mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-25 13:10:13 +00:00
Compare commits
59 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e8ef4f53ae | |||
| a0d10b8471 | |||
| ad3632e9fb | |||
| 05e1569503 | |||
| ba3581e253 | |||
| 829ec4c969 | |||
| b2d2d37c6f | |||
| e0ad51b965 | |||
| bbb55034e3 | |||
| 58c8cf87a4 | |||
| 71d2c0e47f | |||
| 3b2f5ed46d | |||
| 7329f12217 | |||
| c9a9df99f1 | |||
| 757427ffea | |||
| 71832b9deb | |||
| ce4d884abc | |||
| b56c774103 | |||
| 68767f436f | |||
| c7b06be8b7 | |||
| 479247cdae | |||
| b6ba87f072 | |||
| fa640ebe61 | |||
| dbc20a9f66 | |||
| cbda62d131 | |||
| 3951000b40 | |||
| c59842262e | |||
| c3e9621a78 | |||
| 3ed21c37d1 | |||
| 4a1194569e | |||
| d0794fc7c5 | |||
| f34c9d2322 | |||
| 380a8417c8 | |||
| 63826a4869 | |||
| 7df208fac9 | |||
| 3f9576d4a9 | |||
| ee58242d33 | |||
| eca528b9b3 | |||
| 8305f5c850 | |||
| c2c02b929c | |||
| 1821af6f05 | |||
| 62599d972c | |||
| 21b751be64 | |||
| 63ea2e3cee | |||
| 93dcac44a4 | |||
| 72be8ac89f | |||
| cd570c8b3e | |||
| 6dfc564be6 | |||
| afecd18123 | |||
| 5a294ac4f6 | |||
| dbf6e50e2d | |||
| 062962ae41 | |||
| be08660976 | |||
| b032005f5b | |||
| 0ab568f479 | |||
| fced858a7a | |||
| ee947e2ca9 | |||
| 5f08a6ec68 | |||
| a97888ebd5 |
+25
-67
@@ -1,79 +1,37 @@
|
||||
## Workflow Orchestration
|
||||
# Agent Directives: Mechanical Overrides
|
||||
# Agent Directives
|
||||
|
||||
You are operating within a constrained context window and strict system prompts. To produce production-grade code, you MUST adhere to these overrides:
|
||||
- **Plan first.** Enter plan mode for any non-trivial task (3+ steps or an architectural decision) and write the spec before building. If something goes sideways, stop and re-plan instead of pushing on.
|
||||
- **Use subagents liberally** for research, exploration, and parallel analysis — one focus each — to keep the main context clean.
|
||||
- **Verify before claiming done.** Run the tests, check the logs, show the evidence. Diff against `main` where behaviour could regress.
|
||||
- **Fix bugs autonomously.** Given a report, a failing test, or an error, resolve it — no hand-holding.
|
||||
- **Capture corrections.** After any correction from the user, record the pattern and a preventing rule in `tasks/lessons.md`; review it at session start.
|
||||
- Read files over 500 LOC in chunks. Never treat one grep as proof of absence.
|
||||
|
||||
Codex reviews your output once you are done.
|
||||
|
||||
## Code Quality
|
||||
|
||||
### 1. Plan Mode Default
|
||||
- Enter plan mode for ANY non-trivial task (3+ steps or architectural decisions)
|
||||
- If something goes sideways, STOP and re-plan immediately — don't keep pushing
|
||||
- Use plan mode for specification steps, not just building
|
||||
- Write detailed specs upfront to reduce ambiguity
|
||||
|
||||
### 2. Subagent Strategy
|
||||
- Use subagents liberally to keep main context window clean
|
||||
- Offload research, exploration, and parallel analysis to subagents
|
||||
- For complex problems, throw more compute at it via subagents
|
||||
- One tack per subagent for focused execution
|
||||
|
||||
### 3. Self-Improvement Loop
|
||||
- After ANY correction from the user: update `tasks/lessons.md` with the pattern
|
||||
- Write rules for yourself that prevent the same mistake
|
||||
- Ruthlessly iterate on these lessons until mistake rate drops
|
||||
- Review lessons at session start for relevant project
|
||||
|
||||
### 4. Verification Before Done
|
||||
- Never mark a task complete without proving it works
|
||||
- Diff behavior between main and your changes when relevant
|
||||
- Ask yourself: "Would a staff engineer approve this?"
|
||||
- Run tests, check logs, demonstrate correctness
|
||||
|
||||
### 5. Demand Elegance (Balanced)
|
||||
- For non-trivial changes: pause and ask "is there a more elegant way?"
|
||||
- If a fix feels hacky: "Knowing everything I know now, implement the elegant solution"
|
||||
- Skip this for simple, obvious fixes — don't over-engineer
|
||||
- Challenge your own work before presenting it
|
||||
|
||||
### 6. Autonomous Bug Fixing
|
||||
- When given a bug report: just fix it. Don't ask for hand-holding
|
||||
- Point at logs, errors, failing tests — then resolve them
|
||||
- Zero context switching required from the user
|
||||
- Go fix failing CI tests without being told how
|
||||
|
||||
|
||||
|
||||
### Tools Execution
|
||||
- Read any file over 500 LOC in chunks using offset and limit parameters.
|
||||
- Do not assume a single grep caught everything.
|
||||
|
||||
### Code Quality
|
||||
- If architecture is flawed, state is duplicated, or patterns are inconsistent - propose and implement structural fixes. Ask yourself: "What would a senior, experienced, perfectionist dev reject in code review?" Fix all of it.
|
||||
|
||||
|
||||
Codex will review your output once you are done!
|
||||
|
||||
- Simplicity first: minimal, targeted changes. No temporary fixes — find the root cause.
|
||||
- Where architecture is flawed, state is duplicated, or patterns are inconsistent, propose and implement the structural fix. Ask what a perfectionist reviewer would reject, and fix that too.
|
||||
- On non-trivial changes, ask whether there is a more elegant way before presenting. Skip it for simple fixes — don't over-engineer.
|
||||
|
||||
## Task Management
|
||||
|
||||
1. **Plan First**: Write plan to `tasks/todo.md` with checkable items
|
||||
2. **Verify Plan**: Check in before starting implementation
|
||||
3. **Track Progress**: Mark items complete as you go
|
||||
4. **Explain Changes**: High-level summary at each step
|
||||
5. **Document Results**: Add review section to `tasks/todo.md`
|
||||
6. **Capture Lessons**: Update `tasks/lessons.md` after corrections
|
||||
Keep the plan in `tasks/todo.md` as checkable items, confirm it before implementing, tick items off as you go, and add a review section when done.
|
||||
|
||||
## Core Principles
|
||||
## Customer-Facing Claims
|
||||
|
||||
- **Simplicity First**: Make every change as simple as possible. Impact minimal code.
|
||||
- **No Laziness**: Find root causes. No temporary fixes. Senior developer standards.
|
||||
- **Minimal Impact**: Changes should only touch what's necessary. Avoid introducing bugs.
|
||||
`src/Web/autopilot-monitor-web/app/{about,terms,privacy}/page.tsx` and `autopilotmonitor-docs/trust/*` state facts customers and security reviewers rely on. They drift silently.
|
||||
|
||||
## Technical Docs — OKF Knowledge Bundle (`docs/`)
|
||||
- Verify every claim against the code before writing it — never carry one forward because it was already on the page.
|
||||
- Update them in the same change as: runtime/framework versions, roles, notification providers, agent deployment or lifecycle, diagnostics payload, isolation/delegation model, retention caps, sub-processors, or any default governing what is collected.
|
||||
- Describe only what customers can actually use; operator-only infrastructure is not a feature.
|
||||
- Trust pages: durable phrasing over exact figures, and bump their "Last reviewed" date.
|
||||
|
||||
`docs/` is an [Open Knowledge Format (OKF) v0.1](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) knowledge bundle for contributor/AI-facing technical documentation (customer docs live in the separate autopilotmonitor-docs repo).
|
||||
## Technical Docs — OKF Bundle (`docs/`)
|
||||
|
||||
- **Consume first**: start at `docs/index.md` for progressive disclosure before opening individual documents.
|
||||
- **When new durable technical knowledge emerges** (architecture decisions, flows, non-obvious mechanisms), capture it as an OKF concept document in `docs/` — markdown with YAML frontmatter: `type` (mandatory), plus `title`, `description`, `resource`, `tags`, `timestamp` (recommended).
|
||||
- **Maintain the bundle**: add every new document to `docs/index.md`, note changes in `docs/log.md` (grouped by ISO date). Use standard RELATIVE markdown links between docs — never the OKF `/`-prefixed bundle-absolute form (GitHub resolves those from the repo root and navigation breaks). `index.md` and `log.md` are reserved filenames.
|
||||
- Documents are English, structural markdown over prose (`# Schema`, `# Examples`, `# Citations` sections where applicable).
|
||||
Contributor/AI-facing docs (customer docs live in the `autopilotmonitor-docs` repo). Start at `docs/index.md` before opening individual files.
|
||||
|
||||
- New durable knowledge (architecture decisions, flows, non-obvious mechanisms) → an OKF concept doc with YAML frontmatter: `type` (mandatory), plus `title`, `description`, `resource`, `tags`, `timestamp`.
|
||||
- Register it in `docs/index.md`, note the change in `docs/log.md` (ISO date). `index.md` and `log.md` are reserved names.
|
||||
- English, structural markdown (`# Schema`, `# Examples`, `# Citations`). Links between docs are RELATIVE — never `/`-prefixed, that breaks GitHub navigation.
|
||||
|
||||
+1
-1
Submodule .claude/commands updated: 3ed910ab50...af1b8468c7
@@ -0,0 +1,31 @@
|
||||
# CodeQL configuration.
|
||||
#
|
||||
# The default query suites stay active — this file only filters out two C# queries
|
||||
# that fire structurally on this codebase, where the flagged behavior is the feature,
|
||||
# not a defect:
|
||||
#
|
||||
# cs/exposure-of-sensitive-information
|
||||
# Audit and operational logs record WHO performed an action (actor UPN / email)
|
||||
# and notification services deliver mail to a recipient address. Writing that
|
||||
# identity to the log / notification IS the purpose of the field. Every such site
|
||||
# is a legitimate, reviewed use — the query cannot distinguish it from a leak.
|
||||
#
|
||||
# cs/cleartext-storage-of-sensitive-information
|
||||
# The rate limiters and distress guard use the trusted client IP as their bucket
|
||||
# key. The IP is an identifier for throttling, not a secret at rest; storing it in
|
||||
# the in-memory / table rate-limit key is by design.
|
||||
#
|
||||
# Genuine one-off false positives from OTHER queries (e.g. an open-redirect the taint
|
||||
# model can't see is allow-listed, a rate-limited route it doesn't recognize) are
|
||||
# dismissed individually in the Security tab so those queries stay sharp — they are
|
||||
# NOT suppressed here.
|
||||
#
|
||||
# Log-injection (cs/log-forging) is deliberately NOT filtered: user-controlled values
|
||||
# are sanitized at the sink via Helpers/LogSanitizer, so the query stays useful for
|
||||
# catching a future unsanitized sink.
|
||||
|
||||
query-filters:
|
||||
- exclude:
|
||||
id: cs/exposure-of-sensitive-information
|
||||
- exclude:
|
||||
id: cs/cleartext-storage-of-sensitive-information
|
||||
@@ -0,0 +1,47 @@
|
||||
version: 2
|
||||
|
||||
updates:
|
||||
- package-ecosystem: nuget
|
||||
directory: /
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
dotnet:
|
||||
patterns: ['*']
|
||||
update-types: [minor, patch]
|
||||
|
||||
- package-ecosystem: npm
|
||||
directory: /src/Web/autopilot-monitor-web
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
web:
|
||||
patterns: ['*']
|
||||
update-types: [minor, patch]
|
||||
# Major bumps are breaking and reviewed deliberately, not auto-PR'd.
|
||||
# Security updates ignore this filter, so the Security tab stays clean.
|
||||
ignore:
|
||||
- dependency-name: '*'
|
||||
update-types: ['version-update:semver-major']
|
||||
|
||||
- package-ecosystem: npm
|
||||
directory: /src/McpServer/autopilot-monitor-mcp
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
mcp:
|
||||
patterns: ['*']
|
||||
update-types: [minor, patch]
|
||||
# Major bumps are breaking and reviewed deliberately, not auto-PR'd.
|
||||
# Security updates ignore this filter, so the Security tab stays clean.
|
||||
ignore:
|
||||
- dependency-name: '*'
|
||||
update-types: ['version-update:semver-major']
|
||||
|
||||
- package-ecosystem: github-actions
|
||||
directory: /
|
||||
schedule:
|
||||
interval: monthly
|
||||
@@ -0,0 +1,102 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Reserves a monotone build number from a counter blob (ETag compare-and-swap).
|
||||
|
||||
.DESCRIPTION
|
||||
Single source of build numbers for every component that ships from this repo
|
||||
(agent, backend, MCP). Each component owns one counter blob holding a plain
|
||||
integer; this script reads it, writes value+1 back under If-Match, and returns
|
||||
the reserved number.
|
||||
|
||||
Reserve-BEFORE-build: two concurrent builds can never mint the same number.
|
||||
A failed build burns a number, which is harmless -- uniqueness matters,
|
||||
density does not.
|
||||
|
||||
The blob is public-read; only the write needs the container SAS.
|
||||
|
||||
Why a counter and not the published version manifest: manifests are written
|
||||
on publish, not on build (the agent writes version.json only for stable
|
||||
releases), so two dev builds in a row would derive the same number and
|
||||
overwrite each other's versioned artifact.
|
||||
|
||||
.PARAMETER CounterUrl
|
||||
Full URL of the counter blob, without SAS.
|
||||
|
||||
.PARAMETER SasToken
|
||||
Container SAS with create+write permission. A leading '?' is tolerated.
|
||||
|
||||
.PARAMETER Override
|
||||
Explicit build number instead of counter+1. If it is not greater than the
|
||||
current counter the blob is left untouched (re-build of a number that was
|
||||
already reserved). 0 = no override.
|
||||
|
||||
.PARAMETER MaxAttempts
|
||||
CAS retries before giving up.
|
||||
|
||||
.OUTPUTS
|
||||
[int] the reserved build number. Progress goes to the host, not the pipeline.
|
||||
|
||||
.EXAMPLE
|
||||
$n = ./Request-BuildNumber.ps1 -CounterUrl $url -SasToken $sas
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory = $true)][string]$CounterUrl,
|
||||
[Parameter(Mandatory = $true)][string]$SasToken,
|
||||
[int]$Override = 0,
|
||||
[int]$MaxAttempts = 5
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# Tolerate both secret formats ("sp=cw&..." and "?sp=cw&...").
|
||||
$writeUrl = $CounterUrl + '?' + $SasToken.TrimStart('?')
|
||||
|
||||
function Get-HttpStatus {
|
||||
param($ErrorRecord)
|
||||
if ($ErrorRecord.Exception.PSObject.Properties['Response'] -and $ErrorRecord.Exception.Response) {
|
||||
return [int]$ErrorRecord.Exception.Response.StatusCode
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
function Read-Counter {
|
||||
try {
|
||||
$resp = Invoke-WebRequest -Uri $CounterUrl -Method Get -UseBasicParsing
|
||||
} catch {
|
||||
if ((Get-HttpStatus $_) -eq 404) {
|
||||
throw "Counter blob not found at $CounterUrl. Seed it once with the last used build number: PUT the plain integer with 'x-ms-blob-type: BlockBlob' and 'If-None-Match: *'. See docs/versioning.md."
|
||||
}
|
||||
throw
|
||||
}
|
||||
$raw = if ($resp.Content -is [byte[]]) { [System.Text.Encoding]::UTF8.GetString($resp.Content) } else { [string]$resp.Content }
|
||||
# If-Match requires the quoted ETag form; PowerShell strips the quotes on read.
|
||||
$etag = '"' + ([string]($resp.Headers['ETag'] | Select-Object -First 1)).Trim('"') + '"'
|
||||
return [pscustomobject]@{ Value = [int]$raw.Trim(); ETag = $etag }
|
||||
}
|
||||
|
||||
for ($attempt = 1; $attempt -le $MaxAttempts; $attempt++) {
|
||||
$counter = Read-Counter
|
||||
|
||||
if ($Override -gt 0 -and $Override -le $counter.Value) {
|
||||
Write-Host " Build number $Override <= counter $($counter.Value) -- already reserved, counter blob unchanged"
|
||||
return $Override
|
||||
}
|
||||
$target = if ($Override -gt 0) { $Override } else { $counter.Value + 1 }
|
||||
|
||||
try {
|
||||
$headers = @{ 'x-ms-blob-type' = 'BlockBlob'; 'If-Match' = $counter.ETag }
|
||||
Invoke-RestMethod -Uri $writeUrl -Method Put -Headers $headers -Body "$target" -ContentType 'text/plain' | Out-Null
|
||||
Write-Host " Reserved build number $target (CAS write ok, attempt $attempt)"
|
||||
return $target
|
||||
} catch {
|
||||
if ((Get-HttpStatus $_) -eq 412) {
|
||||
Write-Host " CAS conflict on attempt $attempt (another build reserved concurrently) -- retrying"
|
||||
Start-Sleep -Seconds ($attempt * 2)
|
||||
} else {
|
||||
throw
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
throw "Could not reserve a build number after $MaxAttempts CAS attempts"
|
||||
@@ -56,78 +56,27 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v5
|
||||
uses: actions/checkout@v7
|
||||
|
||||
# SDK 10 = single build toolchain (parity with local build.ps1 / global.json).
|
||||
- name: Setup .NET
|
||||
uses: actions/setup-dotnet@v5
|
||||
uses: actions/setup-dotnet@v6
|
||||
with:
|
||||
dotnet-version: '10.0.x'
|
||||
|
||||
# Build numbers come from the shared counter blob (single source of truth for CI
|
||||
# AND local build.ps1). Reserve-before-build with ETag-CAS: concurrent builds can
|
||||
# never mint the same number; a failed build burns a number, which is harmless.
|
||||
# The blob is public-read (like the rest of the container); writes need the SAS.
|
||||
# Build numbers come from the shared counter blob -- one counter and one
|
||||
# implementation for CI and the local build (.github/scripts/Request-BuildNumber.ps1).
|
||||
# Reserve-before-build with ETag-CAS: concurrent builds can never mint the same
|
||||
# number; a failed build burns a number, which is harmless.
|
||||
- name: Reserve build number (counter blob CAS)
|
||||
id: version
|
||||
shell: pwsh
|
||||
run: |
|
||||
$counterBlob = "buildcounter-v$env:LINE_MAJOR.txt"
|
||||
$counterUrl = "https://autopilotmonitoreu.blob.core.windows.net/agent/$counterBlob"
|
||||
# Tolerate both secret formats ("sp=cw&..." and "?sp=cw&...") -- the URL below adds the '?'.
|
||||
$sasToken = "${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}".TrimStart('?')
|
||||
$override = "${{ inputs.build_number }}"
|
||||
$override = 0
|
||||
if ('${{ inputs.build_number }}' -ne '') { $override = [int]'${{ inputs.build_number }}' }
|
||||
|
||||
function Get-Counter {
|
||||
try {
|
||||
$resp = Invoke-WebRequest -Uri $counterUrl -Method Get
|
||||
$raw = if ($resp.Content -is [byte[]]) { [System.Text.Encoding]::UTF8.GetString($resp.Content) } else { [string]$resp.Content }
|
||||
# If-Match requires the quoted ETag form; pwsh strips the quotes on read.
|
||||
$etag = '"' + ([string]($resp.Headers['ETag'] | Select-Object -First 1)).Trim('"') + '"'
|
||||
return @{ Value = [int]$raw.Trim(); ETag = $etag }
|
||||
} catch {
|
||||
$status = if ($_.Exception.Response) { [int]$_.Exception.Response.StatusCode } else { 0 }
|
||||
if ($status -eq 404) {
|
||||
Write-Error "Counter blob '$counterBlob' not found. Seed it once with the last used build number (see docs/agent/build-counter-blob.md) -- do NOT seed from the repo's buildcounter.txt, it is a stale offline fallback."
|
||||
}
|
||||
throw
|
||||
}
|
||||
}
|
||||
|
||||
$buildNumber = $null
|
||||
for ($attempt = 1; $attempt -le 5; $attempt++) {
|
||||
$counter = Get-Counter
|
||||
if ($override -ne '') {
|
||||
$target = [int]$override
|
||||
if ($target -le $counter.Value) {
|
||||
# Re-build of an already-reserved number: counter stays monotonic, no write.
|
||||
Write-Host "Manual build number $target <= counter $($counter.Value) -- counter blob unchanged"
|
||||
$buildNumber = $target
|
||||
break
|
||||
}
|
||||
Write-Host "Manual build number $target > counter $($counter.Value) -- advancing counter"
|
||||
} else {
|
||||
$target = $counter.Value + 1
|
||||
}
|
||||
try {
|
||||
$headers = @{ 'x-ms-blob-type' = 'BlockBlob'; 'If-Match' = $counter.ETag }
|
||||
Invoke-RestMethod -Uri "$counterUrl`?$sasToken" -Method Put -Headers $headers -Body "$target" -ContentType 'text/plain' | Out-Null
|
||||
$buildNumber = $target
|
||||
Write-Host "Reserved build number ${target} (CAS write ok, attempt $attempt)"
|
||||
break
|
||||
} catch {
|
||||
$status = if ($_.Exception.Response) { [int]$_.Exception.Response.StatusCode } else { 0 }
|
||||
if ($status -eq 412) {
|
||||
Write-Host "CAS conflict on attempt $attempt (another build reserved concurrently) -- retrying"
|
||||
Start-Sleep -Seconds ($attempt * 2)
|
||||
} else {
|
||||
throw
|
||||
}
|
||||
}
|
||||
}
|
||||
if ($null -eq $buildNumber) {
|
||||
Write-Error "Could not reserve a build number after 5 CAS attempts"
|
||||
}
|
||||
$counterUrl = "https://autopilotmonitoreu.blob.core.windows.net/agent/buildcounter-v$($env:LINE_MAJOR).txt"
|
||||
$buildNumber = & ./.github/scripts/Request-BuildNumber.ps1 -CounterUrl $counterUrl -SasToken '${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}' -Override $override
|
||||
|
||||
$csproj = Get-Content $env:CSPROJ_PATH -Raw
|
||||
if ($csproj -match '<VersionPrefix>(\d+\.\d+)</VersionPrefix>') {
|
||||
@@ -228,8 +177,13 @@ jobs:
|
||||
Write-Warning "Could not parse bootstrap `$ScriptVersion from $env:BOOTSTRAP_SOURCE"
|
||||
}
|
||||
|
||||
# Enrich the per-line manifest emitted by MSBuild (version-v2.json, read above)
|
||||
# Enrich the per-line manifest emitted by MSBuild (version-v2.json, read above).
|
||||
# commit/buildUtc are the same two fields the backend and MCP manifests carry —
|
||||
# additive, and the bootstrap reads this file by field name, so older installers
|
||||
# ignore them. sha256/bootstrapVersion stay: those are the customer contract.
|
||||
$manifest | Add-Member -NotePropertyName "sha256" -NotePropertyValue $sha256Hash -Force
|
||||
$manifest | Add-Member -NotePropertyName "commit" -NotePropertyValue '${{ github.sha }}'.Substring(0, 7) -Force
|
||||
$manifest | Add-Member -NotePropertyName "buildUtc" -NotePropertyValue ([DateTime]::UtcNow.ToString('o')) -Force
|
||||
if ($bootstrapVersion) {
|
||||
$manifest | Add-Member -NotePropertyName "bootstrapVersion" -NotePropertyValue $bootstrapVersion -Force
|
||||
}
|
||||
@@ -277,7 +231,12 @@ jobs:
|
||||
param([string]$Local, [string]$BlobName, [string]$ContentType)
|
||||
$url = "$containerUrl/$BlobName`?$sasToken"
|
||||
$bytes = [System.IO.File]::ReadAllBytes($Local)
|
||||
$headers = @{ "x-ms-blob-type" = "BlockBlob"; "Content-Type" = $ContentType }
|
||||
# no-cache on EVERY blob: they all rotate in place (even the per-line ZIP is
|
||||
# overwritten each build), and the download alias fronts them with Front Door.
|
||||
# Route caching is disabled there, but a stale manifest paired with a fresh ZIP
|
||||
# (independent cache entries) would fail the bootstrap SHA check -- this header
|
||||
# keeps that class of bug impossible even if caching is ever re-enabled.
|
||||
$headers = @{ "x-ms-blob-type" = "BlockBlob"; "Content-Type" = $ContentType; "x-ms-blob-cache-control" = "no-cache" }
|
||||
if ($ContentType -eq "application/zip") {
|
||||
$md5 = [System.Security.Cryptography.MD5]::Create()
|
||||
$headers["Content-MD5"] = [Convert]::ToBase64String($md5.ComputeHash($bytes))
|
||||
@@ -352,13 +311,13 @@ jobs:
|
||||
subject-path: ${{ steps.artifact.outputs.versioned_zip_release }}
|
||||
|
||||
- name: Upload build artifact
|
||||
uses: actions/upload-artifact@v6
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: AutopilotMonitor-Agent-v${{ steps.version.outputs.version }}
|
||||
path: ${{ steps.artifact.outputs.versioned_zip_release }}
|
||||
|
||||
- name: Upload version metadata
|
||||
uses: actions/upload-artifact@v6
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: version-metadata
|
||||
path: ${{ env.MANIFEST_NAME }}
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
name: CI
|
||||
|
||||
# Tests, typechecks and drift-checks every push to main and every pull request:
|
||||
# the .NET suites, the web and MCP suites, and a guard that the generated rule
|
||||
# files still match rules/guardrails.json.
|
||||
#
|
||||
# The deploy workflows build but never test, so this is the only place the suites
|
||||
# actually run. It does not gate anything — main is deliberately unprotected, so a
|
||||
# red run reports rather than blocks. Read the result and follow up.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: ci-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
dotnet:
|
||||
name: .NET tests (backend + agent)
|
||||
# windows-latest is required: the agent test projects target net48.
|
||||
runs-on: windows-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- uses: actions/setup-dotnet@v6
|
||||
with:
|
||||
dotnet-version: '10.0.x'
|
||||
|
||||
- name: Restore
|
||||
run: dotnet restore AutopilotMonitor.sln
|
||||
|
||||
- name: Build
|
||||
run: dotnet build AutopilotMonitor.sln --configuration Release --no-restore
|
||||
|
||||
- name: Backend tests
|
||||
run: dotnet test src/Backend/AutopilotMonitor.Functions.Tests --configuration Release --no-build --verbosity quiet
|
||||
|
||||
- name: Agent V2 Core tests
|
||||
run: dotnet test src/Agent/AutopilotMonitor.Agent.V2.Core.Tests --configuration Release --no-build --verbosity quiet
|
||||
|
||||
- name: Decision Core tests
|
||||
run: dotnet test src/Agent/AutopilotMonitor.DecisionCore.Tests --configuration Release --no-build --verbosity quiet
|
||||
|
||||
- name: Summary Dialog tests
|
||||
run: dotnet test src/Agent/AutopilotMonitor.SummaryDialog.Tests --configuration Release --no-build --verbosity quiet
|
||||
|
||||
web:
|
||||
name: Web (typecheck + tests)
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: src/Web/autopilot-monitor-web
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: npm
|
||||
cache-dependency-path: src/Web/autopilot-monitor-web/package-lock.json
|
||||
|
||||
- run: npm ci
|
||||
|
||||
- name: Typecheck
|
||||
run: npx tsc --noEmit
|
||||
|
||||
- name: Tests
|
||||
run: npm test
|
||||
|
||||
mcp:
|
||||
name: MCP server tests
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: src/McpServer/autopilot-monitor-mcp
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: npm
|
||||
cache-dependency-path: src/McpServer/autopilot-monitor-mcp/package-lock.json
|
||||
|
||||
- run: npm ci
|
||||
|
||||
- name: Typecheck
|
||||
run: npx tsc --noEmit
|
||||
|
||||
- name: Tests
|
||||
run: npm test
|
||||
|
||||
guardrails-in-sync:
|
||||
name: Generated guardrails match rules/guardrails.json
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Regenerate
|
||||
run: node rules/scripts/combine.js
|
||||
|
||||
- name: Fail if the generated files drifted
|
||||
run: |
|
||||
if ! git diff --quiet; then
|
||||
echo "::error::Generated files are out of date. Run 'node rules/scripts/combine.js' and commit the result."
|
||||
git diff --stat
|
||||
exit 1
|
||||
fi
|
||||
@@ -0,0 +1,73 @@
|
||||
name: CodeQL
|
||||
|
||||
# Documentation-only changes cannot affect what CodeQL analyzes (C# and
|
||||
# TypeScript), so they are filtered out rather than spending ~10 minutes to
|
||||
# re-report an unchanged result. A commit that touches docs AND code still runs:
|
||||
# GitHub skips only when EVERY changed path matches the filter.
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths-ignore:
|
||||
- 'docs/**'
|
||||
- '**/*.md'
|
||||
- 'LICENSE'
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths-ignore:
|
||||
- 'docs/**'
|
||||
- '**/*.md'
|
||||
- 'LICENSE'
|
||||
schedule:
|
||||
# Weekly, so advisories published after a merge still surface. Deliberately
|
||||
# unfiltered — it re-analyzes the whole tree regardless of what changed, which
|
||||
# is also the backstop if the filters above ever skip too much.
|
||||
- cron: '17 3 * * 1'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
name: Analyze ${{ matrix.language }}
|
||||
runs-on: ${{ matrix.runner }}
|
||||
permissions:
|
||||
security-events: write
|
||||
packages: read
|
||||
actions: read
|
||||
contents: read
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
# csharp needs the net48 agent projects to build, so Windows.
|
||||
- language: csharp
|
||||
runner: windows-latest
|
||||
build-mode: manual
|
||||
- language: javascript-typescript
|
||||
runner: ubuntu-latest
|
||||
build-mode: none
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- uses: actions/setup-dotnet@v6
|
||||
if: matrix.language == 'csharp'
|
||||
with:
|
||||
dotnet-version: '10.0.x'
|
||||
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@v4
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
build-mode: ${{ matrix.build-mode }}
|
||||
config-file: .github/codeql/codeql-config.yml
|
||||
|
||||
- name: Build
|
||||
if: matrix.build-mode == 'manual'
|
||||
run: dotnet build AutopilotMonitor.sln --configuration Release
|
||||
|
||||
- name: Analyze
|
||||
uses: github/codeql-action/analyze@v4
|
||||
with:
|
||||
category: /language:${{ matrix.language }}
|
||||
@@ -22,9 +22,9 @@ jobs:
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- uses: actions/setup-node@v5
|
||||
- uses: actions/setup-node@v7
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
|
||||
@@ -7,13 +7,19 @@ permissions:
|
||||
id-token: write # OIDC federated login (secret-less)
|
||||
contents: read
|
||||
|
||||
env:
|
||||
FUNCTION_APP: autopilotmonitor-api-eu
|
||||
HEALTH_URL: https://autopilotmonitor-api-eu.azurewebsites.net/api/health
|
||||
VERSIONS_CONTAINER: https://autopilotmonitoreu.blob.core.windows.net/versions
|
||||
CSPROJ_PATH: src/Backend/AutopilotMonitor.Functions/AutopilotMonitor.Functions.csproj
|
||||
|
||||
jobs:
|
||||
build-and-deploy:
|
||||
runs-on: ubuntu-latest
|
||||
name: Build and Deploy Backend
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Login to Azure (OIDC)
|
||||
uses: azure/login@v3
|
||||
@@ -23,21 +29,95 @@ jobs:
|
||||
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
|
||||
- name: Setup .NET 10
|
||||
uses: actions/setup-dotnet@v5
|
||||
uses: actions/setup-dotnet@v6
|
||||
with:
|
||||
dotnet-version: '10.0.x'
|
||||
|
||||
# Same counter mechanism as the agent line: reserve-before-build with ETag-CAS
|
||||
# against a per-component counter blob. BuildTimestampUtc is pinned here rather
|
||||
# than left to MSBuild's default so the published manifest and the value baked
|
||||
# into the assembly (reported by /api/health) are the same instant.
|
||||
- name: Reserve build number (counter blob CAS)
|
||||
id: version
|
||||
shell: pwsh
|
||||
run: |
|
||||
$buildNumber = & ./.github/scripts/Request-BuildNumber.ps1 -CounterUrl "${{ env.VERSIONS_CONTAINER }}/backend.txt" -SasToken '${{ secrets.AZURE_BLOB_VERSIONS_SAS_TOKEN }}'
|
||||
|
||||
$csproj = Get-Content ${{ env.CSPROJ_PATH }} -Raw
|
||||
if ($csproj -notmatch '<VersionPrefix>(\d+\.\d+)</VersionPrefix>') {
|
||||
throw "VersionPrefix not found in ${{ env.CSPROJ_PATH }}"
|
||||
}
|
||||
$version = "$($Matches[1]).$buildNumber"
|
||||
$buildUtc = [DateTime]::UtcNow.ToString('o')
|
||||
Write-Host "Backend version: $version (built $buildUtc)"
|
||||
|
||||
"build_number=$buildNumber" >> $env:GITHUB_OUTPUT
|
||||
"version=$version" >> $env:GITHUB_OUTPUT
|
||||
"build_utc=$buildUtc" >> $env:GITHUB_OUTPUT
|
||||
|
||||
- name: Restore dependencies
|
||||
run: dotnet restore src/Backend/AutopilotMonitor.Functions/AutopilotMonitor.Functions.csproj
|
||||
run: dotnet restore ${{ env.CSPROJ_PATH }}
|
||||
|
||||
- name: Build
|
||||
run: dotnet build src/Backend/AutopilotMonitor.Functions/AutopilotMonitor.Functions.csproj --configuration Release --no-restore -p:SourceRevisionId=${{ github.sha }}
|
||||
run: dotnet build ${{ env.CSPROJ_PATH }} --configuration Release --no-restore -p:SourceRevisionId=${{ github.sha }} -p:BuildNumber=${{ steps.version.outputs.build_number }} -p:BuildTimestampUtc=${{ steps.version.outputs.build_utc }}
|
||||
|
||||
- name: Publish
|
||||
run: dotnet publish src/Backend/AutopilotMonitor.Functions/AutopilotMonitor.Functions.csproj --configuration Release --output ./publish --no-build -p:SourceRevisionId=${{ github.sha }}
|
||||
run: dotnet publish ${{ env.CSPROJ_PATH }} --configuration Release --output ./publish --no-build -p:SourceRevisionId=${{ github.sha }} -p:BuildNumber=${{ steps.version.outputs.build_number }} -p:BuildTimestampUtc=${{ steps.version.outputs.build_utc }}
|
||||
|
||||
- name: Deploy to Azure Functions
|
||||
uses: Azure/functions-action@v1
|
||||
with:
|
||||
app-name: autopilotmonitor-api-eu
|
||||
app-name: ${{ env.FUNCTION_APP }}
|
||||
package: ./publish
|
||||
|
||||
# A green deploy step only means the package was accepted. This gate makes the
|
||||
# workflow assert what is actually being served, so the manifest published below
|
||||
# can be trusted as "this is live" rather than "this was uploaded".
|
||||
# /api/health is IP rate-limited at 30/min; one poll per 10s stays well under.
|
||||
- name: Verify deployed build is live
|
||||
shell: pwsh
|
||||
run: |
|
||||
$expectedCommit = '${{ github.sha }}'.Substring(0, 7)
|
||||
$expectedVersion = '${{ steps.version.outputs.version }}'
|
||||
$deadline = [DateTime]::UtcNow.AddMinutes(3)
|
||||
$attempt = 0
|
||||
|
||||
while ([DateTime]::UtcNow -lt $deadline) {
|
||||
$attempt++
|
||||
try {
|
||||
$health = Invoke-RestMethod -Uri '${{ env.HEALTH_URL }}' -Method Get -TimeoutSec 30
|
||||
if ($health.commitHash -eq $expectedCommit -and $health.version -eq $expectedVersion) {
|
||||
Write-Host "Live after $attempt attempt(s): $($health.version) @ $($health.commitHash), built $($health.buildUtc)"
|
||||
exit 0
|
||||
}
|
||||
Write-Host "Attempt ${attempt}: serving $($health.version) @ $($health.commitHash) -- waiting for $expectedVersion @ $expectedCommit"
|
||||
} catch {
|
||||
Write-Host "Attempt ${attempt}: /api/health not answering yet -- $($_.Exception.Message)"
|
||||
}
|
||||
Start-Sleep -Seconds 10
|
||||
}
|
||||
|
||||
throw "Backend did not report $expectedVersion @ $expectedCommit within 3 minutes -- deployment not verified, version manifest NOT published"
|
||||
|
||||
# The manifest is the cross-component record of what is deployed where. It is
|
||||
# written last and only on a verified deploy, so a difference between the
|
||||
# manifest and /api/health means the running app drifted after the fact
|
||||
# (rollback, stale instance) rather than a half-finished pipeline.
|
||||
- name: Publish version manifest
|
||||
shell: pwsh
|
||||
run: |
|
||||
$manifest = [ordered]@{
|
||||
component = 'backend'
|
||||
version = '${{ steps.version.outputs.version }}'
|
||||
commit = '${{ github.sha }}'.Substring(0, 7)
|
||||
buildUtc = '${{ steps.version.outputs.build_utc }}'
|
||||
deployedUtc = [DateTime]::UtcNow.ToString('o')
|
||||
runId = '${{ github.run_id }}'
|
||||
} | ConvertTo-Json
|
||||
|
||||
$sas = '${{ secrets.AZURE_BLOB_VERSIONS_SAS_TOKEN }}'.TrimStart('?')
|
||||
$url = "${{ env.VERSIONS_CONTAINER }}/backend.json?$sas"
|
||||
# no-cache: rotating pointer, must never be served stale by any fronting cache.
|
||||
Invoke-RestMethod -Uri $url -Method Put -Headers @{ 'x-ms-blob-type' = 'BlockBlob'; 'x-ms-blob-cache-control' = 'no-cache' } -Body $manifest -ContentType 'application/json' | Out-Null
|
||||
Write-Host "Published versions/backend.json"
|
||||
Write-Host $manifest
|
||||
|
||||
@@ -12,6 +12,9 @@ env:
|
||||
CONTAINER_APP_NAME: autopilotmonitor-mcp
|
||||
RESOURCE_GROUP: ${{ secrets.AZURE_MCP_RESOURCE_GROUP }}
|
||||
IMAGE_TAG: ${{ github.sha }}
|
||||
HEALTH_URL: https://mcp.autopilotmonitor.com/health
|
||||
VERSIONS_CONTAINER: https://autopilotmonitoreu.blob.core.windows.net/versions
|
||||
PACKAGE_JSON: src/McpServer/autopilot-monitor-mcp/package.json
|
||||
|
||||
jobs:
|
||||
build-and-deploy:
|
||||
@@ -19,7 +22,23 @@ jobs:
|
||||
name: Build and Deploy MCP Server
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
# The published customer documentation lives in its own PUBLIC repo and is
|
||||
# baked into the image as the search_docs corpus (see the Dockerfile). No
|
||||
# token is needed. Checking it out here rather than vendoring it keeps the
|
||||
# docs single-sourced — but it also means EDITING THE DOCS DOES NOT UPDATE A
|
||||
# RUNNING SERVER; this workflow must be re-run. /health reports the SHA
|
||||
# captured below so that staleness is checkable.
|
||||
- name: Check out documentation bundle
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
repository: autopilotmonitor/autopilotmonitor-docs
|
||||
path: mcp-docs
|
||||
|
||||
- name: Resolve docs commit
|
||||
id: docs
|
||||
run: echo "sha=$(git -C mcp-docs rev-parse HEAD)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Login to Azure (OIDC)
|
||||
uses: azure/login@v3
|
||||
@@ -28,6 +47,30 @@ jobs:
|
||||
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
|
||||
# Same counter mechanism as agent and backend: reserve-before-build with
|
||||
# ETag-CAS against the component's own counter blob. major.minor stays curated
|
||||
# in package.json; only the build part is minted here.
|
||||
- name: Reserve build number (counter blob CAS)
|
||||
id: version
|
||||
shell: pwsh
|
||||
run: |
|
||||
$buildNumber = & ./.github/scripts/Request-BuildNumber.ps1 -CounterUrl "${{ env.VERSIONS_CONTAINER }}/mcp.txt" -SasToken '${{ secrets.AZURE_BLOB_VERSIONS_SAS_TOKEN }}'
|
||||
|
||||
$pkgVersion = (Get-Content ${{ env.PACKAGE_JSON }} -Raw | ConvertFrom-Json).version
|
||||
$prefix = ($pkgVersion -split '\.')[0..1] -join '.'
|
||||
$version = "$prefix.$buildNumber"
|
||||
$buildUtc = [DateTime]::UtcNow.ToString('o')
|
||||
Write-Host "MCP version: $version (built $buildUtc)"
|
||||
|
||||
# 7 chars, matching what the backend's BackendBuildInfo reports — one commit
|
||||
# shape across all components so the values can be compared as-is.
|
||||
$shortSha = '${{ github.sha }}'.Substring(0, 7)
|
||||
|
||||
"build_number=$buildNumber" >> $env:GITHUB_OUTPUT
|
||||
"version=$version" >> $env:GITHUB_OUTPUT
|
||||
"build_utc=$buildUtc" >> $env:GITHUB_OUTPUT
|
||||
"short_sha=$shortSha" >> $env:GITHUB_OUTPUT
|
||||
|
||||
- name: Login to ACR
|
||||
run: az acr login --name ${{ env.ACR_NAME }}
|
||||
|
||||
@@ -35,6 +78,10 @@ jobs:
|
||||
run: |
|
||||
docker build \
|
||||
-f src/McpServer/autopilot-monitor-mcp/Dockerfile \
|
||||
--build-arg DOCS_COMMIT=${{ steps.docs.outputs.sha }} \
|
||||
--build-arg BUILD_NUMBER=${{ steps.version.outputs.build_number }} \
|
||||
--build-arg BUILD_COMMIT=${{ steps.version.outputs.short_sha }} \
|
||||
--build-arg BUILD_UTC=${{ steps.version.outputs.build_utc }} \
|
||||
-t ${{ env.ACR_NAME }}.azurecr.io/${{ env.CONTAINER_APP_NAME }}:${{ env.IMAGE_TAG }} \
|
||||
-t ${{ env.ACR_NAME }}.azurecr.io/${{ env.CONTAINER_APP_NAME }}:latest \
|
||||
.
|
||||
@@ -46,3 +93,55 @@ jobs:
|
||||
--name ${{ env.CONTAINER_APP_NAME }} \
|
||||
--resource-group ${{ env.RESOURCE_GROUP }} \
|
||||
--image ${{ env.ACR_NAME }}.azurecr.io/${{ env.CONTAINER_APP_NAME }}:${{ env.IMAGE_TAG }}
|
||||
|
||||
# `az containerapp update` returns once the revision is provisioned, not once
|
||||
# it serves traffic — and with minReplicas=0 the first request also pays a cold
|
||||
# start. Polling /health until it reports THIS build is what makes the manifest
|
||||
# below mean "live"; a full 5 minutes because the cold start alone is ~25s.
|
||||
- name: Verify deployed build is live
|
||||
shell: pwsh
|
||||
run: |
|
||||
$expectedCommit = '${{ steps.version.outputs.short_sha }}'
|
||||
$expectedVersion = '${{ steps.version.outputs.version }}'
|
||||
$deadline = [DateTime]::UtcNow.AddMinutes(5)
|
||||
$attempt = 0
|
||||
|
||||
while ([DateTime]::UtcNow -lt $deadline) {
|
||||
$attempt++
|
||||
try {
|
||||
$health = Invoke-RestMethod -Uri '${{ env.HEALTH_URL }}' -Method Get -TimeoutSec 60
|
||||
if ($health.commit -eq $expectedCommit -and $health.version -eq $expectedVersion) {
|
||||
Write-Host "Live after $attempt attempt(s): $($health.version) @ $($health.commit), docs $($health.docs.commit) ($($health.docs.chunks) chunks)"
|
||||
exit 0
|
||||
}
|
||||
Write-Host "Attempt ${attempt}: serving $($health.version) @ $($health.commit) -- waiting for $expectedVersion @ $expectedCommit"
|
||||
} catch {
|
||||
Write-Host "Attempt ${attempt}: /health not answering yet -- $($_.Exception.Message)"
|
||||
}
|
||||
Start-Sleep -Seconds 15
|
||||
}
|
||||
|
||||
throw "MCP did not report $expectedVersion @ $expectedCommit within 5 minutes -- deployment not verified, version manifest NOT published"
|
||||
|
||||
# Cross-component record of what is deployed where; written last and only on a
|
||||
# verified deploy. docsCommit rides along because for this component "which
|
||||
# build" and "which documentation" are two separate questions.
|
||||
- name: Publish version manifest
|
||||
shell: pwsh
|
||||
run: |
|
||||
$manifest = [ordered]@{
|
||||
component = 'mcp'
|
||||
version = '${{ steps.version.outputs.version }}'
|
||||
commit = '${{ steps.version.outputs.short_sha }}'
|
||||
buildUtc = '${{ steps.version.outputs.build_utc }}'
|
||||
deployedUtc = [DateTime]::UtcNow.ToString('o')
|
||||
runId = '${{ github.run_id }}'
|
||||
docsCommit = '${{ steps.docs.outputs.sha }}'.Substring(0, 7)
|
||||
} | ConvertTo-Json
|
||||
|
||||
$sas = '${{ secrets.AZURE_BLOB_VERSIONS_SAS_TOKEN }}'.TrimStart('?')
|
||||
$url = "${{ env.VERSIONS_CONTAINER }}/mcp.json?$sas"
|
||||
# no-cache: rotating pointer, must never be served stale by any fronting cache.
|
||||
Invoke-RestMethod -Uri $url -Method Put -Headers @{ 'x-ms-blob-type' = 'BlockBlob'; 'x-ms-blob-cache-control' = 'no-cache' } -Body $manifest -ContentType 'application/json' | Out-Null
|
||||
Write-Host "Published versions/mcp.json"
|
||||
Write-Host $manifest
|
||||
|
||||
@@ -22,7 +22,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
name: Build and Deploy Job
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
lfs: false
|
||||
fetch-depth: 0 # Full history needed for git-based sitemap lastmod dates
|
||||
@@ -39,6 +39,9 @@ jobs:
|
||||
with:
|
||||
azure_static_web_apps_api_token: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
|
||||
repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
|
||||
# Dependabot PRs run without repo secrets, so the deploy token is absent.
|
||||
# Skip (green) instead of failing the check; real deploys on main still have the token.
|
||||
skip_deploy_on_missing_secrets: true
|
||||
action: "upload"
|
||||
###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
|
||||
# For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
|
||||
|
||||
+5
-2
@@ -117,8 +117,11 @@ scripts/Deployment/
|
||||
# Migration tooling (local only, vorerst) — cross-tenant storage/cert migration scripts
|
||||
scripts/Migration/
|
||||
|
||||
# Infra (local only)
|
||||
infra/
|
||||
# Infra: tracked. The Bicep templates carry no credentials — every secret is a
|
||||
# @secure() parameter with an empty default, injected at deploy time, and image
|
||||
# pull runs on managed identity. Keep it that way: never commit a resolved secret
|
||||
# value or a *.parameters.json holding one.
|
||||
infra/*.parameters.json
|
||||
|
||||
# M2 — local raw session fixtures before anonymization (plan §4 M2)
|
||||
tests/fixtures/enrollment-sessions-local/
|
||||
|
||||
@@ -34,7 +34,7 @@ For detailed information about the system architecture, components, and data flo
|
||||
|
||||
## Documentation
|
||||
|
||||
Full documentation is available at **[autopilotmonitor.com/docs](https://www.autopilotmonitor.com/docs)**
|
||||
Full admin documentation is available at **[docs.autopilotmonitor.com](https://docs.autopilotmonitor.com)**
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -22,26 +22,21 @@ attestation on top.
|
||||
|
||||
# Schema
|
||||
|
||||
- **Counter blob**: `https://autopilotmonitor.blob.core.windows.net/agent/buildcounter-v{LINE}.txt`
|
||||
(per release line, analogous to `version-v{LINE}.json`). Content: the **last used**
|
||||
build number as plain text. Public-read like the rest of the container (the number
|
||||
is visible in `version-v{LINE}.json` anyway); writes require the container SAS.
|
||||
- **Protocol — reserve before build, ETag-CAS**:
|
||||
1. Anonymous `GET` the blob, capture value `N` and the `ETag`.
|
||||
2. `PUT N+1` with `If-Match: <etag>` (quoted form — pwsh strips the quotes on read,
|
||||
they must be re-added or the request is rejected client-side).
|
||||
3. `412 Precondition Failed` → another build reserved concurrently → re-read, retry
|
||||
(5 attempts, linear backoff).
|
||||
4. Build with `BUILD_NUMBER=N+1` in the environment. The csproj honors
|
||||
`BUILD_NUMBER` and then skips its `PersistBuildCounter` target, so the repo file
|
||||
is not touched.
|
||||
- **Failed builds burn a number.** That is intentional: uniqueness matters, density
|
||||
does not. Reserve-before-build is what makes concurrent minting impossible.
|
||||
The reserve-before-build CAS protocol itself is shared with the backend and the MCP
|
||||
server and is documented once in [Version Contract](../versioning.md); the single
|
||||
implementation is `.github/scripts/Request-BuildNumber.ps1`. Agent-specific:
|
||||
|
||||
- **Counter blob**: `https://autopilotmonitoreu.blob.core.windows.net/agent/buildcounter-v{LINE}.txt`
|
||||
— one per release line, analogous to `version-v{LINE}.json`, and kept in the `agent`
|
||||
container next to the artifacts it numbers rather than in `versions/`. Content: the
|
||||
**last used** build number as plain text. Public-read like the rest of the container
|
||||
(the number is visible in `version-v{LINE}.json` anyway); writes require the SAS.
|
||||
- The reserved number reaches the build as `BUILD_NUMBER` in the environment. The
|
||||
csproj honors it and then skips its `PersistBuildCounter` target, so the repo file
|
||||
is not touched.
|
||||
- **CI manual override** (`build_number` input): numbers `<=` counter are re-builds
|
||||
and leave the blob unchanged; numbers `>` counter advance it (monotonic guard).
|
||||
- **Seeding** (one-time per line): `PUT <last used number>` with `If-None-Match: *`
|
||||
so an existing counter can never be clobbered. V2 was seeded with `1356` on
|
||||
2026-07-17.
|
||||
- V2 was seeded with `1356` on 2026-07-17.
|
||||
|
||||
# Consequences
|
||||
|
||||
@@ -62,7 +57,9 @@ attestation on top.
|
||||
|
||||
# Citations
|
||||
|
||||
- [Version Contract](../versioning.md) — the shared CAS protocol and manifest format
|
||||
- `.github/workflows/build-agent.yml` — "Reserve build number (counter blob CAS)" step
|
||||
- `scripts/Deployment/build.ps1` (gitignored, contains SAS) — `Request-BuildNumber`
|
||||
- `.github/scripts/Request-BuildNumber.ps1` — the shared implementation
|
||||
- `scripts/Deployment/build.ps1` (gitignored, contains SAS) — local build path
|
||||
- `src/Agent/AutopilotMonitor.Agent.V2/AutopilotMonitor.Agent.V2.csproj` —
|
||||
`BUILD_NUMBER` property + `PersistBuildCounter` target
|
||||
|
||||
@@ -137,8 +137,17 @@ false` and an arm holds), `EspExiting`, `ImeUserSessionCompleted`,
|
||||
advisory backstop. All routes converge on `CompleteThroughFinalizingOrDefer`:
|
||||
|
||||
1. **Completion gates** (currently only the RealmJoin AND-gate): if RealmJoin was
|
||||
detected and is neither resolved nor timed-out (hard 60-min timeout), completion is
|
||||
*deferred* — the session stays in stage and emits `completion_waiting`.
|
||||
detected and has no terminal outcome yet, completion is *deferred* — the session
|
||||
stays in stage and emits `completion_waiting`. Three terminals open the gate:
|
||||
`Resolved` (DeploymentPhase 110), `FirstDeploymentIncomplete` (phase left the
|
||||
first-deployment window 100/101 for 200/210 without 110 — the RJ ESP was aborted,
|
||||
typically by an interactive logon reclassifying the run as secondary-user; emits
|
||||
the `realmjoin_first_deployment_incomplete` Warning, session 224b2087), and
|
||||
`Timeout` (hard 60-min deadline). The abort rule reads the *persisted*
|
||||
`RealmJoinFacts.LastDeploymentPhase` (updated by the typed `RealmJoinPhaseChanged`
|
||||
signal), so it survives agent restarts — the `RealmJoinDetected` replay path
|
||||
evaluates it too. A first observation of 210 does **not** release (RJ may have
|
||||
completed before agent boot and deploy afterwards).
|
||||
2. `Finalizing` stage + ~5 s `FinalizingGrace` deadline.
|
||||
3. Grace fires → `Completed`, `EnrollmentComplete` outcome, `enrollment_complete`
|
||||
timeline event with full audit payload.
|
||||
|
||||
+1
-1
@@ -7,6 +7,6 @@ How the on-device agent works:
|
||||
* [Logs, Persistence & Crash Recovery](logs-and-persistence.md) - every persisted file and why it exists; signal log vs spool; snapshot/journal; the replay-based recovery flow and quarantine.
|
||||
* [Hello Wizard Un-Skip](hello-wizard-unskip.md) - why a single "WHfB disabled" policy read never decides completion: the HelloWizardStarted signal (prevention + cure) and the HelloTracker confirmation second read.
|
||||
* [Autopilot ZTD Diagnostics](autopilot-ztd-diagnostics.md) - Windows' own diagnostic surfaces for the profile-download flow: ModernDeployment event IDs (807/815/164/...), the Diagnostics\Autopilot registry key, deployment-service endpoints, and the known-issue error-code map.
|
||||
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: 2800 watcher with cross-restart watermark, Shell-Core RebootCoalescing corroboration, ANALYZE-ESP-005 advisory.
|
||||
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated 2800 observations (neutral, flush-time watermark) + ANALYZE-ESP-005 gated on an actually-observed reboot.
|
||||
|
||||
Reading order for newcomers: overview → decision engine → persistence.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: Concept
|
||||
title: MDM Reboot Coalescing — attributing the mid-ESP reboot and second sign-in to policy URIs
|
||||
description: How the agent detects device-assigned MDM policies that force a coalesced reboot during ESP DeviceSetup (DM-Enterprise EventID 2800 + Shell-Core 62407 RebootCoalescing), and the ANALYZE-ESP-005 advisory that makes the invisible "second sign-in" fixable.
|
||||
description: How the agent observes reboot-required MDM policy flags (DM-Enterprise EventID 2800, aggregated) and how ANALYZE-ESP-005 attributes an actually-observed enrollment reboot to them — including why the raw events stay neutral and why the Shell-Core RebootCoalescing match was removed.
|
||||
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals
|
||||
tags:
|
||||
- agent
|
||||
@@ -17,91 +17,82 @@ timestamp: 2026-07-20T00:00:00+02:00
|
||||
Device-assigned MDM policies whose CSP URIs match the OS reboot-required catalog
|
||||
(`HKLM\SOFTWARE\Microsoft\Provisioning\SyncML\RebootRequiredURIs`) force a **coalesced
|
||||
reboot at the end of ESP DeviceSetup** — the user sees an unexpected restart and a
|
||||
**second sign-in screen** before AccountSetup. This is expected OS behavior, invisible in
|
||||
the Intune console; most admins simply live with it, not knowing that reassigning the
|
||||
profiles to user groups removes the reboot entirely. Source research: Patch My PC,
|
||||
**second sign-in screen** before AccountSetup. Invisible in the Intune console; most
|
||||
admins live with it, not knowing that reassigning the profiles to user groups removes the
|
||||
reboot entirely. Source research: Patch My PC,
|
||||
["Avoid the second sign-in screen after Autopilot"](https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/).
|
||||
|
||||
The session already counts reboots (`system_reboot_detected` → `session.rebootCount`);
|
||||
this feature adds the **cause attribution**.
|
||||
|
||||
# Signal chain
|
||||
# Verified signal facts (session b2e890c1, 2026-07-20)
|
||||
|
||||
1. **EventID 2800**, `Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin`
|
||||
— logged by the SyncML engine once per policy URI that matched the reboot-required
|
||||
catalog, during device-policy application in DeviceSetup.
|
||||
2. **EventID 62407**, `Microsoft-Windows-Shell-Core/Operational`, description containing
|
||||
`RebootCoalescing` — ESP/CloudExperienceHost initiating the coalesced reboot at the end
|
||||
of DeviceSetup.
|
||||
3. The reboot kills the agent; the post-reboot restart re-observes both channels via
|
||||
backfill (the .evtx persists across the reboot).
|
||||
* **EventID 2800**, `Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin`,
|
||||
one record per URI, verified description shape:
|
||||
`The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).`
|
||||
Records arrive in sub-second bursts during policy sync.
|
||||
* **Timing semantics (the critical nuance)**: 2800 fires at EVERY policy sync. Only URIs
|
||||
applied **during ESP DeviceSetup** are coalesced into the forced mid-ESP restart. In the
|
||||
verified session the flags fired during AccountSetup (after DeviceSetup completed) —
|
||||
**no reboot occurred**, and `omadmRebootRequiredFlag` read `false`. A detector that
|
||||
claims "reboot incoming" on every 2800 is therefore wrong.
|
||||
* **Shell-Core 62407 "RebootCoalescing" — REMOVED as a signal.** The token appears in the
|
||||
ROUTINE bootstrap marker on every enrollment:
|
||||
`Name: 'CommercialOOBE_BootstrapStatusCategory_SubcategoryProcessing_Started', Value: '{"message":"BootstrapStatus: Starting subcategory DeviceSetup.RebootCoalescing...","errorCode":0}'`
|
||||
— the ESP merely *processes* the RebootCoalescing subcategory (which checks whether a
|
||||
reboot is needed). A substring match on it is a false positive by construction. If a
|
||||
real coalesced-reboot record is ever captured on the E2E VM, a precise pattern can be
|
||||
added then.
|
||||
|
||||
# Detectors
|
||||
# Detector — `MdmRebootPolicyTracker` (`MdmRebootPolicyWatcherHost`)
|
||||
|
||||
## `MdmRebootPolicyTracker` (new collector, `MdmRebootPolicyWatcherHost`)
|
||||
Live `EventLogWatcher` on the DM-Enterprise Admin channel (EventID 2800) + startup
|
||||
backfill (default 60 min lookback). Records are **buffered and debounced (10 s)**, then
|
||||
flushed as **ONE aggregated `mdm_policy_reboot_required` event** per burst:
|
||||
|
||||
Structural clone of `WindowsUpdateTracker`: live `EventLogWatcher` on the DM-Enterprise
|
||||
Admin channel (EventID 2800) + startup backfill (default 60 min lookback) + **cross-restart
|
||||
RecordId watermark** (`mdm-reboot-watermark.json`). The watermark is mandatory here: the
|
||||
coalesced reboot restarts the agent, and without it every reboot-forcing URI would be
|
||||
re-emitted after the very reboot it caused.
|
||||
* `Severity=Info`, neutral message ("Windows flagged N MDM policy URIs as
|
||||
reboot-required: … (+N more)") — the raw event never claims a reboot.
|
||||
* `Data`: `rebootUris` (distinct, sorted, capped 20), `uriCount` (uncapped distinct),
|
||||
`firstRebootUri` (record-time order, drives rule interpolation), `recordCount`,
|
||||
`unparsedCount`/`sampleDescription` (extraction diagnostics), `backfilled` (all-backfill),
|
||||
fail-soft `omadmRebootRequiredFlag` probe of
|
||||
`HKLM\SOFTWARE\Microsoft\Provisioning\OMADM\SyncML\RebootRequired` (null = unknown).
|
||||
* `Timestamp` = earliest record time (backfilled bursts land at their historical instant);
|
||||
`ImmediateUpload=true` (one event per burst is cheap, and a DeviceSetup burst precedes a
|
||||
reboot that kills the agent).
|
||||
* **Watermark is persisted at FLUSH, not at claim** (`mdm-reboot-watermark.json`): if the
|
||||
coalesced reboot kills the process before the debounced emit, nothing is marked — the
|
||||
post-restart backfill re-reads and emits the burst. A flushed burst is never repeated.
|
||||
|
||||
Per 2800 record it emits one `mdm_policy_reboot_required` event (Warning,
|
||||
`ImmediateUpload=true` — the announced reboot will kill the agent, flush now) with
|
||||
`Data.rebootUri` (the CSP URI), `recordId`, `backfilled`, `description` (truncated 1000),
|
||||
and a fail-soft `omadmRebootRequiredFlag` corroboration probe of
|
||||
`HKLM\SOFTWARE\Microsoft\Provisioning\OMADM\SyncML\RebootRequired` (null = unknown, never
|
||||
blocks the emit). Per-URI emission (not aggregated) is deliberate: an aggregate would need
|
||||
a flush point that races the reboot kill.
|
||||
URI extraction: EventData values starting with `./` preferred, permissive regex over the
|
||||
description as fallback (trailing punctuation trimmed — the verified text wraps the URI in
|
||||
parens and ends with a period). Unparsed records still count into the aggregate.
|
||||
|
||||
**Tolerant matching**: the exact 2800 message text is unverified — emission is gated on
|
||||
the EventID only. URI extraction prefers structured EventData values starting with `./`
|
||||
and falls back to a permissive regex over the description (trailing punctuation trimmed).
|
||||
Events with an unparseable URI are still emitted (timeline-visible, rule stays silent, and
|
||||
the captured description tells us how to fix the extraction). Exact texts are validated
|
||||
post-deploy on the E2E VM and recorded here.
|
||||
|
||||
Config (internal steering only, no admin UI — this is an agent-native collector, NOT an
|
||||
Config (internal steering only, no admin UI — agent-native collector, NOT an
|
||||
`IAgentAnalyzer`): `Collectors.MdmRebootPolicyWatcherEnabled` (default true),
|
||||
`Collectors.MdmRebootPolicyBackfillLookbackMinutes` (default 60). ConfigVersion 35.
|
||||
|
||||
## `ShellCoreTracker` — `RebootCoalescing` branch
|
||||
|
||||
The already-watched 62407 stream gets a fourth branch (checked AFTER
|
||||
WhiteGlove/failure/exiting so their semantics stay untouched): description containing
|
||||
`RebootCoalescing` → one `esp_reboot_coalescing` event (Warning, ImmediateUpload), pure
|
||||
informational — no `FinalizingSetup` transition, no C# event raise, no DecisionCore
|
||||
involvement.
|
||||
|
||||
**Deliberate deviation**: unlike the other 62407 branches, the backfill path DOES emit
|
||||
this event. The live emit races the very reboot that kills the process; the post-restart
|
||||
5-min backfill re-emission is the only reliable delivery. A live+backfill cross-process
|
||||
duplicate is possible and accepted (fire-once bool resets per process; both copies carry
|
||||
the identical historical timestamp; the backfill copy is marked `backfilled:true`; the
|
||||
analyze rule uses exists-semantics). The shared 5-min lookback is NOT enlarged — the
|
||||
rule's required condition is the watermarked 2800 event; coalescing is optional
|
||||
corroboration only.
|
||||
|
||||
# Advisory — ANALYZE-ESP-005
|
||||
|
||||
"MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)", severity `warning`,
|
||||
`markSessionAsFailedDefault: false`. Required: `mdm_policy_reboot_required.rebootUri`
|
||||
exists (first URI interpolates as `{{rebootUri}}`). Optional corroboration:
|
||||
`esp_reboot_coalescing` observed (+20 confidence) and ≥2 reboot URIs (+10) on base 70 /
|
||||
threshold 70. Remediation: map each URI to its Intune profile and reassign device → user
|
||||
groups (or exclude the Autopilot device group), or accept + communicate the restart.
|
||||
Because `{{token}}` interpolation is scalar-only, the explanation points at the timeline
|
||||
for the complete URI list — each URI is its own event.
|
||||
The reboot **claim** lives here, and only here. Required conditions (BOTH):
|
||||
|
||||
1. `mdm_policy_reboot_required.firstRebootUri` exists (→ `{{firstRebootUri}}` interpolation)
|
||||
2. `system_reboot_detected` count ≥ 1 — **an actually-observed reboot**
|
||||
|
||||
Policy flags alone (e.g. AccountSetup user-policy sync) stay advisory-silent. Optional
|
||||
confidence: `uriCount ≥ 2` (+10), `omadmRebootRequiredFlag == true` (+15) on base 70 /
|
||||
threshold 70. Remediation: map URIs to Intune profiles, reassign device → user groups (or
|
||||
exclude the Autopilot device group), or accept + communicate the restart.
|
||||
|
||||
# Non-goals / invariants
|
||||
|
||||
* The new event types are **deliberately distinct from `system_reboot_detected`** — the
|
||||
backend RebootCount matches that string exactly in two places
|
||||
(`EventIngestProcessor.Classification` + the terminal recount); attribution events must
|
||||
never inflate the reboot count.
|
||||
* Event type **deliberately distinct from `system_reboot_detected`** — the backend
|
||||
RebootCount matches that string exactly in two places
|
||||
(`EventIngestProcessor.Classification` + terminal recount); attribution must never
|
||||
inflate the reboot count.
|
||||
* No session-level field / SessionInfoCard change in this iteration (follow-up candidate).
|
||||
* The `RebootRequiredURIs` catalog itself is not read (format unverified); only the 2800
|
||||
events (which carry the actually-triggered URIs) are consumed.
|
||||
* The `RebootRequiredURIs` catalog itself is not read; only the 2800 events (which carry
|
||||
the actually-triggered URIs) are consumed.
|
||||
|
||||
# Citations
|
||||
|
||||
|
||||
+9
-2
@@ -12,22 +12,29 @@ https://docs.autopilotmonitor.com (separate repository).
|
||||
# Architecture
|
||||
|
||||
* [Architecture Guide](architecture.md) - High-level architecture overview and solution layout for contributors.
|
||||
* [Version Contract](versioning.md) - One version shape for agent, backend and MCP: the ETag-CAS counter blob that mints build numbers, and the manifests published only after a deploy is verified live.
|
||||
* [URL Registry](url-registry.md) - Every well-known host lives in one registry file per component (Constants.cs / utils/config.ts / src/config.ts), enforced by guard tests; includes the agent-download migration to the download alias and its deploy-order constraint.
|
||||
|
||||
# Agent
|
||||
|
||||
* [V2 Agent](agent/index.md) - How the on-device agent works: [runtime overview](agent/overview.md), [decision engine](agent/decision-engine.md) (reducer, signals, completion arms), [logs & persistence](agent/logs-and-persistence.md) (signal log, snapshot, crash recovery), [Hello wizard un-skip](agent/hello-wizard-unskip.md) (why a single "WHfB disabled" read never decides completion), [Autopilot ZTD diagnostics](agent/autopilot-ztd-diagnostics.md) (event IDs, diagnostic registry, endpoints, error-code map).
|
||||
* [Build Counter Blob](agent/build-counter-blob.md) - How agent build numbers are minted: shared counter blob with ETag-CAS so local and CI builds never collide; CI additionally attests build provenance (Sigstore).
|
||||
* [Agent Endpoint Migration](agent/endpoint-migration.md) - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins.
|
||||
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: DM-Enterprise 2800 watcher with cross-restart watermark, Shell-Core RebootCoalescing corroboration, ANALYZE-ESP-005 advisory.
|
||||
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
|
||||
|
||||
# Rules
|
||||
|
||||
* [Gather Rule Phase Scoping & Emit-on-Change](rules/gather-rule-phase-scoping.md) - Restricting gather rules to enrollment phases (activePhases / activeFromPhase sticky latch) and emitMode "on_change" result-dedup — the anti-spam pair for interval rules.
|
||||
* [Gather Rule Phase Scoping, One-Shot Triggers & Emit-on-Change](rules/gather-rule-phase-scoping.md) - Restricting gather rules to enrollment phases (activePhases / activeFromPhase sticky latch), one-shot collection at a phase's start/end (phase_change / phase_exit), and emitMode "on_change" result-dedup.
|
||||
* [Gather Rule Guardrails](rules/gather-rule-guardrails.md) - Why the allowlists in rules/guardrails.json are enforced on the agent and nowhere else, which hard blocks survive unrestricted mode, and the rule that every collector must call a guard and emit security_warning on refusal.
|
||||
|
||||
# Backend
|
||||
|
||||
* [Business Timestamps](backend/business-timestamps.md) - Why the Azure Tables system Timestamp is never authoritative (migrations reset it), and the OccurredUtc + RowKey-decode compensation for AuditLogs/OpsEvents/Events after the 2026-07-18 migration.
|
||||
|
||||
# MCP Server
|
||||
|
||||
* [MCP Docs Corpus](mcp/docs-corpus.md) - How the published customer documentation becomes the `search_docs` corpus: CRLF-safe heading chunking, build-time embedding as a third precomputed section, and why a docs edit needs an MCP redeploy.
|
||||
|
||||
# Security & Identity
|
||||
|
||||
* [MCP OAuth Flow](mcp-oauth-flow.md) - Who authenticates where when connecting an AI client to the MCP server; two identities, three parties plus Entra ID.
|
||||
|
||||
+29
@@ -1,7 +1,36 @@
|
||||
# Log
|
||||
|
||||
## 2026-07-23
|
||||
|
||||
* **Update**: `url-registry.md` — the first release after the alias migration surfaced Front Door caching as a correctness hazard: the alias served the previous `version.json` (`X-Cache: TCP_REMOTE_HIT`) after the origin had rotated, and since manifest and ZIP are independent cache entries, a stale manifest with a fresh ZIP fails the bootstrap SHA-256 check. Route caching on `/agent/*` is now disabled and every blob upload stamps `Cache-Control: no-cache` (CI `Send-Blob`, local `build.ps1`, `versions/*.json` manifests) — all agent-container blobs rotate in place, so nothing there is safely cacheable.
|
||||
|
||||
* **Update**: `agent/decision-engine.md` — the RealmJoin completion gate has a third terminal, `FirstDeploymentIncomplete`. Session 224b2087 showed why: an interactive logon 17 s after DeploymentPhase 101 makes RealmJoin reclassify the run as a secondary-user deployment, whose branch writes 200/210 and never 110 — the gate stayed closed until the 60-min hard timeout although Hello and desktop were long resolved. `realmjoin_phase_changed` is now a typed `RealmJoinPhaseChanged` signal, so `RealmJoinFacts.LastDeploymentPhase` is persisted across agent restarts (this also fixes every `realmjoin_timeout` event claiming "last phase: 0"); a transition out of 100/101 into 200/210 releases the gate with the `realmjoin_first_deployment_incomplete` Warning, and the `RealmJoinDetected` replay path evaluates the same rule for restarts that swallow the transition. A first observation of 210 deliberately does not release — session 6f1959c0 started at 210 and only deployed afterwards.
|
||||
|
||||
* **Creation**: Added `url-registry.md` — all well-known hosts (own subdomains, both blob accounts, API origin, Graph, Entra login) are now defined once per component (`Constants.cs`, web `utils/config.ts`, MCP `src/config.ts`) and referenced everywhere else; three guard tests fail the build on any literal outside the registries. Written because the EU cutover missed two hardcoded copies of the legacy blob host — `LatestVersionsService` (feeds the portal's version badge) and `ValidateBootstrapCodeFunction` — which kept working only through the fail-soft legacy mirror in `build-agent.yml`; those two plus the agent `SelfUpdater` now use the `download.autopilotmonitor.com` alias (verified serving all four artifacts first). The portal's bootstrap allow-list accepts alias AND legacy host during the transition, which imposes a deploy order: web before (or with) backend. The sweep also surfaced a fourth copy of the www/portal host pair in `middleware.ts` (now derived from the registry via `hostRouting.ts`) and a backend short-link built on the bare apex domain, which only worked through a registrar-level 301.
|
||||
|
||||
## 2026-07-22
|
||||
|
||||
* **Creation**: Added `versioning.md` — agent, backend and MCP now share one version shape (`<major>.<minor>.<build>` + 7-char commit + build timestamp) and one implementation of the build-number reservation (`.github/scripts/Request-BuildNumber.ps1`, previously copy-pasted between `build-agent.yml` and `build.ps1`). The backend had been pinned at a hand-written `1.0.0` since inception and now starts at `1.5.1000`; the MCP server's `package.json` version was bumped by hand and now starts at `1.5.300`, and it reports its own commit on `/health` instead of only the documentation bundle's. Both deploy workflows poll their health endpoint until it reports the build just deployed and publish `versions/<component>.json` only then, so a manifest that disagrees with the live endpoint means the instance drifted rather than the pipeline failing halfway. `agent/build-counter-blob.md` keeps the agent-specific parts and now points at the shared protocol; its counter URL was still the pre-cutover storage account.
|
||||
|
||||
* **Creation**: Added `rules/gather-rule-guardrails.md` — the gather-rule allowlists are enforced only on the agent (the backend validates no `collectorType`, `target`, or `parameters`), so a collector that forgets to call its guard *is* the vulnerability. Written because two had: `LogParserCollector` read any path in `rule.Target`, bypassing even the `C:\Users` hard block that every other file collector honours, and `EventLogCollector` read any channel including `Security` — while the customer-facing documentation already promised both were allowlisted. Both now guard, with `logparser` checking the directory for wildcard targets (`Path.GetFullPath` throws on `*`) and re-checking each resolved path. Added an event-log channel allowlist (`eventLogChannels`) plus hard-blocked channels held in C# rather than JSON, so a parse error cannot lift them. Also fixed the command timeout, which was inert: `ReadToEnd()` blocks until the pipe closes, so `WaitForExit(30000)` was only reached after the process had already exited and a hung command pinned its worker forever — now read asynchronously and killed at the timeout.
|
||||
|
||||
* **Update**: Added `.github/workflows/ci.yml` — the full suite (backend, agent V2 Core, DecisionCore, SummaryDialog, web, MCP) now runs on every pull request and every push to main, plus a job that fails if the generated guardrail files drift from `rules/guardrails.json`. Previously **no** workflow ran tests on a PR: the deploy workflows build without testing, and `build-agent.yml` tested only `DecisionCore.Tests` on manual dispatch — while `trust/security-faq.md` already told customers that automated tests run in CI. Added CodeQL (C# and TypeScript) and Dependabot (nuget, npm ×2, actions). Marked `OfficeBinaryWatcherTests` `[Collection("SerialThreading")]`: its bounded re-probe races the shared ThreadPool and failed once under full-suite load.
|
||||
|
||||
## 2026-07-21
|
||||
|
||||
* **Update**: `search_docs` and `search_knowledge` now state in their tool descriptions that queries must be in English. Both corpora and `all-MiniLM-L6-v2` are English-only, and the failure is total rather than gradual — a German question returns zero results, not weak ones. The caller is a language model, so translating the question before the call and answering in the user's language costs nothing; a multilingual embedding model would be the alternative but is much larger and would have to be measured against the eval harness first, since it changes English retrieval too.
|
||||
|
||||
* **Update**: Added `scripts/eval-docs-search.ts`, an offline retrieval evaluation harness (2465 queries in three classes: chunk headings as a regression detector, rare identifiers, and hand-written questions as the reality anchor), and used it to close a third `search_docs` gap found during the live test. The keyword fallback only fired when semantic search returned *fewer* than `topK`; a pasted identifier like `WOW6432Node` instead returned three chunks at ~0.29 that did not contain the word while the one chunk that did was never considered. It now also fires on a weak semantic head (<0.35), letting exact matches take the front while keeping one semantic result. The threshold is measured, not guessed: 0.35 fixes 335 queries and breaks 0 (identifier recall 66.2% → 81.1%), while 0.40 fixes more but starts demoting hand-written questions. The measurement also resized the problem — identifier queries were only succeeding 66.2% of the time, so the reported case was one of roughly 750.
|
||||
|
||||
* **Update**: Closed the two measured recall gaps in `search_docs` (`mcp/docs-corpus.md`). (1) **Token windows** — 87 of 324 chunks exceeded the model's 256-token context and 13% of all tokens never reached the ranking; a character budget cannot fix this (the corpus ranges 2.43–5.81 chars/token), so long text is now embedded as 250-token windows with 40 overlap and scored by its BEST window (`embeddings: number[][]`, max not mean). Token coverage 87% → 100%; "proxy configuration blocking app downloads" went from absent to 0.559 at rank 1; the ten reference queries kept byte-identical scores. Costs +31% vectors, index 2.63 → 3.12 MB. The rules corpus gained the same treatment (128 rules → 172 vectors). (2) **Keyword fallback** — when the semantic pass returns fewer than `topK`, results are topped up via literal `lexicalMatch` on distinctive query terms, requiring two matching terms for corroboration (one incidental substring match returned a page about LLM providers for a sourdough query). Fuzzy matching was measured and rejected: at a threshold low enough to surface "RealmJoin" it invents 5 hits for "SCEPman", which the corpus never mentions. Also sharpened the tool description with the measured context argument for preferring it over fetching the docs site.
|
||||
|
||||
* **Creation**: Added `mcp/docs-corpus.md` (new `mcp/` section) — the MCP server gains `search_docs`, a third semantic corpus built from the published customer documentation repository (`autopilotmonitor-docs`, checked out into the Docker build context by the deploy workflow). Documents the h2/h3 + `<details>` chunking with breadcrumb prefixes, the two deliberate size exceptions (tables never split, unbroken changelog lists split at line level), the CRLF precondition (`.` does not match `\r` in JS — headings silently stop matching, which produced 286 wrong-but-plausible chunks before it was found), the optional `docs` section in `search-index.json` and its validation asymmetry, embedding rounding to 6 decimals, and the cross-repo coupling: a docs edit reaches clients only after an MCP redeploy, made checkable via `DOCS_COMMIT` on `/health`. Container resources raised to 0.5 vCPU / 1.0 GiB for the cold-start query-embedder load.
|
||||
|
||||
## 2026-07-20
|
||||
|
||||
* **Update**: `rules/gather-rule-phase-scoping.md` — the gather-rule form now shows the phase-scope and emit-mode controls only where they can change behavior (`supportsPhaseScope` / `supportsEmitMode`), nulls hidden-control state out of the payload, and rejects a scope mode selected but left unfilled. Offering all three "when" controls unconditionally allowed combinations that could only suppress a one-shot rule, plus a silent no-op state.
|
||||
* **Update**: Extended `rules/gather-rule-phase-scoping.md` with the `phase_exit` trigger — the closing bookend to `phase_change`, for one-shot collection when a phase is LEFT (e.g. "state at the end of Account Setup"). Documents the evaluation-before-phase-advance rule (triggerPhase match and scope gate see the phase being left), firing into `Failed`, the disjoint `exit:` dedup key space, and why no ConfigVersion bump is needed (no new field — old agents simply never fire the rule).
|
||||
* **Update**: Reworked `agent/mdm-reboot-coalescing.md` after the first real enrollment (session b2e890c1): 2800 records are now **aggregated per burst** into ONE neutral Info event (`rebootUris[]`/`uriCount`/`firstRebootUri`; watermark persisted at flush so a pre-emit reboot kill loses nothing); the raw event no longer claims a reboot — ANALYZE-ESP-005 is gated on an actually-observed `system_reboot_detected`. **Removed** the Shell-Core `esp_reboot_coalescing` event entirely: the "RebootCoalescing" token appears in the routine `SubcategoryProcessing_Started` bootstrap marker on every enrollment (false positive by construction). Verified 2800 description shape recorded.
|
||||
* **Creation**: Added `agent/mdm-reboot-coalescing.md` — the agent now attributes the "unexpected mid-ESP reboot + second sign-in" pattern (PMPC research) to the device-assigned policy URIs that forced it: new `MdmRebootPolicyTracker` (DM-Enterprise EventID 2800, per-URI `mdm_policy_reboot_required` events, cross-restart RecordId watermark), a `RebootCoalescing` branch in `ShellCoreTracker` (`esp_reboot_coalescing`, backfill-emitting by design), and the `ANALYZE-ESP-005` advisory (reassign device → user groups). ConfigVersion 35; event types deliberately distinct from `system_reboot_detected` so RebootCount stays uninflated.
|
||||
|
||||
* **Creation**: Added `backend/business-timestamps.md` (new `backend/` section) — the 2026-07-18 migration reset every row's system Timestamp, exposing that AuditLogs/OpsEvents/Events had (unknowingly) displayed and filtered on it all along; documents the `OccurredUtc` column, the RowKey decoders and RowKey-range date/retention clauses in `BusinessTimestamp`, the GUID-guard trap, the audit/ops backfill endpoint, and the "system Timestamp is never authoritative" migration rule.
|
||||
|
||||
@@ -0,0 +1,305 @@
|
||||
---
|
||||
type: Concept
|
||||
title: MCP Docs Corpus — search_docs and the Cross-Repo Build Coupling
|
||||
description: How the published customer documentation becomes a third semantic search corpus in the MCP server — CRLF-safe heading chunking, build-time embedding, the corpus hash guard, and why editing the docs does not update a running server.
|
||||
resource: /src/McpServer/autopilot-monitor-mcp/src/docs-corpus.ts
|
||||
tags:
|
||||
- mcp
|
||||
- search
|
||||
- embeddings
|
||||
- documentation
|
||||
- build
|
||||
timestamp: 2026-07-21T19:30:00+02:00
|
||||
---
|
||||
|
||||
# Purpose
|
||||
|
||||
The MCP server answers two different kinds of question, and they need two
|
||||
different corpora.
|
||||
|
||||
* *"Why did this enrollment fail?"* → analysis rules, gather rules, IME log
|
||||
patterns. That is `search_knowledge`, backed by `knowledge-base.ts`.
|
||||
* *"How do I deploy the agent?" / "Where is my data stored?"* → the published
|
||||
customer documentation. That is `search_docs`, backed by `docs-corpus.ts`.
|
||||
|
||||
They are deliberately **separate indexes**. Merged into one, they would compete
|
||||
for the same `topK` and dilute each other: a rules query would surface prose
|
||||
pages, and a product question would surface rule definitions.
|
||||
|
||||
# Schema
|
||||
|
||||
## Corpus source
|
||||
|
||||
The corpus is the `autopilotmonitor-docs` repository — a public OKF bundle
|
||||
published at `docs.autopilotmonitor.com`. It is **not** vendored into this repo.
|
||||
The deploy workflow checks it out into `mcp-docs/` of the Docker build context;
|
||||
the Dockerfile copies it to `/app/docs` in both the build and the final stage
|
||||
(byte-identical, or the boot-time corpus hash rejects the precomputed index).
|
||||
|
||||
`SUMMARY.md` and `log.md` are skipped as navigation scaffolding. `index.md` is
|
||||
kept — it is the documentation map and answers "what documentation exists" — but
|
||||
gets no `url`, because the bundle deliberately keeps it out of `SUMMARY.md` and
|
||||
GitBook therefore never renders it.
|
||||
|
||||
## Chunking
|
||||
|
||||
A whole page cannot be one document: `trust/security-faq.md` is 36 KB, while
|
||||
MiniLM-L6-v2 truncates at ~256 tokens. An un-chunked page would silently embed
|
||||
only its first paragraph.
|
||||
|
||||
Chunks are cut at **h2/h3 headings**, which in this bundle are also the semantic
|
||||
boundaries — `security-faq.md` carries 37 h3s, one per question.
|
||||
`<details><summary>Q</summary>` blocks (used by `troubleshooting/faq.md`) are
|
||||
treated the same way, with the summary as the section title. Each chunk is
|
||||
prefixed with a `Page Title › Section › Subsection` breadcrumb so a hit lifted
|
||||
out of the middle of a page is self-describing to both the embedder and the
|
||||
reader.
|
||||
|
||||
Two deliberate exceptions to the ~1500-character budget:
|
||||
|
||||
* **Markdown tables are never split.** A table broken across chunks leaves
|
||||
`| --- |` rows with no header — unreadable in a result and meaningless to
|
||||
embed. Tables carry real answers here (the role matrix, the settings
|
||||
reference), so an over-long table is allowed to exceed the budget.
|
||||
* **Unbroken lists split at line level.** The changelog pages are one bullet
|
||||
list per release with no blank lines, so paragraph splitting cannot touch
|
||||
them; left whole, everything past the model's window would be truncated away
|
||||
and become unsearchable.
|
||||
|
||||
Short sections are **kept**, not filtered. A one-line answer ("Is it free?" →
|
||||
"Yes, the Community plan is free") is often the most valuable hit, and the
|
||||
breadcrumb supplies the context the short body lacks. Only structural residue
|
||||
(under 25 alphanumeric characters) is dropped.
|
||||
|
||||
## CRLF is a correctness precondition
|
||||
|
||||
The docs repo is authored on Windows and ships CRLF. In JavaScript `.` does not
|
||||
match `\r`, so a pattern like `/^(#{2,3})\s+(.*)$/` matches **nothing** on a CRLF
|
||||
file: `(.*)` stops before the `\r` and `$` never reaches end-of-string.
|
||||
|
||||
This was observed during development, and the failure mode is what makes it
|
||||
dangerous: no crash, no error — headings and frontmatter simply stop being
|
||||
recognized and every page silently degrades to unlabelled paragraph splits. The
|
||||
first working build produced 286 plausible-looking chunks that were almost all
|
||||
wrong; after normalizing, `security-faq.md` went from 29 blind splits to 46 real
|
||||
question sections.
|
||||
|
||||
`normalizeNewlines()` runs before any parsing. Do not remove it, and do not
|
||||
"fix" a future parsing bug here by adding `\r?` to one pattern.
|
||||
|
||||
## Token windows: why a chunk can hold several vectors
|
||||
|
||||
Chunking at heading boundaries keeps sections coherent, but it does not bound the
|
||||
result by *tokens* — and it cannot. Measured on this corpus the ratio ranges from
|
||||
**2.43 to 5.81 characters per token**, a factor of 2.4, so no character budget maps
|
||||
reliably onto the model's 256-token window. A budget safe for the worst case
|
||||
(~620 characters) would shred prose and tables alike.
|
||||
|
||||
Left alone, 87 of 324 chunks exceeded the window and **13% of all tokens never
|
||||
reached the ranking** — silently, since the tokenizer truncates without complaint.
|
||||
The effect was concrete: in `built-in-rules.md :: Apps` (734 tokens), the strings
|
||||
`ANALYZE-APP-013`, `ANALYZE-OFFICE-001`, `ANALYZE-CORR-003` and `Click-to-Run` all
|
||||
sit past the cutoff, so *"proxy configuration blocking app downloads"* did not
|
||||
retrieve that section at all.
|
||||
|
||||
So the *chunk* stays whole and the *embedding* is split instead:
|
||||
`VectorSearchProvider.index()` cuts long text into 250-token windows with 40 tokens
|
||||
of overlap and embeds each. A document therefore carries `embeddings: number[][]`,
|
||||
and `search()` scores it by its **best** window.
|
||||
|
||||
- **Max, not mean.** Averaging window vectors would blur a long section's several
|
||||
topics into one mediocre direction — the same dilution that makes large context
|
||||
windows worse than chunking. Max keeps a document findable by any one thing it
|
||||
discusses.
|
||||
- **The tokenizer is a build-time dependency only.** `index()` runs during the
|
||||
Docker build; production boots from precomputed vectors and never loads it. It is
|
||||
also only the vocabulary, not the ONNX weights (~93 ms).
|
||||
- Cost: 324 chunks → 424 vectors (+31%), index 2.63 → 3.12 MB, and 424 instead of
|
||||
324 dot products per query — still microseconds.
|
||||
- The rules corpus benefits too, unplanned: 128 rules now carry 172 vectors.
|
||||
|
||||
Measured effect: token coverage 87% → **100%**, *"Office Click-to-Run install never
|
||||
finished"* 0.318 → **0.442**, *"proxy configuration blocking app downloads"* absent
|
||||
→ **0.559 at rank 1**. The ten reference queries kept byte-identical scores — their
|
||||
chunks are short, hence one window, hence the same vector as before.
|
||||
|
||||
## Keyword fallback
|
||||
|
||||
Semantic ranking alone leaves a gap: proper nouns embed poorly. `RealmJoin` occurs
|
||||
in 3 chunks yet scores under the threshold.
|
||||
|
||||
Fuzzy matching is the wrong remedy, and the corpus says so. At a threshold low
|
||||
enough to surface `RealmJoin` (0.28), Fuse also returns 5 hits for `SCEPman` — a
|
||||
word this corpus contains **zero** times. Literal containment has neither problem.
|
||||
|
||||
So when the semantic pass returns fewer than `topK`, `search_docs` tops the list up
|
||||
via `lexicalMatch`, the substring scan every provider already exposes:
|
||||
|
||||
- Query terms are filtered to distinctive ones (≥4 characters, non-stopword, max 6).
|
||||
- A hit needs **corroboration**: at least two matching terms, unless the query has
|
||||
only one. Without this rule, "how do I bake sourdough bread" matched a page about
|
||||
LLM providers, because "bake" is a substring of "baked into the image".
|
||||
- Hits are **appended, never merged**: they carry no cosine similarity, so semantic
|
||||
results keep the head of the list and keyword hits fill the tail, flagged
|
||||
`matchType: "keyword"` with `keywordFallback` in the response.
|
||||
|
||||
### Language
|
||||
|
||||
The corpus and `all-MiniLM-L6-v2` are both English-only, and the failure is total
|
||||
rather than gradual: "wo werden meine Daten gespeichert" returns **zero** results,
|
||||
not weak ones, because the query vector lands nowhere near any English chunk.
|
||||
|
||||
The fix is a sentence in the tool description telling the caller to query in English
|
||||
and answer in the user's language — the caller is a language model, so translating
|
||||
the question before the call costs nothing. A multilingual embedding model would
|
||||
handle it natively but is substantially larger and would need to be measured against
|
||||
`scripts/eval-docs-search.ts` first, since it changes English retrieval too.
|
||||
`search_knowledge` carries the same note for the same reason.
|
||||
|
||||
### When it fires
|
||||
|
||||
Two triggers, and the second one exists because the first was not enough:
|
||||
|
||||
1. **Fewer than `topK` semantic hits** — the obvious case.
|
||||
2. **A weak semantic head** (best score below `WEAK_SEMANTIC_SCORE`, 0.35). Topping
|
||||
up only a short list misses the more common failure: a pasted identifier like
|
||||
`WOW6432Node` returns three chunks at ~0.29 that do not contain the word, while
|
||||
the single chunk that does never gets considered. When nothing semantic is
|
||||
convincing, exact matches take the front — keeping at least one semantic result,
|
||||
so a weak-but-correct hit is never pushed out entirely.
|
||||
|
||||
The threshold was chosen by measurement, not intuition. `scripts/eval-docs-search.ts`
|
||||
scores strategies over 2465 queries in three classes — every chunk heading
|
||||
(known-item, a regression detector), rare identifiers (the class this fallback
|
||||
serves), and hand-written questions (the reality anchor, not derived from the
|
||||
corpus):
|
||||
|
||||
| strategy | known-item | literal | handwritten MRR | fixed | broken |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| append-only (previous) | 71.9% | 66.2% | 1.000 | — | — |
|
||||
| weak < 0.30 | 71.9% | 72.6% | 1.000 | 142 | 0 |
|
||||
| **weak < 0.35** | **73.7%** | **81.1%** | **1.000** | **335** | **0** |
|
||||
| weak < 0.40 | 76.3% | 88.2% | 0.944 | 502 | 3 |
|
||||
| weak < 0.45 | 77.2% | 93.0% | 0.889 | 613 | 5 |
|
||||
|
||||
0.35 is strictly dominant: better in every class, worse in none. Higher thresholds
|
||||
keep repairing more identifier queries but start demoting hand-written questions —
|
||||
the class that most resembles real use — so the gain is not free past that point.
|
||||
|
||||
The measurement also corrected the size of the problem: identifier queries were
|
||||
succeeding only 66.2% of the time, so `WOW6432Node` was not an outlier but one of
|
||||
roughly 750 such failures.
|
||||
|
||||
## Build-time embedding
|
||||
|
||||
Like the rules corpus, docs are embedded once at Docker build time by
|
||||
`precompute-embeddings.js <rulesDir> <outFile> [docsDir]`, into a third
|
||||
`PrecomputedSection` (`docs`) of `search-index.json`. The container never embeds
|
||||
a corpus at boot — only the incoming query.
|
||||
|
||||
The `docs` section is **optional** in the file format, which is what makes a
|
||||
local checkout without the docs repo work: `validatePrecomputedIndex` skips it
|
||||
when the booting server loaded zero doc chunks. The asymmetry is intentional —
|
||||
if the server *has* a docs corpus and the file does *not*, that is staleness and
|
||||
is rejected like any other mismatch.
|
||||
|
||||
Embeddings are rounded to 6 decimals on serialize. A float64 component
|
||||
serializes to ~20 characters of JSON, so 384 dims cost ~7.8 KB per vector;
|
||||
rounding roughly halves the index file, which is `JSON.parse`d on every
|
||||
scale-to-zero cold start. The induced L2 norm error (~1e-5 over 384 dims) sits
|
||||
two orders of magnitude inside `UNIT_NORM_EPSILON`.
|
||||
|
||||
## Registration
|
||||
|
||||
`search_docs` is registered for **every role**, unlike the platform tools: this
|
||||
is public documentation carrying no tenant data, and a tenant user asking "how do
|
||||
I deploy the agent" has as much business here as a Global Admin. It is skipped
|
||||
entirely when the corpus is empty, so a doc-less build advertises no broken tool.
|
||||
|
||||
# Cold start
|
||||
|
||||
`minReplicas: 0` means every idle period ends in a cold start someone waits through.
|
||||
Measured from outside: **24.5 s** to first byte after `ScaledToZero`, against **60 ms**
|
||||
warm.
|
||||
|
||||
Where those seconds go could not be answered from outside the process. Log Analytics
|
||||
stamps every console line of one ingestion batch with the same time, so the boot
|
||||
sequence collapses to a single instant however it is queried. The server therefore
|
||||
times itself — `[boot +Nms]` marks from `process.uptime()`, which includes Node's own
|
||||
startup and module loading.
|
||||
|
||||
Measured in production on a real cold start (31.5 s to first byte that time):
|
||||
|
||||
```
|
||||
+244ms node started (startup + module loading)
|
||||
+431ms corpora read from disk (+187ms)
|
||||
+491ms index read, validated, hydrated (+60ms <- for 3.12 MB)
|
||||
+504ms listening
|
||||
+677ms query embedder warm (background)
|
||||
```
|
||||
|
||||
**The application accounts for half a second of it.** The other ~31 s — 98% — is
|
||||
Container Apps activation: KEDA reacting to the request, scheduling a pod, mounting
|
||||
the 236 MB image, starting the container. The container also boots *faster* than a
|
||||
development laptop (504 ms against 1273 ms), so nothing here is CPU-starved.
|
||||
|
||||
Three conclusions, and two of them overturn assumptions this file previously stated:
|
||||
|
||||
- **Index size is irrelevant to startup.** 3.12 MB hydrates in 60 ms. Growing the
|
||||
corpus is effectively free, and the embedding rounding earns its keep on file size
|
||||
alone, not on boot time.
|
||||
- **Raising the container to 0.5 vCPU was not justified by cold start.** The stated
|
||||
reason was the embedding model load; prebaked, that is 173 ms and runs in the
|
||||
background, where only the first *search* could ever wait on it. The extra vCPU may
|
||||
still help concurrent requests, but the cold-start argument does not survive
|
||||
measurement.
|
||||
- **The only real lever is `minReplicas: 1`**, which removes activation entirely at
|
||||
roughly 7× the monthly free consumption grant. Shrinking the image would shave the
|
||||
mount, but that is a fraction of platform time, not the bulk of it.
|
||||
|
||||
# Cross-repo coupling
|
||||
|
||||
**Editing the documentation does not update a running MCP server.** The corpus is
|
||||
baked into the image, so only re-running the *Deploy MCP Server* workflow picks
|
||||
up doc changes.
|
||||
|
||||
That coupling is made checkable rather than memorable: the build passes the docs
|
||||
checkout SHA as `--build-arg DOCS_COMMIT`, and `/health` reports
|
||||
`docs: { commit, chunks, sections }`. Comparing that commit against
|
||||
`git rev-parse HEAD` in `autopilotmonitor-docs` answers "is the deployed
|
||||
documentation stale?" directly.
|
||||
|
||||
Automatic redeploy on a docs push was considered and rejected: it would need a
|
||||
PAT secret in the docs repo and would turn every typo fix into a container
|
||||
deployment.
|
||||
|
||||
# Examples
|
||||
|
||||
Retrieval measured against the real bundle (323 chunks, all-MiniLM-L6-v2):
|
||||
|
||||
```
|
||||
"where is my data stored" → 0.566 troubleshooting/faq.md :: General › Where is my data stored?
|
||||
"how do I deploy the agent with Intune" → 0.819 troubleshooting/faq.md :: Setup & Agent › How do I deploy the agent?
|
||||
"which roles exist and what can they do" → 0.500 trust/security-faq.md :: Identity and Access › What roles exist?
|
||||
"how do I set up Teams notifications" → 0.592 reference/settings.md :: Tenant › Notifications
|
||||
"what firewall URLs do I need to allow" → 0.374 reference/network-endpoints.md :: (intro)
|
||||
```
|
||||
|
||||
The 0.25 `minScore` default is tuned to that observed band — short keyword
|
||||
queries on all-MiniLM score low, and a relevant-but-marginal hit lands around
|
||||
0.34.
|
||||
|
||||
# Citations
|
||||
|
||||
* `/src/McpServer/autopilot-monitor-mcp/src/docs-corpus.ts` — loader, frontmatter
|
||||
parser, chunker
|
||||
* `/src/McpServer/autopilot-monitor-mcp/src/tools/search.ts` — `search_docs`
|
||||
registration
|
||||
* `/src/McpServer/autopilot-monitor-mcp/src/precomputed-index.ts` — `docs`
|
||||
section and its validation asymmetry
|
||||
* `/src/McpServer/autopilot-monitor-mcp/Dockerfile` — `mcp-docs/` copy,
|
||||
`DOCS_COMMIT`
|
||||
* `/.github/workflows/deploy-mcp.yml` — docs checkout and SHA capture
|
||||
* `/src/McpServer/autopilot-monitor-mcp/scripts/eval-docs-search.ts` — retrieval
|
||||
evaluation harness; run it before changing anything about ranking
|
||||
* [MCP OAuth Flow](../mcp-oauth-flow.md) — the other MCP concept document
|
||||
@@ -0,0 +1,81 @@
|
||||
---
|
||||
type: Concept
|
||||
title: Gather Rule Guardrails
|
||||
description: >-
|
||||
How gather-rule collectors are fenced in — the allowlists in rules/guardrails.json,
|
||||
the hard blocks that survive unrestricted mode, and why every collector must call a
|
||||
guard rather than trusting rule.Target.
|
||||
resource: src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather
|
||||
tags: [gather-rules, security, guardrails, agent]
|
||||
timestamp: 2026-07-22
|
||||
---
|
||||
|
||||
# Concept
|
||||
|
||||
A gather rule is authored by a **tenant administrator** through `POST /api/rules/gather` and executed by the agent as **SYSTEM** on an enrolling device. The backend does not validate `collectorType`, `target`, or `parameters` at all — it only enforces scope/emit-mode fields, a 1 MB body cap, and that the rule lands in the caller's own tenant partition. Everything that decides *what a rule may reach* is therefore enforced **on the agent**, in `GatherRuleGuards`.
|
||||
|
||||
This is the security boundary. The portal's `ValidationIndicator` is a convenience: it does not block submit, and the API is reachable directly.
|
||||
|
||||
# Schema
|
||||
|
||||
Allowlists come from `rules/guardrails.json`, embedded into the agent assembly as
|
||||
`AutopilotMonitor.Agent.V2.Core.Resources.guardrails.json` (see the `EmbeddedResource`
|
||||
item in `AutopilotMonitor.Agent.V2.Core.csproj`) and regenerated into the portal's
|
||||
`utils/guardrails.generated.ts` by `node rules/scripts/combine.js`.
|
||||
|
||||
| Guard | Source list | Match rule |
|
||||
| --- | --- | --- |
|
||||
| `IsRegistryPathAllowed` | `registryPrefixes` | prefix, boundary `\` or end |
|
||||
| `IsFilePathAllowed` | `filePrefixes` | `Path.GetFullPath` first, then prefix + boundary |
|
||||
| `IsWmiQueryAllowed` | `wmiQueryPrefixes` | prefix, boundary whitespace or end |
|
||||
| `IsCommandAllowed` | `allowedCommands` | **exact** match, trimmed — never prefix |
|
||||
| `IsEventLogChannelAllowed` | `eventLogChannels` | prefix, boundary `/` or end |
|
||||
|
||||
**Hard blocks live in code, not in the JSON**, so a parse error cannot lift them — on a
|
||||
parse failure the allowlists fall back to empty and everything is blocked (fail-closed),
|
||||
but the hard blocks must hold regardless of configuration:
|
||||
|
||||
* `BlockedUsersPrefix` (`C:\Users`) and `AdditionalHardBlockedPathPrefixes`
|
||||
(`C:\Windows\System32\config`) — exception only for `%LOGGED_ON_USER_PROFILE%`
|
||||
under `AppData\Local` / `AppData\Roaming`.
|
||||
* `HardBlockedCommandPatterns` — download, user/group creation, boot config,
|
||||
persistence, destructive operations; plus `MaxCommandLength`.
|
||||
* `HardBlockedEventLogChannels` — `Security` (audit trail of user behaviour) and the
|
||||
PowerShell channels (script-block logging routinely contains secrets in clear text).
|
||||
|
||||
`rules/guardrails.json` mirrors the blocked lists (`blockedFilePrefixes`,
|
||||
`blockedEventLogChannels`) **for display only**. Enforcement is the C# constant.
|
||||
|
||||
# Examples
|
||||
|
||||
Every collector must call its guard and return `context.EmitSecurityWarning(...)` on
|
||||
refusal — that emits the `security_warning` event that makes a block visible in the
|
||||
timeline instead of failing silently.
|
||||
|
||||
```csharp
|
||||
// FileCollector / JsonCollector / XmlCollector
|
||||
if (!GatherRuleGuards.IsFilePathAllowed(filePath, context.UnrestrictedMode, userProfilePath))
|
||||
return context.EmitSecurityWarning(rule, "file", filePath);
|
||||
```
|
||||
|
||||
Two collectors were added to this contract on 2026-07-22 after shipping without it:
|
||||
|
||||
* **`LogParserCollector`** read any path in `rule.Target`, bypassing even the `C:\Users`
|
||||
hard block. It cannot guard `rule.Target` directly — a wildcard filename makes
|
||||
`Path.GetFullPath` throw — so it guards **the directory** when the filename contains
|
||||
`*`/`?`, and re-guards **each resolved path** after wildcard expansion (a junction can
|
||||
surface a file the directory check did not cover).
|
||||
* **`EventLogCollector`** read any channel, including `Security`.
|
||||
|
||||
`ImeLogPathOverride` comes only from the local `--ime-log-path` CLI flag, never from
|
||||
remote config. The operator using it is already a local admin, so it relaxes the
|
||||
allowlist exactly the way unrestricted mode does — hard blocks still apply.
|
||||
|
||||
# Citations
|
||||
|
||||
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleGuards.cs`
|
||||
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleContext.cs` — `EmitSecurityWarning`
|
||||
* `src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/Gather/GatherRuleCollectorGuardTests.cs`
|
||||
* `src/Backend/AutopilotMonitor.Functions/Functions/Rules/GatherRulesFunction.cs` — what the backend does *not* validate
|
||||
* `rules/guardrails.json`, `rules/scripts/combine.js`
|
||||
* Customer-facing counterpart: `autopilotmonitor-docs/rules/gather-rules.md` (§ Security guardrails)
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: Concept
|
||||
title: Gather Rule Phase Scoping & Emit-on-Change
|
||||
description: How gather rules are restricted to enrollment phases (activePhases / activeFromPhase) and how emitMode "on_change" suppresses repeated identical results — the anti-spam pair for interval rules.
|
||||
title: Gather Rule Phase Scoping, One-Shot Triggers & Emit-on-Change
|
||||
description: How gather rules are restricted to enrollment phases (activePhases / activeFromPhase), collect once at a phase boundary (phase_change / phase_exit), and how emitMode "on_change" suppresses repeated identical results.
|
||||
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleExecutor.cs
|
||||
tags:
|
||||
- agent
|
||||
@@ -35,6 +35,35 @@ Backend validation (`GatherRulesFunction.ValidateScopeAndEmitMode`) rejects `Unk
|
||||
numeric tokens, unknown emit modes, and both scope fields set at once (400). The agent
|
||||
defensively prefers `activePhases` if both arrive anyway.
|
||||
|
||||
# One-shot collection at phase boundaries
|
||||
|
||||
Scoping answers "while *may* this rule run". The trigger answers "when does it fire". For
|
||||
one-shot collection the two phase triggers are the bookends of a phase:
|
||||
|
||||
| Trigger | Fires |
|
||||
|---|---|
|
||||
| `phase_change` | Once when `triggerPhase` is **entered** (dedup key `{ruleId}\|{phase}`). |
|
||||
| `phase_exit` | Once when `triggerPhase` is **left** (dedup key `exit:{ruleId}\|{phase}`). |
|
||||
|
||||
Empty `triggerPhase` means every transition. Both keys live in the same
|
||||
`_phaseRulesExecuted` set; the `exit:` prefix keeps the two key spaces disjoint, so an
|
||||
enter-rule and an exit-rule on the same phase never cannibalise each other's slot.
|
||||
|
||||
`phase_exit` semantics that are easy to get wrong:
|
||||
|
||||
* It is evaluated **before** `_currentPhase` advances, so both the `triggerPhase` match and
|
||||
the phase-scope gate see the phase being **left**. A rule scoped to `DeviceSetup` therefore
|
||||
still fires on its own exit, even though the phase being entered is outside that scope.
|
||||
* A transition into `Failed` **does** fire the exit rules of the failing phase — capturing
|
||||
state at the failure boundary is the point.
|
||||
* Exit of `Unknown` never fires (nothing was entered yet), and a repeated identical phase
|
||||
signal is not a transition.
|
||||
* An exit rule on `Complete` only fires if something transitions *away* from Complete. For
|
||||
"at the end of enrollment", use `phase_change` on `Complete` or `on_event` with
|
||||
`enrollment_complete`.
|
||||
* Old agents do not know the trigger value and simply never fire the rule (silent no-op) —
|
||||
which is why `phase_exit` needs no ConfigVersion bump: no new field is transported.
|
||||
|
||||
# Scope semantics (agent, `GatherRuleExecutor`)
|
||||
|
||||
* Scope applies to **all trigger types**:
|
||||
@@ -78,6 +107,16 @@ defensively prefers `activePhases` if both arrive anyway.
|
||||
`always`. The scope UI is a three-way mode (always / during phases / from phase) backed by
|
||||
a canonical `GATHER_PHASES` list — the same list replaced the free-text `triggerPhase`
|
||||
input, whose placeholder used to suggest invalid tokens.
|
||||
* **Controls are shown only where they change behavior** (`supportsPhaseScope` /
|
||||
`supportsEmitMode` in `types.ts`). Offering all three "when" controls unconditionally
|
||||
produced meaningless combinations — a scope on a `phase_change` rule naming a concrete
|
||||
phase can only ever suppress the single firing, and `on_change` has no second result to
|
||||
compare against. Hidden controls are nulled by `buildScopeFields` so state from an earlier
|
||||
trigger choice cannot leak into the payload, and `validateScopeSelection` rejects a scope
|
||||
mode that was selected but left unfilled (which previously serialised to null and silently
|
||||
ran the rule everywhere while the form still read "From a phase onwards").
|
||||
Startup is the deliberate exception: a scope there defers the one-shot rather than
|
||||
suppressing it, so the scope control stays while emit mode is hidden.
|
||||
* The custom-rule toggle PUT (`{enabled, isBuiltIn, isCommunity}`) previously full-replaced
|
||||
the stored rule and wiped Title/Target/Trigger — `GatherRuleService.UpdateRuleAsync` now
|
||||
merges toggle-style partials (empty Title/CollectorType/Target) into the existing row and
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
---
|
||||
type: Concept
|
||||
title: URL Registry — One Place per Component, Guard-Enforced
|
||||
description: Every well-known own or Microsoft host lives in exactly one registry file per component (Constants.cs, utils/config.ts, src/config.ts); guard tests fail the build on any literal outside them. Includes the agent-download host migration to the download alias.
|
||||
resource: /src/Shared/AutopilotMonitor.Shared/Constants.cs
|
||||
tags:
|
||||
- architecture
|
||||
- urls
|
||||
- ci
|
||||
- convention
|
||||
timestamp: 2026-07-23T00:00:00+02:00
|
||||
---
|
||||
|
||||
# URL Registry
|
||||
|
||||
Every well-known base URL — own hosts (portal, www, docs, download alias, MCP,
|
||||
API, both blob accounts) and Microsoft hosts (Graph, Entra login) — is defined
|
||||
in exactly one registry file per component. Code references the constant; the
|
||||
literal appears nowhere else.
|
||||
|
||||
| Component | Registry | Guard test |
|
||||
|---|---|---|
|
||||
| C# (Shared/Backend/Agent) | `AutopilotMonitor.Shared.Constants` (`*BaseUrl`) | `HardcodedUrlGuardTests` |
|
||||
| Web | `utils/config.ts` (`DOCS_URL`, `SITE_URL`, `PORTAL_URL`, `ENTRA_LOGIN_URL`, `API_URL_PROD`, `BLOB_URL_PROD`, `AGENT_DOWNLOAD_HOSTNAMES`) | `utils/__tests__/hardcodedUrls.guard.test.ts` |
|
||||
| MCP | `src/config.ts` (`API_BASE_URL`, `ENTRA_LOGIN_BASE_URL`, `DOCS_BASE_URL`) | `src/__tests__/hardcoded-urls.guard.test.ts` |
|
||||
|
||||
Derived registries build on these instead of repeating hosts:
|
||||
`lib/hostRouting.ts` computes `PUBLIC_HOST`/`PORTAL_HOST`/`APEX_HOST` from the
|
||||
URL constants, and `middleware.ts` imports from it.
|
||||
|
||||
# Schema
|
||||
|
||||
The guards scan non-test source for the enforced host list and fail with
|
||||
file:line on any occurrence outside the registry. Comment lines are ignored
|
||||
(docs may cite URLs). Test files are excluded on purpose: a literal in a test
|
||||
is an independent oracle — `ValidateBootstrapCodeResponseShapeTests` spells the
|
||||
full agent download URL out so a registry change must be acknowledged in two
|
||||
places, backend and portal allow-list, consciously.
|
||||
|
||||
Why enforced rather than agreed: the registry existed before (both
|
||||
`Constants.AgentBlobBaseUrl` and the MCP `config.ts` header document earlier
|
||||
drift), yet the EU cutover still missed two literal copies of the blob host —
|
||||
`LatestVersionsService` and `ValidateBootstrapCodeFunction` kept reading the
|
||||
legacy account, silently kept alive only by the fail-soft legacy mirror in
|
||||
`build-agent.yml`. A convention that is not checked decays; these guards make
|
||||
the next stray literal a red build instead of a production surprise.
|
||||
|
||||
# Agent-download host migration (2026-07-23)
|
||||
|
||||
The two legacy readers above plus the agent's `SelfUpdater` now use
|
||||
`Constants.AgentDownloadBaseUrl` (`download.autopilotmonitor.com/agent`, Front
|
||||
Door in front of the current blob origin — verified serving `version.json`,
|
||||
`version-v2.json` and both ZIPs before the switch). The portal's
|
||||
`bootstrapValidation.ts` allow-lists BOTH the alias and the legacy blob host
|
||||
(`AGENT_DOWNLOAD_HOSTNAMES`) because bootstrap scripts already deployed in
|
||||
customer Intune tenants still carry legacy URLs.
|
||||
|
||||
Deploy-order constraint: the web allow-list must be live BEFORE (or with) the
|
||||
backend that serves alias URLs — an old portal build rejects the alias host and
|
||||
the bootstrap flow dies client-side. Remove the legacy hostname from the
|
||||
allow-list only after the legacy account is torn down.
|
||||
|
||||
**Front Door caching is a correctness hazard here, not a staleness nuisance.**
|
||||
The first release after the migration exposed it: the alias kept serving the
|
||||
previous `version.json` (`X-Cache: TCP_REMOTE_HIT`) while the origin was already
|
||||
updated — and because manifest and ZIP are independent cache entries, a stale
|
||||
manifest paired with a fresh ZIP fails the bootstrap's SHA-256 check. Two
|
||||
defenses, both in place since 2026-07-23: route caching on `/agent/*` is
|
||||
DISABLED, and every blob upload (CI `Send-Blob`, local `build.ps1`, the
|
||||
`versions/*.json` manifests) stamps `Cache-Control: no-cache` — all these blobs
|
||||
rotate in place, even the per-line ZIP. If caching is ever re-enabled, the
|
||||
headers keep the mismatch class impossible.
|
||||
|
||||
Deliberately still on the legacy account: the `HealthCheckService` /
|
||||
`MaintenanceService` legacy-keepalive probes and the fail-soft mirror in
|
||||
`build-agent.yml` — they exist precisely to keep already-deployed customer
|
||||
bootstrap scripts working until the customer migration completes.
|
||||
|
||||
# Citations
|
||||
|
||||
- `src/Shared/AutopilotMonitor.Shared/Constants.cs` — C# registry
|
||||
- `src/Web/autopilot-monitor-web/utils/config.ts` — web registry
|
||||
- `src/McpServer/autopilot-monitor-mcp/src/config.ts` — MCP registry
|
||||
- `src/Backend/AutopilotMonitor.Functions.Tests/HardcodedUrlGuardTests.cs`
|
||||
- [Version Contract](versioning.md) — the release pipeline this migration protects
|
||||
@@ -0,0 +1,108 @@
|
||||
---
|
||||
type: Concept
|
||||
title: Version Contract — Agent, Backend, MCP
|
||||
description: One version shape for all three shipping components, the ETag-CAS counter blob that mints build numbers, and the published manifests that make "what is deployed" checkable from outside.
|
||||
resource: /.github/scripts/Request-BuildNumber.ps1
|
||||
tags:
|
||||
- build
|
||||
- ci
|
||||
- release
|
||||
- versioning
|
||||
timestamp: 2026-07-22T00:00:00+02:00
|
||||
---
|
||||
|
||||
# Version Contract
|
||||
|
||||
Three components ship from this repository — the agent, the Functions backend and the
|
||||
MCP server. They share one version shape, one way of minting build numbers, and one
|
||||
manifest format, so that a version string means the same thing whoever emits it.
|
||||
|
||||
```
|
||||
version = <major>.<minor>.<build> major.minor curated by hand, build minted per build
|
||||
commit = 7-char short SHA of this repo
|
||||
buildUtc = ISO-8601 build moment
|
||||
```
|
||||
|
||||
| Component | major.minor from | Counter blob | Reported by |
|
||||
|---|---|---|---|
|
||||
| Agent V2 | `VersionPrefix` in `AutopilotMonitor.Agent.V2.csproj` | `agent/buildcounter-v2.txt` | `agent/version.json` |
|
||||
| Backend | `VersionPrefix` in `AutopilotMonitor.Functions.csproj` | `versions/backend.txt` | `GET /api/health` |
|
||||
| MCP | `version` in `package.json` | `versions/mcp.txt` | `GET /health`, MCP handshake |
|
||||
|
||||
# Schema
|
||||
|
||||
## Build numbers — reserve before build, ETag-CAS
|
||||
|
||||
`.github/scripts/Request-BuildNumber.ps1` is the only implementation; the CI
|
||||
workflows and the local `build.ps1` all call it.
|
||||
|
||||
1. Anonymous `GET` on the counter blob → value `N` and `ETag`. The blob holds the
|
||||
**last used** number as plain text and is public-read; only the write needs the
|
||||
container SAS.
|
||||
2. `PUT N+1` with `If-Match: <etag>`. The quoted ETag form is required — PowerShell
|
||||
strips the quotes on read, and an unquoted `If-Match` is rejected client-side.
|
||||
3. `412 Precondition Failed` → a concurrent build reserved first → re-read and retry
|
||||
(5 attempts, linear backoff).
|
||||
4. Build with the reserved number.
|
||||
|
||||
Reserving *before* the build is what makes concurrent minting impossible. **A failed
|
||||
build burns a number** — deliberate: uniqueness matters, density does not.
|
||||
|
||||
Why a counter and not the published manifest: manifests are written on publish, not on
|
||||
build. The agent writes `version.json` only for stable releases, so deriving the next
|
||||
number from it would hand two consecutive dev builds the same number and let the second
|
||||
overwrite the first's artifact. A state-free scheme such as `git rev-list --count HEAD`
|
||||
fails for the same reason — two builds of one commit collide.
|
||||
|
||||
**Seeding** (once per component): `PUT <last used number>` with `If-None-Match: *`, so
|
||||
an existing counter can never be clobbered. Agent V2 was seeded with `1356` on
|
||||
2026-07-17; `versions/backend.txt` with `999` and `versions/mcp.txt` with `299` on
|
||||
2026-07-22.
|
||||
|
||||
## Local builds
|
||||
|
||||
No counter is reached, the build number stays `0`, and the commit is empty or
|
||||
`unknown`. A `1.5.0` with no commit hash on `/api/health` is therefore a developer
|
||||
machine, not a release — the fallback is the signal, not a defect. The agent is the
|
||||
exception: `build.ps1` produces uploadable artifacts and reserves a real number unless
|
||||
`-SkipUpload` is passed.
|
||||
|
||||
## Manifests
|
||||
|
||||
`versions/backend.json` and `versions/mcp.json` (public-read) record what is deployed:
|
||||
|
||||
```json
|
||||
{ "component": "backend", "version": "1.5.1000", "commit": "abc1234",
|
||||
"buildUtc": "…", "deployedUtc": "…", "runId": "…" }
|
||||
```
|
||||
|
||||
The MCP manifest adds `docsCommit`, because for that component "which build" and "which
|
||||
documentation" are separate questions.
|
||||
|
||||
Each is written **last and only after the deploy workflow has polled the component's
|
||||
health endpoint until it reports that exact version and commit**. A green deploy step
|
||||
alone means the package was accepted, not that it is being served. Consequently a
|
||||
manifest that disagrees with the live endpoint means the running instance drifted after
|
||||
the fact (rollback, stale revision) rather than a half-finished pipeline.
|
||||
|
||||
The agent keeps its own manifest at `agent/version.json` — the bootstrap derives that
|
||||
URL from the download URL and verifies the ZIP against its `sha256`, so it cannot move.
|
||||
It carries the same `commit` and `buildUtc` fields; `sha256` and `bootstrapVersion`
|
||||
remain, being the customer-facing integrity contract.
|
||||
|
||||
# Consequences
|
||||
|
||||
- Bumping `major.minor` is a human decision, made in the component's own file
|
||||
(`VersionPrefix`, or `version` in `package.json`). Nothing bumps it automatically.
|
||||
- A new component joins by seeding a counter blob and calling the same script; no new
|
||||
mechanism.
|
||||
- The counter blobs and manifests live in the `versions` container on
|
||||
`autopilotmonitoreu` (writes via `AZURE_BLOB_VERSIONS_SAS_TOKEN`). The agent's
|
||||
counter stays in the `agent` container next to the artifacts it numbers.
|
||||
|
||||
# Citations
|
||||
|
||||
- `.github/scripts/Request-BuildNumber.ps1` — the CAS protocol
|
||||
- `.github/workflows/deploy-backend.yml`, `deploy-mcp.yml` — reserve, verify, publish
|
||||
- `.github/workflows/build-agent.yml`, `scripts/Deployment/build.ps1` — agent line
|
||||
- [Build Counter Blob](agent/build-counter-blob.md) — agent-specific consequences
|
||||
@@ -0,0 +1,268 @@
|
||||
// Autopilot-Monitor MCP Server — Azure Container App Infrastructure
|
||||
//
|
||||
// Reconciled against the live deployment on 2026-07-21 (rg-autopilotmonitor-prd-gwc,
|
||||
// Germany West Central). The template had drifted behind two changes made directly
|
||||
// in Azure: the EU cutover (registry/API renamed) and the ACR hardening (admin user
|
||||
// disabled, image pull moved to the app's system-assigned identity).
|
||||
//
|
||||
// !! BEFORE RE-DEPLOYING THIS TEMPLATE, READ THIS !!
|
||||
// The live app carries an ingress custom-domain binding for mcp.autopilotmonitor.com
|
||||
// (SNI-enabled, with a managed certificate). Custom domains are NOT modelled below,
|
||||
// because the certificate is bound out-of-band. A full template deployment would
|
||||
// therefore REMOVE that binding and take the public MCP endpoint offline.
|
||||
// For routine changes (CPU/memory, env vars, image) prefer the surgical CLI form:
|
||||
// az containerapp update -n autopilotmonitor-mcp -g rg-autopilotmonitor-prd-gwc --cpu 0.5 --memory 1.0Gi
|
||||
// Treat this file as the documented desired state and as a rebuild-from-scratch
|
||||
// recipe — not as a routinely-applied deployment.
|
||||
//
|
||||
// On a FROM-SCRATCH deploy the AcrPull assignment below is created only after the
|
||||
// Container App exists (it needs the app's principalId), so the very first revision
|
||||
// can fail to pull its image. Re-run the deployment, or restart the revision, once
|
||||
// the role assignment has propagated. This does not affect updates to a live app.
|
||||
//
|
||||
// Deploy (two-stage; required to set the OAuth public URL safely):
|
||||
// 1. First deploy without mcpPublicUrl:
|
||||
// az deployment group create \
|
||||
// --resource-group <rg-name> \
|
||||
// --template-file infra/mcp-server.bicep \
|
||||
// --parameters apiUrl=https://autopilotmonitor-api.azurewebsites.net \
|
||||
// entraClientSecret=<secret-value>
|
||||
// This emits `containerAppUrl` as an output (e.g. https://autopilotmonitor-mcp.<env-suffix>.<region>.azurecontainerapps.io).
|
||||
//
|
||||
// 2. Re-deploy with that URL pinned, so the OAuth proxy never trusts caller-
|
||||
// supplied X-Forwarded-Host / Host headers when building issuer / WWW-
|
||||
// Authenticate / redirect_uri metadata:
|
||||
// --parameters mcpPublicUrl=<containerAppUrl from step 1>
|
||||
//
|
||||
// When mcpPublicUrl is set, the MCP server uses it verbatim and ignores any
|
||||
// forwarded-headers from the request. Without it, the server logs a startup
|
||||
// warning and falls back to header-derived URLs (acceptable for local dev,
|
||||
// not for production).
|
||||
|
||||
@description('Azure region for all resources')
|
||||
param location string = resourceGroup().location
|
||||
|
||||
@description('Name of the Container App')
|
||||
param containerAppName string = 'autopilotmonitor-mcp'
|
||||
|
||||
@description('Name of the Container App Environment')
|
||||
param environmentName string = 'autopilotmonitor-env'
|
||||
|
||||
@description('Name of the Log Analytics Workspace (dedicated for Container Apps)')
|
||||
param logAnalyticsName string = 'autopilotmonitor-container-logs'
|
||||
|
||||
@description('Name of the Azure Container Registry')
|
||||
param acrName string = 'autopilotmonitoreuacr'
|
||||
|
||||
@description('Backend API URL')
|
||||
param apiUrl string = 'https://autopilotmonitor-api-eu.azurewebsites.net'
|
||||
|
||||
@description('Container image tag')
|
||||
param imageTag string = 'latest'
|
||||
|
||||
@description('Entra ID app registration client_id used for the OAuth proxy. Public value (appears in JWT aud claim); not a secret.')
|
||||
param entraClientId string = '1a400946-62c1-4ab4-aa37-f730ac89704d'
|
||||
|
||||
@secure()
|
||||
@description('Entra ID client secret for OAuth token exchange')
|
||||
param entraClientSecret string = ''
|
||||
|
||||
@description('Comma-separated allowlist of redirect_uri hosts for the OAuth proxy. Loopback is always allowed. Empty = built-in default (Anthropic, OpenAI, Google).')
|
||||
param mcpAllowedRedirectHosts string = ''
|
||||
|
||||
@description('Public URL of the MCP server, used verbatim for OAuth issuer / WWW-Authenticate / redirect metadata. REQUIRED in production: the container refuses to boot without it (host-spoofing defense — see config.ts). Set on the second deploy with the value of the containerAppUrl output from the first deploy. The stage-1 deploy is safe because minReplicas=0 means no container runs until this is pinned. Empty falls back to X-Forwarded-Host headers only outside production (NODE_ENV != production).')
|
||||
param mcpPublicUrl string = ''
|
||||
|
||||
@secure()
|
||||
// NOTE: avoid characters outside cp1252 here (this one used to say a Unicode
|
||||
// >=). `az bicep build --stdout` writes through the Windows console codec and
|
||||
// dies with a UnicodeEncodeError on anything it cannot map. Em dashes are fine.
|
||||
@description('Base64-encoded HMAC key (>=32 bytes after decode) for signing the OAuth proxy state. Naming and format match the backend\'s PaginationTokenSigningKey for consistency. Empty = generate per-instance random key on startup (acceptable for single-replica deployments since state has 10 min lifetime). Generate with PowerShell: [Convert]::ToBase64String([Security.Cryptography.RandomNumberGenerator]::GetBytes(32))')
|
||||
param oAuthStateSigningKey string = ''
|
||||
|
||||
@description('Enable structured tool-call logging to stderr (queryable via Container App Logs)')
|
||||
param toolLoggingEnabled bool = false
|
||||
|
||||
// --- Azure Container Registry ---
|
||||
|
||||
// Admin user is DISABLED: a registry-wide username/password pair is a static,
|
||||
// non-rotating credential that would also have to live in a Container App secret.
|
||||
// The app pulls with its system-assigned identity + AcrPull instead (see below).
|
||||
resource acr 'Microsoft.ContainerRegistry/registries@2023-07-01' = {
|
||||
name: acrName
|
||||
location: location
|
||||
sku: {
|
||||
name: 'Basic'
|
||||
}
|
||||
properties: {
|
||||
adminUserEnabled: false
|
||||
}
|
||||
}
|
||||
|
||||
// AcrPull for the Container App's system-assigned identity — this is what replaces
|
||||
// the admin credentials. Without it the app cannot pull its own image.
|
||||
//
|
||||
// NOTE: this assignment exists on the live registry but could not be enumerated
|
||||
// during reconciliation (the CLI session lacks Microsoft.Authorization read), so it
|
||||
// is reproduced here from the documented pattern rather than read back from Azure.
|
||||
// Verify with:
|
||||
// az role assignment list --scope <acr-id> --assignee <containerApp principalId>
|
||||
var acrPullRoleId = '7f951dda-4ed3-4680-a7ca-43fe172d538d'
|
||||
|
||||
resource acrPull 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
|
||||
scope: acr
|
||||
name: guid(acr.id, containerAppName, acrPullRoleId)
|
||||
properties: {
|
||||
roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', acrPullRoleId)
|
||||
principalId: containerApp.identity.principalId
|
||||
principalType: 'ServicePrincipal'
|
||||
}
|
||||
}
|
||||
|
||||
// --- Log Analytics Workspace (required by Container App Environment) ---
|
||||
|
||||
resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2023-09-01' = {
|
||||
name: logAnalyticsName
|
||||
location: location
|
||||
properties: {
|
||||
sku: {
|
||||
name: 'PerGB2018'
|
||||
}
|
||||
retentionInDays: 30
|
||||
}
|
||||
}
|
||||
|
||||
// --- Container App Environment ---
|
||||
|
||||
resource environment 'Microsoft.App/managedEnvironments@2024-03-01' = {
|
||||
name: environmentName
|
||||
location: location
|
||||
properties: {
|
||||
appLogsConfiguration: {
|
||||
destination: 'log-analytics'
|
||||
logAnalyticsConfiguration: {
|
||||
customerId: logAnalytics.properties.customerId
|
||||
sharedKey: logAnalytics.listKeys().primarySharedKey
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- Container App ---
|
||||
|
||||
resource containerApp 'Microsoft.App/containerApps@2024-03-01' = {
|
||||
name: containerAppName
|
||||
location: location
|
||||
// System-assigned identity: the credential the app uses to pull from the ACR
|
||||
// (see the AcrPull assignment above). Replaces the former admin-user secret.
|
||||
identity: {
|
||||
type: 'SystemAssigned'
|
||||
}
|
||||
properties: {
|
||||
managedEnvironmentId: environment.id
|
||||
configuration: {
|
||||
ingress: {
|
||||
external: true
|
||||
targetPort: 8080
|
||||
transport: 'auto'
|
||||
allowInsecure: false
|
||||
}
|
||||
// 'system' = pull with the app's system-assigned identity. No username, no
|
||||
// password secret — nothing to rotate and nothing to leak from the app config.
|
||||
registries: [
|
||||
{
|
||||
server: acr.properties.loginServer
|
||||
identity: 'system'
|
||||
}
|
||||
]
|
||||
secrets: [
|
||||
{
|
||||
name: 'entra-client-secret'
|
||||
value: entraClientSecret
|
||||
}
|
||||
{
|
||||
name: 'oauth-state-signing-key'
|
||||
value: oAuthStateSigningKey
|
||||
}
|
||||
]
|
||||
activeRevisionsMode: 'Single'
|
||||
}
|
||||
template: {
|
||||
containers: [
|
||||
{
|
||||
name: 'mcp-server'
|
||||
image: '${acr.properties.loginServer}/${containerAppName}:${imageTag}'
|
||||
// 0.5 vCPU, but NOT for the reason originally given. It was raised from
|
||||
// 0.25 to shorten the scale-to-zero cold start, on the assumption that
|
||||
// loading the ONNX embedding model dominated it. Measurement disproved
|
||||
// that: the app boots in 504 ms in production (see the [boot] marks in
|
||||
// the container log and docs/mcp/docs-corpus.md), of which the prebaked
|
||||
// embedder is 173 ms in the background. Of a 31 s cold start, ~98% is
|
||||
// Container Apps activation — pod scheduling, image mount, container
|
||||
// start — which no CPU setting touches.
|
||||
//
|
||||
// It stays at 0.5 because it plausibly helps CONCURRENT requests
|
||||
// (maxReplicas is 1, so one replica serves up to 10 in-flight calls) and
|
||||
// because at scale-to-zero the container bills only while active, making
|
||||
// the difference negligible. If cold start ever needs to actually go
|
||||
// away, the only lever is minReplicas: 1. Container Apps accepts fixed
|
||||
// CPU/memory pairs, so memory moves with the CPU.
|
||||
resources: {
|
||||
cpu: json('0.5')
|
||||
memory: '1.0Gi'
|
||||
}
|
||||
env: [
|
||||
{ name: 'AUTOPILOT_API_URL', value: apiUrl }
|
||||
{ name: 'PORT', value: '8080' }
|
||||
{ name: 'AUTOPILOT_ENTRA_CLIENT_ID', value: entraClientId }
|
||||
{ name: 'AUTOPILOT_ENTRA_CLIENT_SECRET', secretRef: 'entra-client-secret' }
|
||||
{ name: 'MCP_ALLOWED_REDIRECT_HOSTS', value: mcpAllowedRedirectHosts }
|
||||
{ name: 'MCP_PUBLIC_URL', value: mcpPublicUrl }
|
||||
{ name: 'OAuthStateSigningKey', secretRef: 'oauth-state-signing-key' }
|
||||
{ name: 'MCP_TOOL_LOGGING', value: toolLoggingEnabled ? 'true' : 'false' }
|
||||
]
|
||||
probes: [
|
||||
{
|
||||
type: 'Liveness'
|
||||
httpGet: {
|
||||
path: '/health'
|
||||
port: 8080
|
||||
}
|
||||
periodSeconds: 30
|
||||
initialDelaySeconds: 10
|
||||
}
|
||||
{
|
||||
type: 'Readiness'
|
||||
httpGet: {
|
||||
path: '/health'
|
||||
port: 8080
|
||||
}
|
||||
periodSeconds: 10
|
||||
initialDelaySeconds: 5
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
scale: {
|
||||
minReplicas: 0
|
||||
maxReplicas: 1
|
||||
rules: [
|
||||
{
|
||||
name: 'http-scaler'
|
||||
http: {
|
||||
metadata: {
|
||||
concurrentRequests: '10'
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- Outputs ---
|
||||
|
||||
output containerAppUrl string = 'https://${containerApp.properties.configuration.ingress.fqdn}'
|
||||
output mcpEndpoint string = 'https://${containerApp.properties.configuration.ingress.fqdn}/mcp'
|
||||
output acrLoginServer string = acr.properties.loginServer
|
||||
@@ -2,10 +2,10 @@
|
||||
"$schema": "../schema/analyze-rule.schema.json",
|
||||
"ruleId": "ANALYZE-ESP-005",
|
||||
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
|
||||
"description": "Detects the 'unexpected reboot + second sign-in screen' pattern: one or more device-assigned MDM policies matched the OS reboot-required catalog during ESP DeviceSetup, and Windows coalesced them into a restart at the end of DeviceSetup — forcing the user through a second sign-in before AccountSetup. Attributes the reboot to the specific CSP policy URIs so the profiles can be reassigned.",
|
||||
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
|
||||
"severity": "warning",
|
||||
"category": "esp",
|
||||
"version": "1.0.0",
|
||||
"version": "1.1.0",
|
||||
"author": "Autopilot Monitor",
|
||||
"enabled": true,
|
||||
"trigger": "single",
|
||||
@@ -15,37 +15,48 @@
|
||||
"signal": "reboot_forcing_policy_uri",
|
||||
"source": "event_data",
|
||||
"eventType": "mdm_policy_reboot_required",
|
||||
"dataField": "rebootUri",
|
||||
"dataField": "firstRebootUri",
|
||||
"operator": "exists",
|
||||
"required": true
|
||||
},
|
||||
{
|
||||
"signal": "reboot_uri_count",
|
||||
"signal": "reboot_observed",
|
||||
"source": "event_count",
|
||||
"eventType": "mdm_policy_reboot_required",
|
||||
"eventType": "system_reboot_detected",
|
||||
"operator": "count_gte",
|
||||
"value": "1",
|
||||
"required": true
|
||||
},
|
||||
{
|
||||
"signal": "multiple_reboot_uris",
|
||||
"source": "event_data",
|
||||
"eventType": "mdm_policy_reboot_required",
|
||||
"dataField": "uriCount",
|
||||
"operator": "gte",
|
||||
"value": "2",
|
||||
"required": false
|
||||
},
|
||||
{
|
||||
"signal": "esp_coalesced_reboot_observed",
|
||||
"source": "event_type",
|
||||
"eventType": "esp_reboot_coalescing",
|
||||
"operator": "exists",
|
||||
"signal": "omadm_reboot_flag_set",
|
||||
"source": "event_data",
|
||||
"eventType": "mdm_policy_reboot_required",
|
||||
"dataField": "omadmRebootRequiredFlag",
|
||||
"operator": "equals",
|
||||
"value": "true",
|
||||
"required": false
|
||||
}
|
||||
],
|
||||
"confidenceFactors": [
|
||||
{ "signal": "esp_coalesced_reboot_observed", "condition": "exists", "weight": 20 },
|
||||
{ "signal": "mdm_policy_reboot_required", "condition": "count >= 2", "weight": 10 }
|
||||
{ "signal": "multiple_reboot_uris", "condition": "exists", "weight": 10 },
|
||||
{ "signal": "omadm_reboot_flag_set", "condition": "exists", "weight": 15 }
|
||||
],
|
||||
"confidenceThreshold": 70,
|
||||
"explanation": "During ESP DeviceSetup, a device-assigned MDM policy matched the OS reboot-required catalog and Windows scheduled a **coalesced reboot** at the end of DeviceSetup — the user saw an unexpected restart and a **second sign-in screen** before AccountSetup continued.\n\nFirst reboot-forcing policy URI: `{{rebootUri}}`\n\nWhen several policies force a reboot they are coalesced into one restart; **each** reboot-forcing policy appears as its own `mdm_policy_reboot_required` event on the session timeline — check the timeline for the complete URI list.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
|
||||
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
|
||||
"remediation": [
|
||||
{
|
||||
"title": "Reassign the reboot-forcing policies to user groups",
|
||||
"steps": [
|
||||
"Open each `mdm_policy_reboot_required` timeline event and note the CSP URI (e.g. `./Device/Vendor/MSFT/...`)",
|
||||
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
|
||||
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
|
||||
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
|
||||
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
|
||||
|
||||
+27
-16
@@ -1788,10 +1788,10 @@
|
||||
{
|
||||
"ruleId": "ANALYZE-ESP-005",
|
||||
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
|
||||
"description": "Detects the 'unexpected reboot + second sign-in screen' pattern: one or more device-assigned MDM policies matched the OS reboot-required catalog during ESP DeviceSetup, and Windows coalesced them into a restart at the end of DeviceSetup — forcing the user through a second sign-in before AccountSetup. Attributes the reboot to the specific CSP policy URIs so the profiles can be reassigned.",
|
||||
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
|
||||
"severity": "warning",
|
||||
"category": "esp",
|
||||
"version": "1.0.0",
|
||||
"version": "1.1.0",
|
||||
"author": "Autopilot Monitor",
|
||||
"enabled": true,
|
||||
"trigger": "single",
|
||||
@@ -1801,45 +1801,56 @@
|
||||
"signal": "reboot_forcing_policy_uri",
|
||||
"source": "event_data",
|
||||
"eventType": "mdm_policy_reboot_required",
|
||||
"dataField": "rebootUri",
|
||||
"dataField": "firstRebootUri",
|
||||
"operator": "exists",
|
||||
"required": true
|
||||
},
|
||||
{
|
||||
"signal": "reboot_uri_count",
|
||||
"signal": "reboot_observed",
|
||||
"source": "event_count",
|
||||
"eventType": "mdm_policy_reboot_required",
|
||||
"eventType": "system_reboot_detected",
|
||||
"operator": "count_gte",
|
||||
"value": "1",
|
||||
"required": true
|
||||
},
|
||||
{
|
||||
"signal": "multiple_reboot_uris",
|
||||
"source": "event_data",
|
||||
"eventType": "mdm_policy_reboot_required",
|
||||
"dataField": "uriCount",
|
||||
"operator": "gte",
|
||||
"value": "2",
|
||||
"required": false
|
||||
},
|
||||
{
|
||||
"signal": "esp_coalesced_reboot_observed",
|
||||
"source": "event_type",
|
||||
"eventType": "esp_reboot_coalescing",
|
||||
"operator": "exists",
|
||||
"signal": "omadm_reboot_flag_set",
|
||||
"source": "event_data",
|
||||
"eventType": "mdm_policy_reboot_required",
|
||||
"dataField": "omadmRebootRequiredFlag",
|
||||
"operator": "equals",
|
||||
"value": "true",
|
||||
"required": false
|
||||
}
|
||||
],
|
||||
"confidenceFactors": [
|
||||
{
|
||||
"signal": "esp_coalesced_reboot_observed",
|
||||
"signal": "multiple_reboot_uris",
|
||||
"condition": "exists",
|
||||
"weight": 20
|
||||
"weight": 10
|
||||
},
|
||||
{
|
||||
"signal": "mdm_policy_reboot_required",
|
||||
"condition": "count >= 2",
|
||||
"weight": 10
|
||||
"signal": "omadm_reboot_flag_set",
|
||||
"condition": "exists",
|
||||
"weight": 15
|
||||
}
|
||||
],
|
||||
"confidenceThreshold": 70,
|
||||
"explanation": "During ESP DeviceSetup, a device-assigned MDM policy matched the OS reboot-required catalog and Windows scheduled a **coalesced reboot** at the end of DeviceSetup — the user saw an unexpected restart and a **second sign-in screen** before AccountSetup continued.\n\nFirst reboot-forcing policy URI: `{{rebootUri}}`\n\nWhen several policies force a reboot they are coalesced into one restart; **each** reboot-forcing policy appears as its own `mdm_policy_reboot_required` event on the session timeline — check the timeline for the complete URI list.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
|
||||
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
|
||||
"remediation": [
|
||||
{
|
||||
"title": "Reassign the reboot-forcing policies to user groups",
|
||||
"steps": [
|
||||
"Open each `mdm_policy_reboot_required` timeline event and note the CSP URI (e.g. `./Device/Vendor/MSFT/...`)",
|
||||
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
|
||||
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
|
||||
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
|
||||
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
|
||||
|
||||
@@ -109,6 +109,79 @@
|
||||
"SELECT * FROM Win32_LogicalDisk",
|
||||
"SELECT * FROM SoftwareLicensingProduct"
|
||||
],
|
||||
"eventLogChannels": [
|
||||
{
|
||||
"category": "Core Windows logs",
|
||||
"channels": [
|
||||
"Application",
|
||||
"System",
|
||||
"Setup",
|
||||
"Microsoft-Windows-Kernel-Boot",
|
||||
"Microsoft-Windows-Diagnostics-Performance"
|
||||
]
|
||||
},
|
||||
{
|
||||
"category": "MDM / Enrollment",
|
||||
"channels": [
|
||||
"Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider",
|
||||
"Microsoft-Windows-ModernDeployment-Diagnostics-Provider",
|
||||
"Microsoft-Windows-Provisioning-Diagnostics-Provider",
|
||||
"Microsoft-Windows-AAD",
|
||||
"Microsoft-Windows-User Device Registration"
|
||||
]
|
||||
},
|
||||
{
|
||||
"category": "ESP / Shell / Apps",
|
||||
"channels": [
|
||||
"Microsoft-Windows-Shell-Core",
|
||||
"Microsoft-Windows-AppXDeployment",
|
||||
"Microsoft-Windows-AppXDeploymentServer",
|
||||
"Microsoft-Windows-AppReadiness",
|
||||
"Microsoft-Windows-Store"
|
||||
]
|
||||
},
|
||||
{
|
||||
"category": "Security posture / Crypto",
|
||||
"channels": [
|
||||
"Microsoft-Windows-BitLocker-API",
|
||||
"Microsoft-Windows-BitLocker-DrivePreparationTool",
|
||||
"Microsoft-Windows-TPM-WMI",
|
||||
"Microsoft-Windows-CertificateServicesClient-Lifecycle-System"
|
||||
]
|
||||
},
|
||||
{
|
||||
"category": "Update / Servicing",
|
||||
"channels": [
|
||||
"Microsoft-Windows-WindowsUpdateClient"
|
||||
]
|
||||
},
|
||||
{
|
||||
"category": "Logon / Session",
|
||||
"channels": [
|
||||
"Microsoft-Windows-Winlogon",
|
||||
"Microsoft-Windows-User Profile Service",
|
||||
"Microsoft-Windows-GroupPolicy",
|
||||
"Microsoft-Windows-TaskScheduler"
|
||||
]
|
||||
},
|
||||
{
|
||||
"category": "Network",
|
||||
"channels": [
|
||||
"Microsoft-Windows-NetworkProfile",
|
||||
"Microsoft-Windows-Dhcp-Client",
|
||||
"Microsoft-Windows-DNS-Client",
|
||||
"Microsoft-Windows-NCSI",
|
||||
"Microsoft-Windows-WLAN-AutoConfig",
|
||||
"Microsoft-Windows-Time-Service"
|
||||
]
|
||||
}
|
||||
],
|
||||
"blockedEventLogChannels": [
|
||||
"Security",
|
||||
"Microsoft-Windows-PowerShell",
|
||||
"Windows PowerShell",
|
||||
"Microsoft-Windows-Sysmon"
|
||||
],
|
||||
"allowedCommands": [
|
||||
{
|
||||
"category": "TPM and Security",
|
||||
|
||||
@@ -80,8 +80,8 @@
|
||||
},
|
||||
"trigger": {
|
||||
"type": "string",
|
||||
"description": "When to collect data.",
|
||||
"enum": ["startup", "phase_change", "interval", "on_event"]
|
||||
"description": "When to collect data. \"phase_change\" fires once when triggerPhase is entered, \"phase_exit\" once when it is left.",
|
||||
"enum": ["startup", "phase_change", "phase_exit", "interval", "on_event"]
|
||||
},
|
||||
"intervalSeconds": {
|
||||
"type": "integer",
|
||||
@@ -90,7 +90,7 @@
|
||||
},
|
||||
"triggerPhase": {
|
||||
"type": "string",
|
||||
"description": "Phase to trigger on (only used when trigger = \"phase_change\")."
|
||||
"description": "Phase to trigger on (used when trigger = \"phase_change\" or \"phase_exit\"). Empty = every phase transition."
|
||||
},
|
||||
"triggerEventType": {
|
||||
"type": "string",
|
||||
|
||||
@@ -91,6 +91,7 @@ if (fs.existsSync(guardrailsPath)) {
|
||||
// Flatten categorized lists for validation
|
||||
const flatRegistryPrefixes = guardrails.registryPrefixes.flatMap((g) => g.prefixes);
|
||||
const flatCommands = guardrails.allowedCommands.flatMap((g) => g.commands);
|
||||
const flatEventLogChannels = guardrails.eventLogChannels.flatMap((g) => g.channels);
|
||||
|
||||
const ts = `/**
|
||||
* AUTO-GENERATED from rules/guardrails.json — DO NOT EDIT.
|
||||
@@ -114,6 +115,10 @@ export const COMMAND_CATEGORIES: readonly GuardrailCategory[] = [
|
||||
${categorizedArray(guardrails.allowedCommands, 'commands')}
|
||||
];
|
||||
|
||||
export const EVENT_LOG_CHANNEL_CATEGORIES: readonly GuardrailCategory[] = [
|
||||
${categorizedArray(guardrails.eventLogChannels, 'channels')}
|
||||
];
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Flat arrays (for validation logic)
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -141,10 +146,18 @@ ${flatArray(guardrails.diagnosticsPathPrefixes)}
|
||||
export const BLOCKED_FILE_PREFIXES: readonly string[] = [
|
||||
${flatArray(guardrails.blockedFilePrefixes)}
|
||||
];
|
||||
|
||||
export const ALLOWED_EVENT_LOG_CHANNELS: readonly string[] = [
|
||||
${flatArray(flatEventLogChannels)}
|
||||
];
|
||||
|
||||
export const BLOCKED_EVENT_LOG_CHANNELS: readonly string[] = [
|
||||
${flatArray(guardrails.blockedEventLogChannels)}
|
||||
];
|
||||
`;
|
||||
|
||||
fs.writeFileSync(guardrailsOutput, ts, 'utf8');
|
||||
console.log(`guardrails.generated.ts: ${flatRegistryPrefixes.length} registry, ${flatCommands.length} commands, ${guardrails.filePrefixes.length} file prefixes`);
|
||||
console.log(`guardrails.generated.ts: ${flatRegistryPrefixes.length} registry, ${flatCommands.length} commands, ${guardrails.filePrefixes.length} file prefixes, ${flatEventLogChannels.length} event log channels`);
|
||||
} else {
|
||||
console.warn('guardrails.json not found — skipping guardrails generation');
|
||||
}
|
||||
|
||||
@@ -56,6 +56,10 @@
|
||||
PowerShell 5.1 (IME) reads scripts without BOM as ANSI, corrupting multi-byte chars.
|
||||
|
||||
.CHANGELOG
|
||||
2026-07-20 v2.2 Switched default download endpoint from the legacy blob
|
||||
(autopilotmonitor.blob.core.windows.net) to the Front Door alias
|
||||
https://download.autopilotmonitor.com/agent/ (route /agent/*).
|
||||
version.json is still derived from the same container.
|
||||
2026-06-25 v2.1 OOBE-relax: detect genuine OOBE via WinRT
|
||||
SystemSetupInfo.OutOfBoxExperienceState and exempt guards 2+3 when a
|
||||
single freshly-created profile appears in OOBE (Windows Backup for
|
||||
@@ -80,7 +84,7 @@
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory = $false)]
|
||||
[string]$AgentDownloadUrl = "https://autopilotmonitor.blob.core.windows.net/agent/AutopilotMonitor-Agent.zip",
|
||||
[string]$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip",
|
||||
|
||||
[Parameter(Mandatory = $false)]
|
||||
[string]$VersionJsonName = "version.json",
|
||||
@@ -93,7 +97,7 @@ param(
|
||||
)
|
||||
|
||||
# Script version (bump on meaningful changes; see .CHANGELOG above)
|
||||
$ScriptVersion = "2.1"
|
||||
$ScriptVersion = "2.2"
|
||||
|
||||
# Configuration - Everything in ProgramData for easy cleanup
|
||||
$AgentBasePath = "$env:ProgramData\AutopilotMonitor"
|
||||
@@ -268,13 +272,16 @@ try {
|
||||
$downloadAttempt++
|
||||
try {
|
||||
Write-Log "Download attempt ${downloadAttempt}/${maxDownloadAttempts}"
|
||||
$downloadStopwatch = [System.Diagnostics.Stopwatch]::StartNew()
|
||||
Invoke-WebRequest `
|
||||
-Uri $AgentDownloadUrl `
|
||||
-OutFile $zipPath `
|
||||
-UseBasicParsing `
|
||||
-TimeoutSec 30 `
|
||||
-ErrorAction Stop
|
||||
Write-Log "Downloaded agent to $zipPath"
|
||||
$downloadStopwatch.Stop()
|
||||
$downloadSeconds = [math]::Round($downloadStopwatch.Elapsed.TotalSeconds, 1)
|
||||
Write-Log "Downloaded agent to $zipPath (took ${downloadSeconds}s)"
|
||||
break
|
||||
}
|
||||
catch {
|
||||
|
||||
+1
-1
@@ -15,7 +15,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
|
||||
private const string CurrentUrl = "https://autopilotmonitor-api-eu.azurewebsites.net";
|
||||
private const string TargetUrl = "https://autopilotmonitor-api-us.azurewebsites.net";
|
||||
|
||||
private static AgentConfigResponse Config(string migrateUrl) =>
|
||||
private static AgentConfigResponse Config(string? migrateUrl) =>
|
||||
new AgentConfigResponse { MigrateToApiBaseUrl = migrateUrl };
|
||||
|
||||
[Fact]
|
||||
|
||||
+1
@@ -70,6 +70,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
|
||||
// Read the file the production code actually wrote and deserialize with the same lib.
|
||||
var raw = File.ReadAllText(_cachePath);
|
||||
var persisted = JsonConvert.DeserializeObject<AgentConfigResponse>(raw);
|
||||
Assert.NotNull(persisted);
|
||||
|
||||
Assert.False(persisted.UnrestrictedMode);
|
||||
Assert.False(persisted.AllowAgentDowngrade);
|
||||
|
||||
+3
-3
@@ -193,7 +193,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
|
||||
case BackendBehaviour.AlwaysAuthFailure401:
|
||||
throw new BackendAuthException("Unauthorized", 401);
|
||||
case BackendBehaviour.AlwaysReturnNull:
|
||||
return Task.FromResult<AgentConfigResponse>(null);
|
||||
return Task.FromResult<AgentConfigResponse>(null!);
|
||||
default:
|
||||
throw new InvalidOperationException("unknown behaviour");
|
||||
}
|
||||
@@ -204,9 +204,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
|
||||
|
||||
// In-memory + per-instance cache so a prior test's successful fetch can't
|
||||
// bleed into a later test that expects UsedDefaults.
|
||||
private AgentConfigResponse _cached;
|
||||
private AgentConfigResponse? _cached;
|
||||
protected override void CacheConfig(AgentConfigResponse config) { _cached = config; }
|
||||
protected override AgentConfigResponse LoadCachedConfig() => _cached;
|
||||
protected override AgentConfigResponse LoadCachedConfig() => _cached!;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+1
-1
@@ -51,7 +51,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Integration
|
||||
logger: logger,
|
||||
helloTracker: null);
|
||||
|
||||
EspExitedEventArgs captured = null;
|
||||
EspExitedEventArgs? captured = null;
|
||||
tracker.EspExited += (_, args) => captured = args;
|
||||
|
||||
// Source time IS in the past relative to the clock — verify the tracker carries the
|
||||
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
#nullable enable
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Analyzers;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Analyzers
|
||||
{
|
||||
/// <summary>
|
||||
/// LocalAdminAnalyzer.MatchesAllowedEntry — the allowed-list matching contract.
|
||||
/// Entries without wildcard characters match exactly (case-insensitive); entries
|
||||
/// containing * or ? are glob patterns with the same semantics as the hardware
|
||||
/// whitelist. The same matcher is applied to local accounts and profile folders.
|
||||
/// </summary>
|
||||
public sealed class LocalAdminAnalyzerTests
|
||||
{
|
||||
// -------------------------------------------------------------- literal entries
|
||||
|
||||
[Theory]
|
||||
[InlineData("SupportAdmin", "SupportAdmin")]
|
||||
[InlineData("supportadmin", "SupportAdmin")]
|
||||
[InlineData("SUPPORTADMIN", "supportadmin")]
|
||||
public void Literal_entry_matches_exactly_case_insensitive(string name, string entry)
|
||||
{
|
||||
Assert.True(LocalAdminAnalyzer.MatchesAllowedEntry(name, entry));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("SupportAdmin2", "SupportAdmin")]
|
||||
[InlineData("SupportAdmin", "SupportAdmin2")]
|
||||
[InlineData("Support", "SupportAdmin")]
|
||||
public void Literal_entry_does_not_substring_match(string name, string entry)
|
||||
{
|
||||
Assert.False(LocalAdminAnalyzer.MatchesAllowedEntry(name, entry));
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------- glob patterns
|
||||
|
||||
[Theory]
|
||||
[InlineData("adm-12345", "adm-*")]
|
||||
[InlineData("adm-", "adm-*")]
|
||||
[InlineData("ADM-9F3B2C", "adm-*")]
|
||||
[InlineData("lapsAdmin01", "lapsAdmin??")]
|
||||
[InlineData("svc-backup-01", "svc-*-01")]
|
||||
public void Wildcard_entry_matches_glob(string name, string entry)
|
||||
{
|
||||
Assert.True(LocalAdminAnalyzer.MatchesAllowedEntry(name, entry));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("administrator2", "adm-*")] // no literal "adm-" prefix
|
||||
[InlineData("xadm-123", "adm-*")] // pattern is anchored at start
|
||||
[InlineData("lapsAdmin1", "lapsAdmin??")] // ? is exactly one char
|
||||
[InlineData("adm.123", "adm-*")] // "-" is literal, not a regex class
|
||||
public void Wildcard_entry_does_not_overmatch(string name, string entry)
|
||||
{
|
||||
Assert.False(LocalAdminAnalyzer.MatchesAllowedEntry(name, entry));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Regex_metacharacters_in_entry_are_treated_literally()
|
||||
{
|
||||
// "adm.123" as an entry has no wildcard chars → exact match only,
|
||||
// the dot must not act as a regex any-char.
|
||||
Assert.True(LocalAdminAnalyzer.MatchesAllowedEntry("adm.123", "adm.123"));
|
||||
Assert.False(LocalAdminAnalyzer.MatchesAllowedEntry("admX123", "adm.123"));
|
||||
|
||||
// Mixed: wildcard present, remaining metacharacters stay literal.
|
||||
Assert.True(LocalAdminAnalyzer.MatchesAllowedEntry("adm.123", "adm.*"));
|
||||
Assert.False(LocalAdminAnalyzer.MatchesAllowedEntry("admX123", "adm.*"));
|
||||
}
|
||||
}
|
||||
}
|
||||
+139
@@ -405,5 +405,144 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
|
||||
var started = trace.Events.Find(e => e.EventType == SharedEventTypes.DesktopDetectorStarted);
|
||||
Assert.Equal(0, started.Data["noCandidateThresholdMinutes"]);
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// OOBE-state flip (oobe_state_completed) — observational one-shot on the poll
|
||||
// ============================================================================
|
||||
// The WinRT OutOfBoxExperienceState flip in_progress->completed is an owner-
|
||||
// independent desktop corroboration for DAD-starved sessions (4d5a0b78 class).
|
||||
// Tests drive polls synchronously via PollOnceForTest after Stop() so the real
|
||||
// timer cannot race, and pin the owner seams to an excluded user so a real
|
||||
// explorer.exe on the test machine can never set _desktopArrived (which would
|
||||
// short-circuit PollForDesktop before the flip check).
|
||||
|
||||
private static DesktopArrivalDetector BuildFlipDetector(TempDirectory tmp, TraceCapture trace, System.Func<string> oobeProvider)
|
||||
{
|
||||
var detector = new DesktopArrivalDetector(new AgentLogger(tmp.Path, AgentLogLevel.Info), noCandidateTimeoutMinutes: 0)
|
||||
{
|
||||
OnTraceEvent = trace.Sink,
|
||||
OobeStateProvider = oobeProvider,
|
||||
SessionOwnerResolver = _ => "SYSTEM",
|
||||
ProcessOwnerResolver = _ => "SYSTEM",
|
||||
};
|
||||
return detector;
|
||||
}
|
||||
|
||||
/// <summary>Returns the queued states in order, then repeats the last one forever.</summary>
|
||||
private static System.Func<string> StateQueue(params string[] states)
|
||||
{
|
||||
var queue = new Queue<string>(states);
|
||||
var last = states[states.Length - 1];
|
||||
return () => queue.Count > 0 ? queue.Dequeue() : last;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OobeFlip_emits_exactly_once_with_payload()
|
||||
{
|
||||
using var tmp = new TempDirectory();
|
||||
var trace = new TraceCapture();
|
||||
// Baseline sample happens synchronously inside Start()/ArmTimer.
|
||||
using var detector = BuildFlipDetector(tmp, trace, StateQueue("in_progress", "completed"));
|
||||
|
||||
detector.Start();
|
||||
detector.Stop();
|
||||
detector.PollOnceForTest(); // reads "completed" -> flip
|
||||
detector.PollOnceForTest(); // still "completed" -> no second emit
|
||||
|
||||
Assert.Equal(1, trace.CountOf(SharedEventTypes.OobeStateCompleted));
|
||||
var flip = trace.Events.Find(e => e.EventType == SharedEventTypes.OobeStateCompleted);
|
||||
Assert.Equal("in_progress", flip.Data["previousState"]);
|
||||
Assert.True(flip.Data.ContainsKey("pollCount"));
|
||||
Assert.True(flip.Data.ContainsKey("minutesSinceStart"));
|
||||
Assert.False((bool)flip.Data["desktopArrived"]);
|
||||
Assert.True(flip.Data.ContainsKey("wmiErrorCount"));
|
||||
Assert.True(flip.Data.ContainsKey("wtsErrorCount"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OobeFlip_does_not_emit_when_state_is_already_completed()
|
||||
{
|
||||
using var tmp = new TempDirectory();
|
||||
var trace = new TraceCapture();
|
||||
using var detector = BuildFlipDetector(tmp, trace, StateQueue("completed"));
|
||||
|
||||
detector.Start();
|
||||
detector.Stop();
|
||||
detector.PollOnceForTest();
|
||||
detector.PollOnceForTest();
|
||||
|
||||
// Productive machines read "completed" from the very first sample — the flip
|
||||
// event must never fire there (the agent_started field already records the state).
|
||||
Assert.Equal(0, trace.CountOf(SharedEventTypes.OobeStateCompleted));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OobeFlip_does_not_emit_when_in_progress_was_never_seen()
|
||||
{
|
||||
using var tmp = new TempDirectory();
|
||||
var trace = new TraceCapture();
|
||||
using var detector = BuildFlipDetector(tmp, trace, StateQueue("unavailable", "completed"));
|
||||
|
||||
detector.Start();
|
||||
detector.Stop();
|
||||
detector.PollOnceForTest();
|
||||
|
||||
// Old builds without the WinRT contract report "unavailable"; a later
|
||||
// "completed" without a positively observed "in_progress" is not a flip.
|
||||
Assert.Equal(0, trace.CountOf(SharedEventTypes.OobeStateCompleted));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OobeFlip_survives_transient_unavailable_between_polls()
|
||||
{
|
||||
using var tmp = new TempDirectory();
|
||||
var trace = new TraceCapture();
|
||||
using var detector = BuildFlipDetector(tmp, trace, StateQueue("in_progress", "unavailable", "completed"));
|
||||
|
||||
detector.Start();
|
||||
detector.Stop();
|
||||
detector.PollOnceForTest(); // transient reflection failure — must NOT clear the baseline
|
||||
detector.PollOnceForTest(); // "completed" -> flip still detected
|
||||
|
||||
Assert.Equal(1, trace.CountOf(SharedEventTypes.OobeStateCompleted));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OobeFlip_provider_throwing_never_emits_and_never_pollutes_wmi_error_count()
|
||||
{
|
||||
using var tmp = new TempDirectory();
|
||||
var trace = new TraceCapture();
|
||||
using var detector = BuildFlipDetector(tmp, trace, () => throw new System.InvalidOperationException("boom"));
|
||||
|
||||
detector.Start(); // baseline sample throws -> swallowed
|
||||
detector.Stop();
|
||||
detector.PollOnceForTest();
|
||||
detector.PollOnceForTest();
|
||||
|
||||
Assert.Equal(0, trace.CountOf(SharedEventTypes.OobeStateCompleted));
|
||||
// The flip check has its own try/catch — a throw must not fall into
|
||||
// PollForDesktop's outer catch, which increments the WMI error counter the
|
||||
// session-4d5a0b78 liveness diagnostics rely on. first_poll carries that counter.
|
||||
var firstPoll = trace.Events.Find(e => e.EventType == SharedEventTypes.DesktopDetectorFirstPoll);
|
||||
Assert.Equal(0, firstPoll.Data["wmiErrorCount"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OobeFlip_one_shot_flag_survives_ResetForRealUserSwitch()
|
||||
{
|
||||
using var tmp = new TempDirectory();
|
||||
var trace = new TraceCapture();
|
||||
using var detector = BuildFlipDetector(tmp, trace,
|
||||
StateQueue("in_progress", "completed", "in_progress", "completed"));
|
||||
|
||||
detector.Start();
|
||||
detector.Stop();
|
||||
detector.PollOnceForTest(); // flip #1
|
||||
detector.ResetForRealUserSwitch(); // re-arms + re-samples baseline ("in_progress")
|
||||
detector.Stop();
|
||||
detector.PollOnceForTest(); // "completed" again — must NOT re-emit
|
||||
|
||||
Assert.Equal(1, trace.CountOf(SharedEventTypes.OobeStateCompleted));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+406
@@ -0,0 +1,406 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
|
||||
using AutopilotMonitor.Shared;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
|
||||
{
|
||||
/// <summary>
|
||||
/// Allowlist enforcement for the two collectors that used to bypass it:
|
||||
/// <see cref="LogParserCollector"/> read any path the rule named — including
|
||||
/// C:\Users, whose hard block every other file collector honours — and
|
||||
/// <see cref="EventLogCollector"/> read any channel, including Security.
|
||||
/// A tenant admin can author both through the rules API, so the agent is the
|
||||
/// only place this can be enforced.
|
||||
/// </summary>
|
||||
public sealed class GatherRuleCollectorGuardTests : IDisposable
|
||||
{
|
||||
private readonly TempDirectory _tmp = new TempDirectory();
|
||||
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
|
||||
|
||||
// %TEMP% itself lives under C:\Users, which is hard-blocked — the positive
|
||||
// cases need a directory the guard can actually admit, so they run beside
|
||||
// the test assembly instead.
|
||||
private readonly string _outsideUsers = Path.Combine(
|
||||
AppContext.BaseDirectory, "guard-tests-" + Guid.NewGuid().ToString("N"));
|
||||
|
||||
public GatherRuleCollectorGuardTests() => Directory.CreateDirectory(_outsideUsers);
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_tmp.Dispose();
|
||||
try { Directory.Delete(_outsideUsers, recursive: true); } catch { /* best effort */ }
|
||||
}
|
||||
|
||||
private GatherRuleContext Context(bool unrestricted = false, string? imeLogPathOverride = null)
|
||||
=> new GatherRuleContext(
|
||||
new AgentLogger(_tmp.Path, AgentLogLevel.Info),
|
||||
"sess", "tenant",
|
||||
evt => _events.Add(evt),
|
||||
imeLogPathOverride,
|
||||
new LogFilePositionTracker())
|
||||
{
|
||||
UnrestrictedMode = unrestricted
|
||||
};
|
||||
|
||||
private static GatherRule Rule(string collectorType, string target,
|
||||
Dictionary<string, string>? parameters = null) => new GatherRule
|
||||
{
|
||||
RuleId = "GATHER-GUARD-TEST",
|
||||
Title = "guard test",
|
||||
CollectorType = collectorType,
|
||||
Target = target,
|
||||
// Match the model default (empty, never null) so the collectors see the
|
||||
// same shape a deserialized rule without parameters gives them.
|
||||
Parameters = parameters ?? new Dictionary<string, string>(),
|
||||
Trigger = "startup",
|
||||
OutputEventType = "guard_test",
|
||||
};
|
||||
|
||||
private EnrollmentEvent SingleSecurityWarning()
|
||||
=> Assert.Single(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Event log channel allowlist
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Theory]
|
||||
[InlineData("System")]
|
||||
[InlineData("Application")]
|
||||
[InlineData("Setup")]
|
||||
[InlineData("Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin")]
|
||||
[InlineData("Microsoft-Windows-AAD/Operational")]
|
||||
[InlineData("Microsoft-Windows-Shell-Core/Operational")]
|
||||
public void EventLogChannel_OnAllowlist_IsAllowed(string channel)
|
||||
=> Assert.True(GatherRuleGuards.IsEventLogChannelAllowed(channel));
|
||||
|
||||
[Theory]
|
||||
[InlineData("Security")]
|
||||
[InlineData("Microsoft-Windows-PowerShell/Operational")]
|
||||
[InlineData("Windows PowerShell")]
|
||||
[InlineData("Microsoft-Windows-Sysmon/Operational")]
|
||||
public void EventLogChannel_HardBlocked_StaysBlockedEvenUnrestricted(string channel)
|
||||
{
|
||||
Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(channel));
|
||||
Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(channel, unrestrictedMode: true));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("Microsoft-Windows-AADSomethingElse/Operational")] // prefix spoofing
|
||||
[InlineData("Microsoft-Windows-TerminalServices-LocalSessionManager/Operational")]
|
||||
[InlineData("")]
|
||||
public void EventLogChannel_OffAllowlist_IsBlocked(string channel)
|
||||
=> Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(channel));
|
||||
|
||||
[Fact]
|
||||
public void EventLogChannel_Null_IsBlocked()
|
||||
=> Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(null));
|
||||
|
||||
[Fact]
|
||||
public void EventLogChannel_OffAllowlist_IsAllowedInUnrestrictedMode()
|
||||
=> Assert.True(GatherRuleGuards.IsEventLogChannelAllowed(
|
||||
"Microsoft-Windows-TerminalServices-LocalSessionManager/Operational",
|
||||
unrestrictedMode: true));
|
||||
|
||||
[Fact]
|
||||
public void EventLogCollector_SecurityChannel_EmitsSecurityWarningAndReturnsNoData()
|
||||
{
|
||||
var result = new EventLogCollector().Execute(Rule("eventlog", "Security"), Context());
|
||||
|
||||
Assert.Empty(result);
|
||||
var warning = SingleSecurityWarning();
|
||||
Assert.Equal(true, warning.Data["blocked"]);
|
||||
Assert.Equal("Security", warning.Data["target"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void EventLogCollector_SecurityChannel_StaysBlockedInUnrestrictedMode()
|
||||
{
|
||||
var result = new EventLogCollector().Execute(
|
||||
Rule("eventlog", "Security"), Context(unrestricted: true));
|
||||
|
||||
Assert.Empty(result);
|
||||
SingleSecurityWarning();
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Log parser file allowlist
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
private static Dictionary<string, string> TextPattern() => new Dictionary<string, string>
|
||||
{
|
||||
["pattern"] = "(?<line>.+)",
|
||||
["format"] = "text",
|
||||
["trackPosition"] = "false",
|
||||
};
|
||||
|
||||
[Fact]
|
||||
public void LogParser_UserProfilePath_IsBlockedAndEmitsSecurityWarning()
|
||||
{
|
||||
const string target = @"C:\Users\Public\Documents\secret.txt";
|
||||
|
||||
var result = new LogParserCollector().Execute(
|
||||
Rule("logparser", target, TextPattern()), Context());
|
||||
|
||||
Assert.Null(result);
|
||||
var warning = SingleSecurityWarning();
|
||||
Assert.Equal(target, warning.Data["target"]);
|
||||
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LogParser_UserProfilePath_StaysBlockedInUnrestrictedMode()
|
||||
{
|
||||
// C:\Users is a hard block — unrestricted mode must not lift it.
|
||||
new LogParserCollector().Execute(
|
||||
Rule("logparser", @"C:\Users\Public\Documents\secret.txt", TextPattern()),
|
||||
Context(unrestricted: true));
|
||||
|
||||
SingleSecurityWarning();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LogParser_OffAllowlistPath_IsBlockedEvenWhenFileExists()
|
||||
{
|
||||
var path = _tmp.File("evidence.log");
|
||||
File.WriteAllText(path, "line one\nline two\n");
|
||||
|
||||
var result = new LogParserCollector().Execute(
|
||||
Rule("logparser", path, TextPattern()), Context());
|
||||
|
||||
Assert.Null(result);
|
||||
SingleSecurityWarning();
|
||||
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LogParser_WildcardTarget_IsBlockedByItsDirectory()
|
||||
{
|
||||
// Path.GetFullPath rejects '*', so the guard has to check the directory.
|
||||
File.WriteAllText(_tmp.File("evidence.log"), "line one\n");
|
||||
|
||||
new LogParserCollector().Execute(
|
||||
Rule("logparser", Path.Combine(_tmp.Path, "*.log"), TextPattern()), Context());
|
||||
|
||||
SingleSecurityWarning();
|
||||
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LogParser_UnrestrictedMode_ReadsPathOffTheAllowlist()
|
||||
{
|
||||
var path = Path.Combine(_outsideUsers, "evidence.log");
|
||||
File.WriteAllText(path, "line one\nline two\n");
|
||||
|
||||
new LogParserCollector().Execute(
|
||||
Rule("logparser", path, TextPattern()), Context(unrestricted: true));
|
||||
|
||||
Assert.DoesNotContain(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
|
||||
Assert.Equal(2, _events.Count(e => e.EventType == "guard_test"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LogParser_LocalImeLogPathOverride_RelaxesTheAllowlist()
|
||||
{
|
||||
// --ime-log-path is a local CLI flag, never remote config: the operator
|
||||
// using it is already a local admin, so it relaxes the allowlist the way
|
||||
// unrestricted mode does.
|
||||
File.WriteAllText(Path.Combine(_outsideUsers, "AppWorkload.log"), "line one\n");
|
||||
|
||||
new LogParserCollector().Execute(
|
||||
Rule("logparser", @"C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\AppWorkload.log",
|
||||
TextPattern()),
|
||||
Context(imeLogPathOverride: _outsideUsers));
|
||||
|
||||
Assert.DoesNotContain(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
|
||||
Assert.Single(_events, e => e.EventType == "guard_test");
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Command execution
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void Command_OffAllowlist_IsBlockedAndNeverRuns()
|
||||
{
|
||||
var result = new CommandCollector().Execute(
|
||||
Rule("command_allowlisted", "Get-Content C:\\Windows\\System32\\config\\SAM"), Context());
|
||||
|
||||
Assert.Equal(true, result["blocked"]);
|
||||
Assert.False(result.ContainsKey("output"));
|
||||
SingleSecurityWarning();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Command_HardBlockedPattern_StaysBlockedInUnrestrictedMode()
|
||||
{
|
||||
var result = new CommandCollector().Execute(
|
||||
Rule("command_allowlisted", "Invoke-WebRequest https://example.invalid/x"),
|
||||
Context(unrestricted: true));
|
||||
|
||||
Assert.Equal(true, result["blocked"]);
|
||||
SingleSecurityWarning();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Command_ThatHangs_IsKilledAtTheTimeoutRatherThanPinningTheWorker()
|
||||
{
|
||||
// Regression: reading stdout synchronously blocked until the pipe closed,
|
||||
// so WaitForExit's timeout was never reached and a hung command pinned the
|
||||
// worker indefinitely. Unrestricted mode is needed to run an ad-hoc command.
|
||||
var sw = System.Diagnostics.Stopwatch.StartNew();
|
||||
var result = new CommandCollector().Execute(
|
||||
Rule("command_allowlisted", "Start-Sleep -Seconds 120"), Context(unrestricted: true));
|
||||
sw.Stop();
|
||||
|
||||
Assert.Equal(true, result["timed_out"]);
|
||||
Assert.Equal(-1, result["exit_code"]);
|
||||
// The 30s timeout plus drain margin — nowhere near the command's 120s.
|
||||
Assert.True(sw.Elapsed < TimeSpan.FromSeconds(60),
|
||||
$"expected the hung command to be killed at the timeout; took {sw.Elapsed}");
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Every remaining collector actually calls its guard
|
||||
//
|
||||
// The guard functions have their own unit tests, but those prove nothing about
|
||||
// whether a collector consults them before touching the registry, WMI or disk.
|
||||
// These drive the collectors themselves: each blocked target must come back empty,
|
||||
// emit exactly one security_warning naming the target, and produce no rule output.
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
private void AssertBlocked(Dictionary<string, object> result, string target)
|
||||
{
|
||||
Assert.Empty(result);
|
||||
var warning = SingleSecurityWarning();
|
||||
Assert.Equal(true, warning.Data["blocked"]);
|
||||
Assert.Equal(target, warning.Data["target"]);
|
||||
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(@"SAM\SAM\Domains\Account\Users")]
|
||||
[InlineData(@"SECURITY\Policy\Secrets")]
|
||||
[InlineData(@"HKLM\SOFTWARE\Microsoft\EnrollmentsSomethingElse")] // prefix spoofing
|
||||
public void RegistryCollector_OffAllowlistPath_IsBlockedBeforeTheHiveIsOpened(string target)
|
||||
=> AssertBlocked(new RegistryCollector().Execute(Rule("registry", target), Context()), target);
|
||||
|
||||
[Theory]
|
||||
[InlineData("SELECT * FROM Win32_UserAccount")]
|
||||
[InlineData("SELECT * FROM Win32_ShadowCopy")]
|
||||
[InlineData("SELECT * FROM Win32_OperatingSystemExtra")] // prefix spoofing
|
||||
public void WmiCollector_OffAllowlistQuery_IsBlockedBeforeTheQueryRuns(string query)
|
||||
=> AssertBlocked(new WmiCollector().Execute(Rule("wmi", query), Context()), query);
|
||||
|
||||
[Fact]
|
||||
public void JsonCollector_UserProfilePath_IsBlockedBeforeTheFileIsOpened()
|
||||
{
|
||||
const string target = @"C:\Users\Public\Documents\secrets.json";
|
||||
AssertBlocked(
|
||||
new JsonCollector().Execute(
|
||||
Rule("json", target, new Dictionary<string, string> { ["jsonpath"] = "$.token" }),
|
||||
Context()),
|
||||
target);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void XmlCollector_UserProfilePath_IsBlockedBeforeTheFileIsOpened()
|
||||
{
|
||||
const string target = @"C:\Users\Public\Documents\secrets.xml";
|
||||
AssertBlocked(
|
||||
new XmlCollector().Execute(
|
||||
Rule("xml", target, new Dictionary<string, string> { ["xpath"] = "//token" }),
|
||||
Context()),
|
||||
target);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void FileCollector_UserProfilePath_IsBlockedBeforeTheFileIsOpened()
|
||||
{
|
||||
const string target = @"C:\Users\Public\Documents\secrets.txt";
|
||||
AssertBlocked(new FileCollector().Execute(Rule("file", target), Context()), target);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void JsonCollector_AllowedPath_ReadsTheFile()
|
||||
{
|
||||
// Counterpart to the blocked cases: proves the guard is a gate, not a wall —
|
||||
// the collector does reach the file once the path is admitted.
|
||||
var path = Path.Combine(_outsideUsers, "inventory.json");
|
||||
File.WriteAllText(path, "{ \"agent\": { \"version\": \"2.4.0\" } }");
|
||||
|
||||
var result = new JsonCollector().Execute(
|
||||
Rule("json", path, new Dictionary<string, string> { ["jsonpath"] = "$.agent.version" }),
|
||||
Context(unrestricted: true));
|
||||
|
||||
Assert.DoesNotContain(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
|
||||
Assert.Contains("2.4.0", string.Join(",", result.Values));
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Path traversal
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void FilePath_TraversalOutOfAnAllowedPrefix_IsBlocked()
|
||||
{
|
||||
// The raw string starts with an allowed prefix, so a naive StartsWith check would
|
||||
// admit it. Path.GetFullPath runs FIRST, so the allowlist sees C:\Secret.txt.
|
||||
const string traversal =
|
||||
@"C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\..\..\..\..\Secret.txt";
|
||||
|
||||
Assert.Equal(@"C:\Secret.txt", Path.GetFullPath(traversal));
|
||||
Assert.False(GatherRuleGuards.IsFilePathAllowed(traversal));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void FilePath_TraversalIntoTheUsersHardBlock_StaysBlockedEvenUnrestricted()
|
||||
{
|
||||
// Unrestricted mode lifts the allowlist but never the C:\Users privacy block —
|
||||
// and it must not be reachable by spelling the path with '..' either.
|
||||
const string traversal =
|
||||
@"C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\..\..\..\..\Users\Public\ntuser.dat";
|
||||
|
||||
Assert.StartsWith(@"C:\Users\", Path.GetFullPath(traversal), StringComparison.OrdinalIgnoreCase);
|
||||
Assert.False(GatherRuleGuards.IsFilePathAllowed(traversal));
|
||||
Assert.False(GatherRuleGuards.IsFilePathAllowed(traversal, unrestrictedMode: true));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void FilePath_TraversalIntoTheConfigHiveDirectory_StaysBlockedEvenUnrestricted()
|
||||
{
|
||||
const string traversal = @"C:\Windows\Logs\..\System32\config\SAM";
|
||||
|
||||
Assert.False(GatherRuleGuards.IsFilePathAllowed(traversal));
|
||||
Assert.False(GatherRuleGuards.IsFilePathAllowed(traversal, unrestrictedMode: true));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void FileCollector_TraversalTarget_IsBlockedAndReported()
|
||||
{
|
||||
const string traversal =
|
||||
@"C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\..\..\..\..\Users\Public\ntuser.dat";
|
||||
|
||||
AssertBlocked(new FileCollector().Execute(Rule("file", traversal), Context()), traversal);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LogParser_ImeLogPathOverride_StillHonoursTheUsersHardBlock()
|
||||
{
|
||||
new LogParserCollector().Execute(
|
||||
Rule("logparser", @"C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\AppWorkload.log",
|
||||
TextPattern()),
|
||||
Context(imeLogPathOverride: @"C:\Users\Public\Documents"));
|
||||
|
||||
SingleSecurityWarning();
|
||||
}
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -192,7 +192,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
|
||||
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = "1" }),
|
||||
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = 1 }));
|
||||
Assert.NotEqual(
|
||||
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = null }),
|
||||
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = null! }),
|
||||
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = "<null>" }));
|
||||
}
|
||||
|
||||
|
||||
+246
@@ -0,0 +1,246 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Diagnostics;
|
||||
using System.Threading;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
|
||||
{
|
||||
/// <summary>
|
||||
/// The <c>phase_exit</c> trigger: a one-shot collection when a phase is LEFT — the closing
|
||||
/// bookend to <c>phase_change</c> (which fires on ENTER). The distinguishing invariant is that
|
||||
/// both the TriggerPhase match and the phase-scope gate must evaluate against the phase being
|
||||
/// left, i.e. before the executor's current-phase state moves forward.
|
||||
/// Rules use the absent-registry-path pattern so no real system state is touched.
|
||||
/// </summary>
|
||||
[Collection("SerialThreading")] // rules execute on the shared ThreadPool
|
||||
public sealed class GatherRuleExecutorPhaseExitTests : IDisposable
|
||||
{
|
||||
private readonly TempDirectory _tmp = new TempDirectory();
|
||||
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
|
||||
private readonly object _eventsGate = new object();
|
||||
private readonly GatherRuleExecutor _executor;
|
||||
|
||||
private const string AbsentPath =
|
||||
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_9c4d2a7f-0000-0000-0000-000000000000";
|
||||
|
||||
public GatherRuleExecutorPhaseExitTests()
|
||||
{
|
||||
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
|
||||
_executor = new GatherRuleExecutor(
|
||||
"sess", "tenant",
|
||||
evt => { lock (_eventsGate) _events.Add(evt); },
|
||||
logger)
|
||||
{
|
||||
UnrestrictedMode = true
|
||||
};
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_executor.Dispose();
|
||||
_tmp.Dispose();
|
||||
}
|
||||
|
||||
private int EventCount { get { lock (_eventsGate) return _events.Count; } }
|
||||
|
||||
private bool WaitForEventCount(int count, int timeoutMs = 10000)
|
||||
{
|
||||
var sw = Stopwatch.StartNew();
|
||||
while (sw.ElapsedMilliseconds < timeoutMs)
|
||||
{
|
||||
if (EventCount >= count) return true;
|
||||
Thread.Sleep(25);
|
||||
}
|
||||
return EventCount >= count;
|
||||
}
|
||||
|
||||
private static GatherRule ExitRule(
|
||||
string id, string triggerPhase, List<string>? activePhases = null, string? fromPhase = null)
|
||||
=> new GatherRule
|
||||
{
|
||||
RuleId = id,
|
||||
Title = id,
|
||||
CollectorType = "registry",
|
||||
Target = AbsentPath,
|
||||
Trigger = "phase_exit",
|
||||
TriggerPhase = triggerPhase,
|
||||
OutputEventType = "gather_test",
|
||||
Enabled = true,
|
||||
ActivePhases = activePhases,
|
||||
ActiveFromPhase = fromPhase,
|
||||
};
|
||||
|
||||
[Fact]
|
||||
public void FiresOnceWhenTheNamedPhaseIsLeft_NotWhenItIsEntered()
|
||||
{
|
||||
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-001", "AccountSetup") });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
|
||||
Thread.Sleep(300);
|
||||
Assert.Equal(0, EventCount);
|
||||
|
||||
// Entering AccountSetup must NOT fire an exit rule.
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
|
||||
Thread.Sleep(300);
|
||||
Assert.Equal(0, EventCount);
|
||||
|
||||
// Leaving it does.
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
|
||||
Assert.True(WaitForEventCount(1));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DoesNotRefireWhenThePhaseIsLeftTwice()
|
||||
{
|
||||
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-002", "AccountSetup") });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
|
||||
Assert.True(WaitForEventCount(1));
|
||||
|
||||
// A phase can be re-entered (ESP re-reporting); the one-shot slot stays consumed.
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
|
||||
Thread.Sleep(400);
|
||||
Assert.Equal(1, EventCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void EmptyTriggerPhase_FiresOnEveryExit()
|
||||
{
|
||||
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-003", "") });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup); // exit of Unknown — must NOT fire
|
||||
Thread.Sleep(300);
|
||||
Assert.Equal(0, EventCount);
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup); // exit of DeviceSetup
|
||||
Assert.True(WaitForEventCount(1));
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser); // exit of AccountSetup
|
||||
Assert.True(WaitForEventCount(2));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void NeverFiresOnExitOfUnknown()
|
||||
{
|
||||
// Before the first phase signal there is no phase to leave.
|
||||
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-004", "") });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.Start);
|
||||
Thread.Sleep(300);
|
||||
Assert.Equal(0, EventCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RepeatedSamePhaseSignal_IsNotAnExit()
|
||||
{
|
||||
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-005", "") });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup); // same phase — no transition
|
||||
Thread.Sleep(300);
|
||||
Assert.Equal(0, EventCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void FiresOnTransitionIntoFailed_CapturingStateAtTheFailurePoint()
|
||||
{
|
||||
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-006", "AccountSetup") });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.Failed);
|
||||
|
||||
Assert.True(WaitForEventCount(1), "leaving a phase into Failed must still snapshot it");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void FiresOnExitOfCompleteOnly_WhenCompleteIsActuallyLeft()
|
||||
{
|
||||
// "End of enrollment" is better served by phase_change on Complete / on_event; an exit
|
||||
// rule on Complete only fires if a later transition happens at all.
|
||||
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-007", "Complete") });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.FinalizingSetup);
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.Complete);
|
||||
Thread.Sleep(300);
|
||||
Assert.Equal(0, EventCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ScopeGate_EvaluatesAgainstThePhaseBeingLeft()
|
||||
{
|
||||
// Scope = only during DeviceSetup. Leaving DeviceSetup must fire (the rule is still in
|
||||
// scope at that instant) even though the NEW phase is outside the scope.
|
||||
_executor.UpdateRules(new List<GatherRule>
|
||||
{
|
||||
ExitRule("GATHER-PX-010", "DeviceSetup", activePhases: new List<string> { "DeviceSetup" })
|
||||
});
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
|
||||
|
||||
Assert.True(WaitForEventCount(1),
|
||||
"the scope gate must see the phase being left, not the one being entered");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OutOfScopeExit_DoesNotFire_AndDoesNotConsumeTheOneShotSlot()
|
||||
{
|
||||
// Scope = only during AccountSetup; the rule exits DeviceSetup while out of scope.
|
||||
var rule = ExitRule("GATHER-PX-011", "", activePhases: new List<string> { "AccountSetup" });
|
||||
_executor.UpdateRules(new List<GatherRule> { rule });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup); // exit of DeviceSetup: out of scope
|
||||
Thread.Sleep(300);
|
||||
Assert.Equal(0, EventCount);
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser); // exit of AccountSetup: in scope
|
||||
Assert.True(WaitForEventCount(1));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ExitAndChangeRulesForTheSamePhase_BothFireIndependently()
|
||||
{
|
||||
// The dedup key spaces must not collide: same ruleId shape, same phase, different trigger.
|
||||
var enter = new GatherRule
|
||||
{
|
||||
RuleId = "GATHER-PX-020",
|
||||
Title = "enter",
|
||||
CollectorType = "registry",
|
||||
Target = AbsentPath,
|
||||
Trigger = "phase_change",
|
||||
TriggerPhase = "AccountSetup",
|
||||
OutputEventType = "gather_test",
|
||||
Enabled = true,
|
||||
};
|
||||
var exit = ExitRule("GATHER-PX-021", "AccountSetup");
|
||||
_executor.UpdateRules(new List<GatherRule> { enter, exit });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
|
||||
Assert.True(WaitForEventCount(1)); // enter fired
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
|
||||
Assert.True(WaitForEventCount(2)); // exit fired
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ExitRuleWithEmitOnChange_StillEmitsItsFirstResult()
|
||||
{
|
||||
var rule = ExitRule("GATHER-PX-030", "AccountSetup");
|
||||
rule.EmitMode = "on_change";
|
||||
_executor.UpdateRules(new List<GatherRule> { rule });
|
||||
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
|
||||
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
|
||||
|
||||
Assert.True(WaitForEventCount(1));
|
||||
}
|
||||
}
|
||||
}
|
||||
+2
-2
@@ -63,7 +63,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
|
||||
|
||||
/// <summary>Interval rule with a 1h period — the timer never ticks within a test run,
|
||||
/// so these rules exist purely for synchronous scope-matrix assertions.</summary>
|
||||
private static GatherRule ScopeProbeRule(string id, List<string> activePhases = null, string fromPhase = null)
|
||||
private static GatherRule ScopeProbeRule(string id, List<string>? activePhases = null, string? fromPhase = null)
|
||||
=> new GatherRule
|
||||
{
|
||||
RuleId = id,
|
||||
@@ -78,7 +78,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
|
||||
ActiveFromPhase = fromPhase,
|
||||
};
|
||||
|
||||
private static GatherRule StartupRule(string id, List<string> activePhases = null, string fromPhase = null)
|
||||
private static GatherRule StartupRule(string id, List<string>? activePhases = null, string? fromPhase = null)
|
||||
=> new GatherRule
|
||||
{
|
||||
RuleId = id,
|
||||
|
||||
+1
-1
@@ -27,7 +27,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
|
||||
[InlineData(" ")] // whitespace only
|
||||
[InlineData("<![LOG[truncated line with no closing")] // truncated / missing fields
|
||||
[InlineData("garbage prefix <![LOG[msg]LOG]!><time=\"06:08:04.8834397\" date=\"2-8-2026\" component=\"C\" context=\"\" type=\"1\" thread=\"1\" file=\"\">")] // leading junk (non-ignorable) fails the StartsWith gate
|
||||
public void TryParseLine_returns_false_without_throwing_for_malformed_input(string line)
|
||||
public void TryParseLine_returns_false_without_throwing_for_malformed_input(string? line)
|
||||
{
|
||||
var ok = CmTraceLogParser.TryParseLine(line, out var entry);
|
||||
|
||||
|
||||
+3
-2
@@ -24,12 +24,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
|
||||
/// </summary>
|
||||
public sealed class ImeLogPatternReDoSTests
|
||||
{
|
||||
private static ImeLogPattern Pattern(string id, string regex, string action = null) => new ImeLogPattern
|
||||
private static ImeLogPattern Pattern(string id, string regex, string? action = null) => new ImeLogPattern
|
||||
{
|
||||
PatternId = id,
|
||||
Pattern = regex,
|
||||
Category = "always",
|
||||
Action = action,
|
||||
// Action is optional on the wire (absent = null); the model declares it non-nullable.
|
||||
Action = action!,
|
||||
Enabled = true,
|
||||
Parameters = new Dictionary<string, string>(),
|
||||
};
|
||||
|
||||
+1
@@ -17,6 +17,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Office
|
||||
/// cadence and stops at its bound, is idempotent, the initial scan raises when binaries already
|
||||
/// exist, a filesystem create raises, and Dispose cancels the re-probe.
|
||||
/// </summary>
|
||||
[Collection("SerialThreading")] // the bounded re-probe runs on the shared ThreadPool
|
||||
public sealed class OfficeBinaryWatcherTests
|
||||
{
|
||||
private static readonly string[] CoreBinaries = { "WINWORD.EXE", "EXCEL.EXE", "POWERPNT.EXE", "OUTLOOK.EXE" };
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
#nullable enable
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
|
||||
{
|
||||
/// <summary>
|
||||
/// Smoke tests for the WinRT <c>SystemSetupInfo.OutOfBoxExperienceState</c> reader.
|
||||
/// The actual state on the test machine is unknown (CI runners are post-OOBE; Server
|
||||
/// SKUs may lack the UniversalApiContract entirely), so only the fail-soft contract
|
||||
/// is pinned: never throw, always return a member of the known value set.
|
||||
/// </summary>
|
||||
public sealed class OobeStateReaderTests
|
||||
{
|
||||
[Fact]
|
||||
public void Read_never_throws_and_returns_known_value()
|
||||
{
|
||||
var value = OobeStateReader.Read();
|
||||
|
||||
Assert.False(string.IsNullOrEmpty(value));
|
||||
Assert.True(
|
||||
value == "not_started"
|
||||
|| value == "in_progress"
|
||||
|| value == "completed"
|
||||
|| value == OobeStateReader.Unavailable
|
||||
|| value.StartsWith("unknown_"),
|
||||
$"unexpected reader value '{value}'");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Read_is_stable_across_repeated_calls()
|
||||
{
|
||||
// Exercises the cached-PropertyInfo path (second call skips resolution).
|
||||
var first = OobeStateReader.Read();
|
||||
var second = OobeStateReader.Read();
|
||||
|
||||
Assert.Equal(first, second);
|
||||
}
|
||||
}
|
||||
}
|
||||
+212
@@ -0,0 +1,212 @@
|
||||
using System;
|
||||
using System.Diagnostics;
|
||||
using System.IO;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime;
|
||||
using AutopilotMonitor.Agent.V2.Core.Security;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Runtime
|
||||
{
|
||||
/// <summary>
|
||||
/// Executes the generated cleanup script against REAL scratch trees with a real
|
||||
/// powershell.exe — end-to-end replays of the 2026-07-22 incident mechanics that the
|
||||
/// string pins in <see cref="CleanupServiceScriptTests"/> cannot prove:
|
||||
/// <list type="bullet">
|
||||
/// <item>a locked tree must abort WITHOUT deleting anything (old script force-deleted
|
||||
/// marker + state around the locks and orphaned the binaries),</item>
|
||||
/// <item>a straggler agent process (the incident's second instance) must be found by
|
||||
/// NAME, killed, and the cleanup must then complete,</item>
|
||||
/// <item>concurrent cleanup scripts must be serialized by the lock file.</item>
|
||||
/// </list>
|
||||
/// Timings are dialed down via the test-only BuildCleanupScript parameters so each case
|
||||
/// runs in seconds. Every test uses GUID-unique process/task/tree names so nothing real
|
||||
/// (a dev-box agent, a real Scheduled Task) can ever be touched.
|
||||
/// </summary>
|
||||
[Collection("SerialThreading")]
|
||||
public class CleanupServiceScriptIntegrationTests : IDisposable
|
||||
{
|
||||
private readonly string _root;
|
||||
private readonly string _basePath;
|
||||
private readonly string _taskName = $"AmTest-NoSuchTask-{Guid.NewGuid():N}";
|
||||
private readonly string _fakeProcessName = $"AmTestAgent{Guid.NewGuid().ToString("N").Substring(0, 8)}";
|
||||
|
||||
public CleanupServiceScriptIntegrationTests()
|
||||
{
|
||||
_root = Path.Combine(Path.GetTempPath(), $"am-cleanup-it-{Guid.NewGuid():N}");
|
||||
_basePath = Path.Combine(_root, "AutopilotMonitor");
|
||||
Directory.CreateDirectory(Path.Combine(_basePath, "Agent"));
|
||||
Directory.CreateDirectory(Path.Combine(_basePath, "State"));
|
||||
Directory.CreateDirectory(Path.Combine(_basePath, "Logs"));
|
||||
File.WriteAllText(Path.Combine(_basePath, "Agent", "fake.exe"), "x");
|
||||
File.WriteAllText(Path.Combine(_basePath, "State", "enrollment-complete.marker"), "x");
|
||||
File.WriteAllText(Path.Combine(_basePath, "Logs", "agent.log"), "x");
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
try { Directory.Delete(_root, recursive: true); } catch { }
|
||||
}
|
||||
|
||||
private static string LockPath => Path.Combine(Path.GetTempPath(), "AutopilotMonitor-Cleanup.lock");
|
||||
|
||||
private string BuildScript(bool keepLogs = false, int processWaitSeconds = 1)
|
||||
=> CleanupService.BuildCleanupScript(
|
||||
agentProcessName: _fakeProcessName,
|
||||
agentBasePath: _basePath,
|
||||
keepLogs: keepLogs,
|
||||
scheduledTaskName: _taskName,
|
||||
rebootOnComplete: false,
|
||||
initialDelaySeconds: 0,
|
||||
processWaitSeconds: processWaitSeconds,
|
||||
renameRetries: 2,
|
||||
renameRetryDelaySeconds: 1);
|
||||
|
||||
/// <summary>Runs the script synchronously in a real powershell.exe. Returns the script path.</summary>
|
||||
private string RunScript(string script)
|
||||
{
|
||||
var scriptPath = Path.Combine(_root, $"cleanup-{Guid.NewGuid():N}.ps1");
|
||||
File.WriteAllText(scriptPath, script);
|
||||
using (var proc = Process.Start(new ProcessStartInfo
|
||||
{
|
||||
FileName = SystemPaths.PowerShell,
|
||||
Arguments = $"-NoProfile -ExecutionPolicy Bypass -File \"{scriptPath}\"",
|
||||
UseShellExecute = false,
|
||||
CreateNoWindow = true,
|
||||
}))
|
||||
{
|
||||
Assert.True(proc.WaitForExit(90_000), "cleanup script did not finish within 90s");
|
||||
}
|
||||
return scriptPath;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Happy_path_deletes_tree_and_removes_lock_and_script()
|
||||
{
|
||||
var scriptPath = RunScript(BuildScript());
|
||||
|
||||
Assert.False(Directory.Exists(_basePath), "tree should be fully deleted");
|
||||
Assert.Empty(Directory.GetFileSystemEntries(_root, "*.del-*"));
|
||||
Assert.False(File.Exists(scriptPath), "script should self-delete");
|
||||
Assert.False(File.Exists(LockPath), "cleanup lock should be released");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Locked_tree_aborts_completely_marker_and_files_survive()
|
||||
{
|
||||
// The incident regression: with a file locked inside the tree the OLD script
|
||||
// deleted everything else (incl. the enrollment-complete marker) and left the
|
||||
// binaries orphaned with the task already gone. The hardened script must treat
|
||||
// the failed probe-rename as "someone still holds this" and walk away entirely.
|
||||
using (File.Open(Path.Combine(_basePath, "Agent", "fake.exe"),
|
||||
FileMode.Open, FileAccess.Read, FileShare.None))
|
||||
{
|
||||
RunScript(BuildScript());
|
||||
}
|
||||
|
||||
Assert.True(File.Exists(Path.Combine(_basePath, "Agent", "fake.exe")), "binaries must stay in place");
|
||||
Assert.True(File.Exists(Path.Combine(_basePath, "State", "enrollment-complete.marker")),
|
||||
"marker must survive so the next boot retries cleanup");
|
||||
Assert.Empty(Directory.GetFileSystemEntries(_root, "*.del-*"));
|
||||
Assert.False(File.Exists(LockPath));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Straggler_process_holding_the_tree_is_killed_and_cleanup_completes()
|
||||
{
|
||||
// Full incident replay: a SECOND agent-named process (here: a renamed
|
||||
// powershell.exe, matching how only the process NAME identifies agents) holds a
|
||||
// file in the tree open. The old script waited on the launcher PID only and ran
|
||||
// its force-delete against the live locks. The hardened script must find the
|
||||
// straggler by name, kill it after the wait deadline, and then delete everything.
|
||||
var fakeExe = Path.Combine(_root, _fakeProcessName + ".exe");
|
||||
File.Copy(SystemPaths.PowerShell, fakeExe);
|
||||
|
||||
var lockTarget = Path.Combine(_basePath, "Agent", "fake.exe").Replace("'", "''");
|
||||
Process straggler = null;
|
||||
try
|
||||
{
|
||||
straggler = Process.Start(new ProcessStartInfo
|
||||
{
|
||||
FileName = fakeExe,
|
||||
Arguments = "-NoProfile -Command \"$f = [IO.File]::Open('" + lockTarget +
|
||||
"', 'Open', 'Read', 'None'); Start-Sleep -Seconds 120\"",
|
||||
UseShellExecute = false,
|
||||
CreateNoWindow = true,
|
||||
});
|
||||
|
||||
// Give the straggler time to acquire its file handle before cleanup starts.
|
||||
var acquireDeadline = DateTime.UtcNow.AddSeconds(20);
|
||||
var acquired = false;
|
||||
while (!acquired && DateTime.UtcNow < acquireDeadline)
|
||||
{
|
||||
try
|
||||
{
|
||||
using (File.Open(Path.Combine(_basePath, "Agent", "fake.exe"),
|
||||
FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
|
||||
{
|
||||
}
|
||||
}
|
||||
catch (IOException)
|
||||
{
|
||||
acquired = true; // share violation ⇒ straggler holds the file
|
||||
}
|
||||
if (!acquired) System.Threading.Thread.Sleep(200);
|
||||
}
|
||||
Assert.True(acquired, "straggler never acquired its file lock");
|
||||
|
||||
RunScript(BuildScript(processWaitSeconds: 2));
|
||||
|
||||
Assert.True(straggler.HasExited, "straggler must be force-killed by the name-based backstop");
|
||||
Assert.False(Directory.Exists(_basePath), "after the kill the tree must be fully deleted");
|
||||
Assert.Empty(Directory.GetFileSystemEntries(_root, "*.del-*"));
|
||||
}
|
||||
finally
|
||||
{
|
||||
try { if (straggler != null && !straggler.HasExited) straggler.Kill(); } catch { }
|
||||
try { straggler?.Dispose(); } catch { }
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Fresh_foreign_lock_backs_off_without_touching_the_tree()
|
||||
{
|
||||
File.WriteAllText(LockPath, "held by another cleanup");
|
||||
try
|
||||
{
|
||||
var scriptPath = RunScript(BuildScript());
|
||||
|
||||
Assert.True(File.Exists(Path.Combine(_basePath, "Agent", "fake.exe")),
|
||||
"loser script must not touch the tree — the lock holder owns the deletion");
|
||||
Assert.True(File.Exists(LockPath), "foreign lock must not be deleted by the loser");
|
||||
Assert.False(File.Exists(scriptPath), "loser still self-deletes its script");
|
||||
}
|
||||
finally
|
||||
{
|
||||
try { File.Delete(LockPath); } catch { }
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Stale_foreign_lock_is_taken_over_and_cleanup_proceeds()
|
||||
{
|
||||
File.WriteAllText(LockPath, "crashed cleanup from a previous boot");
|
||||
File.SetLastWriteTime(LockPath, DateTime.Now.AddMinutes(-15));
|
||||
|
||||
RunScript(BuildScript());
|
||||
|
||||
Assert.False(Directory.Exists(_basePath), "stale lock must be taken over, cleanup must run");
|
||||
Assert.False(File.Exists(LockPath));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void KeepLogs_variant_deletes_everything_except_logs()
|
||||
{
|
||||
RunScript(BuildScript(keepLogs: true));
|
||||
|
||||
Assert.True(File.Exists(Path.Combine(_basePath, "Logs", "agent.log")), "Logs must survive");
|
||||
Assert.False(Directory.Exists(Path.Combine(_basePath, "Agent")));
|
||||
Assert.False(Directory.Exists(Path.Combine(_basePath, "State")));
|
||||
Assert.Empty(Directory.GetFileSystemEntries(_basePath, "*.del-*"));
|
||||
}
|
||||
}
|
||||
}
|
||||
+135
@@ -0,0 +1,135 @@
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Runtime
|
||||
{
|
||||
/// <summary>
|
||||
/// Pins the LIFE-F7 hardening invariants of the self-destruct cleanup script
|
||||
/// (incident 2026-07-22: a second agent instance held exe + loaded DLLs while the
|
||||
/// old script — waiting on the launcher PID only — force-deleted around the locks
|
||||
/// AFTER removing the Scheduled Task, orphaning the binaries with no retry path).
|
||||
/// These are string-level pins on <see cref="CleanupService.BuildCleanupScript"/>:
|
||||
/// weakening any of them silently re-introduces the orphaned-binaries failure mode.
|
||||
/// </summary>
|
||||
public class CleanupServiceScriptTests
|
||||
{
|
||||
private const string ProcessName = "AutopilotMonitor.Agent";
|
||||
private const string BasePath = @"C:\ProgramData\AutopilotMonitor";
|
||||
private const string TaskName = "AutopilotMonitorAgent";
|
||||
|
||||
private static string Script(bool keepLogs = false, bool reboot = false)
|
||||
=> CleanupService.BuildCleanupScript(ProcessName, BasePath, keepLogs, TaskName, reboot);
|
||||
|
||||
[Theory]
|
||||
[InlineData(false)]
|
||||
[InlineData(true)]
|
||||
public void Waits_by_process_name_not_launcher_pid(bool keepLogs)
|
||||
{
|
||||
var script = Script(keepLogs);
|
||||
|
||||
// LIFE-F7 #2: name-based wait covers marker-retry / self-update / manual instances
|
||||
// and is immune to PID reuse. Wait-Process -Id saw exactly one PID and nothing else.
|
||||
Assert.Contains($"Get-Process -Name '{ProcessName}'", script);
|
||||
Assert.DoesNotContain("Wait-Process", script);
|
||||
|
||||
// Backstop: stragglers are force-killed — their images are deleted right after.
|
||||
Assert.Contains("Stop-Process -Force", script);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(false)]
|
||||
[InlineData(true)]
|
||||
public void Never_deletes_the_original_path(bool keepLogs)
|
||||
{
|
||||
var script = Script(keepLogs);
|
||||
|
||||
// LIFE-F7 #3: deletion happens exclusively on successfully renamed targets
|
||||
// ($entry.Renamed). A Remove-Item against the original tree is what deleted
|
||||
// marker + state around locked binaries in the incident.
|
||||
Assert.DoesNotContain($"Remove-Item -Path '{BasePath}'", script);
|
||||
Assert.Contains("Remove-Item -Path $entry.Renamed", script);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(false)]
|
||||
[InlineData(true)]
|
||||
public void Task_removal_is_gated_on_probe_and_deletion_on_task_gone(bool keepLogs)
|
||||
{
|
||||
var script = Script(keepLogs);
|
||||
|
||||
// Probe-rename first; only a successful probe reaches task removal, and only a
|
||||
// removed task reaches deletion. Both failure branches rename BACK so the next
|
||||
// boot retries with task + marker + files intact.
|
||||
Assert.Contains("if ($probeOk)", script);
|
||||
Assert.Contains("if ($taskGone)", script);
|
||||
Assert.Contains("Rename-Item -Path $entry.Renamed -NewName $entry.Original", script);
|
||||
|
||||
// LIFE-F6 task-removal loop + module-independent fallback stay intact.
|
||||
Assert.Contains("Unregister-ScheduledTask -TaskName $taskName", script);
|
||||
Assert.Contains("schtasks.exe /Delete /TN $taskName /F", script);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(false)]
|
||||
[InlineData(true)]
|
||||
public void Concurrent_cleanup_scripts_are_serialized_by_lock_file(bool keepLogs)
|
||||
{
|
||||
var script = Script(keepLogs);
|
||||
|
||||
// LIFE-F7 #1: CreateNew is the atomic single-flight gate; the loser exits without
|
||||
// touching the tree. Stale locks (>10 min) are taken over.
|
||||
Assert.Contains("AutopilotMonitor-Cleanup.lock", script);
|
||||
Assert.Contains("[System.IO.FileMode]::CreateNew", script);
|
||||
Assert.Contains("$lockAge.TotalMinutes -gt 10", script);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(false)]
|
||||
[InlineData(true)]
|
||||
public void Rename_targets_carry_a_guid_suffix(bool keepLogs)
|
||||
{
|
||||
// A fixed '.del' target collides with leftovers of earlier attempts and with a
|
||||
// concurrent script's rename — the GUID makes every attempt's target unique.
|
||||
Assert.Contains(".del-' + [Guid]::NewGuid().ToString('N')", Script(keepLogs));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void KeepLogs_variant_excludes_logs_directory()
|
||||
{
|
||||
var script = Script(keepLogs: true);
|
||||
Assert.Contains("-Exclude 'Logs'", script);
|
||||
Assert.DoesNotContain($"Rename-Item -Path '{BasePath}' -NewName", script);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Full_variant_renames_the_whole_tree()
|
||||
{
|
||||
var script = Script(keepLogs: false);
|
||||
Assert.Contains($"Rename-Item -Path '{BasePath}' -NewName $renamedPath", script);
|
||||
Assert.DoesNotContain("-Exclude 'Logs'", script);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(false)]
|
||||
[InlineData(true)]
|
||||
public void Reboot_flag_controls_shutdown_invocation(bool reboot)
|
||||
{
|
||||
var script = Script(keepLogs: false, reboot: reboot);
|
||||
if (reboot)
|
||||
Assert.Contains("shutdown.exe /r /t 10", script);
|
||||
else
|
||||
Assert.DoesNotContain("shutdown.exe", script);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(false)]
|
||||
[InlineData(true)]
|
||||
public void Script_and_lock_are_released_in_finally(bool keepLogs)
|
||||
{
|
||||
var script = Script(keepLogs);
|
||||
Assert.Contains("finally", script);
|
||||
Assert.Contains("$lockStream.Dispose()", script);
|
||||
Assert.Contains("Remove-Item -Path $scriptPath -Force", script);
|
||||
}
|
||||
}
|
||||
}
|
||||
-195
@@ -1,195 +0,0 @@
|
||||
#nullable enable
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
|
||||
using AutopilotMonitor.Agent.V2.Core.Orchestration;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
|
||||
using AutopilotMonitor.DecisionCore.Engine;
|
||||
using AutopilotMonitor.DecisionCore.Signals;
|
||||
using AutopilotMonitor.Shared;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
|
||||
{
|
||||
/// <summary>
|
||||
/// Shell-Core 62407 "RebootCoalescing" branch — ESP-initiated coalesced reboot (MDM policies
|
||||
/// forced a mid-ESP restart → second sign-in). Pure informational corroboration for the
|
||||
/// MdmRebootPolicyTracker's per-URI 2800 attribution: no FinalizingSetup transition, no C#
|
||||
/// event raise, and — as a DELIBERATE deviation from the other 62407 backfill branches — the
|
||||
/// backfill path DOES emit (the live emit races the very reboot that kills the process).
|
||||
/// </summary>
|
||||
public sealed class ShellCoreTrackerRebootCoalescingTests
|
||||
{
|
||||
private static readonly DateTime ClockNow = new DateTime(2026, 7, 20, 10, 5, 0, DateTimeKind.Utc);
|
||||
private const string CoalescingDescription =
|
||||
"CloudExperienceHost Web App Event 2. Name: 'CommercialOOBE_DeviceSetup_RebootCoalescing', Value: '...'.";
|
||||
|
||||
private static (ShellCoreTracker tracker, FakeSignalIngressSink sink) MakeTracker(TempDirectory tmp)
|
||||
{
|
||||
var sink = new FakeSignalIngressSink();
|
||||
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info);
|
||||
var tracker = new ShellCoreTracker(
|
||||
sessionId: "S1",
|
||||
tenantId: "T1",
|
||||
post: new InformationalEventPost(sink, new VirtualClock(ClockNow)),
|
||||
logger: logger,
|
||||
helloTracker: null);
|
||||
return (tracker, sink);
|
||||
}
|
||||
|
||||
private static IReadOnlyList<FakeSignalIngressSink.PostedSignal> ByType(FakeSignalIngressSink sink, string eventType) =>
|
||||
sink.Posted.Where(p =>
|
||||
p.Kind == DecisionSignalKind.InformationalEvent
|
||||
&& p.Payload != null
|
||||
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
|
||||
&& et == eventType).ToList();
|
||||
|
||||
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal s) =>
|
||||
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
|
||||
|
||||
[Fact]
|
||||
public void ProcessEvent_62407_RebootCoalescing_EmitsWarning_ImmediateUpload()
|
||||
{
|
||||
using var tmp = new TempDirectory();
|
||||
var (tracker, sink) = MakeTracker(tmp);
|
||||
using (tracker)
|
||||
{
|
||||
var sourceTime = ClockNow.AddSeconds(-5);
|
||||
tracker.ProcessEvent(
|
||||
eventId: ShellCoreTracker.EventId_ShellCore_WebAppEvent,
|
||||
description: CoalescingDescription,
|
||||
timestamp: sourceTime,
|
||||
providerName: "Microsoft-Windows-Shell-Core",
|
||||
isBackfill: false);
|
||||
|
||||
var s = Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
|
||||
Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]);
|
||||
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
|
||||
Assert.Equal(sourceTime, s.OccurredAtUtc);
|
||||
Assert.Equal(CoalescingDescription, Data(s)["description"]);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProcessEvent_62407_RebootCoalescing_FiresOncePerRun()
|
||||
{
|
||||
using var tmp = new TempDirectory();
|
||||
var (tracker, sink) = MakeTracker(tmp);
|
||||
using (tracker)
|
||||
{
|
||||
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
|
||||
CoalescingDescription, ClockNow, "Microsoft-Windows-Shell-Core", isBackfill: false);
|
||||
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
|
||||
CoalescingDescription, ClockNow.AddSeconds(1), "Microsoft-Windows-Shell-Core", isBackfill: false);
|
||||
|
||||
Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProcessEvent_62407_RebootCoalescing_DoesNotTriggerEspSemantics()
|
||||
{
|
||||
// The coalescing record is corroboration only — it must not trip the ESP-exit /
|
||||
// FinalizingSetup / failure machinery.
|
||||
using var tmp = new TempDirectory();
|
||||
var (tracker, sink) = MakeTracker(tmp);
|
||||
using (tracker)
|
||||
{
|
||||
var raised = new List<string>();
|
||||
tracker.FinalizingSetupPhaseTriggered += (_, reason) => raised.Add($"finalizing:{reason}");
|
||||
tracker.EspExited += (_, _) => raised.Add("espExited");
|
||||
tracker.EspFailureDetected += (_, type) => raised.Add($"failure:{type}");
|
||||
tracker.WhiteGloveCompleted += (_, _) => raised.Add("whiteGlove");
|
||||
|
||||
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
|
||||
CoalescingDescription, ClockNow, "Microsoft-Windows-Shell-Core", isBackfill: false);
|
||||
|
||||
Assert.Empty(raised);
|
||||
Assert.False(tracker.IsEspExitedForTest);
|
||||
Assert.Empty(ByType(sink, Constants.EventTypes.EspExiting));
|
||||
Assert.Empty(ByType(sink, Constants.EventTypes.EspFailure));
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProcessEvent_62407_ExistingBranches_Unaffected()
|
||||
{
|
||||
// Ordering regression guard: WhiteGlove/failure/exiting still classify as before —
|
||||
// the RebootCoalescing check comes AFTER them.
|
||||
using var tmp = new TempDirectory();
|
||||
var (tracker, sink) = MakeTracker(tmp);
|
||||
using (tracker)
|
||||
{
|
||||
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
|
||||
"Name: 'CommercialOOBE_ESPProgress_Page_Exiting'", ClockNow, "p", isBackfill: false);
|
||||
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
|
||||
"Name: 'CommercialOOBE_ESPProgress_WhiteGlove_Success'", ClockNow, "p", isBackfill: false);
|
||||
|
||||
Assert.Single(ByType(sink, Constants.EventTypes.EspExiting));
|
||||
Assert.Single(ByType(sink, Constants.EventTypes.WhiteGloveComplete));
|
||||
Assert.Empty(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Backfill_RebootCoalescing_Emits_WithBackfilledTrue_AndHistoricalTimestamp()
|
||||
{
|
||||
// Deliberate deviation from the other 62407 backfill branches: after the coalesced
|
||||
// reboot restarts the agent, the backfill re-emission is the only reliable delivery.
|
||||
using var tmp = new TempDirectory();
|
||||
var (tracker, sink) = MakeTracker(tmp);
|
||||
using (tracker)
|
||||
{
|
||||
var historical = ClockNow.AddMinutes(-3);
|
||||
tracker.HandleBackfillRecord(
|
||||
ShellCoreTracker.EventId_ShellCore_WebAppEvent, CoalescingDescription, historical);
|
||||
|
||||
var s = Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
|
||||
Assert.Equal(historical, s.OccurredAtUtc);
|
||||
Assert.Equal(true, Data(s)["backfilled"]);
|
||||
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Backfill_RebootCoalescing_AfterLiveFire_DoesNotDoubleEmit_SameProcess()
|
||||
{
|
||||
// Same-process dedup via the fire-once flag. (Cross-process live+backfill duplication
|
||||
// is possible and accepted — exists-semantics rule, identical historical timestamp.)
|
||||
using var tmp = new TempDirectory();
|
||||
var (tracker, sink) = MakeTracker(tmp);
|
||||
using (tracker)
|
||||
{
|
||||
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
|
||||
CoalescingDescription, ClockNow, "p", isBackfill: false);
|
||||
tracker.HandleBackfillRecord(
|
||||
ShellCoreTracker.EventId_ShellCore_WebAppEvent, CoalescingDescription, ClockNow.AddMinutes(-1));
|
||||
|
||||
Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Backfill_RebootCoalescing_DoesNotFallThroughToExitOrFailureHandling()
|
||||
{
|
||||
using var tmp = new TempDirectory();
|
||||
var (tracker, sink) = MakeTracker(tmp);
|
||||
using (tracker)
|
||||
{
|
||||
var raised = new List<string>();
|
||||
tracker.FinalizingSetupPhaseTriggered += (_, reason) => raised.Add($"finalizing:{reason}");
|
||||
tracker.EspExited += (_, _) => raised.Add("espExited");
|
||||
tracker.EspFailureDetected += (_, type) => raised.Add($"failure:{type}");
|
||||
|
||||
tracker.HandleBackfillRecord(
|
||||
ShellCoreTracker.EventId_ShellCore_WebAppEvent, CoalescingDescription, ClockNow.AddMinutes(-2));
|
||||
|
||||
Assert.Empty(raised);
|
||||
Assert.False(tracker.IsEspExitedForTest);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+1
-1
@@ -256,7 +256,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
[InlineData(null, false)]
|
||||
[InlineData("", false)]
|
||||
[InlineData("0x8007", false)]
|
||||
public void IsAppxFamilyHresult_MatchesOnlyAppxDeploymentRange(string hresult, bool expected)
|
||||
public void IsAppxFamilyHresult_MatchesOnlyAppxDeploymentRange(string? hresult, bool expected)
|
||||
{
|
||||
Assert.Equal(expected, AppxFailureAnalyzer.IsAppxFamilyHresult(hresult));
|
||||
}
|
||||
|
||||
+176
-75
@@ -15,16 +15,22 @@ using Xunit;
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
{
|
||||
/// <summary>
|
||||
/// MdmRebootPolicyTracker — attributes the "unexpected reboot + second sign-in" pattern to the
|
||||
/// device-assigned policy URIs that forced the coalesced reboot (DM-Enterprise EventID 2800).
|
||||
/// Drives the primitive <see cref="MdmRebootPolicyTracker.ProcessEvent"/> test seam (no real
|
||||
/// EventRecord, which is abstract + Windows-only), mirroring the WindowsUpdateTracker pattern.
|
||||
/// MdmRebootPolicyTracker — aggregates DM-Enterprise EventID 2800 bursts into ONE neutral
|
||||
/// <c>mdm_policy_reboot_required</c> event (session b2e890c1 rework: per-record events were
|
||||
/// spam, and the raw event must not CLAIM a reboot — ANALYZE-ESP-005 does, gated on an
|
||||
/// observed system_reboot_detected). Drives the primitive ProcessEvent seam + explicit
|
||||
/// FlushPending (debounce 0 = manual-flush mode; no timers, no SerialThreading needed).
|
||||
/// The registry probe is always overridden — tests never touch the live registry.
|
||||
/// </summary>
|
||||
public sealed class MdmRebootPolicyTrackerTests : IDisposable
|
||||
{
|
||||
private static readonly DateTime At = new DateTime(2026, 7, 20, 9, 30, 0, DateTimeKind.Utc);
|
||||
private const string SampleUri = "./Device/Vendor/MSFT/Policy/Config/DeviceGuard/EnableVirtualizationBasedSecurity";
|
||||
private static readonly DateTime At = new DateTime(2026, 7, 20, 16, 20, 57, DateTimeKind.Utc);
|
||||
private const string UriSvchost = "./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation";
|
||||
private const string UriSystemGuard = "./Device/Vendor/MSFT/Policy/Config/DeviceGuard/ConfigureSystemGuardLaunch";
|
||||
|
||||
// Verified real 2800 description shape (session b2e890c1, 2026-07-20).
|
||||
private const string RealDescription =
|
||||
"The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).";
|
||||
|
||||
private readonly TempDirectory _tmp = new TempDirectory();
|
||||
private readonly FakeSignalIngressSink _sink;
|
||||
@@ -34,21 +40,28 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
public MdmRebootPolicyTrackerTests()
|
||||
{
|
||||
_sink = new FakeSignalIngressSink();
|
||||
_tracker = MakeTracker(stateDirectory: null);
|
||||
}
|
||||
|
||||
public void Dispose() => _tmp.Dispose();
|
||||
|
||||
private MdmRebootPolicyTracker MakeTracker(string? stateDirectory)
|
||||
{
|
||||
var post = new InformationalEventPost(_sink, new VirtualClock(At));
|
||||
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
|
||||
_tracker = new MdmRebootPolicyTracker(
|
||||
var tracker = new MdmRebootPolicyTracker(
|
||||
sessionId: "sess-mdm",
|
||||
tenantId: "tenant-mdm",
|
||||
post: post,
|
||||
logger: logger,
|
||||
backfillEnabled: false,
|
||||
stateDirectory: null); // in-memory dedup for most tests
|
||||
_tracker.RebootRequiredFlagProbe = () => null; // registry untouched by default
|
||||
stateDirectory: stateDirectory,
|
||||
debounceMilliseconds: 0); // manual-flush mode — tests call FlushPending explicitly
|
||||
tracker.RebootRequiredFlagProbe = () => null; // registry untouched by default
|
||||
return tracker;
|
||||
}
|
||||
|
||||
public void Dispose() => _tmp.Dispose();
|
||||
|
||||
private void Emit(string? rebootUri = SampleUri, string? description = null, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null)
|
||||
private void Buffer(string? rebootUri, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null, string? description = null)
|
||||
{
|
||||
_tracker.ProcessEvent(
|
||||
eventId: MdmRebootPolicyTracker.EventId_PolicyRebootRequired,
|
||||
@@ -70,104 +83,192 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
|
||||
|
||||
[Fact]
|
||||
public void ProcessEvent_EmitsWarning_WithUri_AndImmediateUpload()
|
||||
public void Burst_FlushesAsOneAggregatedEvent_InfoSeverity_ImmediateUpload()
|
||||
{
|
||||
Emit();
|
||||
Buffer(UriSvchost, timeCreatedUtc: At);
|
||||
Buffer(UriSystemGuard, timeCreatedUtc: At.AddMilliseconds(120));
|
||||
_tracker.FlushPending();
|
||||
|
||||
var s = Assert.Single(Emitted());
|
||||
Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]);
|
||||
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
|
||||
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
|
||||
|
||||
var data = Data(s);
|
||||
Assert.Equal(SampleUri, data["rebootUri"]);
|
||||
Assert.Equal(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, data["windowsEventId"]);
|
||||
Assert.Equal(MdmRebootPolicyTracker.Channel, data["eventLogChannel"]);
|
||||
Assert.Equal(2, data["uriCount"]);
|
||||
Assert.Equal(2, data["recordCount"]);
|
||||
Assert.Equal(UriSvchost, data["firstRebootUri"]);
|
||||
var uris = Assert.IsType<List<string>>(data["rebootUris"]);
|
||||
Assert.Equal(2, uris.Count);
|
||||
Assert.Contains(UriSvchost, uris);
|
||||
Assert.Contains(UriSystemGuard, uris);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProcessEvent_NoUri_StillEmits_WithoutRebootUriField()
|
||||
public void Message_IsNeutralObservation_NoRebootClaim()
|
||||
{
|
||||
// Tolerant contract: an unparsed URI must stay timeline-visible (with the captured
|
||||
// description telling us how to fix the extraction) — the analyze rule keys on
|
||||
// "rebootUri exists" and simply stays silent.
|
||||
Emit(rebootUri: null, description: "Some unexpected 2800 wording");
|
||||
// Session b2e890c1: flags raised during AccountSetup have no effect — the raw event
|
||||
// must state the observation only, never a future consequence.
|
||||
Buffer(UriSvchost);
|
||||
_tracker.FlushPending();
|
||||
|
||||
var s = Assert.Single(Emitted());
|
||||
var message = s.Payload![SignalPayloadKeys.Message];
|
||||
Assert.Contains("flagged", message);
|
||||
Assert.DoesNotContain("will restart", message);
|
||||
Assert.DoesNotContain("second sign-in", message);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void FlushPending_Empty_EmitsNothing()
|
||||
{
|
||||
_tracker.FlushPending();
|
||||
Assert.Empty(Emitted());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void AggregateTimestamp_IsEarliestRecordTime()
|
||||
{
|
||||
var early = At.AddMinutes(-30);
|
||||
Buffer(UriSystemGuard, timeCreatedUtc: At, isBackfill: true);
|
||||
Buffer(UriSvchost, timeCreatedUtc: early, isBackfill: true);
|
||||
_tracker.FlushPending();
|
||||
|
||||
var s = Assert.Single(Emitted());
|
||||
Assert.Equal(early, s.OccurredAtUtc);
|
||||
Assert.Equal(true, Data(s)["backfilled"]);
|
||||
// firstRebootUri follows record-time order, not buffer order.
|
||||
Assert.Equal(UriSvchost, Data(s)["firstRebootUri"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Backfilled_False_WhenAnyRecordIsLive()
|
||||
{
|
||||
Buffer(UriSvchost, isBackfill: true);
|
||||
Buffer(UriSystemGuard, isBackfill: false);
|
||||
_tracker.FlushPending();
|
||||
|
||||
Assert.Equal(false, Data(Assert.Single(Emitted()))["backfilled"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DuplicateUris_AreDistinctInAggregate()
|
||||
{
|
||||
Buffer(UriSvchost);
|
||||
Buffer(UriSvchost);
|
||||
_tracker.FlushPending();
|
||||
|
||||
var data = Data(Assert.Single(Emitted()));
|
||||
Assert.Equal(1, data["uriCount"]);
|
||||
Assert.Equal(2, data["recordCount"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void UnparsedRecords_CountedWithSampleDescription_StillEmitted()
|
||||
{
|
||||
Buffer(rebootUri: null, description: "Some unexpected 2800 wording");
|
||||
_tracker.FlushPending();
|
||||
|
||||
var s = Assert.Single(Emitted());
|
||||
var data = Data(s);
|
||||
Assert.False(data.ContainsKey("rebootUri"));
|
||||
Assert.Equal("Some unexpected 2800 wording", data["description"]);
|
||||
Assert.False(data.ContainsKey("rebootUris"));
|
||||
Assert.False(data.ContainsKey("firstRebootUri"));
|
||||
Assert.Equal(0, data["uriCount"]);
|
||||
Assert.Equal(1, data["unparsedCount"]);
|
||||
Assert.Equal("Some unexpected 2800 wording", data["sampleDescription"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BackfilledEvent_UsesEventTimeForTimelineTimestamp_AndMarksBackfilled()
|
||||
public void SameRecordId_BufferedTwice_CountsOnce()
|
||||
{
|
||||
var eventTime = At.AddMinutes(-25); // policy applied well before agent start
|
||||
Emit(isBackfill: true, timeCreatedUtc: eventTime);
|
||||
Buffer(UriSvchost, recordId: 42);
|
||||
Buffer(UriSvchost, recordId: 42);
|
||||
_tracker.FlushPending();
|
||||
|
||||
var s = Assert.Single(Emitted());
|
||||
Assert.Equal(eventTime, s.OccurredAtUtc);
|
||||
Assert.Equal(true, Data(s)["backfilled"]);
|
||||
Assert.Equal(1, Data(Assert.Single(Emitted()))["recordCount"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void SameRecordId_ProcessedTwice_EmitsOnce()
|
||||
public void RecordAfterFlush_GoesIntoNextAggregate()
|
||||
{
|
||||
Emit(recordId: 42);
|
||||
Emit(recordId: 42);
|
||||
|
||||
Assert.Single(Emitted());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BackfillEventWithLowerRecordId_AfterLiveEvent_IsStillEmitted()
|
||||
{
|
||||
// Same contract as WindowsUpdateTracker: the live watcher is armed BEFORE the backfill
|
||||
// scan, so a live record with a higher RecordId can arrive first — the older,
|
||||
// never-emitted backfill record must not be suppressed by a high-water mark.
|
||||
Emit(recordId: 100);
|
||||
Emit(recordId: 50, isBackfill: true);
|
||||
Buffer(UriSvchost, recordId: 10);
|
||||
_tracker.FlushPending();
|
||||
Buffer(UriSystemGuard, recordId: 11);
|
||||
_tracker.FlushPending();
|
||||
|
||||
Assert.Equal(2, Emitted().Count);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BackfillRecordWithLowerRecordId_AfterLiveRecord_IsStillBuffered()
|
||||
{
|
||||
// Live watcher is armed BEFORE the backfill scan — an older, never-seen backfill
|
||||
// record must not be suppressed by a high-water mark.
|
||||
Buffer(UriSvchost, recordId: 100);
|
||||
Buffer(UriSystemGuard, recordId: 50, isBackfill: true);
|
||||
_tracker.FlushPending();
|
||||
|
||||
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void NegativeRecordId_IsNeverDeduped()
|
||||
{
|
||||
Emit(recordId: -1);
|
||||
Emit(recordId: -1);
|
||||
Buffer(UriSvchost, recordId: -1);
|
||||
Buffer(UriSvchost, recordId: -1);
|
||||
_tracker.FlushPending();
|
||||
|
||||
Assert.Equal(2, Emitted().Count);
|
||||
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Watermark_PersistsAcrossTrackerInstances()
|
||||
public void Watermark_PersistedAtFlush_SkipsReReadAcrossInstances()
|
||||
{
|
||||
// The coalesced reboot restarts the agent and the post-reboot backfill re-reads the
|
||||
// same 2800 records — the persisted watermark must make that re-read a no-op.
|
||||
var post = new InformationalEventPost(_sink, new VirtualClock(At));
|
||||
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
|
||||
|
||||
var first = new MdmRebootPolicyTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
|
||||
first.RebootRequiredFlagProbe = () => null;
|
||||
// The coalesced reboot restarts the agent; the post-reboot backfill re-reads the same
|
||||
// records — a FLUSHED record must be skipped by the next instance.
|
||||
var first = MakeTracker(stateDirectory: _tmp.Path);
|
||||
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
|
||||
timeCreatedUtc: At, rebootUri: SampleUri, formattedDescription: null, isBackfill: false);
|
||||
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
|
||||
first.FlushPending();
|
||||
|
||||
var second = new MdmRebootPolicyTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
|
||||
second.RebootRequiredFlagProbe = () => null;
|
||||
var second = MakeTracker(stateDirectory: _tmp.Path);
|
||||
second.LoadWatermark();
|
||||
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
|
||||
timeCreatedUtc: At, rebootUri: SampleUri, formattedDescription: null, isBackfill: true);
|
||||
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
|
||||
second.FlushPending();
|
||||
|
||||
Assert.Single(Emitted());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Watermark_NotPersisted_WhenBufferedButNeverFlushed()
|
||||
{
|
||||
// Process killed by the very reboot the records announce: nothing was flushed, so the
|
||||
// next instance must re-read and emit the records.
|
||||
var first = MakeTracker(stateDirectory: _tmp.Path);
|
||||
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
|
||||
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
|
||||
// no FlushPending — simulated reboot kill
|
||||
|
||||
var second = MakeTracker(stateDirectory: _tmp.Path);
|
||||
second.LoadWatermark();
|
||||
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
|
||||
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
|
||||
second.FlushPending();
|
||||
|
||||
var s = Assert.Single(Emitted());
|
||||
Assert.Equal(true, Data(s)["backfilled"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RebootRequiredFlagProbe_True_AddsField_Null_OmitsIt()
|
||||
{
|
||||
_tracker.RebootRequiredFlagProbe = () => true;
|
||||
Emit(recordId: 1);
|
||||
Buffer(UriSvchost, recordId: 1);
|
||||
_tracker.FlushPending();
|
||||
|
||||
_tracker.RebootRequiredFlagProbe = () => null;
|
||||
Emit(recordId: 2);
|
||||
Buffer(UriSystemGuard, recordId: 2);
|
||||
_tracker.FlushPending();
|
||||
|
||||
var emitted = Emitted();
|
||||
Assert.Equal(2, emitted.Count);
|
||||
@@ -179,10 +280,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
public void RebootRequiredFlagProbe_Throwing_IsFailSoft()
|
||||
{
|
||||
_tracker.RebootRequiredFlagProbe = () => throw new InvalidOperationException("boom");
|
||||
Emit();
|
||||
Buffer(UriSvchost);
|
||||
_tracker.FlushPending();
|
||||
|
||||
var s = Assert.Single(Emitted());
|
||||
Assert.False(Data(s).ContainsKey("omadmRebootRequiredFlag"));
|
||||
Assert.False(Data(Assert.Single(Emitted())).ContainsKey("omadmRebootRequiredFlag"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -194,20 +295,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
// ------------------------------------------------------------------ URI extraction
|
||||
|
||||
[Fact]
|
||||
public void ExtractRebootUri_PrefersEventDataValue_OverDescription()
|
||||
public void ExtractRebootUri_FromVerifiedRealDescription()
|
||||
{
|
||||
var eventData = new Dictionary<string, string> { ["Data0"] = " " + SampleUri + " " };
|
||||
var uri = MdmRebootPolicyTracker.ExtractRebootUri(eventData, "unrelated ./Other/Uri text");
|
||||
Assert.Equal(SampleUri, uri);
|
||||
// Real 2800 text wraps the URI in parentheses and ends with a period — both must be
|
||||
// excluded from the extracted URI.
|
||||
var uri = MdmRebootPolicyTracker.ExtractRebootUri(null, RealDescription);
|
||||
Assert.Equal(UriSvchost, uri);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ExtractRebootUri_RegexFallback_FromDescription()
|
||||
public void ExtractRebootUri_PrefersEventDataValue_OverDescription()
|
||||
{
|
||||
var uri = MdmRebootPolicyTracker.ExtractRebootUri(
|
||||
new Dictionary<string, string> { ["Data0"] = "not a uri" },
|
||||
$"The following URI has triggered a reboot: {SampleUri}.");
|
||||
Assert.Equal(SampleUri, uri);
|
||||
var eventData = new Dictionary<string, string> { ["Data0"] = " " + UriSystemGuard + " " };
|
||||
var uri = MdmRebootPolicyTracker.ExtractRebootUri(eventData, RealDescription);
|
||||
Assert.Equal(UriSystemGuard, uri);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
+9
-9
@@ -39,7 +39,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
public List<EspFailureDetectedEventArgs> EspFailures { get; } = new List<EspFailureDetectedEventArgs>();
|
||||
public ProvisioningStatusTracker Tracker { get; }
|
||||
|
||||
public Fixture(Func<IReadOnlyList<AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime.AppPackageState>> packageStatesProbe = null)
|
||||
public Fixture(Func<IReadOnlyList<AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime.AppPackageState>>? packageStatesProbe = null)
|
||||
{
|
||||
var clock = new VirtualClock(Fixed);
|
||||
var post = new InformationalEventPost(Sink, clock);
|
||||
@@ -179,7 +179,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
}");
|
||||
|
||||
var captured = f.CapturedEvents();
|
||||
Assert.Single(captured.Where(e => e.EventType == "esp_failure_settle_started"));
|
||||
Assert.Single(captured, e => e.EventType == "esp_failure_settle_started");
|
||||
}
|
||||
|
||||
// =====================================================================
|
||||
@@ -214,8 +214,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
// No terminal fire — the failure was retracted.
|
||||
Assert.Empty(f.EspFailures);
|
||||
|
||||
var recovered = Assert.Single(f.CapturedEvents()
|
||||
.Where(e => e.EventType == "esp_failure_settle_recovered"));
|
||||
var recovered = Assert.Single(
|
||||
f.CapturedEvents(), e => e.EventType == "esp_failure_settle_recovered");
|
||||
Assert.Equal("AccountSetup", (string)recovered.Data["category"]);
|
||||
Assert.Equal("Apps", (string)recovered.Data["failedSubcategory"]);
|
||||
Assert.Equal("inProgress", (string)recovered.Data["observedState"]);
|
||||
@@ -280,7 +280,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
f.Tracker.TriggerSettleTimerForTest("AccountSetupCategory.Status");
|
||||
|
||||
Assert.Single(f.EspFailures);
|
||||
Assert.Empty(f.CapturedEvents().Where(e => e.EventType == "esp_failure_settle_recovered"));
|
||||
Assert.DoesNotContain(f.CapturedEvents(), e => e.EventType == "esp_failure_settle_recovered");
|
||||
}
|
||||
|
||||
// =====================================================================
|
||||
@@ -326,8 +326,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
""AppsSubcategory"": {""subcategoryState"":""failed"",""subcategoryStatusText"":""Apps (Error)""}
|
||||
}");
|
||||
|
||||
var settleStarted = Assert.Single(f.CapturedEvents()
|
||||
.Where(e => e.EventType == "esp_failure_settle_started"));
|
||||
var settleStarted = Assert.Single(
|
||||
f.CapturedEvents(), e => e.EventType == "esp_failure_settle_started");
|
||||
Assert.Equal(1, (int)settleStarted.Data["trackedAppsNotCompletedCount"]);
|
||||
var apps = (List<Dictionary<string, object>>)settleStarted.Data["trackedAppsNotCompleted"];
|
||||
var app = Assert.Single(apps);
|
||||
@@ -354,8 +354,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
""SecurityPoliciesSubcategory"": {""subcategoryState"":""failed"",""subcategoryStatusText"":""Failed""}
|
||||
}");
|
||||
|
||||
var settleStarted = Assert.Single(f.CapturedEvents()
|
||||
.Where(e => e.EventType == "esp_failure_settle_started"));
|
||||
var settleStarted = Assert.Single(
|
||||
f.CapturedEvents(), e => e.EventType == "esp_failure_settle_started");
|
||||
Assert.False(settleStarted.Data.ContainsKey("trackedAppsNotCompleted"));
|
||||
Assert.False(settleStarted.Data.ContainsKey("trackedAppsNotCompletedCount"));
|
||||
}
|
||||
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Net.Http;
|
||||
using System.Threading.Tasks;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Transport;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Transport
|
||||
{
|
||||
/// <summary>
|
||||
/// Pins the anti-flood semantics of <see cref="EmergencyReporter.TrySendAsync"/> after the
|
||||
/// delivery-hardening change (2026-07-23): the once-per-session dedup and the per-session
|
||||
/// budget count DELIVERED reports. The previous mark-before-send permanently suppressed an
|
||||
/// error key after its first transport failure — fatal for single-shot callers like the 48h
|
||||
/// emergency break, whose process exits right after the attempt.
|
||||
/// </summary>
|
||||
public class EmergencyReporterTests
|
||||
{
|
||||
private sealed class FakeApiClient : BackendApiClient
|
||||
{
|
||||
public readonly List<TimeSpan?> Timeouts = new List<TimeSpan?>();
|
||||
public readonly Queue<bool> Results = new Queue<bool>();
|
||||
|
||||
public FakeApiClient() : base(
|
||||
httpClient: new HttpClient(),
|
||||
baseUrl: "http://localhost",
|
||||
manufacturer: string.Empty,
|
||||
model: string.Empty,
|
||||
serialNumber: string.Empty,
|
||||
useBootstrapTokenAuth: false,
|
||||
bootstrapToken: null,
|
||||
agentVersion: "0.0.0",
|
||||
logger: null)
|
||||
{
|
||||
}
|
||||
|
||||
public int Calls => Timeouts.Count;
|
||||
|
||||
public override Task<bool> ReportAgentErrorAsync(AgentErrorReport report, TimeSpan? timeout = null)
|
||||
{
|
||||
Timeouts.Add(timeout);
|
||||
return Task.FromResult(Results.Count > 0 && Results.Dequeue());
|
||||
}
|
||||
}
|
||||
|
||||
private static EmergencyReporter Reporter(FakeApiClient client)
|
||||
=> new EmergencyReporter(client, "session", "tenant", "0.0.0", logger: null);
|
||||
|
||||
[Fact]
|
||||
public async Task Failed_send_rolls_back_reservation_so_a_later_call_retries()
|
||||
{
|
||||
var client = new FakeApiClient();
|
||||
client.Results.Enqueue(false); // first call fails
|
||||
client.Results.Enqueue(true); // second call succeeds
|
||||
var reporter = Reporter(client);
|
||||
|
||||
await reporter.TrySendAsync(AgentErrorType.SessionAgeEmergencyBreak, "m");
|
||||
await reporter.TrySendAsync(AgentErrorType.SessionAgeEmergencyBreak, "m");
|
||||
|
||||
// Old behaviour: second call suppressed as "already reported" ⇒ 1 call.
|
||||
Assert.Equal(2, client.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Delivered_send_is_deduplicated_per_key()
|
||||
{
|
||||
var client = new FakeApiClient();
|
||||
client.Results.Enqueue(true);
|
||||
var reporter = Reporter(client);
|
||||
|
||||
await reporter.TrySendAsync(AgentErrorType.SessionAgeEmergencyBreak, "m");
|
||||
await reporter.TrySendAsync(AgentErrorType.SessionAgeEmergencyBreak, "m");
|
||||
|
||||
Assert.Equal(1, client.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Retries_stay_inside_one_reservation_and_stop_on_success()
|
||||
{
|
||||
var client = new FakeApiClient();
|
||||
client.Results.Enqueue(false);
|
||||
client.Results.Enqueue(true); // second attempt delivers — third must not fire
|
||||
var reporter = Reporter(client);
|
||||
|
||||
await reporter.TrySendAsync(
|
||||
AgentErrorType.SessionAgeEmergencyBreak, "m",
|
||||
attempts: 3, perAttemptTimeout: TimeSpan.FromMilliseconds(50), retryDelay: TimeSpan.Zero);
|
||||
|
||||
Assert.Equal(2, client.Calls);
|
||||
|
||||
// Delivered ⇒ dedup holds for the session.
|
||||
await reporter.TrySendAsync(AgentErrorType.SessionAgeEmergencyBreak, "m");
|
||||
Assert.Equal(2, client.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Per_attempt_timeout_is_forwarded_to_the_api_client()
|
||||
{
|
||||
var client = new FakeApiClient();
|
||||
client.Results.Enqueue(true);
|
||||
var reporter = Reporter(client);
|
||||
|
||||
var timeout = TimeSpan.FromSeconds(15);
|
||||
await reporter.TrySendAsync(
|
||||
AgentErrorType.SessionAgeEmergencyBreak, "m",
|
||||
attempts: 1, perAttemptTimeout: timeout);
|
||||
|
||||
Assert.Equal(timeout, Assert.Single(client.Timeouts));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task All_attempts_failing_rolls_back_after_exhausting_attempts()
|
||||
{
|
||||
var client = new FakeApiClient(); // queue empty ⇒ every attempt fails
|
||||
var reporter = Reporter(client);
|
||||
|
||||
await reporter.TrySendAsync(
|
||||
AgentErrorType.SessionAgeEmergencyBreak, "m",
|
||||
attempts: 3, retryDelay: TimeSpan.Zero);
|
||||
Assert.Equal(3, client.Calls);
|
||||
|
||||
// Rolled back ⇒ a later call may try again (budget + dedup + cooldown restored).
|
||||
client.Results.Enqueue(true);
|
||||
await reporter.TrySendAsync(AgentErrorType.SessionAgeEmergencyBreak, "m");
|
||||
Assert.Equal(4, client.Calls);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Different_keys_do_not_suppress_each_other_after_rollback()
|
||||
{
|
||||
var client = new FakeApiClient();
|
||||
client.Results.Enqueue(false); // break report fails → rollback (incl. cooldown restore)
|
||||
client.Results.Enqueue(true); // unrelated report must still go out
|
||||
var reporter = Reporter(client);
|
||||
|
||||
await reporter.TrySendAsync(AgentErrorType.SessionAgeEmergencyBreak, "m");
|
||||
await reporter.TrySendAsync(AgentErrorType.ConfigFetchFailed, "m");
|
||||
|
||||
Assert.Equal(2, client.Calls);
|
||||
}
|
||||
}
|
||||
}
|
||||
+15
-6
@@ -44,7 +44,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
|
||||
private readonly FakeQuarantineSink _quarantine = new FakeQuarantineSink();
|
||||
private readonly FakeSignalIngressSink _sink = new FakeSignalIngressSink();
|
||||
private readonly DecisionEngine _engine = new DecisionEngine();
|
||||
private DecisionStepProcessor _sut;
|
||||
private DecisionStepProcessor? _sut;
|
||||
|
||||
public DecisionStepProcessorParkedTripwireTests()
|
||||
{
|
||||
@@ -107,7 +107,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
|
||||
return (step, signal);
|
||||
}
|
||||
|
||||
private FakeSignalIngressSink.PostedSignal FindTripwirePost()
|
||||
private FakeSignalIngressSink.PostedSignal? FindTripwirePost()
|
||||
{
|
||||
foreach (var posted in _sink.Posted)
|
||||
{
|
||||
@@ -121,6 +121,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
|
||||
return null;
|
||||
}
|
||||
|
||||
/// <summary>The tripwire post, asserted present — for the assertions that only run
|
||||
/// after <c>WaitForTripwire()</c> has already confirmed one was emitted.</summary>
|
||||
private FakeSignalIngressSink.PostedSignal RequireTripwirePost()
|
||||
{
|
||||
var post = FindTripwirePost();
|
||||
Assert.NotNull(post);
|
||||
return post!;
|
||||
}
|
||||
|
||||
private int CountTripwirePosts()
|
||||
{
|
||||
var count = 0;
|
||||
@@ -165,9 +174,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
|
||||
Thread.Sleep(SuppressWait);
|
||||
Assert.Equal(1, CountTripwirePosts());
|
||||
|
||||
var post = FindTripwirePost();
|
||||
var post = RequireTripwirePost();
|
||||
Assert.Equal(DecisionSignalKind.InformationalEvent, post.Kind);
|
||||
Assert.Equal(nameof(SessionStage.EspAccountSetup), post.Payload["stage"]);
|
||||
Assert.Equal(nameof(SessionStage.EspAccountSetup), post.Payload!["stage"]);
|
||||
Assert.Equal("Warning", post.Payload[SignalPayloadKeys.Severity]);
|
||||
Assert.Equal("true", post.Payload[SignalPayloadKeys.ImmediateUpload]);
|
||||
Assert.Contains("esp_final_exit", post.Payload["signalsSeen"]);
|
||||
@@ -222,7 +231,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
|
||||
sut.ApplyStep(step, signal);
|
||||
|
||||
Assert.True(WaitForTripwire());
|
||||
Assert.Equal(DeadlineNames.ClassifierTick, FindTripwirePost().Payload["armedDeadlines"]);
|
||||
Assert.Equal(DeadlineNames.ClassifierTick, RequireTripwirePost().Payload!["armedDeadlines"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -237,7 +246,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
|
||||
sut.ApplyStep(step, signal);
|
||||
|
||||
Assert.True(WaitForTripwire());
|
||||
Assert.Equal(nameof(SessionStage.AwaitingDesktop), FindTripwirePost().Payload["stage"]);
|
||||
Assert.Equal(nameof(SessionStage.AwaitingDesktop), RequireTripwirePost().Payload!["stage"]);
|
||||
}
|
||||
|
||||
// ===================================================================== Dwell semantics
|
||||
|
||||
+2
-2
@@ -256,8 +256,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Program
|
||||
RemoteConfigFetchOutcome outcome,
|
||||
int configVersion = 0,
|
||||
int attempts = 1,
|
||||
string failureType = null,
|
||||
string failureMessage = null,
|
||||
string? failureType = null,
|
||||
string? failureMessage = null,
|
||||
int? authStatusCode = null)
|
||||
{
|
||||
// CurrentConfig is set via the protected SetConfig path — easiest:
|
||||
|
||||
+1
-1
@@ -33,7 +33,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.SignalAdapters
|
||||
[InlineData("Network connections (1 of 1 added)", null)]
|
||||
[InlineData("", null)]
|
||||
[InlineData(null, null)]
|
||||
public void TryExtractErrorCode_extracts_parenthesised_hex_tail(string statusText, string expected)
|
||||
public void TryExtractErrorCode_extracts_parenthesised_hex_tail(string? statusText, string? expected)
|
||||
{
|
||||
var actual = ProvisioningStatusTracker.TryExtractErrorCode(statusText);
|
||||
Assert.Equal(expected, actual);
|
||||
|
||||
+1
-1
@@ -195,7 +195,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.SignalAdapters
|
||||
var all = f.InfoEvents(SharedEventTypes.ScriptCompleted);
|
||||
Assert.Equal(2, all.Count);
|
||||
Assert.Equal("65.00", all[1].Payload!["durationSeconds"]);
|
||||
Assert.Equal("cycle_including_reporting_latency", all[1].Payload["durationBasis"]);
|
||||
Assert.Equal("cycle_including_reporting_latency", all[1].Payload!["durationBasis"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
+8
-2
@@ -151,17 +151,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.SignalAdapters
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void TriggerPhaseChangedFromTest_emits_informational_only_no_decision_signal()
|
||||
public void TriggerPhaseChangedFromTest_emits_typed_signal_and_informational_event()
|
||||
{
|
||||
using var f = new Fixture();
|
||||
using var adapter = new RealmJoinWatcherAdapter(f.Watcher, f.Ingress, f.Clock);
|
||||
|
||||
adapter.TriggerPhaseChangedFromTest(prev: 100, curr: 200);
|
||||
|
||||
// Phase change is observability-only. There is no DecisionSignalKind.RealmJoinPhaseChanged.
|
||||
// 1) Typed signal — persists LastDeploymentPhase and drives the aborted-RJ-ESP
|
||||
// detection in the reducer (session 224b2087). Not Detected/Resolved.
|
||||
Assert.DoesNotContain(f.Ingress.Posted, p => p.Kind == DecisionSignalKind.RealmJoinDetected);
|
||||
Assert.DoesNotContain(f.Ingress.Posted, p => p.Kind == DecisionSignalKind.RealmJoinResolved);
|
||||
var decision = f.Ingress.Posted.Single(p => p.Kind == DecisionSignalKind.RealmJoinPhaseChanged);
|
||||
Assert.Equal("RealmJoinWatcher", decision.SourceOrigin);
|
||||
Assert.Equal("200", decision.Payload![DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase]);
|
||||
Assert.Equal("100", decision.Payload[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase]);
|
||||
|
||||
// 2) Dual-emit informational event keeps the existing timeline entry.
|
||||
var info = f.Ingress.Posted.Single(p =>
|
||||
p.Kind == DecisionSignalKind.InformationalEvent
|
||||
&& p.Payload != null
|
||||
|
||||
+12
-12
@@ -1,4 +1,4 @@
|
||||
using System;
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic;
|
||||
using Xunit;
|
||||
@@ -19,7 +19,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry
|
||||
private static List<BandwidthJobSample> Jobs(params BandwidthJobSample[] jobs)
|
||||
=> new List<BandwidthJobSample>(jobs);
|
||||
|
||||
private static BandwidthJobSample Job(string fileId, long wan, long lan = 0)
|
||||
private static BandwidthJobSample Job(string? fileId, long wan, long lan = 0)
|
||||
=> new BandwidthJobSample { FileId = fileId, WanBytes = wan, LanBytes = lan };
|
||||
|
||||
[Fact]
|
||||
@@ -43,8 +43,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry
|
||||
var estimate = estimator.TryBuildEstimate();
|
||||
Assert.NotNull(estimate);
|
||||
Assert.Equal(1, estimate.WanSampleCount);
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3);
|
||||
Assert.Equal(16.0, estimate.WanMbpsMax.Value, 3);
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90!.Value, 3);
|
||||
Assert.Equal(16.0, estimate.WanMbpsMax!.Value, 3);
|
||||
Assert.Equal(6_000_000, estimate.WanBytesObserved);
|
||||
Assert.Equal("10-50", estimate.Bucket);
|
||||
}
|
||||
@@ -58,7 +58,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry
|
||||
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", 3_000_000), Job("b", 3_000_000)));
|
||||
|
||||
var estimate = estimator.TryBuildEstimate();
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3);
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90!.Value, 3);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -74,7 +74,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry
|
||||
|
||||
var estimate = estimator.TryBuildEstimate();
|
||||
Assert.Equal(2, estimate.WanSampleCount);
|
||||
Assert.Equal(16.0, estimate.WanMbpsMax.Value, 3);
|
||||
Assert.Equal(16.0, estimate.WanMbpsMax!.Value, 3);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -91,7 +91,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry
|
||||
estimator.AddSnapshot(T0.AddSeconds(63), Jobs(Job("a", 106_000_000)));
|
||||
var estimate = estimator.TryBuildEstimate();
|
||||
Assert.Equal(1, estimate.WanSampleCount);
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3);
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90!.Value, 3);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -130,8 +130,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry
|
||||
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", wan: 6_000_000, lan: 375_000_000)));
|
||||
|
||||
var estimate = estimator.TryBuildEstimate();
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3);
|
||||
Assert.Equal(1000.0, estimate.LanMbpsP90.Value, 3);
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90!.Value, 3);
|
||||
Assert.Equal(1000.0, estimate.LanMbpsP90!.Value, 3);
|
||||
Assert.Equal("10-50", estimate.Bucket); // bucket keyed to WAN, not LAN
|
||||
}
|
||||
|
||||
@@ -182,8 +182,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry
|
||||
|
||||
var estimate = estimator.TryBuildEstimate();
|
||||
Assert.Equal(20, estimate.WanSampleCount);
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3); // p90 stays on the steady plateau
|
||||
Assert.Equal(160.0, estimate.WanMbpsMax.Value, 3);
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90!.Value, 3); // p90 stays on the steady plateau
|
||||
Assert.Equal(160.0, estimate.WanMbpsMax!.Value, 3);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -265,7 +265,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry
|
||||
|
||||
var estimate = estimator.TryBuildEstimate();
|
||||
Assert.Equal(1, estimate.WanSampleCount); // only the finite positive sample survived
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3);
|
||||
Assert.Equal(16.0, estimate.WanMbpsP90!.Value, 3);
|
||||
Assert.Equal(0, estimate.WanBytesObserved);
|
||||
}
|
||||
|
||||
|
||||
+96
-2
@@ -17,13 +17,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry.Events
|
||||
FakeTelemetryTransport transport,
|
||||
out EventSequenceCounter counter,
|
||||
string sessionId = "sess-1",
|
||||
string tenantId = "tenant-1")
|
||||
string tenantId = "tenant-1",
|
||||
Func<bool>? traceEventsEnabled = null)
|
||||
{
|
||||
var tmp = new TempDirectory();
|
||||
counter = new EventSequenceCounter(new EventSequencePersistence(tmp.File("seq.json")));
|
||||
return new TelemetryEventEmitter(transport, counter, sessionId, tenantId);
|
||||
return new TelemetryEventEmitter(transport, counter, sessionId, tenantId, traceEventsEnabled);
|
||||
}
|
||||
|
||||
private static EnrollmentEvent NewTraceEvent(string eventType = "outbound_ip") =>
|
||||
new EnrollmentEvent
|
||||
{
|
||||
EventType = eventType,
|
||||
Severity = EventSeverity.Trace,
|
||||
Source = "test",
|
||||
Timestamp = At,
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Message = "trace message",
|
||||
};
|
||||
|
||||
private static EnrollmentEvent NewTestEvent(string eventType = "enrollment_complete") =>
|
||||
new EnrollmentEvent
|
||||
{
|
||||
@@ -49,6 +61,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry.Events
|
||||
Assert.Equal("sess-1", evt.SessionId);
|
||||
Assert.Equal("tenant-1", evt.TenantId);
|
||||
|
||||
Assert.NotNull(item);
|
||||
Assert.Equal(TelemetryItemKind.Event, item.Kind);
|
||||
Assert.Equal("tenant-1_sess-1", item.PartitionKey);
|
||||
Assert.Equal(evt.RowKey, item.RowKey);
|
||||
@@ -174,5 +187,86 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry.Events
|
||||
Assert.Equal("test-reason", (string?)data["reason"]);
|
||||
Assert.Equal("Strong", (string?)data["wgConfidence"]);
|
||||
}
|
||||
|
||||
// ============================================================ Trace-event gate
|
||||
|
||||
[Fact]
|
||||
public void Trace_event_is_dropped_when_gate_is_off()
|
||||
{
|
||||
var transport = new FakeTelemetryTransport();
|
||||
var emitter = Build(transport, out _, traceEventsEnabled: () => false);
|
||||
|
||||
var item = emitter.Emit(NewTraceEvent());
|
||||
|
||||
Assert.Null(item);
|
||||
Assert.Equal(0, transport.EnqueueCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Trace_event_flows_when_gate_is_on()
|
||||
{
|
||||
var transport = new FakeTelemetryTransport();
|
||||
var emitter = Build(transport, out _, traceEventsEnabled: () => true);
|
||||
|
||||
Assert.NotNull(emitter.Emit(NewTraceEvent()));
|
||||
Assert.Equal(1, transport.EnqueueCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Trace_event_flows_when_no_gate_is_wired()
|
||||
{
|
||||
var transport = new FakeTelemetryTransport();
|
||||
var emitter = Build(transport, out _);
|
||||
|
||||
Assert.NotNull(emitter.Emit(NewTraceEvent()));
|
||||
Assert.Equal(1, transport.EnqueueCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Gate_off_does_not_suppress_non_trace_events()
|
||||
{
|
||||
var transport = new FakeTelemetryTransport();
|
||||
var emitter = Build(transport, out _, traceEventsEnabled: () => false);
|
||||
|
||||
foreach (var severity in new[] { EventSeverity.Info, EventSeverity.Warning, EventSeverity.Error })
|
||||
{
|
||||
var evt = NewTestEvent();
|
||||
evt.Severity = severity;
|
||||
Assert.NotNull(emitter.Emit(evt));
|
||||
}
|
||||
|
||||
Assert.Equal(3, transport.EnqueueCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Dropped_trace_event_does_not_consume_a_sequence_number()
|
||||
{
|
||||
var transport = new FakeTelemetryTransport();
|
||||
var traceAllowed = true;
|
||||
// ReSharper disable once AccessToModifiedClosure — deliberate: the gate is read
|
||||
// per event, mirroring a mid-session remote-config merge.
|
||||
var emitter = Build(transport, out _, traceEventsEnabled: () => traceAllowed);
|
||||
|
||||
emitter.Emit(NewTestEvent()); // seq 1
|
||||
traceAllowed = false;
|
||||
emitter.Emit(NewTraceEvent()); // dropped — must not burn seq 2
|
||||
emitter.Emit(NewTestEvent()); // seq 2
|
||||
|
||||
Assert.Equal(2, transport.EnqueueCount);
|
||||
var first = JObject.Parse(transport.Enqueued[0].PayloadJson);
|
||||
var second = JObject.Parse(transport.Enqueued[1].PayloadJson);
|
||||
Assert.Equal(1, (long)first["Sequence"]!);
|
||||
Assert.Equal(2, (long)second["Sequence"]!);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Throwing_gate_fails_open_so_diagnostics_are_never_silently_lost()
|
||||
{
|
||||
var transport = new FakeTelemetryTransport();
|
||||
var emitter = Build(transport, out _, traceEventsEnabled: () => throw new InvalidOperationException("boom"));
|
||||
|
||||
Assert.NotNull(emitter.Emit(NewTraceEvent()));
|
||||
Assert.Equal(1, transport.EnqueueCount);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+75
@@ -61,6 +61,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
internal Func<int, string> SessionOwnerResolver { get; set; }
|
||||
internal Func<int, string> ProcessOwnerResolver { get; set; }
|
||||
|
||||
// Test seam for the WinRT OOBE-state sample (null = OobeStateReader.Read).
|
||||
internal Func<string> OobeStateProvider { get; set; }
|
||||
|
||||
// OOBE-state flip tracking (observational only — see OobeStateReader contract).
|
||||
// _oobeCompletedEmitted is one-shot per agent lifetime and deliberately NOT reset
|
||||
// by ResetForRealUserSwitch: the OOBE flip is a global OS transition, not per-user.
|
||||
private string _lastOobeState;
|
||||
private bool _oobeCompletedEmitted;
|
||||
|
||||
/// <summary>
|
||||
/// System/service account names that should NOT be considered real users.
|
||||
/// </summary>
|
||||
@@ -161,6 +170,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
{
|
||||
_startedAtUtc = DateTime.UtcNow;
|
||||
|
||||
// Baseline OOBE-state sample so the first poll can already detect a flip
|
||||
// (self-contained try/catch inside, like the Emit* helpers).
|
||||
CheckOobeStateFlip();
|
||||
|
||||
_pollingTimer?.Dispose();
|
||||
_pollingTimer = new Timer(
|
||||
PollForDesktop,
|
||||
@@ -186,6 +199,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
|
||||
_pollCount++;
|
||||
|
||||
CheckOobeStateFlip();
|
||||
|
||||
try
|
||||
{
|
||||
var explorerProcesses = Process.GetProcessesByName("explorer");
|
||||
@@ -381,6 +396,66 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
catch (Exception ex) { _logger.Verbose($"DesktopArrivalDetector: detector-started emit failed: {ex.Message}"); }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Samples the WinRT OOBE state (<see cref="OobeStateReader"/>) and emits the one-shot
|
||||
/// <c>oobe_state_completed</c> trace event on the in_progress->completed flip.
|
||||
/// Observational only — an owner-independent desktop corroboration for sessions where
|
||||
/// WTS+WMI owner resolution starves and <c>desktop_arrived</c> never fires. Often absent
|
||||
/// in healthy sessions: the polling timer is disposed at desktop arrival, which is when
|
||||
/// the flip lands (empirical probe, session 9c404ae9).
|
||||
/// <para>
|
||||
/// Self-contained try/catch: a throw here must NOT reach <see cref="PollForDesktop"/>'s
|
||||
/// outer catch, which would inflate <see cref="_wmiErrorCountSinceStart"/> and skew the
|
||||
/// session-4d5a0b78 liveness diagnostics. "unavailable" readings never overwrite the last
|
||||
/// known state, so a transient reflection failure between two polls cannot mask the flip.
|
||||
/// One-shot flag survives <see cref="ResetForRealUserSwitch"/> — the flip is a global OS
|
||||
/// transition, not per-user.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
private void CheckOobeStateFlip()
|
||||
{
|
||||
try
|
||||
{
|
||||
var current = (OobeStateProvider ?? OobeStateReader.Read)();
|
||||
if (string.IsNullOrEmpty(current) || current == OobeStateReader.Unavailable)
|
||||
return;
|
||||
|
||||
if (!_oobeCompletedEmitted
|
||||
&& _lastOobeState == "in_progress"
|
||||
&& current == "completed")
|
||||
{
|
||||
_oobeCompletedEmitted = true;
|
||||
var minutesSinceStart = Math.Round((DateTime.UtcNow - _startedAtUtc).TotalMinutes, 1);
|
||||
_logger.Info("DesktopArrivalDetector: OOBE state flipped in_progress->completed (WinRT SystemSetupInfo)");
|
||||
OnTraceEvent?.Invoke(
|
||||
SharedEventTypes.OobeStateCompleted,
|
||||
$"OOBE state flipped in_progress->completed (WinRT SystemSetupInfo, poll {_pollCount}) — owner-independent desktop corroboration.",
|
||||
new Dictionary<string, object>
|
||||
{
|
||||
{ "previousState", "in_progress" },
|
||||
{ "pollCount", _pollCount },
|
||||
{ "minutesSinceStart", minutesSinceStart },
|
||||
{ "desktopArrived", _desktopArrived },
|
||||
{ "wmiErrorCount", _wmiErrorCountSinceStart },
|
||||
{ "wtsErrorCount", _wtsErrorCountSinceStart },
|
||||
});
|
||||
}
|
||||
|
||||
_lastOobeState = current;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Verbose($"DesktopArrivalDetector: OOBE state check failed: {ex.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Drives a single synchronous poll for unit tests — the production poll is private and
|
||||
/// timer-driven with a fixed 30 s interval (precedent: <see cref="ResolveOwner"/> is
|
||||
/// likewise internal for direct testing).
|
||||
/// </summary>
|
||||
internal void PollOnceForTest() => PollForDesktop(null);
|
||||
|
||||
/// <summary>
|
||||
/// Resolves the owning user of an explorer.exe candidate. Session 4d5a0b78 fix
|
||||
/// (2026-06-11): the WMI <c>Win32_Process.GetOwner</c> call failed on every poll on
|
||||
|
||||
+200
-91
@@ -4,6 +4,7 @@ using System.Diagnostics.Eventing.Reader;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Text.RegularExpressions;
|
||||
using System.Threading;
|
||||
using System.Xml.Linq;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Orchestration;
|
||||
@@ -15,31 +16,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
{
|
||||
/// <summary>
|
||||
/// Watches <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> for
|
||||
/// EventID 2800 — logged once per device-assigned MDM policy whose CSP URI matches the OS
|
||||
/// reboot-required catalog — and forwards each as a <c>mdm_policy_reboot_required</c> event
|
||||
/// carrying the reboot-forcing URI.
|
||||
/// EventID 2800 — logged once per MDM policy URI that matches the OS reboot-required catalog —
|
||||
/// and emits ONE aggregated <c>mdm_policy_reboot_required</c> event per burst carrying the URI
|
||||
/// list (<c>rebootUris</c>, <c>uriCount</c>, <c>firstRebootUri</c>).
|
||||
/// <para>
|
||||
/// This attributes the "unexpected reboot + second sign-in screen" pattern (PMPC research):
|
||||
/// device-assigned policies applied during ESP DeviceSetup can demand a reboot; Windows
|
||||
/// coalesces them into one restart at the end of DeviceSetup, forcing the user through a second
|
||||
/// sign-in before AccountSetup. Admins are blind to this in the Intune console — the attributed
|
||||
/// URIs make it actionable (reassign the profiles to user groups and the reboot disappears).
|
||||
/// Verified against session b2e890c1 (2026-07-20): the 2800 description reads
|
||||
/// <c>The following URI has triggered a reboot: (./Device/...).</c> and records arrive in
|
||||
/// sub-second bursts during policy sync — hence the debounce aggregation instead of per-record
|
||||
/// events. CRITICAL semantics: only URIs applied DURING ESP DeviceSetup cause the coalesced
|
||||
/// mid-ESP reboot ("second sign-in", PMPC research); later syncs (AccountSetup and beyond)
|
||||
/// merely flag the requirement with no forced reboot. The event is therefore a NEUTRAL
|
||||
/// observation (Info) — the reboot claim lives in ANALYZE-ESP-005, which requires an
|
||||
/// actually-observed <c>system_reboot_detected</c> before attributing anything.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Backfill.</b> Device policies can apply before the agent starts, so a startup backfill
|
||||
/// scans the lookback window. The .evtx persists across the coalesced reboot, so the
|
||||
/// post-reboot agent restart re-reads the same records — which is why the
|
||||
/// <b>cross-restart RecordId watermark</b> (WindowsUpdateTracker pattern) is mandatory here:
|
||||
/// without it every reboot-forcing URI would be re-emitted after the very reboot it caused.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Tolerant matching.</b> The exact 2800 message text is unverified (validated post-deploy on
|
||||
/// the E2E VM): emission is gated on the EventID only, never on message wording. URI extraction
|
||||
/// prefers structured EventData values that look like an OMA-DM URI (<c>./…</c>) and falls back
|
||||
/// to a permissive regex over the formatted description. Events with an unparseable URI are
|
||||
/// still emitted (with the captured description) so the timeline shows them and the captured
|
||||
/// text tells us how to fix the extraction — the analyze rule keys on <c>rebootUri exists</c>
|
||||
/// and simply stays silent for those.
|
||||
/// <b>Backfill + watermark.</b> Device policies can apply before the agent starts, and a
|
||||
/// genuine coalesced reboot kills the agent BEFORE the debounced emit — in both cases the
|
||||
/// post-(re)start backfill re-reads the .evtx and emits the aggregate then (historical
|
||||
/// timestamps). The cross-restart RecordId watermark is persisted only at successful flush, so
|
||||
/// a pre-emit death loses nothing and a completed emit is never repeated.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal sealed class MdmRebootPolicyTracker : IDisposable
|
||||
@@ -49,12 +44,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
|
||||
internal const string WatermarkStateFileName = "mdm-reboot-watermark.json";
|
||||
|
||||
/// <summary>Cap for the emitted <c>rebootUris</c> list — <c>uriCount</c> always carries the uncapped distinct total.</summary>
|
||||
internal const int MaxUrisPerEvent = 20;
|
||||
|
||||
/// <summary>Default quiet-period before a burst of 2800 records is flushed as one event.</summary>
|
||||
internal const int DefaultDebounceMilliseconds = 10000;
|
||||
|
||||
/// <summary>Registry key whose <c>RebootRequired</c> value/subkey flags a pending OMA-DM coalesced reboot.</summary>
|
||||
internal const string OmadmSyncMlKeyPath = @"SOFTWARE\Microsoft\Provisioning\OMADM\SyncML";
|
||||
|
||||
// Permissive OMA-DM URI shape: "./Device/Vendor/MSFT/...", "./User/...", "./Vendor/...".
|
||||
// Kept deliberately broad — the exact 2800 text is unverified; post-deploy E2E validates
|
||||
// and tightens this against real records if needed.
|
||||
// Permissive OMA-DM URI shape ("./Device/Vendor/MSFT/...", "./User/..."). The verified 2800
|
||||
// text wraps the URI in parentheses — ')' is deliberately outside the class so the match
|
||||
// stops cleanly; trailing sentence punctuation is trimmed after the match.
|
||||
private static readonly Regex UriPattern = new Regex(
|
||||
@"\./[A-Za-z0-9_\-./\[\]{}%~+]+", RegexOptions.Compiled);
|
||||
|
||||
@@ -65,16 +66,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
private readonly bool _backfillEnabled;
|
||||
private readonly int _backfillLookbackMinutes;
|
||||
private readonly string _stateDirectory;
|
||||
private readonly int _debounceMilliseconds;
|
||||
|
||||
private EventLogWatcher _watcher;
|
||||
private Timer _debounceTimer;
|
||||
|
||||
// Dedup — same contract as WindowsUpdateTracker (see its field docs): immutable
|
||||
// cross-restart boundary from disk + intra-run HashSet (NOT a high-water mark, because the
|
||||
// live watcher is armed before the backfill scan and can deliver higher RecordIds first).
|
||||
private sealed class PendingRecord
|
||||
{
|
||||
public long RecordId;
|
||||
public DateTime TimeUtc;
|
||||
public string Uri; // null when extraction failed
|
||||
public string Description; // may be null
|
||||
public bool IsBackfill;
|
||||
}
|
||||
|
||||
// Pending burst buffer + dedup state, all guarded by _stateLock. Same claim contract as
|
||||
// WindowsUpdateTracker (immutable cross-restart boundary + intra-run HashSet), with ONE
|
||||
// deliberate difference: the watermark is persisted at FLUSH time, not at claim time —
|
||||
// records buffered but never flushed (process killed by the very reboot they announce)
|
||||
// must be re-read and emitted by the post-restart backfill.
|
||||
private readonly List<PendingRecord> _pending = new List<PendingRecord>();
|
||||
private long _restartWatermark = -1;
|
||||
private readonly HashSet<long> _seenThisRun = new HashSet<long>();
|
||||
private long _maxEmittedRecordId = -1;
|
||||
private readonly object _watermarkLock = new object();
|
||||
private long _maxPersistedRecordId = -1;
|
||||
private readonly object _stateLock = new object();
|
||||
|
||||
/// <summary>
|
||||
/// Fail-soft corroboration probe for the OMA-DM RebootRequired flag — null = unknown
|
||||
@@ -89,7 +104,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
AgentLogger logger,
|
||||
bool backfillEnabled = true,
|
||||
int backfillLookbackMinutes = 60,
|
||||
string stateDirectory = null)
|
||||
string stateDirectory = null,
|
||||
int debounceMilliseconds = DefaultDebounceMilliseconds)
|
||||
{
|
||||
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
|
||||
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
|
||||
@@ -98,6 +114,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
_backfillEnabled = backfillEnabled;
|
||||
_backfillLookbackMinutes = backfillLookbackMinutes;
|
||||
_stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null;
|
||||
_debounceMilliseconds = debounceMilliseconds;
|
||||
}
|
||||
|
||||
public void Start()
|
||||
@@ -106,13 +123,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
StartWatcher();
|
||||
|
||||
if (_backfillEnabled && _backfillLookbackMinutes > 0)
|
||||
{
|
||||
BackfillRecentEvents();
|
||||
// The backfill batch is complete — flush immediately instead of waiting out the
|
||||
// debounce (live records that raced in are simply included).
|
||||
FlushPending();
|
||||
}
|
||||
else
|
||||
{
|
||||
_logger.Info("MdmRebootPolicy backfill disabled by config");
|
||||
}
|
||||
}
|
||||
|
||||
public void Stop()
|
||||
{
|
||||
// Flush any un-emitted burst before tearing down (normal agent shutdown path).
|
||||
try { FlushPending(); }
|
||||
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy flush on stop failed: {ex.Message}"); }
|
||||
|
||||
var timer = Interlocked.Exchange(ref _debounceTimer, null);
|
||||
if (timer != null)
|
||||
{
|
||||
try { timer.Dispose(); } catch { }
|
||||
}
|
||||
|
||||
if (_watcher == null) return;
|
||||
try
|
||||
{
|
||||
@@ -168,7 +202,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
$"*[System[(EventID={EventId_PolicyRebootRequired})]]";
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Event processing
|
||||
// Event processing — buffer + debounce
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private void OnEventRecordWritten(object sender, EventRecordWrittenEventArgs e)
|
||||
@@ -212,9 +246,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Core processing extracted to primitive inputs so tests can drive it without synthesizing
|
||||
/// an abstract, Windows-only <see cref="EventRecord"/>. Mirrors the WindowsUpdateTracker
|
||||
/// test-seam pattern.
|
||||
/// Buffers one 2800 record into the pending burst (primitive-input test seam, mirroring the
|
||||
/// WindowsUpdateTracker pattern). The aggregated event is emitted by <see cref="FlushPending"/>
|
||||
/// — after the debounce quiet-period, at backfill completion, or at <see cref="Stop"/>.
|
||||
/// </summary>
|
||||
internal void ProcessEvent(
|
||||
int eventId,
|
||||
@@ -224,53 +258,155 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
string formattedDescription,
|
||||
bool isBackfill)
|
||||
{
|
||||
if (!MarkProcessed(recordId))
|
||||
return; // already emitted (cross-restart or duplicate delivery)
|
||||
lock (_stateLock)
|
||||
{
|
||||
if (recordId >= 0)
|
||||
{
|
||||
// Already emitted in a prior run, or already buffered/emitted this run.
|
||||
if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId))
|
||||
return;
|
||||
}
|
||||
|
||||
_pending.Add(new PendingRecord
|
||||
{
|
||||
RecordId = recordId,
|
||||
TimeUtc = timeCreatedUtc ?? DateTime.UtcNow,
|
||||
Uri = string.IsNullOrEmpty(rebootUri) ? null : rebootUri,
|
||||
Description = formattedDescription,
|
||||
IsBackfill = isBackfill,
|
||||
});
|
||||
}
|
||||
|
||||
ArmDebounce();
|
||||
}
|
||||
|
||||
private void ArmDebounce()
|
||||
{
|
||||
if (_debounceMilliseconds <= 0) return; // manual-flush mode (tests; backfill/Stop flush explicitly)
|
||||
|
||||
var timer = _debounceTimer;
|
||||
if (timer == null)
|
||||
{
|
||||
var created = new Timer(_ =>
|
||||
{
|
||||
try { FlushPending(); }
|
||||
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy debounce flush failed: {ex.Message}"); }
|
||||
}, null, Timeout.Infinite, Timeout.Infinite);
|
||||
|
||||
timer = Interlocked.CompareExchange(ref _debounceTimer, created, null) ?? created;
|
||||
if (!ReferenceEquals(timer, created))
|
||||
{
|
||||
try { created.Dispose(); } catch { }
|
||||
}
|
||||
}
|
||||
|
||||
try { timer.Change(_debounceMilliseconds, Timeout.Infinite); }
|
||||
catch (ObjectDisposedException) { /* raced Stop() — Stop's flush covers the buffer */ }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Emits the pending burst as ONE aggregated <c>mdm_policy_reboot_required</c> event and
|
||||
/// persists the watermark. No-op when nothing is pending. Internal for tests.
|
||||
/// </summary>
|
||||
internal void FlushPending()
|
||||
{
|
||||
List<PendingRecord> batch;
|
||||
long watermarkToPersist = -1;
|
||||
|
||||
lock (_stateLock)
|
||||
{
|
||||
if (_pending.Count == 0) return;
|
||||
batch = new List<PendingRecord>(_pending);
|
||||
_pending.Clear();
|
||||
|
||||
foreach (var r in batch)
|
||||
{
|
||||
if (r.RecordId > _maxPersistedRecordId)
|
||||
_maxPersistedRecordId = r.RecordId;
|
||||
}
|
||||
watermarkToPersist = _maxPersistedRecordId;
|
||||
}
|
||||
|
||||
if (watermarkToPersist >= 0) PersistWatermark(watermarkToPersist);
|
||||
|
||||
batch.Sort((a, b) => a.TimeUtc.CompareTo(b.TimeUtc));
|
||||
|
||||
var distinctUris = batch
|
||||
.Where(r => r.Uri != null)
|
||||
.Select(r => r.Uri)
|
||||
.Distinct(StringComparer.OrdinalIgnoreCase)
|
||||
.ToList();
|
||||
var firstUri = batch.FirstOrDefault(r => r.Uri != null)?.Uri;
|
||||
var unparsedCount = batch.Count(r => r.Uri == null);
|
||||
var allBackfill = batch.All(r => r.IsBackfill);
|
||||
var earliest = batch[0].TimeUtc;
|
||||
|
||||
var data = new Dictionary<string, object>
|
||||
{
|
||||
{ "windowsEventId", eventId },
|
||||
{ "backfilled", isBackfill },
|
||||
{ "windowsEventId", EventId_PolicyRebootRequired },
|
||||
{ "eventLogChannel", Channel },
|
||||
{ "recordCount", batch.Count },
|
||||
{ "uriCount", distinctUris.Count },
|
||||
{ "backfilled", allBackfill },
|
||||
};
|
||||
if (recordId >= 0) data["recordId"] = recordId;
|
||||
if (timeCreatedUtc.HasValue) data["eventTime"] = timeCreatedUtc.Value.ToString("o");
|
||||
if (!string.IsNullOrEmpty(rebootUri)) data["rebootUri"] = rebootUri;
|
||||
if (!string.IsNullOrEmpty(formattedDescription))
|
||||
if (distinctUris.Count > 0)
|
||||
{
|
||||
data["description"] = formattedDescription.Length > 1000
|
||||
? formattedDescription.Substring(0, 1000) + "…"
|
||||
: formattedDescription;
|
||||
data["rebootUris"] = distinctUris
|
||||
.OrderBy(u => u, StringComparer.OrdinalIgnoreCase)
|
||||
.Take(MaxUrisPerEvent)
|
||||
.ToList();
|
||||
data["firstRebootUri"] = firstUri;
|
||||
}
|
||||
if (unparsedCount > 0)
|
||||
{
|
||||
data["unparsedCount"] = unparsedCount;
|
||||
var sample = batch.FirstOrDefault(r => r.Uri == null && !string.IsNullOrEmpty(r.Description))?.Description;
|
||||
if (sample != null)
|
||||
data["sampleDescription"] = sample.Length > 500 ? sample.Substring(0, 500) + "…" : sample;
|
||||
}
|
||||
|
||||
var flag = SafeProbeRebootRequiredFlag();
|
||||
if (flag.HasValue) data["omadmRebootRequiredFlag"] = flag.Value;
|
||||
|
||||
string message;
|
||||
if (distinctUris.Count == 1)
|
||||
message = $"Windows flagged an MDM policy URI as reboot-required: {firstUri}";
|
||||
else if (distinctUris.Count > 1)
|
||||
message = $"Windows flagged {distinctUris.Count} MDM policy URIs as reboot-required: {firstUri} (+{distinctUris.Count - 1} more)";
|
||||
else
|
||||
message = $"Windows flagged {batch.Count} MDM policy record(s) as reboot-required (URI not parsed — see sampleDescription)";
|
||||
|
||||
_post.Emit(new EnrollmentEvent
|
||||
{
|
||||
SessionId = _sessionId,
|
||||
TenantId = _tenantId,
|
||||
// The record's own time, not UtcNow: backfilled pre-agent (and post-reboot re-read)
|
||||
// records must land on the timeline at when the policy actually demanded the reboot.
|
||||
Timestamp = timeCreatedUtc ?? DateTime.UtcNow,
|
||||
// Earliest record time, not UtcNow: backfilled (and post-reboot re-read) bursts
|
||||
// must land on the timeline at when the policies were actually flagged.
|
||||
Timestamp = earliest,
|
||||
EventType = Constants.EventTypes.MdmPolicyRebootRequired,
|
||||
Severity = EventSeverity.Warning,
|
||||
// Neutral observation — reboot-required flags outside ESP DeviceSetup have no
|
||||
// immediate effect. The reboot CLAIM is ANALYZE-ESP-005's job, gated on an
|
||||
// actually-observed system_reboot_detected.
|
||||
Severity = EventSeverity.Info,
|
||||
Source = "MdmRebootPolicyWatcher",
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Message = !string.IsNullOrEmpty(rebootUri)
|
||||
? $"MDM policy requires a reboot during ESP: {rebootUri}"
|
||||
: "MDM policy requires a reboot during ESP (URI not parsed from event — see description)",
|
||||
Message = message,
|
||||
Data = data,
|
||||
// The reboot this event announces will kill the agent — flush now.
|
||||
// Cheap (one event per burst) and still time-critical: a burst DURING DeviceSetup
|
||||
// precedes a coalesced reboot that kills the agent.
|
||||
ImmediateUpload = true,
|
||||
});
|
||||
|
||||
_logger.Info($"MdmRebootPolicy: flushed {batch.Count} record(s) as one event " +
|
||||
$"(uris={distinctUris.Count}, unparsed={unparsedCount}, backfilled={allBackfill})");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Extracts the reboot-forcing OMA-DM URI. Preference order: (1) a structured EventData
|
||||
/// value that starts with <c>./</c> (the CSP-URI shape), (2) a permissive regex over the
|
||||
/// formatted description. Returns null when nothing URI-like is found — the caller still
|
||||
/// emits, just without the <c>rebootUri</c> field.
|
||||
/// value that starts with <c>./</c>, (2) a permissive regex over the formatted description
|
||||
/// (verified shape: <c>The following URI has triggered a reboot: (./Device/...).</c>).
|
||||
/// Returns null when nothing URI-like is found — the record still counts into the
|
||||
/// aggregate (<c>unparsedCount</c> + <c>sampleDescription</c>).
|
||||
/// </summary>
|
||||
internal static string ExtractRebootUri(Dictionary<string, string> eventData, string description)
|
||||
{
|
||||
@@ -289,8 +425,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
var match = UriPattern.Match(description);
|
||||
if (match.Success)
|
||||
{
|
||||
// The permissive pattern can swallow sentence punctuation trailing the URI
|
||||
// ("...Security."). OMA-DM URI segments never end in punctuation — trim it.
|
||||
// The permissive pattern can swallow sentence punctuation trailing the URI.
|
||||
// OMA-DM URI segments never end in punctuation — trim it.
|
||||
var candidate = match.Value.TrimEnd('.', ',', ';', ':', '\'', '"', ')');
|
||||
if (candidate.Length > 2) return candidate;
|
||||
}
|
||||
@@ -349,8 +485,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads the pending-reboot flag under <see cref="OmadmSyncMlKeyPath"/>. The exact shape is
|
||||
/// unverified (PMPC research names the key, not the value type), so both a value and a
|
||||
/// Reads the pending-reboot flag under <see cref="OmadmSyncMlKeyPath"/>. Both a value and a
|
||||
/// subkey named <c>RebootRequired</c> count as "pending". Fail-soft: null = unknown
|
||||
/// (key absent / unreadable), matching the EspSkipConfigurationProbe convention.
|
||||
/// </summary>
|
||||
@@ -424,44 +559,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Watermark dedup (cross-restart) — WindowsUpdateTracker pattern verbatim
|
||||
// Watermark dedup (cross-restart) — persisted at flush, not at claim
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private bool IsAlreadyProcessed(long recordId)
|
||||
{
|
||||
if (recordId < 0) return false; // no RecordId → cannot dedup, let it through
|
||||
lock (_watermarkLock)
|
||||
lock (_stateLock)
|
||||
{
|
||||
return recordId <= _restartWatermark || _seenThisRun.Contains(recordId);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Atomically claims <paramref name="recordId"/>. Returns true if this is the first time we
|
||||
/// see it (caller should emit), false if it was already processed — either in a PRIOR run
|
||||
/// (at/below the restart watermark) or already this run (in the seen-set). Records without
|
||||
/// a RecordId (-1) are always emitted, never tracked or persisted.
|
||||
/// </summary>
|
||||
private bool MarkProcessed(long recordId)
|
||||
{
|
||||
if (recordId < 0) return true;
|
||||
|
||||
long toPersist = -1;
|
||||
lock (_watermarkLock)
|
||||
{
|
||||
if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId))
|
||||
return false;
|
||||
|
||||
if (recordId > _maxEmittedRecordId)
|
||||
{
|
||||
_maxEmittedRecordId = recordId;
|
||||
toPersist = _maxEmittedRecordId;
|
||||
}
|
||||
}
|
||||
if (toPersist >= 0) PersistWatermark(toPersist);
|
||||
return true;
|
||||
}
|
||||
|
||||
internal void LoadWatermark()
|
||||
{
|
||||
if (string.IsNullOrEmpty(_stateDirectory)) return;
|
||||
@@ -475,10 +584,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
var state = JsonConvert.DeserializeObject<WatermarkState>(json);
|
||||
if (state != null)
|
||||
{
|
||||
lock (_watermarkLock)
|
||||
lock (_stateLock)
|
||||
{
|
||||
_restartWatermark = state.LastRecordId;
|
||||
_maxEmittedRecordId = state.LastRecordId;
|
||||
_maxPersistedRecordId = state.LastRecordId;
|
||||
}
|
||||
_logger.Info($"MdmRebootPolicy watermark loaded: lastRecordId={state.LastRecordId} (persisted {state.PersistedUtc:O})");
|
||||
}
|
||||
|
||||
+6
-81
@@ -16,9 +16,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
/// CommercialOOBE_ESPProgress_Page_Exiting — normal ESP exit
|
||||
/// CommercialOOBE_ESPProgress_WhiteGlove_Success — WhiteGlove complete
|
||||
/// CommercialOOBE_ESPProgress_Failure/_Timeout/_Abort/WhiteGlove_Failed — ESP failure
|
||||
/// "RebootCoalescing" (DeviceSetup) — ESP-initiated coalesced reboot
|
||||
/// (MDM policies forced a mid-ESP restart → second sign-in; corroborates the
|
||||
/// MdmRebootPolicyTracker's per-URI 2800 attribution events)
|
||||
///
|
||||
/// NOTE (session b2e890c1, 2026-07-20): a bare "RebootCoalescing" substring match on 62407 was
|
||||
/// tried and REMOVED — the token appears in the ROUTINE SubcategoryProcessing_Started marker
|
||||
/// ("Starting subcategory DeviceSetup.RebootCoalescing...") on every enrollment, so it proves
|
||||
/// nothing about an actual coalesced reboot. Policy-reboot attribution lives entirely in
|
||||
/// MdmRebootPolicyTracker (EventID 2800) + the ANALYZE-ESP-005 rule gated on an observed reboot.
|
||||
///
|
||||
/// Raises <see cref="FinalizingSetupPhaseTriggered"/>, <see cref="WhiteGloveCompleted"/>,
|
||||
/// and <see cref="EspFailureDetected"/>. Cross-notifies the <see cref="HelloTracker"/>
|
||||
@@ -50,7 +53,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
private bool _espExitDetected;
|
||||
private bool _whiteGloveDetected;
|
||||
private bool _helloWizardStartDetected;
|
||||
private bool _rebootCoalescingDetected;
|
||||
private readonly object _stateLock = new object();
|
||||
|
||||
/// <summary>
|
||||
@@ -251,27 +253,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
|
||||
_logger.Info("ESP phase exit detected - detected via Shell-Core event 62407");
|
||||
}
|
||||
else if (description.IndexOf("RebootCoalescing", StringComparison.OrdinalIgnoreCase) >= 0)
|
||||
{
|
||||
// ESP-initiated coalesced reboot (end of DeviceSetup): MDM policies forced a
|
||||
// restart, the user will see a second sign-in screen. Checked AFTER the
|
||||
// WhiteGlove/failure/exiting patterns so their semantics stay untouched even
|
||||
// if a description ever carried both tokens. Fire-once per process run — the
|
||||
// reboot restarts the agent, and the fresh run re-observes it via backfill.
|
||||
lock (_stateLock)
|
||||
{
|
||||
if (_rebootCoalescingDetected) return;
|
||||
_rebootCoalescingDetected = true;
|
||||
}
|
||||
|
||||
eventType = Constants.EventTypes.EspRebootCoalescing;
|
||||
severity = EventSeverity.Warning;
|
||||
message = "ESP initiated a coalesced reboot (RebootCoalescing) — device will restart and show a second sign-in";
|
||||
// No FinalizingSetup transition, no C# event raise — pure informational
|
||||
// corroboration for the MdmRebootPolicyTracker's per-URI attribution.
|
||||
|
||||
_logger.Warning("ESP coalesced reboot detected via Shell-Core event 62407 (RebootCoalescing)");
|
||||
}
|
||||
else
|
||||
{
|
||||
return;
|
||||
@@ -446,29 +427,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
return;
|
||||
}
|
||||
|
||||
if (description.IndexOf("RebootCoalescing", StringComparison.OrdinalIgnoreCase) >= 0)
|
||||
{
|
||||
// DELIBERATE deviation from the other 62407 backfill branches (which only raise C#
|
||||
// events): this is a pure informational type with no reducer/subscriber — the
|
||||
// emission IS the signal, and the live emit races the very reboot that kills the
|
||||
// process. Re-emitting after the restart is the only reliable delivery path. A
|
||||
// live+backfill cross-process duplicate is possible (no watermark here); accepted:
|
||||
// the analyze rule uses exists-semantics, both copies carry the identical historical
|
||||
// timestamp, and the backfill copy is marked backfilled:true.
|
||||
bool shouldEmit;
|
||||
lock (_stateLock)
|
||||
{
|
||||
shouldEmit = !_rebootCoalescingDetected;
|
||||
_rebootCoalescingDetected = true;
|
||||
}
|
||||
if (shouldEmit)
|
||||
{
|
||||
_logger.Info($"Backfill: ESP coalesced reboot (RebootCoalescing) found in recent Shell-Core logs (originalAt={occurredAtUtc:o})");
|
||||
EmitRebootCoalescing(description, occurredAtUtc, isBackfill: true);
|
||||
}
|
||||
return; // a RebootCoalescing record is not an ESP exit/failure — stop here
|
||||
}
|
||||
|
||||
if (EspExitingPattern.IsMatch(description))
|
||||
{
|
||||
bool shouldNotify = false;
|
||||
@@ -507,39 +465,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Emits the <c>esp_reboot_coalescing</c> event with the standard Shell-Core eventData
|
||||
/// shape. Used by the backfill path (the live path flows through the shared emit tail in
|
||||
/// <see cref="ProcessEvent"/>); <paramref name="isBackfill"/> adds the
|
||||
/// <c>backfilled</c> marker that distinguishes an accepted post-restart re-emission.
|
||||
/// </summary>
|
||||
private void EmitRebootCoalescing(string description, DateTime occurredAtUtc, bool isBackfill)
|
||||
{
|
||||
var eventData = new Dictionary<string, object>
|
||||
{
|
||||
{ "windowsEventId", EventId_ShellCore_WebAppEvent },
|
||||
{ "providerName", "" },
|
||||
{ "description", description },
|
||||
{ "eventLogChannel", ShellCoreEventLogChannel },
|
||||
{ "eventTime", occurredAtUtc.ToString("o") },
|
||||
{ "backfilled", isBackfill }
|
||||
};
|
||||
|
||||
_post.Emit(new EnrollmentEvent
|
||||
{
|
||||
SessionId = _sessionId,
|
||||
TenantId = _tenantId,
|
||||
Timestamp = occurredAtUtc,
|
||||
EventType = Constants.EventTypes.EspRebootCoalescing,
|
||||
Severity = EventSeverity.Warning,
|
||||
Source = "EspAndHelloTracker",
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Message = "ESP initiated a coalesced reboot (RebootCoalescing) — device will restart and show a second sign-in",
|
||||
Data = eventData,
|
||||
ImmediateUpload = true
|
||||
});
|
||||
}
|
||||
|
||||
// =====================================================================
|
||||
// Pattern helpers
|
||||
// =====================================================================
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
using System;
|
||||
using System.Reflection;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Interop
|
||||
{
|
||||
/// <summary>
|
||||
/// Reads WinRT <c>Windows.System.Profile.SystemSetupInfo.OutOfBoxExperienceState</c>
|
||||
/// (NotStarted=0 / InProgress=1 / Completed=2) via the CLR's built-in WinRT projection —
|
||||
/// no NuGet package, no winmd reference; the type is resolved at runtime with
|
||||
/// <c>ContentType=WindowsRuntime</c>, the same mechanism the bootstrap script uses from
|
||||
/// PowerShell 5.1.
|
||||
/// <para>
|
||||
/// <b>Contract:</b> observational only — never feed this into decision-engine logic.
|
||||
/// Requires Windows 10 1809+ (build 17763, UniversalApiContract 7.0) and is callable
|
||||
/// from the SYSTEM service context (empirically validated, probe session 9c404ae9).
|
||||
/// On older builds or SKUs without the contract the type resolves to null and
|
||||
/// <see cref="Read"/> returns <c>"unavailable"</c>. <see cref="Read"/> never throws.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Empirical timing (Win11 24H2 probe): the state flips InProgress→Completed at desktop
|
||||
/// arrival — it is a corroboration signal, not an early one. WDP reseal and placeholder
|
||||
/// (fooUser) flip semantics are unverified; another reason this must stay observational.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal static class OobeStateReader
|
||||
{
|
||||
public const string Unavailable = "unavailable";
|
||||
|
||||
// Resolved once per process. Benign race: concurrent first calls resolve the same
|
||||
// PropertyInfo and the assignment is idempotent — no lock needed.
|
||||
private static PropertyInfo _stateProperty;
|
||||
private static bool _initTried;
|
||||
|
||||
/// <summary>
|
||||
/// Returns <c>"not_started"</c> / <c>"in_progress"</c> / <c>"completed"</c>,
|
||||
/// <c>"unknown_<n>"</c> for unexpected enum values, or <c>"unavailable"</c>
|
||||
/// when the WinRT contract is absent or any part of the read fails. Never throws.
|
||||
/// </summary>
|
||||
public static string Read()
|
||||
{
|
||||
try
|
||||
{
|
||||
if (!_initTried)
|
||||
{
|
||||
var type = Type.GetType(
|
||||
"Windows.System.Profile.SystemSetupInfo, Windows.System.Profile, ContentType=WindowsRuntime",
|
||||
throwOnError: false);
|
||||
_stateProperty = type?.GetProperty(
|
||||
"OutOfBoxExperienceState",
|
||||
BindingFlags.Public | BindingFlags.Static);
|
||||
_initTried = true;
|
||||
}
|
||||
|
||||
if (_stateProperty == null)
|
||||
return Unavailable;
|
||||
|
||||
var value = Convert.ToInt32(_stateProperty.GetValue(null));
|
||||
switch (value)
|
||||
{
|
||||
case 0: return "not_started";
|
||||
case 1: return "in_progress";
|
||||
case 2: return "completed";
|
||||
default: return "unknown_" + value;
|
||||
}
|
||||
}
|
||||
catch
|
||||
{
|
||||
return Unavailable;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -31,84 +31,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
|
||||
{
|
||||
_logger.Info($"Executing FULL SELF-DESTRUCT (Scheduled Task + Files{(_configuration.RebootOnComplete ? " + Reboot" : "")})");
|
||||
|
||||
var currentProcessId = Process.GetCurrentProcess().Id;
|
||||
var currentProcess = Process.GetCurrentProcess();
|
||||
var agentBasePath = Path.GetFullPath(Path.Combine(
|
||||
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),
|
||||
"AutopilotMonitor"
|
||||
));
|
||||
|
||||
var logDir = Path.GetFullPath(Path.Combine(agentBasePath, "Logs"));
|
||||
var keepLogs = _configuration.KeepLogFile;
|
||||
|
||||
// Create a self-deleting PowerShell script
|
||||
var cleanupScript = $@"
|
||||
$scriptPath = $MyInvocation.MyCommand.Path
|
||||
|
||||
# Wait for agent process to exit
|
||||
Start-Sleep -Seconds 2
|
||||
Wait-Process -Id {currentProcessId} -ErrorAction SilentlyContinue -Timeout 30
|
||||
Start-Sleep -Seconds 1
|
||||
|
||||
# Remove Scheduled Task — verify removal BEFORE deleting files (LIFE-F6). An orphan task firing
|
||||
# against a deleted exe spams Task Scheduler errors every boot forever and, once the files (incl.
|
||||
# the enrollment-complete marker) are gone, has no cleanup-retry path. So confirm the task is gone
|
||||
# first; if it cannot be removed, SKIP the file deletion and leave everything intact for a clean
|
||||
# cleanup retry on the next boot (the marker survives, so bootstrap re-runs cleanup).
|
||||
$taskName = '{_configuration.ScheduledTaskName}'
|
||||
$taskGone = $false
|
||||
for ($i = 1; $i -le 5; $i++) {{
|
||||
try {{ Stop-ScheduledTask -TaskName $taskName -ErrorAction SilentlyContinue }} catch {{ }}
|
||||
try {{ Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -ErrorAction SilentlyContinue }} catch {{ }}
|
||||
$still = $null
|
||||
try {{ $still = Get-ScheduledTask -TaskName $taskName -ErrorAction SilentlyContinue }} catch {{ }}
|
||||
if (-not $still) {{ $taskGone = $true; break }}
|
||||
Start-Sleep -Seconds 1
|
||||
}}
|
||||
if (-not $taskGone) {{
|
||||
# Fallback: schtasks.exe /Delete does not depend on the ScheduledTasks PowerShell module.
|
||||
try {{ & schtasks.exe /Delete /TN $taskName /F 2>$null | Out-Null }} catch {{ }}
|
||||
try {{ $taskGone = -not (Get-ScheduledTask -TaskName $taskName -ErrorAction SilentlyContinue) }} catch {{ }}
|
||||
}}
|
||||
|
||||
{(keepLogs ? $@"
|
||||
if ($taskGone) {{
|
||||
# Delete everything except Logs directory
|
||||
Get-ChildItem -Path '{agentBasePath}' -Exclude 'Logs' | ForEach-Object {{
|
||||
$dest = $_.FullName + '.del'
|
||||
try {{ Rename-Item -Path $_.FullName -NewName $dest -Force -ErrorAction SilentlyContinue }} catch {{ }}
|
||||
Remove-Item -Path $dest -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}}
|
||||
}}
|
||||
" : $@"
|
||||
if ($taskGone) {{
|
||||
# Rename folder then delete. Retry a few times to let the OS release handles.
|
||||
$renamedPath = '{agentBasePath}.del'
|
||||
$renamed = $false
|
||||
for ($i = 1; $i -le 10; $i++) {{
|
||||
try {{
|
||||
Rename-Item -Path '{agentBasePath}' -NewName $renamedPath -Force -ErrorAction Stop
|
||||
$renamed = $true
|
||||
break
|
||||
}} catch {{
|
||||
Start-Sleep -Seconds 2
|
||||
}}
|
||||
}}
|
||||
if ($renamed) {{
|
||||
Remove-Item -Path $renamedPath -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}} else {{
|
||||
Remove-Item -Path '{agentBasePath}' -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}}
|
||||
}}
|
||||
")}
|
||||
{(_configuration.RebootOnComplete ? @"
|
||||
# shutdown.exe (NOT Restart-Computer): Restart-Computer has no -Comment parameter and -Delay is
|
||||
# only honoured with -Wait, so the previous invocation failed parameter binding and the device
|
||||
# silently never rebooted. shutdown.exe /r /t 10 mirrors the standalone reboot path and reboots
|
||||
# after a 10 s delay (lets this script finish + the agent exit). Review LIFE-F2.
|
||||
shutdown.exe /r /t 10 /c 'Autopilot enrollment completed - Autopilot Monitor is rebooting'
|
||||
" : "")}
|
||||
Remove-Item -Path $scriptPath -Force -ErrorAction SilentlyContinue
|
||||
";
|
||||
var cleanupScript = BuildCleanupScript(
|
||||
agentProcessName: currentProcess.ProcessName,
|
||||
agentBasePath: agentBasePath,
|
||||
keepLogs: _configuration.KeepLogFile,
|
||||
scheduledTaskName: _configuration.ScheduledTaskName,
|
||||
rebootOnComplete: _configuration.RebootOnComplete);
|
||||
|
||||
// Write cleanup script to temp location (outside of agent folder)
|
||||
var tempScriptPath = Path.Combine(Path.GetTempPath(), $"AutopilotMonitor-Cleanup-{Guid.NewGuid()}.ps1");
|
||||
@@ -143,5 +77,171 @@ Remove-Item -Path $scriptPath -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Builds the self-deleting cleanup PowerShell script. Extracted (internal static, pure)
|
||||
/// so tests can pin the hardened invariants without launching processes.
|
||||
/// <para>
|
||||
/// LIFE-F7 hardening (incident 2026-07-22 — exe + loaded DLLs orphaned forever):
|
||||
/// </para>
|
||||
/// <list type="number">
|
||||
/// <item><b>Single-flight lock</b> — the enrollment-complete marker is written BEFORE
|
||||
/// cleanup launches, so any agent start during an in-flight cleanup spawns a SECOND
|
||||
/// cleanup script. Two unsynchronized scripts interleave their rename/delete phases
|
||||
/// and leave partial trees behind. A CreateNew lock file in the SYSTEM temp dir
|
||||
/// serializes them; a stale lock (>10 min) is taken over.</item>
|
||||
/// <item><b>Wait by process NAME, not launcher PID</b> — marker-retry instances,
|
||||
/// self-update restarts and manual runs all load the same images but have PIDs the
|
||||
/// old <c>Wait-Process -Id</c> never saw (and an already-freed PID can be reused by
|
||||
/// an unrelated boot-time process). After the deadline any straggler is force-killed:
|
||||
/// its images are about to be deleted anyway.</item>
|
||||
/// <item><b>Probe-rename BEFORE task removal, never delete the original path</b> — the
|
||||
/// rename is the atomic lock probe. If it fails, something still holds the tree: abort
|
||||
/// with task + files + marker intact so the next boot retries (file-side symmetry to
|
||||
/// the LIFE-F6 task-side rule). If the task cannot be removed afterwards, rename BACK
|
||||
/// and abort for the same reason. Only a fully renamed tree with the task gone is
|
||||
/// deleted — the old <c>Remove-Item</c>-on-original fallback deleted marker + state
|
||||
/// around locked binaries AFTER the task was gone, converting a transient lock into a
|
||||
/// permanently orphaned exe.</item>
|
||||
/// </list>
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The timing parameters exist ONLY so integration tests can replay the incident
|
||||
/// scenarios (locked tree, straggler process) in seconds instead of minutes —
|
||||
/// production call sites always use the defaults.
|
||||
/// </remarks>
|
||||
internal static string BuildCleanupScript(
|
||||
string agentProcessName,
|
||||
string agentBasePath,
|
||||
bool keepLogs,
|
||||
string scheduledTaskName,
|
||||
bool rebootOnComplete,
|
||||
int initialDelaySeconds = 2,
|
||||
int processWaitSeconds = 60,
|
||||
int renameRetries = 10,
|
||||
int renameRetryDelaySeconds = 2)
|
||||
{
|
||||
return $@"
|
||||
$scriptPath = $MyInvocation.MyCommand.Path
|
||||
|
||||
# Single-flight lock (LIFE-F7 #1): serialize concurrent cleanup scripts. CreateNew is
|
||||
# atomic; the loser backs off and lets the winner own the deletion.
|
||||
$lockPath = Join-Path ([System.IO.Path]::GetTempPath()) 'AutopilotMonitor-Cleanup.lock'
|
||||
$lockStream = $null
|
||||
try {{
|
||||
if (Test-Path $lockPath) {{
|
||||
$lockAge = (Get-Date) - (Get-Item $lockPath -ErrorAction Stop).LastWriteTime
|
||||
if ($lockAge.TotalMinutes -gt 10) {{ Remove-Item -Path $lockPath -Force -ErrorAction SilentlyContinue }}
|
||||
}}
|
||||
$lockStream = [System.IO.File]::Open($lockPath, [System.IO.FileMode]::CreateNew, [System.IO.FileAccess]::Write, [System.IO.FileShare]::None)
|
||||
}} catch {{
|
||||
Remove-Item -Path $scriptPath -Force -ErrorAction SilentlyContinue
|
||||
exit 0
|
||||
}}
|
||||
|
||||
try {{
|
||||
# Wait for EVERY agent process to exit, by NAME (LIFE-F7 #2) — covers marker-retry
|
||||
# instances, self-update restarts and manual runs, and is immune to PID reuse.
|
||||
Start-Sleep -Seconds {initialDelaySeconds}
|
||||
$deadline = (Get-Date).AddSeconds({processWaitSeconds})
|
||||
while ((Get-Date) -lt $deadline) {{
|
||||
if (-not (Get-Process -Name '{agentProcessName}' -ErrorAction SilentlyContinue)) {{ break }}
|
||||
Start-Sleep -Seconds 2
|
||||
}}
|
||||
# Backstop: force-kill stragglers — their images are about to be deleted anyway.
|
||||
Get-Process -Name '{agentProcessName}' -ErrorAction SilentlyContinue | Stop-Process -Force -ErrorAction SilentlyContinue
|
||||
Start-Sleep -Seconds 1
|
||||
|
||||
{(keepLogs ? $@"
|
||||
# Probe-rename all children except Logs (LIFE-F7 #3). Any failure ⇒ rename back and
|
||||
# abort with the Scheduled Task intact so the next boot retries cleanly.
|
||||
$renamed = @()
|
||||
$probeOk = $true
|
||||
foreach ($item in @(Get-ChildItem -Path '{agentBasePath}' -Exclude 'Logs' -ErrorAction SilentlyContinue)) {{
|
||||
$dest = $item.FullName + '.del-' + [Guid]::NewGuid().ToString('N')
|
||||
$itemOk = $false
|
||||
for ($i = 1; $i -le {renameRetries}; $i++) {{
|
||||
try {{
|
||||
Rename-Item -Path $item.FullName -NewName $dest -Force -ErrorAction Stop
|
||||
$itemOk = $true
|
||||
break
|
||||
}} catch {{
|
||||
Start-Sleep -Seconds {renameRetryDelaySeconds}
|
||||
}}
|
||||
}}
|
||||
if ($itemOk) {{ $renamed += @{{ Original = $item.FullName; Renamed = $dest }} }}
|
||||
else {{ $probeOk = $false; break }}
|
||||
}}
|
||||
" : $@"
|
||||
# Probe-rename the whole tree (LIFE-F7 #3). Failure ⇒ abort with task + files + marker
|
||||
# intact so the next boot retries cleanly. GUID suffix avoids collisions with leftovers
|
||||
# of earlier attempts.
|
||||
$renamedPath = '{agentBasePath}.del-' + [Guid]::NewGuid().ToString('N')
|
||||
$probeOk = $false
|
||||
for ($i = 1; $i -le {renameRetries}; $i++) {{
|
||||
try {{
|
||||
Rename-Item -Path '{agentBasePath}' -NewName $renamedPath -Force -ErrorAction Stop
|
||||
$probeOk = $true
|
||||
break
|
||||
}} catch {{
|
||||
Start-Sleep -Seconds {renameRetryDelaySeconds}
|
||||
}}
|
||||
}}
|
||||
$renamed = @(@{{ Original = '{agentBasePath}'; Renamed = $renamedPath }})
|
||||
")}
|
||||
if ($probeOk) {{
|
||||
# Remove Scheduled Task — verify removal BEFORE deleting files (LIFE-F6). An orphan
|
||||
# task firing against a deleted exe spams Task Scheduler errors every boot forever
|
||||
# and, once the files (incl. the enrollment-complete marker) are gone, has no
|
||||
# cleanup-retry path.
|
||||
$taskName = '{scheduledTaskName}'
|
||||
$taskGone = $false
|
||||
for ($i = 1; $i -le 5; $i++) {{
|
||||
try {{ Stop-ScheduledTask -TaskName $taskName -ErrorAction SilentlyContinue }} catch {{ }}
|
||||
try {{ Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -ErrorAction SilentlyContinue }} catch {{ }}
|
||||
$still = $null
|
||||
try {{ $still = Get-ScheduledTask -TaskName $taskName -ErrorAction SilentlyContinue }} catch {{ }}
|
||||
if (-not $still) {{ $taskGone = $true; break }}
|
||||
Start-Sleep -Seconds 1
|
||||
}}
|
||||
if (-not $taskGone) {{
|
||||
# Fallback: schtasks.exe /Delete does not depend on the ScheduledTasks PowerShell module.
|
||||
try {{ & schtasks.exe /Delete /TN $taskName /F 2>$null | Out-Null }} catch {{ }}
|
||||
try {{ $taskGone = -not (Get-ScheduledTask -TaskName $taskName -ErrorAction SilentlyContinue) }} catch {{ }}
|
||||
}}
|
||||
|
||||
if ($taskGone) {{
|
||||
# Task gone + tree renamed ⇒ safe to delete. Never touches the original path.
|
||||
foreach ($entry in $renamed) {{
|
||||
Remove-Item -Path $entry.Renamed -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}}
|
||||
}} else {{
|
||||
# Task survived ⇒ rename back and leave everything for the next-boot retry
|
||||
# (the marker survives, so bootstrap re-runs cleanup).
|
||||
foreach ($entry in $renamed) {{
|
||||
try {{ Rename-Item -Path $entry.Renamed -NewName $entry.Original -Force -ErrorAction SilentlyContinue }} catch {{ }}
|
||||
}}
|
||||
}}
|
||||
}} else {{
|
||||
# Probe failed ⇒ something still holds the tree. Rename back whatever succeeded and
|
||||
# abort WITHOUT deleting anything — task + marker stay, next boot retries.
|
||||
foreach ($entry in $renamed) {{
|
||||
try {{ Rename-Item -Path $entry.Renamed -NewName $entry.Original -Force -ErrorAction SilentlyContinue }} catch {{ }}
|
||||
}}
|
||||
}}
|
||||
{(rebootOnComplete ? @"
|
||||
# shutdown.exe (NOT Restart-Computer): Restart-Computer has no -Comment parameter and -Delay is
|
||||
# only honoured with -Wait, so the previous invocation failed parameter binding and the device
|
||||
# silently never rebooted. shutdown.exe /r /t 10 mirrors the standalone reboot path and reboots
|
||||
# after a 10 s delay (lets this script finish + the agent exit). Review LIFE-F2.
|
||||
shutdown.exe /r /t 10 /c 'Autopilot enrollment completed - Autopilot Monitor is rebooting'
|
||||
" : "")}
|
||||
}} finally {{
|
||||
if ($lockStream) {{ $lockStream.Dispose() }}
|
||||
Remove-Item -Path $lockPath -Force -ErrorAction SilentlyContinue
|
||||
Remove-Item -Path $scriptPath -Force -ErrorAction SilentlyContinue
|
||||
}}
|
||||
";
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
@@ -370,7 +370,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
|
||||
{
|
||||
try
|
||||
{
|
||||
var versionUrl = $"{Constants.AgentBlobBaseUrl}/{Constants.AgentVersionFileNameForLine(2)}";
|
||||
// Download alias, not the legacy blob account: the legacy mirror in the
|
||||
// release pipeline is fail-soft, so it can silently lag behind eu.
|
||||
var versionUrl = $"{Constants.AgentDownloadBaseUrl}/{Constants.AgentVersionFileNameForLine(2)}";
|
||||
|
||||
using (var handler = new HttpClientHandler())
|
||||
using (var client = new HttpClient(handler) { Timeout = TimeSpan.FromMilliseconds(VersionCheckTimeoutMs) })
|
||||
@@ -497,7 +499,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
|
||||
{
|
||||
try
|
||||
{
|
||||
var zipUrl = $"{Constants.AgentBlobBaseUrl}/{Constants.AgentZipFileNameForLine(2)}";
|
||||
var zipUrl = $"{Constants.AgentDownloadBaseUrl}/{Constants.AgentZipFileNameForLine(2)}";
|
||||
|
||||
// Clean up any previous download (fragment from aborted run, or symlink/junction).
|
||||
// Pre-delete + FileMode.CreateNew below: CreateNew refuses to open an existing
|
||||
|
||||
+43
-4
@@ -3,6 +3,7 @@ using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Management;
|
||||
using System.Text.RegularExpressions;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
|
||||
using AutopilotMonitor.Agent.V2.Core.Orchestration;
|
||||
@@ -28,6 +29,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Analyzers
|
||||
///
|
||||
/// Emits a single "local_admin_analysis" event at startup and at shutdown,
|
||||
/// enabling delta detection between pre- and post-enrollment state.
|
||||
///
|
||||
/// Tenant-supplied allowed accounts may contain glob wildcards (e.g. "adm-*");
|
||||
/// see <see cref="MatchesAllowedEntry"/>.
|
||||
/// </summary>
|
||||
public class LocalAdminAnalyzer : IAgentAnalyzer
|
||||
{
|
||||
@@ -246,6 +250,43 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Analyzers
|
||||
}
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Allowed-list matching
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/// <summary>
|
||||
/// Matches an account or profile-folder name against one allowed-list entry.
|
||||
/// Entries containing <c>*</c> or <c>?</c> are glob patterns (same semantics as the
|
||||
/// hardware whitelist: <c>*</c> = any run, <c>?</c> = one char, case-insensitive);
|
||||
/// all other entries match exactly. No ambiguity with literal names — <c>*</c> and
|
||||
/// <c>?</c> are invalid characters in Windows account names.
|
||||
/// Internal for direct unit-testing via InternalsVisibleTo.
|
||||
/// </summary>
|
||||
internal static bool MatchesAllowedEntry(string name, string entry)
|
||||
{
|
||||
if (entry.IndexOf('*') < 0 && entry.IndexOf('?') < 0)
|
||||
return string.Equals(entry, name, StringComparison.OrdinalIgnoreCase);
|
||||
|
||||
var regexPattern = "^" + Regex.Escape(entry)
|
||||
.Replace("\\*", ".*")
|
||||
.Replace("\\?", ".") + "$";
|
||||
|
||||
try
|
||||
{
|
||||
// Timeout guards against pathological patterns (ReDoS)
|
||||
return Regex.IsMatch(name, regexPattern, RegexOptions.IgnoreCase, TimeSpan.FromMilliseconds(100));
|
||||
}
|
||||
catch (RegexMatchTimeoutException)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
private static bool IsAllowed(string name, List<string> allowedAccounts)
|
||||
{
|
||||
return allowedAccounts.Any(a => MatchesAllowedEntry(name, a));
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Individual checks
|
||||
// -----------------------------------------------------------------------
|
||||
@@ -308,8 +349,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Analyzers
|
||||
if (disabled)
|
||||
continue;
|
||||
|
||||
if (!allowedAccounts.Any(a =>
|
||||
string.Equals(a, name, StringComparison.OrdinalIgnoreCase)))
|
||||
if (!IsAllowed(name, allowedAccounts))
|
||||
{
|
||||
unexpected.Add(name);
|
||||
_logger.Debug($"{Name}: Unexpected local account: {name}");
|
||||
@@ -349,8 +389,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Analyzers
|
||||
|
||||
allFound.Add(folderName);
|
||||
|
||||
if (!allowedAccounts.Any(a =>
|
||||
string.Equals(a, folderName, StringComparison.OrdinalIgnoreCase)))
|
||||
if (!IsAllowed(folderName, allowedAccounts))
|
||||
{
|
||||
unexpected.Add(folderName);
|
||||
_logger.Debug($"{Name}: Unexpected profile folder: {folderName}");
|
||||
|
||||
+39
-4
@@ -12,6 +12,24 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
{
|
||||
public string CollectorType => "command_allowlisted";
|
||||
|
||||
private const int CommandTimeoutMs = 30000;
|
||||
|
||||
/// <summary>
|
||||
/// Returns what the reader captured, or an empty string if it did not finish
|
||||
/// (a killed process can leave the read faulted or still pending).
|
||||
/// </summary>
|
||||
private static string DrainOrEmpty(System.Threading.Tasks.Task<string> readTask)
|
||||
{
|
||||
try
|
||||
{
|
||||
return readTask.Wait(2000) ? readTask.Result : string.Empty;
|
||||
}
|
||||
catch
|
||||
{
|
||||
return string.Empty;
|
||||
}
|
||||
}
|
||||
|
||||
public Dictionary<string, object> Execute(GatherRule rule, GatherRuleContext context)
|
||||
{
|
||||
var data = new Dictionary<string, object>();
|
||||
@@ -81,12 +99,29 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
|
||||
using (var process = Process.Start(psi))
|
||||
{
|
||||
var output = process.StandardOutput.ReadToEnd();
|
||||
var error = process.StandardError.ReadToEnd();
|
||||
process.WaitForExit(30000); // 30 second timeout
|
||||
// Read asynchronously: a synchronous ReadToEnd() blocks until the pipe
|
||||
// closes, which only happens when the process exits — the WaitForExit
|
||||
// timeout below would never be reached and a hung command would pin
|
||||
// this worker forever.
|
||||
var outputTask = process.StandardOutput.ReadToEndAsync();
|
||||
var errorTask = process.StandardError.ReadToEndAsync();
|
||||
|
||||
var exited = process.WaitForExit(CommandTimeoutMs);
|
||||
if (!exited)
|
||||
{
|
||||
context.Logger.Warning(
|
||||
$"Command timed out after {CommandTimeoutMs / 1000}s, killing: {command} (Rule: {rule.RuleId})");
|
||||
try { process.Kill(); } catch { /* already gone */ }
|
||||
// Give the readers a moment to drain once the pipes close.
|
||||
try { process.WaitForExit(5000); } catch { /* best effort */ }
|
||||
data["timed_out"] = true;
|
||||
}
|
||||
|
||||
data["command"] = command;
|
||||
data["exit_code"] = process.ExitCode;
|
||||
data["exit_code"] = exited ? process.ExitCode : -1;
|
||||
|
||||
var output = DrainOrEmpty(outputTask);
|
||||
var error = DrainOrEmpty(errorTask);
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(output))
|
||||
{
|
||||
|
||||
+4
@@ -17,6 +17,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
if (string.IsNullOrEmpty(logName))
|
||||
return data;
|
||||
|
||||
// Guard: only allow enrollment-relevant channels
|
||||
if (!GatherRuleGuards.IsEventLogChannelAllowed(logName, context.UnrestrictedMode))
|
||||
return context.EmitSecurityWarning(rule, "eventlog", logName);
|
||||
|
||||
// Optional: escalate severity when matching entries are found
|
||||
string severityIfFound = null;
|
||||
rule.Parameters?.TryGetValue("severityIfFound", out severityIfFound);
|
||||
|
||||
+33
-2
@@ -34,11 +34,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
if (filePath == null)
|
||||
return null; // Token present but no user logged on — skip silently
|
||||
|
||||
// Apply IME log path override if set
|
||||
// Apply IME log path override if set. The override comes from the local
|
||||
// --ime-log-path flag only (never from remote config), so the operator
|
||||
// running it is already a local admin — it relaxes the allowlist the same
|
||||
// way unrestricted mode does, while the hard blocks below still apply.
|
||||
var localOverride = false;
|
||||
if (!string.IsNullOrEmpty(context.ImeLogPathOverride))
|
||||
{
|
||||
var fileName = Path.GetFileName(filePath);
|
||||
filePath = Path.Combine(context.ImeLogPathOverride, fileName);
|
||||
localOverride = true;
|
||||
}
|
||||
|
||||
var relaxAllowlist = context.UnrestrictedMode || localOverride;
|
||||
|
||||
// Guard: only allow enrollment-relevant file paths. Checked before resolution
|
||||
// so a blocked target is reported even when no file matches it. A wildcard
|
||||
// target is checked by its directory — Path.GetFullPath rejects '*' and '?'.
|
||||
var guardPath = HasWildcard(Path.GetFileName(filePath))
|
||||
? Path.GetDirectoryName(filePath)
|
||||
: filePath;
|
||||
if (!GatherRuleGuards.IsFilePathAllowed(guardPath, relaxAllowlist, userProfilePath))
|
||||
{
|
||||
context.EmitSecurityWarning(rule, "logparser", filePath);
|
||||
return null;
|
||||
}
|
||||
|
||||
// Get parameters
|
||||
@@ -87,18 +106,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
|
||||
foreach (var resolvedPath in resolvedPaths)
|
||||
{
|
||||
// Re-check each resolved path: wildcard expansion can surface files
|
||||
// reached through a junction that the directory check did not cover.
|
||||
if (!GatherRuleGuards.IsFilePathAllowed(resolvedPath, relaxAllowlist, userProfilePath))
|
||||
{
|
||||
context.EmitSecurityWarning(rule, "logparser", resolvedPath);
|
||||
continue;
|
||||
}
|
||||
|
||||
ProcessLogFile(resolvedPath, rule, pattern, trackPosition, maxLines, isTextMode, context);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
private static bool HasWildcard(string fileNamePart)
|
||||
=> !string.IsNullOrEmpty(fileNamePart) &&
|
||||
(fileNamePart.Contains("*") || fileNamePart.Contains("?"));
|
||||
|
||||
private static List<string> ResolveLogPaths(string filePath, string ruleId, GatherRuleContext context)
|
||||
{
|
||||
var fileNamePart = Path.GetFileName(filePath);
|
||||
|
||||
// No wildcards — single file
|
||||
if (!fileNamePart.Contains("*") && !fileNamePart.Contains("?"))
|
||||
if (!HasWildcard(fileNamePart))
|
||||
{
|
||||
if (File.Exists(filePath))
|
||||
return new List<string> { filePath };
|
||||
|
||||
+48
-1
@@ -176,12 +176,59 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
|
||||
/// <summary>
|
||||
/// Called when a phase change event occurs - executes rules triggered by phase changes
|
||||
/// (<c>phase_change</c> on entering the new phase, <c>phase_exit</c> on leaving the old one).
|
||||
/// </summary>
|
||||
public void OnPhaseChanged(EnrollmentPhase newPhase)
|
||||
{
|
||||
var phaseName = newPhase.ToString();
|
||||
|
||||
// Update the phase + from-phase latches FIRST so deferred startup activation and
|
||||
// phase_exit runs FIRST and against the phase being LEFT: both the TriggerPhase match
|
||||
// and the scope gate must see the old phase, so this block has to complete before
|
||||
// _currentPhase moves below. A transition INTO Failed still fires the exit rules of the
|
||||
// phase that failed — that snapshot is the whole point at a failure boundary.
|
||||
List<GatherRule> exitRules = null;
|
||||
lock (_scopeLock)
|
||||
{
|
||||
var previousPhase = _currentPhase;
|
||||
if (previousPhase != EnrollmentPhase.Unknown && previousPhase != newPhase)
|
||||
{
|
||||
var previousPhaseName = previousPhase.ToString();
|
||||
exitRules = new List<GatherRule>();
|
||||
|
||||
foreach (var rule in _activeRules)
|
||||
{
|
||||
if (rule.Trigger != "phase_exit") continue;
|
||||
if (!string.IsNullOrEmpty(rule.TriggerPhase)
|
||||
&& !string.Equals(rule.TriggerPhase, previousPhaseName, StringComparison.OrdinalIgnoreCase))
|
||||
continue;
|
||||
|
||||
// Scope gate before the dedup add — an out-of-scope pass must not consume
|
||||
// the once-per-(rule, phase) slot. IsRuleInScopeLocked still sees the old phase.
|
||||
if (!IsRuleInScopeLocked(rule)) continue;
|
||||
|
||||
// "exit:" prefix keeps this key space disjoint from the phase_change keys.
|
||||
if (!_phaseRulesExecuted.Add($"exit:{rule.RuleId}|{previousPhaseName}"))
|
||||
{
|
||||
_logger.Debug($"Phase-exit rule {rule.RuleId} already executed for exit of {previousPhaseName}, skipping");
|
||||
continue;
|
||||
}
|
||||
|
||||
_logger.Info($"Phase exit triggered rule {rule.RuleId} (left phase: {previousPhaseName}, now: {phaseName})");
|
||||
exitRules.Add(rule);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (exitRules != null)
|
||||
{
|
||||
foreach (var rule in exitRules)
|
||||
{
|
||||
var exitRule = rule;
|
||||
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(exitRule));
|
||||
}
|
||||
}
|
||||
|
||||
// Update the phase + from-phase latches so deferred startup activation and
|
||||
// the phase_change rules below all evaluate against the NEW phase.
|
||||
List<GatherRule> deferredStartup;
|
||||
lock (_scopeLock)
|
||||
|
||||
+55
@@ -33,6 +33,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
public static readonly IReadOnlyList<string> AllowedFilePrefixes;
|
||||
public static readonly IReadOnlyList<string> AllowedWmiQueryPrefixes;
|
||||
public static readonly IReadOnlyCollection<string> AllowedCommands;
|
||||
public static readonly IReadOnlyList<string> AllowedEventLogChannels;
|
||||
|
||||
// Hard blocks: never allowed, even in unrestricted mode
|
||||
private static readonly string BlockedUsersPrefix = Path.GetFullPath(@"C:\Users");
|
||||
@@ -51,6 +52,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
// Hard limit: maximum command length (even in unrestricted mode)
|
||||
private const int MaxCommandLength = 2000;
|
||||
|
||||
// Event log channels that are never readable, even in unrestricted mode.
|
||||
// Security carries the audit trail of user behaviour; the PowerShell channels
|
||||
// carry script-block logging, which routinely contains secrets in clear text.
|
||||
// Mirrored in rules/guardrails.json ("blockedEventLogChannels") for the portal
|
||||
// to display — enforcement lives here so a parse error cannot lift it.
|
||||
private static readonly string[] HardBlockedEventLogChannels = new[]
|
||||
{
|
||||
"Security",
|
||||
"Microsoft-Windows-PowerShell",
|
||||
"Windows PowerShell",
|
||||
"Microsoft-Windows-Sysmon",
|
||||
};
|
||||
|
||||
// Hard-blocked command patterns: never allowed, even in unrestricted mode.
|
||||
// These prevent privilege escalation, persistence, and data exfiltration.
|
||||
private static readonly string[] HardBlockedCommandPatterns = new[]
|
||||
@@ -85,6 +99,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
|
||||
var commands = FlattenCategorized(obj, "allowedCommands", "commands");
|
||||
AllowedCommands = new HashSet<string>(commands, StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
AllowedEventLogChannels = FlattenCategorized(obj, "eventLogChannels", "channels");
|
||||
return;
|
||||
}
|
||||
}
|
||||
@@ -98,6 +114,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
AllowedFilePrefixes = new List<string>();
|
||||
AllowedWmiQueryPrefixes = new List<string>();
|
||||
AllowedCommands = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
|
||||
AllowedEventLogChannels = new List<string>();
|
||||
}
|
||||
|
||||
private static string LoadEmbeddedGuardrails()
|
||||
@@ -257,6 +274,44 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
(trimmed.Length == prefix.Length || char.IsWhiteSpace(trimmed[prefix.Length])));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Returns true if the event log channel matches an allowed entry with
|
||||
/// boundary matching (next char must be '/' or end of string), so that
|
||||
/// "Microsoft-Windows-AAD" admits "Microsoft-Windows-AAD/Operational"
|
||||
/// but not "Microsoft-Windows-AADSomethingElse".
|
||||
/// </summary>
|
||||
public static bool IsEventLogChannelAllowed(string channel)
|
||||
=> IsEventLogChannelAllowed(channel, unrestrictedMode: false);
|
||||
|
||||
/// <summary>
|
||||
/// Returns true if the event log channel is allowed.
|
||||
/// When unrestrictedMode is true, all channels are allowed except the
|
||||
/// hard-blocked ones (audit trail and script-block logging).
|
||||
/// </summary>
|
||||
public static bool IsEventLogChannelAllowed(string channel, bool unrestrictedMode)
|
||||
{
|
||||
if (string.IsNullOrEmpty(channel))
|
||||
return false;
|
||||
|
||||
var trimmed = channel.Trim();
|
||||
|
||||
// Hard blocks apply even in unrestricted mode
|
||||
foreach (var blocked in HardBlockedEventLogChannels)
|
||||
{
|
||||
if (MatchesChannel(trimmed, blocked))
|
||||
return false;
|
||||
}
|
||||
|
||||
if (unrestrictedMode)
|
||||
return true;
|
||||
|
||||
return AllowedEventLogChannels.Any(allowed => MatchesChannel(trimmed, allowed));
|
||||
}
|
||||
|
||||
private static bool MatchesChannel(string channel, string prefix)
|
||||
=> channel.StartsWith(prefix, StringComparison.OrdinalIgnoreCase) &&
|
||||
(channel.Length == prefix.Length || channel[prefix.Length] == '/');
|
||||
|
||||
/// <summary>
|
||||
/// Returns true if the command (trimmed) exactly matches an entry in the allowlist.
|
||||
/// Exact matching is intentional — no prefix spoofing possible.
|
||||
|
||||
+2
-2
@@ -45,10 +45,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
|
||||
/// </summary>
|
||||
private static readonly (string Name, string Url)[] MdmEndpoints =
|
||||
{
|
||||
("Entra ID", "https://login.microsoftonline.com"),
|
||||
("Entra ID", Constants.EntraLoginBaseUrl),
|
||||
("Device Registration", "https://enterpriseregistration.windows.net"),
|
||||
("Intune Portal", "https://portal.manage.microsoft.com"),
|
||||
("Microsoft Graph", "https://graph.microsoft.com"),
|
||||
("Microsoft Graph", Constants.GraphBaseUrl),
|
||||
};
|
||||
|
||||
public NetworkChangeDetector(
|
||||
|
||||
@@ -169,9 +169,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Transport
|
||||
/// <summary>
|
||||
/// Sends a critical error report to the emergency channel endpoint.
|
||||
/// Fire-and-forget: swallows all exceptions so a failure here never cascades.
|
||||
/// Uses a 5-second per-request timeout so it cannot block the upload loop.
|
||||
/// Default 5-second per-request timeout so it cannot block the upload loop; the
|
||||
/// emergency-break path passes a longer <paramref name="timeout"/> because its first
|
||||
/// request of a freshly booted process pays cold TLS + DNS + a possible backend
|
||||
/// cold start (observed 503s take well over 5 s).
|
||||
/// Returns <c>true</c> when the HTTP round-trip completed (the endpoint always
|
||||
/// answers 200), <c>false</c> on timeout/transport failure — the caller-side
|
||||
/// anti-flood layer uses this to decide whether the report counts as delivered.
|
||||
/// Virtual for tests.
|
||||
/// </summary>
|
||||
public async Task ReportAgentErrorAsync(AgentErrorReport report)
|
||||
public virtual async Task<bool> ReportAgentErrorAsync(AgentErrorReport report, TimeSpan? timeout = null)
|
||||
{
|
||||
try
|
||||
{
|
||||
@@ -190,16 +197,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Transport
|
||||
httpRequest.Headers.Add("X-Tenant-Id", report.TenantId);
|
||||
AddSecurityHeaders(httpRequest);
|
||||
|
||||
using (var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)))
|
||||
using (var cts = new CancellationTokenSource(timeout ?? TimeSpan.FromSeconds(5)))
|
||||
using (var response = await _httpClient.SendAsync(httpRequest, cts.Token).ConfigureAwait(false))
|
||||
{
|
||||
// Response status is deliberately ignored — endpoint always returns 200.
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger?.Debug($"ReportAgentErrorAsync: emergency channel failed: {ex.Message}");
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -57,19 +57,33 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Transport
|
||||
/// (intended to be called as fire-and-forget via <c>_ = reporter.TrySendAsync(...)</c>).
|
||||
///
|
||||
/// All anti-flood checks are applied synchronously before the async HTTP call,
|
||||
/// so repeated calls are cheap when suppressed.
|
||||
/// so repeated calls are cheap when suppressed. The anti-flood reservation is
|
||||
/// taken up front (so concurrent same-key calls cannot double-send) but ROLLED
|
||||
/// BACK when every attempt fails: the budget and the once-per-session dedup are
|
||||
/// meant to count DELIVERED reports — the previous mark-before-send made the very
|
||||
/// first transport failure permanently suppress that error key for the session
|
||||
/// (found via the 48h emergency-break report, which is a single shot at boot).
|
||||
///
|
||||
/// <paramref name="attempts"/>/<paramref name="perAttemptTimeout"/>/<paramref name="retryDelay"/>
|
||||
/// exist for callers whose process is about to exit and cannot come back later
|
||||
/// (emergency break): retries stay INSIDE the one reservation, so they never
|
||||
/// multiply against the per-session budget.
|
||||
/// </summary>
|
||||
public virtual async Task TrySendAsync(
|
||||
AgentErrorType errorType,
|
||||
string message,
|
||||
int? httpStatusCode = null,
|
||||
long? sequenceNumber = null)
|
||||
long? sequenceNumber = null,
|
||||
int attempts = 1,
|
||||
TimeSpan? perAttemptTimeout = null,
|
||||
TimeSpan? retryDelay = null)
|
||||
{
|
||||
// Deduplicate by error type + status code: same failure category sent only once per session.
|
||||
// All anti-flood checks are protected by a lock to prevent race conditions when
|
||||
// TrySendAsync is called concurrently (fire-and-forget from multiple threads).
|
||||
var key = $"{errorType}:{httpStatusCode}";
|
||||
int currentReportCount;
|
||||
DateTime? previousLastReportTime;
|
||||
|
||||
lock (_antiFloodLock)
|
||||
{
|
||||
@@ -92,6 +106,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Transport
|
||||
return;
|
||||
}
|
||||
|
||||
previousLastReportTime = _lastReportTime;
|
||||
_reportedKeys.Add(key);
|
||||
_reportCount++;
|
||||
_lastReportTime = DateTime.UtcNow;
|
||||
@@ -113,15 +128,40 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Transport
|
||||
Timestamp = DateTime.UtcNow
|
||||
};
|
||||
|
||||
var delivered = false;
|
||||
var totalAttempts = attempts < 1 ? 1 : attempts;
|
||||
try
|
||||
{
|
||||
await _apiClient.ReportAgentErrorAsync(report);
|
||||
for (var attempt = 1; attempt <= totalAttempts && !delivered; attempt++)
|
||||
{
|
||||
if (attempt > 1 && retryDelay.HasValue)
|
||||
{
|
||||
await Task.Delay(retryDelay.Value).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
delivered = await _apiClient.ReportAgentErrorAsync(report, perAttemptTimeout).ConfigureAwait(false);
|
||||
if (!delivered && totalAttempts > 1)
|
||||
{
|
||||
_logger?.Debug($"[EmergencyChannel] Attempt {attempt}/{totalAttempts} failed: {key}");
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// Swallow — emergency channel must never cascade failures into the caller
|
||||
_logger?.Debug($"[EmergencyChannel] Failed to send report: {ex.Message}");
|
||||
}
|
||||
|
||||
if (!delivered)
|
||||
{
|
||||
lock (_antiFloodLock)
|
||||
{
|
||||
_reportedKeys.Remove(key);
|
||||
_reportCount--;
|
||||
_lastReportTime = previousLastReportTime;
|
||||
}
|
||||
_logger?.Debug($"[EmergencyChannel] Report not delivered — reservation rolled back: {key}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -86,6 +86,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
// Production always passes null → Start() creates the real DeadlineScheduler.
|
||||
private readonly IDeadlineScheduler? _schedulerOverride;
|
||||
|
||||
// Live accessor for TenantConfiguration.SendTraceEvents — read per event by the
|
||||
// TelemetryEventEmitter so a mid-session remote-config merge takes effect. Null
|
||||
// (tests / legacy callers) means trace events always flow.
|
||||
private readonly Func<bool>? _traceEventsEnabled;
|
||||
|
||||
// Max-lifetime watchdog (M4.6.α). Timer is armed in Start() when _agentMaxLifetime != null.
|
||||
private System.Threading.Timer? _maxLifetimeTimer;
|
||||
private int _terminatedFired;
|
||||
@@ -166,7 +171,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
TimeSpan? terminalDrainTimeout = null,
|
||||
TimeSpan? agentMaxLifetime = null,
|
||||
int uploadBatchSize = 100,
|
||||
IDeadlineScheduler? schedulerOverride = null)
|
||||
IDeadlineScheduler? schedulerOverride = null,
|
||||
Func<bool>? traceEventsEnabled = null)
|
||||
{
|
||||
if (string.IsNullOrEmpty(sessionId)) throw new ArgumentException("SessionId is mandatory.", nameof(sessionId));
|
||||
if (string.IsNullOrEmpty(tenantId)) throw new ArgumentException("TenantId is mandatory.", nameof(tenantId));
|
||||
@@ -206,6 +212,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
_uploadBatchSize = uploadBatchSize;
|
||||
|
||||
_schedulerOverride = schedulerOverride;
|
||||
_traceEventsEnabled = traceEventsEnabled;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -402,7 +409,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
// strukturelle Abhängigkeit erlischt damit, Architektur-Baseline-Test gate
|
||||
// shrinks to exactly two permitted callers.
|
||||
_eventSequenceCounter = new EventSequenceCounter(_eventSequencePersistence);
|
||||
var eventEmitter = new TelemetryEventEmitter(_transport, _eventSequenceCounter, _sessionId, _tenantId);
|
||||
var eventEmitter = new TelemetryEventEmitter(
|
||||
_transport, _eventSequenceCounter, _sessionId, _tenantId, _traceEventsEnabled);
|
||||
_signalEmitter = new TelemetrySignalEmitter(_transport, _sessionId, _tenantId);
|
||||
_transitionEmitter = new TelemetryTransitionEmitter(_transport, _sessionId, _tenantId);
|
||||
_timelineEmitter = new EventTimelineEmitter(eventEmitter);
|
||||
|
||||
@@ -137,7 +137,24 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
|
||||
|
||||
private void OnPhaseChanged(object? sender, RealmJoinPhaseChangedEventArgs e)
|
||||
{
|
||||
// Phase change is observability-only: no DecisionSignalKind, just a timeline event.
|
||||
// Dual-emission like OnDetected/OnResolved: (1) the typed signal persists the phase
|
||||
// into RealmJoinFacts.LastDeploymentPhase (restart-safe) and lets the reducer detect
|
||||
// the aborted-RJ-ESP shape (phase left 100/101 for 200/210 without 110 — session
|
||||
// 224b2087); (2) the InformationalEvent keeps the existing timeline entry.
|
||||
var payload = new Dictionary<string, string>(StringComparer.Ordinal)
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = e.CurrentPhase.ToString(),
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = e.PreviousPhase.ToString(),
|
||||
};
|
||||
_ingress.Post(
|
||||
kind: DecisionSignalKind.RealmJoinPhaseChanged,
|
||||
occurredAtUtc: _clock.UtcNow,
|
||||
sourceOrigin: SourceLabel,
|
||||
evidence: BuildEvidence(
|
||||
"realmjoin-phase-changed-v1",
|
||||
$"RealmJoin DeploymentPhase {e.PreviousPhase} -> {e.CurrentPhase}"),
|
||||
payload: payload);
|
||||
|
||||
// Both raw int (deploymentPhase / previousPhase) and human-readable name
|
||||
// (deploymentPhaseName / previousPhaseName) flow through — int stays for downstream
|
||||
// filters / KQL, name fields make the message + UI rows readable without a lookup.
|
||||
|
||||
@@ -30,12 +30,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
|
||||
private readonly string _sessionId;
|
||||
private readonly string _tenantId;
|
||||
private readonly string _partitionKey;
|
||||
private readonly Func<bool>? _traceEventsEnabled;
|
||||
|
||||
public TelemetryEventEmitter(
|
||||
ITelemetryTransport transport,
|
||||
EventSequenceCounter sequenceCounter,
|
||||
string sessionId,
|
||||
string tenantId)
|
||||
string tenantId,
|
||||
Func<bool>? traceEventsEnabled = null)
|
||||
{
|
||||
if (string.IsNullOrEmpty(sessionId)) throw new ArgumentException("SessionId is mandatory.", nameof(sessionId));
|
||||
if (string.IsNullOrEmpty(tenantId)) throw new ArgumentException("TenantId is mandatory.", nameof(tenantId));
|
||||
@@ -44,13 +46,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
|
||||
_sessionId = sessionId;
|
||||
_tenantId = tenantId;
|
||||
_partitionKey = $"{tenantId}_{sessionId}";
|
||||
_traceEventsEnabled = traceEventsEnabled;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Weist Sequence zu, serialisiert im Legacy-Wire-Format und enqueued via Transport.
|
||||
/// Thread-safe (EventSequenceCounter + ITelemetryTransport.Enqueue beide unter Lock).
|
||||
/// <para>
|
||||
/// Returns <c>null</c> when the event was suppressed by the trace-event gate — the
|
||||
/// only drop path. Both permitted callers ignore the return value.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal TelemetryItem Emit(EnrollmentEvent evt)
|
||||
internal TelemetryItem? Emit(EnrollmentEvent evt)
|
||||
{
|
||||
if (evt == null) throw new ArgumentNullException(nameof(evt));
|
||||
if (string.IsNullOrEmpty(evt.EventType))
|
||||
@@ -58,6 +65,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
|
||||
throw new ArgumentException("EnrollmentEvent.EventType is mandatory.", nameof(evt));
|
||||
}
|
||||
|
||||
// Trace-event gate (TenantConfiguration.SendTraceEvents, default on). Dropped
|
||||
// before the sequence counter advances, so suppression leaves no gaps in the
|
||||
// per-session sequence and the timeline stays contiguous. Evaluated per event
|
||||
// rather than captured at construction: remote config is merged into the live
|
||||
// AgentConfiguration mid-session, and the gate must follow it.
|
||||
if (evt.Severity == EventSeverity.Trace && !TraceEventsEnabled())
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
evt.Sequence = _sequenceCounter.Next();
|
||||
if (string.IsNullOrEmpty(evt.SessionId)) evt.SessionId = _sessionId;
|
||||
if (string.IsNullOrEmpty(evt.TenantId)) evt.TenantId = _tenantId;
|
||||
@@ -77,5 +94,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
|
||||
|
||||
return _transport.Enqueue(draft);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Fail-open: no gate wired (legacy ctor / tests) or a throwing accessor means trace
|
||||
/// events flow. The gate suppresses diagnostics, so a broken accessor must not be able
|
||||
/// to silently blind the timeline.
|
||||
/// </summary>
|
||||
private bool TraceEventsEnabled()
|
||||
{
|
||||
if (_traceEventsEnabled == null) return true;
|
||||
try
|
||||
{
|
||||
return _traceEventsEnabled();
|
||||
}
|
||||
catch
|
||||
{
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ using System.Threading.Tasks;
|
||||
using AutopilotMonitor.Agent.V2.Core.Configuration;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime;
|
||||
using AutopilotMonitor.Agent.V2.Core.Orchestration;
|
||||
using AutopilotMonitor.Agent.V2.Core.Runtime;
|
||||
@@ -93,6 +94,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
|
||||
// correction between agent start and termination no longer skews bootToStartSeconds.
|
||||
private readonly ulong _bootAnchorTickMs;
|
||||
private readonly DateTime _bootAnchorUtc;
|
||||
// WinRT OOBE state at construction (handler is built at agent start) — corroborates
|
||||
// the late-start heuristic: "completed" here means OOBE was already over on arrival.
|
||||
private readonly string _oobeStateAtAgentStart;
|
||||
private int _handled;
|
||||
|
||||
public EnrollmentTerminationHandler(
|
||||
@@ -129,6 +133,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
|
||||
|
||||
_bootAnchorTickMs = ObservationCoverage.CurrentTickMs();
|
||||
_bootAnchorUtc = DateTime.UtcNow;
|
||||
_oobeStateAtAgentStart = OobeStateReader.Read();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -658,6 +663,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
|
||||
{ "agentUptimeSeconds", Math.Round(uptimeSeconds, 1) },
|
||||
{ "deviceBootUtc", deviceBootUtc.ToString("o") },
|
||||
{ "agentStartUtc", _agentStartTimeUtc.ToString("o") },
|
||||
{ "oobeStateAtAgentStart", _oobeStateAtAgentStart },
|
||||
{ "outcome", args.Outcome.ToString() },
|
||||
{ "note", "The agent's bootstrap (an Intune platform script) ran only near the end of the enrollment, so the agent observed the already-decided end-state rather than the failure window. Treat the diagnosis as a post-mortem; check for a platform/remediation script that hung ahead of the bootstrap (see script_timeout_suspected)." },
|
||||
},
|
||||
|
||||
@@ -132,7 +132,10 @@ namespace AutopilotMonitor.Agent.V2.Runtime
|
||||
whiteGloveSealingPatternIds: whiteGloveSealingPatternIds,
|
||||
drainInterval: drainInterval,
|
||||
agentMaxLifetime: agentMaxLifetime,
|
||||
uploadBatchSize: uploadBatchSize))
|
||||
uploadBatchSize: uploadBatchSize,
|
||||
// Live read: remote config is merged into agentConfig mid-session, so the
|
||||
// trace-event gate must follow it rather than snapshot at construction.
|
||||
traceEventsEnabled: () => agentConfig.SendTraceEvents))
|
||||
{
|
||||
using (var shutdown = new ManualResetEventSlim(false))
|
||||
using (var shutdownComplete = new ManualResetEventSlim(false))
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
using System;
|
||||
using System.Net.NetworkInformation;
|
||||
using System.Threading;
|
||||
using AutopilotMonitor.Agent.V2.Core.Configuration;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
@@ -15,28 +17,78 @@ namespace AutopilotMonitor.Agent.V2.Runtime
|
||||
/// classifier terminalize the session instead of waiting out the grace. Fully swallowed — a send
|
||||
/// failure (no network, no cert) must never delay the cleanup/exit the break exists to perform.
|
||||
/// See tasks/enrollment-status-reclassification.md.
|
||||
/// <para>
|
||||
/// Delivery hardening (2026-07-23): this is the FIRST request of a freshly booted process and
|
||||
/// the process never comes back — a miss is gone forever. The old single shot with a 5 s cap
|
||||
/// died on boot-time realities (NIC still associating, cold TLS/DNS, backend cold starts
|
||||
/// observed as >5 s 503s). Now: wait briefly for the NIC, then up to
|
||||
/// <see cref="SendAttempts"/> attempts with a per-attempt timeout that survives a Functions
|
||||
/// cold start. The session is 48 h dead — spending up to ~75 s here is irrelevant, losing the
|
||||
/// only signal that the agent gave up is not.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal static class EmergencyBreakReporter
|
||||
{
|
||||
private static readonly TimeSpan NetworkWaitMax = TimeSpan.FromSeconds(15);
|
||||
private const int SendAttempts = 3;
|
||||
private static readonly TimeSpan PerAttemptTimeout = TimeSpan.FromSeconds(15);
|
||||
private static readonly TimeSpan RetryDelay = TimeSpan.FromSeconds(5);
|
||||
// Upper bound for the blocking wait: NIC wait + 3×15s attempts + 2×5s delays + slack.
|
||||
private static readonly TimeSpan OverallBudget = TimeSpan.FromSeconds(75);
|
||||
|
||||
public static void TrySend(AgentConfiguration agentConfig, string agentVersion, AgentLogger logger)
|
||||
{
|
||||
try
|
||||
{
|
||||
WaitForNetwork(logger);
|
||||
|
||||
var auth = BackendClientFactory.BuildAuthClients(agentConfig, agentVersion, logger);
|
||||
var message =
|
||||
$"Agent absolute session-age emergency break fired (cap {agentConfig.AbsoluteMaxSessionHours}h) — cleaning up and exiting.";
|
||||
|
||||
// The process is about to exit, so block briefly on the otherwise fire-and-forget send
|
||||
// rather than let it be abandoned mid-flight. TrySendAsync swallows its own HTTP errors;
|
||||
// the auth bundle's HttpClients are intentionally left for process teardown.
|
||||
// The process is about to exit, so block on the otherwise fire-and-forget send
|
||||
// rather than let it be abandoned mid-flight. TrySendAsync swallows its own HTTP
|
||||
// errors and keeps the retries inside one anti-flood reservation; the auth
|
||||
// bundle's HttpClients are intentionally left for process teardown.
|
||||
auth.EmergencyReporter
|
||||
.TrySendAsync(AgentErrorType.SessionAgeEmergencyBreak, message)
|
||||
.Wait(TimeSpan.FromSeconds(5));
|
||||
.TrySendAsync(
|
||||
AgentErrorType.SessionAgeEmergencyBreak,
|
||||
message,
|
||||
attempts: SendAttempts,
|
||||
perAttemptTimeout: PerAttemptTimeout,
|
||||
retryDelay: RetryDelay)
|
||||
.Wait(OverallBudget);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
logger?.Debug($"Emergency-break report send failed (best-effort): {ex.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Boot-time NIC grace: the break fires during bootstrap, often seconds after boot while
|
||||
/// Wi-Fi is still associating. Polls the cheap link-level signal — it cannot prove backend
|
||||
/// reachability (the send attempts handle that), it only avoids burning the first attempts
|
||||
/// into a known-dead link. Best-effort: probe errors abort the wait, never the send.
|
||||
/// </summary>
|
||||
private static void WaitForNetwork(AgentLogger logger)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (NetworkInterface.GetIsNetworkAvailable()) return;
|
||||
|
||||
logger?.Info($"Emergency-break report: no network link yet — waiting up to {NetworkWaitMax.TotalSeconds:F0}s.");
|
||||
var deadline = DateTime.UtcNow + NetworkWaitMax;
|
||||
while (DateTime.UtcNow < deadline)
|
||||
{
|
||||
Thread.Sleep(TimeSpan.FromSeconds(1));
|
||||
if (NetworkInterface.GetIsNetworkAvailable()) return;
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
logger?.Debug($"Emergency-break report: network probe failed ({ex.Message}) — sending anyway.");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ using System;
|
||||
using System.Collections.Generic;
|
||||
using AutopilotMonitor.Agent.V2.Core.Configuration;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime;
|
||||
using AutopilotMonitor.Agent.V2.Core.Orchestration;
|
||||
using AutopilotMonitor.Agent.V2.Core.Security;
|
||||
@@ -68,6 +69,10 @@ namespace AutopilotMonitor.Agent.V2.Runtime
|
||||
{ "configVersion", configVersion },
|
||||
{ "remoteConfigFetched", outcome == RemoteConfigFetchOutcome.Succeeded },
|
||||
{ "remoteConfigOutcome", outcome.ToString() },
|
||||
// WinRT OOBE state sampled at this start. Together with previousExitType
|
||||
// this makes late starts provable: "completed" on a first_run means OOBE
|
||||
// was already over when the agent arrived (e.g. IME queue blocked).
|
||||
{ "oobeStateAtAgentStart", OobeStateReader.Read() },
|
||||
};
|
||||
|
||||
if (!string.IsNullOrEmpty(previousExit?.CrashExceptionType))
|
||||
|
||||
@@ -241,6 +241,199 @@ namespace AutopilotMonitor.DecisionCore.Tests
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_timeout");
|
||||
}
|
||||
|
||||
// ============================================================== Aborted first deployment (session 224b2087)
|
||||
|
||||
[Fact]
|
||||
public void PhaseChanged_leaving_first_deployment_for_200_releases_gate_with_warning()
|
||||
{
|
||||
// Session 224b2087: interactive logon 17 s after phase 101 → RJ reclassifies the run
|
||||
// as secondary-user deployment and writes 200/210, never 110. The 101 -> 200
|
||||
// transition is impossible in a healthy first deployment, so it releases the gate
|
||||
// with Outcome=FirstDeploymentIncomplete and a Warning timeline entry.
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.DesktopArrived, T0.AddMinutes(6))).NewState;
|
||||
Assert.NotEqual(SessionStage.Finalizing, state.Stage);
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(10),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "200",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "101",
|
||||
}));
|
||||
|
||||
Assert.Equal(SessionStage.Finalizing, step.NewState.Stage);
|
||||
Assert.Equal("FirstDeploymentIncomplete", step.NewState.RealmJoinFacts.Outcome!.Value);
|
||||
Assert.NotNull(step.NewState.RealmJoinFacts.ResolvedUtc);
|
||||
Assert.Equal(200, step.NewState.RealmJoinFacts.LastDeploymentPhase!.Value);
|
||||
|
||||
// Hard-timeout deadline cancelled in state AND as a scheduler effect.
|
||||
Assert.DoesNotContain(step.NewState.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
Assert.Contains(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.CancelDeadline &&
|
||||
e.CancelDeadlineName == DeadlineNames.RealmJoinTimeout);
|
||||
|
||||
// Warning timeline entry names both phases.
|
||||
var warning = Assert.Single(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_first_deployment_incomplete");
|
||||
Assert.Equal("Warning", warning.Parameters!["severity"]);
|
||||
Assert.Equal("101", warning.Parameters["previousPhase"]);
|
||||
Assert.Equal("200", warning.Parameters["deploymentPhase"]);
|
||||
Assert.Contains("110", warning.Parameters["message"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void PhaseChanged_within_first_deployment_window_is_bookkeeping_only()
|
||||
{
|
||||
// 100 -> 101 stays inside the first-deployment window — no release, but the phase
|
||||
// fact must advance (it is the restart-safe witness for the abort rule).
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "100" })).NewState;
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(6),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "100",
|
||||
}));
|
||||
|
||||
Assert.Equal(101, step.NewState.RealmJoinFacts.LastDeploymentPhase!.Value);
|
||||
Assert.Null(step.NewState.RealmJoinFacts.Outcome);
|
||||
Assert.Contains(step.NewState.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
Assert.Empty(step.Effects);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void PhaseChanged_from_completed_deployment_start_does_not_release()
|
||||
{
|
||||
// Session 6f1959c0 protection: RJ already stood at CompletedDeployment (210) when
|
||||
// the agent booted and only deployed AFTERWARDS (210 -> 200 -> 210). Neither
|
||||
// transition starts from the first-deployment window, so the abort rule must not
|
||||
// fire — this ambiguous shape keeps waiting for 110 / the hard timeout (status quo).
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "210" })).NewState;
|
||||
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(6),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "200",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "210",
|
||||
})).NewState;
|
||||
Assert.Null(state.RealmJoinFacts.Outcome);
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(7),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "210",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "200",
|
||||
}));
|
||||
|
||||
Assert.Null(step.NewState.RealmJoinFacts.Outcome);
|
||||
Assert.Contains(step.NewState.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Detected_replay_after_restart_with_deployment_phase_releases_gate()
|
||||
{
|
||||
// Agent restarted between the 101 and 200 observations: the watcher re-fires
|
||||
// Detected with the CURRENT phase (200) and no PhaseChanged ever carries the
|
||||
// transition. The persisted LastDeploymentPhase=101 is the witness — the Detected
|
||||
// replay path must release the gate exactly like the PhaseChanged path.
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.DesktopArrived, T0.AddMinutes(6))).NewState;
|
||||
Assert.NotEqual(SessionStage.Finalizing, state.Stage);
|
||||
Assert.Equal(101, state.RealmJoinFacts.LastDeploymentPhase!.Value);
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(12),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "200" }));
|
||||
|
||||
Assert.Equal(SessionStage.Finalizing, step.NewState.Stage);
|
||||
Assert.Equal("FirstDeploymentIncomplete", step.NewState.RealmJoinFacts.Outcome!.Value);
|
||||
Assert.Contains(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_first_deployment_incomplete");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Timeout_message_reports_persisted_last_phase_instead_of_zero()
|
||||
{
|
||||
// Before RealmJoinPhaseChanged became a typed signal, LastDeploymentPhase was only
|
||||
// written at detection time — every realmjoin_timeout claimed "last phase: 0" no
|
||||
// matter how far RJ actually got, hiding this whole failure class from ops.
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "100" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(6),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "100",
|
||||
})).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.DesktopArrived, T0.AddMinutes(7))).NewState;
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(8, DecisionSignalKind.DeadlineFired, T0.AddMinutes(65),
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout }));
|
||||
|
||||
var timeout = Assert.Single(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_timeout");
|
||||
Assert.Contains("last phase: 101", timeout.Parameters!["message"]);
|
||||
Assert.Equal("101", timeout.Parameters["lastSeenPhase"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void SelfDeploying_deferred_terminal_released_by_first_deployment_incomplete()
|
||||
{
|
||||
// Same deferred-release contract as RealmJoinResolved: when the SelfDeploying
|
||||
// terminal was deferred on the closed RJ gate, the aborted-first-deployment release
|
||||
// must complete the session directly (Completed + enrollment_complete), with the
|
||||
// Warning entry leading the effects.
|
||||
var engine = new DecisionEngine();
|
||||
var state = DecisionState.CreateInitial("rj-sd-3", "rj-tenant", T0);
|
||||
state = engine.Reduce(state, MakeSignal(0, DecisionSignalKind.SessionStarted, T0)).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(1, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(1),
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.EspPhase] = "DeviceSetup" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(2, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(2),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(3, DecisionSignalKind.DeviceSetupProvisioningComplete, T0.AddMinutes(3))).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(4, DecisionSignalKind.DeadlineFired, T0.AddMinutes(8),
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.DeviceOnlyEspDetection })).NewState;
|
||||
Assert.True(state.RealmJoinFacts.SelfDeployingDeferredCompletion?.Value);
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(10),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "200",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "101",
|
||||
}));
|
||||
|
||||
Assert.Equal(SessionStage.Completed, step.NewState.Stage);
|
||||
Assert.Equal(SessionOutcome.EnrollmentComplete, step.NewState.Outcome);
|
||||
Assert.Equal("FirstDeploymentIncomplete", step.NewState.RealmJoinFacts.Outcome!.Value);
|
||||
Assert.Contains(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_first_deployment_incomplete");
|
||||
Assert.Contains(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "enrollment_complete");
|
||||
}
|
||||
|
||||
// ============================================================== SelfDeploying flow
|
||||
|
||||
[Fact]
|
||||
|
||||
@@ -55,7 +55,7 @@ public class AuthFunctionSideEffectTests
|
||||
{ CallBase = false };
|
||||
|
||||
var previewMock = new Mock<PreviewWhitelistService>(
|
||||
configRepo, cache, Mock.Of<ILogger<PreviewWhitelistService>>())
|
||||
configRepo, cache, Mock.Of<ILogger<PreviewWhitelistService>>(), _tenantConfigMock.Object)
|
||||
{ CallBase = false };
|
||||
|
||||
_telegramMock = new Mock<TelegramNotificationService>(
|
||||
|
||||
@@ -113,9 +113,11 @@ public class EnrollmentTimeoutClassifierTests
|
||||
// Negative Hello terminals leave the agent waiting — must NOT count as resolved.
|
||||
Assert.False(EnrollmentTimeoutClassifier.ExtractRollup(new[] { Evt("hello_provisioning_failed") }).HelloResolved);
|
||||
Assert.False(EnrollmentTimeoutClassifier.ExtractRollup(new[] { Evt("hello_completion_timeout") }).HelloResolved);
|
||||
// Both RealmJoin gate terminals count as resolved (phase 110 or 60-min hard timeout).
|
||||
// All RealmJoin gate terminals count as resolved (phase 110, aborted first
|
||||
// deployment — session 224b2087 — or 60-min hard timeout).
|
||||
Assert.True(EnrollmentTimeoutClassifier.ExtractRollup(new[] { Evt("realmjoin_resolved") }).RealmJoinResolved);
|
||||
Assert.True(EnrollmentTimeoutClassifier.ExtractRollup(new[] { Evt("realmjoin_timeout") }).RealmJoinResolved);
|
||||
Assert.True(EnrollmentTimeoutClassifier.ExtractRollup(new[] { Evt("realmjoin_first_deployment_incomplete") }).RealmJoinResolved);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Enforces the URL registry: every well-known own or Microsoft host must be
|
||||
/// referenced through <c>AutopilotMonitor.Shared.Constants</c>, never as a
|
||||
/// repeated string literal. The EU cutover missed two hardcoded copies of the
|
||||
/// blob host (LatestVersionsService, ValidateBootstrapCodeFunction) precisely
|
||||
/// because they did not go through the registry — this test makes that class
|
||||
/// of drift a build failure instead of a production surprise.
|
||||
///
|
||||
/// Scope: non-test C# source under src/. Comment lines are ignored (docs and
|
||||
/// examples may cite URLs); test projects are excluded because tests use
|
||||
/// literals deliberately as independent oracles.
|
||||
/// </summary>
|
||||
public class HardcodedUrlGuardTests
|
||||
{
|
||||
private static readonly string[] EnforcedHosts =
|
||||
{
|
||||
"portal.autopilotmonitor.com",
|
||||
"www.autopilotmonitor.com",
|
||||
"docs.autopilotmonitor.com",
|
||||
"download.autopilotmonitor.com",
|
||||
"mcp.autopilotmonitor.com",
|
||||
"autopilotmonitor-api-eu.azurewebsites.net",
|
||||
"autopilotmonitor.blob.core.windows.net",
|
||||
"autopilotmonitoreu.blob.core.windows.net",
|
||||
"graph.microsoft.com",
|
||||
"login.microsoftonline.com",
|
||||
};
|
||||
|
||||
/// <summary>Files allowed to carry the literals — the registry itself.</summary>
|
||||
private static readonly string[] RegistryFiles =
|
||||
{
|
||||
Path.Combine("src", "Shared", "AutopilotMonitor.Shared", "Constants.cs"),
|
||||
};
|
||||
|
||||
[Fact]
|
||||
public void WellKnownHosts_OnlyAppearInTheRegistry()
|
||||
{
|
||||
var repoRoot = FindRepoRoot();
|
||||
var srcDir = Path.Combine(repoRoot, "src");
|
||||
var violations = new List<string>();
|
||||
|
||||
foreach (var file in Directory.EnumerateFiles(srcDir, "*.cs", SearchOption.AllDirectories))
|
||||
{
|
||||
var relative = Path.GetRelativePath(repoRoot, file);
|
||||
var normalized = relative.Replace('/', Path.DirectorySeparatorChar);
|
||||
|
||||
// bin/obj artifacts and generated files
|
||||
if (normalized.Contains($"{Path.DirectorySeparatorChar}bin{Path.DirectorySeparatorChar}") ||
|
||||
normalized.Contains($"{Path.DirectorySeparatorChar}obj{Path.DirectorySeparatorChar}"))
|
||||
continue;
|
||||
|
||||
// Test projects: literals there are deliberate independent oracles.
|
||||
if (normalized.Contains("Tests"))
|
||||
continue;
|
||||
|
||||
if (RegistryFiles.Any(r => normalized.EndsWith(r, StringComparison.OrdinalIgnoreCase)))
|
||||
continue;
|
||||
|
||||
var lines = File.ReadAllLines(file);
|
||||
for (var i = 0; i < lines.Length; i++)
|
||||
{
|
||||
var trimmed = lines[i].TrimStart();
|
||||
// Comment approximation: full-line // and /// comments plus the
|
||||
// body lines of XML docs and block comments. A URL after code on
|
||||
// the same line still counts — that is the case being guarded.
|
||||
if (trimmed.StartsWith("//") || trimmed.StartsWith("*") || trimmed.StartsWith("/*"))
|
||||
continue;
|
||||
|
||||
foreach (var host in EnforcedHosts)
|
||||
{
|
||||
if (trimmed.Contains(host, StringComparison.OrdinalIgnoreCase))
|
||||
violations.Add($"{relative}:{i + 1}: {host}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Assert.True(violations.Count == 0,
|
||||
"Hardcoded well-known host(s) found outside the URL registry " +
|
||||
"(use AutopilotMonitor.Shared.Constants.*BaseUrl instead):\n " +
|
||||
string.Join("\n ", violations));
|
||||
}
|
||||
|
||||
private static string FindRepoRoot()
|
||||
{
|
||||
var dir = new DirectoryInfo(AppContext.BaseDirectory);
|
||||
while (dir != null && !File.Exists(Path.Combine(dir.FullName, "AutopilotMonitor.sln")))
|
||||
{
|
||||
dir = dir.Parent;
|
||||
}
|
||||
Assert.NotNull(dir);
|
||||
return dir!.FullName;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
using AutopilotMonitor.Functions.Helpers;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
public class LogSanitizerTests
|
||||
{
|
||||
[Fact]
|
||||
public void Clean_WithCrlf_StripsNewlines()
|
||||
{
|
||||
// The core log-forging vector: a value that injects a fake log line.
|
||||
var forged = "/api/sessions\r\n[Auth Middleware] FAKE: admin granted";
|
||||
var cleaned = LogSanitizer.Clean(forged);
|
||||
Assert.Equal("/api/sessions[Auth Middleware] FAKE: admin granted", cleaned);
|
||||
Assert.DoesNotContain('\r', cleaned!);
|
||||
Assert.DoesNotContain('\n', cleaned!);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Clean_WithBareLineFeedOrCarriageReturn_StripsBoth()
|
||||
{
|
||||
Assert.Equal("ab", LogSanitizer.Clean("a\nb"));
|
||||
Assert.Equal("ab", LogSanitizer.Clean("a\rb"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Clean_WithOtherControlChars_StripsThem()
|
||||
{
|
||||
// Tab (9), NUL (0) and ESC (27, terminal escape) are all removed. Built
|
||||
// from char codes so the source file carries no literal control bytes.
|
||||
var input = $"a{(char)9}b{(char)0}c{(char)27}de";
|
||||
Assert.Equal("abcde", LogSanitizer.Clean(input));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Clean_WithUnicodeLineSeparators_StripsThem()
|
||||
{
|
||||
// U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR are Unicode categories
|
||||
// Zl/Zp, not Cc — so a \p{Cc}-only pass would let them through, and a log viewer
|
||||
// or JavaScript-side reader would still break the line on them. Built from char
|
||||
// codes so the source file carries no literal separator bytes.
|
||||
var forged = $"/api/sessions{(char)0x2028}[Auth Middleware] FAKE: admin granted";
|
||||
var cleaned = LogSanitizer.Clean(forged);
|
||||
|
||||
Assert.Equal("/api/sessions[Auth Middleware] FAKE: admin granted", cleaned);
|
||||
Assert.DoesNotContain((char)0x2028, cleaned!);
|
||||
Assert.Equal("ab", LogSanitizer.Clean($"a{(char)0x2029}b"));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("/api/sessions/abc-123/events")]
|
||||
[InlineData("user@contoso.com")]
|
||||
[InlineData("11111111-2222-3333-4444-555555555555")]
|
||||
[InlineData("GET")]
|
||||
public void Clean_WithLegitimateValue_ReturnsUnchanged(string value)
|
||||
{
|
||||
// Real paths, UPNs, GUIDs and methods never contain control chars, so
|
||||
// tracing fidelity is fully preserved.
|
||||
Assert.Equal(value, LogSanitizer.Clean(value));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Clean_WithNull_ReturnsNull()
|
||||
{
|
||||
Assert.Null(LogSanitizer.Clean(null));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Clean_WithEmpty_ReturnsEmpty()
|
||||
{
|
||||
Assert.Equal("", LogSanitizer.Clean(""));
|
||||
}
|
||||
}
|
||||
+58
@@ -1,4 +1,5 @@
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
@@ -87,4 +88,61 @@ public class MaintenanceServiceExcessiveEventAutoActionTests
|
||||
// — better to no-op than block/kill on an unintended config.
|
||||
Assert.Null(MaintenanceService.DecideAutoAction(eventCount: 5000, autoMode: mode, autoThreshold: 2500));
|
||||
}
|
||||
|
||||
// ── Status eligibility ────────────────────────────────────────────────────
|
||||
// A device may only be blocked over a session that can still upload. Without this gate
|
||||
// a session that finished days ago with a high EventCount got its device blocked on the
|
||||
// next sweep, long after it stopped sending.
|
||||
|
||||
[Fact]
|
||||
public void IsAutoActionEligible_InProgress_IsEligible()
|
||||
{
|
||||
Assert.True(MaintenanceService.IsAutoActionEligible(SessionStatus.InProgress));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(SessionStatus.Succeeded)]
|
||||
[InlineData(SessionStatus.Failed)]
|
||||
[InlineData(SessionStatus.Incomplete)]
|
||||
[InlineData(SessionStatus.Stalled)]
|
||||
[InlineData(SessionStatus.AwaitingUser)]
|
||||
[InlineData(SessionStatus.Pending)]
|
||||
[InlineData(SessionStatus.Unknown)]
|
||||
public void IsAutoActionEligible_EverythingElse_IsRejected(SessionStatus status)
|
||||
{
|
||||
// Matches the time-window watchdog's `Status eq 'InProgress'` filter, so neither
|
||||
// detector can block a device over a session that no longer sends. Stalled and
|
||||
// AwaitingUser can heal back to InProgress — then the next sweep acts.
|
||||
Assert.False(MaintenanceService.IsAutoActionEligible(status));
|
||||
}
|
||||
|
||||
// ── Status parsing feeding that gate ──────────────────────────────────────
|
||||
// The runaway query projects a subset of columns. SessionStatus.InProgress is ordinal 0,
|
||||
// so an unparsed Status would leave every row looking in-progress and the gate above
|
||||
// silently always-true — these pin the fail-closed mapping.
|
||||
|
||||
[Theory]
|
||||
[InlineData("InProgress", SessionStatus.InProgress)]
|
||||
[InlineData("inprogress", SessionStatus.InProgress)]
|
||||
[InlineData("Succeeded", SessionStatus.Succeeded)]
|
||||
[InlineData("AwaitingUser", SessionStatus.AwaitingUser)]
|
||||
public void ParseStatusForAutoAction_KnownNames_RoundTrip(string stored, SessionStatus expected)
|
||||
{
|
||||
Assert.Equal(expected, TableStorageService.ParseStatusForAutoAction(stored));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(null)] // column absent from the projection or the row
|
||||
[InlineData("")]
|
||||
[InlineData(" ")]
|
||||
[InlineData("Enrolling")] // status name that never existed
|
||||
[InlineData("0")] // Enum.TryParse would map this onto InProgress and open the gate
|
||||
[InlineData("3")]
|
||||
[InlineData("-1")]
|
||||
[InlineData("InProgress,Failed")] // flags-style combination is not a real status
|
||||
public void ParseStatusForAutoAction_UnusableInput_IsUnknown(string? stored)
|
||||
{
|
||||
Assert.Equal(SessionStatus.Unknown, TableStorageService.ParseStatusForAutoAction(stored));
|
||||
Assert.False(MaintenanceService.IsAutoActionEligible(TableStorageService.ParseStatusForAutoAction(stored)));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -117,7 +117,6 @@ public class OffboardingSessionEnumeratorTests
|
||||
public Task<List<SessionSummary>> GetLegacyTimeoutFailedSessionsAsync(string tenantId, int maxResults) => throw new NotImplementedException();
|
||||
public Task<List<SessionSummary>> GetSessionsLeanAsync(string tenantId) => throw new NotImplementedException();
|
||||
public Task<List<SessionSummary>> GetAgentSilentSessionsAsync(string tenantId, DateTime silenceCutoff, DateTime hardCutoff) => throw new NotImplementedException();
|
||||
public Task<List<SessionSummary>> GetExcessiveDataSendersAsync(string tenantId, DateTime windowCutoff, int maxSessionWindowHours) => throw new NotImplementedException();
|
||||
public Task<List<string>> GetAllTenantIdsAsync() => throw new NotImplementedException();
|
||||
public Task<int> DeleteSessionEventsAsync(string tenantId, string sessionId) => throw new NotImplementedException();
|
||||
public Task<int> DeleteSessionRuleResultsAsync(string tenantId, string sessionId) => throw new NotImplementedException();
|
||||
|
||||
@@ -868,4 +868,149 @@ public class PolicyEnforcementMiddlewareTests
|
||||
Assert.False(result.Allowed);
|
||||
Assert.Equal(403, result.StatusCode);
|
||||
}
|
||||
|
||||
// ── Empty-UPN contract for device / anonymous routes ───────────────────────────
|
||||
//
|
||||
// UserRateLimitMiddleware buckets on RequestContext.UserPrincipalName and skips only when it is
|
||||
// EMPTY. Publishing the "anonymous" placeholder there collapsed every agent of every tenant into a
|
||||
// single shared user_ratelimit_anonymous bucket, which 429'd agent telemetry fleet-wide once the
|
||||
// per-user limit was reached. These tests pin the contract: no JWT ⇒ empty UPN.
|
||||
|
||||
[Theory]
|
||||
[InlineData("POST", "/api/agent/telemetry")]
|
||||
[InlineData("GET", "/api/agent/config")]
|
||||
[InlineData("POST", "/api/agent/register-session")]
|
||||
[InlineData("POST", "/api/agent/upload-url")]
|
||||
[InlineData("POST", "/api/agent/error")]
|
||||
[InlineData("POST", "/api/bootstrap/register-session")]
|
||||
[InlineData("GET", "/api/bootstrap/config")]
|
||||
[InlineData("POST", "/api/bootstrap/error")]
|
||||
public async Task DeviceRoute_WithoutJwt_PublishesEmptyUpn(string method, string path)
|
||||
{
|
||||
var h = BuildHarness();
|
||||
|
||||
var result = await h.Middleware.DecideAsync(method, path, null, principal: null);
|
||||
|
||||
Assert.True(result.Allowed);
|
||||
// The rate-limit bucket key: empty means "no user bucket applies to this call".
|
||||
Assert.Equal(string.Empty, result.Context!.UserPrincipalName);
|
||||
// The placeholder still renders in log lines — it just never reaches the RequestContext.
|
||||
Assert.Equal("anonymous", result.UserIdentifier);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("GET", "/api/health")]
|
||||
[InlineData("GET", "/api/stats/platform")]
|
||||
[InlineData("POST", "/api/agent/distress")]
|
||||
[InlineData("GET", "/api/diagnostics/download")]
|
||||
public async Task AnonymousRoute_WithoutJwt_PublishesEmptyUpn(string method, string path)
|
||||
{
|
||||
var h = BuildHarness();
|
||||
|
||||
var result = await h.Middleware.DecideAsync(method, path, null, principal: null);
|
||||
|
||||
Assert.True(result.Allowed);
|
||||
Assert.Equal(string.Empty, result.Context!.UserPrincipalName);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task AuthenticatedRoute_StillPublishesTheRealUpn()
|
||||
{
|
||||
// Guard the other direction: the fix must not blank the UPN for genuine JWT callers, or
|
||||
// per-user rate limiting and presence tracking would silently stop working.
|
||||
const string upn = "admin@contoso.com";
|
||||
var h = BuildHarness();
|
||||
h.AsTenantAdmin(TenantA, upn);
|
||||
|
||||
var result = await h.Middleware.DecideAsync("GET", "/api/notifications", null, AuthedPrincipal(TenantA, upn));
|
||||
|
||||
Assert.True(result.Allowed);
|
||||
Assert.Equal(upn, result.Context!.UserPrincipalName);
|
||||
Assert.Equal(upn, result.Context!.CallerId);
|
||||
}
|
||||
|
||||
// ── CallerId: throttle identity survives tokens that carry no UPN ──────────────
|
||||
//
|
||||
// AuthenticationMiddleware validates issuer/audience/lifetime/signature but requires NO upn and no
|
||||
// scp, so an app-only (client-credentials) token authenticates and reaches every AuthenticatedUser
|
||||
// route (feedback, realtime/negotiate, progress/*, health/detailed, …). UserRateLimitMiddleware
|
||||
// skips on an EMPTY throttle identity — so if CallerId fell back to the (absent) UPN, those tokens
|
||||
// would bypass rate limiting entirely. These tests pin the fallback chain.
|
||||
|
||||
/// <summary>Principal with a tid but NO upn/preferred_username — shaped like an app-only token.</summary>
|
||||
private static ClaimsPrincipal AppOnlyPrincipal(string tenantId, params (string Type, string Value)[] claims)
|
||||
{
|
||||
var all = new List<Claim> { new("tid", tenantId) };
|
||||
all.AddRange(claims.Select(c => new Claim(c.Type, c.Value)));
|
||||
return new ClaimsPrincipal(new ClaimsIdentity(all, "TestAuth"));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("oid", "aaaaaaaa-0000-0000-0000-000000000001")]
|
||||
[InlineData("http://schemas.microsoft.com/identity/claims/objectidentifier", "aaaaaaaa-0000-0000-0000-000000000002")]
|
||||
[InlineData("sub", "subject-abc")]
|
||||
[InlineData("appid", "bbbbbbbb-0000-0000-0000-000000000003")]
|
||||
[InlineData("azp", "cccccccc-0000-0000-0000-000000000004")]
|
||||
public async Task AuthenticatedUserRoute_TokenWithoutUpn_StillGetsAThrottleIdentity(string claimType, string claimValue)
|
||||
{
|
||||
var h = BuildHarness();
|
||||
|
||||
// /api/feedback/status is AuthenticatedUser — admits ANY valid principal, role unresolved.
|
||||
var result = await h.Middleware.DecideAsync(
|
||||
"GET", "/api/feedback/status", null, AppOnlyPrincipal(TenantA, (claimType, claimValue)));
|
||||
|
||||
Assert.True(result.Allowed);
|
||||
// No UPN to report — audit/presence correctly see nothing...
|
||||
Assert.Equal(string.Empty, result.Context!.UserPrincipalName);
|
||||
// ...but the throttle still has something stable to bucket on.
|
||||
Assert.Equal(claimValue, result.Context!.CallerId);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task AuthenticatedUserRoute_TokenWithoutUpn_PrefersOidOverSubAndAppId()
|
||||
{
|
||||
var h = BuildHarness();
|
||||
var principal = AppOnlyPrincipal(TenantA,
|
||||
("appid", "cccccccc-0000-0000-0000-000000000009"),
|
||||
("sub", "subject-xyz"),
|
||||
("oid", "aaaaaaaa-0000-0000-0000-00000000000a"));
|
||||
|
||||
var result = await h.Middleware.DecideAsync("GET", "/api/feedback/status", null, principal);
|
||||
|
||||
Assert.True(result.Allowed);
|
||||
Assert.Equal("aaaaaaaa-0000-0000-0000-00000000000a", result.Context!.CallerId);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task AuthenticatedUserRoute_TokenWithNoIdentifyingClaim_FailsClosedToASharedBucket()
|
||||
{
|
||||
// Pathological (every real Entra token has a sub): must NOT resolve to empty, because empty
|
||||
// means "skip the limiter". A shared bucket is the safe answer.
|
||||
var h = BuildHarness();
|
||||
|
||||
var result = await h.Middleware.DecideAsync(
|
||||
"GET", "/api/feedback/status", null, AppOnlyPrincipal(TenantA));
|
||||
|
||||
Assert.True(result.Allowed);
|
||||
Assert.NotEqual(string.Empty, result.Context!.CallerId);
|
||||
Assert.Equal(PolicyEnforcementMiddleware.UnidentifiedCallerBucket, result.Context!.CallerId);
|
||||
// Never reuse the log placeholder as a bucket key — that was the original fleet-wide collapse.
|
||||
Assert.NotEqual("anonymous", result.Context!.CallerId);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("POST", "/api/agent/telemetry")]
|
||||
[InlineData("GET", "/api/health")]
|
||||
[InlineData("GET", "/api/stats/platform")]
|
||||
public async Task UnauthenticatedRoute_HasEmptyCallerId(string method, string path)
|
||||
{
|
||||
// Empty CallerId is the ONLY signal that skips the user limiter — it must mean
|
||||
// "no JWT at all", never "authenticated but unnamed".
|
||||
var h = BuildHarness();
|
||||
|
||||
var result = await h.Middleware.DecideAsync(method, path, null, principal: null);
|
||||
|
||||
Assert.True(result.Allowed);
|
||||
Assert.Equal(string.Empty, result.Context!.CallerId);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,15 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Net.Http;
|
||||
using System.Threading.Tasks;
|
||||
using AutopilotMonitor.Functions.Functions.Ingest;
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Functions.Services.Notifications;
|
||||
using AutopilotMonitor.Shared.DataAccess;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Moq;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
@@ -80,4 +88,116 @@ public class ReportAgentErrorFunctionTests
|
||||
var evt = ReportAgentErrorFunction.BuildAgentEmergencyBreakEvent(Report(Now, message: ""), TenantId, null!, Now);
|
||||
Assert.Contains("emergency break", evt.Message, StringComparison.OrdinalIgnoreCase);
|
||||
}
|
||||
|
||||
// =========================================================================
|
||||
// MaterializeEmergencyBreakArtifactsAsync — the artifact set per break report
|
||||
// (2026-07-23 hardening): timeline event + cross-session EventType index row +
|
||||
// AgentEmergencyBreak ops event, once per session. The index upsert is load-bearing:
|
||||
// StoreEventsBatchAsync alone leaves the event invisible to every search-by-eventType
|
||||
// surface (found in the 2026-07-22 incident analysis).
|
||||
// =========================================================================
|
||||
|
||||
private sealed class Harness
|
||||
{
|
||||
public readonly Mock<ISessionRepository> SessionRepo = new();
|
||||
public readonly List<OpsEventEntry> OpsEvents = new();
|
||||
public readonly OpsEventService OpsService;
|
||||
|
||||
public Harness(List<EnrollmentEvent>? existingEvents = null)
|
||||
{
|
||||
SessionRepo.Setup(r => r.GetSessionEventsAsync(TenantId, SessionId, It.IsAny<int>()))
|
||||
.ReturnsAsync(existingEvents ?? new List<EnrollmentEvent>());
|
||||
SessionRepo.Setup(r => r.StoreEventsBatchAsync(It.IsAny<List<EnrollmentEvent>>()))
|
||||
.ReturnsAsync((List<EnrollmentEvent> e) => e);
|
||||
SessionRepo.Setup(r => r.UpsertEventTypeIndexBatchAsync(
|
||||
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<IEnumerable<EnrollmentEvent>>()))
|
||||
.Returns(Task.CompletedTask);
|
||||
|
||||
var opsRepo = new Mock<IOpsEventRepository>();
|
||||
opsRepo.Setup(r => r.SaveOpsEventAsync(It.IsAny<OpsEventEntry>()))
|
||||
.Callback<OpsEventEntry>(e => { lock (OpsEvents) OpsEvents.Add(e); })
|
||||
.Returns(Task.CompletedTask);
|
||||
|
||||
var adminConfig = new Mock<AdminConfigurationService>(
|
||||
Mock.Of<IConfigRepository>(), NullLogger<AdminConfigurationService>.Instance,
|
||||
new MemoryCache(new MemoryCacheOptions()));
|
||||
var alertDispatch = new OpsAlertDispatchService(
|
||||
adminConfig.Object,
|
||||
new TelegramNotificationService(new HttpClient(), Mock.Of<IConfigRepository>(),
|
||||
NullLogger<TelegramNotificationService>.Instance),
|
||||
new WebhookNotificationService(new HttpClient(),
|
||||
NullLogger<WebhookNotificationService>.Instance),
|
||||
NullLogger<OpsAlertDispatchService>.Instance);
|
||||
OpsService = new OpsEventService(opsRepo.Object, NullLogger<OpsEventService>.Instance, alertDispatch);
|
||||
}
|
||||
|
||||
public Task RunAsync(AgentErrorReport report)
|
||||
=> ReportAgentErrorFunction.MaterializeEmergencyBreakArtifactsAsync(
|
||||
report, TenantId, SessionRepo.Object, OpsService, NullLogger<ReportAgentErrorFunction>.Instance);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task First_break_report_materializes_event_index_row_and_ops_event()
|
||||
{
|
||||
var h = new Harness();
|
||||
|
||||
await h.RunAsync(Report(Now));
|
||||
|
||||
h.SessionRepo.Verify(r => r.StoreEventsBatchAsync(
|
||||
It.Is<List<EnrollmentEvent>>(e => e.Count == 1
|
||||
&& e[0].EventType == AutopilotMonitor.Shared.Constants.EventTypes.AgentEmergencyBreak)), Times.Once);
|
||||
h.SessionRepo.Verify(r => r.UpsertEventTypeIndexBatchAsync(
|
||||
TenantId, SessionId, It.IsAny<IEnumerable<EnrollmentEvent>>()), Times.Once);
|
||||
|
||||
var ops = Assert.Single(h.OpsEvents);
|
||||
Assert.Equal("AgentEmergencyBreak", ops.EventType);
|
||||
Assert.Equal(TenantId, ops.TenantId);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Already_materialized_session_produces_no_second_artifact_set()
|
||||
{
|
||||
var h = new Harness(new List<EnrollmentEvent>
|
||||
{
|
||||
new() { EventType = AutopilotMonitor.Shared.Constants.EventTypes.AgentEmergencyBreak, Sequence = 45 },
|
||||
});
|
||||
|
||||
await h.RunAsync(Report(Now));
|
||||
|
||||
h.SessionRepo.Verify(r => r.StoreEventsBatchAsync(It.IsAny<List<EnrollmentEvent>>()), Times.Never);
|
||||
h.SessionRepo.Verify(r => r.UpsertEventTypeIndexBatchAsync(
|
||||
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<IEnumerable<EnrollmentEvent>>()), Times.Never);
|
||||
Assert.Empty(h.OpsEvents);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Non_break_error_types_and_missing_session_id_are_ignored()
|
||||
{
|
||||
var h = new Harness();
|
||||
|
||||
var other = Report(Now);
|
||||
other.ErrorType = AgentErrorType.ConfigFetchFailed;
|
||||
await h.RunAsync(other);
|
||||
|
||||
var noSession = Report(Now);
|
||||
noSession.SessionId = null!;
|
||||
await h.RunAsync(noSession);
|
||||
|
||||
h.SessionRepo.Verify(r => r.GetSessionEventsAsync(
|
||||
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<int>()), Times.Never);
|
||||
Assert.Empty(h.OpsEvents);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Storage_failure_is_swallowed_and_never_throws()
|
||||
{
|
||||
var h = new Harness();
|
||||
h.SessionRepo.Setup(r => r.StoreEventsBatchAsync(It.IsAny<List<EnrollmentEvent>>()))
|
||||
.ThrowsAsync(new InvalidOperationException("storage down"));
|
||||
|
||||
// Must not throw — the always-200 emergency channel must never turn into a retry loop.
|
||||
await h.RunAsync(Report(Now));
|
||||
|
||||
Assert.Empty(h.OpsEvents);
|
||||
}
|
||||
}
|
||||
|
||||
+49
-10
@@ -78,15 +78,16 @@ public class SessionDeletionMaintenanceFunctionTests
|
||||
[Fact]
|
||||
public async Task RunCore_emits_LongRunning_when_body_runs_past_warning_threshold()
|
||||
{
|
||||
// Set watchdog thresholds to sub-second; make the fanout sleep long enough to trip warning
|
||||
// but not severe.
|
||||
var harness = new Harness(watchdogWarn: TimeSpan.FromMilliseconds(50), watchdogSevere: TimeSpan.FromMilliseconds(500));
|
||||
// Deterministic (no wall-clock race): the fanout body blocks until the WARN watchdog has
|
||||
// actually recorded its OpsEvent, then returns. Severe sits minutes out, so it can never
|
||||
// fire in the moment between warn firing and the body completing (which cancels it).
|
||||
var harness = new Harness(watchdogWarn: TimeSpan.FromMilliseconds(20), watchdogSevere: TimeSpan.FromMinutes(10));
|
||||
harness.SetKillSwitch(false);
|
||||
harness.Fanout.Setup(f => f.RunAsync(It.IsAny<SessionRetentionFanoutService.FanoutResult>(), It.IsAny<DateTime>(), It.IsAny<CancellationToken>()))
|
||||
.Returns(async (SessionRetentionFanoutService.FanoutResult r, DateTime _, CancellationToken ct) =>
|
||||
{
|
||||
r.TenantsProcessed = 1;
|
||||
await Task.Delay(150, ct);
|
||||
await harness.WaitForEventAsync("SessionDeletionMaintenanceLongRunning");
|
||||
});
|
||||
|
||||
await harness.Sut.RunCoreAsync("Test", CancellationToken.None);
|
||||
@@ -102,14 +103,17 @@ public class SessionDeletionMaintenanceFunctionTests
|
||||
// assigned only AFTER the fanout returned, so every LongRunning warning showed zeros.
|
||||
// The fanout now mutates the caller-supplied result incrementally; the watchdog snapshot
|
||||
// must reflect that progress mid-run.
|
||||
var harness = new Harness(watchdogWarn: TimeSpan.FromMilliseconds(50), watchdogSevere: TimeSpan.FromSeconds(5));
|
||||
// Deterministic: counters are set at fanout entry, then the body blocks until the WARN
|
||||
// watchdog fires and snapshots them. Severe sits minutes out and never fires — no
|
||||
// body-sleep-vs-threshold race.
|
||||
var harness = new Harness(watchdogWarn: TimeSpan.FromMilliseconds(50), watchdogSevere: TimeSpan.FromMinutes(10));
|
||||
harness.SetKillSwitch(false);
|
||||
harness.Fanout.Setup(f => f.RunAsync(It.IsAny<SessionRetentionFanoutService.FanoutResult>(), It.IsAny<DateTime>(), It.IsAny<CancellationToken>()))
|
||||
.Returns(async (SessionRetentionFanoutService.FanoutResult r, DateTime _, CancellationToken ct) =>
|
||||
{
|
||||
r.TenantsProcessed = 3;
|
||||
r.SessionsEnqueued = 7;
|
||||
await Task.Delay(200, ct); // long enough for the 50ms watchdog to fire mid-run
|
||||
await harness.WaitForEventAsync("SessionDeletionMaintenanceLongRunning");
|
||||
});
|
||||
|
||||
await harness.Sut.RunCoreAsync("Test", CancellationToken.None);
|
||||
@@ -122,17 +126,20 @@ public class SessionDeletionMaintenanceFunctionTests
|
||||
[Fact]
|
||||
public async Task RunCore_emits_LongRunningSevere_when_body_runs_past_severe_threshold()
|
||||
{
|
||||
var harness = new Harness(watchdogWarn: TimeSpan.FromMilliseconds(50), watchdogSevere: TimeSpan.FromMilliseconds(150));
|
||||
// Deterministic: the body blocks until the SEVERE watchdog records its OpsEvent. Warn has a
|
||||
// smaller threshold, so it is guaranteed to have fired first — neither depends on a body
|
||||
// sleep outlasting a fixed wall-clock threshold.
|
||||
var harness = new Harness(watchdogWarn: TimeSpan.FromMilliseconds(20), watchdogSevere: TimeSpan.FromMilliseconds(60));
|
||||
harness.SetKillSwitch(false);
|
||||
harness.Fanout.Setup(f => f.RunAsync(It.IsAny<SessionRetentionFanoutService.FanoutResult>(), It.IsAny<DateTime>(), It.IsAny<CancellationToken>()))
|
||||
.Returns(async (SessionRetentionFanoutService.FanoutResult _, DateTime _, CancellationToken ct) =>
|
||||
{
|
||||
await Task.Delay(300, ct);
|
||||
await harness.WaitForEventAsync("SessionDeletionMaintenanceLongRunningSevere");
|
||||
});
|
||||
|
||||
await harness.Sut.RunCoreAsync("Test", CancellationToken.None);
|
||||
|
||||
// Both watchdogs fire: warning at 50ms, severe at 150ms.
|
||||
// Both watchdogs fire: warning first (smaller threshold), then severe (which the body waited for).
|
||||
Assert.Contains(harness.OpsEvents, e => e.EventName == "SessionDeletionMaintenanceLongRunning");
|
||||
Assert.Contains(harness.OpsEvents, e => e.EventName == "SessionDeletionMaintenanceLongRunningSevere");
|
||||
}
|
||||
@@ -345,6 +352,12 @@ public class SessionDeletionMaintenanceFunctionTests
|
||||
private readonly Dictionary<string, List<TableEntity>> _sessionsByState = new(StringComparer.Ordinal);
|
||||
private readonly List<DeletionManifestBlobSummary> _ttlBlobs = new();
|
||||
|
||||
// Watchdog OpsEvents are recorded from concurrent watchdog tasks; _opsSync serializes the
|
||||
// OpsEvents append + waiter signalling, and _eventWaiters lets a test's fanout body block
|
||||
// until a specific watchdog OpsEvent has actually fired (see WaitForEventAsync).
|
||||
private readonly object _opsSync = new();
|
||||
private readonly Dictionary<string, TaskCompletionSource> _eventWaiters = new(StringComparer.Ordinal);
|
||||
|
||||
public Harness(TimeSpan? watchdogWarn = null, TimeSpan? watchdogSevere = null)
|
||||
{
|
||||
StorageMock = new Mock<TableStorageService>(Mock.Of<TableServiceClient>(), NullLogger<TableStorageService>.Instance);
|
||||
@@ -376,7 +389,12 @@ public class SessionDeletionMaintenanceFunctionTests
|
||||
opsRepo.Setup(r => r.SaveOpsEventAsync(It.IsAny<OpsEventEntry>()))
|
||||
.Returns<OpsEventEntry>(e =>
|
||||
{
|
||||
OpsEvents.Add(new CapturedOpsEvent(e.EventType, e.Severity, e.Message, e.Details ?? string.Empty));
|
||||
lock (_opsSync)
|
||||
{
|
||||
OpsEvents.Add(new CapturedOpsEvent(e.EventType, e.Severity, e.Message, e.Details ?? string.Empty));
|
||||
if (_eventWaiters.TryGetValue(e.EventType, out var waiter))
|
||||
waiter.TrySetResult();
|
||||
}
|
||||
return Task.CompletedTask;
|
||||
});
|
||||
var alertDispatch = new OpsAlertDispatchService(
|
||||
@@ -426,6 +444,27 @@ public class SessionDeletionMaintenanceFunctionTests
|
||||
public void SetKillSwitch(bool active) =>
|
||||
AdminConfig.Setup(a => a.IsSessionDeletionKillSwitchActiveAsync()).ReturnsAsync(active);
|
||||
|
||||
/// <summary>
|
||||
/// Deterministic watchdog gate: returns a task that completes once an OpsEvent whose type is
|
||||
/// <paramref name="eventName"/> has been recorded. A test's fanout body awaits this instead of
|
||||
/// sleeping a fixed span, so the watchdog is guaranteed to have fired before the body returns —
|
||||
/// removing the wall-clock race (body sleep vs sub-second thresholds) that flaked under CI load.
|
||||
/// </summary>
|
||||
public Task WaitForEventAsync(string eventName)
|
||||
{
|
||||
lock (_opsSync)
|
||||
{
|
||||
if (OpsEvents.Any(e => e.EventName == eventName))
|
||||
return Task.CompletedTask;
|
||||
if (!_eventWaiters.TryGetValue(eventName, out var waiter))
|
||||
{
|
||||
waiter = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
_eventWaiters[eventName] = waiter;
|
||||
}
|
||||
return waiter.Task;
|
||||
}
|
||||
}
|
||||
|
||||
public void SeedTtlBlobs(params (string TenantId, string SessionId, string ManifestId, DateTime LastModified)[] entries)
|
||||
{
|
||||
foreach (var e in entries)
|
||||
|
||||
@@ -0,0 +1,386 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Azure;
|
||||
using Azure.Data.Tables;
|
||||
using AutopilotMonitor.Functions.DataAccess.TableStorage;
|
||||
using AutopilotMonitor.Functions.Functions.Config;
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Shared;
|
||||
using AutopilotMonitor.Shared.DataAccess;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Microsoft.Extensions.Caching.Memory;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Moq;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// ContactEmail is the address enforcement actions and service notices are sent to.
|
||||
/// The seed must never overwrite a value the tenant owns — that is the whole contract.
|
||||
/// </summary>
|
||||
public class TenantContactEmailTests
|
||||
{
|
||||
private const string TenantId = "77777777-7777-7777-7777-777777777777";
|
||||
|
||||
[Fact]
|
||||
public void ContactEmail_defaults_to_null_so_absence_is_distinguishable_from_empty()
|
||||
{
|
||||
var config = new TenantConfiguration();
|
||||
Assert.Null(config.ContactEmail);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ContactEmail_survives_a_round_trip_through_the_model()
|
||||
{
|
||||
var config = new TenantConfiguration { ContactEmail = "it-operations@contoso.com" };
|
||||
Assert.Equal("it-operations@contoso.com", config.ContactEmail);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// Server-side validation (UpdateTenantConfigurationFunction)
|
||||
// ------------------------------------------------------------------
|
||||
|
||||
[Theory]
|
||||
[InlineData("it-operations@contoso.com")]
|
||||
[InlineData("first.last+autopilot@sub.fabrikam.co.uk")]
|
||||
[InlineData("UPPER@CONTOSO.COM")]
|
||||
public void ValidateContactEmail_accepts_real_addresses(string email)
|
||||
=> Assert.Null(UpdateTenantConfigurationFunction.ValidateContactEmail(email));
|
||||
|
||||
[Theory]
|
||||
[InlineData(null)]
|
||||
[InlineData("")]
|
||||
[InlineData(" ")]
|
||||
public void ValidateContactEmail_treats_absence_as_valid(string? email)
|
||||
{
|
||||
// No contact address is a legitimate state — it means we cannot reach this tenant.
|
||||
Assert.Null(UpdateTenantConfigurationFunction.ValidateContactEmail(email));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("not-an-address", "single")] // no @ at all
|
||||
[InlineData("@contoso.com", "single")] // empty local part
|
||||
[InlineData("ops@", "single")] // empty domain
|
||||
[InlineData("ops@@contoso.com", "single")] // two @
|
||||
[InlineData("ops@localhost", "dotted")] // unreachable bare host
|
||||
[InlineData("ops@.contoso.com", "dotted")] // leading dot
|
||||
[InlineData("ops@contoso.", "dotted")] // trailing dot
|
||||
[InlineData("ops@contoso.com, attacker@evil.test", "single address")] // recipient list
|
||||
[InlineData("ops@contoso.com; attacker@evil.test", "single address")]
|
||||
[InlineData("Ops Team <ops@contoso.com>", "single address")] // display-name form
|
||||
public void ValidateContactEmail_rejects_values_that_are_not_a_single_address(string email, string expectedHint)
|
||||
{
|
||||
var error = UpdateTenantConfigurationFunction.ValidateContactEmail(email);
|
||||
Assert.NotNull(error);
|
||||
Assert.Contains(expectedHint, error!, StringComparison.OrdinalIgnoreCase);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ValidateContactEmail_rejects_header_injection()
|
||||
{
|
||||
// Once this address is actually mailed, an embedded CR/LF would forge mail headers.
|
||||
var error = UpdateTenantConfigurationFunction.ValidateContactEmail(
|
||||
$"ops@contoso.com{(char)13}{(char)10}Bcc: attacker@evil.test");
|
||||
|
||||
Assert.NotNull(error);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ValidateContactEmail_rejects_an_over_long_value()
|
||||
{
|
||||
var local = new string('a', UpdateTenantConfigurationFunction.MaxContactEmailLength);
|
||||
var error = UpdateTenantConfigurationFunction.ValidateContactEmail($"{local}@contoso.com");
|
||||
|
||||
Assert.NotNull(error);
|
||||
Assert.Contains("at most", error!);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// Seed contract (TenantConfigurationService — the single owner)
|
||||
// ------------------------------------------------------------------
|
||||
|
||||
private static (TenantConfigurationService service, Mock<IConfigRepository> repo, MemoryCache cache) BuildService()
|
||||
{
|
||||
var repo = new Mock<IConfigRepository>();
|
||||
var cache = new MemoryCache(new MemoryCacheOptions());
|
||||
var service = new TenantConfigurationService(
|
||||
repo.Object, NullLogger<TenantConfigurationService>.Instance, cache);
|
||||
return (service, repo, cache);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TrySeedContactEmail_when_the_write_lands_reports_success_and_drops_the_cached_row()
|
||||
{
|
||||
var (service, repo, cache) = BuildService();
|
||||
repo.Setup(r => r.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com")).ReturnsAsync(true);
|
||||
cache.Set($"tenant-config:{TenantId}", new TenantConfiguration { TenantId = TenantId });
|
||||
|
||||
Assert.True(await service.TrySeedContactEmailAsync(TenantId, "ops@contoso.com"));
|
||||
|
||||
// The repository writes behind this cache; without the eviction the freshly seeded
|
||||
// address would stay invisible for the full 5-minute TTL.
|
||||
Assert.False(cache.TryGetValue($"tenant-config:{TenantId}", out _));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TrySeedContactEmail_when_the_tenant_already_owns_an_address_leaves_the_cache_alone()
|
||||
{
|
||||
var (service, repo, cache) = BuildService();
|
||||
repo.Setup(r => r.TrySeedTenantContactEmailAsync(It.IsAny<string>(), It.IsAny<string>()))
|
||||
.ReturnsAsync(false);
|
||||
var cached = new TenantConfiguration { TenantId = TenantId, ContactEmail = "owner@contoso.com" };
|
||||
cache.Set($"tenant-config:{TenantId}", cached);
|
||||
|
||||
Assert.False(await service.TrySeedContactEmailAsync(TenantId, "ops@contoso.com"));
|
||||
|
||||
Assert.True(cache.TryGetValue($"tenant-config:{TenantId}", out TenantConfiguration? still));
|
||||
Assert.Equal("owner@contoso.com", still!.ContactEmail);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(null)]
|
||||
[InlineData("")]
|
||||
[InlineData(" ")]
|
||||
public async Task TrySeedContactEmail_with_no_address_never_reaches_storage(string? email)
|
||||
{
|
||||
var (service, repo, _) = BuildService();
|
||||
|
||||
Assert.False(await service.TrySeedContactEmailAsync(TenantId, email));
|
||||
|
||||
repo.Verify(r => r.TrySeedTenantContactEmailAsync(It.IsAny<string>(), It.IsAny<string>()), Times.Never);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TrySeedContactEmail_with_no_tenant_never_reaches_storage()
|
||||
{
|
||||
var (service, repo, _) = BuildService();
|
||||
|
||||
Assert.False(await service.TrySeedContactEmailAsync(" ", "ops@contoso.com"));
|
||||
|
||||
repo.Verify(r => r.TrySeedTenantContactEmailAsync(It.IsAny<string>(), It.IsAny<string>()), Times.Never);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TrySeedContactEmail_trims_before_it_reaches_storage()
|
||||
{
|
||||
var (service, repo, _) = BuildService();
|
||||
repo.Setup(r => r.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com")).ReturnsAsync(true);
|
||||
|
||||
Assert.True(await service.TrySeedContactEmailAsync(TenantId, " ops@contoso.com "));
|
||||
|
||||
repo.Verify(r => r.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com"), Times.Once);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TrySeedContactEmail_never_writes_the_whole_configuration()
|
||||
{
|
||||
// Regression guard: the seed used to be a read-modify-write of the full model through
|
||||
// SaveTenantConfigurationAsync, which replaces the row unconditionally — a concurrent
|
||||
// portal save was clobbered by the seeder's stale snapshot of EVERY field.
|
||||
var (service, repo, _) = BuildService();
|
||||
repo.Setup(r => r.TrySeedTenantContactEmailAsync(It.IsAny<string>(), It.IsAny<string>()))
|
||||
.ReturnsAsync(true);
|
||||
|
||||
await service.TrySeedContactEmailAsync(TenantId, "ops@contoso.com");
|
||||
|
||||
repo.Verify(r => r.SaveTenantConfigurationAsync(It.IsAny<TenantConfiguration>()), Times.Never);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// Seed trigger (PreviewWhitelistService)
|
||||
// ------------------------------------------------------------------
|
||||
|
||||
private static (PreviewWhitelistService preview, Mock<IConfigRepository> repo) BuildPreview()
|
||||
{
|
||||
var repo = new Mock<IConfigRepository>();
|
||||
var cache = new MemoryCache(new MemoryCacheOptions());
|
||||
var tenantConfig = new TenantConfigurationService(
|
||||
repo.Object, NullLogger<TenantConfigurationService>.Instance, cache);
|
||||
var preview = new PreviewWhitelistService(
|
||||
repo.Object, cache, NullLogger<PreviewWhitelistService>.Instance, tenantConfig);
|
||||
return (preview, repo);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task SaveNotificationEmail_persists_the_address_and_seeds_the_contact_field()
|
||||
{
|
||||
var (preview, repo) = BuildPreview();
|
||||
repo.Setup(r => r.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com")).ReturnsAsync(true);
|
||||
|
||||
await preview.SaveNotificationEmailAsync(TenantId, "ops@contoso.com");
|
||||
|
||||
repo.Verify(r => r.SaveNotificationEmailAsync(TenantId, "ops@contoso.com"), Times.Once);
|
||||
repo.Verify(r => r.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com"), Times.Once);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task SaveNotificationEmail_clearing_the_address_does_not_seed()
|
||||
{
|
||||
var (preview, repo) = BuildPreview();
|
||||
|
||||
await preview.SaveNotificationEmailAsync(TenantId, null);
|
||||
|
||||
repo.Verify(r => r.SaveNotificationEmailAsync(TenantId, null), Times.Once);
|
||||
repo.Verify(r => r.TrySeedTenantContactEmailAsync(It.IsAny<string>(), It.IsAny<string>()), Times.Never);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task SaveNotificationEmail_still_succeeds_when_the_seed_blows_up()
|
||||
{
|
||||
// The notification email has already been persisted at that point; a failing side
|
||||
// effect must not turn a completed write into a failed request.
|
||||
var (preview, repo) = BuildPreview();
|
||||
repo.Setup(r => r.TrySeedTenantContactEmailAsync(It.IsAny<string>(), It.IsAny<string>()))
|
||||
.ThrowsAsync(new RequestFailedException(503, "throttled"));
|
||||
|
||||
await preview.SaveNotificationEmailAsync(TenantId, "ops@contoso.com");
|
||||
|
||||
repo.Verify(r => r.SaveNotificationEmailAsync(TenantId, "ops@contoso.com"), Times.Once);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// Conditional write (TableConfigRepository) — the actual invariant
|
||||
// ------------------------------------------------------------------
|
||||
|
||||
private sealed class RepoHarness
|
||||
{
|
||||
public TableConfigRepository Sut { get; }
|
||||
public Mock<TableClient> Table { get; } = new();
|
||||
public List<(TableEntity Entity, ETag IfMatch, TableUpdateMode Mode)> Updates { get; } = new();
|
||||
|
||||
public RepoHarness(TableEntity? stored, RequestFailedException? throwOnUpdate = null)
|
||||
{
|
||||
if (stored == null)
|
||||
{
|
||||
Table.Setup(c => c.GetEntityAsync<TableEntity>(
|
||||
It.IsAny<string>(), It.IsAny<string>(),
|
||||
It.IsAny<IEnumerable<string>>(), It.IsAny<CancellationToken>()))
|
||||
.ThrowsAsync(new RequestFailedException(404, "Not Found", "ResourceNotFound", null));
|
||||
}
|
||||
else
|
||||
{
|
||||
Table.Setup(c => c.GetEntityAsync<TableEntity>(
|
||||
It.IsAny<string>(), It.IsAny<string>(),
|
||||
It.IsAny<IEnumerable<string>>(), It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync(Response.FromValue(stored, Mock.Of<Response>()));
|
||||
}
|
||||
|
||||
Table.Setup(c => c.UpdateEntityAsync(
|
||||
It.IsAny<TableEntity>(), It.IsAny<ETag>(),
|
||||
It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()))
|
||||
.Returns<TableEntity, ETag, TableUpdateMode, CancellationToken>((e, tag, mode, _) =>
|
||||
{
|
||||
if (throwOnUpdate != null)
|
||||
throw throwOnUpdate;
|
||||
Updates.Add((e, tag, mode));
|
||||
return Task.FromResult(Mock.Of<Response>());
|
||||
});
|
||||
|
||||
var serviceClient = new Mock<TableServiceClient>();
|
||||
serviceClient.Setup(c => c.GetTableClient(It.IsAny<string>())).Returns(Table.Object);
|
||||
|
||||
Sut = new TableConfigRepository(
|
||||
new TableStorageService(serviceClient.Object, NullLogger<TableStorageService>.Instance),
|
||||
NullLogger<TableConfigRepository>.Instance);
|
||||
}
|
||||
}
|
||||
|
||||
private static TableEntity StoredConfig(string? contactEmail, string etag = "W/\"stored\"")
|
||||
{
|
||||
var entity = new TableEntity(TenantId, "config") { ["DomainName"] = "contoso.com" };
|
||||
if (contactEmail != null)
|
||||
entity["ContactEmail"] = contactEmail;
|
||||
entity.ETag = new ETag(etag);
|
||||
return entity;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Repo_seeds_only_the_contact_field_and_only_conditionally()
|
||||
{
|
||||
var harness = new RepoHarness(StoredConfig(contactEmail: null));
|
||||
|
||||
Assert.True(await harness.Sut.TrySeedTenantContactEmailAsync(TenantId, " ops@contoso.com "));
|
||||
|
||||
var (entity, ifMatch, mode) = Assert.Single(harness.Updates);
|
||||
// Merge, not Replace: every other stored property must survive untouched.
|
||||
Assert.Equal(TableUpdateMode.Merge, mode);
|
||||
Assert.Equal(new ETag("W/\"stored\""), ifMatch);
|
||||
Assert.Equal("ops@contoso.com", entity["ContactEmail"]);
|
||||
Assert.False(entity.ContainsKey("DomainName"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Repo_never_overwrites_an_address_the_tenant_already_owns()
|
||||
{
|
||||
var harness = new RepoHarness(StoredConfig("owner@contoso.com"));
|
||||
|
||||
Assert.False(await harness.Sut.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com"));
|
||||
|
||||
Assert.Empty(harness.Updates);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Repo_treats_a_whitespace_only_stored_value_as_absent()
|
||||
{
|
||||
var harness = new RepoHarness(StoredConfig(" "));
|
||||
|
||||
Assert.True(await harness.Sut.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com"));
|
||||
|
||||
Assert.Single(harness.Updates);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Repo_loses_the_race_rather_than_clobbering_a_concurrent_write()
|
||||
{
|
||||
// Someone wrote the row between our read and our write. The ETag precondition turns
|
||||
// that into a 412; the seed abandons rather than overwriting the winner.
|
||||
var harness = new RepoHarness(
|
||||
StoredConfig(contactEmail: null),
|
||||
throwOnUpdate: new RequestFailedException(412, "Precondition Failed", "UpdateConditionNotSatisfied", null));
|
||||
|
||||
Assert.False(await harness.Sut.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Repo_with_no_configuration_row_reports_no_seed()
|
||||
{
|
||||
var harness = new RepoHarness(stored: null);
|
||||
|
||||
Assert.False(await harness.Sut.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com"));
|
||||
|
||||
Assert.Empty(harness.Updates);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Repo_is_fail_soft_on_a_storage_fault()
|
||||
{
|
||||
// Callers are side effects of an already-committed write — a lost seed is recoverable
|
||||
// on the next maintenance run, a thrown exception is not.
|
||||
var harness = new RepoHarness(
|
||||
StoredConfig(contactEmail: null),
|
||||
throwOnUpdate: new RequestFailedException(503, "Server Busy"));
|
||||
|
||||
Assert.False(await harness.Sut.TrySeedTenantContactEmailAsync(TenantId, "ops@contoso.com"));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(null, "ops@contoso.com")]
|
||||
[InlineData("", "ops@contoso.com")]
|
||||
[InlineData(TenantId, null)]
|
||||
[InlineData(TenantId, " ")]
|
||||
public async Task Repo_rejects_incomplete_input_without_touching_storage(string? tenantId, string? email)
|
||||
{
|
||||
var harness = new RepoHarness(StoredConfig(contactEmail: null));
|
||||
|
||||
Assert.False(await harness.Sut.TrySeedTenantContactEmailAsync(tenantId!, email!));
|
||||
|
||||
Assert.Empty(harness.Updates);
|
||||
harness.Table.Verify(c => c.GetEntityAsync<TableEntity>(
|
||||
It.IsAny<string>(), It.IsAny<string>(),
|
||||
It.IsAny<IEnumerable<string>>(), It.IsAny<CancellationToken>()), Times.Never);
|
||||
}
|
||||
}
|
||||
@@ -87,7 +87,11 @@ public sealed class TenantOffboardFunctionTests
|
||||
return new PreviewWhitelistService(
|
||||
configRepoMock.Object,
|
||||
cache,
|
||||
NullLogger<PreviewWhitelistService>.Instance);
|
||||
NullLogger<PreviewWhitelistService>.Instance,
|
||||
new TenantConfigurationService(
|
||||
configRepoMock.Object,
|
||||
NullLogger<TenantConfigurationService>.Instance,
|
||||
cache));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
|
||||
+11
-7
@@ -64,22 +64,26 @@ public class ValidateBootstrapCodeResponseShapeTests
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The hardcoded agent download URL in <c>ValidateBootstrapCodeFunction</c> must
|
||||
/// The agent download URL served by <c>ValidateBootstrapCodeFunction</c> must
|
||||
/// satisfy the frontend's host allow-list and path regex.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void AgentDownloadUrl_MatchesFrontendAllowList()
|
||||
{
|
||||
// Keep in sync with ValidateBootstrapCodeFunction.cs — if that literal
|
||||
// changes, both this assertion and bootstrapValidation.ts must be updated.
|
||||
const string HardcodedAgentUrl =
|
||||
"https://autopilotmonitor.blob.core.windows.net/agent/AutopilotMonitor-Agent.zip";
|
||||
// Independent oracle, deliberately spelled out: if Constants or the function
|
||||
// change the URL, this literal AND bootstrapValidation.ts (AGENT_DOWNLOAD_HOSTNAMES)
|
||||
// must be updated consciously — that pairing is what keeps portal and backend in sync.
|
||||
const string ExpectedAgentUrl =
|
||||
"https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip";
|
||||
|
||||
Assert.True(Uri.TryCreate(HardcodedAgentUrl, UriKind.Absolute, out var uri),
|
||||
var built = $"{AutopilotMonitor.Shared.Constants.AgentDownloadBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}";
|
||||
Assert.Equal(ExpectedAgentUrl, built);
|
||||
|
||||
Assert.True(Uri.TryCreate(built, UriKind.Absolute, out var uri),
|
||||
"AgentDownloadUrl is not a well-formed absolute URI");
|
||||
|
||||
Assert.Equal(Uri.UriSchemeHttps, uri!.Scheme);
|
||||
Assert.Equal("autopilotmonitor.blob.core.windows.net", uri.Host);
|
||||
Assert.Equal("download.autopilotmonitor.com", uri.Host);
|
||||
Assert.True(AgentDownloadUrlPath.IsMatch(uri.AbsolutePath),
|
||||
$"AgentDownloadUrl path '{uri.AbsolutePath}' does not match the frontend allow-list regex");
|
||||
}
|
||||
|
||||
@@ -12,8 +12,16 @@
|
||||
BuildTimestampUtc is baked into the assembly as AssemblyMetadata so /api/health
|
||||
reports the actual build moment — not the file mtime, which Azure Functions
|
||||
resets to the zip-extract time on every cold start. CI may override via
|
||||
-p:BuildTimestampUtc=<iso8601> to force a specific timestamp. -->
|
||||
<Version>1.0.0</Version>
|
||||
-p:BuildTimestampUtc=<iso8601> to force a specific timestamp.
|
||||
|
||||
VersionPrefix is the curated part (bump by hand for a release); BuildNumber is
|
||||
reserved per build from the counter blob versions/backend.txt and passed in as
|
||||
-p:BuildNumber=<n> by deploy-backend.yml — same mechanism as the agent line.
|
||||
A local build has no counter and stays at x.y.0 with an empty commit hash,
|
||||
which is how /api/health tells a developer machine from a CI artifact. -->
|
||||
<VersionPrefix>1.5</VersionPrefix>
|
||||
<BuildNumber Condition="'$(BuildNumber)' == ''">0</BuildNumber>
|
||||
<Version>$(VersionPrefix).$(BuildNumber)</Version>
|
||||
<GenerateAssemblyInfo>true</GenerateAssemblyInfo>
|
||||
<IncludeSourceRevisionInInformationalVersion>true</IncludeSourceRevisionInInformationalVersion>
|
||||
<BuildTimestampUtc Condition="'$(BuildTimestampUtc)' == ''">$([System.DateTime]::UtcNow.ToString('o'))</BuildTimestampUtc>
|
||||
@@ -47,6 +55,12 @@
|
||||
<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
|
||||
<PackageReference Include="Microsoft.Identity.Web" Version="4.3.0" />
|
||||
<PackageReference Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="10.0.10" />
|
||||
<!-- Security pins: Microsoft.Identity.Web 4.3.0 pulls these transitively at 10.0.0, which carries
|
||||
known advisories (GHSA-9mv3-2cwr-p262 DataProtection; GHSA-37gx-xxp4-5rgx and siblings on
|
||||
Cryptography.Xml). Force the patched 10.0.10 servicing build. Remove once Identity.Web ships
|
||||
a dependency floor past the fix. -->
|
||||
<PackageReference Include="Microsoft.AspNetCore.DataProtection" Version="10.0.10" />
|
||||
<PackageReference Include="System.Security.Cryptography.Xml" Version="10.0.10" />
|
||||
<PackageReference Include="Microsoft.Extensions.Http.Polly" Version="8.0.25" />
|
||||
<PackageReference Include="Resend" Version="0.2.1" />
|
||||
</ItemGroup>
|
||||
|
||||
+50
-4
@@ -71,6 +71,54 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Conditional, single-property seed of ContactEmail (see <see cref="IConfigRepository"/>).
|
||||
/// Merge mode leaves every other stored property untouched, and the ETag precondition turns
|
||||
/// a concurrent write into a 412 rather than a silent overwrite. Both matter: the caller is
|
||||
/// a background/side-effect path that must never beat a tenant admin's own edit.
|
||||
/// </summary>
|
||||
public async Task<bool> TrySeedTenantContactEmailAsync(string tenantId, string email)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(tenantId) || string.IsNullOrWhiteSpace(email))
|
||||
return false;
|
||||
|
||||
try
|
||||
{
|
||||
var existing = await _tenantConfigTableClient.GetEntityAsync<TableEntity>(tenantId, "config");
|
||||
|
||||
// The tenant already owns an address — the seed is one-way and never re-syncs.
|
||||
if (!string.IsNullOrWhiteSpace(existing.Value.GetString("ContactEmail")))
|
||||
return false;
|
||||
|
||||
var patch = new TableEntity(tenantId, "config")
|
||||
{
|
||||
["ContactEmail"] = email.Trim()
|
||||
};
|
||||
|
||||
await _tenantConfigTableClient.UpdateEntityAsync(
|
||||
patch, existing.Value.ETag, TableUpdateMode.Merge);
|
||||
return true;
|
||||
}
|
||||
catch (RequestFailedException ex) when (ex.Status == 404)
|
||||
{
|
||||
// No configuration row yet — nothing to seed onto.
|
||||
return false;
|
||||
}
|
||||
catch (RequestFailedException ex) when (ex.Status == 412)
|
||||
{
|
||||
// Someone wrote the row between the read and the write. They win by construction:
|
||||
// a lost seed is recoverable (the backfill retries), a lost admin edit is not.
|
||||
_logger.LogInformation(
|
||||
"Contact address seed for {TenantId} skipped — the configuration changed concurrently", tenantId);
|
||||
return false;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "Error seeding contact address for tenant {TenantId}", tenantId);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
public async Task<List<TenantConfiguration>> GetAllTenantConfigurationsAsync()
|
||||
{
|
||||
try
|
||||
@@ -336,6 +384,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
{ "LastUpdated", config.LastUpdated },
|
||||
{ "UpdatedBy", config.UpdatedBy },
|
||||
{ "OnboardedBy", config.OnboardedBy },
|
||||
{ "ContactEmail", config.ContactEmail },
|
||||
{ "Disabled", config.Disabled },
|
||||
{ "DisabledReason", config.DisabledReason },
|
||||
{ "DisabledUntil", config.DisabledUntil },
|
||||
@@ -432,6 +481,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
LastUpdated = entity.GetDateTime("LastUpdated") ?? DateTime.UtcNow,
|
||||
UpdatedBy = entity.GetString("UpdatedBy") ?? "Unknown",
|
||||
OnboardedBy = entity.GetString("OnboardedBy"),
|
||||
ContactEmail = entity.GetString("ContactEmail"),
|
||||
Disabled = entity.GetBoolean("Disabled") ?? false,
|
||||
DisabledReason = entity.GetString("DisabledReason"),
|
||||
DisabledUntil = entity.GetDateTime("DisabledUntil"),
|
||||
@@ -540,8 +590,6 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
{ "ExcessiveEventAutoActionMode", config.ExcessiveEventAutoActionMode ?? "Off" },
|
||||
{ "ExcessiveEventAutoActionThreshold", config.ExcessiveEventAutoActionThreshold },
|
||||
{ "ExcessiveEventAutoActionDurationHours", config.ExcessiveEventAutoActionDurationHours },
|
||||
{ "MaxSessionWindowHours", config.MaxSessionWindowHours },
|
||||
{ "MaintenanceBlockDurationHours", config.MaintenanceBlockDurationHours },
|
||||
{ "DiagnosticsGlobalLogPathsJson", config.DiagnosticsGlobalLogPathsJson },
|
||||
{ "ModernDeploymentHarmlessEventIdsJson", config.ModernDeploymentHarmlessEventIdsJson ?? string.Empty },
|
||||
{ "WhiteGloveSealingPatternIdsJson", config.WhiteGloveSealingPatternIdsJson ?? string.Empty },
|
||||
@@ -607,8 +655,6 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
ExcessiveEventAutoActionMode = entity.GetString("ExcessiveEventAutoActionMode") ?? "Off",
|
||||
ExcessiveEventAutoActionThreshold = entity.GetInt32("ExcessiveEventAutoActionThreshold") ?? 2500,
|
||||
ExcessiveEventAutoActionDurationHours = entity.GetInt32("ExcessiveEventAutoActionDurationHours") ?? 24,
|
||||
MaxSessionWindowHours = entity.GetInt32("MaxSessionWindowHours") ?? 24,
|
||||
MaintenanceBlockDurationHours = entity.GetInt32("MaintenanceBlockDurationHours") ?? 12,
|
||||
DiagnosticsGlobalLogPathsJson = entity.GetString("DiagnosticsGlobalLogPathsJson"),
|
||||
ModernDeploymentHarmlessEventIdsJson = entity.GetString("ModernDeploymentHarmlessEventIdsJson"),
|
||||
WhiteGloveSealingPatternIdsJson = entity.GetString("WhiteGloveSealingPatternIdsJson"),
|
||||
|
||||
-3
@@ -69,9 +69,6 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
public Task<List<SessionSummary>> GetAgentSilentSessionsAsync(string tenantId, DateTime silenceCutoff, DateTime hardCutoff)
|
||||
=> _storage.GetAgentSilentSessionsAsync(tenantId, silenceCutoff, hardCutoff);
|
||||
|
||||
public Task<List<SessionSummary>> GetExcessiveDataSendersAsync(string tenantId, DateTime windowCutoff, int maxSessionWindowHours)
|
||||
=> _storage.GetExcessiveDataSendersAsync(tenantId, windowCutoff, maxSessionWindowHours);
|
||||
|
||||
public Task<List<SessionSummary>> GetLegacyTimeoutFailedSessionsAsync(string tenantId, int maxResults)
|
||||
=> _storage.GetLegacyTimeoutFailedSessionsAsync(tenantId, maxResults);
|
||||
|
||||
|
||||
+2
-1
@@ -1,3 +1,4 @@
|
||||
using AutopilotMonitor.Functions.Helpers;
|
||||
using AutopilotMonitor.Functions.Security;
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Shared;
|
||||
@@ -72,7 +73,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
}
|
||||
|
||||
_logger.LogWarning("Failed to increment user usage after {MaxRetries} retries: user={UserId}, endpoint={Endpoint}",
|
||||
maxRetries, userId, endpoint);
|
||||
maxRetries, LogSanitizer.Clean(userId), LogSanitizer.Clean(endpoint));
|
||||
}
|
||||
|
||||
public async Task<List<UserUsageRecord>> GetUsageByUserAsync(string userId, string? dateFrom = null, string? dateTo = null)
|
||||
|
||||
@@ -52,6 +52,34 @@ public static class ClaimsPrincipalExtensions
|
||||
?? principal.FindFirst("preferred_username")?.Value;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Stable per-caller identity for THROTTLING keys — never for authorization or audit.
|
||||
/// <para>
|
||||
/// Prefers the UPN so a human caller keeps the same bucket as before. Falls back to the token
|
||||
/// subject (<c>oid</c> → <c>sub</c> → <c>appid</c>/<c>azp</c>) because a valid token need not carry
|
||||
/// a user identity at all: AuthenticationMiddleware validates issuer/audience/lifetime/signature
|
||||
/// but requires no <c>upn</c>, so an app-only (client-credentials) token authenticates fine and
|
||||
/// reaches every AuthenticatedUser route. Without this fallback such a caller would present an
|
||||
/// empty identity and skip per-user rate limiting entirely.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Both the raw and the WS-Fed-mapped claim names are checked: JwtSecurityTokenHandler runs with
|
||||
/// inbound claim mapping ON, so <c>oid</c>/<c>sub</c> may arrive under their schema URIs.
|
||||
/// </para>
|
||||
/// Returns null only for a principal carrying no identifying claim whatsoever; callers must
|
||||
/// decide fail-closed what that means (see <c>RequestContext.CallerId</c>).
|
||||
/// </summary>
|
||||
public static string? GetCallerId(this ClaimsPrincipal principal)
|
||||
{
|
||||
return principal.GetUserPrincipalName()
|
||||
?? principal.FindFirst("oid")?.Value
|
||||
?? principal.FindFirst("http://schemas.microsoft.com/identity/claims/objectidentifier")?.Value
|
||||
?? principal.FindFirst("sub")?.Value
|
||||
?? principal.FindFirst(ClaimTypes.NameIdentifier)?.Value
|
||||
?? principal.FindFirst("appid")?.Value
|
||||
?? principal.FindFirst("azp")?.Value;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets the user's display name from token claims
|
||||
/// Supports both v1.0 and v2.0 token formats
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user