Compare commits

...

32 Commits

Author SHA1 Message Date
Oliver Kieselbach 21b8e054df docs: correct portal-prefetch root cause (h2, server-side queue) + measured outcome
The HTTP/1.1 / six-connection-cap claim was wrong — it came from the
Schannel curl build that cannot speak HTTP/2. ALPN (openssl s_client)
and .NET HttpClient confirm the portal negotiates h2; the queue is
server-side in the SWA SSR runtime. Doc, index, log and the
GlobalSidebar comment now describe the real mechanism.

Adds the 2026-07-28 outcome section: Standard plan changed nothing
(stratified p50s match, Mann-Whitney p=0.81); prefetch={false} removed
the freezes (worst totalDuration 225s -> 4.9s) and also removed an
accidental keep-alive (~150ms added to cold navigations).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 16:52:19 +02:00
Oliver Kieselbach 249c20e15a feat: mark backfilled pre-agent events as history in timeline + agent summary event
Customer feedback: backfilled modern_deployment_error events (pre-agent
Autopilot event-log errors replayed at agent start) rendered as plain
Error rows with one identical emission timestamp — indistinguishable
from a live failure.

Web (EventTimeline.tsx, effective for already-stored events):
- "⟲ Backfilled" badge (slate, explanatory tooltip) on data.backfilled
- row time now shows the original event-log timeCreated with a
  "(reported …)" suffix; metadata block gains a "Recorded" line;
  raw mode gets a compact ⟲ marker
- severity deliberately NOT downgraded — unresolved pre-agent errors
  must stay visible in the Error filter

Agent (ModernDeploymentTracker):
- backfill scan buffers the capped record set, emits an Info summary
  event first (modern_deployment_log, backfillSummary: true +
  replayedCount/droppedOverCap/lookbackMinutes) announcing the block,
  then replays oldest-first so timeline sequence follows event-log
  chronology (cap semantics unchanged: newest 25 win)

No new EventType, no ConfigVersion bump. Tests: summary emission x2,
tracker subset 12/12; tsc + vitest 651/651 green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 16:10:25 +02:00
Oliver Kieselbach 4ebd60fb1e feat: EnableGatherRuleDebugLog — per-tenant gather-rule debug trace (ConfigVersion 36)
Customers cannot tell why a gather rule produces nothing in the timeline
(e.g. a logparser rule with emit_mode on_change that never emits). New
per-tenant boolean EnableGatherRuleDebugLog (EnableImeMatchLog-style
plumbing end to end): when enabled, the agent writes
%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log with one
pipe-delimited line per evaluation decision across all previously silent
"produced nothing" exits — disabled rules, interval without
intervalSeconds, first-tick-after-one-full-interval, phase-scope skips
incl. currentPhase=Unknown, per-(rule, phase) dedup, empty collector
results (emitOnlyIfExists), on_change hash suppression (hash prefix +
streak), guard blocks, logparser per-file outcomes (position skips,
format=text hint when every line fails CMTrace parsing, maxLines and
wildcard caps), and full exception stacks the agent log drops.

- Backend: ConfigVersion 35 -> 36; TenantConfiguration/AgentConfigResponse
  field + TableConfigRepository Store+Map
- Agent: GatherRuleDebugLog writer (10 MB cap, one .old generation,
  failures swallowed, optional console echo), GatherRuleContext.DebugLog
  (null-safe no-op), trace points in executor + collectors; never writes
  under _scopeLock (reasons captured inside, written after release);
  merger bool->path projection with --gather-debug-log CLI override
  tolerance; deliberately NOT in the offline-cache strip lists;
  --run-gather-rules always traces; zero-rules delivery leaves a one-line
  file instead of nothing
- Side fix: GatherRuleExecutorHost now passes the logger into
  InformationalEventPost so gather-event emissions hit the DEBUG log
- Web: tenant settings toggle (Settings -> Agent) + admin tenant-config
  report rows
- Tests: merger on/off/override, writer format+rotation, executor debug
  lines, logparser debug lines (agent 2131, backend 3561, tsc green)
- Docs: docs/agent/gather-rule-debug-log.md + index/log entries

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 15:59:30 +02:00
Oliver Kieselbach 22093d0824 feat(agent): diagnostics upload at WhiteGlove Part-1 seal (preprov, V1 parity)
Upload the diagnostics package as a pre-provisioning intermediate when
Part 1 seals (suffix "preprov"). No new config field: the existing mode
semantics carry the desired behaviour — Always uploads at the seal,
OnFailure skips (the seal terminates as Succeeded; a Part-1 failure goes
through the regular Failed terminal path, which already uploads), Off
stays off. The upload runs before the whiteglove_part1_complete emit so
diagnostics_* events keep a lower sequence than the Part-1/Part-2
boundary marker. Best-effort: a fast reseal-reboot just loses the
intermediate package. Backend needs nothing — ingest already stamps
diagnostics_uploaded independent of terminal state (proven live by the
on-demand collection path).

Also make the on-demand path honest: CreateAndUploadAsync now takes
bool? enrollmentSucceeded — null means "no outcome yet" (mid-session),
so the OnFailure gate only skips a KNOWN success and sessioninfo.txt
records "In Progress" instead of the former enrollmentSucceeded:false
workaround that stamped in-flight packages "Failed".

10 new tests (WG seal mode matrix, preprov suffix, event ordering vs the
Part-1 marker, unchanged Part-1 exit contract; nullable-outcome gate
matrix via HTTP-free sentinel, sessioninfo labels). Agent suite
2112/2112 green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 15:54:14 +02:00
Oliver Kieselbach dff1532be5 feat: project ConnectionType (WiFi/Ethernet) onto SessionsIndex
The agent already reports connectionType in every network_interface_info
event, but it was only stored as a JSON blob in DeviceSnapshot — answering
"how many machines enroll over Ethernet vs WiFi?" required a slow full-table
scan via deviceProperties. Project it as a typed column instead (template:
the AvgApiLatencyMs chain), making it a cheap indexed one-call query.

- Ingest: TryExtractConnectionType — last network_interface_info emission of
  the batch wins (a device switching media mid-enrollment is relabeled);
  only exact "WiFi"/"Ethernet" accepted, fire-and-forget fail-soft writer
- Storage: UpdateSessionConnectionTypeAsync (Merge on Sessions + index
  mirror, tombstone-safe), BuildSessionIndexEntity superset entry,
  MapToSessionSummary, re-registration preserve in StoreSessionAsync
- Search: connectionType= filter with OData eq push-down on the index,
  client-side backstop on the non-OData paths, fields=connectionType
  projection, pagination fingerprint entry, raw endpoint parity
- MCP: connectionType z.enum param on search_sessions/query_raw_sessions,
  fields doc updated (server redeploy required)
- Web: optional sortable/filterable "Connection" column in the session list
- No backfill: legacy sessions lack the column and are excluded by the
  filter (intended, same semantics as RebootCount); the deviceProperties
  scan path still covers historical data

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 14:25:17 +02:00
Oliver Kieselbach 8608ee6272 feat(agent): netsh WiFi fallback labels for all enrollment-geography languages
Derived the language set from actual enrollment geography
(get_geographic_metrics, 365d, 10k sessions across ~50 countries) and
documented the country->language mapping on the key arrays. Newly
covered: Finnish, Hungarian, Lithuanian, Latvian, Romanian, Croatian/
Bosnian, Slovak, Indonesian/Malay, Vietnamese, Thai, Arabic, Bulgarian
radio-type variant and Swedish "Radiotyp". New fi/id parser tests; the
unknown-language tests now use Greek (outside the geography set).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 14:18:34 +02:00
Oliver Kieselbach f762b932a2 feat(agent): CJK + Russian labels in netsh WiFi fallback parser
Extend the localized label sets with the remaining languages seen in
enrollments: Russian, Japanese, Chinese (Simplified + Traditional,
incl. the 信道/通道 channel variants) and Korean (incl. the two radio-
type spellings). SSID stays untranslated everywhere. New zh-CN and ko
parser tests; suite green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 14:14:07 +02:00
Oliver Kieselbach 173bfc40a9 feat(agent): multi-language label support in netsh WiFi fallback
The netsh fallback previously matched English labels only, silently
dropping fields on localized OS images. The parser now matches the
label variants of the languages commonly seen in enrollments
(EN/DE/FR/ES/PT/IT/NL/PL/CS/TR; Scandinavian locales reuse the EN/DE
spellings) and is split into a pure Parse(output) function covered by
unit tests (EN/DE/ES samples, "94 %" spacing, AP-BSSID collision,
unknown-language degradation - SSID itself is untranslated in all
locales). Unknown labels still fall through harmlessly; the native
WLAN API remains the primary path.

V2.Core suite 2095/2095 green; fallback live-verified against netsh
on dev hardware (identical values to the API path).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 14:08:46 +02:00
Oliver Kieselbach 609d9c47af fix(agent): netsh fallback + step diagnostics for WiFi info collection
Field regression on 2.0.1383 (session f7c56065, HP ProBook 450 G7,
build 26200.8037): the native WLAN API path returned nothing in the
OOBE/SYSTEM context and the null path was silent, so wifi_signal_info
disappeared entirely - while netsh-based 2.0.1380/1382 sessions on the
same build kept reporting SSID/signal the same morning. Suspected (not
yet proven) cause: Wi-Fi location gating of WlanQueryInterface on
24H2+; inbox netsh is unaffected.

- WifiInfoReader: new TryGetCurrentConnection overload reporting WHY
  the read yielded nothing (native rc per step, interface state);
  callers log it at Debug so the agent log explains missing WiFi info.
- NetshWifiFallback (new): parses netsh wlan show interfaces ONLY when
  the WLAN API yields nothing on a WiFi-connected device; SystemPaths.
  Netsh restored for it.
- WifiConnectionInfo.SignalPercent is now int? (netsh fallback cannot
  always parse localized "Signal"); event payload emits the field only
  when present - same shape as before.

V2.Core suite 2085/2085 green; interactive positive path re-verified
(SSID/94%/802.11ax/ch116, empty diagnostics).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 13:55:46 +02:00
Oliver Kieselbach 5496d575c1 style: two-line Started column (date over time) in session list
Narrows the column so more columns fit on one page; matches the
two-line style of Device/Model/Status cells.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 13:37:01 +02:00
Oliver Kieselbach 1c3d29f30b refactor(agent): read WiFi info via native WLAN API instead of netsh parsing
netsh wlan show interfaces output is localized: "Radio type"/"Channel"
(and on some languages even "Signal") never matched on non-English
images, silently dropping fields from wifi_signal_info and network
change snapshots. It also spawned a child process per collection.

Replace both netsh call sites with wlanapi.dll P/Invoke
(WlanQueryInterface: current_connection + channel_number):

- New WlanNativeMethods (raw interop) + WifiInfoReader (fail-soft,
  prefers the active NIC's interface GUID, falls back to the first
  connected WLAN interface; returns null on machines without WLAN).
- DeviceInfoCollector.CollectWiFiSignalInfo and
  NetworkChangeDetector.CollectWiFiInfo now use the reader; RadioType
  strings stay netsh-compatible ("802.11ax" ...), event fields
  (wifiSsid/wifiSignalPercent/wifiRadioType/wifiChannel) unchanged.
- SystemPaths.Netsh removed (no remaining references).

V2.Core suite 2085/2085 green incl. new WifiInfoReader tests; struct
marshaling verified against native sizeof (604/68/36/532).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 13:20:12 +02:00
Oliver Kieselbach bb4accfe96 feat: API Latency as optional column in dashboard session list
Off by default; sortable via avgApiLatencyMs. Data comes from the
SessionsIndex mirror the list already reads — no extra query or table.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 13:11:20 +02:00
Oliver Kieselbach 0ec481b479 feat(agent): outcome-calibrated agent_late_start severity and phrasing
Fleet data (2026-07-28, agent 2.0.1380) showed the late-start detector was
outcome-blind: tenant 659c3a90 collected 12 Warnings with the post-mortem/
hung-script note on enrollments that succeeded — the bootstrap merely ran
late in the ESP queue and the agent observed the healthy final minutes live.

New pure helper ObservationCoverage.DescribeLateStart:
- Succeeded -> Info with a tail-only-coverage note (IME-replay hint,
  bootstrap-ordering review if recurring); no alarm framing.
- Failed/TimedOut -> Warning with the existing post-mortem/hung-script note.
- oobeStateAtAgentStart refines the message only: "completed" = pure
  post-mortem, "in_progress" = live tail observed; not_started/unavailable
  add no claim (value flips to not_started after mid-OOBE reboots and
  "completed" is routine on post-desktop restarts).

The coverage gate (10 min boot-to-start / 5 min uptime) is unchanged, and
the OOBE state is deliberately NOT a gate: the canonical hung-script
failure (f148976f) terminates on the still-shown ESP screen, i.e. with
OOBE reading in_progress.

8 new tests (outcome x oobe-state matrix); V2.Core suite 2067 green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 12:53:17 +02:00
Oliver Kieselbach 3ec716a3ec feat: expose API latency to MCP field projection and tool docs
- SearchSessionsFunction fields whitelist += avgApiLatencyMs,
  apiRequestCount so search_sessions lean sweeps (e.g.
  fields=geoCountry,avgApiLatencyMs,apiRequestCount) can aggregate
  latency without full SessionSummary payloads
- search_sessions tool doc lists the new fields incl. the weighting
  hint; get_geographic_metrics doc names avgApiLatencyMs /
  apiLatencyVsGlobalPct as the backend-region distance metric

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 12:28:01 +02:00
Oliver Kieselbach b0bb7233e4 chore: drop Outbound IP row from Connectivity section
The outbound_ip trace event stays available for analysis (timeline,
MCP); only the UI row is removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 12:20:35 +02:00
Oliver Kieselbach 6758dc59e4 refactor: move API latency into Device Details "Connectivity" section
The extra grid cell in the Session Info card forced a fourth row once
Device Validation was present. The latency reading is a "when needed"
detail, so it moves into the collapsed Device Details card as a new
Connectivity section instead — together with values that were collected
but never displayed anywhere: the estimated internet bandwidth
(network_bandwidth_estimate incl. DO-throttle hint), total telemetry
upload volume (agent_metrics_snapshot), and the outbound IP.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 12:12:08 +02:00
Oliver Kieselbach ffd70e0383 feat: geographic API-latency view (per-location weighted avg by geo)
Surfaces the persisted per-session AvgApiLatencyMs in the geographic
views so operators can compare regions when deciding on new backend
locations (Asia/US):

- ComputeGeographicMetrics: per-location and global request-weighted
  latency (sum(avg*requests)/sum(requests)) + ApiLatencyVsGlobalPct;
  serves both the tenant and the GA cross-tenant endpoint
- GeoMetricsSessionProjection extended (column-projected scan would
  otherwise drop the new fields); equivalence test pins it
- Drilldown: LocationSessionRow carries the per-session values; sessions
  table + summary tile show them
- geographic-performance page: sortable "API Latency" column with
  absolute color buckets (<250 green .. >=800 red), global benchmark
- PlatformMetricsService: session AvgLatency now exact from cumulative
  counters when present (fallback: mean of interval averages)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 11:58:53 +02:00
Oliver Kieselbach c18154d76d feat: persist per-session agent-to-backend API latency (AvgApiLatencyMs)
The V2 agent already measures HTTP round-trip latency per request
(NetworkMetrics, cumulative _totalLatencyMs) but only emitted the
per-interval delta average. Operators deciding on new backend regions
(Asia/US) need the session-wide average, aggregatable by GeoCountry.
Measured baseline: Jakarta sessions average ~600-1050 ms client-side vs
94 ms server-side processing — the gap is pure network distance to GWC.

- Agent: agent_metrics_snapshot now carries cumulative net_total_latency_ms
  (total/requests = whole-session average; idempotent by construction)
- Ingest: last matching snapshot of a batch projects AvgApiLatencyMs +
  ApiRequestCount onto the Session row (fire-and-forget, fail-soft,
  last-write-wins; agents predating the field simply never match)
- SessionsIndex: both fields mirrored (merge + BuildSessionIndexEntity
  superset) so list/GA views can aggregate latency per geo
- Web: session detail shows "API Latency" when present

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 11:44:46 +02:00
Oliver Kieselbach f524473305 Merge branch 'main' of https://github.com/okieselbach/Autopilot-Monitor 2026-07-28 09:54:07 +02:00
Oliver Kieselbach 8fe420b902 updated commands 2026-07-28 09:53:57 +02:00
dependabot[bot] 2e8c2cdb2f chore(deps): bump @modelcontextprotocol/sdk (#110)
Bumps the mcp group in /src/McpServer/autopilot-monitor-mcp with 1 update: [@modelcontextprotocol/sdk](https://github.com/modelcontextprotocol/typescript-sdk).


Updates `@modelcontextprotocol/sdk` from 1.29.0 to 1.30.0
- [Release notes](https://github.com/modelcontextprotocol/typescript-sdk/releases)
- [Commits](https://github.com/modelcontextprotocol/typescript-sdk/compare/v1.29.0...1.30.0)

---
updated-dependencies:
- dependency-name: "@modelcontextprotocol/sdk"
  dependency-version: 1.30.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: mcp
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-28 09:44:13 +02:00
dependabot[bot] e04b75dd53 chore(deps): bump the web group (#111)
Bumps the web group in /src/Web/autopilot-monitor-web with 3 updates: [next](https://github.com/vercel/next.js), [@next/bundle-analyzer](https://github.com/vercel/next.js/tree/HEAD/packages/next-bundle-analyzer) and [eslint-config-next](https://github.com/vercel/next.js/tree/HEAD/packages/eslint-config-next).


Updates `next` from 15.5.21 to 15.5.22
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/compare/v15.5.21...v15.5.22)

Updates `@next/bundle-analyzer` from 16.2.11 to 16.2.12
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/commits/v16.2.12/packages/next-bundle-analyzer)

Updates `eslint-config-next` from 15.5.21 to 15.5.22
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/commits/v15.5.22/packages/eslint-config-next)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 15.5.22
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: web
- dependency-name: "@next/bundle-analyzer"
  dependency-version: 16.2.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: web
- dependency-name: eslint-config-next
  dependency-version: 15.5.22
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: web
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-28 09:41:30 +02:00
Oliver Kieselbach d55b0ff864 test(backend): pin the cross-tenant redaction on metrics/ime-versions
GET metrics/ime-versions is MemberRead and the archive is one global
partition, so every tenant member reads the same platform-wide rows. The only
thing between an ordinary tenant user and another tenant's identifiers was an
inline projection with no test behind it.

There is no HTTP-level harness in this suite - nothing constructs an
HttpRequestData with a RequestContext - so the projection was unreachable from
a test while it lived inside Run. Extracted it to a static
BuildResponsePayload(versions, hasGlobalScope), matching the pattern
GetTpmPssUnsupportedFunction already uses for the same reason. Behaviour is
unchanged: the payload is returned as `object`, and System.Text.Json resolves
an `object` declared type via the runtime type, so both branches keep the wire
format they had inline. A test asserts that rather than trusting it.

10 tests, asserting on serialized JSON in camelCase (Program.cs's naming
policy) because a property dropped from an anonymous type is only a real
redaction once it is absent from the wire:

  - non-global: firstSeenTenantId / firstSeenSessionId absent, and their GUID
    VALUES absent anywhere in the payload (a model rename must not let the
    identifier survive under another key)
  - non-global: exactly {version, firstSeenAt, lastSeenAt, sessionCount}
  - non-global: sessionCount survives - the counts are platform-wide by
    decision, pinned so it cannot drift either way unnoticed
  - global (Admin or Reader): full entry including both identifiers
  - both scopes: array shape, empty archive, row order preserved

Mutation-checked both directions: forcing the global branch fails 3 tests,
dropping sessionCount from the projection fails 2. Full suite 3521 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 19:33:56 +02:00
Oliver Kieselbach 123ba0e55e fix(web): fleet-health crash in GA all-tenants scope - repeatDevices absent
The worker serializer omits null fields (WhenWritingNull), so the
aggregated global/metrics/device-journeys response carries no
repeatDevices at all. The strict `=== null` check missed undefined and
fell through to repeatDevices.length, crashing the page. Verified
against the deployed chunk: crash column 11464 is the `0===o.length`
read behind the `null===o` guard.

Loose `== null` check + optional DTO field; absent and null both render
the "select a tenant" hint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 19:28:14 +02:00
Oliver Kieselbach c8175ff7b1 fix(mcp): drop npm from the runtime image, bump tar to 7.5.22
Defender for Cloud reports CVEs against the mcp container image that the
repo scanners cannot see. The subassessment evidence paths split them in two:

  /usr/local/lib/node_modules/npm/**  - the npm 10.9.8 that node:22-slim
  bundles, vendoring @sigstore/core 2.0.0, brace-expansion 2.0.2, tar 7.5.11
  (critical), sigstore, picomatch, ip-address, plus npm itself (high, no fix
  available). Present in all 20 images including the current one. Dependabot
  reads package-lock.json, so none of this is visible to it.

  /app/node_modules/**  - body-parser 2.2.2, fast-uri 3.1.2, hono, protobufjs,
  tar 7.5.16, all confined to images from 2026-07-21 and earlier and already
  fixed by 7df208fa/c3e9621a. Those findings age out with the old ACR tags.

npm is build-time only here: CMD is `node dist/index.js`, the HEALTHCHECK is
`node -e`, the container app spec overrides no command and probes over HTTP,
and src/ contains no child_process call. So remove npm rather than upgrade it
- npm 11.x still vendors @sigstore/core 3.1.0, itself inside the vulnerable
<= 3.2.0 range, and would clear nothing.

Separately, tar 7.5.20 -> 7.5.22 in our own tree (GHSA-r292-9mhp-454m, via
@huggingface/transformers -> onnxruntime-node). 7.5.22 satisfies the declared
^7.0.1, so no override is needed. fast-uri deliberately stays on 3.1.4: it is
patched against all four fast-uri advisories, and ajv 8.18.0 declares ^3.0.1,
so forcing 4.x would be an override against a major range.

npm audit --omit=dev: 0 vulnerabilities. tsc clean, 455 tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 19:00:07 +02:00
Oliver Kieselbach 0efeb0f734 fix(backend): alert-webhook 500 on non-seekable body - bounded read replaces Length check
With ASP.NET Core integration the HttpRequestData body is a
forward-only stream whose Length property throws NotSupportedException
(verified in prod: 500 + System.NotSupportedException on the first
POST). The size cap is now enforced by reading at most cap+1 chars via
ReadBlockAsync instead of asking the stream for its length.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 18:52:59 +02:00
Oliver Kieselbach 7b975d46b6 feat(backend): Azure Monitor alert webhook - platform alerts join the ops-event bus
New anonymous POST ops/alert-webhook receives Common Alert Schema
notifications from an Action Group webhook receiver and re-emits them
as ops events (category Platform, type AzureMonitorAlert), so Azure
Monitor alerts flow through the existing OpsEventService ->
OpsAlertDispatchService fan-out (Telegram/Teams/Slack + portal
ops-events list) like every internally raised operational event. The
delivery path is independent of the portal: a SWA outage cannot
suppress the notification; the action group's e-mail receiver stays as
the backend-independent fallback.

Security: action-group webhooks cannot send custom headers, so the
shared secret travels as ?secret= and is compared in constant time
(FixedTimeEquals) against the OpsAlertWebhookSecret app setting -
fail-closed while unset. 64 KB payload cap, defensive essentials-only
parsing (only alertRule is mandatory), severity mapped Sev0->Critical,
Sev1->Error, Sev2->Warning, Sev3/4->Info with unknown values falling
back to Info so schema drift can never escalate.

Dual-registered per convention: EndpointAccessPolicyCatalog entry
(PublicAnonymous), OPS_EVENT_TYPES in OpsAlertRulesSection (rule
wiring) and Platform category in OpsEventsSection (badge + filter).

Backend suite 3511 green (incl. new helper tests + catalog
completeness), web tsc + vitest 651 green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 18:39:22 +02:00
Oliver Kieselbach 7680bdff7b fix(web): disable RSC prefetch on nav links - portal freeze root cause
Navbar + GlobalSidebar render the full navigation on every page, so the
App Router's default viewport prefetch fired one ?_rsc= request per
visible link against the SWA runtime (~10.4k requests / 14 days). That
origin serves document, chunks and RSC payloads over HTTP/1.1 (6
connections per origin); when the runtime queued the prefetches (p95
896ms, tail up to 234s), they held every socket and the portal froze
until the queue drained. Measured on the 2026-07-27 17:25 incident:
/admin/presence?_rsc= at 151s while the backend API answered in <2.2s.

All 16 nav links now set prefetch={false}. Every target route is a
client component behind ProtectedRoute that fetches its own data after
mount, so the prefetched RSC payload carried no benefit.

New OKF doc docs/web/portal-navigation-prefetch.md records the full
evidence chain and where browser telemetry lives (separate App Insights
resource, not reachable via MCP query_backend_logs).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 18:27:24 +02:00
Oliver Kieselbach 93094653a2 fix(web): tenant-scope seed races on usage-metrics/sla - own-tenant stats shown for override
A GA with a persisted tenant override opening /usage-metrics (or /sla) saw
their OWN tenant's stats until a manual refresh. useGlobalAdminScope seeded
the persisted selection in a post-mount effect, so the page effect first saw
effectiveTenantId = own tenant and fired an own-tenant fetch immediately
followed by the override fetch; with raw authenticatedFetch (no latest-wins
guard, unlike useAuthenticatedFetch since 82934a37) the last-RESOLVED
response won.

- useGlobalAdminScope: seed the GA selection render-time (same pattern as
  the delegated seed) so consumers never observe a transient own-tenant
  effectiveTenantId - the wrong fetch no longer starts at all.
- usage-metrics/sla pages: latest-wins sequence guard in fetchMetrics so a
  superseded request can neither write data/notifications nor clear the
  newer request's loading state (covers rapid selector switches).

Verified: tsc 0 errors, vitest 651/651.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 16:08:12 +02:00
Oliver Kieselbach ba7d14b070 fix(insights): post-deploy MCP findings - projection drop, aggregate starvation, duration artifacts
Three defects found by measuring production after the 1.5.1012 deploy:

- AppMetricsProjection dropped TerminalState, so the PR0 skip
  classification never reached metrics/app (totalSkipped stuck at 0).
  Column added; equivalence fixture now populates TerminalState/AppId/
  EspBlocking/AppIdCollision and asserts them (proven: test fails
  without the projection fix).
- BuildTimeAttributionAggregates excluded breakdowns on ANY quality
  flag; 903/1000 carry BlockingSetUnknown, starving fleet stats
  (Clean=12 vs 331 excluded on the largest tenant). New
  TimeAttributionFlagQuality.DurationCriticalFlags mask - only
  ClockSkew/PartialObservation/WGAnchors exclude; blocking-set flags
  keep their measured spans (positive-evidence semantics unchanged).
- HasMeasuredDuration gets an upper plausibility bound (6h = default
  AgentMaxLifetimeMinutes): 490 rows carried back-stamped completions
  (up to 117 days; 60175s app rows in a 2697s session) that dominated
  slowestApps with six-figure averages. Read-side, heals old + new rows.

Backend 3484/3484 green; solution build clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 14:50:55 +02:00
Oliver Kieselbach 7f7330e9e5 feat(web): auto-recover from post-deploy stale-chunk failures (one reload, loop-guarded)
A tab still running the previous deployment's bundle requests lazily-loaded
chunks (next/dynamic charts, route chunks) whose content-hashed filenames no
longer exist after the SWA swap - the user is stuck on "Loading chart..." or a
dead navigation until a manual refresh. Standard remedy: reload once so the
fresh document references the new bundle.

- utils/chunkReloadRecovery.ts: chunk-error detection (webpack ChunkLoadError +
  Chromium/Firefox/Safari dynamic-import variants) and a sessionStorage guard -
  at most one auto-reload per tab per 5 minutes; without working storage no
  auto-reload at all, so a reload loop is structurally impossible. Emits a
  ChunkReloadRecovery App Insights event before reloading.
- components/ChunkReloadRecovery.tsx (root layout): catches rejected dynamic
  imports via unhandledrejection and failed /_next/ script/link tags via a
  capture-phase error listener (third-party scripts never trigger a reload).
- app/error.tsx: a chunk error reaching the boundary reloads once instead of
  showing the error card; inside the guard window it falls through to the card.

Verified: tsc clean, vitest 651/651 (8 new tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 12:47:34 +02:00
Oliver Kieselbach 82934a3719 fix(web): tenant-aware unrestricted badge + latest-wins fetch guard on rules pages
GA override on gather-rules called GET /api/global/config/{tenantId}, a route
that does not exist in the backend; the swallowed 404 left the unrestricted
badge showing the GA's HOME tenant's mode for every foreign tenant. Both paths
now read the member-readable config/{tenantId}/feature-flags (MemberRead has a
global-scope bypass and the route is RouteParam-scoped), with a reset-on-switch
so a failed fetch never carries the previous tenant's mode over. The dead
globalConfig.tenant entry is removed from the URL register.

useAuthenticatedFetch now enforces latest-wins: the scope hook seeds a
persisted foreign tenant only after mount, so an own-tenant fetch races the
override fetch and the last-RESOLVED response used to win, leaving the rules
list on the home tenant while the selector showed the override. Stale requests
no longer write data/error/loading; the spinner is owned by the last-started
non-silent request.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 12:26:01 +02:00
116 changed files with 4854 additions and 342 deletions
+13
View File
@@ -119,6 +119,19 @@ normal IME deployment flow. Spam containment, in order:
payload; the event's own timestamp is the emission time, so the session's `StartedAt`
(which the backend aligns to the earliest event) is not dragged backwards.
Presentation (2026-07-28, customer feedback — an unannounced wall of pre-agent Error
events read like a live failure):
- The scan buffers the (capped) record set, first emits an **Info summary event**
(`modern_deployment_log`, payload `backfillSummary: true` + `replayedCount` /
`droppedOverCap` / `lookbackMinutes`) announcing the replay block, then replays the
records **oldest-first** so the timeline sequence follows event-log chronology.
- The web timeline (`EventTimeline.tsx`) badges `backfilled: true` events („⟲ Backfilled",
slate, explanatory tooltip) and renders the original `timeCreated` as the row time with a
"(reported …)" suffix; the expanded metadata block gains a `Recorded` line. Severity is
deliberately **not** downgraded — unresolved pre-agent errors must still surface in the
Error filter.
# Schema — Error codes (backend known-issue map)
Codes documented in the known-issues / troubleshooting-FAQ pages that appear in ESP/enrollment
+108
View File
@@ -0,0 +1,108 @@
---
type: Concept
title: Gather Rule Debug Log — the per-tenant evaluation trace for rules that produce nothing
description: EnableGatherRuleDebugLog (ConfigVersion 36) makes the agent write a local trace file explaining every gather-rule evaluation decision — scope skips, missing intervalSeconds, on_change suppression, empty collector results, logparser position/parse-mode/no-match outcomes — so customers can self-diagnose rules that never show up in the timeline.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleDebugLog.cs
tags:
- agent
- gather-rules
- diagnostics
- tenant-configuration
timestamp: 2026-07-28T00:00:00+02:00
---
# Gather Rule Debug Log
## Schema
### The problem
A gather rule that produces no timeline events is indistinguishable from a rule that
never ran. The pipeline has many *silent-nothing* exits — before this feature, none of
them were visible to the customer:
| # | Decision point | Where |
|---|---|---|
| 1 | Rule delivered but `Enabled=false` | `GatherRuleExecutor.UpdateRules` |
| 2 | `trigger=interval` without `intervalSeconds` — never runs | `UpdateRules` timer filter |
| 3 | Interval timer's first tick comes only after one **full** interval | `UpdateRules` timer creation |
| 4 | Out of phase scope (`activePhases` / `activeFromPhase` not latched); incl. `currentPhase=Unknown` before the first phase signal | `IsRuleInScopeLocked` |
| 5 | phase_change / phase_exit dedup — once per (rule, phase) | `OnPhaseChanged` |
| 6 | Collector returned an **empty** result (e.g. `emitOnlyIfExists=true` and target absent) | `ExecuteRule` |
| 7 | `emit_mode=on_change` — result hash unchanged, emit suppressed | `ShouldEmitOnChange` |
| 8 | Guard blocked the target (emits `security_warning`, but the rule yields no data) | `GatherRuleContext.EmitSecurityWarning` |
| 9 | logparser: no files matched, position tracker says nothing new, regex no-match, plain-text file parsed in default `cmtrace` mode, `maxLines` cap, wildcard capped at 20 files | `LogParserCollector` |
| 10 | Rule execution threw (agent log had only the message, no stack) | `ExecuteRule` catch |
### The setting
`TenantConfiguration.EnableGatherRuleDebugLog` (`bool?`, default null→false), served to the
agent as `AgentConfigResponse.EnableGatherRuleDebugLog` — **ConfigVersion 36**. Portal:
Settings → Agent → Settings, toggle "Gather Rule Debug Log", stored via the usual
tenant-config roundtrip (`TableConfigRepository` Store+Map). Also shown in the admin
tenant-config report.
`RemoteConfigMerger` projects the bool onto a runtime path (same pattern as
`EnableImeMatchLog`): `true` → `Constants.GatherRuleDebugLogPath` =
`%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log`; `false` → null, unless a
`--gather-debug-log <path>` CLI override supplied a non-default path. The field is not
security-sensitive and deliberately **survives the offline config cache**.
### The trace file
`GatherRuleDebugLog` is an append-only writer held by `GatherRuleExecutor` and exposed
to collectors via `GatherRuleContext.DebugLog(ruleId, stage, message)` (null-safe no-op
when disabled — zero cost on the normal path). Line format:
```
2026-07-28T09:15:02.123Z | RULE-ID | stage | message
```
Stages: `config` (registration summary per rule, header, zero-rules note), `trigger`
(timer scheduled, phase/event fired, dedup), `scope` (skips with reason incl. current
phase), `exec` (execution start), `collector` (null/empty results with likely cause),
`guard` (allowlist block), `emit` (event emitted, hash change), `suppress` (on_change
suppression with hash prefix + streak), `logparser` (per-file outcome: lines read,
position range, match count, parse failures, regex timeouts, mode; plus the
`format=text` hint when every line fails CMTrace parsing), `error` (full exception
**with stack trace** — the agent log keeps only the message).
Volume control: 10 MB cap with a single `.old` rotation generation (~20 MB worst case).
Every write failure is swallowed — tracing must never break gathering. The trace never
leaves the device on its own — but because it lives in the agent log folder
(`Constants.LogDirectory`), the diagnostics package (`DiagnosticsPackageService`,
`*.log` pattern, `AgentLogs/` section) picks it up automatically, so a customer-initiated
diagnostics upload hands the trace to support without manual copying.
### Semantics
* **No hot-reload** — the flag is consumed once when `DefaultComponentFactory` builds
the `GatherRuleExecutorHost` at session start (per-enrollment lifecycle, same as
`EnableImeMatchLog`). Enable it in the tenant, then the *next* enrollment writes the
trace.
* **Zero-rules case is traced too** — flag on + backend delivered no gather rules →
a one-shot standalone `config` line, so the file is never just absent.
* **`--run-gather-rules` always traces** (operator diagnostic mode; CLI override or
default path) and mirrors every line to the console under `--console`.
## Examples
A logparser rule with `emit_mode=on_change` that "delivers nothing" becomes:
```
... | LP-01 | config | registered: trigger=interval, collector=logparser, target=C:\...\app.log, unscoped, emitMode=on_change, interval=300s
... | LP-01 | trigger | interval timer scheduled every 300s (first run after one full interval — no immediate execution)
... | LP-01 | exec | executing: collector=logparser, target=C:\...\app.log
... | LP-01 | logparser | app.log: read 1000 lines from position 0->81234, matched 0, parseFailures=1000, regexTimeouts=0, mode=cmtrace
... | LP-01 | logparser | every line failed CMTrace parsing — if this is a plain-text log, set parameter format=text
```
— the answer ("plain-text log parsed in CMTrace mode") is in the file.
## Citations
* Executor + trace points: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleExecutor.cs`
* Writer: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleDebugLog.cs`
* Config projection: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Configuration/RemoteConfigMerger.cs`
* Backend serve + ConfigVersion: `../../src/Backend/AutopilotMonitor.Functions/Functions/Config/GetAgentConfigFunction.cs`
* Phase scoping / emit mode background: `../rules/gather-rule-phase-scoping.md`
+5
View File
@@ -20,6 +20,7 @@ https://docs.autopilotmonitor.com (separate repository).
* [V2 Agent](agent/index.md) - How the on-device agent works: [runtime overview](agent/overview.md), [decision engine](agent/decision-engine.md) (reducer, signals, completion arms), [logs & persistence](agent/logs-and-persistence.md) (signal log, snapshot, crash recovery), [Hello wizard un-skip](agent/hello-wizard-unskip.md) (why a single "WHfB disabled" read never decides completion), [Autopilot ZTD diagnostics](agent/autopilot-ztd-diagnostics.md) (event IDs, diagnostic registry, endpoints, error-code map).
* [Build Counter Blob](agent/build-counter-blob.md) - How agent build numbers are minted: shared counter blob with ETag-CAS so local and CI builds never collide; CI additionally attests build provenance (Sigstore).
* [Agent Endpoint Migration](agent/endpoint-migration.md) - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins.
* [Gather Rule Debug Log](agent/gather-rule-debug-log.md) - EnableGatherRuleDebugLog (ConfigVersion 36): the per-tenant local trace file that explains every gather-rule evaluation decision — scope skips, on_change suppression, empty collector results, logparser position/parse-mode outcomes — for rules that produce no timeline events.
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
* [Secure Boot CA-2023 Detection](agent/secureboot-ca2023-detection.md) - Why ANALYZE-SEC-001 v3 reads the UEFI db/KEK variables directly instead of trusting the SecureBoot\Servicing registry, and how the one-sided uefiCa2023FirmwareConfirmed marker plus a not_exists precondition give absence-tolerant suppression across old and new agents.
* [ESP Policy-Provider Stall Detection](agent/esp-policy-provider-stall.md) - The EnrollmentStatusTracking CSP's no-timeout Setup/Apps wait is keyed to the Intune-registered "Sidecar" provider by NAME: co-management's "ConfigMgr" parks the user ESP at "Apps (Identifying)" for days even with TrackingPoliciesCreated=1 (issue #106); the always-on two-arm esp_policy_provider_stalled tripwire (provider_incomplete | sidecar_provider_missing) and the SOFTWARE\Microsoft\Windows\Autopilot gather-rule allowlist entry that surface it.
@@ -41,6 +42,10 @@ https://docs.autopilotmonitor.com (separate repository).
* [MCP Docs Corpus](mcp/docs-corpus.md) - How the published customer documentation becomes the `search_docs` corpus: CRLF-safe heading chunking, build-time embedding as a third precomputed section, and why a docs edit needs an MCP redeploy.
# Web Portal
* [Portal Navigation Prefetch & the SWA Runtime Bottleneck](web/portal-navigation-prefetch.md) - Why every navigation Link sets `prefetch={false}`: the App Router's viewport prefetch fans out one RSC request per visible link and the Static Web Apps SSR runtime queues them server-side into a multi-minute tail. Records the follow-up measurement (the Standard plan changed nothing; the prefetch storm was an accidental keep-alive), where browser telemetry actually lives, and why this curl build cannot be trusted for HTTP versions.
# Security & Identity
* [MCP OAuth Flow](mcp-oauth-flow.md) - Who authenticates where when connecting an AI client to the MCP server; two identities, three parties plus Entra ID.
+10
View File
@@ -1,7 +1,17 @@
# Log
## 2026-07-28
* **Update**: `agent/autopilot-ztd-diagnostics.md` — presentation layer for the startup error backfill (customer feedback: an unannounced wall of pre-agent Error events with a single identical timestamp read like a live failure). Agent: the backfill scan now buffers the capped record set, emits an Info summary event first (`modern_deployment_log`, `backfillSummary: true` + `replayedCount`/`droppedOverCap`/`lookbackMinutes`), then replays oldest-first so the timeline sequence follows event-log chronology. Web: `EventTimeline.tsx` badges `backfilled: true` events („⟲ Backfilled", tooltip) and renders the original `timeCreated` as the row time with a "(reported …)" suffix plus a `Recorded` metadata line — works retroactively for already-stored events. Severity is deliberately not downgraded.
* **Creation**: Added `agent/gather-rule-debug-log.md` — the new per-tenant setting `EnableGatherRuleDebugLog` (ConfigVersion 35 → 36, portal toggle under Settings → Agent, `EnableImeMatchLog`-style bool→path projection in `RemoteConfigMerger`). When enabled, the agent writes `%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log`: one pipe-delimited line per evaluation decision across ten previously silent "produced nothing" exits — disabled rules, `interval` without `intervalSeconds` (never runs), the first-tick-after-one-full-interval window, phase-scope skips incl. `currentPhase=Unknown`, per-(rule, phase) dedup, empty collector results (`emitOnlyIfExists`), `on_change` hash suppression (hash prefix + streak), guard blocks, logparser per-file outcomes (position skips, parse failures with a `format=text` hint when every line fails CMTrace parsing, `maxLines`/wildcard caps), and full exception stacks that the agent log drops. 10 MB cap + one `.old` generation; no hot-reload (per-enrollment lifecycle); flag survives the offline config cache; `--run-gather-rules` always traces and echoes under `--console`; zero-rules delivery leaves a one-line file instead of nothing. Side fix: `GatherRuleExecutorHost` now passes the logger into `InformationalEventPost`, so gather-event emissions appear in the agent DEBUG log.
* **Correction + Update**: `web/portal-navigation-prefetch.md`. The HTTP/1.1 claim in the 2026-07-27 entry below is **wrong** and has been struck from the document: `portal.autopilotmonitor.com` negotiates **h2**, verified by `openssl s_client -alpn h2,http/1.1` and a .NET `HttpClient` returning `HTTP/2.0`. The false reading came from the Windows/Schannel curl build in this environment, which has no HTTP/2 support and therefore always reports `1.1` — do not use it to determine HTTP versions. There is no six-connection cap in play; the queue is server-side in the SWA SSR runtime, which does not change the fix or the evidence, only the mechanism. Added an `Outcome` section with 20 h of post-change telemetry: the Free → Standard plan change (16:02:44 UTC) produced **no measurable latency gain** once stratified by the gap to the previous request (p50 Free → Standard: `<10s` 85 → 95 ms, `5-30min` 429 → 441 ms; tail rates 1.64 %/0.374 % over 2 s/10 s vs 6 and 1 events at n = 258 against 4.2 and 1.0 expected; `browserTimings` Mann-Whitney z = −0.24, p = 0.81). The freezes were removed by `prefetch={false}` (16:34:13 UTC): worst `browserTimings.totalDuration` 225,277 → 4,856 ms. The prefetch storm was also an **accidental keep-alive** — requests under 10 s apart fell from 85 % to 46 %, and on the identical never-cacheable `/sessions/{id}` route the median rose 87 → 211 ms and p90 465 → 1,480 ms, with one 11,441 ms abandoned request (`resultCode 0`) after 517 minutes idle. Synthetic bursts warm the runtime and must not be used to benchmark this (round 1 p50 288 ms vs rounds 4–25 p50 113 ms).
## 2026-07-27
* **Creation**: Added `web/portal-navigation-prefetch.md` (and a `Web Portal` section to the index) — root cause of the minute-long portal freezes. `GlobalSidebar` and `Navbar` render the whole navigation on every page, so the App Router's default viewport prefetch issued one `?_rsc=` request per visible link against `portal.autopilotmonitor.com` (~10,400 in 14 days). *(The original entry attributed the freeze to HTTP/1.1 and a six-connection-per-origin cap — see the 2026-07-28 correction; the queue is server-side.)* When the Static Web Apps managed runtime queued the prefetches (p50 91 ms, p95 896 ms, but max 234 s / 21 samples over 30 s), the navigation the user wanted was stuck behind that queue. Measured on the 2026-07-27 17:25 CEST incident: `/admin/presence?_rsc=` 151 s, three siblings finishing within 340 ms of each other at ~37.5 s, all `200`, while the backend API answered in 1.6–2.2 s and `browserTimings` showed `networkDuration` 16 ms against `processingDuration` 225 s. Every navigation Link now sets `prefetch={false}` — these routes are client components behind `ProtectedRoute` that fetch their own data, so the prefetched RSC payload carried no benefit. The doc also records that browser telemetry lives in a **separate** App Insights resource (`AUTOPILOT_MONITOR_WEB_APPINSIGHTS_ID`) that the MCP `query_backend_logs` tool cannot reach, which is why the freezes left no trace in the backend logs.
* **Creation**: Added `backend/rule-regression-radar.md` — F3 regression radar (analyze rules only; gather rules wait for their per-batch stats dedup fix): per (tenant, rule) daily hit rate from `RuleStats`, firing only when the trailing 7-day window has ≥5 hit sessions and ≥20 evaluated sessions, the rate is ≥2× the prior 28-day baseline AND the Wilson 95 % intervals are disjoint (deterministic `MetricsMath` primitive with hand-computed test vectors). Suppressed for deleted rules, rules younger than baseline+window (`CreatedAt` grace), rules edited inside the window (`UpdatedAt`), empty baselines, and the `RuleRegressionRadarDisabled` app-setting kill switch. Alert episodes live as `ruleregression|{ruleId}` tracker rows (one bell + one `RuleFrequencyRegression` ops event per episode; numbers refreshed while active; re-armed under 1.5× baseline, on fire-stop, or by the 30-day tracker retention). On-fire dimension correlation over osBuild/model/agentVersion/imeVersion with the "correlated, not necessarily causal" wording contract. Surfaces: `regressions[]` in the rule-stats response (MCP inherits), the analyze-rules page's 30-day sparkline and regression badge.
* **Creation**: Added `backend/device-journeys.md` — F2 device history / First-Time-Right: the device key (trim + case-fold serial, junk/placeholder exclusion incl. the agent's `Unknown` WMI sentinel), journey grouping over the terminal set Succeeded/Failed/Incomplete (first success ends a journey, a >30-day gap from the previous attempt's CompletedAt starts one, WhiteGlove's single session row is ONE attempt, open sessions are never attempts), the `DeviceHistories` chain (cap 20, written inline at the terminal seam, healed by the sweep, deleted-session refs pruned tombstone-driven, deliberately not in the per-session deletion manifest) and the additive `DeviceJourneyAggregates` daily rows (window rate = ratio of sums — no rolling row needed, 180-day retention). Surfaces: the session-detail "Attempt N" banner with server-computed attempt numbers (virtual-attempt rule for live sessions), the Fleet Health First-time-right section (≥20-journey gate, attempt histogram, repeat-devices violator list), and the MCP `get_device_history` tool (serial mode / fleet FTR mode).
+120
View File
@@ -0,0 +1,120 @@
---
type: Concept
title: Portal Navigation Prefetch & the SWA Runtime Bottleneck
description: Why every navigation Link in the portal sets prefetch={false} — the App Router's viewport prefetch fires one RSC request per visible link, and the Static Web Apps SSR runtime queues them server-side into a multi-minute latency tail. Also records what the follow-up measurement showed about the Standard SKU and the keep-alive side effect.
resource: src/Web/autopilot-monitor-web/components/GlobalSidebar.tsx
tags:
- web
- performance
- nextjs
- static-web-apps
- observability
timestamp: 2026-07-27T18:00:00+02:00
---
# Context
The portal is a Next.js 15 App Router application hosted on Azure Static Web Apps. Every page under
the authenticated shell is a client component behind `ProtectedRoute` that fetches its own data from
the Function App (`autopilotmonitor-api-eu`) after mount. The RSC payload a route returns on
navigation therefore carries no page data — only the shell.
`GlobalSidebar` and `Navbar` render the full navigation on every page. With the App Router's default
prefetch behaviour, each visible `<Link>` triggers a background `GET /<route>?_rsc=<hash>` as soon as
it enters the viewport.
# Mechanism
Two properties combine into a portal-wide freeze:
1. **Fan-out.** The navigation is fully rendered on each page load, so a single navigation issues one
RSC request per visible link — measured at ~10,400 requests over 14 days.
2. **A heavy latency tail on the SSR runtime.** RSC responses are prerendered and edge-cached
(`x-nextjs-cache: HIT`, `s-maxage=31536000`), so the median is fast. On a cache miss or cold start
the Static Web Apps managed runtime queues them **server-side**, and the queue drains all at once.
The navigation the user wants is then stuck behind that queue and the tab shows a loading spinner or a
blank page until it drains. Nothing fails — the requests eventually return `200`, which is why the
failure leaves no error trace anywhere.
> **Correction (2026-07-28).** The original version of this document named a third property: that
> `portal.autopilotmonitor.com` negotiates HTTP/1.1 and the prefetches exhaust the six-connection
> browser cap. **That is wrong.** It came from `curl -w '%{http_version}'` on a Windows build of curl
> whose Schannel backend has no HTTP/2 support, so it can only ever report `1.1`. ALPN says otherwise:
> `openssl s_client -alpn h2,http/1.1` negotiates **h2** on the portal host, confirmed by a .NET
> `HttpClient` request returning `HTTP/2.0`. There is no six-connection cap and connection starvation
> was never the mechanism — the queue is server-side. Verify HTTP versions with `openssl s_client
> -alpn` or `HttpClient`, never with this curl build.
# Evidence
Incident of 2026-07-27, 17:25–17:29 CEST. Browser-side dependencies during the freeze:
| Request | Duration | Status |
| --- | --- | --- |
| `/admin/presence?_rsc=…` | 151,461 ms | 200 |
| `/gather-rules?_rsc=…` | 52,283 ms | 200 |
| `/progress?_rsc=…` | 37,878 ms | 200 |
| `/analyze-rules?_rsc=…` | 37,559 ms | 200 |
| `/audit?_rsc=…` | 37,539 ms | 200 |
| Backend API, same window | 1,600–2,200 ms | 200 |
The three ~37.5 s responses complete within 340 ms of each other — a queue draining, not independent
slow requests. The corresponding `browserTimings` row for `/geographic-performance` reports
`networkDuration` = 16 ms against `processingDuration` = 225,095 ms: the network delivered the
document immediately and the browser then waited nearly four minutes for the `load` event.
Fourteen-day distribution of RSC prefetches: n = 10,428, p50 = 91 ms, p95 = 896 ms, max = 234,230 ms,
38 over 10 s, 21 over 30 s.
# Outcome, measured 2026-07-28
Two changes landed within half an hour of each other on 2026-07-27: the SWA plan went Free → Standard
at 16:02:44 UTC, and `prefetch={false}` went live at 16:34:13 UTC. Twenty hours of production
telemetry separate their effects.
**The Standard plan produced no measurable latency gain.** Latency is governed almost entirely by the
gap to the previous request — the SSR runtime cools down — so raw percentiles only measure the traffic
mix. Stratified by that gap, the two eras are indistinguishable (p50, Free → Standard): `<10s`
85 → 95 ms · `10-60s` 100 → 117 ms · `1-5min` 295 → 310 ms · `5-30min` 429 → 441 ms. Tail rates match
too: Free ran 1.64 % over 2 s and 0.374 % over 10 s (n = 10,163); Standard produced 6 and 1 events at
n = 258 against 4.2 and 1.0 expected. `browserTimings.totalDuration` by Mann-Whitney: n = 436 vs 48,
z = −0.24, **p = 0.81**.
**The prefetch fix is what removed the freezes** — worst `browserTimings.totalDuration` fell from
225,277 ms to 4,856 ms, and the 30-second events stopped.
**It also removed an accidental keep-alive.** The prefetch storm kept the SSR runtime warm: 85 % of
requests used to arrive less than 10 s apart, now 46 % do. On an identical never-cacheable route
(`/sessions/{id}`) the median rose from 87 ms to 211 ms and p90 from 465 ms to 1,480 ms. After 517
minutes idle, one request took 11,441 ms and was abandoned (`resultCode 0`). The trade — minute-long
freezes gone, ~150 ms added to a cold navigation — is worth it, but it is a trade.
Do not benchmark this with a synthetic burst. Firing requests in quick succession warms the runtime
and measures the best case: in a 25-round `curl --parallel-max 6` run, round 1 came in at p50 288 ms
and rounds 4–25 at p50 113 ms.
# Diagnosing this class of problem
The backend Application Insights resource is the wrong place to look — a freeze appears there as an
*absence* of requests, never as an error. Browser telemetry lives in a **separate** Application
Insights resource addressed by `AUTOPILOT_MONITOR_WEB_APPINSIGHTS_ID`, which the MCP
`query_backend_logs` tool does not reach. Query it through
`.claude/commands/scripts/query-appinsights.sh` with that ID exported, and read `dependencies`
(client-side fetches, including RSC prefetches) and `browserTimings` (the network/processing split of
a document load).
# Contract
Navigation `<Link>` elements in `GlobalSidebar` and `Navbar` set `prefetch={false}`. The prefetched
RSC payload buys nothing for these routes, so the only remaining cost of a click is the route's JS
chunk, which the browser caches after first use.
Do not remove the prop to "speed up navigation" without first re-measuring the tail in the web
Application Insights resource — the prefetch is cheap only while the runtime is warm.
# Citations
* `src/Web/autopilot-monitor-web/components/GlobalSidebar.tsx` — nav link rendering, rationale comment
* `src/Web/autopilot-monitor-web/components/Navbar.tsx` — top-bar and notification links
* [SWA and Oryx node pinning](../architecture.md) — hosting model for the web app
@@ -285,6 +285,54 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
Assert.Equal(@"C:\Temp\custom-ime.log", agent.ImeMatchLogPath);
}
[Fact]
public void Merge_maps_enable_gather_rule_debug_log_true_to_default_path()
{
var agent = NewAgentConfig();
agent.GatherRuleDebugLogPath = null;
var remote = FullRemote();
remote.EnableGatherRuleDebugLog = true;
RemoteConfigMerger.Merge(agent, remote);
Assert.False(string.IsNullOrEmpty(agent.GatherRuleDebugLogPath));
Assert.Equal(
Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath),
agent.GatherRuleDebugLogPath);
}
[Fact]
public void Merge_maps_enable_gather_rule_debug_log_false_clears_path()
{
var agent = NewAgentConfig();
agent.GatherRuleDebugLogPath = Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath);
var remote = FullRemote();
remote.EnableGatherRuleDebugLog = false;
RemoteConfigMerger.Merge(agent, remote);
Assert.Null(agent.GatherRuleDebugLogPath);
}
[Fact]
public void Merge_preserves_custom_gather_rule_debug_log_path_when_remote_disables()
{
// A dev-time --gather-debug-log <custom> override seeds a non-default path on the
// AgentConfiguration. When the tenant remote flips EnableGatherRuleDebugLog off we
// keep the custom path rather than clobbering a deliberate dev override.
var agent = NewAgentConfig();
agent.GatherRuleDebugLogPath = @"C:\Temp\custom-gather.log";
var remote = FullRemote();
remote.EnableGatherRuleDebugLog = false;
RemoteConfigMerger.Merge(agent, remote);
Assert.Equal(@"C:\Temp\custom-gather.log", agent.GatherRuleDebugLogPath);
}
// ============================================================= Null / partial tolerance
[Fact]
@@ -0,0 +1,104 @@
using System;
using System.IO;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// Writer unit tests for <see cref="GatherRuleDebugLog"/>: line format, lazy directory
/// creation, size-cap rotation, and the null-writer no-op path on the context.
/// </summary>
public sealed class GatherRuleDebugLogTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly AgentLogger _logger;
public GatherRuleDebugLogTests()
{
_logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
}
public void Dispose() => _tmp.Dispose();
private string LogPath => Path.Combine(_tmp.Path, "sub", "gather_rules_debug.log");
[Fact]
public void Write_produces_pipe_delimited_line_and_creates_directory_lazily()
{
var log = new GatherRuleDebugLog(LogPath, _logger);
Assert.False(File.Exists(LogPath)); // lazy — nothing until the first write
log.Write("RULE-1", GatherRuleDebugLog.StageScope, "skipped: out of scope");
var lines = File.ReadAllLines(LogPath);
Assert.Equal(2, lines.Length); // header + entry
Assert.Contains("| - | config | -- gather rule debug log started --", lines[0]);
var parts = lines[1].Split(new[] { " | " }, StringSplitOptions.None);
Assert.Equal(4, parts.Length);
Assert.Equal("RULE-1", parts[1]);
Assert.Equal("scope", parts[2]);
Assert.Equal("skipped: out of scope", parts[3]);
}
[Fact]
public void Write_flattens_newlines_except_for_error_stage()
{
var log = new GatherRuleDebugLog(LogPath, _logger);
log.Write("RULE-1", GatherRuleDebugLog.StageCollector, "line1\r\nline2");
log.Write("RULE-1", GatherRuleDebugLog.StageError, "boom\r\n at Frame()");
var content = File.ReadAllText(LogPath);
Assert.Contains("line1 line2", content);
Assert.Contains(" at Frame()", content); // error stage keeps the stack multi-line
}
[Fact]
public void Write_null_ruleId_renders_dash()
{
var log = new GatherRuleDebugLog(LogPath, _logger);
log.Write(null, GatherRuleDebugLog.StageConfig, "no rules delivered");
Assert.Contains(" | - | config | no rules delivered", File.ReadAllText(LogPath));
}
[Fact]
public void Write_rotates_once_when_file_exceeds_cap()
{
var log = new GatherRuleDebugLog(LogPath, _logger, maxBytes: 512);
for (int i = 0; i < 20; i++)
log.Write("RULE-1", GatherRuleDebugLog.StageExec, new string('x', 100));
var oldPath = Path.Combine(Path.GetDirectoryName(LogPath), "gather_rules_debug.old.log");
Assert.True(File.Exists(oldPath), "expected one .old rotation generation");
Assert.True(File.Exists(LogPath));
Assert.Contains("-- rotated", File.ReadAllText(LogPath) + File.ReadAllText(oldPath));
// Never more than one .old generation
Assert.Single(Directory.GetFiles(Path.GetDirectoryName(LogPath), "*.old.log"));
}
[Fact]
public void WriteStandalone_writes_single_config_line()
{
GatherRuleDebugLog.WriteStandalone(LogPath, "no gather rules delivered by backend — nothing to execute", _logger);
var lines = File.ReadAllLines(LogPath);
Assert.Contains(lines, l => l.Contains("no gather rules delivered by backend"));
}
[Fact]
public void Context_DebugLog_is_noop_without_writer()
{
var context = new GatherRuleContext(
_logger, "sess", "tenant", _ => { }, null, new LogFilePositionTracker());
context.DebugLog("RULE-1", GatherRuleDebugLog.StageScope, "should not throw, should not write");
Assert.False(File.Exists(LogPath));
}
}
}
@@ -0,0 +1,156 @@
using System;
using System.Collections.Generic;
using System.IO;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// Debug-trace coverage of <see cref="GatherRuleExecutor"/> (EnableGatherRuleDebugLog):
/// every "silently produced nothing" outcome must leave an explanatory line in the trace
/// file — registration summary, missing interval, scope skips, on_change suppression,
/// empty collector results.
/// </summary>
[Collection("SerialThreading")] // startup rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorDebugLogTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly GatherRuleExecutor _executor;
private readonly string _debugLogPath;
private const string AbsentPath =
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_7e3a1f4b-0000-0000-0000-000000000000";
public GatherRuleExecutorDebugLogTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_debugLogPath = Path.Combine(_tmp.Path, "gather_rules_debug.log");
_executor = new GatherRuleExecutor(
"sess", "tenant", _ => { }, logger,
debugLogPath: _debugLogPath)
{
UnrestrictedMode = true
};
}
public void Dispose()
{
_executor.Dispose();
_tmp.Dispose();
}
private string TraceContent() => File.Exists(_debugLogPath) ? File.ReadAllText(_debugLogPath) : string.Empty;
private static GatherRule Rule(string id, Action<GatherRule>? mutate = null)
{
var rule = new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "startup",
OutputEventType = "gather_test",
Enabled = true,
};
mutate?.Invoke(rule);
return rule;
}
[Fact]
public void UpdateRules_writes_registration_summary_per_rule()
{
_executor.UpdateRules(new List<GatherRule>
{
Rule("GATHER-REG-001", r => { r.Trigger = "interval"; r.IntervalSeconds = 3600; r.EmitMode = "on_change"; }),
Rule("GATHER-REG-002", r => r.Enabled = false),
});
_executor.WaitForStartupRules(30);
var trace = TraceContent();
Assert.Contains("GATHER-REG-001 | config | registered: trigger=interval, collector=registry", trace);
Assert.Contains("emitMode=on_change", trace);
Assert.Contains("interval=3600s", trace);
Assert.Contains("GATHER-REG-002 | config | rule disabled — will never run", trace);
Assert.Contains("interval timer scheduled every 3600s (first run after one full interval", trace);
}
[Fact]
public void UpdateRules_flags_interval_rule_without_intervalSeconds()
{
_executor.UpdateRules(new List<GatherRule>
{
Rule("GATHER-NOINT-001", r => { r.Trigger = "interval"; r.IntervalSeconds = null; }),
});
Assert.Contains("GATHER-NOINT-001 | error | trigger=interval but intervalSeconds is missing — rule will NEVER run",
TraceContent());
}
[Fact]
public void ScopedRule_skip_traces_unknown_phase_reason()
{
var rule = Rule("GATHER-SCOPE-001", r => { r.Trigger = "interval"; r.IntervalSeconds = 3600; r.ActivePhases = new List<string> { "DeviceSetup" }; });
_executor.UpdateRules(new List<GatherRule> { rule });
// Synchronous internal check mirrors the timer-tick gate; the reason must name Unknown.
string reason;
Assert.False(_executor.IsRuleInScope(rule, out reason));
Assert.Contains("currentPhase=Unknown", reason);
}
[Fact]
public void ScopedStartupRule_deferral_is_traced()
{
_executor.UpdateRules(new List<GatherRule>
{
Rule("GATHER-DEFER-001", r => r.ActivePhases = new List<string> { "DeviceSetup" }),
});
Assert.Contains("GATHER-DEFER-001 | scope | startup rule deferred — currentPhase=Unknown", TraceContent());
}
[Fact]
public void OnChange_suppression_and_reemission_are_traced()
{
var rule = Rule("GATHER-OC-001", r => { r.Trigger = "interval"; r.IntervalSeconds = 3600; r.EmitMode = "on_change"; });
_executor.UpdateRules(new List<GatherRule> { rule });
string hashPrefix;
int streak;
var first = new Dictionary<string, object> { ["exists"] = false };
Assert.True(_executor.ShouldEmitOnChange(rule, first, out hashPrefix, out streak));
Assert.Equal(0, streak);
Assert.False(string.IsNullOrEmpty(hashPrefix));
Assert.False(_executor.ShouldEmitOnChange(rule, new Dictionary<string, object> { ["exists"] = false }, out hashPrefix, out streak));
Assert.Equal(1, streak);
Assert.False(_executor.ShouldEmitOnChange(rule, new Dictionary<string, object> { ["exists"] = false }, out hashPrefix, out streak));
Assert.Equal(2, streak);
var changed = new Dictionary<string, object> { ["exists"] = true };
Assert.True(_executor.ShouldEmitOnChange(rule, changed, out hashPrefix, out streak));
Assert.Equal(2, streak); // the streak that just ended
Assert.Equal(2, changed["suppressedPolls"]);
}
[Fact]
public void EmptyCollectorResult_is_traced_via_startup_execution()
{
_executor.UpdateRules(new List<GatherRule>
{
Rule("GATHER-EMPTY-001", r => r.Parameters = new Dictionary<string, string> { ["emitOnlyIfExists"] = "true" }),
});
Assert.True(_executor.WaitForStartupRules(30));
var trace = TraceContent();
Assert.Contains("GATHER-EMPTY-001 | exec | executing: collector=registry", trace);
Assert.Contains("GATHER-EMPTY-001 | collector | registry key/value not found", trace);
Assert.Contains("GATHER-EMPTY-001 | collector | collector returned EMPTY result — nothing emitted", trace);
}
}
}
@@ -0,0 +1,121 @@
using System;
using System.Collections.Generic;
using System.IO;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// Debug-trace coverage of <see cref="LogParserCollector"/>: the classic
/// "rule delivers nothing" cases must be explained in the trace — zero matches,
/// position-tracker skips, and plain-text logs parsed in the default cmtrace mode.
/// </summary>
public sealed class LogParserCollectorDebugLogTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly GatherRuleContext _context;
private readonly string _debugLogPath;
// %TEMP% lives under C:\Users (hard-blocked) — parse targets sit beside the
// test assembly instead, admitted via UnrestrictedMode.
private readonly string _outsideUsers = Path.Combine(
AppContext.BaseDirectory, "gather-debug-tests-" + Guid.NewGuid().ToString("N"));
public LogParserCollectorDebugLogTests()
{
Directory.CreateDirectory(_outsideUsers);
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_debugLogPath = Path.Combine(_tmp.Path, "gather_rules_debug.log");
_context = new GatherRuleContext(
logger, "sess", "tenant",
evt => _events.Add(evt),
null,
new LogFilePositionTracker(),
new GatherRuleDebugLog(_debugLogPath, logger))
{
UnrestrictedMode = true
};
}
public void Dispose()
{
_tmp.Dispose();
try { Directory.Delete(_outsideUsers, recursive: true); } catch { /* best effort */ }
}
private string TraceContent() => File.Exists(_debugLogPath) ? File.ReadAllText(_debugLogPath) : string.Empty;
private static GatherRule Rule(string target, string pattern, string? format = null) => new GatherRule
{
RuleId = "GATHER-LP-DEBUG",
Title = "logparser debug test",
CollectorType = "logparser",
Target = target,
Parameters = format == null
? new Dictionary<string, string> { ["pattern"] = pattern }
: new Dictionary<string, string> { ["pattern"] = pattern, ["format"] = format },
Trigger = "startup",
OutputEventType = "logparser_debug_test",
};
[Fact]
public void ZeroMatches_writes_per_file_outcome_line()
{
var path = Path.Combine(_outsideUsers, "app.log");
File.WriteAllText(path, "nothing interesting\nstill nothing\n");
new LogParserCollector().Execute(Rule(path, "WILL_NEVER_MATCH_[0-9]+", format: "text"), _context);
Assert.Empty(_events);
var trace = TraceContent();
Assert.Contains("app.log: read 2 lines", trace);
Assert.Contains("matched 0", trace);
Assert.Contains("mode=text", trace);
}
[Fact]
public void SecondRun_without_new_content_traces_position_skip()
{
var path = Path.Combine(_outsideUsers, "twice.log");
File.WriteAllText(path, "one line\n");
var rule = Rule(path, "one", format: "text");
var collector = new LogParserCollector();
collector.Execute(rule, _context); // reads the file, advances the position
collector.Execute(rule, _context); // nothing new
Assert.Contains("twice.log: no new content (position", TraceContent());
}
[Fact]
public void PlainTextFile_in_default_cmtrace_mode_gets_format_hint()
{
var path = Path.Combine(_outsideUsers, "plain.log");
File.WriteAllText(path, "plain line one\nplain line two\n");
new LogParserCollector().Execute(Rule(path, "plain"), _context); // default = cmtrace mode
Assert.Empty(_events);
var trace = TraceContent();
Assert.Contains("parseFailures=2", trace);
Assert.Contains("every line failed CMTrace parsing — if this is a plain-text log, set parameter format=text", trace);
}
[Fact]
public void MissingFile_traces_no_files_matched()
{
var path = Path.Combine(_outsideUsers, "does-not-exist.log");
new LogParserCollector().Execute(Rule(path, "x", format: "text"), _context);
Assert.Contains("no files matched target", TraceContent());
}
}
}
@@ -0,0 +1,209 @@
#nullable enable
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Interop
{
/// <summary>
/// Tests for the pure netsh-output parser of the WiFi fallback path. netsh labels follow
/// the OS UI language — the parser must extract SSID/signal/radio/channel from the label
/// variants of the languages commonly seen in enrollments, and degrade gracefully (fields
/// absent, never throw) on unrecognized ones.
/// </summary>
public sealed class NetshWifiFallbackTests
{
[Fact]
public void Parse_english_output_extracts_all_fields()
{
const string output = @"
There is 1 interface on the system:
Name : Wi-Fi
Description : Intel(R) Wi-Fi 6 AX201 160MHz
GUID : 754b6c07-5ae9-49d5-ba6c-4fbd6e38bbf6
State : connected
SSID : contoso-corp
AP BSSID : 3c:37:12:53:c8:44
Band : 5 GHz
Channel : 116
Radio type : 802.11ax
Signal : 94%
";
var r = NetshWifiFallback.Parse(output);
Assert.NotNull(r);
Assert.Equal("contoso-corp", r!.Ssid);
Assert.Equal(94, r.SignalPercent);
Assert.Equal("802.11ax", r.RadioType);
Assert.Equal(116, r.Channel);
}
[Fact]
public void Parse_german_output_extracts_all_fields()
{
const string output = @"
Es ist 1 Schnittstelle auf dem System vorhanden:
Name : WLAN
Beschreibung : Intel(R) Wi-Fi 6 AX201 160MHz
Status : Verbunden
SSID : fabrikam-wlan
Kanal : 44
Funktyp : 802.11ac
Signal : 87%
";
var r = NetshWifiFallback.Parse(output);
Assert.NotNull(r);
Assert.Equal("fabrikam-wlan", r!.Ssid);
Assert.Equal(87, r.SignalPercent);
Assert.Equal("802.11ac", r.RadioType);
Assert.Equal(44, r.Channel);
}
[Fact]
public void Parse_spanish_output_extracts_signal_and_channel()
{
const string output = @"
Nombre : Wi-Fi
SSID : contoso-es
Canal : 6
Tipo de radio : 802.11n
Señal : 72%
";
var r = NetshWifiFallback.Parse(output);
Assert.NotNull(r);
Assert.Equal("contoso-es", r!.Ssid);
Assert.Equal(72, r.SignalPercent);
Assert.Equal("802.11n", r.RadioType);
Assert.Equal(6, r.Channel);
}
[Fact]
public void Parse_chinese_simplified_output_extracts_all_fields()
{
const string output = @"
名称 : WLAN
SSID : contoso-cn
信道 : 149
无线电类型 : 802.11ac
信号 : 68%
";
var r = NetshWifiFallback.Parse(output);
Assert.NotNull(r);
Assert.Equal("contoso-cn", r!.Ssid);
Assert.Equal(68, r.SignalPercent);
Assert.Equal("802.11ac", r.RadioType);
Assert.Equal(149, r.Channel);
}
[Fact]
public void Parse_korean_output_extracts_all_fields()
{
const string output = @"
이름 : Wi-Fi
SSID : contoso-kr
채널 : 36
라디오 종류 : 802.11ax
신호 : 91%
";
var r = NetshWifiFallback.Parse(output);
Assert.NotNull(r);
Assert.Equal("contoso-kr", r!.Ssid);
Assert.Equal(91, r.SignalPercent);
Assert.Equal("802.11ax", r.RadioType);
Assert.Equal(36, r.Channel);
}
[Fact]
public void Parse_signal_with_space_before_percent_is_parsed()
{
var r = NetshWifiFallback.Parse(" Signal : 81 %\r\n SSID : x\r\n");
Assert.NotNull(r);
Assert.Equal(81, r!.SignalPercent);
}
[Fact]
public void Parse_ap_bssid_does_not_overwrite_ssid()
{
const string output = @"
SSID : contoso-corp
AP BSSID : aa:bb:cc:dd:ee:ff
";
var r = NetshWifiFallback.Parse(output);
Assert.Equal("contoso-corp", r!.Ssid);
}
[Fact]
public void Parse_finnish_output_extracts_all_fields()
{
const string output = @"
Nimi : Wi-Fi
SSID : contoso-fi
Kanava : 100
Radiotyyppi : 802.11ax
Signaali : 79%
";
var r = NetshWifiFallback.Parse(output);
Assert.NotNull(r);
Assert.Equal("contoso-fi", r!.Ssid);
Assert.Equal(79, r.SignalPercent);
Assert.Equal("802.11ax", r.RadioType);
Assert.Equal(100, r.Channel);
}
[Fact]
public void Parse_indonesian_output_extracts_all_fields()
{
const string output = @"
Nama : Wi-Fi
SSID : contoso-id
Saluran : 11
Jenis radio : 802.11n
Sinyal : 64%
";
var r = NetshWifiFallback.Parse(output);
Assert.NotNull(r);
Assert.Equal("contoso-id", r!.Ssid);
Assert.Equal(64, r.SignalPercent);
Assert.Equal("802.11n", r.RadioType);
Assert.Equal(11, r.Channel);
}
[Fact]
public void Parse_unknown_language_without_ssid_returns_null()
{
// Greek is not in the enrollment-geography label set -> nothing to report.
var r = NetshWifiFallback.Parse(" Κατάσταση : συνδεδεμένο\r\n Σήμα : 90%\r\n");
Assert.Null(r);
}
[Fact]
public void Parse_unknown_language_still_yields_ssid()
{
// SSID is untranslated in all locales — partial result beats none.
var r = NetshWifiFallback.Parse(" SSID : contoso-gr\r\n Σήμα : 90%\r\n");
Assert.NotNull(r);
Assert.Equal("contoso-gr", r!.Ssid);
Assert.Null(r.SignalPercent);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("Der WLAN-AutoKonfig-Dienst (wlansvc) wird nicht ausgeführt.")]
public void Parse_empty_or_error_output_returns_null(string? output)
{
Assert.Null(NetshWifiFallback.Parse(output));
}
}
}
@@ -0,0 +1,98 @@
#nullable enable
using System.Text;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
using Xunit;
using static AutopilotMonitor.Agent.V2.Core.Monitoring.Interop.WlanNativeMethods;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Interop
{
/// <summary>
/// Tests for the native-WLAN-API WiFi reader. The WLAN state of the test machine is
/// unknown (CI runners/VMs have no WLAN service), so the live path only pins the
/// fail-soft contract; the pure helpers (SSID decode, PHY-type mapping) are pinned
/// exactly — their output feeds wifi_signal_info event payloads and must stay
/// consistent with the strings netsh historically reported.
/// </summary>
public sealed class WifiInfoReaderTests
{
[Fact]
public void TryGetCurrentConnection_never_throws_and_returns_sane_values()
{
var result = WifiInfoReader.TryGetCurrentConnection();
if (result != null)
{
Assert.True(result.SignalPercent.HasValue, "native API path must always carry signal quality");
Assert.InRange(result.SignalPercent.Value, 0, 100);
if (result.Channel.HasValue)
Assert.True(result.Channel.Value > 0, "channel must be positive when present");
}
}
// ----- DecodeSsid ---------------------------------------------------
[Fact]
public void DecodeSsid_decodes_utf8_bytes()
{
var raw = new byte[32];
var ssid = Encoding.UTF8.GetBytes("Contoso-Büro");
ssid.CopyTo(raw, 0);
Assert.Equal("Contoso-Büro", WifiInfoReader.DecodeSsid(raw, (uint)ssid.Length));
}
[Fact]
public void DecodeSsid_returns_null_for_empty_or_missing()
{
Assert.Null(WifiInfoReader.DecodeSsid(null, 5));
Assert.Null(WifiInfoReader.DecodeSsid(new byte[32], 0));
}
[Fact]
public void DecodeSsid_clamps_length_to_buffer_size()
{
var raw = Encoding.UTF8.GetBytes("abc");
// A corrupt native length beyond the buffer must not throw.
Assert.Equal("abc", WifiInfoReader.DecodeSsid(raw, 200));
}
[Fact]
public void DecodeSsid_uses_only_declared_length()
{
var raw = new byte[32];
Encoding.UTF8.GetBytes("HomeNet-Rest-Ignored").CopyTo(raw, 0);
Assert.Equal("HomeNet", WifiInfoReader.DecodeSsid(raw, 7));
}
// ----- PhyTypeToRadioType -------------------------------------------
// int parameters (cast inside): DOT11_PHY_TYPE is internal and may not appear in a
// public test method signature (CS0051).
[Theory]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_ofdm, "802.11a")]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_hrdsss, "802.11b")]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_erp, "802.11g")]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_ht, "802.11n")]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_vht, "802.11ac")]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_dmg, "802.11ad")]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_he, "802.11ax")]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_eht, "802.11be")]
public void PhyTypeToRadioType_maps_known_generations(int phyType, string expected)
{
Assert.Equal(expected, WifiInfoReader.PhyTypeToRadioType((DOT11_PHY_TYPE)phyType));
}
[Theory]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_unknown)]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_fhss)]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_dsss)]
[InlineData((int)DOT11_PHY_TYPE.dot11_phy_type_irbaseband)]
[InlineData(9999)]
public void PhyTypeToRadioType_returns_null_for_legacy_or_unknown(int phyType)
{
Assert.Null(WifiInfoReader.PhyTypeToRadioType((DOT11_PHY_TYPE)phyType));
}
}
}
@@ -226,6 +226,95 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Runtime
return archive.Entries.Select(e => e.FullName).ToArray();
}
private static string ZipEntryText(byte[] zipBytes, string entryName)
{
using var ms = new MemoryStream(zipBytes);
using var archive = new ZipArchive(ms, ZipArchiveMode.Read);
var entry = archive.GetEntry(entryName);
Assert.NotNull(entry);
using var reader = new StreamReader(entry!.Open(), Encoding.UTF8);
return reader.ReadToEnd();
}
// ── Nullable outcome (on-demand mid-session) — gate + sessioninfo semantics ──
/// <summary>
/// Sentinel subclass: throws before any HTTP so gate-pass-through is observable as a
/// non-null error result (a skip returns null BEFORE the archive is ever built).
/// </summary>
private sealed class ThrowOnBuildService : DiagnosticsPackageService
{
public ThrowOnBuildService(AgentConfiguration cfg, AgentLogger logger, BackendApiClient api)
: base(cfg, logger, api) { }
internal override byte[] BuildArchiveBytes(bool? enrollmentSucceeded) =>
throw new System.InvalidOperationException("gate-passed-sentinel");
}
private static DiagnosticsPackageService BuildGateProbe(AgentLogger logger, string mode)
{
var cfg = Cfg();
cfg.DiagnosticsUploadEnabled = true;
cfg.DiagnosticsUploadMode = mode;
var apiClient = new BackendApiClient(
httpClient: new System.Net.Http.HttpClient(),
baseUrl: "http://localhost",
manufacturer: string.Empty,
model: string.Empty,
serialNumber: string.Empty,
useBootstrapTokenAuth: false,
bootstrapToken: null,
agentVersion: "0.0.0",
logger: logger);
return new ThrowOnBuildService(cfg, logger, apiClient);
}
[Fact]
public async System.Threading.Tasks.Task CreateAndUploadAsync_null_outcome_passes_OnFailure_gate()
{
// On-demand mid-session (no outcome yet): the OnFailure gate must NOT skip —
// there is no known success to skip on. Reaching the archive build (sentinel
// throw → error result) proves the gate was passed; a skip would return null.
using var rig = new Rig();
var result = await BuildGateProbe(rig.Logger, "OnFailure")
.CreateAndUploadAsync(enrollmentSucceeded: null, fileNameSuffix: "server-requested");
Assert.NotNull(result);
Assert.Equal("gate-passed-sentinel", result!.ErrorCode);
}
[Fact]
public async System.Threading.Tasks.Task CreateAndUploadAsync_known_success_still_skipped_by_OnFailure_gate()
{
using var rig = new Rig();
var result = await BuildGateProbe(rig.Logger, "OnFailure")
.CreateAndUploadAsync(enrollmentSucceeded: true);
Assert.Null(result);
}
[Fact]
public async System.Threading.Tasks.Task CreateAndUploadAsync_null_outcome_still_blocked_by_mode_Off()
{
using var rig = new Rig();
var result = await BuildGateProbe(rig.Logger, "Off")
.CreateAndUploadAsync(enrollmentSucceeded: null);
Assert.Null(result);
}
[Theory]
[InlineData(true, "Succeeded")]
[InlineData(false, "Failed")]
[InlineData(null, "In Progress")]
public void BuildArchiveBytes_sessioninfo_reflects_nullable_outcome(bool? outcome, string expected)
{
using var rig = new Rig();
var bytes = rig.Build().BuildArchiveBytes(outcome);
var sessionInfo = ZipEntryText(bytes, "sessioninfo.txt");
Assert.Contains($"Enrollment Result: {expected}", sessionInfo);
}
// ── BuildBlobUploadUrl — destination-aware URL construction ─────────────────
[Theory]
@@ -112,6 +112,36 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
Assert.Equal(false, data["backfilled"]);
}
[Fact]
public void ErrorBackfillSummary_EmitsInfoLogEvent_AnnouncingTheReplayBlock()
{
_tracker.EmitErrorBackfillSummary(replayedCount: 6, droppedOverCap: 0);
var posted = _sink.Posted.Single(p => p.Kind == DecisionSignalKind.InformationalEvent);
Assert.Equal("modern_deployment_log", posted.Payload![SignalPayloadKeys.EventType]);
var data = Assert.IsAssignableFrom<IReadOnlyDictionary<string, object>>(posted.TypedPayload);
Assert.Equal(true, data["backfillSummary"]);
Assert.Equal(6, data["replayedCount"]);
Assert.Equal(0, data["droppedOverCap"]);
Assert.Equal(240, data["lookbackMinutes"]);
var message = (string)posted.Payload[SignalPayloadKeys.Message];
Assert.Contains("Replaying 6 error event(s)", message);
Assert.Contains("before the agent started", message);
Assert.DoesNotContain("cap", message);
}
[Fact]
public void ErrorBackfillSummary_MentionsDroppedRecords_WhenCapWasHit()
{
_tracker.EmitErrorBackfillSummary(replayedCount: 25, droppedOverCap: 7);
var posted = _sink.Posted.Single(p => p.Kind == DecisionSignalKind.InformationalEvent);
var message = (string)posted.Payload![SignalPayloadKeys.Message];
Assert.Contains("7 older record(s) beyond the 25-record cap", message);
}
[Fact]
public void BackfilledHarmlessId_StillRunsThroughRollupSuppression()
{
@@ -511,6 +511,88 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
$"network_bandwidth_estimate (@{bandwidthIdx}) must precede whiteglove_part1_complete (@{part1CompleteIdx}) so the straggler stays in Part 1.");
}
// ===================================================== WG Part-1 preprov diagnostics
// V1 parity (restored 2026-07-28): the seal exit uploads the diagnostics package as a
// pre-provisioning intermediate under the SAME mode gating as the terminal path — the
// seal counts as a success, so Always uploads, OnFailure skips (a Part-1 failure
// terminates via the regular Failed path, which uploads), Off does nothing.
[Theory]
[InlineData("Always", true, 1)]
[InlineData("OnFailure", true, 0)]
[InlineData("Off", true, 0)]
[InlineData("Always", false, 0)]
public void Handle_whiteglove_part1_diagnostics_upload_respects_mode(
string mode, bool enabled, int expected)
{
using var rig = new Rig();
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.WhiteGloveSealed }.Build();
var cfg = rig.BuildConfig(diagEnabled: enabled, diagMode: mode);
rig.Build(cfg).Handle(sender: null!,
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.WhiteGloveSealed));
Assert.Equal(expected, rig.DiagnosticsUploads);
}
[Fact]
public void Handle_whiteglove_part1_diagnostics_upload_uses_preprov_suffix_and_succeeded_flag()
{
using var rig = new Rig();
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.WhiteGloveSealed }.Build();
var cfg = rig.BuildConfig(diagEnabled: true, diagMode: "Always");
rig.Build(cfg).Handle(sender: null!,
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.WhiteGloveSealed));
Assert.Equal(1, rig.DiagnosticsUploads);
Assert.Equal("preprov", rig.LastDiagnosticsSuffix);
Assert.Equal(true, rig.LastDiagnosticsSucceededFlag);
}
[Fact]
public void Handle_whiteglove_part1_diagnostics_events_precede_part1_complete_marker()
{
// whiteglove_part1_complete is the authoritative Part-1/Part-2 boundary for the
// web split (computeWhiteGloveSplitSequence) — diagnostics_collecting/uploaded
// must sequence BEFORE it or they get mis-filed into the resumed Part-2 block.
using var rig = new Rig();
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.WhiteGloveSealed }.Build();
rig.DiagnosticsResult = new DiagnosticsUploadResult { BlobName = "diag-preprov.zip" };
var cfg = rig.BuildConfig(diagEnabled: true, diagMode: "Always");
rig.Build(cfg).Handle(sender: null!,
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.WhiteGloveSealed));
var emitted = rig.EmittedEventTypes.ToList();
var collectingIdx = emitted.IndexOf("diagnostics_collecting");
var uploadedIdx = emitted.IndexOf("diagnostics_uploaded");
var part1CompleteIdx = emitted.IndexOf(Constants.EventTypes.WhiteGlovePart1Complete);
Assert.True(collectingIdx >= 0, "diagnostics_collecting must have been emitted.");
Assert.True(uploadedIdx >= 0, "diagnostics_uploaded must have been emitted.");
Assert.True(part1CompleteIdx >= 0, "whiteglove_part1_complete must have been emitted.");
Assert.True(uploadedIdx < part1CompleteIdx,
$"diagnostics_uploaded (@{uploadedIdx}) must precede whiteglove_part1_complete (@{part1CompleteIdx}) so the upload stays in Part 1.");
}
[Fact]
public void Handle_whiteglove_part1_still_skips_cleanup_and_marker_when_diagnostics_enabled()
{
// The preprov upload must not change the Part-1 exit contract: no self-destruct,
// no enrollment-complete.marker, whiteglove.complete still written.
using var rig = new Rig();
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.WhiteGloveSealed }.Build();
var cfg = rig.BuildConfig(diagEnabled: true, diagMode: "Always");
rig.Build(cfg).Handle(sender: null!,
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.WhiteGloveSealed));
Assert.Equal(0, rig.CleanupService.Invocations);
Assert.False(File.Exists(Path.Combine(rig.StateDir, "enrollment-complete.marker")));
Assert.True(rig.SessionPersistence.IsWhiteGloveResume());
}
[Fact]
public void Handle_standalone_reboot_fires_when_reboot_enabled_and_no_self_destruct()
{
@@ -1,5 +1,7 @@
using System;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Termination;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
@@ -70,5 +72,82 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
Assert.Equal(0, bootToStart, 0);
Assert.Equal(10, uptime, 0);
}
// --- DescribeLateStart: outcome-calibrated severity + phrasing (fleet 2026-07-28) ---
private static void Describe(
EnrollmentTerminationOutcome outcome, string oobeState,
out EventSeverity severity, out string message, out string note)
=> ObservationCoverage.DescribeLateStart(
outcome, oobeState, bootToStartSeconds: 34 * 60, uptimeSeconds: 30,
out severity, out message, out note);
[Fact]
public void Failed_outcome_is_a_warning_with_the_hung_script_post_mortem_note()
{
Describe(EnrollmentTerminationOutcome.Failed, "in_progress",
out var severity, out var message, out var note);
Assert.Equal(EventSeverity.Warning, severity);
Assert.Contains("low coverage of the enrollment window", message);
Assert.Contains("post-mortem", note);
Assert.Contains("hung ahead of the bootstrap", note);
}
[Fact]
public void TimedOut_outcome_is_a_warning_like_a_failure()
{
Describe(EnrollmentTerminationOutcome.TimedOut, "in_progress",
out var severity, out _, out var note);
Assert.Equal(EventSeverity.Warning, severity);
Assert.Contains("post-mortem", note);
}
[Fact]
public void Succeeded_outcome_is_an_info_about_tail_only_coverage_without_alarm_framing()
{
// The 659c3a90 shape: healthy enrollment, bootstrap merely ran late in the ESP queue.
Describe(EnrollmentTerminationOutcome.Succeeded, "in_progress",
out var severity, out _, out var note);
Assert.Equal(EventSeverity.Info, severity);
Assert.Contains("succeeded", note);
Assert.Contains("IME log replay", note);
Assert.DoesNotContain("post-mortem", note);
Assert.DoesNotContain("hung", note);
}
[Fact]
public void Oobe_completed_at_start_phrases_a_pure_post_mortem()
{
Describe(EnrollmentTerminationOutcome.Failed, "completed",
out _, out var message, out _);
Assert.Contains("OOBE was already completed when the agent started", message);
}
[Fact]
public void Oobe_in_progress_at_start_phrases_the_live_tail()
{
Describe(EnrollmentTerminationOutcome.Failed, "in_progress",
out _, out var message, out _);
Assert.Contains("OOBE was still in progress at agent start", message);
Assert.Contains("final 30s live", message);
}
[Theory]
[InlineData("not_started")] // flips in after a mid-OOBE reboot — not interpretable
[InlineData("unavailable")] // WinRT contract absent / read failed
[InlineData("unknown_3")] // unexpected enum value
public void Uninterpretable_oobe_states_add_no_claim_to_the_message(string oobeState)
{
Describe(EnrollmentTerminationOutcome.Failed, oobeState,
out _, out var message, out _);
Assert.DoesNotContain("OOBE", message);
Assert.EndsWith("low coverage of the enrollment window.", message);
}
}
}
@@ -121,6 +121,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
/// </summary>
public string ImeMatchLogPath { get; set; }
/// <summary>
/// Optional path to write a gather-rule evaluation trace (for debugging).
/// If empty, no trace is written. Set from remote config (EnableGatherRuleDebugLog)
/// or the --gather-debug-log CLI override.
/// </summary>
public string GatherRuleDebugLogPath { get; set; }
/// <summary>
/// Path to directory with real IME log files for log replay.
/// When set, the agent replays these logs with time compression instead of reading live logs.
@@ -135,6 +135,24 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
}
}
// ---------------- GatherRuleDebugLog: same bool→path projection as ImeMatchLog
// above. `true` expands Constants.GatherRuleDebugLogPath, `false`
// nulls the runtime path unless a --gather-debug-log CLI override
// supplied a non-default path.
if (remoteConfig.EnableGatherRuleDebugLog)
{
agentConfig.GatherRuleDebugLogPath = Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath);
}
else
{
var defaultGatherDebugPath = Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath);
if (string.IsNullOrEmpty(agentConfig.GatherRuleDebugLogPath)
|| string.Equals(agentConfig.GatherRuleDebugLogPath, defaultGatherDebugPath, StringComparison.OrdinalIgnoreCase))
{
agentConfig.GatherRuleDebugLogPath = null;
}
}
// ---------------- Upload interval / batch — direct, V1 parity (no > 0 gate so a
// misconfigured tenant can in theory break uploads, but this
// surfaces the config bug instead of silently masking it).
@@ -327,6 +327,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
KeepLogFile = false,
EnableGeoLocation = true,
EnableImeMatchLog = false,
EnableGatherRuleDebugLog = false,
Collectors = CollectorConfiguration.CreateDefault(),
GatherRules = new System.Collections.Generic.List<GatherRule>(),
ImeLogPatterns = new System.Collections.Generic.List<ImeLogPattern>()
@@ -631,10 +631,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// the regular <see cref="ProcessRecord"/> pipeline with isBackfill=true, so they arrive
/// as modern_deployment_error events carrying backfilled=true plus the original
/// timeCreated; the event's own timestamp is emission time, so session StartedAt is not
/// dragged backwards. Newest-first, capped, watermark-deduped across restarts.
/// dragged backwards. The scan reads newest-first (the cap keeps the newest records),
/// but replay is buffered so an Info summary event announces the block BEFORE the errors
/// hit the timeline (customer feedback 2026-07-28: an unannounced wall of pre-agent
/// Error events reads like a live failure) and the records emit oldest-first, giving the
/// timeline sequence the original event-log chronology. Watermark-deduped across restarts.
/// </summary>
private void BackfillAutopilotErrors()
{
var buffered = new List<EventRecord>();
try
{
long watermark;
@@ -649,7 +654,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
ReverseDirection = true
};
int processed = 0, skippedWatermark = 0, droppedOverCap = 0;
int skippedWatermark = 0, droppedOverCap = 0;
long maxSeen = watermark;
using (var reader = new EventLogReader(query))
@@ -657,37 +662,48 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
EventRecord record;
while ((record = reader.ReadEvent()) != null)
{
using (record)
var recordId = record.RecordId ?? 0;
if (recordId > maxSeen) maxSeen = recordId;
if (recordId != 0 && recordId <= watermark)
{
var recordId = record.RecordId ?? 0;
if (recordId > maxSeen) maxSeen = recordId;
if (recordId != 0 && recordId <= watermark)
{
skippedWatermark++;
continue;
}
if (processed >= ErrorBackfillMaxRecords)
{
droppedOverCap++;
continue;
}
processed++;
ProcessRecord(record, "Autopilot", AutopilotChannel, isBackfill: true);
skippedWatermark++;
record.Dispose();
continue;
}
if (buffered.Count >= ErrorBackfillMaxRecords)
{
droppedOverCap++;
record.Dispose();
continue;
}
buffered.Add(record);
}
}
if (maxSeen > watermark)
AdvanceAutopilotErrorWatermark(maxSeen);
if (processed == 0 && droppedOverCap == 0)
if (buffered.Count == 0 && droppedOverCap == 0)
{
_logger.Debug($"ModernDeployment error backfill: nothing new (skippedByWatermark={skippedWatermark})");
}
else
_logger.Info($"ModernDeployment error backfill: forwarded {processed} record(s) " +
{
EmitErrorBackfillSummary(buffered.Count, droppedOverCap);
// Scan buffered newest-first — replay in reverse so the emitted sequence
// follows the original event-log chronology.
for (var i = buffered.Count - 1; i >= 0; i--)
{
ProcessRecord(buffered[i], "Autopilot", AutopilotChannel, isBackfill: true);
}
_logger.Info($"ModernDeployment error backfill: forwarded {buffered.Count} record(s) " +
$"(skippedByWatermark={skippedWatermark}, droppedOverCap={droppedOverCap})");
}
}
catch (EventLogNotFoundException)
{
@@ -701,6 +717,49 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
_logger.Warning($"ModernDeployment error backfill failed: {ex.Message}");
}
finally
{
foreach (var record in buffered)
{
try { record.Dispose(); } catch { }
}
}
}
/// <summary>
/// Announces the error-backfill block on the session timeline BEFORE the replayed
/// errors, so pre-agent Error events are readable as history rather than a live
/// failure. The web timeline additionally badges each replayed event via its
/// backfilled=true payload marker; this summary carries backfillSummary=true instead —
/// it is a live emission, not a replayed record.
/// </summary>
internal void EmitErrorBackfillSummary(int replayedCount, int droppedOverCap)
{
var message = $"[Autopilot] Replaying {replayedCount} error event(s) recorded in the Autopilot event log " +
$"before the agent started (lookback {ErrorBackfillLookbackMinutes / 60}h). " +
"These predate monitoring — if enrollment progressed afterwards, they were already resolved by a retry.";
if (droppedOverCap > 0)
{
message += $" {droppedOverCap} older record(s) beyond the {ErrorBackfillMaxRecords}-record cap were not replayed.";
}
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
EventType = Constants.EventTypes.ModernDeploymentLog,
Severity = EventSeverity.Info,
Source = "ModernDeploymentWatcher",
Phase = EnrollmentPhase.Unknown,
Message = message,
Data = new Dictionary<string, object>
{
{ "backfillSummary", true },
{ "replayedCount", replayedCount },
{ "droppedOverCap", droppedOverCap },
{ "lookbackMinutes", ErrorBackfillLookbackMinutes }
}
});
}
/// <summary>
@@ -0,0 +1,221 @@
using System;
using System.Diagnostics;
using AutopilotMonitor.Agent.V2.Core.Security;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Interop
{
/// <summary>
/// Fallback WiFi reader via <c>netsh wlan show interfaces</c> output parsing, used ONLY
/// when <see cref="WifiInfoReader"/> (native WLAN API) yields nothing on a WiFi-connected
/// device. Field evidence 2026-07-28: the WLAN API can return nothing in the OOBE/SYSTEM
/// context on builds where inbox netsh still reports the connection — until that is fully
/// root-caused (suspect: Wi-Fi location gating, 24H2+), netsh stays as the safety net.
/// <para>
/// netsh labels follow the OS UI language, so the parser matches the label variants of
/// every language implied by the enrollment geography seen to date (see the comment on
/// the key arrays for the country→language derivation). Unknown-language labels fall
/// through harmlessly — the affected field is simply absent, same as before. "SSID"
/// itself is untranslated in all known locales.
/// </para>
/// </summary>
internal static class NetshWifiFallback
{
private const int NetshTimeoutMs = 5000;
// Label variants per field, by OS UI language. Matched case-insensitively against the
// full text left of the colon, so acronym-suffixed keys ("AP BSSID") never collide.
// Language set derived from enrollment geography (get_geographic_metrics, 365d):
// US/GB/AU/CA/IE/SG/IN/ZA/NG/PH en · DE/AT/CH de · BE/NL nl · BE/FR/CH fr · DK da ·
// SE sv · PL pl · TR tr · CN zh-CN · HK zh-TW · ES/MX/PE/PY es · CZ cs · SK sk ·
// IT/CH it · ID id · MY ms · FI fi · LT lt · LV lv · HU hu · BR/PT pt · JP ja ·
// KR ko · TH th · VN vi · RO/MD ro · BG bg · BA/HR hr · AE/SA/EG/JO ar ·
// MN/GE/AZ usually en/ru images → ru.
private static readonly string[] SignalKeys =
{
"Signal", // en, de, fr, sv, da, nb, hr
"Señal", // es
"Sinal", // pt
"Segnale", // it
"Signaal", // nl
"Sygnał", // pl
"Signál", // cs, sk
"Sinyal", // tr, id
"Isyarat", // ms
"Signaali", // fi
"Signalas", // lt
"Signāls", // lv
"Jel", // hu
"Semnal", // ro
"Tín hiệu", // vi
"สัญญาณ", // th
"الإشارة", // ar
"إشارة", // ar (variant without article)
"Сигнал", // ru, bg
"シグナル", // ja
"信号", // zh-CN
"訊號", // zh-TW
"신호", // ko
};
private static readonly string[] RadioTypeKeys =
{
"Radio type", // en
"Funktyp", // de
"Type de radio", // fr
"Tipo de radio", // es
"Tipo de rádio", // pt
"Tipo di radio", // it
"Radiotype", // nl, da
"Radiotyp", // sv
"Typ radia", // pl
"Typ rádia", // cs, sk
"Radyo türü", // tr
"Jenis radio", // id, ms
"Radiotyyppi", // fi
"Radijo tipas", // lt
"Radio tips", // lv
"Rádió típusa", // hu
"Tip radio", // ro
"Vrsta radija", // hr, bs
"Loại radio", // vi
"ชนิดวิทยุ", // th
"نوع الراديو", // ar
"Тип радио", // ru
"Тип на радиото", // bg
"無線の種類", // ja
"无线电类型", // zh-CN
"無線電類型", // zh-TW
"라디오 종류", // ko
"무선 종류", // ko (variant)
};
private static readonly string[] ChannelKeys =
{
"Channel", // en
"Kanal", // de, sv, da, nb, tr, hr
"Canal", // fr, es, pt, ro
"Canale", // it
"Kanaal", // nl
"Kanał", // pl
"Kanál", // cs, sk
"Saluran", // id, ms
"Kanava", // fi
"Kanalas", // lt
"Kanāls", // lv
"Csatorna", // hu
"Kênh", // vi
"ช่องสัญญาณ", // th
"ช่อง", // th (variant)
"القناة", // ar
"قناة", // ar (variant without article)
"Канал", // ru, bg
"チャネル", // ja
"信道", // zh-CN
"通道", // zh-TW / zh-CN (variant)
"채널", // ko
};
/// <summary>
/// Runs <c>netsh wlan show interfaces</c> and parses its output. Returns a
/// <see cref="WifiConnectionInfo"/> with the fields that could be parsed, or null when
/// nothing was found (no WLAN, netsh failure, timeout, or unrecognized localized
/// labels). Never throws.
/// </summary>
internal static WifiConnectionInfo TryRead()
{
try
{
var psi = new ProcessStartInfo
{
FileName = SystemPaths.Netsh,
Arguments = "wlan show interfaces",
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true
};
string output;
using (var process = Process.Start(psi))
{
output = process.StandardOutput.ReadToEnd();
if (!process.WaitForExit(NetshTimeoutMs))
{
try { process.Kill(); } catch { }
return null;
}
}
return Parse(output);
}
catch
{
return null;
}
}
/// <summary>
/// Pure parser over the netsh output text. Internal for direct unit-testing via
/// InternalsVisibleTo (TryRead itself spawns a process and stays untested).
/// </summary>
internal static WifiConnectionInfo Parse(string output)
{
if (string.IsNullOrEmpty(output))
return null;
string ssid = null, radioType = null;
int? signal = null, channel = null;
foreach (var line in output.Split(new[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries))
{
var trimmed = line.Trim();
var colonIndex = trimmed.IndexOf(':');
if (colonIndex < 0) continue;
var key = trimmed.Substring(0, colonIndex).Trim();
var value = trimmed.Substring(colonIndex + 1).Trim();
if (key.Equals("SSID", StringComparison.OrdinalIgnoreCase))
{
ssid = value;
}
else if (MatchesAny(key, SignalKeys))
{
// Value formats seen: "94%", "94% " (trailing space), "94 %".
if (int.TryParse(value.TrimEnd('%', ' '), out var sig))
signal = sig;
}
else if (MatchesAny(key, RadioTypeKeys))
{
radioType = value;
}
else if (MatchesAny(key, ChannelKeys))
{
if (int.TryParse(value, out var ch))
channel = ch;
}
}
if (ssid == null && signal == null && radioType == null && channel == null)
return null;
return new WifiConnectionInfo
{
Ssid = ssid,
SignalPercent = signal,
RadioType = radioType,
Channel = channel,
};
}
private static bool MatchesAny(string key, string[] candidates)
{
foreach (var candidate in candidates)
{
if (key.Equals(candidate, StringComparison.OrdinalIgnoreCase))
return true;
}
return false;
}
}
}
@@ -0,0 +1,234 @@
using System;
using System.Runtime.InteropServices;
using System.Text;
using static AutopilotMonitor.Agent.V2.Core.Monitoring.Interop.WlanNativeMethods;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Interop
{
/// <summary>
/// Current WiFi connection details as read from the native WLAN API.
/// </summary>
public sealed class WifiConnectionInfo
{
/// <summary>SSID of the connected network; null when the AP broadcasts an empty SSID.</summary>
public string Ssid { get; internal set; }
/// <summary>
/// Signal quality 0–100 (same semantics as the netsh "Signal" percentage). Always set
/// on the native WLAN API path; null only when the netsh fallback could not parse it.
/// </summary>
public int? SignalPercent { get; internal set; }
/// <summary>Human-readable PHY generation ("802.11ax" …); null for exotic/unknown PHYs.</summary>
public string RadioType { get; internal set; }
/// <summary>Current 802.11 channel number; null when the channel query fails.</summary>
public int? Channel { get; internal set; }
}
/// <summary>
/// Fail-soft reader over <see cref="WlanNativeMethods"/>: returns the current WiFi
/// connection or null — never throws, never blocks on a child process. Machines without
/// the WLAN AutoConfig service (VMs, wired-only devices) simply return null.
/// </summary>
public static class WifiInfoReader
{
/// <summary>
/// Reads the current WiFi connection. When <paramref name="preferredInterfaceId"/> is
/// given (the adapter GUID from <c>NetworkInterface.Id</c>), that interface is queried
/// first; otherwise — or when it is not connected — the first connected WLAN interface
/// wins. Returns null when no interface has an active connection.
/// </summary>
public static WifiConnectionInfo TryGetCurrentConnection(Guid? preferredInterfaceId = null)
=> TryGetCurrentConnection(preferredInterfaceId, out _);
/// <summary>
/// Same as <see cref="TryGetCurrentConnection(Guid?)"/> but reports WHY the read
/// yielded nothing via <paramref name="diagnostics"/> (native return codes per step).
/// The WLAN API can fail in ways that are invisible from the outside (e.g. per-process
/// access gating on newer Windows builds) — callers log this at Debug so field issues
/// are explainable from the agent log instead of a silent missing event.
/// </summary>
public static WifiConnectionInfo TryGetCurrentConnection(Guid? preferredInterfaceId, out string diagnostics)
{
var diag = new StringBuilder();
var clientHandle = IntPtr.Zero;
try
{
var rc = WlanOpenHandle(WLAN_CLIENT_VERSION, IntPtr.Zero, out _, out clientHandle);
if (rc != ERROR_SUCCESS)
{
diagnostics = $"WlanOpenHandle rc={rc}";
return null;
}
var interfaceList = IntPtr.Zero;
try
{
rc = WlanEnumInterfaces(clientHandle, IntPtr.Zero, out interfaceList);
if (rc != ERROR_SUCCESS)
{
diagnostics = $"WlanEnumInterfaces rc={rc}";
return null;
}
var count = (uint)Marshal.ReadInt32(interfaceList);
if (count == 0)
{
diagnostics = "no WLAN interfaces";
return null;
}
var infoSize = Marshal.SizeOf(typeof(WLAN_INTERFACE_INFO));
// List header: DWORD dwNumberOfItems + DWORD dwIndex, then the info array.
var firstEntry = new IntPtr(interfaceList.ToInt64() + 8);
WifiConnectionInfo fallback = null;
for (var i = 0; i < count; i++)
{
var entryPtr = new IntPtr(firstEntry.ToInt64() + i * infoSize);
var info = (WLAN_INTERFACE_INFO)Marshal.PtrToStructure(entryPtr, typeof(WLAN_INTERFACE_INFO));
var connection = QueryCurrentConnection(clientHandle, info.InterfaceGuid, diag);
if (connection == null)
continue;
if (preferredInterfaceId.HasValue && info.InterfaceGuid == preferredInterfaceId.Value)
{
diagnostics = null;
return connection;
}
if (fallback == null)
fallback = connection;
}
diagnostics = fallback == null ? diag.ToString() : null;
return fallback;
}
finally
{
if (interfaceList != IntPtr.Zero)
WlanFreeMemory(interfaceList);
}
}
catch (Exception ex)
{
diagnostics = $"exception {ex.GetType().Name}: {ex.Message}";
return null;
}
finally
{
if (clientHandle != IntPtr.Zero)
WlanCloseHandle(clientHandle, IntPtr.Zero);
}
}
private static WifiConnectionInfo QueryCurrentConnection(IntPtr clientHandle, Guid interfaceGuid, StringBuilder diag)
{
var dataPtr = IntPtr.Zero;
try
{
var rc = WlanQueryInterface(clientHandle, ref interfaceGuid,
WLAN_INTF_OPCODE.wlan_intf_opcode_current_connection,
IntPtr.Zero, out _, out dataPtr, IntPtr.Zero);
if (rc != ERROR_SUCCESS)
{
diag.Append($"[{interfaceGuid:D}] query(current_connection) rc={rc}; ");
return null;
}
var attributes = (WLAN_CONNECTION_ATTRIBUTES)Marshal.PtrToStructure(
dataPtr, typeof(WLAN_CONNECTION_ATTRIBUTES));
if (attributes.isState != WLAN_INTERFACE_STATE.wlan_interface_state_connected)
{
diag.Append($"[{interfaceGuid:D}] state={attributes.isState}; ");
return null;
}
var assoc = attributes.wlanAssociationAttributes;
return new WifiConnectionInfo
{
Ssid = DecodeSsid(assoc.dot11Ssid.ucSSID, assoc.dot11Ssid.uSSIDLength),
SignalPercent = (int)Math.Min(assoc.wlanSignalQuality, 100),
RadioType = PhyTypeToRadioType(assoc.dot11PhyType),
Channel = QueryChannel(clientHandle, interfaceGuid),
};
}
finally
{
if (dataPtr != IntPtr.Zero)
WlanFreeMemory(dataPtr);
}
}
private static int? QueryChannel(IntPtr clientHandle, Guid interfaceGuid)
{
var dataPtr = IntPtr.Zero;
try
{
if (WlanQueryInterface(clientHandle, ref interfaceGuid,
WLAN_INTF_OPCODE.wlan_intf_opcode_channel_number,
IntPtr.Zero, out var dataSize, out dataPtr, IntPtr.Zero) != ERROR_SUCCESS)
return null;
if (dataSize < sizeof(uint) || dataPtr == IntPtr.Zero)
return null;
return Marshal.ReadInt32(dataPtr);
}
catch
{
return null;
}
finally
{
if (dataPtr != IntPtr.Zero)
WlanFreeMemory(dataPtr);
}
}
/// <summary>
/// Decodes a raw DOT11_SSID (UTF-8 by convention) into a string; null for empty SSIDs.
/// Internal for direct unit-testing via InternalsVisibleTo.
/// </summary>
internal static string DecodeSsid(byte[] raw, uint length)
{
if (raw == null || length == 0)
return null;
var byteCount = (int)Math.Min(length, (uint)raw.Length);
try
{
var ssid = Encoding.UTF8.GetString(raw, 0, byteCount);
return string.IsNullOrEmpty(ssid) ? null : ssid;
}
catch
{
return null;
}
}
/// <summary>
/// Maps a DOT11 PHY type to the same strings netsh historically reported as
/// "Radio type" — keeps event payloads consistent across the netsh→WLAN-API switch.
/// Internal for direct unit-testing via InternalsVisibleTo.
/// </summary>
internal static string PhyTypeToRadioType(DOT11_PHY_TYPE phyType)
{
switch (phyType)
{
case DOT11_PHY_TYPE.dot11_phy_type_ofdm: return "802.11a";
case DOT11_PHY_TYPE.dot11_phy_type_hrdsss: return "802.11b";
case DOT11_PHY_TYPE.dot11_phy_type_erp: return "802.11g";
case DOT11_PHY_TYPE.dot11_phy_type_ht: return "802.11n";
case DOT11_PHY_TYPE.dot11_phy_type_vht: return "802.11ac";
case DOT11_PHY_TYPE.dot11_phy_type_dmg: return "802.11ad";
case DOT11_PHY_TYPE.dot11_phy_type_he: return "802.11ax";
case DOT11_PHY_TYPE.dot11_phy_type_eht: return "802.11be";
default: return null;
}
}
}
}
@@ -0,0 +1,142 @@
using System;
using System.Runtime.InteropServices;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Interop
{
/// <summary>
/// Native WLAN API interop (<c>wlanapi.dll</c>) used to read the current WiFi connection
/// (SSID, signal quality, PHY type, channel) without spawning <c>netsh</c>.
/// <para>
/// The WLAN API is language-neutral — unlike parsing <c>netsh wlan show interfaces</c>
/// output, whose labels follow the OS UI language and silently drop fields on
/// non-English images. On machines without the WLAN AutoConfig service (VMs, servers,
/// wired-only devices) <c>WlanOpenHandle</c>/<c>WlanEnumInterfaces</c> fail with a
/// non-zero error code; callers treat that as "no WiFi" (see <see cref="WifiInfoReader"/>).
/// </para>
/// </summary>
internal static class WlanNativeMethods
{
internal const uint ERROR_SUCCESS = 0;
/// <summary>Client version 2 = Vista and later (the only relevant range).</summary>
internal const uint WLAN_CLIENT_VERSION = 2;
internal enum WLAN_INTF_OPCODE : uint
{
wlan_intf_opcode_current_connection = 7,
wlan_intf_opcode_channel_number = 8,
}
internal enum WLAN_INTERFACE_STATE
{
wlan_interface_state_not_ready = 0,
wlan_interface_state_connected = 1,
wlan_interface_state_ad_hoc_network_formed = 2,
wlan_interface_state_disconnecting = 3,
wlan_interface_state_disconnected = 4,
wlan_interface_state_associating = 5,
wlan_interface_state_discovering = 6,
wlan_interface_state_authenticating = 7,
}
internal enum DOT11_PHY_TYPE : uint
{
dot11_phy_type_unknown = 0,
dot11_phy_type_fhss = 1,
dot11_phy_type_dsss = 2,
dot11_phy_type_irbaseband = 3,
dot11_phy_type_ofdm = 4, // 802.11a
dot11_phy_type_hrdsss = 5, // 802.11b
dot11_phy_type_erp = 6, // 802.11g
dot11_phy_type_ht = 7, // 802.11n
dot11_phy_type_vht = 8, // 802.11ac
dot11_phy_type_dmg = 9, // 802.11ad
dot11_phy_type_he = 10, // 802.11ax
dot11_phy_type_eht = 11, // 802.11be
}
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
internal struct WLAN_INTERFACE_INFO
{
public Guid InterfaceGuid;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
public string strInterfaceDescription;
public WLAN_INTERFACE_STATE isState;
}
[StructLayout(LayoutKind.Sequential)]
internal struct DOT11_SSID
{
public uint uSSIDLength;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 32)]
public byte[] ucSSID;
}
[StructLayout(LayoutKind.Sequential)]
internal struct WLAN_ASSOCIATION_ATTRIBUTES
{
public DOT11_SSID dot11Ssid;
public int dot11BssType;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 6)]
public byte[] dot11Bssid;
public DOT11_PHY_TYPE dot11PhyType;
public uint uDot11PhyIndex;
public uint wlanSignalQuality; // 0..100, same semantics as netsh "Signal"
public uint ulRxRate;
public uint ulTxRate;
}
[StructLayout(LayoutKind.Sequential)]
internal struct WLAN_SECURITY_ATTRIBUTES
{
[MarshalAs(UnmanagedType.Bool)]
public bool bSecurityEnabled;
[MarshalAs(UnmanagedType.Bool)]
public bool bOneXEnabled;
public int dot11AuthAlgorithm;
public int dot11CipherAlgorithm;
}
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
internal struct WLAN_CONNECTION_ATTRIBUTES
{
public WLAN_INTERFACE_STATE isState;
public int wlanConnectionMode;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
public string strProfileName;
public WLAN_ASSOCIATION_ATTRIBUTES wlanAssociationAttributes;
public WLAN_SECURITY_ATTRIBUTES wlanSecurityAttributes;
}
[DllImport("wlanapi.dll")]
internal static extern uint WlanOpenHandle(
uint dwClientVersion,
IntPtr pReserved,
out uint pdwNegotiatedVersion,
out IntPtr phClientHandle);
[DllImport("wlanapi.dll")]
internal static extern uint WlanCloseHandle(
IntPtr hClientHandle,
IntPtr pReserved);
[DllImport("wlanapi.dll")]
internal static extern uint WlanEnumInterfaces(
IntPtr hClientHandle,
IntPtr pReserved,
out IntPtr ppInterfaceList);
[DllImport("wlanapi.dll")]
internal static extern uint WlanQueryInterface(
IntPtr hClientHandle,
[In] ref Guid pInterfaceGuid,
WLAN_INTF_OPCODE OpCode,
IntPtr pReserved,
out uint pdwDataSize,
out IntPtr ppData,
IntPtr pWlanOpcodeValueType);
[DllImport("wlanapi.dll")]
internal static extern void WlanFreeMemory(IntPtr pMemory);
}
}
@@ -218,15 +218,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
/// This method is non-fatal: all exceptions are caught and logged.
/// </summary>
/// <param name="enrollmentSucceeded">
/// True for a successful enrollment (affects OnFailure mode check and sessioninfo.txt content).
/// Pass true for WhiteGlove pre-provisioning (it succeeded up to this point).
/// True for a successful enrollment, false for a failed one (drives the OnFailure mode
/// check and sessioninfo.txt content). Pass true for WhiteGlove pre-provisioning (it
/// succeeded up to this point). Pass <b>null</b> when the session has no outcome yet
/// (on-demand server-requested collection mid-enrollment): the OnFailure gate does not
/// apply — there is no success to skip on — and sessioninfo.txt records "In Progress".
/// </param>
/// <param name="fileNameSuffix">
/// Optional suffix inserted before the .zip extension.
/// Example: "preprov" → AgentDiagnostics-{sessionId}-{timestamp}-preprov.zip
/// Null (default) → AgentDiagnostics-{sessionId}-{timestamp}.zip
/// </param>
public virtual async Task<DiagnosticsUploadResult> CreateAndUploadAsync(bool enrollmentSucceeded, string fileNameSuffix = null)
public virtual async Task<DiagnosticsUploadResult> CreateAndUploadAsync(bool? enrollmentSucceeded, string fileNameSuffix = null)
{
try
{
@@ -244,13 +247,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
return null;
}
if (string.Equals(mode, "OnFailure", StringComparison.OrdinalIgnoreCase) && enrollmentSucceeded)
// Null outcome (on-demand mid-session) passes through: the OnFailure gate only
// skips a KNOWN success — an in-flight session has nothing to skip on.
if (string.Equals(mode, "OnFailure", StringComparison.OrdinalIgnoreCase) && enrollmentSucceeded == true)
{
_logger.Info("Diagnostics upload skipped: enrollment succeeded and mode=OnFailure");
return null;
}
_logger.Info($"Creating diagnostics package (mode={mode}, enrollmentSucceeded={enrollmentSucceeded}{(fileNameSuffix != null ? $", suffix={fileNameSuffix}" : "")})...");
_logger.Info($"Creating diagnostics package (mode={mode}, enrollmentSucceeded={(enrollmentSucceeded.HasValue ? enrollmentSucceeded.ToString() : "n/a")}{(fileNameSuffix != null ? $", suffix={fileNameSuffix}" : "")})...");
// Build ZIP in memory
var timestamp = DateTime.UtcNow.ToString("yyyyMMddHHmmss");
@@ -343,7 +348,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
// Builds the diagnostics ZIP body in-memory. Extracted from CreateAndUploadAsync so
// tests can assert archive contents without going through the upload path.
internal virtual byte[] BuildArchiveBytes(bool enrollmentSucceeded)
internal virtual byte[] BuildArchiveBytes(bool? enrollmentSucceeded)
{
var tracker = new BudgetTracker(Budget);
using (var ms = new MemoryStream())
@@ -414,14 +419,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
}
}
private void AddSessionInfo(ZipArchive archive, bool enrollmentSucceeded)
private void AddSessionInfo(ZipArchive archive, bool? enrollmentSucceeded)
{
var sb = new StringBuilder();
sb.AppendLine($"Session ID: {_configuration.SessionId}");
sb.AppendLine($"Tenant ID: {_configuration.TenantId}");
sb.AppendLine($"Device Name: {Environment.MachineName}");
sb.AppendLine($"Timestamp: {DateTime.UtcNow:O}");
sb.AppendLine($"Enrollment Result: {(enrollmentSucceeded ? "Succeeded" : "Failed")}");
sb.AppendLine($"Enrollment Result: {(enrollmentSucceeded == null ? "In Progress" : enrollmentSucceeded.Value ? "Succeeded" : "Failed")}");
// Hardware info via existing DeviceInfoProvider (WMI)
sb.AppendLine($"Manufacturer: {DeviceInfoProvider.GetManufacturer()}");
@@ -815,7 +815,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.DeviceInfo
/// Collects active network interface details (name, type, link speed, MAC) using
/// System.Net.NetworkInformation — instant, no process spawn.
/// If the active interface is WiFi, fires off a separate async task to collect
/// WiFi signal info via netsh (fire-and-forget, never blocks).
/// WiFi signal info via the native WLAN API (fire-and-forget, never blocks).
/// </summary>
private void CollectActiveNetworkInterfaceInfo()
{
@@ -858,7 +858,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.DeviceInfo
// Fire-and-forget WiFi signal collection — separate event, never blocks
if (isWifi)
{
Task.Run(() => CollectWiFiSignalInfo());
Guid? nicGuid = Guid.TryParse(activeNic.Id, out var parsedGuid) ? parsedGuid : (Guid?)null;
Task.Run(() => CollectWiFiSignalInfo(nicGuid));
}
}
catch (Exception ex)
@@ -899,78 +900,41 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.DeviceInfo
}
/// <summary>
/// Collects WiFi signal info via netsh wlan show interfaces and emits a separate
/// wifi_signal_info event. Designed to run as fire-and-forget via Task.Run —
/// never blocks other collection. VMs without WiFi service simply get no event.
/// Collects WiFi signal info via the native WLAN API (<see cref="WifiInfoReader"/>,
/// language-neutral, no process spawn) and emits a separate wifi_signal_info event.
/// Designed to run as fire-and-forget via Task.Run — never blocks other collection.
/// VMs without WiFi service simply get no event.
/// </summary>
private void CollectWiFiSignalInfo()
private void CollectWiFiSignalInfo(Guid? interfaceGuid)
{
try
{
var psi = new ProcessStartInfo
var wifi = WifiInfoReader.TryGetCurrentConnection(interfaceGuid, out var wifiDiag);
if (wifi == null)
{
FileName = SystemPaths.Netsh,
Arguments = "wlan show interfaces",
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true
};
string output;
using (var process = Process.Start(psi))
{
output = process.StandardOutput.ReadToEnd();
process.WaitForExit(5000);
_logger.Debug($"EnrollmentTracker: native WLAN API returned no WiFi info ({wifiDiag ?? "no connected WLAN interface"}) — trying netsh fallback");
wifi = NetshWifiFallback.TryRead();
if (wifi == null)
return;
}
if (string.IsNullOrEmpty(output))
return;
var data = new Dictionary<string, object>();
if (wifi.Ssid != null)
data["wifiSsid"] = wifi.Ssid;
if (wifi.SignalPercent.HasValue)
data["wifiSignalPercent"] = wifi.SignalPercent.Value;
if (wifi.RadioType != null)
data["wifiRadioType"] = wifi.RadioType;
if (wifi.Channel.HasValue)
data["wifiChannel"] = wifi.Channel.Value;
foreach (var line in output.Split(new[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries))
{
var trimmed = line.Trim();
var colonIndex = trimmed.IndexOf(':');
if (colonIndex < 0) continue;
var message = wifi.Ssid != null ? $"WiFi: {wifi.Ssid}" : "WiFi signal info";
if (wifi.SignalPercent.HasValue)
message += $", Signal: {wifi.SignalPercent.Value}%";
if (wifi.RadioType != null)
message += $" ({wifi.RadioType})";
var key = trimmed.Substring(0, colonIndex).Trim();
var value = trimmed.Substring(colonIndex + 1).Trim();
if (key.Equals("SSID", StringComparison.OrdinalIgnoreCase) &&
!key.Equals("BSSID", StringComparison.OrdinalIgnoreCase))
{
data["wifiSsid"] = value;
}
else if (key.Equals("Signal", StringComparison.OrdinalIgnoreCase))
{
if (int.TryParse(value.TrimEnd('%'), out var signal))
data["wifiSignalPercent"] = signal;
}
else if (key.Equals("Radio type", StringComparison.OrdinalIgnoreCase))
{
data["wifiRadioType"] = value;
}
else if (key.Equals("Channel", StringComparison.OrdinalIgnoreCase))
{
if (int.TryParse(value, out var channel))
data["wifiChannel"] = channel;
}
}
if (data.Count > 0)
{
var message = data.ContainsKey("wifiSsid")
? $"WiFi: {data["wifiSsid"]}"
: "WiFi signal info";
if (data.ContainsKey("wifiSignalPercent"))
message += $", Signal: {data["wifiSignalPercent"]}%";
if (data.ContainsKey("wifiRadioType"))
message += $" ({data["wifiRadioType"]})";
EmitDeviceInfoEvent(Constants.EventTypes.WifiSignalInfo, message, data);
}
EmitDeviceInfoEvent(Constants.EventTypes.WifiSignalInfo, message, data);
}
catch (Exception ex)
{
@@ -25,14 +25,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
{
var filePath = rule.Target;
if (string.IsNullOrEmpty(filePath))
{
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser, "empty target — nothing to parse");
return null;
}
// Expand custom tokens (%LOGGED_ON_USER_PROFILE%) and standard environment variables
var userProfilePath = UserProfileResolver.ContainsUserProfileToken(filePath)
? UserProfileResolver.GetLoggedOnUserProfilePath() : null;
filePath = UserProfileResolver.ExpandCustomTokens(filePath);
if (filePath == null)
{
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
"target contains %LOGGED_ON_USER_PROFILE% but no user is logged on — skipped this run");
return null; // Token present but no user logged on — skip silently
}
// Apply IME log path override if set. The override comes from the local
// --ime-log-path flag only (never from remote config), so the operator
@@ -66,6 +73,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
string.IsNullOrEmpty(patternStr))
{
context.Logger.Warning($"LogParser rule {rule.RuleId} has no 'pattern' parameter");
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageError,
"missing 'pattern' parameter — rule can never match");
return null;
}
@@ -93,6 +102,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
catch (Exception ex)
{
context.Logger.Warning($"LogParser rule {rule.RuleId} has invalid regex: {ex.Message}");
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageError,
$"invalid regex pattern — rule can never match: {ex.Message}");
return null;
}
@@ -101,6 +112,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
if (resolvedPaths.Count == 0)
{
context.Logger.Debug($"LogParser rule {rule.RuleId}: no files found for: {filePath}");
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"no files matched target {filePath}" + (HasWildcard(Path.GetFileName(filePath)) ? " (wildcard)" : "") + " — nothing to parse");
return null;
}
@@ -144,7 +157,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
try
{
// Return matched files sorted by last write time (newest first), capped at 20
return Directory.GetFiles(directory, fileNamePart)
var allMatches = Directory.GetFiles(directory, fileNamePart);
if (allMatches.Length > 20)
context.DebugLog(ruleId, GatherRuleDebugLog.StageLogParser,
$"wildcard matched {allMatches.Length} files — capped to the 20 newest");
return allMatches
.OrderByDescending(f => new FileInfo(f).LastWriteTimeUtc)
.Take(20)
.ToList();
@@ -152,6 +169,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
catch (Exception ex)
{
context.Logger.Warning($"LogParser rule {ruleId}: wildcard expansion failed for {filePath}: {ex.Message}");
context.DebugLog(ruleId, GatherRuleDebugLog.StageError, $"wildcard expansion failed for {filePath}: {ex.Message}");
return new List<string>();
}
}
@@ -168,10 +186,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
// Nothing new to read
if (startPosition >= fileInfo.Length)
{
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"{Path.GetFileName(filePath)}: no new content (position {startPosition} >= length {fileInfo.Length}) — trackPosition=true, nothing re-read");
return;
}
int matchCount = 0;
int linesRead = 0;
int parseFailCount = 0;
int timeoutCount = 0;
long endPosition = startPosition;
using (var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
@@ -195,6 +219,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
}
catch (RegexMatchTimeoutException)
{
timeoutCount++;
continue;
}
if (!match.Success)
@@ -239,7 +264,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
// CMTrace mode: parse line as CMTrace format, match regex against message
CmTraceLogEntry entry;
if (!CmTraceLogParser.TryParseLine(line, out entry))
{
parseFailCount++;
continue;
}
Match match;
try
@@ -248,6 +276,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
}
catch (RegexMatchTimeoutException)
{
timeoutCount++;
continue;
}
if (!match.Success)
@@ -301,10 +330,24 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
if (matchCount > 0)
context.Logger.Debug($"LogParser rule {rule.RuleId}: {matchCount} matches from {linesRead} lines in {Path.GetFileName(filePath)}");
// Per-file outcome — always written so a "matched 0" run is visible too.
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"{Path.GetFileName(filePath)}: read {linesRead} lines from position {startPosition}->{endPosition}, " +
$"matched {matchCount}, parseFailures={parseFailCount}, regexTimeouts={timeoutCount}, mode={(isTextMode ? "text" : "cmtrace")}");
if (!isTextMode && linesRead > 0 && parseFailCount == linesRead)
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
"every line failed CMTrace parsing — if this is a plain-text log, set parameter format=text");
if (linesRead == maxLines)
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"stopped at maxLines={maxLines} — remaining content is deferred to the next run");
}
catch (Exception ex)
{
context.Logger.Warning($"LogParser rule {rule.RuleId} failed reading {filePath}: {ex.Message}");
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageError, $"failed reading {filePath}: {ex}");
}
}
@@ -113,7 +113,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
// Suppress the "not found" event when the rule only wants a positive signal.
if (emitOnlyIfExists)
{
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageCollector,
$"registry key/value not found: {path} — emitOnlyIfExists=true, empty result");
return new Dictionary<string, object>(); // empty → executor emits nothing
}
data["exists"] = false;
data["path"] = path;
@@ -22,13 +22,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
public LogFilePositionTracker FilePositionTracker { get; }
public bool UnrestrictedMode { get; set; }
/// <summary>
/// Optional gather-rule debug trace writer (EnableGatherRuleDebugLog). Null when
/// tracing is disabled — use <see cref="DebugLog"/> for null-safe writes.
/// </summary>
public GatherRuleDebugLog DebugWriter { get; }
public GatherRuleContext(
AgentLogger logger,
string sessionId,
string tenantId,
Action<EnrollmentEvent> onEventCollected,
string imeLogPathOverride,
LogFilePositionTracker filePositionTracker)
LogFilePositionTracker filePositionTracker,
GatherRuleDebugLog debugLog = null)
{
Logger = logger ?? throw new ArgumentNullException(nameof(logger));
SessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
@@ -36,8 +43,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
OnEventCollected = onEventCollected ?? throw new ArgumentNullException(nameof(onEventCollected));
ImeLogPathOverride = imeLogPathOverride;
FilePositionTracker = filePositionTracker ?? throw new ArgumentNullException(nameof(filePositionTracker));
DebugWriter = debugLog;
}
/// <summary>
/// Null-safe write to the gather-rule debug trace. No-op when tracing is disabled.
/// </summary>
public void DebugLog(string ruleId, string stage, string message) => DebugWriter?.Write(ruleId, stage, message);
/// <summary>
/// Emits a security_warning event and returns an empty result dictionary.
/// Called when a gather rule targets a path/query not on the allowlist.
@@ -45,6 +58,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
public Dictionary<string, object> EmitSecurityWarning(GatherRule rule, string collectorType, string target)
{
Logger.Warning($"SECURITY: {collectorType} path blocked by guard: {target} (Rule: {rule.RuleId})");
DebugLog(rule.RuleId, GatherRuleDebugLog.StageGuard,
$"{collectorType} target blocked by allowlist guard: {target} — security_warning emitted, rule produced no data");
var data = new Dictionary<string, object>
{
@@ -0,0 +1,115 @@
using System;
using System.IO;
using AutopilotMonitor.Agent.V2.Core.Logging;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
/// <summary>
/// Append-only trace file for gather-rule evaluation, enabled per tenant via
/// EnableGatherRuleDebugLog (or --gather-debug-log). Explains every evaluation
/// decision — including the outcomes that produce no event — so customers can
/// diagnose rules that never show up in the timeline. Every write failure is
/// swallowed: tracing must never break gathering.
/// </summary>
public sealed class GatherRuleDebugLog
{
public const string StageConfig = "config";
public const string StageTrigger = "trigger";
public const string StageScope = "scope";
public const string StageExec = "exec";
public const string StageCollector = "collector";
public const string StageGuard = "guard";
public const string StageEmit = "emit";
public const string StageSuppress = "suppress";
public const string StageLogParser = "logparser";
public const string StageError = "error";
private const long DefaultMaxBytes = 10 * 1024 * 1024;
private readonly string _path;
private readonly long _maxBytes;
private readonly Action<string> _echo;
private readonly object _lock = new object();
private bool _headerWritten;
public GatherRuleDebugLog(string path, AgentLogger logger, Action<string> echo = null, long maxBytes = DefaultMaxBytes)
{
if (string.IsNullOrEmpty(path)) throw new ArgumentNullException(nameof(path));
_path = path;
_maxBytes = maxBytes;
_echo = echo;
logger?.Info($"GatherRuleDebugLog: enabled -> {path}");
}
/// <summary>
/// Writes one trace line: <c>{utc} | {ruleId} | {stage} | {message}</c>.
/// Newlines in <paramref name="message"/> are flattened except for the error
/// stage, where exception stack traces stay multi-line.
/// </summary>
public void Write(string ruleId, string stage, string message)
{
try
{
if (stage != StageError && message != null && message.IndexOf('\n') >= 0)
message = message.Replace("\r", "").Replace('\n', ' ');
var line = $"{DateTime.UtcNow:yyyy-MM-ddTHH:mm:ss.fffZ} | {ruleId ?? "-"} | {stage} | {message}";
lock (_lock)
{
EnsureFileReadyLocked();
File.AppendAllText(_path, line + Environment.NewLine);
}
_echo?.Invoke(line);
}
catch { }
}
/// <summary>
/// One-shot helper for callers that have no executor (e.g. the flag is on but the
/// backend delivered zero gather rules) — the customer's trace file should explain
/// that too instead of silently not existing.
/// </summary>
public static void WriteStandalone(string path, string message, AgentLogger logger)
{
if (string.IsNullOrEmpty(path)) return;
try
{
new GatherRuleDebugLog(path, logger).Write(null, StageConfig, message);
}
catch { }
}
// Lazy on first write: create the directory and emit a header line. Before every
// append, rotate when the file exceeds maxBytes — keeping exactly one .old
// generation (worst case ~2x maxBytes on disk).
private void EnsureFileReadyLocked()
{
if (!_headerWritten)
{
var dir = Path.GetDirectoryName(_path);
if (!string.IsNullOrEmpty(dir))
Directory.CreateDirectory(dir);
}
var info = new FileInfo(_path);
if (info.Exists && info.Length > _maxBytes)
{
var oldPath = Path.ChangeExtension(_path, null) + ".old.log";
if (File.Exists(oldPath))
File.Delete(oldPath);
File.Move(_path, oldPath);
File.AppendAllText(_path,
$"{DateTime.UtcNow:yyyy-MM-ddTHH:mm:ss.fffZ} | - | {StageConfig} | -- rotated (previous file exceeded {_maxBytes} bytes) --" + Environment.NewLine);
}
if (!_headerWritten)
{
File.AppendAllText(_path,
$"{DateTime.UtcNow:yyyy-MM-ddTHH:mm:ss.fffZ} | - | {StageConfig} | -- gather rule debug log started --" + Environment.NewLine);
_headerWritten = true;
}
}
}
}
@@ -22,6 +22,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
public class GatherRuleExecutor : IDisposable
{
private readonly AgentLogger _logger;
private readonly GatherRuleDebugLog _debug; // null unless EnableGatherRuleDebugLog / --gather-debug-log
private readonly GatherRuleContext _context;
private readonly Dictionary<string, IGatherRuleCollector> _collectors;
@@ -61,7 +62,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
}
public GatherRuleExecutor(string sessionId, string tenantId, Action<EnrollmentEvent> onEventCollected,
AgentLogger logger, string imeLogPathOverride = null)
AgentLogger logger, string imeLogPathOverride = null, string debugLogPath = null,
Action<string> debugEcho = null)
{
if (sessionId == null) throw new ArgumentNullException(nameof(sessionId));
if (tenantId == null) throw new ArgumentNullException(nameof(tenantId));
@@ -69,9 +71,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
if (logger == null) throw new ArgumentNullException(nameof(logger));
_logger = logger;
_debug = string.IsNullOrEmpty(debugLogPath) ? null : new GatherRuleDebugLog(debugLogPath, logger, debugEcho);
var filePositionTracker = new LogFilePositionTracker();
_context = new GatherRuleContext(logger, sessionId, tenantId, onEventCollected, imeLogPathOverride, filePositionTracker);
_context = new GatherRuleContext(logger, sessionId, tenantId, onEventCollected, imeLogPathOverride, filePositionTracker, _debug);
// Register all collector strategies
var collectorList = new IGatherRuleCollector[]
@@ -96,6 +99,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
_collectors["command"] = _collectors["command_allowlisted"];
}
/// <summary>
/// Null-safe write to the gather-rule debug trace. Never call while holding
/// <see cref="_scopeLock"/> — capture the message under the lock, write after release.
/// </summary>
private void DebugLog(string ruleId, string stage, string message) => _debug?.Write(ruleId, stage, message);
/// <summary>
/// Updates the active rules and starts/stops execution accordingly
/// </summary>
@@ -111,11 +120,43 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
_activeRules = rules.Where(r => r.Enabled).ToList();
// Registration summary for the debug trace: one line per delivered rule so the
// customer can see exactly what the agent received and how it will be triggered.
if (_debug != null)
{
foreach (var rule in rules)
{
if (!rule.Enabled)
{
DebugLog(rule.RuleId, GatherRuleDebugLog.StageConfig, "rule disabled — will never run");
continue;
}
var scope = rule.ActivePhases != null && rule.ActivePhases.Count > 0
? $"activePhases=[{string.Join(",", rule.ActivePhases)}]"
: !string.IsNullOrEmpty(rule.ActiveFromPhase)
? $"activeFromPhase={rule.ActiveFromPhase}"
: "unscoped";
DebugLog(rule.RuleId, GatherRuleDebugLog.StageConfig,
$"registered: trigger={rule.Trigger}" +
(string.IsNullOrEmpty(rule.TriggerPhase) ? "" : $" triggerPhase={rule.TriggerPhase}") +
(string.IsNullOrEmpty(rule.TriggerEventType) ? "" : $" triggerEventType={rule.TriggerEventType}") +
$", collector={rule.CollectorType}, target={rule.Target}, {scope}" +
$", emitMode={(string.IsNullOrEmpty(rule.EmitMode) ? "always" : rule.EmitMode)}" +
(rule.IntervalSeconds.HasValue ? $", interval={rule.IntervalSeconds}s" : ""));
if (rule.Trigger == "interval" && !rule.IntervalSeconds.HasValue)
DebugLog(rule.RuleId, GatherRuleDebugLog.StageError,
"trigger=interval but intervalSeconds is missing — rule will NEVER run");
}
}
// Execute startup rules — track completion via CountdownEvent so callers can wait.
// Scoped startup rules that are not yet in scope are deferred: they run once from
// OnPhaseChanged when their scope activates (same _startupRulesExecuted dedup — a
// rule runs either here or there, never both). Unscoped rules behave as before.
List<GatherRule> pendingStartup;
var deferredTraces = _debug != null ? new List<KeyValuePair<string, string>>() : null;
lock (_scopeLock)
{
// A config refresh may deliver new from-phase rules after their phase was
@@ -124,7 +165,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
pendingStartup = _activeRules
.Where(r => r.Trigger == "startup" && !_startupRulesExecuted.Contains(r.RuleId))
.Where(r => IsRuleInScopeLocked(r))
.Where(r =>
{
string reason;
var inScope = IsRuleInScopeLocked(r, out reason);
if (!inScope && deferredTraces != null)
deferredTraces.Add(new KeyValuePair<string, string>(r.RuleId,
$"startup rule deferred — {reason}; runs once when its scope activates"));
return inScope;
})
.ToList();
foreach (var rule in pendingStartup)
@@ -133,6 +182,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
}
}
if (deferredTraces != null)
{
foreach (var trace in deferredTraces)
DebugLog(trace.Key, GatherRuleDebugLog.StageScope, trace.Value);
}
if (pendingStartup.Count > 0)
{
_startupRulesLatch?.Dispose();
@@ -158,8 +213,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
var timer = new Timer(
_ =>
{
if (!IsRuleInScope(intervalRule))
string skipReason;
if (!IsRuleInScope(intervalRule, out skipReason))
{
DebugLog(intervalRule.RuleId, GatherRuleDebugLog.StageScope, $"interval tick skipped: {skipReason}");
return;
}
ExecuteRule(intervalRule);
},
null,
@@ -168,6 +227,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
);
_intervalTimers[intervalRule.RuleId] = timer;
_logger.Info($" Interval rule {intervalRule.RuleId} scheduled every {intervalRule.IntervalSeconds}s");
DebugLog(intervalRule.RuleId, GatherRuleDebugLog.StageTrigger,
$"interval timer scheduled every {intervalRule.IntervalSeconds}s (first run after one full interval — no immediate execution)");
}
_logger.Info($"GatherRuleExecutor: {_activeRules.Count(r => r.Trigger == "startup")} startup, " +
@@ -187,6 +248,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// _currentPhase moves below. A transition INTO Failed still fires the exit rules of the
// phase that failed — that snapshot is the whole point at a failure boundary.
List<GatherRule> exitRules = null;
var exitTraces = _debug != null ? new List<KeyValuePair<string, string>>() : null;
lock (_scopeLock)
{
var previousPhase = _currentPhase;
@@ -204,12 +266,22 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// Scope gate before the dedup add — an out-of-scope pass must not consume
// the once-per-(rule, phase) slot. IsRuleInScopeLocked still sees the old phase.
if (!IsRuleInScopeLocked(rule)) continue;
string scopeReason;
if (!IsRuleInScopeLocked(rule, out scopeReason))
{
if (exitTraces != null)
exitTraces.Add(new KeyValuePair<string, string>(rule.RuleId,
$"phase_exit of {previousPhaseName} skipped: {scopeReason}"));
continue;
}
// "exit:" prefix keeps this key space disjoint from the phase_change keys.
if (!_phaseRulesExecuted.Add($"exit:{rule.RuleId}|{previousPhaseName}"))
{
_logger.Debug($"Phase-exit rule {rule.RuleId} already executed for exit of {previousPhaseName}, skipping");
if (exitTraces != null)
exitTraces.Add(new KeyValuePair<string, string>(rule.RuleId,
$"phase_exit already executed for exit of {previousPhaseName} — once per (rule, phase)"));
continue;
}
@@ -219,6 +291,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
}
}
if (exitTraces != null)
{
foreach (var trace in exitTraces)
DebugLog(trace.Key, GatherRuleDebugLog.StageScope, trace.Value);
}
if (exitRules != null)
{
foreach (var rule in exitRules)
@@ -252,6 +330,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
var deferredRule = rule;
_logger.Info($"Phase {phaseName} activated deferred startup rule {deferredRule.RuleId}");
DebugLog(deferredRule.RuleId, GatherRuleDebugLog.StageTrigger,
$"phase {phaseName} activated deferred startup rule");
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(deferredRule));
}
@@ -262,18 +342,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
// Scope gate BEFORE the dedup add — an out-of-scope pass must not consume
// the once-per-(rule, phase) execution slot.
if (!IsRuleInScope(rule))
string scopeReason;
if (!IsRuleInScope(rule, out scopeReason))
{
DebugLog(rule.RuleId, GatherRuleDebugLog.StageScope,
$"phase_change to {phaseName} skipped: {scopeReason}");
continue;
}
// Deduplicate: only fire once per (ruleId, phase) combination
var deduplicationKey = $"{rule.RuleId}|{phaseName}";
if (!_phaseRulesExecuted.Add(deduplicationKey))
{
_logger.Debug($"Phase rule {rule.RuleId} already executed for phase {phaseName}, skipping");
DebugLog(rule.RuleId, GatherRuleDebugLog.StageTrigger,
$"phase_change already executed for phase {phaseName} — once per (rule, phase)");
continue;
}
_logger.Info($"Phase change triggered rule {rule.RuleId} (phase: {phaseName})");
DebugLog(rule.RuleId, GatherRuleDebugLog.StageTrigger, $"phase_change to {phaseName} fired");
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(rule));
}
}
@@ -288,10 +376,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
if (string.Equals(rule.TriggerEventType, eventType, StringComparison.OrdinalIgnoreCase))
{
if (!IsRuleInScope(rule))
string scopeReason;
if (!IsRuleInScope(rule, out scopeReason))
{
DebugLog(rule.RuleId, GatherRuleDebugLog.StageScope,
$"on_event {eventType} skipped: {scopeReason}");
continue;
}
_logger.Info($"Event triggered rule {rule.RuleId} (event: {eventType})");
DebugLog(rule.RuleId, GatherRuleDebugLog.StageTrigger, $"on_event {eventType} fired");
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(rule));
}
}
@@ -302,6 +396,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
try
{
_logger.Info($"Executing gather rule: {rule.RuleId} ({rule.Title})");
DebugLog(rule.RuleId, GatherRuleDebugLog.StageExec, $"executing: collector={rule.CollectorType}, target={rule.Target}");
var collectorType = rule.CollectorType?.ToLower();
@@ -309,6 +404,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
if (collectorType == null || !_collectors.TryGetValue(collectorType, out collector))
{
_logger.Warning($"Unknown collector type: {rule.CollectorType} for rule {rule.RuleId}");
DebugLog(rule.RuleId, GatherRuleDebugLog.StageError, $"unknown collector type '{rule.CollectorType}' — nothing emitted");
return;
}
@@ -316,7 +412,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// LogParser (and potentially others) emit events directly and return null
if (result == null)
{
DebugLog(rule.RuleId, GatherRuleDebugLog.StageCollector,
collectorType == "logparser"
? "logparser emits per-match events directly — see logparser lines above for the per-file outcome"
: "collector returned null — nothing emitted");
return;
}
if (result.Count == 0)
{
DebugLog(rule.RuleId, GatherRuleDebugLog.StageCollector,
"collector returned EMPTY result — nothing emitted (typical causes: target not found with emitOnlyIfExists, or guard block — see guard line if present)");
}
if (result.Count > 0)
{
@@ -325,8 +433,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// empty result never reaches this point, so an emitOnlyIfExists miss neither
// emits nor updates the hash — composing to "one event on appearance, then
// only on change".
if (IsOnChangeMode(rule) && !ShouldEmitOnChange(rule, result))
return;
if (IsOnChangeMode(rule))
{
string hashPrefix;
int suppressedStreak;
if (!ShouldEmitOnChange(rule, result, out hashPrefix, out suppressedStreak))
{
DebugLog(rule.RuleId, GatherRuleDebugLog.StageSuppress,
$"result unchanged (hash {hashPrefix}) — emit suppressed (on_change), suppressed streak={suppressedStreak}");
return;
}
DebugLog(rule.RuleId, GatherRuleDebugLog.StageEmit,
$"result changed (hash {hashPrefix})" + (suppressedStreak > 0 ? $" after {suppressedStreak} suppressed polls" : ""));
}
result["ruleId"] = rule.RuleId;
result["ruleTitle"] = rule.Title;
@@ -357,11 +476,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
Message = $"Gather: {rule.Title}",
Data = result
});
DebugLog(rule.RuleId, GatherRuleDebugLog.StageEmit,
$"emitted {eventType} (severity={severity}, {result.Count} fields)");
}
}
catch (Exception ex)
{
_logger.Warning($"Gather rule {rule.RuleId} failed: {ex.Message}");
// Full exception incl. stack trace only in the debug trace — the agent log stays terse.
DebugLog(rule.RuleId, GatherRuleDebugLog.StageError, $"rule execution failed: {ex}");
}
}
@@ -375,15 +498,33 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// Internal so the scope matrix is testable synchronously (no timer/ThreadPool races).
internal bool IsRuleInScope(GatherRule rule)
{
string reason;
return IsRuleInScope(rule, out reason);
}
/// <summary>
/// Scope gate with a human-readable skip reason for the debug trace. The reason is
/// captured under the lock but must be WRITTEN outside it (never trace under _scopeLock).
/// </summary>
internal bool IsRuleInScope(GatherRule rule, out string reason)
{
lock (_scopeLock)
{
return IsRuleInScopeLocked(rule);
return IsRuleInScopeLocked(rule, out reason);
}
}
private bool IsRuleInScopeLocked(GatherRule rule)
{
string reason;
return IsRuleInScopeLocked(rule, out reason);
}
private bool IsRuleInScopeLocked(GatherRule rule, out string reason)
{
reason = null;
if (!HasPhaseScope(rule))
return true; // unscoped — legacy behavior
@@ -400,7 +541,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// No phase signal yet (or scope tokens that never match, e.g. "Unknown"):
// scoped rules stay inactive.
if (_currentPhase == EnrollmentPhase.Unknown)
{
reason = "currentPhase=Unknown — scoped rules stay inactive until the first phase signal";
return false;
}
var phaseName = _currentPhase.ToString();
foreach (var phase in rule.ActivePhases)
@@ -408,10 +552,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
if (string.Equals(phase, phaseName, StringComparison.OrdinalIgnoreCase))
return true;
}
reason = $"out of scope (currentPhase={phaseName}, activePhases=[{string.Join(",", rule.ActivePhases)}])";
return false;
}
return _fromPhaseLatched.Contains(rule.RuleId);
if (_fromPhaseLatched.Contains(rule.RuleId))
return true;
reason = $"activeFromPhase={rule.ActiveFromPhase} not latched yet (currentPhase={_currentPhase})";
return false;
}
/// <summary>
@@ -457,8 +606,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
/// Internal so the state machine is testable synchronously (no timer/ThreadPool races).
/// </summary>
internal bool ShouldEmitOnChange(GatherRule rule, Dictionary<string, object> result)
{
string hashPrefix;
int suppressedStreak;
return ShouldEmitOnChange(rule, result, out hashPrefix, out suppressedStreak);
}
/// <summary>
/// Overload exposing the hash prefix and suppression streak for the debug trace:
/// on suppress, <paramref name="suppressedStreak"/> is the updated streak count;
/// on emit, it is the streak that just ended (0 when none).
/// </summary>
internal bool ShouldEmitOnChange(GatherRule rule, Dictionary<string, object> result,
out string hashPrefix, out int suppressedStreak)
{
var hash = ComputeCanonicalHash(result);
hashPrefix = hash.Length > 8 ? hash.Substring(0, 8) : hash;
suppressedStreak = 0;
bool emit;
lock (_scopeLock)
{
@@ -469,6 +633,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
_suppressed[rule.RuleId] = _suppressed.TryGetValue(rule.RuleId, out streak)
? (streak.Count + 1, streak.SinceUtc)
: (1, DateTime.UtcNow);
suppressedStreak = _suppressed[rule.RuleId].Count;
emit = false;
}
else
@@ -479,6 +644,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
result["suppressedPolls"] = streak.Count;
result["suppressedSinceUtc"] = streak.SinceUtc.ToString("o", CultureInfo.InvariantCulture);
suppressedStreak = streak.Count;
}
_suppressed.Remove(rule.RuleId);
emit = true;
@@ -77,11 +77,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
protected override void Collect()
{
// Full path writes 18 keys: agent_version + process metrics (5: cpu, ws, private,
// threads, handles) + spool stats (4) + network delta (8: requests, failures,
// bytes_up/down, avg_latency, total_up/down/requests). cap=18 → HashHelpers.GetPrime(18)=23
// buckets → no resize on the 18th key (cap=16 would land at 17 buckets and resize).
var data = new Dictionary<string, object>(capacity: 18, StringComparer.Ordinal)
// Full path writes 19 keys: agent_version + process metrics (5: cpu, ws, private,
// threads, handles) + spool stats (4) + network delta (9: requests, failures,
// bytes_up/down, avg_latency, total_up/down/requests/latency). cap=19 →
// HashHelpers.GetPrime(19)=23 buckets → no resize on the 19th key.
var data = new Dictionary<string, object>(capacity: 19, StringComparer.Ordinal)
{
{ "agent_version", _agentVersion }
};
@@ -164,6 +164,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
data["net_total_bytes_up"] = currentNet.TotalBytesUp;
data["net_total_bytes_down"] = currentNet.TotalBytesDown;
data["net_total_requests"] = currentNet.RequestCount;
// Cumulative latency sum: total/requests = session-wide average HTTP
// round-trip. The backend projects that average onto the Session row
// (AvgApiLatencyMs) — cumulative counters make the write idempotent.
data["net_total_latency_ms"] = currentNet.TotalLatencyMs;
}
catch (Exception ex)
{
@@ -8,8 +8,8 @@ using System.Net.Sockets;
using System.Threading;
using System.Threading.Tasks;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Security;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
@@ -37,7 +37,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
private const int DebounceDelayMs = 3000;
private const int ConnectivityCheckDelayMs = 5000;
private const int ConnectivityCheckTimeoutMs = 8000;
private const int WifiCollectTimeoutMs = 3000;
private const string Source = "NetworkChangeDetector";
/// <summary>
@@ -374,7 +373,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
// Collect WiFi SSID synchronously (needed for before/after comparison)
if (isWifi)
{
var wifiInfo = CollectWiFiInfo();
Guid? nicGuid = Guid.TryParse(activeNic.Id, out var parsedGuid) ? parsedGuid : (Guid?)null;
var wifiInfo = CollectWiFiInfo(nicGuid);
snapshot.WifiSsid = wifiInfo.ssid;
snapshot.WifiSignalPercent = wifiInfo.signal;
snapshot.WifiRadioType = wifiInfo.radioType;
@@ -390,73 +390,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
}
/// <summary>
/// Collects WiFi SSID, signal, and radio type via netsh wlan show interfaces.
/// Synchronous with timeout — needed before emitting the event.
/// Collects WiFi SSID, signal, and radio type via the native WLAN API
/// (<see cref="WifiInfoReader"/> — language-neutral, no process spawn).
/// Synchronous — needed before emitting the event.
/// </summary>
private (string ssid, int? signal, string radioType) CollectWiFiInfo()
private (string ssid, int? signal, string radioType) CollectWiFiInfo(Guid? interfaceGuid)
{
string ssid = null;
int? signal = null;
string radioType = null;
try
{
var psi = new ProcessStartInfo
var wifi = WifiInfoReader.TryGetCurrentConnection(interfaceGuid, out var wifiDiag);
if (wifi == null)
{
FileName = SystemPaths.Netsh,
Arguments = "wlan show interfaces",
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true
};
string output;
using (var process = Process.Start(psi))
{
output = process.StandardOutput.ReadToEnd();
if (!process.WaitForExit(WifiCollectTimeoutMs))
{
try { process.Kill(); } catch { }
_logger.Debug("NetworkChangeDetector: netsh timed out");
return (null, null, null);
}
_logger.Debug($"NetworkChangeDetector: native WLAN API returned no WiFi info ({wifiDiag ?? "no connected WLAN interface"}) — trying netsh fallback");
wifi = NetshWifiFallback.TryRead();
}
if (string.IsNullOrEmpty(output))
return (null, null, null);
foreach (var line in output.Split(new[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries))
{
var trimmed = line.Trim();
var colonIndex = trimmed.IndexOf(':');
if (colonIndex < 0) continue;
var key = trimmed.Substring(0, colonIndex).Trim();
var value = trimmed.Substring(colonIndex + 1).Trim();
if (key.Equals("SSID", StringComparison.OrdinalIgnoreCase) &&
!key.Equals("BSSID", StringComparison.OrdinalIgnoreCase))
{
ssid = value;
}
else if (key.Equals("Signal", StringComparison.OrdinalIgnoreCase))
{
if (int.TryParse(value.TrimEnd('%'), out var sig))
signal = sig;
}
else if (key.Equals("Radio type", StringComparison.OrdinalIgnoreCase))
{
radioType = value;
}
}
if (wifi != null)
return (wifi.Ssid, wifi.SignalPercent, wifi.RadioType);
}
catch (Exception ex)
{
_logger.Debug($"NetworkChangeDetector: WiFi info collection failed: {ex.Message}");
}
return (ssid, signal, radioType);
return (null, null, null);
}
// -------------------------------------------------------------------
@@ -483,7 +483,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
logger: logger,
rules: _remoteConfig.GatherRules,
imeLogPathOverride: _agentConfig.ImeLogPathOverride,
unrestrictedMode: _agentConfig.UnrestrictedMode));
unrestrictedMode: _agentConfig.UnrestrictedMode,
gatherDebugLogPath: _agentConfig.GatherRuleDebugLogPath));
}
else if (!string.IsNullOrEmpty(_agentConfig.GatherRuleDebugLogPath))
{
// Debug trace is on but the backend delivered zero gather rules — the trace
// file must say so instead of silently not existing.
Monitoring.Telemetry.Gather.GatherRuleDebugLog.WriteStandalone(
_agentConfig.GatherRuleDebugLogPath,
"no gather rules delivered by backend — nothing to execute", logger);
}
return new CollectorSurfaces(hosts, imeLogHost, espAndHelloHost, aadJoinHost);
@@ -39,7 +39,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
AgentLogger logger,
List<GatherRule> rules,
string? imeLogPathOverride,
bool unrestrictedMode = false)
bool unrestrictedMode = false,
string? gatherDebugLogPath = null)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
@@ -55,9 +56,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// session mode we wrap post.Emit so every session-mode gather event still
// flows through the InformationalEvent ingress pipe before hitting the
// telemetry spool — Rail-A semantics for ordering / replay determinism.
var post = new InformationalEventPost(ingress, clock);
var post = new InformationalEventPost(ingress, clock, logger);
_executor = new Monitoring.Telemetry.Gather.GatherRuleExecutor(
sessionId, tenantId, evt => post.Emit(evt), logger, imeLogPathOverride);
sessionId, tenantId, evt => post.Emit(evt), logger, imeLogPathOverride,
debugLogPath: gatherDebugLogPath);
_observableIngress = ingress as SignalIngress;
}
@@ -39,8 +39,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
/// <see cref="SessionStage.WhiteGloveSealed"/> (Part-1 exit, session resumes Part 2).</item>
/// <item>WhiteGlove Part-1 path: stop peripheral collectors first (so their stop-time
/// stragglers — e.g. <c>network_bandwidth_estimate</c> — sequence BEFORE the marker),
/// then emit <c>whiteglove_part1_complete</c>, drain spool, and write
/// <c>whiteglove.complete</c> marker via <see cref="SessionIdPersistence.SaveWhiteGloveComplete"/>
/// upload the diagnostics package as a pre-provisioning intermediate (suffix
/// <c>preprov</c>, same mode gating as the terminal path — the seal counts as a
/// success, so <c>OnFailure</c> skips), then emit <c>whiteglove_part1_complete</c>,
/// drain spool, and write <c>whiteglove.complete</c> marker via
/// <see cref="SessionIdPersistence.SaveWhiteGloveComplete"/>
/// so Part-2 resume is detected on the next boot.</item>
/// <item>Signal the caller-owned shutdown <see cref="ManualResetEventSlim"/>.</item>
/// </list>
@@ -241,6 +244,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
// the full Stop() later is a no-op for the hosts.
StopPeripheralCollectorsBestEffort();
// V1 parity (restored 2026-07-28) — upload the diagnostics package as a
// pre-provisioning intermediate. Between reseal and user OOBE the device
// may sit boxed for weeks, and if Part 2 never uploads (never runs, or
// OnFailure + success) the Part-1 forensics would be lost to self-destruct.
// No dedicated flag: the existing mode semantics apply — Always uploads,
// OnFailure skips (the seal is a success; a Part-1 failure terminates via
// the regular Failed path below, which uploads), Off does nothing. Must run
// BEFORE the whiteglove_part1_complete emit so diagnostics_* events keep a
// lower sequence than the Part-1/Part-2 boundary marker. Best-effort: a
// reseal-reboot racing the upload just loses the intermediate package.
RunUploadDiagnosticsWithEvents(args, suffixOverride: "preprov");
EmitEventSafe(new EnrollmentEvent
{
SessionId = _configuration.SessionId,
@@ -623,12 +638,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
}
/// <summary>
/// Emits a one-shot <c>agent_late_start</c> Warning when the agent had low observation
/// Emits a one-shot <c>agent_late_start</c> event when the agent had low observation
/// coverage: it started a long time after device boot AND lived only briefly before this
/// terminal outcome (gate in <see cref="ObservationCoverage"/>). Such a session looks like a
/// normal multi-minute failure in the list (StartedAt is back-dated to the earliest replayed
/// IME-log event) but the agent actually observed only the end-state — so the operator should
/// read the diagnosis as a post-mortem reconstruction, not live evidence. No state mutation.
/// IME-log event) but the agent actually observed only the end-state. Severity and phrasing
/// are outcome-calibrated via <see cref="ObservationCoverage.DescribeLateStart"/>: a failure
/// stays a Warning with the post-mortem/hung-script note, a success is an Info about
/// tail-only coverage (fleet 2026-07-28: tenant 659c3a90 collected 12 Warnings on healthy
/// enrollments whose bootstrap merely ran late). No state mutation.
/// </summary>
private void MaybeEmitAgentLateStart(EnrollmentTerminatedEventArgs args)
{
@@ -642,21 +660,24 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
out var bootToStartSeconds, out var uptimeSeconds))
return;
ObservationCoverage.DescribeLateStart(
args.Outcome, _oobeStateAtAgentStart, bootToStartSeconds, uptimeSeconds,
out var severity, out var message, out var note);
_logger.Info(
$"EnrollmentTerminationHandler: low observation coverage — agent started " +
$"{bootToStartSeconds / 60.0:F1} min after boot and lived {uptimeSeconds:F0}s; " +
$"emitting agent_late_start.");
$"emitting agent_late_start ({severity}, outcome {args.Outcome}).");
_post.Emit(new EnrollmentEvent
{
SessionId = _configuration.SessionId,
TenantId = _configuration.TenantId,
EventType = Constants.EventTypes.AgentLateStart,
Severity = EventSeverity.Warning,
Severity = severity,
Source = "EnrollmentTerminationHandler",
Phase = EnrollmentPhase.Unknown,
Message = $"Agent started {bootToStartSeconds / 60.0:F0} min after boot and observed only " +
$"{uptimeSeconds:F0}s before terminating ({args.Outcome}) — low coverage of the enrollment window.",
Message = message,
Data = new Dictionary<string, object>
{
{ "bootToAgentStartSeconds", Math.Round(bootToStartSeconds, 1) },
@@ -665,7 +686,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
{ "agentStartUtc", _agentStartTimeUtc.ToString("o") },
{ "oobeStateAtAgentStart", _oobeStateAtAgentStart },
{ "outcome", args.Outcome.ToString() },
{ "note", "The agent's bootstrap (an Intune platform script) ran only near the end of the enrollment, so the agent observed the already-decided end-state rather than the failure window. Treat the diagnosis as a post-mortem; check for a platform/remediation script that hung ahead of the bootstrap (see script_timeout_suspected)." },
{ "note", note },
},
ImmediateUpload = true,
});
@@ -771,7 +792,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
StringComparison.Ordinal);
}
private void RunUploadDiagnosticsWithEvents(EnrollmentTerminatedEventArgs args)
/// <param name="args">Termination payload — outcome drives the OnFailure gate.</param>
/// <param name="suffixOverride">
/// Blob-name suffix override. Null (terminal path) derives "success"/"failure" from
/// the outcome; the WhiteGlove Part-1 seal passes "preprov" so the intermediate
/// package is distinguishable from the later Part-2 terminal upload.
/// </param>
private void RunUploadDiagnosticsWithEvents(EnrollmentTerminatedEventArgs args, string suffixOverride = null)
{
var mode = _configuration.DiagnosticsUploadMode ?? "Off";
if (!_configuration.DiagnosticsUploadEnabled || string.Equals(mode, "Off", StringComparison.OrdinalIgnoreCase))
@@ -807,7 +834,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
DiagnosticsUploadResult result = null;
try
{
var suffix = enrollmentSucceeded ? "success" : "failure";
var suffix = suffixOverride ?? (enrollmentSucceeded ? "success" : "failure");
result = _uploadDiagnosticsAsync(enrollmentSucceeded, suffix).GetAwaiter().GetResult();
}
catch (Exception ex)
@@ -1,6 +1,8 @@
#nullable enable
using System;
using System.Runtime.InteropServices;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Shared.Models;
namespace AutopilotMonitor.Agent.V2.Core.Termination
{
@@ -76,5 +78,57 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
uptimeSeconds = Math.Max(0, uptime.TotalSeconds);
return bootToStart >= LateStartBootThreshold && uptime <= LowCoverageUptime;
}
/// <summary>
/// Severity + operator-facing phrasing for the <c>agent_late_start</c> event, calibrated
/// by fleet data (2026-07-28): the gate alone is outcome-blind and attached a
/// "post-mortem / check for a hung script" Warning to sessions that in fact succeeded —
/// there the late bootstrap is deployment hygiene, not an alarm. The OOBE state refines
/// the phrasing but must never gate: the canonical hung-script failure terminates on the
/// still-shown ESP screen, i.e. with OOBE reading <c>in_progress</c>.
/// </summary>
/// <param name="outcome">Terminal outcome the enrollment reached.</param>
/// <param name="oobeStateAtAgentStart"><see cref="Monitoring.Interop.OobeStateReader"/>
/// sample taken at THIS process's start. Only <c>completed</c> / <c>in_progress</c> are
/// interpreted; <c>not_started</c> flips in after mid-OOBE reboots and <c>completed</c> is
/// routine for post-desktop restarts, so no claim is phrased from the other values.</param>
/// <param name="bootToStartSeconds">From <see cref="IsLowObservationCoverage"/>.</param>
/// <param name="uptimeSeconds">From <see cref="IsLowObservationCoverage"/>.</param>
internal static void DescribeLateStart(
EnrollmentTerminationOutcome outcome,
string oobeStateAtAgentStart,
double bootToStartSeconds,
double uptimeSeconds,
out EventSeverity severity,
out string message,
out string note)
{
var succeeded = outcome == EnrollmentTerminationOutcome.Succeeded;
severity = succeeded ? EventSeverity.Info : EventSeverity.Warning;
message = $"Agent started {bootToStartSeconds / 60.0:F0} min after boot and observed only " +
$"{uptimeSeconds:F0}s before terminating ({outcome}) — low coverage of the enrollment window.";
switch (oobeStateAtAgentStart)
{
case "completed":
message += " OOBE was already completed when the agent started — the diagnosis is a " +
"post-mortem reconstruction, not live observation.";
break;
case "in_progress":
message += $" OOBE was still in progress at agent start — the agent did observe the " +
$"final {uptimeSeconds:F0}s live.";
break;
}
note = succeeded
? "The agent's bootstrap (an Intune platform script) ran late in the enrollment, so only the " +
"final stretch was observed live; earlier timeline entries are reconstructed from IME log " +
"replay. The enrollment itself succeeded. If this recurs across sessions, review which " +
"platform/remediation scripts run ahead of the bootstrap (see script_timeout_suspected)."
: "The agent's bootstrap (an Intune platform script) ran only near the end of the enrollment, " +
"so the agent observed the already-decided end-state rather than the failure window. Treat " +
"the diagnosis as a post-mortem; check for a platform/remediation script that hung ahead of " +
"the bootstrap (see script_timeout_suspected).";
}
}
}
@@ -221,8 +221,18 @@ namespace AutopilotMonitor.Agent.V2
},
});
// Step 4 — run the executor and wait.
using (var executor = new GatherRuleExecutor(sessionId, tenantId, emitEvent, logger, config.ImeLogPathOverride))
// Step 4 — run the executor and wait. Diagnostic mode always writes the
// gather-rule debug trace (CLI override or default path) and mirrors it
// to the console in --console mode.
var gatherDebugLogPath = !string.IsNullOrEmpty(config.GatherRuleDebugLogPath)
? config.GatherRuleDebugLogPath
: Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath);
logger.Info($"--run-gather-rules: debug trace -> {gatherDebugLogPath}");
if (consoleMode) Console.WriteLine($"Debug trace: {gatherDebugLogPath}");
using (var executor = new GatherRuleExecutor(sessionId, tenantId, emitEvent, logger, config.ImeLogPathOverride,
debugLogPath: gatherDebugLogPath,
debugEcho: consoleMode ? (Action<string>)(line => Console.WriteLine($" {line}")) : null))
{
// Diagnostic mode has no enrollment phase context — scoped rules
// (activePhases / activeFromPhase) execute unconditionally here.
@@ -303,6 +303,7 @@ namespace AutopilotMonitor.Agent.V2
var apiBaseUrl = GetArgValue(args, "--api-url", "--backend-api") ?? Constants.ApiBaseUrl;
var imeLogPathOverride = GetArgValue(args, "--ime-log-path");
var imeMatchLogPath = GetArgValue(args, "--ime-match-log");
var gatherDebugLogPath = GetArgValue(args, "--gather-debug-log");
// Dev / test — IME log replay (V1 compat mode). Feeds ImeLogTracker.SimulationMode
// + SpeedFactor so recorded raw IME logs are replayed at an accelerated rate; signal
@@ -373,6 +374,7 @@ namespace AutopilotMonitor.Agent.V2
EnableGeoLocation = !disableGeoLocation,
ImeLogPathOverride = imeLogPathOverride,
ImeMatchLogPath = imeMatchLogPath,
GatherRuleDebugLogPath = gatherDebugLogPath,
KeepLogFile = keepLogFile,
SelfDestructOnComplete = !noCleanup,
CommandLineArgs = FormatArgsForLog(args),
@@ -485,6 +487,7 @@ namespace AutopilotMonitor.Agent.V2
Console.Out.WriteLine(" --api-url <URL> Override backend API base URL (alias: --backend-api)");
Console.Out.WriteLine(" --ime-log-path <PATH> Override IME logs directory");
Console.Out.WriteLine(" --ime-match-log <PATH> Write matched IME log lines to file (debug)");
Console.Out.WriteLine(" --gather-debug-log <PATH> Write gather-rule evaluation trace to file (debug)");
Console.Out.WriteLine();
Console.Out.WriteLine("Dev / Test:");
Console.Out.WriteLine(" --replay-log-dir <PATH> Replay IME logs from this directory (simulation mode)");
@@ -46,8 +46,12 @@ namespace AutopilotMonitor.Agent.V2.Runtime
try { var _ = await remoteConfigService.FetchConfigAsync(); return true; }
catch (Exception ex) { logger.Warning($"ServerAction rotate_config failed: {ex.Message}"); return false; }
},
// On-demand request mid-enrollment — the session has no outcome yet. Null keeps
// sessioninfo.txt honest ("In Progress") and passes the OnFailure gate by
// semantics (there is no success to skip on) instead of the former
// enrollmentSucceeded:false workaround that stamped in-flight packages "Failed".
uploadDiagnosticsAsync: async (suffix) =>
await diagnosticsService.CreateAndUploadAsync(enrollmentSucceeded: false, fileNameSuffix: suffix),
await diagnosticsService.CreateAndUploadAsync(enrollmentSucceeded: null, fileNameSuffix: suffix),
onTerminateRequested: action => OnTerminateRequested(
action, agentConfig, orchestrator, terminationHandler, shutdown, shutdownComplete, logger),
post: post);
@@ -34,6 +34,14 @@ public class AppsProjectionEquivalenceTests
["AppType"] = "Win32",
["AppVersion"] = "2.1.0",
["Status"] = "Failed",
// PR0/PR1 classification columns. The fixture MUST populate every column a Build*
// method reads: a column absent here passes the equivalence assert trivially (both
// sides map to default) — exactly how the missing TerminalState projection slipped
// through and zeroed totalSkipped in production (2026-07-27).
["TerminalState"] = "Skipped",
["AppId"] = "9e8d7c6b-5a49-4832-9182-736451234567",
["EspBlocking"] = true,
["AppIdCollision"] = true,
["StartedAt"] = new DateTimeOffset(DateTime.UtcNow.AddHours(-6)),
["CompletedAt"] = new DateTimeOffset(DateTime.UtcNow.AddHours(-5)),
["DurationSeconds"] = 340,
@@ -83,6 +91,9 @@ public class AppsProjectionEquivalenceTests
Assert.Equal(fromFull.AppName, fromProjected.AppName);
Assert.Equal(fromFull.Status, fromProjected.Status);
// IsSkipTerminalState / skip classification input — see the fixture comment.
Assert.Equal(fromFull.TerminalState, fromProjected.TerminalState);
Assert.Equal(fromFull.AppIdCollision, fromProjected.AppIdCollision);
Assert.Equal(fromFull.StartedAt, fromProjected.StartedAt);
Assert.Equal(fromFull.DurationSeconds, fromProjected.DurationSeconds);
Assert.Equal(fromFull.DownloadBytes, fromProjected.DownloadBytes);
@@ -122,6 +133,10 @@ public class AppsProjectionEquivalenceTests
Assert.Equal(fromFull.FailureMessage, fromProjected.FailureMessage);
Assert.Equal(fromFull.ExitCode, fromProjected.ExitCode);
Assert.Equal(fromFull.DetectionResult, fromProjected.DetectionResult);
Assert.Equal(fromFull.TerminalState, fromProjected.TerminalState);
Assert.Equal(fromFull.AppId, fromProjected.AppId);
Assert.Equal(fromFull.EspBlocking, fromProjected.EspBlocking);
Assert.Equal(fromFull.AppIdCollision, fromProjected.AppIdCollision);
}
[Fact]
@@ -0,0 +1,172 @@
using AutopilotMonitor.Functions.Functions.Infrastructure;
using AutopilotMonitor.Shared.DataAccess;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the pure helpers of <see cref="AzureMonitorAlertWebhookFunction"/> — the anonymous,
/// secret-gated bridge that turns Azure Monitor Common Alert Schema notifications into ops
/// events. Load-bearing bits: the secret gate is fail-closed (an unset app setting rejects
/// everything), severity mapping can never escalate on schema drift (unknown → Info), and
/// parsing is defensive (only the alert rule name is mandatory).
/// </summary>
public class AzureMonitorAlertWebhookFunctionTests
{
// ── SecretMatches ──────────────────────────────────────────────────────
[Fact]
public void SecretMatches_accepts_equal_secrets()
{
Assert.True(AzureMonitorAlertWebhookFunction.SecretMatches("s3cret-value", "s3cret-value"));
}
[Theory]
[InlineData("wrong", "right")]
[InlineData("right-but-longer", "right")]
[InlineData("", "configured")]
[InlineData(null, "configured")]
public void SecretMatches_rejects_mismatch_or_missing_provided(string? provided, string configured)
{
Assert.False(AzureMonitorAlertWebhookFunction.SecretMatches(provided, configured));
}
[Theory]
[InlineData(null)]
[InlineData("")]
public void SecretMatches_fails_closed_when_not_configured(string? configured)
{
// While the app setting is unset, NOTHING matches — not even an empty provided value.
Assert.False(AzureMonitorAlertWebhookFunction.SecretMatches("anything", configured));
Assert.False(AzureMonitorAlertWebhookFunction.SecretMatches("", configured));
Assert.False(AzureMonitorAlertWebhookFunction.SecretMatches(null, configured));
}
// ── MapSeverity ────────────────────────────────────────────────────────
[Theory]
[InlineData("Sev0", OpsEventSeverity.Critical)]
[InlineData("Sev1", OpsEventSeverity.Error)]
[InlineData("Sev2", OpsEventSeverity.Warning)]
[InlineData("Sev3", OpsEventSeverity.Info)]
[InlineData("Sev4", OpsEventSeverity.Info)]
public void MapSeverity_maps_azure_scale_to_ops_scale(string azure, string expected)
{
Assert.Equal(expected, AzureMonitorAlertWebhookFunction.MapSeverity(azure));
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("Sev5")]
[InlineData("Critical")] // someone posting the ops scale directly must not pass through
public void MapSeverity_falls_back_to_info_on_unknown_values(string? azure)
{
Assert.Equal(OpsEventSeverity.Info, AzureMonitorAlertWebhookFunction.MapSeverity(azure));
}
// ── TryParse ───────────────────────────────────────────────────────────
private const string FullPayload = """
{
"schemaId": "azureMonitorCommonAlertSchema",
"data": {
"essentials": {
"alertId": "/subscriptions/xxx/providers/Microsoft.AlertsManagement/alerts/abc",
"alertRule": "Portal RSC prefetch tail",
"severity": "Sev2",
"signalType": "Log",
"monitorCondition": "Fired",
"monitoringService": "Log Alerts V2",
"alertTargetIDs": [
"/subscriptions/xxx/resourcegroups/rg-autopilotmonitor-prd-gwc/providers/microsoft.insights/components/autopilotmonitorweb"
],
"description": "RSC prefetches exceeding 10s",
"firedDateTime": "2026-07-27T15:30:00.000Z"
},
"alertContext": {
"condition": {
"windowSize": "PT1H",
"allOf": [
{
"searchQuery": "dependencies | where name contains '_rsc='",
"metricValue": 18.0,
"operator": "GreaterThan",
"threshold": "0"
}
]
}
}
}
}
""";
[Fact]
public void TryParse_extracts_essentials_and_metric_value()
{
var alert = AzureMonitorAlertWebhookFunction.TryParse(FullPayload);
Assert.NotNull(alert);
Assert.Equal("Portal RSC prefetch tail", alert!.AlertRule);
Assert.Equal("Sev2", alert.Severity);
Assert.Equal("Fired", alert.MonitorCondition);
Assert.Equal("RSC prefetches exceeding 10s", alert.Description);
Assert.Equal("Log Alerts V2", alert.MonitoringService);
// No targetResourceName in the payload → resource name segment of the first target ID.
Assert.Equal("autopilotmonitorweb", alert.TargetResource);
Assert.Equal(18.0, alert.MetricValue);
}
[Fact]
public void TryParse_prefers_explicit_target_resource_name()
{
const string payload = """
{"data":{"essentials":{"alertRule":"r","targetResourceName":"explicit-name",
"alertTargetIDs":["/sub/x/other-name"]}}}
""";
var alert = AzureMonitorAlertWebhookFunction.TryParse(payload);
Assert.Equal("explicit-name", alert!.TargetResource);
}
[Fact]
public void TryParse_minimal_payload_needs_only_the_alert_rule()
{
var alert = AzureMonitorAlertWebhookFunction.TryParse(
"""{"data":{"essentials":{"alertRule":"minimal"}}}""");
Assert.NotNull(alert);
Assert.Equal("minimal", alert!.AlertRule);
Assert.Null(alert.Severity);
Assert.Equal("Fired", alert.MonitorCondition); // default when absent
Assert.Null(alert.Description);
Assert.Null(alert.MetricValue);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData(" ")]
[InlineData("not json at all")]
[InlineData("{}")] // no data
[InlineData("""{"data":{}}""")] // no essentials
[InlineData("""{"data":{"essentials":{}}}""")] // no alertRule
[InlineData("""{"data":{"essentials":{"alertRule":""}}}""")] // empty alertRule
[InlineData("""{"data":{"essentials":{"alertRule":123}}}""")] // wrong type
public void TryParse_rejects_unusable_payloads(string? body)
{
Assert.Null(AzureMonitorAlertWebhookFunction.TryParse(body!));
}
[Fact]
public void TryParse_tolerates_malformed_alert_context()
{
// alertContext with unexpected shapes must not break essentials extraction.
const string payload = """
{"data":{"essentials":{"alertRule":"r","severity":"Sev0"},
"alertContext":{"condition":{"allOf":"not-an-array"}}}}
""";
var alert = AzureMonitorAlertWebhookFunction.TryParse(payload);
Assert.NotNull(alert);
Assert.Equal("Sev0", alert!.Severity);
Assert.Null(alert.MetricValue);
}
}
@@ -0,0 +1,48 @@
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Cover for the ConnectionType predicate used on every search path that does NOT push the
/// filter to OData — the deviceProperties batch-get (<c>ApplyBasicFilters</c>) and the legacy
/// unpaged scan. Sessions that predate the projection (null ConnectionType) must never match
/// a set filter; an unset filter matches everything.
/// </summary>
public class ConnectionTypeFilterTests
{
private static SessionSummary Session(string? connectionType) =>
new() { SessionId = "s", TenantId = "t", ConnectionType = connectionType };
[Theory]
[InlineData(null)]
[InlineData("WiFi")]
[InlineData("Ethernet")]
public void NoFilter_AlwaysMatches(string? connectionType)
{
var filter = new SessionSearchFilter();
Assert.True(TableStorageService.MatchesConnectionType(Session(connectionType), filter));
}
[Theory]
[InlineData("Ethernet", true)]
[InlineData("ethernet", true)] // case-insensitive, matching the other string filters
[InlineData("WiFi", false)]
[InlineData(null, false)] // legacy session without the projected column
public void EthernetFilter_MatchesOnlyEthernet(string? connectionType, bool expected)
{
var filter = new SessionSearchFilter { ConnectionType = "Ethernet" };
Assert.Equal(expected, TableStorageService.MatchesConnectionType(Session(connectionType), filter));
}
[Theory]
[InlineData("WiFi", true)]
[InlineData("Ethernet", false)]
[InlineData(null, false)]
public void WiFiFilter_MatchesOnlyWiFi(string? connectionType, bool expected)
{
var filter = new SessionSearchFilter { ConnectionType = "WiFi" };
Assert.Equal(expected, TableStorageService.MatchesConnectionType(Session(connectionType), filter));
}
}
@@ -50,6 +50,8 @@ public class GeoMetricsProjectionEquivalenceTests
["GeoCity"] = "Frankfurt",
["GeoLoc"] = "50.11,8.68",
["IsPreProvisioned"] = false,
["AvgApiLatencyMs"] = 217.4,
["ApiRequestCount"] = 270,
// Noise the geo aggregation never reads.
["SerialNumber"] = "SN-FULL",
["Manufacturer"] = "Contoso",
@@ -76,6 +78,8 @@ public class GeoMetricsProjectionEquivalenceTests
Assert.Equal(fromFull.GeoRegion, fromProjected.GeoRegion);
Assert.Equal(fromFull.GeoCity, fromProjected.GeoCity);
Assert.Equal(fromFull.GeoLoc, fromProjected.GeoLoc);
Assert.Equal(fromFull.AvgApiLatencyMs, fromProjected.AvgApiLatencyMs);
Assert.Equal(fromFull.ApiRequestCount, fromProjected.ApiRequestCount);
// The grouping key must be identical too — it is composed from the Geo* fields.
Assert.Equal(
@@ -0,0 +1,124 @@
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Functions.Functions.Metrics;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the API-latency aggregation of
/// <see cref="GetGeographicMetricsFunction.ComputeGeographicMetrics"/> (2026-07-28): both the
/// per-location and the global figure are REQUEST-WEIGHTED averages —
/// Σ(sessionAvg·requests)/Σ(requests) — not averages of session averages, so a long session
/// with many uploads counts proportionally. Sessions without latency data (agents predating
/// <c>AvgApiLatencyMs</c>) drop out of the weighting entirely.
/// </summary>
public class GeographicMetricsApiLatencyTests
{
private static SessionSummary S(string country, string city, double? avgLatencyMs, int? requestCount)
=> new()
{
TenantId = "00000000-0000-0000-0000-000000000fe0",
SessionId = Guid.NewGuid().ToString(),
Status = SessionStatus.Succeeded,
StartedAt = DateTime.UtcNow.AddHours(-2),
GeoCountry = country,
GeoRegion = country,
GeoCity = city,
GeoLoc = "0,0",
AvgApiLatencyMs = avgLatencyMs,
ApiRequestCount = requestCount,
};
[Fact]
public void LocationLatency_IsRequestWeighted_NotAverageOfAverages()
{
var sessions = new List<SessionSummary>
{
S("ID", "Jakarta", 1000, 300),
S("ID", "Jakarta", 500, 100),
};
var result = GetGeographicMetricsFunction.ComputeGeographicMetrics(
sessions, new List<AppInstallSummary>(), "country");
var loc = Assert.Single(result.Locations);
// (1000·300 + 500·100) / 400 = 875 — an unweighted mean would say 750.
Assert.Equal(875d, loc.AvgApiLatencyMs);
Assert.Equal(2, loc.ApiLatencySessionCount);
}
[Fact]
public void SessionsWithoutLatencyData_DropOutOfWeighting_ButStayInSessionCount()
{
var sessions = new List<SessionSummary>
{
S("DE", "Frankfurt", 200, 100),
S("DE", "Frankfurt", avgLatencyMs: null, requestCount: null), // pre-feature agent
S("DE", "Frankfurt", avgLatencyMs: 0, requestCount: 50), // zero = no signal
};
var result = GetGeographicMetricsFunction.ComputeGeographicMetrics(
sessions, new List<AppInstallSummary>(), "country");
var loc = Assert.Single(result.Locations);
Assert.Equal(3, loc.SessionCount);
Assert.Equal(1, loc.ApiLatencySessionCount);
Assert.Equal(200d, loc.AvgApiLatencyMs);
}
[Fact]
public void NoLatencyDataAnywhere_YieldsZeroes_NotDivisionByZero()
{
var sessions = new List<SessionSummary> { S("DE", "Frankfurt", null, null) };
var result = GetGeographicMetricsFunction.ComputeGeographicMetrics(
sessions, new List<AppInstallSummary>(), "country");
var loc = Assert.Single(result.Locations);
Assert.Equal(0d, loc.AvgApiLatencyMs);
Assert.Equal(0, loc.ApiLatencySessionCount);
Assert.Equal(0d, loc.ApiLatencyVsGlobalPct);
Assert.Equal(0d, result.GlobalAverages.AvgApiLatencyMs);
}
[Fact]
public void GlobalLatency_WeightsAcrossLocations_AndVsGlobalPctCompares()
{
var sessions = new List<SessionSummary>
{
S("DE", "Frankfurt", 200, 100),
S("ID", "Jakarta", 800, 100),
};
var result = GetGeographicMetricsFunction.ComputeGeographicMetrics(
sessions, new List<AppInstallSummary>(), "country");
// Equal request weights → global = (200 + 800) / 2 = 500.
Assert.Equal(500d, result.GlobalAverages.AvgApiLatencyMs);
var de = result.Locations.Single(l => l.Country == "DE");
var id = result.Locations.Single(l => l.Country == "ID");
Assert.Equal(-60d, de.ApiLatencyVsGlobalPct); // 200 vs 500
Assert.Equal(60d, id.ApiLatencyVsGlobalPct); // 800 vs 500
}
[Fact]
public void LocationWithoutData_KeepsZeroPct_EvenWhenGlobalHasData()
{
var sessions = new List<SessionSummary>
{
S("DE", "Frankfurt", 200, 100),
S("FR", "Paris", null, null),
};
var result = GetGeographicMetricsFunction.ComputeGeographicMetrics(
sessions, new List<AppInstallSummary>(), "country");
var fr = result.Locations.Single(l => l.Country == "FR");
Assert.Equal(0d, fr.AvgApiLatencyMs);
Assert.Equal(0d, fr.ApiLatencyVsGlobalPct); // no data ≠ "0 ms fast"
}
}
@@ -0,0 +1,189 @@
using System.Text.Json;
using AutopilotMonitor.Functions.Functions.Metrics;
using AutopilotMonitor.Shared.Models;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Guards the cross-tenant redaction on GET metrics/ime-versions.
///
/// The route is <c>MemberRead</c> (EndpointAccessPolicyCatalog) and the archive itself is a
/// single global partition — every tenant member reads the SAME platform-wide rows. The only
/// thing standing between an ordinary tenant user and another tenant's identifiers is the
/// projection in <see cref="GetImeVersionHistoryFunction.BuildResponsePayload"/>. That made it
/// worth pinning: the redaction had no test at all, and a boundary nobody asserts is one a
/// later refactor can open without a single failure.
///
/// The JSON tests are the ones that matter — a property dropped from an anonymous type is only
/// a real redaction once it is also absent from the wire, and the wire is what the customer's
/// MCP client receives. camelCase mirrors Program.cs's PropertyNamingPolicy.
/// </summary>
public class GetImeVersionHistoryRedactionTests
{
private static readonly DateTime First = new(2026, 7, 8, 3, 18, 13, DateTimeKind.Utc);
private static readonly DateTime Last = new(2026, 7, 27, 16, 59, 34, DateTimeKind.Utc);
private static readonly JsonSerializerOptions CamelCase =
new() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase };
// =========================================================================
// Non-global caller (ordinary tenant member) — the redaction
// =========================================================================
[Fact]
public void NonGlobalCaller_DropsForeignTenantAndSessionIds()
{
var json = SerializeFor(hasGlobalScope: false, MakeEntry());
Assert.DoesNotContain("firstSeenTenantId", json, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("firstSeenSessionId", json, StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// The values, not just the property names: a rename of the model property must not let the
/// identifier survive under a different key. Asserting on the GUIDs catches that.
/// </summary>
[Fact]
public void NonGlobalCaller_ForeignIdentifierValuesAppearNowhereInThePayload()
{
var json = SerializeFor(hasGlobalScope: false, MakeEntry());
Assert.DoesNotContain("57f34dd1-6a42-47f9-9bc0-b921fa6caa30", json, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("baae0453-6d19-47b5-aa58-8bed5a572fc8", json, StringComparison.OrdinalIgnoreCase);
}
[Fact]
public void NonGlobalCaller_ExposesExactlyTheFourAllowedProperties()
{
using var doc = JsonDocument.Parse(SerializeFor(hasGlobalScope: false, MakeEntry()));
var keys = doc.RootElement[0].EnumerateObject().Select(p => p.Name).OrderBy(n => n).ToArray();
Assert.Equal(new[] { "firstSeenAt", "lastSeenAt", "sessionCount", "version" }, keys);
}
/// <summary>
/// SessionCount stays on purpose — it is a platform-wide total, and the route exists so any
/// tenant member can track Microsoft's IME rollout. Pinned so that "the counts are global"
/// stays a decision someone made rather than something that quietly drifts either way.
/// </summary>
[Fact]
public void NonGlobalCaller_KeepsVersionDatesAndGlobalSessionCount()
{
using var doc = JsonDocument.Parse(SerializeFor(hasGlobalScope: false, MakeEntry()));
var row = doc.RootElement[0];
Assert.Equal("1.103.101.0", row.GetProperty("version").GetString());
Assert.Equal(First, row.GetProperty("firstSeenAt").GetDateTime());
Assert.Equal(Last, row.GetProperty("lastSeenAt").GetDateTime());
Assert.Equal(3109, row.GetProperty("sessionCount").GetInt32());
}
// =========================================================================
// Global caller (Global Admin or Global Reader) — full archive
// =========================================================================
[Fact]
public void GlobalCaller_KeepsTenantAndSessionIds()
{
using var doc = JsonDocument.Parse(SerializeFor(hasGlobalScope: true, MakeEntry()));
var row = doc.RootElement[0];
Assert.Equal("57f34dd1-6a42-47f9-9bc0-b921fa6caa30", row.GetProperty("firstSeenTenantId").GetString());
Assert.Equal("baae0453-6d19-47b5-aa58-8bed5a572fc8", row.GetProperty("firstSeenSessionId").GetString());
}
[Fact]
public void GlobalCaller_ExposesTheFullEntry()
{
using var doc = JsonDocument.Parse(SerializeFor(hasGlobalScope: true, MakeEntry()));
var keys = doc.RootElement[0].EnumerateObject().Select(p => p.Name).OrderBy(n => n).ToArray();
Assert.Equal(
new[] { "firstSeenAt", "firstSeenSessionId", "firstSeenTenantId", "lastSeenAt", "sessionCount", "version" },
keys);
}
// =========================================================================
// Shape-preserving properties of the projection
// =========================================================================
[Fact]
public void BothScopes_EmitAJsonArray_EvenForASingleRow()
{
foreach (var scope in new[] { true, false })
{
using var doc = JsonDocument.Parse(SerializeFor(scope, MakeEntry()));
Assert.Equal(JsonValueKind.Array, doc.RootElement.ValueKind);
Assert.Equal(1, doc.RootElement.GetArrayLength());
}
}
[Fact]
public void BothScopes_EmptyArchive_SerializesAsEmptyArray()
{
foreach (var scope in new[] { true, false })
{
using var doc = JsonDocument.Parse(SerializeFor(scope));
Assert.Equal(JsonValueKind.Array, doc.RootElement.ValueKind);
Assert.Equal(0, doc.RootElement.GetArrayLength());
}
}
/// <summary>
/// The repository orders by FirstSeenAt descending; the projection must not reshuffle, or a
/// redacted caller would see a different sequence than a Global Admin for the same archive.
/// </summary>
[Fact]
public void NonGlobalCaller_PreservesRowOrderAndCount()
{
var newest = MakeEntry("1.103.101.0", First);
var older = MakeEntry("1.101.111.0", First.AddDays(-48));
var oldest = MakeEntry("1.99.101.0", First.AddDays(-131));
using var doc = JsonDocument.Parse(SerializeFor(false, newest, older, oldest));
Assert.Equal(3, doc.RootElement.GetArrayLength());
Assert.Equal("1.103.101.0", doc.RootElement[0].GetProperty("version").GetString());
Assert.Equal("1.101.111.0", doc.RootElement[1].GetProperty("version").GetString());
Assert.Equal("1.99.101.0", doc.RootElement[2].GetProperty("version").GetString());
}
/// <summary>
/// BuildResponsePayload returns `object`; the branch that keeps everything hands back the
/// entries themselves. System.Text.Json resolves an `object` declared type via the runtime
/// type, which is what keeps the wire format identical to the pre-extraction inline code —
/// assert it rather than trust it, since a regression here would silently emit "{}" rows.
/// </summary>
[Fact]
public void GlobalPayload_SerializedAsObject_StillEmitsPopulatedRows()
{
var payload = GetImeVersionHistoryFunction.BuildResponsePayload([MakeEntry()], hasGlobalScope: true);
var json = JsonSerializer.Serialize(payload, CamelCase);
Assert.Contains("1.103.101.0", json, StringComparison.Ordinal);
Assert.NotEqual("[{}]", json);
}
// =========================================================================
// Helpers
// =========================================================================
private static string SerializeFor(bool hasGlobalScope, params ImeVersionHistoryEntry[] versions) =>
JsonSerializer.Serialize(
GetImeVersionHistoryFunction.BuildResponsePayload(versions, hasGlobalScope),
CamelCase);
private static ImeVersionHistoryEntry MakeEntry(string version = "1.103.101.0", DateTime? firstSeenAt = null) =>
new()
{
Version = version,
FirstSeenAt = firstSeenAt ?? First,
FirstSeenSessionId = "baae0453-6d19-47b5-aa58-8bed5a572fc8",
FirstSeenTenantId = "57f34dd1-6a42-47f9-9bc0-b921fa6caa30",
LastSeenAt = Last,
SessionCount = 3109,
};
}
@@ -0,0 +1,165 @@
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the per-session API-latency projection (2026-07-28): the V2 agent emits cumulative
/// counters <c>net_total_latency_ms</c> / <c>net_total_requests</c> in every
/// <c>agent_metrics_snapshot</c>; ingest derives the session-wide average from the LAST
/// snapshot of the batch and persists it via <c>UpdateSessionNetworkLatencyAsync</c>
/// (last-write-wins on cumulative counters → replay-idempotent).
/// Wire-format note: the ingest body is parsed with Newtonsoft into
/// <c>Dictionary&lt;string, object&gt;</c>, so integer JSON numbers arrive as boxed
/// <c>long</c> — fixtures mirror that, not the agent's in-memory types.
/// </summary>
public class IngestApiLatencyProjectionTests
{
private static EnrollmentEvent Snapshot(object? totalLatencyMs, object? totalRequests)
{
var evt = new EnrollmentEvent
{
EventType = "agent_metrics_snapshot",
Data = new Dictionary<string, object>
{
["agent_cpu_percent"] = 3.62,
["net_requests"] = 50L,
["net_avg_latency_ms"] = 513.0,
},
};
if (totalLatencyMs != null) evt.Data["net_total_latency_ms"] = totalLatencyMs;
if (totalRequests != null) evt.Data["net_total_requests"] = totalRequests;
return evt;
}
[Fact]
public void CumulativeCounters_YieldSessionAverage()
{
var events = new List<EnrollmentEvent> { Snapshot(51_300L, 100L) };
Assert.True(EventIngestProcessor.TryComputeSessionApiLatency(events, out var avg, out var count));
Assert.Equal(513.0, avg);
Assert.Equal(100, count);
}
[Fact]
public void AverageIsRounded_ToOneDecimal()
{
var events = new List<EnrollmentEvent> { Snapshot(1_000L, 3L) };
Assert.True(EventIngestProcessor.TryComputeSessionApiLatency(events, out var avg, out _));
Assert.Equal(333.3, avg);
}
[Fact]
public void LastSnapshotOfBatch_Wins()
{
// Cumulative counters grow monotonically — the last snapshot carries the most
// complete session average.
var events = new List<EnrollmentEvent>
{
Snapshot(51_300L, 100L),
new EnrollmentEvent { EventType = "download_progress" },
Snapshot(120_000L, 200L),
};
Assert.True(EventIngestProcessor.TryComputeSessionApiLatency(events, out var avg, out var count));
Assert.Equal(600.0, avg);
Assert.Equal(200, count);
}
[Fact]
public void OldAgentSnapshot_WithoutTotalLatencyField_DoesNotMatch()
{
// Agents predating net_total_latency_ms emit net_total_requests only.
var events = new List<EnrollmentEvent> { Snapshot(totalLatencyMs: null, totalRequests: 100L) };
Assert.False(EventIngestProcessor.TryComputeSessionApiLatency(events, out _, out _));
}
[Fact]
public void SnapshotWithLatencyButMissingRequests_DoesNotMatch()
{
var events = new List<EnrollmentEvent> { Snapshot(51_300L, totalRequests: null) };
Assert.False(EventIngestProcessor.TryComputeSessionApiLatency(events, out _, out _));
}
[Fact]
public void ZeroRequests_DoesNotMatch()
{
// No requests yet (first snapshot before the first upload completes) — no division.
var events = new List<EnrollmentEvent> { Snapshot(0L, 0L) };
Assert.False(EventIngestProcessor.TryComputeSessionApiLatency(events, out _, out _));
}
[Fact]
public void MatchingLastSnapshot_SkipsOlderAgentSnapshotAfterIt()
{
// A field-less (old-schema) snapshot later in the batch must not shadow an earlier
// one that carries the counters — the predicate selects the last MATCHING snapshot.
var events = new List<EnrollmentEvent>
{
Snapshot(51_300L, 100L),
Snapshot(totalLatencyMs: null, totalRequests: 150L),
};
Assert.True(EventIngestProcessor.TryComputeSessionApiLatency(events, out var avg, out var count));
Assert.Equal(513.0, avg);
Assert.Equal(100, count);
}
[Fact]
public void BatchWithoutSnapshots_DoesNotMatch()
{
var events = new List<EnrollmentEvent>
{
new EnrollmentEvent { EventType = "esp_phase_changed" },
new EnrollmentEvent { EventType = "app_install_started" },
};
Assert.False(EventIngestProcessor.TryComputeSessionApiLatency(events, out _, out _));
}
// -------- TryGetDouble coercion ladder (mirrors PlatformMetricsService.GetDouble) --------
[Theory]
[InlineData(513.5)] // double — decimal JSON number
[InlineData(513)] // int — in-memory constructed events (tests, V1 paths)
[InlineData(513L)] // long — Newtonsoft wire form for integer JSON numbers
[InlineData(513f)] // float
public void TryGetDouble_CoercesBoxedNumericPrimitives(object boxed)
{
var data = new Dictionary<string, object> { ["k"] = boxed };
Assert.True(EventIngestProcessor.TryGetDouble(data, "k", out var value));
Assert.Equal(Convert.ToDouble(boxed), value);
}
[Fact]
public void TryGetDouble_ParsesStrings_InvariantCulture()
{
var data = new Dictionary<string, object> { ["k"] = "513.5" };
Assert.True(EventIngestProcessor.TryGetDouble(data, "k", out var value));
Assert.Equal(513.5, value);
}
[Theory]
[InlineData("not-a-number")]
[InlineData("")]
public void TryGetDouble_RejectsUnparseableStrings(string raw)
{
var data = new Dictionary<string, object> { ["k"] = raw };
Assert.False(EventIngestProcessor.TryGetDouble(data, "k", out _));
}
[Fact]
public void TryGetDouble_MissingKey_ReturnsFalse()
{
Assert.False(EventIngestProcessor.TryGetDouble(new Dictionary<string, object>(), "k", out _));
}
}
@@ -0,0 +1,124 @@
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the per-session connection-type projection (2026-07-28): the agent stamps
/// <c>connectionType</c> ("WiFi"/"Ethernet") into every <c>network_interface_info</c>
/// emission (initial collect + re-collect on change); ingest takes the LAST matching
/// event of the batch and persists it via <c>UpdateSessionConnectionTypeAsync</c>
/// (last-write-wins → a device that switches media mid-enrollment is relabeled).
/// The no-NIC payload ({"status":"no_active_interface"}) carries no connectionType
/// and must never match.
/// </summary>
public class IngestConnectionTypeProjectionTests
{
private static EnrollmentEvent NicInfo(string? connectionType)
{
var evt = new EnrollmentEvent
{
EventType = "network_interface_info",
Data = new Dictionary<string, object>
{
["adapterName"] = "Ethernet 2",
["linkSpeedMbps"] = 1000L,
},
};
if (connectionType != null) evt.Data["connectionType"] = connectionType;
return evt;
}
[Fact]
public void EthernetEmission_IsExtracted()
{
var events = new List<EnrollmentEvent> { NicInfo("Ethernet") };
Assert.True(EventIngestProcessor.TryExtractConnectionType(events, out var type));
Assert.Equal("Ethernet", type);
}
[Fact]
public void LastEmissionOfBatch_Wins()
{
// Media switch mid-batch (dock removed → WiFi) — the most recent state wins.
var events = new List<EnrollmentEvent>
{
NicInfo("Ethernet"),
new EnrollmentEvent { EventType = "download_progress" },
NicInfo("WiFi"),
};
Assert.True(EventIngestProcessor.TryExtractConnectionType(events, out var type));
Assert.Equal("WiFi", type);
}
[Fact]
public void NoActiveInterfacePayload_DoesNotMatch()
{
// The no-NIC emission carries only {"status":"no_active_interface"}.
var events = new List<EnrollmentEvent>
{
new EnrollmentEvent
{
EventType = "network_interface_info",
Data = new Dictionary<string, object> { ["status"] = "no_active_interface" },
},
};
Assert.False(EventIngestProcessor.TryExtractConnectionType(events, out _));
}
[Fact]
public void NoActiveInterfaceAfterValidEmission_DoesNotShadowIt()
{
// The predicate selects the last event that CARRIES connectionType.
var events = new List<EnrollmentEvent>
{
NicInfo("WiFi"),
new EnrollmentEvent
{
EventType = "network_interface_info",
Data = new Dictionary<string, object> { ["status"] = "no_active_interface" },
},
};
Assert.True(EventIngestProcessor.TryExtractConnectionType(events, out var type));
Assert.Equal("WiFi", type);
}
[Theory]
[InlineData("Cellular")]
[InlineData("wifi")] // agent emits exact casing — anything else is unexpected
[InlineData("")]
public void UnexpectedValues_AreDroppedDefensively(string raw)
{
var events = new List<EnrollmentEvent> { NicInfo(raw) };
Assert.False(EventIngestProcessor.TryExtractConnectionType(events, out _));
}
[Fact]
public void BatchWithoutNicEvents_DoesNotMatch()
{
var events = new List<EnrollmentEvent>
{
new EnrollmentEvent { EventType = "esp_phase_changed" },
new EnrollmentEvent { EventType = "agent_metrics_snapshot" },
};
Assert.False(EventIngestProcessor.TryExtractConnectionType(events, out _));
}
[Fact]
public void NullData_DoesNotMatch()
{
var events = new List<EnrollmentEvent>
{
new EnrollmentEvent { EventType = "network_interface_info", Data = null! },
};
Assert.False(EventIngestProcessor.TryExtractConnectionType(events, out _));
}
}
@@ -207,6 +207,34 @@ public class MetricsMathAppPayloadTests
Assert.Equal(0, app.GetProperty("skipped").GetInt32());
}
[Fact]
public void ImplausibleDurations_AreExcludedFromAverages_AndCountedAsUnmeasured()
{
// A duration beyond the agent's max observation lifetime is a back-stamped emission
// artifact (CompletedAt pinned at session end — production: 60 175 s app rows inside a
// 2 697 s session), not a slow install. It must fall out of the averages exactly like
// an unobserved start. Boundary: exactly 6 h is still plausible/measured.
var summaries = new List<AppInstallSummary>
{
new() { AppName = "App A", Status = "Succeeded", TerminalState = "Installed", DurationSeconds = 100 },
new() { AppName = "App A", Status = "Succeeded", TerminalState = "Installed", DurationSeconds = 200 },
new() { AppName = "App A", Status = "Succeeded", TerminalState = "Installed",
DurationSeconds = MetricsMath.MaxPlausibleInstallDurationSeconds }, // boundary: measured
new() { AppName = "App A", Status = "Succeeded", TerminalState = "Installed",
DurationSeconds = MetricsMath.MaxPlausibleInstallDurationSeconds + 1 }, // artifact: unmeasured
};
var root = Build(summaries);
Assert.Equal(1, root.GetProperty("totalUnmeasured").GetInt32());
var slowest = root.GetProperty("slowestApps");
Assert.Equal(1, slowest.GetArrayLength());
Assert.Equal(3, slowest[0].GetProperty("measuredInstalls").GetInt32());
// avg over {100, 200, 21600} — the 21601 artifact contributes nothing.
Assert.Equal(7300d, slowest[0].GetProperty("avgDurationSeconds").GetDouble());
Assert.Equal(MetricsMath.MaxPlausibleInstallDurationSeconds, slowest[0].GetProperty("maxDurationSeconds").GetInt32());
}
[Fact]
public void SlowestApps_GateOnMeasuredInstalls_NotOnSucceededCount()
{
@@ -45,6 +45,12 @@ public class SessionIndexProjectionTests
["IsSelfDeployingProfile"] = true,
// Device-validation path recorded at registration — shown on the session detail page.
["ValidatedBy"] = "AutopilotV1",
// Agent→backend HTTP latency projection (UpdateSessionNetworkLatencyAsync merges these).
["AvgApiLatencyMs"] = 513.5,
["ApiRequestCount"] = 100,
// Connection-type projection (UpdateSessionConnectionTypeAsync merges this) —
// search-filterable (connectionType= pushes an OData eq on the index).
["ConnectionType"] = "Ethernet",
};
var idx = TableStorageService.BuildSessionIndexEntity(session, IndexRowKey, StartedAt);
@@ -67,6 +73,9 @@ public class SessionIndexProjectionTests
Assert.True(idx.GetBoolean("ExcessiveEventsAutoActioned"));
Assert.True(idx.GetBoolean("IsSelfDeployingProfile"));
Assert.Equal("AutopilotV1", idx.GetString("ValidatedBy"));
Assert.Equal(513.5, idx.GetDouble("AvgApiLatencyMs"));
Assert.Equal(100, idx.GetInt32("ApiRequestCount"));
Assert.Equal("Ethernet", idx.GetString("ConnectionType"));
}
[Fact]
@@ -86,6 +95,8 @@ public class SessionIndexProjectionTests
Assert.False(idx.ContainsKey("FailureSource"));
Assert.False(idx.ContainsKey("StalledAt"));
Assert.False(idx.ContainsKey("ValidatedBy"));
Assert.False(idx.ContainsKey("AvgApiLatencyMs"));
Assert.False(idx.ContainsKey("ConnectionType"));
// Always-present counts/flags mirror the Sessions defaults.
Assert.Equal(0, idx.GetInt32("PlatformScriptCount"));
Assert.Equal(0, idx.GetInt32("RemediationScriptCount"));
@@ -40,6 +40,7 @@ public class StoreSessionReregistrationPreserveTests
["PlatformScriptCount"] = 3,
["RemediationScriptCount"] = 5,
["RebootCount"] = 7,
["ConnectionType"] = "Ethernet",
};
existing.ETag = new ETag("0xEXISTING");
@@ -71,6 +72,9 @@ public class StoreSessionReregistrationPreserveTests
Assert.Equal(7, harness.Written.GetInt32("RebootCount"));
// EventCount preservation is the established sibling behaviour — assert it as a sanity anchor.
Assert.Equal(42, harness.Written.GetInt32("EventCount"));
// Merge-mode field (UpdateSessionConnectionTypeAsync) — must survive the Replace so the
// value isn't lost in the window until the restarted agent re-emits network_interface_info.
Assert.Equal("Ethernet", harness.Written.GetString("ConnectionType"));
}
[Fact]
@@ -299,7 +299,7 @@ public class TimeAttributionPersistenceAndAggregationTests
{
var pairs = new List<(SessionSummary, SessionTimeBreakdown)>
{
(Session(TenantA, T0), Breakdown(flags: TimeAttributionFlags.BlockingSetUnknown)),
(Session(TenantA, T0), Breakdown(flags: TimeAttributionFlags.ClockSkewDropped)),
};
var aggregate = MaintenanceService.BuildTimeAttributionAggregates(
@@ -312,6 +312,32 @@ public class TimeAttributionPersistenceAndAggregationTests
Assert.Empty(aggregate.TopBlockingApps);
}
[Fact]
public void Aggregates_BlockingOnlyFlags_StayInSegmentStats()
{
// BlockingSetUnknown/Truncated limit per-app blocking EVIDENCE, not the measured spans.
// Gating on any flag starved the fleet aggregates in production (903/1000 breakdowns
// carried BlockingSetUnknown, 2026-07-27) — blocking-only flagged sessions must count
// as clean; only duration-critical flags exclude.
var pairs = new List<(SessionSummary, SessionTimeBreakdown)>
{
(Session(TenantA, T0), Breakdown(flags: TimeAttributionFlags.BlockingSetUnknown, espAppsSeconds: 100)),
(Session(TenantA, T0), Breakdown(flags: TimeAttributionFlags.BlockingSetTruncated, espAppsSeconds: 300)),
(Session(TenantA, T0), Breakdown(
flags: TimeAttributionFlags.BlockingSetUnknown | TimeAttributionFlags.PartialObservation,
espAppsSeconds: 99999)), // duration-critical bit set → still excluded
};
var aggregate = MaintenanceService.BuildTimeAttributionAggregates(
pairs, Array.Empty<SessionSummary>(), T0)
.Single(a => a.TenantId == TenantA);
Assert.Equal(2, aggregate.CleanSessionCount);
Assert.Equal(1, aggregate.FlaggedExcludedCount);
// Median of {100, 300} = 100 (nearest-rank, lower value — repo convention).
Assert.Equal(100, aggregate.SegmentStats.Single(s => s.SegmentKey == TimeAttributionSegments.EspApps).MedianSeconds);
}
[Fact]
public void Aggregates_MissingBreakdowns_AreCounted_NeverGuessed()
{
@@ -413,6 +413,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
{ "EnableTimezoneAutoSet", config.EnableTimezoneAutoSet },
{ "NtpServer", config.NtpServer },
{ "EnableImeMatchLog", config.EnableImeMatchLog },
{ "EnableGatherRuleDebugLog", config.EnableGatherRuleDebugLog },
{ "LogLevel", config.LogLevel },
{ "MaxBatchSize", config.MaxBatchSize },
{ "DiagnosticsBlobSasUrl", config.DiagnosticsBlobSasUrl },
@@ -514,6 +515,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
EnableTimezoneAutoSet = entity.GetBoolean("EnableTimezoneAutoSet"),
NtpServer = string.IsNullOrWhiteSpace(entity.GetString("NtpServer")) ? "time.windows.com" : entity.GetString("NtpServer"),
EnableImeMatchLog = entity.GetBoolean("EnableImeMatchLog"),
EnableGatherRuleDebugLog = entity.GetBoolean("EnableGatherRuleDebugLog"),
LogLevel = entity.GetString("LogLevel"),
MaxBatchSize = entity.GetInt32("MaxBatchSize"),
DiagnosticsBlobSasUrl = entity.GetString("DiagnosticsBlobSasUrl"),
@@ -148,6 +148,14 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
public Task UpdateSessionImeAgentVersionAsync(string tenantId, string sessionId, string version)
=> _storage.UpdateSessionImeAgentVersionAsync(tenantId, sessionId, version);
public Task UpdateSessionNetworkLatencyAsync(string tenantId, string sessionId,
double avgLatencyMs, int requestCount)
=> _storage.UpdateSessionNetworkLatencyAsync(tenantId, sessionId, avgLatencyMs, requestCount);
public Task UpdateSessionConnectionTypeAsync(string tenantId, string sessionId,
string connectionType)
=> _storage.UpdateSessionConnectionTypeAsync(tenantId, sessionId, connectionType);
public Task<List<SessionSummary>> GetSessionsWithEventCountAboveAsync(string tenantId, int threshold)
=> _storage.GetSessionsWithEventCountAboveAsync(tenantId, threshold);
@@ -270,12 +270,13 @@ namespace AutopilotMonitor.Functions.Functions.Config
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new AgentConfigResponse
{
ConfigVersion = 35, // MDM reboot-policy watcher (mdm_policy_reboot_required / esp_reboot_coalescing)
ConfigVersion = 36, // EnableGatherRuleDebugLog (local gather-rule evaluation trace)
UploadIntervalSeconds = 10,
SelfDestructOnComplete = tenantConfig.SelfDestructOnComplete ?? true,
KeepLogFile = tenantConfig.KeepLogFile ?? false,
EnableGeoLocation = tenantConfig.EnableGeoLocation ?? true,
EnableImeMatchLog = tenantConfig.EnableImeMatchLog ?? false,
EnableGatherRuleDebugLog = tenantConfig.EnableGatherRuleDebugLog ?? false,
MaxAuthFailures = tenantConfig.MaxAuthFailures ?? 5,
AuthFailureTimeoutMinutes = tenantConfig.AuthFailureTimeoutMinutes ?? 0,
LogLevel = tenantConfig.LogLevel ?? "Info",
@@ -0,0 +1,216 @@
using System;
using System.IO;
using System.Net;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using AutopilotMonitor.Functions.Services;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Functions.Infrastructure
{
/// <summary>
/// Receives Azure Monitor alert notifications (Common Alert Schema) from an Action Group
/// webhook receiver and re-emits them as ops events, so platform alerts flow through the
/// same OpsEventService → OpsAlertDispatchService fan-out (Telegram/Teams/Slack providers +
/// the portal ops-events list) as every internally raised operational event.
///
/// The delivery path is deliberately independent of the portal: Azure Monitor → this
/// function → providers. A portal (SWA) outage cannot suppress these notifications — the
/// action group's e-mail receiver additionally stays as the backend-independent fallback,
/// because a self-hosted channel cannot alert on its own outage.
///
/// Security: anonymous route (Action Group webhook receivers can only call a bare URI — no
/// custom headers), gated by a shared secret in the query string compared in constant time
/// against the OpsAlertWebhookSecret app setting. Fail-closed: while the setting is unset,
/// every request is rejected. Dispatch to providers only happens if an enabled ops alert
/// rule for EventType "AzureMonitorAlert" matches the mapped severity — the event itself is
/// always stored.
/// </summary>
public class AzureMonitorAlertWebhookFunction
{
internal const string SecretSettingName = "OpsAlertWebhookSecret";
// Common Alert Schema payloads for log-search alerts carry the search query, link URLs
// and dimension sets — a few KB in practice. 64 KB is a generous shape gate, not a throttle.
internal const int MaxContentLength = 64 * 1024;
private readonly OpsEventService _opsEventService;
private readonly ILogger<AzureMonitorAlertWebhookFunction> _logger;
public AzureMonitorAlertWebhookFunction(
OpsEventService opsEventService,
ILogger<AzureMonitorAlertWebhookFunction> logger)
{
_opsEventService = opsEventService;
_logger = logger;
}
[Function("AzureMonitorAlertWebhook")]
public async Task<HttpResponseData> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "ops/alert-webhook")] HttpRequestData req)
{
var configuredSecret = Environment.GetEnvironmentVariable(SecretSettingName);
var providedSecret = System.Web.HttpUtility.ParseQueryString(req.Url.Query ?? string.Empty)["secret"];
if (!SecretMatches(providedSecret, configuredSecret))
{
// Deliberately no detail in the response; the warning makes a misconfigured
// (unset) secret operator-visible in the logs.
_logger.LogWarning(
"Azure Monitor alert webhook rejected: {Reason}",
string.IsNullOrEmpty(configuredSecret) ? "secret not configured (fail-closed)" : "secret mismatch");
return req.CreateResponse(HttpStatusCode.Unauthorized);
}
// Bounded read instead of a Body.Length check: with ASP.NET Core integration the
// request body is a forward-only stream whose Length property throws
// NotSupportedException. Reading at most cap+1 chars detects oversize without ever
// needing to know the total size. (Char-based cap ≈ byte cap for a shape gate.)
string body;
using (var reader = new StreamReader(req.Body, Encoding.UTF8))
{
var buffer = new char[MaxContentLength + 1];
var read = await reader.ReadBlockAsync(buffer, 0, buffer.Length);
if (read > MaxContentLength)
{
_logger.LogWarning("Azure Monitor alert webhook rejected: payload exceeds {Cap} char cap", MaxContentLength);
return req.CreateResponse(HttpStatusCode.BadRequest);
}
body = new string(buffer, 0, read);
}
var alert = TryParse(body);
if (alert == null)
{
_logger.LogWarning("Azure Monitor alert webhook rejected: body is not a Common Alert Schema payload");
return req.CreateResponse(HttpStatusCode.BadRequest);
}
await _opsEventService.RecordAzureMonitorAlertAsync(
alert.AlertRule,
MapSeverity(alert.Severity),
alert.MonitorCondition,
alert.Description,
alert.Severity,
alert.MonitoringService,
alert.TargetResource,
alert.MetricValue);
_logger.LogInformation(
"Azure Monitor alert '{AlertRule}' {Condition} ({Severity}) recorded as ops event",
alert.AlertRule, alert.MonitorCondition, alert.Severity);
return req.CreateResponse(HttpStatusCode.OK);
}
/// <summary>
/// Constant-time secret comparison. Fail-closed: an unset/empty configured secret never
/// matches anything, so the endpoint stays dark until the app setting is provisioned.
/// </summary>
internal static bool SecretMatches(string? provided, string? configured)
{
if (string.IsNullOrEmpty(configured) || string.IsNullOrEmpty(provided))
return false;
return CryptographicOperations.FixedTimeEquals(
Encoding.UTF8.GetBytes(provided),
Encoding.UTF8.GetBytes(configured));
}
/// <summary>
/// Maps Azure Monitor severities (Sev0 = most severe … Sev4 = verbose) onto the ops event
/// severity scale. Unknown values fall back to Info so a schema drift can never escalate.
/// </summary>
internal static string MapSeverity(string? azureSeverity) => azureSeverity switch
{
"Sev0" => Shared.DataAccess.OpsEventSeverity.Critical,
"Sev1" => Shared.DataAccess.OpsEventSeverity.Error,
"Sev2" => Shared.DataAccess.OpsEventSeverity.Warning,
_ => Shared.DataAccess.OpsEventSeverity.Info,
};
/// <summary>
/// Parsed subset of the Common Alert Schema. Only <see cref="AlertRule"/> is mandatory;
/// everything else is best-effort so a partial payload still produces a usable ops event.
/// </summary>
internal sealed record ParsedAlert(
string AlertRule,
string? Severity,
string MonitorCondition,
string? Description,
string? MonitoringService,
string? TargetResource,
double? MetricValue);
/// <summary>
/// Defensively extracts data.essentials (+ the first condition's metricValue) from a
/// Common Alert Schema payload. Returns null when the body is not JSON or carries no
/// alert rule name — the two states in which no meaningful ops event can be formed.
/// </summary>
internal static ParsedAlert? TryParse(string body)
{
if (string.IsNullOrWhiteSpace(body))
return null;
try
{
using var doc = JsonDocument.Parse(body);
if (!doc.RootElement.TryGetProperty("data", out var data) ||
!data.TryGetProperty("essentials", out var essentials))
return null;
var alertRule = GetString(essentials, "alertRule");
if (string.IsNullOrWhiteSpace(alertRule))
return null;
var targetResource = GetString(essentials, "targetResourceName");
if (string.IsNullOrEmpty(targetResource) &&
essentials.TryGetProperty("alertTargetIDs", out var targets) &&
targets.ValueKind == JsonValueKind.Array && targets.GetArrayLength() > 0 &&
targets[0].ValueKind == JsonValueKind.String)
{
// Fall back to the resource name segment of the first target resource ID.
var id = targets[0].GetString() ?? string.Empty;
targetResource = id[(id.LastIndexOf('/') + 1)..];
}
// Log-search alerts carry the observed value under alertContext.condition.allOf[0].
double? metricValue = null;
if (data.TryGetProperty("alertContext", out var ctx) &&
ctx.ValueKind == JsonValueKind.Object &&
ctx.TryGetProperty("condition", out var condition) &&
condition.ValueKind == JsonValueKind.Object &&
condition.TryGetProperty("allOf", out var allOf) &&
allOf.ValueKind == JsonValueKind.Array && allOf.GetArrayLength() > 0 &&
allOf[0].ValueKind == JsonValueKind.Object &&
allOf[0].TryGetProperty("metricValue", out var mv) &&
mv.ValueKind == JsonValueKind.Number)
{
metricValue = mv.GetDouble();
}
return new ParsedAlert(
alertRule!,
GetString(essentials, "severity"),
GetString(essentials, "monitorCondition") ?? "Fired",
GetString(essentials, "description"),
GetString(essentials, "monitoringService"),
targetResource,
metricValue);
}
catch (JsonException)
{
return null;
}
}
private static string? GetString(JsonElement element, string property)
=> element.TryGetProperty(property, out var value) && value.ValueKind == JsonValueKind.String
? value.GetString()
: null;
}
}
@@ -153,6 +153,18 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
.Where(s => appsBySession.ContainsKey(s.SessionId))
.Count(s => appsBySession[s.SessionId].Any(a => a.DoDownloadMode >= 0));
// Agent→backend API latency: weight each session's stored average by its request
// count so the location figure equals Σ(latency·requests)/Σ(requests) — the exact
// whole-population average, not an average of session averages. Sessions from
// agents predating the feature carry no data and drop out.
var latencySessions = group
.Where(s => s.AvgApiLatencyMs is > 0 && s.ApiRequestCount is > 0)
.ToList();
var latencyRequestSum = latencySessions.Sum(s => (double)s.ApiRequestCount!.Value);
var avgApiLatency = latencyRequestSum > 0
? latencySessions.Sum(s => s.AvgApiLatencyMs!.Value * s.ApiRequestCount!.Value) / latencyRequestSum
: 0;
locations.Add(new LocationMetrics
{
LocationKey = group.Key,
@@ -171,6 +183,8 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
MinutesPerApp = Math.Round(minutesPerApp, 2),
AvgThroughputBytesPerSec = Math.Round(avgThroughput, 0),
TotalDownloadBytes = totalBytes,
AvgApiLatencyMs = Math.Round(avgApiLatency, 0),
ApiLatencySessionCount = latencySessions.Count,
// Delivery Optimization
DoSessionCount = doSessionCount,
AvgDoPercentPeerCaching = Math.Round(doAgg.PercentPeerCaching, 1),
@@ -202,6 +216,17 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
.Select(l => l.AvgThroughputBytesPerSec).ToList();
var globalAvgThroughput = allThroughputs.Count > 0 ? allThroughputs.Average() : 0;
// Global API latency: request-weighted over ALL geo sessions (not location averages),
// so big sites weigh in proportionally — same Σ(latency·requests)/Σ(requests) math as
// the per-location figure.
var globalLatencySessions = geoSessions
.Where(s => s.AvgApiLatencyMs is > 0 && s.ApiRequestCount is > 0)
.ToList();
var globalLatencyRequestSum = globalLatencySessions.Sum(s => (double)s.ApiRequestCount!.Value);
var globalAvgApiLatency = globalLatencyRequestSum > 0
? globalLatencySessions.Sum(s => s.AvgApiLatencyMs!.Value * s.ApiRequestCount!.Value) / globalLatencyRequestSum
: 0;
// Global DO metrics (weighted by bytes, not simple average)
var globalDoPeers = locations.Sum(l => l.TotalDoBytesFromPeers);
var globalDoHttp = locations.Sum(l => l.TotalDoBytesFromHttp);
@@ -219,6 +244,10 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
loc.ThroughputVsGlobalPct = globalAvgThroughput > 0
? Math.Round((loc.AvgThroughputBytesPerSec - globalAvgThroughput) / globalAvgThroughput * 100, 1) : 0;
// API latency vs global (positive = slower/farther from the backend region)
loc.ApiLatencyVsGlobalPct = globalAvgApiLatency > 0 && loc.AvgApiLatencyMs > 0
? Math.Round((loc.AvgApiLatencyMs - globalAvgApiLatency) / globalAvgApiLatency * 100, 1) : 0;
// AppLoadScore: normalize minutesPerApp to global median = 100
loc.AppLoadScore = globalMedianMinutesPerApp > 0
? Math.Round(loc.MinutesPerApp / globalMedianMinutesPerApp * 100, 0) : 0;
@@ -243,6 +272,7 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
AvgMinutesPerApp = Math.Round(globalAvgMinutesPerApp, 2),
AvgThroughputBytesPerSec = Math.Round(globalAvgThroughput, 0),
StdDevDurationMinutes = Math.Round(globalStdDev, 1),
AvgApiLatencyMs = Math.Round(globalAvgApiLatency, 0),
AvgDoPercentPeerCaching = Math.Round(globalDoPct, 1),
TotalDoBytesFromPeers = globalDoPeers,
TotalDoBytesFromHttp = globalDoHttp
@@ -1,6 +1,7 @@
using System.Net;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Shared.DataAccess;
using AutopilotMonitor.Shared.Models;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
@@ -34,24 +35,7 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
var versions = await _sessionRepo.GetImeVersionHistoryAsync();
var response = req.CreateResponse(HttpStatusCode.OK);
if (requestCtx.HasGlobalScope)
{
await response.WriteAsJsonAsync(versions);
}
else
{
// Strip sensitive cross-tenant data for non-Global Admins
var redacted = versions.Select(v => new
{
v.Version,
v.FirstSeenAt,
v.LastSeenAt,
v.SessionCount
});
await response.WriteAsJsonAsync(redacted);
}
await response.WriteAsJsonAsync(BuildResponsePayload(versions, requestCtx.HasGlobalScope));
return response;
}
catch (Exception ex)
@@ -62,5 +46,35 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
return errorResponse;
}
}
/// <summary>
/// Projects the archive for the caller's scope.
///
/// Extracted from <see cref="Run"/> so the cross-tenant redaction boundary is
/// directly testable: this suite has no HTTP-level harness, so an inline projection
/// is unreachable from a test — and an unguarded redaction is exactly what a later
/// refactor opens silently.
///
/// FirstSeenTenantId and FirstSeenSessionId name ANOTHER tenant's device and are
/// dropped for non-global callers. Version, dates and SessionCount deliberately stay:
/// the archive is a platform-wide view of Microsoft's IME rollout, and the route is
/// MemberRead precisely so any tenant member can read that view.
///
/// Returned as <see cref="object"/> on purpose — System.Text.Json serializes an
/// `object` declared type using the RUNTIME type, so both branches keep the exact
/// wire format they had while the projection was inline.
/// </summary>
public static object BuildResponsePayload(
IEnumerable<ImeVersionHistoryEntry> versions, bool hasGlobalScope)
{
if (hasGlobalScope)
{
return versions.ToList();
}
return versions
.Select(v => new { v.Version, v.FirstSeenAt, v.LastSeenAt, v.SessionCount })
.ToList();
}
}
}
@@ -130,6 +130,8 @@ namespace AutopilotMonitor.Functions.Functions.Raw
filter.IsSelfDeployingProfile = sdp;
if (int.TryParse(query["rebootCountMin"], out var rcMin)) filter.RebootCountMin = rcMin;
if (int.TryParse(query["rebootCountMax"], out var rcMax)) filter.RebootCountMax = rcMax;
var connectionType = query["connectionType"];
if (!string.IsNullOrEmpty(connectionType)) filter.ConnectionType = connectionType;
var callerTenantId = TenantHelper.GetTenantId(req);
@@ -140,6 +140,7 @@ public class SearchSessionsFunction
ImeAgentVersion = query["imeAgentVersion"],
AgentVersionPrefix = query["agentVersionPrefix"],
ImeAgentVersionPrefix = query["imeAgentVersionPrefix"],
ConnectionType = query["connectionType"],
// Limit field is a no-op in the paged path — pagination drives count.
};
@@ -210,6 +211,9 @@ public class SearchSessionsFunction
if (fields.Contains("imeAgentVersion")) dict["imeAgentVersion"] = s.ImeAgentVersion;
if (fields.Contains("geoCountry")) dict["geoCountry"] = s.GeoCountry;
if (fields.Contains("rebootCount")) dict["rebootCount"] = s.RebootCount;
if (fields.Contains("avgApiLatencyMs")) dict["avgApiLatencyMs"] = s.AvgApiLatencyMs;
if (fields.Contains("apiRequestCount")) dict["apiRequestCount"] = s.ApiRequestCount;
if (fields.Contains("connectionType")) dict["connectionType"] = s.ConnectionType;
if (dict.Count == 0)
{
@@ -32,15 +32,29 @@ public static class MetricsMath
public static bool IsSkipTerminalState(AppInstallSummary s)
=> s.TerminalState == "Skipped" || s.TerminalState == "Postponed";
/// <summary>
/// Upper plausibility bound for an observed install duration: 6 h, the agent's default
/// max observation lifetime (AgentMaxLifetimeMinutes = 360) — nothing longer can have
/// been continuously watched. Rows above it are emission artifacts, not slow installs:
/// a batch emission at session end pins CompletedAt far past the real finish (verified
/// in production 2026-07-27: app rows carrying 60 175 s in a session whose own wall
/// clock was 2 697 s; 490 rows &gt; 6 h overall, worst 117 days — dominating the
/// slowest-apps ranking with six-figure averages).
/// </summary>
public const int MaxPlausibleInstallDurationSeconds = 6 * 3600;
/// <summary>
/// True for a succeeded row whose install duration was actually observed. Zero /
/// absent duration on a non-skip succeeded row means the START was never observed
/// (agent attach window — audit finding, §0.5 of the insights spec): the duration is
/// UNKNOWN, not zero — averaging it in would understate real install times (measured
/// −18 % in production). Duration statistics must only read rows where this is true.
/// (agent attach window — audit finding, §0.5 of the insights spec); a duration above
/// <see cref="MaxPlausibleInstallDurationSeconds"/> means the END was never observed
/// (completion back-stamped at session end). Either way the duration is UNKNOWN —
/// averaging zeros understates (measured −18 % in production), averaging back-stamps
/// inflates. Duration statistics must only read rows where this is true; everything
/// else is reported as "unmeasured".
/// </summary>
public static bool HasMeasuredDuration(AppInstallSummary s)
=> s.DurationSeconds > 0;
=> s.DurationSeconds > 0 && s.DurationSeconds <= MaxPlausibleInstallDurationSeconds;
/// <summary>
/// Builds the complete app-metrics response object from a (pre-time-filtered) set of app
@@ -53,6 +53,7 @@ namespace AutopilotMonitor.Functions.Pagination
new KeyValuePair<string, string?>("startedBefore", filter.StartedBefore?.ToUniversalTime().ToString("o", CultureInfo.InvariantCulture)),
new KeyValuePair<string, string?>("rebootCountMin", filter.RebootCountMin?.ToString(CultureInfo.InvariantCulture)),
new KeyValuePair<string, string?>("rebootCountMax", filter.RebootCountMax?.ToString(CultureInfo.InvariantCulture)),
new KeyValuePair<string, string?>("connectionType", filter.ConnectionType),
};
if (filter.DeviceProperties != null)
{
@@ -170,6 +170,11 @@ public static class EndpointAccessPolicyCatalog
// ticket is the sole authority — authz happened at mint time (diagnostics/download-ticket,
// MemberRead). See DiagnosticsTicketDownloadFunction.
new("GET", "diagnostics/download", EndpointPolicy.PublicAnonymous),
// Azure Monitor action-group webhook (Common Alert Schema → ops event). Action groups
// cannot send custom headers, so the shared secret travels as ?secret= and is compared
// in constant time against the OpsAlertWebhookSecret app setting (fail-closed while
// unset). See AzureMonitorAlertWebhookFunction.
new("POST", "ops/alert-webhook", EndpointPolicy.PublicAnonymous),
// ── DeviceOrBootstrapAuth ───────────────────────────────────────
new("POST", "agent/register-session", EndpointPolicy.DeviceOrBootstrapAuth),
@@ -143,6 +143,22 @@ namespace AutopilotMonitor.Functions.Services
}
}
if (TryComputeSessionApiLatency(request.Events, out var avgLatencyMs, out var apiRequestCount))
{
_ = _sessionRepo.UpdateSessionNetworkLatencyAsync(
request.TenantId, request.SessionId, avgLatencyMs, apiRequestCount)
.ContinueWith(t => _logger.LogWarning(t.Exception?.InnerException,
"Fire-and-forget UpdateSessionNetworkLatencyAsync failed"), TaskContinuationOptions.OnlyOnFaulted);
}
if (TryExtractConnectionType(request.Events, out var connectionType))
{
_ = _sessionRepo.UpdateSessionConnectionTypeAsync(
request.TenantId, request.SessionId, connectionType)
.ContinueWith(t => _logger.LogWarning(t.Exception?.InnerException,
"Fire-and-forget UpdateSessionConnectionTypeAsync failed"), TaskContinuationOptions.OnlyOnFaulted);
}
var classification = ClassifyEvents(storedEvents);
foreach (var summary in classification.AppInstallUpdates.Values)
@@ -395,6 +411,80 @@ namespace AutopilotMonitor.Functions.Services
};
}
/// <summary>
/// Session-wide API latency projection: the agent emits cumulative counters
/// (net_total_latency_ms / net_total_requests) in every agent_metrics_snapshot, so
/// the LAST snapshot of the batch carries the whole-session average up to now and a
/// plain overwrite is idempotent against replays. Agents predating the field simply
/// never match (net_total_latency_ms absent) — no fallback, fleet turns over
/// per-enrollment. Exposed as internal for unit testing.
/// </summary>
internal static bool TryComputeSessionApiLatency(
List<EnrollmentEvent> events, out double avgLatencyMs, out int requestCount)
{
avgLatencyMs = 0;
requestCount = 0;
var snapshot = events.LastOrDefault(e =>
e.EventType == Shared.Constants.EventTypes.AgentMetricsSnapshot &&
e.Data?.ContainsKey("net_total_latency_ms") == true);
if (snapshot?.Data == null ||
!TryGetDouble(snapshot.Data, "net_total_latency_ms", out var totalLatencyMs) ||
!TryGetDouble(snapshot.Data, "net_total_requests", out var totalRequests) ||
totalRequests <= 0)
{
return false;
}
avgLatencyMs = Math.Round(totalLatencyMs / totalRequests, 1);
requestCount = (int)totalRequests;
return true;
}
/// <summary>
/// Connection-type projection: the agent stamps connectionType ("WiFi"/"Ethernet")
/// into every network_interface_info emission (initial collect + re-collect), so the
/// LAST event of the batch carries the current media and a plain overwrite is
/// idempotent against replays. The no-NIC payload ({"status":"no_active_interface"})
/// carries no connectionType and never matches. Values outside WiFi/Ethernet are
/// dropped defensively. Exposed as internal for unit testing.
/// </summary>
internal static bool TryExtractConnectionType(
List<EnrollmentEvent> events, out string connectionType)
{
connectionType = string.Empty;
var nicEvent = events.LastOrDefault(e =>
e.EventType == Shared.Constants.EventTypes.NetworkInterfaceInfo &&
e.Data?.ContainsKey("connectionType") == true);
var raw = nicEvent?.Data?["connectionType"]?.ToString();
if (raw != "WiFi" && raw != "Ethernet")
return false;
connectionType = raw;
return true;
}
/// <summary>
/// Numeric reader for agent event Data values, which arrive as boxed Newtonsoft
/// primitives (integer → long, decimal → double) — same coercion ladder as
/// PlatformMetricsService.GetDouble (the proven parser for these snapshot fields),
/// plus invariant-culture string parsing. Exposed as internal for unit testing.
/// </summary>
internal static bool TryGetDouble(Dictionary<string, object> data, string key, out double value)
{
value = 0;
if (!data.TryGetValue(key, out var raw))
return false;
switch (raw)
{
case double d: value = d; return true;
case int i: value = i; return true;
case long l: value = l; return true;
case float f: value = f; return true;
}
return double.TryParse(raw?.ToString(),
System.Globalization.NumberStyles.Float,
System.Globalization.CultureInfo.InvariantCulture, out value);
}
/// <summary>
/// Stamps authoritative server-side fields onto all events before storage.
/// TenantId and SessionId always come from the validated request metadata,
@@ -394,8 +394,11 @@ namespace AutopilotMonitor.Functions.Services
/// <summary>
/// Pure aggregation core behind <see cref="SweepTimeAttributionAsync"/> — internal static
/// so the bucketing/gating contract is pinned by unit tests. Per (tenant × class × date)
/// bucket plus mirrored "global" rows: only CLEAN breakdowns (QualityFlags == None) form
/// the statistics; flagged and missing ones are counted, never silently dropped (rule 7).
/// bucket plus mirrored "global" rows: only breakdowns without DURATION-critical flags
/// (<see cref="TimeAttributionFlagQuality.DurationCriticalFlags"/>) form the statistics —
/// blocking-set-only flags stay in (their spans are sound; unknown blocking simply
/// contributes no per-app intervals); excluded and missing ones are counted, never
/// silently dropped (rule 7).
/// Rows are written even below the ≥20 UI gate — the UI needs the n (rule 4). Segment
/// stats always carry the five canonical segments + unattributed (a session without a
/// span of a segment contributes 0 — the honest "per enrollment of this class" answer).
@@ -427,7 +430,7 @@ namespace AutopilotMonitor.Functions.Services
foreach (var tenantKey in new[] { session.TenantId, "global" })
{
var key = (tenantKey, date, cls);
if (breakdown.QualityFlags != TimeAttributionFlags.None)
if (TimeAttributionFlagQuality.ExcludesFromFleetStats(breakdown.QualityFlags))
{
Bump(flaggedCounts, key);
// Flagged sessions still materialize the bucket so a day whose sessions
@@ -594,6 +594,24 @@ namespace AutopilotMonitor.Functions.Services
null, "System.SlaEvaluation",
new { tenantsEvaluated, breachesDetected, notificationsSent, durationMs });
// ── Platform (Azure Monitor) ───────────────────────────────────────────
/// <summary>
/// Records an Azure Monitor alert notification delivered via the ops alert webhook
/// (AzureMonitorAlertWebhookFunction). severity is already mapped onto the ops scale;
/// azureSeverity keeps the original Sev0–Sev4 value for the details payload.
/// </summary>
public Task RecordAzureMonitorAlertAsync(string alertRule, string severity, string monitorCondition,
string? description, string? azureSeverity, string? monitoringService, string? targetResource,
double? metricValue)
{
var suffix = string.IsNullOrWhiteSpace(description) ? string.Empty : $": {description}";
return WriteAsync(OpsEventCategory.Platform, "AzureMonitorAlert", severity,
$"Azure Monitor alert '{alertRule}' {monitorCondition}{suffix}",
null, "System.AzureMonitor",
new { alertRule, monitorCondition, azureSeverity, monitoringService, targetResource, metricValue });
}
// ── Core write method ──────────────────────────────────────────────────
private async Task WriteAsync(string category, string eventType, string severity,
@@ -213,6 +213,15 @@ namespace AutopilotMonitor.Functions.Services
var lastSnapshot = snapshots.Last();
// Prefer the exact session average from the cumulative counters (agents ≥ 2026-07-28
// emit net_total_latency_ms); fall back to the unweighted mean of the per-interval
// delta averages for older sessions.
var totalLatencyMs = GetDouble(lastSnapshot, "net_total_latency_ms");
var totalLatencyRequests = GetDouble(lastSnapshot, "net_total_requests");
var sessionAvgLatency = totalLatencyMs > 0 && totalLatencyRequests > 0
? totalLatencyMs / totalLatencyRequests
: (latValues.Count > 0 ? latValues.Average() : 0);
// Resolve agent version: prefer from snapshot, fallback to session
var agentVersion = snapshots
.Select(s => GetString(s, "agent_version"))
@@ -239,7 +248,7 @@ namespace AutopilotMonitor.Functions.Services
AvgWorkingSet = wsValues.Count > 0 ? wsValues.Average() : 0,
MaxWorkingSet = wsValues.Count > 0 ? wsValues.Max() : 0,
AvgPrivateBytes = pbValues.Count > 0 ? pbValues.Average() : 0,
AvgLatency = latValues.Count > 0 ? latValues.Average() : 0,
AvgLatency = sessionAvgLatency,
AvgSpoolDepth = spoolValues.Count > 0 ? spoolValues.Average() : 0,
MaxSpoolDepth = spoolValues.Count > 0 ? spoolValues.Max() : 0,
PeakSpoolDepth = peakValues.Count > 0 ? peakValues.Last() : 0,
@@ -654,6 +654,11 @@ namespace AutopilotMonitor.Functions.Services
parts.Add($"RebootCount ge {filter.RebootCountMin.Value}");
if (filter.RebootCountMax.HasValue)
parts.Add($"RebootCount le {filter.RebootCountMax.Value}");
// ConnectionType exact match pushed to OData. Legacy index rows that predate
// the projected column lack the property and are excluded by the eq (they
// carry no connection data, which is the intended result).
if (!string.IsNullOrEmpty(filter.ConnectionType))
parts.Add($"ConnectionType eq '{ODataSanitizer.EscapeValue(filter.ConnectionType)}'");
return parts.Count == 0 ? null : string.Join(" and ", parts);
}
@@ -670,6 +675,19 @@ namespace AutopilotMonitor.Functions.Services
return true;
}
/// <summary>
/// Client-side ConnectionType predicate — same backstop role as
/// <see cref="MatchesRebootCountBounds"/>: applied on the search paths that do NOT
/// push the filter to OData (device-snapshot batch-get, legacy unpaged scan) and
/// redundantly on the OData scan paths. Sessions without the projected column
/// (null) never match a set filter.
/// </summary>
internal static bool MatchesConnectionType(SessionSummary session, SessionSearchFilter filter)
{
if (string.IsNullOrEmpty(filter.ConnectionType)) return true;
return string.Equals(session.ConnectionType, filter.ConnectionType, StringComparison.OrdinalIgnoreCase);
}
private static bool MatchesScanClientFilters(SessionSummary session, SessionSearchFilter filter)
{
if (!string.IsNullOrEmpty(filter.SerialNumber) &&
@@ -683,9 +701,10 @@ namespace AutopilotMonitor.Functions.Services
return false;
if (filter.StartedAfter.HasValue && session.StartedAt < filter.StartedAfter.Value) return false;
if (filter.StartedBefore.HasValue && session.StartedAt > filter.StartedBefore.Value) return false;
// RebootCount is also pushed to OData in BuildSearchScanFilter; this is a defensive
// backstop so the bound holds even if that push-down is ever weakened.
// RebootCount / ConnectionType are also pushed to OData in BuildSearchScanFilter;
// these are defensive backstops so the bound holds even if that push-down is ever weakened.
if (!MatchesRebootCountBounds(session, filter)) return false;
if (!MatchesConnectionType(session, filter)) return false;
// AgentVersion / ImeAgentVersion (exact + prefix) are pushed to OData
// in BuildSearchScanFilter — the server has already filtered them out
// before we see the page. No client-side check needed.
@@ -951,9 +970,11 @@ namespace AutopilotMonitor.Functions.Services
&& (s.ImeAgentVersion == null
|| !s.ImeAgentVersion.StartsWith(filter.ImeAgentVersionPrefix!, StringComparison.OrdinalIgnoreCase)))
return false;
// Device-snapshot path has no RebootCount OData push-down — enforce it here so a
// deviceProperties + rebootCountMin query can't return sub-threshold sessions.
// Device-snapshot path has no RebootCount/ConnectionType OData push-down —
// enforce them here so a deviceProperties + rebootCountMin/connectionType
// query can't return non-matching sessions.
if (!MatchesRebootCountBounds(s, filter)) return false;
if (!MatchesConnectionType(s, filter)) return false;
return true;
}).ToList();
}
@@ -586,7 +586,8 @@ namespace AutopilotMonitor.Functions.Services
internal static readonly string[] GeoMetricsSessionProjection =
{
"PartitionKey", "RowKey", "StartedAt", "CompletedAt", "Status", "DurationSeconds",
"IsPreProvisioned", "ResumedAt", "GeoCountry", "GeoRegion", "GeoCity", "GeoLoc"
"IsPreProvisioned", "ResumedAt", "GeoCountry", "GeoRegion", "GeoCity", "GeoLoc",
"AvgApiLatencyMs", "ApiRequestCount"
};
/// <summary>
@@ -538,12 +538,14 @@ namespace AutopilotMonitor.Functions.Services
/// <summary>
/// Columns MetricsMath.BuildAppMetricsPayload consumes (slowest/failing ranking + DO rollup):
/// grouping key, status/duration/bytes, failure code and every counter DoAggregator sums.
/// grouping key, status + terminal state (PR0 skip classification — dropping TerminalState
/// silently zeroed totalSkipped in production, found 2026-07-27), duration/bytes, failure
/// code and every counter DoAggregator sums.
/// internal so AppsProjectionEquivalenceTests derives its keep-set from this array.
/// </summary>
internal static readonly string[] AppMetricsProjection =
{
"PartitionKey", "RowKey", "AppName", "Status", "StartedAt",
"PartitionKey", "RowKey", "AppName", "Status", "TerminalState", "StartedAt",
"DurationSeconds", "DownloadBytes", "FailureCode",
"DoDownloadMode", "DoTotalBytesDownloaded", "DoBytesFromPeers", "DoBytesFromHttp",
"DoBytesFromLanPeers", "DoBytesFromGroupPeers", "DoBytesFromInternetPeers",
@@ -215,6 +215,22 @@ namespace AutopilotMonitor.Functions.Services
if (stalledAt.HasValue)
indexEntity["StalledAt"] = EnsureUtc(stalledAt.Value);
// Agent→backend HTTP latency projection (UpdateSessionNetworkLatencyAsync merges
// these) — must be in this superset or a StartedAt-shift full upsert drops them.
var avgApiLatencyMs = sessionEntity.GetDouble("AvgApiLatencyMs");
if (avgApiLatencyMs.HasValue)
indexEntity["AvgApiLatencyMs"] = avgApiLatencyMs.Value;
var apiRequestCount = sessionEntity.GetInt32("ApiRequestCount");
if (apiRequestCount.HasValue)
indexEntity["ApiRequestCount"] = apiRequestCount.Value;
// Connection-type projection (UpdateSessionConnectionTypeAsync merges this) —
// must be in this superset or a StartedAt-shift full upsert drops it.
var connectionType = sessionEntity.GetString("ConnectionType");
if (!string.IsNullOrEmpty(connectionType))
indexEntity["ConnectionType"] = connectionType;
return indexEntity;
}
@@ -388,6 +404,7 @@ namespace AutopilotMonitor.Functions.Services
string geoRegion = string.Empty;
string geoCity = string.Empty;
string geoLoc = string.Empty;
string? connectionType = null;
string? existingIndexRowKey = null;
// PR3: cascade-delete state-machine columns. Re-registration MUST preserve
// these — losing them would silently clear the lock and let the agent's writes
@@ -434,6 +451,7 @@ namespace AutopilotMonitor.Functions.Services
geoRegion = existingEntity.GetString("GeoRegion") ?? string.Empty;
geoCity = existingEntity.GetString("GeoCity") ?? string.Empty;
geoLoc = existingEntity.GetString("GeoLoc") ?? string.Empty;
connectionType = existingEntity.GetString("ConnectionType");
existingIndexRowKey = existingEntity.GetString("IndexRowKey");
existingDeletionState = existingEntity.GetString("DeletionState");
existingPendingDeletionManifestId = existingEntity.GetString("PendingDeletionManifestId");
@@ -543,6 +561,12 @@ namespace AutopilotMonitor.Functions.Services
if (!string.IsNullOrEmpty(geoLoc))
entity["GeoLoc"] = geoLoc;
// Merge-mode field (UpdateSessionConnectionTypeAsync) — preserve through the
// Replace; the agent re-emits network_interface_info after restart, but the
// value must not be lost in the window until it does.
if (!string.IsNullOrEmpty(connectionType))
entity["ConnectionType"] = connectionType;
// Preserve IndexRowKey through the Replace so that:
// 1. Concurrent UpdateSessionStatusAsync calls can find it (prevents index stale-status)
// 2. UpsertSessionIndexAsync can detect and delete old index entries when startedAt shifts
@@ -2655,6 +2679,101 @@ namespace AutopilotMonitor.Functions.Services
}
}
/// <summary>
/// Stores the session-wide average agent→backend HTTP latency (Merge-mode, two fields).
/// Derived from cumulative counters, so a plain last-write-wins overwrite is idempotent
/// against batch replays and monotonically converges to the whole-session average.
/// Same UpdateEntity-not-Upsert rationale as <see cref="UpdateSessionImeAgentVersionAsync"/>:
/// a tombstoned Sessions row must stay tombstoned (404 → no-op).
/// Non-fatal: failures are logged as warnings and do not block ingest.
/// </summary>
public async Task UpdateSessionNetworkLatencyAsync(string tenantId, string sessionId,
double avgLatencyMs, int requestCount)
{
try
{
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.Sessions);
var update = new TableEntity(tenantId, sessionId)
{
["AvgApiLatencyMs"] = avgLatencyMs,
["ApiRequestCount"] = requestCount
};
await tableClient.UpdateEntityAsync(update, ETag.All, TableUpdateMode.Merge);
// Mirror to SessionsIndex — the index is a full denormalized mirror and serves the
// session list / GA cross-tenant views where per-geo latency aggregation happens.
var idxRef = await tableClient.GetEntityAsync<TableEntity>(
tenantId, sessionId, select: new[] { "IndexRowKey" });
var indexRowKey = idxRef.Value.GetString("IndexRowKey");
if (!string.IsNullOrEmpty(indexRowKey))
{
await MergeSessionIndexAsync(tenantId, indexRowKey,
new TableEntity(tenantId, indexRowKey)
{
["AvgApiLatencyMs"] = avgLatencyMs,
["ApiRequestCount"] = requestCount
});
}
}
catch (RequestFailedException ex) when (ex.Status == 404)
{
_logger.LogDebug(
"UpdateSessionNetworkLatency no-op: Sessions row {Tenant}/{Session} is absent (tombstoned or never registered)",
tenantId, sessionId);
}
catch (Exception ex)
{
// Non-fatal — don't block ingest
_logger.LogWarning(ex, "Failed to update network latency for session {SessionId}", sessionId);
}
}
/// <summary>
/// Stores the active network connection type ("WiFi"/"Ethernet") from the latest
/// network_interface_info event (Merge-mode, single field, last-write-wins).
/// Same UpdateEntity-not-Upsert rationale as <see cref="UpdateSessionNetworkLatencyAsync"/>:
/// a tombstoned Sessions row must stay tombstoned (404 → no-op).
/// Non-fatal: failures are logged as warnings and do not block ingest.
/// </summary>
public async Task UpdateSessionConnectionTypeAsync(string tenantId, string sessionId,
string connectionType)
{
try
{
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.Sessions);
var update = new TableEntity(tenantId, sessionId)
{
["ConnectionType"] = connectionType
};
await tableClient.UpdateEntityAsync(update, ETag.All, TableUpdateMode.Merge);
// Mirror to SessionsIndex — serves the session list and the cheap
// connectionType= search filter (OData eq push-down on the index).
var idxRef = await tableClient.GetEntityAsync<TableEntity>(
tenantId, sessionId, select: new[] { "IndexRowKey" });
var indexRowKey = idxRef.Value.GetString("IndexRowKey");
if (!string.IsNullOrEmpty(indexRowKey))
{
await MergeSessionIndexAsync(tenantId, indexRowKey,
new TableEntity(tenantId, indexRowKey)
{
["ConnectionType"] = connectionType
});
}
}
catch (RequestFailedException ex) when (ex.Status == 404)
{
_logger.LogDebug(
"UpdateSessionConnectionType no-op: Sessions row {Tenant}/{Session} is absent (tombstoned or never registered)",
tenantId, sessionId);
}
catch (Exception ex)
{
// Non-fatal — don't block ingest
_logger.LogWarning(ex, "Failed to update connection type for session {SessionId}", sessionId);
}
}
// ===== SESSION/EVENT MAPPING HELPERS =====
/// <summary>
@@ -2748,6 +2867,9 @@ namespace AutopilotMonitor.Functions.Services
PendingActionsQueuedAt = SafeGetDateTime(entity, "PendingActionsQueuedAt"),
EventCount = SafeGetInt32(entity, "EventCount") ?? 0,
DurationSeconds = ComputeEffectiveDuration(entity, status, startedAt, completedAt),
AvgApiLatencyMs = SafeGetDouble(entity, "AvgApiLatencyMs"),
ApiRequestCount = SafeGetInt32(entity, "ApiRequestCount"),
ConnectionType = entity.GetString("ConnectionType"),
EnrollmentType = entity.GetString("EnrollmentType") ?? "v1",
DiagnosticsBlobName = entity.GetString("DiagnosticsBlobName"),
// Legacy rows that predate this field surface as null → download path treats
@@ -159,6 +159,37 @@ namespace AutopilotMonitor.Functions.Services
}
}
/// <summary>
/// Safely reads a Double property from a TableEntity.
/// Returns null instead of throwing when the property has a different type (legacy data).
/// Also accepts Int32/Int64-typed cells — Table Storage stores a whole-number double
/// as Int when written via JSON paths that drop the decimal point.
/// </summary>
private double? SafeGetDouble(TableEntity entity, string key)
{
try
{
return entity.GetDouble(key);
}
catch (InvalidOperationException)
{
if (entity.TryGetValue(key, out var raw))
{
switch (raw)
{
case int i: return i;
case long l: return l;
case string s when double.TryParse(s,
System.Globalization.NumberStyles.Float,
System.Globalization.CultureInfo.InvariantCulture, out var parsed):
return parsed;
}
}
_logger.LogWarning("Property '{Key}' on entity {PK}/{RK} is not Double and could not be coerced", key, entity.PartitionKey, entity.RowKey);
return null;
}
}
/// <summary>
/// Safely reads a DateTime property from a TableEntity.
/// Returns null instead of throwing when the property has a different type (legacy data).
+14 -1
View File
@@ -70,7 +70,20 @@ ENV NODE_ENV=production
ENV HF_CACHE_DIR=/app/.hf-cache
COPY --from=build /app/package.json /app/package-lock.json ./
RUN npm ci --omit=dev --ignore-scripts && npm cache clean --force
# npm is a BUILD-time tool only: the runtime is `node dist/index.js`, the
# HEALTHCHECK is `node -e`, and the server never shells out (no child_process
# anywhere in src/). But the npm that node:22-slim bundles carries its own
# vendored dependency tree under /usr/local/lib/node_modules/npm, and a
# container image scanner reports every CVE in it against this image —
# @sigstore/core, brace-expansion, tar, sigstore, picomatch, ip-address, plus
# npm itself, none of which appear in our package-lock and none of which
# Dependabot can see.
#
# Removing npm clears all of them; upgrading it does not (npm 11.x still
# vendors @sigstore/core 3.1.0, inside the vulnerable <= 3.2.0 range). The rm
# must share this RUN so the deletion lands in the same layer as the install.
RUN npm ci --omit=dev --ignore-scripts && npm cache clean --force && \
rm -rf /usr/local/lib/node_modules/npm /usr/local/bin/npm /usr/local/bin/npx
COPY --from=build /app/dist/ ./dist/
COPY --from=build /app/.hf-cache/ ./.hf-cache/
+11 -8
View File
@@ -9,7 +9,7 @@
"version": "1.5.1",
"dependencies": {
"@huggingface/transformers": "^3.8.1",
"@modelcontextprotocol/sdk": "^1.29.0",
"@modelcontextprotocol/sdk": "^1.30.0",
"compression": "^1.8.1",
"express": "^5.2.1",
"fuse.js": "^7.5.0",
@@ -25,6 +25,9 @@
"tsx": "^4.23.1",
"typescript": "^5.0.0",
"vitest": "^4.1.10"
},
"engines": {
"node": ">=22"
}
},
"node_modules/@emnapi/core": {
@@ -1066,12 +1069,12 @@
"license": "MIT"
},
"node_modules/@modelcontextprotocol/sdk": {
"version": "1.29.0",
"resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.29.0.tgz",
"integrity": "sha512-zo37mZA9hJWpULgkRpowewez1y6ML5GsXJPY8FI0tBBCd77HEvza4jDqRKOXgHNn867PVGCyTdzqpz0izu5ZjQ==",
"version": "1.30.0",
"resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.30.0.tgz",
"integrity": "sha512-xKd8OIzlqNzcqcNumGAa6g+PW2kjD5vrpcKOnfldAUPP3j7lnqMPwlTXQm8gF+UwH72z0lqaRbjr9hqGz0eITA==",
"license": "MIT",
"dependencies": {
"@hono/node-server": "^1.19.9",
"@hono/node-server": "^1.19.9 || ^2.0.5",
"ajv": "^8.17.1",
"ajv-formats": "^3.0.1",
"content-type": "^1.0.5",
@@ -3772,9 +3775,9 @@
"license": "MIT"
},
"node_modules/tar": {
"version": "7.5.20",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.20.tgz",
"integrity": "sha512-9FcyK4PA6+WbzlTM9WhQm6vB5W7cP7dUiPsv1g7YDwEQnQ1CGpK3MGlKk/ITVWMk05kHZuBhmVhiv8LZoy/PFQ==",
"version": "7.5.22",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",
@@ -20,7 +20,7 @@
},
"dependencies": {
"@huggingface/transformers": "^3.8.1",
"@modelcontextprotocol/sdk": "^1.29.0",
"@modelcontextprotocol/sdk": "^1.30.0",
"compression": "^1.8.1",
"express": "^5.2.1",
"fuse.js": "^7.5.0",
@@ -295,7 +295,9 @@ export function registerAdminTools(server: McpServer, ga: boolean, strictGa: boo
title: 'Geographic Metrics',
description:
'Get geographic distribution of enrollments — where devices are enrolling from, with performance comparisons. ' +
'Shows per-location: session counts, success rates, avg/median/p95 duration, throughput, and outlier detection. ' +
'Shows per-location: session counts, success rates, avg/median/p95 duration, throughput, outlier detection, and ' +
'avgApiLatencyMs (agent→backend HTTP round-trip, request-weighted — the metric for judging backend-region distance, ' +
'e.g. when evaluating a new backend region; apiLatencyVsGlobalPct compares against the global weighted average). ' +
(ga ? 'Omit tenantId for cross-tenant view (Global Admin). ' : '') +
'Use get_geographic_sessions to drill into a specific location. ' +
'days accepts any value 1-365 (e.g. 5, 7, 12, 30, 90).',
@@ -892,6 +894,8 @@ export function registerAdminTools(server: McpServer, ga: boolean, strictGa: boo
isPreProvisioned: z.boolean().optional().describe('Filter pre-provisioned (white-glove) sessions'),
isHybridJoin: z.boolean().optional().describe('Filter hybrid AAD-join sessions'),
isSelfDeployingProfile: z.boolean().optional().describe('Filter self-deploying/kiosk profile sessions (CloudAssignedOobeConfig 0x20|0x40)'),
connectionType: z.enum(['WiFi', 'Ethernet']).optional()
.describe('Active network connection type ("WiFi" or "Ethernet", exact match on the indexed column; last emission wins). Sessions predating the projection lack the column and are excluded.'),
fields: z.string().optional().describe('Comma-separated pass-through projection over the literal stored column names (case-insensitive, PascalCase, e.g. "Status,OsEdition,ImeAgentVersion,GeoCity"); narrows the row but never drops a real column. PartitionKey + RowKey are always kept. Omit for the full raw row.'),
pageSize: z.coerce.number().int().min(1).max(1000).optional().default(200)
.describe('Page size (1-1000, default 200). Returns this many sessions per call; follow nextLink to fetch more.'),
@@ -904,14 +908,14 @@ export function registerAdminTools(server: McpServer, ga: boolean, strictGa: boo
try {
const { tenantId: rawTenantId, status, startedAfter, startedBefore, serialNumber, agentVersion, agentVersionPrefix,
imeAgentVersion, imeAgentVersionPrefix, manufacturer, model, enrollmentType, deviceName, osBuild,
geoCountry, isPreProvisioned, isHybridJoin, isSelfDeployingProfile, fields, pageSize, continuation } = args;
geoCountry, isPreProvisioned, isHybridJoin, isSelfDeployingProfile, connectionType, fields, pageSize, continuation } = args;
const tenantId = enforceDelegatedTenantForPage(rawTenantId, continuation);
const basePath = pickGlobalOrTenantPath('/api/global/raw/sessions', '/api/raw/sessions', tenantId);
const path = followNextLink(
basePath,
{ tenantId, status, startedAfter, startedBefore, serialNumber, agentVersion, agentVersionPrefix,
imeAgentVersion, imeAgentVersionPrefix, manufacturer, model, enrollmentType, deviceName, osBuild,
geoCountry, isPreProvisioned, isHybridJoin, isSelfDeployingProfile, fields, pageSize },
geoCountry, isPreProvisioned, isHybridJoin, isSelfDeployingProfile, connectionType, fields, pageSize },
continuation,
);
const data = await apiFetch(path);
@@ -139,12 +139,19 @@ export function registerSessionTools(server: McpServer, ga: boolean, delegated:
'e.g. rebootCountMin=5. Only populated for v2 enrollments; sessions that predate the field are excluded.'),
rebootCountMax: z.coerce.number().int().min(0).optional()
.describe('Maximum number of reboots observed during enrollment (<=).'),
connectionType: z.enum(['WiFi', 'Ethernet']).optional()
.describe('Active network connection type during enrollment ("WiFi" or "Ethernet"), indexed for cheap exact-match ' +
'filtering (e.g. "how many machines enrolled over WiFi?"). Last emission wins — a device that switches media ' +
'mid-enrollment reports the most recent state. Sessions that predate the projection lack the column and are excluded.'),
fields: z.string().optional()
.describe('Comma-separated lean projection (e.g. "sessionId,status,agentVersion,startedAt"). ' +
'Use for counting / aggregation to avoid the response cap. Available: sessionId, tenantId, status, ' +
'serialNumber, manufacturer, model, deviceName, osBuild, osName, startedAt, completedAt, ' +
'durationSeconds, currentPhase, failureReason, eventCount, enrollmentType, isPreProvisioned, ' +
'isUserDriven, isHybridJoin, isSelfDeployingProfile, agentVersion, imeAgentVersion, geoCountry, rebootCount.'),
'isUserDriven, isHybridJoin, isSelfDeployingProfile, agentVersion, imeAgentVersion, geoCountry, rebootCount, ' +
'avgApiLatencyMs, apiRequestCount (agent→backend HTTP round-trip; weight latency by apiRequestCount when aggregating, ' +
'e.g. fields=geoCountry,avgApiLatencyMs,apiRequestCount for a per-country latency sweep; null on sessions from agents predating the field), ' +
'connectionType ("WiFi"/"Ethernet"; null on sessions predating the projection).'),
deviceProperties: z.record(z.string(), z.string()).optional().describe(
'Dynamic device property filters. Keys use "eventType.propertyName" dot notation. ' +
'See the device_properties catalog (call get_resource(name="device_properties")) for all available keys and types. ' +
@@ -62,6 +62,13 @@ namespace AutopilotMonitor.Shared
/// </summary>
public const string ImeMatchLogPath = @"%ProgramData%\AutopilotMonitor\Logs\ime_pattern_matches.log";
/// <summary>
/// Default path for the gather rule debug trace file (debugging/diagnostics).
/// Written when the tenant enables EnableGatherRuleDebugLog; explains every
/// gather-rule evaluation decision, including outcomes that produce no event.
/// </summary>
public const string GatherRuleDebugLogPath = @"%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log";
/// <summary>
/// Scheduled Task name used to run the agent as SYSTEM
/// </summary>
@@ -45,6 +45,8 @@ namespace AutopilotMonitor.Shared.DataAccess
public const string Tenant = "Tenant";
public const string Agent = "Agent";
public const string Sla = "SLA";
/// <summary>Platform infrastructure alerts relayed from Azure Monitor (ops alert webhook).</summary>
public const string Platform = "Platform";
}
/// <summary>
@@ -107,6 +107,21 @@ namespace AutopilotMonitor.Shared.DataAccess
Task UpdateSessionGeoAsync(string tenantId, string sessionId,
string? country, string? region, string? city, string? loc);
Task UpdateSessionImeAgentVersionAsync(string tenantId, string sessionId, string version);
/// <summary>
/// Persists the session-wide average agent→backend HTTP latency derived from the
/// cumulative counters of the latest <c>agent_metrics_snapshot</c>. Last-write-wins
/// (cumulative counters make replays idempotent); fail-soft like the other
/// supplementary-field updaters.
/// </summary>
Task UpdateSessionNetworkLatencyAsync(string tenantId, string sessionId,
double avgLatencyMs, int requestCount);
/// <summary>
/// Persists the active network connection type ("WiFi" or "Ethernet") from the
/// latest <c>network_interface_info</c> event. Last-write-wins; fail-soft like
/// the other supplementary-field updaters.
/// </summary>
Task UpdateSessionConnectionTypeAsync(string tenantId, string sessionId,
string connectionType);
// --- Server→Agent Actions ---
/// <summary>
@@ -40,6 +40,12 @@ namespace AutopilotMonitor.Shared.Models
/// </summary>
public bool EnableImeMatchLog { get; set; } = false;
/// <summary>
/// Whether to write a local gather-rule evaluation trace file.
/// When true, the default path is used: Constants.GatherRuleDebugLogPath.
/// </summary>
public bool EnableGatherRuleDebugLog { get; set; } = false;
public CollectorConfiguration Collectors { get; set; } = default!;
/// <summary>
@@ -334,6 +334,15 @@ namespace AutopilotMonitor.Shared.Models
/// </summary>
public bool? EnableImeMatchLog { get; set; }
/// <summary>
/// Whether the agent writes a local gather-rule evaluation trace file so customers
/// can diagnose rules that produce no timeline events (scope skips, on_change
/// suppression, empty collector results, logparser details). When true, the agent
/// writes to Constants.GatherRuleDebugLogPath. The trace never leaves the device.
/// null = use agent default (false).
/// </summary>
public bool? EnableGatherRuleDebugLog { get; set; }
/// <summary>
/// Log verbosity level override for this tenant's agents.
/// null = use agent default ("Info"). Values: "Info", "Debug", "Verbose", "Trace".
@@ -54,6 +54,16 @@ namespace AutopilotMonitor.Shared.Models
/// <summary>Percentage difference from global avg throughput (positive = faster)</summary>
public double ThroughputVsGlobalPct { get; set; }
/// <summary>
/// Average agent→backend HTTP round-trip (ms) at this location, weighted per session by
/// its request count. 0 when no session here carries latency data (pre-feature agents).
/// </summary>
public double AvgApiLatencyMs { get; set; }
/// <summary>Sessions at this location that carry API-latency data</summary>
public int ApiLatencySessionCount { get; set; }
/// <summary>Percentage difference from global avg API latency (positive = slower/farther)</summary>
public double ApiLatencyVsGlobalPct { get; set; }
public bool IsOutlier { get; set; }
/// <summary>"fast", "slow", or null</summary>
public string? OutlierDirection { get; set; }
@@ -142,6 +152,8 @@ namespace AutopilotMonitor.Shared.Models
GeoRegion = s.GeoRegion,
GeoCity = s.GeoCity,
GeoLoc = s.GeoLoc,
AvgApiLatencyMs = s.AvgApiLatencyMs,
ApiRequestCount = s.ApiRequestCount,
PlatformScriptCount = s.PlatformScriptCount,
RemediationScriptCount = s.RemediationScriptCount,
PendingActionsJson = s.PendingActionsJson,
@@ -173,6 +185,8 @@ namespace AutopilotMonitor.Shared.Models
public double AvgMinutesPerApp { get; set; }
public double AvgThroughputBytesPerSec { get; set; }
public double StdDevDurationMinutes { get; set; }
/// <summary>Global request-weighted average agent→backend API latency (ms); 0 = no data yet</summary>
public double AvgApiLatencyMs { get; set; }
/// <summary>Global weighted average peer caching percentage</summary>
public double AvgDoPercentPeerCaching { get; set; }
/// <summary>Total peer bytes across all locations with DO data</summary>
@@ -64,6 +64,34 @@ namespace AutopilotMonitor.Shared.Models
WhiteGloveAnchorsIncomplete = 16
}
/// <summary>
/// Interpretation helpers for <see cref="TimeAttributionFlags"/>. The duration mask lives
/// here as a const rather than as a composite enum member on purpose: the wire format
/// serializes flag combinations as comma-joined member names (the web splits that string),
/// and a composite member would hijack ToString for combinations that happen to equal it.
/// </summary>
public static class TimeAttributionFlagQuality
{
/// <summary>
/// Flags that make the SEGMENT DURATIONS themselves untrustworthy — dropped anchors,
/// underobserved early phases, best-effort WhiteGlove window boundaries. Only these
/// exclude a breakdown from fleet segment statistics. The blocking-set flags
/// (<see cref="TimeAttributionFlags.BlockingSetUnknown"/> / BlockingSetTruncated) say
/// nothing about the measured spans — they only limit per-app blocking evidence, which
/// is positive-evidence-based anyway (unknown contributes no intervals). Gating fleet
/// stats on ANY flag starved the aggregates in production (2026-07-27: 903/1000
/// breakdowns carried BlockingSetUnknown → CleanSessionCount 12 vs 331 excluded).
/// </summary>
public const TimeAttributionFlags DurationCriticalFlags =
TimeAttributionFlags.ClockSkewDropped |
TimeAttributionFlags.PartialObservation |
TimeAttributionFlags.WhiteGloveAnchorsIncomplete;
/// <summary>True when the breakdown's segment durations are unfit for fleet statistics.</summary>
public static bool ExcludesFromFleetStats(TimeAttributionFlags flags)
=> (flags & DurationCriticalFlags) != TimeAttributionFlags.None;
}
/// <summary>One contiguous attributed span inside an observation window.</summary>
public class TimeAttributionSpan
{
@@ -257,6 +257,32 @@ namespace AutopilotMonitor.Shared.Models
public int EventCount { get; set; }
public int? DurationSeconds { get; set; }
/// <summary>
/// Session-wide average HTTP round-trip latency (ms) of the agent's calls to the
/// backend API, as measured on the device (includes network RTT + TLS + server
/// processing). Projected at ingest from the cumulative counters of the latest
/// <c>agent_metrics_snapshot</c> (<c>net_total_latency_ms / net_total_requests</c>).
/// Null for V1 sessions and sessions from agents that predate the field.
/// Aggregate across sessions weighted by <see cref="ApiRequestCount"/> — e.g. per
/// GeoCountry to compare regional backend reachability.
/// </summary>
public double? AvgApiLatencyMs { get; set; }
/// <summary>
/// Number of HTTP requests behind <see cref="AvgApiLatencyMs"/> (the cumulative
/// <c>net_total_requests</c> at the latest snapshot). Needed as the weight when
/// averaging latency across sessions. Null whenever <see cref="AvgApiLatencyMs"/> is.
/// </summary>
public int? ApiRequestCount { get; set; }
/// <summary>
/// Active network connection type during enrollment: "WiFi" or "Ethernet".
/// Projected at ingest from the latest <c>network_interface_info</c> event
/// (last emission wins — a device that switches media mid-enrollment reports
/// the most recent state). Null for sessions that predate the projection.
/// </summary>
public string? ConnectionType { get; set; }
/// <summary>
/// Enrollment type: "v1" (Autopilot Classic/ESP) or "v2" (Windows Device Preparation).
/// Defaults to "v1" for sessions that predate this field.
@@ -37,6 +37,11 @@ namespace AutopilotMonitor.Shared.Models
public int? RebootCountMin { get; set; }
public int? RebootCountMax { get; set; }
// Active network connection type ("WiFi" or "Ethernet"). Exact match on the
// stored ConnectionType column. Sessions that predate the projection lack the
// property and are excluded (acceptable — they have no connection data).
public string? ConnectionType { get; set; }
public int Limit { get; set; } = 50;
// Dynamic device property filters (key = "eventType.propertyName", value = filter expression)
@@ -110,6 +110,15 @@ const OPS_EVENT_TYPES: Record<string, string[]> = {
// feedback_ops_event_types_dual_register.
"AgentEmergencyBreak",
],
Platform: [
// Azure Monitor alerts relayed through POST ops/alert-webhook (Common Alert Schema →
// AzureMonitorAlertWebhookFunction → RecordAzureMonitorAlertAsync). One event type for ALL
// Azure Monitor alert rules — the rule name travels in the message/details. Severity is
// mapped Sev0→Critical, Sev1→Error, Sev2→Warning, Sev3/4→Info; both Fired and Resolved
// notifications carry the same mapped severity, so a rule here delivers recovery pings too.
// Dual-register per memory feedback_ops_event_types_dual_register.
"AzureMonitorAlert",
],
};
const SEVERITIES = ["Info", "Warning", "Error", "Critical"];
@@ -120,6 +129,7 @@ const CATEGORY_COLORS: Record<string, string> = {
Security: "text-orange-700 dark:text-orange-300",
Tenant: "text-teal-700 dark:text-teal-300",
Agent: "text-cyan-700 dark:text-cyan-300",
Platform: "text-sky-700 dark:text-sky-300",
};
interface OpsAlertRulesSectionProps {
@@ -42,9 +42,10 @@ const CATEGORY_STYLES: Record<string, string> = {
Security: "bg-orange-100 text-orange-800 dark:bg-orange-900/40 dark:text-orange-300",
Tenant: "bg-teal-100 text-teal-800 dark:bg-teal-900/40 dark:text-teal-300",
Agent: "bg-cyan-100 text-cyan-800 dark:bg-cyan-900/40 dark:text-cyan-300",
Platform: "bg-sky-100 text-sky-800 dark:bg-sky-900/40 dark:text-sky-300",
};
const ALL_CATEGORIES = ["Consent", "Maintenance", "Security", "Tenant", "Agent"];
const ALL_CATEGORIES = ["Consent", "Maintenance", "Security", "Tenant", "Agent", "Platform"];
const PAGE_SIZE = 20;
@@ -58,6 +58,7 @@ interface TenantConfig {
enableTimezoneAutoSet?: boolean | null;
ntpServer?: string | null;
enableImeMatchLog?: boolean | null;
enableGatherRuleDebugLog?: boolean | null;
logLevel?: string | null;
maxBatchSize?: number | null;
showEnrollmentSummary?: boolean | null;
@@ -130,6 +131,7 @@ const DEFAULTS: Record<string, unknown> = {
enableTimezoneAutoSet: null,
ntpServer: 'time.windows.com',
enableImeMatchLog: null,
enableGatherRuleDebugLog: null,
logLevel: null,
maxBatchSize: null,
showEnrollmentSummary: null,
@@ -168,6 +170,7 @@ const RUNTIME_DEFAULTS: Record<string, unknown> = {
enableTimezoneAutoSet: false,
ntpServer: 'time.windows.com',
enableImeMatchLog: false,
enableGatherRuleDebugLog: false,
maxAuthFailures: 5,
authFailureTimeoutMinutes: 0,
logLevel: 'Info',
@@ -296,6 +299,7 @@ function computeRuntime(c: TenantConfig): Record<string, unknown> {
enableTimezoneAutoSet: c.enableTimezoneAutoSet ?? false,
ntpServer: c.ntpServer ?? 'time.windows.com',
enableImeMatchLog: c.enableImeMatchLog ?? false,
enableGatherRuleDebugLog: c.enableGatherRuleDebugLog ?? false,
maxAuthFailures: c.maxAuthFailures ?? 5,
authFailureTimeoutMinutes: c.authFailureTimeoutMinutes ?? 0,
logLevel: c.logLevel ?? 'Info',
@@ -575,6 +579,7 @@ export function SectionTenantConfigReport() {
<ConfigRow label="Timezone Auto-Set" value={config.enableTimezoneAutoSet} configKey="enableTimezoneAutoSet" defaults={DEFAULTS} />
<ConfigRow label="NTP Server" value={config.ntpServer} configKey="ntpServer" defaults={DEFAULTS} />
<ConfigRow label="IME Match Log" value={config.enableImeMatchLog} configKey="enableImeMatchLog" defaults={DEFAULTS} />
<ConfigRow label="Gather Rule Debug Log" value={config.enableGatherRuleDebugLog} configKey="enableGatherRuleDebugLog" defaults={DEFAULTS} />
<ConfigRow label="Log Level" value={config.logLevel} configKey="logLevel" defaults={DEFAULTS} />
<ConfigRow label="Max Batch Size" value={config.maxBatchSize} configKey="maxBatchSize" defaults={DEFAULTS} />
<ConfigRow label="Show Script Output" value={config.showScriptOutput} configKey="showScriptOutput" defaults={DEFAULTS} />
@@ -647,6 +652,7 @@ export function SectionTenantConfigReport() {
<ConfigRow label="Timezone Auto-Set" value={runtime.enableTimezoneAutoSet} configKey="enableTimezoneAutoSet" defaults={RUNTIME_DEFAULTS} />
<ConfigRow label="NTP Server" value={runtime.ntpServer} configKey="ntpServer" defaults={RUNTIME_DEFAULTS} />
<ConfigRow label="IME Match Log" value={runtime.enableImeMatchLog} configKey="enableImeMatchLog" defaults={RUNTIME_DEFAULTS} />
<ConfigRow label="Gather Rule Debug Log" value={runtime.enableGatherRuleDebugLog} configKey="enableGatherRuleDebugLog" defaults={RUNTIME_DEFAULTS} />
<ConfigRow label="Log Level" value={runtime.logLevel} configKey="logLevel" defaults={RUNTIME_DEFAULTS} />
<ConfigRow label="Send Trace Events" value={runtime.sendTraceEvents} configKey="sendTraceEvents" defaults={RUNTIME_DEFAULTS} />
<ConfigRow label="Unrestricted Mode" value={runtime.unrestrictedMode} configKey="unrestrictedMode" defaults={RUNTIME_DEFAULTS} />
@@ -30,6 +30,8 @@ const ALL_COLUMNS: ColumnDef[] = [
{ key: "duration", label: "Duration", defaultVisible: true, sortKey: "durationSeconds" },
{ key: "started", label: "Started", defaultVisible: true, sortKey: "startedAt" },
{ key: "country", label: "Country", defaultVisible: false, sortKey: "geoCountry", filterKey: "geoCountry" },
{ key: "apiLatency", label: "API Latency", defaultVisible: false, sortKey: "avgApiLatencyMs" },
{ key: "connection", label: "Connection", defaultVisible: false, sortKey: "connectionType", filterKey: "connectionType" },
{ key: "agentVersion", label: "Agent Version", defaultVisible: false, sortKey: "agentVersion", filterKey: "agentVersion" },
{ key: "osName", label: "OS Name", defaultVisible: false, sortKey: "osName", filterKey: "osName" },
{ key: "osBuild", label: "OS Build", defaultVisible: false, sortKey: "osBuild", filterKey: "osBuild" },
@@ -640,7 +642,7 @@ export function SessionTable({
currentSort={sortColumn}
direction={sortDirection}
onSort={onSort}
className={["eventCount", "duration", "started", "country", "agentVersion", "osName", "osBuild", "osDisplayVersion", "osEdition", "osLanguage"].includes(col.key) ? "px-3" : undefined}
className={["eventCount", "duration", "started", "country", "apiLatency", "connection", "agentVersion", "osName", "osBuild", "osDisplayVersion", "osEdition", "osLanguage"].includes(col.key) ? "px-3" : undefined}
filterKey={col.filterKey}
filterValues={col.filterKey ? uniqueValuesByField[col.filterKey] : undefined}
activeFilter={col.filterKey ? columnFilters[col.filterKey] : undefined}
@@ -884,12 +886,16 @@ function SessionCell({
</td>
);
case "started":
case "started": {
// Two-line date/time keeps the column narrow so more columns fit on one page.
const started = new Date(session.startedAt);
return (
<td className="px-3 py-4 whitespace-nowrap text-sm text-gray-500">
{new Date(session.startedAt).toLocaleString()}
<div>{started.toLocaleDateString()}</div>
<div className="text-xs text-gray-400">{started.toLocaleTimeString()}</div>
</td>
);
}
case "country":
return (
@@ -904,6 +910,32 @@ function SessionCell({
</td>
);
case "apiLatency":
return (
<td className="px-3 py-4 whitespace-nowrap text-sm text-gray-500">
{(session.avgApiLatencyMs ?? 0) > 0 ? (
<span title={`Agent→backend HTTP round-trip${session.apiRequestCount ? ` across ${session.apiRequestCount} requests` : ""}`}>
{Math.round(session.avgApiLatencyMs!)} ms
</span>
) : (
<span className="text-gray-300">—</span>
)}
</td>
);
case "connection":
return (
<td className="px-3 py-4 whitespace-nowrap text-sm text-gray-500">
{session.connectionType ? (
<span title="Active network connection during enrollment (last reported state)">
{session.connectionType}
</span>
) : (
<span className="text-gray-300">—</span>
)}
</td>
);
case "agentVersion": {
const fullVersion = session.agentVersion || "";
const shortVersion = fullVersion.split("+")[0];
@@ -1,6 +1,8 @@
"use client";
import { useEffect } from "react";
import { useRouter } from "next/navigation";
import { isChunkLoadError, tryRecoverFromChunkError } from "@/utils/chunkReloadRecovery";
/**
* Route-level error boundary — catches unhandled exceptions within page
@@ -17,6 +19,15 @@ export default function Error({
}) {
const router = useRouter();
// Stale-bundle chunk failure after a deploy: reload once instead of showing the
// error card (utils/chunkReloadRecovery.ts — the guard prevents reload loops; a
// repeat inside the guard window falls through to the card below).
useEffect(() => {
if (isChunkLoadError(error)) {
tryRecoverFromChunkError("error-boundary");
}
}, [error]);
return (
<div className="min-h-screen bg-gradient-to-br from-blue-50 to-indigo-100 flex items-center justify-center p-4">
<div className="bg-white rounded-lg shadow-xl p-8 max-w-md w-full text-center">
@@ -31,7 +31,8 @@ export interface DeviceJourneyDailyDto {
export interface DeviceJourneyTotalsDto {
completedJourneys: number;
firstTimeRight: number;
ftrRatePct: number | null;
/** Absent (WhenWritingNull) when no completed journeys — treat null and undefined alike. */
ftrRatePct?: number | null;
excludedSessions: number;
attemptHistogram: DeviceJourneyAttemptBucketDto[];
}
@@ -53,7 +54,12 @@ export interface DeviceJourneyResponseDto {
windowDays: number;
totals: DeviceJourneyTotalsDto;
daily: DeviceJourneyDailyDto[];
repeatDevices: RepeatDeviceDto[] | null;
/**
* Absent (not null) in the aggregated all-tenants view — the worker serializes with
* WhenWritingNull, so the field is omitted entirely. Absent/null = no per-device
* drill-down available; [] = tenant scoped but no repeat devices.
*/
repeatDevices?: RepeatDeviceDto[] | null;
}
/** UI gate for the FTR rate (spec: ≥20 completed journeys). */
@@ -197,7 +203,7 @@ export default function DeviceJourneySection({
title="Devices whose current journey took 2 or more terminal attempts, newest activity first. The failure reason is from the most recent failed attempt.">
Repeat devices
</div>
{repeatDevices === null ? (
{repeatDevices == null ? (
<div className="text-sm text-gray-400 py-4">
Select a tenant to see per-device detail — the aggregated view has no device drill-down.
</div>
@@ -104,16 +104,19 @@ export default function GatherRulesPage() {
// Fetch unrestrictedMode from tenant config (for validation indicators).
// The display flag lives in the member-readable feature-flags endpoint so that
// Operators/Viewers can load this page without 403'ing on the admin-only full config.
// GA-override path keeps using globalConfig.tenant (GA-only endpoint).
// The SAME route serves the GA-override path: MemberRead has a global-scope bypass and
// the route is RouteParam-scoped, so a GA reads the SELECTED tenant's flag. (The former
// override URL /api/global/config/{tenantId} does not exist — the swallowed 404 left the
// badge showing the GA's HOME-tenant mode for every foreign tenant.)
useEffect(() => {
if (!effectiveTenantId) return;
// Reset first so a failed/forbidden fetch never carries the previous tenant's mode over.
setUnrestrictedMode(false);
let stale = false;
const fetchConfig = async () => {
try {
const url = isGlobalOverride
? api.globalConfig.tenant(effectiveTenantId)
: api.config.featureFlags(effectiveTenantId);
const response = await authenticatedFetch(url, getAccessToken);
if (response.ok) {
const response = await authenticatedFetch(api.config.featureFlags(effectiveTenantId), getAccessToken);
if (!stale && response.ok) {
const data = await response.json();
setUnrestrictedMode(data.unrestrictedMode ?? false);
}
@@ -122,7 +125,8 @@ export default function GatherRulesPage() {
}
};
fetchConfig();
}, [effectiveTenantId, isGlobalOverride, getAccessToken]);
return () => { stale = true; };
}, [effectiveTenantId, getAccessToken]);
const handleToggleRule = async (rule: GatherRule) => {
setTogglingRule(rule.ruleId);
@@ -42,6 +42,9 @@ interface LocationMetric {
totalDownloadBytes: number;
durationVsGlobalPct: number;
throughputVsGlobalPct: number;
avgApiLatencyMs: number;
apiLatencySessionCount: number;
apiLatencyVsGlobalPct: number;
isOutlier: boolean;
outlierDirection: string | null;
// Delivery Optimization
@@ -60,6 +63,7 @@ interface GlobalAverages {
avgMinutesPerApp: number;
avgThroughputBytesPerSec: number;
stdDevDurationMinutes: number;
avgApiLatencyMs: number;
avgDoPercentPeerCaching: number;
totalDoBytesFromPeers: number;
totalDoBytesFromHttp: number;
@@ -76,7 +80,7 @@ interface GeographicMetricsResponse {
}
type GroupBy = "city" | "region" | "country";
type SortBy = "sessionCount" | "avgDurationMinutes" | "appLoadScore" | "avgThroughputBytesPerSec" | "avgDoPercentPeerCaching";
type SortBy = "sessionCount" | "avgDurationMinutes" | "appLoadScore" | "avgThroughputBytesPerSec" | "avgApiLatencyMs" | "avgDoPercentPeerCaching";
type TimeRange = "7d" | "30d" | "90d";
const durationColor = (value: number, globalAvg: number) => {
@@ -96,6 +100,16 @@ const scoreColor = (score: number) => {
return "text-red-600";
};
// Absolute buckets, not relative-to-global: latency encodes physical distance to the backend
// region, and the decision it supports ("open a closer region?") needs absolute thresholds.
const latencyColor = (ms: number) => {
if (ms <= 0) return "text-gray-400";
if (ms < 250) return "bg-green-100 text-green-800";
if (ms < 500) return "bg-yellow-100 text-yellow-800";
if (ms < 800) return "bg-orange-100 text-orange-700";
return "bg-red-100 text-red-800";
};
const formatThroughput = (bytesPerSec: number) => {
if (bytesPerSec <= 0) return "—";
if (bytesPerSec >= 1024 * 1024) return `${(bytesPerSec / 1024 / 1024).toFixed(1)} MB/s`;
@@ -376,7 +390,7 @@ export default function GeographicPerformancePage() {
{geoMetrics && (
<div className="bg-blue-50 border border-blue-200 rounded-lg p-4 mb-6">
<div className="text-sm font-medium text-blue-800 mb-2">Global Averages (Benchmark)</div>
<div className="grid grid-cols-2 md:grid-cols-5 gap-4 text-sm">
<div className="grid grid-cols-2 md:grid-cols-6 gap-4 text-sm">
<div>
<span className="text-blue-600 font-medium">Avg Duration:</span>{" "}
<span className="text-blue-900">{geoMetrics.globalAverages.avgDurationMinutes} min</span>
@@ -401,6 +415,14 @@ export default function GeographicPerformancePage() {
: "No data"}
</span>
</div>
<div title="Agent→backend HTTP round-trip, request-weighted across sessions with latency data">
<span className="text-blue-600 font-medium">API Latency:</span>{" "}
<span className="text-blue-900">
{geoMetrics.globalAverages.avgApiLatencyMs > 0
? `${Math.round(geoMetrics.globalAverages.avgApiLatencyMs)} ms`
: "No data"}
</span>
</div>
</div>
</div>
)}
@@ -453,6 +475,13 @@ export default function GeographicPerformancePage() {
>
Throughput <SortIcon col="avgThroughputBytesPerSec" />
</th>
<th
className="px-4 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider cursor-pointer hover:text-gray-700"
onClick={() => handleSort("avgApiLatencyMs")}
title="Agent→backend HTTP round-trip measured on the devices — high values indicate network distance to the backend region"
>
API Latency <SortIcon col="avgApiLatencyMs" />
</th>
<th
className="px-4 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider cursor-pointer hover:text-gray-700"
onClick={() => handleSort("avgDoPercentPeerCaching")}
@@ -553,6 +582,18 @@ export default function GeographicPerformancePage() {
<td className="px-4 py-3 text-sm text-gray-700">
{formatThroughput(loc.avgThroughputBytesPerSec)}
</td>
<td className="px-4 py-3 text-sm">
{loc.avgApiLatencyMs > 0 ? (
<div title={`${loc.apiLatencySessionCount} session(s) with latency data${loc.apiLatencyVsGlobalPct !== 0 ? ` · ${loc.apiLatencyVsGlobalPct > 0 ? "+" : ""}${loc.apiLatencyVsGlobalPct.toFixed(0)}% vs global` : ""}`}>
<span className={`inline-flex items-center px-2 py-0.5 rounded text-xs font-medium ${latencyColor(loc.avgApiLatencyMs)}`}>
{Math.round(loc.avgApiLatencyMs)} ms
</span>
<span className="text-xs text-gray-400 ml-1">({loc.apiLatencySessionCount})</span>
</div>
) : (
<span className="text-gray-300" title="No sessions with latency data yet (requires agent ≥ 2.0 with latency reporting)">—</span>
)}
</td>
<td className="px-4 py-3 text-sm">
{loc.doSessionCount > 0 ? (
<div title={
@@ -32,6 +32,10 @@ interface SessionSummary {
failureReason: string | null;
durationSeconds: number | null;
enrollmentType: string;
/** Session-wide average agent→backend HTTP round-trip (ms); absent for pre-feature agents. */
avgApiLatencyMs?: number;
/** Number of HTTP requests behind avgApiLatencyMs. */
apiRequestCount?: number;
// Per-session Delivery Optimization aggregate (added by geographic drilldown endpoint)
hasDoTelemetry?: boolean;
doAppCount?: number;
@@ -170,6 +174,14 @@ function LocationSessionsContent() {
.map((s) => s.durationSeconds!);
return durations.length > 0 ? Math.round(durations.reduce((a, b) => a + b, 0) / durations.length / 60) : 0;
})(),
// Request-weighted API latency across sessions with latency data (matches region aggregate formula)
latencySessions: data.sessions.filter((s) => (s.avgApiLatencyMs ?? 0) > 0 && (s.apiRequestCount ?? 0) > 0).length,
avgApiLatency: (() => {
const withLatency = data.sessions.filter((s) => (s.avgApiLatencyMs ?? 0) > 0 && (s.apiRequestCount ?? 0) > 0);
const totalRequests = withLatency.reduce((a, s) => a + s.apiRequestCount!, 0);
const weightedSum = withLatency.reduce((a, s) => a + s.avgApiLatencyMs! * s.apiRequestCount!, 0);
return totalRequests > 0 ? Math.round(weightedSum / totalRequests) : 0;
})(),
// Weighted peer-caching % across sessions with DO telemetry (matches region aggregate formula)
doSessions: data.sessions.filter((s) => s.hasDoTelemetry).length,
avgDoPercent: (() => {
@@ -235,11 +247,20 @@ function LocationSessionsContent() {
<main className="max-w-7xl mx-auto py-6 px-4 sm:px-6 lg:px-8">
{/* Summary Stats */}
{stats && (
<div className="grid grid-cols-2 md:grid-cols-5 gap-4 mb-6">
<div className="grid grid-cols-2 md:grid-cols-6 gap-4 mb-6">
<div className="bg-white rounded-lg shadow p-4">
<div className="text-sm font-medium text-gray-500">Sessions</div>
<div className="text-2xl font-bold text-gray-900">{stats.total}</div>
</div>
<div className="bg-white rounded-lg shadow p-4" title="Agent→backend HTTP round-trip, request-weighted across sessions with latency data">
<div className="text-sm font-medium text-gray-500">API Latency</div>
<div className={`text-2xl font-bold ${stats.avgApiLatency > 0 ? "text-gray-900" : "text-gray-400"}`}>
{stats.avgApiLatency > 0 ? `${stats.avgApiLatency} ms` : "—"}
</div>
<div className="text-xs text-gray-400">
{stats.latencySessions > 0 ? `${stats.latencySessions} session(s) with data` : "requires latency-reporting agent"}
</div>
</div>
<div className="bg-white rounded-lg shadow p-4">
<div className="text-sm font-medium text-gray-500">Succeeded</div>
<div className="text-2xl font-bold text-green-600">{stats.succeeded}</div>
@@ -311,6 +332,9 @@ function LocationSessionsContent() {
<th className="px-4 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Duration
</th>
<th className="px-4 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider" title="Agent→backend HTTP round-trip measured on the device">
API Latency
</th>
<th className="px-4 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
DO Peers
</th>
@@ -341,6 +365,15 @@ function LocationSessionsContent() {
<td className="px-4 py-3 text-sm text-gray-600">
{formatDuration(session.durationSeconds)}
</td>
<td className="px-4 py-3 text-sm text-gray-600">
{(session.avgApiLatencyMs ?? 0) > 0 ? (
<span title={`across ${session.apiRequestCount ?? "?"} requests`}>
{Math.round(session.avgApiLatencyMs!)} ms
</span>
) : (
<span className="text-gray-300" title="No latency data (requires latency-reporting agent)">—</span>
)}
</td>
<td className="px-4 py-3 text-sm">
{session.hasDoTelemetry ? (
<div
@@ -14,6 +14,7 @@ import FeedbackBubble from "../components/FeedbackBubble";
import { SidebarProvider } from "../contexts/SidebarContext";
import { GlobalSidebar } from "../components/GlobalSidebar";
import AppInsightsInit from "../components/AppInsightsInit";
import ChunkReloadRecovery from "../components/ChunkReloadRecovery";
import { SITE_URL } from "@/utils/config";
const inter = Inter({ subsets: ["latin"] });
@@ -126,6 +127,7 @@ export default function RootLayout({
<TenantProvider>
<SidebarProvider>
<AppInsightsInit />
<ChunkReloadRecovery />
<Navbar />
<GlobalSidebar>
{children}
@@ -1,11 +1,11 @@
"use client";
import { useState, useMemo } from "react";
import { EnrollmentEvent } from "@/types";
import { EnrollmentEvent, Session } from "@/types";
import OobeConfigModal from "./OobeConfigModal";
import { compareVersions, stripGitHashSuffix } from "@/utils/bootstrapVersion";
export default function DeviceDetailsCard({ events, latestAgentVersion }: { events: EnrollmentEvent[]; latestAgentVersion?: string | null }) {
export default function DeviceDetailsCard({ events, latestAgentVersion, session }: { events: EnrollmentEvent[]; latestAgentVersion?: string | null; session?: Session | null }) {
const [expanded, setExpanded] = useState(false);
const [showIpv6, setShowIpv6] = useState<Record<number, boolean>>({});
const [showOobeModal, setShowOobeModal] = useState(false);
@@ -161,8 +161,15 @@ export default function DeviceDetailsCard({ events, latestAgentVersion }: { even
const deviceLocation = getEventData("device_location");
const hwSpec = getEventData("hardware_spec");
// Connectivity: agent→backend measurements. Latency comes from the session row (persisted
// at ingest from the cumulative snapshot counters); bandwidth/volume from their events.
const bandwidthEstimate = getEventData("network_bandwidth_estimate");
const metricsSnapshot = getEventData("agent_metrics_snapshot");
const apiLatencyMs = session?.avgApiLatencyMs ?? 0;
const hasConnectivity = apiLatencyMs > 0 || bandwidthEstimate || metricsSnapshot;
const hasData = agentStarted || bootTime || osInfo || networkAdapters || dnsConfig || proxyConfig || networkInterfaceInfo || wifiSignalInfo ||
autopilotProfile || aadJoinStatus || imeVersion || bitLockerStatus || secureBootStatus || deviceLocation || hwSpec;
autopilotProfile || aadJoinStatus || imeVersion || bitLockerStatus || secureBootStatus || deviceLocation || hwSpec || hasConnectivity;
if (!hasData) return null;
@@ -293,6 +300,32 @@ export default function DeviceDetailsCard({ events, latestAgentVersion }: { even
</DetailSection>
)}
{/* Connectivity: agent→backend measurements collected during the enrollment */}
{hasConnectivity && (
<DetailSection title="Connectivity">
{apiLatencyMs > 0 && (
<DetailRow
label="API Latency (avg)"
value={`${Math.round(apiLatencyMs)} ms${session?.apiRequestCount ? ` (${session.apiRequestCount} requests)` : ""}`}
/>
)}
{bandwidthEstimate?.estimatedWanMbps !== undefined && (
<DetailRow
label="Est. Internet Bandwidth"
value={`~${bandwidthEstimate.estimatedWanMbps} Mbit/s${bandwidthEstimate.confidence ? ` (${bandwidthEstimate.confidence} confidence)` : ""}`}
/>
)}
{bandwidthEstimate?.doThrottleConfigured === true && (
<DetailRow label="DO Throttle Policy" value="Configured (measured rate reflects the throttle)" />
)}
{metricsSnapshot?.net_total_bytes_up !== undefined && (
<DetailRow label="Telemetry Uploaded" value={formatBytesCompact(Number(metricsSnapshot.net_total_bytes_up))} />
)}
{/* Outbound IP is deliberately NOT rendered here — the outbound_ip trace event
stays available for analysis (timeline, MCP). */}
</DetailSection>
)}
{/* Security */}
{(bitLockerStatus || secureBootStatus || tpmStatus) && (
<DetailSection title="Security">
@@ -543,6 +576,13 @@ function DetailSection({ title, children }: { title: string; children: React.Rea
);
}
function formatBytesCompact(bytes: number): string {
if (!bytes || bytes <= 0 || isNaN(bytes)) return "—";
if (bytes >= 1024 * 1024 * 1024) return `${(bytes / 1024 / 1024 / 1024).toFixed(1)} GB`;
if (bytes >= 1024 * 1024) return `${(bytes / 1024 / 1024).toFixed(1)} MB`;
return `${(bytes / 1024).toFixed(0)} KB`;
}
function DetailRow({ label, value }: { label: string; value: string }) {
return (
<div className="flex justify-between text-xs py-0.5">
@@ -355,6 +355,24 @@ function PhaseSection({
);
}
// Backfilled events (ModernDeploymentTracker error backfill): pre-agent event-log records
// replayed at agent start carry backfilled=true plus the original event-log timeCreated —
// the event's own timestamp is agent emission time, so without this the whole block renders
// with one identical (wrong) time and full Error severity, reading like a live failure.
function getBackfillInfo(event: EnrollmentEvent): { isBackfilled: boolean; recordedAt: Date | null } {
const raw = event.data?.backfilled ?? event.data?.Backfilled;
const isBackfilled = raw === true || raw === "true" || raw === "True";
if (!isBackfilled) return { isBackfilled: false, recordedAt: null };
const t = event.data?.timeCreated ?? event.data?.TimeCreated;
const recordedAt = typeof t === "string" && t !== "" && !isNaN(Date.parse(t)) ? new Date(t) : null;
return { isBackfilled, recordedAt };
}
const BACKFILL_TOOLTIP =
"Recorded by Windows before the agent started — replayed from the event log for visibility. " +
"If enrollment progressed afterwards, this error was already resolved by a retry. " +
"The timestamp shown is the original event-log time.";
function EventRow({ event, showScriptOutput }: { event: EnrollmentEvent; showScriptOutput?: boolean }) {
const [showDetails, setShowDetails] = useState(false);
const [showRaw, setShowRaw] = useState(false);
@@ -429,15 +447,28 @@ function EventRow({ event, showScriptOutput }: { event: EnrollmentEvent; showScr
const hasData = isTruncated || (detailData && Object.keys(detailData).length > 0);
const hasDetails = true; // Every event has at least the metadata block
const { isBackfilled, recordedAt } = getBackfillInfo(event);
return (
<div id={`event-${event.eventId}`} className="bg-gray-50 rounded-lg p-3 hover:bg-gray-100 transition-colors">
<div className="flex items-start justify-between">
<div className="flex-1 min-w-0">
<div className="flex items-center gap-3">
<span className="text-xs text-gray-500 font-mono">
{new Date(event.timestamp).toLocaleTimeString()}
{(recordedAt ?? new Date(event.timestamp)).toLocaleTimeString()}
{recordedAt && (
<span className="text-gray-400 font-sans"> (reported {new Date(event.timestamp).toLocaleTimeString()})</span>
)}
</span>
<SeverityBadge severity={event.severity} />
{isBackfilled && (
<span
className="px-2 py-0.5 rounded text-xs font-medium bg-slate-200 text-slate-600"
title={BACKFILL_TOOLTIP}
>
⟲ Backfilled
</span>
)}
<span className="text-sm font-medium text-gray-900">{event.eventType}</span>
</div>
<p className="mt-1 text-sm text-gray-600" title={event.message || undefined}>{shortenBuildHashInMessage(event.message)}</p>
@@ -608,6 +639,15 @@ function EventRow({ event, showScriptOutput }: { event: EnrollmentEvent; showScr
<span className="w-16 flex-shrink-0 text-gray-400">EventId</span>
<span className="font-mono">{event.eventId}</span>
</div>
{recordedAt && (
<div className="flex mt-0.5">
<span className="w-16 flex-shrink-0 text-gray-400">Recorded</span>
<span className="font-mono">
{recordedAt.toISOString().replace('T', ' ').replace('Z', '')}
<span className="text-gray-400 ml-1" title={BACKFILL_TOOLTIP}>(event log, pre-agent)</span>
</span>
</div>
)}
<div className="flex mt-0.5">
<span className="w-16 flex-shrink-0 text-gray-400">Created</span>
<span className="font-mono">{event.timestamp}</span>
@@ -751,6 +791,7 @@ function RawEventRow({ event }: { event: EnrollmentEvent }) {
const [expanded, setExpanded] = useState(false);
const detailData = useMemo(() => normalizeEventDataForDisplay(event.data), [event.data]);
const hasDetails = detailData && Object.keys(detailData).length > 0;
const { isBackfilled, recordedAt } = getBackfillInfo(event);
const sevColor: Record<string, string> = {
Trace: "text-purple-500",
@@ -765,8 +806,11 @@ function RawEventRow({ event }: { event: EnrollmentEvent }) {
<div id={`event-${event.eventId}`} className="py-1.5 text-xs font-mono">
<div className="flex items-start gap-2">
<span className="text-gray-400 w-8 text-right flex-shrink-0">{event.sequence}</span>
<span className="text-gray-500 flex-shrink-0">{new Date(event.timestamp).toLocaleTimeString()}</span>
<span className="text-gray-500 flex-shrink-0">{(recordedAt ?? new Date(event.timestamp)).toLocaleTimeString()}</span>
<span className={`flex-shrink-0 w-14 ${sevColor[event.severity] || "text-gray-500"}`}>{event.severity}</span>
{isBackfilled && (
<span className="text-slate-500 flex-shrink-0" title={BACKFILL_TOOLTIP}>⟲</span>
)}
<span className="text-gray-900 font-medium flex-shrink-0">{event.eventType}</span>
<span className="text-gray-500 truncate flex-1 min-w-0" title={event.message || undefined}>{shortenBuildHashInMessage(event.message)}</span>
{hasDetails && (
@@ -573,7 +573,7 @@ export default function SessionDetailPage() {
)}
{/* Device Details Card (from enrollment tracker events) */}
{!isGatherRulesSession && <div id="section-device-details"><DeviceDetailsCard events={events} latestAgentVersion={latestAgentVersion} /></div>}
{!isGatherRulesSession && <div id="section-device-details"><DeviceDetailsCard events={events} latestAgentVersion={latestAgentVersion} session={session} /></div>}
{/* Phase Timeline */}
{!isGatherRulesSession && session && (
@@ -167,6 +167,8 @@ interface TenantConfigContextValue {
setEnableTimezoneAutoSet: (v: boolean) => void;
enableImeMatchLog: boolean;
setEnableImeMatchLog: (v: boolean) => void;
enableGatherRuleDebugLog: boolean;
setEnableGatherRuleDebugLog: (v: boolean) => void;
logLevel: string;
setLogLevel: (v: string) => void;
showScriptOutput: boolean;
@@ -384,6 +386,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
const [enableGeoLocation, setEnableGeoLocation] = useState(true);
const [enableTimezoneAutoSet, setEnableTimezoneAutoSet] = useState(false);
const [enableImeMatchLog, setEnableImeMatchLog] = useState(false);
const [enableGatherRuleDebugLog, setEnableGatherRuleDebugLog] = useState(false);
const [logLevel, setLogLevel] = useState("Info");
const [showScriptOutput, setShowScriptOutput] = useState(true);
const [showEnrollmentSummary, setShowEnrollmentSummary] = useState(false);
@@ -503,6 +506,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
setEnableGeoLocation(data.enableGeoLocation ?? true);
setEnableTimezoneAutoSet(data.enableTimezoneAutoSet ?? false);
setEnableImeMatchLog(data.enableImeMatchLog ?? false);
setEnableGatherRuleDebugLog(data.enableGatherRuleDebugLog ?? false);
setLogLevel(data.logLevel ?? "Info");
setShowScriptOutput(data.showScriptOutput ?? true);
setShowEnrollmentSummary(data.showEnrollmentSummary ?? false);
@@ -700,6 +704,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
enableGeoLocation,
enableTimezoneAutoSet,
enableImeMatchLog,
enableGatherRuleDebugLog,
logLevel,
showScriptOutput,
showEnrollmentSummary,
@@ -777,7 +782,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
manufacturerWhitelist, modelWhitelist, validateAutopilotDevice, validateCorporateIdentifier, validateDeviceAssociation,
dataRetentionDays, sessionTimeoutHours, enablePerformanceCollector, performanceCollectorInterval,
helloWaitTimeoutSeconds, selfDestructOnComplete, keepLogFile, rebootOnComplete, rebootDelaySeconds,
contactEmail, enableGeoLocation, enableTimezoneAutoSet, enableImeMatchLog, logLevel, showScriptOutput, showEnrollmentSummary,
contactEmail, enableGeoLocation, enableTimezoneAutoSet, enableImeMatchLog, enableGatherRuleDebugLog, logLevel, showScriptOutput, showEnrollmentSummary,
enrollmentSummaryTimeoutSeconds, enrollmentSummaryBrandingImageUrl, enrollmentSummaryLaunchRetrySeconds,
notificationChannels,
slaTargetSuccessRate, slaTargetMaxDurationMinutes, slaTargetAppInstallSuccessRate,
@@ -1113,6 +1118,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
setEnableGeoLocation(config.enableGeoLocation ?? true);
setEnableTimezoneAutoSet(config.enableTimezoneAutoSet ?? false);
setEnableImeMatchLog(config.enableImeMatchLog ?? false);
setEnableGatherRuleDebugLog(config.enableGatherRuleDebugLog ?? false);
setLogLevel(config.logLevel ?? "Info");
setShowScriptOutput(config.showScriptOutput ?? true);
setShowEnrollmentSummary(config.showEnrollmentSummary ?? false);
@@ -1542,6 +1548,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
enableGeoLocation, setEnableGeoLocation,
enableTimezoneAutoSet, setEnableTimezoneAutoSet,
enableImeMatchLog, setEnableImeMatchLog,
enableGatherRuleDebugLog, setEnableGatherRuleDebugLog,
logLevel, setLogLevel,
showScriptOutput, setShowScriptOutput,
showEnrollmentSummary, setShowEnrollmentSummary,
@@ -17,6 +17,7 @@ export function SectionAgentSettings() {
enableGeoLocation, setEnableGeoLocation,
enableTimezoneAutoSet, setEnableTimezoneAutoSet,
enableImeMatchLog, setEnableImeMatchLog,
enableGatherRuleDebugLog, setEnableGatherRuleDebugLog,
logLevel, setLogLevel,
showScriptOutput, setShowScriptOutput,
showEnrollmentSummary, setShowEnrollmentSummary,
@@ -51,6 +52,8 @@ export function SectionAgentSettings() {
setEnableTimezoneAutoSet={setEnableTimezoneAutoSet}
enableImeMatchLog={enableImeMatchLog}
setEnableImeMatchLog={setEnableImeMatchLog}
enableGatherRuleDebugLog={enableGatherRuleDebugLog}
setEnableGatherRuleDebugLog={setEnableGatherRuleDebugLog}
logLevel={logLevel}
setLogLevel={setLogLevel}
showScriptOutput={showScriptOutput}

Some files were not shown because too many files have changed in this diff Show More