mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-25 05:00:07 +00:00
Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 21b8e054df | |||
| 249c20e15a | |||
| 4ebd60fb1e | |||
| 22093d0824 | |||
| dff1532be5 |
@@ -119,6 +119,19 @@ normal IME deployment flow. Spam containment, in order:
|
||||
payload; the event's own timestamp is the emission time, so the session's `StartedAt`
|
||||
(which the backend aligns to the earliest event) is not dragged backwards.
|
||||
|
||||
Presentation (2026-07-28, customer feedback — an unannounced wall of pre-agent Error
|
||||
events read like a live failure):
|
||||
|
||||
- The scan buffers the (capped) record set, first emits an **Info summary event**
|
||||
(`modern_deployment_log`, payload `backfillSummary: true` + `replayedCount` /
|
||||
`droppedOverCap` / `lookbackMinutes`) announcing the replay block, then replays the
|
||||
records **oldest-first** so the timeline sequence follows event-log chronology.
|
||||
- The web timeline (`EventTimeline.tsx`) badges `backfilled: true` events („⟲ Backfilled",
|
||||
slate, explanatory tooltip) and renders the original `timeCreated` as the row time with a
|
||||
"(reported …)" suffix; the expanded metadata block gains a `Recorded` line. Severity is
|
||||
deliberately **not** downgraded — unresolved pre-agent errors must still surface in the
|
||||
Error filter.
|
||||
|
||||
# Schema — Error codes (backend known-issue map)
|
||||
|
||||
Codes documented in the known-issues / troubleshooting-FAQ pages that appear in ESP/enrollment
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
---
|
||||
type: Concept
|
||||
title: Gather Rule Debug Log — the per-tenant evaluation trace for rules that produce nothing
|
||||
description: EnableGatherRuleDebugLog (ConfigVersion 36) makes the agent write a local trace file explaining every gather-rule evaluation decision — scope skips, missing intervalSeconds, on_change suppression, empty collector results, logparser position/parse-mode/no-match outcomes — so customers can self-diagnose rules that never show up in the timeline.
|
||||
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleDebugLog.cs
|
||||
tags:
|
||||
- agent
|
||||
- gather-rules
|
||||
- diagnostics
|
||||
- tenant-configuration
|
||||
timestamp: 2026-07-28T00:00:00+02:00
|
||||
---
|
||||
|
||||
# Gather Rule Debug Log
|
||||
|
||||
## Schema
|
||||
|
||||
### The problem
|
||||
|
||||
A gather rule that produces no timeline events is indistinguishable from a rule that
|
||||
never ran. The pipeline has many *silent-nothing* exits — before this feature, none of
|
||||
them were visible to the customer:
|
||||
|
||||
| # | Decision point | Where |
|
||||
|---|---|---|
|
||||
| 1 | Rule delivered but `Enabled=false` | `GatherRuleExecutor.UpdateRules` |
|
||||
| 2 | `trigger=interval` without `intervalSeconds` — never runs | `UpdateRules` timer filter |
|
||||
| 3 | Interval timer's first tick comes only after one **full** interval | `UpdateRules` timer creation |
|
||||
| 4 | Out of phase scope (`activePhases` / `activeFromPhase` not latched); incl. `currentPhase=Unknown` before the first phase signal | `IsRuleInScopeLocked` |
|
||||
| 5 | phase_change / phase_exit dedup — once per (rule, phase) | `OnPhaseChanged` |
|
||||
| 6 | Collector returned an **empty** result (e.g. `emitOnlyIfExists=true` and target absent) | `ExecuteRule` |
|
||||
| 7 | `emit_mode=on_change` — result hash unchanged, emit suppressed | `ShouldEmitOnChange` |
|
||||
| 8 | Guard blocked the target (emits `security_warning`, but the rule yields no data) | `GatherRuleContext.EmitSecurityWarning` |
|
||||
| 9 | logparser: no files matched, position tracker says nothing new, regex no-match, plain-text file parsed in default `cmtrace` mode, `maxLines` cap, wildcard capped at 20 files | `LogParserCollector` |
|
||||
| 10 | Rule execution threw (agent log had only the message, no stack) | `ExecuteRule` catch |
|
||||
|
||||
### The setting
|
||||
|
||||
`TenantConfiguration.EnableGatherRuleDebugLog` (`bool?`, default null→false), served to the
|
||||
agent as `AgentConfigResponse.EnableGatherRuleDebugLog` — **ConfigVersion 36**. Portal:
|
||||
Settings → Agent → Settings, toggle "Gather Rule Debug Log", stored via the usual
|
||||
tenant-config roundtrip (`TableConfigRepository` Store+Map). Also shown in the admin
|
||||
tenant-config report.
|
||||
|
||||
`RemoteConfigMerger` projects the bool onto a runtime path (same pattern as
|
||||
`EnableImeMatchLog`): `true` → `Constants.GatherRuleDebugLogPath` =
|
||||
`%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log`; `false` → null, unless a
|
||||
`--gather-debug-log <path>` CLI override supplied a non-default path. The field is not
|
||||
security-sensitive and deliberately **survives the offline config cache**.
|
||||
|
||||
### The trace file
|
||||
|
||||
`GatherRuleDebugLog` is an append-only writer held by `GatherRuleExecutor` and exposed
|
||||
to collectors via `GatherRuleContext.DebugLog(ruleId, stage, message)` (null-safe no-op
|
||||
when disabled — zero cost on the normal path). Line format:
|
||||
|
||||
```
|
||||
2026-07-28T09:15:02.123Z | RULE-ID | stage | message
|
||||
```
|
||||
|
||||
Stages: `config` (registration summary per rule, header, zero-rules note), `trigger`
|
||||
(timer scheduled, phase/event fired, dedup), `scope` (skips with reason incl. current
|
||||
phase), `exec` (execution start), `collector` (null/empty results with likely cause),
|
||||
`guard` (allowlist block), `emit` (event emitted, hash change), `suppress` (on_change
|
||||
suppression with hash prefix + streak), `logparser` (per-file outcome: lines read,
|
||||
position range, match count, parse failures, regex timeouts, mode; plus the
|
||||
`format=text` hint when every line fails CMTrace parsing), `error` (full exception
|
||||
**with stack trace** — the agent log keeps only the message).
|
||||
|
||||
Volume control: 10 MB cap with a single `.old` rotation generation (~20 MB worst case).
|
||||
Every write failure is swallowed — tracing must never break gathering. The trace never
|
||||
leaves the device on its own — but because it lives in the agent log folder
|
||||
(`Constants.LogDirectory`), the diagnostics package (`DiagnosticsPackageService`,
|
||||
`*.log` pattern, `AgentLogs/` section) picks it up automatically, so a customer-initiated
|
||||
diagnostics upload hands the trace to support without manual copying.
|
||||
|
||||
### Semantics
|
||||
|
||||
* **No hot-reload** — the flag is consumed once when `DefaultComponentFactory` builds
|
||||
the `GatherRuleExecutorHost` at session start (per-enrollment lifecycle, same as
|
||||
`EnableImeMatchLog`). Enable it in the tenant, then the *next* enrollment writes the
|
||||
trace.
|
||||
* **Zero-rules case is traced too** — flag on + backend delivered no gather rules →
|
||||
a one-shot standalone `config` line, so the file is never just absent.
|
||||
* **`--run-gather-rules` always traces** (operator diagnostic mode; CLI override or
|
||||
default path) and mirrors every line to the console under `--console`.
|
||||
|
||||
## Examples
|
||||
|
||||
A logparser rule with `emit_mode=on_change` that "delivers nothing" becomes:
|
||||
|
||||
```
|
||||
... | LP-01 | config | registered: trigger=interval, collector=logparser, target=C:\...\app.log, unscoped, emitMode=on_change, interval=300s
|
||||
... | LP-01 | trigger | interval timer scheduled every 300s (first run after one full interval — no immediate execution)
|
||||
... | LP-01 | exec | executing: collector=logparser, target=C:\...\app.log
|
||||
... | LP-01 | logparser | app.log: read 1000 lines from position 0->81234, matched 0, parseFailures=1000, regexTimeouts=0, mode=cmtrace
|
||||
... | LP-01 | logparser | every line failed CMTrace parsing — if this is a plain-text log, set parameter format=text
|
||||
```
|
||||
|
||||
— the answer ("plain-text log parsed in CMTrace mode") is in the file.
|
||||
|
||||
## Citations
|
||||
|
||||
* Executor + trace points: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleExecutor.cs`
|
||||
* Writer: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleDebugLog.cs`
|
||||
* Config projection: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Configuration/RemoteConfigMerger.cs`
|
||||
* Backend serve + ConfigVersion: `../../src/Backend/AutopilotMonitor.Functions/Functions/Config/GetAgentConfigFunction.cs`
|
||||
* Phase scoping / emit mode background: `../rules/gather-rule-phase-scoping.md`
|
||||
+2
-1
@@ -20,6 +20,7 @@ https://docs.autopilotmonitor.com (separate repository).
|
||||
* [V2 Agent](agent/index.md) - How the on-device agent works: [runtime overview](agent/overview.md), [decision engine](agent/decision-engine.md) (reducer, signals, completion arms), [logs & persistence](agent/logs-and-persistence.md) (signal log, snapshot, crash recovery), [Hello wizard un-skip](agent/hello-wizard-unskip.md) (why a single "WHfB disabled" read never decides completion), [Autopilot ZTD diagnostics](agent/autopilot-ztd-diagnostics.md) (event IDs, diagnostic registry, endpoints, error-code map).
|
||||
* [Build Counter Blob](agent/build-counter-blob.md) - How agent build numbers are minted: shared counter blob with ETag-CAS so local and CI builds never collide; CI additionally attests build provenance (Sigstore).
|
||||
* [Agent Endpoint Migration](agent/endpoint-migration.md) - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins.
|
||||
* [Gather Rule Debug Log](agent/gather-rule-debug-log.md) - EnableGatherRuleDebugLog (ConfigVersion 36): the per-tenant local trace file that explains every gather-rule evaluation decision — scope skips, on_change suppression, empty collector results, logparser position/parse-mode outcomes — for rules that produce no timeline events.
|
||||
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
|
||||
* [Secure Boot CA-2023 Detection](agent/secureboot-ca2023-detection.md) - Why ANALYZE-SEC-001 v3 reads the UEFI db/KEK variables directly instead of trusting the SecureBoot\Servicing registry, and how the one-sided uefiCa2023FirmwareConfirmed marker plus a not_exists precondition give absence-tolerant suppression across old and new agents.
|
||||
* [ESP Policy-Provider Stall Detection](agent/esp-policy-provider-stall.md) - The EnrollmentStatusTracking CSP's no-timeout Setup/Apps wait is keyed to the Intune-registered "Sidecar" provider by NAME: co-management's "ConfigMgr" parks the user ESP at "Apps (Identifying)" for days even with TrackingPoliciesCreated=1 (issue #106); the always-on two-arm esp_policy_provider_stalled tripwire (provider_incomplete | sidecar_provider_missing) and the SOFTWARE\Microsoft\Windows\Autopilot gather-rule allowlist entry that surface it.
|
||||
@@ -43,7 +44,7 @@ https://docs.autopilotmonitor.com (separate repository).
|
||||
|
||||
# Web Portal
|
||||
|
||||
* [Portal Navigation Prefetch & the SWA Runtime Bottleneck](web/portal-navigation-prefetch.md) - Why every navigation Link sets `prefetch={false}`: the App Router's viewport prefetch fans out one RSC request per visible link against an HTTP/1.1 Static Web Apps runtime, and its latency tail starves the six connections that also serve the document and the route chunks. Includes where browser telemetry actually lives.
|
||||
* [Portal Navigation Prefetch & the SWA Runtime Bottleneck](web/portal-navigation-prefetch.md) - Why every navigation Link sets `prefetch={false}`: the App Router's viewport prefetch fans out one RSC request per visible link and the Static Web Apps SSR runtime queues them server-side into a multi-minute tail. Records the follow-up measurement (the Standard plan changed nothing; the prefetch storm was an accidental keep-alive), where browser telemetry actually lives, and why this curl build cannot be trusted for HTTP versions.
|
||||
|
||||
# Security & Identity
|
||||
|
||||
|
||||
+9
-1
@@ -1,8 +1,16 @@
|
||||
# Log
|
||||
|
||||
## 2026-07-28
|
||||
|
||||
* **Update**: `agent/autopilot-ztd-diagnostics.md` — presentation layer for the startup error backfill (customer feedback: an unannounced wall of pre-agent Error events with a single identical timestamp read like a live failure). Agent: the backfill scan now buffers the capped record set, emits an Info summary event first (`modern_deployment_log`, `backfillSummary: true` + `replayedCount`/`droppedOverCap`/`lookbackMinutes`), then replays oldest-first so the timeline sequence follows event-log chronology. Web: `EventTimeline.tsx` badges `backfilled: true` events („⟲ Backfilled", tooltip) and renders the original `timeCreated` as the row time with a "(reported …)" suffix plus a `Recorded` metadata line — works retroactively for already-stored events. Severity is deliberately not downgraded.
|
||||
|
||||
* **Creation**: Added `agent/gather-rule-debug-log.md` — the new per-tenant setting `EnableGatherRuleDebugLog` (ConfigVersion 35 → 36, portal toggle under Settings → Agent, `EnableImeMatchLog`-style bool→path projection in `RemoteConfigMerger`). When enabled, the agent writes `%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log`: one pipe-delimited line per evaluation decision across ten previously silent "produced nothing" exits — disabled rules, `interval` without `intervalSeconds` (never runs), the first-tick-after-one-full-interval window, phase-scope skips incl. `currentPhase=Unknown`, per-(rule, phase) dedup, empty collector results (`emitOnlyIfExists`), `on_change` hash suppression (hash prefix + streak), guard blocks, logparser per-file outcomes (position skips, parse failures with a `format=text` hint when every line fails CMTrace parsing, `maxLines`/wildcard caps), and full exception stacks that the agent log drops. 10 MB cap + one `.old` generation; no hot-reload (per-enrollment lifecycle); flag survives the offline config cache; `--run-gather-rules` always traces and echoes under `--console`; zero-rules delivery leaves a one-line file instead of nothing. Side fix: `GatherRuleExecutorHost` now passes the logger into `InformationalEventPost`, so gather-event emissions appear in the agent DEBUG log.
|
||||
|
||||
* **Correction + Update**: `web/portal-navigation-prefetch.md`. The HTTP/1.1 claim in the 2026-07-27 entry below is **wrong** and has been struck from the document: `portal.autopilotmonitor.com` negotiates **h2**, verified by `openssl s_client -alpn h2,http/1.1` and a .NET `HttpClient` returning `HTTP/2.0`. The false reading came from the Windows/Schannel curl build in this environment, which has no HTTP/2 support and therefore always reports `1.1` — do not use it to determine HTTP versions. There is no six-connection cap in play; the queue is server-side in the SWA SSR runtime, which does not change the fix or the evidence, only the mechanism. Added an `Outcome` section with 20 h of post-change telemetry: the Free → Standard plan change (16:02:44 UTC) produced **no measurable latency gain** once stratified by the gap to the previous request (p50 Free → Standard: `<10s` 85 → 95 ms, `5-30min` 429 → 441 ms; tail rates 1.64 %/0.374 % over 2 s/10 s vs 6 and 1 events at n = 258 against 4.2 and 1.0 expected; `browserTimings` Mann-Whitney z = −0.24, p = 0.81). The freezes were removed by `prefetch={false}` (16:34:13 UTC): worst `browserTimings.totalDuration` 225,277 → 4,856 ms. The prefetch storm was also an **accidental keep-alive** — requests under 10 s apart fell from 85 % to 46 %, and on the identical never-cacheable `/sessions/{id}` route the median rose 87 → 211 ms and p90 465 → 1,480 ms, with one 11,441 ms abandoned request (`resultCode 0`) after 517 minutes idle. Synthetic bursts warm the runtime and must not be used to benchmark this (round 1 p50 288 ms vs rounds 4–25 p50 113 ms).
|
||||
|
||||
## 2026-07-27
|
||||
|
||||
* **Creation**: Added `web/portal-navigation-prefetch.md` (and a `Web Portal` section to the index) — root cause of the minute-long portal freezes. `GlobalSidebar` and `Navbar` render the whole navigation on every page, so the App Router's default viewport prefetch issued one `?_rsc=` request per visible link against `portal.autopilotmonitor.com` (~10,400 in 14 days). That origin serves the document, the route chunks and the RSC payloads over **HTTP/1.1**, so the six-connection-per-origin cap applies; when the Static Web Apps managed runtime queued the prefetches (p50 91 ms, p95 896 ms, but max 234 s / 21 samples over 30 s), the in-flight prefetches held every socket and the navigation the user wanted could not get one. Measured on the 2026-07-27 17:25 CEST incident: `/admin/presence?_rsc=` 151 s, three siblings finishing within 340 ms of each other at ~37.5 s, all `200`, while the backend API answered in 1.6–2.2 s and `browserTimings` showed `networkDuration` 16 ms against `processingDuration` 225 s. Every navigation Link now sets `prefetch={false}` — these routes are client components behind `ProtectedRoute` that fetch their own data, so the prefetched RSC payload carried no benefit. The doc also records that browser telemetry lives in a **separate** App Insights resource (`AUTOPILOT_MONITOR_WEB_APPINSIGHTS_ID`) that the MCP `query_backend_logs` tool cannot reach, which is why the freezes left no trace in the backend logs.
|
||||
* **Creation**: Added `web/portal-navigation-prefetch.md` (and a `Web Portal` section to the index) — root cause of the minute-long portal freezes. `GlobalSidebar` and `Navbar` render the whole navigation on every page, so the App Router's default viewport prefetch issued one `?_rsc=` request per visible link against `portal.autopilotmonitor.com` (~10,400 in 14 days). *(The original entry attributed the freeze to HTTP/1.1 and a six-connection-per-origin cap — see the 2026-07-28 correction; the queue is server-side.)* When the Static Web Apps managed runtime queued the prefetches (p50 91 ms, p95 896 ms, but max 234 s / 21 samples over 30 s), the navigation the user wanted was stuck behind that queue. Measured on the 2026-07-27 17:25 CEST incident: `/admin/presence?_rsc=` 151 s, three siblings finishing within 340 ms of each other at ~37.5 s, all `200`, while the backend API answered in 1.6–2.2 s and `browserTimings` showed `networkDuration` 16 ms against `processingDuration` 225 s. Every navigation Link now sets `prefetch={false}` — these routes are client components behind `ProtectedRoute` that fetch their own data, so the prefetched RSC payload carried no benefit. The doc also records that browser telemetry lives in a **separate** App Insights resource (`AUTOPILOT_MONITOR_WEB_APPINSIGHTS_ID`) that the MCP `query_backend_logs` tool cannot reach, which is why the freezes left no trace in the backend logs.
|
||||
|
||||
* **Creation**: Added `backend/rule-regression-radar.md` — F3 regression radar (analyze rules only; gather rules wait for their per-batch stats dedup fix): per (tenant, rule) daily hit rate from `RuleStats`, firing only when the trailing 7-day window has ≥5 hit sessions and ≥20 evaluated sessions, the rate is ≥2× the prior 28-day baseline AND the Wilson 95 % intervals are disjoint (deterministic `MetricsMath` primitive with hand-computed test vectors). Suppressed for deleted rules, rules younger than baseline+window (`CreatedAt` grace), rules edited inside the window (`UpdatedAt`), empty baselines, and the `RuleRegressionRadarDisabled` app-setting kill switch. Alert episodes live as `ruleregression|{ruleId}` tracker rows (one bell + one `RuleFrequencyRegression` ops event per episode; numbers refreshed while active; re-armed under 1.5× baseline, on fire-stop, or by the 30-day tracker retention). On-fire dimension correlation over osBuild/model/agentVersion/imeVersion with the "correlated, not necessarily causal" wording contract. Surfaces: `regressions[]` in the rule-stats response (MCP inherits), the analyze-rules page's 30-day sparkline and regression badge.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: Concept
|
||||
title: Portal Navigation Prefetch & the SWA Runtime Bottleneck
|
||||
description: Why every navigation Link in the portal sets prefetch={false} — the App Router's viewport prefetch fires one RSC request per visible link against an HTTP/1.1 Static Web Apps runtime, and its latency tail starves the connection pool that serves the document and the route chunks.
|
||||
description: Why every navigation Link in the portal sets prefetch={false} — the App Router's viewport prefetch fires one RSC request per visible link, and the Static Web Apps SSR runtime queues them server-side into a multi-minute latency tail. Also records what the follow-up measurement showed about the Standard SKU and the keep-alive side effect.
|
||||
resource: src/Web/autopilot-monitor-web/components/GlobalSidebar.tsx
|
||||
tags:
|
||||
- web
|
||||
@@ -25,21 +25,26 @@ it enters the viewport.
|
||||
|
||||
# Mechanism
|
||||
|
||||
Three properties combine into a portal-wide freeze:
|
||||
Two properties combine into a portal-wide freeze:
|
||||
|
||||
1. **Fan-out.** The navigation is fully rendered on each page load, so a single navigation issues one
|
||||
RSC request per visible link — measured at ~10,400 requests over 14 days.
|
||||
2. **Shared origin over HTTP/1.1.** `portal.autopilotmonitor.com` serves the document, the JS route
|
||||
chunks *and* the RSC payloads, and it negotiates HTTP/1.1 — not HTTP/2. Browsers cap concurrent
|
||||
connections per origin at six.
|
||||
3. **A heavy latency tail on the SSR runtime.** RSC responses are prerendered and edge-cached
|
||||
2. **A heavy latency tail on the SSR runtime.** RSC responses are prerendered and edge-cached
|
||||
(`x-nextjs-cache: HIT`, `s-maxage=31536000`), so the median is fast. On a cache miss or cold start
|
||||
the Static Web Apps managed runtime queues them.
|
||||
the Static Web Apps managed runtime queues them **server-side**, and the queue drains all at once.
|
||||
|
||||
When the tail hits, the in-flight prefetches hold all six connections. The document request and the
|
||||
route chunks of the page the user actually wants then have no socket available, and the tab shows a
|
||||
loading spinner or a blank page until the queue drains. Nothing fails — the requests eventually
|
||||
return `200`, which is why the failure leaves no error trace anywhere.
|
||||
The navigation the user wants is then stuck behind that queue and the tab shows a loading spinner or a
|
||||
blank page until it drains. Nothing fails — the requests eventually return `200`, which is why the
|
||||
failure leaves no error trace anywhere.
|
||||
|
||||
> **Correction (2026-07-28).** The original version of this document named a third property: that
|
||||
> `portal.autopilotmonitor.com` negotiates HTTP/1.1 and the prefetches exhaust the six-connection
|
||||
> browser cap. **That is wrong.** It came from `curl -w '%{http_version}'` on a Windows build of curl
|
||||
> whose Schannel backend has no HTTP/2 support, so it can only ever report `1.1`. ALPN says otherwise:
|
||||
> `openssl s_client -alpn h2,http/1.1` negotiates **h2** on the portal host, confirmed by a .NET
|
||||
> `HttpClient` request returning `HTTP/2.0`. There is no six-connection cap and connection starvation
|
||||
> was never the mechanism — the queue is server-side. Verify HTTP versions with `openssl s_client
|
||||
> -alpn` or `HttpClient`, never with this curl build.
|
||||
|
||||
# Evidence
|
||||
|
||||
@@ -62,6 +67,33 @@ document immediately and the browser then waited nearly four minutes for the `lo
|
||||
Fourteen-day distribution of RSC prefetches: n = 10,428, p50 = 91 ms, p95 = 896 ms, max = 234,230 ms,
|
||||
38 over 10 s, 21 over 30 s.
|
||||
|
||||
# Outcome, measured 2026-07-28
|
||||
|
||||
Two changes landed within half an hour of each other on 2026-07-27: the SWA plan went Free → Standard
|
||||
at 16:02:44 UTC, and `prefetch={false}` went live at 16:34:13 UTC. Twenty hours of production
|
||||
telemetry separate their effects.
|
||||
|
||||
**The Standard plan produced no measurable latency gain.** Latency is governed almost entirely by the
|
||||
gap to the previous request — the SSR runtime cools down — so raw percentiles only measure the traffic
|
||||
mix. Stratified by that gap, the two eras are indistinguishable (p50, Free → Standard): `<10s`
|
||||
85 → 95 ms · `10-60s` 100 → 117 ms · `1-5min` 295 → 310 ms · `5-30min` 429 → 441 ms. Tail rates match
|
||||
too: Free ran 1.64 % over 2 s and 0.374 % over 10 s (n = 10,163); Standard produced 6 and 1 events at
|
||||
n = 258 against 4.2 and 1.0 expected. `browserTimings.totalDuration` by Mann-Whitney: n = 436 vs 48,
|
||||
z = −0.24, **p = 0.81**.
|
||||
|
||||
**The prefetch fix is what removed the freezes** — worst `browserTimings.totalDuration` fell from
|
||||
225,277 ms to 4,856 ms, and the 30-second events stopped.
|
||||
|
||||
**It also removed an accidental keep-alive.** The prefetch storm kept the SSR runtime warm: 85 % of
|
||||
requests used to arrive less than 10 s apart, now 46 % do. On an identical never-cacheable route
|
||||
(`/sessions/{id}`) the median rose from 87 ms to 211 ms and p90 from 465 ms to 1,480 ms. After 517
|
||||
minutes idle, one request took 11,441 ms and was abandoned (`resultCode 0`). The trade — minute-long
|
||||
freezes gone, ~150 ms added to a cold navigation — is worth it, but it is a trade.
|
||||
|
||||
Do not benchmark this with a synthetic burst. Firing requests in quick succession warms the runtime
|
||||
and measures the best case: in a 25-round `curl --parallel-max 6` run, round 1 came in at p50 288 ms
|
||||
and rounds 4–25 at p50 113 ms.
|
||||
|
||||
# Diagnosing this class of problem
|
||||
|
||||
The backend Application Insights resource is the wrong place to look — a freeze appears there as an
|
||||
|
||||
+48
@@ -285,6 +285,54 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
|
||||
Assert.Equal(@"C:\Temp\custom-ime.log", agent.ImeMatchLogPath);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Merge_maps_enable_gather_rule_debug_log_true_to_default_path()
|
||||
{
|
||||
var agent = NewAgentConfig();
|
||||
agent.GatherRuleDebugLogPath = null;
|
||||
|
||||
var remote = FullRemote();
|
||||
remote.EnableGatherRuleDebugLog = true;
|
||||
|
||||
RemoteConfigMerger.Merge(agent, remote);
|
||||
|
||||
Assert.False(string.IsNullOrEmpty(agent.GatherRuleDebugLogPath));
|
||||
Assert.Equal(
|
||||
Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath),
|
||||
agent.GatherRuleDebugLogPath);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Merge_maps_enable_gather_rule_debug_log_false_clears_path()
|
||||
{
|
||||
var agent = NewAgentConfig();
|
||||
agent.GatherRuleDebugLogPath = Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath);
|
||||
|
||||
var remote = FullRemote();
|
||||
remote.EnableGatherRuleDebugLog = false;
|
||||
|
||||
RemoteConfigMerger.Merge(agent, remote);
|
||||
|
||||
Assert.Null(agent.GatherRuleDebugLogPath);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Merge_preserves_custom_gather_rule_debug_log_path_when_remote_disables()
|
||||
{
|
||||
// A dev-time --gather-debug-log <custom> override seeds a non-default path on the
|
||||
// AgentConfiguration. When the tenant remote flips EnableGatherRuleDebugLog off we
|
||||
// keep the custom path rather than clobbering a deliberate dev override.
|
||||
var agent = NewAgentConfig();
|
||||
agent.GatherRuleDebugLogPath = @"C:\Temp\custom-gather.log";
|
||||
|
||||
var remote = FullRemote();
|
||||
remote.EnableGatherRuleDebugLog = false;
|
||||
|
||||
RemoteConfigMerger.Merge(agent, remote);
|
||||
|
||||
Assert.Equal(@"C:\Temp\custom-gather.log", agent.GatherRuleDebugLogPath);
|
||||
}
|
||||
|
||||
// ============================================================= Null / partial tolerance
|
||||
|
||||
[Fact]
|
||||
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
using System;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
|
||||
{
|
||||
/// <summary>
|
||||
/// Writer unit tests for <see cref="GatherRuleDebugLog"/>: line format, lazy directory
|
||||
/// creation, size-cap rotation, and the null-writer no-op path on the context.
|
||||
/// </summary>
|
||||
public sealed class GatherRuleDebugLogTests : IDisposable
|
||||
{
|
||||
private readonly TempDirectory _tmp = new TempDirectory();
|
||||
private readonly AgentLogger _logger;
|
||||
|
||||
public GatherRuleDebugLogTests()
|
||||
{
|
||||
_logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
|
||||
}
|
||||
|
||||
public void Dispose() => _tmp.Dispose();
|
||||
|
||||
private string LogPath => Path.Combine(_tmp.Path, "sub", "gather_rules_debug.log");
|
||||
|
||||
[Fact]
|
||||
public void Write_produces_pipe_delimited_line_and_creates_directory_lazily()
|
||||
{
|
||||
var log = new GatherRuleDebugLog(LogPath, _logger);
|
||||
Assert.False(File.Exists(LogPath)); // lazy — nothing until the first write
|
||||
|
||||
log.Write("RULE-1", GatherRuleDebugLog.StageScope, "skipped: out of scope");
|
||||
|
||||
var lines = File.ReadAllLines(LogPath);
|
||||
Assert.Equal(2, lines.Length); // header + entry
|
||||
Assert.Contains("| - | config | -- gather rule debug log started --", lines[0]);
|
||||
var parts = lines[1].Split(new[] { " | " }, StringSplitOptions.None);
|
||||
Assert.Equal(4, parts.Length);
|
||||
Assert.Equal("RULE-1", parts[1]);
|
||||
Assert.Equal("scope", parts[2]);
|
||||
Assert.Equal("skipped: out of scope", parts[3]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Write_flattens_newlines_except_for_error_stage()
|
||||
{
|
||||
var log = new GatherRuleDebugLog(LogPath, _logger);
|
||||
log.Write("RULE-1", GatherRuleDebugLog.StageCollector, "line1\r\nline2");
|
||||
log.Write("RULE-1", GatherRuleDebugLog.StageError, "boom\r\n at Frame()");
|
||||
|
||||
var content = File.ReadAllText(LogPath);
|
||||
Assert.Contains("line1 line2", content);
|
||||
Assert.Contains(" at Frame()", content); // error stage keeps the stack multi-line
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Write_null_ruleId_renders_dash()
|
||||
{
|
||||
var log = new GatherRuleDebugLog(LogPath, _logger);
|
||||
log.Write(null, GatherRuleDebugLog.StageConfig, "no rules delivered");
|
||||
|
||||
Assert.Contains(" | - | config | no rules delivered", File.ReadAllText(LogPath));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Write_rotates_once_when_file_exceeds_cap()
|
||||
{
|
||||
var log = new GatherRuleDebugLog(LogPath, _logger, maxBytes: 512);
|
||||
for (int i = 0; i < 20; i++)
|
||||
log.Write("RULE-1", GatherRuleDebugLog.StageExec, new string('x', 100));
|
||||
|
||||
var oldPath = Path.Combine(Path.GetDirectoryName(LogPath), "gather_rules_debug.old.log");
|
||||
Assert.True(File.Exists(oldPath), "expected one .old rotation generation");
|
||||
Assert.True(File.Exists(LogPath));
|
||||
Assert.Contains("-- rotated", File.ReadAllText(LogPath) + File.ReadAllText(oldPath));
|
||||
// Never more than one .old generation
|
||||
Assert.Single(Directory.GetFiles(Path.GetDirectoryName(LogPath), "*.old.log"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WriteStandalone_writes_single_config_line()
|
||||
{
|
||||
GatherRuleDebugLog.WriteStandalone(LogPath, "no gather rules delivered by backend — nothing to execute", _logger);
|
||||
|
||||
var lines = File.ReadAllLines(LogPath);
|
||||
Assert.Contains(lines, l => l.Contains("no gather rules delivered by backend"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Context_DebugLog_is_noop_without_writer()
|
||||
{
|
||||
var context = new GatherRuleContext(
|
||||
_logger, "sess", "tenant", _ => { }, null, new LogFilePositionTracker());
|
||||
|
||||
context.DebugLog("RULE-1", GatherRuleDebugLog.StageScope, "should not throw, should not write");
|
||||
|
||||
Assert.False(File.Exists(LogPath));
|
||||
}
|
||||
}
|
||||
}
|
||||
+156
@@ -0,0 +1,156 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
|
||||
{
|
||||
/// <summary>
|
||||
/// Debug-trace coverage of <see cref="GatherRuleExecutor"/> (EnableGatherRuleDebugLog):
|
||||
/// every "silently produced nothing" outcome must leave an explanatory line in the trace
|
||||
/// file — registration summary, missing interval, scope skips, on_change suppression,
|
||||
/// empty collector results.
|
||||
/// </summary>
|
||||
[Collection("SerialThreading")] // startup rules execute on the shared ThreadPool
|
||||
public sealed class GatherRuleExecutorDebugLogTests : IDisposable
|
||||
{
|
||||
private readonly TempDirectory _tmp = new TempDirectory();
|
||||
private readonly GatherRuleExecutor _executor;
|
||||
private readonly string _debugLogPath;
|
||||
|
||||
private const string AbsentPath =
|
||||
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_7e3a1f4b-0000-0000-0000-000000000000";
|
||||
|
||||
public GatherRuleExecutorDebugLogTests()
|
||||
{
|
||||
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
|
||||
_debugLogPath = Path.Combine(_tmp.Path, "gather_rules_debug.log");
|
||||
_executor = new GatherRuleExecutor(
|
||||
"sess", "tenant", _ => { }, logger,
|
||||
debugLogPath: _debugLogPath)
|
||||
{
|
||||
UnrestrictedMode = true
|
||||
};
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_executor.Dispose();
|
||||
_tmp.Dispose();
|
||||
}
|
||||
|
||||
private string TraceContent() => File.Exists(_debugLogPath) ? File.ReadAllText(_debugLogPath) : string.Empty;
|
||||
|
||||
private static GatherRule Rule(string id, Action<GatherRule>? mutate = null)
|
||||
{
|
||||
var rule = new GatherRule
|
||||
{
|
||||
RuleId = id,
|
||||
Title = id,
|
||||
CollectorType = "registry",
|
||||
Target = AbsentPath,
|
||||
Trigger = "startup",
|
||||
OutputEventType = "gather_test",
|
||||
Enabled = true,
|
||||
};
|
||||
mutate?.Invoke(rule);
|
||||
return rule;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void UpdateRules_writes_registration_summary_per_rule()
|
||||
{
|
||||
_executor.UpdateRules(new List<GatherRule>
|
||||
{
|
||||
Rule("GATHER-REG-001", r => { r.Trigger = "interval"; r.IntervalSeconds = 3600; r.EmitMode = "on_change"; }),
|
||||
Rule("GATHER-REG-002", r => r.Enabled = false),
|
||||
});
|
||||
_executor.WaitForStartupRules(30);
|
||||
|
||||
var trace = TraceContent();
|
||||
Assert.Contains("GATHER-REG-001 | config | registered: trigger=interval, collector=registry", trace);
|
||||
Assert.Contains("emitMode=on_change", trace);
|
||||
Assert.Contains("interval=3600s", trace);
|
||||
Assert.Contains("GATHER-REG-002 | config | rule disabled — will never run", trace);
|
||||
Assert.Contains("interval timer scheduled every 3600s (first run after one full interval", trace);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void UpdateRules_flags_interval_rule_without_intervalSeconds()
|
||||
{
|
||||
_executor.UpdateRules(new List<GatherRule>
|
||||
{
|
||||
Rule("GATHER-NOINT-001", r => { r.Trigger = "interval"; r.IntervalSeconds = null; }),
|
||||
});
|
||||
|
||||
Assert.Contains("GATHER-NOINT-001 | error | trigger=interval but intervalSeconds is missing — rule will NEVER run",
|
||||
TraceContent());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ScopedRule_skip_traces_unknown_phase_reason()
|
||||
{
|
||||
var rule = Rule("GATHER-SCOPE-001", r => { r.Trigger = "interval"; r.IntervalSeconds = 3600; r.ActivePhases = new List<string> { "DeviceSetup" }; });
|
||||
_executor.UpdateRules(new List<GatherRule> { rule });
|
||||
|
||||
// Synchronous internal check mirrors the timer-tick gate; the reason must name Unknown.
|
||||
string reason;
|
||||
Assert.False(_executor.IsRuleInScope(rule, out reason));
|
||||
Assert.Contains("currentPhase=Unknown", reason);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ScopedStartupRule_deferral_is_traced()
|
||||
{
|
||||
_executor.UpdateRules(new List<GatherRule>
|
||||
{
|
||||
Rule("GATHER-DEFER-001", r => r.ActivePhases = new List<string> { "DeviceSetup" }),
|
||||
});
|
||||
|
||||
Assert.Contains("GATHER-DEFER-001 | scope | startup rule deferred — currentPhase=Unknown", TraceContent());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void OnChange_suppression_and_reemission_are_traced()
|
||||
{
|
||||
var rule = Rule("GATHER-OC-001", r => { r.Trigger = "interval"; r.IntervalSeconds = 3600; r.EmitMode = "on_change"; });
|
||||
_executor.UpdateRules(new List<GatherRule> { rule });
|
||||
|
||||
string hashPrefix;
|
||||
int streak;
|
||||
var first = new Dictionary<string, object> { ["exists"] = false };
|
||||
Assert.True(_executor.ShouldEmitOnChange(rule, first, out hashPrefix, out streak));
|
||||
Assert.Equal(0, streak);
|
||||
Assert.False(string.IsNullOrEmpty(hashPrefix));
|
||||
|
||||
Assert.False(_executor.ShouldEmitOnChange(rule, new Dictionary<string, object> { ["exists"] = false }, out hashPrefix, out streak));
|
||||
Assert.Equal(1, streak);
|
||||
Assert.False(_executor.ShouldEmitOnChange(rule, new Dictionary<string, object> { ["exists"] = false }, out hashPrefix, out streak));
|
||||
Assert.Equal(2, streak);
|
||||
|
||||
var changed = new Dictionary<string, object> { ["exists"] = true };
|
||||
Assert.True(_executor.ShouldEmitOnChange(rule, changed, out hashPrefix, out streak));
|
||||
Assert.Equal(2, streak); // the streak that just ended
|
||||
Assert.Equal(2, changed["suppressedPolls"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void EmptyCollectorResult_is_traced_via_startup_execution()
|
||||
{
|
||||
_executor.UpdateRules(new List<GatherRule>
|
||||
{
|
||||
Rule("GATHER-EMPTY-001", r => r.Parameters = new Dictionary<string, string> { ["emitOnlyIfExists"] = "true" }),
|
||||
});
|
||||
Assert.True(_executor.WaitForStartupRules(30));
|
||||
|
||||
var trace = TraceContent();
|
||||
Assert.Contains("GATHER-EMPTY-001 | exec | executing: collector=registry", trace);
|
||||
Assert.Contains("GATHER-EMPTY-001 | collector | registry key/value not found", trace);
|
||||
Assert.Contains("GATHER-EMPTY-001 | collector | collector returned EMPTY result — nothing emitted", trace);
|
||||
}
|
||||
}
|
||||
}
|
||||
+121
@@ -0,0 +1,121 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
|
||||
{
|
||||
/// <summary>
|
||||
/// Debug-trace coverage of <see cref="LogParserCollector"/>: the classic
|
||||
/// "rule delivers nothing" cases must be explained in the trace — zero matches,
|
||||
/// position-tracker skips, and plain-text logs parsed in the default cmtrace mode.
|
||||
/// </summary>
|
||||
public sealed class LogParserCollectorDebugLogTests : IDisposable
|
||||
{
|
||||
private readonly TempDirectory _tmp = new TempDirectory();
|
||||
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
|
||||
private readonly GatherRuleContext _context;
|
||||
private readonly string _debugLogPath;
|
||||
|
||||
// %TEMP% lives under C:\Users (hard-blocked) — parse targets sit beside the
|
||||
// test assembly instead, admitted via UnrestrictedMode.
|
||||
private readonly string _outsideUsers = Path.Combine(
|
||||
AppContext.BaseDirectory, "gather-debug-tests-" + Guid.NewGuid().ToString("N"));
|
||||
|
||||
public LogParserCollectorDebugLogTests()
|
||||
{
|
||||
Directory.CreateDirectory(_outsideUsers);
|
||||
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
|
||||
_debugLogPath = Path.Combine(_tmp.Path, "gather_rules_debug.log");
|
||||
_context = new GatherRuleContext(
|
||||
logger, "sess", "tenant",
|
||||
evt => _events.Add(evt),
|
||||
null,
|
||||
new LogFilePositionTracker(),
|
||||
new GatherRuleDebugLog(_debugLogPath, logger))
|
||||
{
|
||||
UnrestrictedMode = true
|
||||
};
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_tmp.Dispose();
|
||||
try { Directory.Delete(_outsideUsers, recursive: true); } catch { /* best effort */ }
|
||||
}
|
||||
|
||||
private string TraceContent() => File.Exists(_debugLogPath) ? File.ReadAllText(_debugLogPath) : string.Empty;
|
||||
|
||||
private static GatherRule Rule(string target, string pattern, string? format = null) => new GatherRule
|
||||
{
|
||||
RuleId = "GATHER-LP-DEBUG",
|
||||
Title = "logparser debug test",
|
||||
CollectorType = "logparser",
|
||||
Target = target,
|
||||
Parameters = format == null
|
||||
? new Dictionary<string, string> { ["pattern"] = pattern }
|
||||
: new Dictionary<string, string> { ["pattern"] = pattern, ["format"] = format },
|
||||
Trigger = "startup",
|
||||
OutputEventType = "logparser_debug_test",
|
||||
};
|
||||
|
||||
[Fact]
|
||||
public void ZeroMatches_writes_per_file_outcome_line()
|
||||
{
|
||||
var path = Path.Combine(_outsideUsers, "app.log");
|
||||
File.WriteAllText(path, "nothing interesting\nstill nothing\n");
|
||||
|
||||
new LogParserCollector().Execute(Rule(path, "WILL_NEVER_MATCH_[0-9]+", format: "text"), _context);
|
||||
|
||||
Assert.Empty(_events);
|
||||
var trace = TraceContent();
|
||||
Assert.Contains("app.log: read 2 lines", trace);
|
||||
Assert.Contains("matched 0", trace);
|
||||
Assert.Contains("mode=text", trace);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void SecondRun_without_new_content_traces_position_skip()
|
||||
{
|
||||
var path = Path.Combine(_outsideUsers, "twice.log");
|
||||
File.WriteAllText(path, "one line\n");
|
||||
var rule = Rule(path, "one", format: "text");
|
||||
|
||||
var collector = new LogParserCollector();
|
||||
collector.Execute(rule, _context); // reads the file, advances the position
|
||||
collector.Execute(rule, _context); // nothing new
|
||||
|
||||
Assert.Contains("twice.log: no new content (position", TraceContent());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void PlainTextFile_in_default_cmtrace_mode_gets_format_hint()
|
||||
{
|
||||
var path = Path.Combine(_outsideUsers, "plain.log");
|
||||
File.WriteAllText(path, "plain line one\nplain line two\n");
|
||||
|
||||
new LogParserCollector().Execute(Rule(path, "plain"), _context); // default = cmtrace mode
|
||||
|
||||
Assert.Empty(_events);
|
||||
var trace = TraceContent();
|
||||
Assert.Contains("parseFailures=2", trace);
|
||||
Assert.Contains("every line failed CMTrace parsing — if this is a plain-text log, set parameter format=text", trace);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void MissingFile_traces_no_files_matched()
|
||||
{
|
||||
var path = Path.Combine(_outsideUsers, "does-not-exist.log");
|
||||
|
||||
new LogParserCollector().Execute(Rule(path, "x", format: "text"), _context);
|
||||
|
||||
Assert.Contains("no files matched target", TraceContent());
|
||||
}
|
||||
}
|
||||
}
|
||||
+89
@@ -226,6 +226,95 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Runtime
|
||||
return archive.Entries.Select(e => e.FullName).ToArray();
|
||||
}
|
||||
|
||||
private static string ZipEntryText(byte[] zipBytes, string entryName)
|
||||
{
|
||||
using var ms = new MemoryStream(zipBytes);
|
||||
using var archive = new ZipArchive(ms, ZipArchiveMode.Read);
|
||||
var entry = archive.GetEntry(entryName);
|
||||
Assert.NotNull(entry);
|
||||
using var reader = new StreamReader(entry!.Open(), Encoding.UTF8);
|
||||
return reader.ReadToEnd();
|
||||
}
|
||||
|
||||
// ── Nullable outcome (on-demand mid-session) — gate + sessioninfo semantics ──
|
||||
|
||||
/// <summary>
|
||||
/// Sentinel subclass: throws before any HTTP so gate-pass-through is observable as a
|
||||
/// non-null error result (a skip returns null BEFORE the archive is ever built).
|
||||
/// </summary>
|
||||
private sealed class ThrowOnBuildService : DiagnosticsPackageService
|
||||
{
|
||||
public ThrowOnBuildService(AgentConfiguration cfg, AgentLogger logger, BackendApiClient api)
|
||||
: base(cfg, logger, api) { }
|
||||
internal override byte[] BuildArchiveBytes(bool? enrollmentSucceeded) =>
|
||||
throw new System.InvalidOperationException("gate-passed-sentinel");
|
||||
}
|
||||
|
||||
private static DiagnosticsPackageService BuildGateProbe(AgentLogger logger, string mode)
|
||||
{
|
||||
var cfg = Cfg();
|
||||
cfg.DiagnosticsUploadEnabled = true;
|
||||
cfg.DiagnosticsUploadMode = mode;
|
||||
var apiClient = new BackendApiClient(
|
||||
httpClient: new System.Net.Http.HttpClient(),
|
||||
baseUrl: "http://localhost",
|
||||
manufacturer: string.Empty,
|
||||
model: string.Empty,
|
||||
serialNumber: string.Empty,
|
||||
useBootstrapTokenAuth: false,
|
||||
bootstrapToken: null,
|
||||
agentVersion: "0.0.0",
|
||||
logger: logger);
|
||||
return new ThrowOnBuildService(cfg, logger, apiClient);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async System.Threading.Tasks.Task CreateAndUploadAsync_null_outcome_passes_OnFailure_gate()
|
||||
{
|
||||
// On-demand mid-session (no outcome yet): the OnFailure gate must NOT skip —
|
||||
// there is no known success to skip on. Reaching the archive build (sentinel
|
||||
// throw → error result) proves the gate was passed; a skip would return null.
|
||||
using var rig = new Rig();
|
||||
var result = await BuildGateProbe(rig.Logger, "OnFailure")
|
||||
.CreateAndUploadAsync(enrollmentSucceeded: null, fileNameSuffix: "server-requested");
|
||||
|
||||
Assert.NotNull(result);
|
||||
Assert.Equal("gate-passed-sentinel", result!.ErrorCode);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async System.Threading.Tasks.Task CreateAndUploadAsync_known_success_still_skipped_by_OnFailure_gate()
|
||||
{
|
||||
using var rig = new Rig();
|
||||
var result = await BuildGateProbe(rig.Logger, "OnFailure")
|
||||
.CreateAndUploadAsync(enrollmentSucceeded: true);
|
||||
|
||||
Assert.Null(result);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async System.Threading.Tasks.Task CreateAndUploadAsync_null_outcome_still_blocked_by_mode_Off()
|
||||
{
|
||||
using var rig = new Rig();
|
||||
var result = await BuildGateProbe(rig.Logger, "Off")
|
||||
.CreateAndUploadAsync(enrollmentSucceeded: null);
|
||||
|
||||
Assert.Null(result);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(true, "Succeeded")]
|
||||
[InlineData(false, "Failed")]
|
||||
[InlineData(null, "In Progress")]
|
||||
public void BuildArchiveBytes_sessioninfo_reflects_nullable_outcome(bool? outcome, string expected)
|
||||
{
|
||||
using var rig = new Rig();
|
||||
var bytes = rig.Build().BuildArchiveBytes(outcome);
|
||||
|
||||
var sessionInfo = ZipEntryText(bytes, "sessioninfo.txt");
|
||||
Assert.Contains($"Enrollment Result: {expected}", sessionInfo);
|
||||
}
|
||||
|
||||
// ── BuildBlobUploadUrl — destination-aware URL construction ─────────────────
|
||||
|
||||
[Theory]
|
||||
|
||||
+30
@@ -112,6 +112,36 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
Assert.Equal(false, data["backfilled"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ErrorBackfillSummary_EmitsInfoLogEvent_AnnouncingTheReplayBlock()
|
||||
{
|
||||
_tracker.EmitErrorBackfillSummary(replayedCount: 6, droppedOverCap: 0);
|
||||
|
||||
var posted = _sink.Posted.Single(p => p.Kind == DecisionSignalKind.InformationalEvent);
|
||||
Assert.Equal("modern_deployment_log", posted.Payload![SignalPayloadKeys.EventType]);
|
||||
|
||||
var data = Assert.IsAssignableFrom<IReadOnlyDictionary<string, object>>(posted.TypedPayload);
|
||||
Assert.Equal(true, data["backfillSummary"]);
|
||||
Assert.Equal(6, data["replayedCount"]);
|
||||
Assert.Equal(0, data["droppedOverCap"]);
|
||||
Assert.Equal(240, data["lookbackMinutes"]);
|
||||
|
||||
var message = (string)posted.Payload[SignalPayloadKeys.Message];
|
||||
Assert.Contains("Replaying 6 error event(s)", message);
|
||||
Assert.Contains("before the agent started", message);
|
||||
Assert.DoesNotContain("cap", message);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ErrorBackfillSummary_MentionsDroppedRecords_WhenCapWasHit()
|
||||
{
|
||||
_tracker.EmitErrorBackfillSummary(replayedCount: 25, droppedOverCap: 7);
|
||||
|
||||
var posted = _sink.Posted.Single(p => p.Kind == DecisionSignalKind.InformationalEvent);
|
||||
var message = (string)posted.Payload![SignalPayloadKeys.Message];
|
||||
Assert.Contains("7 older record(s) beyond the 25-record cap", message);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BackfilledHarmlessId_StillRunsThroughRollupSuppression()
|
||||
{
|
||||
|
||||
+82
@@ -511,6 +511,88 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
|
||||
$"network_bandwidth_estimate (@{bandwidthIdx}) must precede whiteglove_part1_complete (@{part1CompleteIdx}) so the straggler stays in Part 1.");
|
||||
}
|
||||
|
||||
// ===================================================== WG Part-1 preprov diagnostics
|
||||
// V1 parity (restored 2026-07-28): the seal exit uploads the diagnostics package as a
|
||||
// pre-provisioning intermediate under the SAME mode gating as the terminal path — the
|
||||
// seal counts as a success, so Always uploads, OnFailure skips (a Part-1 failure
|
||||
// terminates via the regular Failed path, which uploads), Off does nothing.
|
||||
|
||||
[Theory]
|
||||
[InlineData("Always", true, 1)]
|
||||
[InlineData("OnFailure", true, 0)]
|
||||
[InlineData("Off", true, 0)]
|
||||
[InlineData("Always", false, 0)]
|
||||
public void Handle_whiteglove_part1_diagnostics_upload_respects_mode(
|
||||
string mode, bool enabled, int expected)
|
||||
{
|
||||
using var rig = new Rig();
|
||||
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.WhiteGloveSealed }.Build();
|
||||
var cfg = rig.BuildConfig(diagEnabled: enabled, diagMode: mode);
|
||||
|
||||
rig.Build(cfg).Handle(sender: null!,
|
||||
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.WhiteGloveSealed));
|
||||
|
||||
Assert.Equal(expected, rig.DiagnosticsUploads);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Handle_whiteglove_part1_diagnostics_upload_uses_preprov_suffix_and_succeeded_flag()
|
||||
{
|
||||
using var rig = new Rig();
|
||||
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.WhiteGloveSealed }.Build();
|
||||
var cfg = rig.BuildConfig(diagEnabled: true, diagMode: "Always");
|
||||
|
||||
rig.Build(cfg).Handle(sender: null!,
|
||||
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.WhiteGloveSealed));
|
||||
|
||||
Assert.Equal(1, rig.DiagnosticsUploads);
|
||||
Assert.Equal("preprov", rig.LastDiagnosticsSuffix);
|
||||
Assert.Equal(true, rig.LastDiagnosticsSucceededFlag);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Handle_whiteglove_part1_diagnostics_events_precede_part1_complete_marker()
|
||||
{
|
||||
// whiteglove_part1_complete is the authoritative Part-1/Part-2 boundary for the
|
||||
// web split (computeWhiteGloveSplitSequence) — diagnostics_collecting/uploaded
|
||||
// must sequence BEFORE it or they get mis-filed into the resumed Part-2 block.
|
||||
using var rig = new Rig();
|
||||
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.WhiteGloveSealed }.Build();
|
||||
rig.DiagnosticsResult = new DiagnosticsUploadResult { BlobName = "diag-preprov.zip" };
|
||||
var cfg = rig.BuildConfig(diagEnabled: true, diagMode: "Always");
|
||||
|
||||
rig.Build(cfg).Handle(sender: null!,
|
||||
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.WhiteGloveSealed));
|
||||
|
||||
var emitted = rig.EmittedEventTypes.ToList();
|
||||
var collectingIdx = emitted.IndexOf("diagnostics_collecting");
|
||||
var uploadedIdx = emitted.IndexOf("diagnostics_uploaded");
|
||||
var part1CompleteIdx = emitted.IndexOf(Constants.EventTypes.WhiteGlovePart1Complete);
|
||||
|
||||
Assert.True(collectingIdx >= 0, "diagnostics_collecting must have been emitted.");
|
||||
Assert.True(uploadedIdx >= 0, "diagnostics_uploaded must have been emitted.");
|
||||
Assert.True(part1CompleteIdx >= 0, "whiteglove_part1_complete must have been emitted.");
|
||||
Assert.True(uploadedIdx < part1CompleteIdx,
|
||||
$"diagnostics_uploaded (@{uploadedIdx}) must precede whiteglove_part1_complete (@{part1CompleteIdx}) so the upload stays in Part 1.");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Handle_whiteglove_part1_still_skips_cleanup_and_marker_when_diagnostics_enabled()
|
||||
{
|
||||
// The preprov upload must not change the Part-1 exit contract: no self-destruct,
|
||||
// no enrollment-complete.marker, whiteglove.complete still written.
|
||||
using var rig = new Rig();
|
||||
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.WhiteGloveSealed }.Build();
|
||||
var cfg = rig.BuildConfig(diagEnabled: true, diagMode: "Always");
|
||||
|
||||
rig.Build(cfg).Handle(sender: null!,
|
||||
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.WhiteGloveSealed));
|
||||
|
||||
Assert.Equal(0, rig.CleanupService.Invocations);
|
||||
Assert.False(File.Exists(Path.Combine(rig.StateDir, "enrollment-complete.marker")));
|
||||
Assert.True(rig.SessionPersistence.IsWhiteGloveResume());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Handle_standalone_reboot_fires_when_reboot_enabled_and_no_self_destruct()
|
||||
{
|
||||
|
||||
@@ -121,6 +121,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
|
||||
/// </summary>
|
||||
public string ImeMatchLogPath { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Optional path to write a gather-rule evaluation trace (for debugging).
|
||||
/// If empty, no trace is written. Set from remote config (EnableGatherRuleDebugLog)
|
||||
/// or the --gather-debug-log CLI override.
|
||||
/// </summary>
|
||||
public string GatherRuleDebugLogPath { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Path to directory with real IME log files for log replay.
|
||||
/// When set, the agent replays these logs with time compression instead of reading live logs.
|
||||
|
||||
@@ -135,6 +135,24 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------- GatherRuleDebugLog: same bool→path projection as ImeMatchLog
|
||||
// above. `true` expands Constants.GatherRuleDebugLogPath, `false`
|
||||
// nulls the runtime path unless a --gather-debug-log CLI override
|
||||
// supplied a non-default path.
|
||||
if (remoteConfig.EnableGatherRuleDebugLog)
|
||||
{
|
||||
agentConfig.GatherRuleDebugLogPath = Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath);
|
||||
}
|
||||
else
|
||||
{
|
||||
var defaultGatherDebugPath = Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath);
|
||||
if (string.IsNullOrEmpty(agentConfig.GatherRuleDebugLogPath)
|
||||
|| string.Equals(agentConfig.GatherRuleDebugLogPath, defaultGatherDebugPath, StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
agentConfig.GatherRuleDebugLogPath = null;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------- Upload interval / batch — direct, V1 parity (no > 0 gate so a
|
||||
// misconfigured tenant can in theory break uploads, but this
|
||||
// surfaces the config bug instead of silently masking it).
|
||||
|
||||
@@ -327,6 +327,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
|
||||
KeepLogFile = false,
|
||||
EnableGeoLocation = true,
|
||||
EnableImeMatchLog = false,
|
||||
EnableGatherRuleDebugLog = false,
|
||||
Collectors = CollectorConfiguration.CreateDefault(),
|
||||
GatherRules = new System.Collections.Generic.List<GatherRule>(),
|
||||
ImeLogPatterns = new System.Collections.Generic.List<ImeLogPattern>()
|
||||
|
||||
+81
-22
@@ -631,10 +631,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
/// the regular <see cref="ProcessRecord"/> pipeline with isBackfill=true, so they arrive
|
||||
/// as modern_deployment_error events carrying backfilled=true plus the original
|
||||
/// timeCreated; the event's own timestamp is emission time, so session StartedAt is not
|
||||
/// dragged backwards. Newest-first, capped, watermark-deduped across restarts.
|
||||
/// dragged backwards. The scan reads newest-first (the cap keeps the newest records),
|
||||
/// but replay is buffered so an Info summary event announces the block BEFORE the errors
|
||||
/// hit the timeline (customer feedback 2026-07-28: an unannounced wall of pre-agent
|
||||
/// Error events reads like a live failure) and the records emit oldest-first, giving the
|
||||
/// timeline sequence the original event-log chronology. Watermark-deduped across restarts.
|
||||
/// </summary>
|
||||
private void BackfillAutopilotErrors()
|
||||
{
|
||||
var buffered = new List<EventRecord>();
|
||||
try
|
||||
{
|
||||
long watermark;
|
||||
@@ -649,7 +654,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
ReverseDirection = true
|
||||
};
|
||||
|
||||
int processed = 0, skippedWatermark = 0, droppedOverCap = 0;
|
||||
int skippedWatermark = 0, droppedOverCap = 0;
|
||||
long maxSeen = watermark;
|
||||
|
||||
using (var reader = new EventLogReader(query))
|
||||
@@ -657,37 +662,48 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
EventRecord record;
|
||||
while ((record = reader.ReadEvent()) != null)
|
||||
{
|
||||
using (record)
|
||||
var recordId = record.RecordId ?? 0;
|
||||
if (recordId > maxSeen) maxSeen = recordId;
|
||||
|
||||
if (recordId != 0 && recordId <= watermark)
|
||||
{
|
||||
var recordId = record.RecordId ?? 0;
|
||||
if (recordId > maxSeen) maxSeen = recordId;
|
||||
|
||||
if (recordId != 0 && recordId <= watermark)
|
||||
{
|
||||
skippedWatermark++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (processed >= ErrorBackfillMaxRecords)
|
||||
{
|
||||
droppedOverCap++;
|
||||
continue;
|
||||
}
|
||||
|
||||
processed++;
|
||||
ProcessRecord(record, "Autopilot", AutopilotChannel, isBackfill: true);
|
||||
skippedWatermark++;
|
||||
record.Dispose();
|
||||
continue;
|
||||
}
|
||||
|
||||
if (buffered.Count >= ErrorBackfillMaxRecords)
|
||||
{
|
||||
droppedOverCap++;
|
||||
record.Dispose();
|
||||
continue;
|
||||
}
|
||||
|
||||
buffered.Add(record);
|
||||
}
|
||||
}
|
||||
|
||||
if (maxSeen > watermark)
|
||||
AdvanceAutopilotErrorWatermark(maxSeen);
|
||||
|
||||
if (processed == 0 && droppedOverCap == 0)
|
||||
if (buffered.Count == 0 && droppedOverCap == 0)
|
||||
{
|
||||
_logger.Debug($"ModernDeployment error backfill: nothing new (skippedByWatermark={skippedWatermark})");
|
||||
}
|
||||
else
|
||||
_logger.Info($"ModernDeployment error backfill: forwarded {processed} record(s) " +
|
||||
{
|
||||
EmitErrorBackfillSummary(buffered.Count, droppedOverCap);
|
||||
|
||||
// Scan buffered newest-first — replay in reverse so the emitted sequence
|
||||
// follows the original event-log chronology.
|
||||
for (var i = buffered.Count - 1; i >= 0; i--)
|
||||
{
|
||||
ProcessRecord(buffered[i], "Autopilot", AutopilotChannel, isBackfill: true);
|
||||
}
|
||||
|
||||
_logger.Info($"ModernDeployment error backfill: forwarded {buffered.Count} record(s) " +
|
||||
$"(skippedByWatermark={skippedWatermark}, droppedOverCap={droppedOverCap})");
|
||||
}
|
||||
}
|
||||
catch (EventLogNotFoundException)
|
||||
{
|
||||
@@ -701,6 +717,49 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
{
|
||||
_logger.Warning($"ModernDeployment error backfill failed: {ex.Message}");
|
||||
}
|
||||
finally
|
||||
{
|
||||
foreach (var record in buffered)
|
||||
{
|
||||
try { record.Dispose(); } catch { }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Announces the error-backfill block on the session timeline BEFORE the replayed
|
||||
/// errors, so pre-agent Error events are readable as history rather than a live
|
||||
/// failure. The web timeline additionally badges each replayed event via its
|
||||
/// backfilled=true payload marker; this summary carries backfillSummary=true instead —
|
||||
/// it is a live emission, not a replayed record.
|
||||
/// </summary>
|
||||
internal void EmitErrorBackfillSummary(int replayedCount, int droppedOverCap)
|
||||
{
|
||||
var message = $"[Autopilot] Replaying {replayedCount} error event(s) recorded in the Autopilot event log " +
|
||||
$"before the agent started (lookback {ErrorBackfillLookbackMinutes / 60}h). " +
|
||||
"These predate monitoring — if enrollment progressed afterwards, they were already resolved by a retry.";
|
||||
if (droppedOverCap > 0)
|
||||
{
|
||||
message += $" {droppedOverCap} older record(s) beyond the {ErrorBackfillMaxRecords}-record cap were not replayed.";
|
||||
}
|
||||
|
||||
_post.Emit(new EnrollmentEvent
|
||||
{
|
||||
SessionId = _sessionId,
|
||||
TenantId = _tenantId,
|
||||
EventType = Constants.EventTypes.ModernDeploymentLog,
|
||||
Severity = EventSeverity.Info,
|
||||
Source = "ModernDeploymentWatcher",
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Message = message,
|
||||
Data = new Dictionary<string, object>
|
||||
{
|
||||
{ "backfillSummary", true },
|
||||
{ "replayedCount", replayedCount },
|
||||
{ "droppedOverCap", droppedOverCap },
|
||||
{ "lookbackMinutes", ErrorBackfillLookbackMinutes }
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
|
||||
+13
-8
@@ -218,15 +218,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
|
||||
/// This method is non-fatal: all exceptions are caught and logged.
|
||||
/// </summary>
|
||||
/// <param name="enrollmentSucceeded">
|
||||
/// True for a successful enrollment (affects OnFailure mode check and sessioninfo.txt content).
|
||||
/// Pass true for WhiteGlove pre-provisioning (it succeeded up to this point).
|
||||
/// True for a successful enrollment, false for a failed one (drives the OnFailure mode
|
||||
/// check and sessioninfo.txt content). Pass true for WhiteGlove pre-provisioning (it
|
||||
/// succeeded up to this point). Pass <b>null</b> when the session has no outcome yet
|
||||
/// (on-demand server-requested collection mid-enrollment): the OnFailure gate does not
|
||||
/// apply — there is no success to skip on — and sessioninfo.txt records "In Progress".
|
||||
/// </param>
|
||||
/// <param name="fileNameSuffix">
|
||||
/// Optional suffix inserted before the .zip extension.
|
||||
/// Example: "preprov" → AgentDiagnostics-{sessionId}-{timestamp}-preprov.zip
|
||||
/// Null (default) → AgentDiagnostics-{sessionId}-{timestamp}.zip
|
||||
/// </param>
|
||||
public virtual async Task<DiagnosticsUploadResult> CreateAndUploadAsync(bool enrollmentSucceeded, string fileNameSuffix = null)
|
||||
public virtual async Task<DiagnosticsUploadResult> CreateAndUploadAsync(bool? enrollmentSucceeded, string fileNameSuffix = null)
|
||||
{
|
||||
try
|
||||
{
|
||||
@@ -244,13 +247,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
|
||||
return null;
|
||||
}
|
||||
|
||||
if (string.Equals(mode, "OnFailure", StringComparison.OrdinalIgnoreCase) && enrollmentSucceeded)
|
||||
// Null outcome (on-demand mid-session) passes through: the OnFailure gate only
|
||||
// skips a KNOWN success — an in-flight session has nothing to skip on.
|
||||
if (string.Equals(mode, "OnFailure", StringComparison.OrdinalIgnoreCase) && enrollmentSucceeded == true)
|
||||
{
|
||||
_logger.Info("Diagnostics upload skipped: enrollment succeeded and mode=OnFailure");
|
||||
return null;
|
||||
}
|
||||
|
||||
_logger.Info($"Creating diagnostics package (mode={mode}, enrollmentSucceeded={enrollmentSucceeded}{(fileNameSuffix != null ? $", suffix={fileNameSuffix}" : "")})...");
|
||||
_logger.Info($"Creating diagnostics package (mode={mode}, enrollmentSucceeded={(enrollmentSucceeded.HasValue ? enrollmentSucceeded.ToString() : "n/a")}{(fileNameSuffix != null ? $", suffix={fileNameSuffix}" : "")})...");
|
||||
|
||||
// Build ZIP in memory
|
||||
var timestamp = DateTime.UtcNow.ToString("yyyyMMddHHmmss");
|
||||
@@ -343,7 +348,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
|
||||
|
||||
// Builds the diagnostics ZIP body in-memory. Extracted from CreateAndUploadAsync so
|
||||
// tests can assert archive contents without going through the upload path.
|
||||
internal virtual byte[] BuildArchiveBytes(bool enrollmentSucceeded)
|
||||
internal virtual byte[] BuildArchiveBytes(bool? enrollmentSucceeded)
|
||||
{
|
||||
var tracker = new BudgetTracker(Budget);
|
||||
using (var ms = new MemoryStream())
|
||||
@@ -414,14 +419,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
|
||||
}
|
||||
}
|
||||
|
||||
private void AddSessionInfo(ZipArchive archive, bool enrollmentSucceeded)
|
||||
private void AddSessionInfo(ZipArchive archive, bool? enrollmentSucceeded)
|
||||
{
|
||||
var sb = new StringBuilder();
|
||||
sb.AppendLine($"Session ID: {_configuration.SessionId}");
|
||||
sb.AppendLine($"Tenant ID: {_configuration.TenantId}");
|
||||
sb.AppendLine($"Device Name: {Environment.MachineName}");
|
||||
sb.AppendLine($"Timestamp: {DateTime.UtcNow:O}");
|
||||
sb.AppendLine($"Enrollment Result: {(enrollmentSucceeded ? "Succeeded" : "Failed")}");
|
||||
sb.AppendLine($"Enrollment Result: {(enrollmentSucceeded == null ? "In Progress" : enrollmentSucceeded.Value ? "Succeeded" : "Failed")}");
|
||||
|
||||
// Hardware info via existing DeviceInfoProvider (WMI)
|
||||
sb.AppendLine($"Manufacturer: {DeviceInfoProvider.GetManufacturer()}");
|
||||
|
||||
+44
-1
@@ -25,14 +25,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
{
|
||||
var filePath = rule.Target;
|
||||
if (string.IsNullOrEmpty(filePath))
|
||||
{
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser, "empty target — nothing to parse");
|
||||
return null;
|
||||
}
|
||||
|
||||
// Expand custom tokens (%LOGGED_ON_USER_PROFILE%) and standard environment variables
|
||||
var userProfilePath = UserProfileResolver.ContainsUserProfileToken(filePath)
|
||||
? UserProfileResolver.GetLoggedOnUserProfilePath() : null;
|
||||
filePath = UserProfileResolver.ExpandCustomTokens(filePath);
|
||||
if (filePath == null)
|
||||
{
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
|
||||
"target contains %LOGGED_ON_USER_PROFILE% but no user is logged on — skipped this run");
|
||||
return null; // Token present but no user logged on — skip silently
|
||||
}
|
||||
|
||||
// Apply IME log path override if set. The override comes from the local
|
||||
// --ime-log-path flag only (never from remote config), so the operator
|
||||
@@ -66,6 +73,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
string.IsNullOrEmpty(patternStr))
|
||||
{
|
||||
context.Logger.Warning($"LogParser rule {rule.RuleId} has no 'pattern' parameter");
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageError,
|
||||
"missing 'pattern' parameter — rule can never match");
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -93,6 +102,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
catch (Exception ex)
|
||||
{
|
||||
context.Logger.Warning($"LogParser rule {rule.RuleId} has invalid regex: {ex.Message}");
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageError,
|
||||
$"invalid regex pattern — rule can never match: {ex.Message}");
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -101,6 +112,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
if (resolvedPaths.Count == 0)
|
||||
{
|
||||
context.Logger.Debug($"LogParser rule {rule.RuleId}: no files found for: {filePath}");
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
|
||||
$"no files matched target {filePath}" + (HasWildcard(Path.GetFileName(filePath)) ? " (wildcard)" : "") + " — nothing to parse");
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -144,7 +157,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
try
|
||||
{
|
||||
// Return matched files sorted by last write time (newest first), capped at 20
|
||||
return Directory.GetFiles(directory, fileNamePart)
|
||||
var allMatches = Directory.GetFiles(directory, fileNamePart);
|
||||
if (allMatches.Length > 20)
|
||||
context.DebugLog(ruleId, GatherRuleDebugLog.StageLogParser,
|
||||
$"wildcard matched {allMatches.Length} files — capped to the 20 newest");
|
||||
return allMatches
|
||||
.OrderByDescending(f => new FileInfo(f).LastWriteTimeUtc)
|
||||
.Take(20)
|
||||
.ToList();
|
||||
@@ -152,6 +169,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
catch (Exception ex)
|
||||
{
|
||||
context.Logger.Warning($"LogParser rule {ruleId}: wildcard expansion failed for {filePath}: {ex.Message}");
|
||||
context.DebugLog(ruleId, GatherRuleDebugLog.StageError, $"wildcard expansion failed for {filePath}: {ex.Message}");
|
||||
return new List<string>();
|
||||
}
|
||||
}
|
||||
@@ -168,10 +186,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
|
||||
// Nothing new to read
|
||||
if (startPosition >= fileInfo.Length)
|
||||
{
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
|
||||
$"{Path.GetFileName(filePath)}: no new content (position {startPosition} >= length {fileInfo.Length}) — trackPosition=true, nothing re-read");
|
||||
return;
|
||||
}
|
||||
|
||||
int matchCount = 0;
|
||||
int linesRead = 0;
|
||||
int parseFailCount = 0;
|
||||
int timeoutCount = 0;
|
||||
long endPosition = startPosition;
|
||||
|
||||
using (var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
|
||||
@@ -195,6 +219,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
}
|
||||
catch (RegexMatchTimeoutException)
|
||||
{
|
||||
timeoutCount++;
|
||||
continue;
|
||||
}
|
||||
if (!match.Success)
|
||||
@@ -239,7 +264,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
// CMTrace mode: parse line as CMTrace format, match regex against message
|
||||
CmTraceLogEntry entry;
|
||||
if (!CmTraceLogParser.TryParseLine(line, out entry))
|
||||
{
|
||||
parseFailCount++;
|
||||
continue;
|
||||
}
|
||||
|
||||
Match match;
|
||||
try
|
||||
@@ -248,6 +276,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
}
|
||||
catch (RegexMatchTimeoutException)
|
||||
{
|
||||
timeoutCount++;
|
||||
continue;
|
||||
}
|
||||
if (!match.Success)
|
||||
@@ -301,10 +330,24 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
|
||||
if (matchCount > 0)
|
||||
context.Logger.Debug($"LogParser rule {rule.RuleId}: {matchCount} matches from {linesRead} lines in {Path.GetFileName(filePath)}");
|
||||
|
||||
// Per-file outcome — always written so a "matched 0" run is visible too.
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
|
||||
$"{Path.GetFileName(filePath)}: read {linesRead} lines from position {startPosition}->{endPosition}, " +
|
||||
$"matched {matchCount}, parseFailures={parseFailCount}, regexTimeouts={timeoutCount}, mode={(isTextMode ? "text" : "cmtrace")}");
|
||||
|
||||
if (!isTextMode && linesRead > 0 && parseFailCount == linesRead)
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
|
||||
"every line failed CMTrace parsing — if this is a plain-text log, set parameter format=text");
|
||||
|
||||
if (linesRead == maxLines)
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
|
||||
$"stopped at maxLines={maxLines} — remaining content is deferred to the next run");
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
context.Logger.Warning($"LogParser rule {rule.RuleId} failed reading {filePath}: {ex.Message}");
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageError, $"failed reading {filePath}: {ex}");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+4
@@ -113,7 +113,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
|
||||
|
||||
// Suppress the "not found" event when the rule only wants a positive signal.
|
||||
if (emitOnlyIfExists)
|
||||
{
|
||||
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageCollector,
|
||||
$"registry key/value not found: {path} — emitOnlyIfExists=true, empty result");
|
||||
return new Dictionary<string, object>(); // empty → executor emits nothing
|
||||
}
|
||||
|
||||
data["exists"] = false;
|
||||
data["path"] = path;
|
||||
|
||||
+16
-1
@@ -22,13 +22,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
public LogFilePositionTracker FilePositionTracker { get; }
|
||||
public bool UnrestrictedMode { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Optional gather-rule debug trace writer (EnableGatherRuleDebugLog). Null when
|
||||
/// tracing is disabled — use <see cref="DebugLog"/> for null-safe writes.
|
||||
/// </summary>
|
||||
public GatherRuleDebugLog DebugWriter { get; }
|
||||
|
||||
public GatherRuleContext(
|
||||
AgentLogger logger,
|
||||
string sessionId,
|
||||
string tenantId,
|
||||
Action<EnrollmentEvent> onEventCollected,
|
||||
string imeLogPathOverride,
|
||||
LogFilePositionTracker filePositionTracker)
|
||||
LogFilePositionTracker filePositionTracker,
|
||||
GatherRuleDebugLog debugLog = null)
|
||||
{
|
||||
Logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
SessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
|
||||
@@ -36,8 +43,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
OnEventCollected = onEventCollected ?? throw new ArgumentNullException(nameof(onEventCollected));
|
||||
ImeLogPathOverride = imeLogPathOverride;
|
||||
FilePositionTracker = filePositionTracker ?? throw new ArgumentNullException(nameof(filePositionTracker));
|
||||
DebugWriter = debugLog;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Null-safe write to the gather-rule debug trace. No-op when tracing is disabled.
|
||||
/// </summary>
|
||||
public void DebugLog(string ruleId, string stage, string message) => DebugWriter?.Write(ruleId, stage, message);
|
||||
|
||||
/// <summary>
|
||||
/// Emits a security_warning event and returns an empty result dictionary.
|
||||
/// Called when a gather rule targets a path/query not on the allowlist.
|
||||
@@ -45,6 +58,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
public Dictionary<string, object> EmitSecurityWarning(GatherRule rule, string collectorType, string target)
|
||||
{
|
||||
Logger.Warning($"SECURITY: {collectorType} path blocked by guard: {target} (Rule: {rule.RuleId})");
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageGuard,
|
||||
$"{collectorType} target blocked by allowlist guard: {target} — security_warning emitted, rule produced no data");
|
||||
|
||||
var data = new Dictionary<string, object>
|
||||
{
|
||||
|
||||
+115
@@ -0,0 +1,115 @@
|
||||
using System;
|
||||
using System.IO;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
{
|
||||
/// <summary>
|
||||
/// Append-only trace file for gather-rule evaluation, enabled per tenant via
|
||||
/// EnableGatherRuleDebugLog (or --gather-debug-log). Explains every evaluation
|
||||
/// decision — including the outcomes that produce no event — so customers can
|
||||
/// diagnose rules that never show up in the timeline. Every write failure is
|
||||
/// swallowed: tracing must never break gathering.
|
||||
/// </summary>
|
||||
public sealed class GatherRuleDebugLog
|
||||
{
|
||||
public const string StageConfig = "config";
|
||||
public const string StageTrigger = "trigger";
|
||||
public const string StageScope = "scope";
|
||||
public const string StageExec = "exec";
|
||||
public const string StageCollector = "collector";
|
||||
public const string StageGuard = "guard";
|
||||
public const string StageEmit = "emit";
|
||||
public const string StageSuppress = "suppress";
|
||||
public const string StageLogParser = "logparser";
|
||||
public const string StageError = "error";
|
||||
|
||||
private const long DefaultMaxBytes = 10 * 1024 * 1024;
|
||||
|
||||
private readonly string _path;
|
||||
private readonly long _maxBytes;
|
||||
private readonly Action<string> _echo;
|
||||
private readonly object _lock = new object();
|
||||
private bool _headerWritten;
|
||||
|
||||
public GatherRuleDebugLog(string path, AgentLogger logger, Action<string> echo = null, long maxBytes = DefaultMaxBytes)
|
||||
{
|
||||
if (string.IsNullOrEmpty(path)) throw new ArgumentNullException(nameof(path));
|
||||
_path = path;
|
||||
_maxBytes = maxBytes;
|
||||
_echo = echo;
|
||||
logger?.Info($"GatherRuleDebugLog: enabled -> {path}");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Writes one trace line: <c>{utc} | {ruleId} | {stage} | {message}</c>.
|
||||
/// Newlines in <paramref name="message"/> are flattened except for the error
|
||||
/// stage, where exception stack traces stay multi-line.
|
||||
/// </summary>
|
||||
public void Write(string ruleId, string stage, string message)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (stage != StageError && message != null && message.IndexOf('\n') >= 0)
|
||||
message = message.Replace("\r", "").Replace('\n', ' ');
|
||||
|
||||
var line = $"{DateTime.UtcNow:yyyy-MM-ddTHH:mm:ss.fffZ} | {ruleId ?? "-"} | {stage} | {message}";
|
||||
|
||||
lock (_lock)
|
||||
{
|
||||
EnsureFileReadyLocked();
|
||||
File.AppendAllText(_path, line + Environment.NewLine);
|
||||
}
|
||||
|
||||
_echo?.Invoke(line);
|
||||
}
|
||||
catch { }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// One-shot helper for callers that have no executor (e.g. the flag is on but the
|
||||
/// backend delivered zero gather rules) — the customer's trace file should explain
|
||||
/// that too instead of silently not existing.
|
||||
/// </summary>
|
||||
public static void WriteStandalone(string path, string message, AgentLogger logger)
|
||||
{
|
||||
if (string.IsNullOrEmpty(path)) return;
|
||||
try
|
||||
{
|
||||
new GatherRuleDebugLog(path, logger).Write(null, StageConfig, message);
|
||||
}
|
||||
catch { }
|
||||
}
|
||||
|
||||
// Lazy on first write: create the directory and emit a header line. Before every
|
||||
// append, rotate when the file exceeds maxBytes — keeping exactly one .old
|
||||
// generation (worst case ~2x maxBytes on disk).
|
||||
private void EnsureFileReadyLocked()
|
||||
{
|
||||
if (!_headerWritten)
|
||||
{
|
||||
var dir = Path.GetDirectoryName(_path);
|
||||
if (!string.IsNullOrEmpty(dir))
|
||||
Directory.CreateDirectory(dir);
|
||||
}
|
||||
|
||||
var info = new FileInfo(_path);
|
||||
if (info.Exists && info.Length > _maxBytes)
|
||||
{
|
||||
var oldPath = Path.ChangeExtension(_path, null) + ".old.log";
|
||||
if (File.Exists(oldPath))
|
||||
File.Delete(oldPath);
|
||||
File.Move(_path, oldPath);
|
||||
File.AppendAllText(_path,
|
||||
$"{DateTime.UtcNow:yyyy-MM-ddTHH:mm:ss.fffZ} | - | {StageConfig} | -- rotated (previous file exceeded {_maxBytes} bytes) --" + Environment.NewLine);
|
||||
}
|
||||
|
||||
if (!_headerWritten)
|
||||
{
|
||||
File.AppendAllText(_path,
|
||||
$"{DateTime.UtcNow:yyyy-MM-ddTHH:mm:ss.fffZ} | - | {StageConfig} | -- gather rule debug log started --" + Environment.NewLine);
|
||||
_headerWritten = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+177
-11
@@ -22,6 +22,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
public class GatherRuleExecutor : IDisposable
|
||||
{
|
||||
private readonly AgentLogger _logger;
|
||||
private readonly GatherRuleDebugLog _debug; // null unless EnableGatherRuleDebugLog / --gather-debug-log
|
||||
private readonly GatherRuleContext _context;
|
||||
private readonly Dictionary<string, IGatherRuleCollector> _collectors;
|
||||
|
||||
@@ -61,7 +62,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
}
|
||||
|
||||
public GatherRuleExecutor(string sessionId, string tenantId, Action<EnrollmentEvent> onEventCollected,
|
||||
AgentLogger logger, string imeLogPathOverride = null)
|
||||
AgentLogger logger, string imeLogPathOverride = null, string debugLogPath = null,
|
||||
Action<string> debugEcho = null)
|
||||
{
|
||||
if (sessionId == null) throw new ArgumentNullException(nameof(sessionId));
|
||||
if (tenantId == null) throw new ArgumentNullException(nameof(tenantId));
|
||||
@@ -69,9 +71,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
if (logger == null) throw new ArgumentNullException(nameof(logger));
|
||||
|
||||
_logger = logger;
|
||||
_debug = string.IsNullOrEmpty(debugLogPath) ? null : new GatherRuleDebugLog(debugLogPath, logger, debugEcho);
|
||||
|
||||
var filePositionTracker = new LogFilePositionTracker();
|
||||
_context = new GatherRuleContext(logger, sessionId, tenantId, onEventCollected, imeLogPathOverride, filePositionTracker);
|
||||
_context = new GatherRuleContext(logger, sessionId, tenantId, onEventCollected, imeLogPathOverride, filePositionTracker, _debug);
|
||||
|
||||
// Register all collector strategies
|
||||
var collectorList = new IGatherRuleCollector[]
|
||||
@@ -96,6 +99,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
_collectors["command"] = _collectors["command_allowlisted"];
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Null-safe write to the gather-rule debug trace. Never call while holding
|
||||
/// <see cref="_scopeLock"/> — capture the message under the lock, write after release.
|
||||
/// </summary>
|
||||
private void DebugLog(string ruleId, string stage, string message) => _debug?.Write(ruleId, stage, message);
|
||||
|
||||
/// <summary>
|
||||
/// Updates the active rules and starts/stops execution accordingly
|
||||
/// </summary>
|
||||
@@ -111,11 +120,43 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
|
||||
_activeRules = rules.Where(r => r.Enabled).ToList();
|
||||
|
||||
// Registration summary for the debug trace: one line per delivered rule so the
|
||||
// customer can see exactly what the agent received and how it will be triggered.
|
||||
if (_debug != null)
|
||||
{
|
||||
foreach (var rule in rules)
|
||||
{
|
||||
if (!rule.Enabled)
|
||||
{
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageConfig, "rule disabled — will never run");
|
||||
continue;
|
||||
}
|
||||
|
||||
var scope = rule.ActivePhases != null && rule.ActivePhases.Count > 0
|
||||
? $"activePhases=[{string.Join(",", rule.ActivePhases)}]"
|
||||
: !string.IsNullOrEmpty(rule.ActiveFromPhase)
|
||||
? $"activeFromPhase={rule.ActiveFromPhase}"
|
||||
: "unscoped";
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageConfig,
|
||||
$"registered: trigger={rule.Trigger}" +
|
||||
(string.IsNullOrEmpty(rule.TriggerPhase) ? "" : $" triggerPhase={rule.TriggerPhase}") +
|
||||
(string.IsNullOrEmpty(rule.TriggerEventType) ? "" : $" triggerEventType={rule.TriggerEventType}") +
|
||||
$", collector={rule.CollectorType}, target={rule.Target}, {scope}" +
|
||||
$", emitMode={(string.IsNullOrEmpty(rule.EmitMode) ? "always" : rule.EmitMode)}" +
|
||||
(rule.IntervalSeconds.HasValue ? $", interval={rule.IntervalSeconds}s" : ""));
|
||||
|
||||
if (rule.Trigger == "interval" && !rule.IntervalSeconds.HasValue)
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageError,
|
||||
"trigger=interval but intervalSeconds is missing — rule will NEVER run");
|
||||
}
|
||||
}
|
||||
|
||||
// Execute startup rules — track completion via CountdownEvent so callers can wait.
|
||||
// Scoped startup rules that are not yet in scope are deferred: they run once from
|
||||
// OnPhaseChanged when their scope activates (same _startupRulesExecuted dedup — a
|
||||
// rule runs either here or there, never both). Unscoped rules behave as before.
|
||||
List<GatherRule> pendingStartup;
|
||||
var deferredTraces = _debug != null ? new List<KeyValuePair<string, string>>() : null;
|
||||
lock (_scopeLock)
|
||||
{
|
||||
// A config refresh may deliver new from-phase rules after their phase was
|
||||
@@ -124,7 +165,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
|
||||
pendingStartup = _activeRules
|
||||
.Where(r => r.Trigger == "startup" && !_startupRulesExecuted.Contains(r.RuleId))
|
||||
.Where(r => IsRuleInScopeLocked(r))
|
||||
.Where(r =>
|
||||
{
|
||||
string reason;
|
||||
var inScope = IsRuleInScopeLocked(r, out reason);
|
||||
if (!inScope && deferredTraces != null)
|
||||
deferredTraces.Add(new KeyValuePair<string, string>(r.RuleId,
|
||||
$"startup rule deferred — {reason}; runs once when its scope activates"));
|
||||
return inScope;
|
||||
})
|
||||
.ToList();
|
||||
|
||||
foreach (var rule in pendingStartup)
|
||||
@@ -133,6 +182,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
}
|
||||
}
|
||||
|
||||
if (deferredTraces != null)
|
||||
{
|
||||
foreach (var trace in deferredTraces)
|
||||
DebugLog(trace.Key, GatherRuleDebugLog.StageScope, trace.Value);
|
||||
}
|
||||
|
||||
if (pendingStartup.Count > 0)
|
||||
{
|
||||
_startupRulesLatch?.Dispose();
|
||||
@@ -158,8 +213,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
var timer = new Timer(
|
||||
_ =>
|
||||
{
|
||||
if (!IsRuleInScope(intervalRule))
|
||||
string skipReason;
|
||||
if (!IsRuleInScope(intervalRule, out skipReason))
|
||||
{
|
||||
DebugLog(intervalRule.RuleId, GatherRuleDebugLog.StageScope, $"interval tick skipped: {skipReason}");
|
||||
return;
|
||||
}
|
||||
ExecuteRule(intervalRule);
|
||||
},
|
||||
null,
|
||||
@@ -168,6 +227,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
);
|
||||
_intervalTimers[intervalRule.RuleId] = timer;
|
||||
_logger.Info($" Interval rule {intervalRule.RuleId} scheduled every {intervalRule.IntervalSeconds}s");
|
||||
DebugLog(intervalRule.RuleId, GatherRuleDebugLog.StageTrigger,
|
||||
$"interval timer scheduled every {intervalRule.IntervalSeconds}s (first run after one full interval — no immediate execution)");
|
||||
}
|
||||
|
||||
_logger.Info($"GatherRuleExecutor: {_activeRules.Count(r => r.Trigger == "startup")} startup, " +
|
||||
@@ -187,6 +248,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
// _currentPhase moves below. A transition INTO Failed still fires the exit rules of the
|
||||
// phase that failed — that snapshot is the whole point at a failure boundary.
|
||||
List<GatherRule> exitRules = null;
|
||||
var exitTraces = _debug != null ? new List<KeyValuePair<string, string>>() : null;
|
||||
lock (_scopeLock)
|
||||
{
|
||||
var previousPhase = _currentPhase;
|
||||
@@ -204,12 +266,22 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
|
||||
// Scope gate before the dedup add — an out-of-scope pass must not consume
|
||||
// the once-per-(rule, phase) slot. IsRuleInScopeLocked still sees the old phase.
|
||||
if (!IsRuleInScopeLocked(rule)) continue;
|
||||
string scopeReason;
|
||||
if (!IsRuleInScopeLocked(rule, out scopeReason))
|
||||
{
|
||||
if (exitTraces != null)
|
||||
exitTraces.Add(new KeyValuePair<string, string>(rule.RuleId,
|
||||
$"phase_exit of {previousPhaseName} skipped: {scopeReason}"));
|
||||
continue;
|
||||
}
|
||||
|
||||
// "exit:" prefix keeps this key space disjoint from the phase_change keys.
|
||||
if (!_phaseRulesExecuted.Add($"exit:{rule.RuleId}|{previousPhaseName}"))
|
||||
{
|
||||
_logger.Debug($"Phase-exit rule {rule.RuleId} already executed for exit of {previousPhaseName}, skipping");
|
||||
if (exitTraces != null)
|
||||
exitTraces.Add(new KeyValuePair<string, string>(rule.RuleId,
|
||||
$"phase_exit already executed for exit of {previousPhaseName} — once per (rule, phase)"));
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -219,6 +291,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
}
|
||||
}
|
||||
|
||||
if (exitTraces != null)
|
||||
{
|
||||
foreach (var trace in exitTraces)
|
||||
DebugLog(trace.Key, GatherRuleDebugLog.StageScope, trace.Value);
|
||||
}
|
||||
|
||||
if (exitRules != null)
|
||||
{
|
||||
foreach (var rule in exitRules)
|
||||
@@ -252,6 +330,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
{
|
||||
var deferredRule = rule;
|
||||
_logger.Info($"Phase {phaseName} activated deferred startup rule {deferredRule.RuleId}");
|
||||
DebugLog(deferredRule.RuleId, GatherRuleDebugLog.StageTrigger,
|
||||
$"phase {phaseName} activated deferred startup rule");
|
||||
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(deferredRule));
|
||||
}
|
||||
|
||||
@@ -262,18 +342,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
{
|
||||
// Scope gate BEFORE the dedup add — an out-of-scope pass must not consume
|
||||
// the once-per-(rule, phase) execution slot.
|
||||
if (!IsRuleInScope(rule))
|
||||
string scopeReason;
|
||||
if (!IsRuleInScope(rule, out scopeReason))
|
||||
{
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageScope,
|
||||
$"phase_change to {phaseName} skipped: {scopeReason}");
|
||||
continue;
|
||||
}
|
||||
|
||||
// Deduplicate: only fire once per (ruleId, phase) combination
|
||||
var deduplicationKey = $"{rule.RuleId}|{phaseName}";
|
||||
if (!_phaseRulesExecuted.Add(deduplicationKey))
|
||||
{
|
||||
_logger.Debug($"Phase rule {rule.RuleId} already executed for phase {phaseName}, skipping");
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageTrigger,
|
||||
$"phase_change already executed for phase {phaseName} — once per (rule, phase)");
|
||||
continue;
|
||||
}
|
||||
|
||||
_logger.Info($"Phase change triggered rule {rule.RuleId} (phase: {phaseName})");
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageTrigger, $"phase_change to {phaseName} fired");
|
||||
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(rule));
|
||||
}
|
||||
}
|
||||
@@ -288,10 +376,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
{
|
||||
if (string.Equals(rule.TriggerEventType, eventType, StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
if (!IsRuleInScope(rule))
|
||||
string scopeReason;
|
||||
if (!IsRuleInScope(rule, out scopeReason))
|
||||
{
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageScope,
|
||||
$"on_event {eventType} skipped: {scopeReason}");
|
||||
continue;
|
||||
}
|
||||
|
||||
_logger.Info($"Event triggered rule {rule.RuleId} (event: {eventType})");
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageTrigger, $"on_event {eventType} fired");
|
||||
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(rule));
|
||||
}
|
||||
}
|
||||
@@ -302,6 +396,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
try
|
||||
{
|
||||
_logger.Info($"Executing gather rule: {rule.RuleId} ({rule.Title})");
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageExec, $"executing: collector={rule.CollectorType}, target={rule.Target}");
|
||||
|
||||
var collectorType = rule.CollectorType?.ToLower();
|
||||
|
||||
@@ -309,6 +404,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
if (collectorType == null || !_collectors.TryGetValue(collectorType, out collector))
|
||||
{
|
||||
_logger.Warning($"Unknown collector type: {rule.CollectorType} for rule {rule.RuleId}");
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageError, $"unknown collector type '{rule.CollectorType}' — nothing emitted");
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -316,7 +412,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
|
||||
// LogParser (and potentially others) emit events directly and return null
|
||||
if (result == null)
|
||||
{
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageCollector,
|
||||
collectorType == "logparser"
|
||||
? "logparser emits per-match events directly — see logparser lines above for the per-file outcome"
|
||||
: "collector returned null — nothing emitted");
|
||||
return;
|
||||
}
|
||||
|
||||
if (result.Count == 0)
|
||||
{
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageCollector,
|
||||
"collector returned EMPTY result — nothing emitted (typical causes: target not found with emitOnlyIfExists, or guard block — see guard line if present)");
|
||||
}
|
||||
|
||||
if (result.Count > 0)
|
||||
{
|
||||
@@ -325,8 +433,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
// empty result never reaches this point, so an emitOnlyIfExists miss neither
|
||||
// emits nor updates the hash — composing to "one event on appearance, then
|
||||
// only on change".
|
||||
if (IsOnChangeMode(rule) && !ShouldEmitOnChange(rule, result))
|
||||
return;
|
||||
if (IsOnChangeMode(rule))
|
||||
{
|
||||
string hashPrefix;
|
||||
int suppressedStreak;
|
||||
if (!ShouldEmitOnChange(rule, result, out hashPrefix, out suppressedStreak))
|
||||
{
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageSuppress,
|
||||
$"result unchanged (hash {hashPrefix}) — emit suppressed (on_change), suppressed streak={suppressedStreak}");
|
||||
return;
|
||||
}
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageEmit,
|
||||
$"result changed (hash {hashPrefix})" + (suppressedStreak > 0 ? $" after {suppressedStreak} suppressed polls" : ""));
|
||||
}
|
||||
|
||||
result["ruleId"] = rule.RuleId;
|
||||
result["ruleTitle"] = rule.Title;
|
||||
@@ -357,11 +476,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
Message = $"Gather: {rule.Title}",
|
||||
Data = result
|
||||
});
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageEmit,
|
||||
$"emitted {eventType} (severity={severity}, {result.Count} fields)");
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Warning($"Gather rule {rule.RuleId} failed: {ex.Message}");
|
||||
// Full exception incl. stack trace only in the debug trace — the agent log stays terse.
|
||||
DebugLog(rule.RuleId, GatherRuleDebugLog.StageError, $"rule execution failed: {ex}");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -375,15 +498,33 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
|
||||
// Internal so the scope matrix is testable synchronously (no timer/ThreadPool races).
|
||||
internal bool IsRuleInScope(GatherRule rule)
|
||||
{
|
||||
string reason;
|
||||
return IsRuleInScope(rule, out reason);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Scope gate with a human-readable skip reason for the debug trace. The reason is
|
||||
/// captured under the lock but must be WRITTEN outside it (never trace under _scopeLock).
|
||||
/// </summary>
|
||||
internal bool IsRuleInScope(GatherRule rule, out string reason)
|
||||
{
|
||||
lock (_scopeLock)
|
||||
{
|
||||
return IsRuleInScopeLocked(rule);
|
||||
return IsRuleInScopeLocked(rule, out reason);
|
||||
}
|
||||
}
|
||||
|
||||
private bool IsRuleInScopeLocked(GatherRule rule)
|
||||
{
|
||||
string reason;
|
||||
return IsRuleInScopeLocked(rule, out reason);
|
||||
}
|
||||
|
||||
private bool IsRuleInScopeLocked(GatherRule rule, out string reason)
|
||||
{
|
||||
reason = null;
|
||||
|
||||
if (!HasPhaseScope(rule))
|
||||
return true; // unscoped — legacy behavior
|
||||
|
||||
@@ -400,7 +541,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
// No phase signal yet (or scope tokens that never match, e.g. "Unknown"):
|
||||
// scoped rules stay inactive.
|
||||
if (_currentPhase == EnrollmentPhase.Unknown)
|
||||
{
|
||||
reason = "currentPhase=Unknown — scoped rules stay inactive until the first phase signal";
|
||||
return false;
|
||||
}
|
||||
|
||||
var phaseName = _currentPhase.ToString();
|
||||
foreach (var phase in rule.ActivePhases)
|
||||
@@ -408,10 +552,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
if (string.Equals(phase, phaseName, StringComparison.OrdinalIgnoreCase))
|
||||
return true;
|
||||
}
|
||||
reason = $"out of scope (currentPhase={phaseName}, activePhases=[{string.Join(",", rule.ActivePhases)}])";
|
||||
return false;
|
||||
}
|
||||
|
||||
return _fromPhaseLatched.Contains(rule.RuleId);
|
||||
if (_fromPhaseLatched.Contains(rule.RuleId))
|
||||
return true;
|
||||
|
||||
reason = $"activeFromPhase={rule.ActiveFromPhase} not latched yet (currentPhase={_currentPhase})";
|
||||
return false;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -457,8 +606,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
/// Internal so the state machine is testable synchronously (no timer/ThreadPool races).
|
||||
/// </summary>
|
||||
internal bool ShouldEmitOnChange(GatherRule rule, Dictionary<string, object> result)
|
||||
{
|
||||
string hashPrefix;
|
||||
int suppressedStreak;
|
||||
return ShouldEmitOnChange(rule, result, out hashPrefix, out suppressedStreak);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Overload exposing the hash prefix and suppression streak for the debug trace:
|
||||
/// on suppress, <paramref name="suppressedStreak"/> is the updated streak count;
|
||||
/// on emit, it is the streak that just ended (0 when none).
|
||||
/// </summary>
|
||||
internal bool ShouldEmitOnChange(GatherRule rule, Dictionary<string, object> result,
|
||||
out string hashPrefix, out int suppressedStreak)
|
||||
{
|
||||
var hash = ComputeCanonicalHash(result);
|
||||
hashPrefix = hash.Length > 8 ? hash.Substring(0, 8) : hash;
|
||||
suppressedStreak = 0;
|
||||
bool emit;
|
||||
lock (_scopeLock)
|
||||
{
|
||||
@@ -469,6 +633,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
_suppressed[rule.RuleId] = _suppressed.TryGetValue(rule.RuleId, out streak)
|
||||
? (streak.Count + 1, streak.SinceUtc)
|
||||
: (1, DateTime.UtcNow);
|
||||
suppressedStreak = _suppressed[rule.RuleId].Count;
|
||||
emit = false;
|
||||
}
|
||||
else
|
||||
@@ -479,6 +644,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
|
||||
{
|
||||
result["suppressedPolls"] = streak.Count;
|
||||
result["suppressedSinceUtc"] = streak.SinceUtc.ToString("o", CultureInfo.InvariantCulture);
|
||||
suppressedStreak = streak.Count;
|
||||
}
|
||||
_suppressed.Remove(rule.RuleId);
|
||||
emit = true;
|
||||
|
||||
@@ -483,7 +483,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
logger: logger,
|
||||
rules: _remoteConfig.GatherRules,
|
||||
imeLogPathOverride: _agentConfig.ImeLogPathOverride,
|
||||
unrestrictedMode: _agentConfig.UnrestrictedMode));
|
||||
unrestrictedMode: _agentConfig.UnrestrictedMode,
|
||||
gatherDebugLogPath: _agentConfig.GatherRuleDebugLogPath));
|
||||
}
|
||||
else if (!string.IsNullOrEmpty(_agentConfig.GatherRuleDebugLogPath))
|
||||
{
|
||||
// Debug trace is on but the backend delivered zero gather rules — the trace
|
||||
// file must say so instead of silently not existing.
|
||||
Monitoring.Telemetry.Gather.GatherRuleDebugLog.WriteStandalone(
|
||||
_agentConfig.GatherRuleDebugLogPath,
|
||||
"no gather rules delivered by backend — nothing to execute", logger);
|
||||
}
|
||||
|
||||
return new CollectorSurfaces(hosts, imeLogHost, espAndHelloHost, aadJoinHost);
|
||||
|
||||
+5
-3
@@ -39,7 +39,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
AgentLogger logger,
|
||||
List<GatherRule> rules,
|
||||
string? imeLogPathOverride,
|
||||
bool unrestrictedMode = false)
|
||||
bool unrestrictedMode = false,
|
||||
string? gatherDebugLogPath = null)
|
||||
{
|
||||
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
|
||||
if (clock == null) throw new ArgumentNullException(nameof(clock));
|
||||
@@ -55,9 +56,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
// session mode we wrap post.Emit so every session-mode gather event still
|
||||
// flows through the InformationalEvent ingress pipe before hitting the
|
||||
// telemetry spool — Rail-A semantics for ordering / replay determinism.
|
||||
var post = new InformationalEventPost(ingress, clock);
|
||||
var post = new InformationalEventPost(ingress, clock, logger);
|
||||
_executor = new Monitoring.Telemetry.Gather.GatherRuleExecutor(
|
||||
sessionId, tenantId, evt => post.Emit(evt), logger, imeLogPathOverride);
|
||||
sessionId, tenantId, evt => post.Emit(evt), logger, imeLogPathOverride,
|
||||
debugLogPath: gatherDebugLogPath);
|
||||
_observableIngress = ingress as SignalIngress;
|
||||
}
|
||||
|
||||
|
||||
+25
-4
@@ -39,8 +39,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
|
||||
/// <see cref="SessionStage.WhiteGloveSealed"/> (Part-1 exit, session resumes Part 2).</item>
|
||||
/// <item>WhiteGlove Part-1 path: stop peripheral collectors first (so their stop-time
|
||||
/// stragglers — e.g. <c>network_bandwidth_estimate</c> — sequence BEFORE the marker),
|
||||
/// then emit <c>whiteglove_part1_complete</c>, drain spool, and write
|
||||
/// <c>whiteglove.complete</c> marker via <see cref="SessionIdPersistence.SaveWhiteGloveComplete"/>
|
||||
/// upload the diagnostics package as a pre-provisioning intermediate (suffix
|
||||
/// <c>preprov</c>, same mode gating as the terminal path — the seal counts as a
|
||||
/// success, so <c>OnFailure</c> skips), then emit <c>whiteglove_part1_complete</c>,
|
||||
/// drain spool, and write <c>whiteglove.complete</c> marker via
|
||||
/// <see cref="SessionIdPersistence.SaveWhiteGloveComplete"/>
|
||||
/// so Part-2 resume is detected on the next boot.</item>
|
||||
/// <item>Signal the caller-owned shutdown <see cref="ManualResetEventSlim"/>.</item>
|
||||
/// </list>
|
||||
@@ -241,6 +244,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
|
||||
// the full Stop() later is a no-op for the hosts.
|
||||
StopPeripheralCollectorsBestEffort();
|
||||
|
||||
// V1 parity (restored 2026-07-28) — upload the diagnostics package as a
|
||||
// pre-provisioning intermediate. Between reseal and user OOBE the device
|
||||
// may sit boxed for weeks, and if Part 2 never uploads (never runs, or
|
||||
// OnFailure + success) the Part-1 forensics would be lost to self-destruct.
|
||||
// No dedicated flag: the existing mode semantics apply — Always uploads,
|
||||
// OnFailure skips (the seal is a success; a Part-1 failure terminates via
|
||||
// the regular Failed path below, which uploads), Off does nothing. Must run
|
||||
// BEFORE the whiteglove_part1_complete emit so diagnostics_* events keep a
|
||||
// lower sequence than the Part-1/Part-2 boundary marker. Best-effort: a
|
||||
// reseal-reboot racing the upload just loses the intermediate package.
|
||||
RunUploadDiagnosticsWithEvents(args, suffixOverride: "preprov");
|
||||
|
||||
EmitEventSafe(new EnrollmentEvent
|
||||
{
|
||||
SessionId = _configuration.SessionId,
|
||||
@@ -777,7 +792,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
|
||||
StringComparison.Ordinal);
|
||||
}
|
||||
|
||||
private void RunUploadDiagnosticsWithEvents(EnrollmentTerminatedEventArgs args)
|
||||
/// <param name="args">Termination payload — outcome drives the OnFailure gate.</param>
|
||||
/// <param name="suffixOverride">
|
||||
/// Blob-name suffix override. Null (terminal path) derives "success"/"failure" from
|
||||
/// the outcome; the WhiteGlove Part-1 seal passes "preprov" so the intermediate
|
||||
/// package is distinguishable from the later Part-2 terminal upload.
|
||||
/// </param>
|
||||
private void RunUploadDiagnosticsWithEvents(EnrollmentTerminatedEventArgs args, string suffixOverride = null)
|
||||
{
|
||||
var mode = _configuration.DiagnosticsUploadMode ?? "Off";
|
||||
if (!_configuration.DiagnosticsUploadEnabled || string.Equals(mode, "Off", StringComparison.OrdinalIgnoreCase))
|
||||
@@ -813,7 +834,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
|
||||
DiagnosticsUploadResult result = null;
|
||||
try
|
||||
{
|
||||
var suffix = enrollmentSucceeded ? "success" : "failure";
|
||||
var suffix = suffixOverride ?? (enrollmentSucceeded ? "success" : "failure");
|
||||
result = _uploadDiagnosticsAsync(enrollmentSucceeded, suffix).GetAwaiter().GetResult();
|
||||
}
|
||||
catch (Exception ex)
|
||||
|
||||
@@ -221,8 +221,18 @@ namespace AutopilotMonitor.Agent.V2
|
||||
},
|
||||
});
|
||||
|
||||
// Step 4 — run the executor and wait.
|
||||
using (var executor = new GatherRuleExecutor(sessionId, tenantId, emitEvent, logger, config.ImeLogPathOverride))
|
||||
// Step 4 — run the executor and wait. Diagnostic mode always writes the
|
||||
// gather-rule debug trace (CLI override or default path) and mirrors it
|
||||
// to the console in --console mode.
|
||||
var gatherDebugLogPath = !string.IsNullOrEmpty(config.GatherRuleDebugLogPath)
|
||||
? config.GatherRuleDebugLogPath
|
||||
: Environment.ExpandEnvironmentVariables(Constants.GatherRuleDebugLogPath);
|
||||
logger.Info($"--run-gather-rules: debug trace -> {gatherDebugLogPath}");
|
||||
if (consoleMode) Console.WriteLine($"Debug trace: {gatherDebugLogPath}");
|
||||
|
||||
using (var executor = new GatherRuleExecutor(sessionId, tenantId, emitEvent, logger, config.ImeLogPathOverride,
|
||||
debugLogPath: gatherDebugLogPath,
|
||||
debugEcho: consoleMode ? (Action<string>)(line => Console.WriteLine($" {line}")) : null))
|
||||
{
|
||||
// Diagnostic mode has no enrollment phase context — scoped rules
|
||||
// (activePhases / activeFromPhase) execute unconditionally here.
|
||||
|
||||
@@ -303,6 +303,7 @@ namespace AutopilotMonitor.Agent.V2
|
||||
var apiBaseUrl = GetArgValue(args, "--api-url", "--backend-api") ?? Constants.ApiBaseUrl;
|
||||
var imeLogPathOverride = GetArgValue(args, "--ime-log-path");
|
||||
var imeMatchLogPath = GetArgValue(args, "--ime-match-log");
|
||||
var gatherDebugLogPath = GetArgValue(args, "--gather-debug-log");
|
||||
|
||||
// Dev / test — IME log replay (V1 compat mode). Feeds ImeLogTracker.SimulationMode
|
||||
// + SpeedFactor so recorded raw IME logs are replayed at an accelerated rate; signal
|
||||
@@ -373,6 +374,7 @@ namespace AutopilotMonitor.Agent.V2
|
||||
EnableGeoLocation = !disableGeoLocation,
|
||||
ImeLogPathOverride = imeLogPathOverride,
|
||||
ImeMatchLogPath = imeMatchLogPath,
|
||||
GatherRuleDebugLogPath = gatherDebugLogPath,
|
||||
KeepLogFile = keepLogFile,
|
||||
SelfDestructOnComplete = !noCleanup,
|
||||
CommandLineArgs = FormatArgsForLog(args),
|
||||
@@ -485,6 +487,7 @@ namespace AutopilotMonitor.Agent.V2
|
||||
Console.Out.WriteLine(" --api-url <URL> Override backend API base URL (alias: --backend-api)");
|
||||
Console.Out.WriteLine(" --ime-log-path <PATH> Override IME logs directory");
|
||||
Console.Out.WriteLine(" --ime-match-log <PATH> Write matched IME log lines to file (debug)");
|
||||
Console.Out.WriteLine(" --gather-debug-log <PATH> Write gather-rule evaluation trace to file (debug)");
|
||||
Console.Out.WriteLine();
|
||||
Console.Out.WriteLine("Dev / Test:");
|
||||
Console.Out.WriteLine(" --replay-log-dir <PATH> Replay IME logs from this directory (simulation mode)");
|
||||
|
||||
@@ -46,8 +46,12 @@ namespace AutopilotMonitor.Agent.V2.Runtime
|
||||
try { var _ = await remoteConfigService.FetchConfigAsync(); return true; }
|
||||
catch (Exception ex) { logger.Warning($"ServerAction rotate_config failed: {ex.Message}"); return false; }
|
||||
},
|
||||
// On-demand request mid-enrollment — the session has no outcome yet. Null keeps
|
||||
// sessioninfo.txt honest ("In Progress") and passes the OnFailure gate by
|
||||
// semantics (there is no success to skip on) instead of the former
|
||||
// enrollmentSucceeded:false workaround that stamped in-flight packages "Failed".
|
||||
uploadDiagnosticsAsync: async (suffix) =>
|
||||
await diagnosticsService.CreateAndUploadAsync(enrollmentSucceeded: false, fileNameSuffix: suffix),
|
||||
await diagnosticsService.CreateAndUploadAsync(enrollmentSucceeded: null, fileNameSuffix: suffix),
|
||||
onTerminateRequested: action => OnTerminateRequested(
|
||||
action, agentConfig, orchestrator, terminationHandler, shutdown, shutdownComplete, logger),
|
||||
post: post);
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Cover for the ConnectionType predicate used on every search path that does NOT push the
|
||||
/// filter to OData — the deviceProperties batch-get (<c>ApplyBasicFilters</c>) and the legacy
|
||||
/// unpaged scan. Sessions that predate the projection (null ConnectionType) must never match
|
||||
/// a set filter; an unset filter matches everything.
|
||||
/// </summary>
|
||||
public class ConnectionTypeFilterTests
|
||||
{
|
||||
private static SessionSummary Session(string? connectionType) =>
|
||||
new() { SessionId = "s", TenantId = "t", ConnectionType = connectionType };
|
||||
|
||||
[Theory]
|
||||
[InlineData(null)]
|
||||
[InlineData("WiFi")]
|
||||
[InlineData("Ethernet")]
|
||||
public void NoFilter_AlwaysMatches(string? connectionType)
|
||||
{
|
||||
var filter = new SessionSearchFilter();
|
||||
Assert.True(TableStorageService.MatchesConnectionType(Session(connectionType), filter));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("Ethernet", true)]
|
||||
[InlineData("ethernet", true)] // case-insensitive, matching the other string filters
|
||||
[InlineData("WiFi", false)]
|
||||
[InlineData(null, false)] // legacy session without the projected column
|
||||
public void EthernetFilter_MatchesOnlyEthernet(string? connectionType, bool expected)
|
||||
{
|
||||
var filter = new SessionSearchFilter { ConnectionType = "Ethernet" };
|
||||
Assert.Equal(expected, TableStorageService.MatchesConnectionType(Session(connectionType), filter));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("WiFi", true)]
|
||||
[InlineData("Ethernet", false)]
|
||||
[InlineData(null, false)]
|
||||
public void WiFiFilter_MatchesOnlyWiFi(string? connectionType, bool expected)
|
||||
{
|
||||
var filter = new SessionSearchFilter { ConnectionType = "WiFi" };
|
||||
Assert.Equal(expected, TableStorageService.MatchesConnectionType(Session(connectionType), filter));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Pins the per-session connection-type projection (2026-07-28): the agent stamps
|
||||
/// <c>connectionType</c> ("WiFi"/"Ethernet") into every <c>network_interface_info</c>
|
||||
/// emission (initial collect + re-collect on change); ingest takes the LAST matching
|
||||
/// event of the batch and persists it via <c>UpdateSessionConnectionTypeAsync</c>
|
||||
/// (last-write-wins → a device that switches media mid-enrollment is relabeled).
|
||||
/// The no-NIC payload ({"status":"no_active_interface"}) carries no connectionType
|
||||
/// and must never match.
|
||||
/// </summary>
|
||||
public class IngestConnectionTypeProjectionTests
|
||||
{
|
||||
private static EnrollmentEvent NicInfo(string? connectionType)
|
||||
{
|
||||
var evt = new EnrollmentEvent
|
||||
{
|
||||
EventType = "network_interface_info",
|
||||
Data = new Dictionary<string, object>
|
||||
{
|
||||
["adapterName"] = "Ethernet 2",
|
||||
["linkSpeedMbps"] = 1000L,
|
||||
},
|
||||
};
|
||||
if (connectionType != null) evt.Data["connectionType"] = connectionType;
|
||||
return evt;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void EthernetEmission_IsExtracted()
|
||||
{
|
||||
var events = new List<EnrollmentEvent> { NicInfo("Ethernet") };
|
||||
|
||||
Assert.True(EventIngestProcessor.TryExtractConnectionType(events, out var type));
|
||||
Assert.Equal("Ethernet", type);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void LastEmissionOfBatch_Wins()
|
||||
{
|
||||
// Media switch mid-batch (dock removed → WiFi) — the most recent state wins.
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
NicInfo("Ethernet"),
|
||||
new EnrollmentEvent { EventType = "download_progress" },
|
||||
NicInfo("WiFi"),
|
||||
};
|
||||
|
||||
Assert.True(EventIngestProcessor.TryExtractConnectionType(events, out var type));
|
||||
Assert.Equal("WiFi", type);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void NoActiveInterfacePayload_DoesNotMatch()
|
||||
{
|
||||
// The no-NIC emission carries only {"status":"no_active_interface"}.
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
new EnrollmentEvent
|
||||
{
|
||||
EventType = "network_interface_info",
|
||||
Data = new Dictionary<string, object> { ["status"] = "no_active_interface" },
|
||||
},
|
||||
};
|
||||
|
||||
Assert.False(EventIngestProcessor.TryExtractConnectionType(events, out _));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void NoActiveInterfaceAfterValidEmission_DoesNotShadowIt()
|
||||
{
|
||||
// The predicate selects the last event that CARRIES connectionType.
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
NicInfo("WiFi"),
|
||||
new EnrollmentEvent
|
||||
{
|
||||
EventType = "network_interface_info",
|
||||
Data = new Dictionary<string, object> { ["status"] = "no_active_interface" },
|
||||
},
|
||||
};
|
||||
|
||||
Assert.True(EventIngestProcessor.TryExtractConnectionType(events, out var type));
|
||||
Assert.Equal("WiFi", type);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("Cellular")]
|
||||
[InlineData("wifi")] // agent emits exact casing — anything else is unexpected
|
||||
[InlineData("")]
|
||||
public void UnexpectedValues_AreDroppedDefensively(string raw)
|
||||
{
|
||||
var events = new List<EnrollmentEvent> { NicInfo(raw) };
|
||||
|
||||
Assert.False(EventIngestProcessor.TryExtractConnectionType(events, out _));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BatchWithoutNicEvents_DoesNotMatch()
|
||||
{
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
new EnrollmentEvent { EventType = "esp_phase_changed" },
|
||||
new EnrollmentEvent { EventType = "agent_metrics_snapshot" },
|
||||
};
|
||||
|
||||
Assert.False(EventIngestProcessor.TryExtractConnectionType(events, out _));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void NullData_DoesNotMatch()
|
||||
{
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
new EnrollmentEvent { EventType = "network_interface_info", Data = null! },
|
||||
};
|
||||
|
||||
Assert.False(EventIngestProcessor.TryExtractConnectionType(events, out _));
|
||||
}
|
||||
}
|
||||
@@ -45,6 +45,12 @@ public class SessionIndexProjectionTests
|
||||
["IsSelfDeployingProfile"] = true,
|
||||
// Device-validation path recorded at registration — shown on the session detail page.
|
||||
["ValidatedBy"] = "AutopilotV1",
|
||||
// Agent→backend HTTP latency projection (UpdateSessionNetworkLatencyAsync merges these).
|
||||
["AvgApiLatencyMs"] = 513.5,
|
||||
["ApiRequestCount"] = 100,
|
||||
// Connection-type projection (UpdateSessionConnectionTypeAsync merges this) —
|
||||
// search-filterable (connectionType= pushes an OData eq on the index).
|
||||
["ConnectionType"] = "Ethernet",
|
||||
};
|
||||
|
||||
var idx = TableStorageService.BuildSessionIndexEntity(session, IndexRowKey, StartedAt);
|
||||
@@ -67,6 +73,9 @@ public class SessionIndexProjectionTests
|
||||
Assert.True(idx.GetBoolean("ExcessiveEventsAutoActioned"));
|
||||
Assert.True(idx.GetBoolean("IsSelfDeployingProfile"));
|
||||
Assert.Equal("AutopilotV1", idx.GetString("ValidatedBy"));
|
||||
Assert.Equal(513.5, idx.GetDouble("AvgApiLatencyMs"));
|
||||
Assert.Equal(100, idx.GetInt32("ApiRequestCount"));
|
||||
Assert.Equal("Ethernet", idx.GetString("ConnectionType"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -86,6 +95,8 @@ public class SessionIndexProjectionTests
|
||||
Assert.False(idx.ContainsKey("FailureSource"));
|
||||
Assert.False(idx.ContainsKey("StalledAt"));
|
||||
Assert.False(idx.ContainsKey("ValidatedBy"));
|
||||
Assert.False(idx.ContainsKey("AvgApiLatencyMs"));
|
||||
Assert.False(idx.ContainsKey("ConnectionType"));
|
||||
// Always-present counts/flags mirror the Sessions defaults.
|
||||
Assert.Equal(0, idx.GetInt32("PlatformScriptCount"));
|
||||
Assert.Equal(0, idx.GetInt32("RemediationScriptCount"));
|
||||
|
||||
+4
@@ -40,6 +40,7 @@ public class StoreSessionReregistrationPreserveTests
|
||||
["PlatformScriptCount"] = 3,
|
||||
["RemediationScriptCount"] = 5,
|
||||
["RebootCount"] = 7,
|
||||
["ConnectionType"] = "Ethernet",
|
||||
};
|
||||
existing.ETag = new ETag("0xEXISTING");
|
||||
|
||||
@@ -71,6 +72,9 @@ public class StoreSessionReregistrationPreserveTests
|
||||
Assert.Equal(7, harness.Written.GetInt32("RebootCount"));
|
||||
// EventCount preservation is the established sibling behaviour — assert it as a sanity anchor.
|
||||
Assert.Equal(42, harness.Written.GetInt32("EventCount"));
|
||||
// Merge-mode field (UpdateSessionConnectionTypeAsync) — must survive the Replace so the
|
||||
// value isn't lost in the window until the restarted agent re-emits network_interface_info.
|
||||
Assert.Equal("Ethernet", harness.Written.GetString("ConnectionType"));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
+2
@@ -413,6 +413,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
{ "EnableTimezoneAutoSet", config.EnableTimezoneAutoSet },
|
||||
{ "NtpServer", config.NtpServer },
|
||||
{ "EnableImeMatchLog", config.EnableImeMatchLog },
|
||||
{ "EnableGatherRuleDebugLog", config.EnableGatherRuleDebugLog },
|
||||
{ "LogLevel", config.LogLevel },
|
||||
{ "MaxBatchSize", config.MaxBatchSize },
|
||||
{ "DiagnosticsBlobSasUrl", config.DiagnosticsBlobSasUrl },
|
||||
@@ -514,6 +515,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
EnableTimezoneAutoSet = entity.GetBoolean("EnableTimezoneAutoSet"),
|
||||
NtpServer = string.IsNullOrWhiteSpace(entity.GetString("NtpServer")) ? "time.windows.com" : entity.GetString("NtpServer"),
|
||||
EnableImeMatchLog = entity.GetBoolean("EnableImeMatchLog"),
|
||||
EnableGatherRuleDebugLog = entity.GetBoolean("EnableGatherRuleDebugLog"),
|
||||
LogLevel = entity.GetString("LogLevel"),
|
||||
MaxBatchSize = entity.GetInt32("MaxBatchSize"),
|
||||
DiagnosticsBlobSasUrl = entity.GetString("DiagnosticsBlobSasUrl"),
|
||||
|
||||
+4
@@ -152,6 +152,10 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
double avgLatencyMs, int requestCount)
|
||||
=> _storage.UpdateSessionNetworkLatencyAsync(tenantId, sessionId, avgLatencyMs, requestCount);
|
||||
|
||||
public Task UpdateSessionConnectionTypeAsync(string tenantId, string sessionId,
|
||||
string connectionType)
|
||||
=> _storage.UpdateSessionConnectionTypeAsync(tenantId, sessionId, connectionType);
|
||||
|
||||
public Task<List<SessionSummary>> GetSessionsWithEventCountAboveAsync(string tenantId, int threshold)
|
||||
=> _storage.GetSessionsWithEventCountAboveAsync(tenantId, threshold);
|
||||
|
||||
|
||||
@@ -270,12 +270,13 @@ namespace AutopilotMonitor.Functions.Functions.Config
|
||||
var response = req.CreateResponse(HttpStatusCode.OK);
|
||||
await response.WriteAsJsonAsync(new AgentConfigResponse
|
||||
{
|
||||
ConfigVersion = 35, // MDM reboot-policy watcher (mdm_policy_reboot_required / esp_reboot_coalescing)
|
||||
ConfigVersion = 36, // EnableGatherRuleDebugLog (local gather-rule evaluation trace)
|
||||
UploadIntervalSeconds = 10,
|
||||
SelfDestructOnComplete = tenantConfig.SelfDestructOnComplete ?? true,
|
||||
KeepLogFile = tenantConfig.KeepLogFile ?? false,
|
||||
EnableGeoLocation = tenantConfig.EnableGeoLocation ?? true,
|
||||
EnableImeMatchLog = tenantConfig.EnableImeMatchLog ?? false,
|
||||
EnableGatherRuleDebugLog = tenantConfig.EnableGatherRuleDebugLog ?? false,
|
||||
MaxAuthFailures = tenantConfig.MaxAuthFailures ?? 5,
|
||||
AuthFailureTimeoutMinutes = tenantConfig.AuthFailureTimeoutMinutes ?? 0,
|
||||
LogLevel = tenantConfig.LogLevel ?? "Info",
|
||||
|
||||
@@ -130,6 +130,8 @@ namespace AutopilotMonitor.Functions.Functions.Raw
|
||||
filter.IsSelfDeployingProfile = sdp;
|
||||
if (int.TryParse(query["rebootCountMin"], out var rcMin)) filter.RebootCountMin = rcMin;
|
||||
if (int.TryParse(query["rebootCountMax"], out var rcMax)) filter.RebootCountMax = rcMax;
|
||||
var connectionType = query["connectionType"];
|
||||
if (!string.IsNullOrEmpty(connectionType)) filter.ConnectionType = connectionType;
|
||||
|
||||
var callerTenantId = TenantHelper.GetTenantId(req);
|
||||
|
||||
|
||||
@@ -140,6 +140,7 @@ public class SearchSessionsFunction
|
||||
ImeAgentVersion = query["imeAgentVersion"],
|
||||
AgentVersionPrefix = query["agentVersionPrefix"],
|
||||
ImeAgentVersionPrefix = query["imeAgentVersionPrefix"],
|
||||
ConnectionType = query["connectionType"],
|
||||
// Limit field is a no-op in the paged path — pagination drives count.
|
||||
};
|
||||
|
||||
@@ -212,6 +213,7 @@ public class SearchSessionsFunction
|
||||
if (fields.Contains("rebootCount")) dict["rebootCount"] = s.RebootCount;
|
||||
if (fields.Contains("avgApiLatencyMs")) dict["avgApiLatencyMs"] = s.AvgApiLatencyMs;
|
||||
if (fields.Contains("apiRequestCount")) dict["apiRequestCount"] = s.ApiRequestCount;
|
||||
if (fields.Contains("connectionType")) dict["connectionType"] = s.ConnectionType;
|
||||
|
||||
if (dict.Count == 0)
|
||||
{
|
||||
|
||||
@@ -53,6 +53,7 @@ namespace AutopilotMonitor.Functions.Pagination
|
||||
new KeyValuePair<string, string?>("startedBefore", filter.StartedBefore?.ToUniversalTime().ToString("o", CultureInfo.InvariantCulture)),
|
||||
new KeyValuePair<string, string?>("rebootCountMin", filter.RebootCountMin?.ToString(CultureInfo.InvariantCulture)),
|
||||
new KeyValuePair<string, string?>("rebootCountMax", filter.RebootCountMax?.ToString(CultureInfo.InvariantCulture)),
|
||||
new KeyValuePair<string, string?>("connectionType", filter.ConnectionType),
|
||||
};
|
||||
if (filter.DeviceProperties != null)
|
||||
{
|
||||
|
||||
@@ -151,6 +151,14 @@ namespace AutopilotMonitor.Functions.Services
|
||||
"Fire-and-forget UpdateSessionNetworkLatencyAsync failed"), TaskContinuationOptions.OnlyOnFaulted);
|
||||
}
|
||||
|
||||
if (TryExtractConnectionType(request.Events, out var connectionType))
|
||||
{
|
||||
_ = _sessionRepo.UpdateSessionConnectionTypeAsync(
|
||||
request.TenantId, request.SessionId, connectionType)
|
||||
.ContinueWith(t => _logger.LogWarning(t.Exception?.InnerException,
|
||||
"Fire-and-forget UpdateSessionConnectionTypeAsync failed"), TaskContinuationOptions.OnlyOnFaulted);
|
||||
}
|
||||
|
||||
var classification = ClassifyEvents(storedEvents);
|
||||
|
||||
foreach (var summary in classification.AppInstallUpdates.Values)
|
||||
@@ -432,6 +440,28 @@ namespace AutopilotMonitor.Functions.Services
|
||||
return true;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Connection-type projection: the agent stamps connectionType ("WiFi"/"Ethernet")
|
||||
/// into every network_interface_info emission (initial collect + re-collect), so the
|
||||
/// LAST event of the batch carries the current media and a plain overwrite is
|
||||
/// idempotent against replays. The no-NIC payload ({"status":"no_active_interface"})
|
||||
/// carries no connectionType and never matches. Values outside WiFi/Ethernet are
|
||||
/// dropped defensively. Exposed as internal for unit testing.
|
||||
/// </summary>
|
||||
internal static bool TryExtractConnectionType(
|
||||
List<EnrollmentEvent> events, out string connectionType)
|
||||
{
|
||||
connectionType = string.Empty;
|
||||
var nicEvent = events.LastOrDefault(e =>
|
||||
e.EventType == Shared.Constants.EventTypes.NetworkInterfaceInfo &&
|
||||
e.Data?.ContainsKey("connectionType") == true);
|
||||
var raw = nicEvent?.Data?["connectionType"]?.ToString();
|
||||
if (raw != "WiFi" && raw != "Ethernet")
|
||||
return false;
|
||||
connectionType = raw;
|
||||
return true;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Numeric reader for agent event Data values, which arrive as boxed Newtonsoft
|
||||
/// primitives (integer → long, decimal → double) — same coercion ladder as
|
||||
|
||||
@@ -654,6 +654,11 @@ namespace AutopilotMonitor.Functions.Services
|
||||
parts.Add($"RebootCount ge {filter.RebootCountMin.Value}");
|
||||
if (filter.RebootCountMax.HasValue)
|
||||
parts.Add($"RebootCount le {filter.RebootCountMax.Value}");
|
||||
// ConnectionType exact match pushed to OData. Legacy index rows that predate
|
||||
// the projected column lack the property and are excluded by the eq (they
|
||||
// carry no connection data, which is the intended result).
|
||||
if (!string.IsNullOrEmpty(filter.ConnectionType))
|
||||
parts.Add($"ConnectionType eq '{ODataSanitizer.EscapeValue(filter.ConnectionType)}'");
|
||||
return parts.Count == 0 ? null : string.Join(" and ", parts);
|
||||
}
|
||||
|
||||
@@ -670,6 +675,19 @@ namespace AutopilotMonitor.Functions.Services
|
||||
return true;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Client-side ConnectionType predicate — same backstop role as
|
||||
/// <see cref="MatchesRebootCountBounds"/>: applied on the search paths that do NOT
|
||||
/// push the filter to OData (device-snapshot batch-get, legacy unpaged scan) and
|
||||
/// redundantly on the OData scan paths. Sessions without the projected column
|
||||
/// (null) never match a set filter.
|
||||
/// </summary>
|
||||
internal static bool MatchesConnectionType(SessionSummary session, SessionSearchFilter filter)
|
||||
{
|
||||
if (string.IsNullOrEmpty(filter.ConnectionType)) return true;
|
||||
return string.Equals(session.ConnectionType, filter.ConnectionType, StringComparison.OrdinalIgnoreCase);
|
||||
}
|
||||
|
||||
private static bool MatchesScanClientFilters(SessionSummary session, SessionSearchFilter filter)
|
||||
{
|
||||
if (!string.IsNullOrEmpty(filter.SerialNumber) &&
|
||||
@@ -683,9 +701,10 @@ namespace AutopilotMonitor.Functions.Services
|
||||
return false;
|
||||
if (filter.StartedAfter.HasValue && session.StartedAt < filter.StartedAfter.Value) return false;
|
||||
if (filter.StartedBefore.HasValue && session.StartedAt > filter.StartedBefore.Value) return false;
|
||||
// RebootCount is also pushed to OData in BuildSearchScanFilter; this is a defensive
|
||||
// backstop so the bound holds even if that push-down is ever weakened.
|
||||
// RebootCount / ConnectionType are also pushed to OData in BuildSearchScanFilter;
|
||||
// these are defensive backstops so the bound holds even if that push-down is ever weakened.
|
||||
if (!MatchesRebootCountBounds(session, filter)) return false;
|
||||
if (!MatchesConnectionType(session, filter)) return false;
|
||||
// AgentVersion / ImeAgentVersion (exact + prefix) are pushed to OData
|
||||
// in BuildSearchScanFilter — the server has already filtered them out
|
||||
// before we see the page. No client-side check needed.
|
||||
@@ -951,9 +970,11 @@ namespace AutopilotMonitor.Functions.Services
|
||||
&& (s.ImeAgentVersion == null
|
||||
|| !s.ImeAgentVersion.StartsWith(filter.ImeAgentVersionPrefix!, StringComparison.OrdinalIgnoreCase)))
|
||||
return false;
|
||||
// Device-snapshot path has no RebootCount OData push-down — enforce it here so a
|
||||
// deviceProperties + rebootCountMin query can't return sub-threshold sessions.
|
||||
// Device-snapshot path has no RebootCount/ConnectionType OData push-down —
|
||||
// enforce them here so a deviceProperties + rebootCountMin/connectionType
|
||||
// query can't return non-matching sessions.
|
||||
if (!MatchesRebootCountBounds(s, filter)) return false;
|
||||
if (!MatchesConnectionType(s, filter)) return false;
|
||||
return true;
|
||||
}).ToList();
|
||||
}
|
||||
|
||||
@@ -225,6 +225,12 @@ namespace AutopilotMonitor.Functions.Services
|
||||
if (apiRequestCount.HasValue)
|
||||
indexEntity["ApiRequestCount"] = apiRequestCount.Value;
|
||||
|
||||
// Connection-type projection (UpdateSessionConnectionTypeAsync merges this) —
|
||||
// must be in this superset or a StartedAt-shift full upsert drops it.
|
||||
var connectionType = sessionEntity.GetString("ConnectionType");
|
||||
if (!string.IsNullOrEmpty(connectionType))
|
||||
indexEntity["ConnectionType"] = connectionType;
|
||||
|
||||
return indexEntity;
|
||||
}
|
||||
|
||||
@@ -398,6 +404,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
string geoRegion = string.Empty;
|
||||
string geoCity = string.Empty;
|
||||
string geoLoc = string.Empty;
|
||||
string? connectionType = null;
|
||||
string? existingIndexRowKey = null;
|
||||
// PR3: cascade-delete state-machine columns. Re-registration MUST preserve
|
||||
// these — losing them would silently clear the lock and let the agent's writes
|
||||
@@ -444,6 +451,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
geoRegion = existingEntity.GetString("GeoRegion") ?? string.Empty;
|
||||
geoCity = existingEntity.GetString("GeoCity") ?? string.Empty;
|
||||
geoLoc = existingEntity.GetString("GeoLoc") ?? string.Empty;
|
||||
connectionType = existingEntity.GetString("ConnectionType");
|
||||
existingIndexRowKey = existingEntity.GetString("IndexRowKey");
|
||||
existingDeletionState = existingEntity.GetString("DeletionState");
|
||||
existingPendingDeletionManifestId = existingEntity.GetString("PendingDeletionManifestId");
|
||||
@@ -553,6 +561,12 @@ namespace AutopilotMonitor.Functions.Services
|
||||
if (!string.IsNullOrEmpty(geoLoc))
|
||||
entity["GeoLoc"] = geoLoc;
|
||||
|
||||
// Merge-mode field (UpdateSessionConnectionTypeAsync) — preserve through the
|
||||
// Replace; the agent re-emits network_interface_info after restart, but the
|
||||
// value must not be lost in the window until it does.
|
||||
if (!string.IsNullOrEmpty(connectionType))
|
||||
entity["ConnectionType"] = connectionType;
|
||||
|
||||
// Preserve IndexRowKey through the Replace so that:
|
||||
// 1. Concurrent UpdateSessionStatusAsync calls can find it (prevents index stale-status)
|
||||
// 2. UpsertSessionIndexAsync can detect and delete old index entries when startedAt shifts
|
||||
@@ -2714,6 +2728,52 @@ namespace AutopilotMonitor.Functions.Services
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Stores the active network connection type ("WiFi"/"Ethernet") from the latest
|
||||
/// network_interface_info event (Merge-mode, single field, last-write-wins).
|
||||
/// Same UpdateEntity-not-Upsert rationale as <see cref="UpdateSessionNetworkLatencyAsync"/>:
|
||||
/// a tombstoned Sessions row must stay tombstoned (404 → no-op).
|
||||
/// Non-fatal: failures are logged as warnings and do not block ingest.
|
||||
/// </summary>
|
||||
public async Task UpdateSessionConnectionTypeAsync(string tenantId, string sessionId,
|
||||
string connectionType)
|
||||
{
|
||||
try
|
||||
{
|
||||
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.Sessions);
|
||||
var update = new TableEntity(tenantId, sessionId)
|
||||
{
|
||||
["ConnectionType"] = connectionType
|
||||
};
|
||||
await tableClient.UpdateEntityAsync(update, ETag.All, TableUpdateMode.Merge);
|
||||
|
||||
// Mirror to SessionsIndex — serves the session list and the cheap
|
||||
// connectionType= search filter (OData eq push-down on the index).
|
||||
var idxRef = await tableClient.GetEntityAsync<TableEntity>(
|
||||
tenantId, sessionId, select: new[] { "IndexRowKey" });
|
||||
var indexRowKey = idxRef.Value.GetString("IndexRowKey");
|
||||
if (!string.IsNullOrEmpty(indexRowKey))
|
||||
{
|
||||
await MergeSessionIndexAsync(tenantId, indexRowKey,
|
||||
new TableEntity(tenantId, indexRowKey)
|
||||
{
|
||||
["ConnectionType"] = connectionType
|
||||
});
|
||||
}
|
||||
}
|
||||
catch (RequestFailedException ex) when (ex.Status == 404)
|
||||
{
|
||||
_logger.LogDebug(
|
||||
"UpdateSessionConnectionType no-op: Sessions row {Tenant}/{Session} is absent (tombstoned or never registered)",
|
||||
tenantId, sessionId);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
// Non-fatal — don't block ingest
|
||||
_logger.LogWarning(ex, "Failed to update connection type for session {SessionId}", sessionId);
|
||||
}
|
||||
}
|
||||
|
||||
// ===== SESSION/EVENT MAPPING HELPERS =====
|
||||
|
||||
/// <summary>
|
||||
@@ -2809,6 +2869,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
DurationSeconds = ComputeEffectiveDuration(entity, status, startedAt, completedAt),
|
||||
AvgApiLatencyMs = SafeGetDouble(entity, "AvgApiLatencyMs"),
|
||||
ApiRequestCount = SafeGetInt32(entity, "ApiRequestCount"),
|
||||
ConnectionType = entity.GetString("ConnectionType"),
|
||||
EnrollmentType = entity.GetString("EnrollmentType") ?? "v1",
|
||||
DiagnosticsBlobName = entity.GetString("DiagnosticsBlobName"),
|
||||
// Legacy rows that predate this field surface as null → download path treats
|
||||
|
||||
@@ -894,6 +894,8 @@ export function registerAdminTools(server: McpServer, ga: boolean, strictGa: boo
|
||||
isPreProvisioned: z.boolean().optional().describe('Filter pre-provisioned (white-glove) sessions'),
|
||||
isHybridJoin: z.boolean().optional().describe('Filter hybrid AAD-join sessions'),
|
||||
isSelfDeployingProfile: z.boolean().optional().describe('Filter self-deploying/kiosk profile sessions (CloudAssignedOobeConfig 0x20|0x40)'),
|
||||
connectionType: z.enum(['WiFi', 'Ethernet']).optional()
|
||||
.describe('Active network connection type ("WiFi" or "Ethernet", exact match on the indexed column; last emission wins). Sessions predating the projection lack the column and are excluded.'),
|
||||
fields: z.string().optional().describe('Comma-separated pass-through projection over the literal stored column names (case-insensitive, PascalCase, e.g. "Status,OsEdition,ImeAgentVersion,GeoCity"); narrows the row but never drops a real column. PartitionKey + RowKey are always kept. Omit for the full raw row.'),
|
||||
pageSize: z.coerce.number().int().min(1).max(1000).optional().default(200)
|
||||
.describe('Page size (1-1000, default 200). Returns this many sessions per call; follow nextLink to fetch more.'),
|
||||
@@ -906,14 +908,14 @@ export function registerAdminTools(server: McpServer, ga: boolean, strictGa: boo
|
||||
try {
|
||||
const { tenantId: rawTenantId, status, startedAfter, startedBefore, serialNumber, agentVersion, agentVersionPrefix,
|
||||
imeAgentVersion, imeAgentVersionPrefix, manufacturer, model, enrollmentType, deviceName, osBuild,
|
||||
geoCountry, isPreProvisioned, isHybridJoin, isSelfDeployingProfile, fields, pageSize, continuation } = args;
|
||||
geoCountry, isPreProvisioned, isHybridJoin, isSelfDeployingProfile, connectionType, fields, pageSize, continuation } = args;
|
||||
const tenantId = enforceDelegatedTenantForPage(rawTenantId, continuation);
|
||||
const basePath = pickGlobalOrTenantPath('/api/global/raw/sessions', '/api/raw/sessions', tenantId);
|
||||
const path = followNextLink(
|
||||
basePath,
|
||||
{ tenantId, status, startedAfter, startedBefore, serialNumber, agentVersion, agentVersionPrefix,
|
||||
imeAgentVersion, imeAgentVersionPrefix, manufacturer, model, enrollmentType, deviceName, osBuild,
|
||||
geoCountry, isPreProvisioned, isHybridJoin, isSelfDeployingProfile, fields, pageSize },
|
||||
geoCountry, isPreProvisioned, isHybridJoin, isSelfDeployingProfile, connectionType, fields, pageSize },
|
||||
continuation,
|
||||
);
|
||||
const data = await apiFetch(path);
|
||||
|
||||
@@ -139,6 +139,10 @@ export function registerSessionTools(server: McpServer, ga: boolean, delegated:
|
||||
'e.g. rebootCountMin=5. Only populated for v2 enrollments; sessions that predate the field are excluded.'),
|
||||
rebootCountMax: z.coerce.number().int().min(0).optional()
|
||||
.describe('Maximum number of reboots observed during enrollment (<=).'),
|
||||
connectionType: z.enum(['WiFi', 'Ethernet']).optional()
|
||||
.describe('Active network connection type during enrollment ("WiFi" or "Ethernet"), indexed for cheap exact-match ' +
|
||||
'filtering (e.g. "how many machines enrolled over WiFi?"). Last emission wins — a device that switches media ' +
|
||||
'mid-enrollment reports the most recent state. Sessions that predate the projection lack the column and are excluded.'),
|
||||
fields: z.string().optional()
|
||||
.describe('Comma-separated lean projection (e.g. "sessionId,status,agentVersion,startedAt"). ' +
|
||||
'Use for counting / aggregation to avoid the response cap. Available: sessionId, tenantId, status, ' +
|
||||
@@ -146,7 +150,8 @@ export function registerSessionTools(server: McpServer, ga: boolean, delegated:
|
||||
'durationSeconds, currentPhase, failureReason, eventCount, enrollmentType, isPreProvisioned, ' +
|
||||
'isUserDriven, isHybridJoin, isSelfDeployingProfile, agentVersion, imeAgentVersion, geoCountry, rebootCount, ' +
|
||||
'avgApiLatencyMs, apiRequestCount (agent→backend HTTP round-trip; weight latency by apiRequestCount when aggregating, ' +
|
||||
'e.g. fields=geoCountry,avgApiLatencyMs,apiRequestCount for a per-country latency sweep; null on sessions from agents predating the field).'),
|
||||
'e.g. fields=geoCountry,avgApiLatencyMs,apiRequestCount for a per-country latency sweep; null on sessions from agents predating the field), ' +
|
||||
'connectionType ("WiFi"/"Ethernet"; null on sessions predating the projection).'),
|
||||
deviceProperties: z.record(z.string(), z.string()).optional().describe(
|
||||
'Dynamic device property filters. Keys use "eventType.propertyName" dot notation. ' +
|
||||
'See the device_properties catalog (call get_resource(name="device_properties")) for all available keys and types. ' +
|
||||
|
||||
@@ -62,6 +62,13 @@ namespace AutopilotMonitor.Shared
|
||||
/// </summary>
|
||||
public const string ImeMatchLogPath = @"%ProgramData%\AutopilotMonitor\Logs\ime_pattern_matches.log";
|
||||
|
||||
/// <summary>
|
||||
/// Default path for the gather rule debug trace file (debugging/diagnostics).
|
||||
/// Written when the tenant enables EnableGatherRuleDebugLog; explains every
|
||||
/// gather-rule evaluation decision, including outcomes that produce no event.
|
||||
/// </summary>
|
||||
public const string GatherRuleDebugLogPath = @"%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log";
|
||||
|
||||
/// <summary>
|
||||
/// Scheduled Task name used to run the agent as SYSTEM
|
||||
/// </summary>
|
||||
|
||||
@@ -115,6 +115,13 @@ namespace AutopilotMonitor.Shared.DataAccess
|
||||
/// </summary>
|
||||
Task UpdateSessionNetworkLatencyAsync(string tenantId, string sessionId,
|
||||
double avgLatencyMs, int requestCount);
|
||||
/// <summary>
|
||||
/// Persists the active network connection type ("WiFi" or "Ethernet") from the
|
||||
/// latest <c>network_interface_info</c> event. Last-write-wins; fail-soft like
|
||||
/// the other supplementary-field updaters.
|
||||
/// </summary>
|
||||
Task UpdateSessionConnectionTypeAsync(string tenantId, string sessionId,
|
||||
string connectionType);
|
||||
|
||||
// --- Server→Agent Actions ---
|
||||
/// <summary>
|
||||
|
||||
@@ -40,6 +40,12 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// </summary>
|
||||
public bool EnableImeMatchLog { get; set; } = false;
|
||||
|
||||
/// <summary>
|
||||
/// Whether to write a local gather-rule evaluation trace file.
|
||||
/// When true, the default path is used: Constants.GatherRuleDebugLogPath.
|
||||
/// </summary>
|
||||
public bool EnableGatherRuleDebugLog { get; set; } = false;
|
||||
|
||||
public CollectorConfiguration Collectors { get; set; } = default!;
|
||||
|
||||
/// <summary>
|
||||
|
||||
@@ -334,6 +334,15 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// </summary>
|
||||
public bool? EnableImeMatchLog { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Whether the agent writes a local gather-rule evaluation trace file so customers
|
||||
/// can diagnose rules that produce no timeline events (scope skips, on_change
|
||||
/// suppression, empty collector results, logparser details). When true, the agent
|
||||
/// writes to Constants.GatherRuleDebugLogPath. The trace never leaves the device.
|
||||
/// null = use agent default (false).
|
||||
/// </summary>
|
||||
public bool? EnableGatherRuleDebugLog { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Log verbosity level override for this tenant's agents.
|
||||
/// null = use agent default ("Info"). Values: "Info", "Debug", "Verbose", "Trace".
|
||||
|
||||
@@ -275,6 +275,14 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// </summary>
|
||||
public int? ApiRequestCount { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Active network connection type during enrollment: "WiFi" or "Ethernet".
|
||||
/// Projected at ingest from the latest <c>network_interface_info</c> event
|
||||
/// (last emission wins — a device that switches media mid-enrollment reports
|
||||
/// the most recent state). Null for sessions that predate the projection.
|
||||
/// </summary>
|
||||
public string? ConnectionType { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Enrollment type: "v1" (Autopilot Classic/ESP) or "v2" (Windows Device Preparation).
|
||||
/// Defaults to "v1" for sessions that predate this field.
|
||||
|
||||
@@ -37,6 +37,11 @@ namespace AutopilotMonitor.Shared.Models
|
||||
public int? RebootCountMin { get; set; }
|
||||
public int? RebootCountMax { get; set; }
|
||||
|
||||
// Active network connection type ("WiFi" or "Ethernet"). Exact match on the
|
||||
// stored ConnectionType column. Sessions that predate the projection lack the
|
||||
// property and are excluded (acceptable — they have no connection data).
|
||||
public string? ConnectionType { get; set; }
|
||||
|
||||
public int Limit { get; set; } = 50;
|
||||
|
||||
// Dynamic device property filters (key = "eventType.propertyName", value = filter expression)
|
||||
|
||||
@@ -58,6 +58,7 @@ interface TenantConfig {
|
||||
enableTimezoneAutoSet?: boolean | null;
|
||||
ntpServer?: string | null;
|
||||
enableImeMatchLog?: boolean | null;
|
||||
enableGatherRuleDebugLog?: boolean | null;
|
||||
logLevel?: string | null;
|
||||
maxBatchSize?: number | null;
|
||||
showEnrollmentSummary?: boolean | null;
|
||||
@@ -130,6 +131,7 @@ const DEFAULTS: Record<string, unknown> = {
|
||||
enableTimezoneAutoSet: null,
|
||||
ntpServer: 'time.windows.com',
|
||||
enableImeMatchLog: null,
|
||||
enableGatherRuleDebugLog: null,
|
||||
logLevel: null,
|
||||
maxBatchSize: null,
|
||||
showEnrollmentSummary: null,
|
||||
@@ -168,6 +170,7 @@ const RUNTIME_DEFAULTS: Record<string, unknown> = {
|
||||
enableTimezoneAutoSet: false,
|
||||
ntpServer: 'time.windows.com',
|
||||
enableImeMatchLog: false,
|
||||
enableGatherRuleDebugLog: false,
|
||||
maxAuthFailures: 5,
|
||||
authFailureTimeoutMinutes: 0,
|
||||
logLevel: 'Info',
|
||||
@@ -296,6 +299,7 @@ function computeRuntime(c: TenantConfig): Record<string, unknown> {
|
||||
enableTimezoneAutoSet: c.enableTimezoneAutoSet ?? false,
|
||||
ntpServer: c.ntpServer ?? 'time.windows.com',
|
||||
enableImeMatchLog: c.enableImeMatchLog ?? false,
|
||||
enableGatherRuleDebugLog: c.enableGatherRuleDebugLog ?? false,
|
||||
maxAuthFailures: c.maxAuthFailures ?? 5,
|
||||
authFailureTimeoutMinutes: c.authFailureTimeoutMinutes ?? 0,
|
||||
logLevel: c.logLevel ?? 'Info',
|
||||
@@ -575,6 +579,7 @@ export function SectionTenantConfigReport() {
|
||||
<ConfigRow label="Timezone Auto-Set" value={config.enableTimezoneAutoSet} configKey="enableTimezoneAutoSet" defaults={DEFAULTS} />
|
||||
<ConfigRow label="NTP Server" value={config.ntpServer} configKey="ntpServer" defaults={DEFAULTS} />
|
||||
<ConfigRow label="IME Match Log" value={config.enableImeMatchLog} configKey="enableImeMatchLog" defaults={DEFAULTS} />
|
||||
<ConfigRow label="Gather Rule Debug Log" value={config.enableGatherRuleDebugLog} configKey="enableGatherRuleDebugLog" defaults={DEFAULTS} />
|
||||
<ConfigRow label="Log Level" value={config.logLevel} configKey="logLevel" defaults={DEFAULTS} />
|
||||
<ConfigRow label="Max Batch Size" value={config.maxBatchSize} configKey="maxBatchSize" defaults={DEFAULTS} />
|
||||
<ConfigRow label="Show Script Output" value={config.showScriptOutput} configKey="showScriptOutput" defaults={DEFAULTS} />
|
||||
@@ -647,6 +652,7 @@ export function SectionTenantConfigReport() {
|
||||
<ConfigRow label="Timezone Auto-Set" value={runtime.enableTimezoneAutoSet} configKey="enableTimezoneAutoSet" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="NTP Server" value={runtime.ntpServer} configKey="ntpServer" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="IME Match Log" value={runtime.enableImeMatchLog} configKey="enableImeMatchLog" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="Gather Rule Debug Log" value={runtime.enableGatherRuleDebugLog} configKey="enableGatherRuleDebugLog" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="Log Level" value={runtime.logLevel} configKey="logLevel" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="Send Trace Events" value={runtime.sendTraceEvents} configKey="sendTraceEvents" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="Unrestricted Mode" value={runtime.unrestrictedMode} configKey="unrestrictedMode" defaults={RUNTIME_DEFAULTS} />
|
||||
|
||||
@@ -31,6 +31,7 @@ const ALL_COLUMNS: ColumnDef[] = [
|
||||
{ key: "started", label: "Started", defaultVisible: true, sortKey: "startedAt" },
|
||||
{ key: "country", label: "Country", defaultVisible: false, sortKey: "geoCountry", filterKey: "geoCountry" },
|
||||
{ key: "apiLatency", label: "API Latency", defaultVisible: false, sortKey: "avgApiLatencyMs" },
|
||||
{ key: "connection", label: "Connection", defaultVisible: false, sortKey: "connectionType", filterKey: "connectionType" },
|
||||
{ key: "agentVersion", label: "Agent Version", defaultVisible: false, sortKey: "agentVersion", filterKey: "agentVersion" },
|
||||
{ key: "osName", label: "OS Name", defaultVisible: false, sortKey: "osName", filterKey: "osName" },
|
||||
{ key: "osBuild", label: "OS Build", defaultVisible: false, sortKey: "osBuild", filterKey: "osBuild" },
|
||||
@@ -641,7 +642,7 @@ export function SessionTable({
|
||||
currentSort={sortColumn}
|
||||
direction={sortDirection}
|
||||
onSort={onSort}
|
||||
className={["eventCount", "duration", "started", "country", "apiLatency", "agentVersion", "osName", "osBuild", "osDisplayVersion", "osEdition", "osLanguage"].includes(col.key) ? "px-3" : undefined}
|
||||
className={["eventCount", "duration", "started", "country", "apiLatency", "connection", "agentVersion", "osName", "osBuild", "osDisplayVersion", "osEdition", "osLanguage"].includes(col.key) ? "px-3" : undefined}
|
||||
filterKey={col.filterKey}
|
||||
filterValues={col.filterKey ? uniqueValuesByField[col.filterKey] : undefined}
|
||||
activeFilter={col.filterKey ? columnFilters[col.filterKey] : undefined}
|
||||
@@ -922,6 +923,19 @@ function SessionCell({
|
||||
</td>
|
||||
);
|
||||
|
||||
case "connection":
|
||||
return (
|
||||
<td className="px-3 py-4 whitespace-nowrap text-sm text-gray-500">
|
||||
{session.connectionType ? (
|
||||
<span title="Active network connection during enrollment (last reported state)">
|
||||
{session.connectionType}
|
||||
</span>
|
||||
) : (
|
||||
<span className="text-gray-300">—</span>
|
||||
)}
|
||||
</td>
|
||||
);
|
||||
|
||||
case "agentVersion": {
|
||||
const fullVersion = session.agentVersion || "";
|
||||
const shortVersion = fullVersion.split("+")[0];
|
||||
|
||||
+46
-2
@@ -355,6 +355,24 @@ function PhaseSection({
|
||||
);
|
||||
}
|
||||
|
||||
// Backfilled events (ModernDeploymentTracker error backfill): pre-agent event-log records
|
||||
// replayed at agent start carry backfilled=true plus the original event-log timeCreated —
|
||||
// the event's own timestamp is agent emission time, so without this the whole block renders
|
||||
// with one identical (wrong) time and full Error severity, reading like a live failure.
|
||||
function getBackfillInfo(event: EnrollmentEvent): { isBackfilled: boolean; recordedAt: Date | null } {
|
||||
const raw = event.data?.backfilled ?? event.data?.Backfilled;
|
||||
const isBackfilled = raw === true || raw === "true" || raw === "True";
|
||||
if (!isBackfilled) return { isBackfilled: false, recordedAt: null };
|
||||
const t = event.data?.timeCreated ?? event.data?.TimeCreated;
|
||||
const recordedAt = typeof t === "string" && t !== "" && !isNaN(Date.parse(t)) ? new Date(t) : null;
|
||||
return { isBackfilled, recordedAt };
|
||||
}
|
||||
|
||||
const BACKFILL_TOOLTIP =
|
||||
"Recorded by Windows before the agent started — replayed from the event log for visibility. " +
|
||||
"If enrollment progressed afterwards, this error was already resolved by a retry. " +
|
||||
"The timestamp shown is the original event-log time.";
|
||||
|
||||
function EventRow({ event, showScriptOutput }: { event: EnrollmentEvent; showScriptOutput?: boolean }) {
|
||||
const [showDetails, setShowDetails] = useState(false);
|
||||
const [showRaw, setShowRaw] = useState(false);
|
||||
@@ -429,15 +447,28 @@ function EventRow({ event, showScriptOutput }: { event: EnrollmentEvent; showScr
|
||||
const hasData = isTruncated || (detailData && Object.keys(detailData).length > 0);
|
||||
const hasDetails = true; // Every event has at least the metadata block
|
||||
|
||||
const { isBackfilled, recordedAt } = getBackfillInfo(event);
|
||||
|
||||
return (
|
||||
<div id={`event-${event.eventId}`} className="bg-gray-50 rounded-lg p-3 hover:bg-gray-100 transition-colors">
|
||||
<div className="flex items-start justify-between">
|
||||
<div className="flex-1 min-w-0">
|
||||
<div className="flex items-center gap-3">
|
||||
<span className="text-xs text-gray-500 font-mono">
|
||||
{new Date(event.timestamp).toLocaleTimeString()}
|
||||
{(recordedAt ?? new Date(event.timestamp)).toLocaleTimeString()}
|
||||
{recordedAt && (
|
||||
<span className="text-gray-400 font-sans"> (reported {new Date(event.timestamp).toLocaleTimeString()})</span>
|
||||
)}
|
||||
</span>
|
||||
<SeverityBadge severity={event.severity} />
|
||||
{isBackfilled && (
|
||||
<span
|
||||
className="px-2 py-0.5 rounded text-xs font-medium bg-slate-200 text-slate-600"
|
||||
title={BACKFILL_TOOLTIP}
|
||||
>
|
||||
⟲ Backfilled
|
||||
</span>
|
||||
)}
|
||||
<span className="text-sm font-medium text-gray-900">{event.eventType}</span>
|
||||
</div>
|
||||
<p className="mt-1 text-sm text-gray-600" title={event.message || undefined}>{shortenBuildHashInMessage(event.message)}</p>
|
||||
@@ -608,6 +639,15 @@ function EventRow({ event, showScriptOutput }: { event: EnrollmentEvent; showScr
|
||||
<span className="w-16 flex-shrink-0 text-gray-400">EventId</span>
|
||||
<span className="font-mono">{event.eventId}</span>
|
||||
</div>
|
||||
{recordedAt && (
|
||||
<div className="flex mt-0.5">
|
||||
<span className="w-16 flex-shrink-0 text-gray-400">Recorded</span>
|
||||
<span className="font-mono">
|
||||
{recordedAt.toISOString().replace('T', ' ').replace('Z', '')}
|
||||
<span className="text-gray-400 ml-1" title={BACKFILL_TOOLTIP}>(event log, pre-agent)</span>
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex mt-0.5">
|
||||
<span className="w-16 flex-shrink-0 text-gray-400">Created</span>
|
||||
<span className="font-mono">{event.timestamp}</span>
|
||||
@@ -751,6 +791,7 @@ function RawEventRow({ event }: { event: EnrollmentEvent }) {
|
||||
const [expanded, setExpanded] = useState(false);
|
||||
const detailData = useMemo(() => normalizeEventDataForDisplay(event.data), [event.data]);
|
||||
const hasDetails = detailData && Object.keys(detailData).length > 0;
|
||||
const { isBackfilled, recordedAt } = getBackfillInfo(event);
|
||||
|
||||
const sevColor: Record<string, string> = {
|
||||
Trace: "text-purple-500",
|
||||
@@ -765,8 +806,11 @@ function RawEventRow({ event }: { event: EnrollmentEvent }) {
|
||||
<div id={`event-${event.eventId}`} className="py-1.5 text-xs font-mono">
|
||||
<div className="flex items-start gap-2">
|
||||
<span className="text-gray-400 w-8 text-right flex-shrink-0">{event.sequence}</span>
|
||||
<span className="text-gray-500 flex-shrink-0">{new Date(event.timestamp).toLocaleTimeString()}</span>
|
||||
<span className="text-gray-500 flex-shrink-0">{(recordedAt ?? new Date(event.timestamp)).toLocaleTimeString()}</span>
|
||||
<span className={`flex-shrink-0 w-14 ${sevColor[event.severity] || "text-gray-500"}`}>{event.severity}</span>
|
||||
{isBackfilled && (
|
||||
<span className="text-slate-500 flex-shrink-0" title={BACKFILL_TOOLTIP}>⟲</span>
|
||||
)}
|
||||
<span className="text-gray-900 font-medium flex-shrink-0">{event.eventType}</span>
|
||||
<span className="text-gray-500 truncate flex-1 min-w-0" title={event.message || undefined}>{shortenBuildHashInMessage(event.message)}</span>
|
||||
{hasDetails && (
|
||||
|
||||
@@ -167,6 +167,8 @@ interface TenantConfigContextValue {
|
||||
setEnableTimezoneAutoSet: (v: boolean) => void;
|
||||
enableImeMatchLog: boolean;
|
||||
setEnableImeMatchLog: (v: boolean) => void;
|
||||
enableGatherRuleDebugLog: boolean;
|
||||
setEnableGatherRuleDebugLog: (v: boolean) => void;
|
||||
logLevel: string;
|
||||
setLogLevel: (v: string) => void;
|
||||
showScriptOutput: boolean;
|
||||
@@ -384,6 +386,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
|
||||
const [enableGeoLocation, setEnableGeoLocation] = useState(true);
|
||||
const [enableTimezoneAutoSet, setEnableTimezoneAutoSet] = useState(false);
|
||||
const [enableImeMatchLog, setEnableImeMatchLog] = useState(false);
|
||||
const [enableGatherRuleDebugLog, setEnableGatherRuleDebugLog] = useState(false);
|
||||
const [logLevel, setLogLevel] = useState("Info");
|
||||
const [showScriptOutput, setShowScriptOutput] = useState(true);
|
||||
const [showEnrollmentSummary, setShowEnrollmentSummary] = useState(false);
|
||||
@@ -503,6 +506,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
|
||||
setEnableGeoLocation(data.enableGeoLocation ?? true);
|
||||
setEnableTimezoneAutoSet(data.enableTimezoneAutoSet ?? false);
|
||||
setEnableImeMatchLog(data.enableImeMatchLog ?? false);
|
||||
setEnableGatherRuleDebugLog(data.enableGatherRuleDebugLog ?? false);
|
||||
setLogLevel(data.logLevel ?? "Info");
|
||||
setShowScriptOutput(data.showScriptOutput ?? true);
|
||||
setShowEnrollmentSummary(data.showEnrollmentSummary ?? false);
|
||||
@@ -700,6 +704,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
|
||||
enableGeoLocation,
|
||||
enableTimezoneAutoSet,
|
||||
enableImeMatchLog,
|
||||
enableGatherRuleDebugLog,
|
||||
logLevel,
|
||||
showScriptOutput,
|
||||
showEnrollmentSummary,
|
||||
@@ -777,7 +782,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
|
||||
manufacturerWhitelist, modelWhitelist, validateAutopilotDevice, validateCorporateIdentifier, validateDeviceAssociation,
|
||||
dataRetentionDays, sessionTimeoutHours, enablePerformanceCollector, performanceCollectorInterval,
|
||||
helloWaitTimeoutSeconds, selfDestructOnComplete, keepLogFile, rebootOnComplete, rebootDelaySeconds,
|
||||
contactEmail, enableGeoLocation, enableTimezoneAutoSet, enableImeMatchLog, logLevel, showScriptOutput, showEnrollmentSummary,
|
||||
contactEmail, enableGeoLocation, enableTimezoneAutoSet, enableImeMatchLog, enableGatherRuleDebugLog, logLevel, showScriptOutput, showEnrollmentSummary,
|
||||
enrollmentSummaryTimeoutSeconds, enrollmentSummaryBrandingImageUrl, enrollmentSummaryLaunchRetrySeconds,
|
||||
notificationChannels,
|
||||
slaTargetSuccessRate, slaTargetMaxDurationMinutes, slaTargetAppInstallSuccessRate,
|
||||
@@ -1113,6 +1118,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
|
||||
setEnableGeoLocation(config.enableGeoLocation ?? true);
|
||||
setEnableTimezoneAutoSet(config.enableTimezoneAutoSet ?? false);
|
||||
setEnableImeMatchLog(config.enableImeMatchLog ?? false);
|
||||
setEnableGatherRuleDebugLog(config.enableGatherRuleDebugLog ?? false);
|
||||
setLogLevel(config.logLevel ?? "Info");
|
||||
setShowScriptOutput(config.showScriptOutput ?? true);
|
||||
setShowEnrollmentSummary(config.showEnrollmentSummary ?? false);
|
||||
@@ -1542,6 +1548,7 @@ export function TenantConfigProvider({ children }: { children: React.ReactNode }
|
||||
enableGeoLocation, setEnableGeoLocation,
|
||||
enableTimezoneAutoSet, setEnableTimezoneAutoSet,
|
||||
enableImeMatchLog, setEnableImeMatchLog,
|
||||
enableGatherRuleDebugLog, setEnableGatherRuleDebugLog,
|
||||
logLevel, setLogLevel,
|
||||
showScriptOutput, setShowScriptOutput,
|
||||
showEnrollmentSummary, setShowEnrollmentSummary,
|
||||
|
||||
@@ -17,6 +17,7 @@ export function SectionAgentSettings() {
|
||||
enableGeoLocation, setEnableGeoLocation,
|
||||
enableTimezoneAutoSet, setEnableTimezoneAutoSet,
|
||||
enableImeMatchLog, setEnableImeMatchLog,
|
||||
enableGatherRuleDebugLog, setEnableGatherRuleDebugLog,
|
||||
logLevel, setLogLevel,
|
||||
showScriptOutput, setShowScriptOutput,
|
||||
showEnrollmentSummary, setShowEnrollmentSummary,
|
||||
@@ -51,6 +52,8 @@ export function SectionAgentSettings() {
|
||||
setEnableTimezoneAutoSet={setEnableTimezoneAutoSet}
|
||||
enableImeMatchLog={enableImeMatchLog}
|
||||
setEnableImeMatchLog={setEnableImeMatchLog}
|
||||
enableGatherRuleDebugLog={enableGatherRuleDebugLog}
|
||||
setEnableGatherRuleDebugLog={setEnableGatherRuleDebugLog}
|
||||
logLevel={logLevel}
|
||||
setLogLevel={setLogLevel}
|
||||
showScriptOutput={showScriptOutput}
|
||||
|
||||
@@ -24,6 +24,8 @@ interface AgentSettingsSectionProps {
|
||||
setEnableTimezoneAutoSet: (value: boolean) => void;
|
||||
enableImeMatchLog: boolean;
|
||||
setEnableImeMatchLog: (value: boolean) => void;
|
||||
enableGatherRuleDebugLog: boolean;
|
||||
setEnableGatherRuleDebugLog: (value: boolean) => void;
|
||||
logLevel: string;
|
||||
setLogLevel: (value: string) => void;
|
||||
showScriptOutput: boolean;
|
||||
@@ -64,6 +66,8 @@ export default function AgentSettingsSection({
|
||||
setEnableTimezoneAutoSet,
|
||||
enableImeMatchLog,
|
||||
setEnableImeMatchLog,
|
||||
enableGatherRuleDebugLog,
|
||||
setEnableGatherRuleDebugLog,
|
||||
logLevel,
|
||||
setLogLevel,
|
||||
showScriptOutput,
|
||||
@@ -199,6 +203,21 @@ export default function AgentSettingsSection({
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Gather Rule Debug Log */}
|
||||
<div className="flex items-center justify-between p-4 rounded-lg border border-gray-200 hover:border-violet-200 transition-colors">
|
||||
<div>
|
||||
<p className="font-medium text-gray-900">Gather Rule Debug Log</p>
|
||||
<p className="text-sm text-gray-500">
|
||||
Trace gather rule evaluation to a local file to diagnose rules that produce no timeline events. The trace never leaves the device.
|
||||
{enableGatherRuleDebugLog && <span className="block text-xs text-gray-400 mt-0.5 font-mono">%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log</span>}
|
||||
</p>
|
||||
</div>
|
||||
<button onClick={() => setEnableGatherRuleDebugLog(!enableGatherRuleDebugLog)}
|
||||
className={`relative inline-flex h-6 w-11 flex-shrink-0 items-center rounded-full transition-colors ${enableGatherRuleDebugLog ? 'bg-violet-500' : 'bg-gray-300'}`}>
|
||||
<span className={`inline-block h-4 w-4 transform rounded-full bg-white transition-transform ${enableGatherRuleDebugLog ? 'translate-x-6' : 'translate-x-1'}`} />
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Show Script Output */}
|
||||
<div className="flex items-center justify-between p-4 rounded-lg border border-gray-200 hover:border-violet-200 transition-colors">
|
||||
<div className="flex-1">
|
||||
|
||||
@@ -60,6 +60,7 @@ export interface TenantConfiguration {
|
||||
enableGeoLocation?: boolean;
|
||||
enableTimezoneAutoSet?: boolean;
|
||||
enableImeMatchLog?: boolean;
|
||||
enableGatherRuleDebugLog?: boolean;
|
||||
logLevel?: string;
|
||||
// Teams notifications (legacy)
|
||||
teamsWebhookUrl?: string;
|
||||
|
||||
@@ -315,10 +315,9 @@ export function GlobalSidebar({ children }: { children: ReactNode }) {
|
||||
// Every nav Link below sets prefetch={false} on purpose. The sidebar renders the whole
|
||||
// navigation on every page, so the App Router's default viewport prefetch fires one RSC
|
||||
// request per visible link against the SWA-hosted Next.js runtime on each page load
|
||||
// (measured: ~10k requests / 14 days). That host speaks HTTP/1.1, so those prefetches
|
||||
// occupy the browser's 6 connections to the SAME origin that serves the document and the
|
||||
// route chunks — when the runtime queues them (p95 ~0.9s but a tail reaching 150s+), the
|
||||
// portal freezes until the queue drains. Every route here is a client component behind
|
||||
// (measured: ~10k requests / 14 days). The SSR runtime queues that burst server-side and
|
||||
// drains it all at once (p95 ~0.9s but a tail reaching 150s+), so the navigation the user
|
||||
// actually wants is stuck behind it and the portal freezes. Every route here is a client component behind
|
||||
// ProtectedRoute that fetches its own data after mount, so the prefetched RSC payload
|
||||
// buys nothing. See docs/web/portal-navigation-prefetch.md.
|
||||
// --- Render a global nav link ---
|
||||
|
||||
@@ -47,6 +47,8 @@ export interface Session {
|
||||
avgApiLatencyMs?: number;
|
||||
/** Number of HTTP requests behind avgApiLatencyMs — the weight when averaging across sessions. */
|
||||
apiRequestCount?: number;
|
||||
/** Active network connection type during enrollment ("WiFi" or "Ethernet", last emission wins). Absent for sessions predating the projection. */
|
||||
connectionType?: string;
|
||||
/**
|
||||
* Compact JSON snapshot of "last known session state" written by the maintenance
|
||||
* 5h-timeout sweep when a session graduates to terminal Failed (Hybrid User-Driven
|
||||
|
||||
Reference in New Issue
Block a user