mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-24 20:53:06 +00:00
Compare commits
22 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 977c5fb447 | |||
| 27fdd19568 | |||
| 72f7adcb3d | |||
| 06cf1c4c12 | |||
| e612c69178 | |||
| 829a841d9b | |||
| 10eedded13 | |||
| a9f6ccd86d | |||
| 7ba67df1a1 | |||
| e214e8dc1a | |||
| f8d0ccd2f5 | |||
| c9701301ea | |||
| ba2a390a83 | |||
| dbd7d2ac6f | |||
| 8f24876682 | |||
| 2d5ee887d7 | |||
| 81c14bb3ed | |||
| 3a0bd74b12 | |||
| 50e7590060 | |||
| 61e53a2e9b | |||
| 7e7b6f8086 | |||
| 2e20476559 |
@@ -143,11 +143,29 @@ advisory backstop. All routes converge on `CompleteThroughFinalizingOrDefer`:
|
||||
first-deployment window 100/101 for 200/210 without 110 — the RJ ESP was aborted,
|
||||
typically by an interactive logon reclassifying the run as secondary-user; emits
|
||||
the `realmjoin_first_deployment_incomplete` Warning, session 224b2087), and
|
||||
`Timeout` (hard 60-min deadline). The abort rule reads the *persisted*
|
||||
`Timeout` (60-min deadline from detection). The abort rule reads the *persisted*
|
||||
`RealmJoinFacts.LastDeploymentPhase` (updated by the typed `RealmJoinPhaseChanged`
|
||||
signal), so it survives agent restarts — the `RealmJoinDetected` replay path
|
||||
evaluates it too. A first observation of 210 does **not** release (RJ may have
|
||||
completed before agent boot and deploy afterwards).
|
||||
|
||||
**Activity-based timeout extension** (report 55e6afd61c9d — a large RJ package
|
||||
catalog outlived the 60-min window mid-install): when the timeout deadline fires
|
||||
while the first deployment is demonstrably active — `LastDeploymentPhase` 100/101
|
||||
*and* deployment activity (`RealmJoinPhaseChanged` or a package observation, tracked
|
||||
in `RealmJoinFacts.LastActivityUtc`; the phase captured at detection deliberately
|
||||
does not count) within the last 60 min — the handler re-arms the deadline to
|
||||
`lastActivity + 60 min` instead of timing out, emitting an Info
|
||||
`realmjoin_timeout_extended` event. Re-arms clamp to an absolute cap of
|
||||
**detection + 4 h**. Package activity is only observable at package *completion*
|
||||
(RJ writes the registry subkey when the install finishes), so a single install
|
||||
that is registry-silent for over 60 min still times out — the eventual
|
||||
`realmjoin_timeout` carries a `reason` payload (`hard_timeout` / `inactivity` /
|
||||
`absolute_cap`) telling the three shapes apart. A stale fire of an older deadline
|
||||
incarnation (armed due later than the fire time; `OccurredAtUtc = DueAtUtc` per
|
||||
scheduler contract) dead-ends as `realmjoin_timeout_stale_superseded_by_rearm`.
|
||||
The idle case (detected, no deployment activity ever) times out at 60 min
|
||||
unchanged, and phase 200/210 activity never extends.
|
||||
2. `Finalizing` stage + ~5 s `FinalizingGrace` deadline.
|
||||
3. Grace fires → `Completed`, `EnrollmentComplete` outcome, `enrollment_complete`
|
||||
timeline event with full audit payload.
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
---
|
||||
type: Concept
|
||||
title: DO GroupId Auto-Set — Delivery Optimization peering group from a network fingerprint
|
||||
description: EnableDoGroupIdAutoSet (ConfigVersion 40) makes the agent set the DOGroupId policy value to a deterministic GUID derived from the default gateway's IPv4 address and MAC (RealmJoin-compatible byte layout), so devices on the same local network form one DO peering group during enrollment; foreign DOGroupId/DOGroupIdSource policies always win.
|
||||
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Transport/DoGroupIdService.cs
|
||||
tags:
|
||||
- agent
|
||||
- delivery-optimization
|
||||
- network
|
||||
- tenant-configuration
|
||||
timestamp: 2026-08-21T00:00:00+02:00
|
||||
---
|
||||
|
||||
# DO GroupId Auto-Set
|
||||
|
||||
## Schema
|
||||
|
||||
### The problem
|
||||
|
||||
With Delivery Optimization download mode Group (2), peering is scoped by GroupId. When no
|
||||
GroupId is configured, Windows derives it from AD site → authenticated domain SID → Entra
|
||||
tenant ID — for a cloud-native tenant that means **one tenant-wide group**, so "peers on
|
||||
the same network" degrades to "peers anywhere in the tenant" and DO's cloud service has to
|
||||
match them by public IP. A GroupId derived from the local network makes the group exactly
|
||||
"devices behind the same gateway" — including branch offices behind NAT. RealmJoin ships
|
||||
this as its network-fingerprint GroupId; our agent is on the device even earlier (during
|
||||
ESP, when the big app downloads actually happen), so it can seed the same value for
|
||||
Autopilot enrollments.
|
||||
|
||||
### The setting
|
||||
|
||||
`TenantConfiguration.EnableDoGroupIdAutoSet` (`bool?`, default null→false), served as
|
||||
`AgentConfigResponse.EnableDoGroupIdAutoSet` — **ConfigVersion 40**. Portal: Settings →
|
||||
Agent → Settings, toggle "Set Delivery Optimization Group ID", stored via the usual
|
||||
tenant-config roundtrip (`TableConfigRepository` Store+Map). Also shown in the admin
|
||||
tenant-config report. `RemoteConfigMerger` maps the bool 1:1; it survives the offline
|
||||
config cache like every plain feature flag.
|
||||
|
||||
The agent does NOT configure `DODownloadMode` — the fingerprint GroupId only has an
|
||||
effect when the tenant already deploys download mode Group (2) via Intune/GPO.
|
||||
|
||||
### The fingerprint
|
||||
|
||||
`DoGroupIdService.ComputeGroupId(ipv4Bytes, macBytes)` builds a 16-byte GUID:
|
||||
|
||||
```
|
||||
{101,48,6} + {0,0,0} + {ip[1], 0, ip[2], ip[3]} + mac[0..5] → new Guid(byte[16])
|
||||
```
|
||||
|
||||
Gateway discovery reuses the shared `NetworkInterfaceLocator` heuristic (Up, not
|
||||
Loopback/Tunnel, non-0.0.0.0 gateway; IPv4 gateway required), the gateway MAC comes from
|
||||
`ArpNativeMethods` (`iphlpapi!SendARP`). The byte layout — including the IP shuffle and
|
||||
the zero padding — is **RealmJoin-compatible and must never change**: devices managed by
|
||||
RealmJoin on the same network compute the identical GUID and peer with ours. The registry
|
||||
value is written as `Guid.ToString()` (braceless lowercase), identical to RealmJoin's
|
||||
`SetValue(name, Guid)` string form. Deliberate divergence from RealmJoin: **no
|
||||
random-GUID fallback** — a random per-device ID would isolate the device in a one-member
|
||||
group, which is worse than the OS default grouping, so no gateway/no MAC means skip.
|
||||
|
||||
### Ownership + conflict rules (never fight a foreign policy)
|
||||
|
||||
Target: `HKLM\SOFTWARE\Policies\Microsoft\Windows\DeliveryOptimization\DOGroupId`
|
||||
(64-bit registry view — the Policies store is WOW64-redirected; key paths reused from
|
||||
`DoThrottlePolicyReader`). Only this GPO-store value is written; the legacy pre-1607
|
||||
`GroupId` value name is intentionally not.
|
||||
|
||||
Skip conditions (Info event with `skipReason`, nothing written):
|
||||
|
||||
| skipReason | Meaning |
|
||||
|---|---|
|
||||
| `no_ipv4_gateway` | No active interface with an IPv4 default gateway |
|
||||
| `arp_failed` | Gateway MAC could not be resolved via SendARP |
|
||||
| `group_id_source_policy` | `DOGroupIdSource` configured in GPO or MDM store (dynamic group source wins) |
|
||||
| `mdm_policy_present` | `DOGroupId` managed via the Policy CSP store (Intune owns it) |
|
||||
| `foreign_value_present` | Existing GPO value neither matches the computed GUID nor the one we last wrote |
|
||||
|
||||
Ownership is tracked through the `StartupEventGate` fingerprint for the
|
||||
`do_group_id_auto_set` key: `written:<guid>` records the GUID we last wrote, so after a
|
||||
gateway change the agent recognizes its own stale value and updates it — while a value
|
||||
written by anyone else is never overwritten (`DecideAction`: empty→write, equal→no-op,
|
||||
ours→update, else→skip; unparseable existing values count as foreign). Every write is
|
||||
verified by readback.
|
||||
|
||||
### The event
|
||||
|
||||
One `do_group_id_auto_set` event per outcome change (Source `StartupEnvironmentProbes`,
|
||||
gated on the fingerprint — unchanged reruns across reboots stay silent; errors never
|
||||
latch, so each failing start keeps its retry trail). Severity: Info for
|
||||
success/skip, Warning for errors. Data: `groupId`, `previousValue`, `gatewayIp`,
|
||||
`gatewayMac`, `written`, `skipped`, `skipReason`, `success`, `error`.
|
||||
|
||||
## Examples
|
||||
|
||||
First boot on a network with gateway `192.168.178.1` / MAC `00-11-22-33-44-55`:
|
||||
|
||||
```
|
||||
do_group_id_auto_set (Info): Delivery Optimization GroupId set to
|
||||
00063065-0000-00a8-b201-001122334455 (gateway 192.168.178.1)
|
||||
```
|
||||
|
||||
Tenant already deploys a DO GroupId via Intune:
|
||||
|
||||
```
|
||||
do_group_id_auto_set (Info): Delivery Optimization GroupId not set: mdm_policy_present
|
||||
```
|
||||
|
||||
## Citations
|
||||
|
||||
* Service (fingerprint, ownership, write+readback): `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Transport/DoGroupIdService.cs`
|
||||
* ARP interop: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Interop/ArpNativeMethods.cs`
|
||||
* Probe wiring + event + fingerprint dedup: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Runtime/StartupEnvironmentProbes.cs`
|
||||
* Backend serve + ConfigVersion: `../../src/Backend/AutopilotMonitor.Functions/Functions/Config/GetAgentConfigFunction.cs`
|
||||
* DO policy stores (key paths, Registry64 rationale): `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Periodic/DoThrottlePolicyReader.cs`
|
||||
@@ -8,5 +8,8 @@ How the on-device agent works:
|
||||
* [Hello Wizard Un-Skip](hello-wizard-unskip.md) - why a single "WHfB disabled" policy read never decides completion: the HelloWizardStarted signal (prevention + cure) and the HelloTracker confirmation second read.
|
||||
* [Autopilot ZTD Diagnostics](autopilot-ztd-diagnostics.md) - Windows' own diagnostic surfaces for the profile-download flow: ModernDeployment event IDs (807/815/164/...), the Diagnostics\Autopilot registry key, deployment-service endpoints, and the known-issue error-code map.
|
||||
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated 2800 observations (neutral, flush-time watermark) + ANALYZE-ESP-005 gated on an actually-observed reboot.
|
||||
* [Power-State Watcher](power-state-watcher.md) - live AC↔battery transitions and 50/30/15% downward threshold crossings via a WMI push subscription (no polling): power_state_change semantics (latch/debounce/cap, never armed without a battery) and the single-condition analyze rules ANALYZE-DEV-009/-010 on top.
|
||||
* [DO GroupId Auto-Set](do-group-id-auto-set.md) - the optional Delivery Optimization peering-group seed: deterministic GUID from default gateway IPv4 + MAC (RealmJoin-compatible), written to the DOGroupId policy value with readback, ownership-tracked so foreign Intune/GPO values are never overwritten; skip reasons and the do_group_id_auto_set event.
|
||||
* [System Timeline Watcher](system-timeline-watcher.md) - clock steps (system_clock_changed: oldTime/newTime/timeDeltaMs/process, micro-slew floor) and completed sleep episodes (system_sleep_episode: sleep/hibernate/Modern Standby with authoritative enteredAt/exitedAt) from one System-channel EventLogWatcher with pre-agent backfill; feeds the web clock-set jump cause, the standby duration note, time-attribution SleepSpans and ANALYZE-DEV-012.
|
||||
|
||||
Reading order for newcomers: overview → decision engine → persistence.
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
---
|
||||
type: Concept
|
||||
title: Power-State Watcher — live AC/battery transitions and threshold events during enrollment
|
||||
description: How the agent observes AC↔battery switches and downward battery-threshold crossings mid-enrollment via a WMI push subscription (no polling), the power_state_change event semantics (transition field, 50/30/15 ladder, latch/debounce/cap), and the two analyze rules on top (ANALYZE-DEV-009/-010).
|
||||
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Orchestration/Hosts/PowerStateWatcherHost.cs
|
||||
tags:
|
||||
- agent
|
||||
- power
|
||||
- battery
|
||||
- collector
|
||||
- wmi
|
||||
- analyze-rules
|
||||
timestamp: 2026-08-21T00:00:00+02:00
|
||||
---
|
||||
|
||||
# Power-State Watcher
|
||||
|
||||
The startup probe (`power_state_check`, one-shot from `StartupEnvironmentProbes` via
|
||||
`PowerStateProbe`/`GetSystemPowerStatus`) only captures the power state at agent start. A
|
||||
device unplugged mid-enrollment — or draining its battery across a multi-hour ESP — was
|
||||
invisible. The watcher closes that gap **event-driven, with zero polling**.
|
||||
|
||||
# Mechanism
|
||||
|
||||
- `PowerStateWatcherHost` (`Orchestration/Hosts`) arms a WMI event subscription with a
|
||||
two-step chain:
|
||||
1. `SELECT * FROM Win32_PowerManagementEvent WHERE EventCode = 10` (PBT_APMPOWERSTATUSCHANGE:
|
||||
true kernel push, fires on AC↔DC switches and battery-percentage steps).
|
||||
2. **Fallback** `SELECT * FROM __InstanceModificationEvent WITHIN 30 WHERE TargetInstance
|
||||
ISA 'Win32_Battery'` — required because on Windows 11 24H2/25H2 (observed on builds
|
||||
26200 and 26220, session 161b838c) the `MS_Power_Management_Event_Provider` is still
|
||||
registered in the repository but its activation fails with `WBEM_E_NOT_FOUND`: the
|
||||
legacy push provider no longer loads on exactly the OS builds new devices enroll with.
|
||||
The WITHIN sampling runs inside WinMgmt (one Win32_Battery snapshot every 30 s), not in
|
||||
the agent; worst-case detection latency ≈ 35 s incl. debounce. `collector_degraded`
|
||||
(`watcher_arm_failed`) is emitted only when BOTH attempts fail; the agent log's arm
|
||||
line names the active mode.
|
||||
|
||||
Either way the arriving event's payload is ignored — each arrival re-probes via the
|
||||
existing `PowerStateProbe.Probe()` (cheap P/Invoke).
|
||||
- Snapshots are diffed by the pure `PowerStateTransitionTracker`
|
||||
(`Monitoring/Enrollment/SystemSignals`), which decides what to emit. The host only owns
|
||||
WMI, a 5 s trailing-edge debounce timer (dock-flap AC→battery→AC inside the window
|
||||
collapses to a no-diff) and the emit path (`InformationalEventPost`, single rail).
|
||||
- **No battery / probe error ⇒ WMI is never armed** — desktops and VMs pay nothing. WMI
|
||||
arming runs on a background thread (WinMgmt can hang in OOBE; `ConsoleBypassWatcher`
|
||||
precedent); an arm failure emits one `collector_degraded` (`watcher_arm_failed`).
|
||||
- Registered as a plain always-on peripheral host after `NetworkChangeHost`, deliberately
|
||||
NOT under `PeriodicCollectorLifecycleHost`: it is push-based (zero idle cost) and must be
|
||||
alive precisely when the device idles on battery. Kill-switch:
|
||||
`Collectors.EnablePowerStateWatcher` (default true, compile-time default like the other
|
||||
collector toggles; ConfigVersion 38).
|
||||
|
||||
# Event: power_state_change
|
||||
|
||||
`data.transition` discriminates three shapes (payload always carries `onAcPower`,
|
||||
`batteryPercent`, `isCharging`, `batteryLifeMinutes`; unknowns as `"unknown"`):
|
||||
|
||||
| transition | Severity | Immediate | Notes |
|
||||
|---|---|---|---|
|
||||
| `ac_to_battery` | Warning | yes | one per accepted edge |
|
||||
| `battery_to_ac` | Info | no | recovery |
|
||||
| `threshold_crossed` | Info (50) / Warning (30) / Error (15) | 30+15 only | extra field `thresholdPercent` |
|
||||
|
||||
Threshold semantics (tracker):
|
||||
|
||||
- Ladder 50/30/15, evaluated only while ON battery; **each level latches once per agent
|
||||
lifetime** (charge-up + re-drain does not re-emit — no plug-cycle alarm spam).
|
||||
- A multi-level jump emits only the LOWEST newly-crossed level (60%→10% ⇒ one `15` event).
|
||||
- Entering battery power already below a level counts as crossing it; the arm-time baseline
|
||||
does too — an enrollment STARTING at 12% battery emits its `thresholdPercent: 15` event
|
||||
immediately (the analyze rule would otherwise never see sessions that began low).
|
||||
- Lifetime emission cap 20 (storm backstop against a flapping dock / dying battery
|
||||
controller); the host logs one warning when it engages.
|
||||
|
||||
Backend/agent hygiene: `power_state_change` is classified as non-activity on both sides —
|
||||
`SignalActivityClassifier.NonActivityEventTypes` (agent: must not reset the
|
||||
PeriodicCollectorLifecycleHost/StallProbeHost idle clocks) and
|
||||
`EventIngestProcessor.IsPeriodicOrStallEvent` (backend: must not heal a Stalled session —
|
||||
battery drain is not enrollment progress).
|
||||
|
||||
# Analyze Rules
|
||||
|
||||
- **ANALYZE-DEV-009** (high): battery crossed 15% while on battery — matches the
|
||||
agent-stamped `thresholdPercent equals 15`.
|
||||
- **ANALYZE-DEV-010** (warning): AC→battery switch — matches `transition equals
|
||||
ac_to_battery`.
|
||||
- Both: `evaluateOn: ["enrollment_end", "whiteglove_sealed", "on_event:power_state_change"]`
|
||||
(live interim finding + terminal finalization), `markSessionAsFailedDefault: false`.
|
||||
- Both are deliberately **single-required-condition**: `event_data` conditions scan events
|
||||
independently (no same-instance join), so a two-condition rule (e.g. `transition ==
|
||||
threshold_crossed AND batteryPercent < 15`) could false-positive across two different
|
||||
power events. `thresholdPercent` exists only on threshold events and its value pins the
|
||||
ladder level, so one condition fully identifies the instance.
|
||||
- `{{token}}` caveat: `AddDataFieldsToEvidence` whitelists only generic identifiers
|
||||
(appId/errorCode/…), so `{{batteryPercent}}` would render literally. DEV-009 interpolates
|
||||
`{{thresholdPercent}}` (the matched condition's own dataField); the exact percentage lives
|
||||
on the linked event in the timeline.
|
||||
|
||||
# Citations
|
||||
|
||||
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Orchestration/Hosts/PowerStateWatcherHost.cs`
|
||||
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/PowerStateTransitionTracker.cs`
|
||||
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Runtime/PowerStateProbe.cs` (probe, unchanged)
|
||||
- `rules/analyze/ANALYZE-DEV-009.json`, `rules/analyze/ANALYZE-DEV-010.json`
|
||||
- [analyze-rule-triggers](../rules/analyze-rule-triggers.md) (interim-trigger design; the WG battery rule was its motivating case)
|
||||
@@ -0,0 +1,110 @@
|
||||
---
|
||||
type: Concept
|
||||
title: System Timeline Watcher — clock steps and sleep episodes from the System event log
|
||||
description: How the agent turns two enrollment blind spots into timeline ground truth via one EventLogWatcher on the System channel — system_clock_changed (Kernel-General 1, oldTime/newTime/timeDeltaMs/process) and system_sleep_episode (Power-Troubleshooter 1 classic sleep/hibernate, Kernel-Power 507 Modern Standby) — including backfill of pre-agent records, the payload-times-are-authoritative rule, SleepSpans in time attribution, the web clock-set jump cause, and ANALYZE-DEV-012.
|
||||
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/SystemTimelineTracker.cs
|
||||
tags:
|
||||
- agent
|
||||
- eventlog
|
||||
- time
|
||||
- clock-skew
|
||||
- standby
|
||||
- collector
|
||||
- time-attribution
|
||||
- analyze-rules
|
||||
timestamp: 2026-08-21T00:00:00+02:00
|
||||
---
|
||||
|
||||
# System Timeline Watcher
|
||||
|
||||
Two things silently distort every session timeline and its wall-clock duration:
|
||||
|
||||
1. **The OS clock gets stepped** — classically w32time correcting a wrong BIOS clock early
|
||||
in OOBE, but also tzutil/MDM actions. Before this watcher the jump was only *inferred*
|
||||
(TimeJumpBadge, `clock_skew`/DEV-008); Windows records the ground truth.
|
||||
2. **The device sleeps mid-enrollment** — the timeline shows an unexplained gap, the
|
||||
duration looks inflated, and nothing says "the device was simply asleep for 56 minutes".
|
||||
|
||||
`SystemTimelineTracker` (hosted by `SystemTimelineWatcherHost`, config flag
|
||||
`Collectors.EnableSystemTimelineWatcher`, ConfigVersion 39) subscribes to the **System**
|
||||
channel with one provider-qualified XPath and forwards both facts as events.
|
||||
|
||||
# Sources and events
|
||||
|
||||
| Source (System channel) | Emitted event | Key payload |
|
||||
|---|---|---|
|
||||
| `Microsoft-Windows-Kernel-General` EventID 1 | `system_clock_changed` | `oldTime`, `newTime` (ISO-8601), signed `timeDeltaMs`, `reason`/`reasonText` (1=`application_set`, 2=`hardware_clock_sync`), `processName`/`processId` |
|
||||
| `Microsoft-Windows-Power-Troubleshooter` EventID 1 | `system_sleep_episode` | `kind`=`sleep`/`hibernate` (EffectiveState 5 ⇒ hibernate), `enteredAt`=SleepTime, `exitedAt`=WakeTime, `durationSeconds`, `wakeSourceType`/`wakeSourceText` |
|
||||
| `Microsoft-Windows-Kernel-Power` EventID 507 | `system_sleep_episode` | `kind`=`modern_standby`, `exitedAt`=TimeCreated, `enteredAt`=exit − DurationInUs, `durationSeconds` (scenario), `sleepDurationSeconds` (real sleep), `reason`, `onAcPower`, `batteryRemainingCapacityOnExit` |
|
||||
|
||||
**506 is deliberately not consumed.** 507 carries the full episode; pairing 506/507 would
|
||||
need an orphan-tolerant state machine across agent restarts for zero information gain — an
|
||||
episode without a 507 never completed and would not be emitted under any design.
|
||||
|
||||
# Thresholds (constants, not knobs)
|
||||
|
||||
- `MinClockDeltaMs = 2000` — w32time logs Kernel-General 1 for routine **1 ms micro-slews**
|
||||
(verified live, several per hour); only genuine steps are emitted. Suppressed records are
|
||||
still claimed by the dedup watermark.
|
||||
- `ClockDeltaWarningMs = 5 min` — larger steps are Warning + immediate upload (can break
|
||||
token/cert validation windows; matches the web TimeJumpBadge threshold).
|
||||
- `MinSleepDurationSeconds = 60` — **Modern Standby only**: every screen-off scenario logs a
|
||||
506/507 pair, including seconds-long ones with `SleepDurationInUs 0` (`SleepEntered=false`);
|
||||
the single duration gate suppresses both. Classic Power-Troubleshooter episodes have no
|
||||
floor — they only exist for genuinely completed S3/S4 transitions.
|
||||
|
||||
# Payload times are authoritative
|
||||
|
||||
The decisive clock correction happens exactly when the device clock is untrustworthy, and
|
||||
the backend clamps implausible event timestamps (±168 h/24 h) to server time. Every semantic
|
||||
instant therefore travels as an explicit ISO-8601 payload field (`oldTime`/`newTime`,
|
||||
`enteredAt`/`exitedAt`); consumers must never read the event's own `Timestamp` for these.
|
||||
The event `Timestamp` is set to the record's TimeCreated (WindowsUpdateTracker backfill
|
||||
convention) so rows land where the wake/step happened.
|
||||
|
||||
# Backfill and dedup
|
||||
|
||||
Template: `WindowsUpdateTracker`. `LoadWatermark → arm watcher → backfill` with lookback
|
||||
`SystemEventBackfillLookbackMinutes` (default 1440 = 24 h; 0 disables). The System .evtx
|
||||
persists across OOBE reboots, so the pre-agent w32time correction and pre-agent standby
|
||||
episodes are recovered with `backfilled=true`. Cross-restart dedup via RecordId watermark
|
||||
(`system-timeline-watermark.json` in the state directory) plus an intra-run `HashSet` —
|
||||
deliberately **not** a high-water mark, which would drop exactly the older, never-emitted
|
||||
pre-agent records (the live watcher arms before the backfill scan).
|
||||
|
||||
# Non-activity classification (both sides)
|
||||
|
||||
Both event types are environment observation, not enrollment progress:
|
||||
`SignalActivityClassifier.NonActivityEventTypes` (agent — must not reset idle/stall clocks)
|
||||
and `EventIngestProcessor.IsPeriodicOrStallEvent` (backend — a device that slept through its
|
||||
stall must stay Stalled).
|
||||
|
||||
# Consumers
|
||||
|
||||
- **Web timeline** (`EventTimeline.tsx`): badge blocks for both events; the TimeJumpBadge
|
||||
gains the ground-truth cause `clock-set` — `classifyTimeJump` receives the session's
|
||||
`system_clock_changed` deltas and matches a backward display step against a backward clock
|
||||
set within max(60 s, 20 %) (`lib/timeProvenance.ts`).
|
||||
- **Duration story** (`SessionInfoCard` via `useSessionDerivedData.standbySeconds`): the
|
||||
wall-clock duration deliberately keeps the pause (the enrollment really took that long);
|
||||
a `· 🌙 56m standby` note explains it. Episodes are dedup'd on `enteredAt`.
|
||||
- **Time attribution** (`TimeAttributionCalculator.BuildSleepSpans`, AttributionVersion 3):
|
||||
`SleepSpan`s mirror `RebootSpan`s — cross-cutting annotations clipped to the observation
|
||||
windows (a WhiteGlove-pause episode contributes only its in-window flanks), never a slice
|
||||
of the wall-clock partition. Persisted as `SleepSeconds`/`SleepSpansJson`; rendered as the
|
||||
standby chip in `TimeAttributionLane`.
|
||||
- **ANALYZE-DEV-012** (info, `trigger: single`, `on_event:system_sleep_episode`): fires once
|
||||
per session for episodes ≥ 5 min. Deliberately the ONLY new rule: clock issues already
|
||||
have DEV-008 (`clock_skew`) — a second clock rule would double-report the same action item
|
||||
(user decision 2026-08-21); clock steps stay timeline ground truth with badges instead.
|
||||
|
||||
# Citations
|
||||
|
||||
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/SystemTimelineTracker.cs`
|
||||
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Orchestration/Hosts/SystemTimelineWatcherHost.cs`
|
||||
- `src/Backend/AutopilotMonitor.Functions/Helpers/TimeAttributionCalculator.cs` (`BuildSleepSpans`)
|
||||
- `src/Web/autopilot-monitor-web/lib/timeProvenance.ts` (`clock-set`, `findExplainingClockChange`)
|
||||
- `rules/analyze/ANALYZE-DEV-012.json`
|
||||
- Live payload evidence (2026-08-21, Win11 26220): Kernel-General 1 with `TimeDeltaInMs=1`
|
||||
micro-slews; Kernel-Power 507 with `SleepDurationInUs=2324095807`; Power-Troubleshooter 1
|
||||
with `SleepTime`/`WakeTime` across a hibernate.
|
||||
@@ -0,0 +1,82 @@
|
||||
---
|
||||
type: concept
|
||||
title: Attempt-Scoped App Install Durations
|
||||
description: AppInstallSummary.DurationSeconds measures the status-defining (last) install attempt via a LastAttemptStartedAt max-fold anchor — replacing the first-observation-to-last-terminal span that billed multi-pass IME processing gaps to trivial apps; time attribution counts the union of active attempt segments.
|
||||
resource: src/Backend/AutopilotMonitor.Functions/Services/EventIngestProcessor.AppInstall.cs
|
||||
tags: [backend, agent, ingest, app-installs, metrics, time-attribution, durations]
|
||||
timestamp: 2026-08-21
|
||||
---
|
||||
|
||||
# Problem
|
||||
|
||||
The IME processes the ESP app list in multiple passes: a device-ESP evaluation pass can
|
||||
touch an app (often ending `Skipped`) long before the real install runs in a later pass.
|
||||
`AppInstallSummary` is keyed by `{SessionId}_{AppName}` and folded from ALL of the app's
|
||||
events, so `DurationSeconds = last terminal − first observation` spanned those passes.
|
||||
Field case (session `44862cd3`, "Set TimeZone"): evaluation pass 09:16, real install
|
||||
10:22:58 → 10:23:21 — reported 4002 s for a 23-second install. Fleet averages, the
|
||||
Slowest-Apps ranking, p95s, and the version-regression radar all inherited the inflation;
|
||||
the rule engine's `app_install_duration` (already attempt-scoped) contradicted the
|
||||
dashboard for the same install.
|
||||
|
||||
# Schema
|
||||
|
||||
Three durable anchors on the row — each with a distinct, non-interchangeable job:
|
||||
|
||||
| Column | Fold | Role |
|
||||
|---|---|---|
|
||||
| `StartedAt` | min (earliest wins) | First observation. Server-side window/bucket filter of every AppInstallSummaries scan and the radar's version-ordering key. Never re-anchored. |
|
||||
| `LastAttemptStartedAt` | **max** (in-batch and in `ReconcileAppInstallSummaryWithExisting`) | Anchor of `DurationSeconds`. Null = sentinel (row predates 2026-08, or no start observed) → duration falls back to the historical `StartedAt` span. Replay of an older batch can never regress it. |
|
||||
| `InstallPassCount` | additive, guarded by `LastAttemptStartedAt` monotonicity (a batch whose newest start is not newer than the stored anchor adds 0) | Explains "finished long after first seen" (web ×N chip); 2+ is normal. |
|
||||
|
||||
`DurationSeconds = CompletedAt − LastAttemptStartedAt`, computed at the terminal event and
|
||||
re-derived in one central reconcile site covering in-order and Q4 out-of-order batches.
|
||||
A `LastAttemptStartedAt` newer than `CompletedAt` means a fresh pass is in flight — the
|
||||
completed attempt's stored duration stands (the 0 value omits the column; Merge preserves).
|
||||
`DownloadDurationSeconds` pairs the latest `app_download_started` with the NEXT
|
||||
`app_install_started` (attempt-scoped, in-batch). The full span stays derivable as
|
||||
`CompletedAt − StartedAt` — deliberately not a column.
|
||||
|
||||
Weaker-terminal guard (both in-batch and cross-batch): `app_install_completed` with
|
||||
state `Skipped`/`Postponed` never overrides an `Installed`/`Error` terminal, nor its
|
||||
`CompletedAt`/`DurationSeconds` — mirroring the long-pinned `app_install_skipped` rule.
|
||||
|
||||
# Consumers
|
||||
|
||||
* **Fleet metrics / apps dashboard / PDF** (`MetricsMath`, `AppsAnalyticsHelper`): values
|
||||
drop to attempt scale; labels say "install time (final attempt)". The 6 h
|
||||
`MaxPlausibleInstallDurationSeconds` gate remains (an attempt longer than the agent's
|
||||
lifetime is still unobservable).
|
||||
* **AppVersionRegressionRadar**: cutover gate in `MeasuredAppGroups` — only rows with
|
||||
`LastAttemptStartedAt` enter the duration population, on BOTH comparison sides. A mixed
|
||||
population would mask real regressions and falsely re-arm active episodes for up to the
|
||||
35-day horizon. `MinMeasuredInstalls` keeps it silent until enough post-cutover rows
|
||||
exist. Thresholds (2.0× / 300 s) were tuned on span data — re-evaluate ~4 weeks after
|
||||
rollout.
|
||||
* **Time attribution** (`TimeAttributionCalculator.BuildBlockingAppIntervals`):
|
||||
sequence-ordered start→terminal pairing yields one segment per pass/attempt. Per-app
|
||||
seconds = SUM of clamped segments (failed attempts and evaluation passes count — they
|
||||
occupied the path); occupancy merges the segments, never the per-app hulls; hull
|
||||
endpoints remain first-start→last-terminal for display and what-if. Deliberately
|
||||
BROADER than the metrics duration: active time across all passes vs. final attempt.
|
||||
* **Agent payloads** (`AppInstallTiming` / `ImeLogTrackerAdapter.UpdateAppTiming`): a
|
||||
terminal→active transition re-arms BOTH endpoints, so event `durationSeconds` also
|
||||
describes the attempt (needs the agent release carrying this change).
|
||||
* **Rule engine** `app_install_duration`: was already attempt-scoped — now consistent.
|
||||
|
||||
# Rollout
|
||||
|
||||
No backfill (user decision, analog to the CMTrace legacy): pre-cutover rows keep span
|
||||
values, so time-series charts step down visibly at the changeover (documented in the
|
||||
public platform changelog). Sentinel-gated columns follow the standard chain:
|
||||
`BuildAppInstallSummaryEntity` + `MapToAppInstallSummary` + `AppsDashboardProjection`
|
||||
(the only projection whose consumers read the new columns).
|
||||
|
||||
# Citations
|
||||
|
||||
* `src/Backend/AutopilotMonitor.Functions/Services/EventIngestProcessor.AppInstall.cs` — fold + weaker-terminal guard + attempt pairing
|
||||
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Metrics.cs` — reconcile max-fold, pass-count guard, central duration recompute
|
||||
* `src/Backend/AutopilotMonitor.Functions/Helpers/AppVersionRegressionRadar.cs` — cutover gate
|
||||
* `src/Backend/AutopilotMonitor.Functions/Helpers/TimeAttributionCalculator.cs` — per-pass segments + occupancy
|
||||
* `src/Agent/AutopilotMonitor.Agent.V2.Core/SignalAdapters/ImeLogTrackerAdapter.cs` — payload timing re-arm
|
||||
* Tests: `AppInstallTerminalStateAndReconcileTests`, `EventTimestampValidationTests`, `AppVersionRegressionRadarTests`, `TimeAttributionCalculatorTests`
|
||||
@@ -30,6 +30,9 @@ https://docs.autopilotmonitor.com (separate repository).
|
||||
* [CMTrace Time Resolution](agent/cmtrace-time-resolution.md) - Converting bias-less CMTrace local times to UTC without trusting any process's timezone belief: per-line self-anchoring on the 15-minute offset grid for provably fresh lines, uniform marked reader-zone fallback for backlog, and the provenance vocabulary (bias / line-anchored / reader-zone-fallback) that keeps every event re-derivable.
|
||||
* [Registry App-State Second Pillar](agent/registry-app-state-pillar.md) - IME's authoritative per-app state read straight from the registry (Win32Apps EnforcementStateMessage, EspTrackingWin32Apps, StatusServiceReports) via a snapshot-and-diff watcher; registry/log reconciliation doubles as the built-in IME log-pattern drift alarm.
|
||||
* [WDP Scenario Gates & Completion Backstop](agent/wdp-scenario-gates.md) - How the agent gates Autopilot/ESP-only machinery off on Windows Device Preparation sessions (deterministic ExecutionContext marker, fail-toward-Classic), which noise it suppresses (ZTD probe, esp_config unknown, ModernDeployment 100/417/418/1005, starved sweep), and the WDP completion model: High-confidence seed, desktop-based arm D, and the 30-min device_prep_completion backstop.
|
||||
* [Power-State Watcher](agent/power-state-watcher.md) - Live AC↔battery transitions and 50/30/15% downward battery-threshold crossings during enrollment via a WMI Win32_PowerManagementEvent push subscription (zero polling, never armed on battery-less devices): the power_state_change event (transition field, per-level lifetime latch, 5s debounce, emission cap) and the analyze rules ANALYZE-DEV-009 (battery <15%, high) / ANALYZE-DEV-010 (AC→battery, warning) with on_event interim triggers.
|
||||
* [System Timeline Watcher](agent/system-timeline-watcher.md) - Ground truth for two timeline blind spots from one System-channel EventLogWatcher with pre-agent backfill: system_clock_changed (Kernel-General 1 — oldTime/newTime/signed timeDeltaMs/reason/setting process; 2 s micro-slew floor, ≥5 min Warning+immediate) and system_sleep_episode (Power-Troubleshooter 1 sleep/hibernate + Kernel-Power 507 Modern Standby; authoritative enteredAt/exitedAt payload instants, 60 s Modern-Standby floor). Consumed by the TimeJumpBadge clock-set cause, the SessionInfoCard standby note, time-attribution SleepSpans (AttributionVersion 3) and ANALYZE-DEV-012.
|
||||
* [DO GroupId Auto-Set](agent/do-group-id-auto-set.md) - EnableDoGroupIdAutoSet (ConfigVersion 40): the agent seeds the Delivery Optimization DOGroupId policy with a deterministic network-fingerprint GUID (default gateway IPv4 + SendARP MAC, RealmJoin-compatible byte layout) so same-network devices form one DO peering group during ESP; ownership-tracked via the StartupEventGate fingerprint, foreign DOGroupId/DOGroupIdSource policies always win, no random-GUID fallback.
|
||||
* [ESP Policy-Provider Stall Detection](agent/esp-policy-provider-stall.md) - The EnrollmentStatusTracking CSP's no-timeout Setup/Apps wait is keyed to the Intune-registered "Sidecar" provider by NAME: co-management's "ConfigMgr" parks the user ESP at "Apps (Identifying)" for days even with TrackingPoliciesCreated=1 (issue #106); the always-on two-arm esp_policy_provider_stalled tripwire (provider_incomplete | sidecar_provider_missing) and the SOFTWARE\Microsoft\Windows\Autopilot gather-rule allowlist entry that surface it.
|
||||
|
||||
# Rules
|
||||
@@ -37,6 +40,7 @@ https://docs.autopilotmonitor.com (separate repository).
|
||||
* [Gather Rule Phase Scoping, One-Shot Triggers & Emit-on-Change](rules/gather-rule-phase-scoping.md) - Restricting gather rules to enrollment phases (activePhases / activeFromPhase sticky latch), one-shot collection at a phase's start/end (phase_change / phase_exit), and emitMode "on_change" result-dedup.
|
||||
* [Gather Rule Guardrails](rules/gather-rule-guardrails.md) - Why the allowlists in rules/guardrails.json are enforced on the agent and nowhere else, which hard blocks survive unrestricted mode, and the rule that every collector must call a guard and emit security_warning on refusal.
|
||||
* [Rule Authoring Surface](rules/rule-authoring-surface.md) - How customer AIs author rules against the real contract: the baked MCP resources (guide/schemas/guardrails via combine.js), the MCP-local validate_rule pre-flight, and the side-effect-free POST rules/analyze/dryrun whose trace twin shares the production evaluators (parity + side-effect-freedom pinned by tests).
|
||||
* [Rule ID Namespace & Collision Guards](rules/rule-id-namespace.md) - Why rule identity is the human-readable ruleId (no GUIDs) with tenant isolation via Table Storage partitioning, the reserved built-in namespace (RuleIdPolicy), which collisions are possible (custom-vs-custom across tenants: allowed and harmless), and the guards keeping the global RuleStats aggregate catalog-only and the global partition inside the reserved namespace (combine.js, reseed, cross-tenant RuleState GC).
|
||||
* [Analyze Rule Evaluation Triggers](rules/analyze-rule-triggers.md) - Opt-in `evaluateOn` interim triggers (`whiteglove_sealed`, `on_event:<type>`) so rules can fire before a session is terminal, with RuleResults update-semantics (interim → finalize/resolve, NotifiedAt notification dedupe) replacing the permanent dedupe freeze, KO/stat suppression on interim runs, the ingest-side cached trigger registry, and the authoring lints that keep interim rules monotonic.
|
||||
|
||||
# Backend
|
||||
@@ -46,6 +50,7 @@ https://docs.autopilotmonitor.com (separate repository).
|
||||
* [OOBE Bootstrap go-Code Flow](backend/oobe-bootstrap-go-flow.md) - The bootstrap short-code lifecycle (portal-created session → `irm go.autopilotmonitor.com/CODE | iex` at Shift+F10 → GetBootstrapScriptFunction), the always-HTTP-200 text/plain contract, the two-line injection defense (GUID/URL/expiry validator + single-quoted PS literals, golden-pinned against the legacy TS template), and the per-IP rate-limit story behind Front Door.
|
||||
* [Device Journeys & First-Time-Right](backend/device-journeys.md) - How a device's terminal enrollment attempts are grouped into journeys (trim+casefold device key with junk-serial exclusion, first-success end, 30-day gap, WhiteGlove = one attempt), the DeviceHistories chain with its tombstone-driven deleted-session pruning, the additive daily FTR aggregates, and the banner/fleet/MCP surfaces.
|
||||
* [Enrollment Time Attribution](backend/time-attribution.md) - How a terminal session's authoritative wall clock (DurationSeconds, WhiteGlove pause excluded) is partitioned into named segments with an explicit unattributed remainder, ESP-blocking apps as positive evidence with "up to" what-if bounds, reboot spans from event gaps, and the breakdown/aggregate tables the sweep maintains.
|
||||
* [Attempt-Scoped App Install Durations](backend/app-install-duration-attempt.md) - AppInstallSummary.DurationSeconds = status-defining (last) install attempt via the LastAttemptStartedAt max-fold anchor instead of the first-observation→last-terminal span that billed multi-pass IME gaps to trivial apps; InstallPassCount, the weaker-terminal guard, the radar's cutover gate, and time attribution's union-of-attempt-segments occupancy.
|
||||
* [Rule Regression Radar](backend/rule-regression-radar.md) - How analyze-rule hit-rate regressions are detected (7-day window ≥2× the 28-day baseline with disjoint Wilson intervals), which suppressions keep the radar quiet (grace period, edit-in-window, kill switch), how alert episodes live in the ruleregression tracker keyspace with re-arm semantics, and the on-fire dimension correlation with its "correlated, not necessarily causal" wording contract.
|
||||
* [App Version Duration Regression Radar](backend/app-version-regression-radar.md) - How app versions whose median install duration regressed vs. the previous version are detected (first-seen-ordered versions, median lift ≥2× plus a 5-minute absolute floor over measured installs only), how alert episodes live in the appversionregression tracker keyspace with drain/median re-arm semantics, and the bell/ops-event/analytics surfaces.
|
||||
* [Tenant Activation & the Once-Only Welcome Mail](backend/tenant-activation-welcome-mail.md) - How a signup becomes an activated tenant (manual approve or the auto-approve queue), and the write-then-read + conditional-marker arbitration that sends the welcome mail exactly once even though activation and the user-typed notification address race each other.
|
||||
|
||||
+14
@@ -2,6 +2,20 @@
|
||||
|
||||
## 2026-08-21
|
||||
|
||||
* **New**: `agent/do-group-id-auto-set.md` — optional DO peering-group seed (`EnableDoGroupIdAutoSet`, default off, ConfigVersion 40): `DoGroupIdService` derives a deterministic GUID from the default gateway's IPv4 + SendARP MAC in RealmJoin's exact byte layout (cross-product peering) and writes `DOGroupId` in the 64-bit GPO Policies store with readback verification. Conservative by design: no random-GUID fallback (skip on missing gateway/MAC), `DOGroupIdSource`/MDM-managed values/foreign GPO values always win (skip reasons in the `do_group_id_auto_set` event), own stale values are recognized via the `written:<guid>` StartupEventGate fingerprint and updated after a gateway change. Gateway heuristic extracted into the shared `NetworkInterfaceLocator` (dedup of NetworkChangeDetector + DeviceInfoCollector copies).
|
||||
|
||||
* **New**: `agent/system-timeline-watcher.md` — clock-step and sleep-episode ground truth from the System event log: one `EventLogWatcher` (Kernel-General 1, Power-Troubleshooter 1, Kernel-Power 507; 506 deliberately unconsumed) emits `system_clock_changed` (oldTime/newTime/signed timeDeltaMs/reason/process; 2 s micro-slew floor, ≥5 min = Warning+immediate) and `system_sleep_episode` (kind sleep/hibernate/modern_standby, authoritative `enteredAt`/`exitedAt` payload instants, 60 s Modern-Standby floor). WindowsUpdateTracker-style backfill (24 h default, `system-timeline-watermark.json`) recovers pre-agent records; ConfigVersion 39 (`Collectors.EnableSystemTimelineWatcher`). Consumers: TimeJumpBadge cause `clock-set` (ground truth beats inference), SessionInfoCard standby note next to the unchanged wall-clock duration, `SleepSpan`s in time attribution (AttributionVersion 3, RebootSpan clone), and ANALYZE-DEV-012 (info, single, ≥5 min episode; deliberately the only new rule — clock issues stay with DEV-008).
|
||||
|
||||
* **New**: `agent/power-state-watcher.md` — live AC/battery observation during enrollment: `PowerStateWatcherHost` subscribes to the WMI push `Win32_PowerManagementEvent EventCode 10` (no polling; re-probes `GetSystemPowerStatus` per push, 5 s trailing-edge debounce, never armed on battery-less devices, `Collectors.EnablePowerStateWatcher` kill-switch, ConfigVersion 38). The pure `PowerStateTransitionTracker` emits `power_state_change` with `transition` = ac_to_battery (Warning, immediate) / battery_to_ac (Info) / threshold_crossed at 50/30/15% (each level latched once per lifetime, lowest-crossed-only on jumps, baseline-below-level covered, emission cap 20). Classified non-activity on both sides (SignalActivityClassifier + IsPeriodicOrStallEvent). New analyze rules ANALYZE-DEV-009 (thresholdPercent=15, high) and ANALYZE-DEV-010 (transition=ac_to_battery, warning), both with `on_event:power_state_change` interim triggers and deliberately single-required-condition (event_data conditions have no same-instance join).
|
||||
|
||||
* **Update**: `rules/rule-id-namespace.md` — the export→paste round-trip is no longer lossy: the JSON editors now map rule-shaped exports into the form (`gatherRuleToForm` unflattens the nested gather `parameters` object, analyze `jsonToForm` maps `evaluateOn` onto the eval checkboxes; `tags` and the effective `markSessionAsFailed` carry into the created rule). Both editors still accept the serialized-form shape; edit-mode pastes merge, with an explicit `parameters`/`evaluateOn` winning over stale flat fields.
|
||||
|
||||
* **New**: `rules/rule-id-namespace.md` — rule identity model: human-readable `ruleId` strings (no GUIDs), tenant isolation via Table Storage partitioning (`global` = built-in/community, tenantId = custom), the reserved built-in namespace from `RuleIdPolicy`, and the four collision guards: catalog-only global `RuleStats` aggregate (same-ID custom rules of unrelated tenants no longer sum into one `global_{ruleId}` row), combine.js duplicate/namespace fail, GitHub-reseed rejection of non-reserved IDs into the global partition, and the namespace gate on the cross-tenant RuleState GC.
|
||||
|
||||
* **New**: `backend/app-install-duration-attempt.md` — attempt-scoped app install durations: `DurationSeconds` = status-defining (last) attempt anchored on the new max-folded `LastAttemptStartedAt` (StartedAt stays the earliest-wins window/version-ordering key; legacy rows without the anchor keep the span fallback); `InstallPassCount` with replay-guarded additive fold; weaker-terminal guard (completed-with-Skipped never overrides Installed/Error, in-batch and cross-batch); attempt-scoped `DownloadDurationSeconds` pairing; the version-regression radar's both-sides cutover gate; time attribution counts the union of per-pass active segments (occupancy merges segments, hulls stay for display/what-if); agent payload timing re-arms both endpoints on terminal→active. No backfill — pre-cutover rows keep span semantics.
|
||||
|
||||
* **Update**: `agent/decision-engine.md` — activity-based RealmJoin timeout extension (report 55e6afd61c9d: a large RJ package catalog outlived the 60-min window mid-install — the timer arms at RJ-agent-MSI detection, so the first deployment had effectively ~44 min). When the `RealmJoinTimeout` deadline fires with `LastDeploymentPhase` 100/101 and deployment activity (phase change or package observation, persisted in `RealmJoinFacts.LastActivityUtc`; the detection-time phase capture deliberately excluded) inside the last 60 min, the deadline re-arms to `lastActivity + 60 min` (Info event `realmjoin_timeout_extended`), clamped at detection + 4 h. `realmjoin_timeout` now carries `reason` = `hard_timeout` / `inactivity` / `absolute_cap`; a fire of a superseded deadline incarnation dead-ends. Idle detection and phase-200/210 sessions keep the original 60-min behavior.
|
||||
|
||||
* **New**: `backend/send-time-frame-separation.md` — P14: the agent stamps one device-clock send timestamp per upload attempt (`X-Send-Time-Utc`), the ingest function server-stamps it as `SentAt` on every event of the request. `SentAt − OccurredUtc` is pure spool delay, `ReceivedAt − SentAt` is network + device-vs-server clock offset — the two were indistinguishable before and had to be separated statistically. The `clock_skew` condition measures SentAt-carrying batches directly (spool-immune, no spread cap, IME-only sessions become measurable; evidence carries `sentAtBatchCount`), legacy batches keep the median path; the CMTrace tripwire deliberately stays on its differential math (common-mode clock offset cancels there).
|
||||
|
||||
## 2026-08-20
|
||||
|
||||
@@ -98,7 +98,7 @@ Interim evaluation changes what a rule's conditions mean mid-flight. Three new l
|
||||
|
||||
# Application to the Motivating Cases
|
||||
|
||||
- WG battery rule (sits-d): `evaluateOn: ["enrollment_end", "whiteglove_sealed"]` — technician gets the channel alert at the bench; Part-2 terminal pass finalizes or resolves the finding.
|
||||
- WG battery rule (sits-d): shipped 2026-08-21 as ANALYZE-DEV-009 (battery below 15% while on battery) and ANALYZE-DEV-010 (AC→battery switch), both `evaluateOn: ["enrollment_end", "whiteglove_sealed", "on_event:power_state_change"]` on the live power watcher's `power_state_change` events (see `../agent/power-state-watcher.md`) — technician gets the channel alert at the bench; the terminal pass finalizes the finding. Both are deliberately single-required-condition rules: `event_data` conditions scan events independently (no same-instance join), so DEV-009 matches the agent-stamped `thresholdPercent` and DEV-010 matches `transition`.
|
||||
- Hybrid-affinity stuck (esa): ANALYZE-ID-004 gets `evaluateOn: ["enrollment_end", "on_event:hybrid_login_pending"]` — the second overdue-login event (threshold 65) produces a live finding on the still-InProgress session. ANALYZE-APP-015 stays `enrollment_end`-only until its conditions are hardened (lint 1).
|
||||
|
||||
- Stuck after a forced mid-ESP reboot (sits-d Cloud PCs, 2026-08-19): `on_event:session_stalled` is the established trigger for the never-terminal stuck session. The agent emits `session_stalled` exactly once per session (60-minute stall probe), so the trigger is rare by construction and needs no throttle. Five Cloud PCs sat at `InProgress` with **zero findings** for hours — a mid-ESP reboot had killed the agent in `Stage=EspAccountSetup` — before the max-lifetime watchdog eventually terminalised them as `Incomplete`. ANALYZE-ESP-005 carries the trigger and fires on that stream at confidence 95 (verified against session `8110e262`): both its conditions (`mdm_policy_reboot_required.firstRebootUri` exists, `system_reboot_detected` count ≥ 1) are **monotonic** and it has no terminal preconditions, so lint 1 does not apply. ANALYZE-APP-016 is interim-safe for the same reason (`app_install_starved` count ≥ 1 only ever grows).
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
type: concept
|
||||
title: Rule ID Namespace & Collision Guards
|
||||
description: How gather/analyze rule identity is scoped (tenant partition vs global partition), why there are no GUIDs, which collisions are possible, and the guards that keep the reserved built-in namespace and the global RuleStats aggregate clean.
|
||||
resource: backend, rules
|
||||
tags: [rules, ruleid, namespace, collision, rulestats, tenant-isolation]
|
||||
timestamp: 2026-08-21
|
||||
---
|
||||
|
||||
# Schema
|
||||
|
||||
Rule identity is the human-readable `ruleId` string — there are no GUIDs, deliberately: the ID appears in logs, mails, notifications, RuleResults, the agent config payload, and docs, and tenant isolation is provided by Table Storage partitioning instead.
|
||||
|
||||
| Store | PartitionKey | RowKey | Content |
|
||||
|---|---|---|---|
|
||||
| `GatherRules` / `AnalyzeRules` | `"global"` | ruleId | Built-in + community rules (shared, all tenants) |
|
||||
| `GatherRules` / `AnalyzeRules` | tenantId | ruleId | Custom rules (visible only to that tenant) |
|
||||
| `RuleStates` | tenantId | ruleId | Per-tenant enable/notify overrides — built-in/community rules ONLY (custom rules fold state into their own row) |
|
||||
| `RuleStats` | date | `{tenantId}_{ruleId}` and `global_{ruleId}` | Daily fire/eval telemetry; the global row is catalog-only (see guards) |
|
||||
|
||||
Namespace split (`RuleIdPolicy.cs`, shared model):
|
||||
|
||||
- **Reserved built-in namespace**: `(ANALYZE|GATHER)-<CATEGORY>-<NUMBER>` with `CATEGORY != CUSTOM`, case-insensitive, including currently unused numbers (gaps = retired rules that may return).
|
||||
- **Tenant namespace**: everything else — `ANALYZE-CUSTOM-NNN` (portal suggestion), `-CUSTOM` suffixes, org prefixes (`CONTOSO-WIFI-001`), freeform.
|
||||
|
||||
# Collision model
|
||||
|
||||
- **Custom vs built-in**: impossible via API — `CreateRuleAsync`/`UpdateRuleAsync` reject reserved IDs (409) and check existence against `global` ∪ own tenant partition. Legacy debris is neutralized at merge time: global wins, tenant copy dropped with a warning (pinned by `RuleMergeCollisionGuardTests`, motivated by the 5ca2b350 duplicate-key incident).
|
||||
- **Custom vs custom across tenants**: allowed and harmless — separate partitions, separate evaluation, separate RuleResults/tenant stats. Two tenants both using `ANALYZE-CUSTOM-101` never interact.
|
||||
- **JSON copy between tenants** (export → paste into create editor): goes through the same create path, so an ID already taken in the receiving tenant surfaces as 409 — the user renames. There is no bulk import endpoint. The JSON editors accept both shapes — a rule export and the serialized form — via `gatherRuleToForm` / `jsonToForm` (web `types.ts` of each rules page): nested gather `parameters` are unflattened, analyze `evaluateOn` maps onto the eval checkboxes, and `tags` plus the effective `markSessionAsFailed` survive the import.
|
||||
|
||||
# Guards (all enforced, 2026-08-21)
|
||||
|
||||
1. **Global RuleStats aggregate is catalog-only.** The `global_{ruleId}` row would otherwise sum same-ID custom rules of unrelated tenants (title/severity last-writer-wins). Analyze gates on `rule.IsBuiltIn || rule.IsCommunity` (`AnalyzeOnEnrollmentEndHandler`), gather gates on `RuleIdPolicy.IsReservedBuiltInId` (`EventIngestProcessor.RuleStats`) because agent events carry no flag — valid because custom rules can never occupy the reserved pattern.
|
||||
2. **Catalog build fails on namespace violations.** `rules/scripts/combine.js` rejects duplicate IDs (case-insensitive) and any gather/analyze built-in outside the reserved pattern; `BuiltInRuleCatalogPolicyTests` pins the same invariants on the embedded resource actually deployed.
|
||||
3. **GitHub reseed rejects non-reserved IDs** before writing to the `global` partition (`ReseedFromGitHubFunction`) — a seeded `*-CUSTOM-*` rule would silently shadow every tenant's same-ID custom rule at merge time.
|
||||
4. **Cross-tenant RuleState GC is namespace-gated.** `DeleteRuleStatesForRuleIdAcrossTenantsAsync` deletes by bare RowKey across all partitions; it now refuses non-reserved IDs with `failed=-1` (callers then skip the global catalog delete, same contract as enumeration failure).
|
||||
|
||||
Invariant chain: schema pattern + combine.js + reseed guard ⇒ the `global` partition only ever contains reserved-pattern IDs ⇒ the pattern is a safe built-in/custom discriminator everywhere no `IsBuiltIn` flag is available.
|
||||
|
||||
# Citations
|
||||
|
||||
- `src/Shared/AutopilotMonitor.Shared/Models/Rules/RuleIdPolicy.cs` — reserved pattern + rationale
|
||||
- `src/Backend/AutopilotMonitor.Functions/Services/AnalyzeRuleService.cs`, `GatherRuleService.cs` — create/update guards, merge collision guard
|
||||
- `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Rules.cs` — key schemes, cross-tenant GC guard
|
||||
- `src/Backend/AutopilotMonitor.Functions.Tests/BuiltInRuleCatalogPolicyTests.cs`, `RuleIdReservationTests.cs`, `RuleMergeCollisionGuardTests.cs`
|
||||
- `rules/scripts/combine.js` — build-time namespace/duplicate guard
|
||||
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"$schema": "../schema/analyze-rule.schema.json",
|
||||
"ruleId": "ANALYZE-DEV-009",
|
||||
"title": "Battery Critically Low During Enrollment",
|
||||
"description": "Detects that the device's battery dropped to 15% or less while enrolling on battery power (power_state_change threshold event from the agent's live power watcher). A device dying mid-enrollment leaves a half-provisioned state that typically needs a full reset; this rule surfaces the risk while there is still time to plug the device in. Uses a single required condition on the agent-stamped thresholdPercent field (the data field 'transition' on the event is unrelated to this rule's 'trigger' property).",
|
||||
"severity": "high",
|
||||
"category": "device",
|
||||
"version": "1.0.0",
|
||||
"enabled": true,
|
||||
"evaluateOn": [
|
||||
"enrollment_end",
|
||||
"whiteglove_sealed",
|
||||
"on_event:power_state_change"
|
||||
],
|
||||
"trigger": "single",
|
||||
"baseConfidence": 90,
|
||||
"confidenceThreshold": 60,
|
||||
"conditions": [
|
||||
{
|
||||
"signal": "battery_threshold_15",
|
||||
"source": "event_data",
|
||||
"eventType": "power_state_change",
|
||||
"dataField": "thresholdPercent",
|
||||
"operator": "equals",
|
||||
"value": "15",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"explanation": "The device's battery fell below **{{thresholdPercent}}%** while the enrollment was running on battery power. If the battery drains completely mid-enrollment, the device is left in a half-provisioned state — apps, policies, or the user profile may be incomplete, and in most cases the device has to be reset and re-enrolled. The agent detected the threshold crossing live via a Windows power-status notification; the exact percentage at that moment is recorded on the matched power_state_change event in the session timeline.",
|
||||
"remediation": [
|
||||
{
|
||||
"title": "Secure power for the running enrollment",
|
||||
"steps": [
|
||||
"Connect the device to AC power immediately — the enrollment continues normally once power is stable.",
|
||||
"If the device shut down before completing, check whether the session reached the desktop; if not, reset the device (Autopilot reset or reimage) and re-enroll it plugged in.",
|
||||
"For technician/White Glove flows, make AC power part of the bench checklist — pre-provisioning on battery is the most common source of this alert.",
|
||||
"If this fires across many devices of one model, review that model's battery health (the hardware_spec event carries design vs. full-charge capacity)."
|
||||
]
|
||||
}
|
||||
],
|
||||
"markSessionAsFailedDefault": false,
|
||||
"tags": [
|
||||
"device",
|
||||
"power",
|
||||
"battery",
|
||||
"environment"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"$schema": "../schema/analyze-rule.schema.json",
|
||||
"ruleId": "ANALYZE-DEV-010",
|
||||
"title": "Enrollment Switched From AC to Battery Power",
|
||||
"description": "Detects that the device lost AC power and switched to battery while the enrollment was running (power_state_change event with transition ac_to_battery from the agent's live power watcher). Uses a single required condition so the match is pinned to one event instance; the event payload carries the battery percentage at the moment of the switch. The agent debounces power flapping (5s) so a brief unplug/replug does not produce this event.",
|
||||
"severity": "warning",
|
||||
"category": "device",
|
||||
"version": "1.0.0",
|
||||
"enabled": true,
|
||||
"evaluateOn": [
|
||||
"enrollment_end",
|
||||
"whiteglove_sealed",
|
||||
"on_event:power_state_change"
|
||||
],
|
||||
"trigger": "single",
|
||||
"baseConfidence": 85,
|
||||
"confidenceThreshold": 60,
|
||||
"conditions": [
|
||||
{
|
||||
"signal": "switched_to_battery",
|
||||
"source": "event_data",
|
||||
"eventType": "power_state_change",
|
||||
"dataField": "transition",
|
||||
"operator": "equals",
|
||||
"value": "ac_to_battery",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"explanation": "The device switched from AC to battery power mid-enrollment. Enrollments are power-hungry — app downloads, Windows Update and disk encryption drain a battery quickly, and Windows may also throttle CPU and network on battery, slowing the enrollment down. The battery percentage at the moment of the switch is recorded on the matched power_state_change event in the session timeline; if the device stayed on battery, look there for subsequent threshold events (50/30/15%) — a later battery_to_ac event means power was restored.",
|
||||
"remediation": [
|
||||
{
|
||||
"title": "Restore AC power",
|
||||
"steps": [
|
||||
"Reconnect the device to AC power; the enrollment continues normally.",
|
||||
"Check the session timeline for a subsequent battery_to_ac event (power restored) or threshold_crossed events (battery draining) to judge urgency.",
|
||||
"If this alert is frequent in a deployment area, check docking stations/USB-C hubs for loose power delivery — a flapping dock is a common cause.",
|
||||
"Make 'plugged in during setup' part of the end-user/technician enrollment instructions."
|
||||
]
|
||||
}
|
||||
],
|
||||
"markSessionAsFailedDefault": false,
|
||||
"tags": [
|
||||
"device",
|
||||
"power",
|
||||
"battery",
|
||||
"environment"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
{
|
||||
"$schema": "../schema/analyze-rule.schema.json",
|
||||
"ruleId": "ANALYZE-DEV-012",
|
||||
"title": "Device Entered Standby During Enrollment",
|
||||
"description": "Detects that the device completed a sleep, hibernate or Modern Standby episode of at least 5 minutes while the enrollment was running (system_sleep_episode event from the agent's system timeline watcher, recorded retroactively at wake from the Windows event log — pre-agent episodes are backfilled). Uses a single required condition on the episode's durationSeconds so the match is pinned to one event instance and the interpolated duration is the matched value. Short Modern Standby screen-off scenarios (under 60 s of real sleep) never emit the event, and this rule adds a 5-minute floor so brief lid-close moments do not fire it.",
|
||||
"severity": "info",
|
||||
"category": "device",
|
||||
"version": "1.0.0",
|
||||
"enabled": true,
|
||||
"evaluateOn": [
|
||||
"enrollment_end",
|
||||
"whiteglove_sealed",
|
||||
"on_event:system_sleep_episode"
|
||||
],
|
||||
"trigger": "single",
|
||||
"baseConfidence": 90,
|
||||
"confidenceThreshold": 60,
|
||||
"conditions": [
|
||||
{
|
||||
"signal": "slept_during_enrollment",
|
||||
"source": "event_data",
|
||||
"eventType": "system_sleep_episode",
|
||||
"dataField": "durationSeconds",
|
||||
"operator": "gte",
|
||||
"value": "300",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"explanation": "The device was asleep for {{durationSeconds}} seconds during the enrollment window. Nothing progresses while the device sleeps — downloads pause, the IME stops processing and the ESP simply waits — so the wall-clock enrollment duration includes this pause and can look much slower than the enrollment actually was. The matched system_sleep_episode event in the session timeline carries the exact enter/exit times, the episode kind (sleep, hibernate or Modern Standby) and, for Modern Standby, whether the device was on AC power. A long unexplained gap in the timeline right before that event is this pause, not missing telemetry.",
|
||||
"remediation": [
|
||||
{
|
||||
"title": "Keep the device awake during enrollment",
|
||||
"steps": [
|
||||
"Check the matched system_sleep_episode event in the timeline for the episode bounds and kind — the enrollment duration minus the episode duration is the real active time.",
|
||||
"If the device slept on battery, connect AC power during setup; Windows sleeps far more aggressively on battery.",
|
||||
"Instruct users/technicians to keep the lid open and the device powered until the enrollment finishes.",
|
||||
"For recurring cases, review the power plan applied during OOBE/ESP (e.g. via provisioning or Intune policy) — consider disabling sleep on AC for the enrollment window."
|
||||
]
|
||||
}
|
||||
],
|
||||
"markSessionAsFailedDefault": false,
|
||||
"tags": [
|
||||
"device",
|
||||
"power",
|
||||
"standby",
|
||||
"sleep",
|
||||
"environment"
|
||||
]
|
||||
}
|
||||
+142
@@ -1584,6 +1584,148 @@
|
||||
"known-issue"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ruleId": "ANALYZE-DEV-009",
|
||||
"title": "Battery Critically Low During Enrollment",
|
||||
"description": "Detects that the device's battery dropped to 15% or less while enrolling on battery power (power_state_change threshold event from the agent's live power watcher). A device dying mid-enrollment leaves a half-provisioned state that typically needs a full reset; this rule surfaces the risk while there is still time to plug the device in. Uses a single required condition on the agent-stamped thresholdPercent field (the data field 'transition' on the event is unrelated to this rule's 'trigger' property).",
|
||||
"severity": "high",
|
||||
"category": "device",
|
||||
"version": "1.0.0",
|
||||
"enabled": true,
|
||||
"evaluateOn": [
|
||||
"enrollment_end",
|
||||
"whiteglove_sealed",
|
||||
"on_event:power_state_change"
|
||||
],
|
||||
"trigger": "single",
|
||||
"baseConfidence": 90,
|
||||
"confidenceThreshold": 60,
|
||||
"conditions": [
|
||||
{
|
||||
"signal": "battery_threshold_15",
|
||||
"source": "event_data",
|
||||
"eventType": "power_state_change",
|
||||
"dataField": "thresholdPercent",
|
||||
"operator": "equals",
|
||||
"value": "15",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"explanation": "The device's battery fell below **{{thresholdPercent}}%** while the enrollment was running on battery power. If the battery drains completely mid-enrollment, the device is left in a half-provisioned state — apps, policies, or the user profile may be incomplete, and in most cases the device has to be reset and re-enrolled. The agent detected the threshold crossing live via a Windows power-status notification; the exact percentage at that moment is recorded on the matched power_state_change event in the session timeline.",
|
||||
"remediation": [
|
||||
{
|
||||
"title": "Secure power for the running enrollment",
|
||||
"steps": [
|
||||
"Connect the device to AC power immediately — the enrollment continues normally once power is stable.",
|
||||
"If the device shut down before completing, check whether the session reached the desktop; if not, reset the device (Autopilot reset or reimage) and re-enroll it plugged in.",
|
||||
"For technician/White Glove flows, make AC power part of the bench checklist — pre-provisioning on battery is the most common source of this alert.",
|
||||
"If this fires across many devices of one model, review that model's battery health (the hardware_spec event carries design vs. full-charge capacity)."
|
||||
]
|
||||
}
|
||||
],
|
||||
"markSessionAsFailedDefault": false,
|
||||
"tags": [
|
||||
"device",
|
||||
"power",
|
||||
"battery",
|
||||
"environment"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ruleId": "ANALYZE-DEV-010",
|
||||
"title": "Enrollment Switched From AC to Battery Power",
|
||||
"description": "Detects that the device lost AC power and switched to battery while the enrollment was running (power_state_change event with transition ac_to_battery from the agent's live power watcher). Uses a single required condition so the match is pinned to one event instance; the event payload carries the battery percentage at the moment of the switch. The agent debounces power flapping (5s) so a brief unplug/replug does not produce this event.",
|
||||
"severity": "warning",
|
||||
"category": "device",
|
||||
"version": "1.0.0",
|
||||
"enabled": true,
|
||||
"evaluateOn": [
|
||||
"enrollment_end",
|
||||
"whiteglove_sealed",
|
||||
"on_event:power_state_change"
|
||||
],
|
||||
"trigger": "single",
|
||||
"baseConfidence": 85,
|
||||
"confidenceThreshold": 60,
|
||||
"conditions": [
|
||||
{
|
||||
"signal": "switched_to_battery",
|
||||
"source": "event_data",
|
||||
"eventType": "power_state_change",
|
||||
"dataField": "transition",
|
||||
"operator": "equals",
|
||||
"value": "ac_to_battery",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"explanation": "The device switched from AC to battery power mid-enrollment. Enrollments are power-hungry — app downloads, Windows Update and disk encryption drain a battery quickly, and Windows may also throttle CPU and network on battery, slowing the enrollment down. The battery percentage at the moment of the switch is recorded on the matched power_state_change event in the session timeline; if the device stayed on battery, look there for subsequent threshold events (50/30/15%) — a later battery_to_ac event means power was restored.",
|
||||
"remediation": [
|
||||
{
|
||||
"title": "Restore AC power",
|
||||
"steps": [
|
||||
"Reconnect the device to AC power; the enrollment continues normally.",
|
||||
"Check the session timeline for a subsequent battery_to_ac event (power restored) or threshold_crossed events (battery draining) to judge urgency.",
|
||||
"If this alert is frequent in a deployment area, check docking stations/USB-C hubs for loose power delivery — a flapping dock is a common cause.",
|
||||
"Make 'plugged in during setup' part of the end-user/technician enrollment instructions."
|
||||
]
|
||||
}
|
||||
],
|
||||
"markSessionAsFailedDefault": false,
|
||||
"tags": [
|
||||
"device",
|
||||
"power",
|
||||
"battery",
|
||||
"environment"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ruleId": "ANALYZE-DEV-012",
|
||||
"title": "Device Entered Standby During Enrollment",
|
||||
"description": "Detects that the device completed a sleep, hibernate or Modern Standby episode of at least 5 minutes while the enrollment was running (system_sleep_episode event from the agent's system timeline watcher, recorded retroactively at wake from the Windows event log — pre-agent episodes are backfilled). Uses a single required condition on the episode's durationSeconds so the match is pinned to one event instance and the interpolated duration is the matched value. Short Modern Standby screen-off scenarios (under 60 s of real sleep) never emit the event, and this rule adds a 5-minute floor so brief lid-close moments do not fire it.",
|
||||
"severity": "info",
|
||||
"category": "device",
|
||||
"version": "1.0.0",
|
||||
"enabled": true,
|
||||
"evaluateOn": [
|
||||
"enrollment_end",
|
||||
"whiteglove_sealed",
|
||||
"on_event:system_sleep_episode"
|
||||
],
|
||||
"trigger": "single",
|
||||
"baseConfidence": 90,
|
||||
"confidenceThreshold": 60,
|
||||
"conditions": [
|
||||
{
|
||||
"signal": "slept_during_enrollment",
|
||||
"source": "event_data",
|
||||
"eventType": "system_sleep_episode",
|
||||
"dataField": "durationSeconds",
|
||||
"operator": "gte",
|
||||
"value": "300",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"explanation": "The device was asleep for {{durationSeconds}} seconds during the enrollment window. Nothing progresses while the device sleeps — downloads pause, the IME stops processing and the ESP simply waits — so the wall-clock enrollment duration includes this pause and can look much slower than the enrollment actually was. The matched system_sleep_episode event in the session timeline carries the exact enter/exit times, the episode kind (sleep, hibernate or Modern Standby) and, for Modern Standby, whether the device was on AC power. A long unexplained gap in the timeline right before that event is this pause, not missing telemetry.",
|
||||
"remediation": [
|
||||
{
|
||||
"title": "Keep the device awake during enrollment",
|
||||
"steps": [
|
||||
"Check the matched system_sleep_episode event in the timeline for the episode bounds and kind — the enrollment duration minus the episode duration is the real active time.",
|
||||
"If the device slept on battery, connect AC power during setup; Windows sleeps far more aggressively on battery.",
|
||||
"Instruct users/technicians to keep the lid open and the device powered until the enrollment finishes.",
|
||||
"For recurring cases, review the power plan applied during OOBE/ESP (e.g. via provisioning or Intune policy) — consider disabling sleep on AC for the enrollment window."
|
||||
]
|
||||
}
|
||||
],
|
||||
"markSessionAsFailedDefault": false,
|
||||
"tags": [
|
||||
"device",
|
||||
"power",
|
||||
"standby",
|
||||
"sleep",
|
||||
"environment"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ruleId": "ANALYZE-ENRL-001",
|
||||
"title": "Enrollment Failed",
|
||||
|
||||
@@ -13,18 +13,26 @@ const rulesRoot = path.resolve(__dirname, '..');
|
||||
|
||||
// ── Rule combination ────────────────────────────────────────────────────────
|
||||
|
||||
// Shipped gather/analyze rules must stay inside the reserved built-in namespace:
|
||||
// (ANALYZE|GATHER)-<CATEGORY>-<NNN> with CATEGORY !== CUSTOM. The CUSTOM category is
|
||||
// the sanctioned tenant namespace (RuleIdPolicy.cs) — a built-in shipped there would
|
||||
// silently shadow every tenant's same-ID custom rule at merge time (global wins).
|
||||
const RESERVED_BUILTIN_ID = /^(ANALYZE|GATHER)-(?!CUSTOM-)[A-Z]+-\d{3}$/;
|
||||
|
||||
const configs = [
|
||||
{
|
||||
dir: path.join(rulesRoot, 'gather'),
|
||||
output: path.join(rulesRoot, 'dist', 'gather-rules.json'),
|
||||
schema: '../schema/gather-rule.schema.json',
|
||||
idField: 'ruleId'
|
||||
idField: 'ruleId',
|
||||
reservedNamespace: RESERVED_BUILTIN_ID
|
||||
},
|
||||
{
|
||||
dir: path.join(rulesRoot, 'analyze'),
|
||||
output: path.join(rulesRoot, 'dist', 'analyze-rules.json'),
|
||||
schema: '../schema/analyze-rule.schema.json',
|
||||
idField: 'ruleId'
|
||||
idField: 'ruleId',
|
||||
reservedNamespace: RESERVED_BUILTIN_ID
|
||||
},
|
||||
{
|
||||
dir: path.join(rulesRoot, 'ime-log-patterns'),
|
||||
@@ -37,12 +45,27 @@ const configs = [
|
||||
for (const config of configs) {
|
||||
const files = fs.readdirSync(config.dir).filter(f => f.endsWith('.json')).sort();
|
||||
const rules = [];
|
||||
const seenIds = new Map(); // lowercased id -> filename (case variants collide for humans and RuleIdPolicy)
|
||||
|
||||
for (const file of files) {
|
||||
const content = fs.readFileSync(path.join(config.dir, file), 'utf8');
|
||||
const rule = JSON.parse(content);
|
||||
// Remove $schema from individual entries (it's on the wrapper)
|
||||
delete rule['$schema'];
|
||||
|
||||
const id = rule[config.idField] || '';
|
||||
const idKey = id.toLowerCase();
|
||||
if (seenIds.has(idKey)) {
|
||||
console.error(`ERROR: duplicate ${config.idField} "${id}" in ${file} (already defined in ${seenIds.get(idKey)})`);
|
||||
process.exit(1);
|
||||
}
|
||||
seenIds.set(idKey, file);
|
||||
|
||||
if (config.reservedNamespace && !config.reservedNamespace.test(id)) {
|
||||
console.error(`ERROR: ${file}: ${config.idField} "${id}" is outside the reserved built-in namespace ${config.reservedNamespace} — built-ins must never use the tenant CUSTOM namespace`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
rules.push(rule);
|
||||
}
|
||||
|
||||
|
||||
@@ -28,10 +28,13 @@
|
||||
(validates Windows 365 Cloud PCs against the tenant's Cloud PC inventory --
|
||||
Cloud PCs are never Autopilot-registered, so this enables agent monitoring
|
||||
of Cloud PC first-connect enrollment)
|
||||
- All -> every optional permission above, in one run
|
||||
(also works with -Revoke to remove all optional grants at once)
|
||||
|
||||
.PARAMETER Permissions
|
||||
Microsoft Graph application permission names to grant directly. Use when the admin UI
|
||||
rendered a copy-paste command for you, or when you have a specific list in mind.
|
||||
Microsoft Graph application permission names to grant directly. Use when you copied the
|
||||
permission list from the admin UI ("Copy permissions"), or when you have a specific list
|
||||
in mind.
|
||||
|
||||
.PARAMETER TenantId
|
||||
Tenant ID (GUID) or verified domain. Optional -- defaults to the signed-in context.
|
||||
@@ -47,12 +50,16 @@
|
||||
.\Grant-AutopilotMonitorAddOn.ps1 -ClientId "<your-client-id>" -Features ScriptDisplayNames
|
||||
|
||||
.EXAMPLE
|
||||
# Same effect, using the explicit-permission form (what the admin UI generates):
|
||||
# Same effect, using the explicit-permission form:
|
||||
.\Grant-AutopilotMonitorAddOn.ps1 `
|
||||
-ClientId "<your-client-id>" `
|
||||
-Permissions "DeviceManagementScripts.Read.All" `
|
||||
-TenantId "contoso.onmicrosoft.com"
|
||||
|
||||
.EXAMPLE
|
||||
# Grant every optional capability in one run:
|
||||
.\Grant-AutopilotMonitorAddOn.ps1 -ClientId "<your-client-id>" -Features All
|
||||
|
||||
.EXAMPLE
|
||||
# Verify current state only, no changes:
|
||||
.\Grant-AutopilotMonitorAddOn.ps1 -ClientId "<your-client-id>" -Features ScriptDisplayNames -VerifyOnly
|
||||
@@ -65,6 +72,12 @@
|
||||
Requires:
|
||||
- Microsoft.Graph.Authentication PowerShell module (auto-installs if missing).
|
||||
- Sign-in as Global Administrator OR Privileged Role Administrator OR Cloud Application Administrator.
|
||||
|
||||
Sign-in behaviour:
|
||||
In Azure Cloud Shell the script signs in via the ambient identity endpoint (a delegated
|
||||
token for the already-signed-in user) - no device-code prompt, which Conditional Access
|
||||
policies commonly block. Outside Cloud Shell it falls back to a normal interactive
|
||||
Microsoft Graph sign-in. -TenantId only applies to the interactive fallback.
|
||||
- The Autopilot Monitor app must be admin-consented in your tenant first
|
||||
(its service principal must exist before grants can be added).
|
||||
|
||||
@@ -80,7 +93,7 @@ param(
|
||||
[string]$ClientId,
|
||||
|
||||
[Parameter(Mandatory = $true, ParameterSetName = 'ByFeatures')]
|
||||
[ValidateSet('ScriptDisplayNames', 'W365CloudPcValidation')]
|
||||
[ValidateSet('All', 'ScriptDisplayNames', 'W365CloudPcValidation')]
|
||||
[string[]]$Features,
|
||||
|
||||
[Parameter(Mandatory = $true, ParameterSetName = 'ByPermissions')]
|
||||
@@ -106,7 +119,11 @@ $FeatureCatalog = @{
|
||||
}
|
||||
|
||||
# Translate Features -> Permissions when caller used the high-level form.
|
||||
# 'All' is a script-side meta-value: it expands to every catalog feature.
|
||||
if ($PSCmdlet.ParameterSetName -eq 'ByFeatures') {
|
||||
if ($Features -contains 'All') {
|
||||
$Features = @($FeatureCatalog.Keys)
|
||||
}
|
||||
$Permissions = @()
|
||||
foreach ($f in $Features) {
|
||||
$Permissions += $FeatureCatalog[$f]
|
||||
@@ -133,7 +150,25 @@ if ($TenantId) { $connectParams['TenantId'] = $TenantId }
|
||||
|
||||
Write-Host ""
|
||||
Write-Host "[Auth] Connecting to Microsoft Graph..." -ForegroundColor Cyan
|
||||
Connect-MgGraph @connectParams | Out-Null
|
||||
|
||||
# Preferred path: ambient identity. In Azure Cloud Shell the local identity endpoint issues a
|
||||
# delegated Graph token for the signed-in user, which avoids the device-code flow that
|
||||
# Conditional Access policies commonly block. Outside Azure this throws and we fall back to
|
||||
# an interactive sign-in with explicit scopes.
|
||||
$identityConnected = $false
|
||||
try {
|
||||
Connect-MgGraph -Identity -NoWelcome -ErrorAction Stop | Out-Null
|
||||
# An ambient identity can exist yet hold no Graph permissions (e.g. a plain VM managed
|
||||
# identity) - probe with a read before committing to this session.
|
||||
Invoke-MgGraphRequest -Method GET -Uri "https://graph.microsoft.com/v1.0/servicePrincipals?`$select=id&`$top=1" -ErrorAction Stop | Out-Null
|
||||
$identityConnected = $true
|
||||
Write-Host "[Auth] Connected via ambient identity (Azure Cloud Shell / managed identity)." -ForegroundColor Green
|
||||
}
|
||||
catch {
|
||||
if (Get-MgContext) { Disconnect-MgGraph -ErrorAction SilentlyContinue | Out-Null }
|
||||
Write-Host "[Auth] No usable ambient identity - falling back to interactive sign-in..." -ForegroundColor Yellow
|
||||
Connect-MgGraph @connectParams | Out-Null
|
||||
}
|
||||
|
||||
$ctx = Get-MgContext
|
||||
if (-not $ctx) {
|
||||
@@ -144,7 +179,10 @@ Write-Host "[Auth] Signed in as : $($ctx.Account)"
|
||||
Write-Host "[Auth] Tenant : $($ctx.TenantId)"
|
||||
Write-Host "[Auth] Granted scopes: $($ctx.Scopes -join ', ')"
|
||||
|
||||
if (-not ($ctx.Scopes -contains 'AppRoleAssignment.ReadWrite.All')) {
|
||||
# Scope check only applies to the interactive path: an ambient-identity (Cloud Shell) token
|
||||
# carries a fixed scope set (incl. Directory.AccessAsUser.All) instead of the requested scopes,
|
||||
# and its effective rights follow the signed-in user's directory role.
|
||||
if (-not $identityConnected -and -not ($ctx.Scopes -contains 'AppRoleAssignment.ReadWrite.All')) {
|
||||
Write-Warning "Scope 'AppRoleAssignment.ReadWrite.All' was NOT granted. The signed-in user likely lacks the required role (Global Admin / Privileged Role Admin / Cloud App Admin)."
|
||||
if (-not $VerifyOnly) { return }
|
||||
}
|
||||
|
||||
+1
@@ -37,6 +37,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
|
||||
nameof(AgentConfigResponse.EnrollmentSummaryLaunchRetrySeconds),
|
||||
nameof(AgentConfigResponse.NtpServer),
|
||||
nameof(AgentConfigResponse.EnableTimezoneAutoSet),
|
||||
nameof(AgentConfigResponse.EnableDoGroupIdAutoSet),
|
||||
nameof(AgentConfigResponse.DiagnosticsUploadEnabled),
|
||||
nameof(AgentConfigResponse.DiagnosticsUploadMode),
|
||||
nameof(AgentConfigResponse.DiagnosticsLogPaths),
|
||||
|
||||
+2
@@ -43,6 +43,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
|
||||
EnrollmentSummaryLaunchRetrySeconds = 300,
|
||||
NtpServer = "pool.ntp.org",
|
||||
EnableTimezoneAutoSet = true,
|
||||
EnableDoGroupIdAutoSet = true,
|
||||
DiagnosticsUploadEnabled = true,
|
||||
DiagnosticsUploadMode = "OnFailure",
|
||||
DiagnosticsLogPaths = new List<DiagnosticsLogPath>
|
||||
@@ -83,6 +84,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
|
||||
Assert.Equal(300, agent.EnrollmentSummaryLaunchRetrySeconds);
|
||||
Assert.Equal("pool.ntp.org", agent.NtpServer);
|
||||
Assert.True(agent.EnableTimezoneAutoSet);
|
||||
Assert.True(agent.EnableDoGroupIdAutoSet);
|
||||
Assert.True(agent.DiagnosticsUploadEnabled);
|
||||
Assert.Equal("OnFailure", agent.DiagnosticsUploadMode);
|
||||
Assert.Single(agent.DiagnosticsLogPaths);
|
||||
|
||||
+258
@@ -0,0 +1,258 @@
|
||||
#nullable enable
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
|
||||
using AutopilotMonitor.Agent.V2.Core.Runtime;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
{
|
||||
/// <summary>
|
||||
/// Pure state machine behind the power watcher: AC↔battery edges, the 50/30/15 threshold
|
||||
/// ladder (latch once per lifetime, lowest-crossed-only on multi-level jumps, baseline
|
||||
/// coverage for sessions starting low), null-percent tolerance and the storm cap.
|
||||
/// </summary>
|
||||
public sealed class PowerStateTransitionTrackerTests
|
||||
{
|
||||
private static PowerStateResult Ac(int? percent = 80, bool charging = true) => new PowerStateResult
|
||||
{
|
||||
OnAcPower = true,
|
||||
HasBattery = true,
|
||||
BatteryPercent = percent,
|
||||
IsCharging = charging,
|
||||
BatteryLifeMinutes = null,
|
||||
};
|
||||
|
||||
private static PowerStateResult Battery(int? percent, int? lifeMinutes = 90) => new PowerStateResult
|
||||
{
|
||||
OnAcPower = false,
|
||||
HasBattery = true,
|
||||
BatteryPercent = percent,
|
||||
IsCharging = false,
|
||||
BatteryLifeMinutes = lifeMinutes,
|
||||
};
|
||||
|
||||
[Fact]
|
||||
public void Baseline_on_ac_is_silent()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
Assert.Empty(tracker.Baseline(Ac()));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Baseline_without_battery_or_with_probe_error_is_silent()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
Assert.Empty(tracker.Baseline(new PowerStateResult { OnAcPower = true, HasBattery = false }));
|
||||
Assert.Empty(tracker.Baseline(new PowerStateResult { ProbeError = "boom" }));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Baseline_on_battery_above_ladder_is_silent()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
Assert.Empty(tracker.Baseline(Battery(80)));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Baseline_at_12_percent_emits_exactly_the_15_threshold()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
|
||||
var emissions = tracker.Baseline(Battery(12));
|
||||
|
||||
var e = Assert.Single(emissions);
|
||||
Assert.Equal(PowerStateTransitionTracker.TransitionThresholdCrossed, e.Transition);
|
||||
Assert.Equal(15, e.ThresholdPercent);
|
||||
Assert.Equal(12, e.BatteryPercent);
|
||||
Assert.Equal(EventSeverity.Error, e.Severity);
|
||||
Assert.True(e.ImmediateUpload);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Baseline_at_25_percent_emits_30_and_latches_50()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
|
||||
var baseline = tracker.Baseline(Battery(25));
|
||||
var e = Assert.Single(baseline);
|
||||
Assert.Equal(30, e.ThresholdPercent);
|
||||
Assert.Equal(EventSeverity.Warning, e.Severity);
|
||||
Assert.True(e.ImmediateUpload);
|
||||
|
||||
// 50 and 30 are latched by the baseline; draining to 40% stays silent, 14% emits 15.
|
||||
Assert.Empty(tracker.Evaluate(Battery(40)));
|
||||
var next = Assert.Single(tracker.Evaluate(Battery(14)));
|
||||
Assert.Equal(15, next.ThresholdPercent);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Ac_to_battery_emits_warning_with_immediate_upload()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Ac(63));
|
||||
|
||||
var e = Assert.Single(tracker.Evaluate(Battery(63)));
|
||||
|
||||
Assert.Equal(PowerStateTransitionTracker.TransitionAcToBattery, e.Transition);
|
||||
Assert.Null(e.ThresholdPercent);
|
||||
Assert.Equal(EventSeverity.Warning, e.Severity);
|
||||
Assert.True(e.ImmediateUpload);
|
||||
Assert.False(e.OnAcPower);
|
||||
Assert.Equal(63, e.BatteryPercent);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Battery_to_ac_emits_info_without_immediate_upload()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Battery(70));
|
||||
|
||||
var e = Assert.Single(tracker.Evaluate(Ac(70)));
|
||||
|
||||
Assert.Equal(PowerStateTransitionTracker.TransitionBatteryToAc, e.Transition);
|
||||
Assert.Equal(EventSeverity.Info, e.Severity);
|
||||
Assert.False(e.ImmediateUpload);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void No_diff_stays_silent()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Ac());
|
||||
|
||||
Assert.Empty(tracker.Evaluate(Ac()));
|
||||
Assert.Empty(tracker.Evaluate(Ac(79)));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Unplug_below_threshold_emits_transition_and_threshold_together()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Ac(10));
|
||||
|
||||
var emissions = tracker.Evaluate(Battery(10));
|
||||
|
||||
Assert.Equal(2, emissions.Count);
|
||||
Assert.Equal(PowerStateTransitionTracker.TransitionAcToBattery, emissions[0].Transition);
|
||||
Assert.Equal(PowerStateTransitionTracker.TransitionThresholdCrossed, emissions[1].Transition);
|
||||
Assert.Equal(15, emissions[1].ThresholdPercent);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Multi_level_drop_emits_lowest_crossed_level_only()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Battery(60));
|
||||
|
||||
var e = Assert.Single(tracker.Evaluate(Battery(10)));
|
||||
|
||||
Assert.Equal(15, e.ThresholdPercent);
|
||||
// All levels are latched — recovering and re-draining emits nothing more.
|
||||
tracker.Evaluate(Ac(90));
|
||||
Assert.Single(tracker.Evaluate(Battery(45)), x => x.Transition == PowerStateTransitionTracker.TransitionAcToBattery);
|
||||
Assert.Empty(tracker.Evaluate(Battery(13)));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Each_level_latches_once_per_lifetime()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Battery(70));
|
||||
|
||||
Assert.Single(tracker.Evaluate(Battery(48))); // 50
|
||||
Assert.Empty(tracker.Evaluate(Battery(47))); // still inside 50, no re-emit
|
||||
Assert.Single(tracker.Evaluate(Battery(29))); // 30
|
||||
Assert.Single(tracker.Evaluate(Battery(15))); // 15 (boundary: <= level)
|
||||
Assert.Empty(tracker.Evaluate(Battery(5))); // ladder exhausted
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Threshold_severity_ladder_is_info_warning_error()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Battery(70));
|
||||
|
||||
var fifty = Assert.Single(tracker.Evaluate(Battery(50)));
|
||||
Assert.Equal(EventSeverity.Info, fifty.Severity);
|
||||
Assert.False(fifty.ImmediateUpload);
|
||||
|
||||
var thirty = Assert.Single(tracker.Evaluate(Battery(30)));
|
||||
Assert.Equal(EventSeverity.Warning, thirty.Severity);
|
||||
Assert.True(thirty.ImmediateUpload);
|
||||
|
||||
var fifteen = Assert.Single(tracker.Evaluate(Battery(15)));
|
||||
Assert.Equal(EventSeverity.Error, fifteen.Severity);
|
||||
Assert.True(fifteen.ImmediateUpload);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Null_battery_percent_skips_thresholds_but_ac_diff_still_works()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Ac(null));
|
||||
|
||||
var e = Assert.Single(tracker.Evaluate(Battery(null, lifeMinutes: null)));
|
||||
|
||||
Assert.Equal(PowerStateTransitionTracker.TransitionAcToBattery, e.Transition);
|
||||
Assert.Null(e.BatteryPercent);
|
||||
Assert.Contains("battery level unknown", e.Message);
|
||||
Assert.Empty(tracker.Evaluate(Battery(null, lifeMinutes: null)));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Probe_error_mid_run_preserves_state()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Ac());
|
||||
|
||||
Assert.Empty(tracker.Evaluate(new PowerStateResult { ProbeError = "transient" }));
|
||||
// State was not corrupted by the error — the real transition still diffs cleanly.
|
||||
Assert.Single(tracker.Evaluate(Battery(60)));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Evaluate_without_baseline_latches_like_a_baseline()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
|
||||
var e = Assert.Single(tracker.Evaluate(Battery(12)));
|
||||
|
||||
// No phantom ac_to_battery from the implicit default state — only the threshold.
|
||||
Assert.Equal(PowerStateTransitionTracker.TransitionThresholdCrossed, e.Transition);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Emission_cap_suppresses_and_flips_flag_once()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker(maxEmissions: 4);
|
||||
tracker.Baseline(Ac());
|
||||
|
||||
// Flapping power source: each edge is one emission.
|
||||
for (var i = 0; i < 2; i++)
|
||||
{
|
||||
Assert.Single(tracker.Evaluate(Battery(80)));
|
||||
Assert.Single(tracker.Evaluate(Ac(80)));
|
||||
}
|
||||
Assert.False(tracker.EmissionCapReached);
|
||||
|
||||
Assert.Empty(tracker.Evaluate(Battery(80)));
|
||||
Assert.True(tracker.EmissionCapReached);
|
||||
Assert.Empty(tracker.Evaluate(Ac(80)));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Message_carries_percent_and_transition_context()
|
||||
{
|
||||
var tracker = new PowerStateTransitionTracker();
|
||||
tracker.Baseline(Ac(62));
|
||||
|
||||
var toBattery = Assert.Single(tracker.Evaluate(Battery(62)));
|
||||
Assert.Contains("AC to battery", toBattery.Message);
|
||||
Assert.Contains("battery 62%", toBattery.Message);
|
||||
|
||||
var backToAc = Assert.Single(tracker.Evaluate(Ac(63)));
|
||||
Assert.Contains("back to AC", backToAc.Message);
|
||||
Assert.Contains("charging", backToAc.Message);
|
||||
}
|
||||
}
|
||||
}
|
||||
+429
@@ -0,0 +1,429 @@
|
||||
#nullable enable
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
|
||||
using AutopilotMonitor.Agent.V2.Core.Orchestration;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
|
||||
using AutopilotMonitor.DecisionCore.Engine;
|
||||
using AutopilotMonitor.DecisionCore.Signals;
|
||||
using AutopilotMonitor.Shared;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
|
||||
{
|
||||
/// <summary>
|
||||
/// SystemTimelineTracker — clock steps (Kernel-General 1) and completed sleep episodes
|
||||
/// (Power-Troubleshooter 1 / Kernel-Power 507) from the System channel. Drives the primitive
|
||||
/// <see cref="SystemTimelineTracker.ProcessEvent"/> test seam (no real EventRecord, which is
|
||||
/// abstract + Windows-only), mirroring the WindowsUpdateTracker test pattern.
|
||||
/// </summary>
|
||||
public sealed class SystemTimelineTrackerTests : IDisposable
|
||||
{
|
||||
private static readonly DateTime At = new DateTime(2026, 8, 21, 10, 0, 0, DateTimeKind.Utc);
|
||||
|
||||
private readonly TempDirectory _tmp = new TempDirectory();
|
||||
private readonly FakeSignalIngressSink _sink;
|
||||
private readonly SystemTimelineTracker _tracker;
|
||||
private long _nextRecordId = 1;
|
||||
|
||||
public SystemTimelineTrackerTests()
|
||||
{
|
||||
_sink = new FakeSignalIngressSink();
|
||||
var post = new InformationalEventPost(_sink, new VirtualClock(At));
|
||||
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
|
||||
_tracker = new SystemTimelineTracker(
|
||||
sessionId: "sess-stl",
|
||||
tenantId: "tenant-stl",
|
||||
post: post,
|
||||
logger: logger,
|
||||
backfillEnabled: false,
|
||||
stateDirectory: null); // in-memory dedup for most tests
|
||||
}
|
||||
|
||||
public void Dispose() => _tmp.Dispose();
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Emission helpers
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
private void EmitClockChange(
|
||||
DateTime? oldTime, DateTime? newTime, string? rawDeltaMs = null,
|
||||
int reason = 1, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null)
|
||||
{
|
||||
var data = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
|
||||
if (oldTime.HasValue) data["OldTime"] = oldTime.Value.ToString("o");
|
||||
if (newTime.HasValue) data["NewTime"] = newTime.Value.ToString("o");
|
||||
if (rawDeltaMs != null) data["TimeDeltaInMs"] = rawDeltaMs;
|
||||
data["Reason"] = reason.ToString();
|
||||
data["ProcessName"] = @"\Device\HarddiskVolume3\Windows\System32\svchost.exe";
|
||||
data["ProcessID"] = "6892";
|
||||
|
||||
_tracker.ProcessEvent(
|
||||
providerName: SystemTimelineTracker.ProviderKernelGeneral,
|
||||
eventId: SystemTimelineTracker.EventId_ClockChange,
|
||||
recordId: recordId ?? _nextRecordId++,
|
||||
timeCreatedUtc: timeCreatedUtc ?? At,
|
||||
eventData: data,
|
||||
isBackfill: isBackfill);
|
||||
}
|
||||
|
||||
private void EmitClassicResume(
|
||||
DateTime sleepTime, DateTime wakeTime, string? effectiveState = "1",
|
||||
string? wakeSourceText = null, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null)
|
||||
{
|
||||
var data = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["SleepTime"] = sleepTime.ToString("o"),
|
||||
["WakeTime"] = wakeTime.ToString("o"),
|
||||
["WakeSourceType"] = "0",
|
||||
};
|
||||
if (effectiveState != null) data["EffectiveState"] = effectiveState;
|
||||
if (wakeSourceText != null) data["WakeSourceText"] = wakeSourceText;
|
||||
|
||||
_tracker.ProcessEvent(
|
||||
providerName: SystemTimelineTracker.ProviderPowerTroubleshooter,
|
||||
eventId: SystemTimelineTracker.EventId_ClassicResume,
|
||||
recordId: recordId ?? _nextRecordId++,
|
||||
timeCreatedUtc: timeCreatedUtc ?? wakeTime,
|
||||
eventData: data,
|
||||
isBackfill: isBackfill);
|
||||
}
|
||||
|
||||
private void EmitModernStandbyExit(
|
||||
long sleepDurationUs, long durationUs, string powerStateAc = "true",
|
||||
long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null)
|
||||
{
|
||||
var data = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["SleepDurationInUs"] = sleepDurationUs.ToString(),
|
||||
["DurationInUs"] = durationUs.ToString(),
|
||||
["Reason"] = "32",
|
||||
["PowerStateAc"] = powerStateAc,
|
||||
["BatteryRemainingCapacityOnExit"] = "34180",
|
||||
};
|
||||
|
||||
_tracker.ProcessEvent(
|
||||
providerName: SystemTimelineTracker.ProviderKernelPower,
|
||||
eventId: SystemTimelineTracker.EventId_ModernStandbyExit,
|
||||
recordId: recordId ?? _nextRecordId++,
|
||||
timeCreatedUtc: timeCreatedUtc ?? At,
|
||||
eventData: data,
|
||||
isBackfill: isBackfill);
|
||||
}
|
||||
|
||||
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> ByType(string eventType) =>
|
||||
_sink.Posted.Where(p =>
|
||||
p.Kind == DecisionSignalKind.InformationalEvent
|
||||
&& p.Payload != null
|
||||
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
|
||||
&& et == eventType).ToList();
|
||||
|
||||
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal s) =>
|
||||
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// system_clock_changed
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void ClockStep_AboveThreshold_EmitsInfo_WithSignedDeltaAndNormalizedTimes()
|
||||
{
|
||||
var oldTime = At;
|
||||
var newTime = At.AddSeconds(10);
|
||||
EmitClockChange(oldTime, newTime, reason: 1);
|
||||
|
||||
var s = Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
|
||||
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
|
||||
Assert.Equal("false", s.Payload![SignalPayloadKeys.ImmediateUpload]);
|
||||
|
||||
var data = Data(s);
|
||||
Assert.Equal(10_000L, data["timeDeltaMs"]);
|
||||
Assert.Equal("application_set", data["reasonText"]);
|
||||
Assert.Equal(1, data["reason"]);
|
||||
Assert.Equal(oldTime.ToString("o"), data["oldTime"]);
|
||||
Assert.Equal(newTime.ToString("o"), data["newTime"]);
|
||||
Assert.Equal(6892, data["processId"]);
|
||||
Assert.Contains("forward", (string)s.Payload![SignalPayloadKeys.Message]);
|
||||
Assert.Contains("svchost.exe", (string)s.Payload![SignalPayloadKeys.Message]);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(1)]
|
||||
[InlineData(1999)]
|
||||
public void ClockMicroStep_BelowThreshold_Suppressed(int deltaMs)
|
||||
{
|
||||
EmitClockChange(At, At.AddMilliseconds(deltaMs));
|
||||
Assert.Empty(ByType(Constants.EventTypes.SystemClockChanged));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ClockStep_Backward_EmitsNegativeDelta()
|
||||
{
|
||||
EmitClockChange(oldTime: At, newTime: At.AddSeconds(-3));
|
||||
|
||||
var s = Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
|
||||
Assert.Equal(-3000L, Data(s)["timeDeltaMs"]);
|
||||
Assert.Contains("backward", (string)s.Payload![SignalPayloadKeys.Message]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ClockStep_AtLeastFiveMinutes_WarningAndImmediateUpload()
|
||||
{
|
||||
EmitClockChange(At, At.AddMinutes(5), reason: 2);
|
||||
|
||||
var s = Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
|
||||
Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]);
|
||||
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
|
||||
Assert.Equal("hardware_clock_sync", Data(s)["reasonText"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ClockStep_NoUsableTimesOrDelta_Skipped()
|
||||
{
|
||||
EmitClockChange(oldTime: null, newTime: null, rawDeltaMs: "not-a-number");
|
||||
Assert.Empty(ByType(Constants.EventTypes.SystemClockChanged));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ClockStep_FallsBackToTimeDeltaInMs_WhenTimesUnparseable()
|
||||
{
|
||||
// Only the raw delta parses — the direction information of NewTime/OldTime is gone,
|
||||
// but the magnitude still gates and emits.
|
||||
EmitClockChange(oldTime: null, newTime: null, rawDeltaMs: "7200000");
|
||||
|
||||
var s = Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
|
||||
Assert.Equal(7_200_000L, Data(s)["timeDeltaMs"]);
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// system_sleep_episode — classic (Power-Troubleshooter 1)
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void ClassicResume_EffectiveState5_EmitsHibernateEpisode()
|
||||
{
|
||||
var sleepAt = At.AddHours(-2);
|
||||
EmitClassicResume(sleepAt, At, effectiveState: "5", wakeSourceText: "Power Button");
|
||||
|
||||
var s = Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode));
|
||||
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
|
||||
|
||||
var data = Data(s);
|
||||
Assert.Equal("hibernate", data["kind"]);
|
||||
Assert.Equal(7200L, data["durationSeconds"]);
|
||||
Assert.Equal(sleepAt.ToString("o"), data["enteredAt"]);
|
||||
Assert.Equal(At.ToString("o"), data["exitedAt"]);
|
||||
Assert.Equal("Power Button", data["wakeSourceText"]);
|
||||
Assert.Contains("hibernate", (string)s.Payload![SignalPayloadKeys.Message]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ClassicResume_NonHibernateState_EmitsSleepKind()
|
||||
{
|
||||
EmitClassicResume(At.AddMinutes(-30), At, effectiveState: "4");
|
||||
Assert.Equal("sleep", Data(Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode)))["kind"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ClassicResume_ShortEpisode_StillEmits()
|
||||
{
|
||||
// No duration floor for classic sleep: the event only exists for genuinely completed
|
||||
// S3/S4 transitions, so a 30 s real sleep is still signal.
|
||||
EmitClassicResume(At.AddSeconds(-30), At);
|
||||
|
||||
var s = Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode));
|
||||
Assert.Equal(30L, Data(s)["durationSeconds"]);
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// system_sleep_episode — Modern Standby (Kernel-Power 507)
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void ModernStandbyExit_EmitsEpisode_WithDerivedEnterTime()
|
||||
{
|
||||
// Values from a live 507 on this dev machine: 38 min scenario, ~38 min actual sleep.
|
||||
EmitModernStandbyExit(sleepDurationUs: 2_324_095_807, durationUs: 2_330_201_652);
|
||||
|
||||
var s = Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode));
|
||||
var data = Data(s);
|
||||
Assert.Equal("modern_standby", data["kind"]);
|
||||
Assert.Equal(2330L, data["durationSeconds"]);
|
||||
Assert.Equal(2324L, data["sleepDurationSeconds"]);
|
||||
Assert.Equal(true, data["onAcPower"]);
|
||||
Assert.Equal(34180L, data["batteryRemainingCapacityOnExit"]);
|
||||
Assert.Equal(At.ToString("o"), data["exitedAt"]);
|
||||
Assert.Equal(At.AddMilliseconds(-2_330_201.652).ToString("o"), data["enteredAt"]);
|
||||
Assert.Contains("Modern Standby", (string)s.Payload![SignalPayloadKeys.Message]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ModernStandbyExit_SleepBelowFloor_Suppressed()
|
||||
{
|
||||
EmitModernStandbyExit(sleepDurationUs: 30_000_000, durationUs: 90_000_000); // 30 s sleep
|
||||
Assert.Empty(ByType(Constants.EventTypes.SystemSleepEpisode));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ModernStandbyExit_ScreenOffWithoutSleep_Suppressed()
|
||||
{
|
||||
// SleepEntered=false scenarios carry SleepDurationInUs 0 — the single duration gate
|
||||
// suppresses them without a separate branch.
|
||||
EmitModernStandbyExit(sleepDurationUs: 0, durationUs: 6_106_696);
|
||||
Assert.Empty(ByType(Constants.EventTypes.SystemSleepEpisode));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ModernStandbyExit_ScenarioShorterThanSleep_ClampsToSleepDuration()
|
||||
{
|
||||
EmitModernStandbyExit(sleepDurationUs: 120_000_000, durationUs: 0);
|
||||
|
||||
var data = Data(Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode)));
|
||||
Assert.Equal(120L, data["durationSeconds"]);
|
||||
Assert.Equal(120L, data["sleepDurationSeconds"]);
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Backfill semantics
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void BackfilledEvent_UsesEventTimeForTimelineTimestamp_AndFlagsData()
|
||||
{
|
||||
// A pre-agent standby episode (record created before the agent started) must land on
|
||||
// the timeline where the device actually woke, not at agent start.
|
||||
var wakeTime = At.AddHours(-3);
|
||||
EmitClassicResume(wakeTime.AddHours(-1), wakeTime, isBackfill: true, timeCreatedUtc: wakeTime);
|
||||
|
||||
var s = Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode));
|
||||
Assert.Equal(wakeTime, s.OccurredAtUtc);
|
||||
Assert.Equal(true, Data(s)["backfilled"]);
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Dedup / watermark
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void SameRecordId_ProcessedTwice_EmitsOnce()
|
||||
{
|
||||
EmitClockChange(At, At.AddMinutes(1), recordId: 42);
|
||||
EmitClockChange(At, At.AddMinutes(1), recordId: 42);
|
||||
|
||||
Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BackfillEventWithLowerRecordId_AfterLiveEvent_IsStillEmitted()
|
||||
{
|
||||
// The live watcher is armed BEFORE backfill runs, so a live record with a higher
|
||||
// RecordId can be processed first. An older, never-emitted backfill record must NOT be
|
||||
// suppressed — pre-agent episodes are the whole point (high-water-mark trap).
|
||||
EmitClockChange(At, At.AddMinutes(1), recordId: 100); // live, high
|
||||
EmitClockChange(At, At.AddMinutes(2), recordId: 50, isBackfill: true); // backfill, older
|
||||
|
||||
Assert.Equal(2, ByType(Constants.EventTypes.SystemClockChanged).Count);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Watermark_PersistsAcrossTrackerInstances()
|
||||
{
|
||||
var post = new InformationalEventPost(_sink, new VirtualClock(At));
|
||||
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
|
||||
var clockData = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["OldTime"] = At.ToString("o"),
|
||||
["NewTime"] = At.AddMinutes(10).ToString("o"),
|
||||
["Reason"] = "1",
|
||||
};
|
||||
|
||||
var first = new SystemTimelineTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
|
||||
first.ProcessEvent(SystemTimelineTracker.ProviderKernelGeneral, SystemTimelineTracker.EventId_ClockChange,
|
||||
recordId: 500, timeCreatedUtc: At, eventData: clockData, isBackfill: false);
|
||||
|
||||
// A fresh tracker (agent restart) must load the watermark and skip the already-emitted
|
||||
// record when the backfill re-reads it.
|
||||
var second = new SystemTimelineTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
|
||||
second.LoadWatermark();
|
||||
second.ProcessEvent(SystemTimelineTracker.ProviderKernelGeneral, SystemTimelineTracker.EventId_ClockChange,
|
||||
recordId: 500, timeCreatedUtc: At, eventData: clockData, isBackfill: true);
|
||||
|
||||
Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void UnknownProviderOrEventId_Ignored()
|
||||
{
|
||||
_tracker.ProcessEvent("Microsoft-Windows-Kernel-Power", eventId: 566, recordId: _nextRecordId++,
|
||||
timeCreatedUtc: At, eventData: new Dictionary<string, string>(), isBackfill: false);
|
||||
_tracker.ProcessEvent("Some-Other-Provider", eventId: 1, recordId: _nextRecordId++,
|
||||
timeCreatedUtc: At, eventData: new Dictionary<string, string>(), isBackfill: false);
|
||||
|
||||
Assert.Empty(_sink.Posted);
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// XPath + parsing
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void BuildXPath_ContainsAllThreeProviderIdPairs_AndNo506()
|
||||
{
|
||||
var xpath = SystemTimelineTracker.BuildXPath();
|
||||
Assert.Contains("Provider[@Name='Microsoft-Windows-Kernel-General'] and EventID=1", xpath);
|
||||
Assert.Contains("Provider[@Name='Microsoft-Windows-Power-Troubleshooter'] and EventID=1", xpath);
|
||||
Assert.Contains("Provider[@Name='Microsoft-Windows-Kernel-Power'] and EventID=507", xpath);
|
||||
Assert.DoesNotContain("506", xpath);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BuildBackfillXPath_AppendsTimediffClause()
|
||||
{
|
||||
var xpath = SystemTimelineTracker.BuildBackfillXPath(86_400_000);
|
||||
Assert.Contains("TimeCreated[timediff(@SystemTime) <= 86400000]", xpath);
|
||||
Assert.Contains("EventID=507", xpath);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ParseEventData_RealKernelGeneralPayload_YieldsAuthoritativeFields()
|
||||
{
|
||||
// Verbatim (values shortened) from a live Kernel-General 1 record on a 26220 dev machine.
|
||||
const string xml =
|
||||
"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>" +
|
||||
"<System><Provider Name='Microsoft-Windows-Kernel-General'/><EventID>1</EventID></System>" +
|
||||
"<EventData>" +
|
||||
"<Data Name='NewTime'>2026-08-21T09:40:02.1987028Z</Data>" +
|
||||
"<Data Name='OldTime'>2026-08-21T09:40:02.1973028Z</Data>" +
|
||||
"<Data Name='TimeDeltaInMs'>1</Data>" +
|
||||
"<Data Name='Reason'>1</Data>" +
|
||||
"<Data Name='ProcessName'>\\Device\\HarddiskVolume3\\Windows\\System32\\svchost.exe</Data>" +
|
||||
"<Data Name='ProcessID'>6892</Data>" +
|
||||
"</EventData></Event>";
|
||||
|
||||
var data = SystemTimelineTracker.ParseEventData(xml);
|
||||
Assert.Equal("1", data["TimeDeltaInMs"]);
|
||||
Assert.Equal("2026-08-21T09:40:02.1987028Z", data["newtime"]); // case-insensitive keys
|
||||
Assert.NotNull(SystemTimelineTracker.TryParseUtc(data["OldTime"]));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void FormatDuration_CompactForms()
|
||||
{
|
||||
Assert.Equal("2h 05m", SystemTimelineTracker.FormatDuration(TimeSpan.FromMinutes(125)));
|
||||
Assert.Equal("56m 12s", SystemTimelineTracker.FormatDuration(TimeSpan.FromSeconds(56 * 60 + 12)));
|
||||
Assert.Equal("45s", SystemTimelineTracker.FormatDuration(TimeSpan.FromSeconds(45)));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ProcessLeafName_StripsDevicePath()
|
||||
{
|
||||
Assert.Equal("svchost.exe",
|
||||
SystemTimelineTracker.ProcessLeafName(@"\Device\HarddiskVolume3\Windows\System32\svchost.exe"));
|
||||
Assert.Null(SystemTimelineTracker.ProcessLeafName(null));
|
||||
Assert.Null(SystemTimelineTracker.ProcessLeafName(@"\Device\HarddiskVolume3\"));
|
||||
}
|
||||
}
|
||||
}
|
||||
+126
@@ -0,0 +1,126 @@
|
||||
using System;
|
||||
using System.Linq;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Transport;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Transport
|
||||
{
|
||||
public class DoGroupIdServiceTests
|
||||
{
|
||||
// -------------------------------------------------------------------
|
||||
// ComputeGroupId — RealmJoin byte-layout compatibility
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void ComputeGroupId_MatchesHandComputedVector()
|
||||
{
|
||||
// Layout: {101,48,6} + {0,0,0} + {ip[1],0,ip[2],ip[3]} + mac — through new Guid(byte[])
|
||||
// the first three groups print little-endian, the last eight bytes verbatim.
|
||||
var guid = DoGroupIdService.ComputeGroupId(
|
||||
new byte[] { 192, 168, 178, 1 },
|
||||
new byte[] { 0x00, 0x11, 0x22, 0x33, 0x44, 0x55 });
|
||||
|
||||
Assert.Equal("00063065-0000-00a8-b201-001122334455", guid.ToString());
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(new byte[] { 10, 0, 0, 1 }, new byte[] { 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF })]
|
||||
[InlineData(new byte[] { 192, 168, 1, 254 }, new byte[] { 0x00, 0x50, 0x56, 0x01, 0x02, 0x03 })]
|
||||
[InlineData(new byte[] { 172, 16, 254, 3 }, new byte[] { 0xF0, 0x9F, 0xC2, 0x00, 0x00, 0x01 })]
|
||||
public void ComputeGroupId_MatchesRealmJoinReferenceImplementation(byte[] ip, byte[] mac)
|
||||
{
|
||||
Assert.Equal(RealmJoinReferenceGuid(ip, mac), DoGroupIdService.ComputeGroupId(ip, mac));
|
||||
}
|
||||
|
||||
/// <summary>Verbatim port of RealmJoin's ConvertToGuid — the compatibility contract.</summary>
|
||||
private static Guid RealmJoinReferenceGuid(byte[] ipBytes, byte[] address)
|
||||
{
|
||||
byte[] prefix = { 101, 48, 6 };
|
||||
var stuff = new byte[4];
|
||||
stuff[0] = ipBytes[1];
|
||||
stuff[2] = ipBytes[2];
|
||||
stuff[3] = ipBytes[3];
|
||||
|
||||
return new Guid(prefix
|
||||
.Concat(new byte[16 - prefix.Length - stuff.Length - address.Length])
|
||||
.Concat(stuff)
|
||||
.Concat(address)
|
||||
.ToArray());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ComputeGroupId_RejectsNonIpv4AndBadMacLengths()
|
||||
{
|
||||
var ip4 = new byte[] { 10, 0, 0, 1 };
|
||||
var mac = new byte[] { 1, 2, 3, 4, 5, 6 };
|
||||
|
||||
Assert.Throws<ArgumentException>(() => DoGroupIdService.ComputeGroupId(new byte[16], mac));
|
||||
Assert.Throws<ArgumentException>(() => DoGroupIdService.ComputeGroupId(null!, mac));
|
||||
Assert.Throws<ArgumentException>(() => DoGroupIdService.ComputeGroupId(ip4, new byte[8]));
|
||||
Assert.Throws<ArgumentException>(() => DoGroupIdService.ComputeGroupId(ip4, null!));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ComputeGroupId_RegistryFormat_IsBracelessLowercase()
|
||||
{
|
||||
// RealmJoin writes SetValue(name, Guid) which stringifies via Guid.ToString() ("d"
|
||||
// format) — our written value must be the identical string form.
|
||||
var text = DoGroupIdService.ComputeGroupId(
|
||||
new byte[] { 10, 1, 2, 3 }, new byte[] { 0xAB, 0xCD, 0xEF, 0x01, 0x02, 0x03 }).ToString();
|
||||
|
||||
Assert.DoesNotContain("{", text);
|
||||
Assert.Equal(text.ToLowerInvariant(), text);
|
||||
Assert.Equal(36, text.Length);
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// DecideAction — ownership matrix
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
private const string Computed = "00063065-0000-00a8-b201-001122334455";
|
||||
private const string Other = "11111111-2222-3333-4444-555555555555";
|
||||
|
||||
[Theory]
|
||||
[InlineData(null)]
|
||||
[InlineData("")]
|
||||
public void DecideAction_NoExistingValue_Writes(string? existing)
|
||||
{
|
||||
Assert.Equal(DoGroupIdAction.Write, DoGroupIdService.DecideAction(existing, Computed, null));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DecideAction_ExistingEqualsComputed_IsNoOp()
|
||||
{
|
||||
Assert.Equal(DoGroupIdAction.NoOp, DoGroupIdService.DecideAction(Computed, Computed, null));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DecideAction_ExistingEqualsComputed_ToleratesBracesAndCase()
|
||||
{
|
||||
// A value written by another tool in "{...}"/uppercase form is still semantically ours.
|
||||
var braced = "{" + Computed.ToUpperInvariant() + "}";
|
||||
Assert.Equal(DoGroupIdAction.NoOp, DoGroupIdService.DecideAction(braced, Computed, null));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DecideAction_ExistingIsOurStaleValue_WritesUpdate()
|
||||
{
|
||||
// Gateway changed: the registry still holds the GUID we wrote for the old network.
|
||||
Assert.Equal(DoGroupIdAction.Write, DoGroupIdService.DecideAction(Other, Computed, Other));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DecideAction_ForeignValue_Skips()
|
||||
{
|
||||
Assert.Equal(DoGroupIdAction.SkipForeign, DoGroupIdService.DecideAction(Other, Computed, null));
|
||||
Assert.Equal(DoGroupIdAction.SkipForeign, DoGroupIdService.DecideAction(Other, Computed, Computed));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DecideAction_NonGuidGarbageExisting_SkipsAsForeign()
|
||||
{
|
||||
// Never overwrite a value we can't even parse — conservative by design.
|
||||
Assert.Equal(DoGroupIdAction.SkipForeign, DoGroupIdService.DecideAction("not-a-guid", Computed, null));
|
||||
}
|
||||
}
|
||||
}
|
||||
+162
@@ -0,0 +1,162 @@
|
||||
#nullable enable
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Orchestration;
|
||||
using AutopilotMonitor.Agent.V2.Core.Runtime;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
|
||||
using AutopilotMonitor.DecisionCore.Engine;
|
||||
using AutopilotMonitor.DecisionCore.Signals;
|
||||
using AutopilotMonitor.Shared;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
|
||||
{
|
||||
/// <summary>
|
||||
/// PowerStateWatcherHost — no-battery/probe-error devices never arm, a low-battery baseline
|
||||
/// emits its threshold event onto the single rail, and debounced ticks translate tracker
|
||||
/// emissions into <c>power_state_change</c> events. WMI arming itself stays untested
|
||||
/// (ConsoleBypassWatcher precedent); <c>StartCore(armWmi: false)</c> exercises everything else.
|
||||
/// </summary>
|
||||
public sealed class PowerStateWatcherHostTests : IDisposable
|
||||
{
|
||||
private static readonly DateTime At = new DateTime(2026, 8, 21, 9, 0, 0, DateTimeKind.Utc);
|
||||
|
||||
private readonly TempDirectory _tmp = new TempDirectory();
|
||||
private readonly FakeSignalIngressSink _sink = new FakeSignalIngressSink();
|
||||
private readonly AgentLogger _logger;
|
||||
private PowerStateResult _probeResult = new PowerStateResult { OnAcPower = true, HasBattery = true, BatteryPercent = 80 };
|
||||
|
||||
public PowerStateWatcherHostTests()
|
||||
{
|
||||
_logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
|
||||
}
|
||||
|
||||
public void Dispose() => _tmp.Dispose();
|
||||
|
||||
private PowerStateWatcherHost Build() => new PowerStateWatcherHost(
|
||||
sessionId: "sess-power",
|
||||
tenantId: "tenant-power",
|
||||
ingress: _sink,
|
||||
clock: new VirtualClock(At),
|
||||
logger: _logger,
|
||||
probe: () => _probeResult);
|
||||
|
||||
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> PowerEvents() =>
|
||||
_sink.Posted.Where(p =>
|
||||
p.Kind == DecisionSignalKind.InformationalEvent
|
||||
&& p.Payload != null
|
||||
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
|
||||
&& et == Constants.EventTypes.PowerStateChange).ToList();
|
||||
|
||||
[Fact]
|
||||
public void No_battery_emits_nothing()
|
||||
{
|
||||
_probeResult = new PowerStateResult { OnAcPower = true, HasBattery = false };
|
||||
|
||||
Build().StartCore(armWmi: false);
|
||||
|
||||
Assert.Empty(_sink.Posted);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Probe_error_emits_nothing()
|
||||
{
|
||||
_probeResult = new PowerStateResult { ProbeError = "GetSystemPowerStatus returned false" };
|
||||
|
||||
Build().StartCore(armWmi: false);
|
||||
|
||||
Assert.Empty(_sink.Posted);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Baseline_below_threshold_emits_threshold_event_with_immediate_upload()
|
||||
{
|
||||
_probeResult = new PowerStateResult { OnAcPower = false, HasBattery = true, BatteryPercent = 12, BatteryLifeMinutes = 34 };
|
||||
|
||||
Build().StartCore(armWmi: false);
|
||||
|
||||
var s = Assert.Single(PowerEvents());
|
||||
Assert.Equal("Error", s.Payload![SignalPayloadKeys.Severity]);
|
||||
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
|
||||
Assert.Contains("15%", s.Payload![SignalPayloadKeys.Message]);
|
||||
|
||||
var data = (IReadOnlyDictionary<string, object>)s.TypedPayload!;
|
||||
Assert.Equal("threshold_crossed", data["transition"]);
|
||||
Assert.Equal(15, data["thresholdPercent"]);
|
||||
Assert.Equal(12, data["batteryPercent"]);
|
||||
Assert.Equal(false, data["onAcPower"]);
|
||||
Assert.Equal(34, data["batteryLifeMinutes"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Tick_after_unplug_emits_ac_to_battery_warning()
|
||||
{
|
||||
var host = Build();
|
||||
host.StartCore(armWmi: false);
|
||||
Assert.Empty(PowerEvents());
|
||||
|
||||
_probeResult = new PowerStateResult { OnAcPower = false, HasBattery = true, BatteryPercent = 79 };
|
||||
host.TickForTest();
|
||||
|
||||
var s = Assert.Single(PowerEvents());
|
||||
Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]);
|
||||
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
|
||||
|
||||
var data = (IReadOnlyDictionary<string, object>)s.TypedPayload!;
|
||||
Assert.Equal("ac_to_battery", data["transition"]);
|
||||
Assert.Equal(79, data["batteryPercent"]);
|
||||
Assert.False(data.ContainsKey("thresholdPercent"));
|
||||
|
||||
// Replug: Info recovery event without immediate upload.
|
||||
_probeResult = new PowerStateResult { OnAcPower = true, HasBattery = true, BatteryPercent = 79, IsCharging = true };
|
||||
host.TickForTest();
|
||||
|
||||
Assert.Equal(2, PowerEvents().Count);
|
||||
var back = PowerEvents()[1];
|
||||
Assert.Equal("Info", back.Payload![SignalPayloadKeys.Severity]);
|
||||
Assert.Equal("false", back.Payload![SignalPayloadKeys.ImmediateUpload]);
|
||||
Assert.Equal("battery_to_ac", ((IReadOnlyDictionary<string, object>)back.TypedPayload!)["transition"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Tick_with_unchanged_state_emits_nothing()
|
||||
{
|
||||
var host = Build();
|
||||
host.StartCore(armWmi: false);
|
||||
|
||||
host.TickForTest();
|
||||
host.TickForTest();
|
||||
|
||||
Assert.Empty(PowerEvents());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Stopped_host_ignores_ticks()
|
||||
{
|
||||
var host = Build();
|
||||
host.StartCore(armWmi: false);
|
||||
host.Stop();
|
||||
|
||||
_probeResult = new PowerStateResult { OnAcPower = false, HasBattery = true, BatteryPercent = 50 };
|
||||
host.TickForTest();
|
||||
|
||||
Assert.Empty(PowerEvents());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Unknown_battery_percent_serializes_as_unknown()
|
||||
{
|
||||
var host = Build();
|
||||
host.StartCore(armWmi: false);
|
||||
|
||||
_probeResult = new PowerStateResult { OnAcPower = false, HasBattery = true, BatteryPercent = null, BatteryLifeMinutes = null };
|
||||
host.TickForTest();
|
||||
|
||||
var data = (IReadOnlyDictionary<string, object>)Assert.Single(PowerEvents()).TypedPayload!;
|
||||
Assert.Equal("unknown", data["batteryPercent"]);
|
||||
Assert.Equal("unknown", data["batteryLifeMinutes"]);
|
||||
}
|
||||
}
|
||||
}
|
||||
+2
@@ -35,6 +35,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
|
||||
// OS-eventlog forwarding — observability, not enrollment progress (session 8bc1180f:
|
||||
// an EventID-100 burst must not keep the periodic collectors alive).
|
||||
[InlineData("modern_deployment_log")]
|
||||
// Environment observation — battery drain on an idle device must not reset idle clocks.
|
||||
[InlineData("power_state_change")]
|
||||
// Agent health / control / transport — not device/enrollment progress (P2).
|
||||
[InlineData("collector_degraded")]
|
||||
[InlineData("telemetry_upload_poisoned")]
|
||||
|
||||
+57
@@ -0,0 +1,57 @@
|
||||
#nullable enable
|
||||
using System;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Orchestration;
|
||||
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
|
||||
using AutopilotMonitor.DecisionCore.Engine;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
|
||||
{
|
||||
/// <summary>
|
||||
/// SystemTimelineWatcherHost — thin ICollectorHost wrapper over SystemTimelineTracker.
|
||||
/// Real event-log arming stays untested by convention (ConsoleBypassWatcher precedent);
|
||||
/// these tests pin the lifecycle contract only.
|
||||
/// </summary>
|
||||
public sealed class SystemTimelineWatcherHostTests : IDisposable
|
||||
{
|
||||
private readonly TempDirectory _tmp = new TempDirectory();
|
||||
|
||||
public void Dispose() => _tmp.Dispose();
|
||||
|
||||
private SystemTimelineWatcherHost CreateHost() => new SystemTimelineWatcherHost(
|
||||
sessionId: "sess-stl",
|
||||
tenantId: "tenant-stl",
|
||||
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info),
|
||||
ingress: new FakeSignalIngressSink(),
|
||||
clock: new VirtualClock(new DateTime(2026, 8, 21, 10, 0, 0, DateTimeKind.Utc)),
|
||||
backfillLookbackMinutes: 0,
|
||||
stateDirectory: _tmp.Path);
|
||||
|
||||
[Fact]
|
||||
public void Stop_BeforeStart_DoesNotThrow()
|
||||
{
|
||||
using (var host = CreateHost())
|
||||
{
|
||||
host.Stop();
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Dispose_Twice_DoesNotThrow()
|
||||
{
|
||||
var host = CreateHost();
|
||||
host.Dispose();
|
||||
host.Dispose();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Name_IsStable()
|
||||
{
|
||||
using (var host = CreateHost())
|
||||
{
|
||||
Assert.Equal("SystemTimelineTracker", host.Name);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+108
@@ -339,6 +339,114 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Runtime
|
||||
Assert.False(StartupEnvironmentProbes.IsNtpWithinTolerance(null!));
|
||||
}
|
||||
|
||||
// ================================================================= DO GroupId auto-set
|
||||
// Builder/fingerprint tests only — TrySetGroupId itself does live registry + ARP I/O and
|
||||
// must never run in a unit test (it would write HKLM on the build machine).
|
||||
|
||||
[Fact]
|
||||
public void BuildDoGroupIdEvent_written_success_is_info_with_full_data()
|
||||
{
|
||||
var result = new DoGroupIdSetResult
|
||||
{
|
||||
Success = true,
|
||||
Written = true,
|
||||
GroupId = "00063065-0000-00a8-b201-001122334455",
|
||||
PreviousValue = null,
|
||||
GatewayIp = "192.168.178.1",
|
||||
GatewayMac = "00-11-22-33-44-55",
|
||||
};
|
||||
|
||||
var evt = StartupEnvironmentProbes.BuildDoGroupIdEvent(Config(), result);
|
||||
|
||||
Assert.Equal("do_group_id_auto_set", evt.EventType);
|
||||
Assert.Equal(EventSeverity.Info, evt.Severity);
|
||||
Assert.Equal("StartupEnvironmentProbes", evt.Source);
|
||||
Assert.Equal("S1", evt.SessionId);
|
||||
Assert.Equal("T1", evt.TenantId);
|
||||
Assert.Contains("00063065-0000-00a8-b201-001122334455", evt.Message);
|
||||
Assert.Equal("192.168.178.1", evt.Data["gatewayIp"]);
|
||||
Assert.Equal("00-11-22-33-44-55", evt.Data["gatewayMac"]);
|
||||
Assert.Equal(true, evt.Data["written"]);
|
||||
Assert.Equal(true, evt.Data["success"]);
|
||||
Assert.Equal(false, evt.Data["skipped"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BuildDoGroupIdEvent_noop_success_says_already_set()
|
||||
{
|
||||
var result = new DoGroupIdSetResult
|
||||
{
|
||||
Success = true,
|
||||
Written = false,
|
||||
GroupId = "00063065-0000-00a8-b201-001122334455",
|
||||
};
|
||||
|
||||
var evt = StartupEnvironmentProbes.BuildDoGroupIdEvent(Config(), result);
|
||||
|
||||
Assert.Equal(EventSeverity.Info, evt.Severity);
|
||||
Assert.Contains("already set", evt.Message);
|
||||
Assert.Equal(false, evt.Data["written"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BuildDoGroupIdEvent_skip_is_info_with_reason()
|
||||
{
|
||||
var result = new DoGroupIdSetResult
|
||||
{
|
||||
Skipped = true,
|
||||
SkipReason = "foreign_value_present",
|
||||
PreviousValue = "11111111-2222-3333-4444-555555555555",
|
||||
};
|
||||
|
||||
var evt = StartupEnvironmentProbes.BuildDoGroupIdEvent(Config(), result);
|
||||
|
||||
Assert.Equal(EventSeverity.Info, evt.Severity);
|
||||
Assert.Contains("foreign_value_present", evt.Message);
|
||||
Assert.Equal("foreign_value_present", evt.Data["skipReason"]);
|
||||
Assert.Equal("11111111-2222-3333-4444-555555555555", evt.Data["previousValue"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BuildDoGroupIdEvent_error_is_warning()
|
||||
{
|
||||
var result = new DoGroupIdSetResult { Error = "readback mismatch" };
|
||||
|
||||
var evt = StartupEnvironmentProbes.BuildDoGroupIdEvent(Config(), result);
|
||||
|
||||
Assert.Equal(EventSeverity.Warning, evt.Severity);
|
||||
Assert.Contains("readback mismatch", evt.Message);
|
||||
Assert.Equal("readback mismatch", evt.Data["error"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DoGroupIdFingerprint_roundtrips_ownership_marker_for_success()
|
||||
{
|
||||
var fp = StartupEnvironmentProbes.ComputeDoGroupIdFingerprint(new DoGroupIdSetResult
|
||||
{
|
||||
Success = true,
|
||||
GroupId = "00063065-0000-00a8-b201-001122334455",
|
||||
});
|
||||
|
||||
Assert.Equal("written:00063065-0000-00a8-b201-001122334455", fp);
|
||||
Assert.Equal("00063065-0000-00a8-b201-001122334455", StartupEnvironmentProbes.ExtractLastWrittenGroupId(fp));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void DoGroupIdFingerprint_skip_carries_no_ownership_and_errors_never_dedupe()
|
||||
{
|
||||
var skipFp = StartupEnvironmentProbes.ComputeDoGroupIdFingerprint(new DoGroupIdSetResult
|
||||
{
|
||||
Skipped = true,
|
||||
SkipReason = "mdm_policy_present",
|
||||
});
|
||||
Assert.Equal("skipped:mdm_policy_present", skipFp);
|
||||
Assert.Null(StartupEnvironmentProbes.ExtractLastWrittenGroupId(skipFp));
|
||||
|
||||
// Errors: no fingerprint → the probe re-runs AND re-emits each failing start.
|
||||
Assert.Null(StartupEnvironmentProbes.ComputeDoGroupIdFingerprint(new DoGroupIdSetResult { Error = "boom" }));
|
||||
Assert.Null(StartupEnvironmentProbes.ExtractLastWrittenGroupId(null));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task RunAsync_skips_geo_timezone_and_ntp_once_all_succeeded_only_power_state_emits()
|
||||
{
|
||||
|
||||
+11
-9
@@ -245,11 +245,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.SignalAdapters
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Terminal_to_active_retry_rearms_CompletedAt_and_final_terminal_covers_total_occupancy()
|
||||
public void Terminal_to_active_retry_rearms_BothEndpoints_and_final_terminal_covers_last_attempt()
|
||||
{
|
||||
// Audit Q1: the IME retry path (Error -> Downloading -> Installed is permitted by
|
||||
// AppPackageState.UpdateState). Pre-fix, CompletedAt froze at the first Error, so
|
||||
// the final Installed payload carried the first failure's completedAt/duration.
|
||||
// Audit Q1 + 2026-08 attempt-duration change: the IME retry path
|
||||
// (Error -> Downloading -> Installed is permitted by AppPackageState.UpdateState).
|
||||
// A terminal->active transition re-arms BOTH endpoints, so the final Installed
|
||||
// payload describes the retry ATTEMPT (matching the backend's AppInstallSummaries
|
||||
// semantics) — not the first-start -> last-terminal span across attempts.
|
||||
using var f = new ImeLogTrackerAdapterFixture(T0);
|
||||
using var adapter = new ImeLogTrackerAdapter(f.Tracker, f.Ingress, f.Clock);
|
||||
var app = new AppPackageState("app-R", 0);
|
||||
@@ -261,23 +263,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.SignalAdapters
|
||||
adapter.TriggerAppStateFromTest(app, AppInstallationState.Error, AppInstallationState.Downloading);
|
||||
|
||||
// Re-armed while active again: the retry's download event carries no completedAt,
|
||||
// and the FIRST start stamp is kept.
|
||||
// and the start stamp is the RETRY's re-entry time (fresh attempt window).
|
||||
Assert.Null(adapter.AppTimings["app-R"].CompletedAtUtc);
|
||||
var retryDownload = f.InfoEvent(SharedEventTypes.AppDownloadStarted);
|
||||
Assert.False(retryDownload.Payload!.ContainsKey("completedAt"));
|
||||
Assert.False(retryDownload.Payload.ContainsKey("durationSeconds"));
|
||||
Assert.Equal(T0.ToString("o"), retryDownload.Payload["startedAt"]);
|
||||
Assert.Equal(T0.AddSeconds(15).ToString("o"), retryDownload.Payload["startedAt"]);
|
||||
|
||||
f.Clock.Advance(TimeSpan.FromSeconds(20));
|
||||
adapter.TriggerAppStateFromTest(app, AppInstallationState.Downloading, AppInstallationState.Installed);
|
||||
|
||||
var timing = adapter.AppTimings["app-R"];
|
||||
Assert.Equal(T0, timing.StartedAtUtc);
|
||||
Assert.Equal(T0.AddSeconds(15), timing.StartedAtUtc);
|
||||
Assert.Equal(T0.AddSeconds(35), timing.CompletedAtUtc);
|
||||
Assert.Equal(35.0, timing.DurationSeconds);
|
||||
Assert.Equal(20.0, timing.DurationSeconds);
|
||||
|
||||
var completed = Assert.Single(f.InfoEvents(SharedEventTypes.AppInstallComplete));
|
||||
Assert.Equal("35.00", completed.Payload!["durationSeconds"]);
|
||||
Assert.Equal("20.00", completed.Payload!["durationSeconds"]);
|
||||
Assert.Equal(T0.AddSeconds(35).ToString("o"), completed.Payload["completedAt"]);
|
||||
}
|
||||
|
||||
|
||||
@@ -109,6 +109,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
|
||||
/// </summary>
|
||||
public bool EnableTimezoneAutoSet { get; set; } = false;
|
||||
|
||||
/// <summary>
|
||||
/// Whether to set the Delivery Optimization group ID (DOGroupId policy value) from a
|
||||
/// deterministic network fingerprint (default gateway IP + MAC, RealmJoin-compatible).
|
||||
/// Existing DOGroupId/DOGroupIdSource policies are never overwritten.
|
||||
/// Default: false
|
||||
/// </summary>
|
||||
public bool EnableDoGroupIdAutoSet { get; set; } = false;
|
||||
|
||||
/// <summary>
|
||||
/// Continue-Anyway observation mode (operator-set tenant opt-in, delivered via
|
||||
/// remote config): stamped into the esp_config_detected decision-signal payload so
|
||||
|
||||
@@ -94,6 +94,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
|
||||
// a tenant admin can clear a previously-applied custom NTP server.
|
||||
agentConfig.NtpServer = remoteConfig.NtpServer;
|
||||
agentConfig.EnableTimezoneAutoSet = remoteConfig.EnableTimezoneAutoSet;
|
||||
agentConfig.EnableDoGroupIdAutoSet = remoteConfig.EnableDoGroupIdAutoSet;
|
||||
|
||||
// ---------------- Diagnostics — remote wins (including DiagnosticsUploadMode="Off"
|
||||
// which must clear any CLI-initial default).
|
||||
|
||||
+7
-4
@@ -10,10 +10,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
|
||||
/// the <c>app_tracking_summary</c> terminal event.
|
||||
/// <para>
|
||||
/// <b>StartedAtUtc</b> is recorded on the first lifecycle transition into Downloading /
|
||||
/// Installing / InProgress and never overwritten. <b>CompletedAtUtc</b> is recorded on a
|
||||
/// transition to a terminal state (Installed / Skipped / Postponed / Error) and re-armed
|
||||
/// to <c>null</c> when the app goes terminal → active again (IME retry / re-evaluation,
|
||||
/// audit Q1) — so the final terminal event carries first-start → last-terminal.
|
||||
/// Installing / InProgress. <b>CompletedAtUtc</b> is recorded on a transition to a
|
||||
/// terminal state (Installed / Skipped / Postponed / Error). When the app goes
|
||||
/// terminal → active again (IME retry / re-evaluation pass, audit Q1) BOTH endpoints
|
||||
/// re-arm — CompletedAtUtc to <c>null</c>, StartedAtUtc to the re-entry time — so the
|
||||
/// final terminal event carries the status-defining ATTEMPT's window (2026-08
|
||||
/// attempt-duration change, matching the backend's AppInstallSummaries semantics),
|
||||
/// never the first-start → last-terminal span across install passes.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class AppInstallTiming
|
||||
|
||||
+197
@@ -0,0 +1,197 @@
|
||||
#nullable enable
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using AutopilotMonitor.Agent.V2.Core.Runtime;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
{
|
||||
/// <summary>
|
||||
/// Pure state machine behind <c>PowerStateWatcherHost</c>: diffs successive
|
||||
/// <see cref="PowerStateResult"/> snapshots into <c>power_state_change</c> emissions.
|
||||
/// No WMI, no timers, no logging — fully deterministic and unit-testable.
|
||||
/// <para>
|
||||
/// Semantics:
|
||||
/// <list type="bullet">
|
||||
/// <item><b>AC↔battery transitions</b> — one emission per accepted edge
|
||||
/// (<c>ac_to_battery</c> Warning/immediate, <c>battery_to_ac</c> Info).</item>
|
||||
/// <item><b>Threshold ladder</b> (default 50/30/15 %) — evaluated only while ON battery;
|
||||
/// each level latches once per tracker lifetime (no re-emit when the device charges
|
||||
/// back up and drains again — avoids plug-cycle alarm spam). A multi-level jump
|
||||
/// emits only the LOWEST newly-crossed level. Entering battery power already below
|
||||
/// a level counts as crossing it (covers "unplugged at 12 %" and, via
|
||||
/// <see cref="Baseline"/>, "enrollment started at 12 %").</item>
|
||||
/// <item><b>Storm cap</b> — a lifetime emission cap (default 20) as backstop against a
|
||||
/// flapping dock / dying battery controller; <see cref="EmissionCapReached"/> flips
|
||||
/// once so the host can log a single warning.</item>
|
||||
/// </list>
|
||||
/// A snapshot with <see cref="PowerStateResult.ProbeError"/> is ignored (previous state
|
||||
/// preserved); a null <see cref="PowerStateResult.BatteryPercent"/> skips threshold
|
||||
/// evaluation but AC diffing still works.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal sealed class PowerStateTransitionTracker
|
||||
{
|
||||
public const string TransitionAcToBattery = "ac_to_battery";
|
||||
public const string TransitionBatteryToAc = "battery_to_ac";
|
||||
public const string TransitionThresholdCrossed = "threshold_crossed";
|
||||
|
||||
internal static readonly int[] DefaultThresholdLadder = { 50, 30, 15 };
|
||||
internal const int DefaultMaxEmissions = 20;
|
||||
|
||||
private readonly int[] _ladder; // strictly descending
|
||||
private readonly int _maxEmissions;
|
||||
private readonly HashSet<int> _latchedLevels = new HashSet<int>();
|
||||
|
||||
private bool _hasState;
|
||||
private bool _lastOnAc;
|
||||
private int _emitted;
|
||||
|
||||
public PowerStateTransitionTracker(IReadOnlyList<int>? thresholdLadder = null, int maxEmissions = DefaultMaxEmissions)
|
||||
{
|
||||
_ladder = (thresholdLadder ?? DefaultThresholdLadder).OrderByDescending(l => l).ToArray();
|
||||
_maxEmissions = maxEmissions;
|
||||
}
|
||||
|
||||
/// <summary>True once the lifetime emission cap suppressed at least one emission.</summary>
|
||||
public bool EmissionCapReached { get; private set; }
|
||||
|
||||
/// <summary>
|
||||
/// Latches the arm-time state silently — except when the device is already on battery
|
||||
/// at/below a ladder level, which yields one threshold emission for the lowest crossed
|
||||
/// level (an enrollment STARTING at 12 % must still produce the event the analyze rule
|
||||
/// matches on). No AC-transition emission is ever produced from a baseline.
|
||||
/// </summary>
|
||||
public IReadOnlyList<PowerStateEmission> Baseline(PowerStateResult probe)
|
||||
{
|
||||
if (probe == null || probe.ProbeError != null || !probe.HasBattery)
|
||||
return Array.Empty<PowerStateEmission>();
|
||||
|
||||
_hasState = true;
|
||||
_lastOnAc = probe.OnAcPower;
|
||||
|
||||
var emissions = new List<PowerStateEmission>();
|
||||
EvaluateThresholds(probe, emissions);
|
||||
return emissions;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Diffs a fresh (host-debounced) snapshot against the last accepted state and returns
|
||||
/// 0..n emissions — an AC transition and a threshold crossing can co-occur (unplugged
|
||||
/// at 10 % ⇒ <c>ac_to_battery</c> + <c>threshold_crossed:15</c>).
|
||||
/// </summary>
|
||||
public IReadOnlyList<PowerStateEmission> Evaluate(PowerStateResult probe)
|
||||
{
|
||||
if (probe == null || probe.ProbeError != null)
|
||||
return Array.Empty<PowerStateEmission>();
|
||||
if (!_hasState)
|
||||
return Baseline(probe);
|
||||
|
||||
var emissions = new List<PowerStateEmission>();
|
||||
|
||||
if (_lastOnAc && !probe.OnAcPower)
|
||||
{
|
||||
Emit(emissions, new PowerStateEmission(
|
||||
transition: TransitionAcToBattery,
|
||||
thresholdPercent: null,
|
||||
probe: probe,
|
||||
severity: EventSeverity.Warning,
|
||||
immediateUpload: true,
|
||||
message: $"Power source switched from AC to battery ({FormatPercent(probe)} remaining) during enrollment"));
|
||||
}
|
||||
else if (!_lastOnAc && probe.OnAcPower)
|
||||
{
|
||||
Emit(emissions, new PowerStateEmission(
|
||||
transition: TransitionBatteryToAc,
|
||||
thresholdPercent: null,
|
||||
probe: probe,
|
||||
severity: EventSeverity.Info,
|
||||
immediateUpload: false,
|
||||
message: $"Power source switched back to AC ({FormatPercent(probe)}{(probe.IsCharging ? ", charging" : string.Empty)})"));
|
||||
}
|
||||
|
||||
_lastOnAc = probe.OnAcPower;
|
||||
EvaluateThresholds(probe, emissions);
|
||||
return emissions;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// While on battery with a known percentage: emit the LOWEST ladder level at/above the
|
||||
/// current charge that is not yet latched, then latch every level at/above the charge
|
||||
/// (a 60 %→10 % jump emits threshold 15 only, latching 50/30/15).
|
||||
/// </summary>
|
||||
private void EvaluateThresholds(PowerStateResult probe, List<PowerStateEmission> emissions)
|
||||
{
|
||||
if (probe.OnAcPower || !probe.BatteryPercent.HasValue) return;
|
||||
var percent = probe.BatteryPercent.Value;
|
||||
|
||||
int? lowestCrossed = null;
|
||||
foreach (var level in _ladder) // descending — the last match is the lowest crossed level
|
||||
{
|
||||
if (percent <= level && !_latchedLevels.Contains(level))
|
||||
lowestCrossed = level;
|
||||
}
|
||||
if (!lowestCrossed.HasValue) return;
|
||||
|
||||
foreach (var level in _ladder)
|
||||
{
|
||||
if (percent <= level) _latchedLevels.Add(level);
|
||||
}
|
||||
|
||||
var threshold = lowestCrossed.Value;
|
||||
var severity = threshold <= 15 ? EventSeverity.Error
|
||||
: threshold <= 30 ? EventSeverity.Warning
|
||||
: EventSeverity.Info;
|
||||
Emit(emissions, new PowerStateEmission(
|
||||
transition: TransitionThresholdCrossed,
|
||||
thresholdPercent: threshold,
|
||||
probe: probe,
|
||||
severity: severity,
|
||||
immediateUpload: severity >= EventSeverity.Warning,
|
||||
message: $"Battery dropped below {threshold}% ({FormatPercent(probe)} remaining) while enrolling on battery power"));
|
||||
}
|
||||
|
||||
private void Emit(List<PowerStateEmission> emissions, PowerStateEmission emission)
|
||||
{
|
||||
if (_emitted >= _maxEmissions)
|
||||
{
|
||||
EmissionCapReached = true;
|
||||
return;
|
||||
}
|
||||
_emitted++;
|
||||
emissions.Add(emission);
|
||||
}
|
||||
|
||||
private static string FormatPercent(PowerStateResult probe)
|
||||
=> probe.BatteryPercent.HasValue ? $"battery {probe.BatteryPercent.Value}%" : "battery level unknown";
|
||||
}
|
||||
|
||||
/// <summary>One decided <c>power_state_change</c> emission — the host maps this 1:1 onto an event.</summary>
|
||||
internal sealed class PowerStateEmission
|
||||
{
|
||||
public PowerStateEmission(string transition, int? thresholdPercent, PowerStateResult probe,
|
||||
EventSeverity severity, bool immediateUpload, string message)
|
||||
{
|
||||
Transition = transition;
|
||||
ThresholdPercent = thresholdPercent;
|
||||
OnAcPower = probe.OnAcPower;
|
||||
BatteryPercent = probe.BatteryPercent;
|
||||
IsCharging = probe.IsCharging;
|
||||
BatteryLifeMinutes = probe.BatteryLifeMinutes;
|
||||
Severity = severity;
|
||||
ImmediateUpload = immediateUpload;
|
||||
Message = message;
|
||||
}
|
||||
|
||||
public string Transition { get; }
|
||||
public int? ThresholdPercent { get; }
|
||||
public bool OnAcPower { get; }
|
||||
public int? BatteryPercent { get; }
|
||||
public bool IsCharging { get; }
|
||||
public int? BatteryLifeMinutes { get; }
|
||||
public EventSeverity Severity { get; }
|
||||
public bool ImmediateUpload { get; }
|
||||
public string Message { get; }
|
||||
}
|
||||
}
|
||||
+681
@@ -0,0 +1,681 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Diagnostics.Eventing.Reader;
|
||||
using System.Globalization;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Xml.Linq;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Orchestration;
|
||||
using AutopilotMonitor.Shared;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Newtonsoft.Json;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
|
||||
{
|
||||
/// <summary>
|
||||
/// Watches the <c>System</c> event log for two enrollment blind spots and forwards them as
|
||||
/// timeline ground truth:
|
||||
/// <list type="bullet">
|
||||
/// <item><b>System clock steps</b> (Kernel-General EventID 1) → <c>system_clock_changed</c>.
|
||||
/// The payload carries <c>oldTime</c>/<c>newTime</c>/<c>timeDeltaMs</c> plus the
|
||||
/// setting process — the "clock set from X to Y" fact that today has to be inferred
|
||||
/// from timestamp discontinuities. w32time also logs frequent 1 ms micro-slews on the
|
||||
/// same EventID, so emissions are gated on <see cref="MinClockDeltaMs"/>.</item>
|
||||
/// <item><b>Completed sleep episodes</b> → <c>system_sleep_episode</c>. Classic S3/S4 comes
|
||||
/// from Power-Troubleshooter EventID 1 (one event per episode at wake, carrying
|
||||
/// SleepTime + WakeTime). Modern Standby comes from Kernel-Power EventID 507 (exit),
|
||||
/// whose payload carries the scenario duration and the actual sleep residency; the
|
||||
/// enter time is derived as exit − duration. Kernel-Power 506 (enter) is deliberately
|
||||
/// NOT consumed: 507 already carries everything, and pairing 506/507 would need an
|
||||
/// orphan-tolerant state machine across agent restarts for zero information gain —
|
||||
/// an episode without a 507 never completed and would not be emitted under any design.</item>
|
||||
/// </list>
|
||||
/// <para>
|
||||
/// <b>Payload times are authoritative.</b> The decisive clock correction (w32time fixing a
|
||||
/// wrong BIOS clock) happens exactly when the device clock is untrustworthy, and the backend
|
||||
/// clamps implausible event timestamps to server time. All semantic instants therefore travel
|
||||
/// as explicit ISO-8601 payload fields, never only as the event's own <c>Timestamp</c>.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Backfill + cross-restart dedup</b> mirror <see cref="WindowsUpdateTracker"/>: the System
|
||||
/// .evtx persists across the OOBE reboots, so a startup scan recovers pre-agent clock steps
|
||||
/// and standby episodes; a persisted RecordId watermark plus an intra-run seen-set (a HashSet,
|
||||
/// NOT a high-water mark — see the field comment) keep restarts from re-emitting.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal sealed class SystemTimelineTracker : IDisposable
|
||||
{
|
||||
internal const string Channel = "System";
|
||||
|
||||
internal const string ProviderKernelGeneral = "Microsoft-Windows-Kernel-General"; // EventID 1: system time changed
|
||||
internal const string ProviderPowerTroubleshooter = "Microsoft-Windows-Power-Troubleshooter"; // EventID 1: resumed from classic sleep/hibernate
|
||||
internal const string ProviderKernelPower = "Microsoft-Windows-Kernel-Power"; // EventID 507: Modern Standby exit
|
||||
|
||||
internal const int EventId_ClockChange = 1;
|
||||
internal const int EventId_ClassicResume = 1;
|
||||
internal const int EventId_ModernStandbyExit = 507;
|
||||
|
||||
internal const string WatermarkStateFileName = "system-timeline-watermark.json";
|
||||
|
||||
// w32time logs Kernel-General 1 for routine 1 ms slew corrections (verified live: several
|
||||
// per hour). Only genuine steps are timeline-relevant; anything below this is claimed in
|
||||
// the dedup set but never emitted.
|
||||
internal const long MinClockDeltaMs = 2000;
|
||||
|
||||
// A step of at least 5 minutes can invalidate token/cert validation windows and visibly
|
||||
// reorders the timeline (matches the web's TimeJumpBadge threshold) — surface it fast.
|
||||
internal const long ClockDeltaWarningMs = 5 * 60 * 1000;
|
||||
|
||||
// Modern Standby logs a 506/507 pair for every screen-off scenario, including seconds-long
|
||||
// ones where the device never actually slept (SleepDurationInUs 0). Only episodes with
|
||||
// real sleep residency are emitted. Classic Power-Troubleshooter episodes have no floor:
|
||||
// they only exist for genuinely completed S3/S4 transitions.
|
||||
internal const long MinSleepDurationSeconds = 60;
|
||||
|
||||
private readonly AgentLogger _logger;
|
||||
private readonly string _sessionId;
|
||||
private readonly string _tenantId;
|
||||
private readonly InformationalEventPost _post;
|
||||
private readonly bool _backfillEnabled;
|
||||
private readonly int _backfillLookbackMinutes;
|
||||
private readonly string _stateDirectory;
|
||||
|
||||
private EventLogWatcher _watcher;
|
||||
|
||||
// Dedup. Guarded because EventRecordWritten callbacks run concurrently with the backfill scan.
|
||||
//
|
||||
// Cross-restart boundary loaded from disk: the highest RecordId emitted in a PRIOR run.
|
||||
// Immutable during this run — records at/below it were already emitted before this process
|
||||
// started and must not be re-emitted by the backfill scan.
|
||||
private long _restartWatermark = -1;
|
||||
// RecordIds claimed in THIS run — the intra-run dedup set. Deliberately a HashSet, NOT a
|
||||
// high-water mark: the live watcher is armed before the backfill runs, so a live record with
|
||||
// a HIGHER RecordId can be processed before/during backfill. A high-water mark would then
|
||||
// make the dedup skip every older, never-emitted backfill record — silently dropping exactly
|
||||
// the pre-agent clock corrections and standby episodes this tracker exists for. The targeted
|
||||
// records are low-volume, so the set stays tiny.
|
||||
private readonly HashSet<long> _seenThisRun = new HashSet<long>();
|
||||
// Highest RecordId claimed so far (max of restart watermark + this run). Persisted so the
|
||||
// next run's restart watermark is correct.
|
||||
private long _maxEmittedRecordId = -1;
|
||||
private readonly object _watermarkLock = new object();
|
||||
|
||||
public SystemTimelineTracker(
|
||||
string sessionId,
|
||||
string tenantId,
|
||||
InformationalEventPost post,
|
||||
AgentLogger logger,
|
||||
bool backfillEnabled = true,
|
||||
int backfillLookbackMinutes = 1440,
|
||||
string stateDirectory = null)
|
||||
{
|
||||
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
|
||||
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
|
||||
_post = post ?? throw new ArgumentNullException(nameof(post));
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
_backfillEnabled = backfillEnabled;
|
||||
_backfillLookbackMinutes = backfillLookbackMinutes;
|
||||
_stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null;
|
||||
}
|
||||
|
||||
public void Start()
|
||||
{
|
||||
LoadWatermark();
|
||||
StartWatcher();
|
||||
|
||||
if (_backfillEnabled && _backfillLookbackMinutes > 0)
|
||||
BackfillRecentEvents();
|
||||
else
|
||||
_logger.Info("SystemTimeline backfill disabled by config");
|
||||
}
|
||||
|
||||
public void Stop()
|
||||
{
|
||||
if (_watcher == null) return;
|
||||
try
|
||||
{
|
||||
_watcher.Enabled = false;
|
||||
_watcher.Dispose();
|
||||
_logger.Info("SystemTimeline watcher stopped");
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Error("Error stopping SystemTimeline watcher", ex);
|
||||
}
|
||||
finally
|
||||
{
|
||||
_watcher = null;
|
||||
}
|
||||
}
|
||||
|
||||
public void Dispose() => Stop();
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Watcher lifecycle
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private void StartWatcher()
|
||||
{
|
||||
try
|
||||
{
|
||||
var query = new EventLogQuery(Channel, PathType.LogName, BuildXPath());
|
||||
_watcher = new EventLogWatcher(query);
|
||||
_watcher.EventRecordWritten += OnEventRecordWritten;
|
||||
_watcher.Enabled = true;
|
||||
_logger.Info($"SystemTimeline watcher started: {Channel} (clock changes + sleep episodes)");
|
||||
}
|
||||
catch (EventLogNotFoundException)
|
||||
{
|
||||
_logger.Warning($"SystemTimeline event log not found: {Channel} (normal on non-Windows test environments)");
|
||||
}
|
||||
catch (UnauthorizedAccessException ex)
|
||||
{
|
||||
_logger.Warning($"SystemTimeline watcher access denied for {Channel}: {ex.Message}");
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Error($"Failed to start SystemTimeline watcher for {Channel}", ex);
|
||||
CollectorDegradationReporter.Report(_post, _sessionId, _tenantId,
|
||||
collectorName: "SystemTimelineTracker", reason: $"watcher_arm_failed:{Channel}", ex: ex);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Provider-qualified XPath for the three targeted signals. Provider filtering is mandatory
|
||||
/// on the System channel — EventID 1 alone matches many unrelated providers.
|
||||
/// Exposed as internal for tests.
|
||||
/// </summary>
|
||||
internal static string BuildXPath() =>
|
||||
"*[System[" + ProviderIdClauses() + "]]";
|
||||
|
||||
/// <summary>Backfill variant of <see cref="BuildXPath"/> with the lookback window applied.</summary>
|
||||
internal static string BuildBackfillXPath(long lookbackMs) =>
|
||||
"*[System[(" + ProviderIdClauses() + $") and TimeCreated[timediff(@SystemTime) <= {lookbackMs}]]]";
|
||||
|
||||
private static string ProviderIdClauses() =>
|
||||
$"(Provider[@Name='{ProviderKernelGeneral}'] and EventID={EventId_ClockChange})" +
|
||||
$" or (Provider[@Name='{ProviderPowerTroubleshooter}'] and EventID={EventId_ClassicResume})" +
|
||||
$" or (Provider[@Name='{ProviderKernelPower}'] and EventID={EventId_ModernStandbyExit})";
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Event processing
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private void OnEventRecordWritten(object sender, EventRecordWrittenEventArgs e)
|
||||
{
|
||||
if (e.EventRecord == null) return;
|
||||
try
|
||||
{
|
||||
ProcessRecord(e.EventRecord, isBackfill: false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Error("Error processing SystemTimeline event", ex);
|
||||
}
|
||||
}
|
||||
|
||||
private void ProcessRecord(EventRecord record, bool isBackfill)
|
||||
{
|
||||
var recordId = record.RecordId ?? -1;
|
||||
|
||||
// Cheap dedup gate BEFORE the XML rendering work.
|
||||
if (IsAlreadyProcessed(recordId))
|
||||
return;
|
||||
|
||||
string xml = null;
|
||||
try { xml = record.ToXml(); }
|
||||
catch { /* fall through with empty EventData */ }
|
||||
|
||||
ProcessEvent(
|
||||
providerName: record.ProviderName,
|
||||
eventId: record.Id,
|
||||
recordId: recordId,
|
||||
timeCreatedUtc: record.TimeCreated?.ToUniversalTime(),
|
||||
eventData: ParseEventData(xml),
|
||||
isBackfill: isBackfill);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Core processing extracted to primitive inputs so tests can drive it without synthesizing
|
||||
/// an abstract, Windows-only <see cref="EventRecord"/> (WindowsUpdateTracker test-seam
|
||||
/// pattern; a plain dictionary instead of named args because three providers with ~10
|
||||
/// payload fields each make a fixed signature impractical).
|
||||
/// </summary>
|
||||
internal void ProcessEvent(
|
||||
string providerName,
|
||||
int eventId,
|
||||
long recordId,
|
||||
DateTime? timeCreatedUtc,
|
||||
Dictionary<string, string> eventData,
|
||||
bool isBackfill)
|
||||
{
|
||||
if (!MarkProcessed(recordId))
|
||||
return; // already emitted (cross-restart or duplicate delivery)
|
||||
|
||||
eventData = eventData ?? new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
if (string.Equals(providerName, ProviderKernelGeneral, StringComparison.OrdinalIgnoreCase) && eventId == EventId_ClockChange)
|
||||
HandleClockChange(recordId, timeCreatedUtc, eventData, isBackfill);
|
||||
else if (string.Equals(providerName, ProviderPowerTroubleshooter, StringComparison.OrdinalIgnoreCase) && eventId == EventId_ClassicResume)
|
||||
HandleClassicResume(recordId, timeCreatedUtc, eventData, isBackfill);
|
||||
else if (string.Equals(providerName, ProviderKernelPower, StringComparison.OrdinalIgnoreCase) && eventId == EventId_ModernStandbyExit)
|
||||
HandleModernStandbyExit(recordId, timeCreatedUtc, eventData, isBackfill);
|
||||
// Anything else slipped past the XPath filter — ignore.
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// system_clock_changed (Kernel-General EventID 1)
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private void HandleClockChange(long recordId, DateTime? timeCreatedUtc, Dictionary<string, string> eventData, bool isBackfill)
|
||||
{
|
||||
eventData.TryGetValue("NewTime", out var newTimeRaw);
|
||||
eventData.TryGetValue("OldTime", out var oldTimeRaw);
|
||||
var newTime = TryParseUtc(newTimeRaw);
|
||||
var oldTime = TryParseUtc(oldTimeRaw);
|
||||
|
||||
// Signed delta preferably from the authoritative instants (NewTime − OldTime carries
|
||||
// the direction unambiguously); TimeDeltaInMs is the fallback when either fails to
|
||||
// parse. Records with neither are unusable and skipped (already claimed by dedup).
|
||||
long deltaMs;
|
||||
if (newTime.HasValue && oldTime.HasValue)
|
||||
{
|
||||
deltaMs = (long)Math.Round((newTime.Value - oldTime.Value).TotalMilliseconds);
|
||||
}
|
||||
else if (eventData.TryGetValue("TimeDeltaInMs", out var deltaRaw) &&
|
||||
long.TryParse(deltaRaw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var parsedDelta))
|
||||
{
|
||||
deltaMs = parsedDelta;
|
||||
}
|
||||
else
|
||||
{
|
||||
_logger.Debug($"SystemTimeline: clock-change record {recordId} has no usable times/delta — skipped");
|
||||
return;
|
||||
}
|
||||
|
||||
if (Math.Abs(deltaMs) < MinClockDeltaMs)
|
||||
return; // w32time micro-slew — claimed (MarkProcessed) but never emitted
|
||||
|
||||
eventData.TryGetValue("Reason", out var reasonRaw);
|
||||
int.TryParse(reasonRaw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var reason);
|
||||
var reasonText = reason == 1 ? "application_set"
|
||||
: reason == 2 ? "hardware_clock_sync"
|
||||
: "unknown";
|
||||
|
||||
eventData.TryGetValue("ProcessName", out var processName);
|
||||
|
||||
var data = new Dictionary<string, object>
|
||||
{
|
||||
{ "timeDeltaMs", deltaMs },
|
||||
{ "reason", reason },
|
||||
{ "reasonText", reasonText },
|
||||
{ "backfilled", isBackfill },
|
||||
};
|
||||
// Authoritative instants: re-normalized ISO-8601 when parseable, raw pass-through
|
||||
// otherwise — the payload must survive even a malformed provider rendering.
|
||||
data["newTime"] = newTime.HasValue ? newTime.Value.ToString("o") : (object)(newTimeRaw ?? string.Empty);
|
||||
data["oldTime"] = oldTime.HasValue ? oldTime.Value.ToString("o") : (object)(oldTimeRaw ?? string.Empty);
|
||||
if (recordId >= 0) data["recordId"] = recordId;
|
||||
if (timeCreatedUtc.HasValue) data["timeCreated"] = timeCreatedUtc.Value.ToString("o");
|
||||
if (!string.IsNullOrEmpty(processName)) data["processName"] = processName;
|
||||
if (eventData.TryGetValue("ProcessID", out var pidRaw) &&
|
||||
int.TryParse(pidRaw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var pid))
|
||||
{
|
||||
data["processId"] = pid;
|
||||
}
|
||||
|
||||
var direction = deltaMs >= 0 ? "forward" : "backward";
|
||||
var processLeaf = ProcessLeafName(processName);
|
||||
var message = $"System clock stepped {direction} {FormatDuration(TimeSpan.FromMilliseconds(Math.Abs(deltaMs)))}" +
|
||||
$" (reason: {reasonText}" + (processLeaf != null ? $", process: {processLeaf})" : ")");
|
||||
|
||||
var isLargeStep = Math.Abs(deltaMs) >= ClockDeltaWarningMs;
|
||||
_post.Emit(new EnrollmentEvent
|
||||
{
|
||||
SessionId = _sessionId,
|
||||
TenantId = _tenantId,
|
||||
// The record's own creation time, not UtcNow: backfilled pre-agent clock steps must
|
||||
// land on the timeline where they happened (WindowsUpdateTracker convention).
|
||||
Timestamp = timeCreatedUtc ?? DateTime.UtcNow,
|
||||
EventType = Constants.EventTypes.SystemClockChanged,
|
||||
Severity = isLargeStep ? EventSeverity.Warning : EventSeverity.Info,
|
||||
Source = "SystemTimelineWatcher",
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Message = message,
|
||||
Data = data,
|
||||
ImmediateUpload = isLargeStep,
|
||||
});
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// system_sleep_episode — classic sleep/hibernate (Power-Troubleshooter EventID 1)
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private void HandleClassicResume(long recordId, DateTime? timeCreatedUtc, Dictionary<string, string> eventData, bool isBackfill)
|
||||
{
|
||||
eventData.TryGetValue("SleepTime", out var sleepRaw);
|
||||
eventData.TryGetValue("WakeTime", out var wakeRaw);
|
||||
var enteredAt = TryParseUtc(sleepRaw);
|
||||
var exitedAt = TryParseUtc(wakeRaw);
|
||||
|
||||
// EffectiveState 5 = hibernate (S4); TargetState is the fallback when the effective
|
||||
// state is missing (older event versions).
|
||||
eventData.TryGetValue("EffectiveState", out var effectiveState);
|
||||
eventData.TryGetValue("TargetState", out var targetState);
|
||||
var stateValue = !string.IsNullOrEmpty(effectiveState) ? effectiveState : targetState;
|
||||
var kind = stateValue == "5" ? "hibernate" : "sleep";
|
||||
|
||||
long? durationSeconds = null;
|
||||
if (enteredAt.HasValue && exitedAt.HasValue && exitedAt.Value > enteredAt.Value)
|
||||
durationSeconds = (long)Math.Round((exitedAt.Value - enteredAt.Value).TotalSeconds);
|
||||
|
||||
var data = new Dictionary<string, object>
|
||||
{
|
||||
{ "kind", kind },
|
||||
{ "backfilled", isBackfill },
|
||||
};
|
||||
data["enteredAt"] = enteredAt.HasValue ? enteredAt.Value.ToString("o") : (object)(sleepRaw ?? string.Empty);
|
||||
data["exitedAt"] = exitedAt.HasValue ? exitedAt.Value.ToString("o") : (object)(wakeRaw ?? string.Empty);
|
||||
if (durationSeconds.HasValue) data["durationSeconds"] = durationSeconds.Value;
|
||||
if (recordId >= 0) data["recordId"] = recordId;
|
||||
if (timeCreatedUtc.HasValue) data["timeCreated"] = timeCreatedUtc.Value.ToString("o");
|
||||
if (eventData.TryGetValue("WakeSourceType", out var wakeTypeRaw) &&
|
||||
int.TryParse(wakeTypeRaw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var wakeType))
|
||||
{
|
||||
data["wakeSourceType"] = wakeType;
|
||||
}
|
||||
eventData.TryGetValue("WakeSourceText", out var wakeText);
|
||||
if (!string.IsNullOrEmpty(wakeText)) data["wakeSourceText"] = wakeText;
|
||||
|
||||
var durationLabel = durationSeconds.HasValue
|
||||
? " after " + FormatDuration(TimeSpan.FromSeconds(durationSeconds.Value))
|
||||
: string.Empty;
|
||||
var wakeLabel = !string.IsNullOrEmpty(wakeText) ? $" (wake source: {wakeText})" : string.Empty;
|
||||
|
||||
EmitSleepEpisode(timeCreatedUtc, data,
|
||||
$"Device resumed from {kind}{durationLabel}{wakeLabel}");
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// system_sleep_episode — Modern Standby (Kernel-Power EventID 507)
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private void HandleModernStandbyExit(long recordId, DateTime? timeCreatedUtc, Dictionary<string, string> eventData, bool isBackfill)
|
||||
{
|
||||
eventData.TryGetValue("SleepDurationInUs", out var sleepUsRaw);
|
||||
long.TryParse(sleepUsRaw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var sleepUs);
|
||||
var sleepDurationSeconds = sleepUs / 1_000_000;
|
||||
|
||||
// Every screen-off scenario logs a 507, including seconds-long ones where the device
|
||||
// never slept (SleepDurationInUs 0, incl. SleepEntered=false) — one gate covers both.
|
||||
if (sleepDurationSeconds < MinSleepDurationSeconds)
|
||||
return; // claimed but not an episode worth the timeline
|
||||
|
||||
eventData.TryGetValue("DurationInUs", out var durationUsRaw);
|
||||
long.TryParse(durationUsRaw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var durationUs);
|
||||
if (durationUs < sleepUs) durationUs = sleepUs; // scenario duration can never undercut its sleep residency
|
||||
|
||||
var durationSeconds = durationUs / 1_000_000;
|
||||
var exitedAt = timeCreatedUtc; // 507 is written at scenario exit
|
||||
DateTime? enteredAt = exitedAt.HasValue
|
||||
? exitedAt.Value.AddMilliseconds(-(durationUs / 1000.0))
|
||||
: (DateTime?)null;
|
||||
|
||||
var data = new Dictionary<string, object>
|
||||
{
|
||||
{ "kind", "modern_standby" },
|
||||
{ "durationSeconds", durationSeconds },
|
||||
{ "sleepDurationSeconds", sleepDurationSeconds },
|
||||
{ "backfilled", isBackfill },
|
||||
};
|
||||
if (enteredAt.HasValue) data["enteredAt"] = enteredAt.Value.ToString("o");
|
||||
if (exitedAt.HasValue) data["exitedAt"] = exitedAt.Value.ToString("o");
|
||||
if (recordId >= 0) data["recordId"] = recordId;
|
||||
if (timeCreatedUtc.HasValue) data["timeCreated"] = timeCreatedUtc.Value.ToString("o");
|
||||
if (eventData.TryGetValue("Reason", out var reasonRaw) &&
|
||||
int.TryParse(reasonRaw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var reason))
|
||||
{
|
||||
data["reason"] = reason;
|
||||
}
|
||||
bool? onAcPower = null;
|
||||
if (eventData.TryGetValue("PowerStateAc", out var acRaw) && bool.TryParse(acRaw, out var ac))
|
||||
{
|
||||
onAcPower = ac;
|
||||
data["onAcPower"] = ac;
|
||||
}
|
||||
if (eventData.TryGetValue("BatteryRemainingCapacityOnExit", out var battRaw) &&
|
||||
long.TryParse(battRaw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var batt))
|
||||
{
|
||||
data["batteryRemainingCapacityOnExit"] = batt;
|
||||
}
|
||||
|
||||
var powerLabel = onAcPower.HasValue ? (onAcPower.Value ? ", on AC" : ", on battery") : string.Empty;
|
||||
EmitSleepEpisode(timeCreatedUtc, data,
|
||||
$"Device exited Modern Standby after {FormatDuration(TimeSpan.FromSeconds(durationSeconds))}" +
|
||||
$" (slept {FormatDuration(TimeSpan.FromSeconds(sleepDurationSeconds))}{powerLabel})");
|
||||
}
|
||||
|
||||
private void EmitSleepEpisode(DateTime? timeCreatedUtc, Dictionary<string, object> data, string message)
|
||||
{
|
||||
_post.Emit(new EnrollmentEvent
|
||||
{
|
||||
SessionId = _sessionId,
|
||||
TenantId = _tenantId,
|
||||
// Wake/exit moment (the record's creation time) — backfilled pre-agent episodes
|
||||
// must land on the timeline where the device actually woke.
|
||||
Timestamp = timeCreatedUtc ?? DateTime.UtcNow,
|
||||
EventType = Constants.EventTypes.SystemSleepEpisode,
|
||||
Severity = EventSeverity.Info,
|
||||
Source = "SystemTimelineWatcher",
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Message = message,
|
||||
Data = data,
|
||||
ImmediateUpload = false,
|
||||
});
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Parsing helpers
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/// <summary>
|
||||
/// Extracts <c><Data Name="...">value</Data></c> pairs from an event's rendered
|
||||
/// XML, keyed case-insensitively. Namespace-agnostic (matches on local name). Returns an
|
||||
/// empty map on null/malformed XML — never throws.
|
||||
/// </summary>
|
||||
internal static Dictionary<string, string> ParseEventData(string xml)
|
||||
{
|
||||
var result = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
|
||||
if (string.IsNullOrEmpty(xml)) return result;
|
||||
|
||||
try
|
||||
{
|
||||
var doc = XDocument.Parse(xml);
|
||||
foreach (var el in doc.Descendants().Where(e => e.Name.LocalName == "Data"))
|
||||
{
|
||||
var nameAttr = el.Attribute("Name");
|
||||
if (nameAttr == null || string.IsNullOrEmpty(nameAttr.Value)) continue;
|
||||
if (!result.ContainsKey(nameAttr.Value))
|
||||
result[nameAttr.Value] = el.Value;
|
||||
}
|
||||
}
|
||||
catch
|
||||
{
|
||||
// Malformed / unexpected XML — best effort, return what we have.
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
internal static DateTime? TryParseUtc(string value)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(value)) return null;
|
||||
return DateTime.TryParse(value, CultureInfo.InvariantCulture,
|
||||
DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal, out var parsed)
|
||||
? parsed
|
||||
: (DateTime?)null;
|
||||
}
|
||||
|
||||
/// <summary>Compact human duration for messages: "2h 05m", "56m 12s", "45s".</summary>
|
||||
internal static string FormatDuration(TimeSpan span)
|
||||
{
|
||||
if (span.TotalHours >= 1)
|
||||
return $"{(long)span.TotalHours}h {span.Minutes:D2}m";
|
||||
if (span.TotalMinutes >= 1)
|
||||
return $"{span.Minutes}m {span.Seconds:D2}s";
|
||||
return $"{(long)Math.Ceiling(span.TotalSeconds)}s";
|
||||
}
|
||||
|
||||
/// <summary>Leaf file name of a kernel device path like <c>\Device\HarddiskVolume3\Windows\System32\svchost.exe</c>.</summary>
|
||||
internal static string ProcessLeafName(string devicePath)
|
||||
{
|
||||
if (string.IsNullOrEmpty(devicePath)) return null;
|
||||
var idx = devicePath.LastIndexOf('\\');
|
||||
var leaf = idx >= 0 ? devicePath.Substring(idx + 1) : devicePath;
|
||||
return string.IsNullOrEmpty(leaf) ? null : leaf;
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Backfill
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private void BackfillRecentEvents()
|
||||
{
|
||||
try
|
||||
{
|
||||
var lookbackMs = (long)_backfillLookbackMinutes * 60 * 1000;
|
||||
_logger.Info($"SystemTimeline backfill: scanning {Channel} " +
|
||||
$"(lookback={_backfillLookbackMinutes}min, restartWatermark={_restartWatermark})");
|
||||
|
||||
var query = new EventLogQuery(Channel, PathType.LogName, BuildBackfillXPath(lookbackMs));
|
||||
|
||||
int processed = 0;
|
||||
using (var reader = new EventLogReader(query))
|
||||
{
|
||||
EventRecord record;
|
||||
while ((record = reader.ReadEvent()) != null)
|
||||
{
|
||||
using (record)
|
||||
{
|
||||
ProcessRecord(record, isBackfill: true);
|
||||
processed++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (processed == 0)
|
||||
_logger.Debug($"SystemTimeline backfill: no targeted events in last {_backfillLookbackMinutes} minutes");
|
||||
else
|
||||
_logger.Info($"SystemTimeline backfill: scanned {processed} record(s)");
|
||||
}
|
||||
catch (EventLogNotFoundException)
|
||||
{
|
||||
_logger.Warning($"SystemTimeline event log not found during backfill: {Channel} (normal on non-Windows test environments)");
|
||||
}
|
||||
catch (UnauthorizedAccessException ex)
|
||||
{
|
||||
_logger.Warning($"SystemTimeline backfill access denied: {ex.Message}");
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Warning($"SystemTimeline backfill failed: {ex.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Watermark dedup (cross-restart)
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
private bool IsAlreadyProcessed(long recordId)
|
||||
{
|
||||
if (recordId < 0) return false; // no RecordId → cannot dedup, let it through
|
||||
lock (_watermarkLock)
|
||||
{
|
||||
return recordId <= _restartWatermark || _seenThisRun.Contains(recordId);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Atomically claims <paramref name="recordId"/>. Returns true if this is the first time we
|
||||
/// see it (caller should evaluate/emit), false if it was already processed — either in a
|
||||
/// PRIOR run (at/below the restart watermark) or already this run (in the seen-set). Only
|
||||
/// advances + persists the max on a genuinely-new, higher RecordId. Records without a
|
||||
/// RecordId (-1) are always processed, never tracked or persisted. Note: threshold-suppressed
|
||||
/// records are claimed too, so a restart's backfill does not re-evaluate them.
|
||||
/// </summary>
|
||||
private bool MarkProcessed(long recordId)
|
||||
{
|
||||
if (recordId < 0) return true;
|
||||
|
||||
long toPersist = -1;
|
||||
lock (_watermarkLock)
|
||||
{
|
||||
if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId))
|
||||
return false;
|
||||
|
||||
if (recordId > _maxEmittedRecordId)
|
||||
{
|
||||
_maxEmittedRecordId = recordId;
|
||||
toPersist = _maxEmittedRecordId;
|
||||
}
|
||||
}
|
||||
if (toPersist >= 0) PersistWatermark(toPersist);
|
||||
return true;
|
||||
}
|
||||
|
||||
internal void LoadWatermark()
|
||||
{
|
||||
if (string.IsNullOrEmpty(_stateDirectory)) return;
|
||||
|
||||
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
|
||||
if (!File.Exists(filePath)) return;
|
||||
|
||||
try
|
||||
{
|
||||
var json = File.ReadAllText(filePath);
|
||||
var state = JsonConvert.DeserializeObject<WatermarkState>(json);
|
||||
if (state != null)
|
||||
{
|
||||
lock (_watermarkLock)
|
||||
{
|
||||
_restartWatermark = state.LastRecordId;
|
||||
_maxEmittedRecordId = state.LastRecordId;
|
||||
}
|
||||
_logger.Info($"SystemTimeline watermark loaded: lastRecordId={state.LastRecordId} (persisted {state.PersistedUtc:O})");
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Warning($"Failed to load SystemTimeline watermark: {ex.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
private void PersistWatermark(long recordId)
|
||||
{
|
||||
if (string.IsNullOrEmpty(_stateDirectory)) return;
|
||||
|
||||
try
|
||||
{
|
||||
Directory.CreateDirectory(_stateDirectory);
|
||||
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
|
||||
var state = new WatermarkState { LastRecordId = recordId, PersistedUtc = DateTime.UtcNow };
|
||||
var json = JsonConvert.SerializeObject(state, Formatting.Indented);
|
||||
var tempPath = filePath + ".tmp";
|
||||
File.WriteAllText(tempPath, json);
|
||||
File.Copy(tempPath, filePath, overwrite: true);
|
||||
try { File.Delete(tempPath); } catch { }
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Warning($"Failed to persist SystemTimeline watermark: {ex.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
internal sealed class WatermarkState
|
||||
{
|
||||
public long LastRecordId { get; set; }
|
||||
public DateTime PersistedUtc { get; set; }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
#nullable enable
|
||||
using System;
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using System.Runtime.InteropServices;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Interop
|
||||
{
|
||||
/// <summary>
|
||||
/// ARP resolution via <c>iphlpapi!SendARP</c> — resolves the MAC address of an on-link IPv4
|
||||
/// host (in practice: the default gateway). IPv4 only by API design; the OS answers from its
|
||||
/// ARP cache or sends a real ARP request, so the target must be on the local subnet.
|
||||
/// </summary>
|
||||
internal static class ArpNativeMethods
|
||||
{
|
||||
[DllImport("iphlpapi.dll", ExactSpelling = true)]
|
||||
private static extern int SendARP(uint destIP, uint srcIP, byte[] macAddress, ref uint macAddressLength);
|
||||
|
||||
/// <summary>
|
||||
/// Fail-soft MAC resolution. Returns false (with a diagnostic in <paramref name="error"/>)
|
||||
/// for non-IPv4 addresses, SendARP failures, or unexpected reply lengths — never throws.
|
||||
/// </summary>
|
||||
internal static bool TryResolveMac(IPAddress? ipv4Address, out byte[] mac, out string? error)
|
||||
{
|
||||
mac = new byte[6];
|
||||
error = null;
|
||||
try
|
||||
{
|
||||
if (ipv4Address == null || ipv4Address.AddressFamily != AddressFamily.InterNetwork)
|
||||
{
|
||||
error = "not an IPv4 address";
|
||||
return false;
|
||||
}
|
||||
|
||||
// SendARP takes the IPv4 address as a uint in network byte order (first octet in
|
||||
// the lowest byte).
|
||||
var addressBytes = ipv4Address.GetAddressBytes();
|
||||
uint dest = ((uint)addressBytes[3] << 24)
|
||||
| ((uint)addressBytes[2] << 16)
|
||||
| ((uint)addressBytes[1] << 8)
|
||||
| addressBytes[0];
|
||||
|
||||
uint len = (uint)mac.Length;
|
||||
var err = SendARP(dest, 0, mac, ref len);
|
||||
if (err != 0)
|
||||
{
|
||||
error = $"SendARP returned {err}";
|
||||
return false;
|
||||
}
|
||||
if (len != 6)
|
||||
{
|
||||
error = $"SendARP returned unexpected MAC length {len}";
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
error = ex.Message;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
#nullable enable
|
||||
using System.Net;
|
||||
using System.Net.NetworkInformation;
|
||||
using System.Net.Sockets;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Monitoring
|
||||
{
|
||||
/// <summary>
|
||||
/// Shared "which NIC is the device actually using" heuristic: Up, not Loopback/Tunnel,
|
||||
/// carries a non-0.0.0.0 default gateway. Fail-soft — any enumeration error yields null.
|
||||
/// </summary>
|
||||
internal static class NetworkInterfaceLocator
|
||||
{
|
||||
/// <summary>
|
||||
/// Finds the active network interface: Up, not Loopback/Tunnel, has a non-0.0.0.0 gateway.
|
||||
/// </summary>
|
||||
internal static NetworkInterface? FindActiveNetworkInterface()
|
||||
{
|
||||
try
|
||||
{
|
||||
var interfaces = NetworkInterface.GetAllNetworkInterfaces();
|
||||
foreach (var nic in interfaces)
|
||||
{
|
||||
if (nic.OperationalStatus != OperationalStatus.Up)
|
||||
continue;
|
||||
if (nic.NetworkInterfaceType == NetworkInterfaceType.Loopback ||
|
||||
nic.NetworkInterfaceType == NetworkInterfaceType.Tunnel)
|
||||
continue;
|
||||
|
||||
var ipProps = nic.GetIPProperties();
|
||||
foreach (var gw in ipProps.GatewayAddresses)
|
||||
{
|
||||
if (gw.Address.ToString() != "0.0.0.0")
|
||||
return nic;
|
||||
}
|
||||
}
|
||||
}
|
||||
catch
|
||||
{
|
||||
// Caller handles null
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/// <summary>First IPv4 default gateway of the interface (0.0.0.0 excluded), or null.</summary>
|
||||
internal static IPAddress? GetIpv4Gateway(NetworkInterface nic)
|
||||
{
|
||||
try
|
||||
{
|
||||
foreach (var gw in nic.GetIPProperties().GatewayAddresses)
|
||||
{
|
||||
var address = gw?.Address;
|
||||
if (address != null
|
||||
&& address.AddressFamily == AddressFamily.InterNetwork
|
||||
&& address.ToString() != "0.0.0.0")
|
||||
{
|
||||
return address;
|
||||
}
|
||||
}
|
||||
}
|
||||
catch
|
||||
{
|
||||
// Caller handles null
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
+2
-30
@@ -922,36 +922,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.DeviceInfo
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Finds the active network interface: Up, not Loopback/Tunnel, has a non-0.0.0.0 gateway.
|
||||
/// </summary>
|
||||
private static NetworkInterface FindActiveNetworkInterface()
|
||||
{
|
||||
try
|
||||
{
|
||||
var interfaces = NetworkInterface.GetAllNetworkInterfaces();
|
||||
foreach (var nic in interfaces)
|
||||
{
|
||||
if (nic.OperationalStatus != OperationalStatus.Up)
|
||||
continue;
|
||||
if (nic.NetworkInterfaceType == NetworkInterfaceType.Loopback ||
|
||||
nic.NetworkInterfaceType == NetworkInterfaceType.Tunnel)
|
||||
continue;
|
||||
|
||||
var ipProps = nic.GetIPProperties();
|
||||
foreach (var gw in ipProps.GatewayAddresses)
|
||||
{
|
||||
if (gw.Address.ToString() != "0.0.0.0")
|
||||
return nic;
|
||||
}
|
||||
}
|
||||
}
|
||||
catch
|
||||
{
|
||||
// Caller handles null
|
||||
}
|
||||
return null;
|
||||
}
|
||||
private static NetworkInterface FindActiveNetworkInterface() =>
|
||||
Monitoring.NetworkInterfaceLocator.FindActiveNetworkInterface();
|
||||
|
||||
/// <summary>
|
||||
/// Collects WiFi signal info via the native WLAN API (<see cref="WifiInfoReader"/>,
|
||||
|
||||
+2
-30
@@ -420,36 +420,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
|
||||
// Helpers
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
/// <summary>
|
||||
/// Finds the active network interface: Up, not Loopback/Tunnel, has a non-0.0.0.0 gateway.
|
||||
/// </summary>
|
||||
private static NetworkInterface FindActiveNetworkInterface()
|
||||
{
|
||||
try
|
||||
{
|
||||
var interfaces = NetworkInterface.GetAllNetworkInterfaces();
|
||||
foreach (var nic in interfaces)
|
||||
{
|
||||
if (nic.OperationalStatus != OperationalStatus.Up)
|
||||
continue;
|
||||
if (nic.NetworkInterfaceType == NetworkInterfaceType.Loopback ||
|
||||
nic.NetworkInterfaceType == NetworkInterfaceType.Tunnel)
|
||||
continue;
|
||||
|
||||
var ipProps = nic.GetIPProperties();
|
||||
foreach (var gw in ipProps.GatewayAddresses)
|
||||
{
|
||||
if (gw.Address.ToString() != "0.0.0.0")
|
||||
return nic;
|
||||
}
|
||||
}
|
||||
}
|
||||
catch
|
||||
{
|
||||
// Caller handles null
|
||||
}
|
||||
return null;
|
||||
}
|
||||
private static NetworkInterface FindActiveNetworkInterface() =>
|
||||
NetworkInterfaceLocator.FindActiveNetworkInterface();
|
||||
|
||||
private static string BuildChangeMessage(NetworkStateSnapshot before, NetworkStateSnapshot after)
|
||||
{
|
||||
|
||||
@@ -0,0 +1,217 @@
|
||||
#nullable enable
|
||||
using System;
|
||||
using System.Net;
|
||||
using System.Net.Sockets;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic;
|
||||
using Microsoft.Win32;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Transport
|
||||
{
|
||||
public class DoGroupIdSetResult
|
||||
{
|
||||
/// <summary>The DOGroupId value is the computed one after this run (written or already there).</summary>
|
||||
public bool Success { get; set; }
|
||||
|
||||
/// <summary>True when this run actually wrote the registry value (false for the idempotent no-op).</summary>
|
||||
public bool Written { get; set; }
|
||||
|
||||
/// <summary>Deliberate non-action (existing foreign policy/value, no usable network) — not an error.</summary>
|
||||
public bool Skipped { get; set; }
|
||||
public string? SkipReason { get; set; }
|
||||
|
||||
public string? GroupId { get; set; }
|
||||
public string? PreviousValue { get; set; }
|
||||
public string? GatewayIp { get; set; }
|
||||
public string? GatewayMac { get; set; }
|
||||
public string? Error { get; set; }
|
||||
}
|
||||
|
||||
internal enum DoGroupIdAction
|
||||
{
|
||||
Write,
|
||||
NoOp,
|
||||
SkipForeign,
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Sets the Delivery Optimization group ID policy value (<c>DOGroupId</c> under the GPO
|
||||
/// DeliveryOptimization key) from a network fingerprint: a deterministic GUID derived from the
|
||||
/// default gateway's IPv4 address and MAC address, so all devices behind the same gateway land
|
||||
/// in the same DO peering group. The byte layout is RealmJoin-compatible — devices managed by
|
||||
/// RealmJoin on the same network compute the identical GUID and peer with ours.
|
||||
/// <para>
|
||||
/// Deliberately conservative where RealmJoin is not: no random-GUID fallback (a per-device
|
||||
/// random ID would isolate the device in a one-member group — worse than the OS default
|
||||
/// grouping), and existing foreign configuration always wins: a DOGroupIdSource policy, an
|
||||
/// MDM-managed DOGroupId, or a GPO value we did not write ourselves all result in a skip.
|
||||
/// Never throws.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public static class DoGroupIdService
|
||||
{
|
||||
internal const string GroupIdValueName = "DOGroupId";
|
||||
internal const string GroupIdSourceValueName = "DOGroupIdSource";
|
||||
|
||||
// RealmJoin's fingerprint-GUID prefix — part of the compatible byte layout, do not change.
|
||||
private static readonly byte[] GuidPrefix = { 101, 48, 6 };
|
||||
|
||||
/// <summary>
|
||||
/// Computes and applies the fingerprint GroupId. <paramref name="lastWrittenGroupId"/> is
|
||||
/// the GUID a previous agent run wrote (null when none is recorded) — it lets us update our
|
||||
/// own value after a gateway change while never touching a foreign one.
|
||||
/// </summary>
|
||||
public static DoGroupIdSetResult TrySetGroupId(string? lastWrittenGroupId, AgentLogger logger)
|
||||
{
|
||||
var result = new DoGroupIdSetResult();
|
||||
|
||||
try
|
||||
{
|
||||
// ---- Network fingerprint -------------------------------------------------
|
||||
var nic = NetworkInterfaceLocator.FindActiveNetworkInterface();
|
||||
var gateway = nic == null ? null : NetworkInterfaceLocator.GetIpv4Gateway(nic);
|
||||
if (gateway == null)
|
||||
{
|
||||
result.Skipped = true;
|
||||
result.SkipReason = "no_ipv4_gateway";
|
||||
logger.Info("DO GroupId auto-set: no active interface with an IPv4 default gateway — skipping.");
|
||||
return result;
|
||||
}
|
||||
result.GatewayIp = gateway.ToString();
|
||||
|
||||
if (!ArpNativeMethods.TryResolveMac(gateway, out var mac, out var arpError))
|
||||
{
|
||||
result.Skipped = true;
|
||||
result.SkipReason = "arp_failed";
|
||||
result.Error = arpError;
|
||||
logger.Warning($"DO GroupId auto-set: could not resolve gateway MAC ({arpError}) — skipping.");
|
||||
return result;
|
||||
}
|
||||
result.GatewayMac = BitConverter.ToString(mac);
|
||||
|
||||
var computed = ComputeGroupId(gateway.GetAddressBytes(), mac).ToString();
|
||||
result.GroupId = computed;
|
||||
|
||||
// ---- Foreign-configuration checks ----------------------------------------
|
||||
// DOGroupIdSource (either store) selects a dynamic group source — a fingerprint
|
||||
// GroupId would fight it. An MDM-managed DOGroupId means Intune owns the setting.
|
||||
if (ReadPolicyValue(DoThrottlePolicyReader.GpoKeyPath, GroupIdSourceValueName) != null
|
||||
|| ReadPolicyValue(DoThrottlePolicyReader.MdmKeyPath, GroupIdSourceValueName) != null)
|
||||
{
|
||||
result.Skipped = true;
|
||||
result.SkipReason = "group_id_source_policy";
|
||||
logger.Info("DO GroupId auto-set: DOGroupIdSource policy is configured — skipping.");
|
||||
return result;
|
||||
}
|
||||
if (ReadPolicyValue(DoThrottlePolicyReader.MdmKeyPath, GroupIdValueName) != null)
|
||||
{
|
||||
result.Skipped = true;
|
||||
result.SkipReason = "mdm_policy_present";
|
||||
logger.Info("DO GroupId auto-set: DOGroupId is managed via MDM policy — skipping.");
|
||||
return result;
|
||||
}
|
||||
|
||||
// ---- Ownership decision ---------------------------------------------------
|
||||
var existing = ReadPolicyValue(DoThrottlePolicyReader.GpoKeyPath, GroupIdValueName) as string;
|
||||
result.PreviousValue = existing;
|
||||
|
||||
switch (DecideAction(existing, computed, lastWrittenGroupId))
|
||||
{
|
||||
case DoGroupIdAction.NoOp:
|
||||
result.Success = true;
|
||||
logger.Info($"DO GroupId auto-set: already set to {computed} — nothing to do.");
|
||||
return result;
|
||||
|
||||
case DoGroupIdAction.SkipForeign:
|
||||
result.Skipped = true;
|
||||
result.SkipReason = "foreign_value_present";
|
||||
logger.Info($"DO GroupId auto-set: existing DOGroupId '{existing}' was not written by us — skipping.");
|
||||
return result;
|
||||
}
|
||||
|
||||
// ---- Write + readback -----------------------------------------------------
|
||||
using (var hklm = RegistryKey.OpenBaseKey(RegistryHive.LocalMachine, RegistryView.Registry64))
|
||||
using (var key = hklm.CreateSubKey(DoThrottlePolicyReader.GpoKeyPath))
|
||||
{
|
||||
key.SetValue(GroupIdValueName, computed, RegistryValueKind.String);
|
||||
}
|
||||
|
||||
var readback = ReadPolicyValue(DoThrottlePolicyReader.GpoKeyPath, GroupIdValueName) as string;
|
||||
if (!string.Equals(readback, computed, StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
result.Error = $"readback mismatch: wrote '{computed}', read '{readback ?? "<null>"}'";
|
||||
logger.Warning($"DO GroupId auto-set failed: {result.Error}");
|
||||
return result;
|
||||
}
|
||||
|
||||
result.Success = true;
|
||||
result.Written = true;
|
||||
logger.Info($"DO GroupId auto-set: DOGroupId set to {computed} (gateway {result.GatewayIp}, previous: {existing ?? "<none>"})");
|
||||
return result;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
result.Error = ex.Message;
|
||||
logger.Warning($"DO GroupId auto-set failed: {ex.Message}");
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// RealmJoin-compatible fingerprint GUID: 16 bytes laid out as
|
||||
/// prefix {101,48,6} + {0,0,0} + {ip[1],0,ip[2],ip[3]} + mac[6], passed to
|
||||
/// <c>new Guid(byte[])</c>. The IP byte shuffle mirrors RealmJoin's "swap for
|
||||
/// new Guid()" step — the layout must stay bit-identical for cross-product peering.
|
||||
/// </summary>
|
||||
internal static Guid ComputeGroupId(byte[] ipv4Bytes, byte[] macBytes)
|
||||
{
|
||||
if (ipv4Bytes == null || ipv4Bytes.Length != 4)
|
||||
throw new ArgumentException("IPv4 address bytes required", nameof(ipv4Bytes));
|
||||
if (macBytes == null || macBytes.Length != 6)
|
||||
throw new ArgumentException("6-byte MAC required", nameof(macBytes));
|
||||
|
||||
var bytes = new byte[16];
|
||||
Array.Copy(GuidPrefix, 0, bytes, 0, GuidPrefix.Length);
|
||||
// bytes[3..5] stay zero (RealmJoin's zero padding)
|
||||
bytes[6] = ipv4Bytes[1];
|
||||
// bytes[7] stays zero
|
||||
bytes[8] = ipv4Bytes[2];
|
||||
bytes[9] = ipv4Bytes[3];
|
||||
Array.Copy(macBytes, 0, bytes, 10, macBytes.Length);
|
||||
return new Guid(bytes);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Pure ownership decision. GUID comparison tolerates case and braces so a value written
|
||||
/// by another tool in "{...}" form still matches semantically.
|
||||
/// </summary>
|
||||
internal static DoGroupIdAction DecideAction(string? existing, string computed, string? lastWrittenByUs)
|
||||
{
|
||||
if (string.IsNullOrEmpty(existing))
|
||||
return DoGroupIdAction.Write;
|
||||
if (GuidEquals(existing, computed))
|
||||
return DoGroupIdAction.NoOp;
|
||||
if (GuidEquals(existing, lastWrittenByUs))
|
||||
return DoGroupIdAction.Write; // our own stale value — gateway changed
|
||||
return DoGroupIdAction.SkipForeign;
|
||||
}
|
||||
|
||||
private static bool GuidEquals(string? a, string? b)
|
||||
{
|
||||
if (string.IsNullOrEmpty(a) || string.IsNullOrEmpty(b)) return false;
|
||||
return Guid.TryParse(a, out var ga) && Guid.TryParse(b, out var gb) && ga == gb;
|
||||
}
|
||||
|
||||
private static object? ReadPolicyValue(string keyPath, string valueName)
|
||||
{
|
||||
// Registry64 explicitly — SOFTWARE\Policies is WOW64-redirected, and the policy
|
||||
// stores live in the 64-bit view regardless of the agent's bitness.
|
||||
using (var hklm = RegistryKey.OpenBaseKey(RegistryHive.LocalMachine, RegistryView.Registry64))
|
||||
using (var key = hklm.OpenSubKey(keyPath))
|
||||
{
|
||||
return key?.GetValue(valueName);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -557,6 +557,22 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
stateDirectory: _stateDirectory));
|
||||
}
|
||||
|
||||
// System timeline watcher — System channel: Kernel-General 1 (system clock stepped,
|
||||
// ground truth for time jumps incl. old/new time + setting process) and completed
|
||||
// sleep/standby episodes (Power-Troubleshooter 1, Kernel-Power 507). Backfill recovers
|
||||
// the pre-agent OOBE clock correction and standby episodes from the persistent .evtx.
|
||||
if (collectors.EnableSystemTimelineWatcher)
|
||||
{
|
||||
hosts.Add(new SystemTimelineWatcherHost(
|
||||
sessionId: sessionId,
|
||||
tenantId: tenantId,
|
||||
logger: logger,
|
||||
ingress: ingress,
|
||||
clock: clock,
|
||||
backfillLookbackMinutes: collectors.SystemEventBackfillLookbackMinutes,
|
||||
stateDirectory: _stateDirectory));
|
||||
}
|
||||
|
||||
// ----- Peripheral hosts (event-only; driven by remote-config toggles) --------------
|
||||
|
||||
// V1 parity (PeriodicCollectorManager) — combine Performance + AgentSelfMetrics under
|
||||
@@ -595,6 +611,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
logger: logger,
|
||||
apiBaseUrl: _agentConfig.ApiBaseUrl));
|
||||
|
||||
// Live AC/battery watcher — WMI Win32_PowerManagementEvent push (no polling), emits
|
||||
// power_state_change on AC↔battery transitions and 50/30/15 % downward threshold
|
||||
// crossings. Deliberately NOT under PeriodicCollectorLifecycleHost: it is push-based
|
||||
// (zero idle cost) and must stay alive precisely when the device sits idle on battery.
|
||||
// Never arms on devices without a battery. Kill-switchable via remote config.
|
||||
if (collectors.EnablePowerStateWatcher)
|
||||
{
|
||||
hosts.Add(new PowerStateWatcherHost(
|
||||
sessionId: sessionId,
|
||||
tenantId: tenantId,
|
||||
ingress: ingress,
|
||||
clock: clock,
|
||||
logger: logger));
|
||||
}
|
||||
|
||||
// Microsoft 365 Apps (Office C2R) install detector — event-driven (Rev 2): woken by a WMI
|
||||
// Win32_ProcessStartTrace push on OfficeC2RClient.exe (+ startup probe), progress via
|
||||
// RegNotifyChangeKeyValue, stop via Process.Exited. No idle polling. Constructed first so
|
||||
|
||||
+332
@@ -0,0 +1,332 @@
|
||||
#nullable enable
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Management;
|
||||
using System.Threading;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
|
||||
using AutopilotMonitor.Agent.V2.Core.Runtime;
|
||||
using AutopilotMonitor.DecisionCore.Engine;
|
||||
using AutopilotMonitor.Shared;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
{
|
||||
/// <summary>
|
||||
/// Live AC/battery watcher: subscribes to the WMI push event
|
||||
/// <c>Win32_PowerManagementEvent WHERE EventCode = 10</c> (PBT_APMPOWERSTATUSCHANGE — fires on
|
||||
/// AC↔DC switches and battery-percentage steps), falling back to the intrinsic
|
||||
/// <c>__InstanceModificationEvent WITHIN 30 … Win32_Battery</c> where the legacy push provider
|
||||
/// no longer loads (Windows 11 24H2/25H2, builds 26200+: activation fails WBEM_E_NOT_FOUND
|
||||
/// although the provider is still registered — session 161b838c). Either way the arriving
|
||||
/// event's payload is ignored; each arrival re-probes via <see cref="PowerStateProbe"/> and
|
||||
/// diffs the snapshots through the pure <see cref="PowerStateTransitionTracker"/> into
|
||||
/// <c>power_state_change</c> events. No polling in this process (the fallback's WITHIN
|
||||
/// sampling runs inside WinMgmt). Complements the one-shot startup
|
||||
/// <c>power_state_check</c> (<c>StartupEnvironmentProbes</c>), which stays the baseline event.
|
||||
/// <para>
|
||||
/// Devices without a battery (or with a failing probe) never arm the WMI subscription — zero
|
||||
/// overhead on desktops/VMs. WMI arming runs on a background thread (WinMgmt can hang in OOBE;
|
||||
/// same rationale as <c>ConsoleBypassWatcher</c>), and an arm failure is surfaced once as
|
||||
/// <c>collector_degraded</c>. A 5 s trailing-edge debounce collapses docking flaps to a
|
||||
/// no-diff; the tracker's lifetime emission cap is the storm backstop.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal sealed class PowerStateWatcherHost : ICollectorHost
|
||||
{
|
||||
public string Name => "PowerStateWatcher";
|
||||
|
||||
private static readonly TimeSpan DebounceDelay = TimeSpan.FromSeconds(5);
|
||||
|
||||
private readonly string _sessionId;
|
||||
private readonly string _tenantId;
|
||||
private readonly AgentLogger _logger;
|
||||
private readonly InformationalEventPost _post;
|
||||
private readonly Func<PowerStateResult> _probe;
|
||||
private readonly PowerStateTransitionTracker _tracker = new PowerStateTransitionTracker();
|
||||
private readonly object _lock = new object();
|
||||
private readonly object _tickLock = new object();
|
||||
|
||||
private ManagementEventWatcher? _watcher;
|
||||
private Timer? _debounceTimer;
|
||||
private bool _started;
|
||||
private bool _disposed;
|
||||
private bool _capWarned;
|
||||
|
||||
/// <param name="probe">Power-state snapshot source; test seam, defaults to
|
||||
/// <see cref="PowerStateProbe.Probe"/>. The ctor never calls it (the factory constructs
|
||||
/// hosts eagerly; all probing happens in <see cref="Start"/>'s background thread).</param>
|
||||
public PowerStateWatcherHost(
|
||||
string sessionId,
|
||||
string tenantId,
|
||||
ISignalIngressSink ingress,
|
||||
IClock clock,
|
||||
AgentLogger logger,
|
||||
Func<PowerStateResult>? probe = null)
|
||||
{
|
||||
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
|
||||
if (clock == null) throw new ArgumentNullException(nameof(clock));
|
||||
_sessionId = sessionId;
|
||||
_tenantId = tenantId;
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
_post = new InformationalEventPost(ingress, clock, logger);
|
||||
_probe = probe ?? PowerStateProbe.Probe;
|
||||
}
|
||||
|
||||
public void Start()
|
||||
{
|
||||
lock (_lock)
|
||||
{
|
||||
if (_started || _disposed) return;
|
||||
_started = true;
|
||||
}
|
||||
|
||||
// Baseline probe + WMI arming are synchronous WinMgmt-adjacent operations and WinMgmt
|
||||
// can hang during OOBE — never run them inline on the orchestrator's sequential
|
||||
// host-start loop (ConsoleBypassWatcher precedent). Disposal racing the background
|
||||
// start is handled by the _disposed re-checks inside StartCore.
|
||||
System.Threading.Tasks.Task.Run(() => StartCore(armWmi: true));
|
||||
}
|
||||
|
||||
/// <summary>Internal for tests: the synchronous start body; tests pass
|
||||
/// <paramref name="armWmi"/> = false so no real WMI subscription is created
|
||||
/// (precedent: ConsoleBypassWatcher leaves WMI arming untested).</summary>
|
||||
internal void StartCore(bool armWmi)
|
||||
{
|
||||
PowerStateResult baseline;
|
||||
try
|
||||
{
|
||||
baseline = _probe();
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Warning($"[{Name}] baseline probe threw ({ex.Message}) — power-state watcher not armed");
|
||||
return;
|
||||
}
|
||||
|
||||
if (baseline.ProbeError != null)
|
||||
{
|
||||
_logger.Info($"[{Name}] power probe failed ({baseline.ProbeError}) — watcher not armed");
|
||||
return;
|
||||
}
|
||||
if (!baseline.HasBattery)
|
||||
{
|
||||
_logger.Info($"[{Name}] no battery present (desktop/VM) — watcher not armed");
|
||||
return;
|
||||
}
|
||||
|
||||
// Latch the baseline BEFORE arming so the first WMI push diffs against a real state.
|
||||
// A session already on battery below a ladder level emits its threshold event here.
|
||||
EmitAll(_tracker.Baseline(baseline));
|
||||
|
||||
if (!armWmi) return;
|
||||
|
||||
// Arm chain: the extrinsic push event first (zero WinMgmt cost, still alive on older
|
||||
// builds), then the intrinsic Win32_Battery modification event. On Windows 11 24H2/25H2
|
||||
// (observed on builds 26200/26220, session 161b838c) the MS_Power_Management_Event_Provider
|
||||
// is still REGISTERED but its activation fails with WBEM_E_NOT_FOUND — the legacy provider
|
||||
// no longer loads, so the push query throws on exactly the devices that enroll today.
|
||||
// The fallback's WITHIN polling runs inside WinMgmt (a Win32_Battery snapshot every 30 s),
|
||||
// NOT in this process; the handler path is identical (payload ignored, re-probe + diff),
|
||||
// worst-case detection latency ≈ 35 s incl. debounce.
|
||||
if (TryArmWatcher(
|
||||
"SELECT * FROM Win32_PowerManagementEvent WHERE EventCode = 10",
|
||||
"power_management_event (push)", baseline, out var pushError))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (TryArmWatcher(
|
||||
"SELECT * FROM __InstanceModificationEvent WITHIN 30 WHERE TargetInstance ISA 'Win32_Battery'",
|
||||
"battery_instance_poll (WinMgmt WITHIN 30s)", baseline, out var fallbackError))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
_logger.Warning($"[{Name}] could not arm any WMI power watcher — " +
|
||||
$"push: {Describe(pushError)}; fallback: {Describe(fallbackError)} — " +
|
||||
"live power-state transitions will not be observed this run");
|
||||
CollectorDegradationReporter.Report(_post, _sessionId, _tenantId, Name, "watcher_arm_failed",
|
||||
fallbackError ?? pushError);
|
||||
}
|
||||
|
||||
private static string Describe(Exception? ex)
|
||||
=> ex == null ? "n/a" : $"{ex.GetType().Name}: {ex.Message}";
|
||||
|
||||
/// <summary>
|
||||
/// One arm attempt for the given WQL event query, with the full dispose-race dance
|
||||
/// (ConsoleBypassWatcher pattern). Returns true when the chain should stop: the watcher
|
||||
/// armed, or disposal won the race (then arming anything would leak a subscription).
|
||||
/// Returns false with the captured exception so the caller can try the next query.
|
||||
/// </summary>
|
||||
private bool TryArmWatcher(string wql, string modeLabel, PowerStateResult baseline, out Exception? error)
|
||||
{
|
||||
error = null;
|
||||
try
|
||||
{
|
||||
var watcher = new ManagementEventWatcher(new WqlEventQuery(wql));
|
||||
watcher.EventArrived += OnPowerStatusChange;
|
||||
|
||||
lock (_lock)
|
||||
{
|
||||
if (_disposed)
|
||||
{
|
||||
// Dispose won the race against the background start — don't arm.
|
||||
try { watcher.EventArrived -= OnPowerStatusChange; } catch { }
|
||||
try { watcher.Dispose(); } catch { }
|
||||
return true;
|
||||
}
|
||||
_watcher = watcher;
|
||||
}
|
||||
|
||||
watcher.Start();
|
||||
|
||||
// Re-check after arming: Dispose() may have run between the lock release above and
|
||||
// Start() — it then stopped a not-yet-armed watcher and this thread armed it right
|
||||
// back. If disposal won that race, tear the watcher down here.
|
||||
bool disposedDuringArm;
|
||||
lock (_lock)
|
||||
{
|
||||
disposedDuringArm = _disposed;
|
||||
if (disposedDuringArm) _watcher = null;
|
||||
}
|
||||
if (disposedDuringArm)
|
||||
{
|
||||
try { watcher.EventArrived -= OnPowerStatusChange; } catch { }
|
||||
try { watcher.Stop(); } catch { }
|
||||
try { watcher.Dispose(); } catch { }
|
||||
return true;
|
||||
}
|
||||
|
||||
_logger.Info($"[{Name}] armed via {modeLabel} — " +
|
||||
$"baseline: {(baseline.OnAcPower ? "AC" : "battery")}, {baseline.BatteryPercent?.ToString() ?? "?"}%");
|
||||
return true;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
error = ex;
|
||||
_logger.Info($"[{Name}] arm attempt '{modeLabel}' failed ({ex.GetType().Name}: {ex.Message})");
|
||||
lock (_lock) { _watcher = null; }
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
private void OnPowerStatusChange(object sender, EventArrivedEventArgs e)
|
||||
{
|
||||
// Push and fallback events are treated identically: the payload is ignored, the event
|
||||
// only tells us "power status may have changed". Debounce, then re-probe — never call
|
||||
// watcher.Stop() from inside this callback.
|
||||
try { ScheduleDebouncedTick(); }
|
||||
catch (Exception ex) { _logger.Debug($"[{Name}] power event handler error: {ex.Message}"); }
|
||||
}
|
||||
|
||||
private void ScheduleDebouncedTick()
|
||||
{
|
||||
lock (_lock) { if (_disposed) return; }
|
||||
// One trailing-edge tick per change burst (ImeRegistryAppStateHost pattern): a
|
||||
// dock-flap AC→battery→AC inside the window collapses to a no-diff in the tracker.
|
||||
var timer = _debounceTimer;
|
||||
if (timer == null)
|
||||
{
|
||||
var created = new Timer(_ => DebouncedTick(), null, DebounceDelay, Timeout.InfiniteTimeSpan);
|
||||
if (Interlocked.CompareExchange(ref _debounceTimer, created, null) != null)
|
||||
created.Dispose();
|
||||
else
|
||||
return;
|
||||
timer = _debounceTimer;
|
||||
}
|
||||
try { timer?.Change(DebounceDelay, Timeout.InfiniteTimeSpan); }
|
||||
catch (ObjectDisposedException) { /* stopping */ }
|
||||
}
|
||||
|
||||
/// <summary>Internal for tests: runs one debounced probe/diff/emit cycle synchronously,
|
||||
/// exactly what the trailing-edge timer callback executes after a WMI push.</summary>
|
||||
internal void TickForTest() => DebouncedTick();
|
||||
|
||||
private void DebouncedTick()
|
||||
{
|
||||
lock (_lock) { if (_disposed) return; }
|
||||
try
|
||||
{
|
||||
IReadOnlyList<PowerStateEmission> emissions;
|
||||
lock (_tickLock) // tracker is not thread-safe; ticks must not interleave
|
||||
{
|
||||
emissions = _tracker.Evaluate(_probe());
|
||||
}
|
||||
EmitAll(emissions);
|
||||
|
||||
if (_tracker.EmissionCapReached && !_capWarned)
|
||||
{
|
||||
_capWarned = true;
|
||||
_logger.Warning($"[{Name}] emission cap reached — further power_state_change events " +
|
||||
"suppressed this run (flapping power source or dying battery controller)");
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Debug($"[{Name}] debounced power tick failed: {ex.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
private void EmitAll(IReadOnlyList<PowerStateEmission> emissions)
|
||||
{
|
||||
foreach (var emission in emissions)
|
||||
{
|
||||
try
|
||||
{
|
||||
var data = new Dictionary<string, object>
|
||||
{
|
||||
{ "transition", emission.Transition },
|
||||
{ "onAcPower", emission.OnAcPower },
|
||||
{ "batteryPercent", emission.BatteryPercent.HasValue ? (object)emission.BatteryPercent.Value : "unknown" },
|
||||
{ "isCharging", emission.IsCharging },
|
||||
{ "batteryLifeMinutes", emission.BatteryLifeMinutes.HasValue ? (object)emission.BatteryLifeMinutes.Value : "unknown" },
|
||||
};
|
||||
if (emission.ThresholdPercent.HasValue)
|
||||
data["thresholdPercent"] = emission.ThresholdPercent.Value;
|
||||
|
||||
_post.Emit(new EnrollmentEvent
|
||||
{
|
||||
SessionId = _sessionId,
|
||||
TenantId = _tenantId,
|
||||
EventType = Constants.EventTypes.PowerStateChange,
|
||||
Severity = emission.Severity,
|
||||
Source = Name,
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Message = emission.Message,
|
||||
ImmediateUpload = emission.ImmediateUpload,
|
||||
Data = data,
|
||||
});
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.Warning($"[{Name}] failed to emit power_state_change: {ex.Message}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public void Stop()
|
||||
{
|
||||
ManagementEventWatcher? watcher;
|
||||
lock (_lock)
|
||||
{
|
||||
if (_disposed) return;
|
||||
_disposed = true;
|
||||
watcher = _watcher;
|
||||
_watcher = null;
|
||||
}
|
||||
|
||||
if (watcher != null)
|
||||
{
|
||||
try { watcher.EventArrived -= OnPowerStatusChange; } catch { }
|
||||
try { watcher.Stop(); } catch { }
|
||||
try { watcher.Dispose(); } catch { }
|
||||
}
|
||||
|
||||
_debounceTimer?.Dispose();
|
||||
_debounceTimer = null;
|
||||
}
|
||||
|
||||
public void Dispose() => Stop();
|
||||
}
|
||||
}
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
#nullable enable
|
||||
using System;
|
||||
using System.Threading;
|
||||
using AutopilotMonitor.Agent.V2.Core.Logging;
|
||||
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
|
||||
using AutopilotMonitor.DecisionCore.Engine;
|
||||
using AutopilotMonitor.DecisionCore.Signals;
|
||||
|
||||
namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
{
|
||||
/// <summary>
|
||||
/// Standalone host for the <see cref="SystemTimelineTracker"/> — watches the <c>System</c>
|
||||
/// event log for system clock steps (Kernel-General 1) and completed sleep/standby episodes
|
||||
/// (Power-Troubleshooter 1 / Kernel-Power 507) and forwards them via the shared single-rail
|
||||
/// ingress. Mirrors <see cref="WindowsUpdateWatcherHost"/>: builds its own
|
||||
/// <see cref="InformationalEventPost"/> and delegates Start/Stop to the tracker.
|
||||
/// </summary>
|
||||
internal sealed class SystemTimelineWatcherHost : ICollectorHost
|
||||
{
|
||||
public string Name => "SystemTimelineTracker";
|
||||
|
||||
private readonly SystemTimelineTracker _tracker;
|
||||
private readonly AgentLogger _logger;
|
||||
private int _disposed;
|
||||
|
||||
public SystemTimelineWatcherHost(
|
||||
string sessionId,
|
||||
string tenantId,
|
||||
AgentLogger logger,
|
||||
ISignalIngressSink ingress,
|
||||
IClock clock,
|
||||
int backfillLookbackMinutes,
|
||||
string stateDirectory)
|
||||
{
|
||||
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
|
||||
if (clock == null) throw new ArgumentNullException(nameof(clock));
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
|
||||
var post = new InformationalEventPost(ingress, clock, logger);
|
||||
_tracker = new SystemTimelineTracker(
|
||||
sessionId: sessionId,
|
||||
tenantId: tenantId,
|
||||
post: post,
|
||||
logger: logger,
|
||||
backfillEnabled: backfillLookbackMinutes > 0,
|
||||
backfillLookbackMinutes: backfillLookbackMinutes,
|
||||
stateDirectory: stateDirectory);
|
||||
}
|
||||
|
||||
public void Start()
|
||||
{
|
||||
_tracker.Start();
|
||||
_logger.Info("SystemTimelineWatcherHost: started.");
|
||||
}
|
||||
|
||||
public void Stop()
|
||||
{
|
||||
try { _tracker.Stop(); }
|
||||
catch (Exception ex) { _logger.Warning($"SystemTimelineWatcherHost: stop failed: {ex.Message}"); }
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
if (Interlocked.Exchange(ref _disposed, 1) == 1) return;
|
||||
Stop();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -56,6 +56,17 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
|
||||
// real activity (rare, and worth keeping diagnostics running for).
|
||||
SharedConstants.EventTypes.ModernDeploymentLog,
|
||||
|
||||
// Environment observation, not device/enrollment progress: an idle device draining
|
||||
// its battery must not keep the periodic collectors alive or push out stall
|
||||
// detection — the exact scenario the power watcher exists to observe.
|
||||
SharedConstants.EventTypes.PowerStateChange,
|
||||
|
||||
// Same reasoning for the system timeline watcher: a clock step, a wake from standby,
|
||||
// or a backfilled episode at agent start is environment observation and must not
|
||||
// reset the idle/stall clocks.
|
||||
SharedConstants.EventTypes.SystemClockChanged,
|
||||
SharedConstants.EventTypes.SystemSleepEpisode,
|
||||
|
||||
// Agent health / control / transport — not device/enrollment progress.
|
||||
SharedConstants.EventTypes.CollectorDegraded,
|
||||
SharedConstants.EventTypes.TelemetryUploadPoisoned,
|
||||
|
||||
@@ -124,6 +124,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Persistence
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Read-only peek of the key's current fingerprint (possibly restored from a previous
|
||||
/// agent run), or null when none is recorded. Like <see cref="HasFingerprint"/> this makes
|
||||
/// NO claim — use it when the fingerprint carries state the caller needs to read back
|
||||
/// (e.g. the DO GroupId ownership marker).
|
||||
/// </summary>
|
||||
public string? GetFingerprint(string key)
|
||||
{
|
||||
if (string.IsNullOrEmpty(key)) return null;
|
||||
lock (_lock)
|
||||
{
|
||||
return _entries.TryGetValue(key, out var entry) ? entry.Fingerprint : null;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Retry-until-success: true when <see cref="MarkSucceeded"/> latched this key in any run.</summary>
|
||||
public bool AlreadySucceeded(string key)
|
||||
{
|
||||
|
||||
@@ -25,6 +25,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Runtime
|
||||
/// Always emitted — offset >60s is Warning, smaller is Info, failure is Warning.</item>
|
||||
/// <item><b>power_state_check</b> — AC vs. battery snapshot via Win32 <c>GetSystemPowerStatus</c>.
|
||||
/// On battery with <80% charge is Warning, otherwise Info, probe failure is Warning.</item>
|
||||
/// <item><b>do_group_id_auto_set</b> — only when <c>EnableDoGroupIdAutoSet=true</c>. Sets the
|
||||
/// Delivery Optimization <c>DOGroupId</c> policy value from a network fingerprint via
|
||||
/// <see cref="DoGroupIdService"/>; skips (Info) rather than fight foreign policy values.</item>
|
||||
/// </list>
|
||||
/// <para>
|
||||
/// Each probe runs best-effort: failures are logged + optionally emitted as a Warning event,
|
||||
@@ -78,6 +81,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Runtime
|
||||
await RunNtpCheck(configuration, logger, post, gate).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
if (configuration.EnableDoGroupIdAutoSet)
|
||||
RunDoGroupIdAutoSet(configuration, logger, post, gate);
|
||||
|
||||
RunPowerStateCheck(configuration, logger, post);
|
||||
}
|
||||
|
||||
@@ -331,6 +337,99 @@ namespace AutopilotMonitor.Agent.V2.Core.Runtime
|
||||
};
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// One-shot DO GroupId apply with change-based restart dedup: the gate fingerprint doubles
|
||||
/// as the ownership marker ("written:<guid>" records the GUID we last wrote, so a later
|
||||
/// run can distinguish our own stale value from a foreign one after a gateway change).
|
||||
/// Unchanged outcomes (same GUID, same skip reason) re-run silently without a duplicate
|
||||
/// event; errors never claim a fingerprint so each failing start keeps its retry trail.
|
||||
/// </summary>
|
||||
private static void RunDoGroupIdAutoSet(
|
||||
AgentConfiguration configuration,
|
||||
AgentLogger logger,
|
||||
Orchestration.InformationalEventPost post,
|
||||
Persistence.StartupEventGate? gate)
|
||||
{
|
||||
try
|
||||
{
|
||||
var lastWritten = ExtractLastWrittenGroupId(gate?.GetFingerprint(Constants.EventTypes.DoGroupIdAutoSet));
|
||||
var result = DoGroupIdService.TrySetGroupId(lastWritten, logger);
|
||||
|
||||
var fingerprint = ComputeDoGroupIdFingerprint(result);
|
||||
if (fingerprint != null && gate != null
|
||||
&& !gate.ShouldEmit(Constants.EventTypes.DoGroupIdAutoSet, fingerprint))
|
||||
{
|
||||
logger.Debug("StartupEnvironmentProbes: DO GroupId outcome unchanged — skipping duplicate event (restart dedup).");
|
||||
return;
|
||||
}
|
||||
|
||||
SafeEmit(post, logger, BuildDoGroupIdEvent(configuration, result));
|
||||
|
||||
if (fingerprint != null)
|
||||
gate?.MarkEmitted(Constants.EventTypes.DoGroupIdAutoSet);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
logger.Warning($"StartupEnvironmentProbes: DO GroupId auto-set threw: {ex.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
internal const string DoGroupIdWrittenFingerprintPrefix = "written:";
|
||||
|
||||
/// <summary>
|
||||
/// Success (written or already correct) fingerprints as "written:<guid>", a skip as
|
||||
/// "skipped:<reason>"; errors return null so they are never deduped.
|
||||
/// </summary>
|
||||
internal static string? ComputeDoGroupIdFingerprint(DoGroupIdSetResult result)
|
||||
{
|
||||
if (result.Success) return DoGroupIdWrittenFingerprintPrefix + result.GroupId;
|
||||
if (result.Skipped) return "skipped:" + result.SkipReason;
|
||||
return null;
|
||||
}
|
||||
|
||||
/// <summary>Ownership marker readback: the GUID from a "written:" fingerprint, else null.</summary>
|
||||
internal static string? ExtractLastWrittenGroupId(string? fingerprint) =>
|
||||
fingerprint != null && fingerprint.StartsWith(DoGroupIdWrittenFingerprintPrefix, StringComparison.Ordinal)
|
||||
? fingerprint.Substring(DoGroupIdWrittenFingerprintPrefix.Length)
|
||||
: null;
|
||||
|
||||
internal static EnrollmentEvent BuildDoGroupIdEvent(AgentConfiguration configuration, DoGroupIdSetResult result)
|
||||
{
|
||||
string message;
|
||||
if (result.Success && result.Written)
|
||||
message = $"Delivery Optimization GroupId set to {result.GroupId} (gateway {result.GatewayIp})";
|
||||
else if (result.Success)
|
||||
message = $"Delivery Optimization GroupId already set to {result.GroupId}";
|
||||
else if (result.Skipped)
|
||||
message = $"Delivery Optimization GroupId not set: {result.SkipReason}";
|
||||
else
|
||||
message = $"Delivery Optimization GroupId auto-set failed: {result.Error}";
|
||||
|
||||
return new EnrollmentEvent
|
||||
{
|
||||
SessionId = configuration.SessionId,
|
||||
TenantId = configuration.TenantId,
|
||||
EventType = Constants.EventTypes.DoGroupIdAutoSet,
|
||||
Severity = result.Success || result.Skipped ? EventSeverity.Info : EventSeverity.Warning,
|
||||
Source = "StartupEnvironmentProbes",
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Timestamp = DateTime.UtcNow,
|
||||
Message = message,
|
||||
Data = new Dictionary<string, object>
|
||||
{
|
||||
{ "groupId", result.GroupId ?? "" },
|
||||
{ "previousValue", result.PreviousValue ?? "" },
|
||||
{ "gatewayIp", result.GatewayIp ?? "" },
|
||||
{ "gatewayMac", result.GatewayMac ?? "" },
|
||||
{ "written", result.Written },
|
||||
{ "skipped", result.Skipped },
|
||||
{ "skipReason", result.SkipReason ?? "" },
|
||||
{ "success", result.Success },
|
||||
{ "error", result.Error ?? "" },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
private static void RunPowerStateCheck(
|
||||
AgentConfiguration configuration,
|
||||
AgentLogger logger,
|
||||
|
||||
@@ -804,12 +804,15 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
|
||||
/// <see cref="AppInstallTiming"/> so the emitter can inline the values on the
|
||||
/// current event's payload without a second dictionary lookup.
|
||||
/// <para>
|
||||
/// Audit Q1: a terminal→active transition (the IME retry path — <see
|
||||
/// cref="AppPackageState.UpdateState"/> permits e.g. Error→Downloading and
|
||||
/// Skipped→Downloading) re-arms <c>Completed</c> to <c>null</c>, so the NEXT terminal
|
||||
/// transition stamps a fresh CompletedAt. StartedAt keeps its first value — the final
|
||||
/// terminal payload then covers first-start → last-terminal (total occupancy across
|
||||
/// attempts) instead of freezing at the first failure. Observed pre-fix: a user-scope
|
||||
/// Audit Q1 + 2026-08 attempt-duration change: a terminal→active transition (the IME
|
||||
/// retry / re-evaluation path — <see cref="AppPackageState.UpdateState"/> permits e.g.
|
||||
/// Error→Downloading and Skipped→Downloading) re-arms BOTH endpoints: <c>Completed</c>
|
||||
/// to <c>null</c> (the NEXT terminal transition stamps fresh) and <c>Started</c> to
|
||||
/// the re-entry time. The final terminal payload therefore describes the
|
||||
/// status-defining ATTEMPT — matching the backend's AppInstallSummaries semantics —
|
||||
/// instead of first-start → last-terminal across passes: the IME processes the app
|
||||
/// list repeatedly (device-ESP evaluation pass, user-context pass), and the idle gap
|
||||
/// between passes is not install time. Observed pre-fix (old bug shape): a user-scope
|
||||
/// app Skipped early in DeviceSetup, retried after sign-in, emitted its final
|
||||
/// Installed event with the stale early CompletedAt (durationSeconds=-2342).
|
||||
/// </para>
|
||||
@@ -820,11 +823,13 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
|
||||
{
|
||||
_appTimings.TryGetValue(appId, out var current);
|
||||
|
||||
// Terminal→active (retry): the earlier terminal stamp no longer describes this
|
||||
// app's outcome — re-arm so the next terminal transition stamps fresh.
|
||||
// Terminal→active (retry / new pass): the earlier stamps no longer describe
|
||||
// this app's outcome — re-arm both endpoints so the next terminal transition
|
||||
// reports the fresh attempt's window.
|
||||
if (current.Completed != null && IsLifecycleActive(newState))
|
||||
{
|
||||
current.Completed = null;
|
||||
current.Started = now;
|
||||
}
|
||||
|
||||
// First lifecycle-active transition → record StartedAt.
|
||||
|
||||
@@ -372,6 +372,9 @@ namespace AutopilotMonitor.DecisionCore.Tests
|
||||
// Before RealmJoinPhaseChanged became a typed signal, LastDeploymentPhase was only
|
||||
// written at detection time — every realmjoin_timeout claimed "last phase: 0" no
|
||||
// matter how far RJ actually got, hiding this whole failure class from ops.
|
||||
// Since the activity-based extension, the PhaseChanged at T0+6 counts as activity,
|
||||
// so the first fire (59 min later) extends once; the timeout lands on the re-armed
|
||||
// fire a full inactivity window after the last activity.
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
@@ -384,7 +387,13 @@ namespace AutopilotMonitor.DecisionCore.Tests
|
||||
})).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.DesktopArrived, T0.AddMinutes(7))).NewState;
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(8, DecisionSignalKind.DeadlineFired, T0.AddMinutes(65),
|
||||
// First fire at the original due (T0+65): activity 59 min ago → extends to T0+66.
|
||||
state = engine.Reduce(state, MakeSignal(8, DecisionSignalKind.DeadlineFired, T0.AddMinutes(65),
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout })).NewState;
|
||||
var rearmed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
Assert.Equal(T0.AddMinutes(66), rearmed.DueAtUtc);
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(9, DecisionSignalKind.DeadlineFired, rearmed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout }));
|
||||
|
||||
var timeout = Assert.Single(step.Effects, e =>
|
||||
@@ -395,6 +404,248 @@ namespace AutopilotMonitor.DecisionCore.Tests
|
||||
Assert.Equal("101", timeout.Parameters["lastSeenPhase"]);
|
||||
}
|
||||
|
||||
// ============================================================== Activity-based extension (report 55e6afd61c9d)
|
||||
|
||||
[Fact]
|
||||
public void Timeout_fire_with_recent_first_deployment_activity_extends_instead_of_timing_out()
|
||||
{
|
||||
// Report 55e6afd61c9d (Douglas): timer armed at detection (phase 0, RJ agent MSI
|
||||
// install during DeviceSetup), first deployment only started 16 min later, Office
|
||||
// completed 3 s before the deadline — the hard cut truncated an actively working
|
||||
// deployment. The fire must now re-arm to lastActivity + 60 min instead.
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "0" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.DesktopArrived, T0.AddMinutes(6))).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(21),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "0",
|
||||
})).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(8, DecisionSignalKind.RealmJoinPackageCompleted, T0.AddMinutes(62),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PackageId] = "generic-microsoft-office-2016-proplus",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.Scope] = RealmJoinPackageFact.ScopeMachine,
|
||||
[DecisionEngine.RealmJoinPayloadKeys.Success] = "true",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.LastExitCode] = "0",
|
||||
})).NewState;
|
||||
|
||||
var armed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
var step = engine.Reduce(state, MakeSignal(9, DecisionSignalKind.DeadlineFired, armed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout }));
|
||||
|
||||
// No completion, no outcome — the gate stays closed and the window is re-armed.
|
||||
Assert.NotEqual(SessionStage.Finalizing, step.NewState.Stage);
|
||||
Assert.Null(step.NewState.RealmJoinFacts.Outcome);
|
||||
Assert.Equal($"DeadlineFired:{DeadlineNames.RealmJoinTimeout}:Extended", step.Transition.Trigger);
|
||||
|
||||
var rearmed = Assert.Single(step.NewState.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
Assert.Equal(T0.AddMinutes(62 + 60), rearmed.DueAtUtc); // lastActivity + inactivity window
|
||||
|
||||
Assert.Single(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.ScheduleDeadline &&
|
||||
e.Deadline != null &&
|
||||
e.Deadline.Name == DeadlineNames.RealmJoinTimeout &&
|
||||
e.Deadline.DueAtUtc == rearmed.DueAtUtc);
|
||||
|
||||
var extended = Assert.Single(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_timeout_extended");
|
||||
Assert.Equal("Info", extended.Parameters!["severity"]);
|
||||
Assert.Equal("101", extended.Parameters["deploymentPhase"]);
|
||||
Assert.DoesNotContain(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_timeout");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Resolved_after_extension_completes_via_classic_path()
|
||||
{
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.DesktopArrived, T0.AddMinutes(6))).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(40),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "100",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "101",
|
||||
})).NewState;
|
||||
var armed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
state = engine.Reduce(state, MakeSignal(8, DecisionSignalKind.DeadlineFired, armed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout })).NewState;
|
||||
Assert.Null(state.RealmJoinFacts.Outcome); // extended, not timed out
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(9, DecisionSignalKind.RealmJoinResolved, T0.AddMinutes(80),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "110" }));
|
||||
|
||||
Assert.Equal(SessionStage.Finalizing, step.NewState.Stage);
|
||||
Assert.Equal("Resolved", step.NewState.RealmJoinFacts.Outcome!.Value);
|
||||
Assert.DoesNotContain(step.NewState.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Rearmed_fire_after_quiet_window_times_out_with_inactivity_reason()
|
||||
{
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "0" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.DesktopArrived, T0.AddMinutes(6))).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(30),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "0",
|
||||
})).NewState;
|
||||
var armed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
state = engine.Reduce(state, MakeSignal(8, DecisionSignalKind.DeadlineFired, armed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout })).NewState;
|
||||
var rearmed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
Assert.Equal(T0.AddMinutes(90), rearmed.DueAtUtc); // lastActivity (T0+30) + 60 min
|
||||
|
||||
// No further activity — the re-armed fire must now time out for real.
|
||||
var step = engine.Reduce(state, MakeSignal(9, DecisionSignalKind.DeadlineFired, rearmed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout }));
|
||||
|
||||
Assert.Equal(SessionStage.Finalizing, step.NewState.Stage);
|
||||
Assert.Equal("Timeout", step.NewState.RealmJoinFacts.Outcome!.Value);
|
||||
var timeout = Assert.Single(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_timeout");
|
||||
Assert.Equal("inactivity", timeout.Parameters!["reason"]);
|
||||
Assert.Contains("no deployment activity", timeout.Parameters["message"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Extension_is_capped_at_absolute_ceiling_and_cap_fire_times_out_despite_activity()
|
||||
{
|
||||
// Late activity pushes the sliding window beyond detection + 4 h — the re-arm must
|
||||
// clamp to the cap, and the cap fire must time out even though activity is recent.
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.DesktopArrived, T0.AddMinutes(6))).NewState;
|
||||
// Package completion whose sliding window (T0+230+60) would exceed the cap (T0+245).
|
||||
state = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.RealmJoinPackageCompleted, T0.AddMinutes(230),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PackageId] = "generic-big-suite",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.Scope] = RealmJoinPackageFact.ScopeMachine,
|
||||
[DecisionEngine.RealmJoinPayloadKeys.Success] = "true",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.LastExitCode] = "0",
|
||||
})).NewState;
|
||||
|
||||
var armed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
state = engine.Reduce(state, MakeSignal(8, DecisionSignalKind.DeadlineFired, armed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout })).NewState;
|
||||
|
||||
var rearmed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
Assert.Equal(T0.AddMinutes(5).AddHours(4), rearmed.DueAtUtc); // clamped to detection + cap
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(9, DecisionSignalKind.DeadlineFired, rearmed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout }));
|
||||
|
||||
Assert.Equal(SessionStage.Finalizing, step.NewState.Stage);
|
||||
Assert.Equal("Timeout", step.NewState.RealmJoinFacts.Outcome!.Value);
|
||||
var timeout = Assert.Single(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_timeout");
|
||||
Assert.Equal("absolute_cap", timeout.Parameters!["reason"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Stale_fire_of_old_incarnation_after_rearm_is_dead_end()
|
||||
{
|
||||
// Race: the OLD deadline incarnation's fire was already queued when the extension
|
||||
// re-armed. The armed deadline is due LATER than the stale fire — dead-end, no
|
||||
// second extension evaluation, no timeout.
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "0" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(30),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "0",
|
||||
})).NewState;
|
||||
var armed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
state = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.DeadlineFired, armed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout })).NewState;
|
||||
Assert.Null(state.RealmJoinFacts.Outcome); // extended
|
||||
|
||||
var step = engine.Reduce(state, MakeSignal(8, DecisionSignalKind.DeadlineFired, armed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout }));
|
||||
|
||||
Assert.False(step.Transition.Taken);
|
||||
Assert.Null(step.NewState.RealmJoinFacts.Outcome);
|
||||
Assert.Empty(step.Effects);
|
||||
var stillArmed = Assert.Single(step.NewState.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
Assert.Equal(T0.AddMinutes(90), stillArmed.DueAtUtc);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Detection_phase_alone_is_not_activity_idle_case_times_out_at_60_min_unchanged()
|
||||
{
|
||||
// The phase captured AT detection must not count as activity — RJ detected in a
|
||||
// first-deployment phase that then never moves still times out after the original
|
||||
// 60 min (the extension exists for demonstrable progress, not for standing still).
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "101" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.DesktopArrived, T0.AddMinutes(6))).NewState;
|
||||
Assert.Null(state.RealmJoinFacts.LastActivityUtc);
|
||||
|
||||
var armed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
var step = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.DeadlineFired, armed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout }));
|
||||
|
||||
Assert.Equal(SessionStage.Finalizing, step.NewState.Stage);
|
||||
Assert.Equal("Timeout", step.NewState.RealmJoinFacts.Outcome!.Value);
|
||||
var timeout = Assert.Single(step.Effects, e =>
|
||||
e.Kind == DecisionEffectKind.EmitEventTimelineEntry &&
|
||||
e.Parameters != null &&
|
||||
e.Parameters.TryGetValue("eventType", out var et) && et == "realmjoin_timeout");
|
||||
Assert.Equal("hard_timeout", timeout.Parameters!["reason"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Regular_deployment_phase_with_activity_does_not_extend()
|
||||
{
|
||||
// Session 6f1959c0 shape: RJ stood at 210 when the agent booted and deploys
|
||||
// regular (non-first) packages afterwards. Phase 200/210 is outside the
|
||||
// first-deployment window — activity there must NOT hold the session hostage.
|
||||
var engine = new DecisionEngine();
|
||||
var state = PrimeClassicAwaitingDesktop(engine);
|
||||
state = engine.Reduce(state, MakeSignal(5, DecisionSignalKind.RealmJoinDetected, T0.AddMinutes(5),
|
||||
new Dictionary<string, string> { [DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "210" })).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.DesktopArrived, T0.AddMinutes(6))).NewState;
|
||||
state = engine.Reduce(state, MakeSignal(7, DecisionSignalKind.RealmJoinPhaseChanged, T0.AddMinutes(40),
|
||||
new Dictionary<string, string>
|
||||
{
|
||||
[DecisionEngine.RealmJoinPayloadKeys.DeploymentPhase] = "200",
|
||||
[DecisionEngine.RealmJoinPayloadKeys.PreviousPhase] = "210",
|
||||
})).NewState;
|
||||
|
||||
var armed = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.RealmJoinTimeout);
|
||||
var step = engine.Reduce(state, MakeSignal(8, DecisionSignalKind.DeadlineFired, armed.DueAtUtc,
|
||||
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.RealmJoinTimeout }));
|
||||
|
||||
Assert.Equal(SessionStage.Finalizing, step.NewState.Stage);
|
||||
Assert.Equal("Timeout", step.NewState.RealmJoinFacts.Outcome!.Value);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void SelfDeploying_deferred_terminal_released_by_first_deployment_incomplete()
|
||||
{
|
||||
|
||||
+160
@@ -198,4 +198,164 @@ public class AppInstallTerminalStateAndReconcileTests
|
||||
// Column absent → Merge-mode preserves a prior batch's terminal state.
|
||||
Assert.False(empty.ContainsKey("TerminalState"));
|
||||
}
|
||||
|
||||
// ── attempt-duration semantics (2026-08): the field case that motivated it ──
|
||||
// Session 44862cd3, "Set TimeZone": device-ESP evaluation pass ends Skipped, the real
|
||||
// install happens ~66 min later in the user-context pass and takes 23 s. The old span
|
||||
// semantics reported 4002 s.
|
||||
|
||||
[Fact]
|
||||
public void TwoPasses_SingleBatch_DurationIsLastAttempt_NotSpan()
|
||||
{
|
||||
var s = Aggregate(
|
||||
AppEvent("app_install_started", T0),
|
||||
AppEvent("app_install_completed", T0.AddMinutes(6), "Skipped"),
|
||||
AppEvent("app_install_started", T0.AddMinutes(66)),
|
||||
AppEvent("app_install_completed", T0.AddMinutes(66).AddSeconds(23), "Installed"))["App"].Summary;
|
||||
|
||||
Assert.Equal(23, s.DurationSeconds); // last attempt, not 3983 s
|
||||
Assert.Equal(T0, s.StartedAt); // first-seen anchor untouched
|
||||
Assert.Equal(T0.AddMinutes(66), s.LastAttemptStartedAt);
|
||||
Assert.Equal(2, s.InstallPassCount);
|
||||
Assert.Equal("Installed", s.TerminalState);
|
||||
Assert.Equal(T0.AddMinutes(66).AddSeconds(23), s.CompletedAt);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WeakerTerminal_CompletedSkipped_NeverOverridesInstalled_InBatch()
|
||||
{
|
||||
// Re-evaluation after a real install reports Skipped (already present) — the row
|
||||
// keeps describing the install attempt, incl. CompletedAt and duration.
|
||||
var s = Aggregate(
|
||||
AppEvent("app_install_started", T0),
|
||||
AppEvent("app_install_completed", T0.AddSeconds(40), "Installed"),
|
||||
AppEvent("app_install_started", T0.AddMinutes(30)),
|
||||
AppEvent("app_install_completed", T0.AddMinutes(30).AddSeconds(2), "Skipped"))["App"].Summary;
|
||||
|
||||
Assert.Equal("Installed", s.TerminalState);
|
||||
Assert.Equal(T0.AddSeconds(40), s.CompletedAt);
|
||||
Assert.Equal(40, s.DurationSeconds);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void StartInOneBatch_TerminalInNext_ReconcileAnchorsOnStoredAttempt()
|
||||
{
|
||||
// Pass 2 start landed in batch N (stored anchor); the terminal-only batch N+1 must
|
||||
// pair against it — not against the first-seen StartedAt an hour earlier.
|
||||
var existing = new TableEntity("tenant", "session_App")
|
||||
{
|
||||
["Status"] = "InProgress",
|
||||
["StartedAt"] = new DateTimeOffset(T0),
|
||||
["LastAttemptStartedAt"] = new DateTimeOffset(T0.AddMinutes(66)),
|
||||
["InstallPassCount"] = 2,
|
||||
};
|
||||
var summary = new AppInstallSummary
|
||||
{
|
||||
AppName = "App", SessionId = "session", TenantId = "tenant",
|
||||
Status = "Succeeded", TerminalState = "Installed",
|
||||
StartedAt = T0.AddMinutes(66).AddSeconds(23), // batch init stamp (terminal event)
|
||||
CompletedAt = T0.AddMinutes(66).AddSeconds(23),
|
||||
};
|
||||
|
||||
TableStorageService.ReconcileAppInstallSummaryWithExisting(summary, existing);
|
||||
|
||||
Assert.Equal(T0, summary.StartedAt); // earliest wins (window key)
|
||||
Assert.Equal(T0.AddMinutes(66), summary.LastAttemptStartedAt);
|
||||
Assert.Equal(23, summary.DurationSeconds); // attempt-anchored, not 3983 s
|
||||
Assert.Equal(2, summary.InstallPassCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reconcile_ReplayedOlderBatch_NeverRegressesAnchorOrDoubleCountsPasses()
|
||||
{
|
||||
var existing = new TableEntity("tenant", "session_App")
|
||||
{
|
||||
["StartedAt"] = new DateTimeOffset(T0),
|
||||
["LastAttemptStartedAt"] = new DateTimeOffset(T0.AddMinutes(66)),
|
||||
["InstallPassCount"] = 2,
|
||||
};
|
||||
// Replay of the pass-1 batch: its newest start is OLDER than the stored anchor.
|
||||
var summary = new AppInstallSummary
|
||||
{
|
||||
AppName = "App", Status = "InProgress",
|
||||
StartedAt = T0, LastAttemptStartedAt = T0, InstallPassCount = 1,
|
||||
};
|
||||
|
||||
TableStorageService.ReconcileAppInstallSummaryWithExisting(summary, existing);
|
||||
|
||||
Assert.Equal(T0.AddMinutes(66), summary.LastAttemptStartedAt); // max-fold: no regression
|
||||
Assert.Equal(2, summary.InstallPassCount); // not 3 — replay not re-counted
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reconcile_NewerAttemptBatch_AddsPassCount()
|
||||
{
|
||||
var existing = new TableEntity("tenant", "session_App")
|
||||
{
|
||||
["StartedAt"] = new DateTimeOffset(T0),
|
||||
["LastAttemptStartedAt"] = new DateTimeOffset(T0),
|
||||
["InstallPassCount"] = 1,
|
||||
};
|
||||
var summary = new AppInstallSummary
|
||||
{
|
||||
AppName = "App", Status = "InProgress",
|
||||
StartedAt = T0.AddMinutes(66), LastAttemptStartedAt = T0.AddMinutes(66), InstallPassCount = 1,
|
||||
};
|
||||
|
||||
TableStorageService.ReconcileAppInstallSummaryWithExisting(summary, existing);
|
||||
|
||||
Assert.Equal(T0.AddMinutes(66), summary.LastAttemptStartedAt);
|
||||
Assert.Equal(2, summary.InstallPassCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reconcile_WeakerTerminalBatch_NeverOverridesStoredInstalled()
|
||||
{
|
||||
// Cross-batch mirror of the in-batch guard: a batch that only saw the Skipped
|
||||
// re-evaluation must not re-describe an Installed row.
|
||||
var existing = new TableEntity("tenant", "session_App")
|
||||
{
|
||||
["Status"] = "Succeeded",
|
||||
["TerminalState"] = "Installed",
|
||||
["StartedAt"] = new DateTimeOffset(T0),
|
||||
["CompletedAt"] = new DateTimeOffset(T0.AddSeconds(40)),
|
||||
["DurationSeconds"] = 40,
|
||||
};
|
||||
var summary = new AppInstallSummary
|
||||
{
|
||||
AppName = "App", Status = "Succeeded", TerminalState = "Skipped",
|
||||
StartedAt = T0.AddMinutes(30),
|
||||
LastAttemptStartedAt = T0.AddMinutes(30), InstallPassCount = 1,
|
||||
CompletedAt = T0.AddMinutes(30).AddSeconds(2), DurationSeconds = 2,
|
||||
};
|
||||
|
||||
TableStorageService.ReconcileAppInstallSummaryWithExisting(summary, existing);
|
||||
|
||||
Assert.Equal("Installed", summary.TerminalState);
|
||||
// The guard drops the skip-pass endpoints; the Q4 path then re-adopts the STORED
|
||||
// CompletedAt (the surviving install attempt's) — same value as on disk, and the
|
||||
// attempt anchor being newer keeps DurationSeconds at 0, so the omitted column
|
||||
// lets Merge preserve the stored 40 s.
|
||||
Assert.Equal(T0.AddSeconds(40), summary.CompletedAt);
|
||||
Assert.Equal(0, summary.DurationSeconds);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void EntityBuilder_AttemptColumns_AreSentinelGated()
|
||||
{
|
||||
var with = new AppInstallSummary
|
||||
{
|
||||
AppName = "App",
|
||||
LastAttemptStartedAt = T0,
|
||||
InstallPassCount = 2,
|
||||
};
|
||||
var entity = TableStorageService.BuildAppInstallSummaryEntity(with, "rk");
|
||||
Assert.Equal(new DateTimeOffset(T0), entity.GetDateTimeOffset("LastAttemptStartedAt"));
|
||||
Assert.Equal(2, entity.GetInt32("InstallPassCount"));
|
||||
|
||||
var without = new AppInstallSummary { AppName = "App" };
|
||||
var empty = TableStorageService.BuildAppInstallSummaryEntity(without, "rk");
|
||||
Assert.False(empty.ContainsKey("LastAttemptStartedAt"));
|
||||
Assert.False(empty.ContainsKey("InstallPassCount"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -36,7 +36,8 @@ public class AppVersionRegressionRadarTests
|
||||
string appName = App,
|
||||
string status = "Succeeded",
|
||||
string terminalState = "Installed",
|
||||
bool collision = false)
|
||||
bool collision = false,
|
||||
bool attemptAnchored = true)
|
||||
=> new()
|
||||
{
|
||||
TenantId = TenantA,
|
||||
@@ -47,6 +48,9 @@ public class AppVersionRegressionRadarTests
|
||||
TerminalState = terminalState,
|
||||
DurationSeconds = durationSeconds,
|
||||
StartedAt = startedAt ?? Now.AddDays(-1),
|
||||
// Post-cutover rows carry the attempt anchor; the radar's cutover gate
|
||||
// excludes rows without it (attemptAnchored: false = legacy span row).
|
||||
LastAttemptStartedAt = attemptAnchored ? (startedAt ?? Now.AddDays(-1)) : null,
|
||||
AppIdCollision = collision,
|
||||
};
|
||||
|
||||
@@ -164,6 +168,26 @@ public class AppVersionRegressionRadarTests
|
||||
Assert.Equal(900, finding.CurrentMedianSeconds);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Evaluate_CutoverGate_LegacySpanRows_ExcludedOnBothSides()
|
||||
{
|
||||
// 2026-08 attempt-duration cutover: rows without LastAttemptStartedAt carry the old
|
||||
// full-span semantics and are excluded from BOTH comparison sides.
|
||||
|
||||
// Current side: 10 legacy span rows would fire (lift 3.0), but only 5 are
|
||||
// attempt-anchored → below MinMeasuredInstalls → silent.
|
||||
var rows = Batch("1.0", 10, 300, Now.AddDays(-20));
|
||||
rows.AddRange(Enumerable.Range(0, 10).Select(_ => Install("2.0", 900, Now.AddDays(-5), attemptAnchored: false)));
|
||||
rows.AddRange(Enumerable.Range(0, 5).Select(_ => Install("2.0", 900, Now.AddDays(-5))));
|
||||
Assert.Empty(AppVersionRegressionRadar.Evaluate(rows));
|
||||
|
||||
// Previous side entirely legacy → gated away → no comparison basis → silent,
|
||||
// even though the raw medians (300 → 900) would scream regression.
|
||||
var rows2 = Enumerable.Range(0, 10).Select(_ => Install("1.0", 300, Now.AddDays(-20), attemptAnchored: false)).ToList();
|
||||
rows2.AddRange(Batch("2.0", 10, 900, Now.AddDays(-5)));
|
||||
Assert.Empty(AppVersionRegressionRadar.Evaluate(rows2));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Evaluate_DeterministicOrdering_LiftDesc_ThenAppNameOrdinal()
|
||||
{
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Pins the shipped rule catalogs to the reserved built-in ID namespace from
|
||||
/// RuleIdPolicy: every built-in occupies (ANALYZE|GATHER)-<CATEGORY>-<NUMBER> with
|
||||
/// CATEGORY != CUSTOM, and IDs are unique (case-insensitively — case variants are
|
||||
/// indistinguishable to humans and to the policy regex). This is the backend-side
|
||||
/// enforcement of the "platform never ships a built-in in the CUSTOM namespace"
|
||||
/// commitment documented in RuleIdPolicy — without it, a shipped CUSTOM-ID rule
|
||||
/// would silently shadow every tenant's same-ID custom rule at merge time
|
||||
/// (global wins, tenant copy dropped). combine.js mirrors these checks at
|
||||
/// catalog-build time; this test guards the embedded resource actually deployed.
|
||||
/// </summary>
|
||||
public class BuiltInRuleCatalogPolicyTests
|
||||
{
|
||||
[Fact]
|
||||
public void BuiltIn_analyze_ids_are_reserved_namespace_and_unique()
|
||||
{
|
||||
var ids = BuiltInAnalyzeRules.GetAll().Select(r => r.RuleId).ToList();
|
||||
|
||||
Assert.NotEmpty(ids);
|
||||
foreach (var id in ids)
|
||||
{
|
||||
Assert.True(RuleIdPolicy.IsReservedBuiltInId(id),
|
||||
$"Built-in analyze rule '{id}' lies outside the reserved built-in namespace — " +
|
||||
"it would collide with the tenant custom namespace.");
|
||||
}
|
||||
|
||||
var duplicates = ids.GroupBy(id => id, StringComparer.OrdinalIgnoreCase)
|
||||
.Where(g => g.Count() > 1).Select(g => g.Key).ToList();
|
||||
Assert.True(duplicates.Count == 0,
|
||||
$"Duplicate built-in analyze rule IDs: {string.Join(", ", duplicates)}");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void BuiltIn_gather_ids_are_reserved_namespace_and_unique()
|
||||
{
|
||||
var ids = BuiltInGatherRules.GetAll().Select(r => r.RuleId).ToList();
|
||||
|
||||
Assert.NotEmpty(ids);
|
||||
foreach (var id in ids)
|
||||
{
|
||||
Assert.True(RuleIdPolicy.IsReservedBuiltInId(id),
|
||||
$"Built-in gather rule '{id}' lies outside the reserved built-in namespace — " +
|
||||
"it would collide with the tenant custom namespace.");
|
||||
}
|
||||
|
||||
var duplicates = ids.GroupBy(id => id, StringComparer.OrdinalIgnoreCase)
|
||||
.Where(g => g.Count() > 1).Select(g => g.Key).ToList();
|
||||
Assert.True(duplicates.Count == 0,
|
||||
$"Duplicate built-in gather rule IDs: {string.Join(", ", duplicates)}");
|
||||
}
|
||||
}
|
||||
@@ -379,20 +379,50 @@ public class EventTimestampValidationTests
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RecalculateAppDurations_DownloadThenInstall_CorrectDownloadDuration()
|
||||
public void DownloadThenInstall_AttemptScopedPairing_DownloadAndInstallDurations()
|
||||
{
|
||||
// 2026-08 attempt-duration change: DownloadDurationSeconds pairs the latest download
|
||||
// start with the NEXT install start at event-fold time (no first→first span math),
|
||||
// and DurationSeconds anchors on the install attempt (download time is reported
|
||||
// separately, not folded into the install duration).
|
||||
var downloadStart = new DateTime(2026, 3, 30, 10, 0, 0, DateTimeKind.Utc);
|
||||
var installStart = downloadStart.AddSeconds(30);
|
||||
var completed = installStart.AddSeconds(45);
|
||||
|
||||
var summaries = new Dictionary<string, AppInstallAggregationState>(StringComparer.OrdinalIgnoreCase);
|
||||
foreach (var evt in new[]
|
||||
{
|
||||
AppInstallEvent("app_download_started", downloadStart),
|
||||
AppInstallEvent("app_install_started", installStart),
|
||||
AppInstallEvent("app_install_completed", completed, "Installed"),
|
||||
})
|
||||
{
|
||||
EventIngestProcessor.AggregateAppInstallEvent(evt, "tenant", "session", summaries);
|
||||
}
|
||||
|
||||
var summary = summaries["TestApp"].Summary;
|
||||
Assert.Equal(30, summary.DownloadDurationSeconds); // download→install gap
|
||||
Assert.Equal(45, summary.DurationSeconds); // install attempt only
|
||||
Assert.Equal(installStart, summary.LastAttemptStartedAt);
|
||||
Assert.Equal(downloadStart, summary.StartedAt); // first-seen anchor
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RecalculateAppDurations_AttemptAnchor_BeatsFirstSeenSpan()
|
||||
{
|
||||
// With an observed attempt start, the duration is the attempt — never the span
|
||||
// back to the first observation an hour earlier.
|
||||
var firstSeen = new DateTime(2026, 3, 30, 10, 0, 0, DateTimeKind.Utc);
|
||||
var attemptStart = firstSeen.AddMinutes(60);
|
||||
var completed = attemptStart.AddSeconds(45);
|
||||
|
||||
var state = new AppInstallAggregationState
|
||||
{
|
||||
DownloadStartedAt = downloadStart,
|
||||
InstallStartedAt = installStart,
|
||||
Summary = new AppInstallSummary
|
||||
{
|
||||
AppName = "TestApp",
|
||||
StartedAt = downloadStart,
|
||||
StartedAt = firstSeen,
|
||||
LastAttemptStartedAt = attemptStart,
|
||||
CompletedAt = completed,
|
||||
Status = "Succeeded"
|
||||
}
|
||||
@@ -400,8 +430,46 @@ public class EventTimestampValidationTests
|
||||
|
||||
EventIngestProcessor.RecalculateAppDurations(state);
|
||||
|
||||
Assert.Equal(30, state.Summary.DownloadDurationSeconds); // download→install gap
|
||||
Assert.Equal(75, state.Summary.DurationSeconds); // full duration
|
||||
Assert.Equal(45, state.Summary.DurationSeconds);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void RecalculateAppDurations_AttemptNewerThanCompletion_KeepsRecordedDuration()
|
||||
{
|
||||
// A fresh pass is in flight (start after the last terminal): CompletedAt belongs to
|
||||
// the previous attempt — its recorded duration must stand.
|
||||
var attempt1 = new DateTime(2026, 3, 30, 10, 0, 0, DateTimeKind.Utc);
|
||||
var completed = attempt1.AddSeconds(40);
|
||||
var attempt2 = completed.AddMinutes(30);
|
||||
|
||||
var state = new AppInstallAggregationState
|
||||
{
|
||||
Summary = new AppInstallSummary
|
||||
{
|
||||
AppName = "TestApp",
|
||||
StartedAt = attempt1,
|
||||
LastAttemptStartedAt = attempt2,
|
||||
CompletedAt = completed,
|
||||
DurationSeconds = 40,
|
||||
Status = "Succeeded"
|
||||
}
|
||||
};
|
||||
|
||||
EventIngestProcessor.RecalculateAppDurations(state);
|
||||
|
||||
Assert.Equal(40, state.Summary.DurationSeconds);
|
||||
}
|
||||
|
||||
private static EnrollmentEvent AppInstallEvent(string eventType, DateTime ts, string? state = null)
|
||||
{
|
||||
var evt = new EnrollmentEvent
|
||||
{
|
||||
EventType = eventType,
|
||||
Timestamp = ts,
|
||||
Data = new Dictionary<string, object> { ["appName"] = "TestApp" },
|
||||
};
|
||||
if (state != null) evt.Data["state"] = state;
|
||||
return evt;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
@@ -46,6 +46,8 @@ public class FleetHealthPayloadTests
|
||||
Assert.Equal(0, payload.Stats.Total);
|
||||
Assert.Equal(0d, payload.Stats.SuccessRate);
|
||||
Assert.Equal(0, payload.Stats.AvgDurationMinutes);
|
||||
Assert.Equal(0, payload.Stats.MedianDurationMinutes);
|
||||
Assert.Equal(0, payload.Stats.P90DurationMinutes);
|
||||
Assert.Empty(payload.FailureReasons);
|
||||
Assert.Empty(payload.ModelHealth);
|
||||
Assert.Empty(payload.SlowestModels);
|
||||
@@ -109,6 +111,28 @@ public class FleetHealthPayloadTests
|
||||
|
||||
// (600 + 1200) / 2 / 60 = 15 min.
|
||||
Assert.Equal(15, payload.Stats.AvgDurationMinutes);
|
||||
// Same population feeds the percentile stats: sorted [600, 1200].
|
||||
Assert.Equal(10, payload.Stats.MedianDurationMinutes);
|
||||
Assert.Equal(20, payload.Stats.P90DurationMinutes);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void MedianDuration_IsRobustAgainstOutliers_ThatDominateTheAverage()
|
||||
{
|
||||
// The card's raison d'être: 8 typical 10-min enrollments plus a 2 h and a 24 h
|
||||
// outlier (overnight ESP / WhiteGlove late user phase). The mean explodes to ~2.7 h
|
||||
// while the median still answers "how long does a typical enrollment take".
|
||||
var sessions = new List<SessionSummary>();
|
||||
for (int i = 0; i < 8; i++)
|
||||
sessions.Add(S(SessionStatus.Succeeded, durationSeconds: 600));
|
||||
sessions.Add(S(SessionStatus.Succeeded, durationSeconds: 7200));
|
||||
sessions.Add(S(SessionStatus.Succeeded, durationSeconds: 86400));
|
||||
|
||||
var payload = MetricsMath.BuildFleetHealthPayload(sessions, days: 7);
|
||||
|
||||
Assert.Equal(164, payload.Stats.AvgDurationMinutes); // (8*600 + 7200 + 86400) / 10 / 60
|
||||
Assert.Equal(10, payload.Stats.MedianDurationMinutes); // unmoved by the tail
|
||||
Assert.Equal(120, payload.Stats.P90DurationMinutes); // tail stays visible
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
@@ -54,9 +54,13 @@ public class GraphFeatureCatalogSyncTests
|
||||
.Select(m => m.Groups["f"].Value)
|
||||
.ToArray();
|
||||
|
||||
// 'All' is a script-side meta-value that expands to every catalog feature — it is not
|
||||
// (and must never become) a GraphFeatureCatalog entry, but the script must keep offering it.
|
||||
Assert.Contains("All", scriptFeatures);
|
||||
|
||||
Assert.Equal(
|
||||
GraphFeatureCatalog.Features.OrderBy(f => f, StringComparer.OrdinalIgnoreCase),
|
||||
scriptFeatures.OrderBy(f => f, StringComparer.OrdinalIgnoreCase));
|
||||
scriptFeatures.Where(f => f != "All").OrderBy(f => f, StringComparer.OrdinalIgnoreCase));
|
||||
}
|
||||
|
||||
private static string ReadGrantScript()
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Shared.DataAccess;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Moq;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Power watcher analyze rules — pins the firing behaviour of the built-ins that consume the
|
||||
/// agent's live <c>power_state_change</c> events (PowerStateWatcherHost):
|
||||
/// - ANALYZE-DEV-009 (high): battery crossed the 15% threshold while enrolling on battery.
|
||||
/// - ANALYZE-DEV-010 (warning): the device switched from AC to battery mid-enrollment.
|
||||
/// Both rules are deliberately single-required-condition: event_data conditions scan events
|
||||
/// INDEPENDENTLY (no same-instance join), so a two-condition rule could false-positive across
|
||||
/// two different power events. DEV-009 matches the agent-stamped thresholdPercent (only present
|
||||
/// on threshold events, value pins the ladder level); DEV-010 matches transition=ac_to_battery.
|
||||
/// Also pins that the rule's string "15" compares equal against the agent's numeric payload.
|
||||
/// </summary>
|
||||
public class PowerStateRuleTests
|
||||
{
|
||||
private const string TenantId = "a1b2c3d4-e5f6-7890-abcd-ef1234567890";
|
||||
private const string SessionId = "11111111-2222-3333-4444-555555555555";
|
||||
|
||||
[Fact]
|
||||
public async Task ANALYZE_DEV_009_fires_high_on_the_15_percent_threshold_event()
|
||||
{
|
||||
var rule = BuiltInAnalyzeRules.GetAll().First(r => r.RuleId == "ANALYZE-DEV-009");
|
||||
Assert.True(rule.Enabled);
|
||||
Assert.Equal("high", rule.Severity);
|
||||
Assert.False(rule.MarkSessionAsFailedDefault); // environment risk, not a KO verdict
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
ThresholdCrossed(thresholdPercent: 15, batteryPercent: 13),
|
||||
};
|
||||
|
||||
var outcome = await RunAsync(rule, events);
|
||||
|
||||
var result = Assert.Single(outcome.Results);
|
||||
Assert.Equal("ANALYZE-DEV-009", result.RuleId);
|
||||
Assert.Equal("high", result.Severity);
|
||||
|
||||
// The numeric payload (int 15) must satisfy the rule's string value "15"; field/value on
|
||||
// the matched condition feed the {{thresholdPercent}} interpolation token.
|
||||
var matched = AsDict(result.MatchedConditions["battery_threshold_15"]);
|
||||
Assert.Equal("thresholdPercent", AsString(matched["field"]));
|
||||
Assert.Equal("15", AsString(matched["value"]));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(50)]
|
||||
[InlineData(30)]
|
||||
public async Task ANALYZE_DEV_009_does_not_fire_on_higher_threshold_levels(int level)
|
||||
{
|
||||
var rule = BuiltInAnalyzeRules.GetAll().First(r => r.RuleId == "ANALYZE-DEV-009");
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
ThresholdCrossed(thresholdPercent: level, batteryPercent: level - 2),
|
||||
};
|
||||
|
||||
var outcome = await RunAsync(rule, events);
|
||||
Assert.Empty(outcome.Results);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ANALYZE_DEV_009_does_not_fire_on_a_low_battery_transition_event()
|
||||
{
|
||||
// An ac_to_battery event at 13% carries batteryPercent but NO thresholdPercent — the
|
||||
// agent emits the separate threshold event for that; the rule must not fire early here.
|
||||
var rule = BuiltInAnalyzeRules.GetAll().First(r => r.RuleId == "ANALYZE-DEV-009");
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
Transition("ac_to_battery", onAcPower: false, batteryPercent: 13),
|
||||
};
|
||||
|
||||
var outcome = await RunAsync(rule, events);
|
||||
Assert.Empty(outcome.Results);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ANALYZE_DEV_010_fires_warning_on_ac_to_battery()
|
||||
{
|
||||
var rule = BuiltInAnalyzeRules.GetAll().First(r => r.RuleId == "ANALYZE-DEV-010");
|
||||
Assert.True(rule.Enabled);
|
||||
Assert.Equal("warning", rule.Severity);
|
||||
Assert.False(rule.MarkSessionAsFailedDefault);
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
Transition("ac_to_battery", onAcPower: false, batteryPercent: 63),
|
||||
};
|
||||
|
||||
var outcome = await RunAsync(rule, events);
|
||||
|
||||
var result = Assert.Single(outcome.Results);
|
||||
Assert.Equal("ANALYZE-DEV-010", result.RuleId);
|
||||
Assert.Equal("warning", result.Severity);
|
||||
|
||||
var matched = AsDict(result.MatchedConditions["switched_to_battery"]);
|
||||
Assert.Equal("transition", AsString(matched["field"]));
|
||||
Assert.Equal("ac_to_battery", AsString(matched["value"]));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("battery_to_ac")]
|
||||
[InlineData("threshold_crossed")]
|
||||
public async Task ANALYZE_DEV_010_does_not_fire_on_other_transitions(string transition)
|
||||
{
|
||||
var rule = BuiltInAnalyzeRules.GetAll().First(r => r.RuleId == "ANALYZE-DEV-010");
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
Transition(transition, onAcPower: transition == "battery_to_ac", batteryPercent: 40),
|
||||
};
|
||||
|
||||
var outcome = await RunAsync(rule, events);
|
||||
Assert.Empty(outcome.Results);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Neither_rule_fires_on_the_startup_power_state_check()
|
||||
{
|
||||
// The one-shot startup probe event (power_state_check) is a different event type and must
|
||||
// not satisfy either rule, even when it reports a low battery on DC power.
|
||||
var rules = BuiltInAnalyzeRules.GetAll()
|
||||
.Where(r => r.RuleId is "ANALYZE-DEV-009" or "ANALYZE-DEV-010").ToList();
|
||||
Assert.Equal(2, rules.Count);
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
new()
|
||||
{
|
||||
EventId = Guid.NewGuid().ToString(),
|
||||
TenantId = TenantId,
|
||||
SessionId = SessionId,
|
||||
EventType = "power_state_check",
|
||||
Timestamp = DateTime.UtcNow,
|
||||
Sequence = 5,
|
||||
Data = new Dictionary<string, object>
|
||||
{
|
||||
["onAcPower"] = false,
|
||||
["hasBattery"] = true,
|
||||
["batteryPercent"] = 12,
|
||||
["isCharging"] = false,
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
foreach (var rule in rules)
|
||||
{
|
||||
var outcome = await RunAsync(rule, events);
|
||||
Assert.Empty(outcome.Results);
|
||||
}
|
||||
}
|
||||
|
||||
// ===== Event builders — mirror the PowerStateWatcherHost emit shape =====
|
||||
|
||||
private static EnrollmentEvent ThresholdCrossed(int thresholdPercent, int batteryPercent) => new()
|
||||
{
|
||||
EventId = Guid.NewGuid().ToString(),
|
||||
TenantId = TenantId,
|
||||
SessionId = SessionId,
|
||||
EventType = "power_state_change",
|
||||
Timestamp = DateTime.UtcNow,
|
||||
Sequence = 42,
|
||||
Data = new Dictionary<string, object>
|
||||
{
|
||||
["transition"] = "threshold_crossed",
|
||||
["thresholdPercent"] = thresholdPercent,
|
||||
["onAcPower"] = false,
|
||||
["batteryPercent"] = batteryPercent,
|
||||
["isCharging"] = false,
|
||||
["batteryLifeMinutes"] = 25,
|
||||
}
|
||||
};
|
||||
|
||||
private static EnrollmentEvent Transition(string transition, bool onAcPower, int batteryPercent) => new()
|
||||
{
|
||||
EventId = Guid.NewGuid().ToString(),
|
||||
TenantId = TenantId,
|
||||
SessionId = SessionId,
|
||||
EventType = "power_state_change",
|
||||
Timestamp = DateTime.UtcNow,
|
||||
Sequence = 43,
|
||||
Data = new Dictionary<string, object>
|
||||
{
|
||||
["transition"] = transition,
|
||||
["onAcPower"] = onAcPower,
|
||||
["batteryPercent"] = batteryPercent,
|
||||
["isCharging"] = onAcPower,
|
||||
["batteryLifeMinutes"] = 120,
|
||||
}
|
||||
};
|
||||
|
||||
private static Dictionary<string, object> AsDict(object o)
|
||||
{
|
||||
if (o is Dictionary<string, object> d) return d;
|
||||
throw new InvalidOperationException($"Expected Dictionary<string,object>, got {o?.GetType().Name ?? "null"}");
|
||||
}
|
||||
|
||||
private static string AsString(object o) => o?.ToString() ?? string.Empty;
|
||||
|
||||
private static async Task<AnalysisOutcome> RunAsync(AnalyzeRule rule, List<EnrollmentEvent> events)
|
||||
{
|
||||
var ruleRepo = new Mock<IRuleRepository>();
|
||||
ruleRepo.Setup(r => r.GetAnalyzeRulesAsync("global")).ReturnsAsync(new List<AnalyzeRule> { rule });
|
||||
ruleRepo.Setup(r => r.GetAnalyzeRulesAsync(TenantId)).ReturnsAsync(new List<AnalyzeRule>());
|
||||
ruleRepo.Setup(r => r.GetRuleStatesAsync(It.IsAny<string>())).ReturnsAsync(new Dictionary<string, RuleState>());
|
||||
ruleRepo.Setup(r => r.GetRuleResultsAsync(It.IsAny<string>(), It.IsAny<string>())).ReturnsAsync(new List<RuleResult>());
|
||||
|
||||
var sessionRepo = new Mock<ISessionRepository>();
|
||||
sessionRepo.Setup(s => s.GetSessionEventsStrictAsync(TenantId, SessionId, It.IsAny<int>())).ReturnsAsync(events);
|
||||
|
||||
var ruleService = new AnalyzeRuleService(ruleRepo.Object, NullLogger<AnalyzeRuleService>.Instance);
|
||||
var engine = new RuleEngine(ruleService, ruleRepo.Object, sessionRepo.Object, NullLogger<PowerStateRuleTests>.Instance);
|
||||
|
||||
return await engine.AnalyzeSessionAsync(TenantId, SessionId);
|
||||
}
|
||||
}
|
||||
@@ -90,6 +90,7 @@ public class SessionIndexFieldManifestTests
|
||||
["DiagnosticsBlobName"] = "diag-0001.zip",
|
||||
["DiagnosticsBlobDestination"] = "Hosted",
|
||||
["LastEventAt"] = new DateTimeOffset(StartedAt.AddMinutes(44)),
|
||||
["LastIngestAt"] = new DateTimeOffset(StartedAt.AddMinutes(46)),
|
||||
["ResumedAt"] = new DateTimeOffset(StartedAt.AddMinutes(10)),
|
||||
["StalledAt"] = new DateTimeOffset(StartedAt.AddMinutes(20)),
|
||||
["AvgApiLatencyMs"] = 513.5,
|
||||
|
||||
@@ -36,6 +36,8 @@ public class SessionStatsAggregationTests
|
||||
Assert.Equal(0, stats.FailedLastNDays);
|
||||
Assert.Equal(0, stats.SuccessRatePct);
|
||||
Assert.Equal(0, stats.AvgDurationMinutes);
|
||||
Assert.Equal(0, stats.MedianDurationMinutes);
|
||||
Assert.Equal(0, stats.P90DurationMinutes);
|
||||
Assert.Equal(0, stats.TotalToday);
|
||||
Assert.Equal(0, stats.FailedToday);
|
||||
}
|
||||
@@ -116,6 +118,29 @@ public class SessionStatsAggregationTests
|
||||
|
||||
// (30 + 10) / 2 = 20 minutes
|
||||
Assert.Equal(20, stats.AvgDurationMinutes);
|
||||
// Same Succeeded population feeds the percentile stats: sorted [600, 1800].
|
||||
Assert.Equal(10, stats.MedianDurationMinutes);
|
||||
Assert.Equal(30, stats.P90DurationMinutes);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Median_duration_is_robust_against_outliers_that_dominate_the_average()
|
||||
{
|
||||
// 8 typical 10-min enrollments plus a 2 h and a 24 h outlier: the mean explodes
|
||||
// to ~2.7 h while the median still answers "how long does a typical enrollment
|
||||
// take" — the reason the dashboard card leads with the median.
|
||||
var now = DateTime.UtcNow;
|
||||
var sessions = new List<SessionSummary>();
|
||||
for (int i = 0; i < 8; i++)
|
||||
sessions.Add(MakeSession(SessionStatus.Succeeded, now.AddHours(-1), 600));
|
||||
sessions.Add(MakeSession(SessionStatus.Succeeded, now.AddHours(-2), 7200));
|
||||
sessions.Add(MakeSession(SessionStatus.Succeeded, now.AddHours(-3), 86400));
|
||||
|
||||
var stats = TableStorageService.AggregateSessionStats(sessions, days: 7);
|
||||
|
||||
Assert.Equal(164, stats.AvgDurationMinutes); // (8*600 + 7200 + 86400) / 10 / 60
|
||||
Assert.Equal(10, stats.MedianDurationMinutes); // unmoved by the tail
|
||||
Assert.Equal(120, stats.P90DurationMinutes); // tail stays visible
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
+3
-1
@@ -172,7 +172,9 @@ public class SessionStatsProjectionEquivalenceTests
|
||||
Assert.Equal(full.SucceededLastNDays, projected.SucceededLastNDays);
|
||||
Assert.Equal(full.FailedLastNDays, projected.FailedLastNDays);
|
||||
Assert.Equal(full.SuccessRatePct, projected.SuccessRatePct);
|
||||
Assert.Equal(full.AvgDurationMinutes, projected.AvgDurationMinutes); // the duration-sensitive card
|
||||
Assert.Equal(full.AvgDurationMinutes, projected.AvgDurationMinutes); // duration-sensitive
|
||||
Assert.Equal(full.MedianDurationMinutes, projected.MedianDurationMinutes); // the headline card
|
||||
Assert.Equal(full.P90DurationMinutes, projected.P90DurationMinutes);
|
||||
Assert.Equal(full.TotalToday, projected.TotalToday);
|
||||
Assert.Equal(full.FailedToday, projected.FailedToday);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Shared.DataAccess;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Moq;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// ANALYZE-DEV-012 — pins the firing behaviour of the built-in that consumes the agent's
|
||||
/// <c>system_sleep_episode</c> events (SystemTimelineWatcherHost): informational, fires once
|
||||
/// (trigger single) when a completed sleep/hibernate/Modern-Standby episode of at least
|
||||
/// 5 minutes is observed. Single-required-condition on durationSeconds so the match is pinned
|
||||
/// to one event instance and the interpolated duration is the matched value. Also pins that the
|
||||
/// agent's numeric payload (long) satisfies the rule's string "300" via numeric gte.
|
||||
/// </summary>
|
||||
public class SleepEpisodeRuleTests
|
||||
{
|
||||
private const string TenantId = "a1b2c3d4-e5f6-7890-abcd-ef1234567890";
|
||||
private const string SessionId = "66666666-7777-8888-9999-aaaaaaaaaaaa";
|
||||
|
||||
[Fact]
|
||||
public async Task ANALYZE_DEV_012_fires_info_on_a_five_minute_episode()
|
||||
{
|
||||
var rule = BuiltInAnalyzeRules.GetAll().First(r => r.RuleId == "ANALYZE-DEV-012");
|
||||
Assert.True(rule.Enabled);
|
||||
Assert.Equal("info", rule.Severity);
|
||||
Assert.False(rule.MarkSessionAsFailedDefault); // environment observation, never a KO verdict
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
SleepEpisode(durationSeconds: 300, kind: "modern_standby"),
|
||||
};
|
||||
|
||||
var outcome = await RunAsync(rule, events);
|
||||
|
||||
var result = Assert.Single(outcome.Results);
|
||||
Assert.Equal("ANALYZE-DEV-012", result.RuleId);
|
||||
Assert.Equal("info", result.Severity);
|
||||
|
||||
// durationSeconds is the matched condition's dataField, so {{durationSeconds}} in the
|
||||
// explanation interpolates the matched value (the DEV-009 whitelist lesson).
|
||||
var matched = AsDict(result.MatchedConditions["slept_during_enrollment"]);
|
||||
Assert.Equal("durationSeconds", AsString(matched["field"]));
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(60)] // real but short episode — below the rule's 5-minute floor
|
||||
[InlineData(299)] // boundary: gte 300 must not fire at 299
|
||||
public async Task ANALYZE_DEV_012_does_not_fire_below_the_five_minute_floor(long durationSeconds)
|
||||
{
|
||||
var rule = BuiltInAnalyzeRules.GetAll().First(r => r.RuleId == "ANALYZE-DEV-012");
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
SleepEpisode(durationSeconds, kind: "sleep"),
|
||||
};
|
||||
|
||||
var outcome = await RunAsync(rule, events);
|
||||
Assert.Empty(outcome.Results);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ANALYZE_DEV_012_does_not_fire_on_a_clock_change_event()
|
||||
{
|
||||
// The sibling watcher event carries a large timeDeltaMs but is a different event type —
|
||||
// the rule must key on system_sleep_episode only.
|
||||
var rule = BuiltInAnalyzeRules.GetAll().First(r => r.RuleId == "ANALYZE-DEV-012");
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
new()
|
||||
{
|
||||
EventId = Guid.NewGuid().ToString(),
|
||||
TenantId = TenantId,
|
||||
SessionId = SessionId,
|
||||
EventType = "system_clock_changed",
|
||||
Timestamp = DateTime.UtcNow,
|
||||
Sequence = 7,
|
||||
Data = new Dictionary<string, object>
|
||||
{
|
||||
["timeDeltaMs"] = 7_200_000L,
|
||||
["durationSeconds"] = 900L, // adversarial: same field name on the wrong type
|
||||
["reason"] = 1,
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
var outcome = await RunAsync(rule, events);
|
||||
Assert.Empty(outcome.Results);
|
||||
}
|
||||
|
||||
// ===== Event builder — mirrors the SystemTimelineTracker emit shape =====
|
||||
|
||||
private static EnrollmentEvent SleepEpisode(long durationSeconds, string kind) => new()
|
||||
{
|
||||
EventId = Guid.NewGuid().ToString(),
|
||||
TenantId = TenantId,
|
||||
SessionId = SessionId,
|
||||
EventType = "system_sleep_episode",
|
||||
Timestamp = DateTime.UtcNow,
|
||||
Sequence = 42,
|
||||
Data = new Dictionary<string, object>
|
||||
{
|
||||
["kind"] = kind,
|
||||
["enteredAt"] = DateTime.UtcNow.AddSeconds(-durationSeconds).ToString("o"),
|
||||
["exitedAt"] = DateTime.UtcNow.ToString("o"),
|
||||
["durationSeconds"] = durationSeconds,
|
||||
["backfilled"] = false,
|
||||
}
|
||||
};
|
||||
|
||||
private static Dictionary<string, object> AsDict(object o)
|
||||
{
|
||||
if (o is Dictionary<string, object> d) return d;
|
||||
throw new InvalidOperationException($"Expected Dictionary<string,object>, got {o?.GetType().Name ?? "null"}");
|
||||
}
|
||||
|
||||
private static string AsString(object o) => o?.ToString() ?? string.Empty;
|
||||
|
||||
private static async Task<AnalysisOutcome> RunAsync(AnalyzeRule rule, List<EnrollmentEvent> events)
|
||||
{
|
||||
var ruleRepo = new Mock<IRuleRepository>();
|
||||
ruleRepo.Setup(r => r.GetAnalyzeRulesAsync("global")).ReturnsAsync(new List<AnalyzeRule> { rule });
|
||||
ruleRepo.Setup(r => r.GetAnalyzeRulesAsync(TenantId)).ReturnsAsync(new List<AnalyzeRule>());
|
||||
ruleRepo.Setup(r => r.GetRuleStatesAsync(It.IsAny<string>())).ReturnsAsync(new Dictionary<string, RuleState>());
|
||||
ruleRepo.Setup(r => r.GetRuleResultsAsync(It.IsAny<string>(), It.IsAny<string>())).ReturnsAsync(new List<RuleResult>());
|
||||
|
||||
var sessionRepo = new Mock<ISessionRepository>();
|
||||
sessionRepo.Setup(s => s.GetSessionEventsStrictAsync(TenantId, SessionId, It.IsAny<int>())).ReturnsAsync(events);
|
||||
|
||||
var ruleService = new AnalyzeRuleService(ruleRepo.Object, NullLogger<AnalyzeRuleService>.Instance);
|
||||
var engine = new RuleEngine(ruleService, ruleRepo.Object, sessionRepo.Object, NullLogger<SleepEpisodeRuleTests>.Instance);
|
||||
|
||||
return await engine.AnalyzeSessionAsync(TenantId, SessionId);
|
||||
}
|
||||
}
|
||||
@@ -120,6 +120,31 @@ public class TenantConfigTableSerializationTests
|
||||
Assert.Null(mapped.AbsoluteMaxSessionHours);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Roundtrip_EnableDoGroupIdAutoSet_SurvivesStoreAndMap()
|
||||
{
|
||||
var config = new TenantConfiguration
|
||||
{
|
||||
TenantId = TenantId,
|
||||
DomainName = "contoso.com",
|
||||
UpdatedBy = "admin@contoso.com",
|
||||
EnableDoGroupIdAutoSet = true
|
||||
};
|
||||
|
||||
var mapped = TableConfigRepository.ConvertFromTenantTableEntity(
|
||||
TableConfigRepository.ConvertToTenantTableEntity(config));
|
||||
|
||||
Assert.True(mapped.EnableDoGroupIdAutoSet);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Map_LegacyRow_WithoutDoGroupIdColumn_ReadsNull()
|
||||
{
|
||||
// null = "not set" → GetAgentConfigFunction serves the agent default (false).
|
||||
var entity = new TableEntity(TenantId, "config") { { "DomainName", "fabrikam.com" } };
|
||||
Assert.Null(TableConfigRepository.ConvertFromTenantTableEntity(entity).EnableDoGroupIdAutoSet);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Roundtrip_NotificationChannelsJson_SurvivesStoreAndMap()
|
||||
{
|
||||
|
||||
@@ -404,10 +404,12 @@ public class TimeAttributionCalculatorTests
|
||||
Assert.Equal(0, b.BlockingAppCount);
|
||||
}
|
||||
|
||||
// ── retry: occupancy spans to the LAST terminal event (audit Q1) ────────
|
||||
// ── retry: every ATTEMPT counts as active time, idle gaps between them don't ──
|
||||
// (2026-08 attempt-duration change: per-pass segment pairing replaced the old
|
||||
// first-start → last-terminal hull.)
|
||||
|
||||
[Fact]
|
||||
public void Retry_IntervalRunsFromFirstStart_ToLastTerminalEvent()
|
||||
public void Retry_ActiveSegmentsSummed_GapBetweenAttemptsExcluded()
|
||||
{
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
@@ -426,10 +428,41 @@ public class TimeAttributionCalculatorTests
|
||||
|
||||
AssertExactPartition(b);
|
||||
var interval = Assert.Single(b.BlockingApps);
|
||||
// First started (+6m) → LAST terminal (+12m) = 360 s. The payload timing would have
|
||||
// frozen at the first failure (+8m) — event timestamps are the only valid source.
|
||||
Assert.Equal(360, interval.Seconds);
|
||||
Assert.Equal(360, b.EspAppsOccupancySeconds);
|
||||
// Failed attempt (+6m→+8m = 120 s) + retry (+9m→+12m = 180 s) = 300 s active; the
|
||||
// idle minute between the attempts is not this app's time. The failed attempt still
|
||||
// counts — the payload timing would have frozen at the first failure (audit Q1).
|
||||
Assert.Equal(300, interval.Seconds);
|
||||
// Hull endpoints stay first-start → last-terminal for display/what-if.
|
||||
Assert.Equal(T0.AddMinutes(6), interval.StartUtc);
|
||||
Assert.Equal(T0.AddMinutes(12), interval.EndUtc);
|
||||
Assert.Equal(300, b.EspAppsOccupancySeconds);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void TwoPasses_OccupancyCountsActiveSegments_NotTheInterPassGap()
|
||||
{
|
||||
// Field case 44862cd3 ("Set TimeZone"): evaluation pass ends Skipped, the real
|
||||
// install runs much later. The app occupies the critical path only while the IME
|
||||
// actively processes it — the gap belongs to whatever installed in between.
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
Evt(T0, "agent_started", EnrollmentPhase.DeviceSetup),
|
||||
Evt(T0.AddMinutes(1), "esp_config_detected", data: EspLists(win32Ids: new[] { AppA })),
|
||||
Evt(T0.AddMinutes(2), "esp_phase_changed", EnrollmentPhase.AppsDevice),
|
||||
Evt(T0.AddMinutes(6), "app_install_started", data: AppData(AppA, "App A")),
|
||||
Evt(T0.AddMinutes(7), "app_install_completed", data: AppData(AppA, "App A", "Skipped")),
|
||||
Evt(T0.AddMinutes(40), "app_install_started", data: AppData(AppA, "App A")), // real install pass
|
||||
Evt(T0.AddMinutes(41), "app_install_completed", data: AppData(AppA, "App A", "Installed")),
|
||||
Evt(T0.AddMinutes(45), "enrollment_complete"),
|
||||
};
|
||||
|
||||
var b = TimeAttributionCalculator.Compute(
|
||||
Input(events, completedAt: T0.AddMinutes(45), durationSeconds: 2700))!;
|
||||
|
||||
AssertExactPartition(b);
|
||||
var interval = Assert.Single(b.BlockingApps);
|
||||
Assert.Equal(120, interval.Seconds); // 60 s eval + 60 s install — not 2100 s hull
|
||||
Assert.Equal(120, b.EspAppsOccupancySeconds); // gap not on the critical path either
|
||||
}
|
||||
|
||||
// ── reboots: spans from event gaps, never the detection event's stamp ───
|
||||
@@ -466,6 +499,136 @@ public class TimeAttributionCalculatorTests
|
||||
Assert.Equal(1200 - 300, b.GetSegmentTotals()[TimeAttributionSegments.EspApps]);
|
||||
}
|
||||
|
||||
// ── sleep spans: payload instants are authoritative, annotation not partition ─────────
|
||||
|
||||
[Fact]
|
||||
public void SleepSpan_ComesFromPayloadInstants_NotFromEventTimestamp()
|
||||
{
|
||||
var enteredAt = T0.AddMinutes(8);
|
||||
var exitedAt = T0.AddMinutes(14);
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
Evt(T0, "agent_started", EnrollmentPhase.DeviceSetup),
|
||||
Evt(T0.AddMinutes(5), "esp_phase_changed", EnrollmentPhase.AppsDevice),
|
||||
// Emitted at wake (its Timestamp is the wake moment, possibly clamped) — only the
|
||||
// payload carries the true episode bounds.
|
||||
Evt(T0.AddMinutes(14.2), "system_sleep_episode", data: new Dictionary<string, object>
|
||||
{
|
||||
["kind"] = "modern_standby",
|
||||
["enteredAt"] = enteredAt.ToString("o"),
|
||||
["exitedAt"] = exitedAt.ToString("o"),
|
||||
["durationSeconds"] = 360L,
|
||||
}),
|
||||
Evt(T0.AddMinutes(20), "enrollment_complete"),
|
||||
};
|
||||
|
||||
var b = TimeAttributionCalculator.Compute(
|
||||
Input(events, completedAt: T0.AddMinutes(20), durationSeconds: 1200))!;
|
||||
|
||||
AssertExactPartition(b); // sleep is an annotation — the wall-clock partition is untouched
|
||||
var span = Assert.Single(b.SleepSpans);
|
||||
Assert.Equal(enteredAt, span.StartUtc);
|
||||
Assert.Equal(exitedAt, span.EndUtc);
|
||||
Assert.Equal(360, span.Seconds);
|
||||
Assert.Equal(360, b.SleepSeconds);
|
||||
Assert.Equal("modern_standby", span.Kind);
|
||||
Assert.Equal(TimeAttributionSegments.EspApps, span.SegmentKey);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void SleepSpan_DuplicateEpisode_NotDoubleCounted()
|
||||
{
|
||||
var enteredAt = T0.AddMinutes(8);
|
||||
var exitedAt = T0.AddMinutes(10);
|
||||
Dictionary<string, object> Episode() => new Dictionary<string, object>
|
||||
{
|
||||
["kind"] = "sleep",
|
||||
["enteredAt"] = enteredAt.ToString("o"),
|
||||
["exitedAt"] = exitedAt.ToString("o"),
|
||||
};
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
Evt(T0, "agent_started", EnrollmentPhase.DeviceSetup),
|
||||
// Same episode observed twice (e.g. backfilled again after an agent restart from a
|
||||
// different event-log record).
|
||||
Evt(T0.AddMinutes(10.1), "system_sleep_episode", data: Episode()),
|
||||
Evt(T0.AddMinutes(12), "system_sleep_episode", data: Episode()),
|
||||
Evt(T0.AddMinutes(20), "enrollment_complete"),
|
||||
};
|
||||
|
||||
var b = TimeAttributionCalculator.Compute(
|
||||
Input(events, completedAt: T0.AddMinutes(20), durationSeconds: 1200))!;
|
||||
|
||||
Assert.Single(b.SleepSpans);
|
||||
Assert.Equal(120, b.SleepSeconds);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void SleepSpan_MissingOrUnparseablePayload_Ignored()
|
||||
{
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
Evt(T0, "agent_started", EnrollmentPhase.DeviceSetup),
|
||||
Evt(T0.AddMinutes(5), "system_sleep_episode", data: new Dictionary<string, object>
|
||||
{
|
||||
["kind"] = "sleep",
|
||||
["enteredAt"] = "not-a-timestamp",
|
||||
["exitedAt"] = T0.AddMinutes(5).ToString("o"),
|
||||
}),
|
||||
Evt(T0.AddMinutes(6), "system_sleep_episode", data: new Dictionary<string, object>
|
||||
{
|
||||
["kind"] = "sleep", // bounds missing entirely
|
||||
}),
|
||||
// exitedAt <= enteredAt — pathological, no claim.
|
||||
Evt(T0.AddMinutes(7), "system_sleep_episode", data: new Dictionary<string, object>
|
||||
{
|
||||
["kind"] = "sleep",
|
||||
["enteredAt"] = T0.AddMinutes(7).ToString("o"),
|
||||
["exitedAt"] = T0.AddMinutes(6).ToString("o"),
|
||||
}),
|
||||
Evt(T0.AddMinutes(20), "enrollment_complete"),
|
||||
};
|
||||
|
||||
var b = TimeAttributionCalculator.Compute(
|
||||
Input(events, completedAt: T0.AddMinutes(20), durationSeconds: 1200))!;
|
||||
|
||||
Assert.Empty(b.SleepSpans);
|
||||
Assert.Equal(0, b.SleepSeconds);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhiteGlove_SleepAcrossPause_CountsOnlyInWindowFlanks()
|
||||
{
|
||||
var part1End = T0.AddMinutes(20);
|
||||
var resumedAt = T0.AddDays(2);
|
||||
var completedAt = resumedAt.AddMinutes(10);
|
||||
|
||||
var events = new List<EnrollmentEvent>
|
||||
{
|
||||
Evt(T0, "agent_started", EnrollmentPhase.DeviceSetup),
|
||||
Evt(T0.AddMinutes(2), "esp_phase_changed", EnrollmentPhase.AppsDevice),
|
||||
Evt(part1End, "whiteglove_part1_complete"),
|
||||
// Sealed device slept through the pause: episode starts 1 min before the seal and
|
||||
// ends 30 s into part 2 — only the in-window flanks may count (90 s), never the
|
||||
// ~2-day pause (RebootSpan precedent).
|
||||
Evt(resumedAt.AddSeconds(35), "system_sleep_episode", data: new Dictionary<string, object>
|
||||
{
|
||||
["kind"] = "hibernate",
|
||||
["enteredAt"] = part1End.AddMinutes(-1).ToString("o"),
|
||||
["exitedAt"] = resumedAt.AddSeconds(30).ToString("o"),
|
||||
}),
|
||||
Evt(completedAt, "enrollment_complete"),
|
||||
};
|
||||
|
||||
var b = TimeAttributionCalculator.Compute(
|
||||
Input(events, completedAt, durationSeconds: 1800, wg: true, resumedAt: resumedAt))!;
|
||||
|
||||
AssertExactPartition(b);
|
||||
var span = Assert.Single(b.SleepSpans);
|
||||
Assert.Equal(90, span.Seconds);
|
||||
Assert.Equal(90, b.SleepSeconds);
|
||||
}
|
||||
|
||||
// ── WhiteGlove pause vs. intervals/reboots (Codex review P1): the pause lies INSIDE
|
||||
// the windows' hull, so hull-clamping alone would count it — a reboot bracketing the
|
||||
// pause must contribute only its in-window flanks, never the pause itself ─────────────
|
||||
|
||||
+12
@@ -46,6 +46,11 @@ public class TimeAttributionPersistenceAndAggregationTests
|
||||
{
|
||||
new() { StartUtc = T0.AddMinutes(10), EndUtc = T0.AddMinutes(14), Seconds = 240, SegmentKey = TimeAttributionSegments.EspApps },
|
||||
},
|
||||
SleepSeconds = 360,
|
||||
SleepSpans = new List<SleepSpan>
|
||||
{
|
||||
new() { StartUtc = T0.AddMinutes(20), EndUtc = T0.AddMinutes(26), Seconds = 360, SegmentKey = TimeAttributionSegments.EspApps, Kind = "modern_standby" },
|
||||
},
|
||||
BlockingApps = new List<BlockingAppInterval>
|
||||
{
|
||||
new() { AppId = AppX, AppName = "App X", StartUtc = T0.AddMinutes(6), EndUtc = T0.AddMinutes(16), Seconds = 600 },
|
||||
@@ -77,6 +82,13 @@ public class TimeAttributionPersistenceAndAggregationTests
|
||||
Assert.Equal(TimeAttributionSegments.EspApps, reboot.SegmentKey);
|
||||
Assert.Equal(240, reboot.Seconds);
|
||||
|
||||
Assert.Equal(360, mapped.SleepSeconds);
|
||||
var sleep = Assert.Single(mapped.SleepSpans);
|
||||
Assert.Equal("modern_standby", sleep.Kind);
|
||||
Assert.Equal(TimeAttributionSegments.EspApps, sleep.SegmentKey);
|
||||
Assert.Equal(T0.AddMinutes(20), sleep.StartUtc);
|
||||
Assert.Equal(360, sleep.Seconds);
|
||||
|
||||
var app = Assert.Single(mapped.BlockingApps);
|
||||
Assert.Equal(AppX, app.AppId);
|
||||
Assert.Equal("App X", app.AppName);
|
||||
|
||||
+138
@@ -0,0 +1,138 @@
|
||||
using AutopilotMonitor.Functions.Services.Vulnerability;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Pins <see cref="VulnerabilityCorrelateHandler.BuildVulnerabilityReportEvent"/> — the
|
||||
/// backend-materialized <c>vulnerability_report</c> event. The load-bearing logic is the
|
||||
/// Sequence assignment: the correlation runs async off a queue, so without one the event
|
||||
/// defaulted to Sequence 0 and sorted to the TOP of the timeline (a −13m time-jump badge
|
||||
/// on a report that actually arrived last). Default is max+1 (sorts LAST, same contract as
|
||||
/// BuildSessionTimeoutEvent / BuildAgentEmergencyBreakEvent); WhiteGlove Part 1 instead
|
||||
/// anchors at the Part-1 terminal markers so a late Part-1 report can never cross the
|
||||
/// web timeline's sequence-based Pre-Provisioning / User Enrollment split and render as
|
||||
/// the start of User Enrollment.
|
||||
/// </summary>
|
||||
public class VulnerabilityReportEventSequenceTests
|
||||
{
|
||||
private const string TenantId = "a1b2c3d4-e5f6-7890-abcd-ef1234567890";
|
||||
private const string SessionId = "b2c3d4e5-f6a7-8901-bcde-f12345678901";
|
||||
|
||||
private static readonly DateTime NowUtc = new(2026, 8, 21, 13, 25, 46, DateTimeKind.Utc);
|
||||
|
||||
private static Dictionary<string, object> Report(string risk = "high") => new()
|
||||
{
|
||||
["scan_summary"] = new Dictionary<string, object> { ["overall_risk"] = risk },
|
||||
["findings"] = new List<Dictionary<string, object>>(),
|
||||
};
|
||||
|
||||
private static EnrollmentEvent Build(
|
||||
string? phaseLabel, IReadOnlyList<EnrollmentEvent> existing, Dictionary<string, object>? report = null)
|
||||
=> VulnerabilityCorrelateHandler.BuildVulnerabilityReportEvent(
|
||||
TenantId, SessionId, report ?? Report(), phaseLabel, existing, NowUtc);
|
||||
|
||||
[Fact]
|
||||
public void NonWhiteGlove_SortsOnePastTheSessionMax()
|
||||
{
|
||||
var existing = new List<EnrollmentEvent>
|
||||
{
|
||||
new() { Sequence = 3 },
|
||||
new() { Sequence = 41 }, // max — even though out of array order
|
||||
new() { Sequence = 17 },
|
||||
};
|
||||
|
||||
Assert.Equal(42, Build(phaseLabel: null, existing).Sequence);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Sequence_IsOne_WhenNoPriorEvents()
|
||||
{
|
||||
Assert.Equal(1, Build(null, new List<EnrollmentEvent>()).Sequence);
|
||||
Assert.Equal(1, Build(null, null!).Sequence);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhiteGlovePart1_AnchorsAtPart1Markers_NotPastThePart2Boot()
|
||||
{
|
||||
// Part 2 already running when the async Part-1 scan lands. The web split is
|
||||
// agent_started(50) - 1 = 49; max+1 would be 61 and cross it.
|
||||
var existing = new List<EnrollmentEvent>
|
||||
{
|
||||
new() { Sequence = 10, EventType = "esp_phase_changed" },
|
||||
new() { Sequence = 20, EventType = "whiteglove_complete" },
|
||||
new() { Sequence = 30, EventType = "software_inventory_analysis" },
|
||||
new() { Sequence = 40, EventType = "whiteglove_part1_complete" },
|
||||
new() { Sequence = 50, EventType = "agent_started" }, // Part-2 boot
|
||||
new() { Sequence = 60, EventType = "whiteglove_resumed" },
|
||||
};
|
||||
|
||||
var evt = Build("device_setup", existing);
|
||||
|
||||
// Anchored at the highest Part-1 terminal marker; the timestamp tie-break in the
|
||||
// web sort places it right behind that marker, still inside Pre-Provisioning.
|
||||
Assert.Equal(40, evt.Sequence);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhiteGlovePart1_UsesWhiteGloveComplete_WhenPart1CompleteMarkerMissing()
|
||||
{
|
||||
var existing = new List<EnrollmentEvent>
|
||||
{
|
||||
new() { Sequence = 20, EventType = "whiteglove_complete" },
|
||||
new() { Sequence = 50, EventType = "agent_started" },
|
||||
};
|
||||
|
||||
Assert.Equal(20, Build("device_setup", existing).Sequence);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhiteGlovePart1_FallsBackToMaxPlusOne_WithoutAnyMarker()
|
||||
{
|
||||
// No marker ⇒ the web derives no split either, so sorting last is safe.
|
||||
var existing = new List<EnrollmentEvent> { new() { Sequence = 7, EventType = "agent_started" } };
|
||||
|
||||
Assert.Equal(8, Build("device_setup", existing).Sequence);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void WhiteGlovePart2_SortsOnePastTheSessionMax()
|
||||
{
|
||||
var existing = new List<EnrollmentEvent>
|
||||
{
|
||||
new() { Sequence = 40, EventType = "whiteglove_part1_complete" },
|
||||
new() { Sequence = 90, EventType = "esp_phase_changed" },
|
||||
};
|
||||
|
||||
Assert.Equal(91, Build("user_enrollment", existing).Sequence);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Shape_IsUnchangedByTheSequenceFix()
|
||||
{
|
||||
var evt = Build(null, new List<EnrollmentEvent>());
|
||||
|
||||
Assert.Equal("vulnerability_report", evt.EventType);
|
||||
Assert.Equal(AutopilotMonitor.Shared.Constants.EventTypes.VulnerabilityReport, evt.EventType);
|
||||
Assert.Equal("VulnerabilityCorrelationService", evt.Source);
|
||||
Assert.Equal(EventSeverity.Error, evt.Severity); // risk=high → Error
|
||||
// Non-phase event: must stay Unknown so the UI timeline doesn't render a phantom phase boundary.
|
||||
Assert.Equal(EnrollmentPhase.Unknown, evt.Phase);
|
||||
Assert.Equal(NowUtc, evt.Timestamp);
|
||||
Assert.Equal(TenantId, evt.TenantId);
|
||||
Assert.Equal(SessionId, evt.SessionId);
|
||||
Assert.Equal("Vulnerability scan complete (risk=high).", evt.Message);
|
||||
Assert.Equal(true, evt.Data["reportAvailable"]);
|
||||
Assert.Equal("single", evt.Data["phase"]);
|
||||
Assert.IsType<Dictionary<string, object>>(evt.Data["scan_summary"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Data_Phase_CarriesTheLabel()
|
||||
{
|
||||
var existing = new List<EnrollmentEvent> { new() { Sequence = 20, EventType = "whiteglove_complete" } };
|
||||
|
||||
Assert.Equal("device_setup", Build("device_setup", existing).Data["phase"]);
|
||||
Assert.Equal("user_enrollment", Build("user_enrollment", existing).Data["phase"]);
|
||||
}
|
||||
}
|
||||
+2
@@ -620,6 +620,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
{ "RebootDelaySeconds", config.RebootDelaySeconds },
|
||||
{ "EnableGeoLocation", config.EnableGeoLocation },
|
||||
{ "EnableTimezoneAutoSet", config.EnableTimezoneAutoSet },
|
||||
{ "EnableDoGroupIdAutoSet", config.EnableDoGroupIdAutoSet },
|
||||
{ "NtpServer", config.NtpServer },
|
||||
{ "EnableImeMatchLog", config.EnableImeMatchLog },
|
||||
{ "EnableGatherRuleDebugLog", config.EnableGatherRuleDebugLog },
|
||||
@@ -727,6 +728,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
RebootDelaySeconds = entity.GetInt32("RebootDelaySeconds"),
|
||||
EnableGeoLocation = entity.GetBoolean("EnableGeoLocation"),
|
||||
EnableTimezoneAutoSet = entity.GetBoolean("EnableTimezoneAutoSet"),
|
||||
EnableDoGroupIdAutoSet = entity.GetBoolean("EnableDoGroupIdAutoSet"),
|
||||
NtpServer = string.IsNullOrWhiteSpace(entity.GetString("NtpServer")) ? "time.windows.com" : entity.GetString("NtpServer"),
|
||||
EnableImeMatchLog = entity.GetBoolean("EnableImeMatchLog"),
|
||||
EnableGatherRuleDebugLog = entity.GetBoolean("EnableGatherRuleDebugLog"),
|
||||
|
||||
@@ -137,6 +137,18 @@ namespace AutopilotMonitor.Functions.Functions.Admin
|
||||
|
||||
private async Task<(int deleted, int written, int orphanStatesGcd, int sunsetSkipped)> ReseedGatherAsync(List<AutopilotMonitor.Shared.Models.GatherRule> rules)
|
||||
{
|
||||
// Global-partition rules must stay inside the reserved built-in namespace — an ID
|
||||
// outside it (e.g. GATHER-CUSTOM-*) would silently shadow every tenant's same-ID
|
||||
// custom rule at merge time (global wins). Reject instead of import.
|
||||
var rejectedGather = rules.Where(r => !AutopilotMonitor.Shared.Models.RuleIdPolicy.IsReservedBuiltInId(r.RuleId)).ToList();
|
||||
if (rejectedGather.Count > 0)
|
||||
{
|
||||
_logger.LogError(
|
||||
"GitHub reseed gather: rejected {Count} rule(s) outside the reserved built-in namespace: {Ids}",
|
||||
rejectedGather.Count, string.Join(", ", rejectedGather.Select(r => r.RuleId)));
|
||||
rules = rules.Except(rejectedGather).ToList();
|
||||
}
|
||||
|
||||
var existing = await _ruleRepo.GetGatherRulesAsync("global");
|
||||
var newCatalogIds = rules.Select(r => r.RuleId).ToHashSet(System.StringComparer.Ordinal);
|
||||
// A rule the deployed binary already ships IDENTICALLY is embedded-provenance; one whose
|
||||
@@ -199,6 +211,18 @@ namespace AutopilotMonitor.Functions.Functions.Admin
|
||||
|
||||
private async Task<(int deleted, int written, int orphanStatesGcd, int sunsetSkipped)> ReseedAnalyzeAsync(List<AutopilotMonitor.Shared.Models.AnalyzeRule> rules)
|
||||
{
|
||||
// Global-partition rules must stay inside the reserved built-in namespace — an ID
|
||||
// outside it (e.g. ANALYZE-CUSTOM-*) would silently shadow every tenant's same-ID
|
||||
// custom rule at merge time (global wins). Reject instead of import.
|
||||
var rejectedAnalyze = rules.Where(r => !AutopilotMonitor.Shared.Models.RuleIdPolicy.IsReservedBuiltInId(r.RuleId)).ToList();
|
||||
if (rejectedAnalyze.Count > 0)
|
||||
{
|
||||
_logger.LogError(
|
||||
"GitHub reseed analyze: rejected {Count} rule(s) outside the reserved built-in namespace: {Ids}",
|
||||
rejectedAnalyze.Count, string.Join(", ", rejectedAnalyze.Select(r => r.RuleId)));
|
||||
rules = rules.Except(rejectedAnalyze).ToList();
|
||||
}
|
||||
|
||||
var existing = await _ruleRepo.GetAnalyzeRulesAsync("global");
|
||||
var newCatalogIds = rules.Select(r => r.RuleId).ToHashSet(System.StringComparer.Ordinal);
|
||||
// A rule the deployed binary already ships IDENTICALLY is embedded-provenance; one whose
|
||||
|
||||
@@ -513,7 +513,10 @@ namespace AutopilotMonitor.Functions.Functions.Apps
|
||||
exitCode = s.ExitCode,
|
||||
attemptNumber = s.AttemptNumber,
|
||||
startedAt = s.StartedAt,
|
||||
durationSeconds = s.DurationSeconds
|
||||
durationSeconds = s.DurationSeconds,
|
||||
// 2+ = the IME processed this app in multiple passes (device-ESP
|
||||
// evaluation + real install) — explains a completion far after startedAt.
|
||||
installPassCount = s.InstallPassCount
|
||||
};
|
||||
}).ToList();
|
||||
|
||||
|
||||
@@ -274,7 +274,7 @@ namespace AutopilotMonitor.Functions.Functions.Config
|
||||
var response = req.CreateResponse(HttpStatusCode.OK);
|
||||
await response.WriteAsJsonAsync(new AgentConfigResponse
|
||||
{
|
||||
ConfigVersion = 37, // EnableEspContinueAnywayObservation (Continue-Anyway observation mode)
|
||||
ConfigVersion = 40, // EnableDoGroupIdAutoSet (Delivery Optimization group ID from network fingerprint)
|
||||
UploadIntervalSeconds = Shared.Constants.DefaultUploadIntervalSeconds,
|
||||
SelfDestructOnComplete = tenantConfig.SelfDestructOnComplete ?? true,
|
||||
KeepLogFile = tenantConfig.KeepLogFile ?? false,
|
||||
@@ -312,6 +312,7 @@ namespace AutopilotMonitor.Functions.Functions.Config
|
||||
AllowAgentDowngrade = adminConfig.AllowAgentDowngrade,
|
||||
NtpServer = string.IsNullOrEmpty(tenantConfig.NtpServer) ? "time.windows.com" : tenantConfig.NtpServer,
|
||||
EnableTimezoneAutoSet = tenantConfig.EnableTimezoneAutoSet ?? false,
|
||||
EnableDoGroupIdAutoSet = tenantConfig.EnableDoGroupIdAutoSet ?? false,
|
||||
SendTraceEvents = tenantConfig.SendTraceEvents,
|
||||
UnrestrictedMode = TenantEntitlementService.IsUnrestrictedModeActive(tenantConfig, DateTime.UtcNow),
|
||||
GatherRules = gatherRules,
|
||||
|
||||
@@ -35,7 +35,12 @@ public sealed class AppVersionDurationRegressionFinding
|
||||
/// </list>
|
||||
/// Measured population: Succeeded, non-skip terminal state, plausible observed duration
|
||||
/// (<see cref="MetricsMath.HasMeasuredDuration"/>), no AppId collision, non-empty
|
||||
/// AppVersion. Version ORDER comes from first-seen install time (min StartedAt) — version
|
||||
/// AppVersion, and — cutover gate of the 2026-08 attempt-duration change — an
|
||||
/// attempt-anchored duration (<c>LastAttemptStartedAt</c> present). Rows from before the
|
||||
/// change carry the old full-span semantics; a median over a mixed population would both
|
||||
/// mask real regressions (inflated previous vs. attempt-short current) and falsely re-arm
|
||||
/// active episodes, so pre-cutover rows are excluded on BOTH sides of every comparison.
|
||||
/// Version ORDER comes from first-seen install time (min StartedAt) — version
|
||||
/// strings are never sorted lexically ("9.1" vs "2024.10" would lie). "Previous" is the
|
||||
/// version with the latest first-seen strictly before the current version's first-seen;
|
||||
/// with parallel ring rollouts that can be a concurrently-deployed version — accepted for
|
||||
@@ -149,6 +154,9 @@ public static class AppVersionRegressionRadar
|
||||
if (summary.Status != "Succeeded") continue;
|
||||
if (MetricsMath.IsSkipTerminalState(summary)) continue;
|
||||
if (!MetricsMath.HasMeasuredDuration(summary)) continue;
|
||||
// Attempt-semantics cutover gate (see class doc): only attempt-anchored rows
|
||||
// enter the duration population, on both sides of every comparison.
|
||||
if (!summary.LastAttemptStartedAt.HasValue) continue;
|
||||
|
||||
if (!groups.TryGetValue(summary.AppName, out var list))
|
||||
{
|
||||
|
||||
@@ -35,11 +35,14 @@ public static class MetricsMath
|
||||
/// <summary>
|
||||
/// Upper plausibility bound for an observed install duration: 6 h, the agent's default
|
||||
/// max observation lifetime (AgentMaxLifetimeMinutes = 360) — nothing longer can have
|
||||
/// been continuously watched. Rows above it are emission artifacts, not slow installs:
|
||||
/// a batch emission at session end pins CompletedAt far past the real finish (verified
|
||||
/// in production 2026-07-27: app rows carrying 60 175 s in a session whose own wall
|
||||
/// clock was 2 697 s; 490 rows > 6 h overall, worst 117 days — dominating the
|
||||
/// slowest-apps ranking with six-figure averages).
|
||||
/// been continuously watched. Historically this mostly caught full-span pathologies
|
||||
/// (pre-2026-08 rows measured first observation → last terminal across all install
|
||||
/// passes; verified in production 2026-07-27: app rows carrying 60 175 s in a session
|
||||
/// whose own wall clock was 2 697 s; 490 rows > 6 h overall, worst 117 days). Since
|
||||
/// the 2026-08 attempt-duration change DurationSeconds measures the last attempt, so
|
||||
/// far fewer rows trip this bound — it remains as the guard against back-stamped
|
||||
/// completions, where a single ATTEMPT longer than the agent's own lifetime is still
|
||||
/// impossible to have observed.
|
||||
/// </summary>
|
||||
public const int MaxPlausibleInstallDurationSeconds = 6 * 3600;
|
||||
|
||||
@@ -168,14 +171,15 @@ public static class MetricsMath
|
||||
/// drained up to 200k raw sessions into the browser and ran these aggregations on the main
|
||||
/// thread. Success rate follows the SLA convention: Succeeded / (Succeeded + Failed) — finished
|
||||
/// enrollments only, so in-flight sessions and Incomplete (terminal, non-failure) never dilute
|
||||
/// it. Average duration counts every non-in-progress session that carries a positive duration
|
||||
/// (failures included).
|
||||
/// it. Duration stats (average, median, P90) count every non-in-progress session that carries a
|
||||
/// positive duration (failures included); the cards lead with the median because enrollment
|
||||
/// durations are heavily right-skewed and a handful of multi-hour outliers make the mean
|
||||
/// meaningless as a "typical enrollment" answer.
|
||||
/// </summary>
|
||||
public static FleetHealthMetrics BuildFleetHealthPayload(IReadOnlyList<SessionSummary> sessions, int days)
|
||||
{
|
||||
int succeeded = 0, failed = 0, inProgress = 0, incomplete = 0;
|
||||
long completedDurationSeconds = 0;
|
||||
int completedWithDurationCount = 0;
|
||||
var completedDurationSeconds = new List<double>();
|
||||
|
||||
foreach (var s in sessions)
|
||||
{
|
||||
@@ -188,12 +192,10 @@ public static class MetricsMath
|
||||
}
|
||||
|
||||
if (s.Status != SessionStatus.InProgress && s.DurationSeconds is int d && d > 0)
|
||||
{
|
||||
completedDurationSeconds += d;
|
||||
completedWithDurationCount++;
|
||||
}
|
||||
completedDurationSeconds.Add(d);
|
||||
}
|
||||
|
||||
completedDurationSeconds.Sort(); // Percentile() expects ascending order
|
||||
int total = sessions.Count;
|
||||
int finished = succeeded + failed;
|
||||
var stats = new FleetHealthStats
|
||||
@@ -204,9 +206,11 @@ public static class MetricsMath
|
||||
InProgress = inProgress,
|
||||
Incomplete = incomplete,
|
||||
SuccessRate = finished > 0 ? Math.Round((double)succeeded / finished * 100, 1) : 0,
|
||||
AvgDurationMinutes = completedWithDurationCount > 0
|
||||
? (int)Math.Round((double)completedDurationSeconds / completedWithDurationCount / 60.0, MidpointRounding.AwayFromZero)
|
||||
AvgDurationMinutes = completedDurationSeconds.Count > 0
|
||||
? (int)Math.Round(completedDurationSeconds.Average() / 60.0, MidpointRounding.AwayFromZero)
|
||||
: 0,
|
||||
MedianDurationMinutes = SecondsToMinutes(Percentile(completedDurationSeconds, 50)),
|
||||
P90DurationMinutes = SecondsToMinutes(Percentile(completedDurationSeconds, 90)),
|
||||
};
|
||||
|
||||
return new FleetHealthMetrics
|
||||
@@ -369,6 +373,10 @@ public static class MetricsMath
|
||||
/// rounded to one decimal place. Callers MUST pass values pre-sorted ascending.
|
||||
/// Returns 0 for an empty list.
|
||||
/// </summary>
|
||||
/// <summary>Rounds a duration in seconds to whole minutes (away from zero, matching the avg).</summary>
|
||||
public static int SecondsToMinutes(double seconds)
|
||||
=> (int)Math.Round(seconds / 60.0, MidpointRounding.AwayFromZero);
|
||||
|
||||
public static double Percentile(List<double> sortedValues, int percentile)
|
||||
{
|
||||
if (sortedValues.Count == 0) return 0;
|
||||
|
||||
@@ -72,7 +72,9 @@ public static class TimeAttributionCalculator
|
||||
/// <summary>Bump on any semantic change so aggregates never mix definitions (truthfulness rule 8).</summary>
|
||||
// v2: PriorEnrollmentResidue flag — sessions whose phase anchors are distorted by
|
||||
// pre-enrollment state on disk are now flagged and excluded from fleet segment stats.
|
||||
public const int CurrentVersion = 2;
|
||||
// v3: SleepSpans — completed sleep/standby episodes (system_sleep_episode) as cross-cutting
|
||||
// annotations mirroring RebootSpans; the maintenance sweep recomputes older rows.
|
||||
public const int CurrentVersion = 3;
|
||||
|
||||
/// <summary>
|
||||
/// Enrollment class for fleet aggregation — classes are NEVER mixed in one aggregate (a
|
||||
@@ -111,6 +113,10 @@ public static class TimeAttributionCalculator
|
||||
public string AppName = string.Empty;
|
||||
public DateTime? FirstStart;
|
||||
public DateTime? LastTerminal;
|
||||
/// <summary>Start of the currently open activity segment (start seen, terminal pending).</summary>
|
||||
public DateTime? OpenStart;
|
||||
/// <summary>Closed activity segments: one per fully-observed install pass/attempt (start → terminal).</summary>
|
||||
public List<(DateTime Start, DateTime End)> Segments = new();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -152,16 +158,19 @@ public static class TimeAttributionCalculator
|
||||
else if (blockingSets.IsTruncated)
|
||||
flags |= TimeAttributionFlags.BlockingSetTruncated;
|
||||
|
||||
var (blockingApps, blockingAppCount) = BuildBlockingAppIntervals(events, blockingSets, windows, ref flags);
|
||||
var (blockingApps, blockingAppCount, blockingAppSegments) = BuildBlockingAppIntervals(events, blockingSets, windows, ref flags);
|
||||
|
||||
int? occupancy = null;
|
||||
if (blockingSets != null && blockingSets.ListedCount > 0)
|
||||
{
|
||||
var espAppsSpans = spans.Where(s => s.SegmentKey == TimeAttributionSegments.EspApps).ToList();
|
||||
occupancy = ComputeOccupancySeconds(blockingApps, espAppsSpans);
|
||||
// Occupancy merges the per-pass SEGMENTS, not the per-app hulls — a hull spans
|
||||
// the idle gap between install passes, which is not occupied by this app.
|
||||
occupancy = ComputeOccupancySeconds(blockingAppSegments, espAppsSpans);
|
||||
}
|
||||
|
||||
var rebootSpans = BuildRebootSpans(events, windows, spans);
|
||||
var sleepSpans = BuildSleepSpans(events, windows, spans);
|
||||
|
||||
if (events.Any(e => e.EventType == Constants.EventTypes.AgentLateStart))
|
||||
flags |= TimeAttributionFlags.PartialObservation;
|
||||
@@ -185,6 +194,8 @@ public static class TimeAttributionCalculator
|
||||
UnattributedSeconds = unattributed,
|
||||
RebootSeconds = rebootSpans.Sum(r => r.Seconds),
|
||||
RebootSpans = rebootSpans,
|
||||
SleepSeconds = sleepSpans.Sum(s => s.Seconds),
|
||||
SleepSpans = sleepSpans,
|
||||
BlockingApps = blockingApps
|
||||
.OrderByDescending(a => a.Seconds)
|
||||
.ThenBy(a => a.AppName, StringComparer.OrdinalIgnoreCase)
|
||||
@@ -403,12 +414,12 @@ public static class TimeAttributionCalculator
|
||||
|
||||
// ── blocking-app intervals + occupancy ──────────────────────────────────
|
||||
|
||||
private static (List<BlockingAppInterval> Intervals, int MatchedCount) BuildBlockingAppIntervals(
|
||||
private static (List<BlockingAppInterval> Intervals, int MatchedCount, List<BlockingAppInterval> OccupancySegments) BuildBlockingAppIntervals(
|
||||
List<EnrollmentEvent> events, EspBlockingSets? sets, List<Window> windows, ref TimeAttributionFlags flags)
|
||||
{
|
||||
var intervals = new List<BlockingAppInterval>();
|
||||
if (sets == null || sets.ListedCount == 0)
|
||||
return (intervals, 0);
|
||||
return (intervals, 0, intervals);
|
||||
|
||||
// Fold per-app activity from EVENT timestamps (never payload timing — audit Q1).
|
||||
var apps = new Dictionary<string, AppActivity>(StringComparer.OrdinalIgnoreCase);
|
||||
@@ -436,36 +447,75 @@ public static class TimeAttributionCalculator
|
||||
activity.AppName = nameObj?.ToString()?.Trim() ?? string.Empty;
|
||||
}
|
||||
|
||||
if (isStart && (!activity.FirstStart.HasValue || evt.Timestamp < activity.FirstStart.Value))
|
||||
activity.FirstStart = evt.Timestamp;
|
||||
// LAST terminal wins — an IME retry (Error → retry → Installed) extends the
|
||||
// occupancy to the final outcome instead of freezing at the first failure.
|
||||
if (isTerminal && (!activity.LastTerminal.HasValue || evt.Timestamp > activity.LastTerminal.Value))
|
||||
activity.LastTerminal = evt.Timestamp;
|
||||
if (isStart)
|
||||
{
|
||||
if (!activity.FirstStart.HasValue || evt.Timestamp < activity.FirstStart.Value)
|
||||
activity.FirstStart = evt.Timestamp;
|
||||
// A download start followed by an install start is ONE active window —
|
||||
// the earliest open start survives until a terminal closes the segment.
|
||||
activity.OpenStart ??= evt.Timestamp;
|
||||
}
|
||||
if (isTerminal)
|
||||
{
|
||||
if (!activity.LastTerminal.HasValue || evt.Timestamp > activity.LastTerminal.Value)
|
||||
activity.LastTerminal = evt.Timestamp;
|
||||
// Segment pairing (2026-08 attempt-duration change): each fully-observed
|
||||
// pass/attempt (start → terminal, sequence-ordered) is its own activity
|
||||
// segment. The IME processes the app list in multiple passes; the idle gap
|
||||
// between passes belongs to the apps actually running then, not to this one.
|
||||
// A failed attempt still counts as active time — the retry opens a new
|
||||
// segment, so the old "LAST terminal wins" retry intent is preserved.
|
||||
if (activity.OpenStart.HasValue)
|
||||
{
|
||||
if (evt.Timestamp >= activity.OpenStart.Value)
|
||||
activity.Segments.Add((activity.OpenStart.Value, evt.Timestamp));
|
||||
else
|
||||
flags |= TimeAttributionFlags.ClockSkewDropped;
|
||||
activity.OpenStart = null;
|
||||
}
|
||||
// Terminal without an observed start: no segment claim (attach window,
|
||||
// audit §0.5) — same rule as before.
|
||||
}
|
||||
}
|
||||
|
||||
var matched = 0;
|
||||
var occupancySegments = new List<BlockingAppInterval>();
|
||||
foreach (var activity in apps.Values)
|
||||
{
|
||||
if (!sets.Contains(activity.AppId)) continue; // absent ⇒ unknown, never "not blocking"
|
||||
matched++;
|
||||
|
||||
if (!activity.FirstStart.HasValue || !activity.LastTerminal.HasValue)
|
||||
continue; // unobserved endpoint → no interval claim (attach window, audit §0.5)
|
||||
if (activity.LastTerminal.Value < activity.FirstStart.Value)
|
||||
{
|
||||
flags |= TimeAttributionFlags.ClockSkewDropped;
|
||||
continue;
|
||||
}
|
||||
if (activity.Segments.Count == 0)
|
||||
continue; // no fully-observed attempt → no interval claim (attach window, audit §0.5)
|
||||
|
||||
var clamped = ClampToWindows(activity.FirstStart.Value, activity.LastTerminal.Value, windows);
|
||||
if (!clamped.HasValue)
|
||||
// Clamp each segment; the persisted per-app interval is the hull of the clamped
|
||||
// segments with Seconds = the SUM of in-window active time (not the hull span —
|
||||
// the inter-pass idle gap is not this app's time).
|
||||
var total = 0;
|
||||
DateTime? hullStart = null, hullEnd = null;
|
||||
foreach (var segment in activity.Segments)
|
||||
{
|
||||
// Outside the hull entirely → pathological timestamps (flagged, as before).
|
||||
// Inside the hull but touching no window (wholly inside the WhiteGlove pause):
|
||||
// no in-window observation → no interval claim, and nothing wrong with clocks.
|
||||
if (activity.LastTerminal.Value < windows[0].Start ||
|
||||
activity.FirstStart.Value > windows[windows.Count - 1].End)
|
||||
var clamped = ClampToWindows(segment.Start, segment.End, windows);
|
||||
if (!clamped.HasValue) continue;
|
||||
total += clamped.Value.Seconds;
|
||||
if (hullStart == null || clamped.Value.Start < hullStart.Value) hullStart = clamped.Value.Start;
|
||||
if (hullEnd == null || clamped.Value.End > hullEnd.Value) hullEnd = clamped.Value.End;
|
||||
occupancySegments.Add(new BlockingAppInterval
|
||||
{
|
||||
AppId = activity.AppId,
|
||||
AppName = activity.AppName,
|
||||
StartUtc = clamped.Value.Start,
|
||||
EndUtc = clamped.Value.End,
|
||||
Seconds = clamped.Value.Seconds,
|
||||
});
|
||||
}
|
||||
if (hullStart == null || hullEnd == null)
|
||||
{
|
||||
// No segment overlapped any window. Outside the hull entirely →
|
||||
// pathological timestamps (flagged, as before). Wholly inside the
|
||||
// WhiteGlove pause → no in-window observation, nothing wrong with clocks.
|
||||
if (activity.LastTerminal!.Value < windows[0].Start ||
|
||||
activity.FirstStart!.Value > windows[windows.Count - 1].End)
|
||||
{
|
||||
flags |= TimeAttributionFlags.ClockSkewDropped;
|
||||
}
|
||||
@@ -476,14 +526,13 @@ public static class TimeAttributionCalculator
|
||||
{
|
||||
AppId = activity.AppId,
|
||||
AppName = activity.AppName,
|
||||
StartUtc = clamped.Value.Start,
|
||||
EndUtc = clamped.Value.End,
|
||||
// In-window seconds — a WG-pause-straddling interval never counts the pause.
|
||||
Seconds = clamped.Value.Seconds,
|
||||
StartUtc = hullStart.Value,
|
||||
EndUtc = hullEnd.Value,
|
||||
Seconds = total,
|
||||
});
|
||||
}
|
||||
|
||||
return (intervals, matched);
|
||||
return (intervals, matched, occupancySegments);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -665,6 +714,63 @@ public static class TimeAttributionCalculator
|
||||
return result;
|
||||
}
|
||||
|
||||
// ── sleep spans ─────────────────────────────────────────────────────────
|
||||
|
||||
/// <summary>
|
||||
/// One span per completed sleep episode (<c>system_sleep_episode</c>). The payload's
|
||||
/// <c>enteredAt</c>/<c>exitedAt</c> are the authoritative instants — the event's own
|
||||
/// timestamp is stamped at wake and may have been clamped server-side. Structural clone of
|
||||
/// <see cref="BuildRebootSpans"/>: spans are clipped to the observation hull (a WhiteGlove
|
||||
/// pause episode contributes nothing), dedup'd on the enter instant (an episode straddling
|
||||
/// an agent restart can be backfilled twice from different records), and annotate the
|
||||
/// segment they started in — they are never a slice of the wall-clock partition.
|
||||
/// </summary>
|
||||
private static List<SleepSpan> BuildSleepSpans(
|
||||
List<EnrollmentEvent> events, List<Window> windows, List<TimeAttributionSpan> spans)
|
||||
{
|
||||
var result = new List<SleepSpan>();
|
||||
var seenEpisodes = new HashSet<DateTime>();
|
||||
|
||||
foreach (var evt in events)
|
||||
{
|
||||
if (evt.EventType != Constants.EventTypes.SystemSleepEpisode || evt.Data == null) continue;
|
||||
|
||||
var enteredAt = ParseUtcInstant(evt.Data, "enteredAt");
|
||||
var exitedAt = ParseUtcInstant(evt.Data, "exitedAt");
|
||||
if (!enteredAt.HasValue || !exitedAt.HasValue || exitedAt.Value <= enteredAt.Value) continue;
|
||||
if (!seenEpisodes.Add(enteredAt.Value)) continue; // duplicate observation of the same episode
|
||||
|
||||
var clamped = ClampToWindows(enteredAt.Value, exitedAt.Value, windows);
|
||||
if (!clamped.HasValue) continue;
|
||||
var seconds = clamped.Value.Seconds;
|
||||
if (seconds <= 0) continue;
|
||||
|
||||
var kind = evt.Data.TryGetValue("kind", out var kindObj) ? kindObj?.ToString() ?? string.Empty : string.Empty;
|
||||
|
||||
result.Add(new SleepSpan
|
||||
{
|
||||
StartUtc = clamped.Value.Start,
|
||||
EndUtc = clamped.Value.End,
|
||||
Seconds = seconds,
|
||||
SegmentKey = SegmentAt(spans, clamped.Value.Start),
|
||||
Kind = kind,
|
||||
});
|
||||
}
|
||||
|
||||
return result.OrderBy(s => s.StartUtc).ToList();
|
||||
}
|
||||
|
||||
private static DateTime? ParseUtcInstant(Dictionary<string, object> data, string key)
|
||||
{
|
||||
if (!data.TryGetValue(key, out var raw)) return null;
|
||||
var text = raw?.ToString();
|
||||
if (string.IsNullOrEmpty(text)) return null;
|
||||
return DateTime.TryParse(text, CultureInfo.InvariantCulture,
|
||||
DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal, out var parsed)
|
||||
? parsed
|
||||
: (DateTime?)null;
|
||||
}
|
||||
|
||||
private static string SegmentAt(List<TimeAttributionSpan> spans, DateTime ts)
|
||||
{
|
||||
foreach (var span in spans)
|
||||
|
||||
+10
-4
@@ -289,10 +289,16 @@ namespace AutopilotMonitor.Functions.Services.Analyze
|
||||
rule.Title, rule.Category, rule.Severity,
|
||||
fired, confidence).ConfigureAwait(false);
|
||||
|
||||
await _metricsRepo.IncrementRuleStatAsync(
|
||||
today, "global", rule.RuleId, "analyze",
|
||||
rule.Title, rule.Category, rule.Severity,
|
||||
fired, confidence).ConfigureAwait(false);
|
||||
// Global aggregate row is catalog-only: custom-rule IDs are tenant-chosen
|
||||
// and not unique across tenants, so a shared "global_{ruleId}" row would
|
||||
// sum unrelated tenants' counters (title/severity last-writer-wins).
|
||||
if (rule.IsBuiltIn || rule.IsCommunity)
|
||||
{
|
||||
await _metricsRepo.IncrementRuleStatAsync(
|
||||
today, "global", rule.RuleId, "analyze",
|
||||
rule.Title, rule.Category, rule.Severity,
|
||||
fired, confidence).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
|
||||
+62
-20
@@ -112,6 +112,22 @@ namespace AutopilotMonitor.Functions.Services
|
||||
case "app_install_start":
|
||||
if (!state.InstallStartedAt.HasValue || evt.Timestamp < state.InstallStartedAt.Value)
|
||||
state.InstallStartedAt = evt.Timestamp;
|
||||
// Attempt anchor: the LATEST start wins (max-fold). DurationSeconds
|
||||
// measures the status-defining attempt, not the session-wide span — the
|
||||
// IME processes the app list in multiple passes (device-ESP evaluation,
|
||||
// user-context install, retries), often an hour apart.
|
||||
if (!summary.LastAttemptStartedAt.HasValue || evt.Timestamp > summary.LastAttemptStartedAt.Value)
|
||||
summary.LastAttemptStartedAt = evt.Timestamp;
|
||||
summary.InstallPassCount++;
|
||||
// Attempt-scoped download pairing: a pending download start is consumed
|
||||
// by the next install start of the same app.
|
||||
if (state.PendingDownloadStartedAt.HasValue && evt.Timestamp >= state.PendingDownloadStartedAt.Value)
|
||||
{
|
||||
var pairedDl = EventTimestampValidator.SafeDurationSeconds(state.PendingDownloadStartedAt.Value, evt.Timestamp);
|
||||
if (pairedDl > 0)
|
||||
summary.DownloadDurationSeconds = pairedDl;
|
||||
state.PendingDownloadStartedAt = null;
|
||||
}
|
||||
if (summary.Status == "InProgress" || summary.Status == string.Empty)
|
||||
summary.Status = "InProgress";
|
||||
break;
|
||||
@@ -119,34 +135,48 @@ namespace AutopilotMonitor.Functions.Services
|
||||
case "app_download_started":
|
||||
if (!state.DownloadStartedAt.HasValue || evt.Timestamp < state.DownloadStartedAt.Value)
|
||||
state.DownloadStartedAt = evt.Timestamp;
|
||||
// Latest unconsumed download start — pairs with the NEXT install start.
|
||||
if (!state.PendingDownloadStartedAt.HasValue || evt.Timestamp > state.PendingDownloadStartedAt.Value)
|
||||
state.PendingDownloadStartedAt = evt.Timestamp;
|
||||
if (summary.Status == "InProgress" || summary.Status == string.Empty)
|
||||
summary.Status = "InProgress";
|
||||
break;
|
||||
|
||||
case "app_install_completed":
|
||||
case "app_install_complete":
|
||||
// A weaker terminal (Skipped/Postponed re-evaluation pass) never overrides
|
||||
// an Installed/Error already observed — mirrors the app_install_skipped
|
||||
// guard below. It must not move CompletedAt/DurationSeconds either: the
|
||||
// row keeps describing the attempt that produced the surviving terminal.
|
||||
var incomingTerminalState = evt.Data?.ContainsKey("state") == true
|
||||
? evt.Data["state"]?.ToString() : null;
|
||||
if ((summary.TerminalState == "Installed" || summary.TerminalState == "Error") &&
|
||||
(incomingTerminalState == "Skipped" || incomingTerminalState == "Postponed"))
|
||||
break;
|
||||
summary.Status = "Succeeded";
|
||||
summary.CompletedAt = evt.Timestamp;
|
||||
if (summary.StartedAt != DateTime.MinValue)
|
||||
summary.DurationSeconds = Math.Max(1, EventTimestampValidator.SafeDurationSeconds(summary.StartedAt, evt.Timestamp));
|
||||
var attemptStart = summary.LastAttemptStartedAt
|
||||
?? (summary.StartedAt != DateTime.MinValue ? summary.StartedAt : (DateTime?)null);
|
||||
if (attemptStart.HasValue)
|
||||
summary.DurationSeconds = Math.Max(1, EventTimestampValidator.SafeDurationSeconds(attemptStart.Value, evt.Timestamp));
|
||||
// PR0 (2026-07-26): the agent emits app_install_completed for EVERY terminal
|
||||
// transition — including Skipped/Postponed (V1 wire parity). The payload's
|
||||
// `state` field is the only way to tell a real install apart from a no-op
|
||||
// (e.g. WinGet "Update for X" policies that were not applicable). Persist it
|
||||
// so metrics can exclude skips from durations and rates. Unknown/absent state
|
||||
// leaves the sentinel empty — never guessed.
|
||||
var terminalStateRaw = evt.Data?.ContainsKey("state") == true
|
||||
? evt.Data["state"]?.ToString() : null;
|
||||
if (terminalStateRaw == "Installed" || terminalStateRaw == "Skipped" || terminalStateRaw == "Postponed")
|
||||
summary.TerminalState = terminalStateRaw;
|
||||
if (incomingTerminalState == "Installed" || incomingTerminalState == "Skipped" || incomingTerminalState == "Postponed")
|
||||
summary.TerminalState = incomingTerminalState;
|
||||
break;
|
||||
|
||||
case "app_install_failed":
|
||||
summary.Status = "Failed";
|
||||
summary.TerminalState = "Error";
|
||||
summary.CompletedAt = evt.Timestamp;
|
||||
if (summary.StartedAt != DateTime.MinValue)
|
||||
summary.DurationSeconds = Math.Max(1, EventTimestampValidator.SafeDurationSeconds(summary.StartedAt, evt.Timestamp));
|
||||
var failedAttemptStart = summary.LastAttemptStartedAt
|
||||
?? (summary.StartedAt != DateTime.MinValue ? summary.StartedAt : (DateTime?)null);
|
||||
if (failedAttemptStart.HasValue)
|
||||
summary.DurationSeconds = Math.Max(1, EventTimestampValidator.SafeDurationSeconds(failedAttemptStart.Value, evt.Timestamp));
|
||||
// FailureCode preference: canonical `failureType` > raw `errorCode` > empty.
|
||||
// c117946b debrief (2026-05-12): the V2 termination handler tags promoted
|
||||
// "likely stuck" apps with failureType=esp_apps_timeout so the UI can
|
||||
@@ -310,20 +340,30 @@ namespace AutopilotMonitor.Functions.Services
|
||||
summary.StartedAt = effectiveStart;
|
||||
}
|
||||
|
||||
// Download duration: from first download start to first install start.
|
||||
if (state.DownloadStartedAt.HasValue && state.InstallStartedAt.HasValue &&
|
||||
state.InstallStartedAt.Value >= state.DownloadStartedAt.Value)
|
||||
{
|
||||
summary.DownloadDurationSeconds = EventTimestampValidator.SafeDurationSeconds(
|
||||
state.DownloadStartedAt.Value, state.InstallStartedAt.Value);
|
||||
}
|
||||
// Download duration is paired attempt-scoped at the app_install_started event
|
||||
// (latest download start → next install start) — no span computation here.
|
||||
|
||||
// Full duration: from effective start to completion/failure.
|
||||
if (summary.CompletedAt.HasValue && summary.StartedAt != DateTime.MinValue &&
|
||||
summary.CompletedAt.Value >= summary.StartedAt)
|
||||
// Full duration: the status-defining attempt. CompletedAt pairs with the latest
|
||||
// attempt start at or before it. A LastAttemptStartedAt NEWER than CompletedAt
|
||||
// means a fresh pass is in flight — the recorded duration of the completed
|
||||
// attempt stands. Rows without any observed start (legacy agents, terminal-only
|
||||
// batches) keep the historical StartedAt (span) fallback.
|
||||
if (summary.CompletedAt.HasValue)
|
||||
{
|
||||
summary.DurationSeconds = EventTimestampValidator.SafeDurationSeconds(
|
||||
summary.StartedAt, summary.CompletedAt.Value);
|
||||
if (summary.LastAttemptStartedAt.HasValue)
|
||||
{
|
||||
if (summary.LastAttemptStartedAt.Value <= summary.CompletedAt.Value)
|
||||
{
|
||||
summary.DurationSeconds = EventTimestampValidator.SafeDurationSeconds(
|
||||
summary.LastAttemptStartedAt.Value, summary.CompletedAt.Value);
|
||||
}
|
||||
}
|
||||
else if (summary.StartedAt != DateTime.MinValue &&
|
||||
summary.CompletedAt.Value >= summary.StartedAt)
|
||||
{
|
||||
summary.DurationSeconds = EventTimestampValidator.SafeDurationSeconds(
|
||||
summary.StartedAt, summary.CompletedAt.Value);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -333,5 +373,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
public AppInstallSummary Summary { get; set; } = new();
|
||||
public DateTime? DownloadStartedAt { get; set; }
|
||||
public DateTime? InstallStartedAt { get; set; }
|
||||
/// <summary>Latest download start not yet consumed by an install start (attempt-scoped download-duration pairing).</summary>
|
||||
public DateTime? PendingDownloadStartedAt { get; set; }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -378,6 +378,13 @@ namespace AutopilotMonitor.Functions.Services
|
||||
"stall_probe_result" => true,
|
||||
"session_stalled" => true,
|
||||
"modern_deployment_log" => true,
|
||||
// Environment observation (battery drain / AC flap), not enrollment progress —
|
||||
// must not count as "real activity" that heals a Stalled session.
|
||||
"power_state_change" => true,
|
||||
// System timeline watcher: a clock step or a completed standby episode is environment
|
||||
// observation too — a device that slept through its stall must stay Stalled.
|
||||
"system_clock_changed" => true,
|
||||
"system_sleep_episode" => true,
|
||||
_ => false
|
||||
};
|
||||
|
||||
|
||||
@@ -36,10 +36,18 @@ namespace AutopilotMonitor.Functions.Services
|
||||
ruleTitle, "", "",
|
||||
fired: true, confidenceScore: null);
|
||||
|
||||
await _metricsRepo.IncrementRuleStatAsync(
|
||||
today, "global", ruleId, "gather",
|
||||
ruleTitle, "", "",
|
||||
fired: true, confidenceScore: null);
|
||||
// Global aggregate row is catalog-only. The agent event carries no
|
||||
// IsBuiltIn flag, but custom rules can never occupy the reserved
|
||||
// built-in namespace (create/update guard), so the ID pattern is the
|
||||
// discriminator. Custom IDs are tenant-chosen and not unique across
|
||||
// tenants — a shared "global_{ruleId}" row would mix their counters.
|
||||
if (RuleIdPolicy.IsReservedBuiltInId(ruleId))
|
||||
{
|
||||
await _metricsRepo.IncrementRuleStatAsync(
|
||||
today, "global", ruleId, "gather",
|
||||
ruleTitle, "", "",
|
||||
fired: true, confidenceScore: null);
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
|
||||
@@ -78,6 +78,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
"DiagnosticsBlobName",
|
||||
"DiagnosticsBlobDestination",
|
||||
"LastEventAt",
|
||||
"LastIngestAt",
|
||||
"ResumedAt",
|
||||
"StalledAt",
|
||||
"AvgApiLatencyMs",
|
||||
|
||||
@@ -260,29 +260,75 @@ namespace AutopilotMonitor.Functions.Services
|
||||
var existingStartedAt = existing.GetDateTimeOffset("StartedAt")?.UtcDateTime;
|
||||
if (existingStartedAt.HasValue && existingStartedAt.Value != DateTime.MinValue)
|
||||
{
|
||||
// Keep the earlier StartedAt; recalculate duration if CompletedAt is now known
|
||||
// Keep the earlier StartedAt — it is the window/bucket filter column and
|
||||
// the radar's version-ordering key, never re-anchored to a later attempt.
|
||||
// Duration recompute happens attempt-aware further down.
|
||||
if (summary.StartedAt == DateTime.MinValue || existingStartedAt.Value < summary.StartedAt)
|
||||
{
|
||||
summary.StartedAt = existingStartedAt.Value;
|
||||
if (summary.CompletedAt.HasValue && summary.CompletedAt.Value >= summary.StartedAt)
|
||||
{
|
||||
summary.DurationSeconds = (int)(summary.CompletedAt.Value - summary.StartedAt).TotalSeconds;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Attempt anchor: the LATEST start wins across batches (mirror of the
|
||||
// in-batch max-fold — a replayed older batch can never regress it), and the
|
||||
// pass counter only grows when the batch carries genuinely newer starts
|
||||
// (replay guard: a batch whose newest start is not newer than the stored
|
||||
// anchor was already counted).
|
||||
var existingLastAttempt = existing.GetDateTimeOffset("LastAttemptStartedAt")?.UtcDateTime;
|
||||
var existingPassCount = existing.GetInt32("InstallPassCount") ?? 0;
|
||||
if (summary.LastAttemptStartedAt.HasValue &&
|
||||
(!existingLastAttempt.HasValue || summary.LastAttemptStartedAt.Value > existingLastAttempt.Value))
|
||||
{
|
||||
summary.InstallPassCount += existingPassCount;
|
||||
}
|
||||
else
|
||||
{
|
||||
summary.InstallPassCount = Math.Max(summary.InstallPassCount, existingPassCount);
|
||||
if (existingLastAttempt.HasValue)
|
||||
summary.LastAttemptStartedAt = existingLastAttempt;
|
||||
}
|
||||
|
||||
// Cross-batch mirror of the weaker-terminal guard: a batch that only saw a
|
||||
// Skipped/Postponed re-evaluation pass never overrides a stored
|
||||
// Installed/Error, and its CompletedAt/DurationSeconds must not re-describe
|
||||
// the row (nulling them omits the columns → Merge preserves the stored
|
||||
// values of the surviving attempt).
|
||||
var existingTerminalStateForGuard = existing.GetString("TerminalState");
|
||||
if ((existingTerminalStateForGuard == "Installed" || existingTerminalStateForGuard == "Error") &&
|
||||
(summary.TerminalState == "Skipped" || summary.TerminalState == "Postponed"))
|
||||
{
|
||||
summary.TerminalState = existingTerminalStateForGuard!;
|
||||
if (existingStatus == "Succeeded" || existingStatus == "Failed")
|
||||
summary.Status = existingStatus!;
|
||||
summary.CompletedAt = null;
|
||||
summary.DurationSeconds = 0;
|
||||
}
|
||||
|
||||
// Q4 (source-data audit 2026-07-26): out-of-order arrival — the terminal batch
|
||||
// landed FIRST (row already carries CompletedAt), the started batch arrives now
|
||||
// with the earlier StartedAt. Neither batch alone could compute the duration;
|
||||
// recompute it here against the stored CompletedAt so the row doesn't keep a
|
||||
// permanently unset duration despite both endpoints being known.
|
||||
// landed FIRST (row already carries CompletedAt), the started batch arrives now.
|
||||
// Neither batch alone could compute the duration; adopt the stored CompletedAt
|
||||
// so the attempt-aware recompute below can pair both endpoints.
|
||||
if (!summary.CompletedAt.HasValue && summary.StartedAt != DateTime.MinValue)
|
||||
{
|
||||
var existingCompletedAt = existing.GetDateTimeOffset("CompletedAt")?.UtcDateTime;
|
||||
if (existingCompletedAt.HasValue && existingCompletedAt.Value >= summary.StartedAt)
|
||||
{
|
||||
summary.CompletedAt = existingCompletedAt.Value;
|
||||
summary.DurationSeconds = (int)(existingCompletedAt.Value - summary.StartedAt).TotalSeconds;
|
||||
}
|
||||
|
||||
// Attempt-aware duration recompute — single site for both endpoints-known
|
||||
// paths (in-order and Q4 out-of-order). Anchor = latest attempt start at or
|
||||
// before CompletedAt; rows without an observed start keep the historical
|
||||
// StartedAt (span) fallback. A LastAttemptStartedAt NEWER than CompletedAt
|
||||
// means a fresh pass is in flight — the stored duration of the completed
|
||||
// attempt stands (DurationSeconds 0 omits the column, Merge preserves it).
|
||||
if (summary.CompletedAt.HasValue)
|
||||
{
|
||||
if (summary.LastAttemptStartedAt.HasValue)
|
||||
{
|
||||
if (summary.LastAttemptStartedAt.Value <= summary.CompletedAt.Value)
|
||||
summary.DurationSeconds = (int)(summary.CompletedAt.Value - summary.LastAttemptStartedAt.Value).TotalSeconds;
|
||||
}
|
||||
else if (summary.StartedAt != DateTime.MinValue && summary.CompletedAt.Value >= summary.StartedAt)
|
||||
{
|
||||
summary.DurationSeconds = (int)(summary.CompletedAt.Value - summary.StartedAt).TotalSeconds;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -571,7 +617,10 @@ namespace AutopilotMonitor.Functions.Services
|
||||
"PartitionKey", "RowKey", "TenantId", "SessionId", "AppName", "AppType", "AppVersion",
|
||||
"Status", "TerminalState", "StartedAt", "CompletedAt", "DurationSeconds", "DownloadBytes",
|
||||
"AttemptNumber", "InstallerPhase", "FailureCode", "FailureMessage", "ExitCode", "DetectionResult",
|
||||
"AppId", "EspBlocking", "AppIdCollision"
|
||||
"AppId", "EspBlocking", "AppIdCollision",
|
||||
// Attempt-duration columns (2026-08): the radar's cutover gate reads the anchor,
|
||||
// the dashboard sessions table shows the pass count.
|
||||
"LastAttemptStartedAt", "InstallPassCount"
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
@@ -671,6 +720,8 @@ namespace AutopilotMonitor.Functions.Services
|
||||
FailureMessage = entity.GetString("FailureMessage") ?? string.Empty,
|
||||
StartedAt = entity.GetDateTimeOffset("StartedAt")?.UtcDateTime ?? DateTime.MinValue,
|
||||
CompletedAt = entity.GetDateTimeOffset("CompletedAt")?.UtcDateTime,
|
||||
LastAttemptStartedAt = entity.GetDateTimeOffset("LastAttemptStartedAt")?.UtcDateTime,
|
||||
InstallPassCount = entity.GetInt32("InstallPassCount") ?? 0,
|
||||
// Delivery Optimization telemetry
|
||||
DoFileSize = entity.GetInt64("DoFileSize") ?? 0,
|
||||
DoTotalBytesDownloaded = entity.GetInt64("DoTotalBytesDownloaded") ?? 0,
|
||||
@@ -1563,6 +1614,12 @@ namespace AutopilotMonitor.Functions.Services
|
||||
entity["DurationSeconds"] = summary.DurationSeconds;
|
||||
if (summary.CompletedAt.HasValue)
|
||||
entity["CompletedAt"] = EnsureUtc(summary.CompletedAt.Value);
|
||||
// Attempt-duration columns (2026-08): both sentinel-gated — a batch that observed
|
||||
// no install start must not clobber the stored anchor/counter under Merge-mode.
|
||||
if (summary.LastAttemptStartedAt.HasValue)
|
||||
entity["LastAttemptStartedAt"] = EnsureUtc(summary.LastAttemptStartedAt.Value);
|
||||
if (summary.InstallPassCount > 0)
|
||||
entity["InstallPassCount"] = summary.InstallPassCount;
|
||||
if (!string.IsNullOrEmpty(summary.FailureCode))
|
||||
entity["FailureCode"] = summary.FailureCode;
|
||||
if (!string.IsNullOrEmpty(summary.FailureMessage))
|
||||
|
||||
@@ -442,6 +442,20 @@ namespace AutopilotMonitor.Functions.Services
|
||||
public async Task<(int deleted, int failed)> DeleteRuleStatesForRuleIdAcrossTenantsAsync(string ruleId)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(ruleId)) return (0, 0);
|
||||
|
||||
// This is a cross-partition delete keyed on a bare RowKey string, so only
|
||||
// catalog (built-in/community) IDs may pass: custom-rule IDs are tenant-chosen
|
||||
// and not unique across tenants, and a call with one would delete state rows
|
||||
// of unrelated tenants. failed=-1 reuses the enumeration-failure contract —
|
||||
// the caller then refuses the follow-up global catalog delete.
|
||||
if (!RuleIdPolicy.IsReservedBuiltInId(ruleId))
|
||||
{
|
||||
_logger.LogError(
|
||||
"Refusing cross-tenant RuleState GC for '{RuleId}': ID is outside the reserved built-in namespace",
|
||||
ruleId);
|
||||
return (0, -1);
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.RuleStates);
|
||||
|
||||
@@ -225,6 +225,13 @@ namespace AutopilotMonitor.Functions.Services
|
||||
if (lastEventAt.HasValue)
|
||||
indexEntity["LastEventAt"] = EnsureUtc(lastEventAt.Value);
|
||||
|
||||
// Server-frame ingest stamp (IncrementSessionEventCountAsync merges this on every
|
||||
// batch, so the index already carries it) — must be in this superset or a
|
||||
// StartedAt-shift full upsert drops it and index-served reads lose the field.
|
||||
var lastIngestAt = sessionEntity.GetDateTimeOffset("LastIngestAt")?.UtcDateTime;
|
||||
if (lastIngestAt.HasValue)
|
||||
indexEntity["LastIngestAt"] = EnsureUtc(lastIngestAt.Value);
|
||||
|
||||
var resumedAt = sessionEntity.GetDateTimeOffset("ResumedAt")?.UtcDateTime;
|
||||
if (resumedAt.HasValue)
|
||||
indexEntity["ResumedAt"] = EnsureUtc(resumedAt.Value);
|
||||
@@ -1223,8 +1230,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
int succeeded = 0;
|
||||
int failed = 0;
|
||||
int incomplete = 0;
|
||||
long succeededDurationSeconds = 0;
|
||||
int succeededWithDurationCount = 0;
|
||||
var succeededDurationSeconds = new List<double>();
|
||||
int totalToday = 0;
|
||||
int failedToday = 0;
|
||||
|
||||
@@ -1243,10 +1249,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
case SessionStatus.Succeeded:
|
||||
succeeded++;
|
||||
if (s.DurationSeconds is int d && d > 0)
|
||||
{
|
||||
succeededDurationSeconds += d;
|
||||
succeededWithDurationCount++;
|
||||
}
|
||||
succeededDurationSeconds.Add(d);
|
||||
break;
|
||||
case SessionStatus.Failed:
|
||||
failed++;
|
||||
@@ -1268,8 +1271,9 @@ namespace AutopilotMonitor.Functions.Services
|
||||
int successRatePct = terminal > 0
|
||||
? (int)Math.Round((double)succeeded / terminal * 100.0)
|
||||
: 0;
|
||||
int avgDurationMinutes = succeededWithDurationCount > 0
|
||||
? (int)Math.Round((double)succeededDurationSeconds / succeededWithDurationCount / 60.0)
|
||||
succeededDurationSeconds.Sort(); // MetricsMath.Percentile expects ascending order
|
||||
int avgDurationMinutes = succeededDurationSeconds.Count > 0
|
||||
? (int)Math.Round(succeededDurationSeconds.Average() / 60.0)
|
||||
: 0;
|
||||
|
||||
return new SessionStats
|
||||
@@ -1282,6 +1286,8 @@ namespace AutopilotMonitor.Functions.Services
|
||||
IncompleteLastNDays = incomplete,
|
||||
SuccessRatePct = successRatePct,
|
||||
AvgDurationMinutes = avgDurationMinutes,
|
||||
MedianDurationMinutes = MetricsMath.SecondsToMinutes(MetricsMath.Percentile(succeededDurationSeconds, 50)),
|
||||
P90DurationMinutes = MetricsMath.SecondsToMinutes(MetricsMath.Percentile(succeededDurationSeconds, 90)),
|
||||
TotalToday = totalToday,
|
||||
FailedToday = failedToday,
|
||||
ComputedAt = DateTime.UtcNow,
|
||||
|
||||
@@ -117,10 +117,12 @@ namespace AutopilotMonitor.Functions.Services
|
||||
["WallClockSeconds"] = b.WallClockSeconds,
|
||||
["UnattributedSeconds"] = b.UnattributedSeconds,
|
||||
["RebootSeconds"] = b.RebootSeconds,
|
||||
["SleepSeconds"] = b.SleepSeconds,
|
||||
["BlockingAppCount"] = b.BlockingAppCount,
|
||||
["QualityFlags"] = (int)b.QualityFlags,
|
||||
["SegmentsJson"] = JsonSerializer.Serialize(b.Segments),
|
||||
["RebootSpansJson"] = JsonSerializer.Serialize(b.RebootSpans),
|
||||
["SleepSpansJson"] = JsonSerializer.Serialize(b.SleepSpans),
|
||||
["BlockingAppsJson"] = JsonSerializer.Serialize(b.BlockingApps),
|
||||
["ComputedAt"] = DateTime.UtcNow,
|
||||
};
|
||||
@@ -142,11 +144,13 @@ namespace AutopilotMonitor.Functions.Services
|
||||
WallClockSeconds = entity.GetInt32("WallClockSeconds") ?? 0,
|
||||
UnattributedSeconds = entity.GetInt32("UnattributedSeconds") ?? 0,
|
||||
RebootSeconds = entity.GetInt32("RebootSeconds") ?? 0,
|
||||
SleepSeconds = entity.GetInt32("SleepSeconds") ?? 0,
|
||||
BlockingAppCount = entity.GetInt32("BlockingAppCount") ?? 0,
|
||||
EspAppsOccupancySeconds = entity.GetInt32("EspAppsOccupancySeconds"),
|
||||
QualityFlags = (TimeAttributionFlags)(entity.GetInt32("QualityFlags") ?? 0),
|
||||
Segments = DeserializeJsonColumn<TimeAttributionSpan>(entity.GetString("SegmentsJson")),
|
||||
RebootSpans = DeserializeJsonColumn<RebootSpan>(entity.GetString("RebootSpansJson")),
|
||||
SleepSpans = DeserializeJsonColumn<SleepSpan>(entity.GetString("SleepSpansJson")),
|
||||
BlockingApps = DeserializeJsonColumn<BlockingAppInterval>(entity.GetString("BlockingAppsJson")),
|
||||
};
|
||||
}
|
||||
|
||||
+93
-27
@@ -4,6 +4,7 @@ using System.Linq;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using AutopilotMonitor.Functions.Services.Analyze;
|
||||
using AutopilotMonitor.Shared;
|
||||
using AutopilotMonitor.Shared.DataAccess;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Microsoft.Extensions.Logging;
|
||||
@@ -255,36 +256,23 @@ namespace AutopilotMonitor.Functions.Services.Vulnerability
|
||||
{
|
||||
try
|
||||
{
|
||||
var scanSummary = reportData.TryGetValue("scan_summary", out var summaryObj)
|
||||
&& summaryObj is Dictionary<string, object> summaryDict
|
||||
? summaryDict
|
||||
: new Dictionary<string, object>();
|
||||
// Soft read: a storage hiccup degrades the event to Sequence 1 instead of
|
||||
// suppressing it entirely — the downstream re-analyze (ANALYZE-ID-003) needs
|
||||
// the event to exist more than it needs a perfect timeline position.
|
||||
var existingEvents = await _sessionRepo
|
||||
.GetSessionEventsAsync(tenantId, sessionId).ConfigureAwait(false);
|
||||
|
||||
var overallRisk = scanSummary.TryGetValue("overall_risk", out var riskObj)
|
||||
? riskObj?.ToString()
|
||||
: null;
|
||||
|
||||
var data = new Dictionary<string, object>(StringComparer.Ordinal)
|
||||
{
|
||||
["scan_summary"] = scanSummary,
|
||||
["reportAvailable"] = true,
|
||||
["phase"] = phaseLabel ?? "single",
|
||||
};
|
||||
|
||||
var evt = new EnrollmentEvent
|
||||
{
|
||||
TenantId = tenantId,
|
||||
SessionId = sessionId,
|
||||
EventType = "vulnerability_report",
|
||||
Severity = MapVulnerabilityRiskToSeverity(overallRisk),
|
||||
Source = "VulnerabilityCorrelationService",
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Message = $"Vulnerability scan complete (risk={overallRisk ?? "unknown"}).",
|
||||
Timestamp = DateTime.UtcNow,
|
||||
Data = data,
|
||||
};
|
||||
var evt = BuildVulnerabilityReportEvent(
|
||||
tenantId, sessionId, reportData, phaseLabel, existingEvents, DateTime.UtcNow);
|
||||
|
||||
await _sessionRepo.StoreEventsBatchAsync(new List<EnrollmentEvent> { evt }).ConfigureAwait(false);
|
||||
|
||||
// StoreEventsBatchAsync writes the Events partition only; the cross-session
|
||||
// EventType index is normally written by EventIngestProcessor, which this
|
||||
// backend-materialized event never passes through (see the identical upsert in
|
||||
// ReportAgentErrorFunction for the incident that established the pattern).
|
||||
await _sessionRepo.UpsertEventTypeIndexBatchAsync(
|
||||
tenantId, sessionId, new List<EnrollmentEvent> { evt }).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
@@ -293,6 +281,84 @@ namespace AutopilotMonitor.Functions.Services.Vulnerability
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Builds the synthetic <c>vulnerability_report</c> timeline event. Static + pure
|
||||
/// (analog to <c>ReportAgentErrorFunction.BuildAgentEmergencyBreakEvent</c>) so the
|
||||
/// field shape and the Sequence assignment are unit-testable.
|
||||
/// <para>
|
||||
/// <b>Sequence:</b> the correlation runs async off a queue, so the event arrives after
|
||||
/// the agent stream. Default is one past the session's max Sequence so it sorts LAST.
|
||||
/// Exception — WhiteGlove Part 1 (<c>phaseLabel == "device_setup"</c>): the web timeline
|
||||
/// splits Pre-Provisioning from User Enrollment strictly by Sequence
|
||||
/// (<c>computeWhiteGloveSplitSequence</c> in eventHelpers.ts), and if Part 2 is already
|
||||
/// running when the Part-1 scan lands, max+1 would push the device_setup report across
|
||||
/// the split and render it as the start of User Enrollment. The Part-1 report therefore
|
||||
/// anchors AT the max Sequence of the Part-1 terminal markers
|
||||
/// (<c>whiteglove_part1_complete</c> / <c>whiteglove_complete</c>) — provably ≤ the web
|
||||
/// split in every split branch, with the timestamp tie-break sorting it right behind the
|
||||
/// marker. Falls back to max+1 when no marker exists (then the web derives no split either).
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal static EnrollmentEvent BuildVulnerabilityReportEvent(
|
||||
string tenantId,
|
||||
string sessionId,
|
||||
Dictionary<string, object> reportData,
|
||||
string? phaseLabel,
|
||||
IReadOnlyList<EnrollmentEvent> existingEvents,
|
||||
DateTime nowUtc)
|
||||
{
|
||||
var scanSummary = reportData.TryGetValue("scan_summary", out var summaryObj)
|
||||
&& summaryObj is Dictionary<string, object> summaryDict
|
||||
? summaryDict
|
||||
: new Dictionary<string, object>();
|
||||
|
||||
var overallRisk = scanSummary.TryGetValue("overall_risk", out var riskObj)
|
||||
? riskObj?.ToString()
|
||||
: null;
|
||||
|
||||
var data = new Dictionary<string, object>(StringComparer.Ordinal)
|
||||
{
|
||||
["scan_summary"] = scanSummary,
|
||||
["reportAvailable"] = true,
|
||||
["phase"] = phaseLabel ?? "single",
|
||||
};
|
||||
|
||||
return new EnrollmentEvent
|
||||
{
|
||||
TenantId = tenantId,
|
||||
SessionId = sessionId,
|
||||
EventType = Constants.EventTypes.VulnerabilityReport,
|
||||
Severity = MapVulnerabilityRiskToSeverity(overallRisk),
|
||||
Source = "VulnerabilityCorrelationService",
|
||||
Phase = EnrollmentPhase.Unknown,
|
||||
Message = $"Vulnerability scan complete (risk={overallRisk ?? "unknown"}).",
|
||||
Timestamp = nowUtc,
|
||||
Sequence = ComputeReportEventSequence(phaseLabel, existingEvents),
|
||||
Data = data,
|
||||
};
|
||||
}
|
||||
|
||||
private static long ComputeReportEventSequence(
|
||||
string? phaseLabel, IReadOnlyList<EnrollmentEvent> existingEvents)
|
||||
{
|
||||
if (existingEvents == null || existingEvents.Count == 0)
|
||||
return 1L;
|
||||
|
||||
if (string.Equals(phaseLabel, "device_setup", StringComparison.Ordinal))
|
||||
{
|
||||
var anchor = existingEvents
|
||||
.Where(e =>
|
||||
string.Equals(e.EventType, Constants.EventTypes.WhiteGlovePart1Complete, StringComparison.OrdinalIgnoreCase)
|
||||
|| string.Equals(e.EventType, Constants.EventTypes.WhiteGloveComplete, StringComparison.OrdinalIgnoreCase))
|
||||
.Select(e => (long?)e.Sequence)
|
||||
.Max();
|
||||
if (anchor.HasValue)
|
||||
return anchor.Value;
|
||||
}
|
||||
|
||||
return existingEvents.Max(e => e.Sequence) + 1;
|
||||
}
|
||||
|
||||
private async Task SafeUpsertCveIndexAsync(
|
||||
VulnerabilityCorrelateEnvelope envelope,
|
||||
string sessionPrefix,
|
||||
|
||||
@@ -108,8 +108,11 @@ export const EVENT_TYPES_CATALOG = {
|
||||
'device_location',
|
||||
'outbound_ip',
|
||||
'timezone_auto_set',
|
||||
'do_group_id_auto_set',
|
||||
'ntp_time_check',
|
||||
'power_state_check',
|
||||
'power_state_change',
|
||||
'system_clock_changed',
|
||||
],
|
||||
identity_events: [
|
||||
'aad_placeholder_user_detected',
|
||||
@@ -121,6 +124,7 @@ export const EVENT_TYPES_CATALOG = {
|
||||
'realmjoin_phase_changed',
|
||||
'realmjoin_resolved',
|
||||
'realmjoin_timeout',
|
||||
'realmjoin_timeout_extended',
|
||||
'realmjoin_first_deployment_incomplete',
|
||||
'realmjoin_package_started',
|
||||
'realmjoin_package_completed',
|
||||
@@ -154,6 +158,7 @@ export const EVENT_TYPES_CATALOG = {
|
||||
'ime_session_change',
|
||||
'ime_process_exited',
|
||||
'system_reboot_detected',
|
||||
'system_sleep_episode',
|
||||
'prior_run_died_with_state',
|
||||
'previous_crash_detected',
|
||||
'agent_emergency_break',
|
||||
|
||||
@@ -160,7 +160,11 @@ export const RULE_AUTHORING_GUIDE = {
|
||||
'Match a phase (eventType=esp_phase_changed, dataField=espPhase, operator=equals, ' +
|
||||
'value=<phase>); the evidence carries durationSeconds. Combine with a confidenceFactor ' +
|
||||
'"phase_duration > N" for the actual duration check.',
|
||||
app_install_duration: 'Compare app install duration in seconds (operator gt/lt/gte/lte, value=seconds).',
|
||||
app_install_duration:
|
||||
'Compare app install duration in seconds (operator gt/lt/gte/lte, value=seconds). ' +
|
||||
'Attempt-scoped: pairs each completion with the LATEST app_install_started before it ' +
|
||||
'(sequence order), so an IME re-evaluation pass an hour earlier does not inflate the ' +
|
||||
'value — same semantics as the apps dashboard since 2026-08.',
|
||||
event_correlation:
|
||||
'Join two event types over a shared field: eventType + correlateEventType + joinField, ' +
|
||||
'optional timeWindowSeconds, optional eventAFilterField/-Operator/-Value to pre-filter the ' +
|
||||
|
||||
@@ -1483,7 +1483,10 @@ export function registerAdminTools(server: McpServer, ga: boolean, strictGa: boo
|
||||
description:
|
||||
'Get aggregated app-install health for Autopilot enrollments over a time window: the top failing apps ' +
|
||||
'(with failure counts, failure rate, and their most common failure codes), the slowest apps by average ' +
|
||||
'install duration, and a fleet "deliveryOptimization" rollup — total bytes downloaded and how much came ' +
|
||||
'install duration (durations measure the FINAL install attempt — the IME processes the app list in ' +
|
||||
'multiple passes, and evaluation passes / time queued behind other apps are not counted; rows written ' +
|
||||
'before 2026-08 carry the old first-observation-to-last-terminal span), and a fleet "deliveryOptimization" ' +
|
||||
'rollup — total bytes downloaded and how much came ' +
|
||||
'from peers / Microsoft Connected Cache (MCC) vs. the CDN, plus a peerOffloadPercent (bandwidth saved by ' +
|
||||
'not pulling from the internet). Use this to answer "which app breaks or slows down my enrollments?" and ' +
|
||||
'"how much install bandwidth is served locally?". ' +
|
||||
@@ -1528,7 +1531,10 @@ export function registerAdminTools(server: McpServer, ga: boolean, strictGa: boo
|
||||
'time breakdown — wall-clock partition into device_prep / esp_apps / identity_hello / user_esp / ' +
|
||||
'desktop_handoff spans plus an EXPLICIT unattributed remainder (sums are exact against the session\'s ' +
|
||||
'authoritative duration; the WhiteGlove pause is excluded by design), the ESP-blocking app install ' +
|
||||
'intervals with critical-path occupancy, reboot outage spans, and data-quality flags. Omit sessionId ' +
|
||||
'intervals with critical-path occupancy (an app\'s seconds = its ACTIVE install time summed across all ' +
|
||||
'IME passes/attempts — deliberately broader than get_app_install_metrics\' final-attempt duration, ' +
|
||||
'because a failed attempt and an evaluation pass still occupied the critical path), reboot outage ' +
|
||||
'spans, and data-quality flags. Omit sessionId ' +
|
||||
'for the fleet rollup: rolling 30-day median/p75/p90 per segment per enrollment class (classes are ' +
|
||||
'never mixed) plus the top time-consuming ESP-blocking apps with "up to" what-if savings bounds. ' +
|
||||
'breakdown=null means the session has no computable attribution (pre-feature, non-terminal, or ' +
|
||||
|
||||
@@ -29,7 +29,7 @@ export const KEY_EVENT_TYPES = new Set([
|
||||
'desktop_arrived', 'hello_policy_detected', 'waiting_for_hello', 'hello_completion_timeout',
|
||||
'agent_started', 'agent_shutdown', 'agent_shutting_down', 'agent_trace',
|
||||
'script_started', 'script_completed', 'script_failed', 'historic_ime_replay_detected',
|
||||
'vulnerability_report',
|
||||
'vulnerability_report', 'system_sleep_episode',
|
||||
]);
|
||||
// Phase-defining events promoted to the top of the triage timeline. Module-level
|
||||
// (not handler-local) so the same drift test can validate it.
|
||||
|
||||
@@ -151,6 +151,10 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
{
|
||||
data["realmjoinReleaseChannel"] = postState.RealmJoinFacts.ReleaseChannel.Value;
|
||||
}
|
||||
if (postState.RealmJoinFacts.LastActivityUtc != null)
|
||||
{
|
||||
data["realmjoinLastActivityUtc"] = FormatUtc(postState.RealmJoinFacts.LastActivityUtc.Value);
|
||||
}
|
||||
data["realmjoinPackages"] = BuildRealmJoinPackageSummary(postState.RealmJoinFacts.Packages);
|
||||
}
|
||||
|
||||
|
||||
@@ -11,16 +11,30 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
// RealmJoin (RJ) deployment-tracking handlers. Plan: tasks/zany-gathering-oasis plan.
|
||||
// The agent posts six DecisionSignalKinds when it observes the
|
||||
// HKLM\SYSTEM\CurrentControlSet\Services\realmjoin\Parameters key + HKLM\SOFTWARE\RealmJoin\Packages
|
||||
// (and the HKU\<sid>\... user-scope counterpart). Detection arms a 60-min hard timeout
|
||||
// (and the HKU\<sid>\... user-scope counterpart). Detection arms a 60-min timeout
|
||||
// and gates the enrollment-completion AND-gate. Resolution (phase 110), the
|
||||
// aborted-first-deployment release (phase left 100/101 for 200/210 without 110 —
|
||||
// session 224b2087) or the timeout releases the gate so the session can complete.
|
||||
// When the timeout fires while the first deployment is demonstrably still active
|
||||
// (phase 100/101 with deployment activity inside the last 60 min), the deadline is
|
||||
// re-armed instead — up to an absolute 4-h cap from detection (report 55e6afd61c9d:
|
||||
// a large package catalog outlived the 60-min window mid-install).
|
||||
public sealed partial class DecisionEngine
|
||||
{
|
||||
// Hard 60-minute deadline from RJ-detected. Not configurable (per design choice —
|
||||
// the trigger is intentionally aggressive, the timeout has to bound it).
|
||||
// 60-minute deadline from RJ-detected. Not configurable (per design choice —
|
||||
// the trigger is intentionally aggressive, the timeout has to bound it). Doubles
|
||||
// as the inactivity window for the activity-based extension: RJ package writes are
|
||||
// only observable at package COMPLETION (RJ creates the registry subkey when the
|
||||
// install finishes), so a single long install is registry-silent for its whole
|
||||
// duration and the window must not be tightened below the original 60 min.
|
||||
private static readonly TimeSpan s_realmJoinHardTimeout = TimeSpan.FromMinutes(60);
|
||||
|
||||
// Absolute ceiling for activity-based extensions, measured from RJ detection.
|
||||
// Preserves the original design intent (the aggressive detection trigger stays
|
||||
// bounded) and sits safely inside AgentMaxLifetimeMinutes (360) — the agent's own
|
||||
// lifetime watchdog would otherwise be the only backstop.
|
||||
private static readonly TimeSpan s_realmJoinAbsoluteTimeout = TimeSpan.FromHours(4);
|
||||
|
||||
// Payload keys consumed by the new handlers (set by the agent's RealmJoinWatcherAdapter).
|
||||
// Public because the V2.Core agent adapter assembles signal payloads with these keys.
|
||||
public static class RealmJoinPayloadKeys
|
||||
@@ -64,14 +78,16 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
|| facts.Outcome != null;
|
||||
|
||||
/// <summary>
|
||||
/// Build the 60-min RJ hard-timeout deadline. Floored at <see cref="DecisionState.AgentBootUtc"/>
|
||||
/// via <see cref="EffectiveDeadlineBase"/> so a replayed RealmJoinDetected signal cannot
|
||||
/// collapse the timer into immediate-fire at boot.
|
||||
/// Build the RJ timeout deadline for an explicit due time. Initial arm: detection
|
||||
/// (floored at <see cref="DecisionState.AgentBootUtc"/> via <see cref="EffectiveDeadlineBase"/>
|
||||
/// so a replayed RealmJoinDetected signal cannot collapse the timer into immediate-fire
|
||||
/// at boot) + 60 min. Re-arm (activity-based extension): last activity + 60 min,
|
||||
/// capped at detection + 4 h.
|
||||
/// </summary>
|
||||
private static ActiveDeadline BuildRealmJoinTimeoutDeadline(DateTime fromUtc) =>
|
||||
private static ActiveDeadline BuildRealmJoinTimeoutDeadline(DateTime dueAtUtc) =>
|
||||
new ActiveDeadline(
|
||||
name: DeadlineNames.RealmJoinTimeout,
|
||||
dueAtUtc: fromUtc.Add(s_realmJoinHardTimeout),
|
||||
dueAtUtc: dueAtUtc,
|
||||
firesSignalKind: DecisionSignalKind.DeadlineFired,
|
||||
firesPayload: new Dictionary<string, string>
|
||||
{
|
||||
@@ -134,7 +150,8 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
var effects = Array.Empty<DecisionEffect>();
|
||||
if (!alreadyDetected)
|
||||
{
|
||||
var deadline = BuildRealmJoinTimeoutDeadline(EffectiveDeadlineBase(state, signal));
|
||||
var deadline = BuildRealmJoinTimeoutDeadline(
|
||||
EffectiveDeadlineBase(state, signal).Add(s_realmJoinHardTimeout));
|
||||
builder.AddDeadline(deadline);
|
||||
effects = new[]
|
||||
{
|
||||
@@ -224,13 +241,20 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
private DecisionStep HandleRealmJoinTimeoutDeadlineFired(DecisionState state, DecisionSignal signal)
|
||||
{
|
||||
var alreadyResolvedOrTimedOut = state.RealmJoinFacts.Outcome != null;
|
||||
var deadlineStillArmed = false;
|
||||
ActiveDeadline? armedDeadline = null;
|
||||
foreach (var d in state.Deadlines)
|
||||
{
|
||||
if (d.Name == DeadlineNames.RealmJoinTimeout) { deadlineStillArmed = true; break; }
|
||||
if (d.Name == DeadlineNames.RealmJoinTimeout) { armedDeadline = d; break; }
|
||||
}
|
||||
|
||||
if (alreadyResolvedOrTimedOut || !deadlineStillArmed)
|
||||
// Third staleness shape (introduced with the activity-based extension): the fire
|
||||
// belongs to an OLDER deadline incarnation that a re-arm has since replaced —
|
||||
// recognizable because the armed deadline is due LATER than this fire
|
||||
// (OccurredAtUtc = DueAtUtc per the scheduler contract). Without this guard the
|
||||
// stale fire would evaluate the timeout/extension decision ahead of schedule.
|
||||
var supersededByRearm = armedDeadline != null && armedDeadline.DueAtUtc > signal.OccurredAtUtc;
|
||||
|
||||
if (alreadyResolvedOrTimedOut || armedDeadline == null || supersededByRearm)
|
||||
{
|
||||
var bookkept = BumpStepBookkeeping(state, signal);
|
||||
var staleTransition = BuildDeadEndTransition(
|
||||
@@ -240,12 +264,60 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
trigger: $"DeadlineFired:{DeadlineNames.RealmJoinTimeout}",
|
||||
deadEndReason: alreadyResolvedOrTimedOut
|
||||
? "realmjoin_timeout_stale_outcome_already_set"
|
||||
: "realmjoin_timeout_stale_deadline_not_armed");
|
||||
: armedDeadline == null
|
||||
? "realmjoin_timeout_stale_deadline_not_armed"
|
||||
: "realmjoin_timeout_stale_superseded_by_rearm");
|
||||
return new DecisionStep(bookkept, staleTransition, Array.Empty<DecisionEffect>());
|
||||
}
|
||||
|
||||
var nextStep = state.StepIndex + 1;
|
||||
|
||||
// Activity-based extension (report 55e6afd61c9d): when the first deployment is
|
||||
// demonstrably still active — phase 100/101 with deployment activity (phase change
|
||||
// or package observation) inside the last 60 min — re-arm instead of cutting off
|
||||
// mid-install. Bounded by an absolute 4-h cap from detection so the original
|
||||
// "aggressive trigger stays bounded" design intent survives. The idle case
|
||||
// (detected, but the first deployment never produced any activity) is unchanged:
|
||||
// LastActivityUtc stays null and the 60-min timeout fires exactly as before.
|
||||
var facts = state.RealmJoinFacts;
|
||||
var now = signal.OccurredAtUtc;
|
||||
var lastActivity = facts.LastActivityUtc?.Value;
|
||||
var detectedUtc = facts.DetectedUtc?.Value;
|
||||
var inFirstDeployment = facts.LastDeploymentPhase != null
|
||||
&& IsRealmJoinFirstDeploymentPhase(facts.LastDeploymentPhase.Value);
|
||||
var recentActivity = lastActivity != null && now - lastActivity.Value < s_realmJoinHardTimeout;
|
||||
var withinCap = detectedUtc != null && now - detectedUtc.Value < s_realmJoinAbsoluteTimeout;
|
||||
|
||||
if (inFirstDeployment && recentActivity && withinCap)
|
||||
{
|
||||
var windowDue = lastActivity!.Value.Add(s_realmJoinHardTimeout);
|
||||
var capDue = detectedUtc!.Value.Add(s_realmJoinAbsoluteTimeout);
|
||||
var newDue = windowDue < capDue ? windowDue : capDue;
|
||||
|
||||
var rearm = BuildRealmJoinTimeoutDeadline(newDue);
|
||||
var rearmBuilder = state.ToBuilder()
|
||||
.WithStepIndex(nextStep)
|
||||
.WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal)
|
||||
.CancelDeadline(DeadlineNames.RealmJoinTimeout)
|
||||
.AddDeadline(rearm);
|
||||
|
||||
var rearmState = rearmBuilder.Build();
|
||||
var rearmTransition = BuildTakenTransition(
|
||||
before: state,
|
||||
signal: signal,
|
||||
toStage: state.Stage,
|
||||
nextStepIndex: nextStep,
|
||||
trigger: $"DeadlineFired:{DeadlineNames.RealmJoinTimeout}:Extended");
|
||||
|
||||
var rearmEffects = new[]
|
||||
{
|
||||
new DecisionEffect(DecisionEffectKind.ScheduleDeadline, deadline: rearm),
|
||||
BuildRealmJoinTimeoutExtendedEvent(facts, lastActivity.Value, newDue),
|
||||
};
|
||||
|
||||
return new DecisionStep(rearmState, rearmTransition, rearmEffects);
|
||||
}
|
||||
|
||||
var builder = state.ToBuilder()
|
||||
.WithStepIndex(nextStep)
|
||||
.WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal)
|
||||
@@ -253,7 +325,15 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
|
||||
builder.RealmJoinFacts = state.RealmJoinFacts.WithTimeoutOutcome(signal.SessionSignalOrdinal);
|
||||
|
||||
var timeoutEffect = BuildRealmJoinTimeoutEvent(state);
|
||||
// Message differentiates WHY the gate gave up: absolute cap exhausted while still
|
||||
// active, activity went quiet, or the original hard timeout (no activity ever seen).
|
||||
var reason = inFirstDeployment && recentActivity && !withinCap
|
||||
? RealmJoinTimeoutReasonAbsoluteCap
|
||||
: inFirstDeployment && lastActivity != null
|
||||
? RealmJoinTimeoutReasonInactivity
|
||||
: RealmJoinTimeoutReasonHardTimeout;
|
||||
|
||||
var timeoutEffect = BuildRealmJoinTimeoutEvent(state, reason);
|
||||
|
||||
return CompleteIfDeferredOrBookkeep(
|
||||
state: state,
|
||||
@@ -331,7 +411,9 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
var builder = state.ToBuilder()
|
||||
.WithStepIndex(nextStep)
|
||||
.WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal);
|
||||
builder.RealmJoinFacts = state.RealmJoinFacts.WithLastPhase(currentPhase.Value, signal.SessionSignalOrdinal);
|
||||
builder.RealmJoinFacts = state.RealmJoinFacts
|
||||
.WithLastPhase(currentPhase.Value, signal.SessionSignalOrdinal)
|
||||
.WithActivity(signal.OccurredAtUtc, signal.SessionSignalOrdinal);
|
||||
|
||||
if (!IsFirstDeploymentAbortTransition(state.RealmJoinFacts, previousPhase, currentPhase.Value))
|
||||
{
|
||||
@@ -442,12 +524,14 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
var builder = state.ToBuilder()
|
||||
.WithStepIndex(nextStep)
|
||||
.WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal);
|
||||
builder.RealmJoinFacts = state.RealmJoinFacts.WithPackageStarted(
|
||||
packageId: packageId!,
|
||||
displayName: displayName,
|
||||
version: version,
|
||||
scope: scope,
|
||||
startedUtc: signal.OccurredAtUtc);
|
||||
builder.RealmJoinFacts = state.RealmJoinFacts
|
||||
.WithPackageStarted(
|
||||
packageId: packageId!,
|
||||
displayName: displayName,
|
||||
version: version,
|
||||
scope: scope,
|
||||
startedUtc: signal.OccurredAtUtc)
|
||||
.WithActivity(signal.OccurredAtUtc, signal.SessionSignalOrdinal);
|
||||
|
||||
var newState = builder.Build();
|
||||
var transition = BuildTakenTransition(
|
||||
@@ -490,14 +574,16 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
var builder = state.ToBuilder()
|
||||
.WithStepIndex(nextStep)
|
||||
.WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal);
|
||||
builder.RealmJoinFacts = state.RealmJoinFacts.WithPackageCompleted(
|
||||
packageId: packageId!,
|
||||
displayName: displayName,
|
||||
version: version,
|
||||
scope: scope,
|
||||
completedUtc: signal.OccurredAtUtc,
|
||||
success: success,
|
||||
lastExitCode: lastExitCode);
|
||||
builder.RealmJoinFacts = state.RealmJoinFacts
|
||||
.WithPackageCompleted(
|
||||
packageId: packageId!,
|
||||
displayName: displayName,
|
||||
version: version,
|
||||
scope: scope,
|
||||
completedUtc: signal.OccurredAtUtc,
|
||||
success: success,
|
||||
lastExitCode: lastExitCode)
|
||||
.WithActivity(signal.OccurredAtUtc, signal.SessionSignalOrdinal);
|
||||
|
||||
var newState = builder.Build();
|
||||
var transition = BuildTakenTransition(
|
||||
@@ -654,6 +740,15 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
return null;
|
||||
}
|
||||
|
||||
// Reason strings carried in the realmjoin_timeout event payload ("reason") and
|
||||
// reflected in its message. HardTimeout = original semantics (no deployment activity
|
||||
// ever observed, 60 min from detection); Inactivity = the first deployment produced
|
||||
// activity at some point but went quiet for a full 60-min window; AbsoluteCap = still
|
||||
// active but the 4-h extension ceiling from detection is exhausted.
|
||||
internal const string RealmJoinTimeoutReasonHardTimeout = "hard_timeout";
|
||||
internal const string RealmJoinTimeoutReasonInactivity = "inactivity";
|
||||
internal const string RealmJoinTimeoutReasonAbsoluteCap = "absolute_cap";
|
||||
|
||||
/// <summary>
|
||||
/// Build the <c>realmjoin_timeout</c> timeline-entry effect emitted by the deadline-
|
||||
/// fired handler. Unlike <see cref="DecisionSignalKind.RealmJoinDetected"/> /
|
||||
@@ -661,7 +756,7 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
/// InformationalEvent), the timeout is a synthetic deadline so the reducer owns its
|
||||
/// timeline visibility.
|
||||
/// </summary>
|
||||
private static DecisionEffect BuildRealmJoinTimeoutEvent(DecisionState state)
|
||||
private static DecisionEffect BuildRealmJoinTimeoutEvent(DecisionState state, string reason)
|
||||
{
|
||||
var facts = state.RealmJoinFacts;
|
||||
var lastPhase = facts.LastDeploymentPhase?.Value ?? 0;
|
||||
@@ -672,17 +767,60 @@ namespace AutopilotMonitor.DecisionCore.Engine
|
||||
if (facts.Packages[i].CompletedUtc != null) completed++;
|
||||
}
|
||||
|
||||
var message = reason == RealmJoinTimeoutReasonAbsoluteCap
|
||||
? $"RealmJoin did not reach phase 110 within {(int)s_realmJoinAbsoluteTimeout.TotalHours} h of detection (last phase: {lastPhase}) — monitoring window exhausted despite recent deployment activity."
|
||||
: reason == RealmJoinTimeoutReasonInactivity
|
||||
? $"RealmJoin did not reach phase 110 — no deployment activity for {(int)s_realmJoinHardTimeout.TotalMinutes} min (last phase: {lastPhase})."
|
||||
: $"RealmJoin did not reach phase 110 within {(int)s_realmJoinHardTimeout.TotalMinutes} min (last phase: {lastPhase}).";
|
||||
|
||||
return new DecisionEffect(
|
||||
kind: DecisionEffectKind.EmitEventTimelineEntry,
|
||||
parameters: new Dictionary<string, string>(StringComparer.Ordinal)
|
||||
{
|
||||
[SignalPayloadKeys.EventType] = SharedConstants.EventTypes.RealmJoinTimeout,
|
||||
[SignalPayloadKeys.Source] = "DecisionEngine",
|
||||
[SignalPayloadKeys.Severity] = "Warning",
|
||||
[SignalPayloadKeys.Message] = message,
|
||||
["lastSeenPhase"] = lastPhase.ToString(CultureInfo.InvariantCulture),
|
||||
["packagesTracked"] = tracked.ToString(CultureInfo.InvariantCulture),
|
||||
["packagesCompleted"] = completed.ToString(CultureInfo.InvariantCulture),
|
||||
["reason"] = reason,
|
||||
});
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Build the <c>realmjoin_timeout_extended</c> timeline-entry effect emitted when the
|
||||
/// timeout deadline fires but the first deployment is demonstrably still active and
|
||||
/// the monitoring window is re-armed instead. Reducer-owned like
|
||||
/// <see cref="BuildRealmJoinTimeoutEvent"/> (synthetic deadline, no agent dual-emit).
|
||||
/// </summary>
|
||||
private static DecisionEffect BuildRealmJoinTimeoutExtendedEvent(
|
||||
RealmJoinFacts facts,
|
||||
DateTime lastActivityUtc,
|
||||
DateTime newDueUtc)
|
||||
{
|
||||
var lastPhase = facts.LastDeploymentPhase?.Value ?? 0;
|
||||
var tracked = facts.Packages.Count;
|
||||
var completed = 0;
|
||||
for (var i = 0; i < facts.Packages.Count; i++)
|
||||
{
|
||||
if (facts.Packages[i].CompletedUtc != null) completed++;
|
||||
}
|
||||
|
||||
return new DecisionEffect(
|
||||
kind: DecisionEffectKind.EmitEventTimelineEntry,
|
||||
parameters: new Dictionary<string, string>(StringComparer.Ordinal)
|
||||
{
|
||||
[SignalPayloadKeys.EventType] = SharedConstants.EventTypes.RealmJoinTimeout,
|
||||
[SignalPayloadKeys.EventType] = SharedConstants.EventTypes.RealmJoinTimeoutExtended,
|
||||
[SignalPayloadKeys.Source] = "DecisionEngine",
|
||||
[SignalPayloadKeys.Severity] = "Warning",
|
||||
[SignalPayloadKeys.Message] = $"RealmJoin did not reach phase 110 within 60 min (last phase: {lastPhase}).",
|
||||
["lastSeenPhase"] = lastPhase.ToString(CultureInfo.InvariantCulture),
|
||||
[SignalPayloadKeys.Severity] = "Info",
|
||||
[SignalPayloadKeys.Message] =
|
||||
$"RealmJoin first deployment still active (phase {lastPhase}, {completed}/{tracked} packages completed, last activity {lastActivityUtc:HH:mm:ss} UTC) — extending monitoring window to {newDueUtc:HH:mm:ss} UTC (cap {(int)s_realmJoinAbsoluteTimeout.TotalHours} h after detection).",
|
||||
["deploymentPhase"] = lastPhase.ToString(CultureInfo.InvariantCulture),
|
||||
["packagesTracked"] = tracked.ToString(CultureInfo.InvariantCulture),
|
||||
["packagesCompleted"] = completed.ToString(CultureInfo.InvariantCulture),
|
||||
["lastActivityUtc"] = lastActivityUtc.ToString("O", CultureInfo.InvariantCulture),
|
||||
["extendedUntilUtc"] = newDueUtc.ToString("O", CultureInfo.InvariantCulture),
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -39,7 +39,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: null,
|
||||
productVersion: null,
|
||||
releaseChannel: null,
|
||||
packages: Array.Empty<RealmJoinPackageFact>());
|
||||
packages: Array.Empty<RealmJoinPackageFact>(),
|
||||
lastActivityUtc: null);
|
||||
|
||||
public RealmJoinFacts(
|
||||
SignalFact<DateTime>? detectedUtc,
|
||||
@@ -49,7 +50,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
SignalFact<bool>? selfDeployingDeferredCompletion,
|
||||
SignalFact<string>? productVersion,
|
||||
SignalFact<string>? releaseChannel,
|
||||
IReadOnlyList<RealmJoinPackageFact> packages)
|
||||
IReadOnlyList<RealmJoinPackageFact> packages,
|
||||
SignalFact<DateTime>? lastActivityUtc = null)
|
||||
{
|
||||
DetectedUtc = detectedUtc;
|
||||
ResolvedUtc = resolvedUtc;
|
||||
@@ -59,6 +61,7 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
ProductVersion = productVersion;
|
||||
ReleaseChannel = releaseChannel;
|
||||
Packages = packages ?? Array.Empty<RealmJoinPackageFact>();
|
||||
LastActivityUtc = lastActivityUtc;
|
||||
}
|
||||
|
||||
/// <summary>First-observation timestamp of the <c>HKLM\SYSTEM\...\realmjoin\Parameters</c> key.</summary>
|
||||
@@ -98,6 +101,15 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
/// <summary>Tracked per-package install rows (machine + user scope combined).</summary>
|
||||
public IReadOnlyList<RealmJoinPackageFact> Packages { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Most recent observed RJ deployment activity: a <c>RealmJoinPhaseChanged</c> or a
|
||||
/// per-package started/completed observation. Deliberately NOT set at detection time —
|
||||
/// the RJ-detected-but-never-deploying idle case must still hit the 60-min hard
|
||||
/// timeout unchanged. Drives the activity-based timeout extension (report 55e6afd61c9d:
|
||||
/// the hard timeout cut off a demonstrably active first deployment mid-install).
|
||||
/// </summary>
|
||||
public SignalFact<DateTime>? LastActivityUtc { get; }
|
||||
|
||||
public RealmJoinFacts WithDetected(DateTime utc, long sourceSignalOrdinal)
|
||||
{
|
||||
if (DetectedUtc != null) return this; // set-once
|
||||
@@ -109,7 +121,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: Packages);
|
||||
packages: Packages,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -129,7 +142,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: new SignalFact<string>(productVersion, sourceSignalOrdinal),
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: Packages);
|
||||
packages: Packages,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -149,7 +163,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: new SignalFact<string>(releaseChannel, sourceSignalOrdinal),
|
||||
packages: Packages);
|
||||
packages: Packages,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
public RealmJoinFacts WithResolved(DateTime utc, int phase, long sourceSignalOrdinal)
|
||||
@@ -163,7 +178,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: Packages);
|
||||
packages: Packages,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -184,7 +200,28 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: Packages);
|
||||
packages: Packages,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Record an RJ deployment-activity observation (phase change or package event).
|
||||
/// Monotonic — an earlier or equal timestamp (replayed signal) is a no-op, so the
|
||||
/// activity fact can never move backwards.
|
||||
/// </summary>
|
||||
public RealmJoinFacts WithActivity(DateTime utc, long sourceSignalOrdinal)
|
||||
{
|
||||
if (LastActivityUtc != null && LastActivityUtc.Value >= utc) return this;
|
||||
return new RealmJoinFacts(
|
||||
detectedUtc: DetectedUtc,
|
||||
resolvedUtc: ResolvedUtc,
|
||||
lastDeploymentPhase: LastDeploymentPhase,
|
||||
outcome: Outcome,
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: Packages,
|
||||
lastActivityUtc: new SignalFact<DateTime>(utc, sourceSignalOrdinal));
|
||||
}
|
||||
|
||||
public RealmJoinFacts WithLastPhase(int phase, long sourceSignalOrdinal)
|
||||
@@ -198,7 +235,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: Packages);
|
||||
packages: Packages,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
public RealmJoinFacts WithTimeoutOutcome(long sourceSignalOrdinal)
|
||||
@@ -212,7 +250,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: Packages);
|
||||
packages: Packages,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
public RealmJoinFacts WithSelfDeployingDeferred(long sourceSignalOrdinal)
|
||||
@@ -226,7 +265,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: new SignalFact<bool>(true, sourceSignalOrdinal),
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: Packages);
|
||||
packages: Packages,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -246,7 +286,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: null,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: Packages);
|
||||
packages: Packages,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -301,7 +342,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: copy);
|
||||
packages: copy,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -370,7 +412,8 @@ namespace AutopilotMonitor.DecisionCore.State
|
||||
selfDeployingDeferredCompletion: SelfDeployingDeferredCompletion,
|
||||
productVersion: ProductVersion,
|
||||
releaseChannel: ReleaseChannel,
|
||||
packages: copy);
|
||||
packages: copy,
|
||||
lastActivityUtc: LastActivityUtc);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -512,12 +512,17 @@ namespace AutopilotMonitor.Shared
|
||||
// RealmJoin (RJ) deployment tracking — driven by HKLM\SYSTEM\...\realmjoin\Parameters
|
||||
// + HKLM/HKU \SOFTWARE\RealmJoin\Packages\<id>. Detection extends the V2 enrollment
|
||||
// session lifetime until DeploymentPhase reaches CompletedFirstDeployment (110) or
|
||||
// the 60-min hard timeout fires. Per-package events surface install lifecycle in
|
||||
// the 60-min timeout fires. Per-package events surface install lifecycle in
|
||||
// the same shape as IME app_install_started / app_install_completed.
|
||||
public const string RealmJoinDetected = "realmjoin_detected";
|
||||
public const string RealmJoinPhaseChanged = "realmjoin_phase_changed";
|
||||
public const string RealmJoinResolved = "realmjoin_resolved";
|
||||
public const string RealmJoinTimeout = "realmjoin_timeout";
|
||||
// Info: the timeout deadline fired while the first deployment was demonstrably
|
||||
// still active (phase 100/101, deployment activity inside the last 60 min) —
|
||||
// the monitoring window was re-armed instead of timing out (capped at 4 h from
|
||||
// detection). Report 55e6afd61c9d.
|
||||
public const string RealmJoinTimeoutExtended = "realmjoin_timeout_extended";
|
||||
// Warning: DeploymentPhase left the first-deployment window (100/101) for 200/210
|
||||
// without CompletedFirstDeployment (110) — the RJ ESP was aborted (typically an
|
||||
// interactive logon during the first deployment reclassified the run as
|
||||
@@ -555,8 +560,12 @@ namespace AutopilotMonitor.Shared
|
||||
public const string DeviceLocation = "device_location";
|
||||
public const string OutboundIp = "outbound_ip";
|
||||
public const string TimezoneAutoSet = "timezone_auto_set";
|
||||
public const string DoGroupIdAutoSet = "do_group_id_auto_set";
|
||||
public const string NtpTimeCheck = "ntp_time_check";
|
||||
public const string PowerStateCheck = "power_state_check";
|
||||
public const string PowerStateChange = "power_state_change";
|
||||
public const string SystemClockChanged = "system_clock_changed"; // Kernel-General EventID 1: OS clock stepped (oldTime/newTime/timeDeltaMs in Data are authoritative — the event's own Timestamp may be clamped server-side).
|
||||
public const string SystemSleepEpisode = "system_sleep_episode"; // Completed sleep/hibernate/modern-standby episode (Power-Troubleshooter 1 / Kernel-Power 507), emitted retroactively at wake.
|
||||
public const string AgentTrace = "agent_trace";
|
||||
|
||||
// Crash / process lifecycle
|
||||
|
||||
@@ -225,6 +225,14 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// </summary>
|
||||
public bool EnableTimezoneAutoSet { get; set; } = false;
|
||||
|
||||
/// <summary>
|
||||
/// Whether to set the Delivery Optimization group ID (DOGroupId policy value) from a
|
||||
/// deterministic network fingerprint (default gateway IP + MAC, RealmJoin-compatible
|
||||
/// byte layout). Existing DOGroupId/DOGroupIdSource policies are never overwritten.
|
||||
/// Default: false
|
||||
/// </summary>
|
||||
public bool EnableDoGroupIdAutoSet { get; set; } = false;
|
||||
|
||||
/// <summary>
|
||||
/// Control-channel mirror of the telemetry-response block signal: the calling device
|
||||
/// (serial) or agent version matched an active admin Block/Kill rule. Delivered on the
|
||||
@@ -355,6 +363,16 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// </summary>
|
||||
public int OfficeInstallSettleSeconds { get; set; } = 10;
|
||||
|
||||
/// <summary>
|
||||
/// Enable the live AC/battery power-state watcher. Event-driven (no polling): a WMI
|
||||
/// Win32_PowerManagementEvent (EventCode 10) push triggers a GetSystemPowerStatus
|
||||
/// re-probe; emits power_state_change on AC↔battery transitions and downward battery
|
||||
/// threshold crossings (50/30/15 %). Never armed on devices without a battery.
|
||||
/// Cheap and high-value → on by default, kept as a kill-switch.
|
||||
/// Default: true
|
||||
/// </summary>
|
||||
public bool EnablePowerStateWatcher { get; set; } = true;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Stall detection (Ebene 2 — StallProbeCollector)
|
||||
// -----------------------------------------------------------------------
|
||||
@@ -520,6 +538,30 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// </summary>
|
||||
public int MdmRebootPolicyBackfillLookbackMinutes { get; set; } = 60;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// System timeline watcher (System channel: clock changes + sleep episodes)
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/// <summary>
|
||||
/// Master switch for the system timeline watcher. When enabled the agent subscribes to the
|
||||
/// System event log for Kernel-General EventID 1 (system clock stepped — ground truth for
|
||||
/// time jumps incl. old/new time, delta and the setting process) and completed sleep
|
||||
/// episodes (Power-Troubleshooter EventID 1 for classic sleep/hibernate, Kernel-Power
|
||||
/// EventID 507 for Modern Standby) and emits system_clock_changed / system_sleep_episode
|
||||
/// events. Explains timeline gaps and time jumps that are otherwise blind spots.
|
||||
/// Default: true.
|
||||
/// </summary>
|
||||
public bool EnableSystemTimelineWatcher { get; set; } = true;
|
||||
|
||||
/// <summary>
|
||||
/// Lookback window in minutes for the system timeline backfill scan on startup. The
|
||||
/// decisive clock correction (w32time fixing a wrong BIOS clock) and pre-agent standby
|
||||
/// episodes happen early in OOBE, often before the agent starts; the System .evtx persists
|
||||
/// across reboots, so a generous backfill recovers them. Bounded well inside the backend's
|
||||
/// 168 h past-tolerance clamp. 0 = backfill disabled. Default: 1440 minutes (24 h).
|
||||
/// </summary>
|
||||
public int SystemEventBackfillLookbackMinutes { get; set; } = 1440;
|
||||
|
||||
/// <summary>
|
||||
/// Creates default collector configuration
|
||||
/// </summary>
|
||||
|
||||
@@ -366,6 +366,16 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// </summary>
|
||||
public bool? EnableTimezoneAutoSet { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Whether to set the Delivery Optimization group ID (DOGroupId policy value) from a
|
||||
/// network fingerprint: a deterministic GUID derived from the default gateway's IP and
|
||||
/// MAC address, so devices on the same local network peer with each other (byte layout
|
||||
/// is RealmJoin-compatible). Only takes effect with DO Download Mode = Group (2);
|
||||
/// existing DOGroupId/DOGroupIdSource policies (Intune/GPO) are never overwritten.
|
||||
/// null = use agent default (false).
|
||||
/// </summary>
|
||||
public bool? EnableDoGroupIdAutoSet { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Whether to write a match log for every IME log line matched by a pattern.
|
||||
/// When true, the agent writes to the default path Constants.ImeMatchLogPath.
|
||||
|
||||
@@ -39,8 +39,17 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// <summary>Succeeded / (Succeeded + Failed) * 100 (one decimal) — finished enrollments only,
|
||||
/// matching the SLA convention. 0 when nothing has finished yet (clients render "—").</summary>
|
||||
public double SuccessRate { get; set; }
|
||||
/// <summary>Average duration in minutes over non-in-progress sessions that carry a duration.</summary>
|
||||
/// <summary>Average duration in minutes over non-in-progress sessions that carry a duration.
|
||||
/// Kept for API compatibility; the cards lead with the median, which a handful of
|
||||
/// multi-hour outliers (overnight ESP, WhiteGlove late user phase) cannot drag around.</summary>
|
||||
public int AvgDurationMinutes { get; set; }
|
||||
|
||||
/// <summary>Median duration in minutes over the same population as <see cref="AvgDurationMinutes"/>.</summary>
|
||||
public int MedianDurationMinutes { get; set; }
|
||||
|
||||
/// <summary>90th-percentile duration in minutes over the same population — the tail signal
|
||||
/// the median alone would hide.</summary>
|
||||
public int P90DurationMinutes { get; set; }
|
||||
}
|
||||
|
||||
public sealed class FleetDailyPoint
|
||||
|
||||
@@ -66,7 +66,14 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// </summary>
|
||||
public string TerminalState { get; set; } = string.Empty;
|
||||
|
||||
/// <summary>Total installation duration in seconds (from start to complete/failed)</summary>
|
||||
/// <summary>
|
||||
/// Duration in seconds of the status-defining (last observed) install attempt:
|
||||
/// <see cref="CompletedAt"/> minus <see cref="LastAttemptStartedAt"/>. Falls back to
|
||||
/// the <see cref="StartedAt"/> anchor (the historical full-span semantics) only when
|
||||
/// no <c>app_install_started</c> was ever observed (legacy agents, terminal-only
|
||||
/// observation). Until 2026-08 this column was the first-observation → last-terminal
|
||||
/// span across ALL install passes — rows from before the change still carry that value.
|
||||
/// </summary>
|
||||
public int DurationSeconds { get; set; }
|
||||
|
||||
/// <summary>Total download size in bytes</summary>
|
||||
@@ -81,12 +88,36 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// <summary>Error message if failed</summary>
|
||||
public string FailureMessage { get; set; } = string.Empty;
|
||||
|
||||
/// <summary>When this app install started</summary>
|
||||
/// <summary>
|
||||
/// When this app was FIRST observed in this session (earliest app-relevant event,
|
||||
/// including download progress). This is the server-side window/bucket filter column
|
||||
/// for every AppInstallSummaries scan and the version-ordering key of the regression
|
||||
/// radar — it is never re-anchored to a later attempt (earliest always wins).
|
||||
/// </summary>
|
||||
public DateTime StartedAt { get; set; }
|
||||
|
||||
/// <summary>When this app install completed or failed</summary>
|
||||
public DateTime? CompletedAt { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Device-frame timestamp of the most recent observed <c>app_install_started</c> for
|
||||
/// this app in this session — the anchor of <see cref="DurationSeconds"/>. Folds via
|
||||
/// max in-batch and across batches (a replayed older batch can never regress it).
|
||||
/// Null = sentinel: no start observed (row predates the 2026-08 attempt-duration
|
||||
/// change, or the agent never emitted a start) — such rows keep the span fallback.
|
||||
/// </summary>
|
||||
public DateTime? LastAttemptStartedAt { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Number of observed install passes (<c>app_install_started</c> events) across the
|
||||
/// session. The IME processes the app list repeatedly (device-ESP evaluation pass,
|
||||
/// user-context pass, retries), so 2+ is normal and explains a
|
||||
/// <see cref="CompletedAt"/> far after <see cref="StartedAt"/>. Additive across
|
||||
/// batches, guarded by <see cref="LastAttemptStartedAt"/> monotonicity so a replayed
|
||||
/// batch is not double-counted. 0 = sentinel (legacy row / never observed).
|
||||
/// </summary>
|
||||
public int InstallPassCount { get; set; }
|
||||
|
||||
// Delivery Optimization telemetry
|
||||
/// <summary>DO: total file size reported by DO</summary>
|
||||
public long DoFileSize { get; set; }
|
||||
|
||||
@@ -147,6 +147,27 @@ namespace AutopilotMonitor.Shared.Models
|
||||
public string SegmentKey { get; set; } = string.Empty;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// One completed sleep episode observed during the session (<c>system_sleep_episode</c>
|
||||
/// payload — <c>enteredAt</c>/<c>exitedAt</c> are the authoritative instants, never the
|
||||
/// event's own timestamp, which is stamped at wake and may be clamped). Cross-cutting
|
||||
/// annotation like <see cref="RebootSpan"/>: sleep overlaps the segment it started in and is
|
||||
/// NOT part of the wall-clock partition — the session really took that long; the span
|
||||
/// explains where the time went.
|
||||
/// </summary>
|
||||
public class SleepSpan
|
||||
{
|
||||
public DateTime StartUtc { get; set; }
|
||||
public DateTime EndUtc { get; set; }
|
||||
public int Seconds { get; set; }
|
||||
|
||||
/// <summary>Segment the episode started in (<see cref="TimeAttributionSegments"/> key, or "unattributed").</summary>
|
||||
public string SegmentKey { get; set; } = string.Empty;
|
||||
|
||||
/// <summary>Episode kind from the agent payload: "sleep", "hibernate" or "modern_standby".</summary>
|
||||
public string Kind { get; set; } = string.Empty;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Per-session enrollment time attribution (F1, insights spec) — an exact partition of the
|
||||
/// session's authoritative wall clock into named segments plus an explicit unattributed
|
||||
@@ -187,6 +208,11 @@ namespace AutopilotMonitor.Shared.Models
|
||||
|
||||
public List<RebootSpan> RebootSpans { get; set; } = new List<RebootSpan>();
|
||||
|
||||
/// <summary>Total observed in-window sleep seconds. Cross-cutting annotation like <see cref="RebootSeconds"/> — the wall clock keeps the pause; this discloses it.</summary>
|
||||
public int SleepSeconds { get; set; }
|
||||
|
||||
public List<SleepSpan> SleepSpans { get; set; } = new List<SleepSpan>();
|
||||
|
||||
/// <summary>
|
||||
/// Install intervals of ESP-blocking apps (positive-evidence join against the latest
|
||||
/// <c>esp_config_detected</c> lists), top 20 by duration. <see cref="BlockingAppCount"/>
|
||||
|
||||
@@ -576,9 +576,17 @@ namespace AutopilotMonitor.Shared.Models
|
||||
/// </summary>
|
||||
public int SuccessRatePct { get; set; }
|
||||
|
||||
/// <summary>Average duration of Succeeded sessions in the window, in minutes (rounded).</summary>
|
||||
/// <summary>Average duration of Succeeded sessions in the window, in minutes (rounded).
|
||||
/// Kept for API compatibility; the dashboard card leads with the median, which
|
||||
/// multi-hour outliers cannot drag around.</summary>
|
||||
public int AvgDurationMinutes { get; set; }
|
||||
|
||||
/// <summary>Median duration in minutes over the same Succeeded population.</summary>
|
||||
public int MedianDurationMinutes { get; set; }
|
||||
|
||||
/// <summary>90th-percentile duration in minutes over the same population — the tail signal.</summary>
|
||||
public int P90DurationMinutes { get; set; }
|
||||
|
||||
/// <summary>Sessions whose StartedAt is on or after UTC midnight of the current day.</summary>
|
||||
public int TotalToday { get; set; }
|
||||
public int FailedToday { get; set; }
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
<!-- BEGIN:nextjs-agent-rules -->
|
||||
|
||||
# This is NOT the Next.js you know
|
||||
|
||||
This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in `node_modules/next/dist/docs/` (resolved from this file's directory; in monorepos the `next` package may not be visible from the repo root) before writing any code. Heed deprecation notices.
|
||||
|
||||
This block is written and re-added by `next dev` — verify at `node_modules/next/dist/server/lib/generate-agent-files.js`. Removing it from a diff only re-creates the uncommitted change; committing it with your work keeps the tree clean.
|
||||
|
||||
<!-- END:nextjs-agent-rules -->
|
||||
@@ -0,0 +1 @@
|
||||
@AGENTS.md
|
||||
@@ -57,6 +57,7 @@ interface TenantConfig {
|
||||
rebootDelaySeconds?: number | null;
|
||||
enableGeoLocation?: boolean | null;
|
||||
enableTimezoneAutoSet?: boolean | null;
|
||||
enableDoGroupIdAutoSet?: boolean | null;
|
||||
ntpServer?: string | null;
|
||||
enableImeMatchLog?: boolean | null;
|
||||
enableGatherRuleDebugLog?: boolean | null;
|
||||
@@ -131,6 +132,7 @@ const DEFAULTS: Record<string, unknown> = {
|
||||
rebootDelaySeconds: null,
|
||||
enableGeoLocation: null,
|
||||
enableTimezoneAutoSet: null,
|
||||
enableDoGroupIdAutoSet: null,
|
||||
ntpServer: 'time.windows.com',
|
||||
enableImeMatchLog: null,
|
||||
enableGatherRuleDebugLog: null,
|
||||
@@ -171,6 +173,7 @@ const RUNTIME_DEFAULTS: Record<string, unknown> = {
|
||||
keepLogFile: false,
|
||||
enableGeoLocation: true,
|
||||
enableTimezoneAutoSet: false,
|
||||
enableDoGroupIdAutoSet: false,
|
||||
ntpServer: 'time.windows.com',
|
||||
enableImeMatchLog: false,
|
||||
enableGatherRuleDebugLog: false,
|
||||
@@ -301,6 +304,7 @@ function computeRuntime(c: TenantConfig): Record<string, unknown> {
|
||||
keepLogFile: c.keepLogFile ?? false,
|
||||
enableGeoLocation: c.enableGeoLocation ?? true,
|
||||
enableTimezoneAutoSet: c.enableTimezoneAutoSet ?? false,
|
||||
enableDoGroupIdAutoSet: c.enableDoGroupIdAutoSet ?? false,
|
||||
ntpServer: c.ntpServer ?? 'time.windows.com',
|
||||
enableImeMatchLog: c.enableImeMatchLog ?? false,
|
||||
enableGatherRuleDebugLog: c.enableGatherRuleDebugLog ?? false,
|
||||
@@ -587,6 +591,7 @@ export function SectionTenantConfigReport() {
|
||||
<ConfigRow label="Reboot Delay (sec)" value={config.rebootDelaySeconds} configKey="rebootDelaySeconds" defaults={DEFAULTS} />
|
||||
<ConfigRow label="Geo-Location" value={config.enableGeoLocation} configKey="enableGeoLocation" defaults={DEFAULTS} />
|
||||
<ConfigRow label="Timezone Auto-Set" value={config.enableTimezoneAutoSet} configKey="enableTimezoneAutoSet" defaults={DEFAULTS} />
|
||||
<ConfigRow label="DO GroupId Auto-Set" value={config.enableDoGroupIdAutoSet} configKey="enableDoGroupIdAutoSet" defaults={DEFAULTS} />
|
||||
<ConfigRow label="NTP Server" value={config.ntpServer} configKey="ntpServer" defaults={DEFAULTS} />
|
||||
<ConfigRow label="IME Match Log" value={config.enableImeMatchLog} configKey="enableImeMatchLog" defaults={DEFAULTS} />
|
||||
<ConfigRow label="Gather Rule Debug Log" value={config.enableGatherRuleDebugLog} configKey="enableGatherRuleDebugLog" defaults={DEFAULTS} />
|
||||
@@ -661,6 +666,7 @@ export function SectionTenantConfigReport() {
|
||||
<ConfigRow label="Reboot Delay (sec)" value={runtime.rebootDelaySeconds} configKey="rebootDelaySeconds" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="Geo-Location" value={runtime.enableGeoLocation} configKey="enableGeoLocation" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="Timezone Auto-Set" value={runtime.enableTimezoneAutoSet} configKey="enableTimezoneAutoSet" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="DO GroupId Auto-Set" value={runtime.enableDoGroupIdAutoSet} configKey="enableDoGroupIdAutoSet" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="NTP Server" value={runtime.ntpServer} configKey="ntpServer" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="IME Match Log" value={runtime.enableImeMatchLog} configKey="enableImeMatchLog" defaults={RUNTIME_DEFAULTS} />
|
||||
<ConfigRow label="Gather Rule Debug Log" value={runtime.enableGatherRuleDebugLog} configKey="enableGatherRuleDebugLog" defaults={RUNTIME_DEFAULTS} />
|
||||
|
||||
+97
@@ -0,0 +1,97 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { EMPTY_FORM, formToEvaluateOn, jsonToForm } from "../types";
|
||||
|
||||
/**
|
||||
* The JSON editor serializes the FORM object (eval* checkbox fields), but what users
|
||||
* paste is usually a rule EXPORT (the "Export" button's shape, e.g. received from
|
||||
* another tenant) which carries `evaluateOn` instead. Before the mapper, a pasted
|
||||
* export silently lost evaluateOn — the imported rule quietly reverted to the
|
||||
* enrollment-end default.
|
||||
*/
|
||||
describe("jsonToForm", () => {
|
||||
const baseExport = {
|
||||
ruleId: "ANALYZE-CUSTOM-101",
|
||||
title: "VPN client install failed",
|
||||
severity: "high",
|
||||
category: "network",
|
||||
conditions: [
|
||||
{
|
||||
signal: "vpn_failed",
|
||||
source: "event_data",
|
||||
eventType: "app_install_failed",
|
||||
dataField: "appName",
|
||||
operator: "contains",
|
||||
value: "VPN",
|
||||
required: true,
|
||||
},
|
||||
],
|
||||
baseConfidence: 70,
|
||||
confidenceThreshold: 50,
|
||||
tags: ["vpn", "network"],
|
||||
};
|
||||
|
||||
it("maps a rule-shaped export's evaluateOn onto the form flags", () => {
|
||||
const form = jsonToForm({
|
||||
...baseExport,
|
||||
evaluateOn: ["whiteglove_sealed", "on_event:vpn_error"],
|
||||
});
|
||||
|
||||
expect(form.evalAtEnrollmentEnd).toBe(false);
|
||||
expect(form.evalAtWhitegloveSealed).toBe(true);
|
||||
expect(form.evalOnEventTypes).toBe("vpn_error");
|
||||
expect(form.conditions).toHaveLength(1);
|
||||
expect(form.tags).toEqual(["vpn", "network"]);
|
||||
});
|
||||
|
||||
it("defaults a terminal-only export (no evaluateOn key) to enrollment end", () => {
|
||||
const form = jsonToForm(baseExport);
|
||||
|
||||
expect(form.evalAtEnrollmentEnd).toBe(true);
|
||||
expect(form.evalAtWhitegloveSealed).toBe(false);
|
||||
expect(form.evalOnEventTypes).toBe("");
|
||||
});
|
||||
|
||||
it("passes form-shaped JSON (eval* fields, no evaluateOn) through unchanged", () => {
|
||||
const form = jsonToForm({
|
||||
...EMPTY_FORM,
|
||||
ruleId: "ANALYZE-CUSTOM-102",
|
||||
title: "Pasted form",
|
||||
evalAtEnrollmentEnd: false,
|
||||
evalAtWhitegloveSealed: true,
|
||||
});
|
||||
|
||||
expect(form.evalAtEnrollmentEnd).toBe(false);
|
||||
expect(form.evalAtWhitegloveSealed).toBe(true);
|
||||
});
|
||||
|
||||
it("prefers an explicit evaluateOn over stale form flags on an edit merge", () => {
|
||||
// Edit-mode paste merges into the current form, so both shapes are present:
|
||||
// the export's evaluateOn must win over the edited rule's stale checkboxes.
|
||||
const form = jsonToForm({
|
||||
...EMPTY_FORM,
|
||||
...baseExport,
|
||||
evalAtEnrollmentEnd: true,
|
||||
evaluateOn: ["whiteglove_sealed"],
|
||||
});
|
||||
|
||||
expect(form.evalAtEnrollmentEnd).toBe(false);
|
||||
expect(form.evalAtWhitegloveSealed).toBe(true);
|
||||
});
|
||||
|
||||
it("folds the tenant markSessionAsFailed override into the imported default", () => {
|
||||
// An exported rule that (effectively) marks sessions failed must keep doing so
|
||||
// when re-created in another tenant from its JSON.
|
||||
expect(jsonToForm({ ...baseExport, markSessionAsFailed: true }).markSessionAsFailedDefault).toBe(true);
|
||||
expect(jsonToForm({ ...baseExport, markSessionAsFailedDefault: true }).markSessionAsFailedDefault).toBe(true);
|
||||
expect(jsonToForm({ ...baseExport, markSessionAsFailedDefault: true, markSessionAsFailed: false }).markSessionAsFailedDefault).toBe(false);
|
||||
expect(jsonToForm(baseExport).markSessionAsFailedDefault).toBe(false);
|
||||
});
|
||||
|
||||
it("round-trips evaluateOn through the form and back", () => {
|
||||
const evaluateOn = ["enrollment_end", "whiteglove_sealed", "on_event:vpn_error"];
|
||||
|
||||
expect(formToEvaluateOn(jsonToForm({ ...baseExport, evaluateOn }))).toEqual(evaluateOn);
|
||||
// Terminal-only stays clean of the field (undefined = backend default).
|
||||
expect(formToEvaluateOn(jsonToForm(baseExport))).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -1,7 +1,7 @@
|
||||
"use client";
|
||||
|
||||
import { useState } from "react";
|
||||
import { AnalyzeRule, RuleForm, getSeverityColor, getCategoryColor, interimTriggerLabels } from "../types";
|
||||
import { AnalyzeRule, RuleForm, PastedAnalyzeJson, getSeverityColor, getCategoryColor, interimTriggerLabels, jsonToForm } from "../types";
|
||||
import { FormJsonToggle, JsonModeToggleButtons, ReadOnlyJsonView } from "@/components/rules/FormJsonToggle";
|
||||
import AnalyzeRuleFormFields from "./AnalyzeRuleFormFields";
|
||||
import { stripInternalFields } from "@/lib/rulePageHelpers";
|
||||
@@ -635,9 +635,11 @@ export default function AnalyzeRuleCard({
|
||||
jsonError={jsonError}
|
||||
onApplyJson={() => {
|
||||
try {
|
||||
const parsed = JSON.parse(jsonText) as RuleForm;
|
||||
const parsed = JSON.parse(jsonText) as PastedAnalyzeJson;
|
||||
if (!parsed.title) throw new Error("JSON must include title");
|
||||
setEditForm({ ...editForm, ...parsed });
|
||||
// Merge keeps unmentioned fields; a rule-shaped paste brings an explicit
|
||||
// evaluateOn, which the mapper prefers over the stale eval* form fields.
|
||||
setEditForm(jsonToForm({ ...editForm, ...parsed }));
|
||||
onSetJsonModeEdit(false);
|
||||
onSetJsonError(null);
|
||||
} catch (e) {
|
||||
@@ -655,9 +657,9 @@ export default function AnalyzeRuleCard({
|
||||
<button onClick={() => {
|
||||
if (jsonModeEdit) {
|
||||
try {
|
||||
const parsed = JSON.parse(jsonText) as RuleForm;
|
||||
const parsed = JSON.parse(jsonText) as PastedAnalyzeJson;
|
||||
onSetJsonError(null);
|
||||
onSaveEdit(rule, { ...editForm, ...parsed });
|
||||
onSaveEdit(rule, jsonToForm({ ...editForm, ...parsed }));
|
||||
} catch (e) {
|
||||
onSetJsonError(e instanceof Error ? e.message : "Invalid JSON");
|
||||
}
|
||||
|
||||
@@ -16,7 +16,7 @@ import { GlobalAdminBanner, globalAdminSubtitle } from "@/components/GlobalAdmin
|
||||
import { TenantScopeSelector } from "@/components/TenantScopeSelector";
|
||||
|
||||
import { isTemplateRule } from "@/lib/analyzeRuleTabs";
|
||||
import { AnalyzeRule, RuleForm, EMPTY_FORM, EMPTY_CONDITION, ruleToForm, formToEvaluateOn, validateOnEventTypes } from "./types";
|
||||
import { AnalyzeRule, RuleForm, PastedAnalyzeJson, EMPTY_FORM, EMPTY_CONDITION, ruleToForm, jsonToForm, formToEvaluateOn, validateOnEventTypes } from "./types";
|
||||
import { BLOCKED_INTERIM_TRIGGER_EVENT_TYPES } from "@/utils/guardrails.generated";
|
||||
|
||||
/** Pre-flight for the evaluation-trigger form fields; returns an error message or null. */
|
||||
@@ -340,6 +340,8 @@ export default function AnalyzeRulesPage() {
|
||||
setCreating(true);
|
||||
const payload = {
|
||||
evaluateOn: formToEvaluateOn(form),
|
||||
tags: form.tags ?? [],
|
||||
markSessionAsFailedDefault: form.markSessionAsFailedDefault ?? false,
|
||||
ruleId: form.ruleId.trim(),
|
||||
title: form.title.trim(),
|
||||
description: form.description.trim(),
|
||||
@@ -398,6 +400,8 @@ export default function AnalyzeRulesPage() {
|
||||
const payload = {
|
||||
...rule,
|
||||
evaluateOn: formToEvaluateOn(form) ?? null,
|
||||
// From the form (not ...rule) so tag changes pasted in JSON mode survive the save.
|
||||
tags: form.tags ?? rule.tags ?? [],
|
||||
title: form.title.trim(),
|
||||
description: form.description.trim(),
|
||||
severity: form.severity,
|
||||
@@ -797,9 +801,11 @@ export default function AnalyzeRulesPage() {
|
||||
jsonError={jsonError}
|
||||
onApplyJson={() => {
|
||||
try {
|
||||
const parsed = JSON.parse(jsonText) as RuleForm;
|
||||
const parsed = JSON.parse(jsonText) as PastedAnalyzeJson;
|
||||
if (!parsed.ruleId && !parsed.title) throw new Error("JSON must include at least ruleId and title");
|
||||
setNewRule({ ...EMPTY_FORM, ...parsed });
|
||||
// Handles both shapes: rule-shaped exports (evaluateOn) and the
|
||||
// serialized form itself (eval* checkbox fields).
|
||||
setNewRule(jsonToForm(parsed));
|
||||
setJsonModeCreate(false);
|
||||
setJsonError(null);
|
||||
} catch (e) {
|
||||
@@ -816,9 +822,9 @@ export default function AnalyzeRulesPage() {
|
||||
<button onClick={() => {
|
||||
if (jsonModeCreate) {
|
||||
try {
|
||||
const parsed = JSON.parse(jsonText) as RuleForm;
|
||||
const parsed = JSON.parse(jsonText) as PastedAnalyzeJson;
|
||||
setJsonError(null);
|
||||
handleCreateRule({ ...EMPTY_FORM, ...parsed });
|
||||
handleCreateRule(jsonToForm(parsed));
|
||||
} catch (e) {
|
||||
setJsonError(e instanceof Error ? e.message : "Invalid JSON");
|
||||
}
|
||||
|
||||
@@ -123,6 +123,10 @@ export interface RuleForm {
|
||||
evalAtWhitegloveSealed: boolean;
|
||||
/** Comma-separated event types for on_event:<type> interim triggers. */
|
||||
evalOnEventTypes: string;
|
||||
/** Carried through create/save so exports and JSON pastes don't lose them (no form UI). */
|
||||
tags: string[];
|
||||
/** Effective "fire → mark session failed" carried as the new rule's default on import. */
|
||||
markSessionAsFailedDefault: boolean;
|
||||
}
|
||||
|
||||
export const CATEGORIES = ["network", "identity", "apps", "device", "esp", "enrollment"] as const;
|
||||
@@ -210,6 +214,8 @@ export const EMPTY_FORM: RuleForm = {
|
||||
evalAtEnrollmentEnd: true,
|
||||
evalAtWhitegloveSealed: false,
|
||||
evalOnEventTypes: "",
|
||||
tags: [],
|
||||
markSessionAsFailedDefault: false,
|
||||
};
|
||||
|
||||
export function ruleToForm(rule: AnalyzeRule): RuleForm {
|
||||
@@ -235,9 +241,52 @@ export function ruleToForm(rule: AnalyzeRule): RuleForm {
|
||||
.filter(t => t.toLowerCase().startsWith("on_event:"))
|
||||
.map(t => t.slice("on_event:".length))
|
||||
.join(", "),
|
||||
tags: [...(rule.tags ?? [])],
|
||||
// Fold the tenant override into the effective value: an exported rule that marks
|
||||
// sessions failed must keep doing so when re-created from its JSON.
|
||||
markSessionAsFailedDefault: rule.markSessionAsFailed ?? rule.markSessionAsFailedDefault ?? false,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* What JSON-mode paste can contain: the serialized form, a rule export, or a mix.
|
||||
* Shared fields take the rule's (nullable) typing so an AnalyzeRule assigns directly;
|
||||
* the form-only eval* fields come from RuleForm.
|
||||
*/
|
||||
export type PastedAnalyzeJson = Partial<AnalyzeRule> & Partial<Omit<RuleForm, keyof AnalyzeRule>>;
|
||||
|
||||
/**
|
||||
* Maps pasted JSON into a RuleForm, accepting BOTH shapes: the serialized form (has the
|
||||
* eval* checkbox fields) and a rule-shaped export (has evaluateOn, or neither when the
|
||||
* rule is terminal-only). Without the rule-shaped path, pasting an export silently
|
||||
* dropped evaluateOn — the rule quietly reverted to the enrollment-end default.
|
||||
*/
|
||||
export function jsonToForm(parsed: PastedAnalyzeJson): RuleForm {
|
||||
// An explicit evaluateOn is authoritative rule-shape; the form fields only decide when
|
||||
// it is absent (a terminal-only export carries neither, which the default handles).
|
||||
const formShaped =
|
||||
parsed.evaluateOn === undefined &&
|
||||
(parsed.evalAtEnrollmentEnd !== undefined ||
|
||||
parsed.evalAtWhitegloveSealed !== undefined ||
|
||||
parsed.evalOnEventTypes !== undefined);
|
||||
if (formShaped) return { ...EMPTY_FORM, ...(parsed as Partial<RuleForm>) };
|
||||
|
||||
return ruleToForm({
|
||||
...parsed,
|
||||
ruleId: parsed.ruleId ?? "",
|
||||
title: parsed.title ?? "",
|
||||
severity: parsed.severity ?? EMPTY_FORM.severity,
|
||||
category: parsed.category ?? EMPTY_FORM.category,
|
||||
trigger: parsed.trigger ?? EMPTY_FORM.trigger,
|
||||
baseConfidence: parsed.baseConfidence ?? EMPTY_FORM.baseConfidence,
|
||||
confidenceThreshold: parsed.confidenceThreshold ?? EMPTY_FORM.confidenceThreshold,
|
||||
conditions: Array.isArray(parsed.conditions) ? parsed.conditions : [],
|
||||
confidenceFactors: Array.isArray(parsed.confidenceFactors) ? parsed.confidenceFactors : [],
|
||||
remediation: Array.isArray(parsed.remediation) ? parsed.remediation : [],
|
||||
relatedDocs: Array.isArray(parsed.relatedDocs) ? parsed.relatedDocs : [],
|
||||
} as AnalyzeRule);
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the evaluateOn payload from the structured form fields. Returns undefined when the
|
||||
* selection equals the default (enrollment_end only) so unchanged rules stay clean of the field.
|
||||
|
||||
@@ -283,7 +283,13 @@ export default function InstallsTab({ scope, timeRange }: InstallsTabProps) {
|
||||
<th className="px-4 py-3 cursor-pointer select-none text-right" onClick={() => toggleSort("succeeded")}>Succeeded {sortIndicator("succeeded")}</th>
|
||||
<th className="px-4 py-3 cursor-pointer select-none text-right" onClick={() => toggleSort("failed")}>Failed {sortIndicator("failed")}</th>
|
||||
<th className="px-4 py-3 cursor-pointer select-none text-right" onClick={() => toggleSort("failureRate")}>Failure Rate {sortIndicator("failureRate")}</th>
|
||||
<th className="px-4 py-3 cursor-pointer select-none text-right" onClick={() => toggleSort("avgDurationSeconds")}>Avg Duration {sortIndicator("avgDurationSeconds")}</th>
|
||||
<th
|
||||
className="px-4 py-3 cursor-pointer select-none text-right"
|
||||
onClick={() => toggleSort("avgDurationSeconds")}
|
||||
title="Average duration of the final install attempt. IME evaluation passes and time queued behind other apps are not counted."
|
||||
>
|
||||
Avg Install Time {sortIndicator("avgDurationSeconds")}
|
||||
</th>
|
||||
<th className="px-4 py-3 cursor-pointer select-none text-right" onClick={() => toggleSort("trend")}>Trend {sortIndicator("trend")}</th>
|
||||
</tr>
|
||||
</thead>
|
||||
|
||||
@@ -142,6 +142,8 @@ interface SessionRow {
|
||||
attemptNumber: number;
|
||||
startedAt: string;
|
||||
durationSeconds: number;
|
||||
/** Observed install passes (2+ = IME evaluation pass + real install); 0 on legacy rows. */
|
||||
installPassCount?: number;
|
||||
}
|
||||
|
||||
// ── Affected Sessions: column metadata (for the column picker) ─────────────
|
||||
@@ -163,7 +165,7 @@ const SESSIONS_COLUMNS: SessionColumn[] = [
|
||||
{ key: "status", label: "Status", defaultVisible: true },
|
||||
{ key: "attempts", label: "Attempts", defaultVisible: true, numeric: true },
|
||||
{ key: "failureCode", label: "Failure Code", defaultVisible: true },
|
||||
{ key: "duration", label: "Duration", defaultVisible: true, numeric: true },
|
||||
{ key: "duration", label: "Install Time", defaultVisible: true, numeric: true },
|
||||
];
|
||||
|
||||
const SESSIONS_COLUMNS_STORAGE_KEY = "appHealthSessions_visibleColumns";
|
||||
@@ -544,12 +546,12 @@ function AppDetailContent() {
|
||||
}
|
||||
/>
|
||||
<SummaryCard
|
||||
label="Avg Duration"
|
||||
label="Avg Install Time"
|
||||
value={formatDuration(analytics.summary.avgDurationSeconds)}
|
||||
hint={
|
||||
analytics.summary.unmeasured > 0
|
||||
? `measured installs only — ${analytics.summary.unmeasured} without observed start`
|
||||
: "measured installs only"
|
||||
? `final attempt, measured installs only — ${analytics.summary.unmeasured} without observed start`
|
||||
: "final attempt, measured installs only"
|
||||
}
|
||||
/>
|
||||
<SummaryCard label="Trend" value={trendText(analytics.summary.trend, analytics.summary.trendDelta)} />
|
||||
@@ -588,18 +590,24 @@ function AppDetailContent() {
|
||||
<EmptyState />
|
||||
)}
|
||||
</Card>
|
||||
<Card title="Avg Install Duration over time">
|
||||
<Card title="Avg Install Time over time">
|
||||
{analytics.timeSeries.length > 0 ? (
|
||||
<AppLineChart
|
||||
data={analytics.timeSeries as unknown as Array<Record<string, unknown>>}
|
||||
xKey="bucketStart"
|
||||
series={[
|
||||
{ dataKey: "avgDurationSeconds", label: "Avg duration (s)", color: chartColors.primary },
|
||||
]}
|
||||
yUnit="s"
|
||||
yDomain={[0, "auto"]}
|
||||
formatXTick={formatBucketTick}
|
||||
/>
|
||||
<>
|
||||
<AppLineChart
|
||||
data={analytics.timeSeries as unknown as Array<Record<string, unknown>>}
|
||||
xKey="bucketStart"
|
||||
series={[
|
||||
{ dataKey: "avgDurationSeconds", label: "Avg install time (s)", color: chartColors.primary },
|
||||
]}
|
||||
yUnit="s"
|
||||
yDomain={[0, "auto"]}
|
||||
formatXTick={formatBucketTick}
|
||||
/>
|
||||
<p className="mt-2 text-xs text-gray-400">
|
||||
Measures the final install attempt since Aug 2026 — earlier points{" "}
|
||||
use the older span measurement and read higher.
|
||||
</p>
|
||||
</>
|
||||
) : (
|
||||
<EmptyState />
|
||||
)}
|
||||
@@ -644,7 +652,7 @@ function AppDetailContent() {
|
||||
</Card>
|
||||
)}
|
||||
{analytics.versionBreakdown.some((v) => v.measuredInstalls > 0) && (
|
||||
<Card title="Median Install Duration by Version">
|
||||
<Card title="Median Install Time by Version">
|
||||
<AppBarChart
|
||||
data={analytics.versionBreakdown.filter((v) => v.measuredInstalls > 0) as unknown as Array<Record<string, unknown>>}
|
||||
categoryKey="appVersion"
|
||||
@@ -952,6 +960,14 @@ function AppDetailContent() {
|
||||
return (
|
||||
<td key={col.key} className="px-3 py-2 text-right text-gray-700">
|
||||
{formatDuration(row.durationSeconds)}
|
||||
{(row.installPassCount ?? 0) > 1 && (
|
||||
<span
|
||||
className="ml-1 text-xs text-gray-400 dark:text-gray-500"
|
||||
title={`Processed in ${row.installPassCount} IME passes (evaluation + install) — install time counts only the final attempt`}
|
||||
>
|
||||
×{row.installPassCount}
|
||||
</span>
|
||||
)}
|
||||
</td>
|
||||
);
|
||||
default:
|
||||
|
||||
@@ -343,7 +343,7 @@ export function SessionTable({
|
||||
className="px-2 py-1.5 rounded-lg text-xs font-medium text-gray-600 bg-gray-100 hover:bg-gray-200 border border-gray-200 transition-colors cursor-pointer focus:outline-none focus:ring-2 focus:ring-green-500"
|
||||
title="Sessions per page"
|
||||
>
|
||||
{[10, 15, 20, 50, 100].map((n) => (
|
||||
{[5, 10, 15, 20, 50, 100].map((n) => (
|
||||
<option key={n} value={n}>{n} per page</option>
|
||||
))}
|
||||
</select>
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user