feat(agent): passive internet-bandwidth estimate from DO counters

New one-shot network_bandwidth_estimate event derived passively from the
Delivery Optimization byte counters the DeliveryOptimizationCollector
already polls every 3s - zero synthetic traffic, zero extra load, no
config change.

- BandwidthEstimator: per-FileId delta windows (a DO job that drops off
  the list and returns is averaged over its own absence window), p90/max
  reduction, WAN (HTTP + internet peers) vs LAN (LAN/group/link-local
  peers + Connected Cache) split so peer-fed downloads cannot inflate
  the WAN figure; bucket (<10/10-50/50-100/100-250/250+) + confidence.
- Emitted at most twice per session (snapshotTrigger): interim snapshot
  on first AccountSetup sighting (device_setup_end - survives sessions
  that starve in the account phase) and authoritative final at collector
  stop (collector_stop, lands before diagnostics_collecting).
- All DO jobs feed it (IME apps, Office CDN, WU/Store/Defender).
- Registered in Constants.EventTypes, MCP resource-catalog
  (network_events) and web gather-rules autocomplete.
- Drive-by: add missing agent_emergency_break to the MCP catalog
  (pre-existing drift, event-types-drift test was red).

13 new estimator unit tests; agent suite 1655/1655, MCP drift 6/6,
web tsc + eventTypes vitest green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Oliver Kieselbach
2026-07-09 11:25:30 +02:00
parent 8b266570cb
commit 076ea7e5fd
7 changed files with 631 additions and 0 deletions
@@ -0,0 +1,224 @@
using System;
using System.Collections.Generic;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry
{
/// <summary>
/// Pins the passive-bandwidth math that feeds the one-shot network_bandwidth_estimate
/// event: per-FileId baselining (first sighting counts nothing), per-file delta windows
/// (a job that vanishes from the DO list and returns must be averaged over its own
/// absence, not one poll), counter-reset and gap re-baselining, the WAN/LAN source
/// split, and the p90/bucket/confidence reduction.
/// </summary>
public class BandwidthEstimatorTests
{
private static readonly DateTime T0 = new DateTime(2026, 7, 9, 12, 0, 0, DateTimeKind.Utc);
private static List<BandwidthJobSample> Jobs(params BandwidthJobSample[] jobs)
=> new List<BandwidthJobSample>(jobs);
private static BandwidthJobSample Job(string fileId, long wan, long lan = 0)
=> new BandwidthJobSample { FileId = fileId, WanBytes = wan, LanBytes = lan };
[Fact]
public void FirstSighting_OnlyBaselines_NoSample()
{
var estimator = new BandwidthEstimator(3);
// 500 MB already on the counter at first sight — must NOT be booked as one interval.
estimator.AddSnapshot(T0, Jobs(Job("a", 500_000_000)));
Assert.Null(estimator.TryBuildEstimate());
}
[Fact]
public void SteadyDownload_ProducesCorrectMbps()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", 0)));
// 6 MB in 3 s = 6e6 * 8 / 3 / 1e6 = 16 Mbps
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", 6_000_000)));
var estimate = estimator.TryBuildEstimate();
Assert.NotNull(estimate);
Assert.Equal(1, estimate.WanSampleCount);
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3);
Assert.Equal(16.0, estimate.WanMbpsMax.Value, 3);
Assert.Equal(6_000_000, estimate.WanBytesObserved);
Assert.Equal("10-50", estimate.Bucket);
}
[Fact]
public void ParallelJobs_RatesAdd()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", 0), Job("b", 0)));
// 3 MB + 3 MB in 3 s = 8 + 8 = 16 Mbps combined
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", 3_000_000), Job("b", 3_000_000)));
var estimate = estimator.TryBuildEstimate();
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3);
}
[Fact]
public void JobVanishesAndReturns_WithinWindow_AveragedOverOwnAbsence()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", 0), Job("b", 0)));
// "a" missing from this poll; "b" moves 3 MB / 3 s = 8 Mbps.
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("b", 3_000_000)));
// "a" returns with 6 MB accumulated over ITS 6 s window → 8 Mbps, not 16.
// "b" adds another 3 MB / 3 s = 8 Mbps → combined sample 16 Mbps.
estimator.AddSnapshot(T0.AddSeconds(6), Jobs(Job("a", 6_000_000), Job("b", 6_000_000)));
var estimate = estimator.TryBuildEstimate();
Assert.Equal(2, estimate.WanSampleCount);
Assert.Equal(16.0, estimate.WanMbpsMax.Value, 3);
}
[Fact]
public void GapBeyondWindow_RebaselinesWithoutSample()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", 0)));
// 60 s dormancy gap (> 3 × 3 s): 100 MB accrued — must NOT become a rate sample.
estimator.AddSnapshot(T0.AddSeconds(60), Jobs(Job("a", 100_000_000)));
Assert.Null(estimator.TryBuildEstimate());
// But the re-baseline must hold: the next in-window delta samples normally.
estimator.AddSnapshot(T0.AddSeconds(63), Jobs(Job("a", 106_000_000)));
var estimate = estimator.TryBuildEstimate();
Assert.Equal(1, estimate.WanSampleCount);
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3);
}
[Fact]
public void CounterReset_RebaselinesWithoutSample()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", 50_000_000)));
// DO job restart: counter moved backwards — no (negative or bogus) sample.
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", 1_000_000)));
Assert.Null(estimator.TryBuildEstimate());
}
[Fact]
public void TinyMovement_BelowThreshold_NoSampleButNotNullWhenOthersExist()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", 0)));
// 100 KB < 256 KB threshold → no rate sample.
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", 100_000)));
Assert.Null(estimator.TryBuildEstimate());
// A real sample afterwards; the tiny delta still counted as observed bytes.
estimator.AddSnapshot(T0.AddSeconds(6), Jobs(Job("a", 6_100_000)));
var estimate = estimator.TryBuildEstimate();
Assert.Equal(1, estimate.WanSampleCount);
Assert.Equal(6_100_000, estimate.WanBytesObserved);
}
[Fact]
public void LanTraffic_DoesNotInflateWanEstimate()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", wan: 0, lan: 0)));
// 375 MB from LAN peers in 3 s (≈1 Gbps LAN) + 6 MB from the internet (16 Mbps).
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", wan: 6_000_000, lan: 375_000_000)));
var estimate = estimator.TryBuildEstimate();
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3);
Assert.Equal(1000.0, estimate.LanMbpsP90.Value, 3);
Assert.Equal("10-50", estimate.Bucket); // bucket keyed to WAN, not LAN
}
[Fact]
public void LanOnlySession_YieldsUnknownBucketAndNoWanFigure()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", wan: 0, lan: 0)));
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", wan: 0, lan: 375_000_000)));
var estimate = estimator.TryBuildEstimate();
Assert.NotNull(estimate);
Assert.Null(estimate.WanMbpsP90);
Assert.Equal(0, estimate.WanSampleCount);
Assert.Equal("unknown", estimate.Bucket);
Assert.Equal(1, estimate.LanSampleCount);
}
[Theory]
[InlineData(3_000_000, "<10")] // 8 Mbps
[InlineData(6_000_000, "10-50")] // 16 Mbps
[InlineData(30_000_000, "50-100")] // 80 Mbps
[InlineData(75_000_000, "100-250")] // 200 Mbps
[InlineData(150_000_000, "250+")] // 400 Mbps
public void Buckets_MapFromWanP90(long bytesPer3s, string expectedBucket)
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", 0)));
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", bytesPer3s)));
Assert.Equal(expectedBucket, estimator.TryBuildEstimate().Bucket);
}
[Fact]
public void P90_ClipsSingleBurstOutlier()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, Jobs(Job("a", 0)));
// 19 steady polls at 16 Mbps, then one DO accounting burst at ~160 Mbps.
long total = 0;
for (int i = 1; i <= 19; i++)
{
total += 6_000_000;
estimator.AddSnapshot(T0.AddSeconds(3 * i), Jobs(Job("a", total)));
}
total += 60_000_000;
estimator.AddSnapshot(T0.AddSeconds(60), Jobs(Job("a", total)));
var estimate = estimator.TryBuildEstimate();
Assert.Equal(20, estimate.WanSampleCount);
Assert.Equal(16.0, estimate.WanMbpsP90.Value, 3); // p90 stays on the steady plateau
Assert.Equal(160.0, estimate.WanMbpsMax.Value, 3);
}
[Fact]
public void Confidence_ScalesWithSamplesAndVolume()
{
// low: 1 sample / 6 MB
var low = new BandwidthEstimator(3);
low.AddSnapshot(T0, Jobs(Job("a", 0)));
low.AddSnapshot(T0.AddSeconds(3), Jobs(Job("a", 6_000_000)));
Assert.Equal("low", low.TryBuildEstimate().Confidence);
// medium: ≥3 samples and ≥25 MB
var medium = new BandwidthEstimator(3);
medium.AddSnapshot(T0, Jobs(Job("a", 0)));
for (int i = 1; i <= 3; i++)
medium.AddSnapshot(T0.AddSeconds(3 * i), Jobs(Job("a", i * 10_000_000L)));
Assert.Equal("medium", medium.TryBuildEstimate().Confidence);
// high: ≥10 samples and ≥150 MB
var high = new BandwidthEstimator(3);
high.AddSnapshot(T0, Jobs(Job("a", 0)));
for (int i = 1; i <= 10; i++)
high.AddSnapshot(T0.AddSeconds(3 * i), Jobs(Job("a", i * 20_000_000L)));
Assert.Equal("high", high.TryBuildEstimate().Confidence);
}
[Fact]
public void EmptyOrNullSnapshots_AreIgnored()
{
var estimator = new BandwidthEstimator(3);
estimator.AddSnapshot(T0, null);
estimator.AddSnapshot(T0, Jobs());
estimator.AddSnapshot(T0.AddSeconds(3), Jobs(Job(null, 5_000_000), Job("", 5_000_000)));
Assert.Null(estimator.TryBuildEstimate());
}
}
}
@@ -0,0 +1,230 @@
using System;
using System.Collections.Generic;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
{
/// <summary>
/// One DO job's cumulative byte counters at a single poll, split by source class.
/// WAN = BytesFromHttp + BytesFromInternetPeers (traffic that traversed the internet line).
/// LAN = LAN/group/link-local peers + Connected Cache (local sources — must NOT count
/// towards the internet-bandwidth estimate or a peer-fed branch office would report
/// hundreds of Mbps on a 10-Mbit line).
/// </summary>
public struct BandwidthJobSample
{
public string FileId;
public long WanBytes;
public long LanBytes;
}
/// <summary>Result of <see cref="BandwidthEstimator.TryBuildEstimate"/>.</summary>
public sealed class BandwidthEstimate
{
public double? WanMbpsP90 { get; set; }
public double? WanMbpsMax { get; set; }
public int WanSampleCount { get; set; }
public long WanBytesObserved { get; set; }
public double? LanMbpsP90 { get; set; }
public double? LanMbpsMax { get; set; }
public int LanSampleCount { get; set; }
public long LanBytesObserved { get; set; }
/// <summary>WAN p90 bucket: "&lt;10" | "10-50" | "50-100" | "100-250" | "250+" | "unknown".</summary>
public string Bucket { get; set; }
/// <summary>"high" | "medium" | "low" — driven by WAN sample count and observed volume.</summary>
public string Confidence { get; set; }
}
/// <summary>
/// Passive bandwidth estimator fed from the DO poll loop. Tracks per-FileId cumulative
/// counters, turns poll-to-poll deltas into Mbps rate samples, and reduces them to a
/// p90/max estimate at enrollment end. Pure arithmetic on data the collector already
/// polls — deliberately generates zero network traffic and zero measurable CPU load.
///
/// Delta windows are tracked PER FileId (not per snapshot): DO's job list churns, and a
/// job that drops out for a few polls and returns with accumulated bytes must have its
/// delta averaged over ITS OWN absence window — booking it into a single poll interval
/// would overshoot the rate by the number of missed polls.
///
/// Sampling rules (each keeps a distinct distortion out of the estimate):
/// - A FileId's first appearance only sets its baseline (its pre-existing bytes would
/// otherwise be booked into one interval and overshoot massively).
/// - A counter that moved backwards re-baselines that FileId (DO job restart).
/// - A per-file window outside [MinElapsedSeconds, MaxElapsedFactor × interval]
/// re-baselines that file without a rate contribution (dormancy, long list absence).
/// - Poll sums below MinSampleBytes produce no rate sample (idle/installing polls would
/// flood the distribution with near-zero noise) but still count as observed bytes.
/// Taking the p90 of the surviving samples approximates the sustained line capacity
/// from below: ramp-up and tail-end polls sit in the lower quantiles, and single-poll
/// accounting bursts above the p90 are clipped by not using the max as the headline.
/// </summary>
public sealed class BandwidthEstimator
{
private const long MinSampleBytes = 256 * 1024; // ignore polls with < 256 KB movement
private const double MinElapsedSeconds = 1.0;
private const int MaxElapsedFactor = 3; // per-file gap > 3× interval → re-baseline only
private const int MaxSamples = 4096; // safety cap (≈ 3.4 h at 3 s polls)
// Confidence thresholds (WAN): enough independent samples AND enough volume that
// TCP has demonstrably saturated the line at least part of the time.
private const int HighSampleCount = 10;
private const long HighBytes = 150L * 1024 * 1024; // 150 MB
private const int MediumSampleCount = 3;
private const long MediumBytes = 25L * 1024 * 1024; // 25 MB
private sealed class FileBaseline
{
public long WanBytes;
public long LanBytes;
public DateTime LastSeenUtc;
}
private readonly object _sync = new object();
private readonly double _maxElapsedSeconds;
private readonly Dictionary<string, FileBaseline> _baselines =
new Dictionary<string, FileBaseline>(StringComparer.OrdinalIgnoreCase);
private readonly List<double> _wanSamplesMbps = new List<double>();
private readonly List<double> _lanSamplesMbps = new List<double>();
private long _wanBytesObserved;
private long _lanBytesObserved;
public BandwidthEstimator(int pollIntervalSeconds)
{
if (pollIntervalSeconds <= 0) pollIntervalSeconds = 3;
_maxElapsedSeconds = pollIntervalSeconds * MaxElapsedFactor;
}
/// <summary>
/// Feeds one poll's job list. <paramref name="utcNow"/> is injected (not read from the
/// clock) so tests can drive synthetic timelines.
/// </summary>
public void AddSnapshot(DateTime utcNow, IReadOnlyList<BandwidthJobSample> jobs)
{
if (jobs == null || jobs.Count == 0) return;
lock (_sync)
{
// Per-poll aggregates. Rates add across parallel jobs; each job's rate is
// averaged over its own last-seen window so list churn cannot distort it.
double wanRateMbps = 0, lanRateMbps = 0;
long wanDelta = 0, lanDelta = 0;
for (int i = 0; i < jobs.Count; i++)
{
var job = jobs[i];
if (string.IsNullOrEmpty(job.FileId)) continue;
FileBaseline baseline;
if (!_baselines.TryGetValue(job.FileId, out baseline))
{
_baselines[job.FileId] = new FileBaseline
{
WanBytes = job.WanBytes,
LanBytes = job.LanBytes,
LastSeenUtc = utcNow
};
continue;
}
var elapsed = (utcNow - baseline.LastSeenUtc).TotalSeconds;
var fileWanDelta = job.WanBytes - baseline.WanBytes;
var fileLanDelta = job.LanBytes - baseline.LanBytes;
// Counter reset (job restart) or an implausible window → re-baseline only.
var valid = fileWanDelta >= 0 && fileLanDelta >= 0 &&
elapsed >= MinElapsedSeconds && elapsed <= _maxElapsedSeconds;
baseline.WanBytes = job.WanBytes;
baseline.LanBytes = job.LanBytes;
baseline.LastSeenUtc = utcNow;
if (!valid) continue;
wanDelta += fileWanDelta;
lanDelta += fileLanDelta;
wanRateMbps += fileWanDelta * 8.0 / elapsed / 1_000_000.0;
lanRateMbps += fileLanDelta * 8.0 / elapsed / 1_000_000.0;
}
_wanBytesObserved += wanDelta;
_lanBytesObserved += lanDelta;
if (wanDelta >= MinSampleBytes && _wanSamplesMbps.Count < MaxSamples)
_wanSamplesMbps.Add(wanRateMbps);
if (lanDelta >= MinSampleBytes && _lanSamplesMbps.Count < MaxSamples)
_lanSamplesMbps.Add(lanRateMbps);
}
}
/// <summary>Returns the estimate, or null when not a single valid rate sample exists.</summary>
public BandwidthEstimate TryBuildEstimate()
{
lock (_sync)
{
if (_wanSamplesMbps.Count == 0 && _lanSamplesMbps.Count == 0) return null;
var estimate = new BandwidthEstimate
{
WanSampleCount = _wanSamplesMbps.Count,
WanBytesObserved = _wanBytesObserved,
LanSampleCount = _lanSamplesMbps.Count,
LanBytesObserved = _lanBytesObserved,
};
if (_wanSamplesMbps.Count > 0)
{
estimate.WanMbpsP90 = Percentile90(_wanSamplesMbps);
estimate.WanMbpsMax = Max(_wanSamplesMbps);
}
if (_lanSamplesMbps.Count > 0)
{
estimate.LanMbpsP90 = Percentile90(_lanSamplesMbps);
estimate.LanMbpsMax = Max(_lanSamplesMbps);
}
estimate.Bucket = ToBucket(estimate.WanMbpsP90);
estimate.Confidence = ToConfidence(_wanSamplesMbps.Count, _wanBytesObserved);
return estimate;
}
}
private static string ToBucket(double? wanMbpsP90)
{
if (!wanMbpsP90.HasValue) return "unknown";
var v = wanMbpsP90.Value;
if (v < 10) return "<10";
if (v < 50) return "10-50";
if (v < 100) return "50-100";
if (v < 250) return "100-250";
return "250+";
}
private static string ToConfidence(int wanSamples, long wanBytes)
{
if (wanSamples >= HighSampleCount && wanBytes >= HighBytes) return "high";
if (wanSamples >= MediumSampleCount && wanBytes >= MediumBytes) return "medium";
return "low";
}
private static double Percentile90(List<double> samples)
{
var sorted = new List<double>(samples);
sorted.Sort();
var index = (int)Math.Ceiling(0.9 * sorted.Count) - 1;
if (index < 0) index = 0;
return sorted[index];
}
private static double Max(List<double> samples)
{
var max = double.MinValue;
for (int i = 0; i < samples.Count; i++)
if (samples[i] > max) max = samples[i];
return max;
}
}
}
@@ -73,6 +73,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
// Change detection for JSONL log: only write when overall state changed
private long _lastSnapshotFingerprint;
// Passive session-level bandwidth estimate: fed with EVERY DO job's byte counters
// (IME apps, Office CDN, WU/Store/Defender — all real traffic) on each poll; reduced
// to a network_bandwidth_estimate event. Emitted (at most) twice per session: an
// interim snapshot when the ESP leaves DeviceSetup (the bulk of Win32 downloads is
// done by then — keeps the estimate available for analysis even when the session
// later starves in AccountSetup and the agent never stops cleanly) and the
// authoritative final one when the collector stops at enrollment end.
// Zero extra traffic/load — pure arithmetic on the existing poll.
private readonly BandwidthEstimator _bandwidthEstimator;
private readonly int _pollIntervalSeconds;
private bool _bandwidthEstimateEmitted;
private int _interimBandwidthEmitted; // Interlocked guard — phase callback runs on the IME tracker thread
public DeliveryOptimizationCollector(
string sessionId,
string tenantId,
@@ -89,6 +102,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
_onDoTelemetryReceived = onDoTelemetryReceived;
_logFilePath = Path.Combine(logDirectory, LogFileName);
_onOfficeDoSample = onOfficeDoSample;
_pollIntervalSeconds = intervalSeconds;
_bandwidthEstimator = new BandwidthEstimator(intervalSeconds);
}
/// <summary>Start dormant — timer does not fire until WakeUp() is called.</summary>
@@ -101,6 +116,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
protected override void OnAfterStop()
{
// Emit before the runspace teardown: the termination handler stops peripheral
// collectors BEFORE the diagnostics ZIP is built, so this one-shot still lands
// ahead of diagnostics_collecting in the timeline.
EmitBandwidthEstimateOnce();
DisposeRunspace();
}
@@ -358,6 +377,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
// Build JSONL snapshot for log (only if fingerprint changes)
var logEntries = new JArray();
// Every DO job feeds the passive bandwidth estimate — including non-IME,
// non-Office jobs (WU, Store, Defender): all of it is real line traffic.
var bandwidthJobs = new List<BandwidthJobSample>(results.Count);
foreach (var result in results)
{
var fileId = GetPropString(result, "FileId");
@@ -378,6 +401,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
var sourceUrl = GetPropString(result, "SourceURL");
var cacheHost = GetPropUriString(result, "CacheHost");
bandwidthJobs.Add(new BandwidthJobSample
{
FileId = fileId,
WanBytes = bytesFromHttp + bytesInternetPeers,
LanBytes = bytesLanPeers + bytesGroupPeers + bytesLinkLocalPeers + bytesFromCacheServer
});
// Build log entry for JSONL
logEntries.Add(new JObject
{
@@ -503,6 +533,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
// Decrement the bounded registry-hint probe window only while no Office job is being seen.
if (!officeJobsThisPoll && _officeExpectedPolls > 0) _officeExpectedPolls--;
_bandwidthEstimator.AddSnapshot(DateTime.UtcNow, bandwidthJobs);
// Write JSONL log only when overall state changed
var fingerprint = ComputeFingerprint(logEntries);
if (fingerprint != _lastSnapshotFingerprint)
@@ -533,6 +565,115 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
return false;
}
// -----------------------------------------------------------------------
// Passive bandwidth estimate (one-shot at collector stop)
// -----------------------------------------------------------------------
/// <summary>
/// Interim snapshot trigger: the IME tracker reports an ESP phase. On the FIRST
/// AccountSetup sighting (= DeviceSetup, and with it the bulk of Win32 downloads, is
/// over) the current estimate is emitted with snapshotTrigger=device_setup_end — so
/// an analysis still has a bandwidth figure when the session later starves in
/// AccountSetup and the final collector-stop emission never happens. The tracker
/// invokes the callback on every phase RE-match too (IME re-logs the phase string
/// periodically), hence the Interlocked once-guard. Called from the tracker thread.
/// </summary>
public void NotifyEspPhaseChanged(string phase)
{
if (!"AccountSetup".Equals(phase, StringComparison.OrdinalIgnoreCase)) return;
if (Interlocked.CompareExchange(ref _interimBandwidthEmitted, 1, 0) != 0) return;
EmitBandwidthEstimate("device_setup_end");
}
/// <summary>
/// Emits the authoritative session-level network_bandwidth_estimate once, at collector
/// stop. Guarded against the double OnAfterStop from Stop() + Dispose().
/// </summary>
private void EmitBandwidthEstimateOnce()
{
if (_bandwidthEstimateEmitted) return;
_bandwidthEstimateEmitted = true;
EmitBandwidthEstimate("collector_stop");
}
/// <summary>
/// Builds and emits one network_bandwidth_estimate event. Silently skipped when no
/// valid rate sample exists yet (e.g. no DO-tracked downloads happened so far).
/// </summary>
private void EmitBandwidthEstimate(string trigger)
{
try
{
var estimate = _bandwidthEstimator.TryBuildEstimate();
if (estimate == null)
{
Logger.Info($"[DeliveryOptimizationCollector] No bandwidth estimate at {trigger} — no valid DO rate samples yet");
return;
}
var data = new Dictionary<string, object>
{
["wanSampleCount"] = estimate.WanSampleCount,
["wanBytesObserved"] = estimate.WanBytesObserved,
["lanSampleCount"] = estimate.LanSampleCount,
["lanBytesObserved"] = estimate.LanBytesObserved,
["bandwidthBucket"] = estimate.Bucket,
["confidence"] = estimate.Confidence,
["pollIntervalSeconds"] = _pollIntervalSeconds,
["snapshotTrigger"] = trigger,
["doSource"] = "os_cmdlet"
};
if (estimate.WanMbpsP90.HasValue)
{
data["estimatedWanMbps"] = Math.Round(estimate.WanMbpsP90.Value, 1);
data["wanMbpsMax"] = Math.Round(estimate.WanMbpsMax.Value, 1);
}
if (estimate.LanMbpsP90.HasValue)
{
data["estimatedLanMbps"] = Math.Round(estimate.LanMbpsP90.Value, 1);
data["lanMbpsMax"] = Math.Round(estimate.LanMbpsMax.Value, 1);
}
var interimSuffix = trigger == "device_setup_end" ? " [interim, after DeviceSetup]" : "";
string message;
if (estimate.WanMbpsP90.HasValue)
{
message = $"Estimated internet bandwidth ~{estimate.WanMbpsP90.Value:0.#} Mbit/s " +
$"(bucket {estimate.Bucket}, confidence {estimate.Confidence}; " +
$"{estimate.WanSampleCount} samples / {estimate.WanBytesObserved / (1024 * 1024)} MB via internet)" +
interimSuffix;
}
else
{
// All observed download traffic came from LAN peers / Connected Cache —
// the internet line was never exercised enough to estimate it.
message = $"No internet-path bandwidth estimate — downloads were LAN/cache-fed " +
$"(~{estimate.LanMbpsP90.Value:0.#} Mbit/s from LAN, " +
$"{estimate.LanBytesObserved / (1024 * 1024)} MB observed)" +
interimSuffix;
}
Post.Emit(new EnrollmentEvent
{
SessionId = SessionId,
TenantId = TenantId,
EventType = Constants.EventTypes.NetworkBandwidthEstimate,
Severity = EventSeverity.Info,
Source = "DeliveryOptimizationCollector",
Phase = EnrollmentPhase.Unknown,
Message = message,
Data = data,
ImmediateUpload = true
});
Logger.Info($"[DeliveryOptimizationCollector] Bandwidth estimate emitted: {message}");
}
catch (Exception ex)
{
Logger.Warning($"[DeliveryOptimizationCollector] Bandwidth estimate emission failed: {ex.Message}");
}
}
// -----------------------------------------------------------------------
// PSObject property helpers (safe extraction, no exceptions on missing props)
// -----------------------------------------------------------------------
@@ -20,6 +20,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
private readonly OfficeInstallDetectorHost? _officeHost;
private Action<AppPackageState, AppInstallationState, AppInstallationState>? _prevStateChanged;
private Action<AppPackageState, AppInstallationState, AppInstallationState>? _chainedHandler;
private Action<string>? _prevEspPhaseChanged;
private Action<string>? _chainedEspPhaseHandler;
private int _disposed;
public DeliveryOptimizationHost(
@@ -75,6 +77,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
};
_imeHost.Tracker.OnAppStateChanged = _chainedHandler;
// Chain into the ESP phase callback (same preserve/restore pattern): the collector
// emits an interim bandwidth snapshot on the first AccountSetup sighting, so the
// estimate survives sessions that later starve in the account phase and never
// reach the clean collector-stop emission.
_prevEspPhaseChanged = _imeHost.Tracker.OnEspPhaseChanged;
_chainedEspPhaseHandler = phase =>
{
try { _prevEspPhaseChanged?.Invoke(phase); }
catch (Exception ex) { _logger.Warning($"DeliveryOptimizationHost: previous OnEspPhaseChanged handler threw: {ex.Message}"); }
try { _collector.NotifyEspPhaseChanged(phase); }
catch (Exception ex) { _logger.Warning($"DeliveryOptimizationHost: NotifyEspPhaseChanged threw: {ex.Message}"); }
};
_imeHost.Tracker.OnEspPhaseChanged = _chainedEspPhaseHandler;
// Office C2R wake sources (no IME package keeps the collector polling otherwise):
// - the worker process up/down (keep-awake while a worker runs), and
// - the early registry hint (Scenario\INSTALL) so we probe for the Office-CDN job before
@@ -106,6 +123,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
{
_imeHost.Tracker.OnAppStateChanged = _prevStateChanged;
}
if (_chainedEspPhaseHandler != null
&& object.ReferenceEquals(_imeHost.Tracker.OnEspPhaseChanged, _chainedEspPhaseHandler))
{
_imeHost.Tracker.OnEspPhaseChanged = _prevEspPhaseChanged;
}
if (_officeHost != null)
{
try { _officeHost.ProcessWatcher.Started -= OnOfficeStarted; } catch { }
@@ -82,6 +82,7 @@ export const EVENT_TYPES_CATALOG = {
'dns_configuration',
'proxy_configuration',
'wifi_signal_info',
'network_bandwidth_estimate',
],
device_info_events: [
'os_info',
@@ -138,6 +139,7 @@ export const EVENT_TYPES_CATALOG = {
'system_reboot_detected',
'prior_run_died_with_state',
'previous_crash_detected',
'agent_emergency_break',
'performance_collector_stopped',
'agent_metrics_collector_stopped',
'keep_awake_engaged',
@@ -337,6 +337,16 @@ namespace AutopilotMonitor.Shared
public const string DoTelemetry = "do_telemetry";
public const string AllAppsCompleted = "all_apps_completed";
public const string AppTrackingSummary = "app_tracking_summary"; // Plan §5 Fix 4b — terminal per-session app summary
// Session-level internet-bandwidth estimate derived PASSIVELY from the DO byte counters
// the DeliveryOptimizationCollector already polls — no synthetic traffic, no extra load.
// At most twice per session (snapshotTrigger disambiguates): an interim snapshot at the
// first AccountSetup sighting (device_setup_end — survives sessions that starve in the
// account phase) and the authoritative final at collector stop (collector_stop).
// WAN (HTTP + internet peers) and LAN
// (LAN/group/link-local peers + Connected Cache) are reported separately so peer-fed
// downloads cannot inflate the WAN figure; p90 of per-poll rates ≈ lower bound of the
// effective line capacity during enrollment ("DSL 16 vs 250").
public const string NetworkBandwidthEstimate = "network_bandwidth_estimate";
// Microsoft 365 Apps (Office Click-to-Run) install lifecycle — emitted by the V2
// OfficeInstallDetector which watches the C2R client (registry + process) directly.
@@ -134,6 +134,8 @@ export const KNOWN_EVENT_TYPES: EventTypeEntry[] = [
description: "Certificate validation result." },
{ value: "network_state_change", label: "network_state_change", category: "diagnostics",
description: "Network state changed (online/offline/interface)." },
{ value: "network_bandwidth_estimate", label: "network_bandwidth_estimate", category: "diagnostics",
description: "Session-level internet-bandwidth estimate derived passively from the Delivery Optimization byte counters (no synthetic traffic). At most twice per session: an interim snapshot after DeviceSetup (snapshotTrigger=device_setup_end — survives account-phase starvation) and the authoritative final at enrollment end (collector_stop). p90/max Mbit/s for internet-path (HTTP + internet peers) and LAN sources (peers/Connected Cache) separately, plus bucket (<10 / 10-50 / 50-100 / 100-250 / 250+) and confidence." },
{ value: "vulnerability_report", label: "vulnerability_report", category: "diagnostics",
description: "Aggregated vulnerability scan finding." },
{ value: "software_inventory_analysis", label: "software_inventory_analysis", category: "diagnostics",