Commit Graph

1017 Commits

Author SHA1 Message Date
Oliver Kieselbach 44dd8161df fix(agent): ImeProcessWatcher verifies process identity before attaching and re-arms discovery after an exit — it attached to the first IntuneManagementExtension name match, so a standard user starting a renamed binary during AccountSetup could forge an ime_process_exited Warning and mute the real signal for the rest of the session (CWE-807), and even a legitimate IME service restart was never watched again; ImeProcessIdentity requires session 0 + image under the IME install root (QueryFullProcessImageName, cross-bitness), prefers the oldest trusted candidate, logs untrusted matches once per PID, and OnImeExited restarts the discovery timer (ReferenceEquals guard against duplicate events)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 17:07:18 +02:00
Oliver Kieselbach 74902bf185 fix(agent): IME per-line matching budget counts Regex.Match time only — the wall-clock window also held the match handlers and scheduler stalls, and session 946ccbd6 (Hyper-V guest, 5–10 s process-wide stalls, 1 MB log, no line over 5 KB) broke it twice on genuine lines and silently skipped their remaining patterns; ime_pattern_hits is Debug severity (operator/backend signal, hidden from the default timeline — the IME Tracker card and the ingest fold read it regardless); the session page's IME Tracker card moves to the right column under Hardware
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 16:00:28 +02:00
Oliver Kieselbach 7a3516e570 feat(observability): IME pattern-drift loop — nothing the log tracker skips or stops matching stays silent: the agent keeps cumulative tracker health counters and a per-pattern match histogram (restart-safe in ImeTrackerState), reports ime_* keys in agent_metrics_snapshot (files tailed, backlog bytes, lines read, matches, oversized/timeouts/budget breaks/held tails/unanchored), raises a one-shot immediate-upload ime_tracker_degraded Warning on the first pass that skipped work, and emits ime_pattern_hits at termination (every enabled pattern, zeros included, IME version); the backend folds the histogram into the permanent per-IME-version ImePatternStats table (built-in IDs only, plausible version only, one partition read + one transactional batch) and evaluates drift without a hard-coded list — baseline = the version with the most reporting sessions (≥100), expected = ≥80 % hit rate there, drift = an expected pattern matched in none of ≥25 sessions on a newer version — raising ImePatternDriftSuspected once per version×pattern (ETag-stamped, dual-registered in the web ops-event list with a new guard test); GET metrics/ime-pattern-health (GlobalReadOrAdmin) + Global Admin → Metrics → IME Pattern Health matrix (drift/silent/low/few cells, open alerts, retired patterns) + session-page "IME Tracker" rows (latest snapshot by max sequence) + MCP get_ime_pattern_health (GA/Reader)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 15:12:38 +02:00
Oliver Kieselbach f4a5eac539 chore(rules): IME pattern-pack hygiene against the IME decompiles (1.97.107.0 / 1.104.102.0 / 1.105.103.0) — no wording drift in any shipped pattern; the 11 disabled legacy patterns whose literals last existed in IME 1.50 (fleet minimum 1.97) are deleted instead of left as re-enable traps (IME-DETECTED-OLD, IME-SET-CURRENT-1/2/3, IME-SKIP-APPLICABILITY-NOTMET, IME-SKIP-DEPENDENCY-DETECT, IME-SKIP-DETECTED, IME-SKIP-UNINSTALL-NOTDETECTED-2, IME-UPDATE-NAME-1/2/3); IME-DO-TIMEOUT-1/2 match the exact source literal "DO downloading is not finished within timeout, jobId = …, fileId = …" instead of two lookaheads; IME-DO-TEL documented as emitter-less from 1.104 on and kept only for the 1.101/1.103 fleet share
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 14:23:14 +02:00
Oliver Kieselbach 09492e908c fix(security): IME log-pattern pack is anchored and linear (all 90 patterns start with ^, IME-APP-VERSION spells the report-delta JSON out instead of a chained [^}]*? scan) and the ImeLogTracker read loop is bounded — BoundedLineReader (32 MB per line, evidence: real 5.6 MB "Get policies" lines; exact byte bookmarks, no decoder read-ahead loss on cancel), 2 s per-line matching budget across all patterns, held-back unterminated EOF tails (≤1 MB, 1 s settle) instead of dropping them and raw-matching fragments, a complete entry after a dropped one is no longer swallowed as its close, one Warning per pass (oversizedLines/regexTimeouts/lineBudgetBreaks) instead of silent Debug timeouts; unanchored PS-SCRIPT-CONTEXT false positives on [Win32App] SideCarScript… lines (which overwrote the pending platform script's RunContext) are gone; --run-ime-matching assembles multiline entries like the tracker
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 14:03:50 +02:00
Oliver Kieselbach f432353efa fix(security): startup self-update no longer trusts the on-disk remote-config.json cache — Program.LoadCachedSelfUpdateContext (raw deserialize, bypassed the read-side strip) is removed, the boot-time check runs with allowDowngrade:false and the version-v2.json manifest hash as its only integrity gate; LatestAgentSha256 (update-ZIP hash, ranked above the manifest hash in SelfUpdater) joins the CacheConfig/LoadCachedConfig strip lists so neither the startup nor the FromCache runtime path can take a planted ZIP hash or downgrade flag; verify+extract already share one exclusive handle (1daf631a)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 13:53:00 +02:00
Oliver Kieselbach 1daf631a02 fix(security): self-update verifies and extracts the ZIP through one exclusive FileShare.None handle — SHA-256 is computed on the same stream the ZipArchive reads, so the file can no longer be swapped between verification and extraction (TOCTOU); handle is released before cleanup, open failure aborts with integrity_mismatch marker
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 13:03:50 +02:00
Oliver Kieselbach 66f5543fb7 fix(security): ConsoleBypassWatcher mirrors cmd.exe first-switch-wins parsing — only a leading /c classifies as scripted; a /c inside a /k payload (cmd /k rem /c, cmd /k "title /c") no longer silences a persistent interactive console, which is surfaced as interactive_console_with_command (low)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 02:05:55 +02:00
Oliver Kieselbach 84d06e1cf6 fix(security): LocalAdminAnalyzer never exempts a logged-in local SAM account from the shutdown scan (CWE-807) and matches built-ins by well-known RID so localized Guest/Administrator are not flagged
The shutdown run allowed every logged-in user (explorer.exe owner) dynamically, so a backdoor account created after the startup scan and signed in before enrollment completed was scored 0. The WMI inventory is now read first and a logged-in name that is a local account is rejected for the account and the profile check (reported as logged_in_local_accounts); only Entra/domain identities — the only ones seen in real sessions — remain dynamically allowed for their profile folder.

Built-ins are recognised by RID 500/501/503/504 on the machine SID in addition to their English names: since disabled accounts are inventoried (88dfd331), localized Gast/Invité/Administrateur would otherwise flag every non-English device. kioskUser0 (Assigned Access autologon) joins the built-in list; the profile check skips junctions and $<32 hex> folders (cannot be an account name, pure payload noise). ANALYZE-ID-002 1.3.0 explanation mentions the new field.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 01:42:51 +02:00
Oliver Kieselbach 88dfd331dd fix(security): LocalAdminAnalyzer inventories disabled accounts and reads Administrators-group membership from the SAM (NetLocalGroupGetMembers on S-1-5-32-544); a dormant /active:no admin backdoor now surfaces as unexpected_admin_members (+40, Error), built-in Administrator enabled-state reported for evidence
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 01:00:29 +02:00
Oliver Kieselbach d294415d29 fix(security): CmTraceLogParser parses in linear time (first open / last parseable trailer, timeout-backed trailer regex) instead of a greedy .* regex that backtracked quadratically on hostile lines without a timeout; agent multiline buffer gains a 1 MB char cap next to the 100-line cap, both logged at Warning, and the tail of a dropped entry is skipped instead of raw-matched
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 00:48:29 +02:00
Oliver Kieselbach a47e94f9a5 fix(security): diagnostics packager reads every file through a handle validated against its pinned source folder; a subdirectory swapped for a junction between enumeration and open is skipped instead of copied
The path guards, the enumeration's reparse-point skip and the per-file attribute check all judged path strings, while the FileStream feeding the ZIP opened that string later (after earlier files were compressed). Under any recursive source folder a local user can write to, a subdirectory turned into a junction in that window made SYSTEM copy the junction target into the tenant-uploaded package.

PinnedSourceFolder holds each section folder open for the whole section (final path must equal the validated path, 8.3-expanded; FILE_LIST_DIRECTORY without delete share, so NTFS refuses renaming it or any ancestor) and opens each candidate with FILE_FLAG_OPEN_REPARSE_POINT, requiring the handle's final path to be exactly <canonical folder>\<relative path>; the archive is fed from that same handle. Manifest: FOLDER REJECTED (...) / SKIPPED (resolved outside validated folder) / FAILED (open). The race is pinned by a test that swaps the directory from the BeforeSourceFileOpen seam; on the previous code the same test copies the junction target.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-30 00:06:50 +02:00
Oliver Kieselbach 6e3487756a fix(security): gather-rule on_event feedback loop — timeline feed carries the event source and never dispatches on_event for gather-emitted events; per-rule on_event cap (100/session); backend rejects rules triggering on their own output type / gather_result
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 23:54:44 +02:00
Oliver Kieselbach c41c2327ab fix(security): validate enrollment summary branding image URL (https-only, no quotes); launcher drops unsafe URLs instead of interpolating them into the dialog command line
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 23:15:07 +02:00
Oliver Kieselbach 1733f4fd48 fix(agent): SignalIngress worker re-entrancy — never block the single consumer on its own post
EffectRunner posts ClassifierVerdictIssued from inside DecisionStepProcessor.ApplyStep,
i.e. on the SignalIngress worker thread. With the bounded channel (256) full, Post's
back-pressure slow path blocked the channel's only consumer as a producer: nothing could
ever drain, every collector Post blocked behind it, and the whole decision/monitoring
pipeline froze silently until the max-lifetime watchdog.

Post now detects a call from the worker thread itself and routes the item to a
worker-local follow-up queue that WorkerLoop drains (iteratively, ahead of the channel)
right after the current item. Same processing path — ordinals, SignalLog, emit,
reduce, ApplyStep — just never blocking. Worker posts are accepted during the Stop
drain too, since the verdict belongs to the step being finished. ReentrantPostCount
exposed for observability/tests.

Tests: full channel + worker post releases a blocked foreign producer and processes the
verdict before the next channel item (fails against the previous code by hanging);
follow-up chain drained in order without recursion; verdict posted during Stop drain is
not lost. Docs: agent/decision-engine.md, agent/overview.md, log.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 22:11:25 +02:00
Oliver Kieselbach fc46b4c78e feat(security): session-owner binding stage 1 (shadow) + dormant agent session rotation
Agent endpoints authenticated per tenant but authorized session writes on a caller-supplied
session id. Sessions rows now carry the creating device identity (cert thumbprint + Intune
device id from the CN, or bootstrap code + serial) in primary-only Owner* columns.
SessionOwnershipPolicy evaluates register / telemetry / error-report requests against it;
SessionOwnerBindingObserver records the outcome on the request row, as a warning and as a
throttled SessionOwnerMismatch ops event. Nothing is rejected in this stage.

Agent: errorCode parsing on 401/403, one-shot session rotation on session_owner_mismatch
without feeding the auth-failure tracker, spool drop, agent_started.sessionRotated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 21:32:24 +02:00
Oliver Kieselbach b3bbcf1300 fix(agent): rotate_config applies the fetched config; diagnostics skip gates report their reason
rotate_config only stored the fetched config in RemoteConfigService and never
merged it into the running AgentConfiguration, so the portal quick-config flow
(enable diagnostics upload, then collect) ran the diagnostics gate against the
stale mode=Off and failed with a generic diagnostics_upload_failed 4 ms later.
The action now runs the same ApplyFetchedConfig path as the startup fetch and
reports config_fetch_failed when the live fetch fell back to cache/defaults.
Skipped uploads return a DiagnosticsUploadResult.Skipped with the gate name
(diagnostics_skipped_mode_off / _not_configured / _onfailure_success) instead
of null, so server_action_failed names the actual reason.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 19:30:00 +02:00
Oliver Kieselbach 0f9a77c848 feat(agent,web,mcp): surface the 24H2 location gate instead of only logging it
A missing WiFi signal reading is indistinguishable from a bug unless the payload
says why, and the device-level location setting is not something an admin would
think to check. Since 058248d9 the SSID survives the gate via the WinRT tier —
but signal, radio and channel silently vanish with no explanation anywhere.

When the WinRT tier wins AND the native tier reported rc=5, wifi_signal_info now
carries wifiDataLimitedReason = "location_services_off" and the event message
gains "— signal unavailable, Location services off", so the reason reads straight
off the timeline. The portal's WiFi card matches that literal and names the
Intune setting that restores the full payload.

The claim is evidence-only: any other native failure (no WLAN service, no
connected interface) leaves the field null rather than blaming a privacy setting
we did not observe. IsLocationGateDenied is the single predicate, pinned at its
boundaries (rc=5 yes; rc=50, rc=5023, rc=1062 no), and the literal is pinned as
the published agent/portal/MCP contract.

New wifi_signal_info group in the MCP DEVICE_PROPERTIES_CATALOG, so a fleet can
be sized before anyone changes a policy:
search_sessions(deviceProperties: {
  "wifi_signal_info.wifiDataLimitedReason": "location_services_off" })

Backend is untouched — the event payload is opaque DataJson with no projection.
network_state_change and the network timeline deliberately do not carry the
reason yet; that is a second payload plus web change for no extra insight.

Verified: build 0 warnings, agent suite 2553/2553, web tsc + eslint (0 warnings)
+ vitest 889/889, MCP tsc.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 13:58:31 +02:00
Oliver Kieselbach 058248d95c fix(agent): read WiFi SSID via ungated WinRT API — 24H2 location gate
wifi_signal_info was missing on most Windows 11 24H2+ WiFi enrollments
(build 22631 3/3, 26100 37/178, 26200 43/95 over n=276 sessions since 08-01).
Not a parser or language problem: every event that exists carries a full SSID,
so the failure is binary at the reader level.

Since 24H2 Windows gates WlanQueryInterface with
wlan_intf_opcode_current_connection — the only opcode we use — behind
precise-location consent and returns ERROR_ACCESS_DENIED. The consent prompt
only fires for a process in the user's context outside System32, so the agent
(SYSTEM, session 0, before the OOBE privacy page) is denied silently, and it
resolves consent from the HKLM ConsentStore because the S-1-5-18 hive has none
— while the interactive user on the same machine reads the SSID fine via the
per-user NonPackaged value. That is why it tracks the image's location setting,
not hardware, driver or language.

netsh wlan hangs off the same gate, so the 2026-07 netsh fallback and its
multi-language labels never covered the case they were built for; more labels
cannot fix it.

- New WinRtWifiSsidReader: Microsoft's documented replacement
  WlanConnectionProfileDetails.GetConnectedSsid (ungated, SSID only), bound via
  the CLR WinRT projection — no winmd, no NuGet, no csproj change — the same
  mechanism OobeStateReader already uses from the SYSTEM context.
- New WifiInfoProvider owns the tier order (native -> WinRT -> netsh) that both
  collectors had duplicated.
- The step diagnostics added in 2026-07 were logged at Debug while the default
  level is Info, so they never reached a field log. The all-tiers-failed case is
  now a Warning that decodes rc=5 into the location-gate explanation.

Signal, PHY and channel have no ungated source and stay null on tier 2;
deriving a percentage from GetSignalBars would fabricate precision.

Verified: build 0 warnings, agent suite 2542/2542, interop struct sizes measured
live (532/604), and the full WinRT chain resolves and returns the SSID under
net48.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 13:27:36 +02:00
Oliver Kieselbach c91d8d4501 fix(agent): complete enrollments with zero user-targeted apps instead of failing them
IME never writes its "Completed user session" line when the user has no
assigned Win32 apps - its zero-app branch returns before that statement
(ApplicationPoller.cs:713-721, identical in 1.80/1.83/1.97/1.104). So
ImeUserSessionCompletedUtc stayed null, every arm of
ShouldTransitionToAwaitingHello was unsatisfiable, and the 30-min
advisory_completion backstop resolved healthy enrollments as
esp_exit_without_completion_evidence. One tenant: 10 of 30 sessions in a week,
all false positives - DAD-validated desktop, Hello provisioned, RealmJoin still
installing. Every Succeeded session there carries ime_user_session_completed;
not one of the failed ones does.

IME-USER-SESSION-ZERO-APPS reads the line IME does write there and routes it
through the same OnUserSessionCompleted callback. That is IME's own verdict
rather than a weakened gate: the zero-app branch locks the ESP in with an empty
application list, and the sibling emit site returns ProvisioningComplete right
after logging it. Guarded to the AccountSetup phase so it cannot burn the
adapter fire-once flag on a pre-sign-in timestamp, and replayed on the phase
transition when the observation predates the phase marker. AreUserEspAppsSettled
accepts an empty app list only with that explicit observation, never on
emptiness alone.

Second, independent defect: advisory_completion (30 min) overran the RealmJoin
completion gate, which deliberately holds the session and carries its own
bounded resolution (60 min from detection, activity-extended, 4 h cap). One
session installed 25 RealmJoin packages inside the window, the last 2 min before
the failure, while completion_waiting still read "waiting on:
realmjoin_resolution". The handler now re-arms while that gate is closed and its
own deadline is still armed; convergence comes from the gate itself, which
always ends with an Outcome set.

ime_user_session_completed gains an evidence field (zero_user_apps_enumerated /
user_session_completed) - no new event type, so the Constants ->
resource-catalog -> shared-manifests triple stays untouched. The pattern ships
via the config API and reaches existing agents, where an unknown action is a
single Debug line; the behaviour needs the agent release.

Build 0 warnings. DecisionCore 558/558, V2.Core 2530/2530, Backend 4634/4634,
MCP event-types-drift 6/6.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 12:27:42 +02:00
Oliver Kieselbach 249e39bc68 feat(security): cert tenant binding, stage 1 shadow mode
A chain-valid Intune MDM certificate only proves "issued to some tenant" —
the pinned roots are shared by every Intune tenant. Tenant scoping rested
entirely on the device serial in the downstream validators, which is not a
secret. Bind the tenant stamped into the auth cert to the requested tenant.

The Entra tenant id sits in OID 1.2.840.113556.5.14 of the cert the agent
already presents for mTLS, so this needs no Graph call, no consent and no
agent release. Not 5.6 (Intune Account id, a different GUID) and not 5.4
(device id, stored as 16 bare bytes) — the decoder handles both encodings
and now lives once in Shared, used by agent and backend.

Stage 1 observes only: ObserveCertTenantBinding returns void and swallows
its own exceptions, so it cannot fail an otherwise valid request. Match is
logged only on cert-cache miss, keeping the denominator without per-request
volume. Grep marker for stage 2: CERT-TENANT-BINDING-SHADOW.

Backend 4604/4604, Agent V2.Core 2519/2519, 0 warnings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:03:05 +02:00
Oliver Kieselbach 6bd9139869 fix(agent): cert GUID parser accepts bare 16-byte extension values
Intune's MDM device cert stores the DeviceId in OID 1.2.840.113556.5.4 as a
bare 16-byte GUID with no ASN.1 wrapper, so the strict OCTET-STRING parser
bailed on the tag check and returned null. Verified against a live Intune
device cert: 5.4 is 16 raw bytes, while 5.6 / 5.11 / 5.14 carry the usual
04 10 wrapper.

- ParseGuidFromAsn1OctetString -> ParseGuidFromExtension (the old name no
  longer describes what it parses); both encodings now handled, with the
  shared LE-GUID construction pulled into ToGuid(). The forms cannot
  collide: a wrapped 16-byte GUID is always >= 18 bytes.
- Document the Intune MDM cert OID arc on the class: 5.4 DeviceId,
  5.6 Intune AccountId (NOT the Entra tenant), 5.14 Entra TenantId,
  5.11 undocumented GUID, 5.15/.16/.17/.18/.19/.23 INTEGER enums.
- 13 tests pinning both encodings plus the malformed-input rejections,
  including a bare GUID whose first byte is the 0x04 OCTET STRING tag.

No behaviour change for the MS-Organization-Access probe the agent uses;
TryGetTenantIdFromCert is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:41:00 +02:00
Oliver Kieselbach 73373a2c9e feat(agent): compiled-in fallback rule catalogs for the no-config default
The built-in default config shipped empty ImeLogPatterns/GatherRules lists, so a
session that got neither a live fetch nor a cached config ran with the entire IME
tracking pipeline dead — no app installs, no script results, no health-script
compliance, no IME ESP-phase signals — although the agent binary carries all 37
action handlers. The scalar knobs never drift (shared AgentConfigResponse property
initializers are also the backend's fallbacks), but the catalogs cannot come from
initializers.

BuiltInConfigCatalog embeds the SAME CI-combined rules/dist artifacts the backend
embeds and serves (ime-log-patterns.json, gather-rules.json) and feeds the enabled
entries into CreateDefaultConfig. Single source by construction: each agent release
snapshots the catalog state of its own build, so the offline fallback can never
drift further than the running agent version itself. A live fetch (startup,
post-registration recovery, rotate_config) always wins. Fail-soft: a load error
degrades to the previous empty-list behaviour.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 12:14:30 +02:00
Oliver Kieselbach 6d607c9045 fix(agent): network-link gate before initial config fetch + post-registration recovery
Root cause (tenant aebdce78 audit 2026-08-24, session ba828038): on boot-time
relaunches the agent starts before the Wi-Fi link is up. The Phase 4 config fetch
had no network gate (98f8114d only guarded Phase 6 registration), burned its
10/30/60s retry budget into a dead link and stranded the whole session on built-in
defaults (ConfigVersion=0) — up to ~50% of that tenant's sessions, with no
self-heal short of a manual rotate_config action. Not a version regression: rates
varied wildly per day within the same agent build; the failed requests never
reached the backend.

Two complementary layers:
- RemoteConfigService gates the initial fetch on the shared NetworkLinkWait poll
  (60s cap — longer than registration's 15s because the fetch runs first), so the
  retry budget starts on a live link. rotate_config stays ungated (latency-bound).
- Phase 6.5: after registration proves the backend reachable, TryRecoverConfigAsync
  refetches once and AgentRuntimeConfig.TryRecoverAfterRegistration re-applies the
  live config (same merge/integrity pipeline as Phase 4). LastFetch* keeps the
  original failure record; remote_config_fetch_failed reports it with a recovered
  flag at Info severity, agent_started flags remoteConfigRecovered.

Deliberately not re-run in recovery: kill-signal check (telemetry-channel control
signals cover post-registration kills) and endpoint migration (next start).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 11:53:04 +02:00
Oliver Kieselbach 8e8bb66371 test(agent): thread-safe FakeSignalIngressSink (parked-tripwire CI flake)
Timer-thread Post() appended to a bare List while the test polled it in a
SpinWait foreach -> 'Collection was modified' (run 32644425162, no assert
text). Posted now returns a locked snapshot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 16:28:58 +02:00
Oliver Kieselbach 98f8114d90 fix(classification): self-deploying profiles reconcile to Succeeded after Device Setup, never AwaitingUser
Kiosk tenant audit 2026-08-23: 836 of ~970 self-deploying sessions in one week
were parked AwaitingUser ("Account Setup 0/5") and graduated to Incomplete 51h
later although the devices were finished — the profile has no user phase, so the
classifier's rule 5 was asking for evidence that can never arrive.

- EnrollmentTimeoutClassifier: new rule 1c IsSelfDeployingProvisioned (profile flag +
  Device Setup all-succeeded + no explicit failure) => Succeeded reconcile with an
  honest reason; isSelfDeployingProfile passed by every caller (sweep stage 1+2,
  max-lifetime verdict, late-telemetry heal, legacy retro mode).
- MaintenanceService stage 1: TryReconcileSelfDeployingAsync gate before the Stalled
  marker (mirrors the WhiteGlove Part-2 gate), audit counter SessionsSelfDeployingReconciled.
- LegacyReclassificationService: new admin mode self_deploying_silent (heal-only,
  dry-run by default) + GetSelfDeployingSilentSessionsAsync for the existing backlog.
- Agent SessionRegistrationHelper: wait up to 15s for a network link and retry 6x
  (~62s) instead of 5x (~30s) so the BootTrigger relaunch on Wi-Fi no longer exits 7
  before the link is back (the mechanism that left these sessions silent).
- Tests: 6 classifier cases (gate, within/past grace, user-driven guard, failure,
  pre-Device-Setup), 2 registration tests; docs/backend/silent-session-classification.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 09:55:06 +02:00
Oliver Kieselbach 93de130cd1 feat(diagnostics): built-in collection catalog visible in the portal, RealmJoin + Device Preparation sections gated
The diagnostics package's built-in collection is now ONE catalog in Shared
(DiagnosticsBuiltInSections.All): the agent iterates it when building the ZIP and
the backend serves the same list to the portal (GET /api/diagnostics/paths,
MemberRead), so what administrators see is exactly what the agent collects.

Agent
- New conditional sections: RealmJoin client/package/Chocolatey/per-user logs
  behind the tenant's RealmJoin Watcher toggle (mirrored onto
  AgentConfiguration.EnableRealmJoinWatcher by RemoteConfigMerger) and the Device
  Preparation bootstrapper event log behind the deterministic WDP marker (probe
  once per build, fail toward Classic). ZIP layout mirrors the disk
  (RealmJoinLogs/Windows|Choco|User). Every skip is a package-manifest line.
- Built-ins bypass DiagnosticsPathGuards by design (reviewed code); caps,
  reparse-point skip and evtx export still apply. No ConfigVersion bump.

Backend
- GET diagnostics/paths (MemberRead, JWT-scoped) with the catalog + global paths.
- MergeDiagnosticsLogPaths: global ∪ tenant dedup (trimmed, case-insensitive) —
  duplicates used to produce duplicate ZIP entry names.

Web
- Collapsed read-only built-in list (RealmJoin on/off in the tenant header,
  neutral in the Global-Admin view), one-line DiagnosticsPathRow for every entry,
  add-row above the lists; tenant context no longer fetches /api/global/config.

Tests: agent RealmJoin on/off, no user session, bootstrapper evtx, probe failure,
merger mirror; backend catalog invariants, payload shape, dedup, policy; web
builtInSectionDisplay. Docs: docs/agent/diagnostics-package.md (+index/log).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 04:09:50 +02:00
Oliver Kieselbach 27fdd19568 feat(agent): optional Delivery Optimization GroupId from network fingerprint
EnableDoGroupIdAutoSet (ConfigVersion 40, default off): the agent seeds the
DOGroupId policy value with a deterministic GUID derived from the default
gateway's IPv4 + SendARP MAC, RealmJoin-compatible byte layout so mixed
fleets peer together. Foreign DOGroupId/DOGroupIdSource policies always win
(ownership tracked via StartupEventGate fingerprint); no random-GUID
fallback. New do_group_id_auto_set event, portal toggle, admin report row,
manifest/catalog sync, OKF doc. Gateway heuristic deduplicated into
NetworkInterfaceLocator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 20:09:45 +02:00
Oliver Kieselbach 06cf1c4c12 feat(agent): system timeline watcher — clock steps + sleep episodes as ground truth
Two enrollment blind spots become timeline ground truth from the System event
log via one EventLogWatcher (WindowsUpdateTracker pattern, 24h backfill with
RecordId watermark recovering pre-agent records):

- system_clock_changed (Kernel-General 1): oldTime/newTime/signed timeDeltaMs,
  reason and the setting process. 2s floor suppresses w32time 1ms micro-slews;
  steps >= 5min are Warning + immediate upload. Payload instants are the
  authoritative source (event timestamps can be clamped server-side).
- system_sleep_episode (Power-Troubleshooter 1 classic sleep/hibernate,
  Kernel-Power 507 Modern Standby; 506 deliberately unconsumed): kind,
  enteredAt/exitedAt, durationSeconds, wake source / AC state. 60s real-sleep
  floor filters Modern Standby screen-off scenarios.

ConfigVersion 39 (Collectors.EnableSystemTimelineWatcher +
SystemEventBackfillLookbackMinutes). Both types are non-activity on agent and
backend (idle/stall clocks unaffected; a sleeping device stays Stalled).

Consumers: TimeJumpBadge gains the ground-truth cause "clock-set"
(classifyTimeJump matches backward display steps against recorded clock sets);
EventTimeline badge blocks for both events plus the previously bare
power_state_change; SessionInfoCard keeps the wall-clock duration and adds a
standby note; TimeAttributionCalculator builds SleepSpans (RebootSpan clone,
AttributionVersion 3) persisted as SleepSeconds/SleepSpansJson and rendered in
TimeAttributionLane; ANALYZE-DEV-012 (info, single) fires once per session for
episodes >= 5min - deliberately the only new rule, clock findings stay with
DEV-008 to avoid duplicate-finding flood.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 19:41:56 +02:00
Oliver Kieselbach ba2a390a83 fix(agent): battery-instance fallback for the dead Win32_PowerManagementEvent provider
On Windows 11 24H2/25H2 (observed on builds 26200 and 26220, session
161b838c) the MS_Power_Management_Event_Provider is still registered in the
WMI repository but activating the extrinsic push query fails with
WBEM_E_NOT_FOUND - the legacy provider no longer loads on exactly the OS
builds new devices enroll with, so the power watcher degraded fleet-wide.

Arm chain: try the push event first (still alive on older builds), then
fall back to the intrinsic __InstanceModificationEvent WITHIN 30 on
Win32_Battery. The WITHIN sampling runs inside WinMgmt, not in the agent;
the handler path is unchanged (payload ignored, re-probe + diff), worst-case
detection latency ~35s incl. debounce. collector_degraded now fires only
when BOTH attempts fail; the arm log line names the active mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 15:49:03 +02:00
Oliver Kieselbach 8f24876682 feat(agent): live power-state watcher + battery analyze rules DEV-009/010
Event-driven AC/battery observation during enrollment (no polling): a WMI
Win32_PowerManagementEvent (EventCode 10) push subscription re-probes
GetSystemPowerStatus and diffs snapshots through a pure tracker into
power_state_change events - transition ac_to_battery (Warning, immediate) /
battery_to_ac (Info) / threshold_crossed at 50/30/15% (per-level lifetime
latch, lowest-crossed-only on jumps, arm-time baseline covers sessions
starting low, 5s trailing-edge debounce, emission cap 20). Never armed on
devices without a battery; arm failure surfaces as collector_degraded.
Kill-switch Collectors.EnablePowerStateWatcher (ConfigVersion 38).

power_state_change is classified non-activity on both sides
(SignalActivityClassifier + IsPeriodicOrStallEvent) so battery drain can
neither keep idle collectors alive nor heal a Stalled session.

New analyze rules, both with on_event:power_state_change interim triggers
and deliberately single-required-condition (event_data conditions scan
events independently - no same-instance join):
- ANALYZE-DEV-009 (high): thresholdPercent equals 15
- ANALYZE-DEV-010 (warning): transition equals ac_to_battery

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 15:24:41 +02:00
Oliver Kieselbach 50e7590060 feat: app install durations measure the status-defining attempt
The IME processes the ESP app list in multiple passes (device-ESP evaluation
ending Skipped, real install up to an hour later), and AppInstallSummary
folded first observation -> last terminal across them: a 23-second app late
in the queue reported 4002 s (session 44862cd3, "Set TimeZone").

Backend ingest: new max-folded LastAttemptStartedAt anchors DurationSeconds
(CompletedAt - last attempt start; legacy rows without the anchor keep the
span fallback), replay-guarded additive InstallPassCount, attempt-scoped
DownloadDurationSeconds pairing, and a weaker-terminal guard (completed with
state Skipped/Postponed never overrides Installed/Error, in-batch and in the
cross-batch reconcile). StartedAt stays the earliest-wins window/version-
ordering key. AppsDashboardProjection carries the new columns.

AppVersionRegressionRadar: cutover gate — only attempt-anchored rows enter
the duration population on BOTH comparison sides, so mixed-semantics medians
can neither mask real regressions nor falsely re-arm active episodes.

Time attribution: per-pass segment pairing replaces the first-start ->
last-terminal hull. An app's seconds = sum of its ACTIVE segments (failed
attempts and evaluation passes count; inter-pass idle gaps don't), occupancy
merges the segments, hull endpoints remain for display and what-if bounds.

Agent: a terminal->active transition re-arms BOTH timing endpoints, so event
payload durationSeconds also describes the attempt.

Web/PDF/MCP: labels and tooltips say "install time (final attempt)", the
apps sessions table gets a xN multi-pass chip, the time-series chart carries
a cutover footnote, MCP tool descriptions document the semantics split
between get_app_install_metrics and get_time_attribution.

No backfill: pre-cutover rows keep span values (documented in the public
changelog). OKF doc: docs/backend/app-install-duration-attempt.md.

Suites: Functions 4383, Agent 2407, MCP 564, Web 814 - all green, build 0/0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 13:40:47 +02:00
Oliver Kieselbach 2e20476559 feat(agent): extend RealmJoin timeout while first deployment is demonstrably active
The 60-min RealmJoinTimeout deadline arms at RJ detection (the RJ agent MSI
install during DeviceSetup) and used to cut off monitoring mid-install: report
55e6afd61c9d (session 46a96078) lost the tail of a large package catalog whose
last observed package completed 3 seconds before the deadline, with the first
deployment only starting 16 minutes into the window.

When the deadline now fires with LastDeploymentPhase 100/101 and deployment
activity (phase change or package observation, persisted restart-safe in
RealmJoinFacts.LastActivityUtc; the detection-time phase capture deliberately
does not count) inside the last 60 min, the handler re-arms to
lastActivity + 60 min instead of timing out — Info event
realmjoin_timeout_extended — clamped at an absolute cap of detection + 4 h.
realmjoin_timeout carries a reason payload (hard_timeout / inactivity /
absolute_cap); a fire of a deadline incarnation superseded by a re-arm
dead-ends via the OccurredAtUtc == DueAtUtc scheduler contract. The idle case
(no activity ever) still times out at 60 min, and phase-200/210 activity never
extends. Packages are only registry-visible at completion, so the inactivity
window keeps the full original 60 min.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 01:11:48 +02:00
Oliver Kieselbach 990641b19e feat: P14 - per-upload-batch send time separates spool delay from clock offset
Agent stamps X-Send-Time-Utc (device clock, ISO-8601 round-trip) per upload
ATTEMPT; ingest parses it (year>=2000 guard, deliberately no upper clamp) and
server-stamps SentAt onto every event of the request. SentAt - OccurredUtc is
pure spool delay, ReceivedAt - SentAt is network + device-vs-server clock
offset - previously indistinguishable and only separable via median statistics.

clock_skew condition: batches carrying SentAt measure the frame directly
(spool-immune by construction - no spread cap, IME/clamped exclusions
irrelevant, IME-only sessions become measurable; evidence sentAtBatchCount);
legacy batches keep the per-event median path unchanged. CmTraceSkewTripwire
deliberately untouched: its differential median math cancels clock offset as
common mode.

Full persistence chain: Store x2 + Map, fields= projection (sentAt), MCP
get_session_events field list + rule-authoring-guide. New OKF doc
backend/send-time-frame-separation.md.

Suites: Functions 4370 (+16), Agent 2407 (+1), MCP vitest 564 + tsc 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 00:44:29 +02:00
Oliver Kieselbach 34fd40c3ad fix(agent): refuse cross-file calibration anchors, make rejections visible
Field forensics from the 1412 verification run (session 067797d8): the
label/anchor crossing recurred on a provably genuine binary, and the
new anchor-timestamp tripwire delivered — the "+02:00 measured for
IntuneManagementExtension.log" anchor (local 19:51:26.948) does not
exist in ANY log file on the device, and IME.log never produced its
expected first measurement during 24 s of growing passes beforehand.
Mechanism still unexplained; per-line anchoring keeps resolution
immune (all 58 line-anchored events of the session were correct), but
the observational layer must convict the culprit next time:

- CalibrateFrom now receives the iteration's ACTUAL file name next to
  the calibration key. A mismatch — the smoking gun the two field
  occurrences imply but static analysis says is impossible — logs a
  Warning with both names plus the anchor, and refuses the anchor, so
  even the unexplained path can no longer poison the measurement store.
- The first calibrator rejection per file logs at Debug (anchor local
  time + clock). The 24 s of silent non-measurement is exactly what
  made the phenomenon undiagnosable from the agent log.

232 IME tracker tests green against a verified-clean build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 20:18:35 +02:00
Oliver Kieselbach 64336c0c89 feat(agent): resolve fresh CMTrace lines by per-line self-anchoring
The era-aware successor to the reverted per-file offset application
(04b1a7c6). At a 100 ms poll, every line of a provably fresh pass was
written essentially "now", so its own distance to the agent clock,
rounded to the 15-minute offset grid, IS its writer's UTC offset — no
cross-line state, which is what interleaved writer eras break. The line
timestamp still contributes sub-poll precision and ordering; the grid
absorbs poll latency. Origin: LineAnchored → "line-anchored".

Freshness is stricter than "the file grew": every poll look stamps
LastCheckedUtc (empty first sights included, so AgentExecutor.log
anchors from its very first content), a pass anchors only when the gap
since the last look is ≤ 30 s (FreshLineMaxAge — far above the poll,
far below the 13-minute zone where a line's AGE rounds onto the grid),
and a restart-restored bookmark deliberately carries no freshness at
all — downtime backlog must never anchor. Everything not provably
fresh keeps the uniform reader-zone fallback: wrong in absolute terms,
self-consistent for durations, and marked as such (the revert's core
lesson — partially corrected is strictly worse than uniformly wrong).

Accepted, bounded edge (characterization test): a freshly WRITTEN line
carrying a replayed timestamp aged exactly N×15 min ± 2 min anchors to
≈now — an anchored line lands at now ± 2 min by construction, and
sub-24h replays are treated as current by the historic-replay guard.

The per-file measurement stays observational (Info log +
measuredWriterOffsetMinutes) and now logs the anchor's local timestamp
as a tripwire: session e9753578 logged "+02:00 measured for
IntuneManagementExtension.log" although that file provably held no
+2-era line — a label/anchor pairing the committed code could not be
shown to produce (iteration-local anchor, single sequential call site,
green repro of the exact field scenario, no +2-capable chunk in the
window). The mechanism remains open; the resolution path is immune to
it by design, and CmTraceCalibratorFileIsolationTests pins the file
isolation so a real regression can never ship silently.

Tests: 13 new — resolves-to-T across five writer beliefs, interleaved
eras inside one chunk, first-sight/empty-first-sight/poll-gap/restart
freshness boundaries (including the nastiest backlog: exactly 30 min
old, on-grid), the grid-replay edge, and the committed two-writer-era
fixture replayed at production poll cadence across its write silences.
Agent suite 2406 green against a verified-clean build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 19:28:27 +02:00
Oliver Kieselbach 59c944618b test(agent): move CMTrace fixture to tests/fixtures and anonymize it
Two corrections to 3c25172d, both found by asking why the repo suddenly had two
fixture locations.

Location: committed test data lives under tests/fixtures/. I had put this one
inside the agent test project without checking, creating a second parallel home.
It does NOT belong in tests/fixtures/enrollment-sessions/ either — that is a
governed corpus of anonymized DecisionSignal JSONL for the Replay Harness, with a
schema, category prefixes and a paired-scenario-test rule that raw log text cannot
satisfy. So: a sibling corpus, tests/fixtures/cmtrace-logs/, with its own README
stating why it is separate.

Anonymization: the fixture carried three real Intune policy/script GUIDs from a
customer tenant. Point 1 of the existing fixture policy requires tenant-scoped
identifiers to be replaced, and I committed without applying it — the check
belonged BEFORE the commit, not after. Replaced with
policy-anon-NNNN-0000-0000-0000-000000000000, consistent with the tenant-anon /
device-anon / user-anon convention already in use. The all-zero GUID is a literal
in IME's own output and stays; the session ID stays as our diagnostic anchor and
is already referenced from the commit history and tasks/todo.md.

The new README carries the commit policy for this corpus, including the rules that
were implicit before: verbatim content (the parser is tested against real
formatting), a header stating what the fixture pins, versioned filenames so a
pinned test cannot silently change meaning, and .cmtrace rather than .log because
*.log is globally gitignored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:54:55 +02:00
Oliver Kieselbach 3c25172dfb test(agent): real two-writer-era CMTrace fixture from session e9753578
Captured from ImeLogs/AgentExecutor.log of the session that exposed the regression
(agent 2.0.1410; device booted on the Pacific OOBE default, agent set W. Europe at
12:13:22Z).

One file, two writer eras: 371 lines rendered at local hour 05 by a writer still
on PDT (UTC-7) and 195 at hour 14 by one already on CEST (UTC+2) - nine hours
apart, interleaved, and no line carries a bias suffix, so nothing in the file
itself says which is which. Exactly the case the per-file calibration could not
express, and the test that was missing.

Analysing the archive also turned up a SECOND, independent defect the revert
commit did not know about: the agent log reports "IntuneManagementExtension.log
writer offset measured 02:00:00" while that file contains ONLY hour-05 lines
(2022 of them, zero at hour 14). A +2 measurement is impossible from its own
content, so the calibration anchor leaked across file boundaries - files are
walked in sorted order and AgentExecutor.log precedes it. Two-eras-per-file and
anchor-leak are separate bugs; both must be fixed.

Extension is .cmtrace.txt, not .log: *.log is globally gitignored, and a fixture
has no reason to claim that extension.

Fixture only, nothing consumes it yet - the era-aware calibrator is next.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:47:15 +02:00
Oliver Kieselbach 04b1a7c6e7 revert(agent): stop applying the measured CMTrace offset — one offset per file is wrong
Field regression, session e9753578 (agent 2.0.1410, PST-defaulted device in DE).
Five script_started events resolved 9 h early while their script_completed
counterparts resolved correctly, inflating every platform-script duration to
~32,400 s. Reverts the application introduced in e9dba11b.

Root cause: calibration keeps ONE offset per log file, but a single file can hold
lines from two writer eras. IME restarting across a timezone change leaves older
lines in the old zone and newer ones in the new zone, in the same file. The anchor
is the newest line, so its offset was also applied to the older lines, for which
it does not hold. The class doc even claimed "self-healing on writer restart" —
that reasoning only ever covered FUTURE lines, never the ones already on disk.

The more expensive mistake was the second one: correcting ABSOLUTE timestamps
without checking what depends on their DIFFERENCES. Applying the reader zone
uniformly is wrong in absolute terms but self-consistent — both ends of a duration
are wrong by the same amount, so the duration is right. Correcting only some
timestamps produced a mixed frame inside one session, which is strictly worse than
being uniformly wrong. This is the same asymmetry the original analysis described
for AppInstallSummaries (68 min reported against 8 min real) and then reproduced.

What stays, deliberately:
- Bias-carrying lines still resolve via the writer-declared offset; that path was
  never implicated.
- The measurement keeps running. It costs nothing, feeds the Info log, and is the
  field data the era-aware fix has to be built on — the first anlauf got the era
  question wrong from assumption rather than observation.
- Provenance now separates measured from applied (measuredWriterOffsetMinutes vs
  sourceOffsetMinutes). The broken event was tagged "calibrated" while carrying an
  offset that did not hold for its line; that confusion must not be expressible.
- The fallback path reports the offset it actually applied. An event stating no
  offset cannot be recomputed later, which defeats the point of provenance.

Provenance earned its keep here: 57 events measured -420 (the PDT writer) against
5 at +120 made the diagnosis a two-minute query instead of median statistics over
a session.

Next: era-aware calibration keyed on the IME-STARTED marker, which we already
track, plus a test over a log file containing two writer eras — the case that was
missing.

Agent 2393 green against a verified-clean build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:31:50 +02:00
Oliver Kieselbach 3b59dab2df feat(agent): carry timestamp provenance on IME-derived events
Every IME-sourced event now states what the writer wrote, what offset was applied,
and how that offset was obtained:

  sourceLocalTs       raw local time, verbatim, NO zone suffix
  sourceOffsetMinutes the offset subtracted
  sourceOffsetOrigin  bias | calibrated | reader-zone-fallback

Why: until now the only record of a conversion was the converted value, so a wrong
offset was indistinguishable from a right one. Proving the field defect needed the
diagnostics archive — and of the 5,386 sessions that went through a timezone
change, essentially none had one, so the analysis had to fall back to median
statistics across thousands of sessions to establish what one stored value makes
obvious per session. It also makes a stored timestamp recomputable after the fact,
without an agent release.

No zone suffix on sourceLocalTs on purpose: the value genuinely carries none, and
formatting one in would re-introduce the assumption this whole change removes.

sourceOffsetMinutes is normalised to one sense (local = UTC + offset) across both
origins. The wire bias uses the opposite GetTimeZoneInformation convention
(UTC = local + bias), so a "+480" line is reported as -480; without that the field
would flip sign depending on where the offset came from and be useless to compare.

Added centrally in TagDerivedTimestamp, which all 14 emit sites already call, so
this does not scatter through the adapter. DataJson only — no table schema change,
so no Store/Map/seed chain and no backend deploy needed to start collecting it.

Agent 2393 / backend 4313 green against a verified-clean build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:05:32 +02:00
Oliver Kieselbach 389426f8f2 fix(agent): future-skew guard that a one-hour timezone error can actually trip
ResolveOccurredAt rejected a source timestamp when ageHours < -1.0. That bound
could not fire for the single most common timezone error: with a one-hour offset
mistake the real value is parseLag - 3600 s, i.e. just ABOVE -1.0. It sat exactly
on the boundary of the case it was meant to catch, which is why every skewed
session sailed past it.

It was asymmetric too — a past-shifted error had to exceed 24 h before anything
noticed, while the measured field errors reached -17 h.

The bound is now derived rather than guessed: writer and reader share one system
clock, and the tracker only sees a line after it was written, so a correctly
converted source timestamp is always at or behind "now". Anything ahead means the
conversion was wrong. MaxFutureSourceSkew is 2 minutes of jitter tolerance —
deliberately far below the hour a timezone mistake produces.

The rejection log now names which bound tripped and by how much, instead of
printing a bare ageHours that reads the same for both directions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:05:31 +02:00
Oliver Kieselbach e9dba11b96 fix(agent): resolve IME log timestamps with the writer's measured offset
This is the behaviour change the previous two commits prepared. ImeLogTracker no
longer substitutes its own timezone belief for the IME process's; it measures the
writer's offset from a freshly appended line and converts with that.

Freshness is stricter than "the file grew". GetSafePosition returns 0 for an
unknown file, so the first pass reads the whole file and its newest line may be
hours old — anchoring there would measure the line's age, not the writer's offset.
LogFilePositionTracker.HasSeen(), captured BEFORE GetSafePosition creates the
entry, gates calibration on growth against a size we had already observed.

Calibration runs at the END of a qualifying pass, so that pass still uses the
previously measured offset — at most one poll (100 ms) old, and an offset only
moves when a process restarts. Calibrating first would mean buffering the pass,
which is not viable: the first pass over AppWorkload.log can be hundreds of MB.
The cost is a warm-up of one growing pass, during which lines fall back to the
reader zone.

Bias-carrying lines are never used as anchors: they already state the writer's
offset, so they need no measurement and must not overwrite one.

CalibrateFrom logs at Info when the measured offset disagrees with this process's
own zone, naming the correction applied. That disagreement is precisely the
condition that silently corrupted every IME-derived timestamp until now; agreement
stays at Debug so the normal case does not add noise.

Tests are deliberately machine-independent: the assertion is always "a line
written at instant T resolves to T", asserted across writer beliefs of +2, +1, 0,
-7 and +10. That one property covers both field cases — the writer disagreeing
with us (the bug) and the writer agreeing with us (the majority of sessions, where
two stale beliefs cancel and which a fix to the agent's own cache would have
broken). Plus: no calibration on first sight, following a writer restart into a
new zone, and a declared bias honoured without being measured.

CheckLogFilesAsync is internal as a test seam — calibration depends on the pass
structure, which only a real read cycle exercises.

Agent 2390 / backend 4313 green against a verified-clean build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 13:49:14 +02:00
Oliver Kieselbach 828cbb5cd0 refactor(shared): CMTrace parser stops guessing the writer's timezone
Behaviour-neutral: 2381 agent and 4313 backend tests pass unchanged.

The parser resolved a bias-less line with DateTimeStyles.AssumeLocal followed by
ToUniversalTime(), i.e. with the READING process's cached zone. That silently
assumes reader and writer agree. They do not — each caches its zone at process
start and neither follows a later tzutil or Windows auto-timezone change — and the
assumption was buried inside a shared helper where no call site could see it.

CmTraceLogEntry now exposes what the line actually contains: LocalTimestamp
(Kind=Unspecified, because the value is meaningless without knowing the writer's
zone) and BiasMinutes. TimestampUtc is populated only when the writer declared its
bias, which is the one case where the offset is genuinely known. Timestamp is
removed rather than deprecated, so the compiler names every site that has to
decide — it immediately turned up ProcessLogMessageForTest, which grep had missed.

The defective path still exists, but now as one named function,
ResolveUtcAssumingReaderZone(), whose documentation states when it is wrong. All
three consumers call it and so keep their current behaviour exactly:
- ImeLogTracker routes every use through ResolveEntryUtc(), the single point where
  the measured offset replaces the fallback in the follow-up change;
- StallProbeCollector and LogParserCollector read file snapshots rather than
  tailing, so they cannot measure an offset and stay on the fallback for now.

The parser also no longer invents a timestamp. An unparseable time used to be
stamped DateTime.UtcNow inside the parser, making a fabricated value
indistinguishable from a real one downstream; it now reports HasTimestamp=false
and leaves the fallback to the caller, which alone knows what is safe in context
(StallProbeCollector still treats such a line as fresh — calling it stale would
hide progress).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 13:35:43 +02:00
Oliver Kieselbach 93a42ba5e4 feat(agent): measure the CMTrace writer's UTC offset instead of assuming it
A CMTrace line carries local time and no offset — IME 1.104 writes
DateTime.Now.TimeOfDay in both of its trace listeners. Converting it with the
agent's own TimeZoneInfo.Local assumes both processes agree on the zone. They do
not: each caches its zone at process start and neither follows a later tzutil or a
Windows auto-timezone change, so the error is Offset_writer - Offset_agent and is
zero only by coincidence.

Measured over 11,068 sessions: 26 carry a real zone-offset error, +1 h (GMT to
W. Europe, session bfff9b1c) through -17 h (a writer still on the Pacific OOBE
default while the agent had already moved to E. Australia). Note the sign — this
is why "make the agent's own clock honest" is the wrong fix: in the -17 h cases
the agent was correct and the writer was stale, and clearing the agent's cache
would have broken the majority of sessions where both are stale and the two errors
cancel.

The tracker tails at a 100 ms poll, so a line that appeared since the previous
pass was written essentially "now". That makes the writer's offset measurable:
round(lineLocal - agentUtcNow) onto the 15-minute grid every real UTC offset sits
on. Per source file, because IntuneManagementExtension.log and AppWorkload.log are
written by different processes; re-measured on every fresh anchor, so a writer
restart is picked up within one poll.

The freshness requirement is the load-bearing part and cannot be replaced by the
residual guard: a backlog line exactly 30 minutes old is itself a clean multiple
of the grid, rounds with zero residual, and would calibrate the offset half an
hour wrong. RejectsBacklogLine_EvenWhenItRoundsCleanly pins that the calibrator
cannot detect this on its own — only the caller's freshness rule prevents it.

Additive only; no call site uses it yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 13:25:48 +02:00
Oliver Kieselbach 78cc038426 fix(agent): record real bytes-down when AutomaticDecompression strips Content-Length
The pipeline's AutomaticDecompression (GZip|Deflate) removes the response
Content-Length header on virtually every backend JSON response, so
net_total_bytes_down in agent_metrics_snapshot read 0 on healthy sessions.
When the header is absent, buffer the content (LoadIntoBufferAsync, 16 MB cap)
and record the DECOMPRESSED payload size — HttpClient buffers responses anyway
(default ResponseContentRead), so no extra memory cost. Fail-soft: a buffering
error keeps 0 and leaves the failure for the caller to observe.
AutomaticDecompression itself stays untouched (pinned by
MtlsHttpClientFactoryTests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 01:25:41 +02:00
Oliver Kieselbach 022cc0354b fix(engine): let the observed user-ESP skip stand in for the exit evidence
The five stuck sits-d Cloud PCs carried SkipUserStatusPage=True — verified
across three agent processes (10+ registry reads, True every time) and six
hours of live Shell-Core watching with zero 62407: Windows never rendered a
user ESP page, so no exit and no AccountSetup provisioning registry can ever
exist on this flow. Arm B of ShouldTransitionToAwaitingHello was open from
the bootstrap EspConfigDetected onward, but every evaluation site hangs off
a signal the flow cannot produce, and the AdvisoryCompletion backstop arms
exclusively from an EspExiting signal — an open gate nobody knocks on, six
hours to the max-lifetime watchdog. The earlier lost-exit-through-reboot
diagnosis is retired by the same evidence.

HandleImeUserSessionCompletedV1's completion attempt now accepts the
observed skip in place of IsPostAccountSetupFinalExit (anchor + genuine IME
user-session completion + real-user desktop stay mandatory; Device
Preparation excluded). The knock deliberately lives on the IME edge, not on
desktop arrival: Fix 10's AccountSetup bounce-back would undo a desktop-side
promotion two signals later, while the IME phase line always precedes the
user-session-complete line in log order, so a bounced promotion is
re-knocked by the very next re-emission — restart re-parse included.

Docs and comments no longer claim an ESP-exit recovery that does not exist
(stray duplicate summary in ShellCoreTracker, ImeLogHost restore-callback
comment); the OKF doc is reframed on the evidence and names the residual
SkipUser=false exit-in-downtime gap honestly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 00:40:23 +02:00
Oliver Kieselbach 3aa8b95676 fix(agent): narrow the Shell-Core replay to the Hello-wizard rail
Round-four review, confirmed — and the cause sits a layer below where I had been
fixing. The IsBackfill return added last round only guarded the coordinator's own
completion logic. Upstream of it, ShellCoreTracker still raised
FinalizingSetupPhaseTriggered, the coordinator still forwarded EspExited to the
adapter before that return, and the adapter discards the args entirely
(OnEspExited(...) => EmitEspExiting()). So a replayed exit still arrived at the
reducer as an ordinary, unmarked EspExiting.

There it is decisive: HandleEspExitingV1 passes espFinalExitInFlight: true for
every arriving exit, and IsPostAccountSetupFinalExit orders exits by ingest
ordinal rather than timestamp — so a historic exit replayed today reads as
post-AccountSetup by construction. With state restored from the snapshot
(AccountSetup entered, a genuine IME user session, desktop arrived) arm C of
ShouldTransitionToAwaitingHello opens on the Device→Account handoff. The reducer
is itself a completion gate and cannot tell the two apart, so the fix has to be
upstream.

The replay is therefore narrowed to what is provably safe: the Hello-wizard start
(62404). That fact is conservative — it vetoes a premature "Hello is disabled"
skip and can never by itself complete a session, so replaying it can only make the
agent wait longer. It is also the observation session 772fe502 wrote the replay
for. A 62407 is no longer raised at all, neither exit nor failure: the exit
because Windows writes the identical description for the intermediate and the
final one and nothing downstream can order it, the failure because re-injecting it
as fresh can fail a session that recovered on retry (ANALYZE-ESP-006). Skipped
records are reported once as an agent_trace with counts and bounds — silently
dropping evidence is how this class of bug survives.

The cross-layer proof is now a pair, as the review asked:
ClassicEspExitingOnRestoredStateTests pins that the reducer opens on any arriving
exit (including one with an hour-old source timestamp), and
ShellCoreTrackerReplayScopeTests pins that the replay never produces one.

Dead machinery removed with it: EspExitedEventArgs.IsBackfill, allowEspExit, the
newest-exit selection, the IsBackfill return. The method is now called
BackfillRecentHelloWizardStart.

One pre-existing integration test drove EspExited through HandleBackfillRecord
because the replay is easier to call than the event-log watcher, and its class doc
claimed that was the production path — it never was, the replay had no caller when
that test was written. Switched to the live path and the doc corrected.

Agent V2.Core 2361, DecisionCore 538, Backend 4287, all green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:05:14 +02:00
Oliver Kieselbach ec945a93d2 fix(agent): bar replayed ESP exits from the completion gate entirely
Round-three review, confirmed. The previous round stopped a replayed exit from
arming the DEFERRED re-check but left the immediate synthesis attempt right below
it untouched. My justification for that — "pre-existing behaviour" — was simply
wrong: the immediate attempt on a replayed exit never existed, because the
backfill had no caller. I introduced it, and by moving the replay behind the IME
state restore I made the dangerous state reachable by design: if the newest 62407
in the window is only the intermediate DeviceSetup→AccountSetup exit while the
restored app states are already terminal, that attempt opens the strong
AccountSetup gate before the final exit has even happened. A test was pinning the
faulty assumption as expected behaviour.

A replayed exit now returns before touching completion at all — no deferred edge,
no immediate attempt. Nothing available to it can establish its position in time:
Shell-Core writes the identical description for both exits, and every probe that
could order it reads state as it is now rather than at the event's time. The
replay keeps the job it was written for in session 772fe502 — feeding EspExiting,
FinalizingSetup and the Hello wizard to the reducer for the window in which no
agent process was observing. Completion is the exclusive business of exits this
agent watched live.

With the synthesis attempt gone, the replay no longer needs restored app states,
so it moves back into EspAndHelloHost.Start(); that also keeps this host's startup
independent of the IME host's. ImeLogHost.onStateRestored goes back to doing just
the re-check, which is still needed for a live exit observed before the restore
finishes.

The test asserting the faulty assumption is inverted, plus coverage that the
replay is not neutered (EspExiting still forwards) and that a live exit following
a replayed one completes normally.

Agent V2.Core 2364, DecisionCore 533, Backend 4287, all green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:25:22 +02:00
Oliver Kieselbach b36cf8ff36 fix(agent): order the ESP-exit replay by evidence, size its window by measurement
Round-two review, both findings confirmed against the code.

A replayed exit was being confirmed as post-AccountSetup by reading the registry
at replay time. That read reflects NOW, not the event's time, so an agent that was
down across the DeviceSetup→AccountSetup transition would confirm the stale
intermediate exit and, once the apps settled, synthesise a completion while the
AccountSetup page was still up. Windows offers no way to tell the two apart from
the event itself — Shell-Core writes the identical description
CommercialOOBE_ESPProgress_Page_Exiting for both. So the deferred edge is now
armed by LIVE exits only (EspExitedEventArgs.IsBackfill): only for those is the
continuous-observation argument valid. A replayed exit keeps the single immediate
synthesis attempt it always had.

For that attempt to be worth anything it must see real data, so the replay moved
out of EspAndHelloHost.Start() into ReplayEspExitBackfill(), invoked from
ImeLogHost.onStateRestored — after LoadState() has restored the package states.
That is what makes "the agent came back long after the ESP finished" recoverable.

The five-minute window for non-reboot restarts was unsound: the scheduled task
carries a BootTrigger only, with no restart-on-failure, so after an
exception_crash the agent does not return until the next boot — hours later, long
after Windows wrote the final exit. The window is now measured rather than
guessed: back to the last moment the previous run is known to have been alive,
taking the wider of the snapshot mtime (rewritten every decision step, available
for every exit type) and LastBootUtc. The five-minute default is the floor,
ClampLookbackMinutes the ceiling, and a future timestamp from clock skew is
ignored instead of producing a negative window.

Also switched the backfill tests to the real Shell-Core wording — the invented
string matched the regex but hid the fact that both exits look identical.

9 new tests. Agent V2.Core 2362, DecisionCore 533, Backend 4287, all green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:06:14 +02:00
Oliver Kieselbach c6ee7808f1 fix(agent): scope the ESP-exit replay to restart recovery only
Follow-up to the review hardening: the replay ran on EVERY agent start with a
5-minute window, first run included. On a first run there is no gap to recover —
no earlier agent process existed — and a 62407 already sitting in the Shell-Core
log belongs to a phase transition this agent was never meant to observe. Replaying
it pushes an EspExiting into the reducer that the pre-fix agent never saw, i.e. a
happy-path behaviour change for no gain.

ResolveEspExitBackfillLookbackMinutes now keys the window to the previous exit
type: first_run (or unknown) disables the replay entirely, clean/hard_kill/
exception_crash keep the 5-minute default because the process is back within
seconds, and reboot_kill widens to "since LastBootUtc" because that gap spans the
forced restart until the post-reboot logon. The result is still clamped to
BackfillLookbackMaxMinutes, so policy and backstop together cannot build an
unbounded event-log query.

With this the happy path is byte-identical to pre-fix behaviour: on a first run
nothing replays, no edge is remembered, and the deferred re-check returns on a
volatile bool read.

No other backfill is affected — the Hello, MDM-reboot, Windows-Update and
ModernDeployment trackers each own a private backfill invoked from their own
Start(), none of which this change touches.

13 policy tests including clock skew across the reboot, a missing boot timestamp
and the clamp backstop. Agent V2.Core 2353, DecisionCore 533, Backend 4287.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 17:37:40 +02:00