mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-25 13:10:13 +00:00
7b975d46b626afda62ea5e7e287c5889bd65ee72
737 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oliver Kieselbach
|
7b975d46b6 |
feat(backend): Azure Monitor alert webhook - platform alerts join the ops-event bus
New anonymous POST ops/alert-webhook receives Common Alert Schema notifications from an Action Group webhook receiver and re-emits them as ops events (category Platform, type AzureMonitorAlert), so Azure Monitor alerts flow through the existing OpsEventService -> OpsAlertDispatchService fan-out (Telegram/Teams/Slack + portal ops-events list) like every internally raised operational event. The delivery path is independent of the portal: a SWA outage cannot suppress the notification; the action group's e-mail receiver stays as the backend-independent fallback. Security: action-group webhooks cannot send custom headers, so the shared secret travels as ?secret= and is compared in constant time (FixedTimeEquals) against the OpsAlertWebhookSecret app setting - fail-closed while unset. 64 KB payload cap, defensive essentials-only parsing (only alertRule is mandatory), severity mapped Sev0->Critical, Sev1->Error, Sev2->Warning, Sev3/4->Info with unknown values falling back to Info so schema drift can never escalate. Dual-registered per convention: EndpointAccessPolicyCatalog entry (PublicAnonymous), OPS_EVENT_TYPES in OpsAlertRulesSection (rule wiring) and Platform category in OpsEventsSection (badge + filter). Backend suite 3511 green (incl. new helper tests + catalog completeness), web tsc + vitest 651 green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
ba7d14b070 |
fix(insights): post-deploy MCP findings - projection drop, aggregate starvation, duration artifacts
Three defects found by measuring production after the 1.5.1012 deploy: - AppMetricsProjection dropped TerminalState, so the PR0 skip classification never reached metrics/app (totalSkipped stuck at 0). Column added; equivalence fixture now populates TerminalState/AppId/ EspBlocking/AppIdCollision and asserts them (proven: test fails without the projection fix). - BuildTimeAttributionAggregates excluded breakdowns on ANY quality flag; 903/1000 carry BlockingSetUnknown, starving fleet stats (Clean=12 vs 331 excluded on the largest tenant). New TimeAttributionFlagQuality.DurationCriticalFlags mask - only ClockSkew/PartialObservation/WGAnchors exclude; blocking-set flags keep their measured spans (positive-evidence semantics unchanged). - HasMeasuredDuration gets an upper plausibility bound (6h = default AgentMaxLifetimeMinutes): 490 rows carried back-stamped completions (up to 117 days; 60175s app rows in a 2697s session) that dominated slowestApps with six-figure averages. Read-side, heals old + new rows. Backend 3484/3484 green; solution build clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
7842d121a0 |
fix(insights): codex review findings - WG pause clamping, late-event healing, stale buckets, date windows, banner count
All six review findings verified against code and real; five fixed here, the agent-side race lands in the companion agent commit. [P1] WhiteGlove pause counted as time: ClampToWindows clipped intervals against the HULL of both WG windows, but the excluded pause lies inside the hull - a reboot gap or app interval straddling the pause counted the entire pause (a ~2-day "reboot" in a 30-minute enrollment was possible). Seconds are now the per-window intersection sum (chronological endpoints kept); an interval wholly inside the pause makes no claim and no longer raises ClockSkewDropped. What-if savings are additionally capped at the session's WallClockSeconds - endpoint deltas can still straddle the pause, and no removal can save more than the enrollment took. [P1] Late-arriving events never updated breakdowns: the terminal-seam compute is one-shot and the sweep only healed missing/version-stale rows. SessionTimeBreakdown now stores EventCountAtCompute (0 = pre-fix row, recomputed once); the sweep recomputes on EventCount drift and then also re-runs ResolveEspBlockingForSessionAsync (newly exposed on IMetricsRepository) so late esp_config_detected re-emits and app batches reach both the breakdown and the blocking evidence. [P2] Stale aggregate buckets: the sweeps now reconcile every partition they wrote - buckets that existed but were not regenerated (sessions deleted, class left the window) are deleted via new targeted deletes; rolling30 rows additionally get a 48h ComputedAt freshness filter at read time, so a fully quiet tenant stops serving year-old numbers as "last 30 days". [P2] Date windows included an extra calendar day: both inclusive range ends meant days=30 returned 31 keys and days=1 summed yesterday AND today. InclusiveWindowStart(today, days) = today-(days-1) in both response builders; the sweep's rolling30 source is filtered to the same 30 calendar days (its query window stays one day wider for backfill). [P3] Device-history banner counted later sessions as "previous enrollments" when viewing an older session: selectPriorSessions (deviceHistoryPrior.ts) anchors on the session's own chain entry (live sessions: new sessionStartedAt prop), sessionId tie-break mirroring the server chain sort. Verified: solution build clean, Backend 3480/3480 (+6: three WG-pause golden fixtures, window-start facts, RK builder/delete consistency, EventCountAtCompute roundtrip), Web tsc + vitest 633/633 (+6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
1bf37f6761 |
feat(insights): F3 regression radar - Wilson-gated rule hit-rate drift detection (PR6)
Analyze rules only (gather stats stay per-batch-deduped until Q6 is
fixed - never silently included). Per (tenant, rule) the radar compares
the trailing 7-day hit rate (FireCount/SessionsEvaluated from the daily
RuleStats rows) against the prior 28-day baseline and fires only when
the window has >=5 hit sessions and >=20 evaluated sessions, the rate
is >=2x baseline AND the Wilson 95% intervals are disjoint
(MetricsMath.WilsonInterval/RateIncreaseSeparated, hand-computed test
vectors). Suppressed for deleted rules, rules inside the CreatedAt
grace period (<35d), rules edited inside the window, empty baselines,
and the RuleRegressionRadarDisabled app-setting kill switch.
Alert episodes live as ruleregression|{ruleId} rows in the notification
tracker (third keyspace): AddEntity-409 dedup = one bell + one
RuleFrequencyRegression ops event (category Tenant, Warning, full
numbers, dual-registered in the web OPS_EVENT_TYPES) per episode;
numbers refresh while active without moving FirstNotifiedAt; episodes
re-arm under 1.5x baseline, on fire-stop, or via the tracker's 30d
retention prune (spec-intended reminder). On-fire dimension correlation
(osBuild/model/agentVersion/imeVersion; hit set from a RuleResults
range scan intersected with the window sessions) reports both shares
with the "correlated, not necessarily causal" wording contract or
honestly "no clear dimension concentration". Radar runs in the daily
maintenance pass after the rule-stats aggregation, anchored on
yesterday (whole days only).
Surfaces: regressions[] in the rule-stats response (MCP get_rule_stats
inherits it), tenant bell deep-linking to the rule card, and the
analyze-rules page gains a densified 30-day fire sparkline plus a
regression badge while an episode is active. OKF doc
backend/rule-regression-radar.md + index/log.
Backend 3474/3474 (27 new: Wilson vectors, every suppression branch,
window boundary dates, re-arm matrix, dimension gates, tracker
round-trips incl. tri-states, keyspace dedup/prune, wording pins),
web tsc + vitest 627/627, MCP tsc + 455 passed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
88da317bf4 |
feat(insights): F2 surfaces - device-history banner, fleet FTR panel, API routes, MCP tool (PR5)
Three read routes over the PR4 data: metrics/device-history
(MemberRead + tenant query param - one route serves the web banner AND
MCP, GA cross-tenant via ?tenantId= with a sessionId-based owner
fallback; history:null is a normal outcome), metrics/device-journeys
and global/metrics/device-journeys (days selector honored because FTR
counts are additive - window rate = ratio of summed daily rows; the
tenant-less global variant serves the "global" partition and discloses
repeatDevices:null). Attempt numbers are server-computed
(DeviceJourneyCalculator.ComputeAttemptNumber: chain position within
the journey; live sessions via a virtual non-successful attempt so the
redeploy and 30d-gap rules apply) - no consumer re-derives journey
semantics. Repeat-devices violator list: last journey >=2 attempts with
newest terminal ref in-window, attempts desc then recency, cap 10,
failure reason from the newest failed attempt (bounded point-reads).
Web: DeviceHistoryBanner on session detail ("Attempt N for this device
- View history", expandable terminal chain, durations strictly the
authoritative durationSeconds, WG/DevPrep/admin-marked badges) and the
Fleet Health "First-time-right" section (rate gated at >=20 completed
journeys with disclosed n, junk-serial exclusion note, Monday-UTC
weekly trend, attempt histogram, repeat devices) following the 7/30/90
selector. MCP: get_device_history (serial mode / fleet FTR mode,
role-aware routing, all roles - GA_FULL privilege snapshot extended
deliberately). Docs: OKF backend/device-journeys.md + index/log.
Backend 3447/3447 (14 new: attempt-number vectors incl. gap boundaries
and live sessions, additive sums, days clamping, repeat-device
gate/sort/cap), web tsc + vitest 627/627, MCP tsc + 455 passed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
1e6de47433 |
feat(insights): F2 journey grouping + DeviceHistories chains + daily FTR aggregates (PR4)
Device-key normalization (trim+casefold, junk-serial exclusion incl. the agent's "Unknown" WMI sentinel), pure journey grouping (terminal set Succeeded/Failed/Incomplete; journey ends at first success; >30d gap from the previous attempt's CompletedAt starts a new journey; WG Pending stays an open session, never an attempt), and the DeviceHistories table (PK tenant, RK encoded normalized serial): terminal-only session-ref chains capped at 20 with derived journey counts, written inline at the session-terminal seam (all three terminal statuses, DeletionState-guarded) and healed by a rolling 30d maintenance sweep. The sweep prunes chain refs of deleted sessions tombstone-driven (SessionTombstones cover cascade AND retention deletions), deletes emptied rows, and recomputes daily FTR aggregates (DeviceJourneyAggregates, PK tenant/"global", RK date): completed journeys, first-time-right count, attempt histogram and junk-serial disclosure — additive counts, bucketed on the completing success session's StartedAt date. Chain refs carry the authoritative DurationSeconds verbatim (never CompletedAt-StartedAt) plus an adminMarked flag. Both tables registered in Constants.All + offboarding TenantPartitionTables; aggregates under the 180d retention block; deliberately NO per-session deletion-manifest step for DeviceHistories. Backend 3433/3433 (51 new tests: junk/normalization vectors, terminal set, exact 30d gap boundaries, WG one-row-one-attempt, redeploy split, deletion pruning, entity round-trips incl. row-key escaping, FTR aggregation core), Agent 2044+493 green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
6de21e821a |
feat(insights): F1 surfaces — session lane, fleet panel, API routes, MCP tool (PR3)
Insights expansion PR3 (tasks/insights-expansion-spec.md §F1 "Surfaces").
Completes F1: schema (PR1) -> persistence/aggregation (PR2) -> surfaces.
- Rolling range rows: the sweep now also writes "rolling30|{class}"
aggregate rows — range statistics over the window's actual sessions. A
median of per-day medians is not the range median, so the fleet panel
reads these and is deliberately fixed to "last 30 days"; the daily rows
stay in the payload for trends/MCP.
- Routes (all registered in EndpointAccessPolicyCatalog):
GET sessions/{sessionId}/time-attribution (MemberRead + tenant query
param, GA fallback via FindSessionTenantId; breakdown=null is SUCCESS —
a missing lane is a normal state, not an error), GET
metrics/time-attribution (MemberRead) and GET
global/metrics/time-attribution (GlobalReadOrAdmin + tenantId filter),
both returning { windowDays, classes (rolling), daily }.
- Web: TimeAttributionLane under the session-detail phase timeline —
proportional stacked bar of segment totals incl. the explicit
unattributed remainder (bar always sums to the session's authoritative
duration), ESP-blocking apps with critical-path occupancy, reboot badge
(cross-cutting, never a bar slice), quality-flag chips; fetched
fail-soft for terminal sessions only. Fleet Health gains the "Time
attribution" section: per-class stacked medians gated at n>=20
("insufficient data (n=...)" below), flagged/missing exclusions
disclosed, and a top-blocking-apps table with class-scoped rows and
"up to X (p75 Y)" what-if wording. Deliberate deviation: the spec's
per-app trend column is omitted — per-day app rows are too sparse under
the daily n>=5 gate; revisit once rolling history exists.
- MCP: get_time_attribution (one tool, session + fleet mode, role-aware
routing like get_app_install_metrics, registered for all roles) — the
privilege-leak-guard catalog snapshot extended deliberately.
- Docs: OKF concept doc backend/time-attribution.md (invariant, windows,
segment mapping, positive-evidence blocking, Q7 reboot spans,
persistence + surfaces) registered in index.md + log.md.
Backend 3382/3382 (+1 rolling-aggregate test), web tsc clean + vitest
627/627, MCP tsc clean + 455 passed. Post-deploy: web deploy +
MCP redeploy (new tool + docs corpus).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
55c898bb9c |
feat(insights): F1 breakdown persistence + daily attribution aggregates (PR2)
Insights expansion PR2 (tasks/insights-expansion-spec.md §F1 "Data & compute
changes"):
- SessionTimeBreakdowns table (PK=TenantId, RK=SessionId): span/reboot/
blocking-app JSON columns, QualityFlags, AttributionVersion,
EspAppsOccupancySeconds kept tri-state (absent column = unknown, never 0).
Written once at the terminal single-writer seam (TableSessionRepository,
next to the counter reconcile + EspBlocking resolution; Succeeded/Failed
only — Incomplete deliberately stores no duration). Deleted with its
session: new deletion-manifest step 16 (PkRkExact; tombstone moves to 19)
plus tenant-offboarding partition wipe — no separate age sweep.
- Self-healing 30d maintenance sweep instead of a one-off backfill (rides
the existing 2h tick + manual parity): backfills pre-deploy sessions,
heals inline-compute misses, folds late-terminating sessions into their
StartedAt-date aggregates, and enforces AttributionVersion purity — a
stale-version breakdown whose events aged out counts as missing rather
than mixing algorithm semantics (truthfulness rule 8). Skips sessions
with a deletion cascade in flight.
- TimeAttributionAggregates table (PK=TenantId or "global",
RK="{date}|{class}"): per enrollment class (whiteglove /
device_preparation / self_deploying / user_driven — never mixed),
median/p75/p90 per segment over the fixed six-segment stack (incl.
unattributed; absent segment = honest 0), clean/flagged/missing counts
disclosed (rule 7), rows written below the n>=20 UI gate on purpose
(the UI needs the n — rule 4). 180d retention alongside UsageMetrics.
- What-if bound: WhatIfSavingSeconds implements the spec formula
max(0, cpEnd - cpEndWithoutX) including the idle gap before X (the ESP
waits for X to start, too); sole-interval fallback = own start. Per-app
aggregate rows gate at >=5 sessions, top 20 by median interval, carry
median/p75 of interval and saving ("up to" wording is the consumer
contract).
Backend 3381/3381 (13 new tests: entity round-trips incl. tri-state,
aggregation bucketing/disclosure/gates, what-if vectors, class precedence,
manifest step-count pins 17/19), solution build clean, web untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
e637317bdb |
feat(insights): F1 schema + time attribution calculator (PR1)
Insights expansion PR1 (tasks/insights-expansion-spec.md §F1, audit §0.5): - AppInstallSummary + AppId / EspBlocking (bool?, tri-state) / AppIdCollision: sentinel-gated columns + mapper + both projections. appId adopted from app events (first-seen wins); a second appId under the same display name flags the name-keyed row as a collision (audit Q3) — in-batch and cross-batch (sticky in the reconcile). Collision rows leave every per-app fleet aggregate (app metrics payload, apps list, app analytics) with a disclosed exclusion count; the fleet DO rollup and per-app session drill-down keep them. - EspBlocking resolution at the session-terminal single-writer seam (TableSessionRepository.UpdateSessionStatusAsync, next to the counter reconcile — covers ingest, admin marks, maintenance sweep, rule engine): joins the session's app rows against the LATEST list-carrying esp_config_detected emission as a positive-evidence set (audit Q2) — listed => true, absent => stays unknown, never false. EspBlockingSets parser handles truncation via the uncapped espTracked*Count fields. - TimeAttributionCalculator (pure, AttributionVersion 1) + SessionTimeBreakdown models: exact partition of the session's authoritative DurationSeconds into device_prep / esp_apps / identity_hello / user_esp / desktop_handoff spans plus an explicit unattributed remainder — invariant sum(spans)+unattributed == wall clock enforced in every test. WhiteGlove = two observation windows in ONE session (part1 anchored on whiteglove_part1_complete, pause never attributed). Per-app blocking intervals from EVENT timestamps (first start -> LAST terminal, audit Q1 retry freeze), occupancy = merged union clipped to esp_apps. Reboot spans from event gaps around lastBootUtc (audit Q7), deduped per boot. QualityFlags: ClockSkewDropped, PartialObservation, BlockingSetUnknown, BlockingSetTruncated, WhiteGloveAnchorsIncomplete. Persistence (SessionTimeBreakdowns) and surfaces follow in PR2/PR3. Backend 3368/3368 (35 new tests: 19 calculator golden fixtures + 16 identity/blocking/aggregate-exclusion), solution build clean, web untouched (new response fields are additive). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
4eac7d042c |
fix(metrics): app duration/rate stats over real installs only — TerminalState column + out-of-order reconcile
Source-data audit (tasks/insights-expansion-spec.md §0.5) verified against production: 17.9% of Succeeded AppInstallSummaries rows carry no duration — 78% are genuine skips (app_install_completed with payload state=Skipped, "Update for X" WinGet policies not applicable) whose state the ingest discarded, the rest are real installs whose start the agent never observed (attach window). Averaging them as 0s understated avg install duration by 18% and distorted the slowest-apps ranking. - AppInstallSummary.TerminalState (Installed/Skipped/Postponed/Error) from the closing event's payload state; sentinel-gated column + mapper + AppsDashboardProjection. Status stays "Succeeded" for compatibility. - MetricsMath.IsSkipTerminalState/HasMeasuredDuration: durations computed over measured rows only; skips leave the failure/success rate (decision 2026-07-26 — a skip is not an install attempt); per-app skipped/unmeasured counts + totals exposed; slowest ranking gates on measured installs. Applied consistently: app metrics payload, apps dashboard (list/analytics/ time series/trend/versions/device models), SLA weekly/snapshot/breach. - StoreAppInstallSummaryAsync: reconcile extracted to internal static ReconcileAppInstallSummaryWithExisting (test seam). Fixes Q4 (terminal batch before started batch left duration permanently unset) and hardens terminal status against late/replayed InProgress batches. - Web: measured-only hints on app detail cards, skip suffix in apps list, fleet-health "measured installs" caption. Backend 3333/3333 (20 new tests), web tsc clean, vitest 627/627. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
f311e8746b |
docs(trust): on-demand log collection in customer-facing claims + tracker dedup tests
The "Collect Logs" feature (
|
||
|
Oliver Kieselbach
|
8a794a1b52 |
fix(rules): cross-tenant GA rule edits no longer misroute to home tenant
A Global Admin editing a gather/analyze rule while viewing a FOREIGN
tenant (override) got a green "saved" banner, but the write landed in
the GA's OWN home tenant: the JWT-scoped rules/{type}/{ruleId} routes
resolve the tenant from the caller's token, and no cross-tenant write
route existed (reads had /api/global/rules/*, writes did not). The
reload re-read the foreign tenant and showed the unchanged rule.
Fix: symmetric cross-tenant WRITE routes mirroring the global reads.
Backend:
- GlobalRulesFunction: PUT/DELETE global/rules/gather/{ruleId} and
global/rules/analyze/{ruleId}; tenant from ?tenantId= (400 if
missing), same 1 MB guard + ValidateScopeAndEmitMode as the JWT
variants, same service calls (toggle-partial/built-in-state/full-edit
semantics are tenant-parameterized and identical).
- Catalog: 4 entries GlobalAdminOnly + TenantScoping.QueryParam — a
write tier, so the delegated-read rescue never fires; only a platform
admin may mutate another tenant's rules. No global CREATE route:
create stays JWT-only and the portal hides Create in override.
Web:
- api.rules.globalGatherRule/globalAnalyzeRule helpers; both rule pages
route save/toggle/mark-failed/notify/delete through a
*RuleWriteUrl(ruleId) switch on isGlobalOverride.
- Analyze template-enable (a create) is blocked in override with a
clear message instead of silently writing to the wrong tenant.
Tests: CrossTenantAccessTests guards the 4 routes are GA-only +
QueryParam and that a GA reaches the named target tenant. Backend
3295/3295, web vitest 627/627, tsc clean.
Note: the old bug planted orphan copies of foreign rules into the GA's
home tenant on each attempted cross-tenant edit; those are not
auto-cleaned.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
6c53dc33e7 |
feat(distress): tenant-visible TpmPssUnsupported devices + one-shot bell
TpmPssUnsupported distress reports (TPM cannot RSA-PSS-sign, Schannel drops
the client cert on Win11 25H2+, agent can never authenticate) were operator-
only, yet the remediation is tenant-side: TPM firmware update or device
replacement.
- New GET audit/tpm-pss-unsupported (MemberRead), aggregated per serial,
mirroring audit/device-not-registered
- Bell notification "tpm_pss_unsupported" (Admin audience) once per
(tenant, serial); dedup via the existing rejection tracker table with a
disjoint "tpmpss|{serial}" row-key space; no webhook by design
- Web: TpmIncompatibleDevicesInsights panel in the Hardware Whitelist
section; renders only when a device is affected (no permanent card)
No agent change, no ConfigVersion bump. Backend tests 3287/3287, tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
1f6efca95b |
fix(backend): stamp DiagnosticsBlobName from on-demand server_action_executed events
Session 8e4cc4ae: two "Collect Logs" round trips completed on the agent (both
ZIPs landed in hosted storage) but the portal never flipped to Download and
the button hung at "Collecting..." — the ServerActionDispatcher confirms an
on-demand upload only via server_action_executed(actionType=request_diagnostics,
blobName=...) and emits NO diagnostics_uploaded (only the terminal upload path
does), so UpdateSessionDiagnosticsBlobAsync never ran.
The ingest classification now treats such an executed event as the upload
confirmation (a real diagnostics_uploaded in the same batch still wins — it
carries destination). Because the executed event has no destination field,
InferDiagnosticsDestination derives "Hosted" from the backend-built
"{tenantId}/{filename}" blob-name shape (CustomerSas blobs are bare
filenames); without it the download route would look in the customer's
container for a hosted blob.
Backend-only — works with the already-deployed agent fleet, no agent release.
Backend suite 3274/3274.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
9ec8644be2 |
feat(settings): operator read-only settings view with real values
Operators previously saw editable settings forms bound to hardcoded defaults
whose Save always 403'd. Now they see the tenant's real configuration
read-only, with no save affordances at all.
Backend:
- GET config/{tenantId}: TenantAdminOrGlobalReader -> MemberRead. Safe because
the handler redacts by default (CanViewSecrets: only GA or own-tenant Admin
see secrets); Operators/Viewers receive RedactedCopyForReader().
- POST diagnostics/files: TenantAdminOrGA -> TenantAdminOrOperator (submitting
logs is a troubleshooting action; Viewer and read-only GlobalReader excluded).
- Tests: policy pins updated; new middleware tests (own-tenant Operator allowed
+ redacted, cross-tenant Operator 403); CanViewSecrets theory for
Operator/Viewer.
Web:
- TenantConfigContext loads the full config for all settings users (403
fallback to feature-flags minimal config for deploy-order safety) and
exposes canEditConfig; saveConfiguration guards on it.
- New ReadOnlyFieldset wraps section content (native fieldset disable, values
stay copyable); SaveResetBar is not rendered at all for read-only viewers.
Applied to Contact, SLA Targets, Hardware Whitelist (+ rejection-insights
toggle), Notifications, Agent Settings, Agent Analyzers, Diagnostics, Data
Management.
- Hidden entirely from Operators (in-page admin-only card + sidebar sub-item
gate): Autopilot Validation, Access Management, Offboarding, Unrestricted
Mode. GlobalReader sidebar parity kept via hasGlobalScope.
Tests: backend 3257/3257, web vitest 627/627, tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
2fee50276e |
feat(backend): operator-tier session actions + diagnostics groundwork for on-demand log collection
- New EndpointPolicy.TenantAdminOrOperator (Admin/Operator/GA, no Viewer) with
middleware evaluator; POST sessions/{sessionId}/actions moves to this tier so
troubleshooting operators can queue request_diagnostics ("Collect Logs").
- QueueSessionActionFunction re-gates per type: terminate_session and
rotate_config stay Admin/GA-only (403 for operators).
- GET config/{tenantId}/feature-flags exposes diagnosticsUploadConfigured so
members below Admin can see whether an on-demand upload would work.
- Offboarding cascade: hosted diagnostics deletion now sweeps the session
prefix {tenantId}/AgentDiagnostics-{sessionId}- (plus the manifest-named
blob) so on-demand packages overwritten on the Sessions row by the terminal
package no longer survive deletion. CustomerSas branch unchanged.
- Notification dismiss routes deliberately NOT migrated: dismissal is still
tenant-shared; operators would clear admin-tier notifications they cannot see.
Tests: middleware role matrix on the actions route, per-type gate matrix,
feature-flag matrix, prefix-sweep + real hosted-branch cascade tests.
Backend suite 3253/3253.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
a44e37a7c2 |
fix(metrics): keep full failure-reason text for fleet-health display
BuildFleetFailureReasons truncated each reason to a 50-char prefix + "..." and used that as both the grouping key AND the display value. The full text never reached the UI, so its tooltip/tap-to-expand had nothing to reveal and near-identical rows were indistinguishable past the cut. Group by the 50-char prefix as before (variants still collapse into one row), but carry the longest full reason as the display value. The UI truncates and expands it on demand, so the complete text is now available end-to-end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
71eda93e8b |
fix(metrics): app-install failure rate over finished installs only
Second half of the terminal-only rate convention (follows
|
||
|
Oliver Kieselbach
|
1cfc32c9aa |
fix(metrics): success rate over finished enrollments only (geo + fleet health)
Customer feedback: the geographic performance pane showed a poor success rate for a site mid-rollout because in-progress/pending sessions counted against the rate. Success rate is an outcome quota — align Geo and Fleet Health with the SLA convention already used by dashboard stats, usage snapshots and the fleet rollup: Succeeded / (Succeeded + Failed). Incomplete (terminal, non-failure) stays excluded. - Geo: per-location SuccessRate over finished sessions; table badge and map tooltip render an em dash instead of a red 0% when nothing has finished yet - Fleet Health: Stats.SuccessRate terminal-based; card shows "X of Y finished enrollments" and a neutral em dash state; per-model health gets a Failed count (additive wire field) so the client rate excludes in-flight devices; TopFailingModels FailureRate now failed/finished - Tests: FleetHealthPayloadTests flipped to the new convention plus zero-finished, model-failed and in-flight-dilution cases; new GeographicMetricsSuccessRateTests pin the geo semantics Backend 3216/3216, web tsc clean, vitest 602/602. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
540f543834 |
feat(agent): extend historic IME-replay suppression to app events + app aggregation
Part 2 of the eaf3d8c4 fix (follows |
||
|
Oliver Kieselbach
|
088c3b0479 |
fix(agent): suppress historic IME-log script replay + timeline-consistent durations
Session eaf3d8c4: a previous enrollment's IME logs survived on disk (dated 2026-07-16, 7 days old) and the agent replayed them at startup — 156 phantom script_completed events in a 7-minute session, each showing ~170h durations (clock-clamped completion minus RAW stale start), plus an immediateUpload flood (3x ingress_backpressure) and RemediationScriptCount inflated to 156. Agent: - Script events whose source line is >24h stale are suppressed entirely; a one-shot historic_script_replay_detected Info event (new EventType) dates the replayed window via earliestRejectedSourceTimestamp. Future-skew rejections and clock-only fallbacks still emit. - durationSeconds now uses a timeline-consistent pair: on rejected-timestamp emissions the raw source completion pairs with the raw start (true runtime instead of cross-timeline garbage). 24h plausibility backstop omits implausible values; same consistency for script_timeout_suspected. - Stale pending platform-script slots (>24h older than the current start line) are discarded so an old orphaned start can't pair with a fresh run (extracted HandlePlatformScriptStarted as internal test seam). Web: partitionHistoricScriptEvents hides legacy-agent replay events (muted "N historic executions hidden" note) and the reducer blanks >24h durations — covers sessions recorded before the agent fix rolls out. Backend: IsHistoricScriptReplay guard keeps replayed events from inflating Platform-/RemediationScriptCount during the rollout window. Registered historic_script_replay_detected in Constants.EventTypes, MCP resource-catalog (script_events) + KEY_EVENT_TYPES, gather-rules eventTypes. Tests: 8 agent (suppression/one-shot/raw-pair/backstop/slot-hardening), 8 web (partition/blanking), 10 backend (predicate). Agent 2026, backend 3210, web 595, MCP 449 all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
ad3632e9fb |
fix(emergency-break): survivable self-destruct, reliable break report, ops visibility
Incident 2026-07-22: the 48h emergency break left exe + loaded DLLs orphaned forever — a second agent instance held the images while the cleanup script (waiting on the launcher PID only) force-deleted marker + state around the locks AFTER removing the Scheduled Task, destroying the retry path. Agent — cleanup script (LIFE-F7): - wait for ALL agent processes by name (PID-reuse-proof), then force-kill stragglers; single-flight lock file serializes concurrent cleanup scripts - probe-rename (GUID target) BEFORE task removal; any failure renames back and aborts with task + marker + files intact for a next-boot retry; the destructive Remove-Item-on-original fallback is gone - script generation extracted to BuildCleanupScript, pinned by 13 tests; all 4 variants parser-validated + E2E-smoked (happy path, locked tree) Agent — break report delivery: - EmergencyReporter counts DELIVERED reports: reservation rolls back on failure (mark-before-send permanently suppressed a key after its first transport failure); optional in-reservation retries - break path: NIC wait <=15s, 3 attempts x 15s timeout (survives Functions cold starts observed as >5s 503s), 75s overall bound Backend: - backend-materialized events (agent_emergency_break, session_timeout) now upsert the cross-session EventType index — they were invisible to every search-by-eventType surface (portal + MCP) - new AgentEmergencyBreak ops event (Agent/Warning, once per session) so operators see silently-lost agents; dual-registered in OPS_EVENT_TYPES - materialization extracted to a static seam, covered by 4 new tests Agent 1963/1963, Backend 3198/3198, tsc clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
b2d2d37c6f |
feat(realmjoin): detect aborted RJ ESP (missing 110) and release the completion gate
An interactive logon during the RealmJoin first deployment (phase 100/101)
makes RJ reclassify the run as a secondary-user deployment, whose branch
writes 200/210 and never CompletedFirstDeployment (110) — no crash, no
reboot request. The V2 completion gate waited for exactly 110, so these
sessions hung until the 60-min hard timeout (session 224b2087; siblings
558bf067/3cc18724 delayed ~50 min the same way).
- promote realmjoin_phase_changed to a typed RealmJoinPhaseChanged signal
(dual-emitted by the adapter) so RealmJoinFacts.LastDeploymentPhase is
persisted across agent restarts
- abort rule: prev in {100,101} and current in {200,210} with no outcome
yet -> emit realmjoin_first_deployment_incomplete (Warning), cancel the
hard-timeout deadline, release the gate with outcome
FirstDeploymentIncomplete; also evaluated on the RealmJoinDetected
replay path (restart swallows the transition)
- deliberately NOT triggered by 101->0 (RJ service restart may still
complete the first deployment) nor by a first observation of 210
(session 6f1959c0: RJ already completed before agent boot)
- fix realmjoin_timeout always claiming "last phase: 0" - the persisted
phase now feeds the message
- backend EnrollmentTimeoutClassifier counts the new event as a gate
terminal; MCP resource catalog dual-synced
- CompleteIfDeferredOrBookkeep widened to a leading-effect list
Verified by replaying session 224b2087's signal log: gate releases at
15:05:17 (101->200) instead of hanging until the 15:54 deadline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
e0ad51b965 |
refactor(urls): one guard-enforced URL registry per component, agent downloads move to the alias
Every well-known host -- own subdomains, both blob accounts, the API origin, Graph, Entra login -- now lives in exactly one registry per component: Constants.cs (*BaseUrl) for C#, utils/config.ts for the web app, src/config.ts for the MCP server. lib/hostRouting.ts derives its hostnames from the registry and middleware.ts imports from it, retiring what turned out to be a fourth independent copy of the www/portal host pair. Three new guard tests (backend xUnit, web vitest, MCP vitest) scan non-test source and fail on any enforced host outside its registry. This is the lesson of the EU cutover: the registry existed then too, but LatestVersionsService (feeds the portal's version badge) and ValidateBootstrapCodeFunction carried their own literal copies of the legacy blob host and were missed -- kept working only by the fail-soft legacy mirror in build-agent.yml, whose silent failure would have pinned badge and self-update to stale bits with a green release. Both guards proved themselves on their first run, flagging the middleware host copy and a literal in an MCP tool description. Test files stay exempt on purpose: literals there are independent oracles, and ValidateBootstrapCodeResponseShapeTests now pins the full alias URL against the Constants-built one so registry changes must be acknowledged consciously. The two legacy readers plus the agent SelfUpdater now use Constants.AgentDownloadBaseUrl (download.autopilotmonitor.com/agent); the alias was verified live for version.json, version-v2.json and both ZIPs before the switch. The portal's bootstrap allow-list accepts the alias AND the legacy host (AGENT_DOWNLOAD_HOSTNAMES) for already-deployed customer scripts, which imposes a deploy order: web before or with backend. The legacy keepalive probes and the release mirror stay untouched -- they exist for exactly this transition. Also: the bootstrap short-link now targets www directly instead of riding the registrar's apex 301, and the welcome-mail docs button points at docs. instead of bouncing through www./docs. Suites: backend 3194, agent core 1925, DecisionCore 474, web 582, MCP 449 -- all green; next build verified (it, not tsc, proves the next.config.ts registry import). Docs: docs/url-registry.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
bbb55034e3 |
feat(versioning): one version contract for agent, backend and MCP
All three components now report <major>.<minor>.<build> plus a 7-char commit and a build timestamp. The backend had been pinned at a hand-written 1.0.0 since inception and starts at 1.5.1000; the MCP server's package.json version was bumped by hand and starts at 1.5.300, and it now reports its own commit on /health instead of only the documentation bundle's. The build number keeps coming from a counter blob reserved before the build with ETag-CAS, but there is now one implementation instead of two copies: .github/scripts/Request-BuildNumber.ps1, called by all three workflows and by the local build.ps1. It lives under .github/ because scripts/Deployment/ is gitignored (those scripts carry SAS tokens) and would therefore be absent from a CI checkout; scripts/Build/ and scripts/Release/ are shadowed by the [Bb]uild/ and [Rr]elease/ artifact ignores. The counter cannot be replaced by the published manifest, which is the obvious simplification to reach for: version.json is written only on stable publish, so two consecutive dev builds would derive the same number and the second would overwrite the first's artifact. git rev-list --count fails the same way, two builds of one commit collide. Both deploy workflows now poll their health endpoint until it reports exactly the version and commit just deployed, and publish versions/<component>.json only then. A green deploy step alone means the package was accepted, not that it is being served -- so a manifest that disagrees with the live endpoint now means the running instance drifted (rollback, stale revision) rather than a pipeline that stopped halfway. The agent manifest gains the same commit and buildUtc fields additively; the bootstrap reads it by field name, so older installers ignore them, and sha256/bootstrapVersion stay untouched. Requires the versions container plus AZURE_BLOB_VERSIONS_SAS_TOKEN and the two seeded counters (backend 999, mcp 299) before the first deploy run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
71d2c0e47f |
fix(maintenance): remove the time-window auto-block, it never measured data volume
GetExcessiveDataSendersAsync gated on LastEventAt - StartedAt alone. No event count, no bytes, no rate — the name asserted something the code never checked. What it actually selected was every enrollment session that spanned a night, which for Autopilot is the ordinary case: device unboxed in the evening, ESP runs, machine powered off, user signs in the next morning. The per-device alerts from the previous commit made this visible. All three devices blocked at the time of the analysis were false positives: CND3320949 / 64ed21e8 23 events over 15.5h (1.5 events/hour) 5CG4303CPT / db5751bc 192 events over 16.8h (blocked mid Account Setup 1/5) PF5YANFB / 2069de87 781 events over 31.5h 14 such blocks in 30 days across 9 customer tenants. Over the same window the event-count detector fired 5 times and was right every time (2742, 2714, 2712, 2506 events). Each false block costs a live enrollment 8 hours of telemetry. Repairing it rather than removing it does not pay: once a volume condition is added, "many events AND long span" is a subset of "many events", which is what the count detector already does. Volume is the quantity that costs anything; a 23-event session spanning 15 hours costs nothing. So the detector, its two admin settings, the ExcessiveDataBlocked ops event and the portal section go, and the event-count path is the only automatic block left. MaxSessionWindowHours and MaintenanceBlockDurationHours are deleted rather than deprecated: the config row is written with Replace, so the columns drop out on the next portal save, and nothing reads them meanwhile. A comment on the surviving detector records why there is no second one, so it does not get rebuilt a year from now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
3b2f5ed46d |
feat(runaway-detection): name the device in the time-window alert, link it in the block list
The two runaway detectors emitted alerts of unequal usefulness. The event-count
path fires per session and carries sessionId + serialNumber, so the Ops Events
detail modal can offer View session / Block / Kill. The time-window path
aggregated a whole tenant into one event with {devicesBlocked, windowHours} —
no device, no session, no shortcuts, on precisely the alert where an operator
wants to act. It now writes one event per blocked device with the same payload
shape; the aggregate count stays in the audit entry, where a sum is the useful
view.
The Active Blocks table showed the serial as plain text although the API already
returns BlockedSessionIds. Maintenance blocks now link the serial to the session
that triggered them; manual whole-device blocks have no session and stay text.
Two defects in the event-count path, found while comparing the detectors:
- No isBlocked guard, so the auto-action re-blocked an already-blocked device and
overwrote BlockedAt/UnblockAt/Reason of the existing block — usually the
time-window block on the same session, whose reason then vanished from the list.
- No status filter, so a session that finished days ago with a high EventCount
still cost its device a 24h block. The fix needed a layer below the obvious one:
the runaway query never selected Status, and SessionStatus.InProgress is
ordinal 0, so a guard on the unpopulated field would have been silently
always-true. ParseStatusForAutoAction is stricter than MapToSessionSummary --
Enum.TryParse maps "0" onto InProgress and ORs comma-separated names into an
unrelated member, both of which would open the gate.
Not addressed, and deliberately: a block is session-scoped and stops the upload,
so whichever detector fires first freezes the other for that session. With
MaxSessionWindowHours=12 the time detector wins in practice. That is correct
behaviour, not a defect; merging the detectors would be its own change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
757427ffea |
chore(build): clear the compiler and xUnit analyzer warnings CI annotates
The CI test job surfaced 10 annotations, but GitHub caps them at 10 per job — a full rebuild showed 31 nullable warnings plus 26 xUnit analyzer warnings. Production code: - CertificateValidator: X509Certificate2(byte[]) is obsolete (SYSLIB0057) -> X509CertificateLoader.LoadCertificate. - AuthenticationMiddleware: IsValidTenantId now carries [NotNullWhen(true)], so the guard above BuildTenantAuthority is visible to flow analysis. - AgentEndpointMigrationRules: the out params are genuinely nullable on the false path; netstandard2.0 has no usable [NotNullWhen], so the contract is documented instead. - Constants.ClassifyEspAppsFailure: errorCode is optional by design (the "no HRESULT" branch is the timeout fallback) - signature says so now. Tests: nullable annotations on helper signatures, Assert.Single/DoesNotContain overloads instead of pre-filtering with Where, and [InlineData(null)] theories declare nullable parameters. GatherRule helpers now seed Parameters with the model default (empty) rather than null, matching a deserialized rule. No behaviour change. 5616 tests pass; solution builds warning-free apart from the two pre-existing NU1510 pruning notices on the pinned security packages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
b56c774103 |
fix(logging): strip Unicode line separators in LogSanitizer
The sanitizer matched \p{Cc}, which covers the C0/C1 control ranges but not
U+2028 LINE SEPARATOR or U+2029 PARAGRAPH SEPARATOR - those are categories
Zl/Zp. A log viewer, a JSON-lines consumer or any JavaScript-side reader still
breaks a line on them, so they were a residual log-forging vector.
Matching \p{Zl} and \p{Zp} rather than listing escapes: the two categories
contain exactly those two characters, and it keeps the source free of
invisible literals.
Verified the test is not vacuous - the old pattern left "a<U+2028>b" as "a b",
the new one yields "ab".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
68767f436f |
fix(config): validate ContactEmail and seed it conditionally
The tenant contact address had no server-side validation, so a direct API caller could store any string in the field service notices are sent to. UpdateTenantConfiguration now rejects non-addresses, recipient lists (a comma would silently widen who receives those notices), display-name forms and control characters (mail-header injection), and normalizes what it stores. Empty stays valid - having no contact address is a legitimate state. The seed path had a worse problem than missing validation. It was a read-modify-write of the whole model through SaveTenantConfigurationAsync, which upserts with TableUpdateMode.Replace and no ETag: a concurrent portal save was clobbered by the seeder's stale snapshot of EVERY field, not just ContactEmail. It also wrote straight through IConfigRepository, bypassing the TenantConfigurationService cache, so a seeded address stayed invisible for the full 5-minute TTL. TrySeedTenantContactEmailAsync now writes ETag-conditionally in Merge mode - one property, and a 412 means someone else wrote the row so the seed abandons rather than overwriting the winner. A lost seed is recoverable on the next maintenance run; a lost admin edit is not. The same seed logic existed twice (preview notification-email save and the maintenance backfill). Both now call a single owner on TenantConfigurationService, which holds the never-overwrite invariant, the cache eviction and the fail-soft guarantee. Fail-soft lives in the service rather than the repository so it holds for any implementation. Tests cover the validator, the service contract, and the repository against a mocked TableClient: Merge mode, the ETag actually passed, only ContactEmail in the patch, the 412 race, 404, storage faults. Plus a regression pin that SaveTenantConfigurationAsync is never called from the seed path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
c7b06be8b7 |
fix(ratelimit): key the user throttle on CallerId, not the UPN
Follow-up to
|
||
|
Oliver Kieselbach
|
479247cdae |
fix(ratelimit): stop agent fleet sharing one anonymous user bucket
Device and anonymous routes published the "anonymous" placeholder as RequestContext.UserPrincipalName. UserRateLimitMiddleware skips only on an EMPTY UPN, so every agent of every tenant was bucketed under user_ratelimit_anonymous and throttled against the 120/min per-user limit. On 2026-07-21 21:00-21:30 UTC that rejected 1821 agent requests with 429 (/api/agent/telemetry, config, register-session, error); the 7-day trend shows it recurring and growing with fleet size. - PolicyEnforcementMiddleware: RequestContext.UserPrincipalName carries a genuine JWT UPN or nothing. decision.UserIdentifier keeps the placeholder for log lines. - Device/bootstrap routes already hold their own limit in SecurityValidator (per cert thumbprint / per bootstrap token), so they lose nothing. - stats/platform and diagnostics/download had NO limiter of their own and were riding on the accidental shared bucket. Both now limit per trusted client IP (60/min and 30/min) via ClientIpExtractor, matching the HealthCheck pattern. - 13 new tests pin the empty-UPN contract across all 12 device/anonymous routes, plus a guard that an authenticated caller still surfaces its real UPN. Solution suite 5552 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
b6ba87f072 |
fix(deps): pin patched DataProtection + Cryptography.Xml to clear NuGet advisories
Microsoft.Identity.Web 4.3.0 pulls Microsoft.AspNetCore.DataProtection and System.Security.Cryptography.Xml transitively at 10.0.0, which carry known advisories (GHSA-9mv3-2cwr-p262 critical on DataProtection; GHSA-37gx-xxp4-5rgx and siblings, high, on Cryptography.Xml). Add explicit references pinning both to the patched 10.0.10 servicing build (matching the existing JwtBearer pin) — the NuGet equivalent of the npm overrides used for the web/MCP alerts. Pinned in the production Functions project so the fix flows to the deployed artifact and to Functions.Tests via project reference. `dotnet list package --vulnerable` now reports 0 across every project; build clean; 3110 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
fa640ebe61 |
fix(security): resolve CodeQL log-forging findings and suppress by-design queries
Bucket B (real fixes) + Bucket A (config) of the CodeQL alert triage. Log-forging (cs/log-forging, 8 sinks): add Helpers/LogSanitizer that strips CR/LF and other control characters, and apply it to the logged copy of every user-controlled value in AuthenticationMiddleware, PolicyEnforcementMiddleware (Invoke + DecideAsync), McpQuotaEnforcementMiddleware and TableUserUsageRepository. Only the logged value is sanitized — routing, authorization and storage keys are untouched, so tracing fidelity for legitimate values (paths, UPNs, GUIDs) is preserved. Structured logging already isolated the fields; this closes the CRLF vector as defense-in-depth and clears the query. Three genuine JS/TS fixes: - guardValidation.ts: replace the no-op `.replace(/%/g, "%")` with a real escapeRegExp() — the author intended to escape; this also closes the latent metacharacter gap (js/identity-replacement). - access-guard.ts: use a `%s` format argument instead of interpolating the UPN into the console.error format string (js/tainted-format-string). - docs-corpus.ts: strip HTML tags with a loop-until-stable stripHtmlTags() so the sanitizer is robust and recognized as complete (js/incomplete-multi-char-sanitization). CodeQL config: filter the two queries that fire structurally on this codebase, where the flagged behavior is the feature — cs/exposure-of-sensitive-information (audit/notification logs legitimately record the actor UPN / recipient email) and cs/cleartext-storage-of-sensitive-information (rate limiters key on the trusted client IP). Default suites stay active; genuine one-off false positives from other queries are dismissed individually so those queries stay sharp. Tests: Backend xUnit 3110 (+9 LogSanitizer), MCP vitest 19 (+1 tag-strip), Web vitest 49, all tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
72be8ac89f |
feat(config): tenant contact address, seeded once from the preview email
Enforcement and service notices need somewhere to go. Until now the only address on file was the Private Preview notification email, which is onboarding data in the PreviewWhitelist table and not something a tenant can maintain. Adds TenantConfiguration.ContactEmail, owned and edited by the tenant's own admins under Settings -> Tenant -> Contact: - Seeded once when a preview notification address is saved, and only while ContactEmail is still empty. It is never re-synced, so an edit in the portal cannot be overwritten later by the onboarding source. - Backfilled for tenants onboarded before the field existed, as a manual-trigger maintenance task alongside the other backfills. It skips any tenant that already has an address, so it converges to a no-op and can be dropped once everyone is covered. - Surfaced read-only in Admin -> Tenants next to the preview address, so an operator who needs to warn a tenant does not have to read the table. Purpose is stated where it is entered and in the Privacy Policy: service matters only - a technical problem, a security matter, or a change needing an administrator's attention. Never marketing, never shared. The Terms already say a warning is sent "where a tenant contact address is on file"; this is the field that makes that promise meaningful. Wired through the full tenant-config round-trip (types, context state, hydrate, save payload, reset, provider) so a save from any other section cannot clobber it. Verified: solution builds clean, 3101 backend tests and 557 web tests pass, tsc --noEmit clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
a654c7309e |
feat(agent): attribute mid-ESP coalesced reboots to MDM policy URIs (second sign-in detection)
Device-assigned MDM policies whose CSP URIs match the OS reboot-required
catalog force a coalesced reboot at the end of ESP DeviceSetup - the user
sees an unexpected restart and a second sign-in before AccountSetup.
Invisible in the Intune console; admins simply lived with it. Now attributed
and actionable (PMPC research).
- MdmRebootPolicyTracker (+host): watches DeviceManagement-Enterprise-
Diagnostics-Provider/Admin EventID 2800, emits one mdm_policy_reboot_required
per reboot-forcing URI (Warning, ImmediateUpload; tolerant EventData-first
URI extraction with regex fallback; fail-soft OMADM RebootRequired probe).
Cross-restart RecordId watermark (mdm-reboot-watermark.json) - mandatory,
the coalesced reboot restarts the agent and the backfill re-reads the log.
- ShellCoreTracker: RebootCoalescing branch on the already-watched 62407
stream -> esp_reboot_coalescing corroboration event; deliberate deviation:
emits on backfill too (live emit races the reboot kill).
- ANALYZE-ESP-005 advisory (warning): names the first URI via {{rebootUri}},
remediation = reassign profiles device -> user groups; +20 confidence on
coalescing corroboration, +10 on >=2 URIs.
- Config: Collectors.MdmRebootPolicyWatcherEnabled (default ON, internal
steering only - agent-native collector, NOT an IAgentAnalyzer, no admin UI),
MdmRebootPolicyBackfillLookbackMinutes=60; ConfigVersion 35.
- Event types deliberately distinct from system_reboot_detected so the
backend RebootCount (exact-string match, 2 sites) stays uninflated.
- Docs: docs/agent/mdm-reboot-coalescing.md (+index/log). Exact 2800/62407
message texts are unverified until E2E VM validation (tolerant matching).
Tests: Agent.V2.Core 1868/1868 (21 new), Functions 3099/3099, MCP 400 passed
(event-types drift), solution build 0 errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
6d1ff8d7f0 |
fix: code-review follow-ups (watcher dispose race, revoke-enforcement tests, scan guard)
- ConsoleBypassWatcher: re-check _disposed after watcher.Start() — a Dispose() racing the background arm could leave an armed WMI subscription no code path could ever stop (leaked until process exit). - SignalRContext: collapse rapid accessRevoked pushes into one restart via an in-flight guard — a second start() on a non-Disconnected connection throws and burns the retry budget. - config/all: negative-cache managed ids a rescue scan proved missing (5 min TTL) — one stale grant (offboarded tenant) no longer makes every delegated config/all call pay a full table scan. - DelegatedAdmin/TenantGroup functions: extract testable revoke-enforcement cores (RevokeCoreAsync, SetEnabledCoreAsync, UnassignCoreAsync, DeleteGroupCoreAsync, ValidateGrantRequest) and cover the security flows: 404-on-missing-row writes no audit row and fires no disconnect; real revoke/disable/unassign/group-delete always cuts live streams; grant validation is fail-closed (GUID tenantId, recognized roles only). - dashboard/page.tsx: drop unused next/link import left by the banner removal. Tests: backend 3099 passed, web 557 passed, agent ConsoleBypassWatcher suite passed, tsc clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
da958b5398 |
fix(backend): business timestamps survive storage migrations (OccurredUtc + RowKey decode)
The 2026-07-18 storage migration reset every row's system Timestamp, exposing that AuditLogs, OpsEvents and Events displayed, filtered and retention-swept on it (custom "Timestamp" writes were silent no-ops — reserved system property). All migrated rows read as 2026-07-18. - New Helpers/BusinessTimestamp: RowKey decoders (audit !revtick_guid, ops bare revtick, event yyyyMMddHHmmssfff_seq), RowKey-range OData clause builders (tick-exact, with the mandatory RowKey lt '"' guard so no lower bound can ever match legacy bare-GUID audit rows), and dual DateTimeOffset/DateTime column reads. - Writes in all three tables now persist OccurredUtc (custom column, copied verbatim by future migrations); no-op ["Timestamp"] removed. - Read mappers resolve OccurredUtc -> RowKey decode -> system Timestamp; GetEarliestSessionEventTimestampAsync uses the same resolution so StartedAt/Duration reconciliation can no longer absorb the migration date on migrated sessions. - Date-window filters and retention moved off the system Timestamp onto RowKey ranges; ops retention loses its full-table scan and the post-migration mass-delete cliff; audit retention runs a temporary second pass for legacy GUID rows (TODO: remove after 2027-03-01). - New GlobalAdmin backfill endpoint POST /api/maintenance/ backfill-occurred-utc (dry-run by default, batched, idempotent MERGE of only OccurredUtc, table=audit|ops) — Events rows are deliberately never mutated; their read path decodes the RowKey instead. - Tests: decoders, clause boundary proofs incl. GUID-safety pins, mapper precedence, backfill row decisions (Functions.Tests 3078). - Docs: docs/backend/business-timestamps.md (system Timestamp is never authoritative; migration checklist rule). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
21dd35f787 |
feat(backend,shared): gather-rule phase scoping + emit-mode fields, custom-rule toggle wipe fix
Step 0-2 of the gather-rule anti-spam plan:
- GatherRuleService.UpdateRuleAsync: toggle-style partial PUTs ({enabled,
isBuiltIn, isCommunity}) previously full-replaced the stored custom rule and
wiped Title/Target/Trigger/Parameters. Partials now merge Enabled into the
existing row; full edit payloads preserve the original CreatedAt.
- GatherRule: new opt-in fields ActivePhases (run only during listed phases),
ActiveFromPhase (sticky latch from a phase onwards), EmitMode (always |
on_change). Absent fields = legacy behavior; schema updated accordingly.
- TableStorageService.Rules: Store+Map for the three columns (absent column /
stored-empty -> null); MapToGatherRule internal for roundtrip tests.
- ContentEquivalent covers the new fields, null-tolerant for ActivePhases so
pre-existing rows never diff against seeds (no reseed churn).
- GatherRulesFunction.ValidateScopeAndEmitMode: canonical EnrollmentPhase name
list Start..Complete (Unknown/Failed and numeric tokens rejected), mutual
exclusion of the two scope fields, emitMode whitelist; 400 on violation.
- ConfigVersion 33 -> 34 (33 was taken by MigrateToApiBaseUrl).
Tests: GatherRuleUpdatePartialMergeTests (merge/replace/unknown-rule/built-in
path), GatherRuleScopeFieldsTests (Store<->Map roundtrip incl. legacy rows,
ContentEquivalent matrix, validation accept/reject). Functions.Tests 3029/3029.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
9933c039ed |
refactor(shared,backend): MigrateToApiBaseUrl nullable, drop null-forgiving operator
null = no migration is the documented contract and the agent already assigns null when stripping the field from the on-disk cache; annotate the property as string? so the backend assignment needs no ! suppression. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
ef67dec263 |
fix(backend): system-health URLs post EU-migration + GA endpoint visibility
- Agent Binaries check + ops watcher now probe the canonical download alias (download.autopilotmonitor.com) AND the legacy blob keepalive; alias failure is unhealthy, legacy failure a warning - McpServerBaseUrl fallback moved from the stale kindwave WEU ACA FQDN to the stable custom domain mcp.autopilotmonitor.com (CNAME to the new GWC instance) - health/detailed and health/mcp include the probed endpoint URLs in check details for Global Admins only (storage table endpoint, WEBSITE_HOSTNAME, alias/legacy download URLs, MCP server URL) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
b005c20598 |
feat(agent,backend,web): config-channel agent endpoint migration (MigrateToApiBaseUrl)
Re-home agents to a new API base URL through the agent/config response (backend move or per-tenant EU->US region move) - the lesson from the 2026-07-18 WEU->GWC cutover, where the compiled-in API hostname hard-cut every in-flight agent (no DNS indirection possible: mTLS breaks behind a proxy, Flex custom-domain TLS is broken). Backend (ConfigVersion 33): - AdminConfiguration.AgentMigrateApiBaseUrl (global) + AgentMigrateTenantOverridesJson (tenantId->URL; empty value pins a tenant against the global target; override wins) - GetAgentConfigFunction.ResolveMigrateTarget serves the validated target; AgentMigrateServed / AgentMigrateRejected warnings as delivery evidence - TableConfigRepository Store+Map; admin portal Global Settings round-trip Agent (kill-switch-mirrored hardening): - Shared AgentEndpointMigrationRules: https-only, *.azurewebsites.net on a label boundary, no path/port/userinfo - enforced on BOTH sides - Live fetch only: RemoteConfigService cache strips the field on write AND read; EndpointMigration.ResolveTarget re-checks the fetch outcome - AgentRuntimeConfig.Resolve returns before merge/integrity/bootstrap cleanup on a pending migration (no spurious IntegrityCheckFailed against a stale migration stub), Program.RunAgent rebuilds Phase 3+4 against the new URL - single hop, kill signal wins before and after the hop Tests: AgentEndpointMigrationTests (backend rules + resolution), EndpointMigrationTests + cache-hardening strip (agent). Full suites green (Functions 2992, Agent Core 1822, vitest 557, tsc silent). Doc: docs/agent/endpoint-migration.md Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
2d5f87140f |
chore: retarget SignalR quota watcher to Standard S1 limits + CI build hygiene
- SignalR quota watcher/health card: defaults 20/20k (Free tier) -> 1000/1M (Standard S1 x1 unit); settings renamed SignalRFreeTier* -> SignalRConnectionLimit/ SignalRDailyMessageLimit (no Azure overrides were set); all operator-facing texts tier-neutral (connections 429 hard, message overage is billed not throttled) - web: drop obsolete edge runtime from opengraph/twitter/apple-icon image routes (next/og runs on Node since 14; images now statically generated at build) - web: update caniuse-lite browser db (silences 6-months-old Browserslist warning) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
e2d711dc30 | Merge branch 'migration/net10' | ||
|
Oliver Kieselbach
|
417983d688 |
feat(sessions): persist + surface which device-validation path admitted the device
The backend admits agent sessions via up to three device-validation paths (Autopilot S/N lookup, Corporate Identifier, Device Association once DevPrep becomes a hard gate) plus the pre-MDM Bootstrap token — but the winning path was only returned to the agent, never stored, so operators could not see HOW a device was accepted. - SessionRegistration.ValidatedBy: server-populated in ProcessRegisterAsync from the SecurityValidationResult (agent-sent values are overwritten unconditionally) - Sessions row + SessionsIndex mirror + MapToSessionSummary carry ValidatedBy; latest non-Unknown validation wins (Bootstrap sessions upgrade to the cert-path validator on re-register), Unknown preserves the earlier value (tenants with device validation off) - Web: "Device Validation" item on the session detail Session Info card (hidden for legacy sessions / validation-off tenants) - MCP: get_session overview exposes validatedBy; raw endpoints carry it verbatim automatically - Tests: SessionIndexProjectionTests pin the new index column Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
d02fa454a7 |
fix(status): honest verdicts across all misclassification audit findings
Misclassification audit 2026-07-16 (1000 non-succeeded sessions, 22 deep
dives) surfaced six systematic misdeclaration sources. This implements
all of them:
1. DecisionCore: HelloNeverObservedPromote in the esp-exit advisory
handler — when the Hello policy read never committed and no wizard
was ever seen, hello_resolution is structurally unresolvable; promote
to AwaitingHello + hello_safety instead of hard-failing a provably
finished enrollment (sessions 2dac8298/7635ab18). Placed after the
enforcement-progress re-arm so active installs still win; advisory-
anchor variant stays exempt.
2. Ingest: both max-lifetime shapes (enrollment_failed/agent_timeout and
agent_shutting_down/max_lifetime) now route through the
EnrollmentTimeoutClassifier instead of hard Failed (tenant a53e67ec:
fully provisioned WG Part 2, user never logged in => AwaitingUser).
agent_timeout no longer counts as HasExplicitFailure; failure
webhooks fire only on actual failure verdicts.
3. Maintenance sweep: dropped the too-broad "IsPreProvisioned ne true"
exclusion (sealed devices are protected by their Pending status);
resumed WG Part-2 runs sweep normally with the window anchored to
ResumedAt. New stage-2 silence guard also fixes a latent bug where an
actively-reporting >5h enrollment could be terminalized mid-run.
4. Registration supersede: a fresh registration resolves older
non-terminal sessions of the same device (serial, placeholder-guarded)
as Incomplete("Superseded by ...") with live SignalR updates — closes
the forever-Pending orphan gap when WG Part 2 runs under a new id.
5. Admin retro-reconcile: POST /api/maintenance/reclassify-legacy
(GlobalAdminOnly, DRY-RUN BY DEFAULT) re-runs the classifier over the
pre-classifier "Session timed out" Failed backlog (~495 rows, sample
projected ~87% misdeclared) and resolves Pending orphans. Requires the
new opt-in allowTerminalReclassification override in the status
writer (admin-marked rows stay untouchable).
6. Polish: "0 of N" AccountSetup rollups keep their total (no more
fabricated "0/0" reasons, "never observed" wording when absent);
late-telemetry reconcile heals Incomplete/AwaitingUser to Succeeded
when straggler evidence proves completion; SLA page knows
AwaitingUser/Incomplete ordinals; script_failed on Success/exit-0 now
names the stderr rule in its message.
Tests: backend 2965, DecisionCore 474 (+6 new), agent 1777, web 557 —
all green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
7521158e65 |
backend: bump to .NET 10 (dotnet-isolated 10.0) for the Flex recreate
TargetFramework net8.0 -> net10.0 (Functions + Tests), setup-dotnet 8.0.x -> 10.0.x. JwtBearer 8.0.11 -> 10.0.10 (Microsoft.Identity.Web 4.3.0 requires >= 10.0.0 on net10). Drops the System.Formats.Asn1 / System.Security.Cryptography.Xml vuln-override pins - superseded by the net10 framework versions (NU1510). Build clean, 2937/2937 backend tests green. Merge on cutover Saturday BEFORE running deploy-backend (the new Flex app is created as dotnet-isolated 10.0). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
87cf893b03 |
feat(reconcile): surface silence duration + decision time in timeout reconcile
The "Backend reconciled" badge showed only the reason text with no timestamps, so operators/customers could not tell "user finished setup and powered the device off" apart from "the mechanism declared success too early" (session efbc17ff): last-contact, silence window and the exact decision moment were all invisible, and the stage-1 "Agent silent for Nmin" note is wiped by the Succeeded-write reconcile hygiene. - EnrollmentTimeoutClassifier.ClassifyTimedOutSession takes an optional lastEventAtUtc and appends a timing clause to the two Succeeded-reconcile reasons (AccountSetup-complete + desktop/Hello): last agent contact, silence duration, and the sweep's declaration time. Lives in the reason string so it survives the hygiene and reaches badge/MCP/exports. - MaintenanceService passes session.LastEventAt through. - Web: add Session.stalledAt; reconcile banner renders structured UTC anchors (last agent contact + first-flagged-silent with the observed silence gap). Verified: build 0 errors; classifier 36/36 (2 new timing tests); maintenance/status/projection 160/160; web tsc clean; vitest 557/557. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
b0105b123a |
feat(maintenance): session-deletion run budget, live watchdog counters, manual trigger
- Watchdog fix: FanoutResult is caller-allocated and mutated live per tenant/session, so 30/60min LongRunning warnings report real progress instead of the always-zero counters (today's warning showed 0/0 while 5 tenants / 54 sessions were already done). - 50min run budget: the retention fanout stops cleanly at the deadline (tenant boundary + per-session check) and reports AbortedByBudget — no more risk of the 60min Flex functionTimeout hard-aborting mid-run. New Warning OpsEvent SessionDeletionMaintenanceBudgetExceeded; Completed gains abortedByBudget. - Blob-lease serialization: dedicated sentinel _lock/session-deletion-maintenance.lock (deletion-manifests container) via SessionDeletionMaintenanceLockStore + reused MaintenanceLeaseHolder; contended runs emit SessionDeletionMaintenanceSkippedLocked and return. - Manual trigger: POST /api/global/session-deletions/maintenance/trigger (GlobalAdminOnly, policy catalog) -> fail-hard queue producer -> SessionDeletionMaintenanceQueueWorker runs the same RunCoreAsync body with triggeredBy = requesting admin; lease probe returns 409 while a run is active. New Started OpsEvent carries triggeredBy. - Web: MaintenanceStatusBanner on the Session Cleanup page derives active / budget-exceeded / failed / idle from the lifecycle ops events and offers a Run-now button (202/409 handling). New event types dual-registered in OPS_EVENT_TYPES. - Tests: 7 new (budget abort immediate/mid-tenant, live counter increments, watchdog live-progress regression, BudgetExceeded + Completed flag, SkippedLocked, Started triggeredBy); suites green (backend 2935, web 557, tsc clean). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
3a4d80cd7f |
hardening(auth): validate + audit EntraId:AdditionalClientIds trust-set expansion
Expanding the token audience trust set via EntraId:AdditionalClientIds was completely silent: no format validation, no operator-visible logging. During the tenant-move window a typo'd GUID would be indistinguishable from "the new app registration doesn't work" (daily-review follow-up, 2026-07-14). - ResolveConfiguredClientIds: additional entries must parse as GUIDs; malformed entries are dropped (fail-closed - they could never match a token aud) and reported via out param. Accepted entries are normalized to the canonical lowercase dashed GUID form so brace/uppercase paste variants still match the ordinal audience comparison. Primary ClientId handling stays verbatim. - AuditClientIdTrustSet: one Warning per distinct config value per instance for (a) dropped malformed entries (truncated - never log a full pasted value in case it is a secret) and (b) trust-set expansion with the accepted ID list. Warning level because nothing below Warning reaches App Insights. - Tests: existing ResolveConfiguredClientIds tests moved to GUID values (signature change, no compat alias) + rejection, primary-verbatim, normalization and TruncateForLog coverage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |