mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-24 20:53:06 +00:00
ae85dc325cfc484d35cd20765327a2560cfbb7c9
231 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oliver Kieselbach
|
57775d6ba2 |
mcp: errorMessage in tool_call telemetry + get_rule_stats trend opt-in
Telemetry follow-ups from the 7d tool-quality review: - tool_call log lines now carry a capped errorMessage (thrown message or leading toolError text) so error drilldowns name the failure class instead of guessing from args; absent on success. - get_rule_stats omits per-rule daily trend rows by default (they drove 50% of calls over the response cap at platform scope); new includeTrends arg opts back in, trendsNote marks the omission. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
e12da0c99b |
geo metrics: median API latency as display statistic
A single corrupt session average (request spanning sleep/hibernate counts wall-clock hours) inflated the request-weighted location mean by orders of magnitude (Amersfoort: 73286 ms from one 1.38M-ms session). Add MedianApiLatencyMs per location + global (upper median over per-session averages, duration convention), base ApiLatencyVsGlobalPct on medians, and switch web display/sort to the median with the weighted avg in the tooltip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
3ac647a680 |
mcp tests: exact hostname match in Entra fetch stub (CodeQL alert #76)
The test-only fetch stub matched the Entra authority via substring (includes), which CodeQL flags as incomplete URL sanitization. Parse the URL and compare the hostname exactly instead — also strictly more precise for what the stub intends (intercept only the Entra host). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
01e76a6ff7 |
mcp: OAuth-proxy hardening — Entra token timeout, https boot guards, replay note
- /oauth/token: AbortSignal.timeout(15s) on the outbound Entra fetch; TimeoutError maps to 504 token_exchange_timeout (502 stays for hard failures). Undici defaults previously allowed ~300s per stalled request. - Boot guards (fail-fast, matching config.ts posture): AUTOPILOT_ENTRA_AUTHORITY must be https (sovereign clouds allowed, host not pinned); MCP_PUBLIC_URL pin must be https (http tolerated for loopback dev only), trailing slashes stripped; production log hint when AUTOPILOT_ENTRA_CLIENT_SECRET is unset (public-client vs AADSTS7000218 misconfig). - State one-time-use: documented as accepted residual risk (a nonce store would reintroduce the shared state the stateless multi-replica design removed; impact bounded by single-use PKCE-bound Entra codes + callback re-validation). - Tests: oauth-boot-guards.test.ts (validators) + token-timeout route tests with Entra-host-scoped fetch stub. Suite: 559 passed, tsc clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
98fd173fb8 |
rules+backend+mcp: rule-verify sweep fixes (dead factor, {{token}} evidence, stats cap, list stringify)
Full verify pass over all 54 built-in rules (validate_rule sweep, 30d/7d
get_rule_stats, test_analyze_rule spot checks). Fixes:
- ANALYZE-APP-007 v1.1.0: declare enrollment_failed as a non-required
condition — the exists-factor referenced an undeclared signal and could
never apply.
- ANALYZE-APP-014 v1.1.0: add non-required event_data conditions for
espCategory/espErrorCode/topCandidatePackage/candidateCount/confidence
so all explanation {{tokens}} resolve instead of rendering literally.
- ANALYZE-ENRL-002 v1.2.0: capture session_timeout.timeoutHours so
{{timeoutHours}} interpolates (verified via dry-run: renders "5").
- RuleEngine TryFlatLookup/ResolveDotPath: flatten IEnumerable data
values to a comma-joined string — List fields (e.g. app_tracking_summary
.pendingNames) stringified to "System.Collections.Generic.List`1[…]",
breaking contains-matching and {{pendingNames}} interpolation.
- GetRuleStatsAsync: raise the 500-row cap to 10000 and log when hit.
The scan walks dates ascending, so the old cap silently dropped the
NEWEST dates of a 30d window — newly added rules (e.g. ANALYZE-SEC-008)
were entirely missing from global stats.
- MCP validate_rule lint: accept event_data_array itemField as a {{token}}
resolver — {{identity}} in ANALYZE-SEC-005/006 was a false positive.
Backend 4157 tests green, MCP 548 green + tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
22a8c7a51d |
rules: hard-block high-frequency event types as evaluateOn on_event triggers
A warning does not stop an end user — telemetry-cadence types (performance_snapshot, agent_metrics_snapshot, download_progress, network_state_change/connectivity_check, log_entry, agent_trace, stall_probe_check) would turn every ingest batch into an analyze run. Now enforced in three layers: - AnalyzeRuleTriggers.BlockedOnEventTypes (code-only enforcement): CRUD write is rejected with an explicit message, and OnEventTypes() drops blocked types at runtime so even a persisted row (pre-validation data, direct table write) is inert for the trigger registry and the engine alike. - rules/guardrails.json gains blockedInterimTriggerEventTypes as the display/ pre-flight mirror (baked into the MCP RULE_GUARDRAILS via combine.js); a parity test pins mirror and code set against drift — same pattern as the gather hard blocks. - MCP validate_rule: the high-frequency finding is now an ERROR sourced from the baked mirror, not a hardcoded warning. Suites: Backend 4155, MCP 548. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
72136f0273 |
backend+mcp+web: evaluateOn interim triggers for analyze rules
Implements docs/rules/analyze-rule-triggers.md (spec approved today): - AnalyzeRule.EvaluateOn (absent = [enrollment_end], today's behavior bit-for-bit) with the shared AnalyzeRuleTriggers grammar/matching helpers; grammar enforced in tenant rule CRUD and the analyze-rule JSON schema. - RuleEngine: AnalyzeRunContext (Terminal/Interim/Reanalyze), interim rule filter, KO suppression on interim runs, and RuleResult update-semantics replacing the permanent dedupe freeze — interim rows refresh, the terminal pass finalizes or resolves them (ResolvedAt, kept for audit), Reanalyze preserves lifecycle markers. - Notification dedupe moves onto RuleResult.NotifiedAt (one send per session+rule, stamped after a successful send; survives the manual reanalyze rebuild). - Handler: whiteglove_sealed / interim_trigger reasons — notify + SignalR yes, IssuesDetected + rule stats terminal-only (Rule Regression Radar baselines clean). - Ingest: InterimTriggerRegistry (per-tenant 5-min TTL cache, fail-soft) gates the interim_trigger enqueue (batch eventType intersection) and the whiteglove_sealed enqueue on the first genuine WhiteGlove seal. - Storage: EvaluateOnJson on AnalyzeRules; FirstDetectedAt/LastEvaluatedAt/ IsInterim/ResolvedAt/NotifiedAt on RuleResults (nullable — legacy rows read as final). - MCP: three authoring lints (terminal-precondition trap, unknown/high-frequency on_event types, KO-on-interim), authoring guide + baked schema updated; get_session_summary marks interim and excludes resolved findings. - Web: preliminary badge on interim findings, resolved findings behind a toggle, issue counts exclude resolved (analysis section + diagnosis flow). - ANALYZE-ID-004 v1.1.0: first interim-enabled rule (on_event:hybrid_login_pending), threshold raised to 75 so only repeated overdue-login signals fire mid-run. Suites: Backend 4152, MCP 548, Web 762 + tsc + lint clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
66d25f57db |
Cloud-PC-aware autopilot_profile_missing: Info instead of Warning on W365
On Windows 365 Cloud PCs a missing Autopilot profile (ProfileAvailable=0) is the expected state - Cloud PCs are provisioned by the Windows 365 service, not by Autopilot - so the warning shown for physical devices is misleading there. - Agent: emit autopilot_profile_missing as Info with a Cloud-PC-specific message when CloudPcDetector matches; payload now carries isCloudPc. Evidence collection (ZTD verdict, registry, endpoint probe) unchanged. - Web: DeviceDetailsCard renders a neutral info note instead of the amber warning for Cloud PC sessions (session.isCloudPc, sticky, covers existing sessions; or the new event marker). - MCP: resource-catalog documents the new isCloudPc data field. - Docs: Cloud PC exception noted in agent/autopilot-ztd-diagnostics.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
dependabot[bot]
|
c28d64fe27 |
chore(deps-dev): bump tsx (#128)
Bumps the mcp group in /src/McpServer/autopilot-monitor-mcp with 1 update: [tsx](https://github.com/privatenumber/tsx). Updates `tsx` from 4.23.1 to 4.23.11 - [Release notes](https://github.com/privatenumber/tsx/releases) - [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs) - [Commits](https://github.com/privatenumber/tsx/compare/v4.23.1...v4.23.11) --- updated-dependencies: - dependency-name: tsx dependency-version: 4.23.11 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: mcp ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Oliver Kieselbach
|
2c48968ec6 |
session annotations: role-separated human verdicts on sessions (flywheel labels)
New SessionAnnotations table (PK=tenantId, RK={sessionId}_{lane}, lanes
operator/tenantadmin/globaladmin) storing structured verdicts + notes with
server-stamped authorship and a fired-rule-id snapshot for join-free
rule-quality evaluation. Per-lane write matrix re-gated in-function with
own-tenant binding; the globaladmin lane is platform-internal and filtered
from tenant callers. Portal: SessionAnnotationsCard on session detail.
MCP: list_session_annotations (ga) + annotate_session (strictGa, GA lane
only) + annotations key in get_session_summary. Lifecycle: cascade steps
17-19, offboarding wipe, critical-backup member. Privacy page gains the
customer-entered-content line.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
2f8ad82b0a |
gather-rules: mirror hard blocks in portal/MCP, stamp author on PUT-upsert; agent: bootstrap retry tests
Codex follow-ups to
|
||
|
Oliver Kieselbach
|
1ebc2675b1 |
gather-rules: WQL property projections, portal enable-gating, author from JWT
- WMI guard accepts SELECT <property-list> FROM <class> for any class already
allowed via SELECT * (strict data subset); same algorithm in agent, portal
validator, and MCP validate_rule (incl. trim fix); guardrails.json unchanged
- Portal refuses to enable custom rules with not-allowed targets: create lands
disabled, invalid edit force-disables, "Blocked on devices" badge, GitHub-issue
hint for allowlist requests; agent stays the enforcement boundary
- Rule Author is stamped from the creator's token (display name > UPN) at create
and preserved through every update, incl. cross-tenant GA edits
- bumpVersion keeps the segment count ("1.0.0" -> "1.1.0")
Customer feedback: SELECT BatteryStatus FROM Win32_Battery + on_change was
impossible because SELECT * re-emits on every charge-percent fluctuation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
1c5f69a9fe |
w365: CloudPc device validator — cert-CN-bound fallback for Cloud PCs (#127)
* w365: CloudPc device validator — cert-CN-bound fallback for Cloud PCs Windows 365 Cloud PCs are never Autopilot-registered, so the serial lookup (and the Corporate Identifier fallback) structurally reject them and Cloud PC first-connect enrollment could not be monitored. New stage-3 fallback in the device-validation chain: extract the Intune device id from the chain-validated MDM client certificate's Subject CN (field-verified: CN == managedDevice id == cloudPC.managedDeviceId) and require a matching cloudPC object in the claimed tenant (Graph v1.0 virtualEndpoint/cloudPCs, $filter=managedDeviceId eq). Only service-provisioned Cloud PCs have such an object — a regular enrolled non-Autopilot device still ends in the 403, and local W365 markers are never consulted for auth. - Shared: ValidatorType.CloudPc = 5, TenantConfiguration.ValidateCloudPcDevice (default false, tenant-admin writable), GraphAppPermissions.CloudPCReadAll, GraphFeatureCatalog feature W365CloudPcValidation -> CloudPC.Read.All - Backend: CloudPcDeviceValidator (AutopilotDeviceValidator sibling contract: 30/5 min cache, 2 attempts, transient->503; GUID gate before the OData filter; exact-match guard; Graph 403 = definitive "grant missing", not transient), SecurityValidator CN extraction + fallback stage + hard gate, wiring through the 5 agent endpoints, TableConfigRepository persistence - Consent script: W365CloudPcValidation in catalog/ValidateSet/help; NEW GraphFeatureCatalogSyncTests pins C# catalog <-> grant script (previously comment-only lock-step) - Bootstrap dev script v2.3-dev.3: guard 4 (uptime window) exempted while the Cloud-PC relax is active — Cloud PCs run headless for days before first connect; the fresh-profile window is the time anchor there (+ Pester tests) - Web: validation toggle + context plumbing, ValidatedBy "CloudPc" label, privacy-page validation claim generalized (Last updated bumped) - Docs: OKF concept doc backend/cloudpc-device-validation.md + index + log Verified: web tsc clean, ESLint 0 errors (99 pre-existing warnings = ratchet cap, none added). .NET build/tests and Pester run in CI — SDK download and pwsh are unavailable in this sandbox's network policy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqEFrcAHhWgCvgJgHGzMuv * w365: agent-side IsCloudPc flag + engine expectation "no Device-ESP phase" Companion to the CloudPc validator: the agent now classifies Windows 365 Cloud PCs locally with the bootstrap's field-verified marker AND (HKLM\SOFTWARE\Microsoft\Windows365 key + installed CloudManagedDesktopExtension service, probed via its Services registry key so net48 needs no System.ServiceProcess). SKIP-safe: any error resolves false. Never an auth input - backend admission stays exclusively the cert-CN-bound Graph lookup (ValidatedBy=CloudPc); the two signals are independent and may disagree. Two transport rails: - Session metadata: SessionRegistration.IsCloudPc -> Sessions table (sticky-true OR across re-registrations, IsSelfDeployingProfile pattern) -> SessionsIndex full mirror -> SessionSummary -> new isCloudPc search filter end to end (SearchSessions + QueryRawSessions + pagination fingerprint + all three in-memory predicates; MCP search_sessions / query_raw_sessions params + summary projection; web type, Cloud PC pill, SessionInfoCard enrollment label, CSV export appended column). - Decision engine: isCloudPc rides the EnrollmentFactsObserved payload, recorded set-once for BOTH values in EnrollmentScenarioObservations.CloudPc and annotates the scenario profile reason cloud_pc_first_connect:no_device_esp_expected - the engine-recorded expectation that Device-ESP already ran headless at provisioning and the session starts at Account Setup. Deliberately reason+observation only: no completion arm depends on DeviceSetup, Mode stays AccountSetup/IME- classified (Classic), EspConfig keeps configured-FirstSync semantics. Positive marker surfaces as cloud_pc_marker in the audit-trail census. Updater idempotency: the cloudPcAnnotates gate keeps the generic v1 reason branch from re-claiming the slot on repeat posts (would bump EvidenceOrdinal every post). Additive-nullable observation - no state schema bump (v4 convention), serializer roundtrip pinned. Tests: CloudPcDetector resolver (marker AND, throwing probe), registration body, facts-signal payload shape, updater annotation/idempotency/ self-deploying interplay, reducer observation both values + census fold, serializer roundtrip, index projection superset, sticky-true both directions. Suites: DecisionCore 500, Functions 3941, Agent V2 2154, vitest 632 - all green; tsc web+MCP clean; ESLint 0 errors (cap 99). Docs: cloudpc-device-validation.md follow-up section replaced with the shipped mechanism + log entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
4def73f193 |
rules: reserve the built-in rule-ID namespace against tenant custom rules
Tenant rule IDs were free-form with only a point-in-time existence check, so a customer could occupy a gap or retired ID like ANALYZE-SEC-002; a later built-in with that ID would then silently shadow the tenant's rule at merge time (global wins, tenant copy skipped) and their RuleStates would start toggling our rule. Enforcement: RuleIdPolicy (Shared) reserves (ANALYZE|GATHER)-<CAT>-<NUM> case-insensitively, with the CUSTOM category as the sanctioned tenant namespace (we never ship built-ins in a category named CUSTOM). Both rule services reject reserved IDs on create AND on the custom-branch of update (updates upsert, so PUT was a bypass); update endpoints now map InvalidOperationException to 409 like create. Web forms mirror the check inline; MCP validate_rule emits a warning finding and the authoring guide now documents the reservation (guide templates moved to GATHER-CUSTOM-101/ANALYZE-CUSTOM-101 — the old numeric examples would be rejected). No existing prod custom rule violates the pattern (verified against all 13 tenant-partition rows), so nothing needs grandfathering. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
75197a4184 |
Logparser pattern testing with the agent's exact .NET engine (test_log_pattern) (#125)
* updated commands * feat(backend): logparser pattern test endpoint with agent-exact semantics POST /api/rules/gather/test-pattern evaluates a logparser gather-rule regex against pasted sample lines exactly the way the agent would on the device. Motivated by a field incident: a customer verified a pattern with a PHP regex tester and the .NET engine matched differently. - CmTraceLogParser moves Agent -> AutopilotMonitor.Shared.Logging so agent (IME tracker, logparser collector, stall probe, IME matching mode) and backend run the SAME parsing implementation. Pure mechanical move, agent behavior unchanged (V2.Core suite 2147/2147 green). - TestLogPatternFunction mirrors LogParserCollector verbatim: Regex(Compiled, 1s timeout, NO IgnoreCase -> case-sensitive), first match per line, group "0" excluded / unsuccessful groups omitted, format cmtrace (default, match against parsed message) vs text (raw line), per-line outcomes incl. parse_failed/regex_timeout, and the agent's "set parameter format=text" hint verbatim. Pure compute, no tenant data. Caps: 200 lines, 8KB/line. - Policy: TenantAdminOrGlobalReader (same tier as the analyze dry-run). - Tests pin the parity showcases: case-sensitivity, envelope invisibility in cmtrace mode, optional-group omission, format hint, request validation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(mcp): test_log_pattern tool + logparser lint and guide hardening - test_log_pattern: wraps POST /api/rules/gather/test-pattern — tests a logparser regex against pasted sample lines with the agent-exact .NET semantics (case-sensitive, cmtrace/text mode). Description steers clients away from JS/PHP/Python regex testing. - validate_rule: logparser parameter lint (missing/non-compiling pattern, unknown format value = silently cmtrace on the agent) + info pointing to test_log_pattern; nextStep for valid logparser drafts routes there too. - rule_authoring_guide: logparser section now states the matching semantics (case-SENSITIVE, .NET, first match per line, cmtrace default) and two new pitfalls (engine mismatch, plain-text log under cmtrace default). - Catalog snapshot + handler/validator tests extended (MCP suite 532 green). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(okf): test-pattern endpoint + shared CmTraceLogParser in rule authoring surface Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
4966d23a46 |
fix(mcp): rejection sniffer must match the isError-result shape, not a JSON-RPC error envelope
Second live miss, root cause found via end-to-end repro against the real SDK
stack: SDK 1.30 catches every McpError around the tool handler (Zod input
validation, unknown/disabled tool) and answers with a SUCCESSFUL JSON-RPC
response whose result is a CallToolResult { isError: true, text: "MCP error
<code>: ..." } — a JSON-RPC error envelope never exists for these. The sniffer
now recognizes both shapes and extracts the code from the text prefix; handler
soft errors (toolError) never carry that prefix, so they are not double-logged.
Repro verified: the sniffer logs tool_call_rejected against a live
express + StreamableHTTPServerTransport + zod-tool stack.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
59ba8301a8 |
fix(mcp): rejection sniffer must capture res.write chunks, not only res.end
Live verification caught the miss: tool_call and search_zero_hit lines arrived, tool_call_rejected did not. SDK 1.30 bridges the web-standard Response through Hono's getRequestListener, which pipes ReadableStream bodies as Uint8Array chunks via res.write followed by a BARE res.end() — the end(chunk) inspection never saw the envelope. The sniffer (now attachToolCallRejectionSniffer in telemetry.ts, unit-testable) wraps both write and end, reassembles up to 10 KB and decodes string/Uint8Array chunks; tests pin the streaming path, the direct end(string) path, the over-cap silence and the disabled no-wrap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
afa8750cbb |
feat(mcp): tool-call quality telemetry — soft errors, result size, zero-hit searches, arg rejections
withToolTelemetry now receives the tool args and inspects the RESULT: handlers never throw (toolError returns isError:true), so the previous isError flag was effectively always false. Each MCP_TOOL_LOGGING line now carries soft-error status, resultChars + overCap (vs. the anthropic/maxResultSizeChars hint), a size-bounded args summary and the caller scope — the only telemetry that covers backend-free tools (search_docs, validate_rule, get_resource). New signals: search_zero_hit on the three search tools (unmet-demand signal) and tool_call_rejected for JSON-RPC-level Zod/unknown-tool rejections, sniffed at the /mcp route since the SDK answers those before any handler runs. Bicep default toolLoggingEnabled=true (desired state; live flag via az containerapp update). Docs: docs/mcp/tool-telemetry.md with KQL examples. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
3c1229a301 |
MCP rule authoring: deterministic context, validate_rule, and session dry-run (#124)
* feat(backend): dry-run endpoint for draft analyze rules (rule authoring)
POST /api/rules/analyze/dryrun evaluates a DRAFT analyze rule against one
session's events and returns a full diagnostic trace: per-condition
matched/evidence (incl. per-eventType occurrence counts), per-factor
confidence breakdown, verdict, and the would-be MatchedConditions map for
{{token}} interpolation preview. Persists nothing: no RuleResult row, no
session-status change, no notifications.
- RuleEngine.DryRun.cs: DryRunRuleAsync / DryRunEvaluate share the exact
evaluators (EvaluatePrecondition/EvaluateCondition/EvaluateConfidenceFactor)
with the production path but evaluate ALL conditions instead of breaking on
the first failed required one; parity with EvaluateRule is pinned by tests.
- DryRunAnalyzeRuleFunction: TenantAdminOrGlobalReader (read-in-effect POST),
GA/GlobalReader cross-tenant via session tenant resolution, 1 MB body cap,
structural draft validation (unknown source/operator, correlation without
joinField, invalid regex, non-evaluable confidenceFactor DSL) with
actionable error messages.
- EndpointAccessPolicyCatalog: register the new route.
- RuleEngineDryRunTests: trace semantics, production parity matrix,
side-effect freedom (VerifyNoOtherCalls), draft validation matrix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(mcp): rule-authoring resources — guide, schemas, guardrails
Deterministic authoring context for customer AIs, replacing chunked
search_docs retrieval for rule creation: get_resource (and the MCP-protocol
resources) now serve
- rule_authoring_guide: hand-written, complete authoring knowledge (collector
types, triggers, condition sources, operators, confidence model incl. the
exact factor-condition DSL shapes, {{token}} interpolation order, guardrail
matching semantics, pitfalls, validate->dry-run workflow)
- rule_schemas: the JSON Schemas (2020-12) for gather + analyze rules
- rule_guardrails: the on-device collection allowlists
Schemas + guardrails are generated into rule-authoring.generated.ts by a new
combine.js emitter (same pattern as the web guardrails mirror; rules/ stays
the single source of truth and CI guardrails-in-sync guards the file).
rule-authoring-drift.test.ts additionally pins generated == repo source,
guide enums == schema enums, and the get_resource size cap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(mcp): validate_rule + test_analyze_rule tools and author-rule prompt
Closes the authoring loop for customer AIs: draft -> validate -> dry-run
against a real session -> iterate -> create in the portal.
- validate_rule (MCP-local, no backend call): ajv 2020-12 validation against
the baked schemas, guardrail pre-flight mirroring the agent matching
semantics (hive-stripped segment-bounded registry/file prefixes,
whitespace-bounded WMI, exact commands, /-bounded event log channels, hard
blocks), and semantic lint for schema-valid-but-dead rules (unreachable
threshold, non-evaluable factor DSL, unknown eventTypes, unanchored
not_regex allowlists, unresolvable {{tokens}}, trigger cross-fields,
outputEventType conventions/collisions).
- test_analyze_rule: calls the new POST /api/rules/analyze/dryrun and adds
the {{token}}-interpolated explanation/remediation preview via the
existing interpolate-rule-template.
- author-rule MCP prompt guiding scenario -> resources -> draft -> validate
-> dry-run.
- Both tools are visible to every role (backend enforces
TenantAdminOrGlobalReader on the dryrun route); catalog snapshot updated.
- deps: ajv + ajv-formats promoted to direct dependencies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(okf): rule authoring surface concept doc
docs/rules/rule-authoring-surface.md: the AI-assisted authoring stack
(deterministic MCP resources via combine.js baking, MCP-local validate_rule
layers, the dry-run trace twin with its parity + side-effect invariants, and
what is deliberately absent); registered in index.md, logged 2026-08-05.
Customer docs (rules/ai-assisted-rule-authoring.md + SUMMARY + FAQ entry) are
committed in the autopilotmonitor-docs repository separately.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
ed5fcf3dc3 |
chore(deps): bump transitive npm deps to resolve Dependabot alerts
- mcp: hono 4.12.31->4.13.0 (ReDoS in CORS middleware), fast-uri 3.1.4->3.1.5 (host confusion), ip-address 10.1.0->10.4.0 (SSRF bypasses), postcss 8.5.21->8.5.25 (sourceMappingURL file read) - web: postcss 8.5.22->8.5.25, brace-expansion (DoS, npm audit) Lockfile-only changes within existing semver ranges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
9d8b0fc254 |
feat: tenant-config field schema - MCP writes get types right on the first attempt (#121)
Follow-up to the E2E finding (95 vs 95.0): a model driving update_tenant_config had to guess field names and JSON types from a config read. New reflection- generated schema surface makes guessing unnecessary: - Backend: GET config/fields-schema (GlobalAdminOnly) - built from the TenantConfiguration model + the patch service deny-lists at runtime, so it can NEVER drift from what the patch endpoint enforces. Per field: camelCase name, JSON type (string/boolean/integer/number), nullability (null clears), format hints (json / date-time), writability + deny reason, GA-only marker (phase 2), revert-protected marker. Literal route; FindPolicy literal-over-template precedence pinned by test (same mechanism as config/all). - MCP: new strictGa READ_ONLY tool get_tenant_config_schema; update_tenant_config description + GA instructions headline steer the model to call it before composing a patch. - Tests: drift canary (every model property appears exactly once), deny-list consistency, type exemplars incl. the decimal SLA column class, policy + route-precedence pins, MCP catalog/registration/request-shape. Backend 3854 green, vitest 465 green, tsc clean. Deploy order: backend first, then MCP redeploy. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
395fdd49df |
Transactional tenant-config writes via MCP + ConfigurationBackups safety net (#120)
* feat(backend): pre-write config backup net - every tenant/admin config save snapshots first New ConfigurationBackups table (PK=tenantId or GlobalConfig, RK=reverse-ticks+guid, newest-first) holds the raw stored row as EntityJson before any overwriting save of TenantConfiguration or AdminConfiguration. Hook lives in TableConfigRepository, the single choke point all writers funnel through - fail-soft (a backup hiccup never breaks a save), skips row creations and noise-only changes (LastAuthClientId flips), prunes to the newest 2 per partition. Source/reason tagging via a new SaveTenantConfigurationAsync overload (not optional params - Moq expression trees, CS0854); portal-put/plan/auth/app-homing/offboard call sites tagged. Offboarding: backup partition wiped in the bulk phase (TenantPartitionTables) and by the marker-cleanup tombstone sweep; the GlobalConfig partition never matches a tenant GUID and survives. Table registered in TableNames.All + CriticalBackupTables.All. Groundwork for the transactional MCP config-write flow (patch/revert endpoints follow). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(backend): transactional tenant-config field patch + revert with verified writes New GA-only surface (policy catalog: GlobalAdminOnly + RouteParam): - PATCH config/{tenantId}/fields - field-level patch through TenantConfigPatchService: fresh ETag read -> field gate (deny-list incl. plan/trial, HomedAppClientId, auth provenance; unknown fields 400 with the name) -> patch onto a clone -> shared validation -> fail-CLOSED backup -> CAS replace -> fresh re-read -> verify that EXACTLY the intended fields changed -> automatic ETag-conditional rollback on drift (catches Store/Map serialization asymmetry). - POST config/{tenantId}/revert - restore from a ConfigurationBackups snapshot (latest by default); snapshots current state first (revert is revertible), preserves plan/homing/provenance fields unless includeProtectedFields. - GET config/{tenantId}/backups - metadata + masked diff only, never EntityJson. Supporting changes: IConfigRepository gains GetTenantConfigurationWithEtagAsync + TryReplaceTenantConfigurationAsync (412 -> false, no implicit backup hook); validators moved from the PUT into shared TenantConfigValidation (PUT re-pointed, forwarding shims keep test call sites); GET config/{tenantId} honors ?view=redacted so a GA's MCP session never pulls clear-text secrets into model context. Caller-tier deny-lists (TenantAdmin/DelegatedAdmin) ship now for the phase-2 flip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(mcp): tenant-config write tools - update/revert with snapshot-backed verified writes First mutating surface on the MCP server, strictGa (real Global Admin) only: - get_tenant_config: full config read, ALWAYS ?view=redacted (secrets never reach model context; pinned by config-write-tools tests) - update_tenant_config: field-level patch via PATCH config/{tenantId}/fields; required reason; response = applied fields + masked diff + backupId - list_tenant_config_backups: snapshot metadata + masked diff only - revert_tenant_config: restore latest/named snapshot; protected fields stay current unless includeProtectedFields; revert is itself revertible New MUTATING annotation + TenantGuidSchema (path-traversal guard) in shared.ts; role-aware instructions headline (non-GA keeps the exact READ-ONLY wording); catalog snapshot test extends GA_FULL + new CONFIG_WRITE_GA_STRICT difference list (Global Reader/tenant/delegated see none of the four). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(backend): de-flake session-deletion heartbeat test - condition-gated handler instead of wall-clock window The heartbeat test held the handler for a FIXED 350ms and expected the 50ms heartbeat timer to tick >=3 times inside that window. On loaded CI runners the heartbeat Task.Delay stretches disproportionately to the handler delay, the handler completes with <3 heartbeats fired, and the awaited condition can never become true afterwards (CI failure on PR #120). The handler now holds the message in-flight UNTIL the test observed 3 heartbeats (with ct + own 15s deadline as termination guards), so the assertion is reached deterministically regardless of scheduler jitter. Verified: 12/12 local runs green, full suite green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
946d503be0 |
feat(mcp): emit RFC 9207 iss parameter in OAuth authorization response
MCP spec 2026-07-28 requires clients to validate the iss parameter against the recorded issuer before redeeming the authorization code (AS mix-up defense). The /oauth/callback redirect now carries iss=<public base URL> — the same value the RFC 8414 metadata reports as issuer — and the metadata advertises authorization_response_iss_parameter_supported. Pre-9207 clients ignore unknown response parameters (RFC 6749 s4.1.2), so nothing regresses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
dff1532be5 |
feat: project ConnectionType (WiFi/Ethernet) onto SessionsIndex
The agent already reports connectionType in every network_interface_info event, but it was only stored as a JSON blob in DeviceSnapshot — answering "how many machines enroll over Ethernet vs WiFi?" required a slow full-table scan via deviceProperties. Project it as a typed column instead (template: the AvgApiLatencyMs chain), making it a cheap indexed one-call query. - Ingest: TryExtractConnectionType — last network_interface_info emission of the batch wins (a device switching media mid-enrollment is relabeled); only exact "WiFi"/"Ethernet" accepted, fire-and-forget fail-soft writer - Storage: UpdateSessionConnectionTypeAsync (Merge on Sessions + index mirror, tombstone-safe), BuildSessionIndexEntity superset entry, MapToSessionSummary, re-registration preserve in StoreSessionAsync - Search: connectionType= filter with OData eq push-down on the index, client-side backstop on the non-OData paths, fields=connectionType projection, pagination fingerprint entry, raw endpoint parity - MCP: connectionType z.enum param on search_sessions/query_raw_sessions, fields doc updated (server redeploy required) - Web: optional sortable/filterable "Connection" column in the session list - No backfill: legacy sessions lack the column and are excluded by the filter (intended, same semantics as RebootCount); the deviceProperties scan path still covers historical data Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
3ec716a3ec |
feat: expose API latency to MCP field projection and tool docs
- SearchSessionsFunction fields whitelist += avgApiLatencyMs, apiRequestCount so search_sessions lean sweeps (e.g. fields=geoCountry,avgApiLatencyMs,apiRequestCount) can aggregate latency without full SessionSummary payloads - search_sessions tool doc lists the new fields incl. the weighting hint; get_geographic_metrics doc names avgApiLatencyMs / apiLatencyVsGlobalPct as the backend-region distance metric Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
dependabot[bot]
|
2e8c2cdb2f |
chore(deps): bump @modelcontextprotocol/sdk (#110)
Bumps the mcp group in /src/McpServer/autopilot-monitor-mcp with 1 update: [@modelcontextprotocol/sdk](https://github.com/modelcontextprotocol/typescript-sdk). Updates `@modelcontextprotocol/sdk` from 1.29.0 to 1.30.0 - [Release notes](https://github.com/modelcontextprotocol/typescript-sdk/releases) - [Commits](https://github.com/modelcontextprotocol/typescript-sdk/compare/v1.29.0...1.30.0) --- updated-dependencies: - dependency-name: "@modelcontextprotocol/sdk" dependency-version: 1.30.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: mcp ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Oliver Kieselbach
|
c8175ff7b1 |
fix(mcp): drop npm from the runtime image, bump tar to 7.5.22
Defender for Cloud reports CVEs against the mcp container image that the repo scanners cannot see. The subassessment evidence paths split them in two: /usr/local/lib/node_modules/npm/** - the npm 10.9.8 that node:22-slim bundles, vendoring @sigstore/core 2.0.0, brace-expansion 2.0.2, tar 7.5.11 (critical), sigstore, picomatch, ip-address, plus npm itself (high, no fix available). Present in all 20 images including the current one. Dependabot reads package-lock.json, so none of this is visible to it. /app/node_modules/** - body-parser 2.2.2, fast-uri 3.1.2, hono, protobufjs, tar 7.5.16, all confined to images from 2026-07-21 and earlier and already fixed by 7df208fa/c3e9621a. Those findings age out with the old ACR tags. npm is build-time only here: CMD is `node dist/index.js`, the HEALTHCHECK is `node -e`, the container app spec overrides no command and probes over HTTP, and src/ contains no child_process call. So remove npm rather than upgrade it - npm 11.x still vendors @sigstore/core 3.1.0, itself inside the vulnerable <= 3.2.0 range, and would clear nothing. Separately, tar 7.5.20 -> 7.5.22 in our own tree (GHSA-r292-9mhp-454m, via @huggingface/transformers -> onnxruntime-node). 7.5.22 satisfies the declared ^7.0.1, so no override is needed. fast-uri deliberately stays on 3.1.4: it is patched against all four fast-uri advisories, and ajv 8.18.0 declares ^3.0.1, so forcing 4.x would be an override against a major range. npm audit --omit=dev: 0 vulnerabilities. tsc clean, 455 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
1bf37f6761 |
feat(insights): F3 regression radar - Wilson-gated rule hit-rate drift detection (PR6)
Analyze rules only (gather stats stay per-batch-deduped until Q6 is
fixed - never silently included). Per (tenant, rule) the radar compares
the trailing 7-day hit rate (FireCount/SessionsEvaluated from the daily
RuleStats rows) against the prior 28-day baseline and fires only when
the window has >=5 hit sessions and >=20 evaluated sessions, the rate
is >=2x baseline AND the Wilson 95% intervals are disjoint
(MetricsMath.WilsonInterval/RateIncreaseSeparated, hand-computed test
vectors). Suppressed for deleted rules, rules inside the CreatedAt
grace period (<35d), rules edited inside the window, empty baselines,
and the RuleRegressionRadarDisabled app-setting kill switch.
Alert episodes live as ruleregression|{ruleId} rows in the notification
tracker (third keyspace): AddEntity-409 dedup = one bell + one
RuleFrequencyRegression ops event (category Tenant, Warning, full
numbers, dual-registered in the web OPS_EVENT_TYPES) per episode;
numbers refresh while active without moving FirstNotifiedAt; episodes
re-arm under 1.5x baseline, on fire-stop, or via the tracker's 30d
retention prune (spec-intended reminder). On-fire dimension correlation
(osBuild/model/agentVersion/imeVersion; hit set from a RuleResults
range scan intersected with the window sessions) reports both shares
with the "correlated, not necessarily causal" wording contract or
honestly "no clear dimension concentration". Radar runs in the daily
maintenance pass after the rule-stats aggregation, anchored on
yesterday (whole days only).
Surfaces: regressions[] in the rule-stats response (MCP get_rule_stats
inherits it), tenant bell deep-linking to the rule card, and the
analyze-rules page gains a densified 30-day fire sparkline plus a
regression badge while an episode is active. OKF doc
backend/rule-regression-radar.md + index/log.
Backend 3474/3474 (27 new: Wilson vectors, every suppression branch,
window boundary dates, re-arm matrix, dimension gates, tracker
round-trips incl. tri-states, keyspace dedup/prune, wording pins),
web tsc + vitest 627/627, MCP tsc + 455 passed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
88da317bf4 |
feat(insights): F2 surfaces - device-history banner, fleet FTR panel, API routes, MCP tool (PR5)
Three read routes over the PR4 data: metrics/device-history
(MemberRead + tenant query param - one route serves the web banner AND
MCP, GA cross-tenant via ?tenantId= with a sessionId-based owner
fallback; history:null is a normal outcome), metrics/device-journeys
and global/metrics/device-journeys (days selector honored because FTR
counts are additive - window rate = ratio of summed daily rows; the
tenant-less global variant serves the "global" partition and discloses
repeatDevices:null). Attempt numbers are server-computed
(DeviceJourneyCalculator.ComputeAttemptNumber: chain position within
the journey; live sessions via a virtual non-successful attempt so the
redeploy and 30d-gap rules apply) - no consumer re-derives journey
semantics. Repeat-devices violator list: last journey >=2 attempts with
newest terminal ref in-window, attempts desc then recency, cap 10,
failure reason from the newest failed attempt (bounded point-reads).
Web: DeviceHistoryBanner on session detail ("Attempt N for this device
- View history", expandable terminal chain, durations strictly the
authoritative durationSeconds, WG/DevPrep/admin-marked badges) and the
Fleet Health "First-time-right" section (rate gated at >=20 completed
journeys with disclosed n, junk-serial exclusion note, Monday-UTC
weekly trend, attempt histogram, repeat devices) following the 7/30/90
selector. MCP: get_device_history (serial mode / fleet FTR mode,
role-aware routing, all roles - GA_FULL privilege snapshot extended
deliberately). Docs: OKF backend/device-journeys.md + index/log.
Backend 3447/3447 (14 new: attempt-number vectors incl. gap boundaries
and live sessions, additive sums, days clamping, repeat-device
gate/sort/cap), web tsc + vitest 627/627, MCP tsc + 455 passed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
6de21e821a |
feat(insights): F1 surfaces — session lane, fleet panel, API routes, MCP tool (PR3)
Insights expansion PR3 (tasks/insights-expansion-spec.md §F1 "Surfaces").
Completes F1: schema (PR1) -> persistence/aggregation (PR2) -> surfaces.
- Rolling range rows: the sweep now also writes "rolling30|{class}"
aggregate rows — range statistics over the window's actual sessions. A
median of per-day medians is not the range median, so the fleet panel
reads these and is deliberately fixed to "last 30 days"; the daily rows
stay in the payload for trends/MCP.
- Routes (all registered in EndpointAccessPolicyCatalog):
GET sessions/{sessionId}/time-attribution (MemberRead + tenant query
param, GA fallback via FindSessionTenantId; breakdown=null is SUCCESS —
a missing lane is a normal state, not an error), GET
metrics/time-attribution (MemberRead) and GET
global/metrics/time-attribution (GlobalReadOrAdmin + tenantId filter),
both returning { windowDays, classes (rolling), daily }.
- Web: TimeAttributionLane under the session-detail phase timeline —
proportional stacked bar of segment totals incl. the explicit
unattributed remainder (bar always sums to the session's authoritative
duration), ESP-blocking apps with critical-path occupancy, reboot badge
(cross-cutting, never a bar slice), quality-flag chips; fetched
fail-soft for terminal sessions only. Fleet Health gains the "Time
attribution" section: per-class stacked medians gated at n>=20
("insufficient data (n=...)" below), flagged/missing exclusions
disclosed, and a top-blocking-apps table with class-scoped rows and
"up to X (p75 Y)" what-if wording. Deliberate deviation: the spec's
per-app trend column is omitted — per-day app rows are too sparse under
the daily n>=5 gate; revisit once rolling history exists.
- MCP: get_time_attribution (one tool, session + fleet mode, role-aware
routing like get_app_install_metrics, registered for all roles) — the
privilege-leak-guard catalog snapshot extended deliberately.
- Docs: OKF concept doc backend/time-attribution.md (invariant, windows,
segment mapping, positive-evidence blocking, Q7 reboot spans,
persistence + surfaces) registered in index.md + log.md.
Backend 3382/3382 (+1 rolling-aggregate test), web tsc clean + vitest
627/627, MCP tsc clean + 455 passed. Post-deploy: web deploy +
MCP redeploy (new tool + docs corpus).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
a5d12ba1d0 |
chore(ci): unify Node toolchain on 22 across CI + engines pins
Node 20 reached EOL (2026-04-30); Node 22 is the current maintenance LTS and already the runtime the MCP container ships (node:22-slim). Straighten the version skew so per-component CI major == deploy runtime == engines floor: - ci.yml: web, mcp and guardrails setup-node 20 -> 22 - combine-rules.yml: setup-node 20 -> 22 - mcp package.json: add engines.node ">=22" (matches Dockerfile) - web package.json: add engines.node ">=22" (first explicit pin; SWA/Oryx no longer picks a Node version implicitly) engine-strict stays off (no .npmrc), so the pins are advisory and cannot hard-break local installs on other Node versions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
540f543834 |
feat(agent): extend historic IME-replay suppression to app events + app aggregation
Part 2 of the eaf3d8c4 fix (follows |
||
|
Oliver Kieselbach
|
088c3b0479 |
fix(agent): suppress historic IME-log script replay + timeline-consistent durations
Session eaf3d8c4: a previous enrollment's IME logs survived on disk (dated 2026-07-16, 7 days old) and the agent replayed them at startup — 156 phantom script_completed events in a 7-minute session, each showing ~170h durations (clock-clamped completion minus RAW stale start), plus an immediateUpload flood (3x ingress_backpressure) and RemediationScriptCount inflated to 156. Agent: - Script events whose source line is >24h stale are suppressed entirely; a one-shot historic_script_replay_detected Info event (new EventType) dates the replayed window via earliestRejectedSourceTimestamp. Future-skew rejections and clock-only fallbacks still emit. - durationSeconds now uses a timeline-consistent pair: on rejected-timestamp emissions the raw source completion pairs with the raw start (true runtime instead of cross-timeline garbage). 24h plausibility backstop omits implausible values; same consistency for script_timeout_suspected. - Stale pending platform-script slots (>24h older than the current start line) are discarded so an old orphaned start can't pair with a fresh run (extracted HandlePlatformScriptStarted as internal test seam). Web: partitionHistoricScriptEvents hides legacy-agent replay events (muted "N historic executions hidden" note) and the reducer blanks >24h durations — covers sessions recorded before the agent fix rolls out. Backend: IsHistoricScriptReplay guard keeps replayed events from inflating Platform-/RemediationScriptCount during the rollout window. Registered historic_script_replay_detected in Constants.EventTypes, MCP resource-catalog (script_events) + KEY_EVENT_TYPES, gather-rules eventTypes. Tests: 8 agent (suppression/one-shot/raw-pair/backstop/slot-hardening), 8 web (partition/blanking), 10 backend (predicate). Agent 2026, backend 3210, web 595, MCP 449 all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
e7ea431ccf |
feat(esp): recovery story for terminally reported ESP failures (session 4910a5a5)
DeviceSetup/Apps failed, the user pressed "Try again" ~23 min later, apps recovered to 10/10 and DeviceSetup resolved - yet the advisory_completion window un-defanged the stale failure and failed the live enrollment, because the advisory variant was exempt from all re-arm/rebase guards. Engine fix (DecisionCore): - New facts EspAdvisoryFailureResolvedUtc + EspAdvisoryFailureCategory (state-borne; deadline payload would be lost on re-arm/rebase). - TryResolveAdvisoryOnCategoryRecovery in both ProvisioningComplete handlers: category match sets the fact + emits esp_failure_advisory_resolved. - Guards now (!hasAdvisoryAnchor || advisoryResolved): enforcement-progress re-arm, hello-never-observed promote and the reboot rebase apply again once the failure demonstrably un-happened; unresolved advisories keep the legacy un-defang exactly. - Truthful terminal reason esp_recovered_without_completion_evidence (+ failureRecovered=true, LastFailureTrigger=DeadlineFired so the likely-stuck app promotion stays off). Story events (agent tracker): - esp_failure_retry_detected: failed subcategory leaves "failed" after the terminal fire - consistent with the user pressing "Try again". - esp_failure_recovered: previously failed category reaches success. - EspFailureDetectedEventArgs.LikelyCulpritApps (never-started first, cap 3) rides the EspTerminalFailure payload so esp_failure_advisory / enrollment_failed name the culprit app directly. Surfacing: MCP resource-catalog, web timeline badges (Recovered / User retry pills, culprit chip on advisory), gather-rule autocomplete, ANALYZE-ESP-006 "ESP Failure Recovered After User Retry". Tests: AdvisoryRecoveryTests (13, incl. the 527/611/635/671 field-ordinal replay that no longer fails the session), EspFailureRecoveryStoryTests (5), snapshot-builder guards 18 -> 20. All suites green (DecisionCore 493, V2.Core 2018, Backend 3201, web vitest 589, MCP 449 incl. drift). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
9e24b92fbf |
feat(agent): ESP policy-provider stall detector + Autopilot registry guardrail
Customer field case: pre-provisioned devices with the ConfigMgr client as a
blocking app occasionally park the user ESP at "Apps (Identifying)" for days.
Root cause (verified against the EnrollmentStatusTracking CSP doc): only a
"ConfigMgr" provider is registered under Device\Setup\Apps\PolicyProviders
(no "Sidecar" = IME) and TrackingPoliciesCreated never becomes 1 - a wait the
CSP defines WITHOUT any timeout. Renaming the key to Sidecar unblocks
immediately. Microsoft documents that ESP providers are not aware of each
other and that pre-provisioning is unsupported with co-management.
New always-on esp_policy_provider_stalled tripwire, generalized to any
provider in both kinds (Setup/Apps TrackingPoliciesCreated != 1,
DevicePreparation InstallationState not in {2,3}):
- EspPolicyProviderProbe: fail-soft registry read of the
EnrollmentStatusTracking mirror (device + user-SID scopes), defensive
value parsing (missing/unparseable = incomplete), ScopedOverride test seam.
Research and CSP contract quotes captured in the XML doc header.
- EspPolicyProviderStallDetector: pure EvaluateStallTransition on a 15-min
continuous-incompleteness dwell (resets on completion/disappearance;
absent key = normal early enrollment), one-shot per provider,
StartupEventGate fingerprint over the sorted stalled key set for
cross-restart dedup. Warning + ImmediateUpload via InformationalEventPost;
purely observational, no decision-engine coupling, no config toggle,
no ConfigVersion bump (disk_space_low precedent).
- EspPolicyProviderStallHost: 60-s wall-clock tick, deliberately outside the
StallProbe idle gate (idle clock resets on session activity) and the
idle-stopped periodic collectors.
- Registered in Constants.EventTypes, MCP resource-catalog (health_events),
web gather-rule autocomplete (stall category).
Guardrail companion: SOFTWARE\Microsoft\Windows\Autopilot joins the
gather-rule registry allowlist ("Autopilot / OOBE / Setup") so tenants can
build custom rules on the subtree (e.g. not_exists on the Sidecar provider
key); generated web mirror regenerated via combine.js; positive + sibling-
spoof guard tests added.
Docs: OKF concept doc agent/esp-policy-provider-stall.md (+ index/log).
Tests: 22 new detector tests (pure transition matrix + emission surface incl.
the ConfigMgr-only customer scenario and gate-across-restart dedup); agent
suite 2013/2013, web vitest 589/589 + tsc clean, MCP 449 passed incl.
event-types-drift.
Both the detector and the embedded allowlist take effect with the next agent
release.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
90012d9a66 |
feat(agent): verify Secure Boot CA-2023 state against firmware, not just servicing registry
Customer report: ANALYZE-SEC-001 warned on devices whose firmware already carried the Windows UEFI CA 2023 certificate — the agent only read the SecureBoot\Servicing registry value, which mirrors the Windows Update rollout and lags behind reality (OEM factory image, WinCS, WU not run yet). Agent: new UefiSecureBootCertReader reads the UEFI db/KEK variables natively (GetFirmwareEnvironmentVariableExW; SeSystemEnvironmentPrivilege enabled on the process token and restored — first privilege-adjustment code in the repo). Fail-soft, never throws; legacy BIOS maps to not_uefi. secureboot_status gains per-certificate booleans, uefiFirmwareReadStatus, and the one-sided marker uefiCa2023FirmwareConfirmed (emitted only on positive confirmation, never as false). Rule v3.0.0: not_exists precondition on the marker suppresses the rule when the firmware confirms the certificate; the marker's absence keeps old agents on exact v2 behavior, so the rule is safe to seed before the agent rollout. Firmware-verified-missing raises confidence (+20 db, +10 KEK 2K CA 2023). Verified: full solution build; backend 3201, agent 1988, web 582 tests green; tsc/ajv clean; elevated cross-check on real firmware matches Get-SecureBootUEFI on all seven certificate probes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
e8ef4f53ae |
feat(agent): OOBE-state observability via WinRT SystemSetupInfo
Sample Windows.System.Profile.SystemSetupInfo.OutOfBoxExperienceState
(build >= 17763, SYSTEM-callable, CLR-native WinRT projection — no NuGet)
as a purely observational signal, mirroring the bootstrap guard marker:
- oobeStateAtAgentStart on agent_started and agent_late_start payloads:
makes late starts provable instead of heuristic ("completed" on a
first_run = OOBE was already over when the agent arrived) and validates
the bootstrap OOBE guard fleet-wide.
- One-shot oobe_state_completed trace event on the in_progress->completed
flip, piggybacked on the DesktopArrivalDetector 30s poll: an
owner-independent desktop corroboration for DAD-starved sessions
(4d5a0b78 class — WTS+WMI owner resolution dead, desktop_arrived never
fires). Often absent in healthy sessions since the DAD timer is
disposed at desktop arrival, which is when the flip lands.
OobeStateReader is fail-soft (never throws, "unavailable" on missing
contract); the flip check has its own try/catch so it cannot pollute the
DAD wmiErrorCount liveness diagnostics, "unavailable" never overwrites
the baseline, and the one-shot flag survives ResetForRealUserSwitch.
No decision-engine involvement, no config/backend changes. Event type
registered in Constants.EventTypes, MCP resource-catalog (phase_events)
and web gather-rule autocomplete.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
b2d2d37c6f |
feat(realmjoin): detect aborted RJ ESP (missing 110) and release the completion gate
An interactive logon during the RealmJoin first deployment (phase 100/101)
makes RJ reclassify the run as a secondary-user deployment, whose branch
writes 200/210 and never CompletedFirstDeployment (110) — no crash, no
reboot request. The V2 completion gate waited for exactly 110, so these
sessions hung until the 60-min hard timeout (session 224b2087; siblings
558bf067/3cc18724 delayed ~50 min the same way).
- promote realmjoin_phase_changed to a typed RealmJoinPhaseChanged signal
(dual-emitted by the adapter) so RealmJoinFacts.LastDeploymentPhase is
persisted across agent restarts
- abort rule: prev in {100,101} and current in {200,210} with no outcome
yet -> emit realmjoin_first_deployment_incomplete (Warning), cancel the
hard-timeout deadline, release the gate with outcome
FirstDeploymentIncomplete; also evaluated on the RealmJoinDetected
replay path (restart swallows the transition)
- deliberately NOT triggered by 101->0 (RJ service restart may still
complete the first deployment) nor by a first observation of 210
(session 6f1959c0: RJ already completed before agent boot)
- fix realmjoin_timeout always claiming "last phase: 0" - the persisted
phase now feeds the message
- backend EnrollmentTimeoutClassifier counts the new event as a gate
terminal; MCP resource catalog dual-synced
- CompleteIfDeferredOrBookkeep widened to a leading-effect list
Verified by replaying session 224b2087's signal log: gate releases at
15:05:17 (101->200) instead of hanging until the 15:54 deadline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
e0ad51b965 |
refactor(urls): one guard-enforced URL registry per component, agent downloads move to the alias
Every well-known host -- own subdomains, both blob accounts, the API origin, Graph, Entra login -- now lives in exactly one registry per component: Constants.cs (*BaseUrl) for C#, utils/config.ts for the web app, src/config.ts for the MCP server. lib/hostRouting.ts derives its hostnames from the registry and middleware.ts imports from it, retiring what turned out to be a fourth independent copy of the www/portal host pair. Three new guard tests (backend xUnit, web vitest, MCP vitest) scan non-test source and fail on any enforced host outside its registry. This is the lesson of the EU cutover: the registry existed then too, but LatestVersionsService (feeds the portal's version badge) and ValidateBootstrapCodeFunction carried their own literal copies of the legacy blob host and were missed -- kept working only by the fail-soft legacy mirror in build-agent.yml, whose silent failure would have pinned badge and self-update to stale bits with a green release. Both guards proved themselves on their first run, flagging the middleware host copy and a literal in an MCP tool description. Test files stay exempt on purpose: literals there are independent oracles, and ValidateBootstrapCodeResponseShapeTests now pins the full alias URL against the Constants-built one so registry changes must be acknowledged consciously. The two legacy readers plus the agent SelfUpdater now use Constants.AgentDownloadBaseUrl (download.autopilotmonitor.com/agent); the alias was verified live for version.json, version-v2.json and both ZIPs before the switch. The portal's bootstrap allow-list accepts the alias AND the legacy host (AGENT_DOWNLOAD_HOSTNAMES) for already-deployed customer scripts, which imposes a deploy order: web before or with backend. The legacy keepalive probes and the release mirror stay untouched -- they exist for exactly this transition. Also: the bootstrap short-link now targets www directly instead of riding the registrar's apex 301, and the welcome-mail docs button points at docs. instead of bouncing through www./docs. Suites: backend 3194, agent core 1925, DecisionCore 474, web 582, MCP 449 -- all green; next build verified (it, not tsc, proves the next.config.ts registry import). Docs: docs/url-registry.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
bbb55034e3 |
feat(versioning): one version contract for agent, backend and MCP
All three components now report <major>.<minor>.<build> plus a 7-char commit and a build timestamp. The backend had been pinned at a hand-written 1.0.0 since inception and starts at 1.5.1000; the MCP server's package.json version was bumped by hand and starts at 1.5.300, and it now reports its own commit on /health instead of only the documentation bundle's. The build number keeps coming from a counter blob reserved before the build with ETag-CAS, but there is now one implementation instead of two copies: .github/scripts/Request-BuildNumber.ps1, called by all three workflows and by the local build.ps1. It lives under .github/ because scripts/Deployment/ is gitignored (those scripts carry SAS tokens) and would therefore be absent from a CI checkout; scripts/Build/ and scripts/Release/ are shadowed by the [Bb]uild/ and [Rr]elease/ artifact ignores. The counter cannot be replaced by the published manifest, which is the obvious simplification to reach for: version.json is written only on stable publish, so two consecutive dev builds would derive the same number and the second would overwrite the first's artifact. git rev-list --count fails the same way, two builds of one commit collide. Both deploy workflows now poll their health endpoint until it reports exactly the version and commit just deployed, and publish versions/<component>.json only then. A green deploy step alone means the package was accepted, not that it is being served -- so a manifest that disagrees with the live endpoint now means the running instance drifted (rollback, stale revision) rather than a pipeline that stopped halfway. The agent manifest gains the same commit and buildUtc fields additively; the bootstrap reads it by field name, so older installers ignore them, and sha256/bootstrapVersion stay untouched. Requires the versions container plus AZURE_BLOB_VERSIONS_SAS_TOKEN and the two seeded counters (backend 999, mcp 299) before the first deploy run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
fa640ebe61 |
fix(security): resolve CodeQL log-forging findings and suppress by-design queries
Bucket B (real fixes) + Bucket A (config) of the CodeQL alert triage. Log-forging (cs/log-forging, 8 sinks): add Helpers/LogSanitizer that strips CR/LF and other control characters, and apply it to the logged copy of every user-controlled value in AuthenticationMiddleware, PolicyEnforcementMiddleware (Invoke + DecideAsync), McpQuotaEnforcementMiddleware and TableUserUsageRepository. Only the logged value is sanitized — routing, authorization and storage keys are untouched, so tracing fidelity for legitimate values (paths, UPNs, GUIDs) is preserved. Structured logging already isolated the fields; this closes the CRLF vector as defense-in-depth and clears the query. Three genuine JS/TS fixes: - guardValidation.ts: replace the no-op `.replace(/%/g, "%")` with a real escapeRegExp() — the author intended to escape; this also closes the latent metacharacter gap (js/identity-replacement). - access-guard.ts: use a `%s` format argument instead of interpolating the UPN into the console.error format string (js/tainted-format-string). - docs-corpus.ts: strip HTML tags with a loop-until-stable stripHtmlTags() so the sanitizer is robust and recognized as complete (js/incomplete-multi-char-sanitization). CodeQL config: filter the two queries that fire structurally on this codebase, where the flagged behavior is the feature — cs/exposure-of-sensitive-information (audit/notification logs legitimately record the actor UPN / recipient email) and cs/cleartext-storage-of-sensitive-information (rate limiters key on the trusted client IP). Default suites stay active; genuine one-off false positives from other queries are dismissed individually so those queries stay sharp. Tests: Backend xUnit 3110 (+9 LogSanitizer), MCP vitest 19 (+1 tag-strip), Web vitest 49, all tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
dependabot[bot]
|
c3e9621a78 |
chore(deps): bump the mcp group across 1 directory with 5 updates (#103)
Bumps the mcp group with 5 updates in the /src/McpServer/autopilot-monitor-mcp directory: | Package | From | To | | --- | --- | --- | | [@modelcontextprotocol/sdk](https://github.com/modelcontextprotocol/typescript-sdk) | `1.28.0` | `1.29.0` | | [fuse.js](https://github.com/krisk/Fuse) | `7.1.0` | `7.5.0` | | [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) | `22.19.15` | `22.20.1` | | [tsx](https://github.com/privatenumber/tsx) | `4.22.4` | `4.23.1` | | [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) | `4.1.2` | `4.1.10` | Updates `@modelcontextprotocol/sdk` from 1.28.0 to 1.29.0 - [Release notes](https://github.com/modelcontextprotocol/typescript-sdk/releases) - [Commits](https://github.com/modelcontextprotocol/typescript-sdk/compare/v1.28.0...v1.29.0) Updates `fuse.js` from 7.1.0 to 7.5.0 - [Release notes](https://github.com/krisk/Fuse/releases) - [Changelog](https://github.com/krisk/Fuse/blob/main/CHANGELOG.md) - [Commits](https://github.com/krisk/Fuse/compare/v7.1.0...v7.5.0) Updates `@types/node` from 22.19.15 to 22.20.1 - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) Updates `tsx` from 4.22.4 to 4.23.1 - [Release notes](https://github.com/privatenumber/tsx/releases) - [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs) - [Commits](https://github.com/privatenumber/tsx/compare/v4.22.4...v4.23.1) Updates `vitest` from 4.1.2 to 4.1.10 - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md) - [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.10/packages/vitest) --- updated-dependencies: - dependency-name: "@modelcontextprotocol/sdk" dependency-version: 1.29.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: mcp - dependency-name: fuse.js dependency-version: 7.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: mcp - dependency-name: "@types/node" dependency-version: 22.20.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: mcp - dependency-name: tsx dependency-version: 4.23.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: mcp - dependency-name: vitest dependency-version: 4.1.10 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: mcp ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Oliver Kieselbach
|
7df208fac9 |
fix(deps): pin patched transitive deps via overrides to clear Dependabot alerts
Force the security-patched versions of transitive dependencies without downgrading their parents (next 15, @modelcontextprotocol/sdk 1.28), which is what `npm audit fix` would otherwise do: - sharp ^0.35.0 (web via next, mcp via @huggingface/transformers) — libvips CVEs - fast-uri ^3.1.4 (mcp via ajv) — host confusion - hono ^4.12.27 (mcp via sdk) — jsx context/XSS/header advisories - @hono/node-server ^2.0.5 (mcp via sdk) npm audit: 0 vulnerabilities in both projects. Builds clean; MCP 447 tests and Web 572 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
ee58242d33 |
fix(mcp): tell callers to query the English-only corpora in English
A German question returns zero results from search_docs, not poor ones: both the documentation and the rules are English, and all-MiniLM-L6-v2 is an English-only model, so the query vector lands nowhere near any chunk. For a German-speaking admin driving an MCP client in their own language, the tool is effectively blind and gives no hint why. The caller is itself a language model, so the cheapest correct fix is to say so: both tool descriptions now instruct it to translate the question to English before searching and to answer the user in their own language. No infrastructure, no model change, no ranking impact. A multilingual embedding model would handle it natively and was considered, but it is substantially larger — which matters on a scale-to-zero container — and it would change English retrieval as well, so it would have to be measured against scripts/eval-docs-search.ts rather than swapped in on the assumption that more languages cannot hurt. Verified: 447 tests pass, tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
8305f5c850 |
feat(mcp): time the boot so the cold start can be attributed
A cold start after scale-to-zero measures 24.5s to first byte against 60ms warm, and nothing outside the process could say why. Log Analytics stamps every console line of one ingestion batch with the same time, so the boot sequence collapses to a single instant however it is queried, and piping stdout locally buffers until exit. So the server times itself: `[boot +Nms]` marks derived from process.uptime(), which unlike an in-process timer also covers Node's own startup and module loading. Measuring the precomputed path locally already contradicts two assumptions worth recording: +860ms node started (startup + module loading) +1199ms corpora read from disk (+339ms) +1242ms index read, validated, hydrated (+43ms for 3.12 MB) +1273ms listening (+31ms) +1497ms query embedder warm (+224ms, background) The index is 43ms, not seconds — its size is not the cold-start cost, and growing the corpus is cheap. The prebaked embedder is 224ms and loads in the background, so only the first search could ever wait on it. That was the stated reason for raising the container to 0.5 vCPU, which now looks like a weaker justification than it did; the production marks will settle it. If they show the app listening after a few seconds, the rest of the 24.5s is platform activation that neither CPU nor index changes can touch. Verified: 447 tests pass, marks appear on both boot paths locally. Incidentally the run also confirmed the corpus hash guard and the staleness signal both work — the local docs repo has moved to 6333d56 while the deployed image reports d8cb7f4, so the precomputed index was correctly rejected and recomputed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
1821af6f05 |
fix(mcp): fire the search_docs keyword fallback on a weak semantic head
The live test found a third recall gap. The fallback only topped up when semantic
search returned FEWER than topK results — but the more common failure is a full
list of unconvincing ones. Querying "WOW6432Node" returned three chunks scoring
~0.29, none containing the word, while the single chunk that does was never
considered at all.
The fix is a ranking change, so it was measured rather than reasoned about. Adds
scripts/eval-docs-search.ts, which scores strategies over 2465 queries in three
deliberately different classes: every chunk heading (easy by construction, so a
regression detector), rare identifiers (the class this fallback serves), and
hand-written questions (the reality anchor — not derived from the corpus, so it
shares little vocabulary with it). Embeddings are computed once and each strategy
is a pure function over the same candidates, so sweeping a threshold costs no
extra inference.
known-item literal handwritten MRR fixed broken
append-only (before) 71.9% 66.2% 1.000 - -
weak < 0.30 71.9% 72.6% 1.000 142 0
weak < 0.35 73.7% 81.1% 1.000 335 0
weak < 0.40 76.3% 88.2% 0.944 502 3
weak < 0.45 77.2% 93.0% 0.889 613 5
0.35 is strictly dominant — better in every class, worse in none — so that is
what ships. Higher thresholds keep repairing identifier queries but start
demoting hand-written questions, the class that most resembles real use, so the
gain stops being free. When the head is weak, exact matches move to the front
while at least one semantic result is kept, so a weak-but-correct hit is never
pushed out entirely.
The measurement also resized the problem: identifier queries were succeeding only
66.2% of the time, so the reported case was one of roughly 750, not an outlier.
That is the part a single anecdote could not have shown, and the reason the
harness exists before the fix rather than after it.
Verified: 447 tests pass, including new cases for the weak path and for leaving a
confident ranking untouched. On the real corpus "WOW6432Node" now returns the
chunk that contains it at rank 1, while "how do I deploy the agent with Intune"
(0.819) and "what firewall URLs do I need to allow" (0.374) are unchanged and the
negative controls stay empty.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
62599d972c |
fix(mcp): put every doc token back into search_docs ranking
Two measured recall gaps in search_docs, both silent. 1. Truncation. MiniLM's context is 256 tokens; 87 of 324 chunks exceeded it and 13% of all tokens never reached the ranking — the tokenizer drops the tail without complaint. Concretely, in built-in-rules.md :: Apps (734 tokens) the strings ANALYZE-APP-013, ANALYZE-OFFICE-001, ANALYZE-CORR-003 and Click-to-Run all sit past the cutoff, so "proxy configuration blocking app downloads" did not retrieve that section at all. Shrinking chunks cannot fix this: the corpus ranges 2.43 to 5.81 characters per token, so no character budget bounds tokens, and a budget safe for the worst case (~620 chars) would shred prose and tables alike. So the chunk stays whole and the embedding is split instead — index() cuts long text into 250-token windows with 40 overlap, a document carries embeddings[][], and search() scores it by its BEST window. Max rather than mean: averaging would blur a long section's several topics into one mediocre direction, the same dilution that makes large context windows worse than chunking. The tokenizer is a build-time dependency only (vocabulary, not ONNX weights, ~93 ms); production boots from precomputed vectors and never loads it. 2. No general fallback. Proper nouns embed poorly — "RealmJoin" occurs in 3 chunks yet scores under the threshold. Fuzzy matching was built, measured and rejected: at a threshold low enough to surface RealmJoin (0.28), Fuse also returns 5 hits for "SCEPman", a word this corpus contains zero times. Literal containment has neither problem, and every provider already exposes it. So a short result set is now topped up via lexicalMatch on distinctive query terms, requiring two matching terms as corroboration — without that rule "how do I bake sourdough bread" matched a page about LLM providers, because "bake" is a substring of "baked into the image". Keyword hits are appended and flagged, never sorted together with cosine scores. Also sharpens the tool description with the measured reason to prefer this over fetching the docs site (~1k tokens for topK=3 against ~9k raw / ~600k rendered, version-matched, and matching on meaning), and replaces the growing tail of positional docs parameters with a DocsSearchBundle. Verified: token coverage 87% -> 100%; "Office Click-to-Run install never finished" 0.318 -> 0.442; "proxy configuration blocking app downloads" absent -> 0.559 at rank 1; the ten reference queries keep byte-identical scores and rank 1 (short chunks are one window, so the same vector as before — no long-chunk bias). The fallback is not a dead path: across 904 rare corpus tokens queried alone it contributes hits for 431, while negative controls stay empty. Precompute output is accepted by the boot validator and a mutated corpus still rejected; index 2.63 -> 3.12 MB for 324 chunks / 424 vectors. The rules corpus gains the same treatment unplanned: 128 rules now carry 172 vectors. 445 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
63ea2e3cee |
feat(mcp): search_docs — customer documentation as a third search corpus
The MCP server could answer telemetry questions ("why did this session fail")
but not product ones ("how do I deploy the agent", "where is my data stored").
Those answers already exist in the autopilotmonitor-docs bundle; search_docs
makes them semantically searchable.
Deliberately a separate index from search_knowledge: rules and prose answer
different questions, and sharing one topK would dilute both. Registered for
every role — published documentation carries no tenant data — and skipped
entirely when no corpus was baked in, so a doc-less build advertises no
broken tool.
Corpus handling:
- Chunked at h2/h3 and <details>/<summary>, each chunk prefixed with a
"Page > Section > Subsection" breadcrumb so a mid-page hit is self-describing.
- Newlines are normalized before any parsing. This is a correctness
precondition, not cosmetics: the bundle ships CRLF and JS `.` does not match
\r, so /^(#{2,3})\s+(.*)$/ matched zero headings — silently, degrading every
page to unlabelled paragraph splits.
- Two size exceptions: tables are never split (a split table leaves `| --- |`
rows with no header), and unbroken changelog lists split at line level (left
whole, their tails fall outside the model window and become unsearchable).
- Short sections are kept — a one-line answer is often the best hit, and the
breadcrumb supplies the context a short body lacks.
Build and delivery:
- Docs are embedded at image build time into an optional third
PrecomputedSection. Optional so a checkout without the docs repo does not
invalidate the rules and event-type vectors; the reverse (server has docs,
file does not) is staleness and is rejected.
- The deploy workflow checks the public docs repo into the build context and
passes its SHA as DOCS_COMMIT, which /health reports alongside the chunk
count — editing the docs does not update a running server, so staleness is
made checkable rather than memorable.
- Embeddings are rounded to 6 decimals on serialize: 55% smaller index (2.63
vs 5.88 MB) for a ~1e-5 norm error against a 1e-3 tolerance. indexPrecomputed
now asserts unit norm, since the hydrate path never checked it.
Infra: the Bicep template was two changes behind the live deployment (EU
cutover and ACR hardening, both applied directly in Azure). Reconciled against
the running resources — managed-identity image pull replaces the admin
credentials and the acr-password secret — and infra/ is no longer gitignored so
it stops drifting. Container raised to 0.5 vCPU / 1.0 GiB for the cold-start
query-embedder load. The template does not model the mcp.autopilotmonitor.com
custom domain, so a full deployment would remove it; the header warns about it.
Verified: 428 tests pass; 45 pages produce 324 chunks; sample queries hit the
right page top-1/2; all 8 derived URLs return 200 live; the precompute output
is accepted by the boot validator and a mutated corpus is rejected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
5a294ac4f6 |
fix(mcp): patch transitive protobufjs/tar/body-parser advisories
Dependabot alert #62 (protobufjs 7.6.4, DoS via infinite loop in .proto option parsing, GHSA fixed in 7.6.5) plus the tar (critical) and body-parser (low) advisories npm audit surfaced alongside it. All three are transitive under @huggingface/transformers and resolve within their existing semver ranges, so only the lockfile changes. Lockfile also picks up the pending 1.5.0 -> 1.5.1 package version sync. npx tsc --noEmit clean, vitest 400 passed / 41 skipped, npm audit reports 0 vulnerabilities. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
a97888ebd5 |
fix(agent): neutral aggregated 2800 observations; reboot claim gated on observed reboot
Rework after the first real enrollment (session b2e890c1) exposed three
defects in
|