mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-24 12:42:38 +00:00
v2.0.1415
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oliver Kieselbach
|
e1b03de5b3 |
fix(agent): recover ESP completion after a forced mid-ESP reboot
Seven sits-d Cloud PCs enrolled the same day with identical configuration (~138 required apps at the user-ESP gate, DeviceGuard/DmaGuard device-assigned). The two that finished AccountSetup before the coalesced reboot landed succeeded; the five the reboot killed in Stage=EspAccountSetup never derived a completion again and ended Incomplete on healthy devices. Two defects behind that: ShellCoreTracker.BackfillRecentEspExitEvents() has existed since session 772fe502 but no caller ever invoked it, so the Shell-Core watcher only saw records written after Start(). Every agent restart silently dropped an ESP exit (62407) or Hello-wizard start (62404) from its downtime. EspAndHelloHost.Start() now calls it, and the fixed 5-minute window is sized to the actual downtime when the previous run died to a reboot (LastBootUtc, clamped to 360 min). The user-apps-settled synthesis ran only on the EspExited edge. When the ESP page exits with apps still in flight the single attempt misses and nothing re-checks — 8110e262 and a89aac2d both reached 138/138 apps, 0 failed, and still never completed. The exit edge is now recorded and the synthesis re-evaluated from ImeLogTracker.OnAppStateChanged. No new completion path: the gate conditions are unchanged, this only gives the existing conservative check a second look. The fire-once claim moves to Interlocked (edge and re-check run on different threads) and the re-check does not re-emit app_install_starved. Rules: ANALYZE-ESP-005 gains on_event:session_stalled so a hung session produces a finding at the 60-minute stall instead of staying finding-less for hours (verified against 8110e262 — fired, confidence 95). New ANALYZE-APP-016 turns the already-emitted app_install_starved events into a customer finding and, unlike ANALYZE-APP-015, fires even on a succeeded session: 3b9291aa was recorded Succeeded with 99 required apps that never started installing. ANALYZE-APP-015 and ANALYZE-ESP-001 were considered for the stall trigger and deliberately left alone — APP-015 is the documented worked example of the terminal-precondition trap, and ESP-001's phase_duration gate on a still-open phase is unverified mid-run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
22a8c7a51d |
rules: hard-block high-frequency event types as evaluateOn on_event triggers
A warning does not stop an end user — telemetry-cadence types (performance_snapshot, agent_metrics_snapshot, download_progress, network_state_change/connectivity_check, log_entry, agent_trace, stall_probe_check) would turn every ingest batch into an analyze run. Now enforced in three layers: - AnalyzeRuleTriggers.BlockedOnEventTypes (code-only enforcement): CRUD write is rejected with an explicit message, and OnEventTypes() drops blocked types at runtime so even a persisted row (pre-validation data, direct table write) is inert for the trigger registry and the engine alike. - rules/guardrails.json gains blockedInterimTriggerEventTypes as the display/ pre-flight mirror (baked into the MCP RULE_GUARDRAILS via combine.js); a parity test pins mirror and code set against drift — same pattern as the gather hard blocks. - MCP validate_rule: the high-frequency finding is now an ERROR sourced from the baked mirror, not a hardcoded warning. Suites: Backend 4155, MCP 548. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
72136f0273 |
backend+mcp+web: evaluateOn interim triggers for analyze rules
Implements docs/rules/analyze-rule-triggers.md (spec approved today): - AnalyzeRule.EvaluateOn (absent = [enrollment_end], today's behavior bit-for-bit) with the shared AnalyzeRuleTriggers grammar/matching helpers; grammar enforced in tenant rule CRUD and the analyze-rule JSON schema. - RuleEngine: AnalyzeRunContext (Terminal/Interim/Reanalyze), interim rule filter, KO suppression on interim runs, and RuleResult update-semantics replacing the permanent dedupe freeze — interim rows refresh, the terminal pass finalizes or resolves them (ResolvedAt, kept for audit), Reanalyze preserves lifecycle markers. - Notification dedupe moves onto RuleResult.NotifiedAt (one send per session+rule, stamped after a successful send; survives the manual reanalyze rebuild). - Handler: whiteglove_sealed / interim_trigger reasons — notify + SignalR yes, IssuesDetected + rule stats terminal-only (Rule Regression Radar baselines clean). - Ingest: InterimTriggerRegistry (per-tenant 5-min TTL cache, fail-soft) gates the interim_trigger enqueue (batch eventType intersection) and the whiteglove_sealed enqueue on the first genuine WhiteGlove seal. - Storage: EvaluateOnJson on AnalyzeRules; FirstDetectedAt/LastEvaluatedAt/ IsInterim/ResolvedAt/NotifiedAt on RuleResults (nullable — legacy rows read as final). - MCP: three authoring lints (terminal-precondition trap, unknown/high-frequency on_event types, KO-on-interim), authoring guide + baked schema updated; get_session_summary marks interim and excludes resolved findings. - Web: preliminary badge on interim findings, resolved findings behind a toggle, issue counts exclude resolved (analysis section + diagnosis flow). - ANALYZE-ID-004 v1.1.0: first interim-enabled rule (on_event:hybrid_login_pending), threshold raised to 75 so only repeated overdue-login signals fire mid-run. Suites: Backend 4152, MCP 548, Web 762 + tsc + lint clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
9c96881242 |
docs: spec for analyze-rule evaluateOn triggers (interim evaluation)
OKF concept doc (status SPEC): opt-in interim analyze runs via evaluateOn (enrollment_end default, whiteglove_sealed, on_event:<type>), RuleResults update-semantics replacing the permanent dedupe freeze, KO/stat suppression on interim runs, notification marker decoupled from row existence, and the validate_rule lints that keep interim rules monotonic. Closes the design for both the sits-d WhiteGlove notification gap and the esa never-terminal stuck-session gap with one mechanism. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
2f8ad82b0a |
gather-rules: mirror hard blocks in portal/MCP, stamp author on PUT-upsert; agent: bootstrap retry tests
Codex follow-ups to
|
||
|
Oliver Kieselbach
|
1ebc2675b1 |
gather-rules: WQL property projections, portal enable-gating, author from JWT
- WMI guard accepts SELECT <property-list> FROM <class> for any class already
allowed via SELECT * (strict data subset); same algorithm in agent, portal
validator, and MCP validate_rule (incl. trim fix); guardrails.json unchanged
- Portal refuses to enable custom rules with not-allowed targets: create lands
disabled, invalid edit force-disables, "Blocked on devices" badge, GitHub-issue
hint for allowlist requests; agent stays the enforcement boundary
- Rule Author is stamped from the creator's token (display name > UPN) at create
and preserved through every update, incl. cross-tenant GA edits
- bumpVersion keeps the segment count ("1.0.0" -> "1.1.0")
Customer feedback: SELECT BatteryStatus FROM Win32_Battery + on_change was
impossible because SELECT * re-emits on every charge-percent fluctuation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
75197a4184 |
Logparser pattern testing with the agent's exact .NET engine (test_log_pattern) (#125)
* updated commands * feat(backend): logparser pattern test endpoint with agent-exact semantics POST /api/rules/gather/test-pattern evaluates a logparser gather-rule regex against pasted sample lines exactly the way the agent would on the device. Motivated by a field incident: a customer verified a pattern with a PHP regex tester and the .NET engine matched differently. - CmTraceLogParser moves Agent -> AutopilotMonitor.Shared.Logging so agent (IME tracker, logparser collector, stall probe, IME matching mode) and backend run the SAME parsing implementation. Pure mechanical move, agent behavior unchanged (V2.Core suite 2147/2147 green). - TestLogPatternFunction mirrors LogParserCollector verbatim: Regex(Compiled, 1s timeout, NO IgnoreCase -> case-sensitive), first match per line, group "0" excluded / unsuccessful groups omitted, format cmtrace (default, match against parsed message) vs text (raw line), per-line outcomes incl. parse_failed/regex_timeout, and the agent's "set parameter format=text" hint verbatim. Pure compute, no tenant data. Caps: 200 lines, 8KB/line. - Policy: TenantAdminOrGlobalReader (same tier as the analyze dry-run). - Tests pin the parity showcases: case-sensitivity, envelope invisibility in cmtrace mode, optional-group omission, format hint, request validation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(mcp): test_log_pattern tool + logparser lint and guide hardening - test_log_pattern: wraps POST /api/rules/gather/test-pattern — tests a logparser regex against pasted sample lines with the agent-exact .NET semantics (case-sensitive, cmtrace/text mode). Description steers clients away from JS/PHP/Python regex testing. - validate_rule: logparser parameter lint (missing/non-compiling pattern, unknown format value = silently cmtrace on the agent) + info pointing to test_log_pattern; nextStep for valid logparser drafts routes there too. - rule_authoring_guide: logparser section now states the matching semantics (case-SENSITIVE, .NET, first match per line, cmtrace default) and two new pitfalls (engine mismatch, plain-text log under cmtrace default). - Catalog snapshot + handler/validator tests extended (MCP suite 532 green). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(okf): test-pattern endpoint + shared CmTraceLogParser in rule authoring surface Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
3c1229a301 |
MCP rule authoring: deterministic context, validate_rule, and session dry-run (#124)
* feat(backend): dry-run endpoint for draft analyze rules (rule authoring)
POST /api/rules/analyze/dryrun evaluates a DRAFT analyze rule against one
session's events and returns a full diagnostic trace: per-condition
matched/evidence (incl. per-eventType occurrence counts), per-factor
confidence breakdown, verdict, and the would-be MatchedConditions map for
{{token}} interpolation preview. Persists nothing: no RuleResult row, no
session-status change, no notifications.
- RuleEngine.DryRun.cs: DryRunRuleAsync / DryRunEvaluate share the exact
evaluators (EvaluatePrecondition/EvaluateCondition/EvaluateConfidenceFactor)
with the production path but evaluate ALL conditions instead of breaking on
the first failed required one; parity with EvaluateRule is pinned by tests.
- DryRunAnalyzeRuleFunction: TenantAdminOrGlobalReader (read-in-effect POST),
GA/GlobalReader cross-tenant via session tenant resolution, 1 MB body cap,
structural draft validation (unknown source/operator, correlation without
joinField, invalid regex, non-evaluable confidenceFactor DSL) with
actionable error messages.
- EndpointAccessPolicyCatalog: register the new route.
- RuleEngineDryRunTests: trace semantics, production parity matrix,
side-effect freedom (VerifyNoOtherCalls), draft validation matrix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(mcp): rule-authoring resources — guide, schemas, guardrails
Deterministic authoring context for customer AIs, replacing chunked
search_docs retrieval for rule creation: get_resource (and the MCP-protocol
resources) now serve
- rule_authoring_guide: hand-written, complete authoring knowledge (collector
types, triggers, condition sources, operators, confidence model incl. the
exact factor-condition DSL shapes, {{token}} interpolation order, guardrail
matching semantics, pitfalls, validate->dry-run workflow)
- rule_schemas: the JSON Schemas (2020-12) for gather + analyze rules
- rule_guardrails: the on-device collection allowlists
Schemas + guardrails are generated into rule-authoring.generated.ts by a new
combine.js emitter (same pattern as the web guardrails mirror; rules/ stays
the single source of truth and CI guardrails-in-sync guards the file).
rule-authoring-drift.test.ts additionally pins generated == repo source,
guide enums == schema enums, and the get_resource size cap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(mcp): validate_rule + test_analyze_rule tools and author-rule prompt
Closes the authoring loop for customer AIs: draft -> validate -> dry-run
against a real session -> iterate -> create in the portal.
- validate_rule (MCP-local, no backend call): ajv 2020-12 validation against
the baked schemas, guardrail pre-flight mirroring the agent matching
semantics (hive-stripped segment-bounded registry/file prefixes,
whitespace-bounded WMI, exact commands, /-bounded event log channels, hard
blocks), and semantic lint for schema-valid-but-dead rules (unreachable
threshold, non-evaluable factor DSL, unknown eventTypes, unanchored
not_regex allowlists, unresolvable {{tokens}}, trigger cross-fields,
outputEventType conventions/collisions).
- test_analyze_rule: calls the new POST /api/rules/analyze/dryrun and adds
the {{token}}-interpolated explanation/remediation preview via the
existing interpolate-rule-template.
- author-rule MCP prompt guiding scenario -> resources -> draft -> validate
-> dry-run.
- Both tools are visible to every role (backend enforces
TenantAdminOrGlobalReader on the dryrun route); catalog snapshot updated.
- deps: ajv + ajv-formats promoted to direct dependencies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(okf): rule authoring surface concept doc
docs/rules/rule-authoring-surface.md: the AI-assisted authoring stack
(deterministic MCP resources via combine.js baking, MCP-local validate_rule
layers, the dry-run trace twin with its parity + side-effect invariants, and
what is deliberately absent); registered in index.md, logged 2026-08-05.
Customer docs (rules/ai-assisted-rule-authoring.md + SUMMARY + FAQ entry) are
committed in the autopilotmonitor-docs repository separately.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
eb94c612e7 |
fix(agent): gather phase/event triggers follow the engine-reduced timeline
phase_change/phase_exit/on_event gather triggers (and the phase-scope gates) were driven by the pre-reduce SignalIngress.SignalPosted stream, so any raw signal carrying an EspPhase payload fired them before the decision engine had applied its gates. In session 32312a32 (RealmJoin tenant) a Phase Start rule on FinalizingSetup fired at the raw ESP-exit EspPhaseChanged(FinalizingSetup) signal, 7 minutes before the engine's RealmJoin-gated phase_transition(FinalizingSetup) - and read HKLM\SOFTWARE\RealmJoin\Custom\BIOS before the RealmJoin package that creates it had run (exists=false). GatherRuleExecutorHost now subscribes to the new TimelineEventStream: EventTimelineEmitter publishes every emitted event's (eventType, phase) post-reduce, so phase triggers fire exactly when the UI timeline shows the phase, and on_event sees engine-emitted event types (enrollment_complete, phase_transition, realmjoin_resolved, ...) that never existed on the raw signal stream - restoring V1 semantics and the documented "On Event with enrollment_complete" pattern. Raw-signal consumers with OS-state semantics (DeviceInfoHost, UserEspKeepAwakeHost, ProvisioningPackageHost) stay on the raw feed deliberately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
3f9576d4a9 |
fix(agent): enforce the gather-rule allowlist in every collector
LogParserCollector and EventLogCollector called no guard at all, while File/Json/Xml did — and rules/gather-rules.md already promised customers that log-parser paths were allowlisted. A tenant admin could author a logparser rule against C:\Users via POST /api/rules/gather and have up to 1000 lines uploaded as telemetry, bypassing the hard C:\Users block the same page guarantees; eventlog could read any channel, Security included. The backend validates no collectorType, target, or parameters, so the agent is the only place this can be enforced. Both now guard. logparser cannot check rule.Target directly (GetFullPath throws on a wildcard), so it checks the directory for wildcard targets and re-checks each resolved path. Event log channels get an allowlist matched on a "/" boundary. Hard-blocked channels live in a C# constant rather than guardrails.json: a parse error drops the allowlists to empty, which is fail-closed, but would have lifted a block sourced from the same file. Also fixes an inert timeout in CommandCollector: ReadToEnd() blocks until the pipe closes, so WaitForExit(30000) was only reached once the process had already exited and a hung command pinned its worker indefinitely. Adds the first real CI gate. No workflow ran tests on a pull request — the deploy workflows build without testing and build-agent.yml covered only DecisionCore.Tests on manual dispatch — while trust/security-faq.md already told customers automated tests run in CI. ci.yml now runs all six suites on every PR and merge, plus a job that fails on generated-guardrail drift, alongside CodeQL and Dependabot. OfficeBinaryWatcherTests is marked SerialThreading; its bounded re-probe races the shared ThreadPool and failed once under full-suite load. Verified: 3101 backend, 1911 agent V2, 474 decision core, 44 dialog, 572 web, 447 MCP. The timeout fix is verified by runtime, not assertion — a Start-Sleep 120 command now returns after ~30s. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
c2c02b929c |
fix(web): show gather-rule scope/emit controls only where they change behavior
The form offered trigger, phase scope and emit mode unconditionally, which produced combinations that cannot do anything plus one silently wrong state: - A scope on a Phase Start / Phase End rule that names a concrete phase can only ever suppress the single firing — the phase IS the moment. Same for emit mode: a one-shot has no second result to dedup against. - Selecting "From a phase onwards" (or "Only during specific phases") and not filling the detail dropdown serialised to null, i.e. the rule ran everywhere while the form still read "From a phase onwards". supportsPhaseScope / supportsEmitMode now drive visibility; buildScopeFields nulls hidden-control state so a value from an earlier trigger choice cannot leak into the payload; validateScopeSelection rejects an unfilled scope mode on both create and save. Startup is the deliberate exception — a scope there defers the one-shot instead of suppressing it, so the scope control stays and only emit mode is hidden. One-shot triggers now state in-form when they collect. Sub-dropdown hints no longer claim "No selection = all phases" (they turn amber and name the required action instead). Tests: app/gather-rules/__tests__/gatherRuleFormLogic.test.ts (15) pins the applicability matrix, the payload nulling incl. the trigger-switch leak, the unfilled-scope rejection, and the JSON-mode scopeMode derivation. vitest 572/572; tsc silent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
dbf6e50e2d |
feat(agent,web,shared): phase_exit gather trigger — one-shot collection at a phase's end
Follow-up to gather-rule phase scoping. One-shot collection at a phase START
already existed (phase_change dedups per (rule, phase)) and at enrollment END
(on_event + enrollment_complete); the END of a specific phase had no trigger.
New trigger value "phase_exit": fires once when TriggerPhase is LEFT (empty =
every phase transition). Together with phase_change it forms the two one-shot
bookends of a phase.
Agent (GatherRuleExecutor.OnPhaseChanged):
- Exit rules are evaluated BEFORE _currentPhase advances, so both the
TriggerPhase match and the phase-scope gate see the phase being LEFT. A rule
scoped to DeviceSetup therefore fires on its own exit even though the phase
being entered is out of scope.
- A transition into Failed fires the exit rules of the failing phase — the
snapshot at a failure boundary is the point of the trigger.
- Exit of Unknown never fires; a repeated identical phase signal is not a
transition.
- Dedup key "exit:{ruleId}|{phase}" shares _phaseRulesExecuted with the
phase_change keys but stays in a disjoint key space, so an enter-rule and an
exit-rule on the same phase cannot consume each other's slot.
Portal: trigger labels are now "Phase Start" / "Phase End"; the canonical phase
dropdown and the create/save payloads cover both via PHASE_TRIGGERS; the card
renders "entering X" / "leaving X".
No ConfigVersion bump: no new field is transported. Old agents do not know the
value and simply never fire the rule (silent no-op).
Tests: GatherRuleExecutorPhaseExitTests (11) — enter-vs-exit, no refire on
re-entry, empty-triggerPhase, exit-of-Unknown, same-phase-signal, into-Failed,
Complete, scope-against-left-phase, out-of-scope slot preservation, key-space
disjointness, on_change interplay. Build 0 Errors; V2.Core.Tests 1884/1884;
Functions.Tests 3099/3099; vitest 557/557; tsc silent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
e17eb607ca |
docs: gather-rule phase scoping + emit-on-change concept doc (OKF)
New rules/ section in the knowledge bundle: semantics table, from-phase latch behavior (never via Failed, sticky through it), on_change x emitOnlyIfExists composition, restart re-emit, no-phase-signal caveat, old-agent degradation, and the prerequisite toggle-wipe fix. Index + log updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |