Files
AutopilotMonitor/rules/scripts/combine.js
T
Oliver Kieselbach 3c1229a301 MCP rule authoring: deterministic context, validate_rule, and session dry-run (#124)
* feat(backend): dry-run endpoint for draft analyze rules (rule authoring)

POST /api/rules/analyze/dryrun evaluates a DRAFT analyze rule against one
session's events and returns a full diagnostic trace: per-condition
matched/evidence (incl. per-eventType occurrence counts), per-factor
confidence breakdown, verdict, and the would-be MatchedConditions map for
{{token}} interpolation preview. Persists nothing: no RuleResult row, no
session-status change, no notifications.

- RuleEngine.DryRun.cs: DryRunRuleAsync / DryRunEvaluate share the exact
  evaluators (EvaluatePrecondition/EvaluateCondition/EvaluateConfidenceFactor)
  with the production path but evaluate ALL conditions instead of breaking on
  the first failed required one; parity with EvaluateRule is pinned by tests.
- DryRunAnalyzeRuleFunction: TenantAdminOrGlobalReader (read-in-effect POST),
  GA/GlobalReader cross-tenant via session tenant resolution, 1 MB body cap,
  structural draft validation (unknown source/operator, correlation without
  joinField, invalid regex, non-evaluable confidenceFactor DSL) with
  actionable error messages.
- EndpointAccessPolicyCatalog: register the new route.
- RuleEngineDryRunTests: trace semantics, production parity matrix,
  side-effect freedom (VerifyNoOtherCalls), draft validation matrix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(mcp): rule-authoring resources — guide, schemas, guardrails

Deterministic authoring context for customer AIs, replacing chunked
search_docs retrieval for rule creation: get_resource (and the MCP-protocol
resources) now serve

- rule_authoring_guide: hand-written, complete authoring knowledge (collector
  types, triggers, condition sources, operators, confidence model incl. the
  exact factor-condition DSL shapes, {{token}} interpolation order, guardrail
  matching semantics, pitfalls, validate->dry-run workflow)
- rule_schemas: the JSON Schemas (2020-12) for gather + analyze rules
- rule_guardrails: the on-device collection allowlists

Schemas + guardrails are generated into rule-authoring.generated.ts by a new
combine.js emitter (same pattern as the web guardrails mirror; rules/ stays
the single source of truth and CI guardrails-in-sync guards the file).
rule-authoring-drift.test.ts additionally pins generated == repo source,
guide enums == schema enums, and the get_resource size cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(mcp): validate_rule + test_analyze_rule tools and author-rule prompt

Closes the authoring loop for customer AIs: draft -> validate -> dry-run
against a real session -> iterate -> create in the portal.

- validate_rule (MCP-local, no backend call): ajv 2020-12 validation against
  the baked schemas, guardrail pre-flight mirroring the agent matching
  semantics (hive-stripped segment-bounded registry/file prefixes,
  whitespace-bounded WMI, exact commands, /-bounded event log channels, hard
  blocks), and semantic lint for schema-valid-but-dead rules (unreachable
  threshold, non-evaluable factor DSL, unknown eventTypes, unanchored
  not_regex allowlists, unresolvable {{tokens}}, trigger cross-fields,
  outputEventType conventions/collisions).
- test_analyze_rule: calls the new POST /api/rules/analyze/dryrun and adds
  the {{token}}-interpolated explanation/remediation preview via the
  existing interpolate-rule-template.
- author-rule MCP prompt guiding scenario -> resources -> draft -> validate
  -> dry-run.
- Both tools are visible to every role (backend enforces
  TenantAdminOrGlobalReader on the dryrun route); catalog snapshot updated.
- deps: ajv + ajv-formats promoted to direct dependencies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(okf): rule authoring surface concept doc

docs/rules/rule-authoring-surface.md: the AI-assisted authoring stack
(deterministic MCP resources via combine.js baking, MCP-local validate_rule
layers, the dry-run trace twin with its parity + side-effect invariants, and
what is deliberately absent); registered in index.md, logged 2026-08-05.

Customer docs (rules/ai-assisted-rule-authoring.md + SUMMARY + FAQ entry) are
committed in the autopilotmonitor-docs repository separately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 17:24:28 +02:00

206 lines
8.1 KiB
JavaScript

#!/usr/bin/env node
/**
* Combines individual rule JSON files into dist/ array files and generates
* TypeScript guardrails from rules/guardrails.json.
*
* Run: node rules/scripts/combine.js
*/
const fs = require('fs');
const path = require('path');
const rulesRoot = path.resolve(__dirname, '..');
// ── Rule combination ────────────────────────────────────────────────────────
const configs = [
{
dir: path.join(rulesRoot, 'gather'),
output: path.join(rulesRoot, 'dist', 'gather-rules.json'),
schema: '../schema/gather-rule.schema.json',
idField: 'ruleId'
},
{
dir: path.join(rulesRoot, 'analyze'),
output: path.join(rulesRoot, 'dist', 'analyze-rules.json'),
schema: '../schema/analyze-rule.schema.json',
idField: 'ruleId'
},
{
dir: path.join(rulesRoot, 'ime-log-patterns'),
output: path.join(rulesRoot, 'dist', 'ime-log-patterns.json'),
schema: '../schema/ime-log-pattern.schema.json',
idField: 'patternId'
}
];
for (const config of configs) {
const files = fs.readdirSync(config.dir).filter(f => f.endsWith('.json')).sort();
const rules = [];
for (const file of files) {
const content = fs.readFileSync(path.join(config.dir, file), 'utf8');
const rule = JSON.parse(content);
// Remove $schema from individual entries (it's on the wrapper)
delete rule['$schema'];
rules.push(rule);
}
// Sort by ID for deterministic output
rules.sort((a, b) => (a[config.idField] || '').localeCompare(b[config.idField] || ''));
const wrapper = {
$schema: config.schema,
rules: rules
};
fs.mkdirSync(path.dirname(config.output), { recursive: true });
fs.writeFileSync(config.output, JSON.stringify(wrapper, null, 2) + '\n', 'utf8');
console.log(`${path.basename(config.output)}: ${rules.length} rules combined`);
}
// ── Guardrails generation ───────────────────────────────────────────────────
const guardrailsPath = path.join(rulesRoot, 'guardrails.json');
// NOTE: consumers import from "@/utils/guardrails.generated" (see guardValidation.ts
// and SectionGatherRules.tsx). The web re-org (commit a3e8e044) moved this file to
// utils/ but left this output path pointing at the now-orphaned lib/ copy, which
// silently froze the live allowlist. Keep this in sync with the import path.
const guardrailsOutput = path.resolve(
rulesRoot, '..', 'src', 'Web', 'autopilot-monitor-web', 'utils', 'guardrails.generated.ts'
);
if (fs.existsSync(guardrailsPath)) {
const guardrails = JSON.parse(fs.readFileSync(guardrailsPath, 'utf8'));
/** Escape a string for use inside a TS string literal (double-quoted). */
const esc = (s) => s.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
/** Render an array of strings as a TS readonly array. */
const flatArray = (items) =>
items.map((s) => ` "${esc(s)}",`).join('\n');
/** Render a categorized list as a TS readonly array of { category, items }. */
const categorizedArray = (groups, itemsKey) =>
groups.map((g) => {
const items = g[itemsKey].map((s) => ` "${esc(s)}",`).join('\n');
return ` {\n category: "${esc(g.category)}",\n items: [\n${items}\n ],\n },`;
}).join('\n');
// Flatten categorized lists for validation
const flatRegistryPrefixes = guardrails.registryPrefixes.flatMap((g) => g.prefixes);
const flatCommands = guardrails.allowedCommands.flatMap((g) => g.commands);
const flatEventLogChannels = guardrails.eventLogChannels.flatMap((g) => g.channels);
const ts = `/**
* AUTO-GENERATED from rules/guardrails.json — DO NOT EDIT.
* Run: node rules/scripts/combine.js
*/
// ---------------------------------------------------------------------------
// Categorized data (for documentation / UI display)
// ---------------------------------------------------------------------------
export interface GuardrailCategory {
readonly category: string;
readonly items: readonly string[];
}
export const REGISTRY_PREFIX_CATEGORIES: readonly GuardrailCategory[] = [
${categorizedArray(guardrails.registryPrefixes, 'prefixes')}
];
export const COMMAND_CATEGORIES: readonly GuardrailCategory[] = [
${categorizedArray(guardrails.allowedCommands, 'commands')}
];
export const EVENT_LOG_CHANNEL_CATEGORIES: readonly GuardrailCategory[] = [
${categorizedArray(guardrails.eventLogChannels, 'channels')}
];
// ---------------------------------------------------------------------------
// Flat arrays (for validation logic)
// ---------------------------------------------------------------------------
export const ALLOWED_REGISTRY_PREFIXES: readonly string[] = [
${flatArray(flatRegistryPrefixes)}
];
export const ALLOWED_FILE_PREFIXES: readonly string[] = [
${flatArray(guardrails.filePrefixes)}
];
export const ALLOWED_WMI_QUERY_PREFIXES: readonly string[] = [
${flatArray(guardrails.wmiQueryPrefixes)}
];
export const ALLOWED_COMMANDS_LIST: readonly string[] = [
${flatArray(flatCommands)}
];
export const ALLOWED_DIAGNOSTICS_PATH_PREFIXES: readonly string[] = [
${flatArray(guardrails.diagnosticsPathPrefixes)}
];
export const BLOCKED_FILE_PREFIXES: readonly string[] = [
${flatArray(guardrails.blockedFilePrefixes)}
];
export const ALLOWED_EVENT_LOG_CHANNELS: readonly string[] = [
${flatArray(flatEventLogChannels)}
];
export const BLOCKED_EVENT_LOG_CHANNELS: readonly string[] = [
${flatArray(guardrails.blockedEventLogChannels)}
];
`;
fs.writeFileSync(guardrailsOutput, ts, 'utf8');
console.log(`guardrails.generated.ts: ${flatRegistryPrefixes.length} registry, ${flatCommands.length} commands, ${guardrails.filePrefixes.length} file prefixes, ${flatEventLogChannels.length} event log channels`);
} else {
console.warn('guardrails.json not found — skipping guardrails generation');
}
// ── MCP rule-authoring module generation ────────────────────────────────────
// The MCP server exposes the rule schemas + guardrails as get_resource content
// so a customer's AI can author rules against the real contract instead of
// retrieval fragments. The server's tsconfig (rootDir: src, no resolveJsonModule)
// and the Docker build (rules/schema not copied) rule out importing the JSON
// directly — so we bake it into a TS module, exactly like the web mirror above.
// CI's guardrails-in-sync job re-runs this script and fails on any diff, which
// makes drift between rules/* and this generated module impossible to merge.
const mcpAuthoringOutput = path.resolve(
rulesRoot, '..', 'src', 'McpServer', 'autopilot-monitor-mcp', 'src', 'rule-authoring.generated.ts'
);
{
const guardrails = JSON.parse(fs.readFileSync(guardrailsPath, 'utf8'));
const gatherSchema = JSON.parse(fs.readFileSync(path.join(rulesRoot, 'schema', 'gather-rule.schema.json'), 'utf8'));
const analyzeSchema = JSON.parse(fs.readFileSync(path.join(rulesRoot, 'schema', 'analyze-rule.schema.json'), 'utf8'));
const ts = `/**
* AUTO-GENERATED from rules/guardrails.json + rules/schema/*.schema.json — DO NOT EDIT.
* Run: node rules/scripts/combine.js
*
* Consumed by the MCP server's rule-authoring surface (get_resource +
* validate_rule): the JSON Schemas are the validation contract, the guardrails
* are the agent-side collection allowlists. Single source of truth is rules/;
* the CI guardrails-in-sync job guards this file against drift.
*/
/** JSON Schema (2020-12) for gather rules — rules/schema/gather-rule.schema.json verbatim. */
export const GATHER_RULE_SCHEMA: Record<string, unknown> = ${JSON.stringify(gatherSchema, null, 2)};
/** JSON Schema (2020-12) for analyze rules — rules/schema/analyze-rule.schema.json verbatim. */
export const ANALYZE_RULE_SCHEMA: Record<string, unknown> = ${JSON.stringify(analyzeSchema, null, 2)};
/** Gather-rule collection guardrails — rules/guardrails.json verbatim. */
export const RULE_GUARDRAILS = ${JSON.stringify(guardrails, null, 2)} as const;
`;
fs.writeFileSync(mcpAuthoringOutput, ts, 'utf8');
console.log('rule-authoring.generated.ts: schemas + guardrails baked for MCP server');
}