9 Commits

Author	SHA1	Message	Date
Anso	e05099f2a1	fix(fleet-sync): make control-identity-mismatch sticky and surface in UI (#1117) ... Treat 409 CONTROL_IDENTITY_MISMATCH from a replica as a non-retriable failure instead of looping the same 409 through the 5-minute retry service forever and silently writing identical failure rows. Backend - DatabaseService: add `sticky_error_code`, `sticky_error_expected`, `sticky_error_got` columns to `fleet_sync_status` via an idempotent migration. New methods setFleetSyncSticky, getFleetSyncStickyCode, clearFleetSyncStickyForNode. recordFleetSyncSuccess clears the sticky flag on a clean push. getFailedSyncTargets SQL adds `AND sticky_error_code IS NULL` so the retry loop skips sticky rows. - FleetSyncService.executePushToNode: short-circuits at the top when sticky is set (covers event-driven pushResourceAsync calls). On a 409 with code CONTROL_IDENTITY_MISMATCH, records the failure once and pins sticky with the expected/got fingerprints carried in the 409 body. - routes/nodes.ts: new POST /api/nodes/:id/fleet-sync/reset-anchor. Admin + paid + node:manage. Proxies POST /api/fleet/role/reanchor to the peer with `{override:true}` using the stored Bearer node_proxy token. On peer 200, clears every sticky row for the node so the next push re-anchors and resumes replication. Distinct 502 / 504 responses for peer-rejected / peer-unreachable so the UI can show a useful toast. Frontend - New lib/fleetSyncApi.ts + hooks/useFleetSyncStatus.ts. Polling hook (30s visibilityInterval) skips fetch when !isPaid. - NodeManager.tsx: destructive banner per affected node listing both fingerprints, with `Reset anchor on peer` and `Remove node` buttons. Hidden for community-tier users via empty hook data. - FleetConfiguration.tsx (Fleet -> Status): read-only `Policy sync` SummaryRow per remote node card. In sync / degraded / paused with a tooltip; no action buttons (the action lives in NodeManager). Tests - fleet-sync-service.test.ts: 4 new cases for sticky-set on first mismatch, short-circuit on subsequent pushes, null fingerprints, and non-mismatch failures not setting sticky. - database-fleet-sync-sticky.test.ts (new): 6 cases pinning the DB contract incl. retry-loop SQL filter and migration idempotency. - nodes-fleet-sync-reset-anchor.test.ts (new): 6 cases covering happy path, peer 401 -> 502, peer unreachable -> 504, local-node rejection, unknown node id, and community-tier 403. Gate parity (Directive 30): the new POST .../reset-anchor enforces requireAdmin + requirePaid + node:manage (matches the existing read at GET /api/fleet/sync-status). UI banner + SummaryRow only render when the hook returns data, which it only does for paid-tier authed users. No existing tier-gate file moved; this is greenfield parity. Auth audit: the peer's POST /api/fleet/role/reanchor route already uses requireAdmin, which accepts the central's stored node_proxy Bearer token because authMiddleware maps `scope === 'node_proxy'` to `req.user = { username: 'node-proxy', role: 'admin', userId: 0 }`. No widening required. Backend tsc clean. Frontend tsc -b clean. 59 fleet-sync tests pass; full backend suite green minus the pre-existing Windows-only file-lock flake on filesystem-backup.test.ts that reproduces unchanged on main.	2026-05-19 19:49:16 -04:00
Anso	3b650523c1	Audit-hardening pass for secret and misconfiguration scanning (#977) ... * fix(security): dedupe concurrent compose-stack scans Track stack scans in scanningImages keyed stack:<nodeId>:<stackName>. The /scan/stack route returns 409 when an in-flight scan exists, and the service-side check is the real correctness barrier (the route pre-check is a fast-path optimization that mirrors scanImage). The dedup key release lives in a try/finally so failed scans free the slot for retry. Why: scanComposeStack had no equivalent of scanImage's scanningImages guard, so two simultaneous calls for the same stack would both run trivy config, both insert a vulnerability_scans row, and double- process the result. * feat(security): acknowledge misconfig findings Adds a parallel acknowledgement system for Trivy misconfig findings that mirrors cve_suppressions: a new misconfig_acknowledgements table, read-time enrichment via the new misconfig-ack-filter utility, REST CRUD endpoints, fleet-sync replication from control to replicas, a Settings panel, and an Acknowledge button on the Misconfigs tab. Schema and behavior parity with cve_suppressions: - UNIQUE(rule_id, COALESCE(stack_pattern, '')) so fleet-wide acks collide as expected - blockIfReplica on every write - Audit-log entries name the scope (rule_id, stack_pattern) but never the reason text - replicated_from_control flag controls UI delete affordance and drives clearReplicatedRows on demote/reanchor - Validators reused: validateStackPatternForRedos for glob safety, sanitizeForLog for log fragments SARIF export emits an external/accepted suppression entry per acknowledged misconfig, matching the CVE pattern. Per-row Acknowledge dialog prefills stack_pattern with the scan's stack_context so the default scope is "rule + this stack only" and an operator must broaden explicitly. Tests: misconfig-ack-filter (15) and misconfig-ack-routes (23) including the duplicate-409 case for both pinned and fleet-wide acks. * fix(security): reap orphaned trivy tmp dirs at startup When the buildEnv path writes a per-scan DOCKER_CONFIG dir under os.tmpdir() and the process crashes before the finally block runs, the dir leaks. Mirrors GitSourceService.sweepStaleTempDirs: exported sweepStaleTrivyTempDirs is fire-and-forget at boot, removes prefix-matching dirs older than 1 hour, swallows permission/race failures, logs a single line if any were reaped. * perf(security): emit per-batch summary for scanAllNodeImages Adds one diag() line at the end of scanAllNodeImages summarising unique image count, scanned, skipped, failed, violation count, and elapsed time. Per-image diag inside scanImage stays useful for debugging individual scans; the summary gives operators a single fleet-level checkpoint when developer_mode is on. * perf(security): cap SARIF export at 5000 findings per type Replace the unbounded fetchAllPages walk on /scans/:id/sarif with a hard limit of 5000 findings per type. When any type trips the cap, emit run-level properties.truncated=true plus row_limit and per-type totals so downstream tooling can flag the export as partial. Console-warns for ops visibility. A scan with 50k vulns previously streamed every row into memory before serialising; the cap bounds memory and serialisation time at the cost of completeness on pathological scans. * docs(env): document TRIVY_BIN host-binary override The env var is honored by TrivyService.detectTrivy as a fallback when no managed install is present, but it was undocumented in .env.example. Adds the var with a comment explaining precedence (managed > TRIVY_BIN > PATH). * test(security): cover scanComposeStack failure modes Two new cases drive the existing try/catch through real failure paths: - Malformed Trivy stdout: row flips to status='failed' with the parser error preserved on `error`. - execFile rejection: row flips to status='failed' with a string error message. Pairs with the existing dedup tests so the failure path now also verifies the scan row state, not just the thrown exception. * test(e2e): security scanner + misconfig acknowledgement flow Seven Playwright tests covering the scanner UI and the new acknowledgement system end-to-end: - Trivy availability gate (skips suite when binary absent so CI without Trivy can opt out via E2E_SKIP_TRIVY=1) - Stack config scan completes and records misconfig findings - Concurrent stack scan returns 409 from the dedup gate - Misconfig ack POST creates and lists on Settings - Duplicate (rule_id, stack_pattern) returns 409 - Malformed rule_id (shell metacharacters) returns 400 - Misconfigs tab renders against a real stack scan Tests drive the API for behaviour assertions and the UI only for shell-rendering checks; the visual snapshot suite owns screenshots. * docs(features): add misconfig acknowledgement workflow and SARIF cap Refreshes vulnerability-scanning.mdx with: - Misconfig acknowledgements section covering the per-row dialog, Settings panel, scope/matching rules, and SARIF emission - Tier table row for the new feature - SARIF section note on the 5000 row-per-type cap and the properties.truncated marker for partial exports - Troubleshooting entries: SARIF cap, hidden Acknowledge button, findings resurfacing after delete, Trivy DB phone-home, and 409 on concurrent compose-stack scans * fix(ci): clear backend lint and CodeQL alerts - Remove the dead fetchAllPages helper in routes/security.ts. It lost its callers when the SARIF endpoint switched to direct paged reads for the truncation cap. ESLint flagged it as unused. - Switch the trivy-tmp-cleanup test helper to fs.mkdtempSync. Building paths under os.tmpdir() with predictable names tripped CodeQL's js/insecure-temporary-file rule (high severity), which warns about symlink-pre-creation attacks even in test code. mkdtempSync appends a process-random suffix and creates the dir atomically; the sencho-trivy- prefix is preserved so the production sweep still matches the test fixtures.	2026-05-07 19:23:11 -04:00
Anso	4007709590	fix(fleet-sync): hygiene pass on receiver behavior and cleanup (#972) ... A bundle of small file-local fixes to the receiver path and node-deletion flow. Changes: - F4 receiver audit log: applyIncomingSync now writes a system audit entry on every applied push so mirrored security-rule changes show up in the replica's audit panel with a clear control-side origin. - F7 pilot-agent skip: pushResource explicitly excludes pilot-agent nodes (they have no api_url for HTTP push) and warns once per node id so the operator sees they will not receive replicated policies. - B4 identity-drift notification: when targetIdentity differs from the cached fleet_self_identity, dispatch a warning so the operator can audit any identity-scoped policies that may need re-targeting. - B6 stack_pattern ReDoS guard: reject patterns with 4+ consecutive wildcards or more than 8 wildcards total. Both control-side validators (POST/PUT scan policies) and the receiver-side row validator share the helper. - B9 deleteNode cascade: clear fleet_sync_status rows for the node inside the existing transaction so the sync-status panel does not render ghost entries after a node is removed. - S6 last_error redaction: formatError strips Bearer tokens and JWT-shaped values from error messages and caps at 500 chars before storing in fleet_sync_status.last_error or logging. Tests: - 8 new vitest cases covering audit-log entry, identity-drift alert, pilot-agent warn-once, formatError redaction (Bearer + JWT), ReDoS validator rejection, and a backtracking-time smoke test. - New database-fleet-sync-cascade.test.ts: deleteNode removes fleet_sync_status rows for the deleted node and leaves siblings untouched. - Full backend suite: 1792 pass / 5 skipped.	2026-05-07 13:41:10 -04:00
Anso	33b15d6cba	feat(fleet-sync): retry failed pushes and backfill on add-node (#970) ... A control instance now retries fleet-sync pushes that hit a transient failure and backfills the security state on a freshly registered remote without waiting for the next policy edit. New service: - FleetSyncRetryService (singleton, start/stop) wakes 30s after boot and ticks every 5min. For each fleet resource, queries getFailedSyncTargets within a 24h window and re-pushes via FleetSyncService.pushResourceToNode through the same per-node mutex, so a normal fanout in flight serializes naturally with a retry. - After STALE_THRESHOLD_MS (1h) of continuous failure for a previously-working node, dispatches a single warning notification per cooldown window. Brand-new nodes that have never succeeded do not alert via this path; misconfigured remotes are caught by the test-connection affordance at registration time. - Wired into bootstrap startup/shutdown next to AutoHealService. Public surface: - FleetSyncService.pushResourceToNode(node, resource): targeted push to one node that re-uses the per-node mutex. Used by the retry service and any future targeted-resync flow. - routes/nodes.ts POST /api/nodes fires pushResourceAsync for both resources after a remote-proxy node row commits. Tuning constants centralized in fleetSyncConstants.ts: - RETRY_MAX_AGE_MS = 24h - STALE_THRESHOLD_MS = 1h Tests: - 8 vitest cases covering replica skip, retry dispatch, missing-node skip, alert-once-per-cooldown across the threshold window, no-alert for recent failures, no-alert for brand-new never-succeeded nodes, no-alert when the retry itself succeeds, start/stop idempotency. - Full backend suite: 1781 pass / 5 skipped.	2026-05-07 13:16:24 -04:00
Anso	7dde257e1f	feat(fleet-sync): replica self-demote endpoint and role UX (#969) ... A replica admin can now demote the instance back to a standalone control without raw SQLite access. The Settings → Security UI surfaces a confirm-gated button when the role is replica; the role probe also surfaces a soft banner when it cannot determine fleet role rather than silently defaulting to control. Backend: - POST /api/fleet/role/demote (admin, requires `{confirm: true}`): flips fleet_role to 'control', clears fleet_self_identity, fleet_control_identity, and both received_pushed_at:* watermarks, drops every replicated_from_control row from scan_policies and cve_suppressions, nulls out any orphaned policy_evaluation cache. Returns 409 ALREADY_CONTROL when invoked on a control. - DatabaseService gains `clearOrphanPolicyEvaluations()` and `clearReplicatedRows()` helpers. Reanchor consolidates onto clearReplicatedRows so it shares the same code path. - `FleetSyncService.demote()` returns boolean for the route to translate into 200 or 409. Frontend: - SecuritySection probes /fleet/role and now records explicit success vs failure rather than silently treating an error as control. A soft banner appears when probe fails. - Replica banner gains a "Demote to control" button and a destructive ConfirmModal explaining the wipe. Tests: - 4 new route-level vitest cases (401, 400 without confirm, end-to-end demote with replica setup, 409 ALREADY_CONTROL with explicit precondition). - Service unit test asserts the consolidated clearReplicatedRows path. - Full backend suite: 1773 pass / 5 skipped. Frontend: 185 pass.	2026-05-07 13:08:21 -04:00
Anso	f3757b43c6	feat(fleet-sync): anchor replicas to a control fingerprint (#968) ... A replica now binds to the first control that pushes to it. Subsequent pushes from a different control are rejected with 409 CONTROL_IDENTITY_MISMATCH until an admin explicitly reanchors. Closes the cross-control hijack window where any node_proxy bearer signed against the replica's secret could overwrite security policies. Wire protocol: - Sender includes a stable 16-hex-char `controlIdentity` derived by SHA-256-truncating `system_state.instance_id` (the local UUID written once by LicenseService.initialize on first boot). Hostname rotations do not flag drift; only a SQLite reset or explicit reanchor breaks the binding. - Receiver caches the fingerprint inside the same transaction that applies the row replacement and watermark write. Three states: null (fresh install), '' (post-reanchor), '<fingerprint>' (anchored). - Empty `controlIdentity` is treated as legacy and accepted, so older controls keep working during rollout. New endpoint: - POST /api/fleet/role/reanchor (admin, requires `{override: true}`): clears the cached fingerprint, both `received_pushed_at:*` watermarks, and replicated rows of both resources, all in one transaction. The static cached fingerprint is also flushed defensively. Public surface additions: - `FleetSyncService.getControlIdentity()`: stable fingerprint for the outgoing push body. - `FleetSyncService.reanchor()`: admin-driven anchor reset. - `ControlIdentityMismatchError`: typed sentinel the route translates to 409 with structured body `{error, code, expected, got}`. Tests: - 9 new vitest cases covering first-sync persistence, mismatch rejection, matching acceptance, empty-incoming back-compat, post-reanchor un-anchored state, fingerprint stability, missing instance_id fallback, route-level mismatch, route-level reanchor with override gating. - One ordered route-level scenario instead of cross-dependent it() blocks so test reordering cannot silently break the suite. - Full backend suite: 1769 pass / 5 skipped.	2026-05-07 13:00:21 -04:00
Anso	27660f622b	fix(fleet-sync): version the wire protocol and serialize per-node pushes (#967) ... Hardens the scan-policy and CVE-suppression replication channel as the foundation of a multi-PR fleet sync hardening track. No new endpoints, no new tables, no schema changes; receivers still tolerate legacy payloads (absent pushedAt and controlIdentity) for rollout safety. Wire protocol: - Sender stamps every push with a strictly-increasing pushedAt and a placeholder controlIdentity. Receiver rejects strictly-older pushedAt with 409 STALE_SYNC_PUSH so the next write retries. - pushedAt comparison plus row replacement plus watermark write run in a single SQLite transaction; a partial-write window cannot leave the watermark behind the row state. Concurrency and limits: - Per-node mutex on the sender so concurrent control writes serialize per remote and never apply older state on top of newer. - Sender-side row cap at MAX_SYNC_ROWS=5000 with a 6-hour throttled truncation alert so flapping configs cannot flood the operator. - Route-level body limit raised to 5MB on POST /api/fleet/sync/:resource only; the global 100KB cap is unchanged. Oversize bodies return a structured 413 SYNC_PAYLOAD_TOO_LARGE. Determinism and hygiene: - getMatchingPolicy gains an id-ASC tiebreaker so two replicas resolve the same winner when policies tie on scope class. - Inline comment documents why getMatchingPolicy filters node_id at SQL yet still relies on JS identity matching for replicated rows. - Comment on the receive endpoint documents why no requirePaid is enforced (control's tier authorizes; replica trusts the bearer). - STALE_SYNC_PUSH 409s no longer record a node failure; they are expected protocol outcomes, not health issues. Public surface additions: - DatabaseService.transaction(fn): generic SAVEPOINT-friendly wrapper. - DatabaseService.getLocalScanPolicies / getLocalCveSuppressions: SQL filter on replicated_from_control = 0. - StaleSyncPushError: typed sentinel the route translates to 409. - fleetSyncConstants: shared MAX_SYNC_ROWS, body limit, state-key and error-code maps so the wire protocol has one source of truth. Tests: - 24 new vitest cases across fleet-sync-service, fleet-sync-routes, and database-matching-policy covering monotonic pushedAt, per-node serialization, row truncation and throttle, stale-push suppression, receiver back-compat, oversize-body 413, deterministic matching. - Full backend suite: 1757 pass / 5 skipped.	2026-05-07 12:53:13 -04:00
Anso	732fc95415	feat(security): fleet-replicated CVE suppression list (#650) ... Operators can accept known-benign findings once and have Sencho filter them out of scan drawers, comparison views, and other read surfaces. Suppressions replicate from the control instance to every remote node. * New cve_suppressions table with a COALESCE-based unique index so NULL scope slots collide the way users expect * Admin + paid-tier CRUD routes; writes are rejected on replicas * Read-time filter enriches vulnerability details and compare payloads without mutating stored counts * Settings > Security panel for managing rules, per-CVE suppress action in the scan drawer, dimmed rows with a shield-off indicator * Vitest unit tests for the filter (glob, expiry, specificity) and route tests (auth, tier, replica, UNIQUE conflict)	2026-04-17 05:16:34 -04:00
Anso	708d15b2b3	feat(fleet): replicate scan policies across managed nodes (#649) ... Scan policies now propagate from the control Sencho instance to every registered remote. The control is the source of truth; replicas render rules read-only with a managed-by-control banner. Pushes fire on every policy write, record per-node success and failure on a new fleet_sync_status table, and use node_proxy Bearer tokens exclusively so only sibling Senchos can apply incoming sync payloads. Policy scope now travels as a string identity (api_url or a local sentinel) so node-scoped rules evaluate correctly on each target.	2026-04-16 23:57:08 -04:00