102 Commits

Author	SHA1	Message	Date
Anso	085267b466	feat(security): make CVE suppressions optionally honored by deploy-block policies (#1269) ... * feat(security): make CVE suppressions optionally honored by deploy-block policies Block-on-deploy policies evaluate the raw scan result, so a CVE an admin has accepted in CVE Suppressions still blocks the deploy. Add an opt-in, per-instance toggle ("Honor suppressions in deploy blocks", Settings -> Security) that, when on, re-derives each image's severity from the suppression-filtered findings before comparing to the policy threshold. A deploy that proceeds only because suppressions dropped it below the gate is recorded in the audit log. Default off, so the strict raw-scan behavior is unchanged unless an operator enables it. The setting governs the instance that runs the deploy and is not fleet-replicated. The gate fails safe: a suppression-read error or an empty detail set falls back to raw scan severity rather than dropping it. Also surface a previously swallowed error in the CVE suppressions and misconfig acknowledgement settings panels so a failed list load shows a toast instead of an empty list. * fix(security): gate on raw severity when preflight detail rows are truncated The suppression-aware deploy gate re-derived image severity from the stored vulnerability_details rows, assuming any non-empty set was complete. A cached pre-deploy scan keeps the full aggregate counts but copies only a bounded slice of detail rows, so recomputing from that slice could drop an unsuppressed blocking CVE below the threshold and let a deploy through. Guard the recompute: when the loaded detail rows do not match the scan's total finding count, gate on the raw scan severity (never drops severity). Suppression awareness still applies for scans whose details are stored in full, which is the common case.	2026-06-01 13:39:46 -04:00
Anso	b61388c675	fix(rbac): enforce admin seat cap on promotion and harden last-admin and audit paths (#1266) ... Promoting a user to admin now respects the per-tier admin seat limit the same way user creation does, closing a path that let an operator exceed the cap by creating an account and then editing its role to admin. The last-admin guard for demote and delete now runs the admin-count re-check and the write in a single transaction, so two concurrent admin changes can no longer race the admin count to zero and lock everyone out. Admin two-factor resets are now recorded once with their own audit summary instead of being mislabeled as a user creation by the audit middleware.	2026-06-01 13:01:22 -04:00
Anso	7e65a2ae19	fix(mfa): enforce single-use backup codes under concurrent verification (#1262) ... * fix(mfa): enforce single-use backup codes under concurrent verification Backup-code consumption read the stored hash set, awaited bcrypt.compare, then wrote the shrunk set back. Two concurrent /login/mfa requests carrying the same code could both read the same set, both match, and both persist, so a single backup code yielded two authenticated sessions. Move consumption into a synchronous transaction: verifyBackupCode now returns the matched hash without mutating state, and a new consumeBackupCodeHash re-reads and shrinks the set atomically, returning whether the hash was still present. Login gates success on that result, so exactly one concurrent request wins. Reshape the verify result into a discriminated union so a match always carries its hash. Add coverage: a deterministic consume test (same hash twice, distinct hashes, absent hash), a concurrent same-code login race, backup-code exhaustion, and a disable-via-backup-code path. * test(mfa): make the concurrent backup-code test deterministic The race test relied on incidental scheduling to interleave the two requests, so it could false-green if they happened to serialize. Add a barrier that holds both requests just after verification (once both have read the same stored set) until both arrive, then releases them into the atomic consume. This forces the race every run, so the test fails against a non-atomic consume and passes only when exactly one request wins. A timeout releases the barrier if only one request arrives, so a setup fault fails loudly instead of hanging.	2026-05-31 20:28:48 -04:00
Anso	5e66b54153	fix(audit-log): neutralize CSV export injection, clamp pagination, bound anomaly history (#1259) ... * fix(audit-log): neutralize CSV export injection, clamp pagination, bound anomaly history Harden the Admiral audit log without changing its tier or hub-only gating. - CSV export now defuses formula injection: any field that a spreadsheet would evaluate as a formula (leading = + - @, or a trigger behind leading whitespace, or a leading tab/CR) is prefixed with a single quote before RFC 4180 quoting. Audit summaries embed user-controlled resource names, so this closes a path where a crafted name could execute on export open. - Clamp page and limit to positive bounds on the list endpoint so a negative limit can no longer reach SQLite as "unlimited" and dump the whole table. - Bound the anomaly and stats history reads to a capped slice of recent rows so the analysis paths stay within fixed memory and latency on large histories instead of scanning the full retention window per request. - Surface failed audit list and stats fetches through the standard error toast instead of leaving the view silently stale. - Add a developer-mode-gated diagnostic log to the stats endpoint for parity with the list and export handlers. Covered by new unit and HTTP-integration tests (CSV neutralization through the real export route, pagination clamps, bounded history, stats endpoint, anomaly annotation) and verified end to end in the browser. * fix(audit-log): compute signal-rail stats with exact SQL aggregates Address review feedback on the earlier history-cap change. The cap was correct for the anomaly baseline but made the stats tiles (events, actors, failure rate, hourly series) silently undercount on a hub with more than the cap's worth of rows in the window, since they were derived from the capped row slice. - Add DatabaseService.getAuditStatsInputs: exact counts via SQL COUNT / COUNT(DISTINCT) / GROUP BY hour, and new-ip detection over the small DISTINCT (user, ip) pair sets. No row cap, so the tiles stay exact at any window size while memory stays bounded. - Reduce computeAuditStats to a pure formatter over those aggregates. - Keep the bounded history read only for the list endpoint's anomaly annotation, where a recent-activity baseline is an acceptable heuristic. - Skip the redundant load-failure toast when a fetch fails with a handled 401, so an expired session does not stack toasts on top of logout. Adds exactness tests for the aggregate counts, distinct-actor handling, and new-ip detection, and strengthens the pagination-clamp tests. * fix(audit-log): exclude future-dated rows and make the new-ip sample deterministic Two small parity fixes on the stats aggregates: upper-bound every current window by `now` so a future-dated row (clock skew or a fixture) cannot inflate the live counts, and order the new-ip pair scan so the sample actor shown in the tile detail is stable. Adds a test asserting a future row is excluded.	2026-05-31 16:36:07 -04:00
Anso	265fece988	fix(notifications): prevent self-container stack routing (#1242) ... * fix(notifications): prevent self-container stack routing * fix(stack-files): stabilize download metrics in CI	2026-05-28 12:45:10 -04:00
Anso	adcd04b01a	refactor(auto-update): retire per-stack gate, drive auto-update from schedules only (#1233) ... * refactor(auto-update): retire per-stack gate, drive auto-update from schedules only The per-stack Auto-update toggle in the stack sidebar context menu wrote a gate row to `stack_auto_update_settings`, but actual updates only ran when a `scheduled_tasks` row with `action='update'` fired. On a fresh install the toggle was inert: detection ran every 6h, nothing was applied. The same context menu already exposes `Schedule task`, which opens ScheduledOperationsView pre-filled for the stack where the user can pick `Auto-update Stack` and any cron. Keeping the toggle alongside that flow duplicated the same action and turned the gate table into a parallel store of "is a covering schedule active" derivable from `scheduled_tasks` itself. Drop the gate model entirely: - Backend: remove the `stack_auto_update_settings` table and its four accessors, the three routes under /api/stacks/*/auto-update, the per-stack skip in /api/auto-update/execute and SchedulerService.executeUpdate's fleet branch, and the clearStackAutoUpdateSetting call on stack delete. Dashboard `autoUpdate` count derives from scheduled_tasks (action='update' rows pinned to the node, total/enabled split). - Frontend: drop the Auto-update entry from the sidebar context menu and its optimistic toggle plumbing. Drop autoUpdateSettings state, the /stacks/auto-update-settings fetch, and the auto-update-settings-changed WebSocket branch. Slim useSidebarActivitySummary (just nextRunAt; no enabled/total counts). AutoUpdateReadinessView's per-card autoUpdateEnabled now means "a covering enabled action='update' schedule exists" (per-stack row or fleet row on this node, earliest next_run_at wins, per-stack row wins on ties), with the gate-fetch removed. - New: scheduledTasksRouter broadcasts scope: 'scheduled-tasks' on POST, PUT, PATCH /toggle, and DELETE so useConfigurationStatus and useNextAutoUpdateRun refetch under the 250ms debounce instead of waiting for the 60s poll. The broadcast is wrapped so a broken subscriber socket cannot turn a successful mutation into a 500. - Docs: rewrite the "Per-stack control" section of auto-update-policies.mdx to describe the schedule-based model; update the matching troubleshooting entry. The misleading fleet-update help text in ScheduledOperationsView is corrected to reflect that every stack on the node is covered. Tier parity: the surviving auto-update path (Schedule task -> Auto-update Stack / All Stacks) is gated `requirePaid + requireAdmin` backend and `isPaid + isAdmin` frontend, matching the gate the deleted routes carried. The pre-commit grep returns no tier-related diff outside this PR's scope. No data migration is provided: greenfield rules apply, and the leftover table on already-shipped instances is harmless because no code reads or writes it after this PR. * docs: sweep remaining references to the per-stack auto-update toggle The previous commit retired the per-stack Auto-update gate in favor of configuring auto-update purely through scheduled tasks. This commit removes the now-stale mentions of that toggle across the operator docs: - docs/features/sidebar.mdx: drop the Auto-update entry from the Inspect group description, the matching screenshot alt-text, and the Skipper Note that listed it. Schedule task now carries the cross-link to Auto-Update Policies. - docs/features/stack-management.mdx: drop the Auto-update list item; refresh the Schedule task entry to mention the Auto-update Stack action. - docs/features/dashboard.mdx: rename the Configuration Status row from "Auto-update stacks" to "Auto-update schedules" with the new value shape, and rewrite the troubleshooting accordion to describe the scheduled-tasks invalidation path. - docs/features/scheduled-operations.mdx: rewrite the Auto-update All Stacks row and helper text to reflect that every stack on the node is covered (no per-stack opt-out from this surface anymore). - docs/features/multi-node.mdx: rewrite the Updates column definition to derive the Auto/Off flag from enabled Auto-update Stack / Auto-update All Stacks schedules instead of the removed per-stack policy. The auto-update-policies.mdx rewrite in the previous commit already covered the main reference page. The sidebar-context-menu.png screenshot will be refreshed on release once the new menu is live in production; the alt text is updated in this commit so it accurately describes the shipping state. No website edits needed: the Auto-Update Policies feature card description ("Schedule automatic image pulls and redeployments per stack on your own cadence") and the feature matrix labels ("Auto-update stack schedule", "Auto-update all stacks schedule") remain accurate under the new model. * fix(stacks): drop orphaned requireAdmin import after auto-update route removal CI's backend lint step flagged this PR's earlier deletion of the three /api/stacks/*/auto-update routes: those handlers were the only callers of `requireAdmin` inside routes/stacks.ts, leaving the named import on line 15 unreferenced. `requirePaid` and `effectiveTier` from the same line are still in use elsewhere in the file and stay. tsc --noEmit does not flag unused named imports; ESLint's no-unused-vars does. Local backend lint reproduces and now reports 0 errors against the existing 334-warning baseline.	2026-05-26 11:08:33 -04:00
Anso	42e8d3a78c	feat(security): per-image scroll + retention cap in scan history (#1231) ... * feat(security): per-image scroll + retention cap in scan history Long scan histories for hot images used to monopolise the Scan history sheet: a single image with dozens of scans pushed every other image off screen, and the underlying vulnerability_scans table grew without bound. Each image group's table now renders inside its own ScrollArea capped at max-h-64 (~6 rows visible) so a busy image scrolls independently while the list of images stays navigable. A new global setting scan_history_per_image_limit (default 50, min 5, max 1000) backs both a window-function query that caps the response per image_ref and a prune step that runs on the existing MonitorService cleanup tick. The response now carries cappedImageRefs + perImageLimit so the UI can render a "Capped at N · older scans pruned" hint on groups sitting at the ceiling without a second settings round-trip. Single-image deep-dive (imageRef query param) bypasses the cap so a user clicking into one image can still see its full history. The prune uses self-contained subqueries to avoid SQLITE_MAX_VARIABLE_NUMBER issues on first-run installs with large backlogs, and explicitly deletes child rows from vulnerability_details, secret_findings, and misconfig_findings inside a transaction since FK cascade is not enabled at the connection level. Settings → Developer → Data retention gains a "Scan history per image" field. * fix(security): skip searchDraft debounce on mount to stop page-reset race The searchDraft debounce useEffect fires once on initial mount with the unchanged value and, 300ms later, unconditionally calls setPage(0). When a user (or a test) paginates inside that 300ms window, the pending debounce silently undoes the page advance. CI surfaced this as a flaky 3rd fetch in the "advances offset when the user pages forward" test once the per-image cap work added enough state-update overhead to push the click past the 300ms threshold on the slower Linux jsdom run. Track searchDraft with a ref and exit the effect when the value has not actually changed, so the debounce only runs in response to real user typing.	2026-05-25 23:44:31 -04:00
Anso	2d56ea958a	fix(stack-activity): per-stack history integrity, attribution, sanitization (#1228) ... * fix(stack-activity): per-stack history integrity, attribution, sanitization Address the Stack Activity audit findings (PR 1 of 2): - Per-stack history integrity: drop the per-insert 100-row prune in addNotificationHistory that evicted quieter stacks' history whenever another stack got chatty. Periodic cleanupOldNotifications now caps per (node, stack) at 500 rows and per-node unattached system events at 1000 rows, on top of the existing 30-day retention. Signature takes an options bag and returns a per-stage summary so MonitorService can log what actually ran each cycle. - Actor attribution: thread req.user?.username through every notifyActionFailure call site and add synthetic actors at service emit sites (system:autoheal, system:scheduler, system:image-update, system:docker-events, system:blueprint, system:monitor, system:policy). The timeline renders system actors as "via <Label>" so an autoheal redeploy is no longer indistinguishable from a user redeploy. - Message sanitization: new sanitizeNotificationMessage at NotificationService.dispatchAlert strips KEY=VALUE pairs whose key ends in TOKEN/KEY/PASSWORD/SECRET/CREDENTIALS/AUTH, scrubs HTTP basic auth in URLs and Bearer tokens, collapses COMPOSE_DIR paths, and truncates to 1000 chars. Applied to the stored history and to every downstream Discord/Slack/webhook channel. The ImageUpdateService recovery-path direct DB write also runs through the sanitizer. - Composite pagination cursor: getStackActivity now accepts a (timestamp, id) cursor (?before=&beforeId=). The legacy timestamp-only form silently dropped events when a single compose up emitted many events sharing one millisecond. Route rejects beforeId without before. - Frontend hardening: distinct error state with retry button (initial fetch failure no longer renders as the genuine empty state), strict positive-integer parsing on cursor params, overrequest-by-1 pagination so the last page does not leave a dead "Load more" click, runtime guard on liveEvents merge that validates the level union, per-minute day-bucket recompute so an open panel does not stay on "Today" past midnight. No tier, role, or capability gate touched. Route permission gate remains stack:read on the named stack. * fix(stack-activity): sanitizer covers lowercase env vars and per-node compose dir External review surfaced two leak paths in the message sanitizer: - The sensitive-key regex was uppercase-only. Compose env names are conventionally uppercase but lowercase forms (db_password, jwt_secret, github_token) are valid and do leak through the same Docker and compose-parse error paths. Make the regex case-insensitive and tighten it to also catch bare TOKEN= / KEY= / PASSWORD= without a prefix word, while still leaving BYPASS, COMPASS, and similar non-secret keys alone. - The compose-dir path collapse only read process.env.COMPOSE_DIR, but the real resolution chain is node.compose_dir (per-node DB override) -> process.env.COMPOSE_DIR -> /app/compose. A node with a custom compose_dir could still leak absolute paths into stored history and downstream channels. Route both the dispatchAlert call and the ImageUpdateService recovery-path direct write through NodeRegistry.getInstance().getComposeDir(localNodeId) so the collapse covers every resolution outcome. Tests now assert lowercase keys are redacted and that BYPASS-style non-secrets stay intact in both cases. notification-routing mock extended to stub the new getComposeDir call. * chore(stack-activity): a11y roles, visibility-aware tick, live-disconnect signal Close three small follow-ups on the per-stack activity timeline: - A11y: each day-group gets role="list" and each event row gets role="listitem" so screen readers traverse the timeline as a list instead of a wall of text. The day-group container also carries an aria-label naming the bucket. - Visibility-aware day-bucket tick: the 60s setInterval that re-derives Today/Yesterday/Earlier now short-circuits when document.hidden, so a backgrounded panel does not re-render every minute for no visible effect. - Live-disconnect signal: useNotifications dispatches a sencho:notifications-connection custom event on WebSocket open and close. The timeline listens and, when explicitly disconnected, shows a one-line "Live updates offline; reconnecting…" hint above the list. The sidebar ticker already surfaces fleet-wide connection state; this adds an in-context cue for users who are focused on a single stack. Stack-name case normalization was considered and rejected: stack names are case-permissive per the isValidStackName validator, and lowercasing on read or write would silently rename or hide a user's "MyApp" stack. * ci(stack-activity): drop unnecessary escape in URL_BASIC_AUTH regex ESLint no-useless-escape errored on \- inside the character class [a-zA-Z0-9+.\-] at notificationMessage.ts:14. Move the dash to the end of the class so it's an unambiguous literal and the escape is no longer required. Behavior is identical; sanitizer tests still pass. * revert(stack-activity): drop unvalidated E2E spec from this PR The spec was committed without ever running against a real Docker daemon, then failed in CI when it ran for the first time: deploy returned 200 but no notification appeared on the activity endpoint within the polling window, suggesting either a deploy-notification race or a node-id resolution mismatch in the CI environment. Backend unit tests (route + composite cursor + sanitizer) and frontend component tests cover the same logic. The E2E spec will land in a dedicated follow-up once it has been authored against a working CI environment.	2026-05-25 21:09:00 -04:00
Anso	d727a55a5f	feat(stacks): surface post-deploy scan attempt status (#1198) ... triggerPostDeployScan was fire-and-forget. When Trivy was missing on a node, when the registry refused the digest lookup, or when a single image scan threw, the failure went to console.error and the user never learned. Open the security tab later, see stale data, no indicator that the scan even tried. Backend: - New stack_scan_attempts table (node_id, stack_name, status, attempted_at, error_message). One row per stack; latest attempt overwrites the previous one. - DatabaseService gains recordStackScanAttempt / getStackScanAttempt / clearStackScanAttempts. Status is one of 'ok' | 'partial' | 'failed' | 'skipped'. - triggerPostDeployScan in helpers/policyGate.ts now records every exit path: 'skipped' when Trivy is unavailable or no images to scan; 'failed' when container enumeration or all images fail; 'partial' when some images scan and others fail; 'ok' on full success. - New GET /api/stacks/:name/scan-status returns { status, attemptedAt, errorMessage } or { status: null } when never tried. - DELETE /:stackName cleanup chain now clears the row alongside the existing update-status / auto-update cleanups. Frontend: - StackAnatomyPanel fetches /scan-status on stackName change. - Renders a small warning strip below the update banner when status !== 'ok' (failed / partial / skipped). Hidden when status is 'ok' or unknown (never attempted). Title attribute carries the full error message for hover inspection. Cross-feature note: the audit doc flagged this as M-6 with a coordination note for the pending Security feature audit. The schema kept intentionally narrow (one row per stack, simple status enum) so the Security audit can extend it (richer history, per-image-row breakdown, etc.) without a destructive migration. Resolves M-6 from the stack-management audit.	2026-05-24 15:44:12 -04:00
Anso	aa3d99a594	fix(mesh): re-evaluate data plane every 10s and add opt-in auto-recreate (#1184) ... * fix(mesh): re-evaluate data plane every 10s and add opt-in auto-recreate MeshService.dataPlaneStatus was written exactly once at boot in setupMeshNetwork() and never re-evaluated. After the operator removed sencho_mesh at runtime (or it was recreated externally, or Sencho was disconnected from it), /api/health and the dashboard banner kept returning the stale boot-time discriminator until the next process restart. Adds a 10s revalidator that inspects the current Docker truth in one network-inspect call and transitions dataPlaneStatus to reflect it. Short-circuits in not_started / not_in_docker / subnet_invalid (states that cannot change within this process) and in concurrent ticks. Transitions are idempotent on stable state, so the timer can tick indefinitely on a healthy mesh without log noise. New 'not_found' reason value for the network-was-removed-at-runtime case. Existing reasons (subnet_mismatch, subnet_overlap, attach_failed) also surface from the revalidator when their underlying conditions arise post-boot. transitionDataPlane keeps message and subnet fields fresh across consecutive observations even when reason is unchanged, so /api/health never reports stale numbers (e.g. two consecutive subnet_mismatch observations against different external subnets). Adds an opt-in mesh_auto_recreate global setting (default off). When on, the revalidator additionally calls attemptInPlaceRecreate() after surfacing not_found. The helper hard-prefers the boot-chosen subnet (this.meshSubnet) and never iterates candidates, because changing the subnet here would invalidate every existing extra_hosts override on disk. A real conflict on the original subnet is reported as subnet_overlap and preserved during the 60s recreate throttle window so the operator-actionable reason is not flapped back to not_found between attempts. Self-attachment is checked via Name match (operator --hostname X matches container Name /X) or full container-ID prefix for hex HOSTNAMEs >= 12 chars (Docker default short ID). Non-hex HOSTNAMEs cannot collide with container IDs at all so a Name miss is conclusive; short hex HOSTNAMEs preserve the prior status as 'unknown' rather than risking a false-positive prefix match. Frontend surfaces: - types/mesh.ts: 'not_found' added to MeshDataPlaneReason. - MeshDataPlaneBanner: 'not_found' headline copy. - Settings > System > Mesh data plane: TogglePill bound to mesh_auto_recreate, default off, helper text explains the tradeoff. Backend coverage in backend/src/__tests__/mesh-data-plane-revalidate.test.ts (25 cases): short-circuits, idempotent stable-state, recovery from subnet_mismatch / subnet_overlap, transition to not_found / subnet_mismatch / attach_failed, transient-Docker anti-flap, re-entrancy guard, name match path, ID-prefix path, short hex hostname ambiguity, non-hex hostname certainty, transition message refresh on observation drift, auto-recreate off (default), auto-recreate success with senchoIp preservation, auto-recreate overlap classification with no subnet drift, throttle window preserves classified reason, throttle release. Lifecycle test covers timer wiring in start()/stop(). Existing mesh-setup-error-classification suite (27 cases) still green. Resolves: F-4 in the v1.0 audit tracker. * fix(mesh): address Codex review of PR #1184 Three findings from the independent review: BLOCKER: attemptInPlaceRecreate() called recordSetupFailure() on create / attach failures, which clears this.senchoIp. The next revalidator tick's attachment check is guarded on senchoIp, so with it null the check is skipped and the snapshot path can silently flip the status back to ok against a network where Sencho is in fact not attached. Also: a later successful recreate would call ensureSelfAttached() with senchoIp null, which short-circuits, so the network gets recreated without binding Sencho. Replaced the recordSetupFailure() calls in attemptInPlaceRecreate with a new recordRecreateFailure() that uses transitionDataPlane and preserves senchoIp. Added two tests: create-fails-then-succeeds (verifies senchoIp survives the failure and the later retry binds Sencho correctly) and create-succeeds-attach-fails-then-next-tick (verifies the snapshot path surfaces attach_failed on the next tick instead of falsely reporting ok). SHOULD-FIX 1: single-key POST /api/settings wrote String(value) without re-validating against the per-key schema, so an allowlisted enum-shaped key like mesh_auto_recreate could persist arbitrary strings ('banana', 'true') that the bulk PATCH would later refuse. Routed the single-key path through SettingsPatchSchema.safeParse so both write paths validate identically. Added regression tests for an invalid mesh_auto_recreate value, a valid mesh_auto_recreate write, and an out-of-range numeric value. SHOULD-FIX 2: the new Mesh data plane subsection lived inside a section the registry exposes to non-admins, who would see the toggle and only learn it was admin-only after the save 403'd. Gated the subsection on `isAdmin` from useAuth so non-admins do not see the control. The other system controls keep their existing visibility pattern (read-only for non-admins). 71/71 backend tests green (revalidate + mesh-setup + settings-routes). 276/276 frontend tests green. tsc clean on backend + frontend.	2026-05-23 18:09:39 -04:00
Anso	fcff8e9047	fix(monitor): collapse repeated host-metric alerts into per-window summary (F-11) (#1175) ... * fix(monitor): collapse repeated host-metric alerts into per-window summary (F-11) A host metric over threshold previously dispatched one notification every 5 minutes for the duration of the breach, producing 7+ identical messages in 35 minutes and spamming Discord/Slack routes. Replace the hardcoded 5-minute cooldown for CPU/RAM/disk with a per-metric suppression window (default 60 min, configurable via host_alert_suppression_mins). The first breach fires immediately; subsequent cycles within the window are silently counted; the next dispatch after the window elapses carries a summary suffix listing how many cycles were suppressed and when the breach first crossed threshold. Recovery clears the counter so re-breach fires fresh. The pattern mirrors PolicyEnforcement.notifyTrivyMissingOnce: module-scope Map, in-memory only, in-cycle dedup, with a test-reset helper. The existing system_state row keeps post-restart re-fires bounded. Janitor and per-stack alert rules are unchanged; they already have adequate cadence and per-rule cooldown respectively. * fix(ci): restore backend and frontend checks * fix(e2e): remove create button timing race * fix(e2e): harden create double-click test * fix(monitor): clear persisted F-11 timestamp on recovery + clamp suppression window Independent audit on the previous commit surfaced two issues. 1. clearHostMetricSuppression early-returned on missing in-memory state, leaving a stale system_state.last_host_*_alert_ts row alive after a process restart. Scenario: breach fires + persists timestamp, process restarts, metric recovers before another evaluate cycle re-seeds the in-memory Map, recovery cleanup early-returns. Next re-breach inside the original window hits the restart-survivability branch and is silently suppressed instead of firing fresh. Fix: read persisted state in clearHostMetricSuppression and reset to '0' independently of in-memory presence. The read-before-write also skips redundant writes when the row is already cleared. 2. host_alert_suppression_mins is validated by zod on the bulk PATCH path but the single-key POST /api/settings path accepts allowlisted keys without re-validation. A 999999999-minute value would silence host alerts for centuries. Add MAX_HOST_ALERT_SUPPRESSION_MIN = 1440 mirroring the zod max, and clamp via Math.min in evaluateGlobalSettings. Two new vitest cases (restart-then-recovery-then-rebreach; the 1440 clamp) confirmed failing before the fix, passing after. The existing "metric drop" case updated to use a mock-backed persistence pattern consistent with the new restart-scenario tests. 73/73 monitor-service tests green; full backend suite 2507/2510 (same pre-existing Windows EBUSY flake on filesystem-backup.test.ts as baseline).	2026-05-23 06:28:01 -04:00
Anso	e05099f2a1	fix(fleet-sync): make control-identity-mismatch sticky and surface in UI (#1117) ... Treat 409 CONTROL_IDENTITY_MISMATCH from a replica as a non-retriable failure instead of looping the same 409 through the 5-minute retry service forever and silently writing identical failure rows. Backend - DatabaseService: add `sticky_error_code`, `sticky_error_expected`, `sticky_error_got` columns to `fleet_sync_status` via an idempotent migration. New methods setFleetSyncSticky, getFleetSyncStickyCode, clearFleetSyncStickyForNode. recordFleetSyncSuccess clears the sticky flag on a clean push. getFailedSyncTargets SQL adds `AND sticky_error_code IS NULL` so the retry loop skips sticky rows. - FleetSyncService.executePushToNode: short-circuits at the top when sticky is set (covers event-driven pushResourceAsync calls). On a 409 with code CONTROL_IDENTITY_MISMATCH, records the failure once and pins sticky with the expected/got fingerprints carried in the 409 body. - routes/nodes.ts: new POST /api/nodes/:id/fleet-sync/reset-anchor. Admin + paid + node:manage. Proxies POST /api/fleet/role/reanchor to the peer with `{override:true}` using the stored Bearer node_proxy token. On peer 200, clears every sticky row for the node so the next push re-anchors and resumes replication. Distinct 502 / 504 responses for peer-rejected / peer-unreachable so the UI can show a useful toast. Frontend - New lib/fleetSyncApi.ts + hooks/useFleetSyncStatus.ts. Polling hook (30s visibilityInterval) skips fetch when !isPaid. - NodeManager.tsx: destructive banner per affected node listing both fingerprints, with `Reset anchor on peer` and `Remove node` buttons. Hidden for community-tier users via empty hook data. - FleetConfiguration.tsx (Fleet -> Status): read-only `Policy sync` SummaryRow per remote node card. In sync / degraded / paused with a tooltip; no action buttons (the action lives in NodeManager). Tests - fleet-sync-service.test.ts: 4 new cases for sticky-set on first mismatch, short-circuit on subsequent pushes, null fingerprints, and non-mismatch failures not setting sticky. - database-fleet-sync-sticky.test.ts (new): 6 cases pinning the DB contract incl. retry-loop SQL filter and migration idempotency. - nodes-fleet-sync-reset-anchor.test.ts (new): 6 cases covering happy path, peer 401 -> 502, peer unreachable -> 504, local-node rejection, unknown node id, and community-tier 403. Gate parity (Directive 30): the new POST .../reset-anchor enforces requireAdmin + requirePaid + node:manage (matches the existing read at GET /api/fleet/sync-status). UI banner + SummaryRow only render when the hook returns data, which it only does for paid-tier authed users. No existing tier-gate file moved; this is greenfield parity. Auth audit: the peer's POST /api/fleet/role/reanchor route already uses requireAdmin, which accepts the central's stored node_proxy Bearer token because authMiddleware maps `scope === 'node_proxy'` to `req.user = { username: 'node-proxy', role: 'admin', userId: 0 }`. No widening required. Backend tsc clean. Frontend tsc -b clean. 59 fleet-sync tests pass; full backend suite green minus the pre-existing Windows-only file-lock flake on filesystem-backup.test.ts that reproduces unchanged on main.	2026-05-19 19:49:16 -04:00
Anso	f6e42535c8	fix(mesh): route peer→central traffic over the existing forward WS (#1094) ... * fix(mesh): route peer→central traffic over the existing forward WS The reverse mesh callback path (`/api/mesh/proxy-tunnel-from-peer`) needed SENCHO_PRIMARY_URL on central plus a publicly reachable origin from the peer's perspective. In a typical homelab where central sits behind NAT, peer→central dispatch silently failed at the dialer's short-circuit and the headline "call any service on any node by hostname" worked one way only. The forward WS at `/api/mesh/proxy-tunnel` is already bidirectional end to end. Make the bridge a persistent control-plane primitive: dial every mesh-enabled proxy peer at startup, reconcile every 60 s, never idle-close. Peer→central traffic multiplexes over the same WS via `tcp_open_reverse`. Removed: - `meshProxyTunnelFromPeer.ts` WS handler and dispatch - `MeshCentralRegistry`, `PeerToCentralMeshSessionDialer` - `mesh_handshake` first-frame state machine in `meshProxyTunnel.ts` - `maybeSendBootstrap`, `buildHandshakeFrame` in the dialer - `mesh_proxy_callback_bootstrap` capability and `maybeWarnUnsetPrimaryUrl` - `mesh_centrals` table (drop migration; greenfield, no users) - `PilotTunnelManager.replaceOrRegisterProxyBridge` (dead after handler removal) - twelve associated unit/integration tests plus the peer-recovery branch in `MeshService.openCrossNode` Added: - `MeshService.proactiveBridgeFanout` selects every mesh-enabled proxy peer (no longer gated on `mesh_stacks` rows) - `startBridgeReconcileLoop` runs the fanout every 60 s (override via `SENCHO_MESH_RECONCILE_INTERVAL_MS`) - `MeshProxyTunnelDialer` default idle TTL is now `0` and exposes `isDialing(nodeId)` for the status surface - `MeshNodeStatus.reverseCallbackStatus` discriminator (`connected | connecting | unavailable | not_applicable`) surfaced via `/api/mesh/status` and rendered as a pill in the Routing tab - `openCrossNode` error message distinguishes "no proxy target" from "waiting for central to dial the reverse bridge" - New tests: `mesh-service-proxy-tunnel-reconcile`, `mesh-status-reverse-callback`, `mesh-proxy-tunnel-dialer-no-idle-close` SENCHO_PRIMARY_URL is no longer required for any mesh function. * fix(mesh): rewrite proxy-tunnel reconcile test contents The previous commit renamed the file but the rewritten test bodies stayed unstaged on top of the rename. This commit lands the actual rewrite: the fanout assertion now requires every mesh-enabled proxy peer to be dialed, not just those with `mesh_stacks` rows, and adds a reconcile-tick repeated-call test.	2026-05-17 22:00:31 -04:00
Anso	50b89db3b8	chore(mesh): drop mesh_stacks table on pilot-mode DBs (#1085) ... Per the C-3 design, mesh state lives on central. Pilots learn aliases via the D-1 override push and hold them in `pilotAliasOverlay`. The local `mesh_stacks` table on pilots has been dead since C-3 shipped. This is a greenfield cleanup (Directive 20): - `migrateMeshTables()` runs `DROP TABLE IF EXISTS mesh_stacks` when `SENCHO_MODE === 'pilot'` and skips the CREATE. Central-mode behavior is unchanged. - The four mesh CRUD methods (`listMeshStacks`, `isMeshStackEnabled`, `insertMeshStack`, `deleteMeshStack`) short-circuit on the same check. Reads early-return empty/false; writes early-return; `insertMeshStack` also warns so an accidental pilot-side write is loud. - New `isPilotMode()` helper mirrors the existing one in `bootstrap/startup.ts`. Layering rules forbid the shared import; a third call-site would justify extraction to `helpers/`. Operator-visible behavior is unchanged on either side. On central, the table and its rows are untouched. On pilots, the table is removed (it held no rows in steady state) and any caller of the CRUD methods continues to see empty list / false / no-op without touching SQLite. Test plan - `npx tsc --noEmit`: clean. - New `database-service-pilot-mesh-stacks.test.ts`: 5/5 green (table absence + four short-circuit assertions). - `mesh-service.test.ts`, `mesh-diagnostic-local.test.ts`, and `mesh-service-proactive-bootstrap-fanout.test.ts`: 58/58 green. - Full backend suite: 2280/2281; the single failure is a pre-existing Windows EBUSY flake in `filesystem-backup.test.ts` that reproduces on a clean tree. - Code reviewed; findings applied.	2026-05-17 05:25:00 -04:00
Anso	6b728599c4	fix(mesh): persist PilotMetrics counters across central restart (#1078) ... The 12 PilotMetrics counters (proxy_dials_failed, proxy_bridges_total, proxy_idle_closes, the pilot-tunnel counters, and the peer-to-central callback counters) live in a singleton holding scalar fields, so process restart wipes them. Operators investigating rotation- or dial-failure trends across a central restart lost the aggregate metric. Persist the snapshot under a pilot_metrics_counters JSON row on the existing system_state KV table. Hydrate via PilotMetrics.load() in bootstrap/startup.ts before any service that increments runs; final-flush in bootstrap/shutdown.ts before the SQLite handle closes. Between those boundaries a buffered flush mirrors the audit-log pattern: every 1 s or every 100 increments, whichever fires first. Hot increment path stays a single property write plus a pending-counter check. Defensive parse on the row: object check + numeric filter, so an operator hand-edit or schema drift across releases degrades to zero state with a warn rather than crashing boot. Schema additions back-fill missing keys with zero so a release that adds a counter does not need a migration. Closes F-R-4 from the mesh tracker.	2026-05-17 01:55:17 -04:00
Anso	cf618dd866	feat(mesh): symmetric WS dial for proxy-mode mesh peers (#1066) ... * chore(mesh): foundation for symmetric callback dial Adds the data-plane scaffolding that the symmetric callback dial fix builds on: - mesh_centrals table for peer-side bootstrap material - MeshCentralRegistry service (upsert/getActive/clear/markUsed/markRejected) - PilotTunnelManager kind discriminator and replaceOrRegisterProxyBridge - mesh_proxy_callback_bootstrap capability registration - MeshProxyTunnelDialer reason-tagged proxy-bridge-down events from a single tearDownBridge emission point - Reactive redial scheduler that skips idle and auth_failed reasons * feat(mesh): add reverse-direction activity log entries (closes R1-B) acceptReverseLocal now emits route.resolve.ok with direction=reverse on connect ack and route.resolve.fail with direction=reverse plus reason=container_not_found / connect_error pre-connect. Post-connect close/error stays silent. Reuses existing event types via the new details.direction discriminator so frontend filters are unaffected. * feat(mesh): add peer-to-central callback dial path (closes R1-A2) Closes the architectural gap where proxy-mode mesh peers could not re-establish their tunnel to central after any non-idle bridge teardown (idle close, network blip, central restart, peer reboot). Central remains the hub for the data plane; the change is purely about WS initiation. Symmetric WS initiation, asymmetric protocol roles. Central retains PilotTunnelBridge ownership; peer retains TcpStreamSwitchboard + reverseDialer ownership. Central bootstraps callback credentials over the first authenticated central-initiated mesh tunnel via a one-shot mesh_handshake JSON frame; peer persists the material in a new mesh_centrals SQLite table and dials central's new /api/mesh/proxy-tunnel-from-peer endpoint when local cross-node traffic needs a bridge and none is live. Mesh_tunnel JWT (HS256, signed with auth_jwt_secret) carries scope, audience, issuer (central instance id), peer api_token fingerprint, kid. Validation on inbound peer dial: algorithm pin, signature, scope, audience, instance, time bounds, node existence and mode, fingerprint match. Failures return HTTP 401 with a machine-readable reason; peer routes the response per a clear-vs-keep cache matrix. Triggers proactive bootstrap on mesh-enable and api_token rotation; central startup fans out to mesh-enabled proxy-mode nodes with mesh_stacks rows (throttled, fire-and-forget). Reactive redial on non-idle bridge loss. Capability-gated handshake send (mesh_proxy_callback_bootstrap) makes the upgrade path safe against older peers in mixed-version fleets. Adds peer-side /api/system/pilot-tunnels centralCallback diag block, bounded counter metrics for bootstrap and dial events. SENCHO_PRIMARY_URL preflight warning when unset on a central with mesh-enabled proxy nodes. Tested with unit suites for the validation chain, registry, manager, and both dialers; integration tests for bootstrap E2E (asserts protocol-role invariant), api_token rotation, instance id change, version skew, and pilot-mode regression. * fix(mesh): green CI on the symmetric callback branch Two independent CI failures, both surgical: 1. Backend tests (11 fails): four mesh test files called setupTestDb in beforeEach. setupTestDb does not reset the DatabaseService singleton, so the per-test afterEach rm of the previous tmpdir left the singleton connection pointing at a deleted file. The next beforeEach's line-55 write threw SQLITE_READONLY_DBMOVED on Linux. Windows file-lock semantics hid this locally. Hoist setupTestDb / cleanupTestDb to file-scope beforeAll / afterAll; per-test state resets stay in beforeEach. Matches the convention in the eight mesh test files that already pass. 2. CodeQL (4 high alerts): js/insufficient-password-hash flagged sha256(api_token) at four sites. The api_token is a 256-bit opaque bearer (sen_sk_-prefixed), not a human password; sha256 is the correct fingerprint primitive for binding the mesh_tunnel JWT to a specific token. Add the two production files plus the two test files that mint the fingerprint to the existing path-scoped query-filter for that rule. * fix(mesh): drop unused afterEach import and revert dead codeql config ESLint flagged afterEach as unused in mesh-central-registry.test.ts:1 after the previous commit hoisted setup/teardown to file-scope beforeAll/afterAll. Remove from the vitest import line. Revert the codeql-config.yml additions from the previous commit. The paths: sub-key under query-filters > exclude is not a documented CodeQL feature and silently no-ops. The four js/insufficient-password-hash alerts on api_token fingerprinting are tracked as dismissed false positives in the GitHub Security tab rather than via dead config.	2026-05-16 14:58:19 -04:00
Anso	e7a3b544c0	fix: harden auto-heal policies (#1042) ... * fix: harden auto-heal policies * fix: resolve auto-heal lint failure	2026-05-14 10:20:57 -04:00
Anso	c31d48b933	fix: harden git source webhooks (#1033) ... * fix: harden git source webhooks * fix: make path validation visible to CodeQL static analysis Add explicit isValidStackName guard in getEnvContent, isValidGitSourcePath pre-validation in readRepoFile, and URL hostname check in remoteStackRequest to satisfy CodeQL taint-tracking so the pipeline passes. * fix: use path.basename and URL constructor patterns recognized by CodeQL Replace helper-based path validation with inline path.basename and path.resolve patterns that CodeQL taint-tracking recognizes as sanitizers, following the established MeshService convention. Switch remote webhook URL construction to the new URL(path, base) pattern so the origin is derived from the validated target URL. * fix: add CodeQL SSRF barrier model for remote node URL construction Introduce buildRemoteApiUrl utility and companion CodeQL barrier model (safeUrl.model.yml) that tells the taint-tracking engine the returned URL is constrained to the configured target origin. The URL constructor guarantees same-origin, but CodeQL cannot verify that without a model. * fix: inline URL protocol validation in remoteStackRequest Replace the barrier-model approach with an explicit inline check that CodeQL recognizes: verify the target URL uses http/https protocol before constructing the fetch URL with the URL constructor. * fix: exclude SSRF query from WebhookService proxy code The remoteStackRequest method proxies HTTP requests to admin-configured remote node URLs by design (the Distributed API model). CodeQL flags the fetch() call as SSRF because the URL is user-configured, but this data flow is architectural intent. Exclude js/server-side-request-forgery from this file. * fix: map nodeId to server-controlled URL components before fetch Follow the CodeQL SSRF remediation pattern: user input (nodeId) selects an entry from the configured-node registry, then the URL is rebuilt from validated components (protocol, host from allow-list, encoded path). Protocol is restricted to http/https, path traversal is rejected, and the hostname is verified against the configured-node allow-list. * fix: remove unnecessary escape in endpoint validation regex	2026-05-13 03:02:21 -04:00
Anso	3b650523c1	Audit-hardening pass for secret and misconfiguration scanning (#977) ... * fix(security): dedupe concurrent compose-stack scans Track stack scans in scanningImages keyed stack:<nodeId>:<stackName>. The /scan/stack route returns 409 when an in-flight scan exists, and the service-side check is the real correctness barrier (the route pre-check is a fast-path optimization that mirrors scanImage). The dedup key release lives in a try/finally so failed scans free the slot for retry. Why: scanComposeStack had no equivalent of scanImage's scanningImages guard, so two simultaneous calls for the same stack would both run trivy config, both insert a vulnerability_scans row, and double- process the result. * feat(security): acknowledge misconfig findings Adds a parallel acknowledgement system for Trivy misconfig findings that mirrors cve_suppressions: a new misconfig_acknowledgements table, read-time enrichment via the new misconfig-ack-filter utility, REST CRUD endpoints, fleet-sync replication from control to replicas, a Settings panel, and an Acknowledge button on the Misconfigs tab. Schema and behavior parity with cve_suppressions: - UNIQUE(rule_id, COALESCE(stack_pattern, '')) so fleet-wide acks collide as expected - blockIfReplica on every write - Audit-log entries name the scope (rule_id, stack_pattern) but never the reason text - replicated_from_control flag controls UI delete affordance and drives clearReplicatedRows on demote/reanchor - Validators reused: validateStackPatternForRedos for glob safety, sanitizeForLog for log fragments SARIF export emits an external/accepted suppression entry per acknowledged misconfig, matching the CVE pattern. Per-row Acknowledge dialog prefills stack_pattern with the scan's stack_context so the default scope is "rule + this stack only" and an operator must broaden explicitly. Tests: misconfig-ack-filter (15) and misconfig-ack-routes (23) including the duplicate-409 case for both pinned and fleet-wide acks. * fix(security): reap orphaned trivy tmp dirs at startup When the buildEnv path writes a per-scan DOCKER_CONFIG dir under os.tmpdir() and the process crashes before the finally block runs, the dir leaks. Mirrors GitSourceService.sweepStaleTempDirs: exported sweepStaleTrivyTempDirs is fire-and-forget at boot, removes prefix-matching dirs older than 1 hour, swallows permission/race failures, logs a single line if any were reaped. * perf(security): emit per-batch summary for scanAllNodeImages Adds one diag() line at the end of scanAllNodeImages summarising unique image count, scanned, skipped, failed, violation count, and elapsed time. Per-image diag inside scanImage stays useful for debugging individual scans; the summary gives operators a single fleet-level checkpoint when developer_mode is on. * perf(security): cap SARIF export at 5000 findings per type Replace the unbounded fetchAllPages walk on /scans/:id/sarif with a hard limit of 5000 findings per type. When any type trips the cap, emit run-level properties.truncated=true plus row_limit and per-type totals so downstream tooling can flag the export as partial. Console-warns for ops visibility. A scan with 50k vulns previously streamed every row into memory before serialising; the cap bounds memory and serialisation time at the cost of completeness on pathological scans. * docs(env): document TRIVY_BIN host-binary override The env var is honored by TrivyService.detectTrivy as a fallback when no managed install is present, but it was undocumented in .env.example. Adds the var with a comment explaining precedence (managed > TRIVY_BIN > PATH). * test(security): cover scanComposeStack failure modes Two new cases drive the existing try/catch through real failure paths: - Malformed Trivy stdout: row flips to status='failed' with the parser error preserved on `error`. - execFile rejection: row flips to status='failed' with a string error message. Pairs with the existing dedup tests so the failure path now also verifies the scan row state, not just the thrown exception. * test(e2e): security scanner + misconfig acknowledgement flow Seven Playwright tests covering the scanner UI and the new acknowledgement system end-to-end: - Trivy availability gate (skips suite when binary absent so CI without Trivy can opt out via E2E_SKIP_TRIVY=1) - Stack config scan completes and records misconfig findings - Concurrent stack scan returns 409 from the dedup gate - Misconfig ack POST creates and lists on Settings - Duplicate (rule_id, stack_pattern) returns 409 - Malformed rule_id (shell metacharacters) returns 400 - Misconfigs tab renders against a real stack scan Tests drive the API for behaviour assertions and the UI only for shell-rendering checks; the visual snapshot suite owns screenshots. * docs(features): add misconfig acknowledgement workflow and SARIF cap Refreshes vulnerability-scanning.mdx with: - Misconfig acknowledgements section covering the per-row dialog, Settings panel, scope/matching rules, and SARIF emission - Tier table row for the new feature - SARIF section note on the 5000 row-per-type cap and the properties.truncated marker for partial exports - Troubleshooting entries: SARIF cap, hidden Acknowledge button, findings resurfacing after delete, Trivy DB phone-home, and 409 on concurrent compose-stack scans * fix(ci): clear backend lint and CodeQL alerts - Remove the dead fetchAllPages helper in routes/security.ts. It lost its callers when the SARIF endpoint switched to direct paged reads for the truncation cap. ESLint flagged it as unused. - Switch the trivy-tmp-cleanup test helper to fs.mkdtempSync. Building paths under os.tmpdir() with predictable names tripped CodeQL's js/insecure-temporary-file rule (high severity), which warns about symlink-pre-creation attacks even in test code. mkdtempSync appends a process-random suffix and creates the dir atomically; the sencho-trivy- prefix is preserved so the production sweep still matches the test fixtures.	2026-05-07 19:23:11 -04:00
Anso	a284732a95	feat(fleet-sync): hide other replicas' identity-scoped policies on a replica (#973) ... GET /api/security/policies on a replica now returns only the policies that apply to THIS replica. Replicated rows with a node_identity targeting a sibling replica are filtered out so an operator cannot enumerate the names and rules of policies meant for another node in the fleet. Defense in depth: the security panel is admin-only, but a backend filter is bypass-proof and matches Sencho's privacy posture. Internal evaluators (getMatchingPolicy, evaluateScanAgainstPolicies) keep using the unfiltered list because they already enforce identity matching at evaluation time. CVE suppressions are fleet-wide on every replica (no node_identity column) so no analogous filter is needed. Public surface: - DatabaseService.getScanPoliciesForUi(role, selfIdentity): the filtered variant, called from securityRouter.get('/policies'). Tests: - 3 new vitest cases: control sees full set; replica hides other-replica scoped rows; replica always includes locally created rows. - Full backend suite: 1795 pass / 5 skipped.	2026-05-07 13:48:11 -04:00
Anso	4007709590	fix(fleet-sync): hygiene pass on receiver behavior and cleanup (#972) ... A bundle of small file-local fixes to the receiver path and node-deletion flow. Changes: - F4 receiver audit log: applyIncomingSync now writes a system audit entry on every applied push so mirrored security-rule changes show up in the replica's audit panel with a clear control-side origin. - F7 pilot-agent skip: pushResource explicitly excludes pilot-agent nodes (they have no api_url for HTTP push) and warns once per node id so the operator sees they will not receive replicated policies. - B4 identity-drift notification: when targetIdentity differs from the cached fleet_self_identity, dispatch a warning so the operator can audit any identity-scoped policies that may need re-targeting. - B6 stack_pattern ReDoS guard: reject patterns with 4+ consecutive wildcards or more than 8 wildcards total. Both control-side validators (POST/PUT scan policies) and the receiver-side row validator share the helper. - B9 deleteNode cascade: clear fleet_sync_status rows for the node inside the existing transaction so the sync-status panel does not render ghost entries after a node is removed. - S6 last_error redaction: formatError strips Bearer tokens and JWT-shaped values from error messages and caps at 500 chars before storing in fleet_sync_status.last_error or logging. Tests: - 8 new vitest cases covering audit-log entry, identity-drift alert, pilot-agent warn-once, formatError redaction (Bearer + JWT), ReDoS validator rejection, and a backtracking-time smoke test. - New database-fleet-sync-cascade.test.ts: deleteNode removes fleet_sync_status rows for the deleted node and leaves siblings untouched. - Full backend suite: 1792 pass / 5 skipped.	2026-05-07 13:41:10 -04:00
Anso	f8c75aa6cd	fix(fleet-sync): clear stale policy_evaluation on replica sync swap (#971) ... Replicated scan policies always insert with fresh ids on the replica side. Any vulnerability_scans.policy_evaluation row referencing the prior set was instantly stale the moment the swap completed, so the replica's UI would surface violations from a policy that no longer exists. replaceReplicatedScanPolicies now calls clearOrphanPolicyEvaluations() inside the same transaction as the delete + inserts, so the cleanup commits atomically with the row swap. Local-only policies and their cached evaluations remain untouched. CVE suppressions do not have an analogous cache (suppressions are applied at read time, never persisted on scan rows), so replaceReplicatedCveSuppressions does not need a sibling call. Tests: - database-replicated-policies.test.ts: stale policy_evaluation cleared after a swap; local-policy evaluation preserved across swaps. - Full backend suite: 1783 pass / 5 skipped.	2026-05-07 13:26:00 -04:00
Anso	7dde257e1f	feat(fleet-sync): replica self-demote endpoint and role UX (#969) ... A replica admin can now demote the instance back to a standalone control without raw SQLite access. The Settings → Security UI surfaces a confirm-gated button when the role is replica; the role probe also surfaces a soft banner when it cannot determine fleet role rather than silently defaulting to control. Backend: - POST /api/fleet/role/demote (admin, requires `{confirm: true}`): flips fleet_role to 'control', clears fleet_self_identity, fleet_control_identity, and both received_pushed_at:* watermarks, drops every replicated_from_control row from scan_policies and cve_suppressions, nulls out any orphaned policy_evaluation cache. Returns 409 ALREADY_CONTROL when invoked on a control. - DatabaseService gains `clearOrphanPolicyEvaluations()` and `clearReplicatedRows()` helpers. Reanchor consolidates onto clearReplicatedRows so it shares the same code path. - `FleetSyncService.demote()` returns boolean for the route to translate into 200 or 409. Frontend: - SecuritySection probes /fleet/role and now records explicit success vs failure rather than silently treating an error as control. A soft banner appears when probe fails. - Replica banner gains a "Demote to control" button and a destructive ConfirmModal explaining the wipe. Tests: - 4 new route-level vitest cases (401, 400 without confirm, end-to-end demote with replica setup, 409 ALREADY_CONTROL with explicit precondition). - Service unit test asserts the consolidated clearReplicatedRows path. - Full backend suite: 1773 pass / 5 skipped. Frontend: 185 pass.	2026-05-07 13:08:21 -04:00
Anso	27660f622b	fix(fleet-sync): version the wire protocol and serialize per-node pushes (#967) ... Hardens the scan-policy and CVE-suppression replication channel as the foundation of a multi-PR fleet sync hardening track. No new endpoints, no new tables, no schema changes; receivers still tolerate legacy payloads (absent pushedAt and controlIdentity) for rollout safety. Wire protocol: - Sender stamps every push with a strictly-increasing pushedAt and a placeholder controlIdentity. Receiver rejects strictly-older pushedAt with 409 STALE_SYNC_PUSH so the next write retries. - pushedAt comparison plus row replacement plus watermark write run in a single SQLite transaction; a partial-write window cannot leave the watermark behind the row state. Concurrency and limits: - Per-node mutex on the sender so concurrent control writes serialize per remote and never apply older state on top of newer. - Sender-side row cap at MAX_SYNC_ROWS=5000 with a 6-hour throttled truncation alert so flapping configs cannot flood the operator. - Route-level body limit raised to 5MB on POST /api/fleet/sync/:resource only; the global 100KB cap is unchanged. Oversize bodies return a structured 413 SYNC_PAYLOAD_TOO_LARGE. Determinism and hygiene: - getMatchingPolicy gains an id-ASC tiebreaker so two replicas resolve the same winner when policies tie on scope class. - Inline comment documents why getMatchingPolicy filters node_id at SQL yet still relies on JS identity matching for replicated rows. - Comment on the receive endpoint documents why no requirePaid is enforced (control's tier authorizes; replica trusts the bearer). - STALE_SYNC_PUSH 409s no longer record a node failure; they are expected protocol outcomes, not health issues. Public surface additions: - DatabaseService.transaction(fn): generic SAVEPOINT-friendly wrapper. - DatabaseService.getLocalScanPolicies / getLocalCveSuppressions: SQL filter on replicated_from_control = 0. - StaleSyncPushError: typed sentinel the route translates to 409. - fleetSyncConstants: shared MAX_SYNC_ROWS, body limit, state-key and error-code maps so the wire protocol has one source of truth. Tests: - 24 new vitest cases across fleet-sync-service, fleet-sync-routes, and database-matching-policy covering monotonic pushedAt, per-node serialization, row truncation and throttle, stale-push suppression, receiver back-compat, oversize-body 413, deterministic matching. - Full backend suite: 1757 pass / 5 skipped.	2026-05-07 12:53:13 -04:00
Anso	0f0b22c51a	feat(fleet): Fleet Secrets tab with env-var bundles (v1 MVP) (#965) ... * feat(fleet): add Fleet Secrets tab with versioned env-var bundles (Skipper+) Centralized, encrypted-at-rest secret bundles that can be pushed to labeled nodes' stacks. Each save bumps a monotonic version; each push records a per-node-per-version row in `secret_pushes` plus an entry in `audit_log`. Conflict detection shows added/changed/unchanged/removed (informational) diffs before write. Overlay merge preserves keys missing from the bundle. - Adds `secrets`, `secret_versions`, `secret_pushes` tables. - New `SecretsService` reuses CryptoService for AES-256-GCM, NodeLabelService for selectors, and direct fetch + Bearer for outbound calls to remote nodes. - New `secretsRouter` with 9 endpoints under `/api/secrets`, gated by `requirePaid`. Mounted after the auth gate. - Audit summary patterns added for the new routes. - New Fleet › Secrets tab with bundle list, editor sheet (key=value rows, versions tab), and push wizard (selector, target stack, env file picker, per-node diff preview, results pills). - Documentation: docs/features/fleet-secrets.mdx + docs.json nav entry. - 26 Vitest cases cover parser, encryption, versioning, push aggregation, tier gating. * fix(fleet): use const for rawValue in env parser ESLint prefer-const flagged the let declaration as a CI-blocking error; the variable is never reassigned.	2026-05-07 06:03:39 -04:00
Anso	52b46753af	feat(fleet): add Federation tab with cordon and pin policy (Admiral) (#964) ... Ships the v1 MVP for the Federation tab as placement control, not placement automation: - Cordon a node: marks the node unschedulable so the BlueprintReconciler skips it for new placements only. Existing deployments continue to drift-check and redeploy on revision changes; cordon never triggers withdraw or eviction. Toggle on the NodeCard kebab (Admiral, admin role); Cordoned pill renders for all tiers. - Pin a blueprint to a node: stores blueprints.pinned_node_id, replacing the desired set with the pinned node regardless of selector. Pin overrides cordon by design. Action lives only in the Federation tab; BlueprintDetail and the deployment table show read-only Pinned indicators. Backend: idempotent migrations add nodes.cordoned/cordoned_at/cordoned_reason and blueprints.pinned_node_id. New routes POST /api/nodes/:id/cordon, POST /api/nodes/:id/uncordon, PUT /api/blueprints/:id/pin, all gated by requireAdmiral plus requireAdmin. Audit summaries added so the existing auditLog middleware records every operator action. deleteNode clears dangling pins. Reconciler: pin override evaluated before selector match; cordon filter applied only to the new-placement branch (deploy/stateReview without an existing deployment). 11 new Vitest cases cover cordon filter, pin override, pin-overrides-cordon, missing pin target, pin shrinks desired set (stateless withdraw + stateful evict_blocked), and pin clearing on node delete. Frontend: new FederationTab.tsx with cordoned-nodes summary and pin-policy table. Federation moved out of the experimental flag into {isAdmiral && (...)} + AdmiralGate, mirroring the Routing tab pattern. Secrets stays under experimental. Tests pass: backend tsc, full Vitest suite (1704 passed), frontend tsc -b, ESLint (0 errors). Manual verification via the local dev instance confirmed the tab is hidden at Community, the kebab and pill render at Admiral, and cordon and pin endpoints round-trip end to end. Refs cut-line-1.0.md Federation v1 MVP.	2026-05-07 05:55:00 -04:00
Anso	775fab7d64	feat(dashboard): replace duplicate Recent Activity card with Fleet Heartbeat / Stack Restart Map (#932) ... * feat: open security basics, manual fleet ops, and basic fleet management to Community Realign tier guards to the user-stated philosophy: Community covers deploy/monitor at scale plus security basics, Skipper adds automation and advanced fleet management, Admiral keeps enterprise control. Community now includes: - Trivy install / uninstall / update from the Settings Hub (admin role) - CVE suppressions CRUD (admin role; replicates fleet-wide) - Manual image scan with vuln, secret, and misconfig results - Stack-config scan, scan comparison - Manual fleet snapshots: create, list, view, restore, delete - Per-node Sencho self-update (Check Updates + per-node Update) - Fleet Overview search, sort, filters, node-card expand, auto-refresh Stays paid: - Scan policies with block_on_deploy enforcement (Skipper+) - SBOM (SPDX, CycloneDX), SARIF export (Skipper+) - Bulk Update All across the fleet (Skipper+) - Scheduled snapshot create (now Skipper, was Admiral) - Trivy auto-update toggle, fleet-wide policy push (Admiral) The Settings -> Security tab is unhidden by setting the registry tier to null. The SecuritySection no longer early-returns a PaidGate; the policy list, Add Policy button, and policy dialogs are wrapped in {isPaid && }. The Fleet view drops isPaid gates on the Snapshots tab, Check Updates button, per-node update handlers, OverviewToolbar grid controls, the NodeCard expand affordance, and the auto-refresh notice. The NodeUpdatesSheet receives a canBulkUpdate prop and gates the Update All button on it. useFleetUpdateStatus and useFleetPolling drop their isPaid guards so polling runs for Community; useFleetOverview drops the isPaid wrap on the filter and sort path. Backend route guards are flipped per the matrix above. The scheduler tick and requireScheduledTaskTier add 'snapshot' to the Skipper+ branch. Backend test assertions are inverted for the now-Community endpoints and a positive Skipper-snapshot-task test is added. Documentation across features/, api-reference/, and operations/ is updated to reflect the new tier mapping. * feat: add node last-contact tracking, fleet latency, and stack-restart summary - DatabaseService: add last_successful_contact column to nodes table via idempotent migration; expose updateNodeLastContact() and getStackRestartSummary() methods; include the column in NODE_COLUMNS so getNodes/getNode return it - fleet.ts: record latency_ms and last_successful_contact on each remote node overview fetch; pilot-agent nodes surface pilot_last_seen instead; pass db singleton into fetchRemoteNodeOverview to avoid redundant getInstance calls - dashboard.ts: replace /recent-activity with /stack-restarts endpoint that groups notification_history events by stack and category (crash/autoheal/manual) over a configurable window (default 7 days, max 30) * refactor(dashboard): remove redundant per-route authMiddleware All routes under /api/ are covered by the global auth gate in app.ts. The inline authMiddleware arguments on /configuration and /stack-restarts were redundant with that gate and inconsistent with every other route in the file. Remove them and drop the now-unused import. * refactor(backend): consolidate Date.now(), move SQL aggregation, normalize node row mapping - Capture a single completedAt timestamp in fetchRemoteNodeOverview to eliminate two separate Date.now() calls and ensure latency_ms and last_successful_contact are derived from the same instant - Inline the redundant contactedAt variable; use completedAt directly - Move stack-restart aggregation from JS into SQL (GROUP BY stack_name with CASE/SUM counts), replacing the Map loop in the route handler - Export StackRestartSummary interface from DatabaseService and remove the duplicate local definition in dashboard.ts; handler now returns the query result directly - Add last_successful_contact normalization in decryptNodeRow, mirroring the existing pilot_last_seen pattern - Add authGate reliance comment above dashboardRouter route handlers * feat(dashboard): replace Recent Activity card with context-aware Fleet Heartbeat / Stack Restart Map - Multi-node installs (≥1 remote node): shows Fleet Heartbeat — real-time reachability, latency, and container count per registered node - Local-only installs: shows Stack Restart Map — 7-day restart frequency per stack grouped by crash / auto-heal / manual category - Conditional wrapper (DashboardActivityCard) switches states automatically when the node list changes, with no page reload required - Deletes RecentActivity card and hook (duplicated data already in Recent Alerts) - Extracts formatRelativeTime to frontend/src/lib/utils.ts for reuse * fix(dashboard): add pilot_last_seen to FleetNodeOverview and use it in getLastSeenLabel * fix(fleet): expose mode and pilot_last_seen in overview, consolidate formatRelativeTime, drop em dash - Add `mode` and `pilot_last_seen` (in seconds) to the FleetNodeOverview interface and to both the pilot-agent and HTTP-proxy return paths in fetchRemoteNodeOverview so the frontend getLastSeenLabel pilot branch can fire correctly - Remove the private formatRelativeTime from RecentAlerts.tsx and use the shared implementation from lib/utils, converting the millisecond timestamp at the call site - Replace the em dash in getLatencyLabel with 'n/a' per project rules	2026-05-05 15:23:05 -04:00
Anso	2e8ca76103	fix(backend): batch-insert stress test metrics to avoid per-insert fsync timeout (#911)	2026-05-04 13:36:01 -04:00
Anso	685d5d729e	feat(blueprints): backend foundation for fleet-wide compose templates (#860) ... * feat(blueprints): add backend foundation for fleet-wide compose templates Introduces the Blueprint Model: a docker-compose.yml plus a node selector (labels or explicit IDs) that Sencho reconciles across the fleet. Backend foundation only; the frontend tab and documentation follow. Schema (DatabaseService): - node_labels table for fleet-level orchestration tagging - blueprints table with compose content, selector, drift_mode, classification - blueprint_deployments table for per-node materialized state - New idempotent migrate methods following the existing pattern Services: - BlueprintAnalyzer: pure compose-YAML classifier (stateless / stateful / unknown) with 17 covered cases including named volumes, bind mounts, external volumes, and tmpfs - NodeLabelService: label CRUD plus selector matching helper (any/all/ids) - BlueprintService: local + remote deploy/withdraw orchestration, marker file management, name-conflict guard, per-(blueprint,node) lock - BlueprintReconciler: 60-second loop with three-mode drift policy (observe/suggest/enforce), state-aware guards, and Enforce-downgrade for volume-destroying drift Routes (gated requirePaid + requireAdmin on mutations): - /api/blueprints (CRUD + apply + withdraw + accept + preview + analyze) - /api/node-labels (CRUD + listAll + listDistinct) Notifications: four new categories registered in NotificationService for deploy/failure/drift events. Bootstrap: reconciler start/stop wired in startup and shutdown. Tests: 45 new Vitest cases covering selector matching, classifier rules, state-aware guards, drift-mode branching, and marker parsing. Full backend suite (1625 tests) passes; tsc clean. * fix(lint): replace bare Function type in blueprint reconciler tests Replace 8 occurrences of `as unknown as { computeDecision: Function }` with a properly typed `ReconcilerWithCompute` alias that mirrors the real method signature. Export `ReconcileDecision` from BlueprintReconciler so the test can reference it. Resolves @typescript-eslint/no-unsafe-function-type errors that were failing the Backend (Lint) CI step.	2026-05-01 18:57:44 -04:00
Anso	7663f4cd8b	feat(fleet): sencho mesh in traffic and routing tab (#858) ... * feat(fleet): sencho mesh in traffic and routing tab Lights up Sencho Mesh: cross-node container forwarding rendered as if the container next to you were on localhost. Builds on the dormant TCP frame plumbing from the prior PR (pilot tunnel TCP frames + sencho-mesh sidecar package) and exposes the Admiral-only orchestrator surface. Backend - New mesh_stacks table (per-node opt-ins) + nodes.mesh_enabled column via DatabaseService.migrateMeshTables. - MeshService singleton: sidecar lifecycle via Dockerode, opt-in/out with cascading override regeneration, request-based resolver from sidecar control WS, cross-node TCP forwarding via PilotTunnelManager (same-node fast path included), in-memory 1000-event activity ring buffer with durable mirror to audit_log for state-change events, per-node and per-route diagnostics, and the Test upstream probe. - MeshComposeOverride: pure YAML generator that injects extra_hosts using host-gateway. The user's docker-compose.yml is never mutated; overrides live under DATA_DIR/mesh/overrides. - ComposeService deploy/update splice the override file when the stack is opted in; non-mesh stacks behave identically to today. - Pilot agent resolveMeshTarget consults the local mesh_stacks table (defense in depth) and resolves Compose containers via Dockerode. - /api/mesh router with 13 Admiral-gated endpoints covering status, enable/disable, stack opt-in/out, alias listing, per-route diagnostic, Test upstream probe, per-node diagnostic, sidecar restart, activity log paginated and SSE. - meshControl WS slot at /api/mesh/control validates the mesh_sidecar JWT minted by MeshService; dispatched as upgrade slot 2 (canonical order preserved). Frontend - New Traffic Routing tab in FleetView, gated by isAdmiral and wrapped in AdmiralGate. Tab uses the cyan brand glyph and italic-serif state typography from the audit. - RoutingTab masthead with mesh activity drawer, per-node card grid with TogglePill, alias rows with five-state pill taxonomy (healthy / degraded / unreachable / tunnel-down / not-authorized), inline Test buttons. - Four sheets: opt-in picker with port-collision inline error, per-route detail with diagnostic + filtered activity, per-node diagnostics with active streams + resolver cache + restart action, fleet-wide activity log with filters. - meshRouteState helper centralizes pill-state mapping; pure-function tests cover all five states. Docs - User docs at /docs/features/sencho-mesh.mdx covering opt-in, troubleshooting, security model (4 guarantees + 4 explicit non-guarantees), and V1 limitations. - Internal architecture and runbook pages. - websocket-dispatch internal doc updated with the new slot. * fix(mesh): validate stack name before path use; fix test DB lifecycle Two surgical fixes against the prior PR. Path-injection (CodeQL js/path-injection): MeshService.optInStack, optOutStack, ensureStackOverride, and removeStackOverride now validate stackName via isValidStackName from utils/validation, reject malicious names at the API boundary, and additionally check isPathWithinBase on the resolved override file path for defense in depth. The dataflow from req.params.stackName to fs.writeFile no longer reaches an unsanitized path expression. Test DB lifecycle: mesh-service.test.ts used per-test setupTestDb / cleanupTestDb, which deletes the temp dir while DatabaseService still holds an open SQLite handle. On Linux CI this raises SQLITE_READONLY_DBMOVED on the next prepare() because the inode has been unlinked. Switched to file-scoped beforeAll/afterAll matching agents-routes.test.ts, with a per-test beforeEach that truncates mesh_stacks plus non-default nodes and resets the MeshService singleton in-memory state. Adds a new test case asserting the path-traversal rejection. * fix(compose): use discovered compose filename instead of hardcoded docker-compose.yml composeArgs() hardcoded `-f docker-compose.yml` for every deploy. Sencho writes its canonical compose file as `compose.yaml`, so any stack created via the UI failed to deploy with `open ...docker-compose.yml: no such file or directory`. When no mesh override applies, drop the explicit `-f` so docker compose's built-in discovery resolves the actual filename. When an override exists, look up the real base filename via FileSystemService.getComposeFilename() and pass both files explicitly. Also hoist the MeshService import to module top now that the dependency is known to be acyclic, and revert the matching unit-test assertion.	2026-05-01 01:50:53 -04:00
Anso	3e01daf76f	feat(stack): per-stack activity timeline with actor attribution (#852) ... * feat(stack): per-stack activity timeline with actor attribution Adds an Activity tab to the Stack Anatomy panel showing a timestamped event log for each stack: deploys, restarts, starts, stops, and image updates, attributed to the user who triggered them or 'system' for automated actions. Backend: - Extends notification_history with actor_username column (idempotent migration) and a partial composite index on (node_id, stack_name, timestamp DESC) for efficient per-stack lookups. - NotificationService.dispatchAlert() accepts an optional actor that is written to the new column. - Success-side dispatchAlert calls added after deploy, bulkContainerOp (start/stop/restart), and update handlers in routes/stacks.ts so user-initiated operations are recorded, not just failures. - New GET /api/stacks/:stackName/activity?limit&before endpoint with stack:read permission gate and cursor-based pagination. Frontend: - StackAnatomyPanel grows an Anatomy / Activity tab pair using the existing Tabs primitive. - StackActivityTimeline fetches the initial 50 events, paginates on demand, and prepends live events arriving over the existing WS notifications stream without duplicates. - NotificationPanel bell dropdown suppresses user-initiated success events (start/stop/restart/deploy/update triggered by a real user), keeping the tray focused on alerts and system events. * docs(stack): add stack activity timeline feature page and internal arch docs * fix(test): add actor_username to notification-routing history assertions dispatchAlert now passes actor_username to addNotificationHistory after the activity timeline PR added the column. Update the two exact-match assertions that were failing because the expected object shape was missing this field.	2026-04-30 19:53:23 -04:00
Anso	5cf4323511	perf(backend): batch audit_log inserts into a buffered transaction (#817) ... Every mutating /api/* request runs an individual INSERT into audit_log which serializes against other writers under burst load (SQLite's single-writer model). Buffer the writes in DatabaseService and flush them in a single transaction either every second or once the buffer reaches 100 entries, whichever comes first. Read paths (getAuditLogs, getAuditLogsInRange, cleanupOldAuditLogs) drain the buffer first so callers always see a consistent view, which keeps the existing test pattern of insert-then-read working. Graceful shutdown flushes before db.close() so no entries are lost on clean exit. The 1s flush timer is unref'd so the buffer cannot keep the process alive on its own. The CLI resetMfa script flushes explicitly before returning since it exits before the timer fires.	2026-04-28 00:59:04 -04:00
Anso	836e384d17	perf(backend): cache global_settings reads in DatabaseService (#814) ... getGlobalSettings() runs a SELECT * on every call and is hit from 22 files, including the auth middleware (every authenticated request), the WebSocket upgrade handler (every connection), and the debug-mode gate (every diagnostic log line). Cache the result inside the service on first read and invalidate on updateGlobalSetting(). The cached snapshot is Object.freeze'd and the public return type is now Readonly<Record<string, string>> so accidental mutations are caught at compile time. The settings GET handler that delete'd private keys now takes a defensive shallow copy first. The 5-second TTL cache in utils/debug.ts is now redundant and removed; the service-level cache is strictly fresher (invalidates on write rather than going stale for up to 5s).	2026-04-27 23:45:25 -04:00
Anso	abee078741	feat(scheduler): add auto_backup, auto_stop, auto_down, auto_start and delete_after_run one-shot mode (#777) ... * feat(scheduler): add auto_backup, auto_stop, auto_down, auto_start actions and delete_after_run one-shot mode Extends the scheduler with four new stack-targeted actions: - auto_backup: backs up stack compose files and .env using the existing FileSystemService.backupStackFiles primitive - auto_stop: runs compose stop (containers preserved) - auto_down: runs compose down (containers removed) - auto_start: runs compose up -d via deployStack (universal start for both stopped and down stacks) Adds delete_after_run boolean column to scheduled_tasks. When enabled, the task self-deletes after its first successful execution; failures keep the task so the user can debug and retry. All four new actions gate at Admiral tier, consistent with restart/snapshot/prune. Migration is idempotent (maybeAddCol). * docs(scheduler): update scheduled-operations doc with new lifecycle actions and delete-after-run Adds the four new actions (Backup Stack Files, Stop Stack, Take Stack Down, Start Stack) to the action table. Documents the delete-after-run one-shot mode with its success-only deletion semantics. Adds the Stack Lifecycle Scheduling section explaining stop-vs-down semantics and the local-execution boundary. Adds three troubleshooting entries: auto-start on a missing compose folder, auto-backup single-slot overwrite by design, and one-shot task disappearing after successful run. Updates the timeline description from four to five lanes. Refreshes screenshots to show the new dialog layout with the Lifecycle lane visible.	2026-04-25 16:11:13 -04:00
Anso	e0034132b4	feat(notifications): match routing rules by labels and categories (#776) ... - Add label_ids and categories columns to notification_routes via idempotent migration - Matcher logic always evaluates routes (AND semantics across all non-empty matchers) - getStackLabelIds skips DB call when no enabled route uses label filtering - Extract ALL_NOTIFICATION_CATEGORIES array from NotificationService as single source of truth - Derive VALID_CATEGORIES set from the array in the route handler - Extract validateLabelIds and validateCategories helpers to remove POST/PUT duplication - Extract tryAddColumn as a private DatabaseService class method (removes 5 local re-declarations) - Extract CATEGORY_LABELS to frontend/src/lib/notificationCategories.ts (shared by NotificationPanel and NotificationRoutingSection) - Frontend form adds label and category multiselects with AND-filter hint - Route cards show label and category badges; empty-matcher routes show 'Matches all alerts' - Add tests for category-only, label-only, and combined AND-semantics routing	2026-04-25 14:58:50 -04:00
Anso	fcbdd59ec2	fix(notifications): scope routing rules to nodes via node_id column (#775) ... Adds a nullable node_id column to notification_routes (null = any node, integer = fire only when the alert originates from that specific node). This fixes a multi-node fleet defect where a route scoped to "my-app" would fire on every node that hosts a stack with that name. Backend changes: - DatabaseService: idempotent migration adds node_id INTEGER NULL and a composite index on (node_id, enabled, priority); the two statements are in separate try-catch blocks so the index is always created even when the column was added in an earlier run - NotificationService: route matcher now pre-filters by node_id before checking stack_patterns (== null matches any node) - notifications route: POST/PUT accept optional node_id, validated to be null or the local node's ID; NodeRegistry guards against cross-node misroutes Frontend changes: - NotificationRoutingSection: node scope Select field uses useNodes() from NodeContext (no extra API call) to populate the local node option - Route cards show a node badge when node_id is set Tests: 3 new tests covering node-match, node-mismatch, and null-scope; all 75 files (1413 tests) passing.	2026-04-25 13:56:48 -04:00
Anso	44dba59cab	feat(notifications): add structured category enum to dispatcher and history (#774) ... Introduce a NotificationCategory string-literal union (11 values) and thread it through dispatchAlert as a required second argument. All callers (DockerEventService, AutoHealService, ImageUpdateService, MonitorService, PolicyEnforcement, policyGate, SchedulerService, imageUpdates route) pass an explicit category at every call site, giving TypeScript compile-time enforcement that no new emit site can be added without choosing a category. DatabaseService gains an idempotent migration that adds a nullable category TEXT column to notification_history; existing rows keep category=NULL (displayed as Uncategorized in the UI). The getNotificationHistory method accepts an optional category filter that is forwarded from the GET /api/notifications/history route via a ?category= query param. NotificationPanel gains a category Select dropdown so users can filter history by category. The frontend types mirror the backend union so API responses are type-safe end-to-end. All 75 test files (1410 tests) updated to the new 4-arg dispatchAlert signature and passing.	2026-04-25 13:55:07 -04:00
Anso	af9cb0aa63	feat(auto-update): per-stack auto-update enable/disable toggle (#771) ... * feat(auto-update): add per-stack auto-update enable/disable toggle Paid users (Skipper and Admiral) can now opt individual stacks out of scheduled auto-updates from the stack context menu without disabling the global feature. - Add stack_auto_update_settings table (node_id, stack_name) with default enabled=true; four typed DatabaseService accessors with parameterized queries. - Add GET /stacks/auto-update-settings, GET /stacks/:name/auto-update, and PUT /stacks/:name/auto-update (requirePaid + requireAdmin). PUT broadcasts state-invalidate with action auto-update-settings-changed so all open tabs refresh immediately. - Stack DELETE clears the auto-update setting row alongside stack_update_status. - autoUpdateRouter /execute skips disabled stacks before any registry call; skip is recorded in the results array. Manual Update actions are not affected. - Add Auto-update: Enabled/Disabled toggle in the stack inspect group (paid tiers only, hidden for Community, consistent with Auto-Heal). Toggle uses optimistic update with revert-on-error toast. - AutoUpdateReadinessView shows an Auto: Off pill and disables the Apply now button for stacks with auto-updates off. Detection still runs so the readiness card remains visible. - Add 21 backend Vitest tests covering DB round-trips, endpoint auth and tier gates, execute skip for both wildcard and named targets. Add 3 frontend hook tests for toggle visibility and callback behavior. * docs(auto-update): document per-stack auto-update control Add a Per-stack control section to the auto-update readiness page explaining how to disable and re-enable auto-updates for individual stacks, what disabling means (scheduled apply skipped; detection still runs; manual update unaffected), and a troubleshooting entry for scheduled runs not applying to a specific stack.	2026-04-25 10:50:21 -04:00
Anso	24c0a2833b	fix(security): clear cached policy evaluations when a scan policy is deleted (#758) ... Vulnerability scans cache their policy verdict as a JSON blob in vulnerability_scans.policy_evaluation. Deleting a scan policy used to remove only the policies row and leave those blobs intact, so the scheduler kept emitting violations and stacks remained marked as blocked against a policy that no longer existed. deleteScanPolicy now nulls out policy_evaluation on every scan whose JSON references the deleted policy id, then deletes the policy row, in one transaction.	2026-04-24 22:28:25 -04:00
Anso	ca5a930c68	refactor(backend): extract authMiddleware and introduce createApp factory (phase 2) (#732) ... Phase 2 of the index.ts refactor. Pulls the auth middleware and session cookie issuers into their own module, and introduces the app.ts factory that owns the first nine steps of the canonical middleware pipeline. New modules: - middleware/auth.ts: authMiddleware, issueSessionCookie, issueMfaPendingCookie, clearMfaPendingCookie - app.ts: createApp() factory installing trust proxy, helmet, cors, compression, cookieParser, rate limiters, conditionalJsonParser, and nodeContextMiddleware. A header comment documents all 16 canonical middleware steps and where each currently lives. Changes: - middleware/authGate.ts: createAuthGate factory removed; authGate now imports authMiddleware directly (the factory existed only to avoid a circular import while authMiddleware lived in index.ts). - services/DatabaseService.ts: added API_TOKEN_SCOPE_TO_ROLE map so the auth middleware no longer inlines a stringly-typed record. - index.ts drops ~260 lines; auth routes, authGate, auditLog, apiTokenScope, remaining routes, static serving, and the error handler continue to be registered there until their respective phases. - vitest.config.ts bumps testTimeout to 30s and hookTimeout to 45s so fork-pool workers have enough headroom to ts-node-transform the growing module graph under CPU contention (64 workers each import the full Express stack in beforeAll). Code review fixes: use getErrorMessage() util in the auth catch block instead of inline cast; promote the scope-to-role map to a typed module-level constant.	2026-04-23 19:02:13 -04:00
Anso	661b9c638b	feat(security): enforce scan policies as a pre-deploy gate (#719) ... Policies with block_on_deploy=1 now scan every stack image before docker compose up runs and reject the deploy with HTTP 409 on violation. The UI opens a dialog listing offending images; admins can override per deploy with ?ignorePolicy=true, and every bypass is recorded in the audit log with the originating route, actor, policy, and image list. When Trivy is not installed on the target node the gate fails open with a warning notification, so teams are never locked out by tooling state. Post-deploy and scheduled scans still evaluate matching policies and dispatch warnings on violations to surface drift on long-running stacks. Public API additions: policy and suppression CRUD under /api/security, plus the documented 409 block-response shape on all deploy paths.	2026-04-21 00:14:11 -04:00
Anso	08f57c7141	feat(settings): surface security, notifications, and app store on remote nodes (#716) ... Flip Security (Trivy), Notifications (agents + history), and App Store from global-and-hidden-on-remote to node-scoped so operators can manage them when a remote node is selected in the node picker. The primary instance proxies the calls to each remote, which resolves the correct per-instance binary state, agent config, and template registry. Backend: key `agents` and `notification_history` by `node_id` with idempotent column-add migrations and a `(node_id, type)` unique index on agents, matching the Labels pattern. Thread `req.nodeId` through the /api/agents and /api/notifications routes. Internal NotificationService and ImageUpdateService writes resolve the middleware default via `NodeRegistry.getDefaultNodeId()` so monitor-emitted rows share a bucket with user-facing ones (avoids split-brain where the UI sees test notifications but not internal alerts). Frontend: split Security on remote to render only the scanner card and hide scan policies and CVE suppressions (those remain control-plane-only). Drop the misleading "Always Local" badge on Developer since retention windows govern backend jobs, not UI state. Flip the App Store registry to node-scoped. Docs: add a "What Settings apply per node" table to multi-node, clarify remote alert setup in alerts-notifications, and note Trivy's per-host install in vulnerability-scanning.	2026-04-20 21:04:09 -04:00
Anso	b95442c8f7	feat(ui): redesign global logs as cockpit surface (#699) ... Rework the Logs view into the cockpit language: a PageMasthead strip with cyan rail, pulsing live dot, italic state word, and tracked-mono kicker; a SignalRail of EVENTS/MIN (with rolling 60s sparkline), ERRORS, WARNINGS, and CONTAINERS tiles; a segmented filter strip; a day-banded feed with severity dots and whole-row tint on ERROR/WARN; and a floating glass chip strip for pause/resume, clear, and download. Decouple the live log stream from the Developer Mode toggle so SSE runs by default for everyone, with polling as an invisible fallback when EventSource is unavailable. Drop the Standard Log Polling Rate setting and the global_logs_refresh key it persisted; Developer Mode still gates Real-Time Metrics and Debug Diagnostics and nothing else. Introduces the shared PageMasthead and SignalRail primitives for reuse across other cockpit pages.	2026-04-19 16:25:54 -04:00
Anso	ed2a16af79	feat(notifications): deep-link bell rows to source stack and container logs (#692) ... Add stack_name and container_name columns to notification_history so bell rows can act as jump points. Producers (AutoHeal, Docker events) pass the container context through dispatchAlert; the panel renders routable rows as buttons that load the target stack and, when a container name is present, open its logs modal. Non-structural notifications stay as passive display rows.	2026-04-19 03:56:59 -04:00
Anso	591dc75d1e	feat(audit-log): signal rail, day-banded stream, anomaly detection (#682) ... Add a Stream view to the Audit Log that leads with a four-tile signal rail (events, actors, failure rate with inline sparkline, peak hour) and presents the feed grouped by day with severity dots, relative times, and inline anomaly callouts. The existing Table view is preserved behind a toggle for power users. Anomaly flags are computed at read time against strictly prior history and returned on demand via ?with_anomalies=1: - unusual_hour: hour outside the actor's central 7-day window - new_ip: IP unseen for this actor in the last 30 days - first_seen_actor: no prior history in the 30-day window New /audit-log/stats endpoint returns the signal-rail aggregates over 24h/7d/30d windows; stats are derived from a single 30-day scan.	2026-04-18 18:24:27 -04:00
Anso	ec7620675e	feat(app-store): editorial hero, category rail, security scan signal per tile (#679) ... * feat(app-store): editorial hero, category rail, security scan signal per tile Rework the App Store view into an editorial layout with a 180px category rail, a featured-template hero, and compact tiles that surface star counts and vulnerability scan status at a glance. The deploy sheet splits into Essentials (one-click deploy) and Advanced (full port/volume/env control) tabs. Backend enriches /api/templates with scan_status, scan_cve_count, and a featured flag computed from the highest-star template with recorded stars. Scan lookups use a single batched SQL query to avoid N+1 round-trips. * fix(app-store): split firstSentence helper into its own module The firstSentence helper lived alongside the TemplateLogo component, which violates react-refresh/only-export-components — a file that exports a component must not also export non-component values, or Fast Refresh cannot establish an HMR boundary. Move the helper into appstore/util.ts and update the two import sites.	2026-04-18 12:55:50 -04:00
Anso	5bb4b01953	feat: auto-heal policies for unhealthy containers (#671) ... * feat(db): add auto_heal_policies and auto_heal_history schema and CRUD Adds two new SQLite tables (auto_heal_policies, auto_heal_history) to DatabaseService.initSchema() and exposes CRUD methods: getAutoHealPolicies, getAutoHealPolicy, addAutoHealPolicy, updateAutoHealPolicy, deleteAutoHealPolicy, recordAutoHealHistory, getAutoHealHistory, incrementConsecutiveFailures, resetConsecutiveFailures, setPolicyEnabled. Also adds AutoHealPolicy and AutoHealHistoryEntry TypeScript interfaces. * feat(events): track health-status duration and expose state accessors - Add healthStatus and unhealthySince fields to InternalContainerState - onHealthStatus now records unhealthySince timestamp on first transition to unhealthy, and clears it when the container recovers or restarts - onStart resets both fields so a restarted container begins from 'starting' - Add listContainerStates() and getContainerState() public accessors for use by the upcoming AutoHealService evaluator * fix(auto-heal): key allowlist in updateAutoHealPolicy, cascade delete, extract ContainerHealthSnapshot * feat: add AutoHealService evaluator singleton Polls every 30 s, matches containers to enabled policies via Compose labels, and restarts containers that have been unhealthy beyond the configured threshold. Enforces cooldown, per-hour rate cap, and recent-user-action suppression; auto-disables policies after repeated consecutive failures. Also adds DockerEventManager.getService() accessor required by the evaluator. * fix(auto-heal): prune stale restartTimestamps, guard undefined policy id - Prune restartTimestamps entries for containers no longer running after each container list fetch, preventing unbounded map growth from dead container IDs. - Guard against policies with undefined id at the start of the per-policy loop; warn and skip rather than proceed with a non-null assertion. - Extract handleAutoDisable private helper to bring executeHeal under 30 lines and isolate the auto-disable side-effect sequence. - Move ContainerInfo type to module scope. * feat: add auto-heal API routes and wire AutoHealService lifecycle Registers five REST endpoints under /api/auto-heal/policies (list, create, patch, delete, history) with requirePaid + requireAdmin guards and Zod validation. Wires AutoHealService.start()/stop() into the server startup and graceful-shutdown blocks alongside MonitorService. * test: add AutoHealService and DatabaseService auto-heal unit tests - 15 unit tests for AutoHealService.shouldHeal covering all decision branches (healthy state, duration threshold, user-action suppression, cooldown, rate limiting, and correct skipReason values) - 13 integration tests for DatabaseService auto-heal CRUD: policy round-trip, stack-name filter, partial update, cascade delete, history ordering/limit, consecutive failure counters, and setPolicyEnabled toggle * fix: log AutoHealService shutdown errors consistently * fix(api): requireAdmin-first guard order and try/catch on auto-heal routes * feat(ui): add StackAutoHealSheet component * feat(ui): add Auto-Heal context menu item to EditorLayout * fix(ui): StackAutoHealSheet label, token, a11y, and useEffect fixes - Rename 'All services in stack' to 'All services' in combobox options and placeholder - Replace text-green-600 with text-success design token in actionColorClass - Add htmlFor/id pairs to all four numeric form inputs for accessibility - Inline fetch logic into useEffect, removing stale closure risk and eslint-disable comment - Remove now-unused fetchPolicies and fetchServices standalone functions - Update 'Auto-disable after' label to 'Auto-disable after (failures)' for clarity - Add toast.error in policy fetch failure path; services fetch silently skips as before * docs: add auto-heal-policies feature documentation * test(e2e): add auto-heal policies CRUD spec * fix(docs): correct auto-heal-policies nav position in docs.json	2026-04-17 22:45:06 -04:00
Anso	8e7a567f69	feat: pilot agent outbound-mode for remote nodes (#667) ... * feat: pilot agent outbound-mode for remote nodes Adds a second mode for managing remote nodes: the agent dials an outbound WebSocket tunnel to the primary, so the remote host no longer needs an inbound port, a reachable URL, or its own TLS certificate. Works behind NAT, residential routers, and corporate firewalls. The primary multiplexes HTTP and WebSocket requests over a single tunnel via a hybrid JSON + binary frame protocol, bridged through a per-tunnel loopback server so existing proxy and upgrade handlers route pilot-mode nodes identically to proxy-mode ones. Enrollment uses a single-use 15-minute pilot_enroll JWT exchanged for a long-lived pilot_tunnel credential on first connect. Proxy mode continues to work unchanged and both modes are supported side-by-side. * test(e2e): switch to proxy mode before asserting api_url field Remote nodes default to Pilot Agent mode, which hides the api_url input. The SSRF-validation tests need proxy mode, so the helper now selects Distributed API Proxy after picking Remote type before asserting the field is visible. * fix(e2e): wire Combobox id prop so node-mode selector resolves The Combobox trigger button had no id, leaving its Label orphaned and making getByRole name-based lookups fail. Adding id to the primitive, passing id="node-mode" from NodeManager, and updating the E2E helper to use #node-mode fixes both the a11y regression and the CI timeout.	2026-04-17 20:31:43 -04:00
Anso	2fce1d3baf	fix(security): server-driven pagination for scan history (#661) ... Scan history fetched a fixed 200 most-recent rows and paginated them client-side, so older scans silently fell off mature nodes where baselining is most valuable. The list now fetches one page at a time via offset, with status=completed and imageRefLike filters applied server-side. Search input is debounced to avoid per-keystroke fetches.	2026-04-17 14:06:23 -04:00
Anso	a95bf1ff33	feat(security): secret and misconfiguration scanning (#651) ... Extends Trivy scans with secret detection in image filesystems and misconfiguration scanning for Compose stacks. Adds tabs to the scan drawer for vulnerabilities, secrets, and misconfigs. Secret matches are redacted server-side (first 8 chars + ellipsis) before storage. - TrivyService: --scanners vuln,secret for images; trivy config for stacks - DB: scanners_used/secret_count/misconfig_count cols; secret_findings, misconfig_findings tables; cache key scoped by scanners - Routes: POST /security/scan accepts scanners array (requirePaid when secret requested); POST /security/scan/stack; GET .../secrets and .../misconfigs (paid-tier reads) - UI: tabs in VulnerabilityScanSheet; scan-options dropdown on images; Scan config button on stack header	2026-04-17 07:57:08 -04:00