12 Commits

Author	SHA1	Message	Date
Anso	6b728599c4	fix(mesh): persist PilotMetrics counters across central restart (#1078) ... The 12 PilotMetrics counters (proxy_dials_failed, proxy_bridges_total, proxy_idle_closes, the pilot-tunnel counters, and the peer-to-central callback counters) live in a singleton holding scalar fields, so process restart wipes them. Operators investigating rotation- or dial-failure trends across a central restart lost the aggregate metric. Persist the snapshot under a pilot_metrics_counters JSON row on the existing system_state KV table. Hydrate via PilotMetrics.load() in bootstrap/startup.ts before any service that increments runs; final-flush in bootstrap/shutdown.ts before the SQLite handle closes. Between those boundaries a buffered flush mirrors the audit-log pattern: every 1 s or every 100 increments, whichever fires first. Hot increment path stays a single property write plus a pending-counter check. Defensive parse on the row: object check + numeric filter, so an operator hand-edit or schema drift across releases degrades to zero state with a warn rather than crashing boot. Schema additions back-fill missing keys with zero so a release that adds a counter does not need a migration. Closes F-R-4 from the mesh tracker.	2026-05-17 01:55:17 -04:00
Anso	8dd0fce621	fix(fleet): show capabilities, version, metrics, and stacks for pilot-agent nodes (#1044) ... When a pilot-agent node was the active node, the UI rendered "does not advertise this capability" across most tabs, a perpetual "Update available" badge, and a Fleet card body with blank CPU/RAM/Disk and "No stacks found". The cause was central-side aggregators in /api/fleet/* and /api/nodes/:id/meta only fanning out to proxy-mode remotes via node.api_url + node.api_token, which are null for pilot-agent. Route every affected aggregator through NodeRegistry.getProxyTarget so the loopback URL backed by the active pilot tunnel is used uniformly: - /api/nodes/:id/meta and /api/fleet/update-status fetch via the new NodeRegistry.fetchMetaForNode helper (resolves the target, delegates to fetchRemoteMeta, returns the shared OFFLINE_META on null). - fetchRemoteNodeOverview, /api/fleet/configuration, /api/fleet/node/:nodeId/stacks, and the stack-containers drilldown fetch through target.apiUrl with conditional Authorization. - fetchRemoteMeta omits the Authorization header when the token is empty (pilot-agent loopback) instead of sending a malformed Bearer string. - Pilot-agent rows preserve pilot_last_seen and mirror it into last_successful_contact so the Fleet "last seen" cell renders the recent tunnel timestamp during a brief reconnect. Pilot-mode capability filter excludes capabilities whose central-pilot path is not yet wired (host-console, self-update). Without this, the Console tab would surface for an Admiral pilot session and click through to central's host because the WS upgrade handler still gates on api_url + api_token. Filtered capabilities are removed at boot via applyPilotModeCapabilityFilter when SENCHO_MODE=pilot. Cache invalidation on tunnel-up: the meta cache for a reconnecting pilot is dropped so the next request rebuilds capabilities and version through the live bridge instead of waiting for the 3-minute TTL. The namespace constant moves to helpers/cacheInvalidation.ts alongside the new invalidateRemoteMetaCache helper. Husky commit-msg hook: add the missing shebang and a .gitattributes rule pinning .husky/* to LF line endings so commits do not fail with "Exec format error" on Windows shells where autocrlf=true converts the hook to CRLF. Tests cover Authorization-header behavior, pilot-mode filter idempotency, fetchMetaForNode dispatch (offline target, pilot-agent loopback, proxy-mode), and the four affected fleet routes for pilot-agent both when the tunnel is up and when it is down.	2026-05-14 10:21:08 -04:00
Anso	3d1cf0f8f1	fix(pilot): generate auth_jwt_secret on pilot-agent boot (#997) ... Pilot-agent hosts never run the first-run setup wizard, so the wizard path that normally generates auth_jwt_secret (routes/auth.ts) never fires. Without that secret, the agent-side loopback auth helper added in PR #990 (pilot/agent.ts::getLoopbackAuthHeader) returned null at mint time, no Authorization header was attached on the forwarded loopback request, and the local Sencho's authMiddleware rejected every proxied call with 401 "Authentication required". This was reproducible end-to-end on v0.74.1, .2, and .3. Add ensurePilotJwtSecret() to bootstrap/startup, called early in startServer (before LicenseService.initialize, which only reads system_state and does not touch global_settings, so the order is safe). When SENCHO_MODE=pilot and global_settings.auth_jwt_secret is empty, generate 64 random bytes (hex-encoded, matching the wizard's pattern in routes/auth.ts) and persist. Subsequent boots see the persisted value and no-op. Outside pilot mode the function is a no-op since the wizard owns the lifecycle. The helper is exported so the regression test can exercise the real function rather than maintaining a hand-mirrored copy. Three vitest cases cover: first-boot generation, no-op on persisted secret, no-op in non-pilot mode. Together with PR #989 (proxy-side bridge dispatch), PR #990 (agent-side loopback auth injection), PR #992 (cross-node control plane via HTTP proxy), and PR #994 (local-node mesh-reachable diagnostic), this completes the central, bridge, agent, local-Sencho HTTP path for pilot-agent-mode nodes.	2026-05-08 11:59:02 -04:00
Anso	3b650523c1	Audit-hardening pass for secret and misconfiguration scanning (#977) ... * fix(security): dedupe concurrent compose-stack scans Track stack scans in scanningImages keyed stack:<nodeId>:<stackName>. The /scan/stack route returns 409 when an in-flight scan exists, and the service-side check is the real correctness barrier (the route pre-check is a fast-path optimization that mirrors scanImage). The dedup key release lives in a try/finally so failed scans free the slot for retry. Why: scanComposeStack had no equivalent of scanImage's scanningImages guard, so two simultaneous calls for the same stack would both run trivy config, both insert a vulnerability_scans row, and double- process the result. * feat(security): acknowledge misconfig findings Adds a parallel acknowledgement system for Trivy misconfig findings that mirrors cve_suppressions: a new misconfig_acknowledgements table, read-time enrichment via the new misconfig-ack-filter utility, REST CRUD endpoints, fleet-sync replication from control to replicas, a Settings panel, and an Acknowledge button on the Misconfigs tab. Schema and behavior parity with cve_suppressions: - UNIQUE(rule_id, COALESCE(stack_pattern, '')) so fleet-wide acks collide as expected - blockIfReplica on every write - Audit-log entries name the scope (rule_id, stack_pattern) but never the reason text - replicated_from_control flag controls UI delete affordance and drives clearReplicatedRows on demote/reanchor - Validators reused: validateStackPatternForRedos for glob safety, sanitizeForLog for log fragments SARIF export emits an external/accepted suppression entry per acknowledged misconfig, matching the CVE pattern. Per-row Acknowledge dialog prefills stack_pattern with the scan's stack_context so the default scope is "rule + this stack only" and an operator must broaden explicitly. Tests: misconfig-ack-filter (15) and misconfig-ack-routes (23) including the duplicate-409 case for both pinned and fleet-wide acks. * fix(security): reap orphaned trivy tmp dirs at startup When the buildEnv path writes a per-scan DOCKER_CONFIG dir under os.tmpdir() and the process crashes before the finally block runs, the dir leaks. Mirrors GitSourceService.sweepStaleTempDirs: exported sweepStaleTrivyTempDirs is fire-and-forget at boot, removes prefix-matching dirs older than 1 hour, swallows permission/race failures, logs a single line if any were reaped. * perf(security): emit per-batch summary for scanAllNodeImages Adds one diag() line at the end of scanAllNodeImages summarising unique image count, scanned, skipped, failed, violation count, and elapsed time. Per-image diag inside scanImage stays useful for debugging individual scans; the summary gives operators a single fleet-level checkpoint when developer_mode is on. * perf(security): cap SARIF export at 5000 findings per type Replace the unbounded fetchAllPages walk on /scans/:id/sarif with a hard limit of 5000 findings per type. When any type trips the cap, emit run-level properties.truncated=true plus row_limit and per-type totals so downstream tooling can flag the export as partial. Console-warns for ops visibility. A scan with 50k vulns previously streamed every row into memory before serialising; the cap bounds memory and serialisation time at the cost of completeness on pathological scans. * docs(env): document TRIVY_BIN host-binary override The env var is honored by TrivyService.detectTrivy as a fallback when no managed install is present, but it was undocumented in .env.example. Adds the var with a comment explaining precedence (managed > TRIVY_BIN > PATH). * test(security): cover scanComposeStack failure modes Two new cases drive the existing try/catch through real failure paths: - Malformed Trivy stdout: row flips to status='failed' with the parser error preserved on `error`. - execFile rejection: row flips to status='failed' with a string error message. Pairs with the existing dedup tests so the failure path now also verifies the scan row state, not just the thrown exception. * test(e2e): security scanner + misconfig acknowledgement flow Seven Playwright tests covering the scanner UI and the new acknowledgement system end-to-end: - Trivy availability gate (skips suite when binary absent so CI without Trivy can opt out via E2E_SKIP_TRIVY=1) - Stack config scan completes and records misconfig findings - Concurrent stack scan returns 409 from the dedup gate - Misconfig ack POST creates and lists on Settings - Duplicate (rule_id, stack_pattern) returns 409 - Malformed rule_id (shell metacharacters) returns 400 - Misconfigs tab renders against a real stack scan Tests drive the API for behaviour assertions and the UI only for shell-rendering checks; the visual snapshot suite owns screenshots. * docs(features): add misconfig acknowledgement workflow and SARIF cap Refreshes vulnerability-scanning.mdx with: - Misconfig acknowledgements section covering the per-row dialog, Settings panel, scope/matching rules, and SARIF emission - Tier table row for the new feature - SARIF section note on the 5000 row-per-type cap and the properties.truncated marker for partial exports - Troubleshooting entries: SARIF cap, hidden Acknowledge button, findings resurfacing after delete, Trivy DB phone-home, and 409 on concurrent compose-stack scans * fix(ci): clear backend lint and CodeQL alerts - Remove the dead fetchAllPages helper in routes/security.ts. It lost its callers when the SARIF endpoint switched to direct paged reads for the truncation cap. ESLint flagged it as unused. - Switch the trivy-tmp-cleanup test helper to fs.mkdtempSync. Building paths under os.tmpdir() with predictable names tripped CodeQL's js/insecure-temporary-file rule (high severity), which warns about symlink-pre-creation attacks even in test code. mkdtempSync appends a process-random suffix and creates the dir atomically; the sencho-trivy- prefix is preserved so the production sweep still matches the test fixtures.	2026-05-07 19:23:11 -04:00
Anso	33b15d6cba	feat(fleet-sync): retry failed pushes and backfill on add-node (#970) ... A control instance now retries fleet-sync pushes that hit a transient failure and backfills the security state on a freshly registered remote without waiting for the next policy edit. New service: - FleetSyncRetryService (singleton, start/stop) wakes 30s after boot and ticks every 5min. For each fleet resource, queries getFailedSyncTargets within a 24h window and re-pushes via FleetSyncService.pushResourceToNode through the same per-node mutex, so a normal fanout in flight serializes naturally with a retry. - After STALE_THRESHOLD_MS (1h) of continuous failure for a previously-working node, dispatches a single warning notification per cooldown window. Brand-new nodes that have never succeeded do not alert via this path; misconfigured remotes are caught by the test-connection affordance at registration time. - Wired into bootstrap startup/shutdown next to AutoHealService. Public surface: - FleetSyncService.pushResourceToNode(node, resource): targeted push to one node that re-uses the per-node mutex. Used by the retry service and any future targeted-resync flow. - routes/nodes.ts POST /api/nodes fires pushResourceAsync for both resources after a remote-proxy node row commits. Tuning constants centralized in fleetSyncConstants.ts: - RETRY_MAX_AGE_MS = 24h - STALE_THRESHOLD_MS = 1h Tests: - 8 vitest cases covering replica skip, retry dispatch, missing-node skip, alert-once-per-cooldown across the threshold window, no-alert for recent failures, no-alert for brand-new never-succeeded nodes, no-alert when the retry itself succeeds, start/stop idempotency. - Full backend suite: 1781 pass / 5 skipped.	2026-05-07 13:16:24 -04:00
Anso	e5b1c7b22b	refactor(backend): collapse entitlement provider abstraction back to LicenseService (#889) ... Removes backend/src/entitlements/ (registry, loadProvider, CommunityEntitlementProvider, types, headers, normalize) and the two abstraction-only tests. Relocates headers/normalize/types to services/license-*.ts. Swaps 22 consumer call sites from getEntitlementProvider() to LicenseService.getInstance(). Drops the Dockerfile install step plus PRO_PACKAGE_VERSION build-arg and github_token BuildKit secret in docker-publish.yml. Removes the now stale no-restricted-imports rule in backend/eslint.config.mjs. Net: 37 files changed, ~700 lines removed, no behavior change. Local dev no longer requires GitHub Packages auth to start the backend. Rationale and revisit conditions in docs/internal/adrs/2026-05-02-collapse-entitlement-provider.md.	2026-05-02 23:45:44 -04:00
Anso	3324616e59	refactor(backend): extract EntitlementProvider abstraction (Phase 1) (#878) ... * refactor(backend): extract EntitlementProvider abstraction (Phase 1) Phase 1 of the open-core hybrid extraction described in docs/internal/adrs/2026-05-02-open-core-hybrid-strategy.md. Introduces the abstraction without moving any code out of the public repo; Phase 2 will actually move services/LicenseService.ts to a private @studio-saelix/sencho-pro package. The new backend/src/entitlements/ module contains: - types.ts. The EntitlementProvider interface plus all tier/license types (LicenseTier, LicenseVariant, LicenseInfo, SeatLimits, ActivationResult, etc.). The interface mirrors the existing LicenseService public surface so the migration was mechanical. - registry.ts. Module-scope holder for the active provider with setEntitlementProvider, getEntitlementProvider, and a test-only reset helper. getEntitlementProvider throws if called before bootstrap registers a provider; the throw is intentional fail-fast on a bootstrap-order bug rather than a silent degradation. - CommunityEntitlementProvider.ts. Phase 2 fallback that returns community tier and rejects activate(). NOT instantiated in production today; a smoke test keeps it covered against bitrot. - loadProvider.ts. Async resolver. Phase 1 returns LicenseService.getInstance() directly. The async signature matches what Phase 2 needs (dynamic import of @studio-saelix/sencho-pro with a "module not found" vs "construction threw" narrowing); the call site does not change between phases. - headers.ts. PROXY_TIER_HEADER and PROXY_VARIANT_HEADER constants. These are part of the wire contract between Sencho instances and belong in the public core regardless of which entitlement provider is bound. - normalize.ts. isLicenseTier, isLicenseVariant, normalizeTier, normalizeVariant. Domain knowledge about Sencho's tier model (legacy name maps from pre-0.38.1 versions), not LemonSqueezy internals. Phase 2 keeps these in the public core. services/LicenseService.ts now imports its types from entitlements/types and adds an "implements EntitlementProvider" clause. Re-exports the types for back-compat with ~20 type-only consumers; a follow-up PR will sweep those imports to entitlements/ types directly before Phase 2 deletes the file. bootstrap/startup.ts awaits loadEntitlementProvider, registers the result, then calls initialize. shutdown.ts calls getEntitlementProvider().destroy() instead of the LicenseService singleton. middleware/tierGates.ts, the chokepoint for ~154 tier-check call sites, now reads through getEntitlementProvider. Sixteen other production files (routes/{fleet,imageUpdates,license,permissions, scheduledTasks,security,stacks,templates,users,webhooks}, services/{BlueprintService,CloudBackupService,SchedulerService, SSOService}, proxy/remoteNodeProxy, websocket/{hostConsole, remoteForwarder}, middleware/auth) had their LicenseService.getInstance calls and utility-export imports redirected to the entitlements module. The only remaining LicenseService.getInstance in production code is in entitlements/loadProvider.ts itself, which is the intentional Phase-1 binding site. Test infrastructure: setupTestDb registers LicenseService.getInstance() as the active provider so existing test files using the helper need no changes. The mocking pattern many tests use, vi.spyOn(LicenseService.getInstance(), 'getTier'), keeps working because LicenseService.getInstance() and getEntitlementProvider() return the same singleton in Phase 1. scheduler-service.test.ts is the only test that does not use setupTestDb but exercises tier-gating; it now mocks entitlements/registry alongside its existing LicenseService mock. Adds a smoke test for CommunityEntitlementProvider so the Phase 2 fallback class stays covered. Adds an architecture doc at docs/internal/architecture/entitlement-provider.md covering the runtime registry, bootstrap order invariants, and the Phase 1 vs Phase 2 binding table. Test results: 89/89 backend test files pass, 1657 passing tests, 5 pre-existing skips. The pre-existing database-metrics > handles 1000+ metrics stress test continues to flake under parallel load and pass when re-run solo, same flake observed in PRs #862, #863. * chore(backend): drop unused entitlement type imports from LicenseService Phase 1 of the EntitlementProvider extraction left five type imports (ActivationResult, BillingPortalError, BillingPortalResult, DeactivationResult, ValidationResult) unreferenced after the runtime methods that produced them began inferring their result shapes via the EntitlementProvider interface contract. ESLint's no-unused-vars rule flagged them as errors and failed the lint step in CI.	2026-05-02 05:07:00 -04:00
Anso	685d5d729e	feat(blueprints): backend foundation for fleet-wide compose templates (#860) ... * feat(blueprints): add backend foundation for fleet-wide compose templates Introduces the Blueprint Model: a docker-compose.yml plus a node selector (labels or explicit IDs) that Sencho reconciles across the fleet. Backend foundation only; the frontend tab and documentation follow. Schema (DatabaseService): - node_labels table for fleet-level orchestration tagging - blueprints table with compose content, selector, drift_mode, classification - blueprint_deployments table for per-node materialized state - New idempotent migrate methods following the existing pattern Services: - BlueprintAnalyzer: pure compose-YAML classifier (stateless / stateful / unknown) with 17 covered cases including named volumes, bind mounts, external volumes, and tmpfs - NodeLabelService: label CRUD plus selector matching helper (any/all/ids) - BlueprintService: local + remote deploy/withdraw orchestration, marker file management, name-conflict guard, per-(blueprint,node) lock - BlueprintReconciler: 60-second loop with three-mode drift policy (observe/suggest/enforce), state-aware guards, and Enforce-downgrade for volume-destroying drift Routes (gated requirePaid + requireAdmin on mutations): - /api/blueprints (CRUD + apply + withdraw + accept + preview + analyze) - /api/node-labels (CRUD + listAll + listDistinct) Notifications: four new categories registered in NotificationService for deploy/failure/drift events. Bootstrap: reconciler start/stop wired in startup and shutdown. Tests: 45 new Vitest cases covering selector matching, classifier rules, state-aware guards, drift-mode branching, and marker parsing. Full backend suite (1625 tests) passes; tsc clean. * fix(lint): replace bare Function type in blueprint reconciler tests Replace 8 occurrences of `as unknown as { computeDecision: Function }` with a properly typed `ReconcilerWithCompute` alias that mirrors the real method signature. Export `ReconcileDecision` from BlueprintReconciler so the test can reference it. Resolves @typescript-eslint/no-unsafe-function-type errors that were failing the Backend (Lint) CI step.	2026-05-01 18:57:44 -04:00
Anso	7663f4cd8b	feat(fleet): sencho mesh in traffic and routing tab (#858) ... * feat(fleet): sencho mesh in traffic and routing tab Lights up Sencho Mesh: cross-node container forwarding rendered as if the container next to you were on localhost. Builds on the dormant TCP frame plumbing from the prior PR (pilot tunnel TCP frames + sencho-mesh sidecar package) and exposes the Admiral-only orchestrator surface. Backend - New mesh_stacks table (per-node opt-ins) + nodes.mesh_enabled column via DatabaseService.migrateMeshTables. - MeshService singleton: sidecar lifecycle via Dockerode, opt-in/out with cascading override regeneration, request-based resolver from sidecar control WS, cross-node TCP forwarding via PilotTunnelManager (same-node fast path included), in-memory 1000-event activity ring buffer with durable mirror to audit_log for state-change events, per-node and per-route diagnostics, and the Test upstream probe. - MeshComposeOverride: pure YAML generator that injects extra_hosts using host-gateway. The user's docker-compose.yml is never mutated; overrides live under DATA_DIR/mesh/overrides. - ComposeService deploy/update splice the override file when the stack is opted in; non-mesh stacks behave identically to today. - Pilot agent resolveMeshTarget consults the local mesh_stacks table (defense in depth) and resolves Compose containers via Dockerode. - /api/mesh router with 13 Admiral-gated endpoints covering status, enable/disable, stack opt-in/out, alias listing, per-route diagnostic, Test upstream probe, per-node diagnostic, sidecar restart, activity log paginated and SSE. - meshControl WS slot at /api/mesh/control validates the mesh_sidecar JWT minted by MeshService; dispatched as upgrade slot 2 (canonical order preserved). Frontend - New Traffic Routing tab in FleetView, gated by isAdmiral and wrapped in AdmiralGate. Tab uses the cyan brand glyph and italic-serif state typography from the audit. - RoutingTab masthead with mesh activity drawer, per-node card grid with TogglePill, alias rows with five-state pill taxonomy (healthy / degraded / unreachable / tunnel-down / not-authorized), inline Test buttons. - Four sheets: opt-in picker with port-collision inline error, per-route detail with diagnostic + filtered activity, per-node diagnostics with active streams + resolver cache + restart action, fleet-wide activity log with filters. - meshRouteState helper centralizes pill-state mapping; pure-function tests cover all five states. Docs - User docs at /docs/features/sencho-mesh.mdx covering opt-in, troubleshooting, security model (4 guarantees + 4 explicit non-guarantees), and V1 limitations. - Internal architecture and runbook pages. - websocket-dispatch internal doc updated with the new slot. * fix(mesh): validate stack name before path use; fix test DB lifecycle Two surgical fixes against the prior PR. Path-injection (CodeQL js/path-injection): MeshService.optInStack, optOutStack, ensureStackOverride, and removeStackOverride now validate stackName via isValidStackName from utils/validation, reject malicious names at the API boundary, and additionally check isPathWithinBase on the resolved override file path for defense in depth. The dataflow from req.params.stackName to fs.writeFile no longer reaches an unsanitized path expression. Test DB lifecycle: mesh-service.test.ts used per-test setupTestDb / cleanupTestDb, which deletes the temp dir while DatabaseService still holds an open SQLite handle. On Linux CI this raises SQLITE_READONLY_DBMOVED on the next prepare() because the inode has been unlinked. Switched to file-scoped beforeAll/afterAll matching agents-routes.test.ts, with a per-test beforeEach that truncates mesh_stacks plus non-default nodes and resets the MeshService singleton in-memory state. Adds a new test case asserting the path-traversal rejection. * fix(compose): use discovered compose filename instead of hardcoded docker-compose.yml composeArgs() hardcoded `-f docker-compose.yml` for every deploy. Sencho writes its canonical compose file as `compose.yaml`, so any stack created via the UI failed to deploy with `open ...docker-compose.yml: no such file or directory`. When no mesh override applies, drop the explicit `-f` so docker compose's built-in discovery resolves the actual filename. When an override exists, look up the real base filename via FileSystemService.getComposeFilename() and pass both files explicitly. Also hoist the MeshService import to module top now that the dependency is known to be acyclic, and revert the matching unit-test assertion.	2026-05-01 01:50:53 -04:00
Anso	5cf4323511	perf(backend): batch audit_log inserts into a buffered transaction (#817) ... Every mutating /api/* request runs an individual INSERT into audit_log which serializes against other writers under burst load (SQLite's single-writer model). Buffer the writes in DatabaseService and flush them in a single transaction either every second or once the buffer reaches 100 entries, whichever comes first. Read paths (getAuditLogs, getAuditLogsInRange, cleanupOldAuditLogs) drain the buffer first so callers always see a consistent view, which keeps the existing test pattern of insert-then-read working. Graceful shutdown flushes before db.close() so no entries are lost on clean exit. The 1s flush timer is unref'd so the buffer cannot keep the process alive on its own. The CLI resetMfa script flushes explicitly before returning since it exits before the timer fires.	2026-04-28 00:59:04 -04:00
Anso	18cf2e65e8	perf(backend): parallelize independent startup initializers (#816) ... The boot path awaited SelfUpdateService.initialize, DockerEventManager.start, and TrivyService.initialize one at a time even though none of them depend on each other. Group them into a single Promise.all so total cold-start time is the slowest one rather than the sum. Also convert the inner `docker compose version` probe in SelfUpdateService.initialize from execFileSync to execFileAsync. Without that, the synchronous spawn would block the event loop for up to 5 seconds and silently serialize the other two members of the parallel block, defeating the parallelization win for in-container deployments. The synchronous service starts (Monitor, AutoHeal, ImageUpdate, Scheduler, Mfa) are grouped together up front. They schedule timers whose first ticks fire 5+ seconds out, so they safely run alongside the awaited block.	2026-04-28 00:33:17 -04:00
Anso	e9fce15010	refactor(backend): extract bootstrap into startup/shutdown modules (phase 5) (#745) ... Move the startup and shutdown lifecycles out of index.ts: - bootstrap/startup.ts exports startServer(server) - migration check, service initialization, background watchdogs, HTTP listen, pilot-agent loopback bind. - bootstrap/shutdown.ts exports installShutdownHandlers(server) - SIGTERM/SIGINT handlers, in-order service stop chain, 10s force-exit guard, SQLite close. Restructure MfaService to add an instance + lifecycle so the replay purge timer no longer lives as a module-scope setInterval in index.ts. MfaService keeps all existing static methods (generateSecret, verifyTotp, currentWindow, generateBackupCodes, hashBackupCodes, verifyBackupCode, formatBackupCodeForDisplay, normalizeBackupCode, buildOtpauthUri) so every existing caller stays unchanged. The new start() / stop() pair is idempotent and calls .unref() so test shutdown is not blocked. bootstrap/startup calls MfaService.getInstance().start(). bootstrap/shutdown calls MfaService.getInstance().stop(). index.ts drops from 305 to 147 lines and now contains only the Express app composition: createApp, route mounts, remote proxy, createServer, attachUpgrade, static/SPA fallback, errorHandler, installShutdownHandlers, and the require.main guard that boots the server when run directly. Behavior is byte-for-byte identical: shutdown service order, log strings, force-exit timer, pilot-agent loopback logic, and the MFA purge cadence and debug logging all preserved verbatim.	2026-04-23 23:44:00 -04:00