3 Commits

Author	SHA1	Message	Date
Anso	f599110386	feat(mesh): collapse sidecar into Sencho process via in-process forwarder (#1000) ... The separate saelix/sencho-mesh sidecar container is gone. The forwarder logic that previously lived in mesh-sidecar/src/forwarder.ts moves into the Sencho process as backend/src/services/MeshForwarder.ts, a thin per-port net.Server lifecycle wrapper. MeshService implements the host interface and owns resolve plus splice; MeshForwarder owns listener boilerplate. One container per node, no separate image to publish, no control WebSocket. Operator-facing change: the Sencho container now runs in network_mode: host so the forwarder can bind alias ports on the host network where meshed containers' extra_hosts host-gateway entries point. Without host network mode the listeners would land in the container's namespace and inbound traffic from peers would never reach them. The 1852:1852 port publish becomes a no-op under host mode and is commented out in the operator template. Same-node forward path now dials the target container's bridge IP via Dockerode (preferring the compose default network for deterministic selection across daemon versions) instead of 127.0.0.1. The legacy 127.0.0.1 path only worked when the target service published its port to the host; the IP path works regardless. Cross-node mesh routing in this phase is central -> pilot direction only via PilotTunnelManager.openTcpStream. Pilot -> central and pilot <-> pilot via central relay land in Phase B with the tcp_open_reverse frame. Deletions: - mesh-sidecar/ package entirely (Dockerfile, package, sources, tests) - backend/src/websocket/meshControl.ts - MeshService sidecar lifecycle: spawnSidecar, stopSidecar, isSidecarRunning, mintSidecarToken, verifySidecarToken, attachSidecarSocket, handleSidecarResolve, sendSidecar - POST /api/mesh/nodes/:id/sidecar/restart route - /api/mesh/control WS dispatch in upgradeHandler Type cleanup: 'sidecar' literal removed from MeshActivitySource and MeshProbeResult.where (also the frontend mirror). MeshNodeStatus sidecarRunning becomes localForwarderListening (boolean | null) so non-local nodes get a null instead of an unconditional false; the honest semantic is "this view only knows the local forwarder state; remote forwarder status lands in Phase B." MeshNodeDiagnostic sidecar object becomes forwarder { listening, listenerCount }. Frontend MeshDiagnosticsSheet drops the restart-sidecar action and sidecar liveness card; surfaces forwarder state plus a "runs in-process; no separate container" caption. Resolves audit findings C-1 (data plane non-functional), C-2 (sidecar control WS not loopback-enforced), C-4 (sidecar lifecycle Dockerode- on-remote, PR #999 closed), and C-5 (saelix/sencho-mesh:latest unreachable). C-3 (PR #992) is unchanged. M-12 (PR #994) is unchanged.	2026-05-08 15:51:39 -04:00
Anso	7663f4cd8b	feat(fleet): sencho mesh in traffic and routing tab (#858) ... * feat(fleet): sencho mesh in traffic and routing tab Lights up Sencho Mesh: cross-node container forwarding rendered as if the container next to you were on localhost. Builds on the dormant TCP frame plumbing from the prior PR (pilot tunnel TCP frames + sencho-mesh sidecar package) and exposes the Admiral-only orchestrator surface. Backend - New mesh_stacks table (per-node opt-ins) + nodes.mesh_enabled column via DatabaseService.migrateMeshTables. - MeshService singleton: sidecar lifecycle via Dockerode, opt-in/out with cascading override regeneration, request-based resolver from sidecar control WS, cross-node TCP forwarding via PilotTunnelManager (same-node fast path included), in-memory 1000-event activity ring buffer with durable mirror to audit_log for state-change events, per-node and per-route diagnostics, and the Test upstream probe. - MeshComposeOverride: pure YAML generator that injects extra_hosts using host-gateway. The user's docker-compose.yml is never mutated; overrides live under DATA_DIR/mesh/overrides. - ComposeService deploy/update splice the override file when the stack is opted in; non-mesh stacks behave identically to today. - Pilot agent resolveMeshTarget consults the local mesh_stacks table (defense in depth) and resolves Compose containers via Dockerode. - /api/mesh router with 13 Admiral-gated endpoints covering status, enable/disable, stack opt-in/out, alias listing, per-route diagnostic, Test upstream probe, per-node diagnostic, sidecar restart, activity log paginated and SSE. - meshControl WS slot at /api/mesh/control validates the mesh_sidecar JWT minted by MeshService; dispatched as upgrade slot 2 (canonical order preserved). Frontend - New Traffic Routing tab in FleetView, gated by isAdmiral and wrapped in AdmiralGate. Tab uses the cyan brand glyph and italic-serif state typography from the audit. - RoutingTab masthead with mesh activity drawer, per-node card grid with TogglePill, alias rows with five-state pill taxonomy (healthy / degraded / unreachable / tunnel-down / not-authorized), inline Test buttons. - Four sheets: opt-in picker with port-collision inline error, per-route detail with diagnostic + filtered activity, per-node diagnostics with active streams + resolver cache + restart action, fleet-wide activity log with filters. - meshRouteState helper centralizes pill-state mapping; pure-function tests cover all five states. Docs - User docs at /docs/features/sencho-mesh.mdx covering opt-in, troubleshooting, security model (4 guarantees + 4 explicit non-guarantees), and V1 limitations. - Internal architecture and runbook pages. - websocket-dispatch internal doc updated with the new slot. * fix(mesh): validate stack name before path use; fix test DB lifecycle Two surgical fixes against the prior PR. Path-injection (CodeQL js/path-injection): MeshService.optInStack, optOutStack, ensureStackOverride, and removeStackOverride now validate stackName via isValidStackName from utils/validation, reject malicious names at the API boundary, and additionally check isPathWithinBase on the resolved override file path for defense in depth. The dataflow from req.params.stackName to fs.writeFile no longer reaches an unsanitized path expression. Test DB lifecycle: mesh-service.test.ts used per-test setupTestDb / cleanupTestDb, which deletes the temp dir while DatabaseService still holds an open SQLite handle. On Linux CI this raises SQLITE_READONLY_DBMOVED on the next prepare() because the inode has been unlinked. Switched to file-scoped beforeAll/afterAll matching agents-routes.test.ts, with a per-test beforeEach that truncates mesh_stacks plus non-default nodes and resets the MeshService singleton in-memory state. Adds a new test case asserting the path-traversal rejection. * fix(compose): use discovered compose filename instead of hardcoded docker-compose.yml composeArgs() hardcoded `-f docker-compose.yml` for every deploy. Sencho writes its canonical compose file as `compose.yaml`, so any stack created via the UI failed to deploy with `open ...docker-compose.yml: no such file or directory`. When no mesh override applies, drop the explicit `-f` so docker compose's built-in discovery resolves the actual filename. When an override exists, look up the real base filename via FileSystemService.getComposeFilename() and pass both files explicitly. Also hoist the MeshService import to module top now that the dependency is known to be acyclic, and revert the matching unit-test assertion.	2026-05-01 01:50:53 -04:00
Anso	dc3699189d	refactor(backend): extract remote proxy, WebSocket upgrade handler, and server factory (phase 3) (#733) ... Phase 3 of the index.ts refactor. Pulls the remote HTTP/WS proxy plumbing, the WebSocket upgrade dispatcher, and the http/WSS construction out of the monolith. index.ts drops roughly 620 lines. New modules: - proxy/websocketProxy.ts: shared httpProxy.createProxyServer singleton (used by both the HTTP proxy middleware and the remote WS forwarder) - proxy/remoteNodeProxy.ts: createRemoteProxyMiddleware() factory; consumes the isProxyExemptPath helper instead of open-coding the prefix list - server.ts: createServer(app) returns { server, wss, pilotTunnelWss } - services/FleetUpdateTrackerService.ts: singleton wrapping the in-flight fleet update tracker Map with create()/resolve() helpers - helpers/consoleSession.ts: mintConsoleSession(), isConsoleSessionScope() - websocket/upgradeHandler.ts: attachUpgrade(server, deps) dispatcher that runs the manual cookie/JWT verify and delegates to sub-handlers - websocket/pilotTunnel.ts: handlePilotTunnel (pilot_enroll consumption and pilot_tunnel registration) - websocket/notifications.ts: /ws/notifications local subscriber - websocket/remoteForwarder.ts: remote-node WS proxy with console_session token exchange for interactive paths - websocket/logs.ts: /api/stacks/:name/logs supervisor stream - websocket/hostConsole.ts: /api/system/host-console PTY, Admiral-gated - websocket/generic.ts: /ws exec + streamStats action dispatch, owns the terminalWs single-instance reference - websocket/reject.ts: shared rejectUpgrade helper (replaces five copies) Service extension: - NotificationService: setBroadcaster(fn) replaced by subscribe(ws) that returns an unsubscriber; broadcastToSubscribers is now internal. Subscriber set lives on the service rather than in index.ts. Wiring in index.ts: - const app = createApp() already in place from Phase 2 - const { server, wss, pilotTunnelWss } = createServer(app) - attachUpgrade(server, { wss, pilotTunnelWss }) - app.use('/api/', createRemoteProxyMiddleware()) - /api/system/console-token route now uses mintConsoleSession() - deploy/down/update routes read the streaming target via getTerminalWs() (return type is WebSocket | undefined so the || undefined fallback is gone) Code review fixes: five duplicated reject helpers collapsed into websocket/reject.ts; dropped the createTracker/resolveTracker bind aliases in index.ts so call sites go through the service directly; removed em dashes; replaced req.url! with req.url || '/'.	2026-04-23 19:31:16 -04:00