3 Commits

Author	SHA1	Message	Date
Anso	982c7830b1	fix(mesh): address audit follow-ups (early-data, recompose, admiral gate, error codes, docs) (#1126) ... * fix(mesh): buffer early TcpData on reverse-relay path The reverse-relay code dropped the local-shaped reservation and any TcpData frames buffered in it before acceptReverseRelay had wired up the target stream. A peer that sent a request body immediately after tcp_open_reverse lost those bytes when the target lived on a third node, since reverse_local buffers them but reverse_relay did not. Carry the reservation through acceptReverseRelay: transplant its pendingData and pendingBytes into the new reverse_relay state, gate TcpData writes on targetOpen, and flush buffered frames into the target stream before sending tcp_open_ack. Mirrors the reverse_local pattern. Regression test fires tcp_open_reverse plus an immediate TcpData while ensureBridge is in flight and verifies the bytes arrive intact, in order, before the ack. * fix(mesh): recompose affected stacks when node-level mesh is disabled disableForNode used to clear DB rows and override files but leave the running containers attached to sencho_mesh with stale /etc/hosts alias entries until an operator redeployed every stack by hand. Mirror optOutStack: after the existing alias/forwarder cleanup, call regenerateOverridesAcrossFleet, cascadeRecomposeAcrossFleet, and triggerRedeploy for each previously meshed stack on the disabled node so containers detach from sencho_mesh and shed the alias entries they owned. The disabled node's mesh_stacks rows are deleted before the cascade so listMeshStacks returns the right set with no skip tuple required. Route threads the actor through actorFor(req) for parity with optInStack/optOutStack. Tests cover the redeploy fan-out, the cascade no-skip-tuple invariant, and the default actor fallback for non-route callers. * fix(mesh): require Admiral on the WS proxy-tunnel upgrade HTTP mesh routes in routes/mesh.ts all enforce requireAdmiral, but the /api/mesh/proxy-tunnel WS upgrade accepted any node_proxy or full-admin api_token regardless of the receiver's license. A node downgraded from Admiral kept serving mesh data-plane traffic to a sibling central while refusing every mesh management call. Read the receiver's local LicenseService at the upgrade and 403 when the tier is not paid+admiral. The check sits after the existing credential gate and uses LicenseService directly rather than effectiveTier (which trusts forwarded proxy headers); a remote peer dialing in cannot be trusted to assert our entitlement. Dialer and node_proxy token format are unchanged. Three regression tests cover community-tier node_proxy, skipper-tier node_proxy, and community-tier full-admin api_token all rejected with 403. * fix(mesh): handle no_target alongside push_failed in inspectStackServices proxyFetch throws MeshError('no_target') when getProxyTarget returns null (pilot tunnel offline, proxy bridge unreachable), but the inspectStackServices catch branch only matched push_failed. Offline remotes fell through to the generic 'remote unreachable' error log, which the Routing tab surfaces as an unexpected fault. Match both error codes and emit the operator-friendly warn message that names the unreachable node and the error code. Regression test spies on console.warn/console.error to pin the branch. * docs(mesh): align env defaults and forwarder comments with current architecture SENCHO_MESH_PROXY_TUNNEL_IDLE_MS in .env.example carried the old five-minute idle-close value (=300000), but the code default is DEFAULT_IDLE_TTL_MS=0 (persistent tunnel). Copying the example silently reintroduced the idle-close behavior the dialer removed. MeshForwarder.ts's leading docblock and inline listen comment still described host-network mode plus extra_hosts: host-gateway as required for forwarder reachability. Sencho runs in standard bridge mode and attaches to the shared sencho_mesh network at a stable IP; meshed user containers reach the forwarder by that IP directly. Flip the env default to 0, rewrite the env comment to describe the persistent behavior and the opt-in for idle teardown, and rewrite both forwarder comments to match the bridge-network reality. * fix(mesh): trust forwarded tier on proxy-tunnel WS and remove remote overrides on disable Admiral entitlement on the WS data plane now follows the same trust model as the HTTP mesh routes: the central asserts its tier via x-sencho-tier and x-sencho-variant on the WS handshake, and the receiver trusts those headers only when the upgrade carries a node_proxy credential. When no headers are present or the credential is a full-admin api_token, the receiver falls back to its own local license. Without this an Admiral central could be rejected by a Community remote and a Community central could dial a locally-Admiral remote. disableForNode now routes through removeOverrideFromNode so override files pushed earlier via applyLocalOverride are removed on remote nodes via DELETE /api/mesh/local-override/:stack. Sequential awaits are wrapped in Promise.allSettled to match regenerateOverridesForNode's parallel push pattern. Other changes: - Cover the post-state-swap buffering window in the reverse-relay test (TcpData arriving after openTcpStream returns but before target open). - Refresh stale MeshService comments that still referenced the removed sidecar layer and host-network listener model. * test(mesh): pin removeOverrideFromNode remote HTTP shape The disableForNode regression test mocks removeOverrideFromNode itself, so a regression inside the helper would not be caught. Add a narrow contract test that spies on global fetch and asserts the request shape: DELETE /api/mesh/local-override/:stack against the resolved proxy target, with Authorization Bearer plus the x-sencho-tier and x-sencho-variant headers. Also covers the encodeURIComponent path and the swallowed-network-error behavior the disable cascade relies on. * test(mesh): use createTestApiToken helper in proxy-tunnel api_token gate test The full-admin api_token branch of the WS Admiral gate test inlined the canonical generateApiToken + sha256 + addApiToken triple that already lives in the createTestApiToken helper. Switching to the helper removes the duplicated insertion logic and aligns this test with the helper used by the other api_token call sites.	2026-05-20 12:53:29 -04:00
Anso	a318e6b3c1	fix(mesh): close data-plane race that dropped early TcpData frames (#1086) ... The tcp_open and tcp_open_reverse receivers registered their stream entry only after awaiting resolveTarget / resolveContainerIp. The peer, which is free to send TcpData immediately after the open frame, hit the lookup-miss path in handleBinaryFrame and the bytes were silently dropped. Probes passed because they only exercise the handshake; real HTTP hung with 0 bytes received. Reserve the stream entry synchronously, buffer early TcpData in a per-stream pendingData queue capped at 1 MiB, and flush onto the local socket inside the connect handler before sending tcp_open_ack. The payload is copied because decodeBinaryFrame returns a subarray view of the WS receive buffer; holding the view would pin the parent buffer past its lifecycle. Both sides of the data plane are patched: - tcpStreamSwitchboard.onTcpOpen (forward, agent + proxy-mode peer) - PilotTunnelBridge.handleTcpOpenReverse / acceptReverseLocal (reverse, primary acting as the local dial target) Adds unit coverage for the race window and for the overflow path. The cap constant lives in pilot/protocol.ts alongside the other per-stream limits.	2026-05-17 14:01:39 -04:00
Anso	567e524450	feat(mesh): bidirectional routing via tcp_open_reverse and central relay (#1003) ... * feat(mesh): bidirectional routing via tcp_open_reverse and central relay Phase B of the mesh redesign. Adds the reverse-direction protocol so a pilot's MeshForwarder can route cross-node traffic to central or to another pilot. Central relays pilot-to-pilot streams transparently; pilots keep their existing single outbound WS to central. Protocol additions (backend/src/pilot/protocol.ts): - TcpOpenReverseFrame { s, targetNodeId, stack, service, port } sent agent to primary. Reuses existing tcp_open_ack, tcp_close, and TcpData binary frames for the response and byte plane. - AGENT_REVERSE_ID_BASE = 0x40000001 splits the 32-bit id space so agent-allocated reverse stream ids never collide with primary- allocated forward ids on the same tunnel. - StreamIdAllocator now wraps to its configured start (not always 1) so an allocator parameterized with the agent base stays in the agent half across the wrap. Agent (backend/src/pilot/agent.ts): - New ReverseTcpStreamHandle exported class, EventEmitter facade matching PilotTunnelBridge.TcpStream's surface (write, end, destroy plus open, data, error, close events). - Public openMeshTcpStream(target) allocates a reverse id, sends tcp_open_reverse, returns the handle. - onTcpOpenAckReverse handles inbound ack, dispatches open or error+close to the matching handle. - TcpData and tcp_close binary/JSON paths route ids in the reverse range to reverseTcpStreams; existing primary-allocated paths unchanged. - streamCount, cleanupAfterDisconnect, onStreamIdle include reverse streams. The allocator is reset to a fresh base on disconnect so long-lived agents that reconnect many times do not drift up the id space. - startPilotAgent registers the agent as MeshService's reverse dialer via lazy import. Bridge (backend/src/services/PilotTunnelBridge.ts): - Two new StreamState kinds: reverse_local (target = central, dial Dockerode container IP) and reverse_relay (target = another pilot, open a forward TcpStream on the target pilot's bridge). - handleTcpOpenReverse validates the agent-id range and stream cap, then dispatches to acceptReverseLocal or acceptReverseRelay via lazy imports of MeshService, NodeRegistry, and PilotTunnelManager (avoids module cycles). - acceptReverseLocal calls MeshService.resolveContainerIp (now public), opens net.createConnection, splices bytes through the tunnel. Pre-connect error handler is removed inside connect to avoid double-firing with the mid-stream error path. - acceptReverseRelay opens a forward TcpStream on the target bridge, splices bytes between the two tunnels. - Existing tcp_close and TcpData handlers extended for the new state kinds. teardownStream extended. MeshService (backend/src/services/MeshService.ts): - resolveContainerIp made public so the bridge's local-target path can dial the same shape. - New reverseDialer field plus setReverseDialer setter. Pilot mode registers the agent at boot; central mode leaves it null. - New private dialMeshTcpStream dispatcher: pilot side uses the reverse dialer, central side uses PilotTunnelManager.getBridge. - openCrossNode refactored to call the dispatcher and use the active-stream record's id for activity logging. - New exported MeshTcpStreamLike interface that both PilotTunnelBridge.TcpStream and ReverseTcpStreamHandle satisfy structurally; ReverseMeshDialer is the contract for the setter. Tests (3 files, 16 new cases, 172 total pass): - pilot-protocol-tcp.test.ts: tcp_open_reverse round-trip; agent base invariant. - pilot-agent-reverse-stream.test.ts: openMeshTcpStream allocation, ws-not-open, frame shape, write encoding, end emits tcp_close, ack-success, ack-failure, low-id ack ignored, inbound TcpData routing. - pilot-bridge-reverse.test.ts: id-range validation, no-target failure, successful local dial against a real upstream server. Together with PR #1000 (Phase A) and PR #1001 (deps), Phase B completes the central-pilot-pilot mesh routing matrix. Phase C (mesh over proxy-mode remotes) is the planned follow-up. * test(pilot): drop unused encodeJsonFrame import Lint failed on pilot-agent-reverse-stream.test.ts after the test changed from constructing tcp_open_reverse frames inline to driving the agent's frame-dispatch path with synthetic objects. The import is no longer referenced; ESLint's no-unused-vars rule rejected it.	2026-05-08 16:21:34 -04:00