gsadmin/sencho
1
0
Fork 0
mirror of https://github.com/Studio-Saelix/sencho.git synced 2026-08-23 08:29:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(mesh): bidirectional routing via tcp_open_reverse and central relay (#1003)

Browse Source 
* feat(mesh): bidirectional routing via tcp_open_reverse and central relay

Phase B of the mesh redesign. Adds the reverse-direction protocol so a
pilot's MeshForwarder can route cross-node traffic to central or to
another pilot. Central relays pilot-to-pilot streams transparently;
pilots keep their existing single outbound WS to central.

Protocol additions (backend/src/pilot/protocol.ts):
- TcpOpenReverseFrame { s, targetNodeId, stack, service, port } sent
  agent to primary. Reuses existing tcp_open_ack, tcp_close, and
  TcpData binary frames for the response and byte plane.
- AGENT_REVERSE_ID_BASE = 0x40000001 splits the 32-bit id space so
  agent-allocated reverse stream ids never collide with primary-
  allocated forward ids on the same tunnel.
- StreamIdAllocator now wraps to its configured start (not always 1)
  so an allocator parameterized with the agent base stays in the
  agent half across the wrap.

Agent (backend/src/pilot/agent.ts):
- New ReverseTcpStreamHandle exported class, EventEmitter facade
  matching PilotTunnelBridge.TcpStream's surface (write, end,
  destroy plus open, data, error, close events).
- Public openMeshTcpStream(target) allocates a reverse id, sends
  tcp_open_reverse, returns the handle.
- onTcpOpenAckReverse handles inbound ack, dispatches open or
  error+close to the matching handle.
- TcpData and tcp_close binary/JSON paths route ids in the reverse
  range to reverseTcpStreams; existing primary-allocated paths
  unchanged.
- streamCount, cleanupAfterDisconnect, onStreamIdle include reverse
  streams. The allocator is reset to a fresh base on disconnect so
  long-lived agents that reconnect many times do not drift up the id
  space.
- startPilotAgent registers the agent as MeshService's reverse
  dialer via lazy import.

Bridge (backend/src/services/PilotTunnelBridge.ts):
- Two new StreamState kinds: reverse_local (target = central, dial
  Dockerode container IP) and reverse_relay (target = another pilot,
  open a forward TcpStream on the target pilot's bridge).
- handleTcpOpenReverse validates the agent-id range and stream cap,
  then dispatches to acceptReverseLocal or acceptReverseRelay via
  lazy imports of MeshService, NodeRegistry, and PilotTunnelManager
  (avoids module cycles).
- acceptReverseLocal calls MeshService.resolveContainerIp (now
  public), opens net.createConnection, splices bytes through the
  tunnel. Pre-connect error handler is removed inside connect to
  avoid double-firing with the mid-stream error path.
- acceptReverseRelay opens a forward TcpStream on the target
  bridge, splices bytes between the two tunnels.
- Existing tcp_close and TcpData handlers extended for the new
  state kinds. teardownStream extended.

MeshService (backend/src/services/MeshService.ts):
- resolveContainerIp made public so the bridge's local-target path
  can dial the same shape.
- New reverseDialer field plus setReverseDialer setter. Pilot mode
  registers the agent at boot; central mode leaves it null.
- New private dialMeshTcpStream dispatcher: pilot side uses the
  reverse dialer, central side uses PilotTunnelManager.getBridge.
- openCrossNode refactored to call the dispatcher and use the
  active-stream record's id for activity logging.
- New exported MeshTcpStreamLike interface that both
  PilotTunnelBridge.TcpStream and ReverseTcpStreamHandle satisfy
  structurally; ReverseMeshDialer is the contract for the setter.

Tests (3 files, 16 new cases, 172 total pass):
- pilot-protocol-tcp.test.ts: tcp_open_reverse round-trip; agent
  base invariant.
- pilot-agent-reverse-stream.test.ts: openMeshTcpStream allocation,
  ws-not-open, frame shape, write encoding, end emits tcp_close,
  ack-success, ack-failure, low-id ack ignored, inbound TcpData
  routing.
- pilot-bridge-reverse.test.ts: id-range validation, no-target
  failure, successful local dial against a real upstream server.

Together with PR #1000 (Phase A) and PR #1001 (deps), Phase B
completes the central-pilot-pilot mesh routing matrix. Phase C
(mesh over proxy-mode remotes) is the planned follow-up.

* test(pilot): drop unused encodeJsonFrame import

Lint failed on pilot-agent-reverse-stream.test.ts after the test
changed from constructing tcp_open_reverse frames inline to driving
the agent's frame-dispatch path with synthetic objects. The import
is no longer referenced; ESLint's no-unused-vars rule rejected it.
This commit is contained in:
Anso
2026-05-08 16:21:34 -04:00
committed by GitHub
parent f599110386
commit 567e524450
7 changed files with 884 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/src/services/MeshService.ts
+74 -14
View File
@@ -590,8 +590,14 @@ export class MeshService extends EventEmitter implements MeshForwarderHost {
src.on('close', () => teardown());
}
/** Find the bridge-network IP of the first container of `<stack>/<service>`. */
private async resolveContainerIp(target: MeshTarget): Promise<string | null> {
/**
* Find the bridge-network IP of the first container of
* `<stack>/<service>`. Public so the central-side
* `PilotTunnelBridge` reverse-open handler (Phase B) can dial the same
* target shape when a pilot's mesh forwarder routes traffic back to
* central.
*/
public async resolveContainerIp(target: { stack: string; service: string }): Promise<string | null> {
try {
const docker = DockerController.getInstance().getDocker();
// Compose default container name pattern; -1 is the first replica.
@@ -644,28 +650,54 @@ export class MeshService extends EventEmitter implements MeshForwarderHost {
return info.NetworkSettings?.IPAddress || null;
}
private openCrossNode(target: MeshTarget, src: net.Socket): void {
/**
* Pluggable reverse dialer. Set by `PilotAgent` on a pilot host; left
* null on a central host. When set, `openCrossNode` routes outbound
* mesh dials through the agent's `tcp_open_reverse` path; when unset,
* `openCrossNode` uses the central-side `PilotTunnelManager.getBridge`
* directly. Lets the same MeshService code work on both sides.
*/
private reverseDialer: ReverseMeshDialer | null = null;
public setReverseDialer(dialer: ReverseMeshDialer | null): void {
this.reverseDialer = dialer;
}
private dialMeshTcpStream(target: MeshTarget): MeshTcpStreamLike | null {
if (this.reverseDialer) {
return this.reverseDialer.openMeshTcpStream({
nodeId: target.nodeId,
stack: target.stack,
service: target.service,
port: target.port,
});
}
const ptm = PilotTunnelManager.getInstance();
if (!ptm.hasActiveTunnel(target.nodeId)) {
if (!ptm.hasActiveTunnel(target.nodeId)) return null;
const bridge = ptm.getBridge(target.nodeId);
if (!bridge) return null;
return bridge.openTcpStream({ stack: target.stack, service: target.service, port: target.port });
}
private openCrossNode(target: MeshTarget, src: net.Socket): void {
const tcpStream = this.dialMeshTcpStream(target);
if (!tcpStream) {
this.logActivity({
source: 'pilot', level: 'error', type: 'tunnel.fail',
nodeId: target.nodeId, alias: target.alias,
message: `no active pilot tunnel to node ${target.nodeId}`,
message: this.reverseDialer
? `cannot open reverse mesh stream to node ${target.nodeId}`
: `no active pilot tunnel to node ${target.nodeId}`,
});
try { src.destroy(); } catch { /* ignore */ }
return;
}
const bridge = ptm.getBridge(target.nodeId);
if (!bridge) { try { src.destroy(); } catch { /* ignore */ } return; }
const tcpStream = bridge.openTcpStream({ stack: target.stack, service: target.service, port: target.port });
if (!tcpStream) { try { src.destroy(); } catch { /* ignore */ } return; }
const record = this.registerActiveStream(target.alias, tcpStream.streamId);
const t0 = Date.now();
tcpStream.on('open', () => {
this.logActivity({
source: 'mesh', level: 'info', type: 'route.resolve.ok',
nodeId: target.nodeId, alias: target.alias, streamId: tcpStream.streamId,
nodeId: target.nodeId, alias: target.alias, streamId: record.streamId,
message: `cross-node connect to ${target.alias}`,
});
this.routeLatencyMap.set(target.alias, Date.now() - t0);
@@ -677,14 +709,14 @@ export class MeshService extends EventEmitter implements MeshForwarderHost {
tcpStream.on('error', (err: Error) => {
this.logActivity({
source: 'pilot', level: 'error', type: 'tunnel.fail',
nodeId: target.nodeId, alias: target.alias, streamId: tcpStream.streamId,
nodeId: target.nodeId, alias: target.alias, streamId: record.streamId,
message: err.message,
});
this.activeStreams.delete(tcpStream.streamId);
this.activeStreams.delete(record.streamId);
try { src.destroy(); } catch { /* ignore */ }
});
tcpStream.on('close', () => {
this.activeStreams.delete(tcpStream.streamId);
this.activeStreams.delete(record.streamId);
try { src.end(); } catch { /* ignore */ }
});
src.on('data', (chunk: Buffer) => {
@@ -912,3 +944,31 @@ export class MeshError extends Error {
this.code = code;
}
}
/**
* Common surface of an outbound mesh TCP stream as MeshService consumes
* it. Both the central-side `PilotTunnelBridge.TcpStream` and the
* pilot-side `ReverseTcpStreamHandle` (from `pilot/agent.ts`) implement
* this shape structurally so MeshService.openCrossNode can splice bytes
* against either without caring which side initiated the stream.
*/
export interface MeshTcpStreamLike {
readonly streamId: number;
write(chunk: Buffer): boolean;
end(): void;
destroy(): void;
on(event: 'open', listener: () => void): this;
on(event: 'data', listener: (chunk: Buffer) => void): this;
on(event: 'error', listener: (err: Error) => void): this;
on(event: 'close', listener: () => void): this;
}
/**
* Pilot-side reverse dialer. Set by `PilotAgent` when the agent boots
* (Phase B); leaves MeshService.openCrossNode able to route outbound
* cross-node mesh traffic over the agent's outbound `tcp_open_reverse`
* frame instead of central's `PilotTunnelManager.getBridge`.
*/
export interface ReverseMeshDialer {
openMeshTcpStream(target: { nodeId: number; stack: string; service: string; port: number }): MeshTcpStreamLike | null;
}
backend/src/services/PilotTunnelBridge.ts
+188 -10
View File
@@ -1,8 +1,9 @@
import http, { IncomingMessage, Server as HttpServer, ServerResponse } from 'http';
import { Socket } from 'net';
import net, { Socket } from 'net';
import { EventEmitter } from 'events';
import { WebSocket, WebSocketServer } from 'ws';
import {
AGENT_REVERSE_ID_BASE,
BinaryFrameType,
DecodedBinaryFrame,
MAX_STREAMS_PER_TUNNEL,
@@ -57,7 +58,34 @@ interface TcpStreamState extends StreamMeta {
accepted: boolean;
}
type StreamState = HttpStreamState | WsStreamState | TcpStreamState;
/**
* Pilot-initiated reverse stream where the agent's mesh forwarder asked the
* primary to dial a service ON the primary's local Docker host. The primary
* holds the resulting `net.Socket` and pumps bytes between the socket and
* the tunnel `TcpData` frames keyed on the agent-allocated `s`.
*/
interface ReverseLocalTcpStreamState extends StreamMeta {
kind: 'reverse_local';
socket: Socket;
bytesIn: number;
bytesOut: number;
}
/**
* Pilot-initiated reverse stream where the target is *another* pilot. The
* primary acts as a transparent relay: it allocates a forward `TcpStream`
* on the target pilot's bridge and splices bytes between this bridge's
* incoming `TcpData` frames (keyed on the source agent's `s`) and the
* target stream's `write` / `'data'`.
*/
interface ReverseRelayTcpStreamState extends StreamMeta {
kind: 'reverse_relay';
target: TcpStream;
bytesIn: number;
bytesOut: number;
}
type StreamState = HttpStreamState | WsStreamState | TcpStreamState | ReverseLocalTcpStreamState | ReverseRelayTcpStreamState;
/**
* Sencho Mesh TCP stream handle. EventEmitter-based duplex-like surface that
@@ -513,11 +541,23 @@ export class PilotTunnelBridge extends EventEmitter implements MeshTunnelHandle
}
break;
}
case 'tcp_open_reverse': {
this.handleTcpOpenReverse(frame);
break;
}
case 'tcp_close': {
const s = this.streams.get(frame.s);
if (!s || s.kind !== 'tcp') return;
this.removeStream(frame.s);
s.handle.emit('close');
if (!s) return;
if (s.kind === 'tcp') {
this.removeStream(frame.s);
s.handle.emit('close');
} else if (s.kind === 'reverse_local') {
this.removeStream(frame.s);
try { s.socket.destroy(); } catch { /* ignore */ }
} else if (s.kind === 'reverse_relay') {
this.removeStream(frame.s);
try { s.target.destroy(); } catch { /* ignore */ }
}
break;
}
default:
@@ -551,10 +591,19 @@ export class PilotTunnelBridge extends EventEmitter implements MeshTunnelHandle
// Agent never originates request bodies; ignore for defense-in-depth.
break;
case BinaryFrameType.TcpData: {
if (s.kind !== 'tcp') return;
s.bytesIn += frame.payload.length;
this.refreshIdleTimer(frame.streamId, s);
s.handle.emit('data', frame.payload);
if (s.kind === 'tcp') {
s.bytesIn += frame.payload.length;
this.refreshIdleTimer(frame.streamId, s);
s.handle.emit('data', frame.payload);
} else if (s.kind === 'reverse_local') {
s.bytesIn += frame.payload.length;
this.refreshIdleTimer(frame.streamId, s);
try { s.socket.write(frame.payload); } catch { /* ignore */ }
} else if (s.kind === 'reverse_relay') {
s.bytesIn += frame.payload.length;
this.refreshIdleTimer(frame.streamId, s);
s.target.write(frame.payload);
}
break;
}
default:
@@ -673,11 +722,140 @@ export class PilotTunnelBridge extends EventEmitter implements MeshTunnelHandle
} else if (state.rawSocket) {
try { state.rawSocket.destroy(); } catch { /* ignore */ }
}
} else {
} else if (state.kind === 'tcp') {
try {
if (!state.accepted) state.handle.emit('error', new Error('tunnel closed before accept'));
state.handle.emit('close');
} catch { /* ignore */ }
} else if (state.kind === 'reverse_local') {
try { state.socket.destroy(); } catch { /* ignore */ }
} else if (state.kind === 'reverse_relay') {
try { state.target.destroy(); } catch { /* ignore */ }
}
}
// ---- Phase B: pilot-initiated reverse mesh streams ----
/**
* Handle an inbound `tcp_open_reverse` from the agent. The agent's
* `MeshForwarder` accepted a connection destined for a node other than
* the agent's own and is asking the primary to dial the target. Two
* cases:
*
* - target is the primary's own node: dial a local container directly
* and splice bytes between the resulting socket and the tunnel.
* - target is another pilot: open a forward `TcpStream` on the target
* pilot's bridge and relay bytes between the two tunnels (primary
* in the middle).
*
* Stream id is allocated by the agent; the primary stores state keyed
* on the agent's id. Agent ids are required to be in the upper half of
* the 32-bit space so they cannot collide with primary-allocated ids
* for forward `tcp_open` streams on the same tunnel.
*/
private handleTcpOpenReverse(frame: { s: number; targetNodeId: number; stack: string; service: string; port: number }): void {
const { s, targetNodeId, stack, service, port } = frame;
if (s < AGENT_REVERSE_ID_BASE) {
this.sendJson({ t: 'tcp_open_ack', s, ok: false, err: 'agent_error' });
return;
}
if (this.streams.has(s) || this.streams.size >= MAX_STREAMS_PER_TUNNEL) {
this.sendJson({ t: 'tcp_open_ack', s, ok: false, err: 'agent_error' });
return;
}
// Lazy-import services that import this module to avoid a cycle.
void (async () => {
const { NodeRegistry } = await import('./NodeRegistry');
const localNodeId = NodeRegistry.getInstance().getDefaultNodeId();
if (targetNodeId === localNodeId) {
await this.acceptReverseLocal(s, { stack, service, port });
} else {
await this.acceptReverseRelay(s, targetNodeId, { stack, service, port });
}
})().catch((err) => {
if (isDebugEnabled()) console.warn('[PilotBridge:diag] reverse-open dispatch failed:', sanitizeForLog((err as Error).message));
this.sendJson({ t: 'tcp_open_ack', s, ok: false, err: 'agent_error' });
});
}
private async acceptReverseLocal(s: number, target: { stack: string; service: string; port: number }): Promise<void> {
const { MeshService } = await import('./MeshService');
const ip = await MeshService.getInstance().resolveContainerIp({ stack: target.stack, service: target.service });
if (!ip) {
this.sendJson({ t: 'tcp_open_ack', s, ok: false, err: 'no_target' });
return;
}
const socket = net.createConnection({ host: ip, port: target.port });
const state: ReverseLocalTcpStreamState = { kind: 'reverse_local', socket, bytesIn: 0, bytesOut: 0 };
this.streams.set(s, state);
this.refreshIdleTimer(s, state);
const teardown = (sendClose: boolean) => {
if (!this.streams.has(s)) return;
this.removeStream(s);
try { socket.destroy(); } catch { /* ignore */ }
if (sendClose) this.sendJson({ t: 'tcp_close', s });
};
// Pre-connect failure: ack-fail and drop. The handler is removed in
// 'connect' below so post-connect errors fall through to the
// mid-stream teardown path instead of double-firing.
const onPreConnectError = () => {
if (!this.streams.has(s)) return;
this.streams.delete(s);
this.sendJson({ t: 'tcp_open_ack', s, ok: false, err: 'unreachable' });
};
socket.once('error', onPreConnectError);
socket.once('connect', () => {
socket.off('error', onPreConnectError);
this.sendJson({ t: 'tcp_open_ack', s, ok: true });
socket.on('data', (chunk: Buffer) => {
const cur = this.streams.get(s);
if (!cur || cur.kind !== 'reverse_local') return;
this.sendBinary(BinaryFrameType.TcpData, s, chunk);
cur.bytesOut += chunk.length;
this.refreshIdleTimer(s, cur);
});
socket.on('close', () => teardown(true));
socket.on('error', () => teardown(true));
});
}
private async acceptReverseRelay(s: number, targetNodeId: number, target: { stack: string; service: string; port: number }): Promise<void> {
const { PilotTunnelManager } = await import('./PilotTunnelManager');
const targetBridge = PilotTunnelManager.getInstance().getBridge(targetNodeId);
if (!targetBridge) {
this.sendJson({ t: 'tcp_open_ack', s, ok: false, err: 'unreachable' });
return;
}
const targetStream = targetBridge.openTcpStream(target);
if (!targetStream) {
this.sendJson({ t: 'tcp_open_ack', s, ok: false, err: 'unreachable' });
return;
}
const state: ReverseRelayTcpStreamState = { kind: 'reverse_relay', target: targetStream, bytesIn: 0, bytesOut: 0 };
this.streams.set(s, state);
this.refreshIdleTimer(s, state);
targetStream.once('open', () => {
this.sendJson({ t: 'tcp_open_ack', s, ok: true });
});
targetStream.on('data', (chunk: Buffer) => {
const cur = this.streams.get(s);
if (!cur || cur.kind !== 'reverse_relay') return;
this.sendBinary(BinaryFrameType.TcpData, s, chunk);
cur.bytesOut += chunk.length;
this.refreshIdleTimer(s, cur);
});
targetStream.once('error', () => {
if (!this.streams.has(s)) return;
this.streams.delete(s);
this.sendJson({ t: 'tcp_open_ack', s, ok: false, err: 'unreachable' });
});
targetStream.once('close', () => {
if (!this.streams.has(s)) return;
this.removeStream(s);
this.sendJson({ t: 'tcp_close', s });
});
}
}