fix(pilot): post-merge audit followups (WS via getProxyTarget, closeTunnel lifecycle, mesh source buffer, docs) (#1128)

* fix(pilot): route remote WS upgrades through NodeRegistry.getProxyTarget

Pilot-mode nodes carry empty api_url and api_token by design and expose
their API on a per-tunnel loopback bridge. The upgrade handler gated the
remote-forwarder branch on `node.api_url && node.api_token`, so WS
requests targeting pilot nodes silently fell through to the local
handlers (live logs, exec, generic) instead of tunneling to the agent.

Resolve the target via NodeRegistry.getProxyTarget so pilot and proxy
modes share one dispatch path, mirroring the HTTP proxy.
handleRemoteForwarder now takes the resolved target and, when the target
is the pilot loopback (empty token), skips the console-token exchange
and the Authorization injection so the tunnel-side auth is the only
source of truth on that path. Unresolvable targets reject the upgrade
with HTTP 503 instead of being served gateway-local data.

* fix(pilot): emit tunnel-down and mark node offline on closeTunnel

PilotTunnelManager.closeTunnel closed the underlying WebSocket but
skipped the cleanup the natural-disconnect path runs, so explicit
closures (enrollment regenerate, node deletion) left the node row at
status='online' until the next reconnect. The dashboard kept showing
the stale state for the entire interval.

closeTunnel now writes nodes.status='offline' and emits tunnel-down for
pilot bridges, and emits proxy-bridge-down for central-initiated proxy
bridges. The maps are cleared before bridge.close() so the natural
'closed' handler's bridge-identity guard short-circuits and we do not
double-emit.

* fix(mesh): buffer cross-node source data until tcp_open_ack arrives

openCrossNode piped src socket data straight to tcpStream.write before
the forward TcpStream emitted 'open'. The first packet on a fresh
cross-node stream raced ahead of the agent's tcp_open_ack on the wire,
which broke protocols that send immediately after connect (HTTP, TLS,
Redis, Postgres) on Pilot and proxy mesh paths.

Buffer src chunks in a local array capped at STREAM_PENDING_DATA_MAX_BYTES
until tcpStream emits 'open', then flush them in order before any
post-open writes. Tear down both sockets if the buffer overflows so a
misbehaving source cannot exhaust gateway memory while waiting for the
ack.

* docs(pilot): clarify host-console non-parity and narrow the parity claim

Pilot mode disables the host-console capability at the capability
registry (the agent container has no useful host shell to surface), but
the public docs listed host console among the WebSockets that ride
through the tunnel and described pilot as behaving identically to proxy
mode. State the shared-capability claim more carefully and call out the
intentional non-parity in a dedicated subsection.

backend/src/services/MeshService.ts

@@ -15,6 +15,7 @@ import { PilotTunnelManager } from './PilotTunnelManager';
 import { MeshProxyTunnelDialer, type DialFailureCode } from './MeshProxyTunnelDialer';
 import { generateOverrideYaml, MeshAlias, SENCHO_MESH_NETWORK } from './MeshComposeOverride';
 import { lookupContainerIp } from '../mesh/containerLookup';
+import { STREAM_PENDING_DATA_MAX_BYTES } from '../pilot/protocol';
 import { sanitizeForLog } from '../utils/safeLog';
 import { isPathWithinBase, isValidStackName } from '../utils/validation';
 import { PORT as SENCHO_LISTEN_PORT } from '../helpers/constants';
@@ -1893,6 +1894,15 @@ export class MeshService extends EventEmitter implements MeshForwarderHost {
         const record = this.registerActiveStream(target.alias, tcpStream.streamId);
         const t0 = Date.now();
 
+        // Hold src bytes until tcp_open_ack arrives. Writing through to the
+        // tunnel before 'open' fires races the first packet ahead of the
+        // ack, which breaks protocols that send immediately after connect
+        // (HTTP, TLS, Redis, Postgres). Cap matches the bridge's reservation
+        // cap so a misbehaving source cannot exhaust gateway memory.
+        let tcpOpen = false;
+        const pending: Buffer[] = [];
+        let pendingBytes = 0;
+
         // Timer guards against the agent never returning a tcp_open_ack
         // (broken pilot, frame dropped, dial stuck after handshake).
         // Without this the stream sits forever and the operator sees
@@ -1929,6 +1939,14 @@ export class MeshService extends EventEmitter implements MeshForwarderHost {
                 message: `cross-node connect to ${target.alias}`,
             });
             this.routeLatencyMap.set(target.alias, Date.now() - t0);
+            // Flush any bytes that arrived before tcp_open_ack so the upstream
+            // sees them in order, ahead of anything that lands post-ack.
+            for (const buf of pending) {
+                try { tcpStream.write(buf); } catch { /* ignore */ }
+            }
+            pending.length = 0;
+            pendingBytes = 0;
+            tcpOpen = true;
         });
         tcpStream.on('data', (chunk: Buffer) => {
             record.bytesIn += chunk.length;
@@ -1949,7 +1967,17 @@ export class MeshService extends EventEmitter implements MeshForwarderHost {
         });
         src.on('data', (chunk: Buffer) => {
             record.bytesOut += chunk.length;
-            tcpStream.write(chunk);
+            if (tcpOpen) {
+                tcpStream.write(chunk);
+                return;
+            }
+            if (pendingBytes + chunk.length > STREAM_PENDING_DATA_MAX_BYTES) {
+                try { src.destroy(); } catch { /* ignore */ }
+                try { tcpStream.destroy(); } catch { /* ignore */ }
+                return;
+            }
+            pending.push(Buffer.from(chunk));
+            pendingBytes += chunk.length;
         });
         src.on('end', () => tcpStream.end());
         src.on('close', () => { cleanupRecord(); try { tcpStream.destroy(); } catch { /* ignore */ } });

backend/src/services/PilotTunnelManager.ts

@@ -295,14 +295,28 @@ export class PilotTunnelManager extends EventEmitter {
     }
 
     /**
-     * Force-close a tunnel (e.g., on node deletion).
+     * Force-close a tunnel (e.g., on node deletion, enrollment regenerate).
+     *
+     * Mirrors the cleanup the bridge's natural `'closed'` event handler runs
+     * so explicit closure and natural disconnect produce identical state:
+     * pilot tunnels write `nodes.status='offline'` and emit `tunnel-down`;
+     * proxy bridges emit `proxy-bridge-down`. The maps are cleared before
+     * `bridge.close()` so the natural handler's `=== bridge` check
+     * short-circuits and we do not double-emit.
      */
     public closeTunnel(nodeId: number, code = 1000, reason = 'closed by primary'): void {
         const bridge = this.bridges.get(nodeId);
         if (!bridge) return;
-        bridge.close(code, reason);
+        const kind = this.bridgeKinds.get(nodeId);
         this.bridges.delete(nodeId);
         this.bridgeKinds.delete(nodeId);
+        if (kind === 'pilot') {
+            DatabaseService.getInstance().updateNodeStatus(nodeId, 'offline');
+            this.emit('tunnel-down', nodeId);
+        } else if (kind === 'proxy') {
+            this.emit('proxy-bridge-down', nodeId);
+        }
+        bridge.close(code, reason);
     }
 
 }