fix(pilot): post-merge audit followups (WS via getProxyTarget, closeTunnel lifecycle, mesh source buffer, docs) (#1128)

* fix(pilot): route remote WS upgrades through NodeRegistry.getProxyTarget

Pilot-mode nodes carry empty api_url and api_token by design and expose
their API on a per-tunnel loopback bridge. The upgrade handler gated the
remote-forwarder branch on `node.api_url && node.api_token`, so WS
requests targeting pilot nodes silently fell through to the local
handlers (live logs, exec, generic) instead of tunneling to the agent.

Resolve the target via NodeRegistry.getProxyTarget so pilot and proxy
modes share one dispatch path, mirroring the HTTP proxy.
handleRemoteForwarder now takes the resolved target and, when the target
is the pilot loopback (empty token), skips the console-token exchange
and the Authorization injection so the tunnel-side auth is the only
source of truth on that path. Unresolvable targets reject the upgrade
with HTTP 503 instead of being served gateway-local data.

* fix(pilot): emit tunnel-down and mark node offline on closeTunnel

PilotTunnelManager.closeTunnel closed the underlying WebSocket but
skipped the cleanup the natural-disconnect path runs, so explicit
closures (enrollment regenerate, node deletion) left the node row at
status='online' until the next reconnect. The dashboard kept showing
the stale state for the entire interval.

closeTunnel now writes nodes.status='offline' and emits tunnel-down for
pilot bridges, and emits proxy-bridge-down for central-initiated proxy
bridges. The maps are cleared before bridge.close() so the natural
'closed' handler's bridge-identity guard short-circuits and we do not
double-emit.

* fix(mesh): buffer cross-node source data until tcp_open_ack arrives

openCrossNode piped src socket data straight to tcpStream.write before
the forward TcpStream emitted 'open'. The first packet on a fresh
cross-node stream raced ahead of the agent's tcp_open_ack on the wire,
which broke protocols that send immediately after connect (HTTP, TLS,
Redis, Postgres) on Pilot and proxy mesh paths.

Buffer src chunks in a local array capped at STREAM_PENDING_DATA_MAX_BYTES
until tcpStream emits 'open', then flush them in order before any
post-open writes. Tear down both sockets if the buffer overflows so a
misbehaving source cannot exhaust gateway memory while waiting for the
ack.

* docs(pilot): clarify host-console non-parity and narrow the parity claim

Pilot mode disables the host-console capability at the capability
registry (the agent container has no useful host shell to surface), but
the public docs listed host console among the WebSockets that ride
through the tunnel and described pilot as behaving identically to proxy
mode. State the shared-capability claim more carefully and call out the
intentional non-parity in a dedicated subsection.
This commit is contained in:
Anso
2026-05-21 01:00:47 -04:00
committed by GitHub
parent 8fd02ef39b
commit e65c5e8551
8 changed files with 320 additions and 22 deletions
+27 -15
View File
@@ -1,6 +1,5 @@
import type { IncomingMessage } from 'http';
import type { Duplex } from 'stream';
import type { Node } from '../services/DatabaseService';
import { PROXY_TIER_HEADER, PROXY_VARIANT_HEADER } from '../services/license-headers';
import { LicenseService } from '../services/LicenseService';
import { wsProxyServer } from '../proxy/websocketProxy';
@@ -13,35 +12,41 @@ import { rejectUpgrade as reject } from './reject';
* api_token never reaches an interactive terminal (the remote's upgrade
* handler rejects node_proxy tokens on those paths).
*
* The caller must have already established that `node.type === 'remote'`
* and that `api_url` + `api_token` are present.
* Target resolution lives in the caller (parallel to the HTTP proxy in
* `proxy/remoteNodeProxy.ts`), so this function works uniformly for
* proxy-mode remotes (api_url + api_token) and pilot-mode remotes (loopback
* URL + empty token). When the target carries an empty token, the loopback
* bridge sits on 127.0.0.1 and demuxes onto an already-authenticated tunnel:
* we skip the console-token exchange and do not inject a Bearer.
*/
export async function handleRemoteForwarder(
req: IncomingMessage,
socket: Duplex,
head: Buffer,
opts: { node: Node; pathname: string },
opts: { pathname: string; target: { apiUrl: string; apiToken: string } },
): Promise<void> {
const { node, pathname } = opts;
// Guaranteed non-null by caller; assert here so the rest of the function is nullsafe.
if (!node.api_url || !node.api_token) return reject(socket, 503, 'Service Unavailable');
const { pathname, target } = opts;
if (!target.apiUrl) return reject(socket, 503, 'Service Unavailable');
const wsTarget = node.api_url.replace(/\/$/, '').replace(/^https?/, (m) => m === 'https' ? 'wss' : 'ws');
const wsTarget = target.apiUrl.replace(/\/$/, '').replace(/^https?/, (m) => m === 'https' ? 'wss' : 'ws');
const isPilotLoopback = target.apiToken === '';
// Interactive console paths (host console / container exec) are guarded on
// the remote by an isProxyToken check that rejects the long-lived api_token.
// Exchange it for a short-lived console_session token before forwarding so
// the remote allows the connection while keeping the guard intact for
// direct api_token access.
// direct api_token access. Pilot loopback targets skip this: there is no
// long-lived api_token to exchange, and host-console is disabled on pilot
// mode at the capability registry anyway.
const isInteractiveConsolePath = pathname === '/api/system/host-console' || pathname === '/ws';
let bearerTokenForProxy = node.api_token;
if (isInteractiveConsolePath) {
let bearerTokenForProxy = target.apiToken;
if (isInteractiveConsolePath && !isPilotLoopback) {
try {
const consoleHeaders = LicenseService.getInstance().getProxyHeaders();
const tokenRes = await fetch(`${node.api_url.replace(/\/$/, '')}/api/system/console-token`, {
const tokenRes = await fetch(`${target.apiUrl.replace(/\/$/, '')}/api/system/console-token`, {
method: 'POST',
headers: {
'Authorization': `Bearer ${node.api_token}`,
'Authorization': `Bearer ${target.apiToken}`,
[PROXY_TIER_HEADER]: consoleHeaders.tier,
[PROXY_VARIANT_HEADER]: consoleHeaders.variant || '',
},
@@ -58,11 +63,18 @@ export async function handleRemoteForwarder(
}
}
req.headers['authorization'] = `Bearer ${bearerTokenForProxy}`;
if (isPilotLoopback) {
// The loopback bridge does not authenticate inbound traffic; an
// Authorization header inherited from the browser would only confuse
// the agent. Strip it explicitly.
delete req.headers['authorization'];
} else {
req.headers['authorization'] = `Bearer ${bearerTokenForProxy}`;
}
delete req.headers['x-node-id'];
// Strip the browser's session cookie: signed by this instance's JWT secret
// and would fail verification on the remote. Auth is handled exclusively
// via the Bearer token.
// via the Bearer token (or, for pilot loopback, the tunnel itself).
delete req.headers['cookie'];
const fwdHeaders = LicenseService.getInstance().getProxyHeaders();
req.headers[PROXY_TIER_HEADER] = fwdHeaders.tier;
+12 -2
View File
@@ -182,8 +182,18 @@ export function attachUpgrade(
return;
}
if (node && node.type === 'remote' && node.api_url && node.api_token) {
await handleRemoteForwarder(req, socket, head, { node, pathname });
if (node && node.type === 'remote') {
// Resolve the proxy target through NodeRegistry so pilot-mode nodes
// (empty api_url + api_token, loopback bridge instead) and proxy-mode
// nodes share one dispatch path. Mirrors proxy/remoteNodeProxy.ts.
const target = NodeRegistry.getInstance().getProxyTarget(nodeId);
if (!target) {
// Pilot tunnel disconnected, or proxy-mode node missing credentials.
// Reject the upgrade cleanly; falling through to local handlers would
// serve gateway-local data for a request that named a remote node.
return reject(socket, 503, 'Service Unavailable');
}
await handleRemoteForwarder(req, socket, head, { pathname, target });
return;
}