Anso a38a3e0226 feat(mesh): route mesh traffic over Distributed API remotes (#1048)
* refactor(mesh): extract shared TCP stream switchboard

Pull the `tcp_open` / `tcp_open_ack` / `tcp_open_reverse` / `tcp_close`
+ `TcpData` handling out of the pilot agent into a reusable module so a
second caller (the upcoming proxy-mode WS handler) can run the same
frame parser, stream allocator, idle timers, and Compose-label
resolver. One parser, two callers, zero drift on a
security-sensitive protocol surface.

The pilot agent keeps its public `openMeshTcpStream` API and delegates
to a per-connection switchboard constructed in `connect()` and torn
down in `cleanupAfterDisconnect`. Existing reverse-stream and resolver
tests are rewritten to exercise the shared module directly.

* feat(mesh): route mesh traffic over Distributed API remotes

Sencho Mesh now works against remotes in Distributed API mode in
addition to Pilot Agent mode. Central opens a short-lived WebSocket
tunnel to the remote's new `/api/mesh/proxy-tunnel` endpoint on demand
and tears it down after 5 minutes of idle (configurable via
`SENCHO_MESH_PROXY_TUNNEL_IDLE_MS`). Mesh dispatch is mode-agnostic at
the routing layer; `PilotTunnelManager.ensureBridge` resolves an
existing tunnel or asks the new `MeshProxyTunnelDialer` to dial.

The Routing tab badges now use a `reachableMode` classifier: `★ Local`
for local, `pilot offline` red badge for a pilot with a down tunnel,
and a new `unreachable` red badge surfacing the specific reason
(missing token, scope not full-admin, TLS failure, remote does not
support proxy mesh). Distributed API remotes with valid credentials
show no negative badge; the tunnel opens on first dial.

Auth: the proxy-tunnel WS upgrade requires an `Authorization: Bearer`
API token with the `full-admin` scope. Lower-scoped tokens are
rejected at upgrade time so a leaked read-only token cannot reach the
mesh data plane.

Bidirectional: the proxy-mode WS handler registers itself as the
local `MeshService` reverse dialer (compare-and-swap), so meshed
containers on a Distributed API remote can dial cross-node aliases
via `tcp_open_reverse` over the same tunnel. Cross-node relays
(`PilotTunnelBridge.acceptReverseRelay`) await `ensureBridge` so
proxy-to-proxy mesh works without standing tunnels.

* docs(mesh): describe Distributed API mesh and unreachable troubleshooting

Refresh the user-facing mesh documentation to reflect that mesh works
over both Pilot Agent and Distributed API remotes. Adds a
Troubleshooting accordion entry for the new `unreachable` Routing tab
badge so operators can match a tooltip reason ("api token rejected
(scope must be full-admin)", "remote does not support proxy mesh",
"TLS handshake failed", "api_url not set", "api token missing") to a
concrete fix.

* refactor(mesh): apply /simplify review findings

Five cleanups surfaced by the post-implementation code review pass.
No behaviour change for fleets running on `main`; only internal
structure improves.

- **Deterministic container IP shared.** Extract the compose-default
  network preference (`pickContainerIp`) and the conventional-name
  fast path (`lookupContainerIp`) into `backend/src/mesh/containerLookup.ts`.
  Both `MeshService.resolveContainerIp` (existing same-node fast
  path) and the switchboard's `resolveByComposeLabels` (proxy/pilot
  inbound dial) now use the same logic. Earlier the switchboard
  helper grabbed the first `Object.values(Networks)` IP, which
  could flip across daemon versions on multi-network containers;
  fixed.
- **WS URL upgrade extracted.** New `backend/src/utils/wsUrl.ts`
  exposes `httpUrlToWs(baseUrl)` that maps `http://` to `ws://` and
  `https://` to `wss://`. Used in both `pilot/agent.ts` and the
  proxy-tunnel dialer. The previous inline `replace(/^http/, 'ws')`
  silently downgraded `https://` to `ws://` (cleartext).
- **`computeReachable` takes the pre-fetched node.** `getStatus`
  already iterated `db.getNodes()`; the helper previously re-queried
  by id per node (N+1 reads). Pass the row in.
- **Reachable-reason ternary -> const map.** Replace the nested
  ternary in `MeshService.computeReachable` with a
  `Record<DialFailureCode, string>` lookup keyed on the dialer's
  failure code.
- **Activity-type ternary -> const map.** Same flattening inside
  `MeshProxyTunnelDialer.logActivity`.

All mesh tests pass (85/85). Full backend suite green (2126/2126).

* fix(mesh): cache proxy dial failures and contain WS handshake errors

`ensureBridge` now consults the recent-failure cache before dialing so a
continuous mesh workload against a misconfigured proxy-mode remote does
not produce one upgrade attempt per cross-node TCP open. `recordFailure`
emits at most one activity-log entry per cache window per (nodeId, code)
so a connect-loop on a single bad remote cannot flush the ring buffer.
Failure messages run through `redactSensitiveText` before reaching the
log so any embedded Bearer / JWT / inline-URL credentials are scrubbed.

`stop()` clears the inflight map and `dial()` checks `this.stopped`
between awaits so a shutdown does not leak a half-opened bridge.

When `awaitOpen` rejects (401, 4403, TLS), the dialer attaches a noop
'error' listener before calling `ws.close()`. Without it the ws library
emits a tail 'error' on a still-CONNECTING socket that propagates as an
unhandled exception. Surfaced by a live-network test against a real
proxy-mode peer.

Adds `mesh-proxy-tunnel-live.test.ts` (skipped unless MESH_AUDIT_URL
and MESH_AUDIT_TOKEN_FILE env vars are set) covering both the happy
path and the auth-rejected path against an actual remote Sencho.

* feat(mesh): instrument proxy tunnel observability

Adds three counters to `PilotMetrics`:
- `proxy_bridges_total` (incremented on `registerProxyBridge` success)
- `proxy_dials_failed` (every failed dial attempt, not deduped)
- `proxy_idle_closes` (idle-sweep teardowns)

All three surface automatically via `GET /api/system/pilot-tunnels`
since the route returns the full `Counters` snapshot.

Gates two pre-existing always-on `console.warn` calls in
`tcpStreamSwitchboard.ts` (mid-stream socket errors and Docker resolve
failures) behind `isDebugEnabled()`. Both fire on per-stream events and
would otherwise violate the diagnostic-log safety rule under load.

Adds `mesh-tcp-stream-switchboard.test.ts` covering the forward
`tcp_open` path: real localhost dial, resolver errors (no_target,
denied), per-tunnel cap saturation, frame-routing fall-through
invariants, `tcp_close` socket teardown.

Drops `MESH_CONNECT_TIMEOUT_MS` from the public surface; only the
switchboard itself uses it.

* fix(mesh): tighten Routing tab unreachable handling

`RoutingNodeCard.tsx`: the **Add stack to mesh** button now disables
when `reachableMode === 'unreachable'`, matching the existing
TogglePill behavior. Without this, an operator on an unreachable node
could open the opt-in sheet, confirm, and watch the redeploy proceed
against a target whose mesh data plane will silently fail to route.
Also drops the leftover `status.nodeId !== -1` guard on the pilot-
offline badge.

`MeshService.ts`: renames `isMeshReachable` to `isMeshConfigured` with
the new predicate that returns true for proxy-mode remotes whose creds
are valid (the tunnel is opened on demand). `getRouteDiagnostic` now
distinguishes "routable" (configured) from "pilotLive" (live tunnel
state, only meaningful for pilot mode); without the split, every idle
proxy-mode route would report `tunnel down`.

`MeshService.setReverseDialer` warns when the unconditional install
path silently overwrites a non-null current dialer. By topology a
Sencho is either pilot or central, so the branch flags a misconfigured
deployment rather than an expected race.

Drops the dead `export { ReverseTcpStreamHandle }` re-export from
`pilot/agent.ts`; its only consumer imports straight from
`mesh/tcpStreamSwitchboard.ts`. Fixes a stale doc comment in
`MeshService.ts` that referenced the wrong source file.

Adds `mesh-proxy-tunnel-handler.test.ts` covering the WS handler
lifecycle: pilot-mode 404 rejection, post-upgrade reverse-dialer
install, concurrent-upgrade 1013 rejection (single-tenant slot), and
error-path teardown.

Updates the troubleshooting accordion in the user docs to mention the
disabled-state behavior.

* fix(mesh): close ESLint and CodeQL findings on the proxy-tunnel diag logs

Remove the unused `reverseDialer` local in `meshProxyTunnel.ts`; the
closure target is `localDialer` above and this assignment was always
dead. Strip line breaks inline on the three `MeshProxyTunnelDialer`
diag log lines so CodeQL `js/log-injection` data flow recognises the
sanitisation that `sanitizeForLog` already performs.

No behaviour change; the diag logs render the same characters they
do today.
2026-05-14 20:51:42 -04:00
2026-03-25 00:40:06 -04:00

Sencho

Self-hosted Docker Compose management for one machine or a fleet.

Docs · Website · Discussions · Sponsor

Latest release Docker Pulls CI License Discussions


Sencho dashboard

What Sencho is

Sencho is for homelab operators, small DevOps teams, and platform engineers who run services on Docker Compose, want a graphical interface without giving up file-on-disk workflows, and need to manage more than one machine without SSH gymnastics or a VPN.

It runs as a single container on your hardware and gives you a UI for the work you currently do over SSH on compose stacks: deploying, editing files, watching logs, restarting containers, browsing volumes, and recovering from failures. Your compose files stay on the host filesystem and remain the source of truth.

A Sencho instance is autonomous. To manage another machine, you install a second Sencho on it and connect them with a long-lived API token; the primary dashboard then acts as a transparent HTTPS proxy across your fleet. There is no SSH and no exposed Docker socket. For nodes behind NAT or strict firewalls, the Pilot Agent establishes a single outbound WebSocket tunnel to the primary, so the remote host opens no inbound port at all.

Most capabilities are free in the Community tier. A few advanced automation and fleet-control features ship in paid tiers; pricing lives at sencho.io/pricing.


Capabilities

Stacks

  • Full Compose lifecycle: create, deploy, restart, stop, pull
  • Monaco editor with diff preview before save and one-click rollback
  • Git-sourced stacks pulled and synced from any repository
  • File explorer for compose, env, and supporting files
  • Stack labels for grouping and bulk operations
  • App Store with LinuxServer.io templates

Observability

  • Aggregated log search and stream across every container in the fleet
  • Live container stats, health checks, and image-update notifications
  • Threshold alerts for CPU, memory, and network
  • Read-only audit log of every action
  • Network topology view of containers, networks, and nodes

Fleet

  • Multi-node management via authenticated HTTP and WebSocket proxy
  • Fleet view with grid and topology layouts
  • Fleet snapshots of compose and env across the fleet
  • Pilot Agent for nodes behind NAT or strict firewalls
  • Node compatibility checks before deploying

Automation

Security

Operations


Quick start

Sencho runs in a single container.

services:
  sencho:
    image: saelix/sencho:latest
    container_name: sencho
    restart: unless-stopped
    ports:
      - "1852:1852"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./data:/app/data
      # 1:1 Compose Path Rule: the host path MUST match the container path
      - /opt/docker:/opt/docker
    environment:
      - COMPOSE_DIR=/opt/docker
      - DATA_DIR=/app/data
docker compose up -d

Open http://your-server:1852 and create your admin account.

Always front Sencho with a TLS-terminating reverse proxy in production. See the self-hosting guide for hardening, environment variables, and reverse-proxy examples.

Run with docker run instead
docker run -d --name sencho \
  -p 1852:1852 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v sencho_data:/app/data \
  -e COMPOSE_DIR=/opt/docker \
  saelix/sencho:latest

For the full walkthrough, see the quickstart guide.


Adding remote nodes

To manage a second machine, install Sencho on it the same way, then add it from the primary dashboard with its URL and a long-lived API token. The primary proxies authenticated HTTPS and WebSocket requests to the remote instance. No SSH, no exposed Docker socket, no agent process on the remote. Nodes behind NAT or strict firewalls can opt into the Pilot Agent for outbound-only connectivity.

See the multi-node guide for the full token-bearer flow.


Screenshots

Stacks Editor
Fleet Logs

Documentation, community, and license


Contributors

S
Description
Self-hosted Docker Compose management platform. Great for homelabs, small DevOps teams, and platform engineers.
Readme AGPL-3.0 177 MiB
Languages
TypeScript 99.2%
JavaScript 0.4%
CSS 0.3%
Dockerfile 0.1%