6 Commits

Author	SHA1	Message	Date
Anso	865d792874	feat(pricing): collapse to two tiers (#1309) ... * feat(pricing): collapse to two tiers (Community + Admiral) Collapse Sencho's pricing from three tiers (Community / Skipper / Admiral) to two: a generous free Community tier and a single paid Admiral tier. The Skipper tier is removed. Now free in Community: auto-heal, auto-update, scheduled operations, webhooks, notification routing, Fleet Actions and bulk operations, SSO preset providers (Google / GitHub / Okta), unlimited users with admin and viewer roles, and deploy safety (atomic deploys, auto-rollback, and one-click rollback). Admiral (paid) is focused on running and governing a fleet: blueprints, Fleet Secrets, deploy enforcement, vulnerability report export, audit log, host console, private registries, mesh networking, node cordon, managed cloud backup, LDAP / Active Directory SSO, and the advanced RBAC roles (deployer, node-admin, auditor) with per-resource scoped assignments. Internally the license variant distinction is removed so tier is binary (community / paid). License validation still verifies the Lemon Squeezy store and product before granting paid status. Docs and the contributor guide are updated to the two-tier model. * docs(pricing): correct licensing page to two-tier pricing and tidy stale tier wording The licensing docs page kept the old Admiral pricing plus a Founder Lifetime column and an Enterprise paragraph after the two-tier collapse. Update it to $12/month or $99/year, drop the lifetime and Enterprise content, and link to the pricing page for current pricing. Also fix stale "Skipper" wording in CLA.md, SUPPORT.md, one test title, and three test comments. Historical CHANGELOG entries and the retired-Skipper license-guard test are intentionally left as-is. * docs: align licensing and SSO pages with the two-tier model Correct the SSO overview so the Google, GitHub, and Okta presets read as available on every tier, matching the provider table; only LDAP and Active Directory require Sencho Admiral. Remove the lifetime-plan references from the licensing, settings, and troubleshooting pages so they reflect subscription-only Admiral pricing. * fix(rbac): omit scoped permissions from /me on the Community tier Scoped role assignments only take effect on the paid tier, but GET /api/permissions/me returned them unconditionally, so a downgraded instance with leftover assignments rendered per-resource affordances the API then rejected with 403. The endpoint now mirrors the permission middleware and includes scoped permissions only on the paid tier. Adds a regression test covering the downgrade case. * docs: use custom-pricing wording on the contact page The two-tier model has no Enterprise tier; reword the contact page's enterprise pricing/deals to custom pricing/deals so it does not imply a tier that no longer exists.	2026-06-04 17:45:53 -04:00
Anso	982c7830b1	fix(mesh): address audit follow-ups (early-data, recompose, admiral gate, error codes, docs) (#1126) ... * fix(mesh): buffer early TcpData on reverse-relay path The reverse-relay code dropped the local-shaped reservation and any TcpData frames buffered in it before acceptReverseRelay had wired up the target stream. A peer that sent a request body immediately after tcp_open_reverse lost those bytes when the target lived on a third node, since reverse_local buffers them but reverse_relay did not. Carry the reservation through acceptReverseRelay: transplant its pendingData and pendingBytes into the new reverse_relay state, gate TcpData writes on targetOpen, and flush buffered frames into the target stream before sending tcp_open_ack. Mirrors the reverse_local pattern. Regression test fires tcp_open_reverse plus an immediate TcpData while ensureBridge is in flight and verifies the bytes arrive intact, in order, before the ack. * fix(mesh): recompose affected stacks when node-level mesh is disabled disableForNode used to clear DB rows and override files but leave the running containers attached to sencho_mesh with stale /etc/hosts alias entries until an operator redeployed every stack by hand. Mirror optOutStack: after the existing alias/forwarder cleanup, call regenerateOverridesAcrossFleet, cascadeRecomposeAcrossFleet, and triggerRedeploy for each previously meshed stack on the disabled node so containers detach from sencho_mesh and shed the alias entries they owned. The disabled node's mesh_stacks rows are deleted before the cascade so listMeshStacks returns the right set with no skip tuple required. Route threads the actor through actorFor(req) for parity with optInStack/optOutStack. Tests cover the redeploy fan-out, the cascade no-skip-tuple invariant, and the default actor fallback for non-route callers. * fix(mesh): require Admiral on the WS proxy-tunnel upgrade HTTP mesh routes in routes/mesh.ts all enforce requireAdmiral, but the /api/mesh/proxy-tunnel WS upgrade accepted any node_proxy or full-admin api_token regardless of the receiver's license. A node downgraded from Admiral kept serving mesh data-plane traffic to a sibling central while refusing every mesh management call. Read the receiver's local LicenseService at the upgrade and 403 when the tier is not paid+admiral. The check sits after the existing credential gate and uses LicenseService directly rather than effectiveTier (which trusts forwarded proxy headers); a remote peer dialing in cannot be trusted to assert our entitlement. Dialer and node_proxy token format are unchanged. Three regression tests cover community-tier node_proxy, skipper-tier node_proxy, and community-tier full-admin api_token all rejected with 403. * fix(mesh): handle no_target alongside push_failed in inspectStackServices proxyFetch throws MeshError('no_target') when getProxyTarget returns null (pilot tunnel offline, proxy bridge unreachable), but the inspectStackServices catch branch only matched push_failed. Offline remotes fell through to the generic 'remote unreachable' error log, which the Routing tab surfaces as an unexpected fault. Match both error codes and emit the operator-friendly warn message that names the unreachable node and the error code. Regression test spies on console.warn/console.error to pin the branch. * docs(mesh): align env defaults and forwarder comments with current architecture SENCHO_MESH_PROXY_TUNNEL_IDLE_MS in .env.example carried the old five-minute idle-close value (=300000), but the code default is DEFAULT_IDLE_TTL_MS=0 (persistent tunnel). Copying the example silently reintroduced the idle-close behavior the dialer removed. MeshForwarder.ts's leading docblock and inline listen comment still described host-network mode plus extra_hosts: host-gateway as required for forwarder reachability. Sencho runs in standard bridge mode and attaches to the shared sencho_mesh network at a stable IP; meshed user containers reach the forwarder by that IP directly. Flip the env default to 0, rewrite the env comment to describe the persistent behavior and the opt-in for idle teardown, and rewrite both forwarder comments to match the bridge-network reality. * fix(mesh): trust forwarded tier on proxy-tunnel WS and remove remote overrides on disable Admiral entitlement on the WS data plane now follows the same trust model as the HTTP mesh routes: the central asserts its tier via x-sencho-tier and x-sencho-variant on the WS handshake, and the receiver trusts those headers only when the upgrade carries a node_proxy credential. When no headers are present or the credential is a full-admin api_token, the receiver falls back to its own local license. Without this an Admiral central could be rejected by a Community remote and a Community central could dial a locally-Admiral remote. disableForNode now routes through removeOverrideFromNode so override files pushed earlier via applyLocalOverride are removed on remote nodes via DELETE /api/mesh/local-override/:stack. Sequential awaits are wrapped in Promise.allSettled to match regenerateOverridesForNode's parallel push pattern. Other changes: - Cover the post-state-swap buffering window in the reverse-relay test (TcpData arriving after openTcpStream returns but before target open). - Refresh stale MeshService comments that still referenced the removed sidecar layer and host-network listener model. * test(mesh): pin removeOverrideFromNode remote HTTP shape The disableForNode regression test mocks removeOverrideFromNode itself, so a regression inside the helper would not be caught. Add a narrow contract test that spies on global fetch and asserts the request shape: DELETE /api/mesh/local-override/:stack against the resolved proxy target, with Authorization Bearer plus the x-sencho-tier and x-sencho-variant headers. Also covers the encodeURIComponent path and the swallowed-network-error behavior the disable cascade relies on. * test(mesh): use createTestApiToken helper in proxy-tunnel api_token gate test The full-admin api_token branch of the WS Admiral gate test inlined the canonical generateApiToken + sha256 + addApiToken triple that already lives in the createTestApiToken helper. Switching to the helper removes the duplicated insertion logic and aligns this test with the helper used by the other api_token call sites.	2026-05-20 12:53:29 -04:00
Anso	f6e42535c8	fix(mesh): route peer→central traffic over the existing forward WS (#1094) ... * fix(mesh): route peer→central traffic over the existing forward WS The reverse mesh callback path (`/api/mesh/proxy-tunnel-from-peer`) needed SENCHO_PRIMARY_URL on central plus a publicly reachable origin from the peer's perspective. In a typical homelab where central sits behind NAT, peer→central dispatch silently failed at the dialer's short-circuit and the headline "call any service on any node by hostname" worked one way only. The forward WS at `/api/mesh/proxy-tunnel` is already bidirectional end to end. Make the bridge a persistent control-plane primitive: dial every mesh-enabled proxy peer at startup, reconcile every 60 s, never idle-close. Peer→central traffic multiplexes over the same WS via `tcp_open_reverse`. Removed: - `meshProxyTunnelFromPeer.ts` WS handler and dispatch - `MeshCentralRegistry`, `PeerToCentralMeshSessionDialer` - `mesh_handshake` first-frame state machine in `meshProxyTunnel.ts` - `maybeSendBootstrap`, `buildHandshakeFrame` in the dialer - `mesh_proxy_callback_bootstrap` capability and `maybeWarnUnsetPrimaryUrl` - `mesh_centrals` table (drop migration; greenfield, no users) - `PilotTunnelManager.replaceOrRegisterProxyBridge` (dead after handler removal) - twelve associated unit/integration tests plus the peer-recovery branch in `MeshService.openCrossNode` Added: - `MeshService.proactiveBridgeFanout` selects every mesh-enabled proxy peer (no longer gated on `mesh_stacks` rows) - `startBridgeReconcileLoop` runs the fanout every 60 s (override via `SENCHO_MESH_RECONCILE_INTERVAL_MS`) - `MeshProxyTunnelDialer` default idle TTL is now `0` and exposes `isDialing(nodeId)` for the status surface - `MeshNodeStatus.reverseCallbackStatus` discriminator (`connected | connecting | unavailable | not_applicable`) surfaced via `/api/mesh/status` and rendered as a pill in the Routing tab - `openCrossNode` error message distinguishes "no proxy target" from "waiting for central to dial the reverse bridge" - New tests: `mesh-service-proxy-tunnel-reconcile`, `mesh-status-reverse-callback`, `mesh-proxy-tunnel-dialer-no-idle-close` SENCHO_PRIMARY_URL is no longer required for any mesh function. * fix(mesh): rewrite proxy-tunnel reconcile test contents The previous commit renamed the file but the rewritten test bodies stayed unstaged on top of the rename. This commit lands the actual rewrite: the fanout assertion now requires every mesh-enabled proxy peer to be dialed, not just those with `mesh_stacks` rows, and adds a reconcile-tick repeated-call test.	2026-05-17 22:00:31 -04:00
Anso	cf618dd866	feat(mesh): symmetric WS dial for proxy-mode mesh peers (#1066) ... * chore(mesh): foundation for symmetric callback dial Adds the data-plane scaffolding that the symmetric callback dial fix builds on: - mesh_centrals table for peer-side bootstrap material - MeshCentralRegistry service (upsert/getActive/clear/markUsed/markRejected) - PilotTunnelManager kind discriminator and replaceOrRegisterProxyBridge - mesh_proxy_callback_bootstrap capability registration - MeshProxyTunnelDialer reason-tagged proxy-bridge-down events from a single tearDownBridge emission point - Reactive redial scheduler that skips idle and auth_failed reasons * feat(mesh): add reverse-direction activity log entries (closes R1-B) acceptReverseLocal now emits route.resolve.ok with direction=reverse on connect ack and route.resolve.fail with direction=reverse plus reason=container_not_found / connect_error pre-connect. Post-connect close/error stays silent. Reuses existing event types via the new details.direction discriminator so frontend filters are unaffected. * feat(mesh): add peer-to-central callback dial path (closes R1-A2) Closes the architectural gap where proxy-mode mesh peers could not re-establish their tunnel to central after any non-idle bridge teardown (idle close, network blip, central restart, peer reboot). Central remains the hub for the data plane; the change is purely about WS initiation. Symmetric WS initiation, asymmetric protocol roles. Central retains PilotTunnelBridge ownership; peer retains TcpStreamSwitchboard + reverseDialer ownership. Central bootstraps callback credentials over the first authenticated central-initiated mesh tunnel via a one-shot mesh_handshake JSON frame; peer persists the material in a new mesh_centrals SQLite table and dials central's new /api/mesh/proxy-tunnel-from-peer endpoint when local cross-node traffic needs a bridge and none is live. Mesh_tunnel JWT (HS256, signed with auth_jwt_secret) carries scope, audience, issuer (central instance id), peer api_token fingerprint, kid. Validation on inbound peer dial: algorithm pin, signature, scope, audience, instance, time bounds, node existence and mode, fingerprint match. Failures return HTTP 401 with a machine-readable reason; peer routes the response per a clear-vs-keep cache matrix. Triggers proactive bootstrap on mesh-enable and api_token rotation; central startup fans out to mesh-enabled proxy-mode nodes with mesh_stacks rows (throttled, fire-and-forget). Reactive redial on non-idle bridge loss. Capability-gated handshake send (mesh_proxy_callback_bootstrap) makes the upgrade path safe against older peers in mixed-version fleets. Adds peer-side /api/system/pilot-tunnels centralCallback diag block, bounded counter metrics for bootstrap and dial events. SENCHO_PRIMARY_URL preflight warning when unset on a central with mesh-enabled proxy nodes. Tested with unit suites for the validation chain, registry, manager, and both dialers; integration tests for bootstrap E2E (asserts protocol-role invariant), api_token rotation, instance id change, version skew, and pilot-mode regression. * fix(mesh): green CI on the symmetric callback branch Two independent CI failures, both surgical: 1. Backend tests (11 fails): four mesh test files called setupTestDb in beforeEach. setupTestDb does not reset the DatabaseService singleton, so the per-test afterEach rm of the previous tmpdir left the singleton connection pointing at a deleted file. The next beforeEach's line-55 write threw SQLITE_READONLY_DBMOVED on Linux. Windows file-lock semantics hid this locally. Hoist setupTestDb / cleanupTestDb to file-scope beforeAll / afterAll; per-test state resets stay in beforeEach. Matches the convention in the eight mesh test files that already pass. 2. CodeQL (4 high alerts): js/insufficient-password-hash flagged sha256(api_token) at four sites. The api_token is a 256-bit opaque bearer (sen_sk_-prefixed), not a human password; sha256 is the correct fingerprint primitive for binding the mesh_tunnel JWT to a specific token. Add the two production files plus the two test files that mint the fingerprint to the existing path-scoped query-filter for that rule. * fix(mesh): drop unused afterEach import and revert dead codeql config ESLint flagged afterEach as unused in mesh-central-registry.test.ts:1 after the previous commit hoisted setup/teardown to file-scope beforeAll/afterAll. Remove from the vitest import line. Revert the codeql-config.yml additions from the previous commit. The paths: sub-key under query-filters > exclude is not a documented CodeQL feature and silently no-ops. The four js/insufficient-password-hash alerts on api_token fingerprinting are tracked as dismissed false positives in the GitHub Security tab rather than via dead config.	2026-05-16 14:58:19 -04:00
Anso	6185206323	fix(mesh): proxy peer learns its central-namespace nodeId at tunnel upgrade (#1055) ... Proxy-mode peers receive an alias overlay from central with nodeIds in central's namespace, but had no source for their own nodeId in that namespace. selfCentralNodeId fell back to the peer's local DB default (always 1), and handleAccept falsely matched cross-node aliases (nodeId=1, central's Local) against the peer's self-id. Cross-node aliases routed to openSameNode and surfaced as route.resolve.denied. Central now appends ?nodeId=<central-namespace-id> to the proxy-tunnel WS URL. The peer's upgrade handler parses it with a strict integer regex, installs into MeshService before the reverse dialer is registered, and clears on disconnect. handleAccept resolves self-id in priority order: proxy-tunnel install > SENCHO_ENROLL_TOKEN > local default. A non-null overwrite to a different value warns loudly (misconfigured deployment). Missing or malformed query param logs a warn and proceeds, leaving reverse-direction dispatch undefined until both sides are upgraded. Adds 5 test cases: nodeId install + teardown, missing/malformed input (including decimal, leading zeros, exponent, whitespace), overwrite warn, and CAS-rejected reverse-dialer leaving the slot unchanged.	2026-05-15 09:00:05 -04:00
Anso	a38a3e0226	feat(mesh): route mesh traffic over Distributed API remotes (#1048) ... * refactor(mesh): extract shared TCP stream switchboard Pull the `tcp_open` / `tcp_open_ack` / `tcp_open_reverse` / `tcp_close` + `TcpData` handling out of the pilot agent into a reusable module so a second caller (the upcoming proxy-mode WS handler) can run the same frame parser, stream allocator, idle timers, and Compose-label resolver. One parser, two callers, zero drift on a security-sensitive protocol surface. The pilot agent keeps its public `openMeshTcpStream` API and delegates to a per-connection switchboard constructed in `connect()` and torn down in `cleanupAfterDisconnect`. Existing reverse-stream and resolver tests are rewritten to exercise the shared module directly. * feat(mesh): route mesh traffic over Distributed API remotes Sencho Mesh now works against remotes in Distributed API mode in addition to Pilot Agent mode. Central opens a short-lived WebSocket tunnel to the remote's new `/api/mesh/proxy-tunnel` endpoint on demand and tears it down after 5 minutes of idle (configurable via `SENCHO_MESH_PROXY_TUNNEL_IDLE_MS`). Mesh dispatch is mode-agnostic at the routing layer; `PilotTunnelManager.ensureBridge` resolves an existing tunnel or asks the new `MeshProxyTunnelDialer` to dial. The Routing tab badges now use a `reachableMode` classifier: `★ Local` for local, `pilot offline` red badge for a pilot with a down tunnel, and a new `unreachable` red badge surfacing the specific reason (missing token, scope not full-admin, TLS failure, remote does not support proxy mesh). Distributed API remotes with valid credentials show no negative badge; the tunnel opens on first dial. Auth: the proxy-tunnel WS upgrade requires an `Authorization: Bearer` API token with the `full-admin` scope. Lower-scoped tokens are rejected at upgrade time so a leaked read-only token cannot reach the mesh data plane. Bidirectional: the proxy-mode WS handler registers itself as the local `MeshService` reverse dialer (compare-and-swap), so meshed containers on a Distributed API remote can dial cross-node aliases via `tcp_open_reverse` over the same tunnel. Cross-node relays (`PilotTunnelBridge.acceptReverseRelay`) await `ensureBridge` so proxy-to-proxy mesh works without standing tunnels. * docs(mesh): describe Distributed API mesh and unreachable troubleshooting Refresh the user-facing mesh documentation to reflect that mesh works over both Pilot Agent and Distributed API remotes. Adds a Troubleshooting accordion entry for the new `unreachable` Routing tab badge so operators can match a tooltip reason ("api token rejected (scope must be full-admin)", "remote does not support proxy mesh", "TLS handshake failed", "api_url not set", "api token missing") to a concrete fix. * refactor(mesh): apply /simplify review findings Five cleanups surfaced by the post-implementation code review pass. No behaviour change for fleets running on `main`; only internal structure improves. - **Deterministic container IP shared.** Extract the compose-default network preference (`pickContainerIp`) and the conventional-name fast path (`lookupContainerIp`) into `backend/src/mesh/containerLookup.ts`. Both `MeshService.resolveContainerIp` (existing same-node fast path) and the switchboard's `resolveByComposeLabels` (proxy/pilot inbound dial) now use the same logic. Earlier the switchboard helper grabbed the first `Object.values(Networks)` IP, which could flip across daemon versions on multi-network containers; fixed. - **WS URL upgrade extracted.** New `backend/src/utils/wsUrl.ts` exposes `httpUrlToWs(baseUrl)` that maps `http://` to `ws://` and `https://` to `wss://`. Used in both `pilot/agent.ts` and the proxy-tunnel dialer. The previous inline `replace(/^http/, 'ws')` silently downgraded `https://` to `ws://` (cleartext). - **`computeReachable` takes the pre-fetched node.** `getStatus` already iterated `db.getNodes()`; the helper previously re-queried by id per node (N+1 reads). Pass the row in. - **Reachable-reason ternary -> const map.** Replace the nested ternary in `MeshService.computeReachable` with a `Record<DialFailureCode, string>` lookup keyed on the dialer's failure code. - **Activity-type ternary -> const map.** Same flattening inside `MeshProxyTunnelDialer.logActivity`. All mesh tests pass (85/85). Full backend suite green (2126/2126). * fix(mesh): cache proxy dial failures and contain WS handshake errors `ensureBridge` now consults the recent-failure cache before dialing so a continuous mesh workload against a misconfigured proxy-mode remote does not produce one upgrade attempt per cross-node TCP open. `recordFailure` emits at most one activity-log entry per cache window per (nodeId, code) so a connect-loop on a single bad remote cannot flush the ring buffer. Failure messages run through `redactSensitiveText` before reaching the log so any embedded Bearer / JWT / inline-URL credentials are scrubbed. `stop()` clears the inflight map and `dial()` checks `this.stopped` between awaits so a shutdown does not leak a half-opened bridge. When `awaitOpen` rejects (401, 4403, TLS), the dialer attaches a noop 'error' listener before calling `ws.close()`. Without it the ws library emits a tail 'error' on a still-CONNECTING socket that propagates as an unhandled exception. Surfaced by a live-network test against a real proxy-mode peer. Adds `mesh-proxy-tunnel-live.test.ts` (skipped unless MESH_AUDIT_URL and MESH_AUDIT_TOKEN_FILE env vars are set) covering both the happy path and the auth-rejected path against an actual remote Sencho. * feat(mesh): instrument proxy tunnel observability Adds three counters to `PilotMetrics`: - `proxy_bridges_total` (incremented on `registerProxyBridge` success) - `proxy_dials_failed` (every failed dial attempt, not deduped) - `proxy_idle_closes` (idle-sweep teardowns) All three surface automatically via `GET /api/system/pilot-tunnels` since the route returns the full `Counters` snapshot. Gates two pre-existing always-on `console.warn` calls in `tcpStreamSwitchboard.ts` (mid-stream socket errors and Docker resolve failures) behind `isDebugEnabled()`. Both fire on per-stream events and would otherwise violate the diagnostic-log safety rule under load. Adds `mesh-tcp-stream-switchboard.test.ts` covering the forward `tcp_open` path: real localhost dial, resolver errors (no_target, denied), per-tunnel cap saturation, frame-routing fall-through invariants, `tcp_close` socket teardown. Drops `MESH_CONNECT_TIMEOUT_MS` from the public surface; only the switchboard itself uses it. * fix(mesh): tighten Routing tab unreachable handling `RoutingNodeCard.tsx`: the **Add stack to mesh** button now disables when `reachableMode === 'unreachable'`, matching the existing TogglePill behavior. Without this, an operator on an unreachable node could open the opt-in sheet, confirm, and watch the redeploy proceed against a target whose mesh data plane will silently fail to route. Also drops the leftover `status.nodeId !== -1` guard on the pilot- offline badge. `MeshService.ts`: renames `isMeshReachable` to `isMeshConfigured` with the new predicate that returns true for proxy-mode remotes whose creds are valid (the tunnel is opened on demand). `getRouteDiagnostic` now distinguishes "routable" (configured) from "pilotLive" (live tunnel state, only meaningful for pilot mode); without the split, every idle proxy-mode route would report `tunnel down`. `MeshService.setReverseDialer` warns when the unconditional install path silently overwrites a non-null current dialer. By topology a Sencho is either pilot or central, so the branch flags a misconfigured deployment rather than an expected race. Drops the dead `export { ReverseTcpStreamHandle }` re-export from `pilot/agent.ts`; its only consumer imports straight from `mesh/tcpStreamSwitchboard.ts`. Fixes a stale doc comment in `MeshService.ts` that referenced the wrong source file. Adds `mesh-proxy-tunnel-handler.test.ts` covering the WS handler lifecycle: pilot-mode 404 rejection, post-upgrade reverse-dialer install, concurrent-upgrade 1013 rejection (single-tenant slot), and error-path teardown. Updates the troubleshooting accordion in the user docs to mention the disabled-state behavior. * fix(mesh): close ESLint and CodeQL findings on the proxy-tunnel diag logs Remove the unused `reverseDialer` local in `meshProxyTunnel.ts`; the closure target is `localDialer` above and this assignment was always dead. Strip line breaks inline on the three `MeshProxyTunnelDialer` diag log lines so CodeQL `js/log-injection` data flow recognises the sanitisation that `sanitizeForLog` already performs. No behaviour change; the diag logs render the same characters they do today.	2026-05-14 20:51:42 -04:00