mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-07-26 11:49:16 +00:00
412d3696c8efea927ea44d64896012c10390a66f
1154 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Anso
|
412d3696c8 |
ci: drop sencho-mesh sidecar publish job (#1011)
The shared sencho_mesh Docker network replaced host-mode mesh, so the sidecar image is no longer built or distributed. Remove the obsolete push_mesh_sidecar job from the release workflow. |
||
|
sencho-quartermaster[bot]
|
e5e55d7f10 |
chore(main): release 0.76.0 (#1008)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
Anso
|
23bbee4f45 |
feat(mesh): replace host-mode with shared sencho_mesh Docker network (#1009)
* feat(mesh): replace host-mode with shared sencho_mesh Docker network Phase D of the mesh redesign: drop the operator's `network_mode: host` requirement and the `host-gateway` extra_hosts pattern that did not work on cloud iptables-restrictive distros (OCI, etc.) or Docker Desktop. Each Sencho creates a shared `sencho_mesh` Docker bridge network on boot (default subnet 172.30.0.0/24, override via SENCHO_MESH_SUBNET), pins itself at `<network>+2`, and attaches every meshed user service to the same bridge. Compose overrides now emit IP-based `extra_hosts` plus a top-level `networks` block declaring `sencho_mesh` external. Override delivery: central renders for local stacks; for remote stacks it sends the fleet alias list to the remote's new `PUT /api/mesh/local- override/:stackName` endpoint, which renders against the remote's OWN local senchoIp and writes under its OWN DATA_DIR. Each node may use a different subnet without coordination beyond the env var. Opt-in / opt-out now trigger an automatic redeploy of the affected stack via the existing deploy code path (local: ComposeService; remote: HTTP POST through proxyFetch). The frontend opt-in sheet shows a confirmation modal (ConfirmModal) before the mutation. Failed redeploys emit both a mesh activity event and a durable audit-log row. Hardening: - Reserve port 1852 at opt-in (prevents user containers from racing the Sencho API listener). - ensureMeshNetwork refuses to continue if `sencho_mesh` exists with a mismatched subnet rather than silently routing to the wrong IP. - Idempotent network connect/disconnect helpers in DockerController. - optInStack rolls back the DB row if the just-inserted stack's override push fails (no half-states surviving across calls). - regenerateOverridesForNode runs in parallel and skips the just- pushed stack on opt-in. Operator template: drop `network_mode: host`, restore `ports: ["1852:1852"]`. Mesh now works identically on Linux LAN, OCI, and Docker Desktop without firewall changes. Docs: rewrite docs/features/sencho-mesh.mdx around the shared bridge network, document SENCHO_MESH_SUBNET, surface the host-network-service opt-in restriction, and cross-link with the Pilot Agent docs. BREAKING CHANGE: the operator's `docker-compose.yml` no longer uses `network_mode: host`. After upgrading, redeploy any meshed stacks once so they pick up the new IP-based override and join `sencho_mesh`. * fix(mesh): wrap stackName with path.basename in local-override fs ops CodeQL flagged js/path-injection on the new applyLocalOverride and removeLocalOverride methods because they are publicly reachable and its data-flow model does not recognize isValidStackName / isPathWithinBase as sanitizers. The validation IS sufficient (the allowlist regex blocks path separators, the path-prefix check blocks escape), but path.basename is a model CodeQL recognizes and is purely defensive: for any input that already passes isValidStackName, basename is the identity. |
||
|
Anso
|
ccad5c925b |
feat(nodes): hide hub-only views when active node is remote (#1007)
* feat(nodes): hide hub-only views when active node is remote Fleet, Schedules, Audit, Logs, and Auto-Update operate on hub-owned state (node registry, fleet schedules, centralized audit, fleet-wide log aggregation, fleet-wide update preview). When the active node is remote, proxying those surfaces would show that remote's own disconnected state instead of the hub's. Hide them from the nav strip and force-redirect to Home if one was open during the node switch. Backend hubOnlyGuard middleware sits between nodeContextMiddleware and the remote proxy and rejects /api/scheduled-tasks, /api/audit-log, and /api/notification-routes with 403 + HUB_ONLY_ENDPOINT when nodeId resolves to a remote, closing the script-bypass path the UI gating cannot reach. Settings sub-sections were already gated via the hiddenOnRemote registry; this extends the same model to top-level views. * docs(nodes): note hub-only visibility on Fleet, Schedules, Audit, Logs, Auto-Update Each of the five hub-only feature pages now points readers to the canonical "What top-level views show when a remote node is active" section in multi-node.mdx, so users landing directly on a feature page understand why the nav item disappears when they switch to a remote node. |
||
|
dependabot[bot]
|
7ad9381ede |
chore(deps-dev): bump fast-uri from 3.1.0 to 3.1.2 (#1006)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.0 to 3.1.2. - [Release notes](https://github.com/fastify/fast-uri/releases) - [Commits](https://github.com/fastify/fast-uri/compare/v3.1.0...v3.1.2) --- updated-dependencies: - dependency-name: fast-uri dependency-version: 3.1.2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
sencho-quartermaster[bot]
|
1c20739e90 |
chore(main): release 0.75.1 (#1005)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
Anso
|
10a469ecb4 |
fix(mesh): bind every alias port on every node, not just local-owned (#1004)
syncForwarderListeners filtered the bind set to ports owned by the local node. That broke cross-node mesh routing end to end: meshed containers' extra_hosts: <alias>:host-gateway entries resolve to the SOURCE node's gateway, so the source node is where the inbound TCP connection lands. With the filter, the source node never bound the remote alias's port and the connection went nowhere. The architecture model documented in docs/internal/architecture/ mesh.md (data flow step 3) is explicit: monolith's Sencho process accepts the connection on its MeshForwarder listener bound to port 5432 — where monolith is the source and opsix is the target. Every meshed node binds every alias port. handleAccept then resolves the alias and dispatches to openSameNode (when target.nodeId equals the local node) or openCrossNode (otherwise). Both branches were already correct; only the bind filter was wrong. Fleet-wide port collisions remain blocked at opt-in time (optInStack checks aliasByPort), so binding every alias port is unambiguous. New vitest case asserts the want-set includes both local-owned and remote-owned ports. Discovered while running PR 0 verification on v0.75.0 against Local + sencho-pilot-test: connections from a Local prober to the pilot's alias port hit Local's host with no listener. |
||
|
sencho-quartermaster[bot]
|
e4411f3247 |
chore(main): release 0.75.0 (#1002)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
Anso
|
567e524450 |
feat(mesh): bidirectional routing via tcp_open_reverse and central relay (#1003)
* feat(mesh): bidirectional routing via tcp_open_reverse and central relay
Phase B of the mesh redesign. Adds the reverse-direction protocol so a
pilot's MeshForwarder can route cross-node traffic to central or to
another pilot. Central relays pilot-to-pilot streams transparently;
pilots keep their existing single outbound WS to central.
Protocol additions (backend/src/pilot/protocol.ts):
- TcpOpenReverseFrame { s, targetNodeId, stack, service, port } sent
agent to primary. Reuses existing tcp_open_ack, tcp_close, and
TcpData binary frames for the response and byte plane.
- AGENT_REVERSE_ID_BASE = 0x40000001 splits the 32-bit id space so
agent-allocated reverse stream ids never collide with primary-
allocated forward ids on the same tunnel.
- StreamIdAllocator now wraps to its configured start (not always 1)
so an allocator parameterized with the agent base stays in the
agent half across the wrap.
Agent (backend/src/pilot/agent.ts):
- New ReverseTcpStreamHandle exported class, EventEmitter facade
matching PilotTunnelBridge.TcpStream's surface (write, end,
destroy plus open, data, error, close events).
- Public openMeshTcpStream(target) allocates a reverse id, sends
tcp_open_reverse, returns the handle.
- onTcpOpenAckReverse handles inbound ack, dispatches open or
error+close to the matching handle.
- TcpData and tcp_close binary/JSON paths route ids in the reverse
range to reverseTcpStreams; existing primary-allocated paths
unchanged.
- streamCount, cleanupAfterDisconnect, onStreamIdle include reverse
streams. The allocator is reset to a fresh base on disconnect so
long-lived agents that reconnect many times do not drift up the id
space.
- startPilotAgent registers the agent as MeshService's reverse
dialer via lazy import.
Bridge (backend/src/services/PilotTunnelBridge.ts):
- Two new StreamState kinds: reverse_local (target = central, dial
Dockerode container IP) and reverse_relay (target = another pilot,
open a forward TcpStream on the target pilot's bridge).
- handleTcpOpenReverse validates the agent-id range and stream cap,
then dispatches to acceptReverseLocal or acceptReverseRelay via
lazy imports of MeshService, NodeRegistry, and PilotTunnelManager
(avoids module cycles).
- acceptReverseLocal calls MeshService.resolveContainerIp (now
public), opens net.createConnection, splices bytes through the
tunnel. Pre-connect error handler is removed inside connect to
avoid double-firing with the mid-stream error path.
- acceptReverseRelay opens a forward TcpStream on the target
bridge, splices bytes between the two tunnels.
- Existing tcp_close and TcpData handlers extended for the new
state kinds. teardownStream extended.
MeshService (backend/src/services/MeshService.ts):
- resolveContainerIp made public so the bridge's local-target path
can dial the same shape.
- New reverseDialer field plus setReverseDialer setter. Pilot mode
registers the agent at boot; central mode leaves it null.
- New private dialMeshTcpStream dispatcher: pilot side uses the
reverse dialer, central side uses PilotTunnelManager.getBridge.
- openCrossNode refactored to call the dispatcher and use the
active-stream record's id for activity logging.
- New exported MeshTcpStreamLike interface that both
PilotTunnelBridge.TcpStream and ReverseTcpStreamHandle satisfy
structurally; ReverseMeshDialer is the contract for the setter.
Tests (3 files, 16 new cases, 172 total pass):
- pilot-protocol-tcp.test.ts: tcp_open_reverse round-trip; agent
base invariant.
- pilot-agent-reverse-stream.test.ts: openMeshTcpStream allocation,
ws-not-open, frame shape, write encoding, end emits tcp_close,
ack-success, ack-failure, low-id ack ignored, inbound TcpData
routing.
- pilot-bridge-reverse.test.ts: id-range validation, no-target
failure, successful local dial against a real upstream server.
Together with PR #1000 (Phase A) and PR #1001 (deps), Phase B
completes the central-pilot-pilot mesh routing matrix. Phase C
(mesh over proxy-mode remotes) is the planned follow-up.
* test(pilot): drop unused encodeJsonFrame import
Lint failed on pilot-agent-reverse-stream.test.ts after the test
changed from constructing tcp_open_reverse frames inline to driving
the agent's frame-dispatch path with synthetic objects. The import
is no longer referenced; ESLint's no-unused-vars rule rejected it.
|
||
|
Anso
|
f599110386 |
feat(mesh): collapse sidecar into Sencho process via in-process forwarder (#1000)
The separate saelix/sencho-mesh sidecar container is gone. The
forwarder logic that previously lived in mesh-sidecar/src/forwarder.ts
moves into the Sencho process as backend/src/services/MeshForwarder.ts,
a thin per-port net.Server lifecycle wrapper. MeshService implements
the host interface and owns resolve plus splice; MeshForwarder owns
listener boilerplate. One container per node, no separate image to
publish, no control WebSocket.
Operator-facing change: the Sencho container now runs in
network_mode: host so the forwarder can bind alias ports on the host
network where meshed containers' extra_hosts host-gateway entries
point. Without host network mode the listeners would land in the
container's namespace and inbound traffic from peers would never
reach them. The 1852:1852 port publish becomes a no-op under host
mode and is commented out in the operator template.
Same-node forward path now dials the target container's bridge IP
via Dockerode (preferring the compose default network for
deterministic selection across daemon versions) instead of 127.0.0.1.
The legacy 127.0.0.1 path only worked when the target service
published its port to the host; the IP path works regardless.
Cross-node mesh routing in this phase is central -> pilot direction
only via PilotTunnelManager.openTcpStream. Pilot -> central and
pilot <-> pilot via central relay land in Phase B with the
tcp_open_reverse frame.
Deletions:
- mesh-sidecar/ package entirely (Dockerfile, package, sources, tests)
- backend/src/websocket/meshControl.ts
- MeshService sidecar lifecycle: spawnSidecar, stopSidecar,
isSidecarRunning, mintSidecarToken, verifySidecarToken,
attachSidecarSocket, handleSidecarResolve, sendSidecar
- POST /api/mesh/nodes/:id/sidecar/restart route
- /api/mesh/control WS dispatch in upgradeHandler
Type cleanup: 'sidecar' literal removed from MeshActivitySource and
MeshProbeResult.where (also the frontend mirror). MeshNodeStatus
sidecarRunning becomes localForwarderListening (boolean | null) so
non-local nodes get a null instead of an unconditional false; the
honest semantic is "this view only knows the local forwarder state;
remote forwarder status lands in Phase B." MeshNodeDiagnostic
sidecar object becomes forwarder { listening, listenerCount }.
Frontend MeshDiagnosticsSheet drops the restart-sidecar action and
sidecar liveness card; surfaces forwarder state plus a "runs
in-process; no separate container" caption.
Resolves audit findings C-1 (data plane non-functional), C-2 (sidecar
control WS not loopback-enforced), C-4 (sidecar lifecycle Dockerode-
on-remote, PR #999 closed), and C-5 (saelix/sencho-mesh:latest
unreachable). C-3 (PR #992) is unchanged. M-12 (PR #994) is
unchanged.
|
||
|
Anso
|
b5463e1771 |
chore(deps): bump fast-uri to 3.1.1+ to clear GHSA-q3j6-qgpj-74h6 (#1001)
GitHub published a high-severity advisory against fast-uri <= 3.1.0 (path traversal via percent-encoded dot segments, GHSA-q3j6-qgpj-74h6) which CI's npm audit step now flags. The vulnerability is not exploitable in Sencho's usage: The dependency chain is composerize@1.7.5 -> composeverter@1.7.5 -> ajv@8.18.0 -> fast-uri@3.1.0. Composerize is consumed only by POST /api/convert/ in routes/convert.ts, which parses CLI-style docker run flags (not URLs) and returns YAML. ajv's use of fast-uri is for internal JSON-Schema $ref/$id resolution, not user input. The CVE requires attacker-controlled URLs flowing into fast-uri.normalize() or equal() for path-based policy bypass; that flow does not exist in Sencho. Even though the vulnerability is not exploitable, npm audit blocks CI regardless of exploitability assessment (it does not read VEX). This commit is a lockfile-only bump produced by npm audit fix. ajv@8.18.0's range already accepts fast-uri >= 3.1.1 so no package.json change is needed. Verification: npm audit reports 0 vulnerabilities post-fix; backend test suite (1921 tests, 5 skipped) is green; tsc --noEmit is clean. |
||
|
Anso
|
1803512f70 |
feat: drop stack labels and network topology to Community tier (#995)
* feat(labels): drop tier gate to Community for organization endpoints Stack Labels CRUD and per-stack assignment are now Community-tier features: list, create, update, delete labels, and assign labels to a single stack. The two automation surfaces stay Skipper+: per-label bulk deploy / stop / restart (POST /api/labels/:id/action) and the Fleet Actions tab's bulk-assign card (POST /api/fleet-actions/labels/bulk-assign). Tier story is now organize free, automate paid. Add a route-level test that proves the CRUD endpoints succeed on a Community license while the bulk-action endpoint still returns 403. Update overview, licensing, and stack-labels docs to reflect the new tier placement and to note that bulk actions on a label still require Skipper or Admiral. * feat(topology): drop tier gate to Community for network topology view The Resources tab's Networks > Topology view is now available on every tier. Drops requirePaid from GET /api/system/networks/topology, removes the isPaid wrapper around the List | Topology toggle in ResourcesView, and removes the PaidGate around the topology graph. CapabilityGate stays in place so a node running on a build without the network-topology capability still renders its lock card instead of the graph. Add a route-level test that proves the endpoint returns 200 on a Community license. Update licensing and resources docs to reflect the new tier placement. * fix(labels): expose Settings > Labels tab on Community tier The settings registry entry for the Labels tab still carried tier: 'skipper', which kept the tab hidden in the Settings sidebar even though the underlying CRUD endpoints now serve Community. Drop the tier flag so Community users can discover and reach the section that backs the already-Community-tier label endpoints. |
||
|
sencho-quartermaster[bot]
|
0f3380897a |
chore(main): release 0.74.4 (#998)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
Anso
|
3d1cf0f8f1 |
fix(pilot): generate auth_jwt_secret on pilot-agent boot (#997)
Pilot-agent hosts never run the first-run setup wizard, so the wizard path that normally generates auth_jwt_secret (routes/auth.ts) never fires. Without that secret, the agent-side loopback auth helper added in PR #990 (pilot/agent.ts::getLoopbackAuthHeader) returned null at mint time, no Authorization header was attached on the forwarded loopback request, and the local Sencho's authMiddleware rejected every proxied call with 401 "Authentication required". This was reproducible end-to-end on v0.74.1, .2, and .3. Add ensurePilotJwtSecret() to bootstrap/startup, called early in startServer (before LicenseService.initialize, which only reads system_state and does not touch global_settings, so the order is safe). When SENCHO_MODE=pilot and global_settings.auth_jwt_secret is empty, generate 64 random bytes (hex-encoded, matching the wizard's pattern in routes/auth.ts) and persist. Subsequent boots see the persisted value and no-op. Outside pilot mode the function is a no-op since the wizard owns the lifecycle. The helper is exported so the regression test can exercise the real function rather than maintaining a hand-mirrored copy. Three vitest cases cover: first-boot generation, no-op on persisted secret, no-op in non-pilot mode. Together with PR #989 (proxy-side bridge dispatch), PR #990 (agent-side loopback auth injection), PR #992 (cross-node control plane via HTTP proxy), and PR #994 (local-node mesh-reachable diagnostic), this completes the central, bridge, agent, local-Sencho HTTP path for pilot-agent-mode nodes. |
||
|
sencho-quartermaster[bot]
|
ff4be9019a |
chore(main): release 0.74.3 (#996)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
Anso
|
34581e55e7 |
fix(mesh): treat local-node aliases as always reachable in diagnostics (#994)
MeshService.getRouteDiagnostic called PilotTunnelManager.hasActiveTunnel unconditionally for the alias's target node. Local nodes never establish pilot tunnels because they do not need them (mesh same-node traffic uses the localhost fast path), so the call always returned false and the diagnostic flipped to state: 'tunnel down' even on a working local route. The route detail sheet and node card consequently rendered every local mesh alias as a destructive 'tunnel down' pill. Added a private isMeshReachable helper that returns true for any local-type node and otherwise delegates to hasActiveTunnel. Used it in both getRouteDiagnostic and getStatus, replacing the inline ternary in getStatus that already had the right shape but lived as duplicated logic. The helper falls back to false when the node row is missing (orphaned alias from a partial cascade), matching the pre-existing conservative behavior. Verified live: GET /api/mesh/aliases/echo.audit-mesh-prod.Local.sencho/diagnostic previously returned state='tunnel down' on a healthy local route. |
||
|
sencho-quartermaster[bot]
|
f7ff7955ee |
chore(main): release 0.74.2 (#993)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
Anso
|
e176ae9f17 |
fix(mesh): inspect remote stacks via HTTP proxy not Dockerode (#992)
MeshService.inspectStackServices used DockerController.getInstance(nodeId)
to enumerate compose-labeled containers, but NodeRegistry.getDocker
explicitly throws for any node with type='remote' by design. The throw
was silently caught and an empty array returned, so opt-in for any
remote node failed with a misleading "no running services" error and
refreshAliasCache only ever populated LOCAL aliases. Cross-node mesh
routing was therefore impossible end-to-end regardless of pilot-tunnel
state.
Split the inspector. inspectLocalStackServices keeps the Dockerode
listContainers path and always queries the local Docker daemon; it is
public so the new route can call it. inspectStackServices is now a
dispatcher: local nodes fall through to the Dockerode path, remote
nodes (proxy mode and pilot-agent) HTTP-fetch
/api/mesh/local-services/:stackName against the URL resolved by
NodeRegistry.getProxyTarget, with the persisted node_proxy Bearer
token and license tier headers attached. The remote's MeshService
enumerates its own LOCAL Docker daemon and returns the
{service, ports[]} envelope.
refreshAliasCache now inspects every opted-in stack in parallel via
Promise.allSettled so a slow or unreachable remote does not stall the
60-second loop. The new /api/mesh/local-services/:stackName route is
gated by requireAdmiral plus isValidStackName and always queries the
caller's own Sencho instance.
Together with PR #989 (proxy-side bridge dispatch) and PR #990
(agent-side loopback auth), this makes the central, bridge, agent,
local-Sencho mesh control plane functional end-to-end for both
proxy-mode and pilot-agent remotes.
|
||
|
sencho-quartermaster[bot]
|
ee6e761f70 |
chore(main): release 0.74.1 (#991)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
Anso
|
a6d3e5d052 |
fix(pilot): inject loopback auth on agent-side HTTP/WS forwarding (#990)
After the central proxy forwards a request through the pilot tunnel, the agent receives an http_req or ws_open frame and opens a local loopback request to its own Sencho server. The forwarder previously copied the inbound headers verbatim with no Authorization header, so the local authMiddleware rejected every proxied call with 401 Authentication required. The central cannot sign the missing token because each Sencho instance has its own auth_jwt_secret generated at first-run setup. The agent process runs in the same Node process as the local Sencho server and shares the local DatabaseService, so it can mint a fresh pilot_tunnel-scoped JWT signed with the local secret. Token TTL is 5 minutes with a refresh-on-use 4-minute cache to avoid jwt.sign on every request. The header is built once per request via buildLoopbackHeaders and applied to both onHttpReq and onWsOpen so the two forwarder paths stay in lockstep. The local authMiddleware accepts the scope through its existing pilot_tunnel branch with no special-case bypass. Together with the proxy-side change in the previous commit, this completes the central, bridge, agent, local-Sencho HTTP path for pilot-agent-mode nodes. |
||
|
sencho-quartermaster[bot]
|
269ba2909b |
chore(main): release 0.74.0 (#980)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
Anso
|
94ce7c71d2 |
fix(proxy): route pilot-agent HTTP via PilotTunnelBridge loopback (#989)
The remote-node HTTP proxy resolved targets by reading nodes.api_url and
nodes.api_token directly from the database. Both fields are empty for
pilot-agent nodes by design, which produced a misleading 503 ("no API URL
or token configured. Update it in Settings, Nodes.") for any API call
targeting a healthy pilot-agent: stack creation, log retrieval, and every
other resource a pilot-agent should serve.
NodeRegistry.getProxyTarget already encapsulates the correct dispatch.
For proxy mode it returns the persisted api_url and api_token. For
pilot-agent it returns the loopback URL of the active PilotTunnelBridge
with an empty token, since the bridge re-authenticates implicitly via the
pre-verified tunnel socket.
Switch all three lookup sites in remoteNodeProxy to this helper, cache
the resolved target on req.proxyTarget so the http-proxy router and
proxyReq callbacks do not re-resolve, and split the 503 message so
pilot-agent operators see "Pilot tunnel to X is disconnected" instead of
the proxy-mode hint.
|
||
|
Anso
|
c9452337e1 |
test(pilot): in-process simulation of mid-tunnel disconnects (#987)
* test(pilot): in-process simulation of mid-tunnel disconnects The PR #979 hardening added per-stream idle timers, drain handling, paused-request maps, and a tcpAwaitingDrain set on the bridge. None of those cleanup paths had an end-to-end test driving them through a real disconnect. A future refactor that inverts the order of clearIdleTimer + streams.delete (or that misses one of the aux maps) would leak per-tunnel memory in a way unit tests cannot catch. Spin up a real http.Server with attachUpgrade and a real ws.WebSocket client (same self-contained pattern as pilot-tunnel-integration.test.ts). Four cases: - HTTP request mid-flight: open a real loopback HTTP request, wait for the bridge to forward http_req to the agent, terminate the agent WS, assert the loopback request resolves with 502 (the teardownStream path) and the manager marks the node Offline. - TCP stream with bytes outstanding: openTcpStream from the bridge, ack open from the test agent, write some bytes, terminate the tunnel, assert the TcpStream emits 'close' and the bridge is gone from the manager. - Reconnect with the long-lived token after a clean close does NOT bump tunnels_replaced (no live tunnel to replace). - Reconnect WHILE the prior tunnel is still live DOES bump tunnels_replaced. Inverse case proves the counter wiring is correct in both directions. Self-contained: no shared helper modules, no dependencies on other in-flight pilot test files. Can land independently of PRs #982, #983, or #985. * test(pilot): address PR E code-review on disconnect tests Two small findings: - Defensive: attach a no-op 'error' listener on the TcpStream in test #2 after the 'open' event has fired. teardownStream only emits 'error' for unaccepted streams today, so the listener is not exercised; the listener exists so a future refactor that delays accepted=true past 'open' cannot crash the worker on an unhandled 'error' event. - Removed a stale cross-file reference in the header comment that pointed at pilot-tunnel-integration.test.ts; that file lives in a separate in-flight branch and does not exist on main. No behavior change. All 4 disconnect tests pass. * chore(test): drop unused PROTOCOL_VERSION import The import was leftover from an earlier draft and ESLint flagged it as no-unused-vars (error level for symbols not prefixed with _), failing the lint job in CI. |
||
|
Anso
|
3112f58a88 |
fix(templates): align App Store deploy gate with stack:create permission (#986)
The POST /api/templates/deploy handler gated on requireAdmin, while every
other create-stack endpoint in routes/stacks.ts uses
requirePermission('stack:create'). Per the role table in
middleware/permissions.ts, node-admin holds stack:create — so a node-admin
could create stacks the regular way but got 403 ADMIN_REQUIRED from the
App Store. The cockpit's Deploy button is gated on can('stack:create'),
so the button looked enabled and the click silently failed.
Swap the gate to requirePermission(req, res, 'stack:create'). Admin still
passes through the global bypass; node-admin now passes via the role
permissions table; deployer, viewer, and auditor stay denied. Cache-refresh
on the same router keeps requireAdmin since cache invalidation has no
per-resource scope.
Adds backend/src/__tests__/templates-deploy-rbac.test.ts with six
parameterised supertest cases (one per role plus an unauthenticated
case) so the matrix is locked in. The two passing-role cases assert
the request clears the gate (status !== 403, code !== PERMISSION_DENIED)
without depending on Docker being available in the test environment.
|
||
|
Anso
|
8f13a7faf3 |
fix(pilot): stop silently swallowing fs errors in agent token helpers (#985)
* fix(pilot): stop silently swallowing fs errors in agent token helpers
The pilot-agent audit found that both filesystem-touching helpers in
the agent process discarded every fs error class:
- readPersistedToken caught all errors and returned null. ENOENT
(normal first boot) and EACCES / EIO / EROFS (a real disk or
volume-permission failure) were indistinguishable; the latter
silently fell back to "no persisted token", and on a node where
the enrollment token had already been consumed the agent would
enter a re-enrollment loop with no log signal pointing at the disk.
- persistToken caught all errors and logged at console.warn. The
operator saw the next-boot loop with the same diagnostic gap.
Replace both with errno-aware handling:
- readPersistedToken calls fs.readFileSync directly (no TOCTOU race
against existsSync), treats ENOENT as silent, and logs every other
errno at ERROR with the path.
- persistToken logs at ERROR (not WARN) with the failing errno and an
explicit "next agent restart will require re-enrollment until the
volume is writable" message. The agent still continues with the
in-memory token so the current session is unaffected.
Both helpers are now exported (marked @internal) so the new test file
can mock fs and assert the error-class-vs-log-level matrix.
12 unit cases in pilot-agent-fs-errors.test.ts cover ENOENT,
EACCES, EIO, EROFS, ENOSPC, missing errno, empty file, and the
happy path. Mock pattern follows backend/src/__tests__/filesystem.test.ts.
* fix(pilot): address code-review on the fs-error branch
Three findings from the review pass:
- Em dash in a test description (Directive 18). Rewritten as
"does not throw, so the in-memory token stays usable for the
current session".
- persistToken still had an existsSync + mkdirSync pair around
the token-write. mkdirSync({ recursive: true }) is idempotent on
existing directories, so the existsSync was redundant and added
a TOCTOU window where the directory could be removed between the
probe and the write. Dropped the existsSync; the test that
previously primed mockExistsSync now asserts mockExistsSync is
NOT called, locking the TOCTOU removal.
- The two new exports used the @internal JSDoc tag, but this
repo's existing pattern for "public-by-convention-for-tests"
helpers (e.g. RegistryService.ts:481) is plain prose
"Exposed for unit tests." Switched to that style.
No behavior change beyond the TOCTOU removal.
|
||
|
Anso
|
86abd901f6 |
feat(app-store): show inline port-conflict message on deploy sheet (#984)
* feat(app-store): show inline port-conflict message on deploy sheet
Replace the hover-only cursor tooltip on the Advanced tab with an inline
"in use by <stack>" message rendered next to the port input. The
pulsating warning dot stays as the live-data indicator.
Add a Port-conflict warning rail to the Essentials tab that lists each
conflicting default port and the application using it. The rail replaces
the "Deploy with defaults" hint while conflicts exist, and its left
accent pulses to echo the live-data signal.
Update docs/features/app-store.mdx to describe the new visible behavior
on both tabs.
* docs(app-store): drop docs hunk from this PR
The deploy-sheet docs are being rewritten on the docs/v1-refresh branch
(PR #966). To avoid a merge conflict between the two PRs, the prose
update for the inline port-conflict messaging now lives there as a
follow-up commit on top of
|
||
|
Anso
|
e380ee7b16 |
test(pilot): in-process integration test and Playwright E2E (#983)
* test(pilot): in-process integration test for the tunnel handshake
Layered unit tests cover the protocol decoder, the bridge cap, the
manager, and the DB-layer enrollment lifecycle. None exercise the
glue: the WebSocket dispatch order in upgradeHandler.ts, the actual
hello / enroll_ack round-trip, the long-lived token swap. A
regression that moves /api/pilot/tunnel below the auth gate, breaks
the hello / enroll_ack ordering, or changes the token shape would not
be caught today.
Spin up a real http.Server with attachUpgrade wired and a real
ws.WebSocket client. Three tests:
- Enroll-ack happy path: connect with an enrollment token, receive
hello + ctrl enroll_ack, assert the manager has registered the
tunnel.
- Replay rejection on the wire: connect, complete enroll-ack, close,
reconnect with the same enrollment token, assert HTTP 401 from the
upgrade handshake.
- Long-lived token reconnect: capture the enroll_ack token, close,
reconnect with the long-lived pilot_tunnel JWT, assert hello but
no enroll_ack and the tunnel is registered.
The test does not mock the agent side beyond the framing protocol.
The goal is to confirm the wires connect end-to-end, not to drive a
full request round-trip (which would require a synthetic agent bridge
and adds little marginal coverage over the existing bridge unit tests).
This file also establishes the in-process WS-pair test pattern for
any future WebSocket integration work; no such pattern existed in the
suite before.
* test(e2e): cover operator-side pilot-agent enrollment in Playwright
Backend integration is covered by pilot-tunnel-integration.test.ts
and the vitest enrollment suites. The browser-only surfaces (mode
selector default, enrollment dialog, docker run code block,
regenerate affordance on an existing pilot-mode node) had no
automated coverage; a typo in the mode label, a styling regression
that hid the docker-run code, or a backend response shape change
that broke the rendered command would all ship unchecked today.
Two specs reusing the existing loginAs helper and the same
Settings -> Nodes navigation pattern from nodes.spec.ts:
- Create flow: open Add Node, switch type to Remote, confirm pilot
mode is the default (api_url field stays absent), submit, assert
the enrollment dialog renders a docker run command containing
SENCHO_MODE=pilot, SENCHO_PRIMARY_URL=, and a JWT-shaped
SENCHO_ENROLL_TOKEN=. Cleanup deletes the row.
- Regenerate flow: create a pilot node, capture the first token,
open the row's edit dialog, click Regenerate enrollment token,
assert the second token differs from the first. Cleanup deletes
the row.
Out of scope for this E2E: simulating an agent connecting to flip the
row to Online. The integration test covers the wire side; this spec
keeps focus on the operator-visible UI.
Each test name-suffixes with Date.now() so re-runs do not collide on
the UNIQUE name constraint and so leftover rows from a partial run
get distinct names instead of stacking on the same row.
* fix(pilot): address PR B code-review findings
Three code-review findings:
- High: NodeManager row action buttons were icon-only with no
accessible name. Added aria-label="Edit node",
aria-label="Delete node", aria-label="Test connection" so the
Playwright E2E can target them by role/name (currently the
accessible name is rendered into a Radix tooltip portal that
Playwright cannot resolve as the button's name).
- High: Replaced UI-driven cleanup in the E2E with API-based
deletion via page.request, run in both beforeEach (sweep
leftovers) and afterEach (sweep this test's row even on failure).
Targets every node whose name starts with the test prefix so a
crashed previous run cannot affect the next.
- Medium: Replaced two fixed 50ms sleeps in the integration test
with vi.waitFor polling on hasActiveTunnel(nodeId) === false,
eliminating the race between the WS close hop chain and the next
test step on slow CI runners.
Plus two cleanup items:
- The integration test's afterAll now removes 'tunnel-up' /
'tunnel-down' listeners from the manager singleton so this
file's runs do not leak listeners into other test files in the
same Vitest worker.
- Tightened the WS error swallowing in the rejection-path test:
only swallow the expected "Unexpected server response" error
shape; surface anything else (ECONNREFUSED etc.) instead of
silently masking it.
No em dashes added (Directive 18 verified clean).
|
||
|
Anso
|
f5a52e44dc |
refactor(pilot): narrow Mesh handle, prune dead code, add replay-route test (#982)
* refactor(pilot): narrow PilotTunnelManager.getBridge to MeshTunnelHandle
The manager handed out the entire PilotTunnelBridge to its only consumer,
MeshService, which let any current or future caller reach into transport
internals: loopback URL, per-stream maps, the close API, the underscored
_writeTcpData / _closeTcpStream, the diagnostic helpers. None of that
was load-bearing for Mesh.
Introduce a MeshTunnelHandle interface alongside TcpStream in
PilotTunnelBridge.ts that exposes only the two methods Mesh actually
calls (openTcpStream and getBufferedAmount), have PilotTunnelBridge
declare implements MeshTunnelHandle, and change getBridge to return
the interface. MeshService continues to compile unchanged because its
existing call sites only touch the narrowed surface.
A future alternative transport (a stub for tests, a different routing
strategy) now has a one-method-and-a-getter contract to satisfy
instead of the full bridge.
* refactor(pilot): drop unused tunnel manager and bridge surface
Three public methods predating the hardening pass had zero callers
across the entire codebase (verified via grep across backend, frontend,
e2e):
- PilotTunnelManager.touch(nodeId): never invoked. The pilot_last_seen
timestamp is updated by the manager itself on registerTunnel and by
the persistence layer on heartbeat events.
- PilotTunnelManager.listActive(): never invoked. The metrics endpoint
added in PR #979 returns its own per-node breakdown via
getMetricsSnapshot, which is the canonical observability surface.
- PilotTunnelBridge.listTcpStreams() and the supporting
TcpStreamSummary interface: never invoked. The Mesh diagnostics
sheet that would have consumed it is not wired and would use
getMetricsSnapshot if/when it ships.
Removing them tightens the public surface and prevents accidental new
dependencies on speculative future-proofing.
* test(pilot): cover enrollment replay rejection at the route layer
The DB-level test in pilot-enrollment.test.ts already verifies that
consumePilotEnrollment is one-shot. That guards the persistence layer
but not the route handler: a refactor of handlePilotTunnel that swaps
the order of consume vs upgrade, or that grants the WebSocket
upgrade before checking the consume result, would silently break the
security invariant while DB-layer tests stay green.
Drive handlePilotTunnel directly with a stub IncomingMessage and
Duplex socket. Six cases:
- Already-consumed enrollment row -> 401.
- Token whose hash matches no row -> 401.
- Row whose expires_at has passed -> 401.
- Missing Authorization header -> 401.
- JWT signed with a wrong secret -> 401.
- pilot_tunnel JWT for an unknown node -> 404.
The stub captures HTTP/1.1 status writes so the test asserts the
exact rejection lands on the wire, not just that the function
returned without throwing.
* docs(debug): warn future committers off per-frame isDebugEnabled calls
Code-review feedback on PR #979 noted that isDebugEnabled is fine in
the cadences it has today (per-tunnel, per-request, error paths) but
is fragile against a future commit that drops it into a per-frame
WebSocket loop. The function does a try/catch + Node require cache
lookup + a method call into DatabaseService on every invocation;
acceptable at hundreds of calls per second, expensive at thousands.
Add a comment block above the function spelling out the acceptable
and unacceptable cadences and the snapshot-outside-the-loop
mitigation, so the next person to add a diag log there sees the
constraint.
* fix(pilot): address PR A code-review findings
Code review on this branch surfaced four items:
- Em-dash directive (CLAUDE.md Directive 18) violations in four
comment sites; replaced with colons or restructured.
- debug.ts perf comment claimed try/catch frame setup as the cost
driver. V8 inlines those; the load-bearing cost is the require
lookup and singleton dispatch. Reword.
- Test file header described coverage as 'replay rejection at the
route layer' but the file now also covers missing-header,
wrong-secret, expired-row, never-stored, and unknown-node
rejection paths. Widen the JSDoc and the describe label to match.
- Stub-cast comment in the replay test now explicitly lists the
IncomingMessage and Duplex surface the stub satisfies, so a
future commit that grows handlePilotTunnel's surface (rate
limiting, socket options) updates both stubs instead of
silently no-opping against them.
No behavior change.
|
||
|
Anso
|
4867234eac |
chore(resources): remove Largest 5 and Recently changed cards from Volumes tab (#981)
The two landing cards above the volumes table did not earn their space and made the Volumes view feel cluttered. Drop the cards, the unused TabLanding component, and the helpers that only fed it. |
||
|
Anso
|
0be479100c |
docs: v1 docs refresh (#966)
* docs(introduction): rewrite intro page and refresh screenshots
Rewrite the Getting Started introduction to reflect the current product:
adds the Mesh, Blueprints, Pilot, Fleet, Resources concepts up front;
restructures capability sections around Stacks, Fleet (now including
Fleet Actions), Mesh, Blueprints, Monitoring, Resources, Security,
Automation, and Pilot/Remote ops; cross-links every claim to the
matching feature page.
Replaces the dashboard hero shot with a fresh capture against the
redesigned cockpit chrome and adds three inline shots (running stack
with anatomy and logs, fleet command deck, resources hub with
reclaim header). All screenshots taken at 1920x900, dark theme,
with node names, usernames, IPs, and home paths neutralized.
Drops outdated claims: stale "190+ templates" count, the
"viewer accounts" RBAC summary, and the "atomic" deployment label
that did not match the actual rollback mechanism.
* docs(introduction): add Federation and Fleet Secrets
Federation has shipped (Admiral) with cordon and pin policy as
operator overrides on the blueprint reconciler. Fleet Secrets is
landing as a Skipper+ tab for versioned env-var bundles encrypted
at rest, with diff preview and target push.
Mirror those in the Run-one-machine-or-many bullet list and in the
Tiers paragraph so the introduction matches the current product.
* docs(introduction): re-shoot screenshots against v0.72.0 production
Re-capture all four introduction screenshots from the upgraded
production node so the fleet view shows the current full tab strip
(Overview / Snapshots / Status / Deployments / Traffic / Federation
/ Fleet Actions / Secrets) instead of the older Overview / Snapshots
/ Status only. Same 1920x900 dark-theme capture and the same PII
scrub applied (node names, usernames, IPs, home paths neutralized).
* docs(quickstart): rewrite around v0.72.0 cockpit and add screenshots
Replaces the bare install snippet with a five-minute walkthrough that
matches the redesigned UI. Leads with a docker-compose.yml block (the
bare docker run command is collapsed in an Accordion), keeps the 1:1
path rule, and adds two new sections that show the user what happens
on first boot.
Adds two screenshots at 1920x900 dark theme:
- setup-cold-start.png: the Cold start card with Username, Password,
Confirm password fields, and the Initialize console button.
- dashboard.png: the post-sign-in dashboard captured against v0.72.0
with the full top nav (Home, Fleet, Resources, App Store, Logs,
Auto-Update, Console, Audit, Schedules) and the populated Stack
health table sorted by load.
Where-to-next now uses CardGroup cols=2 to match the introduction
page's pattern.
* docs(configuration): align env var reference with current backend
Bring docs/getting-started/configuration.mdx up to date with the
v0.72.0 backend:
- Remove PORT from the optional env vars table. The listen port is
hardcoded to 1852 in backend/src/helpers/constants.ts and is never
read from the environment. Replace it with a Listen port section
that explains the fixed port and host-port remapping.
- Document API_RATE_LIMIT (default 200) and API_POLLING_RATE_LIMIT
(default 300), both applied in production only.
- Note the /app/compose fallback default for COMPOSE_DIR while still
pointing readers at the 1:1 path rule.
- Point the SSO env var section at the new SSO Quickstart page and
keep the SSO feature reference as the deeper dive.
- Tighten First boot and cross-link to Quickstart so the screenshot
is not duplicated.
- Add a Where to next CardGroup matching the refreshed Introduction
and Quickstart pages.
Drop stale PORT=1852 and JWT_SECRET=your-secure-jwt-secret-here
lines from .env.example so the example no longer contradicts the
docs (PORT is hardcoded, JWT_SECRET is auto-generated and persisted
to the database during initial setup).
* docs(configuration): replace em-dash-substitute hyphens in prose
Three sentences used ` - ` (space-hyphen-space) as an em-dash
substitute. Replaced with the punctuation that fits each case:
- "How Sencho organizes your compose directory": semicolon between
the two related clauses.
- Data directory Warning: parentheses around the parenthetical
insertion.
- 1:1 path rule explanation: comma before the contrastive clause.
Heading slugs and YAML code-block comments are unchanged.
* docs(sso): refresh setup guide and drop misleading "one-click" wording
Brings the SSO Setup Guide in line with how SSO actually works in the
current build, and corrects misleading copy across both SSO docs pages.
- Setup Guide: explains the env-var-seeds-once / DB-is-authoritative
config model up front, replacing per-section "Restart Sencho"
wording that implied a restart was always required.
- Setup Guide: promotes the per-provider Test Connection button out
of the LDAP-only paragraph into a generic intro callout, and adds
a self-signed LDAPS tip.
- Setup Guide: notes that all OIDC providers accept a *_DISPLAY_NAME
override for the login button label, and adds a commented LDAP TLS
toggle to the full compose example.
- Setup Guide: adds two screenshots of the redesigned Settings > SSO
panel (overview + LDAP card expanded with form).
- Both pages: replaces "one-click presets" / "one-click configuration"
with "preset providers". The Skipper-tier presets still require an
OAuth app provisioned in the provider's console; what they actually
buy is provider-aware defaults and a branded login button. The old
wording overpromised.
* docs(features-overview): regroup catalog and add 17 missing features
Restructure the Features Overview into the same six groups the docs
sidebar uses (Stacks & Deployments, Observability, Fleet & Multi-Node,
Security & Identity, Automation, Platform) plus a short Reference tail.
Add catalog entries for 17 shipped features that the previous overview
never mentioned: Stack Activity, Stack File Explorer, Deploy Progress,
Deploy Enforcement, Blueprints, Git Sources, Global Search, Pilot Agent,
Sencho Mesh, Fleet Federation, Fleet Actions, Fleet Sync, Fleet Secrets,
Two-Factor Authentication, CVE Suppressions, Auto-Heal Policies, Stack
Sidebar.
Fix two factual inaccuracies:
- Fleet View blurb wrongly gated search, sort, filter, and stack
drill-down behind Skipper. The deep-dive is explicit that those are
available on every tier; only the bulk Update All action inside the
Node Updates modal is paid.
- Auto-update entry was titled and described as a scheduling system.
The deep-dive page is the Auto-Update Readiness board (risk tags,
changelog previews, rollback targets); the scheduler lives under
Scheduled Operations.
Add three hero screenshots captured from production at 1920x900,
illustrating the redesigned cockpit visual language: Home dashboard,
stack anatomy, and fleet topology.
* docs(stack-management): refresh page around v0.72.0 cockpit and add screenshots
Updates the Stack Management page to match the current UI: sidebar with
filter chips and label groups, bulk mode, restructured kebab menu,
two-tab anatomy panel, and three-source New stack dialog (Empty, From
Git, From Docker Run).
Adds sections for Filter chips, Pinned and label groups, and Bulk mode.
Restructures the Stack context menu around the inspect, organize,
lifecycle, and destructive groups with their keyboard shortcuts.
Documents the From Git tab and cross-links Git Sources for the full
sync flow.
Replaces every existing screenshot with fresh captures from the current
UI and adds eight new captures for the new sections. Cross-links Stack
Activity, Stack Labels, Stack File Explorer, Compose Editor, Atomic
Deployments, Scheduled Operations, Auto-Heal Policies, Auto-Update
Policies, and Alerts and Notifications for features documented on
their own pages.
* docs(stack-activity): refresh page around v1 cockpit and recapture screenshots
Realign the page with the current Anatomy panel tab strip and the
StackActivityTimeline component:
- Frame the Activity tab as a sibling of Anatomy under the right-hand
panel, with files/edit actions belonging to the strip.
- Expand the category guidance: list the five iconized categories and
call out that other stack-scoped notifications (deploy failure,
available image updates, auto-heal triggers, monitor alerts, scan
findings) flow into the timeline with a generic icon.
- Tighten the actor-attribution rule to match the component: omitted
for events without an actor and for system-driven events.
- Add the day-format example to the relative-time row.
- Recapture both screenshots from a populated stack (Today + Yesterday
+ Earlier with three distinct icons) and an empty stack.
- Convert troubleshooting blurbs to H3 for anchor links and consistency
with the v1-refresh sibling pages.
* docs(stack-activity): wrap troubleshooting entries in Accordion blocks
Match the foldable troubleshooting pattern established by
docs/features/deploy-progress.mdx so the page stays compact and
readers can scan to their issue. This is the canonical formatting
for the /features section's troubleshooting blurbs going forward.
* docs(editor): refresh page around v1 cockpit and recapture screenshots
Rewrites the page around the dual-mode right panel (Anatomy by default,
Monaco when the user clicks edit), the redesigned Command Center action
bar, the new container row layout with status badges and live stats, the
Structured / Raw terminal logs toggle, the Git Source toolbar button, and
the opt-in diff preview. Drops the obsolete persistent embedded terminal
section. Preserves the #diff-preview-before-save and #log-viewer anchors
referenced from settings.mdx and global-observability.mdx. Replaces
editor-overview.png and container-exec-modal.png with fresh captures
against v0.72.0 production at 1920x900 dark theme; renames container
-actions.png to containers-list.png; adds command-center.png and
editor-edit-mode.png. PII scrubbed (host paths normalized) and the
compose-diff-preview/diff-modal.png shared asset is left untouched after
visual diff against the live UI showed no chrome change.
* docs(features): wrap troubleshooting accordions in AccordionGroup
Wraps the loose <Accordion> blocks in editor.mdx and stack-activity.mdx
in a single <AccordionGroup> to match the troubleshooting design used
by fleet-federation.mdx. Structural only; no content changes.
* docs(stack-file-explorer): refresh page around v1 explorer and add screenshots
Full rewrite to match the live two-pane Files tab. Replaces the false
'Edit' toolbar flow with the read-only chip + always-on Save model,
fixes the protected-files list (5 names) and dedicated-tab redirect
list (3 names), drops the fabricated 100 MB download cap, and corrects
the upload claim to single file at a time.
Adds sections for New File, Rename, Permissions, the type-to-confirm
protected-file delete flow, the non-empty folder delete confirmation,
the 500-entry tree display cap, and the symlink rendering. Restructures
troubleshooting around <AccordionGroup> + <Accordion> to match
fleet-federation.mdx, and adds two new entries (display cap, 403 on
Community admin write).
Ships nine screenshots captured against v0.72.0 production at 1920x900
in dark theme: overview, two-pane layout, protected-tree-marker,
viewer-edit-mode, new-file-dialog, context-menu-folder, context-menu-file,
permissions-dialog, delete-protected-confirm.
* docs(deploy-progress): rewrite around current modal + capture v1 screenshots
Aligns the page with the v0.72.0 implementation and standardizes the
troubleshooting layout with the rest of the docs refresh.
Setting and gating
- Renames the Settings field to "Deploy progress modal" and quotes the
current helper text verbatim. Documents that the toggle is off by
default, lives under Settings > Appearance > Display, is stored in
localStorage, and syncs across tabs in the same browser.
Modal anatomy
- Names every visible UI string: header verbs, status indicator (with
the "closes in <n>s" countdown that was previously undocumented),
empty-body strings, footer toggle that flips between "Raw output" and
"Hide raw", and the destructive border on ERR rows vs the softer warn
tint on WARN rows.
- Replaces the vague "after a few seconds" with the actual 4-second
auto-close timer; documents hover-to-pause and the
leave-hover-restarts-the-countdown behavior.
- Documents the truncated error message in the failed-state header and
the manual close-only requirement.
- Notes that the pill is portal-mounted and survives navigation.
Stage badges
- Keeps the 9-badge table but adds an honest note that most lines render
as LOG because the badges are gated on Compose's "[+]" progress
prefix, which Compose only emits in TTY mode and Sencho spawns it
without one.
Entry points
- Splits the supported actions into the four that produce a populated
structured-log body (Deploy, Update, Install, Git Apply) and the two
that bypass compose and finish with 0 lines (Restart, Stop). Drops the
"Down" claim from the user-facing list since no UI control currently
triggers it; mentions the down route as an automation surface only.
Troubleshooting
- Wraps the existing accordions in an <AccordionGroup> matching the
pattern used by the editor and stack-activity refreshes. Adds two new
entries: one explaining the Restart/Stop "0 lines" outcome, one
explaining the LOG-everywhere case for non-TTY compose output.
Screenshots (six PNGs in docs/images/deploy-progress/, 1920x900, dark
theme, captured against the upgraded production node)
- setting-toggle.png: the Display section with the toggle enabled.
- modal-streaming.png: a real update in flight at 19s, 554 lines.
- modal-succeeded.png: succeeded state with the live closes-in
countdown visible.
- modal-raw-output.png: structured rows with the Raw output panel
expanded beneath.
- pill.png: minimized pill anchored bottom-center on a stack editor
view.
- modal-failed.png: failed state with the truncated error in the
header and ERR rows highlighted.
* docs(resources): refresh Resources Hub page for v1 redesign and feature additions
Rewrite the page to match the shipping UI and replace stale screenshots
with fresh captures of the redesigned chrome.
- Document the admin-only Reclaim hero and clarify the per-tile Sencho-only
vs. All Docker (includes external) split in Quick Clean.
- Add coverage of the Scan history toolbar button and the per-row severity
badge plus shield-icon scan dropdown in the Images tab; cross-link to the
vulnerability scanning page.
- Correct the Volumes column list (no Size column; size lives on the Largest
5 landing card) and call out admin gates on browse and delete.
- Spell out the List/Topology view-mode toggle and that Create Network is
admin-only and List-mode only.
- Rewrite the Unmanaged tab section around the project-grouped layout, the
Select all + Purge Selected (N) admin-only multi-select, and the empty
state copy.
- Replace screenshots: resources-reclaim, networks-list, create-network,
network-inspect, network-topology, network-topology-toggle. Add fresh
resources-volumes-tab and resources-unmanaged-tab captures.
* docs(app-store): refresh page around v1 deploy sheet, scan integration, and registry settings
Rewrites the App Store reference to match the current cockpit:
- Documents the weekly-rotated featured banner picked from the top-5 by GitHub stars and the star-descending grid sort.
- Adds the deploy-sheet structure (breadcrumb, meta line, About panel with Read more) and splits the Advanced tab into Ports, Volumes, Environment variables, Custom variables, and Security subsections.
- Documents the Trivy-gated Security checkbox, atomic vs non-atomic deploys by tier, and the rollback semantics driven by error class.
- Adds a Watching the deploy section linking to the deploy-progress modal.
- Rewrites the Custom registry section against the new two-panel settings layout (Default + Custom) with the URL validation rule and the using default / using custom hint.
- Adds a four-entry Accordion troubleshooting block in the house style.
- Replaces three screenshots and adds two (Advanced tab, Settings registry panel) captured against the production node.
Permissions wording aligns with current backend (admin only); the broader stack:create gate will land in a follow-up fix branch.
* docs(app-store): describe inline port-conflict messaging on deploy sheet
Update the deploy-sheet section to match the visible port-conflict
behavior: the Essentials tab surfaces a Port-conflict warning that
replaces the defaults hint when any default port is already bound, and
the Advanced tab shows an inline "in use by {stack}" message next to
the container port instead of a hover-only tooltip. Refresh the
screenshot alt-text and the troubleshooting Accordion to match.
* docs(app-store): align permissions note with stack:create gate
Pairs with the backend gate swap in fix/templates-deploy-rbac (#986)
which moves POST /api/templates/deploy from requireAdmin to
requirePermission('stack:create'). Updates the Note block under
'Watching the deploy' so the docs match the new behavior: admin and
node-admin can deploy templates from the App Store; viewer, deployer,
and auditor cannot.
|
||
|
dependabot[bot]
|
8cbeb8708e |
chore(deps): bump the all-npm-backend group across 1 directory with 8 updates (#943)
Bumps the all-npm-backend group with 8 updates in the /backend directory: | Package | From | To | | --- | --- | --- | | [axios](https://github.com/axios/axios) | `1.15.2` | `1.16.0` | | [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware) | `3.0.5` | `4.0.0` | | [yaml](https://github.com/eemeli/yaml) | `2.8.3` | `2.8.4` | | [zod](https://github.com/colinhacks/zod) | `4.3.6` | `4.4.3` | | [eslint](https://github.com/eslint/eslint) | `10.2.1` | `10.3.0` | | [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) | `8.59.1` | `8.59.2` | | [@aws-sdk/client-ecr](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-ecr) | `3.1038.0` | `3.1043.0` | | [@aws-sdk/client-s3](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-s3) | `3.1038.0` | `3.1043.0` | Updates `axios` from 1.15.2 to 1.16.0 - [Release notes](https://github.com/axios/axios/releases) - [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md) - [Commits](https://github.com/axios/axios/compare/v1.15.2...v1.16.0) Updates `http-proxy-middleware` from 3.0.5 to 4.0.0 - [Release notes](https://github.com/chimurai/http-proxy-middleware/releases) - [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/master/CHANGELOG.md) - [Commits](https://github.com/chimurai/http-proxy-middleware/compare/v3.0.5...v4.0.0) Updates `yaml` from 2.8.3 to 2.8.4 - [Release notes](https://github.com/eemeli/yaml/releases) - [Commits](https://github.com/eemeli/yaml/compare/v2.8.3...v2.8.4) Updates `zod` from 4.3.6 to 4.4.3 - [Release notes](https://github.com/colinhacks/zod/releases) - [Commits](https://github.com/colinhacks/zod/compare/v4.3.6...v4.4.3) Updates `eslint` from 10.2.1 to 10.3.0 - [Release notes](https://github.com/eslint/eslint/releases) - [Commits](https://github.com/eslint/eslint/compare/v10.2.1...v10.3.0) Updates `typescript-eslint` from 8.59.1 to 8.59.2 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.59.2/packages/typescript-eslint) Updates `@aws-sdk/client-ecr` from 3.1038.0 to 3.1043.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-ecr/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1043.0/clients/client-ecr) Updates `@aws-sdk/client-s3` from 3.1038.0 to 3.1043.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-s3/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1043.0/clients/client-s3) --- updated-dependencies: - dependency-name: axios dependency-version: 1.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-npm-backend - dependency-name: http-proxy-middleware dependency-version: 4.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: all-npm-backend - dependency-name: yaml dependency-version: 2.8.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all-npm-backend - dependency-name: zod dependency-version: 4.4.3 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-npm-backend - dependency-name: eslint dependency-version: 10.3.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: all-npm-backend - dependency-name: typescript-eslint dependency-version: 8.59.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: all-npm-backend - dependency-name: "@aws-sdk/client-ecr" dependency-version: 3.1043.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-npm-backend - dependency-name: "@aws-sdk/client-s3" dependency-version: 3.1043.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all-npm-backend ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anso <dev@saelix.com> |
||
|
Anso
|
3d9489648e |
fix(pilot): harden outbound reverse-tunnel against resource exhaustion (#979)
* fix(pilot): cap tunnel frame size, concurrent streams, and stream idle time
Pilot tunnels carry every HTTP, WS, and Mesh-TCP byte for a remote node
through a single multiplexed WebSocket. A buggy or compromised peer that
sent oversized frames, opened streams in a tight loop, or left streams
parked indefinitely could exhaust gateway memory.
Three protocol-level limits applied symmetrically on both ends:
- MAX_FRAME_SIZE_BYTES (8 MB): set as the ws maxPayload on the
gateway-side WebSocketServer and the agent-side WebSocket client,
plus a defense-in-depth length check in decodeBinaryFrame and
decodeJsonFrame.
- MAX_STREAMS_PER_TUNNEL (1024): bridge refuses new loopback HTTP /
upgrade / TCP allocations with 503 once at the cap; the agent
rejects new incoming http_req / ws_open / tcp_open with the
appropriate error frame.
- STREAM_IDLE_TIMEOUT_MS (10 min): every stream gets a per-stream
timer refreshed on each inbound or outbound activity. Expiry tears
down the local half and notifies the peer.
The constants are colocated in pilot/protocol.ts so any future agent
build picks them up via the protocol module.
* fix(pilot): rate-limit pilot enrollment endpoints to 10 per minute
Pilot enrollment mints a JWT and writes a pilot_enrollments row, both
privileged operations that should not share the global API budget
(200/min). Add a dedicated express-rate-limit instance keyed by user or
IP at 10/min in production (100/min in dev so the local enrollment
test loop is not throttled).
The limiter is applied directly on POST /api/nodes/:id/pilot/enroll.
On POST /api/nodes the limiter's skip function reads the parsed body
and exempts proxy-mode creates; only requests that resolve to
mode=pilot_agent count against the enrollment budget.
* fix(pilot): cap concurrent pilot tunnels system-wide
The PilotTunnelManager held an unbounded Map of bridges. A reconnect
storm, runaway enrollment, or operator misconfiguration could grow the
map without limit, even though every tunnel still carries a valid JWT.
Cap at 256 concurrent tunnels per primary instance:
- Soft warning at 128 (logged once per crossing).
- Hard refusal at 256: registerTunnel throws PilotTunnelCapacityError
and the upgrade handler closes the WebSocket with 1013 (Try Again
Later) so the agent backs off rather than tight-looping.
A node that already has a registered tunnel does not consume a new
slot when it reconnects; the existing bridge is closed first.
* fix(pilot): release backpressure as soon as the tunnel buffer drains
Previously the bridge paused HTTP request bodies when the tunnel
WebSocket's bufferedAmount climbed above 4 MB but never resumed them
explicitly; the next data event re-checked the threshold, and TCP
streams only saw a 'drain' fan-out on the 30 s ping cycle. Slow-consumer
peers held buffered bytes for tens of seconds longer than needed.
Replace both with an on-demand drain check: when at least one stream
is paused, sample bufferedAmount every 100 ms; once it drops below the
high-water mark, resume every paused request and emit 'drain' to every
accepted TCP stream, then stop the timer. Dormant when no stream is
paused, so steady-state cost is zero.
* fix(pilot): trust internal CAs via SENCHO_PILOT_CA_FILE
Self-hosted deployments often terminate TLS with an internal CA. The
only previous escape hatch was NODE_TLS_REJECT_UNAUTHORIZED=0, which
disables verification across every outbound connection in the agent
process and is the wrong shape of fix.
Add SENCHO_PILOT_CA_FILE: when set, the agent reads the file as a PEM
bundle and passes it to ws as the `ca` option for the tunnel
WebSocket. rejectUnauthorized remains true. Failure to read the file
exits with a clear error so the operator does not silently fall back
to the default trust store.
There is no flag to disable TLS verification entirely; that would
defeat the credential trust model.
* fix(pilot): gate frame-rate-adjacent logs behind developer_mode
The agent's malformed-frame warning fires per inbound frame and could
flood logs under attack or against a buggy primary. The bridge swallowed
parse errors silently, leaving operators blind to protocol drift, and
the manager had no signal at all on tunnel registration.
Route per-frame and per-tunnel diagnostic logs through the existing
isDebugEnabled() helper (same pattern used by middleware/auth, RBAC,
and websocket/logs). Production stays quiet by default; the toggle is
the existing developer_mode setting in global_settings — no new env
var, no new dependency.
* feat(pilot): expose per-tunnel metrics via /api/system/pilot-tunnels
Operator support had no signal beyond raw logs to answer "is this
tunnel flapping?" or "is one bad node hiding behind the aggregate?".
Add an in-process counter set covering tunnels_total, tunnels_replaced,
tunnels_rejected_capacity, enroll_acks, frame_decode_errors, plus a
per-node array carrying the connectedAt and bufferedAmount so a single
tunnel sitting on a stuck buffer remains visible.
Counters live in services/PilotMetrics.ts; the gateway has no shared
in-process metrics facility today, so this is a per-feature pattern
documented as such for future consolidation. Counters are strictly
process-local — no telemetry, no export, no phone-home — consistent
with Sencho's privacy posture.
Surfaced read-only via GET /api/system/pilot-tunnels behind admin auth,
mirroring the existing cache-stats endpoint.
* fix(pilot): tighten reconnect backoff and log handling
Three small follow-ups to the hardening pass:
- Sanitize the primary URL on the agent's connect log so a malicious
SENCHO_PRIMARY_URL with embedded control characters cannot inject
fake log lines.
- Move the reconnect-backoff reset from the 'open' callback to the
'hello' frame handler. A peer that always rejects the handshake
(incompatible version, consumed enrollment token) used to reset
the backoff on every TCP-level connect and tight-loop reconnects;
now the reset waits for a clean protocol round-trip.
- Document the StreamIdAllocator wrap behavior so future readers
can see the relationship between the 2^31 wrap and the
MAX_STREAMS_PER_TUNNEL cap without re-deriving it.
* test(pilot): add coverage for enrollment, rate limiter, and tunnel caps
Adds two test files exercising the hardening pass surfaces that were
previously uncovered:
- pilot-enrollment.test.ts: end-to-end through POST /api/nodes
(pilot mode), POST /api/nodes/:id/pilot/enroll, the SHA256
token-hash persistence, replay protection, expired enrollments,
and the rate-limit header wiring (enrollment limiter on pilot
paths, global limiter elsewhere).
- pilot-bridge-limits.test.ts: oversize-frame rejection at both
binary and JSON decoders, and the per-tunnel concurrent stream
cap as observed via openTcpStream returning null and the loopback
HTTP server returning 503 once the bridge is at capacity.
Bridge cap test fills the slot map with TCP stream handles rather than
real HTTP requests so the OS socket pool stays out of the picture.
* test(pilot): cover manager metrics snapshot and capacity-error shape
Adds direct coverage for:
- PilotTunnelCapacityError exposing the limit so the upgrade
handler can format a useful close-frame reason.
- PilotMetrics.snapshot returning a defensive copy (callers must not
mutate the live counter set).
- getMetricsSnapshot returning the open count, per-node breakdown,
and counter set in a stable shape, with tunnels_total bumping on
a successful registerTunnel.
The manager test seeds a real pilot-mode node row first because
updateNode throws on missing rows; this models the production path
where a node is created before its enrollment is consumed.
* docs(pilot): document tunnel limits, custom CA, and new failure modes
Refresh the Pilot Agent feature doc to cover the hardening surfaces:
- New 'Self-signed primary TLS certs' section walking through the
SENCHO_PILOT_CA_FILE env var with an example docker run command.
- New 'Resource limits' section listing the per-tunnel and
system-wide ceilings so operators know what the wire enforces.
- Four new troubleshooting entries: WebSocket close 1013 (system
cap), loopback 503 (stream cap), close 1002 with protocol error
(frame size / malformed), and HTTP 429 on enrollment.
Also expand .env.example with a new pilot-agent block covering
SENCHO_MODE, SENCHO_PRIMARY_URL, SENCHO_ENROLL_TOKEN, and
SENCHO_PILOT_CA_FILE so operators do not have to read code or the
feature doc to discover the agent-side config surface.
* fix(pilot): address code-review findings on the hardening pass
Critical:
- Bridge: TCP backpressure now starts the drain timer and tracks
streams awaiting drain, so a TcpStream caller waiting on 'drain'
no longer hangs when no HTTP request is in flight.
- Bridge: every direct streams.delete call now goes through the
removeStream helper so per-stream idle timers are cleared
consistently and pausedReqs / tcpAwaitingDrain stay aligned.
- Bridge close(): resume any paused IncomingMessage before clearing
the map so a parser does not stay stuck across teardown.
High:
- Protocol: decodeJsonFrame now compares Buffer.byteLength(raw,
'utf8') against the cap; raw.length (UTF-16 code units) let
multi-byte payloads sneak ~3x the byte budget through.
- Agent: the optional CA bundle is read once at construction and
cached on the instance, so a missing or rotated SENCHO_PILOT_CA_FILE
no longer process-exits on every reconnect attempt.
Medium / Low:
- Bridge limits test: assert mockWs.sent.length to confirm slot
allocations actually serialized a tcp_open frame and that a
rejected (cap+1) attempt did not consume a stream id.
- Manager: defer the tunnels_replaced increment until after the
cap check passes, so a rejected reconnect does not double-count
as both replacement and capacity rejection.
- Manager: clarify the protocol comment about MAX_FRAME_SIZE_BYTES
enforcement layering (ws maxPayload is authoritative; decoder
check is for tests and defense-in-depth).
- Limiter: defensive `if (!req.body) return false` so a future
refactor that delays body parsing cannot silently skip the
enrollment limiter.
Deferred (documented as follow-ups, not in this branch):
- M2: narrowing public surface of PilotTunnelManager.getBridge to a
MeshTunnelHandle interface (cross-cutting refactor).
- H4: end-to-end replay test driving the upgrade handler twice
with the same enrollment JWT (needs WS test harness).
|
||
|
Anso
|
fe3bc3e9c3 |
docs: refresh GitHub README for 1.0 launch (#978)
New hero with light and dark logo via <picture>, fresh production screenshots (dashboard hero plus four-image gallery covering stacks, editor, fleet, and logs), tighter capability list grouped by surface, compose-first quick start with TLS reverse-proxy callout, dedicated remote-nodes section, and a combined documentation/community/license footer. Also adds .github/FUNDING.yml pointing at the Buy Me a Coffee page so the Sponsor button surfaces on the repo page. |
||
|
sencho-quartermaster[bot]
|
1d7d2b255e |
chore(main): release 0.73.0 (#975)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
SaelixCode
|
887d8fb8f0 |
feat(security): audit-hardening pass for secret and misconfiguration scanning (#977)
Releases the audit-hardening pass merged in #977 with a conventional commit subject so release-please picks it up. The squash-merge of #977 used a bare descriptive subject ("Audit-hardening pass for..."), which release-please skips by design. Body of #977 already lists every individual change including the new misconfig acknowledgement system, scan dedup, tmp-dir sweep, SARIF cap, and tests. |
||
|
SaelixCode
|
060bc300ac |
feat(security): audit-hardening pass for fleet-replicated CVE suppressions (#976)
Releases the audit-hardening pass merged in #976 with a conventional commit subject so release-please picks it up. The squash-merge of #976 used a bare descriptive subject ("Audit-hardening pass for..."), which release-please skips by design. Body of #976 already lists every individual change. |
||
|
Anso
|
3b650523c1 |
Audit-hardening pass for secret and misconfiguration scanning (#977)
* fix(security): dedupe concurrent compose-stack scans
Track stack scans in scanningImages keyed stack:<nodeId>:<stackName>.
The /scan/stack route returns 409 when an in-flight scan exists, and
the service-side check is the real correctness barrier (the route
pre-check is a fast-path optimization that mirrors scanImage). The
dedup key release lives in a try/finally so failed scans free the
slot for retry.
Why: scanComposeStack had no equivalent of scanImage's scanningImages
guard, so two simultaneous calls for the same stack would both run
trivy config, both insert a vulnerability_scans row, and double-
process the result.
* feat(security): acknowledge misconfig findings
Adds a parallel acknowledgement system for Trivy misconfig findings
that mirrors cve_suppressions: a new misconfig_acknowledgements table,
read-time enrichment via the new misconfig-ack-filter utility, REST
CRUD endpoints, fleet-sync replication from control to replicas, a
Settings panel, and an Acknowledge button on the Misconfigs tab.
Schema and behavior parity with cve_suppressions:
- UNIQUE(rule_id, COALESCE(stack_pattern, '')) so fleet-wide acks
collide as expected
- blockIfReplica on every write
- Audit-log entries name the scope (rule_id, stack_pattern) but
never the reason text
- replicated_from_control flag controls UI delete affordance and
drives clearReplicatedRows on demote/reanchor
- Validators reused: validateStackPatternForRedos for glob safety,
sanitizeForLog for log fragments
SARIF export emits an external/accepted suppression entry per
acknowledged misconfig, matching the CVE pattern.
Per-row Acknowledge dialog prefills stack_pattern with the scan's
stack_context so the default scope is "rule + this stack only" and an
operator must broaden explicitly.
Tests: misconfig-ack-filter (15) and misconfig-ack-routes (23)
including the duplicate-409 case for both pinned and fleet-wide acks.
* fix(security): reap orphaned trivy tmp dirs at startup
When the buildEnv path writes a per-scan DOCKER_CONFIG dir under
os.tmpdir() and the process crashes before the finally block runs,
the dir leaks. Mirrors GitSourceService.sweepStaleTempDirs:
exported sweepStaleTrivyTempDirs is fire-and-forget at boot,
removes prefix-matching dirs older than 1 hour, swallows
permission/race failures, logs a single line if any were reaped.
* perf(security): emit per-batch summary for scanAllNodeImages
Adds one diag() line at the end of scanAllNodeImages summarising
unique image count, scanned, skipped, failed, violation count, and
elapsed time. Per-image diag inside scanImage stays useful for
debugging individual scans; the summary gives operators a single
fleet-level checkpoint when developer_mode is on.
* perf(security): cap SARIF export at 5000 findings per type
Replace the unbounded fetchAllPages walk on /scans/:id/sarif with a
hard limit of 5000 findings per type. When any type trips the cap,
emit run-level properties.truncated=true plus row_limit and per-type
totals so downstream tooling can flag the export as partial.
Console-warns for ops visibility.
A scan with 50k vulns previously streamed every row into memory
before serialising; the cap bounds memory and serialisation time at
the cost of completeness on pathological scans.
* docs(env): document TRIVY_BIN host-binary override
The env var is honored by TrivyService.detectTrivy as a fallback when
no managed install is present, but it was undocumented in
.env.example. Adds the var with a comment explaining precedence
(managed > TRIVY_BIN > PATH).
* test(security): cover scanComposeStack failure modes
Two new cases drive the existing try/catch through real failure
paths:
- Malformed Trivy stdout: row flips to status='failed' with the
parser error preserved on `error`.
- execFile rejection: row flips to status='failed' with a string
error message.
Pairs with the existing dedup tests so the failure path now also
verifies the scan row state, not just the thrown exception.
* test(e2e): security scanner + misconfig acknowledgement flow
Seven Playwright tests covering the scanner UI and the new
acknowledgement system end-to-end:
- Trivy availability gate (skips suite when binary absent so CI
without Trivy can opt out via E2E_SKIP_TRIVY=1)
- Stack config scan completes and records misconfig findings
- Concurrent stack scan returns 409 from the dedup gate
- Misconfig ack POST creates and lists on Settings
- Duplicate (rule_id, stack_pattern) returns 409
- Malformed rule_id (shell metacharacters) returns 400
- Misconfigs tab renders against a real stack scan
Tests drive the API for behaviour assertions and the UI only for
shell-rendering checks; the visual snapshot suite owns screenshots.
* docs(features): add misconfig acknowledgement workflow and SARIF cap
Refreshes vulnerability-scanning.mdx with:
- Misconfig acknowledgements section covering the per-row dialog,
Settings panel, scope/matching rules, and SARIF emission
- Tier table row for the new feature
- SARIF section note on the 5000 row-per-type cap and the
properties.truncated marker for partial exports
- Troubleshooting entries: SARIF cap, hidden Acknowledge button,
findings resurfacing after delete, Trivy DB phone-home, and
409 on concurrent compose-stack scans
* fix(ci): clear backend lint and CodeQL alerts
- Remove the dead fetchAllPages helper in routes/security.ts. It lost
its callers when the SARIF endpoint switched to direct paged reads
for the truncation cap. ESLint flagged it as unused.
- Switch the trivy-tmp-cleanup test helper to fs.mkdtempSync. Building
paths under os.tmpdir() with predictable names tripped CodeQL's
js/insecure-temporary-file rule (high severity), which warns about
symlink-pre-creation attacks even in test code. mkdtempSync appends
a process-random suffix and creates the dir atomically; the
sencho-trivy- prefix is preserved so the production sweep still
matches the test fixtures.
|
||
|
Anso
|
4b1de35dda |
Audit-hardening pass for fleet-replicated CVE suppressions (#976)
* perf(security): bucket CVE suppressions by id at read time
applySuppressions now builds a Map<cve_id, suppression[]> once before the
per-finding loop, dropping per-finding work to O(matching-cve-suppressions)
rather than O(all-suppressions). At a fleet-wide cap of 10000 rows against
a multi-thousand-finding scan, the prior linear-per-finding shape drifted
into tens of millions of comparisons per render.
Public API of findSuppression and applySuppressions is unchanged.
Specificity scoring, expiry handling, and image-glob matching are
preserved bit-for-bit. Adds a regression guard that pins a 10000x2000
workload under 1.5s and asserts at least one match was actually returned.
* feat(security): record audit-log entries on control-side CVE suppression CRUD
The replica receive path already wrote an audit-log entry on apply; the
control-side POST/PUT/DELETE handlers did not. Operators reading the
audit panel saw mirrored security-rule changes from the replica view but
could not see who originated them on the control. Symmetric logging
closes that gap.
The summary records the CVE id and pinned scope (pkg, image) but never
the suppression's reason text. Reasons are free-form admin input that
replicate fleet-wide and may carry incident-tracker IDs or vendor
context the operator did not intend to broadcast.
The summary is also sanitised before emission so an operator-supplied
package name or image pattern carrying a smuggled newline plus a forged
"cve_suppression.delete:" prefix cannot inject a fake row into the audit
panel. Control characters become "?" and the field is capped to its
validator length.
Adds a Control-side audit log block to suppression-routes.test.ts that
asserts the privacy contract on each verb (scope present, reason absent),
plus a log-injection guard test, plus a regression that GET still works
when the local instance is a replica.
* test(security): cover cve_suppressions fleet-sync receive path
The existing fleet-sync route tests covered the protocol mechanics
(auth, anchor, stale push, reanchor, demote) for cve_suppressions only
with empty-rows payloads. The suppression-specific concerns were
unverified end-to-end:
- actual rows replace prior replicated rows on a fresh push
- the receive path writes an audit-log entry naming the source
fingerprint and row count
- a malformed suppression row is rejected at the validator before any
DB write
Adds a focused block exercising those three properties against the real
Express app, real SQLite, and the real apply transaction.
* docs(features): refresh CVE suppressions troubleshooting and field guidance
Converts the troubleshooting section to Mintlify Accordion blocks so
each entry is foldable and the page stays scannable. Adds entries for:
- control-identity mismatch on a replica (anchor-aware reanchor flow)
- mirrored rules persisting after a control demote
- the 10000-row truncation cap on a fleet sync push
Adds a privacy note to the Reason field guidance: do not paste
credentials, tokens, or vendor secrets there, since the field replicates
fleet-wide and surfaces on every node's suppressions panel.
|
||
|
Anso
|
d8e8d500c9 |
docs(fleet-sync): refresh hardening, anchor, retry, demote behaviors (#974)
The fleet-sync.mdx page reflected an earlier shape of the feature. Refreshed to cover everything the receiver and sender now do: - Both scan policies AND CVE suppressions replicate over the same channel today (was previously framed as "future" for suppressions). - Control anchor: replicas bind to the first control fingerprint and reject pushes from a different control until an admin reanchors. Documented the reanchor curl call. - Push ordering: monotonic pushedAt rejects strictly-older pushes with 409 STALE_SYNC_PUSH; legacy controls without timestamps still accepted for back-compat. - Automatic retry: the control retries failed pushes every 5 minutes for 24h and emits one warning notification per hour-long failure window for previously-working remotes. - Demote to control: documented the admin-only button and what it wipes. Replaced the "remove the node from the control" workaround with the proper flow. - Pilot-agent nodes: explicitly documented as out of fleet-sync scope. - Tiebreaker: documented lowest-id wins for ties. - Replica policy filtering: identity-scoped policies for other replicas no longer surface in this replica's UI. - Troubleshooting expanded with 409 codes (STALE_SYNC_PUSH, CONTROL_IDENTITY_MISMATCH). Cross-link added from vulnerability-scanning.mdx (Scan policies section) so anyone reading about policies finds the replication docs. |
||
|
Anso
|
a284732a95 |
feat(fleet-sync): hide other replicas' identity-scoped policies on a replica (#973)
GET /api/security/policies on a replica now returns only the policies
that apply to THIS replica. Replicated rows with a node_identity
targeting a sibling replica are filtered out so an operator cannot
enumerate the names and rules of policies meant for another node in
the fleet. Defense in depth: the security panel is admin-only, but a
backend filter is bypass-proof and matches Sencho's privacy posture.
Internal evaluators (getMatchingPolicy, evaluateScanAgainstPolicies)
keep using the unfiltered list because they already enforce identity
matching at evaluation time.
CVE suppressions are fleet-wide on every replica (no node_identity
column) so no analogous filter is needed.
Public surface:
- DatabaseService.getScanPoliciesForUi(role, selfIdentity): the
filtered variant, called from securityRouter.get('/policies').
Tests:
- 3 new vitest cases: control sees full set; replica hides
other-replica scoped rows; replica always includes locally created
rows.
- Full backend suite: 1795 pass / 5 skipped.
|
||
|
Anso
|
4007709590 |
fix(fleet-sync): hygiene pass on receiver behavior and cleanup (#972)
A bundle of small file-local fixes to the receiver path and node-deletion flow. Changes: - F4 receiver audit log: applyIncomingSync now writes a system audit entry on every applied push so mirrored security-rule changes show up in the replica's audit panel with a clear control-side origin. - F7 pilot-agent skip: pushResource explicitly excludes pilot-agent nodes (they have no api_url for HTTP push) and warns once per node id so the operator sees they will not receive replicated policies. - B4 identity-drift notification: when targetIdentity differs from the cached fleet_self_identity, dispatch a warning so the operator can audit any identity-scoped policies that may need re-targeting. - B6 stack_pattern ReDoS guard: reject patterns with 4+ consecutive wildcards or more than 8 wildcards total. Both control-side validators (POST/PUT scan policies) and the receiver-side row validator share the helper. - B9 deleteNode cascade: clear fleet_sync_status rows for the node inside the existing transaction so the sync-status panel does not render ghost entries after a node is removed. - S6 last_error redaction: formatError strips Bearer tokens and JWT-shaped values from error messages and caps at 500 chars before storing in fleet_sync_status.last_error or logging. Tests: - 8 new vitest cases covering audit-log entry, identity-drift alert, pilot-agent warn-once, formatError redaction (Bearer + JWT), ReDoS validator rejection, and a backtracking-time smoke test. - New database-fleet-sync-cascade.test.ts: deleteNode removes fleet_sync_status rows for the deleted node and leaves siblings untouched. - Full backend suite: 1792 pass / 5 skipped. |
||
|
Anso
|
f8c75aa6cd |
fix(fleet-sync): clear stale policy_evaluation on replica sync swap (#971)
Replicated scan policies always insert with fresh ids on the replica side. Any vulnerability_scans.policy_evaluation row referencing the prior set was instantly stale the moment the swap completed, so the replica's UI would surface violations from a policy that no longer exists. replaceReplicatedScanPolicies now calls clearOrphanPolicyEvaluations() inside the same transaction as the delete + inserts, so the cleanup commits atomically with the row swap. Local-only policies and their cached evaluations remain untouched. CVE suppressions do not have an analogous cache (suppressions are applied at read time, never persisted on scan rows), so replaceReplicatedCveSuppressions does not need a sibling call. Tests: - database-replicated-policies.test.ts: stale policy_evaluation cleared after a swap; local-policy evaluation preserved across swaps. - Full backend suite: 1783 pass / 5 skipped. |
||
|
Anso
|
33b15d6cba |
feat(fleet-sync): retry failed pushes and backfill on add-node (#970)
A control instance now retries fleet-sync pushes that hit a transient failure and backfills the security state on a freshly registered remote without waiting for the next policy edit. New service: - FleetSyncRetryService (singleton, start/stop) wakes 30s after boot and ticks every 5min. For each fleet resource, queries getFailedSyncTargets within a 24h window and re-pushes via FleetSyncService.pushResourceToNode through the same per-node mutex, so a normal fanout in flight serializes naturally with a retry. - After STALE_THRESHOLD_MS (1h) of continuous failure for a previously-working node, dispatches a single warning notification per cooldown window. Brand-new nodes that have never succeeded do not alert via this path; misconfigured remotes are caught by the test-connection affordance at registration time. - Wired into bootstrap startup/shutdown next to AutoHealService. Public surface: - FleetSyncService.pushResourceToNode(node, resource): targeted push to one node that re-uses the per-node mutex. Used by the retry service and any future targeted-resync flow. - routes/nodes.ts POST /api/nodes fires pushResourceAsync for both resources after a remote-proxy node row commits. Tuning constants centralized in fleetSyncConstants.ts: - RETRY_MAX_AGE_MS = 24h - STALE_THRESHOLD_MS = 1h Tests: - 8 vitest cases covering replica skip, retry dispatch, missing-node skip, alert-once-per-cooldown across the threshold window, no-alert for recent failures, no-alert for brand-new never-succeeded nodes, no-alert when the retry itself succeeds, start/stop idempotency. - Full backend suite: 1781 pass / 5 skipped. |
||
|
Anso
|
7dde257e1f |
feat(fleet-sync): replica self-demote endpoint and role UX (#969)
A replica admin can now demote the instance back to a standalone
control without raw SQLite access. The Settings → Security UI surfaces
a confirm-gated button when the role is replica; the role probe also
surfaces a soft banner when it cannot determine fleet role rather
than silently defaulting to control.
Backend:
- POST /api/fleet/role/demote (admin, requires `{confirm: true}`):
flips fleet_role to 'control', clears fleet_self_identity,
fleet_control_identity, and both received_pushed_at:* watermarks,
drops every replicated_from_control row from scan_policies and
cve_suppressions, nulls out any orphaned policy_evaluation cache.
Returns 409 ALREADY_CONTROL when invoked on a control.
- DatabaseService gains `clearOrphanPolicyEvaluations()` and
`clearReplicatedRows()` helpers. Reanchor consolidates onto
clearReplicatedRows so it shares the same code path.
- `FleetSyncService.demote()` returns boolean for the route to
translate into 200 or 409.
Frontend:
- SecuritySection probes /fleet/role and now records explicit success
vs failure rather than silently treating an error as control. A
soft banner appears when probe fails.
- Replica banner gains a "Demote to control" button and a destructive
ConfirmModal explaining the wipe.
Tests:
- 4 new route-level vitest cases (401, 400 without confirm,
end-to-end demote with replica setup, 409 ALREADY_CONTROL with
explicit precondition).
- Service unit test asserts the consolidated clearReplicatedRows path.
- Full backend suite: 1773 pass / 5 skipped. Frontend: 185 pass.
|
||
|
Anso
|
f3757b43c6 |
feat(fleet-sync): anchor replicas to a control fingerprint (#968)
A replica now binds to the first control that pushes to it. Subsequent
pushes from a different control are rejected with 409
CONTROL_IDENTITY_MISMATCH until an admin explicitly reanchors. Closes
the cross-control hijack window where any node_proxy bearer signed
against the replica's secret could overwrite security policies.
Wire protocol:
- Sender includes a stable 16-hex-char `controlIdentity` derived by
SHA-256-truncating `system_state.instance_id` (the local UUID written
once by LicenseService.initialize on first boot). Hostname rotations
do not flag drift; only a SQLite reset or explicit reanchor breaks
the binding.
- Receiver caches the fingerprint inside the same transaction that
applies the row replacement and watermark write. Three states:
null (fresh install), '' (post-reanchor), '<fingerprint>' (anchored).
- Empty `controlIdentity` is treated as legacy and accepted, so older
controls keep working during rollout.
New endpoint:
- POST /api/fleet/role/reanchor (admin, requires `{override: true}`):
clears the cached fingerprint, both `received_pushed_at:*` watermarks,
and replicated rows of both resources, all in one transaction. The
static cached fingerprint is also flushed defensively.
Public surface additions:
- `FleetSyncService.getControlIdentity()`: stable fingerprint for the
outgoing push body.
- `FleetSyncService.reanchor()`: admin-driven anchor reset.
- `ControlIdentityMismatchError`: typed sentinel the route translates
to 409 with structured body `{error, code, expected, got}`.
Tests:
- 9 new vitest cases covering first-sync persistence, mismatch
rejection, matching acceptance, empty-incoming back-compat,
post-reanchor un-anchored state, fingerprint stability, missing
instance_id fallback, route-level mismatch, route-level reanchor
with override gating.
- One ordered route-level scenario instead of cross-dependent it()
blocks so test reordering cannot silently break the suite.
- Full backend suite: 1769 pass / 5 skipped.
|
||
|
Anso
|
27660f622b |
fix(fleet-sync): version the wire protocol and serialize per-node pushes (#967)
Hardens the scan-policy and CVE-suppression replication channel as the foundation of a multi-PR fleet sync hardening track. No new endpoints, no new tables, no schema changes; receivers still tolerate legacy payloads (absent pushedAt and controlIdentity) for rollout safety. Wire protocol: - Sender stamps every push with a strictly-increasing pushedAt and a placeholder controlIdentity. Receiver rejects strictly-older pushedAt with 409 STALE_SYNC_PUSH so the next write retries. - pushedAt comparison plus row replacement plus watermark write run in a single SQLite transaction; a partial-write window cannot leave the watermark behind the row state. Concurrency and limits: - Per-node mutex on the sender so concurrent control writes serialize per remote and never apply older state on top of newer. - Sender-side row cap at MAX_SYNC_ROWS=5000 with a 6-hour throttled truncation alert so flapping configs cannot flood the operator. - Route-level body limit raised to 5MB on POST /api/fleet/sync/:resource only; the global 100KB cap is unchanged. Oversize bodies return a structured 413 SYNC_PAYLOAD_TOO_LARGE. Determinism and hygiene: - getMatchingPolicy gains an id-ASC tiebreaker so two replicas resolve the same winner when policies tie on scope class. - Inline comment documents why getMatchingPolicy filters node_id at SQL yet still relies on JS identity matching for replicated rows. - Comment on the receive endpoint documents why no requirePaid is enforced (control's tier authorizes; replica trusts the bearer). - STALE_SYNC_PUSH 409s no longer record a node failure; they are expected protocol outcomes, not health issues. Public surface additions: - DatabaseService.transaction(fn): generic SAVEPOINT-friendly wrapper. - DatabaseService.getLocalScanPolicies / getLocalCveSuppressions: SQL filter on replicated_from_control = 0. - StaleSyncPushError: typed sentinel the route translates to 409. - fleetSyncConstants: shared MAX_SYNC_ROWS, body limit, state-key and error-code maps so the wire protocol has one source of truth. Tests: - 24 new vitest cases across fleet-sync-service, fleet-sync-routes, and database-matching-policy covering monotonic pushedAt, per-node serialization, row truncation and throttle, stale-push suppression, receiver back-compat, oversize-body 413, deterministic matching. - Full backend suite: 1757 pass / 5 skipped. |
||
|
sencho-quartermaster[bot]
|
14afc7c8d3 |
chore(main): release 0.72.0 (#959)
Co-authored-by: sencho-quartermaster[bot] <275163604+sencho-quartermaster[bot]@users.noreply.github.com> |
||
|
Anso
|
0f0b22c51a |
feat(fleet): Fleet Secrets tab with env-var bundles (v1 MVP) (#965)
* feat(fleet): add Fleet Secrets tab with versioned env-var bundles (Skipper+) Centralized, encrypted-at-rest secret bundles that can be pushed to labeled nodes' stacks. Each save bumps a monotonic version; each push records a per-node-per-version row in `secret_pushes` plus an entry in `audit_log`. Conflict detection shows added/changed/unchanged/removed (informational) diffs before write. Overlay merge preserves keys missing from the bundle. - Adds `secrets`, `secret_versions`, `secret_pushes` tables. - New `SecretsService` reuses CryptoService for AES-256-GCM, NodeLabelService for selectors, and direct fetch + Bearer for outbound calls to remote nodes. - New `secretsRouter` with 9 endpoints under `/api/secrets`, gated by `requirePaid`. Mounted after the auth gate. - Audit summary patterns added for the new routes. - New Fleet › Secrets tab with bundle list, editor sheet (key=value rows, versions tab), and push wizard (selector, target stack, env file picker, per-node diff preview, results pills). - Documentation: docs/features/fleet-secrets.mdx + docs.json nav entry. - 26 Vitest cases cover parser, encryption, versioning, push aggregation, tier gating. * fix(fleet): use const for rawValue in env parser ESLint prefer-const flagged the let declaration as a CI-blocking error; the variable is never reassigned. |
||
|
Anso
|
52b46753af |
feat(fleet): add Federation tab with cordon and pin policy (Admiral) (#964)
Ships the v1 MVP for the Federation tab as placement control, not
placement automation:
- Cordon a node: marks the node unschedulable so the BlueprintReconciler
skips it for new placements only. Existing deployments continue to
drift-check and redeploy on revision changes; cordon never triggers
withdraw or eviction. Toggle on the NodeCard kebab (Admiral, admin
role); Cordoned pill renders for all tiers.
- Pin a blueprint to a node: stores blueprints.pinned_node_id, replacing
the desired set with the pinned node regardless of selector. Pin
overrides cordon by design. Action lives only in the Federation tab;
BlueprintDetail and the deployment table show read-only Pinned
indicators.
Backend: idempotent migrations add nodes.cordoned/cordoned_at/cordoned_reason
and blueprints.pinned_node_id. New routes POST /api/nodes/:id/cordon,
POST /api/nodes/:id/uncordon, PUT /api/blueprints/:id/pin, all gated by
requireAdmiral plus requireAdmin. Audit summaries added so the existing
auditLog middleware records every operator action. deleteNode clears
dangling pins.
Reconciler: pin override evaluated before selector match; cordon filter
applied only to the new-placement branch (deploy/stateReview without an
existing deployment). 11 new Vitest cases cover cordon filter, pin
override, pin-overrides-cordon, missing pin target, pin shrinks
desired set (stateless withdraw + stateful evict_blocked), and pin
clearing on node delete.
Frontend: new FederationTab.tsx with cordoned-nodes summary and
pin-policy table. Federation moved out of the experimental flag into
{isAdmiral && (...)} + AdmiralGate, mirroring the Routing tab pattern.
Secrets stays under experimental.
Tests pass: backend tsc, full Vitest suite (1704 passed), frontend
tsc -b, ESLint (0 errors). Manual verification via the local dev
instance confirmed the tab is hidden at Community, the kebab and pill
render at Admiral, and cordon and pin endpoints round-trip end to end.
Refs cut-line-1.0.md Federation v1 MVP.
|
||
|
Anso
|
77d5ff58d3 |
feat(fleet): add Fleet Actions tab for cross-node bulk operations (#963)
* feat(fleet): add Fleet Actions tab for cross-node bulk operations Introduces a new "Actions" sub-tab in Fleet view with two Skipper+ cards that fill gaps in the existing surface: - Stop fleet by label: matches a label name across every node and stops every stack assigned to it, reporting per-node and per-stack results. - Bulk label assign: applies the same label set to many stacks on one node in a single round trip. Other bulk operations stay in their existing homes (sidebar bulk mode, Schedules, NodeUpdatesSheet) to avoid duplicate surfaces. Backend: - POST /api/fleet/labels/fleet-stop (gateway-orchestrated, multi-node) - POST /api/fleet-actions/labels/bulk-assign (per-node, capped at 1000) - Tightens /api/fleet proxy-exempt prefix to /api/fleet/ so /api/fleet-actions/* is routed through the proxy for per-node calls. - Exports activeBulkActions from labels.ts so fleet-stop and label-action share the per-node lock and cannot double-stop the same containers. - Extracts containerActionForStack helper from stacks.ts for reuse. * chore(fleet): rename Actions tab to Fleet Actions and reorder Fleet sub-tabs - Tab label "Actions" -> "Fleet Actions" so the surface is unambiguous alongside Schedules and the sidebar bulk bar. - Reorder Fleet sub-tabs as Overview / Snapshots / Status | Deployments / Traffic / Fleet Actions, with the separator after Status. - Rename "Traffic · Routing" -> "Traffic" and update Sencho Mesh docs to match the shorter label. - Update Fleet Actions docs to the new tab name and placement. |