Commit Graph

72 Commits

Author SHA1 Message Date
Anso 7cca68f84a fix(mesh): operator-triggered override regen and boot lifecycle hardening (#1016)
Audit-driven follow-up to the F6+F7 fix. Closes the High-severity findings
from the mesh-dial-path audit without expanding into the architectural
follow-ups (JWT TTL, TLS enforcement, per-stack JWT scope) which need
their own design discussions.

Changes:

1. New `POST /api/mesh/regen-overrides` Admiral-gated endpoint that calls
   `MeshService.regenerateAllOverrides()` and returns a structured
   `MeshRegenSummary { regenerated, failures, skipped, reason? }`. Lets
   the operator rerun boot regen after fixing a remote node that was
   offline at central startup, instead of opt-out + opt-in for every
   meshed stack on that node. Audit-log row carries the outcome
   (success / partial / skipped / error).

2. `regenerateAllOverrides` now aggregates per-stack outcomes into a
   single summary log row (`mesh override regen complete: N succeeded,
   M failed across K node(s)`) with `failedNodeIds` in `details`. Per-
   stack warnings still emitted. When skipped because `senchoIp` is
   null, emits an explicit warn instead of silently no-oping.

3. `MeshService.start()` now wraps `refreshAliasCache` and
   `syncForwarderListeners` in their own try/catch so a throw from
   either step is logged and the boot continues to override regen.
   The closing log line surfaces data-plane state ("MeshService started
   (data plane ok)" or "(data plane unavailable (<reason>))") so an
   operator grepping startup output sees a half-init mesh immediately.

4. `sanitizeForLog` wrap on four previously unsanitized `err.message`
   writes (cross-node tcpStream error handler + three probe error
   paths). Container IPs and local socket details no longer leak into
   the activity buffer or the probe response body.

5. Updated three existing F6 regression tests for the new return shape
   and aggregated summary message. Added two new tests: version-skew
   404-push handling, and concurrent opt-in during regen.
2026-05-09 03:45:04 -04:00
Anso 693f9b4495 fix(fleet): drop tier gate from stack and container list endpoints (#1012)
Community tier opened the Fleet Overview drilldown (node card → stack
list → container list) but two read-only metadata endpoints in
fleet.ts kept their requirePaid gate. Stack names still rendered
because they ride on the un-gated /api/fleet/overview payload, but
expanding a stack always re-fetched containers and hit the orphan
gate, surfacing as a "Failed to load containers for X" toast on every
node card on Community.

Drop requirePaid from GET /node/:nodeId/stacks and
GET /node/:nodeId/stacks/:stackName/containers. Both handlers stay
behind authMiddleware, validate inputs, and proxy to the per-node
api_token for remote nodes. Paid mutations (bulk update, scheduled
snapshots, label bulk actions, Fleet Actions cards) keep their gates
in fleetActions.ts. Add positive Community-tier tests in fleet.test.ts
and clear the stale "Requires Skipper or Admiral license" line from
the OpenAPI descriptions.
2026-05-09 01:55:56 -04:00
Anso 23bbee4f45 feat(mesh): replace host-mode with shared sencho_mesh Docker network (#1009)
* feat(mesh): replace host-mode with shared sencho_mesh Docker network

Phase D of the mesh redesign: drop the operator's `network_mode: host`
requirement and the `host-gateway` extra_hosts pattern that did not work
on cloud iptables-restrictive distros (OCI, etc.) or Docker Desktop.

Each Sencho creates a shared `sencho_mesh` Docker bridge network on
boot (default subnet 172.30.0.0/24, override via SENCHO_MESH_SUBNET),
pins itself at `<network>+2`, and attaches every meshed user service to
the same bridge. Compose overrides now emit IP-based `extra_hosts` plus
a top-level `networks` block declaring `sencho_mesh` external.

Override delivery: central renders for local stacks; for remote stacks
it sends the fleet alias list to the remote's new `PUT /api/mesh/local-
override/:stackName` endpoint, which renders against the remote's OWN
local senchoIp and writes under its OWN DATA_DIR. Each node may use a
different subnet without coordination beyond the env var.

Opt-in / opt-out now trigger an automatic redeploy of the affected
stack via the existing deploy code path (local: ComposeService; remote:
HTTP POST through proxyFetch). The frontend opt-in sheet shows a
confirmation modal (ConfirmModal) before the mutation. Failed
redeploys emit both a mesh activity event and a durable audit-log row.

Hardening:
- Reserve port 1852 at opt-in (prevents user containers from racing
  the Sencho API listener).
- ensureMeshNetwork refuses to continue if `sencho_mesh` exists with a
  mismatched subnet rather than silently routing to the wrong IP.
- Idempotent network connect/disconnect helpers in DockerController.
- optInStack rolls back the DB row if the just-inserted stack's
  override push fails (no half-states surviving across calls).
- regenerateOverridesForNode runs in parallel and skips the just-
  pushed stack on opt-in.

Operator template: drop `network_mode: host`, restore
`ports: ["1852:1852"]`. Mesh now works identically on Linux LAN, OCI,
and Docker Desktop without firewall changes.

Docs: rewrite docs/features/sencho-mesh.mdx around the shared bridge
network, document SENCHO_MESH_SUBNET, surface the host-network-service
opt-in restriction, and cross-link with the Pilot Agent docs.

BREAKING CHANGE: the operator's `docker-compose.yml` no longer uses
`network_mode: host`. After upgrading, redeploy any meshed stacks once
so they pick up the new IP-based override and join `sencho_mesh`.

* fix(mesh): wrap stackName with path.basename in local-override fs ops

CodeQL flagged js/path-injection on the new applyLocalOverride and
removeLocalOverride methods because they are publicly reachable and
its data-flow model does not recognize isValidStackName /
isPathWithinBase as sanitizers. The validation IS sufficient (the
allowlist regex blocks path separators, the path-prefix check blocks
escape), but path.basename is a model CodeQL recognizes and is purely
defensive: for any input that already passes isValidStackName,
basename is the identity.
2026-05-09 00:11:09 -04:00
Anso f599110386 feat(mesh): collapse sidecar into Sencho process via in-process forwarder (#1000)
The separate saelix/sencho-mesh sidecar container is gone. The
forwarder logic that previously lived in mesh-sidecar/src/forwarder.ts
moves into the Sencho process as backend/src/services/MeshForwarder.ts,
a thin per-port net.Server lifecycle wrapper. MeshService implements
the host interface and owns resolve plus splice; MeshForwarder owns
listener boilerplate. One container per node, no separate image to
publish, no control WebSocket.

Operator-facing change: the Sencho container now runs in
network_mode: host so the forwarder can bind alias ports on the host
network where meshed containers' extra_hosts host-gateway entries
point. Without host network mode the listeners would land in the
container's namespace and inbound traffic from peers would never
reach them. The 1852:1852 port publish becomes a no-op under host
mode and is commented out in the operator template.

Same-node forward path now dials the target container's bridge IP
via Dockerode (preferring the compose default network for
deterministic selection across daemon versions) instead of 127.0.0.1.
The legacy 127.0.0.1 path only worked when the target service
published its port to the host; the IP path works regardless.

Cross-node mesh routing in this phase is central -> pilot direction
only via PilotTunnelManager.openTcpStream. Pilot -> central and
pilot <-> pilot via central relay land in Phase B with the
tcp_open_reverse frame.

Deletions:
- mesh-sidecar/ package entirely (Dockerfile, package, sources, tests)
- backend/src/websocket/meshControl.ts
- MeshService sidecar lifecycle: spawnSidecar, stopSidecar,
  isSidecarRunning, mintSidecarToken, verifySidecarToken,
  attachSidecarSocket, handleSidecarResolve, sendSidecar
- POST /api/mesh/nodes/:id/sidecar/restart route
- /api/mesh/control WS dispatch in upgradeHandler

Type cleanup: 'sidecar' literal removed from MeshActivitySource and
MeshProbeResult.where (also the frontend mirror). MeshNodeStatus
sidecarRunning becomes localForwarderListening (boolean | null) so
non-local nodes get a null instead of an unconditional false; the
honest semantic is "this view only knows the local forwarder state;
remote forwarder status lands in Phase B." MeshNodeDiagnostic
sidecar object becomes forwarder { listening, listenerCount }.

Frontend MeshDiagnosticsSheet drops the restart-sidecar action and
sidecar liveness card; surfaces forwarder state plus a "runs
in-process; no separate container" caption.

Resolves audit findings C-1 (data plane non-functional), C-2 (sidecar
control WS not loopback-enforced), C-4 (sidecar lifecycle Dockerode-
on-remote, PR #999 closed), and C-5 (saelix/sencho-mesh:latest
unreachable). C-3 (PR #992) is unchanged. M-12 (PR #994) is
unchanged.
2026-05-08 15:51:39 -04:00
Anso 1803512f70 feat: drop stack labels and network topology to Community tier (#995)
* feat(labels): drop tier gate to Community for organization endpoints

Stack Labels CRUD and per-stack assignment are now Community-tier features:
list, create, update, delete labels, and assign labels to a single stack.

The two automation surfaces stay Skipper+: per-label bulk deploy / stop /
restart (POST /api/labels/:id/action) and the Fleet Actions tab's
bulk-assign card (POST /api/fleet-actions/labels/bulk-assign). Tier story
is now organize free, automate paid.

Add a route-level test that proves the CRUD endpoints succeed on a Community
license while the bulk-action endpoint still returns 403. Update overview,
licensing, and stack-labels docs to reflect the new tier placement and to
note that bulk actions on a label still require Skipper or Admiral.

* feat(topology): drop tier gate to Community for network topology view

The Resources tab's Networks > Topology view is now available on every tier.
Drops requirePaid from GET /api/system/networks/topology, removes the
isPaid wrapper around the List | Topology toggle in ResourcesView, and
removes the PaidGate around the topology graph.

CapabilityGate stays in place so a node running on a build without the
network-topology capability still renders its lock card instead of the
graph. Add a route-level test that proves the endpoint returns 200 on a
Community license. Update licensing and resources docs to reflect the new
tier placement.

* fix(labels): expose Settings > Labels tab on Community tier

The settings registry entry for the Labels tab still carried tier:
'skipper', which kept the tab hidden in the Settings sidebar even though
the underlying CRUD endpoints now serve Community. Drop the tier flag so
Community users can discover and reach the section that backs the
already-Community-tier label endpoints.
2026-05-08 15:24:25 -04:00
Anso e176ae9f17 fix(mesh): inspect remote stacks via HTTP proxy not Dockerode (#992)
MeshService.inspectStackServices used DockerController.getInstance(nodeId)
to enumerate compose-labeled containers, but NodeRegistry.getDocker
explicitly throws for any node with type='remote' by design. The throw
was silently caught and an empty array returned, so opt-in for any
remote node failed with a misleading "no running services" error and
refreshAliasCache only ever populated LOCAL aliases. Cross-node mesh
routing was therefore impossible end-to-end regardless of pilot-tunnel
state.

Split the inspector. inspectLocalStackServices keeps the Dockerode
listContainers path and always queries the local Docker daemon; it is
public so the new route can call it. inspectStackServices is now a
dispatcher: local nodes fall through to the Dockerode path, remote
nodes (proxy mode and pilot-agent) HTTP-fetch
/api/mesh/local-services/:stackName against the URL resolved by
NodeRegistry.getProxyTarget, with the persisted node_proxy Bearer
token and license tier headers attached. The remote's MeshService
enumerates its own LOCAL Docker daemon and returns the
{service, ports[]} envelope.

refreshAliasCache now inspects every opted-in stack in parallel via
Promise.allSettled so a slow or unreachable remote does not stall the
60-second loop. The new /api/mesh/local-services/:stackName route is
gated by requireAdmiral plus isValidStackName and always queries the
caller's own Sencho instance.

Together with PR #989 (proxy-side bridge dispatch) and PR #990
(agent-side loopback auth), this makes the central, bridge, agent,
local-Sencho mesh control plane functional end-to-end for both
proxy-mode and pilot-agent remotes.
2026-05-08 11:12:29 -04:00
Anso 3112f58a88 fix(templates): align App Store deploy gate with stack:create permission (#986)
The POST /api/templates/deploy handler gated on requireAdmin, while every
other create-stack endpoint in routes/stacks.ts uses
requirePermission('stack:create'). Per the role table in
middleware/permissions.ts, node-admin holds stack:create — so a node-admin
could create stacks the regular way but got 403 ADMIN_REQUIRED from the
App Store. The cockpit's Deploy button is gated on can('stack:create'),
so the button looked enabled and the click silently failed.

Swap the gate to requirePermission(req, res, 'stack:create'). Admin still
passes through the global bypass; node-admin now passes via the role
permissions table; deployer, viewer, and auditor stay denied. Cache-refresh
on the same router keeps requireAdmin since cache invalidation has no
per-resource scope.

Adds backend/src/__tests__/templates-deploy-rbac.test.ts with six
parameterised supertest cases (one per role plus an unauthenticated
case) so the matrix is locked in. The two passing-role cases assert
the request clears the gate (status !== 403, code !== PERMISSION_DENIED)
without depending on Docker being available in the test environment.
2026-05-08 03:50:03 -04:00
Anso 3d9489648e fix(pilot): harden outbound reverse-tunnel against resource exhaustion (#979)
* fix(pilot): cap tunnel frame size, concurrent streams, and stream idle time

Pilot tunnels carry every HTTP, WS, and Mesh-TCP byte for a remote node
through a single multiplexed WebSocket. A buggy or compromised peer that
sent oversized frames, opened streams in a tight loop, or left streams
parked indefinitely could exhaust gateway memory.

Three protocol-level limits applied symmetrically on both ends:

  - MAX_FRAME_SIZE_BYTES (8 MB): set as the ws maxPayload on the
    gateway-side WebSocketServer and the agent-side WebSocket client,
    plus a defense-in-depth length check in decodeBinaryFrame and
    decodeJsonFrame.
  - MAX_STREAMS_PER_TUNNEL (1024): bridge refuses new loopback HTTP /
    upgrade / TCP allocations with 503 once at the cap; the agent
    rejects new incoming http_req / ws_open / tcp_open with the
    appropriate error frame.
  - STREAM_IDLE_TIMEOUT_MS (10 min): every stream gets a per-stream
    timer refreshed on each inbound or outbound activity. Expiry tears
    down the local half and notifies the peer.

The constants are colocated in pilot/protocol.ts so any future agent
build picks them up via the protocol module.

* fix(pilot): rate-limit pilot enrollment endpoints to 10 per minute

Pilot enrollment mints a JWT and writes a pilot_enrollments row, both
privileged operations that should not share the global API budget
(200/min). Add a dedicated express-rate-limit instance keyed by user or
IP at 10/min in production (100/min in dev so the local enrollment
test loop is not throttled).

The limiter is applied directly on POST /api/nodes/:id/pilot/enroll.
On POST /api/nodes the limiter's skip function reads the parsed body
and exempts proxy-mode creates; only requests that resolve to
mode=pilot_agent count against the enrollment budget.

* fix(pilot): cap concurrent pilot tunnels system-wide

The PilotTunnelManager held an unbounded Map of bridges. A reconnect
storm, runaway enrollment, or operator misconfiguration could grow the
map without limit, even though every tunnel still carries a valid JWT.

Cap at 256 concurrent tunnels per primary instance:

  - Soft warning at 128 (logged once per crossing).
  - Hard refusal at 256: registerTunnel throws PilotTunnelCapacityError
    and the upgrade handler closes the WebSocket with 1013 (Try Again
    Later) so the agent backs off rather than tight-looping.

A node that already has a registered tunnel does not consume a new
slot when it reconnects; the existing bridge is closed first.

* fix(pilot): release backpressure as soon as the tunnel buffer drains

Previously the bridge paused HTTP request bodies when the tunnel
WebSocket's bufferedAmount climbed above 4 MB but never resumed them
explicitly; the next data event re-checked the threshold, and TCP
streams only saw a 'drain' fan-out on the 30 s ping cycle. Slow-consumer
peers held buffered bytes for tens of seconds longer than needed.

Replace both with an on-demand drain check: when at least one stream
is paused, sample bufferedAmount every 100 ms; once it drops below the
high-water mark, resume every paused request and emit 'drain' to every
accepted TCP stream, then stop the timer. Dormant when no stream is
paused, so steady-state cost is zero.

* fix(pilot): trust internal CAs via SENCHO_PILOT_CA_FILE

Self-hosted deployments often terminate TLS with an internal CA. The
only previous escape hatch was NODE_TLS_REJECT_UNAUTHORIZED=0, which
disables verification across every outbound connection in the agent
process and is the wrong shape of fix.

Add SENCHO_PILOT_CA_FILE: when set, the agent reads the file as a PEM
bundle and passes it to ws as the `ca` option for the tunnel
WebSocket. rejectUnauthorized remains true. Failure to read the file
exits with a clear error so the operator does not silently fall back
to the default trust store.

There is no flag to disable TLS verification entirely; that would
defeat the credential trust model.

* fix(pilot): gate frame-rate-adjacent logs behind developer_mode

The agent's malformed-frame warning fires per inbound frame and could
flood logs under attack or against a buggy primary. The bridge swallowed
parse errors silently, leaving operators blind to protocol drift, and
the manager had no signal at all on tunnel registration.

Route per-frame and per-tunnel diagnostic logs through the existing
isDebugEnabled() helper (same pattern used by middleware/auth, RBAC,
and websocket/logs). Production stays quiet by default; the toggle is
the existing developer_mode setting in global_settings — no new env
var, no new dependency.

* feat(pilot): expose per-tunnel metrics via /api/system/pilot-tunnels

Operator support had no signal beyond raw logs to answer "is this
tunnel flapping?" or "is one bad node hiding behind the aggregate?".
Add an in-process counter set covering tunnels_total, tunnels_replaced,
tunnels_rejected_capacity, enroll_acks, frame_decode_errors, plus a
per-node array carrying the connectedAt and bufferedAmount so a single
tunnel sitting on a stuck buffer remains visible.

Counters live in services/PilotMetrics.ts; the gateway has no shared
in-process metrics facility today, so this is a per-feature pattern
documented as such for future consolidation. Counters are strictly
process-local — no telemetry, no export, no phone-home — consistent
with Sencho's privacy posture.

Surfaced read-only via GET /api/system/pilot-tunnels behind admin auth,
mirroring the existing cache-stats endpoint.

* fix(pilot): tighten reconnect backoff and log handling

Three small follow-ups to the hardening pass:

  - Sanitize the primary URL on the agent's connect log so a malicious
    SENCHO_PRIMARY_URL with embedded control characters cannot inject
    fake log lines.
  - Move the reconnect-backoff reset from the 'open' callback to the
    'hello' frame handler. A peer that always rejects the handshake
    (incompatible version, consumed enrollment token) used to reset
    the backoff on every TCP-level connect and tight-loop reconnects;
    now the reset waits for a clean protocol round-trip.
  - Document the StreamIdAllocator wrap behavior so future readers
    can see the relationship between the 2^31 wrap and the
    MAX_STREAMS_PER_TUNNEL cap without re-deriving it.

* test(pilot): add coverage for enrollment, rate limiter, and tunnel caps

Adds two test files exercising the hardening pass surfaces that were
previously uncovered:

  - pilot-enrollment.test.ts: end-to-end through POST /api/nodes
    (pilot mode), POST /api/nodes/:id/pilot/enroll, the SHA256
    token-hash persistence, replay protection, expired enrollments,
    and the rate-limit header wiring (enrollment limiter on pilot
    paths, global limiter elsewhere).
  - pilot-bridge-limits.test.ts: oversize-frame rejection at both
    binary and JSON decoders, and the per-tunnel concurrent stream
    cap as observed via openTcpStream returning null and the loopback
    HTTP server returning 503 once the bridge is at capacity.

Bridge cap test fills the slot map with TCP stream handles rather than
real HTTP requests so the OS socket pool stays out of the picture.

* test(pilot): cover manager metrics snapshot and capacity-error shape

Adds direct coverage for:

  - PilotTunnelCapacityError exposing the limit so the upgrade
    handler can format a useful close-frame reason.
  - PilotMetrics.snapshot returning a defensive copy (callers must not
    mutate the live counter set).
  - getMetricsSnapshot returning the open count, per-node breakdown,
    and counter set in a stable shape, with tunnels_total bumping on
    a successful registerTunnel.

The manager test seeds a real pilot-mode node row first because
updateNode throws on missing rows; this models the production path
where a node is created before its enrollment is consumed.

* docs(pilot): document tunnel limits, custom CA, and new failure modes

Refresh the Pilot Agent feature doc to cover the hardening surfaces:

  - New 'Self-signed primary TLS certs' section walking through the
    SENCHO_PILOT_CA_FILE env var with an example docker run command.
  - New 'Resource limits' section listing the per-tunnel and
    system-wide ceilings so operators know what the wire enforces.
  - Four new troubleshooting entries: WebSocket close 1013 (system
    cap), loopback 503 (stream cap), close 1002 with protocol error
    (frame size / malformed), and HTTP 429 on enrollment.

Also expand .env.example with a new pilot-agent block covering
SENCHO_MODE, SENCHO_PRIMARY_URL, SENCHO_ENROLL_TOKEN, and
SENCHO_PILOT_CA_FILE so operators do not have to read code or the
feature doc to discover the agent-side config surface.

* fix(pilot): address code-review findings on the hardening pass

Critical:
  - Bridge: TCP backpressure now starts the drain timer and tracks
    streams awaiting drain, so a TcpStream caller waiting on 'drain'
    no longer hangs when no HTTP request is in flight.
  - Bridge: every direct streams.delete call now goes through the
    removeStream helper so per-stream idle timers are cleared
    consistently and pausedReqs / tcpAwaitingDrain stay aligned.
  - Bridge close(): resume any paused IncomingMessage before clearing
    the map so a parser does not stay stuck across teardown.

High:
  - Protocol: decodeJsonFrame now compares Buffer.byteLength(raw,
    'utf8') against the cap; raw.length (UTF-16 code units) let
    multi-byte payloads sneak ~3x the byte budget through.
  - Agent: the optional CA bundle is read once at construction and
    cached on the instance, so a missing or rotated SENCHO_PILOT_CA_FILE
    no longer process-exits on every reconnect attempt.

Medium / Low:
  - Bridge limits test: assert mockWs.sent.length to confirm slot
    allocations actually serialized a tcp_open frame and that a
    rejected (cap+1) attempt did not consume a stream id.
  - Manager: defer the tunnels_replaced increment until after the
    cap check passes, so a rejected reconnect does not double-count
    as both replacement and capacity rejection.
  - Manager: clarify the protocol comment about MAX_FRAME_SIZE_BYTES
    enforcement layering (ws maxPayload is authoritative; decoder
    check is for tests and defense-in-depth).
  - Limiter: defensive `if (!req.body) return false` so a future
    refactor that delays body parsing cannot silently skip the
    enrollment limiter.

Deferred (documented as follow-ups, not in this branch):
  - M2: narrowing public surface of PilotTunnelManager.getBridge to a
    MeshTunnelHandle interface (cross-cutting refactor).
  - H4: end-to-end replay test driving the upgrade handler twice
    with the same enrollment JWT (needs WS test harness).
2026-05-07 21:28:35 -04:00
Anso 3b650523c1 Audit-hardening pass for secret and misconfiguration scanning (#977)
* fix(security): dedupe concurrent compose-stack scans

Track stack scans in scanningImages keyed stack:<nodeId>:<stackName>.
The /scan/stack route returns 409 when an in-flight scan exists, and
the service-side check is the real correctness barrier (the route
pre-check is a fast-path optimization that mirrors scanImage). The
dedup key release lives in a try/finally so failed scans free the
slot for retry.

Why: scanComposeStack had no equivalent of scanImage's scanningImages
guard, so two simultaneous calls for the same stack would both run
trivy config, both insert a vulnerability_scans row, and double-
process the result.

* feat(security): acknowledge misconfig findings

Adds a parallel acknowledgement system for Trivy misconfig findings
that mirrors cve_suppressions: a new misconfig_acknowledgements table,
read-time enrichment via the new misconfig-ack-filter utility, REST
CRUD endpoints, fleet-sync replication from control to replicas, a
Settings panel, and an Acknowledge button on the Misconfigs tab.

Schema and behavior parity with cve_suppressions:
  - UNIQUE(rule_id, COALESCE(stack_pattern, '')) so fleet-wide acks
    collide as expected
  - blockIfReplica on every write
  - Audit-log entries name the scope (rule_id, stack_pattern) but
    never the reason text
  - replicated_from_control flag controls UI delete affordance and
    drives clearReplicatedRows on demote/reanchor
  - Validators reused: validateStackPatternForRedos for glob safety,
    sanitizeForLog for log fragments

SARIF export emits an external/accepted suppression entry per
acknowledged misconfig, matching the CVE pattern.

Per-row Acknowledge dialog prefills stack_pattern with the scan's
stack_context so the default scope is "rule + this stack only" and an
operator must broaden explicitly.

Tests: misconfig-ack-filter (15) and misconfig-ack-routes (23)
including the duplicate-409 case for both pinned and fleet-wide acks.

* fix(security): reap orphaned trivy tmp dirs at startup

When the buildEnv path writes a per-scan DOCKER_CONFIG dir under
os.tmpdir() and the process crashes before the finally block runs,
the dir leaks. Mirrors GitSourceService.sweepStaleTempDirs:
exported sweepStaleTrivyTempDirs is fire-and-forget at boot,
removes prefix-matching dirs older than 1 hour, swallows
permission/race failures, logs a single line if any were reaped.

* perf(security): emit per-batch summary for scanAllNodeImages

Adds one diag() line at the end of scanAllNodeImages summarising
unique image count, scanned, skipped, failed, violation count, and
elapsed time. Per-image diag inside scanImage stays useful for
debugging individual scans; the summary gives operators a single
fleet-level checkpoint when developer_mode is on.

* perf(security): cap SARIF export at 5000 findings per type

Replace the unbounded fetchAllPages walk on /scans/:id/sarif with a
hard limit of 5000 findings per type. When any type trips the cap,
emit run-level properties.truncated=true plus row_limit and per-type
totals so downstream tooling can flag the export as partial.
Console-warns for ops visibility.

A scan with 50k vulns previously streamed every row into memory
before serialising; the cap bounds memory and serialisation time at
the cost of completeness on pathological scans.

* docs(env): document TRIVY_BIN host-binary override

The env var is honored by TrivyService.detectTrivy as a fallback when
no managed install is present, but it was undocumented in
.env.example. Adds the var with a comment explaining precedence
(managed > TRIVY_BIN > PATH).

* test(security): cover scanComposeStack failure modes

Two new cases drive the existing try/catch through real failure
paths:
  - Malformed Trivy stdout: row flips to status='failed' with the
    parser error preserved on `error`.
  - execFile rejection: row flips to status='failed' with a string
    error message.

Pairs with the existing dedup tests so the failure path now also
verifies the scan row state, not just the thrown exception.

* test(e2e): security scanner + misconfig acknowledgement flow

Seven Playwright tests covering the scanner UI and the new
acknowledgement system end-to-end:
  - Trivy availability gate (skips suite when binary absent so CI
    without Trivy can opt out via E2E_SKIP_TRIVY=1)
  - Stack config scan completes and records misconfig findings
  - Concurrent stack scan returns 409 from the dedup gate
  - Misconfig ack POST creates and lists on Settings
  - Duplicate (rule_id, stack_pattern) returns 409
  - Malformed rule_id (shell metacharacters) returns 400
  - Misconfigs tab renders against a real stack scan

Tests drive the API for behaviour assertions and the UI only for
shell-rendering checks; the visual snapshot suite owns screenshots.

* docs(features): add misconfig acknowledgement workflow and SARIF cap

Refreshes vulnerability-scanning.mdx with:
  - Misconfig acknowledgements section covering the per-row dialog,
    Settings panel, scope/matching rules, and SARIF emission
  - Tier table row for the new feature
  - SARIF section note on the 5000 row-per-type cap and the
    properties.truncated marker for partial exports
  - Troubleshooting entries: SARIF cap, hidden Acknowledge button,
    findings resurfacing after delete, Trivy DB phone-home, and
    409 on concurrent compose-stack scans

* fix(ci): clear backend lint and CodeQL alerts

- Remove the dead fetchAllPages helper in routes/security.ts. It lost
  its callers when the SARIF endpoint switched to direct paged reads
  for the truncation cap. ESLint flagged it as unused.
- Switch the trivy-tmp-cleanup test helper to fs.mkdtempSync. Building
  paths under os.tmpdir() with predictable names tripped CodeQL's
  js/insecure-temporary-file rule (high severity), which warns about
  symlink-pre-creation attacks even in test code. mkdtempSync appends
  a process-random suffix and creates the dir atomically; the
  sencho-trivy- prefix is preserved so the production sweep still
  matches the test fixtures.
2026-05-07 19:23:11 -04:00
Anso 4b1de35dda Audit-hardening pass for fleet-replicated CVE suppressions (#976)
* perf(security): bucket CVE suppressions by id at read time

applySuppressions now builds a Map<cve_id, suppression[]> once before the
per-finding loop, dropping per-finding work to O(matching-cve-suppressions)
rather than O(all-suppressions). At a fleet-wide cap of 10000 rows against
a multi-thousand-finding scan, the prior linear-per-finding shape drifted
into tens of millions of comparisons per render.

Public API of findSuppression and applySuppressions is unchanged.
Specificity scoring, expiry handling, and image-glob matching are
preserved bit-for-bit. Adds a regression guard that pins a 10000x2000
workload under 1.5s and asserts at least one match was actually returned.

* feat(security): record audit-log entries on control-side CVE suppression CRUD

The replica receive path already wrote an audit-log entry on apply; the
control-side POST/PUT/DELETE handlers did not. Operators reading the
audit panel saw mirrored security-rule changes from the replica view but
could not see who originated them on the control. Symmetric logging
closes that gap.

The summary records the CVE id and pinned scope (pkg, image) but never
the suppression's reason text. Reasons are free-form admin input that
replicate fleet-wide and may carry incident-tracker IDs or vendor
context the operator did not intend to broadcast.

The summary is also sanitised before emission so an operator-supplied
package name or image pattern carrying a smuggled newline plus a forged
"cve_suppression.delete:" prefix cannot inject a fake row into the audit
panel. Control characters become "?" and the field is capped to its
validator length.

Adds a Control-side audit log block to suppression-routes.test.ts that
asserts the privacy contract on each verb (scope present, reason absent),
plus a log-injection guard test, plus a regression that GET still works
when the local instance is a replica.

* test(security): cover cve_suppressions fleet-sync receive path

The existing fleet-sync route tests covered the protocol mechanics
(auth, anchor, stale push, reanchor, demote) for cve_suppressions only
with empty-rows payloads. The suppression-specific concerns were
unverified end-to-end:

  - actual rows replace prior replicated rows on a fresh push
  - the receive path writes an audit-log entry naming the source
    fingerprint and row count
  - a malformed suppression row is rejected at the validator before any
    DB write

Adds a focused block exercising those three properties against the real
Express app, real SQLite, and the real apply transaction.

* docs(features): refresh CVE suppressions troubleshooting and field guidance

Converts the troubleshooting section to Mintlify Accordion blocks so
each entry is foldable and the page stays scannable. Adds entries for:

  - control-identity mismatch on a replica (anchor-aware reanchor flow)
  - mirrored rules persisting after a control demote
  - the 10000-row truncation cap on a fleet sync push

Adds a privacy note to the Reason field guidance: do not paste
credentials, tokens, or vendor secrets there, since the field replicates
fleet-wide and surfaces on every node's suppressions panel.
2026-05-07 16:18:19 -04:00
Anso a284732a95 feat(fleet-sync): hide other replicas' identity-scoped policies on a replica (#973)
GET /api/security/policies on a replica now returns only the policies
that apply to THIS replica. Replicated rows with a node_identity
targeting a sibling replica are filtered out so an operator cannot
enumerate the names and rules of policies meant for another node in
the fleet. Defense in depth: the security panel is admin-only, but a
backend filter is bypass-proof and matches Sencho's privacy posture.

Internal evaluators (getMatchingPolicy, evaluateScanAgainstPolicies)
keep using the unfiltered list because they already enforce identity
matching at evaluation time.

CVE suppressions are fleet-wide on every replica (no node_identity
column) so no analogous filter is needed.

Public surface:
- DatabaseService.getScanPoliciesForUi(role, selfIdentity): the
  filtered variant, called from securityRouter.get('/policies').

Tests:
- 3 new vitest cases: control sees full set; replica hides
  other-replica scoped rows; replica always includes locally created
  rows.
- Full backend suite: 1795 pass / 5 skipped.
2026-05-07 13:48:11 -04:00
Anso 4007709590 fix(fleet-sync): hygiene pass on receiver behavior and cleanup (#972)
A bundle of small file-local fixes to the receiver path and
node-deletion flow.

Changes:
- F4 receiver audit log: applyIncomingSync now writes a system audit
  entry on every applied push so mirrored security-rule changes show
  up in the replica's audit panel with a clear control-side origin.
- F7 pilot-agent skip: pushResource explicitly excludes pilot-agent
  nodes (they have no api_url for HTTP push) and warns once per node
  id so the operator sees they will not receive replicated policies.
- B4 identity-drift notification: when targetIdentity differs from
  the cached fleet_self_identity, dispatch a warning so the operator
  can audit any identity-scoped policies that may need re-targeting.
- B6 stack_pattern ReDoS guard: reject patterns with 4+ consecutive
  wildcards or more than 8 wildcards total. Both control-side
  validators (POST/PUT scan policies) and the receiver-side row
  validator share the helper.
- B9 deleteNode cascade: clear fleet_sync_status rows for the node
  inside the existing transaction so the sync-status panel does not
  render ghost entries after a node is removed.
- S6 last_error redaction: formatError strips Bearer tokens and
  JWT-shaped values from error messages and caps at 500 chars before
  storing in fleet_sync_status.last_error or logging.

Tests:
- 8 new vitest cases covering audit-log entry, identity-drift alert,
  pilot-agent warn-once, formatError redaction (Bearer + JWT), ReDoS
  validator rejection, and a backtracking-time smoke test.
- New database-fleet-sync-cascade.test.ts: deleteNode removes
  fleet_sync_status rows for the deleted node and leaves siblings
  untouched.
- Full backend suite: 1792 pass / 5 skipped.
2026-05-07 13:41:10 -04:00
Anso 33b15d6cba feat(fleet-sync): retry failed pushes and backfill on add-node (#970)
A control instance now retries fleet-sync pushes that hit a transient
failure and backfills the security state on a freshly registered remote
without waiting for the next policy edit.

New service:
- FleetSyncRetryService (singleton, start/stop) wakes 30s after boot
  and ticks every 5min. For each fleet resource, queries
  getFailedSyncTargets within a 24h window and re-pushes via
  FleetSyncService.pushResourceToNode through the same per-node mutex,
  so a normal fanout in flight serializes naturally with a retry.
- After STALE_THRESHOLD_MS (1h) of continuous failure for a
  previously-working node, dispatches a single warning notification
  per cooldown window. Brand-new nodes that have never succeeded do
  not alert via this path; misconfigured remotes are caught by the
  test-connection affordance at registration time.
- Wired into bootstrap startup/shutdown next to AutoHealService.

Public surface:
- FleetSyncService.pushResourceToNode(node, resource): targeted push
  to one node that re-uses the per-node mutex. Used by the retry
  service and any future targeted-resync flow.
- routes/nodes.ts POST /api/nodes fires pushResourceAsync for both
  resources after a remote-proxy node row commits.

Tuning constants centralized in fleetSyncConstants.ts:
- RETRY_MAX_AGE_MS = 24h
- STALE_THRESHOLD_MS = 1h

Tests:
- 8 vitest cases covering replica skip, retry dispatch, missing-node
  skip, alert-once-per-cooldown across the threshold window, no-alert
  for recent failures, no-alert for brand-new never-succeeded nodes,
  no-alert when the retry itself succeeds, start/stop idempotency.
- Full backend suite: 1781 pass / 5 skipped.
2026-05-07 13:16:24 -04:00
Anso 7dde257e1f feat(fleet-sync): replica self-demote endpoint and role UX (#969)
A replica admin can now demote the instance back to a standalone
control without raw SQLite access. The Settings → Security UI surfaces
a confirm-gated button when the role is replica; the role probe also
surfaces a soft banner when it cannot determine fleet role rather
than silently defaulting to control.

Backend:
- POST /api/fleet/role/demote (admin, requires `{confirm: true}`):
  flips fleet_role to 'control', clears fleet_self_identity,
  fleet_control_identity, and both received_pushed_at:* watermarks,
  drops every replicated_from_control row from scan_policies and
  cve_suppressions, nulls out any orphaned policy_evaluation cache.
  Returns 409 ALREADY_CONTROL when invoked on a control.
- DatabaseService gains `clearOrphanPolicyEvaluations()` and
  `clearReplicatedRows()` helpers. Reanchor consolidates onto
  clearReplicatedRows so it shares the same code path.
- `FleetSyncService.demote()` returns boolean for the route to
  translate into 200 or 409.

Frontend:
- SecuritySection probes /fleet/role and now records explicit success
  vs failure rather than silently treating an error as control. A
  soft banner appears when probe fails.
- Replica banner gains a "Demote to control" button and a destructive
  ConfirmModal explaining the wipe.

Tests:
- 4 new route-level vitest cases (401, 400 without confirm,
  end-to-end demote with replica setup, 409 ALREADY_CONTROL with
  explicit precondition).
- Service unit test asserts the consolidated clearReplicatedRows path.
- Full backend suite: 1773 pass / 5 skipped. Frontend: 185 pass.
2026-05-07 13:08:21 -04:00
Anso f3757b43c6 feat(fleet-sync): anchor replicas to a control fingerprint (#968)
A replica now binds to the first control that pushes to it. Subsequent
pushes from a different control are rejected with 409
CONTROL_IDENTITY_MISMATCH until an admin explicitly reanchors. Closes
the cross-control hijack window where any node_proxy bearer signed
against the replica's secret could overwrite security policies.

Wire protocol:
- Sender includes a stable 16-hex-char `controlIdentity` derived by
  SHA-256-truncating `system_state.instance_id` (the local UUID written
  once by LicenseService.initialize on first boot). Hostname rotations
  do not flag drift; only a SQLite reset or explicit reanchor breaks
  the binding.
- Receiver caches the fingerprint inside the same transaction that
  applies the row replacement and watermark write. Three states:
  null (fresh install), '' (post-reanchor), '<fingerprint>' (anchored).
- Empty `controlIdentity` is treated as legacy and accepted, so older
  controls keep working during rollout.

New endpoint:
- POST /api/fleet/role/reanchor (admin, requires `{override: true}`):
  clears the cached fingerprint, both `received_pushed_at:*` watermarks,
  and replicated rows of both resources, all in one transaction. The
  static cached fingerprint is also flushed defensively.

Public surface additions:
- `FleetSyncService.getControlIdentity()`: stable fingerprint for the
  outgoing push body.
- `FleetSyncService.reanchor()`: admin-driven anchor reset.
- `ControlIdentityMismatchError`: typed sentinel the route translates
  to 409 with structured body `{error, code, expected, got}`.

Tests:
- 9 new vitest cases covering first-sync persistence, mismatch
  rejection, matching acceptance, empty-incoming back-compat,
  post-reanchor un-anchored state, fingerprint stability, missing
  instance_id fallback, route-level mismatch, route-level reanchor
  with override gating.
- One ordered route-level scenario instead of cross-dependent it()
  blocks so test reordering cannot silently break the suite.
- Full backend suite: 1769 pass / 5 skipped.
2026-05-07 13:00:21 -04:00
Anso 27660f622b fix(fleet-sync): version the wire protocol and serialize per-node pushes (#967)
Hardens the scan-policy and CVE-suppression replication channel as the
foundation of a multi-PR fleet sync hardening track. No new endpoints,
no new tables, no schema changes; receivers still tolerate legacy
payloads (absent pushedAt and controlIdentity) for rollout safety.

Wire protocol:
- Sender stamps every push with a strictly-increasing pushedAt and a
  placeholder controlIdentity. Receiver rejects strictly-older pushedAt
  with 409 STALE_SYNC_PUSH so the next write retries.
- pushedAt comparison plus row replacement plus watermark write run in a
  single SQLite transaction; a partial-write window cannot leave the
  watermark behind the row state.

Concurrency and limits:
- Per-node mutex on the sender so concurrent control writes serialize
  per remote and never apply older state on top of newer.
- Sender-side row cap at MAX_SYNC_ROWS=5000 with a 6-hour throttled
  truncation alert so flapping configs cannot flood the operator.
- Route-level body limit raised to 5MB on POST /api/fleet/sync/:resource
  only; the global 100KB cap is unchanged. Oversize bodies return a
  structured 413 SYNC_PAYLOAD_TOO_LARGE.

Determinism and hygiene:
- getMatchingPolicy gains an id-ASC tiebreaker so two replicas resolve
  the same winner when policies tie on scope class.
- Inline comment documents why getMatchingPolicy filters node_id at SQL
  yet still relies on JS identity matching for replicated rows.
- Comment on the receive endpoint documents why no requirePaid is
  enforced (control's tier authorizes; replica trusts the bearer).
- STALE_SYNC_PUSH 409s no longer record a node failure; they are
  expected protocol outcomes, not health issues.

Public surface additions:
- DatabaseService.transaction(fn): generic SAVEPOINT-friendly wrapper.
- DatabaseService.getLocalScanPolicies / getLocalCveSuppressions: SQL
  filter on replicated_from_control = 0.
- StaleSyncPushError: typed sentinel the route translates to 409.
- fleetSyncConstants: shared MAX_SYNC_ROWS, body limit, state-key and
  error-code maps so the wire protocol has one source of truth.

Tests:
- 24 new vitest cases across fleet-sync-service, fleet-sync-routes, and
  database-matching-policy covering monotonic pushedAt, per-node
  serialization, row truncation and throttle, stale-push suppression,
  receiver back-compat, oversize-body 413, deterministic matching.
- Full backend suite: 1757 pass / 5 skipped.
2026-05-07 12:53:13 -04:00
Anso 0f0b22c51a feat(fleet): Fleet Secrets tab with env-var bundles (v1 MVP) (#965)
* feat(fleet): add Fleet Secrets tab with versioned env-var bundles (Skipper+)

Centralized, encrypted-at-rest secret bundles that can be pushed to labeled
nodes' stacks. Each save bumps a monotonic version; each push records a
per-node-per-version row in `secret_pushes` plus an entry in `audit_log`.
Conflict detection shows added/changed/unchanged/removed (informational)
diffs before write. Overlay merge preserves keys missing from the bundle.

- Adds `secrets`, `secret_versions`, `secret_pushes` tables.
- New `SecretsService` reuses CryptoService for AES-256-GCM, NodeLabelService
  for selectors, and direct fetch + Bearer for outbound calls to remote nodes.
- New `secretsRouter` with 9 endpoints under `/api/secrets`, gated by
  `requirePaid`. Mounted after the auth gate.
- Audit summary patterns added for the new routes.
- New Fleet › Secrets tab with bundle list, editor sheet (key=value rows,
  versions tab), and push wizard (selector, target stack, env file picker,
  per-node diff preview, results pills).
- Documentation: docs/features/fleet-secrets.mdx + docs.json nav entry.
- 26 Vitest cases cover parser, encryption, versioning, push aggregation,
  tier gating.

* fix(fleet): use const for rawValue in env parser

ESLint prefer-const flagged the let declaration as a CI-blocking error;
the variable is never reassigned.
2026-05-07 06:03:39 -04:00
Anso 52b46753af feat(fleet): add Federation tab with cordon and pin policy (Admiral) (#964)
Ships the v1 MVP for the Federation tab as placement control, not
placement automation:

- Cordon a node: marks the node unschedulable so the BlueprintReconciler
  skips it for new placements only. Existing deployments continue to
  drift-check and redeploy on revision changes; cordon never triggers
  withdraw or eviction. Toggle on the NodeCard kebab (Admiral, admin
  role); Cordoned pill renders for all tiers.
- Pin a blueprint to a node: stores blueprints.pinned_node_id, replacing
  the desired set with the pinned node regardless of selector. Pin
  overrides cordon by design. Action lives only in the Federation tab;
  BlueprintDetail and the deployment table show read-only Pinned
  indicators.

Backend: idempotent migrations add nodes.cordoned/cordoned_at/cordoned_reason
and blueprints.pinned_node_id. New routes POST /api/nodes/:id/cordon,
POST /api/nodes/:id/uncordon, PUT /api/blueprints/:id/pin, all gated by
requireAdmiral plus requireAdmin. Audit summaries added so the existing
auditLog middleware records every operator action. deleteNode clears
dangling pins.

Reconciler: pin override evaluated before selector match; cordon filter
applied only to the new-placement branch (deploy/stateReview without an
existing deployment). 11 new Vitest cases cover cordon filter, pin
override, pin-overrides-cordon, missing pin target, pin shrinks
desired set (stateless withdraw + stateful evict_blocked), and pin
clearing on node delete.

Frontend: new FederationTab.tsx with cordoned-nodes summary and
pin-policy table. Federation moved out of the experimental flag into
{isAdmiral && (...)} + AdmiralGate, mirroring the Routing tab pattern.
Secrets stays under experimental.

Tests pass: backend tsc, full Vitest suite (1704 passed), frontend
tsc -b, ESLint (0 errors). Manual verification via the local dev
instance confirmed the tab is hidden at Community, the kebab and pill
render at Admiral, and cordon and pin endpoints round-trip end to end.

Refs cut-line-1.0.md Federation v1 MVP.
2026-05-07 05:55:00 -04:00
Anso 77d5ff58d3 feat(fleet): add Fleet Actions tab for cross-node bulk operations (#963)
* feat(fleet): add Fleet Actions tab for cross-node bulk operations

Introduces a new "Actions" sub-tab in Fleet view with two Skipper+ cards
that fill gaps in the existing surface:

- Stop fleet by label: matches a label name across every node and stops
  every stack assigned to it, reporting per-node and per-stack results.
- Bulk label assign: applies the same label set to many stacks on one
  node in a single round trip.

Other bulk operations stay in their existing homes (sidebar bulk mode,
Schedules, NodeUpdatesSheet) to avoid duplicate surfaces.

Backend:
- POST /api/fleet/labels/fleet-stop (gateway-orchestrated, multi-node)
- POST /api/fleet-actions/labels/bulk-assign (per-node, capped at 1000)
- Tightens /api/fleet proxy-exempt prefix to /api/fleet/ so
  /api/fleet-actions/* is routed through the proxy for per-node calls.
- Exports activeBulkActions from labels.ts so fleet-stop and label-action
  share the per-node lock and cannot double-stop the same containers.
- Extracts containerActionForStack helper from stacks.ts for reuse.

* chore(fleet): rename Actions tab to Fleet Actions and reorder Fleet sub-tabs

- Tab label "Actions" -> "Fleet Actions" so the surface is unambiguous
  alongside Schedules and the sidebar bulk bar.
- Reorder Fleet sub-tabs as Overview / Snapshots / Status | Deployments /
  Traffic / Fleet Actions, with the separator after Status.
- Rename "Traffic · Routing" -> "Traffic" and update Sencho Mesh docs to
  match the shorter label.
- Update Fleet Actions docs to the new tab name and placement.
2026-05-07 05:41:53 -04:00
Anso 7fe90d9f3a feat(blueprints): capture compose snapshot before stateful eviction (#957)
Wire the snapshot_then_evict withdraw mode to actually persist the
blueprint's compose YAML to fleet_snapshots before running the
eviction. The mode previously recorded intent only. Capture failure
aborts the eviction with HTTP 500 rather than silently falling
through to a destructive withdraw.

Volume bytes remain out of scope: the snapshot holds the compose
definition only. UI copy and the Blueprints docs (Withdraw note,
Migrating stateful data section, two new Troubleshooting entries)
clarify that operators must move volumes by hand if they need the
data on another node.

Adds 9 route-level tests covering the success path, snapshot DB
write failure, orphan-row cleanup when insertSnapshotFiles fails,
empty compose_content, evict_and_destroy unchanged, stateless
unchanged, evict_blocked gate, omitted confirm field, and bad
confirm value.
2026-05-06 21:58:45 -04:00
Anso 0c3ce4b224 feat: implement file explorer context menus and dialogs (#934) 2026-05-06 08:46:02 -04:00
Anso 775fab7d64 feat(dashboard): replace duplicate Recent Activity card with Fleet Heartbeat / Stack Restart Map (#932)
* feat: open security basics, manual fleet ops, and basic fleet management to Community

Realign tier guards to the user-stated philosophy: Community covers
deploy/monitor at scale plus security basics, Skipper adds automation
and advanced fleet management, Admiral keeps enterprise control.

Community now includes:
- Trivy install / uninstall / update from the Settings Hub (admin role)
- CVE suppressions CRUD (admin role; replicates fleet-wide)
- Manual image scan with vuln, secret, and misconfig results
- Stack-config scan, scan comparison
- Manual fleet snapshots: create, list, view, restore, delete
- Per-node Sencho self-update (Check Updates + per-node Update)
- Fleet Overview search, sort, filters, node-card expand, auto-refresh

Stays paid:
- Scan policies with block_on_deploy enforcement (Skipper+)
- SBOM (SPDX, CycloneDX), SARIF export (Skipper+)
- Bulk Update All across the fleet (Skipper+)
- Scheduled snapshot create (now Skipper, was Admiral)
- Trivy auto-update toggle, fleet-wide policy push (Admiral)

The Settings -> Security tab is unhidden by setting the registry tier to
null. The SecuritySection no longer early-returns a PaidGate; the policy
list, Add Policy button, and policy dialogs are wrapped in {isPaid && }.
The Fleet view drops isPaid gates on the Snapshots tab, Check Updates
button, per-node update handlers, OverviewToolbar grid controls, the
NodeCard expand affordance, and the auto-refresh notice. The
NodeUpdatesSheet receives a canBulkUpdate prop and gates the Update All
button on it. useFleetUpdateStatus and useFleetPolling drop their isPaid
guards so polling runs for Community; useFleetOverview drops the isPaid
wrap on the filter and sort path.

Backend route guards are flipped per the matrix above. The scheduler
tick and requireScheduledTaskTier add 'snapshot' to the Skipper+ branch.
Backend test assertions are inverted for the now-Community endpoints
and a positive Skipper-snapshot-task test is added.

Documentation across features/, api-reference/, and operations/ is
updated to reflect the new tier mapping.

* feat: add node last-contact tracking, fleet latency, and stack-restart summary

- DatabaseService: add last_successful_contact column to nodes table via
  idempotent migration; expose updateNodeLastContact() and getStackRestartSummary()
  methods; include the column in NODE_COLUMNS so getNodes/getNode return it
- fleet.ts: record latency_ms and last_successful_contact on each remote
  node overview fetch; pilot-agent nodes surface pilot_last_seen instead;
  pass db singleton into fetchRemoteNodeOverview to avoid redundant getInstance calls
- dashboard.ts: replace /recent-activity with /stack-restarts endpoint that
  groups notification_history events by stack and category (crash/autoheal/manual)
  over a configurable window (default 7 days, max 30)

* refactor(dashboard): remove redundant per-route authMiddleware

All routes under /api/ are covered by the global auth gate in app.ts.
The inline authMiddleware arguments on /configuration and /stack-restarts
were redundant with that gate and inconsistent with every other route in
the file. Remove them and drop the now-unused import.

* refactor(backend): consolidate Date.now(), move SQL aggregation, normalize node row mapping

- Capture a single completedAt timestamp in fetchRemoteNodeOverview to
  eliminate two separate Date.now() calls and ensure latency_ms and
  last_successful_contact are derived from the same instant
- Inline the redundant contactedAt variable; use completedAt directly
- Move stack-restart aggregation from JS into SQL (GROUP BY stack_name
  with CASE/SUM counts), replacing the Map loop in the route handler
- Export StackRestartSummary interface from DatabaseService and remove
  the duplicate local definition in dashboard.ts; handler now returns
  the query result directly
- Add last_successful_contact normalization in decryptNodeRow, mirroring
  the existing pilot_last_seen pattern
- Add authGate reliance comment above dashboardRouter route handlers

* feat(dashboard): replace Recent Activity card with context-aware Fleet Heartbeat / Stack Restart Map

- Multi-node installs (≥1 remote node): shows Fleet Heartbeat — real-time
  reachability, latency, and container count per registered node
- Local-only installs: shows Stack Restart Map — 7-day restart frequency
  per stack grouped by crash / auto-heal / manual category
- Conditional wrapper (DashboardActivityCard) switches states automatically
  when the node list changes, with no page reload required
- Deletes RecentActivity card and hook (duplicated data already in Recent Alerts)
- Extracts formatRelativeTime to frontend/src/lib/utils.ts for reuse

* fix(dashboard): add pilot_last_seen to FleetNodeOverview and use it in getLastSeenLabel

* fix(fleet): expose mode and pilot_last_seen in overview, consolidate formatRelativeTime, drop em dash

- Add `mode` and `pilot_last_seen` (in seconds) to the FleetNodeOverview
  interface and to both the pilot-agent and HTTP-proxy return paths in
  fetchRemoteNodeOverview so the frontend getLastSeenLabel pilot branch
  can fire correctly
- Remove the private formatRelativeTime from RecentAlerts.tsx and use
  the shared implementation from lib/utils, converting the millisecond
  timestamp at the call site
- Replace the em dash in getLatencyLabel with 'n/a' per project rules
2026-05-05 15:23:05 -04:00
Anso ecf4dd5d52 feat: open security basics, manual fleet ops, and basic fleet management to Community (#930)
Realign tier guards to the user-stated philosophy: Community covers
deploy/monitor at scale plus security basics, Skipper adds automation
and advanced fleet management, Admiral keeps enterprise control.

Community now includes:
- Trivy install / uninstall / update from the Settings Hub (admin role)
- CVE suppressions CRUD (admin role; replicates fleet-wide)
- Manual image scan with vuln, secret, and misconfig results
- Stack-config scan, scan comparison
- Manual fleet snapshots: create, list, view, restore, delete
- Per-node Sencho self-update (Check Updates + per-node Update)
- Fleet Overview search, sort, filters, node-card expand, auto-refresh

Stays paid:
- Scan policies with block_on_deploy enforcement (Skipper+)
- SBOM (SPDX, CycloneDX), SARIF export (Skipper+)
- Bulk Update All across the fleet (Skipper+)
- Scheduled snapshot create (now Skipper, was Admiral)
- Trivy auto-update toggle, fleet-wide policy push (Admiral)

The Settings -> Security tab is unhidden by setting the registry tier to
null. The SecuritySection no longer early-returns a PaidGate; the policy
list, Add Policy button, and policy dialogs are wrapped in {isPaid && }.
The Fleet view drops isPaid gates on the Snapshots tab, Check Updates
button, per-node update handlers, OverviewToolbar grid controls, the
NodeCard expand affordance, and the auto-refresh notice. The
NodeUpdatesSheet receives a canBulkUpdate prop and gates the Update All
button on it. useFleetUpdateStatus and useFleetPolling drop their isPaid
guards so polling runs for Community; useFleetOverview drops the isPaid
wrap on the filter and sort path.

Backend route guards are flipped per the matrix above. The scheduler
tick and requireScheduledTaskTier add 'snapshot' to the Skipper+ branch.
Backend test assertions are inverted for the now-Community endpoints
and a positive Skipper-snapshot-task test is added.

Documentation across features/, api-reference/, and operations/ is
updated to reflect the new tier mapping.
2026-05-05 12:54:26 -04:00
Anso 49d775c61f feat(volumes): add read-only volume browser (#926)
* feat(volumes): add read-only volume browser

Adds a browser for the contents of any Docker named volume. Click the
folder icon on a volume row (admin only) to open a sheet with a directory
tree on the left and a file viewer on the right.

Backend
-------
New VolumeBrowserService spawns a one-shot Alpine 3.20 helper container
with the target volume mounted read-only at /v. The container runs as
nobody (65534:65534) with a read-only rootfs, no network, all caps
dropped, no-new-privileges, and capped at 64 PIDs and 128 MiB. The
helper image is pulled on first use per node.

Listing and stat use a portable busybox-compatible shell loop (find
-printf is not available on Alpine). Reads use head -c with an
explicit -- separator; the helper's working directory is /v so user
paths are passed as ./<path> argv elements and never as flags. The
container lifecycle is managed manually (create, attach, start, wait,
remove) to avoid the AutoRemove race where dockerode sees a 404 on
its post-exit container lookup.

Path safety: relative paths are sanitized server-side, rejecting
parent-escape segments, absolute paths, null bytes, and oversized
input. Symlinks are listed but never followed on read. Files larger
than 5 MB are truncated; binary content is detected via null-byte
scan and returned base64-encoded. Non-zero helper exits map to
404, 403, or 500 by classifying stderr.

Routes mounted at /api/volumes:
- GET /:name/list?path=
- GET /:name/stat?path=
- GET /:name/read?path=

All three require admin. The read endpoint always inserts an audit
log row (success or failure) with the actual response status code,
volume name, and relative path.

Frontend
--------
FileTree generalized to take a loadDir callback and a sourceKey
instead of a hard-coded stackName. The single existing consumer
(StackFileExplorer) was updated and its tests rewritten. The loader
is read through a ref so re-creating the arrow on every parent
render does not re-trigger the root fetch effect.

New VolumeBrowserSheet renders the tree against the volume API,
shows file content (hex view for binaries), and surfaces truncation.
Rapid sheet open and reopen on different volumes is generation-
checked to avoid stomping the visible result with a stale read.

A persistent footnote reminds the user that file reads are recorded
in the audit log, and the docs page warns about the typical contents
of database volumes.

Tests
-----
15 new vitest cases cover the pure helpers (path traversal, volume
name validation, binary detection). The Docker-facing exec path is
exercised by manual end-to-end via curl against a seeded volume.

* fix(volumes): truncate long volume names in browser sheet header

Wide volume names overlapped the close X. Reserve right padding on
the header, set min-w-0 on the flex title, mark the icon and refresh
button shrink-0, and truncate the name span.

* fix(volumes): satisfy lint on volume browser additions

prefer-const on sanitizeRelPath's local; drop unused FileTree entry
arg from the file-select callback (variance lets the arrow take fewer
params than the contract).
2026-05-05 00:09:40 -04:00
Anso 7e5dc2d9ea feat(resources): add image details sheet with layer history (#925)
Adds a read-only inspect panel for Docker images. Click the eye icon on
any image row to open a sheet showing:

- Overview: ID (with copy), size, created date, arch/OS, author, tags
- Config: Cmd, Entrypoint, WorkingDir, User, exposed ports, env (collapsible),
  labels (collapsible)
- Layers: ordered history list with size, age, and build command per layer.
  Empty layers (metadata-only) are dimmed.

Backend adds DockerController.inspectImage(id) which combines image.inspect()
and image.history() in parallel, exposed via GET /api/system/images/:id.
The route accepts both bare hex IDs and sha256-prefixed IDs, since the list
endpoint surfaces the prefixed form. Returns 400 for malformed IDs and 404
for missing images.

Documents the new panel in docs/features/resources.mdx under Images.
2026-05-04 23:45:54 -04:00
Anso e5b1c7b22b refactor(backend): collapse entitlement provider abstraction back to LicenseService (#889)
Removes backend/src/entitlements/ (registry, loadProvider,
CommunityEntitlementProvider, types, headers, normalize) and the two
abstraction-only tests. Relocates headers/normalize/types to
services/license-*.ts. Swaps 22 consumer call sites from
getEntitlementProvider() to LicenseService.getInstance(). Drops the
Dockerfile install step plus PRO_PACKAGE_VERSION build-arg and
github_token BuildKit secret in docker-publish.yml. Removes the now
stale no-restricted-imports rule in backend/eslint.config.mjs.

Net: 37 files changed, ~700 lines removed, no behavior change. Local
dev no longer requires GitHub Packages auth to start the backend.

Rationale and revisit conditions in
docs/internal/adrs/2026-05-02-collapse-entitlement-provider.md.
2026-05-02 23:45:44 -04:00
Anso d69fb9f1da feat(meta): gate deferred Fleet tabs behind SENCHO_EXPERIMENTAL flag (#886)
Hide the Traffic / Routing, Deployments, Federation and Secrets Fleet
tabs by default. They re-appear when the operator opts in by setting
SENCHO_EXPERIMENTAL=true. Backend routes and database tables are
unchanged; this is a UI discovery gate only.

The /api/meta endpoint now returns experimental as a boolean. A new
useExperimental hook reads it once per page load and feeds the four
tab triggers and tab content panels in FleetView.
2026-05-02 18:01:06 -04:00
Anso 4b18109286 refactor(entitlements): migrate type-only consumers to entitlements/types (#879)
Follows the Phase 1 EntitlementProvider abstraction. Two files
imported tier types from services/LicenseService via the back-compat
re-export added in Phase 1; this PR points them at the canonical
location at entitlements/types and drops the re-export block.

Migrated:
  - backend/src/types/express.ts
  - backend/src/routes/dashboard.ts

After this PR, services/LicenseService.ts has no public type re-
exports. The remaining imports of services/LicenseService are:
  - entitlements/loadProvider.ts: runtime import of the
    LicenseService class itself, the intentional Phase 1 binding
    site.
  - __tests__/license-service-id-validation.test.ts: imports
    SENCHO_LS_* catalog constants and resolveSenchoVariantFromMeta;
    these are LemonSqueezy-implementation-specific and stay in
    services/LicenseService until Phase 2 moves the file to
    @studio-saelix/sencho-pro.

Phase 2's deletion of services/LicenseService.ts now requires zero
public-core consumer changes outside the loader and the LS-specific
test file.

Test results: 89/89 backend test files clean, 1657 passing tests, 5
pre-existing skips, plus the same pre-existing database-metrics
stress test flake under parallel load that consistently passes solo.
2026-05-02 05:21:26 -04:00
Anso 3324616e59 refactor(backend): extract EntitlementProvider abstraction (Phase 1) (#878)
* refactor(backend): extract EntitlementProvider abstraction (Phase 1)

Phase 1 of the open-core hybrid extraction described in
docs/internal/adrs/2026-05-02-open-core-hybrid-strategy.md. Introduces
the abstraction without moving any code out of the public repo; Phase
2 will actually move services/LicenseService.ts to a private
@studio-saelix/sencho-pro package.

The new backend/src/entitlements/ module contains:

- types.ts. The EntitlementProvider interface plus all tier/license
  types (LicenseTier, LicenseVariant, LicenseInfo, SeatLimits,
  ActivationResult, etc.). The interface mirrors the existing
  LicenseService public surface so the migration was mechanical.

- registry.ts. Module-scope holder for the active provider with
  setEntitlementProvider, getEntitlementProvider, and a test-only
  reset helper. getEntitlementProvider throws if called before
  bootstrap registers a provider; the throw is intentional fail-fast
  on a bootstrap-order bug rather than a silent degradation.

- CommunityEntitlementProvider.ts. Phase 2 fallback that returns
  community tier and rejects activate(). NOT instantiated in
  production today; a smoke test keeps it covered against bitrot.

- loadProvider.ts. Async resolver. Phase 1 returns
  LicenseService.getInstance() directly. The async signature matches
  what Phase 2 needs (dynamic import of @studio-saelix/sencho-pro
  with a "module not found" vs "construction threw" narrowing); the
  call site does not change between phases.

- headers.ts. PROXY_TIER_HEADER and PROXY_VARIANT_HEADER constants.
  These are part of the wire contract between Sencho instances and
  belong in the public core regardless of which entitlement provider
  is bound.

- normalize.ts. isLicenseTier, isLicenseVariant, normalizeTier,
  normalizeVariant. Domain knowledge about Sencho's tier model
  (legacy name maps from pre-0.38.1 versions), not LemonSqueezy
  internals. Phase 2 keeps these in the public core.

services/LicenseService.ts now imports its types from
entitlements/types and adds an "implements EntitlementProvider"
clause. Re-exports the types for back-compat with ~20 type-only
consumers; a follow-up PR will sweep those imports to entitlements/
types directly before Phase 2 deletes the file.

bootstrap/startup.ts awaits loadEntitlementProvider, registers the
result, then calls initialize. shutdown.ts calls
getEntitlementProvider().destroy() instead of the LicenseService
singleton.

middleware/tierGates.ts, the chokepoint for ~154 tier-check call
sites, now reads through getEntitlementProvider. Sixteen other
production files (routes/{fleet,imageUpdates,license,permissions,
scheduledTasks,security,stacks,templates,users,webhooks},
services/{BlueprintService,CloudBackupService,SchedulerService,
SSOService}, proxy/remoteNodeProxy, websocket/{hostConsole,
remoteForwarder}, middleware/auth) had their LicenseService.getInstance
calls and utility-export imports redirected to the entitlements
module. The only remaining LicenseService.getInstance in production
code is in entitlements/loadProvider.ts itself, which is the
intentional Phase-1 binding site.

Test infrastructure: setupTestDb registers
LicenseService.getInstance() as the active provider so existing
test files using the helper need no changes. The mocking pattern
many tests use, vi.spyOn(LicenseService.getInstance(), 'getTier'),
keeps working because LicenseService.getInstance() and
getEntitlementProvider() return the same singleton in Phase 1.
scheduler-service.test.ts is the only test that does not use
setupTestDb but exercises tier-gating; it now mocks
entitlements/registry alongside its existing LicenseService mock.

Adds a smoke test for CommunityEntitlementProvider so the Phase 2
fallback class stays covered.

Adds an architecture doc at
docs/internal/architecture/entitlement-provider.md covering the
runtime registry, bootstrap order invariants, and the Phase 1 vs
Phase 2 binding table.

Test results: 89/89 backend test files pass, 1657 passing tests, 5
pre-existing skips. The pre-existing database-metrics > handles
1000+ metrics stress test continues to flake under parallel load
and pass when re-run solo, same flake observed in PRs #862, #863.

* chore(backend): drop unused entitlement type imports from LicenseService

Phase 1 of the EntitlementProvider extraction left five type imports
(ActivationResult, BillingPortalError, BillingPortalResult,
DeactivationResult, ValidationResult) unreferenced after the runtime
methods that produced them began inferring their result shapes via the
EntitlementProvider interface contract. ESLint's no-unused-vars rule
flagged them as errors and failed the lint step in CI.
2026-05-02 05:07:00 -04:00
Anso 685d5d729e feat(blueprints): backend foundation for fleet-wide compose templates (#860)
* feat(blueprints): add backend foundation for fleet-wide compose templates

Introduces the Blueprint Model: a docker-compose.yml plus a node selector
(labels or explicit IDs) that Sencho reconciles across the fleet. Backend
foundation only; the frontend tab and documentation follow.

Schema (DatabaseService):
- node_labels table for fleet-level orchestration tagging
- blueprints table with compose content, selector, drift_mode, classification
- blueprint_deployments table for per-node materialized state
- New idempotent migrate methods following the existing pattern

Services:
- BlueprintAnalyzer: pure compose-YAML classifier (stateless / stateful /
  unknown) with 17 covered cases including named volumes, bind mounts,
  external volumes, and tmpfs
- NodeLabelService: label CRUD plus selector matching helper (any/all/ids)
- BlueprintService: local + remote deploy/withdraw orchestration, marker
  file management, name-conflict guard, per-(blueprint,node) lock
- BlueprintReconciler: 60-second loop with three-mode drift policy
  (observe/suggest/enforce), state-aware guards, and Enforce-downgrade for
  volume-destroying drift

Routes (gated requirePaid + requireAdmin on mutations):
- /api/blueprints (CRUD + apply + withdraw + accept + preview + analyze)
- /api/node-labels (CRUD + listAll + listDistinct)

Notifications: four new categories registered in NotificationService for
deploy/failure/drift events.

Bootstrap: reconciler start/stop wired in startup and shutdown.

Tests: 45 new Vitest cases covering selector matching, classifier rules,
state-aware guards, drift-mode branching, and marker parsing. Full backend
suite (1625 tests) passes; tsc clean.

* fix(lint): replace bare Function type in blueprint reconciler tests

Replace 8 occurrences of `as unknown as { computeDecision: Function }`
with a properly typed `ReconcilerWithCompute` alias that mirrors the
real method signature. Export `ReconcileDecision` from
BlueprintReconciler so the test can reference it.

Resolves @typescript-eslint/no-unsafe-function-type errors that were
failing the Backend (Lint) CI step.
2026-05-01 18:57:44 -04:00
Anso 7663f4cd8b feat(fleet): sencho mesh in traffic and routing tab (#858)
* feat(fleet): sencho mesh in traffic and routing tab

Lights up Sencho Mesh: cross-node container forwarding rendered as if the
container next to you were on localhost. Builds on the dormant TCP frame
plumbing from the prior PR (pilot tunnel TCP frames + sencho-mesh sidecar
package) and exposes the Admiral-only orchestrator surface.

Backend
- New mesh_stacks table (per-node opt-ins) + nodes.mesh_enabled column
  via DatabaseService.migrateMeshTables.
- MeshService singleton: sidecar lifecycle via Dockerode, opt-in/out with
  cascading override regeneration, request-based resolver from sidecar
  control WS, cross-node TCP forwarding via PilotTunnelManager (same-node
  fast path included), in-memory 1000-event activity ring buffer with
  durable mirror to audit_log for state-change events, per-node and
  per-route diagnostics, and the Test upstream probe.
- MeshComposeOverride: pure YAML generator that injects extra_hosts using
  host-gateway. The user's docker-compose.yml is never mutated; overrides
  live under DATA_DIR/mesh/overrides.
- ComposeService deploy/update splice the override file when the stack
  is opted in; non-mesh stacks behave identically to today.
- Pilot agent resolveMeshTarget consults the local mesh_stacks table
  (defense in depth) and resolves Compose containers via Dockerode.
- /api/mesh router with 13 Admiral-gated endpoints covering status,
  enable/disable, stack opt-in/out, alias listing, per-route diagnostic,
  Test upstream probe, per-node diagnostic, sidecar restart, activity
  log paginated and SSE.
- meshControl WS slot at /api/mesh/control validates the mesh_sidecar
  JWT minted by MeshService; dispatched as upgrade slot 2 (canonical
  order preserved).

Frontend
- New Traffic Routing tab in FleetView, gated by isAdmiral and wrapped
  in AdmiralGate. Tab uses the cyan brand glyph and italic-serif state
  typography from the audit.
- RoutingTab masthead with mesh activity drawer, per-node card grid
  with TogglePill, alias rows with five-state pill taxonomy
  (healthy / degraded / unreachable / tunnel-down / not-authorized),
  inline Test buttons.
- Four sheets: opt-in picker with port-collision inline error,
  per-route detail with diagnostic + filtered activity, per-node
  diagnostics with active streams + resolver cache + restart action,
  fleet-wide activity log with filters.
- meshRouteState helper centralizes pill-state mapping; pure-function
  tests cover all five states.

Docs
- User docs at /docs/features/sencho-mesh.mdx covering opt-in,
  troubleshooting, security model (4 guarantees + 4 explicit
  non-guarantees), and V1 limitations.
- Internal architecture and runbook pages.
- websocket-dispatch internal doc updated with the new slot.

* fix(mesh): validate stack name before path use; fix test DB lifecycle

Two surgical fixes against the prior PR.

Path-injection (CodeQL js/path-injection): MeshService.optInStack,
optOutStack, ensureStackOverride, and removeStackOverride now validate
stackName via isValidStackName from utils/validation, reject malicious
names at the API boundary, and additionally check isPathWithinBase on
the resolved override file path for defense in depth. The dataflow from
req.params.stackName to fs.writeFile no longer reaches an unsanitized
path expression.

Test DB lifecycle: mesh-service.test.ts used per-test setupTestDb /
cleanupTestDb, which deletes the temp dir while DatabaseService still
holds an open SQLite handle. On Linux CI this raises
SQLITE_READONLY_DBMOVED on the next prepare() because the inode has
been unlinked. Switched to file-scoped beforeAll/afterAll matching
agents-routes.test.ts, with a per-test beforeEach that truncates
mesh_stacks plus non-default nodes and resets the MeshService singleton
in-memory state. Adds a new test case asserting the path-traversal
rejection.

* fix(compose): use discovered compose filename instead of hardcoded docker-compose.yml

composeArgs() hardcoded `-f docker-compose.yml` for every deploy. Sencho
writes its canonical compose file as `compose.yaml`, so any stack created
via the UI failed to deploy with `open ...docker-compose.yml: no such
file or directory`.

When no mesh override applies, drop the explicit `-f` so docker compose's
built-in discovery resolves the actual filename. When an override exists,
look up the real base filename via FileSystemService.getComposeFilename()
and pass both files explicitly.

Also hoist the MeshService import to module top now that the dependency
is known to be acyclic, and revert the matching unit-test assertion.
2026-05-01 01:50:53 -04:00
Anso 3e01daf76f feat(stack): per-stack activity timeline with actor attribution (#852)
* feat(stack): per-stack activity timeline with actor attribution

Adds an Activity tab to the Stack Anatomy panel showing a timestamped
event log for each stack: deploys, restarts, starts, stops, and image
updates, attributed to the user who triggered them or 'system' for
automated actions.

Backend:
- Extends notification_history with actor_username column (idempotent
  migration) and a partial composite index on (node_id, stack_name,
  timestamp DESC) for efficient per-stack lookups.
- NotificationService.dispatchAlert() accepts an optional actor that
  is written to the new column.
- Success-side dispatchAlert calls added after deploy, bulkContainerOp
  (start/stop/restart), and update handlers in routes/stacks.ts so
  user-initiated operations are recorded, not just failures.
- New GET /api/stacks/:stackName/activity?limit&before endpoint with
  stack:read permission gate and cursor-based pagination.

Frontend:
- StackAnatomyPanel grows an Anatomy / Activity tab pair using the
  existing Tabs primitive.
- StackActivityTimeline fetches the initial 50 events, paginates on
  demand, and prepends live events arriving over the existing WS
  notifications stream without duplicates.
- NotificationPanel bell dropdown suppresses user-initiated success
  events (start/stop/restart/deploy/update triggered by a real user),
  keeping the tray focused on alerts and system events.

* docs(stack): add stack activity timeline feature page and internal arch docs

* fix(test): add actor_username to notification-routing history assertions

dispatchAlert now passes actor_username to addNotificationHistory after
the activity timeline PR added the column. Update the two exact-match
assertions that were failing because the expected object shape was missing
this field.
2026-04-30 19:53:23 -04:00
Anso 7d4390a7e4 fix(backend): resolve ts-node dynamic import and TS2322 narrowing errors (#839)
* fix(backend): resolve ts-node dynamic import and TS2322 narrowing errors

* fix(backend): disable triple-slash reference lint error in convert.ts
2026-04-29 01:34:21 -04:00
Anso 219dee720e fix(convert): resolve TS7016 and TS2322 for composerize dynamic import (#837)
* fix(convert): resolve TS7016 and TS2322 for composerize dynamic import

* fix(convert): remove triple-slash reference banned by ESLint
2026-04-29 01:18:23 -04:00
Anso 329b4ec4e2 perf(backend): lazy-load composerize and isomorphic-git (#819)
Both modules are opt-in:
  - composerize (~2 MB) is only used by /api/convert when a user pastes a
    docker run command into the converter UI.
  - isomorphic-git plus isomorphic-git/http/node (~5 MB combined) only fire
    when a stack is created from a Git source.

Previously each was imported at module scope, parsing the whole package on
every cold start regardless of whether the feature was used. Wrap them in
small load-and-cache helpers so the first call resolves the module via
Node's loader and every subsequent call returns the cached reference.

The pattern matches the existing dynamic import of @aws-sdk/client-ecr in
RegistryService. Existing tests using vi.mock('isomorphic-git', ...) and
vi.mock('isomorphic-git/http/node', ...) keep working without changes
because dynamic and static imports share the same module registry.
2026-04-28 01:37:39 -04:00
Anso 836e384d17 perf(backend): cache global_settings reads in DatabaseService (#814)
getGlobalSettings() runs a SELECT * on every call and is hit from 22
files, including the auth middleware (every authenticated request),
the WebSocket upgrade handler (every connection), and the debug-mode
gate (every diagnostic log line). Cache the result inside the service
on first read and invalidate on updateGlobalSetting().

The cached snapshot is Object.freeze'd and the public return type is
now Readonly<Record<string, string>> so accidental mutations are
caught at compile time. The settings GET handler that delete'd private
keys now takes a defensive shallow copy first.

The 5-second TTL cache in utils/debug.ts is now redundant and removed;
the service-level cache is strictly fresher (invalidates on write
rather than going stale for up to 5s).
2026-04-27 23:45:25 -04:00
Anso ae8211c0b4 fix(fleet): forward main node tier to remote config fetch and hide local-only fields (#811)
The /fleet/configuration endpoint fetched remote node config via a direct
backend-to-backend fetch that omitted the distributed license headers
(x-sencho-tier, x-sencho-variant). Remote nodes evaluated their own
Community tier and returned locked: true for Webhooks, Scanning, and
Backup even when the main node held a Skipper/Admiral license.

Forward the same tier/variant headers that remoteNodeProxy already
injects so tier gates on remote nodes honour the main node's license.

MFA and Backup are also hidden for remote node cards in the Status tab:
- MFA is a user session feature managed on the main node; remote nodes
  are accessed via node_proxy Bearer tokens with no userId, so the value
  was always "Not set" and provided no useful information.
- Backup (Sencho Cloud Backup) runs fleet-wide from the main node and
  captures all nodes' compose files; remote nodes never configure it
  independently, so showing it there was misleading.

Removing both fields from remote cards keeps the grid at 6 items (3
even pairs) and eliminates stale or irrelevant data.
2026-04-27 15:47:51 -04:00
Anso 6ac02c792a fix(backend): use URL parser for registry scheme + template host check (#808)
Two small hardenings flagged by CodeQL:

- routes/registries.ts: drop the redundant startsWith block-list and rely
  solely on URL parsing + protocol allow-list. The startsWith pass was
  unreachable defense (any non-http/https scheme already fails the
  protocol check below it) and was tripping js/incomplete-url-scheme-check.
- services/TemplateService.ts: replace the .includes('api.linuxserver.io')
  substring match with new URL(registryUrl).hostname comparison. The
  substring form would mis-classify a malicious admin-set URL like
  https://evil.example/api.linuxserver.io/... as the LSIO registry and
  apply the LSIO response parser to its payload. Hostname compare closes
  that.

Behavioral parity for the happy path: every previously-accepted URL still
parses; the LSIO branch still triggers when the hostname is exactly
api.linuxserver.io.
2026-04-27 11:02:10 -04:00
Anso 4e5ba17710 refactor(backend): sanitize user input before logging to close CRLF injection (#807)
* refactor(backend): sanitize user input before logging to close CRLF injection

Adds a small sanitizeForLog helper that strips CR, LF, tab, and ASCII
control characters (0x00-0x1F, 0x7F) from a value before it is embedded
in a console.log/warn/error/debug call. Wraps every call site where a
user-controlled value (req.params, req.body, req.query, or a value
derived from them) flows into a log message.

Closes the bulk of the open CodeQL alerts in this family:
- 96 js/log-injection
- 28 js/tainted-format-string

The helper is in backend/src/utils/safeLog.ts. Routes still pre-validate
input at the request boundary; this is the second line of defense and
gives static analyzers a sanitizer they can trace through. JSON
responses, Docker filter labels, and other non-log call sites are
intentionally left unwrapped.

* refactor(backend): printf-style format strings for tainted-log call sites

CodeQL's js/tainted-format-string rule flags template literals in the first
arg of console.X when any interpolated value is user-controlled, regardless
of whether each value is sanitized inline. The canonical mitigation is to
use a static format string and pass values as positional args.

Converts the 28 flagged template literals to printf-style ("%s") format
strings, with sanitizeForLog applied to each positional arg. Also fills in
the log-injection wraps on 9 sites where a user-controlled value was
missed in the first sweep (agents, fleet, gitSources, imageUpdates,
GitSourceService).

No behavior change at runtime. Node's util.format substitutes %s tokens
identically to template-literal interpolation.

* fix(backend): wrap nodeId/snapshotId in fleet restore debug log

CodeQL flagged the unwrapped numeric args even though they cannot
contain control chars in practice. Apply the sanitizer for taint-flow
recognition.
2026-04-27 10:47:23 -04:00
Anso f037b435f3 refactor(backend): use stacksRouter.param for stackName validation (#799)
* refactor(backend): use stacksRouter.param for stackName validation

Registers a router-level param validator on :stackName so the 400
'Invalid stack name' guard runs once per route entry instead of being
duplicated in every handler. Removes ~22 inline isValidStackName checks
across the stacks router (deploy, down, env, files, services,
update-preview, rollback, backup, etc.).

Validation now fires before per-handler tier and permission checks,
which matches the standard input-validate-first pattern. The body-field
validators in POST / and POST /from-git remain inline because they
operate on req.body, not the route param.

Closes #752

* fix(stacks): remove unused stackName local in upload multer wrapper

The multer middleware wrapper for POST /:stackName/files/upload no longer
needs a local stackName binding now that param-level validation handles
the check. Removes the stale assignment that ESLint flagged and corrects
the leftover indentation on the requirePaid line.
2026-04-27 00:39:42 -04:00
Anso 4c352c74c8 refactor(backend): hoist parseIntParam helper to utils (#798)
Adds backend/src/utils/parseIntParam.ts with a shared parseIntParam
helper that writes a 400 'Invalid <label>' response and returns null on
non-numeric route params. Consolidates the parseInt + isNaN + 400 shape
that was inlined or duplicated across multiple routers.

Updated:
- routes/fleet.ts: replaced the local parseIdParam wrapper.
- routes/autoHeal.ts: replaced the local parsePolicyId wrapper.
- routes/notifications.ts: replaced parseRouteId wrapper plus an inline
  notification-id site.
- routes/apiTokens.ts, routes/labels.ts, routes/registries.ts,
  routes/scheduledTasks.ts, routes/users.ts: replaced inline copies.

Out of scope (route handlers without an existing isNaN check, kept
intentionally untouched to avoid introducing new 400 responses): alerts,
nodes, webhooks, and several user-routes handlers that rely on a
downstream 404 instead.

Closes #748
2026-04-27 00:39:28 -04:00
Anso add3abaece refactor(backend): replica guard helper for security routes (#797)
* refactor(backend): extract replica guard helper for security routes

Adds blockIfReplica(res, resource) in middleware/fleetSyncGuards.ts and
replaces six inline FleetSyncService.getRole() === 'replica' checks
across the security policies and CVE suppressions endpoints.

Error responses now use a uniform shape:
  403 { error: 'Cannot modify <resource> on a replica instance.
    Connect to the primary.', code: 'REPLICA_READ_ONLY' }

The new code field gives callers a stable discriminator without
matching prose.

Closes #750

* test(suppressions): match stable REPLICA_READ_ONLY code instead of prose

The replica guard helper exposes a stable code field for callers to
discriminate without grepping the human-readable error string. Switch
the replica-rejection assertion to use that code so the test no longer
breaks when the unified error template wording is tuned.
2026-04-27 00:29:38 -04:00
Anso 1747de1962 refactor(backend): extract bulkContainerOp helper for stack lifecycle routes (#796)
Collapses the three near-identical /:stackName/restart, /:stackName/stop,
and /:stackName/start handlers in routes/stacks.ts into a single
bulkContainerOp helper. Preserves the asymmetric notifyActionFailure
behavior (restart and stop notify, start does not).

Closes #751
2026-04-27 00:05:34 -04:00
Anso 8460ae9ede refactor(backend): hoist severity Sets to utils/severity.ts (#795)
Replaces five inline new Set([...]) literals scattered across
routes/security.ts and routes/fleet.ts with two shared constants
(FINDING_SEVERITIES, POLICY_SEVERITIES) exported from
utils/severity.ts. Pure refactor: no error response or status code
changes.

Closes #749
2026-04-27 00:05:12 -04:00
Anso 38a9f277c6 feat(stacks): add optional volume prune to delete confirmation (#788)
The Delete Stack dialog now includes an opt-in checkbox to also remove
associated Docker volumes when the stack is deleted. The checkbox is
unchecked by default and resets to unchecked on every open.

Backend: DELETE /stacks/:name accepts ?pruneVolumes=true and calls
pruneManagedOnly for volumes labeled with the stack project name after
bringing the stack down. Prune failure is non-fatal and logged; the
delete proceeds regardless.
2026-04-26 18:56:30 -04:00
Anso dcf8794047 feat(app-store): sort grid by stars and rotate featured weekly (#787)
Grid templates are now sorted by star count (descending) so popular
apps surface naturally rather than appearing in registry fetch order.

Featured hero rotates weekly among the top 5 starred apps instead of
always pinning the single highest-starred entry. Rotation is seeded
by week number so all nodes show the same featured app throughout a
given week. Registries with no star data gracefully skip the featured
hero and preserve their natural ordering.
2026-04-26 18:56:09 -04:00
Anso d7d8f9bfe8 feat(dashboard): replace 24h charts with Configuration Status and Recent Activity (#785)
* feat(dashboard): replace 24h charts with Configuration Status and Recent Activity

The 24-hour CPU/Memory area charts summed per-container metrics normalized
to each container's CPU quota, producing numbers that bore no honest
relationship to host load. The live ResourceGauges strip already shows
accurate host-level stats, making the historical charts both inaccurate
and redundant.

This commit replaces that row with two side-by-side cards:

- **Configuration Status**: aggregates every toggleable feature on the
  active node (notification agents, alert rules, routing rules, auto-heal,
  auto-update, webhooks, scheduled tasks, MFA, SSO, vulnerability scanning,
  cloud backup, and alert thresholds) into a single at-a-glance card.
  Tier-locked rows display an upgrade indicator instead of a value.
  Each row is clickable and navigates to the relevant settings section.
  Data refreshes every 60 s and immediately on state-invalidate events.

- **Recent Activity**: lists the ten most recent notification-history events
  for the active node (deployments, image updates, auto-heal actions, scan
  findings, cloud backup events, system notices) with category icons and
  relative timestamps. Refreshes every 30 s.

New backend endpoints:
- GET /api/dashboard/configuration - per-node feature status with locked/
  requiredTier markers so the frontend renders upgrade chips without extra
  calls. The endpoint sits after authGate and before the remote proxy so
  remote-node requests are transparently forwarded.
- GET /api/dashboard/recent-activity?limit=N - thin wrapper over
  DatabaseService.getNotificationHistory.
- GET /api/fleet/configuration - fleet-wide fan-out using the same
  Promise.allSettled dead-node-tolerant pattern as /fleet/overview.
  Exposed as the new "Status" tab on the Fleet page (after Snapshots).

Shared utilities:
- visibilityInterval and formatCount extracted to frontend/src/lib/utils.ts
  so the three polling hooks and two components share a single copy.

* docs(dashboard): fix stale alt text referencing removed historical charts
2026-04-26 18:42:28 -04:00
Anso 03f91cd5bb feat(cloud-backup): mirror fleet snapshots to S3-compatible storage (#782)
* feat(cloud-backup): mirror fleet snapshots to S3-compatible storage

Add an Admiral-tier Cloud Backup feature that replicates every fleet
snapshot to off-site storage, with two provider modes that share the
same `@aws-sdk/client-s3` code path:

- Sencho Cloud Backup: zero-config, 500 MB allowance backed by
  Cloudflare R2, provisioned via the sencho.io worker against the
  user's Lemon Squeezy license.
- Custom S3 (BYOB): any S3-compatible bucket (AWS, MinIO, Backblaze
  B2, Wasabi, R2 with own keys), with credentials encrypted via
  `CryptoService` before storage.

API-triggered snapshots upload fire-and-forget so the UI returns
immediately; scheduled snapshots block on the upload so the task's
success/failure reflects cloud durability. Object keys include the
instance_id segment to prevent collisions when the same Admiral
license is activated on multiple Sencho instances.

* fix(cloud-backup): drop ES2022-only Error cause arg breaking ES2020 build

The backend tsconfig pins lib to ES2020. The two-argument
`Error(message, { cause })` form requires ES2022, so tsc rejected it
with TS2554. Revert to single-argument throw to match the
convention used elsewhere in the backend services.
2026-04-26 15:42:21 -04:00
Anso 801a098a5b feat(files): per-stack file explorer (#780)
* feat(files): backend foundation for stack file explorer

Install multer for multipart file upload handling. Add
isValidRelativeStackPath to validation.ts to guard client-supplied
relative paths against traversal, absolute paths, NUL bytes, backslash
injection, and double-slash segments. Add isBinaryBuffer to a new
binaryDetect.ts utility for heuristic text/binary detection via
NUL-byte fast exit and non-printable byte ratio sampling.

* fix(files): reject bare dot segments in isValidRelativeStackPath

* feat(files): add safe stack-scoped file I/O methods to FileSystemService

Adds FileEntry interface and seven new public methods to FileSystemService
for stack-scoped file operations: listStackDirectory, readStackFile,
streamStackFile, writeStackFile, deleteStackPath, mkdirStackPath, and
statStackEntry.

Each method routes through a private resolveSafeStackPath helper that
enforces two-phase path containment: a pre-realpath lexical check plus a
post-realpath symlink-escape check. ENOENT targets are handled by walking
up to the deepest existing ancestor, realpaths that ancestor, and
reattaching the remaining suffix.

Binary detection delegates to isBinaryBuffer; path safety delegates to
isPathWithinBase. Protected file names and the MIME map are module-level
constants to avoid repeated allocation.

* feat(files): frontend API wrappers and Monaco language helper

* fix(files): tighten stackFilesApi error handling and localOnly support

* fix(files): FileSystemService safety and correctness fixes

* feat(files): add file explorer API endpoints to stacks router

* feat(files): FileTree and FileTreeNode components

* fix(files): route security hardening and stream cleanup

* fix(files): FileTree accessibility, icon stroke, stale fetch guard

Add strokeWidth={1.5} to all Lucide icons in FileTreeNode to match the
design system. Add aria-expanded to directory rows for accessibility.
Guard handleDirClick .then() callbacks against stale stack name
references when the component re-renders with a new stack. Add
toast.info fallbacks when compose.yaml or .env is clicked without a
navigation callback registered.

* feat(files): FileViewer, FileUploadDropzone, NewFolderDialog, DeleteFileConfirm

* fix(files): resolve code quality findings in file explorer components

- Move editorOptions useMemo above conditional returns in FileViewer (Rules of Hooks fix)
- Fix blob download: append anchor to DOM before click, defer URL revoke 100ms
- Keep protected-file confirm input visible during NOT_EMPTY recursive retry in DeleteFileConfirm
- Remove non-functional cursor-pointer/onClick from Community upgrade pill in FileUploadDropzone
- Add success toast on folder creation in NewFolderDialog
- Switch all (e as Error).message casts to instanceof Error narrowing

* test(files): unit tests for binary detection, stack path safety, and file explorer routes

- binary-detection.test.ts: covers isBinaryBuffer edge cases (empty, NUL,
  PNG header, threshold boundary, sampleBytes parameter)
- filesystem-stack-paths.test.ts: covers isValidRelativeStackPath (accepts/
  rejects matrix) and FileSystemService stack methods against a real temp dir
  (listStackDirectory sort and protection flags, readStackFile text/binary/
  oversized paths, writeStackFile/Buffer, deleteStackPath, mkdirStackPath,
  traversal guard); platform-specific empty-dir/NOT_EMPTY cases skip on Windows
- stack-files-routes.test.ts: route-level integration tests for all seven
  file explorer endpoints; covers auth gating, Community-tier 403 gates,
  input validation, 413 TOO_LARGE upload limit, and 204/200 happy paths

* feat(files): StackFileExplorer container with lazy tree, viewer, and action bar

* fix(files): add Download button to explorer toolbar, fix Community upgrade pill, reset state on stack change

* test(files): add missing test coverage for file explorer routes and service

* feat(files): add Files tab to EditorLayout with StackFileExplorer integration

* fix(files): add defensive activeTab guard to saveFile and discardChanges

* test(files): unit tests for FileTree expand/collapse and FileViewer render modes

Covers the three FileViewer content modes (text/Monaco, binary panel,
oversized panel) and the FileTree expand/collapse/cache cycle: first
expand fetches the subdirectory, second click collapses without a fetch,
third click re-expands from the in-memory cache without a second fetch.

* test(e2e): file explorer community and skipper+ flows

Covers the full file-explorer feature surface in two describe blocks:

Community (read-only): intercepts /api/license to simulate community
tier, confirms the upgrade pill is visible in the left pane, and
asserts that the Save button is absent after opening a text file.

Skipper+ (full CRUD): uploads a text file and confirms it appears in
the tree; edits config/app.conf and saves via Monaco; deletes an
uploaded file and asserts the tree entry is gone; issues a raw HTTP
request to the download endpoint and checks for status 200 and the
content-disposition: attachment header.

Also adds data-testid="file-action-delete" to the action bar Delete
button in StackFileExplorer for stable targeting, and exports
waitForStacksLoaded from e2e/helpers.ts to eliminate the three
identical local copies in stacks, deploy-log-panel, and stack-files
spec files.

* fix(e2e): improve test isolation and selector stability in stack-files spec

Move beforeEach seed to beforeAll/afterAll so fixtures are created once per
suite, not before every test. Extract shared seedSuite/teardownSuite helpers
to eliminate the duplicate beforeAll/afterAll blocks. Wrap teardown in
try/catch so failures log a warning rather than masking test results.

Replace waitForTimeout(500) with a deterministic expect on the file tree
sentinel. Add data-testid="anatomy-files-btn" and data-testid="delete-confirm-btn"
to replace the fragile button text/positional selectors. Assert Save button
starts disabled before editing.

* docs(files): add stack file explorer documentation

Add user-facing guide for the stack file explorer feature covering
tier access (Community read-only, Skipper+ read-write), viewing
limits, upload/download caps, protected file routing, and
troubleshooting. Update the editor page to reference the new guide
and register the page in the navigation.

* fix(docs): use canonical Skipper tier name in file explorer overview card

* fix(files): resolve lint errors blocking CI

Remove unnecessary backslash escape before double-quote in the
Content-Disposition regex (no-useless-escape). Replace five synchronous
setState resets at the top of the FileTree mount effect with a React key
prop on the FileTree element in StackFileExplorer so remounting resets
state automatically, eliminating the react-hooks/set-state-in-effect
violation.

* test(files): fix e2e seeding to work on community-tier CI

Replace the browser-side paid upload/mkdir API calls in seedTestStack with
direct Node fs writes. The upload and folder endpoints require Skipper+ so
they returned 403 on CI, which runs with no license set. Stack creation
via POST /api/stacks stays as an API call since it is community-allowed and
keeps the backend registry in sync.

Add a per-test tier check in the Skipper+ beforeEach that skips gracefully
when the instance is community, matching the pattern in auto-heal-policies.
2026-04-26 13:05:19 -04:00
Anso dd9d33813b feat(deploy-logs): opt-in deploy progress modal with structured log rows (#779)
* feat(notifications): dispatch deploy_failure alert on stack action errors

* feat(terminal): add onReady and onMessage callback props

* feat(deploy-logs): add DeployLogContext with runWithLog API

* feat(deploy-logs): add DeployLogPanel bottom drawer with resize and minimize

* feat(deploy-logs): wire DeployLogContext to App and EditorLayout action runners

* test(deploy-logs): add E2E test for deploy log panel open, failure, and minimize

* docs(deploy-logs): add user-facing and internal architecture docs

* feat(deploy-logs): redesign as opt-in modal with structured log rows

Replace the full-width bottom drawer (DeployLogPanel) with a centered
modal that streams structured log output for deploy, stop, restart,
update, install, and Git apply operations. The modal is disabled by
default; users opt in from Settings -> Appearance.

Core changes:
- New DeployFeedbackContext with runWithLog() API: if opt-in is off,
  silently bypasses the UI so all call sites degrade to the existing
  toast behavior without code changes.
- composeLogParser.ts: pure parser that strips ANSI escapes and
  classifies compose output into stage badges (PULL, BUILD, CREATE,
  START, STOP, DOWN, WARN, ERR, LOG). 15 unit tests.
- StructuredLogRow.tsx: memoized row with timestamp, stage badge, and
  message. Error rows get a rose left rail; warn rows get a tinted bg.
- DeployFeedbackModal: Dialog-based, max-w-640px/max-h-70vh, elapsed
  timer, auto-close 4s on success (hover cancels), persistent on
  failure. Raw xterm output collapsible in footer.
- DeployFeedbackPill: minimized state anchored top-right, survives
  navigation, click restores modal.
- Wires App Store install (action: install), Git apply (action: deploy),
  and Git pull (action: update) in addition to the existing EditorLayout
  actions.
- Fixes Terminal.tsx WS URL in generic mode (was connecting to root path
  not proxied by Vite; now uses /ws).
- Settings: adds "Show deploy progress modal" checkbox to Appearance.
- Docs: renames deploy-logs.mdx to deploy-progress.mdx; updates
  internal architecture doc.

* fix(deploy-logs): connect Terminal in generic mode and move pill to bottom-center

Terminal was passed stackName which routes it to the stack logs WS
(container stdout). In that mode onReady is never called, so the
deployStarted gate never resolves and the compose command never runs.
Remove stackName so Terminal uses generic WS mode, which calls onReady
on open and streams compose output.

Also reposition the minimized pill from top-right to bottom-center
(fixed bottom-6 left-1/2 -translate-x-1/2) per UX feedback.

* docs(deploy-logs): update pill position to bottom center

* test(deploy-logs): rewrite E2E spec for deploy feedback modal

The old spec targeted the removed bottom-drawer DeployLogPanel and used
the wrong field name when calling POST /api/stacks (sent 'name' but the
endpoint reads 'stackName'), causing every test to fail with a 400 before
any UI assertions ran.

Fixes:
- POST /api/stacks body now uses 'stackName' matching the API contract
- All locators updated to target the new DeployFeedbackModal and
  DeployFeedbackPill components (data-testid attributes added)
- Added enableDeployFeedback helper to opt-in via localStorage before
  each test that expects the modal (feature is off by default)
- Added opt-in OFF test to confirm the modal is suppressed when disabled
- Minimize/expand test now asserts the pill appears and contains the
  stack name before clicking to restore the modal

* test(deploy-logs): fix compose file write endpoint in E2E helper

createStackViaApi was calling PUT /api/stacks/:name/files/docker-compose.yml
which does not exist. The correct endpoint is PUT /api/stacks/:name with
{ content } in the body.

* test(deploy-logs): use addInitScript to persist opt-in across reloads

The opt-in flag was set via page.evaluate before setupDeployStack, which
calls page.reload() and loginAs (a second navigation). Although localStorage
should persist across same-origin reloads, the React tree was reading
'false' on remount in CI. Switching to addInitScript guarantees the
localStorage value is set before any page script on every navigation, so
useDeployFeedbackEnabled's useState initializer always sees the right
value when React mounts.

* test(deploy-logs): verify localStorage and re-dispatch event before deploy

Adds syncDeployFeedbackState() called right before each deploy click in
the ON tests. It both verifies localStorage is set (failing the test
loudly with a clear message if not) and re-dispatches the
SENCHO_SETTINGS_CHANGED event to defeat any stale React state after
navigation. If the modal still does not appear with the assertion green,
the issue is downstream of localStorage and we have a clear signal.

* test(deploy-logs): wait for React re-render after dispatching opt-in event

After syncDeployFeedbackState dispatches SENCHO_SETTINGS_CHANGED, React
schedules the state update but does not flush it synchronously. The
click that follows can fire against the stale closure where isEnabled is
still false, so runWithLog takes its early-return path and the modal
never opens. A 200ms wait is enough to let React commit the new state
before the next interaction.

* test(deploy-logs): wait for stack file fetch before clicking deploy

deployStack() in EditorLayout returns early at 'if (!selectedFile)'
without calling runWithLog. selectedFile is set inside loadFile() after
GET /api/stacks/:name resolves. The previous setup clicked the stack in
the sidebar and immediately asked the test to click Deploy, racing the
fetch. CI backend logs confirmed no deploy POST ever fired for the ON
tests, while the OFF test passed only because it asserts non-existence.

Now setup awaits both the stack click and the file response together,
then verifies the action bar's deploy button is visible before returning.

* test(deploy-logs): wait for network idle and capture browser logs

Adds a networkidle wait plus a 500ms settle after the stack click so
React commits selectedFile and any follow-up env/container/backup
fetches drain before the deploy click. Also mirrors browser console
errors and pageerrors into the Playwright output so the next failure
ships with the React stack trace instead of just a 'modal not visible'
message.

* test(deploy-logs): temporary debug logging in runWithLog

Adds a console.log at the entry of runWithLog so we can see in CI logs
whether it is being called and what isEnabled value the closure has.
Also widens the test's console capture to include these debug lines.

This is diagnostic only and will be removed once the root cause of the
modal-not-opening-in-CI failure is identified.

* test(deploy-logs): debug log at deployStack entry to trace click path

Adds console.log at the first line of deployStack handler so we can
confirm in CI whether the click is reaching it at all and what
selectedFile/isStackBusy resolve to. Combined with the existing
runWithLog debug logs, this isolates whether the modal failure is in
deployStack guarding out, runWithLog early-returning, or something
else entirely.

* test(deploy-logs): drop filter, log every browser console msg

The previous filter only emitted error/warning plus the deploy-feedback
substring. The deploy-feedback debug logs never appeared, so we don't
yet know whether the log itself is firing. Remove the filter so the
full console stream shows up in CI.

* test(deploy-logs): app-level console log to verify capture pipeline

If even an unconditional log at App component render time does not
appear in CI browser logs, then the console capture listener is broken
or the dispatched logs are being filtered upstream of Playwright. This
isolates whether the issue is in the production code or the test
harness.

* test(deploy-logs): use testid locator for stack action button

Replaces the regex-based getByRole locator (/Deploy|Start/i) with
getByTestId('stack-deploy-button'). The regex matched something other
than the actual deploy button: backend logs proved no deploy POST ever
fired, and instrumentation confirmed neither deployStack nor runWithLog
ran on click despite the test claiming success.

Adds data-testid='stack-deploy-button' to both the Restart and Start
button branches in EditorLayout's action bar so the same locator works
whether the stack is running or not.

Also drops the temporary debug console.log entries in deployStack,
runWithLog, and App, and restores the test's console listener filter
to only emit error and warning messages.

* test(deploy-logs): park cursor in corner so auto-close countdown fires

After clicking the deploy button, the cursor lands inside the centered
modal. The modal pauses its 4s auto-close countdown on hover, so the
HAPPY test was waiting for a close that never happened. page.mouse.move
to (0,0) parks the cursor outside the modal before the success banner
appears, letting the countdown complete.

* test(deploy-logs): drop redundant loginAs after page.reload

page.reload preserves auth cookies, so the page lands back on the
dashboard without needing a fresh login. The loginAs call after reload
was racing on isLoginPage(): a transient login-page state during page
load made loginAs commit to filling #username, then the dashboard
committed and #username never came back. Playwright's auto-wait then
hung the fill until the test's 120s timeout, which also dragged later
stacks.spec tests down with collateral timeouts.

waitForStacksLoaded is enough to confirm we're on the dashboard with
the sidebar populated before clicking the new stack.

* test(e2e): make loginAs race-safe when login page is a false positive

isLoginPage() reports the page as a login screen if the Login button
locator reports visible at the moment of the check. Under CI load (more
real container deploys from the deploy-log-panel suite), the auth
context can render the login form for one paint, then redirect to the
dashboard. The original code committed to filling #username and hung
until the test timeout when the field was no longer there.

Now the login branch waits up to 2s for #username to actually appear
before filling. If it never appears, we fall through to the dashboard
check instead of hanging.
2026-04-26 00:43:54 -04:00