* feat: move core Blueprint orchestration to Community tier
Blueprints CRUD, reconciliation, and drift modes are now available on
Community. Pin remains Admiral-only via Federation placement controls.
* test: update NodeCard cordon tests for Admiral-only tier gate
Cordon now requires both isPaid and node:manage permission, matching the
backend requirePaid + requirePermission guard. Three tests still used
isPaid:false but expected the menu to be visible.
Detect services with build: in the update preview and run compose build --pull
plus pull --ignore-buildable when Update is triggered on those stacks, while
keeping the existing pull-only path for image-only stacks.
* fix: enforce 1:1 compose path mapping for Pilot agent mounts
Pilot enrollment now generates validated 1:1 bind mounts so every
agent path maps to a unique compose directory. Persisted agent paths
reconcile during startup to catch drift. Unsafe relative-bind redeploys
are blocked before container removal to prevent path escapes.
- Add composePathMapping utility with strict path validation
- Generate COMPOSE_DIR and validated mounts during Pilot enrollment
- Reconcile persisted agent paths during startup bootstrap
- Block redeploy when a relative-bind mount would escape the compose root
- Default Pilot UI path to /opt/docker/sencho
- Update multi-node and pilot-agent documentation
- Add regression tests for enrollment, bootstrap, compose-service,
and environment-check paths
* fix: update E2E enrollment regexes for YAML-quoted token values
* feat: show container name in structured log output
Prepend a normalized container name prefix to each line in
ComposeService.streamLogs() so both the structured log viewer
and the raw terminal identify which container produced each entry.
- Backend: prepend displayName (normalized via normalizeContainerName)
before LogFormatter.process() in sendOutput and flushBuffer.
- LogFormatter: refactor process() to handle both prefix-first and
timestamp-first input orders via a while-loop; widen PREFIX_REGEX
to accept dotted service names.
- Frontend: add containerName to LogRow, extract prefix in parseLine,
render as an inline mono chip in the message column, and include
the name in downloaded logs (omitting the bracket prefix when null).
- Tests: 14 new tests across log-formatter, compose-service streamLogs,
and StructuredLogViewer chip rendering + download formatting.
* fix: guard LogFormatter loop to at most one prefix and one timestamp
The while-loop refactored for order-agnostic prefix/timestamp
parsing could continue matching beyond the intended single prefix
and timestamp. A log line like "redis | 2024-...Z api | started"
would falsely colorize "api |" as a second container prefix in
raw terminal output.
Add prefixFound/timestampFound boolean guards so the loop stops
after one prefix and one timestamp, regardless of input order.
* feat: per-service color alternation for log container chips
Add an Appearance setting that lets users switch between unified
cyan and per-service label-token colors for the container name chips
in the structured log viewer.
- Extract HUE_VARS and hashLabel() from NodeLabelPill into a shared
utility at frontend/src/lib/label-colors.ts.
- Add useLogChipColorMode hook (browser-local localStorage,
sencho.log-chip-color-mode key, unified by default).
- Add SegmentedControl in Settings > Appearance > Display.
- Apply inline label-token styles via style attribute in per-service
mode; keep current text-brand/80 bg-brand/10 classes in unified mode.
- 14 new tests across label-colors, hook, and viewer chip rendering.
* feat(security): surface Compose internet-reachability exposure in posture
Builds a per-stack per-service exposure descriptor from the rendered
effective Compose model, cached at deploy/update time, and joins it into
the Security action posture. A service is publicly exposed when it
publishes a port on a non-loopback host IP or uses host networking.
The exposure cache lives in a new stack_exposure table, refreshed inside
ComposeService.deployStack and updateStack (covering all funneled paths:
manual, scheduler, mesh, templates, labels, App Store, Git, webhooks).
Cleanup runs on stack delete, blueprint withdrawal, and node delete.
The overview route intersects the exposed image set with the existing
per-image suppression-aware Critical/High tally, so a clean public
nginx does not escalate posture. The scan sheet shows a "Published
service" or "Internal only" evidence badge per image.
* fix(test): provide fresh auto-close proc for exposure spawn in stall tests
Two deployStack idle-stall tests used mockSpawn.mockReturnValue(proc)
which returned the same already-closed process for the new config spawn
added by the exposure refresh. The renderConfig promise hung waiting for
a close event that had already fired.
The fix uses mockImplementation to return the controlled proc for the
first spawn (up) and a fresh auto-closing proc for the second spawn
(config via refreshExposureCache).
* fix(security): tighten loopback detection, clarify exposure semantics, drop internal-only badge
- Expand isLoopback to cover full 127.0.0.0/8 range (127.0.0.2 etc)
- Clarify that exposure is configured (Compose model), not live topology
- Remove "Internal only" badge: false is not proof of non-exposure when
other stacks using the same image may lack a cached descriptor
When a single-file stack is opted into Sencho Mesh, the deploy builds an
explicit `docker compose -f <base> -f <mesh override>` list. Passing any
explicit -f disables Compose's automatic discovery of compose.override.yml
(and the docker-compose.override variants), so a user's hand-authored
override was silently dropped from the effective deploy once Mesh was on.
Resolve the user's override file (first existing variant, with the same
stack-name and symlink-containment guards as the base compose file) and
insert it between the base and the mesh override, so it layers exactly as
Compose's implicit discovery would, with the mesh override still taking
precedence. A transient read failure during the lookup degrades to "no
override" rather than failing the deploy; a stack-name or containment-guard
rejection still aborts. Multi-file Git-source stacks and non-mesh deploys
are unaffected.
* fix(drift): reconcile the drift ledger on deploy and timestamp its history
The drift ledger (persisted history + activity timeline) only advanced
when someone clicked re-check on a stack's Drift tab, so the history could
sit indefinitely out of sync with the live status: a stack reading
"drifted" live while its history still said "resolved". Two corrections:
- Deploy and update reconcile the ledger against the just-deployed runtime
(the rollback route re-deploys through deployStack, so it is covered),
resolving what the change fixed and recording what it left.
- Every authoritative reconcile stamps the dossier last-checked time, and
the Drift tab labels its history "checked {time}" so a stale finding
reads as history, not a claim about the live status above it.
Adds the last_drift_check_at column and tests across the ledger reconcile
stamp, reconcileStack, the deploy hook, and the panel.
* fix(drift): stamp last-checked inside the ledger transaction
Move the dossier last-checked stamp into the same transaction as the
finding insert/resolve, so the "checked {time}" the Drift tab shows can
never persist without the ledger update it describes. The stamp still runs
on a no-op authoritative check (a transaction that only stamps), keeping
the history "as of" honest. Adds a test that a failed deploy does not
reconcile the ledger.
* feat(stacks): per-stack environment inventory and secret-safe guardrails
Add an Environment tab to Stack Anatomy that derives a per-stack inventory
of environment variables from the compose files and env files. Each variable
shows its source, whether Compose interpolates it or injects it into a
container, and a status (present, missing, unused, duplicate, or shell-only),
plus likely-secret classification. The inventory works from variable names
only: a value is never read, returned, or logged, and a likely secret shows
presence only. A copy env checklist action exports names and status without
values.
Surface a missing required env_file as a Compose Doctor preflight finding,
and add an opt-in node setting that refuses a deploy or update when a
required ${VAR:?...} variable is unset or empty, before any backup, pull, or
up runs. Default off.
The Environment tab is capability-gated so it hides on older remote nodes.
* fix(stacks): harden env-file reader against a stat-then-open race
Open the env-file handle first and fstat the open handle instead of
stat-ing the path before opening, removing the check-then-use window in
readEnvFileKeys. Use a secure mkdtemp directory for the out-of-base test
path instead of a predictable name in the temp root.
* fix(stacks): resolve nested env_file paths per compose file, reconcile inline keys per service
Resolve each env_file relative to the directory of the compose file that
declared it, so a nested multi-file Git override (infra/prod.yml referencing
./prod.env) lands next to that file instead of the stack root. The root
compose file is unaffected, since its directory is the stack directory.
Reconcile inline environment provenance per service, so a key an override
removed from one service's effective env is not labeled compose-inline just
because another service injects the same name from a different source.
* fix: resolve the root .env at deploy and render time for Git context-dir stacks
A Git multi-file source with a context dir set --project-directory to that
dir, so Docker Compose looked for .env there and missed the root .env Sencho
writes. Validation already passed the root .env with --env-file, so a stack
could validate with one effective config but deploy or render another.
Add authoredComposeEnvFileArgs, which appends --env-file <stackDir>/.env when
the applied deploy spec has a context dir and a root .env exists, and wire it
into the deploy/update, image-scan, render, and container-listing compose
invocations so they all resolve env from the same file the validator used. A
non-ENOENT access error surfaces instead of silently dropping the flag.
* fix: base multi-file Git dossier and doc-drift on the effective Compose model
The Stack Dossier and its documentation-drift check parsed only the stored root
compose file. For a multi-file Git source, services, ports, networks, or volumes
that an override file adds were invisible, so the dossier showed incomplete facts
and doc-drift could falsely warn that a documented port is unpublished when an
override actually publishes it.
Add a secret-safe GET /stacks/:name/effective-anatomy that renders the merged
effective model and extracts only structural facts (services, ports, volumes,
networks, restart), never env, label, or command values. StackAnatomyPanel
fetches it for multi-file Git stacks and feeds those facts into the dossier and
doc-drift, falling back to the root-only parse for single-file or non-git stacks
and whenever the render is unavailable.
* fix: add an inline path-injection barrier to the Git env-file resolver
CodeQL js/path-injection flagged the fs.access in authoredComposeEnvFileArgs
because the env path derives from the route-supplied stack name and the only
containment check lived in the callers, not at the sink. Resolve the stack dir
against the compose base and assert containment with startsWith inline, then
derive the .env path from the validated dir, mirroring the existing inline guards
in renderConfig and validateCompose. Valid stack names are unaffected; a name
that escapes the base now yields no --env-file.
* test: stabilize the dossier doc-drift e2e against the dossier-load race
The first assertion filled the access_urls field as soon as the Dossier panel
was visible, but the panel's GET /stacks/:name/dossier resolves by overwriting
the fields from the server (empty access_urls) and only then flips the doc-drift
gate on. When the GET landed after the fill, it clobbered the typed value and the
warning never rendered, so the test failed intermittently under CI timing. Wait
for that GET to land before typing, mirroring the spec's openStack helper.
* fix: harden deploy/update concurrency and node-targeting safety
Release stabilization for deploy/update operational safety.
Per-stack operation locking is now global. Background lifecycle paths
(scheduler auto stop/down/start/backup/update, webhook execute, Git source
auto-deploy, image auto-update, label bulk actions, fleet snapshot redeploy,
and mesh redeploy) acquire the per-node, per-stack lock through a new
StackOpLockService.runExclusive helper and skip rather than race a manual
deploy/update/rollback/backup on the same stack and node. Skips surface
honestly (a failed scheduled run, a recorded webhook failure, a per-stack
batch result, or a thrown error) instead of a silent no-op.
Update readiness and policy-bypass now run against the node captured when the
dialog opened, not the live active node, so switching nodes while a dialog is
open cannot retarget the update or the bypass retry.
Rollback readiness no longer presents a moving-tag or unpinned image as a ready
image revert. Restoring files does not revert a moving tag, so those stacks
read as partial, and the rollback success message states that the compose and
env files were restored.
* fix: lock blueprint reconcile against manual ops and correct rollback wording
Follow-up to the deploy/update safety hardening, closing two more gaps from a
verification pass.
BlueprintService.deployLocal and withdrawLocal called ComposeService directly,
so blueprint reconciliation could race a manual deploy/update/rollback/backup on
an owned stack. Both now run their compose lifecycle call through
StackOpLockService.runExclusive and skip (recorded as a failed reconcile,
retried on the next cycle) on conflict. The withdraw holds the lock across both
the compose down and the directory delete so neither races a manual operation.
The runtime rollback messages overstated recovery: a rollback restores the
compose and env files and recreates containers, but does not revert an image
behind a moving tag. The auto-rollback deploy-progress output, the recovery
panel and chip, the failure toasts, and the manual rollback route message now
state that the compose and env files were restored, with the matching OpenAPI
example and atomic-deployments doc updated.
* fix: acquire stack lock before blueprint deploy mutates compose and marker files
Local blueprint deploy wrote the compose and marker files and ran the policy
assert before acquiring the per-stack lock; the lock only wrapped the deploy
itself. A reconcile could therefore rewrite an owned stack's files while a
manual deploy/update/rollback/backup was running. The lock now wraps the whole
critical section (create, write compose, write marker, policy assert, deploy),
so on conflict nothing is written and the reconcile records a failed outcome.
Adds a test asserting a deploy under a held lock records failed, writes no
marker file, and leaves the manual lock untouched.
* fix: make remote blueprint apply atomic under the receiving node's stack lock
Remote blueprint deploy wrote the compose and marker files to the target node
via separate HTTP calls and only locked on the final deploy, so the file writes
could race a manual operation on that node. A node's operation lock is
process-local and cannot be held by the hub across HTTP calls, so the locked
create/write/deploy now runs on the receiving node.
The locked critical section is extracted into BlueprintService.applyLocalUnderLock
and exposed via POST /api/blueprints/apply-local. The hub posts the blueprint to
that endpoint in one call; the receiving node runs create + write compose+marker
+ deploy under its own per-stack lock. Older nodes without the route answer 404
and fall back to the legacy multi-call flow. The endpoint is gated by paid tier
and the same per-stack stack:edit and stack:deploy permissions as the
PUT-compose + deploy it bundles, validates the stack name, compose size, and
marker structure, and returns 409 on a lock conflict without writing anything.
Adds tests for the atomic single-call path, the 404 legacy fallback, the 409
lock-conflict mapping, the route validation and permission paths, and the
write-compose-then-marker-then-deploy ordering of the shared locked apply.
* fix(deps): bump undici to 7.28.0 to clear high-severity advisory
The frontend CI npm audit gate (--audit-level=high) failed on a transitive
undici 7.25.0 (a dev-only dependency via jsdom): TLS certificate validation
bypass (GHSA-vmh5-mc38-953g) and cross-user cache information disclosure
(GHSA-pr7r-676h-xcf6). Bumping undici within jsdom's existing ^7.25.0 range to
7.28.0 clears the high-severity advisory and unblocks the frontend job. Lockfile
only; no direct dependency or source change.
* feat: ordered multi-file Compose for Git sources
Extend Git sources to deploy an ordered list of compose files merged with
docker compose -f base.yaml -f override.yaml ..., plus an optional project
directory.
- Pick and reorder compose files from the repository tree (drag to reorder on
desktop, up/down arrows on phones); manual path entry is also supported.
- The ordered set drives every stack-scoped compose command (deploy, update,
start/stop/restart/down, image scans, Compose Doctor) and the container
lookup, so a service or image declared only in an override is handled too.
- Runtime keys off the materialized set, not the saved configuration: saving a
source does not change deploy args until the pull is applied, and apply
materializes from the pending snapshot rather than live config.
- The project directory is passed as --project-directory, with -p <stack>
pinning the Compose project so container labels stay stable.
- The Mesh override is layered last; single-file sources are byte-identical to
before, and existing rows keep working via the single-path fallback.
Docs cover the picker, ordering, project directory, and the new troubleshooting
and limitations (referenced files are not materialized; the dependency graph,
drift, and networking views read the primary file).
* fix: harden multi-file Git source (hash, unlink, collisions, node id)
- hashContent folds ordered file CONTENTS (not paths) so a clean multi-file
stack is not flagged as locally edited: create/apply hash the fetched files
(repo paths) while pull hashes the on-disk files (materialized paths), which
previously disagreed and showed a false "local edits detected".
- Block unlinking a multi-file or project-directory Git source (409): the deploy
spec lives on the source row, so removing it would silently revert deploys to
root compose.yaml. Single-file sources still unlink.
- Reject materialized-path collisions in the selection validator: an additional
file equal to or nested under compose.yaml, an ancestor/descendant overlap
between selected files, and a project directory nested under a compose file
(previously a 500 at materialization).
- DockerController.getContainersByStack uses the controller's node compose dir
and passes its node id to the authored prefix, instead of the process default.
* fix: CI failures on multi-file Git source (test crash, aria query, path barrier)
- GitSourceFields no longer crashes when repoUrl/branch are falsy: the canBrowse
trim() is optional-chained, so a reusable field component tolerates partial
props. Fixes the apply-binding panel test, which feeds a minimal source object.
- GitSourcePanel tests query the footer Remove button by its exact name, so the
picker's per-file "Remove <path>" buttons no longer collide with the broad
/remove/i match (the test intent, footer Remove present/absent, is unchanged).
- validateCompose uses an inline resolve + startsWith barrier at the context-dir
mkdir sink (CodeQL does not credit the wrapped isPathWithinBase helper),
clearing the js/path-injection alert. The containment check is equivalent and
contextDir is also validated upstream.
* test: update Git source E2E spec for the multi-file compose picker
The compose-file picker replaced the single #git-source-path input and added
per-file Remove buttons, so the E2E spec drove selectors that no longer exist:
- Drop the redundant compose.yaml fills (the picker defaults to compose.yaml).
- Select the footer Remove button by exact name so the picker's per-file
"Remove <path>" buttons no longer make the locator ambiguous.
- Set a custom compose path through the picker (add via the manual input, press
Enter, then remove the default compose.yaml).
* test: match the footer Remove button with an exact Playwright name
Playwright's getByRole name option is a substring match by default, so
{ name: 'Remove' } also matched the picker's "Remove <path>" buttons. Require an
exact match so only the footer Remove button is selected.
* feat: add Compose Doctor preflight checks for stacks
Add an on-demand, advisory preflight that renders a stack's effective
Compose model with `docker compose config` and runs a registry of
deterministic checks before deploy, surfacing findings grouped by
severity (blocker, high, warning, info) with a remediation for each.
Findings cover unset env vars, host-port conflicts on the node, broad
0.0.0.0 exposure, missing bind-mount paths, a mounted Docker socket,
privileged and host networking, moving image tags, missing restart
policy and healthcheck, Swarm-only deploy fields, missing external
networks or volumes, and container_name collisions.
The report is node-scoped and stored as the last run per stack, and the
route auto-proxies to the active node so a remote stack is checked on
the node that owns it. A new Doctor tab on the stack detail panel runs
preflight and shows the grouped findings, with a severity dot on the tab
when the last run has blocker or high findings. The tab is gated on a
compose-doctor capability so older nodes hide it.
No environment value is ever stored, returned, or logged: only env key
names and structural facts are read, and render failures surface a
generic message or the missing required-variable names, never raw
stderr.
* fix: scroll the stack tab strip when its tabs overflow
Adding the Doctor tab can push the per-stack Anatomy tab strip past the
panel width on narrower layouts. Make the tab row scroll horizontally
with subtle edge fades that appear only while there is more to scroll in
that direction, so a panel wide enough to show every tab is unchanged.
* fix: add clickable arrows and wheel scroll to the stack tab strip
Hiding the scrollbar left mouse users with no way to scroll the
overflowing tab row: a vertical wheel does not move a horizontal overflow
and native rows do not drag-scroll. Replace the passive edge fades with
clickable chevron arrows shown only when the row overflows that edge, and
translate a vertical wheel over the row into horizontal scroll.
* fix: inline the path-injection barrier in renderConfig
CodeQL's path-injection check does not credit the wrapped isPathWithinBase
helper as a sanitizer, so move the containment check inline at the spawn
cwd sink, matching the canonical barrier used elsewhere in the codebase.
Behavior is unchanged: the resolved stack directory must be contained in
the compose base and may not be the base itself.
* fix: hoist the compose-config spawn into the path-barrier scope
The earlier inline barrier sat in a different scope than the spawn cwd
sink (separated by the Promise-executor closure) and used a compound
guard, so CodeQL did not credit it. Use the exact canonical startsWith
barrier and hoist the spawn into the same scope as the check. Behavior
is unchanged: the executor runs synchronously in the same tick as the
spawn, so handlers still attach before any event can fire.
* feat: detect stalled stack updates and add in-app recovery actions
Add a backend idle-output backstop that stops a deploy/update compose step
that has gone silent (SENCHO_COMPOSE_STALL_TIMEOUT_MS, default 10m), so a
hung image pull surfaces a fast failure instead of spinning indefinitely.
Surface failed, timed-out, and stalled operations with recovery actions on
the stack page: a desktop chip plus popover menu and an inline mobile card
offering retry, restart, roll back (when a backup exists), refresh state,
and copy diagnostics, all gated by deploy permission. The streaming
deploy/update progress modal is now on by default and warns when output
goes quiet. Container state is refreshed after a failed or stalled
operation, and the UI never sits in an indefinite spinner.
* fix: harden rollback against policy-blocked file mutation and refine recovery
Address review findings on the stalled-update recovery work:
- The rollback route restored backup files before running the policy gate, so
a policy-blocked rollback could leave the on-disk config rolled back while the
deployed containers were unchanged. Snapshot the current files first and
revert them when the gate blocks; if that revert itself fails, escalate it on
the persistent alert feed since the 409 is already sent.
- Refresh container state after a successful manual rollback (rollback
redeploys), without mis-recording a refetch failure as a rollback failure.
- Suppress the stalled-output warning once live progress is unavailable.
* test: mock snapshotStackFiles in the atomic-deploy rollback route tests
The rollback route now snapshots stack files before restoring a backup, so its
FileSystemService mock needs snapshotStackFiles. Without it the mocked call
threw and the route returned 500, failing the success-path rollback assertions.
* feat(stacks): persist a drift ledger with temporal source-change detection
Build on the read-only compose-vs-runtime drift check so a stack's drift is
remembered over time, not just shown at a glance.
- Record a deploy baseline: on a successful deploy, update, or rollback, store
the deployed compose file's source and rendered-model hashes on the stack so
the Drift tab can tell whether the file has changed since the last deploy.
- Surface temporal drift in the Drift tab: "matches last deploy", "source
changed since last deploy" (distinguishing a model change from a
formatting-only edit), or "no deploy baseline yet".
- Persist findings into a drift ledger: a re-check reconciles the current
findings, recording newly detected ones and resolving cleared ones, and shows
a short drift history under the findings. The drift report read stays
side-effect-free; only an explicit re-check (and a deploy) writes the ledger.
- Write drift detected/resolved events to the stack Activity timeline so the
provenance sits alongside deploys and restarts.
Node-local and available on the Community tier. Reconciliation is skipped when
a check is not authoritative (Docker unreachable or a compose parse error) so an
open finding is never falsely cleared.
* fix(stacks): record the drift baseline for every deploy path and harden the ledger
Address review feedback on the drift ledger:
- Record the deploy baseline in ComposeService.deployStack/updateStack instead of
only the manual route, so bulk, Git-source, App Store, scheduler, and webhook
deploys all capture source/rendered hashes. Reconciliation stays on the explicit
re-check.
- Store no rendered baseline when the local parser cannot model the compose (for
example a file over the parse cap) rather than a sentinel that would make a later
real change read as unchanged.
- Let temporal-overlay failures surface as a 500 instead of being hidden behind a
neutral "no baseline"; only the compose read stays best-effort.
- Omit the temporal card entirely when a report (for example from an older remote
node) carries no temporal data, instead of showing a misleading "no baseline".
- Keep drift_detected / drift_resolved history-only by excluding them from the
routable-category whitelist, so they are never offered as a channel route that
would never fire.
- Use a JSON separator for the finding identity key so the source file is plain
text (no embedded control byte).
* fix(stacks): sanitize logged errors in the drift report handlers
The drift report and re-check handlers logged the caught error object
raw alongside the stack name, which a code scan flagged as a
log-injection vector: a crafted stack name surfacing inside an error
message or stack could forge log lines. Route the error through the log
sanitizer so control characters are stripped before writing. Render it
with util.inspect first so the stack trace, cause chain, and underlying
error codes are preserved for debugging.
* feat(updates): auto-prune dangling images after updates
Each update pulls a fresh image and recreates containers, leaving the
replaced image behind as a dangling layer that previously had to be
pruned by hand. A new "Prune dangling images after updates" toggle under
Settings > System > Docker hygiene reclaims these automatically.
The setting is on by default and opt-out. When enabled, a successful
stack update (manual or scheduled) and a Sencho self-update each remove
the dangling image layers they orphaned. Only untagged layers are
touched; tagged images, volumes, and data are never removed. The toggle
requires an admin account and is per node: each instance honors its own
value, so a remote node self-update applies that node's own preference.
A prune failure never affects the update result: on the stack path it is
caught and logged after the update has already succeeded, and on the
self-update path the helper-shell prune runs only after a clean recreate
and cannot change the exit code or the recorded update error.
* security(self-update): shell-quote label-derived values in helper command
Address review feedback on the prune-on-update change:
- The self-update helper command interpolated the compose service name and
config-file paths (both read from Docker Compose labels) straight into a
shell string. Shell-quote them via shQuote so a label carrying shell
metacharacters stays inert data and cannot break the exit-code capture,
error-file write, or prune guard.
- Correct the settings copy and docs: the prune is a standard dangling-image
prune, so it reclaims every untagged layer on the node, not only the one the
current update orphaned. Tagged images, volumes, and data remain untouched.
- Add tests: shell-metacharacter neutralization and prune-output suppression in
the self-update command, and an atomic-update case asserting a prune failure
does not trigger a rollback.
* fix(updates): omit the reclaim figure when the daemon reports zero bytes
End-to-end testing on a Docker daemon backed by the containerd image store
showed the post-update prune removing a dangling image while the prune API
returned SpaceReclaimed=0, so the stream printed "reclaimed 0.0 MB" even though
an image was removed. Show the reclaimed figure only when the daemon reports a
non-zero value; otherwise the line reads "=== Pruned dangling images ===". The
overlay2 store still reports real figures and shows them. Add a test covering
both branches.
* fix(deploy-progress): decouple deploys from the live progress stream
The deploy progress modal streamed compose output over a WebSocket, but
the deploy itself was coupled to that socket in two ways that could break
or silently abort a deploy:
- The deploy request was gated on the progress socket connecting, so any
upgrade failure (a reverse proxy blocking WebSocket upgrades, or the
admin-only stream rejecting a scoped deployer) left the modal stuck on
"Connecting..." and the deploy never fired.
- The backend terminated the running compose process when that socket
closed, so minimizing the modal, navigating away, or a network blip
aborted an in-flight deploy.
Make the progress socket output-only: the deploy is owned by its request
and runs to completion (or the existing command timeout) regardless of
the stream. The modal now degrades to a "Live progress unavailable" state
and still reports success or failure from the request result. Connect
failures, drops, and a connect timeout all release the deploy instead of
blocking it.
Also route progress output per deploy: the frontend sends a correlation
id on both the connectTerminal message and the deploy request header, and
the backend keys progress sockets by that id so concurrent deploys from
different tabs or users no longer cross-stream each other's output.
Cap the in-memory parsed log rows so a very long deploy cannot grow the
modal's state unbounded.
* fix(deploy-progress): generate the deploy session id with a CSPRNG
The per-deploy correlation id keys which WebSocket receives a deploy's
live output, so a guessable id lets one authenticated client register a
victim's id and read its compose output. It was built from Math.random()
plus a timestamp, which is not cryptographically secure.
Generate it with crypto.getRandomValues (128 bits, hex). That is the one
Crypto member available in insecure contexts, so it still works over LAN
HTTP where crypto.randomUUID is unavailable.
* fix(deploy-progress): stop headerless ops bleeding into a keyed progress modal
Address review findings on the progress-stream routing:
- Only an id-less connectTerminal registration may become the id-less
fallback socket. Previously every connectTerminal (including keyed deploy
modals) set the fallback, so a headerless operation (bulk update, rollback,
or a legacy client) resolved via getTerminalWs() into another user's keyed
deploy modal. Keyed sockets are now excluded from the fallback, and a socket
that adopts a session id is removed from it.
- The connect-timeout fallback now also flags the modal as "Live progress
unavailable" instead of leaving it on "Connecting..." while the deploy runs.
- Log only a short prefix of the deploy session id in developer diagnostics,
not the full capability value.
Operators previously saw "spawn docker ENOENT" or "spawn /bin/sh ENOENT" when
the host was under memory pressure, which sent them down a missing-binary
debugging path. Linux libuv's posix_spawn can fail to allocate its argv /
path-search arena under low free memory and surface the underlying ENOMEM
as ENOENT.
Centralizes spawn-error mapping in a new utils/spawnErrors.ts helper:
- Explicit ENOMEM is rewritten to "Out of memory while launching <command>
(host free memory: X MiB of Y MiB)".
- ENOENT under the 128 MiB free-memory floor is rewritten with the same
wording plus a "reported as ENOENT under memory pressure" hint.
- ENOENT for docker on a healthy host preserves the existing
"Docker CLI unavailable on this node" mapping.
- Other errors pass through unchanged.
Applied at the four named offenders: ComposeService.execute(),
ComposeService.captureCompose(), DockerController.getContainersByStack(),
and FileSystemService.getStacks() (which gets an ENOMEM-aware log line
for the scandir failure).
Startup also logs host free/total MiB once and warns when free memory is
below the 128 MiB floor, so the diagnostic surfaces before the first
spawn attempt rather than after it fails.
37 tests cover the mapping function directly and the ComposeService /
FileSystemService integration paths.
POST /api/stacks/:name/{deploy,down,update} previously returned HTTP 500
with body {"error":"spawn docker ENOENT"} when invoked against a stack
whose compose directory was missing. The status code was wrong (the
named resource did not exist, so 404 is the right answer) and the
message misled operators into thinking the docker CLI was unavailable.
Add a small requireStackExists(nodeId, stackName, res) helper in
routes/stacks.ts that validates the stack name and confirms a compose
file is present via FileSystemService.hasComposeFile before any of the
three handlers spawn docker compose. The helper is called immediately
after requirePermission and before runPolicyGate so unauthorized
callers still get 403 first and the policy gate never runs against a
phantom stack.
In ComposeService.execute(), narrow the child.on('error') handler so
the genuine docker-binary-missing case (ENOENT on the spawn itself)
rejects with "Docker CLI unavailable on this node" instead of the raw
"spawn docker ENOENT". This is defense in depth for the rare case the
pre-check cannot cover, and it fixes the misleading-message half of
the bug as well.
Cover the new contract with stack-actions-missing-stack.test.ts (four
cases: deploy/down/update return 404, invalid name returns 400). Mock
ComposeService as a tripwire so a future code path that bypasses the
guard would fail loudly. Fix stacks-failure-notifications.test.ts by
adding hasComposeFile to its FileSystemService partial mock so the
existing happy-path-error-handling cases continue to flow into
ComposeService.
* fix: harden deploy enforcement paths
* fix: update Docker toolchain to Go 1.26.3
* fix: repair Dockerfile tr argument split across lines
* fix: bump protobufjs to clear npm audit high-severity advisories
* fix(test): add execFile to child_process mock in compose-images test
* fix: resolve merge conflicts with main
* fix: resolve merge conflicts with main
* fix: resolve merge conflicts with main
* feat(fleet): sencho mesh in traffic and routing tab
Lights up Sencho Mesh: cross-node container forwarding rendered as if the
container next to you were on localhost. Builds on the dormant TCP frame
plumbing from the prior PR (pilot tunnel TCP frames + sencho-mesh sidecar
package) and exposes the Admiral-only orchestrator surface.
Backend
- New mesh_stacks table (per-node opt-ins) + nodes.mesh_enabled column
via DatabaseService.migrateMeshTables.
- MeshService singleton: sidecar lifecycle via Dockerode, opt-in/out with
cascading override regeneration, request-based resolver from sidecar
control WS, cross-node TCP forwarding via PilotTunnelManager (same-node
fast path included), in-memory 1000-event activity ring buffer with
durable mirror to audit_log for state-change events, per-node and
per-route diagnostics, and the Test upstream probe.
- MeshComposeOverride: pure YAML generator that injects extra_hosts using
host-gateway. The user's docker-compose.yml is never mutated; overrides
live under DATA_DIR/mesh/overrides.
- ComposeService deploy/update splice the override file when the stack
is opted in; non-mesh stacks behave identically to today.
- Pilot agent resolveMeshTarget consults the local mesh_stacks table
(defense in depth) and resolves Compose containers via Dockerode.
- /api/mesh router with 13 Admiral-gated endpoints covering status,
enable/disable, stack opt-in/out, alias listing, per-route diagnostic,
Test upstream probe, per-node diagnostic, sidecar restart, activity
log paginated and SSE.
- meshControl WS slot at /api/mesh/control validates the mesh_sidecar
JWT minted by MeshService; dispatched as upgrade slot 2 (canonical
order preserved).
Frontend
- New Traffic Routing tab in FleetView, gated by isAdmiral and wrapped
in AdmiralGate. Tab uses the cyan brand glyph and italic-serif state
typography from the audit.
- RoutingTab masthead with mesh activity drawer, per-node card grid
with TogglePill, alias rows with five-state pill taxonomy
(healthy / degraded / unreachable / tunnel-down / not-authorized),
inline Test buttons.
- Four sheets: opt-in picker with port-collision inline error,
per-route detail with diagnostic + filtered activity, per-node
diagnostics with active streams + resolver cache + restart action,
fleet-wide activity log with filters.
- meshRouteState helper centralizes pill-state mapping; pure-function
tests cover all five states.
Docs
- User docs at /docs/features/sencho-mesh.mdx covering opt-in,
troubleshooting, security model (4 guarantees + 4 explicit
non-guarantees), and V1 limitations.
- Internal architecture and runbook pages.
- websocket-dispatch internal doc updated with the new slot.
* fix(mesh): validate stack name before path use; fix test DB lifecycle
Two surgical fixes against the prior PR.
Path-injection (CodeQL js/path-injection): MeshService.optInStack,
optOutStack, ensureStackOverride, and removeStackOverride now validate
stackName via isValidStackName from utils/validation, reject malicious
names at the API boundary, and additionally check isPathWithinBase on
the resolved override file path for defense in depth. The dataflow from
req.params.stackName to fs.writeFile no longer reaches an unsanitized
path expression.
Test DB lifecycle: mesh-service.test.ts used per-test setupTestDb /
cleanupTestDb, which deletes the temp dir while DatabaseService still
holds an open SQLite handle. On Linux CI this raises
SQLITE_READONLY_DBMOVED on the next prepare() because the inode has
been unlinked. Switched to file-scoped beforeAll/afterAll matching
agents-routes.test.ts, with a per-test beforeEach that truncates
mesh_stacks plus non-default nodes and resets the MeshService singleton
in-memory state. Adds a new test case asserting the path-traversal
rejection.
* fix(compose): use discovered compose filename instead of hardcoded docker-compose.yml
composeArgs() hardcoded `-f docker-compose.yml` for every deploy. Sencho
writes its canonical compose file as `compose.yaml`, so any stack created
via the UI failed to deploy with `open ...docker-compose.yml: no such
file or directory`.
When no mesh override applies, drop the explicit `-f` so docker compose's
built-in discovery resolves the actual filename. When an override exists,
look up the real base filename via FileSystemService.getComposeFilename()
and pass both files explicitly.
Also hoist the MeshService import to module top now that the dependency
is known to be acyclic, and revert the matching unit-test assertion.
* refactor(backend): sanitize user input before logging to close CRLF injection
Adds a small sanitizeForLog helper that strips CR, LF, tab, and ASCII
control characters (0x00-0x1F, 0x7F) from a value before it is embedded
in a console.log/warn/error/debug call. Wraps every call site where a
user-controlled value (req.params, req.body, req.query, or a value
derived from them) flows into a log message.
Closes the bulk of the open CodeQL alerts in this family:
- 96 js/log-injection
- 28 js/tainted-format-string
The helper is in backend/src/utils/safeLog.ts. Routes still pre-validate
input at the request boundary; this is the second line of defense and
gives static analyzers a sanitizer they can trace through. JSON
responses, Docker filter labels, and other non-log call sites are
intentionally left unwrapped.
* refactor(backend): printf-style format strings for tainted-log call sites
CodeQL's js/tainted-format-string rule flags template literals in the first
arg of console.X when any interpolated value is user-controlled, regardless
of whether each value is sanitized inline. The canonical mitigation is to
use a static format string and pass values as positional args.
Converts the 28 flagged template literals to printf-style ("%s") format
strings, with sanitizeForLog applied to each positional arg. Also fills in
the log-injection wraps on 9 sites where a user-controlled value was
missed in the first sweep (agents, fleet, gitSources, imageUpdates,
GitSourceService).
No behavior change at runtime. Node's util.format substitutes %s tokens
identically to template-literal interpolation.
* fix(backend): wrap nodeId/snapshotId in fleet restore debug log
CodeQL flagged the unwrapped numeric args even though they cannot
contain control chars in practice. Apply the sanitizer for taint-flow
recognition.
Policies with block_on_deploy=1 now scan every stack image before
docker compose up runs and reject the deploy with HTTP 409 on violation.
The UI opens a dialog listing offending images; admins can override per
deploy with ?ignorePolicy=true, and every bypass is recorded in the
audit log with the originating route, actor, policy, and image list.
When Trivy is not installed on the target node the gate fails open with
a warning notification, so teams are never locked out by tooling state.
Post-deploy and scheduled scans still evaluate matching policies and
dispatch warnings on violations to surface drift on long-running stacks.
Public API additions: policy and suppression CRUD under /api/security,
plus the documented 409 block-response shape on all deploy paths.
* feat(stack-view): per-container health strip and structured logs viewer
Replaces the flat container list with a per-container health strip showing
healthcheck state, uptime, port mapping with an open-app link, and live
cpu/memory/network sparklines fed by a 60-sample ring buffer on the stats
WebSocket.
Adds a structured logs viewer that parses docker timestamps (emitted by the
-t flag on the logs stream) and classifies each line by level. Rows render
as a DOM grid with filter pills (all / info / warn / err with count),
following indicator, and plain-text download. A segmented toggle switches
between the structured viewer and the original xterm view; the choice is
persisted in localStorage.
* fix(stack-view): disable no-control-regex for ANSI escape pattern
ANSI escape sequences start with ESC (0x1B), which is a control character.
The regex is intentional and cannot be rewritten without it.
* feat(registries): add stateless test endpoint, ECR caching, URL and host hardening
Adds a POST /api/registries/test endpoint so credentials can be verified
before being persisted. Caches ECR authorization tokens in memory until
their AWS-reported expiry (minus a safety margin) instead of fetching on
every compose invocation. Normalizes registry URLs on save so the
stored values match the keys Docker expects in ~/.docker/config.json,
fixes a bidirectional host-match bug in getAuthForRegistry that could
cross-match overlapping hostnames, and surfaces per-registry decryption
failures as warnings in the deploy log stream instead of swallowing
them. Also strips the Authorization header on cross-host redirects in
the test probe, rejects non-http(s) schemes on save, and validates the
shape of returned ECR authorization tokens before use.
* refactor(registries): align UI with design system and add in-form test button
Swaps the registry type dropdown from shadcn Select to the project's
Combobox, applies the canonical card bevel and top-border hover styling
to the form container and each registry row, restyles the delete
button to the ghost + muted destructive pattern, uses strokeWidth 1.5
on every Lucide icon, and routes all toast errors through the
standard defensive chain. Adds a Test connection button inside the
form so credentials can be verified before saving.
* test(registries): cover RegistryService and deploy warnings surface
Adds unit coverage for URL normalization, the encrypt/decrypt round
trip through create and resolveDockerConfig, exact-host matching in
getAuthForRegistry, resolveDockerConfig warnings on decryption
failure, ECR token cache hit/miss and invalidation on update, the
stateless testWithCredentials path for 200, 401 with and without a
challenge, network errors, and ECR success and failure including
malformed tokens. Extends the ComposeService tests to verify that
warnings from resolveDockerConfig reach the deploy log stream.
* docs(registries): document test-before-save flow and troubleshooting
Describes the in-form Test connection button, the two-point testing
flow from the registries list, the cached ECR token behavior during
deploys, the per-registry warning Sencho emits when a stored secret
cannot be decrypted, and adds a Troubleshooting section covering
common 401 causes, ECR token handling, warning interpretation, and
per-node credential scoping.
* fix(stacks): harden stack management with security fixes, validation alignment, and logging
Validate WebSocket stack names with isValidStackName() to close a
path-traversal gap on the /api/stacks/:stackName/logs WS endpoint.
Align POST /api/stacks to use the canonical validator (allows underscores).
Replace error: any catch blocks with error: unknown + type narrowing.
Add cache invalidation to PUT /api/stacks/:stackName/env.
Rename DELETE param from :name to :stackName for consistency.
Add standard [Stacks] lifecycle logs and diagnostic [Stacks:debug] logs
gated behind the Developer Mode toggle (with 5s TTL cache).
Extract shared isDebugEnabled() and getErrorMessage() utilities.
Frontend: roll back optimistic status on API failure, guard unsaved
changes when switching stacks, pre-check duplicate names in App Store.
* docs(settings): update Developer Mode description to mention debug diagnostics
Add console.warn/console.error logging to 22 silent catch blocks across
10 files. Errors in cleanup, migrations, SSO, fleet snapshots, shutdown,
and validation are now visible in logs. ENOENT guards added to
file-system catches to distinguish missing files from permission errors.
No control flow changes.
* fix(stacks): resolve permission denied error when deleting stacks with root-owned files
When Docker Compose creates files as root inside a stack directory, the
non-root Sencho process cannot remove them. This adds a Docker-based
fallback: if fsPromises.rm fails with EACCES/EPERM, Sencho spawns a
short-lived Alpine container to clean up the root-owned files.
Also enhances docker compose down with --volumes --remove-orphans to let
Docker clean up its own resources before filesystem deletion.
* docs: clarify that pre-existing root-owned stacks can be deleted
* fix(stacks): include Docker stderr in fallback deletion error message
Fixes CI lint failure: 'stderr' was assigned but never read in
forceDeleteViaDocker(). Now surfaces Docker stderr output in the error
message when the fallback cleanup fails.
Add centralized credential storage for private Docker registries with
support for Docker Hub, GHCR, AWS ECR, and self-hosted registries.
- New `registries` table with AES-256-GCM encrypted secrets
- RegistryService with CRUD, test connectivity, Docker config generation
- 5 API endpoints gated by requireTeamPro + requireAdmin
- ComposeService injects credentials via temp DOCKER_CONFIG on deploy/pull
- ImageUpdateService passes stored credentials for private registry checks
- AWS ECR just-in-time token refresh via @aws-sdk/client-ecr
- RegistriesSection UI in Settings Hub with type-aware form
- Documentation with screenshots
* feat: add RBAC viewer accounts, atomic deployments, and fleet-wide backups (Pro)
Introduces three Pro-tier features:
- RBAC: Multi-user system with admin/viewer roles, user management UI,
automatic migration from single-admin credentials, viewer restrictions
across the entire UI (read-only editor, hidden action buttons)
- Atomic Deployments: Pre-deploy file backup to .sencho-backup/, automatic
rollback on health probe failure, manual rollback button, health probes
added to stack updates, webhook-triggered deploys use atomic rollback
- Fleet-Wide Backups: Point-in-time snapshots of compose files across all
nodes (local + remote), stored centrally in SQLite, per-stack restore
with optional redeploy, graceful handling of offline nodes
* fix(settings): use correct ProGate prop name in UsersSection
* fix(settings): remove unused isPro prop from UsersSection
* fix(auth): fetch user info after login and setup so isAdmin is set correctly
* feat(pricing): revise pricing strategy and enforce variant-based seat limits
Raise Personal Pro from $49/yr to $69/yr with 3 viewer seats (up from 1).
Add $15/mo billing option for Team Pro. Mark lifetime pricing as a
90-day early-adopter offer. Store Lemon Squeezy variant_name on
activation/validation and enforce seat limits server-side per variant.
* feat(licensing): add Lemon Squeezy checkout, webhook, and billing portal integration
Server-side checkout URL generation (POST /api/checkout) with admin email
pre-fill and instance_id custom data. HMAC-SHA256 verified webhook endpoint
(POST /api/webhooks/lemonsqueezy) handling order, subscription, and payment
lifecycle events for automatic license activation. Customer billing portal
link stored from webhook events and exposed via GET /api/billing/portal.
In-app checkout buttons in Settings with manual license key fallback.
* fix(licensing): exempt Lemon Squeezy webhook from auth middleware
The catch-all auth middleware on /api/* was blocking the public webhook
endpoint. Added /webhooks/lemonsqueezy to the exemption list alongside
/auth/* and /webhooks/:id/trigger.
* feat(pricing): update pricing to final live rates
Personal Pro: $7.99/month, $69.99/year, $249 lifetime.
Team Pro: $49.99/month, $499.99/year, $1,499 lifetime.
Added personal_monthly checkout variant across backend, frontend, and website.
* refactor(licensing): remove server-side checkout/webhook for self-hosted model
Sencho is self-hosted — each user runs their own instance, so there is
no central server to receive webhooks or hold the store API key. Replaced
in-app checkout buttons with a "View Pricing" redirect to sencho.io and
kept manual license key activation as the primary flow.
- Delete LemonSqueezyService (checkout, webhook, HMAC verification)
- Remove POST /api/checkout, GET /api/billing/portal, POST /api/webhooks/lemonsqueezy
- Remove raw body parser and auth exemption for webhook route
- Remove all LEMONSQUEEZY_* env vars from .env.example
- Replace checkout buttons in SettingsModal with single "View Pricing" button
- Simplify LicenseContext checkout to open sencho.io pricing page
- Update licensing docs to reflect website-based purchase flow
* chore: normalize em-dashes to hyphens across codebase (linter)
* chore: remove accidentally tracked directories from index
* feat: add RBAC viewer accounts, atomic deployments, and fleet-wide backups (Pro)
Introduces three Pro-tier features:
- RBAC: Multi-user system with admin/viewer roles, user management UI,
automatic migration from single-admin credentials, viewer restrictions
across the entire UI (read-only editor, hidden action buttons)
- Atomic Deployments: Pre-deploy file backup to .sencho-backup/, automatic
rollback on health probe failure, manual rollback button, health probes
added to stack updates, webhook-triggered deploys use atomic rollback
- Fleet-Wide Backups: Point-in-time snapshots of compose files across all
nodes (local + remote), stored centrally in SQLite, per-stack restore
with optional redeploy, graceful handling of offline nodes
* fix(settings): use correct ProGate prop name in UsersSection
* fix(settings): remove unused isPro prop from UsersSection
* fix(auth): fetch user info after login and setup so isAdmin is set correctly
Add Node modal — type selector & state reset:
- Restored a Local/Remote <Select> dropdown in renderFormFields so users can
explicitly choose the node type instead of it defaulting silently to 'remote'.
- Switching type clears api_url and api_token so no stale remote credentials
carry over if a user switches from Remote to Local mid-form.
- Replaced the static "Add Remote Node" title with a dynamic one that reflects
the currently selected type ("Add Local Node" / "Add Remote Node").
- onOpenChange now resets formData to defaultFormData whenever the dialog
opens, preventing stale values from a previous session leaking in.
Remote connection details — real metrics:
- testRemoteConnection previously returned hard-coded '-' for containers,
images, and cpus after a successful auth/check ping.
- Now fires three parallel requests (Promise.allSettled) after auth passes:
/api/stats → containers total + running count
/api/system/stats → cpu.cores
/api/system/images → image list length
- Each field falls back to '-' gracefully if an endpoint is unavailable,
so a slow or older remote instance never breaks the connection test.
DEP0060 util._extend suppression:
- http-proxy@1.18.1 calls util._extend when createProxyServer() is first
invoked at runtime (NOT at import time). A process.emitWarning override
placed before the proxy instantiations intercepts only DEP0060 without
suppressing any other warnings. No package version changes needed.
Also includes linter/formatter normalisation across multiple files.
Anso