The self-update helper container runs `docker compose up -d
--force-recreate` to recreate the main Sencho container. Previously it
only mounted the docker socket, the compose working directory, and the
data directory. If the user's docker-compose.yml references env_file,
configs, or secrets at paths outside the compose working directory
(e.g. /opt/docker/env/globals.env), the helper could not resolve them
and compose failed with "env file not found".
Now during initialize(), SelfUpdateService collects all host bind mounts
from the container inspect data (filtered to Type=bind). In
triggerUpdate(), these are forwarded to the helper as read-only mounts
at their original host paths (source:source:ro), skipping the socket,
data dir, and compose working dir which are already mounted explicitly.
This lets docker compose resolve any host-path reference the user has
configured, without needing to parse compose files for specific
directives.
The helper container that runs `docker compose up --force-recreate` was
spawned with `docker run -d`, so the command returned immediately with
just the container ID. Any failure happening INSIDE the helper (bad
compose file, image mismatch, permission issue, socket problem) was
invisible: execFile's callback only fired for `docker run` command
errors, never for errors inside the detached helper. The UI fell back to
the generic 3-minute "Local update did not complete" heuristic with no
actionable information.
The helper now runs attached, so execFile's callback receives the
helper's exit code and stderr directly for any failure that happens
before the recreate kills this process. Additionally, the helper
persists exit code + stderr to `/app/data/.sencho-update-error` before
exiting, so the error survives the gateway's own death. On startup,
`SelfUpdateService.recoverPreviousError()` reads and deletes that file,
routing the real error through the existing `getLastError()` path so the
freshly booted gateway reports exactly why the previous attempt failed
instead of the generic timeout.
The "Updating Sencho..." overlay used to dismiss prematurely while the
image pull was still running, after which the local node card would get
stuck in "updating" and eventually surface a generic "Timed Out" error
while the container remained on the old version.
Three root causes are addressed:
1. The image pull was synchronous (`execFileSync`), which blocked the
Node event loop. The overlay's health probe saw the server come back
the moment the pull finished and reloaded the page, even though the
container had not restarted yet. The pull is now async via
`promisify(execFile)`, so /api/health and /api/fleet/update-status
keep serving throughout.
2. The overlay reloaded on the first 200 from /api/health regardless of
whether the underlying process had actually restarted. /api/health
now exposes the gateway boot timestamp, and the overlay captures it
pre-update and only reloads when it observes a different value. A
wasOffline-then-online fallback handles the case where the pre-update
fetch failed.
3. Helper container spawn errors from `docker run` were silently
discarded, so a failed compose recreate never surfaced anywhere.
Errors are now captured into `lastUpdateError` via the execFile
callback and surfaced through the existing /api/fleet/update-status
error path.
A 3-minute early-fail heuristic on the local node block surfaces a clear
failure message when the helper fails silently, instead of waiting the
full 5-minute timeout for an unknown failure.
The self-update feature failed on remote nodes because SelfUpdateService
ran `docker compose -f <host_path>` inside the container, where the host
compose file path does not exist. The fix splits the update into two
steps: (1) pull the latest image directly via `docker pull`, and (2)
spawn a short-lived helper container that mounts the compose directory
from the host and runs `docker compose up --force-recreate`.
Additional changes:
- Use execFileSync/execFile with argument arrays instead of shell strings
to eliminate shell injection surface from Docker label values
- Add Signal 4 completion detection: mark update as completed when the
remote version matches the gateway version (with 15s elapsed guard)
- Extend early failure heuristic from 90s to 3 minutes for slow pulls
- Distinguish "node unreachable" from "node lacks self-update capability"
in error messages; use silent skip in update-all to avoid res crashes
- Add requireAdmin guard to POST /api/system/update
- Handle comma-separated compose config file paths (multiple -f flags)
- Update fleet docs with self-update mechanism, troubleshooting entries
The execSync call in SelfUpdateService used cwd from Docker Compose
labels (host-side path) which does not exist inside the container.
Removed cwd (the -f flag provides the absolute compose file path),
added explicit shell, added Docker Compose CLI check at startup,
and improved error capture via stderr.
* fix(fleet): resolve stuck update states and improve update UX
The fleet node update flow had several bugs: the in-memory update tracker
never cleared terminal states (timeout, failed, completed), leaving nodes
permanently stuck with no way to retry or dismiss. The Recheck button
only re-fetched stale state without clearing it, and the POST trigger
rejected retries with 409 even after timeout.
Backend fixes:
- Add DELETE endpoints (single node + batch) to clear tracker entries
- Fix 409 race: detect expired timeouts and clear terminal states before
re-triggering
- Populate error messages in the tracker for timeouts and failures
- Include error field in the update-status API response
- Auto-expire completed entries after 60 seconds
Frontend fixes:
- Add retry (RotateCcw) and dismiss (X) buttons on failed/timed-out badges
- Show error details via animated cursor hover (CursorFollow pattern)
- Recheck button now batch-clears all terminal states before fetching
- Recheck shows loading spinner and disables while checking
- Extract NodeCardProps interface for readability
* fix(fleet): detect update completion via process start time
Remote nodes that cannot report their version (e.g. older builds)
caused updates to always time out because completion detection
relied solely on version comparison. The gateway now tracks the
remote node's process start time from /api/meta and detects
container restarts by comparing it across polls.
Also extracts a createTracker() factory to eliminate repeated
object construction across 5 call sites.
* docs: add troubleshooting for first-update timeout on old nodes
Adds a new troubleshooting entry explaining why the first remote
update on nodes running pre-v0.40.0 always times out (neither
version nor process start time can be detected). Documents the
fix: dismiss, recheck, and confirm the node updated.
Also adds a screenshot of the timed-out state with retry/dismiss
buttons to the remote updates feature page.
* fix(fleet): detect update completion via offline detection and error reporting
The update completion detection relied on version change and process
start time, both of which fail on nodes running older Sencho versions
that report "unknown" and lack the startedAt field. This caused every
update to time out after 5 minutes.
Add three-signal detection: version change, process restart (startedAt),
and offline/online detection (node went unreachable during update and
came back). Also add a 90-second early failure heuristic for when the
remote image pull fails silently, and surface pull errors from
SelfUpdateService via /api/meta so the gateway can report them
immediately.
* fix(deps): bump vite to 8.0.5 to resolve high severity vulnerabilities
Fixes GHSA-4w7w-66w2-5vf9, GHSA-v2wj-q39q-566r, GHSA-p9ff-h696-f583.
* fix(deps): bump vite in backend lockfile to resolve audit failures
Vitest pulls in vite as a transitive dependency. Bumps to 8.0.5.
Add the ability to check for outdated nodes and trigger over-the-air
updates from Fleet View. Nodes self-update by pulling the latest Docker
image and recreating their container via the "last breath" pattern.
Backend:
- SelfUpdateService: self-container identification via HOSTNAME + Docker
Compose labels, triggers pull + force-recreate
- CapabilityRegistry: runtime capability disabling via disableCapability()
- POST /api/system/update (202 + deferred self-update)
- GET /api/fleet/update-status (version comparison across fleet)
- POST /api/fleet/nodes/:nodeId/update (single node)
- POST /api/fleet/update-all (bulk remote update)
- In-memory update tracker with 5-min timeout
Frontend:
- Node Updates modal with summary stats, search filter, table layout,
per-node Update buttons, and bulk Update All
- Version badges and update-available indicators on node cards
- ReconnectingOverlay for local node updates (polls /api/health)
- 5s fast-poll when any node is actively updating
- UpdateStatusBadge shared component for consistent badge rendering
Requires Skipper (Pro) tier. Nodes must be deployed via Docker Compose
with Docker socket access.
Anso