* fix(notifications): stop Sencho version notifications from silently skipping
Three independent defects combined to make version-update notifications
silently fail even while Fleet overview correctly surfaced an update button:
- The in-memory 6-hour cooldown was advanced before the network fetch,
so a single transient failure at boot could lock the check for the
rest of the container lifetime. Moved the cooldown update inside the
success branch so failures retry on the next eval cycle.
- MonitorService called the raw version fetch directly, bypassing the
CacheService wrapper (TTL, inflight dedup, stale-on-error) that Fleet
uses, so the two paths could diverge. Unified both on a shared
getLatestVersion() helper in utils/version-check.ts.
- The dedup key could carry stale state from a previous build and never
self-clear. It now self-heals when the running version reaches the
previously-notified version, so future releases re-fire as expected.
Added diagnostic logs gated on debug mode for each skip branch, plus
three regression tests covering cooldown-on-failure, cooldown-on-success,
and dedup self-heal.
* docs(notifications): drop legacy-upgrade framing from alerts troubleshooting
Sencho has not shipped publicly, so troubleshooting entries written in
'this used to happen but now does Y' mode reference a past that does not
exist for any reader. Rewrote the version-notification, image-update,
and crash-alert troubleshooting entries to describe current behavior
positively without referring to prior builds, upgrade paths, or legacy
fixes.
* chore(security): accept CVE-2026-33810 in bundled Docker CLI 29.4.0
Trivy now flags CVE-2026-33810 (Go stdlib crypto/x509 DNS constraint
bypass, fixed in Go 1.26.2) in the Docker CLI static binary we ship.
Docker CLI 29.4.0 is the latest upstream release and still links Go
1.26.1; no newer static binary exists yet.
Same exposure profile as the already-accepted CVE-2026-32280: the
Docker CLI and compose plugin only validate certificates from
well-known registry CAs and the local Docker socket, not from
attacker-controlled CAs with crafted DNS name constraints. Revisit
on the next Docker CLI release that rebuilds against Go 1.26.2 or
later.
* fix(docker-events): harden crash detection against edge cases
- Isolate per-container failures in the reconcile path so one failed
container inspect cannot abort classification for the rest of the
batch after a reconnect.
- Fall back to inspecting State.OOMKilled when a container exits with
code 137 and no oom event preceded the die, so cgroup OOM kills that
lose the oom event are still classified correctly. The dedup check
runs before the fallback so crashloops do not hammer the daemon.
- Guard the intentional-kill window against wildly future-dated Docker
timestamps by switching to a signed age comparison with a bounded
negative-skew tolerance, so clock skew cannot flip a genuine crash
into an intentional stop.
- Align diagnostic logging with the codebase [Service:diag] convention
and add one informational lifecycle log on boot and shutdown so
operators running in developer mode can confirm the watcher started.
- New tests cover gap-inspect isolation, the OOM inspect fallback (both
success and failure paths), duplicate die events collapsing within
the grace window, and clock-skew bounds on the classifier.
* docs(alerts): add troubleshooting entries for crash detection toggle and rate limits
* fix(notifications): replace polling with Docker event stream for container lifecycle detection
Replaces the 30-second MonitorService crash-detection poll with a causal,
per-node Docker events stream. Eliminates false crash alerts on intentional
stops (docker stop, compose down, stack restart/update), detects OOM kills
as a distinct alert category, and surfaces real crashes in real time.
A new DockerEventManager spawns one DockerEventService per local node. Each
service consumes the filtered container event stream, classifies die events
against recent kill/oom state, and reconciles container state via snapshot
diffing on connect and reconnect. Rate limiting, exponential backoff with
jitter, and parse-error tolerance keep the stream resilient under load and
during daemon interruptions.
MonitorService retains host limits, janitor, version check, and stack metric
alerts; crash and healthcheck detection move out entirely.
* fix(tests): silence require-imports lint in hoisted mock factory
- Read running Sencho version from the packaged manifest via getSenchoVersion() instead of process.env.npm_package_version, which is undefined when launched via node dist/index.js (production Docker). Skip the update check entirely when the version cannot be resolved.
- Add a one-time backfill pass in ImageUpdateService so users who upgraded to a Sencho version with the notification pipeline receive a catch-up entry for stacks already flagged as having updates before the upgrade.
- Surface dispatch failures as error-level entries in the in-app notification bell via a direct notification_history write, so misconfigured webhooks are visible without tailing logs.
- Extend test coverage for both paths: mock getSenchoVersion (including the null-version case) and add dispatch-path tests for transition, no-re-fire, backfill, and error surfacing.
- Expand alerts-notifications docs with per-instance 6-hour check cadence, an example version message, a backfill note, and three new troubleshooting entries.
* fix(network-topology): harden with edge-case fixes, logging, and test coverage
Fixes stale data on node switch (topology now refreshes when active node changes),
adds manual refresh button to toolbar, and guards against containers with empty
Names arrays by falling back to a 12-char short ID. Adds operational and diagnostic
logging around topology fetches, and introduces 25 unit tests covering happy paths,
system network filtering, stack resolution, container deduplication, edge cases,
and error handling.
* docs: refresh screenshots
* fix(container-exec): harden with security fixes, validation, and test coverage
- Enforce admin role at WebSocket upgrade for container exec sessions
- Validate container is running before creating exec
- Fix bash-to-sh fallback (move .start() inside try/catch)
- Register container-exec as a capability for fleet visibility
- Add standard and diagnostic logging for exec lifecycle
- Fix false Admiral license claim in API docs
- Fix design system violations in BashExecModal (hardcoded colors)
- Remove duplicate legacy xterm dependencies
- Add 18-test suite covering auth, validation, fallback, and cleanup
* fix(container-exec): use correct Duplex type for exec stream
The stream variable was typed as NodeJS.ReadWriteStream, which lacks
.destroy(). Dockerode's Exec.start() returns stream.Duplex per its
type definitions. This caused tsc to fail while Vitest (which skips
full type checking) passed.
* fix(alerts): harden with security fixes, design compliance, and test coverage
Add authMiddleware to all alert endpoints, validate notification test
dispatch inputs, fix restart_count metric via Docker inspect, correct
network metric units, replace any types with DockerContainerStats
interface, add webhook timeouts and dispatch error tracking.
Frontend: migrate Select to Combobox, add ScrollArea and delete
confirmation AlertDialog, fix icon strokeWidth to 1.5.
Add update availability notifications for both Sencho version updates
(6-hour check in MonitorService) and stack image updates (state
transition detection in ImageUpdateService). Extract shared version
fetch logic into utils/version-check.ts.
Add diagnostic logging gated behind developer_mode for MonitorService
breach state machine and NotificationService dispatch routing.
Tests: 24 new alert API integration tests, restart_count and version
check unit tests (688 total passing). Docs updated with HTTPS
requirement, update notifications section, and troubleshooting guide.
* fix(alerts): remove unused TEST_USERNAME import in alerts-api tests
Rate-limit OIDC callback route and clear state cookie on all paths.
Validate GitHub API responses and add server-side config validation.
Sync SSO user roles on every login respecting seat limits.
Add LDAP connection timeout, bind DN warning, and multiple entry logging.
Replace Select with Combobox and apply card design tokens in SSOSection.
Expose OIDC scopes field in settings UI.
Fix hardcoded colors to use design system tokens.
Add standard and diagnostic logging throughout SSO flow.
Add tests for role sync, seat limits, LDAP escaping, and config validation.
Remove unused login-form.tsx template.
Update docs and screenshots for SSO feature.
* fix(audit): harden audit log with summary fixes, design compliance, and test coverage
Replace stale /compose/* route summaries with current /stacks/* patterns,
add ~30 missing route summaries for container ops, labels, fleet updates,
SSO, templates, and auto-update. Fix X-Forwarded-For extracting full
proxy chain instead of first client IP. Pre-sort pattern table at module
load for O(1) lookup instead of sorting per request.
Add Calendar and DatePicker UI components (react-day-picker v9 + date-fns)
replacing browser-native date inputs. Align AuditLogView with design system:
Combobox instead of Select, design token colors, strokeWidth 1.5, card bevel,
tabular-nums on all numeric cells.
Add diagnostic logging gated behind developer mode for audit middleware,
query, and export endpoints. Extract getAuditSummary to testable utility.
Add 32 tests covering summary resolution, DB round-trips, API permissions,
export formats, and middleware integration.
* fix(audit): use correct Chevron prop types for react-day-picker compatibility
Security fixes:
- Map deploy-only API tokens to deployer role (not admin)
- Add token versioning to invalidate sessions on password/role changes
- Reject deleted users immediately in auth middleware (no 24h JWT grace)
- Use DB role instead of JWT role so changes take effect instantly
- Block password setting on SSO-provisioned users
- Check proxy variant in scoped permission resolver
Cleanup and logging:
- Remove orphaned role assignments when stacks/nodes are deleted
- Add standard logging for login, user CRUD, role assignments, password changes
- Add diagnostic logging gated behind developer_mode setting
- Extract issueSessionCookie helper (DRY across 5 JWT signing sites)
Frontend:
- Replace Select with Combobox in UsersSection (design system compliance)
- Add strokeWidth={1.5} to Lucide icons
- Hide password fields for SSO-provisioned users
Testing:
- Add 42-test RBAC suite covering user CRUD, token versioning, scoped
assignments, permissions endpoint, password management, seat limits,
last-admin protection, and orphan cleanup
Extract duplicated snapshot capture functions from index.ts and
SchedulerService.ts into a shared module (snapshot-capture.ts). Fix
audit log route patterns that used singular 'snapshot' instead of
plural 'snapshots'. Apply design system to FleetSnapshots component
(card styling, strokeWidth, font-mono, tabular-nums, ScrollArea).
Add safePage pagination, loading toast for creation, and fix the
restore dialog race condition with a controlled AlertDialog. Add
diagnostic logging gated behind Developer Mode. Add tests for
restore endpoint, admin role enforcement, and edge cases. Update
docs with troubleshooting section and refresh screenshots.
* fix(scheduler): clean up stale runs on startup and auto-disable invalid cron tasks
- Add markStaleRunsAsFailed() bulk DB method with status index
- Clean up orphaned 'running' records on scheduler startup
- Auto-disable tasks when cron expression becomes invalid at execution time
- Promote CRUD debug logs to standard logs for scheduled task admin actions
- Add diagnostic logging for task pre-checks, action timing, and prune fallback
* refactor(scheduling): extract shared types and fix design system violations
- Extract ScheduledTask, TaskRun, NodeOption to shared types file
- Extract getCronDescription and formatTimestamp to shared utilities
- Fix formatTimestamp falsy-zero null check
- Tighten last_status type to 'success' | 'failure' | null
- Add strokeWidth={1.5} to all action icons per design system
- Add sr-only DialogDescription for accessibility
- Wrap Sheet run history in ScrollArea
- Fix delete button styling to match design system pattern
- Change manual trigger toast from "executed" to "triggered"
* test(scheduler): add tests for snapshot, remote update, stale cleanup, and cron invalidation
- Add stale run cleanup tests (bulk markStaleRunsAsFailed, logging)
- Add cron invalidation test (auto-disable, error message)
- Add executeSnapshot tests (fleet capture, empty stacks)
- Add executeUpdateRemote tests (proxy success, remote error)
- Document stale run cleanup and cron auto-disable in troubleshooting docs
* fix(scheduler): harden auto-update policies with cascade deletes, error reporting, and UI fixes
- Fix orphaned task runs on policy/node deletion with transaction-wrapped cascade deletes
- Make manual trigger non-blocking (202 Accepted) to prevent proxy timeouts
- Distinguish registry check failures from clean "no update" results via structured ImageCheckResult
- Trim whitespace-only policy names in both frontend and backend validation
- Add strokeWidth={1.5} to action icons per design system
- Add sr-only DialogDescription for Radix accessibility
- Replace Select with Combobox for frequency picker
- Wrap run history sheet content in ScrollArea
- Support concurrent Run Now indicators via Set-based state
- Abort stale stack fetches on node switch with AbortController
- Add standard and diagnostic logging to SchedulerService and ImageUpdateService
- Add tests for cascade deletes, image checking, and scheduler edge cases
- Add troubleshooting section to auto-update docs
* fix(tests): resolve lint errors in image-update-service tests
Remove unused mock variables (mockGetImage, mockGetDocker) and unused
ImageCheckResult type import. Replace CommonJS require('yaml') with
ESM import to satisfy no-require-imports rule.
* chore(deps): bump Docker CLI to 29.4.0 and Compose to v5.1.2
Resolves Trivy CVE-2026-32282 (Go stdlib symlink follow in Root.Chmod)
by upgrading to releases that ship Go 1.25.9. Compose v5.1.2 also bumps
grpc to 1.80.0, resolving CVE-2026-33186.
* chore(security): accept CVE-2026-32282 in .trivyignore, update stale refs
Go stdlib symlink-following in Root.Chmod (CVE-2026-32282) affects both
Docker CLI 29.4.0 (Go 1.26.1) and Compose v5.1.2 (Go 1.25.8). Fix
requires Go 1.25.9 or 1.26.2; no upstream static binary ships a patched
runtime yet. The vulnerable code path requires a chroot context with
attacker-controlled filesystem, which does not apply to our usage.
Also updates version references from v5.1.1/v29.3.1 to v5.1.2/v29.4.0
for existing CVE entries, and notes that Compose v5.1.2 resolved
CVE-2026-33186 (grpc bumped to 1.80.0) for the compose binary.
- Extract log parsing utilities (normalizeContainerName, parseLogTimestamp,
detectLogLevel, stripControlChars, demuxDockerLog) to shared module,
eliminating duplication between polling and SSE endpoints
- Fix timestamp regex to accept timezone offsets (+HH:MM/-HH:MM), not just Z
- Replace any[] with typed GlobalLogEntry interface
- Add SSE heartbeat (30s) to prevent reverse proxy timeouts
- Add X-Accel-Buffering: no header for nginx SSE compatibility
- Replace swallowed catch blocks with console.warn diagnostics
- Fix SSE not reconnecting on node switch (missing dep in effect array)
- Add settings change event so devMode/pollRate updates apply without remount
- Add log level filter (ALL/ERROR/WARN/INFO) to toolbar
- Replace overflow-auto div with ScrollArea for design system compliance
- Add strokeWidth={1.5} to all toolbar icons
- Include stack name in download format
- Surface fetch errors with inline banner
- Add 39 unit tests for all log parsing utilities
- Update docs with level filter documentation and refreshed screenshot
Add authMiddleware to all 13 fleet endpoints that were previously
accessible without authentication. Add NaN validation for parseInt
params, stackName validation on snapshot restore, and description
length cap on snapshot creation. Clean up updateTracker entries on
node deletion to prevent memory leaks.
Replace hardcoded colors with design system tokens, swap Select for
Combobox, replace overflow-y-auto with ScrollArea, fix card styling
(shadow-card-bevel, border tokens). Fix stale container data by
always refetching on stack expand with a loading guard against
concurrent requests.
Add operational logging for state-changing fleet operations and
diagnostic logging gated behind Developer Mode. Add 20 fleet tests
covering auth enforcement, input validation, tier gating, and
snapshot CRUD lifecycle.
Show a pulsating warning dot next to host ports that are already in
use by a running container. Hovering over the dot reveals which
Sencho-managed stack or external app occupies the port.
Adds GET /api/ports/in-use endpoint that returns a map of bound host
ports with ownership info, and a getPortsInUse method on
DockerController that reuses the existing container-to-stack
resolution logic.
* fix(app-store): handle orphaned stack directories on template deploy
When a stack deployed via the App Store is later removed through Docker
Desktop or the CLI (instead of through Sencho), its directory remains on
disk without a compose file. The deploy endpoint previously rejected any
re-deploy with a 409 if the directory existed, even if empty.
Now the endpoint checks for a compose file before rejecting. If the
directory exists but contains no compose file, it is treated as an
orphaned remnant: cleaned up automatically and the deploy proceeds.
Also makes FileSystemService.hasComposeFile public so the deploy
endpoint can reuse it instead of duplicating the compose file check.
* docs(app-store): document orphaned stack directory cleanup behavior
- Sanitize error messages in all delete/prune/create/inspect endpoints
to prevent Docker internals from leaking to the frontend
- Add CIDR, IPv4, and Docker resource ID input validation
- Add requirePaid gate to network topology endpoint
- Add invalidateNodeCaches after image/volume/network mutations
- Fix design system violations: card borders, destructive button variant,
visible DialogDescription, overflow-auto replaced with ScrollArea,
hardcoded Tailwind colors replaced with tokens
- Gate purge button behind isAdmin to prevent silent 403s
- Fix shared inspect loading state to be per-network-row
- Parse error response bodies for meaningful toast messages
- Add clipboard API fallback for non-HTTPS contexts
- Render Options section in network inspect sheet
- Add operational and diagnostic logging for resource operations
- Extend validation and DockerController test suites
- Update docs with Options field in network inspect
* fix(app-store): harden App Store with auth, validation, bug fixes, and design compliance
Add authMiddleware to GET /api/templates and POST /api/templates/deploy
endpoints. Add isValidStackName and isPathWithinBase checks to the deploy
endpoint. Replace fs.existsSync with async fsPromises.access. Extract
FileSystemService to local variable to avoid repeated getInstance calls.
Fix template mutation bug where PUID/PGID/TZ duplicated on re-open by
working on a copy instead of mutating state. Make env_file conditional
in generated compose YAML (only when env vars exist). Add port validation
(range 1-65535) with visual feedback and deploy blocking. Add env key
collision warning toast for custom variables.
Replace any types with proper LSIO API interfaces. Change catch types
from any to unknown with getErrorMessage. Add structured logging with
[Templates] prefix and diagnostic logging gated behind Developer Mode.
Align with design system: remove hardcoded bg-white and text-red-500,
use ScrollArea, fix destructive button variant, add tabular-nums and
strokeWidth 1.5, use cn() for conditional classes. Add empty-registry
state distinct from no-search-results.
Add 16 unit tests for TemplateService covering compose generation,
conditional env_file, env string generation, and cache clearing.
Update App Store docs with port validation and env collision details.
* refactor(app-store): remove unused interface exports
Remove export keyword from interfaces that are only used within their
own file: TemplateEnv, TemplateVolume, and TemplatesResponse in
TemplateService.ts; TemplateEnv and Template in AppStoreView.tsx.
No external consumers import these types.
* fix(app-store): remove unused fs default import
The fs.existsSync call was replaced with fsPromises.access in the
deploy endpoint, leaving the fs default import unused. Remove it
to fix the ESLint no-unused-vars error in CI.
* fix(dashboard): harden real-time dashboard with bug fixes and design compliance
- Fix host alert spam: add 5-minute cooldown for CPU/RAM/disk threshold
alerts, preventing duplicate notifications every 30s during sustained
breaches. Extract shared dispatchWithCooldown helper (also used by
Docker janitor alerts).
- Fix memory metric inflation: subtract filesystem cache from stored
memory_mb values, matching the existing calculateMemoryPercent logic.
- Fix crash detection reliability: replace fragile 'seconds ago' string
matching with a tracked Set of alerted container IDs. Containers are
only alerted once per crash event, with automatic cleanup when they
start running again or after a 1-hour TTL.
- Fix health status bar: exited containers now trigger 'degraded' state
independently of unread error notifications.
- Fix CPU chart Y-axis: auto-scale when aggregate container CPU exceeds
100% instead of silently clipping at the hardcoded domain ceiling.
- Fix grammar: 'actives' to 'active' in container count label.
- Add shadow-card-bevel to all dashboard cards per design system.
- Update dashboard docs to reflect revised health status thresholds.
* test(dashboard): update monitor service tests for new alert signatures
- Add container Id fields to crash detection test fixtures
- Update host alert assertions to match dispatchWithCooldown 3-arg call
- Fix unhealthy container test to use State: 'unhealthy' instead of
State: 'running' (running containers are now skipped in crash detect)
The self-update helper container runs `docker compose up -d
--force-recreate` to recreate the main Sencho container. Previously it
only mounted the docker socket, the compose working directory, and the
data directory. If the user's docker-compose.yml references env_file,
configs, or secrets at paths outside the compose working directory
(e.g. /opt/docker/env/globals.env), the helper could not resolve them
and compose failed with "env file not found".
Now during initialize(), SelfUpdateService collects all host bind mounts
from the container inspect data (filtered to Type=bind). In
triggerUpdate(), these are forwarded to the helper as read-only mounts
at their original host paths (source:source:ro), skipping the socket,
data dir, and compose working dir which are already mounted explicitly.
This lets docker compose resolve any host-path reference the user has
configured, without needing to parse compose files for specific
directives.
* feat(dashboard): drop CPU column and relative timestamp from Stack Health and status bar
The Stack Health table's CPU column duplicated data already surfaced in
the top ResourceGauges and the CPU Usage historical chart. The health
status bar's 'just now' timestamp was cosmetic: no consumer relied on
lastUpdated state for polling, staleness detection, or conditional
rendering. Removing both tightens the dashboard and eliminates a dead
prop chain through useDashboardData.
* refactor: remove dead admin_email field from setup flow
The Setup form captured an admin email under 'Used for license recovery.
Never shared with third parties.' but the value was written to
global_settings and read nowhere: no license recovery, SMTP, or support
contact flow consumed it. Rather than building UI on top of the dead
field, delete the input, the payload key, and the backend persistence.
Any orphaned row from prior setups is harmless and the frontend ignores
unknown settings keys.
* feat(settings): use Radix ScrollArea with per-section scroll memory
Settings Hub used a native-scroll div that snapped to the top every time
the user switched subsections and exposed the default browser scrollbar.
Wrap the nav and content panes with the shadcn ScrollArea (Radix under
the hood, type='hover') and expose a viewportRef so the modal can stash
each section's scrollTop in a ref and restore it via useLayoutEffect on
switch. Style the thumb with translucent foreground tokens so it reads
as glass against popovers and dialogs. Replaces a hand-rolled scroll
hook and ad-hoc CSS utility.
The Skipper/Admiral atomic deploy/update path used to create
.sencho-backup/ inside the user's stack folder, which silently failed
with EACCES whenever a container had chowned the bind mount (swag,
tautulli, linuxserver/* images, etc). That broke auto-rollback and the
manual rollback endpoint for those stacks. Stack backups now live under
<DATA_DIR>/backups/<stackName>/ next to sencho.db, which is always
writable by the Sencho user.
While stress-testing the same scenario, MonitorService also flooded the
error log with "Error parsing stats for container ... 404 no such
container" because per-container stats polls (30s tick) raced with
docker compose recreating containers. The 404 case is now skipped
silently; non-404 stats failures still log at error level.
The "Updating Sencho..." overlay used to dismiss prematurely while the
image pull was still running, after which the local node card would get
stuck in "updating" and eventually surface a generic "Timed Out" error
while the container remained on the old version.
Three root causes are addressed:
1. The image pull was synchronous (`execFileSync`), which blocked the
Node event loop. The overlay's health probe saw the server come back
the moment the pull finished and reloaded the page, even though the
container had not restarted yet. The pull is now async via
`promisify(execFile)`, so /api/health and /api/fleet/update-status
keep serving throughout.
2. The overlay reloaded on the first 200 from /api/health regardless of
whether the underlying process had actually restarted. /api/health
now exposes the gateway boot timestamp, and the overlay captures it
pre-update and only reloads when it observes a different value. A
wasOffline-then-online fallback handles the case where the pre-update
fetch failed.
3. Helper container spawn errors from `docker run` were silently
discarded, so a failed compose recreate never surfaced anywhere.
Errors are now captured into `lastUpdateError` via the execFile
callback and surfaced through the existing /api/fleet/update-status
error path.
A 3-minute early-fail heuristic on the local node block surfaces a clear
failure message when the helper fails silently, instead of waiting the
full 5-minute timeout for an unknown failure.
* fix(fleet): detect updates via GitHub Releases instead of gateway self-comparison
The fleet update check compared each node's version against the gateway's
own version, so the local node could never appear outdated. Now fetches
the actual latest release from GitHub Releases API with a 30-minute
in-memory cache and thundering-herd protection. The Recheck button
invalidates this cache via ?recheck=true to force a fresh lookup.
* docs(fleet): update docs to reflect GitHub Releases version detection
Replace "Gateway version" references with "Latest version" to match the
new label. Document that version comparison uses the latest GitHub release
rather than the gateway's own version, and that Recheck refreshes the
cached latest version.
The self-update feature failed on remote nodes because SelfUpdateService
ran `docker compose -f <host_path>` inside the container, where the host
compose file path does not exist. The fix splits the update into two
steps: (1) pull the latest image directly via `docker pull`, and (2)
spawn a short-lived helper container that mounts the compose directory
from the host and runs `docker compose up --force-recreate`.
Additional changes:
- Use execFileSync/execFile with argument arrays instead of shell strings
to eliminate shell injection surface from Docker label values
- Add Signal 4 completion detection: mark update as completed when the
remote version matches the gateway version (with 15s elapsed guard)
- Extend early failure heuristic from 90s to 3 minutes for slow pulls
- Distinguish "node unreachable" from "node lacks self-update capability"
in error messages; use silent skip in update-all to avoid res crashes
- Add requireAdmin guard to POST /api/system/update
- Handle comma-separated compose config file paths (multiple -f flags)
- Update fleet docs with self-update mechanism, troubleshooting entries
Lifetime licenses have no recurring subscription, so the Lemon Squeezy
customer portal cannot generate a URL. The Manage Subscription button in
Settings already had the isLifetime guard, but the Billing button in the
profile dropdown did not, causing a confusing "No billing portal
available" error.
- Add !license.isLifetime guard to UserProfileDropdown (matches
LicenseSection pattern)
- Move lifetime detection into getBillingPortalUrl() so the service owns
all billing eligibility logic
- Change return type to { url } | { error } discriminated union for
clear error propagation
Show a loading notification with spinner and indeterminate progress bar
while Resource Hub operations are in progress, replacing the dead moment
between confirmation and result.
* fix(auto-update): proxy update execution to remote nodes via Distributed API
Remote auto-update policies previously failed because the scheduler tried
to access the Docker daemon directly on remote nodes. Now the scheduler
detects remote nodes and proxies the update execution via HTTP to the
remote Sencho instance's new /api/auto-update/execute endpoint, which
runs image checks and compose updates locally on the remote machine.
* test(auto-update): add getNode mock to NodeRegistry in scheduler tests
The executeUpdate method now calls NodeRegistry.getNode() to detect
remote nodes. The test mock for NodeRegistry was missing this method,
causing the two executeUpdate tests to fail.
* fix(fleet): resolve stuck update states and improve update UX
The fleet node update flow had several bugs: the in-memory update tracker
never cleared terminal states (timeout, failed, completed), leaving nodes
permanently stuck with no way to retry or dismiss. The Recheck button
only re-fetched stale state without clearing it, and the POST trigger
rejected retries with 409 even after timeout.
Backend fixes:
- Add DELETE endpoints (single node + batch) to clear tracker entries
- Fix 409 race: detect expired timeouts and clear terminal states before
re-triggering
- Populate error messages in the tracker for timeouts and failures
- Include error field in the update-status API response
- Auto-expire completed entries after 60 seconds
Frontend fixes:
- Add retry (RotateCcw) and dismiss (X) buttons on failed/timed-out badges
- Show error details via animated cursor hover (CursorFollow pattern)
- Recheck button now batch-clears all terminal states before fetching
- Recheck shows loading spinner and disables while checking
- Extract NodeCardProps interface for readability
* fix(fleet): detect update completion via process start time
Remote nodes that cannot report their version (e.g. older builds)
caused updates to always time out because completion detection
relied solely on version comparison. The gateway now tracks the
remote node's process start time from /api/meta and detects
container restarts by comparing it across polls.
Also extracts a createTracker() factory to eliminate repeated
object construction across 5 call sites.
* docs: add troubleshooting for first-update timeout on old nodes
Adds a new troubleshooting entry explaining why the first remote
update on nodes running pre-v0.40.0 always times out (neither
version nor process start time can be detected). Documents the
fix: dismiss, recheck, and confirm the node updated.
Also adds a screenshot of the timed-out state with retry/dismiss
buttons to the remote updates feature page.
* fix(fleet): detect update completion via offline detection and error reporting
The update completion detection relied on version change and process
start time, both of which fail on nodes running older Sencho versions
that report "unknown" and lack the startedAt field. This caused every
update to time out after 5 minutes.
Add three-signal detection: version change, process restart (startedAt),
and offline/online detection (node went unreachable during update and
came back). Also add a 90-second early failure heuristic for when the
remote image pull fails silently, and surface pull errors from
SelfUpdateService via /api/meta so the gateway can report them
immediately.
* fix(deps): bump vite to 8.0.5 to resolve high severity vulnerabilities
Fixes GHSA-4w7w-66w2-5vf9, GHSA-v2wj-q39q-566r, GHSA-p9ff-h696-f583.
* fix(deps): bump vite in backend lockfile to resolve audit failures
Vitest pulls in vite as a transitive dependency. Bumps to 8.0.5.
* fix(fleet): resolve version detection pipeline for Docker builds
The Dockerfile backend-builder stage was missing a COPY of the root
package.json, causing generate-version.js to fall back to "0.0.0-dev"
at build time. At runtime, the filesystem walk also failed (root
package.json not in the final image), producing the string "unknown"
which the frontend rendered as "vunknown".
Changes:
- Dockerfile: copy root package.json into backend-builder stage
- CapabilityRegistry: return null (not "unknown") for unresolvable
versions; add isValidVersion() type guard; normalize remote meta
responses to strip "unknown"/"0.0.0-dev" sentinel values
- Fleet endpoints: hoist gateway version validation outside per-node
loops; treat unresolvable remote versions as "potentially outdated"
instead of silently marking them up to date
- FleetView: guard all version display points (card badge, update
button, gateway label, modal columns) via shared formatVersion()
- EditorLayout, CapabilityGate: use shared isValidVersion utility
- New frontend/src/lib/version.ts shared utility
- Docs: add troubleshooting section for version display edge cases
- Screenshots: updated Fleet Overview and Node Updates modal
* docs: update fleet node updates screenshot with live remote node
* fix(licensing): resolve Admiral variant detection and lifetime license handling
The Lemon Squeezy variant name for Admiral licenses contains "Admiral"
(not "Team"), but getVariant() only checked for "team" and "personal".
This caused Admiral licenses to be misidentified as Skipper, locking all
Admiral-exclusive features.
- Map "admiral" variant names to internal "team" value, "skipper" to "personal"
- Add isLifetime field to LicenseInfo API response
- Hide "Manage Subscription" button for lifetime licenses (no billing portal)
- Show "Duration: Lifetime" instead of empty renewal date
- Hide upgrade cards for active Admiral users
- Add 23 unit tests covering variant resolution, tier computation, and lifetime detection
- Add troubleshooting entries for wrong tier label, locked features, and billing portal errors
* fix(licensing): address code review findings
- Fix nested ternary in LicenseSection JSX; restore conditional rendering
to avoid showing an empty "N/A" row for non-subscription states
- Clean up test file: use shared svc variable, remove redundant comments,
add trialDaysRemaining assertions, rename describe block
Eliminate all references to "Pro" across backend, frontend, and docs.
Internal tier value renamed from 'pro' to 'paid'; user-facing text now
uses the thematic tier names (Community, Skipper, Admiral).
- Rename LicenseTier 'pro' to 'paid' in backend and frontend types
- Rename requirePro guard to requirePaid, error code PRO_REQUIRED to PAID_REQUIRED
- Rename ProGate.tsx to PaidGate.tsx with updated copy
- Fix: trial users can now see upgrade/purchase cards in Settings
- Update all docs and openapi.yaml to use correct tier names
* feat(stacks): state-aware sidebar context menu and Open App action
- Context menu now adapts to stack state: running stacks show
Stop/Restart/Update, stopped stacks show Deploy only
- Added "Open App" shortcut to open a stack's web UI directly
from the sidebar (visible when running with a published port)
- Backend bulk status endpoint enriched with mainPort detection
- Reduced manual image update check cooldown from 10 to 2 minutes
- Rate limit error message now derives from the configured constant
* fix(stacks): use const for bulkPorts (prefer-const lint)
* feat(license): distributed license enforcement across multi-node setups
The primary instance's license tier is now asserted to remote nodes on
every proxied HTTP and WebSocket request via trusted headers. Remote
nodes honor the assertion only when the request carries a valid
node_proxy JWT, preventing unauthorized elevation from browsers or API
tokens. Falls back to local license tier for direct access.
* fix(test): remove unused vi import in distributed-license tests
Add the ability to check for outdated nodes and trigger over-the-air
updates from Fleet View. Nodes self-update by pulling the latest Docker
image and recreating their container via the "last breath" pattern.
Backend:
- SelfUpdateService: self-container identification via HOSTNAME + Docker
Compose labels, triggers pull + force-recreate
- CapabilityRegistry: runtime capability disabling via disableCapability()
- POST /api/system/update (202 + deferred self-update)
- GET /api/fleet/update-status (version comparison across fleet)
- POST /api/fleet/nodes/:nodeId/update (single node)
- POST /api/fleet/update-all (bulk remote update)
- In-memory update tracker with 5-min timeout
Frontend:
- Node Updates modal with summary stats, search filter, table layout,
per-node Update buttons, and bulk Update All
- Version badges and update-available indicators on node cards
- ReconnectingOverlay for local node updates (polls /api/health)
- 5s fast-poll when any node is actively updating
- UpdateStatusBadge shared component for consistent badge rendering
Requires Skipper (Pro) tier. Nodes must be deployed via Docker Compose
with Docker socket access.
* feat(nodes): add capability-based node compatibility negotiation
Each Sencho instance now exposes /api/meta with its version and supported
capabilities. When the user switches nodes, the frontend fetches this
metadata and disables features the remote node doesn't support via a
CapabilityGate overlay. Version is shown in the node switcher dropdown
and connection test results.
- Backend: CapabilityRegistry with static capability list and fetchRemoteMeta helper
- Backend: /api/meta (public) and /api/nodes/:id/meta (auth) endpoints
- Frontend: NodeContext enhanced with per-node meta caching (5min TTL)
- Frontend: CapabilityGate component with typed Capability union
- Frontend: 13 features wrapped with capability gates
- Docs: node-compatibility.mdx + OpenAPI spec updates
* fix(nodes): revert to require() for package.json version reading
The static import fails in the Docker multi-stage build because the
root package.json is not copied into the backend-builder stage. The
require() call resolves at runtime when the file is available.
Route stack alerts to specific Discord, Slack, or webhook channels instead
of the single global endpoint. Includes per-rule enable/disable, priority
ordering, and automatic fallback to global agents when no rule matches.
- Add notification_routes table, interface, and CRUD in DatabaseService
- Add routing logic in NotificationService.dispatchAlert with optional stackName
- Pass stack context from MonitorService (crash/health) and SchedulerService
- Add 5 API endpoints gated with requireAdmin + requireAdmiral
- Add NotificationRoutingSection UI with Combobox stack picker, channel tabs
- Parallel webhook dispatch via Promise.allSettled
- 10 unit tests covering routing, fallback, and edge cases
- Documentation with screenshots at docs/features/notification-routing.mdx
* feat(labels): add stack_labels schema and DatabaseService CRUD methods
* feat(labels): add label CRUD, assignment, and bulk action API routes
* feat(labels): add oklch label color palette for light and dark themes
* feat(labels): add LabelPill and LabelDot reusable components
* feat(labels): add LabelAssignPopover component for inline label management
* feat(labels): add label pill bar, label dots, and label assignment to sidebar
* feat(labels): add label filtering and label dots to fleet view
* feat(labels): add label-scoped bulk actions (deploy/stop/restart all)
* docs: add Stack Labels feature documentation
* fix(labels): use context menu sub-menu for label assignment and add settings integration
Replace broken Popover-inside-ContextMenu pattern with native Radix
ContextMenuSub for reliable label toggling on right-click. Wrap
ContextMenuSubContent in a Portal to prevent overflow clipping. Add
"Manage labels..." item that opens Settings directly to Labels section.
Fix close button overlap in LabelsSection header. Add LabelsSection
settings component with full CRUD, assignment counts, and ProGate.
Add initialSection prop to SettingsModal for deep-linking. Include
screenshots for documentation.
* docs: update stack labels documentation with screenshots and corrected instructions
* fix(labels): address security and quality issues from code review
- Add NaN validation on parseInt(req.params.id) in label routes
- Scope updateLabel/deleteLabel by nodeId to prevent cross-node IDOR
- Validate labelIds belong to correct node in setStackLabels
- Add requireAdmin check on bulk action endpoint
- Replace error: any with error: unknown and proper narrowing
- Remove unused Label import from index.ts
- Remove unused isPro prop from LabelsSection
- Add strokeWidth={1.5} to Check icons per design system
* chore: update CHANGELOG with stack labels feature
* feat(nodes): add per-node scheduling and update visibility
Add Schedules and Updates columns to the Nodes table showing active
task counts, next run times, and auto-update status per node. A calendar
action button navigates to filtered schedule/auto-update views.
Backend changes:
- Add node_id to stack_update_status table (migration + unique index)
- Cascade cleanup on node deletion (scheduled_tasks + update status)
- Pre-check target node existence/status before executing scheduled tasks
- New GET /api/nodes/scheduling-summary endpoint
- New GET /api/image-updates/fleet endpoint with 2-minute cache
- Parallelize remote node fetches with Promise.allSettled
- Wrap deleteNode cascade in a transaction
Frontend changes:
- NodeManager: Schedules/Updates columns with summary data fetch
- EditorLayout: sencho-navigate event listener for cross-component nav
- ScheduledOperationsView/AutoUpdatePoliciesView: filterNodeId prop,
filter bar UI, pre-selected node in create dialog
* feat(labels): add stack_labels schema and DatabaseService CRUD methods
* feat(labels): add label CRUD, assignment, and bulk action API routes
* feat(labels): add oklch label color palette for light and dark themes
* feat(labels): add LabelPill and LabelDot reusable components
* feat(labels): add LabelAssignPopover component for inline label management
* feat(labels): add label pill bar, label dots, and label assignment to sidebar
* feat(labels): add label filtering and label dots to fleet view
* feat(labels): add label-scoped bulk actions (deploy/stop/restart all)
* docs: add Stack Labels feature documentation
* fix(labels): use context menu sub-menu for label assignment and add settings integration
Replace broken Popover-inside-ContextMenu pattern with native Radix
ContextMenuSub for reliable label toggling on right-click. Wrap
ContextMenuSubContent in a Portal to prevent overflow clipping. Add
"Manage labels..." item that opens Settings directly to Labels section.
Fix close button overlap in LabelsSection header. Add LabelsSection
settings component with full CRUD, assignment counts, and ProGate.
Add initialSection prop to SettingsModal for deep-linking. Include
screenshots for documentation.
* docs: update stack labels documentation with screenshots and corrected instructions
* fix(labels): address security and quality issues from code review
- Add NaN validation on parseInt(req.params.id) in label routes
- Scope updateLabel/deleteLabel by nodeId to prevent cross-node IDOR
- Validate labelIds belong to correct node in setStackLabels
- Add requireAdmin check on bulk action endpoint
- Replace error: any with error: unknown and proper narrowing
- Remove unused Label import from index.ts
- Remove unused isPro prop from LabelsSection
- Add strokeWidth={1.5} to Check icons per design system
* chore: update CHANGELOG with stack labels feature
* feat(resources): add network management with create, inspect, and topology visualization
Add full Docker network CRUD: create networks with custom drivers, subnets,
and IPAM config; inspect network details including connected containers and
IP addresses; interactive topology graph visualization (Pro-gated to
Skipper/Admiral tiers). Includes backend routes, DockerController methods,
unit tests, documentation with screenshots, and updated changelog.
* refactor(resources): address code review findings for network management
- Add batch GET /api/system/networks/topology endpoint to eliminate N+1
HTTP calls from the topology view
- Export DockerNetwork type from ResourcesView, remove duplicate in
NetworkTopologyView
- Wire up isInspectLoading state to Eye button (spinner + disabled)
- Remove unnecessary wrapper div around FilterToggle
- NetworkTopologyView is now self-contained (fetches from batch endpoint,
no longer needs networks prop)
* fix(resources): align pre-existing UI with design system standards
- Replace hardcoded red-500 on image/volume delete buttons with
text-destructive/60 hover:bg-destructive tokens
- Replace shadow-sm with shadow-card-bevel on all cards (Disk Footprint,
Quick Clean, Resource Tabs, Unmanaged Containers)
- Add strokeWidth={1.5} to all action button Trash2 icons
- Type network drivers as union type instead of raw strings (backend
NetworkDriver type + frontend NETWORK_DRIVERS constant)
* fix(resources): add generic type args to useNodesState/useEdgesState
Fixes TS2345 build error in CI where tsc -b (strict mode via
tsconfig.app.json) infers never[] from untyped empty array literals
passed to React Flow hooks.
* fix(resources): replace explicit any in catch blocks with unknown narrowing
ESLint no-explicit-any errors in CI for three catch blocks added by the
network management feature.
* feat(stack-management): add scan stacks folder button to detect manually-placed compose files
Users who place Docker Compose files directly into the stacks directory
(via SCP, file manager, etc.) can now click the folder-search icon next
to "Create Stack" to immediately discover and surface those stacks in
the sidebar without a full page reload.
* docs(troubleshooting): add scan stacks folder troubleshooting section
Covers common issues: compose file not in subdirectory, unrecognized
filenames, empty directories, and already-tracked stacks.
* docs: remove security-sensitive implementation details from public documentation
Generalize or remove internal architecture details that could aid targeted
attacks — CVE tables, database schema, rate limit thresholds, proxy internals,
encryption algorithm names, and WebSocket middleware bypass info.
* test(metrics): fix flaky minute-bucket aggregation test
Floor baseTime to the start of the current minute so baseTime + 5000
never crosses a minute boundary and produces 2 buckets instead of 1.
- Webhook HMAC: capture raw request bytes via express.json verify callback
instead of re-serializing with JSON.stringify
- AES-256-GCM: use NIST-recommended 12-byte IV (backward compatible with
existing 16-byte IVs)
- Node proxy tokens: add 1-year default expiry (previously no expiry)
- Host console env filtering: pattern-based approach blocking SECRET,
PASSWORD, TOKEN, KEY, CREDENTIAL keywords (previously only 4 explicit keys)
- CORS: deny cross-origin requests when FRONTEND_URL is unset in production
(previously fell back to allowing all origins)
Anso