Extract the Secrets tab's header and empty-card markup into shared
primitives (FleetTabHeading, FleetEmptyState, FleetEmptyCard) and adopt
them across Snapshots, Deployments, Routing, Federation, and Secrets so
the Fleet area presents one consistent layout. The empty card sits
vertically centered below a title/subtitle header. Deployments keeps its
onboarding steps inside the shared shell; Federation's sections are
unchanged.
* feat(mesh): refine the Routing tab and add per-route removal
Tighten the Fleet > Routing tab so its node cards and diagnostics read
honestly for proxy-connected fleets, and remove a couple of dead ends.
- Node cards drop the "pilot connected/offline" line, which was meaningless
for nodes that connect over the HTTP API proxy. The compact card drops the
matching agent cell and now reads stacks / aliases / bridge.
- Enabling mesh on a proxy node shows a transient "Connecting" state and
settles to meshed on its own, instead of flashing "Degraded" with a manual
refresh while the bridge finishes dialing.
- The alias detail sheet gains a "Remove from mesh" action that opts the
alias's owning stack out (the confirmation says how many aliases that
drops), and a "Topology" tab. The standalone topology sheet and its
opt-in-sheet shortcut are removed in favor of the tab.
- The "Add stack" affordance stays reachable on a meshed node, so a node
that already has aliases is no longer a dead end.
- Diagnostics and the alias detail sheet show a transport-aware line
("local", "API proxy bridge", or "Pilot tunnel" with its state) instead of
a fixed "Pilot tunnel" label.
Adds unit coverage for the new state derivation, transport descriptor, the
enable auto-converge, and the admin-gated removal, and updates the mesh docs.
* fix(mesh): harden Routing tab toggle and alias sheet against async races
Independent review surfaced two narrow async races in the new code.
- RoutingNodeCard: cancel the converge re-poll batch at the start of any
toggle (so a slow disable can't let a prior enable's re-polls fire), and
guard the toast, refresh, timer scheduling, and setToggling behind a mounted
ref so nothing runs after the card unmounts while the enable POST is still
in flight.
- MeshRouteDetailSheet: re-check the cancelled flag after reading the
diagnostic body, not just before it. Reading the body is itself async, so a
superseded alias's diagnostic could otherwise call setDiag and expose Remove
for the wrong stack.
Adds tests for unmount-before-enable-resolves and the deferred-diagnostic
alias switch.
The Create Stack button left a redundant `mr-2` on its Plus icon. The
Button base already applies `gap-2`, so the margin doubled the spacing
and pushed the centered icon-plus-label block right of center. Removing
`mr-2` restores the single correct gap.
The meshed-state routing CTA rendered both a Plus icon (via ctaIconFor)
and a literal "+ " prefix in the text, showing two plus glyphs. Dropped
the literal prefix so only the icon renders.
* feat(pricing): collapse to two tiers (Community + Admiral)
Collapse Sencho's pricing from three tiers (Community / Skipper / Admiral)
to two: a generous free Community tier and a single paid Admiral tier. The
Skipper tier is removed.
Now free in Community: auto-heal, auto-update, scheduled operations,
webhooks, notification routing, Fleet Actions and bulk operations, SSO
preset providers (Google / GitHub / Okta), unlimited users with admin and
viewer roles, and deploy safety (atomic deploys, auto-rollback, and
one-click rollback).
Admiral (paid) is focused on running and governing a fleet: blueprints,
Fleet Secrets, deploy enforcement, vulnerability report export, audit log,
host console, private registries, mesh networking, node cordon, managed
cloud backup, LDAP / Active Directory SSO, and the advanced RBAC roles
(deployer, node-admin, auditor) with per-resource scoped assignments.
Internally the license variant distinction is removed so tier is binary
(community / paid). License validation still verifies the Lemon Squeezy
store and product before granting paid status.
Docs and the contributor guide are updated to the two-tier model.
* docs(pricing): correct licensing page to two-tier pricing and tidy stale tier wording
The licensing docs page kept the old Admiral pricing plus a Founder
Lifetime column and an Enterprise paragraph after the two-tier collapse.
Update it to $12/month or $99/year, drop the lifetime and Enterprise
content, and link to the pricing page for current pricing.
Also fix stale "Skipper" wording in CLA.md, SUPPORT.md, one test title,
and three test comments. Historical CHANGELOG entries and the
retired-Skipper license-guard test are intentionally left as-is.
* docs: align licensing and SSO pages with the two-tier model
Correct the SSO overview so the Google, GitHub, and Okta presets read as
available on every tier, matching the provider table; only LDAP and Active
Directory require Sencho Admiral. Remove the lifetime-plan references from the
licensing, settings, and troubleshooting pages so they reflect subscription-only
Admiral pricing.
* fix(rbac): omit scoped permissions from /me on the Community tier
Scoped role assignments only take effect on the paid tier, but GET /api/permissions/me returned them unconditionally, so a downgraded instance with leftover assignments rendered per-resource affordances the API then rejected with 403. The endpoint now mirrors the permission middleware and includes scoped permissions only on the paid tier. Adds a regression test covering the downgrade case.
* docs: use custom-pricing wording on the contact page
The two-tier model has no Enterprise tier; reword the contact page's enterprise pricing/deals to custom pricing/deals so it does not imply a tier that no longer exists.
* feat(appearance): add theme, accent, contrast, and typography personalization
Expand Settings to Appearance into a full personalization surface and add a
quick switcher to the top bar (the palette button between search and
notifications). Choices are saved to the browser, sync across tabs, and apply
before first paint so there is no flash on reload.
- Themes: Dim (the default raised charcoal), OLED true black, Light, and Auto
(follows the OS and re-resolves live when it flips).
- Accent: an eight-hue wheel (cyan default) that drives the one data color
across charts, rails, focus rings, active states, and the ambient glow.
- Fine-tune sliders: a master Contrast that spreads page, ink, and borders
together, plus Border brightness and Ambient glow.
- Typography: swappable interface (Geist / IBM Plex Sans / Hanken Grotesk) and
data (Geist Mono / IBM Plex Mono / Fira Code) faces, and a text-size control
(continuous slider in Settings, S/M/L/XL presets in the popover, kept in
sync). The display serif stays locked as the signature face.
Surfaces, borders, and ink derive from per-theme lightness values so the live
knobs scale the whole UI through CSS, and the opaque directional borders read on
every panel including true black. A live preview reflects changes in real time.
Adds a documentation page under Features.
* fix(appearance): keep contrast-driven tokens in gamut and add picker keyboard nav
Clamp the contrast and knob driven surface, border, and ink lightness so the
full slider range stays a valid color. The page now always sits below the card
tone, so page/card separation no longer collapses at high contrast in Light; the
lit border edge never reaches white in Light at the low end of the knobs; and the
OLED and Dim extremes resolve to valid black/white instead of out-of-range values.
Add the radiogroup keyboard model (roving tabindex plus Arrow / Home / End) to the
accent and type pickers through a shared hook, matching the segmented control. One
item is tabbable and arrow keys move focus and selection together.
* fix(settings): reflect role, tier, and node scope in settings panels
Three gate fixes so the Settings panels match what the backend enforces.
Non-admin roles saw editable fields and a Save button on System Limits,
Developer, and App Store, but writes require admin, so Save always failed.
These panels now render read-only for non-admins (controls disabled, Save
hidden) while still showing the values.
The Developer panel is node-scoped but read and wrote the controlling
instance regardless of the selected node, so a remote node's debug mode and
retention windows could not be changed from the UI. It now targets the
active node like System Limits.
The settings shell derived the Admiral entitlement locally; it now consumes
the backend-provided value the API authorizes against, and that value is
corrected to require an active paid tier so an expired Admiral license no
longer reports as Admiral.
* fix(settings): gate audit_retention_days writes behind Admiral
audit_retention_days configures the Admiral-only audit log (the audit-log
routes require Admiral, and the Developer settings UI only shows the field to
Admiral operators), but the settings POST/PATCH handlers only required an admin
role. A non-Admiral admin (for example Skipper, or an expired-Admiral admin
whose tier dropped to community) could still set it through the API.
Gate writes to that key with requireAdmiral on both the single-key POST and the
bulk PATCH paths, matching the audit-log routes and the UI. Other keys remain
writable by any admin.
* fix(settings): reject unknown keys on PATCH instead of silently stripping
The bulk settings PATCH validated the body with a Zod object schema that
strips unknown keys by default, so a request carrying a disallowed key (for
example an auth_* secret) returned 200 as a no-op instead of being rejected.
No secret was written, but it diverged from the single-key POST path, which
rejects disallowed keys, and could hide client drift.
PATCH now rejects any key outside the allowlist with a 400 before validation
or write, keeping the bulk path fail-closed and consistent with POST.
The settings read returned the entire global_settings map minus a small
denylist of auth keys. Other subsystems persist their config in the same
table, so cloud backup values (endpoint, bucket, access key) were returned
to any authenticated user, including read-only roles, bypassing the
redaction the dedicated cloud-backup endpoint applies.
Project the response from the operational allowlist instead, so only the
keys the settings UI reads are returned and any key written to
global_settings by another subsystem is excluded by default. Remove the
now-unused denylist constant.
* fix(metrics): host memory usage excludes reclaimable page cache
Host RAM was computed as mem.used / mem.total via systeminformation, but mem.used counts
reclaimable buffers/cache as used, so a busy Linux host read ~99%. Switch the dashboard stats,
the fleet node self-report, and the host-RAM alert threshold to mem.active / mem.total
(cache-excluded), and report used: mem.active and free: mem.available so the byte readout stays
consistent with the percentage. Add regression tests for a cache-heavy host (no false alert) and
a genuinely busy host (alert still fires).
* test(metrics): assert system stats memory excludes reclaimable cache
The cached /api/system/stats test mocked si.mem() without active/available, so after
the route switched to the cache-excluded fields it produced a NaN percentage that the
shape-only assertions did not catch. Fill the mock with a realistic shape and assert the
route reports the active working set, not the cache-inclusive used/free.
Show the first-run environment preflight step in the quickstart so the
setup walkthrough is fully illustrated between the Cold start card and the
dashboard. The screenshot renders the six checks (Docker engine, Compose,
compose directory, path mapping, TLS, disk) with their results, the Re-run
control, and the Enter Sencho action.
* fix(api-tokens): scope per-token rate limits to live tokens
Forged or token-shaped Authorization headers no longer mint their own
rate-limit budget. The key generator now grants a per-token budget only to
a real, active token and falls back to per-IP keying for anything else, so
a single source cannot evade the global limiter by rotating fake tokens.
The validated token is memoized on the request, so authentication reuses
it without a second database lookup.
Token validation (format, checksum, lookup, revocation, expiry) is now a
single shared helper used by the HTTP auth middleware, the WebSocket
upgrade handler, and the rate-limit key generator, replacing two
near-identical inline copies that could drift apart. The last-used
timestamp write is throttled so a busy token no longer writes to the
database on every request.
* fix(api-tokens): surface token list-load failures with a retry
A failed load of the API tokens list was swallowed: a server error
rendered the empty "no tokens yet" state with no sign that anything went
wrong. The list now shows an error card with a Retry action and raises a
toast on any non-ok response or network error, matching the create and
revoke flows. Adds a troubleshooting entry for the error.
* test(api-tokens): seed tokens via the shared test helper
The new hardening and WS-scope suites computed sha256 of a raw token
directly, which CodeQL flags as js/insufficient-password-hash (a false
positive: these are 256-bit CSPRNG opaque tokens, not passwords). Route
token creation through the existing apiTokenTestHelper and read the stored
token_hash back from the row, so the suites no longer hash anything
themselves. Also removes the duplicated createToken helpers.
* fix(api-tokens): key the rate limiter by the same credential auth uses
The rate-limit key generator checked the session cookie before the
Authorization bearer, while authMiddleware authenticates bearer-over-cookie
(bearerToken || cookieToken). A request could send a Bearer API token plus a
forged cookie and be keyed by the cookie's (forgeable, rotatable) username,
sidestepping the per-token / per-IP keying the limiter applies to API tokens:
a valid token would lose its own bucket, and a forged token-shaped bearer
would no longer collapse to per-IP.
Reorder the generator to mirror auth: process the bearer first (validate the
API token and key per-token or fall back to per-IP; otherwise decode the JWT
by username/sub), and consult the cookie only when there is no bearer.
Regression tests cover a valid and a forged sen_sk_ bearer, each sent with a
forged cookie.
The dashboard probes GET /api/stacks/<name>/git-source for every stack to
decide whether to show a Git badge. For a stack with no Git source attached
the endpoint answered 404, so a fleet of unlinked stacks painted a red 404
per stack in the browser console.
Return 200 { linked: false } when the stack exists but has no Git source,
and reserve 404 for the genuine "stack does not exist" case (mirroring the
existence guard the PUT handler already uses). The two consumers that read
this endpoint now treat the { linked: false } sentinel as unlinked rather
than as a configured source.
The "records the download metric exactly once" test in stack-files-routes
intermittently failed with "expected 2 to be 1". The download route records
its metric asynchronously, on the file stream's end/close and the response's
close event, so a prior download test's record could land after the next test
called resetFileExplorerMetrics(), leaking a count across the reset. The
route's per-request recording is correct (a synchronous guard); the leak was
purely cross-test in the suite.
Add an afterEach in the download suite that waits for the download count to
settle (bounded poll) before the next test runs, so no pending record survives
the reset, and assert the count actually settled so a runaway record surfaces
rather than being silently swallowed.
* feat(onboarding): add first-run environment checker
Add a preflight that checks whether the host can run Docker deploys before a
deploy fails for an avoidable reason. It verifies the Docker engine is reachable
and permitted, the Compose plugin is present, the compose directory is writable
and mounted at a matching host path, the dashboard is behind TLS, and the
compose volume has disk headroom. Each result that needs attention carries a
specific fix rather than a generic error, and the checks never block: an
operator who knows their setup can continue.
The checks run as the final step of first-boot setup and can be re-run any time
from the Recovery settings tab. A new admin-only endpoint,
GET /api/diagnostics/environment, backs both surfaces.
* fix(onboarding): distinguish unverified path mapping and support parent binds
Treat a container whose self-inspect fails as an unverified path-mapping warning
instead of a false "not containerized" pass, so an unverifiable mapping never
reads as healthy. Resolve the compose directory through the longest-prefix bind
mount and compare the host path it resolves to, so a parent bind such as
-v /opt:/opt correctly covers COMPOSE_DIR=/opt/compose instead of warning that
the directory is not bind-mounted.
* test(e2e): advance the setup wizard past the environment step in loginAs
The first-run setup helper clicked "Initialize console" and immediately waited
for the dashboard, but setup now shows an environment-preflight step before
landing the console. Click "Enter Sencho" to complete onboarding before
asserting the dashboard, so the first test on a fresh instance passes.
The notifications panel opened a per-node WebSocket and a 60s REST poll for
every remote node regardless of reachability. A remote node marked offline
(for example a pilot node whose tunnel dropped) still got a socket, which
failed the handshake and reconnected forever, plus a poll that returned 502
on every cycle.
Filter remote nodes by status before both fan-out points so an offline node
gets no socket and no poll. The existing cleanup loop closes the socket of any
node that leaves the active set, so a node transitioning to offline is torn
down too; an unprobed unknown-status node still subscribes. Mirrors the
existing skip-offline idiom used by the cross-node stack search.
The create endpoint reads the new stack name from the body field
stackName, but a missing or non-string value returned "Stack name is
required and must be a string", which points at a value problem and
leaves anyone scripting against the stacks API unsure which field to
fix. Name the field in the message so the cause is unambiguous, and add
regression coverage for the missing and non-string cases.
- Note that managing the mesh (the per-node toggle and stack opt-in or
opt-out) requires an administrator and that non-admin users see the
Routing tab read-only, matching the access model the tab enforces.
- Document the [Mesh:diag] developer-mode logs in the Diagnostics section
so operators can trace routing decisions and operation timing.
- Add SENCHO_MESH_SUBNET to .env.example; it was documented on the feature
page but missing from the env template.
* feat(mesh): add developer-mode diagnostics to the mesh data plane
Add a developer-mode-gated [Mesh:diag] diagnostic log to MeshService,
following the existing diagnostic-logging pattern: gated on the shared
developer_mode setting (off in production by default) and every value run
through sanitizeForLog. The diagnostics cover the forwarder dispatch
decision (which self-node resolution won, and whether the route is
same-node or cross-node), the entry of the opt-in, opt-out, enable, and
disable operations, and completion timing for opt-in, node disable, and
the alias-cache refresh. This lets an operator trace a slow or stuck mesh
operation from the logs without attaching a debugger. The calls sit only
at per-operation, per-accept, and per-refresh cadences, never inside the
per-frame byte-relay loops.
Add tests covering the developer_mode on/off gating and a guard that a
node's api token never reaches a diagnostic log or the activity buffer.
* refactor(mesh): redact secret-shaped values in developer-mode diagnostics
Run each [Mesh:diag] detail through redactSensitiveText before
sanitizeForLog so a future caller cannot leak a Bearer token, JWT, or
credentialed URL through a diagnostic line, regardless of which call site
emits it. No current call site passes a secret; this is defense in depth.
Use a JWT-shaped canary in the no-leak test so it guards the real credential
class.
* feat(recovery): add safe-mode recovery surface and emergency CLI
Add a read-only Recovery tab under Settings (admin-only) backed by a new
GET /api/diagnostics endpoint reporting app version, database integrity,
encryption-key status, Docker reachability, account and SSO counts, and
non-secret configuration. The endpoint loads without Docker or live metrics
so it stays available when the dashboard does not, requires a genuine admin
session, and builds its config block from a non-secret allowlist so no
credentials are ever exposed.
Expand the emergency command-line toolkit beyond the two-factor reset with
seven host-level commands: reset-password, create-emergency-admin,
clear-sessions, disable-sso, diagnostics, validate-db, and backup-data. Each
prints its result, exits with a meaningful status code, and writes an audit
entry where it changes state.
Document the toolkit in a new operator guide and link it from the recovery
and two-factor pages.
* feat(recovery): download the emergency command reference as a text file
The recovery commands are needed exactly when the dashboard is unreachable,
so reading them only in-app is a chicken-and-egg problem. Add a Download
button to the command-line section that saves the full
`docker compose exec sencho ...` reference as a text file, letting operators
keep it on hand before they need it. Reuses a shared download helper with the
existing diagnostics export.
* fix(recovery): harden diagnostics, backup, and emergency-admin against edge cases
Address findings from an independent review of the recovery toolkit:
- DiagnosticsService now degrades instead of throwing when a queried table is
missing or corrupt: each read falls back and is folded into database.ok, so a
broken database reports "problem detected" rather than failing the whole
endpoint or showing a misleading healthy state with zeroed counts.
- backup-data refuses a destination that resolves to the live database, which
would otherwise report success while producing no separate copy.
- create-emergency-admin now applies the same username rule as the user-
management route, extracted to a shared helper so both stay in sync.
Adds tests for a missing read table, a malformed emergency-admin username, and
the backup same-target rejection.
* feat(stacks): add guided first stack import flow
Add an Import mode to the Create Stack dialog and a zero-stacks empty
state so a new user who already has compose files on disk can land their
first stack without reading the docs first.
A read-only scan of the compose directory (GET /api/stacks/import/scan)
lists the compose files it finds with a dry preview of each file's
services, ports, volumes, and env files. Each result is labelled by
placement: already a stack, loose at the root of the compose directory,
or one folder too deep, with the exact path to move misplaced files to.
The scan never writes, moves, or changes any files.
Manual stack creation (Empty, From Git, From Docker Run) is unchanged.
* fix(stacks): read import-scan candidates via a single file handle
Open the compose file once and stat plus read on the same descriptor so
the size check and the read observe the same inode, instead of resolving
the path twice (stat then readFile), which is a time-of-check/time-of-use
race. Mirrors the existing handle-based readers in FileSystemService.
* fix(stacks): confine import scan to the compose dir and refine the empty state
Harden the read-only import scan:
- Resolve symlinks and confirm the real target stays inside the compose
directory before reading a candidate, and reject non-regular files, so a
symlinked compose file or parent cannot expose a file outside the compose
directory through the preview (matches resolveSafeStackPath).
- Read at most the stat-reported size (bounded by the 1 MiB cap) from the open
handle, so a file that grows after the size check cannot exceed the cap.
- Log when the compose directory or a subdirectory cannot be read, so an access
failure is not silently reported as "no compose files found".
Only show the first-run "No stacks yet" prompt when no filter chip is active, so
a filter that matches nothing is not mistaken for an empty fleet.
* fix(mesh): hide node and stack management controls from non-admins
The Routing tab rendered the per-node mesh enable/disable toggle and the
stack opt-in/opt-out controls for any Admiral-tier user, but those backend
routes require the admin role. A non-admin viewer on an Admiral instance
saw controls that returned 403.
Thread a canManage flag (true only for admins) from the Fleet view into
the Routing tab, its node cards, and the opt-in sheet so non-admins get a
read-only Routing tab: the enable/disable toggle, add-stack, and
opt-in/opt-out controls are hidden, while status, aliases, topology,
activity, diagnostics, and the alias test probe stay available. This
mirrors the Federation tab's existing read-only treatment for non-admins.
Add backend route-gating tests covering the tier and admin-role guards on
every mesh route, and frontend render-gate tests for the node card and the
opt-in sheet in both density layouts.
* refactor(mesh): require canManage on the routing-node-card primitive
Remove the permissive `canManage = true` default on the shared
routing-node-card primitive so a new call site cannot render the
management controls without an explicit decision. Every current caller
already passes the flag; the type now enforces it. Drop the omitted-prop
test, which covered a state the compiler now prevents.
Add a consolidated Recovery page that answers "what do I do if Sencho
breaks?" in one place. It covers getting back to a working state when
Sencho itself fails, a stack deploy fails, an admin is locked out of
sign-in, Docker is unavailable, or a remote node is unreachable, and
summarizes each path with a link to the authoritative detail page rather
than duplicating it.
The page opens by stating the core safety fact: Sencho keeps its state in
DATA_DIR and COMPOSE_DIR and treats compose files on disk as the source of
truth, so recovery is mostly file-level and running containers are
unaffected by a Sencho outage. It points to Backup & Restore as the
prerequisite.
Link the guide from the README documentation section, the quickstart
"where to next" cards, and the top of the troubleshooting page.
Node read endpoints now return a client-safe projection that omits the
stored api_token and exposes a has_token boolean instead, so a node's
long-lived proxy credential is never serialized to a browser or API
token client. The token stays encrypted at rest and is read server-side
only by the components that need it (the remote proxy, the connection
test, and the mesh dialer).
The edit form opens the API Token field blank, and a blank value keeps
the existing credential, so saving an edit without retyping the token no
longer clears it; a non-empty value rotates it. The backend enforces the
same rule defensively.
Node management actions (add, edit, delete, test connection, generate
node token) are gated in the UI to match their server-side permission
checks, so operators no longer see an action the API would reject. The
test-connection route also gains the missing server-side permission and
token-scope guards.
Also validate the x-node-id header and fall back to the default node for
malformed values instead of an obscure 404, and return 400 (not 500)
when deleting the default node.
* fix(nodes): gate node-management actions by role and release pilot tunnels on delete
The node-management write actions in the Nodes panel (add, edit, delete,
generate node token, reset fleet-sync anchor) rendered for every signed-in role,
but the API enforces the manage-nodes permission on them, so lower-privilege
roles saw buttons that returned 403. The panel now renders each action against
the same permission its route enforces; the read-only node table stays visible
to every role.
Deleting a node now also tears down its live pilot-agent tunnel (and any mesh
bridge) immediately, releasing the loopback server, heartbeat timer, and open
streams instead of leaving them until the agent next disconnects, matching the
cleanup the re-enrollment path already performed.
Adds backend route tests for the permission boundaries and tunnel teardown, and
a Nodes panel render test covering the viewer, admin, and node-admin views.
* fix(nodes): close proxy mesh bridges via the dialer on node delete to skip a redial
Deleting a node closed any active mesh bridge through PilotTunnelManager, but a
proxy-mode bridge is owned by the mesh dialer, whose close listener then treated
the close as unexpected and scheduled a reactive redial against the node being
removed. The delete handler now closes a proxy bridge through the dialer's
intentional-close path first (which suppresses the redial), then closes a
pilot-agent tunnel as before. Adds a backend test that primes a live proxy
bridge and asserts deletion closes it without scheduling a redial.
* fix(stacks): reflect apply progress and clear the anatomy update banner
The "Update available" banner in the stack Anatomy panel ran its apply
in the background but gave no feedback: the apply button stayed idle and
the banner never went away once the update finished.
Pass an in-flight flag to the panel so the apply button disables and
shows progress while the update runs, and re-check the update preview
when the apply finishes so the banner clears on success and stays in
place when the update did not take effect.
* fix(stacks): scope the update re-check to its stack and keep the banner on refresh failure
The post-apply re-check of the Anatomy update banner tracked only the
in-flight flag, so switching stacks while the first was still updating
looked like a completion for the newly selected stack and triggered a
stray preview request that could clear that stack's valid banner. Track
the stack name alongside the flag and only re-check when the completion
belongs to the stack on screen.
Also keep the banner when the re-check itself fails (non-OK response or
network error) instead of clearing it, so a transient read failure can
no longer hide an update that may still be pending. Both failure paths
now log for diagnosis.
* fix(blueprints): stop the blueprint detail sheet flickering while open
The Fleet > Deployments blueprint detail sheet reloaded its data on a short
timer while open, flickering the body through its loading skeleton every few
seconds.
The sheet's load effect was keyed on the refresh callback, which was memoized
with the parent's onOpenChange prop. The parent passes a fresh onOpenChange
closure on every render, so each parent re-render (driven by the Fleet view's
polling) recreated refresh, re-ran the load effect, and refetched the
blueprint. Every refetch flipped the loading flag, swapping the populated body
for skeletons and back.
Hold onOpenChange in a ref so refresh depends only on blueprintId. The load
effect now runs on open and on blueprint change, not on every parent render.
Switching to a different blueprint still refetches.
* fix(blueprints): keep the blueprint detail body visible during a refresh
After the sheet has loaded, a reload triggered by an action (apply, withdraw,
accept, enable/disable, save) flipped the loading flag and swapped the whole
body for skeletons until the reload finished, a brief flash on every action.
Gate the skeleton on whether the blueprint has loaded yet, not on the loading
flag. The skeleton now shows only on the first load; later reloads keep the
populated body, deployment table, and compose source on screen while they run.
* fix(blueprints): show progress while a fresh deploy runs
Confirming a fresh deploy from the blueprint detail sheet starts a remote
deploy that can take several seconds. The button only became disabled with no
other change, so the action looked frozen.
Show a spinner and a "Deploying" label on the button while the deploy runs, and
dim it, so the click clearly registers as in progress.
The file-download handler recorded its in-process download metric off the
source stream's end/close events. When the response consumer closed right
after receiving the whole file, those source events could be dropped and the
metric was never recorded, so the per-node download counter undercounted
successful reads under load.
Record the success directly on response close once the file has been fully
read, instead of waiting on source events that may never arrive. The abort
path (partial read) is unchanged and the recorder stays idempotent, so a
later source end/close cannot double-count. Adds a deterministic regression
test for the response-close-after-full-read path.
* fix(federation): gate node cordon control on node:manage to match backend
The cordon and uncordon control on the node card rendered whenever the
instance held an Admiral license, but the backend route also requires the
node:manage permission. Non-admin users without that permission (deployer,
viewer, auditor) saw a Cordon action the API rejected with 403. Gate the
control on the same permission the route enforces, so it renders only for
users who can use it.
Add developer-mode diagnostic logging to the cordon and pin handlers, and
route-level tests covering the cordon and uncordon permission, tier,
API-token, and input-validation boundaries.
* fix(federation): reject non-numeric node ids in cordon/uncordon
The cordon and uncordon routes validated the node id with parseInt, which
accepts numeric-prefix strings (parseInt('1abc', 10) === 1), so a request to
/api/nodes/1abc/cordon would operate on node 1. Require a strict positive
integer before parsing, and add tests for both routes.
* fix(federation): sanitize user-derived values in cordon and pin diagnostics
Route the node id, blueprint id, target node id, and reason length through
the shared log sanitizer before they reach the developer-mode diagnostic
log lines, and switch the format specifiers to %s to match. The values are
already validated integers, so this is defense in depth at the log sink and
keeps the diagnostics on the same sanitize-every-interpolated-value pattern
used elsewhere. No runtime behavior change: the lines stay gated behind the
developer_mode setting, off by default.
Three improvements to the Fleet Snapshots detail view, matching the admin-only
access of the existing per-stack restore:
- Restore all: a control in the snapshot header restores every captured stack
across the fleet in one action, with an optional "redeploy all after restore"
checkbox. Each stack is restored independently, so a failure on one (removed
node, offline remote, blocked deploy) is reported per stack while the rest
still proceed. Backed by POST /api/fleet/snapshots/:id/restore-all.
- Per-file download: each compose or .env file in a snapshot can be saved to
disk individually from its row.
- Scrollable preview: the inline file preview is now a bounded, scrollable
panel, so a long compose file can be read in full instead of being clipped.
Remote restore and redeploy failures now carry the remote node's status and
reason, so a per-stack failure in Restore all is actionable.
* fix(fleet-secrets): restrict bundle management to admin hub sessions
Fleet Secrets exposes decrypted environment-variable values and writes
credentials across the fleet, so every route now requires an admin role,
runs only on the instance you are signed into, and rejects long-lived API
tokens:
- Add an admin-role check to all secrets routes; the frontend Secrets tab
renders only for admin users so the affordance matches the backend gate.
- Add /api/secrets/ to the hub-only path list so a request carrying a
remote node id cannot be proxied to read another node's decrypted values.
- Reject API tokens on every secrets route (browser admin sessions only),
matching how registry credentials are handled.
Also adds lifecycle and developer-mode diagnostic logging (never the secret
values) and tests covering the admin boundary on every endpoint, API-token
rejection, hub-only enforcement, and diagnostic gating.
* fix(fleet-secrets): require a signed-in user session for all secrets routes
The earlier API-token rejection only blocked opaque API tokens. node_proxy
and pilot_tunnel JWTs are mapped to an admin role by the auth middleware
without an API-token scope, so they still passed the admin gate and could
read decrypted bundles via GET /api/secrets/:id.
Replace the API-token check with requireUserSession, which rejects API tokens
and node_proxy / pilot_tunnel machine credentials (userId 0) on every secrets
route, returning SESSION_REQUIRED. The admin role is still enforced after.
Tests now assert SESSION_REQUIRED for a full-admin API token across all nine
routes and for node_proxy and pilot_tunnel JWTs.
* test(fleet-secrets): mint the rejection-test token via the real endpoint
The machine-credential test reconstructed an API token by sha256-hashing a
raw key inline. That duplicated a hashing sink that CodeQL's
js/insufficient-password-hash query flags (a false positive for a 256-bit
random token, but a new occurrence in the diff). Create the token through
POST /api/api-tokens instead, so the hashing stays in the production path
and the test carries none of its own. Behavior and coverage are unchanged.
* fix(fleet-snapshots): gate reads on admin role and encrypt content at rest
Fleet snapshots capture every node's compose.yaml and .env, so the data is
as sensitive as the live stacks. This hardens access and reliability across
the snapshot pipeline.
- Restrict snapshot reads to administrators. GET /api/fleet/snapshots and
/:id now require the admin role, matching create, restore, and delete; the
Fleet "Snapshots" tab and its panel render only for admins. Previously any
authenticated user could enumerate snapshots and read every node's .env.
- Encrypt snapshot file contents at rest with the instance key. Restore and
cloud-archive paths decrypt on read, so cloud archives stay portable and a
database copy no longer exposes stack secrets in plaintext. Rows written
before this change still read back as plaintext.
- Surface partial captures. A stack whose compose file cannot be read or
fetched, or a file over the 1 MB capture cap, is recorded as a warning and
shown on the snapshot instead of being silently dropped, so a snapshot is
never mistaken for complete. Remote .env read errors are now distinguished
from a genuinely absent .env.
Adds route-authz, capture-warning, and encryption round-trip tests; updates
the Fleet-Wide Backups feature docs.
* fix(fleet-snapshots): gate cloud snapshot reads on admin role
The cloud snapshot read routes were guarded by provider/license only, not by
role, while their write counterparts (upload, delete) already required admin
and the Cloud Backup settings surface is admin-only. Because a downloaded
archive contains plaintext compose and .env files, a non-admin could list and
download cloud snapshots and read every node's secrets, the same exposure the
local snapshot reads were just closed against.
- Require admin on GET /api/cloud-backup/snapshots, /status/:id, and
/object/:keyB64/download, matching the local snapshot reads and the
admin-only Cloud Backup settings section.
- When capturing a remote node, treat a 200 response carrying X-Env-Exists:
false as a stack with no .env (matching the local ENOENT path) instead of
storing an empty .env that restore would later write back.
Adds non-admin authorization tests for the cloud read routes and a remote
absent-.env capture test.
* fix(fleet): gate node update actions to admins and harden update tracking
Node update affordances now render only for admins, matching the admin-only
routes behind them. Previously a non-admin could open the Fleet view and see the
per-node Update button, Update all, retry, dismiss, and Recheck controls, then
get a 403 on click. Those controls are now hidden for non-admins, who still see
read-only update status.
Both update-status clear routes (per-node and bulk) now require admin, and the
bulk recheck throttles its forced "latest published version" lookup so a caller
cannot loop it to hammer the upstream registries; the response reports whether
the refresh actually ran so the UI can surface a "checked recently" note.
Completion detection no longer reports a node as Updated when it merely blips
offline and returns on the same version with an unchanged process start time.
That case stays in progress and is decided by the existing early-fail and
timeout heuristics, so a momentary network glitch is not mistaken for a
successful update. Failed and timed-out updates now emit an operator-visible
warning, and a periodic safety-net sweep bounds in-flight trackers when no
client is polling for status.
* fix(fleet): harden update completion and recheck failure handling
Refinements from review of the node self-update hardening:
- Completion signal 1 now requires a valid version, not merely a different one.
A node whose /api/meta momentarily omits or mangles its version (online, same
process) reported version=null, which compared unequal to the previous version
and falsely marked the update completed. It now stays in progress and is
decided by the early-fail/timeout heuristics.
- Terminal resolution is atomic: it re-reads the live tracker and transitions
only if it is still in flight with the same start time, so two concurrent
status polls cannot both warn or clobber each other's transition.
- The operator warning for a failed or timed-out update now redacts
secret-shaped text (bearer/basic/token/password, credentialed URLs) from the
underlying error before logging, in addition to stripping control characters.
- The Recheck button now surfaces an error toast when the request throws
(network or auth failure), matching the existing non-ok-response path instead
of only logging to the console.
* feat(fleet-sync): gate sync-status polling on admin role
The fleet sync status poll requires a paid admin on the server, but the
useFleetSyncStatus hook only checked the paid tier. A paid non-admin who
opened the Fleet Status tab or Settings, Nodes would poll the endpoint
every 30 seconds and get a 403 each time, with the policy sync rows
silently never loading. Gate the hook on both paid tier and admin role
so the client mirrors the server and the request only fires when it can
succeed.
Also add a one-time log when an instance transitions from control to
replica, and a real-database integration test covering the full receive
path: role flip, identity and watermark persistence, wholesale row
replacement with local rows preserved, replica read-only enforcement,
and the stale-watermark and control-anchor rejection branches. Includes
a hook test asserting the gate across the paid and admin matrix.
* fix(fleet-sync): drop sync-status rows that resolve after the gate flips
A sync-status fetch started while the user was an eligible paid admin
could resolve after the user lost eligibility mid-flight (role or tier
change), re-populating the status rows that the gate-false path had
already cleared. That briefly re-enabled the policy-sync rows and the
anchor-mismatch banner for a now-ineligible client.
Track the latest gate state in a ref and skip the state writes when the
fetch resolves after eligibility was lost, so the rows stay empty under
role and tier transitions. Adds a regression test for the late-resolve
path.
* fix(security): sanitize control identity before logging replica transition
The control to replica transition log interpolated the control fingerprint
straight from the inbound sync payload, so a sibling pushing a forged
controlIdentity with embedded CR/LF could split or spoof log lines. Wrap the
value in the shared sanitizeForLog helper, matching every other tainted log
sink, so control characters are stripped before the entry is written.
* fix(fleet-actions): stop-by-label works on Community remote nodes
Fleet-stop's remote leg fanned out to POST /api/labels/:id/action, which
is gated to Skipper/Admiral, so on a Community fleet the control node
stopped its own stacks but every remote node returned 403. Fleet-stop
itself is admin-only and available on every license, so the remote leg
contradicted the feature's own gate.
Extract the label-match plus bulk-stop logic into a shared
runLocalLabelStop helper and add an admin-only, every-license
POST /api/fleet-actions/labels/local-stop receiver. The control now fans
out to that receiver, so remote stacks stop on every tier. Each node runs
under its own per-node bulk lock, so a fleet-stop and a per-label action
still serialize cleanly instead of double-stopping containers.
Also degrade the control's own leg per-node instead of failing the whole
fan-out when its filesystem read throws, and gate fleet-stop and
fleet-prune diagnostics behind developer_mode.
Tests: local-stop auth, tier, validation, and behavior; a remote-leg
routing guard that asserts the fan-out targets local-stop and never the
paid route; local-leg graceful degradation; and the three Fleet Action
card UIs.
* fix(fleet-actions): honor the remote stop receiver's matched flag
The control reached the remote leg only because its own mirror had the
label, then hardcoded matched:true and trusted results without guarding
its shape. A mirror-skewed control (mirror has the label, remote does
not) then showed a remote mismatch as "matched, 0 stacks" instead of
"no matching label", and a malformed 200 body could flow a non-array
into the per-stack renderers.
Honor the remote's own matched flag and coerce results to an array when
the body is malformed. Add regression tests for the matched:false skew
case and the non-array results case.
* feat(security): make CVE suppressions optionally honored by deploy-block policies
Block-on-deploy policies evaluate the raw scan result, so a CVE an admin
has accepted in CVE Suppressions still blocks the deploy. Add an opt-in,
per-instance toggle ("Honor suppressions in deploy blocks", Settings ->
Security) that, when on, re-derives each image's severity from the
suppression-filtered findings before comparing to the policy threshold. A
deploy that proceeds only because suppressions dropped it below the gate is
recorded in the audit log. Default off, so the strict raw-scan behavior is
unchanged unless an operator enables it.
The setting governs the instance that runs the deploy and is not
fleet-replicated. The gate fails safe: a suppression-read error or an
empty detail set falls back to raw scan severity rather than dropping it.
Also surface a previously swallowed error in the CVE suppressions and
misconfig acknowledgement settings panels so a failed list load shows a
toast instead of an empty list.
* fix(security): gate on raw severity when preflight detail rows are truncated
The suppression-aware deploy gate re-derived image severity from the stored
vulnerability_details rows, assuming any non-empty set was complete. A cached
pre-deploy scan keeps the full aggregate counts but copies only a bounded slice
of detail rows, so recomputing from that slice could drop an unsuppressed
blocking CVE below the threshold and let a deploy through.
Guard the recompute: when the loaded detail rows do not match the scan's total
finding count, gate on the raw scan severity (never drops severity). Suppression
awareness still applies for scans whose details are stored in full, which is the
common case.
* feat(fleet): show stack-label filtering in Fleet View on every tier
Stack labels and their assignments are a Community feature: the per-node
label reads are available to any authenticated user and are already used in
the per-node Stacks view. Fleet View, however, only fetched the fleet-wide
label palette and per-stack chips when the instance was on a paid tier, so a
Community user who had labelled their stacks saw no label dots on node cards
and no Tags filter in the Overview toolbar.
Drop the paid gate on the fleet label fetch so the palette, the per-stack
chips, and the Tags filter render for everyone who has labels. Node-level tag
aggregation (used for topology grouping) stays paid and is unchanged.
* fix(fleet): surface update-status failures and soften the reconnect timeout
Three reliability fixes in the Fleet View update path:
- The fleet update-status poll swallowed fetch errors in an empty catch, so a
failing poll left a silently stale table with no breadcrumb. It now logs the
failure (both thrown errors and non-ok HTTP responses) without toasting on
every tick, and keeps the last-known statuses.
- The local-update reconnecting overlay declared "Update timed out" after five
minutes, which falsely reported failure when a large image pull simply ran
longer than the reconnect window. It now shows a non-failure "Taking longer
than expected" state with a "Reload to check" action, and the timeout is a
named constant that mirrors the backend update timeout.
- The fleet overview fan-out already logged a node that failed to report; the
update-status and update-all fan-outs now log the rejected node and reason
too instead of discarding it.
* test(fleet): backfill Fleet View hook, component, and update-tracker coverage
Adds unit and component coverage for the previously untested Fleet View
surface: all six Overview hooks (overview, update-status, polling cadence,
preferences, fleet labels, node labels), the NodeCard, OverviewTab,
OverviewToolbar, NodeUpdatesSheet, UpdateStatusBadge, and ReconnectingOverlay
components, and the FleetUpdateTrackerService state transitions.
* test(fleet): assert update-status poll preserves last-known statuses on failure
Adds an explicit case that seeds statuses from a successful poll, then fails
the next poll, and verifies the table keeps the seeded statuses (and logs
without toasting) rather than relying on the implementation implicitly.
Registry credentials are stored and resolved per instance, and the UI
only ever calls registry endpoints with localOnly. The endpoints were
not on the hub-only proxy list, so a scripted request carrying an
x-node-id for a remote node would be forwarded by the remote proxy,
carrying a plaintext registry secret to (or reading stored credentials
from) the remote instance.
Add /api/registries/ to HUB_ONLY_PREFIXES so a remote-targeted registry
request is rejected with 403 before the proxy can forward it, matching
the existing defense-in-depth for the audit log, schedules, and the
global logs feed. Add regression tests for the collection, a sub-path,
and the local pass-through. Correct a stale 409 reference in the guard
comments and test header to the 403 the guard actually returns.
Promoting a user to admin now respects the per-tier admin seat limit the same
way user creation does, closing a path that let an operator exceed the cap by
creating an account and then editing its role to admin.
The last-admin guard for demote and delete now runs the admin-count re-check
and the write in a single transaction, so two concurrent admin changes can no
longer race the admin count to zero and lock everyone out.
Admin two-factor resets are now recorded once with their own audit summary
instead of being mislabeled as a user creation by the audit middleware.
The SSO provider settings previously swallowed three failure paths: the
provider-config load on mount, the connection test, and provider removal.
A transient backend error, a tier-gate rejection, or a failed removal left
the admin with no feedback at all.
Each path now surfaces the backend message (or a clear fallback) through the
standard error toast, matching the existing save handler. The connection-test
handler also checks the response status before treating the body as a test
result, so an API or authorization failure is no longer reported as a failed
identity-provider connection.
Adds component tests covering the non-ok and network-failure branches for each
handler, including the unparseable-error-body fallback.
* docs(registries): correct multi-node credential model
Private Registries are resolved per instance: ComposeService reads the
local instance's own registries at deploy time, and a remote deploy is
proxied to the remote Sencho instance, which resolves its own. The
control's credentials are never shipped to remote nodes.
The page previously described fleet-wide credential injection that does
not exist, which would leave an operator unable to explain a private
image pull failing with 401 on a remote node. Rewrite the Note, the
multi-node section, and the affected troubleshooting entries to describe
the real per-instance model: configure registries on each instance that
deploys private images, and manage a remote node's registries by signing
into that node directly.
* docs(registries): tighten wording on credential scope and tier mention
Address review feedback on the multi-node section: state that Sencho does
not automatically share credentials between instances rather than the
absolute "never copied" phrasing, since an admin can still direct a write
at a remote node through the distributed-API proxy. Drop the redundant
tier and role restatement from "Where to find it"; the requirement is
already stated in the page note.
sencho-quartermaster[bot]