The log viewer's per-line timestamp now renders HH:mm:ss again. The
HH:mm:ss.SSS variant was busier than the actual log content needed and
made the column harder to scan at a glance. The underlying ISO
timestamp from docker logs -t is still preserved on each row, so
download / copy still carries the original precision.
The rAF-based flush from the same area stays in place; that is what
makes lines feel real-time, not the timestamp width.
Dashboard and sidebar status indicators previously only refreshed on a
5-30 second polling cadence: a container restart, a degraded -> healthy
transition, or a stack update was invisible until the next tick.
Add a lightweight, non-persisted "state-invalidate" envelope on the
existing /ws/notifications WebSocket:
Backend
- NotificationService.broadcastEvent: sibling of dispatchAlert that
pushes an arbitrary {type, ...} envelope to every subscriber WITHOUT
writing to the alerts history (these are pure ephemeral signals).
- DockerEventService.handleEvent: emit the envelope for state-changing
container actions (start/die/kill/destroy/create/restart/pause/
unpause/health_status/rename/update). Carries node id, stack name
(from the compose project label), container id, action, and
timestamp.
Frontend
- EditorLayout's two notification WebSocket handlers (local plus
per-remote-node) branch on type. On state-invalidate they re-emit a
window CustomEvent and trigger a debounced (250ms) refreshStacks so
a burst of events from compose recreating multiple services
collapses to one refetch. The refresh callback is held in a ref so
the long-lived WS effect never closes over a stale function.
- useDashboardData listens for the same window event and refetches
/stats, /system/stats, and /stacks/statuses on every signal.
Historical metrics stay on their 60s polling cadence (10-minute
trend data, not a live indicator).
Tests
- Three new docker-event-service cases assert broadcastEvent fires on
start and health_status events with the correct envelope shape, and
does not fire on non-state actions like exec_create.
- Existing 28 cases updated with the broadcastEvent mock so the
subscriber stub matches the new shape.
Polling stays as a safety net at the same intervals; the WS path is
the fast path. Multi-node fleets benefit on the local node today;
extending the remote forwarder to relay state-invalidate is a
recommended follow-up.
Previously, fetching the .env file for a stack with no env files at all
returned a 404 with a JSON error body. The frontend's secondary loader
(changeEnvFile) called res.text() without checking res.ok, which caused
the error body to be stuffed directly into the editor as if it were
file content.
Two-part fix:
Backend (routes/stacks.ts):
- For the default GET /stacks/:name/env (no ?file= query) when the
stack has no env files, respond 200 with an empty body and an
X-Env-Exists: false header instead of 404.
- For an explicit ?file= query that resolves to a missing file, keep
the 404 (the caller asked for something specific).
- Catch a TOCTOU ENOENT between access() and readFile() and return the
same friendly empty-body shape, not a generic 500.
Frontend (EditorLayout.tsx::changeEnvFile):
- Check res.ok before reading the body. On a non-OK response, clear the
editor content and surface a friendly toast instead of pasting the
server's JSON error string into the file.
When a registry pushes a new build of an image at the same tag (digest
changes, tag does not), the preview service set next_tag to the same
string as current_tag and the readiness view rendered '10.11 -> 10.11',
which reads as a UI bug.
Add an update_kind field to UpdatePreviewSummary that distinguishes:
- 'tag' - at least one image has a strictly newer tag
- 'digest' - the only updates available are same-tag rebuilds
- 'none' - nothing to apply
The frontend now branches on update_kind and renders 'Rebuild available'
next to the current tag for the digest case, leaving the version-arrow
diff for genuine tag bumps.
Three new buildSummary cases lock in the kind classification.
The Docker janitor watchdog had three problems on multi-node fleets:
1. The alert text said "Your system has accumulated X GB" with no node
identifier, so on a fleet view the operator could not tell which
node was complaining. Resolve the local node via NodeRegistry and
put the node name in the message.
2. The threshold gate was a single comparison against the user's
configured GB value. A small or accidentally tiny threshold made
the alert fire on hosts with effectively no waste. Add a 100 MB
absolute floor so trivial cruft never triggers a notification.
3. The unit parser only matched uppercase "GB|MB|KB|B" and dropped
"TB" entirely. Modern Docker emits "kB" with a lowercase k, which
silently contributed zero bytes to the running total. Normalise the
unit to uppercase before the comparison and add the TB case.
Two compounding issues made logs feel laggy and timestamps look duplicated:
1. State updates were batched in a 250ms setInterval, so a burst of
lines all rendered together every quarter second. Replace with a
requestAnimationFrame scheduler: lines flush on the next paint
(~16ms at 60Hz) while still collapsing a single burst into one
React commit. Cleanup uses cancelAnimationFrame.
2. The timestamp formatter rendered HH:mm:ss only, dropping the
sub-second precision that docker logs -t already emits. Two lines
logged within the same second appeared identically. Render
HH:mm:ss.SSS so successive lines remain visually distinct.
The toolbar Update button calls updateStack(), which refreshed
containers and stacks but never re-fetched the image-updates list.
The sidebar's blue "update available" dot therefore stayed visible
until the 5-minute polling interval. The right-click context menu
path (executeStackActionByFile, action='update') already calls
fetchImageUpdates() on success; mirror that call here so both paths
behave the same.
Vulnerability scans cache their policy verdict as a JSON blob in
vulnerability_scans.policy_evaluation. Deleting a scan policy used to
remove only the policies row and leave those blobs intact, so the
scheduler kept emitting violations and stacks remained marked as blocked
against a policy that no longer existed.
deleteScanPolicy now nulls out policy_evaluation on every scan whose
JSON references the deleted policy id, then deletes the policy row, in
one transaction.
The Clipboard API requires a secure context, so navigator.clipboard is
undefined when Sencho is accessed over HTTP on a LAN IP. Most copy
buttons therefore failed silently and a few even fired success toasts
without writing anything to the clipboard.
Extract a shared copyToClipboard helper that prefers the modern API in
secure contexts and falls back to a hidden-textarea execCommand path
otherwise, then route every existing call site through it.
Updates the backend listen port, Vite dev proxy target, Docker EXPOSE,
compose port mapping, .env.example default, GitHub Actions smoke-test
default, healthcheck URLs, and every doc/example reference. Test fixtures
that include example URLs were updated for consistency, though their
assertions are port-agnostic.
The rate-limit value of 3000 in middleware/rateLimiters.ts and the
3000 entry in WEB_UI_PORTS (which detects user containers like Grafana)
are intentionally untouched.
Fresh installs land on the Community tier. The 14-day Admiral trial is now
issued by Lemon Squeezy via their hosted checkout: the user enters email +
card, receives a license key by email, and pastes it into the existing
Settings > License activation field.
Backend changes:
- LicenseService.initialize() no longer auto-creates a license_status='trial'
row on first boot. It now only ensures an instance_id exists and starts
periodic validation.
- Drop the TRIAL_DURATION_DAYS constant.
- Drop the status='trial' early-return in getVariant() so LS-issued trials
resolve through the normal variant metadata path (variant_name / product_name).
- Trial branches in getTier() and getLicenseInfo() are retained for future
work that may detect trial state from Lemon Squeezy metadata; they are
currently unreachable via the Sencho code paths.
Frontend changes:
- Settings > License surfaces a new "Try Admiral free for 14 days" CTA block
with Start monthly trial and Start annual trial buttons that open Lemon
Squeezy hosted checkout. The CTA is visible only when the user has no paid
access and is not already on a trial.
- Reserve the Admiral upgrade card for the Skipper-active upgrade path so
unlicensed users see one Admiral path (the trial CTA) instead of two.
- Pull the inline Lemon Squeezy checkout URLs into named module constants so
the Skipper, Admiral monthly, and Admiral annual endpoints are defined in
one place.
Test changes:
- license-service.test.ts covers the no-auto-trial startup path and updates
the trial-variant test to reflect the metadata-driven resolution.
- afterAll in the initialize() describe block calls destroy() so the 72-hour
validation interval does not leak into sibling test files.
Docs:
- Rewrite the Free trial section in features/licensing.mdx to document the
new LS checkout flow (email + card required, auto-converts on day 14 unless
cancelled).
- Add an operations/troubleshooting entry for cases where the trial license
key email does not arrive.
Custom OIDC stays on Community so self-hosters can wire any spec-compliant
OIDC identity provider (Authelia, Keycloak, Authentik, Zitadel, and others).
Google, GitHub, and Okta one-click presets move to Skipper. LDAP / Active
Directory and scoped RBAC are Admiral-only.
Backend enforces the split via a new requireTierForSsoProvider helper in
middleware/tierGates.ts, applied after requireAdmin in all four ssoConfig
mutation handlers. GET /sso/config (list) stays ungated so downgraded
admins can still see previously-configured providers. Invalid provider ids
now 400 before the tier check to avoid leaking tier information.
Frontend adds a compact mode to PaidGate and AdmiralGate for inline
list-item locks, and SSOSection reorders the provider cards as
Custom OIDC > Google > GitHub > Okta > LDAP to reinforce the
free-to-paid progression.
Stale 'SSO is Admiral' copy in AdmiralGate, PaidGate, and the Admiral
upgrade card on the License settings page has been replaced to reflect the
new split. User-facing licensing, SSO, overview, quickstart, and security
docs have been updated with the per-tier provider matrix.
The canonical middleware-order comment in app.ts carried historical
notes from the index.ts refactor ("before Phase 4 finishes", "moves to
routes/* in Phase 4", "moves here in Phase 5") that are no longer
active-voice descriptions of current state. Replace with plain
descriptions matching the final module layout. The 16-step enumeration
and the invariant paragraph about public routers (metaRouter, authRouter,
mfaRouter, ssoRouter) sitting before the auth gate are preserved.
No behavior change.
The index.ts refactor inverted the proxy mount order. The pre-refactor
monolith mounted `app.use('/api/', remoteNodeProxy)` before any inline
route, so remote-nodeId requests short-circuited into the proxy. After
the refactor the proxy was registered after every per-group router, so
Express matched local routers first and remote-nodeId requests were
silently handled with the control instance's local state (e.g.
GET /api/stacks with x-node-id=<remote> returned local stacks rather
than the remote's).
Fix moves createRemoteProxyMiddleware() between enforceApiTokenScope
and the first per-group router, matching middleware-order.md step 13
and restoring pre-refactor behavior. PROXY_EXEMPT_PREFIXES continues to
cover gateway-level paths (auth, nodes, license, fleet, webhooks, meta)
that must stay local even when x-node-id targets a remote.
Add four regression guards that would have caught this:
- json-parser-bypass.test.ts: asserts conditionalJsonParser leaves the
request stream intact on proxy-eligible paths so http-proxy can pipe
the raw body to the upstream; spins up a local echo server and
verifies the bytes arrive.
- proxy-mount-order.test.ts: asserts a remote-nodeId GET short-circuits
into the proxy (502 from unreachable upstream) instead of matching a
local router (200 from local state).
- upgrade-order.test.ts: pins WebSocket dispatch order by observing
handler-specific side effects for notifications, remote forwarder,
logs, and pilot tunnel.
- remote-console-session.test.ts: asserts the HTTP console-token route
mints a JWT with the same claim shape as the shared mintConsoleSession
helper, so gateway and WS forwarder tokens remain interchangeable.
Full suite: 73 files, 1,358 tests, all passing.
Reserve docs/internal/ for internal engineering documentation that should
not ship on docs.sencho.io. Mintlify's docs.json does not reference
internal/, so the folder is also invisible to the published site even if
it were tracked. The .gitignore entry prevents accidental commits.
The folder holds architecture deep-dives, module responsibilities,
shared-state inventory, WebSocket dispatch order, and refactor history
that are load-bearing for engineers but not appropriate for end-user
documentation.
Move the startup and shutdown lifecycles out of index.ts:
- bootstrap/startup.ts exports startServer(server) - migration check,
service initialization, background watchdogs, HTTP listen, pilot-agent
loopback bind.
- bootstrap/shutdown.ts exports installShutdownHandlers(server) -
SIGTERM/SIGINT handlers, in-order service stop chain, 10s force-exit
guard, SQLite close.
Restructure MfaService to add an instance + lifecycle so the replay
purge timer no longer lives as a module-scope setInterval in index.ts.
MfaService keeps all existing static methods (generateSecret, verifyTotp,
currentWindow, generateBackupCodes, hashBackupCodes, verifyBackupCode,
formatBackupCodeForDisplay, normalizeBackupCode, buildOtpauthUri) so
every existing caller stays unchanged. The new start() / stop() pair
is idempotent and calls .unref() so test shutdown is not blocked.
bootstrap/startup calls MfaService.getInstance().start().
bootstrap/shutdown calls MfaService.getInstance().stop().
index.ts drops from 305 to 147 lines and now contains only the Express
app composition: createApp, route mounts, remote proxy, createServer,
attachUpgrade, static/SPA fallback, errorHandler, installShutdownHandlers,
and the require.main guard that boots the server when run directly.
Behavior is byte-for-byte identical: shutdown service order, log
strings, force-exit timer, pilot-agent loopback logic, and the MFA
purge cadence and debug logging all preserved verbatim.
Move the nine /api/nodes/* endpoints out of index.ts into routes/nodes.ts
(list, scheduling-summary, get, create, pilot-enroll, update, delete,
test, meta). mintPilotEnrollment and the REMOTE_META_* constants move
with the router as local helpers.
Handlers moved verbatim. Two safe cleanups applied during the move:
- Inline req.apiTokenScope 403 blocks replaced with the shared
rejectApiTokenScope helper; payload shape unchanged.
- catch (error: any) rewritten to catch (error: unknown) with explicit
instanceof Error narrowing to satisfy the no-any strictness rule.
Response body shapes unchanged.
Removes now-unused imports from index.ts: jwt, crypto, authMiddleware,
isValidRemoteUrl, PilotTunnelManager, PilotCloseCode, CAPABILITIES,
getSenchoVersion, fetchRemoteMeta, RemoteMeta, FleetUpdateTrackerService,
plus the module-scope updateTracker alias.
index.ts drops from 1364 to 1021 lines. Only the stacks group remains
inline for the final phase 4c slice.
Move the six /api/containers/* and /api/ports/in-use endpoints out of
index.ts. Handlers moved verbatim. routes/containers.ts exports two
routers:
- containersRouter (mounted at /api/containers): list, stream logs,
start, stop, restart.
- portsRouter (mounted at /api/ports): /in-use host port inventory.
Removes the now-unused requireAdmin import from index.ts. Middleware
chains, response shapes, and error messages are all preserved.
index.ts drops from 1437 to 1364 lines. Remaining inline groups:
stacks and nodes.
Move the entire /api/security/* surface out of index.ts:
- Trivy lifecycle: status, install, uninstall, update-check, update, auto-update toggle
- Scanning: POST /scan (image), POST /scan/stack (compose)
- Scan queries: list, get, vulnerabilities, secrets, misconfigs, image-summaries, SARIF export
- SBOM generation
- Scan policies CRUD (fleet-replicated; replica writes rejected)
- CVE suppressions CRUD (fleet-replicated; replica writes rejected)
- GET /compare: diff two scans
Handlers moved verbatim. Local CVE_ID_RE, parseScannersInput, and
shapeScanForResponse helpers move with the router. The previously-closure
fetchAll inside the SARIF handler is hoisted to module scope as
fetchAllPages so it is not re-allocated per request.
Removes nine now-unused imports from index.ts (parsePolicyEvaluation,
VulnerabilityScan, FleetSyncService, requireAdmiral, trivyInstallLimiter,
SbomFormat, TrivyInstaller, validateImageRef, applySuppressions,
generateSarif).
index.ts drops from 2130 to 1437 lines. Remaining inline groups:
containers, stacks, and nodes.
Move three more Round C route groups out of index.ts:
- /api/registries/* -> routes/registries.ts
- /api/system/{orphans,prune,docker-df,resources,images,volumes,networks,...} -> routes/systemMaintenance.ts
- /api/templates/* -> routes/templates.ts
Handlers moved verbatim. Middleware chains, response shapes, and error
messages are preserved. Registry scope-denial now uses the shared
rejectApiTokenScope helper (same payload shape).
index.ts drops from 2739 to 2130 lines. Remaining inline groups:
containers, stacks, security (Trivy/scans/SBOM/policies), and nodes.
Move four small Round C route groups out of index.ts:
- /api/auto-heal/* -> routes/autoHeal.ts
- /api/notifications/*, /api/notification-routes/* -> routes/notifications.ts
- /api/system/console-token -> routes/console.ts
- /api/sso/config/* -> routes/ssoConfig.ts
Share NOTIFICATION_CHANNEL_TYPES, cleanStackPatterns, and validateHttpsUrl
via a new helpers/notificationChannels.ts so agents.ts and notifications.ts
consume the same allowlist and URL validator.
Handlers moved verbatim. Middleware chains and response shapes preserved.
index.ts drops from 3231 to 2739 lines.
Wraps up Phase 4 Round B by tackling the two deferred groups. 25 new
integration tests land first and run green against the inline monolith,
then each group is extracted byte-for-byte.
index.ts drops from ~3,678 to ~3,231 lines; test count rises 1,320 → 1,345.
New coverage:
- metrics-routes.test.ts (11) — auth + shape checks for /api/stats,
/api/metrics/historical, /api/system/stats, /api/system/cache-stats
(admin-only), and SSE headers for /api/logs/global/stream
- image-updates-routes.test.ts (14) — auth, admin gating, rate-limit
tolerance on /refresh, fleet aggregation, /auto-update/execute input
validation and no-stacks short-circuit
New route files:
- routes/metrics.ts — /stats, /metrics/historical, /logs/global (+ SSE
/stream), /system/stats, /system/cache-stats. Mounted at /api so the
mixed sub-paths line up.
- routes/imageUpdates.ts — /api/image-updates CRUD + fleet aggregation,
plus a separate autoUpdateRouter mounted at /api/auto-update that
owns the /execute handler. Same split pattern as license.ts +
systemUpdateRouter.
index.ts trims unused imports left behind by the extraction:
globalDockerNetwork, si, STATS_CACHE_TTL_MS, SYSTEM_STATS_CACHE_TTL_MS,
GlobalLogEntry + log-parsing helpers.
Round B of Phase 4. Writes integration tests for three under-covered
route groups BEFORE extracting them, then does the extraction once the
new tests pass against the monolith. index.ts drops from ~4,206 to
~3,678 lines.
New test coverage (42 new assertions):
- settings-routes.test.ts (14) — auth, admin gating, private-key stripping,
allowlist, single-key write, bulk PATCH validation + partial update
- scheduled-tasks-routes.test.ts (18) — list/create/get/toggle/delete/runs,
action+target_type matrix, cron validation, tier gating on non-admin
- agents-routes.test.ts (10) — GET/POST, admin gating, channel type +
HTTPS URL validation, boolean enabled check, upsert semantics
Each suite was verified against the inline monolith first, then the
route extraction was performed byte-for-byte and all suites re-run to
ensure no regression.
New route files:
- routes/settings.ts — GET/POST/PATCH with PRIVATE_SETTINGS_KEYS strip,
ALLOWED_SETTING_KEYS allowlist, and SettingsPatchSchema zod bulk schema
- routes/scheduledTasks.ts — 9 endpoints (list, create, get, update,
delete, toggle, run-now, runs history, runs CSV export). File-local
helpers parseTaskId, validateActionTarget, validateOptionalFields
collapse duplication across create+update handlers. Uses shared
escapeCsvField from utils/csv.ts.
- routes/agents.ts — notification-channel GET/POST. Owns
NOTIFICATION_CHANNEL_TYPES and validateHttpsUrl locally because the
notification-routes block still inlines identical copies; the helpers
will converge once those routes extract in a later slice.
Final slice of Phase 4 Round A. Pulls the four remaining well-tested route
groups out of index.ts. index.ts drops from ~5,930 to ~4,206 lines.
New route files:
- routes/webhooks.ts: /api/webhooks CRUD + HMAC-authenticated trigger.
Uses shared webhookTriggerLimiter. Trigger preserves the raw-body path
established by the conditional JSON parser for HMAC validation.
- routes/users.ts: /api/users CRUD + /:id/mfa/reset + /:id/roles
scoped-assignment surface. Uses rejectApiTokenScope across every
handler, validateUsername helper, BCRYPT_SALT_ROUNDS, and
isSqliteUniqueViolation for the role-assignment UNIQUE guard.
- routes/gitSources.ts: /api/git-sources + /api/stacks/:name/git-source/*.
Exports two routers (gitSourcesRouter + stackGitSourceRouter) because
the per-stack paths need to mount at /api/stacks alongside the label
routes extracted in phase 4a-1. String length limits are now named
constants so the 400 responses stay truthful if the bounds change.
- routes/fleet.ts: /api/fleet role, sync, overview, node drill-down,
update-status + trigger (single + fleet-wide), and snapshot CRUD +
restore. Local parseIdParam helper collapses seven copies of the
parseInt/isNaN route-param pattern.
Bugs fixed during review:
- users.ts :id/roles POST — replace the fragile
(err as Error).message?.includes('UNIQUE constraint') check with
isSqliteUniqueViolation from utils/errors.ts.
index.ts carries forward three symbols (updateTracker alias,
CVE_ID_RE, parseScannersInput) until the corresponding security /
nodes / scan routes get extracted in a later slice.
Second slice of Phase 4. Pulls the three auth-family route groups out of
index.ts into focused routers. All handlers move verbatim; index.ts drops
~845 lines.
New route files:
- routes/auth.ts: /api/auth core (status, setup, login, password, logout,
check, generate-node-token)
- routes/mfa.ts: /api/auth/login/mfa + full /api/auth/mfa/* surface
(status, enroll start/confirm, disable, backup-codes/regenerate,
sso-bypass)
- routes/sso.ts: /api/auth/sso/{providers,ldap,oidc/:provider/authorize,
oidc/:provider/callback} + getSSOBaseUrl helper. Module-load calls
SSOService.getInstance().seedFromEnv() so env-seeded providers are
available on the first request.
Shared lifts:
- helpers/constants.ts: MFA_REPLAY_TTL_MS + MFA_REPLAY_PURGE_INTERVAL_MS
(used by mfa.ts and the startup purge timer in index.ts) and
BCRYPT_SALT_ROUNDS (shared between setup and password-change handlers).
- middleware/auth.ts: new reissueSessionAfterTokenBump(req, res, userId)
helper collapses three copies of "bump → fetch user → re-sign cookie"
across auth.ts (password change) and mfa.ts (enrol confirm, disable).
Code review fixes:
- File-local requireEnrolledMfaUser helper in mfa.ts eliminates four
copies of "auth check + rejectApiTokenScope + load enrolled MFA" with
near-identical shape.
- Applied BCRYPT_SALT_ROUNDS to auth.ts setup + password handlers.
Mount order in index.ts: authRouter / mfaRouter / ssoRouter sit before
authGate because login / setup / SSO-callback are public; handlers that
need auth use authMiddleware directly on the route.
First slice of Phase 4 (route extraction). Pulls 8 well-tested, mostly
independent route groups out of index.ts into focused Router files. No
behavior change; every handler body moves verbatim.
New route files under backend/src/routes/:
- meta.ts /api/health, /api/meta (mounted before authGate)
- license.ts /api/license/* + /api/system/update,
exports scheduleLocalUpdate for the fleet route
- permissions.ts /api/permissions/me
- convert.ts POST /api/convert
- alerts.ts /api/alerts/*
- labels.ts /api/labels/* + PUT /api/stacks/:name/labels
(exported as stackLabelsRouter)
- apiTokens.ts /api/api-tokens/*
- auditLog.ts /api/audit-log/*
Shared helper lifts:
- helpers/cacheInvalidation.ts: invalidateNodeCaches()
- middleware/tierGates.ts: requireBody (was inline in index.ts)
- utils/errors.ts: isSqliteUniqueViolation (was inline in index.ts)
- middleware/apiTokenScope.ts: rejectApiTokenScope() helper (new)
- utils/csv.ts: escapeCsvField() (new)
index.ts drops from ~7520 to ~6775 lines and now mounts the routers right
after enforceApiTokenScope. The remote proxy and fleet/auth/webhooks/users
routes remain inline in index.ts pending later Phase 4 slices.
Code review fixes: rejectApiTokenScope helper replaces duplicated
`if (req.apiTokenScope) 403 SCOPE_DENIED` blocks in apiTokens.ts and
license.ts; escapeCsvField replaces the inline CSV escape in auditLog.ts.
Phase 3 of the index.ts refactor. Pulls the remote HTTP/WS proxy plumbing,
the WebSocket upgrade dispatcher, and the http/WSS construction out of the
monolith. index.ts drops roughly 620 lines.
New modules:
- proxy/websocketProxy.ts: shared httpProxy.createProxyServer singleton
(used by both the HTTP proxy middleware and the remote WS forwarder)
- proxy/remoteNodeProxy.ts: createRemoteProxyMiddleware() factory; consumes
the isProxyExemptPath helper instead of open-coding the prefix list
- server.ts: createServer(app) returns { server, wss, pilotTunnelWss }
- services/FleetUpdateTrackerService.ts: singleton wrapping the in-flight
fleet update tracker Map with create()/resolve() helpers
- helpers/consoleSession.ts: mintConsoleSession(), isConsoleSessionScope()
- websocket/upgradeHandler.ts: attachUpgrade(server, deps) dispatcher that
runs the manual cookie/JWT verify and delegates to sub-handlers
- websocket/pilotTunnel.ts: handlePilotTunnel (pilot_enroll consumption and
pilot_tunnel registration)
- websocket/notifications.ts: /ws/notifications local subscriber
- websocket/remoteForwarder.ts: remote-node WS proxy with console_session
token exchange for interactive paths
- websocket/logs.ts: /api/stacks/:name/logs supervisor stream
- websocket/hostConsole.ts: /api/system/host-console PTY, Admiral-gated
- websocket/generic.ts: /ws exec + streamStats action dispatch, owns the
terminalWs single-instance reference
- websocket/reject.ts: shared rejectUpgrade helper (replaces five copies)
Service extension:
- NotificationService: setBroadcaster(fn) replaced by subscribe(ws) that
returns an unsubscriber; broadcastToSubscribers is now internal. Subscriber
set lives on the service rather than in index.ts.
Wiring in index.ts:
- const app = createApp() already in place from Phase 2
- const { server, wss, pilotTunnelWss } = createServer(app)
- attachUpgrade(server, { wss, pilotTunnelWss })
- app.use('/api/', createRemoteProxyMiddleware())
- /api/system/console-token route now uses mintConsoleSession()
- deploy/down/update routes read the streaming target via getTerminalWs()
(return type is WebSocket | undefined so the || undefined fallback is gone)
Code review fixes: five duplicated reject helpers collapsed into
websocket/reject.ts; dropped the createTracker/resolveTracker bind
aliases in index.ts so call sites go through the service directly;
removed em dashes; replaced req.url! with req.url || '/'.
Phase 2 of the index.ts refactor. Pulls the auth middleware and session
cookie issuers into their own module, and introduces the app.ts factory
that owns the first nine steps of the canonical middleware pipeline.
New modules:
- middleware/auth.ts: authMiddleware, issueSessionCookie,
issueMfaPendingCookie, clearMfaPendingCookie
- app.ts: createApp() factory installing trust proxy, helmet, cors,
compression, cookieParser, rate limiters, conditionalJsonParser, and
nodeContextMiddleware. A header comment documents all 16 canonical
middleware steps and where each currently lives.
Changes:
- middleware/authGate.ts: createAuthGate factory removed; authGate now
imports authMiddleware directly (the factory existed only to avoid a
circular import while authMiddleware lived in index.ts).
- services/DatabaseService.ts: added API_TOKEN_SCOPE_TO_ROLE map so the
auth middleware no longer inlines a stringly-typed record.
- index.ts drops ~260 lines; auth routes, authGate, auditLog,
apiTokenScope, remaining routes, static serving, and the error handler
continue to be registered there until their respective phases.
- vitest.config.ts bumps testTimeout to 30s and hookTimeout to 45s so
fork-pool workers have enough headroom to ts-node-transform the
growing module graph under CPU contention (64 workers each import the
full Express stack in beforeAll).
Code review fixes: use getErrorMessage() util in the auth catch block
instead of inline cast; promote the scope-to-role map to a typed
module-level constant.
Phase 1 of the index.ts monolith refactor. Extracts all non-auth middleware
into focused modules. Routes and authMiddleware itself stay in index.ts for
now; Phase 2 introduces createApp() and extracts authMiddleware.
New modules:
- middleware/rateLimiters.ts: globalApiLimiter, pollingLimiter,
webhookTriggerLimiter, authRateLimiter, ssoRateLimiter, trivyInstallLimiter
plus the hybrid rateLimitKeyGenerator and isNodeProxyRequest helper
- middleware/jsonParser.ts: conditionalJsonParser that preserves the raw
stream for remote-proxy forwarding (via helpers/proxyExemptPaths)
- middleware/nodeContext.ts: nodeContextMiddleware
- middleware/apiTokenScope.ts: enforceApiTokenScope + DEPLOY_ALLOWED_PATTERNS
- middleware/authGate.ts: createAuthGate(authMiddleware) factory + auditLog.
Factory takes authMiddleware as a dependency to avoid a circular import
until Phase 2 extracts the auth module.
- middleware/errorHandler.ts: central error handler that preserves
err.status / err.expose from body-parser and other HTTP errors
- helpers/routePatterns.ts: WEBHOOK_TRIGGER_RE shared by rateLimiters and
authGate
index.ts drops ~290 lines. Middleware registration order is unchanged.
All 1278 tests pass.
Code review fixes: typed ApiTokenScope in apiTokenScope.ts; replaced 2
em dashes with colons (Directive 18); added local CachedProxyFlagReq
type alias for the node_proxy memoization cast; extracted deny() helper.
* refactor(backend): extract types, constants, and guards from index.ts (phase 0)
Additive, behavior-preserving first step of the modular backend refactor.
Moves purely static artifacts out of backend/src/index.ts so later phases can
extract routes and middleware without touching shared symbols.
New modules:
- types/express.ts: Express Request augmentation
- helpers/constants.ts: PORT, password policy, label colors, cookie names,
MFA TTLs, hot-path cache TTLs
- helpers/proxyExemptPaths.ts: PROXY_EXEMPT_PREFIXES + isProxyExemptPath
- helpers/cookies.ts: isSecureRequest, getCookieOptions
- helpers/policyGate.ts: buildPolicyGateOptions, runPolicyGate,
triggerPostDeployScan
- middleware/permissions.ts: ROLE_PERMISSIONS, checkPermission,
requirePermission
- middleware/tierGates.ts: requirePaid, requireAdmiral, requireAdmin,
requireNodeProxy, requireScheduledTaskTier + effectiveTier/Variant
index.ts shrinks by ~260 lines; no runtime behavior changes. All 64 vitest
files and 1,278 tests pass.
* refactor(backend): drop unused imports left after phase 0 extraction
LicenseTier, LicenseVariant, DIGEST_CACHE_TTL_MS, and isProxyExemptPath
were imported into index.ts but no longer referenced there after the
phase 0 move; CI lint flagged them as errors.
isProxyExemptPath will be re-imported in phase 1 when the JSON parser
bypass and nodeContext middleware get extracted. Silence the
no-namespace warning on the Express augmentation since the namespace
syntax is required for TypeScript module augmentation.
* feat(sidebar): implement keyboard shortcuts for stack menu actions
Shortcut labels shown in the context menu and kebab menu were purely
decorative. This wires them up to the corresponding actions on the
currently selected stack.
Cmd/Ctrl shortcuts: Enter (deploy), . (stop), R (restart), Up (update),
Backspace (delete). Single-key shortcuts: a (alerts), h (auto-heal,
paid), u (check updates), p (pin/unpin).
Guards: shortcuts are blocked when an input element is focused, when the
global command palette dialog is open, when no stack is selected, or
when the stack is busy. All visibility and busy flags from the menu
context are respected.
* docs(sidebar): document keyboard shortcuts for stack actions
- Remove redundant SENCHO prefix from kicker (already shown in card header)
- Animate the left accent bar with a slow back-and-forth shimmer sweep
- Replace static status dot with animated ping pulse
* feat(security): polish scan sheets, fix CVE links, surface policy violations
Adds cveUrl helper that rewrites Trivy's 404-ing avd.aquasec.com links to
cve.org for CVE-prefixed IDs (GHSA and misconfig URLs pass through unchanged).
Redesigns both scan sheets with shadow-card-bevel chips, tracked-mono kickers,
severity row tinting with a left accent rail, and tabular-nums timestamps.
Surfaces a destructive policy-violation banner on scans whose policy_evaluation
row flags a block, and fixes the compare sheet's delta ribbon so CRITICAL
net-positive deltas render in destructive (not warning) tone. Backend parses
the JSON policy_evaluation column at the API boundary so the UI receives a
structured object.
* chore(security): suppress CVE-2026-32281 and CVE-2026-32283 in Trivy scan
Both CVEs affect Go stdlib crypto/x509 and TLS in Docker CLI 29.4.0
(Go 1.26.1) and Compose v5.1.2 (Go 1.25.8). No upstream static binary
has been released with the patched Go 1.26.2 or 1.25.9 runtimes yet.
Exposure analysis: the Docker CLI and compose plugin connect to the local
Docker socket (Unix socket, no TLS) and to public registries with well-known
CAs. Neither CVE is exploitable in this configuration. Added alongside
sibling entries already in .trivyignore for the same binary versions.
Revisit on next Docker CLI and Compose upstream release.
Scan history is now a right-side sheet that layers over the current view
(typically Resources Hub) instead of a full-page activeView branch. The
sheet opens via the existing navigation event, fetches only when open,
dismisses on Escape or overlay click, and preserves the nested scan-details
and scan-compare sheets intact via Radix portal stacking.
The fetch effect now resets selection and page state on active-node change
exactly once, avoiding a double-fetch on node switches.
Policies with block_on_deploy=1 now scan every stack image before
docker compose up runs and reject the deploy with HTTP 409 on violation.
The UI opens a dialog listing offending images; admins can override per
deploy with ?ignorePolicy=true, and every bypass is recorded in the
audit log with the originating route, actor, policy, and image list.
When Trivy is not installed on the target node the gate fails open with
a warning notification, so teams are never locked out by tooling state.
Post-deploy and scheduled scans still evaluate matching policies and
dispatch warnings on violations to surface drift on long-running stacks.
Public API additions: policy and suppression CRUD under /api/security,
plus the documented 409 block-response shape on all deploy paths.
The `trivy config` subcommand does not accept `--no-progress`; the flag
exists only on `trivy image`. Every stack configuration scan therefore
failed with `FATAL Fatal error unknown flag: --no-progress`, and the
"Scan configuration" action on the stack details page has been
non-functional since it shipped.
Removing the flag is the complete fix. `trivy config` is silent by
default, so the flag was redundant even if it had been accepted.
A new Vitest spec pins the exact argument vector
(`['config', '--format', 'json', '--quiet', <stackPath>]`) so a future
edit cannot reintroduce the bug, and exercises the success path end to
end: status transitions to `completed`, misconfig severities tally
correctly, and `highest_severity` rolls up to the worst finding.
Two polish improvements to the aggregated notifications inbox:
- Per-node filter dropdown in the bell panel (hidden on single-node
installs) so fleet operators can triage events from a specific box.
Selected node falls back to "All nodes" automatically if that node is
removed from the registry.
- 60-second safety-net poll that reconciles the list so events missed
during a WebSocket reconnect backoff appear without a manual refresh.
Uses a ref indirection to pin the interval to the latest
fetchNotifications closure.
Flip Security (Trivy), Notifications (agents + history), and App Store from
global-and-hidden-on-remote to node-scoped so operators can manage them when a
remote node is selected in the node picker. The primary instance proxies the
calls to each remote, which resolves the correct per-instance binary state,
agent config, and template registry.
Backend: key `agents` and `notification_history` by `node_id` with idempotent
column-add migrations and a `(node_id, type)` unique index on agents, matching
the Labels pattern. Thread `req.nodeId` through the /api/agents and
/api/notifications routes. Internal NotificationService and ImageUpdateService
writes resolve the middleware default via `NodeRegistry.getDefaultNodeId()` so
monitor-emitted rows share a bucket with user-facing ones (avoids split-brain
where the UI sees test notifications but not internal alerts).
Frontend: split Security on remote to render only the scanner card and hide
scan policies and CVE suppressions (those remain control-plane-only). Drop the
misleading "Always Local" badge on Developer since retention windows govern
backend jobs, not UI state. Flip the App Store registry to node-scoped.
Docs: add a "What Settings apply per node" table to multi-node, clarify
remote alert setup in alerts-notifications, and note Trivy's per-host install
in vulnerability-scanning.
Replace the static SVG fleet topology with a ReactFlow canvas laid out
via dagre. Each node renders as a rack card with status pill, type
badge, CPU/MEM/DISK bars, and stack/running counts. Pan, zoom, drag,
and minimap are enabled; user-dragged positions persist across the
30-second poll so live metric updates no longer reset layout.
* feat(fleet): reorganize overview page for clarity and density
Scope Grid/Topology to the Overview tab (moved from above the tab bar so it
no longer implies it applies to Snapshots). Move Check Updates and Refresh
onto the tab bar row, right-aligned.
Compact the overview toolbar: constrain the sort combobox to a fixed width so
it no longer stretches full-page, and collapse the Status, Type, Severity,
and Tags pill groups into a single Filters popover with an active-count
badge and an inline Clear all filters action.
Render the local node card in the same responsive grid as remote nodes
instead of a dedicated full-width row. Visual distinction is preserved
through the existing brand gradient, cyan rail, ring, and "Local" badge.
* fix(fleet): stop local card from stretching when remote expands
Merging the local and remote node cards into a single responsive grid
meant CSS grid's default align-items: stretch made every cell in a row
match the tallest one. Expanding stack details on a remote card pulled
the local card up with it.
Add items-start on the merged grid so each cell sizes to its own
content.
* feat(search): add global command palette with cross-node stacks
Press Ctrl+K from anywhere to open a search palette that jumps to
pages, switches nodes, or opens stacks on any online node in the
fleet. Trigger lives as an icon in the top bar. Replaces the former
sidebar-scoped Ctrl+K handler.
The cross-node stack search fan-out is extracted into a shared hook
so the sidebar and palette stay in sync on the same debounce +
abort shape.
* fix(search): drop render-time ref write and effect-based query reset
Replace `nodesRef.current = nodes` during render with direct use of
`nodes` in the stack select callback, and fold the query-reset logic
into a single `onOpenChange` handler so it no longer runs via an
effect. Both changes resolve react-hooks rule violations that broke
frontend lint in CI.
Labels settings are now visible on remote nodes (scope: node, no longer
hidden on remote context). The LabelsSection already routes through the
active-node proxy, so edits land on whichever node the operator is viewing.
Fleet overview's label filter previously fetched only the control-plane
node's labels and assignments, so remote stacks could never match the
filter. Rewrote aggregation to fan out /labels and /labels/assignments
to every online node via fetchForNode + Promise.allSettled with a 5s
timeout per request. The palette dedupes by (name, color) so identical
labels on multiple nodes collapse into one entry while same-name +
different-color stay distinct. The assignment map is nested by nodeId
to avoid cross-node stack-name collisions.
Keyed the label refetch effect on a stable online-node id signature
rather than the nodes array reference, so the existing 30s overview
poll (and 5s fast-poll during updates) does not cascade into repeated
fleet-wide label fetches.
Apply design audit section 13's proposed chroma/lightness values to
frontend/src/index.css. Dark-theme accents now match the audit swatches
exactly: brand 0.78 0.11 195, success 0.78 0.16 155, warning 0.82 0.14
75, destructive 0.72 0.18 20. Surface hierarchy shifts to a warm pebble
cast (hue 80, chroma 0.003 to 0.01); card anchors to the audit's
0.14 0.005 80 surface swatch; popover bumps to 0.16 to preserve the 4
percent lightness gap against card. Light theme mirrors symmetrically.
NetworkTopologyView keeps the react-flow inline-style escape hatch but
hoists the brand literal into a BRAND_COLOR constant so MiniMap and
EDGE_COLORS share one source of truth.
Also normalize em-dash comment separators to colons across index.css
and drop a duplicated dark-background literal in favor of
var(--background).
The pulsing dot signaling an available image update used --info (blue
hue 250), the same hue as --label-blue, so the two blended visually
when a stack carried a blue label. Introduce a dedicated --update
token in vibrant fuchsia (hue 320) sitting outside the label palette,
and render both the sidebar StackRow and the fleet NodeManager
indicator as a notification-dot pattern (solid stationary core with
animate-ping halo). Aligns with the design rule that static state
should not pulse.
Document the hidden constraints that PR #706 left in the codebase so they
are not accidentally removed in a future cleanup pass:
- DropdownMenuSubContent and ContextMenuSubContent must stay Portal-wrapped
so sub-menus escape ancestors with overflow-x-hidden.
- refreshLabels must stay stable via useCallback because it is captured by
buildMenuCtx's memoization and passed as a prop.
Also delete LabelAssignPopover.tsx, which had zero consumers after the
create-and-assign flow moved into the menu-layer inline form.
Anso