* feat(cloud-backup): mirror fleet snapshots to S3-compatible storage
Add an Admiral-tier Cloud Backup feature that replicates every fleet
snapshot to off-site storage, with two provider modes that share the
same `@aws-sdk/client-s3` code path:
- Sencho Cloud Backup: zero-config, 500 MB allowance backed by
Cloudflare R2, provisioned via the sencho.io worker against the
user's Lemon Squeezy license.
- Custom S3 (BYOB): any S3-compatible bucket (AWS, MinIO, Backblaze
B2, Wasabi, R2 with own keys), with credentials encrypted via
`CryptoService` before storage.
API-triggered snapshots upload fire-and-forget so the UI returns
immediately; scheduled snapshots block on the upload so the task's
success/failure reflects cloud durability. Object keys include the
instance_id segment to prevent collisions when the same Admiral
license is activated on multiple Sencho instances.
* fix(cloud-backup): drop ES2022-only Error cause arg breaking ES2020 build
The backend tsconfig pins lib to ES2020. The two-argument
`Error(message, { cause })` form requires ES2022, so tsc rejected it
with TS2554. Revert to single-argument throw to match the
convention used elsewhere in the backend services.
* feat(files): backend foundation for stack file explorer
Install multer for multipart file upload handling. Add
isValidRelativeStackPath to validation.ts to guard client-supplied
relative paths against traversal, absolute paths, NUL bytes, backslash
injection, and double-slash segments. Add isBinaryBuffer to a new
binaryDetect.ts utility for heuristic text/binary detection via
NUL-byte fast exit and non-printable byte ratio sampling.
* fix(files): reject bare dot segments in isValidRelativeStackPath
* feat(files): add safe stack-scoped file I/O methods to FileSystemService
Adds FileEntry interface and seven new public methods to FileSystemService
for stack-scoped file operations: listStackDirectory, readStackFile,
streamStackFile, writeStackFile, deleteStackPath, mkdirStackPath, and
statStackEntry.
Each method routes through a private resolveSafeStackPath helper that
enforces two-phase path containment: a pre-realpath lexical check plus a
post-realpath symlink-escape check. ENOENT targets are handled by walking
up to the deepest existing ancestor, realpaths that ancestor, and
reattaching the remaining suffix.
Binary detection delegates to isBinaryBuffer; path safety delegates to
isPathWithinBase. Protected file names and the MIME map are module-level
constants to avoid repeated allocation.
* feat(files): frontend API wrappers and Monaco language helper
* fix(files): tighten stackFilesApi error handling and localOnly support
* fix(files): FileSystemService safety and correctness fixes
* feat(files): add file explorer API endpoints to stacks router
* feat(files): FileTree and FileTreeNode components
* fix(files): route security hardening and stream cleanup
* fix(files): FileTree accessibility, icon stroke, stale fetch guard
Add strokeWidth={1.5} to all Lucide icons in FileTreeNode to match the
design system. Add aria-expanded to directory rows for accessibility.
Guard handleDirClick .then() callbacks against stale stack name
references when the component re-renders with a new stack. Add
toast.info fallbacks when compose.yaml or .env is clicked without a
navigation callback registered.
* feat(files): FileViewer, FileUploadDropzone, NewFolderDialog, DeleteFileConfirm
* fix(files): resolve code quality findings in file explorer components
- Move editorOptions useMemo above conditional returns in FileViewer (Rules of Hooks fix)
- Fix blob download: append anchor to DOM before click, defer URL revoke 100ms
- Keep protected-file confirm input visible during NOT_EMPTY recursive retry in DeleteFileConfirm
- Remove non-functional cursor-pointer/onClick from Community upgrade pill in FileUploadDropzone
- Add success toast on folder creation in NewFolderDialog
- Switch all (e as Error).message casts to instanceof Error narrowing
* test(files): unit tests for binary detection, stack path safety, and file explorer routes
- binary-detection.test.ts: covers isBinaryBuffer edge cases (empty, NUL,
PNG header, threshold boundary, sampleBytes parameter)
- filesystem-stack-paths.test.ts: covers isValidRelativeStackPath (accepts/
rejects matrix) and FileSystemService stack methods against a real temp dir
(listStackDirectory sort and protection flags, readStackFile text/binary/
oversized paths, writeStackFile/Buffer, deleteStackPath, mkdirStackPath,
traversal guard); platform-specific empty-dir/NOT_EMPTY cases skip on Windows
- stack-files-routes.test.ts: route-level integration tests for all seven
file explorer endpoints; covers auth gating, Community-tier 403 gates,
input validation, 413 TOO_LARGE upload limit, and 204/200 happy paths
* feat(files): StackFileExplorer container with lazy tree, viewer, and action bar
* fix(files): add Download button to explorer toolbar, fix Community upgrade pill, reset state on stack change
* test(files): add missing test coverage for file explorer routes and service
* feat(files): add Files tab to EditorLayout with StackFileExplorer integration
* fix(files): add defensive activeTab guard to saveFile and discardChanges
* test(files): unit tests for FileTree expand/collapse and FileViewer render modes
Covers the three FileViewer content modes (text/Monaco, binary panel,
oversized panel) and the FileTree expand/collapse/cache cycle: first
expand fetches the subdirectory, second click collapses without a fetch,
third click re-expands from the in-memory cache without a second fetch.
* test(e2e): file explorer community and skipper+ flows
Covers the full file-explorer feature surface in two describe blocks:
Community (read-only): intercepts /api/license to simulate community
tier, confirms the upgrade pill is visible in the left pane, and
asserts that the Save button is absent after opening a text file.
Skipper+ (full CRUD): uploads a text file and confirms it appears in
the tree; edits config/app.conf and saves via Monaco; deletes an
uploaded file and asserts the tree entry is gone; issues a raw HTTP
request to the download endpoint and checks for status 200 and the
content-disposition: attachment header.
Also adds data-testid="file-action-delete" to the action bar Delete
button in StackFileExplorer for stable targeting, and exports
waitForStacksLoaded from e2e/helpers.ts to eliminate the three
identical local copies in stacks, deploy-log-panel, and stack-files
spec files.
* fix(e2e): improve test isolation and selector stability in stack-files spec
Move beforeEach seed to beforeAll/afterAll so fixtures are created once per
suite, not before every test. Extract shared seedSuite/teardownSuite helpers
to eliminate the duplicate beforeAll/afterAll blocks. Wrap teardown in
try/catch so failures log a warning rather than masking test results.
Replace waitForTimeout(500) with a deterministic expect on the file tree
sentinel. Add data-testid="anatomy-files-btn" and data-testid="delete-confirm-btn"
to replace the fragile button text/positional selectors. Assert Save button
starts disabled before editing.
* docs(files): add stack file explorer documentation
Add user-facing guide for the stack file explorer feature covering
tier access (Community read-only, Skipper+ read-write), viewing
limits, upload/download caps, protected file routing, and
troubleshooting. Update the editor page to reference the new guide
and register the page in the navigation.
* fix(docs): use canonical Skipper tier name in file explorer overview card
* fix(files): resolve lint errors blocking CI
Remove unnecessary backslash escape before double-quote in the
Content-Disposition regex (no-useless-escape). Replace five synchronous
setState resets at the top of the FileTree mount effect with a React key
prop on the FileTree element in StackFileExplorer so remounting resets
state automatically, eliminating the react-hooks/set-state-in-effect
violation.
* test(files): fix e2e seeding to work on community-tier CI
Replace the browser-side paid upload/mkdir API calls in seedTestStack with
direct Node fs writes. The upload and folder endpoints require Skipper+ so
they returned 403 on CI, which runs with no license set. Stack creation
via POST /api/stacks stays as an API call since it is community-allowed and
keeps the backend registry in sync.
Add a per-test tier check in the Skipper+ beforeEach that skips gracefully
when the instance is community, matching the pattern in auto-heal-policies.
* feat(auth): add TOTP two-factor authentication with backup codes
Adds RFC 6238 time-based one-time password support to every tier,
integrated with the existing password and SSO login paths.
Backend:
- New MfaService wrapping otplib with a plus or minus 1 step tolerance,
base32 secret generation, and hashed single-use backup codes (bcrypt).
- user_mfa and mfa_used_tokens tables in DatabaseService. The second
table is a DB-backed replay blacklist, purged on a 60s interval.
- authMiddleware now recognizes an mfa_pending scope. A token carrying
that scope is rejected on every route except the MFA challenge and
logout, so no API surface is reachable before the second factor
clears.
- /api/auth/login issues only a short-lived mfa_pending cookie when the
user has MFA enrolled. /api/auth/login/mfa consumes that cookie,
verifies the code (or backup code), and swaps in a real session.
- /api/auth/mfa/* routes for status, enrol/start, enrol/confirm,
disable, backup-code regenerate, and SSO-bypass opt-in.
- Admin recovery path: POST /api/users/:id/mfa/reset clears the target's
MFA state, bumps token_version, and writes an audit log entry.
- CLI emergency fallback: backend/src/cli/resetMfa.ts is wired via
`npm run reset-mfa <username>` and also exported for tests.
- SSO flows (LDAP and OIDC) gate on user_mfa.sso_enforce_mfa before
issuing a session; default behaviour keeps the SSO path frictionless.
- Per-user lockout after 5 consecutive failed codes (15 min).
Frontend:
- AppStatus gains an mfa-challenge branch driven by /api/auth/status.
- New MfaChallenge screen, MfaEnrollDialog (QR plus manual secret plus
backup codes), MfaDisableDialog, MfaBackupCodesDialog.
- Account section shows a Two-factor authentication card with enrol,
regenerate, disable, and the SSO-enforce toggle (shown only when SSO
providers are configured).
- Users section gains a Reset 2FA action for admins.
Docs:
- New user guide at features/two-factor-authentication.mdx.
- New admin guide at operations/two-factor-admin.mdx.
- SSO page cross-links to the 2FA doc.
* fix(mfa): drop unused TEST_PASSWORD import and stale eslint disable
* fix(mfa): simplify e2e openAccountSettings helper to match working pattern
* fix(mfa): make e2e suite self-contained and always clean up
Test #2 called loginAs() before the MFA challenge step, which waited for
the dashboard indicator that never appears once the previous test enrolled
the user. That timeout skipped the rest of the serial block, including
the disable step, leaving MFA enabled and breaking every later spec.
Two fixes:
- Tests #2 and #3 now navigate directly to the login page instead of
piggybacking on loginAs, which only handles the password-only path.
- A new afterAll hook unconditionally disables MFA via the API using two
unused backup codes, so the DB is reset even if a test fails midway.
* fix(e2e): use backup code for mfa recovery to avoid totp replay race
The final recovery step in the backup-code replay test previously
generated a fresh TOTP to sign back in. When the timing landed inside
the same 30-second window that test #2 consumed, the server's replay
blacklist correctly rejected it, producing a ~50% flake rate. Backup
codes are single-use and sidestep the replay window, so the recovery
becomes deterministic.
* fix(e2e): drive mfa disable test through the challenge screen
Test #4 called loginAs after test #3 left MFA enabled, but loginAs
waits for the dashboard indicator and does not handle the challenge
screen, so it timed out. Drive the login manually, satisfy the
challenge with a backup code, and use a backup code for the disable
step too to avoid any TOTP replay-window race against earlier tests
in the serial block.
* feat(git-sources): link stacks to Git repositories with diff-and-apply workflow
Add Git Sources so any stack can point at an HTTPS Git repository, branch, and
compose file path. Pulls fetch + validate the incoming commit, store a
diffable pending snapshot, and apply writes only after explicit confirmation
(or automatically, per the configured apply mode). Sibling .env sync is
optional. Works on the Community tier.
Apply modes:
- Review only: mark pending, wait for manual apply in the diff dialog
- Auto-write: write compose + env to disk, do not redeploy
- Auto-deploy: write files and run docker compose up -d
Webhook integration: webhooks can target the new "git-pull" action to trigger
a sync from CI. Per-source debounce prevents runaway pipelines from hammering
the repository host. Tokens are encrypted at rest and never returned to the
frontend.
Docs and tests included. Screenshots and Playwright E2E flows to follow.
* fix(git-sources): drop unnecessary useMemo on commit sha slice
React Compiler's lint rule rejected the manual dependency list because the
inferred dep ('pull') was less specific than the written one ('pull?.commitSha').
The computation is a cheap 7-char slice, so drop the useMemo entirely rather
than fight the rule.
* test(git-sources): add Playwright E2E flows and drop orphan source rows on stack delete
- E2E coverage: non-HTTPS URL rejected client-side, unreachable repo surfaces
a toast error on save, and configure+remove walks the AlertDialog confirm path.
- Deleting a stack now also drops its linked Git source row so a future stack
with the same name starts clean rather than inheriting a stale config.
Replaces five ad-hoc in-process caches (project name map, templates, latest
version, fleet update status, remote node meta) with a single internal
CacheService that provides TTL, inflight-promise deduplication to protect
against thundering herd, stale-on-error fallback, and per-namespace
hit/miss/stale/size counters for observability.
Wraps the hot-path dashboard endpoints in the cache with write-path
invalidation: /api/stats (2s), /api/system/stats (3s), and
/api/stacks/statuses (3s). Keys are namespaced by nodeId so switching nodes
never serves another node's data. Every route that mutates container or
stack state calls invalidateNodeCaches(nodeId), which also drops the global
project-name-map, so user actions stay instantly reflected in the UI.
For /api/system/stats the cheap per-request network rx/tx block is kept
outside the cache so live-updating charts stay smooth while the expensive
systeminformation.currentLoad() CPU sample (~200ms) is reused across the
TTL.
Adds admin-only GET /api/system/cache-stats returning per-namespace
counters for operators who want to observe cache effectiveness.
Enables the compression middleware site-wide for JSON responses. Large
payloads like /api/templates shrink roughly 5x on the wire. SSE endpoints
are explicitly excluded via a Content-Type filter so live log tails and
metric streams are not buffered.
Bumps vitest hookTimeout to match testTimeout (15s) so parallel fork
workers do not hit the default 10s hook limit under CPU contention.
Adds 35 new tests (26 unit for CacheService, 9 integration for cached
endpoints) covering TTL expiry, inflight dedup, stale-on-error,
namespace invalidation, entry-cap safety guard, and write-path
invalidation end-to-end through Express routes.
* fix(api): add tiered rate limiting to prevent polling lockouts
Replace the single global rate limiter (100 req/min/IP) with a tiered
system that separates high-frequency polling endpoints from standard
API traffic:
- Polling tier (300/min): /stats, /system/stats, /stacks/statuses,
/metrics/historical, /health, /meta, /auth/status, /auth/sso/providers,
/license. Exempt from the global limiter but governed by their own
safety net to prevent resource exhaustion.
- Standard tier (200/min): All other endpoints, raised from 100.
- Webhook tier (500/min): POST /webhooks/:id/trigger, dedicated limiter
for CI/CD platforms sharing datacenter IPs.
- Auth tier: Unchanged (5-10 attempts / 15 min).
Enterprise adaptations:
- Authenticated requests keyed by user session (JWT sub/username) instead
of IP, preventing shared NAT/VPN environments from pooling budgets.
- Internal node-to-node traffic (node_proxy tokens) bypasses all rate
limiters entirely.
Includes comprehensive stress tests (21 cases) validating tier
separation, node proxy bypass, and per-user keying.
* chore(deps): bump axios to 1.15.0 to fix SSRF vulnerability
Addresses GHSA-3p68-rc4w-qgx5 (NO_PROXY hostname normalization bypass).
* fix(fleet): resolve getSenchoVersion crash in Docker containers
Replace the runtime __dirname walk (which fails in Docker because the
root package.json is absent from the final image) with a build-time
version injection. A prebuild script reads the root package.json and
generates src/generated/version.ts with the version baked in as a
constant. The compiled output carries the correct version regardless
of the container's filesystem layout.
Retains a filesystem walk fallback for dev environments where the
prebuild hook may not have run.
* fix(fleet): skip postinstall version generation when script is absent
The Docker prod-deps stage copies only package.json (no source files)
before running npm ci --omit=dev. The postinstall hook now checks for
the script file before executing, so it no-ops in stages where
scripts/generate-version.js has not been copied.
Add the ability to check for outdated nodes and trigger over-the-air
updates from Fleet View. Nodes self-update by pulling the latest Docker
image and recreating their container via the "last breath" pattern.
Backend:
- SelfUpdateService: self-container identification via HOSTNAME + Docker
Compose labels, triggers pull + force-recreate
- CapabilityRegistry: runtime capability disabling via disableCapability()
- POST /api/system/update (202 + deferred self-update)
- GET /api/fleet/update-status (version comparison across fleet)
- POST /api/fleet/nodes/:nodeId/update (single node)
- POST /api/fleet/update-all (bulk remote update)
- In-memory update tracker with 5-min timeout
Frontend:
- Node Updates modal with summary stats, search filter, table layout,
per-node Update buttons, and bulk Update All
- Version badges and update-available indicators on node cards
- ReconnectingOverlay for local node updates (polls /api/health)
- 5s fast-poll when any node is actively updating
- UpdateStatusBadge shared component for consistent badge rendering
Requires Skipper (Pro) tier. Nodes must be deployed via Docker Compose
with Docker socket access.
Add centralized credential storage for private Docker registries with
support for Docker Hub, GHCR, AWS ECR, and self-hosted registries.
- New `registries` table with AES-256-GCM encrypted secrets
- RegistryService with CRUD, test connectivity, Docker config generation
- 5 API endpoints gated by requireTeamPro + requireAdmin
- ComposeService injects credentials via temp DOCKER_CONFIG on deploy/pull
- ImageUpdateService passes stored credentials for private registry checks
- AWS ECR just-in-time token refresh via @aws-sdk/client-ecr
- RegistriesSection UI in Settings Hub with type-aware form
- Documentation with screenshots
Adds the ability to schedule recurring Docker operations (stack restarts,
fleet snapshots, system prunes) via cron expressions with full execution
history logging. Includes Run Now for on-demand execution.
* feat: SSO & LDAP authentication for Team Pro
Add SSO integration allowing Team Pro users to authenticate via LDAP/Active Directory, Google, GitHub, and Okta identity providers. SSO works alongside password authentication with auto-provisioning and role mapping.
- LDAP bind+search authentication with group-based role mapping
- OIDC/OAuth2 flows with PKCE and CSRF protection for Google, GitHub, Okta
- Auto-provisioning: first SSO login creates a Sencho account automatically
- Role mapping via LDAP group membership or OIDC JWT claims
- SSO settings UI in Settings → SSO with per-provider config and test connection
- SSO login buttons on login page with LDAP toggle
- Environment variable seeding for infrastructure-as-code workflows
- Secrets encrypted at rest via CryptoService (AES-256-GCM)
- Seat limit enforcement during auto-provisioning
- Full documentation: feature docs, quickstart guides, env var reference
* fix: resolve ESLint errors in SSO feature
- Remove unnecessary escape characters in regex character classes
- Remove unused `issuer` variable from OIDC callback handler
- Fix setState-in-effect lint error in Login.tsx by using useState initializer
- Suppress set-state-in-effect for SSOSection fetch pattern (matches existing codebase convention)
* chore(deps): upgrade ESLint 9 → 10 with plugin compatibility fixes
- eslint: ^9 → ^10.0.0 (backend + frontend)
- @eslint/js: ^9 → ^10.0.0 (backend + frontend)
- eslint-plugin-react-refresh: ^0.4.24 → ^0.5.2 (ESM, config factory API)
- Update frontend eslint.config.js: destructured import for react-refresh,
call configs.vite() as factory function (0.5 API change)
- Downgrade new ESLint 10 rules (no-useless-assignment, preserve-caught-error)
to warnings for existing code patterns
- eslint-plugin-react-hooks stays at 7.0.1 (stable) with --legacy-peer-deps
until a stable release adds ESLint 10 peer support
* chore(deps): upgrade recharts 2.x to 3.8 with chart.tsx type fixes
recharts 3.x moved Tooltip/Legend props to context-based API.
Updated chart.tsx to use explicit prop interfaces with internal
recharts type imports (LegendPayload, TooltipPayload, TooltipFormatter).
* chore(deps): upgrade TypeScript 5.9 to 6.0
- Remove deprecated baseUrl from frontend tsconfig (paths works standalone in TS 6)
- Add react-is dependency required by recharts 3.x at build time
- Backend and frontend both compile and lint cleanly
* chore(deps): upgrade Vite 7.3 to 8.0 and @vitejs/plugin-react to 6.0
Vite 8 replaces Rollup+esbuild with Rolldown, significantly improving
build speed (~2s vs ~18s). No config changes required.
* fix(ci): add .npmrc with legacy-peer-deps for CI and Docker builds
typescript-eslint@8.x requires typescript <6.0.0 and
eslint-plugin-react-hooks@7.0.1 requires eslint <=9. Until upstream
packages release compatible versions, legacy-peer-deps is needed.
* docs: add logo assets and re-ignore CLAUDE.md
* fix(ci): copy .npmrc into prod-deps Docker stage
The prod-deps stage also runs npm ci with backend/package.json but was
missing the .npmrc needed to bypass peer dep conflicts.
- Split serial build-and-test job into parallel backend, frontend, docker-validate, and e2e jobs
- backend job: build → test → lint (new ESLint) → npm audit --audit-level=high
- frontend job: build → lint → npm audit --audit-level=high
- docker-validate job: builds image on every PR without pushing; Trivy scans for CRITICAL/HIGH CVEs (informational)
- e2e job: runs full Playwright suite against live dev servers after backend+frontend pass
- Add backend/eslint.config.mjs and lint script to backend/package.json
SECURITY (critical fixes):
- Add authMiddleware to /api/system/console-token (was publicly accessible)
- Validate api_url on node create/update to prevent SSRF (rejects localhost/loopback)
- Add rate limiting (5 req/15 min/IP) to /api/auth/login and /api/auth/setup
- Fix path traversal in env_file resolution — absolute/escaping paths rejected
- Add stack name validation to GET routes (was only on PUT/POST)
- Add helmet security headers middleware
- Restrict CORS to FRONTEND_URL in production
PRODUCTION READINESS:
- Add GET /api/health public endpoint + HEALTHCHECK in Dockerfile
- Add SIGTERM/SIGINT graceful shutdown handler (drains connections, closes DB)
- Run container as non-root sencho user in Dockerfile
QUALITY:
- Fix 4 silent empty catch{} blocks in EditorLayout (now show toast.error)
- Connect ErrorBoundary to root App in main.tsx
- Replace WebSocket.Server with named WebSocketServer import (ESM compat)
TESTING (new automated test suite):
- Install Vitest; 38 backend tests across 4 suites covering validation utilities,
health endpoint, auth middleware, login flows, SSRF protection, and path traversal
- Extract isValidStackName/isValidRemoteUrl/isPathWithinBase to utils/validation.ts
- Playwright E2E scaffolding: auth, stacks, nodes specs + shared login helper
- CI: run Vitest + ESLint on every PR
Security:
- Strip auth credential keys (auth_username, auth_password_hash,
auth_jwt_secret) from GET /api/settings response
- Add allowlist guard to POST /api/settings — rejects unknown or
auth-namespace keys with a 400
Backend:
- Add PATCH /api/settings bulk endpoint with Zod schema validation
(type coercion, range checks, URL format) and atomic SQLite transaction
- Add system_state table — moves last_janitor_alert_timestamp out of
global_settings; adds getSystemState/setSystemState on DatabaseService
- Add metrics_retention_hours and log_retention_days configurable settings;
MonitorService reads both dynamically each evaluation cycle
- Add cleanupOldNotifications(days) to DatabaseService, called each cycle
Frontend:
- Replace single isLoading flag with per-operation states
(isSavingSystem, isSavingDeveloper, isSavingPassword, isSavingRegistry,
isSavingAgent/isTestingAgent per agent type)
- Add skeleton loader that blocks interaction until fetchSettings resolves
- Explicit key-picking in fetchSettings — auth keys cannot enter state
- Unsaved-changes amber dot on System Limits and Developer sidebar items
- Separate saveSystemSettings / saveDeveloperSettings — no cross-tab clobber
- Developer tab gains Data Retention section (metrics hours, log days)
- All settings saves use new PATCH /api/settings endpoint
Anso