Commit Graph

42 Commits

Author SHA1 Message Date
Anso 2d56ea958a fix(stack-activity): per-stack history integrity, attribution, sanitization (#1228)
* fix(stack-activity): per-stack history integrity, attribution, sanitization

Address the Stack Activity audit findings (PR 1 of 2):

- Per-stack history integrity: drop the per-insert 100-row prune in
  addNotificationHistory that evicted quieter stacks' history whenever
  another stack got chatty. Periodic cleanupOldNotifications now caps
  per (node, stack) at 500 rows and per-node unattached system events
  at 1000 rows, on top of the existing 30-day retention. Signature
  takes an options bag and returns a per-stage summary so MonitorService
  can log what actually ran each cycle.

- Actor attribution: thread req.user?.username through every
  notifyActionFailure call site and add synthetic actors at service
  emit sites (system:autoheal, system:scheduler, system:image-update,
  system:docker-events, system:blueprint, system:monitor, system:policy).
  The timeline renders system actors as "via <Label>" so an autoheal
  redeploy is no longer indistinguishable from a user redeploy.

- Message sanitization: new sanitizeNotificationMessage at
  NotificationService.dispatchAlert strips KEY=VALUE pairs whose key
  ends in TOKEN/KEY/PASSWORD/SECRET/CREDENTIALS/AUTH, scrubs HTTP basic
  auth in URLs and Bearer tokens, collapses COMPOSE_DIR paths, and
  truncates to 1000 chars. Applied to the stored history and to every
  downstream Discord/Slack/webhook channel. The ImageUpdateService
  recovery-path direct DB write also runs through the sanitizer.

- Composite pagination cursor: getStackActivity now accepts a
  (timestamp, id) cursor (?before=&beforeId=). The legacy timestamp-only
  form silently dropped events when a single compose up emitted many
  events sharing one millisecond. Route rejects beforeId without before.

- Frontend hardening: distinct error state with retry button (initial
  fetch failure no longer renders as the genuine empty state), strict
  positive-integer parsing on cursor params, overrequest-by-1 pagination
  so the last page does not leave a dead "Load more" click, runtime
  guard on liveEvents merge that validates the level union, per-minute
  day-bucket recompute so an open panel does not stay on "Today" past
  midnight.

No tier, role, or capability gate touched. Route permission gate
remains stack:read on the named stack.

* fix(stack-activity): sanitizer covers lowercase env vars and per-node compose dir

External review surfaced two leak paths in the message sanitizer:

- The sensitive-key regex was uppercase-only. Compose env names are
  conventionally uppercase but lowercase forms (db_password, jwt_secret,
  github_token) are valid and do leak through the same Docker and
  compose-parse error paths. Make the regex case-insensitive and tighten
  it to also catch bare TOKEN= / KEY= / PASSWORD= without a prefix word,
  while still leaving BYPASS, COMPASS, and similar non-secret keys alone.

- The compose-dir path collapse only read process.env.COMPOSE_DIR, but
  the real resolution chain is node.compose_dir (per-node DB override)
  -> process.env.COMPOSE_DIR -> /app/compose. A node with a custom
  compose_dir could still leak absolute paths into stored history and
  downstream channels. Route both the dispatchAlert call and the
  ImageUpdateService recovery-path direct write through
  NodeRegistry.getInstance().getComposeDir(localNodeId) so the
  collapse covers every resolution outcome.

Tests now assert lowercase keys are redacted and that BYPASS-style
non-secrets stay intact in both cases. notification-routing mock
extended to stub the new getComposeDir call.

* chore(stack-activity): a11y roles, visibility-aware tick, live-disconnect signal

Close three small follow-ups on the per-stack activity timeline:

- A11y: each day-group gets role="list" and each event row gets
  role="listitem" so screen readers traverse the timeline as a list
  instead of a wall of text. The day-group container also carries an
  aria-label naming the bucket.

- Visibility-aware day-bucket tick: the 60s setInterval that re-derives
  Today/Yesterday/Earlier now short-circuits when document.hidden, so a
  backgrounded panel does not re-render every minute for no visible
  effect.

- Live-disconnect signal: useNotifications dispatches a
  sencho:notifications-connection custom event on WebSocket open and
  close. The timeline listens and, when explicitly disconnected, shows
  a one-line "Live updates offline; reconnecting…" hint above the list.
  The sidebar ticker already surfaces fleet-wide connection state; this
  adds an in-context cue for users who are focused on a single stack.

Stack-name case normalization was considered and rejected: stack names
are case-permissive per the isValidStackName validator, and lowercasing
on read or write would silently rename or hide a user's "MyApp" stack.

* ci(stack-activity): drop unnecessary escape in URL_BASIC_AUTH regex

ESLint no-useless-escape errored on \- inside the character class
[a-zA-Z0-9+.\-] at notificationMessage.ts:14. Move the dash to the
end of the class so it's an unambiguous literal and the escape is no
longer required. Behavior is identical; sanitizer tests still pass.

* revert(stack-activity): drop unvalidated E2E spec from this PR

The spec was committed without ever running against a real Docker
daemon, then failed in CI when it ran for the first time: deploy
returned 200 but no notification appeared on the activity endpoint
within the polling window, suggesting either a deploy-notification
race or a node-id resolution mismatch in the CI environment.

Backend unit tests (route + composite cursor + sanitizer) and
frontend component tests cover the same logic. The E2E spec will
land in a dedicated follow-up once it has been authored against a
working CI environment.
2026-05-25 21:09:00 -04:00
Anso f03c9dc7b6 fix(webhooks): address Codex review of PR #1177 (#1181)
* fix(webhooks): close HMAC timing oracle on trigger reject paths

The trigger handler in PR #1177 returned a uniform 404 for every
unauthenticated rejection, but only the wrong-signature path computed
an HMAC over the request body. The other reject paths (unknown id,
disabled webhook, non-paid tier, missing X-Webhook-Signature header,
missing rawBody) short-circuited before any HMAC work. Repeated
near-rate-limit probes with a large attacker-controlled body could
distinguish a valid-and-enabled paid webhook id from the other reject
cases through response latency.

WebhookService.validateSignature is now constant-time over every input
shape: it always runs crypto.createHmac and crypto.timingSafeEqual
against fixed-length 32-byte buffers regardless of whether the
signature is missing, has the wrong prefix, is malformed hex, or is
the wrong length. The trigger handler calls it unconditionally before
any reject branch fires, using a stable per-process decoy secret
(WebhookService.getDecoySecret) when the webhook does not exist and an
empty buffer when the request has no body. Response timing now depends
only on the size of the request body, which the attacker already
controls and which reveals nothing webhook-specific.

Six new tests pin the behaviour: validateSignature is observed firing
on the unknown-id and missing-signature paths through a spy assertion,
and four direct-call tests confirm validateSignature returns false
without throwing for empty, wrong-prefix, malformed-hex, and
wrong-length signatures.

* fix(safe-log): redact Basic auth and lowercase Windows drive letters

The redactSensitiveText helper now covers two cases the prior chain
missed:

* Authorization: Basic <base64> previously left the base64 payload
  intact. The existing key/value regex caught only the literal word
  Basic before stopping at the space. A new Basic\s+[A-Za-z0-9+/=]+
  replacement runs before the key/value regex so the credential is
  scrubbed first.
* Windows homedir paths like c:\Users\<user>\... with a lowercase
  drive letter previously slipped through because the regex required
  [A-Z]. Changed to [A-Za-z] so both letter cases are covered.

Two new tests pin both fixes.

* docs(webhooks): document 429, fix shared schema, comply with D27/D31

* Trigger endpoint declares the 429 response that webhookTriggerLimiter
  can return (500 requests per minute per source IP); both
  docs/openapi.yaml and the response table in docs/features/webhooks.mdx
  carry the new row, and a new troubleshooting accordion explains the
  shared-NAT scenario.
* Shared Webhook schema in docs/openapi.yaml extends the action enum to
  include git-pull and documents the node_id property. The GET list
  endpoint returns these fields; the prior schema would have failed
  validation for any git-pull row.
* docs/features/webhooks.mdx:7 rewritten from a customer-side role
  enumeration ("non-admins on a paid tier can view the list but cannot
  manage it") to a single requirement statement ("Webhooks require a
  Skipper or Admiral license. Managing webhooks is admin-only.") per
  CLAUDE.md D27/D31; the prior phrasing was customer-side fence-spec.
* Two em dashes in webhook description strings I had touched in the
  prior OpenAPI sync commit replaced with semicolons per D18.
2026-05-23 16:03:48 -04:00
Anso 21ec5e7e0a fix(webhooks): harden trigger response surface (#1177)
* fix(webhooks): harden trigger response surface

Bundles six audit findings on the incoming-webhooks trigger path. All
changes preserve the documented happy path: a CI caller signing the exact
request body with the webhook secret still receives 202 Accepted.

* Uniform 404 on every unauthenticated rejection (missing webhook,
  disabled webhook, non-paid tier, missing signature header, missing
  raw body, signature mismatch). The four-way response surface previously
  let an unauthenticated probe enumerate webhook ids and fingerprint the
  instance's licence tier; callers now see one shape for any failed auth.
* Fail closed when express.json()'s verify callback did not capture the
  raw request body. Previously the handler fell back to
  JSON.stringify(req.body), which compares the HMAC against a
  re-serialised payload that is not byte-equal to what the client signed.
* Pass the already-loaded webhook through to WebhookService.execute()
  instead of re-fetching by id. Closes the delete-during-execution race
  where an admin deletion between the trigger handler's load and the async
  dispatch silently dropped the execution row. The webhook_executions
  table has ON DELETE CASCADE, so recordExecution now wraps the insert in
  try/catch and logs a warning when the FK constraint trips because the
  parent webhook was deleted mid-flight.
* Redact bearer tokens, JWTs, URL credentials, and homedir paths from
  error strings before persisting to webhook_executions.error. The
  execution history is readable by any paid user via GET /webhooks/:id/
  history; redactSensitiveText gains three home-directory patterns
  (/home/<user>, /Users/<user>, <drive>:\Users\<user>) and now runs on
  every error stored from this path.
* Cap webhook name at 100 characters on both POST and PUT, rejecting
  non-string and oversized values with 400 before they reach the DB.
* Validate the body's action override against a typed allowlist
  (isWebhookAction type guard) on the trigger endpoint, returning 400
  before queueing execution. An unknown override no longer reaches
  recordExecution as a stored failure row.

Tests updated to pass db.getWebhook(id)! instead of the raw id to the new
execute() signature. Docs at docs/features/webhooks.mdx updated to reflect
the new uniform 404 response, the new 400-on-invalid-action behaviour, and
a rewritten troubleshooting accordion that walks operators through every
cause of the uniform 404.

* test(webhooks): cover trigger handler auth, race, and redaction paths

Adds 21 vitest cases for the public webhook trigger handler and the
WebhookService.execute / recordExecution pipeline, plus 3 cases for the
new homedir patterns in redactSensitiveText.

webhooks-trigger.test.ts covers, per audit finding:

* M1 + H3 uniform 404: id unknown, webhook disabled, non-paid tier,
  missing signature header, missing rawBody, sha1= prefix, malformed
  hex signature, sig mismatch. Each asserts identical 404 body so a
  future regression that re-introduces 401 / 403 / PAID_REQUIRED breaks
  one of the 8 tests.
* Happy path: 202 with configured action, valid action override,
  unknown action override returns 400 after auth succeeds (L2),
  non-string action override returns 400.
* L1 name cap: POST and PUT both reject names over 100 chars and
  non-string names; 100-char boundary still accepted; PUT allows
  partial updates that omit name.
* M5 race: deleting the parent webhook before recordExecution runs no
  longer crashes the async dispatch; the FK cascade is swallowed with
  a console.warn, and a happy-path test pins the recordExecution row.
* M6 redaction: stubs ComposeService.runCommand to throw errors
  containing a bearer token and a homedir path, then asserts the
  persisted webhook_executions.error has both scrubbed.

safe-log.test.ts gains three unit tests pinning the new homedir
patterns in redactSensitiveText (Linux, macOS, Windows). The existing
credentials test is untouched.

Tests use prototype spies on FileSystemService and ComposeService (both
hand out a fresh instance per nodeId), so per-test mocks do not leak.
beforeEach restores all mocks and reapplies the LicenseService 'paid'
spy. Closes audit finding H2 (zero trigger-path test coverage).

* docs(webhooks): sync openapi spec with new trigger response surface

Brings docs/openapi.yaml in line with the behaviour changes from the
trigger hardening commit. Mintlify auto-generates the per-endpoint
reference pages from this spec, so the spec drift would surface as
wrong response codes in the public API reference.

POST /api/webhooks and PUT /api/webhooks/🆔
  * name: maxLength 100 (matches MAX_WEBHOOK_NAME_LENGTH on the route).
  * action enum: add git-pull (pre-existing omission; the route has
    always accepted it).
  * node_id: documented as an integer property (pre-existing omission).

POST /api/webhooks/:id/trigger:
  * requestBody required: true (body is now mandatory; the H3
    fail-closed branch rejects a missing rawBody).
  * action override: enum restricted to the allowlist.
  * 401 and 403 responses removed.
  * 404 response: description rewritten to reflect uniform-404
    behaviour; the body is { error: "Webhook not found or signature
    invalid" } for every unauthenticated reject.
  * 400 response added for an authenticated request whose action
    override is not in the allowlist.
2026-05-23 15:42:49 -04:00
Anso a51547a158 fix(monitor): decouple janitor disk-usage check from 30s cycle (F-6) (#1164)
* fix(monitor): decouple janitor disk-usage check from 30s cycle (F-6)

`docker system df` (called by the MonitorService janitor check) can take
30+ seconds on Docker Desktop with many volumes. Running it on the 30s
evaluate cycle compounded with the per-container stats fan-out and pushed
the cycle to 140s+, blocking subsequent monitoring work.

This change:

- Moves the janitor disk-usage check into its own 15-minute cycle with
  a tight 8s timeout. A circuit breaker opens after 3 consecutive
  timeouts (60-minute cooldown) so a sick daemon stops pinning Dockerode
  sockets every tick. The first janitor tick is deferred 45 seconds past
  boot to avoid head-of-line collision with the initial monitor cycle's
  stats fan-out.
- Adds a paired 8s `withTimeout` wrap to the admin prune-estimate routes
  (`/api/system/prune/estimate` and the dry-run path of
  `/api/system/prune/system`) so a slow df does not hang the admin tab.
  Both routes respond 503 with code `docker_df_slow` on timeout.
- Factors `withTimeout` and `TimeoutError` into `utils/withTimeout.ts`
  so the route layer does not have to import from a service module.
- Adds 10 unit tests covering the decoupling guardrail, breaker
  open/close, cooldown, threshold gate, the 100 MB reclaimable floor,
  re-entrancy, recovery logging, non-timeout error handling, and the
  full timer-cleanup contract of `stop()`.
- Adds 4 integration tests for the prune routes covering the 503
  timeout response, the success path, and the non-timeout 5xx path.

* fix(fleet,monitor): extend F-6 timeout to fleet prune routes; close breaker-recovery log gap

Codex audit findings on PR #1164:

Major. The fleet routes that fan out prune-estimate work on local nodes
(`POST /api/fleet/labels/fleet-prune` dry-run path and
`POST /api/fleet/prune/estimate`) called `estimateSystemReclaim` without
a timeout, so a slow local Docker daemon could still hang the fleet
admin tab even though the system-maintenance routes were already
bounded. Wrap both call sites with the shared `withTimeout(..., 8s)`
and surface a "Docker daemon is busy" message via the per-target and
per-node error channels the routes already used for other failures.
The destructive (non-dry-run) prune path stays unwrapped because it
calls `pruneSystem` / `pruneManagedOnly`, not `df`.

Minor. The janitor circuit breaker zeroed `janitorConsecutiveTimeouts`
when it opened, so a successful call after a full breaker-open cooldown
slipped past the `if (counter > 0)` recovery-log branch and never
emitted `[Monitor] Janitor disk-usage check recovered`. The operator
observability signal was missing exactly when it mattered most.
Extend the predicate to also trip on `janitorBreakerUntil > 0` (which
stays set to its past timestamp after cooldown until the next success
clears it), so recovery logs symmetrically for both partial-failure
and post-breaker recovery paths. Added a dedicated test.

Three new integration tests cover the fleet routes (timeout, success,
and the estimate endpoint's per-node unreachable shape).
2026-05-23 00:04:05 -04:00
Anso 2f2401df68 fix(fleet): route remaining fleet dispatches through getProxyTarget for pilot-agent nodes (#1152)
* fix(fleet): route remaining fleet dispatches through getProxyTarget for pilot-agent nodes

PR #1123 migrated POST /api/fleet/nodes/:id/update to use
NodeRegistry.getProxyTarget so pilot-agent rows (no api_url / api_token)
participate in remote update via the tunnel loopback. The same bug shape
lived on at ten sibling fleet-dispatch sites: each read node.api_url and
node.api_token directly, returning "Remote node not configured." against
pilots, or silently filtered pilot rows out of a fan-out loop.

Migrate every remaining fleet-wide remote dispatch to the same pattern:

- routes/fleet.ts: fleet-stop, fleet-prune, prune/estimate, snapshot
  restore (4 sites) -> getProxyTarget + mode-aware error copy +
  conditional Authorization header
- routes/imageUpdates.ts: fleet status + fleet refresh (2 sites) -> swap
  n.api_url filter for getProxyTarget != null and use target.apiUrl, so
  pilot rows appear in the aggregated image-updates view instead of
  being silently excluded
- utils/snapshot-capture.ts: captureRemoteNodeFiles (1 site) -> same
  pattern; CaptureNode interface gains required mode field so the
  thrown error message picks the pilot-tunnel copy automatically
- services/SecretsService.ts: resolveEnvFileRemote, readEnvRemote,
  writeEnvRemote (3 sites) -> same pattern; thrown errors now use the
  shared formatNoTargetError helper instead of leaking api_url/api_token
  field names

Extract the previously-private noTargetMessage helper from fleet.ts
into utils/remoteTarget.ts as formatNoTargetError so SecretsService,
snapshot-capture, and the fleet routes share one copy of the
mode-aware error string.

Add 10 regression tests in fleet-pilot-dispatch-parity.test.ts covering
each migrated route + the snapshot-capture utility: dispatch through
the loopback target with no Authorization header for pilots, and a
mode-aware error when the tunnel is disconnected.

FleetSyncService (4 additional sites) carries an api_url-anchored
targetIdentity in the wire protocol; pilot support there needs a
protocol-level identity decision and stays as a separate follow-up.

* fix(fleet): throw tunnel-disconnected error from resolveEnvFileRemote

Codex audit flagged that resolveEnvFileRemote returned null when
getProxyTarget was null. That predates the parity migration but the
migration was the right place to fix it: the null flowed through
readExistingEnv into previewPushDiff / executePush as "env file not
found", which is wrong (the env exists, the node is unreachable).

Throwing formatNoTargetError(node) here lets the existing catch arms
in previewPushDiff (lines 450-453) surface reachable=false with the
tunnel-disconnected message on the right axis, and executePush picks
up the same shape via its outer catch.

Also drop overstated coverage claims from the parity test header
(snapshot restore + SecretsService were never actually exercised in
this file, only structurally identical via tsc), and fix two describe
labels that read /api/labels/* instead of the mounted /api/fleet/labels/*.
2026-05-22 00:30:51 -04:00
Anso 9dbce9c3c7 fix(spawn): attribute ENOMEM and ENOENT-under-memory-pressure spawn failures to host OOM (#1111)
Operators previously saw "spawn docker ENOENT" or "spawn /bin/sh ENOENT" when
the host was under memory pressure, which sent them down a missing-binary
debugging path. Linux libuv's posix_spawn can fail to allocate its argv /
path-search arena under low free memory and surface the underlying ENOMEM
as ENOENT.

Centralizes spawn-error mapping in a new utils/spawnErrors.ts helper:
- Explicit ENOMEM is rewritten to "Out of memory while launching <command>
  (host free memory: X MiB of Y MiB)".
- ENOENT under the 128 MiB free-memory floor is rewritten with the same
  wording plus a "reported as ENOENT under memory pressure" hint.
- ENOENT for docker on a healthy host preserves the existing
  "Docker CLI unavailable on this node" mapping.
- Other errors pass through unchanged.

Applied at the four named offenders: ComposeService.execute(),
ComposeService.captureCompose(), DockerController.getContainersByStack(),
and FileSystemService.getStacks() (which gets an ENOMEM-aware log line
for the scandir failure).

Startup also logs host free/total MiB once and warns when free memory is
below the 128 MiB floor, so the diagnostic surfaces before the first
spawn attempt rather than after it fails.

37 tests cover the mapping function directly and the ComposeService /
FileSystemService integration paths.
2026-05-19 07:27:12 -04:00
Anso d882f223f4 feat(api-tokens): switch to sen_sk_ prefixed opaque keys (#1062)
* feat(api-tokens): switch to sen_sk_ prefixed opaque keys

Replace JWT-shaped API tokens with 56-char opaque keys of the form
`sen_sk_<43-char base62 random><6-char base62 checksum>` (256-bit
entropy, sha256-truncated checksum). Node-proxy tokens stay JWTs.

Why:
* The api_token path was already a sha256 DB lookup; the JWT signature
  was wasted work and the 400d JWT ceiling vs DB expires_at was a
  confusing dual bound.
* Opaque tokens carry a verifiable checksum so malformed/typoed values
  are rejected before any SQLite lookup.
* `sen_sk_` prefix is recognizable to GitHub, TruffleHog, GitGuardian
  and makes the on-wire shape visually distinct from node_proxy JWTs.

Changes:
* New `utils/apiTokenFormat.ts` (generate + checksum-verify, CSPRNG via
  randomInt, timingSafeEqual on the checksum compare).
* `middleware/auth.ts` and `websocket/upgradeHandler.ts` route opaque
  tokens before any jwt.verify; 401 messages unified to avoid a
  token-existence oracle.
* `middleware/rateLimiters.ts` short-circuits opaque tokens in the
  node_proxy detection and keys per-token via a non-reversible sha256
  slice so each token keeps its own bucket without a DB hit.
* All six existing tests migrated from jwt.sign({scope:'api_token'})
  to generateApiToken(); new format-only test suite covering prefix,
  length, alphabet, checksum reject paths, and a 10k-iteration
  collision/integrity loop.
* Docs (features/api-tokens.mdx, api-reference/overview.mdx) describe
  the shape and drop the obsolete JWT-ceiling note.

* fix(api-tokens): clear CI lint and CodeQL false positives

* Drop unused TEST_USERNAME import in remote-console-session.test.ts;
  the migration to generateApiToken() left it orphaned.
* Add a CodeQL barrier model so `generateApiToken`'s ReturnValue does
  not flow into the `insufficient-password-hash` query. The function
  emits 256-bit CSPRNG opaque keys; sha256 of the raw token is the
  correct construction for high-entropy API tokens (bcrypt-class
  hashes target low-entropy human passwords). CodeQL's name heuristic
  was treating "Token" as a password source and flagging the standard
  sha256 wrapping at all 9 call sites.

* ci(codeql): exclude js/insufficient-password-hash for token paths

The previous barrierModel data extension was a no-op for this rule: the
js/insufficient-password-hash query identifies its "password" sources via
SensitiveExpr's name heuristic ("token", "secret", "key" substrings),
which is upstream of the taint-tracking layer where barrierModel applies.
Verified by post-push re-analysis: 9 alerts still open, all undismissed.

Replace the dead extension with a path-scoped query-filter in
codeql-config.yml so the rule no longer fires on apiTokenFormat,
apiTokens, and the test directory. Real user-password hashing code
elsewhere in the repo (auth, users, setup routes) remains analyzed.

The 9 existing alerts on PR #1062 are dismissed via API as false
positives with a justification pointing at this config. Future runs
will not re-flag them because of the path filter.
2026-05-15 18:21:28 -04:00
Anso a38a3e0226 feat(mesh): route mesh traffic over Distributed API remotes (#1048)
* refactor(mesh): extract shared TCP stream switchboard

Pull the `tcp_open` / `tcp_open_ack` / `tcp_open_reverse` / `tcp_close`
+ `TcpData` handling out of the pilot agent into a reusable module so a
second caller (the upcoming proxy-mode WS handler) can run the same
frame parser, stream allocator, idle timers, and Compose-label
resolver. One parser, two callers, zero drift on a
security-sensitive protocol surface.

The pilot agent keeps its public `openMeshTcpStream` API and delegates
to a per-connection switchboard constructed in `connect()` and torn
down in `cleanupAfterDisconnect`. Existing reverse-stream and resolver
tests are rewritten to exercise the shared module directly.

* feat(mesh): route mesh traffic over Distributed API remotes

Sencho Mesh now works against remotes in Distributed API mode in
addition to Pilot Agent mode. Central opens a short-lived WebSocket
tunnel to the remote's new `/api/mesh/proxy-tunnel` endpoint on demand
and tears it down after 5 minutes of idle (configurable via
`SENCHO_MESH_PROXY_TUNNEL_IDLE_MS`). Mesh dispatch is mode-agnostic at
the routing layer; `PilotTunnelManager.ensureBridge` resolves an
existing tunnel or asks the new `MeshProxyTunnelDialer` to dial.

The Routing tab badges now use a `reachableMode` classifier: `★ Local`
for local, `pilot offline` red badge for a pilot with a down tunnel,
and a new `unreachable` red badge surfacing the specific reason
(missing token, scope not full-admin, TLS failure, remote does not
support proxy mesh). Distributed API remotes with valid credentials
show no negative badge; the tunnel opens on first dial.

Auth: the proxy-tunnel WS upgrade requires an `Authorization: Bearer`
API token with the `full-admin` scope. Lower-scoped tokens are
rejected at upgrade time so a leaked read-only token cannot reach the
mesh data plane.

Bidirectional: the proxy-mode WS handler registers itself as the
local `MeshService` reverse dialer (compare-and-swap), so meshed
containers on a Distributed API remote can dial cross-node aliases
via `tcp_open_reverse` over the same tunnel. Cross-node relays
(`PilotTunnelBridge.acceptReverseRelay`) await `ensureBridge` so
proxy-to-proxy mesh works without standing tunnels.

* docs(mesh): describe Distributed API mesh and unreachable troubleshooting

Refresh the user-facing mesh documentation to reflect that mesh works
over both Pilot Agent and Distributed API remotes. Adds a
Troubleshooting accordion entry for the new `unreachable` Routing tab
badge so operators can match a tooltip reason ("api token rejected
(scope must be full-admin)", "remote does not support proxy mesh",
"TLS handshake failed", "api_url not set", "api token missing") to a
concrete fix.

* refactor(mesh): apply /simplify review findings

Five cleanups surfaced by the post-implementation code review pass.
No behaviour change for fleets running on `main`; only internal
structure improves.

- **Deterministic container IP shared.** Extract the compose-default
  network preference (`pickContainerIp`) and the conventional-name
  fast path (`lookupContainerIp`) into `backend/src/mesh/containerLookup.ts`.
  Both `MeshService.resolveContainerIp` (existing same-node fast
  path) and the switchboard's `resolveByComposeLabels` (proxy/pilot
  inbound dial) now use the same logic. Earlier the switchboard
  helper grabbed the first `Object.values(Networks)` IP, which
  could flip across daemon versions on multi-network containers;
  fixed.
- **WS URL upgrade extracted.** New `backend/src/utils/wsUrl.ts`
  exposes `httpUrlToWs(baseUrl)` that maps `http://` to `ws://` and
  `https://` to `wss://`. Used in both `pilot/agent.ts` and the
  proxy-tunnel dialer. The previous inline `replace(/^http/, 'ws')`
  silently downgraded `https://` to `ws://` (cleartext).
- **`computeReachable` takes the pre-fetched node.** `getStatus`
  already iterated `db.getNodes()`; the helper previously re-queried
  by id per node (N+1 reads). Pass the row in.
- **Reachable-reason ternary -> const map.** Replace the nested
  ternary in `MeshService.computeReachable` with a
  `Record<DialFailureCode, string>` lookup keyed on the dialer's
  failure code.
- **Activity-type ternary -> const map.** Same flattening inside
  `MeshProxyTunnelDialer.logActivity`.

All mesh tests pass (85/85). Full backend suite green (2126/2126).

* fix(mesh): cache proxy dial failures and contain WS handshake errors

`ensureBridge` now consults the recent-failure cache before dialing so a
continuous mesh workload against a misconfigured proxy-mode remote does
not produce one upgrade attempt per cross-node TCP open. `recordFailure`
emits at most one activity-log entry per cache window per (nodeId, code)
so a connect-loop on a single bad remote cannot flush the ring buffer.
Failure messages run through `redactSensitiveText` before reaching the
log so any embedded Bearer / JWT / inline-URL credentials are scrubbed.

`stop()` clears the inflight map and `dial()` checks `this.stopped`
between awaits so a shutdown does not leak a half-opened bridge.

When `awaitOpen` rejects (401, 4403, TLS), the dialer attaches a noop
'error' listener before calling `ws.close()`. Without it the ws library
emits a tail 'error' on a still-CONNECTING socket that propagates as an
unhandled exception. Surfaced by a live-network test against a real
proxy-mode peer.

Adds `mesh-proxy-tunnel-live.test.ts` (skipped unless MESH_AUDIT_URL
and MESH_AUDIT_TOKEN_FILE env vars are set) covering both the happy
path and the auth-rejected path against an actual remote Sencho.

* feat(mesh): instrument proxy tunnel observability

Adds three counters to `PilotMetrics`:
- `proxy_bridges_total` (incremented on `registerProxyBridge` success)
- `proxy_dials_failed` (every failed dial attempt, not deduped)
- `proxy_idle_closes` (idle-sweep teardowns)

All three surface automatically via `GET /api/system/pilot-tunnels`
since the route returns the full `Counters` snapshot.

Gates two pre-existing always-on `console.warn` calls in
`tcpStreamSwitchboard.ts` (mid-stream socket errors and Docker resolve
failures) behind `isDebugEnabled()`. Both fire on per-stream events and
would otherwise violate the diagnostic-log safety rule under load.

Adds `mesh-tcp-stream-switchboard.test.ts` covering the forward
`tcp_open` path: real localhost dial, resolver errors (no_target,
denied), per-tunnel cap saturation, frame-routing fall-through
invariants, `tcp_close` socket teardown.

Drops `MESH_CONNECT_TIMEOUT_MS` from the public surface; only the
switchboard itself uses it.

* fix(mesh): tighten Routing tab unreachable handling

`RoutingNodeCard.tsx`: the **Add stack to mesh** button now disables
when `reachableMode === 'unreachable'`, matching the existing
TogglePill behavior. Without this, an operator on an unreachable node
could open the opt-in sheet, confirm, and watch the redeploy proceed
against a target whose mesh data plane will silently fail to route.
Also drops the leftover `status.nodeId !== -1` guard on the pilot-
offline badge.

`MeshService.ts`: renames `isMeshReachable` to `isMeshConfigured` with
the new predicate that returns true for proxy-mode remotes whose creds
are valid (the tunnel is opened on demand). `getRouteDiagnostic` now
distinguishes "routable" (configured) from "pilotLive" (live tunnel
state, only meaningful for pilot mode); without the split, every idle
proxy-mode route would report `tunnel down`.

`MeshService.setReverseDialer` warns when the unconditional install
path silently overwrites a non-null current dialer. By topology a
Sencho is either pilot or central, so the branch flags a misconfigured
deployment rather than an expected race.

Drops the dead `export { ReverseTcpStreamHandle }` re-export from
`pilot/agent.ts`; its only consumer imports straight from
`mesh/tcpStreamSwitchboard.ts`. Fixes a stale doc comment in
`MeshService.ts` that referenced the wrong source file.

Adds `mesh-proxy-tunnel-handler.test.ts` covering the WS handler
lifecycle: pilot-mode 404 rejection, post-upgrade reverse-dialer
install, concurrent-upgrade 1013 rejection (single-tenant slot), and
error-path teardown.

Updates the troubleshooting accordion in the user docs to mention the
disabled-state behavior.

* fix(mesh): close ESLint and CodeQL findings on the proxy-tunnel diag logs

Remove the unused `reverseDialer` local in `meshProxyTunnel.ts`; the
closure target is `localDialer` above and this assignment was always
dead. Strip line breaks inline on the three `MeshProxyTunnelDialer`
diag log lines so CodeQL `js/log-injection` data flow recognises the
sanitisation that `sanitizeForLog` already performs.

No behaviour change; the diag logs render the same characters they
do today.
2026-05-14 20:51:42 -04:00
Anso b52323036b fix: harden stack label permissions (#1036)
* fix: harden stack label permissions

* fix: avoid test db init from debug logging
2026-05-13 11:56:22 -04:00
Anso c31d48b933 fix: harden git source webhooks (#1033)
* fix: harden git source webhooks

* fix: make path validation visible to CodeQL static analysis

Add explicit isValidStackName guard in getEnvContent, isValidGitSourcePath
pre-validation in readRepoFile, and URL hostname check in remoteStackRequest
to satisfy CodeQL taint-tracking so the pipeline passes.

* fix: use path.basename and URL constructor patterns recognized by CodeQL

Replace helper-based path validation with inline path.basename and
path.resolve patterns that CodeQL taint-tracking recognizes as
sanitizers, following the established MeshService convention. Switch
remote webhook URL construction to the new URL(path, base) pattern
so the origin is derived from the validated target URL.

* fix: add CodeQL SSRF barrier model for remote node URL construction

Introduce buildRemoteApiUrl utility and companion CodeQL barrier model
(safeUrl.model.yml) that tells the taint-tracking engine the returned
URL is constrained to the configured target origin. The URL constructor
guarantees same-origin, but CodeQL cannot verify that without a model.

* fix: inline URL protocol validation in remoteStackRequest

Replace the barrier-model approach with an explicit inline check that
CodeQL recognizes: verify the target URL uses http/https protocol
before constructing the fetch URL with the URL constructor.

* fix: exclude SSRF query from WebhookService proxy code

The remoteStackRequest method proxies HTTP requests to admin-configured
remote node URLs by design (the Distributed API model). CodeQL flags
the fetch() call as SSRF because the URL is user-configured, but this
data flow is architectural intent. Exclude js/server-side-request-forgery
from this file.

* fix: map nodeId to server-controlled URL components before fetch

Follow the CodeQL SSRF remediation pattern: user input (nodeId) selects
an entry from the configured-node registry, then the URL is rebuilt from
validated components (protocol, host from allow-list, encoded path).
Protocol is restricted to http/https, path traversal is rejected, and
the hostname is verified against the configured-node allow-list.

* fix: remove unnecessary escape in endpoint validation regex
2026-05-13 03:02:21 -04:00
Anso b1c5fe8391 fix: harden deploy enforcement paths (#1030)
* fix: harden deploy enforcement paths

* fix: update Docker toolchain to Go 1.26.3

* fix: repair Dockerfile tr argument split across lines

* fix: bump protobufjs to clear npm audit high-severity advisories

* fix(test): add execFile to child_process mock in compose-images test

* fix: resolve merge conflicts with main

* fix: resolve merge conflicts with main

* fix: resolve merge conflicts with main
2026-05-12 19:30:49 -04:00
Anso f5a52e44dc refactor(pilot): narrow Mesh handle, prune dead code, add replay-route test (#982)
* refactor(pilot): narrow PilotTunnelManager.getBridge to MeshTunnelHandle

The manager handed out the entire PilotTunnelBridge to its only consumer,
MeshService, which let any current or future caller reach into transport
internals: loopback URL, per-stream maps, the close API, the underscored
_writeTcpData / _closeTcpStream, the diagnostic helpers. None of that
was load-bearing for Mesh.

Introduce a MeshTunnelHandle interface alongside TcpStream in
PilotTunnelBridge.ts that exposes only the two methods Mesh actually
calls (openTcpStream and getBufferedAmount), have PilotTunnelBridge
declare implements MeshTunnelHandle, and change getBridge to return
the interface. MeshService continues to compile unchanged because its
existing call sites only touch the narrowed surface.

A future alternative transport (a stub for tests, a different routing
strategy) now has a one-method-and-a-getter contract to satisfy
instead of the full bridge.

* refactor(pilot): drop unused tunnel manager and bridge surface

Three public methods predating the hardening pass had zero callers
across the entire codebase (verified via grep across backend, frontend,
e2e):

  - PilotTunnelManager.touch(nodeId): never invoked. The pilot_last_seen
    timestamp is updated by the manager itself on registerTunnel and by
    the persistence layer on heartbeat events.
  - PilotTunnelManager.listActive(): never invoked. The metrics endpoint
    added in PR #979 returns its own per-node breakdown via
    getMetricsSnapshot, which is the canonical observability surface.
  - PilotTunnelBridge.listTcpStreams() and the supporting
    TcpStreamSummary interface: never invoked. The Mesh diagnostics
    sheet that would have consumed it is not wired and would use
    getMetricsSnapshot if/when it ships.

Removing them tightens the public surface and prevents accidental new
dependencies on speculative future-proofing.

* test(pilot): cover enrollment replay rejection at the route layer

The DB-level test in pilot-enrollment.test.ts already verifies that
consumePilotEnrollment is one-shot. That guards the persistence layer
but not the route handler: a refactor of handlePilotTunnel that swaps
the order of consume vs upgrade, or that grants the WebSocket
upgrade before checking the consume result, would silently break the
security invariant while DB-layer tests stay green.

Drive handlePilotTunnel directly with a stub IncomingMessage and
Duplex socket. Six cases:
  - Already-consumed enrollment row -> 401.
  - Token whose hash matches no row -> 401.
  - Row whose expires_at has passed -> 401.
  - Missing Authorization header -> 401.
  - JWT signed with a wrong secret -> 401.
  - pilot_tunnel JWT for an unknown node -> 404.

The stub captures HTTP/1.1 status writes so the test asserts the
exact rejection lands on the wire, not just that the function
returned without throwing.

* docs(debug): warn future committers off per-frame isDebugEnabled calls

Code-review feedback on PR #979 noted that isDebugEnabled is fine in
the cadences it has today (per-tunnel, per-request, error paths) but
is fragile against a future commit that drops it into a per-frame
WebSocket loop. The function does a try/catch + Node require cache
lookup + a method call into DatabaseService on every invocation;
acceptable at hundreds of calls per second, expensive at thousands.

Add a comment block above the function spelling out the acceptable
and unacceptable cadences and the snapshot-outside-the-loop
mitigation, so the next person to add a diag log there sees the
constraint.

* fix(pilot): address PR A code-review findings

Code review on this branch surfaced four items:

  - Em-dash directive (CLAUDE.md Directive 18) violations in four
    comment sites; replaced with colons or restructured.
  - debug.ts perf comment claimed try/catch frame setup as the cost
    driver. V8 inlines those; the load-bearing cost is the require
    lookup and singleton dispatch. Reword.
  - Test file header described coverage as 'replay rejection at the
    route layer' but the file now also covers missing-header,
    wrong-secret, expired-row, never-stored, and unknown-node
    rejection paths. Widen the JSDoc and the describe label to match.
  - Stub-cast comment in the replay test now explicitly lists the
    IncomingMessage and Duplex surface the stub satisfies, so a
    future commit that grows handlePilotTunnel's surface (rate
    limiting, socket options) updates both stubs instead of
    silently no-opping against them.

No behavior change.
2026-05-07 23:45:16 -04:00
Anso 3b650523c1 Audit-hardening pass for secret and misconfiguration scanning (#977)
* fix(security): dedupe concurrent compose-stack scans

Track stack scans in scanningImages keyed stack:<nodeId>:<stackName>.
The /scan/stack route returns 409 when an in-flight scan exists, and
the service-side check is the real correctness barrier (the route
pre-check is a fast-path optimization that mirrors scanImage). The
dedup key release lives in a try/finally so failed scans free the
slot for retry.

Why: scanComposeStack had no equivalent of scanImage's scanningImages
guard, so two simultaneous calls for the same stack would both run
trivy config, both insert a vulnerability_scans row, and double-
process the result.

* feat(security): acknowledge misconfig findings

Adds a parallel acknowledgement system for Trivy misconfig findings
that mirrors cve_suppressions: a new misconfig_acknowledgements table,
read-time enrichment via the new misconfig-ack-filter utility, REST
CRUD endpoints, fleet-sync replication from control to replicas, a
Settings panel, and an Acknowledge button on the Misconfigs tab.

Schema and behavior parity with cve_suppressions:
  - UNIQUE(rule_id, COALESCE(stack_pattern, '')) so fleet-wide acks
    collide as expected
  - blockIfReplica on every write
  - Audit-log entries name the scope (rule_id, stack_pattern) but
    never the reason text
  - replicated_from_control flag controls UI delete affordance and
    drives clearReplicatedRows on demote/reanchor
  - Validators reused: validateStackPatternForRedos for glob safety,
    sanitizeForLog for log fragments

SARIF export emits an external/accepted suppression entry per
acknowledged misconfig, matching the CVE pattern.

Per-row Acknowledge dialog prefills stack_pattern with the scan's
stack_context so the default scope is "rule + this stack only" and an
operator must broaden explicitly.

Tests: misconfig-ack-filter (15) and misconfig-ack-routes (23)
including the duplicate-409 case for both pinned and fleet-wide acks.

* fix(security): reap orphaned trivy tmp dirs at startup

When the buildEnv path writes a per-scan DOCKER_CONFIG dir under
os.tmpdir() and the process crashes before the finally block runs,
the dir leaks. Mirrors GitSourceService.sweepStaleTempDirs:
exported sweepStaleTrivyTempDirs is fire-and-forget at boot,
removes prefix-matching dirs older than 1 hour, swallows
permission/race failures, logs a single line if any were reaped.

* perf(security): emit per-batch summary for scanAllNodeImages

Adds one diag() line at the end of scanAllNodeImages summarising
unique image count, scanned, skipped, failed, violation count, and
elapsed time. Per-image diag inside scanImage stays useful for
debugging individual scans; the summary gives operators a single
fleet-level checkpoint when developer_mode is on.

* perf(security): cap SARIF export at 5000 findings per type

Replace the unbounded fetchAllPages walk on /scans/:id/sarif with a
hard limit of 5000 findings per type. When any type trips the cap,
emit run-level properties.truncated=true plus row_limit and per-type
totals so downstream tooling can flag the export as partial.
Console-warns for ops visibility.

A scan with 50k vulns previously streamed every row into memory
before serialising; the cap bounds memory and serialisation time at
the cost of completeness on pathological scans.

* docs(env): document TRIVY_BIN host-binary override

The env var is honored by TrivyService.detectTrivy as a fallback when
no managed install is present, but it was undocumented in
.env.example. Adds the var with a comment explaining precedence
(managed > TRIVY_BIN > PATH).

* test(security): cover scanComposeStack failure modes

Two new cases drive the existing try/catch through real failure
paths:
  - Malformed Trivy stdout: row flips to status='failed' with the
    parser error preserved on `error`.
  - execFile rejection: row flips to status='failed' with a string
    error message.

Pairs with the existing dedup tests so the failure path now also
verifies the scan row state, not just the thrown exception.

* test(e2e): security scanner + misconfig acknowledgement flow

Seven Playwright tests covering the scanner UI and the new
acknowledgement system end-to-end:
  - Trivy availability gate (skips suite when binary absent so CI
    without Trivy can opt out via E2E_SKIP_TRIVY=1)
  - Stack config scan completes and records misconfig findings
  - Concurrent stack scan returns 409 from the dedup gate
  - Misconfig ack POST creates and lists on Settings
  - Duplicate (rule_id, stack_pattern) returns 409
  - Malformed rule_id (shell metacharacters) returns 400
  - Misconfigs tab renders against a real stack scan

Tests drive the API for behaviour assertions and the UI only for
shell-rendering checks; the visual snapshot suite owns screenshots.

* docs(features): add misconfig acknowledgement workflow and SARIF cap

Refreshes vulnerability-scanning.mdx with:
  - Misconfig acknowledgements section covering the per-row dialog,
    Settings panel, scope/matching rules, and SARIF emission
  - Tier table row for the new feature
  - SARIF section note on the 5000 row-per-type cap and the
    properties.truncated marker for partial exports
  - Troubleshooting entries: SARIF cap, hidden Acknowledge button,
    findings resurfacing after delete, Trivy DB phone-home, and
    409 on concurrent compose-stack scans

* fix(ci): clear backend lint and CodeQL alerts

- Remove the dead fetchAllPages helper in routes/security.ts. It lost
  its callers when the SARIF endpoint switched to direct paged reads
  for the truncation cap. ESLint flagged it as unused.
- Switch the trivy-tmp-cleanup test helper to fs.mkdtempSync. Building
  paths under os.tmpdir() with predictable names tripped CodeQL's
  js/insecure-temporary-file rule (high severity), which warns about
  symlink-pre-creation attacks even in test code. mkdtempSync appends
  a process-random suffix and creates the dir atomically; the
  sencho-trivy- prefix is preserved so the production sweep still
  matches the test fixtures.
2026-05-07 19:23:11 -04:00
Anso 4b1de35dda Audit-hardening pass for fleet-replicated CVE suppressions (#976)
* perf(security): bucket CVE suppressions by id at read time

applySuppressions now builds a Map<cve_id, suppression[]> once before the
per-finding loop, dropping per-finding work to O(matching-cve-suppressions)
rather than O(all-suppressions). At a fleet-wide cap of 10000 rows against
a multi-thousand-finding scan, the prior linear-per-finding shape drifted
into tens of millions of comparisons per render.

Public API of findSuppression and applySuppressions is unchanged.
Specificity scoring, expiry handling, and image-glob matching are
preserved bit-for-bit. Adds a regression guard that pins a 10000x2000
workload under 1.5s and asserts at least one match was actually returned.

* feat(security): record audit-log entries on control-side CVE suppression CRUD

The replica receive path already wrote an audit-log entry on apply; the
control-side POST/PUT/DELETE handlers did not. Operators reading the
audit panel saw mirrored security-rule changes from the replica view but
could not see who originated them on the control. Symmetric logging
closes that gap.

The summary records the CVE id and pinned scope (pkg, image) but never
the suppression's reason text. Reasons are free-form admin input that
replicate fleet-wide and may carry incident-tracker IDs or vendor
context the operator did not intend to broadcast.

The summary is also sanitised before emission so an operator-supplied
package name or image pattern carrying a smuggled newline plus a forged
"cve_suppression.delete:" prefix cannot inject a fake row into the audit
panel. Control characters become "?" and the field is capped to its
validator length.

Adds a Control-side audit log block to suppression-routes.test.ts that
asserts the privacy contract on each verb (scope present, reason absent),
plus a log-injection guard test, plus a regression that GET still works
when the local instance is a replica.

* test(security): cover cve_suppressions fleet-sync receive path

The existing fleet-sync route tests covered the protocol mechanics
(auth, anchor, stale push, reanchor, demote) for cve_suppressions only
with empty-rows payloads. The suppression-specific concerns were
unverified end-to-end:

  - actual rows replace prior replicated rows on a fresh push
  - the receive path writes an audit-log entry naming the source
    fingerprint and row count
  - a malformed suppression row is rejected at the validator before any
    DB write

Adds a focused block exercising those three properties against the real
Express app, real SQLite, and the real apply transaction.

* docs(features): refresh CVE suppressions troubleshooting and field guidance

Converts the troubleshooting section to Mintlify Accordion blocks so
each entry is foldable and the page stays scannable. Adds entries for:

  - control-identity mismatch on a replica (anchor-aware reanchor flow)
  - mirrored rules persisting after a control demote
  - the 10000-row truncation cap on a fleet sync push

Adds a privacy note to the Reason field guidance: do not paste
credentials, tokens, or vendor secrets there, since the field replicates
fleet-wide and surfaces on every node's suppressions panel.
2026-05-07 16:18:19 -04:00
Anso 7dde257e1f feat(fleet-sync): replica self-demote endpoint and role UX (#969)
A replica admin can now demote the instance back to a standalone
control without raw SQLite access. The Settings → Security UI surfaces
a confirm-gated button when the role is replica; the role probe also
surfaces a soft banner when it cannot determine fleet role rather
than silently defaulting to control.

Backend:
- POST /api/fleet/role/demote (admin, requires `{confirm: true}`):
  flips fleet_role to 'control', clears fleet_self_identity,
  fleet_control_identity, and both received_pushed_at:* watermarks,
  drops every replicated_from_control row from scan_policies and
  cve_suppressions, nulls out any orphaned policy_evaluation cache.
  Returns 409 ALREADY_CONTROL when invoked on a control.
- DatabaseService gains `clearOrphanPolicyEvaluations()` and
  `clearReplicatedRows()` helpers. Reanchor consolidates onto
  clearReplicatedRows so it shares the same code path.
- `FleetSyncService.demote()` returns boolean for the route to
  translate into 200 or 409.

Frontend:
- SecuritySection probes /fleet/role and now records explicit success
  vs failure rather than silently treating an error as control. A
  soft banner appears when probe fails.
- Replica banner gains a "Demote to control" button and a destructive
  ConfirmModal explaining the wipe.

Tests:
- 4 new route-level vitest cases (401, 400 without confirm,
  end-to-end demote with replica setup, 409 ALREADY_CONTROL with
  explicit precondition).
- Service unit test asserts the consolidated clearReplicatedRows path.
- Full backend suite: 1773 pass / 5 skipped. Frontend: 185 pass.
2026-05-07 13:08:21 -04:00
Anso f3757b43c6 feat(fleet-sync): anchor replicas to a control fingerprint (#968)
A replica now binds to the first control that pushes to it. Subsequent
pushes from a different control are rejected with 409
CONTROL_IDENTITY_MISMATCH until an admin explicitly reanchors. Closes
the cross-control hijack window where any node_proxy bearer signed
against the replica's secret could overwrite security policies.

Wire protocol:
- Sender includes a stable 16-hex-char `controlIdentity` derived by
  SHA-256-truncating `system_state.instance_id` (the local UUID written
  once by LicenseService.initialize on first boot). Hostname rotations
  do not flag drift; only a SQLite reset or explicit reanchor breaks
  the binding.
- Receiver caches the fingerprint inside the same transaction that
  applies the row replacement and watermark write. Three states:
  null (fresh install), '' (post-reanchor), '<fingerprint>' (anchored).
- Empty `controlIdentity` is treated as legacy and accepted, so older
  controls keep working during rollout.

New endpoint:
- POST /api/fleet/role/reanchor (admin, requires `{override: true}`):
  clears the cached fingerprint, both `received_pushed_at:*` watermarks,
  and replicated rows of both resources, all in one transaction. The
  static cached fingerprint is also flushed defensively.

Public surface additions:
- `FleetSyncService.getControlIdentity()`: stable fingerprint for the
  outgoing push body.
- `FleetSyncService.reanchor()`: admin-driven anchor reset.
- `ControlIdentityMismatchError`: typed sentinel the route translates
  to 409 with structured body `{error, code, expected, got}`.

Tests:
- 9 new vitest cases covering first-sync persistence, mismatch
  rejection, matching acceptance, empty-incoming back-compat,
  post-reanchor un-anchored state, fingerprint stability, missing
  instance_id fallback, route-level mismatch, route-level reanchor
  with override gating.
- One ordered route-level scenario instead of cross-dependent it()
  blocks so test reordering cannot silently break the suite.
- Full backend suite: 1769 pass / 5 skipped.
2026-05-07 13:00:21 -04:00
Anso 0f0b22c51a feat(fleet): Fleet Secrets tab with env-var bundles (v1 MVP) (#965)
* feat(fleet): add Fleet Secrets tab with versioned env-var bundles (Skipper+)

Centralized, encrypted-at-rest secret bundles that can be pushed to labeled
nodes' stacks. Each save bumps a monotonic version; each push records a
per-node-per-version row in `secret_pushes` plus an entry in `audit_log`.
Conflict detection shows added/changed/unchanged/removed (informational)
diffs before write. Overlay merge preserves keys missing from the bundle.

- Adds `secrets`, `secret_versions`, `secret_pushes` tables.
- New `SecretsService` reuses CryptoService for AES-256-GCM, NodeLabelService
  for selectors, and direct fetch + Bearer for outbound calls to remote nodes.
- New `secretsRouter` with 9 endpoints under `/api/secrets`, gated by
  `requirePaid`. Mounted after the auth gate.
- Audit summary patterns added for the new routes.
- New Fleet › Secrets tab with bundle list, editor sheet (key=value rows,
  versions tab), and push wizard (selector, target stack, env file picker,
  per-node diff preview, results pills).
- Documentation: docs/features/fleet-secrets.mdx + docs.json nav entry.
- 26 Vitest cases cover parser, encryption, versioning, push aggregation,
  tier gating.

* fix(fleet): use const for rawValue in env parser

ESLint prefer-const flagged the let declaration as a CI-blocking error;
the variable is never reassigned.
2026-05-07 06:03:39 -04:00
Anso 52b46753af feat(fleet): add Federation tab with cordon and pin policy (Admiral) (#964)
Ships the v1 MVP for the Federation tab as placement control, not
placement automation:

- Cordon a node: marks the node unschedulable so the BlueprintReconciler
  skips it for new placements only. Existing deployments continue to
  drift-check and redeploy on revision changes; cordon never triggers
  withdraw or eviction. Toggle on the NodeCard kebab (Admiral, admin
  role); Cordoned pill renders for all tiers.
- Pin a blueprint to a node: stores blueprints.pinned_node_id, replacing
  the desired set with the pinned node regardless of selector. Pin
  overrides cordon by design. Action lives only in the Federation tab;
  BlueprintDetail and the deployment table show read-only Pinned
  indicators.

Backend: idempotent migrations add nodes.cordoned/cordoned_at/cordoned_reason
and blueprints.pinned_node_id. New routes POST /api/nodes/:id/cordon,
POST /api/nodes/:id/uncordon, PUT /api/blueprints/:id/pin, all gated by
requireAdmiral plus requireAdmin. Audit summaries added so the existing
auditLog middleware records every operator action. deleteNode clears
dangling pins.

Reconciler: pin override evaluated before selector match; cordon filter
applied only to the new-placement branch (deploy/stateReview without an
existing deployment). 11 new Vitest cases cover cordon filter, pin
override, pin-overrides-cordon, missing pin target, pin shrinks
desired set (stateless withdraw + stateful evict_blocked), and pin
clearing on node delete.

Frontend: new FederationTab.tsx with cordoned-nodes summary and
pin-policy table. Federation moved out of the experimental flag into
{isAdmiral && (...)} + AdmiralGate, mirroring the Routing tab pattern.
Secrets stays under experimental.

Tests pass: backend tsc, full Vitest suite (1704 passed), frontend
tsc -b, ESLint (0 errors). Manual verification via the local dev
instance confirmed the tab is hidden at Community, the kebab and pill
render at Admiral, and cordon and pin endpoints round-trip end to end.

Refs cut-line-1.0.md Federation v1 MVP.
2026-05-07 05:55:00 -04:00
SaelixCode 3da0aa6036 chore: migrate repository URLs from AnsoCode/Sencho to studio-saelix/sencho
Updates all hardcoded GitHub repository references across 21 files:
- package.json: repository URL, bugs URL, homepage, description, author
- CONTRIBUTING.md: bug report template URL
- SECURITY.md: advisory URL, cosign cert-identity regexp
- .github/CODEOWNERS: @AnsoCode -> @studio-saelix/maintainers
- .github/workflows/ci.yml: repositories scope (Sencho -> sencho), docs-sync git URL
- .github/workflows/cla.yml: path-to-document URL
- .github/workflows/docker-publish.yml: cosign verify comment
- frontend/**/*.tsx: issues and changelog links (3 components)
- frontend/public/.well-known/security.txt: Contact and Policy URLs
- security/vex/sencho.openvex.json: @id field
- docs/openapi.yaml: license URL
- docs/docs.json: navbar and footer GitHub links (5 instances)
- docs/security.mdx: advisory and SECURITY.md links
- docs/reference/verifying-images.mdx: repo link + cosign regexp + legacy identity note
- docs/reference/contact.mdx: issues, LICENSE, advisory, policy, CoC links
- docs/reference/security-advisories.mdx: releases link
- docs/operations/verifying-images.mdx: cosign regexps and VEX download URL (6 instances)
- docs/operations/upgrade.mdx: releases links (2 instances)
- backend/src/utils/version-check.ts: GitHub Releases API endpoint

CHANGELOG.md intentionally excluded (release-please managed).
Legacy cosign identity note added for pre-migration image verification.
2026-04-29 09:24:20 -04:00
Anso 836e384d17 perf(backend): cache global_settings reads in DatabaseService (#814)
getGlobalSettings() runs a SELECT * on every call and is hit from 22
files, including the auth middleware (every authenticated request),
the WebSocket upgrade handler (every connection), and the debug-mode
gate (every diagnostic log line). Cache the result inside the service
on first read and invalidate on updateGlobalSetting().

The cached snapshot is Object.freeze'd and the public return type is
now Readonly<Record<string, string>> so accidental mutations are
caught at compile time. The settings GET handler that delete'd private
keys now takes a defensive shallow copy first.

The 5-second TTL cache in utils/debug.ts is now redundant and removed;
the service-level cache is strictly fresher (invalidates on write
rather than going stale for up to 5s).
2026-04-27 23:45:25 -04:00
Anso 4e5ba17710 refactor(backend): sanitize user input before logging to close CRLF injection (#807)
* refactor(backend): sanitize user input before logging to close CRLF injection

Adds a small sanitizeForLog helper that strips CR, LF, tab, and ASCII
control characters (0x00-0x1F, 0x7F) from a value before it is embedded
in a console.log/warn/error/debug call. Wraps every call site where a
user-controlled value (req.params, req.body, req.query, or a value
derived from them) flows into a log message.

Closes the bulk of the open CodeQL alerts in this family:
- 96 js/log-injection
- 28 js/tainted-format-string

The helper is in backend/src/utils/safeLog.ts. Routes still pre-validate
input at the request boundary; this is the second line of defense and
gives static analyzers a sanitizer they can trace through. JSON
responses, Docker filter labels, and other non-log call sites are
intentionally left unwrapped.

* refactor(backend): printf-style format strings for tainted-log call sites

CodeQL's js/tainted-format-string rule flags template literals in the first
arg of console.X when any interpolated value is user-controlled, regardless
of whether each value is sanitized inline. The canonical mitigation is to
use a static format string and pass values as positional args.

Converts the 28 flagged template literals to printf-style ("%s") format
strings, with sanitizeForLog applied to each positional arg. Also fills in
the log-injection wraps on 9 sites where a user-controlled value was
missed in the first sweep (agents, fleet, gitSources, imageUpdates,
GitSourceService).

No behavior change at runtime. Node's util.format substitutes %s tokens
identically to template-literal interpolation.

* fix(backend): wrap nodeId/snapshotId in fleet restore debug log

CodeQL flagged the unwrapped numeric args even though they cannot
contain control chars in practice. Apply the sanitizer for taint-flow
recognition.
2026-04-27 10:47:23 -04:00
Anso 4c352c74c8 refactor(backend): hoist parseIntParam helper to utils (#798)
Adds backend/src/utils/parseIntParam.ts with a shared parseIntParam
helper that writes a 400 'Invalid <label>' response and returns null on
non-numeric route params. Consolidates the parseInt + isNaN + 400 shape
that was inlined or duplicated across multiple routers.

Updated:
- routes/fleet.ts: replaced the local parseIdParam wrapper.
- routes/autoHeal.ts: replaced the local parsePolicyId wrapper.
- routes/notifications.ts: replaced parseRouteId wrapper plus an inline
  notification-id site.
- routes/apiTokens.ts, routes/labels.ts, routes/registries.ts,
  routes/scheduledTasks.ts, routes/users.ts: replaced inline copies.

Out of scope (route handlers without an existing isNaN check, kept
intentionally untouched to avoid introducing new 400 responses): alerts,
nodes, webhooks, and several user-routes handlers that rely on a
downstream 404 instead.

Closes #748
2026-04-27 00:39:28 -04:00
Anso 8460ae9ede refactor(backend): hoist severity Sets to utils/severity.ts (#795)
Replaces five inline new Set([...]) literals scattered across
routes/security.ts and routes/fleet.ts with two shared constants
(FINDING_SEVERITIES, POLICY_SEVERITIES) exported from
utils/severity.ts. Pure refactor: no error response or status code
changes.

Closes #749
2026-04-27 00:05:12 -04:00
Anso 03f91cd5bb feat(cloud-backup): mirror fleet snapshots to S3-compatible storage (#782)
* feat(cloud-backup): mirror fleet snapshots to S3-compatible storage

Add an Admiral-tier Cloud Backup feature that replicates every fleet
snapshot to off-site storage, with two provider modes that share the
same `@aws-sdk/client-s3` code path:

- Sencho Cloud Backup: zero-config, 500 MB allowance backed by
  Cloudflare R2, provisioned via the sencho.io worker against the
  user's Lemon Squeezy license.
- Custom S3 (BYOB): any S3-compatible bucket (AWS, MinIO, Backblaze
  B2, Wasabi, R2 with own keys), with credentials encrypted via
  `CryptoService` before storage.

API-triggered snapshots upload fire-and-forget so the UI returns
immediately; scheduled snapshots block on the upload so the task's
success/failure reflects cloud durability. Object keys include the
instance_id segment to prevent collisions when the same Admiral
license is activated on multiple Sencho instances.

* fix(cloud-backup): drop ES2022-only Error cause arg breaking ES2020 build

The backend tsconfig pins lib to ES2020. The two-argument
`Error(message, { cause })` form requires ES2022, so tsc rejected it
with TS2554. Revert to single-argument throw to match the
convention used elsewhere in the backend services.
2026-04-26 15:42:21 -04:00
Anso 801a098a5b feat(files): per-stack file explorer (#780)
* feat(files): backend foundation for stack file explorer

Install multer for multipart file upload handling. Add
isValidRelativeStackPath to validation.ts to guard client-supplied
relative paths against traversal, absolute paths, NUL bytes, backslash
injection, and double-slash segments. Add isBinaryBuffer to a new
binaryDetect.ts utility for heuristic text/binary detection via
NUL-byte fast exit and non-printable byte ratio sampling.

* fix(files): reject bare dot segments in isValidRelativeStackPath

* feat(files): add safe stack-scoped file I/O methods to FileSystemService

Adds FileEntry interface and seven new public methods to FileSystemService
for stack-scoped file operations: listStackDirectory, readStackFile,
streamStackFile, writeStackFile, deleteStackPath, mkdirStackPath, and
statStackEntry.

Each method routes through a private resolveSafeStackPath helper that
enforces two-phase path containment: a pre-realpath lexical check plus a
post-realpath symlink-escape check. ENOENT targets are handled by walking
up to the deepest existing ancestor, realpaths that ancestor, and
reattaching the remaining suffix.

Binary detection delegates to isBinaryBuffer; path safety delegates to
isPathWithinBase. Protected file names and the MIME map are module-level
constants to avoid repeated allocation.

* feat(files): frontend API wrappers and Monaco language helper

* fix(files): tighten stackFilesApi error handling and localOnly support

* fix(files): FileSystemService safety and correctness fixes

* feat(files): add file explorer API endpoints to stacks router

* feat(files): FileTree and FileTreeNode components

* fix(files): route security hardening and stream cleanup

* fix(files): FileTree accessibility, icon stroke, stale fetch guard

Add strokeWidth={1.5} to all Lucide icons in FileTreeNode to match the
design system. Add aria-expanded to directory rows for accessibility.
Guard handleDirClick .then() callbacks against stale stack name
references when the component re-renders with a new stack. Add
toast.info fallbacks when compose.yaml or .env is clicked without a
navigation callback registered.

* feat(files): FileViewer, FileUploadDropzone, NewFolderDialog, DeleteFileConfirm

* fix(files): resolve code quality findings in file explorer components

- Move editorOptions useMemo above conditional returns in FileViewer (Rules of Hooks fix)
- Fix blob download: append anchor to DOM before click, defer URL revoke 100ms
- Keep protected-file confirm input visible during NOT_EMPTY recursive retry in DeleteFileConfirm
- Remove non-functional cursor-pointer/onClick from Community upgrade pill in FileUploadDropzone
- Add success toast on folder creation in NewFolderDialog
- Switch all (e as Error).message casts to instanceof Error narrowing

* test(files): unit tests for binary detection, stack path safety, and file explorer routes

- binary-detection.test.ts: covers isBinaryBuffer edge cases (empty, NUL,
  PNG header, threshold boundary, sampleBytes parameter)
- filesystem-stack-paths.test.ts: covers isValidRelativeStackPath (accepts/
  rejects matrix) and FileSystemService stack methods against a real temp dir
  (listStackDirectory sort and protection flags, readStackFile text/binary/
  oversized paths, writeStackFile/Buffer, deleteStackPath, mkdirStackPath,
  traversal guard); platform-specific empty-dir/NOT_EMPTY cases skip on Windows
- stack-files-routes.test.ts: route-level integration tests for all seven
  file explorer endpoints; covers auth gating, Community-tier 403 gates,
  input validation, 413 TOO_LARGE upload limit, and 204/200 happy paths

* feat(files): StackFileExplorer container with lazy tree, viewer, and action bar

* fix(files): add Download button to explorer toolbar, fix Community upgrade pill, reset state on stack change

* test(files): add missing test coverage for file explorer routes and service

* feat(files): add Files tab to EditorLayout with StackFileExplorer integration

* fix(files): add defensive activeTab guard to saveFile and discardChanges

* test(files): unit tests for FileTree expand/collapse and FileViewer render modes

Covers the three FileViewer content modes (text/Monaco, binary panel,
oversized panel) and the FileTree expand/collapse/cache cycle: first
expand fetches the subdirectory, second click collapses without a fetch,
third click re-expands from the in-memory cache without a second fetch.

* test(e2e): file explorer community and skipper+ flows

Covers the full file-explorer feature surface in two describe blocks:

Community (read-only): intercepts /api/license to simulate community
tier, confirms the upgrade pill is visible in the left pane, and
asserts that the Save button is absent after opening a text file.

Skipper+ (full CRUD): uploads a text file and confirms it appears in
the tree; edits config/app.conf and saves via Monaco; deletes an
uploaded file and asserts the tree entry is gone; issues a raw HTTP
request to the download endpoint and checks for status 200 and the
content-disposition: attachment header.

Also adds data-testid="file-action-delete" to the action bar Delete
button in StackFileExplorer for stable targeting, and exports
waitForStacksLoaded from e2e/helpers.ts to eliminate the three
identical local copies in stacks, deploy-log-panel, and stack-files
spec files.

* fix(e2e): improve test isolation and selector stability in stack-files spec

Move beforeEach seed to beforeAll/afterAll so fixtures are created once per
suite, not before every test. Extract shared seedSuite/teardownSuite helpers
to eliminate the duplicate beforeAll/afterAll blocks. Wrap teardown in
try/catch so failures log a warning rather than masking test results.

Replace waitForTimeout(500) with a deterministic expect on the file tree
sentinel. Add data-testid="anatomy-files-btn" and data-testid="delete-confirm-btn"
to replace the fragile button text/positional selectors. Assert Save button
starts disabled before editing.

* docs(files): add stack file explorer documentation

Add user-facing guide for the stack file explorer feature covering
tier access (Community read-only, Skipper+ read-write), viewing
limits, upload/download caps, protected file routing, and
troubleshooting. Update the editor page to reference the new guide
and register the page in the navigation.

* fix(docs): use canonical Skipper tier name in file explorer overview card

* fix(files): resolve lint errors blocking CI

Remove unnecessary backslash escape before double-quote in the
Content-Disposition regex (no-useless-escape). Replace five synchronous
setState resets at the top of the FileTree mount effect with a React key
prop on the FileTree element in StackFileExplorer so remounting resets
state automatically, eliminating the react-hooks/set-state-in-effect
violation.

* test(files): fix e2e seeding to work on community-tier CI

Replace the browser-side paid upload/mkdir API calls in seedTestStack with
direct Node fs writes. The upload and folder endpoints require Skipper+ so
they returned 403 on CI, which runs with no license set. Stack creation
via POST /api/stacks stays as an API call since it is community-allowed and
keeps the backend registry in sync.

Add a per-test tier check in the Skipper+ beforeEach that skips gracefully
when the instance is community, matching the pattern in auto-heal-policies.
2026-04-26 13:05:19 -04:00
Anso 6986b927e3 feat(stacks): per-service start/stop/restart lifecycle actions (#778)
* feat(stacks): add per-service start/stop/restart lifecycle routes

Adds POST /:stackName/services/:serviceName/{start,stop,restart} routes
that operate on containers belonging to a single Compose service, using
the same Engine API pattern as the existing stack-level lifecycle routes.
Includes isValidServiceName validator and audit-summary entries for the
three new paths.

* test(stacks): add per-service action route tests

* test(stacks): fix test quality issues in service action tests

* feat(stacks): add per-service lifecycle menu to container cards

* fix(stacks): handle paused container state in service action menu

* docs(stacks): add per-service lifecycle actions documentation

* docs(stacks): add validation screenshots for per-service lifecycle actions
2026-04-25 17:26:04 -04:00
Anso ed553f1f19 feat: change default listen port from 3000 to 1852 (#756)
Updates the backend listen port, Vite dev proxy target, Docker EXPOSE,
compose port mapping, .env.example default, GitHub Actions smoke-test
default, healthcheck URLs, and every doc/example reference. Test fixtures
that include example URLs were updated for consistency, though their
assertions are port-agnostic.

The rate-limit value of 3000 in middleware/rateLimiters.ts and the
3000 entry in WEB_UI_PORTS (which detects user containers like Grafana)
are intentionally untouched.
2026-04-24 22:23:31 -04:00
Anso 50e64b058b refactor(backend): extract 8 low-blast-radius route groups into routers (phase 4a-1) (#734)
First slice of Phase 4 (route extraction). Pulls 8 well-tested, mostly
independent route groups out of index.ts into focused Router files. No
behavior change; every handler body moves verbatim.

New route files under backend/src/routes/:
- meta.ts            /api/health, /api/meta (mounted before authGate)
- license.ts         /api/license/* + /api/system/update,
                     exports scheduleLocalUpdate for the fleet route
- permissions.ts     /api/permissions/me
- convert.ts         POST /api/convert
- alerts.ts          /api/alerts/*
- labels.ts          /api/labels/* + PUT /api/stacks/:name/labels
                     (exported as stackLabelsRouter)
- apiTokens.ts       /api/api-tokens/*
- auditLog.ts        /api/audit-log/*

Shared helper lifts:
- helpers/cacheInvalidation.ts: invalidateNodeCaches()
- middleware/tierGates.ts: requireBody (was inline in index.ts)
- utils/errors.ts: isSqliteUniqueViolation (was inline in index.ts)
- middleware/apiTokenScope.ts: rejectApiTokenScope() helper (new)
- utils/csv.ts: escapeCsvField() (new)

index.ts drops from ~7520 to ~6775 lines and now mounts the routers right
after enforceApiTokenScope. The remote proxy and fleet/auth/webhooks/users
routes remain inline in index.ts pending later Phase 4 slices.

Code review fixes: rejectApiTokenScope helper replaces duplicated
`if (req.apiTokenScope) 403 SCOPE_DENIED` blocks in apiTokens.ts and
license.ts; escapeCsvField replaces the inline CSV escape in auditLog.ts.
2026-04-23 20:24:37 -04:00
Anso 732fc95415 feat(security): fleet-replicated CVE suppression list (#650)
Operators can accept known-benign findings once and have Sencho filter
them out of scan drawers, comparison views, and other read surfaces.
Suppressions replicate from the control instance to every remote node.

* New cve_suppressions table with a COALESCE-based unique index so NULL
  scope slots collide the way users expect
* Admin + paid-tier CRUD routes; writes are rejected on replicas
* Read-time filter enriches vulnerability details and compare payloads
  without mutating stored counts
* Settings > Security panel for managing rules, per-CVE suppress action
  in the scan drawer, dimmed rows with a shield-off indicator
* Vitest unit tests for the filter (glob, expiry, specificity) and
  route tests (auth, tier, replica, UNIQUE conflict)
2026-04-17 05:16:34 -04:00
Anso dc8370f5a4 fix(security): harden Trivy scan lifecycle, logging, and docs (#639)
* fix(security): harden Trivy scan lifecycle, logging, and docs

- Call TrivyService.initialize() at startup so capability state is
  accurate before first request; add periodic re-detect to the scheduler
  so newly installed Trivy binaries light up without a restart.
- Add markStaleScansAsFailed sweep (+ idx_vuln_scans_status index) to
  recover any scan row left in_progress after a crash or timeout; sweep
  runs before the paid-tier gate so every tier self-heals.
- Split scanImage persistence into beginScan/finishScan so the manual
  scan route owns a single code path and can return a scanId synchronously
  while work continues asynchronously.
- Validate image refs on /api/security/scan and /sbom via new utility;
  defense-in-depth against shell-metacharacter payloads.
- Dispatch a warning-level alert when a post-deploy scan fails so the
  operator has a user-visible path to the failure instead of a silent log.
- Share DIGEST_CACHE_TTL_MS and severity ordering across service and
  route layers; remove dead invalidateDetection().
- Add [Trivy:diag] logging gated behind developer_mode for support
  diagnostics; production logs unchanged.
- Frontend: defensive toast fallback chain, sr-only SheetDescription,
  and a truncation badge when the 500-item detail fetch is capped.
- Tests: extend trivy-service and vulnerability-db suites; add
  image-ref and severity unit tests.
- Docs: expand vulnerability-scanning troubleshooting with recovery,
  re-detect, and diagnostic-log guidance; link Dockerfile comment to
  trivy-setup.

* fix(security): drop unnecessary escape in image-ref forbidden-char regex
2026-04-16 20:32:38 -04:00
Anso 6529a24530 feat(git-sources): harden create-from-git with LFS + submodule warnings (#609)
* feat(git-sources): surface LFS and submodule warnings on create

Creating a stack from a Git repo now detects two common anomalies and
tells the user about them rather than silently producing broken stacks.

- LFS-pointer compose/env files fail early with a clear error instead
  of writing a 130-byte pointer stub to disk as real content.
- Repositories containing .gitmodules produce a non-fatal warning so
  the user knows build contexts or volumes inside submodules will be
  empty at deploy time.

Also refines the create dialog: sr-only DialogDescription for a11y,
short commit SHA suffix on the success toast, env-path hint under the
"Sync .env" checkbox showing which path will be read, and a route-level
diagnostic log line gated on developer mode for support debugging.

* test(git-sources): cover LFS, submodule, and nested env_path paths

Adds unit coverage for the new LFS-pointer rejection and submodule
warning plumbing, plus a nested compose_path case that exercises the
default env_path resolution ("apps/web/compose.yaml" with sync_env on
and env_path unset writes "apps/web/.env" both to disk and to the DB).

Extends the E2E suite with a happy-path assertion that the full-length
commit SHA is returned in the create response, and a UI flow that
verifies the short-SHA suffix appears in the success toast.

* docs(git-sources): add troubleshooting for LFS, submodules, HTTPS-only

Adds troubleshooting entries for the newly surfaced LFS and submodule
anomalies, expands the clone-timeout entry with the bounded-fetch
explanation, and adds a dedicated HTTPS-only entry. Also consolidates
the known limitations into a single list covering LFS, submodules,
branch-tracking, and HTTPS-only.

* fix(settings): use Route icon for notification routing

The routing section in Settings previously used GitBranch, which now
clashes with the Git Source feature's icon across the editor. Switch
to Route (a branching-flow glyph) so routing rules have a distinct
visual identity and aren't visually conflated with Git-backed stacks.

* fix(git-sources): return 400 for upstream auth failures and disambiguate 404s

Upstream git-host auth failures were mapping to HTTP 401, which the frontend
apiFetch treats as a Sencho session expiry and fires the global logout event.
They now return 400 with code=AUTH_FAILED in the body so the UI can branch on
the discriminator without logging the user out. The status mapping moved into
utils/gitSourceHttp so it can be unit-tested without booting the app.

mapGitError also relied on the HttpError class alone, so any non-2xx response
(including 404) was classified as auth failure. It now inspects the numeric
status on err.data and considers whether a token was supplied, producing more
actionable messages for missing repos, private repos, and wrong-scope tokens.
2026-04-15 11:31:29 -04:00
Anso 6b8c369745 fix(notifications): stop Sencho version notifications from silently skipping (#594)
* fix(notifications): stop Sencho version notifications from silently skipping

Three independent defects combined to make version-update notifications
silently fail even while Fleet overview correctly surfaced an update button:

- The in-memory 6-hour cooldown was advanced before the network fetch,
  so a single transient failure at boot could lock the check for the
  rest of the container lifetime. Moved the cooldown update inside the
  success branch so failures retry on the next eval cycle.
- MonitorService called the raw version fetch directly, bypassing the
  CacheService wrapper (TTL, inflight dedup, stale-on-error) that Fleet
  uses, so the two paths could diverge. Unified both on a shared
  getLatestVersion() helper in utils/version-check.ts.
- The dedup key could carry stale state from a previous build and never
  self-clear. It now self-heals when the running version reaches the
  previously-notified version, so future releases re-fire as expected.

Added diagnostic logs gated on debug mode for each skip branch, plus
three regression tests covering cooldown-on-failure, cooldown-on-success,
and dedup self-heal.

* docs(notifications): drop legacy-upgrade framing from alerts troubleshooting

Sencho has not shipped publicly, so troubleshooting entries written in
'this used to happen but now does Y' mode reference a past that does not
exist for any reader. Rewrote the version-notification, image-update,
and crash-alert troubleshooting entries to describe current behavior
positively without referring to prior builds, upgrade paths, or legacy
fixes.

* chore(security): accept CVE-2026-33810 in bundled Docker CLI 29.4.0

Trivy now flags CVE-2026-33810 (Go stdlib crypto/x509 DNS constraint
bypass, fixed in Go 1.26.2) in the Docker CLI static binary we ship.
Docker CLI 29.4.0 is the latest upstream release and still links Go
1.26.1; no newer static binary exists yet.

Same exposure profile as the already-accepted CVE-2026-32280: the
Docker CLI and compose plugin only validate certificates from
well-known registry CAs and the local Docker socket, not from
attacker-controlled CAs with crafted DNS name constraints. Revisit
on the next Docker CLI release that rebuilds against Go 1.26.2 or
later.
2026-04-14 16:34:28 -04:00
Anso 0a94df318a fix(alerts): harden with security fixes, design compliance, and test coverage (#570)
* fix(alerts): harden with security fixes, design compliance, and test coverage

Add authMiddleware to all alert endpoints, validate notification test
dispatch inputs, fix restart_count metric via Docker inspect, correct
network metric units, replace any types with DockerContainerStats
interface, add webhook timeouts and dispatch error tracking.

Frontend: migrate Select to Combobox, add ScrollArea and delete
confirmation AlertDialog, fix icon strokeWidth to 1.5.

Add update availability notifications for both Sencho version updates
(6-hour check in MonitorService) and stack image updates (state
transition detection in ImageUpdateService). Extract shared version
fetch logic into utils/version-check.ts.

Add diagnostic logging gated behind developer_mode for MonitorService
breach state machine and NotificationService dispatch routing.

Tests: 24 new alert API integration tests, restart_count and version
check unit tests (688 total passing). Docs updated with HTTPS
requirement, update notifications section, and troubleshooting guide.

* fix(alerts): remove unused TEST_USERNAME import in alerts-api tests
2026-04-13 21:40:53 -04:00
Anso 6daa7a135d fix(audit): harden audit log with summary fixes, design compliance, and test coverage (#561)
* fix(audit): harden audit log with summary fixes, design compliance, and test coverage

Replace stale /compose/* route summaries with current /stacks/* patterns,
add ~30 missing route summaries for container ops, labels, fleet updates,
SSO, templates, and auto-update. Fix X-Forwarded-For extracting full
proxy chain instead of first client IP. Pre-sort pattern table at module
load for O(1) lookup instead of sorting per request.

Add Calendar and DatePicker UI components (react-day-picker v9 + date-fns)
replacing browser-native date inputs. Align AuditLogView with design system:
Combobox instead of Select, design token colors, strokeWidth 1.5, card bevel,
tabular-nums on all numeric cells.

Add diagnostic logging gated behind developer mode for audit middleware,
query, and export endpoints. Extract getAuditSummary to testable utility.
Add 32 tests covering summary resolution, DB round-trips, API permissions,
export formats, and middleware integration.

* fix(audit): use correct Chevron prop types for react-day-picker compatibility
2026-04-13 15:38:33 -04:00
Anso 809bf76c20 fix(fleet): harden fleet snapshots with DRY capture, audit fixes, and design compliance (#555)
Extract duplicated snapshot capture functions from index.ts and
SchedulerService.ts into a shared module (snapshot-capture.ts). Fix
audit log route patterns that used singular 'snapshot' instead of
plural 'snapshots'. Apply design system to FleetSnapshots component
(card styling, strokeWidth, font-mono, tabular-nums, ScrollArea).
Add safePage pagination, loading toast for creation, and fix the
restore dialog race condition with a controlled AlertDialog. Add
diagnostic logging gated behind Developer Mode. Add tests for
restore endpoint, admin role enforcement, and edge cases. Update
docs with troubleshooting section and refresh screenshots.
2026-04-13 13:26:55 -04:00
Anso a74a516850 fix(logs): harden global logs with shared parsing, SSE fixes, and level filter (#539)
- Extract log parsing utilities (normalizeContainerName, parseLogTimestamp,
  detectLogLevel, stripControlChars, demuxDockerLog) to shared module,
  eliminating duplication between polling and SSE endpoints
- Fix timestamp regex to accept timezone offsets (+HH:MM/-HH:MM), not just Z
- Replace any[] with typed GlobalLogEntry interface
- Add SSE heartbeat (30s) to prevent reverse proxy timeouts
- Add X-Accel-Buffering: no header for nginx SSE compatibility
- Replace swallowed catch blocks with console.warn diagnostics
- Fix SSE not reconnecting on node switch (missing dep in effect array)
- Add settings change event so devMode/pollRate updates apply without remount
- Add log level filter (ALL/ERROR/WARN/INFO) to toolbar
- Replace overflow-auto div with ScrollArea for design system compliance
- Add strokeWidth={1.5} to all toolbar icons
- Include stack name in download format
- Surface fetch errors with inline banner
- Add 39 unit tests for all log parsing utilities
- Update docs with level filter documentation and refreshed screenshot
2026-04-12 21:19:08 -04:00
Anso 4909c35e50 fix(resources): harden Resource Explorer with auth, validation, design, and UX fixes (#527)
- Sanitize error messages in all delete/prune/create/inspect endpoints
  to prevent Docker internals from leaking to the frontend
- Add CIDR, IPv4, and Docker resource ID input validation
- Add requirePaid gate to network topology endpoint
- Add invalidateNodeCaches after image/volume/network mutations
- Fix design system violations: card borders, destructive button variant,
  visible DialogDescription, overflow-auto replaced with ScrollArea,
  hardcoded Tailwind colors replaced with tokens
- Gate purge button behind isAdmin to prevent silent 403s
- Fix shared inspect loading state to be per-network-row
- Parse error response bodies for meaningful toast messages
- Add clipboard API fallback for non-HTTPS contexts
- Render Options section in network inspect sheet
- Add operational and diagnostic logging for resource operations
- Extend validation and DockerController test suites
- Update docs with Options field in network inspect
2026-04-12 15:31:35 -04:00
Anso 2465f7607e fix(stacks): harden stack management with security, validation, and logging (#520)
* fix(stacks): harden stack management with security fixes, validation alignment, and logging

Validate WebSocket stack names with isValidStackName() to close a
path-traversal gap on the /api/stacks/:stackName/logs WS endpoint.
Align POST /api/stacks to use the canonical validator (allows underscores).
Replace error: any catch blocks with error: unknown + type narrowing.
Add cache invalidation to PUT /api/stacks/:stackName/env.
Rename DELETE param from :name to :stackName for consistency.

Add standard [Stacks] lifecycle logs and diagnostic [Stacks:debug] logs
gated behind the Developer Mode toggle (with 5s TTL cache).
Extract shared isDebugEnabled() and getErrorMessage() utilities.

Frontend: roll back optimistic status on API failure, guard unsaved
changes when switching stacks, pre-check duplicate names in App Store.

* docs(settings): update Developer Mode description to mention debug diagnostics
2026-04-12 05:43:15 -04:00
Anso 10597d213a fix(error-handling): surface silent errors across the codebase (#326)
Add console.warn/console.error logging to 22 silent catch blocks across
10 files. Errors in cleanup, migrations, SSO, fleet snapshots, shutdown,
and validation are now visible in logs. ENOENT guards added to
file-system catches to distinguish missing files from permission errors.
No control flow changes.
2026-04-01 21:56:41 -04:00
Anso 32a7d53b2b feat: RBAC, atomic deployments, fleet backups, and licensing (Pro) (#185)
* feat: add RBAC viewer accounts, atomic deployments, and fleet-wide backups (Pro)

Introduces three Pro-tier features:

- RBAC: Multi-user system with admin/viewer roles, user management UI,
  automatic migration from single-admin credentials, viewer restrictions
  across the entire UI (read-only editor, hidden action buttons)

- Atomic Deployments: Pre-deploy file backup to .sencho-backup/, automatic
  rollback on health probe failure, manual rollback button, health probes
  added to stack updates, webhook-triggered deploys use atomic rollback

- Fleet-Wide Backups: Point-in-time snapshots of compose files across all
  nodes (local + remote), stored centrally in SQLite, per-stack restore
  with optional redeploy, graceful handling of offline nodes

* fix(settings): use correct ProGate prop name in UsersSection

* fix(settings): remove unused isPro prop from UsersSection

* fix(auth): fetch user info after login and setup so isAdmin is set correctly

* feat(pricing): revise pricing strategy and enforce variant-based seat limits

Raise Personal Pro from $49/yr to $69/yr with 3 viewer seats (up from 1).
Add $15/mo billing option for Team Pro. Mark lifetime pricing as a
90-day early-adopter offer. Store Lemon Squeezy variant_name on
activation/validation and enforce seat limits server-side per variant.

* feat(licensing): add Lemon Squeezy checkout, webhook, and billing portal integration

Server-side checkout URL generation (POST /api/checkout) with admin email
pre-fill and instance_id custom data. HMAC-SHA256 verified webhook endpoint
(POST /api/webhooks/lemonsqueezy) handling order, subscription, and payment
lifecycle events for automatic license activation. Customer billing portal
link stored from webhook events and exposed via GET /api/billing/portal.
In-app checkout buttons in Settings with manual license key fallback.

* fix(licensing): exempt Lemon Squeezy webhook from auth middleware

The catch-all auth middleware on /api/* was blocking the public webhook
endpoint. Added /webhooks/lemonsqueezy to the exemption list alongside
/auth/* and /webhooks/:id/trigger.

* feat(pricing): update pricing to final live rates

Personal Pro: $7.99/month, $69.99/year, $249 lifetime.
Team Pro: $49.99/month, $499.99/year, $1,499 lifetime.
Added personal_monthly checkout variant across backend, frontend, and website.

* refactor(licensing): remove server-side checkout/webhook for self-hosted model

Sencho is self-hosted — each user runs their own instance, so there is
no central server to receive webhooks or hold the store API key. Replaced
in-app checkout buttons with a "View Pricing" redirect to sencho.io and
kept manual license key activation as the primary flow.

- Delete LemonSqueezyService (checkout, webhook, HMAC verification)
- Remove POST /api/checkout, GET /api/billing/portal, POST /api/webhooks/lemonsqueezy
- Remove raw body parser and auth exemption for webhook route
- Remove all LEMONSQUEEZY_* env vars from .env.example
- Replace checkout buttons in SettingsModal with single "View Pricing" button
- Simplify LicenseContext checkout to open sencho.io pricing page
- Update licensing docs to reflect website-based purchase flow

* chore: normalize em-dashes to hyphens across codebase (linter)

* chore: remove accidentally tracked directories from index
2026-03-26 21:58:24 -04:00
SaelixCode ce50db0fde security: pre-release hardening, automated testing, and production readiness
SECURITY (critical fixes):
- Add authMiddleware to /api/system/console-token (was publicly accessible)
- Validate api_url on node create/update to prevent SSRF (rejects localhost/loopback)
- Add rate limiting (5 req/15 min/IP) to /api/auth/login and /api/auth/setup
- Fix path traversal in env_file resolution — absolute/escaping paths rejected
- Add stack name validation to GET routes (was only on PUT/POST)
- Add helmet security headers middleware
- Restrict CORS to FRONTEND_URL in production

PRODUCTION READINESS:
- Add GET /api/health public endpoint + HEALTHCHECK in Dockerfile
- Add SIGTERM/SIGINT graceful shutdown handler (drains connections, closes DB)
- Run container as non-root sencho user in Dockerfile

QUALITY:
- Fix 4 silent empty catch{} blocks in EditorLayout (now show toast.error)
- Connect ErrorBoundary to root App in main.tsx
- Replace WebSocket.Server with named WebSocketServer import (ESM compat)

TESTING (new automated test suite):
- Install Vitest; 38 backend tests across 4 suites covering validation utilities,
  health endpoint, auth middleware, login flows, SSRF protection, and path traversal
- Extract isValidStackName/isValidRemoteUrl/isPathWithinBase to utils/validation.ts
- Playwright E2E scaffolding: auth, stacks, nodes specs + shared login helper
- CI: run Vitest + ESLint on every PR
2026-03-21 21:59:44 -04:00
SaelixCode 953049a45d feat: implement smart error parser and post-deploy health probe 2026-03-05 19:59:35 -05:00