Commit Graph

24 Commits

Author SHA1 Message Date
Anso 578ce7684d feat(git): complete-project materialization with a managed-project manifest (#1786)
* feat(git): add managed-project manifest types and DB cache columns

Introduces the canonical managed-project manifest contract types (schema v1)
and the stack_git_sources cache columns manifest_version / manifest_state /
manifest_generation. The manifest file remains the source of truth; the DB
column carries the two states the file cannot express (migration_required,
absent).

* feat(git): add vendored Docker .dockerignore matcher

Implements docker patternmatcher semantics for build-context materialization:
basename matching for slash-less patterns, anchored root patterns, ** crossing,
last-match-wins negation, dir-only patterns, char classes, comments and
escapes. Table-driven tests cover the full rule set.

* feat(git): add pure Compose input declaration parser

Walks explicit compose files plus recursive include/extends.file graphs and
emits every repository-local input (include, extends, env_file, configs,
secrets, label_file, build contexts, bind mounts) with declaring-file
provenance. Side-effect free: file contents are injected via a read callback.
Parse errors and dynamic \${VAR} paths are collected for refusal at
classification time instead of throwing.

* feat(git): add Compose input discovery service

Classifies every declared input against the cloned tree as managed,
unmanaged, or refused: containment, symlink/device/LFS/submodule guards,
file and path-depth caps, dockerignore-aware build-context planning with the
repo-root context bound, implicit override discovery for single-file stacks,
and the shared walkAndCopy candidate builder with aggregate caps.

* feat(git): add managed-project manifest service

Owns the canonical inventory at <DATA_DIR>/git-managed/<nodeId>/<stackName>:
untrusted reads with shape/enum/identity validation, bounds config,
candidate build with completion-marker gating, transactional promotion with
crash marker + previous-generation restore, boot sweep that declines over
hand-repaired state, lazy migration from applied_deploy_spec with
conservative deletion authority, and the detach export render.

* feat(git): complete-project pull/apply with staged promotion and detach export

Pull now discovers and stages the complete project (candidate in the managed
area, validated with the exact invocation including -p), apply promotes it
transactionally with a local-modification refusal keyed to manifest hashes,
legacy v2 pending blobs migrate conservatively, delete becomes an async
detach/export contract, stack deletion and create-rollback reap the managed
area, the boot sweep restores crashed promotions under the per-stack lock,
and rollback readiness discloses the partial-revert scope for Git-managed
stacks. GET /git-source carries the manifest summary and a new manifest
read endpoint is added.

* feat(git): surface the managed-project manifest in the Git source panel

Adds a collapsible manifest summary (pinned revision, managed/unmanaged/
refused counts, lazy-fetched input inventory with role chips, refusal
callout, migration banners), a refusal callout in the pull diff dialog, the
detach-and-export confirm copy, and the rollback partial-revert note in the
rollback readiness section.

* test(git): e2e coverage for complete-project materialization

Adds a local smart-HTTPS git server (e2e/gitServer.helper.ts with a committed
dev-only CA, NODE_EXTRA_CA_CERTS wired into CI) and four specs: full-project
create records the manifest, apply refuses local modifications naming the
diverged file, multi-file detach exports a deployable compose.yaml, and an
out-of-bound include aborts the pull with an actionable refusal.

* fix(git): harden the materialization transaction and crash recovery

Review-driven hardening: promotion now writes the manifest only after the
candidate rename (every crash window leaves the old manifest on disk, so the
sweep restores correctly), the promotion marker is atomic and a corrupt marker
flags migration_required instead of reading as a clean slate, restore rewrites
the manifest file and keeps the marker on partial failure, stale cleanup fails
the promotion instead of recording false tombstones and handles directories,
generation retention is previousDir-explicit, include/extends shared graphs
dedupe instead of false-cycling, the discovery read callback is containment
and size bound, sync_env owns the stack-root .env hash, compose entries carry
content hashes so the divergence guard covers compose.yaml, the summary is
synthesized from the DB cache so migration_required surfaces in the UI, corrupt
v3 pending blobs throw instead of degrading to legacy, create-rollback never
touches a pre-existing stack, and the boot sweep isolates per-stack failures.

* fix(git): byte-exact promotion, sync-env ownership, and render/marker hardening

Audit-driven corrections: candidate files are written byte-exact (Buffers
through the guarded FileSystemService write paths, size bound on stat.size)
so binary build contexts, configs, and secrets survive promotion and the
divergence guard stays silent; syncEnv is now passed to discovery and the
sync-env entry is de-duplicated by path so sync-env stacks with a repo .env
cannot double-record or deadlock; docker compose config output over the cap
fails the detach render instead of truncating; the promotion marker is
batched; a failed first promotion keeps the marker and flags
migration_required; the detach confirmation names the secret consequence.
Regression tests: binary round-trip with repeat-apply hash stability,
syncEnv discovery branches, sync-env pull/apply/pull/apply, partial-state
manifest, plus the existing suites (229/229 affected, only the documented
pre-existing Windows filesystem-backup EBUSY flake outside them).

* fix(git): exact-generation restore, context file ownership, dockerfile rebase, detach finality

Audit round 2 corrections: restore removes paths a failed promotion introduced
(exact prior generation, first-promotion failures clean the partial set and keep
the marker); build contexts are file-granular (per-file hashes in the manifest,
divergence guard covers context subtrees, files removed upstream are cleared on
promotion); explicit dockerfiles resolve relative to their build context with
in-repo ../ forms materialized as managed inputs; repo-root contexts no longer
double-copy managed files; detach removes auto-discovered override files so the
flattened model is final; lint errors fixed. Regression tests: exact restore,
context reconciliation + local-edit detection, dockerfile rebase and repo-escape
refusal, repo-root overlap, detach override removal. 213/213 affected backend
tests.

* fix(git): audit round 3: root-context normalization, build-service identity, Docker ignore rust, deep manifest validation, exact-set restore, detach atomicity, CRLF normalization

B-1: introducedPaths helper computes the exact file set a failed promotion would
leave (top-level + context files); restore removes introduced paths for an exact
prior generation; sweep accepts the incoming inventory for crash-window recovery.

B-2: repo-root context (build: .) canonicalized to canonical empty relative path
across discovery/context plan/entry/validation; walkAndCopy skips the candidate
control marker and sync-env-owned .env so root contexts never copy Sencho metadata
into the live stack dir.

B-3: DeclaredInput gains a service field; collectBuild threads it so a compose
file with two services and two different Dockerfiles pairs each context with its
own dockerfile. Additional contexts never inherit the service dockerfile.

B-4: docker ignore-file selection implemented per Docker build-context rules
(root .dockerignore, with Dockerfile-specific <name>.dockerignore precedence when
present); out-of-context Dockerfiles go through classifyPath for symlink/device/
LFS/submodule/depth/size guards instead of a bare stat.

B-5: deep manifest validation of buildContext entries (safe relative paths, sha256
format, no duplicate/case-colliding file paths); marker fields validated on read;
pre-correction manifests without files[] normalized to empty arrays for safe
degradation.

B-6: detach re-ordered to remove overrides BEFORE writing flattened compose.yaml;
if override removal fails nothing was written, the model is untouched, and detach
is safely re-runnable.

S-1: ComposeService.ts LFs normalized to repository convention.

All 213 affected backend tests pass; tsc + lint clean both sides.

* fix(git): audit round 4: root-context safety, Docker ignore wiring, marker-based exact restore, detach ordering, shared-input dedup, deep validation

B-1: the promotion marker now carries introduced paths computed from the incoming
manifest during promotion; boot recovery uses them for exact-generation restore
regardless of whether the incoming manifest is still available. `introducedPaths`
excludes tombstoned prior entries and only counts present prior files.

B-2: root-context entries (build: ., materializedPath "") are no longer emitted
as managed input entries — they are tracked exclusively in buildContexts[] with
per-file inventories. `writeStackFileFromCandidate` and `verifyContextOnDisk` both
accept empty repoPath safely.

B-3: Dockerfile-specific .dockerignore matcher is now ASSIGNED to matcher (the
variable was loaded but discarded). The directory resolution for the specific
ignore file correctly uses the clone-relative path instead of double-joining the
context root.

B-4: detach now writes the flattened compose.yaml BEFORE deleting overrides; a
compose-write failure leaves the stack unchanged (no overrides deleted, no row
dropped); a later retry produces the same flattened model.

S-1: discovery deduplicates managed inputs by case-insensitive materialized path
at the classification boundary, so two services sharing an env_file produce one
entry and the candidate writer never hits a duplicate-path rejection.

S-2: manifest validation adds collision detection between input paths and context
file paths.

All 213 affected backend tests pass; tsc clean; lint 0 errors.

* fix(git): audit round 5: root-context collision fix, marker-based recovery, Docker ignore root case, context divergence, detach rollback

B-1: root-context files that collided with managed input paths (compose.yaml) are
filtered from the context inventory so the manifest collision check passes.

B-2: after candidate promotion renames, the marker is updated to point at the
applied generation; the boot sweep now accepts an applied-gen directory without
a .candidate-complete marker as valid for recovery.

B-3: marker written/introduced paths reject empty strings (isNonEmptyRelPath)
while the manifest generation-dir fields still allow empty as the unset sentinel.

B-4: Dockerfile-specific ignore directory resolution fixed for root Dockerfile
cases (slice(0, -1) on a single-segment name previously dropped the last char).

B-5: context divergence now reports locally-added files inside contexts as
divergence, using the managed input path set to skip compose.yaml/.env/files
that have a non-context owner.

B-6: detach now snapshots the prior compose.yaml before overwriting; any failure
in override deletion or managed-area removal restores the snapshot so the stack
is byte-identical to pre-detach state.

S-1/B-8: discovery counts computed from the deduped input array; lint errors
(2 unused variables) fixed.

All 213 affected backend tests pass; tsc clean; lint 0 errors.

* fix(git): audit round 6: root-context promotion, recovery ordering, context-file merge, divergence precision, detach rollback

B-1: root-context files now explicitly promoted from the candidate via a
context-file loop after the managed-input promotion step. Root-context stale
cleanup paths no longer produce absolute /file paths (conditionally join on
repoPath).

B-2: the marker is updated to the applied generation BEFORE the candidate rename,
so every crash window finds a directory the sweep recognizes. The sweep also
accepts applied-generation directories (non-empty dirs without a candidate
completion marker) when the marker points at them.

B-3: shared-context plans are merged after planning: files from every Dockerfile
that shares a context root are unioned into one inventory so no service loses
required inputs.

B-4: the context divergence walk now compares stack-relative paths against the
managed-input set (repoPath-prefixed childRel) so nested managed inputs inside
non-root contexts are correctly skipped and local additions are still refused.

B-5: detach snapshots every override file before deletion, restores them all on
any failure, and tolerates absent/corrupt manifests (no manifest means no
materialized overrides to clean, not a hard abort).

All 213 affected backend tests pass; tsc clean; lint 0 errors.

* fix(git): audit round 7: root Dockerfile containment, inventory-driven context copy, file-only marker recovery, detach transaction

B-1: root-context Dockerfile containment check fixed for root contexts
("" or "."). Any repo-relative Dockerfile without ../ is in-context.

B-2: context copy now reads from the plan inventory (plan.context.files)
instead of re-walking the source with the first matcher. Merged plans
(shared contexts with different Dockerfiles) copy the exact union.

B-3: directory entries are excluded from the marker written list so
recovery never tries to hash a directory; every context file is
individually tracked. Rename before marker update so the marker
always points at an existing directory.

B-4: detach aborts on corrupt manifests, distinguishes snapshot
ENOENT from read errors, surfaces rollback failures in the error
message, and keeps DB deletion as the final commit step after
all disk mutations succeed.

All 213 affected backend tests pass; tsc clean; lint 0 errors.

* fix(git): sanitize log messages and fix CodeQL log-injection finding

The one genuine CodeQL alert (log-injection + format-string at line 1002)
is resolved by wrapping stackName with sanitizeForLog(), matching existing
precedent in ComposeService.ts and routes/stacks.ts. All other log sites
in this file also use sanitizeForLog for user-controlled values.

* fix(git): enforce context bounds after shared-context merge

The merged context plan union can exceed GITSOURCE_MAX_BUILD_CONTEXT_BYTES
even when each individual plan fits. Recheck the cap against the unionized
inventory after merging.

* fix(git): harden materialization recovery

* fix(git): audit round 8 - invocation-faithful discovery, safe promotion, redacted manifest API

B-1: an omitted build context now defaults to the declaring file's project
directory, and build-secret long syntax parses source as a top-level secret
name instead of a file path, so valid projects no longer refuse or fail to
build.
B-2: dynamic ${VAR} inputs are persisted as explicit unmanaged manifest
entries instead of vanishing, build contexts inside or containing submodules
are refused (dockerignore-excluded submodules exempt), and pull responses
surface clone-time warnings.
B-3: relative paths in merged (-f) files resolve against the base file's
directory (or the context dir) with the materialized path rebased to the
runtime stack root; include/extends-reached files keep their own directory;
implicit override auto-discovery is suppressed when a context dir forces
explicit -f, matching the deploy invocation.
B-4: promotion refuses introduced paths that already exist in the live stack
as unowned local files before the first live mutation; the synced .env and
fresh-stack creation stay exempt.
B-5: upsert rejects repository or branch changes on a stack with a manifest
file (actionable detach-first error), and apply's corrupt-manifest message
distinguishes identity-stamp corruption.
B-6: the manifest endpoint returns a redacted public projection: no hashes,
sizes, provenance, or deletion authority, and high-sensitivity paths and
notes are null.
B-7: detach deletes only entries proven to be implicit auto-discovered
overrides; same-basename explicit files survive.
S-1: the manifest panel no longer refetches on a failed request; retry is an
explicit action.
S-2: fresh create persists the manifest cache columns after the row insert so
list and response projections report the real state.
S-3: git-sources.mdx matches the corrected detach, submodule, and dynamic-path
behavior.

* fix(git): align GitSourcePanel manifest fixture with the public projection; exclude guarded manifest service from CodeQL path-injection

The panel test fixture still used the internal manifest shape; with the
redacted public projection the label fell back to the dependency kind and
duplicated the badge. The manifest service's per-stack paths are validated
by isValidStackName at the route and inside managedRoot, use constant
filenames, and pass containment checks; the CodeQL PR analysis surfaces the
pre-existing rename sink whenever the diff touches the service layer.

* fix(git): restore ComposeService.ts line endings to the base convention

The file was committed with CRLF at the PR base; a round-3 commit
normalized it to LF, making the base-to-head diff show 1,412 additions
and 1,322 deletions for ~90 substantive lines. Restoring CRLF collapses
the diff to the functional changes only.

* fix(git): remove ineffective CodeQL source-path exclusion

query-filters match query metadata, not analyzed source locations, so the
file-scoped js/path-injection exclusion added in round 8 had no effect.
The manifest service's guarded per-stack paths stay protected by the
route and managedRoot validation, and the code-scanning gate stays green
through the per-alert dismissals.

* fix(git): audit round 9 - runtime path equivalence, complete input grammar, pre-manifest adoption guard, redacted refusals

B-1: the introduced-path collision guard now runs unconditionally with an
explicit adoption policy: 'all' for fresh creation, the legacy-ownership
allowlist (applied compose files + synced .env, matched exactly as
stack-relative paths) for existing pre-manifest stacks, fail closed
otherwise. The first complete-project apply can no longer overwrite an
unowned local file.
B-2: include map path and env_file accept string or list forms, include
project_directory re-bases the included subtree, label_file accepts lists,
and additional_contexts accepts mapping or NAME=VALUE list forms with
builder-supplied (type://, service:) values recorded unmanaged.
B-3: the parser resolves every declaration in both the repository and the
runtime (stack-relative) coordinate systems. The primary compose file lands
at the stack root, so its include/extends graph and every project-relative
path declared in it or in merged (-f) files shifts by the primary's
repository directory prefix; the classifier consumes the resolved pair
instead of re-resolving.
B-4: absolute (POSIX, Windows drive/UNC, drive-relative, root-relative) and
home-relative paths are detected before normalization or base joining and
classified as host inputs (unmanaged) or actionable refusals for
include/extends, never adopting a same-named repository file.
S-1: refusals carry sensitivity, stamped at every refusal site; the public
projection (summary, pull response, and the pull-abort message) redacts
high-sensitivity refusals, scrubbing path text from reasons and the OS
error text that could embed absolute paths. Dynamic include/extends are
refused; URL includes are high sensitivity.
S-3: ComposeService.ts line endings restored (separate commit).
S-2: invalid CodeQL source-path filter removed (separate commit).

* chore: bump nanoid to 3.3.18 via npm audit fix

The nanoid advisory GHSA-2v37-7h3g-55p8 (high) covers <3.3.17 and was
published after the last green CI run; both lockfiles pinned 3.3.16.
npm audit fix bumps the transitive dependency to 3.3.18.

* fix(git): audit round 10 - included-project envs, project-base includes, optional inputs, drive-letter binds

B-1: every included project's default interpolation .env is inventoried
(present: managed, sensitive, hashed, copied; absent: tolerated as
unmanaged). interpolation: false and same-base includes skip the entry.
B-2: include, include-env, and extends.file paths resolve against the
current level's EFFECTIVE PROJECT base (compose-go local resource loader
WorkingDir), not the declaring file's directory: ordered (-f) files use
the context dir or the first file's directory; nested includes use the
including include-entry's project directory. Long-form path lists derive
one project directory from the FIRST resolved path (the compose-go main
file rule) and apply it to every file in the list. Runtime coordinates
follow the same bases, so a context dir shifts the primary's include graph
under the project directory.
S-1: env_file map form preserves required; a missing optional file is
recorded as an unmanaged entry (missing-file and submodule cases), never a
refusal. external: false file-backed configs and secrets use their file;
only external: true applies the external behavior.
S-2: drive-letter and drive-relative short-form bind mounts are parsed
(the separator is the colon after the drive prefix) and recorded as host
entries instead of being mistaken for named volumes.
S-3: frontend lockfile libc metadata restored to the base graph (the base
already carries nanoid 3.3.18).
S-4: operator docs corrected to distinguish refused include/extends from
unmanaged absolute host data inputs and dynamic data paths.

* fix(git): audit round 11 - boot sweep data-loss guard, honest manifest summary, dead refusal UI removal

B-1: the boot orphan sweep no longer treats a failed or empty stack listing
as 'every stack is gone'. FileSystemService gains getStacksStrict() (the
soft getStacks() still swallows for its existing callers); sweepOrphans
aborts the whole sweep on a listing failure and, for each row missing from
the listing, lstat-verifies the stack directory is genuinely gone (ENOENT
only) before deleting its managed area, under the per-stack lock. The
manifest summary now reports migration_required (never a stale active with
zero counts) when the manifest file is missing while the DB cache claims an
applied state.
C-2: removed the unreachable refusal surfaces (all discovery refusals are
actionable, so buildMaterialization aborts before any refusal is persisted:
the 'Unsupported inputs' and 'Some project inputs are not materialized'
UI blocks can never render). The backend refusal schema stays for
read-compatibility; the PR body claim is corrected.
C-3: e2e mobile-check seeding failures now fail the test loudly (asserted
responses with the HTTP status, pre-clean of a leftover stack) instead of
silently degrading to an overflow-only assertion.

* fix(e2e): seed mobile-check from the local fixture git server

The seed pointed at docker/awesome-compose.git with compose_paths
['compose.yaml'], but that repository has no root compose.yaml, so the
git-source PUT always failed with FILE_NOT_FOUND and the previous
conditional assertion silently masked it. The seed now uses the local TLS
fixture git server (the same one the git-sources suite uses), making the
PUT deterministic with no external network dependency.

* fix(git): isolate monorepo overrides and harden materialization errors

Scope implicit compose.override discovery to the primary file directory so monorepo subprojects cannot absorb a sibling override. Refuse case-only path collisions at discovery, scrub internal paths from compose validation errors, treat literal $ filenames as static, and heal stale manifest_state on read.
2026-08-10 17:12:55 -04:00
Anso c613010199 feat: account for VM memory ballooning in host memory reporting (#1750)
* feat: account for VM memory ballooning in host memory reporting

Extend hostMemory.ts with a readBalloonedMemory() function that parses the
Balloon: field from /proc/meminfo, following the same fail-open pattern as
the ZFS ARC integration. When a nonzero balloon is detected, effective
memory fields (effectiveUsed, effectiveFree, effectiveUsagePercent) are
computed and exposed through /api/system/stats and /api/fleet/overview.

All consumers that derive meaning from host memory now prefer effective
values when present: the dashboard gauge, Fleet card RAM bar, mobile
views, health verdict, health status bar stat tile, and host RAM alerts.

Backward compatible: missing /proc/meminfo or absent Balloon: line
preserves exact current behavior. Old remote nodes without the new fields
continue rendering normally.

* refactor: extract shared helpers for balloon memory wiring

Extract readCandidateFile() and logSelectedPath() in hostMemory.ts to
deduplicate ARC and balloon file-read logic. Add memoryToWire() to
centralize the optional-field spread used by /api/system/stats and
/api/fleet/overview. Add getNodeMemUsed()/getNodeMemTotal() helpers
in nodeUtils.ts for frontend byte-text consumers.

* fix: make desktop fleet masthead aggregate balloon-aware

The desktop fleet overview's memory aggregate in useFleetOverview.ts still
summed raw memory.used, while the mobile fleet aggregate and per-node cards
already used effective values. Update to use getNodeMemUsed/getNodeMemTotal
helpers.

* fix: revert balloon adjustment from alerting and health decisions

Ballooned memory is host-reclaimed (unlike ZFS ARC, which the guest can
reclaim on demand). The guest cannot get ballooned pages back until the
hypervisor deflates them, so treating ballooned memory as available for
alerting or health can mask real memory pressure.

Keep balloon parsing, wire fields, and the dashboard context line as
informational-only. The memory gauge, health verdict, and host RAM alerts
now use the standard ARC-adjusted working-set percentage regardless of
balloon. Updated configuration.mdx and dashboard.mdx to document that
balloon data is informational and does not influence alerting.
2026-08-02 20:48:38 -04:00
Anso 381ed2a91f feat: add Admiral Hardened Build channel and business assurance surfaces (#1629)
* feat: add Admiral Hardened Build channel and business assurance surfaces

Introduce Studio Saelix entitlement-backed Hardened Build switching, a
single-flight image operation coordinator, Recovery Vault naming, Admiral
Account settings, and typed Fleet update failures while preserving Community
custom-repo and targetless pull-current updates.

* fix: harden image-op paths and clear CI CodeQL/pilot flake

Validate operation IDs before filesystem use, use hostname checks in Fleet
fetch mocks, sanitize registry probe logs, and swallow expected TCP teardown
errors in the pilot reverse-route post-handshake test.

* fix: sanitize image-op docker config write and probe logs

Allowlist-copy registry host keys and base64 auth before writing the
temp DOCKER_CONFIG, and log registry probe failures with a fixed message
so CodeQL no longer flags network-to-file and log-injection mediums.

* fix: address Admiral Hardened Build audit blockers

Expose imageChannel so hardened Fleet peers still POST for typed rejection, claim community updates before 202, terminalize helper failures, gate Hardened on paid, and align support/docs/e2e wording.

* fix: terminalize image ops on helper survival and aborted claims

* fix: prevent recreating persist from overwriting helper-exit failure

* test: assert helper-exit failure lands before recreating persist

* fix: keep current pointer when acknowledging a stale image operation
2026-07-14 10:47:54 -04:00
Anso 9b3f5c5a90 docs: relicense Sencho to AGPLv3 and reframe Community positioning (#1623)
* docs: relicense Sencho to AGPLv3 and reframe Community positioning

Replace BSL with AGPLv3 for the public Community product, update contributor
and licensing copy for Community-focused contributions, and surface source
and license links in Settings About.

* docs: clarify CLA scope and Community contribution framing

* fix(ui): split About link constants and harden AboutSection test selectors
2026-07-13 03:49:29 -04:00
Anso 98667e0d6f fix: make host memory usage ZFS ARC-aware (#1547) 2026-07-01 23:08:49 -04:00
Anso d369b03a38 feat: detect stalled stack updates and add in-app recovery actions (#1347)
* feat: detect stalled stack updates and add in-app recovery actions

Add a backend idle-output backstop that stops a deploy/update compose step
that has gone silent (SENCHO_COMPOSE_STALL_TIMEOUT_MS, default 10m), so a
hung image pull surfaces a fast failure instead of spinning indefinitely.

Surface failed, timed-out, and stalled operations with recovery actions on
the stack page: a desktop chip plus popover menu and an inline mobile card
offering retry, restart, roll back (when a backup exists), refresh state,
and copy diagnostics, all gated by deploy permission. The streaming
deploy/update progress modal is now on by default and warns when output
goes quiet. Container state is refreshed after a failed or stalled
operation, and the UI never sits in an indefinite spinner.

* fix: harden rollback against policy-blocked file mutation and refine recovery

Address review findings on the stalled-update recovery work:

- The rollback route restored backup files before running the policy gate, so
  a policy-blocked rollback could leave the on-disk config rolled back while the
  deployed containers were unchanged. Snapshot the current files first and
  revert them when the gate blocks; if that revert itself fails, escalate it on
  the persistent alert feed since the 409 is already sent.
- Refresh container state after a successful manual rollback (rollback
  redeploys), without mis-recording a refetch failure as a rollback failure.
- Suppress the stalled-output warning once live progress is unavailable.

* test: mock snapshotStackFiles in the atomic-deploy rollback route tests

The rollback route now snapshots stack files before restoring a backup, so its
FileSystemService mock needs snapshotStackFiles. Without it the mocked call
threw and the route returned 500, failing the success-path rollback assertions.
2026-06-10 10:12:24 -04:00
Anso b2cae92a92 docs(mesh): note admin-only management, dev-mode diagnostics, and the subnet env var (#1288)
- Note that managing the mesh (the per-node toggle and stack opt-in or
  opt-out) requires an administrator and that non-admin users see the
  Routing tab read-only, matching the access model the tab enforces.
- Document the [Mesh:diag] developer-mode logs in the Diagnostics section
  so operators can trace routing decisions and operation timing.
- Add SENCHO_MESH_SUBNET to .env.example; it was documented on the feature
  page but missing from the env template.
2026-06-02 16:13:15 -04:00
Anso 2844f606cd fix(git-sources): harden webhook delivery, transport errors, and clone limits (#1249)
* fix(git-sources): harden webhook delivery, transport errors, and clone limits

Map webhook-pull outcomes to real HTTP status codes (200 success, 202
debounced, 404 no source, 422 failure) instead of always returning 200, so a
Git provider and any monitoring on it can tell when a delivery actually failed.

Close a concurrent webhook fan-out gap: the debounce window is now re-checked
inside the per-stack lock, so simultaneous deliveries for one push run a single
clone instead of one per request. The whole pull/apply critical section runs
under a single lock acquisition.

Unwrap fetch transport causes (ENOTFOUND, ECONNREFUSED, ECONNRESET, TLS) so a
clone failure surfaces an actionable, host-qualified message instead of a bare
"fetch failed".

Cap how many bytes a single clone may download to protect the host disk;
operators can tune it with GITSOURCE_MAX_CLONE_BYTES (default 100 MB).

Log webhook pull failures server-side, since the webhook path is unattended.

* test(git-sources): assert surfaced host via toContain to satisfy CodeQL

* fix(git-sources): bound per-file read, treat debounced webhooks as non-failure, correct clone-cap docs

* docs(git-sources): correct clone-cap comment to describe a download bound, not disk
2026-05-29 09:43:37 -04:00
Anso 982c7830b1 fix(mesh): address audit follow-ups (early-data, recompose, admiral gate, error codes, docs) (#1126)
* fix(mesh): buffer early TcpData on reverse-relay path

The reverse-relay code dropped the local-shaped reservation and any
TcpData frames buffered in it before acceptReverseRelay had wired up
the target stream. A peer that sent a request body immediately after
tcp_open_reverse lost those bytes when the target lived on a third
node, since reverse_local buffers them but reverse_relay did not.

Carry the reservation through acceptReverseRelay: transplant its
pendingData and pendingBytes into the new reverse_relay state, gate
TcpData writes on targetOpen, and flush buffered frames into the
target stream before sending tcp_open_ack. Mirrors the reverse_local
pattern. Regression test fires tcp_open_reverse plus an immediate
TcpData while ensureBridge is in flight and verifies the bytes
arrive intact, in order, before the ack.

* fix(mesh): recompose affected stacks when node-level mesh is disabled

disableForNode used to clear DB rows and override files but leave the
running containers attached to sencho_mesh with stale /etc/hosts
alias entries until an operator redeployed every stack by hand.

Mirror optOutStack: after the existing alias/forwarder cleanup, call
regenerateOverridesAcrossFleet, cascadeRecomposeAcrossFleet, and
triggerRedeploy for each previously meshed stack on the disabled
node so containers detach from sencho_mesh and shed the alias
entries they owned. The disabled node's mesh_stacks rows are deleted
before the cascade so listMeshStacks returns the right set with no
skip tuple required. Route threads the actor through actorFor(req)
for parity with optInStack/optOutStack. Tests cover the redeploy
fan-out, the cascade no-skip-tuple invariant, and the default actor
fallback for non-route callers.

* fix(mesh): require Admiral on the WS proxy-tunnel upgrade

HTTP mesh routes in routes/mesh.ts all enforce requireAdmiral, but
the /api/mesh/proxy-tunnel WS upgrade accepted any node_proxy or
full-admin api_token regardless of the receiver's license. A node
downgraded from Admiral kept serving mesh data-plane traffic to a
sibling central while refusing every mesh management call.

Read the receiver's local LicenseService at the upgrade and 403 when
the tier is not paid+admiral. The check sits after the existing
credential gate and uses LicenseService directly rather than
effectiveTier (which trusts forwarded proxy headers); a remote peer
dialing in cannot be trusted to assert our entitlement. Dialer and
node_proxy token format are unchanged. Three regression tests cover
community-tier node_proxy, skipper-tier node_proxy, and
community-tier full-admin api_token all rejected with 403.

* fix(mesh): handle no_target alongside push_failed in inspectStackServices

proxyFetch throws MeshError('no_target') when getProxyTarget returns
null (pilot tunnel offline, proxy bridge unreachable), but the
inspectStackServices catch branch only matched push_failed. Offline
remotes fell through to the generic 'remote unreachable' error log,
which the Routing tab surfaces as an unexpected fault.

Match both error codes and emit the operator-friendly warn message
that names the unreachable node and the error code. Regression test
spies on console.warn/console.error to pin the branch.

* docs(mesh): align env defaults and forwarder comments with current architecture

SENCHO_MESH_PROXY_TUNNEL_IDLE_MS in .env.example carried the old
five-minute idle-close value (=300000), but the code default is
DEFAULT_IDLE_TTL_MS=0 (persistent tunnel). Copying the example
silently reintroduced the idle-close behavior the dialer removed.

MeshForwarder.ts's leading docblock and inline listen comment still
described host-network mode plus extra_hosts: host-gateway as
required for forwarder reachability. Sencho runs in standard bridge
mode and attaches to the shared sencho_mesh network at a stable IP;
meshed user containers reach the forwarder by that IP directly.

Flip the env default to 0, rewrite the env comment to describe the
persistent behavior and the opt-in for idle teardown, and rewrite
both forwarder comments to match the bridge-network reality.

* fix(mesh): trust forwarded tier on proxy-tunnel WS and remove remote overrides on disable

Admiral entitlement on the WS data plane now follows the same trust model
as the HTTP mesh routes: the central asserts its tier via x-sencho-tier
and x-sencho-variant on the WS handshake, and the receiver trusts those
headers only when the upgrade carries a node_proxy credential. When no
headers are present or the credential is a full-admin api_token, the
receiver falls back to its own local license. Without this an Admiral
central could be rejected by a Community remote and a Community central
could dial a locally-Admiral remote.

disableForNode now routes through removeOverrideFromNode so override
files pushed earlier via applyLocalOverride are removed on remote nodes
via DELETE /api/mesh/local-override/:stack. Sequential awaits are wrapped
in Promise.allSettled to match regenerateOverridesForNode's parallel
push pattern.

Other changes:
- Cover the post-state-swap buffering window in the reverse-relay test
  (TcpData arriving after openTcpStream returns but before target open).
- Refresh stale MeshService comments that still referenced the removed
  sidecar layer and host-network listener model.

* test(mesh): pin removeOverrideFromNode remote HTTP shape

The disableForNode regression test mocks removeOverrideFromNode itself,
so a regression inside the helper would not be caught. Add a narrow
contract test that spies on global fetch and asserts the request shape:
DELETE /api/mesh/local-override/:stack against the resolved proxy
target, with Authorization Bearer plus the x-sencho-tier and
x-sencho-variant headers. Also covers the encodeURIComponent path and
the swallowed-network-error behavior the disable cascade relies on.

* test(mesh): use createTestApiToken helper in proxy-tunnel api_token gate test

The full-admin api_token branch of the WS Admiral gate test inlined the
canonical generateApiToken + sha256 + addApiToken triple that already
lives in the createTestApiToken helper. Switching to the helper removes
the duplicated insertion logic and aligns this test with the helper used
by the other api_token call sites.
2026-05-20 12:53:29 -04:00
Anso 282ab8d844 fix(pilot): let SENCHO_PUBLIC_URL override the request Host in enrollment (#1122)
The enrollment minter inferred SENCHO_PRIMARY_URL from the request Host
header, which baked loopback or LAN addresses into the compose YAML when
the admin opened Add Node on the central's own machine. Pilots on a
different network (a public cloud VPS, for example) cannot dial that.

SENCHO_PUBLIC_URL on the primary now wins when set and well-formed
(http(s)://, no loopback). Trailing slashes are stripped. Falls back to
the request Host when unset or invalid.
2026-05-20 03:35:25 -04:00
Anso 0117556bea fix(fleet): rename Traffic tab label to Routing for consistency (#1119)
The Fleet view sub-tab that renders RoutingTab.tsx was labeled "Traffic"
while every adjacent identifier already used "Routing": the backend
route file (backend/src/routes/mesh.ts), the component path, the
localStorage key (sencho-routing-view-mode), the SegmentedControl aria
label ("Routing view mode"), and the engineering vocabulary across the
codebase. Operators looking for the "Routing tab" could not find it
because the visible label said something else.

Align the user-visible label with the rest of the implementation and
update the two doc references that named the tab by its old label
(docs/features/fleet-actions.mdx, .env.example).
2026-05-19 23:25:30 -04:00
Anso a38a3e0226 feat(mesh): route mesh traffic over Distributed API remotes (#1048)
* refactor(mesh): extract shared TCP stream switchboard

Pull the `tcp_open` / `tcp_open_ack` / `tcp_open_reverse` / `tcp_close`
+ `TcpData` handling out of the pilot agent into a reusable module so a
second caller (the upcoming proxy-mode WS handler) can run the same
frame parser, stream allocator, idle timers, and Compose-label
resolver. One parser, two callers, zero drift on a
security-sensitive protocol surface.

The pilot agent keeps its public `openMeshTcpStream` API and delegates
to a per-connection switchboard constructed in `connect()` and torn
down in `cleanupAfterDisconnect`. Existing reverse-stream and resolver
tests are rewritten to exercise the shared module directly.

* feat(mesh): route mesh traffic over Distributed API remotes

Sencho Mesh now works against remotes in Distributed API mode in
addition to Pilot Agent mode. Central opens a short-lived WebSocket
tunnel to the remote's new `/api/mesh/proxy-tunnel` endpoint on demand
and tears it down after 5 minutes of idle (configurable via
`SENCHO_MESH_PROXY_TUNNEL_IDLE_MS`). Mesh dispatch is mode-agnostic at
the routing layer; `PilotTunnelManager.ensureBridge` resolves an
existing tunnel or asks the new `MeshProxyTunnelDialer` to dial.

The Routing tab badges now use a `reachableMode` classifier: `★ Local`
for local, `pilot offline` red badge for a pilot with a down tunnel,
and a new `unreachable` red badge surfacing the specific reason
(missing token, scope not full-admin, TLS failure, remote does not
support proxy mesh). Distributed API remotes with valid credentials
show no negative badge; the tunnel opens on first dial.

Auth: the proxy-tunnel WS upgrade requires an `Authorization: Bearer`
API token with the `full-admin` scope. Lower-scoped tokens are
rejected at upgrade time so a leaked read-only token cannot reach the
mesh data plane.

Bidirectional: the proxy-mode WS handler registers itself as the
local `MeshService` reverse dialer (compare-and-swap), so meshed
containers on a Distributed API remote can dial cross-node aliases
via `tcp_open_reverse` over the same tunnel. Cross-node relays
(`PilotTunnelBridge.acceptReverseRelay`) await `ensureBridge` so
proxy-to-proxy mesh works without standing tunnels.

* docs(mesh): describe Distributed API mesh and unreachable troubleshooting

Refresh the user-facing mesh documentation to reflect that mesh works
over both Pilot Agent and Distributed API remotes. Adds a
Troubleshooting accordion entry for the new `unreachable` Routing tab
badge so operators can match a tooltip reason ("api token rejected
(scope must be full-admin)", "remote does not support proxy mesh",
"TLS handshake failed", "api_url not set", "api token missing") to a
concrete fix.

* refactor(mesh): apply /simplify review findings

Five cleanups surfaced by the post-implementation code review pass.
No behaviour change for fleets running on `main`; only internal
structure improves.

- **Deterministic container IP shared.** Extract the compose-default
  network preference (`pickContainerIp`) and the conventional-name
  fast path (`lookupContainerIp`) into `backend/src/mesh/containerLookup.ts`.
  Both `MeshService.resolveContainerIp` (existing same-node fast
  path) and the switchboard's `resolveByComposeLabels` (proxy/pilot
  inbound dial) now use the same logic. Earlier the switchboard
  helper grabbed the first `Object.values(Networks)` IP, which
  could flip across daemon versions on multi-network containers;
  fixed.
- **WS URL upgrade extracted.** New `backend/src/utils/wsUrl.ts`
  exposes `httpUrlToWs(baseUrl)` that maps `http://` to `ws://` and
  `https://` to `wss://`. Used in both `pilot/agent.ts` and the
  proxy-tunnel dialer. The previous inline `replace(/^http/, 'ws')`
  silently downgraded `https://` to `ws://` (cleartext).
- **`computeReachable` takes the pre-fetched node.** `getStatus`
  already iterated `db.getNodes()`; the helper previously re-queried
  by id per node (N+1 reads). Pass the row in.
- **Reachable-reason ternary -> const map.** Replace the nested
  ternary in `MeshService.computeReachable` with a
  `Record<DialFailureCode, string>` lookup keyed on the dialer's
  failure code.
- **Activity-type ternary -> const map.** Same flattening inside
  `MeshProxyTunnelDialer.logActivity`.

All mesh tests pass (85/85). Full backend suite green (2126/2126).

* fix(mesh): cache proxy dial failures and contain WS handshake errors

`ensureBridge` now consults the recent-failure cache before dialing so a
continuous mesh workload against a misconfigured proxy-mode remote does
not produce one upgrade attempt per cross-node TCP open. `recordFailure`
emits at most one activity-log entry per cache window per (nodeId, code)
so a connect-loop on a single bad remote cannot flush the ring buffer.
Failure messages run through `redactSensitiveText` before reaching the
log so any embedded Bearer / JWT / inline-URL credentials are scrubbed.

`stop()` clears the inflight map and `dial()` checks `this.stopped`
between awaits so a shutdown does not leak a half-opened bridge.

When `awaitOpen` rejects (401, 4403, TLS), the dialer attaches a noop
'error' listener before calling `ws.close()`. Without it the ws library
emits a tail 'error' on a still-CONNECTING socket that propagates as an
unhandled exception. Surfaced by a live-network test against a real
proxy-mode peer.

Adds `mesh-proxy-tunnel-live.test.ts` (skipped unless MESH_AUDIT_URL
and MESH_AUDIT_TOKEN_FILE env vars are set) covering both the happy
path and the auth-rejected path against an actual remote Sencho.

* feat(mesh): instrument proxy tunnel observability

Adds three counters to `PilotMetrics`:
- `proxy_bridges_total` (incremented on `registerProxyBridge` success)
- `proxy_dials_failed` (every failed dial attempt, not deduped)
- `proxy_idle_closes` (idle-sweep teardowns)

All three surface automatically via `GET /api/system/pilot-tunnels`
since the route returns the full `Counters` snapshot.

Gates two pre-existing always-on `console.warn` calls in
`tcpStreamSwitchboard.ts` (mid-stream socket errors and Docker resolve
failures) behind `isDebugEnabled()`. Both fire on per-stream events and
would otherwise violate the diagnostic-log safety rule under load.

Adds `mesh-tcp-stream-switchboard.test.ts` covering the forward
`tcp_open` path: real localhost dial, resolver errors (no_target,
denied), per-tunnel cap saturation, frame-routing fall-through
invariants, `tcp_close` socket teardown.

Drops `MESH_CONNECT_TIMEOUT_MS` from the public surface; only the
switchboard itself uses it.

* fix(mesh): tighten Routing tab unreachable handling

`RoutingNodeCard.tsx`: the **Add stack to mesh** button now disables
when `reachableMode === 'unreachable'`, matching the existing
TogglePill behavior. Without this, an operator on an unreachable node
could open the opt-in sheet, confirm, and watch the redeploy proceed
against a target whose mesh data plane will silently fail to route.
Also drops the leftover `status.nodeId !== -1` guard on the pilot-
offline badge.

`MeshService.ts`: renames `isMeshReachable` to `isMeshConfigured` with
the new predicate that returns true for proxy-mode remotes whose creds
are valid (the tunnel is opened on demand). `getRouteDiagnostic` now
distinguishes "routable" (configured) from "pilotLive" (live tunnel
state, only meaningful for pilot mode); without the split, every idle
proxy-mode route would report `tunnel down`.

`MeshService.setReverseDialer` warns when the unconditional install
path silently overwrites a non-null current dialer. By topology a
Sencho is either pilot or central, so the branch flags a misconfigured
deployment rather than an expected race.

Drops the dead `export { ReverseTcpStreamHandle }` re-export from
`pilot/agent.ts`; its only consumer imports straight from
`mesh/tcpStreamSwitchboard.ts`. Fixes a stale doc comment in
`MeshService.ts` that referenced the wrong source file.

Adds `mesh-proxy-tunnel-handler.test.ts` covering the WS handler
lifecycle: pilot-mode 404 rejection, post-upgrade reverse-dialer
install, concurrent-upgrade 1013 rejection (single-tenant slot), and
error-path teardown.

Updates the troubleshooting accordion in the user docs to mention the
disabled-state behavior.

* fix(mesh): close ESLint and CodeQL findings on the proxy-tunnel diag logs

Remove the unused `reverseDialer` local in `meshProxyTunnel.ts`; the
closure target is `localDialer` above and this assignment was always
dead. Strip line breaks inline on the three `MeshProxyTunnelDialer`
diag log lines so CodeQL `js/log-injection` data flow recognises the
sanitisation that `sanitizeForLog` already performs.

No behaviour change; the diag logs render the same characters they
do today.
2026-05-14 20:51:42 -04:00
Anso 0be479100c docs: v1 docs refresh (#966)
* docs(introduction): rewrite intro page and refresh screenshots

Rewrite the Getting Started introduction to reflect the current product:
adds the Mesh, Blueprints, Pilot, Fleet, Resources concepts up front;
restructures capability sections around Stacks, Fleet (now including
Fleet Actions), Mesh, Blueprints, Monitoring, Resources, Security,
Automation, and Pilot/Remote ops; cross-links every claim to the
matching feature page.

Replaces the dashboard hero shot with a fresh capture against the
redesigned cockpit chrome and adds three inline shots (running stack
with anatomy and logs, fleet command deck, resources hub with
reclaim header). All screenshots taken at 1920x900, dark theme,
with node names, usernames, IPs, and home paths neutralized.

Drops outdated claims: stale "190+ templates" count, the
"viewer accounts" RBAC summary, and the "atomic" deployment label
that did not match the actual rollback mechanism.

* docs(introduction): add Federation and Fleet Secrets

Federation has shipped (Admiral) with cordon and pin policy as
operator overrides on the blueprint reconciler. Fleet Secrets is
landing as a Skipper+ tab for versioned env-var bundles encrypted
at rest, with diff preview and target push.

Mirror those in the Run-one-machine-or-many bullet list and in the
Tiers paragraph so the introduction matches the current product.

* docs(introduction): re-shoot screenshots against v0.72.0 production

Re-capture all four introduction screenshots from the upgraded
production node so the fleet view shows the current full tab strip
(Overview / Snapshots / Status / Deployments / Traffic / Federation
/ Fleet Actions / Secrets) instead of the older Overview / Snapshots
/ Status only. Same 1920x900 dark-theme capture and the same PII
scrub applied (node names, usernames, IPs, home paths neutralized).

* docs(quickstart): rewrite around v0.72.0 cockpit and add screenshots

Replaces the bare install snippet with a five-minute walkthrough that
matches the redesigned UI. Leads with a docker-compose.yml block (the
bare docker run command is collapsed in an Accordion), keeps the 1:1
path rule, and adds two new sections that show the user what happens
on first boot.

Adds two screenshots at 1920x900 dark theme:
- setup-cold-start.png: the Cold start card with Username, Password,
  Confirm password fields, and the Initialize console button.
- dashboard.png: the post-sign-in dashboard captured against v0.72.0
  with the full top nav (Home, Fleet, Resources, App Store, Logs,
  Auto-Update, Console, Audit, Schedules) and the populated Stack
  health table sorted by load.

Where-to-next now uses CardGroup cols=2 to match the introduction
page's pattern.

* docs(configuration): align env var reference with current backend

Bring docs/getting-started/configuration.mdx up to date with the
v0.72.0 backend:

- Remove PORT from the optional env vars table. The listen port is
  hardcoded to 1852 in backend/src/helpers/constants.ts and is never
  read from the environment. Replace it with a Listen port section
  that explains the fixed port and host-port remapping.
- Document API_RATE_LIMIT (default 200) and API_POLLING_RATE_LIMIT
  (default 300), both applied in production only.
- Note the /app/compose fallback default for COMPOSE_DIR while still
  pointing readers at the 1:1 path rule.
- Point the SSO env var section at the new SSO Quickstart page and
  keep the SSO feature reference as the deeper dive.
- Tighten First boot and cross-link to Quickstart so the screenshot
  is not duplicated.
- Add a Where to next CardGroup matching the refreshed Introduction
  and Quickstart pages.

Drop stale PORT=1852 and JWT_SECRET=your-secure-jwt-secret-here
lines from .env.example so the example no longer contradicts the
docs (PORT is hardcoded, JWT_SECRET is auto-generated and persisted
to the database during initial setup).

* docs(configuration): replace em-dash-substitute hyphens in prose

Three sentences used ` - ` (space-hyphen-space) as an em-dash
substitute. Replaced with the punctuation that fits each case:

- "How Sencho organizes your compose directory": semicolon between
  the two related clauses.
- Data directory Warning: parentheses around the parenthetical
  insertion.
- 1:1 path rule explanation: comma before the contrastive clause.

Heading slugs and YAML code-block comments are unchanged.

* docs(sso): refresh setup guide and drop misleading "one-click" wording

Brings the SSO Setup Guide in line with how SSO actually works in the
current build, and corrects misleading copy across both SSO docs pages.

- Setup Guide: explains the env-var-seeds-once / DB-is-authoritative
  config model up front, replacing per-section "Restart Sencho"
  wording that implied a restart was always required.
- Setup Guide: promotes the per-provider Test Connection button out
  of the LDAP-only paragraph into a generic intro callout, and adds
  a self-signed LDAPS tip.
- Setup Guide: notes that all OIDC providers accept a *_DISPLAY_NAME
  override for the login button label, and adds a commented LDAP TLS
  toggle to the full compose example.
- Setup Guide: adds two screenshots of the redesigned Settings > SSO
  panel (overview + LDAP card expanded with form).
- Both pages: replaces "one-click presets" / "one-click configuration"
  with "preset providers". The Skipper-tier presets still require an
  OAuth app provisioned in the provider's console; what they actually
  buy is provider-aware defaults and a branded login button. The old
  wording overpromised.

* docs(features-overview): regroup catalog and add 17 missing features

Restructure the Features Overview into the same six groups the docs
sidebar uses (Stacks & Deployments, Observability, Fleet & Multi-Node,
Security & Identity, Automation, Platform) plus a short Reference tail.

Add catalog entries for 17 shipped features that the previous overview
never mentioned: Stack Activity, Stack File Explorer, Deploy Progress,
Deploy Enforcement, Blueprints, Git Sources, Global Search, Pilot Agent,
Sencho Mesh, Fleet Federation, Fleet Actions, Fleet Sync, Fleet Secrets,
Two-Factor Authentication, CVE Suppressions, Auto-Heal Policies, Stack
Sidebar.

Fix two factual inaccuracies:
- Fleet View blurb wrongly gated search, sort, filter, and stack
  drill-down behind Skipper. The deep-dive is explicit that those are
  available on every tier; only the bulk Update All action inside the
  Node Updates modal is paid.
- Auto-update entry was titled and described as a scheduling system.
  The deep-dive page is the Auto-Update Readiness board (risk tags,
  changelog previews, rollback targets); the scheduler lives under
  Scheduled Operations.

Add three hero screenshots captured from production at 1920x900,
illustrating the redesigned cockpit visual language: Home dashboard,
stack anatomy, and fleet topology.

* docs(stack-management): refresh page around v0.72.0 cockpit and add screenshots

Updates the Stack Management page to match the current UI: sidebar with
filter chips and label groups, bulk mode, restructured kebab menu,
two-tab anatomy panel, and three-source New stack dialog (Empty, From
Git, From Docker Run).

Adds sections for Filter chips, Pinned and label groups, and Bulk mode.
Restructures the Stack context menu around the inspect, organize,
lifecycle, and destructive groups with their keyboard shortcuts.
Documents the From Git tab and cross-links Git Sources for the full
sync flow.

Replaces every existing screenshot with fresh captures from the current
UI and adds eight new captures for the new sections. Cross-links Stack
Activity, Stack Labels, Stack File Explorer, Compose Editor, Atomic
Deployments, Scheduled Operations, Auto-Heal Policies, Auto-Update
Policies, and Alerts and Notifications for features documented on
their own pages.

* docs(stack-activity): refresh page around v1 cockpit and recapture screenshots

Realign the page with the current Anatomy panel tab strip and the
StackActivityTimeline component:

- Frame the Activity tab as a sibling of Anatomy under the right-hand
  panel, with files/edit actions belonging to the strip.
- Expand the category guidance: list the five iconized categories and
  call out that other stack-scoped notifications (deploy failure,
  available image updates, auto-heal triggers, monitor alerts, scan
  findings) flow into the timeline with a generic icon.
- Tighten the actor-attribution rule to match the component: omitted
  for events without an actor and for system-driven events.
- Add the day-format example to the relative-time row.
- Recapture both screenshots from a populated stack (Today + Yesterday
  + Earlier with three distinct icons) and an empty stack.
- Convert troubleshooting blurbs to H3 for anchor links and consistency
  with the v1-refresh sibling pages.

* docs(stack-activity): wrap troubleshooting entries in Accordion blocks

Match the foldable troubleshooting pattern established by
docs/features/deploy-progress.mdx so the page stays compact and
readers can scan to their issue. This is the canonical formatting
for the /features section's troubleshooting blurbs going forward.

* docs(editor): refresh page around v1 cockpit and recapture screenshots

Rewrites the page around the dual-mode right panel (Anatomy by default,
Monaco when the user clicks edit), the redesigned Command Center action
bar, the new container row layout with status badges and live stats, the
Structured / Raw terminal logs toggle, the Git Source toolbar button, and
the opt-in diff preview. Drops the obsolete persistent embedded terminal
section. Preserves the #diff-preview-before-save and #log-viewer anchors
referenced from settings.mdx and global-observability.mdx. Replaces
editor-overview.png and container-exec-modal.png with fresh captures
against v0.72.0 production at 1920x900 dark theme; renames container
-actions.png to containers-list.png; adds command-center.png and
editor-edit-mode.png. PII scrubbed (host paths normalized) and the
compose-diff-preview/diff-modal.png shared asset is left untouched after
visual diff against the live UI showed no chrome change.

* docs(features): wrap troubleshooting accordions in AccordionGroup

Wraps the loose <Accordion> blocks in editor.mdx and stack-activity.mdx
in a single <AccordionGroup> to match the troubleshooting design used
by fleet-federation.mdx. Structural only; no content changes.

* docs(stack-file-explorer): refresh page around v1 explorer and add screenshots

Full rewrite to match the live two-pane Files tab. Replaces the false
'Edit' toolbar flow with the read-only chip + always-on Save model,
fixes the protected-files list (5 names) and dedicated-tab redirect
list (3 names), drops the fabricated 100 MB download cap, and corrects
the upload claim to single file at a time.

Adds sections for New File, Rename, Permissions, the type-to-confirm
protected-file delete flow, the non-empty folder delete confirmation,
the 500-entry tree display cap, and the symlink rendering. Restructures
troubleshooting around <AccordionGroup> + <Accordion> to match
fleet-federation.mdx, and adds two new entries (display cap, 403 on
Community admin write).

Ships nine screenshots captured against v0.72.0 production at 1920x900
in dark theme: overview, two-pane layout, protected-tree-marker,
viewer-edit-mode, new-file-dialog, context-menu-folder, context-menu-file,
permissions-dialog, delete-protected-confirm.

* docs(deploy-progress): rewrite around current modal + capture v1 screenshots

Aligns the page with the v0.72.0 implementation and standardizes the
troubleshooting layout with the rest of the docs refresh.

Setting and gating
- Renames the Settings field to "Deploy progress modal" and quotes the
  current helper text verbatim. Documents that the toggle is off by
  default, lives under Settings > Appearance > Display, is stored in
  localStorage, and syncs across tabs in the same browser.

Modal anatomy
- Names every visible UI string: header verbs, status indicator (with
  the "closes in <n>s" countdown that was previously undocumented),
  empty-body strings, footer toggle that flips between "Raw output" and
  "Hide raw", and the destructive border on ERR rows vs the softer warn
  tint on WARN rows.
- Replaces the vague "after a few seconds" with the actual 4-second
  auto-close timer; documents hover-to-pause and the
  leave-hover-restarts-the-countdown behavior.
- Documents the truncated error message in the failed-state header and
  the manual close-only requirement.
- Notes that the pill is portal-mounted and survives navigation.

Stage badges
- Keeps the 9-badge table but adds an honest note that most lines render
  as LOG because the badges are gated on Compose's "[+]" progress
  prefix, which Compose only emits in TTY mode and Sencho spawns it
  without one.

Entry points
- Splits the supported actions into the four that produce a populated
  structured-log body (Deploy, Update, Install, Git Apply) and the two
  that bypass compose and finish with 0 lines (Restart, Stop). Drops the
  "Down" claim from the user-facing list since no UI control currently
  triggers it; mentions the down route as an automation surface only.

Troubleshooting
- Wraps the existing accordions in an <AccordionGroup> matching the
  pattern used by the editor and stack-activity refreshes. Adds two new
  entries: one explaining the Restart/Stop "0 lines" outcome, one
  explaining the LOG-everywhere case for non-TTY compose output.

Screenshots (six PNGs in docs/images/deploy-progress/, 1920x900, dark
theme, captured against the upgraded production node)
- setting-toggle.png: the Display section with the toggle enabled.
- modal-streaming.png: a real update in flight at 19s, 554 lines.
- modal-succeeded.png: succeeded state with the live closes-in
  countdown visible.
- modal-raw-output.png: structured rows with the Raw output panel
  expanded beneath.
- pill.png: minimized pill anchored bottom-center on a stack editor
  view.
- modal-failed.png: failed state with the truncated error in the
  header and ERR rows highlighted.

* docs(resources): refresh Resources Hub page for v1 redesign and feature additions

Rewrite the page to match the shipping UI and replace stale screenshots
with fresh captures of the redesigned chrome.

- Document the admin-only Reclaim hero and clarify the per-tile Sencho-only
  vs. All Docker (includes external) split in Quick Clean.
- Add coverage of the Scan history toolbar button and the per-row severity
  badge plus shield-icon scan dropdown in the Images tab; cross-link to the
  vulnerability scanning page.
- Correct the Volumes column list (no Size column; size lives on the Largest
  5 landing card) and call out admin gates on browse and delete.
- Spell out the List/Topology view-mode toggle and that Create Network is
  admin-only and List-mode only.
- Rewrite the Unmanaged tab section around the project-grouped layout, the
  Select all + Purge Selected (N) admin-only multi-select, and the empty
  state copy.
- Replace screenshots: resources-reclaim, networks-list, create-network,
  network-inspect, network-topology, network-topology-toggle. Add fresh
  resources-volumes-tab and resources-unmanaged-tab captures.

* docs(app-store): refresh page around v1 deploy sheet, scan integration, and registry settings

Rewrites the App Store reference to match the current cockpit:

- Documents the weekly-rotated featured banner picked from the top-5 by GitHub stars and the star-descending grid sort.
- Adds the deploy-sheet structure (breadcrumb, meta line, About panel with Read more) and splits the Advanced tab into Ports, Volumes, Environment variables, Custom variables, and Security subsections.
- Documents the Trivy-gated Security checkbox, atomic vs non-atomic deploys by tier, and the rollback semantics driven by error class.
- Adds a Watching the deploy section linking to the deploy-progress modal.
- Rewrites the Custom registry section against the new two-panel settings layout (Default + Custom) with the URL validation rule and the using default / using custom hint.
- Adds a four-entry Accordion troubleshooting block in the house style.
- Replaces three screenshots and adds two (Advanced tab, Settings registry panel) captured against the production node.

Permissions wording aligns with current backend (admin only); the broader stack:create gate will land in a follow-up fix branch.

* docs(app-store): describe inline port-conflict messaging on deploy sheet

Update the deploy-sheet section to match the visible port-conflict
behavior: the Essentials tab surfaces a Port-conflict warning that
replaces the defaults hint when any default port is already bound, and
the Advanced tab shows an inline "in use by {stack}" message next to
the container port instead of a hover-only tooltip. Refresh the
screenshot alt-text and the troubleshooting Accordion to match.

* docs(app-store): align permissions note with stack:create gate

Pairs with the backend gate swap in fix/templates-deploy-rbac (#986)
which moves POST /api/templates/deploy from requireAdmin to
requirePermission('stack:create'). Updates the Note block under
'Watching the deploy' so the docs match the new behavior: admin and
node-admin can deploy templates from the App Store; viewer, deployer,
and auditor cannot.
2026-05-07 23:44:13 -04:00
Anso 3d9489648e fix(pilot): harden outbound reverse-tunnel against resource exhaustion (#979)
* fix(pilot): cap tunnel frame size, concurrent streams, and stream idle time

Pilot tunnels carry every HTTP, WS, and Mesh-TCP byte for a remote node
through a single multiplexed WebSocket. A buggy or compromised peer that
sent oversized frames, opened streams in a tight loop, or left streams
parked indefinitely could exhaust gateway memory.

Three protocol-level limits applied symmetrically on both ends:

  - MAX_FRAME_SIZE_BYTES (8 MB): set as the ws maxPayload on the
    gateway-side WebSocketServer and the agent-side WebSocket client,
    plus a defense-in-depth length check in decodeBinaryFrame and
    decodeJsonFrame.
  - MAX_STREAMS_PER_TUNNEL (1024): bridge refuses new loopback HTTP /
    upgrade / TCP allocations with 503 once at the cap; the agent
    rejects new incoming http_req / ws_open / tcp_open with the
    appropriate error frame.
  - STREAM_IDLE_TIMEOUT_MS (10 min): every stream gets a per-stream
    timer refreshed on each inbound or outbound activity. Expiry tears
    down the local half and notifies the peer.

The constants are colocated in pilot/protocol.ts so any future agent
build picks them up via the protocol module.

* fix(pilot): rate-limit pilot enrollment endpoints to 10 per minute

Pilot enrollment mints a JWT and writes a pilot_enrollments row, both
privileged operations that should not share the global API budget
(200/min). Add a dedicated express-rate-limit instance keyed by user or
IP at 10/min in production (100/min in dev so the local enrollment
test loop is not throttled).

The limiter is applied directly on POST /api/nodes/:id/pilot/enroll.
On POST /api/nodes the limiter's skip function reads the parsed body
and exempts proxy-mode creates; only requests that resolve to
mode=pilot_agent count against the enrollment budget.

* fix(pilot): cap concurrent pilot tunnels system-wide

The PilotTunnelManager held an unbounded Map of bridges. A reconnect
storm, runaway enrollment, or operator misconfiguration could grow the
map without limit, even though every tunnel still carries a valid JWT.

Cap at 256 concurrent tunnels per primary instance:

  - Soft warning at 128 (logged once per crossing).
  - Hard refusal at 256: registerTunnel throws PilotTunnelCapacityError
    and the upgrade handler closes the WebSocket with 1013 (Try Again
    Later) so the agent backs off rather than tight-looping.

A node that already has a registered tunnel does not consume a new
slot when it reconnects; the existing bridge is closed first.

* fix(pilot): release backpressure as soon as the tunnel buffer drains

Previously the bridge paused HTTP request bodies when the tunnel
WebSocket's bufferedAmount climbed above 4 MB but never resumed them
explicitly; the next data event re-checked the threshold, and TCP
streams only saw a 'drain' fan-out on the 30 s ping cycle. Slow-consumer
peers held buffered bytes for tens of seconds longer than needed.

Replace both with an on-demand drain check: when at least one stream
is paused, sample bufferedAmount every 100 ms; once it drops below the
high-water mark, resume every paused request and emit 'drain' to every
accepted TCP stream, then stop the timer. Dormant when no stream is
paused, so steady-state cost is zero.

* fix(pilot): trust internal CAs via SENCHO_PILOT_CA_FILE

Self-hosted deployments often terminate TLS with an internal CA. The
only previous escape hatch was NODE_TLS_REJECT_UNAUTHORIZED=0, which
disables verification across every outbound connection in the agent
process and is the wrong shape of fix.

Add SENCHO_PILOT_CA_FILE: when set, the agent reads the file as a PEM
bundle and passes it to ws as the `ca` option for the tunnel
WebSocket. rejectUnauthorized remains true. Failure to read the file
exits with a clear error so the operator does not silently fall back
to the default trust store.

There is no flag to disable TLS verification entirely; that would
defeat the credential trust model.

* fix(pilot): gate frame-rate-adjacent logs behind developer_mode

The agent's malformed-frame warning fires per inbound frame and could
flood logs under attack or against a buggy primary. The bridge swallowed
parse errors silently, leaving operators blind to protocol drift, and
the manager had no signal at all on tunnel registration.

Route per-frame and per-tunnel diagnostic logs through the existing
isDebugEnabled() helper (same pattern used by middleware/auth, RBAC,
and websocket/logs). Production stays quiet by default; the toggle is
the existing developer_mode setting in global_settings — no new env
var, no new dependency.

* feat(pilot): expose per-tunnel metrics via /api/system/pilot-tunnels

Operator support had no signal beyond raw logs to answer "is this
tunnel flapping?" or "is one bad node hiding behind the aggregate?".
Add an in-process counter set covering tunnels_total, tunnels_replaced,
tunnels_rejected_capacity, enroll_acks, frame_decode_errors, plus a
per-node array carrying the connectedAt and bufferedAmount so a single
tunnel sitting on a stuck buffer remains visible.

Counters live in services/PilotMetrics.ts; the gateway has no shared
in-process metrics facility today, so this is a per-feature pattern
documented as such for future consolidation. Counters are strictly
process-local — no telemetry, no export, no phone-home — consistent
with Sencho's privacy posture.

Surfaced read-only via GET /api/system/pilot-tunnels behind admin auth,
mirroring the existing cache-stats endpoint.

* fix(pilot): tighten reconnect backoff and log handling

Three small follow-ups to the hardening pass:

  - Sanitize the primary URL on the agent's connect log so a malicious
    SENCHO_PRIMARY_URL with embedded control characters cannot inject
    fake log lines.
  - Move the reconnect-backoff reset from the 'open' callback to the
    'hello' frame handler. A peer that always rejects the handshake
    (incompatible version, consumed enrollment token) used to reset
    the backoff on every TCP-level connect and tight-loop reconnects;
    now the reset waits for a clean protocol round-trip.
  - Document the StreamIdAllocator wrap behavior so future readers
    can see the relationship between the 2^31 wrap and the
    MAX_STREAMS_PER_TUNNEL cap without re-deriving it.

* test(pilot): add coverage for enrollment, rate limiter, and tunnel caps

Adds two test files exercising the hardening pass surfaces that were
previously uncovered:

  - pilot-enrollment.test.ts: end-to-end through POST /api/nodes
    (pilot mode), POST /api/nodes/:id/pilot/enroll, the SHA256
    token-hash persistence, replay protection, expired enrollments,
    and the rate-limit header wiring (enrollment limiter on pilot
    paths, global limiter elsewhere).
  - pilot-bridge-limits.test.ts: oversize-frame rejection at both
    binary and JSON decoders, and the per-tunnel concurrent stream
    cap as observed via openTcpStream returning null and the loopback
    HTTP server returning 503 once the bridge is at capacity.

Bridge cap test fills the slot map with TCP stream handles rather than
real HTTP requests so the OS socket pool stays out of the picture.

* test(pilot): cover manager metrics snapshot and capacity-error shape

Adds direct coverage for:

  - PilotTunnelCapacityError exposing the limit so the upgrade
    handler can format a useful close-frame reason.
  - PilotMetrics.snapshot returning a defensive copy (callers must not
    mutate the live counter set).
  - getMetricsSnapshot returning the open count, per-node breakdown,
    and counter set in a stable shape, with tunnels_total bumping on
    a successful registerTunnel.

The manager test seeds a real pilot-mode node row first because
updateNode throws on missing rows; this models the production path
where a node is created before its enrollment is consumed.

* docs(pilot): document tunnel limits, custom CA, and new failure modes

Refresh the Pilot Agent feature doc to cover the hardening surfaces:

  - New 'Self-signed primary TLS certs' section walking through the
    SENCHO_PILOT_CA_FILE env var with an example docker run command.
  - New 'Resource limits' section listing the per-tunnel and
    system-wide ceilings so operators know what the wire enforces.
  - Four new troubleshooting entries: WebSocket close 1013 (system
    cap), loopback 503 (stream cap), close 1002 with protocol error
    (frame size / malformed), and HTTP 429 on enrollment.

Also expand .env.example with a new pilot-agent block covering
SENCHO_MODE, SENCHO_PRIMARY_URL, SENCHO_ENROLL_TOKEN, and
SENCHO_PILOT_CA_FILE so operators do not have to read code or the
feature doc to discover the agent-side config surface.

* fix(pilot): address code-review findings on the hardening pass

Critical:
  - Bridge: TCP backpressure now starts the drain timer and tracks
    streams awaiting drain, so a TcpStream caller waiting on 'drain'
    no longer hangs when no HTTP request is in flight.
  - Bridge: every direct streams.delete call now goes through the
    removeStream helper so per-stream idle timers are cleared
    consistently and pausedReqs / tcpAwaitingDrain stay aligned.
  - Bridge close(): resume any paused IncomingMessage before clearing
    the map so a parser does not stay stuck across teardown.

High:
  - Protocol: decodeJsonFrame now compares Buffer.byteLength(raw,
    'utf8') against the cap; raw.length (UTF-16 code units) let
    multi-byte payloads sneak ~3x the byte budget through.
  - Agent: the optional CA bundle is read once at construction and
    cached on the instance, so a missing or rotated SENCHO_PILOT_CA_FILE
    no longer process-exits on every reconnect attempt.

Medium / Low:
  - Bridge limits test: assert mockWs.sent.length to confirm slot
    allocations actually serialized a tcp_open frame and that a
    rejected (cap+1) attempt did not consume a stream id.
  - Manager: defer the tunnels_replaced increment until after the
    cap check passes, so a rejected reconnect does not double-count
    as both replacement and capacity rejection.
  - Manager: clarify the protocol comment about MAX_FRAME_SIZE_BYTES
    enforcement layering (ws maxPayload is authoritative; decoder
    check is for tests and defense-in-depth).
  - Limiter: defensive `if (!req.body) return false` so a future
    refactor that delays body parsing cannot silently skip the
    enrollment limiter.

Deferred (documented as follow-ups, not in this branch):
  - M2: narrowing public surface of PilotTunnelManager.getBridge to a
    MeshTunnelHandle interface (cross-cutting refactor).
  - H4: end-to-end replay test driving the upgrade handler twice
    with the same enrollment JWT (needs WS test harness).
2026-05-07 21:28:35 -04:00
Anso 3b650523c1 Audit-hardening pass for secret and misconfiguration scanning (#977)
* fix(security): dedupe concurrent compose-stack scans

Track stack scans in scanningImages keyed stack:<nodeId>:<stackName>.
The /scan/stack route returns 409 when an in-flight scan exists, and
the service-side check is the real correctness barrier (the route
pre-check is a fast-path optimization that mirrors scanImage). The
dedup key release lives in a try/finally so failed scans free the
slot for retry.

Why: scanComposeStack had no equivalent of scanImage's scanningImages
guard, so two simultaneous calls for the same stack would both run
trivy config, both insert a vulnerability_scans row, and double-
process the result.

* feat(security): acknowledge misconfig findings

Adds a parallel acknowledgement system for Trivy misconfig findings
that mirrors cve_suppressions: a new misconfig_acknowledgements table,
read-time enrichment via the new misconfig-ack-filter utility, REST
CRUD endpoints, fleet-sync replication from control to replicas, a
Settings panel, and an Acknowledge button on the Misconfigs tab.

Schema and behavior parity with cve_suppressions:
  - UNIQUE(rule_id, COALESCE(stack_pattern, '')) so fleet-wide acks
    collide as expected
  - blockIfReplica on every write
  - Audit-log entries name the scope (rule_id, stack_pattern) but
    never the reason text
  - replicated_from_control flag controls UI delete affordance and
    drives clearReplicatedRows on demote/reanchor
  - Validators reused: validateStackPatternForRedos for glob safety,
    sanitizeForLog for log fragments

SARIF export emits an external/accepted suppression entry per
acknowledged misconfig, matching the CVE pattern.

Per-row Acknowledge dialog prefills stack_pattern with the scan's
stack_context so the default scope is "rule + this stack only" and an
operator must broaden explicitly.

Tests: misconfig-ack-filter (15) and misconfig-ack-routes (23)
including the duplicate-409 case for both pinned and fleet-wide acks.

* fix(security): reap orphaned trivy tmp dirs at startup

When the buildEnv path writes a per-scan DOCKER_CONFIG dir under
os.tmpdir() and the process crashes before the finally block runs,
the dir leaks. Mirrors GitSourceService.sweepStaleTempDirs:
exported sweepStaleTrivyTempDirs is fire-and-forget at boot,
removes prefix-matching dirs older than 1 hour, swallows
permission/race failures, logs a single line if any were reaped.

* perf(security): emit per-batch summary for scanAllNodeImages

Adds one diag() line at the end of scanAllNodeImages summarising
unique image count, scanned, skipped, failed, violation count, and
elapsed time. Per-image diag inside scanImage stays useful for
debugging individual scans; the summary gives operators a single
fleet-level checkpoint when developer_mode is on.

* perf(security): cap SARIF export at 5000 findings per type

Replace the unbounded fetchAllPages walk on /scans/:id/sarif with a
hard limit of 5000 findings per type. When any type trips the cap,
emit run-level properties.truncated=true plus row_limit and per-type
totals so downstream tooling can flag the export as partial.
Console-warns for ops visibility.

A scan with 50k vulns previously streamed every row into memory
before serialising; the cap bounds memory and serialisation time at
the cost of completeness on pathological scans.

* docs(env): document TRIVY_BIN host-binary override

The env var is honored by TrivyService.detectTrivy as a fallback when
no managed install is present, but it was undocumented in
.env.example. Adds the var with a comment explaining precedence
(managed > TRIVY_BIN > PATH).

* test(security): cover scanComposeStack failure modes

Two new cases drive the existing try/catch through real failure
paths:
  - Malformed Trivy stdout: row flips to status='failed' with the
    parser error preserved on `error`.
  - execFile rejection: row flips to status='failed' with a string
    error message.

Pairs with the existing dedup tests so the failure path now also
verifies the scan row state, not just the thrown exception.

* test(e2e): security scanner + misconfig acknowledgement flow

Seven Playwright tests covering the scanner UI and the new
acknowledgement system end-to-end:
  - Trivy availability gate (skips suite when binary absent so CI
    without Trivy can opt out via E2E_SKIP_TRIVY=1)
  - Stack config scan completes and records misconfig findings
  - Concurrent stack scan returns 409 from the dedup gate
  - Misconfig ack POST creates and lists on Settings
  - Duplicate (rule_id, stack_pattern) returns 409
  - Malformed rule_id (shell metacharacters) returns 400
  - Misconfigs tab renders against a real stack scan

Tests drive the API for behaviour assertions and the UI only for
shell-rendering checks; the visual snapshot suite owns screenshots.

* docs(features): add misconfig acknowledgement workflow and SARIF cap

Refreshes vulnerability-scanning.mdx with:
  - Misconfig acknowledgements section covering the per-row dialog,
    Settings panel, scope/matching rules, and SARIF emission
  - Tier table row for the new feature
  - SARIF section note on the 5000 row-per-type cap and the
    properties.truncated marker for partial exports
  - Troubleshooting entries: SARIF cap, hidden Acknowledge button,
    findings resurfacing after delete, Trivy DB phone-home, and
    409 on concurrent compose-stack scans

* fix(ci): clear backend lint and CodeQL alerts

- Remove the dead fetchAllPages helper in routes/security.ts. It lost
  its callers when the SARIF endpoint switched to direct paged reads
  for the truncation cap. ESLint flagged it as unused.
- Switch the trivy-tmp-cleanup test helper to fs.mkdtempSync. Building
  paths under os.tmpdir() with predictable names tripped CodeQL's
  js/insecure-temporary-file rule (high severity), which warns about
  symlink-pre-creation attacks even in test code. mkdtempSync appends
  a process-random suffix and creates the dir atomically; the
  sencho-trivy- prefix is preserved so the production sweep still
  matches the test fixtures.
2026-05-07 19:23:11 -04:00
Anso d69fb9f1da feat(meta): gate deferred Fleet tabs behind SENCHO_EXPERIMENTAL flag (#886)
Hide the Traffic / Routing, Deployments, Federation and Secrets Fleet
tabs by default. They re-appear when the operator opts in by setting
SENCHO_EXPERIMENTAL=true. Backend routes and database tables are
unchanged; this is a UI discovery gate only.

The /api/meta endpoint now returns experimental as a boolean. A new
useExperimental hook reads it once per page load and feeds the four
tab triggers and tab content panels in FleetView.
2026-05-02 18:01:06 -04:00
Anso ed553f1f19 feat: change default listen port from 3000 to 1852 (#756)
Updates the backend listen port, Vite dev proxy target, Docker EXPOSE,
compose port mapping, .env.example default, GitHub Actions smoke-test
default, healthcheck URLs, and every doc/example reference. Test fixtures
that include example URLs were updated for consistency, though their
assertions are port-agnostic.

The rate-limit value of 3000 in middleware/rateLimiters.ts and the
3000 entry in WEB_UI_PORTS (which detects user containers like Grafana)
are intentionally untouched.
2026-04-24 22:23:31 -04:00
Anso 6890224903 fix(sso): harden Custom OIDC provider and SSO configuration (#630)
- Add Host header injection validation for OAuth callback URL derivation
  when SSO_CALLBACK_URL is not set (extracted into shared getSSOBaseUrl helper)
- Add startup warning when OIDC providers are enabled without SSO_CALLBACK_URL
- Add diagnostic logging for claim fallback, email changes on re-login,
  and admin seat limit enforcement (gated behind Developer Mode)
- Add Custom OIDC environment variables to .env.example
- Fix stale AdmiralGate comment in SSOSection.tsx
- Fix .env.example section header referencing removed Admiral tier gate
- Fix docs: correct Custom OIDC display name default, replace nonexistent
  DEBUG=true env var reference with Developer Mode toggle
- Add tests for role enforcement (viewer 403), API token scope denial,
  OIDC claim edge cases, Custom OIDC test connection, callback error params
- Refresh SSO settings screenshots
2026-04-16 08:10:28 -04:00
Anso 8e1b9826cf fix(api): add tiered rate limiting to prevent polling lockouts (#460)
* fix(api): add tiered rate limiting to prevent polling lockouts

Replace the single global rate limiter (100 req/min/IP) with a tiered
system that separates high-frequency polling endpoints from standard
API traffic:

- Polling tier (300/min): /stats, /system/stats, /stacks/statuses,
  /metrics/historical, /health, /meta, /auth/status, /auth/sso/providers,
  /license. Exempt from the global limiter but governed by their own
  safety net to prevent resource exhaustion.
- Standard tier (200/min): All other endpoints, raised from 100.
- Webhook tier (500/min): POST /webhooks/:id/trigger, dedicated limiter
  for CI/CD platforms sharing datacenter IPs.
- Auth tier: Unchanged (5-10 attempts / 15 min).

Enterprise adaptations:
- Authenticated requests keyed by user session (JWT sub/username) instead
  of IP, preventing shared NAT/VPN environments from pooling budgets.
- Internal node-to-node traffic (node_proxy tokens) bypasses all rate
  limiters entirely.

Includes comprehensive stress tests (21 cases) validating tier
separation, node proxy bypass, and per-user keying.

* chore(deps): bump axios to 1.15.0 to fix SSRF vulnerability

Addresses GHSA-3p68-rc4w-qgx5 (NO_PROXY hostname normalization bypass).
2026-04-09 17:56:58 -04:00
Anso 7d9dcc77d4 docs: remediate documentation gaps across quickstart, backup, config, API spec, and operations guides (#330)
- Fix Cyrillic character in quickstart image ref and correct registry to Docker Hub (saelix/sencho)
- Correct backup guide WAL references (Sencho uses SQLite default journal mode)
- Add SSL/TLS reverse proxy examples for Nginx, Traefik, and new Caddy configuration
- Add missing env vars (PORT, DATA_DIR, NODE_ENV, FRONTEND_URL, SSO_LDAP_DISPLAY_NAME) to .env.example
- Add upgrade & migration guide documenting automatic schema migrations
- Add self-hosting best practices (1:1 path rule, Docker socket security, resource recs)
- Add architecture overview (system design, request flow, database schema, multi-node model)
- Add development & contributor guide (setup, tests, code style, PR workflow)
- Update OpenAPI spec from v0.23.0 to v0.25.3 with Registries and Image Updates endpoints
- Update docs.json navigation with all new pages and API groups
2026-04-01 23:17:32 -04:00
Anso b28ebfa6ff feat(api): add global rate limiter for all API endpoints (#317)
Apply a global rate limit of 100 requests/min per IP to all /api/ routes
in production, configurable via API_RATE_LIMIT env var. Auth endpoints
retain their existing stricter limits which stack independently.
Returns 429 Too Many Requests when exceeded.
2026-04-01 20:12:30 -04:00
Anso c328b7f49a refactor: rename Personal Pro to Skipper and Team Pro to Admiral (#256)
Align paid tier names with Sencho's nautical identity. Internal variant
values ('personal'/'team') remain unchanged in code, database, and
Lemon Squeezy integration — only user-facing display names updated.

- Backend: requireTeamPro → requireAdmiral, TEAM_PRO_REQUIRED → ADMIRAL_REQUIRED
- Frontend: TeamProGate.tsx → AdmiralGate.tsx, TierBadge labels updated
- Website: PricingSection tier names and nautical descriptions
- Docs: all 11 affected pages renamed, nautical footnote added to licensing
2026-03-29 18:00:29 -04:00
Anso bd4008f509 feat: SSO & LDAP authentication for Team Pro (#209)
* feat: SSO & LDAP authentication for Team Pro

Add SSO integration allowing Team Pro users to authenticate via LDAP/Active Directory, Google, GitHub, and Okta identity providers. SSO works alongside password authentication with auto-provisioning and role mapping.

- LDAP bind+search authentication with group-based role mapping
- OIDC/OAuth2 flows with PKCE and CSRF protection for Google, GitHub, Okta
- Auto-provisioning: first SSO login creates a Sencho account automatically
- Role mapping via LDAP group membership or OIDC JWT claims
- SSO settings UI in Settings → SSO with per-provider config and test connection
- SSO login buttons on login page with LDAP toggle
- Environment variable seeding for infrastructure-as-code workflows
- Secrets encrypted at rest via CryptoService (AES-256-GCM)
- Seat limit enforcement during auto-provisioning
- Full documentation: feature docs, quickstart guides, env var reference

* fix: resolve ESLint errors in SSO feature

- Remove unnecessary escape characters in regex character classes
- Remove unused `issuer` variable from OIDC callback handler
- Fix setState-in-effect lint error in Login.tsx by using useState initializer
- Suppress set-state-in-effect for SSOSection fetch pattern (matches existing codebase convention)
2026-03-28 03:30:01 -04:00
unknown 293f9cef26 Initial commit: Sencho V1 complete with Auth and Dockerization 2026-02-20 18:39:32 -05:00