412 Commits

Author	SHA1	Message	Date
Anso	aa3d99a594	fix(mesh): re-evaluate data plane every 10s and add opt-in auto-recreate (#1184) ... * fix(mesh): re-evaluate data plane every 10s and add opt-in auto-recreate MeshService.dataPlaneStatus was written exactly once at boot in setupMeshNetwork() and never re-evaluated. After the operator removed sencho_mesh at runtime (or it was recreated externally, or Sencho was disconnected from it), /api/health and the dashboard banner kept returning the stale boot-time discriminator until the next process restart. Adds a 10s revalidator that inspects the current Docker truth in one network-inspect call and transitions dataPlaneStatus to reflect it. Short-circuits in not_started / not_in_docker / subnet_invalid (states that cannot change within this process) and in concurrent ticks. Transitions are idempotent on stable state, so the timer can tick indefinitely on a healthy mesh without log noise. New 'not_found' reason value for the network-was-removed-at-runtime case. Existing reasons (subnet_mismatch, subnet_overlap, attach_failed) also surface from the revalidator when their underlying conditions arise post-boot. transitionDataPlane keeps message and subnet fields fresh across consecutive observations even when reason is unchanged, so /api/health never reports stale numbers (e.g. two consecutive subnet_mismatch observations against different external subnets). Adds an opt-in mesh_auto_recreate global setting (default off). When on, the revalidator additionally calls attemptInPlaceRecreate() after surfacing not_found. The helper hard-prefers the boot-chosen subnet (this.meshSubnet) and never iterates candidates, because changing the subnet here would invalidate every existing extra_hosts override on disk. A real conflict on the original subnet is reported as subnet_overlap and preserved during the 60s recreate throttle window so the operator-actionable reason is not flapped back to not_found between attempts. Self-attachment is checked via Name match (operator --hostname X matches container Name /X) or full container-ID prefix for hex HOSTNAMEs >= 12 chars (Docker default short ID). Non-hex HOSTNAMEs cannot collide with container IDs at all so a Name miss is conclusive; short hex HOSTNAMEs preserve the prior status as 'unknown' rather than risking a false-positive prefix match. Frontend surfaces: - types/mesh.ts: 'not_found' added to MeshDataPlaneReason. - MeshDataPlaneBanner: 'not_found' headline copy. - Settings > System > Mesh data plane: TogglePill bound to mesh_auto_recreate, default off, helper text explains the tradeoff. Backend coverage in backend/src/__tests__/mesh-data-plane-revalidate.test.ts (25 cases): short-circuits, idempotent stable-state, recovery from subnet_mismatch / subnet_overlap, transition to not_found / subnet_mismatch / attach_failed, transient-Docker anti-flap, re-entrancy guard, name match path, ID-prefix path, short hex hostname ambiguity, non-hex hostname certainty, transition message refresh on observation drift, auto-recreate off (default), auto-recreate success with senchoIp preservation, auto-recreate overlap classification with no subnet drift, throttle window preserves classified reason, throttle release. Lifecycle test covers timer wiring in start()/stop(). Existing mesh-setup-error-classification suite (27 cases) still green. Resolves: F-4 in the v1.0 audit tracker. * fix(mesh): address Codex review of PR #1184 Three findings from the independent review: BLOCKER: attemptInPlaceRecreate() called recordSetupFailure() on create / attach failures, which clears this.senchoIp. The next revalidator tick's attachment check is guarded on senchoIp, so with it null the check is skipped and the snapshot path can silently flip the status back to ok against a network where Sencho is in fact not attached. Also: a later successful recreate would call ensureSelfAttached() with senchoIp null, which short-circuits, so the network gets recreated without binding Sencho. Replaced the recordSetupFailure() calls in attemptInPlaceRecreate with a new recordRecreateFailure() that uses transitionDataPlane and preserves senchoIp. Added two tests: create-fails-then-succeeds (verifies senchoIp survives the failure and the later retry binds Sencho correctly) and create-succeeds-attach-fails-then-next-tick (verifies the snapshot path surfaces attach_failed on the next tick instead of falsely reporting ok). SHOULD-FIX 1: single-key POST /api/settings wrote String(value) without re-validating against the per-key schema, so an allowlisted enum-shaped key like mesh_auto_recreate could persist arbitrary strings ('banana', 'true') that the bulk PATCH would later refuse. Routed the single-key path through SettingsPatchSchema.safeParse so both write paths validate identically. Added regression tests for an invalid mesh_auto_recreate value, a valid mesh_auto_recreate write, and an out-of-range numeric value. SHOULD-FIX 2: the new Mesh data plane subsection lived inside a section the registry exposes to non-admins, who would see the toggle and only learn it was admin-only after the save 403'd. Gated the subsection on `isAdmin` from useAuth so non-admins do not see the control. The other system controls keep their existing visibility pattern (read-only for non-admins). 71/71 backend tests green (revalidate + mesh-setup + settings-routes). 276/276 frontend tests green. tsc clean on backend + frontend.	2026-05-23 18:09:39 -04:00
Anso	f03c9dc7b6	fix(webhooks): address Codex review of PR #1177 (#1181) ... * fix(webhooks): close HMAC timing oracle on trigger reject paths The trigger handler in PR #1177 returned a uniform 404 for every unauthenticated rejection, but only the wrong-signature path computed an HMAC over the request body. The other reject paths (unknown id, disabled webhook, non-paid tier, missing X-Webhook-Signature header, missing rawBody) short-circuited before any HMAC work. Repeated near-rate-limit probes with a large attacker-controlled body could distinguish a valid-and-enabled paid webhook id from the other reject cases through response latency. WebhookService.validateSignature is now constant-time over every input shape: it always runs crypto.createHmac and crypto.timingSafeEqual against fixed-length 32-byte buffers regardless of whether the signature is missing, has the wrong prefix, is malformed hex, or is the wrong length. The trigger handler calls it unconditionally before any reject branch fires, using a stable per-process decoy secret (WebhookService.getDecoySecret) when the webhook does not exist and an empty buffer when the request has no body. Response timing now depends only on the size of the request body, which the attacker already controls and which reveals nothing webhook-specific. Six new tests pin the behaviour: validateSignature is observed firing on the unknown-id and missing-signature paths through a spy assertion, and four direct-call tests confirm validateSignature returns false without throwing for empty, wrong-prefix, malformed-hex, and wrong-length signatures. * fix(safe-log): redact Basic auth and lowercase Windows drive letters The redactSensitiveText helper now covers two cases the prior chain missed: * Authorization: Basic <base64> previously left the base64 payload intact. The existing key/value regex caught only the literal word Basic before stopping at the space. A new Basic\s+[A-Za-z0-9+/=]+ replacement runs before the key/value regex so the credential is scrubbed first. * Windows homedir paths like c:\Users\<user>\... with a lowercase drive letter previously slipped through because the regex required [A-Z]. Changed to [A-Za-z] so both letter cases are covered. Two new tests pin both fixes. * docs(webhooks): document 429, fix shared schema, comply with D27/D31 * Trigger endpoint declares the 429 response that webhookTriggerLimiter can return (500 requests per minute per source IP); both docs/openapi.yaml and the response table in docs/features/webhooks.mdx carry the new row, and a new troubleshooting accordion explains the shared-NAT scenario. * Shared Webhook schema in docs/openapi.yaml extends the action enum to include git-pull and documents the node_id property. The GET list endpoint returns these fields; the prior schema would have failed validation for any git-pull row. * docs/features/webhooks.mdx:7 rewritten from a customer-side role enumeration ("non-admins on a paid tier can view the list but cannot manage it") to a single requirement statement ("Webhooks require a Skipper or Admiral license. Managing webhooks is admin-only.") per CLAUDE.md D27/D31; the prior phrasing was customer-side fence-spec. * Two em dashes in webhook description strings I had touched in the prior OpenAPI sync commit replaced with semicolons per D18.	2026-05-23 16:03:48 -04:00
Anso	21ec5e7e0a	fix(webhooks): harden trigger response surface (#1177) ... * fix(webhooks): harden trigger response surface Bundles six audit findings on the incoming-webhooks trigger path. All changes preserve the documented happy path: a CI caller signing the exact request body with the webhook secret still receives 202 Accepted. * Uniform 404 on every unauthenticated rejection (missing webhook, disabled webhook, non-paid tier, missing signature header, missing raw body, signature mismatch). The four-way response surface previously let an unauthenticated probe enumerate webhook ids and fingerprint the instance's licence tier; callers now see one shape for any failed auth. * Fail closed when express.json()'s verify callback did not capture the raw request body. Previously the handler fell back to JSON.stringify(req.body), which compares the HMAC against a re-serialised payload that is not byte-equal to what the client signed. * Pass the already-loaded webhook through to WebhookService.execute() instead of re-fetching by id. Closes the delete-during-execution race where an admin deletion between the trigger handler's load and the async dispatch silently dropped the execution row. The webhook_executions table has ON DELETE CASCADE, so recordExecution now wraps the insert in try/catch and logs a warning when the FK constraint trips because the parent webhook was deleted mid-flight. * Redact bearer tokens, JWTs, URL credentials, and homedir paths from error strings before persisting to webhook_executions.error. The execution history is readable by any paid user via GET /webhooks/:id/ history; redactSensitiveText gains three home-directory patterns (/home/<user>, /Users/<user>, <drive>:\Users\<user>) and now runs on every error stored from this path. * Cap webhook name at 100 characters on both POST and PUT, rejecting non-string and oversized values with 400 before they reach the DB. * Validate the body's action override against a typed allowlist (isWebhookAction type guard) on the trigger endpoint, returning 400 before queueing execution. An unknown override no longer reaches recordExecution as a stored failure row. Tests updated to pass db.getWebhook(id)! instead of the raw id to the new execute() signature. Docs at docs/features/webhooks.mdx updated to reflect the new uniform 404 response, the new 400-on-invalid-action behaviour, and a rewritten troubleshooting accordion that walks operators through every cause of the uniform 404. * test(webhooks): cover trigger handler auth, race, and redaction paths Adds 21 vitest cases for the public webhook trigger handler and the WebhookService.execute / recordExecution pipeline, plus 3 cases for the new homedir patterns in redactSensitiveText. webhooks-trigger.test.ts covers, per audit finding: * M1 + H3 uniform 404: id unknown, webhook disabled, non-paid tier, missing signature header, missing rawBody, sha1= prefix, malformed hex signature, sig mismatch. Each asserts identical 404 body so a future regression that re-introduces 401 / 403 / PAID_REQUIRED breaks one of the 8 tests. * Happy path: 202 with configured action, valid action override, unknown action override returns 400 after auth succeeds (L2), non-string action override returns 400. * L1 name cap: POST and PUT both reject names over 100 chars and non-string names; 100-char boundary still accepted; PUT allows partial updates that omit name. * M5 race: deleting the parent webhook before recordExecution runs no longer crashes the async dispatch; the FK cascade is swallowed with a console.warn, and a happy-path test pins the recordExecution row. * M6 redaction: stubs ComposeService.runCommand to throw errors containing a bearer token and a homedir path, then asserts the persisted webhook_executions.error has both scrubbed. safe-log.test.ts gains three unit tests pinning the new homedir patterns in redactSensitiveText (Linux, macOS, Windows). The existing credentials test is untouched. Tests use prototype spies on FileSystemService and ComposeService (both hand out a fresh instance per nodeId), so per-test mocks do not leak. beforeEach restores all mocks and reapplies the LicenseService 'paid' spy. Closes audit finding H2 (zero trigger-path test coverage). * docs(webhooks): sync openapi spec with new trigger response surface Brings docs/openapi.yaml in line with the behaviour changes from the trigger hardening commit. Mintlify auto-generates the per-endpoint reference pages from this spec, so the spec drift would surface as wrong response codes in the public API reference. POST /api/webhooks and PUT /api/webhooks/🆔 * name: maxLength 100 (matches MAX_WEBHOOK_NAME_LENGTH on the route). * action enum: add git-pull (pre-existing omission; the route has always accepted it). * node_id: documented as an integer property (pre-existing omission). POST /api/webhooks/:id/trigger: * requestBody required: true (body is now mandatory; the H3 fail-closed branch rejects a missing rawBody). * action override: enum restricted to the allowlist. * 401 and 403 responses removed. * 404 response: description rewritten to reflect uniform-404 behaviour; the body is { error: "Webhook not found or signature invalid" } for every unauthenticated reject. * 400 response added for an authenticated request whose action override is not in the allowlist.	2026-05-23 15:42:49 -04:00
Anso	fcff8e9047	fix(monitor): collapse repeated host-metric alerts into per-window summary (F-11) (#1175) ... * fix(monitor): collapse repeated host-metric alerts into per-window summary (F-11) A host metric over threshold previously dispatched one notification every 5 minutes for the duration of the breach, producing 7+ identical messages in 35 minutes and spamming Discord/Slack routes. Replace the hardcoded 5-minute cooldown for CPU/RAM/disk with a per-metric suppression window (default 60 min, configurable via host_alert_suppression_mins). The first breach fires immediately; subsequent cycles within the window are silently counted; the next dispatch after the window elapses carries a summary suffix listing how many cycles were suppressed and when the breach first crossed threshold. Recovery clears the counter so re-breach fires fresh. The pattern mirrors PolicyEnforcement.notifyTrivyMissingOnce: module-scope Map, in-memory only, in-cycle dedup, with a test-reset helper. The existing system_state row keeps post-restart re-fires bounded. Janitor and per-stack alert rules are unchanged; they already have adequate cadence and per-rule cooldown respectively. * fix(ci): restore backend and frontend checks * fix(e2e): remove create button timing race * fix(e2e): harden create double-click test * fix(monitor): clear persisted F-11 timestamp on recovery + clamp suppression window Independent audit on the previous commit surfaced two issues. 1. clearHostMetricSuppression early-returned on missing in-memory state, leaving a stale system_state.last_host_*_alert_ts row alive after a process restart. Scenario: breach fires + persists timestamp, process restarts, metric recovers before another evaluate cycle re-seeds the in-memory Map, recovery cleanup early-returns. Next re-breach inside the original window hits the restart-survivability branch and is silently suppressed instead of firing fresh. Fix: read persisted state in clearHostMetricSuppression and reset to '0' independently of in-memory presence. The read-before-write also skips redundant writes when the row is already cleared. 2. host_alert_suppression_mins is validated by zod on the bulk PATCH path but the single-key POST /api/settings path accepts allowlisted keys without re-validation. A 999999999-minute value would silence host alerts for centuries. Add MAX_HOST_ALERT_SUPPRESSION_MIN = 1440 mirroring the zod max, and clamp via Math.min in evaluateGlobalSettings. Two new vitest cases (restart-then-recovery-then-rebreach; the 1440 clamp) confirmed failing before the fix, passing after. The existing "metric drop" case updated to use a mock-backed persistence pattern consistent with the new restart-scenario tests. 73/73 monitor-service tests green; full backend suite 2507/2510 (same pre-existing Windows EBUSY flake on filesystem-backup.test.ts as baseline).	2026-05-23 06:28:01 -04:00
Anso	0947da13ae	fix(security): collapse repeated trivy-missing pre-deploy notifications (#1166) ... Scan policies on a node without Trivy installed previously fired one "Pre-deploy scan skipped" warning per deploy, flooding the notification feed during CI loops. Add a 60-minute per-(node, stack) cooldown so an operator sees one actionable warning, not one per deploy. The boot log line and the one-click managed install in Settings > Security are unchanged; this only reshapes the per-deploy fanout. Also tighten the vulnerability-scanning entry in /docs/features/overview to point first-touch users at the one-click install on first use.	2026-05-23 01:18:30 -04:00
Anso	64bf6344a3	fix(mesh): bias Sencho static IP via IPAM IPRange (F-13) (#1162) ... * fix(mesh): reserve Sencho static IP via IPAM auxiliary address (F-13) Sencho pins itself to <network>+2 on sencho_mesh, but the IPAM block only declared Subnet, so Docker freely handed that address to any meshed workload that restarted while Sencho was offline. A real-world hit on arrapps-prod during a Sencho upgrade had tautulli grab 172.30.0.2, which blocked the new Sencho container's mesh attach with "Address already in use" and left compose in a half-state needing manual disconnect/recreate. The fix reserves <network>+2 via AuxiliaryAddresses on the IPAM Config when creating sencho_mesh. Aux-listed addresses are removed from the auto-allocatable pool, so Docker refuses to hand the IP to any container that does not explicitly request it. Sencho's own attach via connectContainerToNetwork({ ipv4Address }) is unaffected because explicit pins still bind aux-reserved addresses. Workload containers without a pinned IP get .3 and up. Wire format verified against the Docker Engine REST v1.33 OpenAPI spec: the JSON key on POST /networks/create (and on the inspect response) is AuxiliaryAddresses inside each IPAM.Config item, value { sencho: <ip> }. Adopt-existing path: when Sencho boots against a sencho_mesh that pre-dates this reservation, the data plane still comes up but a one-time warn fires (mesh.enable activity at level: 'warn' plus a [Mesh] console line so docker logs surfaces it). The advisory explains the squat risk and gives the recreate recipe. Tests: 5 cases added to mesh-setup-error-classification.test.ts covering the explicit-env create payload, the candidate-iteration winner payload, the adopt-legacy warn (env-unset), the adopt-already-reserved silent path, and the TOCTOU 409 race-winner adopt-legacy warn. Docs: one sentence added to docs/features/sencho-mesh.mdx under "Customising the mesh subnet" describing the reservation positively. No tier/role/capability/flag gates touched (no frontend changes). * fix(mesh): use IPRange upper-half bias instead of aux-address reservation The initial F-13 fix used IPAM AuxiliaryAddresses to reserve <network>+2 on sencho_mesh. Empirical probe against Docker 29.4.3 confirmed this also blocks explicit pins via EndpointConfig.IPAMConfig.IPv4Address: libnetwork's RequestAddress() rejects a preferred-address request when the bit is already set by the aux reservation. Result: the freshly- reserved network refuses Sencho's own ensureSelfAttached, and the data plane never comes up. The pivot uses IPRange instead. IPRange constrains Docker's auto- allocation to the configured CIDR; preferred-address requests via RequestAddress(prefAddress) skip the IPRange check and only consult the subnet-wide bitmap. So setting IPRange to the upper half of the subnet (e.g. 172.30.0.128/25 for 172.30.0.0/24) biases workloads without an explicit IP to <network>+128 and up, while Sencho's explicit pin to <network>+2 still succeeds. Verified on Docker 29.4.3 with the same workstation that produced the original audit: - `docker run --rm --network N --ip 10.99.99.2` against a network with `--aux-address sencho=10.99.99.2` → "Address already in use" (rejects explicit pin). - Same `--ip` against a network with `--ip-range 10.99.99.128/25` → succeeds (10.99.99.2 is outside the range but inside the subnet). - Auto-allocated workload on the IPRange network lands at .129. Adopt-existing legacy detection now compares IPRange instead of AuxiliaryAddresses. inspectExistingMeshSubnet returns { subnet, ipRange } and the warn fires when ipRange differs from the expected upper-half CIDR. Same once-per-process semantics as before. createMeshNetwork now derives the IPRange via a new getMeshIpRangeFromSubnet helper. The five regression tests assert IPRange = <network>+128/<prefix+1> in the create payload and the expected/actual IPRange in the legacy-warn details.	2026-05-22 21:08:56 -04:00
Anso	474290081d	fix(mesh): log boot state to console so docker logs surfaces mesh status (#1159) ... MeshService records its boot summary and setup failures only through logActivity (in-memory ring buffer + WS listeners), which the Routing tab consumes. The docker-logs surface was silent for the mesh subsystem, so operators running Sencho-in-Docker had no boot-time visibility into whether the data plane came up cleanly. Mirror the existing logActivity entries to console without replacing them: - MeshService.start() success summary: console.log / console.warn / console.error gated on the summary level. Format: [Mesh] data plane ok, self attached at <ip>, subnet <X> [Mesh] data plane unavailable (<reason>: <message>) - recordSetupFailure: console.warn for the expected dev-mode not_in_docker case, console.error for real failures. Format: [Mesh] data plane unavailable (<reason>, subnet <X>): <sanitized> The activity entries that already drive the Routing tab banner stay intact; the console lines are purely additive for the docker logs workflow. Two new unit tests assert the failure and not_in_docker console mirrors fire with the [Mesh] prefix. Fixes F-5.	2026-05-22 16:30:10 -04:00
Anso	e4fe4cfced	fix(mesh): address Codex audit findings on F-1 PR (#1158) ... - docs(sencho-mesh): split subnet_overlap troubleshooting into env-set vs env-unset paths; rewrite the "Customising the mesh subnet" intro to describe the candidate list and the adopt-existing behavior. - backend(MeshService): preserve idempotent 409 handling in the explicit-env path. On createNetwork 409 (TOCTOU race against another process), re-inspect and treat the race-winner as success when its subnet matches the operator's request; subnet_mismatch otherwise. - frontend(MeshDataPlaneBanner): trim the card variant to a true one-line strip (headline only, truncate min-w-0). Full recovery hint stays on the Routing tab variant and in docs. - tests(mesh): add five cases covering the previously untested branches — candidate-loop non-overlap bail, adopt-existing with unparseable subnet, explicit-env generic createNetwork failure, TOCTOU 409 race-winner match, TOCTOU 409 race-winner mismatch. Architecture map (gitignored per Directive 11) updated locally with the new useMeshDataPlane hook node and the mesh.dashboardBanner flow so the local interactive viewer stays accurate.	2026-05-22 13:39:12 -04:00
Anso	1a03cf82af	fix(mesh): auto-fallback through candidate subnets when default overlaps (#1156) ... The default mesh subnet 172.30.0.0/24 is fully contained in linuxserver/* default networks (sonarr_default 172.30.0.0/16, etc.), so libnetwork rejects the IPAM allocation with "Pool overlaps with other one on this address space" on a typical homelab Docker host. The single hard-coded default left first-run operators with a silently broken mesh. MeshService.setupMeshNetwork now resolves the subnet via three paths: 1. Operator-explicit (SENCHO_MESH_SUBNET set): use that subnet, strict. Pre-existing sencho_mesh with a different subnet still raises subnet_mismatch. 2. Adopt-existing (env unset, sencho_mesh already on the daemon): adopt the existing subnet. Docker is the source of truth across restarts. 3. Candidate iteration (env unset, no existing network): walk 172.30.0.0/24, 172.31.0.0/24, 10.42.0.0/24, 10.43.0.0/24 in order. First subnet Docker accepts wins. If every candidate overlaps, record subnet_overlap with a message naming every attempt. The dashboard's Fleet Heartbeat card now surfaces the down state via a compact banner above the per-node rows, plus a "mesh down" counter suffix on the right of the title. The existing Routing-tab banner is extracted into a shared MeshDataPlaneBanner component with tab and card variants. Dashboard polling is gated on Admiral tier so non-paid users do not fire the Admiral-only /mesh/status endpoint. Six new tests in mesh-setup-error-classification cover: iterates past first overlap, all candidates overlap, adopts existing network, inspectNetwork non-404 failure classified as attach_failed, env-matches- existing skip-create, and operator-explicit strict (no fallback). Fixes F-1 in the pre-1.0 audit. Closes the silent-failure mode that left the mesh down on the most common homelab Docker layout.	2026-05-22 13:20:27 -04:00
Anso	519a59ed2e	feat(fleet): open Fleet Actions tab to Community (admin-only) (#1153) ... * feat(fleet): open Fleet Actions tab to Community (admin-only) Removes the requirePaid guard from the five Fleet Actions endpoints (fleet-stop, fleet-prune, match-preview, prune/estimate, bulk-assign) and drops the matching isPaid parent gate on FleetActionsTab so Community admins can run fleet-wide bulk operations. requireAdmin stays on every endpoint; operator and viewer roles still 403 on apply. Tests flipped from "403 PAID_REQUIRED on community" to positive "reachable on community + admin" assertions. Docs (fleet-actions, fleet-view, licensing, overview, stack-labels) rewritten to state the admin-role requirement once and drop the prior Skipper framing. * fix(fleet): apply audit findings from PR #1153 review - stack-labels.mdx: fix the page intro that still framed fleet label actions as "Operators on a Skipper or Admiral license". The cards are now Community + admin, so the intro reads "Admins also get a pair of fleet-wide actions". - Collapse redundant role-rule statements on the two affected pages. fleet-actions.mdx now states the admin gate once in the lead-in Note and again only in the troubleshooting accordion (the Prerequisites row was duplicative). stack-labels.mdx trims the "Limits and rules" bullet to the value-add half (label authoring is open to every role) and drops the Fleet Actions repetition. - Strip now-no-op mockTier('paid') calls from non-tier tests across the three fleet test files, plus the test-wide default in the fleet-action-card-endpoints beforeEach. Those mocks were misleading after the routes stopped consulting tier; if a future change re-adds requirePaid the tests will fail loudly instead of silently passing.	2026-05-22 01:27:27 -04:00
Anso	60f893a81f	feat(fleet): move bulk Remote OTA updates to Community tier (#1151) ... Drops `requirePaid` from `POST /api/fleet/update-all` so Community admins can dispatch bulk node updates. Per-node OTA was already Community- reachable (admin-only); this completes the move so the full Remote OTA surface ships at Community. Frontend mirrors the backend: removes `canBulkUpdate` from NodeUpdatesSheet so the "Update all (N)" affordance is purely data- driven on `updatableRemoteCount > 0`. Docs realigned to drop fence-spec and Skipper-only phrasing on the Update all bulk action: - features/licensing.mdx: Community line now lists Remote OTA (per-node and Update all); Skipper Fleet Actions parenthetical drops "bulk update all" - features/remote-updates.mdx: Note rewritten to role-only requirement - features/fleet-view.mdx: Update all (n) bullet drops the tier clause - features/overview.mdx: Fleet View and Remote updates blurbs drop the Skipper/Admiral fences - operations/upgrade.mdx: Note rephrased without naming tiers Test coverage: - fleet.test.ts: tier-gating spec flipped to assert Community access - fleet-pilot-update.test.ts: bulk-OTA dispatch suite now spies tier to Community so it doubles as a regression guard	2026-05-21 23:54:45 -04:00
Anso	8a3889dc67	feat(security): move managed Trivy auto-update to Skipper tier (#1150) ... * feat(security): move managed Trivy auto-update to Skipper tier Drop the gate on the managed Trivy auto-update toggle from Admiral to Skipper so it lives alongside the rest of Sencho's automation features (auto-heal, scheduled ops, per-stack image auto-update, scan policies) instead of behind the enterprise-control tier. Backend: `PUT /api/security/trivy-auto-update` switches from `requireAdmiral` to `requirePaid`. The 24h scheduler tick in SchedulerService that reads the setting is tier-neutral and picks the new gate up automatically. Frontend: SecuritySection.tsx swaps the inline `isAdmiral` conditional on the toggle render for `isPaid`. The local `isAdmiral` derivation and the `useLicense` import become unused and are removed. Docs: licensing, overview, vulnerability-scanning matrix, trivy-setup, and the settings reference now read Skipper consistently for this feature. * fix(security): require admin role on trivy-auto-update toggle Independent audit of the prior commit flagged that PUT /api/security/trivy-auto-update had no admin-role guard. The route was authenticated and tier-gated, but the global /api authGate only authenticates and `requirePaid` only checks tier. Any paid viewer could flip the global trivy_auto_update setting via a direct API call. Add `requireAdmin` ahead of `requirePaid`, matching the pattern used by every other mutating route in this file (sbom, policies, suppressions, misconfig-acks). Add route tests covering paid admin allowed, paid viewer rejected, community admin rejected, and unauthenticated rejected.	2026-05-21 23:54:01 -04:00
Anso	b740dd1078	feat(resources): protect Sencho's own image, network, volumes from deletion (#1149) ... * feat(resources): protect Sencho's own image, network, volumes from deletion Adds SelfIdentityService that reads HOSTNAME at startup and inspects the running Sencho container via Dockerode to record its image ID, attached networks, named volumes, and container ID. The classification API marks these with isSencho:true, destructive delete routes return 423 Locked when the target matches self, the orphan-containers API filters the Sencho container out so it cannot be selected and purged from the Unmanaged tab, and the managed-prune path adds an explicit self filter for defense-in-depth on top of Docker's in-use semantics. The Resources view renders a Sencho pill alongside the managed badge on matching rows and disables the trash control with a hover tooltip. When Sencho runs outside Docker (dev mode), inspect returns 404, the service stays empty, and every isOwn* returns false so today's behaviour is preserved. * fix(resources): handle sha256-prefixed image IDs and custom hostnames Addresses independent-review findings on PR #1149: - Strip sha256: prefix in POST /api/system/images/delete before validating the ID, matching the inspect route's handling. Without this, /system/images responses round-trip through the UI as sha256:<hex> and got 400 Invalid image ID format before rejectIfSelf could run. - Add /proc/self/cgroup fallback to SelfIdentityService so custom --hostname, Compose hostname:, or --uts=host setups still self-identify. HOSTNAME inspect runs first; on 404 the service parses the cgroup file for a 64-hex container ID (cgroupv1 docker, cgroupv2 docker, podman libpod formats all covered) and retries inspect with that ID. - Restrict prefix matching in isOwnNetwork / matchesId to hex-shaped candidates (12 to 64 hex chars), so a non-Sencho network whose name happens to start with a hex prefix of Sencho's network ID is no longer flagged as self. - Trim the resources.mdx Note to customer-visible behaviour without enumerating every tab. - New tests: prefixed-image-ID 200 path, three cgroup file format parses (v1, v2, podman) plus the no-match and missing-file cases, HOSTNAME-404-then-cgroup-success fallback path, name-collision regression for the hex-only prefix rule, and an empty-cache no-regression check. Test hygiene: mockReset on the inspect stub and restoreAllMocks in afterEach so spies do not leak across tests. * chore(security): VEX not_affected for CVE-2026-46680 (containerd in docker-compose) Trivy now flags CVE-2026-46680 HIGH on usr/local/lib/docker/cli-plugins/docker-compose, which statically embeds github.com/containerd/containerd/v2 v2.2.3 (compose v5.1.3's resolved module graph). The CVE is a runtime-executor flaw: containerd's runc invocation can be tricked into running a Kubernetes pod marked runAsNonRoot as root via crafted user ID handling. The vulnerable code path is reached only by containerd-shim executing a container with a populated OCI runtime spec on the daemon side. docker-compose vendors the containerd Go module purely as a client (gRPC stubs, API types, shared utilities); it never executes containers and never enforces runAsNonRoot. Sencho's compose usage (up / down / ps against user-authored files) cannot construct a Kubernetes pod security context. The vulnerable path is unreachable. Adds a not_affected entry to security/vex/sencho.openvex.json with justification vulnerable_code_not_in_execute_path, bumps version 5 to 6, and updates last_updated to 2026-05-22 per Directive 23.	2026-05-21 23:10:13 -04:00
Anso	e57799d4df	docs: expand Features subgroups by default (#1148) ... Mintlify nested groups collapse unless `expanded: true` is set on each group object. Adding it to all six Features subgroups so users land on a fully scannable sidebar instead of six folded headers.	2026-05-21 22:02:07 -04:00
Anso	5f7a887ed6	docs: reorganize navigation for feature discoverability (#1147) ... Restructures the Documentation tab so each group answers one operator question. - Split the 12-page "Stacks & Deployments" into Stacks (per-stack work) and Deployment (the act of deploying); promote Resources Hub to a standalone item. - Dissolve the 2-page "Platform" junk drawer: Sidebar moves to Stacks, Host Console moves to Fleet. - Rename "Fleet & Multi-Node" to "Fleet"; absorb Node Compatibility from Reference. Move Scheduled Operations from Fleet to Automation (now a 4-page group covering Scheduled Ops, Auto-Update, Auto-Heal, Webhooks). - Clean up the Reference tab: drop misplaced node-compatibility, move root-level security.mdx into reference/, delete the orphan reference/verifying-images.mdx after porting its Available Tags table into operations/verifying-images.mdx. - Reorder top-level groups: Operations moves above Reference. - Rename two misleading page titles: "Deploy Progress Modal" becomes "Deploy Progress" (drops the UI implementation leak); "Auto-Update Readiness" becomes "Auto-Update Policies" (matches filename and sibling "Auto-Heal Policies"). Verified: docs.json parses as valid JSON, 59 disk .mdx files match 59 nav entries with zero orphans and zero broken refs.	2026-05-21 21:25:23 -04:00
Anso	be22e3ded1	docs(api-tokens): deep rewrite for v1, expand to fleet automation guide (#1140) ... Rewrite docs/features/api-tokens.mdx (115 → 442 lines) as a full product + technical guide: mental model, scope ladder, prerequisites, step-by-step usage with HTTP/WS/multi-node examples, complete universal- restriction table, cross-node proxy behaviour, lifecycle, rate-limit ceiling, security model, limitations, three example workflows, troubleshooting accordion, FAQ accordion. Replace the single populated-list screenshot with five captured against production: empty state, create form, reveal banner (token value redacted), populated list with all three scope variants, revoke modal. Sibling-doc edits keep tier statements coherent now that API tokens are available on every tier: - docs/api-reference/overview.mdx: drop the Admiral-only Note callout, drop API Tokens from the Admiral-gated license-tier table, correct the rate-limit table to say tokens are keyed per-credential - docs/features/overview.mdx: drop the "Admiral only." sentence - docs/security.mdx: drop "Admiral tier.", move API tokens row to every tier in the security matrix, repoint image to the new populated shot	2026-05-21 21:20:59 -04:00
Anso	66b84932e0	feat(notifications): move Notification Routing to Skipper tier (#1145) ... * feat(notifications): move Notification Routing to Skipper tier Notification routing is automation (route alerts to channels by rules), not enterprise compliance. Aligning the gate with Skipper makes the tier boundary read consistently with the rest of the automation surface (webhooks, auto-update, auto-heal, scheduled tasks). Backend: requireAdmiral -> requirePaid on the five /api/notification-routes endpoints. Dashboard configuration-status now exposes the routing-rules row to any paid tier. Frontend: settings registry tier flipped to skipper; the Admiral wrapper around NotificationRoutingSection is removed (the inner CapabilityGate stays, preserving forward-compat with older remote nodes). Tests: added a tier-enforcement describe block covering Skipper (200) and Community (403 PAID_REQUIRED on all five endpoints). Docs: refreshed alerts-notifications, licensing, overview, dashboard, troubleshooting, and reference/settings; cleaned one fence-spec line per Directive 31. * fix(notifications): address audit findings on tier-move PR Docs: rewrite three lines that survived the initial sweep. The dashboard "you do not see a locked placeholder" clause and the settings.mdx "hidden on Community and Skipper" phrase were Directive 31 fence-spec. The alerts-notifications troubleshooting note still said "an Admiral routing rule" and contradicted the tier move. Tests: the Community-negative cases on POST/PUT/DELETE/POST :id/test could not distinguish requirePaid from a stray requireAdmiral, because Community fails on the tier check before variant is read. Adding Skipper-positive coverage per endpoint locks the gate identity in. Replace the leaky mockReturnValueOnce with a per-test mockReturnValue plus an afterEach restore so spies cannot bleed across tests.	2026-05-21 20:57:34 -04:00
Anso	535023b350	feat(files): open stack file explorer to every tier (#1144) ... * feat(files): open stack file explorer to every tier Drop the `requirePaid` guard from the seven stack-file write routes (download, upload, write-content, delete, mkdir, rename, chmod) and remove every matching `isPaid` check from the file-explorer frontend. Stack edit permission (RBAC) continues to gate every write end-to-end. The file explorer is the primary way a user touches a stack's on-disk surface; gating it behind a paid tier conflicted with the principle that Community covers single user-initiated actions while paid tiers add automation and governance. * docs(files): treat download as a read action, not a write Download has no `requirePermission('stack:edit')` on the route and no `canEdit` gate in the UI, so viewer accounts can download. Update the top paragraph to list download under reads, and rewrite the troubleshooting accordion to describe the actual gating (a file must be selected) instead of asserting a role gate that does not exist. * test(e2e): align stack-files spec with the new tier rule The community-tier describe block asserted that the Upload control is absent and the editor shows a `Read-only` chip; the admin-tier block skipped on Community via `test.skip(tier !== 'paid')`. Both rules reflected the previous gate, where writes required a paid tier. Writes are now gated on the `stack:edit` role, not on the license tier. Repurpose the community describe to assert that a Community admin under a mocked community license still sees the Upload control and an editable Save button. Drop the obsolete tier-skip in the admin describe so upload, edit, delete, and download exercise on every tier. Update stale comments to reference the role gate.	2026-05-21 19:35:31 -04:00
Anso	380ed6fd50	feat(cloud-backup): make Custom S3-compatible target available on every tier (#1143) ... * feat(cloud-backup): make Custom S3-compatible target available on every tier Sencho Cloud Backup remains an Admiral feature; the bring-your-own-bucket Custom S3 target is now reachable on Community and Skipper as well. Backend splits the per-route Admiral gate into two helpers: operations that touch the saved provider use gateForCurrentProvider, PUT /config uses gateForRequestedProvider against the body. /provision and /usage stay requireAdmiral because they are Sencho-only by definition; GET /config is ungated so any tier can read its own stored configuration. Frontend drops the AdmiralGate wrapper on the Cloud Backup section, filters the Sencho provider option out of the dropdown for non-Admiral users, and gates the per-snapshot cloud-upload affordance on "cloud-backup configured" instead of Admiral tier. Dashboard Configuration row is no longer locked on lower tiers. Sidebar registry tier on cloud-backup goes from 'admiral' to null. Docs and licensing breakdown restate the rule once per page without fence-spec. * fix(cloud-backup): keep downgraded sencho config off the upload surface If an Admiral configured Sencho Cloud Backup and the license later drops to Skipper or Community, the saved provider is still 'sencho'. The FleetSnapshots cloud-upload affordance now requires either provider= custom (every tier) or provider=sencho with an active Admiral license, so a downgraded admin never sees an upload button that the backend would 403 on click. Also tidies the Fleet Backups doc, which still claimed the cloud-upload icon was Admiral only; the icon now renders whenever a Cloud Backup target is configured.	2026-05-21 18:11:16 -04:00
Anso	c491d309c1	feat(schedules): make every scheduled action available on Skipper (#1141) ... Collapse the Admiral carveout that restricted restart, prune, auto_backup, auto_stop, auto_down, and auto_start schedules to the Admiral variant. Scheduled Operations stays at Skipper+ (paid). The action picker now lists every supported operation for any paid admin, and the scheduler runner executes every action on either variant.	2026-05-21 16:42:41 -04:00
Anso	d0e140444a	feat(api-tokens): make API tokens available on every tier (#1136) ... API tokens are credential management, not a tier-gated capability. Remove the Admiral gate from the POST/GET/DELETE handlers, drop the AdmiralGate wrapper from the settings UI, set the registry entry's tier to null so the tab renders on every tier, and update the docs Note to state availability plainly. The three permission scopes (read-only, deploy-only, full-admin), the 25-token-per-user cap, the per-token 200 req/min rate limit, the sen_sk_ prefix format, and the SHA-256 hashed storage are all unchanged. The Vitest suite now runs at Community tier to prove every code path works without a paid license. A new "API token tier accessibility" describe block mints all three scopes via POST /api/api-tokens to lock the behavior.	2026-05-21 11:45:37 -04:00
Anso	9090de3a38	docs(licensing): refresh tier prices and rename Lifetime to Founder Lifetime (#1134) ... Update the docs pricing table and surrounding prose to match the website: - Skipper: $5.75/mo billed yearly ($69/yr), $9.99/mo, $149 Founder Lifetime - Admiral: $20.75/mo billed yearly ($249/yr), $39.99/mo, $499 Founder Lifetime - Enterprise: custom pricing (was a fixed annual minimum) - Rename the lifetime cycle to "Founder Lifetime" in the table column, the EA blurb, and the trial-section caveat. The internal license-duration type ("DURATION: lifetime" masthead pill, "Lifetime licenses" subsection) is unchanged because it describes app runtime behavior, not the SKU.	2026-05-21 10:18:24 -04:00
Anso	e65c5e8551	fix(pilot): post-merge audit followups (WS via getProxyTarget, closeTunnel lifecycle, mesh source buffer, docs) (#1128) ... * fix(pilot): route remote WS upgrades through NodeRegistry.getProxyTarget Pilot-mode nodes carry empty api_url and api_token by design and expose their API on a per-tunnel loopback bridge. The upgrade handler gated the remote-forwarder branch on `node.api_url && node.api_token`, so WS requests targeting pilot nodes silently fell through to the local handlers (live logs, exec, generic) instead of tunneling to the agent. Resolve the target via NodeRegistry.getProxyTarget so pilot and proxy modes share one dispatch path, mirroring the HTTP proxy. handleRemoteForwarder now takes the resolved target and, when the target is the pilot loopback (empty token), skips the console-token exchange and the Authorization injection so the tunnel-side auth is the only source of truth on that path. Unresolvable targets reject the upgrade with HTTP 503 instead of being served gateway-local data. * fix(pilot): emit tunnel-down and mark node offline on closeTunnel PilotTunnelManager.closeTunnel closed the underlying WebSocket but skipped the cleanup the natural-disconnect path runs, so explicit closures (enrollment regenerate, node deletion) left the node row at status='online' until the next reconnect. The dashboard kept showing the stale state for the entire interval. closeTunnel now writes nodes.status='offline' and emits tunnel-down for pilot bridges, and emits proxy-bridge-down for central-initiated proxy bridges. The maps are cleared before bridge.close() so the natural 'closed' handler's bridge-identity guard short-circuits and we do not double-emit. * fix(mesh): buffer cross-node source data until tcp_open_ack arrives openCrossNode piped src socket data straight to tcpStream.write before the forward TcpStream emitted 'open'. The first packet on a fresh cross-node stream raced ahead of the agent's tcp_open_ack on the wire, which broke protocols that send immediately after connect (HTTP, TLS, Redis, Postgres) on Pilot and proxy mesh paths. Buffer src chunks in a local array capped at STREAM_PENDING_DATA_MAX_BYTES until tcpStream emits 'open', then flush them in order before any post-open writes. Tear down both sockets if the buffer overflows so a misbehaving source cannot exhaust gateway memory while waiting for the ack. * docs(pilot): clarify host-console non-parity and narrow the parity claim Pilot mode disables the host-console capability at the capability registry (the agent container has no useful host shell to surface), but the public docs listed host console among the WebSockets that ride through the tunnel and described pilot as behaving identically to proxy mode. State the shared-capability claim more carefully and call out the intentional non-parity in a dedicated subsection.	2026-05-21 01:00:47 -04:00
Anso	08caa914ce	docs: v1 docs refresh (batch 2) (#988) ... * docs(atomic-deployments): refresh page around current UI and behavior Rewrites the page to match the v1 docs refresh template. Corrects several factual errors against the current code, fills in missing detail, and adds a screenshot of the rollback overflow menu. Notable corrections: - Scheduled tasks do not run atomically; only stack editor Deploy and Update, App Store installs, webhook triggers, and image auto-updates pass the atomic flag through to ComposeService. - Rollback lives in the stack editor's More actions overflow menu, not on the action bar directly. The backup timestamp renders as a sub-line of the menu item. - Health probe is a 3-second window with an exit-code check on every container labelled with the compose project name; describe this exactly rather than as 'waits briefly'. - Document where backups live (DATA_DIR/backups/<stack>/), why they are kept outside the compose folder, and that the slot is one per stack with overwrite semantics. - Document the four streamed log markers users see in the deploy progress modal during the atomic flow. - Add a troubleshooting accordion group covering missing menu entry, late crashes outside the probe window, manual-intervention message, and the single-slot retention edge case. * docs(deploy-enforcement): refresh page for v1 and align with current enforcement paths Update the page to match the current pre-flight gate behavior, the v1 modal chrome on the block dialog, and the AccordionGroup troubleshooting pattern used across the v1 docs. Drift items corrected: - Replace the broken vulnerability-scanning/deploy-blocked-dialog.png reference with three fresh captures under docs/images/deploy-enforcement/ (policy list, policy editor, block dialog). - Drop "Recreate from the stack actions menu" and the git-source apply pre-flight claim; neither path runs the gate. - Add bulk label deploy and the auto-update scheduler to the enforced code paths, with a dedicated subsection for the auto-update interaction (alert-and-skip, not 409). - Drop the false claim that severity chips in the block dialog are clickable; the dialog is informational. - Document the compose-parse-fails-closed branch with its synthetic violation label. - Refresh dialog copy to reflect the v1 ModalDestructiveHeader (kicker, title, button variants). - Convert the troubleshooting Q&A into AccordionGroup blocks and add accordions for the compose-parse-error case and the auto-update-skipped case. - Quote the verbatim audit-log summary format. * docs(blueprints): refresh against v1 UI and add federation/state-review coverage * docs(git-sources): refresh page against v1 UI and current behavior Rewrites the page against the v1 docs refresh template (Note tier-gate, sectioned anatomy, AccordionGroup troubleshooting), aligning prose with the live UI labels and the current code paths. Corrections: - Authentication toggle reads "Public (no auth)" / "Personal Access Token" (not "None"), and apply mode "Auto-write files" (not "Auto-write"). - Diff dialog kicker is GIT . PULL PREVIEW; local-edits state opens an Overwrite local edits? confirmation modal whose primary button is Overwrite and apply. - Sidebar pending indicator is a small GitBranch icon, not a brand-color dot, and the image-update dot takes priority over it on the same row. - Pending update banner appears in the panel; Review re-fetches the commit and opens the diff (no client-side payload caching). Adds coverage for: - Anatomy of the panel (pending banner, form, last-applied stat strip, footer actions). - 10-second webhook debounce window. - Pending compose/env content is encrypted at rest in the database, not just the token. - Auth/host failures map to HTTP 400, never 401, so they do not sign the user out. - Per-stack lock serializes pull, apply, and create-from-git so a webhook firing during a manual apply waits rather than racing. - Compose validation has a 10-second budget; clone fetches have a 30-second timeout. - New troubleshooting accordion for Pending commit has changed since this pull was fetched. Recaptures all five screenshots from the v0.74.x production node, signed in as admin: panel, create-from-git tab, pull-preview diff dialog, sidebar GitBranch pending icon, webhook Action select with Git source sync highlighted. * docs(stack-labels): refresh page for v1 sidebar grouping and fleet-action surface - Lead with the v1 behavior the previous page did not cover: the sidebar groups stacks under collapsible label headers (PINNED first, label buckets sorted by stack count desc then name asc, UNLABELED last) with a count chip per group. Trailing colored dots on each row (max 3 + N overflow, paid-only) supplement the headers. - Drop the stale claim that a label-pill filter bar lives between search and the stack list; that UI no longer exists. - Drop the right-click-on-pill bulk actions table (Deploy all / Stop all / Restart all). The legacy per-node action endpoint stays in the backend but no longer has a UI binding, so the page documents only what users can click today. - Document the two Skipper+ Fleet Action cards: Stop fleet by label (name match across nodes, autocomplete, per-node breakdown, HTTP 429 on per-node concurrency) and Bulk label assign (per-node, replace semantics, clear on empty selection). - Document the inline 'New label' form inside the stack right-click / three-dot Labels submenu, the Settings - Advanced - Labels masthead N/50 stat, the LABELS - NEW / EDIT modal kickers, and the LABELS - DELETE - IRREVERSIBLE confirmation copy verbatim. - Document the Fleet Overview Tags multi-select filter (filters by stack labels aggregated across nodes), with cross-link to fleet-view. - Capture every screenshot fresh from production signed in as admin: sidebar-grouping, context-menu-labels, inline-create-form, settings-labels, create-label-dialog, fleet-tags-filter, fleet-actions. Drop the now-stale sidebar-with-labels, sidebar-filtered, and bulk-actions-menu captures. * docs(dashboard): refresh page for v1 layout (status masthead, gauges, fleet heartbeat, restart map) Aligns docs/features/dashboard.mdx with the redesigned Home tab. Replaces the obsolete Recent Activity feed coverage with the actual DashboardActivityCard split (Fleet Heartbeat when remote nodes are registered, Stack Restarts (7d) otherwise) and recaptures every screenshot from the v0.74.x production node. * docs(global-search): refresh page for v1 palette - Note tier and role gating on the Pages list (Auto-Update, Console, Schedules, Audit) so the prose matches what the top bar exposes. - Document the ACTIVE chip on the currently active node row. - Document the 50-result cap counter and the Searching... loading state. - Mention the ~250 ms debounce and clarify that filename matching includes the file extension. - Replace stack screenshot with a redesigned capture and add empty-state Pages and Nodes captures showing the ACTIVE chip. * docs(global-observability): refresh page for v1 layout (masthead, signal rail, filter strip, paused-resume chip) Full rewrite against the current Logs tab and the v1 docs refresh template (hero Frame, sectioned anatomy, AccordionGroup troubleshooting, refresh-cadence table). Replaces the single overview screenshot with seven captures under docs/images/global-observability/ (overview, masthead, signal-rail, filter-strip, feed-bands, paused-resume-chip, error-only-filter), all from the v0.75.x production node signed in as admin with PII scrubbed (profile chip patched to AD, in-feed LAN IPs and third-party hostnames substituted via DOM injection while the stream was paused). Aligns prose with the actual UI labels and code: - Masthead kicker reads LIVE LOGS · NODE · <NAME> with LOCAL for the local node; state word toggles Streaming / Idle / Offline; SESSION uses uppercase letter suffixes (1H 43M / 0M 12S) per formatUptime. - Signal rail tile counts are scoped to the 2000-entry buffer and reset with Clear; CONTAINERS is buffer-bound, not a monotonic accumulator. - Filter strip controls quoted verbatim (Stacks · All / Stacks · n, segmented controls All / Out / Err and All / Info / Warn / Error). - Feed row anatomy: severity dot, timestamp, brand-cyan container name with stack/container tooltip, message tinted by source. Row tint follows detected level, which is regex-based, so an STDOUT line containing ERROR: still classifies as ERROR. - Day bands: NOW, Nm AGO, Nh AGO, calendar date. - Empty states: two-tier kicker over caption (Awaiting events / No matches). - Pause keeps the SSE buffer filling up to the 2000-entry cap; resume pill reads <n> NEW · RESUME and counts the queue, not total arrivals during the pause. - Download filename and row format quoted: sencho-logs-<ISO8601>.txt and [<ISO>] [<stack>/<container>] <LEVEL>: <message>. Documents behavior the previous page never covered: - Active-node scoping; node switch resets the stream and the buffer. - SSE primary transport with 30-second server heartbeat and a 5-second polling fallback against /api/logs/global (server-capped at 500 lines per snapshot). - Initial replay of the last 500 lines per container when the SSE connection opens, so the feed has context immediately. - Display limits (2000 client buffer, 300 rendered rows, Showing last 300 of N overflow notice). - Refresh cadence table covering UI tick, flush cadence, polling cadence, SSE heartbeat, sparkline window, and the Idle threshold. Adds a seven-accordion troubleshooting block (Offline state, gray Idle dot, ERROR-without-tint, growing Resume pill, Clear-cutoff lag, node-switch buffer drop, fleet-wide aggregation expectations). Tightens the closing Note so it makes clear that Notification Log Retention does not govern this live container stream. * docs(alerts-notifications): refresh page for v1 and absorb notification-routing Full v1 template rewrite of /features/alerts-notifications. Bundles in the entire Notification Routing page so a reader sees channels, routing, per-stack rules, and retention in one place; deletes the standalone notification-routing.mdx and points all five cross-link sites at the new in-page anchor. * docs(alerts-notifications): drop "What's not in scope" section The page should describe what Sencho does, not enumerate what it does not ship. Users find missing integrations through the Webhook section and the routing matcher reference; the explicit disclaimer added noise without adding guidance. * docs(audit-log): refresh page for v1 layout, expanded action list, troubleshooting accordion - Clarify that the search/method/date filter strip lives in Table view only. Stream view always shows the unfiltered chronological feed. - Fold the total-entries readout into the card subtitle wording where it actually renders, instead of describing it as a separate header element. - Sharpen the Peak hour off-hours window to the literal 08:00 to 17:59 working window the tile keys off, plus the 5% / 20% failure-rate tints. - Note that the Actors tile names a sample actor alongside the new-IP count. - Expand the example actions list to cover surfaces that have shipped since the last edit: per-service stack lifecycle, node cordon/uncordon, fleet replica role changes, Sencho Cloud Backup operations, Fleet Secrets, and blueprint federation pin updates. - Correct the Settings path: Settings · Developer · Data retention card, Audit log input, Save settings button. - Add a Troubleshooting AccordionGroup matching the rest of the v1-refresh pages: missing tab, filter scope, anomaly thresholds, export cap, and retention pruning. - Replace all four screenshots with fresh captures of the current UI. * docs(multi-node): refresh page for v1 layout, pilot agent mode, refreshed table columns Rewrites docs/features/multi-node.mdx against the current product. The previous page predated the v1 Settings hub redesign and the Pilot Agent enrollment model, so it documented only the Distributed API Proxy add-node flow and missed the new Mode, Endpoint, and Labels columns on the Nodes table. Restructures the page into 13 sections: intro, How it works, the local node, Choose a remote mode (decision table comparing Pilot Agent vs Distributed API Proxy), Add a remote node: Pilot Agent (three steps plus re-enrollment), Add a remote node: Distributed API Proxy (three steps), Switching between nodes, the Nodes table (full column reference), What Settings apply per node (verified against settings/registry.ts), License enforcement across nodes, Editing and deleting nodes, Security (token security, transport encryption, why no application-layer TLS), and Troubleshooting (AccordionGroup matching the v1 template used on audit-log, atomic-deployments, and deploy-progress pages). Refreshes seven screenshots against the production node signed in as admin, scrubbing IPs and usernames before capture: full Nodes panel overview, Generate Node Token card with a placeholder token, Add node modal in Pilot Agent mode, Add node modal in Distributed API Proxy mode (with the inline plain-HTTP warning visible), Edit modal showing the Regenerate enrollment token card for a pilot agent, Pilot enrollment modal with the docker run command, refreshed node switcher popover, and a close-up of the table columns. Drops the obsolete add-node-form.png, http-warning.png, and per-node-scheduling/ folder. * docs(fleet-view): refresh page for v1 layout, expanded tabs, cordon, sheet-based updates - Aligns the Overview, Status, and Node Updates content with today's UI: the masthead's `The fleet` headline plus CPU / MEM / CONTAINERS stat tiles, the eight-tab strip (Overview, Snapshots, Status, Deployments, Traffic, Federation, Fleet Actions, Secrets) with per-tier visibility, and the Check Updates surface that is now a system sheet rather than a modal. - Documents the toolbar (search, sort, filter popover with Status / Type / Severity / Tags sections) and the Grid / Topology segmented control including the topology graph's status pill (Online / Critical / Offline), connector colouring, ReactFlow controls and minimap. - Documents the per-card surfaces that were missing from the prior page: Cordoned badge with cross-reference to Fleet Federation, fleet stack label dots in the drill-down, container drill-down rows (state dot, badge, image, status, open-in-editor hover button), and the Admiral three-dot Node actions menu for cordon / uncordon. - Documents the Node Updates sheet anatomy (Recheck and Update all (n) header actions, four summary cards, node table columns, Update flow, reconnecting overlay timing, admin enforcement) and the GitHub Releases with Docker Hub fallback resolution path with its 30-minute cache. - Replaces every stale screenshot with a fresh capture (overview, topology, drill-down, status tab, node updates sheet) and removes the obsolete files plus the empty docs/images/fleet/ folder. - Reformats troubleshooting as an AccordionGroup matching dashboard, multi-node, and audit-log refreshes. * docs(fleet-backups): refresh page for redesigned fleet and settings UI Replace all six screenshots with current production captures. Update content to reflect the new fleet header card, eight-tab layout, full- page Cloud Backup settings with header stats, and corrected navigation paths. Add cloud backup rows to the access control table. * docs(fleet-backups): convert troubleshooting to AccordionGroup pattern Match the foldable-accordion pattern used across the v1 docs refresh batch. Merges the standalone Cloud Backup troubleshooting subsection into a single Troubleshooting section at the bottom of the page with seven accordions covering skipped nodes, two restore failure modes, three cloud-upload failure modes, and a diagnostic logging entry. * docs(remote-updates): refresh page for v1 sheet, accordion troubleshooting, factual fixes Rewrites the page against the v1 docs refresh template (Note tier gate, sectioned mechanism deep-dive, Frame screenshots with detailed alt text, inline AccordionGroup troubleshooting), bringing it in line with the recently-refreshed fleet-view, fleet-backups, dashboard, and audit-log pages. The page is repositioned as the mechanism deep-dive (prerequisites, what runs on a node during an update, completion and failure detection, recovery actions). The full UI tour for the Node updates sheet remains in fleet-view so the two pages stop overlapping; remote-updates now links into fleet-view#node-updates instead of restating the table anatomy. Captures three screenshots from the production node, signed in as admin: fleet-node-updates.png shows the Node updates sheet with eight nodes and seven remote updates available; local-update-confirm.png shows the LOCAL · UPDATE alert dialog with the Cancel and Update & restart buttons; node-card-update-available.png shows the Opsix card with the Update available pill and the Update to v0.76.7 outline button. Corrects several factual claims that no longer matched the current code: - The remote early-fail threshold is about 3 minutes, matching EARLY_FAIL_MS in backend/src/routes/fleet.ts, not 90 seconds. - The Recheck button sits in the sheet header, not the footer. - The component is a SystemSheet, so the page now consistently calls it the Node updates sheet instead of a dialog, with lowercase "Node updates" and lowercase "Update all (n)" matching the live UI. - Reconnecting overlay polls /api/health every 3 seconds, not "every few seconds". - The local Failed badge surfaces as soon as the helper writes its error file, by the 3-minute mark at the latest. Documents the LocalUpdateConfirmDialog kicker, title, body, and CTA verbatim, the Triggering... loading state on the Update buttons, the four completion signals the gateway accepts (version change, process startedAt change, offline-then-online transition, version at or above the comparison target after 15 seconds), and the 60-second auto-clear of the Updated badge. Drops references to two screenshots that never existed (fleet-node-updating.png, fleet-node-failed.png); the in-flight and failed states are described in prose instead, the same way fleet-view handles them. * docs(scheduled-operations): refresh page for v1 timeline, fleet-wide update action, sheet-based run history Rewrites the Scheduled Operations page against the v1 template (Note tier gate, sectioned anatomy, Frame screenshots, AccordionGroup troubleshooting) applied to sibling pages in this batch. Captures seven fresh screenshots against the production node signed in as admin (timeline, all-tasks, action-picker, create-restart, create-prune, create-scan, run-history) and removes every legacy PNG. Documents the new "Auto-update All Stacks" action that was absent from the page, extends the Skipper allow-list to all four Skipper actions (Auto-update Stack, Auto-update All Stacks, Fleet Snapshot, Vulnerability Scan) and clarifies that the action picker hides operations the active tier cannot run. Corrects several factual claims that no longer matched the code: - Scheduled scan completion is `info`/`scan_finding` on a clean run and `warning`/`scan_finding` when findings are present (not `info`/`system` as previously stated). Cross-link now points at `alerts-notifications#vulnerability-scanning`. - Lifecycle actions (auto_backup, auto_stop, auto_down, auto_start) execute against the local Sencho instance only; only Auto-update Stack / All Stacks have a remote-proxy code path. The page reinstates the guidance to schedule remote lifecycle operations from that node's own UI. - Run history lives in a right-side sheet with a "Schedules › <task> › Runs" breadcrumb and a Download CSV secondary action. - Timeline masthead is described in terms of the v1 visual (`NEXT 24 HOURS` kicker, italic display heading, monospace date range, right-anchored Next pill with countdown, glowing cyan now rail, six-tick bottom axis). * docs(rbac): refresh RBAC & user management page against v1 template Bring /features/rbac onto the v1 docs refresh template (Note tier gate, sectioned anatomy, Frame screenshots, AccordionGroup troubleshooting). Recapture five screenshots from the production node signed in as admin and remove the three stale captures under docs/images/rbac/. Corrections vs. the prior page: - Deployer no longer claims node:read in the permission matrix; the backend grants only stack:read and stack:deploy. - Add the system:registries row (container registry management). - Document the form as inline below the Add user button (not a modal). - Note the (you) marker on the signed-in admin's row and the disabled delete icon on that row. Additions: - Settings nav location and hub-only visibility. - 2FA reset row action with verbatim modal kicker, title, and body. - Five-failure / 15-minute MFA lockout behavior and admin reset recovery. - Token-version session-security table covering deletion, role change, password change, and admin 2FA reset. - SSO password-fields-hidden line quoted verbatim and the per-provider Require MFA toggle. - Audit-log emissions list for every user-management mutation. - API tokens cross-link explaining the user-vs-machine boundary. - Scoped permissions section retightened: scoped role picker is Deployer / Node Admin / Admin only; resource type is Stack or Node. AccordionGroup with eight troubleshooting entries covering missing nav, greyed role options, seat-limit errors, unexpected sign-outs, scoped deployer mismatches, missing shield icon, re-locking MFA accounts, and SSO role drift at provisioning. * docs(2fa): refresh two-factor authentication and admin guide against v1 template Bring /features/two-factor-authentication and /operations/two-factor-admin onto the v1 docs refresh template (Note tier gate, sectioned anatomy, Frame screenshots with descriptive alt text, AccordionGroup troubleshooting, verbatim modal copy with kicker callouts). Recapture every screenshot under docs/images/two-factor-auth/ from a fresh session and add six new captures for surfaces the prior page did not document. Corrections vs the prior pages: - Panel rename: Settings -> Account & Security is now Settings -> Account, under the Identity group of the settings sidebar. Replaced every occurrence on both pages. - Enrol dialog titles match the current modal: Pair your authenticator, Confirm the pairing, Save your recovery codes (was: Set up 2FA, Confirm, Save your backup codes). Step rail 01 PAIR / 02 CONFIRM / 03 ARCHIVE documented. - Manual-entry affordance is the always-visible Secret manual entry row with a copy icon, not the toggleable Can't scan Show secret key link. - Confirm step auto-submits on the sixth digit; no submit button. Verified in MfaChallenge.tsx and MfaEnrollDialog.tsx and called out explicitly. - Authenticator-app list trimmed to match in-app copy (1Password, Bitwarden, Google Authenticator, or any TOTP app). Authy and Microsoft Authenticator dropped because the dialog does not mention them. - Disable dialog: kicker SECURITY MFA DISABLE, title Turn off two-factor, destructive header, Disable button. Replaces the prior Disable 2FA paragraph that did not describe the dialog chrome. - Regenerate dialog: two-step flow with kicker SECURITY BACKUP CODES, Confirm identity then New recovery codes, with the verbatim PREVIOUS CODES HAVE BEEN INVALIDATED warn rail on the show step. Documented that the dialog only accepts a TOTP, not a backup code. - Per-user SSO toggle label corrected: Require 2FA on SSO sign-in (was: Require 2FA even when signing in via SSO). Added the per-provider vs per-user distinction on both pages (admins can also enable Require MFA on the SSO provider config, which is independent of the per-user toggle). - Admin reset modal: verbatim USERS RESET 2FA kicker, Reset 2FA for <username> title, full-body copy reproduced. Documented that the reset bumps the target's token version and invalidates active sessions. Additions: - Sign-in throttle: five failed verifications lock the account for 15 minutes, server returns 423 with Retry-After, UI shows the Retry in MM:SS countdown plus Rate limited label. Lockout recovery section explains that the counter only clears on a successful sign-in, so retries after the window expires re-lock immediately. - Account panel anatomy section enumerates the three rows (Authenticator app, Backup codes, Require 2FA on SSO sign-in) plus the destructive Disable 2FA link, and the masthead 2FA on / BACKUP N left chips. - Recovery codes section now covers all three count states (3 plus, 1 to 2, 0) with verbatim helper text, tone, and the standalone No backup codes left callout that renders at zero. New screenshots for the 2-remaining and 0-remaining states. - Cross-references to the admin operations page (CLI fallback, token version rotation, what a reset changes in the DB), the SSO page, and the RBAC page (per-provider Require MFA toggle, SSO auto-provisioning). Troubleshooting on the feature page rewritten as an AccordionGroup with nine entries: clock drift, wrong account selected, QR will not scan, lost phone with no codes, lost codes with authenticator, ran out of codes, unexpected SSO prompt (with both toggle causes), repeated lockout after the window expires, missing shield icon on Users panel. The admin operations page also gains the SSO + 2FA two-toggles table so administrators can answer the per-user vs per-provider question without context-switching between pages. Six new images added; six existing images replaced. Total 14 captures. * docs(rbac,host-console): drop enforcement-boundary detail from tier-gate notes Operator-facing docs should state tier or role requirements once, in plain customer-facing language, and leave the enforcement chain to the source. Two surfaces on the v1-refreshed pages over-specified the gate: - `features/rbac.mdx::Scoped permissions`: the Note enumerated both the UI hide on Skipper and the `/api/users/:id/roles` write rejection. The first half ("Scoped permissions require Admiral.") is the operator-relevant fact; the rest reads as a fence specification, which is awkward for an open-core product where the gate is readable in source anyway. Trimmed to just the tier claim. - `features/host-console.mdx::Availability`: the paragraph already says who can use the console and that the Console tab is hidden on Community or Skipper. The trailing "Attempting to access the console endpoint directly without the correct license or role is rejected" is the same bypass-prevention coda. Dropped. No functional behavior change; the gates themselves are untouched. * docs(sso): refresh SSO & LDAP authentication page against v1 template Rewrites docs/features/sso.mdx against the v1 docs refresh template (intro + tier callout, sectioned Configuration anatomy, Frame screenshots, AccordionGroup troubleshooting), bringing it in line with the previously refreshed two-factor-authentication and rbac pages on this branch. Recaptures all four screenshots from the production node signed in as admin: sso-settings (overview with the five collapsible provider cards), sso-settings-ldap (LDAP form expanded), sso-settings-oidc (Google form expanded), sso-settings-custom-oidc (Custom OIDC form expanded with all eleven fields). Refreshes the Settings UI section to match the redesigned panel: each provider is a collapsible card with an Active badge on the header, an enable / disable toggle pill, and a footer with Save, Test Connection (green check or red X next to the button), and Remove (only after a config has been saved). Documents the static callback-URL helper that sits below all five cards. Clarifies that the per-OIDC claim mapping environment variables (SSO_OIDC_*_ID_CLAIM, *_USERNAME_CLAIM, *_EMAIL_CLAIM) are accepted for Google, GitHub, and Okta, not just Custom OIDC. The Settings UI hides those fields on the presets because the defaults match. Converts the troubleshooting section to an AccordionGroup with five entries (Test Connection discovery failure, issuer validation error, wrong username or missing email after sign-in, invalid redirect URI, SSO buttons missing on the login page). Cross-links the operations troubleshooting page for setup-time errors. Tightens the LDAP TLS env var note to spell out the literal string 'false' requirement. Syncs the Combining SSO with 2FA section to use the live toggle label 'Require 2FA on SSO sign-in'. * docs(sso): drop the Community-tier Custom OIDC workaround tip The Tip walked through how a Community-tier operator could integrate Google, GitHub, or Okta by pointing Custom OIDC at the provider's discovery URL, bypassing the Skipper preset gate. Operator docs should state the tier rule once and stop; they should not describe how to circumvent it. The tier matrix above the removed block already names which providers are paid; the Custom OIDC row already lists "any spec-compliant OIDC provider" as its scope. That is enough. * docs(vulnerability-scanning): refresh page for v1 UI and corrected tier mapping The page was last revised before the v1 visual redesign and before the tier-mapping changes shipped in v0.81.2 (open Community access to secret scanning, compose misconfig scanning, scan history, and scan comparison). This refresh: - Rewrites the tier matrix to match the shipped Community / Skipper / Admiral split. Secret detection, compose misconfig scanning, scan history, scan comparison, and misconfig acknowledgements are now correctly marked as Community. Scheduled fleet scans, scan policies with block_on_deploy, SBOM, SARIF, and Trivy auto-update stay paid. - Drops two stale Notes that said secret detection and compose misconfig scanning required Skipper or Admiral. The page now states each tier requirement once, in plain language. - Refreshes all six existing screenshots from the production node: resources-badges, scan-details-sheet, scan-history-sheet, scan-compare-sheet, security-settings, app-store-toggle. - Adds a new scan-config-button screenshot showing the stack-page overflow menu where Scan config now lives. - Describes the scan drawer header accurately: Re-scan + Compare + CSV + SARIF as top-level buttons, with SBOM as a separate button below the summary. - Updates the compose misconfig flow to point at the stack overflow menu (not the Deploy controls). - Converts the troubleshooting section to a single AccordionGroup per the v1 template, and audits each entry for legacy phrasing and the removed tier claims. - Adds a TRIVY_BIN reference to the How it works section so operators know about the host-binary override. * docs(cve-suppressions): refresh page for v1 UI and corrected suppression specifics - Recapture all three screenshots from the production node signed in as admin under `docs/images/cve-suppressions/` (`settings-panel`, `create-dialog`, `suppressed-row`). The previous file referenced three image paths that did not exist in the repo. - Align prose with the actual UI labels: - Dialog kicker `SUPPRESSIONS . NEW`, title `New suppression`. - Field labels match the form: `CVE or advisory ID`, `Package (optional)`, `Image pattern (optional)`, `Reason`, `Expires in (days, optional)`. - Remove confirmation reads `Remove suppression` with kicker `SUPPRESSIONS . REMOVE . IRREVERSIBLE`. - Factual corrections: - Fleet sync truncation cap is 5,000 rows (not 10,000). - State the admin-role requirement once in the lead Note. - Drop references to a `Fleet . Sync status` page and a `Reanchor` button; neither exists in the UI. The reanchor flow is an admin API call and is documented in /features/fleet-sync. - Sharpen the specificity scoring section (package + image scores 3, package only 2, image only 1, neither 0) so the order matches the read-time filter logic. - Note that the image-pattern glob is case-sensitive. - New coverage: - Suppressing directly from a scan result, including which fields are read-only in that inline flow and when to fall back to Settings to broaden scope. - The `replicated` and `expired` row badges in the panel. - Hovering the package column on a suppressed row to surface the Reason. - Two distinct read-only modes: viewing a remote node from the hub (panel hidden, banner shown) versus signing into a replica instance (panel visible, read-only). - SARIF export carries suppressions through as `kind: external, status: accepted`, cross-linked to the Vulnerability Scanning page. - Convert troubleshooting to AccordionGroup with six entries; update the truncation entry to reflect the 5,000-row cap. * docs(private-registries): refresh page for v1 UI and fleet-wide credential model Rewrites the page against the v1 docs template (Note tier gate, opening Frame, sectioned anatomy, AccordionGroup troubleshooting) and replaces every screenshot with a fresh capture taken against the current product. Corrects several factual claims that no longer matched the current code: - Registries are stored once on the control instance and applied fleet-wide, not configured per node. The old Multi-node behavior section and the matching troubleshooting entry described a per-node model that the product no longer has. - The Registries section is hidden on remote nodes (global scope) and on Sencho versions that do not surface the feature. New troubleshooting entries explain both visibility states. - The feature is admin-only on Admiral. Non-admin operators do not see the section even on Admiral; previous copy implied any Admiral license user could manage credentials. - Registry endpoints are not reachable from API tokens; only an admin browser session can manage credentials. The Security section now states this without naming internal route paths. Documents UI behavior the previous page omitted: the inline form (not modal), the four type-specific form variants, the Docker Hub read-only URL field, the destructive delete confirmation with its stack-pull warning, the masthead REGISTRIES count, and the empty-state callout copy. Screenshots replaced: - registries-overview.png: section with one configured GHCR card and the masthead stat at one. - registries-empty.png: empty state with the Add registry button and callout. - registries-add-form.png: inline form with the Docker Hub default and the read-only URL field. - registries-ecr-form.png: form switched to ECR, showing the AWS Region field and the relabelled AWS credential inputs. - registries-card-detail.png: card close-up with the three action icons and the metadata row. - registries-delete-confirm.png: destructive ConfirmModal with the kicker, title, and stack-pull warning body. - registries-with-entry.png removed (superseded by registries-overview.png and registries-card-detail.png). * docs(auto-update): refresh readiness page for v1 redesign Bring the Auto-Update Readiness doc in line with the shipped UI: - Replace the hero screenshot with a fresh capture of the redesigned board (italic-display hero, brand-cyan accent, per-node groups with local/remote pills, dashed-border changelog separator). - Rewrite the card-anatomy list. Drop the rollback-target bullet (the field exists in the backend payload but is not rendered). Add the "Rebuild available" inline label and the primary-image / multi-service count line. - Rewrite the risk-tags table as a risk-badges table using the actual badge labels and colors emitted by the UI (Safe / Review / Blocked with the corresponding icons; Digest rebuild for non-semver tags). - Add an Empty state section and document the per-node group header. - Tighten the hero subtitle paragraph to match the actual UI string (only major-bump count is surfaced separately; preview failures are not). - Fix workflow step 4: major-bump apply path is the stack lifecycle Update action, not the Schedules editor (a scheduled task hits the same block). - Add the 2-minute manual-refresh cooldown to the Recheck workflow. - Remove the broken cross-link to the non-existent /features/image-update-detection page and inline the 6-hour cadence fact from ImageUpdateService.INTERVAL_MS. - Convert troubleshooting to AccordionGroup format per the troubleshoot ing convention used on /features/deploy-progress. - Sync the Auto-Update entry in /features/overview.mdx to the new badge labels and the corrected hero-counter description. * docs(auto-update): fix Auto-Update entry point in Workflow step 1 Workflow step 1 said "Open the Auto-Update view from the sidebar." The Auto-Update view is opened from the top nav strip (alongside Home, Fleet, Resources, App Store, Logs, Schedules, Console, Audit). The sidebar carries the stack list and the per-stack right-click / kebab context menu that toggles auto-updates on or off; it does not house the Auto-Update top-level view. * docs(auto-update): trim enforcement detail from per-stack control note State the tier requirement once and stop, per Directive 27. The "The toggle does not appear on Community" sentence enumerates the enforcement effect of the gate, which the source already reflects; operator docs do not need to narrate it. * docs(auto-heal): refresh page for v1 UI and policy hardening Rewrite Auto-Heal Policies docs against the current Stack Monitor sheet: corrects the Max restarts / hr field label, documents the per-policy enable toggle, the consecutive-failures pill, the full Recent activity action set (including Docker unavailable), the 30s evaluation cadence, multi-node behavior, notification dispatches, and the dashboard Configuration status counter. Replaces the broken /images/auto-heal-policies/policy-sheet.png reference with three fresh screenshots captured against a live node: the sheet on the Auto-heal tab, a single policy row, and the expanded Recent activity panel. * docs(webhooks): refresh page for v1 UI, correct tier and add Git source sync - Fix tier note: gate is Skipper or Admiral, management is admin-only. - Update Settings path to Settings -> Alerts -> Webhooks; document the read-only Node field and the green secret-reveal callout. - Add the missing Git source sync action and the git-pull override value. - Refresh the configured-webhooks card description: action/stack/node badges, On/Off toggle, copy URL, and the Recent executions disclosure. - Tighten the trigger section with a constant-time signature check note and a status/body/meaning response table. - Add an Accordion troubleshooting block covering common signature failures, the 404 case, no-op actions on 202, and git-pull prereqs. - Re-capture all three screenshots from the v1 UI. * docs(webhooks): wrap troubleshooting accordions in AccordionGroup * docs(sidebar): refresh page for v1 redesign with filter chips, bulk mode, row anatomy, and troubleshooting Rewrites the Stack Sidebar page against the live v1 sidebar and the v1 docs refresh template (Frame screenshots, Note tier callouts, AccordionGroup troubleshooting). Recaptures all four existing screenshots and adds three new captures: filter chips, row anatomy, and bulk mode. Adds coverage for features the previous page omitted entirely: the ALL / UP / DOWN / UPDATES filter chips with their counts cap and collapse toggle; bulk mode (B key, sticky toolbar with Start / Stop / Restart, and Update gated on Skipper or Admiral); stack-row anatomy (status pill, label dots with +N overflow, image-update dot vs Git source icon priority, hover kebab); the Auto-update toggle, Schedule task, and Open App entries in the context menu; the B shortcut for bulk mode. Corrects three claims that no longer matched the code or UI: Auto-Heal is gated on Skipper or Admiral, not universal; the global Ctrl+K opens the command palette, not the sidebar search box; the activity footer kicker reads LIVE / IDLE with the verbatim copy from SidebarActivityTicker. Documents the in-menu ↗ and L › glyphs as visual hints rather than global keybindings to match useStackKeyboardShortcuts.ts. * docs(sidebar): trim enforcement-effect sentence from context-menu tier note State the Skipper / Admiral requirement once and stop, per Directive 27. The "They do not appear in the menu on Community" clause described the enforcement effect alongside the gate, which the directive bans in operator-facing docs. * docs(host-console): refresh page for v1 UI and clarify shell metadata Rewrite the Host Console page to match the current Cockpit layout (masthead + terminal well + chip strip), replace the legacy PowerShell screenshot with a fresh bash capture, and document the masthead tone states, kicker, metadata pills, and session/heartbeat behavior. Trim the security section to state the tier and role rule once. * docs(licensing): refresh page for v1 UI, corrected pricing, and trial flow Rewrites the Licensing & Billing page to match the redesigned v1 Settings layout. The previous draft still described the legacy Settings Hub: in-app "Upgrade your plan" Skipper/Admiral cards, "Start monthly trial" / "Start annual trial" buttons, the "Have a license key?" field, "Manage Subscription" with a capital S, "Deactivate License" as the button label, and the license-active.png asset rendering the literal "Sencho Pro" string in the card title. None of that exists in the current product. - Refreshes the Plans table to the live pricing on sencho.io/pricing and adds an Enterprise mention with the floor price ($3,500/year). Skipper now $11.99 annual / $14.99 monthly / $449 lifetime, Admiral now $69.99 annual / $89.99 monthly / $2,499 lifetime. - Rebuilds the Feature breakdown from a code-level audit of every requirePaid, requireAdmiral, requireScheduledTaskTier, and requireTierForSsoProvider call site in backend/src/routes, not from the marketing page. Notable code-grounded items: CVE suppressions on Community (no requirePaid guard), manual fleet snapshots on Community (scheduled snapshots on Skipper), Sencho Mesh under Admiral (entire mesh.ts router is requireAdmiral), and scheduled-task tiering names update/scan/snapshot as the Skipper subset with everything else under Admiral. - Rewrites the Free trial flow end to end. The previous steps told operators to click in-app "Start monthly trial" or "Start annual trial" buttons; no such buttons exist. The new flow starts on sencho.io/pricing, switches to the Annual or Monthly tab, clicks "Start 14-day trial" on the Admiral card, completes the Lemon Squeezy checkout (card-required, no charge before day 14), and pastes the issued key into Settings -> License -> License key. - Adds a new "The Plan section" anatomy block describing the masthead SCOPE / PLAN / DURATION (or RENEWS, TRIAL, STATUS) stat pills and the Plan card fields (Customer, Product, masked License key, status helper). - Adds a new "License states" reference table covering Community / Trial / Active subscription / Active lifetime / Expired / Disabled, what each surface renders, and which of the Plan / Activate / Pricing sections is visible in each state. - Corrects every UI label that drifted: section heading is Activate, field label is License key (not "Have a license key?"), buttons are Manage subscription (lowercase s) and Deactivate (not "Deactivate License"), and the action-row hint reads "Lemon Squeezy manages billing". - Documents the redesigned profile dropdown: identity header with initials chip, role badge, and tier badge, then Settings, conditional Billing, Documentation, Feedback, an Appearance segmented control, and Log Out. Billing only appears when the license is an active non-lifetime subscription. - Replaces all four screenshots under docs/images/licensing/: license-admiral-active.png (production Admiral lifetime view), profile-menu.png (redesigned popover), and two new captures for the Community-tier surfaces (license-activate-section.png, license-community.png). Removes the stale license-active.png (legacy "Sencho Pro" card) and profile-billing.png (legacy dropdown). * docs(settings-reference): refresh page for v1 UI with new sections and masthead Rewrites docs/reference/settings.mdx against the current Settings Hub so a reader encounters an accurate map of every section. Adds the previously missing **Cloud Backup** and **Security** sections, restructures **System Limits** into Host thresholds and Docker hygiene subsections (GiB units, "Global crash capture" toggle), fixes the Account password minimum to 8 chars and documents the two-factor subsection, refreshes License/Routing/Webhooks/App Store with the field labels actually rendered today, and documents the masthead pills (SCOPE/NODE, EDITED, plus the per-section stats like 2FA, PLAN, CHANNELS, ROUTES, WEBHOOKS, LABELS, TRIVY, POLICIES, PROVIDER, USED, SNAPSHOTS, DEV MODE). Replaces five existing screenshots that predated the v1 redesign and adds five new captures: Account with the 2FA card, License panel, System Limits with both subsections, Security with the Trivy installer, and Cloud Backup with Sencho Cloud Backup provisioned. All shots taken against the production node. * docs(licensing): drop billing-provider name from operator-facing copy The previous draft named the third-party billing provider in nine places (checkout, receipt email, error toast verbatim, Customer / Product field descriptions, the action-row hint, the billing portal, and the validation API). Operator docs don't need to advertise which vendor sits behind the checkout, billing portal, and validation calls. Rewrite each instance to describe what the operator sees and does without naming the upstream service. * docs(node-compatibility): refresh page for v1 UI with lock card visuals and current capability list - Replaces the legacy "dim + blur + pill overlay" description with the current CapabilityGate behavior: a centered lock card with an Unplug icon, title "<feature> is not available on this node", and a body line that names the node and its running version. - Corrects the tier-interaction section: on the wrong tier the entry point is hidden entirely, so the lock card only appears for users who already cleared the license gate. - Documents the public /api/meta endpoint, the 5-minute success cache, the 30-second failure cache, and the lazy-fetch behavior visible in the switcher (the version pill appears once a node has been visited). - Refreshes the capability table against the current CapabilityRegistry list, adding container-exec and vulnerability-scanning, with a note that vulnerability-scanning is only advertised when Trivy is installed. - Adds three production screenshots captured on the live fleet: switcher popover with mixed-version pills (one node on v0.76.9, rest on v0.81.11), a real lock card on an older pilot agent, and the Connection Details panel from Settings · Nodes. * docs(security): refresh security architecture page for v1 UI Add Fleet Secrets and Webhook signatures cards plus tier-matrix rows for shipped-but-undocumented features. Rename SSO presets from "one-click" to "preset providers" (presets still require OAuth-app provisioning on the upstream IdP). Update settings paths to the v1 middle-dot convention: Settings · Users, Settings · Account, Settings · Developer · Data retention. Extend the encryption-at-rest list with registry credentials and Fleet Secrets bundle payloads (both sealed with the same AES-256-GCM data key) and clarify the password section with bcrypt cost factor 10. Add a Webhook signature authentication subsection covering the per-webhook HMAC-SHA256 secret, one-shot display, masked preview thereafter, and constant-time comparison on inbound triggers. Replace the API Tokens screenshot with a fresh capture against the v1 Settings · Identity · API Tokens panel. * docs(security-advisories): retire reference page The reference/security-advisories page does not survive the v1 docs refresh: - Misuses the term "Security Advisories", which industry-wide refers to published notices for confirmed product CVEs (ID, severity, affected versions, fix version, remediation). The retired page was a narrative changelog of internal hardening work between v0.19 and v0.25.2. - The narrative is also frozen at v0.25.2 (April 2026) while current release is v0.81.11. Refreshing it would require backfilling ~56 release entries' worth of hardening copy. - The framing is uniformly "improved from prior behavior" (minimum 8 characters up from 6, 1-year token expiry previously without expiry, CORS previously allowed all origins, users should upgrade promptly). Sencho has not shipped publicly; there are no users to address as upgraders. All operationally relevant content already lives elsewhere: the security architecture page covers the current posture, verifying-images covers the supply-chain attestations, cve-suppressions covers operator acknowledgment, vulnerability-scanning covers the in-app scanner, and contact + the security architecture page both surface the disclosure path. Published Sencho-product advisories, when any exist, will appear on the GitHub Security tab, which is already linked from those pages. Inbound-link audit returned a single hit on the nav entry itself; no other doc, README, or operator artifact deep-links the slug. * docs: rewrite Pilot Agent page with deep architecture and operations reference Reframes docs/features/pilot-agent.mdx as the architecture-and-operations companion to the operator walkthrough in Multi-Node Management. Adds a mental model section, an explicit security and trust model, a full agent env-var reference, an honest limitations list, and a 5-item FAQ. Verifies every constant and label against the current backend source. Refreshes four production screenshots (admin login, scrubbed) and resolves the previously-broken /images/pilot-agent/enrollment-dialog.png reference. Adjacent edits keep the cross-linking coherent: - multi-node.mdx adds a one-line forward link to the rewritten page - security.mdx adds a Pilot Agent tunnel credentials subsection * docs(fleet-federation): deep rewrite with production screenshots Rewrites the Fleet Federation page against the v1 docs refresh template following the recent fleet-view, pilot-agent, and multi-node refreshes. Doubles the page length (92 to 204 lines) while keeping the cut-line v1 MVP scope: operator-driven placement controls (cordon + pin) for Blueprints, no expansion into mesh/sync/pilot territory. What changed: - Adds four production-captured screenshots under docs/images/fleet-federation/: the Federation tab with a cordoned node populated, the node-card kebab menu showing the Cordon node entry, the cordon confirmation dialog with a reason filled in, and a node card displaying the Cordoned pill. - Expands the page to eleven sections: opening summary, philosophy (kept), key capabilities, prerequisites, step-by-step usage with embedded screenshots, behaviour and lifecycle table, security and audit, limitations and non-goals (expanded), practical workflows (new: OS patching, host-to-host migration, gateway pinning), troubleshooting (eight accordions, up from five), and a Where Federation fits cross-link table. - Documents the exact production UI strings observed: the cordon dialog description, the uncordon confirmation copy, the reason field cap (256 chars), and the audit log action names (node.cordon, node.uncordon, blueprint.pin). - Documents the audit visibility surface so operators know how to filter the Audit view for cordon and pin history. - Adds eight cross-links to sibling pages (Fleet View, Multi-Node, Pilot Agent, Mesh, Fleet Actions, Fleet Sync, Blueprints, Licensing) with one-line scope contrasts so newcomers can place Federation in the broader fleet picture. - Tightens lifecycle table to operator-relevant terms (no DB column names) and audit section to operator-facing wording (no middleware names), keeping the page operator-focused rather than implementation-focused. Validation: - Captured screenshots against the production node logged in as admin, using Playwright MCP. Cordoned and pinned actions reverted; audit log confirmed the matched cordon/uncordon pair. - Verified every cross-link target exists in the v1-refresh worktree (/features/fleet-view, /features/multi-node, /features/pilot-agent, /features/sencho-mesh, /features/fleet-actions, /features/fleet-sync, /features/blueprint-model, /features/licensing). - Compliance: no em dashes, no PII, no "previously"/"used to" framing, no fence-spec language, tier rule stated once in plain language. * docs(fleet-federation): drop fence-spec phrasing in the open-core context Sencho is open-core: anyone can clone the repo and read the tier gate. Operator docs that name exactly where the UI gate sits ("hidden at the Community and Skipper tiers", "lower-tier users do not see the toggle", "only the Federation tab is gated") work as a dig-target for a tech-savvy reader and undercut the open-core posture. Directive 27 already bans enforcement-chain spelling; the open-core threat model makes the same phrasings risky even when they describe UI surfaces rather than route guards. Removes three instances of the pattern on this page: - Top Note callout: drops "The tab is hidden at the Community and Skipper tiers." Keeps the one-line requirement: "Federation is an Admiral feature. Cordon and pin actions require an admin user role." - Security and audit section: drops the sentence enumerating which UI affordances are hidden from which tiers. Keeps the customer-visible behavior (the Cordoned pill stays visible at every tier as a read-only signal). - Troubleshooting "Federation tab is not visible" accordion: rewrites to lead with the requirement and the role check, drops the "Federation is hidden by design" and "only the toggle and the Federation tab are gated" phrasings. Other claims on the page unchanged; rule is still stated once in plain language at the top of the page. * docs(fleet-sync): deep rewrite with production screenshots Replace fleet-sync.mdx with a verified end-to-end reference. The previous page named two replicated resources but the code syncs three, described a sync-status panel and a fleet-vs-node scope picker that do not exist in the shipped UI, and was missing prerequisites and several edge cases. Highlights of the rewrite: - Names all three replicated resources (scan policies, CVE suppressions, misconfig acknowledgements) and treats them uniformly. - Drops the sync-status-panel and node-scope-picker UI claims; both move to the Limitations section as honest caveats. - Adds prerequisites covering the paid-tier requirement on the control, admin-role requirement, proxy-mode remotes, and reachability. - Expands lifecycle coverage: per-node serialised pushes, add-node backfill, monotonic pushedAt, per-resource watermarks, identity-drift notifications, the 5000-row truncation cap, stale-target warnings, audit-log entries on the replica. - New "Where Fleet Sync fits" closing table cross-linking to Fleet View, Multi-Node Management, Pilot Agent, Vulnerability Scanning, CVE Suppressions, Fleet Federation, Fleet Actions, and Licensing. - Two fresh production screenshots: control Security panel and the "Scanner is per-node" callout shown when proxying to a remote. * docs(fleet-actions): deep rewrite with production screenshots Three cards are documented end to end: Stop fleet by label, Bulk label assign, and Prune Docker resources fleet-wide. Adds the execution-path distinction (control-orchestrated fan-out vs single-node proxy), per-card behaviour and partial-failure semantics, prerequisites, limitations, practical workflows, an Accordion troubleshooting section, and a Where Fleet Actions fits comparison table linking the surrounding Fleet view features. Corrects the prior page's tab-neighborhood claim, confirm-dialog wording, autocomplete-vs-request scope, and missing batch ceiling. Replaces the ten-day-old single screenshot with five fresh production captures under docs/images/fleet-actions/. * docs(fleet-secrets): deep rewrite with production screenshots Full rewrite of /features/fleet-secrets matching the fleet-actions structure. Replaces the sparse v1 page (no Frames, inline Q&A) with a gold-standard layout: opening Frame, single Note for the tier gate, 'What it covers' table, mental model, prerequisites, create + edit + versions + push (Target / Preview / Results) sections each with a production Frame, Import from stack section, behaviour and lifecycle table, audit-trail mapping with the six exact audit strings, limitations and non-goals, practical workflows, AccordionGroup troubleshooting, and a Where-it-fits cross-link table. Adds six fresh production screenshots under docs/images/fleet-secrets/ : overview, create, versions, target, preview, and results. Documents the Import-from-stack flow (depends on the bundle editor's new Import action) and uses the post-rename 'Send' wording on the bundle-row action (depends on the aria-label fix). Corrects three factual drifts vs the code: env-key regex described as 'letter or underscore, then letters/digits/underscores; case- sensitive' to match ^[A-Za-z_][A-Za-z0-9_]*$ ; documents only the 'ok' and 'failed' status pills (the 'skipped' enum value is unused); replaces the bogus 'stack not found' troubleshooting entry with the real 'env file not declared' cause. Drops the fence-spec phrasing 'The tab is hidden on Community.' per Directive 31; the tier requirement is now stated once in plain language. * docs(sencho-mesh): deep rewrite with mental model, lifecycle, security, screenshots Replace the feature-reference page with a deep product + technical guide. Adds: - Opening hook framing audience and problem (cross-node service-to-service without a separate VPN or service-mesh sidecar). - Mental model: three moving parts (sencho_mesh bridge, alias registry, cross-node transport) with direction-of-flow described in prose. - Key capabilities, prerequisites, step-by-step usage with inline screenshots. - Full lifecycle section covering opt-in, opt-out, sticky stack-stopped state, peer reconnect, and the proxy-mode bridge with its real default (persistent, env-override for idle). - Security and trust boundaries split into authentication, inbound exposure, encryption, audit, and app-layer caveats. - Limitations and non-goals: one-alias-per-port, port 1852 reserved, central-relay for remote-to-remote, shared 1024-stream pool with the Pilot tunnel, no L7, host-network unsupported, in-memory activity log. - Three concrete workflow examples and a complete troubleshooting accordion (every data-plane reason, every probe stage, every unreachable cause) plus a Common questions FAQ. - Where Mesh fits CardGroup linking Pilot Agent, Multi-Node, Federation, Licensing. Corrections vs prior text: - Tab is labelled Traffic in the UI (not Routing); all navigation references updated. - Proxy-mode bridge default is no idle close (env-overridable to opt into idle teardown); prior 5-minute-teardown claim removed. - Audit trail scope tightened: only opt-in / opt-out write durable rows; tunnel-state and probe events live in the in-memory activity log. Adds seven production screenshots under docs/images/sencho-mesh covering Table view, opt-in sheet, graph (Tunnels and Aliases edge modes), Diagnostics, activity log, and per-stack topology. * docs(blueprints): add missing detail-state-review screenshot Captures the Blueprint detail sheet with a deployment row in the "Awaiting confirmation" status (stateful first-deploy gate), to fix the broken image referenced at blueprint-model.mdx:132. mint broken-links now reports zero broken references. * docs(blueprints): deep rewrite with mental model, lifecycle, security, prerequisites Restructures the Blueprints page against the v1-refresh template used by the recently-refreshed mesh, secrets, and atomic-deployments pages. Adds a mental model, prerequisites table, lifecycle and status-transition map, security and trust boundaries section, practical workflows, common questions accordion, and a Where Blueprints fits CardGroup. Removes the internal-style rollout and watch-plan section. Replaces all nine production screenshots with fresh captures against the production node signed in as admin, and adds two new captures (federation pin policy table, stateless eviction dialog). Rewrites the tier-gate Note to drop the fence-spec phrasing that violated Directive 31. Every retained claim is anchored to current backend or frontend code. * docs(pilot-agent): recapture enrollment dialog with compose payload Replaces the pre-0.84 docker-run capture with the current dialog (Compose file, two-step instructions, "Copy compose file" button) and refines the alt text to describe the captured content. URL and token redacted to placeholder values during capture.	2026-05-20 08:43:18 -04:00
Anso	282ab8d844	fix(pilot): let SENCHO_PUBLIC_URL override the request Host in enrollment (#1122) ... The enrollment minter inferred SENCHO_PRIMARY_URL from the request Host header, which baked loopback or LAN addresses into the compose YAML when the admin opened Add Node on the central's own machine. Pilots on a different network (a public cloud VPS, for example) cannot dial that. SENCHO_PUBLIC_URL on the primary now wins when set and well-formed (http(s)://, no loopback). Trailing slashes are stripped. Falls back to the request Host when unset or invalid.	2026-05-20 03:35:25 -04:00
Anso	3ad6ea9c5d	feat(pilot): make Docker Compose the canonical pilot enrollment payload (#1121) ... * feat(pilot): make Docker Compose the canonical pilot enrollment payload The Add Node dialog for a pilot-agent now returns a Compose snippet instead of a single-line docker run command, and the enrollment dialog walks the operator through a save-and-up flow. The compose project name and container name align with what SelfUpdateService looks up at boot, so a Compose-deployed pilot can be updated remotely through the Fleet view without intervention on the remote host. Docs (pilot-agent, remote-updates) were rewritten to match. * test(e2e): align pilot enrollment spec with Compose payload The spec was written against the docker-run payload; it now asserts the Compose YAML the dialog renders.	2026-05-20 02:05:21 -04:00
Anso	0117556bea	fix(fleet): rename Traffic tab label to Routing for consistency (#1119) ... The Fleet view sub-tab that renders RoutingTab.tsx was labeled "Traffic" while every adjacent identifier already used "Routing": the backend route file (backend/src/routes/mesh.ts), the component path, the localStorage key (sencho-routing-view-mode), the SegmentedControl aria label ("Routing view mode"), and the engineering vocabulary across the codebase. Operators looking for the "Routing tab" could not find it because the visible label said something else. Align the user-visible label with the rest of the implementation and update the two doc references that named the tab by its old label (docs/features/fleet-actions.mdx, .env.example).	2026-05-19 23:25:30 -04:00
Anso	1f673073ca	feat(fleet): add fleet-wide Docker prune to Fleet Actions (#1104) ... Adds a third card to the Fleet Actions tab that fans out Docker prune (images, volumes, networks) across every node in one submit. Local nodes call DockerController under a bulk-prune lock; remote nodes receive one POST /api/system/prune/system per target. Per-node + per-target results with reclaimed bytes are surfaced inline via ResultsList. Tier: Skipper / Admiral (requirePaid + requireAdmin), matching the rest of Fleet Actions. The frontend card is mounted inside the existing isPaid branch at FleetActionsTab; no new frontend gate is required. The card uses an amber accent rail and the Eraser icon so it reads as 'cleanup' rather than 'destructive stop'. Scope toggle defaults to Managed only (Sencho-tagged resources) with an All unused option that escalates the destructive-confirm copy.	2026-05-19 00:13:32 -04:00
Anso	554f662563	fix(mesh): surface stopped-stack opt-ins on routing node cards (#1098) ... Add `currentlyResolvable: boolean` per entry in `MeshNodeStatus.optedInStacks`, derived from the existing alias cache so the new field stays consistent with `/api/mesh/aliases` without any extra Dockerode or cross-node inspect on the status path. The Routing tab renders an amber `suspended` pill for entries whose stack is opted in but currently has no running services, plus a single explanatory caption below the suspended list. Resolves the contradictory `Mesh stacks: 1 / Aliases: 0 / No mesh services on this node yet` copy on the node card when a meshed stack's container has been stopped; the misleading line is now only shown when the node truly has no opt-ins. The opt-in itself remains sticky: when the stack starts again, its aliases reappear automatically on the next refresh. Defensive de-dup in the UI filters suspended entries against the live alias snapshot to handle the transient gap where `/mesh/status` and `/mesh/aliases` return slightly inconsistent views from their separate fetches. Tests: - new `mesh-status-resolvability.test.ts` (6 cases) locks the resolvable / suspended / mixed / empty / per-node-scoping / stale-alias-no-phantom invariants for `getStatus`. - `mesh-topology-layout.test.ts` gains a `stacksKey` resolvability-flip case and a `meshNodeStateEqual` case asserting a resolvability flip on an otherwise identical stack registers as a state change so the topology layout re-runs. Operator docs gain one new troubleshooting accordion in `/docs/features/sencho-mesh.mdx` explaining the suspended state.	2026-05-18 01:52:18 -04:00
Anso	f6e42535c8	fix(mesh): route peer→central traffic over the existing forward WS (#1094) ... * fix(mesh): route peer→central traffic over the existing forward WS The reverse mesh callback path (`/api/mesh/proxy-tunnel-from-peer`) needed SENCHO_PRIMARY_URL on central plus a publicly reachable origin from the peer's perspective. In a typical homelab where central sits behind NAT, peer→central dispatch silently failed at the dialer's short-circuit and the headline "call any service on any node by hostname" worked one way only. The forward WS at `/api/mesh/proxy-tunnel` is already bidirectional end to end. Make the bridge a persistent control-plane primitive: dial every mesh-enabled proxy peer at startup, reconcile every 60 s, never idle-close. Peer→central traffic multiplexes over the same WS via `tcp_open_reverse`. Removed: - `meshProxyTunnelFromPeer.ts` WS handler and dispatch - `MeshCentralRegistry`, `PeerToCentralMeshSessionDialer` - `mesh_handshake` first-frame state machine in `meshProxyTunnel.ts` - `maybeSendBootstrap`, `buildHandshakeFrame` in the dialer - `mesh_proxy_callback_bootstrap` capability and `maybeWarnUnsetPrimaryUrl` - `mesh_centrals` table (drop migration; greenfield, no users) - `PilotTunnelManager.replaceOrRegisterProxyBridge` (dead after handler removal) - twelve associated unit/integration tests plus the peer-recovery branch in `MeshService.openCrossNode` Added: - `MeshService.proactiveBridgeFanout` selects every mesh-enabled proxy peer (no longer gated on `mesh_stacks` rows) - `startBridgeReconcileLoop` runs the fanout every 60 s (override via `SENCHO_MESH_RECONCILE_INTERVAL_MS`) - `MeshProxyTunnelDialer` default idle TTL is now `0` and exposes `isDialing(nodeId)` for the status surface - `MeshNodeStatus.reverseCallbackStatus` discriminator (`connected | connecting | unavailable | not_applicable`) surfaced via `/api/mesh/status` and rendered as a pill in the Routing tab - `openCrossNode` error message distinguishes "no proxy target" from "waiting for central to dial the reverse bridge" - New tests: `mesh-service-proxy-tunnel-reconcile`, `mesh-status-reverse-callback`, `mesh-proxy-tunnel-dialer-no-idle-close` SENCHO_PRIMARY_URL is no longer required for any mesh function. * fix(mesh): rewrite proxy-tunnel reconcile test contents The previous commit renamed the file but the rewritten test bodies stayed unstaged on top of the rename. This commit lands the actual rewrite: the fanout assertion now requires every mesh-enabled proxy peer to be dialed, not just those with `mesh_stacks` rows, and adds a reconcile-tick repeated-call test.	2026-05-17 22:00:31 -04:00
Anso	578cac89da	fix(mesh): surface data-plane failures in health, meta, and Routing tab (#1088) ... The three previously-silent console.warn paths in MeshService.setupMeshNetwork now route through a typed recordSetupFailure helper that classifies the failure (subnet_invalid, subnet_overlap, subnet_mismatch, ip_in_use, attach_failed, not_in_docker), emits a mesh.disable activity entry at the matching level (error for real failures, warn for the expected dev-mode not_in_docker case), and strips mesh_proxy_callback_bootstrap from advertised capabilities via CapabilityRegistry. /api/health gains a mesh.dataPlane block carrying the typed status. /api/mesh/status carries localDataPlane at the top level so the Routing tab renders a red banner with an operator-actionable recovery hint (set SENCHO_MESH_SUBNET to a free /24 and recreate the container) when the data plane is down. The success path re-enables the capability and flips the status to ok. Generalizes the previously-documented F-0 failure mode (IP-in-subnet collision) to also cover the subnet-pool-overlap case where another Docker bridge on the host already owns the requested CIDR.	2026-05-17 17:31:57 -04:00
Anso	cf618dd866	feat(mesh): symmetric WS dial for proxy-mode mesh peers (#1066) ... * chore(mesh): foundation for symmetric callback dial Adds the data-plane scaffolding that the symmetric callback dial fix builds on: - mesh_centrals table for peer-side bootstrap material - MeshCentralRegistry service (upsert/getActive/clear/markUsed/markRejected) - PilotTunnelManager kind discriminator and replaceOrRegisterProxyBridge - mesh_proxy_callback_bootstrap capability registration - MeshProxyTunnelDialer reason-tagged proxy-bridge-down events from a single tearDownBridge emission point - Reactive redial scheduler that skips idle and auth_failed reasons * feat(mesh): add reverse-direction activity log entries (closes R1-B) acceptReverseLocal now emits route.resolve.ok with direction=reverse on connect ack and route.resolve.fail with direction=reverse plus reason=container_not_found / connect_error pre-connect. Post-connect close/error stays silent. Reuses existing event types via the new details.direction discriminator so frontend filters are unaffected. * feat(mesh): add peer-to-central callback dial path (closes R1-A2) Closes the architectural gap where proxy-mode mesh peers could not re-establish their tunnel to central after any non-idle bridge teardown (idle close, network blip, central restart, peer reboot). Central remains the hub for the data plane; the change is purely about WS initiation. Symmetric WS initiation, asymmetric protocol roles. Central retains PilotTunnelBridge ownership; peer retains TcpStreamSwitchboard + reverseDialer ownership. Central bootstraps callback credentials over the first authenticated central-initiated mesh tunnel via a one-shot mesh_handshake JSON frame; peer persists the material in a new mesh_centrals SQLite table and dials central's new /api/mesh/proxy-tunnel-from-peer endpoint when local cross-node traffic needs a bridge and none is live. Mesh_tunnel JWT (HS256, signed with auth_jwt_secret) carries scope, audience, issuer (central instance id), peer api_token fingerprint, kid. Validation on inbound peer dial: algorithm pin, signature, scope, audience, instance, time bounds, node existence and mode, fingerprint match. Failures return HTTP 401 with a machine-readable reason; peer routes the response per a clear-vs-keep cache matrix. Triggers proactive bootstrap on mesh-enable and api_token rotation; central startup fans out to mesh-enabled proxy-mode nodes with mesh_stacks rows (throttled, fire-and-forget). Reactive redial on non-idle bridge loss. Capability-gated handshake send (mesh_proxy_callback_bootstrap) makes the upgrade path safe against older peers in mixed-version fleets. Adds peer-side /api/system/pilot-tunnels centralCallback diag block, bounded counter metrics for bootstrap and dial events. SENCHO_PRIMARY_URL preflight warning when unset on a central with mesh-enabled proxy nodes. Tested with unit suites for the validation chain, registry, manager, and both dialers; integration tests for bootstrap E2E (asserts protocol-role invariant), api_token rotation, instance id change, version skew, and pilot-mode regression. * fix(mesh): green CI on the symmetric callback branch Two independent CI failures, both surgical: 1. Backend tests (11 fails): four mesh test files called setupTestDb in beforeEach. setupTestDb does not reset the DatabaseService singleton, so the per-test afterEach rm of the previous tmpdir left the singleton connection pointing at a deleted file. The next beforeEach's line-55 write threw SQLITE_READONLY_DBMOVED on Linux. Windows file-lock semantics hid this locally. Hoist setupTestDb / cleanupTestDb to file-scope beforeAll / afterAll; per-test state resets stay in beforeEach. Matches the convention in the eight mesh test files that already pass. 2. CodeQL (4 high alerts): js/insufficient-password-hash flagged sha256(api_token) at four sites. The api_token is a 256-bit opaque bearer (sen_sk_-prefixed), not a human password; sha256 is the correct fingerprint primitive for binding the mesh_tunnel JWT to a specific token. Add the two production files plus the two test files that mint the fingerprint to the existing path-scoped query-filter for that rule. * fix(mesh): drop unused afterEach import and revert dead codeql config ESLint flagged afterEach as unused in mesh-central-registry.test.ts:1 after the previous commit hoisted setup/teardown to file-scope beforeAll/afterAll. Remove from the vitest import line. Revert the codeql-config.yml additions from the previous commit. The paths: sub-key under query-filters > exclude is not a documented CodeQL feature and silently no-ops. The four js/insufficient-password-hash alerts on api_token fingerprinting are tracked as dismissed false positives in the GitHub Security tab rather than via dead config.	2026-05-16 14:58:19 -04:00
Anso	16774ae515	feat(fleet): add node management actions to Fleet Overview (#1064) ... * refactor(nodes): extract node create/edit/delete modals into useNodeActions hook Pulls the inline Add/Edit/Delete/Pilot-enrollment modal stack out of NodeManager.tsx and into a reusable useNodeActions() hook in components/nodes/. Settings continues to consume the same modals via this hook, with an onTestResult callback used by Settings to render the existing connection-detail panel after a successful test. The hook also extends the auto-test-on-save behavior so that saving a proxy-mode remote node from the Edit dialog re-runs the connection test when the API URL or token has actually changed (skipped when only name or compose dir was edited). * feat(fleet): surface Add/Edit/Delete node actions on Fleet Overview Adds an admin-only Add node button to the right of Refresh on the Fleet header, opening the same Add Node dialog used by Settings. Each node card's three-dot menu now exposes Edit node and Delete node items (routed through the shared useNodeActions hook) alongside the existing Cordon item, so operators can manage node lifecycle without leaving the Fleet page. The card kebab is shown to admins regardless of tier; Cordon stays Admiral-only. Delete is hidden on the local default node. After any Add/Edit/Delete the Fleet overview refetches so the grid reflects the change immediately. * docs(fleet): document Add/Edit/Delete node actions on Fleet Overview Updates the Action buttons table to cover the new Add node entry point on the Fleet header, and adds a new Node actions menu section describing the per-card Edit/Delete/Cordon items, their tier and permission gating, and the auto connection test that fires after saving a proxy-mode remote.	2026-05-15 19:54:23 -04:00
Anso	d882f223f4	feat(api-tokens): switch to sen_sk_ prefixed opaque keys (#1062) ... * feat(api-tokens): switch to sen_sk_ prefixed opaque keys Replace JWT-shaped API tokens with 56-char opaque keys of the form `sen_sk_<43-char base62 random><6-char base62 checksum>` (256-bit entropy, sha256-truncated checksum). Node-proxy tokens stay JWTs. Why: * The api_token path was already a sha256 DB lookup; the JWT signature was wasted work and the 400d JWT ceiling vs DB expires_at was a confusing dual bound. * Opaque tokens carry a verifiable checksum so malformed/typoed values are rejected before any SQLite lookup. * `sen_sk_` prefix is recognizable to GitHub, TruffleHog, GitGuardian and makes the on-wire shape visually distinct from node_proxy JWTs. Changes: * New `utils/apiTokenFormat.ts` (generate + checksum-verify, CSPRNG via randomInt, timingSafeEqual on the checksum compare). * `middleware/auth.ts` and `websocket/upgradeHandler.ts` route opaque tokens before any jwt.verify; 401 messages unified to avoid a token-existence oracle. * `middleware/rateLimiters.ts` short-circuits opaque tokens in the node_proxy detection and keys per-token via a non-reversible sha256 slice so each token keeps its own bucket without a DB hit. * All six existing tests migrated from jwt.sign({scope:'api_token'}) to generateApiToken(); new format-only test suite covering prefix, length, alphabet, checksum reject paths, and a 10k-iteration collision/integrity loop. * Docs (features/api-tokens.mdx, api-reference/overview.mdx) describe the shape and drop the obsolete JWT-ceiling note. * fix(api-tokens): clear CI lint and CodeQL false positives * Drop unused TEST_USERNAME import in remote-console-session.test.ts; the migration to generateApiToken() left it orphaned. * Add a CodeQL barrier model so `generateApiToken`'s ReturnValue does not flow into the `insufficient-password-hash` query. The function emits 256-bit CSPRNG opaque keys; sha256 of the raw token is the correct construction for high-entropy API tokens (bcrypt-class hashes target low-entropy human passwords). CodeQL's name heuristic was treating "Token" as a password source and flagging the standard sha256 wrapping at all 9 call sites. * ci(codeql): exclude js/insufficient-password-hash for token paths The previous barrierModel data extension was a no-op for this rule: the js/insufficient-password-hash query identifies its "password" sources via SensitiveExpr's name heuristic ("token", "secret", "key" substrings), which is upstream of the taint-tracking layer where barrierModel applies. Verified by post-push re-analysis: 9 alerts still open, all undismissed. Replace the dead extension with a path-scoped query-filter in codeql-config.yml so the rule no longer fires on apiTokenFormat, apiTokens, and the test directory. Real user-password hashing code elsewhere in the repo (auth, users, setup routes) remains analyzed. The 9 existing alerts on PR #1062 are dismissed via API as false positives with a justification pointing at this config. Future runs will not re-flag them because of the path filter.	2026-05-15 18:21:28 -04:00
Anso	3323a59003	fix(mesh): poll topology data and tighten stack-membership equality (#1059) ... Adds a visibility-aware 30s poll on the Routing tab so the graph reflects tunnel state, alias publishes, and remote-node disconnects without a manual reload. Polling pauses while the tab is hidden and wakes immediately on focus via the shared visibilityInterval helper. Tightens nodeStateEqual so a stack swap on the same node (opt out A, opt in B) is detected even when the count is unchanged. Extracts the helper, the minimap colour mapping, and the minimap colour literals to mesh-topology-layout so they can be unit-tested and to drop a now-unused isOwnerView field from MeshNodeData. Persists the Table/Graph and Tunnels/Aliases toggles to localStorage so a Routing tab session keeps the operator's last-used view. Adds a legend line to the per-stack topology sheet clarifying that consumer edges mean meshed peers that could reach the aliases via DNS; whether containers dial them depends on each consumer's own opt-in stacks. Adds unit tests for stacksKey, meshNodeStateEqual, and miniMapColorFor, plus troubleshooting entries and a refresh-cadence note to the mesh docs.	2026-05-15 09:52:10 -04:00
Anso	2e82eb44fe	feat(fleet): multi-mode topology with label grouping and persisted positions (#1054) ... The Topology view now offers three layouts so the canvas adapts to how the fleet is organised, not the other way around: - Hub: the gateway anchored on the left with remotes radiating right - Grouped (Skipper+): remotes cluster by their primary node label, with the local node in its own cluster and unlabeled remotes in an Unlabeled cluster - Free (Skipper+): drag any node; positions persist per browser via local storage Node cards gain label pills (Skipper+), a cordon banner with reason tooltip, a latency chip for online remotes, and a stale pilot-heartbeat glyph. All fields are sourced from /api/fleet/overview, which already returns cordon, latency, and pilot timestamps; no backend changes required. Community continues to see the single Hub layout (without the toolbar, no gating cues), matching the visibility principle that paid affordances are hidden from lower tiers rather than displayed as locked teasers. The backend /api/node-labels route is already requirePaid, so the data gate is honoured end to end. Includes unit tests for the layout module (hub/grouped/free coverage) and the preferences hook (round-trip plus corrupt-JSON fallback).	2026-05-15 08:59:52 -04:00
Anso	9e4b969dfb	feat(mesh): add topology graph view and per-stack drill sheet (#1052) ... Adds a Table/Graph toggle to Fleet → Traffic. Graph mode draws the fleet as a ReactFlow diagram with a second toggle that switches edge encoding between Tunnels (one edge per node pair, coloured by tunnel state) and Aliases (same edges labelled with alias counts). Clicking a node card opens the existing opt-in sheet, where each opted-in stack now has a Topology button that opens a focused side sheet showing the stack at the centre, the aliases it publishes, and the meshed consumer nodes with per-tunnel state. Reuses the existing /mesh/status and /mesh/aliases endpoints; no backend changes. Inherits the Admiral gate from the parent Routing tab. Layout helpers and the per-stack sheet are unit-tested.	2026-05-15 08:07:34 -04:00
Anso	946db9df52	fix(mesh): accept node_proxy at the proxy-tunnel WS upgrade (#1050) ... The mesh proxy-tunnel WS handler was gated on full-admin api_token scope only, so node_proxy JWTs (the credential the Add Remote Node dialog tells operators to generate via Settings -> Nodes -> Generate Token) were silently rejected with HTTP 403. Operators followed the dialog's instructions, enrolled the node cleanly, saw reachableMode='proxy' with no negative badge, opted a stack into mesh, watched the redeploy complete, and only then discovered no bytes flow. node_proxy already authorizes every /api/* surface on the remote (deploy, exec, host console, filesystem), which is a strict superset of what the mesh proxy-tunnel does. Gating mesh more strictly was theatre, not security, and it created a UX trap with no in-product signal pointing at the scope mismatch. Change the upgrade dispatcher to accept any machine-to-machine credential: node_proxy JWT or full-admin api_token. Session cookies and restricted api_token scopes (read-only, deploy-only) remain rejected through the same paths as before. Adds positive/negative coverage in upgrade-order.test.ts for all four credential shapes so the gate stays pinned against future re-tightening. Updates user-facing copy in the mesh docs and Settings -> API Tokens description so the documented path matches the actual code. If we ever introduce a demoted node_proxy variant (audit-only, read-only fleet member), the right move is differentiated node_proxy scope claims inside the tunnel JWT (tracker F-A3), not requiring a separate token type for mesh.	2026-05-14 23:26:41 -04:00
Anso	a38a3e0226	feat(mesh): route mesh traffic over Distributed API remotes (#1048) ... * refactor(mesh): extract shared TCP stream switchboard Pull the `tcp_open` / `tcp_open_ack` / `tcp_open_reverse` / `tcp_close` + `TcpData` handling out of the pilot agent into a reusable module so a second caller (the upcoming proxy-mode WS handler) can run the same frame parser, stream allocator, idle timers, and Compose-label resolver. One parser, two callers, zero drift on a security-sensitive protocol surface. The pilot agent keeps its public `openMeshTcpStream` API and delegates to a per-connection switchboard constructed in `connect()` and torn down in `cleanupAfterDisconnect`. Existing reverse-stream and resolver tests are rewritten to exercise the shared module directly. * feat(mesh): route mesh traffic over Distributed API remotes Sencho Mesh now works against remotes in Distributed API mode in addition to Pilot Agent mode. Central opens a short-lived WebSocket tunnel to the remote's new `/api/mesh/proxy-tunnel` endpoint on demand and tears it down after 5 minutes of idle (configurable via `SENCHO_MESH_PROXY_TUNNEL_IDLE_MS`). Mesh dispatch is mode-agnostic at the routing layer; `PilotTunnelManager.ensureBridge` resolves an existing tunnel or asks the new `MeshProxyTunnelDialer` to dial. The Routing tab badges now use a `reachableMode` classifier: `★ Local` for local, `pilot offline` red badge for a pilot with a down tunnel, and a new `unreachable` red badge surfacing the specific reason (missing token, scope not full-admin, TLS failure, remote does not support proxy mesh). Distributed API remotes with valid credentials show no negative badge; the tunnel opens on first dial. Auth: the proxy-tunnel WS upgrade requires an `Authorization: Bearer` API token with the `full-admin` scope. Lower-scoped tokens are rejected at upgrade time so a leaked read-only token cannot reach the mesh data plane. Bidirectional: the proxy-mode WS handler registers itself as the local `MeshService` reverse dialer (compare-and-swap), so meshed containers on a Distributed API remote can dial cross-node aliases via `tcp_open_reverse` over the same tunnel. Cross-node relays (`PilotTunnelBridge.acceptReverseRelay`) await `ensureBridge` so proxy-to-proxy mesh works without standing tunnels. * docs(mesh): describe Distributed API mesh and unreachable troubleshooting Refresh the user-facing mesh documentation to reflect that mesh works over both Pilot Agent and Distributed API remotes. Adds a Troubleshooting accordion entry for the new `unreachable` Routing tab badge so operators can match a tooltip reason ("api token rejected (scope must be full-admin)", "remote does not support proxy mesh", "TLS handshake failed", "api_url not set", "api token missing") to a concrete fix. * refactor(mesh): apply /simplify review findings Five cleanups surfaced by the post-implementation code review pass. No behaviour change for fleets running on `main`; only internal structure improves. - **Deterministic container IP shared.** Extract the compose-default network preference (`pickContainerIp`) and the conventional-name fast path (`lookupContainerIp`) into `backend/src/mesh/containerLookup.ts`. Both `MeshService.resolveContainerIp` (existing same-node fast path) and the switchboard's `resolveByComposeLabels` (proxy/pilot inbound dial) now use the same logic. Earlier the switchboard helper grabbed the first `Object.values(Networks)` IP, which could flip across daemon versions on multi-network containers; fixed. - **WS URL upgrade extracted.** New `backend/src/utils/wsUrl.ts` exposes `httpUrlToWs(baseUrl)` that maps `http://` to `ws://` and `https://` to `wss://`. Used in both `pilot/agent.ts` and the proxy-tunnel dialer. The previous inline `replace(/^http/, 'ws')` silently downgraded `https://` to `ws://` (cleartext). - **`computeReachable` takes the pre-fetched node.** `getStatus` already iterated `db.getNodes()`; the helper previously re-queried by id per node (N+1 reads). Pass the row in. - **Reachable-reason ternary -> const map.** Replace the nested ternary in `MeshService.computeReachable` with a `Record<DialFailureCode, string>` lookup keyed on the dialer's failure code. - **Activity-type ternary -> const map.** Same flattening inside `MeshProxyTunnelDialer.logActivity`. All mesh tests pass (85/85). Full backend suite green (2126/2126). * fix(mesh): cache proxy dial failures and contain WS handshake errors `ensureBridge` now consults the recent-failure cache before dialing so a continuous mesh workload against a misconfigured proxy-mode remote does not produce one upgrade attempt per cross-node TCP open. `recordFailure` emits at most one activity-log entry per cache window per (nodeId, code) so a connect-loop on a single bad remote cannot flush the ring buffer. Failure messages run through `redactSensitiveText` before reaching the log so any embedded Bearer / JWT / inline-URL credentials are scrubbed. `stop()` clears the inflight map and `dial()` checks `this.stopped` between awaits so a shutdown does not leak a half-opened bridge. When `awaitOpen` rejects (401, 4403, TLS), the dialer attaches a noop 'error' listener before calling `ws.close()`. Without it the ws library emits a tail 'error' on a still-CONNECTING socket that propagates as an unhandled exception. Surfaced by a live-network test against a real proxy-mode peer. Adds `mesh-proxy-tunnel-live.test.ts` (skipped unless MESH_AUDIT_URL and MESH_AUDIT_TOKEN_FILE env vars are set) covering both the happy path and the auth-rejected path against an actual remote Sencho. * feat(mesh): instrument proxy tunnel observability Adds three counters to `PilotMetrics`: - `proxy_bridges_total` (incremented on `registerProxyBridge` success) - `proxy_dials_failed` (every failed dial attempt, not deduped) - `proxy_idle_closes` (idle-sweep teardowns) All three surface automatically via `GET /api/system/pilot-tunnels` since the route returns the full `Counters` snapshot. Gates two pre-existing always-on `console.warn` calls in `tcpStreamSwitchboard.ts` (mid-stream socket errors and Docker resolve failures) behind `isDebugEnabled()`. Both fire on per-stream events and would otherwise violate the diagnostic-log safety rule under load. Adds `mesh-tcp-stream-switchboard.test.ts` covering the forward `tcp_open` path: real localhost dial, resolver errors (no_target, denied), per-tunnel cap saturation, frame-routing fall-through invariants, `tcp_close` socket teardown. Drops `MESH_CONNECT_TIMEOUT_MS` from the public surface; only the switchboard itself uses it. * fix(mesh): tighten Routing tab unreachable handling `RoutingNodeCard.tsx`: the **Add stack to mesh** button now disables when `reachableMode === 'unreachable'`, matching the existing TogglePill behavior. Without this, an operator on an unreachable node could open the opt-in sheet, confirm, and watch the redeploy proceed against a target whose mesh data plane will silently fail to route. Also drops the leftover `status.nodeId !== -1` guard on the pilot- offline badge. `MeshService.ts`: renames `isMeshReachable` to `isMeshConfigured` with the new predicate that returns true for proxy-mode remotes whose creds are valid (the tunnel is opened on demand). `getRouteDiagnostic` now distinguishes "routable" (configured) from "pilotLive" (live tunnel state, only meaningful for pilot mode); without the split, every idle proxy-mode route would report `tunnel down`. `MeshService.setReverseDialer` warns when the unconditional install path silently overwrites a non-null current dialer. By topology a Sencho is either pilot or central, so the branch flags a misconfigured deployment rather than an expected race. Drops the dead `export { ReverseTcpStreamHandle }` re-export from `pilot/agent.ts`; its only consumer imports straight from `mesh/tcpStreamSwitchboard.ts`. Fixes a stale doc comment in `MeshService.ts` that referenced the wrong source file. Adds `mesh-proxy-tunnel-handler.test.ts` covering the WS handler lifecycle: pilot-mode 404 rejection, post-upgrade reverse-dialer install, concurrent-upgrade 1013 rejection (single-tenant slot), and error-path teardown. Updates the troubleshooting accordion in the user docs to mention the disabled-state behavior. * fix(mesh): close ESLint and CodeQL findings on the proxy-tunnel diag logs Remove the unused `reverseDialer` local in `meshProxyTunnel.ts`; the closure target is `localDialer` above and this assignment was always dead. Strip line breaks inline on the three `MeshProxyTunnelDialer` diag log lines so CodeQL `js/log-injection` data flow recognises the sanitisation that `sanitizeForLog` already performs. No behaviour change; the diag logs render the same characters they do today.	2026-05-14 20:51:42 -04:00
Anso	69b6ac1f3b	fix: harden stack file explorer operations (#1028) ... * fix: harden stack file explorer operations * fix: update Docker toolchain to Go 1.26.3 * fix: repair Dockerfile tr argument split across lines * fix: bump protobufjs to clear npm audit high-severity advisories	2026-05-12 15:49:51 -04:00
Anso	19cdb3681d	fix: harden blueprint deployment guardrails (#1027) ... * fix: harden blueprint deployment guardrails * fix: update Docker toolchain to Go 1.26.3 * fix: repair Dockerfile tr argument split across lines * fix: bump protobufjs to clear npm audit high-severity advisories	2026-05-12 15:49:19 -04:00
Anso	693f9b4495	fix(fleet): drop tier gate from stack and container list endpoints (#1012) ... Community tier opened the Fleet Overview drilldown (node card → stack list → container list) but two read-only metadata endpoints in fleet.ts kept their requirePaid gate. Stack names still rendered because they ride on the un-gated /api/fleet/overview payload, but expanding a stack always re-fetched containers and hit the orphan gate, surfacing as a "Failed to load containers for X" toast on every node card on Community. Drop requirePaid from GET /node/:nodeId/stacks and GET /node/:nodeId/stacks/:stackName/containers. Both handlers stay behind authMiddleware, validate inputs, and proxy to the per-node api_token for remote nodes. Paid mutations (bulk update, scheduled snapshots, label bulk actions, Fleet Actions cards) keep their gates in fleetActions.ts. Add positive Community-tier tests in fleet.test.ts and clear the stale "Requires Skipper or Admiral license" line from the OpenAPI descriptions.	2026-05-09 01:55:56 -04:00
Anso	23bbee4f45	feat(mesh): replace host-mode with shared sencho_mesh Docker network (#1009) ... * feat(mesh): replace host-mode with shared sencho_mesh Docker network Phase D of the mesh redesign: drop the operator's `network_mode: host` requirement and the `host-gateway` extra_hosts pattern that did not work on cloud iptables-restrictive distros (OCI, etc.) or Docker Desktop. Each Sencho creates a shared `sencho_mesh` Docker bridge network on boot (default subnet 172.30.0.0/24, override via SENCHO_MESH_SUBNET), pins itself at `<network>+2`, and attaches every meshed user service to the same bridge. Compose overrides now emit IP-based `extra_hosts` plus a top-level `networks` block declaring `sencho_mesh` external. Override delivery: central renders for local stacks; for remote stacks it sends the fleet alias list to the remote's new `PUT /api/mesh/local- override/:stackName` endpoint, which renders against the remote's OWN local senchoIp and writes under its OWN DATA_DIR. Each node may use a different subnet without coordination beyond the env var. Opt-in / opt-out now trigger an automatic redeploy of the affected stack via the existing deploy code path (local: ComposeService; remote: HTTP POST through proxyFetch). The frontend opt-in sheet shows a confirmation modal (ConfirmModal) before the mutation. Failed redeploys emit both a mesh activity event and a durable audit-log row. Hardening: - Reserve port 1852 at opt-in (prevents user containers from racing the Sencho API listener). - ensureMeshNetwork refuses to continue if `sencho_mesh` exists with a mismatched subnet rather than silently routing to the wrong IP. - Idempotent network connect/disconnect helpers in DockerController. - optInStack rolls back the DB row if the just-inserted stack's override push fails (no half-states surviving across calls). - regenerateOverridesForNode runs in parallel and skips the just- pushed stack on opt-in. Operator template: drop `network_mode: host`, restore `ports: ["1852:1852"]`. Mesh now works identically on Linux LAN, OCI, and Docker Desktop without firewall changes. Docs: rewrite docs/features/sencho-mesh.mdx around the shared bridge network, document SENCHO_MESH_SUBNET, surface the host-network-service opt-in restriction, and cross-link with the Pilot Agent docs. BREAKING CHANGE: the operator's `docker-compose.yml` no longer uses `network_mode: host`. After upgrading, redeploy any meshed stacks once so they pick up the new IP-based override and join `sencho_mesh`. * fix(mesh): wrap stackName with path.basename in local-override fs ops CodeQL flagged js/path-injection on the new applyLocalOverride and removeLocalOverride methods because they are publicly reachable and its data-flow model does not recognize isValidStackName / isPathWithinBase as sanitizers. The validation IS sufficient (the allowlist regex blocks path separators, the path-prefix check blocks escape), but path.basename is a model CodeQL recognizes and is purely defensive: for any input that already passes isValidStackName, basename is the identity.	2026-05-09 00:11:09 -04:00
Anso	ccad5c925b	feat(nodes): hide hub-only views when active node is remote (#1007) ... * feat(nodes): hide hub-only views when active node is remote Fleet, Schedules, Audit, Logs, and Auto-Update operate on hub-owned state (node registry, fleet schedules, centralized audit, fleet-wide log aggregation, fleet-wide update preview). When the active node is remote, proxying those surfaces would show that remote's own disconnected state instead of the hub's. Hide them from the nav strip and force-redirect to Home if one was open during the node switch. Backend hubOnlyGuard middleware sits between nodeContextMiddleware and the remote proxy and rejects /api/scheduled-tasks, /api/audit-log, and /api/notification-routes with 403 + HUB_ONLY_ENDPOINT when nodeId resolves to a remote, closing the script-bypass path the UI gating cannot reach. Settings sub-sections were already gated via the hiddenOnRemote registry; this extends the same model to top-level views. * docs(nodes): note hub-only visibility on Fleet, Schedules, Audit, Logs, Auto-Update Each of the five hub-only feature pages now points readers to the canonical "What top-level views show when a remote node is active" section in multi-node.mdx, so users landing directly on a feature page understand why the nav item disappears when they switch to a remote node.	2026-05-08 22:54:58 -04:00
Anso	1803512f70	feat: drop stack labels and network topology to Community tier (#995) ... * feat(labels): drop tier gate to Community for organization endpoints Stack Labels CRUD and per-stack assignment are now Community-tier features: list, create, update, delete labels, and assign labels to a single stack. The two automation surfaces stay Skipper+: per-label bulk deploy / stop / restart (POST /api/labels/:id/action) and the Fleet Actions tab's bulk-assign card (POST /api/fleet-actions/labels/bulk-assign). Tier story is now organize free, automate paid. Add a route-level test that proves the CRUD endpoints succeed on a Community license while the bulk-action endpoint still returns 403. Update overview, licensing, and stack-labels docs to reflect the new tier placement and to note that bulk actions on a label still require Skipper or Admiral. * feat(topology): drop tier gate to Community for network topology view The Resources tab's Networks > Topology view is now available on every tier. Drops requirePaid from GET /api/system/networks/topology, removes the isPaid wrapper around the List | Topology toggle in ResourcesView, and removes the PaidGate around the topology graph. CapabilityGate stays in place so a node running on a build without the network-topology capability still renders its lock card instead of the graph. Add a route-level test that proves the endpoint returns 200 on a Community license. Update licensing and resources docs to reflect the new tier placement. * fix(labels): expose Settings > Labels tab on Community tier The settings registry entry for the Labels tab still carried tier: 'skipper', which kept the tab hidden in the Settings sidebar even though the underlying CRUD endpoints now serve Community. Drop the tier flag so Community users can discover and reach the section that backs the already-Community-tier label endpoints.	2026-05-08 15:24:25 -04:00
Anso	0be479100c	docs: v1 docs refresh (#966) ... * docs(introduction): rewrite intro page and refresh screenshots Rewrite the Getting Started introduction to reflect the current product: adds the Mesh, Blueprints, Pilot, Fleet, Resources concepts up front; restructures capability sections around Stacks, Fleet (now including Fleet Actions), Mesh, Blueprints, Monitoring, Resources, Security, Automation, and Pilot/Remote ops; cross-links every claim to the matching feature page. Replaces the dashboard hero shot with a fresh capture against the redesigned cockpit chrome and adds three inline shots (running stack with anatomy and logs, fleet command deck, resources hub with reclaim header). All screenshots taken at 1920x900, dark theme, with node names, usernames, IPs, and home paths neutralized. Drops outdated claims: stale "190+ templates" count, the "viewer accounts" RBAC summary, and the "atomic" deployment label that did not match the actual rollback mechanism. * docs(introduction): add Federation and Fleet Secrets Federation has shipped (Admiral) with cordon and pin policy as operator overrides on the blueprint reconciler. Fleet Secrets is landing as a Skipper+ tab for versioned env-var bundles encrypted at rest, with diff preview and target push. Mirror those in the Run-one-machine-or-many bullet list and in the Tiers paragraph so the introduction matches the current product. * docs(introduction): re-shoot screenshots against v0.72.0 production Re-capture all four introduction screenshots from the upgraded production node so the fleet view shows the current full tab strip (Overview / Snapshots / Status / Deployments / Traffic / Federation / Fleet Actions / Secrets) instead of the older Overview / Snapshots / Status only. Same 1920x900 dark-theme capture and the same PII scrub applied (node names, usernames, IPs, home paths neutralized). * docs(quickstart): rewrite around v0.72.0 cockpit and add screenshots Replaces the bare install snippet with a five-minute walkthrough that matches the redesigned UI. Leads with a docker-compose.yml block (the bare docker run command is collapsed in an Accordion), keeps the 1:1 path rule, and adds two new sections that show the user what happens on first boot. Adds two screenshots at 1920x900 dark theme: - setup-cold-start.png: the Cold start card with Username, Password, Confirm password fields, and the Initialize console button. - dashboard.png: the post-sign-in dashboard captured against v0.72.0 with the full top nav (Home, Fleet, Resources, App Store, Logs, Auto-Update, Console, Audit, Schedules) and the populated Stack health table sorted by load. Where-to-next now uses CardGroup cols=2 to match the introduction page's pattern. * docs(configuration): align env var reference with current backend Bring docs/getting-started/configuration.mdx up to date with the v0.72.0 backend: - Remove PORT from the optional env vars table. The listen port is hardcoded to 1852 in backend/src/helpers/constants.ts and is never read from the environment. Replace it with a Listen port section that explains the fixed port and host-port remapping. - Document API_RATE_LIMIT (default 200) and API_POLLING_RATE_LIMIT (default 300), both applied in production only. - Note the /app/compose fallback default for COMPOSE_DIR while still pointing readers at the 1:1 path rule. - Point the SSO env var section at the new SSO Quickstart page and keep the SSO feature reference as the deeper dive. - Tighten First boot and cross-link to Quickstart so the screenshot is not duplicated. - Add a Where to next CardGroup matching the refreshed Introduction and Quickstart pages. Drop stale PORT=1852 and JWT_SECRET=your-secure-jwt-secret-here lines from .env.example so the example no longer contradicts the docs (PORT is hardcoded, JWT_SECRET is auto-generated and persisted to the database during initial setup). * docs(configuration): replace em-dash-substitute hyphens in prose Three sentences used ` - ` (space-hyphen-space) as an em-dash substitute. Replaced with the punctuation that fits each case: - "How Sencho organizes your compose directory": semicolon between the two related clauses. - Data directory Warning: parentheses around the parenthetical insertion. - 1:1 path rule explanation: comma before the contrastive clause. Heading slugs and YAML code-block comments are unchanged. * docs(sso): refresh setup guide and drop misleading "one-click" wording Brings the SSO Setup Guide in line with how SSO actually works in the current build, and corrects misleading copy across both SSO docs pages. - Setup Guide: explains the env-var-seeds-once / DB-is-authoritative config model up front, replacing per-section "Restart Sencho" wording that implied a restart was always required. - Setup Guide: promotes the per-provider Test Connection button out of the LDAP-only paragraph into a generic intro callout, and adds a self-signed LDAPS tip. - Setup Guide: notes that all OIDC providers accept a *_DISPLAY_NAME override for the login button label, and adds a commented LDAP TLS toggle to the full compose example. - Setup Guide: adds two screenshots of the redesigned Settings > SSO panel (overview + LDAP card expanded with form). - Both pages: replaces "one-click presets" / "one-click configuration" with "preset providers". The Skipper-tier presets still require an OAuth app provisioned in the provider's console; what they actually buy is provider-aware defaults and a branded login button. The old wording overpromised. * docs(features-overview): regroup catalog and add 17 missing features Restructure the Features Overview into the same six groups the docs sidebar uses (Stacks & Deployments, Observability, Fleet & Multi-Node, Security & Identity, Automation, Platform) plus a short Reference tail. Add catalog entries for 17 shipped features that the previous overview never mentioned: Stack Activity, Stack File Explorer, Deploy Progress, Deploy Enforcement, Blueprints, Git Sources, Global Search, Pilot Agent, Sencho Mesh, Fleet Federation, Fleet Actions, Fleet Sync, Fleet Secrets, Two-Factor Authentication, CVE Suppressions, Auto-Heal Policies, Stack Sidebar. Fix two factual inaccuracies: - Fleet View blurb wrongly gated search, sort, filter, and stack drill-down behind Skipper. The deep-dive is explicit that those are available on every tier; only the bulk Update All action inside the Node Updates modal is paid. - Auto-update entry was titled and described as a scheduling system. The deep-dive page is the Auto-Update Readiness board (risk tags, changelog previews, rollback targets); the scheduler lives under Scheduled Operations. Add three hero screenshots captured from production at 1920x900, illustrating the redesigned cockpit visual language: Home dashboard, stack anatomy, and fleet topology. * docs(stack-management): refresh page around v0.72.0 cockpit and add screenshots Updates the Stack Management page to match the current UI: sidebar with filter chips and label groups, bulk mode, restructured kebab menu, two-tab anatomy panel, and three-source New stack dialog (Empty, From Git, From Docker Run). Adds sections for Filter chips, Pinned and label groups, and Bulk mode. Restructures the Stack context menu around the inspect, organize, lifecycle, and destructive groups with their keyboard shortcuts. Documents the From Git tab and cross-links Git Sources for the full sync flow. Replaces every existing screenshot with fresh captures from the current UI and adds eight new captures for the new sections. Cross-links Stack Activity, Stack Labels, Stack File Explorer, Compose Editor, Atomic Deployments, Scheduled Operations, Auto-Heal Policies, Auto-Update Policies, and Alerts and Notifications for features documented on their own pages. * docs(stack-activity): refresh page around v1 cockpit and recapture screenshots Realign the page with the current Anatomy panel tab strip and the StackActivityTimeline component: - Frame the Activity tab as a sibling of Anatomy under the right-hand panel, with files/edit actions belonging to the strip. - Expand the category guidance: list the five iconized categories and call out that other stack-scoped notifications (deploy failure, available image updates, auto-heal triggers, monitor alerts, scan findings) flow into the timeline with a generic icon. - Tighten the actor-attribution rule to match the component: omitted for events without an actor and for system-driven events. - Add the day-format example to the relative-time row. - Recapture both screenshots from a populated stack (Today + Yesterday + Earlier with three distinct icons) and an empty stack. - Convert troubleshooting blurbs to H3 for anchor links and consistency with the v1-refresh sibling pages. * docs(stack-activity): wrap troubleshooting entries in Accordion blocks Match the foldable troubleshooting pattern established by docs/features/deploy-progress.mdx so the page stays compact and readers can scan to their issue. This is the canonical formatting for the /features section's troubleshooting blurbs going forward. * docs(editor): refresh page around v1 cockpit and recapture screenshots Rewrites the page around the dual-mode right panel (Anatomy by default, Monaco when the user clicks edit), the redesigned Command Center action bar, the new container row layout with status badges and live stats, the Structured / Raw terminal logs toggle, the Git Source toolbar button, and the opt-in diff preview. Drops the obsolete persistent embedded terminal section. Preserves the #diff-preview-before-save and #log-viewer anchors referenced from settings.mdx and global-observability.mdx. Replaces editor-overview.png and container-exec-modal.png with fresh captures against v0.72.0 production at 1920x900 dark theme; renames container -actions.png to containers-list.png; adds command-center.png and editor-edit-mode.png. PII scrubbed (host paths normalized) and the compose-diff-preview/diff-modal.png shared asset is left untouched after visual diff against the live UI showed no chrome change. * docs(features): wrap troubleshooting accordions in AccordionGroup Wraps the loose <Accordion> blocks in editor.mdx and stack-activity.mdx in a single <AccordionGroup> to match the troubleshooting design used by fleet-federation.mdx. Structural only; no content changes. * docs(stack-file-explorer): refresh page around v1 explorer and add screenshots Full rewrite to match the live two-pane Files tab. Replaces the false 'Edit' toolbar flow with the read-only chip + always-on Save model, fixes the protected-files list (5 names) and dedicated-tab redirect list (3 names), drops the fabricated 100 MB download cap, and corrects the upload claim to single file at a time. Adds sections for New File, Rename, Permissions, the type-to-confirm protected-file delete flow, the non-empty folder delete confirmation, the 500-entry tree display cap, and the symlink rendering. Restructures troubleshooting around <AccordionGroup> + <Accordion> to match fleet-federation.mdx, and adds two new entries (display cap, 403 on Community admin write). Ships nine screenshots captured against v0.72.0 production at 1920x900 in dark theme: overview, two-pane layout, protected-tree-marker, viewer-edit-mode, new-file-dialog, context-menu-folder, context-menu-file, permissions-dialog, delete-protected-confirm. * docs(deploy-progress): rewrite around current modal + capture v1 screenshots Aligns the page with the v0.72.0 implementation and standardizes the troubleshooting layout with the rest of the docs refresh. Setting and gating - Renames the Settings field to "Deploy progress modal" and quotes the current helper text verbatim. Documents that the toggle is off by default, lives under Settings > Appearance > Display, is stored in localStorage, and syncs across tabs in the same browser. Modal anatomy - Names every visible UI string: header verbs, status indicator (with the "closes in <n>s" countdown that was previously undocumented), empty-body strings, footer toggle that flips between "Raw output" and "Hide raw", and the destructive border on ERR rows vs the softer warn tint on WARN rows. - Replaces the vague "after a few seconds" with the actual 4-second auto-close timer; documents hover-to-pause and the leave-hover-restarts-the-countdown behavior. - Documents the truncated error message in the failed-state header and the manual close-only requirement. - Notes that the pill is portal-mounted and survives navigation. Stage badges - Keeps the 9-badge table but adds an honest note that most lines render as LOG because the badges are gated on Compose's "[+]" progress prefix, which Compose only emits in TTY mode and Sencho spawns it without one. Entry points - Splits the supported actions into the four that produce a populated structured-log body (Deploy, Update, Install, Git Apply) and the two that bypass compose and finish with 0 lines (Restart, Stop). Drops the "Down" claim from the user-facing list since no UI control currently triggers it; mentions the down route as an automation surface only. Troubleshooting - Wraps the existing accordions in an <AccordionGroup> matching the pattern used by the editor and stack-activity refreshes. Adds two new entries: one explaining the Restart/Stop "0 lines" outcome, one explaining the LOG-everywhere case for non-TTY compose output. Screenshots (six PNGs in docs/images/deploy-progress/, 1920x900, dark theme, captured against the upgraded production node) - setting-toggle.png: the Display section with the toggle enabled. - modal-streaming.png: a real update in flight at 19s, 554 lines. - modal-succeeded.png: succeeded state with the live closes-in countdown visible. - modal-raw-output.png: structured rows with the Raw output panel expanded beneath. - pill.png: minimized pill anchored bottom-center on a stack editor view. - modal-failed.png: failed state with the truncated error in the header and ERR rows highlighted. * docs(resources): refresh Resources Hub page for v1 redesign and feature additions Rewrite the page to match the shipping UI and replace stale screenshots with fresh captures of the redesigned chrome. - Document the admin-only Reclaim hero and clarify the per-tile Sencho-only vs. All Docker (includes external) split in Quick Clean. - Add coverage of the Scan history toolbar button and the per-row severity badge plus shield-icon scan dropdown in the Images tab; cross-link to the vulnerability scanning page. - Correct the Volumes column list (no Size column; size lives on the Largest 5 landing card) and call out admin gates on browse and delete. - Spell out the List/Topology view-mode toggle and that Create Network is admin-only and List-mode only. - Rewrite the Unmanaged tab section around the project-grouped layout, the Select all + Purge Selected (N) admin-only multi-select, and the empty state copy. - Replace screenshots: resources-reclaim, networks-list, create-network, network-inspect, network-topology, network-topology-toggle. Add fresh resources-volumes-tab and resources-unmanaged-tab captures. * docs(app-store): refresh page around v1 deploy sheet, scan integration, and registry settings Rewrites the App Store reference to match the current cockpit: - Documents the weekly-rotated featured banner picked from the top-5 by GitHub stars and the star-descending grid sort. - Adds the deploy-sheet structure (breadcrumb, meta line, About panel with Read more) and splits the Advanced tab into Ports, Volumes, Environment variables, Custom variables, and Security subsections. - Documents the Trivy-gated Security checkbox, atomic vs non-atomic deploys by tier, and the rollback semantics driven by error class. - Adds a Watching the deploy section linking to the deploy-progress modal. - Rewrites the Custom registry section against the new two-panel settings layout (Default + Custom) with the URL validation rule and the using default / using custom hint. - Adds a four-entry Accordion troubleshooting block in the house style. - Replaces three screenshots and adds two (Advanced tab, Settings registry panel) captured against the production node. Permissions wording aligns with current backend (admin only); the broader stack:create gate will land in a follow-up fix branch. * docs(app-store): describe inline port-conflict messaging on deploy sheet Update the deploy-sheet section to match the visible port-conflict behavior: the Essentials tab surfaces a Port-conflict warning that replaces the defaults hint when any default port is already bound, and the Advanced tab shows an inline "in use by {stack}" message next to the container port instead of a hover-only tooltip. Refresh the screenshot alt-text and the troubleshooting Accordion to match. * docs(app-store): align permissions note with stack:create gate Pairs with the backend gate swap in fix/templates-deploy-rbac (#986) which moves POST /api/templates/deploy from requireAdmin to requirePermission('stack:create'). Updates the Note block under 'Watching the deploy' so the docs match the new behavior: admin and node-admin can deploy templates from the App Store; viewer, deployer, and auditor cannot.	2026-05-07 23:44:13 -04:00
Anso	3d9489648e	fix(pilot): harden outbound reverse-tunnel against resource exhaustion (#979) ... * fix(pilot): cap tunnel frame size, concurrent streams, and stream idle time Pilot tunnels carry every HTTP, WS, and Mesh-TCP byte for a remote node through a single multiplexed WebSocket. A buggy or compromised peer that sent oversized frames, opened streams in a tight loop, or left streams parked indefinitely could exhaust gateway memory. Three protocol-level limits applied symmetrically on both ends: - MAX_FRAME_SIZE_BYTES (8 MB): set as the ws maxPayload on the gateway-side WebSocketServer and the agent-side WebSocket client, plus a defense-in-depth length check in decodeBinaryFrame and decodeJsonFrame. - MAX_STREAMS_PER_TUNNEL (1024): bridge refuses new loopback HTTP / upgrade / TCP allocations with 503 once at the cap; the agent rejects new incoming http_req / ws_open / tcp_open with the appropriate error frame. - STREAM_IDLE_TIMEOUT_MS (10 min): every stream gets a per-stream timer refreshed on each inbound or outbound activity. Expiry tears down the local half and notifies the peer. The constants are colocated in pilot/protocol.ts so any future agent build picks them up via the protocol module. * fix(pilot): rate-limit pilot enrollment endpoints to 10 per minute Pilot enrollment mints a JWT and writes a pilot_enrollments row, both privileged operations that should not share the global API budget (200/min). Add a dedicated express-rate-limit instance keyed by user or IP at 10/min in production (100/min in dev so the local enrollment test loop is not throttled). The limiter is applied directly on POST /api/nodes/:id/pilot/enroll. On POST /api/nodes the limiter's skip function reads the parsed body and exempts proxy-mode creates; only requests that resolve to mode=pilot_agent count against the enrollment budget. * fix(pilot): cap concurrent pilot tunnels system-wide The PilotTunnelManager held an unbounded Map of bridges. A reconnect storm, runaway enrollment, or operator misconfiguration could grow the map without limit, even though every tunnel still carries a valid JWT. Cap at 256 concurrent tunnels per primary instance: - Soft warning at 128 (logged once per crossing). - Hard refusal at 256: registerTunnel throws PilotTunnelCapacityError and the upgrade handler closes the WebSocket with 1013 (Try Again Later) so the agent backs off rather than tight-looping. A node that already has a registered tunnel does not consume a new slot when it reconnects; the existing bridge is closed first. * fix(pilot): release backpressure as soon as the tunnel buffer drains Previously the bridge paused HTTP request bodies when the tunnel WebSocket's bufferedAmount climbed above 4 MB but never resumed them explicitly; the next data event re-checked the threshold, and TCP streams only saw a 'drain' fan-out on the 30 s ping cycle. Slow-consumer peers held buffered bytes for tens of seconds longer than needed. Replace both with an on-demand drain check: when at least one stream is paused, sample bufferedAmount every 100 ms; once it drops below the high-water mark, resume every paused request and emit 'drain' to every accepted TCP stream, then stop the timer. Dormant when no stream is paused, so steady-state cost is zero. * fix(pilot): trust internal CAs via SENCHO_PILOT_CA_FILE Self-hosted deployments often terminate TLS with an internal CA. The only previous escape hatch was NODE_TLS_REJECT_UNAUTHORIZED=0, which disables verification across every outbound connection in the agent process and is the wrong shape of fix. Add SENCHO_PILOT_CA_FILE: when set, the agent reads the file as a PEM bundle and passes it to ws as the `ca` option for the tunnel WebSocket. rejectUnauthorized remains true. Failure to read the file exits with a clear error so the operator does not silently fall back to the default trust store. There is no flag to disable TLS verification entirely; that would defeat the credential trust model. * fix(pilot): gate frame-rate-adjacent logs behind developer_mode The agent's malformed-frame warning fires per inbound frame and could flood logs under attack or against a buggy primary. The bridge swallowed parse errors silently, leaving operators blind to protocol drift, and the manager had no signal at all on tunnel registration. Route per-frame and per-tunnel diagnostic logs through the existing isDebugEnabled() helper (same pattern used by middleware/auth, RBAC, and websocket/logs). Production stays quiet by default; the toggle is the existing developer_mode setting in global_settings — no new env var, no new dependency. * feat(pilot): expose per-tunnel metrics via /api/system/pilot-tunnels Operator support had no signal beyond raw logs to answer "is this tunnel flapping?" or "is one bad node hiding behind the aggregate?". Add an in-process counter set covering tunnels_total, tunnels_replaced, tunnels_rejected_capacity, enroll_acks, frame_decode_errors, plus a per-node array carrying the connectedAt and bufferedAmount so a single tunnel sitting on a stuck buffer remains visible. Counters live in services/PilotMetrics.ts; the gateway has no shared in-process metrics facility today, so this is a per-feature pattern documented as such for future consolidation. Counters are strictly process-local — no telemetry, no export, no phone-home — consistent with Sencho's privacy posture. Surfaced read-only via GET /api/system/pilot-tunnels behind admin auth, mirroring the existing cache-stats endpoint. * fix(pilot): tighten reconnect backoff and log handling Three small follow-ups to the hardening pass: - Sanitize the primary URL on the agent's connect log so a malicious SENCHO_PRIMARY_URL with embedded control characters cannot inject fake log lines. - Move the reconnect-backoff reset from the 'open' callback to the 'hello' frame handler. A peer that always rejects the handshake (incompatible version, consumed enrollment token) used to reset the backoff on every TCP-level connect and tight-loop reconnects; now the reset waits for a clean protocol round-trip. - Document the StreamIdAllocator wrap behavior so future readers can see the relationship between the 2^31 wrap and the MAX_STREAMS_PER_TUNNEL cap without re-deriving it. * test(pilot): add coverage for enrollment, rate limiter, and tunnel caps Adds two test files exercising the hardening pass surfaces that were previously uncovered: - pilot-enrollment.test.ts: end-to-end through POST /api/nodes (pilot mode), POST /api/nodes/:id/pilot/enroll, the SHA256 token-hash persistence, replay protection, expired enrollments, and the rate-limit header wiring (enrollment limiter on pilot paths, global limiter elsewhere). - pilot-bridge-limits.test.ts: oversize-frame rejection at both binary and JSON decoders, and the per-tunnel concurrent stream cap as observed via openTcpStream returning null and the loopback HTTP server returning 503 once the bridge is at capacity. Bridge cap test fills the slot map with TCP stream handles rather than real HTTP requests so the OS socket pool stays out of the picture. * test(pilot): cover manager metrics snapshot and capacity-error shape Adds direct coverage for: - PilotTunnelCapacityError exposing the limit so the upgrade handler can format a useful close-frame reason. - PilotMetrics.snapshot returning a defensive copy (callers must not mutate the live counter set). - getMetricsSnapshot returning the open count, per-node breakdown, and counter set in a stable shape, with tunnels_total bumping on a successful registerTunnel. The manager test seeds a real pilot-mode node row first because updateNode throws on missing rows; this models the production path where a node is created before its enrollment is consumed. * docs(pilot): document tunnel limits, custom CA, and new failure modes Refresh the Pilot Agent feature doc to cover the hardening surfaces: - New 'Self-signed primary TLS certs' section walking through the SENCHO_PILOT_CA_FILE env var with an example docker run command. - New 'Resource limits' section listing the per-tunnel and system-wide ceilings so operators know what the wire enforces. - Four new troubleshooting entries: WebSocket close 1013 (system cap), loopback 503 (stream cap), close 1002 with protocol error (frame size / malformed), and HTTP 429 on enrollment. Also expand .env.example with a new pilot-agent block covering SENCHO_MODE, SENCHO_PRIMARY_URL, SENCHO_ENROLL_TOKEN, and SENCHO_PILOT_CA_FILE so operators do not have to read code or the feature doc to discover the agent-side config surface. * fix(pilot): address code-review findings on the hardening pass Critical: - Bridge: TCP backpressure now starts the drain timer and tracks streams awaiting drain, so a TcpStream caller waiting on 'drain' no longer hangs when no HTTP request is in flight. - Bridge: every direct streams.delete call now goes through the removeStream helper so per-stream idle timers are cleared consistently and pausedReqs / tcpAwaitingDrain stay aligned. - Bridge close(): resume any paused IncomingMessage before clearing the map so a parser does not stay stuck across teardown. High: - Protocol: decodeJsonFrame now compares Buffer.byteLength(raw, 'utf8') against the cap; raw.length (UTF-16 code units) let multi-byte payloads sneak ~3x the byte budget through. - Agent: the optional CA bundle is read once at construction and cached on the instance, so a missing or rotated SENCHO_PILOT_CA_FILE no longer process-exits on every reconnect attempt. Medium / Low: - Bridge limits test: assert mockWs.sent.length to confirm slot allocations actually serialized a tcp_open frame and that a rejected (cap+1) attempt did not consume a stream id. - Manager: defer the tunnels_replaced increment until after the cap check passes, so a rejected reconnect does not double-count as both replacement and capacity rejection. - Manager: clarify the protocol comment about MAX_FRAME_SIZE_BYTES enforcement layering (ws maxPayload is authoritative; decoder check is for tests and defense-in-depth). - Limiter: defensive `if (!req.body) return false` so a future refactor that delays body parsing cannot silently skip the enrollment limiter. Deferred (documented as follow-ups, not in this branch): - M2: narrowing public surface of PilotTunnelManager.getBridge to a MeshTunnelHandle interface (cross-cutting refactor). - H4: end-to-end replay test driving the upgrade handler twice with the same enrollment JWT (needs WS test harness).	2026-05-07 21:28:35 -04:00
Anso	fe3bc3e9c3	docs: refresh GitHub README for 1.0 launch (#978) ... New hero with light and dark logo via <picture>, fresh production screenshots (dashboard hero plus four-image gallery covering stacks, editor, fleet, and logs), tighter capability list grouped by surface, compose-first quick start with TLS reverse-proxy callout, dedicated remote-nodes section, and a combined documentation/community/license footer. Also adds .github/FUNDING.yml pointing at the Buy Me a Coffee page so the Sponsor button surfaces on the repo page.	2026-05-07 20:06:19 -04:00
Anso	3b650523c1	Audit-hardening pass for secret and misconfiguration scanning (#977) ... * fix(security): dedupe concurrent compose-stack scans Track stack scans in scanningImages keyed stack:<nodeId>:<stackName>. The /scan/stack route returns 409 when an in-flight scan exists, and the service-side check is the real correctness barrier (the route pre-check is a fast-path optimization that mirrors scanImage). The dedup key release lives in a try/finally so failed scans free the slot for retry. Why: scanComposeStack had no equivalent of scanImage's scanningImages guard, so two simultaneous calls for the same stack would both run trivy config, both insert a vulnerability_scans row, and double- process the result. * feat(security): acknowledge misconfig findings Adds a parallel acknowledgement system for Trivy misconfig findings that mirrors cve_suppressions: a new misconfig_acknowledgements table, read-time enrichment via the new misconfig-ack-filter utility, REST CRUD endpoints, fleet-sync replication from control to replicas, a Settings panel, and an Acknowledge button on the Misconfigs tab. Schema and behavior parity with cve_suppressions: - UNIQUE(rule_id, COALESCE(stack_pattern, '')) so fleet-wide acks collide as expected - blockIfReplica on every write - Audit-log entries name the scope (rule_id, stack_pattern) but never the reason text - replicated_from_control flag controls UI delete affordance and drives clearReplicatedRows on demote/reanchor - Validators reused: validateStackPatternForRedos for glob safety, sanitizeForLog for log fragments SARIF export emits an external/accepted suppression entry per acknowledged misconfig, matching the CVE pattern. Per-row Acknowledge dialog prefills stack_pattern with the scan's stack_context so the default scope is "rule + this stack only" and an operator must broaden explicitly. Tests: misconfig-ack-filter (15) and misconfig-ack-routes (23) including the duplicate-409 case for both pinned and fleet-wide acks. * fix(security): reap orphaned trivy tmp dirs at startup When the buildEnv path writes a per-scan DOCKER_CONFIG dir under os.tmpdir() and the process crashes before the finally block runs, the dir leaks. Mirrors GitSourceService.sweepStaleTempDirs: exported sweepStaleTrivyTempDirs is fire-and-forget at boot, removes prefix-matching dirs older than 1 hour, swallows permission/race failures, logs a single line if any were reaped. * perf(security): emit per-batch summary for scanAllNodeImages Adds one diag() line at the end of scanAllNodeImages summarising unique image count, scanned, skipped, failed, violation count, and elapsed time. Per-image diag inside scanImage stays useful for debugging individual scans; the summary gives operators a single fleet-level checkpoint when developer_mode is on. * perf(security): cap SARIF export at 5000 findings per type Replace the unbounded fetchAllPages walk on /scans/:id/sarif with a hard limit of 5000 findings per type. When any type trips the cap, emit run-level properties.truncated=true plus row_limit and per-type totals so downstream tooling can flag the export as partial. Console-warns for ops visibility. A scan with 50k vulns previously streamed every row into memory before serialising; the cap bounds memory and serialisation time at the cost of completeness on pathological scans. * docs(env): document TRIVY_BIN host-binary override The env var is honored by TrivyService.detectTrivy as a fallback when no managed install is present, but it was undocumented in .env.example. Adds the var with a comment explaining precedence (managed > TRIVY_BIN > PATH). * test(security): cover scanComposeStack failure modes Two new cases drive the existing try/catch through real failure paths: - Malformed Trivy stdout: row flips to status='failed' with the parser error preserved on `error`. - execFile rejection: row flips to status='failed' with a string error message. Pairs with the existing dedup tests so the failure path now also verifies the scan row state, not just the thrown exception. * test(e2e): security scanner + misconfig acknowledgement flow Seven Playwright tests covering the scanner UI and the new acknowledgement system end-to-end: - Trivy availability gate (skips suite when binary absent so CI without Trivy can opt out via E2E_SKIP_TRIVY=1) - Stack config scan completes and records misconfig findings - Concurrent stack scan returns 409 from the dedup gate - Misconfig ack POST creates and lists on Settings - Duplicate (rule_id, stack_pattern) returns 409 - Malformed rule_id (shell metacharacters) returns 400 - Misconfigs tab renders against a real stack scan Tests drive the API for behaviour assertions and the UI only for shell-rendering checks; the visual snapshot suite owns screenshots. * docs(features): add misconfig acknowledgement workflow and SARIF cap Refreshes vulnerability-scanning.mdx with: - Misconfig acknowledgements section covering the per-row dialog, Settings panel, scope/matching rules, and SARIF emission - Tier table row for the new feature - SARIF section note on the 5000 row-per-type cap and the properties.truncated marker for partial exports - Troubleshooting entries: SARIF cap, hidden Acknowledge button, findings resurfacing after delete, Trivy DB phone-home, and 409 on concurrent compose-stack scans * fix(ci): clear backend lint and CodeQL alerts - Remove the dead fetchAllPages helper in routes/security.ts. It lost its callers when the SARIF endpoint switched to direct paged reads for the truncation cap. ESLint flagged it as unused. - Switch the trivy-tmp-cleanup test helper to fs.mkdtempSync. Building paths under os.tmpdir() with predictable names tripped CodeQL's js/insecure-temporary-file rule (high severity), which warns about symlink-pre-creation attacks even in test code. mkdtempSync appends a process-random suffix and creates the dir atomically; the sencho-trivy- prefix is preserved so the production sweep still matches the test fixtures.	2026-05-07 19:23:11 -04:00
Anso	4b1de35dda	Audit-hardening pass for fleet-replicated CVE suppressions (#976) ... * perf(security): bucket CVE suppressions by id at read time applySuppressions now builds a Map<cve_id, suppression[]> once before the per-finding loop, dropping per-finding work to O(matching-cve-suppressions) rather than O(all-suppressions). At a fleet-wide cap of 10000 rows against a multi-thousand-finding scan, the prior linear-per-finding shape drifted into tens of millions of comparisons per render. Public API of findSuppression and applySuppressions is unchanged. Specificity scoring, expiry handling, and image-glob matching are preserved bit-for-bit. Adds a regression guard that pins a 10000x2000 workload under 1.5s and asserts at least one match was actually returned. * feat(security): record audit-log entries on control-side CVE suppression CRUD The replica receive path already wrote an audit-log entry on apply; the control-side POST/PUT/DELETE handlers did not. Operators reading the audit panel saw mirrored security-rule changes from the replica view but could not see who originated them on the control. Symmetric logging closes that gap. The summary records the CVE id and pinned scope (pkg, image) but never the suppression's reason text. Reasons are free-form admin input that replicate fleet-wide and may carry incident-tracker IDs or vendor context the operator did not intend to broadcast. The summary is also sanitised before emission so an operator-supplied package name or image pattern carrying a smuggled newline plus a forged "cve_suppression.delete:" prefix cannot inject a fake row into the audit panel. Control characters become "?" and the field is capped to its validator length. Adds a Control-side audit log block to suppression-routes.test.ts that asserts the privacy contract on each verb (scope present, reason absent), plus a log-injection guard test, plus a regression that GET still works when the local instance is a replica. * test(security): cover cve_suppressions fleet-sync receive path The existing fleet-sync route tests covered the protocol mechanics (auth, anchor, stale push, reanchor, demote) for cve_suppressions only with empty-rows payloads. The suppression-specific concerns were unverified end-to-end: - actual rows replace prior replicated rows on a fresh push - the receive path writes an audit-log entry naming the source fingerprint and row count - a malformed suppression row is rejected at the validator before any DB write Adds a focused block exercising those three properties against the real Express app, real SQLite, and the real apply transaction. * docs(features): refresh CVE suppressions troubleshooting and field guidance Converts the troubleshooting section to Mintlify Accordion blocks so each entry is foldable and the page stays scannable. Adds entries for: - control-identity mismatch on a replica (anchor-aware reanchor flow) - mirrored rules persisting after a control demote - the 10000-row truncation cap on a fleet sync push Adds a privacy note to the Reason field guidance: do not paste credentials, tokens, or vendor secrets there, since the field replicates fleet-wide and surfaces on every node's suppressions panel.	2026-05-07 16:18:19 -04:00