6 Commits

Author	SHA1	Message	Date
Anso	3f1f15a6f4	fix: keep running containers until stack pull/build succeeds (#1657) ... * fix: keep running containers until stack pull/build succeeds Acquire images before reconcile, capture a recovery generation for compensation, and only remove classified orphans after handoff. * fix: address recovery audit blockers for safe stack updates Retire abandoned and expired recovery artifacts, probe compensated runtimes before reporting rollback success, preserve local Docker when deleting a node, validate the exact Compose invocation before capture, and repair updateStack return-contract fixtures. * fix: resolve ESLint errors blocking CI on this branch Unused-import and unused-variable errors left over from the stack deletion refactor: MeshService in stacks.ts (its opt-out cascade moved into DeployedStackDeletionService), a redundant pruneVolumes destructure in deleteDeployedStack (the real one is re-derived from the same input object inside runDeletionBody), and an unused beforeAll import in a Docker-integration test stub. Also scopes the webhook pull-action case body in a block to satisfy no-case-declarations; purely syntactic, no behavior change. * fix: harden recovery probe, cleanup retry, and failed-pull Docker test Reject absent or unhealthy expected replicas before reporting rollback success, keep cleanup records until artifacts are actually removed, fail closed when a mesh override cannot be generated, and assert a real failed pull leaves the original container running. * fix: verify recovery probe image identity and stack-scoped override paths Reject recovered runtimes that use the wrong image or leave scale-zero services running, and confine tombstone override deletion to the intent stack directory so forged cross-stack paths cannot be swept. * test: batch notification cap fixtures in a SQLite transaction Unbatched 1200-row inserts were timing out at the default 30s under CI load even though the same assertions pass in under 2s when green.	2026-07-21 12:18:01 -04:00
Anso	63213c0960	feat: add service-scoped Compose update and restore (#1648) ... * feat: add service-scoped Compose update and restore Allow updating or rebuilding one declared Compose service on multi-service stacks without recreating siblings, with recovery snapshots, health-gate observation, and prune holds for rollback images. Full-stack update paths and single-service UX stay unchanged. * fix: sanitize service-scoped update log messages for CodeQL * fix: address service-scoped update audit findings B-01 through B-07 * fix: complete service-scoped update audit metadata and surfaces * test: wrap Updates readiness tests for deploy-feedback context * fix: keep service recovery reachable without Deploy Progress Make failed service-gate recovery discoverable when Deploy Progress is disabled or dismissed, suppress stale image-scan notification side effects, normalize ComposeService line endings, and add focused regression coverage. * fix: resurface ContainersHealth density and expand on multi-service stacks Service grouping hid the summary strip and Compact/Detailed/Expand controls that still applied to multi-container stacks.	2026-07-19 02:42:29 -04:00
Anso	38aabe7064	feat: health-gated updates and rollback readiness (#1354) ... * feat: classify stack deploy and update failures with suggested next actions Failed deploy and update responses now carry a failure classification (cause category, headline, and suggested next step) derived from the compose error output. The recovery panel and chip render the classification and include it in copied diagnostics, and gateway-style failures surface as a node-unreachable cause. * feat: add update and rollback readiness reports for stacks Before a manual update, Sencho now shows an advisory readiness verdict computed from the stored preflight result, open drift findings, live container health, the pending image change, the rollback backup slot, and node disk headroom. The Stack Dossier gains a rollback readiness section that states what a rollback can restore and explicitly discloses that volume and bind-mounted data are not covered. Toolbar and sidebar updates now share one update path, and admins can create a fleet snapshot from the readiness dialog before updating. Nodes that do not advertise the capability keep the direct update flow. * feat: observe stack health after updates with a post-deploy health gate After a deploy or update succeeds, Sencho now watches the stack for a configurable observation window and records a passed, failed, or unknown verdict: containers must stay running, healthchecks must report healthy, and restart loops or disappearing containers fail the gate. The deploy panel shows the observation live and holds off auto-closing until the verdict lands, a failed gate surfaces the existing recovery actions including rollback, and the stack timeline records update started and gate verdict events. Scheduled, webhook, bulk, and git-source updates are gated the same way; rollbacks and installs are deliberately not. The gate is observational only and can be tuned or disabled per node under host alert settings. * docs: document health-gated updates and rollback readiness New operator page covering the update readiness dialog, the post-update health gate and its settings, the rollback readiness disclosure, and classified failures, with cross-links from the atomic deployments and deploy progress pages. The API reference gains the readiness and health-gate endpoints, the healthGateId success field, and the failure classification schema on deploy and update error responses. * feat: withhold the success verdict while the health gate observes An update used to show a green Succeeded that a failed health gate then contradicted moments later. The deploy modal now reports Verifying health while the gate observes, shows success only when the gate passes, and makes a failed or unknown gate the headline result; success toasts soften to a verifying message while a gate runs. The mobile recovery card groups its actions behind one bottom-right Take action menu so it stays compact on a phone, with the classified cause still visible on the card. A successful image update now also counts as the last known-good marker in rollback readiness, and the docs gain screenshots of the readiness dialog, gate states, dossier section, and settings. * fix: harden log format strings and the env existence path check Log calls that interpolated the stack name into the console format string now use constant format strings with placeholder arguments, and envExists validates path containment inline at its filesystem access, matching the established patterns used elsewhere in the same files. * test: adapt deploy modal success specs to the post-deploy health gate The deploy feedback modal now withholds its success verdict while the health gate observes the new containers, showing "Verifying health" until the gate passes. The two success-path E2E tests waited for "Succeeded" within the gate's 90s default window and timed out. Shorten the observation window to the 15s minimum for these tests via the settings API, assert the verify-then-succeed sequence the modal actually renders, and restore the default window afterward so the test value does not leak into later runs. * fix: serialize health gate polling and harden gate observation Address race conditions in the post-update health gate found in review. Backend: the gate poller used setInterval, so a Docker observe slower than the 5s tick could overlap the next poll and corrupt the restart and missing-container accounting, and a wedged socket could leave a poll pending forever. Polling is now single-flight: each cycle self-schedules the next only after it settles, and the observe is bounded by an 8s timeout so a hung probe counts as a poll error and resolves the gate unknown after three in a row. Frontend: the gate poller could overlap requests, letting a slow earlier "observing" response overwrite an already-applied terminal verdict. It is now single-flight with a terminal latch, so a late response can never roll the UI back from passed or failed. Also reject a non-digit nodeId on the snapshot coverage route instead of letting parseInt coerce it, document that turning off the deploy progress panel opts out of the live gate UI while the gate still runs server-side, and add gate-coverage tests for the webhook, git source, and auto-update apply paths plus the new single-flight, observe-timeout, and recovery cases.	2026-06-11 00:26:26 -04:00
Anso	865d792874	feat(pricing): collapse to two tiers (#1309) ... * feat(pricing): collapse to two tiers (Community + Admiral) Collapse Sencho's pricing from three tiers (Community / Skipper / Admiral) to two: a generous free Community tier and a single paid Admiral tier. The Skipper tier is removed. Now free in Community: auto-heal, auto-update, scheduled operations, webhooks, notification routing, Fleet Actions and bulk operations, SSO preset providers (Google / GitHub / Okta), unlimited users with admin and viewer roles, and deploy safety (atomic deploys, auto-rollback, and one-click rollback). Admiral (paid) is focused on running and governing a fleet: blueprints, Fleet Secrets, deploy enforcement, vulnerability report export, audit log, host console, private registries, mesh networking, node cordon, managed cloud backup, LDAP / Active Directory SSO, and the advanced RBAC roles (deployer, node-admin, auditor) with per-resource scoped assignments. Internally the license variant distinction is removed so tier is binary (community / paid). License validation still verifies the Lemon Squeezy store and product before granting paid status. Docs and the contributor guide are updated to the two-tier model. * docs(pricing): correct licensing page to two-tier pricing and tidy stale tier wording The licensing docs page kept the old Admiral pricing plus a Founder Lifetime column and an Enterprise paragraph after the two-tier collapse. Update it to $12/month or $99/year, drop the lifetime and Enterprise content, and link to the pricing page for current pricing. Also fix stale "Skipper" wording in CLA.md, SUPPORT.md, one test title, and three test comments. Historical CHANGELOG entries and the retired-Skipper license-guard test are intentionally left as-is. * docs: align licensing and SSO pages with the two-tier model Correct the SSO overview so the Google, GitHub, and Okta presets read as available on every tier, matching the provider table; only LDAP and Active Directory require Sencho Admiral. Remove the lifetime-plan references from the licensing, settings, and troubleshooting pages so they reflect subscription-only Admiral pricing. * fix(rbac): omit scoped permissions from /me on the Community tier Scoped role assignments only take effect on the paid tier, but GET /api/permissions/me returned them unconditionally, so a downgraded instance with leftover assignments rendered per-resource affordances the API then rejected with 403. The endpoint now mirrors the permission middleware and includes scoped permissions only on the paid tier. Adds a regression test covering the downgrade case. * docs: use custom-pricing wording on the contact page The two-tier model has no Enterprise tier; reword the contact page's enterprise pricing/deals to custom pricing/deals so it does not imply a tier that no longer exists.	2026-06-04 17:45:53 -04:00
Anso	f03c9dc7b6	fix(webhooks): address Codex review of PR #1177 (#1181) ... * fix(webhooks): close HMAC timing oracle on trigger reject paths The trigger handler in PR #1177 returned a uniform 404 for every unauthenticated rejection, but only the wrong-signature path computed an HMAC over the request body. The other reject paths (unknown id, disabled webhook, non-paid tier, missing X-Webhook-Signature header, missing rawBody) short-circuited before any HMAC work. Repeated near-rate-limit probes with a large attacker-controlled body could distinguish a valid-and-enabled paid webhook id from the other reject cases through response latency. WebhookService.validateSignature is now constant-time over every input shape: it always runs crypto.createHmac and crypto.timingSafeEqual against fixed-length 32-byte buffers regardless of whether the signature is missing, has the wrong prefix, is malformed hex, or is the wrong length. The trigger handler calls it unconditionally before any reject branch fires, using a stable per-process decoy secret (WebhookService.getDecoySecret) when the webhook does not exist and an empty buffer when the request has no body. Response timing now depends only on the size of the request body, which the attacker already controls and which reveals nothing webhook-specific. Six new tests pin the behaviour: validateSignature is observed firing on the unknown-id and missing-signature paths through a spy assertion, and four direct-call tests confirm validateSignature returns false without throwing for empty, wrong-prefix, malformed-hex, and wrong-length signatures. * fix(safe-log): redact Basic auth and lowercase Windows drive letters The redactSensitiveText helper now covers two cases the prior chain missed: * Authorization: Basic <base64> previously left the base64 payload intact. The existing key/value regex caught only the literal word Basic before stopping at the space. A new Basic\s+[A-Za-z0-9+/=]+ replacement runs before the key/value regex so the credential is scrubbed first. * Windows homedir paths like c:\Users\<user>\... with a lowercase drive letter previously slipped through because the regex required [A-Z]. Changed to [A-Za-z] so both letter cases are covered. Two new tests pin both fixes. * docs(webhooks): document 429, fix shared schema, comply with D27/D31 * Trigger endpoint declares the 429 response that webhookTriggerLimiter can return (500 requests per minute per source IP); both docs/openapi.yaml and the response table in docs/features/webhooks.mdx carry the new row, and a new troubleshooting accordion explains the shared-NAT scenario. * Shared Webhook schema in docs/openapi.yaml extends the action enum to include git-pull and documents the node_id property. The GET list endpoint returns these fields; the prior schema would have failed validation for any git-pull row. * docs/features/webhooks.mdx:7 rewritten from a customer-side role enumeration ("non-admins on a paid tier can view the list but cannot manage it") to a single requirement statement ("Webhooks require a Skipper or Admiral license. Managing webhooks is admin-only.") per CLAUDE.md D27/D31; the prior phrasing was customer-side fence-spec. * Two em dashes in webhook description strings I had touched in the prior OpenAPI sync commit replaced with semicolons per D18.	2026-05-23 16:03:48 -04:00
Anso	21ec5e7e0a	fix(webhooks): harden trigger response surface (#1177) ... * fix(webhooks): harden trigger response surface Bundles six audit findings on the incoming-webhooks trigger path. All changes preserve the documented happy path: a CI caller signing the exact request body with the webhook secret still receives 202 Accepted. * Uniform 404 on every unauthenticated rejection (missing webhook, disabled webhook, non-paid tier, missing signature header, missing raw body, signature mismatch). The four-way response surface previously let an unauthenticated probe enumerate webhook ids and fingerprint the instance's licence tier; callers now see one shape for any failed auth. * Fail closed when express.json()'s verify callback did not capture the raw request body. Previously the handler fell back to JSON.stringify(req.body), which compares the HMAC against a re-serialised payload that is not byte-equal to what the client signed. * Pass the already-loaded webhook through to WebhookService.execute() instead of re-fetching by id. Closes the delete-during-execution race where an admin deletion between the trigger handler's load and the async dispatch silently dropped the execution row. The webhook_executions table has ON DELETE CASCADE, so recordExecution now wraps the insert in try/catch and logs a warning when the FK constraint trips because the parent webhook was deleted mid-flight. * Redact bearer tokens, JWTs, URL credentials, and homedir paths from error strings before persisting to webhook_executions.error. The execution history is readable by any paid user via GET /webhooks/:id/ history; redactSensitiveText gains three home-directory patterns (/home/<user>, /Users/<user>, <drive>:\Users\<user>) and now runs on every error stored from this path. * Cap webhook name at 100 characters on both POST and PUT, rejecting non-string and oversized values with 400 before they reach the DB. * Validate the body's action override against a typed allowlist (isWebhookAction type guard) on the trigger endpoint, returning 400 before queueing execution. An unknown override no longer reaches recordExecution as a stored failure row. Tests updated to pass db.getWebhook(id)! instead of the raw id to the new execute() signature. Docs at docs/features/webhooks.mdx updated to reflect the new uniform 404 response, the new 400-on-invalid-action behaviour, and a rewritten troubleshooting accordion that walks operators through every cause of the uniform 404. * test(webhooks): cover trigger handler auth, race, and redaction paths Adds 21 vitest cases for the public webhook trigger handler and the WebhookService.execute / recordExecution pipeline, plus 3 cases for the new homedir patterns in redactSensitiveText. webhooks-trigger.test.ts covers, per audit finding: * M1 + H3 uniform 404: id unknown, webhook disabled, non-paid tier, missing signature header, missing rawBody, sha1= prefix, malformed hex signature, sig mismatch. Each asserts identical 404 body so a future regression that re-introduces 401 / 403 / PAID_REQUIRED breaks one of the 8 tests. * Happy path: 202 with configured action, valid action override, unknown action override returns 400 after auth succeeds (L2), non-string action override returns 400. * L1 name cap: POST and PUT both reject names over 100 chars and non-string names; 100-char boundary still accepted; PUT allows partial updates that omit name. * M5 race: deleting the parent webhook before recordExecution runs no longer crashes the async dispatch; the FK cascade is swallowed with a console.warn, and a happy-path test pins the recordExecution row. * M6 redaction: stubs ComposeService.runCommand to throw errors containing a bearer token and a homedir path, then asserts the persisted webhook_executions.error has both scrubbed. safe-log.test.ts gains three unit tests pinning the new homedir patterns in redactSensitiveText (Linux, macOS, Windows). The existing credentials test is untouched. Tests use prototype spies on FileSystemService and ComposeService (both hand out a fresh instance per nodeId), so per-test mocks do not leak. beforeEach restores all mocks and reapplies the LicenseService 'paid' spy. Closes audit finding H2 (zero trigger-path test coverage). * docs(webhooks): sync openapi spec with new trigger response surface Brings docs/openapi.yaml in line with the behaviour changes from the trigger hardening commit. Mintlify auto-generates the per-endpoint reference pages from this spec, so the spec drift would surface as wrong response codes in the public API reference. POST /api/webhooks and PUT /api/webhooks/🆔 * name: maxLength 100 (matches MAX_WEBHOOK_NAME_LENGTH on the route). * action enum: add git-pull (pre-existing omission; the route has always accepted it). * node_id: documented as an integer property (pre-existing omission). POST /api/webhooks/:id/trigger: * requestBody required: true (body is now mandatory; the H3 fail-closed branch rejects a missing rawBody). * action override: enum restricted to the allowlist. * 401 and 403 responses removed. * 404 response: description rewritten to reflect uniform-404 behaviour; the body is { error: "Webhook not found or signature invalid" } for every unauthenticated reject. * 400 response added for an authenticated request whose action override is not in the allowlist.	2026-05-23 15:42:49 -04:00