mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-18 14:33:19 +00:00
b70a529656a232df0012c56b3280aed9aa49c4fa
22 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Anso
|
b70a529656 |
feat: add developer-mode startup and stack hydration timing (#1619)
* feat: add developer-mode startup and stack hydration timing Instrument boot-to-list and detail hydration with commit-aligned milestones, truthful request stages, and destination/gateway debug duration logs so performance work is guided by measurements. * fix: redact stack names and complete hydration request stages Stop logging stack identifiers in containers debug timing, and record state_dispatch (plus detail fetch spans) so copied reports match the advertised stage breakdown. |
||
|
Anso
|
d9b7911f12 |
fix: distinguish failed image-update checks from "up to date" (#1470)
* fix: distinguish failed image-update checks from "up to date" The image-update detector collapsed every failure (registry unreachable, missing auth, rate limit, unresolved local digest) into hasUpdate:false and dropped the captured reason, so a failed check was indistinguishable from a current image and never raised a notification, even while a manual stack update still pulled a newer image. Detection now records a tri-state per stack (ok / partial / failed) with the failure reason, exposed via a new GET /api/image-updates/detail (the boolean GET / is unchanged so fleet aggregation is unaffected). A fully-failed check preserves the last known has_update, so a transient outage neither erases a real update nor flaps the notification state. The sidebar shows a muted "couldn't check" indicator with the reason on hover, and the Update board lists stacks whose check failed in a "could not be checked" advisory. Detector hardening: the manifest digest lookup issues HEAD first (falling back to GET) so it no longer draws down Docker Hub's anonymous pull-rate budget, and local RepoDigest matching is normalized so official library/* images resolve their digest instead of falling through to a silent "no update". * fix: preserve confirmed updates through partial checks; tighten failure surfacing Address review findings on the tri-state image-update detection: - A partial check (some images errored) no longer erases a previously confirmed update; only a fully-ok check can lower has_update, so a single image's registry blip cannot drop the stack's update and re-fire the notification on recovery. Adds a regression test. - The image-level catch stores getErrorMessage(e) rather than raw String(e), since that value surfaces verbatim in the sidebar tooltip and readiness advisory. - useImageUpdates and the readiness detail fetch now log unexpected non-ok responses instead of silently leaving stale state. - Remove an unused checkFailedCount derivation (the row indicator is driven by the checkStatus prop). - Reword the recordStackCheckFailure docstring and the HEAD-first comment. |
||
|
Anso
|
ca496c89dc |
fix: name matched risk inputs in policy block messages (#1471)
The auto-update, bulk-label, scheduler, and blueprint deploy block messages hardcoded "image(s) exceed <max_severity>", which is wrong under the risk-first policy model: a block can be driven by a known-exploited (KEV) or fixable Critical/High input while the severity threshold was never the trigger. In those cases the message named a severity ceiling the policy did not enforce. Route all four message paths through a shared summarizeBlockReasons helper (the same reason text the deploy-gate 409 response and the block dialog already use), so every surface names the inputs that actually matched. Falls back to a generic phrase when no reason was recorded. |
||
|
Anso
|
7320a86579 |
feat: add cron scheduling mode for image update checks (#1460)
* feat: add cron scheduling mode for image update checks Adds a cron scheduling mode alongside the existing fixed-interval dropdown in Settings > Automation > Image update checks. Users can now set a 5-field cron expression (e.g. "0 3 * * 1") for precise time-of-day scheduling of registry polls. - Backend: ImageUpdateService gains mode/cronExpression fields and cron-based nextDelayMs() using the existing cron-parser dependency. PUT /api/image-updates/interval extended with transactional writes and server-authoritative cron validation matching the Scheduled Operations contract. Nicknames like @daily are supported. - Frontend: UpdatesSection gains a SegmentedControl toggle and cron text input with cronstrue-powered live description. The frontend does advisory validation only; backend 400s are surfaced inline. SettingsPrimaryButton used for explicit "Save schedule" action. - No cron jitter (the user chose a specific time). Interval mode keeps existing ±10% jitter. - Tests: 15 new backend tests covering valid cron, invalid cron, 6-field rejection, nickname support, backward compat, runtime fallback, and transactional writes. - Docs: auto-update-policies.mdx, alerts-notifications.mdx, and openapi.yaml updated with new scheduling mode. * fix: add mode and cronExpression to UpdatesSection test fixtures The existing tests failed because the mock status object was missing the new required fields (mode, cronExpression) added with cron scheduling support. Without them, status.mode was undefined, causing uiMode to never match 'interval' and the Select combobox to not render. * fix: prevent SegmentedControl from stretching full-width in SettingsField The flex-col container defaults items to align-self: stretch, making the Interval/Cron toggle bar span the full card width. Add self-start so it sizes to its content. |
||
|
Anso
|
5f1baa7522 |
fix: harden deploy/update concurrency and node-targeting safety (#1390)
* fix: harden deploy/update concurrency and node-targeting safety Release stabilization for deploy/update operational safety. Per-stack operation locking is now global. Background lifecycle paths (scheduler auto stop/down/start/backup/update, webhook execute, Git source auto-deploy, image auto-update, label bulk actions, fleet snapshot redeploy, and mesh redeploy) acquire the per-node, per-stack lock through a new StackOpLockService.runExclusive helper and skip rather than race a manual deploy/update/rollback/backup on the same stack and node. Skips surface honestly (a failed scheduled run, a recorded webhook failure, a per-stack batch result, or a thrown error) instead of a silent no-op. Update readiness and policy-bypass now run against the node captured when the dialog opened, not the live active node, so switching nodes while a dialog is open cannot retarget the update or the bypass retry. Rollback readiness no longer presents a moving-tag or unpinned image as a ready image revert. Restoring files does not revert a moving tag, so those stacks read as partial, and the rollback success message states that the compose and env files were restored. * fix: lock blueprint reconcile against manual ops and correct rollback wording Follow-up to the deploy/update safety hardening, closing two more gaps from a verification pass. BlueprintService.deployLocal and withdrawLocal called ComposeService directly, so blueprint reconciliation could race a manual deploy/update/rollback/backup on an owned stack. Both now run their compose lifecycle call through StackOpLockService.runExclusive and skip (recorded as a failed reconcile, retried on the next cycle) on conflict. The withdraw holds the lock across both the compose down and the directory delete so neither races a manual operation. The runtime rollback messages overstated recovery: a rollback restores the compose and env files and recreates containers, but does not revert an image behind a moving tag. The auto-rollback deploy-progress output, the recovery panel and chip, the failure toasts, and the manual rollback route message now state that the compose and env files were restored, with the matching OpenAPI example and atomic-deployments doc updated. * fix: acquire stack lock before blueprint deploy mutates compose and marker files Local blueprint deploy wrote the compose and marker files and ran the policy assert before acquiring the per-stack lock; the lock only wrapped the deploy itself. A reconcile could therefore rewrite an owned stack's files while a manual deploy/update/rollback/backup was running. The lock now wraps the whole critical section (create, write compose, write marker, policy assert, deploy), so on conflict nothing is written and the reconcile records a failed outcome. Adds a test asserting a deploy under a held lock records failed, writes no marker file, and leaves the manual lock untouched. * fix: make remote blueprint apply atomic under the receiving node's stack lock Remote blueprint deploy wrote the compose and marker files to the target node via separate HTTP calls and only locked on the final deploy, so the file writes could race a manual operation on that node. A node's operation lock is process-local and cannot be held by the hub across HTTP calls, so the locked create/write/deploy now runs on the receiving node. The locked critical section is extracted into BlueprintService.applyLocalUnderLock and exposed via POST /api/blueprints/apply-local. The hub posts the blueprint to that endpoint in one call; the receiving node runs create + write compose+marker + deploy under its own per-stack lock. Older nodes without the route answer 404 and fall back to the legacy multi-call flow. The endpoint is gated by paid tier and the same per-stack stack:edit and stack:deploy permissions as the PUT-compose + deploy it bundles, validates the stack name, compose size, and marker structure, and returns 409 on a lock conflict without writing anything. Adds tests for the atomic single-call path, the 404 legacy fallback, the 409 lock-conflict mapping, the route validation and permission paths, and the write-compose-then-marker-then-deploy ordering of the shared locked apply. * fix(deps): bump undici to 7.28.0 to clear high-severity advisory The frontend CI npm audit gate (--audit-level=high) failed on a transitive undici 7.25.0 (a dev-only dependency via jsdom): TLS certificate validation bypass (GHSA-vmh5-mc38-953g) and cross-user cache information disclosure (GHSA-pr7r-676h-xcf6). Bumping undici within jsdom's existing ^7.25.0 range to 7.28.0 clears the high-severity advisory and unblocks the frontend job. Lockfile only; no direct dependency or source change. |
||
|
Anso
|
058cf8f2c7 |
feat: make image-update check cadence configurable and visible (#1377)
* feat: make image-update check cadence configurable and visible The background image-update scanner polled registries on a hardcoded 6-hour interval, with no way to see when it last ran or when the next run was due. Operators testing updates read this as auto-update being unreliable: a manual update checks the registry immediately and applies, so the slow background scan rarely raised the "update available" notification before the stack was already current. Backend: - ImageUpdateService reads image_update_check_interval_minutes (15-1440, default 120) and drives a single generation-guarded self-rescheduling timer with 10% per-run jitter so fleet nodes do not poll in lockstep. restartPolling() applies a new interval live, with no restart, and cannot leave a duplicate timer when a save lands mid-scan. - GET /api/image-updates/status now returns checking, intervalMinutes, lastCheckedAt, nextCheckAt, and the manual-cooldown fields. New admin-only PUT /api/image-updates/interval persists the setting and reschedules. Frontend: - New Settings > Automation > Image update checks section to choose the interval (read-only for non-admins; admin enforced on the backend). - The Auto-Update readiness view shows last-checked, next-check, and a ticking manual-recheck cooldown, and the copy distinguishes registry detection from scheduled auto-update execution. Adds backend unit and route tests and frontend component tests, and updates the auto-update documentation. * fix: drop stale image-update status response in the readiness strip loadCadence() ran on mount and again after a Recheck with no request token, so a slow initial /image-updates/status response could resolve after the recheck-triggered one and overwrite the fresh cooldown with stale data, or set state after the view unmounted. Guard setCadence with a monotonic token mirroring loadReadiness, and bump it on unmount. Adds a regression test for the out-of-order resolution. |
||
|
Anso
|
38aabe7064 |
feat: health-gated updates and rollback readiness (#1354)
* feat: classify stack deploy and update failures with suggested next actions Failed deploy and update responses now carry a failure classification (cause category, headline, and suggested next step) derived from the compose error output. The recovery panel and chip render the classification and include it in copied diagnostics, and gateway-style failures surface as a node-unreachable cause. * feat: add update and rollback readiness reports for stacks Before a manual update, Sencho now shows an advisory readiness verdict computed from the stored preflight result, open drift findings, live container health, the pending image change, the rollback backup slot, and node disk headroom. The Stack Dossier gains a rollback readiness section that states what a rollback can restore and explicitly discloses that volume and bind-mounted data are not covered. Toolbar and sidebar updates now share one update path, and admins can create a fleet snapshot from the readiness dialog before updating. Nodes that do not advertise the capability keep the direct update flow. * feat: observe stack health after updates with a post-deploy health gate After a deploy or update succeeds, Sencho now watches the stack for a configurable observation window and records a passed, failed, or unknown verdict: containers must stay running, healthchecks must report healthy, and restart loops or disappearing containers fail the gate. The deploy panel shows the observation live and holds off auto-closing until the verdict lands, a failed gate surfaces the existing recovery actions including rollback, and the stack timeline records update started and gate verdict events. Scheduled, webhook, bulk, and git-source updates are gated the same way; rollbacks and installs are deliberately not. The gate is observational only and can be tuned or disabled per node under host alert settings. * docs: document health-gated updates and rollback readiness New operator page covering the update readiness dialog, the post-update health gate and its settings, the rollback readiness disclosure, and classified failures, with cross-links from the atomic deployments and deploy progress pages. The API reference gains the readiness and health-gate endpoints, the healthGateId success field, and the failure classification schema on deploy and update error responses. * feat: withhold the success verdict while the health gate observes An update used to show a green Succeeded that a failed health gate then contradicted moments later. The deploy modal now reports Verifying health while the gate observes, shows success only when the gate passes, and makes a failed or unknown gate the headline result; success toasts soften to a verifying message while a gate runs. The mobile recovery card groups its actions behind one bottom-right Take action menu so it stays compact on a phone, with the classified cause still visible on the card. A successful image update now also counts as the last known-good marker in rollback readiness, and the docs gain screenshots of the readiness dialog, gate states, dossier section, and settings. * fix: harden log format strings and the env existence path check Log calls that interpolated the stack name into the console format string now use constant format strings with placeholder arguments, and envExists validates path containment inline at its filesystem access, matching the established patterns used elsewhere in the same files. * test: adapt deploy modal success specs to the post-deploy health gate The deploy feedback modal now withholds its success verdict while the health gate observes the new containers, showing "Verifying health" until the gate passes. The two success-path E2E tests waited for "Succeeded" within the gate's 90s default window and timed out. Shorten the observation window to the 15s minimum for these tests via the settings API, assert the verify-then-succeed sequence the modal actually renders, and restore the default window afterward so the test value does not leak into later runs. * fix: serialize health gate polling and harden gate observation Address race conditions in the post-update health gate found in review. Backend: the gate poller used setInterval, so a Docker observe slower than the 5s tick could overlap the next poll and corrupt the restart and missing-container accounting, and a wedged socket could leave a poll pending forever. Polling is now single-flight: each cycle self-schedules the next only after it settles, and the observe is bounded by an 8s timeout so a hung probe counts as a poll error and resolves the gate unknown after three in a row. Frontend: the gate poller could overlap requests, letting a slow earlier "observing" response overwrite an already-applied terminal verdict. It is now single-flight with a terminal latch, so a late response can never roll the UI back from passed or failed. Also reject a non-digit nodeId on the snapshot coverage route instead of letting parseInt coerce it, document that turning off the deploy progress panel opts out of the live gate UI while the gate still runs server-side, and add gate-coverage tests for the webhook, git source, and auto-update apply paths plus the new single-flight, observe-timeout, and recovery cases. |
||
|
Anso
|
865d792874 |
feat(pricing): collapse to two tiers (#1309)
* feat(pricing): collapse to two tiers (Community + Admiral) Collapse Sencho's pricing from three tiers (Community / Skipper / Admiral) to two: a generous free Community tier and a single paid Admiral tier. The Skipper tier is removed. Now free in Community: auto-heal, auto-update, scheduled operations, webhooks, notification routing, Fleet Actions and bulk operations, SSO preset providers (Google / GitHub / Okta), unlimited users with admin and viewer roles, and deploy safety (atomic deploys, auto-rollback, and one-click rollback). Admiral (paid) is focused on running and governing a fleet: blueprints, Fleet Secrets, deploy enforcement, vulnerability report export, audit log, host console, private registries, mesh networking, node cordon, managed cloud backup, LDAP / Active Directory SSO, and the advanced RBAC roles (deployer, node-admin, auditor) with per-resource scoped assignments. Internally the license variant distinction is removed so tier is binary (community / paid). License validation still verifies the Lemon Squeezy store and product before granting paid status. Docs and the contributor guide are updated to the two-tier model. * docs(pricing): correct licensing page to two-tier pricing and tidy stale tier wording The licensing docs page kept the old Admiral pricing plus a Founder Lifetime column and an Enterprise paragraph after the two-tier collapse. Update it to $12/month or $99/year, drop the lifetime and Enterprise content, and link to the pricing page for current pricing. Also fix stale "Skipper" wording in CLA.md, SUPPORT.md, one test title, and three test comments. Historical CHANGELOG entries and the retired-Skipper license-guard test are intentionally left as-is. * docs: align licensing and SSO pages with the two-tier model Correct the SSO overview so the Google, GitHub, and Okta presets read as available on every tier, matching the provider table; only LDAP and Active Directory require Sencho Admiral. Remove the lifetime-plan references from the licensing, settings, and troubleshooting pages so they reflect subscription-only Admiral pricing. * fix(rbac): omit scoped permissions from /me on the Community tier Scoped role assignments only take effect on the paid tier, but GET /api/permissions/me returned them unconditionally, so a downgraded instance with leftover assignments rendered per-resource affordances the API then rejected with 403. The endpoint now mirrors the permission middleware and includes scoped permissions only on the paid tier. Adds a regression test covering the downgrade case. * docs: use custom-pricing wording on the contact page The two-tier model has no Enterprise tier; reword the contact page's enterprise pricing/deals to custom pricing/deals so it does not imply a tier that no longer exists. |
||
|
Anso
|
ca346916c1 |
fix(auto-update): paid-gate execute route and harden image-check watchdog (#1257)
* fix(auto-update): paid-gate execute route and harden image-check watchdog Auto-update execution is a paid capability, but POST /api/auto-update/execute was reachable by any admin regardless of license. Add the paid guard so it matches the rest of the surface (scheduled-task management and fleet refresh). The scheduler dispatch to remote nodes still works because the controlling instance forwards its tier with the request. Restrict GET /api/image-updates/fleet to admins. The single-node status endpoint that drives the sidebar update dot stays open to all roles. Replace the image-check watchdog timer that released the run lock after five minutes. On a healthy but slow scan it let a manual refresh start a second concurrent check, duplicating notifications and racing the status writes. The scan now owns its lock for its full duration, and every Docker socket and filesystem read is bounded so a wedged daemon or mount cannot stall a scan forever. * test(auto-update): assert the debug skip-log branch in the image-check guard The concurrency-guard test covered the warn branch for a trigger arriving past the long-run threshold but never exercised the developer-mode debug skip log. Add a case that enables developer mode and asserts the debug line fires for a mid-scan trigger under the threshold. |
||
|
Anso
|
b33a0e8422 |
fix(deploy-enforcement): surface scan-policy blocks on update and sidebar deploys (#1248)
* fix(deploy-enforcement): surface scan-policy blocks on update and sidebar deploys A blocked deploy only opened the policy dialog from the editor deploy button. The update action and the sidebar context-menu deploy/update fell through to a generic error toast, so an admin could not review the violations or bypass the block from those entry points. Route the 409 policy response through a shared handler on all three paths and make the "Deploy anyway" bypass retry the originating action (deploy or update) so an update bypass still re-pulls images. Also: - Correct the "Block on deploy" policy-editor helper text, which described post-deploy alerting rather than the pre-flight rejection it actually performs. - Dispatch the documented scan_finding warning (policy name and the offending images) when a scheduled auto-update or auto-start is blocked, instead of recording an opaque failure. - Add a standard log line when the gate blocks a deploy, plus developer-mode diagnostics for the matched policy and per-image severity decision. - Fix deploy-enforcement docs: complete the enforced entry-point list, correct the policy-precedence wording, and remove inaccurate tier and audit-actor claims. * fix(deploy-enforcement): surface policy block on rollback and name images in remote auto-update alert Addresses two gaps found in independent review: - Rollback is a policy-gated deploy path (it restores the saved files then re-runs the gate before redeploying), but the frontend treated a blocked rollback as a generic error toast. Route the 409 through the same handler as deploy and update so the block dialog opens, and let an admin "Deploy anyway" retry the rollback with the bypass flag (the rollback route already honors it). - The remote auto-update path dispatched its policy-block warning without the offending image refs, unlike the local scheduler. Append the images so the alert matches the documented contract on every node. Also list rollback as an enforced entry point in the docs and clarify that Git Source enforcement covers both the create-time deploy and a manual apply-with-deploy. |
||
|
Anso
|
adcd04b01a |
refactor(auto-update): retire per-stack gate, drive auto-update from schedules only (#1233)
* refactor(auto-update): retire per-stack gate, drive auto-update from schedules only
The per-stack Auto-update toggle in the stack sidebar context menu wrote a
gate row to `stack_auto_update_settings`, but actual updates only ran when a
`scheduled_tasks` row with `action='update'` fired. On a fresh install the
toggle was inert: detection ran every 6h, nothing was applied.
The same context menu already exposes `Schedule task`, which opens
ScheduledOperationsView pre-filled for the stack where the user can pick
`Auto-update Stack` and any cron. Keeping the toggle alongside that flow
duplicated the same action and turned the gate table into a parallel store
of "is a covering schedule active" derivable from `scheduled_tasks` itself.
Drop the gate model entirely:
- Backend: remove the `stack_auto_update_settings` table and its four
accessors, the three routes under /api/stacks/*/auto-update, the per-stack
skip in /api/auto-update/execute and SchedulerService.executeUpdate's
fleet branch, and the clearStackAutoUpdateSetting call on stack delete.
Dashboard `autoUpdate` count derives from scheduled_tasks (action='update'
rows pinned to the node, total/enabled split).
- Frontend: drop the Auto-update entry from the sidebar context menu and its
optimistic toggle plumbing. Drop autoUpdateSettings state, the
/stacks/auto-update-settings fetch, and the auto-update-settings-changed
WebSocket branch. Slim useSidebarActivitySummary (just nextRunAt; no
enabled/total counts). AutoUpdateReadinessView's per-card autoUpdateEnabled
now means "a covering enabled action='update' schedule exists" (per-stack
row or fleet row on this node, earliest next_run_at wins, per-stack row
wins on ties), with the gate-fetch removed.
- New: scheduledTasksRouter broadcasts scope: 'scheduled-tasks' on POST,
PUT, PATCH /toggle, and DELETE so useConfigurationStatus and
useNextAutoUpdateRun refetch under the 250ms debounce instead of waiting
for the 60s poll. The broadcast is wrapped so a broken subscriber socket
cannot turn a successful mutation into a 500.
- Docs: rewrite the "Per-stack control" section of auto-update-policies.mdx
to describe the schedule-based model; update the matching troubleshooting
entry. The misleading fleet-update help text in ScheduledOperationsView
is corrected to reflect that every stack on the node is covered.
Tier parity: the surviving auto-update path (Schedule task -> Auto-update
Stack / All Stacks) is gated `requirePaid + requireAdmin` backend and
`isPaid + isAdmin` frontend, matching the gate the deleted routes carried.
The pre-commit grep returns no tier-related diff outside this PR's scope.
No data migration is provided: greenfield rules apply, and the leftover
table on already-shipped instances is harmless because no code reads or
writes it after this PR.
* docs: sweep remaining references to the per-stack auto-update toggle
The previous commit retired the per-stack Auto-update gate in favor of
configuring auto-update purely through scheduled tasks. This commit
removes the now-stale mentions of that toggle across the operator docs:
- docs/features/sidebar.mdx: drop the Auto-update entry from the Inspect
group description, the matching screenshot alt-text, and the Skipper
Note that listed it. Schedule task now carries the cross-link to
Auto-Update Policies.
- docs/features/stack-management.mdx: drop the Auto-update list item;
refresh the Schedule task entry to mention the Auto-update Stack action.
- docs/features/dashboard.mdx: rename the Configuration Status row from
"Auto-update stacks" to "Auto-update schedules" with the new value
shape, and rewrite the troubleshooting accordion to describe the
scheduled-tasks invalidation path.
- docs/features/scheduled-operations.mdx: rewrite the Auto-update All
Stacks row and helper text to reflect that every stack on the node is
covered (no per-stack opt-out from this surface anymore).
- docs/features/multi-node.mdx: rewrite the Updates column definition to
derive the Auto/Off flag from enabled Auto-update Stack / Auto-update
All Stacks schedules instead of the removed per-stack policy.
The auto-update-policies.mdx rewrite in the previous commit already
covered the main reference page. The sidebar-context-menu.png screenshot
will be refreshed on release once the new menu is live in production;
the alt text is updated in this commit so it accurately describes the
shipping state.
No website edits needed: the Auto-Update Policies feature card description
("Schedule automatic image pulls and redeployments per stack on your own
cadence") and the feature matrix labels ("Auto-update stack schedule",
"Auto-update all stacks schedule") remain accurate under the new model.
* fix(stacks): drop orphaned requireAdmin import after auto-update route removal
CI's backend lint step flagged this PR's earlier deletion of the three
/api/stacks/*/auto-update routes: those handlers were the only callers of
`requireAdmin` inside routes/stacks.ts, leaving the named import on line 15
unreferenced. `requirePaid` and `effectiveTier` from the same line are still
in use elsewhere in the file and stay.
tsc --noEmit does not flag unused named imports; ESLint's no-unused-vars
does. Local backend lint reproduces and now reports 0 errors against the
existing 334-warning baseline.
|
||
|
Anso
|
2d56ea958a |
fix(stack-activity): per-stack history integrity, attribution, sanitization (#1228)
* fix(stack-activity): per-stack history integrity, attribution, sanitization Address the Stack Activity audit findings (PR 1 of 2): - Per-stack history integrity: drop the per-insert 100-row prune in addNotificationHistory that evicted quieter stacks' history whenever another stack got chatty. Periodic cleanupOldNotifications now caps per (node, stack) at 500 rows and per-node unattached system events at 1000 rows, on top of the existing 30-day retention. Signature takes an options bag and returns a per-stage summary so MonitorService can log what actually ran each cycle. - Actor attribution: thread req.user?.username through every notifyActionFailure call site and add synthetic actors at service emit sites (system:autoheal, system:scheduler, system:image-update, system:docker-events, system:blueprint, system:monitor, system:policy). The timeline renders system actors as "via <Label>" so an autoheal redeploy is no longer indistinguishable from a user redeploy. - Message sanitization: new sanitizeNotificationMessage at NotificationService.dispatchAlert strips KEY=VALUE pairs whose key ends in TOKEN/KEY/PASSWORD/SECRET/CREDENTIALS/AUTH, scrubs HTTP basic auth in URLs and Bearer tokens, collapses COMPOSE_DIR paths, and truncates to 1000 chars. Applied to the stored history and to every downstream Discord/Slack/webhook channel. The ImageUpdateService recovery-path direct DB write also runs through the sanitizer. - Composite pagination cursor: getStackActivity now accepts a (timestamp, id) cursor (?before=&beforeId=). The legacy timestamp-only form silently dropped events when a single compose up emitted many events sharing one millisecond. Route rejects beforeId without before. - Frontend hardening: distinct error state with retry button (initial fetch failure no longer renders as the genuine empty state), strict positive-integer parsing on cursor params, overrequest-by-1 pagination so the last page does not leave a dead "Load more" click, runtime guard on liveEvents merge that validates the level union, per-minute day-bucket recompute so an open panel does not stay on "Today" past midnight. No tier, role, or capability gate touched. Route permission gate remains stack:read on the named stack. * fix(stack-activity): sanitizer covers lowercase env vars and per-node compose dir External review surfaced two leak paths in the message sanitizer: - The sensitive-key regex was uppercase-only. Compose env names are conventionally uppercase but lowercase forms (db_password, jwt_secret, github_token) are valid and do leak through the same Docker and compose-parse error paths. Make the regex case-insensitive and tighten it to also catch bare TOKEN= / KEY= / PASSWORD= without a prefix word, while still leaving BYPASS, COMPASS, and similar non-secret keys alone. - The compose-dir path collapse only read process.env.COMPOSE_DIR, but the real resolution chain is node.compose_dir (per-node DB override) -> process.env.COMPOSE_DIR -> /app/compose. A node with a custom compose_dir could still leak absolute paths into stored history and downstream channels. Route both the dispatchAlert call and the ImageUpdateService recovery-path direct write through NodeRegistry.getInstance().getComposeDir(localNodeId) so the collapse covers every resolution outcome. Tests now assert lowercase keys are redacted and that BYPASS-style non-secrets stay intact in both cases. notification-routing mock extended to stub the new getComposeDir call. * chore(stack-activity): a11y roles, visibility-aware tick, live-disconnect signal Close three small follow-ups on the per-stack activity timeline: - A11y: each day-group gets role="list" and each event row gets role="listitem" so screen readers traverse the timeline as a list instead of a wall of text. The day-group container also carries an aria-label naming the bucket. - Visibility-aware day-bucket tick: the 60s setInterval that re-derives Today/Yesterday/Earlier now short-circuits when document.hidden, so a backgrounded panel does not re-render every minute for no visible effect. - Live-disconnect signal: useNotifications dispatches a sencho:notifications-connection custom event on WebSocket open and close. The timeline listens and, when explicitly disconnected, shows a one-line "Live updates offline; reconnecting…" hint above the list. The sidebar ticker already surfaces fleet-wide connection state; this adds an in-context cue for users who are focused on a single stack. Stack-name case normalization was considered and rejected: stack names are case-permissive per the isValidStackName validator, and lowercasing on read or write would silently rename or hide a user's "MyApp" stack. * ci(stack-activity): drop unnecessary escape in URL_BASIC_AUTH regex ESLint no-useless-escape errored on \- inside the character class [a-zA-Z0-9+.\-] at notificationMessage.ts:14. Move the dash to the end of the class so it's an unambiguous literal and the escape is no longer required. Behavior is identical; sanitizer tests still pass. * revert(stack-activity): drop unvalidated E2E spec from this PR The spec was committed without ever running against a real Docker daemon, then failed in CI when it ran for the first time: deploy returned 200 but no notification appeared on the activity endpoint within the polling window, suggesting either a deploy-notification race or a node-id resolution mismatch in the CI environment. Backend unit tests (route + composite cursor + sanitizer) and frontend component tests cover the same logic. The E2E spec will land in a dedicated follow-up once it has been authored against a working CI environment. |
||
|
Anso
|
2f2401df68 |
fix(fleet): route remaining fleet dispatches through getProxyTarget for pilot-agent nodes (#1152)
* fix(fleet): route remaining fleet dispatches through getProxyTarget for pilot-agent nodes PR #1123 migrated POST /api/fleet/nodes/:id/update to use NodeRegistry.getProxyTarget so pilot-agent rows (no api_url / api_token) participate in remote update via the tunnel loopback. The same bug shape lived on at ten sibling fleet-dispatch sites: each read node.api_url and node.api_token directly, returning "Remote node not configured." against pilots, or silently filtered pilot rows out of a fan-out loop. Migrate every remaining fleet-wide remote dispatch to the same pattern: - routes/fleet.ts: fleet-stop, fleet-prune, prune/estimate, snapshot restore (4 sites) -> getProxyTarget + mode-aware error copy + conditional Authorization header - routes/imageUpdates.ts: fleet status + fleet refresh (2 sites) -> swap n.api_url filter for getProxyTarget != null and use target.apiUrl, so pilot rows appear in the aggregated image-updates view instead of being silently excluded - utils/snapshot-capture.ts: captureRemoteNodeFiles (1 site) -> same pattern; CaptureNode interface gains required mode field so the thrown error message picks the pilot-tunnel copy automatically - services/SecretsService.ts: resolveEnvFileRemote, readEnvRemote, writeEnvRemote (3 sites) -> same pattern; thrown errors now use the shared formatNoTargetError helper instead of leaking api_url/api_token field names Extract the previously-private noTargetMessage helper from fleet.ts into utils/remoteTarget.ts as formatNoTargetError so SecretsService, snapshot-capture, and the fleet routes share one copy of the mode-aware error string. Add 10 regression tests in fleet-pilot-dispatch-parity.test.ts covering each migrated route + the snapshot-capture utility: dispatch through the loopback target with no Authorization header for pilots, and a mode-aware error when the tunnel is disconnected. FleetSyncService (4 additional sites) carries an api_url-anchored targetIdentity in the wire protocol; pilot support there needs a protocol-level identity decision and stays as a separate follow-up. * fix(fleet): throw tunnel-disconnected error from resolveEnvFileRemote Codex audit flagged that resolveEnvFileRemote returned null when getProxyTarget was null. That predates the parity migration but the migration was the right place to fix it: the null flowed through readExistingEnv into previewPushDiff / executePush as "env file not found", which is wrong (the env exists, the node is unreachable). Throwing formatNoTargetError(node) here lets the existing catch arms in previewPushDiff (lines 450-453) surface reachable=false with the tunnel-disconnected message on the right axis, and executePush picks up the same shape via its outer catch. Also drop overstated coverage claims from the parity test header (snapshot restore + SecretsService were never actually exercised in this file, only structurally identical via tsc), and fix two describe labels that read /api/labels/* instead of the mounted /api/fleet/labels/*. |
||
|
Anso
|
6722335a79 |
fix(stack-update): refresh frontend state automatically after a stack update (#1113)
After applying a stack update the sidebar's "update available" dot stayed visible and the stack's status indicator was stuck on the optimistic value until the page was manually refreshed. Two root causes: 1. Image-updates state refresh was a fire-and-forget call in some paths and entirely missing from the bulk-update, auto-update, and state-invalidate WebSocket-handler paths. 2. stackActionsRef.current was resynced only at render time, so the post- update refreshStacks(true) running in the action's finally block read a stale "busy" map and preserved the optimistic mask via prev[file] ?? status. Backend now broadcasts a state-invalidate event with scope='image-updates' and action='stack-updated' after every successful update (single-stack route and auto-update loop). The frontend useNotifications hook routes this to a new onImageUpdatesChange callback wired to fetchImageUpdates in EditorLayout, so every connected client refreshes the dot through the same code path. Bulk update also calls fetchImageUpdates directly for fast local feedback, and setStackAction/clearStackAction now keep stackActionsRef synchronously in sync with state so the busy-stack check inside refreshStacks observes the cleared map immediately. Adds 3 unit tests covering the new WS branch (positive, scope-mismatch negative, auto-update-settings-changed negative). |
||
|
Anso
|
74ae2ce0c6 |
fix: harden atomic deployment rollback (#1029)
* fix: harden atomic deployment rollback * fix: update Docker toolchain to Go 1.26.3 * fix: repair Dockerfile tr argument split across lines * fix: bump protobufjs to clear npm audit high-severity advisories * fix: sanitize error objects in console.error to prevent log injection |
||
|
Anso
|
e5b1c7b22b |
refactor(backend): collapse entitlement provider abstraction back to LicenseService (#889)
Removes backend/src/entitlements/ (registry, loadProvider, CommunityEntitlementProvider, types, headers, normalize) and the two abstraction-only tests. Relocates headers/normalize/types to services/license-*.ts. Swaps 22 consumer call sites from getEntitlementProvider() to LicenseService.getInstance(). Drops the Dockerfile install step plus PRO_PACKAGE_VERSION build-arg and github_token BuildKit secret in docker-publish.yml. Removes the now stale no-restricted-imports rule in backend/eslint.config.mjs. Net: 37 files changed, ~700 lines removed, no behavior change. Local dev no longer requires GitHub Packages auth to start the backend. Rationale and revisit conditions in docs/internal/adrs/2026-05-02-collapse-entitlement-provider.md. |
||
|
Anso
|
3324616e59 |
refactor(backend): extract EntitlementProvider abstraction (Phase 1) (#878)
* refactor(backend): extract EntitlementProvider abstraction (Phase 1)
Phase 1 of the open-core hybrid extraction described in
docs/internal/adrs/2026-05-02-open-core-hybrid-strategy.md. Introduces
the abstraction without moving any code out of the public repo; Phase
2 will actually move services/LicenseService.ts to a private
@studio-saelix/sencho-pro package.
The new backend/src/entitlements/ module contains:
- types.ts. The EntitlementProvider interface plus all tier/license
types (LicenseTier, LicenseVariant, LicenseInfo, SeatLimits,
ActivationResult, etc.). The interface mirrors the existing
LicenseService public surface so the migration was mechanical.
- registry.ts. Module-scope holder for the active provider with
setEntitlementProvider, getEntitlementProvider, and a test-only
reset helper. getEntitlementProvider throws if called before
bootstrap registers a provider; the throw is intentional fail-fast
on a bootstrap-order bug rather than a silent degradation.
- CommunityEntitlementProvider.ts. Phase 2 fallback that returns
community tier and rejects activate(). NOT instantiated in
production today; a smoke test keeps it covered against bitrot.
- loadProvider.ts. Async resolver. Phase 1 returns
LicenseService.getInstance() directly. The async signature matches
what Phase 2 needs (dynamic import of @studio-saelix/sencho-pro
with a "module not found" vs "construction threw" narrowing); the
call site does not change between phases.
- headers.ts. PROXY_TIER_HEADER and PROXY_VARIANT_HEADER constants.
These are part of the wire contract between Sencho instances and
belong in the public core regardless of which entitlement provider
is bound.
- normalize.ts. isLicenseTier, isLicenseVariant, normalizeTier,
normalizeVariant. Domain knowledge about Sencho's tier model
(legacy name maps from pre-0.38.1 versions), not LemonSqueezy
internals. Phase 2 keeps these in the public core.
services/LicenseService.ts now imports its types from
entitlements/types and adds an "implements EntitlementProvider"
clause. Re-exports the types for back-compat with ~20 type-only
consumers; a follow-up PR will sweep those imports to entitlements/
types directly before Phase 2 deletes the file.
bootstrap/startup.ts awaits loadEntitlementProvider, registers the
result, then calls initialize. shutdown.ts calls
getEntitlementProvider().destroy() instead of the LicenseService
singleton.
middleware/tierGates.ts, the chokepoint for ~154 tier-check call
sites, now reads through getEntitlementProvider. Sixteen other
production files (routes/{fleet,imageUpdates,license,permissions,
scheduledTasks,security,stacks,templates,users,webhooks},
services/{BlueprintService,CloudBackupService,SchedulerService,
SSOService}, proxy/remoteNodeProxy, websocket/{hostConsole,
remoteForwarder}, middleware/auth) had their LicenseService.getInstance
calls and utility-export imports redirected to the entitlements
module. The only remaining LicenseService.getInstance in production
code is in entitlements/loadProvider.ts itself, which is the
intentional Phase-1 binding site.
Test infrastructure: setupTestDb registers
LicenseService.getInstance() as the active provider so existing
test files using the helper need no changes. The mocking pattern
many tests use, vi.spyOn(LicenseService.getInstance(), 'getTier'),
keeps working because LicenseService.getInstance() and
getEntitlementProvider() return the same singleton in Phase 1.
scheduler-service.test.ts is the only test that does not use
setupTestDb but exercises tier-gating; it now mocks
entitlements/registry alongside its existing LicenseService mock.
Adds a smoke test for CommunityEntitlementProvider so the Phase 2
fallback class stays covered.
Adds an architecture doc at
docs/internal/architecture/entitlement-provider.md covering the
runtime registry, bootstrap order invariants, and the Phase 1 vs
Phase 2 binding table.
Test results: 89/89 backend test files pass, 1657 passing tests, 5
pre-existing skips. The pre-existing database-metrics > handles
1000+ metrics stress test continues to flake under parallel load
and pass when re-run solo, same flake observed in PRs #862, #863.
* chore(backend): drop unused entitlement type imports from LicenseService
Phase 1 of the EntitlementProvider extraction left five type imports
(ActivationResult, BillingPortalError, BillingPortalResult,
DeactivationResult, ValidationResult) unreferenced after the runtime
methods that produced them began inferring their result shapes via the
EntitlementProvider interface contract. ESLint's no-unused-vars rule
flagged them as errors and failed the lint step in CI.
|
||
|
Anso
|
4e5ba17710 |
refactor(backend): sanitize user input before logging to close CRLF injection (#807)
* refactor(backend): sanitize user input before logging to close CRLF injection
Adds a small sanitizeForLog helper that strips CR, LF, tab, and ASCII
control characters (0x00-0x1F, 0x7F) from a value before it is embedded
in a console.log/warn/error/debug call. Wraps every call site where a
user-controlled value (req.params, req.body, req.query, or a value
derived from them) flows into a log message.
Closes the bulk of the open CodeQL alerts in this family:
- 96 js/log-injection
- 28 js/tainted-format-string
The helper is in backend/src/utils/safeLog.ts. Routes still pre-validate
input at the request boundary; this is the second line of defense and
gives static analyzers a sanitizer they can trace through. JSON
responses, Docker filter labels, and other non-log call sites are
intentionally left unwrapped.
* refactor(backend): printf-style format strings for tainted-log call sites
CodeQL's js/tainted-format-string rule flags template literals in the first
arg of console.X when any interpolated value is user-controlled, regardless
of whether each value is sanitized inline. The canonical mitigation is to
use a static format string and pass values as positional args.
Converts the 28 flagged template literals to printf-style ("%s") format
strings, with sanitizeForLog applied to each positional arg. Also fills in
the log-injection wraps on 9 sites where a user-controlled value was
missed in the first sweep (agents, fleet, gitSources, imageUpdates,
GitSourceService).
No behavior change at runtime. Node's util.format substitutes %s tokens
identically to template-literal interpolation.
* fix(backend): wrap nodeId/snapshotId in fleet restore debug log
CodeQL flagged the unwrapped numeric args even though they cannot
contain control chars in practice. Apply the sanitizer for taint-flow
recognition.
|
||
|
Anso
|
44dba59cab |
feat(notifications): add structured category enum to dispatcher and history (#774)
Introduce a NotificationCategory string-literal union (11 values) and thread it through dispatchAlert as a required second argument. All callers (DockerEventService, AutoHealService, ImageUpdateService, MonitorService, PolicyEnforcement, policyGate, SchedulerService, imageUpdates route) pass an explicit category at every call site, giving TypeScript compile-time enforcement that no new emit site can be added without choosing a category. DatabaseService gains an idempotent migration that adds a nullable category TEXT column to notification_history; existing rows keep category=NULL (displayed as Uncategorized in the UI). The getNotificationHistory method accepts an optional category filter that is forwarded from the GET /api/notifications/history route via a ?category= query param. NotificationPanel gains a category Select dropdown so users can filter history by category. The frontend types mirror the backend union so API responses are type-safe end-to-end. All 75 test files (1410 tests) updated to the new 4-arg dispatchAlert signature and passing. |
||
|
Anso
|
af9cb0aa63 |
feat(auto-update): per-stack auto-update enable/disable toggle (#771)
* feat(auto-update): add per-stack auto-update enable/disable toggle Paid users (Skipper and Admiral) can now opt individual stacks out of scheduled auto-updates from the stack context menu without disabling the global feature. - Add stack_auto_update_settings table (node_id, stack_name) with default enabled=true; four typed DatabaseService accessors with parameterized queries. - Add GET /stacks/auto-update-settings, GET /stacks/:name/auto-update, and PUT /stacks/:name/auto-update (requirePaid + requireAdmin). PUT broadcasts state-invalidate with action auto-update-settings-changed so all open tabs refresh immediately. - Stack DELETE clears the auto-update setting row alongside stack_update_status. - autoUpdateRouter /execute skips disabled stacks before any registry call; skip is recorded in the results array. Manual Update actions are not affected. - Add Auto-update: Enabled/Disabled toggle in the stack inspect group (paid tiers only, hidden for Community, consistent with Auto-Heal). Toggle uses optimistic update with revert-on-error toast. - AutoUpdateReadinessView shows an Auto: Off pill and disables the Apply now button for stacks with auto-updates off. Detection still runs so the readiness card remains visible. - Add 21 backend Vitest tests covering DB round-trips, endpoint auth and tier gates, execute skip for both wildcard and named targets. Add 3 frontend hook tests for toggle visibility and callback behavior. * docs(auto-update): document per-stack auto-update control Add a Per-stack control section to the auto-update readiness page explaining how to disable and re-enable auto-updates for individual stacks, what disabling means (scheduled apply skipped; detection still runs; manual update unaffected), and a troubleshooting entry for scheduled runs not applying to a specific stack. |
||
|
Anso
|
58df1a50b3 |
feat(auto-update): show pending image updates fleet-wide on the Auto-Updates page (#770)
Group readiness cards by node so updates pending on every reachable node are visible without having to switch the active node. Apply now targets the owning node directly, and Recheck fans out to every reachable node in parallel; per-node cooldowns are surfaced in the toast. Adds POST /image-updates/fleet/refresh and invalidates the fleet aggregation cache after auto-update execute so the next read reflects the new state immediately. A small banner appears under the hero when some online nodes did not respond within the request timeout. |
||
|
Anso
|
f5eb993f48 |
refactor(backend): add tests then extract metrics and image-updates routers (phase 4b follow-up) (#738)
Wraps up Phase 4 Round B by tackling the two deferred groups. 25 new integration tests land first and run green against the inline monolith, then each group is extracted byte-for-byte. index.ts drops from ~3,678 to ~3,231 lines; test count rises 1,320 → 1,345. New coverage: - metrics-routes.test.ts (11) — auth + shape checks for /api/stats, /api/metrics/historical, /api/system/stats, /api/system/cache-stats (admin-only), and SSE headers for /api/logs/global/stream - image-updates-routes.test.ts (14) — auth, admin gating, rate-limit tolerance on /refresh, fleet aggregation, /auto-update/execute input validation and no-stacks short-circuit New route files: - routes/metrics.ts — /stats, /metrics/historical, /logs/global (+ SSE /stream), /system/stats, /system/cache-stats. Mounted at /api so the mixed sub-paths line up. - routes/imageUpdates.ts — /api/image-updates CRUD + fleet aggregation, plus a separate autoUpdateRouter mounted at /api/auto-update that owns the /execute handler. Same split pattern as license.ts + systemUpdateRouter. index.ts trims unused imports left behind by the extraction: globalDockerNetwork, si, STATS_CACHE_TTL_MS, SYSTEM_STATS_CACHE_TTL_MS, GlobalLogEntry + log-parsing helpers. |