mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-09-04 06:35:29 +00:00
6a40f0ff3de61be26676aad0317c63734e1119e0
72 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Anso
|
f5178889eb |
feat(recovery): complete authored-project atomic rollback generations (#1819)
* feat(recovery): capture complete authored Compose project for atomic rollback Replace the root-compose-only backup slot with staged recovery generations that record the managed inventory, exact Compose invocation, and prior image identity, and wire the same engine through deploy, update, manual rollback, and Git apply. * fix(recovery): satisfy CodeQL path barriers and update-guard mock Inline resolve+startsWith checks at generation/inventory fs sinks and stub getCurrentStackUpdateRecovery in UpdateGuardService tests. * fix(recovery): drop unused FileSystemService import in generation store test * fix(recovery): harden authored-project rollback for upgrade and restore safety Preserve legacy UUID backup rows, restore Git deploy state with files, make multi-file restore recoverable, evaluate policy on the restored target, and fail closed when Git capture cannot cover an apply. * fix(recovery): unblock Git apply unit tests and CodeQL pre-restore TOCTOU Mock recovery capture in git-source-service tests after fail-closed apply capture, and re-resolve live paths immediately before pre-restore snapshot reads. * fix(recovery): fall back to authored inventory when Git manifesto is missing First Git apply captures before promote, so a missing managed-project manifesto must not block rollback capture when the live stack already has authored files. * fix(recovery): make authored-project rollback atomic across Git state Restore the managed-project manifesto with files, keep nullable Git identity on first-apply captures, persist Git side-state in restore intents for startup reconcile, compensate legacy materialize failures, and refuse directory collisions before mutation. * fix(recovery): satisfy CodeQL path and TOCTOU barriers on manifesto restore Add inline resolve barriers for manifesto read/clear sinks and remove the access-then-read race when restoring a generation manifesto snapshot. * fix(recovery): close third-audit rollback generation blockers Fail closed on incomplete Git inventory fallbacks, execute captured Compose invocation during recovery, refuse startup and mutations while restore intents remain unresolved, propagate legacy stale-delete failures, and add Docker-level exact prior-image coverage plus regression tests. * fix(recovery): mark acquired before handoff in prior-image Docker test Match the production updateStack CAS sequence so the exact prior-image integration test does not fail handoff from the captured phase. * fix(recovery): close fourth-audit rollback safety blockers Evaluate policy against held images, use index-based pre-restore snapshots, hold the shared stack lock across Git apply, replay Mesh and empty captured invocations exactly, restore POSIX modes with fail-closed sensitive permissions, keep case-sensitive paths, and link Git auto-deploy health gates. Add regression coverage for these cases. * test(recovery): fix mocks for health-gate link and authored compose args Add linkGateOrRetain to the Git apply recovery mock, and mock authoredComposeArgs so the case-collision inventory test is not masked by a missing getComposeDir stub. * fix(recovery): close fifth-audit rollback safety blockers Share git_apply locking for webhook auto-apply, fail closed on malformed recovery service records, refuse mixed-image capture, and require exact probe counts with hold-tag eligibility checks. * fix(recovery): close sixth-audit rollback safety blockers Preserve the legacy backup slot during generation capture, encrypt sensitive pre-restore snapshots, revert files on a failed health probe without committing Git, fail closed when an absent-file revert would delete a directory, skip Compose one-offs, route manual and scheduled backup through the current generation, and persist runtime image platform identity. * fix(recovery): close seventh-audit rollback safety blockers Fleet snapshot restore and restore-all now capture a recovery generation under the stack lock before any authored file write, including on remote nodes. * fix(recovery): keep pre-deploy generations during health-gate observe Link deploy recovery generations to the observing gate so backup cannot replace them mid-observe. Distinguish missing hold tags from probe failures, refuse generation release when services metadata is corrupt, classify mixed-replica and coverage refusals, and toast the backend rollback message. * fix(recovery): wrap webhook deploy case for eslint const bindings in an unbraced switch case trip no-case-declarations. Match the pull case block. |
||
|
Anso
|
0d5ff26a8f |
fix(fleet): reject negative or non-finite remote reclaimableBytes in prune estimate (#1803)
A remote node returning HTTP 200 with a negative reclaimableBytes value passed the typeof-only validation in the fleet prune estimate fan-out and was folded into the per-node sum and the fleet total, silently shrinking the estimate. Extend the invalid-response check to reject non-finite and negative values, matching the existing guard for destructive prune plans. Adds route-level tests covering negative, mixed partial, non-finite (1e999 to Infinity), and zero-valued remote estimates. |
||
|
Anso
|
b9e73f7bd8 |
fix(fleet-sync): open status and reset-anchor to Community admins (#1792)
Baseline Fleet Sync already replicates security policy on Community code paths, but status and anchor recovery still required a paid entitlement. Drop the residual paid gates while keeping admin and node:manage authorization boundaries. |
||
|
Anso
|
575848e017 |
fix: keep fleet prune estimate bytes after a target timeout (#1783)
* fix: keep fleet prune estimate bytes after a target timeout Accumulate successful per-target reclaimable bytes on both the local serial loop and the remote fan-out, mark partial nodes, and surface that state in the Fleet prune card instead of zeroing the whole node. * test: clarify zero-byte partial estimate card case Rename the FleetPruneCard assertion so it matches fold semantics: partial with zero bytes means a successful zero-byte target plus a failure, not an all-failed node. * fix: keep fleet prune estimate resilient on init and partial errors Guard DockerController init so a deleted-node race cannot 500 the whole fleet estimate, surface partial-node failure text in the card row title, and pin the generic-rejection partial path with a fast route test. * test: destroy reverse-route connect-ack sockets on teardown Unguarded accepted sockets could emit a late ECONNRESET after assertions passed, failing the Backend Vitest job with zero assertion failures. |
||
|
Anso
|
e2fc3a58a0 |
fix: reduce prune estimate work and add managed-scope timeout (#1768)
* fix: reduce prune estimate work and add managed-scope timeout estimateSystemReclaim previously called getDiskUsageClassified, which walks the full classified-resources pipeline (6+ Docker API calls and filesystem I/O) under the 8 s timeout, but only reads the three reclaimable* fields that getDiskUsage() (a single docker.df() call) already provides. Switch to getDiskUsage() so the timeout actually bounds the work the comment describes. Additionally, the managed-scope estimateManagedReclaim path had no timeout on either the remote route or the fleet local path. Wrap both call sites in withTimeout so a slow daemon surfaces the actionable 'Docker daemon is busy' message within 8 s instead of hanging until the hub's 15 s fetch abort fires. * fix: skip getStacks() for all scope in prune estimate route The remote handler unconditionally walked the compose directory before starting the 8 s estimate timer, but for 'all' scope the knownStackNames parameter is now unused (estimateSystemReclaim uses only docker system df). Mirror the fleet route's conditional so the walk only happens for managed scope, where estimateManagedReclaim genuinely needs stack names. Found during QA: on a Pilot node with real tunnel latency, this unbounded walk added latency outside the timeout budget. * fix: raise prune estimate budget to 12s for large image stores docker.df() cost scales with image-store size: measured ~7.4s on a 34GB / 96-image store, alone nearly exhausting the previous 8s budget before tunnel transport overhead. A healthy Pilot node could flip to 'Docker daemon is busy' at idle load. Raise PRUNE_ESTIMATE_TIMEOUT_MS and FLEET_DF_TIMEOUT_MS to 12s, which sits strictly below the hub's 15s AbortSignal.timeout on the fleet estimate fetch, keeping the remote 503 the actionable failure. The MonitorService janitor keeps its own 8s budget for destructive paths. Found in QA pass 2: single-target estimate failed at ~8.05s on a node where docker.df() alone takes ~7.4s. |
||
|
Anso
|
c613010199 |
feat: account for VM memory ballooning in host memory reporting (#1750)
* feat: account for VM memory ballooning in host memory reporting Extend hostMemory.ts with a readBalloonedMemory() function that parses the Balloon: field from /proc/meminfo, following the same fail-open pattern as the ZFS ARC integration. When a nonzero balloon is detected, effective memory fields (effectiveUsed, effectiveFree, effectiveUsagePercent) are computed and exposed through /api/system/stats and /api/fleet/overview. All consumers that derive meaning from host memory now prefer effective values when present: the dashboard gauge, Fleet card RAM bar, mobile views, health verdict, health status bar stat tile, and host RAM alerts. Backward compatible: missing /proc/meminfo or absent Balloon: line preserves exact current behavior. Old remote nodes without the new fields continue rendering normally. * refactor: extract shared helpers for balloon memory wiring Extract readCandidateFile() and logSelectedPath() in hostMemory.ts to deduplicate ARC and balloon file-read logic. Add memoryToWire() to centralize the optional-field spread used by /api/system/stats and /api/fleet/overview. Add getNodeMemUsed()/getNodeMemTotal() helpers in nodeUtils.ts for frontend byte-text consumers. * fix: make desktop fleet masthead aggregate balloon-aware The desktop fleet overview's memory aggregate in useFleetOverview.ts still summed raw memory.used, while the mobile fleet aggregate and per-node cards already used effective values. Update to use getNodeMemUsed/getNodeMemTotal helpers. * fix: revert balloon adjustment from alerting and health decisions Ballooned memory is host-reclaimed (unlike ZFS ARC, which the guest can reclaim on demand). The guest cannot get ballooned pages back until the hypervisor deflates them, so treating ballooned memory as available for alerting or health can mask real memory pressure. Keep balloon parsing, wire fields, and the dashboard context line as informational-only. The memory gauge, health verdict, and host RAM alerts now use the standard ARC-adjusted working-set percentage regardless of balloon. Updated configuration.mdx and dashboard.mdx to document that balloon data is informational and does not influence alerting. |
||
|
Anso
|
c704cb54d2 | fix(rbac): enforce operational permission parity (#1736) | ||
|
Anso
|
44d6078241 |
feat(fleet): show itemized prune plans (#1734)
* feat(fleet): itemize prune review plans Build and display fingerprint-bound prune candidates for every reviewed fleet node. Preflight all node plans before mutation and preserve detailed removed, skipped, failed, and partial outcomes. Add safe resource metadata projection, managed ownership attribution, runtime contract validation, transport parity coverage, and operator docs. Closes #1724 * fix(security): harden stack path lookup Use a Map for Compose working-directory ownership resolution so untrusted path strings cannot become object property writes. * fix(fleet): harden prune execution safeguards |
||
|
Anso
|
b0b423b234 |
feat(fleet): reapply Compose configuration without a version update (#1716)
* feat(fleet): reapply Compose configuration without a version update Add a distinct Fleet Reapply configuration path so Compose-managed nodes can recreate Sencho from the current on-disk project when already up to date, without pulling or rewriting the image reference. * fix(fleet): confirm remote reapply and close concurrent tracker race Require confirmation for remote compose reapply, and lock dispatch before the remote POST so a second request cannot overwrite a successful in-flight tracker. * fix(ui): icon-only Reapply control so Up to date badge can breathe Collapse the Node updates Reapply label into a tooltip so the status pill no longer wraps in the Status column. * feat(editor): Save & Reapply self-stack via fleet compose reapply (#1726) * feat(editor): Save & Reapply self-stack via fleet compose reapply Eligible admins can apply on-disk Compose edits to Sencho's own stack from the editor using the same confirm, dispatch, and reconnect path as Fleet Node Updates. * fix(editor): gate Save & Reapply label to self-stack only Ordinary stacks were labeled Save & Reapply whenever the node was reapply-eligible. Require the selected file to be the self-stack for the toolbar label and diff confirm CTA. * fix(ui): move compose diff action label helper out of dialog module Keep ComposeDiffPreviewDialog component-only so react-refresh Fast Refresh lint passes after the Save and reapply stacked merge. |
||
|
Anso
|
c90e9606f1 |
fix(dashboard): scope alert-rule count to stacks on the active node (#1722)
Intersect stack_alerts with FileSystemService.getStacks() so Configuration Status (dashboard and fleet local row) counts only rules for stacks that exist on that node, not orphaned names in the global table. |
||
|
Anso
|
3b027957c4 |
fix(fleet): isolate corrupt snapshot file decrypt failures (#1650)
* fix(fleet): isolate corrupt snapshot file decrypt failures A single damaged encrypted fleet-snapshot row no longer fails detail, restore, or off-site upload for the whole snapshot. Unavailable members are marked, restore is blocked before mutation, and cloud upload fails closed with no PutObject. * fix(fleet): fail closed on damaged enc snapshot envelopes Unrecognized enc: payloads no longer fall through as usable plaintext. Only clear legacy prose stays readable; delimiter-byte and similar envelope damage stays unavailable through restore and cloud upload. * fix(fleet): subordinate legacy enc prose to envelope shape Legacy exceptions no longer trigger from = or whitespace alone. Encryption-shaped payloads (length and hex density) stay unavailable through restore and cloud upload, while short genuine prose such as enc:hello remains usable. * fix(fleet): preserve non-envelope enc legacy plaintext Any non-empty enc: payload that is not encryption-shaped is kept verbatim, including punctuation forms such as enc:hello-world, while envelope-shaped damage remains unavailable. |
||
|
Anso
|
972f2b9483 |
feat: stack glob patterns and route severity levels (#1651)
* feat: add stack glob patterns and route severity levels Operators can filter notification routes and mute rules with anchored * globs, and routes can target info, warning, or error. Matching is fail-closed for unsafe stored patterns; write paths keep partial-PUT semantics and ReDoS caps. * test: split mute and routing chip tests to avoid dialog race * fix: move stack pattern client validator out of PatternChips * fix: bound stack glob matching and atomic pattern chip saves |
||
|
Anso
|
63213c0960 |
feat: add service-scoped Compose update and restore (#1648)
* feat: add service-scoped Compose update and restore Allow updating or rebuilding one declared Compose service on multi-service stacks without recreating siblings, with recovery snapshots, health-gate observation, and prune holds for rollback images. Full-stack update paths and single-service UX stay unchanged. * fix: sanitize service-scoped update log messages for CodeQL * fix: address service-scoped update audit findings B-01 through B-07 * fix: complete service-scoped update audit metadata and surfaces * test: wrap Updates readiness tests for deploy-feedback context * fix: keep service recovery reachable without Deploy Progress Make failed service-gate recovery discoverable when Deploy Progress is disabled or dismissed, suppress stale image-scan notification side effects, normalize ComposeService line endings, and add focused regression coverage. * fix: resurface ContainersHealth density and expand on multi-service stacks Service grouping hid the summary strip and Compact/Detailed/Expand controls that still applied to multi-container stacks. |
||
|
Anso
|
83b3d932e5 |
feat: add Apprise as a fourth notification channel (#1644)
* feat: add Apprise as a fourth notification channel Support keyed and stateless Apprise endpoints with secret-safe public DTOs, fail-closed malformed config, and mode-specific Settings UI. Docs and screenshots updated for four-channel Channels and routing. * fix: harden Apprise secrets at rest and preserve-on-write saves Encrypt Apprise endpoint and config with CryptoService so a downgrade cannot leak via SELECT *. Align channel and routing saves so blank destination fields omit config on same-mode URL edits, enforce keyed notify IDs, and keep secrets_redacted truthful. * fix: harden Apprise route type changes and mixed-version config UI Require a raw channel_url when switching notification-route types so ciphertext cannot strand under Discord/Slack/webhook. Default missing remote apprise status, replace Channels state on node switch, and exercise the production config-column migrator. * fix: tolerate stub fleet configuration payloads without agents Normalize remote Apprise agent status only when notifications.agents is present so successful Pilot/stub fetches stay online instead of throwing into the offline catch path. * fix: correct TypeScript in configuration normalize tests * fix: ignore stale Channels agent bodies after node switch Compare the active node after response JSON parsing so a slow body cannot overwrite the newly selected node's channel state. * fix: isolate corrupt Apprise crypto and keep keyed Tags visible Decrypt failures on one Apprise row no longer 500 agent/route lists or suppress sibling channel dispatch. Treat public /notify/<redacted> as keyed so Tags remain editable after reload. |
||
|
Anso
|
35bb74425b |
feat: guide missing external network creation during deploy (#1645)
* feat: guide missing external network creation during deploy Detect missing external networks before Compose runs, prompt or auto-create safe bridge networks, and keep unsupported declarations blocked with trusted deploy provenance. * test: align deploy context and settings fixtures with missing-network gate Update caller spies, EffResource expectations, StacksSection save keys, and git-source spy cleanup so CI matches the new deployStack context and auto-create setting. * fix: drop unused renderError binding in missing-network resolver Satisfies no-unused-vars so backend ESLint CI passes; callers already key only on model presence. * fix: use HTTP-safe clipboard helper in missing-network dialog navigator.clipboard fails on plain HTTP LAN hosts; route copy actions through copyToClipboard so Docker and Compose copy buttons work on self-hosted instances. * fix: simplify missing-network dialog actions and copy label Drop the Compose snippet escape hatch, move secondary actions under More, and rename the terminal copy action to Copy create command so the footer is a clear Cancel / Create decision. |
||
|
Anso
|
381ed2a91f |
feat: add Admiral Hardened Build channel and business assurance surfaces (#1629)
* feat: add Admiral Hardened Build channel and business assurance surfaces Introduce Studio Saelix entitlement-backed Hardened Build switching, a single-flight image operation coordinator, Recovery Vault naming, Admiral Account settings, and typed Fleet update failures while preserving Community custom-repo and targetless pull-current updates. * fix: harden image-op paths and clear CI CodeQL/pilot flake Validate operation IDs before filesystem use, use hostname checks in Fleet fetch mocks, sanitize registry probe logs, and swallow expected TCP teardown errors in the pilot reverse-route post-handshake test. * fix: sanitize image-op docker config write and probe logs Allowlist-copy registry host keys and base64 auth before writing the temp DOCKER_CONFIG, and log registry probe failures with a fixed message so CodeQL no longer flags network-to-file and log-injection mediums. * fix: address Admiral Hardened Build audit blockers Expose imageChannel so hardened Fleet peers still POST for typed rejection, claim community updates before 202, terminalize helper failures, gate Hardened on paid, and align support/docs/e2e wording. * fix: terminalize image ops on helper survival and aborted claims * fix: prevent recreating persist from overwriting helper-exit failure * test: assert helper-exit failure lands before recreating persist * fix: keep current pointer when acknowledging a stale image operation |
||
|
Anso
|
e12602091a |
fix: repin semver compose tags during fleet self-update (#1587)
* fix: repin semver compose tags during fleet self-update Fleet updates failed when docker-compose.yml pinned a semver tag because recreate reused the on-disk pin. Pull the target image first, rewrite semver pins via the update helper, and block digest or unresolved pins with fast 409s. * fix: update OFFLINE_META shape in capability and node-registry meta tests |
||
|
Anso
|
4a350e7a0a | feat: add Docker label audit across Fleet and Stack views (#1531) | ||
|
Anso
|
98667e0d6f | fix: make host memory usage ZFS ARC-aware (#1547) | ||
|
Anso
|
a7144d4e71 |
fix: probe remote RBAC capability live and enforce exact stop-result membership (#1510)
The cross-node capability gate cached its verdict, so a remote replaced by older code at the same URL stayed trusted until the cache expired, reopening the non-admin HTTP escalation and the over-broad stop. The probe now hits the remote's live /api/meta on every gated action (concurrent calls deduped, never cached across requests, fail-closed), so a downgraded remote is detected immediately. Two stop-result gaps are also closed: - A remote stop result must now cover exactly the confirmed stacks (one per stack, no extras, no omissions), not merely exclude extras, so a dropped confirmed stack is no longer accepted as clean. runLocalLabelStop reports one result per confirmed stack even when the label has vanished, so a current remote always satisfies the check. - The local stop exception path now reports the full confirmed set, so a confirmed stack that lost its label is not dropped when the local stop throws. |
||
|
Anso
|
997a6bb79a |
fix: gate cross-node HTTP and stop-by-label on remote RBAC capability (#1509)
An older remote node ignores the forwarded actor-role header (running proxied requests as admin) and ignores the stop-by-label stack allowlist (stopping every label-matched stack). The control could neither detect nor prevent this on a mixed-version fleet. Instances now advertise a cross-node-rbac capability, and the control refuses to act when a remote lacks it: - HTTP proxy: a non-admin user's request is not forwarded to a remote that does not advertise the capability (fails closed when it cannot be determined). Admins are unaffected. - Stop-by-label: a real stop bound to a confirmed stack set is not sent to a remote lacking the capability; the node is reported as needing an upgrade. As defense in depth, a node whose results name stacks outside the confirmed set is failed rather than rendered as a clean stop. Separately, the stop's lock-contention path now reports every confirmed stack as a contention failure (including one that lost its label), so a confirmed stack is never silently dropped and the result is never empty. |
||
|
Anso
|
dd76b13d55 |
fix: require node:read for fleet topology reads and hide Fleet without it (#1507)
The fleet overview, configuration, dependency-map, and networking-summary reads were authentication-only, so a role without node:read (deployer) could read node names, host stats, and cross-node topology. They now require node:read, matching the role model where every role except deployer holds it. For parity, the Fleet nav entry is gated on node:read (hiding it from the top nav, mobile menu, and command palette), the Fleet view redirects to the dashboard when reached without it, and the dashboard fleet heartbeat falls back to the single-node restart map for a role that cannot read fleet data. |
||
|
Anso
|
1dc12f7da8 |
fix: bind fleet stop-by-label to the exact confirmed nodes and stacks (#1506)
A fleet stop re-matched stacks by label name at execution, so a stack that gained the label between the operator's preview and confirmation could be stopped even though it never appeared in the confirmation. A confirmed node that was deleted after the preview also vanished from the results, letting the remaining successes read as a clean stop. The confirm flow now sends the exact node and stack list resolved in the preview. Each node's stop is bound to that set: only stacks that are still label-matched and confirmed are stopped, and a confirmed node missing from the registry is reported as an explicit failure rather than dropped. |
||
|
Anso
|
05c483f213 |
fix: harden cross-node fleet label actions and guard container reads (#1503)
* fix: harden cross-node fleet label actions and guard container reads
Release-stabilization fixes for the Fleet Actions surface:
- Stop-by-label binds execution to the nodes shown in the confirmed
preview. The real stop sends the confirmed node ids and the backend
restricts the fan-out to them, so a node that was unreachable during
preview and reconnects before the stop can no longer enter execution
and have unlisted stacks stopped.
- Bulk label assign validates each remote node's result against the
stacks it was asked to label: a body whose results are empty, partial,
duplicated, or shaped wrong is a per-node failure instead of reading as
a successful zero-stack assign. The card mirrors this, rejecting a
missing or non-array results body and only reporting success when at
least one stack was assigned.
- Bulk label assign re-reads authoritative per-node stacks and labels on
demand via a Refresh control, and the confirmation lists the affected
node and stack names rather than bare counts.
- The stack-specific and fleet container/stack read routes require the
stack:read permission, matching the generic container and stack routes.
Every shipped role already carries stack:read, so reachability is
unchanged; the guard closes the routes that were auth-only.
Adds unit coverage for the assign-result validator, route coverage for
the stop allowlist and assign membership checks, and authorization
coverage for the newly guarded reads.
* test: assert the confirmed node allowlist in the fleet stop-card test
The stop-card component test pinned the real-stop request body to
{ labelName, dryRun } and broke once the stop began carrying the
confirmed-preview node ids. Update it to expect the nodeIds allowlist
derived from the resolved preview, so the test asserts the binding
rather than the pre-fix shape.
|
||
|
Anso
|
4cf04056c9 |
fix: bind the node-update changelog to the advertised release version (#1492)
The changelog tab fetched release notes once and held them in component state without tying them to a version, and the endpoint did not report which release the notes belonged to. When a newer release surfaced while the sheet stayed mounted, reopening the changelog could show the previous version's notes, and a GitHub/Docker Hub fallback or independent cache timing could leave the notes out of sync with the advertised latest version. The release-notes endpoint now returns the release version (normalized tag_name). The changelog keys its loaded notes to the advertised latest version, refetching when that version changes, and labels the notes with the version they belong to so the displayed content is always explicit. |
||
|
Anso
|
315e8b6379 |
feat: add node update alerts with changelog tab and skip-version handling (#1463)
* feat: add node update alerts with changelog tab and skip-version handling - Add node_update_available notification category with blue/brand bell dot - Route node_update_available notifications to Fleet -> Node updates sheet - Add Changelog tab to NodeUpdatesSheet with GitHub release notes - Add per-node skip-version persistence (node_update_skips table) - Skip hides update CTA on node card and sheet; re-surfaces on newer version - Skipped nodes excluded from Update all backend filter - Add pulsating dot indicator on Changelog tab when updates available - Always-visible View changelog action in notification row bottom - Admin-only for all mutating controls (skip, unskip, update) - Backend tests for skip-version semantics (15 tests) - Update fleet-view.mdx, remote-updates.mdx, and OpenAPI spec * fix: address audit findings - nested button, stale changelog, semver normalization, mobile intent - Move View changelog button outside routable button (sibling element) - Fix aria-label for node_update_available notification rows - Support ?recheck=true on release-notes endpoint - Invalidate release notes cache on forced recheck - Store normalized semver (semver.valid strips v prefix) - Skip fleetUpdatesIntent on mobile (desktop only) - Add v-prefix normalization test * fix: restore View changelog on same line as timestamp, opposite sides The button is always visible at the bottom right of the notification card, on the same row as the timestamp (just now), using justify-between layout. * fix: update tests for node_update_available category and release-notes fetch - Backend: monitor-service tests now expect node_update_available instead of system - Frontend: NodeUpdatesSheet tests mock release-notes API call to prevent undefined then() * fix: resolve ci lint failures |
||
|
Anso
|
6527bc971b |
feat(security): gate deploys on exploitation risk, not just severity (#1432)
Scan-policy deploy gates can now block on a known-exploited CVE (CISA KEV) and on a fixable Critical/High finding, in addition to an optional severity threshold. New policies default risk-first (KEV and fixable on, severity off); existing policies keep their severity-only behavior. CVSS stays captured for context but is never the sole basis for a block, and a finding whose exploitability cannot be confirmed is treated as risky rather than safe (incomplete scan detail fails closed on KEV/fixable inputs). The decision logic is shared between the pre-deploy gate and the informational post-scan banner via a pure helper, so the two never disagree. Block messages and the block dialog now name the conditions an image matched. Backend and frontend gates move together, the new inputs replicate across the fleet, and a blocking policy with no active input is rejected on both sides. |
||
|
Anso
|
d26ab58189 |
feat(fleet): cross-node bulk label assign with authoritative label discovery (#1389)
* feat(fleet): cross-node bulk label assign with authoritative label discovery Make Fleet Actions > Bulk label assign work across the fleet. Pick a stack label that exists anywhere in the fleet, select stacks on one or more nodes, and the control orchestrates: each target node resolves the label by name, creating it with the same name and color if missing, then adds it to the selected stacks while preserving their existing labels. The local node runs in process; each remote runs its own admin-only local-assign receiver over the node proxy. Per-node failures (unknown node, no proxy target, unreachable, mixed-version remote) degrade that node only and are reported per node in the result. Assignment writes use a transactional INSERT OR IGNORE so the add-preserve path is idempotent and race-free. Also make the shared fleet label discovery authoritative: suggestions, match-preview, and the fleet-stop remote leg now read each node's labels live over the proxy instead of the control database, which does not mirror remote labels. A propagated label therefore appears in, and is stoppable by, Stop-by-label across the fleet, and unreachable nodes are surfaced rather than silently dropped. Fleet Actions runs against the unfiltered node list, so overview filters no longer narrow its scope. The previous node-scoped, replace-by-id bulk-assign endpoint is removed. * fix(fleet): treat malformed remote label responses as per-node failures A 200 response from a remote node whose body is not the expected shape was treated as a benign empty result, so a malformed remote could read as a clean zero-stack assign or a "matched, nothing to stop" no-op and even surface a success toast. Validate the wire shape in the bulk-assign and fleet-stop remote legs and in the authoritative label discovery fan-out; on a malformed body, report the node as a per-node failure with the error attributed to its stacks instead of silently dropping it. * chore: drop accidentally committed temp file |
||
|
Anso
|
2821e87d3f |
fix: gate fleet stop-by-label on a resolved preview and validate remote stop responses (#1392)
* fix: gate fleet stop-by-label on a resolved preview and validate remote stop responses The destructive "Stop fleet by label" action could run before its blast radius was known, and a malformed remote response could be rendered as a successful zero-stack stop. Both are release blockers for the fleet stop-by-label flow. Gate the "Stop fleet" button on a resolved blast radius: the live match-preview resolving to at least one matching stack, or a dry run of the current label when the preview endpoint is unavailable. Carry the resolved node and stack list into the confirm modal so the operator confirms against the concrete targets rather than a label name alone. Editing the label invalidates a prior dry-run snapshot, so a stale blast radius cannot re-enable the action. Validate the remote local-stop 200 body before trusting it. A body that is not the local-stop contract (missing matched flag, non-array results, or a malformed result element) now fails that node with a clear error, consistent with the non-ok and unreachable paths, instead of defaulting matched to true and results to empty. * fix: ignore stale fleet stop-by-label preview responses after the label changes The debounced match-preview callback set the preview state unconditionally, so a request issued for one label that resolved after the operator switched to another label would mark the preview ready with the old label's blast radius. That re-enabled the destructive Stop and showed stale nodes/stacks in the confirm modal for a label that no longer matched them. Guard the in-flight request with a per-run cancelled flag flipped in the effect cleanup, so a response for a label that is no longer current cannot set the preview. This mirrors the cancellation pattern already used by the suggestions effect. Add a test that holds a preview request in-flight, switches the label, then resolves the stale response and asserts Stop stays disabled and the old targets do not leak. |
||
|
Anso
|
5f1baa7522 |
fix: harden deploy/update concurrency and node-targeting safety (#1390)
* fix: harden deploy/update concurrency and node-targeting safety Release stabilization for deploy/update operational safety. Per-stack operation locking is now global. Background lifecycle paths (scheduler auto stop/down/start/backup/update, webhook execute, Git source auto-deploy, image auto-update, label bulk actions, fleet snapshot redeploy, and mesh redeploy) acquire the per-node, per-stack lock through a new StackOpLockService.runExclusive helper and skip rather than race a manual deploy/update/rollback/backup on the same stack and node. Skips surface honestly (a failed scheduled run, a recorded webhook failure, a per-stack batch result, or a thrown error) instead of a silent no-op. Update readiness and policy-bypass now run against the node captured when the dialog opened, not the live active node, so switching nodes while a dialog is open cannot retarget the update or the bypass retry. Rollback readiness no longer presents a moving-tag or unpinned image as a ready image revert. Restoring files does not revert a moving tag, so those stacks read as partial, and the rollback success message states that the compose and env files were restored. * fix: lock blueprint reconcile against manual ops and correct rollback wording Follow-up to the deploy/update safety hardening, closing two more gaps from a verification pass. BlueprintService.deployLocal and withdrawLocal called ComposeService directly, so blueprint reconciliation could race a manual deploy/update/rollback/backup on an owned stack. Both now run their compose lifecycle call through StackOpLockService.runExclusive and skip (recorded as a failed reconcile, retried on the next cycle) on conflict. The withdraw holds the lock across both the compose down and the directory delete so neither races a manual operation. The runtime rollback messages overstated recovery: a rollback restores the compose and env files and recreates containers, but does not revert an image behind a moving tag. The auto-rollback deploy-progress output, the recovery panel and chip, the failure toasts, and the manual rollback route message now state that the compose and env files were restored, with the matching OpenAPI example and atomic-deployments doc updated. * fix: acquire stack lock before blueprint deploy mutates compose and marker files Local blueprint deploy wrote the compose and marker files and ran the policy assert before acquiring the per-stack lock; the lock only wrapped the deploy itself. A reconcile could therefore rewrite an owned stack's files while a manual deploy/update/rollback/backup was running. The lock now wraps the whole critical section (create, write compose, write marker, policy assert, deploy), so on conflict nothing is written and the reconcile records a failed outcome. Adds a test asserting a deploy under a held lock records failed, writes no marker file, and leaves the manual lock untouched. * fix: make remote blueprint apply atomic under the receiving node's stack lock Remote blueprint deploy wrote the compose and marker files to the target node via separate HTTP calls and only locked on the final deploy, so the file writes could race a manual operation on that node. A node's operation lock is process-local and cannot be held by the hub across HTTP calls, so the locked create/write/deploy now runs on the receiving node. The locked critical section is extracted into BlueprintService.applyLocalUnderLock and exposed via POST /api/blueprints/apply-local. The hub posts the blueprint to that endpoint in one call; the receiving node runs create + write compose+marker + deploy under its own per-stack lock. Older nodes without the route answer 404 and fall back to the legacy multi-call flow. The endpoint is gated by paid tier and the same per-stack stack:edit and stack:deploy permissions as the PUT-compose + deploy it bundles, validates the stack name, compose size, and marker structure, and returns 409 on a lock conflict without writing anything. Adds tests for the atomic single-call path, the 404 legacy fallback, the 409 lock-conflict mapping, the route validation and permission paths, and the write-compose-then-marker-then-deploy ordering of the shared locked apply. * fix(deps): bump undici to 7.28.0 to clear high-severity advisory The frontend CI npm audit gate (--audit-level=high) failed on a transitive undici 7.25.0 (a dev-only dependency via jsdom): TLS certificate validation bypass (GHSA-vmh5-mc38-953g) and cross-user cache information disclosure (GHSA-pr7r-676h-xcf6). Bumping undici within jsdom's existing ^7.25.0 range to 7.28.0 clears the high-severity advisory and unblocks the frontend job. Lockfile only; no direct dependency or source change. |
||
|
Anso
|
cb58cc423f |
fix(fleet): resolve Stop-by-label stack labels across all nodes (#1382)
* fix(fleet): resolve Stop-by-label stack labels across all nodes The Fleet Actions "Stop by label" card only ever saw the control node's own stack labels. The stack-label routes are proxied, so each node stores its labels in its own database and the control holds no mirror for remote nodes. The suggestions, match-preview, and fleet-stop endpoints all read that nonexistent mirror, so remote-only labels were invisible and remote stacks were skipped before the remote was ever asked. Make all three authoritative across the fleet with a new collectFleetLabelSummaries helper: the local node reads its own database, and each remote is queried live through its labels and label-assignments endpoints over the proxy, with fail-closed parsing and per-node reachability. fleet-stop drops the mirror pre-check and always calls each reachable remote's local-stop receiver, reporting unreachable nodes at the node level so they never block the reachable ones. The picker now surfaces remote-only labels, aggregates shared names once with combined counts and the carrying node names, and flags incomplete coverage when a node is unreachable. The preview groups matches per node, lists unreachable nodes separately, and distinguishes no matching stacks from "label exists but no stacks" from "remote unavailable". * fix(fleet): harden Stop-by-label card against malformed responses Guard the match-preview and fleet-stop response bodies so a malformed but 200 reply degrades instead of crashing or misreporting. match-preview now validates the per-node shape before rendering (the preview reads it outside any try/catch) and logs a malformed body; fleet-stop distinguishes a non-array results body (a server bug, now logged and surfaced as an unexpected-response error) from a genuine empty fleet, and guards per-node stackResults. Docs: an unreachable or errored remote is reported once per node, not as a per-stack error row. |
||
|
Anso
|
4610a433e6 |
fix(fleet): scope Stop-by-label to stack labels with a typed suggestion source (#1368)
* fix(fleet): scope Stop-by-label to stack labels with a typed suggestion source The Fleet Actions "Stop by label" card labelled its target field generically as "Label", so a same-named node label could look like a valid stop target in a destructive workflow. The action has always matched stack labels only, but nothing in the copy or the data flow made that explicit. Add a stack-label-only suggestions endpoint and make the scope unmistakable: - New GET /api/fleet/labels/suggestions aggregates the per-node stack labels into a name-keyed list with stack and node counts (admin-only, central DB, covers every configured node including offline remotes). Node labels are never folded in. - The card now sources its autocomplete from that endpoint and renders each suggestion with its stack and node counts via a typed FleetStopLabelSuggestion model, so node-label data cannot be fed into this destructive card. - Copy is explicit throughout: "Stack label" target field with a helper line that node labels are not used, a clear "0 matching stacks" readout and a "No stacks are assigned to this stack label" empty preview, and confirm and result copy that references stacks and the stack label. - Docs updated (fleet-actions, stack-labels) and tests added on both sides, including node-only exclusion, name collision, multi-node counts, the zero-stack preview, and the non-fatal suggestions-load path. * docs: correct stale Stop-by-label button and helper references The Stop-by-label walkthrough referenced a "Stop matching stacks" button and a warning callout that no longer exist on the card. Align the docs with the live card: the primary action is "Stop fleet", and the scope is stated by the helper line under the input. |
||
|
Anso
|
77f1611971 |
feat: Compose Network Inspector and exposure intent guard (#1360)
* feat: add Compose Network Inspector facts engine Render a stack's authored effective model and pair it with the live Docker snapshot to derive per-stack networking facts: project networks with external and internal flags, service-to-network membership and aliases, published ports with host-binding scope, network_mode, and extra_hosts, plus runtime drift (runtime-only attachments, foreign networks, and declared-but-unused or missing networks). Extend the effective-model parser with service network membership, extra_hosts, and label keys (key names only, never values), and add a key-space normalized network model with adapters from both the rendered model and the raw declared compose so the Inspector and drift share one comparison. Expose GET /api/stacks/:stackName/networking: advisory and read-only, it renders the authored model only and never returns or logs raw stderr, env values, or label values. * feat: store and edit per-stack and per-service exposure intent Add a stack_exposure_intent table (intent values constrained by a CHECK, unique per node, stack, and service) with DAO methods to read, upsert, clear one row, and clear all rows for a stack. The classification is stored independently of the generated networking facts so a later mismatch stays detectable; service rows are kept separately from the stack-level row (service ''). Expose GET and PUT /api/stacks/:stackName/exposure: GET requires read access, PUT requires edit access and validates the intent against the allowed set. Sending intent null clears that row, returning the scope to unset so a service inherits the stack intent again. Intent rows are cleared when the stack is deleted and when the owning node is removed, so a later same-named stack never picks up stale classification. * feat: add exposure-aware Compose Doctor findings Feed the Compose Doctor's effective-model context with the stored exposure intent (resolved into a stack-level value plus per-service overrides) and the dossier's documented access-URL ports, read fail-soft so a metadata read error skips these checks rather than failing the preflight. Add five deterministic findings on top of that context: - a service classified internal or same-node that publishes a host port (same-node tolerates a loopback bind), - a sensitive database or admin image published on all interfaces, - a port-publishing stack with no exposure intent set, - a published port not reflected in the documented access URLs, - reverse-proxy labels with no documented URL or reverse-proxy intent. The rules stay pure functions over the preflight context; the registry completeness test pins the new rule set. * feat: detect compose network drift in the drift ledger Extend the spatial drift engine with two network-level findings: a running container attached to a stack-owned or foreign network that compose does not declare (one finding per service), and a declared network that no running service uses or that is absent from the runtime (one stack-level finding, every network named by its resolved runtime name). The comparison reuses the same helper the Network Inspector uses, so the two surfaces never disagree. Network drift runs only when the stack has running containers and the runtime is reachable, preserving the existing missing-runtime, parse-error, and unreachable behavior. The findings persist through the existing drift ledger and surface on the Drift tab, which now labels the two new kinds. * feat: link a Docker network back to its owning stack Add a cross-component open-stack event and make the owning-stack badge on a managed network in Resources a link: clicking it loads that stack on its node and opens the editor, reusing the existing fleet navigation. A latest-ref keeps the window listener current without re-subscribing each render. Image and volume badges are unchanged; only a managed network opts in via the new optional handler. * feat: add the Networking tab to the stack detail panel Add a capability-gated Networking tab that reads the per-stack networking facts and exposure intent. It shows the project networks (with external, internal, and created-by-stack flags), per-service network membership and aliases, published ports with their host-binding scope, network_mode and extra_hosts, and runtime drift, degrading to the declared model when the runtime is unavailable. Users can classify the stack and each service (internal, LAN, reverse proxy, public, and so on) or clear a row to inherit; the controls are read-only when the user cannot edit, and a broken exposure response never tears down the facts view. A new compose-networking capability is added to both registries so older nodes hide the tab, and the tab cross-links to the Doctor for the deploy and security findings. * docs: document the Compose Networking tab Add a feature page covering the Networking tab: the network facts, published ports and host bindings, the exposure-intent classification and inheritance, the exposure-aware Doctor findings, runtime drift, and a troubleshooting section. Register it in the docs navigation next to Compose Doctor. * feat: add a redacted network summary to the Stack Dossier export Append a network exposure section to the dossier Markdown: the stack and per-service exposure intents, the networks with their external and internal flags, and each service's published ports with their binding scope. It carries only names, intents, port numbers, and scope, never an env value or a label value. The summary is fetched only when the user exports (copy or download), so opening the panel costs nothing, and it degrades to omitting the section when the data is unavailable. The whole-fleet dossier export collects the same summary per stack, rethrowing the unauthorized sentinel like the sibling loaders. * feat: add a Fleet networking filter for exposure and drift Add a per-node networking summary that classifies a node's stacks as exposed (a host port published beyond loopback), unknown-exposure (publishes ports with no exposure intent set), or network-drift. It reads each stack's compose with the light dependency parser and one Docker snapshot, so it stays cheap across a node's full stack set, and it skips drift when the runtime is unreachable rather than inventing it. Serve it node-locally at GET /api/networking/summary, and aggregate it fleet-wide at GET /api/fleet/networking-summary: the hub computes its own summary in-process and reaches each remote through its node-local route, degrading an unreachable or older node to a skip. Because the aggregate lives under the proxy-exempt /api/fleet prefix it is never wrongly proxied. The Fleet overview gains a networking filter chip backed by that aggregate, fetched fail-soft and detached so it never gates the grid. * fix: spin the Networking refresh button while it reloads The refresh button silently refetched the same data, so a click gave no feedback. Track a refreshing state and spin the icon while the load is in flight, disabling the button, matching the Compose Doctor preflight button. * fix: apply effective per-service exposure intent to unclassified checks The "unclassified exposure" decisions only consulted the stack-level intent row, so a service classified directly (with no stack row) was still reported as unclassified, and a service explicitly marked unknown over a classified stack was missed. Both the exposure-unclassified preflight rule and the networking summary's unknown-exposure bucket now resolve the effective intent per publishing service (service row overrides stack row), matching the precedence already used by the exposure-internal-published rule. * fix: resolve drift network names via the compose top-level name When a compose file sets a top-level name:, Docker prefixes resource names with that project name instead of the stack directory. The light dependency parser dropped name:, so network-drift normalization compared runtime networks against directory-prefixed names and reported false network-undeclared / network-missing findings. Carry the parsed project name through DeclaredCompose and use it when normalizing declared networks for drift, while still filtering containers by the stack directory. |
||
|
Anso
|
38aabe7064 |
feat: health-gated updates and rollback readiness (#1354)
* feat: classify stack deploy and update failures with suggested next actions Failed deploy and update responses now carry a failure classification (cause category, headline, and suggested next step) derived from the compose error output. The recovery panel and chip render the classification and include it in copied diagnostics, and gateway-style failures surface as a node-unreachable cause. * feat: add update and rollback readiness reports for stacks Before a manual update, Sencho now shows an advisory readiness verdict computed from the stored preflight result, open drift findings, live container health, the pending image change, the rollback backup slot, and node disk headroom. The Stack Dossier gains a rollback readiness section that states what a rollback can restore and explicitly discloses that volume and bind-mounted data are not covered. Toolbar and sidebar updates now share one update path, and admins can create a fleet snapshot from the readiness dialog before updating. Nodes that do not advertise the capability keep the direct update flow. * feat: observe stack health after updates with a post-deploy health gate After a deploy or update succeeds, Sencho now watches the stack for a configurable observation window and records a passed, failed, or unknown verdict: containers must stay running, healthchecks must report healthy, and restart loops or disappearing containers fail the gate. The deploy panel shows the observation live and holds off auto-closing until the verdict lands, a failed gate surfaces the existing recovery actions including rollback, and the stack timeline records update started and gate verdict events. Scheduled, webhook, bulk, and git-source updates are gated the same way; rollbacks and installs are deliberately not. The gate is observational only and can be tuned or disabled per node under host alert settings. * docs: document health-gated updates and rollback readiness New operator page covering the update readiness dialog, the post-update health gate and its settings, the rollback readiness disclosure, and classified failures, with cross-links from the atomic deployments and deploy progress pages. The API reference gains the readiness and health-gate endpoints, the healthGateId success field, and the failure classification schema on deploy and update error responses. * feat: withhold the success verdict while the health gate observes An update used to show a green Succeeded that a failed health gate then contradicted moments later. The deploy modal now reports Verifying health while the gate observes, shows success only when the gate passes, and makes a failed or unknown gate the headline result; success toasts soften to a verifying message while a gate runs. The mobile recovery card groups its actions behind one bottom-right Take action menu so it stays compact on a phone, with the classified cause still visible on the card. A successful image update now also counts as the last known-good marker in rollback readiness, and the docs gain screenshots of the readiness dialog, gate states, dossier section, and settings. * fix: harden log format strings and the env existence path check Log calls that interpolated the stack name into the console format string now use constant format strings with placeholder arguments, and envExists validates path containment inline at its filesystem access, matching the established patterns used elsewhere in the same files. * test: adapt deploy modal success specs to the post-deploy health gate The deploy feedback modal now withholds its success verdict while the health gate observes the new containers, showing "Verifying health" until the gate passes. The two success-path E2E tests waited for "Succeeded" within the gate's 90s default window and timed out. Shorten the observation window to the 15s minimum for these tests via the settings API, assert the verify-then-succeed sequence the modal actually renders, and restore the default window afterward so the test value does not leak into later runs. * fix: serialize health gate polling and harden gate observation Address race conditions in the post-update health gate found in review. Backend: the gate poller used setInterval, so a Docker observe slower than the 5s tick could overlap the next poll and corrupt the restart and missing-container accounting, and a wedged socket could leave a poll pending forever. Polling is now single-flight: each cycle self-schedules the next only after it settles, and the observe is bounded by an 8s timeout so a hung probe counts as a poll error and resolves the gate unknown after three in a row. Frontend: the gate poller could overlap requests, letting a slow earlier "observing" response overwrite an already-applied terminal verdict. It is now single-flight with a terminal latch, so a late response can never roll the UI back from passed or failed. Also reject a non-digit nodeId on the snapshot coverage route instead of letting parseInt coerce it, document that turning off the deploy progress panel opts out of the live gate UI while the gate still runs server-side, and add gate-coverage tests for the webhook, git source, and auto-update apply paths plus the new single-flight, observe-timeout, and recovery cases. |
||
|
Anso
|
710647a44f |
feat(snapshots): preserve stack dossiers with fleet snapshots (#1339)
* feat(snapshots): preserve stack dossiers with fleet snapshots Fleet snapshots can now optionally capture each stack's Dossier notes alongside its compose and .env files, so a recovery restores the operational knowledge around a stack, not just its configuration. - Opt-in global setting "snapshot_documentation" (default off), toggled from the renamed Fleet settings section. - Capture reads local dossiers from the database and remote dossiers over the Distributed API proxy; only stacks with notes are recorded, and secret values are never included. - Captured notes are stored encrypted at rest in a new fleet_snapshots column and surfaced in the snapshot detail view behind a badge. - Cloud and downloaded archives gain a documentation.json (archive_version 2). - Restore stays conservative: dossier notes are written back only when the operator explicitly opts in, on both single-stack and restore-all paths. - Existing snapshots and archives remain valid; behavior is unchanged when the setting is off. * fix(snapshots): harden dossier-notes restore against bad input and partial failures Address review findings on the documentation-snapshots restore path: - Parse `restoreNotes` strictly (=== true) on single-stack restore, matching restore-all, so a stray non-boolean can never opt in to overwriting notes. - Guard findSnapshotDossier: require an array of stacks and real dossier content, so a malformed or all-blank entry can't clobber current notes. - Make the dossier-notes write non-fatal relative to the file restore: a notes failure (e.g. a remote dossier PUT) is caught, reported via `notesError`, and no longer 500s the single restore or fails the stack in restore-all once the files are already written. - Surface the partial outcome in the UI: a warning toast on single restore, a summary note on restore-all, and gate the "Documentation captured" badge and restore-all notes control on captured stacks while rendering capture warnings. Adds tests for strict parsing, malformed/blank blobs, remote notes restore (success + non-fatal failure, single and bulk), and scheduled capture-on. * fix(snapshots): drop unused binding in restore-all remote notes test The restore-all remote notes test destructured a node id it never uses (restore-all is driven by snapshot id alone), tripping no-unused-vars and failing the lint step. Bind only the snapshot id. |
||
|
Anso
|
af4083175c |
feat(fleet): add read-only dependency map tab (#1324)
* feat(fleet): add read-only dependency map tab Add a fleet-wide Dependencies tab to Fleet view that maps how stacks, services, networks, volumes, and ports relate, with flags for missing dependencies, port conflicts, orphaned resources, and cross-stack shared resources. Read-only; filterable by stack, node, and flag; collapsed by default with a list-view fallback at scale. The graph is derived at request time from Docker and compose metadata, so no new table or persisted state is introduced. A per-node graph endpoint feeds a hub aggregation endpoint that fans out across the fleet and degrades gracefully, surfacing unreachable or unparseable nodes inline while the rest of the map still renders. * fix(fleet): harden dependency map flag detection and remote merge Address review findings on the dependency map: - Port-conflict detection now does pairwise host-scope overlap, so an unrelated bind on the same port and protocol but a different specific host IP is no longer flagged, and the flag lands on the exact scoped port node. - A running service's depends_on target is only considered satisfied when it is actually running, so a crashed (exited) dependency is surfaced while a deliberately stopped stack stays quiet. - Declared external networks and volumes are reported missing when they do not exist on the host instead of being assumed present. - The hub deep-validates each remote node-graph payload before merging, so a reachable-but-malformed remote degrades to a single node error rather than failing the whole fleet map, and the validation failure is logged. - Searching or filtering on a network, volume, or port now also reveals the services that claim it and their stacks. |
||
|
Anso
|
865d792874 |
feat(pricing): collapse to two tiers (#1309)
* feat(pricing): collapse to two tiers (Community + Admiral) Collapse Sencho's pricing from three tiers (Community / Skipper / Admiral) to two: a generous free Community tier and a single paid Admiral tier. The Skipper tier is removed. Now free in Community: auto-heal, auto-update, scheduled operations, webhooks, notification routing, Fleet Actions and bulk operations, SSO preset providers (Google / GitHub / Okta), unlimited users with admin and viewer roles, and deploy safety (atomic deploys, auto-rollback, and one-click rollback). Admiral (paid) is focused on running and governing a fleet: blueprints, Fleet Secrets, deploy enforcement, vulnerability report export, audit log, host console, private registries, mesh networking, node cordon, managed cloud backup, LDAP / Active Directory SSO, and the advanced RBAC roles (deployer, node-admin, auditor) with per-resource scoped assignments. Internally the license variant distinction is removed so tier is binary (community / paid). License validation still verifies the Lemon Squeezy store and product before granting paid status. Docs and the contributor guide are updated to the two-tier model. * docs(pricing): correct licensing page to two-tier pricing and tidy stale tier wording The licensing docs page kept the old Admiral pricing plus a Founder Lifetime column and an Enterprise paragraph after the two-tier collapse. Update it to $12/month or $99/year, drop the lifetime and Enterprise content, and link to the pricing page for current pricing. Also fix stale "Skipper" wording in CLA.md, SUPPORT.md, one test title, and three test comments. Historical CHANGELOG entries and the retired-Skipper license-guard test are intentionally left as-is. * docs: align licensing and SSO pages with the two-tier model Correct the SSO overview so the Google, GitHub, and Okta presets read as available on every tier, matching the provider table; only LDAP and Active Directory require Sencho Admiral. Remove the lifetime-plan references from the licensing, settings, and troubleshooting pages so they reflect subscription-only Admiral pricing. * fix(rbac): omit scoped permissions from /me on the Community tier Scoped role assignments only take effect on the paid tier, but GET /api/permissions/me returned them unconditionally, so a downgraded instance with leftover assignments rendered per-resource affordances the API then rejected with 403. The endpoint now mirrors the permission middleware and includes scoped permissions only on the paid tier. Adds a regression test covering the downgrade case. * docs: use custom-pricing wording on the contact page The two-tier model has no Enterprise tier; reword the contact page's enterprise pricing/deals to custom pricing/deals so it does not imply a tier that no longer exists. |
||
|
Anso
|
42fc8048fe |
fix(metrics): host memory usage excludes reclaimable page cache (#1297)
* fix(metrics): host memory usage excludes reclaimable page cache Host RAM was computed as mem.used / mem.total via systeminformation, but mem.used counts reclaimable buffers/cache as used, so a busy Linux host read ~99%. Switch the dashboard stats, the fleet node self-report, and the host-RAM alert threshold to mem.active / mem.total (cache-excluded), and report used: mem.active and free: mem.available so the byte readout stays consistent with the percentage. Add regression tests for a cache-heavy host (no false alert) and a genuinely busy host (alert still fires). * test(metrics): assert system stats memory excludes reclaimable cache The cached /api/system/stats test mocked si.mem() without active/available, so after the route switched to the cache-excluded fields it produced a NaN percentage that the shape-only assertions did not catch. Fill the mock with a realistic shape and assert the route reports the active working set, not the cache-inclusive used/free. |
||
|
Anso
|
e60c1c0525 |
feat(fleet-snapshots): add restore-all, per-file download, and scrollable preview (#1276)
Three improvements to the Fleet Snapshots detail view, matching the admin-only access of the existing per-stack restore: - Restore all: a control in the snapshot header restores every captured stack across the fleet in one action, with an optional "redeploy all after restore" checkbox. Each stack is restored independently, so a failure on one (removed node, offline remote, blocked deploy) is reported per stack while the rest still proceed. Backed by POST /api/fleet/snapshots/:id/restore-all. - Per-file download: each compose or .env file in a snapshot can be saved to disk individually from its row. - Scrollable preview: the inline file preview is now a bounded, scrollable panel, so a long compose file can be read in full instead of being clipped. Remote restore and redeploy failures now carry the remote node's status and reason, so a per-stack failure in Restore all is actionable. |
||
|
Anso
|
c11a550b6a |
fix(fleet-snapshots): gate reads on admin role and encrypt content at rest (#1273)
* fix(fleet-snapshots): gate reads on admin role and encrypt content at rest Fleet snapshots capture every node's compose.yaml and .env, so the data is as sensitive as the live stacks. This hardens access and reliability across the snapshot pipeline. - Restrict snapshot reads to administrators. GET /api/fleet/snapshots and /:id now require the admin role, matching create, restore, and delete; the Fleet "Snapshots" tab and its panel render only for admins. Previously any authenticated user could enumerate snapshots and read every node's .env. - Encrypt snapshot file contents at rest with the instance key. Restore and cloud-archive paths decrypt on read, so cloud archives stay portable and a database copy no longer exposes stack secrets in plaintext. Rows written before this change still read back as plaintext. - Surface partial captures. A stack whose compose file cannot be read or fetched, or a file over the 1 MB capture cap, is recorded as a warning and shown on the snapshot instead of being silently dropped, so a snapshot is never mistaken for complete. Remote .env read errors are now distinguished from a genuinely absent .env. Adds route-authz, capture-warning, and encryption round-trip tests; updates the Fleet-Wide Backups feature docs. * fix(fleet-snapshots): gate cloud snapshot reads on admin role The cloud snapshot read routes were guarded by provider/license only, not by role, while their write counterparts (upload, delete) already required admin and the Cloud Backup settings surface is admin-only. Because a downloaded archive contains plaintext compose and .env files, a non-admin could list and download cloud snapshots and read every node's secrets, the same exposure the local snapshot reads were just closed against. - Require admin on GET /api/cloud-backup/snapshots, /status/:id, and /object/:keyB64/download, matching the local snapshot reads and the admin-only Cloud Backup settings section. - When capturing a remote node, treat a 200 response carrying X-Env-Exists: false as a stack with no .env (matching the local ENOENT path) instead of storing an empty .env that restore would later write back. Adds non-admin authorization tests for the cloud read routes and a remote absent-.env capture test. |
||
|
Anso
|
0953025036 |
fix(fleet): gate node update actions to admins and harden update tracking (#1272)
* fix(fleet): gate node update actions to admins and harden update tracking Node update affordances now render only for admins, matching the admin-only routes behind them. Previously a non-admin could open the Fleet view and see the per-node Update button, Update all, retry, dismiss, and Recheck controls, then get a 403 on click. Those controls are now hidden for non-admins, who still see read-only update status. Both update-status clear routes (per-node and bulk) now require admin, and the bulk recheck throttles its forced "latest published version" lookup so a caller cannot loop it to hammer the upstream registries; the response reports whether the refresh actually ran so the UI can surface a "checked recently" note. Completion detection no longer reports a node as Updated when it merely blips offline and returns on the same version with an unchanged process start time. That case stays in progress and is decided by the existing early-fail and timeout heuristics, so a momentary network glitch is not mistaken for a successful update. Failed and timed-out updates now emit an operator-visible warning, and a periodic safety-net sweep bounds in-flight trackers when no client is polling for status. * fix(fleet): harden update completion and recheck failure handling Refinements from review of the node self-update hardening: - Completion signal 1 now requires a valid version, not merely a different one. A node whose /api/meta momentarily omits or mangles its version (online, same process) reported version=null, which compared unequal to the previous version and falsely marked the update completed. It now stays in progress and is decided by the early-fail/timeout heuristics. - Terminal resolution is atomic: it re-reads the live tracker and transitions only if it is still in flight with the same start time, so two concurrent status polls cannot both warn or clobber each other's transition. - The operator warning for a failed or timed-out update now redacts secret-shaped text (bearer/basic/token/password, credentialed URLs) from the underlying error before logging, in addition to stripping control characters. - The Recheck button now surfaces an error toast when the request throws (network or auth failure), matching the existing non-ok-response path instead of only logging to the console. |
||
|
Anso
|
7e0cffa376 |
fix(fleet-actions): stop-by-label works on Community remote nodes (#1270)
* fix(fleet-actions): stop-by-label works on Community remote nodes Fleet-stop's remote leg fanned out to POST /api/labels/:id/action, which is gated to Skipper/Admiral, so on a Community fleet the control node stopped its own stacks but every remote node returned 403. Fleet-stop itself is admin-only and available on every license, so the remote leg contradicted the feature's own gate. Extract the label-match plus bulk-stop logic into a shared runLocalLabelStop helper and add an admin-only, every-license POST /api/fleet-actions/labels/local-stop receiver. The control now fans out to that receiver, so remote stacks stop on every tier. Each node runs under its own per-node bulk lock, so a fleet-stop and a per-label action still serialize cleanly instead of double-stopping containers. Also degrade the control's own leg per-node instead of failing the whole fan-out when its filesystem read throws, and gate fleet-stop and fleet-prune diagnostics behind developer_mode. Tests: local-stop auth, tier, validation, and behavior; a remote-leg routing guard that asserts the fan-out targets local-stop and never the paid route; local-leg graceful degradation; and the three Fleet Action card UIs. * fix(fleet-actions): honor the remote stop receiver's matched flag The control reached the remote leg only because its own mirror had the label, then hardcoded matched:true and trusted results without guarding its shape. A mirror-skewed control (mirror has the label, remote does not) then showed a remote mismatch as "matched, 0 stacks" instead of "no matching label", and a malformed 200 body could flow a non-array into the per-stack renderers. Honor the remote's own matched flag and coerce results to an array when the body is malformed. Add regression tests for the matched:false skew case and the non-array results case. |
||
|
Anso
|
d8f73f8203 |
feat(fleet): show stack-label filtering in Fleet View on every tier (#1268)
* feat(fleet): show stack-label filtering in Fleet View on every tier Stack labels and their assignments are a Community feature: the per-node label reads are available to any authenticated user and are already used in the per-node Stacks view. Fleet View, however, only fetched the fleet-wide label palette and per-stack chips when the instance was on a paid tier, so a Community user who had labelled their stacks saw no label dots on node cards and no Tags filter in the Overview toolbar. Drop the paid gate on the fleet label fetch so the palette, the per-stack chips, and the Tags filter render for everyone who has labels. Node-level tag aggregation (used for topology grouping) stays paid and is unchanged. * fix(fleet): surface update-status failures and soften the reconnect timeout Three reliability fixes in the Fleet View update path: - The fleet update-status poll swallowed fetch errors in an empty catch, so a failing poll left a silently stale table with no breadcrumb. It now logs the failure (both thrown errors and non-ok HTTP responses) without toasting on every tick, and keeps the last-known statuses. - The local-update reconnecting overlay declared "Update timed out" after five minutes, which falsely reported failure when a large image pull simply ran longer than the reconnect window. It now shows a non-failure "Taking longer than expected" state with a "Reload to check" action, and the timeout is a named constant that mirrors the backend update timeout. - The fleet overview fan-out already logged a node that failed to report; the update-status and update-all fan-outs now log the rejected node and reason too instead of discarding it. * test(fleet): backfill Fleet View hook, component, and update-tracker coverage Adds unit and component coverage for the previously untested Fleet View surface: all six Overview hooks (overview, update-status, polling cadence, preferences, fleet labels, node labels), the NodeCard, OverviewTab, OverviewToolbar, NodeUpdatesSheet, UpdateStatusBadge, and ReconnectingOverlay components, and the FleetUpdateTrackerService state transitions. * test(fleet): assert update-status poll preserves last-known statuses on failure Adds an explicit case that seeds statuses from a successful poll, then fails the next poll, and verifies the table keeps the seeded statuses (and logs without toasting) rather than relying on the implementation implicitly. |
||
|
Anso
|
d03d97d964 |
fix(nodes): close capability-gating gaps in node compatibility (#1261)
* fix(nodes): close capability-gating gaps in node compatibility Vulnerability scanning is now gated correctly on whether the active node advertises support for it: - A node without the Trivy binary stops advertising the scanning capability. Previously the capability was toggled only on a state change, so a node that booted without Trivy kept advertising scanning it could not perform. - The control node's own capability list now reflects features disabled at runtime, matching what it advertises to peers. - The scan history surface shows a clear "not available on this node" card, with its header actions hidden, instead of attempting a request that fails. A node's version and capability metadata now refreshes immediately after a connection test or a completed update, rather than waiting out the cache. Capability gates fail closed to the unavailable card when a node's metadata request errors, instead of staying open until the next fetch. Adds a test that fails if the frontend and backend capability lists drift, plus coverage for the metadata error path, the runtime-disabled local meta, the scanning capability sync, and the metadata cache invalidation paths. * fix(nodes): refresh node metadata client-side after a connection test A connection test dropped the server-side metadata cache, but the dashboard kept its own cached copy until the client TTL expired, so version and capability gates could stay stale in the browser. The test now forces a client-side metadata refresh for that node, so the version pill and gates reflect the node's current state immediately. Also strips any URL userinfo before logging the metadata fetch target, and makes the scanning-capability detection test deterministically exercise the no-binary disable path rather than depending on whether the runner has Trivy. |
||
|
Anso
|
a51547a158 |
fix(monitor): decouple janitor disk-usage check from 30s cycle (F-6) (#1164)
* fix(monitor): decouple janitor disk-usage check from 30s cycle (F-6) `docker system df` (called by the MonitorService janitor check) can take 30+ seconds on Docker Desktop with many volumes. Running it on the 30s evaluate cycle compounded with the per-container stats fan-out and pushed the cycle to 140s+, blocking subsequent monitoring work. This change: - Moves the janitor disk-usage check into its own 15-minute cycle with a tight 8s timeout. A circuit breaker opens after 3 consecutive timeouts (60-minute cooldown) so a sick daemon stops pinning Dockerode sockets every tick. The first janitor tick is deferred 45 seconds past boot to avoid head-of-line collision with the initial monitor cycle's stats fan-out. - Adds a paired 8s `withTimeout` wrap to the admin prune-estimate routes (`/api/system/prune/estimate` and the dry-run path of `/api/system/prune/system`) so a slow df does not hang the admin tab. Both routes respond 503 with code `docker_df_slow` on timeout. - Factors `withTimeout` and `TimeoutError` into `utils/withTimeout.ts` so the route layer does not have to import from a service module. - Adds 10 unit tests covering the decoupling guardrail, breaker open/close, cooldown, threshold gate, the 100 MB reclaimable floor, re-entrancy, recovery logging, non-timeout error handling, and the full timer-cleanup contract of `stop()`. - Adds 4 integration tests for the prune routes covering the 503 timeout response, the success path, and the non-timeout 5xx path. * fix(fleet,monitor): extend F-6 timeout to fleet prune routes; close breaker-recovery log gap Codex audit findings on PR #1164: Major. The fleet routes that fan out prune-estimate work on local nodes (`POST /api/fleet/labels/fleet-prune` dry-run path and `POST /api/fleet/prune/estimate`) called `estimateSystemReclaim` without a timeout, so a slow local Docker daemon could still hang the fleet admin tab even though the system-maintenance routes were already bounded. Wrap both call sites with the shared `withTimeout(..., 8s)` and surface a "Docker daemon is busy" message via the per-target and per-node error channels the routes already used for other failures. The destructive (non-dry-run) prune path stays unwrapped because it calls `pruneSystem` / `pruneManagedOnly`, not `df`. Minor. The janitor circuit breaker zeroed `janitorConsecutiveTimeouts` when it opened, so a successful call after a full breaker-open cooldown slipped past the `if (counter > 0)` recovery-log branch and never emitted `[Monitor] Janitor disk-usage check recovered`. The operator observability signal was missing exactly when it mattered most. Extend the predicate to also trip on `janitorBreakerUntil > 0` (which stays set to its past timestamp after cooldown until the next success clears it), so recovery logs symmetrically for both partial-failure and post-breaker recovery paths. Added a dedicated test. Three new integration tests cover the fleet routes (timeout, success, and the estimate endpoint's per-node unreachable shape). |
||
|
Anso
|
519a59ed2e |
feat(fleet): open Fleet Actions tab to Community (admin-only) (#1153)
* feat(fleet): open Fleet Actions tab to Community (admin-only) Removes the requirePaid guard from the five Fleet Actions endpoints (fleet-stop, fleet-prune, match-preview, prune/estimate, bulk-assign) and drops the matching isPaid parent gate on FleetActionsTab so Community admins can run fleet-wide bulk operations. requireAdmin stays on every endpoint; operator and viewer roles still 403 on apply. Tests flipped from "403 PAID_REQUIRED on community" to positive "reachable on community + admin" assertions. Docs (fleet-actions, fleet-view, licensing, overview, stack-labels) rewritten to state the admin-role requirement once and drop the prior Skipper framing. * fix(fleet): apply audit findings from PR #1153 review - stack-labels.mdx: fix the page intro that still framed fleet label actions as "Operators on a Skipper or Admiral license". The cards are now Community + admin, so the intro reads "Admins also get a pair of fleet-wide actions". - Collapse redundant role-rule statements on the two affected pages. fleet-actions.mdx now states the admin gate once in the lead-in Note and again only in the troubleshooting accordion (the Prerequisites row was duplicative). stack-labels.mdx trims the "Limits and rules" bullet to the value-add half (label authoring is open to every role) and drops the Fleet Actions repetition. - Strip now-no-op mockTier('paid') calls from non-tier tests across the three fleet test files, plus the test-wide default in the fleet-action-card-endpoints beforeEach. Those mocks were misleading after the routes stopped consulting tier; if a future change re-adds requirePaid the tests will fail loudly instead of silently passing. |
||
|
Anso
|
2f2401df68 |
fix(fleet): route remaining fleet dispatches through getProxyTarget for pilot-agent nodes (#1152)
* fix(fleet): route remaining fleet dispatches through getProxyTarget for pilot-agent nodes PR #1123 migrated POST /api/fleet/nodes/:id/update to use NodeRegistry.getProxyTarget so pilot-agent rows (no api_url / api_token) participate in remote update via the tunnel loopback. The same bug shape lived on at ten sibling fleet-dispatch sites: each read node.api_url and node.api_token directly, returning "Remote node not configured." against pilots, or silently filtered pilot rows out of a fan-out loop. Migrate every remaining fleet-wide remote dispatch to the same pattern: - routes/fleet.ts: fleet-stop, fleet-prune, prune/estimate, snapshot restore (4 sites) -> getProxyTarget + mode-aware error copy + conditional Authorization header - routes/imageUpdates.ts: fleet status + fleet refresh (2 sites) -> swap n.api_url filter for getProxyTarget != null and use target.apiUrl, so pilot rows appear in the aggregated image-updates view instead of being silently excluded - utils/snapshot-capture.ts: captureRemoteNodeFiles (1 site) -> same pattern; CaptureNode interface gains required mode field so the thrown error message picks the pilot-tunnel copy automatically - services/SecretsService.ts: resolveEnvFileRemote, readEnvRemote, writeEnvRemote (3 sites) -> same pattern; thrown errors now use the shared formatNoTargetError helper instead of leaking api_url/api_token field names Extract the previously-private noTargetMessage helper from fleet.ts into utils/remoteTarget.ts as formatNoTargetError so SecretsService, snapshot-capture, and the fleet routes share one copy of the mode-aware error string. Add 10 regression tests in fleet-pilot-dispatch-parity.test.ts covering each migrated route + the snapshot-capture utility: dispatch through the loopback target with no Authorization header for pilots, and a mode-aware error when the tunnel is disconnected. FleetSyncService (4 additional sites) carries an api_url-anchored targetIdentity in the wire protocol; pilot support there needs a protocol-level identity decision and stays as a separate follow-up. * fix(fleet): throw tunnel-disconnected error from resolveEnvFileRemote Codex audit flagged that resolveEnvFileRemote returned null when getProxyTarget was null. That predates the parity migration but the migration was the right place to fix it: the null flowed through readExistingEnv into previewPushDiff / executePush as "env file not found", which is wrong (the env exists, the node is unreachable). Throwing formatNoTargetError(node) here lets the existing catch arms in previewPushDiff (lines 450-453) surface reachable=false with the tunnel-disconnected message on the right axis, and executePush picks up the same shape via its outer catch. Also drop overstated coverage claims from the parity test header (snapshot restore + SecretsService were never actually exercised in this file, only structurally identical via tsc), and fix two describe labels that read /api/labels/* instead of the mounted /api/fleet/labels/*. |
||
|
Anso
|
60f893a81f |
feat(fleet): move bulk Remote OTA updates to Community tier (#1151)
Drops `requirePaid` from `POST /api/fleet/update-all` so Community admins can dispatch bulk node updates. Per-node OTA was already Community- reachable (admin-only); this completes the move so the full Remote OTA surface ships at Community. Frontend mirrors the backend: removes `canBulkUpdate` from NodeUpdatesSheet so the "Update all (N)" affordance is purely data- driven on `updatableRemoteCount > 0`. Docs realigned to drop fence-spec and Skipper-only phrasing on the Update all bulk action: - features/licensing.mdx: Community line now lists Remote OTA (per-node and Update all); Skipper Fleet Actions parenthetical drops "bulk update all" - features/remote-updates.mdx: Note rewritten to role-only requirement - features/fleet-view.mdx: Update all (n) bullet drops the tier clause - features/overview.mdx: Fleet View and Remote updates blurbs drop the Skipper/Admiral fences - operations/upgrade.mdx: Note rephrased without naming tiers Test coverage: - fleet.test.ts: tier-gating spec flipped to assert Community access - fleet-pilot-update.test.ts: bulk-OTA dispatch suite now spies tier to Community so it doubles as a regression guard |
||
|
Anso
|
6f301e005a |
feat(fleet): redesign Fleet Action cards to the System Sheet recipe (#1137)
Lift <FleetActionCard> primitive from audit §20 / DESIGN §9.12. Migrate the three v1 cards (stop-by-label, bulk-label-assign, prune-fleet-wide) to consume it: cyan rail on every card, action class as a chip, blast radius as a live readout in the toolbar, preview section replaces the warning banner, footer carries reversibility plus freshness. Drop the per-card tone rail, the icon prop, and cards/tone.ts. Add /fleet/labels/match-preview and /fleet/prune/estimate for the live blast readouts (chrome falls back to "preview unavailable" if either 404s). Add dryRun: true to the existing fleet-stop and fleet-prune endpoints so the Dry run button rehearses the full code path (locks, per-node fan-out, remote propagation) without firing the destructive leaf call. Result flows into the existing ResultsList. Extend <SheetSection> with optional meta. Add --action-transformative semantic token. |
||
|
Anso
|
0b50c88eb3 |
fix(fleet): route remote update trigger through getProxyTarget (#1123)
POST /api/fleet/nodes/:id/update and POST /api/fleet/update-all read node.api_url + node.api_token directly, so a pilot-agent row (which carries neither) returned "Remote node not configured." and was filtered out of bulk update. Both routes now resolve the target via NodeRegistry.getProxyTarget(), which returns the loopback URL for an active pilot tunnel and the configured api_url for proxy-mode remotes. fetchMetaForNode replaces the direct fetchRemoteMeta call for the self-update capability check. self-update is removed from PILOT_DISABLED_CAPABILITIES so a Compose-deployed pilot can advertise it; the host-console filter stays. SelfUpdateService still gates the local-end capability on the container actually carrying docker-compose labels, so a docker-run pilot self-disables and the Fleet UI sees a clean 503 instead of an ambiguous failure. Tests: new fleet-pilot-update covers the four single-node branches (success via loopback, null target, no self-update capability, meta offline) and two update-all cases (mixed-fleet dispatch, all-targets- null skip). capability-registry-pilot rewired to assert the new filter set. |