* feat(recovery): capture complete authored Compose project for atomic rollback
Replace the root-compose-only backup slot with staged recovery generations that
record the managed inventory, exact Compose invocation, and prior image identity,
and wire the same engine through deploy, update, manual rollback, and Git apply.
* fix(recovery): satisfy CodeQL path barriers and update-guard mock
Inline resolve+startsWith checks at generation/inventory fs sinks and stub getCurrentStackUpdateRecovery in UpdateGuardService tests.
* fix(recovery): drop unused FileSystemService import in generation store test
* fix(recovery): harden authored-project rollback for upgrade and restore safety
Preserve legacy UUID backup rows, restore Git deploy state with files, make multi-file restore recoverable, evaluate policy on the restored target, and fail closed when Git capture cannot cover an apply.
* fix(recovery): unblock Git apply unit tests and CodeQL pre-restore TOCTOU
Mock recovery capture in git-source-service tests after fail-closed apply capture, and re-resolve live paths immediately before pre-restore snapshot reads.
* fix(recovery): fall back to authored inventory when Git manifesto is missing
First Git apply captures before promote, so a missing managed-project manifesto must not block rollback capture when the live stack already has authored files.
* fix(recovery): make authored-project rollback atomic across Git state
Restore the managed-project manifesto with files, keep nullable Git identity on first-apply captures, persist Git side-state in restore intents for startup reconcile, compensate legacy materialize failures, and refuse directory collisions before mutation.
* fix(recovery): satisfy CodeQL path and TOCTOU barriers on manifesto restore
Add inline resolve barriers for manifesto read/clear sinks and remove the access-then-read race when restoring a generation manifesto snapshot.
* fix(recovery): close third-audit rollback generation blockers
Fail closed on incomplete Git inventory fallbacks, execute captured Compose
invocation during recovery, refuse startup and mutations while restore intents
remain unresolved, propagate legacy stale-delete failures, and add Docker-level
exact prior-image coverage plus regression tests.
* fix(recovery): mark acquired before handoff in prior-image Docker test
Match the production updateStack CAS sequence so the exact prior-image
integration test does not fail handoff from the captured phase.
* fix(recovery): close fourth-audit rollback safety blockers
Evaluate policy against held images, use index-based pre-restore snapshots, hold the shared stack lock across Git apply, replay Mesh and empty captured invocations exactly, restore POSIX modes with fail-closed sensitive permissions, keep case-sensitive paths, and link Git auto-deploy health gates. Add regression coverage for these cases.
* test(recovery): fix mocks for health-gate link and authored compose args
Add linkGateOrRetain to the Git apply recovery mock, and mock authoredComposeArgs so the case-collision inventory test is not masked by a missing getComposeDir stub.
* fix(recovery): close fifth-audit rollback safety blockers
Share git_apply locking for webhook auto-apply, fail closed on malformed recovery service records, refuse mixed-image capture, and require exact probe counts with hold-tag eligibility checks.
* fix(recovery): close sixth-audit rollback safety blockers
Preserve the legacy backup slot during generation capture, encrypt sensitive pre-restore snapshots, revert files on a failed health probe without committing Git, fail closed when an absent-file revert would delete a directory, skip Compose one-offs, route manual and scheduled backup through the current generation, and persist runtime image platform identity.
* fix(recovery): close seventh-audit rollback safety blockers
Fleet snapshot restore and restore-all now capture a recovery generation under the stack lock before any authored file write, including on remote nodes.
* fix(recovery): keep pre-deploy generations during health-gate observe
Link deploy recovery generations to the observing gate so backup cannot replace them mid-observe. Distinguish missing hold tags from probe failures, refuse generation release when services metadata is corrupt, classify mixed-replica and coverage refusals, and toast the backend rollback message.
* fix(recovery): wrap webhook deploy case for eslint
const bindings in an unbraced switch case trip no-case-declarations. Match the pull case block.
Remove the leftover paid-only blockingEnabled switch so enabled
block-on-deploy policies enforce on every tier, matching the
documented every-tier security surface. Existing Community policies
begin blocking immediately with no migration.
When the pre-deploy gate could not scan or evaluate an image (a compose parse
error, a scan failure, an invalid image reference, or an evaluation error), it
pushed a synthetic violation with zero counts and no reason. The block dialog
then showed "0 critical, 0 high" with no explanation and only Close or admin
bypass, so an operator could not tell why the deploy was blocked or what to fix.
The synthetic violation now carries the failure reason in an error field, which
flows through the existing 409 block payload. The block dialog renders that
reason under a "Could not be scanned" label instead of a misleading zero-count
row, and shows a recovery hint pointing at the fix-and-deploy-again path.
* fix(scheduler): reject 6-field cron in Scheduled Operations
Create and edit validation parsed cron with cron-parser, which accepts both
5- and 6-field expressions, while the form, presets, and docs all describe a
5-field cron. Because the scheduler ticks once per minute, a leading seconds
field can never improve precision, so a 6-field expression was silently
accepted but never honored on its stated schedule.
Add a field-count guard on both sides: the API rejects 6-field input at
create and edit with a clear message, and the form surfaces the same error
inline and disables save. Cron nicknames such as @daily still pass. Document
the five-field requirement in the cron reference.
* chore: merge main into scheduled cron validation
* fix: avoid logging policy bypass actor in debug output
Scan-policy deploy gates can now block on a known-exploited CVE (CISA KEV)
and on a fixable Critical/High finding, in addition to an optional severity
threshold. New policies default risk-first (KEV and fixable on, severity off);
existing policies keep their severity-only behavior. CVSS stays captured for
context but is never the sole basis for a block, and a finding whose
exploitability cannot be confirmed is treated as risky rather than safe
(incomplete scan detail fails closed on KEV/fixable inputs).
The decision logic is shared between the pre-deploy gate and the informational
post-scan banner via a pure helper, so the two never disagree. Block messages
and the block dialog now name the conditions an image matched. Backend and
frontend gates move together, the new inputs replicate across the fleet, and a
blocking policy with no active input is rejected on both sides.
* feat(security): make CVE suppressions optionally honored by deploy-block policies
Block-on-deploy policies evaluate the raw scan result, so a CVE an admin
has accepted in CVE Suppressions still blocks the deploy. Add an opt-in,
per-instance toggle ("Honor suppressions in deploy blocks", Settings ->
Security) that, when on, re-derives each image's severity from the
suppression-filtered findings before comparing to the policy threshold. A
deploy that proceeds only because suppressions dropped it below the gate is
recorded in the audit log. Default off, so the strict raw-scan behavior is
unchanged unless an operator enables it.
The setting governs the instance that runs the deploy and is not
fleet-replicated. The gate fails safe: a suppression-read error or an
empty detail set falls back to raw scan severity rather than dropping it.
Also surface a previously swallowed error in the CVE suppressions and
misconfig acknowledgement settings panels so a failed list load shows a
toast instead of an empty list.
* fix(security): gate on raw severity when preflight detail rows are truncated
The suppression-aware deploy gate re-derived image severity from the stored
vulnerability_details rows, assuming any non-empty set was complete. A cached
pre-deploy scan keeps the full aggregate counts but copies only a bounded slice
of detail rows, so recomputing from that slice could drop an unsuppressed
blocking CVE below the threshold and let a deploy through.
Guard the recompute: when the loaded detail rows do not match the scan's total
finding count, gate on the raw scan severity (never drops severity). Suppression
awareness still applies for scans whose details are stored in full, which is the
common case.
* fix(deploy-enforcement): surface scan-policy blocks on update and sidebar deploys
A blocked deploy only opened the policy dialog from the editor deploy
button. The update action and the sidebar context-menu deploy/update
fell through to a generic error toast, so an admin could not review the
violations or bypass the block from those entry points. Route the 409
policy response through a shared handler on all three paths and make the
"Deploy anyway" bypass retry the originating action (deploy or update)
so an update bypass still re-pulls images.
Also:
- Correct the "Block on deploy" policy-editor helper text, which
described post-deploy alerting rather than the pre-flight rejection it
actually performs.
- Dispatch the documented scan_finding warning (policy name and the
offending images) when a scheduled auto-update or auto-start is
blocked, instead of recording an opaque failure.
- Add a standard log line when the gate blocks a deploy, plus
developer-mode diagnostics for the matched policy and per-image
severity decision.
- Fix deploy-enforcement docs: complete the enforced entry-point list,
correct the policy-precedence wording, and remove inaccurate tier and
audit-actor claims.
* fix(deploy-enforcement): surface policy block on rollback and name images in remote auto-update alert
Addresses two gaps found in independent review:
- Rollback is a policy-gated deploy path (it restores the saved files then
re-runs the gate before redeploying), but the frontend treated a blocked
rollback as a generic error toast. Route the 409 through the same handler
as deploy and update so the block dialog opens, and let an admin "Deploy
anyway" retry the rollback with the bypass flag (the rollback route already
honors it).
- The remote auto-update path dispatched its policy-block warning without the
offending image refs, unlike the local scheduler. Append the images so the
alert matches the documented contract on every node.
Also list rollback as an enforced entry point in the docs and clarify that
Git Source enforcement covers both the create-time deploy and a manual
apply-with-deploy.
* fix(stack-activity): per-stack history integrity, attribution, sanitization
Address the Stack Activity audit findings (PR 1 of 2):
- Per-stack history integrity: drop the per-insert 100-row prune in
addNotificationHistory that evicted quieter stacks' history whenever
another stack got chatty. Periodic cleanupOldNotifications now caps
per (node, stack) at 500 rows and per-node unattached system events
at 1000 rows, on top of the existing 30-day retention. Signature
takes an options bag and returns a per-stage summary so MonitorService
can log what actually ran each cycle.
- Actor attribution: thread req.user?.username through every
notifyActionFailure call site and add synthetic actors at service
emit sites (system:autoheal, system:scheduler, system:image-update,
system:docker-events, system:blueprint, system:monitor, system:policy).
The timeline renders system actors as "via <Label>" so an autoheal
redeploy is no longer indistinguishable from a user redeploy.
- Message sanitization: new sanitizeNotificationMessage at
NotificationService.dispatchAlert strips KEY=VALUE pairs whose key
ends in TOKEN/KEY/PASSWORD/SECRET/CREDENTIALS/AUTH, scrubs HTTP basic
auth in URLs and Bearer tokens, collapses COMPOSE_DIR paths, and
truncates to 1000 chars. Applied to the stored history and to every
downstream Discord/Slack/webhook channel. The ImageUpdateService
recovery-path direct DB write also runs through the sanitizer.
- Composite pagination cursor: getStackActivity now accepts a
(timestamp, id) cursor (?before=&beforeId=). The legacy timestamp-only
form silently dropped events when a single compose up emitted many
events sharing one millisecond. Route rejects beforeId without before.
- Frontend hardening: distinct error state with retry button (initial
fetch failure no longer renders as the genuine empty state), strict
positive-integer parsing on cursor params, overrequest-by-1 pagination
so the last page does not leave a dead "Load more" click, runtime
guard on liveEvents merge that validates the level union, per-minute
day-bucket recompute so an open panel does not stay on "Today" past
midnight.
No tier, role, or capability gate touched. Route permission gate
remains stack:read on the named stack.
* fix(stack-activity): sanitizer covers lowercase env vars and per-node compose dir
External review surfaced two leak paths in the message sanitizer:
- The sensitive-key regex was uppercase-only. Compose env names are
conventionally uppercase but lowercase forms (db_password, jwt_secret,
github_token) are valid and do leak through the same Docker and
compose-parse error paths. Make the regex case-insensitive and tighten
it to also catch bare TOKEN= / KEY= / PASSWORD= without a prefix word,
while still leaving BYPASS, COMPASS, and similar non-secret keys alone.
- The compose-dir path collapse only read process.env.COMPOSE_DIR, but
the real resolution chain is node.compose_dir (per-node DB override)
-> process.env.COMPOSE_DIR -> /app/compose. A node with a custom
compose_dir could still leak absolute paths into stored history and
downstream channels. Route both the dispatchAlert call and the
ImageUpdateService recovery-path direct write through
NodeRegistry.getInstance().getComposeDir(localNodeId) so the
collapse covers every resolution outcome.
Tests now assert lowercase keys are redacted and that BYPASS-style
non-secrets stay intact in both cases. notification-routing mock
extended to stub the new getComposeDir call.
* chore(stack-activity): a11y roles, visibility-aware tick, live-disconnect signal
Close three small follow-ups on the per-stack activity timeline:
- A11y: each day-group gets role="list" and each event row gets
role="listitem" so screen readers traverse the timeline as a list
instead of a wall of text. The day-group container also carries an
aria-label naming the bucket.
- Visibility-aware day-bucket tick: the 60s setInterval that re-derives
Today/Yesterday/Earlier now short-circuits when document.hidden, so a
backgrounded panel does not re-render every minute for no visible
effect.
- Live-disconnect signal: useNotifications dispatches a
sencho:notifications-connection custom event on WebSocket open and
close. The timeline listens and, when explicitly disconnected, shows
a one-line "Live updates offline; reconnecting…" hint above the list.
The sidebar ticker already surfaces fleet-wide connection state; this
adds an in-context cue for users who are focused on a single stack.
Stack-name case normalization was considered and rejected: stack names
are case-permissive per the isValidStackName validator, and lowercasing
on read or write would silently rename or hide a user's "MyApp" stack.
* ci(stack-activity): drop unnecessary escape in URL_BASIC_AUTH regex
ESLint no-useless-escape errored on \- inside the character class
[a-zA-Z0-9+.\-] at notificationMessage.ts:14. Move the dash to the
end of the class so it's an unambiguous literal and the escape is no
longer required. Behavior is identical; sanitizer tests still pass.
* revert(stack-activity): drop unvalidated E2E spec from this PR
The spec was committed without ever running against a real Docker
daemon, then failed in CI when it ran for the first time: deploy
returned 200 but no notification appeared on the activity endpoint
within the polling window, suggesting either a deploy-notification
race or a node-id resolution mismatch in the CI environment.
Backend unit tests (route + composite cursor + sanitizer) and
frontend component tests cover the same logic. The E2E spec will
land in a dedicated follow-up once it has been authored against a
working CI environment.
Scan policies on a node without Trivy installed previously fired one
"Pre-deploy scan skipped" warning per deploy, flooding the notification
feed during CI loops. Add a 60-minute per-(node, stack) cooldown so an
operator sees one actionable warning, not one per deploy. The boot log
line and the one-click managed install in Settings > Security are
unchanged; this only reshapes the per-deploy fanout.
Also tighten the vulnerability-scanning entry in /docs/features/overview
to point first-touch users at the one-click install on first use.
* fix: harden deploy enforcement paths
* fix: update Docker toolchain to Go 1.26.3
* fix: repair Dockerfile tr argument split across lines
* fix: bump protobufjs to clear npm audit high-severity advisories
* fix(test): add execFile to child_process mock in compose-images test
* fix: resolve merge conflicts with main
* fix: resolve merge conflicts with main
* fix: resolve merge conflicts with main
* refactor(backend): sanitize user input before logging to close CRLF injection
Adds a small sanitizeForLog helper that strips CR, LF, tab, and ASCII
control characters (0x00-0x1F, 0x7F) from a value before it is embedded
in a console.log/warn/error/debug call. Wraps every call site where a
user-controlled value (req.params, req.body, req.query, or a value
derived from them) flows into a log message.
Closes the bulk of the open CodeQL alerts in this family:
- 96 js/log-injection
- 28 js/tainted-format-string
The helper is in backend/src/utils/safeLog.ts. Routes still pre-validate
input at the request boundary; this is the second line of defense and
gives static analyzers a sanitizer they can trace through. JSON
responses, Docker filter labels, and other non-log call sites are
intentionally left unwrapped.
* refactor(backend): printf-style format strings for tainted-log call sites
CodeQL's js/tainted-format-string rule flags template literals in the first
arg of console.X when any interpolated value is user-controlled, regardless
of whether each value is sanitized inline. The canonical mitigation is to
use a static format string and pass values as positional args.
Converts the 28 flagged template literals to printf-style ("%s") format
strings, with sanitizeForLog applied to each positional arg. Also fills in
the log-injection wraps on 9 sites where a user-controlled value was
missed in the first sweep (agents, fleet, gitSources, imageUpdates,
GitSourceService).
No behavior change at runtime. Node's util.format substitutes %s tokens
identically to template-literal interpolation.
* fix(backend): wrap nodeId/snapshotId in fleet restore debug log
CodeQL flagged the unwrapped numeric args even though they cannot
contain control chars in practice. Apply the sanitizer for taint-flow
recognition.
Introduce a NotificationCategory string-literal union (11 values) and
thread it through dispatchAlert as a required second argument. All
callers (DockerEventService, AutoHealService, ImageUpdateService,
MonitorService, PolicyEnforcement, policyGate, SchedulerService,
imageUpdates route) pass an explicit category at every call site,
giving TypeScript compile-time enforcement that no new emit site can
be added without choosing a category.
DatabaseService gains an idempotent migration that adds a nullable
category TEXT column to notification_history; existing rows keep
category=NULL (displayed as Uncategorized in the UI). The
getNotificationHistory method accepts an optional category filter
that is forwarded from the GET /api/notifications/history route via
a ?category= query param.
NotificationPanel gains a category Select dropdown so users can
filter history by category. The frontend types mirror the backend
union so API responses are type-safe end-to-end.
All 75 test files (1410 tests) updated to the new 4-arg dispatchAlert
signature and passing.
Policies with block_on_deploy=1 now scan every stack image before
docker compose up runs and reject the deploy with HTTP 409 on violation.
The UI opens a dialog listing offending images; admins can override per
deploy with ?ignorePolicy=true, and every bypass is recorded in the
audit log with the originating route, actor, policy, and image list.
When Trivy is not installed on the target node the gate fails open with
a warning notification, so teams are never locked out by tooling state.
Post-deploy and scheduled scans still evaluate matching policies and
dispatch warnings on violations to surface drift on long-running stacks.
Public API additions: policy and suppression CRUD under /api/security,
plus the documented 409 block-response shape on all deploy paths.
Anso