* fix(blueprints): fail closed on marker ownership for apply and withdraw
Require a matching .blueprint.json under the stack lock, persist required_blueprint_id on deletion intents, remove the legacy remote apply fallback, and protect the marker in the file explorer.
* fix(blueprints): add CodeQL path barriers on ownership probes
Use the canonical resolve-and-startsWith sanitizer inline at the marker and stack-directory fs sinks so js/path-injection clears.
* fix(blueprints): block delete on failed withdraw and defer marker write
Refuse Blueprint DELETE when pre-delete withdraw does not complete, and write .blueprint.json only after a successful deploy so failed applies cannot orphan stacks or claim an unapplied revision.
* test(blueprints): align lock-order assert with deferred marker write
Update the per-stack lock ordering expectations to compose, cleanup, deploy, then marker after the partial-apply fix.
* fix(deps): bump postcss past GHSA-r28c-9q8g-f849 for npm audit
Raise the Vitest/Vite transitive postcss to 8.5.23 so Backend CI audit --audit-level=high passes.
* feat: add service-scoped stack alert rules
Stack alerts can target one Compose service or all services. Breach timers
are per container and cooldowns are per service so a healthy sibling no
longer clears another container's timer or silences a different service.
* fix: gate remote scoped alert creates without losing the body
Remote hops skip JSON parsing so the proxy stream stays pipeable, which
left service_name invisible to the capability gate. Buffer POST /alerts
bodies for inspection, fail closed when the remote lacks the capability,
and rewrite the buffered bytes on forward. Restore alert-panel alt text
to match the unchanged screenshot.
* fix: bound remote alert body buffer and reject encoded JSON
Cap proxied POST /alerts buffering at the local 100KB JSON limit with
structured 413 cleanup, reject non-identity Content-Encoding with 415 so
compressed scoped bodies cannot bypass the mixed-version gate, and cover
oversized, chunked, and gzip regressions.
* fix: harden service-scoped alert delete, cooldown, and proxy gates
Reject non-digit alert ids, dual-write last_fired_at for rollback safety,
gate cooldown on persisted notification history, fail-fast oversized proxy
bodies with 413, and clarify Not in compose UI semantics.
* test: expect dispatchAlert persisted result in crash-safety cases
Update notification-routing assertions for the new { persisted } return
shape so CI matches the cooldown-gating contract.
Main already clears stack-scoped role_assignments through
DeployedStackDeletionService (used by local blueprint withdraw and
stack DELETE). Add call-path regressions for successful cleanup,
ENOENT-as-absent, non-ENOENT filesystem failure, and db_failed when
RBAC cleanup throws, plus remote hub isolation.
createStack scaffolds compose.yaml; Blueprint was writing docker-compose.yml, so Compose discovery ran the nginx boilerplate. Align Blueprint writes with the canonical filename, clear alternate root Compose siblings on local/modern apply, and cover the regression paths.
* feat: guide missing external network creation during deploy
Detect missing external networks before Compose runs, prompt or auto-create safe bridge networks, and keep unsupported declarations blocked with trusted deploy provenance.
* test: align deploy context and settings fixtures with missing-network gate
Update caller spies, EffResource expectations, StacksSection save keys, and git-source spy cleanup so CI matches the new deployStack context and auto-create setting.
* fix: drop unused renderError binding in missing-network resolver
Satisfies no-unused-vars so backend ESLint CI passes; callers already key only on model presence.
* fix: use HTTP-safe clipboard helper in missing-network dialog
navigator.clipboard fails on plain HTTP LAN hosts; route copy actions through copyToClipboard so Docker and Compose copy buttons work on self-hosted instances.
* fix: simplify missing-network dialog actions and copy label
Drop the Compose snippet escape hatch, move secondary actions under More, and rename the terminal copy action to Copy create command so the footer is a clear Cancel / Create decision.
* fix(notifications): neutralize satellite-local node names in alert bodies
Fleet-aggregated alerts embedded each instance seed name (often Local)
while the hub badge already named the remote. Drop identity prefixes and
use type-aware local wording so attribution stays on the badge.
* fix(docs): correct image-update default check cadence
Operator docs still said six-hour polling; the seeded default is two
hours in interval mode, and the cadence is configurable or cron-based.
* test(notifications): assert hub stamps roster name on neutral remote bodies
Cover the fan-in path that attaches hub roster identity while leaving the
satellite message body unchanged.
* fix: harden deploy/update concurrency and node-targeting safety
Release stabilization for deploy/update operational safety.
Per-stack operation locking is now global. Background lifecycle paths
(scheduler auto stop/down/start/backup/update, webhook execute, Git source
auto-deploy, image auto-update, label bulk actions, fleet snapshot redeploy,
and mesh redeploy) acquire the per-node, per-stack lock through a new
StackOpLockService.runExclusive helper and skip rather than race a manual
deploy/update/rollback/backup on the same stack and node. Skips surface
honestly (a failed scheduled run, a recorded webhook failure, a per-stack
batch result, or a thrown error) instead of a silent no-op.
Update readiness and policy-bypass now run against the node captured when the
dialog opened, not the live active node, so switching nodes while a dialog is
open cannot retarget the update or the bypass retry.
Rollback readiness no longer presents a moving-tag or unpinned image as a ready
image revert. Restoring files does not revert a moving tag, so those stacks
read as partial, and the rollback success message states that the compose and
env files were restored.
* fix: lock blueprint reconcile against manual ops and correct rollback wording
Follow-up to the deploy/update safety hardening, closing two more gaps from a
verification pass.
BlueprintService.deployLocal and withdrawLocal called ComposeService directly,
so blueprint reconciliation could race a manual deploy/update/rollback/backup on
an owned stack. Both now run their compose lifecycle call through
StackOpLockService.runExclusive and skip (recorded as a failed reconcile,
retried on the next cycle) on conflict. The withdraw holds the lock across both
the compose down and the directory delete so neither races a manual operation.
The runtime rollback messages overstated recovery: a rollback restores the
compose and env files and recreates containers, but does not revert an image
behind a moving tag. The auto-rollback deploy-progress output, the recovery
panel and chip, the failure toasts, and the manual rollback route message now
state that the compose and env files were restored, with the matching OpenAPI
example and atomic-deployments doc updated.
* fix: acquire stack lock before blueprint deploy mutates compose and marker files
Local blueprint deploy wrote the compose and marker files and ran the policy
assert before acquiring the per-stack lock; the lock only wrapped the deploy
itself. A reconcile could therefore rewrite an owned stack's files while a
manual deploy/update/rollback/backup was running. The lock now wraps the whole
critical section (create, write compose, write marker, policy assert, deploy),
so on conflict nothing is written and the reconcile records a failed outcome.
Adds a test asserting a deploy under a held lock records failed, writes no
marker file, and leaves the manual lock untouched.
* fix: make remote blueprint apply atomic under the receiving node's stack lock
Remote blueprint deploy wrote the compose and marker files to the target node
via separate HTTP calls and only locked on the final deploy, so the file writes
could race a manual operation on that node. A node's operation lock is
process-local and cannot be held by the hub across HTTP calls, so the locked
create/write/deploy now runs on the receiving node.
The locked critical section is extracted into BlueprintService.applyLocalUnderLock
and exposed via POST /api/blueprints/apply-local. The hub posts the blueprint to
that endpoint in one call; the receiving node runs create + write compose+marker
+ deploy under its own per-stack lock. Older nodes without the route answer 404
and fall back to the legacy multi-call flow. The endpoint is gated by paid tier
and the same per-stack stack:edit and stack:deploy permissions as the
PUT-compose + deploy it bundles, validates the stack name, compose size, and
marker structure, and returns 409 on a lock conflict without writing anything.
Adds tests for the atomic single-call path, the 404 legacy fallback, the 409
lock-conflict mapping, the route validation and permission paths, and the
write-compose-then-marker-then-deploy ordering of the shared locked apply.
* fix(deps): bump undici to 7.28.0 to clear high-severity advisory
The frontend CI npm audit gate (--audit-level=high) failed on a transitive
undici 7.25.0 (a dev-only dependency via jsdom): TLS certificate validation
bypass (GHSA-vmh5-mc38-953g) and cross-user cache information disclosure
(GHSA-pr7r-676h-xcf6). Bumping undici within jsdom's existing ^7.25.0 range to
7.28.0 clears the high-severity advisory and unblocks the frontend job. Lockfile
only; no direct dependency or source change.
Ships the v1 MVP for the Federation tab as placement control, not
placement automation:
- Cordon a node: marks the node unschedulable so the BlueprintReconciler
skips it for new placements only. Existing deployments continue to
drift-check and redeploy on revision changes; cordon never triggers
withdraw or eviction. Toggle on the NodeCard kebab (Admiral, admin
role); Cordoned pill renders for all tiers.
- Pin a blueprint to a node: stores blueprints.pinned_node_id, replacing
the desired set with the pinned node regardless of selector. Pin
overrides cordon by design. Action lives only in the Federation tab;
BlueprintDetail and the deployment table show read-only Pinned
indicators.
Backend: idempotent migrations add nodes.cordoned/cordoned_at/cordoned_reason
and blueprints.pinned_node_id. New routes POST /api/nodes/:id/cordon,
POST /api/nodes/:id/uncordon, PUT /api/blueprints/:id/pin, all gated by
requireAdmiral plus requireAdmin. Audit summaries added so the existing
auditLog middleware records every operator action. deleteNode clears
dangling pins.
Reconciler: pin override evaluated before selector match; cordon filter
applied only to the new-placement branch (deploy/stateReview without an
existing deployment). 11 new Vitest cases cover cordon filter, pin
override, pin-overrides-cordon, missing pin target, pin shrinks
desired set (stateless withdraw + stateful evict_blocked), and pin
clearing on node delete.
Frontend: new FederationTab.tsx with cordoned-nodes summary and
pin-policy table. Federation moved out of the experimental flag into
{isAdmiral && (...)} + AdmiralGate, mirroring the Routing tab pattern.
Secrets stays under experimental.
Tests pass: backend tsc, full Vitest suite (1704 passed), frontend
tsc -b, ESLint (0 errors). Manual verification via the local dev
instance confirmed the tab is hidden at Community, the kebab and pill
render at Admiral, and cordon and pin endpoints round-trip end to end.
Refs cut-line-1.0.md Federation v1 MVP.
* feat(blueprints): add backend foundation for fleet-wide compose templates
Introduces the Blueprint Model: a docker-compose.yml plus a node selector
(labels or explicit IDs) that Sencho reconciles across the fleet. Backend
foundation only; the frontend tab and documentation follow.
Schema (DatabaseService):
- node_labels table for fleet-level orchestration tagging
- blueprints table with compose content, selector, drift_mode, classification
- blueprint_deployments table for per-node materialized state
- New idempotent migrate methods following the existing pattern
Services:
- BlueprintAnalyzer: pure compose-YAML classifier (stateless / stateful /
unknown) with 17 covered cases including named volumes, bind mounts,
external volumes, and tmpfs
- NodeLabelService: label CRUD plus selector matching helper (any/all/ids)
- BlueprintService: local + remote deploy/withdraw orchestration, marker
file management, name-conflict guard, per-(blueprint,node) lock
- BlueprintReconciler: 60-second loop with three-mode drift policy
(observe/suggest/enforce), state-aware guards, and Enforce-downgrade for
volume-destroying drift
Routes (gated requirePaid + requireAdmin on mutations):
- /api/blueprints (CRUD + apply + withdraw + accept + preview + analyze)
- /api/node-labels (CRUD + listAll + listDistinct)
Notifications: four new categories registered in NotificationService for
deploy/failure/drift events.
Bootstrap: reconciler start/stop wired in startup and shutdown.
Tests: 45 new Vitest cases covering selector matching, classifier rules,
state-aware guards, drift-mode branching, and marker parsing. Full backend
suite (1625 tests) passes; tsc clean.
* fix(lint): replace bare Function type in blueprint reconciler tests
Replace 8 occurrences of `as unknown as { computeDecision: Function }`
with a properly typed `ReconcilerWithCompute` alias that mirrors the
real method signature. Export `ReconcileDecision` from
BlueprintReconciler so the test can reference it.
Resolves @typescript-eslint/no-unsafe-function-type errors that were
failing the Backend (Lint) CI step.
Anso