63 Commits

Author	SHA1	Message	Date
Anso	341511a2e0	feat: deliver hub registry credentials to remote Compose targets (#1866) ... * feat: deliver hub registry credentials to remote Compose targets When a hub forwards stack operations to a remote node over confidential transport, discover private image hosts on the target, attach an attested credential envelope, and materialize DOCKER_CONFIG at the Compose seam. Capability-gated with pass-through when delivery is unavailable. * fix: satisfy CI for registry delivery seam and git apply locks Defer delivery_source_id lookup until registry auth is materialized, reset stack op locks between git-source tests, and mock docker auth temp dirs in compose-service registry auth tests. * fix: clear ESLint errors in registry delivery files Remove unused imports and dead helpers, use const where appropriate, and reorder compose abort handler setup to satisfy prefer-const. * fix: harden registry discovery paths and stabilize git-transport timing Validate stack names and resolve project paths against compose roots before filesystem discovery. Widen the git-transport termination race margin in CI. * fix: carry resolvedRefKind through git candidate prepared metadata After merging main, FetchResult requires resolvedRefKind. Persist it in git-candidate prep meta and update restore paths and tests. * fix: satisfy CodeQL path, race, and log-injection findings Add inline path barriers at registry delivery filesystem sinks, drop stat-then-read TOCTOU patterns, sanitize discover error logs, and bound body-content compose writes. * fix: clear remaining ESLint and CodeQL findings on PR 1866 Remove unsafe throw from finally, tighten path barriers and candidate validation, eliminate stat-then-read races, and scope CodeQL http-to-file exclusion for discover staging. * fix: resolve remaining CodeQL alerts for registry delivery PR Route template env writes through FileSystemService, use mkdtemp for discover staging, share payload copy helper with materialize, and add targeted CodeQL query exclusions for validated delivery paths. * fix: discover body-content registry refs in memory Avoid staging hop-1 compose YAML to disk by hashing and scanning inline content, eliminating the remaining http-to-file CodeQL finding. * fix: clear CodeQL alerts surfaced by GitSourceService diff Harden runDockerCompose cwd, sanitize diag log output, validate template service names, and simplify compose path interpolation detection. * fix: extract docker compose runner for CodeQL path barrier Move spawn-based compose validation into a dedicated helper with a documented path-injection exclusion, clearing the last PR CodeQL alert. * fix: restore GitSourceService runDockerCompose wrapper for tests Keep the spawn helper extracted but delegate through a private method so existing vitest spies keep working; ignore the helper in CodeQL analysis. * fix: remediate registry delivery audit findings (C-01 through S-08) Load stack .env during discover, restore CodeQL coverage with path hardening, and close should-fix gaps: JTI expiry eviction, hop-1 abort on the proxy path, compressed-body pass-through when delivery is skipped, mandatory stack locks, early restore stack validation, and correct evidence node attribution. * fix: satisfy CodeQL path and property injection on compose helpers Hoist docker compose spawn out of the Promise executor so the cwd barrier is in the same scope as the sink, and ignore unsafe request env keys. * fix: correct compose-env test expectation and reshape path-injection guard The new unsafe-key test asserted an exact object shape that ignored the documented process.env override layer, failing wherever process.env is non-empty. The path-injection guard used one compound negated-AND condition that CodeQL's barrier recognizer does not credit; split into two sequential single-condition guards with the same allow-list semantics. * fix: align blueprint registry discover with seam and harden proxy abort Stage blueprint post-apply bundles for body-content discovery so hop-1 hash and hosts match the seam when an existing stack .env is present. Restore prior compose.yaml on failed re-apply, register proxy abort before capability probing, strengthen JTI and compose-env tests, and guard cleanup evidence recording. * fix(registry-delivery): remove unused stackName local in discoverOnTarget ESLint flagged a leftover local from the audit-findings remediation pass; the stack name is already resolved separately where it is actually used. * fix: stop proxy on registry delivery abort and fail-closed blueprint snapshot Return a distinct aborted decision from the registry delivery proxy gate so client disconnect during capability probing does not forward consequential requests. Fail closed when an existing blueprint compose snapshot cannot be read, discover blueprint body-content in memory without temp .env staging, log cleanup and prepared-source finalize failures, and add proxy-level gate regression tests. * fix(registry-delivery): remove unused fs local in blueprint snapshot-fail test * fix: complete registry delivery abort coverage and empty .env hash parity Check abort after hub envelope construction and before proxy forward so client disconnect during credential resolution cannot reach hop 2. Include zero-byte stack .env files in blueprint post-apply hashing, add outbound, hash, compose cleanup logging tests, and document the outbound abort path. * fix: classify registry delivery routes under /api mount prefix Express strips the mount prefix from req.path when registryDeliveryMiddleware is installed at app.use('/api', ...). Normalize to /api${req.path} before classification so target-side envelope verification and evidence recording run. Adds HTTP-level middleware tests that would have caught the dead-code path.	2026-08-29 19:07:36 -04:00
Anso	38ee4527b1	feat(git): add GitOps revision store and Direct apply transitions (#1835) ... * feat(git): add GitOps revision store and Direct apply transitions Adds the canonical GitOps revision state model: schema, store, history, approval resolution, Direct apply transitions, and a first-cut read projection. Also widens recovery capture with generation, artifact, and source-acceptance bindings, rejects credentials and query strings in new Git repository URLs, and keeps Blueprint and node-label routes on the hub. * feat(git): add create-from-Git activation and teardown transitions Adds the durable half of create-from-Git: the single transaction that persists the application, resolved commit, generation, checkpoint, and candidate together, and the teardown that tombstones a create which never reached its success boundary. Adds the staging marker and operation-owned cleanup rules that decide what a crashed create is allowed to delete. * feat(git): make create-from-Git crash-safe Wires the create path through the GitOps state model: the staging marker is written before the candidate is built, the activation transaction persists the application, generation, and checkpoint together, and the source row plus accepted pointers now commit as one success boundary. A create that fails before that boundary removes only what it staged, then tombstones itself. The managed-area sweep no longer deletes a directory it cannot prove it owns, counts in-flight creates as claimed, and is awaited at startup so an interrupted create is settled before any mutation service runs. * feat(git): settle interrupted create-from-Git at startup Adds boot-time recovery for creates a previous process left in flight. A create whose project was already committed to disk is finished, including its source row and accepted pointers. Anything earlier has its files removed first and is then tombstoned, so the stack name becomes usable again instead of staying locked by a half-created application. A source row that outlived its application is always preserved. * fix(git): close create-from-Git ownership and reporting gaps Removes two ways a failed create could destroy files it did not own: boot recovery no longer deletes a stack directory the create never recorded making, and the managed-root teardown is gated on having created that root. Cleanup now rejects a path that is not a generation directory, so a malformed row cannot widen the blast radius. A create that fails during materialization now clears its staging marker, which previously stayed behind and made the stack name uncreatable until restart. Generation paths no longer resolve through a module cycle that left them pointing at a directory that never existed. Boot recovery restores the deploy spec and manifest cache a finished create needs, settles each application independently so one bad row cannot strand the others, and distinguishes a missing directory from one it could not read. A create that succeeds now clears its checkpoint, and a failure after the success boundary says the stack was created rather than reporting a bare error. * fix(git): reap managed areas that nothing has ever claimed The boot sweep now distinguishes a missing staging marker from an unreadable one. A missing marker means no stack, no create, and no marker claims the directory, which is the ordinary orphan a crashed stack deletion leaves behind, so it is removed as it always was. An unreadable marker is evidence of a claim that cannot be verified, so the area is preserved. Without this an orphaned managed area survived every boot forever, since a completed create deletes its marker and nothing else would ever claim it. * feat(git): complete the Direct source and target transitions Adds the remaining Direct events the state model owed: an invalid fetch that advances the resolved commit without minting a candidate, a blocked candidate that is visible but cannot apply, dismissal that leaves the running workload alone, material configuration change that invalidates a staged candidate while keeping accepted and applied pointers, the two deploy failure classes, and application and target tombstones. Widens the application update to every mutable column. It previously wrote about half of them, so a transition assigning a pause, suspend, recovery, or intent field would have type-checked, appeared in the history snapshot, and been silently dropped at commit. * feat(git): record Direct Git operations in the revision state Wires fetch, apply, and detach through the state model. A pull records the fetch and, when it produces a candidate, the generation behind it, marking a blocked change plan as a blocker that cannot apply. An apply records the acceptance that binds the generation to the workload, and closes its operation on failure so a throw cannot leave the source reporting work in progress. Detach tombstones the application and its target in the same transaction that removes the source row. Every producer is a no-op for a stack with no live application, so installs whose Git stacks predate this model are untouched until migration runs. * test(git): cover the Direct Git producers end to end Adds a harness that stubs only the git transport and rollback capture, so fetch, candidate materialization, change-plan classification, apply, and detach all run for real and are asserted against the resulting revision state. Covers the seam the transition-level tests could not: that a fetch advances the resolved commit without moving the accepted generation, that the apply binds the exact candidate the fetch recorded and its acceptance proves that generation and no other, that detach tombstones while keeping configured identity as a frozen fact, that a failed fetch closes its operation and leaves every pointer alone, and that a stack with no application produces no writes. * feat(git): bind the deployed generation from the Compose adapter Makes ComposeService the sole producer of deploy events. Every deploy path funnels through deployStack, so recording it there keeps one start and one terminal row per mutation instead of each caller reporting its own. A successful deploy binds the applied generation, a failure records the class and leaves the deployed pointer alone. The class is conservative: once the compose command is handed off we cannot prove the workload was untouched, and claiming it was intact is the more dangerous error. A stack with no live application, or one with nothing applied, has no deploy identity to record and is skipped entirely. * feat(git): promote healthy and last-known-good from health verdicts A stack health run now records which generation it observed, and its verdict drives the revision state. Promotion is narrow by design: healthy and last-known-good move only when the run passed, watched the whole stack, and the generation it watched is still deployed. A stale or service-scoped verdict records history and moves nothing. Last-known-good keeps the artifact expectation only when that expectation belongs to the promoted generation. Otherwise the generation is still good and its executable identity simply is not proven, so the pointer is left null rather than borrowed from another generation. beginStack now takes the deployed generation explicitly rather than reading current state, because a verdict is only meaningful for the generation the run actually observed. Update-path callers pass null until updateStack reports what it deployed. * feat(git): bind the deployed generation on the update path Gives updateStack the same treatment deployStack has: it opens a deploy operation at the point Compose is handed the mutation, closes it on both outcomes, and reports the generation it bound. The orchestrator carries that binding through, and all five update call sites now pass it to the health gate instead of null, so a passed health run after an update promotes healthy and last-known-good exactly as it does after a deploy. The operation is opened at the compose call rather than at entry, so an update that fails during capture or classification records nothing: it never touched the workload, and reporting a deploy failure for it would make the projection claim something that did not happen. * feat(git): track linked stacks and material config changes Linking an existing stack to a Git source now brings it into the revision state: the application starts live with nothing desired or accepted, so the projection asks for a fetch rather than claiming a state it has not observed. Editing a source's material configuration invalidates its staged candidate in the same transaction that writes the row and clears the pending pull. A candidate built from a different repository, ref, or file set can no longer be applied, and clearing pending without invalidating it would leave the model offering an apply the operator cannot produce. Credential-only and policy-only edits change nothing material and leave the candidate alone. * feat(git): retire the application when a stack is deleted Stack deletion now tombstones its GitOps application and targets in the same transaction that commits the deletion. A deleted stack that kept a live application would go on claiming the stack name and block re-creating it through the unique live-application index. The tombstone is driven from the deletion service rather than from inside DatabaseService, so the store keeps its transitions and their history in one place and the two modules do not form an import cycle. * feat(git): retire node targets when a node is removed Deleting a node now tombstones the GitOps targets that lived on it, in the same transaction as the delete and while those rows still exist. Otherwise a part-way failure would leave targets pointing at a node that is gone. Applications stay live: a Direct application still describes a real stack, and a Blueprint one may have targets on other nodes. Both the local and remote node paths go through the same helper, so neither can skip the retirement. * feat(git): add proof-bound recovery transitions A restore moves a target back to an older generation, the one case where a target and its application legitimately disagree about what is current. These transitions decide what may move with it. Pointers move only when the restore is provable: the recovery point named a generation, that generation still exists, and it belongs to this application. The artifact expectation comes from what the recovery point captured, never from what the application expects now, and the acceptance is kept only if it still proves the restored generation rather than borrowing one that authorized a newer one. A last-known-good survives unless the generation behind it is gone or turns out to belong elsewhere, and then the reason is recorded so the projection can say unavailable rather than none. An unproven restore is still recorded as a real operational recovery and moves nothing, because there is no evidence to move pointers to. * fix(git): close GitOps operations that cannot be recorded Recording a transition still never fails the operation it describes, but a rejected terminal event no longer leaves that operation open. A start that never terminates reported work in flight for ever and offered no actions, and because a new fetch refuses to open a second operation, one rejection silently stopped the model tracking that stack until a restart. Startup now reclassifies operations the previous process left open, which the transition already supported but nothing called. An interrupted restore is closed there too, since only the terminal recovery events clear it. The pull records its fetch, generation, and candidate as one transaction, and the apply verifies the candidate was built from the commit being applied. Separately those two allowed an apply to accept a generation whose files were never on disk while the projection reported the older commit as current. Also: the candidate now records its own invocation rather than the one it replaces, changing material configuration is refused while an operation is in flight, a tombstoned target cannot be repopulated by a late health verdict, a deploy handle is not returned when its start was not recorded, the mutation handoff is marked only once Compose is genuinely about to receive it, and a rejected tombstone fails one deletion rather than aborting the boot sweep. * feat(git): record why unprovable evidence was dropped The projection carried a limitations array that only read-time derivation ever wrote to. When a transition cleared a pointer it could not prove, the result was indistinguishable from that pointer never having existed, and in two cases it made the target read healthier: a dropped artifact expectation silently disables the runtime drift check, and an unproven restore left every pointer agreeing with itself so the target reported as synced and healthy. Adds a persisted, fail-closed evidence record on the application and target rows. The transitions that drop an artifact expectation, a last-known-good artifact, or a source acceptance now say why, an unproven restore is marked as such, and the deriver folds all of it into the limitations the projection already exposes. Clearing a code when the evidence becomes provable again is part of the contract, so a stale doubt cannot outlive its cause. This lands before the migration matrix because migration is the largest producer of evidence that cannot be proven, and the plan requires those to surface as bounded limitation evidence rather than as fabricated pointers. * feat(git): migrate pre-existing Git stacks into the revision state Git stacks created before this model had no application at all, so every producer was a no-op for them and the projection could not describe them. They are now brought in at boot. The governing rule is that a canonical pointer is written only when the evidence proves that exact generation under the repository and ref configured now. A legacy applied commit is not that proof on its own: the manifest may be gone, unreadable, or stamped for a repository the stack no longer points at. In each of those the commit is kept as recorded limitation evidence and the pointers stay null, so the stack asks for a fetch instead of asserting a state nobody verified. Deployed, healthy, and last-known-good are never invented, because a manifest proves what was materialized, not what is running, and no source acceptance is written because nobody approved through the model. Replay is decided by scope, schema version, and configuration fingerprint. A changed fingerprint re-runs the matrix, but a stack that already has an application is skipped rather than rebuilt, so migration cannot overwrite pointers written with proof it does not have. A stack whose directory has vanished migrates to a tombstone rather than claiming a name it cannot back. * feat(git): record rollbacks in the revision state A restore now opens a recovery in the model before any file moves, so a crash mid-restore leaves a target that says what it was doing rather than one that merely looks broken, and closes it on both outcomes. The failure is classified by whether the files had already been restored, because only a failure before that leaves the previous workload provably intact. Pointers move only when the restore is provable: the recovery point named a generation, that generation still exists, and the manifest actually restored carries the same commit and manifest version. Anything less is still recorded as a real recovery, it simply has nothing to bind, and the transition marks it unproven rather than guessing. The deployed pointer is never claimed here. This path drives Compose through a callback that reports nothing back, so binding cannot be proven and applied moves without it. * feat(git): add the deferred rollout-state transitions Retry scheduling, suspension, pause, and partial rollout complete the transition store. They have no production writer by design, but implementing them now means the deriver has no branch a writer cannot reach, and the shape a future producer has to satisfy is pinned rather than inferred from the read side. None of them is a statement about health. A scheduled retry leaves the failure that caused it visible, so a stack that keeps failing does not read as merely busy. A suspended source keeps everything it had accepted, and an operation in flight is interrupted rather than abandoned so it cannot report as running for ever. A paused or partially rolled out target keeps whatever was deployed, and the partial record never stands in for a deployed pointer. * test(git): stop the Compose producer tests reading Docker from the host Both deploy cases asserted that the compose command would reject, which was only true on a workstation with no Docker daemon. On Linux the command succeeded, the promise resolved, and the assertions failed. The compose subprocess now reports an exit code each test chooses, so the adapter is what decides the outcome rather than the machine. Only the verbs that need a daemon and travel through spawn are answered; `config` still runs for real, because a host with the CLI and no daemon parses compose files exactly as CI does, and failing that call breaks every create in the file. That also makes the bound path reachable, so the deploy test now covers it: a failed compose leaves the deployed pointer alone and classifies post-mutation, and a successful one binds the applied generation, returns it for health to bind against, and clears the earlier failure. * fix(git): contain every managed-area path at the call that uses it A stack name reaches the managed root without passing through `isValidStackName` on this path, so the marker, the cleanup, and the create's root probe all built filesystem paths from unvalidated input. Each call now resolves its target against the managed area and checks containment in its own scope, which is also the form the security scan credits: it does not follow the barrier through the shared `isPathWithinBase` helper, so the check has to sit with the call it protects. `cleanupUnclaimedManagedRoot` removes a whole root recursively and gets the same check, even though it was not among the reported calls. Clearing a staging marker can now fail, so the two boot-recovery branches that dropped the checkpoint first were reordered. Losing the checkpoint while the marker survives would report the area as settled and leave the stack name uncreatable. The deploy adapter's two failure logs built their format string from the stack name; both now use a constant format with the name as an argument. The marker and cleanup fixtures stood a bare temp directory in for a managed root, so nothing exercised the invariant these checks enforce. Both now build a real managed area over a scoped data directory. * fix(git): refuse to claim a managed area whose marker cannot be read A marker that exists but will not parse is still someone's claim, yet the write path refused only a readable foreign marker and wrote straight over a corrupt one, discarding the reason and logging nothing. A transient permission error, a marker truncated by a crash between write and rename, or a full disk all read as corrupt, and each one let a second create take ownership of an area the first still owned. Every other path in the module preserves on corrupt; this one now does too, and says why. Clearing a settled create's staging marker can fail, and reporting that as a retained create sent a reader looking for an unfinished create that had finished. It reports `marker_retained` instead. The checkpoint is still kept for the retry, so its encrypted token can outlive the create while the clear keeps failing; that tradeoff is stated at the call, because the alternative leaves a marker no later create can get past. The reaper also computed why it was preserving an area and threw the reason away, so a directory could survive every boot with nothing said about it. The containment checks these paths rest on had no tests and were silently deletable, so the out-of-area read, write, delete, and reap now have them. * feat(git): bind and observe a recovered generation A proven restore could never claim the deployed pointer, because the restore path drives Compose through a callback that reported nothing back: a caller that restored some other way resolved identically, so binding on a resolved promise would have claimed a workload nobody launched. The Compose wrapper now returns what it did, and only that answer binds. With binding reachable, a bound recovery claims its health run inside the same transaction as the pointers it describes, and arms the timer once that transaction lands. Committing the two together is the point: a crash between them would otherwise leave a restored workload nothing was watching. Anything that stops the timer starting writes the run off immediately, so an observing row never outlives the timer meant to watch it, and a reservation is never armed across a restart. The reservation is handed to the transition rather than reached for, because the health gate reports its verdicts back through the transition store and importing it there would close a module cycle. Startup now finalizes each interrupted observation on its own instead of sweeping them with one update, so the revision state hears a verdict for every run rather than watching the rows change under it. * feat(git): add the rollout-scoped rollback transitions Completes the deferred-state set. The three aliases write the same recovery columns under the same rules as the recovery events, differing only in provenance: Direct Git recovery emits `recovery_*`, and a later rollout producer emits these. Nothing in this PR writes them, which is why they are tested directly rather than through a caller. `partial` is the one failure class they add, for a rollback that reached some targets and not others. Completing has no unproven variant on purpose: a rollback nobody can bind to a generation has nothing to complete against, so it refuses without one and refuses a generation another application owns. Also covers the recovery health reservation end to end: reserving writes and links the row without arming anything, a replay reuses the run rather than opening a second observation of the same restore, arming inserts nothing and supersedes a conflicting stack gate, an unarmed reservation is written off once, and a reservation that outlived its process is finalized rather than armed. The per-row startup sweep is pinned too, since one row whose write fails leaving the others finalized is the entire reason it replaced a bulk update. * feat(git): add the Blueprint source and deployment transitions The first slice of the Blueprint work: the store side, with no production caller yet. The routes and the reconciler are wired to these next, and pinning the contract first means a caller cannot quietly satisfy a different one. What these enforce is that a terminal event names the request it answers. A deploy records the intent and candidate it was launched for, and a terminal is accepted only against that same request, matching stage and identity on one side rather than either in isolation. Without that pairing a deploy could be acknowledged out of an in-flight withdraw, or one request's intent stored against another's candidate. An acknowledgement carries the candidate from the matched request rather than from its own payload, for the same reason. Releasing an operation now releases its identity with it, and a resolved interruption is retired rather than left to match again. Both were leaking through the shared helpers: a stale identity let a later start resurrect a superseded intent as live, and a surviving interruption both reported completion as unknown for ever and let a late acknowledgement regress the target after two later deploys had succeeded. Fixing the helpers fixes the Direct paths too. A start refuses to displace an unrelated operation, which would otherwise abandon it with no terminal event and no history saying so. Observations write the latest stage, since the placement facet reads it to decide whether a stateful deployment is waiting on confirmation, and history alone could never reach a reader. Minting is left to the caller. A no-op Blueprint edit must mint no intent, because a fresh identity would invalidate acknowledgements that are still accurate. * feat(git): record Blueprint create, edit, pin, and delete Wires the Blueprint routes to the revision state. Each producer writes the Blueprint source row and its GitOps rows in one transaction, so an operator never sees a Blueprint that exists with nothing describing what it means, or an intent for a Blueprint that failed to save. The question these answer is when an edit invalidates what the fleet already acknowledged. Changing the name, the compose content, the selector, the drift mode, or whether it is enabled changes what nodes run or where, so each mints a new intent and candidate. Changing the description or the classification changes how the Blueprint reads and nothing a node can observe, so it updates the source row alone: minting there would make every acknowledgement report as stale over a reworded sentence. An edit is measured by value, not by which fields were submitted. The editor sends every field on every save, and the source layer decides what to invalidate from which keys are present, so only the keys that genuinely differ are passed down. Without that the two disagreed: rewording a description advanced the revision past the one the current intent describes and cleared the approval, while this layer classified it as metadata and minted nothing. Selectors compare by value too, so reordering a list that names the same nodes is not a placement change. That measurement also changes one behaviour: a save that alters nothing now leaves the approval alone, where before any save cleared it. Nothing changed, so nothing is invalidated. Pinning revises placement the same way a selector edit does, and re-pinning the node already pinned is not a change. Deleting tombstones the application and its live targets so the Blueprint stops claiming its slot, and withdraws nothing itself, because the route has already done that and doing it twice would record removals that never happened. The required node set is stored in a canonical order, so reordering it is not mistaken for a placement change. A Blueprint that predates the model has no application yet and is left for migration rather than given a first intent here, which would claim a starting point for deployments nobody has reconciled. Desired nodes are passed in rather than computed here: the reconciler that knows how to compute them reaches this layer, and importing it back would close a module cycle. * feat(git): record the placement a label or cordon moves A label and a cordon say nothing about any one Blueprint, but both change which nodes a selector matches. Each now revises placement for whichever Blueprints the change actually moved, by comparing the desired nodes either side of the write rather than reacting to the event. That comparison is the whole point. Labelling a node no selector mentions, or cordoning one no Blueprint wanted, moves nothing, and minting an intent for it would invalidate every acknowledgement in the fleet over an edit no node can observe. The same comparison covers a Blueprint pinned to a cordoned node: a cordon governs automatic placement only, so the pinned target still wants that node and its set does not move, without needing a case of its own. Nodes are compared as a set, so the same nodes returned in a different order is not a change either. Each route wraps its existing write and the recording in one transaction, so a recording failure cannot leave a fleet selecting on a label nothing recorded. The write itself is untouched, which keeps label validation and the per-node limit where they already live rather than restating them in a producer. Node deletion already retires its targets, so this adds nothing there. * feat(git): record Blueprint deployments by what caused them Every production write to a Blueprint deployment row now goes through one funnel that records the cause. The cause is carried rather than inferred from the resulting status, because several causes land on the same one: a deploy that failed and a withdraw that failed both read `failed`, and they mean opposite things about whether the deployment is still on the node. Recording is skipped when the status did not move, so a reconciler tick that re-asserts a state it already reported does not append a second event describing the same fact. The write still happens either way. A terminal answers the request the target says it was given, not whatever the Blueprint currently wants: an acknowledgement matched against the current intent would accept work for a revision that node was never sent. The first deploy to a node creates its target, since a Blueprint application has no targets until something is sent somewhere. Recording never fails the deployment. The rollout already happened, and turning a bookkeeping problem into a stuck rollout would be the worse outcome. The four reconciler observations record what was seen and nothing else. Preview cleanup stays outside this path deliberately: it reverses a projection nobody deployed, so recording it would report removals that never happened. * fix(git): record an Inline Blueprint coming into existence Creating a Blueprint inserted its application row directly, so the history began at the first intent and described an application nothing recorded coming into existence. Every other application-creation path emits the activation event, and the event's own definition covers this case: an Inline Blueprint inserts the application, the first intent, and the first candidate. Activation goes through a transition now, which also enforces the one thing the direct insert could not: a Blueprint gets one live application, refused rather than silently duplicated. No target is created. A Blueprint application has no targets until something is deployed somewhere, unlike a Direct one which always has the node its stack lives on. * fix(git): file withdraw failures as withdraw failures Four of the five withdraw failure paths were recorded as failed deploys. Only the thrown one was tagged correctly, and it is the rare case: a delete-lock conflict, a network error, a remote 409, and a remote non-200 all return rather than throw, and all four were filed under deploy. That inverted the thing this funnel exists to prevent. Recording a deploy failure clears the in-flight operation, so a routine "another operation is already in progress" wiped the withdraw that had just started and left the target reporting a failed deploy while the deployment was still on the node. A withdraw refused because the on-disk stack is not ours is now its own cause too, rather than borrowing the deploy-side name conflict. The status guard no longer suppresses starts. A start writes the identity that terminals are matched against, so skipping one because the row already read `deploying` let a later acknowledgement answer a request that had been superseded, reporting a node as converged on a revision it was not running. The guard now covers observations only, which is where repetition is the reconciler re-asserting a state it already reported. A newer deploy supersedes an older one so a redeploy of a stuck request can take over; anything else in flight is still refused rather than displaced with no terminal event. A rejection is logged differently from an infrastructure error, because a target the model keeps refusing holds its active slot and stops recording anything further, and that is worth seeing. * feat(git): migrate pre-existing Blueprints into the revision state Blueprints that predate the model are brought in at boot: an application, an intent describing what the Blueprint currently asks for, and a candidate marked as coming from the legacy inline record. None of it is an acknowledgement. The Blueprint revision is carried for display and nothing else, because a revision number is not evidence that any node is running what it names, and recording it as agreement would report a fleet as converged on an intent nobody verified. No targets are created for the same reason, and the candidate's required set is empty: migration records what is asked for, never which nodes currently satisfy it. An approval authorizes the intent it was given for, so a Blueprint that was never approved, or edited since, is recorded as needing reapproval rather than left blank. Blank would read the same as an approval that is merely absent. Replay is decided by the same scope, schema version, and fingerprint the Git migration uses, and a Blueprint the new path already described is skipped outright, since its rows were written with proof this pass does not have. Blueprints migrate in their own pass. Coupling them to the Git migration would let one unreadable Git stack keep every Blueprint outside the model. * fix(git): contain the blueprint stack-directory probe at its call The ownership probe resolved a path from the blueprint name and checked it through the shared helper, which the security scan does not credit as a barrier. The check now sits with the call it protects, matching the form used elsewhere for the same reason. The path was already validated, so this changes no behaviour. It surfaced now because an earlier commit in this branch rewrote the file's line endings, which made every line read as new and brought a pre-existing call into the scan's changed-code window. * feat(git): read GitOps history and carry revision state on source reads Adds the instance-local history API and the additive revision fields the source reads have been missing. `GET /api/git-sources` and `GET /api/stacks/:stackName/git-source` now carry `gitopsRevision` and `stackResourcePresent`. Only the instance that owns the files can say whether a stack directory is really there, so that answer travels with the response rather than being inferred by a reader that has never seen the filesystem. It is read through the strict stack listing: the lenient one answers a failed directory read with an empty list, which here would read as every stack having vanished and would turn an unreadable disk into an empty audit trail. Two history routes land with them: `GET /api/git-sources/history` across stacks, and `GET /api/stacks/:stackName/git-source/history` for one. Both page on `(created_at, id)` so rows written inside the same millisecond survive a page boundary, and both bound how far one request will scan. A malformed cursor or an unusable filter value is refused rather than ignored, because answering "show me the failures" with everything reads as an answer instead of a non-answer. Authorization is decided per row. A row reduces to a stack read only when it names a stack, its application is live, and the stack is present on disk; anything unprovable stays with Admin. The cross-stack routes classify every row, while the per-stack route is authorized whole by name, so reading a stack shows its full history including entries from an earlier application. That distinction is carried in the scope type rather than a flag, so skipping the row classifier without pinning the query to the authorized stack cannot be expressed. History entries record the fields each transition moved rather than a whole revision, so lifecycle for the classifier comes from the owning application row. That keeps authorization off the recorded payload entirely: an entry whose detail cannot be read still returns its identity, stage, and outcome with a stated limitation, and the decode failure is logged rather than passed over in silence. * feat(git): let auditors read GitOps history they cannot tie to a stack A history entry that cannot be tied to a readable stack fell to Admin, which left the auditor role seeing exactly what a viewer sees on the one surface it exists to read. History entries are an audit trail: insert-only, and recording the actor, trigger, stage, and outcome of every transition. The request audit log is already gated on the audit permission, so an entry whose audience cannot be narrowed now falls there too. Withholding it protected nothing in any case, since the request log already records that a Git-source mutation happened and who made it. The source collection deliberately keeps its Admin fallback. Those rows are live Git configuration, not a record of events, and a mandate to audit does not extend to reading the repository, ref, and credentials policy of stacks that were deleted or never finished being created. The fallback is now part of the requirement type rather than a role comparison, so each surface states which audience it falls back to and the compiler requires every case to be answered. * feat(git): correct remote node identities on GitOps reads A remote instance numbers its own nodes and has never heard of the hub's numbering, so every node id it reports is a statement in its own namespace. Read straight through, a hub joining two nodes showed two different machines as the same node. Four GETs now take a second hop that buffers the response and corrects those identities: the git-source list and detail, and both history routes. Everything else keeps streaming exactly as before, which is the reason this is a separate hop rather than a mode of the existing one: logs, downloads, and event streams must never be buffered, and the enumerated positions rewritten here do not appear in them. Only JSON numbers are replaced. A null node is preserved, since "no node" is a fact the remote is entitled to state and inventing one there would claim a placement that does not exist. Strings, application ids, and stack names travel untouched. The hub re-authorizes what comes back. A remote authorized its rows for the machine account the hub proxies with, which says nothing about the person behind the request, so every row is classified again against the signed-in user. To make that possible without the hub holding another instance's database, each history entry now carries the lifecycle of its owning application alongside the stack-presence answer already there. Both are validated fail-closed, and a verdict a remote might volunteer is ignored: the hub decides, the remote supplies evidence. Asking one node for another node's history is refused before the hop rather than forwarded, because the remote would answer about itself and the page would read as an answer to a question nobody asked. When the node asked for is the node being talked to, the hub translates that into a flag the remote resolves to its own default node. Only the hub may set that flag, and a remote honours it only on a proxied hop. Every way this hop can end converges on one terminal answer: rewritten, passed through, too large, undecodable, failed upstream, or abandoned by the client. A body the hub could not read never borrows the remote's success status. * test(git): assert the unlinked git-source response field by field The detach test compared the whole response to `{ linked: false }`, which stopped holding once that route started carrying the additive revision fields. The equivalent backend route test was updated when those fields landed; this one was missed. Asserted field by field rather than loosened to a partial match, so the two new fields are actually checked: a detached stack has no application to project while its directory is still on disk. * feat(git): carry GitOps revisions on blueprint, node, and drift responses Completes the additive JSON half of the revision-state read contract, so the Blueprint, node, and drift surfaces report GitOps state through the same projection the Git-source routes already return instead of leaving it unreadable. Blueprint list, detail, create, update, and pin carry gitopsRevision. Node-label add, cordon, uncordon, and node delete carry gitopsRevisions, ordered by blueprint id and covering only the Blueprints the mutation actually moved: an edit no selector reacts to reports an empty list rather than invalidating the whole catalog. Node deletion reads its Blueprint owners inside the deletion transaction and before the tombstone, which is the last moment a target row can be traced back to an application, and an orphaned target is logged rather than being silently indistinguishable from a Direct one. Both DELETEs that answered 204 still answer 204. The drift GET and re-check gain the same field alongside the existing ledger, which is untouched; no GitOps class is written into stack_drift_findings. Reads and mutations treat a projection fault differently, on purpose. On a read the revision is part of the answer, so a fault surfaces. On a mutation the write has already committed, so the revision is decoration and is built defensively: letting it throw would answer a successful cordon or a completed node deletion with a 500, sending the operator to retry a hard delete that already happened and be told the node does not exist. It degrades to an empty list and logs. Because both drift routes answer with that projection, the identity hop now intercepts them too, matching its route table per method instead of rejecting every non-GET. Left as they were, one object would carry the hub's node numbering or the remote's depending only on which route asked for it. The allowlist names a single mutation rather than opening a verb, and the hub's collection filter still keys off the git-source paths alone, so drift payloads are rewritten without being re-authorized: they are per-stack and already authorized by name before the hop. Tests cover the exact shapes, the preserved 204s, revision ordering, the empty-list cases, node deletion reporting its tombstoned Blueprints and surviving a projection failure, and the proxy's new route matching, drift rewrite, and filter exclusion. * fix(git): resolve every application a GitOps surface can own Three gaps found while building the additive JSON, all the same shape: an id or a sentinel stood in for a row, so "there is nothing here" and "the thing that should be here is unreachable" collapsed into one answer nobody could tell apart. A Blueprint application is stored with stack_name NULL, so no lookup by stack name could reach it, while the reconciler materializes every Blueprint as a stack directory of that name. A stack's Drift tab and Git panel therefore reported no GitOps at all for a stack GitOps was actively managing, while the Blueprint page reported a live application for the same thing. Stack surfaces now bridge the two through the deployment row for the asking node. The deployment row is what makes it safe: Blueprint and stack names share one namespace, so matching on name alone would let a Blueprint claim an unrelated stack of the same name on a node it never targeted. A retired application was unreachable for the same reason, because every entry point filtered to the live rows. Tombstoning deliberately keeps the configured identity and SHA pointers as frozen facts so the projection can still say what an application was, and the source deriver has a not_live status waiting for exactly that, but nothing could hand it one. A detached Blueprint or stack now reports what it was instead of reading as one that never existed. The not-applicable projection was typed so it could not carry a reason, which made a row that vanished between resolving it and re-reading it by id identical to a stack the model was never asked about. That one field widens to hold limitations, and the missing case now says so with the id as evidence. Every other missing-row path in the deriver already worked this way. Also logs, rather than silently skipping, a live application whose blueprint row is gone: with cascade off that is a real integrity fault, the placement did move, and the response would otherwise report that nothing had. Cordon and uncordon are deliberately left reporting nothing. A cordon governs whether new placements may be made, not what a Blueprint asks for, and the reconciler applies it only to new placements, so revising intent for it would invalidate acknowledgements fleet-wide over a change that evicts nothing. Two comments implied otherwise and now state the contract. * fix(git): keep stack-state resolution out of the Git-source read path Review of the previous commit found the fallback it added had reached further than intended. The resolver is now split by the question it answers. projectStackRevision answers "what Git source is attached to this stack" and stays Direct-only. projectManagedStackRevision answers "what manages this stack" and is what the drift routes use. The Git-source routes must not be answered with another application's identity, and there was a second reason: the row classifier takes the stack name from the Git-source row but the lifecycle from whatever the projection resolved, so a Blueprint application arriving there would have turned an Admin-only row into one any stack grant could read. The Blueprint that unlocked it could be the very one holding name_conflict because of that stack. The Blueprint-to-stack bridge also needed the right predicate rather than a present deployment row. name_conflict is written precisely when a stack of that name already exists on the node and Sencho does not own it, so guarding only against withdrawn admitted exactly the collision the bridge exists to prevent. It now requires last_deployed_at, which proves this Blueprint wrote the directory, and excludes name_conflict and withdrawn: the same predicate the delete and withdraw paths use. Only a live Blueprint application qualifies, since a retired one has no claim on the directory. The detached fallback no longer resolves deleted applications. Detached means the files are still on disk and still describe that stack; deleted means the stack is gone, so any directory of that name now belongs to something else and reporting the old repository and SHA against it would disclose one stack's Git identity through another's name. The authorization side already refused deleted rows for that reason, so the two now agree. Ordering breaks ties on rowid rather than a random UUID, and two partial indexes cover the new lookups, which the existing live-only unique indexes could not serve. The not-applicable variant's limitations are readonly, so pushing onto the shared frozen instance is a compile error again rather than a runtime throw. Tests cover a Blueprint claiming a stack it never deployed onto and one it hit a name conflict on, a detached Direct source projecting not_live, and a deleted one staying unresolved for a reused name. Drift fixtures move to afterEach so a failing assertion cannot leak state into the next test. * fix(git): stop the stack resolver falling through proven ownership A second review round found the resolution chain could answer with the wrong application, and that part of the previous commit was reading a state nothing writes. The Blueprint bridge returned one value for two different facts. Once the deployment row proved a Blueprint had written the directory, a missing application row for it still answered "no Blueprint here", so the chain continued to the detached-Direct lookup. A stack that once had Direct Git, was detached, and whose directory a Blueprint later took over would then report the old Direct application's repository, ref, and SHA as that directory's state. The bridge now separates "not mine" from "mine, but broken": the second logs and returns its own limitation, so the fault is visible and no unrelated identity is offered in its place. A Blueprint projection also carried the whole placement roster into a route authorized by a grant on one stack name. Targets are now scoped to the node being asked about, which is both the safe answer and the accurate one, since the question is what manages this directory here. The detached-Blueprint lookup, its index, and its fallback are removed. Blueprint retirement writes deleted, never detached, and the only detached writer is the Direct detach path, so the getter could never match and the index covered an empty set. The test that seemed to cover it built the state by hand and was green against something the product cannot produce; it now drives the real delete route and asserts what that path actually writes. The Git-source resolver goes back to live applications only. Detach removes the source row in the same transaction that tombstones the application, so a source row beside a detached one is not producible, and the route's row classifier takes its stack name from the source row but its lifecycle from the projection. Keeping that resolver narrow is what stops a resolution change from quietly moving who may read a row. The comment saying so is restored and now explains why it must stay true. * refactor(git): share the GitOps application fixture across the route tests Two test files carried near-identical 58-line GitOpsApplicationRow literals that differed only in a repo URL nothing asserted on. Both now import one helper. The drift tests also repeated a Blueprint seed and its activation across four cases, and re-imported DatabaseService per test; those become local helpers and one hoisted import, which lets the cleanup hook go synchronous. Node lookup throws on a missing default instead of casting. projectBlueprintRevisions is module-private again: its only caller is the committed-projection wrapper beside it. A third copy of the same fixture remains in git-source-routes.test.ts. It is left alone deliberately: it pins timestamps at 1 rather than now, and that file has ordering-sensitive history assertions that would need auditing first. * feat(git): mirror the GitOps revision contract on the frontend Adds the client-side read contract for the GitOps revision projection, with no consumers yet. types/gitops.ts is a hand-written mirror of the backend read contract, in the same convention as the other domain type files: the frontend never imports backend. The projection's two arms are named separately so a component can take the live one as a prop without re-narrowing, and the absent arm keeps its absent keys rather than nulls so reaching for a lifecycle status without narrowing is a compile error. The limitation code stays an open string, since the backend adds codes without a schema bump and an exhaustive switch would silently stop rendering the newest ones. lib/gitopsState.ts is the one place a facet status becomes words and a colour, so a state reads the same in a sidebar tooltip, a panel banner and the drift tab. Both maps are keyed on the closed status unions, so widening the mirror without naming the new state fails this build. Every line of copy states the condition the deriver actually tests, which is not always what the status name suggests. recovery_required is a recovery already running rather than one that is needed. candidate_ready is reached only when review is not required, so it is ready to apply, not ready to review. synced_and_healthy is also reached with the health gate switched off and never compares the deployed generation against the accepted one, so it claims neither. never_reconciled means nothing has been accepted; a fetch that produced no materialization still records its commit. pendingSourceStatus keys "an update is waiting" on the candidate pointer rather than the status name, because source_reconcile_required is reachable both from a stale candidate and from an accepted generation with no candidate at all. Retirement is excluded first: tombstoning keeps the candidate pointer as a frozen fact, so a stack detached mid-review still carries one. absentFault separates the two facts the empty projection carries. Empty limitations means the model was never asked about this stack and the right rendering is nothing; a non-empty list means an application that was expected could not be reached. A live application's limitations are caveats on state that is being reported, not faults, so they are deliberately excluded. * feat(git): name the Git source state on the stack and sidebar surfaces The sidebar indicator and the anatomy source row both read a raw pending commit pointer, so a candidate blocked by local conflicts, held for review, or stale against the configuration in force all render identically to one that is ready to apply. The projection distinguishes them; nothing was reading it. refreshGitSourcePending now derives each stack's state from the projection and the pending map carries that state rather than a boolean. Presence in the map still means exactly what it meant before, because it is keyed on the candidate pointer rather than the status name: source_reconcile_required is reachable both from a stale candidate and from an accepted generation with no candidate, and only the first is something to review. One narrow fallback remains, for a row with no projection at all: a failed GitOps write is logged and swallowed while the pending commit still commits, so the flat pointer is the only thing that can answer there. Wherever a projection exists it is the sole authority. The sidebar keeps one indicator, in the same slot, at the same size and colour, with the same position in the priority ladder. Only the tooltip changes, so a blocked plan now says so instead of reading as an ordinary update. A test asserts the rendered indicator is identical across states, which is what keeps the rendered sidebar unchanged. The anatomy source row keeps its pulsing dot alone for the ordinary case and adds the state word only for the states the dot cannot express. Inline rather than in a tooltip, since this is the reason something is stuck. * feat(git): surface derived GitOps state in the Git source and Drift panels The Git source panel's pending banner had two things it could say, read from a raw commit pointer and a single blocked boolean. It now names which of four states the candidate is actually in, so a plan blocked by local conflicts, one held for review, and one gone stale against the configuration in force stop rendering as the same ordinary update. The short commit sha stays. A new card above it reports an application the projection could not reach. That case renders as nothing at all today, which is indistinguishable from a stack the model was never asked about. Empty limitations stays silent, because that is the ordinary answer for most stacks and a header over an empty block would be worse than nothing. The summary block gains one row naming the source state. It is the first place the panel can show applying, retry scheduled, suspended, recovering, or a detached source, none of which have a pending commit and so never reach the banner. Both existing rows are untouched and the last applied commit stays a display fact. The revision lives in its own state rather than on the Git source type, because the PUT that saves this panel answers with a bare source and no revision. A save drops it instead of rendering a state the write has already moved. The Drift tab gains a third block below the two it already has. The compose versus runtime card and the since-last-deploy card keep their exact positions and copy; the new block answers a different question, which generation this node was asked to run and whether it got there. A Blueprint-owned stack shows its target rows and no source card, since a Blueprint application has no Git source and inventing one would be a claim the model never made. The drift class list renders against the type and expects no rows, because nothing populates it yet. * feat(git): declare revision fields on the blueprint and node clients Types first. The blueprint list, detail, create, update and pin responses all carry a GitOps revision, and the node label add, cordon, uncordon and delete responses carry a list of them. None of that was modelled, so the fields were being dropped silently and the next reader would have taken these for bare records. Most of it is declared and deliberately unread, with the reason written where the type lives. Create, update and pin are followed immediately by a re-read of the catalog or the detail that carries the same projection, so rendering the mutation's copy would show the same fact twice with one of them stale. Cordon and uncordon always answer with an empty list by design, because a cordon governs whether new placements may be made rather than what a Blueprint asks for, and the reconciler leaves existing deployments where they are. The comment on that type is the point: the risk there is a future reader building a consumer for a list that is never populated. Two places do report something. Deleting a node and adding a node label both re-place Blueprints, and both now say how many. The count only, and only when it is not zero: an empty list means both that nothing moved and that the projection faulted after the write had already committed, so it can never be reported as the first. Blueprint detail gains the same unreachable-application card the Git source panel has. A Blueprint with no live application row is a fact nothing in the product could previously express, which is a different problem from having two ways to say the same thing. * fix(git): close the gaps review found in the GitOps frontend consumers Six defects, each verified against the code before being fixed. The pending-map read assumed every /git-sources row carries a projection. That route is proxied, so a node predating the model answers rows without one, and dereferencing it threw inside the loop. The catch swallowed the throw, the map was never set, and every stack's Git indicator froze at its previous value for as long as that node was selected. The field is optional now and a row without one falls through to the same branch as a row with nothing to project. That fallback also branched on the arm alone, so a projection reporting an unreachable application plus a stale commit pointer was reported as a candidate ready to apply. It now excludes a faulted projection: a fault means an application was expected and could not be read, so the pointer is not evidence that anything is ready, and naming a state there would be a guess. Detaching a Git source cleared the source but not the revision, and the pending card is derived from the revision alone. The panel kept advertising a waiting commit for a stack Git no longer managed, behind a Review button that silently did nothing. A read that threw had the same problem across stacks: the panel is reused, so stack A's state could render under stack B's header. The panel also lost the flat-pointer fallback the sidebar keeps, so the two surfaces disagreed: the sidebar would show an indicator and clicking through showed no card. Both now apply the same rule. Node delete parsed the success body unguarded. The delete has already committed at that point, so a malformed body would have reported a completed deletion as a failure and skipped the refresh, leaving the deleted node on screen. Tests: five mutations that previously survived now fail. The zero-pixel claim compared the wrong element and passed when the indicator vanished entirely; the stale-read test asserted during the loading window, when the body is skeletons regardless; the sidebar passthrough test proved only that an indicator existed, not that the state reached it. Added coverage for the drift row, an unknown node, the not-applicable source guard, the save clearing, the Review button, and a live application carrying a caveat, which must not read as a fault. The fixture set drops two source statuses whose identity defaults describe a state they cannot be in, and gains a drift-item builder. * refactor(git): share the GitOps fault card and live-facet derivation The unreachable-application card was built inline on three surfaces, and the copies had already drifted: the Git source panel used one icon while the Drift tab and the Blueprint sheet used another, so the same failure rendered two different ways. It is one component now, which also owns the state key and the test id those surfaces assert on. Deriving the live source facet was likewise repeated, along with the two semantic decisions behind it: the absent arm has no facets, and a source facet of not applicable means a Blueprint owns the stack rather than that something is missing. Both now live next to pendingSourceStatus, which already encoded the same exclusion. The Git source panel's seven derived values collapse to three, with the pending-commit rule extracted to a named function so the four cases read straight through instead of as nested ternaries. Behaviour is unchanged in every case. The two flat-pointer fallbacks are deliberately not unified: they look alike but the panel also treats a live application whose source facet is not applicable as unanswered, and merging the predicates would change what that case renders. * feat(git): count and announce committed GitOps transitions Every history row that is actually inserted now produces one metric increment and one state-invalidate event, so the surfaces that read GitOps state hear about a change instead of waiting for the next poll. Announcement is buffered and drained on a macrotask rather than fired inline. better-sqlite3 is synchronous, so waiting for the macrotask puts the drain after the transaction that wrote the row, and after any outer transaction wrapping it, without having to detect which nesting depth it is in. The drain confirms each row is still present before announcing it, so a rolled-back transaction says nothing on its own, and the insert declines to queue a dedupe replay, so a retry says nothing either. The broadcaster is injected at startup rather than imported, keeping the GitOps layer free of a cycle back into the notification stack. Counters are process-local and in-memory, and their keyspace is finite by construction: history stages are now a closed union that the build enforces, and outcomes were already a closed set. The payload names no stack, node, repository, or actor, because a counter carrying those would be an audit trail with no retention rules and no per-row authorization, which is what the history routes provide. GET /api/gitops-metrics is Admin-only and instance-local, so selecting a node answers with that node's counters. On the client, a gitops-scoped invalidate refreshes the derived state through a 250ms trailing window, matching the existing stack refresh. One operation commits several transitions in a row and a first-boot migration commits a great many, so refetching per event would thrash the API for a picture that only settles at the end. * feat(git): show GitOps source state on the stack dashboards The dashboards list every stack on a node, which is where a fleet-wide reading of Git state is most useful and where, until now, a Git-backed stack looked exactly like a local one. Each row that the model has something to say about carries a chip naming its source state. The chip reads from the same status vocabulary the panels and the sidebar use, so a stack cannot be "pending update" in one place and something else in another. It sits beside the stack name rather than replacing the source column: the column says where the files come from, the chip says what GitOps makes of them, and those are different facts. The label is a word and the title is a whole sentence, so the state never rests on colour alone; the phone rows keep the word on screen, since touch has no hover. The join is by stack name, which is what the dashboards have, and its source is the same proxied route the sidebar reads. A row without a revision, a Blueprint-owned application, and a projection fault all leave the stack unbadged rather than inventing a state for it. State arrives by announcement rather than by poll, since it only moves on a transition, and a node switch blanks the map first: stack names repeat across nodes, so a slow answer for the node just left is discarded rather than allowed to label the wrong stacks. * feat(git): say which part of a GitOps state could not be proven Eighteen conditions can qualify a live projection: a manifest that does not match the branch configured now, an approval that could not be restored after a recovery, an artifact record that has gone. Every one was already recorded and none of them reached the operator, so a state with a hole in it read exactly like one without. Each now has operator wording under the state it qualifies, checked against the site that emits it rather than against the code's name, since several names describe something narrower or wider than the condition actually tested. The stored messages stay out of the UI: they are written for a log reader, and some are raw decoder errors. The presentation is deliberately quiet. A caveat is not a failure: the state above it is real, and one piece of evidence behind it is missing, so the reader learns which part to distrust without being told the whole thing is broken. Faults keep their own card, because those replace the state rather than qualifying it, and the two arms are read through separate helpers so neither can be rendered as the other. A code with no wording names itself rather than vanishing, so a node running ahead of this build degrades to something honest instead of reporting full confidence in a state its own backend flagged. * fix(git): record a stateful first placement and keep a half-built Blueprint editable Two faults that today's product cannot reach, and that the Git-backed Blueprint mode would reach immediately. The live-application lookup answers with an application that is active or still being created, because its other callers ask whether the Blueprint already holds the live slot, where a half-built row counts. Three producers passed that answer straight into transitions that accept only an active application and reject anything else by throwing, inside the caller's own transaction. A Blueprint edited or pinned while its application was still being created would have failed with a server error and rolled the Blueprint write back with it. The producers now narrow to what they actually require, through one shared predicate, leaving the slot check honest. Separately, a stateful placement is held for operator review before anything is deployed, so no target exists when that hold is recorded, and the observation was dropped for want of one. The hold left no trace: no history row, and nothing on the target to say the node had been asked to hold anything. First contact now creates the target, exactly as the first deploy does. Nothing else changes with it: no intent, no generation, no operation, and connectivity stays unset, because a node that has only been asked to hold something has not been contacted. Observations that follow a deploy already have a target, so they are unaffected, and a drift or evict report for a node nothing was ever sent to is still dropped. * fix(git): keep the Git source panel's state after a save The save answers with the source row and no revision, so the panel had nothing to replace its copy with and dropped it. The result was a stack that had just been reconfigured showing no GitOps state at all until the panel was reopened, which reads as a stack the model knows nothing about rather than one whose state has just moved. It re-reads instead. Keeping the old copy was not an option either: a material configuration change clears the staged candidate server side, so the state genuinely has moved and the panel would have gone on offering a commit that is no longer there. Only the server can say what replaced it. The alternative was returning the revision on the save itself. That route is proxied, and the identity hop rewrites node numbering for an enumerated set of routes, so adding a revision to a response outside that set would hand back a remote node's numbering unrewritten. Re-reading costs one request on a low-frequency action and needs no change to the hop. * docs(git): document what GitOps state means for an operator A commit SHA says which files Sencho wrote. It does not say whether that commit was accepted, whether something newer is waiting, or whether an operation was interrupted halfway, and until now the docs had no words for any of that. Git Sources gains a source-state table and a section on what happens when part of a state could not be proven, since a stack with missing evidence looks identical to one with complete evidence unless the product says otherwise. Drift Detection gains the gitops section and explains why it answers a different question from the two signals above it: those compare files and containers as they are now, this reports what has been proven over time. The dashboard page describes the state chip and, importantly, that it and the row tint are independent, so a healthy row carrying "pending update" is not a contradiction. The Blueprint page now separates its revision counter from GitOps state. The counter labels the spec; it says nothing about whether a given node has got there, and reading it as fleet truth is the mistake the note prevents. The tutorial's verification step names the source state it should show, so a reader can tell the difference between files written and a commit accepted. Audited and left unchanged: health-gated updates and fleet federation. Neither presents a SHA or a revision as canonical GitOps truth, so neither needed correcting. * docs(git): name which saves actually clear a staged commit The note claimed saving the form always clears the staged commit and moves the state to reconcile required. Only a change to what gets materialized does that: the repository, the ref, the compose paths, the project directory, or the env sync. Changing the token or the apply behavior leaves a staged commit alone, because neither changes what would be written, and telling an operator otherwise would have them pull again for nothing. * fix(git): say when a committed transition has nobody to announce it to Two silences worth breaking, both found reviewing the error paths in this slice. A server that never installs the event sink still counts every transition and still writes every history row, so the only symptom is that no client ever refreshes and the UI is quietly as stale as it was before any of this existed. It now says so, once rather than per row, since a boot migration would otherwise fill the log with one fact. The dashboard's source-state fetch dropped a non-ok response without a word. A refusal there freezes every badge at its last value, which looks exactly like a fleet where nothing has changed, so the status code is logged rather than inferred from badges that stopped moving. * fix(git): close the defects review found in the announcement slice Five findings, two of them able to reach an operator. The event sink was installed one line after the deletion reconcile, which tombstones applications and targets and awaits inside its own loop. A drain therefore landed while the sink was still absent, so those boot-time transitions were counted and never announced, and the warning added for an unwired sink would have fired on every boot with a prepared deletion intent: the fastest way to teach an operator to ignore it when it means something. The sink now precedes every reconcile and migration pass. The first-placement target was created outside the transition that records it. The observation runs in its own savepoint and can refuse, and the caller deliberately lets the deployment commit whatever the record says, so a refusal left an active target with no generation, no stage and no history: a placement relationship the model never established, which the delete path would later tombstone as if it were real. Both writes now succeed or fail together. The badge looked its status up in a map closed at compile time, behind two casts, while the value arrives over a proxy from a node that may run a newer vocabulary. An unmapped key dereferenced undefined inside a stack row and would have taken the whole table down with it. The facet and its status are now a discriminated union, the lookup is optional, and an unrecognised status renders nothing, which is what the join already does for a stack it has no state for. The dashboard's source-state loop derived each row inside the shared try, so one row in an unexpected shape abandoned the loop and froze every badge at its last value with nothing on screen to say so. Each row is now guarded on its own and the count of unreadable ones is logged. Two tests claimed things they did not check: both asserted a word was on screen with a matcher that also matches screen-reader-only text, so a compact badge would have kept them green, and one compared a label by containment where the shorter label is a prefix of the longer. Both now assert the visible node by equality. Added coverage for the window event the dashboard actually refetches on, an unknown status, a row that cannot be read, node-switch blanking, and a caveat recorded twice. One fixture built a status pair the deriver cannot emit and was corrected. Also: the guard narrowing Blueprint producers to an active application excludes nothing today, because every Blueprint-mode application is inserted as active and only Direct mode can be creating. The guard stays, since the getter's slot semantics and the transitions' requirement have drifted apart once already, but its rationale and its tests now say plainly that they pin a defensive guard rather than reachable behaviour. * refactor(git): make three runtime guards visible to the compiler Type review found the same shape three times: a correct guard whose necessity was asserted in a comment the type system could not see. This project does not set noUncheckedIndexedAccess, so each of these read as dead code to anything that trusts the types, and the guards protect the behaviours that matter most here. The limitation copy map typed its values optional, so the fallback for an unrecognised code is now something the compiler requires rather than something a comment explains. The badge reads its status through partial views of the two vocabularies, so a miss is a fact TypeScript derives: deleting the guard that stops one unknown status taking down a stack list now fails the build, where before it compiled. Neither needed a cast. The event payload has to be a type alias rather than an interface, because only the former gets an implicit index signature and the broadcaster takes an open envelope. That requirement now fails at the declaration instead of surfacing in the startup wiring as an unexplained index-signature error. Also added the reverse of the copy-coverage check: a code retired from the backend leaving stale wording behind was invisible, since the fallback only fires for entries that are missing rather than ones that linger. * fix(git): make the state cards survive an unknown status, and correct the copy Comment review found the protection added for the dashboard chip was claimed more widely than it held. Five other surfaces indexed the status vocabularies directly and passed the result straight to a card that dereferences it, so the exact input the comments said would take a list down still threw in the Git source panel, the Drift tab, the sidebar tooltip and the stack anatomy row. The card now takes an optional state and renders nothing without one, which also means a new surface cannot reintroduce the dereference by rendering a card the ordinary way, and the remaining four read through the lookups. Copy corrections, each against the site that emits it rather than the name of the code: The manifest-identity caveat said the manifest named a different repository or branch. It also fires when the manifest has no identity block at all, or identifies another node or another stack, so it now says it does not identify this stack on this node from the repository configured now. The last-known-good artifact caveat covered a mismatch but not the row being gone, which is the other half of the same condition. Doc corrections: a Blueprint-delivered stack was described as carrying the dashboard chip and a source state on the Drift tab, and it carries neither, because a Blueprint has no Git source of its own. Per-node Blueprint state was attributed to the detail sheet's GitOps section, which only reports what could not be read or proven; the Deployments table is where per-node status lives. Clearing a staged commit was said to always land on reconcile required, but a stack that has never had a commit accepted lands on never reconciled. The caveat lines were described as sitting under the state when they render above the form. And the pending banner was still documented as webhook-only, which this branch changed: it now appears for any staged commit and its heading is the source state. Also corrected two claims in the code: the observation branch does not skip for a Blueprint that migration brought in, because migration creates no targets, and the only refusal reachable from that call site is a tombstoned target. * refactor(git): simplify the announcement slice without moving any behaviour A simplification sweep over the new code, all of it behaviour-preserving and re-checked by breaking each guard afterwards. The drain now separates policy from mechanism: it decides whether a row survived, counts it, and either warns or hands it to a small announce helper that owns the envelope and its error handling. The row-to-state pass in the dashboard hook lifts out of the fetch, so one function does the request and another does the derivation. The caveat dedupe collapses to a set, which preserves first-insertion order and so keeps the ordering its own docstring promises. The badge renders one span with a conditional class rather than two that differed only by class. The metrics service drops an internal type that was byte-identical to the one it exports. One assertion goes: the observation branch tested membership and then re-asserted the same fact to index the stage map, and a type predicate lets the compiler carry it instead. Everything the sweep was told to leave alone is intact: the buffered drain, the row-existence re-check, the injected sink, the one-shot warning, the partial-view lookups and their guards, the optional copy map with its fallback, and the generation counter with its per-row guard. * fix(git): close audit findings in the GitOps revision model Five corrections from the pre-merge audit of this branch. Migration no longer certifies a commit it cannot prove. The manifest read now carries its resolved commit, and a manifest that names a different commit than the source row records as applied is refused: the applied directory materializes one commit while the row names another, so trusting them together would mint a generation pointing at files that are not the ones it claims. A manifest that names no commit at all, which is what adoption writes, is a separate answer rather than a disagreement, because reporting it as one would name a commit the manifest does not contain. Both are recorded as evidence and the projection asks for a fetch. Create cleanup proves containment against the real filesystem before it deletes. Every check on these paths was lexical, so a symlink or Windows junction above the target read as contained while the recursive delete followed it out of the managed area. Resolution keeps "is not there" apart from "could not be read": only a genuinely missing path lets the walk climb to an ancestor, because treating an unreadable one as absent would infer containment for the single path whose link status could not be established. Both sides are resolved, so relocating the data directory onto another volume keeps working, and an area that does not exist at all is nothing to delete rather than a suspected escape. The marker write takes the same barrier as the marker delete, so a link cannot be written through and then refused on the way out, wedging the stack name. Startup stops while an interrupted create is unresolved, matching the restore reconcile above it. A create that could not be settled leaves a stack directory the deploy path cannot tell apart from a finished one, so the alternative to stopping is letting a scheduler, webhook or operator act on a half-built stack. Only that blocks: once the staged directories are gone the create is torn down, and a staging marker nobody could unlink is reported rather than thrown, so one failed unlink is not the difference between an instance that boots and one that does not. Per-stack history no longer exposes a predecessor through a reused stack name. The grant covers the application holding the name now, which is what keeps a stack's own entries readable while it is still being created; every other row on that name is classified per row, so entries belonging to an application that held the name earlier need the audit permission, as they already did across stacks. A detached predecessor is still readable, which is the classifier's own standing decision about detach rather than a gap here, and is now pinned by a test. Blueprint observations reach a reader. The reconciler recorded state review, evict blocked, drifted and correcting against the target and nothing projected them, so a deployed Blueprint could report itself as never applied. The runtime facet now projects all four, below the states a live or failed mutation puts the target in and above the pointer checks, and any later transition supersedes the observation. The map is declared total over the stages the reconciler can record, so a stage added without a projection fails the build rather than silently reading as never applied again. * fix(git): confine managed-area cleanup to each stack's own location Cleanup proved only that a resolved path landed somewhere inside the managed area. That is satisfied by every other stack and every other node in it, so a link from one stack's generations directory into another's passed the check while the delete took a generation belonging to someone else. The check is now positional: the managed area is resolved once, so relocating the data directory onto another volume still works, and every segment below it must be reached without redirection. The create-path sinks use it on the write as well as the delete, so a claim cannot be written through a link and then refused by the hardened delete, and the manifest service's pre-existing deletion sinks (generation pruning, boot-sweep orphan reaping, detach staging and finalization, and whole-area removal, including after a restored snapshot) run it before their recursive deletes too. Each of those sinks also keeps a literal containment comparison beside the positional check, because static analysis credits only a comparison at the call and reports the delete otherwise. A refusal names both paths in the log, because it can hold the boot gate. Boot recovery dropped the checkpoint of an application that had left the creating state without first clearing its staging marker. That left a claim on the stack name with nothing to retry it, and every later create for that name was refused by a marker nothing could remove. The settled branches now share one exit, so the marker ordering holds for all of them. A detached application's history was readable on a stack grant because its files are still the stack standing at that name. Nothing in these tables can prove it still is: a Blueprint successor records the name off the application row, and a plain Compose stack recreated at the name leaves no trace at all, so no detection-based allowance can be made sound. Detach now moves an application's trail to the audit audience outright, the same answer deleted and creating predecessors get, while reading the stack itself stays where it was. * fix(gitops): close four audit findings in revision state and identity proxy A1: derive runtime and health against target desired_generation_id - deriveRuntime returns 'applied_not_deployed' when desired != deployed, so a stale deployment stays deploy-pending instead of reading synced_and_healthy - deriveHealth judges against desired_generation_id falling back to deployed when null; control case proves null-desired rows unchanged - collectRuntimeDrift emits the plan-pinned runtime drift item for exact/ qualified observation mismatches; equal/non-comparable observations emit nothing; ordering pin keeps deploy question first A2: stop upstream 304 responses from bypassing hub rewrite and reauthorization - identity proxy answers no-store on every terminal (rewrite, 204, 304, generated failures); validators (etag, last-modified, cache-control, expires, vary) no longer forwarded - conditional request headers (if-none-match, if-modified-since, if-match, if-unmodified-since) stripped on the identity-hop branch of the shared proxyReq handler; streaming hop untouched (optimistic-concurrency file writes depend on If-Match/If-Unmodified-Since) - supertest integration test drives the real middleware through a loopback capture server and asserts if-none-match absent while accept survives S1: migrate legacy query/userinfo URLs via secret-free path - parseLegacyRepoUrl strips userinfo, query, fragment instead of refusing; parseHttpsRepoUrl remains the strict gate for user-driven paths - migrationDirectSourceIdentity uses the tolerant parser; operational stack_git_sources.repo_url untouched; fingerprint convergence proven - trusted-manifest-on-legacy-URL test covers the worst real-world instance S2: emit runtime artifact drift for current evidence - top-level drift array carries the seven-class runtime item; frontend comments updated to reflect backend now emits runtime drift All regression tests added and passing. CI green (7,374 backend + 2,790 frontend tests; only pre-existing Windows EBUSY teardown failure unrelated). * test(frontend): fix DriftPanel drift item rendering test Update test assertions to match the new backend drift item format. The function now renders artifact_set expected identities as 'artifact <id> · <qualification>' and runtime_artifact observed identities as the raw identity string. * fix(gitops): emit a runtime drift item for desired-versus-deployed mismatch The projection reported applied_not_deployed with no entry in drift, so the canonical drift list contradicted the runtime facet it travels with. collectRuntimeDrift now emits the r27 generation-mismatch item for that state: desired generation as expected, deployed generation as observed, ComposeService as owner. The action mirrors what availableActions offers: deploy, unless an application-level fetch or apply is in flight or a recovery is in progress, which withhold deploying without removing the fact of the mismatch. The item clears once the desired generation deploys. The stale-deployment test pins the exact item shape, its stability across re-derivation, its convergence removal, and the withheld-action case; the ordering pin keeps the artifact observation suppressed while the deploy question stands. Frontend fixture aligns its artifact-mismatch example with the backend's none action. * fix(gitops): keep generation drift visible across failure and pause states The desired-versus-deployed drift item was keyed to the applied_not_deployed runtime status, but paused, failed, recovering, interrupted, and in-flight statuses all outrank the pointer comparison in the deriver. A failed deploy of a new generation over a running older one therefore dropped the drift report exactly while the old workload was still serving. The item is now judged from the pointers themselves: known and different means reported, whatever presentation status the target carries. Retired targets are excluded because nothing can rebind them, so their surviving pointer divergence would be permanently unresolvable noise. The action follows what the payload offers: deploy only when no application-level operation or recovery withholds it and availableActions lists deploying, none otherwise. The failed-redeploy regression test pins the exact item shape after a pre-mutation deploy failure, its stability across re-derivation, continued artifact suppression for the replaced workload, and convergence removal; a second pin keeps retirement silent. * fix(gitops): complete the available-action legality matrix The action list transcribed only part of the approved rules. An interrupted apply offered apply without checking that its recorded generation was still the current candidate; deploy fired for any target reading applied_not_deployed regardless of target mode, with no retry for an interrupted Direct deploy and none of the Blueprint interruption rules; approve_legacy sat in the type union with no producer, leaving a pending Inline placement review permanently unactionable through the projection; and drift items inherited whatever deploy recommendation any sibling target earned. Actions now follow the matrix. Deploy is decided per target through one predicate shared by deriveActions and the drift items: a paused or failed sibling can no longer inherit another target's legal deploy, a Blueprint-mode divergence never advertises Direct deployment, an interrupted Direct deploy retries against the generation still applied, and an interrupted Blueprint deploy or withdraw repeats only while its recorded intent revision and rollout candidate still equal what the application requires, where an absent pair matches because inline Blueprints carry no candidate until the later-phase producer lands. Interrupted apply requires the recorded generation to still be the current candidate and the source not to have been suspended meanwhile. A reachable Inline placement review now offers approve_legacy. Six tests pin the matrix: sibling isolation across failed and paused targets, the Blueprint mode guard, vacuous and matched Blueprint retry identities plus the superseded case, matching-versus-stale Direct deploy and apply interruptions, and the legacy review action. * fix(gitops): withhold apply retry while the candidate is blocked The interrupted-apply retry checked that the recorded generation still matched the current candidate and that the source was not suspended, but a later classification can also block that candidate, and applyStarted refuses a blocked one outright. The retry is offered only when every precondition the transition enforces still holds. * fix(gitops): prove apply preconditions and limit fetch to Direct The interrupted-apply retry checked identity, suspension, and blockage but never loaded the candidate generation, so apply could be recommended for a row that was missing, owned by another application, or built from a superseded materialization fingerprint; applyStarted refuses all three, and the positive fixture itself described a state no transition would accept. The gate now proves existence, ownership, and fingerprint before offering apply. Its fixture is rebuilt around a transition-legal candidate whose projected action is executed against applyStarted, with negatives for every refusal including suspension. Fetch was offered to any application whose source looked unreconciled, but the approved rules reserve fetch for live Direct applications; a Git-backed Blueprint divergence now advertises nothing of the sort. Controls pin both modes against equivalent source state. * fix(gitops): fail closed when a candidate row is missing or foreign Ordinary candidate_ready checked a fingerprint only when the candidate generation row existed, so a dangling id or one owned by another application fell through to ready and offered an apply that applyStarted would refuse on sight. The source now requires the row to exist under this application before readiness, records a limitation for the anomaly with its operator copy mirrored in the frontend, and reports reconcile-required instead. The Blueprint retry test also pins that a superseded rollout candidate alone, with the intent still matching, suppresses deployment. * fix(gitops): fail accepted-generation derivation closed The acceptance branch compared fingerprint and sha only when the accepted generation row loaded, so a dangling id or one owned by another application fell through and reported application_generation_accepted with no evidence behind it, while also withholding fetch as the recovery. The source now requires the row to exist under this application before any comparison, records an accepted_generation_invalid limitation naming the pointer with its operator copy and inventory entry mirrored on the frontend, and reports reconcile-required instead. Five scenarios pin valid, missing, foreign, fingerprint-mismatched, and sha-mismatched acceptances. * fix(gitops): enforce canonical state in dismiss, reconcile, and pulls - Route dismiss-pending through the canonical dismissed transition so a refusal while an operation is in flight surfaces as 409 instead of clearing staged state behind the model's back - Treat tombstoned targets as authoritative in both reconciler decision surfaces so ticks never redeploy onto a placement the model severed; an explicit deploy revives the target and records the revival delta - Keep application-level rows in node-scoped history pages so proxied hub views do not read as if the application never came into being - Expose repoIdentity and configuredRef on history items to match the server-side identity filters - Stand down when a pull resolves to exactly the live candidate (same commit, source fingerprint, plan verdict); pulls after an acceptance still open a fresh staging generation as the apply target * fix(rbac): classify git-source manifest reads as stack:read GET /stacks/:name/git-source/manifest had no rule in the hub route classifier, so proxied requests were refused with 403 before reaching the remote node even for callers holding stack read access. * fix(gitops): surface an error toast when dismiss is refused as in-flight Dismissing a pending Git update while a fetch or apply is still running now returns 409 OPERATION_IN_FLIGHT, but the frontend handler had no else branch for a non-2xx response, so the refusal was swallowed with no operator feedback at all. Add the same error-toast pattern already used by the sibling apply handler, and order the success toast before its side effects so a downstream failure cannot invert the outcome.	2026-08-27 10:25:27 -04:00
Anso	4c93947004	fix: stop compose-ps spam after deleting a stack (#1831) ... * fix: stop compose-ps spam after deleting a stack A live logs WebSocket kept calling docker compose ps against the removed stack directory every 2s. Missing cwd surfaces as spawn ENOENT, which was logged as Docker CLI unavailable. Skip compose when the dir is gone and idle the log stream instead of retrying. * fix: contain stack directory stats inside the compose root Resolve the stack path against the node's compose directory and refuse paths that escape it before calling fs.stat, matching the existing filesystem containment pattern at other sinks.	2026-08-14 09:24:48 -04:00
Anso	f5178889eb	feat(recovery): complete authored-project atomic rollback generations (#1819) ... * feat(recovery): capture complete authored Compose project for atomic rollback Replace the root-compose-only backup slot with staged recovery generations that record the managed inventory, exact Compose invocation, and prior image identity, and wire the same engine through deploy, update, manual rollback, and Git apply. * fix(recovery): satisfy CodeQL path barriers and update-guard mock Inline resolve+startsWith checks at generation/inventory fs sinks and stub getCurrentStackUpdateRecovery in UpdateGuardService tests. * fix(recovery): drop unused FileSystemService import in generation store test * fix(recovery): harden authored-project rollback for upgrade and restore safety Preserve legacy UUID backup rows, restore Git deploy state with files, make multi-file restore recoverable, evaluate policy on the restored target, and fail closed when Git capture cannot cover an apply. * fix(recovery): unblock Git apply unit tests and CodeQL pre-restore TOCTOU Mock recovery capture in git-source-service tests after fail-closed apply capture, and re-resolve live paths immediately before pre-restore snapshot reads. * fix(recovery): fall back to authored inventory when Git manifesto is missing First Git apply captures before promote, so a missing managed-project manifesto must not block rollback capture when the live stack already has authored files. * fix(recovery): make authored-project rollback atomic across Git state Restore the managed-project manifesto with files, keep nullable Git identity on first-apply captures, persist Git side-state in restore intents for startup reconcile, compensate legacy materialize failures, and refuse directory collisions before mutation. * fix(recovery): satisfy CodeQL path and TOCTOU barriers on manifesto restore Add inline resolve barriers for manifesto read/clear sinks and remove the access-then-read race when restoring a generation manifesto snapshot. * fix(recovery): close third-audit rollback generation blockers Fail closed on incomplete Git inventory fallbacks, execute captured Compose invocation during recovery, refuse startup and mutations while restore intents remain unresolved, propagate legacy stale-delete failures, and add Docker-level exact prior-image coverage plus regression tests. * fix(recovery): mark acquired before handoff in prior-image Docker test Match the production updateStack CAS sequence so the exact prior-image integration test does not fail handoff from the captured phase. * fix(recovery): close fourth-audit rollback safety blockers Evaluate policy against held images, use index-based pre-restore snapshots, hold the shared stack lock across Git apply, replay Mesh and empty captured invocations exactly, restore POSIX modes with fail-closed sensitive permissions, keep case-sensitive paths, and link Git auto-deploy health gates. Add regression coverage for these cases. * test(recovery): fix mocks for health-gate link and authored compose args Add linkGateOrRetain to the Git apply recovery mock, and mock authoredComposeArgs so the case-collision inventory test is not masked by a missing getComposeDir stub. * fix(recovery): close fifth-audit rollback safety blockers Share git_apply locking for webhook auto-apply, fail closed on malformed recovery service records, refuse mixed-image capture, and require exact probe counts with hold-tag eligibility checks. * fix(recovery): close sixth-audit rollback safety blockers Preserve the legacy backup slot during generation capture, encrypt sensitive pre-restore snapshots, revert files on a failed health probe without committing Git, fail closed when an absent-file revert would delete a directory, skip Compose one-offs, route manual and scheduled backup through the current generation, and persist runtime image platform identity. * fix(recovery): close seventh-audit rollback safety blockers Fleet snapshot restore and restore-all now capture a recovery generation under the stack lock before any authored file write, including on remote nodes. * fix(recovery): keep pre-deploy generations during health-gate observe Link deploy recovery generations to the observing gate so backup cannot replace them mid-observe. Distinguish missing hold tags from probe failures, refuse generation release when services metadata is corrupt, classify mixed-replica and coverage refusals, and toast the backend rollback message. * fix(recovery): wrap webhook deploy case for eslint const bindings in an unbraced switch case trip no-case-declarations. Match the pull case block.	2026-08-13 03:48:09 -04:00
Anso	578ce7684d	feat(git): complete-project materialization with a managed-project manifest (#1786) ... * feat(git): add managed-project manifest types and DB cache columns Introduces the canonical managed-project manifest contract types (schema v1) and the stack_git_sources cache columns manifest_version / manifest_state / manifest_generation. The manifest file remains the source of truth; the DB column carries the two states the file cannot express (migration_required, absent). * feat(git): add vendored Docker .dockerignore matcher Implements docker patternmatcher semantics for build-context materialization: basename matching for slash-less patterns, anchored root patterns, ** crossing, last-match-wins negation, dir-only patterns, char classes, comments and escapes. Table-driven tests cover the full rule set. * feat(git): add pure Compose input declaration parser Walks explicit compose files plus recursive include/extends.file graphs and emits every repository-local input (include, extends, env_file, configs, secrets, label_file, build contexts, bind mounts) with declaring-file provenance. Side-effect free: file contents are injected via a read callback. Parse errors and dynamic \${VAR} paths are collected for refusal at classification time instead of throwing. * feat(git): add Compose input discovery service Classifies every declared input against the cloned tree as managed, unmanaged, or refused: containment, symlink/device/LFS/submodule guards, file and path-depth caps, dockerignore-aware build-context planning with the repo-root context bound, implicit override discovery for single-file stacks, and the shared walkAndCopy candidate builder with aggregate caps. * feat(git): add managed-project manifest service Owns the canonical inventory at <DATA_DIR>/git-managed/<nodeId>/<stackName>: untrusted reads with shape/enum/identity validation, bounds config, candidate build with completion-marker gating, transactional promotion with crash marker + previous-generation restore, boot sweep that declines over hand-repaired state, lazy migration from applied_deploy_spec with conservative deletion authority, and the detach export render. * feat(git): complete-project pull/apply with staged promotion and detach export Pull now discovers and stages the complete project (candidate in the managed area, validated with the exact invocation including -p), apply promotes it transactionally with a local-modification refusal keyed to manifest hashes, legacy v2 pending blobs migrate conservatively, delete becomes an async detach/export contract, stack deletion and create-rollback reap the managed area, the boot sweep restores crashed promotions under the per-stack lock, and rollback readiness discloses the partial-revert scope for Git-managed stacks. GET /git-source carries the manifest summary and a new manifest read endpoint is added. * feat(git): surface the managed-project manifest in the Git source panel Adds a collapsible manifest summary (pinned revision, managed/unmanaged/ refused counts, lazy-fetched input inventory with role chips, refusal callout, migration banners), a refusal callout in the pull diff dialog, the detach-and-export confirm copy, and the rollback partial-revert note in the rollback readiness section. * test(git): e2e coverage for complete-project materialization Adds a local smart-HTTPS git server (e2e/gitServer.helper.ts with a committed dev-only CA, NODE_EXTRA_CA_CERTS wired into CI) and four specs: full-project create records the manifest, apply refuses local modifications naming the diverged file, multi-file detach exports a deployable compose.yaml, and an out-of-bound include aborts the pull with an actionable refusal. * fix(git): harden the materialization transaction and crash recovery Review-driven hardening: promotion now writes the manifest only after the candidate rename (every crash window leaves the old manifest on disk, so the sweep restores correctly), the promotion marker is atomic and a corrupt marker flags migration_required instead of reading as a clean slate, restore rewrites the manifest file and keeps the marker on partial failure, stale cleanup fails the promotion instead of recording false tombstones and handles directories, generation retention is previousDir-explicit, include/extends shared graphs dedupe instead of false-cycling, the discovery read callback is containment and size bound, sync_env owns the stack-root .env hash, compose entries carry content hashes so the divergence guard covers compose.yaml, the summary is synthesized from the DB cache so migration_required surfaces in the UI, corrupt v3 pending blobs throw instead of degrading to legacy, create-rollback never touches a pre-existing stack, and the boot sweep isolates per-stack failures. * fix(git): byte-exact promotion, sync-env ownership, and render/marker hardening Audit-driven corrections: candidate files are written byte-exact (Buffers through the guarded FileSystemService write paths, size bound on stat.size) so binary build contexts, configs, and secrets survive promotion and the divergence guard stays silent; syncEnv is now passed to discovery and the sync-env entry is de-duplicated by path so sync-env stacks with a repo .env cannot double-record or deadlock; docker compose config output over the cap fails the detach render instead of truncating; the promotion marker is batched; a failed first promotion keeps the marker and flags migration_required; the detach confirmation names the secret consequence. Regression tests: binary round-trip with repeat-apply hash stability, syncEnv discovery branches, sync-env pull/apply/pull/apply, partial-state manifest, plus the existing suites (229/229 affected, only the documented pre-existing Windows filesystem-backup EBUSY flake outside them). * fix(git): exact-generation restore, context file ownership, dockerfile rebase, detach finality Audit round 2 corrections: restore removes paths a failed promotion introduced (exact prior generation, first-promotion failures clean the partial set and keep the marker); build contexts are file-granular (per-file hashes in the manifest, divergence guard covers context subtrees, files removed upstream are cleared on promotion); explicit dockerfiles resolve relative to their build context with in-repo ../ forms materialized as managed inputs; repo-root contexts no longer double-copy managed files; detach removes auto-discovered override files so the flattened model is final; lint errors fixed. Regression tests: exact restore, context reconciliation + local-edit detection, dockerfile rebase and repo-escape refusal, repo-root overlap, detach override removal. 213/213 affected backend tests. * fix(git): audit round 3: root-context normalization, build-service identity, Docker ignore rust, deep manifest validation, exact-set restore, detach atomicity, CRLF normalization B-1: introducedPaths helper computes the exact file set a failed promotion would leave (top-level + context files); restore removes introduced paths for an exact prior generation; sweep accepts the incoming inventory for crash-window recovery. B-2: repo-root context (build: .) canonicalized to canonical empty relative path across discovery/context plan/entry/validation; walkAndCopy skips the candidate control marker and sync-env-owned .env so root contexts never copy Sencho metadata into the live stack dir. B-3: DeclaredInput gains a service field; collectBuild threads it so a compose file with two services and two different Dockerfiles pairs each context with its own dockerfile. Additional contexts never inherit the service dockerfile. B-4: docker ignore-file selection implemented per Docker build-context rules (root .dockerignore, with Dockerfile-specific <name>.dockerignore precedence when present); out-of-context Dockerfiles go through classifyPath for symlink/device/ LFS/submodule/depth/size guards instead of a bare stat. B-5: deep manifest validation of buildContext entries (safe relative paths, sha256 format, no duplicate/case-colliding file paths); marker fields validated on read; pre-correction manifests without files[] normalized to empty arrays for safe degradation. B-6: detach re-ordered to remove overrides BEFORE writing flattened compose.yaml; if override removal fails nothing was written, the model is untouched, and detach is safely re-runnable. S-1: ComposeService.ts LFs normalized to repository convention. All 213 affected backend tests pass; tsc + lint clean both sides. * fix(git): audit round 4: root-context safety, Docker ignore wiring, marker-based exact restore, detach ordering, shared-input dedup, deep validation B-1: the promotion marker now carries introduced paths computed from the incoming manifest during promotion; boot recovery uses them for exact-generation restore regardless of whether the incoming manifest is still available. `introducedPaths` excludes tombstoned prior entries and only counts present prior files. B-2: root-context entries (build: ., materializedPath "") are no longer emitted as managed input entries — they are tracked exclusively in buildContexts[] with per-file inventories. `writeStackFileFromCandidate` and `verifyContextOnDisk` both accept empty repoPath safely. B-3: Dockerfile-specific .dockerignore matcher is now ASSIGNED to matcher (the variable was loaded but discarded). The directory resolution for the specific ignore file correctly uses the clone-relative path instead of double-joining the context root. B-4: detach now writes the flattened compose.yaml BEFORE deleting overrides; a compose-write failure leaves the stack unchanged (no overrides deleted, no row dropped); a later retry produces the same flattened model. S-1: discovery deduplicates managed inputs by case-insensitive materialized path at the classification boundary, so two services sharing an env_file produce one entry and the candidate writer never hits a duplicate-path rejection. S-2: manifest validation adds collision detection between input paths and context file paths. All 213 affected backend tests pass; tsc clean; lint 0 errors. * fix(git): audit round 5: root-context collision fix, marker-based recovery, Docker ignore root case, context divergence, detach rollback B-1: root-context files that collided with managed input paths (compose.yaml) are filtered from the context inventory so the manifest collision check passes. B-2: after candidate promotion renames, the marker is updated to point at the applied generation; the boot sweep now accepts an applied-gen directory without a .candidate-complete marker as valid for recovery. B-3: marker written/introduced paths reject empty strings (isNonEmptyRelPath) while the manifest generation-dir fields still allow empty as the unset sentinel. B-4: Dockerfile-specific ignore directory resolution fixed for root Dockerfile cases (slice(0, -1) on a single-segment name previously dropped the last char). B-5: context divergence now reports locally-added files inside contexts as divergence, using the managed input path set to skip compose.yaml/.env/files that have a non-context owner. B-6: detach now snapshots the prior compose.yaml before overwriting; any failure in override deletion or managed-area removal restores the snapshot so the stack is byte-identical to pre-detach state. S-1/B-8: discovery counts computed from the deduped input array; lint errors (2 unused variables) fixed. All 213 affected backend tests pass; tsc clean; lint 0 errors. * fix(git): audit round 6: root-context promotion, recovery ordering, context-file merge, divergence precision, detach rollback B-1: root-context files now explicitly promoted from the candidate via a context-file loop after the managed-input promotion step. Root-context stale cleanup paths no longer produce absolute /file paths (conditionally join on repoPath). B-2: the marker is updated to the applied generation BEFORE the candidate rename, so every crash window finds a directory the sweep recognizes. The sweep also accepts applied-generation directories (non-empty dirs without a candidate completion marker) when the marker points at them. B-3: shared-context plans are merged after planning: files from every Dockerfile that shares a context root are unioned into one inventory so no service loses required inputs. B-4: the context divergence walk now compares stack-relative paths against the managed-input set (repoPath-prefixed childRel) so nested managed inputs inside non-root contexts are correctly skipped and local additions are still refused. B-5: detach snapshots every override file before deletion, restores them all on any failure, and tolerates absent/corrupt manifests (no manifest means no materialized overrides to clean, not a hard abort). All 213 affected backend tests pass; tsc clean; lint 0 errors. * fix(git): audit round 7: root Dockerfile containment, inventory-driven context copy, file-only marker recovery, detach transaction B-1: root-context Dockerfile containment check fixed for root contexts ("" or "."). Any repo-relative Dockerfile without ../ is in-context. B-2: context copy now reads from the plan inventory (plan.context.files) instead of re-walking the source with the first matcher. Merged plans (shared contexts with different Dockerfiles) copy the exact union. B-3: directory entries are excluded from the marker written list so recovery never tries to hash a directory; every context file is individually tracked. Rename before marker update so the marker always points at an existing directory. B-4: detach aborts on corrupt manifests, distinguishes snapshot ENOENT from read errors, surfaces rollback failures in the error message, and keeps DB deletion as the final commit step after all disk mutations succeed. All 213 affected backend tests pass; tsc clean; lint 0 errors. * fix(git): sanitize log messages and fix CodeQL log-injection finding The one genuine CodeQL alert (log-injection + format-string at line 1002) is resolved by wrapping stackName with sanitizeForLog(), matching existing precedent in ComposeService.ts and routes/stacks.ts. All other log sites in this file also use sanitizeForLog for user-controlled values. * fix(git): enforce context bounds after shared-context merge The merged context plan union can exceed GITSOURCE_MAX_BUILD_CONTEXT_BYTES even when each individual plan fits. Recheck the cap against the unionized inventory after merging. * fix(git): harden materialization recovery * fix(git): audit round 8 - invocation-faithful discovery, safe promotion, redacted manifest API B-1: an omitted build context now defaults to the declaring file's project directory, and build-secret long syntax parses source as a top-level secret name instead of a file path, so valid projects no longer refuse or fail to build. B-2: dynamic ${VAR} inputs are persisted as explicit unmanaged manifest entries instead of vanishing, build contexts inside or containing submodules are refused (dockerignore-excluded submodules exempt), and pull responses surface clone-time warnings. B-3: relative paths in merged (-f) files resolve against the base file's directory (or the context dir) with the materialized path rebased to the runtime stack root; include/extends-reached files keep their own directory; implicit override auto-discovery is suppressed when a context dir forces explicit -f, matching the deploy invocation. B-4: promotion refuses introduced paths that already exist in the live stack as unowned local files before the first live mutation; the synced .env and fresh-stack creation stay exempt. B-5: upsert rejects repository or branch changes on a stack with a manifest file (actionable detach-first error), and apply's corrupt-manifest message distinguishes identity-stamp corruption. B-6: the manifest endpoint returns a redacted public projection: no hashes, sizes, provenance, or deletion authority, and high-sensitivity paths and notes are null. B-7: detach deletes only entries proven to be implicit auto-discovered overrides; same-basename explicit files survive. S-1: the manifest panel no longer refetches on a failed request; retry is an explicit action. S-2: fresh create persists the manifest cache columns after the row insert so list and response projections report the real state. S-3: git-sources.mdx matches the corrected detach, submodule, and dynamic-path behavior. * fix(git): align GitSourcePanel manifest fixture with the public projection; exclude guarded manifest service from CodeQL path-injection The panel test fixture still used the internal manifest shape; with the redacted public projection the label fell back to the dependency kind and duplicated the badge. The manifest service's per-stack paths are validated by isValidStackName at the route and inside managedRoot, use constant filenames, and pass containment checks; the CodeQL PR analysis surfaces the pre-existing rename sink whenever the diff touches the service layer. * fix(git): restore ComposeService.ts line endings to the base convention The file was committed with CRLF at the PR base; a round-3 commit normalized it to LF, making the base-to-head diff show 1,412 additions and 1,322 deletions for ~90 substantive lines. Restoring CRLF collapses the diff to the functional changes only. * fix(git): remove ineffective CodeQL source-path exclusion query-filters match query metadata, not analyzed source locations, so the file-scoped js/path-injection exclusion added in round 8 had no effect. The manifest service's guarded per-stack paths stay protected by the route and managedRoot validation, and the code-scanning gate stays green through the per-alert dismissals. * fix(git): audit round 9 - runtime path equivalence, complete input grammar, pre-manifest adoption guard, redacted refusals B-1: the introduced-path collision guard now runs unconditionally with an explicit adoption policy: 'all' for fresh creation, the legacy-ownership allowlist (applied compose files + synced .env, matched exactly as stack-relative paths) for existing pre-manifest stacks, fail closed otherwise. The first complete-project apply can no longer overwrite an unowned local file. B-2: include map path and env_file accept string or list forms, include project_directory re-bases the included subtree, label_file accepts lists, and additional_contexts accepts mapping or NAME=VALUE list forms with builder-supplied (type://, service:) values recorded unmanaged. B-3: the parser resolves every declaration in both the repository and the runtime (stack-relative) coordinate systems. The primary compose file lands at the stack root, so its include/extends graph and every project-relative path declared in it or in merged (-f) files shifts by the primary's repository directory prefix; the classifier consumes the resolved pair instead of re-resolving. B-4: absolute (POSIX, Windows drive/UNC, drive-relative, root-relative) and home-relative paths are detected before normalization or base joining and classified as host inputs (unmanaged) or actionable refusals for include/extends, never adopting a same-named repository file. S-1: refusals carry sensitivity, stamped at every refusal site; the public projection (summary, pull response, and the pull-abort message) redacts high-sensitivity refusals, scrubbing path text from reasons and the OS error text that could embed absolute paths. Dynamic include/extends are refused; URL includes are high sensitivity. S-3: ComposeService.ts line endings restored (separate commit). S-2: invalid CodeQL source-path filter removed (separate commit). * chore: bump nanoid to 3.3.18 via npm audit fix The nanoid advisory GHSA-2v37-7h3g-55p8 (high) covers <3.3.17 and was published after the last green CI run; both lockfiles pinned 3.3.16. npm audit fix bumps the transitive dependency to 3.3.18. * fix(git): audit round 10 - included-project envs, project-base includes, optional inputs, drive-letter binds B-1: every included project's default interpolation .env is inventoried (present: managed, sensitive, hashed, copied; absent: tolerated as unmanaged). interpolation: false and same-base includes skip the entry. B-2: include, include-env, and extends.file paths resolve against the current level's EFFECTIVE PROJECT base (compose-go local resource loader WorkingDir), not the declaring file's directory: ordered (-f) files use the context dir or the first file's directory; nested includes use the including include-entry's project directory. Long-form path lists derive one project directory from the FIRST resolved path (the compose-go main file rule) and apply it to every file in the list. Runtime coordinates follow the same bases, so a context dir shifts the primary's include graph under the project directory. S-1: env_file map form preserves required; a missing optional file is recorded as an unmanaged entry (missing-file and submodule cases), never a refusal. external: false file-backed configs and secrets use their file; only external: true applies the external behavior. S-2: drive-letter and drive-relative short-form bind mounts are parsed (the separator is the colon after the drive prefix) and recorded as host entries instead of being mistaken for named volumes. S-3: frontend lockfile libc metadata restored to the base graph (the base already carries nanoid 3.3.18). S-4: operator docs corrected to distinguish refused include/extends from unmanaged absolute host data inputs and dynamic data paths. * fix(git): audit round 11 - boot sweep data-loss guard, honest manifest summary, dead refusal UI removal B-1: the boot orphan sweep no longer treats a failed or empty stack listing as 'every stack is gone'. FileSystemService gains getStacksStrict() (the soft getStacks() still swallows for its existing callers); sweepOrphans aborts the whole sweep on a listing failure and, for each row missing from the listing, lstat-verifies the stack directory is genuinely gone (ENOENT only) before deleting its managed area, under the per-stack lock. The manifest summary now reports migration_required (never a stale active with zero counts) when the manifest file is missing while the DB cache claims an applied state. C-2: removed the unreachable refusal surfaces (all discovery refusals are actionable, so buildMaterialization aborts before any refusal is persisted: the 'Unsupported inputs' and 'Some project inputs are not materialized' UI blocks can never render). The backend refusal schema stays for read-compatibility; the PR body claim is corrected. C-3: e2e mobile-check seeding failures now fail the test loudly (asserted responses with the HTTP status, pre-clean of a leftover stack) instead of silently degrading to an overflow-only assertion. * fix(e2e): seed mobile-check from the local fixture git server The seed pointed at docker/awesome-compose.git with compose_paths ['compose.yaml'], but that repository has no root compose.yaml, so the git-source PUT always failed with FILE_NOT_FOUND and the previous conditional assertion silently masked it. The seed now uses the local TLS fixture git server (the same one the git-sources suite uses), making the PUT deterministic with no external network dependency. * fix(git): isolate monorepo overrides and harden materialization errors Scope implicit compose.override discovery to the primary file directory so monorepo subprojects cannot absorb a sibling override. Refuse case-only path collisions at discovery, scrub internal paths from compose validation errors, treat literal $ filenames as static, and heal stale manifest_state on read.	2026-08-10 17:12:55 -04:00
Anso	92d974b13e	fix: condition --volumes in downStack() on the removeVolumes option (#1764) ... * fix: condition --volumes in downStack() on the removeVolumes option ComposeService.downStack() hardcoded --volumes on every stack delete, ignoring the "Also remove associated volumes" checkbox and destroying volumes the operator asked to keep. The sibling Take-down path (runDown) already conditions --volumes correctly. - Add options?: { removeVolumes?: boolean } to downStack() - Default to data-preserving (no --volumes when option absent) - DeletedStackDeletionService reads the persisted intent flag - Templates rollback passes removeVolumes: true (clean up failed deploy) - Blueprint withdraw passes removeVolumes: false (volumes preserved) * docs: update Delete row to reflect conditional volume removal The Delete row now describes that volumes are removed only when the operator opts in, matching the behavior introduced by the downStack fix. * fix: add capability gate for delete pruneVolumes and fix QA findings Four P0 issues found in live QA: P0-1/P0-4 - No capability gate on delete's pruneVolumes: Add stack-delete-prune-volumes capability so the frontend hides the "Also remove associated volumes" checkbox on nodes that don't support conditional volume removal on delete. Without this, an operator on an old node sees a VOLUMES KEPT promise the old node silently breaks. Frontend-only gate: no API or proxy gate because the old node's fallback (always destroy) is correct for the checked case. P0-2 - Checkbox state leaked across dialogs: Reset pruneVolumes in onConfirm before calling the parent, so a previously checked box doesn't appear pre-checked when the dialog opens for a different stack. P0-3 - Delete not bound to the active node: Capture activeNode.id at delete time and pass it as an explicit nodeId to apiFetch, matching the Take Down pattern. Without this, switching the active node while the dialog is open silently deletes the wrong stack on the wrong node. * fix: update test assertions for nodeId binding and showVolumeOption gate P0-3 added nodeId to apiFetch DELETE calls — two useStackActions tests now expect the parameter. P0-1 gated the volume checkbox behind showVolumeOption — the confirming test now passes the prop. * fix: gate volume hint on showVolumeOption to prevent false promise On nodes without stack-delete-prune-volumes, volumes are always destroyed. Showing VOLUMES KEPT was a lie. Now the hint is hidden entirely when the capability is absent. * fix: gate delete against nodes that cannot guarantee volume preservation Hiding the checkbox and the misleading hint stopped the false promise but not the data loss: an unchecked delete against a node lacking stack-delete-prune-volumes still reached that node and its downStack() still destroyed volumes unconditionally, now with no warning at all. - remoteNodeProxy.ts: block an unacknowledged DELETE /stacks/:name (no pruneVolumes=true) to a remote lacking the capability, mirroring the existing removeVolumes gate on the down route. An explicit pruneVolumes=true always proxies through since that matches what an unsupported remote does anyway. - DeleteStackDialog: rework around a three-state model (supported / unsupported / unknown) instead of a boolean. A node whose capabilities have not been confirmed (meta not yet fetched, or a failed probe) is now treated like a supported node, not forced onto the destructive path just because its state is unresolved. - Fix deleteStack's error toast, which surfaced the raw JSON response body instead of the parsed error message. - Fix CreateStackDialog's orphan-stack rollback (docker-run import), which silently no-op'd against a node requiring acknowledgement. - Update node-compatibility.mdx and stack-management.mdx to describe the new gate. * test: advertise stack-delete-prune-volumes on the scoped-evidence fixtures These mock remotes simulate nodes capable enough to run scoped-stack-auth-evidence RBAC and were pinned before stack-delete-prune-volumes existed, so the new delete gate now blocked their unacknowledged DELETE calls before reaching the mock server, failing the grant-tuple-cleanup assertions the tests actually check.	2026-08-04 13:04:21 -04:00
Anso	41bf075eb0	feat(recovery): make rollback-recovery image lifecycle visible and controllable (#1753) ... * feat(recovery): make rollback-recovery image lifecycle visible and controllable GitHub discussion #1751 asked why Sencho creates sencho-rb/<id>/<service>:hold images during automatic updates and how to clean them up. That surfaced a real safety bug alongside the missing visibility: the manual single-image delete route did not consult the held-image predicate every other deletion path already honors, so a user could delete a rollback-protected image straight through the Images tab and silently break automatic recovery for that update. A short/truncated id also bypassed the predicate's full-id lookup. Fixes: - POST /images/delete now resolves the submitted id to its canonical form and checks the unified held-image predicate before deleting, returning 409 IMAGE_HELD_FOR_ROLLBACK for a protected image. - The Images tab no longer mislabels a protected image as plain "Unused"; a fully-synthetic hold image is kept out of the generic inventory entirely and surfaced instead in a new Resources -> Rollback tab, with an additive "Rollback protected" badge for images that still carry a normal tag too. New capability: - Two settings (Deploy Guardrails): superseded-generation retention (days, replaces a hardcoded 7) and a cap on retained generations per stack. - A new Resources -> Rollback tab lists every generation (stack, short id, state, retention) with an admin-gated manual release action, including releasing the current generation with an explicit warning that automatic rollback becomes unavailable until the next successful update. Release is a single atomic, server-revalidated transition so a stale UI read can never release a row that has since become ineligible. Also consolidated three near-duplicate implementations of the held-image predicate (two of which relied on a require() of a sibling .ts file that silently failed to resolve under the test runner and was never actually exercised by a real test before this change) into one shared module. Known follow-up, not fixed here: an orphaned sencho-rb tag whose recovery row no longer exists (DB restore, node re-add) is invisible in both the Images and Rollback tabs with no UI path to reclaim it. * fix(audit): add summary mapping for rollback generation release * fix(security): sanitize prune target in log sinks and cover release RBAC Closes two open js/log-injection findings on the system prune route by applying the same inline sanitizeForLog barrier the rest of the file already uses. The prune target is validated against an enum by parsePruneTargets before reaching these sinks, so the findings were false positives, but the barrier is cheap and removes the standing alerts on a file this change already touches. Also wraps the generation id in the release log line for consistency with the stack name beside it. Adds coverage for gaps a QA pass identified: - Release endpoint refuses a viewer and a deployer (Admin-only), leaving the generation and its artifacts untouched. - Viewer can still read the generations list, matching the sibling Resources routes. - The predicate the prune routes build reports full-stack rollback holds, not just service-scoped ones, and re-reads per call so a hold taken between plan and delete still gates the delete. - After releasing the current generation, no rollback point is claimed for the stack through any consumer of the current-generation lookup.	2026-08-02 21:55:22 -04:00
Anso	3f1f15a6f4	fix: keep running containers until stack pull/build succeeds (#1657) ... * fix: keep running containers until stack pull/build succeeds Acquire images before reconcile, capture a recovery generation for compensation, and only remove classified orphans after handoff. * fix: address recovery audit blockers for safe stack updates Retire abandoned and expired recovery artifacts, probe compensated runtimes before reporting rollback success, preserve local Docker when deleting a node, validate the exact Compose invocation before capture, and repair updateStack return-contract fixtures. * fix: resolve ESLint errors blocking CI on this branch Unused-import and unused-variable errors left over from the stack deletion refactor: MeshService in stacks.ts (its opt-out cascade moved into DeployedStackDeletionService), a redundant pruneVolumes destructure in deleteDeployedStack (the real one is re-derived from the same input object inside runDeletionBody), and an unused beforeAll import in a Docker-integration test stub. Also scopes the webhook pull-action case body in a block to satisfy no-case-declarations; purely syntactic, no behavior change. * fix: harden recovery probe, cleanup retry, and failed-pull Docker test Reject absent or unhealthy expected replicas before reporting rollback success, keep cleanup records until artifacts are actually removed, fail closed when a mesh override cannot be generated, and assert a real failed pull leaves the original container running. * fix: verify recovery probe image identity and stack-scoped override paths Reject recovered runtimes that use the wrong image or leave scale-zero services running, and confine tombstone override deletion to the intent stack directory so forged cross-stack paths cannot be swept. * test: batch notification cap fixtures in a SQLite transaction Unbatched 1200-row inserts were timing out at the default 30s under CI load even though the same assertions pass in under 2s when green.	2026-07-21 12:18:01 -04:00
Anso	63213c0960	feat: add service-scoped Compose update and restore (#1648) ... * feat: add service-scoped Compose update and restore Allow updating or rebuilding one declared Compose service on multi-service stacks without recreating siblings, with recovery snapshots, health-gate observation, and prune holds for rollback images. Full-stack update paths and single-service UX stay unchanged. * fix: sanitize service-scoped update log messages for CodeQL * fix: address service-scoped update audit findings B-01 through B-07 * fix: complete service-scoped update audit metadata and surfaces * test: wrap Updates readiness tests for deploy-feedback context * fix: keep service recovery reachable without Deploy Progress Make failed service-gate recovery discoverable when Deploy Progress is disabled or dismissed, suppress stale image-scan notification side effects, normalize ComposeService line endings, and add focused regression coverage. * fix: resurface ContainersHealth density and expand on multi-service stacks Service grouping hid the summary strip and Compact/Detailed/Expand controls that still applied to multi-container stacks.	2026-07-19 02:42:29 -04:00
Anso	35bb74425b	feat: guide missing external network creation during deploy (#1645) ... * feat: guide missing external network creation during deploy Detect missing external networks before Compose runs, prompt or auto-create safe bridge networks, and keep unsupported declarations blocked with trusted deploy provenance. * test: align deploy context and settings fixtures with missing-network gate Update caller spies, EffResource expectations, StacksSection save keys, and git-source spy cleanup so CI matches the new deployStack context and auto-create setting. * fix: drop unused renderError binding in missing-network resolver Satisfies no-unused-vars so backend ESLint CI passes; callers already key only on model presence. * fix: use HTTP-safe clipboard helper in missing-network dialog navigator.clipboard fails on plain HTTP LAN hosts; route copy actions through copyToClipboard so Docker and Compose copy buttons work on self-hosted instances. * fix: simplify missing-network dialog actions and copy label Drop the Compose snippet escape hatch, move secondary actions under More, and rename the terminal copy action to Copy create command so the footer is a clear Cancel / Create decision.	2026-07-17 14:51:00 -04:00
Anso	d113004359	feat: add confirmed Take down stack action with optional volume removal (#1599) ... * feat: add confirmed Take down stack action with optional volume removal Expose Take down in the stack header and sidebar with a confirmation dialog that runs compose down while keeping the stack definition on disk. Optional volume removal is gated by node capability and stack:deploy permission, with remote gateway preflight before proxying removeVolumes requests. Closes #1582 * fix: reset take-down volume checkbox when dialog closes * test: align getStackMenuVisibility assertions with showTakeDown key getStackMenuVisibility now returns a fifth lifecycle flag, showTakeDown, but three exhaustive toEqual assertions still listed only the prior four keys and failed. Add the expected showTakeDown value to each: true for the partial and exited running-stack cases, false for the self stack. * test: cover Take down visibility for running non-self stacks The getStackMenuVisibility assertions exercised the partial and exited branches and the self-stack guard, but not the raw === 'running' literal that drives showTakeDown for a normal running stack. Add a case so a regression dropping 'running' from that check is caught. * fix: drop Take down from header overflow and wire activity shortcut Remove duplicate Take down from More actions. Keep inline button when running, sidebar menu, and Cmd+ArrowDown. Record stack_taken_down in activity on successful POST /down.	2026-07-09 12:20:13 -04:00
Anso	b3bc223c45	fix(deploy): selective compose recreate on Save and Deploy (#1565) (#1568) ... * feat: move core Blueprint orchestration to Community tier Blueprints CRUD, reconciliation, and drift modes are now available on Community. Pin remains Admiral-only via Federation placement controls. * test: update NodeCard cordon tests for Admiral-only tier gate Cordon now requires both isPaid and node:manage permission, matching the backend requirePaid + requirePermission guard. Three tests still used isPaid:false but expected the menu to be visible.	2026-07-05 21:55:32 -04:00
Anso	f2b5c68d84	feat: add build-aware compose stack updates (#1561) ... Detect services with build: in the update preview and run compose build --pull plus pull --ignore-buildable when Update is triggered on those stacks, while keeping the existing pull-only path for image-only stacks.	2026-07-05 04:45:45 -04:00
Anso	41dc339c26	fix: enforce 1:1 compose path mapping for Pilot agent mounts (#1516) ... * fix: enforce 1:1 compose path mapping for Pilot agent mounts Pilot enrollment now generates validated 1:1 bind mounts so every agent path maps to a unique compose directory. Persisted agent paths reconcile during startup to catch drift. Unsafe relative-bind redeploys are blocked before container removal to prevent path escapes. - Add composePathMapping utility with strict path validation - Generate COMPOSE_DIR and validated mounts during Pilot enrollment - Reconcile persisted agent paths during startup bootstrap - Block redeploy when a relative-bind mount would escape the compose root - Default Pilot UI path to /opt/docker/sencho - Update multi-node and pilot-agent documentation - Add regression tests for enrollment, bootstrap, compose-service, and environment-check paths * fix: update E2E enrollment regexes for YAML-quoted token values	2026-06-29 14:35:50 -04:00
Anso	f1f64ec7f6	feat: show container name in structured log output (#1452) ... * feat: show container name in structured log output Prepend a normalized container name prefix to each line in ComposeService.streamLogs() so both the structured log viewer and the raw terminal identify which container produced each entry. - Backend: prepend displayName (normalized via normalizeContainerName) before LogFormatter.process() in sendOutput and flushBuffer. - LogFormatter: refactor process() to handle both prefix-first and timestamp-first input orders via a while-loop; widen PREFIX_REGEX to accept dotted service names. - Frontend: add containerName to LogRow, extract prefix in parseLine, render as an inline mono chip in the message column, and include the name in downloaded logs (omitting the bracket prefix when null). - Tests: 14 new tests across log-formatter, compose-service streamLogs, and StructuredLogViewer chip rendering + download formatting. * fix: guard LogFormatter loop to at most one prefix and one timestamp The while-loop refactored for order-agnostic prefix/timestamp parsing could continue matching beyond the intended single prefix and timestamp. A log line like "redis | 2024-...Z api | started" would falsely colorize "api |" as a second container prefix in raw terminal output. Add prefixFound/timestampFound boolean guards so the loop stops after one prefix and one timestamp, regardless of input order. * feat: per-service color alternation for log container chips Add an Appearance setting that lets users switch between unified cyan and per-service label-token colors for the container name chips in the structured log viewer. - Extract HUE_VARS and hashLabel() from NodeLabelPill into a shared utility at frontend/src/lib/label-colors.ts. - Add useLogChipColorMode hook (browser-local localStorage, sencho.log-chip-color-mode key, unified by default). - Add SegmentedControl in Settings > Appearance > Display. - Apply inline label-token styles via style attribute in per-service mode; keep current text-brand/80 bg-brand/10 classes in unified mode. - 14 new tests across label-colors, hook, and viewer chip rendering.	2026-06-25 14:13:38 -04:00
Anso	3a22f59057	feat(security): surface Compose internet-reachability exposure in posture (#1442) ... * feat(security): surface Compose internet-reachability exposure in posture Builds a per-stack per-service exposure descriptor from the rendered effective Compose model, cached at deploy/update time, and joins it into the Security action posture. A service is publicly exposed when it publishes a port on a non-loopback host IP or uses host networking. The exposure cache lives in a new stack_exposure table, refreshed inside ComposeService.deployStack and updateStack (covering all funneled paths: manual, scheduler, mesh, templates, labels, App Store, Git, webhooks). Cleanup runs on stack delete, blueprint withdrawal, and node delete. The overview route intersects the exposed image set with the existing per-image suppression-aware Critical/High tally, so a clean public nginx does not escalate posture. The scan sheet shows a "Published service" or "Internal only" evidence badge per image. * fix(test): provide fresh auto-close proc for exposure spawn in stall tests Two deployStack idle-stall tests used mockSpawn.mockReturnValue(proc) which returned the same already-closed process for the new config spawn added by the exposure refresh. The renderConfig promise hung waiting for a close event that had already fired. The fix uses mockImplementation to return the controlled proc for the first spawn (up) and a fresh auto-closing proc for the second spawn (config via refreshExposureCache). * fix(security): tighten loopback detection, clarify exposure semantics, drop internal-only badge - Expand isLoopback to cover full 127.0.0.0/8 range (127.0.0.2 etc) - Clarify that exposure is configured (Compose model), not live topology - Remove "Internal only" badge: false is not proof of non-exposure when other stacks using the same image may lack a cached descriptor	2026-06-24 23:22:13 -04:00
Anso	b753d2d5e0	fix(deploy): preserve compose.override.yml when Mesh is enabled (#1420) ... When a single-file stack is opted into Sencho Mesh, the deploy builds an explicit `docker compose -f <base> -f <mesh override>` list. Passing any explicit -f disables Compose's automatic discovery of compose.override.yml (and the docker-compose.override variants), so a user's hand-authored override was silently dropped from the effective deploy once Mesh was on. Resolve the user's override file (first existing variant, with the same stack-name and symlink-containment guards as the base compose file) and insert it between the base and the mesh override, so it layers exactly as Compose's implicit discovery would, with the mesh override still taking precedence. A transient read failure during the lookup degrades to "no override" rather than failing the deploy; a stack-name or containment-guard rejection still aborts. Multi-file Git-source stacks and non-mesh deploys are unaffected.	2026-06-23 10:44:49 -04:00
Anso	f9c6c5fd09	fix(drift): reconcile the drift ledger on deploy and timestamp its history (#1405) ... * fix(drift): reconcile the drift ledger on deploy and timestamp its history The drift ledger (persisted history + activity timeline) only advanced when someone clicked re-check on a stack's Drift tab, so the history could sit indefinitely out of sync with the live status: a stack reading "drifted" live while its history still said "resolved". Two corrections: - Deploy and update reconcile the ledger against the just-deployed runtime (the rollback route re-deploys through deployStack, so it is covered), resolving what the change fixed and recording what it left. - Every authoritative reconcile stamps the dossier last-checked time, and the Drift tab labels its history "checked {time}" so a stale finding reads as history, not a claim about the live status above it. Adds the last_drift_check_at column and tests across the ledger reconcile stamp, reconcileStack, the deploy hook, and the panel. * fix(drift): stamp last-checked inside the ledger transaction Move the dossier last-checked stamp into the same transaction as the finding insert/resolve, so the "checked {time}" the Drift tab shows can never persist without the ledger update it describes. The stamp still runs on a no-op authoritative check (a transaction that only stamps), keeping the history "as of" honest. Adds a test that a failed deploy does not reconcile the ledger.	2026-06-21 18:20:57 -04:00
Anso	57a0856ffc	feat(stacks): per-stack environment inventory and secret-safe guardrails (#1397) ... * feat(stacks): per-stack environment inventory and secret-safe guardrails Add an Environment tab to Stack Anatomy that derives a per-stack inventory of environment variables from the compose files and env files. Each variable shows its source, whether Compose interpolates it or injects it into a container, and a status (present, missing, unused, duplicate, or shell-only), plus likely-secret classification. The inventory works from variable names only: a value is never read, returned, or logged, and a likely secret shows presence only. A copy env checklist action exports names and status without values. Surface a missing required env_file as a Compose Doctor preflight finding, and add an opt-in node setting that refuses a deploy or update when a required ${VAR:?...} variable is unset or empty, before any backup, pull, or up runs. Default off. The Environment tab is capability-gated so it hides on older remote nodes. * fix(stacks): harden env-file reader against a stat-then-open race Open the env-file handle first and fstat the open handle instead of stat-ing the path before opening, removing the check-then-use window in readEnvFileKeys. Use a secure mkdtemp directory for the out-of-base test path instead of a predictable name in the temp root. * fix(stacks): resolve nested env_file paths per compose file, reconcile inline keys per service Resolve each env_file relative to the directory of the compose file that declared it, so a nested multi-file Git override (infra/prod.yml referencing ./prod.env) lands next to that file instead of the stack root. The root compose file is unaffected, since its directory is the stack directory. Reconcile inline environment provenance per service, so a key an override removed from one service's effective env is not labeled compose-inline just because another service injects the same name from a different source.	2026-06-20 11:58:42 -04:00
Anso	ba09e6f69e	fix: base Git multi-file Compose deploy env and dossier on the effective config (#1391) ... * fix: resolve the root .env at deploy and render time for Git context-dir stacks A Git multi-file source with a context dir set --project-directory to that dir, so Docker Compose looked for .env there and missed the root .env Sencho writes. Validation already passed the root .env with --env-file, so a stack could validate with one effective config but deploy or render another. Add authoredComposeEnvFileArgs, which appends --env-file <stackDir>/.env when the applied deploy spec has a context dir and a root .env exists, and wire it into the deploy/update, image-scan, render, and container-listing compose invocations so they all resolve env from the same file the validator used. A non-ENOENT access error surfaces instead of silently dropping the flag. * fix: base multi-file Git dossier and doc-drift on the effective Compose model The Stack Dossier and its documentation-drift check parsed only the stored root compose file. For a multi-file Git source, services, ports, networks, or volumes that an override file adds were invisible, so the dossier showed incomplete facts and doc-drift could falsely warn that a documented port is unpublished when an override actually publishes it. Add a secret-safe GET /stacks/:name/effective-anatomy that renders the merged effective model and extracts only structural facts (services, ports, volumes, networks, restart), never env, label, or command values. StackAnatomyPanel fetches it for multi-file Git stacks and feeds those facts into the dossier and doc-drift, falling back to the root-only parse for single-file or non-git stacks and whenever the render is unavailable. * fix: add an inline path-injection barrier to the Git env-file resolver CodeQL js/path-injection flagged the fs.access in authoredComposeEnvFileArgs because the env path derives from the route-supplied stack name and the only containment check lived in the callers, not at the sink. Resolve the stack dir against the compose base and assert containment with startsWith inline, then derive the .env path from the validated dir, mirroring the existing inline guards in renderConfig and validateCompose. Valid stack names are unaffected; a name that escapes the base now yields no --env-file. * test: stabilize the dossier doc-drift e2e against the dossier-load race The first assertion filled the access_urls field as soon as the Dossier panel was visible, but the panel's GET /stacks/:name/dossier resolves by overwriting the fields from the server (empty access_urls) and only then flips the doc-drift gate on. When the GET landed after the fill, it clobbered the typed value and the warning never rendered, so the test failed intermittently under CI timing. Wait for that GET to land before typing, mirroring the spec's openStack helper.	2026-06-18 18:25:58 -04:00
Anso	5f1baa7522	fix: harden deploy/update concurrency and node-targeting safety (#1390) ... * fix: harden deploy/update concurrency and node-targeting safety Release stabilization for deploy/update operational safety. Per-stack operation locking is now global. Background lifecycle paths (scheduler auto stop/down/start/backup/update, webhook execute, Git source auto-deploy, image auto-update, label bulk actions, fleet snapshot redeploy, and mesh redeploy) acquire the per-node, per-stack lock through a new StackOpLockService.runExclusive helper and skip rather than race a manual deploy/update/rollback/backup on the same stack and node. Skips surface honestly (a failed scheduled run, a recorded webhook failure, a per-stack batch result, or a thrown error) instead of a silent no-op. Update readiness and policy-bypass now run against the node captured when the dialog opened, not the live active node, so switching nodes while a dialog is open cannot retarget the update or the bypass retry. Rollback readiness no longer presents a moving-tag or unpinned image as a ready image revert. Restoring files does not revert a moving tag, so those stacks read as partial, and the rollback success message states that the compose and env files were restored. * fix: lock blueprint reconcile against manual ops and correct rollback wording Follow-up to the deploy/update safety hardening, closing two more gaps from a verification pass. BlueprintService.deployLocal and withdrawLocal called ComposeService directly, so blueprint reconciliation could race a manual deploy/update/rollback/backup on an owned stack. Both now run their compose lifecycle call through StackOpLockService.runExclusive and skip (recorded as a failed reconcile, retried on the next cycle) on conflict. The withdraw holds the lock across both the compose down and the directory delete so neither races a manual operation. The runtime rollback messages overstated recovery: a rollback restores the compose and env files and recreates containers, but does not revert an image behind a moving tag. The auto-rollback deploy-progress output, the recovery panel and chip, the failure toasts, and the manual rollback route message now state that the compose and env files were restored, with the matching OpenAPI example and atomic-deployments doc updated. * fix: acquire stack lock before blueprint deploy mutates compose and marker files Local blueprint deploy wrote the compose and marker files and ran the policy assert before acquiring the per-stack lock; the lock only wrapped the deploy itself. A reconcile could therefore rewrite an owned stack's files while a manual deploy/update/rollback/backup was running. The lock now wraps the whole critical section (create, write compose, write marker, policy assert, deploy), so on conflict nothing is written and the reconcile records a failed outcome. Adds a test asserting a deploy under a held lock records failed, writes no marker file, and leaves the manual lock untouched. * fix: make remote blueprint apply atomic under the receiving node's stack lock Remote blueprint deploy wrote the compose and marker files to the target node via separate HTTP calls and only locked on the final deploy, so the file writes could race a manual operation on that node. A node's operation lock is process-local and cannot be held by the hub across HTTP calls, so the locked create/write/deploy now runs on the receiving node. The locked critical section is extracted into BlueprintService.applyLocalUnderLock and exposed via POST /api/blueprints/apply-local. The hub posts the blueprint to that endpoint in one call; the receiving node runs create + write compose+marker + deploy under its own per-stack lock. Older nodes without the route answer 404 and fall back to the legacy multi-call flow. The endpoint is gated by paid tier and the same per-stack stack:edit and stack:deploy permissions as the PUT-compose + deploy it bundles, validates the stack name, compose size, and marker structure, and returns 409 on a lock conflict without writing anything. Adds tests for the atomic single-call path, the 404 legacy fallback, the 409 lock-conflict mapping, the route validation and permission paths, and the write-compose-then-marker-then-deploy ordering of the shared locked apply. * fix(deps): bump undici to 7.28.0 to clear high-severity advisory The frontend CI npm audit gate (--audit-level=high) failed on a transitive undici 7.25.0 (a dev-only dependency via jsdom): TLS certificate validation bypass (GHSA-vmh5-mc38-953g) and cross-user cache information disclosure (GHSA-pr7r-676h-xcf6). Bumping undici within jsdom's existing ^7.25.0 range to 7.28.0 clears the high-severity advisory and unblocks the frontend job. Lockfile only; no direct dependency or source change.	2026-06-18 13:38:19 -04:00
Anso	f23b7e1bac	feat: ordered multi-file Compose for Git sources (#1380) ... * feat: ordered multi-file Compose for Git sources Extend Git sources to deploy an ordered list of compose files merged with docker compose -f base.yaml -f override.yaml ..., plus an optional project directory. - Pick and reorder compose files from the repository tree (drag to reorder on desktop, up/down arrows on phones); manual path entry is also supported. - The ordered set drives every stack-scoped compose command (deploy, update, start/stop/restart/down, image scans, Compose Doctor) and the container lookup, so a service or image declared only in an override is handled too. - Runtime keys off the materialized set, not the saved configuration: saving a source does not change deploy args until the pull is applied, and apply materializes from the pending snapshot rather than live config. - The project directory is passed as --project-directory, with -p <stack> pinning the Compose project so container labels stay stable. - The Mesh override is layered last; single-file sources are byte-identical to before, and existing rows keep working via the single-path fallback. Docs cover the picker, ordering, project directory, and the new troubleshooting and limitations (referenced files are not materialized; the dependency graph, drift, and networking views read the primary file). * fix: harden multi-file Git source (hash, unlink, collisions, node id) - hashContent folds ordered file CONTENTS (not paths) so a clean multi-file stack is not flagged as locally edited: create/apply hash the fetched files (repo paths) while pull hashes the on-disk files (materialized paths), which previously disagreed and showed a false "local edits detected". - Block unlinking a multi-file or project-directory Git source (409): the deploy spec lives on the source row, so removing it would silently revert deploys to root compose.yaml. Single-file sources still unlink. - Reject materialized-path collisions in the selection validator: an additional file equal to or nested under compose.yaml, an ancestor/descendant overlap between selected files, and a project directory nested under a compose file (previously a 500 at materialization). - DockerController.getContainersByStack uses the controller's node compose dir and passes its node id to the authored prefix, instead of the process default. * fix: CI failures on multi-file Git source (test crash, aria query, path barrier) - GitSourceFields no longer crashes when repoUrl/branch are falsy: the canBrowse trim() is optional-chained, so a reusable field component tolerates partial props. Fixes the apply-binding panel test, which feeds a minimal source object. - GitSourcePanel tests query the footer Remove button by its exact name, so the picker's per-file "Remove <path>" buttons no longer collide with the broad /remove/i match (the test intent, footer Remove present/absent, is unchanged). - validateCompose uses an inline resolve + startsWith barrier at the context-dir mkdir sink (CodeQL does not credit the wrapped isPathWithinBase helper), clearing the js/path-injection alert. The containment check is equivalent and contextDir is also validated upstream. * test: update Git source E2E spec for the multi-file compose picker The compose-file picker replaced the single #git-source-path input and added per-file Remove buttons, so the E2E spec drove selectors that no longer exist: - Drop the redundant compose.yaml fills (the picker defaults to compose.yaml). - Select the footer Remove button by exact name so the picker's per-file "Remove <path>" buttons no longer make the locator ambiguous. - Set a custom compose path through the picker (add via the manual input, press Enter, then remove the default compose.yaml). * test: match the footer Remove button with an exact Playwright name Playwright's getByRole name option is a substring match by default, so { name: 'Remove' } also matched the picker's "Remove <path>" buttons. Require an exact match so only the footer Remove button is selected.	2026-06-17 13:24:55 -04:00
Anso	52ff0725f4	feat: add Compose Doctor preflight checks for stacks (#1348) ... * feat: add Compose Doctor preflight checks for stacks Add an on-demand, advisory preflight that renders a stack's effective Compose model with `docker compose config` and runs a registry of deterministic checks before deploy, surfacing findings grouped by severity (blocker, high, warning, info) with a remediation for each. Findings cover unset env vars, host-port conflicts on the node, broad 0.0.0.0 exposure, missing bind-mount paths, a mounted Docker socket, privileged and host networking, moving image tags, missing restart policy and healthcheck, Swarm-only deploy fields, missing external networks or volumes, and container_name collisions. The report is node-scoped and stored as the last run per stack, and the route auto-proxies to the active node so a remote stack is checked on the node that owns it. A new Doctor tab on the stack detail panel runs preflight and shows the grouped findings, with a severity dot on the tab when the last run has blocker or high findings. The tab is gated on a compose-doctor capability so older nodes hide it. No environment value is ever stored, returned, or logged: only env key names and structural facts are read, and render failures surface a generic message or the missing required-variable names, never raw stderr. * fix: scroll the stack tab strip when its tabs overflow Adding the Doctor tab can push the per-stack Anatomy tab strip past the panel width on narrower layouts. Make the tab row scroll horizontally with subtle edge fades that appear only while there is more to scroll in that direction, so a panel wide enough to show every tab is unchanged. * fix: add clickable arrows and wheel scroll to the stack tab strip Hiding the scrollbar left mouse users with no way to scroll the overflowing tab row: a vertical wheel does not move a horizontal overflow and native rows do not drag-scroll. Replace the passive edge fades with clickable chevron arrows shown only when the row overflows that edge, and translate a vertical wheel over the row into horizontal scroll. * fix: inline the path-injection barrier in renderConfig CodeQL's path-injection check does not credit the wrapped isPathWithinBase helper as a sanitizer, so move the containment check inline at the spawn cwd sink, matching the canonical barrier used elsewhere in the codebase. Behavior is unchanged: the resolved stack directory must be contained in the compose base and may not be the base itself. * fix: hoist the compose-config spawn into the path-barrier scope The earlier inline barrier sat in a different scope than the spawn cwd sink (separated by the Promise-executor closure) and used a compound guard, so CodeQL did not credit it. Use the exact canonical startsWith barrier and hoist the spawn into the same scope as the check. Behavior is unchanged: the executor runs synchronously in the same tick as the spawn, so handlers still attach before any event can fire.	2026-06-10 11:35:39 -04:00
Anso	d369b03a38	feat: detect stalled stack updates and add in-app recovery actions (#1347) ... * feat: detect stalled stack updates and add in-app recovery actions Add a backend idle-output backstop that stops a deploy/update compose step that has gone silent (SENCHO_COMPOSE_STALL_TIMEOUT_MS, default 10m), so a hung image pull surfaces a fast failure instead of spinning indefinitely. Surface failed, timed-out, and stalled operations with recovery actions on the stack page: a desktop chip plus popover menu and an inline mobile card offering retry, restart, roll back (when a backup exists), refresh state, and copy diagnostics, all gated by deploy permission. The streaming deploy/update progress modal is now on by default and warns when output goes quiet. Container state is refreshed after a failed or stalled operation, and the UI never sits in an indefinite spinner. * fix: harden rollback against policy-blocked file mutation and refine recovery Address review findings on the stalled-update recovery work: - The rollback route restored backup files before running the policy gate, so a policy-blocked rollback could leave the on-disk config rolled back while the deployed containers were unchanged. Snapshot the current files first and revert them when the gate blocks; if that revert itself fails, escalate it on the persistent alert feed since the 409 is already sent. - Refresh container state after a successful manual rollback (rollback redeploys), without mis-recording a refetch failure as a rollback failure. - Suppress the stalled-output warning once live progress is unavailable. * test: mock snapshotStackFiles in the atomic-deploy rollback route tests The rollback route now snapshots stack files before restoring a backup, so its FileSystemService mock needs snapshotStackFiles. Without it the mocked call threw and the route returned 500, failing the success-path rollback assertions.	2026-06-10 10:12:24 -04:00
Anso	b21324f97a	feat(stacks): persist a drift ledger with temporal source-change detection (#1333) ... * feat(stacks): persist a drift ledger with temporal source-change detection Build on the read-only compose-vs-runtime drift check so a stack's drift is remembered over time, not just shown at a glance. - Record a deploy baseline: on a successful deploy, update, or rollback, store the deployed compose file's source and rendered-model hashes on the stack so the Drift tab can tell whether the file has changed since the last deploy. - Surface temporal drift in the Drift tab: "matches last deploy", "source changed since last deploy" (distinguishing a model change from a formatting-only edit), or "no deploy baseline yet". - Persist findings into a drift ledger: a re-check reconciles the current findings, recording newly detected ones and resolving cleared ones, and shows a short drift history under the findings. The drift report read stays side-effect-free; only an explicit re-check (and a deploy) writes the ledger. - Write drift detected/resolved events to the stack Activity timeline so the provenance sits alongside deploys and restarts. Node-local and available on the Community tier. Reconciliation is skipped when a check is not authoritative (Docker unreachable or a compose parse error) so an open finding is never falsely cleared. * fix(stacks): record the drift baseline for every deploy path and harden the ledger Address review feedback on the drift ledger: - Record the deploy baseline in ComposeService.deployStack/updateStack instead of only the manual route, so bulk, Git-source, App Store, scheduler, and webhook deploys all capture source/rendered hashes. Reconciliation stays on the explicit re-check. - Store no rendered baseline when the local parser cannot model the compose (for example a file over the parse cap) rather than a sentinel that would make a later real change read as unchanged. - Let temporal-overlay failures surface as a 500 instead of being hidden behind a neutral "no baseline"; only the compose read stays best-effort. - Omit the temporal card entirely when a report (for example from an older remote node) carries no temporal data, instead of showing a misleading "no baseline". - Keep drift_detected / drift_resolved history-only by excluding them from the routable-category whitelist, so they are never offered as a channel route that would never fire. - Use a JSON separator for the finding identity key so the source file is plain text (no embedded control byte). * fix(stacks): sanitize logged errors in the drift report handlers The drift report and re-check handlers logged the caught error object raw alongside the stack name, which a code scan flagged as a log-injection vector: a crafted stack name surfacing inside an error message or stack could forge log lines. Route the error through the log sanitizer so control characters are stripped before writing. Render it with util.inspect first so the stack trace, cause chain, and underlying error codes are preserved for debugging.	2026-06-07 20:44:22 -04:00
Anso	716daf77d0	feat(updates): auto-prune dangling images after updates (#1316) ... * feat(updates): auto-prune dangling images after updates Each update pulls a fresh image and recreates containers, leaving the replaced image behind as a dangling layer that previously had to be pruned by hand. A new "Prune dangling images after updates" toggle under Settings > System > Docker hygiene reclaims these automatically. The setting is on by default and opt-out. When enabled, a successful stack update (manual or scheduled) and a Sencho self-update each remove the dangling image layers they orphaned. Only untagged layers are touched; tagged images, volumes, and data are never removed. The toggle requires an admin account and is per node: each instance honors its own value, so a remote node self-update applies that node's own preference. A prune failure never affects the update result: on the stack path it is caught and logged after the update has already succeeded, and on the self-update path the helper-shell prune runs only after a clean recreate and cannot change the exit code or the recorded update error. * security(self-update): shell-quote label-derived values in helper command Address review feedback on the prune-on-update change: - The self-update helper command interpolated the compose service name and config-file paths (both read from Docker Compose labels) straight into a shell string. Shell-quote them via shQuote so a label carrying shell metacharacters stays inert data and cannot break the exit-code capture, error-file write, or prune guard. - Correct the settings copy and docs: the prune is a standard dangling-image prune, so it reclaims every untagged layer on the node, not only the one the current update orphaned. Tagged images, volumes, and data remain untouched. - Add tests: shell-metacharacter neutralization and prune-output suppression in the self-update command, and an atomic-update case asserting a prune failure does not trigger a rollback. * fix(updates): omit the reclaim figure when the daemon reports zero bytes End-to-end testing on a Docker daemon backed by the containerd image store showed the post-update prune removing a dangling image while the prune API returned SpaceReclaimed=0, so the stream printed "reclaimed 0.0 MB" even though an image was removed. Show the reclaimed figure only when the daemon reports a non-zero value; otherwise the line reads "=== Pruned dangling images ===". The overlay2 store still reports real figures and shows them. Add a test covering both branches.	2026-06-05 18:12:37 -04:00
Anso	5dea040ec8	fix(deploy-progress): decouple deploys from the live progress stream (#1246) ... * fix(deploy-progress): decouple deploys from the live progress stream The deploy progress modal streamed compose output over a WebSocket, but the deploy itself was coupled to that socket in two ways that could break or silently abort a deploy: - The deploy request was gated on the progress socket connecting, so any upgrade failure (a reverse proxy blocking WebSocket upgrades, or the admin-only stream rejecting a scoped deployer) left the modal stuck on "Connecting..." and the deploy never fired. - The backend terminated the running compose process when that socket closed, so minimizing the modal, navigating away, or a network blip aborted an in-flight deploy. Make the progress socket output-only: the deploy is owned by its request and runs to completion (or the existing command timeout) regardless of the stream. The modal now degrades to a "Live progress unavailable" state and still reports success or failure from the request result. Connect failures, drops, and a connect timeout all release the deploy instead of blocking it. Also route progress output per deploy: the frontend sends a correlation id on both the connectTerminal message and the deploy request header, and the backend keys progress sockets by that id so concurrent deploys from different tabs or users no longer cross-stream each other's output. Cap the in-memory parsed log rows so a very long deploy cannot grow the modal's state unbounded. * fix(deploy-progress): generate the deploy session id with a CSPRNG The per-deploy correlation id keys which WebSocket receives a deploy's live output, so a guessable id lets one authenticated client register a victim's id and read its compose output. It was built from Math.random() plus a timestamp, which is not cryptographically secure. Generate it with crypto.getRandomValues (128 bits, hex). That is the one Crypto member available in insecure contexts, so it still works over LAN HTTP where crypto.randomUUID is unavailable. * fix(deploy-progress): stop headerless ops bleeding into a keyed progress modal Address review findings on the progress-stream routing: - Only an id-less connectTerminal registration may become the id-less fallback socket. Previously every connectTerminal (including keyed deploy modals) set the fallback, so a headerless operation (bulk update, rollback, or a legacy client) resolved via getTerminalWs() into another user's keyed deploy modal. Keyed sockets are now excluded from the fallback, and a socket that adopts a session id is removed from it. - The connect-timeout fallback now also flags the modal as "Live progress unavailable" instead of leaving it on "Connecting..." while the deploy runs. - Log only a short prefix of the deploy session id in developer diagnostics, not the full capability value.	2026-05-28 20:51:45 -04:00
Anso	9dbce9c3c7	fix(spawn): attribute ENOMEM and ENOENT-under-memory-pressure spawn failures to host OOM (#1111) ... Operators previously saw "spawn docker ENOENT" or "spawn /bin/sh ENOENT" when the host was under memory pressure, which sent them down a missing-binary debugging path. Linux libuv's posix_spawn can fail to allocate its argv / path-search arena under low free memory and surface the underlying ENOMEM as ENOENT. Centralizes spawn-error mapping in a new utils/spawnErrors.ts helper: - Explicit ENOMEM is rewritten to "Out of memory while launching <command> (host free memory: X MiB of Y MiB)". - ENOENT under the 128 MiB free-memory floor is rewritten with the same wording plus a "reported as ENOENT under memory pressure" hint. - ENOENT for docker on a healthy host preserves the existing "Docker CLI unavailable on this node" mapping. - Other errors pass through unchanged. Applied at the four named offenders: ComposeService.execute(), ComposeService.captureCompose(), DockerController.getContainersByStack(), and FileSystemService.getStacks() (which gets an ENOMEM-aware log line for the scandir failure). Startup also logs host free/total MiB once and warns when free memory is below the 128 MiB floor, so the diagnostic surfaces before the first spawn attempt rather than after it fails. 37 tests cover the mapping function directly and the ComposeService / FileSystemService integration paths.	2026-05-19 07:27:12 -04:00
Anso	523ba5854c	fix(stacks): return 404 for nonexistent stacks on deploy/down/update (F-7) (#1108) ... POST /api/stacks/:name/{deploy,down,update} previously returned HTTP 500 with body {"error":"spawn docker ENOENT"} when invoked against a stack whose compose directory was missing. The status code was wrong (the named resource did not exist, so 404 is the right answer) and the message misled operators into thinking the docker CLI was unavailable. Add a small requireStackExists(nodeId, stackName, res) helper in routes/stacks.ts that validates the stack name and confirms a compose file is present via FileSystemService.hasComposeFile before any of the three handlers spawn docker compose. The helper is called immediately after requirePermission and before runPolicyGate so unauthorized callers still get 403 first and the policy gate never runs against a phantom stack. In ComposeService.execute(), narrow the child.on('error') handler so the genuine docker-binary-missing case (ENOENT on the spawn itself) rejects with "Docker CLI unavailable on this node" instead of the raw "spawn docker ENOENT". This is defense in depth for the rare case the pre-check cannot cover, and it fixes the misleading-message half of the bug as well. Cover the new contract with stack-actions-missing-stack.test.ts (four cases: deploy/down/update return 404, invalid name returns 400). Mock ComposeService as a tripwire so a future code path that bypasses the guard would fail loudly. Fix stacks-failure-notifications.test.ts by adding hasComposeFile to its FileSystemService partial mock so the existing happy-path-error-handling cases continue to flow into ComposeService.	2026-05-19 00:13:57 -04:00
Anso	5461bc316b	fix: harden stack management operations (#1046)	2026-05-14 10:21:18 -04:00
Anso	b1c5fe8391	fix: harden deploy enforcement paths (#1030) ... * fix: harden deploy enforcement paths * fix: update Docker toolchain to Go 1.26.3 * fix: repair Dockerfile tr argument split across lines * fix: bump protobufjs to clear npm audit high-severity advisories * fix(test): add execFile to child_process mock in compose-images test * fix: resolve merge conflicts with main * fix: resolve merge conflicts with main * fix: resolve merge conflicts with main	2026-05-12 19:30:49 -04:00
Anso	74ae2ce0c6	fix: harden atomic deployment rollback (#1029) ... * fix: harden atomic deployment rollback * fix: update Docker toolchain to Go 1.26.3 * fix: repair Dockerfile tr argument split across lines * fix: bump protobufjs to clear npm audit high-severity advisories * fix: sanitize error objects in console.error to prevent log injection	2026-05-12 15:58:30 -04:00
Anso	7663f4cd8b	feat(fleet): sencho mesh in traffic and routing tab (#858) ... * feat(fleet): sencho mesh in traffic and routing tab Lights up Sencho Mesh: cross-node container forwarding rendered as if the container next to you were on localhost. Builds on the dormant TCP frame plumbing from the prior PR (pilot tunnel TCP frames + sencho-mesh sidecar package) and exposes the Admiral-only orchestrator surface. Backend - New mesh_stacks table (per-node opt-ins) + nodes.mesh_enabled column via DatabaseService.migrateMeshTables. - MeshService singleton: sidecar lifecycle via Dockerode, opt-in/out with cascading override regeneration, request-based resolver from sidecar control WS, cross-node TCP forwarding via PilotTunnelManager (same-node fast path included), in-memory 1000-event activity ring buffer with durable mirror to audit_log for state-change events, per-node and per-route diagnostics, and the Test upstream probe. - MeshComposeOverride: pure YAML generator that injects extra_hosts using host-gateway. The user's docker-compose.yml is never mutated; overrides live under DATA_DIR/mesh/overrides. - ComposeService deploy/update splice the override file when the stack is opted in; non-mesh stacks behave identically to today. - Pilot agent resolveMeshTarget consults the local mesh_stacks table (defense in depth) and resolves Compose containers via Dockerode. - /api/mesh router with 13 Admiral-gated endpoints covering status, enable/disable, stack opt-in/out, alias listing, per-route diagnostic, Test upstream probe, per-node diagnostic, sidecar restart, activity log paginated and SSE. - meshControl WS slot at /api/mesh/control validates the mesh_sidecar JWT minted by MeshService; dispatched as upgrade slot 2 (canonical order preserved). Frontend - New Traffic Routing tab in FleetView, gated by isAdmiral and wrapped in AdmiralGate. Tab uses the cyan brand glyph and italic-serif state typography from the audit. - RoutingTab masthead with mesh activity drawer, per-node card grid with TogglePill, alias rows with five-state pill taxonomy (healthy / degraded / unreachable / tunnel-down / not-authorized), inline Test buttons. - Four sheets: opt-in picker with port-collision inline error, per-route detail with diagnostic + filtered activity, per-node diagnostics with active streams + resolver cache + restart action, fleet-wide activity log with filters. - meshRouteState helper centralizes pill-state mapping; pure-function tests cover all five states. Docs - User docs at /docs/features/sencho-mesh.mdx covering opt-in, troubleshooting, security model (4 guarantees + 4 explicit non-guarantees), and V1 limitations. - Internal architecture and runbook pages. - websocket-dispatch internal doc updated with the new slot. * fix(mesh): validate stack name before path use; fix test DB lifecycle Two surgical fixes against the prior PR. Path-injection (CodeQL js/path-injection): MeshService.optInStack, optOutStack, ensureStackOverride, and removeStackOverride now validate stackName via isValidStackName from utils/validation, reject malicious names at the API boundary, and additionally check isPathWithinBase on the resolved override file path for defense in depth. The dataflow from req.params.stackName to fs.writeFile no longer reaches an unsanitized path expression. Test DB lifecycle: mesh-service.test.ts used per-test setupTestDb / cleanupTestDb, which deletes the temp dir while DatabaseService still holds an open SQLite handle. On Linux CI this raises SQLITE_READONLY_DBMOVED on the next prepare() because the inode has been unlinked. Switched to file-scoped beforeAll/afterAll matching agents-routes.test.ts, with a per-test beforeEach that truncates mesh_stacks plus non-default nodes and resets the MeshService singleton in-memory state. Adds a new test case asserting the path-traversal rejection. * fix(compose): use discovered compose filename instead of hardcoded docker-compose.yml composeArgs() hardcoded `-f docker-compose.yml` for every deploy. Sencho writes its canonical compose file as `compose.yaml`, so any stack created via the UI failed to deploy with `open ...docker-compose.yml: no such file or directory`. When no mesh override applies, drop the explicit `-f` so docker compose's built-in discovery resolves the actual filename. When an override exists, look up the real base filename via FileSystemService.getComposeFilename() and pass both files explicitly. Also hoist the MeshService import to module top now that the dependency is known to be acyclic, and revert the matching unit-test assertion.	2026-05-01 01:50:53 -04:00
Anso	4e5ba17710	refactor(backend): sanitize user input before logging to close CRLF injection (#807) ... * refactor(backend): sanitize user input before logging to close CRLF injection Adds a small sanitizeForLog helper that strips CR, LF, tab, and ASCII control characters (0x00-0x1F, 0x7F) from a value before it is embedded in a console.log/warn/error/debug call. Wraps every call site where a user-controlled value (req.params, req.body, req.query, or a value derived from them) flows into a log message. Closes the bulk of the open CodeQL alerts in this family: - 96 js/log-injection - 28 js/tainted-format-string The helper is in backend/src/utils/safeLog.ts. Routes still pre-validate input at the request boundary; this is the second line of defense and gives static analyzers a sanitizer they can trace through. JSON responses, Docker filter labels, and other non-log call sites are intentionally left unwrapped. * refactor(backend): printf-style format strings for tainted-log call sites CodeQL's js/tainted-format-string rule flags template literals in the first arg of console.X when any interpolated value is user-controlled, regardless of whether each value is sanitized inline. The canonical mitigation is to use a static format string and pass values as positional args. Converts the 28 flagged template literals to printf-style ("%s") format strings, with sanitizeForLog applied to each positional arg. Also fills in the log-injection wraps on 9 sites where a user-controlled value was missed in the first sweep (agents, fleet, gitSources, imageUpdates, GitSourceService). No behavior change at runtime. Node's util.format substitutes %s tokens identically to template-literal interpolation. * fix(backend): wrap nodeId/snapshotId in fleet restore debug log CodeQL flagged the unwrapped numeric args even though they cannot contain control chars in practice. Apply the sanitizer for taint-flow recognition.	2026-04-27 10:47:23 -04:00
Anso	661b9c638b	feat(security): enforce scan policies as a pre-deploy gate (#719) ... Policies with block_on_deploy=1 now scan every stack image before docker compose up runs and reject the deploy with HTTP 409 on violation. The UI opens a dialog listing offending images; admins can override per deploy with ?ignorePolicy=true, and every bypass is recorded in the audit log with the originating route, actor, policy, and image list. When Trivy is not installed on the target node the gate fails open with a warning notification, so teams are never locked out by tooling state. Post-deploy and scheduled scans still evaluate matching policies and dispatch warnings on violations to surface drift on long-running stacks. Public API additions: policy and suppression CRUD under /api/security, plus the documented 409 block-response shape on all deploy paths.	2026-04-21 00:14:11 -04:00
Anso	a65a1c0e86	feat(stack-view): per-container health strip and structured logs viewer (#689) ... * feat(stack-view): per-container health strip and structured logs viewer Replaces the flat container list with a per-container health strip showing healthcheck state, uptime, port mapping with an open-app link, and live cpu/memory/network sparklines fed by a 60-sample ring buffer on the stats WebSocket. Adds a structured logs viewer that parses docker timestamps (emitted by the -t flag on the logs stream) and classifies each line by level. Rows render as a DOM grid with filter pills (all / info / warn / err with count), following indicator, and plain-text download. A segmented toggle switches between the structured viewer and the original xterm view; the choice is persisted in localStorage. * fix(stack-view): disable no-control-regex for ANSI escape pattern ANSI escape sequences start with ESC (0x1B), which is a control character. The regex is intentional and cannot be rewritten without it.	2026-04-19 00:17:51 -04:00
Anso	6275adc6b3	feat(registries): harden Private Registry Credentials feature (#597) ... * feat(registries): add stateless test endpoint, ECR caching, URL and host hardening Adds a POST /api/registries/test endpoint so credentials can be verified before being persisted. Caches ECR authorization tokens in memory until their AWS-reported expiry (minus a safety margin) instead of fetching on every compose invocation. Normalizes registry URLs on save so the stored values match the keys Docker expects in ~/.docker/config.json, fixes a bidirectional host-match bug in getAuthForRegistry that could cross-match overlapping hostnames, and surfaces per-registry decryption failures as warnings in the deploy log stream instead of swallowing them. Also strips the Authorization header on cross-host redirects in the test probe, rejects non-http(s) schemes on save, and validates the shape of returned ECR authorization tokens before use. * refactor(registries): align UI with design system and add in-form test button Swaps the registry type dropdown from shadcn Select to the project's Combobox, applies the canonical card bevel and top-border hover styling to the form container and each registry row, restyles the delete button to the ghost + muted destructive pattern, uses strokeWidth 1.5 on every Lucide icon, and routes all toast errors through the standard defensive chain. Adds a Test connection button inside the form so credentials can be verified before saving. * test(registries): cover RegistryService and deploy warnings surface Adds unit coverage for URL normalization, the encrypt/decrypt round trip through create and resolveDockerConfig, exact-host matching in getAuthForRegistry, resolveDockerConfig warnings on decryption failure, ECR token cache hit/miss and invalidation on update, the stateless testWithCredentials path for 200, 401 with and without a challenge, network errors, and ECR success and failure including malformed tokens. Extends the ComposeService tests to verify that warnings from resolveDockerConfig reach the deploy log stream. * docs(registries): document test-before-save flow and troubleshooting Describes the in-form Test connection button, the two-point testing flow from the registries list, the cached ECR token behavior during deploys, the per-registry warning Sencho emits when a stored secret cannot be decrypted, and adds a Troubleshooting section covering common 401 causes, ECR token handling, warning interpretation, and per-node credential scoping.	2026-04-14 17:58:57 -04:00
Anso	2465f7607e	fix(stacks): harden stack management with security, validation, and logging (#520) ... * fix(stacks): harden stack management with security fixes, validation alignment, and logging Validate WebSocket stack names with isValidStackName() to close a path-traversal gap on the /api/stacks/:stackName/logs WS endpoint. Align POST /api/stacks to use the canonical validator (allows underscores). Replace error: any catch blocks with error: unknown + type narrowing. Add cache invalidation to PUT /api/stacks/:stackName/env. Rename DELETE param from :name to :stackName for consistency. Add standard [Stacks] lifecycle logs and diagnostic [Stacks:debug] logs gated behind the Developer Mode toggle (with 5s TTL cache). Extract shared isDebugEnabled() and getErrorMessage() utilities. Frontend: roll back optimistic status on API failure, guard unsaved changes when switching stacks, pre-check duplicate names in App Store. * docs(settings): update Developer Mode description to mention debug diagnostics	2026-04-12 05:43:15 -04:00
Anso	10597d213a	fix(error-handling): surface silent errors across the codebase (#326) ... Add console.warn/console.error logging to 22 silent catch blocks across 10 files. Errors in cleanup, migrations, SSO, fleet snapshots, shutdown, and validation are now visible in logs. ENOENT guards added to file-system catches to distinguish missing files from permission errors. No control flow changes.	2026-04-01 21:56:41 -04:00
Anso	116f15dae9	fix(stacks): resolve permission denied error on stack deletion (#261) ... * fix(stacks): resolve permission denied error when deleting stacks with root-owned files When Docker Compose creates files as root inside a stack directory, the non-root Sencho process cannot remove them. This adds a Docker-based fallback: if fsPromises.rm fails with EACCES/EPERM, Sencho spawns a short-lived Alpine container to clean up the root-owned files. Also enhances docker compose down with --volumes --remove-orphans to let Docker clean up its own resources before filesystem deletion. * docs: clarify that pre-existing root-owned stacks can be deleted * fix(stacks): include Docker stderr in fallback deletion error message Fixes CI lint failure: 'stderr' was assigned but never read in forceDeleteViaDocker(). Now surfaces Docker stderr output in the error message when the fallback cleanup fails.	2026-03-29 21:32:05 -04:00
Anso	244c83a0c3	feat(registries): add private registry credential management (Team Pro) (#240) ... Add centralized credential storage for private Docker registries with support for Docker Hub, GHCR, AWS ECR, and self-hosted registries. - New `registries` table with AES-256-GCM encrypted secrets - RegistryService with CRUD, test connectivity, Docker config generation - 5 API endpoints gated by requireTeamPro + requireAdmin - ComposeService injects credentials via temp DOCKER_CONFIG on deploy/pull - ImageUpdateService passes stored credentials for private registry checks - AWS ECR just-in-time token refresh via @aws-sdk/client-ecr - RegistriesSection UI in Settings Hub with type-aware form - Documentation with screenshots	2026-03-29 12:56:30 -04:00
Anso	32a7d53b2b	feat: RBAC, atomic deployments, fleet backups, and licensing (Pro) (#185) ... * feat: add RBAC viewer accounts, atomic deployments, and fleet-wide backups (Pro) Introduces three Pro-tier features: - RBAC: Multi-user system with admin/viewer roles, user management UI, automatic migration from single-admin credentials, viewer restrictions across the entire UI (read-only editor, hidden action buttons) - Atomic Deployments: Pre-deploy file backup to .sencho-backup/, automatic rollback on health probe failure, manual rollback button, health probes added to stack updates, webhook-triggered deploys use atomic rollback - Fleet-Wide Backups: Point-in-time snapshots of compose files across all nodes (local + remote), stored centrally in SQLite, per-stack restore with optional redeploy, graceful handling of offline nodes * fix(settings): use correct ProGate prop name in UsersSection * fix(settings): remove unused isPro prop from UsersSection * fix(auth): fetch user info after login and setup so isAdmin is set correctly * feat(pricing): revise pricing strategy and enforce variant-based seat limits Raise Personal Pro from $49/yr to $69/yr with 3 viewer seats (up from 1). Add $15/mo billing option for Team Pro. Mark lifetime pricing as a 90-day early-adopter offer. Store Lemon Squeezy variant_name on activation/validation and enforce seat limits server-side per variant. * feat(licensing): add Lemon Squeezy checkout, webhook, and billing portal integration Server-side checkout URL generation (POST /api/checkout) with admin email pre-fill and instance_id custom data. HMAC-SHA256 verified webhook endpoint (POST /api/webhooks/lemonsqueezy) handling order, subscription, and payment lifecycle events for automatic license activation. Customer billing portal link stored from webhook events and exposed via GET /api/billing/portal. In-app checkout buttons in Settings with manual license key fallback. * fix(licensing): exempt Lemon Squeezy webhook from auth middleware The catch-all auth middleware on /api/* was blocking the public webhook endpoint. Added /webhooks/lemonsqueezy to the exemption list alongside /auth/* and /webhooks/:id/trigger. * feat(pricing): update pricing to final live rates Personal Pro: $7.99/month, $69.99/year, $249 lifetime. Team Pro: $49.99/month, $499.99/year, $1,499 lifetime. Added personal_monthly checkout variant across backend, frontend, and website. * refactor(licensing): remove server-side checkout/webhook for self-hosted model Sencho is self-hosted — each user runs their own instance, so there is no central server to receive webhooks or hold the store API key. Replaced in-app checkout buttons with a "View Pricing" redirect to sencho.io and kept manual license key activation as the primary flow. - Delete LemonSqueezyService (checkout, webhook, HMAC verification) - Remove POST /api/checkout, GET /api/billing/portal, POST /api/webhooks/lemonsqueezy - Remove raw body parser and auth exemption for webhook route - Remove all LEMONSQUEEZY_* env vars from .env.example - Replace checkout buttons in SettingsModal with single "View Pricing" button - Simplify LicenseContext checkout to open sencho.io pricing page - Update licensing docs to reflect website-based purchase flow * chore: normalize em-dashes to hyphens across codebase (linter) * chore: remove accidentally tracked directories from index	2026-03-26 21:58:24 -04:00
Anso	db73d7671a	feat: RBAC, atomic deployments, and fleet-wide backups (Pro) (#181) ... * feat: add RBAC viewer accounts, atomic deployments, and fleet-wide backups (Pro) Introduces three Pro-tier features: - RBAC: Multi-user system with admin/viewer roles, user management UI, automatic migration from single-admin credentials, viewer restrictions across the entire UI (read-only editor, hidden action buttons) - Atomic Deployments: Pre-deploy file backup to .sencho-backup/, automatic rollback on health probe failure, manual rollback button, health probes added to stack updates, webhook-triggered deploys use atomic rollback - Fleet-Wide Backups: Point-in-time snapshots of compose files across all nodes (local + remote), stored centrally in SQLite, per-stack restore with optional redeploy, graceful handling of offline nodes * fix(settings): use correct ProGate prop name in UsersSection * fix(settings): remove unused isPro prop from UsersSection * fix(auth): fetch user info after login and setup so isAdmin is set correctly	2026-03-26 12:51:30 -04:00
SaelixCode	e876a91a2e	fix(lint): resolve all backend ESLint errors to pass CI lint step ... Config changes (eslint.config.mjs): - no-unused-vars: caughtErrors=none (catch clause vars are intentionally ignored throughout) - varsIgnorePattern/argsIgnorePattern: ^_ (allow _-prefixed intentional ignores) - ban-ts-comment, no-namespace, no-empty: downgraded to warn (pre-existing patterns) - no-control-regex: off (terminal output processing intentionally uses control chars) Dead code removed: - index.ts: remove unused spawn/exec/promisify imports and dead execAsync variable - FileSystemService.ts: remove duplicate fs default import (fsPromises already imported) - LogFormatter.ts: remove unused parsed variable (JSON.parse used only for validation) Auto-fixed via eslint --fix: - prefer-const: ComposeService, DockerController, MonitorService, index.ts	2026-03-22 01:09:18 -04:00
SaelixCode	eb0c0263c7	fix: Distributed API UI & metrics polish + DEP0060 suppression ... Add Node modal — type selector & state reset: - Restored a Local/Remote <Select> dropdown in renderFormFields so users can explicitly choose the node type instead of it defaulting silently to 'remote'. - Switching type clears api_url and api_token so no stale remote credentials carry over if a user switches from Remote to Local mid-form. - Replaced the static "Add Remote Node" title with a dynamic one that reflects the currently selected type ("Add Local Node" / "Add Remote Node"). - onOpenChange now resets formData to defaultFormData whenever the dialog opens, preventing stale values from a previous session leaking in. Remote connection details — real metrics: - testRemoteConnection previously returned hard-coded '-' for containers, images, and cpus after a successful auth/check ping. - Now fires three parallel requests (Promise.allSettled) after auth passes: /api/stats → containers total + running count /api/system/stats → cpu.cores /api/system/images → image list length - Each field falls back to '-' gracefully if an endpoint is unavailable, so a slow or older remote instance never breaks the connection test. DEP0060 util._extend suppression: - http-proxy@1.18.1 calls util._extend when createProxyServer() is first invoked at runtime (NOT at import time). A process.emitWarning override placed before the proxy instantiations intercepts only DEP0060 without suppressing any other warnings. No package version changes needed. Also includes linter/formatter normalisation across multiple files.	2026-03-19 16:06:47 -04:00
SaelixCode	c91c9ed7fd	refactor: pivot from ssh proxy to distributed api model ... - Delete SSHFileAdapter, IFileAdapter, LocalFileAdapter (SSH/SFTP stack) - Remove ssh2, ssh2-sftp-client dependencies; add http-proxy-middleware, http-proxy - NodeRegistry: remote nodes no longer use Dockerode TCP; new getProxyTarget() returns {apiUrl, apiToken} - NodeRegistry.testConnection: remote nodes use HTTP GET /api/auth/check instead of docker.info() - DatabaseService: Node interface swaps SSH/TLS fields for api_url + api_token; legacy columns preserved for DB compat - FileSystemService: reverted to clean local-only fs.promises; adapter pattern fully removed - ComposeService: executeRemote() and SSH log streaming deleted; local-only execution remains - index.ts: add /api/auth/generate-node-token endpoint (long-lived JWT, scope:node_proxy) - index.ts: authMiddleware now accepts Bearer token in addition to cookie (Sencho-to-Sencho auth) - index.ts: remote HTTP proxy middleware intercepts all /api/ requests for remote nodes, strips x-node-id, injects Authorization header, proxies to api_url - index.ts: WS upgrade handler proxies WebSocket connections for remote nodes via http-proxy wsProxyServer - NodeManager.tsx: form reduced to Name, API URL, API Token; Generate Node Token button added inline - NodeContext.tsx: Node interface updated to api_url/api_token Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-03-19 12:20:32 -04:00
SaelixCode	26b8f62968	fix: separate Docker API port from SSH port, add SSH credential UI, fix compose_dir routing	2026-03-18 21:04:43 -04:00
SaelixCode	8c51198468	feat: Remote Nodes Wiring & SSH Adapters	2026-03-18 12:55:30 -04:00
SaelixCode	fb3a28834e	fix: implement two-stage teardown for reliable atomic rollbacks	2026-03-06 10:18:47 -05:00
SaelixCode	953049a45d	feat: implement smart error parser and post-deploy health probe	2026-03-05 19:59:35 -05:00