60 Commits

Author	SHA1	Message	Date
Anso	38ee4527b1	feat(git): add GitOps revision store and Direct apply transitions (#1835) ... * feat(git): add GitOps revision store and Direct apply transitions Adds the canonical GitOps revision state model: schema, store, history, approval resolution, Direct apply transitions, and a first-cut read projection. Also widens recovery capture with generation, artifact, and source-acceptance bindings, rejects credentials and query strings in new Git repository URLs, and keeps Blueprint and node-label routes on the hub. * feat(git): add create-from-Git activation and teardown transitions Adds the durable half of create-from-Git: the single transaction that persists the application, resolved commit, generation, checkpoint, and candidate together, and the teardown that tombstones a create which never reached its success boundary. Adds the staging marker and operation-owned cleanup rules that decide what a crashed create is allowed to delete. * feat(git): make create-from-Git crash-safe Wires the create path through the GitOps state model: the staging marker is written before the candidate is built, the activation transaction persists the application, generation, and checkpoint together, and the source row plus accepted pointers now commit as one success boundary. A create that fails before that boundary removes only what it staged, then tombstones itself. The managed-area sweep no longer deletes a directory it cannot prove it owns, counts in-flight creates as claimed, and is awaited at startup so an interrupted create is settled before any mutation service runs. * feat(git): settle interrupted create-from-Git at startup Adds boot-time recovery for creates a previous process left in flight. A create whose project was already committed to disk is finished, including its source row and accepted pointers. Anything earlier has its files removed first and is then tombstoned, so the stack name becomes usable again instead of staying locked by a half-created application. A source row that outlived its application is always preserved. * fix(git): close create-from-Git ownership and reporting gaps Removes two ways a failed create could destroy files it did not own: boot recovery no longer deletes a stack directory the create never recorded making, and the managed-root teardown is gated on having created that root. Cleanup now rejects a path that is not a generation directory, so a malformed row cannot widen the blast radius. A create that fails during materialization now clears its staging marker, which previously stayed behind and made the stack name uncreatable until restart. Generation paths no longer resolve through a module cycle that left them pointing at a directory that never existed. Boot recovery restores the deploy spec and manifest cache a finished create needs, settles each application independently so one bad row cannot strand the others, and distinguishes a missing directory from one it could not read. A create that succeeds now clears its checkpoint, and a failure after the success boundary says the stack was created rather than reporting a bare error. * fix(git): reap managed areas that nothing has ever claimed The boot sweep now distinguishes a missing staging marker from an unreadable one. A missing marker means no stack, no create, and no marker claims the directory, which is the ordinary orphan a crashed stack deletion leaves behind, so it is removed as it always was. An unreadable marker is evidence of a claim that cannot be verified, so the area is preserved. Without this an orphaned managed area survived every boot forever, since a completed create deletes its marker and nothing else would ever claim it. * feat(git): complete the Direct source and target transitions Adds the remaining Direct events the state model owed: an invalid fetch that advances the resolved commit without minting a candidate, a blocked candidate that is visible but cannot apply, dismissal that leaves the running workload alone, material configuration change that invalidates a staged candidate while keeping accepted and applied pointers, the two deploy failure classes, and application and target tombstones. Widens the application update to every mutable column. It previously wrote about half of them, so a transition assigning a pause, suspend, recovery, or intent field would have type-checked, appeared in the history snapshot, and been silently dropped at commit. * feat(git): record Direct Git operations in the revision state Wires fetch, apply, and detach through the state model. A pull records the fetch and, when it produces a candidate, the generation behind it, marking a blocked change plan as a blocker that cannot apply. An apply records the acceptance that binds the generation to the workload, and closes its operation on failure so a throw cannot leave the source reporting work in progress. Detach tombstones the application and its target in the same transaction that removes the source row. Every producer is a no-op for a stack with no live application, so installs whose Git stacks predate this model are untouched until migration runs. * test(git): cover the Direct Git producers end to end Adds a harness that stubs only the git transport and rollback capture, so fetch, candidate materialization, change-plan classification, apply, and detach all run for real and are asserted against the resulting revision state. Covers the seam the transition-level tests could not: that a fetch advances the resolved commit without moving the accepted generation, that the apply binds the exact candidate the fetch recorded and its acceptance proves that generation and no other, that detach tombstones while keeping configured identity as a frozen fact, that a failed fetch closes its operation and leaves every pointer alone, and that a stack with no application produces no writes. * feat(git): bind the deployed generation from the Compose adapter Makes ComposeService the sole producer of deploy events. Every deploy path funnels through deployStack, so recording it there keeps one start and one terminal row per mutation instead of each caller reporting its own. A successful deploy binds the applied generation, a failure records the class and leaves the deployed pointer alone. The class is conservative: once the compose command is handed off we cannot prove the workload was untouched, and claiming it was intact is the more dangerous error. A stack with no live application, or one with nothing applied, has no deploy identity to record and is skipped entirely. * feat(git): promote healthy and last-known-good from health verdicts A stack health run now records which generation it observed, and its verdict drives the revision state. Promotion is narrow by design: healthy and last-known-good move only when the run passed, watched the whole stack, and the generation it watched is still deployed. A stale or service-scoped verdict records history and moves nothing. Last-known-good keeps the artifact expectation only when that expectation belongs to the promoted generation. Otherwise the generation is still good and its executable identity simply is not proven, so the pointer is left null rather than borrowed from another generation. beginStack now takes the deployed generation explicitly rather than reading current state, because a verdict is only meaningful for the generation the run actually observed. Update-path callers pass null until updateStack reports what it deployed. * feat(git): bind the deployed generation on the update path Gives updateStack the same treatment deployStack has: it opens a deploy operation at the point Compose is handed the mutation, closes it on both outcomes, and reports the generation it bound. The orchestrator carries that binding through, and all five update call sites now pass it to the health gate instead of null, so a passed health run after an update promotes healthy and last-known-good exactly as it does after a deploy. The operation is opened at the compose call rather than at entry, so an update that fails during capture or classification records nothing: it never touched the workload, and reporting a deploy failure for it would make the projection claim something that did not happen. * feat(git): track linked stacks and material config changes Linking an existing stack to a Git source now brings it into the revision state: the application starts live with nothing desired or accepted, so the projection asks for a fetch rather than claiming a state it has not observed. Editing a source's material configuration invalidates its staged candidate in the same transaction that writes the row and clears the pending pull. A candidate built from a different repository, ref, or file set can no longer be applied, and clearing pending without invalidating it would leave the model offering an apply the operator cannot produce. Credential-only and policy-only edits change nothing material and leave the candidate alone. * feat(git): retire the application when a stack is deleted Stack deletion now tombstones its GitOps application and targets in the same transaction that commits the deletion. A deleted stack that kept a live application would go on claiming the stack name and block re-creating it through the unique live-application index. The tombstone is driven from the deletion service rather than from inside DatabaseService, so the store keeps its transitions and their history in one place and the two modules do not form an import cycle. * feat(git): retire node targets when a node is removed Deleting a node now tombstones the GitOps targets that lived on it, in the same transaction as the delete and while those rows still exist. Otherwise a part-way failure would leave targets pointing at a node that is gone. Applications stay live: a Direct application still describes a real stack, and a Blueprint one may have targets on other nodes. Both the local and remote node paths go through the same helper, so neither can skip the retirement. * feat(git): add proof-bound recovery transitions A restore moves a target back to an older generation, the one case where a target and its application legitimately disagree about what is current. These transitions decide what may move with it. Pointers move only when the restore is provable: the recovery point named a generation, that generation still exists, and it belongs to this application. The artifact expectation comes from what the recovery point captured, never from what the application expects now, and the acceptance is kept only if it still proves the restored generation rather than borrowing one that authorized a newer one. A last-known-good survives unless the generation behind it is gone or turns out to belong elsewhere, and then the reason is recorded so the projection can say unavailable rather than none. An unproven restore is still recorded as a real operational recovery and moves nothing, because there is no evidence to move pointers to. * fix(git): close GitOps operations that cannot be recorded Recording a transition still never fails the operation it describes, but a rejected terminal event no longer leaves that operation open. A start that never terminates reported work in flight for ever and offered no actions, and because a new fetch refuses to open a second operation, one rejection silently stopped the model tracking that stack until a restart. Startup now reclassifies operations the previous process left open, which the transition already supported but nothing called. An interrupted restore is closed there too, since only the terminal recovery events clear it. The pull records its fetch, generation, and candidate as one transaction, and the apply verifies the candidate was built from the commit being applied. Separately those two allowed an apply to accept a generation whose files were never on disk while the projection reported the older commit as current. Also: the candidate now records its own invocation rather than the one it replaces, changing material configuration is refused while an operation is in flight, a tombstoned target cannot be repopulated by a late health verdict, a deploy handle is not returned when its start was not recorded, the mutation handoff is marked only once Compose is genuinely about to receive it, and a rejected tombstone fails one deletion rather than aborting the boot sweep. * feat(git): record why unprovable evidence was dropped The projection carried a limitations array that only read-time derivation ever wrote to. When a transition cleared a pointer it could not prove, the result was indistinguishable from that pointer never having existed, and in two cases it made the target read healthier: a dropped artifact expectation silently disables the runtime drift check, and an unproven restore left every pointer agreeing with itself so the target reported as synced and healthy. Adds a persisted, fail-closed evidence record on the application and target rows. The transitions that drop an artifact expectation, a last-known-good artifact, or a source acceptance now say why, an unproven restore is marked as such, and the deriver folds all of it into the limitations the projection already exposes. Clearing a code when the evidence becomes provable again is part of the contract, so a stale doubt cannot outlive its cause. This lands before the migration matrix because migration is the largest producer of evidence that cannot be proven, and the plan requires those to surface as bounded limitation evidence rather than as fabricated pointers. * feat(git): migrate pre-existing Git stacks into the revision state Git stacks created before this model had no application at all, so every producer was a no-op for them and the projection could not describe them. They are now brought in at boot. The governing rule is that a canonical pointer is written only when the evidence proves that exact generation under the repository and ref configured now. A legacy applied commit is not that proof on its own: the manifest may be gone, unreadable, or stamped for a repository the stack no longer points at. In each of those the commit is kept as recorded limitation evidence and the pointers stay null, so the stack asks for a fetch instead of asserting a state nobody verified. Deployed, healthy, and last-known-good are never invented, because a manifest proves what was materialized, not what is running, and no source acceptance is written because nobody approved through the model. Replay is decided by scope, schema version, and configuration fingerprint. A changed fingerprint re-runs the matrix, but a stack that already has an application is skipped rather than rebuilt, so migration cannot overwrite pointers written with proof it does not have. A stack whose directory has vanished migrates to a tombstone rather than claiming a name it cannot back. * feat(git): record rollbacks in the revision state A restore now opens a recovery in the model before any file moves, so a crash mid-restore leaves a target that says what it was doing rather than one that merely looks broken, and closes it on both outcomes. The failure is classified by whether the files had already been restored, because only a failure before that leaves the previous workload provably intact. Pointers move only when the restore is provable: the recovery point named a generation, that generation still exists, and the manifest actually restored carries the same commit and manifest version. Anything less is still recorded as a real recovery, it simply has nothing to bind, and the transition marks it unproven rather than guessing. The deployed pointer is never claimed here. This path drives Compose through a callback that reports nothing back, so binding cannot be proven and applied moves without it. * feat(git): add the deferred rollout-state transitions Retry scheduling, suspension, pause, and partial rollout complete the transition store. They have no production writer by design, but implementing them now means the deriver has no branch a writer cannot reach, and the shape a future producer has to satisfy is pinned rather than inferred from the read side. None of them is a statement about health. A scheduled retry leaves the failure that caused it visible, so a stack that keeps failing does not read as merely busy. A suspended source keeps everything it had accepted, and an operation in flight is interrupted rather than abandoned so it cannot report as running for ever. A paused or partially rolled out target keeps whatever was deployed, and the partial record never stands in for a deployed pointer. * test(git): stop the Compose producer tests reading Docker from the host Both deploy cases asserted that the compose command would reject, which was only true on a workstation with no Docker daemon. On Linux the command succeeded, the promise resolved, and the assertions failed. The compose subprocess now reports an exit code each test chooses, so the adapter is what decides the outcome rather than the machine. Only the verbs that need a daemon and travel through spawn are answered; `config` still runs for real, because a host with the CLI and no daemon parses compose files exactly as CI does, and failing that call breaks every create in the file. That also makes the bound path reachable, so the deploy test now covers it: a failed compose leaves the deployed pointer alone and classifies post-mutation, and a successful one binds the applied generation, returns it for health to bind against, and clears the earlier failure. * fix(git): contain every managed-area path at the call that uses it A stack name reaches the managed root without passing through `isValidStackName` on this path, so the marker, the cleanup, and the create's root probe all built filesystem paths from unvalidated input. Each call now resolves its target against the managed area and checks containment in its own scope, which is also the form the security scan credits: it does not follow the barrier through the shared `isPathWithinBase` helper, so the check has to sit with the call it protects. `cleanupUnclaimedManagedRoot` removes a whole root recursively and gets the same check, even though it was not among the reported calls. Clearing a staging marker can now fail, so the two boot-recovery branches that dropped the checkpoint first were reordered. Losing the checkpoint while the marker survives would report the area as settled and leave the stack name uncreatable. The deploy adapter's two failure logs built their format string from the stack name; both now use a constant format with the name as an argument. The marker and cleanup fixtures stood a bare temp directory in for a managed root, so nothing exercised the invariant these checks enforce. Both now build a real managed area over a scoped data directory. * fix(git): refuse to claim a managed area whose marker cannot be read A marker that exists but will not parse is still someone's claim, yet the write path refused only a readable foreign marker and wrote straight over a corrupt one, discarding the reason and logging nothing. A transient permission error, a marker truncated by a crash between write and rename, or a full disk all read as corrupt, and each one let a second create take ownership of an area the first still owned. Every other path in the module preserves on corrupt; this one now does too, and says why. Clearing a settled create's staging marker can fail, and reporting that as a retained create sent a reader looking for an unfinished create that had finished. It reports `marker_retained` instead. The checkpoint is still kept for the retry, so its encrypted token can outlive the create while the clear keeps failing; that tradeoff is stated at the call, because the alternative leaves a marker no later create can get past. The reaper also computed why it was preserving an area and threw the reason away, so a directory could survive every boot with nothing said about it. The containment checks these paths rest on had no tests and were silently deletable, so the out-of-area read, write, delete, and reap now have them. * feat(git): bind and observe a recovered generation A proven restore could never claim the deployed pointer, because the restore path drives Compose through a callback that reported nothing back: a caller that restored some other way resolved identically, so binding on a resolved promise would have claimed a workload nobody launched. The Compose wrapper now returns what it did, and only that answer binds. With binding reachable, a bound recovery claims its health run inside the same transaction as the pointers it describes, and arms the timer once that transaction lands. Committing the two together is the point: a crash between them would otherwise leave a restored workload nothing was watching. Anything that stops the timer starting writes the run off immediately, so an observing row never outlives the timer meant to watch it, and a reservation is never armed across a restart. The reservation is handed to the transition rather than reached for, because the health gate reports its verdicts back through the transition store and importing it there would close a module cycle. Startup now finalizes each interrupted observation on its own instead of sweeping them with one update, so the revision state hears a verdict for every run rather than watching the rows change under it. * feat(git): add the rollout-scoped rollback transitions Completes the deferred-state set. The three aliases write the same recovery columns under the same rules as the recovery events, differing only in provenance: Direct Git recovery emits `recovery_*`, and a later rollout producer emits these. Nothing in this PR writes them, which is why they are tested directly rather than through a caller. `partial` is the one failure class they add, for a rollback that reached some targets and not others. Completing has no unproven variant on purpose: a rollback nobody can bind to a generation has nothing to complete against, so it refuses without one and refuses a generation another application owns. Also covers the recovery health reservation end to end: reserving writes and links the row without arming anything, a replay reuses the run rather than opening a second observation of the same restore, arming inserts nothing and supersedes a conflicting stack gate, an unarmed reservation is written off once, and a reservation that outlived its process is finalized rather than armed. The per-row startup sweep is pinned too, since one row whose write fails leaving the others finalized is the entire reason it replaced a bulk update. * feat(git): add the Blueprint source and deployment transitions The first slice of the Blueprint work: the store side, with no production caller yet. The routes and the reconciler are wired to these next, and pinning the contract first means a caller cannot quietly satisfy a different one. What these enforce is that a terminal event names the request it answers. A deploy records the intent and candidate it was launched for, and a terminal is accepted only against that same request, matching stage and identity on one side rather than either in isolation. Without that pairing a deploy could be acknowledged out of an in-flight withdraw, or one request's intent stored against another's candidate. An acknowledgement carries the candidate from the matched request rather than from its own payload, for the same reason. Releasing an operation now releases its identity with it, and a resolved interruption is retired rather than left to match again. Both were leaking through the shared helpers: a stale identity let a later start resurrect a superseded intent as live, and a surviving interruption both reported completion as unknown for ever and let a late acknowledgement regress the target after two later deploys had succeeded. Fixing the helpers fixes the Direct paths too. A start refuses to displace an unrelated operation, which would otherwise abandon it with no terminal event and no history saying so. Observations write the latest stage, since the placement facet reads it to decide whether a stateful deployment is waiting on confirmation, and history alone could never reach a reader. Minting is left to the caller. A no-op Blueprint edit must mint no intent, because a fresh identity would invalidate acknowledgements that are still accurate. * feat(git): record Blueprint create, edit, pin, and delete Wires the Blueprint routes to the revision state. Each producer writes the Blueprint source row and its GitOps rows in one transaction, so an operator never sees a Blueprint that exists with nothing describing what it means, or an intent for a Blueprint that failed to save. The question these answer is when an edit invalidates what the fleet already acknowledged. Changing the name, the compose content, the selector, the drift mode, or whether it is enabled changes what nodes run or where, so each mints a new intent and candidate. Changing the description or the classification changes how the Blueprint reads and nothing a node can observe, so it updates the source row alone: minting there would make every acknowledgement report as stale over a reworded sentence. An edit is measured by value, not by which fields were submitted. The editor sends every field on every save, and the source layer decides what to invalidate from which keys are present, so only the keys that genuinely differ are passed down. Without that the two disagreed: rewording a description advanced the revision past the one the current intent describes and cleared the approval, while this layer classified it as metadata and minted nothing. Selectors compare by value too, so reordering a list that names the same nodes is not a placement change. That measurement also changes one behaviour: a save that alters nothing now leaves the approval alone, where before any save cleared it. Nothing changed, so nothing is invalidated. Pinning revises placement the same way a selector edit does, and re-pinning the node already pinned is not a change. Deleting tombstones the application and its live targets so the Blueprint stops claiming its slot, and withdraws nothing itself, because the route has already done that and doing it twice would record removals that never happened. The required node set is stored in a canonical order, so reordering it is not mistaken for a placement change. A Blueprint that predates the model has no application yet and is left for migration rather than given a first intent here, which would claim a starting point for deployments nobody has reconciled. Desired nodes are passed in rather than computed here: the reconciler that knows how to compute them reaches this layer, and importing it back would close a module cycle. * feat(git): record the placement a label or cordon moves A label and a cordon say nothing about any one Blueprint, but both change which nodes a selector matches. Each now revises placement for whichever Blueprints the change actually moved, by comparing the desired nodes either side of the write rather than reacting to the event. That comparison is the whole point. Labelling a node no selector mentions, or cordoning one no Blueprint wanted, moves nothing, and minting an intent for it would invalidate every acknowledgement in the fleet over an edit no node can observe. The same comparison covers a Blueprint pinned to a cordoned node: a cordon governs automatic placement only, so the pinned target still wants that node and its set does not move, without needing a case of its own. Nodes are compared as a set, so the same nodes returned in a different order is not a change either. Each route wraps its existing write and the recording in one transaction, so a recording failure cannot leave a fleet selecting on a label nothing recorded. The write itself is untouched, which keeps label validation and the per-node limit where they already live rather than restating them in a producer. Node deletion already retires its targets, so this adds nothing there. * feat(git): record Blueprint deployments by what caused them Every production write to a Blueprint deployment row now goes through one funnel that records the cause. The cause is carried rather than inferred from the resulting status, because several causes land on the same one: a deploy that failed and a withdraw that failed both read `failed`, and they mean opposite things about whether the deployment is still on the node. Recording is skipped when the status did not move, so a reconciler tick that re-asserts a state it already reported does not append a second event describing the same fact. The write still happens either way. A terminal answers the request the target says it was given, not whatever the Blueprint currently wants: an acknowledgement matched against the current intent would accept work for a revision that node was never sent. The first deploy to a node creates its target, since a Blueprint application has no targets until something is sent somewhere. Recording never fails the deployment. The rollout already happened, and turning a bookkeeping problem into a stuck rollout would be the worse outcome. The four reconciler observations record what was seen and nothing else. Preview cleanup stays outside this path deliberately: it reverses a projection nobody deployed, so recording it would report removals that never happened. * fix(git): record an Inline Blueprint coming into existence Creating a Blueprint inserted its application row directly, so the history began at the first intent and described an application nothing recorded coming into existence. Every other application-creation path emits the activation event, and the event's own definition covers this case: an Inline Blueprint inserts the application, the first intent, and the first candidate. Activation goes through a transition now, which also enforces the one thing the direct insert could not: a Blueprint gets one live application, refused rather than silently duplicated. No target is created. A Blueprint application has no targets until something is deployed somewhere, unlike a Direct one which always has the node its stack lives on. * fix(git): file withdraw failures as withdraw failures Four of the five withdraw failure paths were recorded as failed deploys. Only the thrown one was tagged correctly, and it is the rare case: a delete-lock conflict, a network error, a remote 409, and a remote non-200 all return rather than throw, and all four were filed under deploy. That inverted the thing this funnel exists to prevent. Recording a deploy failure clears the in-flight operation, so a routine "another operation is already in progress" wiped the withdraw that had just started and left the target reporting a failed deploy while the deployment was still on the node. A withdraw refused because the on-disk stack is not ours is now its own cause too, rather than borrowing the deploy-side name conflict. The status guard no longer suppresses starts. A start writes the identity that terminals are matched against, so skipping one because the row already read `deploying` let a later acknowledgement answer a request that had been superseded, reporting a node as converged on a revision it was not running. The guard now covers observations only, which is where repetition is the reconciler re-asserting a state it already reported. A newer deploy supersedes an older one so a redeploy of a stuck request can take over; anything else in flight is still refused rather than displaced with no terminal event. A rejection is logged differently from an infrastructure error, because a target the model keeps refusing holds its active slot and stops recording anything further, and that is worth seeing. * feat(git): migrate pre-existing Blueprints into the revision state Blueprints that predate the model are brought in at boot: an application, an intent describing what the Blueprint currently asks for, and a candidate marked as coming from the legacy inline record. None of it is an acknowledgement. The Blueprint revision is carried for display and nothing else, because a revision number is not evidence that any node is running what it names, and recording it as agreement would report a fleet as converged on an intent nobody verified. No targets are created for the same reason, and the candidate's required set is empty: migration records what is asked for, never which nodes currently satisfy it. An approval authorizes the intent it was given for, so a Blueprint that was never approved, or edited since, is recorded as needing reapproval rather than left blank. Blank would read the same as an approval that is merely absent. Replay is decided by the same scope, schema version, and fingerprint the Git migration uses, and a Blueprint the new path already described is skipped outright, since its rows were written with proof this pass does not have. Blueprints migrate in their own pass. Coupling them to the Git migration would let one unreadable Git stack keep every Blueprint outside the model. * fix(git): contain the blueprint stack-directory probe at its call The ownership probe resolved a path from the blueprint name and checked it through the shared helper, which the security scan does not credit as a barrier. The check now sits with the call it protects, matching the form used elsewhere for the same reason. The path was already validated, so this changes no behaviour. It surfaced now because an earlier commit in this branch rewrote the file's line endings, which made every line read as new and brought a pre-existing call into the scan's changed-code window. * feat(git): read GitOps history and carry revision state on source reads Adds the instance-local history API and the additive revision fields the source reads have been missing. `GET /api/git-sources` and `GET /api/stacks/:stackName/git-source` now carry `gitopsRevision` and `stackResourcePresent`. Only the instance that owns the files can say whether a stack directory is really there, so that answer travels with the response rather than being inferred by a reader that has never seen the filesystem. It is read through the strict stack listing: the lenient one answers a failed directory read with an empty list, which here would read as every stack having vanished and would turn an unreadable disk into an empty audit trail. Two history routes land with them: `GET /api/git-sources/history` across stacks, and `GET /api/stacks/:stackName/git-source/history` for one. Both page on `(created_at, id)` so rows written inside the same millisecond survive a page boundary, and both bound how far one request will scan. A malformed cursor or an unusable filter value is refused rather than ignored, because answering "show me the failures" with everything reads as an answer instead of a non-answer. Authorization is decided per row. A row reduces to a stack read only when it names a stack, its application is live, and the stack is present on disk; anything unprovable stays with Admin. The cross-stack routes classify every row, while the per-stack route is authorized whole by name, so reading a stack shows its full history including entries from an earlier application. That distinction is carried in the scope type rather than a flag, so skipping the row classifier without pinning the query to the authorized stack cannot be expressed. History entries record the fields each transition moved rather than a whole revision, so lifecycle for the classifier comes from the owning application row. That keeps authorization off the recorded payload entirely: an entry whose detail cannot be read still returns its identity, stage, and outcome with a stated limitation, and the decode failure is logged rather than passed over in silence. * feat(git): let auditors read GitOps history they cannot tie to a stack A history entry that cannot be tied to a readable stack fell to Admin, which left the auditor role seeing exactly what a viewer sees on the one surface it exists to read. History entries are an audit trail: insert-only, and recording the actor, trigger, stage, and outcome of every transition. The request audit log is already gated on the audit permission, so an entry whose audience cannot be narrowed now falls there too. Withholding it protected nothing in any case, since the request log already records that a Git-source mutation happened and who made it. The source collection deliberately keeps its Admin fallback. Those rows are live Git configuration, not a record of events, and a mandate to audit does not extend to reading the repository, ref, and credentials policy of stacks that were deleted or never finished being created. The fallback is now part of the requirement type rather than a role comparison, so each surface states which audience it falls back to and the compiler requires every case to be answered. * feat(git): correct remote node identities on GitOps reads A remote instance numbers its own nodes and has never heard of the hub's numbering, so every node id it reports is a statement in its own namespace. Read straight through, a hub joining two nodes showed two different machines as the same node. Four GETs now take a second hop that buffers the response and corrects those identities: the git-source list and detail, and both history routes. Everything else keeps streaming exactly as before, which is the reason this is a separate hop rather than a mode of the existing one: logs, downloads, and event streams must never be buffered, and the enumerated positions rewritten here do not appear in them. Only JSON numbers are replaced. A null node is preserved, since "no node" is a fact the remote is entitled to state and inventing one there would claim a placement that does not exist. Strings, application ids, and stack names travel untouched. The hub re-authorizes what comes back. A remote authorized its rows for the machine account the hub proxies with, which says nothing about the person behind the request, so every row is classified again against the signed-in user. To make that possible without the hub holding another instance's database, each history entry now carries the lifecycle of its owning application alongside the stack-presence answer already there. Both are validated fail-closed, and a verdict a remote might volunteer is ignored: the hub decides, the remote supplies evidence. Asking one node for another node's history is refused before the hop rather than forwarded, because the remote would answer about itself and the page would read as an answer to a question nobody asked. When the node asked for is the node being talked to, the hub translates that into a flag the remote resolves to its own default node. Only the hub may set that flag, and a remote honours it only on a proxied hop. Every way this hop can end converges on one terminal answer: rewritten, passed through, too large, undecodable, failed upstream, or abandoned by the client. A body the hub could not read never borrows the remote's success status. * test(git): assert the unlinked git-source response field by field The detach test compared the whole response to `{ linked: false }`, which stopped holding once that route started carrying the additive revision fields. The equivalent backend route test was updated when those fields landed; this one was missed. Asserted field by field rather than loosened to a partial match, so the two new fields are actually checked: a detached stack has no application to project while its directory is still on disk. * feat(git): carry GitOps revisions on blueprint, node, and drift responses Completes the additive JSON half of the revision-state read contract, so the Blueprint, node, and drift surfaces report GitOps state through the same projection the Git-source routes already return instead of leaving it unreadable. Blueprint list, detail, create, update, and pin carry gitopsRevision. Node-label add, cordon, uncordon, and node delete carry gitopsRevisions, ordered by blueprint id and covering only the Blueprints the mutation actually moved: an edit no selector reacts to reports an empty list rather than invalidating the whole catalog. Node deletion reads its Blueprint owners inside the deletion transaction and before the tombstone, which is the last moment a target row can be traced back to an application, and an orphaned target is logged rather than being silently indistinguishable from a Direct one. Both DELETEs that answered 204 still answer 204. The drift GET and re-check gain the same field alongside the existing ledger, which is untouched; no GitOps class is written into stack_drift_findings. Reads and mutations treat a projection fault differently, on purpose. On a read the revision is part of the answer, so a fault surfaces. On a mutation the write has already committed, so the revision is decoration and is built defensively: letting it throw would answer a successful cordon or a completed node deletion with a 500, sending the operator to retry a hard delete that already happened and be told the node does not exist. It degrades to an empty list and logs. Because both drift routes answer with that projection, the identity hop now intercepts them too, matching its route table per method instead of rejecting every non-GET. Left as they were, one object would carry the hub's node numbering or the remote's depending only on which route asked for it. The allowlist names a single mutation rather than opening a verb, and the hub's collection filter still keys off the git-source paths alone, so drift payloads are rewritten without being re-authorized: they are per-stack and already authorized by name before the hop. Tests cover the exact shapes, the preserved 204s, revision ordering, the empty-list cases, node deletion reporting its tombstoned Blueprints and surviving a projection failure, and the proxy's new route matching, drift rewrite, and filter exclusion. * fix(git): resolve every application a GitOps surface can own Three gaps found while building the additive JSON, all the same shape: an id or a sentinel stood in for a row, so "there is nothing here" and "the thing that should be here is unreachable" collapsed into one answer nobody could tell apart. A Blueprint application is stored with stack_name NULL, so no lookup by stack name could reach it, while the reconciler materializes every Blueprint as a stack directory of that name. A stack's Drift tab and Git panel therefore reported no GitOps at all for a stack GitOps was actively managing, while the Blueprint page reported a live application for the same thing. Stack surfaces now bridge the two through the deployment row for the asking node. The deployment row is what makes it safe: Blueprint and stack names share one namespace, so matching on name alone would let a Blueprint claim an unrelated stack of the same name on a node it never targeted. A retired application was unreachable for the same reason, because every entry point filtered to the live rows. Tombstoning deliberately keeps the configured identity and SHA pointers as frozen facts so the projection can still say what an application was, and the source deriver has a not_live status waiting for exactly that, but nothing could hand it one. A detached Blueprint or stack now reports what it was instead of reading as one that never existed. The not-applicable projection was typed so it could not carry a reason, which made a row that vanished between resolving it and re-reading it by id identical to a stack the model was never asked about. That one field widens to hold limitations, and the missing case now says so with the id as evidence. Every other missing-row path in the deriver already worked this way. Also logs, rather than silently skipping, a live application whose blueprint row is gone: with cascade off that is a real integrity fault, the placement did move, and the response would otherwise report that nothing had. Cordon and uncordon are deliberately left reporting nothing. A cordon governs whether new placements may be made, not what a Blueprint asks for, and the reconciler applies it only to new placements, so revising intent for it would invalidate acknowledgements fleet-wide over a change that evicts nothing. Two comments implied otherwise and now state the contract. * fix(git): keep stack-state resolution out of the Git-source read path Review of the previous commit found the fallback it added had reached further than intended. The resolver is now split by the question it answers. projectStackRevision answers "what Git source is attached to this stack" and stays Direct-only. projectManagedStackRevision answers "what manages this stack" and is what the drift routes use. The Git-source routes must not be answered with another application's identity, and there was a second reason: the row classifier takes the stack name from the Git-source row but the lifecycle from whatever the projection resolved, so a Blueprint application arriving there would have turned an Admin-only row into one any stack grant could read. The Blueprint that unlocked it could be the very one holding name_conflict because of that stack. The Blueprint-to-stack bridge also needed the right predicate rather than a present deployment row. name_conflict is written precisely when a stack of that name already exists on the node and Sencho does not own it, so guarding only against withdrawn admitted exactly the collision the bridge exists to prevent. It now requires last_deployed_at, which proves this Blueprint wrote the directory, and excludes name_conflict and withdrawn: the same predicate the delete and withdraw paths use. Only a live Blueprint application qualifies, since a retired one has no claim on the directory. The detached fallback no longer resolves deleted applications. Detached means the files are still on disk and still describe that stack; deleted means the stack is gone, so any directory of that name now belongs to something else and reporting the old repository and SHA against it would disclose one stack's Git identity through another's name. The authorization side already refused deleted rows for that reason, so the two now agree. Ordering breaks ties on rowid rather than a random UUID, and two partial indexes cover the new lookups, which the existing live-only unique indexes could not serve. The not-applicable variant's limitations are readonly, so pushing onto the shared frozen instance is a compile error again rather than a runtime throw. Tests cover a Blueprint claiming a stack it never deployed onto and one it hit a name conflict on, a detached Direct source projecting not_live, and a deleted one staying unresolved for a reused name. Drift fixtures move to afterEach so a failing assertion cannot leak state into the next test. * fix(git): stop the stack resolver falling through proven ownership A second review round found the resolution chain could answer with the wrong application, and that part of the previous commit was reading a state nothing writes. The Blueprint bridge returned one value for two different facts. Once the deployment row proved a Blueprint had written the directory, a missing application row for it still answered "no Blueprint here", so the chain continued to the detached-Direct lookup. A stack that once had Direct Git, was detached, and whose directory a Blueprint later took over would then report the old Direct application's repository, ref, and SHA as that directory's state. The bridge now separates "not mine" from "mine, but broken": the second logs and returns its own limitation, so the fault is visible and no unrelated identity is offered in its place. A Blueprint projection also carried the whole placement roster into a route authorized by a grant on one stack name. Targets are now scoped to the node being asked about, which is both the safe answer and the accurate one, since the question is what manages this directory here. The detached-Blueprint lookup, its index, and its fallback are removed. Blueprint retirement writes deleted, never detached, and the only detached writer is the Direct detach path, so the getter could never match and the index covered an empty set. The test that seemed to cover it built the state by hand and was green against something the product cannot produce; it now drives the real delete route and asserts what that path actually writes. The Git-source resolver goes back to live applications only. Detach removes the source row in the same transaction that tombstones the application, so a source row beside a detached one is not producible, and the route's row classifier takes its stack name from the source row but its lifecycle from the projection. Keeping that resolver narrow is what stops a resolution change from quietly moving who may read a row. The comment saying so is restored and now explains why it must stay true. * refactor(git): share the GitOps application fixture across the route tests Two test files carried near-identical 58-line GitOpsApplicationRow literals that differed only in a repo URL nothing asserted on. Both now import one helper. The drift tests also repeated a Blueprint seed and its activation across four cases, and re-imported DatabaseService per test; those become local helpers and one hoisted import, which lets the cleanup hook go synchronous. Node lookup throws on a missing default instead of casting. projectBlueprintRevisions is module-private again: its only caller is the committed-projection wrapper beside it. A third copy of the same fixture remains in git-source-routes.test.ts. It is left alone deliberately: it pins timestamps at 1 rather than now, and that file has ordering-sensitive history assertions that would need auditing first. * feat(git): mirror the GitOps revision contract on the frontend Adds the client-side read contract for the GitOps revision projection, with no consumers yet. types/gitops.ts is a hand-written mirror of the backend read contract, in the same convention as the other domain type files: the frontend never imports backend. The projection's two arms are named separately so a component can take the live one as a prop without re-narrowing, and the absent arm keeps its absent keys rather than nulls so reaching for a lifecycle status without narrowing is a compile error. The limitation code stays an open string, since the backend adds codes without a schema bump and an exhaustive switch would silently stop rendering the newest ones. lib/gitopsState.ts is the one place a facet status becomes words and a colour, so a state reads the same in a sidebar tooltip, a panel banner and the drift tab. Both maps are keyed on the closed status unions, so widening the mirror without naming the new state fails this build. Every line of copy states the condition the deriver actually tests, which is not always what the status name suggests. recovery_required is a recovery already running rather than one that is needed. candidate_ready is reached only when review is not required, so it is ready to apply, not ready to review. synced_and_healthy is also reached with the health gate switched off and never compares the deployed generation against the accepted one, so it claims neither. never_reconciled means nothing has been accepted; a fetch that produced no materialization still records its commit. pendingSourceStatus keys "an update is waiting" on the candidate pointer rather than the status name, because source_reconcile_required is reachable both from a stale candidate and from an accepted generation with no candidate at all. Retirement is excluded first: tombstoning keeps the candidate pointer as a frozen fact, so a stack detached mid-review still carries one. absentFault separates the two facts the empty projection carries. Empty limitations means the model was never asked about this stack and the right rendering is nothing; a non-empty list means an application that was expected could not be reached. A live application's limitations are caveats on state that is being reported, not faults, so they are deliberately excluded. * feat(git): name the Git source state on the stack and sidebar surfaces The sidebar indicator and the anatomy source row both read a raw pending commit pointer, so a candidate blocked by local conflicts, held for review, or stale against the configuration in force all render identically to one that is ready to apply. The projection distinguishes them; nothing was reading it. refreshGitSourcePending now derives each stack's state from the projection and the pending map carries that state rather than a boolean. Presence in the map still means exactly what it meant before, because it is keyed on the candidate pointer rather than the status name: source_reconcile_required is reachable both from a stale candidate and from an accepted generation with no candidate, and only the first is something to review. One narrow fallback remains, for a row with no projection at all: a failed GitOps write is logged and swallowed while the pending commit still commits, so the flat pointer is the only thing that can answer there. Wherever a projection exists it is the sole authority. The sidebar keeps one indicator, in the same slot, at the same size and colour, with the same position in the priority ladder. Only the tooltip changes, so a blocked plan now says so instead of reading as an ordinary update. A test asserts the rendered indicator is identical across states, which is what keeps the rendered sidebar unchanged. The anatomy source row keeps its pulsing dot alone for the ordinary case and adds the state word only for the states the dot cannot express. Inline rather than in a tooltip, since this is the reason something is stuck. * feat(git): surface derived GitOps state in the Git source and Drift panels The Git source panel's pending banner had two things it could say, read from a raw commit pointer and a single blocked boolean. It now names which of four states the candidate is actually in, so a plan blocked by local conflicts, one held for review, and one gone stale against the configuration in force stop rendering as the same ordinary update. The short commit sha stays. A new card above it reports an application the projection could not reach. That case renders as nothing at all today, which is indistinguishable from a stack the model was never asked about. Empty limitations stays silent, because that is the ordinary answer for most stacks and a header over an empty block would be worse than nothing. The summary block gains one row naming the source state. It is the first place the panel can show applying, retry scheduled, suspended, recovering, or a detached source, none of which have a pending commit and so never reach the banner. Both existing rows are untouched and the last applied commit stays a display fact. The revision lives in its own state rather than on the Git source type, because the PUT that saves this panel answers with a bare source and no revision. A save drops it instead of rendering a state the write has already moved. The Drift tab gains a third block below the two it already has. The compose versus runtime card and the since-last-deploy card keep their exact positions and copy; the new block answers a different question, which generation this node was asked to run and whether it got there. A Blueprint-owned stack shows its target rows and no source card, since a Blueprint application has no Git source and inventing one would be a claim the model never made. The drift class list renders against the type and expects no rows, because nothing populates it yet. * feat(git): declare revision fields on the blueprint and node clients Types first. The blueprint list, detail, create, update and pin responses all carry a GitOps revision, and the node label add, cordon, uncordon and delete responses carry a list of them. None of that was modelled, so the fields were being dropped silently and the next reader would have taken these for bare records. Most of it is declared and deliberately unread, with the reason written where the type lives. Create, update and pin are followed immediately by a re-read of the catalog or the detail that carries the same projection, so rendering the mutation's copy would show the same fact twice with one of them stale. Cordon and uncordon always answer with an empty list by design, because a cordon governs whether new placements may be made rather than what a Blueprint asks for, and the reconciler leaves existing deployments where they are. The comment on that type is the point: the risk there is a future reader building a consumer for a list that is never populated. Two places do report something. Deleting a node and adding a node label both re-place Blueprints, and both now say how many. The count only, and only when it is not zero: an empty list means both that nothing moved and that the projection faulted after the write had already committed, so it can never be reported as the first. Blueprint detail gains the same unreachable-application card the Git source panel has. A Blueprint with no live application row is a fact nothing in the product could previously express, which is a different problem from having two ways to say the same thing. * fix(git): close the gaps review found in the GitOps frontend consumers Six defects, each verified against the code before being fixed. The pending-map read assumed every /git-sources row carries a projection. That route is proxied, so a node predating the model answers rows without one, and dereferencing it threw inside the loop. The catch swallowed the throw, the map was never set, and every stack's Git indicator froze at its previous value for as long as that node was selected. The field is optional now and a row without one falls through to the same branch as a row with nothing to project. That fallback also branched on the arm alone, so a projection reporting an unreachable application plus a stale commit pointer was reported as a candidate ready to apply. It now excludes a faulted projection: a fault means an application was expected and could not be read, so the pointer is not evidence that anything is ready, and naming a state there would be a guess. Detaching a Git source cleared the source but not the revision, and the pending card is derived from the revision alone. The panel kept advertising a waiting commit for a stack Git no longer managed, behind a Review button that silently did nothing. A read that threw had the same problem across stacks: the panel is reused, so stack A's state could render under stack B's header. The panel also lost the flat-pointer fallback the sidebar keeps, so the two surfaces disagreed: the sidebar would show an indicator and clicking through showed no card. Both now apply the same rule. Node delete parsed the success body unguarded. The delete has already committed at that point, so a malformed body would have reported a completed deletion as a failure and skipped the refresh, leaving the deleted node on screen. Tests: five mutations that previously survived now fail. The zero-pixel claim compared the wrong element and passed when the indicator vanished entirely; the stale-read test asserted during the loading window, when the body is skeletons regardless; the sidebar passthrough test proved only that an indicator existed, not that the state reached it. Added coverage for the drift row, an unknown node, the not-applicable source guard, the save clearing, the Review button, and a live application carrying a caveat, which must not read as a fault. The fixture set drops two source statuses whose identity defaults describe a state they cannot be in, and gains a drift-item builder. * refactor(git): share the GitOps fault card and live-facet derivation The unreachable-application card was built inline on three surfaces, and the copies had already drifted: the Git source panel used one icon while the Drift tab and the Blueprint sheet used another, so the same failure rendered two different ways. It is one component now, which also owns the state key and the test id those surfaces assert on. Deriving the live source facet was likewise repeated, along with the two semantic decisions behind it: the absent arm has no facets, and a source facet of not applicable means a Blueprint owns the stack rather than that something is missing. Both now live next to pendingSourceStatus, which already encoded the same exclusion. The Git source panel's seven derived values collapse to three, with the pending-commit rule extracted to a named function so the four cases read straight through instead of as nested ternaries. Behaviour is unchanged in every case. The two flat-pointer fallbacks are deliberately not unified: they look alike but the panel also treats a live application whose source facet is not applicable as unanswered, and merging the predicates would change what that case renders. * feat(git): count and announce committed GitOps transitions Every history row that is actually inserted now produces one metric increment and one state-invalidate event, so the surfaces that read GitOps state hear about a change instead of waiting for the next poll. Announcement is buffered and drained on a macrotask rather than fired inline. better-sqlite3 is synchronous, so waiting for the macrotask puts the drain after the transaction that wrote the row, and after any outer transaction wrapping it, without having to detect which nesting depth it is in. The drain confirms each row is still present before announcing it, so a rolled-back transaction says nothing on its own, and the insert declines to queue a dedupe replay, so a retry says nothing either. The broadcaster is injected at startup rather than imported, keeping the GitOps layer free of a cycle back into the notification stack. Counters are process-local and in-memory, and their keyspace is finite by construction: history stages are now a closed union that the build enforces, and outcomes were already a closed set. The payload names no stack, node, repository, or actor, because a counter carrying those would be an audit trail with no retention rules and no per-row authorization, which is what the history routes provide. GET /api/gitops-metrics is Admin-only and instance-local, so selecting a node answers with that node's counters. On the client, a gitops-scoped invalidate refreshes the derived state through a 250ms trailing window, matching the existing stack refresh. One operation commits several transitions in a row and a first-boot migration commits a great many, so refetching per event would thrash the API for a picture that only settles at the end. * feat(git): show GitOps source state on the stack dashboards The dashboards list every stack on a node, which is where a fleet-wide reading of Git state is most useful and where, until now, a Git-backed stack looked exactly like a local one. Each row that the model has something to say about carries a chip naming its source state. The chip reads from the same status vocabulary the panels and the sidebar use, so a stack cannot be "pending update" in one place and something else in another. It sits beside the stack name rather than replacing the source column: the column says where the files come from, the chip says what GitOps makes of them, and those are different facts. The label is a word and the title is a whole sentence, so the state never rests on colour alone; the phone rows keep the word on screen, since touch has no hover. The join is by stack name, which is what the dashboards have, and its source is the same proxied route the sidebar reads. A row without a revision, a Blueprint-owned application, and a projection fault all leave the stack unbadged rather than inventing a state for it. State arrives by announcement rather than by poll, since it only moves on a transition, and a node switch blanks the map first: stack names repeat across nodes, so a slow answer for the node just left is discarded rather than allowed to label the wrong stacks. * feat(git): say which part of a GitOps state could not be proven Eighteen conditions can qualify a live projection: a manifest that does not match the branch configured now, an approval that could not be restored after a recovery, an artifact record that has gone. Every one was already recorded and none of them reached the operator, so a state with a hole in it read exactly like one without. Each now has operator wording under the state it qualifies, checked against the site that emits it rather than against the code's name, since several names describe something narrower or wider than the condition actually tested. The stored messages stay out of the UI: they are written for a log reader, and some are raw decoder errors. The presentation is deliberately quiet. A caveat is not a failure: the state above it is real, and one piece of evidence behind it is missing, so the reader learns which part to distrust without being told the whole thing is broken. Faults keep their own card, because those replace the state rather than qualifying it, and the two arms are read through separate helpers so neither can be rendered as the other. A code with no wording names itself rather than vanishing, so a node running ahead of this build degrades to something honest instead of reporting full confidence in a state its own backend flagged. * fix(git): record a stateful first placement and keep a half-built Blueprint editable Two faults that today's product cannot reach, and that the Git-backed Blueprint mode would reach immediately. The live-application lookup answers with an application that is active or still being created, because its other callers ask whether the Blueprint already holds the live slot, where a half-built row counts. Three producers passed that answer straight into transitions that accept only an active application and reject anything else by throwing, inside the caller's own transaction. A Blueprint edited or pinned while its application was still being created would have failed with a server error and rolled the Blueprint write back with it. The producers now narrow to what they actually require, through one shared predicate, leaving the slot check honest. Separately, a stateful placement is held for operator review before anything is deployed, so no target exists when that hold is recorded, and the observation was dropped for want of one. The hold left no trace: no history row, and nothing on the target to say the node had been asked to hold anything. First contact now creates the target, exactly as the first deploy does. Nothing else changes with it: no intent, no generation, no operation, and connectivity stays unset, because a node that has only been asked to hold something has not been contacted. Observations that follow a deploy already have a target, so they are unaffected, and a drift or evict report for a node nothing was ever sent to is still dropped. * fix(git): keep the Git source panel's state after a save The save answers with the source row and no revision, so the panel had nothing to replace its copy with and dropped it. The result was a stack that had just been reconfigured showing no GitOps state at all until the panel was reopened, which reads as a stack the model knows nothing about rather than one whose state has just moved. It re-reads instead. Keeping the old copy was not an option either: a material configuration change clears the staged candidate server side, so the state genuinely has moved and the panel would have gone on offering a commit that is no longer there. Only the server can say what replaced it. The alternative was returning the revision on the save itself. That route is proxied, and the identity hop rewrites node numbering for an enumerated set of routes, so adding a revision to a response outside that set would hand back a remote node's numbering unrewritten. Re-reading costs one request on a low-frequency action and needs no change to the hop. * docs(git): document what GitOps state means for an operator A commit SHA says which files Sencho wrote. It does not say whether that commit was accepted, whether something newer is waiting, or whether an operation was interrupted halfway, and until now the docs had no words for any of that. Git Sources gains a source-state table and a section on what happens when part of a state could not be proven, since a stack with missing evidence looks identical to one with complete evidence unless the product says otherwise. Drift Detection gains the gitops section and explains why it answers a different question from the two signals above it: those compare files and containers as they are now, this reports what has been proven over time. The dashboard page describes the state chip and, importantly, that it and the row tint are independent, so a healthy row carrying "pending update" is not a contradiction. The Blueprint page now separates its revision counter from GitOps state. The counter labels the spec; it says nothing about whether a given node has got there, and reading it as fleet truth is the mistake the note prevents. The tutorial's verification step names the source state it should show, so a reader can tell the difference between files written and a commit accepted. Audited and left unchanged: health-gated updates and fleet federation. Neither presents a SHA or a revision as canonical GitOps truth, so neither needed correcting. * docs(git): name which saves actually clear a staged commit The note claimed saving the form always clears the staged commit and moves the state to reconcile required. Only a change to what gets materialized does that: the repository, the ref, the compose paths, the project directory, or the env sync. Changing the token or the apply behavior leaves a staged commit alone, because neither changes what would be written, and telling an operator otherwise would have them pull again for nothing. * fix(git): say when a committed transition has nobody to announce it to Two silences worth breaking, both found reviewing the error paths in this slice. A server that never installs the event sink still counts every transition and still writes every history row, so the only symptom is that no client ever refreshes and the UI is quietly as stale as it was before any of this existed. It now says so, once rather than per row, since a boot migration would otherwise fill the log with one fact. The dashboard's source-state fetch dropped a non-ok response without a word. A refusal there freezes every badge at its last value, which looks exactly like a fleet where nothing has changed, so the status code is logged rather than inferred from badges that stopped moving. * fix(git): close the defects review found in the announcement slice Five findings, two of them able to reach an operator. The event sink was installed one line after the deletion reconcile, which tombstones applications and targets and awaits inside its own loop. A drain therefore landed while the sink was still absent, so those boot-time transitions were counted and never announced, and the warning added for an unwired sink would have fired on every boot with a prepared deletion intent: the fastest way to teach an operator to ignore it when it means something. The sink now precedes every reconcile and migration pass. The first-placement target was created outside the transition that records it. The observation runs in its own savepoint and can refuse, and the caller deliberately lets the deployment commit whatever the record says, so a refusal left an active target with no generation, no stage and no history: a placement relationship the model never established, which the delete path would later tombstone as if it were real. Both writes now succeed or fail together. The badge looked its status up in a map closed at compile time, behind two casts, while the value arrives over a proxy from a node that may run a newer vocabulary. An unmapped key dereferenced undefined inside a stack row and would have taken the whole table down with it. The facet and its status are now a discriminated union, the lookup is optional, and an unrecognised status renders nothing, which is what the join already does for a stack it has no state for. The dashboard's source-state loop derived each row inside the shared try, so one row in an unexpected shape abandoned the loop and froze every badge at its last value with nothing on screen to say so. Each row is now guarded on its own and the count of unreadable ones is logged. Two tests claimed things they did not check: both asserted a word was on screen with a matcher that also matches screen-reader-only text, so a compact badge would have kept them green, and one compared a label by containment where the shorter label is a prefix of the longer. Both now assert the visible node by equality. Added coverage for the window event the dashboard actually refetches on, an unknown status, a row that cannot be read, node-switch blanking, and a caveat recorded twice. One fixture built a status pair the deriver cannot emit and was corrected. Also: the guard narrowing Blueprint producers to an active application excludes nothing today, because every Blueprint-mode application is inserted as active and only Direct mode can be creating. The guard stays, since the getter's slot semantics and the transitions' requirement have drifted apart once already, but its rationale and its tests now say plainly that they pin a defensive guard rather than reachable behaviour. * refactor(git): make three runtime guards visible to the compiler Type review found the same shape three times: a correct guard whose necessity was asserted in a comment the type system could not see. This project does not set noUncheckedIndexedAccess, so each of these read as dead code to anything that trusts the types, and the guards protect the behaviours that matter most here. The limitation copy map typed its values optional, so the fallback for an unrecognised code is now something the compiler requires rather than something a comment explains. The badge reads its status through partial views of the two vocabularies, so a miss is a fact TypeScript derives: deleting the guard that stops one unknown status taking down a stack list now fails the build, where before it compiled. Neither needed a cast. The event payload has to be a type alias rather than an interface, because only the former gets an implicit index signature and the broadcaster takes an open envelope. That requirement now fails at the declaration instead of surfacing in the startup wiring as an unexplained index-signature error. Also added the reverse of the copy-coverage check: a code retired from the backend leaving stale wording behind was invisible, since the fallback only fires for entries that are missing rather than ones that linger. * fix(git): make the state cards survive an unknown status, and correct the copy Comment review found the protection added for the dashboard chip was claimed more widely than it held. Five other surfaces indexed the status vocabularies directly and passed the result straight to a card that dereferences it, so the exact input the comments said would take a list down still threw in the Git source panel, the Drift tab, the sidebar tooltip and the stack anatomy row. The card now takes an optional state and renders nothing without one, which also means a new surface cannot reintroduce the dereference by rendering a card the ordinary way, and the remaining four read through the lookups. Copy corrections, each against the site that emits it rather than the name of the code: The manifest-identity caveat said the manifest named a different repository or branch. It also fires when the manifest has no identity block at all, or identifies another node or another stack, so it now says it does not identify this stack on this node from the repository configured now. The last-known-good artifact caveat covered a mismatch but not the row being gone, which is the other half of the same condition. Doc corrections: a Blueprint-delivered stack was described as carrying the dashboard chip and a source state on the Drift tab, and it carries neither, because a Blueprint has no Git source of its own. Per-node Blueprint state was attributed to the detail sheet's GitOps section, which only reports what could not be read or proven; the Deployments table is where per-node status lives. Clearing a staged commit was said to always land on reconcile required, but a stack that has never had a commit accepted lands on never reconciled. The caveat lines were described as sitting under the state when they render above the form. And the pending banner was still documented as webhook-only, which this branch changed: it now appears for any staged commit and its heading is the source state. Also corrected two claims in the code: the observation branch does not skip for a Blueprint that migration brought in, because migration creates no targets, and the only refusal reachable from that call site is a tombstoned target. * refactor(git): simplify the announcement slice without moving any behaviour A simplification sweep over the new code, all of it behaviour-preserving and re-checked by breaking each guard afterwards. The drain now separates policy from mechanism: it decides whether a row survived, counts it, and either warns or hands it to a small announce helper that owns the envelope and its error handling. The row-to-state pass in the dashboard hook lifts out of the fetch, so one function does the request and another does the derivation. The caveat dedupe collapses to a set, which preserves first-insertion order and so keeps the ordering its own docstring promises. The badge renders one span with a conditional class rather than two that differed only by class. The metrics service drops an internal type that was byte-identical to the one it exports. One assertion goes: the observation branch tested membership and then re-asserted the same fact to index the stage map, and a type predicate lets the compiler carry it instead. Everything the sweep was told to leave alone is intact: the buffered drain, the row-existence re-check, the injected sink, the one-shot warning, the partial-view lookups and their guards, the optional copy map with its fallback, and the generation counter with its per-row guard. * fix(git): close audit findings in the GitOps revision model Five corrections from the pre-merge audit of this branch. Migration no longer certifies a commit it cannot prove. The manifest read now carries its resolved commit, and a manifest that names a different commit than the source row records as applied is refused: the applied directory materializes one commit while the row names another, so trusting them together would mint a generation pointing at files that are not the ones it claims. A manifest that names no commit at all, which is what adoption writes, is a separate answer rather than a disagreement, because reporting it as one would name a commit the manifest does not contain. Both are recorded as evidence and the projection asks for a fetch. Create cleanup proves containment against the real filesystem before it deletes. Every check on these paths was lexical, so a symlink or Windows junction above the target read as contained while the recursive delete followed it out of the managed area. Resolution keeps "is not there" apart from "could not be read": only a genuinely missing path lets the walk climb to an ancestor, because treating an unreadable one as absent would infer containment for the single path whose link status could not be established. Both sides are resolved, so relocating the data directory onto another volume keeps working, and an area that does not exist at all is nothing to delete rather than a suspected escape. The marker write takes the same barrier as the marker delete, so a link cannot be written through and then refused on the way out, wedging the stack name. Startup stops while an interrupted create is unresolved, matching the restore reconcile above it. A create that could not be settled leaves a stack directory the deploy path cannot tell apart from a finished one, so the alternative to stopping is letting a scheduler, webhook or operator act on a half-built stack. Only that blocks: once the staged directories are gone the create is torn down, and a staging marker nobody could unlink is reported rather than thrown, so one failed unlink is not the difference between an instance that boots and one that does not. Per-stack history no longer exposes a predecessor through a reused stack name. The grant covers the application holding the name now, which is what keeps a stack's own entries readable while it is still being created; every other row on that name is classified per row, so entries belonging to an application that held the name earlier need the audit permission, as they already did across stacks. A detached predecessor is still readable, which is the classifier's own standing decision about detach rather than a gap here, and is now pinned by a test. Blueprint observations reach a reader. The reconciler recorded state review, evict blocked, drifted and correcting against the target and nothing projected them, so a deployed Blueprint could report itself as never applied. The runtime facet now projects all four, below the states a live or failed mutation puts the target in and above the pointer checks, and any later transition supersedes the observation. The map is declared total over the stages the reconciler can record, so a stage added without a projection fails the build rather than silently reading as never applied again. * fix(git): confine managed-area cleanup to each stack's own location Cleanup proved only that a resolved path landed somewhere inside the managed area. That is satisfied by every other stack and every other node in it, so a link from one stack's generations directory into another's passed the check while the delete took a generation belonging to someone else. The check is now positional: the managed area is resolved once, so relocating the data directory onto another volume still works, and every segment below it must be reached without redirection. The create-path sinks use it on the write as well as the delete, so a claim cannot be written through a link and then refused by the hardened delete, and the manifest service's pre-existing deletion sinks (generation pruning, boot-sweep orphan reaping, detach staging and finalization, and whole-area removal, including after a restored snapshot) run it before their recursive deletes too. Each of those sinks also keeps a literal containment comparison beside the positional check, because static analysis credits only a comparison at the call and reports the delete otherwise. A refusal names both paths in the log, because it can hold the boot gate. Boot recovery dropped the checkpoint of an application that had left the creating state without first clearing its staging marker. That left a claim on the stack name with nothing to retry it, and every later create for that name was refused by a marker nothing could remove. The settled branches now share one exit, so the marker ordering holds for all of them. A detached application's history was readable on a stack grant because its files are still the stack standing at that name. Nothing in these tables can prove it still is: a Blueprint successor records the name off the application row, and a plain Compose stack recreated at the name leaves no trace at all, so no detection-based allowance can be made sound. Detach now moves an application's trail to the audit audience outright, the same answer deleted and creating predecessors get, while reading the stack itself stays where it was. * fix(gitops): close four audit findings in revision state and identity proxy A1: derive runtime and health against target desired_generation_id - deriveRuntime returns 'applied_not_deployed' when desired != deployed, so a stale deployment stays deploy-pending instead of reading synced_and_healthy - deriveHealth judges against desired_generation_id falling back to deployed when null; control case proves null-desired rows unchanged - collectRuntimeDrift emits the plan-pinned runtime drift item for exact/ qualified observation mismatches; equal/non-comparable observations emit nothing; ordering pin keeps deploy question first A2: stop upstream 304 responses from bypassing hub rewrite and reauthorization - identity proxy answers no-store on every terminal (rewrite, 204, 304, generated failures); validators (etag, last-modified, cache-control, expires, vary) no longer forwarded - conditional request headers (if-none-match, if-modified-since, if-match, if-unmodified-since) stripped on the identity-hop branch of the shared proxyReq handler; streaming hop untouched (optimistic-concurrency file writes depend on If-Match/If-Unmodified-Since) - supertest integration test drives the real middleware through a loopback capture server and asserts if-none-match absent while accept survives S1: migrate legacy query/userinfo URLs via secret-free path - parseLegacyRepoUrl strips userinfo, query, fragment instead of refusing; parseHttpsRepoUrl remains the strict gate for user-driven paths - migrationDirectSourceIdentity uses the tolerant parser; operational stack_git_sources.repo_url untouched; fingerprint convergence proven - trusted-manifest-on-legacy-URL test covers the worst real-world instance S2: emit runtime artifact drift for current evidence - top-level drift array carries the seven-class runtime item; frontend comments updated to reflect backend now emits runtime drift All regression tests added and passing. CI green (7,374 backend + 2,790 frontend tests; only pre-existing Windows EBUSY teardown failure unrelated). * test(frontend): fix DriftPanel drift item rendering test Update test assertions to match the new backend drift item format. The function now renders artifact_set expected identities as 'artifact <id> · <qualification>' and runtime_artifact observed identities as the raw identity string. * fix(gitops): emit a runtime drift item for desired-versus-deployed mismatch The projection reported applied_not_deployed with no entry in drift, so the canonical drift list contradicted the runtime facet it travels with. collectRuntimeDrift now emits the r27 generation-mismatch item for that state: desired generation as expected, deployed generation as observed, ComposeService as owner. The action mirrors what availableActions offers: deploy, unless an application-level fetch or apply is in flight or a recovery is in progress, which withhold deploying without removing the fact of the mismatch. The item clears once the desired generation deploys. The stale-deployment test pins the exact item shape, its stability across re-derivation, its convergence removal, and the withheld-action case; the ordering pin keeps the artifact observation suppressed while the deploy question stands. Frontend fixture aligns its artifact-mismatch example with the backend's none action. * fix(gitops): keep generation drift visible across failure and pause states The desired-versus-deployed drift item was keyed to the applied_not_deployed runtime status, but paused, failed, recovering, interrupted, and in-flight statuses all outrank the pointer comparison in the deriver. A failed deploy of a new generation over a running older one therefore dropped the drift report exactly while the old workload was still serving. The item is now judged from the pointers themselves: known and different means reported, whatever presentation status the target carries. Retired targets are excluded because nothing can rebind them, so their surviving pointer divergence would be permanently unresolvable noise. The action follows what the payload offers: deploy only when no application-level operation or recovery withholds it and availableActions lists deploying, none otherwise. The failed-redeploy regression test pins the exact item shape after a pre-mutation deploy failure, its stability across re-derivation, continued artifact suppression for the replaced workload, and convergence removal; a second pin keeps retirement silent. * fix(gitops): complete the available-action legality matrix The action list transcribed only part of the approved rules. An interrupted apply offered apply without checking that its recorded generation was still the current candidate; deploy fired for any target reading applied_not_deployed regardless of target mode, with no retry for an interrupted Direct deploy and none of the Blueprint interruption rules; approve_legacy sat in the type union with no producer, leaving a pending Inline placement review permanently unactionable through the projection; and drift items inherited whatever deploy recommendation any sibling target earned. Actions now follow the matrix. Deploy is decided per target through one predicate shared by deriveActions and the drift items: a paused or failed sibling can no longer inherit another target's legal deploy, a Blueprint-mode divergence never advertises Direct deployment, an interrupted Direct deploy retries against the generation still applied, and an interrupted Blueprint deploy or withdraw repeats only while its recorded intent revision and rollout candidate still equal what the application requires, where an absent pair matches because inline Blueprints carry no candidate until the later-phase producer lands. Interrupted apply requires the recorded generation to still be the current candidate and the source not to have been suspended meanwhile. A reachable Inline placement review now offers approve_legacy. Six tests pin the matrix: sibling isolation across failed and paused targets, the Blueprint mode guard, vacuous and matched Blueprint retry identities plus the superseded case, matching-versus-stale Direct deploy and apply interruptions, and the legacy review action. * fix(gitops): withhold apply retry while the candidate is blocked The interrupted-apply retry checked that the recorded generation still matched the current candidate and that the source was not suspended, but a later classification can also block that candidate, and applyStarted refuses a blocked one outright. The retry is offered only when every precondition the transition enforces still holds. * fix(gitops): prove apply preconditions and limit fetch to Direct The interrupted-apply retry checked identity, suspension, and blockage but never loaded the candidate generation, so apply could be recommended for a row that was missing, owned by another application, or built from a superseded materialization fingerprint; applyStarted refuses all three, and the positive fixture itself described a state no transition would accept. The gate now proves existence, ownership, and fingerprint before offering apply. Its fixture is rebuilt around a transition-legal candidate whose projected action is executed against applyStarted, with negatives for every refusal including suspension. Fetch was offered to any application whose source looked unreconciled, but the approved rules reserve fetch for live Direct applications; a Git-backed Blueprint divergence now advertises nothing of the sort. Controls pin both modes against equivalent source state. * fix(gitops): fail closed when a candidate row is missing or foreign Ordinary candidate_ready checked a fingerprint only when the candidate generation row existed, so a dangling id or one owned by another application fell through to ready and offered an apply that applyStarted would refuse on sight. The source now requires the row to exist under this application before readiness, records a limitation for the anomaly with its operator copy mirrored in the frontend, and reports reconcile-required instead. The Blueprint retry test also pins that a superseded rollout candidate alone, with the intent still matching, suppresses deployment. * fix(gitops): fail accepted-generation derivation closed The acceptance branch compared fingerprint and sha only when the accepted generation row loaded, so a dangling id or one owned by another application fell through and reported application_generation_accepted with no evidence behind it, while also withholding fetch as the recovery. The source now requires the row to exist under this application before any comparison, records an accepted_generation_invalid limitation naming the pointer with its operator copy and inventory entry mirrored on the frontend, and reports reconcile-required instead. Five scenarios pin valid, missing, foreign, fingerprint-mismatched, and sha-mismatched acceptances. * fix(gitops): enforce canonical state in dismiss, reconcile, and pulls - Route dismiss-pending through the canonical dismissed transition so a refusal while an operation is in flight surfaces as 409 instead of clearing staged state behind the model's back - Treat tombstoned targets as authoritative in both reconciler decision surfaces so ticks never redeploy onto a placement the model severed; an explicit deploy revives the target and records the revival delta - Keep application-level rows in node-scoped history pages so proxied hub views do not read as if the application never came into being - Expose repoIdentity and configuredRef on history items to match the server-side identity filters - Stand down when a pull resolves to exactly the live candidate (same commit, source fingerprint, plan verdict); pulls after an acceptance still open a fresh staging generation as the apply target * fix(rbac): classify git-source manifest reads as stack:read GET /stacks/:name/git-source/manifest had no rule in the hub route classifier, so proxied requests were refused with 403 before reaching the remote node even for callers holding stack read access. * fix(gitops): surface an error toast when dismiss is refused as in-flight Dismissing a pending Git update while a fetch or apply is still running now returns 409 OPERATION_IN_FLIGHT, but the frontend handler had no else branch for a non-2xx response, so the refusal was swallowed with no operator feedback at all. Add the same error-toast pattern already used by the sibling apply handler, and order the success toast before its side effects so a downstream failure cannot invert the outcome.	2026-08-27 10:25:27 -04:00
Anso	f5178889eb	feat(recovery): complete authored-project atomic rollback generations (#1819) ... * feat(recovery): capture complete authored Compose project for atomic rollback Replace the root-compose-only backup slot with staged recovery generations that record the managed inventory, exact Compose invocation, and prior image identity, and wire the same engine through deploy, update, manual rollback, and Git apply. * fix(recovery): satisfy CodeQL path barriers and update-guard mock Inline resolve+startsWith checks at generation/inventory fs sinks and stub getCurrentStackUpdateRecovery in UpdateGuardService tests. * fix(recovery): drop unused FileSystemService import in generation store test * fix(recovery): harden authored-project rollback for upgrade and restore safety Preserve legacy UUID backup rows, restore Git deploy state with files, make multi-file restore recoverable, evaluate policy on the restored target, and fail closed when Git capture cannot cover an apply. * fix(recovery): unblock Git apply unit tests and CodeQL pre-restore TOCTOU Mock recovery capture in git-source-service tests after fail-closed apply capture, and re-resolve live paths immediately before pre-restore snapshot reads. * fix(recovery): fall back to authored inventory when Git manifesto is missing First Git apply captures before promote, so a missing managed-project manifesto must not block rollback capture when the live stack already has authored files. * fix(recovery): make authored-project rollback atomic across Git state Restore the managed-project manifesto with files, keep nullable Git identity on first-apply captures, persist Git side-state in restore intents for startup reconcile, compensate legacy materialize failures, and refuse directory collisions before mutation. * fix(recovery): satisfy CodeQL path and TOCTOU barriers on manifesto restore Add inline resolve barriers for manifesto read/clear sinks and remove the access-then-read race when restoring a generation manifesto snapshot. * fix(recovery): close third-audit rollback generation blockers Fail closed on incomplete Git inventory fallbacks, execute captured Compose invocation during recovery, refuse startup and mutations while restore intents remain unresolved, propagate legacy stale-delete failures, and add Docker-level exact prior-image coverage plus regression tests. * fix(recovery): mark acquired before handoff in prior-image Docker test Match the production updateStack CAS sequence so the exact prior-image integration test does not fail handoff from the captured phase. * fix(recovery): close fourth-audit rollback safety blockers Evaluate policy against held images, use index-based pre-restore snapshots, hold the shared stack lock across Git apply, replay Mesh and empty captured invocations exactly, restore POSIX modes with fail-closed sensitive permissions, keep case-sensitive paths, and link Git auto-deploy health gates. Add regression coverage for these cases. * test(recovery): fix mocks for health-gate link and authored compose args Add linkGateOrRetain to the Git apply recovery mock, and mock authoredComposeArgs so the case-collision inventory test is not masked by a missing getComposeDir stub. * fix(recovery): close fifth-audit rollback safety blockers Share git_apply locking for webhook auto-apply, fail closed on malformed recovery service records, refuse mixed-image capture, and require exact probe counts with hold-tag eligibility checks. * fix(recovery): close sixth-audit rollback safety blockers Preserve the legacy backup slot during generation capture, encrypt sensitive pre-restore snapshots, revert files on a failed health probe without committing Git, fail closed when an absent-file revert would delete a directory, skip Compose one-offs, route manual and scheduled backup through the current generation, and persist runtime image platform identity. * fix(recovery): close seventh-audit rollback safety blockers Fleet snapshot restore and restore-all now capture a recovery generation under the stack lock before any authored file write, including on remote nodes. * fix(recovery): keep pre-deploy generations during health-gate observe Link deploy recovery generations to the observing gate so backup cannot replace them mid-observe. Distinguish missing hold tags from probe failures, refuse generation release when services metadata is corrupt, classify mixed-replica and coverage refusals, and toast the backend rollback message. * fix(recovery): wrap webhook deploy case for eslint const bindings in an unbraced switch case trip no-case-declarations. Match the pull case block.	2026-08-13 03:48:09 -04:00
Anso	5c52ae26eb	fix(rbac): make complete built-in RBAC available on Community (#1793) ... Open all five built-in global roles and stack/node scoped assignments on Community. Remove paid fences from user role create/update, scoped assignment CRUD, permission evaluation, and the Users settings UI. Admiral continues to own extended audit governance, LDAP directory integration, and other organizational assurance features. Built-in scoped RBAC is no longer marketed or enforced as paid-only.	2026-08-07 23:50:53 -04:00
Anso	209c9c5d53	feat: add target-aware RBAC authorization to Scheduled Operations (#1745) ... * feat: add target-aware RBAC authorization to Scheduled Operations Replace the blanket requireAdmin gate on all 9 scheduled-tasks endpoints with per-action permission checks derived from the centralized action registry. Each scheduled action now declares the existing permission it requires: stack lifecycle actions need stack:deploy on the target stack, node-wide operations need node:manage on the target node, prune stays admin-only via system:settings, and snapshot requires unscoped node:manage. Key changes: - Backend registry: add permission field and resolveTaskPermissionScope - Routes: replace requireAdmin with requireTaskPermission, filter GET listing by permission, add two-phase PUT check - Scheduler: revalidate creator permission at execution time, auto-disable on revocation (TaskAuthorizationError) - Database: new creator_user_id column with migration and backfill - Frontend: canScheduleAction/canScheduleAny helpers, reachability gate via scheduledOpsAccessible, stack context menu uses canDeploy not isAdmin, permissions field on all ScheduledActionDefinitions - Expose checkPermissionForSubject for in-process callers (scheduler) No new PermissionAction values are added. Uses the existing stack:deploy, node:manage, and system:settings matrix. Scoped Admiral grants work on the exact (nodeId, stackName) target per SEN-438. * test: add RBAC coverage for scheduled operations authorization * test: update frontend tests for scheduled-ops RBAC gate changes - useStackMenuItems: gate Schedule task on canDeploy not isAdmin; add test for canDeploy=true, non-admin case - buildNavigationModel: default scheduledOpsAccessible to true (default ctx is admin, who can always schedule) - useViewNavigationState: add stack:deploy to admin can() mocks so canScheduleAny resolves correctly * fix: keep checkPermission unchanged, add checkPermissionForSubject standalone The earlier refactoring that made checkPermission delegate to checkPermissionForSubject changed the call order of effectiveTier(req) relative to the admin bypass, which subtly broke the Community tier clamping in the audit-log route. Keep checkPermission byte-identical to the original and expose checkPermissionForSubject as a standalone function used only by the scheduler revalidation path. * fix: prefix unused selectorType parameter in resolveTaskPermissionScope * fix: address audit findings — existence oracle, revalidation test, action filtering Three corrections from the independent PR audit: RC-1 (action filtering): Wire canScheduleActionAnywhere into the action picker in ScheduledOperationsView so actions the user can never schedule (prune without system:settings, node:manage actions without a scoped grant) are filtered from the picker entirely. Add canScheduleAction check on the Create button against the currently selected target, so the submit button is disabled when the caller cannot schedule the chosen action on the chosen target. RC-2 (existence oracle): Six by-ID endpoints (GET /:id, DELETE /:id, PATCH /:id/toggle, POST /:id/run, GET /:id/runs/export, GET /:id/runs) now return a uniform 404 when permission is denied on an existing task, so an unauthorized caller cannot distinguish "task does not exist" from "task exists but you are not authorized." Added requireTaskExistsPermission helper for the 404 variant; POST create and PUT merged-scope checks keep requireTaskPermission (403). RC-3 (revalidation test): Fixed the orphan-creator test to assert the post-execution task state (auto-disabled, explicit error message) rather than wrapping executeTask in a try/catch with a no-op else branch, since executeTask catches TaskAuthorizationError internally and returns normally. Added canScheduleActionAnywhere helper and AuthContext mock to ScheduledOperationsView tests (38/38 pass). * fix: remove unused TaskAuthorizationError import from test * fix: use 404 on PUT phase-1 unauthorized-access check The PUT two-phase check's first phase (ownership verification) now uses requireTaskExistsPermission (404) instead of a manual 403, consistent with the six other by-ID endpoints. This prevents a caller from probing task-ID existence via the PUT route. * fix: address QA findings — reorder prechecks, surface permission reason Two live-confirmed fixes from the 3-node fleet QA pass: Finding #4 (offline-node error ordering): Swap the order of the node- reachability precheck and the creator-permission revalidation in SchedulerService.executeTask. Authorization now runs first, so a revoked grant is always surfaced as an explicit auto-disable with a clear error message, even when the target node is offline. Previously the reachability check ran first, hiding the revocation behind a misleading "target node is offline" error and leaving the task enabled indefinitely while the node was down. Finding #7 (unexplained Save dead-end): When the Create/Update button is disabled because canScheduleAction denies the selected action-target combination, a muted text line now appears below the button: "You do not have permission to schedule this action on the selected target." This gives scoped users meaningful feedback instead of a silently disabled button with no explanation. * chore: remove redundant !!formName guards The earlier short-circuit conditions in isSaveDisabled and saveDisabledReason already guarantee formName is truthy by the time the canSaveWithCurrentTarget check runs. The !!formName guard is a no-op — flagged by GitHub code-quality as 'Useless conditional: This negation always evaluates to true.'	2026-08-02 01:15:37 -04:00
Anso	72cdbb0eaa	feat(schedules): auto-update stacks by Stack Label (#1717) ... * feat(schedules): auto-update stacks by Stack Label Add a reusable selector_type/selector_value on scheduled tasks so admins can schedule image updates against live Stack Label membership across the fleet or one node, reusing fleet label resolution and the existing auto-update orchestrator. * fix(image-updates): sanitize auto-update execute failure logs Use a static format string and sanitizeForLog so CodeQL no longer flags user-controlled stack names and error text in the execute catch. * fix(ui): space Scope label from fleet/node segmented control Match the Schedule row layout so the inline SegmentedControl no longer sits flush against the Scope label. * fix(ui): remove redundant wrapper around Scope segmented control	2026-07-28 13:00:47 -04:00
Anso	fa503ddf27	feat: add node-scoped opt-out for image update detection (#1715) ... * feat: add node-scoped opt-out for image update detection Operators who use an external update authority can disable Sencho registry polling per node without losing explicit stack Update, pull, or redeploy. * test: fix mocks and lint for image-update checks opt-out Scheduler tests need isChecksEnabled on the ImageUpdateService mock, and the UpdatesSection older-node fixture must not leave an unused binding. * fix: gate update-preview and recheck when detection is off Anatomy was still calling stack update-preview (and contacting registries) while checks were disabled. Short-circuit those routes and skip recheckStack writes so disabled nodes stay quiet until detection is re-enabled.	2026-07-28 10:10:04 -04:00
Anso	719180f156	fix(fleet): verify update status before removing readiness cards (#1697) ... * fix(fleet): verify update status before removing readiness cards Full-stack Apply now rechecks persisted status after the health gate starts, reloads the live preview before dropping a card, and invalidates the hub fleet aggregation so cleared updates cannot resurrect from a stale cache. Closes #1686 * fix(fleet): align persisted update status with preview semver detection Share digest-plus-tag detection so post-Apply sidebar status matches Fleet and Anatomy. * fix(fleet): keep tag-only updates advisory for Compose automation Expose digestUpdate vs tagUpdate from checkImage so scheduled and API auto-update only apply same-tag digest drift Compose can pull. * docs: clarify scheduled auto-update applies digest drift only Document that higher pinned tags stay advisory until Compose is changed, matching schedule and Run Now behavior. * docs: require Compose pin edits for higher-tag advisories Stop recommending Apply now or Update as remedies that cannot rewrite a pinned image tag. * docs: clarify Apply now pulls pinned tags only Align the detection-cadence bullet with digest-rebuild vs higher-tag guidance. * fix(fleet): keep tag advisories after apply and scheduled updates Tag-only previews were treated as cleared on Fleet reload, and scheduled/ Run Now paths wiped status without rechecking. Align post-update verification with the manual Apply path (health gate first, recheck, no blind clear) and block digest apply when sibling image checks failed. * fix(fleet): clear eslint unused-arg and containers assignment	2026-07-26 03:09:21 -04:00
Anso	0daddfde00	fix: reconcile sticky update indicators with Anatomy preview (#1698) ... * fix: reconcile sticky update indicators with Anatomy preview Sidebar, Updates filter, and Fleet treated retained partial/failed scanner has_update as confirmed. Keep raw state for retention/notifications, project confirmed-only to APIs, show distinct incomplete indicators, and clear sticky rows only after an authoritative-negative preview. Closes #1685 * test: align sidebar truncate E2E with failed-over-retained precedence Purple update indicators are confirmed-only; hasUpdate with a failed check correctly shows the failed trailing icon. * fix: clear confirmed update rows on authoritative-negative preview Address audit SF-1/SF-2/SF-3: observation-watermark clears for older ok+has_update rows (DB + memory gens), Fleet checkability parity with backend not_checkable, and Updates chip confirmed-only regressions. * fix: tombstone equal-generation writers on preview clear Advance the per-stack write generation when clearing at the observation watermark so a scanner reserved before preview cannot recreate the row after an authoritative-negative reconcile. * fix: clear sticky updates with digest and tag preview parity Share detection across scanner and preview, keep GET read-only with POST reconcile, gate Apply to digest and rebuild updates, and invalidate the hub fleet cache on clear. * test: set digestUpdate on auto-update checkImage mocks Scheduler and execute routes now gate Compose on digest drift; fixtures that expect an apply need digestUpdate so they exercise the update path. * fix: clear unused lint errors on sticky update branch Drop unused partial helper and fleet invalidate import; keep the CacheService inflight self-ref as let with an eslint exception so tsc stays green. * fix: use inflight holder for CacheService prefer-const Keep generation-aware ownership without a let self-reference that fights ESLint and tsc.	2026-07-25 15:42:19 -04:00
Anso	3f1f15a6f4	fix: keep running containers until stack pull/build succeeds (#1657) ... * fix: keep running containers until stack pull/build succeeds Acquire images before reconcile, capture a recovery generation for compensation, and only remove classified orphans after handoff. * fix: address recovery audit blockers for safe stack updates Retire abandoned and expired recovery artifacts, probe compensated runtimes before reporting rollback success, preserve local Docker when deleting a node, validate the exact Compose invocation before capture, and repair updateStack return-contract fixtures. * fix: resolve ESLint errors blocking CI on this branch Unused-import and unused-variable errors left over from the stack deletion refactor: MeshService in stacks.ts (its opt-out cascade moved into DeployedStackDeletionService), a redundant pruneVolumes destructure in deleteDeployedStack (the real one is re-derived from the same input object inside runDeletionBody), and an unused beforeAll import in a Docker-integration test stub. Also scopes the webhook pull-action case body in a block to satisfy no-case-declarations; purely syntactic, no behavior change. * fix: harden recovery probe, cleanup retry, and failed-pull Docker test Reject absent or unhealthy expected replicas before reporting rollback success, keep cleanup records until artifacts are actually removed, fail closed when a mesh override cannot be generated, and assert a real failed pull leaves the original container running. * fix: verify recovery probe image identity and stack-scoped override paths Reject recovered runtimes that use the wrong image or leave scale-zero services running, and confine tombstone override deletion to the intent stack directory so forged cross-stack paths cannot be swept. * test: batch notification cap fixtures in a SQLite transaction Unbatched 1200-row inserts were timing out at the default 30s under CI load even though the same assertions pass in under 2s when green.	2026-07-21 12:18:01 -04:00
Anso	63213c0960	feat: add service-scoped Compose update and restore (#1648) ... * feat: add service-scoped Compose update and restore Allow updating or rebuilding one declared Compose service on multi-service stacks without recreating siblings, with recovery snapshots, health-gate observation, and prune holds for rollback images. Full-stack update paths and single-service UX stay unchanged. * fix: sanitize service-scoped update log messages for CodeQL * fix: address service-scoped update audit findings B-01 through B-07 * fix: complete service-scoped update audit metadata and surfaces * test: wrap Updates readiness tests for deploy-feedback context * fix: keep service recovery reachable without Deploy Progress Make failed service-gate recovery discoverable when Deploy Progress is disabled or dismissed, suppress stale image-scan notification side effects, normalize ComposeService line endings, and add focused regression coverage. * fix: resurface ContainersHealth density and expand on multi-service stacks Service grouping hid the summary strip and Compact/Detailed/Expand controls that still applied to multi-container stacks.	2026-07-19 02:42:29 -04:00
Anso	35bb74425b	feat: guide missing external network creation during deploy (#1645) ... * feat: guide missing external network creation during deploy Detect missing external networks before Compose runs, prompt or auto-create safe bridge networks, and keep unsupported declarations blocked with trusted deploy provenance. * test: align deploy context and settings fixtures with missing-network gate Update caller spies, EffResource expectations, StacksSection save keys, and git-source spy cleanup so CI matches the new deployStack context and auto-create setting. * fix: drop unused renderError binding in missing-network resolver Satisfies no-unused-vars so backend ESLint CI passes; callers already key only on model presence. * fix: use HTTP-safe clipboard helper in missing-network dialog navigator.clipboard fails on plain HTTP LAN hosts; route copy actions through copyToClipboard so Docker and Compose copy buttons work on self-hosted instances. * fix: simplify missing-network dialog actions and copy label Drop the Compose snippet escape hatch, move secondary actions under More, and rename the terminal copy action to Copy create command so the footer is a clear Cancel / Create decision.	2026-07-17 14:51:00 -04:00
Anso	d8e4ede94f	fix(notifications): neutralize satellite-local node names in alert bodies (#1640) ... * fix(notifications): neutralize satellite-local node names in alert bodies Fleet-aggregated alerts embedded each instance seed name (often Local) while the hub badge already named the remote. Drop identity prefixes and use type-aware local wording so attribution stays on the badge. * fix(docs): correct image-update default check cadence Operator docs still said six-hour polling; the seeded default is two hours in interval mode, and the cadence is configurable or cron-based. * test(notifications): assert hub stamps roster name on neutral remote bodies Cover the fan-in path that attaches hub roster identity while leaving the satellite message body unchanged.	2026-07-16 15:51:52 -04:00
Anso	b91025dc8b	fix: unlock Community deploy policy hard-blocking (#1643) ... Remove the leftover paid-only blockingEnabled switch so enabled block-on-deploy policies enforce on every tier, matching the documented every-tier security surface. Existing Community policies begin blocking immediately with no migration.	2026-07-16 15:24:05 -04:00
Anso	8fd526ba05	feat(scheduler): include node identifier in remote proxy error messages (#1609) ... * feat(scheduler): include node identifier in remote proxy error messages Remote lifecycle proxy errors from postToRemoteStack, executeUpdateRemote, postToRemoteContainer, and getRemoteContainers now include a stable node label (name and numeric id) so a hub operator reading run history can correlate failures to a node without cross-referencing the task table. The prefix matches the existing containerNotFoundMessage pattern in the same class. Transport-level failures (DNS, ECONNREFUSED, timeout) also receive the node label. A startsWith guard prevents double- prefixing within the same method. The restart fan-out catch relies on the inner postToRemoteStack error for node context so the label appears exactly once. * fix(scheduler): pilot-aware remote proxy no-target error context Use formatNoTargetError for null proxy targets so pilot tunnel disconnects are not misdiagnosed as missing credentials. Standardize the failure prefix to Remote node name/id, share require/rethrow helpers, and cover pilot no-target plus remote container action failures in tests.	2026-07-10 13:05:03 -04:00
Anso	10fb93dcb1	feat(scheduler): schedule container restart, stop, and start (#1526) ... * feat(scheduler): schedule container restart, stop, and start Add container as a scheduled-task target type so operators can automate lifecycle actions against standalone containers by node and name, with matching UI pickers, validation, execution on local and remote nodes, and tests. * fix(scheduler): stack service matching and container picker hygiene Backfill Service on smartFallback containers so per-service stack restarts work when container_name is set. Match services by compose label and container name in stack routes and scheduled restarts. Exclude Sencho from GET /api/containers lists. Hide the Restart Stack service picker when a stack has only one service. * test(scheduler): scope service checkbox assertion to Services block The create dialog also has a Delete after run checkbox. Count checkboxes only inside the Services section so CI does not include unrelated form controls. * fix(scheduler): narrow closest() result to HTMLElement in schedule test The service-checkbox assertion passed an Element from closest() into within(), which requires an HTMLElement, failing tsc -b in the frontend build and Docker build stages. Use the closest<HTMLElement>() type argument so the value type-checks without an unsafe cast. * fix(scheduler): hide Sencho container on remote node picker lists Remote container lists are proxied from peer Sencho instances, so id-only self filtering missed peers on older builds. Await SelfIdentity init, match ImageID, and drop official saelix/sencho images. Apply the same heuristic in the scheduled-operations UI and when the hub fetches remote containers for scheduled runs. * test(monitor): add missing DatabaseService mocks for scan history cleanup * test(scheduler): add missing markStaleScansAsFailed mock SchedulerService.tick() calls db.markStaleScansAsFailed() to sweep stale vulnerability scans. The scheduler-service test was missing this method in its DatabaseService mock, causing TypeError failures during test initialization. Added mockMarkStaleScansAsFailed to hoisted mocks and DatabaseService mock object, returning safe default of 0 scans marked as failed. * test(compose): add missing FileSystemService mocks for getStackContent/getEnvContent * test(containers-route): mock SelfIdentityService to prevent initialize() crash The excludeSelfContainers() helper calls SelfIdentityService.initialize(), which tries to access DockerController. Without a proper SelfIdentityService mock, the initialize() call fails silently, causing a 500 error on GET /api/containers. Added SelfIdentityService mock with initialize(), isOwnContainer(), and isOwnImage() methods to prevent the crash.	2026-07-02 22:31:29 -04:00
Anso	1de49f8b1a	fix: name matched risk inputs in policy scan banner and alerts (#1473) ... The pre-deploy gate names the inputs that matched a scan policy (a known-exploited CVE, a fixable Critical/High, or a severity threshold), but the informational post-scan surfaces still framed every violation as a severity ceiling. The scan detail banner read "blocks severities at or above X, highest severity is Y" and the scheduled-scan alert read "<severity> exceeds <maxSeverity>", which is wrong for a KEV- or fixable-only policy that never gated on severity. Persist the matched reasons on the policy evaluation, carry them on the scheduled-scan violation, and render them on the banner so every policy surface names the input that actually matched. Evaluations persisted before this change carry no reasons: the parser defaults the field to an empty array and the banner falls back to a plain violation notice.	2026-06-26 16:43:26 -04:00
Anso	ca496c89dc	fix: name matched risk inputs in policy block messages (#1471) ... The auto-update, bulk-label, scheduler, and blueprint deploy block messages hardcoded "image(s) exceed <max_severity>", which is wrong under the risk-first policy model: a block can be driven by a known-exploited (KEV) or fixable Critical/High input while the severity threshold was never the trigger. In those cases the message named a severity ceiling the policy did not enforce. Route all four message paths through a shared summarizeBlockReasons helper (the same reason text the deploy-gate 409 response and the block dialog already use), so every surface names the inputs that actually matched. Falls back to a generic phrase when no reason was recorded.	2026-06-26 15:34:24 -04:00
Anso	db8bb70b7d	fix(scheduler): reject 6-field cron in Scheduled Operations (#1435) ... * fix(scheduler): reject 6-field cron in Scheduled Operations Create and edit validation parsed cron with cron-parser, which accepts both 5- and 6-field expressions, while the form, presets, and docs all describe a 5-field cron. Because the scheduler ticks once per minute, a leading seconds field can never improve precision, so a 6-field expression was silently accepted but never honored on its stated schedule. Add a field-count guard on both sides: the API rejects 6-field input at create and edit with a clear message, and the form surfaces the same error inline and disables save. Cron nicknames such as @daily still pass. Document the five-field requirement in the cron reference. * chore: merge main into scheduled cron validation * fix: avoid logging policy bypass actor in debug output	2026-06-24 23:00:56 -04:00
Anso	bc8c051962	feat(scheduler): consistent action targeting in Scheduled Operations (#1431) ... Give every scheduled action an explicit, predictable target model (Action then Node then Stack then Options then Schedule): - System Prune now exposes a Node picker and requires a node, so it can no longer run silently on the default node. - Vulnerability Scan and System Prune list local nodes only; both run on the hub-local Docker daemon and reject remote nodes on the backend. - Restart Stack service discovery loads services from the selected node via fetchForNode instead of the active or local node. - Fleet Snapshot shows a read-only "Scope: Entire fleet" summary. Backend gains a shared local-node guard and prune node validation on create and update, plus an executor-level remote-node guard, so the frontend and backend validation now agree for every action.	2026-06-24 21:02:15 -04:00
Anso	5f1baa7522	fix: harden deploy/update concurrency and node-targeting safety (#1390) ... * fix: harden deploy/update concurrency and node-targeting safety Release stabilization for deploy/update operational safety. Per-stack operation locking is now global. Background lifecycle paths (scheduler auto stop/down/start/backup/update, webhook execute, Git source auto-deploy, image auto-update, label bulk actions, fleet snapshot redeploy, and mesh redeploy) acquire the per-node, per-stack lock through a new StackOpLockService.runExclusive helper and skip rather than race a manual deploy/update/rollback/backup on the same stack and node. Skips surface honestly (a failed scheduled run, a recorded webhook failure, a per-stack batch result, or a thrown error) instead of a silent no-op. Update readiness and policy-bypass now run against the node captured when the dialog opened, not the live active node, so switching nodes while a dialog is open cannot retarget the update or the bypass retry. Rollback readiness no longer presents a moving-tag or unpinned image as a ready image revert. Restoring files does not revert a moving tag, so those stacks read as partial, and the rollback success message states that the compose and env files were restored. * fix: lock blueprint reconcile against manual ops and correct rollback wording Follow-up to the deploy/update safety hardening, closing two more gaps from a verification pass. BlueprintService.deployLocal and withdrawLocal called ComposeService directly, so blueprint reconciliation could race a manual deploy/update/rollback/backup on an owned stack. Both now run their compose lifecycle call through StackOpLockService.runExclusive and skip (recorded as a failed reconcile, retried on the next cycle) on conflict. The withdraw holds the lock across both the compose down and the directory delete so neither races a manual operation. The runtime rollback messages overstated recovery: a rollback restores the compose and env files and recreates containers, but does not revert an image behind a moving tag. The auto-rollback deploy-progress output, the recovery panel and chip, the failure toasts, and the manual rollback route message now state that the compose and env files were restored, with the matching OpenAPI example and atomic-deployments doc updated. * fix: acquire stack lock before blueprint deploy mutates compose and marker files Local blueprint deploy wrote the compose and marker files and ran the policy assert before acquiring the per-stack lock; the lock only wrapped the deploy itself. A reconcile could therefore rewrite an owned stack's files while a manual deploy/update/rollback/backup was running. The lock now wraps the whole critical section (create, write compose, write marker, policy assert, deploy), so on conflict nothing is written and the reconcile records a failed outcome. Adds a test asserting a deploy under a held lock records failed, writes no marker file, and leaves the manual lock untouched. * fix: make remote blueprint apply atomic under the receiving node's stack lock Remote blueprint deploy wrote the compose and marker files to the target node via separate HTTP calls and only locked on the final deploy, so the file writes could race a manual operation on that node. A node's operation lock is process-local and cannot be held by the hub across HTTP calls, so the locked create/write/deploy now runs on the receiving node. The locked critical section is extracted into BlueprintService.applyLocalUnderLock and exposed via POST /api/blueprints/apply-local. The hub posts the blueprint to that endpoint in one call; the receiving node runs create + write compose+marker + deploy under its own per-stack lock. Older nodes without the route answer 404 and fall back to the legacy multi-call flow. The endpoint is gated by paid tier and the same per-stack stack:edit and stack:deploy permissions as the PUT-compose + deploy it bundles, validates the stack name, compose size, and marker structure, and returns 409 on a lock conflict without writing anything. Adds tests for the atomic single-call path, the 404 legacy fallback, the 409 lock-conflict mapping, the route validation and permission paths, and the write-compose-then-marker-then-deploy ordering of the shared locked apply. * fix(deps): bump undici to 7.28.0 to clear high-severity advisory The frontend CI npm audit gate (--audit-level=high) failed on a transitive undici 7.25.0 (a dev-only dependency via jsdom): TLS certificate validation bypass (GHSA-vmh5-mc38-953g) and cross-user cache information disclosure (GHSA-pr7r-676h-xcf6). Bumping undici within jsdom's existing ^7.25.0 range to 7.28.0 clears the high-severity advisory and unblocks the frontend job. Lockfile only; no direct dependency or source change.	2026-06-18 13:38:19 -04:00
Anso	38aabe7064	feat: health-gated updates and rollback readiness (#1354) ... * feat: classify stack deploy and update failures with suggested next actions Failed deploy and update responses now carry a failure classification (cause category, headline, and suggested next step) derived from the compose error output. The recovery panel and chip render the classification and include it in copied diagnostics, and gateway-style failures surface as a node-unreachable cause. * feat: add update and rollback readiness reports for stacks Before a manual update, Sencho now shows an advisory readiness verdict computed from the stored preflight result, open drift findings, live container health, the pending image change, the rollback backup slot, and node disk headroom. The Stack Dossier gains a rollback readiness section that states what a rollback can restore and explicitly discloses that volume and bind-mounted data are not covered. Toolbar and sidebar updates now share one update path, and admins can create a fleet snapshot from the readiness dialog before updating. Nodes that do not advertise the capability keep the direct update flow. * feat: observe stack health after updates with a post-deploy health gate After a deploy or update succeeds, Sencho now watches the stack for a configurable observation window and records a passed, failed, or unknown verdict: containers must stay running, healthchecks must report healthy, and restart loops or disappearing containers fail the gate. The deploy panel shows the observation live and holds off auto-closing until the verdict lands, a failed gate surfaces the existing recovery actions including rollback, and the stack timeline records update started and gate verdict events. Scheduled, webhook, bulk, and git-source updates are gated the same way; rollbacks and installs are deliberately not. The gate is observational only and can be tuned or disabled per node under host alert settings. * docs: document health-gated updates and rollback readiness New operator page covering the update readiness dialog, the post-update health gate and its settings, the rollback readiness disclosure, and classified failures, with cross-links from the atomic deployments and deploy progress pages. The API reference gains the readiness and health-gate endpoints, the healthGateId success field, and the failure classification schema on deploy and update error responses. * feat: withhold the success verdict while the health gate observes An update used to show a green Succeeded that a failed health gate then contradicted moments later. The deploy modal now reports Verifying health while the gate observes, shows success only when the gate passes, and makes a failed or unknown gate the headline result; success toasts soften to a verifying message while a gate runs. The mobile recovery card groups its actions behind one bottom-right Take action menu so it stays compact on a phone, with the classified cause still visible on the card. A successful image update now also counts as the last known-good marker in rollback readiness, and the docs gain screenshots of the readiness dialog, gate states, dossier section, and settings. * fix: harden log format strings and the env existence path check Log calls that interpolated the stack name into the console format string now use constant format strings with placeholder arguments, and envExists validates path containment inline at its filesystem access, matching the established patterns used elsewhere in the same files. * test: adapt deploy modal success specs to the post-deploy health gate The deploy feedback modal now withholds its success verdict while the health gate observes the new containers, showing "Verifying health" until the gate passes. The two success-path E2E tests waited for "Succeeded" within the gate's 90s default window and timed out. Shorten the observation window to the 15s minimum for these tests via the settings API, assert the verify-then-succeed sequence the modal actually renders, and restore the default window afterward so the test value does not leak into later runs. * fix: serialize health gate polling and harden gate observation Address race conditions in the post-update health gate found in review. Backend: the gate poller used setInterval, so a Docker observe slower than the 5s tick could overlap the next poll and corrupt the restart and missing-container accounting, and a wedged socket could leave a poll pending forever. Polling is now single-flight: each cycle self-schedules the next only after it settles, and the observe is bounded by an 8s timeout so a hung probe counts as a poll error and resolves the gate unknown after three in a row. Frontend: the gate poller could overlap requests, letting a slow earlier "observing" response overwrite an already-applied terminal verdict. It is now single-flight with a terminal latch, so a late response can never roll the UI back from passed or failed. Also reject a non-digit nodeId on the snapshot coverage route instead of letting parseInt coerce it, document that turning off the deploy progress panel opts out of the live gate UI while the gate still runs server-side, and add gate-coverage tests for the webhook, git source, and auto-update apply paths plus the new single-flight, observe-timeout, and recovery cases.	2026-06-11 00:26:26 -04:00
Anso	710647a44f	feat(snapshots): preserve stack dossiers with fleet snapshots (#1339) ... * feat(snapshots): preserve stack dossiers with fleet snapshots Fleet snapshots can now optionally capture each stack's Dossier notes alongside its compose and .env files, so a recovery restores the operational knowledge around a stack, not just its configuration. - Opt-in global setting "snapshot_documentation" (default off), toggled from the renamed Fleet settings section. - Capture reads local dossiers from the database and remote dossiers over the Distributed API proxy; only stacks with notes are recorded, and secret values are never included. - Captured notes are stored encrypted at rest in a new fleet_snapshots column and surfaced in the snapshot detail view behind a badge. - Cloud and downloaded archives gain a documentation.json (archive_version 2). - Restore stays conservative: dossier notes are written back only when the operator explicitly opts in, on both single-stack and restore-all paths. - Existing snapshots and archives remain valid; behavior is unchanged when the setting is off. * fix(snapshots): harden dossier-notes restore against bad input and partial failures Address review findings on the documentation-snapshots restore path: - Parse `restoreNotes` strictly (=== true) on single-stack restore, matching restore-all, so a stray non-boolean can never opt in to overwriting notes. - Guard findSnapshotDossier: require an array of stacks and real dossier content, so a malformed or all-blank entry can't clobber current notes. - Make the dossier-notes write non-fatal relative to the file restore: a notes failure (e.g. a remote dossier PUT) is caught, reported via `notesError`, and no longer 500s the single restore or fails the stack in restore-all once the files are already written. - Surface the partial outcome in the UI: a warning toast on single restore, a summary note on restore-all, and gate the "Documentation captured" badge and restore-all notes control on captured stacks while rendering capture warnings. Adds tests for strict parsing, malformed/blank blobs, remote notes restore (success + non-fatal failure, single and bulk), and scheduled capture-on. * fix(snapshots): drop unused binding in restore-all remote notes test The restore-all remote notes test destructured a node id it never uses (restore-all is driven by snapshot id alone), tripping no-unused-vars and failing the lint step. Bind only the snapshot id.	2026-06-08 08:44:59 -04:00
Anso	865d792874	feat(pricing): collapse to two tiers (#1309) ... * feat(pricing): collapse to two tiers (Community + Admiral) Collapse Sencho's pricing from three tiers (Community / Skipper / Admiral) to two: a generous free Community tier and a single paid Admiral tier. The Skipper tier is removed. Now free in Community: auto-heal, auto-update, scheduled operations, webhooks, notification routing, Fleet Actions and bulk operations, SSO preset providers (Google / GitHub / Okta), unlimited users with admin and viewer roles, and deploy safety (atomic deploys, auto-rollback, and one-click rollback). Admiral (paid) is focused on running and governing a fleet: blueprints, Fleet Secrets, deploy enforcement, vulnerability report export, audit log, host console, private registries, mesh networking, node cordon, managed cloud backup, LDAP / Active Directory SSO, and the advanced RBAC roles (deployer, node-admin, auditor) with per-resource scoped assignments. Internally the license variant distinction is removed so tier is binary (community / paid). License validation still verifies the Lemon Squeezy store and product before granting paid status. Docs and the contributor guide are updated to the two-tier model. * docs(pricing): correct licensing page to two-tier pricing and tidy stale tier wording The licensing docs page kept the old Admiral pricing plus a Founder Lifetime column and an Enterprise paragraph after the two-tier collapse. Update it to $12/month or $99/year, drop the lifetime and Enterprise content, and link to the pricing page for current pricing. Also fix stale "Skipper" wording in CLA.md, SUPPORT.md, one test title, and three test comments. Historical CHANGELOG entries and the retired-Skipper license-guard test are intentionally left as-is. * docs: align licensing and SSO pages with the two-tier model Correct the SSO overview so the Google, GitHub, and Okta presets read as available on every tier, matching the provider table; only LDAP and Active Directory require Sencho Admiral. Remove the lifetime-plan references from the licensing, settings, and troubleshooting pages so they reflect subscription-only Admiral pricing. * fix(rbac): omit scoped permissions from /me on the Community tier Scoped role assignments only take effect on the paid tier, but GET /api/permissions/me returned them unconditionally, so a downgraded instance with leftover assignments rendered per-resource affordances the API then rejected with 403. The endpoint now mirrors the permission middleware and includes scoped permissions only on the paid tier. Adds a regression test covering the downgrade case. * docs: use custom-pricing wording on the contact page The two-tier model has no Enterprise tier; reword the contact page's enterprise pricing/deals to custom pricing/deals so it does not imply a tier that no longer exists.	2026-06-04 17:45:53 -04:00
Anso	c11a550b6a	fix(fleet-snapshots): gate reads on admin role and encrypt content at rest (#1273) ... * fix(fleet-snapshots): gate reads on admin role and encrypt content at rest Fleet snapshots capture every node's compose.yaml and .env, so the data is as sensitive as the live stacks. This hardens access and reliability across the snapshot pipeline. - Restrict snapshot reads to administrators. GET /api/fleet/snapshots and /:id now require the admin role, matching create, restore, and delete; the Fleet "Snapshots" tab and its panel render only for admins. Previously any authenticated user could enumerate snapshots and read every node's .env. - Encrypt snapshot file contents at rest with the instance key. Restore and cloud-archive paths decrypt on read, so cloud archives stay portable and a database copy no longer exposes stack secrets in plaintext. Rows written before this change still read back as plaintext. - Surface partial captures. A stack whose compose file cannot be read or fetched, or a file over the 1 MB capture cap, is recorded as a warning and shown on the snapshot instead of being silently dropped, so a snapshot is never mistaken for complete. Remote .env read errors are now distinguished from a genuinely absent .env. Adds route-authz, capture-warning, and encryption round-trip tests; updates the Fleet-Wide Backups feature docs. * fix(fleet-snapshots): gate cloud snapshot reads on admin role The cloud snapshot read routes were guarded by provider/license only, not by role, while their write counterparts (upload, delete) already required admin and the Cloud Backup settings surface is admin-only. Because a downloaded archive contains plaintext compose and .env files, a non-admin could list and download cloud snapshots and read every node's secrets, the same exposure the local snapshot reads were just closed against. - Require admin on GET /api/cloud-backup/snapshots, /status/:id, and /object/:keyB64/download, matching the local snapshot reads and the admin-only Cloud Backup settings section. - When capturing a remote node, treat a 200 response carrying X-Env-Exists: false as a stack with no .env (matching the local ENOENT path) instead of storing an empty .env that restore would later write back. Adds non-admin authorization tests for the cloud read routes and a remote absent-.env capture test.	2026-06-01 17:27:59 -04:00
Anso	6fc7f200a6	fix(scheduled-ops): run stack lifecycle schedules on remote nodes and harden run visibility (#1260) ... * fix(scheduled-ops): run stack lifecycle schedules on remote nodes and harden run visibility Stack lifecycle schedules (Restart, Stop, Take Down, Start, Backup Stack Files) now run against whichever node the schedule targets, local or remote. Each remote run proxies to that node's own stack-operation endpoint, so a hub-managed schedule reaches the node that actually holds the stack. Restart with a service subset restarts each selected service and, if one fails, names the services already restarted so run history reflects the stack's partial state. Auto-start on a remote node runs that node's own pre-deploy scan-policy check against the images it holds. Add POST /api/stacks/:name/backup to trigger an on-demand backup of a stack's compose and env files (the same rollback snapshot a deploy takes); it backs the remote backup schedule and is available to operators on its own. A scheduled task that reaches execution on an unpaid licence is now skipped and written to run history as a failed run, so a manual trigger that returned a queued response never silently disappears. Test plan: - Backend unit + integration: scheduler-service (remote proxy per action, per-service fan-out, auto-start policy delegation, remote-failure and no-credentials paths, unpaid-tier skip), stack-backup-route (auth/role/paid/404/400/500), scheduled-tasks-routes. - Frontend component test for the schedules view (list, prefill, node filter, create payload). - tsc and lint clean on both packages. * fix(scheduled-ops): lock the stack-files backup route against concurrent stack ops The stack-files backup writes the same slot the pre-deploy rollback snapshot uses, so running it while a deploy, update, or rollback is in flight on the same stack could overwrite the rollback point. The backup route now takes the per-stack operation lock (as deploy/down/restart do) and returns 409 when the stack is busy, keeping the rollback snapshot intact. Adds the 'backup' action to the stack-op lock type and a busy participle for the 409 message. * fix(scheduled-ops): enforce backup-path containment inline at the filesystem sink The on-demand backup route passes the stack name straight into backupStackFiles, so resolve the backup directory against the backup root and confirm containment with an inline startsWith check before the mkdir/copy/write sinks, matching the barrier restoreStackFiles already uses. The stack name is validated at the route and again by resolveStackDir, so this is defense in depth that also closes a static path-injection finding on the new call path.	2026-05-31 17:47:34 -04:00
Anso	b33a0e8422	fix(deploy-enforcement): surface scan-policy blocks on update and sidebar deploys (#1248) ... * fix(deploy-enforcement): surface scan-policy blocks on update and sidebar deploys A blocked deploy only opened the policy dialog from the editor deploy button. The update action and the sidebar context-menu deploy/update fell through to a generic error toast, so an admin could not review the violations or bypass the block from those entry points. Route the 409 policy response through a shared handler on all three paths and make the "Deploy anyway" bypass retry the originating action (deploy or update) so an update bypass still re-pulls images. Also: - Correct the "Block on deploy" policy-editor helper text, which described post-deploy alerting rather than the pre-flight rejection it actually performs. - Dispatch the documented scan_finding warning (policy name and the offending images) when a scheduled auto-update or auto-start is blocked, instead of recording an opaque failure. - Add a standard log line when the gate blocks a deploy, plus developer-mode diagnostics for the matched policy and per-image severity decision. - Fix deploy-enforcement docs: complete the enforced entry-point list, correct the policy-precedence wording, and remove inaccurate tier and audit-actor claims. * fix(deploy-enforcement): surface policy block on rollback and name images in remote auto-update alert Addresses two gaps found in independent review: - Rollback is a policy-gated deploy path (it restores the saved files then re-runs the gate before redeploying), but the frontend treated a blocked rollback as a generic error toast. Route the 409 through the same handler as deploy and update so the block dialog opens, and let an admin "Deploy anyway" retry the rollback with the bypass flag (the rollback route already honors it). - The remote auto-update path dispatched its policy-block warning without the offending image refs, unlike the local scheduler. Append the images so the alert matches the documented contract on every node. Also list rollback as an enforced entry point in the docs and clarify that Git Source enforcement covers both the create-time deploy and a manual apply-with-deploy.	2026-05-29 08:48:50 -04:00
Anso	45844b92ca	fix(atomic-deploy): harden rollback locking, restore fidelity, and tier gating (#1247) ... * fix(atomic-deploy): harden rollback locking, restore fidelity, and tier gating Hardens the Atomic Deployments feature found during a full audit: - Rollback now holds the per-stack lifecycle lock (deploy/update already do), so a rollback can no longer race a concurrent deploy on the same compose files. Adds a 'rollback' lifecycle action and releases the lock in finally. - restoreStackFiles is now a faithful revert: it removes managed compose/.env files added after the backup before copying, so a rollback no longer leaves a hybrid of old and new configuration. Scope is the protected file set only; user data is untouched. Aborts (rather than reporting success) if a stale managed file cannot be removed. - The scheduled image-update path derives the atomic flag from the licence tier instead of hardcoding it on, keeping the paid capability explicit at the call site (the scheduler is already paid-gated; this prevents silent drift). - The backup-metadata read (GET /stacks/:name/backup) now requires a paid licence, matching the rollback flow that is the only caller. - Manual rollback dispatches a success/failure notification, alongside the existing audit-log entry. Adds route integration tests (lock acquisition/release, tier 403, notifications, no-backup 404), filesystem tests for the faithful restore (orphan removal, variant switch, abort path, non-managed files preserved), a community-tier scheduler test, and a developer-mode logging matrix. Documents the restore semantics and reconciles the scheduled-update wording in the feature guide. * fix(atomic-deploy): assert restore target stays within the compose dir before unlink The orphan-removal step in restoreStackFiles joins the stack directory with a managed filename and unlinks it. The stack directory is already validated and contained by resolveStackDir (allowlist stack name + within-base assertion), but the containment guard was not reapplied to the joined target at the delete sink, so static analysis flagged the path as derived from user input. Reassert containment on the final path before unlinking, matching the barrier the other write/read helpers in this service already apply. No behavior change for valid stacks; defense-in-depth at the sink. * fix(atomic-deploy): inline the path-containment barrier at the restore unlink sink The wrapped within-base assertion was not recognized as a sanitizer by the static path-injection analysis, which still traced the stack name to the unlink sink. Replace it with the inline path.resolve + startsWith containment check the other write helpers in this service already use (the recognized barrier), kept in the same scope as the sink. Behavior is unchanged for valid stack names. * fix(atomic-deploy): clear stale managed files from the backup slot before writing The backup directory is reused across runs and was only ever added to, never cleared. A managed file removed from the stack since the last backup (e.g. a deleted .env or a switched compose variant) lingered in the slot, so a later rollback restored a file that did not exist immediately before the failed run, contradicting the faithful-revert guarantee. Clear the protected file set from the slot before copying the current files, with the same inline containment barrier the restore path uses. A clear failure is logged, not fatal, since it only risks a stale future rollback and should not block a valid deploy.	2026-05-29 00:12:39 -04:00
Anso	adcd04b01a	refactor(auto-update): retire per-stack gate, drive auto-update from schedules only (#1233) ... * refactor(auto-update): retire per-stack gate, drive auto-update from schedules only The per-stack Auto-update toggle in the stack sidebar context menu wrote a gate row to `stack_auto_update_settings`, but actual updates only ran when a `scheduled_tasks` row with `action='update'` fired. On a fresh install the toggle was inert: detection ran every 6h, nothing was applied. The same context menu already exposes `Schedule task`, which opens ScheduledOperationsView pre-filled for the stack where the user can pick `Auto-update Stack` and any cron. Keeping the toggle alongside that flow duplicated the same action and turned the gate table into a parallel store of "is a covering schedule active" derivable from `scheduled_tasks` itself. Drop the gate model entirely: - Backend: remove the `stack_auto_update_settings` table and its four accessors, the three routes under /api/stacks/*/auto-update, the per-stack skip in /api/auto-update/execute and SchedulerService.executeUpdate's fleet branch, and the clearStackAutoUpdateSetting call on stack delete. Dashboard `autoUpdate` count derives from scheduled_tasks (action='update' rows pinned to the node, total/enabled split). - Frontend: drop the Auto-update entry from the sidebar context menu and its optimistic toggle plumbing. Drop autoUpdateSettings state, the /stacks/auto-update-settings fetch, and the auto-update-settings-changed WebSocket branch. Slim useSidebarActivitySummary (just nextRunAt; no enabled/total counts). AutoUpdateReadinessView's per-card autoUpdateEnabled now means "a covering enabled action='update' schedule exists" (per-stack row or fleet row on this node, earliest next_run_at wins, per-stack row wins on ties), with the gate-fetch removed. - New: scheduledTasksRouter broadcasts scope: 'scheduled-tasks' on POST, PUT, PATCH /toggle, and DELETE so useConfigurationStatus and useNextAutoUpdateRun refetch under the 250ms debounce instead of waiting for the 60s poll. The broadcast is wrapped so a broken subscriber socket cannot turn a successful mutation into a 500. - Docs: rewrite the "Per-stack control" section of auto-update-policies.mdx to describe the schedule-based model; update the matching troubleshooting entry. The misleading fleet-update help text in ScheduledOperationsView is corrected to reflect that every stack on the node is covered. Tier parity: the surviving auto-update path (Schedule task -> Auto-update Stack / All Stacks) is gated `requirePaid + requireAdmin` backend and `isPaid + isAdmin` frontend, matching the gate the deleted routes carried. The pre-commit grep returns no tier-related diff outside this PR's scope. No data migration is provided: greenfield rules apply, and the leftover table on already-shipped instances is harmless because no code reads or writes it after this PR. * docs: sweep remaining references to the per-stack auto-update toggle The previous commit retired the per-stack Auto-update gate in favor of configuring auto-update purely through scheduled tasks. This commit removes the now-stale mentions of that toggle across the operator docs: - docs/features/sidebar.mdx: drop the Auto-update entry from the Inspect group description, the matching screenshot alt-text, and the Skipper Note that listed it. Schedule task now carries the cross-link to Auto-Update Policies. - docs/features/stack-management.mdx: drop the Auto-update list item; refresh the Schedule task entry to mention the Auto-update Stack action. - docs/features/dashboard.mdx: rename the Configuration Status row from "Auto-update stacks" to "Auto-update schedules" with the new value shape, and rewrite the troubleshooting accordion to describe the scheduled-tasks invalidation path. - docs/features/scheduled-operations.mdx: rewrite the Auto-update All Stacks row and helper text to reflect that every stack on the node is covered (no per-stack opt-out from this surface anymore). - docs/features/multi-node.mdx: rewrite the Updates column definition to derive the Auto/Off flag from enabled Auto-update Stack / Auto-update All Stacks schedules instead of the removed per-stack policy. The auto-update-policies.mdx rewrite in the previous commit already covered the main reference page. The sidebar-context-menu.png screenshot will be refreshed on release once the new menu is live in production; the alt text is updated in this commit so it accurately describes the shipping state. No website edits needed: the Auto-Update Policies feature card description ("Schedule automatic image pulls and redeployments per stack on your own cadence") and the feature matrix labels ("Auto-update stack schedule", "Auto-update all stacks schedule") remain accurate under the new model. * fix(stacks): drop orphaned requireAdmin import after auto-update route removal CI's backend lint step flagged this PR's earlier deletion of the three /api/stacks/*/auto-update routes: those handlers were the only callers of `requireAdmin` inside routes/stacks.ts, leaving the named import on line 15 unreferenced. `requirePaid` and `effectiveTier` from the same line are still in use elsewhere in the file and stay. tsc --noEmit does not flag unused named imports; ESLint's no-unused-vars does. Local backend lint reproduces and now reports 0 errors against the existing 334-warning baseline.	2026-05-26 11:08:33 -04:00
Anso	2d56ea958a	fix(stack-activity): per-stack history integrity, attribution, sanitization (#1228) ... * fix(stack-activity): per-stack history integrity, attribution, sanitization Address the Stack Activity audit findings (PR 1 of 2): - Per-stack history integrity: drop the per-insert 100-row prune in addNotificationHistory that evicted quieter stacks' history whenever another stack got chatty. Periodic cleanupOldNotifications now caps per (node, stack) at 500 rows and per-node unattached system events at 1000 rows, on top of the existing 30-day retention. Signature takes an options bag and returns a per-stage summary so MonitorService can log what actually ran each cycle. - Actor attribution: thread req.user?.username through every notifyActionFailure call site and add synthetic actors at service emit sites (system:autoheal, system:scheduler, system:image-update, system:docker-events, system:blueprint, system:monitor, system:policy). The timeline renders system actors as "via <Label>" so an autoheal redeploy is no longer indistinguishable from a user redeploy. - Message sanitization: new sanitizeNotificationMessage at NotificationService.dispatchAlert strips KEY=VALUE pairs whose key ends in TOKEN/KEY/PASSWORD/SECRET/CREDENTIALS/AUTH, scrubs HTTP basic auth in URLs and Bearer tokens, collapses COMPOSE_DIR paths, and truncates to 1000 chars. Applied to the stored history and to every downstream Discord/Slack/webhook channel. The ImageUpdateService recovery-path direct DB write also runs through the sanitizer. - Composite pagination cursor: getStackActivity now accepts a (timestamp, id) cursor (?before=&beforeId=). The legacy timestamp-only form silently dropped events when a single compose up emitted many events sharing one millisecond. Route rejects beforeId without before. - Frontend hardening: distinct error state with retry button (initial fetch failure no longer renders as the genuine empty state), strict positive-integer parsing on cursor params, overrequest-by-1 pagination so the last page does not leave a dead "Load more" click, runtime guard on liveEvents merge that validates the level union, per-minute day-bucket recompute so an open panel does not stay on "Today" past midnight. No tier, role, or capability gate touched. Route permission gate remains stack:read on the named stack. * fix(stack-activity): sanitizer covers lowercase env vars and per-node compose dir External review surfaced two leak paths in the message sanitizer: - The sensitive-key regex was uppercase-only. Compose env names are conventionally uppercase but lowercase forms (db_password, jwt_secret, github_token) are valid and do leak through the same Docker and compose-parse error paths. Make the regex case-insensitive and tighten it to also catch bare TOKEN= / KEY= / PASSWORD= without a prefix word, while still leaving BYPASS, COMPASS, and similar non-secret keys alone. - The compose-dir path collapse only read process.env.COMPOSE_DIR, but the real resolution chain is node.compose_dir (per-node DB override) -> process.env.COMPOSE_DIR -> /app/compose. A node with a custom compose_dir could still leak absolute paths into stored history and downstream channels. Route both the dispatchAlert call and the ImageUpdateService recovery-path direct write through NodeRegistry.getInstance().getComposeDir(localNodeId) so the collapse covers every resolution outcome. Tests now assert lowercase keys are redacted and that BYPASS-style non-secrets stay intact in both cases. notification-routing mock extended to stub the new getComposeDir call. * chore(stack-activity): a11y roles, visibility-aware tick, live-disconnect signal Close three small follow-ups on the per-stack activity timeline: - A11y: each day-group gets role="list" and each event row gets role="listitem" so screen readers traverse the timeline as a list instead of a wall of text. The day-group container also carries an aria-label naming the bucket. - Visibility-aware day-bucket tick: the 60s setInterval that re-derives Today/Yesterday/Earlier now short-circuits when document.hidden, so a backgrounded panel does not re-render every minute for no visible effect. - Live-disconnect signal: useNotifications dispatches a sencho:notifications-connection custom event on WebSocket open and close. The timeline listens and, when explicitly disconnected, shows a one-line "Live updates offline; reconnecting…" hint above the list. The sidebar ticker already surfaces fleet-wide connection state; this adds an in-context cue for users who are focused on a single stack. Stack-name case normalization was considered and rejected: stack names are case-permissive per the isValidStackName validator, and lowercasing on read or write would silently rename or hide a user's "MyApp" stack. * ci(stack-activity): drop unnecessary escape in URL_BASIC_AUTH regex ESLint no-useless-escape errored on \- inside the character class [a-zA-Z0-9+.\-] at notificationMessage.ts:14. Move the dash to the end of the class so it's an unambiguous literal and the escape is no longer required. Behavior is identical; sanitizer tests still pass. * revert(stack-activity): drop unvalidated E2E spec from this PR The spec was committed without ever running against a real Docker daemon, then failed in CI when it ran for the first time: deploy returned 200 but no notification appeared on the activity endpoint within the polling window, suggesting either a deploy-notification race or a node-id resolution mismatch in the CI environment. Backend unit tests (route + composite cursor + sanitizer) and frontend component tests cover the same logic. The E2E spec will land in a dedicated follow-up once it has been authored against a working CI environment.	2026-05-25 21:09:00 -04:00
Anso	c491d309c1	feat(schedules): make every scheduled action available on Skipper (#1141) ... Collapse the Admiral carveout that restricted restart, prune, auto_backup, auto_stop, auto_down, and auto_start schedules to the Admiral variant. Scheduled Operations stays at Skipper+ (paid). The action picker now lists every supported operation for any paid admin, and the scheduler runner executes every action on either variant.	2026-05-21 16:42:41 -04:00
Anso	328a98439d	fix: harden vulnerability scan scheduling (#1035)	2026-05-13 09:40:45 -04:00
Anso	b1c5fe8391	fix: harden deploy enforcement paths (#1030) ... * fix: harden deploy enforcement paths * fix: update Docker toolchain to Go 1.26.3 * fix: repair Dockerfile tr argument split across lines * fix: bump protobufjs to clear npm audit high-severity advisories * fix(test): add execFile to child_process mock in compose-images test * fix: resolve merge conflicts with main * fix: resolve merge conflicts with main * fix: resolve merge conflicts with main	2026-05-12 19:30:49 -04:00
Anso	74ae2ce0c6	fix: harden atomic deployment rollback (#1029) ... * fix: harden atomic deployment rollback * fix: update Docker toolchain to Go 1.26.3 * fix: repair Dockerfile tr argument split across lines * fix: bump protobufjs to clear npm audit high-severity advisories * fix: sanitize error objects in console.error to prevent log injection	2026-05-12 15:58:30 -04:00
Anso	ecf4dd5d52	feat: open security basics, manual fleet ops, and basic fleet management to Community (#930) ... Realign tier guards to the user-stated philosophy: Community covers deploy/monitor at scale plus security basics, Skipper adds automation and advanced fleet management, Admiral keeps enterprise control. Community now includes: - Trivy install / uninstall / update from the Settings Hub (admin role) - CVE suppressions CRUD (admin role; replicates fleet-wide) - Manual image scan with vuln, secret, and misconfig results - Stack-config scan, scan comparison - Manual fleet snapshots: create, list, view, restore, delete - Per-node Sencho self-update (Check Updates + per-node Update) - Fleet Overview search, sort, filters, node-card expand, auto-refresh Stays paid: - Scan policies with block_on_deploy enforcement (Skipper+) - SBOM (SPDX, CycloneDX), SARIF export (Skipper+) - Bulk Update All across the fleet (Skipper+) - Scheduled snapshot create (now Skipper, was Admiral) - Trivy auto-update toggle, fleet-wide policy push (Admiral) The Settings -> Security tab is unhidden by setting the registry tier to null. The SecuritySection no longer early-returns a PaidGate; the policy list, Add Policy button, and policy dialogs are wrapped in {isPaid && }. The Fleet view drops isPaid gates on the Snapshots tab, Check Updates button, per-node update handlers, OverviewToolbar grid controls, the NodeCard expand affordance, and the auto-refresh notice. The NodeUpdatesSheet receives a canBulkUpdate prop and gates the Update All button on it. useFleetUpdateStatus and useFleetPolling drop their isPaid guards so polling runs for Community; useFleetOverview drops the isPaid wrap on the filter and sort path. Backend route guards are flipped per the matrix above. The scheduler tick and requireScheduledTaskTier add 'snapshot' to the Skipper+ branch. Backend test assertions are inverted for the now-Community endpoints and a positive Skipper-snapshot-task test is added. Documentation across features/, api-reference/, and operations/ is updated to reflect the new tier mapping.	2026-05-05 12:54:26 -04:00
Anso	e5b1c7b22b	refactor(backend): collapse entitlement provider abstraction back to LicenseService (#889) ... Removes backend/src/entitlements/ (registry, loadProvider, CommunityEntitlementProvider, types, headers, normalize) and the two abstraction-only tests. Relocates headers/normalize/types to services/license-*.ts. Swaps 22 consumer call sites from getEntitlementProvider() to LicenseService.getInstance(). Drops the Dockerfile install step plus PRO_PACKAGE_VERSION build-arg and github_token BuildKit secret in docker-publish.yml. Removes the now stale no-restricted-imports rule in backend/eslint.config.mjs. Net: 37 files changed, ~700 lines removed, no behavior change. Local dev no longer requires GitHub Packages auth to start the backend. Rationale and revisit conditions in docs/internal/adrs/2026-05-02-collapse-entitlement-provider.md.	2026-05-02 23:45:44 -04:00
Anso	3324616e59	refactor(backend): extract EntitlementProvider abstraction (Phase 1) (#878) ... * refactor(backend): extract EntitlementProvider abstraction (Phase 1) Phase 1 of the open-core hybrid extraction described in docs/internal/adrs/2026-05-02-open-core-hybrid-strategy.md. Introduces the abstraction without moving any code out of the public repo; Phase 2 will actually move services/LicenseService.ts to a private @studio-saelix/sencho-pro package. The new backend/src/entitlements/ module contains: - types.ts. The EntitlementProvider interface plus all tier/license types (LicenseTier, LicenseVariant, LicenseInfo, SeatLimits, ActivationResult, etc.). The interface mirrors the existing LicenseService public surface so the migration was mechanical. - registry.ts. Module-scope holder for the active provider with setEntitlementProvider, getEntitlementProvider, and a test-only reset helper. getEntitlementProvider throws if called before bootstrap registers a provider; the throw is intentional fail-fast on a bootstrap-order bug rather than a silent degradation. - CommunityEntitlementProvider.ts. Phase 2 fallback that returns community tier and rejects activate(). NOT instantiated in production today; a smoke test keeps it covered against bitrot. - loadProvider.ts. Async resolver. Phase 1 returns LicenseService.getInstance() directly. The async signature matches what Phase 2 needs (dynamic import of @studio-saelix/sencho-pro with a "module not found" vs "construction threw" narrowing); the call site does not change between phases. - headers.ts. PROXY_TIER_HEADER and PROXY_VARIANT_HEADER constants. These are part of the wire contract between Sencho instances and belong in the public core regardless of which entitlement provider is bound. - normalize.ts. isLicenseTier, isLicenseVariant, normalizeTier, normalizeVariant. Domain knowledge about Sencho's tier model (legacy name maps from pre-0.38.1 versions), not LemonSqueezy internals. Phase 2 keeps these in the public core. services/LicenseService.ts now imports its types from entitlements/types and adds an "implements EntitlementProvider" clause. Re-exports the types for back-compat with ~20 type-only consumers; a follow-up PR will sweep those imports to entitlements/ types directly before Phase 2 deletes the file. bootstrap/startup.ts awaits loadEntitlementProvider, registers the result, then calls initialize. shutdown.ts calls getEntitlementProvider().destroy() instead of the LicenseService singleton. middleware/tierGates.ts, the chokepoint for ~154 tier-check call sites, now reads through getEntitlementProvider. Sixteen other production files (routes/{fleet,imageUpdates,license,permissions, scheduledTasks,security,stacks,templates,users,webhooks}, services/{BlueprintService,CloudBackupService,SchedulerService, SSOService}, proxy/remoteNodeProxy, websocket/{hostConsole, remoteForwarder}, middleware/auth) had their LicenseService.getInstance calls and utility-export imports redirected to the entitlements module. The only remaining LicenseService.getInstance in production code is in entitlements/loadProvider.ts itself, which is the intentional Phase-1 binding site. Test infrastructure: setupTestDb registers LicenseService.getInstance() as the active provider so existing test files using the helper need no changes. The mocking pattern many tests use, vi.spyOn(LicenseService.getInstance(), 'getTier'), keeps working because LicenseService.getInstance() and getEntitlementProvider() return the same singleton in Phase 1. scheduler-service.test.ts is the only test that does not use setupTestDb but exercises tier-gating; it now mocks entitlements/registry alongside its existing LicenseService mock. Adds a smoke test for CommunityEntitlementProvider so the Phase 2 fallback class stays covered. Adds an architecture doc at docs/internal/architecture/entitlement-provider.md covering the runtime registry, bootstrap order invariants, and the Phase 1 vs Phase 2 binding table. Test results: 89/89 backend test files pass, 1657 passing tests, 5 pre-existing skips. The pre-existing database-metrics > handles 1000+ metrics stress test continues to flake under parallel load and pass when re-run solo, same flake observed in PRs #862, #863. * chore(backend): drop unused entitlement type imports from LicenseService Phase 1 of the EntitlementProvider extraction left five type imports (ActivationResult, BillingPortalError, BillingPortalResult, DeactivationResult, ValidationResult) unreferenced after the runtime methods that produced them began inferring their result shapes via the EntitlementProvider interface contract. ESLint's no-unused-vars rule flagged them as errors and failed the lint step in CI.	2026-05-02 05:07:00 -04:00
Anso	4e5ba17710	refactor(backend): sanitize user input before logging to close CRLF injection (#807) ... * refactor(backend): sanitize user input before logging to close CRLF injection Adds a small sanitizeForLog helper that strips CR, LF, tab, and ASCII control characters (0x00-0x1F, 0x7F) from a value before it is embedded in a console.log/warn/error/debug call. Wraps every call site where a user-controlled value (req.params, req.body, req.query, or a value derived from them) flows into a log message. Closes the bulk of the open CodeQL alerts in this family: - 96 js/log-injection - 28 js/tainted-format-string The helper is in backend/src/utils/safeLog.ts. Routes still pre-validate input at the request boundary; this is the second line of defense and gives static analyzers a sanitizer they can trace through. JSON responses, Docker filter labels, and other non-log call sites are intentionally left unwrapped. * refactor(backend): printf-style format strings for tainted-log call sites CodeQL's js/tainted-format-string rule flags template literals in the first arg of console.X when any interpolated value is user-controlled, regardless of whether each value is sanitized inline. The canonical mitigation is to use a static format string and pass values as positional args. Converts the 28 flagged template literals to printf-style ("%s") format strings, with sanitizeForLog applied to each positional arg. Also fills in the log-injection wraps on 9 sites where a user-controlled value was missed in the first sweep (agents, fleet, gitSources, imageUpdates, GitSourceService). No behavior change at runtime. Node's util.format substitutes %s tokens identically to template-literal interpolation. * fix(backend): wrap nodeId/snapshotId in fleet restore debug log CodeQL flagged the unwrapped numeric args even though they cannot contain control chars in practice. Apply the sanitizer for taint-flow recognition.	2026-04-27 10:47:23 -04:00
Anso	03f91cd5bb	feat(cloud-backup): mirror fleet snapshots to S3-compatible storage (#782) ... * feat(cloud-backup): mirror fleet snapshots to S3-compatible storage Add an Admiral-tier Cloud Backup feature that replicates every fleet snapshot to off-site storage, with two provider modes that share the same `@aws-sdk/client-s3` code path: - Sencho Cloud Backup: zero-config, 500 MB allowance backed by Cloudflare R2, provisioned via the sencho.io worker against the user's Lemon Squeezy license. - Custom S3 (BYOB): any S3-compatible bucket (AWS, MinIO, Backblaze B2, Wasabi, R2 with own keys), with credentials encrypted via `CryptoService` before storage. API-triggered snapshots upload fire-and-forget so the UI returns immediately; scheduled snapshots block on the upload so the task's success/failure reflects cloud durability. Object keys include the instance_id segment to prevent collisions when the same Admiral license is activated on multiple Sencho instances. * fix(cloud-backup): drop ES2022-only Error cause arg breaking ES2020 build The backend tsconfig pins lib to ES2020. The two-argument `Error(message, { cause })` form requires ES2022, so tsc rejected it with TS2554. Revert to single-argument throw to match the convention used elsewhere in the backend services.	2026-04-26 15:42:21 -04:00
Anso	abee078741	feat(scheduler): add auto_backup, auto_stop, auto_down, auto_start and delete_after_run one-shot mode (#777) ... * feat(scheduler): add auto_backup, auto_stop, auto_down, auto_start actions and delete_after_run one-shot mode Extends the scheduler with four new stack-targeted actions: - auto_backup: backs up stack compose files and .env using the existing FileSystemService.backupStackFiles primitive - auto_stop: runs compose stop (containers preserved) - auto_down: runs compose down (containers removed) - auto_start: runs compose up -d via deployStack (universal start for both stopped and down stacks) Adds delete_after_run boolean column to scheduled_tasks. When enabled, the task self-deletes after its first successful execution; failures keep the task so the user can debug and retry. All four new actions gate at Admiral tier, consistent with restart/snapshot/prune. Migration is idempotent (maybeAddCol). * docs(scheduler): update scheduled-operations doc with new lifecycle actions and delete-after-run Adds the four new actions (Backup Stack Files, Stop Stack, Take Stack Down, Start Stack) to the action table. Documents the delete-after-run one-shot mode with its success-only deletion semantics. Adds the Stack Lifecycle Scheduling section explaining stop-vs-down semantics and the local-execution boundary. Adds three troubleshooting entries: auto-start on a missing compose folder, auto-backup single-slot overwrite by design, and one-shot task disappearing after successful run. Updates the timeline description from four to five lanes. Refreshes screenshots to show the new dialog layout with the Lifecycle lane visible.	2026-04-25 16:11:13 -04:00
Anso	44dba59cab	feat(notifications): add structured category enum to dispatcher and history (#774) ... Introduce a NotificationCategory string-literal union (11 values) and thread it through dispatchAlert as a required second argument. All callers (DockerEventService, AutoHealService, ImageUpdateService, MonitorService, PolicyEnforcement, policyGate, SchedulerService, imageUpdates route) pass an explicit category at every call site, giving TypeScript compile-time enforcement that no new emit site can be added without choosing a category. DatabaseService gains an idempotent migration that adds a nullable category TEXT column to notification_history; existing rows keep category=NULL (displayed as Uncategorized in the UI). The getNotificationHistory method accepts an optional category filter that is forwarded from the GET /api/notifications/history route via a ?category= query param. NotificationPanel gains a category Select dropdown so users can filter history by category. The frontend types mirror the backend union so API responses are type-safe end-to-end. All 75 test files (1410 tests) updated to the new 4-arg dispatchAlert signature and passing.	2026-04-25 13:55:07 -04:00
Anso	a74564fd61	feat(scheduler): support fleet-wide auto-update schedules per node (#773) ... Allow a scheduled task with action='update' and target_type='fleet' to update every eligible stack on a node in a single schedule entry. The executor respects each stack's per-stack auto-update policy via a single batch query, skipping stacks that have opted out. For remote nodes the request proxies to the remote Sencho instance, which already enforces the same policy in its /api/auto-update/execute endpoint. Backend route validation now accepts update+fleet as a valid combo (previously only update+stack was allowed) and requires node_id. Frontend adds an "Auto-update All Stacks" option to the scheduled-task creation form with a node selector and descriptive helper text.	2026-04-25 12:17:09 -04:00
Anso	661b9c638b	feat(security): enforce scan policies as a pre-deploy gate (#719) ... Policies with block_on_deploy=1 now scan every stack image before docker compose up runs and reject the deploy with HTTP 409 on violation. The UI opens a dialog listing offending images; admins can override per deploy with ?ignorePolicy=true, and every bypass is recorded in the audit log with the originating route, actor, policy, and image list. When Trivy is not installed on the target node the gate fails open with a warning notification, so teams are never locked out by tooling state. Post-deploy and scheduled scans still evaluate matching policies and dispatch warnings on violations to surface drift on long-running stacks. Public API additions: policy and suppression CRUD under /api/security, plus the documented 409 block-response shape on all deploy paths.	2026-04-21 00:14:11 -04:00
Anso	95278843cf	feat(schedules): next-24h timeline + merge auto-update into schedules (#681) ... * feat(backend): add stack update-preview endpoint for readiness board Adds GET /api/stacks/:stackName/update-preview that returns per-image semver diff, bump classification, and a stack-level summary powering the Auto-Update readiness board. - New UpdatePreviewService parses compose images, inspects local digests, fetches remote digests and tag lists, and finds the highest compatible semver tag. - Major bumps are flagged blocked until human review; unknown bumps rank below real semver so they cannot mask a major. - Rollback target is reconstructed through parseImageRef to preserve registry ports and drop the Docker Hub library/ prefix. - Registry helpers (httpGet, auth token, digest, tag list, ref parse) are extracted into registry-api.ts and shared with ImageUpdateService. - 28 Vitest cases cover parse, selection, bump math, digest rebuilds, blocked policy, and rollback target construction. * feat(schedules): next-24h timeline, merge auto-update crud, add readiness board Replace the flat task table with a Timeline view as the default, showing the next 24 hours of scheduled work across four lanes (Restart, Update, Scan, Prune) with a live now rail and per-firing pills. The All tasks tab preserves the existing CRUD surface. Merge Auto-update Stack into Schedules as a first-class action and replace the standalone Auto-Update Policies view with a per-stack Readiness board that surfaces version diffs, risk tags, changelog previews, and rollback targets sourced from the stack update-preview endpoint.	2026-04-18 17:48:02 -04:00
Anso	29ed0524c1	feat(security): severity-aware scheduled scan notifications (#654) ... Enrich scheduled vulnerability scan completion notifications with per-severity CVE counts so recipients can triage from the message body alone. Expose the scan action in the schedule creation UI, require an explicit node_id, and harden fire-and-forget alert dispatches so a failing webhook cannot crash the scheduler. Notification body now reports scanned/skipped/failed counts plus critical/high/medium totals aggregated across fresh and cached scans, reflecting the current node posture rather than only what was newly scanned on this run.	2026-04-17 11:08:46 -04:00
Anso	e660d2a658	feat(scheduler): notify on scheduled scan completion (#646) ... Scheduled scan tasks now dispatch a completion alert through the existing notification system: info when every image scanned cleanly, warning when one or more images failed. The alert includes the task name and the run's scanned/cached/failed summary so operators do not need to open the task history.	2026-04-16 22:55:30 -04:00
Anso	61bac08027	feat(security): one-click managed Trivy install (#643) ... * feat(security): one-click managed Trivy install Add a Vulnerability Scanner card to Settings, Security with install, update, uninstall, and auto-update controls (Admiral-only). The installer downloads a verified Trivy release into the existing data volume at /app/data/bin/trivy and defaults the cache to /app/data/trivy-cache, so no host mounts or extra env vars are required. Detection probes the managed path, a TRIVY_BIN override, and the host PATH, distinguishing managed vs host installs. A daily scheduled check surfaces available Trivy updates, installs them automatically when opted in, and dedupes notifications per version. * fix(frontend): silence react-hooks/set-state-in-effect in useTrivyStatus The initial status fetch and managed-source update check both call setState from the effect body. Match the existing pattern used in useDashboardData / SSOSection and disable the rule at the call site.	2026-04-16 21:29:44 -04:00
Anso	dc8370f5a4	fix(security): harden Trivy scan lifecycle, logging, and docs (#639) ... * fix(security): harden Trivy scan lifecycle, logging, and docs - Call TrivyService.initialize() at startup so capability state is accurate before first request; add periodic re-detect to the scheduler so newly installed Trivy binaries light up without a restart. - Add markStaleScansAsFailed sweep (+ idx_vuln_scans_status index) to recover any scan row left in_progress after a crash or timeout; sweep runs before the paid-tier gate so every tier self-heals. - Split scanImage persistence into beginScan/finishScan so the manual scan route owns a single code path and can return a scanId synchronously while work continues asynchronously. - Validate image refs on /api/security/scan and /sbom via new utility; defense-in-depth against shell-metacharacter payloads. - Dispatch a warning-level alert when a post-deploy scan fails so the operator has a user-visible path to the failure instead of a silent log. - Share DIGEST_CACHE_TTL_MS and severity ordering across service and route layers; remove dead invalidateDetection(). - Add [Trivy:diag] logging gated behind developer_mode for support diagnostics; production logs unchanged. - Frontend: defensive toast fallback chain, sr-only SheetDescription, and a truncation badge when the 500-item detail fetch is capped. - Tests: extend trivy-service and vulnerability-db suites; add image-ref and severity unit tests. - Docs: expand vulnerability-scanning troubleshooting with recovery, re-detect, and diagnostic-log guidance; link Dockerfile comment to trivy-setup. * fix(security): drop unnecessary escape in image-ref forbidden-char regex	2026-04-16 20:32:38 -04:00
Anso	c9cd6990d2	feat(images): Trivy-powered vulnerability scanning (#635) ... * feat(images): Trivy-powered vulnerability scanning Scan container images for known CVEs via Trivy. On-demand scanning and severity badges are available on every tier; scheduled scans, scan policies, SBOM generation, and scan history are gated to Skipper+. - New TrivyService (binary detection, per-image scan, SBOM, digest cache) - Three new tables: vulnerability_scans, vulnerability_details, scan_policies - 12 routes under /api/security (scan, results, summaries, SBOM, policies, compare) - Post-deploy async scans wired into all five deploy paths, with a per-deploy opt-out toggle in the App Store deploy sheet - "scan" action type added to SchedulerService for fleet-wide recurring scans - Frontend: severity badges in Resources Hub with animated cursor detail, scan results drawer with vulnerability table and filters, and a new Security section in Settings for scan policy CRUD - Policy threshold violations dispatch a warning or critical alert based on the policy's block_on_deploy flag; deploys themselves are never blocked * fix(security): compute scan age in useEffect to satisfy react-hooks/purity	2026-04-16 15:03:36 -04:00
Anso	809bf76c20	fix(fleet): harden fleet snapshots with DRY capture, audit fixes, and design compliance (#555) ... Extract duplicated snapshot capture functions from index.ts and SchedulerService.ts into a shared module (snapshot-capture.ts). Fix audit log route patterns that used singular 'snapshot' instead of plural 'snapshots'. Apply design system to FleetSnapshots component (card styling, strokeWidth, font-mono, tabular-nums, ScrollArea). Add safePage pagination, loading toast for creation, and fix the restore dialog race condition with a controlled AlertDialog. Add diagnostic logging gated behind Developer Mode. Add tests for restore endpoint, admin role enforcement, and edge cases. Update docs with troubleshooting section and refresh screenshots.	2026-04-13 13:26:55 -04:00
Anso	44e8fdfba9	fix(scheduler): harden scheduled operations with stale cleanup, cron validation, and design fixes (#549) ... * fix(scheduler): clean up stale runs on startup and auto-disable invalid cron tasks - Add markStaleRunsAsFailed() bulk DB method with status index - Clean up orphaned 'running' records on scheduler startup - Auto-disable tasks when cron expression becomes invalid at execution time - Promote CRUD debug logs to standard logs for scheduled task admin actions - Add diagnostic logging for task pre-checks, action timing, and prune fallback * refactor(scheduling): extract shared types and fix design system violations - Extract ScheduledTask, TaskRun, NodeOption to shared types file - Extract getCronDescription and formatTimestamp to shared utilities - Fix formatTimestamp falsy-zero null check - Tighten last_status type to 'success' | 'failure' | null - Add strokeWidth={1.5} to all action icons per design system - Add sr-only DialogDescription for accessibility - Wrap Sheet run history in ScrollArea - Fix delete button styling to match design system pattern - Change manual trigger toast from "executed" to "triggered" * test(scheduler): add tests for snapshot, remote update, stale cleanup, and cron invalidation - Add stale run cleanup tests (bulk markStaleRunsAsFailed, logging) - Add cron invalidation test (auto-disable, error message) - Add executeSnapshot tests (fleet capture, empty stacks) - Add executeUpdateRemote tests (proxy success, remote error) - Document stale run cleanup and cron auto-disable in troubleshooting docs	2026-04-13 11:32:09 -04:00