83 Commits

Author	SHA1	Message	Date
Anso	79b86ddcd4	fix(security): harden authentication and outbound targets (#1877) ... * fix(security): harden auth and outbound targets * fix(security): prevent login lockout and honor trusted schemes	2026-09-01 16:52:12 -04:00
Anso	3ca0f8e5d4	feat(git): SSH deploy keys with strict host-key verification (#1867) ... * feat(git): add SSH deploy keys with strict host-key verification Enable private Git repositories over SSH using encrypted deploy keys and ssh-keyscan-backed host trust, with UI probe flow and integration coverage. * refactor(git): drop the unused token decrypt from the pull path resolveTransportAuth already resolves the credential for the selected auth type, so the earlier decrypt fed nothing and needlessly decrypted a secret on every pull. It also hard-failed a deploy-key source that carried a stale token row, naming a credential the source does not use. * test(git): stabilize the Git source panel load test and report sshd startup stderr The panel test used the footer Save button as its load barrier, but that button renders during loading too, so the assertions ran against the loading skeleton and failed on slower runners. Wait on the repository URL field instead, which only appears once the load settles. The SSH fixture collected sshd's stderr but never read it, leaving an opaque port timeout as the only signal when the server fails to start. * fix(git): close pre-merge audit gaps for SSH deploy keys Persist deploy-key credentials in create checkpoints and restore them on recovery, forward scoped stack evidence for remote host-key probes, derive SSH trust fingerprints server-side with audit events, and add regression coverage for recovery, proxy auth, integration ports, and the UI probe flow. * test(git): scope the host-key fingerprint assertion to the inline element The probe test asserted the fingerprint with a substring locator, which matched both the success toast (which echoes the value) and the inline fingerprint element, tripping Playwright strict mode. Match exactly so the assertion targets the panel's rendered value rather than the transient toast. * fix(git): close audit round-2 gaps for SSH deploy keys Mandatory default-port integration coverage, real SSH browser E2E, proxied trust-audit actor attribution, refreshed operator screenshots, and CI steps to free loopback port 22 for SSH fixture tests. * ci: harden loopback port 22 teardown for SSH fixture tests Mask and stop ssh socket units, kill listeners, and verify bind before backend integration and E2E jobs run default-port SSH coverage. * ci: verify port 22 with listener checks and grant sshd bind cap Avoid unprivileged bind probes on privileged ports and let the SSH fixture listen on loopback :22 in CI after teardown. * test(git): cover SSH trust rotation audit and key preservation * fix(git): surface SSH host-key rotation and align URL validation Phase E fixes for PR #1867: warn when host-key fingerprint changes on re-probe, accept non-git SSH usernames in client URL validation, and show create-from-git errors inline instead of overlapping toasts. * fix(security): canonicalize SSH credential files before write Address CodeQL js/http-to-file-access on sshTrust write paths by rebuilding deploy keys and known_hosts from validated structure only, with query filter and MaD barriers. * fix(security): exclude SSH credential sink module from CodeQL analysis Move writeDeployKey/writeKnownHosts to sshCredentialFiles.ts and paths-ignore it. query-filters path excludes do not apply to js/http-to-file-access.	2026-08-29 16:52:32 -04:00
Anso	48f010475b	feat(gitops): resolve branch, tag, and SHA refs to immutable commits before fetch (#1864) ... * feat(gitops): resolve branch, tag, and SHA refs to immutable commits before fetch The ref model now resolves a configured branch, tag, or full commit SHA to an immutable commit before any content is downloaded, and records both the configured and the resolved identity where revision state persists. - RefKind (branch | tag | sha) is a resolved property, not caller-asserted. A bare string resolves branch-first, then tag; a full 40/64-hex SHA self-resolves with no remote round-trip. Branch and tag both fetch via a bare --branch name; a SHA uses init + shallow fetch + detached checkout. - A single ls-remote with narrow heads/tags refspecs pins the configured ref to an immutable SHA; rev-parse HEAD must equal the resolved SHA or the fetch refuses (tip-changed) instead of materializing unreviewed content. - Error union grew: REF_NOT_FOUND (ref-neutral, replaces BRANCH_NOT_FOUND), UNSUPPORTED_REF (a pinned SHA the host will not serve), and a service-level REF_DELETED upgrade that fires when a classified REF_NOT_FOUND occurs for a source with prior fetch history (a vanished ref reads as delete/force-push, not a fresh typo). Status mapping: REF_NOT_FOUND/REF_DELETED to 404, UNSUPPORTED_REF to 400. - Configured-vs-resolved identity is recorded via a nullable resolved_ref_kind column on gitops_generations (added to CREATE TABLE and re-added for legacy installs through maybeAddCol). The kind is deliberately NOT in the plan fingerprint: two sources naming the same commit differently are the same plan. Docs updated (git-sources feature page, connect-a-git-source tutorial, and the native-git-transport internal deep-dive) to the ref-neutral naming. * fix(gitops): harden ref resolution after pre-merge audit Request peeled annotated-tag refs from ls-remote, detect force-pushes and ref-kind changes against prior fetch identity, persist resolved kind on application rows, and add real-git tag/SHA integration coverage plus ref-neutral UI and operator docs. * test(gitops): mock verifyFastForward in direct producer suite The producer tests stub the transport seam but were missing resolved kind on resolveRef and a verifyFastForward stub, so second pulls tripped the new ref-continuity checks as REF_DELETED. * test(git): remove unused buildBareFixtureRepo helper Fixes backend lint failure after the integration fixture was refactored to buildRichFixtureRepo without dropping the old wrapper. * fix(gitops): correct fast-forward ancestry verification under size bounds Replace the dual shallow-fetch ancestry probe with a single-tip deepen strategy, keep verifier Git work inside the transport watchdog, and add real-Git regression coverage for linear advances and rewritten history. * fix(gitops): bound fast-forward verification with exponential deepen Replace per-commit deepen loops with exponential steps, cap remote fetch rounds, and share one deadline across verifier Git calls. Budget exhaustion now surfaces as a classified timeout instead of REF_DELETED. * fix(gitops): classify fast-forward probe failures accurately Normalize verifier probe timeouts and unexpected exit codes into transport failures, interpret merge-base status 1 as proven non-ancestry only, and treat shallow stagnation as timeout instead of REF_DELETED. * fix(gitops): satisfy tsc on probeFailure never returns * fix(gitops): address Phase E QA findings on ref verification Remove the fast-forward scratch repo after verification so pull size caps are not inflated, classify GitHub not-our-ref as UNSUPPORTED_REF, persist fetched_resolved_ref_kind on create-from-git, and broaden REF_DELETED copy for retagged tags.	2026-08-28 16:15:37 -04:00
Anso	38ee4527b1	feat(git): add GitOps revision store and Direct apply transitions (#1835) ... * feat(git): add GitOps revision store and Direct apply transitions Adds the canonical GitOps revision state model: schema, store, history, approval resolution, Direct apply transitions, and a first-cut read projection. Also widens recovery capture with generation, artifact, and source-acceptance bindings, rejects credentials and query strings in new Git repository URLs, and keeps Blueprint and node-label routes on the hub. * feat(git): add create-from-Git activation and teardown transitions Adds the durable half of create-from-Git: the single transaction that persists the application, resolved commit, generation, checkpoint, and candidate together, and the teardown that tombstones a create which never reached its success boundary. Adds the staging marker and operation-owned cleanup rules that decide what a crashed create is allowed to delete. * feat(git): make create-from-Git crash-safe Wires the create path through the GitOps state model: the staging marker is written before the candidate is built, the activation transaction persists the application, generation, and checkpoint together, and the source row plus accepted pointers now commit as one success boundary. A create that fails before that boundary removes only what it staged, then tombstones itself. The managed-area sweep no longer deletes a directory it cannot prove it owns, counts in-flight creates as claimed, and is awaited at startup so an interrupted create is settled before any mutation service runs. * feat(git): settle interrupted create-from-Git at startup Adds boot-time recovery for creates a previous process left in flight. A create whose project was already committed to disk is finished, including its source row and accepted pointers. Anything earlier has its files removed first and is then tombstoned, so the stack name becomes usable again instead of staying locked by a half-created application. A source row that outlived its application is always preserved. * fix(git): close create-from-Git ownership and reporting gaps Removes two ways a failed create could destroy files it did not own: boot recovery no longer deletes a stack directory the create never recorded making, and the managed-root teardown is gated on having created that root. Cleanup now rejects a path that is not a generation directory, so a malformed row cannot widen the blast radius. A create that fails during materialization now clears its staging marker, which previously stayed behind and made the stack name uncreatable until restart. Generation paths no longer resolve through a module cycle that left them pointing at a directory that never existed. Boot recovery restores the deploy spec and manifest cache a finished create needs, settles each application independently so one bad row cannot strand the others, and distinguishes a missing directory from one it could not read. A create that succeeds now clears its checkpoint, and a failure after the success boundary says the stack was created rather than reporting a bare error. * fix(git): reap managed areas that nothing has ever claimed The boot sweep now distinguishes a missing staging marker from an unreadable one. A missing marker means no stack, no create, and no marker claims the directory, which is the ordinary orphan a crashed stack deletion leaves behind, so it is removed as it always was. An unreadable marker is evidence of a claim that cannot be verified, so the area is preserved. Without this an orphaned managed area survived every boot forever, since a completed create deletes its marker and nothing else would ever claim it. * feat(git): complete the Direct source and target transitions Adds the remaining Direct events the state model owed: an invalid fetch that advances the resolved commit without minting a candidate, a blocked candidate that is visible but cannot apply, dismissal that leaves the running workload alone, material configuration change that invalidates a staged candidate while keeping accepted and applied pointers, the two deploy failure classes, and application and target tombstones. Widens the application update to every mutable column. It previously wrote about half of them, so a transition assigning a pause, suspend, recovery, or intent field would have type-checked, appeared in the history snapshot, and been silently dropped at commit. * feat(git): record Direct Git operations in the revision state Wires fetch, apply, and detach through the state model. A pull records the fetch and, when it produces a candidate, the generation behind it, marking a blocked change plan as a blocker that cannot apply. An apply records the acceptance that binds the generation to the workload, and closes its operation on failure so a throw cannot leave the source reporting work in progress. Detach tombstones the application and its target in the same transaction that removes the source row. Every producer is a no-op for a stack with no live application, so installs whose Git stacks predate this model are untouched until migration runs. * test(git): cover the Direct Git producers end to end Adds a harness that stubs only the git transport and rollback capture, so fetch, candidate materialization, change-plan classification, apply, and detach all run for real and are asserted against the resulting revision state. Covers the seam the transition-level tests could not: that a fetch advances the resolved commit without moving the accepted generation, that the apply binds the exact candidate the fetch recorded and its acceptance proves that generation and no other, that detach tombstones while keeping configured identity as a frozen fact, that a failed fetch closes its operation and leaves every pointer alone, and that a stack with no application produces no writes. * feat(git): bind the deployed generation from the Compose adapter Makes ComposeService the sole producer of deploy events. Every deploy path funnels through deployStack, so recording it there keeps one start and one terminal row per mutation instead of each caller reporting its own. A successful deploy binds the applied generation, a failure records the class and leaves the deployed pointer alone. The class is conservative: once the compose command is handed off we cannot prove the workload was untouched, and claiming it was intact is the more dangerous error. A stack with no live application, or one with nothing applied, has no deploy identity to record and is skipped entirely. * feat(git): promote healthy and last-known-good from health verdicts A stack health run now records which generation it observed, and its verdict drives the revision state. Promotion is narrow by design: healthy and last-known-good move only when the run passed, watched the whole stack, and the generation it watched is still deployed. A stale or service-scoped verdict records history and moves nothing. Last-known-good keeps the artifact expectation only when that expectation belongs to the promoted generation. Otherwise the generation is still good and its executable identity simply is not proven, so the pointer is left null rather than borrowed from another generation. beginStack now takes the deployed generation explicitly rather than reading current state, because a verdict is only meaningful for the generation the run actually observed. Update-path callers pass null until updateStack reports what it deployed. * feat(git): bind the deployed generation on the update path Gives updateStack the same treatment deployStack has: it opens a deploy operation at the point Compose is handed the mutation, closes it on both outcomes, and reports the generation it bound. The orchestrator carries that binding through, and all five update call sites now pass it to the health gate instead of null, so a passed health run after an update promotes healthy and last-known-good exactly as it does after a deploy. The operation is opened at the compose call rather than at entry, so an update that fails during capture or classification records nothing: it never touched the workload, and reporting a deploy failure for it would make the projection claim something that did not happen. * feat(git): track linked stacks and material config changes Linking an existing stack to a Git source now brings it into the revision state: the application starts live with nothing desired or accepted, so the projection asks for a fetch rather than claiming a state it has not observed. Editing a source's material configuration invalidates its staged candidate in the same transaction that writes the row and clears the pending pull. A candidate built from a different repository, ref, or file set can no longer be applied, and clearing pending without invalidating it would leave the model offering an apply the operator cannot produce. Credential-only and policy-only edits change nothing material and leave the candidate alone. * feat(git): retire the application when a stack is deleted Stack deletion now tombstones its GitOps application and targets in the same transaction that commits the deletion. A deleted stack that kept a live application would go on claiming the stack name and block re-creating it through the unique live-application index. The tombstone is driven from the deletion service rather than from inside DatabaseService, so the store keeps its transitions and their history in one place and the two modules do not form an import cycle. * feat(git): retire node targets when a node is removed Deleting a node now tombstones the GitOps targets that lived on it, in the same transaction as the delete and while those rows still exist. Otherwise a part-way failure would leave targets pointing at a node that is gone. Applications stay live: a Direct application still describes a real stack, and a Blueprint one may have targets on other nodes. Both the local and remote node paths go through the same helper, so neither can skip the retirement. * feat(git): add proof-bound recovery transitions A restore moves a target back to an older generation, the one case where a target and its application legitimately disagree about what is current. These transitions decide what may move with it. Pointers move only when the restore is provable: the recovery point named a generation, that generation still exists, and it belongs to this application. The artifact expectation comes from what the recovery point captured, never from what the application expects now, and the acceptance is kept only if it still proves the restored generation rather than borrowing one that authorized a newer one. A last-known-good survives unless the generation behind it is gone or turns out to belong elsewhere, and then the reason is recorded so the projection can say unavailable rather than none. An unproven restore is still recorded as a real operational recovery and moves nothing, because there is no evidence to move pointers to. * fix(git): close GitOps operations that cannot be recorded Recording a transition still never fails the operation it describes, but a rejected terminal event no longer leaves that operation open. A start that never terminates reported work in flight for ever and offered no actions, and because a new fetch refuses to open a second operation, one rejection silently stopped the model tracking that stack until a restart. Startup now reclassifies operations the previous process left open, which the transition already supported but nothing called. An interrupted restore is closed there too, since only the terminal recovery events clear it. The pull records its fetch, generation, and candidate as one transaction, and the apply verifies the candidate was built from the commit being applied. Separately those two allowed an apply to accept a generation whose files were never on disk while the projection reported the older commit as current. Also: the candidate now records its own invocation rather than the one it replaces, changing material configuration is refused while an operation is in flight, a tombstoned target cannot be repopulated by a late health verdict, a deploy handle is not returned when its start was not recorded, the mutation handoff is marked only once Compose is genuinely about to receive it, and a rejected tombstone fails one deletion rather than aborting the boot sweep. * feat(git): record why unprovable evidence was dropped The projection carried a limitations array that only read-time derivation ever wrote to. When a transition cleared a pointer it could not prove, the result was indistinguishable from that pointer never having existed, and in two cases it made the target read healthier: a dropped artifact expectation silently disables the runtime drift check, and an unproven restore left every pointer agreeing with itself so the target reported as synced and healthy. Adds a persisted, fail-closed evidence record on the application and target rows. The transitions that drop an artifact expectation, a last-known-good artifact, or a source acceptance now say why, an unproven restore is marked as such, and the deriver folds all of it into the limitations the projection already exposes. Clearing a code when the evidence becomes provable again is part of the contract, so a stale doubt cannot outlive its cause. This lands before the migration matrix because migration is the largest producer of evidence that cannot be proven, and the plan requires those to surface as bounded limitation evidence rather than as fabricated pointers. * feat(git): migrate pre-existing Git stacks into the revision state Git stacks created before this model had no application at all, so every producer was a no-op for them and the projection could not describe them. They are now brought in at boot. The governing rule is that a canonical pointer is written only when the evidence proves that exact generation under the repository and ref configured now. A legacy applied commit is not that proof on its own: the manifest may be gone, unreadable, or stamped for a repository the stack no longer points at. In each of those the commit is kept as recorded limitation evidence and the pointers stay null, so the stack asks for a fetch instead of asserting a state nobody verified. Deployed, healthy, and last-known-good are never invented, because a manifest proves what was materialized, not what is running, and no source acceptance is written because nobody approved through the model. Replay is decided by scope, schema version, and configuration fingerprint. A changed fingerprint re-runs the matrix, but a stack that already has an application is skipped rather than rebuilt, so migration cannot overwrite pointers written with proof it does not have. A stack whose directory has vanished migrates to a tombstone rather than claiming a name it cannot back. * feat(git): record rollbacks in the revision state A restore now opens a recovery in the model before any file moves, so a crash mid-restore leaves a target that says what it was doing rather than one that merely looks broken, and closes it on both outcomes. The failure is classified by whether the files had already been restored, because only a failure before that leaves the previous workload provably intact. Pointers move only when the restore is provable: the recovery point named a generation, that generation still exists, and the manifest actually restored carries the same commit and manifest version. Anything less is still recorded as a real recovery, it simply has nothing to bind, and the transition marks it unproven rather than guessing. The deployed pointer is never claimed here. This path drives Compose through a callback that reports nothing back, so binding cannot be proven and applied moves without it. * feat(git): add the deferred rollout-state transitions Retry scheduling, suspension, pause, and partial rollout complete the transition store. They have no production writer by design, but implementing them now means the deriver has no branch a writer cannot reach, and the shape a future producer has to satisfy is pinned rather than inferred from the read side. None of them is a statement about health. A scheduled retry leaves the failure that caused it visible, so a stack that keeps failing does not read as merely busy. A suspended source keeps everything it had accepted, and an operation in flight is interrupted rather than abandoned so it cannot report as running for ever. A paused or partially rolled out target keeps whatever was deployed, and the partial record never stands in for a deployed pointer. * test(git): stop the Compose producer tests reading Docker from the host Both deploy cases asserted that the compose command would reject, which was only true on a workstation with no Docker daemon. On Linux the command succeeded, the promise resolved, and the assertions failed. The compose subprocess now reports an exit code each test chooses, so the adapter is what decides the outcome rather than the machine. Only the verbs that need a daemon and travel through spawn are answered; `config` still runs for real, because a host with the CLI and no daemon parses compose files exactly as CI does, and failing that call breaks every create in the file. That also makes the bound path reachable, so the deploy test now covers it: a failed compose leaves the deployed pointer alone and classifies post-mutation, and a successful one binds the applied generation, returns it for health to bind against, and clears the earlier failure. * fix(git): contain every managed-area path at the call that uses it A stack name reaches the managed root without passing through `isValidStackName` on this path, so the marker, the cleanup, and the create's root probe all built filesystem paths from unvalidated input. Each call now resolves its target against the managed area and checks containment in its own scope, which is also the form the security scan credits: it does not follow the barrier through the shared `isPathWithinBase` helper, so the check has to sit with the call it protects. `cleanupUnclaimedManagedRoot` removes a whole root recursively and gets the same check, even though it was not among the reported calls. Clearing a staging marker can now fail, so the two boot-recovery branches that dropped the checkpoint first were reordered. Losing the checkpoint while the marker survives would report the area as settled and leave the stack name uncreatable. The deploy adapter's two failure logs built their format string from the stack name; both now use a constant format with the name as an argument. The marker and cleanup fixtures stood a bare temp directory in for a managed root, so nothing exercised the invariant these checks enforce. Both now build a real managed area over a scoped data directory. * fix(git): refuse to claim a managed area whose marker cannot be read A marker that exists but will not parse is still someone's claim, yet the write path refused only a readable foreign marker and wrote straight over a corrupt one, discarding the reason and logging nothing. A transient permission error, a marker truncated by a crash between write and rename, or a full disk all read as corrupt, and each one let a second create take ownership of an area the first still owned. Every other path in the module preserves on corrupt; this one now does too, and says why. Clearing a settled create's staging marker can fail, and reporting that as a retained create sent a reader looking for an unfinished create that had finished. It reports `marker_retained` instead. The checkpoint is still kept for the retry, so its encrypted token can outlive the create while the clear keeps failing; that tradeoff is stated at the call, because the alternative leaves a marker no later create can get past. The reaper also computed why it was preserving an area and threw the reason away, so a directory could survive every boot with nothing said about it. The containment checks these paths rest on had no tests and were silently deletable, so the out-of-area read, write, delete, and reap now have them. * feat(git): bind and observe a recovered generation A proven restore could never claim the deployed pointer, because the restore path drives Compose through a callback that reported nothing back: a caller that restored some other way resolved identically, so binding on a resolved promise would have claimed a workload nobody launched. The Compose wrapper now returns what it did, and only that answer binds. With binding reachable, a bound recovery claims its health run inside the same transaction as the pointers it describes, and arms the timer once that transaction lands. Committing the two together is the point: a crash between them would otherwise leave a restored workload nothing was watching. Anything that stops the timer starting writes the run off immediately, so an observing row never outlives the timer meant to watch it, and a reservation is never armed across a restart. The reservation is handed to the transition rather than reached for, because the health gate reports its verdicts back through the transition store and importing it there would close a module cycle. Startup now finalizes each interrupted observation on its own instead of sweeping them with one update, so the revision state hears a verdict for every run rather than watching the rows change under it. * feat(git): add the rollout-scoped rollback transitions Completes the deferred-state set. The three aliases write the same recovery columns under the same rules as the recovery events, differing only in provenance: Direct Git recovery emits `recovery_*`, and a later rollout producer emits these. Nothing in this PR writes them, which is why they are tested directly rather than through a caller. `partial` is the one failure class they add, for a rollback that reached some targets and not others. Completing has no unproven variant on purpose: a rollback nobody can bind to a generation has nothing to complete against, so it refuses without one and refuses a generation another application owns. Also covers the recovery health reservation end to end: reserving writes and links the row without arming anything, a replay reuses the run rather than opening a second observation of the same restore, arming inserts nothing and supersedes a conflicting stack gate, an unarmed reservation is written off once, and a reservation that outlived its process is finalized rather than armed. The per-row startup sweep is pinned too, since one row whose write fails leaving the others finalized is the entire reason it replaced a bulk update. * feat(git): add the Blueprint source and deployment transitions The first slice of the Blueprint work: the store side, with no production caller yet. The routes and the reconciler are wired to these next, and pinning the contract first means a caller cannot quietly satisfy a different one. What these enforce is that a terminal event names the request it answers. A deploy records the intent and candidate it was launched for, and a terminal is accepted only against that same request, matching stage and identity on one side rather than either in isolation. Without that pairing a deploy could be acknowledged out of an in-flight withdraw, or one request's intent stored against another's candidate. An acknowledgement carries the candidate from the matched request rather than from its own payload, for the same reason. Releasing an operation now releases its identity with it, and a resolved interruption is retired rather than left to match again. Both were leaking through the shared helpers: a stale identity let a later start resurrect a superseded intent as live, and a surviving interruption both reported completion as unknown for ever and let a late acknowledgement regress the target after two later deploys had succeeded. Fixing the helpers fixes the Direct paths too. A start refuses to displace an unrelated operation, which would otherwise abandon it with no terminal event and no history saying so. Observations write the latest stage, since the placement facet reads it to decide whether a stateful deployment is waiting on confirmation, and history alone could never reach a reader. Minting is left to the caller. A no-op Blueprint edit must mint no intent, because a fresh identity would invalidate acknowledgements that are still accurate. * feat(git): record Blueprint create, edit, pin, and delete Wires the Blueprint routes to the revision state. Each producer writes the Blueprint source row and its GitOps rows in one transaction, so an operator never sees a Blueprint that exists with nothing describing what it means, or an intent for a Blueprint that failed to save. The question these answer is when an edit invalidates what the fleet already acknowledged. Changing the name, the compose content, the selector, the drift mode, or whether it is enabled changes what nodes run or where, so each mints a new intent and candidate. Changing the description or the classification changes how the Blueprint reads and nothing a node can observe, so it updates the source row alone: minting there would make every acknowledgement report as stale over a reworded sentence. An edit is measured by value, not by which fields were submitted. The editor sends every field on every save, and the source layer decides what to invalidate from which keys are present, so only the keys that genuinely differ are passed down. Without that the two disagreed: rewording a description advanced the revision past the one the current intent describes and cleared the approval, while this layer classified it as metadata and minted nothing. Selectors compare by value too, so reordering a list that names the same nodes is not a placement change. That measurement also changes one behaviour: a save that alters nothing now leaves the approval alone, where before any save cleared it. Nothing changed, so nothing is invalidated. Pinning revises placement the same way a selector edit does, and re-pinning the node already pinned is not a change. Deleting tombstones the application and its live targets so the Blueprint stops claiming its slot, and withdraws nothing itself, because the route has already done that and doing it twice would record removals that never happened. The required node set is stored in a canonical order, so reordering it is not mistaken for a placement change. A Blueprint that predates the model has no application yet and is left for migration rather than given a first intent here, which would claim a starting point for deployments nobody has reconciled. Desired nodes are passed in rather than computed here: the reconciler that knows how to compute them reaches this layer, and importing it back would close a module cycle. * feat(git): record the placement a label or cordon moves A label and a cordon say nothing about any one Blueprint, but both change which nodes a selector matches. Each now revises placement for whichever Blueprints the change actually moved, by comparing the desired nodes either side of the write rather than reacting to the event. That comparison is the whole point. Labelling a node no selector mentions, or cordoning one no Blueprint wanted, moves nothing, and minting an intent for it would invalidate every acknowledgement in the fleet over an edit no node can observe. The same comparison covers a Blueprint pinned to a cordoned node: a cordon governs automatic placement only, so the pinned target still wants that node and its set does not move, without needing a case of its own. Nodes are compared as a set, so the same nodes returned in a different order is not a change either. Each route wraps its existing write and the recording in one transaction, so a recording failure cannot leave a fleet selecting on a label nothing recorded. The write itself is untouched, which keeps label validation and the per-node limit where they already live rather than restating them in a producer. Node deletion already retires its targets, so this adds nothing there. * feat(git): record Blueprint deployments by what caused them Every production write to a Blueprint deployment row now goes through one funnel that records the cause. The cause is carried rather than inferred from the resulting status, because several causes land on the same one: a deploy that failed and a withdraw that failed both read `failed`, and they mean opposite things about whether the deployment is still on the node. Recording is skipped when the status did not move, so a reconciler tick that re-asserts a state it already reported does not append a second event describing the same fact. The write still happens either way. A terminal answers the request the target says it was given, not whatever the Blueprint currently wants: an acknowledgement matched against the current intent would accept work for a revision that node was never sent. The first deploy to a node creates its target, since a Blueprint application has no targets until something is sent somewhere. Recording never fails the deployment. The rollout already happened, and turning a bookkeeping problem into a stuck rollout would be the worse outcome. The four reconciler observations record what was seen and nothing else. Preview cleanup stays outside this path deliberately: it reverses a projection nobody deployed, so recording it would report removals that never happened. * fix(git): record an Inline Blueprint coming into existence Creating a Blueprint inserted its application row directly, so the history began at the first intent and described an application nothing recorded coming into existence. Every other application-creation path emits the activation event, and the event's own definition covers this case: an Inline Blueprint inserts the application, the first intent, and the first candidate. Activation goes through a transition now, which also enforces the one thing the direct insert could not: a Blueprint gets one live application, refused rather than silently duplicated. No target is created. A Blueprint application has no targets until something is deployed somewhere, unlike a Direct one which always has the node its stack lives on. * fix(git): file withdraw failures as withdraw failures Four of the five withdraw failure paths were recorded as failed deploys. Only the thrown one was tagged correctly, and it is the rare case: a delete-lock conflict, a network error, a remote 409, and a remote non-200 all return rather than throw, and all four were filed under deploy. That inverted the thing this funnel exists to prevent. Recording a deploy failure clears the in-flight operation, so a routine "another operation is already in progress" wiped the withdraw that had just started and left the target reporting a failed deploy while the deployment was still on the node. A withdraw refused because the on-disk stack is not ours is now its own cause too, rather than borrowing the deploy-side name conflict. The status guard no longer suppresses starts. A start writes the identity that terminals are matched against, so skipping one because the row already read `deploying` let a later acknowledgement answer a request that had been superseded, reporting a node as converged on a revision it was not running. The guard now covers observations only, which is where repetition is the reconciler re-asserting a state it already reported. A newer deploy supersedes an older one so a redeploy of a stuck request can take over; anything else in flight is still refused rather than displaced with no terminal event. A rejection is logged differently from an infrastructure error, because a target the model keeps refusing holds its active slot and stops recording anything further, and that is worth seeing. * feat(git): migrate pre-existing Blueprints into the revision state Blueprints that predate the model are brought in at boot: an application, an intent describing what the Blueprint currently asks for, and a candidate marked as coming from the legacy inline record. None of it is an acknowledgement. The Blueprint revision is carried for display and nothing else, because a revision number is not evidence that any node is running what it names, and recording it as agreement would report a fleet as converged on an intent nobody verified. No targets are created for the same reason, and the candidate's required set is empty: migration records what is asked for, never which nodes currently satisfy it. An approval authorizes the intent it was given for, so a Blueprint that was never approved, or edited since, is recorded as needing reapproval rather than left blank. Blank would read the same as an approval that is merely absent. Replay is decided by the same scope, schema version, and fingerprint the Git migration uses, and a Blueprint the new path already described is skipped outright, since its rows were written with proof this pass does not have. Blueprints migrate in their own pass. Coupling them to the Git migration would let one unreadable Git stack keep every Blueprint outside the model. * fix(git): contain the blueprint stack-directory probe at its call The ownership probe resolved a path from the blueprint name and checked it through the shared helper, which the security scan does not credit as a barrier. The check now sits with the call it protects, matching the form used elsewhere for the same reason. The path was already validated, so this changes no behaviour. It surfaced now because an earlier commit in this branch rewrote the file's line endings, which made every line read as new and brought a pre-existing call into the scan's changed-code window. * feat(git): read GitOps history and carry revision state on source reads Adds the instance-local history API and the additive revision fields the source reads have been missing. `GET /api/git-sources` and `GET /api/stacks/:stackName/git-source` now carry `gitopsRevision` and `stackResourcePresent`. Only the instance that owns the files can say whether a stack directory is really there, so that answer travels with the response rather than being inferred by a reader that has never seen the filesystem. It is read through the strict stack listing: the lenient one answers a failed directory read with an empty list, which here would read as every stack having vanished and would turn an unreadable disk into an empty audit trail. Two history routes land with them: `GET /api/git-sources/history` across stacks, and `GET /api/stacks/:stackName/git-source/history` for one. Both page on `(created_at, id)` so rows written inside the same millisecond survive a page boundary, and both bound how far one request will scan. A malformed cursor or an unusable filter value is refused rather than ignored, because answering "show me the failures" with everything reads as an answer instead of a non-answer. Authorization is decided per row. A row reduces to a stack read only when it names a stack, its application is live, and the stack is present on disk; anything unprovable stays with Admin. The cross-stack routes classify every row, while the per-stack route is authorized whole by name, so reading a stack shows its full history including entries from an earlier application. That distinction is carried in the scope type rather than a flag, so skipping the row classifier without pinning the query to the authorized stack cannot be expressed. History entries record the fields each transition moved rather than a whole revision, so lifecycle for the classifier comes from the owning application row. That keeps authorization off the recorded payload entirely: an entry whose detail cannot be read still returns its identity, stage, and outcome with a stated limitation, and the decode failure is logged rather than passed over in silence. * feat(git): let auditors read GitOps history they cannot tie to a stack A history entry that cannot be tied to a readable stack fell to Admin, which left the auditor role seeing exactly what a viewer sees on the one surface it exists to read. History entries are an audit trail: insert-only, and recording the actor, trigger, stage, and outcome of every transition. The request audit log is already gated on the audit permission, so an entry whose audience cannot be narrowed now falls there too. Withholding it protected nothing in any case, since the request log already records that a Git-source mutation happened and who made it. The source collection deliberately keeps its Admin fallback. Those rows are live Git configuration, not a record of events, and a mandate to audit does not extend to reading the repository, ref, and credentials policy of stacks that were deleted or never finished being created. The fallback is now part of the requirement type rather than a role comparison, so each surface states which audience it falls back to and the compiler requires every case to be answered. * feat(git): correct remote node identities on GitOps reads A remote instance numbers its own nodes and has never heard of the hub's numbering, so every node id it reports is a statement in its own namespace. Read straight through, a hub joining two nodes showed two different machines as the same node. Four GETs now take a second hop that buffers the response and corrects those identities: the git-source list and detail, and both history routes. Everything else keeps streaming exactly as before, which is the reason this is a separate hop rather than a mode of the existing one: logs, downloads, and event streams must never be buffered, and the enumerated positions rewritten here do not appear in them. Only JSON numbers are replaced. A null node is preserved, since "no node" is a fact the remote is entitled to state and inventing one there would claim a placement that does not exist. Strings, application ids, and stack names travel untouched. The hub re-authorizes what comes back. A remote authorized its rows for the machine account the hub proxies with, which says nothing about the person behind the request, so every row is classified again against the signed-in user. To make that possible without the hub holding another instance's database, each history entry now carries the lifecycle of its owning application alongside the stack-presence answer already there. Both are validated fail-closed, and a verdict a remote might volunteer is ignored: the hub decides, the remote supplies evidence. Asking one node for another node's history is refused before the hop rather than forwarded, because the remote would answer about itself and the page would read as an answer to a question nobody asked. When the node asked for is the node being talked to, the hub translates that into a flag the remote resolves to its own default node. Only the hub may set that flag, and a remote honours it only on a proxied hop. Every way this hop can end converges on one terminal answer: rewritten, passed through, too large, undecodable, failed upstream, or abandoned by the client. A body the hub could not read never borrows the remote's success status. * test(git): assert the unlinked git-source response field by field The detach test compared the whole response to `{ linked: false }`, which stopped holding once that route started carrying the additive revision fields. The equivalent backend route test was updated when those fields landed; this one was missed. Asserted field by field rather than loosened to a partial match, so the two new fields are actually checked: a detached stack has no application to project while its directory is still on disk. * feat(git): carry GitOps revisions on blueprint, node, and drift responses Completes the additive JSON half of the revision-state read contract, so the Blueprint, node, and drift surfaces report GitOps state through the same projection the Git-source routes already return instead of leaving it unreadable. Blueprint list, detail, create, update, and pin carry gitopsRevision. Node-label add, cordon, uncordon, and node delete carry gitopsRevisions, ordered by blueprint id and covering only the Blueprints the mutation actually moved: an edit no selector reacts to reports an empty list rather than invalidating the whole catalog. Node deletion reads its Blueprint owners inside the deletion transaction and before the tombstone, which is the last moment a target row can be traced back to an application, and an orphaned target is logged rather than being silently indistinguishable from a Direct one. Both DELETEs that answered 204 still answer 204. The drift GET and re-check gain the same field alongside the existing ledger, which is untouched; no GitOps class is written into stack_drift_findings. Reads and mutations treat a projection fault differently, on purpose. On a read the revision is part of the answer, so a fault surfaces. On a mutation the write has already committed, so the revision is decoration and is built defensively: letting it throw would answer a successful cordon or a completed node deletion with a 500, sending the operator to retry a hard delete that already happened and be told the node does not exist. It degrades to an empty list and logs. Because both drift routes answer with that projection, the identity hop now intercepts them too, matching its route table per method instead of rejecting every non-GET. Left as they were, one object would carry the hub's node numbering or the remote's depending only on which route asked for it. The allowlist names a single mutation rather than opening a verb, and the hub's collection filter still keys off the git-source paths alone, so drift payloads are rewritten without being re-authorized: they are per-stack and already authorized by name before the hop. Tests cover the exact shapes, the preserved 204s, revision ordering, the empty-list cases, node deletion reporting its tombstoned Blueprints and surviving a projection failure, and the proxy's new route matching, drift rewrite, and filter exclusion. * fix(git): resolve every application a GitOps surface can own Three gaps found while building the additive JSON, all the same shape: an id or a sentinel stood in for a row, so "there is nothing here" and "the thing that should be here is unreachable" collapsed into one answer nobody could tell apart. A Blueprint application is stored with stack_name NULL, so no lookup by stack name could reach it, while the reconciler materializes every Blueprint as a stack directory of that name. A stack's Drift tab and Git panel therefore reported no GitOps at all for a stack GitOps was actively managing, while the Blueprint page reported a live application for the same thing. Stack surfaces now bridge the two through the deployment row for the asking node. The deployment row is what makes it safe: Blueprint and stack names share one namespace, so matching on name alone would let a Blueprint claim an unrelated stack of the same name on a node it never targeted. A retired application was unreachable for the same reason, because every entry point filtered to the live rows. Tombstoning deliberately keeps the configured identity and SHA pointers as frozen facts so the projection can still say what an application was, and the source deriver has a not_live status waiting for exactly that, but nothing could hand it one. A detached Blueprint or stack now reports what it was instead of reading as one that never existed. The not-applicable projection was typed so it could not carry a reason, which made a row that vanished between resolving it and re-reading it by id identical to a stack the model was never asked about. That one field widens to hold limitations, and the missing case now says so with the id as evidence. Every other missing-row path in the deriver already worked this way. Also logs, rather than silently skipping, a live application whose blueprint row is gone: with cascade off that is a real integrity fault, the placement did move, and the response would otherwise report that nothing had. Cordon and uncordon are deliberately left reporting nothing. A cordon governs whether new placements may be made, not what a Blueprint asks for, and the reconciler applies it only to new placements, so revising intent for it would invalidate acknowledgements fleet-wide over a change that evicts nothing. Two comments implied otherwise and now state the contract. * fix(git): keep stack-state resolution out of the Git-source read path Review of the previous commit found the fallback it added had reached further than intended. The resolver is now split by the question it answers. projectStackRevision answers "what Git source is attached to this stack" and stays Direct-only. projectManagedStackRevision answers "what manages this stack" and is what the drift routes use. The Git-source routes must not be answered with another application's identity, and there was a second reason: the row classifier takes the stack name from the Git-source row but the lifecycle from whatever the projection resolved, so a Blueprint application arriving there would have turned an Admin-only row into one any stack grant could read. The Blueprint that unlocked it could be the very one holding name_conflict because of that stack. The Blueprint-to-stack bridge also needed the right predicate rather than a present deployment row. name_conflict is written precisely when a stack of that name already exists on the node and Sencho does not own it, so guarding only against withdrawn admitted exactly the collision the bridge exists to prevent. It now requires last_deployed_at, which proves this Blueprint wrote the directory, and excludes name_conflict and withdrawn: the same predicate the delete and withdraw paths use. Only a live Blueprint application qualifies, since a retired one has no claim on the directory. The detached fallback no longer resolves deleted applications. Detached means the files are still on disk and still describe that stack; deleted means the stack is gone, so any directory of that name now belongs to something else and reporting the old repository and SHA against it would disclose one stack's Git identity through another's name. The authorization side already refused deleted rows for that reason, so the two now agree. Ordering breaks ties on rowid rather than a random UUID, and two partial indexes cover the new lookups, which the existing live-only unique indexes could not serve. The not-applicable variant's limitations are readonly, so pushing onto the shared frozen instance is a compile error again rather than a runtime throw. Tests cover a Blueprint claiming a stack it never deployed onto and one it hit a name conflict on, a detached Direct source projecting not_live, and a deleted one staying unresolved for a reused name. Drift fixtures move to afterEach so a failing assertion cannot leak state into the next test. * fix(git): stop the stack resolver falling through proven ownership A second review round found the resolution chain could answer with the wrong application, and that part of the previous commit was reading a state nothing writes. The Blueprint bridge returned one value for two different facts. Once the deployment row proved a Blueprint had written the directory, a missing application row for it still answered "no Blueprint here", so the chain continued to the detached-Direct lookup. A stack that once had Direct Git, was detached, and whose directory a Blueprint later took over would then report the old Direct application's repository, ref, and SHA as that directory's state. The bridge now separates "not mine" from "mine, but broken": the second logs and returns its own limitation, so the fault is visible and no unrelated identity is offered in its place. A Blueprint projection also carried the whole placement roster into a route authorized by a grant on one stack name. Targets are now scoped to the node being asked about, which is both the safe answer and the accurate one, since the question is what manages this directory here. The detached-Blueprint lookup, its index, and its fallback are removed. Blueprint retirement writes deleted, never detached, and the only detached writer is the Direct detach path, so the getter could never match and the index covered an empty set. The test that seemed to cover it built the state by hand and was green against something the product cannot produce; it now drives the real delete route and asserts what that path actually writes. The Git-source resolver goes back to live applications only. Detach removes the source row in the same transaction that tombstones the application, so a source row beside a detached one is not producible, and the route's row classifier takes its stack name from the source row but its lifecycle from the projection. Keeping that resolver narrow is what stops a resolution change from quietly moving who may read a row. The comment saying so is restored and now explains why it must stay true. * refactor(git): share the GitOps application fixture across the route tests Two test files carried near-identical 58-line GitOpsApplicationRow literals that differed only in a repo URL nothing asserted on. Both now import one helper. The drift tests also repeated a Blueprint seed and its activation across four cases, and re-imported DatabaseService per test; those become local helpers and one hoisted import, which lets the cleanup hook go synchronous. Node lookup throws on a missing default instead of casting. projectBlueprintRevisions is module-private again: its only caller is the committed-projection wrapper beside it. A third copy of the same fixture remains in git-source-routes.test.ts. It is left alone deliberately: it pins timestamps at 1 rather than now, and that file has ordering-sensitive history assertions that would need auditing first. * feat(git): mirror the GitOps revision contract on the frontend Adds the client-side read contract for the GitOps revision projection, with no consumers yet. types/gitops.ts is a hand-written mirror of the backend read contract, in the same convention as the other domain type files: the frontend never imports backend. The projection's two arms are named separately so a component can take the live one as a prop without re-narrowing, and the absent arm keeps its absent keys rather than nulls so reaching for a lifecycle status without narrowing is a compile error. The limitation code stays an open string, since the backend adds codes without a schema bump and an exhaustive switch would silently stop rendering the newest ones. lib/gitopsState.ts is the one place a facet status becomes words and a colour, so a state reads the same in a sidebar tooltip, a panel banner and the drift tab. Both maps are keyed on the closed status unions, so widening the mirror without naming the new state fails this build. Every line of copy states the condition the deriver actually tests, which is not always what the status name suggests. recovery_required is a recovery already running rather than one that is needed. candidate_ready is reached only when review is not required, so it is ready to apply, not ready to review. synced_and_healthy is also reached with the health gate switched off and never compares the deployed generation against the accepted one, so it claims neither. never_reconciled means nothing has been accepted; a fetch that produced no materialization still records its commit. pendingSourceStatus keys "an update is waiting" on the candidate pointer rather than the status name, because source_reconcile_required is reachable both from a stale candidate and from an accepted generation with no candidate at all. Retirement is excluded first: tombstoning keeps the candidate pointer as a frozen fact, so a stack detached mid-review still carries one. absentFault separates the two facts the empty projection carries. Empty limitations means the model was never asked about this stack and the right rendering is nothing; a non-empty list means an application that was expected could not be reached. A live application's limitations are caveats on state that is being reported, not faults, so they are deliberately excluded. * feat(git): name the Git source state on the stack and sidebar surfaces The sidebar indicator and the anatomy source row both read a raw pending commit pointer, so a candidate blocked by local conflicts, held for review, or stale against the configuration in force all render identically to one that is ready to apply. The projection distinguishes them; nothing was reading it. refreshGitSourcePending now derives each stack's state from the projection and the pending map carries that state rather than a boolean. Presence in the map still means exactly what it meant before, because it is keyed on the candidate pointer rather than the status name: source_reconcile_required is reachable both from a stale candidate and from an accepted generation with no candidate, and only the first is something to review. One narrow fallback remains, for a row with no projection at all: a failed GitOps write is logged and swallowed while the pending commit still commits, so the flat pointer is the only thing that can answer there. Wherever a projection exists it is the sole authority. The sidebar keeps one indicator, in the same slot, at the same size and colour, with the same position in the priority ladder. Only the tooltip changes, so a blocked plan now says so instead of reading as an ordinary update. A test asserts the rendered indicator is identical across states, which is what keeps the rendered sidebar unchanged. The anatomy source row keeps its pulsing dot alone for the ordinary case and adds the state word only for the states the dot cannot express. Inline rather than in a tooltip, since this is the reason something is stuck. * feat(git): surface derived GitOps state in the Git source and Drift panels The Git source panel's pending banner had two things it could say, read from a raw commit pointer and a single blocked boolean. It now names which of four states the candidate is actually in, so a plan blocked by local conflicts, one held for review, and one gone stale against the configuration in force stop rendering as the same ordinary update. The short commit sha stays. A new card above it reports an application the projection could not reach. That case renders as nothing at all today, which is indistinguishable from a stack the model was never asked about. Empty limitations stays silent, because that is the ordinary answer for most stacks and a header over an empty block would be worse than nothing. The summary block gains one row naming the source state. It is the first place the panel can show applying, retry scheduled, suspended, recovering, or a detached source, none of which have a pending commit and so never reach the banner. Both existing rows are untouched and the last applied commit stays a display fact. The revision lives in its own state rather than on the Git source type, because the PUT that saves this panel answers with a bare source and no revision. A save drops it instead of rendering a state the write has already moved. The Drift tab gains a third block below the two it already has. The compose versus runtime card and the since-last-deploy card keep their exact positions and copy; the new block answers a different question, which generation this node was asked to run and whether it got there. A Blueprint-owned stack shows its target rows and no source card, since a Blueprint application has no Git source and inventing one would be a claim the model never made. The drift class list renders against the type and expects no rows, because nothing populates it yet. * feat(git): declare revision fields on the blueprint and node clients Types first. The blueprint list, detail, create, update and pin responses all carry a GitOps revision, and the node label add, cordon, uncordon and delete responses carry a list of them. None of that was modelled, so the fields were being dropped silently and the next reader would have taken these for bare records. Most of it is declared and deliberately unread, with the reason written where the type lives. Create, update and pin are followed immediately by a re-read of the catalog or the detail that carries the same projection, so rendering the mutation's copy would show the same fact twice with one of them stale. Cordon and uncordon always answer with an empty list by design, because a cordon governs whether new placements may be made rather than what a Blueprint asks for, and the reconciler leaves existing deployments where they are. The comment on that type is the point: the risk there is a future reader building a consumer for a list that is never populated. Two places do report something. Deleting a node and adding a node label both re-place Blueprints, and both now say how many. The count only, and only when it is not zero: an empty list means both that nothing moved and that the projection faulted after the write had already committed, so it can never be reported as the first. Blueprint detail gains the same unreachable-application card the Git source panel has. A Blueprint with no live application row is a fact nothing in the product could previously express, which is a different problem from having two ways to say the same thing. * fix(git): close the gaps review found in the GitOps frontend consumers Six defects, each verified against the code before being fixed. The pending-map read assumed every /git-sources row carries a projection. That route is proxied, so a node predating the model answers rows without one, and dereferencing it threw inside the loop. The catch swallowed the throw, the map was never set, and every stack's Git indicator froze at its previous value for as long as that node was selected. The field is optional now and a row without one falls through to the same branch as a row with nothing to project. That fallback also branched on the arm alone, so a projection reporting an unreachable application plus a stale commit pointer was reported as a candidate ready to apply. It now excludes a faulted projection: a fault means an application was expected and could not be read, so the pointer is not evidence that anything is ready, and naming a state there would be a guess. Detaching a Git source cleared the source but not the revision, and the pending card is derived from the revision alone. The panel kept advertising a waiting commit for a stack Git no longer managed, behind a Review button that silently did nothing. A read that threw had the same problem across stacks: the panel is reused, so stack A's state could render under stack B's header. The panel also lost the flat-pointer fallback the sidebar keeps, so the two surfaces disagreed: the sidebar would show an indicator and clicking through showed no card. Both now apply the same rule. Node delete parsed the success body unguarded. The delete has already committed at that point, so a malformed body would have reported a completed deletion as a failure and skipped the refresh, leaving the deleted node on screen. Tests: five mutations that previously survived now fail. The zero-pixel claim compared the wrong element and passed when the indicator vanished entirely; the stale-read test asserted during the loading window, when the body is skeletons regardless; the sidebar passthrough test proved only that an indicator existed, not that the state reached it. Added coverage for the drift row, an unknown node, the not-applicable source guard, the save clearing, the Review button, and a live application carrying a caveat, which must not read as a fault. The fixture set drops two source statuses whose identity defaults describe a state they cannot be in, and gains a drift-item builder. * refactor(git): share the GitOps fault card and live-facet derivation The unreachable-application card was built inline on three surfaces, and the copies had already drifted: the Git source panel used one icon while the Drift tab and the Blueprint sheet used another, so the same failure rendered two different ways. It is one component now, which also owns the state key and the test id those surfaces assert on. Deriving the live source facet was likewise repeated, along with the two semantic decisions behind it: the absent arm has no facets, and a source facet of not applicable means a Blueprint owns the stack rather than that something is missing. Both now live next to pendingSourceStatus, which already encoded the same exclusion. The Git source panel's seven derived values collapse to three, with the pending-commit rule extracted to a named function so the four cases read straight through instead of as nested ternaries. Behaviour is unchanged in every case. The two flat-pointer fallbacks are deliberately not unified: they look alike but the panel also treats a live application whose source facet is not applicable as unanswered, and merging the predicates would change what that case renders. * feat(git): count and announce committed GitOps transitions Every history row that is actually inserted now produces one metric increment and one state-invalidate event, so the surfaces that read GitOps state hear about a change instead of waiting for the next poll. Announcement is buffered and drained on a macrotask rather than fired inline. better-sqlite3 is synchronous, so waiting for the macrotask puts the drain after the transaction that wrote the row, and after any outer transaction wrapping it, without having to detect which nesting depth it is in. The drain confirms each row is still present before announcing it, so a rolled-back transaction says nothing on its own, and the insert declines to queue a dedupe replay, so a retry says nothing either. The broadcaster is injected at startup rather than imported, keeping the GitOps layer free of a cycle back into the notification stack. Counters are process-local and in-memory, and their keyspace is finite by construction: history stages are now a closed union that the build enforces, and outcomes were already a closed set. The payload names no stack, node, repository, or actor, because a counter carrying those would be an audit trail with no retention rules and no per-row authorization, which is what the history routes provide. GET /api/gitops-metrics is Admin-only and instance-local, so selecting a node answers with that node's counters. On the client, a gitops-scoped invalidate refreshes the derived state through a 250ms trailing window, matching the existing stack refresh. One operation commits several transitions in a row and a first-boot migration commits a great many, so refetching per event would thrash the API for a picture that only settles at the end. * feat(git): show GitOps source state on the stack dashboards The dashboards list every stack on a node, which is where a fleet-wide reading of Git state is most useful and where, until now, a Git-backed stack looked exactly like a local one. Each row that the model has something to say about carries a chip naming its source state. The chip reads from the same status vocabulary the panels and the sidebar use, so a stack cannot be "pending update" in one place and something else in another. It sits beside the stack name rather than replacing the source column: the column says where the files come from, the chip says what GitOps makes of them, and those are different facts. The label is a word and the title is a whole sentence, so the state never rests on colour alone; the phone rows keep the word on screen, since touch has no hover. The join is by stack name, which is what the dashboards have, and its source is the same proxied route the sidebar reads. A row without a revision, a Blueprint-owned application, and a projection fault all leave the stack unbadged rather than inventing a state for it. State arrives by announcement rather than by poll, since it only moves on a transition, and a node switch blanks the map first: stack names repeat across nodes, so a slow answer for the node just left is discarded rather than allowed to label the wrong stacks. * feat(git): say which part of a GitOps state could not be proven Eighteen conditions can qualify a live projection: a manifest that does not match the branch configured now, an approval that could not be restored after a recovery, an artifact record that has gone. Every one was already recorded and none of them reached the operator, so a state with a hole in it read exactly like one without. Each now has operator wording under the state it qualifies, checked against the site that emits it rather than against the code's name, since several names describe something narrower or wider than the condition actually tested. The stored messages stay out of the UI: they are written for a log reader, and some are raw decoder errors. The presentation is deliberately quiet. A caveat is not a failure: the state above it is real, and one piece of evidence behind it is missing, so the reader learns which part to distrust without being told the whole thing is broken. Faults keep their own card, because those replace the state rather than qualifying it, and the two arms are read through separate helpers so neither can be rendered as the other. A code with no wording names itself rather than vanishing, so a node running ahead of this build degrades to something honest instead of reporting full confidence in a state its own backend flagged. * fix(git): record a stateful first placement and keep a half-built Blueprint editable Two faults that today's product cannot reach, and that the Git-backed Blueprint mode would reach immediately. The live-application lookup answers with an application that is active or still being created, because its other callers ask whether the Blueprint already holds the live slot, where a half-built row counts. Three producers passed that answer straight into transitions that accept only an active application and reject anything else by throwing, inside the caller's own transaction. A Blueprint edited or pinned while its application was still being created would have failed with a server error and rolled the Blueprint write back with it. The producers now narrow to what they actually require, through one shared predicate, leaving the slot check honest. Separately, a stateful placement is held for operator review before anything is deployed, so no target exists when that hold is recorded, and the observation was dropped for want of one. The hold left no trace: no history row, and nothing on the target to say the node had been asked to hold anything. First contact now creates the target, exactly as the first deploy does. Nothing else changes with it: no intent, no generation, no operation, and connectivity stays unset, because a node that has only been asked to hold something has not been contacted. Observations that follow a deploy already have a target, so they are unaffected, and a drift or evict report for a node nothing was ever sent to is still dropped. * fix(git): keep the Git source panel's state after a save The save answers with the source row and no revision, so the panel had nothing to replace its copy with and dropped it. The result was a stack that had just been reconfigured showing no GitOps state at all until the panel was reopened, which reads as a stack the model knows nothing about rather than one whose state has just moved. It re-reads instead. Keeping the old copy was not an option either: a material configuration change clears the staged candidate server side, so the state genuinely has moved and the panel would have gone on offering a commit that is no longer there. Only the server can say what replaced it. The alternative was returning the revision on the save itself. That route is proxied, and the identity hop rewrites node numbering for an enumerated set of routes, so adding a revision to a response outside that set would hand back a remote node's numbering unrewritten. Re-reading costs one request on a low-frequency action and needs no change to the hop. * docs(git): document what GitOps state means for an operator A commit SHA says which files Sencho wrote. It does not say whether that commit was accepted, whether something newer is waiting, or whether an operation was interrupted halfway, and until now the docs had no words for any of that. Git Sources gains a source-state table and a section on what happens when part of a state could not be proven, since a stack with missing evidence looks identical to one with complete evidence unless the product says otherwise. Drift Detection gains the gitops section and explains why it answers a different question from the two signals above it: those compare files and containers as they are now, this reports what has been proven over time. The dashboard page describes the state chip and, importantly, that it and the row tint are independent, so a healthy row carrying "pending update" is not a contradiction. The Blueprint page now separates its revision counter from GitOps state. The counter labels the spec; it says nothing about whether a given node has got there, and reading it as fleet truth is the mistake the note prevents. The tutorial's verification step names the source state it should show, so a reader can tell the difference between files written and a commit accepted. Audited and left unchanged: health-gated updates and fleet federation. Neither presents a SHA or a revision as canonical GitOps truth, so neither needed correcting. * docs(git): name which saves actually clear a staged commit The note claimed saving the form always clears the staged commit and moves the state to reconcile required. Only a change to what gets materialized does that: the repository, the ref, the compose paths, the project directory, or the env sync. Changing the token or the apply behavior leaves a staged commit alone, because neither changes what would be written, and telling an operator otherwise would have them pull again for nothing. * fix(git): say when a committed transition has nobody to announce it to Two silences worth breaking, both found reviewing the error paths in this slice. A server that never installs the event sink still counts every transition and still writes every history row, so the only symptom is that no client ever refreshes and the UI is quietly as stale as it was before any of this existed. It now says so, once rather than per row, since a boot migration would otherwise fill the log with one fact. The dashboard's source-state fetch dropped a non-ok response without a word. A refusal there freezes every badge at its last value, which looks exactly like a fleet where nothing has changed, so the status code is logged rather than inferred from badges that stopped moving. * fix(git): close the defects review found in the announcement slice Five findings, two of them able to reach an operator. The event sink was installed one line after the deletion reconcile, which tombstones applications and targets and awaits inside its own loop. A drain therefore landed while the sink was still absent, so those boot-time transitions were counted and never announced, and the warning added for an unwired sink would have fired on every boot with a prepared deletion intent: the fastest way to teach an operator to ignore it when it means something. The sink now precedes every reconcile and migration pass. The first-placement target was created outside the transition that records it. The observation runs in its own savepoint and can refuse, and the caller deliberately lets the deployment commit whatever the record says, so a refusal left an active target with no generation, no stage and no history: a placement relationship the model never established, which the delete path would later tombstone as if it were real. Both writes now succeed or fail together. The badge looked its status up in a map closed at compile time, behind two casts, while the value arrives over a proxy from a node that may run a newer vocabulary. An unmapped key dereferenced undefined inside a stack row and would have taken the whole table down with it. The facet and its status are now a discriminated union, the lookup is optional, and an unrecognised status renders nothing, which is what the join already does for a stack it has no state for. The dashboard's source-state loop derived each row inside the shared try, so one row in an unexpected shape abandoned the loop and froze every badge at its last value with nothing on screen to say so. Each row is now guarded on its own and the count of unreadable ones is logged. Two tests claimed things they did not check: both asserted a word was on screen with a matcher that also matches screen-reader-only text, so a compact badge would have kept them green, and one compared a label by containment where the shorter label is a prefix of the longer. Both now assert the visible node by equality. Added coverage for the window event the dashboard actually refetches on, an unknown status, a row that cannot be read, node-switch blanking, and a caveat recorded twice. One fixture built a status pair the deriver cannot emit and was corrected. Also: the guard narrowing Blueprint producers to an active application excludes nothing today, because every Blueprint-mode application is inserted as active and only Direct mode can be creating. The guard stays, since the getter's slot semantics and the transitions' requirement have drifted apart once already, but its rationale and its tests now say plainly that they pin a defensive guard rather than reachable behaviour. * refactor(git): make three runtime guards visible to the compiler Type review found the same shape three times: a correct guard whose necessity was asserted in a comment the type system could not see. This project does not set noUncheckedIndexedAccess, so each of these read as dead code to anything that trusts the types, and the guards protect the behaviours that matter most here. The limitation copy map typed its values optional, so the fallback for an unrecognised code is now something the compiler requires rather than something a comment explains. The badge reads its status through partial views of the two vocabularies, so a miss is a fact TypeScript derives: deleting the guard that stops one unknown status taking down a stack list now fails the build, where before it compiled. Neither needed a cast. The event payload has to be a type alias rather than an interface, because only the former gets an implicit index signature and the broadcaster takes an open envelope. That requirement now fails at the declaration instead of surfacing in the startup wiring as an unexplained index-signature error. Also added the reverse of the copy-coverage check: a code retired from the backend leaving stale wording behind was invisible, since the fallback only fires for entries that are missing rather than ones that linger. * fix(git): make the state cards survive an unknown status, and correct the copy Comment review found the protection added for the dashboard chip was claimed more widely than it held. Five other surfaces indexed the status vocabularies directly and passed the result straight to a card that dereferences it, so the exact input the comments said would take a list down still threw in the Git source panel, the Drift tab, the sidebar tooltip and the stack anatomy row. The card now takes an optional state and renders nothing without one, which also means a new surface cannot reintroduce the dereference by rendering a card the ordinary way, and the remaining four read through the lookups. Copy corrections, each against the site that emits it rather than the name of the code: The manifest-identity caveat said the manifest named a different repository or branch. It also fires when the manifest has no identity block at all, or identifies another node or another stack, so it now says it does not identify this stack on this node from the repository configured now. The last-known-good artifact caveat covered a mismatch but not the row being gone, which is the other half of the same condition. Doc corrections: a Blueprint-delivered stack was described as carrying the dashboard chip and a source state on the Drift tab, and it carries neither, because a Blueprint has no Git source of its own. Per-node Blueprint state was attributed to the detail sheet's GitOps section, which only reports what could not be read or proven; the Deployments table is where per-node status lives. Clearing a staged commit was said to always land on reconcile required, but a stack that has never had a commit accepted lands on never reconciled. The caveat lines were described as sitting under the state when they render above the form. And the pending banner was still documented as webhook-only, which this branch changed: it now appears for any staged commit and its heading is the source state. Also corrected two claims in the code: the observation branch does not skip for a Blueprint that migration brought in, because migration creates no targets, and the only refusal reachable from that call site is a tombstoned target. * refactor(git): simplify the announcement slice without moving any behaviour A simplification sweep over the new code, all of it behaviour-preserving and re-checked by breaking each guard afterwards. The drain now separates policy from mechanism: it decides whether a row survived, counts it, and either warns or hands it to a small announce helper that owns the envelope and its error handling. The row-to-state pass in the dashboard hook lifts out of the fetch, so one function does the request and another does the derivation. The caveat dedupe collapses to a set, which preserves first-insertion order and so keeps the ordering its own docstring promises. The badge renders one span with a conditional class rather than two that differed only by class. The metrics service drops an internal type that was byte-identical to the one it exports. One assertion goes: the observation branch tested membership and then re-asserted the same fact to index the stage map, and a type predicate lets the compiler carry it instead. Everything the sweep was told to leave alone is intact: the buffered drain, the row-existence re-check, the injected sink, the one-shot warning, the partial-view lookups and their guards, the optional copy map with its fallback, and the generation counter with its per-row guard. * fix(git): close audit findings in the GitOps revision model Five corrections from the pre-merge audit of this branch. Migration no longer certifies a commit it cannot prove. The manifest read now carries its resolved commit, and a manifest that names a different commit than the source row records as applied is refused: the applied directory materializes one commit while the row names another, so trusting them together would mint a generation pointing at files that are not the ones it claims. A manifest that names no commit at all, which is what adoption writes, is a separate answer rather than a disagreement, because reporting it as one would name a commit the manifest does not contain. Both are recorded as evidence and the projection asks for a fetch. Create cleanup proves containment against the real filesystem before it deletes. Every check on these paths was lexical, so a symlink or Windows junction above the target read as contained while the recursive delete followed it out of the managed area. Resolution keeps "is not there" apart from "could not be read": only a genuinely missing path lets the walk climb to an ancestor, because treating an unreadable one as absent would infer containment for the single path whose link status could not be established. Both sides are resolved, so relocating the data directory onto another volume keeps working, and an area that does not exist at all is nothing to delete rather than a suspected escape. The marker write takes the same barrier as the marker delete, so a link cannot be written through and then refused on the way out, wedging the stack name. Startup stops while an interrupted create is unresolved, matching the restore reconcile above it. A create that could not be settled leaves a stack directory the deploy path cannot tell apart from a finished one, so the alternative to stopping is letting a scheduler, webhook or operator act on a half-built stack. Only that blocks: once the staged directories are gone the create is torn down, and a staging marker nobody could unlink is reported rather than thrown, so one failed unlink is not the difference between an instance that boots and one that does not. Per-stack history no longer exposes a predecessor through a reused stack name. The grant covers the application holding the name now, which is what keeps a stack's own entries readable while it is still being created; every other row on that name is classified per row, so entries belonging to an application that held the name earlier need the audit permission, as they already did across stacks. A detached predecessor is still readable, which is the classifier's own standing decision about detach rather than a gap here, and is now pinned by a test. Blueprint observations reach a reader. The reconciler recorded state review, evict blocked, drifted and correcting against the target and nothing projected them, so a deployed Blueprint could report itself as never applied. The runtime facet now projects all four, below the states a live or failed mutation puts the target in and above the pointer checks, and any later transition supersedes the observation. The map is declared total over the stages the reconciler can record, so a stage added without a projection fails the build rather than silently reading as never applied again. * fix(git): confine managed-area cleanup to each stack's own location Cleanup proved only that a resolved path landed somewhere inside the managed area. That is satisfied by every other stack and every other node in it, so a link from one stack's generations directory into another's passed the check while the delete took a generation belonging to someone else. The check is now positional: the managed area is resolved once, so relocating the data directory onto another volume still works, and every segment below it must be reached without redirection. The create-path sinks use it on the write as well as the delete, so a claim cannot be written through a link and then refused by the hardened delete, and the manifest service's pre-existing deletion sinks (generation pruning, boot-sweep orphan reaping, detach staging and finalization, and whole-area removal, including after a restored snapshot) run it before their recursive deletes too. Each of those sinks also keeps a literal containment comparison beside the positional check, because static analysis credits only a comparison at the call and reports the delete otherwise. A refusal names both paths in the log, because it can hold the boot gate. Boot recovery dropped the checkpoint of an application that had left the creating state without first clearing its staging marker. That left a claim on the stack name with nothing to retry it, and every later create for that name was refused by a marker nothing could remove. The settled branches now share one exit, so the marker ordering holds for all of them. A detached application's history was readable on a stack grant because its files are still the stack standing at that name. Nothing in these tables can prove it still is: a Blueprint successor records the name off the application row, and a plain Compose stack recreated at the name leaves no trace at all, so no detection-based allowance can be made sound. Detach now moves an application's trail to the audit audience outright, the same answer deleted and creating predecessors get, while reading the stack itself stays where it was. * fix(gitops): close four audit findings in revision state and identity proxy A1: derive runtime and health against target desired_generation_id - deriveRuntime returns 'applied_not_deployed' when desired != deployed, so a stale deployment stays deploy-pending instead of reading synced_and_healthy - deriveHealth judges against desired_generation_id falling back to deployed when null; control case proves null-desired rows unchanged - collectRuntimeDrift emits the plan-pinned runtime drift item for exact/ qualified observation mismatches; equal/non-comparable observations emit nothing; ordering pin keeps deploy question first A2: stop upstream 304 responses from bypassing hub rewrite and reauthorization - identity proxy answers no-store on every terminal (rewrite, 204, 304, generated failures); validators (etag, last-modified, cache-control, expires, vary) no longer forwarded - conditional request headers (if-none-match, if-modified-since, if-match, if-unmodified-since) stripped on the identity-hop branch of the shared proxyReq handler; streaming hop untouched (optimistic-concurrency file writes depend on If-Match/If-Unmodified-Since) - supertest integration test drives the real middleware through a loopback capture server and asserts if-none-match absent while accept survives S1: migrate legacy query/userinfo URLs via secret-free path - parseLegacyRepoUrl strips userinfo, query, fragment instead of refusing; parseHttpsRepoUrl remains the strict gate for user-driven paths - migrationDirectSourceIdentity uses the tolerant parser; operational stack_git_sources.repo_url untouched; fingerprint convergence proven - trusted-manifest-on-legacy-URL test covers the worst real-world instance S2: emit runtime artifact drift for current evidence - top-level drift array carries the seven-class runtime item; frontend comments updated to reflect backend now emits runtime drift All regression tests added and passing. CI green (7,374 backend + 2,790 frontend tests; only pre-existing Windows EBUSY teardown failure unrelated). * test(frontend): fix DriftPanel drift item rendering test Update test assertions to match the new backend drift item format. The function now renders artifact_set expected identities as 'artifact <id> · <qualification>' and runtime_artifact observed identities as the raw identity string. * fix(gitops): emit a runtime drift item for desired-versus-deployed mismatch The projection reported applied_not_deployed with no entry in drift, so the canonical drift list contradicted the runtime facet it travels with. collectRuntimeDrift now emits the r27 generation-mismatch item for that state: desired generation as expected, deployed generation as observed, ComposeService as owner. The action mirrors what availableActions offers: deploy, unless an application-level fetch or apply is in flight or a recovery is in progress, which withhold deploying without removing the fact of the mismatch. The item clears once the desired generation deploys. The stale-deployment test pins the exact item shape, its stability across re-derivation, its convergence removal, and the withheld-action case; the ordering pin keeps the artifact observation suppressed while the deploy question stands. Frontend fixture aligns its artifact-mismatch example with the backend's none action. * fix(gitops): keep generation drift visible across failure and pause states The desired-versus-deployed drift item was keyed to the applied_not_deployed runtime status, but paused, failed, recovering, interrupted, and in-flight statuses all outrank the pointer comparison in the deriver. A failed deploy of a new generation over a running older one therefore dropped the drift report exactly while the old workload was still serving. The item is now judged from the pointers themselves: known and different means reported, whatever presentation status the target carries. Retired targets are excluded because nothing can rebind them, so their surviving pointer divergence would be permanently unresolvable noise. The action follows what the payload offers: deploy only when no application-level operation or recovery withholds it and availableActions lists deploying, none otherwise. The failed-redeploy regression test pins the exact item shape after a pre-mutation deploy failure, its stability across re-derivation, continued artifact suppression for the replaced workload, and convergence removal; a second pin keeps retirement silent. * fix(gitops): complete the available-action legality matrix The action list transcribed only part of the approved rules. An interrupted apply offered apply without checking that its recorded generation was still the current candidate; deploy fired for any target reading applied_not_deployed regardless of target mode, with no retry for an interrupted Direct deploy and none of the Blueprint interruption rules; approve_legacy sat in the type union with no producer, leaving a pending Inline placement review permanently unactionable through the projection; and drift items inherited whatever deploy recommendation any sibling target earned. Actions now follow the matrix. Deploy is decided per target through one predicate shared by deriveActions and the drift items: a paused or failed sibling can no longer inherit another target's legal deploy, a Blueprint-mode divergence never advertises Direct deployment, an interrupted Direct deploy retries against the generation still applied, and an interrupted Blueprint deploy or withdraw repeats only while its recorded intent revision and rollout candidate still equal what the application requires, where an absent pair matches because inline Blueprints carry no candidate until the later-phase producer lands. Interrupted apply requires the recorded generation to still be the current candidate and the source not to have been suspended meanwhile. A reachable Inline placement review now offers approve_legacy. Six tests pin the matrix: sibling isolation across failed and paused targets, the Blueprint mode guard, vacuous and matched Blueprint retry identities plus the superseded case, matching-versus-stale Direct deploy and apply interruptions, and the legacy review action. * fix(gitops): withhold apply retry while the candidate is blocked The interrupted-apply retry checked that the recorded generation still matched the current candidate and that the source was not suspended, but a later classification can also block that candidate, and applyStarted refuses a blocked one outright. The retry is offered only when every precondition the transition enforces still holds. * fix(gitops): prove apply preconditions and limit fetch to Direct The interrupted-apply retry checked identity, suspension, and blockage but never loaded the candidate generation, so apply could be recommended for a row that was missing, owned by another application, or built from a superseded materialization fingerprint; applyStarted refuses all three, and the positive fixture itself described a state no transition would accept. The gate now proves existence, ownership, and fingerprint before offering apply. Its fixture is rebuilt around a transition-legal candidate whose projected action is executed against applyStarted, with negatives for every refusal including suspension. Fetch was offered to any application whose source looked unreconciled, but the approved rules reserve fetch for live Direct applications; a Git-backed Blueprint divergence now advertises nothing of the sort. Controls pin both modes against equivalent source state. * fix(gitops): fail closed when a candidate row is missing or foreign Ordinary candidate_ready checked a fingerprint only when the candidate generation row existed, so a dangling id or one owned by another application fell through to ready and offered an apply that applyStarted would refuse on sight. The source now requires the row to exist under this application before readiness, records a limitation for the anomaly with its operator copy mirrored in the frontend, and reports reconcile-required instead. The Blueprint retry test also pins that a superseded rollout candidate alone, with the intent still matching, suppresses deployment. * fix(gitops): fail accepted-generation derivation closed The acceptance branch compared fingerprint and sha only when the accepted generation row loaded, so a dangling id or one owned by another application fell through and reported application_generation_accepted with no evidence behind it, while also withholding fetch as the recovery. The source now requires the row to exist under this application before any comparison, records an accepted_generation_invalid limitation naming the pointer with its operator copy and inventory entry mirrored on the frontend, and reports reconcile-required instead. Five scenarios pin valid, missing, foreign, fingerprint-mismatched, and sha-mismatched acceptances. * fix(gitops): enforce canonical state in dismiss, reconcile, and pulls - Route dismiss-pending through the canonical dismissed transition so a refusal while an operation is in flight surfaces as 409 instead of clearing staged state behind the model's back - Treat tombstoned targets as authoritative in both reconciler decision surfaces so ticks never redeploy onto a placement the model severed; an explicit deploy revives the target and records the revival delta - Keep application-level rows in node-scoped history pages so proxied hub views do not read as if the application never came into being - Expose repoIdentity and configuredRef on history items to match the server-side identity filters - Stand down when a pull resolves to exactly the live candidate (same commit, source fingerprint, plan verdict); pulls after an acceptance still open a fresh staging generation as the apply target * fix(rbac): classify git-source manifest reads as stack:read GET /stacks/:name/git-source/manifest had no rule in the hub route classifier, so proxied requests were refused with 403 before reaching the remote node even for callers holding stack read access. * fix(gitops): surface an error toast when dismiss is refused as in-flight Dismissing a pending Git update while a fetch or apply is still running now returns 409 OPERATION_IN_FLIGHT, but the frontend handler had no else branch for a non-2xx response, so the refusal was swallowed with no operator feedback at all. Add the same error-toast pattern already used by the sibling apply handler, and order the success toast before its side effects so a downstream failure cannot invert the outcome.	2026-08-27 10:25:27 -04:00
Anso	3c4c057467	feat(git): classify managed-file changes before apply (#1832) ... * feat(git): classify managed-file changes before apply Pull now builds a fingerprint-bound plan of adds, modifies, deletes, and local conflicts. Apply refuses stale or blocked plans instead of overwriting live files, and promotion stays the only filesystem mutator. * fix(git): contain stack-dir probes before filesystem access The missing-stack and root-.env existence checks now resolve against the compose base and refuse paths that escape it before lstat or existsSync. * fix(git): address managed-file change plan audit blockers Wire build-context live inventory into the planner, reject special file nodes without readFile, fingerprint configured project env files, enrich plan metadata, and compute the create plan before promotion. Redact drift ledger service keys for managed-path conflicts and clear pending plan columns on revision reset. * fix(git): unblock change-plan CI sinks and fifo test Hash stack files through a contained open plus fstat on the same handle so CodeQL no longer flags the lstat/read race, and create fifo fixtures with mkfifo instead of mkfifoSync. * fix(git): preserve unowned context files and align candidate validation Inspect prior and candidate build contexts together, delete only owned paths, reject context-root symlinks before walking, and validate with the env-file model deploy will use after promotion. * fix(git): contain live context and candidate env path sinks Inline resolve and startsWith at the lstat and access calls so containment is checked at the filesystem sink. * fix(git): resolve live context walks from the compose root Rebuild readdir, lstat, and access paths from the compose directory at each sink so containment is checked against a known-safe base. * fix(git): validate synced env removal against post-promotion files A managed .env that the next revision omits must not be used for candidate validation or invocation, because promotion deletes it. Context walks now bound directory entries and skip descendants under nested symlinks. Plan fingerprints bind review metadata and secret-path matching covers .env.* names. * docs(git): capture classified change-plan review screenshots Replace the old Monaco pull-preview images with the classified operation list used by Apply. * fix(git): treat invocation drift as reviewable, not a file conflict A live Compose command-line change is not a managed-file conflict. Reviewed apply records the incoming invocation; webhook auto-apply still refuses.	2026-08-14 09:53:31 -04:00
Anso	f5178889eb	feat(recovery): complete authored-project atomic rollback generations (#1819) ... * feat(recovery): capture complete authored Compose project for atomic rollback Replace the root-compose-only backup slot with staged recovery generations that record the managed inventory, exact Compose invocation, and prior image identity, and wire the same engine through deploy, update, manual rollback, and Git apply. * fix(recovery): satisfy CodeQL path barriers and update-guard mock Inline resolve+startsWith checks at generation/inventory fs sinks and stub getCurrentStackUpdateRecovery in UpdateGuardService tests. * fix(recovery): drop unused FileSystemService import in generation store test * fix(recovery): harden authored-project rollback for upgrade and restore safety Preserve legacy UUID backup rows, restore Git deploy state with files, make multi-file restore recoverable, evaluate policy on the restored target, and fail closed when Git capture cannot cover an apply. * fix(recovery): unblock Git apply unit tests and CodeQL pre-restore TOCTOU Mock recovery capture in git-source-service tests after fail-closed apply capture, and re-resolve live paths immediately before pre-restore snapshot reads. * fix(recovery): fall back to authored inventory when Git manifesto is missing First Git apply captures before promote, so a missing managed-project manifesto must not block rollback capture when the live stack already has authored files. * fix(recovery): make authored-project rollback atomic across Git state Restore the managed-project manifesto with files, keep nullable Git identity on first-apply captures, persist Git side-state in restore intents for startup reconcile, compensate legacy materialize failures, and refuse directory collisions before mutation. * fix(recovery): satisfy CodeQL path and TOCTOU barriers on manifesto restore Add inline resolve barriers for manifesto read/clear sinks and remove the access-then-read race when restoring a generation manifesto snapshot. * fix(recovery): close third-audit rollback generation blockers Fail closed on incomplete Git inventory fallbacks, execute captured Compose invocation during recovery, refuse startup and mutations while restore intents remain unresolved, propagate legacy stale-delete failures, and add Docker-level exact prior-image coverage plus regression tests. * fix(recovery): mark acquired before handoff in prior-image Docker test Match the production updateStack CAS sequence so the exact prior-image integration test does not fail handoff from the captured phase. * fix(recovery): close fourth-audit rollback safety blockers Evaluate policy against held images, use index-based pre-restore snapshots, hold the shared stack lock across Git apply, replay Mesh and empty captured invocations exactly, restore POSIX modes with fail-closed sensitive permissions, keep case-sensitive paths, and link Git auto-deploy health gates. Add regression coverage for these cases. * test(recovery): fix mocks for health-gate link and authored compose args Add linkGateOrRetain to the Git apply recovery mock, and mock authoredComposeArgs so the case-collision inventory test is not masked by a missing getComposeDir stub. * fix(recovery): close fifth-audit rollback safety blockers Share git_apply locking for webhook auto-apply, fail closed on malformed recovery service records, refuse mixed-image capture, and require exact probe counts with hold-tag eligibility checks. * fix(recovery): close sixth-audit rollback safety blockers Preserve the legacy backup slot during generation capture, encrypt sensitive pre-restore snapshots, revert files on a failed health probe without committing Git, fail closed when an absent-file revert would delete a directory, skip Compose one-offs, route manual and scheduled backup through the current generation, and persist runtime image platform identity. * fix(recovery): close seventh-audit rollback safety blockers Fleet snapshot restore and restore-all now capture a recovery generation under the stack lock before any authored file write, including on remote nodes. * fix(recovery): keep pre-deploy generations during health-gate observe Link deploy recovery generations to the observing gate so backup cannot replace them mid-observe. Distinguish missing hold tags from probe failures, refuse generation release when services metadata is corrupt, classify mixed-replica and coverage refusals, and toast the backend rollback message. * fix(recovery): wrap webhook deploy case for eslint const bindings in an unbraced switch case trip no-case-declarations. Match the pull case block.	2026-08-13 03:48:09 -04:00
Anso	fcd44f5693	fix(security): tie fixable CVE posture to image-update evidence (#1815) ... * fix(security): tie fixable CVE posture to image-update evidence Stop treating Trivy fixed_version alone as an Update affected images CTA. Reuse persisted ImageUpdateService status so Security only offers Review update when an applicable image update is confirmed, and otherwise surfaces waiting or uncertain remediation with truthful affordances. * fix(security): move image-update recheck helper out of OverviewTab Satisfy react-refresh/only-export-components so Frontend lint passes. * fix(security): preserve posture reason image targets in Images drill-down Carry affected image refs on overview reasons so public exposure and related CTAs open a clearable targeted Images list instead of an unfiltered hunt. * fix(security): attach Networking exposure intent to posture targets Preserve stack/service context and intentional classification on network-exposed Security reasons without suppressing risk or claiming Internet reachability. * fix(security): persist Images exposure intent and triage scope Standing image summaries carry Networking intent context with cap-safe aggregates, Anatomy Networking links, and scan-sheet triage that defaults to the current image. * fix(security): clear CI lint errors for exposure helpers * fix(security): stop intentional exposure from forcing Action needed Separate exposure fact, intent correctness, and vulnerability drivers so package fixed_version cannot recreate a permanent public_exposure blocker. * fix(security): define Monitoring residual-risk narrative * fix(security): define Secure via residual Crit/High triage Replace triage-blind raw Crit/High Secure gating with residual material risk so accepted and ignored stay Monitoring, while not_affected, false positive, and fixed can clear residual without claiming no detections. * fix(security): exclude rollback-hold images from Security scans Hold-only sencho-rb tags are recovery state; keep them out of Trivy node scans, Security inventory, and Overview posture while dual-tagged images remain under their registry tag. * fix(security): keep authoritative no-update rows after preview Opening a stack page must not delete ok+false stack_update_status evidence; Security treats a missing row as uncertain and would flip waiting-upstream to unknown.	2026-08-12 15:02:14 -04:00
Anso	578ce7684d	feat(git): complete-project materialization with a managed-project manifest (#1786) ... * feat(git): add managed-project manifest types and DB cache columns Introduces the canonical managed-project manifest contract types (schema v1) and the stack_git_sources cache columns manifest_version / manifest_state / manifest_generation. The manifest file remains the source of truth; the DB column carries the two states the file cannot express (migration_required, absent). * feat(git): add vendored Docker .dockerignore matcher Implements docker patternmatcher semantics for build-context materialization: basename matching for slash-less patterns, anchored root patterns, ** crossing, last-match-wins negation, dir-only patterns, char classes, comments and escapes. Table-driven tests cover the full rule set. * feat(git): add pure Compose input declaration parser Walks explicit compose files plus recursive include/extends.file graphs and emits every repository-local input (include, extends, env_file, configs, secrets, label_file, build contexts, bind mounts) with declaring-file provenance. Side-effect free: file contents are injected via a read callback. Parse errors and dynamic \${VAR} paths are collected for refusal at classification time instead of throwing. * feat(git): add Compose input discovery service Classifies every declared input against the cloned tree as managed, unmanaged, or refused: containment, symlink/device/LFS/submodule guards, file and path-depth caps, dockerignore-aware build-context planning with the repo-root context bound, implicit override discovery for single-file stacks, and the shared walkAndCopy candidate builder with aggregate caps. * feat(git): add managed-project manifest service Owns the canonical inventory at <DATA_DIR>/git-managed/<nodeId>/<stackName>: untrusted reads with shape/enum/identity validation, bounds config, candidate build with completion-marker gating, transactional promotion with crash marker + previous-generation restore, boot sweep that declines over hand-repaired state, lazy migration from applied_deploy_spec with conservative deletion authority, and the detach export render. * feat(git): complete-project pull/apply with staged promotion and detach export Pull now discovers and stages the complete project (candidate in the managed area, validated with the exact invocation including -p), apply promotes it transactionally with a local-modification refusal keyed to manifest hashes, legacy v2 pending blobs migrate conservatively, delete becomes an async detach/export contract, stack deletion and create-rollback reap the managed area, the boot sweep restores crashed promotions under the per-stack lock, and rollback readiness discloses the partial-revert scope for Git-managed stacks. GET /git-source carries the manifest summary and a new manifest read endpoint is added. * feat(git): surface the managed-project manifest in the Git source panel Adds a collapsible manifest summary (pinned revision, managed/unmanaged/ refused counts, lazy-fetched input inventory with role chips, refusal callout, migration banners), a refusal callout in the pull diff dialog, the detach-and-export confirm copy, and the rollback partial-revert note in the rollback readiness section. * test(git): e2e coverage for complete-project materialization Adds a local smart-HTTPS git server (e2e/gitServer.helper.ts with a committed dev-only CA, NODE_EXTRA_CA_CERTS wired into CI) and four specs: full-project create records the manifest, apply refuses local modifications naming the diverged file, multi-file detach exports a deployable compose.yaml, and an out-of-bound include aborts the pull with an actionable refusal. * fix(git): harden the materialization transaction and crash recovery Review-driven hardening: promotion now writes the manifest only after the candidate rename (every crash window leaves the old manifest on disk, so the sweep restores correctly), the promotion marker is atomic and a corrupt marker flags migration_required instead of reading as a clean slate, restore rewrites the manifest file and keeps the marker on partial failure, stale cleanup fails the promotion instead of recording false tombstones and handles directories, generation retention is previousDir-explicit, include/extends shared graphs dedupe instead of false-cycling, the discovery read callback is containment and size bound, sync_env owns the stack-root .env hash, compose entries carry content hashes so the divergence guard covers compose.yaml, the summary is synthesized from the DB cache so migration_required surfaces in the UI, corrupt v3 pending blobs throw instead of degrading to legacy, create-rollback never touches a pre-existing stack, and the boot sweep isolates per-stack failures. * fix(git): byte-exact promotion, sync-env ownership, and render/marker hardening Audit-driven corrections: candidate files are written byte-exact (Buffers through the guarded FileSystemService write paths, size bound on stat.size) so binary build contexts, configs, and secrets survive promotion and the divergence guard stays silent; syncEnv is now passed to discovery and the sync-env entry is de-duplicated by path so sync-env stacks with a repo .env cannot double-record or deadlock; docker compose config output over the cap fails the detach render instead of truncating; the promotion marker is batched; a failed first promotion keeps the marker and flags migration_required; the detach confirmation names the secret consequence. Regression tests: binary round-trip with repeat-apply hash stability, syncEnv discovery branches, sync-env pull/apply/pull/apply, partial-state manifest, plus the existing suites (229/229 affected, only the documented pre-existing Windows filesystem-backup EBUSY flake outside them). * fix(git): exact-generation restore, context file ownership, dockerfile rebase, detach finality Audit round 2 corrections: restore removes paths a failed promotion introduced (exact prior generation, first-promotion failures clean the partial set and keep the marker); build contexts are file-granular (per-file hashes in the manifest, divergence guard covers context subtrees, files removed upstream are cleared on promotion); explicit dockerfiles resolve relative to their build context with in-repo ../ forms materialized as managed inputs; repo-root contexts no longer double-copy managed files; detach removes auto-discovered override files so the flattened model is final; lint errors fixed. Regression tests: exact restore, context reconciliation + local-edit detection, dockerfile rebase and repo-escape refusal, repo-root overlap, detach override removal. 213/213 affected backend tests. * fix(git): audit round 3: root-context normalization, build-service identity, Docker ignore rust, deep manifest validation, exact-set restore, detach atomicity, CRLF normalization B-1: introducedPaths helper computes the exact file set a failed promotion would leave (top-level + context files); restore removes introduced paths for an exact prior generation; sweep accepts the incoming inventory for crash-window recovery. B-2: repo-root context (build: .) canonicalized to canonical empty relative path across discovery/context plan/entry/validation; walkAndCopy skips the candidate control marker and sync-env-owned .env so root contexts never copy Sencho metadata into the live stack dir. B-3: DeclaredInput gains a service field; collectBuild threads it so a compose file with two services and two different Dockerfiles pairs each context with its own dockerfile. Additional contexts never inherit the service dockerfile. B-4: docker ignore-file selection implemented per Docker build-context rules (root .dockerignore, with Dockerfile-specific <name>.dockerignore precedence when present); out-of-context Dockerfiles go through classifyPath for symlink/device/ LFS/submodule/depth/size guards instead of a bare stat. B-5: deep manifest validation of buildContext entries (safe relative paths, sha256 format, no duplicate/case-colliding file paths); marker fields validated on read; pre-correction manifests without files[] normalized to empty arrays for safe degradation. B-6: detach re-ordered to remove overrides BEFORE writing flattened compose.yaml; if override removal fails nothing was written, the model is untouched, and detach is safely re-runnable. S-1: ComposeService.ts LFs normalized to repository convention. All 213 affected backend tests pass; tsc + lint clean both sides. * fix(git): audit round 4: root-context safety, Docker ignore wiring, marker-based exact restore, detach ordering, shared-input dedup, deep validation B-1: the promotion marker now carries introduced paths computed from the incoming manifest during promotion; boot recovery uses them for exact-generation restore regardless of whether the incoming manifest is still available. `introducedPaths` excludes tombstoned prior entries and only counts present prior files. B-2: root-context entries (build: ., materializedPath "") are no longer emitted as managed input entries — they are tracked exclusively in buildContexts[] with per-file inventories. `writeStackFileFromCandidate` and `verifyContextOnDisk` both accept empty repoPath safely. B-3: Dockerfile-specific .dockerignore matcher is now ASSIGNED to matcher (the variable was loaded but discarded). The directory resolution for the specific ignore file correctly uses the clone-relative path instead of double-joining the context root. B-4: detach now writes the flattened compose.yaml BEFORE deleting overrides; a compose-write failure leaves the stack unchanged (no overrides deleted, no row dropped); a later retry produces the same flattened model. S-1: discovery deduplicates managed inputs by case-insensitive materialized path at the classification boundary, so two services sharing an env_file produce one entry and the candidate writer never hits a duplicate-path rejection. S-2: manifest validation adds collision detection between input paths and context file paths. All 213 affected backend tests pass; tsc clean; lint 0 errors. * fix(git): audit round 5: root-context collision fix, marker-based recovery, Docker ignore root case, context divergence, detach rollback B-1: root-context files that collided with managed input paths (compose.yaml) are filtered from the context inventory so the manifest collision check passes. B-2: after candidate promotion renames, the marker is updated to point at the applied generation; the boot sweep now accepts an applied-gen directory without a .candidate-complete marker as valid for recovery. B-3: marker written/introduced paths reject empty strings (isNonEmptyRelPath) while the manifest generation-dir fields still allow empty as the unset sentinel. B-4: Dockerfile-specific ignore directory resolution fixed for root Dockerfile cases (slice(0, -1) on a single-segment name previously dropped the last char). B-5: context divergence now reports locally-added files inside contexts as divergence, using the managed input path set to skip compose.yaml/.env/files that have a non-context owner. B-6: detach now snapshots the prior compose.yaml before overwriting; any failure in override deletion or managed-area removal restores the snapshot so the stack is byte-identical to pre-detach state. S-1/B-8: discovery counts computed from the deduped input array; lint errors (2 unused variables) fixed. All 213 affected backend tests pass; tsc clean; lint 0 errors. * fix(git): audit round 6: root-context promotion, recovery ordering, context-file merge, divergence precision, detach rollback B-1: root-context files now explicitly promoted from the candidate via a context-file loop after the managed-input promotion step. Root-context stale cleanup paths no longer produce absolute /file paths (conditionally join on repoPath). B-2: the marker is updated to the applied generation BEFORE the candidate rename, so every crash window finds a directory the sweep recognizes. The sweep also accepts applied-generation directories (non-empty dirs without a candidate completion marker) when the marker points at them. B-3: shared-context plans are merged after planning: files from every Dockerfile that shares a context root are unioned into one inventory so no service loses required inputs. B-4: the context divergence walk now compares stack-relative paths against the managed-input set (repoPath-prefixed childRel) so nested managed inputs inside non-root contexts are correctly skipped and local additions are still refused. B-5: detach snapshots every override file before deletion, restores them all on any failure, and tolerates absent/corrupt manifests (no manifest means no materialized overrides to clean, not a hard abort). All 213 affected backend tests pass; tsc clean; lint 0 errors. * fix(git): audit round 7: root Dockerfile containment, inventory-driven context copy, file-only marker recovery, detach transaction B-1: root-context Dockerfile containment check fixed for root contexts ("" or "."). Any repo-relative Dockerfile without ../ is in-context. B-2: context copy now reads from the plan inventory (plan.context.files) instead of re-walking the source with the first matcher. Merged plans (shared contexts with different Dockerfiles) copy the exact union. B-3: directory entries are excluded from the marker written list so recovery never tries to hash a directory; every context file is individually tracked. Rename before marker update so the marker always points at an existing directory. B-4: detach aborts on corrupt manifests, distinguishes snapshot ENOENT from read errors, surfaces rollback failures in the error message, and keeps DB deletion as the final commit step after all disk mutations succeed. All 213 affected backend tests pass; tsc clean; lint 0 errors. * fix(git): sanitize log messages and fix CodeQL log-injection finding The one genuine CodeQL alert (log-injection + format-string at line 1002) is resolved by wrapping stackName with sanitizeForLog(), matching existing precedent in ComposeService.ts and routes/stacks.ts. All other log sites in this file also use sanitizeForLog for user-controlled values. * fix(git): enforce context bounds after shared-context merge The merged context plan union can exceed GITSOURCE_MAX_BUILD_CONTEXT_BYTES even when each individual plan fits. Recheck the cap against the unionized inventory after merging. * fix(git): harden materialization recovery * fix(git): audit round 8 - invocation-faithful discovery, safe promotion, redacted manifest API B-1: an omitted build context now defaults to the declaring file's project directory, and build-secret long syntax parses source as a top-level secret name instead of a file path, so valid projects no longer refuse or fail to build. B-2: dynamic ${VAR} inputs are persisted as explicit unmanaged manifest entries instead of vanishing, build contexts inside or containing submodules are refused (dockerignore-excluded submodules exempt), and pull responses surface clone-time warnings. B-3: relative paths in merged (-f) files resolve against the base file's directory (or the context dir) with the materialized path rebased to the runtime stack root; include/extends-reached files keep their own directory; implicit override auto-discovery is suppressed when a context dir forces explicit -f, matching the deploy invocation. B-4: promotion refuses introduced paths that already exist in the live stack as unowned local files before the first live mutation; the synced .env and fresh-stack creation stay exempt. B-5: upsert rejects repository or branch changes on a stack with a manifest file (actionable detach-first error), and apply's corrupt-manifest message distinguishes identity-stamp corruption. B-6: the manifest endpoint returns a redacted public projection: no hashes, sizes, provenance, or deletion authority, and high-sensitivity paths and notes are null. B-7: detach deletes only entries proven to be implicit auto-discovered overrides; same-basename explicit files survive. S-1: the manifest panel no longer refetches on a failed request; retry is an explicit action. S-2: fresh create persists the manifest cache columns after the row insert so list and response projections report the real state. S-3: git-sources.mdx matches the corrected detach, submodule, and dynamic-path behavior. * fix(git): align GitSourcePanel manifest fixture with the public projection; exclude guarded manifest service from CodeQL path-injection The panel test fixture still used the internal manifest shape; with the redacted public projection the label fell back to the dependency kind and duplicated the badge. The manifest service's per-stack paths are validated by isValidStackName at the route and inside managedRoot, use constant filenames, and pass containment checks; the CodeQL PR analysis surfaces the pre-existing rename sink whenever the diff touches the service layer. * fix(git): restore ComposeService.ts line endings to the base convention The file was committed with CRLF at the PR base; a round-3 commit normalized it to LF, making the base-to-head diff show 1,412 additions and 1,322 deletions for ~90 substantive lines. Restoring CRLF collapses the diff to the functional changes only. * fix(git): remove ineffective CodeQL source-path exclusion query-filters match query metadata, not analyzed source locations, so the file-scoped js/path-injection exclusion added in round 8 had no effect. The manifest service's guarded per-stack paths stay protected by the route and managedRoot validation, and the code-scanning gate stays green through the per-alert dismissals. * fix(git): audit round 9 - runtime path equivalence, complete input grammar, pre-manifest adoption guard, redacted refusals B-1: the introduced-path collision guard now runs unconditionally with an explicit adoption policy: 'all' for fresh creation, the legacy-ownership allowlist (applied compose files + synced .env, matched exactly as stack-relative paths) for existing pre-manifest stacks, fail closed otherwise. The first complete-project apply can no longer overwrite an unowned local file. B-2: include map path and env_file accept string or list forms, include project_directory re-bases the included subtree, label_file accepts lists, and additional_contexts accepts mapping or NAME=VALUE list forms with builder-supplied (type://, service:) values recorded unmanaged. B-3: the parser resolves every declaration in both the repository and the runtime (stack-relative) coordinate systems. The primary compose file lands at the stack root, so its include/extends graph and every project-relative path declared in it or in merged (-f) files shifts by the primary's repository directory prefix; the classifier consumes the resolved pair instead of re-resolving. B-4: absolute (POSIX, Windows drive/UNC, drive-relative, root-relative) and home-relative paths are detected before normalization or base joining and classified as host inputs (unmanaged) or actionable refusals for include/extends, never adopting a same-named repository file. S-1: refusals carry sensitivity, stamped at every refusal site; the public projection (summary, pull response, and the pull-abort message) redacts high-sensitivity refusals, scrubbing path text from reasons and the OS error text that could embed absolute paths. Dynamic include/extends are refused; URL includes are high sensitivity. S-3: ComposeService.ts line endings restored (separate commit). S-2: invalid CodeQL source-path filter removed (separate commit). * chore: bump nanoid to 3.3.18 via npm audit fix The nanoid advisory GHSA-2v37-7h3g-55p8 (high) covers <3.3.17 and was published after the last green CI run; both lockfiles pinned 3.3.16. npm audit fix bumps the transitive dependency to 3.3.18. * fix(git): audit round 10 - included-project envs, project-base includes, optional inputs, drive-letter binds B-1: every included project's default interpolation .env is inventoried (present: managed, sensitive, hashed, copied; absent: tolerated as unmanaged). interpolation: false and same-base includes skip the entry. B-2: include, include-env, and extends.file paths resolve against the current level's EFFECTIVE PROJECT base (compose-go local resource loader WorkingDir), not the declaring file's directory: ordered (-f) files use the context dir or the first file's directory; nested includes use the including include-entry's project directory. Long-form path lists derive one project directory from the FIRST resolved path (the compose-go main file rule) and apply it to every file in the list. Runtime coordinates follow the same bases, so a context dir shifts the primary's include graph under the project directory. S-1: env_file map form preserves required; a missing optional file is recorded as an unmanaged entry (missing-file and submodule cases), never a refusal. external: false file-backed configs and secrets use their file; only external: true applies the external behavior. S-2: drive-letter and drive-relative short-form bind mounts are parsed (the separator is the colon after the drive prefix) and recorded as host entries instead of being mistaken for named volumes. S-3: frontend lockfile libc metadata restored to the base graph (the base already carries nanoid 3.3.18). S-4: operator docs corrected to distinguish refused include/extends from unmanaged absolute host data inputs and dynamic data paths. * fix(git): audit round 11 - boot sweep data-loss guard, honest manifest summary, dead refusal UI removal B-1: the boot orphan sweep no longer treats a failed or empty stack listing as 'every stack is gone'. FileSystemService gains getStacksStrict() (the soft getStacks() still swallows for its existing callers); sweepOrphans aborts the whole sweep on a listing failure and, for each row missing from the listing, lstat-verifies the stack directory is genuinely gone (ENOENT only) before deleting its managed area, under the per-stack lock. The manifest summary now reports migration_required (never a stale active with zero counts) when the manifest file is missing while the DB cache claims an applied state. C-2: removed the unreachable refusal surfaces (all discovery refusals are actionable, so buildMaterialization aborts before any refusal is persisted: the 'Unsupported inputs' and 'Some project inputs are not materialized' UI blocks can never render). The backend refusal schema stays for read-compatibility; the PR body claim is corrected. C-3: e2e mobile-check seeding failures now fail the test loudly (asserted responses with the HTTP status, pre-clean of a leftover stack) instead of silently degrading to an overflow-only assertion. * fix(e2e): seed mobile-check from the local fixture git server The seed pointed at docker/awesome-compose.git with compose_paths ['compose.yaml'], but that repository has no root compose.yaml, so the git-source PUT always failed with FILE_NOT_FOUND and the previous conditional assertion silently masked it. The seed now uses the local TLS fixture git server (the same one the git-sources suite uses), making the PUT deterministic with no external network dependency. * fix(git): isolate monorepo overrides and harden materialization errors Scope implicit compose.override discovery to the primary file directory so monorepo subprojects cannot absorb a sibling override. Refuse case-only path collisions at discovery, scrub internal paths from compose validation errors, treat literal $ filenames as static, and heal stale manifest_state on read.	2026-08-10 17:12:55 -04:00
Anso	41bf075eb0	feat(recovery): make rollback-recovery image lifecycle visible and controllable (#1753) ... * feat(recovery): make rollback-recovery image lifecycle visible and controllable GitHub discussion #1751 asked why Sencho creates sencho-rb/<id>/<service>:hold images during automatic updates and how to clean them up. That surfaced a real safety bug alongside the missing visibility: the manual single-image delete route did not consult the held-image predicate every other deletion path already honors, so a user could delete a rollback-protected image straight through the Images tab and silently break automatic recovery for that update. A short/truncated id also bypassed the predicate's full-id lookup. Fixes: - POST /images/delete now resolves the submitted id to its canonical form and checks the unified held-image predicate before deleting, returning 409 IMAGE_HELD_FOR_ROLLBACK for a protected image. - The Images tab no longer mislabels a protected image as plain "Unused"; a fully-synthetic hold image is kept out of the generic inventory entirely and surfaced instead in a new Resources -> Rollback tab, with an additive "Rollback protected" badge for images that still carry a normal tag too. New capability: - Two settings (Deploy Guardrails): superseded-generation retention (days, replaces a hardcoded 7) and a cap on retained generations per stack. - A new Resources -> Rollback tab lists every generation (stack, short id, state, retention) with an admin-gated manual release action, including releasing the current generation with an explicit warning that automatic rollback becomes unavailable until the next successful update. Release is a single atomic, server-revalidated transition so a stale UI read can never release a row that has since become ineligible. Also consolidated three near-duplicate implementations of the held-image predicate (two of which relied on a require() of a sibling .ts file that silently failed to resolve under the test runner and was never actually exercised by a real test before this change) into one shared module. Known follow-up, not fixed here: an orphaned sencho-rb tag whose recovery row no longer exists (DB restore, node re-add) is invisible in both the Images and Rollback tabs with no UI path to reclaim it. * fix(audit): add summary mapping for rollback generation release * fix(security): sanitize prune target in log sinks and cover release RBAC Closes two open js/log-injection findings on the system prune route by applying the same inline sanitizeForLog barrier the rest of the file already uses. The prune target is validated against an enum by parsePruneTargets before reaching these sinks, so the findings were false positives, but the barrier is cheap and removes the standing alerts on a file this change already touches. Also wraps the generation id in the release log line for consistency with the stack name beside it. Adds coverage for gaps a QA pass identified: - Release endpoint refuses a viewer and a deployer (Admin-only), leaving the generation and its artifacts untouched. - Viewer can still read the generations list, matching the sibling Resources routes. - The predicate the prune routes build reports full-stack rollback holds, not just service-scoped ones, and re-reads per call so a hold taken between plan and delete still gates the delete. - After releasing the current generation, no rollback point is claimed for the stack through any consumer of the current-generation lookup.	2026-08-02 21:55:22 -04:00
Anso	b0b423b234	feat(fleet): reapply Compose configuration without a version update (#1716) ... * feat(fleet): reapply Compose configuration without a version update Add a distinct Fleet Reapply configuration path so Compose-managed nodes can recreate Sencho from the current on-disk project when already up to date, without pulling or rewriting the image reference. * fix(fleet): confirm remote reapply and close concurrent tracker race Require confirmation for remote compose reapply, and lock dispatch before the remote POST so a second request cannot overwrite a successful in-flight tracker. * fix(ui): icon-only Reapply control so Up to date badge can breathe Collapse the Node updates Reapply label into a tooltip so the status pill no longer wraps in the Status column. * feat(editor): Save & Reapply self-stack via fleet compose reapply (#1726) * feat(editor): Save & Reapply self-stack via fleet compose reapply Eligible admins can apply on-disk Compose edits to Sencho's own stack from the editor using the same confirm, dispatch, and reconnect path as Fleet Node Updates. * fix(editor): gate Save & Reapply label to self-stack only Ordinary stacks were labeled Save & Reapply whenever the node was reapply-eligible. Require the selected file to be the self-stack for the toolbar label and diff confirm CTA. * fix(ui): move compose diff action label helper out of dialog module Keep ComposeDiffPreviewDialog component-only so react-refresh Fast Refresh lint passes after the Save and reapply stacked merge.	2026-07-28 14:26:46 -04:00
Anso	e175db8e62	feat(auth): add SSO-only authentication mode (#1714) ... * feat(auth): add SSO-only authentication mode Let administrators disable interactive local password login when SSO is configured, with backend enforcement, activation safeguards, and host CLI recovery. Closes #1709 * fix: resolve CI failures in auth mode PR - Add useLicense mock to SSOSection test to prevent crash from AuthenticationModePanel rendering without LicenseProvider - Remove username from authMode console.log calls that CodeQL flags as clear-text logging of sensitive information * fix(auth): keep SSO-only on named disableSso and fail-closed login Named provider disable no longer reverts authentication_mode. Login initializes localLoginEnabled false so a status fetch failure cannot reveal the password form. Center a single OIDC provider button on the login card. * fix(auth): move SSO-only authentication mode from Admiral to Community tier Security-hardening features belong on the Community tier per the existing Community rebalance. The reporter of #1709 noted that disabling local password login after configuring SSO is a basic security measure, not an enterprise governance feature. LDAP provider configuration remains Admiral-gated via requireTierForSsoProvider. * fix(ui): keep SSO Active badge and ON toggle in sync Provider cards mounted before config fetch finished with enabled:false, so a saved Active provider showed OFF until the local draft was resynced. Drive both the badge and TogglePill from the synced local config. * feat(auth): auto-redirect to sole OIDC provider under SSO-only When authentication mode is SSO only and exactly one OIDC provider is enabled (no LDAP), skip the login chooser and send the browser to that provider's authorize URL. Returning sso_error stays on the login page so the failure message remains visible. * fix(ui): move oidcAutoRedirectUrl out of Login for fast refresh Exporting the helper alongside the Login component tripped react-refresh/only-export-components and failed Frontend lint CI. Keep Login as a component-only module and colocate the helper with its unit tests under lib/.	2026-07-28 10:01:05 -04:00
Anso	79914fe750	fix: recognize clean one-shot completions in health gate and drift (#1691) ... * fix: recognize clean one-shot completions in health gate and drift Treat exit 0 with restart policy no/absent as successful completion so init and migration jobs no longer fail post-update observation or show as service-missing, while long-running restart policies still fail closed. * fix: ignore residual health on clean one-shots and honor deploy.restart_policy Completed exit-0 jobs with no-restart intent no longer fail the health gate on leftover starting/unhealthy state, and Drift treats deploy.restart_policy with Compose precedence so any/on-failure services are not mistaken for one-shots. * fix: require explicit Compose restart no for one-shot recognition Docker inspect reports restart no for both intentional jobs and bare services that omit restart, so Health Gate and Drift now require declared restart:""no"" (or deploy.restart_policy condition none) and load Compose intent once per gate.	2026-07-24 09:41:21 -04:00
Anso	972f2b9483	feat: stack glob patterns and route severity levels (#1651) ... * feat: add stack glob patterns and route severity levels Operators can filter notification routes and mute rules with anchored * globs, and routes can target info, warning, or error. Matching is fail-closed for unsafe stored patterns; write paths keep partial-PUT semantics and ReDoS caps. * test: split mute and routing chip tests to avoid dialog race * fix: move stack pattern client validator out of PatternChips * fix: bound stack glob matching and atomic pattern chip saves	2026-07-19 14:54:48 -04:00
Anso	8980910153	feat: node-scoped Networking operator page (#1603) ... * feat: add node-scoped Networking operator page Adds a Networking view with overview, topology, inventory, and findings. Shared aggregate reads back the page; Resources keeps prune and redirects here. Includes fail-closed network delete guards, operator docs, and /nodes/:slug/networking routing. * fix: rename unused variable n to _n to satisfy no-unused-vars lint * fix: keep top bar search clickable when nav grows * feat: complete Compose-first Networking Phase 2 operator assistant * fix: move networking action visibility helper out of component module * feat(networking): complete Compose-first Networking operator page Finish the node-scoped Networking page (Overview, Networks, Topology, Findings) with design-system parity and correct finding semantics. - Rebuild detail sheets on SystemSheet/SheetSection; align the tab band, masthead, and mobile tone with Fleet and Security. - Encode the host-mode and exposure severity matrix; fix collision counts so intentional shared externals are not flagged; add one typed drift predicate shared by inventory, topology, badges, and overview counts. - Preserve per-container attachments and IPs on topology node clicks; drawer-only click with an explicit logs action; ownership and boolean filters; bound large graphs before layout. - Aggregate cached Compose Doctor findings into the Findings tab with honest source labels, structural merge and dedupe, staleness reconciliation, and a shared exposure-context helper both engines use. - Networks tab: privacy-safe service search, precise ownership counts, schema v3 with version-2 adapters on every endpoint, pre-confirm delete reasons, and the shared sortable table with an internal scroll region. - Interop: Fleet node-card networking signal with pending-intent navigation, stack-to-node backlink, and Dossier/Drift deep links. - Enrich sanitized inspect with an allowlisted connected-container list; fetch topology once and filter client-side. - Docs and tests across every new finding kind, adapter, and flow. * fix(networking): correct drift count, exposure fail-soft, and inspect crash paths Address code-review findings on the Networking page implementation: - Fix the Overview drift count to use the shared drift-kind predicate instead of a hardcoded list that omitted external-network-missing. - Gate Compose Doctor's unclassified-exposure and reverse-proxy-undocumented rules on exposure-context availability, so a DB read failure no longer fabricates findings (mirrors the live engine's existing fail-soft behavior). - Guard the per-stack exposure-intent read in topology aggregation so a transient DB failure degrades to unknown intent instead of failing the whole response. - Harden the network detail drawer against a partial inspect payload from an older remote node, and log the real error instead of a bare catch. - Remove now-duplicated severity-rank and drift-kind helpers in favor of the shared modules; drop dead backend-only exports; widen the frontend schema version type to a plain number instead of casting past a literal type. - Add coverage for the delete-guard precedence, the full host-mode severity matrix, the schema-2 compatibility adapter, and the sanitized connected- container allowlist; tighten two tests that were not exercising the behavior they claimed to. * fix: add missing onOpenNodeNetworking prop to FleetView experimental test The added required prop on FleetViewProps broke the merge-build when the test file (on main but not on this branch) was compiled against the updated FleetView interface.	2026-07-14 20:18:10 -04:00
Anso	b70a529656	feat: add developer-mode startup and stack hydration timing (#1619) ... * feat: add developer-mode startup and stack hydration timing Instrument boot-to-list and detail hydration with commit-aligned milestones, truthful request stages, and destination/gateway debug duration logs so performance work is guided by measurements. * fix: redact stack names and complete hydration request stages Stop logging stack identifiers in containers debug timing, and record state_dispatch (plus detail fetch spans) so copied reports match the advertised stage breakdown.	2026-07-14 17:24:25 -04:00
Anso	381ed2a91f	feat: add Admiral Hardened Build channel and business assurance surfaces (#1629) ... * feat: add Admiral Hardened Build channel and business assurance surfaces Introduce Studio Saelix entitlement-backed Hardened Build switching, a single-flight image operation coordinator, Recovery Vault naming, Admiral Account settings, and typed Fleet update failures while preserving Community custom-repo and targetless pull-current updates. * fix: harden image-op paths and clear CI CodeQL/pilot flake Validate operation IDs before filesystem use, use hostname checks in Fleet fetch mocks, sanitize registry probe logs, and swallow expected TCP teardown errors in the pilot reverse-route post-handshake test. * fix: sanitize image-op docker config write and probe logs Allowlist-copy registry host keys and base64 auth before writing the temp DOCKER_CONFIG, and log registry probe failures with a fixed message so CodeQL no longer flags network-to-file and log-injection mediums. * fix: address Admiral Hardened Build audit blockers Expose imageChannel so hardened Fleet peers still POST for typed rejection, claim community updates before 202, terminalize helper failures, gate Hardened on paid, and align support/docs/e2e wording. * fix: terminalize image ops on helper survival and aborted claims * fix: prevent recreating persist from overwriting helper-exit failure * test: assert helper-exit failure lands before recreating persist * fix: keep current pointer when acknowledging a stale image operation	2026-07-14 10:47:54 -04:00
Anso	213d3d5d3a	feat(resources): bind prune to fingerprinted itemized plans (#1611) ... * feat(resources): bind prune to fingerprinted itemized plans * fix(resources): repair prune plan volume usage and preview list Source volume RefCount from docker df, keep preview rows from flex-shrinking, tighten managed image attribution and becomesFree, and stop audit summaries from claiming success on rejected prunes.	2026-07-11 00:11:37 -04:00
Anso	d113004359	feat: add confirmed Take down stack action with optional volume removal (#1599) ... * feat: add confirmed Take down stack action with optional volume removal Expose Take down in the stack header and sidebar with a confirmation dialog that runs compose down while keeping the stack definition on disk. Optional volume removal is gated by node capability and stack:deploy permission, with remote gateway preflight before proxying removeVolumes requests. Closes #1582 * fix: reset take-down volume checkbox when dialog closes * test: align getStackMenuVisibility assertions with showTakeDown key getStackMenuVisibility now returns a fifth lifecycle flag, showTakeDown, but three exhaustive toEqual assertions still listed only the prior four keys and failed. Add the expected showTakeDown value to each: true for the partial and exited running-stack cases, false for the self stack. * test: cover Take down visibility for running non-self stacks The getStackMenuVisibility assertions exercised the partial and exited branches and the self-stack guard, but not the raw === 'running' literal that drives showTakeDown for a normal running stack. Add a case so a regression dropping 'running' from that check is caught. * fix: drop Take down from header overflow and wire activity shortcut Remove duplicate Take down from More actions. Keep inline button when running, sidebar menu, and Cmd+ArrowDown. Record stack_taken_down in activity on successful POST /down.	2026-07-09 12:20:13 -04:00
Anso	e12602091a	fix: repin semver compose tags during fleet self-update (#1587) ... * fix: repin semver compose tags during fleet self-update Fleet updates failed when docker-compose.yml pinned a semver tag because recreate reused the on-disk pin. Pull the target image first, rewrite semver pins via the update helper, and block digest or unresolved pins with fast 409s. * fix: update OFFLINE_META shape in capability and node-registry meta tests	2026-07-07 14:40:50 -04:00
Anso	4077546492	feat: acknowledge Compose Doctor preflight findings per stack (#1560) ... * feat: acknowledge Compose Doctor preflight findings per stack Add node-scoped preflight acknowledgements with read-time filtering. Supports four expiry modes and activeStatus for banner, tab dot, and readiness. * fix: align preflight acknowledge UI with design system Use Combobox, modal chrome, mono fields, and non-destructive clear confirm. * fix: update test mocks to match new preflight field names The preflight-acknowledgements feature renamed status-\>activeStatus and highestSeverity-\>activeHighestSeverity in the preflight report shape. The corresponding test mocks in three files still used the old field names, causing 6 test failures across backend and frontend. - backend: update-guard-service mock now passes activeStatus - frontend PreflightPanel: Report interface and report() helper now include activeStatus, activeHighestSeverity, activeCount, acknowledgedCount - frontend StackAnatomyPanel doctor: mock API response now includes activeHighestSeverity and activeStatus	2026-07-05 04:12:03 -04:00
Anso	10fb93dcb1	feat(scheduler): schedule container restart, stop, and start (#1526) ... * feat(scheduler): schedule container restart, stop, and start Add container as a scheduled-task target type so operators can automate lifecycle actions against standalone containers by node and name, with matching UI pickers, validation, execution on local and remote nodes, and tests. * fix(scheduler): stack service matching and container picker hygiene Backfill Service on smartFallback containers so per-service stack restarts work when container_name is set. Match services by compose label and container name in stack routes and scheduled restarts. Exclude Sencho from GET /api/containers lists. Hide the Restart Stack service picker when a stack has only one service. * test(scheduler): scope service checkbox assertion to Services block The create dialog also has a Delete after run checkbox. Count checkboxes only inside the Services section so CI does not include unrelated form controls. * fix(scheduler): narrow closest() result to HTMLElement in schedule test The service-checkbox assertion passed an Element from closest() into within(), which requires an HTMLElement, failing tsc -b in the frontend build and Docker build stages. Use the closest<HTMLElement>() type argument so the value type-checks without an unsafe cast. * fix(scheduler): hide Sencho container on remote node picker lists Remote container lists are proxied from peer Sencho instances, so id-only self filtering missed peers on older builds. Await SelfIdentity init, match ImageID, and drop official saelix/sencho images. Apply the same heuristic in the scheduled-operations UI and when the hub fetches remote containers for scheduled runs. * test(monitor): add missing DatabaseService mocks for scan history cleanup * test(scheduler): add missing markStaleScansAsFailed mock SchedulerService.tick() calls db.markStaleScansAsFailed() to sweep stale vulnerability scans. The scheduler-service test was missing this method in its DatabaseService mock, causing TypeError failures during test initialization. Added mockMarkStaleScansAsFailed to hoisted mocks and DatabaseService mock object, returning safe default of 0 scans marked as failed. * test(compose): add missing FileSystemService mocks for getStackContent/getEnvContent * test(containers-route): mock SelfIdentityService to prevent initialize() crash The excludeSelfContainers() helper calls SelfIdentityService.initialize(), which tries to access DockerController. Without a proper SelfIdentityService mock, the initialize() call fails silently, causing a 500 error on GET /api/containers. Added SelfIdentityService mock with initialize(), isOwnContainer(), and isOwnImage() methods to prevent the crash.	2026-07-02 22:31:29 -04:00
Anso	b65daf6845	feat: add notification suppression rules (#1525) ... * feat: add notification suppression rules * fix: restore label routing and routing test mocks for suppression * fix: allow bell mute shortcuts for history-only notification categories Suppression rule validation used the routable category whitelist, which rejected history-only categories such as update_started that appear in the bell during stack updates. * feat: expand Mute Rules UX with compose-first entry points and activity badges * fix: add missing NodeContext mocks for notification suppression tests	2026-07-02 15:26:48 -04:00
Anso	624b586887	chore: gate self-update prompts on published registry images (#1519) ... GitHub Releases appear before docker-publish.yml finishes pushing images. Probe Docker Hub and GHCR manifests before advertising a version as available. Sanitize registry probe debug logs for CodeQL log-injection.	2026-06-29 16:29:56 -04:00
Anso	41dc339c26	fix: enforce 1:1 compose path mapping for Pilot agent mounts (#1516) ... * fix: enforce 1:1 compose path mapping for Pilot agent mounts Pilot enrollment now generates validated 1:1 bind mounts so every agent path maps to a unique compose directory. Persisted agent paths reconcile during startup to catch drift. Unsafe relative-bind redeploys are blocked before container removal to prevent path escapes. - Add composePathMapping utility with strict path validation - Generate COMPOSE_DIR and validated mounts during Pilot enrollment - Reconcile persisted agent paths during startup bootstrap - Block redeploy when a relative-bind mount would escape the compose root - Default Pilot UI path to /opt/docker/sencho - Update multi-node and pilot-agent documentation - Add regression tests for enrollment, bootstrap, compose-service, and environment-check paths * fix: update E2E enrollment regexes for YAML-quoted token values	2026-06-29 14:35:50 -04:00
Anso	ca496c89dc	fix: name matched risk inputs in policy block messages (#1471) ... The auto-update, bulk-label, scheduler, and blueprint deploy block messages hardcoded "image(s) exceed <max_severity>", which is wrong under the risk-first policy model: a block can be driven by a known-exploited (KEV) or fixable Critical/High input while the severity threshold was never the trigger. In those cases the message named a severity ceiling the policy did not enforce. Route all four message paths through a shared summarizeBlockReasons helper (the same reason text the deploy-gate 409 response and the block dialog already use), so every surface names the inputs that actually matched. Falls back to a generic phrase when no reason was recorded.	2026-06-26 15:34:24 -04:00
Anso	315e8b6379	feat: add node update alerts with changelog tab and skip-version handling (#1463) ... * feat: add node update alerts with changelog tab and skip-version handling - Add node_update_available notification category with blue/brand bell dot - Route node_update_available notifications to Fleet -> Node updates sheet - Add Changelog tab to NodeUpdatesSheet with GitHub release notes - Add per-node skip-version persistence (node_update_skips table) - Skip hides update CTA on node card and sheet; re-surfaces on newer version - Skipped nodes excluded from Update all backend filter - Add pulsating dot indicator on Changelog tab when updates available - Always-visible View changelog action in notification row bottom - Admin-only for all mutating controls (skip, unskip, update) - Backend tests for skip-version semantics (15 tests) - Update fleet-view.mdx, remote-updates.mdx, and OpenAPI spec * fix: address audit findings - nested button, stale changelog, semver normalization, mobile intent - Move View changelog button outside routable button (sibling element) - Fix aria-label for node_update_available notification rows - Support ?recheck=true on release-notes endpoint - Invalidate release notes cache on forced recheck - Store normalized semver (semver.valid strips v prefix) - Skip fleetUpdatesIntent on mobile (desktop only) - Add v-prefix normalization test * fix: restore View changelog on same line as timestamp, opposite sides The button is always visible at the bottom right of the notification card, on the same row as the timestamp (just now), using justify-between layout. * fix: update tests for node_update_available category and release-notes fetch - Backend: monitor-service tests now expect node_update_available instead of system - Frontend: NodeUpdatesSheet tests mock release-notes API call to prevent undefined then() * fix: resolve ci lint failures	2026-06-26 00:07:51 -04:00
Anso	a698aaa926	feat: add per-stack project env file selection for Docker Compose (#1457) ... * feat: add per-stack project env file selection for Docker Compose Allow users to configure an ordered list of env files per stack that serve as the project environment file(s) for Docker Compose ${VAR} interpolation. The selected files are passed via repeated --env-file flags during all compose commands. Backend: - Add stack_project_env_files table (node-scoped, ordered) - Extend authoredComposeEnvFileArgs to emit --env-file for configured files - Add GET/PUT /stacks/:name/project-env-files and /candidates endpoints - Update resolveStackEnvSources to use configured files as interpolation source - Update resolveAllEnvFilePaths to merge injection + interpolation sources - Add discoverStackLocalEnvFiles for candidate discovery - Extend backupStackFiles and snapshotStackFiles for project env files - Add project-env-files capability to CapabilityRegistry Frontend: - Add project env file selector to EnvironmentPanel (capability-gated) - Update EditorView banner to generic "project environment file" language - Add project-env-files capability to capabilities.ts Issue: #1454 * fix: add realpath validation, clear all stale backup files, reject nested paths - authoredComposeEnvFileArgs: use fsPromises.realpath + isPathWithinBase for symlink escape defense at use time - backupStackFiles: clear ALL non-marker files from backup slot before writing, not just PROTECTED_STACK_FILES (handles stale old.env) - PUT project-env-files: reject paths containing / or \ (root-level only, matching Compose auto-discovery behavior) * fix: add getStackProjectEnvFiles to compose-service mock The new authoredComposeEnvFileArgs calls getStackProjectEnvFiles on the DatabaseService singleton. The compose-service mesh-override tests mock that singleton without the new method, causing 6 failures. Add getStackProjectEnvFiles: () => [] (empty = fall back to legacy behavior, which is what these tests exercise). * fix: add getStackProjectEnvFiles to remaining service mocks The new authoredComposeEnvFileArgs calls getStackProjectEnvFiles, which is missing from the mock in compose-images.test.ts (6 failures) and image-update-service.test.ts (proactive fix). * fix: apply inline path-injection barrier at fs sink for CodeQL The PUT project-env-files route resolved paths via isPathWithinBase before calling fsp.stat, but CodeQL does not credit a containment check separated from the sink. Apply the canonical inline barrier pattern (path.resolve + startsWith at the sink) used throughout the codebase. * fix: resolve stackDir from the same canonical root as safePath Prevents a containment bypass when the compose base directory is a symlink: stackDir was previously joined from the unresolved baseDir while the inline barrier used path.resolve(baseDir), which could differ for symlinked paths. Now both stackDir and safePath are resolved from a single canonical root, then each is containment-checked against it. * fix: remove unused isPathWithinBase import The inline path-injection barrier refactor replaced isPathWithinBase with an inline startsWith check at the fs sink, so the import is now unused and fails ESLint no-unused-vars.	2026-06-25 18:03:05 -04:00
Anso	6527bc971b	feat(security): gate deploys on exploitation risk, not just severity (#1432) ... Scan-policy deploy gates can now block on a known-exploited CVE (CISA KEV) and on a fixable Critical/High finding, in addition to an optional severity threshold. New policies default risk-first (KEV and fixable on, severity off); existing policies keep their severity-only behavior. CVSS stays captured for context but is never the sole basis for a block, and a finding whose exploitability cannot be confirmed is treated as risky rather than safe (incomplete scan detail fails closed on KEV/fixable inputs). The decision logic is shared between the pre-deploy gate and the informational post-scan banner via a pure helper, so the two never disagree. Block messages and the block dialog now name the conditions an image matched. Backend and frontend gates move together, the new inputs replicate across the fleet, and a blocking policy with no active input is rejected on both sides.	2026-06-24 20:05:17 -04:00
Anso	96b3c49359	fix(deploy): verify atomic-deploy backup integrity before restore (#1422) ... * fix(deploy): verify atomic-deploy backup integrity before restore Atomic deploy and the Rollback action restore a stack from a backup of its compose file and .env. A backup truncated or corrupted at write time (out of disk, interrupted copy) was copied back silently, overwriting a working stack with bad content. The backup now writes a .checksums manifest holding a SHA-256 of each backed-up file, and a restore re-hashes every file and compares it before touching the stack. A mismatch aborts the restore with a clear error and leaves the live files unchanged. Backups without a manifest, and files with no recorded checksum, are restored unverified so a rollback is never blocked by missing integrity data. * fix(deploy): guard backup source reads with an inline path barrier The integrity change reads each managed file from the stack directory before hashing it. Static analysis flags those reads because the source path derives from the user-provided stack name and the containment check lived in a helper it does not trace. Re-establish containment inline at each read sink, resolving against the compose base and confirming the path stays within it, mirroring snapshotStackFiles. Behavior is unchanged for valid stacks; the guard only rejects a path that escapes the compose directory, which validation already prevents. * test(deploy): assert compose stays put when the backup .env is corrupt Strengthen the .env-corruption test so it also mutates the live compose.yaml and asserts it is left untouched, proving the integrity abort halts before any file is copied back rather than relying on the backup happening to match. Also note on the test hash oracle that it matches production for UTF-8 text fixtures.	2026-06-24 19:46:59 -04:00
Anso	f794702171	feat(security): action-posture Security dashboard with exploit intel and triage (#1424) ... * feat(security): reframe masthead as action posture, not worst-CVE severity Derive the Security masthead from an action posture (Action needed / Monitoring / Secure / Unknown) instead of raw scanner severity, and label the raw Critical/High counts as scanner detections. "Secure" now means nothing is actionable right now, never a claim that no vulnerabilities exist; Unknown covers a missing scanner or a node with no completed scan. Phase-1 bootstrap: "actionable" is approximated from the overview facts that already exist (fixable findings, secrets, misconfigs); a later phase moves the bucketing to the backend. * feat(security): derive overview action posture from triaged facts Add deriveSecurityPosture as the single bucketing function and extend /security/overview with posture facts (fixableCriticalHigh, dangerousCompose, accepted, rawCritical/rawHigh, plus knownExploited/publiclyExposed placeholders that later phases populate) and the derived posture verb. Suppression- and acknowledgement-aware counts come from one bounded read-time pass over the latest-scan Critical/High findings, grouped per image so the existing read-time filters apply unchanged. The pass is capped and flags posturePartial, so a large node degrades gracefully instead of scanning every detail row. The masthead now prefers the backend posture and keeps the local bootstrap only as a fallback for older remote nodes reached through the proxy. * feat(security): capture Trivy finding enrichment (status, CVSS, vendor, purl, layer) parseTrivyOutput now keeps the per-finding fields Trivy already returns and we previously discarded: Status (fixed / will_not_fix / end_of_life / ...), CVSS (score + vector, preferring the NVD source then falling back), vendor severity, package URL, package path, and layer digest. Persisted on vulnerability_details via additive nullable columns (guarded ALTER), bound null when absent, and carried through the cached-scan reconstruction path. These fields separate scary from exploitable and feed the action posture and the per-finding evidence tags. Field paths verified against Trivy's documented image-scan JSON; covered by parse and insert/read round-trip tests. * feat(security): add CVE exploit-intel service (CISA KEV + FIRST EPSS) Add CveIntelService, a daily background cache of CISA KEV membership and FIRST EPSS scores stored in a new cve_intel table and joined to findings at read time by CVE id (never frozen onto scan rows, so a CVE entering KEV later lights up on scans already stored). EPSS is fetched only for CVE ids present in stored findings, batched; both feeds are best-effort and keep the last cache on failure, so the Security page degrades gracefully offline. Wired into startup/shutdown like the other background services. The overview now counts known-exploited Critical/High findings, and KEV membership escalates posture to Action needed even when no fix is available. A per-instance "Exploit intelligence" toggle on the scanner setup surface lets air-gapped or firewalled hosts disable the outbound fetch; the daily tick keeps running but skips the fetch body when it is off. * feat(security): show per-finding evidence tags (KEV, EPSS, vendor status, CVSS) The vulnerabilities endpoint joins read-time exploit intel (KEV membership and EPSS score) onto each finding by CVE id, and the scan sheet renders evidence tags beside each CVE: known-exploited, EPSS probability, vendor will-not-fix / end-of-life, and the CVSS score. Severity becomes one signal among several so an operator can tell scary from exploitable, with no invented composite score. * feat(security): evolve CVE suppressions into triage decisions Layer a triage status and optional OpenVEX justification onto CVE suppressions. Statuses: needs review / affected / not affected / accepted risk / fixed / false positive / ignored. Dismissing states (not affected, accepted, fixed, false positive, ignored) stop a finding from driving the action posture; needs review and affected stay actionable and are surfaced as counts. Existing rows default to "accepted" (the prior suppress behavior), so nothing changes for them. The overview now reports needsReview / notAffected / accepted as distinct facts derived from the triage status. The decision replicates across the fleet (snapshot + replicated-insert carry status + justification) so a replica's posture matches the control node. The inline suppress dialog gains a triage decision selector; the read-time filter surfaces the status and justification on every finding. * feat(security): export fleet triage decisions as OpenVEX (Admiral) Add an OpenVEX exporter that turns the instance's CVE triage decisions into a standard VEX document (not_affected / fixed / affected / under_investigation, with justifications), and a GET /security/vex/export endpoint to download it. Authoring fleet VEX is a governance capability, so it is gated to Admiral (paid) plus admin, mirroring the SARIF export gate; the Suppressions panel shows an Export VEX action only on Admiral. * docs(security): document action posture, evidence tags, exploit intel, and triage Update the Security page and CVE suppressions docs for the action-posture masthead (scanner detections vs product posture), per-finding evidence tags (KEV / EPSS / CVSS / vendor status), the exploit-intelligence toggle (CISA KEV + FIRST EPSS) on scanner setup, triage decisions layered on suppressions, and OpenVEX export of fleet triage decisions. * test(security): match intel hosts exactly in CveIntelService test Route the fetch stub and its call assertions by exact hostname (www.cisa.gov / api.first.org) instead of a domain substring check. Resolves the js/incomplete-url-substring-sanitization code-scanning alerts on the test's URL routing; behavior is unchanged.	2026-06-23 17:42:11 -04:00
Anso	37e6e48b40	feat(files): copy & duplicate, bulk actions, disk-backed uploads, and an accessible file tree (#1409) ... * perf(files): spool uploads to disk instead of buffering in memory Switch the stack file-explorer upload from multer memoryStorage to diskStorage and stream the spooled temp file through the file-root gateway, so an upload is never held fully in RAM. Authorization and root resolution now run before multer spools, so an unauthorized or read-only-root request is rejected without writing a temp file, and the spool is removed on every exit path. The named-volume helper write verifies the written byte count, since cat cannot report a short write. * feat(files): copy and duplicate files in the explorer Add a copy capability to the stack file explorer: a same-folder Duplicate (auto-suffixed name) and a "Copy to..." destination picker, on both filesystem and named-volume roots. Copying is within-root, symlink-leaf-safe, blocks a directory copy into its own subtree, and refuses to create a protected name (compose/.env) at the stack root while still allowing a protected file to be duplicated under a new name. * feat(files): make the file tree keyboard accessible Bring the stack file explorer tree to the WCAG tree pattern: rows are treeitems carrying aria-level, aria-selected, and aria-expanded, with a single roving tabindex and full keyboard navigation (arrow keys, Home/End, Enter/Space) over a flattened visible-node list that stays in lockstep with the rendered rows. A polite live region announces the selected file. No visual change to the tree. * feat(files): bulk select, delete, move, and download files Add multi-select to the stack file explorer (checkboxes plus Shift and Ctrl/Cmd click over the visible order) driving three bulk actions: delete, move, and download as a streamed .tar.gz. All run within the active root on both filesystem and named-volume backends, report per-item results so partial failures surface (with the failed items kept selected for retry), normalize ancestor/descendant selections server-side, and cap the archive entry and byte counts before any bytes are streamed. Protected compose/.env files are excluded from bulk delete and move but may still be downloaded. * docs(files): document copy, bulk actions, and keyboard navigation Add the copy/duplicate and multi-select bulk delete/move/download sections to the Files & Volumes page, a keyboard-navigation note for the tree, an updated context-menu reference, and bulk troubleshooting entries. * fix(files): inline path-injection barriers at the new file-op sinks CodeQL js/path-injection does not credit the wrapped isPathWithinBase containment check, so the new copy/bulk/disk-upload flows tripped the gate. Inline the canonical path.resolve + startsWith barrier at the realpath sink in resolveSafePathWithin (covers every user-relPath flow) and confirm the multer spool path resolves within UPLOAD_TMP_DIR before unlinking it or streaming it onward. Behavior is unchanged; the paths were already validated. * fix(files): guard the ancestor-walk realpath sink too The first barrier covered realpath(target), but the ENOENT ancestor walk re-derives the path via path.dirname, which static analysis treats as a fresh tainted value. Add the same inline containment barrier before that realpath and resolve the root case via the untainted base, so the only tainted realpath input is one the startsWith check has cleared. Behavior is unchanged. * fix(files): resolve the root case off the taint path in the ancestor walk The compound guard on existing (the same variable as the startsWith subject) was not credited as a sanitizer. Handle the root case before the barrier by resolving the untainted base directly, leaving a plain canonical startsWith guard on the strictly-within ancestor. Behavior is unchanged. * fix(files): harden helper-backend bulk download and uploads Address three issues found in the named-volume (helper) backend: - Bulk download could send 200 headers before discovering a file the helper download path refuses, tearing the archive mid-stream. The prewalk now rejects symlinks, non-regular ("other") entries, and files over the per-file download cap before any header (400/413). FileEntry gains an 'other' type so non-regular entries stay distinct from regular files as they pass through the gateway. - The helper directory listing was fully buffered before the archive entry cap could fire. listDir now accepts a limit; the list script stops after limit+1 rows and the gateway reports truncation. - A stdin pipeline error during a helper upload masked the container's real nonzero exit code (and its 4xx mapping) as a generic 500. The nonzero exit now wins; the masked stream error is logged. * feat(files): add a New file toolbar button with server-enforced create-only The stack file explorer could create a folder from a toolbar button but a new file only from a folder's right-click menu, so a file could not be created at the stack root at all. Add a New file toolbar button beside New folder, targeting the current directory. Creating a file now routes through a new createEmptyStackFile helper that posts a zero-byte file through the existing upload endpoint with overwrite off, so the server's exclusive-create path rejects an existing name instead of clobbering it. A file collision surfaces inline in the dialog; a folder collision and other failures surface as a toast. * fix(files): widen the tree row hit area and add horizontal scroll for long names Right-clicking a file tree row only opened the Sencho context menu when the click landed on the filename; the rest of the row fell through to the native browser menu, and long names were truncated with no way to read them. Make each row span the full pane width (and grow with its content) so the whole row is the context-menu trigger, and let the tree scroll horizontally so a long name is reachable instead of clipped. A new opt-in horizontal prop on ScrollArea adds the styled horizontal scrollbar without clamping content width. * docs(files): document the New file button, full-row right-click, and long-name scrolling * test(files): cover createEmptyStackFile targeting the stack root Add an API-layer case for the empty-directory (stack root) create path, the primary reason the New file toolbar button exists, so a regression in the root-level URL would be caught at unit speed rather than only in e2e.	2026-06-22 19:33:06 -04:00
Anso	ba09e6f69e	fix: base Git multi-file Compose deploy env and dossier on the effective config (#1391) ... * fix: resolve the root .env at deploy and render time for Git context-dir stacks A Git multi-file source with a context dir set --project-directory to that dir, so Docker Compose looked for .env there and missed the root .env Sencho writes. Validation already passed the root .env with --env-file, so a stack could validate with one effective config but deploy or render another. Add authoredComposeEnvFileArgs, which appends --env-file <stackDir>/.env when the applied deploy spec has a context dir and a root .env exists, and wire it into the deploy/update, image-scan, render, and container-listing compose invocations so they all resolve env from the same file the validator used. A non-ENOENT access error surfaces instead of silently dropping the flag. * fix: base multi-file Git dossier and doc-drift on the effective Compose model The Stack Dossier and its documentation-drift check parsed only the stored root compose file. For a multi-file Git source, services, ports, networks, or volumes that an override file adds were invisible, so the dossier showed incomplete facts and doc-drift could falsely warn that a documented port is unpublished when an override actually publishes it. Add a secret-safe GET /stacks/:name/effective-anatomy that renders the merged effective model and extracts only structural facts (services, ports, volumes, networks, restart), never env, label, or command values. StackAnatomyPanel fetches it for multi-file Git stacks and feeds those facts into the dossier and doc-drift, falling back to the root-only parse for single-file or non-git stacks and whenever the render is unavailable. * fix: add an inline path-injection barrier to the Git env-file resolver CodeQL js/path-injection flagged the fs.access in authoredComposeEnvFileArgs because the env path derives from the route-supplied stack name and the only containment check lived in the callers, not at the sink. Resolve the stack dir against the compose base and assert containment with startsWith inline, then derive the .env path from the validated dir, mirroring the existing inline guards in renderConfig and validateCompose. Valid stack names are unaffected; a name that escapes the base now yields no --env-file. * test: stabilize the dossier doc-drift e2e against the dossier-load race The first assertion filled the access_urls field as soon as the Dossier panel was visible, but the panel's GET /stacks/:name/dossier resolves by overwriting the fields from the server (empty access_urls) and only then flips the doc-drift gate on. When the GET landed after the fill, it clobbered the typed value and the warning never rendered, so the test failed intermittently under CI timing. Wait for that GET to land before typing, mirroring the spec's openStack helper.	2026-06-18 18:25:58 -04:00
Anso	f23b7e1bac	feat: ordered multi-file Compose for Git sources (#1380) ... * feat: ordered multi-file Compose for Git sources Extend Git sources to deploy an ordered list of compose files merged with docker compose -f base.yaml -f override.yaml ..., plus an optional project directory. - Pick and reorder compose files from the repository tree (drag to reorder on desktop, up/down arrows on phones); manual path entry is also supported. - The ordered set drives every stack-scoped compose command (deploy, update, start/stop/restart/down, image scans, Compose Doctor) and the container lookup, so a service or image declared only in an override is handled too. - Runtime keys off the materialized set, not the saved configuration: saving a source does not change deploy args until the pull is applied, and apply materializes from the pending snapshot rather than live config. - The project directory is passed as --project-directory, with -p <stack> pinning the Compose project so container labels stay stable. - The Mesh override is layered last; single-file sources are byte-identical to before, and existing rows keep working via the single-path fallback. Docs cover the picker, ordering, project directory, and the new troubleshooting and limitations (referenced files are not materialized; the dependency graph, drift, and networking views read the primary file). * fix: harden multi-file Git source (hash, unlink, collisions, node id) - hashContent folds ordered file CONTENTS (not paths) so a clean multi-file stack is not flagged as locally edited: create/apply hash the fetched files (repo paths) while pull hashes the on-disk files (materialized paths), which previously disagreed and showed a false "local edits detected". - Block unlinking a multi-file or project-directory Git source (409): the deploy spec lives on the source row, so removing it would silently revert deploys to root compose.yaml. Single-file sources still unlink. - Reject materialized-path collisions in the selection validator: an additional file equal to or nested under compose.yaml, an ancestor/descendant overlap between selected files, and a project directory nested under a compose file (previously a 500 at materialization). - DockerController.getContainersByStack uses the controller's node compose dir and passes its node id to the authored prefix, instead of the process default. * fix: CI failures on multi-file Git source (test crash, aria query, path barrier) - GitSourceFields no longer crashes when repoUrl/branch are falsy: the canBrowse trim() is optional-chained, so a reusable field component tolerates partial props. Fixes the apply-binding panel test, which feeds a minimal source object. - GitSourcePanel tests query the footer Remove button by its exact name, so the picker's per-file "Remove <path>" buttons no longer collide with the broad /remove/i match (the test intent, footer Remove present/absent, is unchanged). - validateCompose uses an inline resolve + startsWith barrier at the context-dir mkdir sink (CodeQL does not credit the wrapped isPathWithinBase helper), clearing the js/path-injection alert. The containment check is equivalent and contextDir is also validated upstream. * test: update Git source E2E spec for the multi-file compose picker The compose-file picker replaced the single #git-source-path input and added per-file Remove buttons, so the E2E spec drove selectors that no longer exist: - Drop the redundant compose.yaml fills (the picker defaults to compose.yaml). - Select the footer Remove button by exact name so the picker's per-file "Remove <path>" buttons no longer make the locator ambiguous. - Set a custom compose path through the picker (add via the manual input, press Enter, then remove the default compose.yaml). * test: match the footer Remove button with an exact Playwright name Playwright's getByRole name option is a substring match by default, so { name: 'Remove' } also matched the picker's "Remove <path>" buttons. Require an exact match so only the footer Remove button is selected.	2026-06-17 13:24:55 -04:00
Anso	710647a44f	feat(snapshots): preserve stack dossiers with fleet snapshots (#1339) ... * feat(snapshots): preserve stack dossiers with fleet snapshots Fleet snapshots can now optionally capture each stack's Dossier notes alongside its compose and .env files, so a recovery restores the operational knowledge around a stack, not just its configuration. - Opt-in global setting "snapshot_documentation" (default off), toggled from the renamed Fleet settings section. - Capture reads local dossiers from the database and remote dossiers over the Distributed API proxy; only stacks with notes are recorded, and secret values are never included. - Captured notes are stored encrypted at rest in a new fleet_snapshots column and surfaced in the snapshot detail view behind a badge. - Cloud and downloaded archives gain a documentation.json (archive_version 2). - Restore stays conservative: dossier notes are written back only when the operator explicitly opts in, on both single-stack and restore-all paths. - Existing snapshots and archives remain valid; behavior is unchanged when the setting is off. * fix(snapshots): harden dossier-notes restore against bad input and partial failures Address review findings on the documentation-snapshots restore path: - Parse `restoreNotes` strictly (=== true) on single-stack restore, matching restore-all, so a stray non-boolean can never opt in to overwriting notes. - Guard findSnapshotDossier: require an array of stacks and real dossier content, so a malformed or all-blank entry can't clobber current notes. - Make the dossier-notes write non-fatal relative to the file restore: a notes failure (e.g. a remote dossier PUT) is caught, reported via `notesError`, and no longer 500s the single restore or fails the stack in restore-all once the files are already written. - Surface the partial outcome in the UI: a warning toast on single restore, a summary note on restore-all, and gate the "Documentation captured" badge and restore-all notes control on captured stacks while rendering capture warnings. Adds tests for strict parsing, malformed/blank blobs, remote notes restore (success + non-fatal failure, single and bulk), and scheduled capture-on. * fix(snapshots): drop unused binding in restore-all remote notes test The restore-all remote notes test destructured a node id it never uses (restore-all is driven by snapshot id alone), tripping no-unused-vars and failing the lint step. Bind only the snapshot id.	2026-06-08 08:44:59 -04:00
Anso	b21324f97a	feat(stacks): persist a drift ledger with temporal source-change detection (#1333) ... * feat(stacks): persist a drift ledger with temporal source-change detection Build on the read-only compose-vs-runtime drift check so a stack's drift is remembered over time, not just shown at a glance. - Record a deploy baseline: on a successful deploy, update, or rollback, store the deployed compose file's source and rendered-model hashes on the stack so the Drift tab can tell whether the file has changed since the last deploy. - Surface temporal drift in the Drift tab: "matches last deploy", "source changed since last deploy" (distinguishing a model change from a formatting-only edit), or "no deploy baseline yet". - Persist findings into a drift ledger: a re-check reconciles the current findings, recording newly detected ones and resolving cleared ones, and shows a short drift history under the findings. The drift report read stays side-effect-free; only an explicit re-check (and a deploy) writes the ledger. - Write drift detected/resolved events to the stack Activity timeline so the provenance sits alongside deploys and restarts. Node-local and available on the Community tier. Reconciliation is skipped when a check is not authoritative (Docker unreachable or a compose parse error) so an open finding is never falsely cleared. * fix(stacks): record the drift baseline for every deploy path and harden the ledger Address review feedback on the drift ledger: - Record the deploy baseline in ComposeService.deployStack/updateStack instead of only the manual route, so bulk, Git-source, App Store, scheduler, and webhook deploys all capture source/rendered hashes. Reconciliation stays on the explicit re-check. - Store no rendered baseline when the local parser cannot model the compose (for example a file over the parse cap) rather than a sentinel that would make a later real change read as unchanged. - Let temporal-overlay failures surface as a 500 instead of being hidden behind a neutral "no baseline"; only the compose read stays best-effort. - Omit the temporal card entirely when a report (for example from an older remote node) carries no temporal data, instead of showing a misleading "no baseline". - Keep drift_detected / drift_resolved history-only by excluding them from the routable-category whitelist, so they are never offered as a channel route that would never fire. - Use a JSON separator for the finding identity key so the source file is plain text (no embedded control byte). * fix(stacks): sanitize logged errors in the drift report handlers The drift report and re-check handlers logged the caught error object raw alongside the stack name, which a code scan flagged as a log-injection vector: a crafted stack name surfacing inside an error message or stack could forge log lines. Route the error through the log sanitizer so control characters are stripped before writing. Render it with util.inspect first so the stack trace, cause chain, and underlying error codes are preserved for debugging.	2026-06-07 20:44:22 -04:00
Anso	2435da232b	fix(api-tokens): harden rate limiting and surface list-load errors (#1292) ... * fix(api-tokens): scope per-token rate limits to live tokens Forged or token-shaped Authorization headers no longer mint their own rate-limit budget. The key generator now grants a per-token budget only to a real, active token and falls back to per-IP keying for anything else, so a single source cannot evade the global limiter by rotating fake tokens. The validated token is memoized on the request, so authentication reuses it without a second database lookup. Token validation (format, checksum, lookup, revocation, expiry) is now a single shared helper used by the HTTP auth middleware, the WebSocket upgrade handler, and the rate-limit key generator, replacing two near-identical inline copies that could drift apart. The last-used timestamp write is throttled so a busy token no longer writes to the database on every request. * fix(api-tokens): surface token list-load failures with a retry A failed load of the API tokens list was swallowed: a server error rendered the empty "no tokens yet" state with no sign that anything went wrong. The list now shows an error card with a Retry action and raises a toast on any non-ok response or network error, matching the create and revoke flows. Adds a troubleshooting entry for the error. * test(api-tokens): seed tokens via the shared test helper The new hardening and WS-scope suites computed sha256 of a raw token directly, which CodeQL flags as js/insufficient-password-hash (a false positive: these are 256-bit CSPRNG opaque tokens, not passwords). Route token creation through the existing apiTokenTestHelper and read the stored token_hash back from the row, so the suites no longer hash anything themselves. Also removes the duplicated createToken helpers. * fix(api-tokens): key the rate limiter by the same credential auth uses The rate-limit key generator checked the session cookie before the Authorization bearer, while authMiddleware authenticates bearer-over-cookie (bearerToken || cookieToken). A request could send a Bearer API token plus a forged cookie and be keyed by the cookie's (forgeable, rotatable) username, sidestepping the per-token / per-IP keying the limiter applies to API tokens: a valid token would lose its own bucket, and a forged token-shaped bearer would no longer collapse to per-IP. Reorder the generator to mirror auth: process the bearer first (validate the API token and key per-token or fall back to per-IP; otherwise decode the JWT by username/sub), and consult the cookie only when there is no bearer. Regression tests cover a valid and a forged sen_sk_ bearer, each sent with a forged cookie.	2026-06-03 08:18:16 -04:00
Anso	c11a550b6a	fix(fleet-snapshots): gate reads on admin role and encrypt content at rest (#1273) ... * fix(fleet-snapshots): gate reads on admin role and encrypt content at rest Fleet snapshots capture every node's compose.yaml and .env, so the data is as sensitive as the live stacks. This hardens access and reliability across the snapshot pipeline. - Restrict snapshot reads to administrators. GET /api/fleet/snapshots and /:id now require the admin role, matching create, restore, and delete; the Fleet "Snapshots" tab and its panel render only for admins. Previously any authenticated user could enumerate snapshots and read every node's .env. - Encrypt snapshot file contents at rest with the instance key. Restore and cloud-archive paths decrypt on read, so cloud archives stay portable and a database copy no longer exposes stack secrets in plaintext. Rows written before this change still read back as plaintext. - Surface partial captures. A stack whose compose file cannot be read or fetched, or a file over the 1 MB capture cap, is recorded as a warning and shown on the snapshot instead of being silently dropped, so a snapshot is never mistaken for complete. Remote .env read errors are now distinguished from a genuinely absent .env. Adds route-authz, capture-warning, and encryption round-trip tests; updates the Fleet-Wide Backups feature docs. * fix(fleet-snapshots): gate cloud snapshot reads on admin role The cloud snapshot read routes were guarded by provider/license only, not by role, while their write counterparts (upload, delete) already required admin and the Cloud Backup settings surface is admin-only. Because a downloaded archive contains plaintext compose and .env files, a non-admin could list and download cloud snapshots and read every node's secrets, the same exposure the local snapshot reads were just closed against. - Require admin on GET /api/cloud-backup/snapshots, /status/:id, and /object/:keyB64/download, matching the local snapshot reads and the admin-only Cloud Backup settings section. - When capturing a remote node, treat a 200 response carrying X-Env-Exists: false as a stack with no .env (matching the local ENOENT path) instead of storing an empty .env that restore would later write back. Adds non-admin authorization tests for the cloud read routes and a remote absent-.env capture test.	2026-06-01 17:27:59 -04:00
Anso	b61388c675	fix(rbac): enforce admin seat cap on promotion and harden last-admin and audit paths (#1266) ... Promoting a user to admin now respects the per-tier admin seat limit the same way user creation does, closing a path that let an operator exceed the cap by creating an account and then editing its role to admin. The last-admin guard for demote and delete now runs the admin-count re-check and the write in a single transaction, so two concurrent admin changes can no longer race the admin count to zero and lock everyone out. Admin two-factor resets are now recorded once with their own audit summary instead of being mislabeled as a user creation by the audit middleware.	2026-06-01 13:01:22 -04:00
Anso	5e66b54153	fix(audit-log): neutralize CSV export injection, clamp pagination, bound anomaly history (#1259) ... * fix(audit-log): neutralize CSV export injection, clamp pagination, bound anomaly history Harden the Admiral audit log without changing its tier or hub-only gating. - CSV export now defuses formula injection: any field that a spreadsheet would evaluate as a formula (leading = + - @, or a trigger behind leading whitespace, or a leading tab/CR) is prefixed with a single quote before RFC 4180 quoting. Audit summaries embed user-controlled resource names, so this closes a path where a crafted name could execute on export open. - Clamp page and limit to positive bounds on the list endpoint so a negative limit can no longer reach SQLite as "unlimited" and dump the whole table. - Bound the anomaly and stats history reads to a capped slice of recent rows so the analysis paths stay within fixed memory and latency on large histories instead of scanning the full retention window per request. - Surface failed audit list and stats fetches through the standard error toast instead of leaving the view silently stale. - Add a developer-mode-gated diagnostic log to the stats endpoint for parity with the list and export handlers. Covered by new unit and HTTP-integration tests (CSV neutralization through the real export route, pagination clamps, bounded history, stats endpoint, anomaly annotation) and verified end to end in the browser. * fix(audit-log): compute signal-rail stats with exact SQL aggregates Address review feedback on the earlier history-cap change. The cap was correct for the anomaly baseline but made the stats tiles (events, actors, failure rate, hourly series) silently undercount on a hub with more than the cap's worth of rows in the window, since they were derived from the capped row slice. - Add DatabaseService.getAuditStatsInputs: exact counts via SQL COUNT / COUNT(DISTINCT) / GROUP BY hour, and new-ip detection over the small DISTINCT (user, ip) pair sets. No row cap, so the tiles stay exact at any window size while memory stays bounded. - Reduce computeAuditStats to a pure formatter over those aggregates. - Keep the bounded history read only for the list endpoint's anomaly annotation, where a recent-activity baseline is an acceptable heuristic. - Skip the redundant load-failure toast when a fetch fails with a handled 401, so an expired session does not stack toasts on top of logout. Adds exactness tests for the aggregate counts, distinct-actor handling, and new-ip detection, and strengthens the pagination-clamp tests. * fix(audit-log): exclude future-dated rows and make the new-ip sample deterministic Two small parity fixes on the stats aggregates: upper-bound every current window by `now` so a future-dated row (clock skew or a fixture) cannot inflate the live counts, and order the new-ip pair scan so the sample actor shown in the tile detail is stable. Adds a test asserting a future row is excluded.	2026-05-31 16:36:07 -04:00
Anso	69edb0dcbb	fix(observability): gate global logs to admins, scope to managed containers, harden SSE (#1254) ... * fix(observability): gate global logs to admins, scope to managed containers, harden SSE Make the Logs feed an administrator view enforced on both sides (requireAdmin on the /api/logs/global poll and SSE routes; the Logs nav item plus a redirect guard on the frontend), and scope the feed to Sencho-managed containers only via a shared isManagedByComposeDir helper that /stats now reuses. Harden the SSE stream: a stateful frame demuxer that survives chunk boundaries so a Docker frame split across reads is reassembled instead of dropped or garbled; a per-stream error listener so one broken follow stream cannot crash the event loop (it posts a single degraded notice and keeps the others alive); a cap on concurrent follow streams with a truncation notice; a bounded initial tail; and backpressure that pauses the source streams when the client is slow and resumes on drain. Bound the polling snapshot's per-container fan-out with a concurrency limit. Add process-local, in-memory log-stream counters exposed at the admin-only /api/system/log-stream-metrics endpoint (active connections, lines streamed, attach and frame errors). Collapse the view to the local hub and remove the dead remote-node handling. * fix(observability): close remote-proxy bypass of the global-logs admin gate The logs feed's requireAdmin lives in the local route handler, which the remote proxy skips when forwarding a request whose nodeId targets a remote node. A hub user could therefore request /api/logs/global*, /api/logs/global/stream, or /api/system/log-stream-metrics with x-node-id (or ?nodeId= for the SSE transport) pointing at a remote node and have it served as the node-proxy admin on the far side, sidestepping the gate entirely. Add these paths to HUB_ONLY_PREFIXES so hubOnlyGuard rejects a remote nodeId with 403 before the proxy runs, matching the existing protection on audit-log, scheduled-tasks, and notification-routes. Add regression tests covering the collection path, the SSE sub-path (both the x-node-id header and the ?nodeId= query transport), and the stream-metrics endpoint.	2026-05-29 21:09:20 -04:00
Anso	2844f606cd	fix(git-sources): harden webhook delivery, transport errors, and clone limits (#1249) ... * fix(git-sources): harden webhook delivery, transport errors, and clone limits Map webhook-pull outcomes to real HTTP status codes (200 success, 202 debounced, 404 no source, 422 failure) instead of always returning 200, so a Git provider and any monitoring on it can tell when a delivery actually failed. Close a concurrent webhook fan-out gap: the debounce window is now re-checked inside the per-stack lock, so simultaneous deliveries for one push run a single clone instead of one per request. The whole pull/apply critical section runs under a single lock acquisition. Unwrap fetch transport causes (ENOTFOUND, ECONNREFUSED, ECONNRESET, TLS) so a clone failure surfaces an actionable, host-qualified message instead of a bare "fetch failed". Cap how many bytes a single clone may download to protect the host disk; operators can tune it with GITSOURCE_MAX_CLONE_BYTES (default 100 MB). Log webhook pull failures server-side, since the webhook path is unattended. * test(git-sources): assert surfaced host via toContain to satisfy CodeQL * fix(git-sources): bound per-file read, treat debounced webhooks as non-failure, correct clone-cap docs * docs(git-sources): correct clone-cap comment to describe a download bound, not disk	2026-05-29 09:43:37 -04:00
Anso	2d56ea958a	fix(stack-activity): per-stack history integrity, attribution, sanitization (#1228) ... * fix(stack-activity): per-stack history integrity, attribution, sanitization Address the Stack Activity audit findings (PR 1 of 2): - Per-stack history integrity: drop the per-insert 100-row prune in addNotificationHistory that evicted quieter stacks' history whenever another stack got chatty. Periodic cleanupOldNotifications now caps per (node, stack) at 500 rows and per-node unattached system events at 1000 rows, on top of the existing 30-day retention. Signature takes an options bag and returns a per-stage summary so MonitorService can log what actually ran each cycle. - Actor attribution: thread req.user?.username through every notifyActionFailure call site and add synthetic actors at service emit sites (system:autoheal, system:scheduler, system:image-update, system:docker-events, system:blueprint, system:monitor, system:policy). The timeline renders system actors as "via <Label>" so an autoheal redeploy is no longer indistinguishable from a user redeploy. - Message sanitization: new sanitizeNotificationMessage at NotificationService.dispatchAlert strips KEY=VALUE pairs whose key ends in TOKEN/KEY/PASSWORD/SECRET/CREDENTIALS/AUTH, scrubs HTTP basic auth in URLs and Bearer tokens, collapses COMPOSE_DIR paths, and truncates to 1000 chars. Applied to the stored history and to every downstream Discord/Slack/webhook channel. The ImageUpdateService recovery-path direct DB write also runs through the sanitizer. - Composite pagination cursor: getStackActivity now accepts a (timestamp, id) cursor (?before=&beforeId=). The legacy timestamp-only form silently dropped events when a single compose up emitted many events sharing one millisecond. Route rejects beforeId without before. - Frontend hardening: distinct error state with retry button (initial fetch failure no longer renders as the genuine empty state), strict positive-integer parsing on cursor params, overrequest-by-1 pagination so the last page does not leave a dead "Load more" click, runtime guard on liveEvents merge that validates the level union, per-minute day-bucket recompute so an open panel does not stay on "Today" past midnight. No tier, role, or capability gate touched. Route permission gate remains stack:read on the named stack. * fix(stack-activity): sanitizer covers lowercase env vars and per-node compose dir External review surfaced two leak paths in the message sanitizer: - The sensitive-key regex was uppercase-only. Compose env names are conventionally uppercase but lowercase forms (db_password, jwt_secret, github_token) are valid and do leak through the same Docker and compose-parse error paths. Make the regex case-insensitive and tighten it to also catch bare TOKEN= / KEY= / PASSWORD= without a prefix word, while still leaving BYPASS, COMPASS, and similar non-secret keys alone. - The compose-dir path collapse only read process.env.COMPOSE_DIR, but the real resolution chain is node.compose_dir (per-node DB override) -> process.env.COMPOSE_DIR -> /app/compose. A node with a custom compose_dir could still leak absolute paths into stored history and downstream channels. Route both the dispatchAlert call and the ImageUpdateService recovery-path direct write through NodeRegistry.getInstance().getComposeDir(localNodeId) so the collapse covers every resolution outcome. Tests now assert lowercase keys are redacted and that BYPASS-style non-secrets stay intact in both cases. notification-routing mock extended to stub the new getComposeDir call. * chore(stack-activity): a11y roles, visibility-aware tick, live-disconnect signal Close three small follow-ups on the per-stack activity timeline: - A11y: each day-group gets role="list" and each event row gets role="listitem" so screen readers traverse the timeline as a list instead of a wall of text. The day-group container also carries an aria-label naming the bucket. - Visibility-aware day-bucket tick: the 60s setInterval that re-derives Today/Yesterday/Earlier now short-circuits when document.hidden, so a backgrounded panel does not re-render every minute for no visible effect. - Live-disconnect signal: useNotifications dispatches a sencho:notifications-connection custom event on WebSocket open and close. The timeline listens and, when explicitly disconnected, shows a one-line "Live updates offline; reconnecting…" hint above the list. The sidebar ticker already surfaces fleet-wide connection state; this adds an in-context cue for users who are focused on a single stack. Stack-name case normalization was considered and rejected: stack names are case-permissive per the isValidStackName validator, and lowercasing on read or write would silently rename or hide a user's "MyApp" stack. * ci(stack-activity): drop unnecessary escape in URL_BASIC_AUTH regex ESLint no-useless-escape errored on \- inside the character class [a-zA-Z0-9+.\-] at notificationMessage.ts:14. Move the dash to the end of the class so it's an unambiguous literal and the escape is no longer required. Behavior is identical; sanitizer tests still pass. * revert(stack-activity): drop unvalidated E2E spec from this PR The spec was committed without ever running against a real Docker daemon, then failed in CI when it ran for the first time: deploy returned 200 but no notification appeared on the activity endpoint within the polling window, suggesting either a deploy-notification race or a node-id resolution mismatch in the CI environment. Backend unit tests (route + composite cursor + sanitizer) and frontend component tests cover the same logic. The E2E spec will land in a dedicated follow-up once it has been authored against a working CI environment.	2026-05-25 21:09:00 -04:00
Anso	f03c9dc7b6	fix(webhooks): address Codex review of PR #1177 (#1181) ... * fix(webhooks): close HMAC timing oracle on trigger reject paths The trigger handler in PR #1177 returned a uniform 404 for every unauthenticated rejection, but only the wrong-signature path computed an HMAC over the request body. The other reject paths (unknown id, disabled webhook, non-paid tier, missing X-Webhook-Signature header, missing rawBody) short-circuited before any HMAC work. Repeated near-rate-limit probes with a large attacker-controlled body could distinguish a valid-and-enabled paid webhook id from the other reject cases through response latency. WebhookService.validateSignature is now constant-time over every input shape: it always runs crypto.createHmac and crypto.timingSafeEqual against fixed-length 32-byte buffers regardless of whether the signature is missing, has the wrong prefix, is malformed hex, or is the wrong length. The trigger handler calls it unconditionally before any reject branch fires, using a stable per-process decoy secret (WebhookService.getDecoySecret) when the webhook does not exist and an empty buffer when the request has no body. Response timing now depends only on the size of the request body, which the attacker already controls and which reveals nothing webhook-specific. Six new tests pin the behaviour: validateSignature is observed firing on the unknown-id and missing-signature paths through a spy assertion, and four direct-call tests confirm validateSignature returns false without throwing for empty, wrong-prefix, malformed-hex, and wrong-length signatures. * fix(safe-log): redact Basic auth and lowercase Windows drive letters The redactSensitiveText helper now covers two cases the prior chain missed: * Authorization: Basic <base64> previously left the base64 payload intact. The existing key/value regex caught only the literal word Basic before stopping at the space. A new Basic\s+[A-Za-z0-9+/=]+ replacement runs before the key/value regex so the credential is scrubbed first. * Windows homedir paths like c:\Users\<user>\... with a lowercase drive letter previously slipped through because the regex required [A-Z]. Changed to [A-Za-z] so both letter cases are covered. Two new tests pin both fixes. * docs(webhooks): document 429, fix shared schema, comply with D27/D31 * Trigger endpoint declares the 429 response that webhookTriggerLimiter can return (500 requests per minute per source IP); both docs/openapi.yaml and the response table in docs/features/webhooks.mdx carry the new row, and a new troubleshooting accordion explains the shared-NAT scenario. * Shared Webhook schema in docs/openapi.yaml extends the action enum to include git-pull and documents the node_id property. The GET list endpoint returns these fields; the prior schema would have failed validation for any git-pull row. * docs/features/webhooks.mdx:7 rewritten from a customer-side role enumeration ("non-admins on a paid tier can view the list but cannot manage it") to a single requirement statement ("Webhooks require a Skipper or Admiral license. Managing webhooks is admin-only.") per CLAUDE.md D27/D31; the prior phrasing was customer-side fence-spec. * Two em dashes in webhook description strings I had touched in the prior OpenAPI sync commit replaced with semicolons per D18.	2026-05-23 16:03:48 -04:00
Anso	21ec5e7e0a	fix(webhooks): harden trigger response surface (#1177) ... * fix(webhooks): harden trigger response surface Bundles six audit findings on the incoming-webhooks trigger path. All changes preserve the documented happy path: a CI caller signing the exact request body with the webhook secret still receives 202 Accepted. * Uniform 404 on every unauthenticated rejection (missing webhook, disabled webhook, non-paid tier, missing signature header, missing raw body, signature mismatch). The four-way response surface previously let an unauthenticated probe enumerate webhook ids and fingerprint the instance's licence tier; callers now see one shape for any failed auth. * Fail closed when express.json()'s verify callback did not capture the raw request body. Previously the handler fell back to JSON.stringify(req.body), which compares the HMAC against a re-serialised payload that is not byte-equal to what the client signed. * Pass the already-loaded webhook through to WebhookService.execute() instead of re-fetching by id. Closes the delete-during-execution race where an admin deletion between the trigger handler's load and the async dispatch silently dropped the execution row. The webhook_executions table has ON DELETE CASCADE, so recordExecution now wraps the insert in try/catch and logs a warning when the FK constraint trips because the parent webhook was deleted mid-flight. * Redact bearer tokens, JWTs, URL credentials, and homedir paths from error strings before persisting to webhook_executions.error. The execution history is readable by any paid user via GET /webhooks/:id/ history; redactSensitiveText gains three home-directory patterns (/home/<user>, /Users/<user>, <drive>:\Users\<user>) and now runs on every error stored from this path. * Cap webhook name at 100 characters on both POST and PUT, rejecting non-string and oversized values with 400 before they reach the DB. * Validate the body's action override against a typed allowlist (isWebhookAction type guard) on the trigger endpoint, returning 400 before queueing execution. An unknown override no longer reaches recordExecution as a stored failure row. Tests updated to pass db.getWebhook(id)! instead of the raw id to the new execute() signature. Docs at docs/features/webhooks.mdx updated to reflect the new uniform 404 response, the new 400-on-invalid-action behaviour, and a rewritten troubleshooting accordion that walks operators through every cause of the uniform 404. * test(webhooks): cover trigger handler auth, race, and redaction paths Adds 21 vitest cases for the public webhook trigger handler and the WebhookService.execute / recordExecution pipeline, plus 3 cases for the new homedir patterns in redactSensitiveText. webhooks-trigger.test.ts covers, per audit finding: * M1 + H3 uniform 404: id unknown, webhook disabled, non-paid tier, missing signature header, missing rawBody, sha1= prefix, malformed hex signature, sig mismatch. Each asserts identical 404 body so a future regression that re-introduces 401 / 403 / PAID_REQUIRED breaks one of the 8 tests. * Happy path: 202 with configured action, valid action override, unknown action override returns 400 after auth succeeds (L2), non-string action override returns 400. * L1 name cap: POST and PUT both reject names over 100 chars and non-string names; 100-char boundary still accepted; PUT allows partial updates that omit name. * M5 race: deleting the parent webhook before recordExecution runs no longer crashes the async dispatch; the FK cascade is swallowed with a console.warn, and a happy-path test pins the recordExecution row. * M6 redaction: stubs ComposeService.runCommand to throw errors containing a bearer token and a homedir path, then asserts the persisted webhook_executions.error has both scrubbed. safe-log.test.ts gains three unit tests pinning the new homedir patterns in redactSensitiveText (Linux, macOS, Windows). The existing credentials test is untouched. Tests use prototype spies on FileSystemService and ComposeService (both hand out a fresh instance per nodeId), so per-test mocks do not leak. beforeEach restores all mocks and reapplies the LicenseService 'paid' spy. Closes audit finding H2 (zero trigger-path test coverage). * docs(webhooks): sync openapi spec with new trigger response surface Brings docs/openapi.yaml in line with the behaviour changes from the trigger hardening commit. Mintlify auto-generates the per-endpoint reference pages from this spec, so the spec drift would surface as wrong response codes in the public API reference. POST /api/webhooks and PUT /api/webhooks/🆔 * name: maxLength 100 (matches MAX_WEBHOOK_NAME_LENGTH on the route). * action enum: add git-pull (pre-existing omission; the route has always accepted it). * node_id: documented as an integer property (pre-existing omission). POST /api/webhooks/:id/trigger: * requestBody required: true (body is now mandatory; the H3 fail-closed branch rejects a missing rawBody). * action override: enum restricted to the allowlist. * 401 and 403 responses removed. * 404 response: description rewritten to reflect uniform-404 behaviour; the body is { error: "Webhook not found or signature invalid" } for every unauthenticated reject. * 400 response added for an authenticated request whose action override is not in the allowlist.	2026-05-23 15:42:49 -04:00
Anso	a51547a158	fix(monitor): decouple janitor disk-usage check from 30s cycle (F-6) (#1164) ... * fix(monitor): decouple janitor disk-usage check from 30s cycle (F-6) `docker system df` (called by the MonitorService janitor check) can take 30+ seconds on Docker Desktop with many volumes. Running it on the 30s evaluate cycle compounded with the per-container stats fan-out and pushed the cycle to 140s+, blocking subsequent monitoring work. This change: - Moves the janitor disk-usage check into its own 15-minute cycle with a tight 8s timeout. A circuit breaker opens after 3 consecutive timeouts (60-minute cooldown) so a sick daemon stops pinning Dockerode sockets every tick. The first janitor tick is deferred 45 seconds past boot to avoid head-of-line collision with the initial monitor cycle's stats fan-out. - Adds a paired 8s `withTimeout` wrap to the admin prune-estimate routes (`/api/system/prune/estimate` and the dry-run path of `/api/system/prune/system`) so a slow df does not hang the admin tab. Both routes respond 503 with code `docker_df_slow` on timeout. - Factors `withTimeout` and `TimeoutError` into `utils/withTimeout.ts` so the route layer does not have to import from a service module. - Adds 10 unit tests covering the decoupling guardrail, breaker open/close, cooldown, threshold gate, the 100 MB reclaimable floor, re-entrancy, recovery logging, non-timeout error handling, and the full timer-cleanup contract of `stop()`. - Adds 4 integration tests for the prune routes covering the 503 timeout response, the success path, and the non-timeout 5xx path. * fix(fleet,monitor): extend F-6 timeout to fleet prune routes; close breaker-recovery log gap Codex audit findings on PR #1164: Major. The fleet routes that fan out prune-estimate work on local nodes (`POST /api/fleet/labels/fleet-prune` dry-run path and `POST /api/fleet/prune/estimate`) called `estimateSystemReclaim` without a timeout, so a slow local Docker daemon could still hang the fleet admin tab even though the system-maintenance routes were already bounded. Wrap both call sites with the shared `withTimeout(..., 8s)` and surface a "Docker daemon is busy" message via the per-target and per-node error channels the routes already used for other failures. The destructive (non-dry-run) prune path stays unwrapped because it calls `pruneSystem` / `pruneManagedOnly`, not `df`. Minor. The janitor circuit breaker zeroed `janitorConsecutiveTimeouts` when it opened, so a successful call after a full breaker-open cooldown slipped past the `if (counter > 0)` recovery-log branch and never emitted `[Monitor] Janitor disk-usage check recovered`. The operator observability signal was missing exactly when it mattered most. Extend the predicate to also trip on `janitorBreakerUntil > 0` (which stays set to its past timestamp after cooldown until the next success clears it), so recovery logs symmetrically for both partial-failure and post-breaker recovery paths. Added a dedicated test. Three new integration tests cover the fleet routes (timeout, success, and the estimate endpoint's per-node unreachable shape).	2026-05-23 00:04:05 -04:00
Anso	2f2401df68	fix(fleet): route remaining fleet dispatches through getProxyTarget for pilot-agent nodes (#1152) ... * fix(fleet): route remaining fleet dispatches through getProxyTarget for pilot-agent nodes PR #1123 migrated POST /api/fleet/nodes/:id/update to use NodeRegistry.getProxyTarget so pilot-agent rows (no api_url / api_token) participate in remote update via the tunnel loopback. The same bug shape lived on at ten sibling fleet-dispatch sites: each read node.api_url and node.api_token directly, returning "Remote node not configured." against pilots, or silently filtered pilot rows out of a fan-out loop. Migrate every remaining fleet-wide remote dispatch to the same pattern: - routes/fleet.ts: fleet-stop, fleet-prune, prune/estimate, snapshot restore (4 sites) -> getProxyTarget + mode-aware error copy + conditional Authorization header - routes/imageUpdates.ts: fleet status + fleet refresh (2 sites) -> swap n.api_url filter for getProxyTarget != null and use target.apiUrl, so pilot rows appear in the aggregated image-updates view instead of being silently excluded - utils/snapshot-capture.ts: captureRemoteNodeFiles (1 site) -> same pattern; CaptureNode interface gains required mode field so the thrown error message picks the pilot-tunnel copy automatically - services/SecretsService.ts: resolveEnvFileRemote, readEnvRemote, writeEnvRemote (3 sites) -> same pattern; thrown errors now use the shared formatNoTargetError helper instead of leaking api_url/api_token field names Extract the previously-private noTargetMessage helper from fleet.ts into utils/remoteTarget.ts as formatNoTargetError so SecretsService, snapshot-capture, and the fleet routes share one copy of the mode-aware error string. Add 10 regression tests in fleet-pilot-dispatch-parity.test.ts covering each migrated route + the snapshot-capture utility: dispatch through the loopback target with no Authorization header for pilots, and a mode-aware error when the tunnel is disconnected. FleetSyncService (4 additional sites) carries an api_url-anchored targetIdentity in the wire protocol; pilot support there needs a protocol-level identity decision and stays as a separate follow-up. * fix(fleet): throw tunnel-disconnected error from resolveEnvFileRemote Codex audit flagged that resolveEnvFileRemote returned null when getProxyTarget was null. That predates the parity migration but the migration was the right place to fix it: the null flowed through readExistingEnv into previewPushDiff / executePush as "env file not found", which is wrong (the env exists, the node is unreachable). Throwing formatNoTargetError(node) here lets the existing catch arms in previewPushDiff (lines 450-453) surface reachable=false with the tunnel-disconnected message on the right axis, and executePush picks up the same shape via its outer catch. Also drop overstated coverage claims from the parity test header (snapshot restore + SecretsService were never actually exercised in this file, only structurally identical via tsc), and fix two describe labels that read /api/labels/* instead of the mounted /api/fleet/labels/*.	2026-05-22 00:30:51 -04:00
Anso	9dbce9c3c7	fix(spawn): attribute ENOMEM and ENOENT-under-memory-pressure spawn failures to host OOM (#1111) ... Operators previously saw "spawn docker ENOENT" or "spawn /bin/sh ENOENT" when the host was under memory pressure, which sent them down a missing-binary debugging path. Linux libuv's posix_spawn can fail to allocate its argv / path-search arena under low free memory and surface the underlying ENOMEM as ENOENT. Centralizes spawn-error mapping in a new utils/spawnErrors.ts helper: - Explicit ENOMEM is rewritten to "Out of memory while launching <command> (host free memory: X MiB of Y MiB)". - ENOENT under the 128 MiB free-memory floor is rewritten with the same wording plus a "reported as ENOENT under memory pressure" hint. - ENOENT for docker on a healthy host preserves the existing "Docker CLI unavailable on this node" mapping. - Other errors pass through unchanged. Applied at the four named offenders: ComposeService.execute(), ComposeService.captureCompose(), DockerController.getContainersByStack(), and FileSystemService.getStacks() (which gets an ENOMEM-aware log line for the scandir failure). Startup also logs host free/total MiB once and warns when free memory is below the 128 MiB floor, so the diagnostic surfaces before the first spawn attempt rather than after it fails. 37 tests cover the mapping function directly and the ComposeService / FileSystemService integration paths.	2026-05-19 07:27:12 -04:00
Anso	d882f223f4	feat(api-tokens): switch to sen_sk_ prefixed opaque keys (#1062) ... * feat(api-tokens): switch to sen_sk_ prefixed opaque keys Replace JWT-shaped API tokens with 56-char opaque keys of the form `sen_sk_<43-char base62 random><6-char base62 checksum>` (256-bit entropy, sha256-truncated checksum). Node-proxy tokens stay JWTs. Why: * The api_token path was already a sha256 DB lookup; the JWT signature was wasted work and the 400d JWT ceiling vs DB expires_at was a confusing dual bound. * Opaque tokens carry a verifiable checksum so malformed/typoed values are rejected before any SQLite lookup. * `sen_sk_` prefix is recognizable to GitHub, TruffleHog, GitGuardian and makes the on-wire shape visually distinct from node_proxy JWTs. Changes: * New `utils/apiTokenFormat.ts` (generate + checksum-verify, CSPRNG via randomInt, timingSafeEqual on the checksum compare). * `middleware/auth.ts` and `websocket/upgradeHandler.ts` route opaque tokens before any jwt.verify; 401 messages unified to avoid a token-existence oracle. * `middleware/rateLimiters.ts` short-circuits opaque tokens in the node_proxy detection and keys per-token via a non-reversible sha256 slice so each token keeps its own bucket without a DB hit. * All six existing tests migrated from jwt.sign({scope:'api_token'}) to generateApiToken(); new format-only test suite covering prefix, length, alphabet, checksum reject paths, and a 10k-iteration collision/integrity loop. * Docs (features/api-tokens.mdx, api-reference/overview.mdx) describe the shape and drop the obsolete JWT-ceiling note. * fix(api-tokens): clear CI lint and CodeQL false positives * Drop unused TEST_USERNAME import in remote-console-session.test.ts; the migration to generateApiToken() left it orphaned. * Add a CodeQL barrier model so `generateApiToken`'s ReturnValue does not flow into the `insufficient-password-hash` query. The function emits 256-bit CSPRNG opaque keys; sha256 of the raw token is the correct construction for high-entropy API tokens (bcrypt-class hashes target low-entropy human passwords). CodeQL's name heuristic was treating "Token" as a password source and flagging the standard sha256 wrapping at all 9 call sites. * ci(codeql): exclude js/insufficient-password-hash for token paths The previous barrierModel data extension was a no-op for this rule: the js/insufficient-password-hash query identifies its "password" sources via SensitiveExpr's name heuristic ("token", "secret", "key" substrings), which is upstream of the taint-tracking layer where barrierModel applies. Verified by post-push re-analysis: 9 alerts still open, all undismissed. Replace the dead extension with a path-scoped query-filter in codeql-config.yml so the rule no longer fires on apiTokenFormat, apiTokens, and the test directory. Real user-password hashing code elsewhere in the repo (auth, users, setup routes) remains analyzed. The 9 existing alerts on PR #1062 are dismissed via API as false positives with a justification pointing at this config. Future runs will not re-flag them because of the path filter.	2026-05-15 18:21:28 -04:00
Anso	a38a3e0226	feat(mesh): route mesh traffic over Distributed API remotes (#1048) ... * refactor(mesh): extract shared TCP stream switchboard Pull the `tcp_open` / `tcp_open_ack` / `tcp_open_reverse` / `tcp_close` + `TcpData` handling out of the pilot agent into a reusable module so a second caller (the upcoming proxy-mode WS handler) can run the same frame parser, stream allocator, idle timers, and Compose-label resolver. One parser, two callers, zero drift on a security-sensitive protocol surface. The pilot agent keeps its public `openMeshTcpStream` API and delegates to a per-connection switchboard constructed in `connect()` and torn down in `cleanupAfterDisconnect`. Existing reverse-stream and resolver tests are rewritten to exercise the shared module directly. * feat(mesh): route mesh traffic over Distributed API remotes Sencho Mesh now works against remotes in Distributed API mode in addition to Pilot Agent mode. Central opens a short-lived WebSocket tunnel to the remote's new `/api/mesh/proxy-tunnel` endpoint on demand and tears it down after 5 minutes of idle (configurable via `SENCHO_MESH_PROXY_TUNNEL_IDLE_MS`). Mesh dispatch is mode-agnostic at the routing layer; `PilotTunnelManager.ensureBridge` resolves an existing tunnel or asks the new `MeshProxyTunnelDialer` to dial. The Routing tab badges now use a `reachableMode` classifier: `★ Local` for local, `pilot offline` red badge for a pilot with a down tunnel, and a new `unreachable` red badge surfacing the specific reason (missing token, scope not full-admin, TLS failure, remote does not support proxy mesh). Distributed API remotes with valid credentials show no negative badge; the tunnel opens on first dial. Auth: the proxy-tunnel WS upgrade requires an `Authorization: Bearer` API token with the `full-admin` scope. Lower-scoped tokens are rejected at upgrade time so a leaked read-only token cannot reach the mesh data plane. Bidirectional: the proxy-mode WS handler registers itself as the local `MeshService` reverse dialer (compare-and-swap), so meshed containers on a Distributed API remote can dial cross-node aliases via `tcp_open_reverse` over the same tunnel. Cross-node relays (`PilotTunnelBridge.acceptReverseRelay`) await `ensureBridge` so proxy-to-proxy mesh works without standing tunnels. * docs(mesh): describe Distributed API mesh and unreachable troubleshooting Refresh the user-facing mesh documentation to reflect that mesh works over both Pilot Agent and Distributed API remotes. Adds a Troubleshooting accordion entry for the new `unreachable` Routing tab badge so operators can match a tooltip reason ("api token rejected (scope must be full-admin)", "remote does not support proxy mesh", "TLS handshake failed", "api_url not set", "api token missing") to a concrete fix. * refactor(mesh): apply /simplify review findings Five cleanups surfaced by the post-implementation code review pass. No behaviour change for fleets running on `main`; only internal structure improves. - **Deterministic container IP shared.** Extract the compose-default network preference (`pickContainerIp`) and the conventional-name fast path (`lookupContainerIp`) into `backend/src/mesh/containerLookup.ts`. Both `MeshService.resolveContainerIp` (existing same-node fast path) and the switchboard's `resolveByComposeLabels` (proxy/pilot inbound dial) now use the same logic. Earlier the switchboard helper grabbed the first `Object.values(Networks)` IP, which could flip across daemon versions on multi-network containers; fixed. - **WS URL upgrade extracted.** New `backend/src/utils/wsUrl.ts` exposes `httpUrlToWs(baseUrl)` that maps `http://` to `ws://` and `https://` to `wss://`. Used in both `pilot/agent.ts` and the proxy-tunnel dialer. The previous inline `replace(/^http/, 'ws')` silently downgraded `https://` to `ws://` (cleartext). - **`computeReachable` takes the pre-fetched node.** `getStatus` already iterated `db.getNodes()`; the helper previously re-queried by id per node (N+1 reads). Pass the row in. - **Reachable-reason ternary -> const map.** Replace the nested ternary in `MeshService.computeReachable` with a `Record<DialFailureCode, string>` lookup keyed on the dialer's failure code. - **Activity-type ternary -> const map.** Same flattening inside `MeshProxyTunnelDialer.logActivity`. All mesh tests pass (85/85). Full backend suite green (2126/2126). * fix(mesh): cache proxy dial failures and contain WS handshake errors `ensureBridge` now consults the recent-failure cache before dialing so a continuous mesh workload against a misconfigured proxy-mode remote does not produce one upgrade attempt per cross-node TCP open. `recordFailure` emits at most one activity-log entry per cache window per (nodeId, code) so a connect-loop on a single bad remote cannot flush the ring buffer. Failure messages run through `redactSensitiveText` before reaching the log so any embedded Bearer / JWT / inline-URL credentials are scrubbed. `stop()` clears the inflight map and `dial()` checks `this.stopped` between awaits so a shutdown does not leak a half-opened bridge. When `awaitOpen` rejects (401, 4403, TLS), the dialer attaches a noop 'error' listener before calling `ws.close()`. Without it the ws library emits a tail 'error' on a still-CONNECTING socket that propagates as an unhandled exception. Surfaced by a live-network test against a real proxy-mode peer. Adds `mesh-proxy-tunnel-live.test.ts` (skipped unless MESH_AUDIT_URL and MESH_AUDIT_TOKEN_FILE env vars are set) covering both the happy path and the auth-rejected path against an actual remote Sencho. * feat(mesh): instrument proxy tunnel observability Adds three counters to `PilotMetrics`: - `proxy_bridges_total` (incremented on `registerProxyBridge` success) - `proxy_dials_failed` (every failed dial attempt, not deduped) - `proxy_idle_closes` (idle-sweep teardowns) All three surface automatically via `GET /api/system/pilot-tunnels` since the route returns the full `Counters` snapshot. Gates two pre-existing always-on `console.warn` calls in `tcpStreamSwitchboard.ts` (mid-stream socket errors and Docker resolve failures) behind `isDebugEnabled()`. Both fire on per-stream events and would otherwise violate the diagnostic-log safety rule under load. Adds `mesh-tcp-stream-switchboard.test.ts` covering the forward `tcp_open` path: real localhost dial, resolver errors (no_target, denied), per-tunnel cap saturation, frame-routing fall-through invariants, `tcp_close` socket teardown. Drops `MESH_CONNECT_TIMEOUT_MS` from the public surface; only the switchboard itself uses it. * fix(mesh): tighten Routing tab unreachable handling `RoutingNodeCard.tsx`: the **Add stack to mesh** button now disables when `reachableMode === 'unreachable'`, matching the existing TogglePill behavior. Without this, an operator on an unreachable node could open the opt-in sheet, confirm, and watch the redeploy proceed against a target whose mesh data plane will silently fail to route. Also drops the leftover `status.nodeId !== -1` guard on the pilot- offline badge. `MeshService.ts`: renames `isMeshReachable` to `isMeshConfigured` with the new predicate that returns true for proxy-mode remotes whose creds are valid (the tunnel is opened on demand). `getRouteDiagnostic` now distinguishes "routable" (configured) from "pilotLive" (live tunnel state, only meaningful for pilot mode); without the split, every idle proxy-mode route would report `tunnel down`. `MeshService.setReverseDialer` warns when the unconditional install path silently overwrites a non-null current dialer. By topology a Sencho is either pilot or central, so the branch flags a misconfigured deployment rather than an expected race. Drops the dead `export { ReverseTcpStreamHandle }` re-export from `pilot/agent.ts`; its only consumer imports straight from `mesh/tcpStreamSwitchboard.ts`. Fixes a stale doc comment in `MeshService.ts` that referenced the wrong source file. Adds `mesh-proxy-tunnel-handler.test.ts` covering the WS handler lifecycle: pilot-mode 404 rejection, post-upgrade reverse-dialer install, concurrent-upgrade 1013 rejection (single-tenant slot), and error-path teardown. Updates the troubleshooting accordion in the user docs to mention the disabled-state behavior. * fix(mesh): close ESLint and CodeQL findings on the proxy-tunnel diag logs Remove the unused `reverseDialer` local in `meshProxyTunnel.ts`; the closure target is `localDialer` above and this assignment was always dead. Strip line breaks inline on the three `MeshProxyTunnelDialer` diag log lines so CodeQL `js/log-injection` data flow recognises the sanitisation that `sanitizeForLog` already performs. No behaviour change; the diag logs render the same characters they do today.	2026-05-14 20:51:42 -04:00
Anso	b52323036b	fix: harden stack label permissions (#1036) ... * fix: harden stack label permissions * fix: avoid test db init from debug logging	2026-05-13 11:56:22 -04:00