mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-09-03 14:18:02 +00:00
feat(git): add GitOps revision store and Direct apply transitions (#1835)
* feat(git): add GitOps revision store and Direct apply transitions
Adds the canonical GitOps revision state model: schema, store, history,
approval resolution, Direct apply transitions, and a first-cut read
projection. Also widens recovery capture with generation, artifact, and
source-acceptance bindings, rejects credentials and query strings in new
Git repository URLs, and keeps Blueprint and node-label routes on the hub.
* feat(git): add create-from-Git activation and teardown transitions
Adds the durable half of create-from-Git: the single transaction that persists
the application, resolved commit, generation, checkpoint, and candidate
together, and the teardown that tombstones a create which never reached its
success boundary. Adds the staging marker and operation-owned cleanup rules
that decide what a crashed create is allowed to delete.
* feat(git): make create-from-Git crash-safe
Wires the create path through the GitOps state model: the staging marker is
written before the candidate is built, the activation transaction persists the
application, generation, and checkpoint together, and the source row plus
accepted pointers now commit as one success boundary. A create that fails
before that boundary removes only what it staged, then tombstones itself.
The managed-area sweep no longer deletes a directory it cannot prove it owns,
counts in-flight creates as claimed, and is awaited at startup so an
interrupted create is settled before any mutation service runs.
* feat(git): settle interrupted create-from-Git at startup
Adds boot-time recovery for creates a previous process left in flight. A create
whose project was already committed to disk is finished, including its source
row and accepted pointers. Anything earlier has its files removed first and is
then tombstoned, so the stack name becomes usable again instead of staying
locked by a half-created application. A source row that outlived its
application is always preserved.
* fix(git): close create-from-Git ownership and reporting gaps
Removes two ways a failed create could destroy files it did not own: boot
recovery no longer deletes a stack directory the create never recorded making,
and the managed-root teardown is gated on having created that root. Cleanup
now rejects a path that is not a generation directory, so a malformed row
cannot widen the blast radius.
A create that fails during materialization now clears its staging marker, which
previously stayed behind and made the stack name uncreatable until restart.
Generation paths no longer resolve through a module cycle that left them
pointing at a directory that never existed.
Boot recovery restores the deploy spec and manifest cache a finished create
needs, settles each application independently so one bad row cannot strand the
others, and distinguishes a missing directory from one it could not read. A
create that succeeds now clears its checkpoint, and a failure after the success
boundary says the stack was created rather than reporting a bare error.
* fix(git): reap managed areas that nothing has ever claimed
The boot sweep now distinguishes a missing staging marker from an unreadable
one. A missing marker means no stack, no create, and no marker claims the
directory, which is the ordinary orphan a crashed stack deletion leaves behind,
so it is removed as it always was. An unreadable marker is evidence of a claim
that cannot be verified, so the area is preserved.
Without this an orphaned managed area survived every boot forever, since a
completed create deletes its marker and nothing else would ever claim it.
* feat(git): complete the Direct source and target transitions
Adds the remaining Direct events the state model owed: an invalid fetch that
advances the resolved commit without minting a candidate, a blocked candidate
that is visible but cannot apply, dismissal that leaves the running workload
alone, material configuration change that invalidates a staged candidate while
keeping accepted and applied pointers, the two deploy failure classes, and
application and target tombstones.
Widens the application update to every mutable column. It previously wrote
about half of them, so a transition assigning a pause, suspend, recovery, or
intent field would have type-checked, appeared in the history snapshot, and
been silently dropped at commit.
* feat(git): record Direct Git operations in the revision state
Wires fetch, apply, and detach through the state model. A pull records the
fetch and, when it produces a candidate, the generation behind it, marking a
blocked change plan as a blocker that cannot apply. An apply records the
acceptance that binds the generation to the workload, and closes its operation
on failure so a throw cannot leave the source reporting work in progress.
Detach tombstones the application and its target in the same transaction that
removes the source row.
Every producer is a no-op for a stack with no live application, so installs
whose Git stacks predate this model are untouched until migration runs.
* test(git): cover the Direct Git producers end to end
Adds a harness that stubs only the git transport and rollback capture, so
fetch, candidate materialization, change-plan classification, apply, and detach
all run for real and are asserted against the resulting revision state.
Covers the seam the transition-level tests could not: that a fetch advances the
resolved commit without moving the accepted generation, that the apply binds
the exact candidate the fetch recorded and its acceptance proves that
generation and no other, that detach tombstones while keeping configured
identity as a frozen fact, that a failed fetch closes its operation and leaves
every pointer alone, and that a stack with no application produces no writes.
* feat(git): bind the deployed generation from the Compose adapter
Makes ComposeService the sole producer of deploy events. Every deploy path
funnels through deployStack, so recording it there keeps one start and one
terminal row per mutation instead of each caller reporting its own.
A successful deploy binds the applied generation, a failure records the class
and leaves the deployed pointer alone. The class is conservative: once the
compose command is handed off we cannot prove the workload was untouched, and
claiming it was intact is the more dangerous error. A stack with no live
application, or one with nothing applied, has no deploy identity to record and
is skipped entirely.
* feat(git): promote healthy and last-known-good from health verdicts
A stack health run now records which generation it observed, and its verdict
drives the revision state. Promotion is narrow by design: healthy and
last-known-good move only when the run passed, watched the whole stack, and the
generation it watched is still deployed. A stale or service-scoped verdict
records history and moves nothing.
Last-known-good keeps the artifact expectation only when that expectation
belongs to the promoted generation. Otherwise the generation is still good and
its executable identity simply is not proven, so the pointer is left null
rather than borrowed from another generation.
beginStack now takes the deployed generation explicitly rather than reading
current state, because a verdict is only meaningful for the generation the run
actually observed. Update-path callers pass null until updateStack reports what
it deployed.
* feat(git): bind the deployed generation on the update path
Gives updateStack the same treatment deployStack has: it opens a deploy
operation at the point Compose is handed the mutation, closes it on both
outcomes, and reports the generation it bound. The orchestrator carries that
binding through, and all five update call sites now pass it to the health gate
instead of null, so a passed health run after an update promotes healthy and
last-known-good exactly as it does after a deploy.
The operation is opened at the compose call rather than at entry, so an update
that fails during capture or classification records nothing: it never touched
the workload, and reporting a deploy failure for it would make the projection
claim something that did not happen.
* feat(git): track linked stacks and material config changes
Linking an existing stack to a Git source now brings it into the revision
state: the application starts live with nothing desired or accepted, so the
projection asks for a fetch rather than claiming a state it has not observed.
Editing a source's material configuration invalidates its staged candidate in
the same transaction that writes the row and clears the pending pull. A
candidate built from a different repository, ref, or file set can no longer be
applied, and clearing pending without invalidating it would leave the model
offering an apply the operator cannot produce. Credential-only and policy-only
edits change nothing material and leave the candidate alone.
* feat(git): retire the application when a stack is deleted
Stack deletion now tombstones its GitOps application and targets in the same
transaction that commits the deletion. A deleted stack that kept a live
application would go on claiming the stack name and block re-creating it
through the unique live-application index.
The tombstone is driven from the deletion service rather than from inside
DatabaseService, so the store keeps its transitions and their history in one
place and the two modules do not form an import cycle.
* feat(git): retire node targets when a node is removed
Deleting a node now tombstones the GitOps targets that lived on it, in the same
transaction as the delete and while those rows still exist. Otherwise a
part-way failure would leave targets pointing at a node that is gone.
Applications stay live: a Direct application still describes a real stack, and
a Blueprint one may have targets on other nodes. Both the local and remote node
paths go through the same helper, so neither can skip the retirement.
* feat(git): add proof-bound recovery transitions
A restore moves a target back to an older generation, the one case where a
target and its application legitimately disagree about what is current. These
transitions decide what may move with it.
Pointers move only when the restore is provable: the recovery point named a
generation, that generation still exists, and it belongs to this application.
The artifact expectation comes from what the recovery point captured, never
from what the application expects now, and the acceptance is kept only if it
still proves the restored generation rather than borrowing one that authorized
a newer one. A last-known-good survives unless the generation behind it is gone
or turns out to belong elsewhere, and then the reason is recorded so the
projection can say unavailable rather than none.
An unproven restore is still recorded as a real operational recovery and moves
nothing, because there is no evidence to move pointers to.
* fix(git): close GitOps operations that cannot be recorded
Recording a transition still never fails the operation it describes, but a
rejected terminal event no longer leaves that operation open. A start that
never terminates reported work in flight for ever and offered no actions, and
because a new fetch refuses to open a second operation, one rejection silently
stopped the model tracking that stack until a restart.
Startup now reclassifies operations the previous process left open, which the
transition already supported but nothing called. An interrupted restore is
closed there too, since only the terminal recovery events clear it.
The pull records its fetch, generation, and candidate as one transaction, and
the apply verifies the candidate was built from the commit being applied.
Separately those two allowed an apply to accept a generation whose files were
never on disk while the projection reported the older commit as current.
Also: the candidate now records its own invocation rather than the one it
replaces, changing material configuration is refused while an operation is in
flight, a tombstoned target cannot be repopulated by a late health verdict, a
deploy handle is not returned when its start was not recorded, the mutation
handoff is marked only once Compose is genuinely about to receive it, and a
rejected tombstone fails one deletion rather than aborting the boot sweep.
* feat(git): record why unprovable evidence was dropped
The projection carried a limitations array that only read-time derivation ever
wrote to. When a transition cleared a pointer it could not prove, the result
was indistinguishable from that pointer never having existed, and in two cases
it made the target read healthier: a dropped artifact expectation silently
disables the runtime drift check, and an unproven restore left every pointer
agreeing with itself so the target reported as synced and healthy.
Adds a persisted, fail-closed evidence record on the application and target
rows. The transitions that drop an artifact expectation, a last-known-good
artifact, or a source acceptance now say why, an unproven restore is marked as
such, and the deriver folds all of it into the limitations the projection
already exposes. Clearing a code when the evidence becomes provable again is
part of the contract, so a stale doubt cannot outlive its cause.
This lands before the migration matrix because migration is the largest
producer of evidence that cannot be proven, and the plan requires those to
surface as bounded limitation evidence rather than as fabricated pointers.
* feat(git): migrate pre-existing Git stacks into the revision state
Git stacks created before this model had no application at all, so every
producer was a no-op for them and the projection could not describe them. They
are now brought in at boot.
The governing rule is that a canonical pointer is written only when the
evidence proves that exact generation under the repository and ref configured
now. A legacy applied commit is not that proof on its own: the manifest may be
gone, unreadable, or stamped for a repository the stack no longer points at. In
each of those the commit is kept as recorded limitation evidence and the
pointers stay null, so the stack asks for a fetch instead of asserting a state
nobody verified. Deployed, healthy, and last-known-good are never invented,
because a manifest proves what was materialized, not what is running, and no
source acceptance is written because nobody approved through the model.
Replay is decided by scope, schema version, and configuration fingerprint. A
changed fingerprint re-runs the matrix, but a stack that already has an
application is skipped rather than rebuilt, so migration cannot overwrite
pointers written with proof it does not have. A stack whose directory has
vanished migrates to a tombstone rather than claiming a name it cannot back.
* feat(git): record rollbacks in the revision state
A restore now opens a recovery in the model before any file moves, so a crash
mid-restore leaves a target that says what it was doing rather than one that
merely looks broken, and closes it on both outcomes. The failure is classified
by whether the files had already been restored, because only a failure before
that leaves the previous workload provably intact.
Pointers move only when the restore is provable: the recovery point named a
generation, that generation still exists, and the manifest actually restored
carries the same commit and manifest version. Anything less is still recorded
as a real recovery, it simply has nothing to bind, and the transition marks it
unproven rather than guessing.
The deployed pointer is never claimed here. This path drives Compose through a
callback that reports nothing back, so binding cannot be proven and applied
moves without it.
* feat(git): add the deferred rollout-state transitions
Retry scheduling, suspension, pause, and partial rollout complete the
transition store. They have no production writer by design, but implementing
them now means the deriver has no branch a writer cannot reach, and the shape a
future producer has to satisfy is pinned rather than inferred from the read
side.
None of them is a statement about health. A scheduled retry leaves the failure
that caused it visible, so a stack that keeps failing does not read as merely
busy. A suspended source keeps everything it had accepted, and an operation in
flight is interrupted rather than abandoned so it cannot report as running for
ever. A paused or partially rolled out target keeps whatever was deployed, and
the partial record never stands in for a deployed pointer.
* test(git): stop the Compose producer tests reading Docker from the host
Both deploy cases asserted that the compose command would reject, which was
only true on a workstation with no Docker daemon. On Linux the command
succeeded, the promise resolved, and the assertions failed.
The compose subprocess now reports an exit code each test chooses, so the
adapter is what decides the outcome rather than the machine. Only the verbs
that need a daemon and travel through spawn are answered; `config` still runs
for real, because a host with the CLI and no daemon parses compose files
exactly as CI does, and failing that call breaks every create in the file.
That also makes the bound path reachable, so the deploy test now covers it:
a failed compose leaves the deployed pointer alone and classifies
post-mutation, and a successful one binds the applied generation, returns it
for health to bind against, and clears the earlier failure.
* fix(git): contain every managed-area path at the call that uses it
A stack name reaches the managed root without passing through
`isValidStackName` on this path, so the marker, the cleanup, and the create's
root probe all built filesystem paths from unvalidated input. Each call now
resolves its target against the managed area and checks containment in its own
scope, which is also the form the security scan credits: it does not follow the
barrier through the shared `isPathWithinBase` helper, so the check has to sit
with the call it protects.
`cleanupUnclaimedManagedRoot` removes a whole root recursively and gets the
same check, even though it was not among the reported calls.
Clearing a staging marker can now fail, so the two boot-recovery branches that
dropped the checkpoint first were reordered. Losing the checkpoint while the
marker survives would report the area as settled and leave the stack name
uncreatable.
The deploy adapter's two failure logs built their format string from the stack
name; both now use a constant format with the name as an argument.
The marker and cleanup fixtures stood a bare temp directory in for a managed
root, so nothing exercised the invariant these checks enforce. Both now build a
real managed area over a scoped data directory.
* fix(git): refuse to claim a managed area whose marker cannot be read
A marker that exists but will not parse is still someone's claim, yet the write
path refused only a readable foreign marker and wrote straight over a corrupt
one, discarding the reason and logging nothing. A transient permission error, a
marker truncated by a crash between write and rename, or a full disk all read
as corrupt, and each one let a second create take ownership of an area the
first still owned. Every other path in the module preserves on corrupt; this
one now does too, and says why.
Clearing a settled create's staging marker can fail, and reporting that as a
retained create sent a reader looking for an unfinished create that had
finished. It reports `marker_retained` instead. The checkpoint is still kept
for the retry, so its encrypted token can outlive the create while the clear
keeps failing; that tradeoff is stated at the call, because the alternative
leaves a marker no later create can get past.
The reaper also computed why it was preserving an area and threw the reason
away, so a directory could survive every boot with nothing said about it.
The containment checks these paths rest on had no tests and were silently
deletable, so the out-of-area read, write, delete, and reap now have them.
* feat(git): bind and observe a recovered generation
A proven restore could never claim the deployed pointer, because the restore
path drives Compose through a callback that reported nothing back: a caller
that restored some other way resolved identically, so binding on a resolved
promise would have claimed a workload nobody launched. The Compose wrapper now
returns what it did, and only that answer binds.
With binding reachable, a bound recovery claims its health run inside the same
transaction as the pointers it describes, and arms the timer once that
transaction lands. Committing the two together is the point: a crash between
them would otherwise leave a restored workload nothing was watching. Anything
that stops the timer starting writes the run off immediately, so an observing
row never outlives the timer meant to watch it, and a reservation is never
armed across a restart.
The reservation is handed to the transition rather than reached for, because
the health gate reports its verdicts back through the transition store and
importing it there would close a module cycle.
Startup now finalizes each interrupted observation on its own instead of
sweeping them with one update, so the revision state hears a verdict for every
run rather than watching the rows change under it.
* feat(git): add the rollout-scoped rollback transitions
Completes the deferred-state set. The three aliases write the same recovery
columns under the same rules as the recovery events, differing only in
provenance: Direct Git recovery emits `recovery_*`, and a later rollout
producer emits these. Nothing in this PR writes them, which is why they are
tested directly rather than through a caller.
`partial` is the one failure class they add, for a rollback that reached some
targets and not others. Completing has no unproven variant on purpose: a
rollback nobody can bind to a generation has nothing to complete against, so
it refuses without one and refuses a generation another application owns.
Also covers the recovery health reservation end to end: reserving writes and
links the row without arming anything, a replay reuses the run rather than
opening a second observation of the same restore, arming inserts nothing and
supersedes a conflicting stack gate, an unarmed reservation is written off
once, and a reservation that outlived its process is finalized rather than
armed. The per-row startup sweep is pinned too, since one row whose write
fails leaving the others finalized is the entire reason it replaced a bulk
update.
* feat(git): add the Blueprint source and deployment transitions
The first slice of the Blueprint work: the store side, with no production
caller yet. The routes and the reconciler are wired to these next, and pinning
the contract first means a caller cannot quietly satisfy a different one.
What these enforce is that a terminal event names the request it answers. A
deploy records the intent and candidate it was launched for, and a terminal is
accepted only against that same request, matching stage and identity on one
side rather than either in isolation. Without that pairing a deploy could be
acknowledged out of an in-flight withdraw, or one request's intent stored
against another's candidate. An acknowledgement carries the candidate from the
matched request rather than from its own payload, for the same reason.
Releasing an operation now releases its identity with it, and a resolved
interruption is retired rather than left to match again. Both were leaking
through the shared helpers: a stale identity let a later start resurrect a
superseded intent as live, and a surviving interruption both reported
completion as unknown for ever and let a late acknowledgement regress the
target after two later deploys had succeeded. Fixing the helpers fixes the
Direct paths too.
A start refuses to displace an unrelated operation, which would otherwise
abandon it with no terminal event and no history saying so. Observations write
the latest stage, since the placement facet reads it to decide whether a
stateful deployment is waiting on confirmation, and history alone could never
reach a reader.
Minting is left to the caller. A no-op Blueprint edit must mint no intent,
because a fresh identity would invalidate acknowledgements that are still
accurate.
* feat(git): record Blueprint create, edit, pin, and delete
Wires the Blueprint routes to the revision state. Each producer writes the
Blueprint source row and its GitOps rows in one transaction, so an operator
never sees a Blueprint that exists with nothing describing what it means, or an
intent for a Blueprint that failed to save.
The question these answer is when an edit invalidates what the fleet already
acknowledged. Changing the name, the compose content, the selector, the drift
mode, or whether it is enabled changes what nodes run or where, so each mints a
new intent and candidate. Changing the description or the classification
changes how the Blueprint reads and nothing a node can observe, so it updates
the source row alone: minting there would make every acknowledgement report as
stale over a reworded sentence.
An edit is measured by value, not by which fields were submitted. The editor
sends every field on every save, and the source layer decides what to
invalidate from which keys are present, so only the keys that genuinely differ
are passed down. Without that the two disagreed: rewording a description
advanced the revision past the one the current intent describes and cleared the
approval, while this layer classified it as metadata and minted nothing.
Selectors compare by value too, so reordering a list that names the same nodes
is not a placement change.
That measurement also changes one behaviour: a save that alters nothing now
leaves the approval alone, where before any save cleared it. Nothing changed,
so nothing is invalidated.
Pinning revises placement the same way a selector edit does, and re-pinning the
node already pinned is not a change. Deleting tombstones the application and
its live targets so the Blueprint stops claiming its slot, and withdraws
nothing itself, because the route has already done that and doing it twice
would record removals that never happened.
The required node set is stored in a canonical order, so reordering it is not
mistaken for a placement change. A Blueprint that predates the model has no
application yet and is left for migration rather than given a first intent
here, which would claim a starting point for deployments nobody has reconciled.
Desired nodes are passed in rather than computed here: the reconciler that
knows how to compute them reaches this layer, and importing it back would close
a module cycle.
* feat(git): record the placement a label or cordon moves
A label and a cordon say nothing about any one Blueprint, but both change which
nodes a selector matches. Each now revises placement for whichever Blueprints
the change actually moved, by comparing the desired nodes either side of the
write rather than reacting to the event.
That comparison is the whole point. Labelling a node no selector mentions, or
cordoning one no Blueprint wanted, moves nothing, and minting an intent for it
would invalidate every acknowledgement in the fleet over an edit no node can
observe. The same comparison covers a Blueprint pinned to a cordoned node: a
cordon governs automatic placement only, so the pinned target still wants that
node and its set does not move, without needing a case of its own. Nodes are
compared as a set, so the same nodes returned in a different order is not a
change either.
Each route wraps its existing write and the recording in one transaction, so a
recording failure cannot leave a fleet selecting on a label nothing recorded.
The write itself is untouched, which keeps label validation and the per-node
limit where they already live rather than restating them in a producer.
Node deletion already retires its targets, so this adds nothing there.
* feat(git): record Blueprint deployments by what caused them
Every production write to a Blueprint deployment row now goes through one
funnel that records the cause. The cause is carried rather than inferred from
the resulting status, because several causes land on the same one: a deploy
that failed and a withdraw that failed both read `failed`, and they mean
opposite things about whether the deployment is still on the node.
Recording is skipped when the status did not move, so a reconciler tick that
re-asserts a state it already reported does not append a second event
describing the same fact. The write still happens either way.
A terminal answers the request the target says it was given, not whatever the
Blueprint currently wants: an acknowledgement matched against the current
intent would accept work for a revision that node was never sent. The first
deploy to a node creates its target, since a Blueprint application has no
targets until something is sent somewhere.
Recording never fails the deployment. The rollout already happened, and turning
a bookkeeping problem into a stuck rollout would be the worse outcome.
The four reconciler observations record what was seen and nothing else. Preview
cleanup stays outside this path deliberately: it reverses a projection nobody
deployed, so recording it would report removals that never happened.
* fix(git): record an Inline Blueprint coming into existence
Creating a Blueprint inserted its application row directly, so the history
began at the first intent and described an application nothing recorded coming
into existence. Every other application-creation path emits the activation
event, and the event's own definition covers this case: an Inline Blueprint
inserts the application, the first intent, and the first candidate.
Activation goes through a transition now, which also enforces the one thing the
direct insert could not: a Blueprint gets one live application, refused rather
than silently duplicated.
No target is created. A Blueprint application has no targets until something is
deployed somewhere, unlike a Direct one which always has the node its stack
lives on.
* fix(git): file withdraw failures as withdraw failures
Four of the five withdraw failure paths were recorded as failed deploys. Only
the thrown one was tagged correctly, and it is the rare case: a delete-lock
conflict, a network error, a remote 409, and a remote non-200 all return rather
than throw, and all four were filed under deploy.
That inverted the thing this funnel exists to prevent. Recording a deploy
failure clears the in-flight operation, so a routine "another operation is
already in progress" wiped the withdraw that had just started and left the
target reporting a failed deploy while the deployment was still on the node.
A withdraw refused because the on-disk stack is not ours is now its own cause
too, rather than borrowing the deploy-side name conflict.
The status guard no longer suppresses starts. A start writes the identity that
terminals are matched against, so skipping one because the row already read
`deploying` let a later acknowledgement answer a request that had been
superseded, reporting a node as converged on a revision it was not running. The
guard now covers observations only, which is where repetition is the reconciler
re-asserting a state it already reported. A newer deploy supersedes an older
one so a redeploy of a stuck request can take over; anything else in flight is
still refused rather than displaced with no terminal event.
A rejection is logged differently from an infrastructure error, because a
target the model keeps refusing holds its active slot and stops recording
anything further, and that is worth seeing.
* feat(git): migrate pre-existing Blueprints into the revision state
Blueprints that predate the model are brought in at boot: an application, an
intent describing what the Blueprint currently asks for, and a candidate marked
as coming from the legacy inline record.
None of it is an acknowledgement. The Blueprint revision is carried for display
and nothing else, because a revision number is not evidence that any node is
running what it names, and recording it as agreement would report a fleet as
converged on an intent nobody verified. No targets are created for the same
reason, and the candidate's required set is empty: migration records what is
asked for, never which nodes currently satisfy it.
An approval authorizes the intent it was given for, so a Blueprint that was
never approved, or edited since, is recorded as needing reapproval rather than
left blank. Blank would read the same as an approval that is merely absent.
Replay is decided by the same scope, schema version, and fingerprint the Git
migration uses, and a Blueprint the new path already described is skipped
outright, since its rows were written with proof this pass does not have.
Blueprints migrate in their own pass. Coupling them to the Git migration would
let one unreadable Git stack keep every Blueprint outside the model.
* fix(git): contain the blueprint stack-directory probe at its call
The ownership probe resolved a path from the blueprint name and checked it
through the shared helper, which the security scan does not credit as a
barrier. The check now sits with the call it protects, matching the form used
elsewhere for the same reason.
The path was already validated, so this changes no behaviour. It surfaced now
because an earlier commit in this branch rewrote the file's line endings, which
made every line read as new and brought a pre-existing call into the scan's
changed-code window.
* feat(git): read GitOps history and carry revision state on source reads
Adds the instance-local history API and the additive revision fields the
source reads have been missing.
`GET /api/git-sources` and `GET /api/stacks/:stackName/git-source` now
carry `gitopsRevision` and `stackResourcePresent`. Only the instance that
owns the files can say whether a stack directory is really there, so that
answer travels with the response rather than being inferred by a reader
that has never seen the filesystem. It is read through the strict stack
listing: the lenient one answers a failed directory read with an empty
list, which here would read as every stack having vanished and would turn
an unreadable disk into an empty audit trail.
Two history routes land with them: `GET /api/git-sources/history` across
stacks, and `GET /api/stacks/:stackName/git-source/history` for one. Both
page on `(created_at, id)` so rows written inside the same millisecond
survive a page boundary, and both bound how far one request will scan. A
malformed cursor or an unusable filter value is refused rather than
ignored, because answering "show me the failures" with everything reads
as an answer instead of a non-answer.
Authorization is decided per row. A row reduces to a stack read only when
it names a stack, its application is live, and the stack is present on
disk; anything unprovable stays with Admin. The cross-stack routes
classify every row, while the per-stack route is authorized whole by
name, so reading a stack shows its full history including entries from an
earlier application. That distinction is carried in the scope type rather
than a flag, so skipping the row classifier without pinning the query to
the authorized stack cannot be expressed.
History entries record the fields each transition moved rather than a
whole revision, so lifecycle for the classifier comes from the owning
application row. That keeps authorization off the recorded payload
entirely: an entry whose detail cannot be read still returns its identity,
stage, and outcome with a stated limitation, and the decode failure is
logged rather than passed over in silence.
* feat(git): let auditors read GitOps history they cannot tie to a stack
A history entry that cannot be tied to a readable stack fell to Admin,
which left the auditor role seeing exactly what a viewer sees on the one
surface it exists to read.
History entries are an audit trail: insert-only, and recording the actor,
trigger, stage, and outcome of every transition. The request audit log is
already gated on the audit permission, so an entry whose audience cannot
be narrowed now falls there too. Withholding it protected nothing in any
case, since the request log already records that a Git-source mutation
happened and who made it.
The source collection deliberately keeps its Admin fallback. Those rows
are live Git configuration, not a record of events, and a mandate to
audit does not extend to reading the repository, ref, and credentials
policy of stacks that were deleted or never finished being created.
The fallback is now part of the requirement type rather than a role
comparison, so each surface states which audience it falls back to and
the compiler requires every case to be answered.
* feat(git): correct remote node identities on GitOps reads
A remote instance numbers its own nodes and has never heard of the hub's
numbering, so every node id it reports is a statement in its own
namespace. Read straight through, a hub joining two nodes showed two
different machines as the same node.
Four GETs now take a second hop that buffers the response and corrects
those identities: the git-source list and detail, and both history
routes. Everything else keeps streaming exactly as before, which is the
reason this is a separate hop rather than a mode of the existing one:
logs, downloads, and event streams must never be buffered, and the
enumerated positions rewritten here do not appear in them.
Only JSON numbers are replaced. A null node is preserved, since "no node"
is a fact the remote is entitled to state and inventing one there would
claim a placement that does not exist. Strings, application ids, and
stack names travel untouched.
The hub re-authorizes what comes back. A remote authorized its rows for
the machine account the hub proxies with, which says nothing about the
person behind the request, so every row is classified again against the
signed-in user. To make that possible without the hub holding another
instance's database, each history entry now carries the lifecycle of its
owning application alongside the stack-presence answer already there.
Both are validated fail-closed, and a verdict a remote might volunteer is
ignored: the hub decides, the remote supplies evidence.
Asking one node for another node's history is refused before the hop
rather than forwarded, because the remote would answer about itself and
the page would read as an answer to a question nobody asked. When the
node asked for is the node being talked to, the hub translates that into
a flag the remote resolves to its own default node. Only the hub may set
that flag, and a remote honours it only on a proxied hop.
Every way this hop can end converges on one terminal answer: rewritten,
passed through, too large, undecodable, failed upstream, or abandoned by
the client. A body the hub could not read never borrows the remote's
success status.
* test(git): assert the unlinked git-source response field by field
The detach test compared the whole response to `{ linked: false }`, which
stopped holding once that route started carrying the additive revision
fields. The equivalent backend route test was updated when those fields
landed; this one was missed.
Asserted field by field rather than loosened to a partial match, so the
two new fields are actually checked: a detached stack has no application
to project while its directory is still on disk.
* feat(git): carry GitOps revisions on blueprint, node, and drift responses
Completes the additive JSON half of the revision-state read contract, so the
Blueprint, node, and drift surfaces report GitOps state through the same
projection the Git-source routes already return instead of leaving it
unreadable.
Blueprint list, detail, create, update, and pin carry gitopsRevision. Node-label
add, cordon, uncordon, and node delete carry gitopsRevisions, ordered by
blueprint id and covering only the Blueprints the mutation actually moved: an
edit no selector reacts to reports an empty list rather than invalidating the
whole catalog. Node deletion reads its Blueprint owners inside the deletion
transaction and before the tombstone, which is the last moment a target row can
be traced back to an application, and an orphaned target is logged rather than
being silently indistinguishable from a Direct one. Both DELETEs that answered
204 still answer 204.
The drift GET and re-check gain the same field alongside the existing ledger,
which is untouched; no GitOps class is written into stack_drift_findings.
Reads and mutations treat a projection fault differently, on purpose. On a read
the revision is part of the answer, so a fault surfaces. On a mutation the write
has already committed, so the revision is decoration and is built defensively:
letting it throw would answer a successful cordon or a completed node deletion
with a 500, sending the operator to retry a hard delete that already happened
and be told the node does not exist. It degrades to an empty list and logs.
Because both drift routes answer with that projection, the identity hop now
intercepts them too, matching its route table per method instead of rejecting
every non-GET. Left as they were, one object would carry the hub's node
numbering or the remote's depending only on which route asked for it. The
allowlist names a single mutation rather than opening a verb, and the hub's
collection filter still keys off the git-source paths alone, so drift payloads
are rewritten without being re-authorized: they are per-stack and already
authorized by name before the hop.
Tests cover the exact shapes, the preserved 204s, revision ordering, the
empty-list cases, node deletion reporting its tombstoned Blueprints and
surviving a projection failure, and the proxy's new route matching, drift
rewrite, and filter exclusion.
* fix(git): resolve every application a GitOps surface can own
Three gaps found while building the additive JSON, all the same shape: an id or
a sentinel stood in for a row, so "there is nothing here" and "the thing that
should be here is unreachable" collapsed into one answer nobody could tell
apart.
A Blueprint application is stored with stack_name NULL, so no lookup by stack
name could reach it, while the reconciler materializes every Blueprint as a
stack directory of that name. A stack's Drift tab and Git panel therefore
reported no GitOps at all for a stack GitOps was actively managing, while the
Blueprint page reported a live application for the same thing. Stack surfaces
now bridge the two through the deployment row for the asking node. The
deployment row is what makes it safe: Blueprint and stack names share one
namespace, so matching on name alone would let a Blueprint claim an unrelated
stack of the same name on a node it never targeted.
A retired application was unreachable for the same reason, because every entry
point filtered to the live rows. Tombstoning deliberately keeps the configured
identity and SHA pointers as frozen facts so the projection can still say what
an application was, and the source deriver has a not_live status waiting for
exactly that, but nothing could hand it one. A detached Blueprint or stack now
reports what it was instead of reading as one that never existed.
The not-applicable projection was typed so it could not carry a reason, which
made a row that vanished between resolving it and re-reading it by id identical
to a stack the model was never asked about. That one field widens to hold
limitations, and the missing case now says so with the id as evidence. Every
other missing-row path in the deriver already worked this way.
Also logs, rather than silently skipping, a live application whose blueprint row
is gone: with cascade off that is a real integrity fault, the placement did
move, and the response would otherwise report that nothing had.
Cordon and uncordon are deliberately left reporting nothing. A cordon governs
whether new placements may be made, not what a Blueprint asks for, and the
reconciler applies it only to new placements, so revising intent for it would
invalidate acknowledgements fleet-wide over a change that evicts nothing. Two
comments implied otherwise and now state the contract.
* fix(git): keep stack-state resolution out of the Git-source read path
Review of the previous commit found the fallback it added had reached further
than intended. The resolver is now split by the question it answers.
projectStackRevision answers "what Git source is attached to this stack" and
stays Direct-only. projectManagedStackRevision answers "what manages this stack"
and is what the drift routes use. The Git-source routes must not be answered
with another application's identity, and there was a second reason: the row
classifier takes the stack name from the Git-source row but the lifecycle from
whatever the projection resolved, so a Blueprint application arriving there
would have turned an Admin-only row into one any stack grant could read. The
Blueprint that unlocked it could be the very one holding name_conflict because
of that stack.
The Blueprint-to-stack bridge also needed the right predicate rather than a
present deployment row. name_conflict is written precisely when a stack of that
name already exists on the node and Sencho does not own it, so guarding only
against withdrawn admitted exactly the collision the bridge exists to prevent.
It now requires last_deployed_at, which proves this Blueprint wrote the
directory, and excludes name_conflict and withdrawn: the same predicate the
delete and withdraw paths use. Only a live Blueprint application qualifies,
since a retired one has no claim on the directory.
The detached fallback no longer resolves deleted applications. Detached means
the files are still on disk and still describe that stack; deleted means the
stack is gone, so any directory of that name now belongs to something else and
reporting the old repository and SHA against it would disclose one stack's Git
identity through another's name. The authorization side already refused deleted
rows for that reason, so the two now agree. Ordering breaks ties on rowid rather
than a random UUID, and two partial indexes cover the new lookups, which the
existing live-only unique indexes could not serve.
The not-applicable variant's limitations are readonly, so pushing onto the
shared frozen instance is a compile error again rather than a runtime throw.
Tests cover a Blueprint claiming a stack it never deployed onto and one it hit a
name conflict on, a detached Direct source projecting not_live, and a deleted
one staying unresolved for a reused name. Drift fixtures move to afterEach so a
failing assertion cannot leak state into the next test.
* fix(git): stop the stack resolver falling through proven ownership
A second review round found the resolution chain could answer with the wrong
application, and that part of the previous commit was reading a state nothing
writes.
The Blueprint bridge returned one value for two different facts. Once the
deployment row proved a Blueprint had written the directory, a missing
application row for it still answered "no Blueprint here", so the chain
continued to the detached-Direct lookup. A stack that once had Direct Git, was
detached, and whose directory a Blueprint later took over would then report the
old Direct application's repository, ref, and SHA as that directory's state.
The bridge now separates "not mine" from "mine, but broken": the second logs and
returns its own limitation, so the fault is visible and no unrelated identity is
offered in its place.
A Blueprint projection also carried the whole placement roster into a route
authorized by a grant on one stack name. Targets are now scoped to the node
being asked about, which is both the safe answer and the accurate one, since the
question is what manages this directory here.
The detached-Blueprint lookup, its index, and its fallback are removed. Blueprint
retirement writes deleted, never detached, and the only detached writer is the
Direct detach path, so the getter could never match and the index covered an
empty set. The test that seemed to cover it built the state by hand and was
green against something the product cannot produce; it now drives the real
delete route and asserts what that path actually writes.
The Git-source resolver goes back to live applications only. Detach removes the
source row in the same transaction that tombstones the application, so a source
row beside a detached one is not producible, and the route's row classifier
takes its stack name from the source row but its lifecycle from the projection.
Keeping that resolver narrow is what stops a resolution change from quietly
moving who may read a row. The comment saying so is restored and now explains
why it must stay true.
* refactor(git): share the GitOps application fixture across the route tests
Two test files carried near-identical 58-line GitOpsApplicationRow literals that
differed only in a repo URL nothing asserted on. Both now import one helper.
The drift tests also repeated a Blueprint seed and its activation across four
cases, and re-imported DatabaseService per test; those become local helpers and
one hoisted import, which lets the cleanup hook go synchronous. Node lookup
throws on a missing default instead of casting.
projectBlueprintRevisions is module-private again: its only caller is the
committed-projection wrapper beside it.
A third copy of the same fixture remains in git-source-routes.test.ts. It is
left alone deliberately: it pins timestamps at 1 rather than now, and that file
has ordering-sensitive history assertions that would need auditing first.
* feat(git): mirror the GitOps revision contract on the frontend
Adds the client-side read contract for the GitOps revision projection, with
no consumers yet.
types/gitops.ts is a hand-written mirror of the backend read contract, in the
same convention as the other domain type files: the frontend never imports
backend. The projection's two arms are named separately so a component can
take the live one as a prop without re-narrowing, and the absent arm keeps its
absent keys rather than nulls so reaching for a lifecycle status without
narrowing is a compile error. The limitation code stays an open string, since
the backend adds codes without a schema bump and an exhaustive switch would
silently stop rendering the newest ones.
lib/gitopsState.ts is the one place a facet status becomes words and a colour,
so a state reads the same in a sidebar tooltip, a panel banner and the drift
tab. Both maps are keyed on the closed status unions, so widening the mirror
without naming the new state fails this build.
Every line of copy states the condition the deriver actually tests, which is
not always what the status name suggests. recovery_required is a recovery
already running rather than one that is needed. candidate_ready is reached only
when review is not required, so it is ready to apply, not ready to review.
synced_and_healthy is also reached with the health gate switched off and never
compares the deployed generation against the accepted one, so it claims
neither. never_reconciled means nothing has been accepted; a fetch that
produced no materialization still records its commit.
pendingSourceStatus keys "an update is waiting" on the candidate pointer rather
than the status name, because source_reconcile_required is reachable both from
a stale candidate and from an accepted generation with no candidate at all.
Retirement is excluded first: tombstoning keeps the candidate pointer as a
frozen fact, so a stack detached mid-review still carries one.
absentFault separates the two facts the empty projection carries. Empty
limitations means the model was never asked about this stack and the right
rendering is nothing; a non-empty list means an application that was expected
could not be reached. A live application's limitations are caveats on state
that is being reported, not faults, so they are deliberately excluded.
* feat(git): name the Git source state on the stack and sidebar surfaces
The sidebar indicator and the anatomy source row both read a raw pending
commit pointer, so a candidate blocked by local conflicts, held for review, or
stale against the configuration in force all render identically to one that is
ready to apply. The projection distinguishes them; nothing was reading it.
refreshGitSourcePending now derives each stack's state from the projection and
the pending map carries that state rather than a boolean. Presence in the map
still means exactly what it meant before, because it is keyed on the candidate
pointer rather than the status name: source_reconcile_required is reachable
both from a stale candidate and from an accepted generation with no candidate,
and only the first is something to review. One narrow fallback remains, for a
row with no projection at all: a failed GitOps write is logged and swallowed
while the pending commit still commits, so the flat pointer is the only thing
that can answer there. Wherever a projection exists it is the sole authority.
The sidebar keeps one indicator, in the same slot, at the same size and colour,
with the same position in the priority ladder. Only the tooltip changes, so a
blocked plan now says so instead of reading as an ordinary update. A test
asserts the rendered indicator is identical across states, which is what keeps
the rendered sidebar unchanged.
The anatomy source row keeps its pulsing dot alone for the ordinary case and
adds the state word only for the states the dot cannot express. Inline rather
than in a tooltip, since this is the reason something is stuck.
* feat(git): surface derived GitOps state in the Git source and Drift panels
The Git source panel's pending banner had two things it could say, read from a
raw commit pointer and a single blocked boolean. It now names which of four
states the candidate is actually in, so a plan blocked by local conflicts, one
held for review, and one gone stale against the configuration in force stop
rendering as the same ordinary update. The short commit sha stays.
A new card above it reports an application the projection could not reach. That
case renders as nothing at all today, which is indistinguishable from a stack
the model was never asked about. Empty limitations stays silent, because that
is the ordinary answer for most stacks and a header over an empty block would
be worse than nothing.
The summary block gains one row naming the source state. It is the first place
the panel can show applying, retry scheduled, suspended, recovering, or a
detached source, none of which have a pending commit and so never reach the
banner. Both existing rows are untouched and the last applied commit stays a
display fact.
The revision lives in its own state rather than on the Git source type, because
the PUT that saves this panel answers with a bare source and no revision. A
save drops it instead of rendering a state the write has already moved.
The Drift tab gains a third block below the two it already has. The compose
versus runtime card and the since-last-deploy card keep their exact positions
and copy; the new block answers a different question, which generation this node
was asked to run and whether it got there. A Blueprint-owned stack shows its
target rows and no source card, since a Blueprint application has no Git source
and inventing one would be a claim the model never made. The drift class list
renders against the type and expects no rows, because nothing populates it yet.
* feat(git): declare revision fields on the blueprint and node clients
Types first. The blueprint list, detail, create, update and pin responses all
carry a GitOps revision, and the node label add, cordon, uncordon and delete
responses carry a list of them. None of that was modelled, so the fields were
being dropped silently and the next reader would have taken these for bare
records.
Most of it is declared and deliberately unread, with the reason written where
the type lives. Create, update and pin are followed immediately by a re-read of
the catalog or the detail that carries the same projection, so rendering the
mutation's copy would show the same fact twice with one of them stale. Cordon
and uncordon always answer with an empty list by design, because a cordon
governs whether new placements may be made rather than what a Blueprint asks
for, and the reconciler leaves existing deployments where they are. The comment
on that type is the point: the risk there is a future reader building a
consumer for a list that is never populated.
Two places do report something. Deleting a node and adding a node label both
re-place Blueprints, and both now say how many. The count only, and only when
it is not zero: an empty list means both that nothing moved and that the
projection faulted after the write had already committed, so it can never be
reported as the first.
Blueprint detail gains the same unreachable-application card the Git source
panel has. A Blueprint with no live application row is a fact nothing in the
product could previously express, which is a different problem from having two
ways to say the same thing.
* fix(git): close the gaps review found in the GitOps frontend consumers
Six defects, each verified against the code before being fixed.
The pending-map read assumed every /git-sources row carries a projection. That
route is proxied, so a node predating the model answers rows without one, and
dereferencing it threw inside the loop. The catch swallowed the throw, the map
was never set, and every stack's Git indicator froze at its previous value for
as long as that node was selected. The field is optional now and a row without
one falls through to the same branch as a row with nothing to project.
That fallback also branched on the arm alone, so a projection reporting an
unreachable application plus a stale commit pointer was reported as a candidate
ready to apply. It now excludes a faulted projection: a fault means an
application was expected and could not be read, so the pointer is not evidence
that anything is ready, and naming a state there would be a guess.
Detaching a Git source cleared the source but not the revision, and the pending
card is derived from the revision alone. The panel kept advertising a waiting
commit for a stack Git no longer managed, behind a Review button that silently
did nothing. A read that threw had the same problem across stacks: the panel is
reused, so stack A's state could render under stack B's header.
The panel also lost the flat-pointer fallback the sidebar keeps, so the two
surfaces disagreed: the sidebar would show an indicator and clicking through
showed no card. Both now apply the same rule.
Node delete parsed the success body unguarded. The delete has already committed
at that point, so a malformed body would have reported a completed deletion as
a failure and skipped the refresh, leaving the deleted node on screen.
Tests: five mutations that previously survived now fail. The zero-pixel claim
compared the wrong element and passed when the indicator vanished entirely; the
stale-read test asserted during the loading window, when the body is skeletons
regardless; the sidebar passthrough test proved only that an indicator existed,
not that the state reached it. Added coverage for the drift row, an unknown
node, the not-applicable source guard, the save clearing, the Review button, and
a live application carrying a caveat, which must not read as a fault.
The fixture set drops two source statuses whose identity defaults describe a
state they cannot be in, and gains a drift-item builder.
* refactor(git): share the GitOps fault card and live-facet derivation
The unreachable-application card was built inline on three surfaces, and the
copies had already drifted: the Git source panel used one icon while the Drift
tab and the Blueprint sheet used another, so the same failure rendered two
different ways. It is one component now, which also owns the state key and the
test id those surfaces assert on.
Deriving the live source facet was likewise repeated, along with the two
semantic decisions behind it: the absent arm has no facets, and a source facet
of not applicable means a Blueprint owns the stack rather than that something
is missing. Both now live next to pendingSourceStatus, which already encoded
the same exclusion.
The Git source panel's seven derived values collapse to three, with the
pending-commit rule extracted to a named function so the four cases read
straight through instead of as nested ternaries. Behaviour is unchanged in
every case.
The two flat-pointer fallbacks are deliberately not unified: they look alike
but the panel also treats a live application whose source facet is not
applicable as unanswered, and merging the predicates would change what that
case renders.
* feat(git): count and announce committed GitOps transitions
Every history row that is actually inserted now produces one metric
increment and one state-invalidate event, so the surfaces that read GitOps
state hear about a change instead of waiting for the next poll.
Announcement is buffered and drained on a macrotask rather than fired
inline. better-sqlite3 is synchronous, so waiting for the macrotask puts
the drain after the transaction that wrote the row, and after any outer
transaction wrapping it, without having to detect which nesting depth it
is in. The drain confirms each row is still present before announcing it,
so a rolled-back transaction says nothing on its own, and the insert
declines to queue a dedupe replay, so a retry says nothing either. The
broadcaster is injected at startup rather than imported, keeping the
GitOps layer free of a cycle back into the notification stack.
Counters are process-local and in-memory, and their keyspace is finite by
construction: history stages are now a closed union that the build
enforces, and outcomes were already a closed set. The payload names no
stack, node, repository, or actor, because a counter carrying those would
be an audit trail with no retention rules and no per-row authorization,
which is what the history routes provide. GET /api/gitops-metrics is
Admin-only and instance-local, so selecting a node answers with that
node's counters.
On the client, a gitops-scoped invalidate refreshes the derived state
through a 250ms trailing window, matching the existing stack refresh. One
operation commits several transitions in a row and a first-boot migration
commits a great many, so refetching per event would thrash the API for a
picture that only settles at the end.
* feat(git): show GitOps source state on the stack dashboards
The dashboards list every stack on a node, which is where a fleet-wide
reading of Git state is most useful and where, until now, a Git-backed
stack looked exactly like a local one. Each row that the model has
something to say about carries a chip naming its source state.
The chip reads from the same status vocabulary the panels and the sidebar
use, so a stack cannot be "pending update" in one place and something else
in another. It sits beside the stack name rather than replacing the source
column: the column says where the files come from, the chip says what
GitOps makes of them, and those are different facts. The label is a word
and the title is a whole sentence, so the state never rests on colour
alone; the phone rows keep the word on screen, since touch has no hover.
The join is by stack name, which is what the dashboards have, and its
source is the same proxied route the sidebar reads. A row without a
revision, a Blueprint-owned application, and a projection fault all leave
the stack unbadged rather than inventing a state for it. State arrives by
announcement rather than by poll, since it only moves on a transition, and
a node switch blanks the map first: stack names repeat across nodes, so a
slow answer for the node just left is discarded rather than allowed to
label the wrong stacks.
* feat(git): say which part of a GitOps state could not be proven
Eighteen conditions can qualify a live projection: a manifest that does
not match the branch configured now, an approval that could not be
restored after a recovery, an artifact record that has gone. Every one was
already recorded and none of them reached the operator, so a state with a
hole in it read exactly like one without.
Each now has operator wording under the state it qualifies, checked
against the site that emits it rather than against the code's name, since
several names describe something narrower or wider than the condition
actually tested. The stored messages stay out of the UI: they are written
for a log reader, and some are raw decoder errors.
The presentation is deliberately quiet. A caveat is not a failure: the
state above it is real, and one piece of evidence behind it is missing, so
the reader learns which part to distrust without being told the whole
thing is broken. Faults keep their own card, because those replace the
state rather than qualifying it, and the two arms are read through
separate helpers so neither can be rendered as the other.
A code with no wording names itself rather than vanishing, so a node
running ahead of this build degrades to something honest instead of
reporting full confidence in a state its own backend flagged.
* fix(git): record a stateful first placement and keep a half-built Blueprint editable
Two faults that today's product cannot reach, and that the Git-backed
Blueprint mode would reach immediately.
The live-application lookup answers with an application that is active or
still being created, because its other callers ask whether the Blueprint
already holds the live slot, where a half-built row counts. Three
producers passed that answer straight into transitions that accept only an
active application and reject anything else by throwing, inside the
caller's own transaction. A Blueprint edited or pinned while its
application was still being created would have failed with a server error
and rolled the Blueprint write back with it. The producers now narrow to
what they actually require, through one shared predicate, leaving the slot
check honest.
Separately, a stateful placement is held for operator review before
anything is deployed, so no target exists when that hold is recorded, and
the observation was dropped for want of one. The hold left no trace: no
history row, and nothing on the target to say the node had been asked to
hold anything. First contact now creates the target, exactly as the first
deploy does. Nothing else changes with it: no intent, no generation, no
operation, and connectivity stays unset, because a node that has only been
asked to hold something has not been contacted. Observations that follow a
deploy already have a target, so they are unaffected, and a drift or evict
report for a node nothing was ever sent to is still dropped.
* fix(git): keep the Git source panel's state after a save
The save answers with the source row and no revision, so the panel had
nothing to replace its copy with and dropped it. The result was a stack
that had just been reconfigured showing no GitOps state at all until the
panel was reopened, which reads as a stack the model knows nothing about
rather than one whose state has just moved.
It re-reads instead. Keeping the old copy was not an option either: a
material configuration change clears the staged candidate server side, so
the state genuinely has moved and the panel would have gone on offering a
commit that is no longer there. Only the server can say what replaced it.
The alternative was returning the revision on the save itself. That route
is proxied, and the identity hop rewrites node numbering for an enumerated
set of routes, so adding a revision to a response outside that set would
hand back a remote node's numbering unrewritten. Re-reading costs one
request on a low-frequency action and needs no change to the hop.
* docs(git): document what GitOps state means for an operator
A commit SHA says which files Sencho wrote. It does not say whether that
commit was accepted, whether something newer is waiting, or whether an
operation was interrupted halfway, and until now the docs had no words for
any of that.
Git Sources gains a source-state table and a section on what happens when
part of a state could not be proven, since a stack with missing evidence
looks identical to one with complete evidence unless the product says
otherwise. Drift Detection gains the gitops section and explains why it
answers a different question from the two signals above it: those compare
files and containers as they are now, this reports what has been proven
over time. The dashboard page describes the state chip and, importantly,
that it and the row tint are independent, so a healthy row carrying
"pending update" is not a contradiction.
The Blueprint page now separates its revision counter from GitOps state.
The counter labels the spec; it says nothing about whether a given node
has got there, and reading it as fleet truth is the mistake the note
prevents.
The tutorial's verification step names the source state it should show,
so a reader can tell the difference between files written and a commit
accepted.
Audited and left unchanged: health-gated updates and fleet federation.
Neither presents a SHA or a revision as canonical GitOps truth, so neither
needed correcting.
* docs(git): name which saves actually clear a staged commit
The note claimed saving the form always clears the staged commit and moves
the state to reconcile required. Only a change to what gets materialized
does that: the repository, the ref, the compose paths, the project
directory, or the env sync. Changing the token or the apply behavior
leaves a staged commit alone, because neither changes what would be
written, and telling an operator otherwise would have them pull again for
nothing.
* fix(git): say when a committed transition has nobody to announce it to
Two silences worth breaking, both found reviewing the error paths in this
slice.
A server that never installs the event sink still counts every transition
and still writes every history row, so the only symptom is that no client
ever refreshes and the UI is quietly as stale as it was before any of this
existed. It now says so, once rather than per row, since a boot migration
would otherwise fill the log with one fact.
The dashboard's source-state fetch dropped a non-ok response without a
word. A refusal there freezes every badge at its last value, which looks
exactly like a fleet where nothing has changed, so the status code is
logged rather than inferred from badges that stopped moving.
* fix(git): close the defects review found in the announcement slice
Five findings, two of them able to reach an operator.
The event sink was installed one line after the deletion reconcile, which
tombstones applications and targets and awaits inside its own loop. A
drain therefore landed while the sink was still absent, so those boot-time
transitions were counted and never announced, and the warning added for an
unwired sink would have fired on every boot with a prepared deletion
intent: the fastest way to teach an operator to ignore it when it means
something. The sink now precedes every reconcile and migration pass.
The first-placement target was created outside the transition that records
it. The observation runs in its own savepoint and can refuse, and the
caller deliberately lets the deployment commit whatever the record says,
so a refusal left an active target with no generation, no stage and no
history: a placement relationship the model never established, which the
delete path would later tombstone as if it were real. Both writes now
succeed or fail together.
The badge looked its status up in a map closed at compile time, behind two
casts, while the value arrives over a proxy from a node that may run a
newer vocabulary. An unmapped key dereferenced undefined inside a stack
row and would have taken the whole table down with it. The facet and its
status are now a discriminated union, the lookup is optional, and an
unrecognised status renders nothing, which is what the join already does
for a stack it has no state for.
The dashboard's source-state loop derived each row inside the shared
try, so one row in an unexpected shape abandoned the loop and froze every
badge at its last value with nothing on screen to say so. Each row is now
guarded on its own and the count of unreadable ones is logged.
Two tests claimed things they did not check: both asserted a word was on
screen with a matcher that also matches screen-reader-only text, so a
compact badge would have kept them green, and one compared a label by
containment where the shorter label is a prefix of the longer. Both now
assert the visible node by equality. Added coverage for the window event
the dashboard actually refetches on, an unknown status, a row that cannot
be read, node-switch blanking, and a caveat recorded twice. One fixture
built a status pair the deriver cannot emit and was corrected.
Also: the guard narrowing Blueprint producers to an active application
excludes nothing today, because every Blueprint-mode application is
inserted as active and only Direct mode can be creating. The guard stays,
since the getter's slot semantics and the transitions' requirement have
drifted apart once already, but its rationale and its tests now say
plainly that they pin a defensive guard rather than reachable behaviour.
* refactor(git): make three runtime guards visible to the compiler
Type review found the same shape three times: a correct guard whose
necessity was asserted in a comment the type system could not see. This
project does not set noUncheckedIndexedAccess, so each of these read as
dead code to anything that trusts the types, and the guards protect the
behaviours that matter most here.
The limitation copy map typed its values optional, so the fallback for an
unrecognised code is now something the compiler requires rather than
something a comment explains. The badge reads its status through partial
views of the two vocabularies, so a miss is a fact TypeScript derives:
deleting the guard that stops one unknown status taking down a stack list
now fails the build, where before it compiled. Neither needed a cast.
The event payload has to be a type alias rather than an interface, because
only the former gets an implicit index signature and the broadcaster takes
an open envelope. That requirement now fails at the declaration instead of
surfacing in the startup wiring as an unexplained index-signature error.
Also added the reverse of the copy-coverage check: a code retired from the
backend leaving stale wording behind was invisible, since the fallback
only fires for entries that are missing rather than ones that linger.
* fix(git): make the state cards survive an unknown status, and correct the copy
Comment review found the protection added for the dashboard chip was
claimed more widely than it held. Five other surfaces indexed the status
vocabularies directly and passed the result straight to a card that
dereferences it, so the exact input the comments said would take a list
down still threw in the Git source panel, the Drift tab, the sidebar
tooltip and the stack anatomy row. The card now takes an optional state
and renders nothing without one, which also means a new surface cannot
reintroduce the dereference by rendering a card the ordinary way, and the
remaining four read through the lookups.
Copy corrections, each against the site that emits it rather than the
name of the code:
The manifest-identity caveat said the manifest named a different
repository or branch. It also fires when the manifest has no identity
block at all, or identifies another node or another stack, so it now says
it does not identify this stack on this node from the repository
configured now. The last-known-good artifact caveat covered a mismatch but
not the row being gone, which is the other half of the same condition.
Doc corrections: a Blueprint-delivered stack was described as carrying the
dashboard chip and a source state on the Drift tab, and it carries
neither, because a Blueprint has no Git source of its own. Per-node
Blueprint state was attributed to the detail sheet's GitOps section, which
only reports what could not be read or proven; the Deployments table is
where per-node status lives. Clearing a staged commit was said to always
land on reconcile required, but a stack that has never had a commit
accepted lands on never reconciled. The caveat lines were described as
sitting under the state when they render above the form. And the pending
banner was still documented as webhook-only, which this branch changed:
it now appears for any staged commit and its heading is the source state.
Also corrected two claims in the code: the observation branch does not
skip for a Blueprint that migration brought in, because migration creates
no targets, and the only refusal reachable from that call site is a
tombstoned target.
* refactor(git): simplify the announcement slice without moving any behaviour
A simplification sweep over the new code, all of it behaviour-preserving
and re-checked by breaking each guard afterwards.
The drain now separates policy from mechanism: it decides whether a row
survived, counts it, and either warns or hands it to a small announce
helper that owns the envelope and its error handling. The row-to-state
pass in the dashboard hook lifts out of the fetch, so one function does
the request and another does the derivation. The caveat dedupe collapses
to a set, which preserves first-insertion order and so keeps the ordering
its own docstring promises. The badge renders one span with a conditional
class rather than two that differed only by class. The metrics service
drops an internal type that was byte-identical to the one it exports.
One assertion goes: the observation branch tested membership and then
re-asserted the same fact to index the stage map, and a type predicate
lets the compiler carry it instead.
Everything the sweep was told to leave alone is intact: the buffered
drain, the row-existence re-check, the injected sink, the one-shot
warning, the partial-view lookups and their guards, the optional copy map
with its fallback, and the generation counter with its per-row guard.
* fix(git): close audit findings in the GitOps revision model
Five corrections from the pre-merge audit of this branch.
Migration no longer certifies a commit it cannot prove. The manifest read
now carries its resolved commit, and a manifest that names a different
commit than the source row records as applied is refused: the applied
directory materializes one commit while the row names another, so
trusting them together would mint a generation pointing at files that are
not the ones it claims. A manifest that names no commit at all, which is
what adoption writes, is a separate answer rather than a disagreement,
because reporting it as one would name a commit the manifest does not
contain. Both are recorded as evidence and the projection asks for a
fetch.
Create cleanup proves containment against the real filesystem before it
deletes. Every check on these paths was lexical, so a symlink or Windows
junction above the target read as contained while the recursive delete
followed it out of the managed area. Resolution keeps "is not there"
apart from "could not be read": only a genuinely missing path lets the
walk climb to an ancestor, because treating an unreadable one as absent
would infer containment for the single path whose link status could not
be established. Both sides are resolved, so relocating the data directory
onto another volume keeps working, and an area that does not exist at all
is nothing to delete rather than a suspected escape. The marker write
takes the same barrier as the marker delete, so a link cannot be written
through and then refused on the way out, wedging the stack name.
Startup stops while an interrupted create is unresolved, matching the
restore reconcile above it. A create that could not be settled leaves a
stack directory the deploy path cannot tell apart from a finished one, so
the alternative to stopping is letting a scheduler, webhook or operator
act on a half-built stack. Only that blocks: once the staged directories
are gone the create is torn down, and a staging marker nobody could
unlink is reported rather than thrown, so one failed unlink is not the
difference between an instance that boots and one that does not.
Per-stack history no longer exposes a predecessor through a reused stack
name. The grant covers the application holding the name now, which is
what keeps a stack's own entries readable while it is still being
created; every other row on that name is classified per row, so entries
belonging to an application that held the name earlier need the audit
permission, as they already did across stacks. A detached predecessor is
still readable, which is the classifier's own standing decision about
detach rather than a gap here, and is now pinned by a test.
Blueprint observations reach a reader. The reconciler recorded state
review, evict blocked, drifted and correcting against the target and
nothing projected them, so a deployed Blueprint could report itself as
never applied. The runtime facet now projects all four, below the states
a live or failed mutation puts the target in and above the pointer
checks, and any later transition supersedes the observation. The map is
declared total over the stages the reconciler can record, so a stage
added without a projection fails the build rather than silently reading
as never applied again.
* fix(git): confine managed-area cleanup to each stack's own location
Cleanup proved only that a resolved path landed somewhere inside the
managed area. That is satisfied by every other stack and every other node
in it, so a link from one stack's generations directory into another's
passed the check while the delete took a generation belonging to someone
else. The check is now positional: the managed area is resolved once, so
relocating the data directory onto another volume still works, and every
segment below it must be reached without redirection. The create-path
sinks use it on the write as well as the delete, so a claim cannot be
written through a link and then refused by the hardened delete, and the
manifest service's pre-existing deletion sinks (generation pruning,
boot-sweep orphan reaping, detach staging and finalization, and
whole-area removal, including after a restored snapshot) run it before
their recursive deletes too. Each of those sinks also keeps a literal
containment comparison beside the positional check, because static
analysis credits only a comparison at the call and reports the delete
otherwise. A refusal names both paths in the log, because it can hold
the boot gate.
Boot recovery dropped the checkpoint of an application that had left the
creating state without first clearing its staging marker. That left a
claim on the stack name with nothing to retry it, and every later create
for that name was refused by a marker nothing could remove. The settled
branches now share one exit, so the marker ordering holds for all of them.
A detached application's history was readable on a stack grant because
its files are still the stack standing at that name. Nothing in these
tables can prove it still is: a Blueprint successor records the name off
the application row, and a plain Compose stack recreated at the name
leaves no trace at all, so no detection-based allowance can be made
sound. Detach now moves an application's trail to the audit audience
outright, the same answer deleted and creating predecessors get, while
reading the stack itself stays where it was.
* fix(gitops): close four audit findings in revision state and identity proxy
A1: derive runtime and health against target desired_generation_id
- deriveRuntime returns 'applied_not_deployed' when desired != deployed, so a
stale deployment stays deploy-pending instead of reading synced_and_healthy
- deriveHealth judges against desired_generation_id falling back to deployed
when null; control case proves null-desired rows unchanged
- collectRuntimeDrift emits the plan-pinned runtime drift item for exact/
qualified observation mismatches; equal/non-comparable observations emit
nothing; ordering pin keeps deploy question first
A2: stop upstream 304 responses from bypassing hub rewrite and reauthorization
- identity proxy answers no-store on every terminal (rewrite, 204, 304,
generated failures); validators (etag, last-modified, cache-control,
expires, vary) no longer forwarded
- conditional request headers (if-none-match, if-modified-since, if-match,
if-unmodified-since) stripped on the identity-hop branch of the shared
proxyReq handler; streaming hop untouched (optimistic-concurrency file writes
depend on If-Match/If-Unmodified-Since)
- supertest integration test drives the real middleware through a loopback
capture server and asserts if-none-match absent while accept survives
S1: migrate legacy query/userinfo URLs via secret-free path
- parseLegacyRepoUrl strips userinfo, query, fragment instead of refusing;
parseHttpsRepoUrl remains the strict gate for user-driven paths
- migrationDirectSourceIdentity uses the tolerant parser; operational
stack_git_sources.repo_url untouched; fingerprint convergence proven
- trusted-manifest-on-legacy-URL test covers the worst real-world instance
S2: emit runtime artifact drift for current evidence
- top-level drift array carries the seven-class runtime item; frontend
comments updated to reflect backend now emits runtime drift
All regression tests added and passing. CI green (7,374 backend + 2,790
frontend tests; only pre-existing Windows EBUSY teardown failure unrelated).
* test(frontend): fix DriftPanel drift item rendering test
Update test assertions to match the new backend drift item format.
The function now renders artifact_set expected
identities as 'artifact <id> · <qualification>' and runtime_artifact
observed identities as the raw identity string.
* fix(gitops): emit a runtime drift item for desired-versus-deployed mismatch
The projection reported applied_not_deployed with no entry in drift, so
the canonical drift list contradicted the runtime facet it travels with.
collectRuntimeDrift now emits the r27 generation-mismatch item for that
state: desired generation as expected, deployed generation as observed,
ComposeService as owner. The action mirrors what availableActions offers:
deploy, unless an application-level fetch or apply is in flight or a
recovery is in progress, which withhold deploying without removing the
fact of the mismatch. The item clears once the desired generation deploys.
The stale-deployment test pins the exact item shape, its stability across
re-derivation, its convergence removal, and the withheld-action case; the
ordering pin keeps the artifact observation suppressed while the deploy
question stands. Frontend fixture aligns its artifact-mismatch example
with the backend's none action.
* fix(gitops): keep generation drift visible across failure and pause states
The desired-versus-deployed drift item was keyed to the applied_not_deployed
runtime status, but paused, failed, recovering, interrupted, and in-flight
statuses all outrank the pointer comparison in the deriver. A failed deploy
of a new generation over a running older one therefore dropped the drift
report exactly while the old workload was still serving.
The item is now judged from the pointers themselves: known and different
means reported, whatever presentation status the target carries. Retired
targets are excluded because nothing can rebind them, so their surviving
pointer divergence would be permanently unresolvable noise. The action
follows what the payload offers: deploy only when no application-level
operation or recovery withholds it and availableActions lists deploying,
none otherwise.
The failed-redeploy regression test pins the exact item shape after a
pre-mutation deploy failure, its stability across re-derivation, continued
artifact suppression for the replaced workload, and convergence removal;
a second pin keeps retirement silent.
* fix(gitops): complete the available-action legality matrix
The action list transcribed only part of the approved rules. An
interrupted apply offered apply without checking that its recorded
generation was still the current candidate; deploy fired for any target
reading applied_not_deployed regardless of target mode, with no retry
for an interrupted Direct deploy and none of the Blueprint interruption
rules; approve_legacy sat in the type union with no producer, leaving a
pending Inline placement review permanently unactionable through the
projection; and drift items inherited whatever deploy recommendation any
sibling target earned.
Actions now follow the matrix. Deploy is decided per target through one
predicate shared by deriveActions and the drift items: a paused or
failed sibling can no longer inherit another target's legal deploy, a
Blueprint-mode divergence never advertises Direct deployment, an
interrupted Direct deploy retries against the generation still applied,
and an interrupted Blueprint deploy or withdraw repeats only while its
recorded intent revision and rollout candidate still equal what the
application requires, where an absent pair matches because inline
Blueprints carry no candidate until the later-phase producer lands.
Interrupted apply requires the recorded generation to still be the
current candidate and the source not to have been suspended meanwhile.
A reachable Inline placement review now offers approve_legacy.
Six tests pin the matrix: sibling isolation across failed and paused
targets, the Blueprint mode guard, vacuous and matched Blueprint retry
identities plus the superseded case, matching-versus-stale Direct deploy
and apply interruptions, and the legacy review action.
* fix(gitops): withhold apply retry while the candidate is blocked
The interrupted-apply retry checked that the recorded generation still
matched the current candidate and that the source was not suspended, but
a later classification can also block that candidate, and applyStarted
refuses a blocked one outright. The retry is offered only when every
precondition the transition enforces still holds.
* fix(gitops): prove apply preconditions and limit fetch to Direct
The interrupted-apply retry checked identity, suspension, and blockage
but never loaded the candidate generation, so apply could be recommended
for a row that was missing, owned by another application, or built from
a superseded materialization fingerprint; applyStarted refuses all
three, and the positive fixture itself described a state no transition
would accept. The gate now proves existence, ownership, and fingerprint
before offering apply. Its fixture is rebuilt around a transition-legal
candidate whose projected action is executed against applyStarted, with
negatives for every refusal including suspension.
Fetch was offered to any application whose source looked unreconciled,
but the approved rules reserve fetch for live Direct applications; a
Git-backed Blueprint divergence now advertises nothing of the sort.
Controls pin both modes against equivalent source state.
* fix(gitops): fail closed when a candidate row is missing or foreign
Ordinary candidate_ready checked a fingerprint only when the candidate
generation row existed, so a dangling id or one owned by another
application fell through to ready and offered an apply that
applyStarted would refuse on sight. The source now requires the row to
exist under this application before readiness, records a limitation
for the anomaly with its operator copy mirrored in the frontend, and
reports reconcile-required instead. The Blueprint retry test also pins
that a superseded rollout candidate alone, with the intent still
matching, suppresses deployment.
* fix(gitops): fail accepted-generation derivation closed
The acceptance branch compared fingerprint and sha only when the
accepted generation row loaded, so a dangling id or one owned by
another application fell through and reported
application_generation_accepted with no evidence behind it, while also
withholding fetch as the recovery. The source now requires the row to
exist under this application before any comparison, records an
accepted_generation_invalid limitation naming the pointer with its
operator copy and inventory entry mirrored on the frontend, and
reports reconcile-required instead. Five scenarios pin valid, missing,
foreign, fingerprint-mismatched, and sha-mismatched acceptances.
* fix(gitops): enforce canonical state in dismiss, reconcile, and pulls
- Route dismiss-pending through the canonical dismissed transition so a
refusal while an operation is in flight surfaces as 409 instead of
clearing staged state behind the model's back
- Treat tombstoned targets as authoritative in both reconciler decision
surfaces so ticks never redeploy onto a placement the model severed;
an explicit deploy revives the target and records the revival delta
- Keep application-level rows in node-scoped history pages so proxied
hub views do not read as if the application never came into being
- Expose repoIdentity and configuredRef on history items to match the
server-side identity filters
- Stand down when a pull resolves to exactly the live candidate (same
commit, source fingerprint, plan verdict); pulls after an acceptance
still open a fresh staging generation as the apply target
* fix(rbac): classify git-source manifest reads as stack:read
GET /stacks/:name/git-source/manifest had no rule in the hub route
classifier, so proxied requests were refused with 403 before reaching
the remote node even for callers holding stack read access.
* fix(gitops): surface an error toast when dismiss is refused as in-flight
Dismissing a pending Git update while a fetch or apply is still running
now returns 409 OPERATION_IN_FLIGHT, but the frontend handler had no
else branch for a non-2xx response, so the refusal was swallowed with
no operator feedback at all. Add the same error-toast pattern already
used by the sibling apply handler, and order the success toast before
its side effects so a downstream failure cannot invert the outcome.
This commit is contained in:
@@ -83,7 +83,7 @@ afterAll(() => {
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
mockDeployStack.mockReset().mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockReset().mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
mockGetBackupInfo.mockReset().mockResolvedValue({ exists: true, timestamp: Date.now() });
|
||||
mockRestoreStackFiles.mockReset().mockResolvedValue(undefined);
|
||||
mockSnapshotStackFiles.mockReset().mockResolvedValue(async () => {});
|
||||
@@ -96,7 +96,7 @@ afterEach(() => vi.restoreAllMocks());
|
||||
describe('Rollback holds the stack lifecycle lock (H-1)', () => {
|
||||
it('blocks deploy while a rollback is in flight on the same stack', async () => {
|
||||
mockTier('paid');
|
||||
const gate = deferred<{ recoveryId: string | null }>();
|
||||
const gate = deferred<{ recoveryId: string | null; deployedGenerationId: string | null }>();
|
||||
mockDeployStack.mockImplementationOnce(() => gate.promise);
|
||||
|
||||
const rollback = request(app)
|
||||
@@ -113,14 +113,14 @@ describe('Rollback holds the stack lifecycle lock (H-1)', () => {
|
||||
expect(deploy.body.code).toBe('stack_op_in_progress');
|
||||
expect(deploy.body.inProgress.action).toBe('rollback');
|
||||
|
||||
gate.resolve({ recoveryId: null });
|
||||
gate.resolve({ recoveryId: null, deployedGenerationId: null });
|
||||
const rollbackRes = await rollback;
|
||||
expect(rollbackRes.status).toBe(200);
|
||||
});
|
||||
|
||||
it('returns 409 when a rollback lands while a deploy is in flight', async () => {
|
||||
mockTier('paid');
|
||||
const gate = deferred<{ recoveryId: string | null }>();
|
||||
const gate = deferred<{ recoveryId: string | null; deployedGenerationId: string | null }>();
|
||||
mockDeployStack.mockImplementationOnce(() => gate.promise);
|
||||
|
||||
const deploy = request(app)
|
||||
@@ -136,13 +136,13 @@ describe('Rollback holds the stack lifecycle lock (H-1)', () => {
|
||||
expect(rollback.status).toBe(409);
|
||||
expect(rollback.body.inProgress.action).toBe('deploy');
|
||||
|
||||
gate.resolve({ recoveryId: null });
|
||||
gate.resolve({ recoveryId: null, deployedGenerationId: null });
|
||||
await deploy;
|
||||
});
|
||||
|
||||
it('releases the lock after a successful rollback', async () => {
|
||||
mockTier('paid');
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
|
||||
const first = await request(app).post('/api/stacks/web/rollback').set('Cookie', authCookie);
|
||||
expect(first.status).toBe(200);
|
||||
@@ -157,7 +157,7 @@ describe('Rollback holds the stack lifecycle lock (H-1)', () => {
|
||||
const first = await request(app).post('/api/stacks/web/rollback').set('Cookie', authCookie);
|
||||
expect(first.status).toBe(500);
|
||||
|
||||
mockDeployStack.mockResolvedValueOnce({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValueOnce({ recoveryId: null, deployedGenerationId: null });
|
||||
const second = await request(app).post('/api/stacks/web/rollback').set('Cookie', authCookie);
|
||||
expect(second.status).toBe(200);
|
||||
});
|
||||
@@ -166,7 +166,7 @@ describe('Rollback holds the stack lifecycle lock (H-1)', () => {
|
||||
describe('Rollback notifications (M-2)', () => {
|
||||
it('dispatches a success notification when a rollback completes', async () => {
|
||||
mockTier('paid');
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const { NotificationService } = await import('../services/NotificationService');
|
||||
const spy = vi.spyOn(NotificationService.getInstance(), 'dispatchAlert').mockResolvedValue({ persisted: true });
|
||||
|
||||
@@ -214,7 +214,7 @@ describe('Rollback returns 404 when no backup exists', () => {
|
||||
// The 404 is an early return inside the try; the finally must still release
|
||||
// the lock so the stack is not wedged at 409 afterwards.
|
||||
mockGetBackupInfo.mockResolvedValue({ exists: true, timestamp: Date.now() });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const next = await request(app).post('/api/stacks/web/rollback').set('Cookie', authCookie);
|
||||
expect(next.status).toBe(200);
|
||||
});
|
||||
@@ -223,7 +223,7 @@ describe('Rollback returns 404 when no backup exists', () => {
|
||||
describe('Developer Mode logging matrix', () => {
|
||||
it('only emits rollback diagnostic logs when Developer Mode is enabled', async () => {
|
||||
mockTier('paid');
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const { DatabaseService } = await import('../services/DatabaseService');
|
||||
const db = DatabaseService.getInstance();
|
||||
const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
|
||||
@@ -246,7 +246,7 @@ describe('Deploy safety is available on every tier', () => {
|
||||
it('allows rollback on community', async () => {
|
||||
mockTier('community');
|
||||
mockGetBackupInfo.mockResolvedValue({ exists: true, timestamp: 1700000000000 });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const res = await request(app).post('/api/stacks/web/rollback').set('Cookie', authCookie);
|
||||
expect(res.status).toBe(200);
|
||||
expect(mockDeployStack).toHaveBeenCalled();
|
||||
|
||||
@@ -75,7 +75,7 @@ describe('Blueprint compose apply (real filesystem)', () => {
|
||||
const composeContent = 'services:\n web:\n image: traefik:v3\n';
|
||||
const markerContent = JSON.stringify({ blueprintId: 1, revision: 1, lastApplied: Date.now() }, null, 2);
|
||||
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
|
||||
const outcome = await BlueprintService.getInstance().applyLocalUnderLock(
|
||||
nodeId,
|
||||
@@ -120,7 +120,7 @@ describe('Blueprint compose apply (real filesystem)', () => {
|
||||
path.join(stackDir, 'docker-compose.yaml'),
|
||||
path.join(stackDir, 'docker-compose.yml'),
|
||||
);
|
||||
return { recoveryId: null };
|
||||
return { recoveryId: null, deployedGenerationId: null };
|
||||
});
|
||||
|
||||
const outcome = await BlueprintService.getInstance().applyLocalUnderLock(
|
||||
@@ -198,7 +198,7 @@ describe('Blueprint compose apply (real filesystem)', () => {
|
||||
const original = 'services:\n mine:\n image: nginx:alpine\n';
|
||||
await fsPromises.writeFile(path.join(stackDir, 'compose.yaml'), original);
|
||||
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
|
||||
await expect(
|
||||
BlueprintService.getInstance().applyLocalUnderLock(
|
||||
|
||||
@@ -125,6 +125,84 @@ describe('reconcileOne approval gate (real path)', () => {
|
||||
expect(DatabaseService.getInstance().listDeployments(bp.id)).toEqual([]);
|
||||
});
|
||||
|
||||
/** Approve `bp` for placement on `nodeId`, then sever that placement in
|
||||
* the canonical model so every auto-decision path sees it as tombstoned. */
|
||||
async function seedSeveredPlacement(bpId: number, nodeId: number): Promise<void> {
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
const bp = DatabaseService.getInstance().getBlueprint(bpId)!;
|
||||
db.prepare(
|
||||
`UPDATE blueprints SET approval_status = 'approved',
|
||||
approved_intent_fingerprint = ?,
|
||||
approved_blast_json = ?
|
||||
WHERE id = ?`,
|
||||
).run(
|
||||
intentFingerprint(bp),
|
||||
serializeApprovedBlast([{ nodeId, outcome: 'place' as const }]),
|
||||
bpId,
|
||||
);
|
||||
|
||||
const { migrateInlineBlueprints } = await import('../services/gitops/migrate');
|
||||
const { GitOpsStore, emptyTargetRow } = await import('../services/gitops/store');
|
||||
migrateInlineBlueprints();
|
||||
const gitopsApp = GitOpsStore.getInstance().getLiveBlueprintApplication(bpId)!;
|
||||
GitOpsStore.getInstance().upsertTarget({
|
||||
...emptyTargetRow(gitopsApp.id, nodeId, Date.now()),
|
||||
target_status: 'tombstoned',
|
||||
});
|
||||
}
|
||||
|
||||
function seedDeployment(bpId: number, nodeId: number, status: string, appliedRevision: number | null): void {
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
`INSERT INTO blueprint_deployments (blueprint_id, node_id, status, applied_revision, last_deployed_at)
|
||||
VALUES (?, ?, ?, ?, ?)`,
|
||||
).run(bpId, nodeId, status, appliedRevision, Date.now());
|
||||
}
|
||||
|
||||
it('does not auto-place onto a tombstoned target', async () => {
|
||||
// A withdraw (or node delete) severs the placement in the model. The
|
||||
// tick must treat that as authoritative instead of resurrecting the
|
||||
// workload behind the projection's back; only an explicit deploy
|
||||
// re-opens the placement.
|
||||
const node = seedNode();
|
||||
const bp = createBp({ nodeIds: [node.id] });
|
||||
await seedSeveredPlacement(bp.id, node.id);
|
||||
|
||||
const deploySpy = vi.spyOn(BlueprintService.getInstance(), 'deployToNode').mockResolvedValue({ status: 'active' });
|
||||
await BlueprintReconciler.getInstance().reconcileOne(bp.id);
|
||||
|
||||
expect(deploySpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not auto-redeploy a stale revision onto a tombstoned target', async () => {
|
||||
// Severance also blocks the update path: an existing deployment that
|
||||
// lagged behind the blueprint must wait for an explicit deploy, never
|
||||
// catch up on its own while the model says the placement is gone.
|
||||
const node = seedNode();
|
||||
const bp = createBp({ nodeIds: [node.id] });
|
||||
await seedSeveredPlacement(bp.id, node.id);
|
||||
seedDeployment(bp.id, node.id, 'active', bp.revision - 1);
|
||||
|
||||
const deploySpy = vi.spyOn(BlueprintService.getInstance(), 'deployToNode').mockResolvedValue({ status: 'active' });
|
||||
await BlueprintReconciler.getInstance().reconcileOne(bp.id);
|
||||
|
||||
expect(deploySpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not redeploy a failed placement onto a tombstoned target', async () => {
|
||||
// A failed run on a severed placement is evidence of the severance, not
|
||||
// a retry request. Redeploying here would undo the withdraw the model
|
||||
// already recorded.
|
||||
const node = seedNode();
|
||||
const bp = createBp({ nodeIds: [node.id] });
|
||||
await seedSeveredPlacement(bp.id, node.id);
|
||||
seedDeployment(bp.id, node.id, 'failed', bp.revision);
|
||||
|
||||
const deploySpy = vi.spyOn(BlueprintService.getInstance(), 'deployToNode').mockResolvedValue({ status: 'active' });
|
||||
await BlueprintReconciler.getInstance().reconcileOne(bp.id);
|
||||
|
||||
expect(deploySpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not mutate when approval_status is approved but blast JSON is malformed', async () => {
|
||||
const node = seedNode();
|
||||
const bp = createBp({ nodeIds: [node.id] });
|
||||
|
||||
@@ -410,7 +410,7 @@ describe('BlueprintService per-stack lock', () => {
|
||||
vi.spyOn(FileSystemService.prototype, 'createStack').mockResolvedValue(undefined);
|
||||
const writeSpy = vi.spyOn(FileSystemService.prototype, 'writeStackFile').mockResolvedValue(undefined);
|
||||
const cleanupSpy = vi.spyOn(FileSystemService.prototype, 'removeAlternateRootComposeFiles').mockResolvedValue(undefined);
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
|
||||
const outcome = await BlueprintService.getInstance().deployToNode(bp, node);
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ beforeAll(async () => {
|
||||
|
||||
const { ComposeService } = await import('../services/ComposeService');
|
||||
listImagesSpy = vi.spyOn(ComposeService.prototype, 'listStackImages').mockResolvedValue(['nginx:bad']);
|
||||
deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
|
||||
const TrivyService = (await import('../services/TrivyService')).default;
|
||||
const trivy = TrivyService.getInstance();
|
||||
|
||||
@@ -740,7 +740,7 @@ describe('ComposeService - deployStack', () => {
|
||||
const promise = ComposeService.getInstance(1).deployStack('my-stack');
|
||||
await vi.advanceTimersByTimeAsync(3100);
|
||||
|
||||
await expect(promise).resolves.toEqual({ recoveryId: null });
|
||||
await expect(promise).resolves.toEqual({ recoveryId: null, deployedGenerationId: null });
|
||||
expect(mockGetLegacyOrphanContainersByStack).toHaveBeenCalledWith('my-stack');
|
||||
});
|
||||
|
||||
@@ -844,7 +844,7 @@ describe('ComposeService - deployStack', () => {
|
||||
await vi.advanceTimersByTimeAsync(3100);
|
||||
const result = await promise;
|
||||
|
||||
expect(result).toEqual({ recoveryId: 'recovery-1' });
|
||||
expect(result).toEqual({ recoveryId: 'recovery-1', deployedGenerationId: null });
|
||||
expect(mockCaptureCandidate).toHaveBeenCalledWith(expect.objectContaining({
|
||||
stackName: 'my-stack',
|
||||
operationKind: 'deployment',
|
||||
@@ -1221,7 +1221,7 @@ describe('ComposeService - updateStack prune-on-update', () => {
|
||||
|
||||
// The update already succeeded before the prune ran, so a prune failure
|
||||
// must neither reject nor trigger the atomic restore.
|
||||
await expect(promise).resolves.toEqual({ recoveryId: 'recovery-1' });
|
||||
await expect(promise).resolves.toEqual({ recoveryId: 'recovery-1', deployedGenerationId: null });
|
||||
expect(mockRestoreStackFiles).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -1235,7 +1235,7 @@ describe('ComposeService - updateStack prune-on-update', () => {
|
||||
const promise = svc.updateStack('my-stack');
|
||||
await vi.advanceTimersByTimeAsync(3100);
|
||||
|
||||
await expect(promise).resolves.toEqual({ recoveryId: 'recovery-1' });
|
||||
await expect(promise).resolves.toEqual({ recoveryId: 'recovery-1', deployedGenerationId: null });
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -3,23 +3,26 @@
|
||||
* read-only report is reachable on the Community tier (no tier gate). Deep diff
|
||||
* behaviour is covered by drift-detection.test.ts.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll, vi } from 'vitest';
|
||||
import { describe, it, expect, beforeAll, afterAll, afterEach, vi } from 'vitest';
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import request from 'supertest';
|
||||
import jwt from 'jsonwebtoken';
|
||||
import { setupTestDb, cleanupTestDb, TEST_USERNAME, TEST_JWT_SECRET } from './helpers/setupTestDb';
|
||||
import { directApplicationFixture } from './helpers/gitopsFixtures';
|
||||
import DockerController from '../services/DockerController';
|
||||
|
||||
let tmpDir: string;
|
||||
let app: import('express').Express;
|
||||
let authHeader: string;
|
||||
let LicenseService: typeof import('../services/LicenseService').LicenseService;
|
||||
let DatabaseService: typeof import('../services/DatabaseService').DatabaseService;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
({ app } = await import('../index'));
|
||||
({ LicenseService } = await import('../services/LicenseService'));
|
||||
({ DatabaseService } = await import('../services/DatabaseService'));
|
||||
const token = jwt.sign({ username: TEST_USERNAME }, TEST_JWT_SECRET, { expiresIn: '1m' });
|
||||
authHeader = `Bearer ${token}`;
|
||||
});
|
||||
@@ -63,3 +66,197 @@ describe('GET /api/stacks/:stackName/drift', () => {
|
||||
fs.rmSync(stackDir, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('drift payload carries the GitOps revision', () => {
|
||||
const STACK = 'driftgitopstest';
|
||||
|
||||
// Cleanup belongs here, not at the end of each test body. A failing
|
||||
// assertion would otherwise leak a blueprint, a deployment, and an
|
||||
// application into the next test, which reuses this stack name and
|
||||
// asserts not_applicable: one real failure would become two, and the
|
||||
// second would point at innocent code.
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
fs.rmSync(path.join(process.env.COMPOSE_DIR as string, STACK), { recursive: true, force: true });
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
for (const table of ['gitops_applications', 'blueprint_deployments', 'blueprints']) {
|
||||
db.prepare(`DELETE FROM ${table}`).run();
|
||||
}
|
||||
});
|
||||
|
||||
function defaultNodeId(): number {
|
||||
const id = DatabaseService.getInstance().getNodes().find(n => n.is_default)?.id;
|
||||
if (id === undefined) throw new Error('the test database has no default node');
|
||||
return id;
|
||||
}
|
||||
|
||||
/** A Blueprint named after the stack, which is what makes the directory its work. */
|
||||
function seedBlueprint(nodeId: number): import('../services/DatabaseService').Blueprint {
|
||||
return DatabaseService.getInstance().createBlueprint({
|
||||
name: STACK,
|
||||
description: null,
|
||||
compose_content: 'services:\n web:\n image: nginx:1.27\n',
|
||||
selector: { type: 'nodes', ids: [nodeId] },
|
||||
drift_mode: 'suggest',
|
||||
classification: 'stateless',
|
||||
classification_reasons: [],
|
||||
enabled: true,
|
||||
created_by: 'admin',
|
||||
});
|
||||
}
|
||||
|
||||
async function activateBlueprintApplication(blueprintId: number, applicationId: string): Promise<void> {
|
||||
const { GitOpsTransitions } = await import('../services/gitops/transitions');
|
||||
const { blankInlineApplication } = await import('../services/gitops/blueprintProducers');
|
||||
GitOpsTransitions.getInstance().activateInlineBlueprint({
|
||||
application: blankInlineApplication(applicationId, blueprintId, Date.now()),
|
||||
envelope: { operationId: `op-${applicationId}`, actor: 'tester', trigger: 'manual', at: Date.now() },
|
||||
});
|
||||
}
|
||||
|
||||
function stubDockerBoundary(): void {
|
||||
vi.spyOn(DockerController, 'getInstance').mockReturnValue({
|
||||
getDependencySnapshot: vi.fn().mockResolvedValue({ containers: [], networks: [], volumes: [] }),
|
||||
} as unknown as DockerController);
|
||||
}
|
||||
|
||||
function makeStack(): void {
|
||||
const stackDir = path.join(process.env.COMPOSE_DIR as string, STACK);
|
||||
fs.mkdirSync(stackDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(stackDir, 'compose.yaml'), 'services:\n web:\n image: nginx:1.27\n');
|
||||
}
|
||||
|
||||
it('adds gitopsRevision to the GET without disturbing the ledger fields', async () => {
|
||||
makeStack();
|
||||
stubDockerBoundary();
|
||||
|
||||
const res = await request(app).get(`/api/stacks/${STACK}/drift`).set('Authorization', authHeader);
|
||||
expect(res.status).toBe(200);
|
||||
// A stack with no Git source has no application, so the uniform
|
||||
// not-applicable shape is what a reader gets rather than a missing key.
|
||||
expect(res.body.gitopsRevision).toMatchObject({ schemaVersion: 1, targetMode: 'not_applicable' });
|
||||
expect(res.body.gitopsRevision.drift).toEqual([]);
|
||||
// The ledger surface is untouched: this field is additive, not a rewrite.
|
||||
expect(res.body).toMatchObject({ stack: STACK });
|
||||
expect(Array.isArray(res.body.findings)).toBe(true);
|
||||
expect(Array.isArray(res.body.ledger)).toBe(true);
|
||||
expect(res.body.temporal).toBeDefined();
|
||||
});
|
||||
|
||||
it('resolves the Blueprint that owns the stack directory, not just Direct Git', async () => {
|
||||
// A Blueprint application is stored with stack_name NULL, so no lookup by
|
||||
// stack name reaches it, yet the reconciler materializes the Blueprint as a
|
||||
// stack directory of that name. Without the deployment bridge the Drift tab
|
||||
// reports not_applicable for a stack GitOps is actively managing, while the
|
||||
// Blueprint page reports a live application for the very same thing.
|
||||
const nodeId = defaultNodeId();
|
||||
const blueprint = seedBlueprint(nodeId);
|
||||
// last_deployed_at is what proves this Blueprint actually wrote the
|
||||
// directory, which is the predicate the bridge requires.
|
||||
DatabaseService.getInstance().upsertDeployment({
|
||||
blueprint_id: blueprint.id,
|
||||
node_id: nodeId,
|
||||
status: 'active',
|
||||
applied_revision: 1,
|
||||
last_deployed_at: Date.now(),
|
||||
});
|
||||
await activateBlueprintApplication(blueprint.id, 'app-bp-drift');
|
||||
|
||||
makeStack();
|
||||
stubDockerBoundary();
|
||||
const res = await request(app).get(`/api/stacks/${STACK}/drift`).set('Authorization', authHeader);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.gitopsRevision).toMatchObject({
|
||||
applicationId: 'app-bp-drift',
|
||||
targetMode: 'inline_blueprint',
|
||||
blueprintId: blueprint.id,
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses to claim a stack the Blueprint could not deploy onto', async () => {
|
||||
// name_conflict is written precisely when a stack of that name already
|
||||
// exists on the node and Sencho does not own it. A deployment row exists,
|
||||
// so a present-row check would treat it as ownership and hand the unrelated
|
||||
// stack's operator this Blueprint's repository, ref, and SHA pointers: the
|
||||
// exact collision the deployment check is supposed to rule out.
|
||||
const nodeId = defaultNodeId();
|
||||
const blueprint = seedBlueprint(nodeId);
|
||||
DatabaseService.getInstance().upsertDeployment({ blueprint_id: blueprint.id, node_id: nodeId, status: 'name_conflict' });
|
||||
await activateBlueprintApplication(blueprint.id, 'app-bp-conflict');
|
||||
|
||||
makeStack();
|
||||
stubDockerBoundary();
|
||||
const res = await request(app).get(`/api/stacks/${STACK}/drift`).set('Authorization', authHeader);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.gitopsRevision).toMatchObject({ targetMode: 'not_applicable', applicationId: null });
|
||||
});
|
||||
|
||||
it('says a proven Blueprint owner has no application, instead of answering with another one', async () => {
|
||||
// The hazard this pins: a stack that once had Direct Git and was detached,
|
||||
// whose directory a Blueprint later took over, whose application row is
|
||||
// then lost. Falling through the resolution chain would report the old
|
||||
// Direct application's repository, ref, and SHA as this directory's state,
|
||||
// confidently and wrongly.
|
||||
const nodeId = defaultNodeId();
|
||||
const { GitOpsTransitions } = await import('../services/gitops/transitions');
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
|
||||
const stale = directApplicationFixture('app-stale-direct', STACK);
|
||||
tx.activateDirect({
|
||||
application: stale,
|
||||
nodeId,
|
||||
envelope: { operationId: 'op-stale', actor: 'tester', trigger: 'manual', at: Date.now() },
|
||||
});
|
||||
tx.applicationTombstoned(stale.id, 'detached', {
|
||||
operationId: 'op-stale-2', actor: 'tester', trigger: 'manual', at: Date.now(),
|
||||
});
|
||||
|
||||
const blueprint = seedBlueprint(nodeId);
|
||||
// Ownership proven by the deployment row, but no application row exists.
|
||||
DatabaseService.getInstance().upsertDeployment({
|
||||
blueprint_id: blueprint.id,
|
||||
node_id: nodeId,
|
||||
status: 'active',
|
||||
applied_revision: 1,
|
||||
last_deployed_at: Date.now(),
|
||||
});
|
||||
|
||||
makeStack();
|
||||
stubDockerBoundary();
|
||||
const res = await request(app).get(`/api/stacks/${STACK}/drift`).set('Authorization', authHeader);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.gitopsRevision).toMatchObject({ targetMode: 'not_applicable', applicationId: null });
|
||||
// Not the plain sentinel: the fault is named, and the detached Direct
|
||||
// application is nowhere in the answer.
|
||||
expect(res.body.gitopsRevision.limitations).toEqual([
|
||||
expect.objectContaining({ code: 'blueprint_application_missing' }),
|
||||
]);
|
||||
expect(JSON.stringify(res.body.gitopsRevision)).not.toContain('app-stale-direct');
|
||||
});
|
||||
|
||||
it('refuses to claim a stack the Blueprint has never deployed', async () => {
|
||||
// A pending or first-deploy-failed row has nothing of ours on the node
|
||||
// either, so last_deployed_at is what proves the directory is the
|
||||
// Blueprint's work.
|
||||
const nodeId = defaultNodeId();
|
||||
const blueprint = seedBlueprint(nodeId);
|
||||
DatabaseService.getInstance().upsertDeployment({ blueprint_id: blueprint.id, node_id: nodeId, status: 'pending' });
|
||||
await activateBlueprintApplication(blueprint.id, 'app-bp-pending');
|
||||
|
||||
makeStack();
|
||||
stubDockerBoundary();
|
||||
const res = await request(app).get(`/api/stacks/${STACK}/drift`).set('Authorization', authHeader);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.gitopsRevision).toMatchObject({ targetMode: 'not_applicable', applicationId: null });
|
||||
});
|
||||
|
||||
it('adds the same gitopsRevision to the re-check', async () => {
|
||||
makeStack();
|
||||
stubDockerBoundary();
|
||||
|
||||
const res = await request(app).post(`/api/stacks/${STACK}/drift/recheck`).set('Authorization', authHeader);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.gitopsRevision).toMatchObject({ schemaVersion: 1, targetMode: 'not_applicable' });
|
||||
expect(Array.isArray(res.body.ledger)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -298,7 +298,7 @@ describe('Single-stack snapshot restore (behavior lock)', () => {
|
||||
fs.writeFileSync(composePath('corrupt-web'), beforeCompose);
|
||||
fs.writeFileSync(envPath('corrupt-web'), beforeEnv);
|
||||
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const res = await request(app)
|
||||
.post(`/api/fleet/snapshots/${id}/restore`)
|
||||
.set('Cookie', adminCookie)
|
||||
@@ -329,7 +329,7 @@ describe('Single-stack snapshot restore (behavior lock)', () => {
|
||||
fs.writeFileSync(composePath('mixed-web'), beforeCompose);
|
||||
fs.writeFileSync(envPath('mixed-web'), beforeEnv);
|
||||
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const res = await request(app)
|
||||
.post(`/api/fleet/snapshots/${id}/restore`)
|
||||
.set('Cookie', adminCookie)
|
||||
@@ -355,7 +355,7 @@ describe('Single-stack snapshot restore (behavior lock)', () => {
|
||||
const beforeCompose = 'services:\n keep: {}\n';
|
||||
fs.writeFileSync(composePath('delim-web'), beforeCompose);
|
||||
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const res = await request(app)
|
||||
.post(`/api/fleet/snapshots/${id}/restore`)
|
||||
.set('Cookie', adminCookie)
|
||||
@@ -396,7 +396,7 @@ describe('Single-stack snapshot restore (behavior lock)', () => {
|
||||
|
||||
it('redeploys after restore when requested', async () => {
|
||||
vi.spyOn(LicenseService.getInstance(), 'getTier').mockReturnValue('community');
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const db = DatabaseService.getInstance();
|
||||
const id = db.createSnapshot('restore-redeploy', 'admin', 1, 1, '[]', '[]');
|
||||
db.insertSnapshotFiles(id, [
|
||||
@@ -758,7 +758,7 @@ describe('Restore-all', () => {
|
||||
|
||||
it('isolates corrupt decrypt stacks before any mutation with notes and redeploy requested', async () => {
|
||||
vi.spyOn(LicenseService.getInstance(), 'getTier').mockReturnValue('community');
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const db = DatabaseService.getInstance();
|
||||
const id = db.createSnapshot('restore-all-corrupt', 'admin', 1, 2, '[]', '[]');
|
||||
const good = CryptoService.getInstance().encrypt('services:\n app: {}\n');
|
||||
@@ -800,7 +800,7 @@ describe('Restore-all', () => {
|
||||
|
||||
it('isolates delimiter-byte corruption before restore-all mutation', async () => {
|
||||
vi.spyOn(LicenseService.getInstance(), 'getTier').mockReturnValue('community');
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const db = DatabaseService.getInstance();
|
||||
const id = db.createSnapshot('restore-all-delim', 'admin', 1, 2, '[]', '[]');
|
||||
const good = CryptoService.getInstance().encrypt('services:\n app: {}\n');
|
||||
@@ -835,7 +835,7 @@ describe('Restore-all', () => {
|
||||
|
||||
it('redeploys each restored stack when requested', async () => {
|
||||
vi.spyOn(LicenseService.getInstance(), 'getTier').mockReturnValue('community');
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const db = DatabaseService.getInstance();
|
||||
const id = db.createSnapshot('restore-all-redeploy', 'admin', 1, 1, '[]', '[]');
|
||||
db.insertSnapshotFiles(id, [
|
||||
@@ -856,7 +856,7 @@ describe('Restore-all', () => {
|
||||
});
|
||||
|
||||
it('records a policy-blocked redeploy as a per-stack failure and still restores the rest', async () => {
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
vi.spyOn(policyGate, 'assertPolicyGateAllows').mockImplementation(async (stackName: string) => {
|
||||
if (stackName === 'blocked-web') throw new Error('Policy "block-criticals" blocked deploy: 1 image(s) exceed high');
|
||||
});
|
||||
|
||||
@@ -129,6 +129,14 @@ vi.mock('../services/DatabaseService', () => ({
|
||||
setGitSourceLastPlan: mockSetGitSourceLastPlan,
|
||||
addNotificationHistory: mockAddNotificationHistory,
|
||||
getStackProjectEnvFiles: vi.fn().mockReturnValue([]),
|
||||
// The apply path now asks whether this stack has a GitOps application.
|
||||
// These fixtures predate the revision-state model, so the lookup finds
|
||||
// nothing and every GitOps producer stays a no-op, which is exactly the
|
||||
// behavior an install with pre-existing Git stacks gets.
|
||||
getDb: () => ({
|
||||
prepare: () => ({ get: () => undefined, all: () => [], run: () => ({ changes: 0 }) }),
|
||||
transaction: (fn: () => unknown) => () => fn(),
|
||||
}),
|
||||
}),
|
||||
},
|
||||
}));
|
||||
@@ -277,7 +285,7 @@ describe('git-source apply recovery (R1)', () => {
|
||||
|
||||
it('refuses to promote when recovery capture fails', async () => {
|
||||
mockCaptureCandidate.mockRejectedValue(new Error('Exact authored-project rollback coverage is unavailable'));
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
|
||||
const { GitSourceService, GitSourceError } = await import('../services/GitSourceService');
|
||||
const svc = GitSourceService.getInstance();
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
/**
|
||||
* Route-layer tests for the git-source API.
|
||||
*
|
||||
* Covers input-validation and guard behavior that lives in the Express
|
||||
* handlers (not in GitSourceService), specifically:
|
||||
* - HTTPS-only repo URL enforcement
|
||||
* Covers input-validation and guard behavior reachable through the Express
|
||||
* handlers (the URL rules themselves live in services/gitops/repoIdentity.ts,
|
||||
* not in GitSourceService), specifically:
|
||||
* - HTTPS-only repo URL enforcement, including userinfo/query/fragment rejection
|
||||
* - Max-length caps on repo_url / branch / compose_path / env_path / token
|
||||
* - Stack-existence 404 guard on PUT
|
||||
* - 400 on invalid stack names
|
||||
@@ -20,6 +21,70 @@ import { setupTestDb, cleanupTestDb, TEST_USERNAME, TEST_JWT_SECRET } from './he
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { ComposeService } from '../services/ComposeService';
|
||||
import { GitSourceService, GitSourceError } from '../services/GitSourceService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import { insertHistory } from '../services/gitops/history';
|
||||
import type { GitOpsApplicationRow } from '../services/gitops/types';
|
||||
|
||||
/** A minimal live Direct application row for GitOps read-path fixtures. */
|
||||
function directApplicationFixture(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/example/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/example/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yaml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
// ── Hoisted mocks (must come before importing the app) ─────────────────
|
||||
|
||||
@@ -93,6 +158,27 @@ describe('PUT /api/stacks/:stackName/git-source — URL validation', () => {
|
||||
expect(res.body.error).toMatch(/HTTPS/i);
|
||||
});
|
||||
|
||||
it('rejects repo URLs with userinfo, query, or fragment', async () => {
|
||||
const cases = [
|
||||
{ repo_url: 'https://user:pass@github.com/example/repo.git', error: /userinfo/i },
|
||||
{ repo_url: 'https://github.com/example/repo.git?token=1', error: /query/i },
|
||||
{ repo_url: 'https://github.com/example/repo.git#head', error: /fragment/i },
|
||||
];
|
||||
for (const c of cases) {
|
||||
const res = await request(app)
|
||||
.put('/api/stacks/existing-stack/git-source')
|
||||
.set('Authorization', `Bearer ${adminToken()}`)
|
||||
.send({
|
||||
repo_url: c.repo_url,
|
||||
branch: 'main',
|
||||
compose_path: 'compose.yaml',
|
||||
auth_type: 'none',
|
||||
});
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.error).toMatch(c.error);
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects missing repo_url with 400', async () => {
|
||||
const res = await request(app)
|
||||
.put('/api/stacks/existing-stack/git-source')
|
||||
@@ -107,6 +193,28 @@ describe('PUT /api/stacks/:stackName/git-source — URL validation', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('POST /api/git-sources/browse: URL validation', () => {
|
||||
it('rejects non-HTTPS, userinfo, query, and fragment URLs before cloning', async () => {
|
||||
const listRepoTree = vi.spyOn(GitSourceService.getInstance(), 'listRepoTree');
|
||||
const cases = [
|
||||
{ repo_url: 'http://github.com/example/repo.git', error: /HTTPS/i },
|
||||
{ repo_url: 'https://user:pass@github.com/example/repo.git', error: /userinfo/i },
|
||||
{ repo_url: 'https://github.com/example/repo.git?token=1', error: /query/i },
|
||||
{ repo_url: 'https://github.com/example/repo.git#head', error: /fragment/i },
|
||||
];
|
||||
for (const c of cases) {
|
||||
const res = await request(app)
|
||||
.post('/api/git-sources/browse')
|
||||
.set('Authorization', `Bearer ${adminToken()}`)
|
||||
.send({ repo_url: c.repo_url, branch: 'main', auth_type: 'none' });
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.error).toMatch(c.error);
|
||||
}
|
||||
expect(listRepoTree).not.toHaveBeenCalled();
|
||||
listRepoTree.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
describe('PUT /api/stacks/:stackName/git-source — max-length caps', () => {
|
||||
const baseBody = {
|
||||
branch: 'main',
|
||||
@@ -276,7 +384,15 @@ describe('GET /api/stacks/:stackName/git-source', () => {
|
||||
.get('/api/stacks/unlinked-stack/git-source')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body).toEqual({ linked: false });
|
||||
expect(res.body.linked).toBe(false);
|
||||
// The stack is real but carries no Git source, so it has no GitOps
|
||||
// application to project and the directory is still on disk.
|
||||
expect(res.body.stackResourcePresent).toBe(true);
|
||||
expect(res.body.gitopsRevision).toMatchObject({
|
||||
schemaVersion: 1,
|
||||
targetMode: 'not_applicable',
|
||||
applicationId: null,
|
||||
});
|
||||
});
|
||||
|
||||
it('returns 404 when the stack does not exist on the active node', async () => {
|
||||
@@ -479,6 +595,15 @@ describe('POST /api/stacks/from-git', () => {
|
||||
expect(res.body.error).toMatch(/HTTPS/i);
|
||||
});
|
||||
|
||||
it('rejects repo URLs with userinfo, query, or fragment', async () => {
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/from-git')
|
||||
.set('Authorization', `Bearer ${adminToken()}`)
|
||||
.send({ ...validBody, repo_url: 'https://github.com/example/repo.git?token=1' });
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.error).toMatch(/query/i);
|
||||
});
|
||||
|
||||
it('rejects oversized repo_url with 400', async () => {
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/from-git')
|
||||
@@ -1213,3 +1338,371 @@ describe('POST /api/stacks/:stackName/git-source/pull permissions and actor', ()
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('GitOps additive fields and history routes', () => {
|
||||
let viewerCookie: string;
|
||||
let auditorCookie: string;
|
||||
|
||||
async function loginAs(username: string, role: 'viewer' | 'auditor'): Promise<string> {
|
||||
const bcrypt = (await import('bcrypt')).default;
|
||||
const password = `${username}-pass`;
|
||||
DatabaseService.getInstance().addUser({
|
||||
username,
|
||||
password_hash: await bcrypt.hash(password, 1),
|
||||
role,
|
||||
});
|
||||
const login = await request(app).post('/api/auth/login').send({ username, password });
|
||||
const cookies = login.headers['set-cookie'] as string | string[];
|
||||
return Array.isArray(cookies) ? cookies[0] : cookies;
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
viewerCookie = await loginAs('gitops-viewer', 'viewer');
|
||||
auditorCookie = await loginAs('gitops-auditor', 'auditor');
|
||||
});
|
||||
|
||||
function makeStackDir(stackName: string): void {
|
||||
const composeDir = process.env.COMPOSE_DIR!;
|
||||
fs.mkdirSync(path.join(composeDir, stackName), { recursive: true });
|
||||
fs.writeFileSync(path.join(composeDir, stackName, 'compose.yaml'), 'services:\n x:\n image: nginx\n');
|
||||
}
|
||||
|
||||
/** Bring a real Direct application into the store, which also writes its first history row. */
|
||||
function activateApplication(
|
||||
id: string,
|
||||
stackName: string,
|
||||
lifecycleStatus: GitOpsApplicationRow['lifecycle_status'] = 'active',
|
||||
): void {
|
||||
const application: GitOpsApplicationRow = {
|
||||
...directApplicationFixture(id, stackName),
|
||||
lifecycle_status: lifecycleStatus,
|
||||
};
|
||||
GitOpsTransitions.getInstance().activateDirect({
|
||||
application,
|
||||
nodeId: 1,
|
||||
envelope: { operationId: `op-${id}`, actor: 'tester', trigger: 'manual', at: Date.now() },
|
||||
});
|
||||
}
|
||||
|
||||
/** Append one more history row for an existing application. */
|
||||
function recordFetch(applicationId: string, stackName: string, operationId: string, sha: string): void {
|
||||
const application = GitOpsStore.getInstance().getApplication(applicationId)
|
||||
?? directApplicationFixture(applicationId, stackName);
|
||||
insertHistory(DatabaseService.getInstance().getDb(), {
|
||||
application,
|
||||
nodeId: 1,
|
||||
dedupeTarget: 'app',
|
||||
operationId,
|
||||
stage: 'fetched',
|
||||
outcome: 'committed',
|
||||
trigger: 'manual',
|
||||
actor: 'tester',
|
||||
before: { desiredCommitSha: null },
|
||||
after: { desiredCommitSha: sha },
|
||||
commitSha: sha,
|
||||
at: Date.now(),
|
||||
});
|
||||
}
|
||||
|
||||
it('carries gitopsRevision and stackResourcePresent on each git-source row', async () => {
|
||||
makeStackDir('additive-stack');
|
||||
seedGitSource('additive-stack');
|
||||
const res = await request(app)
|
||||
.get('/api/git-sources')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
expect(res.status).toBe(200);
|
||||
const row = res.body.find((r: { stack_name: string }) => r.stack_name === 'additive-stack');
|
||||
expect(row).toBeDefined();
|
||||
expect(row.stackResourcePresent).toBe(true);
|
||||
expect(row.gitopsRevision.schemaVersion).toBe(1);
|
||||
});
|
||||
|
||||
it('withholds a row with no GitOps application from a non-admin', async () => {
|
||||
makeStackDir('unmodelled-stack');
|
||||
seedGitSource('unmodelled-stack');
|
||||
// A viewer holds global stack:read, but a source we cannot tie to a
|
||||
// live application has no lifecycle to prove, so it stays with Admin.
|
||||
const res = await request(app)
|
||||
.get('/api/git-sources')
|
||||
.set('Cookie', viewerCookie);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.map((r: { stack_name: string }) => r.stack_name)).not.toContain('unmodelled-stack');
|
||||
});
|
||||
|
||||
it('projects a live application, not just the not-applicable shape', async () => {
|
||||
makeStackDir('live-app-stack');
|
||||
seedGitSource('live-app-stack');
|
||||
activateApplication('app-live-route', 'live-app-stack');
|
||||
const res = await request(app)
|
||||
.get('/api/git-sources')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
const row = res.body.find((r: { stack_name: string }) => r.stack_name === 'live-app-stack');
|
||||
expect(row.gitopsRevision).toMatchObject({
|
||||
schemaVersion: 1,
|
||||
targetMode: 'direct',
|
||||
applicationId: 'app-live-route',
|
||||
lifecycleStatus: 'active',
|
||||
});
|
||||
expect(row.gitopsRevision.facets).not.toBeNull();
|
||||
});
|
||||
|
||||
it('shows a modelled row to a non-admin holding stack read', async () => {
|
||||
// The deny case alone would pass if the route dropped every row for a
|
||||
// non-admin, so the allow case is what proves the classifier runs.
|
||||
makeStackDir('viewer-visible-stack');
|
||||
seedGitSource('viewer-visible-stack');
|
||||
activateApplication('app-viewer-visible', 'viewer-visible-stack');
|
||||
const res = await request(app)
|
||||
.get('/api/git-sources')
|
||||
.set('Cookie', viewerCookie);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.map((r: { stack_name: string }) => r.stack_name)).toContain('viewer-visible-stack');
|
||||
});
|
||||
|
||||
it('filters cross-stack history per row for a non-admin', async () => {
|
||||
makeStackDir('viewer-hist-stack');
|
||||
activateApplication('app-viewer-hist', 'viewer-hist-stack');
|
||||
// No directory, so this application's rows are unprovable and Admin-only.
|
||||
activateApplication('app-hidden-hist', 'absent-hist-stack');
|
||||
|
||||
const asAdmin = await request(app)
|
||||
.get('/api/git-sources/history?limit=100')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
const adminStacks = asAdmin.body.items.map((i: { stackName: string }) => i.stackName);
|
||||
expect(adminStacks).toContain('viewer-hist-stack');
|
||||
expect(adminStacks).toContain('absent-hist-stack');
|
||||
|
||||
const asViewer = await request(app)
|
||||
.get('/api/git-sources/history?limit=100')
|
||||
.set('Cookie', viewerCookie);
|
||||
const viewerStacks = asViewer.body.items.map((i: { stackName: string }) => i.stackName);
|
||||
expect(viewerStacks).toContain('viewer-hist-stack');
|
||||
expect(viewerStacks).not.toContain('absent-hist-stack');
|
||||
});
|
||||
|
||||
it('shows an auditor the history entries a viewer cannot prove', async () => {
|
||||
// 'absent-hist-stack' has no directory, so its entries cannot be tied
|
||||
// to a readable stack. They are still an audit record, so the audit
|
||||
// permission reaches them where a plain stack grant does not.
|
||||
const asAuditor = await request(app)
|
||||
.get('/api/git-sources/history?limit=100')
|
||||
.set('Cookie', auditorCookie);
|
||||
expect(asAuditor.status).toBe(200);
|
||||
const auditorStacks = asAuditor.body.items.map((i: { stackName: string }) => i.stackName);
|
||||
expect(auditorStacks).toContain('absent-hist-stack');
|
||||
expect(auditorStacks).toContain('viewer-hist-stack');
|
||||
});
|
||||
|
||||
it('does not let the audit permission reach Git configuration', async () => {
|
||||
// The source list is live configuration, not a record of events, so an
|
||||
// auditor sees no more of it than any other non-admin.
|
||||
makeStackDir('auditor-config-stack');
|
||||
seedGitSource('auditor-config-stack');
|
||||
const res = await request(app)
|
||||
.get('/api/git-sources')
|
||||
.set('Cookie', auditorCookie);
|
||||
expect(res.status).toBe(200);
|
||||
// Seeded with no GitOps application, so it stays Admin-only.
|
||||
expect(res.body.map((r: { stack_name: string }) => r.stack_name)).not.toContain('auditor-config-stack');
|
||||
});
|
||||
|
||||
it('advances the cursor past rows the caller may not read', async () => {
|
||||
// The viewer cannot read the absent-stack rows seeded above. Paging
|
||||
// must still move forward, or a narrowly scoped caller re-reads the
|
||||
// same rejected window for ever.
|
||||
const first = await request(app)
|
||||
.get('/api/git-sources/history?limit=1')
|
||||
.set('Cookie', viewerCookie);
|
||||
expect(first.status).toBe(200);
|
||||
expect(first.body.nextCursor).not.toBeNull();
|
||||
|
||||
const second = await request(app)
|
||||
.get(`/api/git-sources/history?limit=1&cursor=${encodeURIComponent(first.body.nextCursor)}`)
|
||||
.set('Cookie', viewerCookie);
|
||||
expect(second.status).toBe(200);
|
||||
const firstIds = first.body.items.map((i: { id: string }) => i.id);
|
||||
const secondIds = second.body.items.map((i: { id: string }) => i.id);
|
||||
expect(secondIds.filter((id: string) => firstIds.includes(id))).toEqual([]);
|
||||
});
|
||||
|
||||
it('hands back a cursor when a page fills and none when the window is spent', async () => {
|
||||
makeStackDir('paging-stack');
|
||||
activateApplication('app-paging', 'paging-stack');
|
||||
recordFetch('app-paging', 'paging-stack', 'op-page-1', 'aaa1111');
|
||||
recordFetch('app-paging', 'paging-stack', 'op-page-2', 'bbb2222');
|
||||
|
||||
const full = await request(app)
|
||||
.get('/api/stacks/paging-stack/git-source/history?limit=2')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
expect(full.body.items).toHaveLength(2);
|
||||
expect(full.body.nextCursor).not.toBeNull();
|
||||
|
||||
const rest = await request(app)
|
||||
.get(`/api/stacks/paging-stack/git-source/history?limit=2&cursor=${encodeURIComponent(full.body.nextCursor)}`)
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
expect(rest.body.items).toHaveLength(1);
|
||||
expect(rest.body.nextCursor).toBeNull();
|
||||
});
|
||||
|
||||
it('keeps a creating stack own history readable through the per-stack route', async () => {
|
||||
// The row classifier sends `creating` to Admin. The per-stack route
|
||||
// authorizes its collection by name instead, which is the whole reason
|
||||
// that distinction exists.
|
||||
makeStackDir('creating-stack');
|
||||
activateApplication('app-creating', 'creating-stack', 'creating');
|
||||
|
||||
recordFetch('app-creating', 'creating-stack', 'op-creating-1', 'creat111');
|
||||
const perStack = await request(app)
|
||||
.get('/api/stacks/creating-stack/git-source/history')
|
||||
.set('Cookie', viewerCookie);
|
||||
expect(perStack.status).toBe(200);
|
||||
// Asserted by identity, not by count. A non-empty page would also be
|
||||
// satisfied by an exemption that had widened to rows it should not
|
||||
// cover, which is the failure this route's scope exists to prevent.
|
||||
expect(perStack.body.items.map((i: { applicationId: string }) => i.applicationId))
|
||||
.toContain('app-creating');
|
||||
expect(perStack.body.items.map((i: { commitSha: string | null }) => i.commitSha))
|
||||
.toContain('creat111');
|
||||
|
||||
const crossStack = await request(app)
|
||||
.get('/api/git-sources/history?limit=100')
|
||||
.set('Cookie', viewerCookie);
|
||||
const stacks = crossStack.body.items.map((i: { stackName: string }) => i.stackName);
|
||||
expect(stacks).not.toContain('creating-stack');
|
||||
});
|
||||
|
||||
it('does not expose a predecessor application through a reused stack name', async () => {
|
||||
// A stack name outlives the applications that hold it. A grant on the
|
||||
// one holding it now says nothing about the repository, actors or
|
||||
// commits of the one that held it before, so those rows stay behind
|
||||
// the audit permission on this route exactly as they do cross-stack.
|
||||
makeStackDir('reused-name');
|
||||
activateApplication('app-reused-old', 'reused-name');
|
||||
recordFetch('app-reused-old', 'reused-name', 'op-reused-old', 'old11111');
|
||||
GitOpsTransitions.getInstance().applicationTombstoned('app-reused-old', 'deleted', {
|
||||
operationId: 'op-reused-old', actor: 'tester', trigger: 'manual', at: Date.now(),
|
||||
});
|
||||
activateApplication('app-reused-new', 'reused-name');
|
||||
recordFetch('app-reused-new', 'reused-name', 'op-reused-new', 'new22222');
|
||||
|
||||
const viewer = await request(app)
|
||||
.get('/api/stacks/reused-name/git-source/history?limit=100')
|
||||
.set('Cookie', viewerCookie);
|
||||
expect(viewer.status).toBe(200);
|
||||
const viewerShas = viewer.body.items.map((i: { commitSha: string | null }) => i.commitSha);
|
||||
expect(viewerShas).toContain('new22222');
|
||||
expect(viewerShas).not.toContain('old11111');
|
||||
|
||||
// The audit trail is not lost, only moved behind the permission that
|
||||
// exists for reading it.
|
||||
const auditor = await request(app)
|
||||
.get('/api/stacks/reused-name/git-source/history?limit=100')
|
||||
.set('Cookie', auditorCookie);
|
||||
expect(auditor.status).toBe(200);
|
||||
const auditorShas = auditor.body.items.map((i: { commitSha: string | null }) => i.commitSha);
|
||||
expect(auditorShas).toContain('old11111');
|
||||
expect(auditorShas).toContain('new22222');
|
||||
});
|
||||
|
||||
it('moves a detached application behind system:audit even with no successor', async () => {
|
||||
// Detach leaves the files on disk, which once justified reading its
|
||||
// trail on a stack grant. A grant covers whatever occupies the name
|
||||
// today, and nothing in these tables can prove the detached
|
||||
// application still does: some successors hide from every lookup this
|
||||
// route could run, so detach joins `deleted` as an audit-only
|
||||
// predecessor.
|
||||
makeStackDir('detached-kept');
|
||||
activateApplication('app-detached-kept', 'detached-kept');
|
||||
recordFetch('app-detached-kept', 'detached-kept', 'op-detached-kept', 'kept1111');
|
||||
GitOpsTransitions.getInstance().applicationTombstoned('app-detached-kept', 'detached', {
|
||||
operationId: 'op-detached-kept', actor: 'tester', trigger: 'manual', at: Date.now(),
|
||||
});
|
||||
|
||||
const viewer = await request(app)
|
||||
.get('/api/stacks/detached-kept/git-source/history?limit=100')
|
||||
.set('Cookie', viewerCookie);
|
||||
expect(viewer.status).toBe(200);
|
||||
const shas = viewer.body.items.map((i: { commitSha: string | null }) => i.commitSha);
|
||||
expect(shas).not.toContain('kept1111');
|
||||
|
||||
// Moved behind the audit permission, not lost.
|
||||
const auditor = await request(app)
|
||||
.get('/api/stacks/detached-kept/git-source/history?limit=100')
|
||||
.set('Cookie', auditorCookie);
|
||||
expect(auditor.status).toBe(200);
|
||||
const auditorShas = auditor.body.items.map((i: { commitSha: string | null }) => i.commitSha);
|
||||
expect(auditorShas).toContain('kept1111');
|
||||
});
|
||||
|
||||
it('keeps a detached predecessor behind system:audit once a successor takes the name', async () => {
|
||||
// The successor makes the reuse visible, but the answer does not depend
|
||||
// on detecting it: a detached trail is audit-only on its own. This pins
|
||||
// that a successor neither restores nor widens what the stack grant
|
||||
// reaches.
|
||||
makeStackDir('reused-detached');
|
||||
activateApplication('app-detached-old', 'reused-detached');
|
||||
recordFetch('app-detached-old', 'reused-detached', 'op-detached-old', 'det11111');
|
||||
GitOpsTransitions.getInstance().applicationTombstoned('app-detached-old', 'detached', {
|
||||
operationId: 'op-detached-old', actor: 'tester', trigger: 'manual', at: Date.now(),
|
||||
});
|
||||
activateApplication('app-detached-new', 'reused-detached');
|
||||
recordFetch('app-detached-new', 'reused-detached', 'op-detached-new', 'det22222');
|
||||
|
||||
const viewer = await request(app)
|
||||
.get('/api/stacks/reused-detached/git-source/history?limit=100')
|
||||
.set('Cookie', viewerCookie);
|
||||
expect(viewer.status).toBe(200);
|
||||
const shas = viewer.body.items.map((i: { commitSha: string | null }) => i.commitSha);
|
||||
expect(shas).toContain('det22222');
|
||||
expect(shas).not.toContain('det11111');
|
||||
|
||||
// Moved behind the audit permission, not lost.
|
||||
const auditor = await request(app)
|
||||
.get('/api/stacks/reused-detached/git-source/history?limit=100')
|
||||
.set('Cookie', auditorCookie);
|
||||
expect(auditor.status).toBe(200);
|
||||
const auditorShas = auditor.body.items.map((i: { commitSha: string | null }) => i.commitSha);
|
||||
expect(auditorShas).toContain('det11111');
|
||||
expect(auditorShas).toContain('det22222');
|
||||
});
|
||||
|
||||
it('rejects a malformed cursor instead of silently restarting', async () => {
|
||||
const res = await request(app)
|
||||
.get('/api/git-sources/history?cursor=123.not-a-uuid')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.error).toMatch(/cursor/i);
|
||||
});
|
||||
|
||||
it('rejects a recognized filter carrying an unusable value', async () => {
|
||||
const outcome = await request(app)
|
||||
.get('/api/git-sources/history?outcome=success')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
expect(outcome.status).toBe(400);
|
||||
expect(outcome.body.error).toMatch(/outcome/i);
|
||||
|
||||
const nodeId = await request(app)
|
||||
.get('/api/git-sources/history?nodeId=abc')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
expect(nodeId.status).toBe(400);
|
||||
expect(nodeId.body.error).toMatch(/nodeId/i);
|
||||
});
|
||||
|
||||
it('rejects an invalid stack name on the per-stack history route', async () => {
|
||||
const res = await request(app)
|
||||
.get('/api/stacks/..%2Fetc/git-source/history')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.error).toMatch(/stack name/i);
|
||||
});
|
||||
|
||||
it('returns an empty page for a stack with no recorded history', async () => {
|
||||
makeStackDir('quiet-stack');
|
||||
const res = await request(app)
|
||||
.get('/api/stacks/quiet-stack/git-source/history')
|
||||
.set('Authorization', `Bearer ${adminToken()}`);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.items).toEqual([]);
|
||||
expect(res.body.nextCursor).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -17,6 +17,14 @@ import fs from 'fs';
|
||||
import path from 'path';
|
||||
import { describe, it, expect, vi, beforeAll, afterAll, beforeEach } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import {
|
||||
buildGenerationRow,
|
||||
directSourceIdentity,
|
||||
newGitOpsId,
|
||||
type DirectSourceConfig,
|
||||
} from '../services/gitops/directApplication';
|
||||
|
||||
// ── Hoisted mocks ──────────────────────────────────────────────────────
|
||||
|
||||
@@ -849,6 +857,98 @@ describe('GitSourceService pending lifecycle', () => {
|
||||
expect(db.getGitSource('pending-stack')?.pending_commit_sha).toBeNull();
|
||||
});
|
||||
|
||||
it('dismissPending clears the canonical candidate and records a dismissed history row', async () => {
|
||||
mockSuccessfulClone();
|
||||
const svc = GitSourceService.getInstance();
|
||||
const db = DatabaseService.getInstance();
|
||||
const stackName = 'dismiss-canonical';
|
||||
await svc.upsert({
|
||||
stackName,
|
||||
repoUrl: 'https://github.com/example/repo.git',
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
db.setGitSourcePending(stackName, 'sha-xxx', 'services: {}', null);
|
||||
const { appId, generationId } = seedDirectCandidate(stackName);
|
||||
expect(GitOpsStore.getInstance().getApplication(appId)?.candidate_generation_id).toBe(generationId);
|
||||
|
||||
svc.dismissPending(stackName, 'operator-1');
|
||||
|
||||
const app = GitOpsStore.getInstance().getApplication(appId)!;
|
||||
expect(app.candidate_generation_id).toBeNull();
|
||||
expect(app.candidate_plan_blocked).toBe(0);
|
||||
expect(app.review_required).toBe(0);
|
||||
expect(db.getGitSource(stackName)?.pending_commit_sha).toBeNull();
|
||||
const stages = (db.getDb().prepare(
|
||||
'SELECT stage, outcome FROM gitops_history WHERE application_id = ? ORDER BY id',
|
||||
).all(appId) as Array<{ stage: string; outcome: string }>).map((r) => `${r.stage}:${r.outcome}`);
|
||||
expect(stages).toContain('dismissed:skipped');
|
||||
});
|
||||
|
||||
it('dismissPending refuses while an operation is in flight and mutates nothing', async () => {
|
||||
mockSuccessfulClone();
|
||||
const svc = GitSourceService.getInstance();
|
||||
const db = DatabaseService.getInstance();
|
||||
const stackName = 'dismiss-in-flight';
|
||||
await svc.upsert({
|
||||
stackName,
|
||||
repoUrl: 'https://github.com/example/repo.git',
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
db.setGitSourcePending(stackName, 'sha-yyy', 'services: {}', null);
|
||||
const { appId, generationId } = seedDirectCandidate(stackName);
|
||||
GitOpsTransitions.getInstance().fetchStarted(appId, testEnvelope());
|
||||
|
||||
let caught: unknown;
|
||||
try {
|
||||
svc.dismissPending(stackName, 'operator-1');
|
||||
} catch (error) {
|
||||
caught = error;
|
||||
}
|
||||
expect(caught).toBeInstanceOf(GitSourceError);
|
||||
if (!(caught instanceof GitSourceError)) throw new Error('expected GitSourceError');
|
||||
expect(caught.code).toBe('OPERATION_IN_FLIGHT');
|
||||
// The refusal is the outcome: neither the model nor the legacy columns move.
|
||||
expect(GitOpsStore.getInstance().getApplication(appId)?.candidate_generation_id).toBe(generationId);
|
||||
expect(db.getGitSource(stackName)?.pending_commit_sha).toBe('sha-yyy');
|
||||
});
|
||||
|
||||
it('dismissPending stays a legacy-only no-op without a canonical application', async () => {
|
||||
mockSuccessfulClone();
|
||||
const svc = GitSourceService.getInstance();
|
||||
const db = DatabaseService.getInstance();
|
||||
const stackName = 'dismiss-legacy-only';
|
||||
await svc.upsert({
|
||||
stackName,
|
||||
repoUrl: 'https://github.com/example/repo.git',
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
db.setGitSourcePending(stackName, 'sha-zzz', 'services: {}', null);
|
||||
|
||||
expect(() => svc.dismissPending(stackName, 'operator-1')).not.toThrow();
|
||||
expect(db.getGitSource(stackName)?.pending_commit_sha).toBeNull();
|
||||
});
|
||||
|
||||
it('clearGitSourceAppliedRevision clears pending plan columns', async () => {
|
||||
mockSuccessfulClone();
|
||||
const svc = GitSourceService.getInstance();
|
||||
@@ -1206,6 +1306,144 @@ describe('GitSourceService.pull', () => {
|
||||
const svc = GitSourceService.getInstance();
|
||||
await expect(svc.pull('does-not-exist')).rejects.toMatchObject({ code: 'GIT_ERROR' });
|
||||
});
|
||||
|
||||
function generationCount(stackName: string): number {
|
||||
const app = GitOpsStore.getInstance().getLiveDirectApplication(stackName)!;
|
||||
return (DatabaseService.getInstance().getDb()
|
||||
.prepare('SELECT COUNT(*) AS n FROM gitops_generations WHERE application_id = ?')
|
||||
.get(app.id) as { n: number }).n;
|
||||
}
|
||||
|
||||
async function createFromGit(stackName: string, sha: string, autoApplyOnWebhook = false): Promise<void> {
|
||||
const svc = GitSourceService.getInstance();
|
||||
mockSuccessfulClone({ compose: 'services:\n web:\n image: nginx\n', sha });
|
||||
const validateSpy = vi.spyOn(svc, 'validateCompose').mockResolvedValue({ ok: true });
|
||||
try {
|
||||
await svc.createStackFromGit({
|
||||
stackName,
|
||||
repoUrl: 'https://github.com/example/repo.git',
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
} finally {
|
||||
validateSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
|
||||
it('an up-to-date pull against an accepted commit opens a fresh staging generation', async () => {
|
||||
// Deliberate counterpart to the dedupe below: once the candidate was
|
||||
// accepted, nothing is staged, and staging again is a new dispatch
|
||||
// cycle that apply needs as its acceptance target.
|
||||
const svc = GitSourceService.getInstance();
|
||||
await createFromGit('pull-after-apply', '1111111111111111111111111111111111111111');
|
||||
const base = generationCount('pull-after-apply');
|
||||
|
||||
await svc.pull('pull-after-apply');
|
||||
expect(generationCount('pull-after-apply')).toBe(base + 1);
|
||||
const app = GitOpsStore.getInstance().getLiveDirectApplication('pull-after-apply')!;
|
||||
expect(app.candidate_generation_id).toBeTruthy();
|
||||
expect(DatabaseService.getInstance().getGitSource('pull-after-apply')?.pending_commit_sha).toBeTruthy();
|
||||
await cleanupStackDir('pull-after-apply');
|
||||
});
|
||||
|
||||
it('repeat pulls of an unapplied update keep one candidate', async () => {
|
||||
const svc = GitSourceService.getInstance();
|
||||
await createFromGit('pull-repeat', '2222222222222222222222222222222222222222');
|
||||
const base = generationCount('pull-repeat');
|
||||
|
||||
const updatedSha = '3333333333333333333333333333333333333333';
|
||||
mockSuccessfulClone({
|
||||
compose: 'services:\n web:\n image: nginx:1.29\n',
|
||||
sha: updatedSha,
|
||||
});
|
||||
await svc.pull('pull-repeat');
|
||||
const stagedId = GitOpsStore.getInstance().getLiveDirectApplication('pull-repeat')!.candidate_generation_id;
|
||||
expect(stagedId).toBeTruthy();
|
||||
expect(generationCount('pull-repeat')).toBe(base + 1);
|
||||
|
||||
mockSuccessfulClone({
|
||||
compose: 'services:\n web:\n image: nginx:1.29\n',
|
||||
sha: updatedSha,
|
||||
});
|
||||
await svc.pull('pull-repeat');
|
||||
expect(generationCount('pull-repeat')).toBe(base + 1);
|
||||
expect(GitOpsStore.getInstance().getLiveDirectApplication('pull-repeat')!.candidate_generation_id).toBe(stagedId);
|
||||
expect(DatabaseService.getInstance().getGitSource('pull-repeat')?.pending_commit_sha).toBe(updatedSha);
|
||||
await cleanupStackDir('pull-repeat');
|
||||
});
|
||||
|
||||
it('a pull whose source fingerprint drifted from the staged candidate mints anew', async () => {
|
||||
const svc = GitSourceService.getInstance();
|
||||
await createFromGit('pull-fp-drift', '4444444444444444444444444444444444444444');
|
||||
const base = generationCount('pull-fp-drift');
|
||||
const updatedSha = '5555555555555555555555555555555555555555';
|
||||
mockSuccessfulClone({
|
||||
compose: 'services:\n web:\n image: nginx:1.29\n',
|
||||
sha: updatedSha,
|
||||
});
|
||||
await svc.pull('pull-fp-drift');
|
||||
const stagedId = GitOpsStore.getInstance().getLiveDirectApplication('pull-fp-drift')!.candidate_generation_id;
|
||||
expect(stagedId).toBeTruthy();
|
||||
|
||||
// Simulates a standing candidate produced under different source
|
||||
// wiring than the configuration in effect now. Commit and plan
|
||||
// verdict are unchanged, but the fingerprint term alone must defeat
|
||||
// equivalence so the candidate never misrepresents what a pull stages.
|
||||
DatabaseService.getInstance().getDb()
|
||||
.prepare('UPDATE gitops_generations SET materialization_fingerprint = ? WHERE id = ?')
|
||||
.run('drifted-fingerprint', stagedId);
|
||||
|
||||
mockSuccessfulClone({
|
||||
compose: 'services:\n web:\n image: nginx:1.29\n',
|
||||
sha: updatedSha,
|
||||
});
|
||||
await svc.pull('pull-fp-drift');
|
||||
expect(generationCount('pull-fp-drift')).toBe(base + 2);
|
||||
expect(GitOpsStore.getInstance().getLiveDirectApplication('pull-fp-drift')!.candidate_generation_id).not.toBe(stagedId);
|
||||
await cleanupStackDir('pull-fp-drift');
|
||||
});
|
||||
|
||||
it('a pull whose plan verdict differs from the staged candidate mints anew', async () => {
|
||||
const svc = GitSourceService.getInstance();
|
||||
await createFromGit('pull-verdict-flip', '6666666666666666666666666666666666666666');
|
||||
const base = generationCount('pull-verdict-flip');
|
||||
const updatedSha = '7777777777777777777777777777777777777777';
|
||||
mockSuccessfulClone({
|
||||
compose: 'services:\n web:\n image: nginx:1.29\n',
|
||||
sha: updatedSha,
|
||||
});
|
||||
await svc.pull('pull-verdict-flip');
|
||||
const stagedId = GitOpsStore.getInstance().getLiveDirectApplication('pull-verdict-flip')!.candidate_generation_id;
|
||||
expect(stagedId).toBeTruthy();
|
||||
const seeded = DatabaseService.getInstance().getDb()
|
||||
.prepare('SELECT plan_blocked FROM gitops_generations WHERE id = ?')
|
||||
.get(stagedId) as { plan_blocked: number };
|
||||
expect(seeded.plan_blocked).toBe(0);
|
||||
|
||||
// The plan is re-evaluated on every pull and can flip without a new
|
||||
// commit, for example when stack policy changes between pulls.
|
||||
// Simulating a candidate staged under the other verdict proves the
|
||||
// verdict term defeats equivalence on its own.
|
||||
DatabaseService.getInstance().getDb()
|
||||
.prepare('UPDATE gitops_generations SET plan_blocked = 1 WHERE id = ?')
|
||||
.run(stagedId);
|
||||
|
||||
mockSuccessfulClone({
|
||||
compose: 'services:\n web:\n image: nginx:1.29\n',
|
||||
sha: updatedSha,
|
||||
});
|
||||
await svc.pull('pull-verdict-flip');
|
||||
expect(generationCount('pull-verdict-flip')).toBe(base + 2);
|
||||
expect(GitOpsStore.getInstance().getLiveDirectApplication('pull-verdict-flip')!.candidate_generation_id).not.toBe(stagedId);
|
||||
await cleanupStackDir('pull-verdict-flip');
|
||||
});
|
||||
});
|
||||
|
||||
describe('GitSourceService.createStackFromGit', () => {
|
||||
@@ -1537,7 +1775,7 @@ describe('GitSourceService.apply', () => {
|
||||
const { ComposeService } = await import('../services/ComposeService');
|
||||
const { HealthGateService } = await import('../services/HealthGateService');
|
||||
const saveSpy = vi.spyOn(FileSystemService.prototype, 'saveStackContent').mockResolvedValue();
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const beginSpy = vi.spyOn(HealthGateService.getInstance(), 'beginStack').mockReturnValue('gate-git');
|
||||
const nodeId = DatabaseService.getInstance().getDefaultNode()!.id!;
|
||||
|
||||
@@ -1548,7 +1786,7 @@ describe('GitSourceService.apply', () => {
|
||||
source: 'git_apply',
|
||||
actor: 'system:git-source',
|
||||
});
|
||||
expect(beginSpy).toHaveBeenCalledWith(nodeId, 'apply-deploy-gate', 'deploy', 'system:git-source');
|
||||
expect(beginSpy).toHaveBeenCalledWith(nodeId, 'apply-deploy-gate', 'deploy', 'system:git-source', { deployedGenerationId: null });
|
||||
expect(mockRecoveryLinkGateOrRetain).toHaveBeenCalledWith('rec-test-1', 'gate-git');
|
||||
} finally {
|
||||
validateSpy.mockRestore();
|
||||
@@ -1648,7 +1886,7 @@ describe('GitSourceService.apply', () => {
|
||||
const TrivyService = (await import('../services/TrivyService')).default;
|
||||
const saveSpy = vi.spyOn(FileSystemService.prototype, 'saveStackContent').mockResolvedValue();
|
||||
const listImagesSpy = vi.spyOn(ComposeService.prototype, 'listStackImages').mockResolvedValue(['nginx:bad']);
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null });
|
||||
const deploySpy = vi.spyOn(ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const trivy = TrivyService.getInstance();
|
||||
const trivyAvailableSpy = vi.spyOn(trivy, 'isTrivyAvailable').mockReturnValue(true);
|
||||
const scanSpy = vi.spyOn(trivy, 'scanImagePreflight').mockResolvedValue({
|
||||
@@ -2646,3 +2884,50 @@ describe('GitSourceService classified plan fingerprint', () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── Canonical dismissal fixtures ───────────────────────────────────────
|
||||
|
||||
function testEnvelope(): { operationId: string; actor: string; trigger: string; at: number } {
|
||||
return { operationId: newGitOpsId(), actor: 'test', trigger: 'test', at: Date.now() };
|
||||
}
|
||||
|
||||
/**
|
||||
* Mint an unblocked candidate for `stackName`, the same shape a pull produces,
|
||||
* without driving a real fetch. Reuses the live application the preceding
|
||||
* `svc.upsert` created; the identity is re-derived from the same configuration
|
||||
* so the generation fingerprint matches the application's.
|
||||
*/
|
||||
function seedDirectCandidate(stackName: string): { appId: string; generationId: string } {
|
||||
const at = Date.now();
|
||||
const config: DirectSourceConfig = {
|
||||
repoUrl: 'https://github.com/example/repo.git',
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
};
|
||||
const identity = directSourceIdentity(config);
|
||||
const app = GitOpsStore.getInstance().getLiveDirectApplication(stackName);
|
||||
if (!app) throw new Error(`no live direct application for ${stackName}`);
|
||||
const appId = app.id;
|
||||
const generationId = newGitOpsId();
|
||||
GitOpsStore.getInstance().insertGeneration(buildGenerationRow({
|
||||
id: generationId,
|
||||
applicationId: appId,
|
||||
commitSha: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
|
||||
identity,
|
||||
configuredRef: 'main',
|
||||
candidateRelPath: 'generations/cand',
|
||||
appliedRelPath: 'applied/1',
|
||||
manifestVersion: 1,
|
||||
expectedInvocation: null,
|
||||
changePlanFingerprint: 'fp-seed',
|
||||
operationId: newGitOpsId(),
|
||||
trigger: 'test',
|
||||
actor: 'test',
|
||||
at,
|
||||
}));
|
||||
GitOpsTransitions.getInstance().candidateReady(appId, generationId, false, testEnvelope());
|
||||
return { appId, generationId };
|
||||
}
|
||||
|
||||
@@ -0,0 +1,393 @@
|
||||
/**
|
||||
* Exact API shapes for the additive GitOps revision fields on the Blueprint,
|
||||
* node-label, and node surfaces.
|
||||
*
|
||||
* Two things are being defended here. The first is that the fields are
|
||||
* genuinely additive: the routes keep their status codes, the two DELETEs stay
|
||||
* 204 with no body, and the pre-existing keys are untouched. The second is that
|
||||
* `gitopsRevisions` reports only what a mutation actually moved. A label or a
|
||||
* cordon that no selector reacts to must answer with an empty list rather than
|
||||
* every Blueprint in the fleet, because a consumer reading that list as "these
|
||||
* changed" would otherwise invalidate the whole catalog over an edit nobody can
|
||||
* observe.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll, beforeEach, vi } from 'vitest';
|
||||
import request from 'supertest';
|
||||
import { setupTestDb, cleanupTestDb, loginAsTestAdmin } from './helpers/setupTestDb';
|
||||
import { directApplicationFixture } from './helpers/gitopsFixtures';
|
||||
|
||||
let tmpDir: string;
|
||||
let app: import('express').Express;
|
||||
let DatabaseService: typeof import('../services/DatabaseService').DatabaseService;
|
||||
let LicenseService: typeof import('../services/LicenseService').LicenseService;
|
||||
let adminCookie: string;
|
||||
let counter = 0;
|
||||
|
||||
/** A fresh non-default node row, so a cordon or delete never touches the default node. */
|
||||
function seedNode(): number {
|
||||
counter += 1;
|
||||
const result = DatabaseService.getInstance().getDb().prepare(
|
||||
`INSERT INTO nodes (name, type, mode, compose_dir, is_default, status, created_at)
|
||||
VALUES (?, 'local', 'proxy', '/tmp/compose', 0, 'online', ?)`,
|
||||
).run(`additive-node-${counter}`, Date.now());
|
||||
return result.lastInsertRowid as number;
|
||||
}
|
||||
|
||||
/** A Blueprint inserted straight into the table, so no GitOps application exists for it. */
|
||||
function seedUnmodelledBlueprint() {
|
||||
counter += 1;
|
||||
return DatabaseService.getInstance().createBlueprint({
|
||||
name: `additive-unmodelled-${counter}`,
|
||||
description: null,
|
||||
compose_content: 'services:\n app:\n image: nginx\n',
|
||||
selector: { type: 'nodes', ids: [] },
|
||||
drift_mode: 'suggest',
|
||||
classification: 'stateless',
|
||||
classification_reasons: [],
|
||||
enabled: true,
|
||||
created_by: 'admin',
|
||||
});
|
||||
}
|
||||
|
||||
/** Create through the route, which is the path that activates an application. */
|
||||
async function createBlueprint(selector: { type: string; ids?: number[]; all?: string[]; any?: string[] }) {
|
||||
counter += 1;
|
||||
const res = await request(app)
|
||||
.post('/api/blueprints')
|
||||
.set('Cookie', adminCookie)
|
||||
.send({
|
||||
name: `additive-bp-${counter}`,
|
||||
compose_content: 'services:\n app:\n image: nginx\n',
|
||||
selector,
|
||||
});
|
||||
expect(res.status).toBe(201);
|
||||
return res;
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
({ DatabaseService } = await import('../services/DatabaseService'));
|
||||
({ LicenseService } = await import('../services/LicenseService'));
|
||||
vi.spyOn(LicenseService.getInstance(), 'getTier').mockReturnValue('paid');
|
||||
({ app } = await import('../index'));
|
||||
adminCookie = await loginAsTestAdmin(app);
|
||||
});
|
||||
|
||||
afterAll(() => cleanupTestDb(tmpDir));
|
||||
|
||||
beforeEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
vi.spyOn(LicenseService.getInstance(), 'getTier').mockReturnValue('paid');
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
for (const table of [
|
||||
'blueprint_deployments', 'gitops_history', 'gitops_target_current', 'gitops_rollout_candidates',
|
||||
'gitops_intent_revisions', 'gitops_applications', 'blueprints', 'node_labels',
|
||||
]) {
|
||||
db.prepare(`DELETE FROM ${table}`).run();
|
||||
}
|
||||
// The default node is the one every stack route resolves against, so only
|
||||
// the seeded ones go.
|
||||
db.prepare('DELETE FROM nodes WHERE is_default = 0').run();
|
||||
});
|
||||
|
||||
describe('Blueprint routes carry gitopsRevision', () => {
|
||||
it('projects the live application the create activated, and reports it identically on list and detail', async () => {
|
||||
const created = await createBlueprint({ type: 'nodes', ids: [] });
|
||||
expect(created.body.gitopsRevision).toMatchObject({
|
||||
schemaVersion: 1,
|
||||
targetMode: 'inline_blueprint',
|
||||
lifecycleStatus: 'active',
|
||||
blueprintId: created.body.id,
|
||||
});
|
||||
const applicationId = created.body.gitopsRevision.applicationId;
|
||||
expect(typeof applicationId).toBe('string');
|
||||
|
||||
// The same application id has to come back from every surface, or two
|
||||
// views of one Blueprint would disagree about which application is live.
|
||||
const detail = await request(app).get(`/api/blueprints/${created.body.id}`).set('Cookie', adminCookie);
|
||||
expect(detail.status).toBe(200);
|
||||
expect(detail.body.gitopsRevision.applicationId).toBe(applicationId);
|
||||
|
||||
const list = await request(app).get('/api/blueprints').set('Cookie', adminCookie);
|
||||
expect(list.status).toBe(200);
|
||||
const row = list.body.find((b: { id: number }) => b.id === created.body.id);
|
||||
expect(row.gitopsRevision.applicationId).toBe(applicationId);
|
||||
});
|
||||
|
||||
it('gives a Blueprint with no application the uniform not-applicable shape', async () => {
|
||||
const bp = seedUnmodelledBlueprint();
|
||||
const detail = await request(app).get(`/api/blueprints/${bp.id}`).set('Cookie', adminCookie);
|
||||
expect(detail.status).toBe(200);
|
||||
// Not an omitted key and not a throw: the catalog needs one shape across
|
||||
// rows whether or not migration has brought a Blueprint into the model.
|
||||
expect(detail.body.gitopsRevision).toMatchObject({
|
||||
schemaVersion: 1,
|
||||
targetMode: 'not_applicable',
|
||||
applicationId: null,
|
||||
facets: null,
|
||||
});
|
||||
});
|
||||
|
||||
it('carries gitopsRevision on update and on pin, and leaves the existing keys alone', async () => {
|
||||
const nodeId = seedNode();
|
||||
const created = await createBlueprint({ type: 'nodes', ids: [nodeId] });
|
||||
|
||||
const updated = await request(app)
|
||||
.put(`/api/blueprints/${created.body.id}`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ description: 'revised' });
|
||||
expect(updated.status).toBe(200);
|
||||
expect(updated.body.description).toBe('revised');
|
||||
expect(updated.body.id).toBe(created.body.id);
|
||||
expect(updated.body.gitopsRevision.applicationId).toBe(created.body.gitopsRevision.applicationId);
|
||||
|
||||
const pinned = await request(app)
|
||||
.put(`/api/blueprints/${created.body.id}/pin`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ nodeId });
|
||||
expect(pinned.status).toBe(200);
|
||||
expect(pinned.body.pinned_node_id).toBe(nodeId);
|
||||
expect(pinned.body.gitopsRevision.applicationId).toBe(created.body.gitopsRevision.applicationId);
|
||||
});
|
||||
|
||||
it('keeps DELETE at 204 with no body', async () => {
|
||||
const created = await createBlueprint({ type: 'nodes', ids: [] });
|
||||
const res = await request(app).delete(`/api/blueprints/${created.body.id}`).set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(204);
|
||||
expect(res.body).toEqual({});
|
||||
expect(res.text).toBeFalsy();
|
||||
});
|
||||
});
|
||||
|
||||
describe('Projection resolution reaches every application that owns a surface', () => {
|
||||
it('retires a deleted Blueprint to the plain not-applicable shape', async () => {
|
||||
// Driven through the real delete route rather than a hand-made
|
||||
// tombstone. Blueprint retirement writes `deleted`, never `detached`,
|
||||
// which is why no detached-Blueprint lookup exists: one would index and
|
||||
// query a state the product cannot produce. Asserting it here keeps
|
||||
// that fact tied to the path that decides it.
|
||||
const created = await createBlueprint({ type: 'nodes', ids: [] });
|
||||
const applicationId = created.body.gitopsRevision.applicationId;
|
||||
const del = await request(app).delete(`/api/blueprints/${created.body.id}`).set('Cookie', adminCookie);
|
||||
expect(del.status).toBe(204);
|
||||
|
||||
const store = (await import('../services/gitops/store')).GitOpsStore.getInstance();
|
||||
expect(store.getApplication(applicationId)?.lifecycle_status).toBe('deleted');
|
||||
expect(store.getLiveBlueprintApplication(created.body.id)).toBeUndefined();
|
||||
|
||||
const { projectBlueprintRevision } = await import('../helpers/gitopsResponse');
|
||||
expect(projectBlueprintRevision(created.body.id)).toMatchObject({ targetMode: 'not_applicable' });
|
||||
});
|
||||
|
||||
it('reports a detached Direct source as not_live, which nothing could reach before', async () => {
|
||||
const store = (await import('../services/gitops/store')).GitOpsStore.getInstance();
|
||||
const tx = (await import('../services/gitops/transitions')).GitOpsTransitions.getInstance();
|
||||
const application = directApplicationFixture('app-direct-detach', 'detached-direct-stack');
|
||||
const nodeId = seedNode();
|
||||
tx.activateDirect({
|
||||
application,
|
||||
nodeId,
|
||||
envelope: { operationId: 'op-direct-detach', actor: 'tester', trigger: 'manual', at: Date.now() },
|
||||
});
|
||||
tx.applicationTombstoned(application.id, 'detached', {
|
||||
operationId: 'op-direct-detach-2', actor: 'tester', trigger: 'manual', at: Date.now(),
|
||||
});
|
||||
expect(store.getLiveDirectApplication('detached-direct-stack')).toBeUndefined();
|
||||
expect(store.getDetachedDirectApplication('detached-direct-stack')?.id).toBe(application.id);
|
||||
|
||||
// The tombstone keeps repository, ref, and SHA pointers as frozen facts
|
||||
// so the projection can still say what was there. Before this lookup
|
||||
// existed, the source deriver's not_live branch had no way to be
|
||||
// reached and a deliberate detach read as "never had Git".
|
||||
const { projectManagedStackRevision, projectStackRevision } = await import('../helpers/gitopsResponse');
|
||||
const projection = projectManagedStackRevision('detached-direct-stack', nodeId);
|
||||
expect(projection).toMatchObject({ applicationId: application.id, lifecycleStatus: 'detached' });
|
||||
if (projection.targetMode === 'not_applicable') throw new Error('expected an application');
|
||||
expect(projection.facets.source).toMatchObject({ status: 'not_live', lifecycleStatus: 'detached' });
|
||||
|
||||
// The Git-source resolver stays live-only, so it cannot feed a detached
|
||||
// lifecycle to the row classifier that decides who may read the row.
|
||||
expect(projectStackRevision('detached-direct-stack')).toMatchObject({ targetMode: 'not_applicable' });
|
||||
});
|
||||
|
||||
it('does not resurrect a deleted application for a stack name that gets reused', async () => {
|
||||
const tx = (await import('../services/gitops/transitions')).GitOpsTransitions.getInstance();
|
||||
const application = directApplicationFixture('app-reuse', 'reused-name-stack');
|
||||
tx.activateDirect({
|
||||
application,
|
||||
nodeId: seedNode(),
|
||||
envelope: { operationId: 'op-reuse', actor: 'tester', trigger: 'manual', at: Date.now() },
|
||||
});
|
||||
tx.applicationTombstoned(application.id, 'deleted', {
|
||||
operationId: 'op-reuse-2', actor: 'tester', trigger: 'manual', at: Date.now(),
|
||||
});
|
||||
|
||||
// Deletion means the stack is gone, so a directory of that name now is
|
||||
// a different stack. Reporting the old repository and SHA against it
|
||||
// would disclose one stack's Git identity through another's name.
|
||||
const { projectManagedStackRevision } = await import('../helpers/gitopsResponse');
|
||||
expect(projectManagedStackRevision('reused-name-stack', 1)).toMatchObject({ targetMode: 'not_applicable' });
|
||||
});
|
||||
|
||||
it('says why when the application it resolved has gone missing', async () => {
|
||||
const created = await createBlueprint({ type: 'nodes', ids: [] });
|
||||
const store = (await import('../services/gitops/store')).GitOpsStore.getInstance();
|
||||
const live = store.getLiveBlueprintApplication(created.body.id);
|
||||
// Resolve the row, then delete it before the projection re-reads it by
|
||||
// id. That is the window the two non-transactional reads leave open.
|
||||
vi.spyOn(store, 'getLiveBlueprintApplication').mockImplementation((id: number) => {
|
||||
DatabaseService.getInstance().getDb()
|
||||
.prepare('DELETE FROM gitops_applications WHERE blueprint_id = ?').run(id);
|
||||
return live;
|
||||
});
|
||||
|
||||
const detail = await request(app).get(`/api/blueprints/${created.body.id}`).set('Cookie', adminCookie);
|
||||
expect(detail.status).toBe(200);
|
||||
expect(detail.body.gitopsRevision.targetMode).toBe('not_applicable');
|
||||
// The distinguishing fact: an unmodelled Blueprint carries no limitation.
|
||||
expect(detail.body.gitopsRevision.limitations).toEqual([
|
||||
expect.objectContaining({ code: 'application_row_missing' }),
|
||||
]);
|
||||
});
|
||||
|
||||
it('leaves an unmodelled Blueprint with no limitation, so the two stay distinguishable', async () => {
|
||||
const bp = seedUnmodelledBlueprint();
|
||||
const detail = await request(app).get(`/api/blueprints/${bp.id}`).set('Cookie', adminCookie);
|
||||
expect(detail.body.gitopsRevision.limitations).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Node-label routes report only the Blueprints a label moved', () => {
|
||||
it('carries gitopsRevisions for a Blueprint whose selector reacts to the label', async () => {
|
||||
const nodeId = seedNode();
|
||||
const created = await createBlueprint({ type: 'labels', all: ['edge'] });
|
||||
|
||||
const res = await request(app)
|
||||
.post(`/api/node-labels/${nodeId}`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ label: 'edge' });
|
||||
expect(res.status).toBe(201);
|
||||
expect(res.body).toMatchObject({ nodeId, label: 'edge' });
|
||||
expect(res.body.gitopsRevisions).toHaveLength(1);
|
||||
expect(res.body.gitopsRevisions[0]).toMatchObject({
|
||||
blueprintId: created.body.id,
|
||||
applicationId: created.body.gitopsRevision.applicationId,
|
||||
});
|
||||
});
|
||||
|
||||
it('reports an empty list for a label no selector mentions', async () => {
|
||||
const nodeId = seedNode();
|
||||
await createBlueprint({ type: 'nodes', ids: [] });
|
||||
|
||||
const res = await request(app)
|
||||
.post(`/api/node-labels/${nodeId}`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ label: 'unrelated' });
|
||||
expect(res.status).toBe(201);
|
||||
expect(res.body.gitopsRevisions).toEqual([]);
|
||||
});
|
||||
|
||||
it('keeps DELETE at 204 with no body', async () => {
|
||||
const nodeId = seedNode();
|
||||
await request(app)
|
||||
.post(`/api/node-labels/${nodeId}`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ label: 'edge' });
|
||||
|
||||
const res = await request(app)
|
||||
.delete(`/api/node-labels/${nodeId}/edge`)
|
||||
.set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(204);
|
||||
expect(res.body).toEqual({});
|
||||
expect(res.text).toBeFalsy();
|
||||
});
|
||||
});
|
||||
|
||||
describe('Node routes carry gitopsRevisions', () => {
|
||||
it('carries the field on cordon, empty because a cordon revises no intent', async () => {
|
||||
const nodeId = seedNode();
|
||||
await createBlueprint({ type: 'nodes', ids: [nodeId] });
|
||||
|
||||
const res = await request(app).post(`/api/nodes/${nodeId}/cordon`).set('Cookie', adminCookie).send({});
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.cordoned).toBe(true);
|
||||
// Deliberately empty, and asserted so the reason is not lost. A cordon
|
||||
// suppresses new placements; it does not change what a Blueprint asks
|
||||
// for, and `listDesiredNodes` reports what is asked for. The set is
|
||||
// therefore identical either side of the write, so nothing is revised
|
||||
// and nothing is reported. The field is still present, so a consumer
|
||||
// reads one shape across every mutation.
|
||||
expect(res.body.gitopsRevisions).toEqual([]);
|
||||
});
|
||||
|
||||
it('carries the field on uncordon', async () => {
|
||||
const nodeId = seedNode();
|
||||
await createBlueprint({ type: 'nodes', ids: [nodeId] });
|
||||
await request(app).post(`/api/nodes/${nodeId}/cordon`).set('Cookie', adminCookie).send({});
|
||||
|
||||
const res = await request(app).post(`/api/nodes/${nodeId}/uncordon`).set('Cookie', adminCookie).send({});
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.cordoned).toBe(false);
|
||||
expect(res.body.gitopsRevisions).toEqual([]);
|
||||
});
|
||||
|
||||
it('orders revisions by blueprintId ascending when a mutation moves several', async () => {
|
||||
const nodeId = seedNode();
|
||||
const first = await createBlueprint({ type: 'labels', all: ['fleet'] });
|
||||
const second = await createBlueprint({ type: 'labels', all: ['fleet'] });
|
||||
|
||||
const res = await request(app)
|
||||
.post(`/api/node-labels/${nodeId}`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ label: 'fleet' });
|
||||
expect(res.status).toBe(201);
|
||||
// Ordering is the contract, not the order the producer happened to visit
|
||||
// the Blueprints in, which is a Map iteration order.
|
||||
const ids = res.body.gitopsRevisions.map((r: { blueprintId: number }) => r.blueprintId);
|
||||
expect(ids).toEqual([first.body.id, second.body.id].sort((a, b) => a - b));
|
||||
});
|
||||
|
||||
it('reports the Blueprints that lost a target when a node is deleted', async () => {
|
||||
const nodeId = seedNode();
|
||||
const bp = await createBlueprint({ type: 'nodes', ids: [nodeId] });
|
||||
const applicationId = bp.body.gitopsRevision.applicationId;
|
||||
// A target has to exist on the node for the deletion to retire one. The
|
||||
// route reads the owners before the tombstone, which is the only moment
|
||||
// the link from target back to Blueprint still exists.
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
`INSERT INTO gitops_target_current (application_id, node_id, target_status, updated_at)
|
||||
VALUES (?, ?, 'active', ?)`,
|
||||
).run(applicationId, nodeId, Date.now());
|
||||
|
||||
const res = await request(app).delete(`/api/nodes/${nodeId}`).set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.success).toBe(true);
|
||||
expect(res.body.gitopsRevisions.map((r: { blueprintId: number }) => r.blueprintId)).toEqual([bp.body.id]);
|
||||
});
|
||||
|
||||
it('still reports a node deletion as successful when the revision projection fails', async () => {
|
||||
const nodeId = seedNode();
|
||||
await createBlueprint({ type: 'nodes', ids: [nodeId] });
|
||||
// The write commits before the decoration is built. If a projection
|
||||
// fault escaped, the operator would be told a hard delete failed and
|
||||
// would retry it, and the retry answers "Node not found": two wrong
|
||||
// answers about an operation that actually succeeded.
|
||||
const store = (await import('../services/gitops/store')).GitOpsStore.getInstance();
|
||||
vi.spyOn(store, 'getLiveBlueprintApplication').mockImplementation(() => {
|
||||
throw new Error('projection exploded');
|
||||
});
|
||||
|
||||
const res = await request(app).delete(`/api/nodes/${nodeId}`).set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body).toMatchObject({ success: true, gitopsRevisions: [] });
|
||||
// And the node really is gone, so the success it reported was true.
|
||||
expect(DatabaseService.getInstance().getNode(nodeId)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('reports an empty list when a deleted node held no Blueprint target', async () => {
|
||||
const nodeId = seedNode();
|
||||
const res = await request(app).delete(`/api/nodes/${nodeId}`).set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body).toMatchObject({ success: true, gitopsRevisions: [] });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,353 @@
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { encodeGitOpsApprovedTargetEffectJson, encodeGitOpsRequiredTargetsJson } from '../services/gitops/json';
|
||||
import type {
|
||||
GitOpsApplicationRow,
|
||||
GitOpsApprovalRow,
|
||||
GitOpsGenerationRow,
|
||||
GitOpsIntentRevisionRow,
|
||||
GitOpsRolloutCandidateRow,
|
||||
} from '../services/gitops/types';
|
||||
|
||||
describe('gitops approvals', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertApplication(directApp('app-a', 'stack-a'));
|
||||
store.insertGeneration(generation('gen-a', 'app-a'));
|
||||
store.insertGeneration(generation('gen-b', 'app-a'));
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('rejects exact kind/authority mismatches at the CHECK floor', async () => {
|
||||
const { DatabaseService } = await import('../services/DatabaseService');
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
const insert = db.prepare(
|
||||
`INSERT INTO gitops_approvals (
|
||||
id, kind, authority, authoritative, application_id, generation_id, intent_revision_id,
|
||||
artifact_set_id, rollout_candidate_id, rollout_generation_id, source_acceptance_ref,
|
||||
placement_approval_ref, required_targets_json, preflight_fingerprint, fingerprint,
|
||||
blast_json, policy_provenance_json, actor, created_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
);
|
||||
expect(() => insert.run(
|
||||
'bad-src-legacy', 'source_acceptance', 'legacy_combined', 1, 'app-a', 'gen-a',
|
||||
null, null, null, null, null, null, null, null, null, null, null, 'tester', 1,
|
||||
)).toThrow();
|
||||
expect(() => insert.run(
|
||||
'bad-src-auth0', 'source_acceptance', 'operator', 0, 'app-a', 'gen-a',
|
||||
null, null, null, null, null, null, null, null, null, null, null, 'tester', 1,
|
||||
)).toThrow();
|
||||
expect(() => insert.run(
|
||||
'bad-legacy-auth1', 'legacy_combined', 'legacy_combined', 1, 'app-a', null,
|
||||
null, null, null, null, null, null, null, null, null, null, null, 'tester', 1,
|
||||
)).toThrow();
|
||||
expect(() => insert.run(
|
||||
'bad-legacy-op', 'legacy_combined', 'operator', 0, 'app-a', null,
|
||||
null, null, null, null, null, null, null, null, null, null, null, 'tester', 1,
|
||||
)).toThrow();
|
||||
});
|
||||
|
||||
it('resolves source acceptance only for the expected generation', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertApproval(sourceAcceptance('acc-a', 'app-a', 'gen-a'));
|
||||
store.insertApproval(sourceAcceptance('acc-b', 'app-a', 'gen-b'));
|
||||
expect(store.resolveApprovalRef('acc-a', {
|
||||
kind: 'source_acceptance',
|
||||
applicationId: 'app-a',
|
||||
generationId: 'gen-a',
|
||||
})?.id).toBe('acc-a');
|
||||
expect(store.resolveApprovalRef('acc-a', {
|
||||
kind: 'source_acceptance',
|
||||
applicationId: 'app-a',
|
||||
generationId: 'gen-b',
|
||||
})).toBeNull();
|
||||
expect(store.newestSourceAcceptanceId('app-a', 'gen-a')).toBe('acc-a');
|
||||
});
|
||||
|
||||
it('validates placement effects against required nodes without set equality', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertIntentRevision(intent('intent-1', 'app-a'));
|
||||
store.insertApproval(placement('place-subset', 'app-a', 'intent-1', [
|
||||
{ nodeId: 2, outcome: 'place' },
|
||||
]));
|
||||
store.insertApproval(placement('place-empty', 'app-a', 'intent-1', []));
|
||||
store.insertApproval(placement('place-remove-extra', 'app-a', 'intent-1', [
|
||||
{ nodeId: 3, outcome: 'remove' },
|
||||
]));
|
||||
const required = [1, 2];
|
||||
expect(store.resolveApprovalRef('place-subset', {
|
||||
kind: 'placement_approval',
|
||||
applicationId: 'app-a',
|
||||
intentRevisionId: 'intent-1',
|
||||
requiredNodeIds: required,
|
||||
})?.id).toBe('place-subset');
|
||||
expect(store.resolveApprovalRef('place-empty', {
|
||||
kind: 'placement_approval',
|
||||
applicationId: 'app-a',
|
||||
intentRevisionId: 'intent-1',
|
||||
requiredNodeIds: required,
|
||||
})?.id).toBe('place-empty');
|
||||
expect(store.resolveApprovalRef('place-remove-extra', {
|
||||
kind: 'placement_approval',
|
||||
applicationId: 'app-a',
|
||||
intentRevisionId: 'intent-1',
|
||||
requiredNodeIds: required,
|
||||
})?.id).toBe('place-remove-extra');
|
||||
store.insertApproval(placement('place-bad-required', 'app-a', 'intent-1', [
|
||||
{ nodeId: 9, outcome: 'place' },
|
||||
]));
|
||||
expect(store.resolveApprovalRef('place-bad-required', {
|
||||
kind: 'placement_approval',
|
||||
applicationId: 'app-a',
|
||||
intentRevisionId: 'intent-1',
|
||||
requiredNodeIds: required,
|
||||
})).toBeNull();
|
||||
store.insertApproval(placement('remove-required', 'app-a', 'intent-1', [
|
||||
{ nodeId: 1, outcome: 'remove' },
|
||||
]));
|
||||
expect(store.resolveApprovalRef('remove-required', {
|
||||
kind: 'placement_approval',
|
||||
applicationId: 'app-a',
|
||||
intentRevisionId: 'intent-1',
|
||||
requiredNodeIds: required,
|
||||
})).toBeNull();
|
||||
});
|
||||
|
||||
it('refuses to persist an approval whose evidence JSON cannot be decoded', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertApplication(directApp('app-badjson', 'badjson-web'));
|
||||
store.insertGeneration(generation('gen-badjson', 'app-badjson'));
|
||||
store.insertIntentRevision(intent('intent-badjson', 'app-badjson'));
|
||||
expect(() => store.insertApproval({
|
||||
...placement('appr-badblast', 'app-badjson', 'intent-badjson', []),
|
||||
generation_id: null,
|
||||
blast_json: '[{"nodeId":2,"outcome":"place"},{"nodeId":1,"outcome":"remove"}]',
|
||||
})).toThrow();
|
||||
expect(() => store.insertApproval({
|
||||
...placement('appr-badtargets', 'app-badjson', 'intent-badjson', []),
|
||||
generation_id: null,
|
||||
required_targets_json: '{"nodeIds":[2,1]}',
|
||||
})).toThrow();
|
||||
expect(store.getApproval('appr-badblast')).toBeUndefined();
|
||||
expect(store.getApproval('appr-badtargets')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('does not treat a CHECK-valid row as proof when expected identity differs', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertIntentRevision(intent('intent-2', 'app-a'));
|
||||
store.insertRolloutCandidate(candidate('cand-1', 'app-a', 'intent-2', 'gen-a'));
|
||||
store.insertApproval(sourceAcceptance('acc-bind-a', 'app-a', 'gen-a'));
|
||||
store.insertApproval(placement('place-bind', 'app-a', 'intent-2', [
|
||||
{ nodeId: 1, outcome: 'place' },
|
||||
]));
|
||||
const fingerprint = 'ab'.repeat(32);
|
||||
store.insertApproval({
|
||||
id: 'rollout-1',
|
||||
kind: 'rollout_authorization',
|
||||
authority: 'operator',
|
||||
authoritative: 1,
|
||||
application_id: 'app-a',
|
||||
generation_id: 'gen-a',
|
||||
intent_revision_id: 'intent-2',
|
||||
artifact_set_id: 'art-missing',
|
||||
rollout_candidate_id: 'cand-1',
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: 'acc-bind-a',
|
||||
placement_approval_ref: 'place-bind',
|
||||
required_targets_json: encodeGitOpsRequiredTargetsJson([1]),
|
||||
preflight_fingerprint: fingerprint,
|
||||
fingerprint: null,
|
||||
blast_json: null,
|
||||
policy_provenance_json: null,
|
||||
actor: 'tester',
|
||||
created_at: 10,
|
||||
});
|
||||
expect(store.resolveApprovalRef('rollout-1', {
|
||||
kind: 'rollout_authorization',
|
||||
applicationId: 'app-a',
|
||||
binding: {
|
||||
rolloutCandidateId: 'cand-1',
|
||||
acceptedGenerationId: 'gen-b',
|
||||
artifactSetId: 'art-missing',
|
||||
intentRevisionId: 'intent-2',
|
||||
requiredNodeIds: [1],
|
||||
sourceAcceptanceRef: 'acc-bind-a',
|
||||
placementApprovalRef: 'place-bind',
|
||||
preflightFingerprint: fingerprint,
|
||||
},
|
||||
})).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
function sourceAcceptance(id: string, applicationId: string, generationId: string): GitOpsApprovalRow {
|
||||
return {
|
||||
id,
|
||||
kind: 'source_acceptance',
|
||||
authority: 'operator',
|
||||
authoritative: 1,
|
||||
application_id: applicationId,
|
||||
generation_id: generationId,
|
||||
intent_revision_id: null,
|
||||
artifact_set_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
required_targets_json: null,
|
||||
preflight_fingerprint: null,
|
||||
fingerprint: null,
|
||||
blast_json: null,
|
||||
policy_provenance_json: null,
|
||||
actor: 'tester',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function placement(
|
||||
id: string,
|
||||
applicationId: string,
|
||||
intentRevisionId: string,
|
||||
effect: Array<{ nodeId: number; outcome: 'place' | 'remove' }>,
|
||||
): GitOpsApprovalRow {
|
||||
return {
|
||||
...sourceAcceptance(id, applicationId, 'gen-a'),
|
||||
kind: 'placement_approval',
|
||||
generation_id: null,
|
||||
intent_revision_id: intentRevisionId,
|
||||
blast_json: encodeGitOpsApprovedTargetEffectJson(effect),
|
||||
};
|
||||
}
|
||||
|
||||
function intent(id: string, applicationId: string): GitOpsIntentRevisionRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
blueprint_id: 1,
|
||||
compose_content_sha256: 'c'.repeat(64),
|
||||
blueprint_revision: 1,
|
||||
deploy_stack_name: 'web',
|
||||
selector_json: '{}',
|
||||
pinned_node_id: null,
|
||||
cordon_implications_json: '[]',
|
||||
rollout_strategy_json: '{}',
|
||||
runtime_drift_policy: null,
|
||||
stateful_policy_json: null,
|
||||
health_failure_rollback_policy_json: null,
|
||||
operation_id: 'op-intent',
|
||||
actor: 'tester',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function candidate(
|
||||
id: string,
|
||||
applicationId: string,
|
||||
intentRevisionId: string,
|
||||
acceptedGenerationId: string,
|
||||
): GitOpsRolloutCandidateRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
intent_revision_id: intentRevisionId,
|
||||
compose_content_sha256: 'c'.repeat(64),
|
||||
accepted_generation_id: acceptedGenerationId,
|
||||
artifact_set_id: null,
|
||||
required_targets_json: encodeGitOpsRequiredTargetsJson([1]),
|
||||
authoritative: 0,
|
||||
provenance: 'legacy_inline',
|
||||
operation_id: 'op-cand',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function directApp(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function generation(id: string, applicationId: string): GitOpsGenerationRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
commit_sha: id,
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
configured_ref: 'main',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
manifest_version: 0,
|
||||
candidate_dir: `generations/candidate-${id}`,
|
||||
applied_dir: `generations/applied-${id}-0`,
|
||||
expected_invocation_json: '{"composeFileOrder":[],"projectName":null,"projectDirectory":null,"envFileOrder":[]}',
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
validation_ok: 1,
|
||||
plan_blocked: 0,
|
||||
change_plan_fingerprint: null,
|
||||
operation_id: `op-${id}`,
|
||||
trigger: 'manual',
|
||||
actor: 'tester',
|
||||
previous_generation_id: null,
|
||||
redacted_limitations_json: '[]',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,272 @@
|
||||
/**
|
||||
* Blueprint deployment writes, recorded by cause.
|
||||
*
|
||||
* The cause has to be carried rather than inferred, because several land on the
|
||||
* same deployment status. A deploy that failed and a withdraw that failed both
|
||||
* read `failed`, and they mean opposite things about whether the deployment is
|
||||
* still on the node.
|
||||
*/
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService, type Blueprint } from '../services/DatabaseService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import { commitBlueprintCreate, commitBlueprintUpdate } from '../services/gitops/blueprintProducers';
|
||||
import {
|
||||
commitBlueprintDeploymentCause,
|
||||
commitBlueprintDeploymentRemoved,
|
||||
} from '../services/gitops/blueprintDeploymentProducers';
|
||||
|
||||
const NODE = 1;
|
||||
const NEXT_COMPOSE = 'services:\n web:\n image: nginx:1.29\n';
|
||||
|
||||
describe('gitops blueprint deployment causes', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('creates the target on the first deploy and records what was requested', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('dc-first');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
// A Blueprint application has no targets until something is sent somewhere.
|
||||
expect(store.getTarget(app.id, NODE)).toBeUndefined();
|
||||
|
||||
deploying(blueprint);
|
||||
|
||||
const target = store.getTarget(app.id, NODE)!;
|
||||
expect(target.active_operation_stage).toBe('blueprint_deploy_started');
|
||||
expect(target.active_intent_revision_id).toBe(app.intent_revision_id);
|
||||
});
|
||||
|
||||
it('acknowledges the intent the node was actually sent', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('dc-ack');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
deploying(blueprint);
|
||||
|
||||
commitBlueprintDeploymentCause('deploy_ack', blueprint.id, NODE, {
|
||||
status: 'active', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
|
||||
const target = store.getTarget(app.id, NODE)!;
|
||||
expect(target.intent_revision_id).toBe(app.intent_revision_id);
|
||||
expect(target.active_operation_stage).toBeNull();
|
||||
});
|
||||
|
||||
it('records nothing when an observation repeats the state it already reported', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const db = DatabaseService.getInstance();
|
||||
const blueprint = create('dc-repeat');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
deploying(blueprint);
|
||||
commitBlueprintDeploymentCause('drift_observed', blueprint.id, NODE, {
|
||||
status: 'drifted', last_checked_at: Date.now(), drift_summary: 'moved',
|
||||
}, 'tester');
|
||||
const before = historyCount(db, app.id);
|
||||
|
||||
// A reconciler tick re-asserting a state it already reported must not
|
||||
// append a second event describing the same fact.
|
||||
commitBlueprintDeploymentCause('drift_observed', blueprint.id, NODE, {
|
||||
status: 'drifted', last_checked_at: Date.now(), drift_summary: 'moved again',
|
||||
}, 'tester');
|
||||
expect(historyCount(db, app.id)).toBe(before);
|
||||
});
|
||||
|
||||
it('supersedes a stuck deploy rather than answering the request it replaced', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('dc-stuck');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
deploying(blueprint);
|
||||
const stale = store.getTarget(app.id, NODE)!.active_intent_revision_id;
|
||||
|
||||
// The Blueprint changed while the row sat at `deploying`, so a redeploy is
|
||||
// asking for something new. The status does not move, and gating the start
|
||||
// on that would let the later acknowledgement answer the stale request.
|
||||
commitBlueprintUpdate(
|
||||
blueprint.id, { compose_content: NEXT_COMPOSE }, 'tester', () => [NODE],
|
||||
);
|
||||
const revised = store.getLiveBlueprintApplication(blueprint.id)!.intent_revision_id;
|
||||
expect(revised).not.toBe(stale);
|
||||
|
||||
deploying(blueprint);
|
||||
expect(store.getTarget(app.id, NODE)!.active_intent_revision_id).toBe(revised);
|
||||
|
||||
commitBlueprintDeploymentCause('deploy_ack', blueprint.id, NODE, {
|
||||
status: 'active', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
// Converged on what was actually asked for last, not on the superseded one.
|
||||
expect(store.getTarget(app.id, NODE)!.intent_revision_id).toBe(revised);
|
||||
});
|
||||
|
||||
it('keeps a failed deploy distinct from a failed withdraw', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const failed = create('dc-deploy-fail');
|
||||
deploying(failed);
|
||||
commitBlueprintDeploymentCause('deploy_fail', failed.id, NODE, {
|
||||
status: 'failed', last_checked_at: Date.now(), last_error: 'boom',
|
||||
}, 'tester');
|
||||
const deployTarget = store.getTarget(store.getLiveBlueprintApplication(failed.id)!.id, NODE)!;
|
||||
expect(deployTarget.failure_stage).toBe('blueprint_deploy');
|
||||
expect(deployTarget.target_status).toBe('active');
|
||||
|
||||
const withdrawn = create('dc-withdraw-fail');
|
||||
deploying(withdrawn);
|
||||
commitBlueprintDeploymentCause('deploy_ack', withdrawn.id, NODE, {
|
||||
status: 'active', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
commitBlueprintDeploymentCause('withdraw_start', withdrawn.id, NODE, {
|
||||
status: 'withdrawing', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
commitBlueprintDeploymentCause('withdraw_fail', withdrawn.id, NODE, {
|
||||
status: 'failed', last_checked_at: Date.now(), last_error: 'boom',
|
||||
}, 'tester');
|
||||
|
||||
const withdrawTarget = store.getTarget(store.getLiveBlueprintApplication(withdrawn.id)!.id, NODE)!;
|
||||
// Same deployment status, opposite meaning: the deployment is still there.
|
||||
expect(withdrawTarget.failure_stage).toBe('blueprint_withdraw');
|
||||
expect(withdrawTarget.target_status).toBe('active');
|
||||
});
|
||||
|
||||
it('classifies a name conflict as its own failure', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('dc-conflict');
|
||||
deploying(blueprint);
|
||||
commitBlueprintDeploymentCause('name_conflict', blueprint.id, NODE, {
|
||||
status: 'name_conflict', last_checked_at: Date.now(), last_error: 'taken',
|
||||
}, 'tester');
|
||||
|
||||
const target = store.getTarget(store.getLiveBlueprintApplication(blueprint.id)!.id, NODE)!;
|
||||
expect(target.failure_class).toBe('name_conflict');
|
||||
});
|
||||
|
||||
it('tombstones the target when the deployment row is removed', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('dc-removed');
|
||||
deploying(blueprint);
|
||||
commitBlueprintDeploymentCause('deploy_ack', blueprint.id, NODE, {
|
||||
status: 'active', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
commitBlueprintDeploymentCause('withdraw_start', blueprint.id, NODE, {
|
||||
status: 'withdrawing', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
|
||||
commitBlueprintDeploymentRemoved(blueprint.id, NODE, 'tester');
|
||||
|
||||
const target = store.getTarget(store.getLiveBlueprintApplication(blueprint.id)!.id, NODE)!;
|
||||
expect(target.target_status).toBe('tombstoned');
|
||||
});
|
||||
|
||||
it('re-opens a severed target when a deploy starts again', () => {
|
||||
// After a withdraw the reconciler must not redeploy onto the severed
|
||||
// placement unrecorded, while an explicit deploy must land in the model
|
||||
// instead of being refused and ignored.
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('dc-revive');
|
||||
deploying(blueprint);
|
||||
commitBlueprintDeploymentCause('deploy_ack', blueprint.id, NODE, {
|
||||
status: 'active', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
commitBlueprintDeploymentCause('withdraw_start', blueprint.id, NODE, {
|
||||
status: 'withdrawing', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
commitBlueprintDeploymentRemoved(blueprint.id, NODE, 'tester');
|
||||
|
||||
const appId = store.getLiveBlueprintApplication(blueprint.id)!.id;
|
||||
expect(store.getTarget(appId, NODE)!.target_status).toBe('tombstoned');
|
||||
|
||||
deploying(blueprint);
|
||||
|
||||
const revived = store.getTarget(appId, NODE)!;
|
||||
expect(revived.target_status).toBe('active');
|
||||
expect(revived.active_operation_stage).toBe('blueprint_deploy_started');
|
||||
});
|
||||
|
||||
it('observes without acknowledging anything', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('dc-observe');
|
||||
deploying(blueprint);
|
||||
|
||||
commitBlueprintDeploymentCause('drift_observed', blueprint.id, NODE, {
|
||||
status: 'drifted', last_checked_at: Date.now(), drift_summary: 'image moved',
|
||||
}, 'tester');
|
||||
|
||||
const target = store.getTarget(store.getLiveBlueprintApplication(blueprint.id)!.id, NODE)!;
|
||||
expect(target.latest_stage).toBe('blueprint_drifted');
|
||||
// An observation says what was seen, never what was agreed.
|
||||
expect(target.intent_revision_id).toBeNull();
|
||||
});
|
||||
|
||||
it('records a stateful first placement, which happens before any deploy', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('dc-first-placement');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
// No deploy has run, so there is no target: this is the case that used to
|
||||
// drop the observation and leave the hold unrecorded.
|
||||
expect(store.getTarget(app.id, NODE)).toBeUndefined();
|
||||
|
||||
commitBlueprintDeploymentCause('await_state_review', blueprint.id, NODE, {
|
||||
status: 'pending_state_review', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
|
||||
const target = store.getTarget(app.id, NODE)!;
|
||||
expect(target.latest_stage).toBe('blueprint_state_review');
|
||||
// First contact only. Nothing has been sent, applied or agreed.
|
||||
expect(target.intent_revision_id).toBeNull();
|
||||
expect(target.desired_generation_id).toBeNull();
|
||||
expect(target.active_operation_stage).toBeNull();
|
||||
// Unset rather than reachable: a node that has only been asked to hold
|
||||
// something has not been contacted, and claiming reachability here would
|
||||
// make the rollout facet answer for a node nobody has spoken to.
|
||||
expect(target.connectivity).toBeNull();
|
||||
});
|
||||
|
||||
it('still drops an observation for a node with no target and no placement', () => {
|
||||
// Only the first-placement hold creates a target. A drift or evict report
|
||||
// for a node nothing was ever sent to describes a deployment this model
|
||||
// does not have, so it stays dropped.
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('dc-observe-no-target');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
commitBlueprintDeploymentCause('drift_observed', blueprint.id, NODE, {
|
||||
status: 'drifted', last_checked_at: Date.now(), drift_summary: 'image moved',
|
||||
}, 'tester');
|
||||
|
||||
expect(store.getTarget(app.id, NODE)).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
function historyCount(db: DatabaseService, applicationId: string): number {
|
||||
return (db.getDb()
|
||||
.prepare('SELECT COUNT(*) AS n FROM gitops_history WHERE application_id = ?')
|
||||
.get(applicationId) as { n: number }).n;
|
||||
}
|
||||
|
||||
function deploying(blueprint: Blueprint): void {
|
||||
commitBlueprintDeploymentCause('deploy_start', blueprint.id, NODE, {
|
||||
status: 'deploying', last_checked_at: Date.now(),
|
||||
}, 'tester');
|
||||
}
|
||||
|
||||
function create(name: string): Blueprint {
|
||||
return commitBlueprintCreate({
|
||||
name,
|
||||
description: null,
|
||||
compose_content: 'services:\n web:\n image: nginx:1.27\n',
|
||||
selector: { type: 'nodes', ids: [NODE] },
|
||||
drift_mode: 'suggest',
|
||||
classification: 'stateless',
|
||||
classification_reasons: [],
|
||||
enabled: true,
|
||||
created_by: 'tester',
|
||||
}, () => [NODE]);
|
||||
}
|
||||
@@ -0,0 +1,338 @@
|
||||
/**
|
||||
* Blueprint source-mutation producers.
|
||||
*
|
||||
* These are the seam between the Blueprint routes and the revision state, and
|
||||
* the question they exist to answer is when an edit invalidates what the fleet
|
||||
* already acknowledged. Renaming or re-selecting does; rewording a description
|
||||
* does not, and minting an intent for the latter would make every node's
|
||||
* acknowledgement read as stale over a change no node can observe.
|
||||
*/
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService, type Blueprint } from '../services/DatabaseService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import {
|
||||
classifyBlueprintChange,
|
||||
commitBlueprintCreate,
|
||||
commitBlueprintDelete,
|
||||
commitBlueprintPin,
|
||||
commitBlueprintUpdate,
|
||||
} from '../services/gitops/blueprintProducers';
|
||||
|
||||
const DESIRED = [1];
|
||||
const desiredNodeIdsFor = (): number[] => DESIRED;
|
||||
|
||||
describe('gitops blueprint producers', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('creates the Blueprint, its application, and the first intent and candidate together', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-create');
|
||||
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
expect(app.target_mode).toBe('inline_blueprint');
|
||||
expect(app.lifecycle_status).toBe('active');
|
||||
// No Git identity: an Inline Blueprint has no generations to point at.
|
||||
expect(app.stack_name).toBeNull();
|
||||
expect(app.configured_repo_url).toBeNull();
|
||||
|
||||
const intent = store.getIntentRevision(app.intent_revision_id!)!;
|
||||
expect(intent.blueprint_id).toBe(blueprint.id);
|
||||
expect(intent.deploy_stack_name).toBe('bp-create');
|
||||
|
||||
const candidate = store.getRolloutCandidate(app.rollout_candidate_id!)!;
|
||||
expect(candidate.intent_revision_id).toBe(intent.id);
|
||||
expect(JSON.parse(candidate.required_targets_json)).toEqual({ nodeIds: DESIRED });
|
||||
|
||||
// History starts where the application does. Beginning at the first intent
|
||||
// would describe an application nothing records coming into existence.
|
||||
const stages = DatabaseService.getInstance().getDb().prepare(
|
||||
'SELECT stage FROM gitops_history WHERE application_id = ? ORDER BY rowid ASC',
|
||||
).all(app.id) as Array<{ stage: string }>;
|
||||
expect(stages.map(row => row.stage))
|
||||
.toEqual(['application_activated', 'intent_revised', 'rollout_candidate_opened']);
|
||||
|
||||
// No targets until something is deployed somewhere.
|
||||
expect(store.listTargets(app.id)).toEqual([]);
|
||||
});
|
||||
|
||||
it('refuses a second live application for the same Blueprint', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-single');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
|
||||
expect(() => tx.activateInlineBlueprint({
|
||||
application: { ...app, id: 'second-app' },
|
||||
envelope: { operationId: 'op-dup', actor: 'tester', trigger: 'manual', at: Date.now() },
|
||||
})).toThrow(/already exists/);
|
||||
});
|
||||
|
||||
it('mints nothing when an edit changes no value', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-noop');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
const result = commitBlueprintUpdate(
|
||||
blueprint.id, { name: 'bp-noop', drift_mode: blueprint.drift_mode }, 'tester', desiredNodeIdsFor,
|
||||
);
|
||||
|
||||
expect(result.change).toBe('none');
|
||||
const after = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
expect(after.intent_revision_id).toBe(app.intent_revision_id);
|
||||
expect(after.rollout_candidate_id).toBe(app.rollout_candidate_id);
|
||||
});
|
||||
|
||||
it('leaves the acknowledged intent alone when only the description changes', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-meta');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
const result = commitBlueprintUpdate(
|
||||
blueprint.id, { description: 'now with a longer explanation' }, 'tester', desiredNodeIdsFor,
|
||||
);
|
||||
|
||||
expect(result.change).toBe('metadata_only');
|
||||
expect(result.blueprint?.description).toBe('now with a longer explanation');
|
||||
// The source row moved and the intent did not: no node's acknowledgement
|
||||
// became stale because someone reworded the description.
|
||||
const after = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
expect(after.intent_revision_id).toBe(app.intent_revision_id);
|
||||
});
|
||||
|
||||
it('mints a new intent and candidate when the deployed content changes', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-op');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
const result = commitBlueprintUpdate(
|
||||
blueprint.id, { compose_content: 'services:\n web:\n image: nginx:1.29\n' }, 'tester', desiredNodeIdsFor,
|
||||
);
|
||||
|
||||
expect(result.change).toBe('operational');
|
||||
const after = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
expect(after.intent_revision_id).not.toBe(app.intent_revision_id);
|
||||
expect(after.rollout_candidate_id).not.toBe(app.rollout_candidate_id);
|
||||
const intent = store.getIntentRevision(after.intent_revision_id!)!;
|
||||
expect(intent.compose_content_sha256).not.toBe(
|
||||
store.getIntentRevision(app.intent_revision_id!)!.compose_content_sha256,
|
||||
);
|
||||
});
|
||||
|
||||
it('treats a pin as a placement change, and re-pinning the same node as nothing', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-pin');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
const pinned = commitBlueprintPin(blueprint.id, 1, 'tester', desiredNodeIdsFor);
|
||||
expect(pinned.changed).toBe(true);
|
||||
const afterPin = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
expect(afterPin.intent_revision_id).not.toBe(app.intent_revision_id);
|
||||
expect(store.getRolloutCandidate(afterPin.rollout_candidate_id!)?.provenance).toBe('roster_change');
|
||||
|
||||
const again = commitBlueprintPin(blueprint.id, 1, 'tester', desiredNodeIdsFor);
|
||||
expect(again.changed).toBe(false);
|
||||
expect(store.getLiveBlueprintApplication(blueprint.id)?.intent_revision_id)
|
||||
.toBe(afterPin.intent_revision_id);
|
||||
});
|
||||
|
||||
it('records the required set in a canonical order so a reorder is not a change', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-order');
|
||||
commitBlueprintUpdate(blueprint.id, { name: 'bp-order-2' }, 'tester', () => [3, 1, 2]);
|
||||
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
expect(JSON.parse(store.getRolloutCandidate(app.rollout_candidate_id!)!.required_targets_json))
|
||||
.toEqual({ nodeIds: [1, 2, 3] });
|
||||
});
|
||||
|
||||
it('retires the application when the Blueprint is deleted', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-delete');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
expect(commitBlueprintDelete(blueprint.id, 'tester')).toBe(true);
|
||||
|
||||
// The live slot has to be released, or the Blueprint name cannot be used
|
||||
// again while a record of a deleted one still claims it.
|
||||
expect(store.getLiveBlueprintApplication(blueprint.id)).toBeUndefined();
|
||||
expect(store.getApplication(app.id)?.lifecycle_status).toBe('deleted');
|
||||
});
|
||||
|
||||
it('does not bump the revision or void approval when only the description changed', () => {
|
||||
const db = DatabaseService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-full-save');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
const intentBefore = store.getIntentRevision(app.intent_revision_id!)!;
|
||||
|
||||
// What the editor actually sends: every field, every save. The source layer
|
||||
// decides what to invalidate from which keys are present, so submitting an
|
||||
// unchanged compose body used to advance the revision past the one the
|
||||
// current intent describes, and clear the approval, while this layer
|
||||
// classified it as metadata and minted nothing.
|
||||
const result = commitBlueprintUpdate(blueprint.id, {
|
||||
name: blueprint.name,
|
||||
description: 'reworded',
|
||||
compose_content: blueprint.compose_content,
|
||||
selector: blueprint.selector,
|
||||
drift_mode: blueprint.drift_mode,
|
||||
enabled: blueprint.enabled,
|
||||
bumpRevision: true,
|
||||
}, 'tester', desiredNodeIdsFor);
|
||||
|
||||
expect(result.change).toBe('metadata_only');
|
||||
const after = db.getBlueprint(blueprint.id)!;
|
||||
expect(after.description).toBe('reworded');
|
||||
expect(after.revision).toBe(blueprint.revision);
|
||||
expect(after.approval_status).toBe(blueprint.approval_status);
|
||||
// The intent still describes the revision that is actually stored.
|
||||
expect(store.getIntentRevision(app.intent_revision_id!)!.blueprint_revision).toBe(after.revision);
|
||||
expect(store.getLiveBlueprintApplication(blueprint.id)!.intent_revision_id).toBe(intentBefore.id);
|
||||
});
|
||||
|
||||
it('treats a reordered selector naming the same nodes as no change', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-reorder');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
const result = commitBlueprintUpdate(
|
||||
blueprint.id, { selector: { type: 'nodes', ids: [1] } }, 'tester', desiredNodeIdsFor,
|
||||
);
|
||||
|
||||
expect(result.change).toBe('none');
|
||||
expect(store.getLiveBlueprintApplication(blueprint.id)!.intent_revision_id).toBe(app.intent_revision_id);
|
||||
});
|
||||
|
||||
it('rolls the Blueprint back when recording it fails', () => {
|
||||
const db = DatabaseService.getInstance();
|
||||
const applicationsBefore = (db.getDb()
|
||||
.prepare("SELECT COUNT(*) AS n FROM gitops_applications WHERE target_mode = 'inline_blueprint'")
|
||||
.get() as { n: number }).n;
|
||||
|
||||
// The source write and its record commit together or not at all, so a
|
||||
// Blueprint can never exist with nothing describing what it means.
|
||||
expect(() => commitBlueprintCreate({
|
||||
name: 'bp-rollback',
|
||||
description: null,
|
||||
compose_content: 'services:\n web:\n image: nginx:1.27\n',
|
||||
selector: { type: 'nodes', ids: [1] },
|
||||
drift_mode: 'suggest',
|
||||
classification: 'stateless',
|
||||
classification_reasons: [],
|
||||
enabled: true,
|
||||
created_by: 'tester',
|
||||
}, () => { throw new Error('placement lookup failed'); })).toThrow(/placement lookup failed/);
|
||||
|
||||
expect(db.getBlueprintByName('bp-rollback')).toBeUndefined();
|
||||
// And no orphan application survived the rolled-back source write.
|
||||
expect(db.getDb()
|
||||
.prepare("SELECT COUNT(*) AS n FROM gitops_applications WHERE target_mode = 'inline_blueprint'")
|
||||
.get() as { n: number }).toEqual({ n: applicationsBefore });
|
||||
});
|
||||
|
||||
it('leaves every other Blueprint untouched when one is edited', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const other = create('bp-bystander');
|
||||
const edited = create('bp-edited');
|
||||
const otherBefore = store.getLiveBlueprintApplication(other.id)!;
|
||||
|
||||
commitBlueprintUpdate(edited.id, { name: 'bp-edited-2' }, 'tester', desiredNodeIdsFor);
|
||||
|
||||
const otherAfter = store.getLiveBlueprintApplication(other.id)!;
|
||||
expect(otherAfter.intent_revision_id).toBe(otherBefore.intent_revision_id);
|
||||
expect(otherAfter.rollout_candidate_id).toBe(otherBefore.rollout_candidate_id);
|
||||
expect(store.getIntentRevision(otherAfter.intent_revision_id!)!.deploy_stack_name).toBe('bp-bystander');
|
||||
});
|
||||
|
||||
it('classifies each field without touching the database', () => {
|
||||
const before = {
|
||||
name: 'a', description: 'd', compose_content: 'c', selector: { type: 'nodes', ids: [1] },
|
||||
drift_mode: 'suggest', enabled: true, classification: 'stateless', classification_reasons: [],
|
||||
} as unknown as Blueprint;
|
||||
|
||||
expect(classifyBlueprintChange(before, {})).toBe('none');
|
||||
expect(classifyBlueprintChange(before, { name: 'a' })).toBe('none');
|
||||
expect(classifyBlueprintChange(before, { description: 'd' })).toBe('none');
|
||||
expect(classifyBlueprintChange(before, { description: 'other' })).toBe('metadata_only');
|
||||
expect(classifyBlueprintChange(before, { name: 'b' })).toBe('operational');
|
||||
expect(classifyBlueprintChange(before, { enabled: false })).toBe('operational');
|
||||
// Selector equality is by value: the same set written again is not a change.
|
||||
expect(classifyBlueprintChange(before, { selector: { type: 'nodes', ids: [1] } })).toBe('none');
|
||||
expect(classifyBlueprintChange(before, { selector: { type: 'nodes', ids: [2] } })).toBe('operational');
|
||||
// An operational change alongside a metadata one is still operational.
|
||||
expect(classifyBlueprintChange(before, { name: 'b', description: 'other' })).toBe('operational');
|
||||
});
|
||||
|
||||
describe('an application that is not yet active', () => {
|
||||
// The live-slot lookup answers with `active` or `creating`, because its
|
||||
// other callers ask whether the slot is taken. The transitions that mint
|
||||
// intents accept only `active` and reject anything else by throwing, inside
|
||||
// the caller's own transaction, so a `creating` row reaching one would fail
|
||||
// the operator's edit and roll the Blueprint write back with it.
|
||||
//
|
||||
// The state is written directly here because no production path creates a
|
||||
// Blueprint-mode application in `creating`: they all go through
|
||||
// `blankInlineApplication`, which hardcodes `active`. These cases pin a
|
||||
// deliberately defensive guard rather than a reachable behaviour, and are
|
||||
// marked as such so nobody later reads them as live coverage. The Git
|
||||
// backed `blueprint` mode is what makes the state producible.
|
||||
const toCreating = (blueprintId: number): void => {
|
||||
DatabaseService.getInstance().getDb()
|
||||
.prepare('UPDATE gitops_applications SET lifecycle_status = ? WHERE blueprint_id = ?')
|
||||
.run('creating', blueprintId);
|
||||
};
|
||||
|
||||
it('lets an edit through without minting an intent', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-creating-update');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
const intentBefore = app.intent_revision_id;
|
||||
toCreating(blueprint.id);
|
||||
|
||||
const result = commitBlueprintUpdate(blueprint.id, { name: 'bp-creating-renamed' }, 'tester', desiredNodeIdsFor);
|
||||
|
||||
expect(result.blueprint?.name).toBe('bp-creating-renamed');
|
||||
expect(store.getLiveBlueprintApplication(blueprint.id)!.intent_revision_id).toBe(intentBefore);
|
||||
});
|
||||
|
||||
it('lets a pin through without minting an intent', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('bp-creating-pin');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
const intentBefore = app.intent_revision_id;
|
||||
toCreating(blueprint.id);
|
||||
|
||||
const result = commitBlueprintPin(blueprint.id, 1, 'tester', desiredNodeIdsFor);
|
||||
|
||||
expect(result.changed).toBe(true);
|
||||
expect(result.blueprint?.pinned_node_id).toBe(1);
|
||||
expect(store.getLiveBlueprintApplication(blueprint.id)!.intent_revision_id).toBe(intentBefore);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
function create(name: string): Blueprint {
|
||||
return commitBlueprintCreate({
|
||||
name,
|
||||
description: null,
|
||||
compose_content: 'services:\n web:\n image: nginx:1.27\n',
|
||||
selector: { type: 'nodes', ids: [1] },
|
||||
drift_mode: 'suggest',
|
||||
classification: 'stateless',
|
||||
classification_reasons: [],
|
||||
enabled: true,
|
||||
created_by: 'tester',
|
||||
}, desiredNodeIdsFor);
|
||||
}
|
||||
@@ -0,0 +1,628 @@
|
||||
/**
|
||||
* Blueprint source and deployment transitions.
|
||||
*
|
||||
* These have no production caller yet; the Blueprint routes and the reconciler
|
||||
* are wired to them in the same step. They are tested directly so the shape a
|
||||
* caller must satisfy is pinned here rather than inferred from the deriver.
|
||||
*
|
||||
* The rule they share is that a terminal event has to name the request it is
|
||||
* answering. A node that acknowledges a superseded intent has not converged on
|
||||
* anything anyone asked for, and recording it as an acknowledgement is how a
|
||||
* fleet comes to report agreement it does not have.
|
||||
*/
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { GitOpsStore, emptyTargetRow } from '../services/gitops/store';
|
||||
import { GitOpsTransitions, type EventEnvelope } from '../services/gitops/transitions';
|
||||
import { projectApplication } from '../services/gitops/derive';
|
||||
import type {
|
||||
GitOpsApplicationRow,
|
||||
GitOpsIntentRevisionRow,
|
||||
GitOpsRolloutCandidateRow,
|
||||
} from '../services/gitops/types';
|
||||
|
||||
describe('gitops blueprint transitions', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('mints an intent and opens a candidate against it', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-intent', 101);
|
||||
|
||||
tx.intentRevised({
|
||||
applicationId: 'app-intent',
|
||||
intent: intent('int-1', 'app-intent', 101),
|
||||
envelope: env('op-int-1'),
|
||||
});
|
||||
expect(store.getApplication('app-intent')?.intent_revision_id).toBe('int-1');
|
||||
|
||||
tx.rolloutCandidateOpened({
|
||||
applicationId: 'app-intent',
|
||||
candidate: candidate('cand-1', 'app-intent', 'int-1'),
|
||||
envelope: env('op-cand-1'),
|
||||
});
|
||||
const app = store.getApplication('app-intent')!;
|
||||
expect(app.rollout_candidate_id).toBe('cand-1');
|
||||
// Candidate-time facts only: nothing here claims anything was authorized.
|
||||
const row = store.getRolloutCandidate('cand-1')!;
|
||||
expect(row.intent_revision_id).toBe('int-1');
|
||||
expect(row.accepted_generation_id).toBeNull();
|
||||
});
|
||||
|
||||
it('refuses a candidate that does not name the current intent', () => {
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-stale-cand', 102);
|
||||
tx.intentRevised({
|
||||
applicationId: 'app-stale-cand',
|
||||
intent: intent('int-2', 'app-stale-cand', 102),
|
||||
envelope: env('op-int-2'),
|
||||
});
|
||||
|
||||
expect(() => tx.rolloutCandidateOpened({
|
||||
applicationId: 'app-stale-cand',
|
||||
candidate: { ...candidate('cand-2', 'app-stale-cand', 'int-nonexistent') },
|
||||
envelope: env('op-cand-2'),
|
||||
})).toThrow(/current intent/);
|
||||
});
|
||||
|
||||
it('records a deploy, then accepts the ack that names it', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-ack', 103, 1);
|
||||
tx.intentRevised({ applicationId: 'app-ack', intent: intent('int-3', 'app-ack', 103), envelope: env('op-int-3') });
|
||||
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-ack',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-3',
|
||||
rolloutCandidateId: null,
|
||||
envelope: env('op-dep-3'),
|
||||
});
|
||||
let target = store.getTarget('app-ack', 1)!;
|
||||
expect(target.active_operation_stage).toBe('blueprint_deploy_started');
|
||||
expect(target.active_intent_revision_id).toBe('int-3');
|
||||
// Nothing is acknowledged yet: the request is in flight, not converged.
|
||||
expect(target.intent_revision_id).toBeNull();
|
||||
|
||||
tx.blueprintAckRecorded({
|
||||
applicationId: 'app-ack',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-3',
|
||||
rolloutCandidateId: null,
|
||||
legacyAppliedRevision: 7,
|
||||
envelope: env('op-ack-3'),
|
||||
});
|
||||
target = store.getTarget('app-ack', 1)!;
|
||||
expect(target.intent_revision_id).toBe('int-3');
|
||||
expect(target.active_operation_stage).toBeNull();
|
||||
expect(target.legacy_applied_revision).toBe(7);
|
||||
// A Blueprint target has no Git generation to point at.
|
||||
expect(target.desired_generation_id).toBeNull();
|
||||
});
|
||||
|
||||
it('ignores an acknowledgement for an intent the target was never asked to run', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-super', 104, 1);
|
||||
tx.intentRevised({ applicationId: 'app-super', intent: intent('int-4', 'app-super', 104), envelope: env('op-int-4') });
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-super',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-4',
|
||||
rolloutCandidateId: null,
|
||||
envelope: env('op-dep-4'),
|
||||
});
|
||||
|
||||
// A newer intent superseded the one this node is running.
|
||||
tx.intentRevised({ applicationId: 'app-super', intent: intent('int-5', 'app-super', 104), envelope: env('op-int-5') });
|
||||
|
||||
expect(() => tx.blueprintAckRecorded({
|
||||
applicationId: 'app-super',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-5',
|
||||
rolloutCandidateId: null,
|
||||
legacyAppliedRevision: null,
|
||||
envelope: env('op-ack-5'),
|
||||
})).toThrow(/was not asked to run/);
|
||||
expect(store.getTarget('app-super', 1)?.intent_revision_id).toBeNull();
|
||||
});
|
||||
|
||||
it('clears a deploy failure only when the next deploy is acknowledged', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-fail', 105, 1);
|
||||
tx.intentRevised({ applicationId: 'app-fail', intent: intent('int-6', 'app-fail', 105), envelope: env('op-int-6') });
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-fail',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-6',
|
||||
rolloutCandidateId: null,
|
||||
envelope: env('op-dep-6'),
|
||||
});
|
||||
tx.blueprintDeployFailed({
|
||||
applicationId: 'app-fail',
|
||||
nodeId: 1,
|
||||
failureClass: 'name_conflict',
|
||||
envelope: env('op-dep-6'),
|
||||
});
|
||||
|
||||
let target = store.getTarget('app-fail', 1)!;
|
||||
expect(target.failure_stage).toBe('blueprint_deploy');
|
||||
expect(target.failure_class).toBe('name_conflict');
|
||||
expect(target.active_operation_stage).toBeNull();
|
||||
// A failure does not acknowledge anything.
|
||||
expect(target.intent_revision_id).toBeNull();
|
||||
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-fail',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-6',
|
||||
rolloutCandidateId: null,
|
||||
envelope: env('op-dep-6b'),
|
||||
});
|
||||
tx.blueprintAckRecorded({
|
||||
applicationId: 'app-fail',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-6',
|
||||
rolloutCandidateId: null,
|
||||
legacyAppliedRevision: null,
|
||||
envelope: env('op-ack-6b'),
|
||||
});
|
||||
target = store.getTarget('app-fail', 1)!;
|
||||
expect(target.failure_stage).toBeNull();
|
||||
expect(target.intent_revision_id).toBe('int-6');
|
||||
});
|
||||
|
||||
it('withdraws against the intent being removed, not a later one', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-wd', 106, 1);
|
||||
tx.intentRevised({ applicationId: 'app-wd', intent: intent('int-7', 'app-wd', 106), envelope: env('op-int-7') });
|
||||
tx.blueprintWithdrawStarted({
|
||||
applicationId: 'app-wd',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-7',
|
||||
envelope: env('op-wd-7'),
|
||||
});
|
||||
|
||||
expect(() => tx.blueprintWithdrawn({
|
||||
applicationId: 'app-wd',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-other',
|
||||
envelope: env('op-wd-7'),
|
||||
})).toThrow(/was not asked to run/);
|
||||
|
||||
tx.blueprintWithdrawn({
|
||||
applicationId: 'app-wd',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-7',
|
||||
envelope: env('op-wd-7'),
|
||||
});
|
||||
const target = store.getTarget('app-wd', 1)!;
|
||||
expect(target.target_status).toBe('tombstoned');
|
||||
expect(target.active_operation_stage).toBeNull();
|
||||
});
|
||||
|
||||
it('re-opens a severed placement when a deploy starts again', () => {
|
||||
// Withdrawal is terminal for the placement, not for the node. A later
|
||||
// explicit deploy re-activates the target and records the revival in the
|
||||
// same event, so the projection and the workload cannot disagree about
|
||||
// whether this node runs the Blueprint.
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-revive', 220, 1);
|
||||
tx.intentRevised({
|
||||
applicationId: 'app-revive',
|
||||
intent: intent('int-rev', 'app-revive', 220),
|
||||
envelope: env('op-rev-int'),
|
||||
});
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-revive', nodeId: 1, intentRevisionId: 'int-rev',
|
||||
rolloutCandidateId: null, envelope: env('op-rev-d1'),
|
||||
});
|
||||
tx.blueprintAckRecorded({
|
||||
applicationId: 'app-revive', nodeId: 1, intentRevisionId: 'int-rev',
|
||||
rolloutCandidateId: null, legacyAppliedRevision: null, envelope: env('op-rev-a1'),
|
||||
});
|
||||
tx.blueprintWithdrawStarted({
|
||||
applicationId: 'app-revive', nodeId: 1, intentRevisionId: 'int-rev',
|
||||
envelope: env('op-rev-w1'),
|
||||
});
|
||||
tx.blueprintWithdrawn({
|
||||
applicationId: 'app-revive', nodeId: 1, intentRevisionId: 'int-rev',
|
||||
envelope: env('op-rev-w2'),
|
||||
});
|
||||
expect(store.getTarget('app-revive', 1)?.target_status).toBe('tombstoned');
|
||||
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-revive', nodeId: 1, intentRevisionId: 'int-rev',
|
||||
rolloutCandidateId: null, envelope: env('op-rev-d2'),
|
||||
});
|
||||
|
||||
const revived = store.getTarget('app-revive', 1)!;
|
||||
expect(revived.target_status).toBe('active');
|
||||
expect(revived.active_operation_stage).toBe('blueprint_deploy_started');
|
||||
// The acknowledged intent survives severance; only a fresh ack rewrites it.
|
||||
expect(revived.intent_revision_id).toBe('int-rev');
|
||||
});
|
||||
|
||||
it('keeps a failed withdraw distinct from a failed deploy', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-wdf', 107, 1);
|
||||
tx.intentRevised({ applicationId: 'app-wdf', intent: intent('int-8', 'app-wdf', 107), envelope: env('op-int-8') });
|
||||
tx.blueprintWithdrawStarted({
|
||||
applicationId: 'app-wdf',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-8',
|
||||
envelope: env('op-wd-8'),
|
||||
});
|
||||
tx.blueprintWithdrawFailed({
|
||||
applicationId: 'app-wdf',
|
||||
nodeId: 1,
|
||||
failureClass: 'post_mutation',
|
||||
envelope: env('op-wd-8'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-wdf', 1)!;
|
||||
expect(target.failure_stage).toBe('blueprint_withdraw');
|
||||
// Still active: a withdraw that failed has not removed the deployment.
|
||||
expect(target.target_status).toBe('active');
|
||||
});
|
||||
|
||||
it('releases the request identity with the operation, not just the stage', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-ident', 120, 1);
|
||||
tx.intentRevised({ applicationId: 'app-ident', intent: intent('int-20', 'app-ident', 120), envelope: env('op-int-20') });
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-ident',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-20',
|
||||
rolloutCandidateId: null,
|
||||
envelope: env('op-dep-20'),
|
||||
});
|
||||
tx.blueprintDeployFailed({
|
||||
applicationId: 'app-ident',
|
||||
nodeId: 1,
|
||||
failureClass: 'pre_mutation',
|
||||
envelope: env('op-dep-20'),
|
||||
});
|
||||
|
||||
// Identity has to go with the stage. Left behind, a later start that only
|
||||
// sets a stage would make the superseded intent read as live again, and a
|
||||
// duplicate ack for it would then be accepted.
|
||||
const target = store.getTarget('app-ident', 1)!;
|
||||
expect(target.active_operation_stage).toBeNull();
|
||||
expect(target.active_intent_revision_id).toBeNull();
|
||||
expect(target.active_rollout_candidate_id).toBeNull();
|
||||
|
||||
expect(() => tx.blueprintAckRecorded({
|
||||
applicationId: 'app-ident',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-20',
|
||||
rolloutCandidateId: null,
|
||||
legacyAppliedRevision: null,
|
||||
envelope: env('op-ack-20'),
|
||||
})).toThrow(/was not asked to run/);
|
||||
});
|
||||
|
||||
it('acknowledges an interrupted deploy, and retires the interruption with it', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-int', 121, 1);
|
||||
tx.intentRevised({ applicationId: 'app-int', intent: intent('int-21', 'app-int', 121), envelope: env('op-int-21') });
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-int',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-21',
|
||||
rolloutCandidateId: 'cand-21',
|
||||
envelope: env('op-dep-21'),
|
||||
});
|
||||
tx.interruptActiveOperations('app-int', env('op-boot-21'));
|
||||
expect(store.getTarget('app-int', 1)?.interruption_stage).toBe('blueprint_deploy_started');
|
||||
|
||||
// An ack that arrives after a restart still names a request this target was
|
||||
// genuinely given, so it is accepted.
|
||||
tx.blueprintAckRecorded({
|
||||
applicationId: 'app-int',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-21',
|
||||
rolloutCandidateId: 'cand-21',
|
||||
legacyAppliedRevision: null,
|
||||
envelope: env('op-ack-21'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-int', 1)!;
|
||||
expect(target.intent_revision_id).toBe('int-21');
|
||||
expect(target.rollout_candidate_id).toBe('cand-21');
|
||||
// Retired, or it would keep matching and let a third ack regress the
|
||||
// pointer after two later deploys had succeeded.
|
||||
expect(target.interruption_stage).toBeNull();
|
||||
expect(target.interruption_intent_revision_id).toBeNull();
|
||||
expect(target.interruption_rollout_candidate_id).toBeNull();
|
||||
});
|
||||
|
||||
it('refuses an acknowledgement that pairs the deployed intent with another candidate', () => {
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-pair', 122, 1);
|
||||
tx.intentRevised({ applicationId: 'app-pair', intent: intent('int-22', 'app-pair', 122), envelope: env('op-int-22') });
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-pair',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-22',
|
||||
rolloutCandidateId: 'cand-22',
|
||||
envelope: env('op-dep-22'),
|
||||
});
|
||||
|
||||
expect(() => tx.blueprintAckRecorded({
|
||||
applicationId: 'app-pair',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-22',
|
||||
rolloutCandidateId: 'cand-other',
|
||||
legacyAppliedRevision: null,
|
||||
envelope: env('op-ack-22'),
|
||||
})).toThrow(/not the one deployed/);
|
||||
});
|
||||
|
||||
it('will not settle a deploy out of a withdraw, or the reverse', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-cross', 123, 1);
|
||||
tx.intentRevised({ applicationId: 'app-cross', intent: intent('int-23', 'app-cross', 123), envelope: env('op-int-23') });
|
||||
tx.blueprintWithdrawStarted({
|
||||
applicationId: 'app-cross',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-23',
|
||||
envelope: env('op-wd-23'),
|
||||
});
|
||||
|
||||
// Same intent, but it names the deploy this target is not running. Taking
|
||||
// it would claim the deployment is live while it is being torn down.
|
||||
expect(() => tx.blueprintAckRecorded({
|
||||
applicationId: 'app-cross',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-23',
|
||||
rolloutCandidateId: null,
|
||||
legacyAppliedRevision: null,
|
||||
envelope: env('op-ack-23'),
|
||||
})).toThrow(/was not asked to run/);
|
||||
expect(store.getTarget('app-cross', 1)?.target_status).toBe('active');
|
||||
expect(store.getTarget('app-cross', 1)?.active_operation_stage).toBe('blueprint_withdraw_started');
|
||||
});
|
||||
|
||||
it('refuses a start that would displace an unrelated operation', () => {
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-conflict', 124, 1);
|
||||
tx.intentRevised({ applicationId: 'app-conflict', intent: intent('int-24', 'app-conflict', 124), envelope: env('op-int-24') });
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-conflict',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-24',
|
||||
rolloutCandidateId: null,
|
||||
envelope: env('op-dep-24'),
|
||||
});
|
||||
|
||||
// Overwriting would leave the displaced operation with no terminal event
|
||||
// and no history saying it was abandoned.
|
||||
expect(() => tx.blueprintWithdrawStarted({
|
||||
applicationId: 'app-conflict',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-24',
|
||||
envelope: env('op-wd-24-other'),
|
||||
})).toThrow(/conflicting target operation/);
|
||||
});
|
||||
|
||||
it('records an observation without acknowledging or minting anything', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-obs', 108, 1);
|
||||
tx.intentRevised({ applicationId: 'app-obs', intent: intent('int-9', 'app-obs', 108), envelope: env('op-int-9') });
|
||||
const before = store.getApplication('app-obs')!;
|
||||
|
||||
for (const stage of ['blueprint_state_review', 'blueprint_evict_blocked', 'blueprint_drifted', 'blueprint_correcting'] as const) {
|
||||
tx.blueprintObservation({ applicationId: 'app-obs', nodeId: 1, stage, envelope: env(`op-obs-${stage}`) });
|
||||
}
|
||||
|
||||
const after = store.getApplication('app-obs')!;
|
||||
expect(after.intent_revision_id).toBe(before.intent_revision_id);
|
||||
expect(after.rollout_candidate_id).toBe(before.rollout_candidate_id);
|
||||
expect(store.getTarget('app-obs', 1)?.intent_revision_id).toBeNull();
|
||||
});
|
||||
|
||||
it('projects every observation stage as its runtime status', () => {
|
||||
// Recording an observation nothing reads would leave a deployed Blueprint
|
||||
// reporting itself as never applied, which is what the pointers alone say.
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
const expected = {
|
||||
blueprint_state_review: 'pending_state_review',
|
||||
blueprint_evict_blocked: 'evict_blocked',
|
||||
blueprint_drifted: 'drifted',
|
||||
blueprint_correcting: 'correcting',
|
||||
} as const;
|
||||
|
||||
// One live application per Blueprint, so each case needs its own id.
|
||||
Object.entries(expected).forEach(([stage, status], index) => {
|
||||
const applicationId = `app-proj-${stage}`;
|
||||
seedInline(applicationId, 200 + index, 1);
|
||||
tx.blueprintObservation({
|
||||
applicationId,
|
||||
nodeId: 1,
|
||||
stage: stage as keyof typeof expected,
|
||||
envelope: env(`op-proj-${stage}`),
|
||||
});
|
||||
|
||||
expect(runtimeStatusOf(applicationId), stage).toBe(status);
|
||||
});
|
||||
});
|
||||
|
||||
it('stops projecting an observation once something else happens to the target', () => {
|
||||
// The observation is what was seen last, not a state the target is stuck
|
||||
// in. A deploy after it has to win, or a corrected stack reads as drifting
|
||||
// for ever. A deploy start rather than a tombstone, so the runtime
|
||||
// assertion is load-bearing: the tombstone check sits above the observation
|
||||
// branch and would hold whatever `latest_stage` said.
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-superseded', 210, 1);
|
||||
tx.intentRevised({
|
||||
applicationId: 'app-superseded',
|
||||
intent: intent('int-sup', 'app-superseded', 210),
|
||||
envelope: env('op-sup-int'),
|
||||
});
|
||||
tx.blueprintObservation({
|
||||
applicationId: 'app-superseded', nodeId: 1, stage: 'blueprint_drifted', envelope: env('op-sup-obs'),
|
||||
});
|
||||
expect(runtimeStatusOf('app-superseded')).toBe('drifted');
|
||||
|
||||
tx.blueprintDeployStarted({
|
||||
applicationId: 'app-superseded',
|
||||
nodeId: 1,
|
||||
intentRevisionId: 'int-sup',
|
||||
rolloutCandidateId: null,
|
||||
envelope: env('op-sup-deploy'),
|
||||
});
|
||||
|
||||
expect(store.getTarget('app-superseded', 1)?.latest_stage).toBe('blueprint_deploy_started');
|
||||
expect(runtimeStatusOf('app-superseded')).not.toBe('drifted');
|
||||
});
|
||||
|
||||
it('does not let an observation mask a failure this node actually hit', () => {
|
||||
// The ordering claim in the deriver, asserted at its upper boundary. A
|
||||
// failed mutation describes what this node did; an observation describes
|
||||
// what was seen about it. Reporting the observation instead would hide a
|
||||
// deploy that broke the running workload.
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedInline('app-failfirst', 211, 1);
|
||||
tx.deployFailed('app-failfirst', 1, 'post_mutation', env('op-fail'));
|
||||
tx.blueprintObservation({
|
||||
applicationId: 'app-failfirst', nodeId: 1, stage: 'blueprint_drifted', envelope: env('op-fail-obs'),
|
||||
});
|
||||
|
||||
expect(runtimeStatusOf('app-failfirst')).toBe('failed_after_mutation');
|
||||
});
|
||||
});
|
||||
|
||||
function runtimeStatusOf(applicationId: string): string | undefined {
|
||||
const projection = projectApplication(applicationId, false);
|
||||
if (projection.targetMode === 'not_applicable') throw new Error('expected application');
|
||||
return projection.targets[0]?.runtime.status;
|
||||
}
|
||||
|
||||
function seedInline(applicationId: string, blueprintId: number, nodeId?: number): void {
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertApplication(inlineApp(applicationId, blueprintId));
|
||||
if (nodeId !== undefined) {
|
||||
store.upsertTarget(emptyTargetRow(applicationId, nodeId, 1));
|
||||
}
|
||||
}
|
||||
|
||||
function env(operationId: string): EventEnvelope {
|
||||
return { operationId, actor: 'tester', trigger: 'manual', at: Date.now() };
|
||||
}
|
||||
|
||||
function intent(id: string, applicationId: string, blueprintId: number): GitOpsIntentRevisionRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
blueprint_id: blueprintId,
|
||||
compose_content_sha256: 'c'.repeat(64),
|
||||
blueprint_revision: 1,
|
||||
deploy_stack_name: 'bp-stack',
|
||||
selector_json: '{"nodeIds":[1]}',
|
||||
pinned_node_id: null,
|
||||
cordon_implications_json: '{}',
|
||||
rollout_strategy_json: '{}',
|
||||
runtime_drift_policy: null,
|
||||
stateful_policy_json: null,
|
||||
health_failure_rollback_policy_json: null,
|
||||
operation_id: `op-${id}`,
|
||||
actor: 'tester',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function candidate(id: string, applicationId: string, intentRevisionId: string): GitOpsRolloutCandidateRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
intent_revision_id: intentRevisionId,
|
||||
compose_content_sha256: 'c'.repeat(64),
|
||||
accepted_generation_id: null,
|
||||
artifact_set_id: null,
|
||||
required_targets_json: '{"nodeIds":[1]}',
|
||||
authoritative: 1,
|
||||
provenance: 'intent_change',
|
||||
operation_id: `op-${id}`,
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function inlineApp(id: string, blueprintId: number): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `blueprint:${blueprintId}`,
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'inline_blueprint',
|
||||
stack_name: null,
|
||||
blueprint_id: blueprintId,
|
||||
configured_repo_url: null,
|
||||
repo_identity_json: null,
|
||||
configured_ref: null,
|
||||
compose_paths_json: null,
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: null,
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,443 @@
|
||||
/**
|
||||
* Boot-time settlement of creates that a previous process left in flight.
|
||||
*
|
||||
* Each case seeds the durable state a crash would have left at one phase, runs
|
||||
* the recovery the startup sweep runs, and asserts the outcome: finish the
|
||||
* create only when its project is already on disk, tear it down only after its
|
||||
* files are gone, and never touch a source row that outlived the application.
|
||||
*/
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions, type EventEnvelope } from '../services/gitops/transitions';
|
||||
import { assertCreatesSettled, resolveInterruptedCreates } from '../services/gitops/createRecovery';
|
||||
import { candidateRelPathForSha, CREATE_STAGING_MARKER_FILENAME } from '../services/gitops/createStagingMarker';
|
||||
import { stackManagedRoot } from '../services/gitops/directApplication';
|
||||
import type {
|
||||
GitOpsApplicationRow,
|
||||
GitOpsCreateCheckpointRow,
|
||||
GitOpsGenerationRow,
|
||||
} from '../services/gitops/types';
|
||||
|
||||
const SHA = 'feed1234';
|
||||
|
||||
describe('gitops interrupted create recovery', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
db.prepare('DELETE FROM gitops_create_checkpoints').run();
|
||||
db.prepare('DELETE FROM gitops_history').run();
|
||||
db.prepare('DELETE FROM gitops_target_current').run();
|
||||
db.prepare('DELETE FROM gitops_generations').run();
|
||||
db.prepare('DELETE FROM gitops_applications').run();
|
||||
});
|
||||
|
||||
it('tears down a create that stopped before the stack existed', async () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
seedCreate('app-pre', 'pre-stack-web', 'pre_stack');
|
||||
const managedRoot = stackManagedRoot('pre-stack-web');
|
||||
fs.mkdirSync(path.join(managedRoot, candidateRelPathForSha(SHA)), { recursive: true });
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled).toEqual([
|
||||
{ stackName: 'pre-stack-web', applicationId: 'app-pre', outcome: 'tombstoned' },
|
||||
]);
|
||||
expect(store.getApplication('app-pre')?.lifecycle_status).toBe('deleted');
|
||||
expect(store.getCreateCheckpoint('app-pre')).toBeUndefined();
|
||||
expect(store.getLiveDirectApplication('pre-stack-web')).toBeUndefined();
|
||||
expect(fs.existsSync(managedRoot)).toBe(false);
|
||||
});
|
||||
|
||||
it('leaves a stack directory alone when the create never recorded making it', async () => {
|
||||
// pre_stack is durable proof that createStack had not returned, so a
|
||||
// directory present now may be the operator's own. Deleting it is the one
|
||||
// mistake recovery cannot take back.
|
||||
seedCreate('app-notours', 'notours-web', 'pre_stack');
|
||||
const composeDir = process.env.COMPOSE_DIR!;
|
||||
const stackDir = path.join(composeDir, 'notours-web');
|
||||
fs.mkdirSync(stackDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(stackDir, 'compose.yaml'), 'services: {}\n');
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled[0].outcome).toBe('tombstoned');
|
||||
expect(fs.existsSync(path.join(stackDir, 'compose.yaml'))).toBe(true);
|
||||
});
|
||||
|
||||
it('removes the stack directory a crashed create had already made', async () => {
|
||||
seedCreate('app-mid', 'mid-web', 'stack_created');
|
||||
const composeDir = process.env.COMPOSE_DIR!;
|
||||
fs.mkdirSync(path.join(composeDir, 'mid-web'), { recursive: true });
|
||||
fs.writeFileSync(path.join(composeDir, 'mid-web', 'compose.yaml'), 'services: {}\n');
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled[0].outcome).toBe('tombstoned');
|
||||
expect(fs.existsSync(path.join(composeDir, 'mid-web'))).toBe(false);
|
||||
});
|
||||
|
||||
it('preserves a managed root the create did not create', async () => {
|
||||
seedCreate('app-shared', 'shared-web', 'pre_stack', { createdManagedRoot: 0 });
|
||||
const managedRoot = stackManagedRoot('shared-web');
|
||||
const sentinel = path.join(managedRoot, 'generations', 'applied-earlier');
|
||||
fs.mkdirSync(sentinel, { recursive: true });
|
||||
fs.mkdirSync(path.join(managedRoot, candidateRelPathForSha(SHA)), { recursive: true });
|
||||
|
||||
await resolveInterruptedCreates();
|
||||
|
||||
expect(fs.existsSync(sentinel)).toBe(true);
|
||||
expect(fs.existsSync(path.join(managedRoot, candidateRelPathForSha(SHA)))).toBe(false);
|
||||
});
|
||||
|
||||
it('finishes a create whose manifest was already committed on disk', async () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const db = DatabaseService.getInstance();
|
||||
seedCreate('app-finish', 'finish-web', 'manifest_committed');
|
||||
const composeDir = process.env.COMPOSE_DIR!;
|
||||
fs.mkdirSync(path.join(composeDir, 'finish-web'), { recursive: true });
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled[0].outcome).toBe('completed');
|
||||
const app = store.getApplication('app-finish')!;
|
||||
expect(app.lifecycle_status).toBe('active');
|
||||
expect(app.accepted_generation_id).toBe('gen-app-finish');
|
||||
expect(app.source_acceptance_ref).not.toBeNull();
|
||||
expect(store.getTarget('app-finish', 1)?.applied_generation_id).toBe('gen-app-finish');
|
||||
expect(db.getGitSource('finish-web')?.last_applied_commit_sha).toBe(SHA);
|
||||
expect(store.getCreateCheckpoint('app-finish')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('clears the checkpoint of a create that already reached its boundary', async () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
seedCreate('app-done', 'done-web', 'pointers_committed');
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_applications SET lifecycle_status = 'active' WHERE id = 'app-done'",
|
||||
).run();
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled[0].outcome).toBe('checkpoint_cleared');
|
||||
expect(store.getApplication('app-done')?.lifecycle_status).toBe('active');
|
||||
expect(store.getCreateCheckpoint('app-done')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('tombstones a creating application with no checkpoint and keeps its source row', async () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const db = DatabaseService.getInstance();
|
||||
seedCreate('app-orphan', 'orphan-web', 'pre_stack');
|
||||
store.deleteCreateCheckpoint('app-orphan');
|
||||
db.upsertGitSource({
|
||||
stack_name: 'orphan-web',
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
branch: 'main',
|
||||
compose_path: 'compose.yml',
|
||||
compose_paths: ['compose.yml'],
|
||||
context_dir: null,
|
||||
sync_env: false,
|
||||
env_path: null,
|
||||
auth_type: 'none',
|
||||
encrypted_token: null,
|
||||
auto_apply_on_webhook: false,
|
||||
auto_deploy_on_apply: false,
|
||||
last_applied_commit_sha: SHA,
|
||||
last_applied_content_hash: null,
|
||||
pending_commit_sha: null,
|
||||
pending_compose_content: null,
|
||||
pending_env_content: null,
|
||||
pending_fetched_at: null,
|
||||
last_debounce_at: null,
|
||||
});
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled).toEqual([
|
||||
{ stackName: 'orphan-web', applicationId: 'app-orphan', outcome: 'source_preserved' },
|
||||
]);
|
||||
expect(store.getApplication('app-orphan')?.lifecycle_status).toBe('deleted');
|
||||
expect(db.getGitSource('orphan-web')).toBeTruthy();
|
||||
expect(store.getLiveDirectApplication('orphan-web')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('retains a create whose files could not be removed, and refuses to start on it', async () => {
|
||||
// A cleanup that cannot finish must not be recorded as a clean failure: the
|
||||
// application stays `creating`, so nothing downstream may treat the stack
|
||||
// name as free. The failure is forced through the containment guard, which
|
||||
// is a real refusal rather than a stubbed one.
|
||||
const store = GitOpsStore.getInstance();
|
||||
seedCreate('app-stuck', 'stuck-web', 'stack_created', { createdManagedRoot: 0 });
|
||||
store.updateCreateCheckpoint('app-stuck', { generationId: 'gen-app-stuck' }, Date.now());
|
||||
const managedRoot = stackManagedRoot('stuck-web');
|
||||
// Outside the managed area, which is what makes the guard refuse.
|
||||
const external = path.join(process.env.DATA_DIR!, 'external-stuck');
|
||||
fs.mkdirSync(external, { recursive: true });
|
||||
fs.mkdirSync(managedRoot, { recursive: true });
|
||||
fs.symlinkSync(external, path.join(managedRoot, 'generations'), 'junction');
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled).toEqual([
|
||||
{ stackName: 'stuck-web', applicationId: 'app-stuck', outcome: 'retained' },
|
||||
]);
|
||||
// Still creating, and its checkpoint survives so the next boot retries.
|
||||
expect(store.getApplication('app-stuck')?.lifecycle_status).toBe('creating');
|
||||
expect(store.getCreateCheckpoint('app-stuck')).toBeDefined();
|
||||
expect(fs.existsSync(external)).toBe(true);
|
||||
|
||||
// What startup does with that outcome: stop, before any mutation service
|
||||
// starts or HTTP binds.
|
||||
expect(() => assertCreatesSettled(settled)).toThrow(/stuck-web/);
|
||||
});
|
||||
|
||||
it('reports a settled create whose marker survived, and still starts', async () => {
|
||||
// The counterpart to the test above, driven through the real code rather
|
||||
// than a hand-built outcome list. Ownership is decided here, so a marker
|
||||
// file that could not be deleted decides nothing and must not stop a boot.
|
||||
// The marker path is made a directory so the unlink fails while everything
|
||||
// else about the create is already settled.
|
||||
const store = GitOpsStore.getInstance();
|
||||
seedCreate('app-marker', 'marker-web', 'pointers_committed');
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_applications SET lifecycle_status = 'active' WHERE id = 'app-marker'",
|
||||
).run();
|
||||
fs.mkdirSync(path.join(stackManagedRoot('marker-web'), CREATE_STAGING_MARKER_FILENAME), { recursive: true });
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled).toEqual([
|
||||
{ stackName: 'marker-web', applicationId: 'app-marker', outcome: 'marker_retained' },
|
||||
]);
|
||||
// The checkpoint is what makes the next boot retry the marker. Dropping it
|
||||
// would leave a claim on the name with nothing left to clear it.
|
||||
expect(store.getCreateCheckpoint('app-marker')).toBeDefined();
|
||||
expect(() => assertCreatesSettled(settled)).not.toThrow();
|
||||
});
|
||||
|
||||
it('clears the marker before the checkpoint for a create that is no longer creating', async () => {
|
||||
// An application tombstoned on some other path leaves a stale checkpoint
|
||||
// behind. It settles like any other finished create, and it has to clear the
|
||||
// marker on the way out: dropping the checkpoint first would leave a claim
|
||||
// on the stack name with nothing left to retry it, and every later create
|
||||
// for that name would be refused by a marker nothing could remove.
|
||||
const store = GitOpsStore.getInstance();
|
||||
seedCreate('app-gone', 'gone-web', 'stack_created', { createdManagedRoot: 0 });
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_applications SET lifecycle_status = 'deleted' WHERE id = 'app-gone'",
|
||||
).run();
|
||||
// A directory at the marker path, so the unlink fails the way a permission
|
||||
// error would and the ordering becomes observable.
|
||||
fs.mkdirSync(path.join(stackManagedRoot('gone-web'), CREATE_STAGING_MARKER_FILENAME), { recursive: true });
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled).toEqual([
|
||||
{ stackName: 'gone-web', applicationId: 'app-gone', outcome: 'marker_retained' },
|
||||
]);
|
||||
expect(store.getCreateCheckpoint('app-gone')).toBeDefined();
|
||||
expect(() => assertCreatesSettled(settled)).not.toThrow();
|
||||
});
|
||||
|
||||
it('drops the checkpoint for a create that is no longer creating once its marker is clear', async () => {
|
||||
// The same route with nothing blocking the marker: this is the ordinary
|
||||
// outcome, and it must still end with the checkpoint gone.
|
||||
const store = GitOpsStore.getInstance();
|
||||
seedCreate('app-gone-ok', 'gone-ok-web', 'stack_created', { createdManagedRoot: 0 });
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_applications SET lifecycle_status = 'deleted' WHERE id = 'app-gone-ok'",
|
||||
).run();
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled[0].outcome).toBe('checkpoint_cleared');
|
||||
expect(store.getCreateCheckpoint('app-gone-ok')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('reports a marker left by a torn-down create without blocking the boot', async () => {
|
||||
// The teardown path reaches the same condition by a different route. Its
|
||||
// staged directories are gone, so nothing deployable survives and the
|
||||
// create is effectively torn down; only the marker is stuck. Treating that
|
||||
// as unresolved would make one failed unlink cost an operator their
|
||||
// instance, which is the opposite of the settled path's answer.
|
||||
const store = GitOpsStore.getInstance();
|
||||
seedCreate('app-tearmark', 'tearmark-web', 'stack_created', { createdManagedRoot: 0 });
|
||||
fs.mkdirSync(path.join(stackManagedRoot('tearmark-web'), CREATE_STAGING_MARKER_FILENAME), { recursive: true });
|
||||
|
||||
const settled = await resolveInterruptedCreates();
|
||||
|
||||
expect(settled[0].outcome).toBe('marker_retained');
|
||||
expect(store.getCreateCheckpoint('app-tearmark')).toBeDefined();
|
||||
expect(() => assertCreatesSettled(settled)).not.toThrow();
|
||||
});
|
||||
|
||||
it('settles a create when the managed area is not on disk at all', async () => {
|
||||
// A database restored without its data directory, or a volume that failed
|
||||
// to mount. Nothing under the area exists, so there is nothing to remove
|
||||
// and the create tears down normally. Reporting this as unresolved would,
|
||||
// with the boot gate, stop the instance starting on every boot over a
|
||||
// directory that is merely absent.
|
||||
const previous = process.env.DATA_DIR;
|
||||
process.env.DATA_DIR = path.join(tmpDir, 'data-without-managed-area');
|
||||
try {
|
||||
seedCreate('app-noarea', 'noarea-web', 'stack_created', { createdManagedRoot: 0 });
|
||||
const settled = await resolveInterruptedCreates();
|
||||
expect(settled[0].outcome).toBe('tombstoned');
|
||||
expect(() => assertCreatesSettled(settled)).not.toThrow();
|
||||
} finally {
|
||||
process.env.DATA_DIR = previous;
|
||||
}
|
||||
});
|
||||
|
||||
it('is idempotent across repeated boots', async () => {
|
||||
seedCreate('app-replay', 'replay-web', 'pre_stack');
|
||||
const first = await resolveInterruptedCreates();
|
||||
const second = await resolveInterruptedCreates();
|
||||
expect(first[0].outcome).toBe('tombstoned');
|
||||
expect(second).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
function seedCreate(
|
||||
applicationId: string,
|
||||
stackName: string,
|
||||
phase: GitOpsCreateCheckpointRow['phase'],
|
||||
options: { createdManagedRoot?: number } = {},
|
||||
): void {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const generationId = `gen-${applicationId}`;
|
||||
GitOpsTransitions.getInstance().activateCreateFromGit({
|
||||
application: creatingApp(applicationId, stackName),
|
||||
nodeId: 1,
|
||||
commitSha: SHA,
|
||||
generation: gen(generationId, applicationId),
|
||||
checkpoint: {
|
||||
application_id: applicationId,
|
||||
stack_name: stackName,
|
||||
phase: 'pre_stack',
|
||||
generation_id: null,
|
||||
operation_id: `op-${applicationId}`,
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
branch: 'main',
|
||||
compose_path: 'compose.yml',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
auth_type: 'none',
|
||||
encrypted_token: null,
|
||||
auto_apply_on_webhook: 0,
|
||||
auto_deploy_on_apply: 0,
|
||||
commit_sha: SHA,
|
||||
applied_spec_json: null,
|
||||
created_managed_root: options.createdManagedRoot ?? 1,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
},
|
||||
envelope: envelope(`op-${applicationId}`),
|
||||
});
|
||||
if (phase !== 'pre_stack') {
|
||||
store.updateCreateCheckpoint(applicationId, { phase }, Date.now());
|
||||
}
|
||||
}
|
||||
|
||||
function envelope(operationId: string): EventEnvelope {
|
||||
return { operationId, actor: 'tester', trigger: 'manual', at: Date.now() };
|
||||
}
|
||||
|
||||
function creatingApp(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'creating',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function gen(id: string, applicationId: string): GitOpsGenerationRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
commit_sha: SHA,
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
configured_ref: 'main',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
manifest_version: 1,
|
||||
candidate_dir: candidateRelPathForSha(SHA),
|
||||
applied_dir: `generations/applied-${SHA}-1`,
|
||||
expected_invocation_json: '{"composeFileOrder":[],"projectName":null,"projectDirectory":null,"envFileOrder":[]}',
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
validation_ok: 1,
|
||||
plan_blocked: 0,
|
||||
change_plan_fingerprint: null,
|
||||
operation_id: `op-${id}`,
|
||||
trigger: 'manual',
|
||||
actor: 'tester',
|
||||
previous_generation_id: null,
|
||||
redacted_limitations_json: '[]',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,746 @@
|
||||
/**
|
||||
* Create-from-Git durability: the activation transaction, the teardown of a
|
||||
* create that never reached `applied`, and the staging marker plus
|
||||
* operation-owned cleanup that together decide what a crashed create is
|
||||
* allowed to delete.
|
||||
*/
|
||||
import fs from 'fs';
|
||||
import fsPromises from 'fs/promises';
|
||||
import path from 'path';
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions, type EventEnvelope } from '../services/gitops/transitions';
|
||||
import {
|
||||
appliedRelPathFor,
|
||||
candidateRelPathForSha,
|
||||
deleteStagingMarker,
|
||||
CREATE_STAGING_MARKER_FILENAME,
|
||||
readStagingMarker,
|
||||
stagingMarkerPath,
|
||||
writeStagingMarker,
|
||||
CreateStagingMarkerError,
|
||||
} from '../services/gitops/createStagingMarker';
|
||||
import { cleanupUnclaimedManagedRoot, removeOperationOwnedPaths } from '../services/gitops/createCleanup';
|
||||
import { GENERATIONS_DIR, MANAGED_ROOT_NAME, managedAreaBase } from '../services/gitops/managedPaths';
|
||||
import type {
|
||||
GitOpsApplicationRow,
|
||||
GitOpsCreateCheckpointRow,
|
||||
GitOpsGenerationRow,
|
||||
} from '../services/gitops/types';
|
||||
|
||||
const SHA = 'a1b2c3d4';
|
||||
|
||||
describe('gitops create-from-git', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('commits application, fetch, generation, checkpoint, and candidate in one transaction', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
const result = tx.activateCreateFromGit({
|
||||
application: creatingApp('app-create', 'create-web'),
|
||||
nodeId: 1,
|
||||
commitSha: SHA,
|
||||
generation: gen('gen-create', 'app-create'),
|
||||
checkpoint: checkpoint('app-create', 'create-web'),
|
||||
envelope: envelope('op-create'),
|
||||
});
|
||||
|
||||
const app = store.getApplication('app-create')!;
|
||||
expect(app.lifecycle_status).toBe('creating');
|
||||
expect(app.desired_commit_sha).toBe(SHA);
|
||||
expect(app.fetched_commit_sha).toBe(SHA);
|
||||
expect(app.candidate_generation_id).toBe('gen-create');
|
||||
expect(app.accepted_generation_id).toBeNull();
|
||||
expect(app.source_acceptance_ref).toBeNull();
|
||||
|
||||
const target = store.getTarget('app-create', 1)!;
|
||||
expect(target.candidate_generation_id).toBe('gen-create');
|
||||
expect(target.desired_generation_id).toBeNull();
|
||||
expect(target.applied_generation_id).toBeNull();
|
||||
|
||||
expect(store.getCreateCheckpoint('app-create')?.generation_id).toBe('gen-create');
|
||||
expect(store.getCreateCheckpoint('app-create')?.phase).toBe('pre_stack');
|
||||
|
||||
expect(result.historyIds).toHaveLength(3);
|
||||
const stages = DatabaseService.getInstance().getDb().prepare(
|
||||
'SELECT stage FROM gitops_history WHERE application_id = ? ORDER BY rowid ASC',
|
||||
).all('app-create') as Array<{ stage: string }>;
|
||||
expect(stages.map((row) => row.stage)).toEqual(['application_activated', 'fetched', 'candidate_ready']);
|
||||
});
|
||||
|
||||
it('refuses to persist a create whose candidate is blocked or stale', () => {
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
expect(() => tx.activateCreateFromGit({
|
||||
application: creatingApp('app-blocked', 'blocked-web'),
|
||||
nodeId: 1,
|
||||
commitSha: SHA,
|
||||
generation: { ...gen('gen-blocked', 'app-blocked'), plan_blocked: 1 },
|
||||
checkpoint: checkpoint('app-blocked', 'blocked-web'),
|
||||
envelope: envelope('op-blocked'),
|
||||
})).toThrow(/invalid or blocked candidate/);
|
||||
|
||||
expect(() => tx.activateCreateFromGit({
|
||||
application: creatingApp('app-stale', 'stale-web'),
|
||||
nodeId: 1,
|
||||
commitSha: SHA,
|
||||
generation: { ...gen('gen-stale', 'app-stale'), materialization_fingerprint: 'b'.repeat(64) },
|
||||
checkpoint: checkpoint('app-stale', 'stale-web'),
|
||||
envelope: envelope('op-stale'),
|
||||
})).toThrow(/fingerprint/);
|
||||
|
||||
expect(GitOpsStore.getInstance().getApplication('app-blocked')).toBeUndefined();
|
||||
expect(GitOpsStore.getInstance().getApplication('app-stale')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('activates the application only at applied, which is the success boundary', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateCreateFromGit({
|
||||
application: creatingApp('app-boundary', 'boundary-web'),
|
||||
nodeId: 1,
|
||||
commitSha: SHA,
|
||||
generation: gen('gen-boundary', 'app-boundary'),
|
||||
checkpoint: checkpoint('app-boundary', 'boundary-web'),
|
||||
envelope: envelope('op-boundary'),
|
||||
});
|
||||
expect(store.getApplication('app-boundary')?.lifecycle_status).toBe('creating');
|
||||
|
||||
tx.applied({
|
||||
applicationId: 'app-boundary',
|
||||
generationId: 'gen-boundary',
|
||||
artifactSetId: 'art-boundary',
|
||||
sourceAcceptanceId: 'acc-boundary',
|
||||
authority: 'operator',
|
||||
envelope: envelope('op-boundary-applied'),
|
||||
activateCreating: true,
|
||||
});
|
||||
|
||||
const app = store.getApplication('app-boundary')!;
|
||||
expect(app.lifecycle_status).toBe('active');
|
||||
expect(app.accepted_generation_id).toBe('gen-boundary');
|
||||
expect(store.getTarget('app-boundary', 1)?.applied_generation_id).toBe('gen-boundary');
|
||||
|
||||
// After the success boundary the create can no longer be torn down.
|
||||
expect(() => tx.createFailed('app-boundary', 'post_boundary', envelope('op-boundary-fail')))
|
||||
.toThrow(/requires a creating application/);
|
||||
expect(store.getApplication('app-boundary')?.lifecycle_status).toBe('active');
|
||||
});
|
||||
|
||||
it('tombstones a failed create, drops its checkpoint, and frees the stack name', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateCreateFromGit({
|
||||
application: creatingApp('app-fail', 'fail-web'),
|
||||
nodeId: 1,
|
||||
commitSha: SHA,
|
||||
generation: gen('gen-fail', 'app-fail'),
|
||||
checkpoint: checkpoint('app-fail', 'fail-web'),
|
||||
envelope: envelope('op-fail'),
|
||||
});
|
||||
tx.createFailed('app-fail', 'validation', envelope('op-fail'));
|
||||
|
||||
const app = store.getApplication('app-fail')!;
|
||||
expect(app.lifecycle_status).toBe('deleted');
|
||||
expect(app.failure_stage).toBe('create');
|
||||
expect(app.failure_class).toBe('validation');
|
||||
expect(store.getCreateCheckpoint('app-fail')).toBeUndefined();
|
||||
expect(store.getTarget('app-fail', 1)?.target_status).toBe('tombstoned');
|
||||
expect(store.getLiveDirectApplication('fail-web')).toBeUndefined();
|
||||
|
||||
// Retry is a brand new application id against the now-free stack name.
|
||||
tx.activateCreateFromGit({
|
||||
application: creatingApp('app-fail-retry', 'fail-web'),
|
||||
nodeId: 1,
|
||||
commitSha: SHA,
|
||||
generation: gen('gen-fail-retry', 'app-fail-retry'),
|
||||
checkpoint: checkpoint('app-fail-retry', 'fail-web'),
|
||||
envelope: envelope('op-fail-retry'),
|
||||
});
|
||||
expect(store.getLiveDirectApplication('fail-web')?.id).toBe('app-fail-retry');
|
||||
});
|
||||
});
|
||||
|
||||
describe('gitops create staging marker', () => {
|
||||
let root: string;
|
||||
let dataDir: string;
|
||||
let priorDataDir: string | undefined;
|
||||
|
||||
beforeAll(() => {
|
||||
// A managed root only ever lives inside the managed area, and the marker
|
||||
// helpers enforce that at every filesystem call, so the fixture has to be a
|
||||
// real managed area rather than a bare temp directory.
|
||||
priorDataDir = process.env.DATA_DIR;
|
||||
dataDir = fs.mkdtempSync(path.join(process.env.TEMP || '/tmp', 'sencho-marker-'));
|
||||
process.env.DATA_DIR = dataDir;
|
||||
root = managedAreaBase();
|
||||
fs.mkdirSync(root, { recursive: true });
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
if (priorDataDir === undefined) delete process.env.DATA_DIR;
|
||||
else process.env.DATA_DIR = priorDataDir;
|
||||
if (dataDir) fs.rmSync(dataDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function areaFor(name: string): string {
|
||||
return path.join(root, name);
|
||||
}
|
||||
|
||||
it('derives generation paths without depending on import order', () => {
|
||||
// These were briefly built from a constant imported across a module cycle,
|
||||
// which evaluated as undefined and produced `undefined/candidate-<sha>`:
|
||||
// a path that passes containment, names nothing, and makes cleanup a no-op.
|
||||
expect(candidateRelPathForSha('abc123')).toBe('generations/candidate-abc123');
|
||||
expect(appliedRelPathFor('abc123', 2)).toBe('generations/applied-abc123-2');
|
||||
});
|
||||
|
||||
it('round-trips a valid marker and refuses a foreign live marker', async () => {
|
||||
const area = areaFor('round-trip');
|
||||
await writeStagingMarker(area, {
|
||||
schemaVersion: 1,
|
||||
operationId: 'op-1',
|
||||
rootPreexisted: true,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
createdAt: 1,
|
||||
});
|
||||
const read = await readStagingMarker(area);
|
||||
expect(read.state).toBe('valid');
|
||||
if (read.state !== 'valid') throw new Error('expected a valid marker');
|
||||
expect(read.marker.operationId).toBe('op-1');
|
||||
expect(read.marker.candidateRelPath).toBe(`generations/candidate-${SHA}`);
|
||||
|
||||
// Same operation may rewrite its own marker; a different one may not.
|
||||
await writeStagingMarker(area, { ...read.marker, createdAt: 2 });
|
||||
await expect(writeStagingMarker(area, { ...read.marker, operationId: 'op-2' }))
|
||||
.rejects.toBeInstanceOf(CreateStagingMarkerError);
|
||||
|
||||
await deleteStagingMarker(area);
|
||||
expect((await readStagingMarker(area)).state).toBe('missing');
|
||||
});
|
||||
|
||||
it('treats every unsafe candidate path as corrupt', async () => {
|
||||
const cases: Array<[string, unknown]> = [
|
||||
['absolute', path.resolve(root, 'elsewhere')],
|
||||
['dotdot', '../escape'],
|
||||
['empty', ''],
|
||||
['wrong prefix', 'applied/candidate-abc'],
|
||||
['escape', 'generations/candidate-../../../etc'],
|
||||
['null', null],
|
||||
];
|
||||
for (const [label, candidateRelPath] of cases) {
|
||||
const area = areaFor(`corrupt-${label.replace(/\s/g, '-')}`);
|
||||
await fsPromises.mkdir(area, { recursive: true });
|
||||
await fsPromises.writeFile(
|
||||
stagingMarkerPath(area),
|
||||
JSON.stringify({ schemaVersion: 1, operationId: 'op-x', rootPreexisted: true, candidateRelPath, createdAt: 1 }),
|
||||
'utf8',
|
||||
);
|
||||
const read = await readStagingMarker(area);
|
||||
expect(read.state, `${label} should be corrupt`).toBe('corrupt');
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects a marker with a bad schema version or missing fields', async () => {
|
||||
const area = areaFor('bad-shape');
|
||||
await fsPromises.mkdir(area, { recursive: true });
|
||||
await fsPromises.writeFile(stagingMarkerPath(area), '{"schemaVersion":2}', 'utf8');
|
||||
expect((await readStagingMarker(area)).state).toBe('corrupt');
|
||||
await fsPromises.writeFile(stagingMarkerPath(area), 'not json', 'utf8');
|
||||
expect((await readStagingMarker(area)).state).toBe('corrupt');
|
||||
});
|
||||
|
||||
it('refuses to claim an area whose marker cannot be read', async () => {
|
||||
// A marker that exists but will not parse is still someone's claim.
|
||||
// Writing over it would hand this operation deletion authority over what
|
||||
// the last one staged.
|
||||
const area = areaFor('unreadable-claim');
|
||||
await fsPromises.mkdir(area, { recursive: true });
|
||||
await fsPromises.writeFile(stagingMarkerPath(area), 'not json', 'utf8');
|
||||
await expect(writeStagingMarker(area, {
|
||||
schemaVersion: 1,
|
||||
operationId: 'op-new',
|
||||
rootPreexisted: true,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
createdAt: 1,
|
||||
})).rejects.toThrow(/unreadable staging marker/);
|
||||
});
|
||||
|
||||
it('refuses every marker operation on a root outside the managed area', async () => {
|
||||
// The stack name reaches this root without being validated here, so each
|
||||
// call checks containment itself. Without these the checks are deletable
|
||||
// and nothing notices.
|
||||
const outside = path.join(dataDir, 'not-the-managed-area', 'web');
|
||||
await fsPromises.mkdir(outside, { recursive: true });
|
||||
|
||||
const read = await readStagingMarker(outside);
|
||||
expect(read.state).toBe('corrupt');
|
||||
if (read.state !== 'corrupt') throw new Error('expected a corrupt result');
|
||||
expect(read.reason).toMatch(/managed area/);
|
||||
|
||||
await expect(writeStagingMarker(outside, {
|
||||
schemaVersion: 1,
|
||||
operationId: 'op-outside',
|
||||
rootPreexisted: true,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
createdAt: 1,
|
||||
})).rejects.toBeInstanceOf(CreateStagingMarkerError);
|
||||
|
||||
await expect(deleteStagingMarker(outside)).rejects.toBeInstanceOf(CreateStagingMarkerError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('gitops create cleanup', () => {
|
||||
let root: string;
|
||||
let dataDir: string;
|
||||
let priorDataDir: string | undefined;
|
||||
|
||||
beforeAll(() => {
|
||||
// Same as the marker describe: cleanup refuses to touch anything outside
|
||||
// the managed area, so the fixture areas have to live inside one.
|
||||
priorDataDir = process.env.DATA_DIR;
|
||||
dataDir = fs.mkdtempSync(path.join(process.env.TEMP || '/tmp', 'sencho-cleanup-'));
|
||||
process.env.DATA_DIR = dataDir;
|
||||
root = managedAreaBase();
|
||||
fs.mkdirSync(root, { recursive: true });
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
if (priorDataDir === undefined) delete process.env.DATA_DIR;
|
||||
else process.env.DATA_DIR = priorDataDir;
|
||||
if (dataDir) fs.rmSync(dataDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
async function seedArea(name: string): Promise<{ area: string; candidateRel: string; sentinel: string }> {
|
||||
const area = path.join(root, name);
|
||||
const candidateRel = candidateRelPathForSha(SHA);
|
||||
await fsPromises.mkdir(path.join(area, candidateRel), { recursive: true });
|
||||
const sentinel = path.join(area, 'generations', 'applied-old');
|
||||
await fsPromises.mkdir(sentinel, { recursive: true });
|
||||
return { area, candidateRel, sentinel };
|
||||
}
|
||||
|
||||
it('removes only the staged candidate when the managed root pre-existed', async () => {
|
||||
const { area, candidateRel, sentinel } = await seedArea('preexisting');
|
||||
await removeOperationOwnedPaths({ stackManagedRoot: area, candidateRelPath: candidateRel, ownsManagedRoot: false });
|
||||
expect(fs.existsSync(path.join(area, candidateRel))).toBe(false);
|
||||
expect(fs.existsSync(sentinel)).toBe(true);
|
||||
expect(fs.existsSync(area)).toBe(true);
|
||||
});
|
||||
|
||||
it('removes the whole root only when the operation created it', async () => {
|
||||
const { area, candidateRel } = await seedArea('owned');
|
||||
await removeOperationOwnedPaths({ stackManagedRoot: area, candidateRelPath: candidateRel, ownsManagedRoot: true });
|
||||
expect(fs.existsSync(area)).toBe(false);
|
||||
});
|
||||
|
||||
it('refuses to remove a managed root outside the managed area', async () => {
|
||||
// The guard that keeps a recursive removal inside the area it is meant to
|
||||
// clean. Without a test it is deletable and nothing notices.
|
||||
const outside = path.join(dataDir, 'not-the-managed-area', 'web');
|
||||
fs.mkdirSync(outside, { recursive: true });
|
||||
await expect(removeOperationOwnedPaths({
|
||||
stackManagedRoot: outside,
|
||||
candidateRelPath: null,
|
||||
ownsManagedRoot: true,
|
||||
})).rejects.toThrow(/outside the managed area/);
|
||||
expect(fs.existsSync(outside)).toBe(true);
|
||||
|
||||
// The reaper reports rather than throws, so it answers `preserved`.
|
||||
expect(await cleanupUnclaimedManagedRoot(outside, {
|
||||
operationId: 'op-outside',
|
||||
rootPreexisted: false,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
})).toBe('preserved');
|
||||
expect(fs.existsSync(outside)).toBe(true);
|
||||
});
|
||||
|
||||
it('refuses to remove a path outside the managed root', async () => {
|
||||
const { area } = await seedArea('escape-guard');
|
||||
await expect(removeOperationOwnedPaths({
|
||||
stackManagedRoot: area,
|
||||
candidateRelPath: '../../outside',
|
||||
ownsManagedRoot: false,
|
||||
})).rejects.toThrow(/outside the managed root/);
|
||||
});
|
||||
|
||||
/**
|
||||
* A directory link that lands outside the managed area.
|
||||
*
|
||||
* `junction` is what Windows can create without elevation, and Node ignores
|
||||
* the type argument everywhere else, so one call covers both platforms.
|
||||
*/
|
||||
async function linkOutside(linkPath: string, name: string, victimRelPath?: string): Promise<string> {
|
||||
const external = path.join(dataDir, 'external', name);
|
||||
await fsPromises.mkdir(external, { recursive: true });
|
||||
await fsPromises.writeFile(path.join(external, 'keepme.txt'), 'not ours', 'utf8');
|
||||
// The path the escaping delete would actually resolve to. Without content
|
||||
// at exactly that path the removal is a no-op even unguarded, and the
|
||||
// survival assertion would pass against the unfixed code too.
|
||||
if (victimRelPath) {
|
||||
const victim = path.join(external, victimRelPath);
|
||||
await fsPromises.mkdir(victim, { recursive: true });
|
||||
await fsPromises.writeFile(path.join(victim, 'victim.txt'), 'would have been deleted', 'utf8');
|
||||
}
|
||||
await fsPromises.mkdir(path.dirname(linkPath), { recursive: true });
|
||||
await fsPromises.symlink(external, linkPath, 'junction');
|
||||
return external;
|
||||
}
|
||||
|
||||
it('refuses to remove a path whose parent links out of the managed area', async () => {
|
||||
// The lexical checks all pass here: `<area>/generations/candidate-*` reads
|
||||
// as contained no matter what `generations` points at. Containment has to
|
||||
// be proven against the real filesystem, because the recursive delete is
|
||||
// what follows the link.
|
||||
const area = path.join(root, 'junction-parent');
|
||||
const candidateRel = candidateRelPathForSha(SHA);
|
||||
const external = await linkOutside(path.join(area, 'generations'), 'parent-escape', `candidate-${SHA}`);
|
||||
|
||||
await expect(removeOperationOwnedPaths({
|
||||
stackManagedRoot: area,
|
||||
candidateRelPath: candidateRel,
|
||||
ownsManagedRoot: false,
|
||||
})).rejects.toThrow(/links outside its managed location/);
|
||||
// The path the escaping delete would have resolved to, not just a bystander
|
||||
// file: this is the data an unguarded removal destroys.
|
||||
expect(fs.existsSync(path.join(external, `candidate-${SHA}`, 'victim.txt'))).toBe(true);
|
||||
expect(fs.existsSync(path.join(external, 'keepme.txt'))).toBe(true);
|
||||
});
|
||||
|
||||
it('refuses to remove a managed root that is itself a link out of the area', async () => {
|
||||
const area = path.join(root, 'junction-root');
|
||||
const external = await linkOutside(area, 'root-escape');
|
||||
|
||||
await expect(removeOperationOwnedPaths({
|
||||
stackManagedRoot: area,
|
||||
candidateRelPath: null,
|
||||
ownsManagedRoot: true,
|
||||
})).rejects.toThrow(/links outside its managed location/);
|
||||
expect(fs.existsSync(path.join(external, 'keepme.txt'))).toBe(true);
|
||||
// The link itself survives too. Unlinking it is the damage an unguarded
|
||||
// delete does here, and it is the operator's own relocation pointer.
|
||||
expect(fs.existsSync(area)).toBe(true);
|
||||
|
||||
// The boot sweep reaches the same root by a different route and must reach
|
||||
// the same answer, reporting rather than throwing as it does everywhere.
|
||||
expect(await cleanupUnclaimedManagedRoot(area, {
|
||||
operationId: 'op-junction',
|
||||
rootPreexisted: false,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
})).toBe('preserved');
|
||||
expect(fs.existsSync(path.join(external, 'keepme.txt'))).toBe(true);
|
||||
expect(fs.existsSync(area)).toBe(true);
|
||||
});
|
||||
|
||||
/**
|
||||
* A directory link that stays *inside* the managed area but lands in another
|
||||
* stack's subtree.
|
||||
*
|
||||
* The area-membership check cannot see this: the link's target is a real path
|
||||
* under the managed area, so "is this inside the area" answers yes while the
|
||||
* delete walks into a generation that belongs to someone else. Containment has
|
||||
* to be proven against the path's own place in the area, not the area itself.
|
||||
*/
|
||||
it('refuses to remove a candidate reached through a junction into a sibling stack', async () => {
|
||||
const victim = await seedArea('sibling-victim');
|
||||
const attacker = path.join(root, 'sibling-attacker');
|
||||
await fsPromises.mkdir(attacker, { recursive: true });
|
||||
await fsPromises.symlink(
|
||||
path.join(victim.area, GENERATIONS_DIR),
|
||||
path.join(attacker, GENERATIONS_DIR),
|
||||
'junction',
|
||||
);
|
||||
|
||||
await expect(removeOperationOwnedPaths({
|
||||
stackManagedRoot: attacker,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
ownsManagedRoot: false,
|
||||
})).rejects.toThrow(/links outside its managed location/);
|
||||
// The other stack's staged generation, which an area-only guard removes.
|
||||
expect(fs.existsSync(path.join(victim.area, victim.candidateRel))).toBe(true);
|
||||
expect(fs.existsSync(victim.sentinel)).toBe(true);
|
||||
});
|
||||
|
||||
it('refuses to remove a managed root that junctions into another node subtree', async () => {
|
||||
// Mirrors the production layout `<area>/<nodeId>/<stackName>`, because the
|
||||
// node segment is the one an area-only guard also fails to pin.
|
||||
const victimRoot = path.join(root, 'node-2', 'shared-name');
|
||||
const victimGeneration = path.join(victimRoot, candidateRelPathForSha(SHA));
|
||||
await fsPromises.mkdir(victimGeneration, { recursive: true });
|
||||
|
||||
const attackerRoot = path.join(root, 'node-1', 'shared-name');
|
||||
await fsPromises.mkdir(path.dirname(attackerRoot), { recursive: true });
|
||||
await fsPromises.symlink(victimRoot, attackerRoot, 'junction');
|
||||
|
||||
await expect(removeOperationOwnedPaths({
|
||||
stackManagedRoot: attackerRoot,
|
||||
candidateRelPath: null,
|
||||
ownsManagedRoot: true,
|
||||
})).rejects.toThrow(/links outside its managed location/);
|
||||
expect(fs.existsSync(victimGeneration)).toBe(true);
|
||||
|
||||
// The boot sweep reaches the same root by another route and must agree.
|
||||
expect(await cleanupUnclaimedManagedRoot(attackerRoot, {
|
||||
operationId: 'op-sibling-node',
|
||||
rootPreexisted: false,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
})).toBe('preserved');
|
||||
expect(fs.existsSync(victimGeneration)).toBe(true);
|
||||
});
|
||||
|
||||
it('refuses to write or delete a staging marker through a junction into a sibling stack', async () => {
|
||||
// The write sink needs the same rule as the delete: a marker written into
|
||||
// another stack's root would hand this operation deletion authority there,
|
||||
// and would overwrite the claim that stack is relying on.
|
||||
const victim = path.join(root, 'sibling-marker-victim');
|
||||
await fsPromises.mkdir(victim, { recursive: true });
|
||||
const attacker = path.join(root, 'sibling-marker-attacker');
|
||||
await fsPromises.symlink(victim, attacker, 'junction');
|
||||
|
||||
// No marker at the victim yet, so the write reaches the containment check
|
||||
// rather than being turned back by the "someone already owns this" guard.
|
||||
await expect(writeStagingMarker(attacker, {
|
||||
schemaVersion: 1,
|
||||
operationId: 'op-sibling-marker',
|
||||
rootPreexisted: false,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
createdAt: 1,
|
||||
})).rejects.toThrow(/links outside its managed location/);
|
||||
expect(fs.existsSync(path.join(victim, CREATE_STAGING_MARKER_FILENAME))).toBe(false);
|
||||
|
||||
// Now the victim holds its own claim, and the delete must not clear it:
|
||||
// that claim is what stops a second create racing this stack.
|
||||
await fsPromises.writeFile(path.join(victim, CREATE_STAGING_MARKER_FILENAME), 'theirs', 'utf8');
|
||||
await expect(deleteStagingMarker(attacker)).rejects.toThrow(/links outside its managed location/);
|
||||
expect(fs.readFileSync(path.join(victim, CREATE_STAGING_MARKER_FILENAME), 'utf8')).toBe('theirs');
|
||||
});
|
||||
|
||||
it('refuses to write a staging marker through a link out of the area', async () => {
|
||||
// The write sink gets the same barrier as the delete. Without it a marker
|
||||
// could be written through a link and then refused by the hardened delete,
|
||||
// wedging the stack name behind a claim nothing could clear.
|
||||
const area = path.join(root, 'junction-write');
|
||||
const external = await linkOutside(area, 'write-escape');
|
||||
|
||||
await expect(writeStagingMarker(area, {
|
||||
schemaVersion: 1,
|
||||
operationId: 'op-write-escape',
|
||||
rootPreexisted: false,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
createdAt: 1,
|
||||
})).rejects.toThrow(/links outside its managed location/);
|
||||
expect(fs.existsSync(path.join(external, CREATE_STAGING_MARKER_FILENAME))).toBe(false);
|
||||
});
|
||||
|
||||
it('refuses to delete a staging marker through a link out of the area', async () => {
|
||||
// Reached only through the real-path barrier: the marker path is lexically
|
||||
// inside the area, so every string check passes.
|
||||
const area = path.join(root, 'junction-marker');
|
||||
const external = await linkOutside(area, 'marker-escape');
|
||||
await fsPromises.writeFile(path.join(external, CREATE_STAGING_MARKER_FILENAME), '{}', 'utf8');
|
||||
|
||||
await expect(deleteStagingMarker(area)).rejects.toThrow(/links outside its managed location/);
|
||||
expect(fs.existsSync(path.join(external, CREATE_STAGING_MARKER_FILENAME))).toBe(true);
|
||||
});
|
||||
|
||||
it('treats a managed area that does not exist as nothing to remove', async () => {
|
||||
// A database restored without its data directory, or a volume that failed
|
||||
// to mount. Every path under the area is absent, so a forced remove is a
|
||||
// no-op. Refusing here instead would make an absent directory look like a
|
||||
// link escape and, with the boot gate, stop the instance starting at all.
|
||||
const missingData = path.join(dataDir, 'no-area-here');
|
||||
const previous = process.env.DATA_DIR;
|
||||
process.env.DATA_DIR = missingData;
|
||||
try {
|
||||
const area = path.join(managedAreaBase(), 'ghost-stack');
|
||||
await expect(removeOperationOwnedPaths({
|
||||
stackManagedRoot: area,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
ownsManagedRoot: false,
|
||||
})).resolves.toBe('cleared');
|
||||
} finally {
|
||||
process.env.DATA_DIR = previous;
|
||||
}
|
||||
});
|
||||
|
||||
it('still cleans up when the managed area itself is relocated onto a link', async () => {
|
||||
// The counterpart to the two tests above: an operator who points the data
|
||||
// directory at another volume has moved the whole area rather than escaped
|
||||
// it, and cleanup must keep working for them.
|
||||
const relocatedData = path.join(dataDir, 'relocated-data');
|
||||
const storage = path.join(dataDir, 'other-volume');
|
||||
await fsPromises.mkdir(relocatedData, { recursive: true });
|
||||
await fsPromises.mkdir(storage, { recursive: true });
|
||||
await fsPromises.symlink(storage, path.join(relocatedData, MANAGED_ROOT_NAME), 'junction');
|
||||
|
||||
const previous = process.env.DATA_DIR;
|
||||
process.env.DATA_DIR = relocatedData;
|
||||
try {
|
||||
const area = path.join(managedAreaBase(), 'relocated-stack');
|
||||
const candidateRel = candidateRelPathForSha(SHA);
|
||||
await fsPromises.mkdir(path.join(area, candidateRel), { recursive: true });
|
||||
|
||||
await removeOperationOwnedPaths({ stackManagedRoot: area, candidateRelPath: candidateRel, ownsManagedRoot: false });
|
||||
expect(fs.existsSync(path.join(area, candidateRel))).toBe(false);
|
||||
} finally {
|
||||
process.env.DATA_DIR = previous;
|
||||
}
|
||||
});
|
||||
|
||||
it('preserves an unclaimed root whose marker is missing or corrupt', async () => {
|
||||
const { area, sentinel } = await seedArea('unclaimed');
|
||||
expect(await cleanupUnclaimedManagedRoot(area, null)).toBe('preserved');
|
||||
expect(fs.existsSync(sentinel)).toBe(true);
|
||||
|
||||
expect(await cleanupUnclaimedManagedRoot(area, {
|
||||
operationId: 'op-x',
|
||||
rootPreexisted: true,
|
||||
candidateRelPath: '../escape',
|
||||
})).toBe('preserved');
|
||||
expect(fs.existsSync(sentinel)).toBe(true);
|
||||
});
|
||||
|
||||
it('applies operation-owned cleanup for an unclaimed root with a valid marker', async () => {
|
||||
const preexisting = await seedArea('unclaimed-preexisting');
|
||||
expect(await cleanupUnclaimedManagedRoot(preexisting.area, {
|
||||
operationId: 'op-x',
|
||||
rootPreexisted: true,
|
||||
candidateRelPath: preexisting.candidateRel,
|
||||
})).toBe('removed_candidate');
|
||||
expect(fs.existsSync(path.join(preexisting.area, preexisting.candidateRel))).toBe(false);
|
||||
expect(fs.existsSync(preexisting.sentinel)).toBe(true);
|
||||
|
||||
const owned = await seedArea('unclaimed-owned');
|
||||
expect(await cleanupUnclaimedManagedRoot(owned.area, {
|
||||
operationId: 'op-x',
|
||||
rootPreexisted: false,
|
||||
candidateRelPath: owned.candidateRel,
|
||||
})).toBe('removed_root');
|
||||
expect(fs.existsSync(owned.area)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
function envelope(operationId: string): EventEnvelope {
|
||||
return { operationId, actor: 'tester', trigger: 'manual', at: Date.now() };
|
||||
}
|
||||
|
||||
function checkpoint(applicationId: string, stackName: string): GitOpsCreateCheckpointRow {
|
||||
return {
|
||||
application_id: applicationId,
|
||||
stack_name: stackName,
|
||||
phase: 'pre_stack',
|
||||
generation_id: null,
|
||||
operation_id: `op-${applicationId}`,
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
branch: 'main',
|
||||
compose_path: 'compose.yml',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
auth_type: 'none',
|
||||
encrypted_token: null,
|
||||
auto_apply_on_webhook: 0,
|
||||
auto_deploy_on_apply: 0,
|
||||
commit_sha: SHA,
|
||||
applied_spec_json: null,
|
||||
created_managed_root: 1,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function creatingApp(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'creating',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function gen(id: string, applicationId: string): GitOpsGenerationRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
commit_sha: SHA,
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
configured_ref: 'main',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
manifest_version: 0,
|
||||
candidate_dir: candidateRelPathForSha(SHA),
|
||||
applied_dir: `generations/applied-${id}-0`,
|
||||
expected_invocation_json: '{"composeFileOrder":[],"projectName":null,"projectDirectory":null,"envFileOrder":[]}',
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
validation_ok: 1,
|
||||
plan_blocked: 0,
|
||||
change_plan_fingerprint: null,
|
||||
operation_id: `op-${id}`,
|
||||
trigger: 'manual',
|
||||
actor: 'tester',
|
||||
previous_generation_id: null,
|
||||
redacted_limitations_json: '[]',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,359 @@
|
||||
/**
|
||||
* Deferred-state events: retry, suspend, pause, and partial rollout.
|
||||
*
|
||||
* These have no production writer by design; later tickets emit them. They are
|
||||
* implemented and tested now so the deriver has no branch a writer cannot
|
||||
* reach, and so the shape a future producer must satisfy is pinned rather than
|
||||
* inferred from the deriver.
|
||||
*
|
||||
* The rule they all share is that none of them is a statement about health. A
|
||||
* suspended source, a paused rollout, and a partial rollout each leave every
|
||||
* success pointer exactly where it was.
|
||||
*/
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions, type EventEnvelope } from '../services/gitops/transitions';
|
||||
import { projectApplication } from '../services/gitops/derive';
|
||||
import type { GitOpsApplicationRow, GitOpsGenerationRow } from '../services/gitops/types';
|
||||
|
||||
describe('gitops deferred state', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('schedules a retry without hiding the failure that caused it', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-retry', 'retry-web');
|
||||
tx.fetchStarted('app-retry', env('op-retry-f'));
|
||||
tx.fetchFailed('app-retry', env('op-retry-f'));
|
||||
|
||||
tx.sourceRetryScheduled('app-retry', 5_000, 2, env('op-retry-s'));
|
||||
|
||||
const app = store.getApplication('app-retry')!;
|
||||
expect(app.retry_at).toBe(5_000);
|
||||
expect(app.retry_count).toBe(2);
|
||||
// A retry is a plan, not a resolution: a stack that keeps failing must not
|
||||
// read as merely busy.
|
||||
expect(app.failure_stage).toBe('fetch');
|
||||
expect(projectOf('app-retry').facets.source.status).toBe('source_failed');
|
||||
|
||||
// Starting the retry clears the schedule and keeps the count.
|
||||
tx.fetchStarted('app-retry', env('op-retry-f2'));
|
||||
expect(store.getApplication('app-retry')?.retry_at).toBeNull();
|
||||
expect(store.getApplication('app-retry')?.retry_count).toBe(2);
|
||||
});
|
||||
|
||||
it('suspends a source without forgetting what it had accepted', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-susp', 'susp-web');
|
||||
const accepted = store.getApplication('app-susp')!.accepted_generation_id;
|
||||
|
||||
tx.sourceSuspended('app-susp', 'operator paused sync', env('op-susp'));
|
||||
|
||||
const app = store.getApplication('app-susp')!;
|
||||
expect(app.suspended_at).not.toBeNull();
|
||||
expect(app.accepted_generation_id).toBe(accepted);
|
||||
expect(projectOf('app-susp').facets.source.status).toBe('source_suspended');
|
||||
// A suspended source refuses new work rather than queueing it.
|
||||
expect(() => tx.fetchStarted('app-susp', env('op-susp-f'))).toThrow(/suspended/);
|
||||
|
||||
tx.sourceUnsuspended('app-susp', env('op-unsusp'));
|
||||
expect(store.getApplication('app-susp')?.suspended_at).toBeNull();
|
||||
expect(projectOf('app-susp').facets.source.status).toBe('application_generation_accepted');
|
||||
});
|
||||
|
||||
it('interrupts an operation in flight when the source is suspended', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-susp2', 'susp2-web');
|
||||
tx.fetchStarted('app-susp2', env('op-susp2-f'));
|
||||
|
||||
tx.sourceSuspended('app-susp2', 'operator paused sync', env('op-susp2'));
|
||||
|
||||
const app = store.getApplication('app-susp2')!;
|
||||
// Abandoning the operation without recording it would leave the source
|
||||
// reporting a fetch in flight that nothing will ever finish.
|
||||
expect(app.active_operation_stage).toBeNull();
|
||||
expect(app.interruption_stage).toBe('fetch_started');
|
||||
expect(app.suspended_at).not.toBeNull();
|
||||
});
|
||||
|
||||
it('pauses a rollout without claiming anything about health', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-pause', 'pause-web');
|
||||
tx.deployStarted('app-pause', 1, 'gen-app-pause', env('op-pause-d'));
|
||||
tx.deployBound('app-pause', 1, 'gen-app-pause', env('op-pause-d'));
|
||||
|
||||
tx.rolloutPaused('app-pause', 1, 'awaiting approval', env('op-pause'));
|
||||
|
||||
const target = store.getTarget('app-pause', 1)!;
|
||||
expect(target.pause_at).not.toBeNull();
|
||||
// What was deployed is still deployed.
|
||||
expect(target.deployed_generation_id).toBe('gen-app-pause');
|
||||
expect(projectOf('app-pause').targets[0]?.runtime.status).toBe('paused');
|
||||
|
||||
tx.rolloutUnpaused('app-pause', 1, env('op-unpause'));
|
||||
expect(store.getTarget('app-pause', 1)?.pause_at).toBeNull();
|
||||
expect(projectOf('app-pause').targets[0]?.runtime.status).not.toBe('paused');
|
||||
});
|
||||
|
||||
it('records a partial rollout without inventing a deployed pointer', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-partial', 'partial-web');
|
||||
|
||||
tx.partiallyRolledOut('app-partial', 1, '{"reached":[1],"pending":[2]}', env('op-partial'));
|
||||
|
||||
const target = store.getTarget('app-partial', 1)!;
|
||||
expect(target.partial_json).toBe('{"reached":[1],"pending":[2]}');
|
||||
expect(target.deployed_generation_id).toBeNull();
|
||||
expect(projectOf('app-partial').targets[0]?.runtime.status).toBe('partially_rolled_out');
|
||||
|
||||
tx.partialCleared('app-partial', 1, env('op-partial-clear'));
|
||||
expect(store.getTarget('app-partial', 1)?.partial_json).toBeNull();
|
||||
});
|
||||
|
||||
it('refuses partial state that is not decodable', () => {
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-partial-bad', 'partial-bad-web');
|
||||
expect(() => tx.partiallyRolledOut('app-partial-bad', 1, 'not json', env('op-partial-bad')))
|
||||
.toThrow();
|
||||
});
|
||||
|
||||
it('reports a rollback in flight on both the application and the target', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-rb-start', 'rb-start-web');
|
||||
const generationId = 'gen-app-rb-start';
|
||||
|
||||
tx.rollbackInProgress({
|
||||
applicationId: 'app-rb-start',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rb-1',
|
||||
recoveryGenerationId: generationId,
|
||||
envelope: env('op-rb-start'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-rb-start', 1)!;
|
||||
expect(target.recovery_phase).toBe('restoring');
|
||||
expect(target.recovery_ref).toBe('rb-1');
|
||||
expect(target.recovery_generation_id).toBe(generationId);
|
||||
// Written to both, because a target-only write left the source facet
|
||||
// reporting whatever the source last did instead of the rollback.
|
||||
expect(store.getApplication('app-rb-start')?.recovery_phase).toBe('restoring');
|
||||
expect(projectOf('app-rb-start').facets.rollout.status).toBe('rollback_in_progress');
|
||||
});
|
||||
|
||||
it('persists the failure class a partial rollback was given', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-rb-partial', 'rb-partial-web');
|
||||
const applied = store.getTarget('app-rb-partial', 1)!.applied_generation_id;
|
||||
|
||||
tx.rollbackPartialFailed({
|
||||
applicationId: 'app-rb-partial',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rb-2',
|
||||
failureClass: 'partial',
|
||||
envelope: env('op-rb-partial'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-rb-partial', 1)!;
|
||||
expect(target.recovery_phase).toBe('failed');
|
||||
expect(target.failure_stage).toBe('recovery');
|
||||
// Reported verbatim: the deriver reads these columns rather than inventing
|
||||
// a class, and `partial` is the one this alias adds over a recovery.
|
||||
expect(target.failure_class).toBe('partial');
|
||||
// A failed rollback moves no success pointer.
|
||||
expect(target.applied_generation_id).toBe(applied);
|
||||
expect(target.healthy_generation_id).toBeNull();
|
||||
});
|
||||
|
||||
it('completes a rollback only against a generation it can prove', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-rb-done', 'rb-done-web');
|
||||
const generationId = 'gen-app-rb-done';
|
||||
|
||||
// Nothing bound yet, so there is no generation to complete against.
|
||||
expect(() => tx.rollbackCompleted({
|
||||
applicationId: 'app-rb-done',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rb-3',
|
||||
capturedArtifactSetId: null,
|
||||
capturedSourceAcceptanceRef: null,
|
||||
envelope: env('op-rb-done-early'),
|
||||
})).toThrow(/bound recovery generation/);
|
||||
|
||||
tx.rollbackInProgress({
|
||||
applicationId: 'app-rb-done',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rb-3',
|
||||
recoveryGenerationId: generationId,
|
||||
envelope: env('op-rb-done-start'),
|
||||
});
|
||||
tx.rollbackCompleted({
|
||||
applicationId: 'app-rb-done',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rb-3',
|
||||
capturedArtifactSetId: null,
|
||||
capturedSourceAcceptanceRef: null,
|
||||
envelope: env('op-rb-done'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-rb-done', 1)!;
|
||||
expect(target.recovery_phase).toBe('complete');
|
||||
expect(target.desired_generation_id).toBe(generationId);
|
||||
expect(target.applied_generation_id).toBe(generationId);
|
||||
// The workload is back but nothing has observed it yet.
|
||||
expect(target.healthy_generation_id).toBeNull();
|
||||
});
|
||||
|
||||
it('refuses to complete a rollback onto another application generation', () => {
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-rb-foreign', 'rb-foreign-web');
|
||||
seedApplied('app-rb-owner', 'rb-owner-web');
|
||||
|
||||
tx.rollbackInProgress({
|
||||
applicationId: 'app-rb-foreign',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rb-4',
|
||||
recoveryGenerationId: 'gen-app-rb-owner',
|
||||
envelope: env('op-rb-foreign-start'),
|
||||
});
|
||||
|
||||
expect(() => tx.rollbackCompleted({
|
||||
applicationId: 'app-rb-foreign',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rb-4',
|
||||
capturedArtifactSetId: null,
|
||||
capturedSourceAcceptanceRef: null,
|
||||
envelope: env('op-rb-foreign'),
|
||||
})).toThrow(/does not own/);
|
||||
});
|
||||
});
|
||||
|
||||
function projectOf(applicationId: string) {
|
||||
const projection = projectApplication(applicationId, true);
|
||||
if (projection.targetMode === 'not_applicable') throw new Error('expected an application');
|
||||
return projection;
|
||||
}
|
||||
|
||||
function seedApplied(applicationId: string, stackName: string): void {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
const generationId = `gen-${applicationId}`;
|
||||
tx.activateDirect({ application: app(applicationId, stackName), nodeId: 1, envelope: env(`op-act-${applicationId}`) });
|
||||
store.insertGeneration(gen(generationId, applicationId));
|
||||
tx.fetchStarted(applicationId, env(`op-f-${applicationId}`));
|
||||
tx.fetched(applicationId, 'abc123', env(`op-f-${applicationId}`));
|
||||
tx.candidateReady(applicationId, generationId, false, env(`op-c-${applicationId}`));
|
||||
tx.applied({
|
||||
applicationId,
|
||||
generationId,
|
||||
artifactSetId: `art-${applicationId}`,
|
||||
sourceAcceptanceId: `acc-${applicationId}`,
|
||||
authority: 'operator',
|
||||
envelope: env(`op-a-${applicationId}`),
|
||||
});
|
||||
}
|
||||
|
||||
function env(operationId: string): EventEnvelope {
|
||||
return { operationId, actor: 'tester', trigger: 'manual', at: Date.now() };
|
||||
}
|
||||
|
||||
function app(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function gen(id: string, applicationId: string): GitOpsGenerationRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
commit_sha: 'abc123',
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
configured_ref: 'main',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
manifest_version: 0,
|
||||
candidate_dir: `generations/candidate-${id}`,
|
||||
applied_dir: `generations/applied-${id}-0`,
|
||||
expected_invocation_json: '{"composeFileOrder":[],"projectName":null,"projectDirectory":null,"envFileOrder":[]}',
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
validation_ok: 1,
|
||||
plan_blocked: 0,
|
||||
change_plan_fingerprint: null,
|
||||
operation_id: `op-${id}`,
|
||||
trigger: 'manual',
|
||||
actor: 'tester',
|
||||
previous_generation_id: null,
|
||||
redacted_limitations_json: '[]',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,652 @@
|
||||
/**
|
||||
* End-to-end coverage for the Direct Git producers.
|
||||
*
|
||||
* Only the two boundaries the host owns are stubbed: `git.clone` writes a real
|
||||
* project into the clone directory and `git.log` returns a commit, and the
|
||||
* compose commands that need a running daemon report an exit code each test
|
||||
* chooses. Compose commands that only parse files, `config` above all, still
|
||||
* shell out for real, so the Compose CLI is a genuine prerequisite here even
|
||||
* though the daemon is not. Everything after that runs for real, so fetch,
|
||||
* candidate materialization, change-plan classification, the apply, the
|
||||
* deploy, and the detach all drive the GitOps state model the way they do in
|
||||
* production.
|
||||
*
|
||||
* This exists because the producer wiring is the seam between the operational
|
||||
* Git path and the revision state, and a mismatch there type-checks and passes
|
||||
* transition-level tests.
|
||||
*/
|
||||
import { EventEmitter } from 'events';
|
||||
import fsPromises from 'fs/promises';
|
||||
import path from 'path';
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
|
||||
const { mockGitClone, mockGitLog, compose } = vi.hoisted(() => ({
|
||||
mockGitClone: vi.fn(),
|
||||
mockGitLog: vi.fn(),
|
||||
/** Exit code the next daemon-dependent compose command reports. */
|
||||
compose: { exitCode: 1 },
|
||||
}));
|
||||
|
||||
vi.mock('isomorphic-git', () => {
|
||||
const api = { clone: mockGitClone, log: mockGitLog };
|
||||
return { default: api, clone: mockGitClone, log: mockGitLog };
|
||||
});
|
||||
vi.mock('isomorphic-git/http/node', () => ({ default: {} }));
|
||||
|
||||
/**
|
||||
* Compose verbs that need a running daemon and are issued through `spawn`.
|
||||
*
|
||||
* `ps` is deliberately absent: it is issued through `execFile`, which this mock
|
||||
* does not replace, so listing it would advertise coverage that is not there.
|
||||
*/
|
||||
const DAEMON_COMPOSE_VERBS = new Set(['up', 'down', 'pull', 'build', 'start', 'stop', 'restart']);
|
||||
const COMPOSE_FLAGS_WITH_VALUE = new Set([
|
||||
'-f', '--file', '-p', '--project-name', '--env-file', '--project-directory',
|
||||
]);
|
||||
|
||||
/**
|
||||
* The verb in a `docker compose …` argv, skipping global flags and their
|
||||
* values so a stack or file named after a verb cannot be mistaken for one.
|
||||
*/
|
||||
function composeVerbOf(args: readonly string[]): string | null {
|
||||
if (args[0] !== 'compose') return null;
|
||||
for (let i = 1; i < args.length; i++) {
|
||||
const token = args[i];
|
||||
if (COMPOSE_FLAGS_WITH_VALUE.has(token)) {
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
if (token.startsWith('-')) continue;
|
||||
return token;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function fakeComposeChild(): EventEmitter {
|
||||
const child = new EventEmitter() as EventEmitter & {
|
||||
stdout: EventEmitter;
|
||||
stderr: EventEmitter;
|
||||
kill: () => boolean;
|
||||
};
|
||||
child.stdout = new EventEmitter();
|
||||
child.stderr = new EventEmitter();
|
||||
child.kill = () => true;
|
||||
// The caller attaches its listeners synchronously after spawn returns, so the
|
||||
// exit cannot be announced until the current turn finishes.
|
||||
setImmediate(() => child.emit('close', compose.exitCode));
|
||||
return child;
|
||||
}
|
||||
|
||||
/**
|
||||
* Only the compose commands that need a daemon are answered here; `config` and
|
||||
* everything else still runs for real.
|
||||
*
|
||||
* That split is the whole point. A deploy failing is otherwise a fact about the
|
||||
* host rather than about the adapter: a workstation with the CLI but no daemon
|
||||
* parses compose files happily and fails `up`, while a CI runner succeeds at
|
||||
* both, so any test that reads "the deploy failed" from the environment says
|
||||
* something different in the two places.
|
||||
*/
|
||||
vi.mock('child_process', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('child_process')>();
|
||||
return {
|
||||
...actual,
|
||||
spawn: (command: string, args: readonly string[], options?: unknown) => {
|
||||
if (command === 'docker' && DAEMON_COMPOSE_VERBS.has(composeVerbOf(args) ?? '')) {
|
||||
return fakeComposeChild();
|
||||
}
|
||||
return (actual.spawn as unknown as (...a: unknown[]) => unknown)(command, args, options);
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
// Rollback capture talks to Docker, which is not available here. Stubbing it
|
||||
// keeps the apply on its success path so the GitOps wiring is what the test
|
||||
// actually exercises.
|
||||
vi.mock('../services/StackUpdateRecoveryService', () => ({
|
||||
StackUpdateRecoveryService: {
|
||||
getInstance: () => ({
|
||||
captureCandidate: vi.fn(async () => ({ id: 'rec-producers-1' })),
|
||||
abandon: vi.fn(async () => true),
|
||||
markAcquired: vi.fn(() => true),
|
||||
handoff: vi.fn(() => true),
|
||||
markReconciling: vi.fn(() => true),
|
||||
markImmediateVerified: vi.fn(() => true),
|
||||
get: vi.fn(() => ({ id: 'rec-producers-1', is_current: 1 })),
|
||||
linkGateOrRetain: vi.fn(),
|
||||
compensateWithCandidate: vi.fn(async () => true),
|
||||
start: vi.fn(),
|
||||
}),
|
||||
},
|
||||
}));
|
||||
|
||||
const REPO = 'https://github.com/example/project.git';
|
||||
const COMPOSE = 'services:\n web:\n image: nginx:1.27\n';
|
||||
const COMPOSE_V2 = 'services:\n web:\n image: nginx:1.28\n';
|
||||
const COMPOSE_PROD = 'services:\n web:\n restart: always\n';
|
||||
|
||||
let tmpDir: string;
|
||||
let GitSourceService: typeof import('../services/GitSourceService').GitSourceService;
|
||||
let GitOpsStore: typeof import('../services/gitops/store').GitOpsStore;
|
||||
let GitOpsTransitions: typeof import('../services/gitops/transitions').GitOpsTransitions;
|
||||
let projectApplication: typeof import('../services/gitops/derive').projectApplication;
|
||||
|
||||
/** Make the next clone produce a project containing this compose content. */
|
||||
function stageRepo(content: string, sha: string, extraFiles: Record<string, string> = {}): void {
|
||||
mockGitClone.mockImplementation(async ({ dir }: { dir: string }) => {
|
||||
await fsPromises.mkdir(dir, { recursive: true });
|
||||
await fsPromises.writeFile(path.join(dir, 'compose.yaml'), content, 'utf8');
|
||||
for (const [name, body] of Object.entries(extraFiles)) {
|
||||
await fsPromises.writeFile(path.join(dir, name), body, 'utf8');
|
||||
}
|
||||
});
|
||||
mockGitLog.mockResolvedValue([{ oid: sha }]);
|
||||
}
|
||||
|
||||
function projectOf(applicationId: string) {
|
||||
const projection = projectApplication(applicationId, true);
|
||||
if (projection.targetMode === 'not_applicable') throw new Error('expected an application');
|
||||
return projection;
|
||||
}
|
||||
|
||||
describe('Direct Git producers drive the revision state', () => {
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
({ GitSourceService } = await import('../services/GitSourceService'));
|
||||
({ GitOpsStore } = await import('../services/gitops/store'));
|
||||
({ GitOpsTransitions } = await import('../services/gitops/transitions'));
|
||||
({ projectApplication } = await import('../services/gitops/derive'));
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
mockGitClone.mockReset();
|
||||
mockGitLog.mockReset();
|
||||
compose.exitCode = 1;
|
||||
});
|
||||
|
||||
// Prototype spies a test installs are restored here rather than in a per-test
|
||||
// finally, so a failing test cannot leak one into the next. Only spies are
|
||||
// touched, so the hoisted git mocks keep the reset above.
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it('creates, fetches, applies, and detaches a Git stack through the state model', async () => {
|
||||
const svc = GitSourceService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const stackName = 'producers-web';
|
||||
|
||||
// ── create ────────────────────────────────────────────────────────────
|
||||
stageRepo(COMPOSE, 'aaaaaaa1');
|
||||
await svc.createStackFromGit({
|
||||
stackName,
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
|
||||
const app = store.getLiveDirectApplication(stackName);
|
||||
expect(app).toBeTruthy();
|
||||
if (!app) throw new Error('expected an application');
|
||||
expect(app.lifecycle_status).toBe('active');
|
||||
expect(app.accepted_generation_id).not.toBeNull();
|
||||
expect(app.desired_commit_sha).toBe('aaaaaaa1');
|
||||
// The create records a secret-free identity, never the operational URL.
|
||||
expect(app.configured_repo_url).toBe('https://github.com/example/project.git');
|
||||
// The success boundary cleared its own checkpoint.
|
||||
expect(store.getCreateCheckpoint(app.id)).toBeUndefined();
|
||||
|
||||
const afterCreate = projectOf(app.id);
|
||||
expect(afterCreate.facets.source.status).toBe('application_generation_accepted');
|
||||
expect(afterCreate.targets[0]?.runtime.status).toBe('applied_not_deployed');
|
||||
|
||||
// ── fetch a newer commit ──────────────────────────────────────────────
|
||||
stageRepo(COMPOSE_V2, 'bbbbbbb2');
|
||||
await svc.pull(stackName, { actor: 'tester' });
|
||||
|
||||
const afterPull = store.getApplication(app.id)!;
|
||||
expect(afterPull.desired_commit_sha).toBe('bbbbbbb2');
|
||||
expect(afterPull.fetched_commit_sha).toBe('bbbbbbb2');
|
||||
// A fetch advances the resolved commit and offers a candidate, but the
|
||||
// accepted generation does not move until the apply.
|
||||
expect(afterPull.accepted_generation_id).toBe(app.accepted_generation_id);
|
||||
expect(afterPull.candidate_generation_id).not.toBeNull();
|
||||
expect(afterPull.candidate_generation_id).not.toBe(app.accepted_generation_id);
|
||||
|
||||
const candidateId = afterPull.candidate_generation_id!;
|
||||
const candidate = store.getGeneration(candidateId)!;
|
||||
expect(candidate.commit_sha).toBe('bbbbbbb2');
|
||||
expect(candidate.application_id).toBe(app.id);
|
||||
expect(candidate.materialization_fingerprint).toBe(afterPull.materialization_fingerprint);
|
||||
expect(store.getTarget(app.id, 1)?.candidate_generation_id).toBe(candidateId);
|
||||
expect(projectOf(app.id).availableActions).toContain('apply');
|
||||
|
||||
// ── apply ─────────────────────────────────────────────────────────────
|
||||
await svc.apply(stackName, 'bbbbbbb2', { requirePlanFingerprint: false, deploy: false, actor: 'tester' });
|
||||
|
||||
const afterApply = store.getApplication(app.id)!;
|
||||
expect(afterApply.accepted_generation_id).toBe(candidateId);
|
||||
expect(afterApply.candidate_generation_id).toBeNull();
|
||||
expect(afterApply.active_operation_stage).toBeNull();
|
||||
expect(afterApply.source_acceptance_ref).not.toBeNull();
|
||||
|
||||
const target = store.getTarget(app.id, 1)!;
|
||||
expect(target.desired_generation_id).toBe(candidateId);
|
||||
expect(target.applied_generation_id).toBe(candidateId);
|
||||
expect(target.candidate_generation_id).toBeNull();
|
||||
|
||||
// The acceptance is provable against the exact generation it authorized.
|
||||
expect(store.resolveApprovalRef(afterApply.source_acceptance_ref!, {
|
||||
kind: 'source_acceptance',
|
||||
applicationId: app.id,
|
||||
generationId: candidateId,
|
||||
})).toBeTruthy();
|
||||
expect(store.resolveApprovalRef(afterApply.source_acceptance_ref!, {
|
||||
kind: 'source_acceptance',
|
||||
applicationId: app.id,
|
||||
generationId: app.accepted_generation_id!,
|
||||
})).toBeNull();
|
||||
|
||||
// ── detach ────────────────────────────────────────────────────────────
|
||||
await svc.detach(stackName);
|
||||
|
||||
expect(store.getLiveDirectApplication(stackName)).toBeUndefined();
|
||||
const tombstoned = store.getApplication(app.id)!;
|
||||
expect(tombstoned.lifecycle_status).toBe('detached');
|
||||
// Configured identity and resolved commit survive as frozen facts.
|
||||
expect(tombstoned.configured_repo_url).toBe('https://github.com/example/project.git');
|
||||
expect(tombstoned.desired_commit_sha).toBe('bbbbbbb2');
|
||||
expect(store.getTarget(app.id, 1)?.target_status).toBe('tombstoned');
|
||||
expect(projectOf(app.id).facets.source.status).toBe('not_live');
|
||||
});
|
||||
|
||||
it('binds the deployed generation through the Compose adapter', async () => {
|
||||
const { ComposeService } = await import('../services/ComposeService');
|
||||
const { default: DockerController } = await import('../services/DockerController');
|
||||
const svc = GitSourceService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const stackName = 'producers-deploy';
|
||||
|
||||
stageRepo(COMPOSE, '11111111');
|
||||
await svc.createStackFromGit({
|
||||
stackName,
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
const app = store.getLiveDirectApplication(stackName)!;
|
||||
const applied = store.getTarget(app.id, 1)!.applied_generation_id;
|
||||
expect(applied).not.toBeNull();
|
||||
expect(store.getTarget(app.id, 1)?.deployed_generation_id).toBeNull();
|
||||
|
||||
// The post-deploy probe asks the daemon what came up. It is not what this
|
||||
// test is about, and `getInstance` hands back a fresh controller each call,
|
||||
// so the stubs go on the prototype. `afterEach` restores them, so a failing
|
||||
// test cannot leak one into the next.
|
||||
const composeSvc = ComposeService.getInstance(1);
|
||||
vi.spyOn(DockerController.prototype, 'getLegacyOrphanContainersByStack')
|
||||
.mockResolvedValue([]);
|
||||
vi.spyOn(DockerController.prototype, 'getDocker')
|
||||
.mockReturnValue({ listContainers: async () => [] } as unknown as ReturnType<
|
||||
typeof DockerController.prototype.getDocker
|
||||
>);
|
||||
|
||||
// ── the compose command fails ──────────────────────────────────────
|
||||
compose.exitCode = 1;
|
||||
await expect(composeSvc.deployStack(stackName)).rejects.toThrow();
|
||||
|
||||
const failed = store.getTarget(app.id, 1)!;
|
||||
expect(failed.deployed_generation_id).toBeNull();
|
||||
expect(failed.failure_stage).toBe('deploy');
|
||||
// Classified conservatively: once the compose command has been handed
|
||||
// off, we cannot prove the workload was untouched, and claiming it was
|
||||
// intact would be the more dangerous error.
|
||||
expect(failed.failure_class).toBe('post_mutation');
|
||||
expect(projectOf(app.id).targets[0]?.runtime.status).toBe('failed_after_mutation');
|
||||
expect(failed.applied_generation_id).toBe(applied);
|
||||
|
||||
// ── the compose command succeeds ───────────────────────────────────
|
||||
compose.exitCode = 0;
|
||||
const result = await composeSvc.deployStack(stackName);
|
||||
|
||||
// The adapter reports the generation it bound, which is what lets the
|
||||
// caller start health against that exact generation rather than against
|
||||
// whatever is applied by the time health runs.
|
||||
expect(result.deployedGenerationId).toBe(applied);
|
||||
|
||||
const bound = store.getTarget(app.id, 1)!;
|
||||
expect(bound.deployed_generation_id).toBe(applied);
|
||||
expect(bound.applied_generation_id).toBe(applied);
|
||||
// A bound deploy clears the earlier failure: the target is no longer in
|
||||
// the state the operator was asked to act on.
|
||||
expect(bound.failure_stage).toBeNull();
|
||||
expect(bound.failure_class).toBeNull();
|
||||
expect(projectOf(app.id).targets[0]?.runtime.status).not.toBe('failed_after_mutation');
|
||||
});
|
||||
|
||||
it('reports the deployed generation from an update so health can bind to it', async () => {
|
||||
const { ComposeService } = await import('../services/ComposeService');
|
||||
const { StackUpdateOrchestrator } = await import('../services/StackUpdateOrchestrator');
|
||||
const svc = GitSourceService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const stackName = 'producers-update';
|
||||
|
||||
stageRepo(COMPOSE, '22222222');
|
||||
await svc.createStackFromGit({
|
||||
stackName,
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
const app = store.getLiveDirectApplication(stackName)!;
|
||||
const applied = store.getTarget(app.id, 1)!.applied_generation_id;
|
||||
|
||||
// The image pull fails, so the update dies during preparation, before the
|
||||
// recreate Compose is handed anything. The deploy operation is opened only
|
||||
// at that recreate, so nothing is recorded: an update that never touched
|
||||
// the workload must not leave a deploy failure behind for the deriver to
|
||||
// report.
|
||||
compose.exitCode = 1;
|
||||
await expect(ComposeService.getInstance(1).updateStack(stackName)).rejects.toThrow();
|
||||
const target = store.getTarget(app.id, 1)!;
|
||||
expect(target.deployed_generation_id).toBeNull();
|
||||
expect(target.failure_stage).toBeNull();
|
||||
expect(target.active_operation_stage).toBeNull();
|
||||
expect(target.applied_generation_id).toBe(applied);
|
||||
expect(projectOf(app.id).targets[0]?.runtime.status).toBe('applied_not_deployed');
|
||||
|
||||
// The same holds through the orchestrator, which is what the update callers
|
||||
// actually use and which carries the binding on to beginStack.
|
||||
await expect(StackUpdateOrchestrator.getInstance().execute(
|
||||
{ nodeId: 1, stackName, target: { scope: 'stack' }, trigger: 'manual', actor: 'tester' },
|
||||
{ atomic: false, terminalWs: null },
|
||||
)).rejects.toThrow();
|
||||
expect(store.getTarget(app.id, 1)?.failure_stage).toBeNull();
|
||||
});
|
||||
|
||||
it('records a failed fetch without moving any pointer', async () => {
|
||||
const svc = GitSourceService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const stackName = 'producers-fail';
|
||||
|
||||
stageRepo(COMPOSE, 'ccccccc3');
|
||||
await svc.createStackFromGit({
|
||||
stackName,
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
const app = store.getLiveDirectApplication(stackName)!;
|
||||
const acceptedBefore = app.accepted_generation_id;
|
||||
|
||||
mockGitClone.mockRejectedValue(new Error('could not resolve host'));
|
||||
await expect(svc.pull(stackName, { actor: 'tester' })).rejects.toThrow();
|
||||
|
||||
const afterFailure = store.getApplication(app.id)!;
|
||||
expect(afterFailure.failure_stage).toBe('fetch');
|
||||
expect(afterFailure.active_operation_stage).toBeNull();
|
||||
expect(afterFailure.accepted_generation_id).toBe(acceptedBefore);
|
||||
expect(afterFailure.candidate_generation_id).toBeNull();
|
||||
|
||||
const projection = projectOf(app.id);
|
||||
expect(projection.facets.source.status).toBe('source_failed');
|
||||
expect(projection.availableActions).toContain('fetch');
|
||||
|
||||
// A later successful fetch clears the failure.
|
||||
stageRepo(COMPOSE_V2, 'ddddddd4');
|
||||
await svc.pull(stackName, { actor: 'tester' });
|
||||
expect(store.getApplication(app.id)?.failure_stage).toBeNull();
|
||||
});
|
||||
|
||||
it('brings a newly linked stack into the model and invalidates its candidate on a material edit', async () => {
|
||||
const svc = GitSourceService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const stackName = 'producers-link';
|
||||
|
||||
const composeDir = process.env.COMPOSE_DIR!;
|
||||
await fsPromises.mkdir(path.join(composeDir, stackName), { recursive: true });
|
||||
await fsPromises.writeFile(path.join(composeDir, stackName, 'compose.yaml'), COMPOSE, 'utf8');
|
||||
|
||||
stageRepo(COMPOSE, '33333333');
|
||||
await svc.upsert({
|
||||
stackName,
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
|
||||
const app = store.getLiveDirectApplication(stackName);
|
||||
expect(app).toBeTruthy();
|
||||
if (!app) throw new Error('expected an application');
|
||||
// Linked, not fetched: nothing is desired or accepted until a pull runs.
|
||||
expect(app.lifecycle_status).toBe('active');
|
||||
expect(app.desired_commit_sha).toBeNull();
|
||||
expect(app.accepted_generation_id).toBeNull();
|
||||
let projection = projectOf(app.id);
|
||||
expect(projection.facets.source.status).toBe('never_reconciled');
|
||||
expect(projection.availableActions).toContain('fetch');
|
||||
|
||||
// A pull produces a candidate against the current configuration.
|
||||
stageRepo(COMPOSE_V2, '44444444');
|
||||
await svc.pull(stackName, { actor: 'tester' });
|
||||
const candidateId = store.getApplication(app.id)!.candidate_generation_id;
|
||||
expect(candidateId).not.toBeNull();
|
||||
|
||||
// A credential-only edit changes nothing material, so the candidate stands.
|
||||
stageRepo(COMPOSE_V2, '44444444');
|
||||
await svc.upsert({
|
||||
stackName,
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook: true,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
expect(store.getApplication(app.id)?.candidate_generation_id).toBe(candidateId);
|
||||
|
||||
// Changing the compose file set does invalidate it: that candidate was
|
||||
// built from a different set and can no longer be applied.
|
||||
stageRepo(COMPOSE_V2, '44444444', { 'compose.prod.yaml': COMPOSE_PROD });
|
||||
await svc.upsert({
|
||||
stackName,
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml', 'compose.prod.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook: true,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
const afterEdit = store.getApplication(app.id)!;
|
||||
expect(afterEdit.candidate_generation_id).toBeNull();
|
||||
expect(afterEdit.desired_commit_sha).toBeNull();
|
||||
expect(store.getTarget(app.id, 1)?.candidate_generation_id).toBeNull();
|
||||
projection = projectOf(app.id);
|
||||
expect(projection.availableActions).toContain('fetch');
|
||||
expect(projection.availableActions).not.toContain('apply');
|
||||
});
|
||||
|
||||
it('retires the application when the stack itself is deleted', async () => {
|
||||
const { DeployedStackDeletionService } = await import('../services/DeployedStackDeletionService');
|
||||
const svc = GitSourceService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const stackName = 'producers-delete';
|
||||
|
||||
stageRepo(COMPOSE, '55555555');
|
||||
await svc.createStackFromGit({
|
||||
stackName,
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
const app = store.getLiveDirectApplication(stackName)!;
|
||||
|
||||
await DeployedStackDeletionService.getInstance().deleteDeployedStack({
|
||||
nodeId: 1,
|
||||
stackName,
|
||||
pruneVolumes: false,
|
||||
actor: 'tester',
|
||||
});
|
||||
|
||||
// A deleted stack must not leave a live application behind: it would keep
|
||||
// claiming the name and block re-creating it.
|
||||
expect(store.getLiveDirectApplication(stackName)).toBeUndefined();
|
||||
expect(store.getApplication(app.id)?.lifecycle_status).toBe('deleted');
|
||||
expect(store.getTarget(app.id, 1)?.target_status).toBe('tombstoned');
|
||||
});
|
||||
|
||||
it('closes the operation when a terminal transition is rejected', async () => {
|
||||
const svc = GitSourceService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const stackName = 'producers-reject';
|
||||
|
||||
stageRepo(COMPOSE, '66666666');
|
||||
await svc.createStackFromGit({
|
||||
stackName,
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
authType: 'none',
|
||||
token: null,
|
||||
autoApplyOnWebhook: false,
|
||||
autoDeployOnApply: false,
|
||||
});
|
||||
const app = store.getLiveDirectApplication(stackName)!;
|
||||
|
||||
// Reject the transition that closes a fetch. Recording must not fail the
|
||||
// pull, but it must not leave the operation open either: a fetch that never
|
||||
// terminates blocks every later pull from being recorded at all.
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
const realFetched = tx.fetched.bind(tx);
|
||||
tx.fetched = () => { throw new Error('rejected for test'); };
|
||||
try {
|
||||
stageRepo(COMPOSE_V2, '77777777');
|
||||
await svc.pull(stackName, { actor: 'tester' });
|
||||
} finally {
|
||||
tx.fetched = realFetched;
|
||||
}
|
||||
|
||||
const afterReject = store.getApplication(app.id)!;
|
||||
expect(afterReject.active_operation_stage).toBeNull();
|
||||
expect(afterReject.failure_stage).toBe('fetch');
|
||||
// The projection reports an error the operator can act on, not a spinner.
|
||||
const projection = projectOf(app.id);
|
||||
expect(projection.facets.source.status).toBe('source_failed');
|
||||
expect(projection.availableActions).toContain('fetch');
|
||||
|
||||
// And the next pull records normally, rather than being locked out.
|
||||
stageRepo(COMPOSE_V2, '88888888');
|
||||
await svc.pull(stackName, { actor: 'tester' });
|
||||
const recovered = store.getApplication(app.id)!;
|
||||
expect(recovered.fetched_commit_sha).toBe('88888888');
|
||||
expect(recovered.failure_stage).toBeNull();
|
||||
expect(recovered.active_operation_stage).toBeNull();
|
||||
});
|
||||
|
||||
it('leaves a stack with no GitOps application untouched', async () => {
|
||||
const svc = GitSourceService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const stackName = 'producers-legacy';
|
||||
|
||||
// A Git stack exactly as an install carries it across an upgrade: the
|
||||
// source row was written before this model existed, so there is no
|
||||
// application and nothing has migrated it yet. Seeded directly, because
|
||||
// linking through the service now creates one.
|
||||
const composeDir = process.env.COMPOSE_DIR!;
|
||||
await fsPromises.mkdir(path.join(composeDir, stackName), { recursive: true });
|
||||
await fsPromises.writeFile(path.join(composeDir, stackName, 'compose.yaml'), COMPOSE, 'utf8');
|
||||
(await import('../services/DatabaseService')).DatabaseService.getInstance().upsertGitSource({
|
||||
stack_name: stackName,
|
||||
repo_url: REPO,
|
||||
branch: 'main',
|
||||
compose_path: 'compose.yaml',
|
||||
compose_paths: ['compose.yaml'],
|
||||
context_dir: null,
|
||||
sync_env: false,
|
||||
env_path: null,
|
||||
auth_type: 'none',
|
||||
encrypted_token: null,
|
||||
auto_apply_on_webhook: false,
|
||||
auto_deploy_on_apply: false,
|
||||
last_applied_commit_sha: 'eeeeeee5',
|
||||
last_applied_content_hash: null,
|
||||
pending_commit_sha: null,
|
||||
pending_compose_content: null,
|
||||
pending_env_content: null,
|
||||
pending_fetched_at: null,
|
||||
last_debounce_at: null,
|
||||
});
|
||||
expect(store.getLiveDirectApplication(stackName)).toBeUndefined();
|
||||
|
||||
stageRepo(COMPOSE_V2, 'fffffff6');
|
||||
await svc.pull(stackName, { actor: 'tester' });
|
||||
|
||||
// The pull succeeded operationally and wrote no GitOps rows.
|
||||
expect(store.getLiveDirectApplication(stackName)).toBeUndefined();
|
||||
const historyRows = (await import('../services/DatabaseService')).DatabaseService
|
||||
.getInstance().getDb()
|
||||
.prepare('SELECT COUNT(*) AS n FROM gitops_history WHERE stack_name = ?')
|
||||
.get(stackName) as { n: number };
|
||||
expect(historyRows.n).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,550 @@
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import {
|
||||
HISTORY_DEFAULT_LIMIT,
|
||||
HISTORY_MAX_LIMIT,
|
||||
HISTORY_SCAN_CAP,
|
||||
decodeHistoryCursor,
|
||||
encodeHistoryCursor,
|
||||
insertHistory,
|
||||
queryHistoryRows,
|
||||
toHistoryItem,
|
||||
} from '../services/gitops/history';
|
||||
import { parseHistoryFilters, parseLimit } from '../helpers/gitopsHistoryPage';
|
||||
import {
|
||||
classifyHistoryRow,
|
||||
classifySourceRow,
|
||||
normalizeStackResourcePresent,
|
||||
} from '../services/gitops/readAuth';
|
||||
import type { GitOpsApplicationRow, GitOpsHistoryRow } from '../services/gitops/types';
|
||||
|
||||
describe('gitops history read layer', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
seedHistory();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
describe('cursor', () => {
|
||||
const SAMPLE_ID = '0b3f4a1c-8d2e-4c7b-9f10-5a6b7c8d9e0f';
|
||||
|
||||
it('round-trips a created_at and id pair', () => {
|
||||
const encoded = encodeHistoryCursor({ createdAt: 1700, id: SAMPLE_ID });
|
||||
expect(decodeHistoryCursor(encoded)).toEqual({ createdAt: 1700, id: SAMPLE_ID });
|
||||
});
|
||||
|
||||
it('rejects malformed cursors rather than guessing a position', () => {
|
||||
expect(decodeHistoryCursor('')).toBeNull();
|
||||
expect(decodeHistoryCursor('nodot')).toBeNull();
|
||||
expect(decodeHistoryCursor(`.${SAMPLE_ID}`)).toBeNull();
|
||||
expect(decodeHistoryCursor('123.')).toBeNull();
|
||||
expect(decodeHistoryCursor(`notanumber.${SAMPLE_ID}`)).toBeNull();
|
||||
expect(decodeHistoryCursor(`-5.${SAMPLE_ID}`)).toBeNull();
|
||||
});
|
||||
|
||||
it('rejects an id that is not a real row id', () => {
|
||||
// A truncated cursor is the likely case, and an unvalidated id would not
|
||||
// fail: it would shift the page boundary and quietly drop or repeat rows.
|
||||
expect(decodeHistoryCursor(`1700.${SAMPLE_ID.slice(0, 12)}`)).toBeNull();
|
||||
expect(decodeHistoryCursor('1700.not-a-uuid')).toBeNull();
|
||||
expect(decodeHistoryCursor(`1700.${SAMPLE_ID.toUpperCase()}`)).toBeNull();
|
||||
});
|
||||
|
||||
it('accepts the cursor it emits for a stored row', () => {
|
||||
const row = query({ commitSha: 'sha-b' })[0] as GitOpsHistoryRow;
|
||||
const encoded = encodeHistoryCursor({ createdAt: row.created_at, id: row.id });
|
||||
expect(decodeHistoryCursor(encoded)).toEqual({ createdAt: row.created_at, id: row.id });
|
||||
});
|
||||
});
|
||||
|
||||
describe('queryHistoryRows', () => {
|
||||
it('returns newest first', () => {
|
||||
const rows = query({ stackName: 'history-web' });
|
||||
expect(rows.map(r => r.commit_sha)).toEqual(['sha-c', 'sha-b', 'sha-a', null]);
|
||||
});
|
||||
|
||||
// Every filter is asserted to reach the right column. A wrong column name
|
||||
// is not a subtly wrong page, it is a "no such column" throw at query
|
||||
// time, so an untested filter is a 500 waiting behind any link carrying it.
|
||||
it.each([
|
||||
['applicationId', { applicationId: 'app-history' }, 'sha-b'],
|
||||
['stackName', { stackName: 'history-web' }, 'sha-b'],
|
||||
['repoIdentity', { repoIdentity: 'https://github.com/org/repo.git' }, 'sha-b'],
|
||||
['configuredRef', { configuredRef: 'main' }, 'sha-b'],
|
||||
['commitSha', { commitSha: 'sha-b' }, 'sha-b'],
|
||||
['nodeId', { nodeId: 7 }, 'sha-b'],
|
||||
['trigger', { trigger: 'webhook' }, 'sha-b'],
|
||||
['actor', { actor: 'operator-2' }, 'sha-b'],
|
||||
['outcome', { outcome: 'failed' as const }, 'sha-c'],
|
||||
['rolloutCandidateId', { rolloutCandidateId: 'cand-1' }, 'sha-c'],
|
||||
])('routes the %s filter to its own column', (_name, filters, expectedSha) => {
|
||||
const rows = query(filters);
|
||||
expect(rows.length).toBeGreaterThan(0);
|
||||
expect(rows.map(r => r.commit_sha)).toContain(expectedSha);
|
||||
});
|
||||
|
||||
it('accepts the identity filters that match nothing in this fixture', () => {
|
||||
// Exercises the remaining column names so a typo still throws here.
|
||||
expect(query({ generationId: 'gen-absent' })).toHaveLength(0);
|
||||
expect(query({ artifactSetId: 'artifact-absent' })).toHaveLength(0);
|
||||
expect(query({ blueprintId: 4242 })).toHaveLength(0);
|
||||
expect(query({ rolloutGenerationId: 'rollout-gen-absent' })).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('narrows node rows while keeping application-level rows', () => {
|
||||
// A proxied hub view filters by its own node id, but activation and
|
||||
// similar stages carry no node. Dropping them under `node_id = ?`
|
||||
// would make the history read as if the application never came into
|
||||
// being.
|
||||
const byNode = query({ nodeId: 7 });
|
||||
expect(byNode.map(r => r.node_id)).toEqual([7, null]);
|
||||
expect(byNode[0]?.commit_sha).toBe('sha-b');
|
||||
expect(query({ stackName: 'no-such-stack' })).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('never matches a rollout candidate id as a rollout generation id', () => {
|
||||
// The candidate is a proposal; a generation is a dispatch that ran.
|
||||
// Answering the generation filter from the candidate column would report
|
||||
// a rollout that never happened.
|
||||
expect(query({ rolloutCandidateId: 'cand-1' })).toHaveLength(1);
|
||||
expect(query({ rolloutGenerationId: 'cand-1' })).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('pages past the cursor without repeating a row', () => {
|
||||
const first = query({ stackName: 'history-web' }, null, 2);
|
||||
expect(first).toHaveLength(2);
|
||||
const last = first[1] as GitOpsHistoryRow;
|
||||
const second = query({ stackName: 'history-web' }, { createdAt: last.created_at, id: last.id }, 2);
|
||||
expect(second).toHaveLength(2);
|
||||
expect(second[0]?.id).not.toBe(last.id);
|
||||
expect(second[0]?.commit_sha).toBe('sha-a');
|
||||
});
|
||||
|
||||
it('separates rows sharing one millisecond by id', () => {
|
||||
const sameMs = query({ commitSha: 'sha-tie' });
|
||||
expect(sameMs).toHaveLength(2);
|
||||
const [newer, older] = sameMs as [GitOpsHistoryRow, GitOpsHistoryRow];
|
||||
expect(newer.id > older.id).toBe(true);
|
||||
const after = query({ commitSha: 'sha-tie' }, { createdAt: newer.created_at, id: newer.id }, 10);
|
||||
expect(after.map(r => r.id)).toEqual([older.id]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('toHistoryItem', () => {
|
||||
it('exposes the producer delta as before and after', () => {
|
||||
const row = query({ commitSha: 'sha-b' })[0] as GitOpsHistoryRow;
|
||||
const item = toHistoryItem(row);
|
||||
expect(item.before).toEqual({ desiredCommitSha: null });
|
||||
expect(item.after).toEqual({ desiredCommitSha: 'sha-b' });
|
||||
expect(item.limitations).toEqual([]);
|
||||
expect(item.stage).toBe('fetched');
|
||||
});
|
||||
|
||||
it('keeps identity and stage when the recorded delta cannot be read', () => {
|
||||
const row = query({ commitSha: 'sha-a' })[0] as GitOpsHistoryRow;
|
||||
const corrupt: GitOpsHistoryRow = { ...row, after_json: '{not json' };
|
||||
const item = toHistoryItem(corrupt);
|
||||
expect(item.after).toBeNull();
|
||||
expect(item.stage).toBe(row.stage);
|
||||
expect(item.applicationId).toBe(row.application_id);
|
||||
expect(item.limitations).toEqual([
|
||||
{
|
||||
code: 'history_json_invalid',
|
||||
message: 'Recorded change detail for this entry could not be read.',
|
||||
evidence: { before: false, after: true },
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('treats a non-object payload as unreadable', () => {
|
||||
const row = query({ commitSha: 'sha-a' })[0] as GitOpsHistoryRow;
|
||||
const item = toHistoryItem({ ...row, before_json: '"a string"' });
|
||||
expect(item.before).toBeNull();
|
||||
expect(item.limitations[0]?.code).toBe('history_json_invalid');
|
||||
});
|
||||
|
||||
it('maps every identity column to its own field', () => {
|
||||
// The four approval refs are same-typed and same-shaped, so a swap
|
||||
// between them is invisible without asserting each one individually.
|
||||
const row = query({ commitSha: 'sha-b' })[0] as GitOpsHistoryRow;
|
||||
const populated: GitOpsHistoryRow = {
|
||||
...row,
|
||||
generation_id: 'gen-x',
|
||||
artifact_set_id: 'art-x',
|
||||
intent_revision_id: 'intent-x',
|
||||
rollout_candidate_id: 'cand-x',
|
||||
rollout_generation_id: 'rgen-x',
|
||||
source_acceptance_ref: 'ref-source',
|
||||
placement_approval_ref: 'ref-placement',
|
||||
rollout_authorization_ref: 'ref-rollout',
|
||||
legacy_combined_approval_ref: 'ref-legacy',
|
||||
blueprint_id: 11,
|
||||
};
|
||||
const item = toHistoryItem(populated);
|
||||
expect(item).toMatchObject({
|
||||
id: row.id,
|
||||
createdAt: row.created_at,
|
||||
applicationId: row.application_id,
|
||||
targetMode: row.target_mode,
|
||||
stackName: row.stack_name,
|
||||
repoIdentity: 'https://github.com/org/repo.git',
|
||||
configuredRef: 'main',
|
||||
blueprintId: 11,
|
||||
nodeId: row.node_id,
|
||||
commitSha: row.commit_sha,
|
||||
generationId: 'gen-x',
|
||||
artifactSetId: 'art-x',
|
||||
intentRevisionId: 'intent-x',
|
||||
rolloutCandidateId: 'cand-x',
|
||||
rolloutGenerationId: 'rgen-x',
|
||||
operationId: row.operation_id,
|
||||
stage: row.stage,
|
||||
outcome: row.outcome,
|
||||
trigger: row.trigger,
|
||||
actor: row.actor,
|
||||
approvals: {
|
||||
sourceAcceptanceRef: 'ref-source',
|
||||
placementApprovalRef: 'ref-placement',
|
||||
rolloutAuthorizationRef: 'ref-rollout',
|
||||
legacyCombinedApprovalRef: 'ref-legacy',
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('stackResourcePresent validation', () => {
|
||||
it('accepts only a real boolean true', () => {
|
||||
expect(normalizeStackResourcePresent(true)).toBe(true);
|
||||
expect(normalizeStackResourcePresent(false)).toBe(false);
|
||||
expect(normalizeStackResourcePresent('true')).toBe(false);
|
||||
expect(normalizeStackResourcePresent(1)).toBe(false);
|
||||
expect(normalizeStackResourcePresent(null)).toBe(false);
|
||||
expect(normalizeStackResourcePresent(undefined)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('source-row classifier', () => {
|
||||
const revision = (lifecycleStatus: unknown): Record<string, unknown> => ({
|
||||
schemaVersion: 1,
|
||||
targetMode: 'direct',
|
||||
lifecycleStatus,
|
||||
});
|
||||
|
||||
it('authorizes a live present stack by stack read', () => {
|
||||
expect(classifySourceRow({
|
||||
stackName: 'web',
|
||||
gitopsRevision: revision('active'),
|
||||
stackResourcePresent: true,
|
||||
})).toEqual({ kind: 'stack_read', stackName: 'web' });
|
||||
});
|
||||
|
||||
it('authorizes a detached stack whose resource is present', () => {
|
||||
expect(classifySourceRow({
|
||||
stackName: 'web',
|
||||
gitopsRevision: revision('detached'),
|
||||
stackResourcePresent: true,
|
||||
})).toEqual({ kind: 'stack_read', stackName: 'web' });
|
||||
});
|
||||
|
||||
it('falls back to Admin for every unprovable row', () => {
|
||||
const admin = { kind: 'admin' };
|
||||
expect(classifySourceRow({ stackName: '', gitopsRevision: revision('active'), stackResourcePresent: true })).toEqual(admin);
|
||||
expect(classifySourceRow({ stackName: null, gitopsRevision: revision('active'), stackResourcePresent: true })).toEqual(admin);
|
||||
expect(classifySourceRow({ stackName: 'web', gitopsRevision: null, stackResourcePresent: true })).toEqual(admin);
|
||||
expect(classifySourceRow({ stackName: 'web', gitopsRevision: 'nope', stackResourcePresent: true })).toEqual(admin);
|
||||
expect(classifySourceRow({ stackName: 'web', gitopsRevision: revision(undefined), stackResourcePresent: true })).toEqual(admin);
|
||||
expect(classifySourceRow({ stackName: 'web', gitopsRevision: revision('deleted'), stackResourcePresent: true })).toEqual(admin);
|
||||
expect(classifySourceRow({ stackName: 'web', gitopsRevision: revision('creating'), stackResourcePresent: true })).toEqual(admin);
|
||||
expect(classifySourceRow({ stackName: 'web', gitopsRevision: revision('active'), stackResourcePresent: false })).toEqual(admin);
|
||||
expect(classifySourceRow({ stackName: 'web', gitopsRevision: revision('active'), stackResourcePresent: 'yes' })).toEqual(admin);
|
||||
});
|
||||
|
||||
it('sends a stack with no GitOps application to Admin', () => {
|
||||
// The not_applicable projection carries no lifecycleStatus at all.
|
||||
expect(classifySourceRow({
|
||||
stackName: 'web',
|
||||
gitopsRevision: { schemaVersion: 1, targetMode: 'not_applicable', applicationId: null },
|
||||
stackResourcePresent: true,
|
||||
})).toEqual({ kind: 'admin' });
|
||||
});
|
||||
});
|
||||
|
||||
describe('history-row classifier', () => {
|
||||
it('authorizes from the application lifecycle, not the recorded delta', () => {
|
||||
expect(classifyHistoryRow({
|
||||
stackName: 'web',
|
||||
applicationLifecycleStatus: 'active',
|
||||
stackResourcePresent: true,
|
||||
})).toEqual({ kind: 'stack_read', stackName: 'web' });
|
||||
});
|
||||
|
||||
it('falls back to the audit audience for every unprovable row', () => {
|
||||
// History entries are an audit trail, so an entry nobody can tie to a
|
||||
// readable stack goes to whoever audits rather than to Admin alone.
|
||||
const audit = { kind: 'audit' };
|
||||
const held = { stackResourcePresent: true };
|
||||
expect(classifyHistoryRow({ stackName: null, applicationLifecycleStatus: 'active', ...held })).toEqual(audit);
|
||||
expect(classifyHistoryRow({ stackName: '', applicationLifecycleStatus: 'active', ...held })).toEqual(audit);
|
||||
expect(classifyHistoryRow({ stackName: 'web', applicationLifecycleStatus: undefined, ...held })).toEqual(audit);
|
||||
expect(classifyHistoryRow({ stackName: 'web', applicationLifecycleStatus: 'deleted', ...held })).toEqual(audit);
|
||||
expect(classifyHistoryRow({ stackName: 'web', applicationLifecycleStatus: 'creating', ...held })).toEqual(audit);
|
||||
expect(classifyHistoryRow({
|
||||
stackName: 'web',
|
||||
applicationLifecycleStatus: 'active',
|
||||
stackResourcePresent: false,
|
||||
})).toEqual(audit);
|
||||
});
|
||||
|
||||
it('sends every detached predecessor to the audit audience', () => {
|
||||
// Detach leaves the files on disk, but a stack grant covers whatever
|
||||
// occupies the name today, and nothing in these tables can prove the
|
||||
// detached application still does: a Blueprint successor records the
|
||||
// name off-row (`deploy_stack_name`) and a plain Compose stack recreated
|
||||
// at the name leaves no trace at all. An allowance that holds only for
|
||||
// the successors this classifier happens to see is worse than none, so
|
||||
// detach joins `deleted` and `creating`.
|
||||
const detached = { stackName: 'web', applicationLifecycleStatus: 'detached', stackResourcePresent: true };
|
||||
expect(classifyHistoryRow({ ...detached })).toEqual({ kind: 'audit' });
|
||||
});
|
||||
|
||||
it('keeps source rows on Admin rather than the audit audience', () => {
|
||||
// Git configuration is not a record of events, so an auditing mandate
|
||||
// does not reach it.
|
||||
expect(classifySourceRow({
|
||||
stackName: 'web',
|
||||
gitopsRevision: { schemaVersion: 1, targetMode: 'direct', lifecycleStatus: 'deleted' },
|
||||
stackResourcePresent: true,
|
||||
})).toEqual({ kind: 'admin' });
|
||||
});
|
||||
});
|
||||
|
||||
it('honours the scan cap as the query bound', () => {
|
||||
// Asserts the cap is actually applied to the read, not merely declared.
|
||||
expect(query({}, null, HISTORY_SCAN_CAP).length).toBeLessThanOrEqual(HISTORY_SCAN_CAP);
|
||||
expect(query({}, null, 1)).toHaveLength(1);
|
||||
});
|
||||
|
||||
describe('filter and limit parsing', () => {
|
||||
it('reads every supported filter off the query string', () => {
|
||||
const parsed = parseHistoryFilters({
|
||||
applicationId: 'app-1',
|
||||
repoIdentity: 'https://github.com/org/repo.git',
|
||||
configuredRef: 'main',
|
||||
commitSha: 'sha-1',
|
||||
generationId: 'gen-1',
|
||||
artifactSetId: 'art-1',
|
||||
blueprintId: '9',
|
||||
rolloutCandidateId: 'cand-1',
|
||||
rolloutGenerationId: 'rgen-1',
|
||||
nodeId: '3',
|
||||
trigger: 'manual',
|
||||
actor: 'operator',
|
||||
outcome: 'failed',
|
||||
});
|
||||
if (!parsed.ok) throw new Error(parsed.message);
|
||||
expect(parsed.filters).toEqual({
|
||||
applicationId: 'app-1',
|
||||
repoIdentity: 'https://github.com/org/repo.git',
|
||||
configuredRef: 'main',
|
||||
commitSha: 'sha-1',
|
||||
generationId: 'gen-1',
|
||||
artifactSetId: 'art-1',
|
||||
blueprintId: 9,
|
||||
rolloutCandidateId: 'cand-1',
|
||||
rolloutGenerationId: 'rgen-1',
|
||||
nodeId: 3,
|
||||
trigger: 'manual',
|
||||
actor: 'operator',
|
||||
outcome: 'failed',
|
||||
});
|
||||
});
|
||||
|
||||
it('never takes stackName from the caller', () => {
|
||||
const parsed = parseHistoryFilters({ stackName: 'somebody-elses-stack' });
|
||||
if (!parsed.ok) throw new Error(parsed.message);
|
||||
expect(parsed.filters.stackName).toBeUndefined();
|
||||
});
|
||||
|
||||
it('rejects a recognized filter with an unusable value', () => {
|
||||
expect(parseHistoryFilters({ outcome: 'success' })).toEqual({
|
||||
ok: false,
|
||||
message: expect.stringContaining('outcome'),
|
||||
});
|
||||
expect(parseHistoryFilters({ nodeId: 'abc' })).toEqual({
|
||||
ok: false,
|
||||
message: expect.stringContaining('nodeId'),
|
||||
});
|
||||
expect(parseHistoryFilters({ blueprintId: '1.5' })).toEqual({
|
||||
ok: false,
|
||||
message: expect.stringContaining('blueprintId'),
|
||||
});
|
||||
});
|
||||
|
||||
it('clamps the page size and falls back for nonsense', () => {
|
||||
expect(parseLimit('5000')).toBe(HISTORY_MAX_LIMIT);
|
||||
expect(parseLimit('10')).toBe(10);
|
||||
expect(parseLimit(undefined)).toBe(HISTORY_DEFAULT_LIMIT);
|
||||
expect(parseLimit('0')).toBe(HISTORY_DEFAULT_LIMIT);
|
||||
expect(parseLimit('-1')).toBe(HISTORY_DEFAULT_LIMIT);
|
||||
expect(parseLimit('abc')).toBe(HISTORY_DEFAULT_LIMIT);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
function query(
|
||||
filters: Parameters<typeof queryHistoryRows>[1],
|
||||
cursor: Parameters<typeof queryHistoryRows>[2] = null,
|
||||
limit = 50,
|
||||
): GitOpsHistoryRow[] {
|
||||
return queryHistoryRows(DatabaseService.getInstance().getDb(), filters, cursor, limit);
|
||||
}
|
||||
|
||||
function seedHistory(): void {
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
const base = application();
|
||||
// The activation event belongs to the application, not to a node, so it
|
||||
// carries no node id. This is the row shape every application-level stage
|
||||
// writes and the one a `node_id = ?` filter silently drops.
|
||||
insertHistory(db, {
|
||||
application: base,
|
||||
nodeId: null,
|
||||
dedupeTarget: 'app',
|
||||
operationId: 'op-app-level',
|
||||
stage: 'application_activated',
|
||||
outcome: 'committed',
|
||||
trigger: 'manual',
|
||||
actor: 'operator-1',
|
||||
before: { lifecycleStatus: null },
|
||||
after: { lifecycleStatus: 'active', targetMode: 'direct' },
|
||||
at: 500,
|
||||
});
|
||||
insertHistory(db, {
|
||||
application: base,
|
||||
nodeId: 1,
|
||||
dedupeTarget: 'app',
|
||||
operationId: 'op-a',
|
||||
stage: 'fetched',
|
||||
outcome: 'committed',
|
||||
trigger: 'manual',
|
||||
actor: 'operator-1',
|
||||
before: { desiredCommitSha: null },
|
||||
after: { desiredCommitSha: 'sha-a' },
|
||||
commitSha: 'sha-a',
|
||||
at: 1000,
|
||||
});
|
||||
insertHistory(db, {
|
||||
application: base,
|
||||
nodeId: 7,
|
||||
dedupeTarget: 'app',
|
||||
operationId: 'op-b',
|
||||
stage: 'fetched',
|
||||
outcome: 'committed',
|
||||
trigger: 'webhook',
|
||||
actor: 'operator-2',
|
||||
before: { desiredCommitSha: null },
|
||||
after: { desiredCommitSha: 'sha-b' },
|
||||
commitSha: 'sha-b',
|
||||
at: 2000,
|
||||
});
|
||||
insertHistory(db, {
|
||||
application: { ...base, rollout_candidate_id: 'cand-1' },
|
||||
nodeId: 1,
|
||||
dedupeTarget: 'app',
|
||||
operationId: 'op-c',
|
||||
stage: 'apply_failed',
|
||||
outcome: 'failed',
|
||||
trigger: 'manual',
|
||||
actor: 'operator-1',
|
||||
before: {},
|
||||
after: { failureClass: 'validation' },
|
||||
commitSha: 'sha-c',
|
||||
rolloutCandidateId: 'cand-1',
|
||||
at: 3000,
|
||||
});
|
||||
// Two rows inside one millisecond, which a real transaction produces.
|
||||
for (const operationId of ['op-tie-1', 'op-tie-2']) {
|
||||
insertHistory(db, {
|
||||
application: { ...base, stack_name: 'tie-web', lifecycle_key: 'direct:tie-web' },
|
||||
nodeId: 1,
|
||||
dedupeTarget: 'app',
|
||||
operationId,
|
||||
stage: 'fetched',
|
||||
outcome: 'committed',
|
||||
trigger: 'manual',
|
||||
actor: 'operator-1',
|
||||
before: {},
|
||||
after: {},
|
||||
commitSha: 'sha-tie',
|
||||
at: 4000,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function application(): GitOpsApplicationRow {
|
||||
return {
|
||||
id: 'app-history',
|
||||
lifecycle_key: 'direct:history-web',
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'direct',
|
||||
stack_name: 'history-web',
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,636 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { IncomingMessage } from 'http';
|
||||
import { Socket } from 'net';
|
||||
import zlib from 'zlib';
|
||||
import {
|
||||
IDENTITY_PROXY_MAX_BYTES,
|
||||
handleIdentityResponse,
|
||||
isGitOpsHistoryRoute,
|
||||
isGitOpsIdentityJsonRoute,
|
||||
prepareIdentityQuery,
|
||||
rewriteIdentityPayload,
|
||||
stripConditionalRequestHeaders,
|
||||
filterIdentityCollection,
|
||||
filterRemoteIdentityPayload,
|
||||
type IdentityResponseSink,
|
||||
type IdentityTerminalKind,
|
||||
} from '../proxy/gitopsIdentityProxy';
|
||||
|
||||
describe('gitops identity proxy', () => {
|
||||
describe('route matching', () => {
|
||||
it('intercepts the identity GETs', () => {
|
||||
for (const path of [
|
||||
'/git-sources',
|
||||
'/git-sources/history',
|
||||
'/stacks/web/git-source',
|
||||
'/stacks/web/git-source/history',
|
||||
'/stacks/web/drift',
|
||||
]) {
|
||||
expect(isGitOpsIdentityJsonRoute(path, 'GET')).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('intercepts the drift re-check, the one mutation that answers with a revision', () => {
|
||||
// The GET beside it is rewritten, and both return the same projection
|
||||
// object. Leaving the re-check on the streaming hop would make one object
|
||||
// carry the hub's node numbering or the remote's depending only on how it
|
||||
// was asked for.
|
||||
expect(isGitOpsIdentityJsonRoute('/stacks/web/drift/recheck', 'POST')).toBe(true);
|
||||
expect(isGitOpsIdentityJsonRoute('/stacks/web/drift/recheck', 'GET')).toBe(false);
|
||||
expect(isGitOpsIdentityJsonRoute('/stacks/web/drift', 'POST')).toBe(false);
|
||||
});
|
||||
|
||||
it('leaves streaming and unrelated routes to the streaming hop', () => {
|
||||
// Buffering any of these to rewrite identities they do not carry would
|
||||
// break streaming or cap a legitimately large response.
|
||||
for (const path of [
|
||||
'/stacks/web/logs',
|
||||
'/containers/abc/logs',
|
||||
'/stacks/web/files/download',
|
||||
'/git-sources/browse',
|
||||
'/stacks/web/git-source/manifest',
|
||||
'/blueprints',
|
||||
]) {
|
||||
expect(isGitOpsIdentityJsonRoute(path, 'GET')).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it('never intercepts a mutation of a git-source route', () => {
|
||||
for (const method of ['POST', 'PUT', 'DELETE', 'PATCH']) {
|
||||
expect(isGitOpsIdentityJsonRoute('/stacks/web/git-source', method)).toBe(false);
|
||||
expect(isGitOpsIdentityJsonRoute('/git-sources', method)).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it('recognizes only the history pair as history', () => {
|
||||
expect(isGitOpsHistoryRoute('/git-sources/history')).toBe(true);
|
||||
expect(isGitOpsHistoryRoute('/stacks/web/git-source/history')).toBe(true);
|
||||
expect(isGitOpsHistoryRoute('/git-sources')).toBe(false);
|
||||
expect(isGitOpsHistoryRoute('/stacks/web/git-source')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('outbound query', () => {
|
||||
const prep = (qs: string, path: string, hubNodeId: number | undefined) =>
|
||||
prepareIdentityQuery(new URLSearchParams(qs), path, hubNodeId);
|
||||
|
||||
it('always strips a caller-supplied local-target flag', () => {
|
||||
// Only the hub may tell a remote to filter to its own node.
|
||||
const result = prep('gitopsLocalTarget=1', '/git-sources', 3);
|
||||
if (result.kind !== 'forward') throw new Error('expected forward');
|
||||
expect(result.search.get('gitopsLocalTarget')).toBeNull();
|
||||
});
|
||||
|
||||
it('strips nodeId on the non-history routes', () => {
|
||||
const result = prep('nodeId=3', '/git-sources', 3);
|
||||
if (result.kind !== 'forward') throw new Error('expected forward');
|
||||
expect(result.search.get('nodeId')).toBeNull();
|
||||
expect(result.search.get('gitopsLocalTarget')).toBeNull();
|
||||
});
|
||||
|
||||
it('translates a matching node into the local-target flag on history', () => {
|
||||
const result = prep('nodeId=3&limit=10', '/git-sources/history', 3);
|
||||
if (result.kind !== 'forward') throw new Error('expected forward');
|
||||
expect(result.search.get('nodeId')).toBeNull();
|
||||
expect(result.search.get('gitopsLocalTarget')).toBe('1');
|
||||
expect(result.search.get('limit')).toBe('10');
|
||||
});
|
||||
|
||||
it('refuses history for a node this hop cannot answer for', () => {
|
||||
// Forwarding would make the remote answer about itself, which reads as an
|
||||
// answer to a question nobody asked.
|
||||
expect(prep('nodeId=7', '/git-sources/history', 3).kind).toBe('refuse');
|
||||
expect(prep('nodeId=7', '/stacks/web/git-source/history', 3).kind).toBe('refuse');
|
||||
expect(prep('nodeId=3', '/git-sources/history', undefined).kind).toBe('refuse');
|
||||
});
|
||||
|
||||
it('narrows to the proxied node even when history names none', () => {
|
||||
// A request routed to this node is a question about this node. Without
|
||||
// the filter the remote answers with rows from all of its own nodes, and
|
||||
// the hub stamps a single id across every row it rewrites, so those rows
|
||||
// would come back claiming to belong to a node they do not.
|
||||
const result = prep('limit=5', '/git-sources/history', 3);
|
||||
if (result.kind !== 'forward') throw new Error('expected forward');
|
||||
expect(result.search.get('gitopsLocalTarget')).toBe('1');
|
||||
expect(result.search.get('limit')).toBe('5');
|
||||
});
|
||||
|
||||
it('leaves the non-history routes without a node filter', () => {
|
||||
// Git sources are per-instance rather than per-node, so there is nothing
|
||||
// to narrow.
|
||||
const result = prep('', '/git-sources', 3);
|
||||
if (result.kind !== 'forward') throw new Error('expected forward');
|
||||
expect(result.search.get('gitopsLocalTarget')).toBeNull();
|
||||
});
|
||||
|
||||
it('strips a forged local-target even when it refuses', () => {
|
||||
expect(prep('nodeId=7&gitopsLocalTarget=1', '/git-sources/history', 3).kind).toBe('refuse');
|
||||
});
|
||||
});
|
||||
|
||||
describe('conditional requests', () => {
|
||||
it('strips every conditional request header before forwarding', () => {
|
||||
const removed: string[] = [];
|
||||
stripConditionalRequestHeaders({ removeHeader: (name) => removed.push(name) });
|
||||
expect(removed).toEqual(['if-none-match', 'if-modified-since', 'if-match', 'if-unmodified-since']);
|
||||
});
|
||||
|
||||
it('reruns the hub filter when a caller revalidates after a permission change', async () => {
|
||||
// First read: the caller may see both rows. The answer is filtered for
|
||||
// them and carries no validator to cache against.
|
||||
const row = (name: string): unknown => ({
|
||||
nodeId: 1,
|
||||
stack_name: name,
|
||||
gitopsRevision: { lifecycleStatus: 'active' },
|
||||
stackResourcePresent: true,
|
||||
});
|
||||
const first = await runResponse({
|
||||
status: 200,
|
||||
headers: { etag: 'W/"upstream-1"' },
|
||||
body: JSON.stringify([row('kept'), row('revoked')]),
|
||||
});
|
||||
expect(first.kind).toBe('rewrite');
|
||||
expect(JSON.parse(first.body.toString())).toHaveLength(2);
|
||||
expect(first.headers.etag).toBeUndefined();
|
||||
expect(first.headers['cache-control']).toBe('no-store');
|
||||
|
||||
// The revalidation attempt: a conditional request is stripped on its way
|
||||
// up, so the remote cannot answer 304 and the hub must classify every
|
||||
// row again under the grants in force now.
|
||||
const outbound: Record<string, string> = { 'if-none-match': 'W/"upstream-1"', accept: 'application/json' };
|
||||
stripConditionalRequestHeaders({ removeHeader: (name) => { delete outbound[name]; } });
|
||||
expect(outbound['if-none-match']).toBeUndefined();
|
||||
expect(outbound.accept).toBe('application/json');
|
||||
|
||||
// The fresh answer reflects the revocation: one row survives.
|
||||
const second = await runResponse({
|
||||
status: 200,
|
||||
body: JSON.stringify([row('kept'), row('revoked')]),
|
||||
transform: (payload) => (filterRemoteIdentityPayload(
|
||||
'/git-sources',
|
||||
payload,
|
||||
// The caller may still prove every row except the revoked stack.
|
||||
(requirement) => !(requirement.kind === 'stack_read' && requirement.stackName === 'revoked'),
|
||||
1,
|
||||
)),
|
||||
});
|
||||
expect(second.kind).toBe('rewrite');
|
||||
expect(JSON.parse(second.body.toString())).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('node id rewriting', () => {
|
||||
it('rewrites every enumerated position on a source row', () => {
|
||||
const payload = [{
|
||||
stack_name: 'web',
|
||||
nodeId: 1,
|
||||
stackResourcePresent: true,
|
||||
targets: [{ nodeId: 1 }, { nodeId: 2 }],
|
||||
gitopsRevision: {
|
||||
targets: [{ nodeId: 1 }],
|
||||
drift: [{ affectedTargets: [{ nodeId: 1 }, { nodeId: null }] }],
|
||||
},
|
||||
gitopsRevisions: [{ targets: [{ nodeId: 9 }] }],
|
||||
}];
|
||||
rewriteIdentityPayload(payload, 42);
|
||||
const row = payload[0];
|
||||
expect(row.nodeId).toBe(42);
|
||||
expect(row.targets.map(t => t.nodeId)).toEqual([42, 42]);
|
||||
expect(row.gitopsRevision.targets[0]?.nodeId).toBe(42);
|
||||
expect(row.gitopsRevision.drift[0]?.affectedTargets[0]?.nodeId).toBe(42);
|
||||
expect(row.gitopsRevisions[0]?.targets[0]?.nodeId).toBe(42);
|
||||
});
|
||||
|
||||
it('preserves a null node rather than inventing a placement', () => {
|
||||
const payload = [{ nodeId: null, targets: [{ nodeId: null }] }];
|
||||
rewriteIdentityPayload(payload, 42);
|
||||
expect(payload[0]?.nodeId).toBeNull();
|
||||
expect(payload[0]?.targets[0]?.nodeId).toBeNull();
|
||||
});
|
||||
|
||||
it('rewrites history items including their before and after projections', () => {
|
||||
const payload = {
|
||||
items: [{
|
||||
nodeId: 1,
|
||||
stackName: 'web',
|
||||
before: { targets: [{ nodeId: 1 }] },
|
||||
after: { targets: [{ nodeId: 1 }], drift: [{ affectedTargets: [{ nodeId: 1 }] }] },
|
||||
}],
|
||||
nextCursor: '100.abc',
|
||||
};
|
||||
rewriteIdentityPayload(payload, 42);
|
||||
const item = payload.items[0];
|
||||
expect(item?.nodeId).toBe(42);
|
||||
expect(item?.before.targets[0]?.nodeId).toBe(42);
|
||||
expect(item?.after.drift[0]?.affectedTargets[0]?.nodeId).toBe(42);
|
||||
expect(payload.nextCursor).toBe('100.abc');
|
||||
});
|
||||
|
||||
it('rewrites the revision on a drift payload without touching the ledger', () => {
|
||||
// The drift payload is a single object whose GitOps content hangs off
|
||||
// `gitopsRevision`. Its own `findings` and `ledger` are the compose vs
|
||||
// runtime record and carry no node identity, so they must come back byte
|
||||
// for byte.
|
||||
const payload = {
|
||||
stack: 'web',
|
||||
status: 'drifted',
|
||||
findings: [{ service: 'app', kind: 'image-mismatch' }],
|
||||
ledger: [{ service: 'app', kind: 'image-mismatch', detectedAt: 5 }],
|
||||
gitopsRevision: {
|
||||
targets: [{ nodeId: 1 }],
|
||||
drift: [{ affectedTargets: [{ nodeId: 1 }] }],
|
||||
},
|
||||
};
|
||||
rewriteIdentityPayload(payload, 42);
|
||||
expect(payload.gitopsRevision.targets[0]?.nodeId).toBe(42);
|
||||
expect(payload.gitopsRevision.drift[0]?.affectedTargets[0]?.nodeId).toBe(42);
|
||||
expect(payload.findings).toEqual([{ service: 'app', kind: 'image-mismatch' }]);
|
||||
expect(payload.ledger).toEqual([{ service: 'app', kind: 'image-mismatch', detectedAt: 5 }]);
|
||||
expect(payload.stack).toBe('web');
|
||||
});
|
||||
|
||||
it('leaves strings and unlisted keys untouched', () => {
|
||||
const payload = {
|
||||
applicationId: 'app-1',
|
||||
stackName: 'web',
|
||||
nodeId: '1',
|
||||
someOtherId: 1,
|
||||
targets: [{ nodeId: 1, stackName: 'web' }],
|
||||
};
|
||||
rewriteIdentityPayload(payload, 42);
|
||||
expect(payload.applicationId).toBe('app-1');
|
||||
expect(payload.stackName).toBe('web');
|
||||
expect(payload.nodeId).toBe('1');
|
||||
expect(payload.someOtherId).toBe(1);
|
||||
expect(payload.targets[0]?.stackName).toBe('web');
|
||||
});
|
||||
});
|
||||
|
||||
describe('collection filtering', () => {
|
||||
it('drops rows in place and preserves order', () => {
|
||||
const payload = [{ id: 'a' }, { id: 'b' }, { id: 'c' }];
|
||||
const filtered = filterIdentityCollection(payload, row => (row as { id: string }).id !== 'b', () => true);
|
||||
expect(filtered).toEqual([{ id: 'a' }, { id: 'c' }]);
|
||||
});
|
||||
|
||||
it('keeps the cursor when a page filters down to nothing', () => {
|
||||
// Otherwise a caller whose grants reject a whole window concludes the
|
||||
// history is empty instead of paging on.
|
||||
const payload = { items: [{ id: 'a' }], nextCursor: '100.abc' };
|
||||
const filtered = filterIdentityCollection(payload, () => true, () => false);
|
||||
expect(filtered).toEqual({ items: [], nextCursor: '100.abc' });
|
||||
});
|
||||
});
|
||||
|
||||
describe('hub re-authorization of remote rows', () => {
|
||||
// A viewer holds global stack:read, so a row that reduces to a stack read
|
||||
// survives while anything unprovable falls to Admin or audit.
|
||||
const asViewer = (requirement: { kind: string }): boolean => requirement.kind === 'stack_read';
|
||||
|
||||
const sourceRow = (stackName: string, lifecycleStatus: string, present: boolean) => ({
|
||||
stack_name: stackName,
|
||||
gitopsRevision: { schemaVersion: 1, targetMode: 'direct', lifecycleStatus },
|
||||
stackResourcePresent: present,
|
||||
});
|
||||
|
||||
const historyItem = (stackName: string, lifecycleStatus: string | null, present: boolean) => ({
|
||||
stackName,
|
||||
applicationLifecycleStatus: lifecycleStatus,
|
||||
stackResourcePresent: present,
|
||||
});
|
||||
|
||||
it('filters a source collection on the pre-rewrite path', () => {
|
||||
// The path must be the one the hub saw before pathRewrite prefixed
|
||||
// `/api`. Passing the rewritten path matches nothing and silently skips
|
||||
// re-authorization on every request.
|
||||
const payload = [
|
||||
sourceRow('web', 'active', true),
|
||||
sourceRow('gone', 'deleted', true),
|
||||
sourceRow('absent', 'active', false),
|
||||
];
|
||||
const filtered = filterRemoteIdentityPayload('/git-sources', payload, asViewer, 7);
|
||||
expect(Array.isArray(filtered)).toBe(true);
|
||||
expect((filtered as Array<{ stack_name: string }>).map(r => r.stack_name)).toEqual(['web']);
|
||||
});
|
||||
|
||||
it('filters a history collection on the pre-rewrite path', () => {
|
||||
const payload = {
|
||||
items: [
|
||||
historyItem('web', 'active', true),
|
||||
historyItem('creating-one', 'creating', true),
|
||||
historyItem('no-app', null, true),
|
||||
],
|
||||
nextCursor: '100.abc',
|
||||
};
|
||||
const filtered = filterRemoteIdentityPayload('/git-sources/history', payload, asViewer, 7);
|
||||
const items = (filtered as { items: Array<{ stackName: string }> }).items;
|
||||
expect(items.map(i => i.stackName)).toEqual(['web']);
|
||||
expect((filtered as { nextCursor: string }).nextCursor).toBe('100.abc');
|
||||
});
|
||||
|
||||
it('leaves a drift payload unfiltered', () => {
|
||||
// The drift routes are per-stack, authorized by name before the hop, and
|
||||
// return one object rather than a cross-stack collection. Re-filtering
|
||||
// them would hide a stack's own drift from the operator who just proved
|
||||
// they may read it.
|
||||
const payload = { stack: 'gone', gitopsRevision: { schemaVersion: 1, targetMode: 'direct', lifecycleStatus: 'deleted' } };
|
||||
expect(filterRemoteIdentityPayload('/stacks/gone/drift', payload, asViewer, 7)).toEqual(payload);
|
||||
expect(filterRemoteIdentityPayload('/stacks/gone/drift/recheck', payload, asViewer, 7)).toEqual(payload);
|
||||
});
|
||||
|
||||
it('does not match the rewritten path, which is why the hop stashes the original', () => {
|
||||
// Pins the defect directly: with `/api` prefixed, nothing is filtered.
|
||||
const payload = [sourceRow('gone', 'deleted', true)];
|
||||
const filtered = filterRemoteIdentityPayload('/api/git-sources', payload, asViewer, 7);
|
||||
expect(filtered).toEqual(payload);
|
||||
});
|
||||
|
||||
it('leaves per-stack routes unfiltered, since they were authorized by name', () => {
|
||||
const payload = { items: [historyItem('web', 'creating', true)] };
|
||||
const filtered = filterRemoteIdentityPayload('/stacks/web/git-source/history', payload, asViewer, 7);
|
||||
expect((filtered as { items: unknown[] }).items).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('terminal response rules', () => {
|
||||
it('rewrites a JSON 200 and reframes the body', async () => {
|
||||
const result = await runResponse({ status: 200, body: JSON.stringify([{ nodeId: 1 }]) });
|
||||
expect(result.kind).toBe('rewrite');
|
||||
expect(result.statusCode).toBe(200);
|
||||
expect(result.headers['content-type']).toBe('application/json; charset=utf-8');
|
||||
expect(JSON.parse(result.body.toString())).toEqual([{ nodeId: 42 }]);
|
||||
expect(result.headers['content-length']).toBe(String(result.body.length));
|
||||
});
|
||||
|
||||
it('decodes a gzipped body before rewriting it', async () => {
|
||||
const result = await runResponse({
|
||||
status: 200,
|
||||
body: zlib.gzipSync(Buffer.from(JSON.stringify([{ nodeId: 1 }]))),
|
||||
headers: { 'content-encoding': 'gzip' },
|
||||
});
|
||||
expect(result.kind).toBe('rewrite');
|
||||
expect(JSON.parse(result.body.toString())).toEqual([{ nodeId: 42 }]);
|
||||
// The upstream framing described bytes that no longer exist, so it must
|
||||
// be gone rather than replayed over a body of a different length.
|
||||
expect(result.headers['content-encoding']).toBeUndefined();
|
||||
expect(result.headers['transfer-encoding']).toBeUndefined();
|
||||
expect(result.headers['content-length']).toBe(String(result.body.length));
|
||||
expect(result.finalizeCalls).toBe(1);
|
||||
});
|
||||
|
||||
it('accumulates across chunks rather than checking one at a time', async () => {
|
||||
// A per-chunk check would let an arbitrarily large body through in small
|
||||
// pieces, so the cap has to be tested against a stream, not a buffer.
|
||||
const result = await runResponse({
|
||||
status: 200,
|
||||
body: '',
|
||||
chunks: Array.from({ length: 40 }, () => Buffer.alloc(32 * 1024, 0x61)),
|
||||
});
|
||||
expect(result.kind).toBe('too_large');
|
||||
expect(result.statusCode).toBe(502);
|
||||
});
|
||||
|
||||
it('allows a body exactly at the ceiling', async () => {
|
||||
const exact = Buffer.concat([
|
||||
Buffer.from('"'),
|
||||
Buffer.alloc(IDENTITY_PROXY_MAX_BYTES - 2, 0x61),
|
||||
Buffer.from('"'),
|
||||
]);
|
||||
const result = await runResponse({ status: 200, body: exact });
|
||||
expect(result.kind).toBe('rewrite');
|
||||
});
|
||||
|
||||
it('passes a non-2xx body through without rewriting', async () => {
|
||||
const result = await runResponse({ status: 403, body: JSON.stringify({ error: 'denied' }) });
|
||||
expect(result.kind).toBe('passthrough');
|
||||
expect(result.statusCode).toBe(403);
|
||||
expect(JSON.parse(result.body.toString())).toEqual({ error: 'denied' });
|
||||
});
|
||||
|
||||
it('refuses a 200 that will not parse instead of relaying it', async () => {
|
||||
// These routes answer with JSON on success, so an unparseable 200 is a
|
||||
// body the hub could not read. Relaying it under the remote's success
|
||||
// status would hand the client an unrewritten, unauthorized payload.
|
||||
const result = await runResponse({ status: 200, body: 'not json at all' });
|
||||
expect(result.kind).toBe('parse_error');
|
||||
expect(result.statusCode).toBe(502);
|
||||
expect(JSON.parse(result.body.toString()).code).toBe('gitops_proxy_unparseable');
|
||||
});
|
||||
|
||||
it('blames itself, not the remote, when its own rewrite throws', async () => {
|
||||
const result = await runResponse({
|
||||
status: 200,
|
||||
body: JSON.stringify([{ nodeId: 1 }]),
|
||||
transform: () => { throw new Error('permission lookup failed'); },
|
||||
});
|
||||
expect(result.kind).toBe('rewrite_failed');
|
||||
// A 500, because everything in the transform runs on this instance.
|
||||
expect(result.statusCode).toBe(500);
|
||||
expect(JSON.parse(result.body.toString()).code).toBe('gitops_proxy_rewrite_failed');
|
||||
});
|
||||
|
||||
it('treats a stream that ends incomplete as truncation', async () => {
|
||||
// Node's own truncation signal, rather than a hand-fired event: the
|
||||
// message ends without `complete`, which is what a remote dying mid-body
|
||||
// actually looks like.
|
||||
const result = await runResponse({ status: 200, body: '[{"nodeId":1}]', endIncomplete: true });
|
||||
expect(result.kind).toBe('upstream_failed');
|
||||
expect(result.statusCode).toBe(502);
|
||||
});
|
||||
|
||||
it('writes no body for 204 and answers a 304 without the upstream validators', async () => {
|
||||
const noContent = await runResponse({ status: 204, body: '' });
|
||||
expect(noContent.statusCode).toBe(204);
|
||||
expect(noContent.body.length).toBe(0);
|
||||
expect(noContent.headers['cache-control']).toBe('no-store');
|
||||
|
||||
const notModified = await runResponse({
|
||||
status: 304,
|
||||
body: '',
|
||||
headers: { etag: 'W/"abc"', 'last-modified': 'Mon, 18 Aug 2026 00:00:00 GMT', 'cache-control': 'max-age=60' },
|
||||
});
|
||||
expect(notModified.statusCode).toBe(304);
|
||||
expect(notModified.body.length).toBe(0);
|
||||
// The upstream validators describe the remote's unfiltered
|
||||
// representation, not the page this hub sends, so relaying them would
|
||||
// let a cached page outlive the authorization it was filtered under.
|
||||
expect(notModified.headers.etag).toBeUndefined();
|
||||
expect(notModified.headers['last-modified']).toBeUndefined();
|
||||
expect(notModified.headers['cache-control']).toBe('no-store');
|
||||
});
|
||||
|
||||
it('answers a rewritten page with no-store and no cache validators', async () => {
|
||||
const result = await runResponse({
|
||||
status: 200,
|
||||
body: JSON.stringify([{ nodeId: 1, stackName: 'web' }]),
|
||||
headers: {
|
||||
etag: 'W/"upstream-1"',
|
||||
'last-modified': 'Mon, 18 Aug 2026 00:00:00 GMT',
|
||||
expires: 'Mon, 18 Aug 2026 01:00:00 GMT',
|
||||
vary: 'Accept-Encoding',
|
||||
'cache-control': 'max-age=60',
|
||||
},
|
||||
});
|
||||
expect(result.kind).toBe('rewrite');
|
||||
expect(result.statusCode).toBe(200);
|
||||
expect(result.headers['cache-control']).toBe('no-store');
|
||||
expect(result.headers.etag).toBeUndefined();
|
||||
expect(result.headers['last-modified']).toBeUndefined();
|
||||
expect(result.headers.expires).toBeUndefined();
|
||||
expect(result.headers.vary).toBeUndefined();
|
||||
});
|
||||
|
||||
it('preserves the location on a redirect', async () => {
|
||||
const result = await runResponse({
|
||||
status: 302,
|
||||
body: '',
|
||||
headers: { location: '/api/git-sources' },
|
||||
});
|
||||
expect(result.statusCode).toBe(302);
|
||||
expect(result.headers.location).toBe('/api/git-sources');
|
||||
// Every answer this hop writes is uncacheable, redirects included.
|
||||
expect(result.headers['cache-control']).toBe('no-store');
|
||||
});
|
||||
|
||||
it('refuses a body past the ceiling with its own status', async () => {
|
||||
const oversized = 'x'.repeat(IDENTITY_PROXY_MAX_BYTES + 1024);
|
||||
const result = await runResponse({ status: 200, body: JSON.stringify([oversized]) });
|
||||
expect(result.kind).toBe('too_large');
|
||||
// Not the upstream 200: the hub could not read the answer.
|
||||
expect(result.statusCode).toBe(502);
|
||||
expect(JSON.parse(result.body.toString()).code).toBe('gitops_proxy_too_large');
|
||||
expect(result.headers['cache-control']).toBe('no-store');
|
||||
});
|
||||
|
||||
it('reports an undecodable body as a decode failure', async () => {
|
||||
const result = await runResponse({
|
||||
status: 200,
|
||||
body: Buffer.from('this is not gzip'),
|
||||
headers: { 'content-encoding': 'gzip' },
|
||||
});
|
||||
expect(result.kind).toBe('decompress_error');
|
||||
expect(result.statusCode).toBe(502);
|
||||
expect(JSON.parse(result.body.toString()).code).toBe('gitops_proxy_decompress_failed');
|
||||
});
|
||||
|
||||
it('reports a truncated upstream as a failure, not an empty success', async () => {
|
||||
const result = await runResponse({ status: 200, body: '[{"nodeId":1}]', abort: true });
|
||||
expect(result.kind).toBe('upstream_failed');
|
||||
expect(result.statusCode).toBe(502);
|
||||
expect(JSON.parse(result.body.toString()).code).toBe('gitops_proxy_upstream_failed');
|
||||
});
|
||||
|
||||
it('finalizes once and writes nothing when the client hangs up', async () => {
|
||||
const result = await runResponse({ status: 200, body: JSON.stringify([{ nodeId: 1 }]), closeEarly: true });
|
||||
expect(result.kind).toBe('downstream_close');
|
||||
expect(result.ended).toBe(false);
|
||||
expect(result.finalizeCalls).toBe(1);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
type ResponseCase = {
|
||||
status: number;
|
||||
body: string | Buffer;
|
||||
headers?: Record<string, string>;
|
||||
chunks?: Buffer[];
|
||||
abort?: boolean;
|
||||
endIncomplete?: boolean;
|
||||
closeEarly?: boolean;
|
||||
transform?: (payload: unknown) => unknown;
|
||||
};
|
||||
|
||||
type ResponseResult = {
|
||||
kind: IdentityTerminalKind;
|
||||
statusCode: number;
|
||||
headers: Record<string, string>;
|
||||
body: Buffer;
|
||||
ended: boolean;
|
||||
finalizeCalls: number;
|
||||
};
|
||||
|
||||
/** Drive one upstream response through the terminal rules and capture what the client sees. */
|
||||
function runResponse(testCase: ResponseCase): Promise<ResponseResult> {
|
||||
return new Promise((resolve) => {
|
||||
const proxyRes = new IncomingMessage(new Socket());
|
||||
proxyRes.statusCode = testCase.status;
|
||||
for (const [name, value] of Object.entries(testCase.headers ?? {})) {
|
||||
proxyRes.headers[name] = value;
|
||||
}
|
||||
|
||||
// Pre-seeded with the framing the streaming hop would have set. Without
|
||||
// this the strip assertions pass vacuously, since removing a header that
|
||||
// was never present is indistinguishable from not stripping at all.
|
||||
const headers: Record<string, string> = {
|
||||
'content-length': '999',
|
||||
'content-encoding': 'gzip',
|
||||
'transfer-encoding': 'chunked',
|
||||
};
|
||||
let ended = false;
|
||||
let written = Buffer.alloc(0);
|
||||
let kind: IdentityTerminalKind | undefined;
|
||||
let finalizeCalls = 0;
|
||||
const closeListeners: Array<() => void> = [];
|
||||
|
||||
const sink: IdentityResponseSink = {
|
||||
headersSent: false,
|
||||
writableEnded: false,
|
||||
statusCode: 0,
|
||||
removeHeader: (name) => { delete headers[name]; },
|
||||
setHeader: (name, value) => { headers[name] = String(value); },
|
||||
end: (body) => {
|
||||
ended = true;
|
||||
sink.writableEnded = true;
|
||||
if (body) written = Buffer.from(body);
|
||||
finish();
|
||||
},
|
||||
on: (_event, listener) => { closeListeners.push(listener); },
|
||||
};
|
||||
|
||||
const finish = (): void => {
|
||||
resolve({
|
||||
kind: kind ?? 'passthrough',
|
||||
statusCode: sink.statusCode,
|
||||
headers,
|
||||
body: written,
|
||||
ended,
|
||||
finalizeCalls,
|
||||
});
|
||||
};
|
||||
|
||||
handleIdentityResponse(proxyRes, sink, {
|
||||
transform: testCase.transform
|
||||
?? ((payload) => { rewriteIdentityPayload(payload, 42); return payload; }),
|
||||
finalizeTiming: (terminal) => {
|
||||
kind = terminal;
|
||||
finalizeCalls += 1;
|
||||
if (terminal === 'downstream_close') setImmediate(finish);
|
||||
},
|
||||
});
|
||||
|
||||
if (testCase.closeEarly) {
|
||||
for (const listener of closeListeners) listener();
|
||||
return;
|
||||
}
|
||||
|
||||
const payload = Buffer.isBuffer(testCase.body) ? testCase.body : Buffer.from(testCase.body);
|
||||
if (testCase.abort) {
|
||||
proxyRes.push(payload.subarray(0, Math.max(1, payload.length - 4)));
|
||||
proxyRes.emit('aborted');
|
||||
return;
|
||||
}
|
||||
if (testCase.endIncomplete) {
|
||||
// `complete` deliberately left false, which is how Node itself reports a
|
||||
// body that stopped early.
|
||||
proxyRes.push(payload.subarray(0, Math.max(1, payload.length - 4)));
|
||||
proxyRes.push(null);
|
||||
return;
|
||||
}
|
||||
for (const chunk of testCase.chunks ?? []) proxyRes.push(chunk);
|
||||
if (payload.length > 0) proxyRes.push(payload);
|
||||
// A real HTTP response sets this once the parser has seen the whole body.
|
||||
// Without it a synthetic message reports every clean end as truncated.
|
||||
proxyRes.complete = true;
|
||||
proxyRes.push(null);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
decodeArtifactEvidenceJson,
|
||||
decodeGitOpsApprovedTargetEffectJson,
|
||||
decodeGitOpsRequiredTargetsJson,
|
||||
decodeObservedArtifactIdentity,
|
||||
encodeGitOpsJson,
|
||||
encodeGitOpsRequiredTargetsJson,
|
||||
GitOpsJsonError,
|
||||
} from '../services/gitops/json';
|
||||
|
||||
describe('gitops json codecs', () => {
|
||||
it('rejects extra keys, non-integer ids, and non-canonical required targets', () => {
|
||||
expect(() => decodeGitOpsRequiredTargetsJson('{"nodeIds":[1],"extra":true}')).toThrow(GitOpsJsonError);
|
||||
expect(() => decodeGitOpsRequiredTargetsJson('{"nodeIds":["1"]}')).toThrow(GitOpsJsonError);
|
||||
expect(() => decodeGitOpsRequiredTargetsJson('{"nodeIds":[2,1]}')).toThrow(GitOpsJsonError);
|
||||
expect(() => decodeGitOpsRequiredTargetsJson('{"nodeIds":[1,1]}')).toThrow(GitOpsJsonError);
|
||||
expect(decodeGitOpsRequiredTargetsJson('{"nodeIds":[1,2]}')).toEqual({ nodeIds: [1, 2] });
|
||||
expect(() => encodeGitOpsRequiredTargetsJson([2, 1])).toThrow(GitOpsJsonError);
|
||||
});
|
||||
|
||||
it('decodes placement blast as a canonical action effect', () => {
|
||||
expect(decodeGitOpsApprovedTargetEffectJson('[]')).toEqual([]);
|
||||
expect(decodeGitOpsApprovedTargetEffectJson(
|
||||
'[{"nodeId":1,"outcome":"place"},{"nodeId":3,"outcome":"remove"}]',
|
||||
)).toEqual([
|
||||
{ nodeId: 1, outcome: 'place' },
|
||||
{ nodeId: 3, outcome: 'remove' },
|
||||
]);
|
||||
expect(() => decodeGitOpsApprovedTargetEffectJson(
|
||||
'[{"nodeId":2,"outcome":"place"},{"nodeId":1,"outcome":"remove"}]',
|
||||
)).toThrow(GitOpsJsonError);
|
||||
expect(() => decodeGitOpsApprovedTargetEffectJson(
|
||||
'[{"nodeId":1,"outcome":"place","extra":1}]',
|
||||
)).toThrow(GitOpsJsonError);
|
||||
});
|
||||
|
||||
it('rejects contradictory artifact evidence', () => {
|
||||
expect(decodeArtifactEvidenceJson('{"kind":"unresolved"}')).toEqual({ kind: 'unresolved' });
|
||||
expect(() => decodeArtifactEvidenceJson('{"kind":"unresolved","identity":"x"}')).toThrow(GitOpsJsonError);
|
||||
expect(() => decodeArtifactEvidenceJson('{"kind":"exact"}')).toThrow(GitOpsJsonError);
|
||||
expect(decodeArtifactEvidenceJson('{"kind":"exact","identity":"sha256:abc"}')).toEqual({
|
||||
kind: 'exact',
|
||||
identity: 'sha256:abc',
|
||||
});
|
||||
});
|
||||
|
||||
it('refuses to encode a value JSON.stringify drops', () => {
|
||||
// JSON.stringify returns undefined rather than throwing for these, and
|
||||
// every JSON column is NOT NULL, so the encoder has to reject them itself.
|
||||
expect(() => encodeGitOpsJson(undefined)).toThrow(GitOpsJsonError);
|
||||
expect(() => encodeGitOpsJson(() => 'x')).toThrow(GitOpsJsonError);
|
||||
expect(() => encodeGitOpsJson(Symbol('x'))).toThrow(GitOpsJsonError);
|
||||
expect(encodeGitOpsJson({ a: 1 })).toBe('{"a":1}');
|
||||
});
|
||||
|
||||
it('treats null observation as unknown and rejects contradictory kinds', () => {
|
||||
expect(decodeObservedArtifactIdentity(null)).toEqual({ kind: 'unknown' });
|
||||
expect(() => decodeObservedArtifactIdentity('{"kind":"missing","identity":"x"}')).toThrow(GitOpsJsonError);
|
||||
expect(() => decodeObservedArtifactIdentity('{"kind":"exact","identity":"x"}')).toThrow(GitOpsJsonError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,182 @@
|
||||
/**
|
||||
* What the boot sweep does with a managed area no database row claims.
|
||||
*
|
||||
* The three staging-marker states drive three different actions, and the
|
||||
* difference between "missing" and "corrupt" is the whole rule: nothing ever
|
||||
* claimed a missing-marker area, so it is an ordinary orphan and is reaped,
|
||||
* while a corrupt marker is evidence of a claim we cannot read, so the area is
|
||||
* preserved. See docs/internal/adrs/2026-08-16-managed-area-orphan-reaping.md.
|
||||
*/
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { GitSourceService } from '../services/GitSourceService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import { candidateRelPathForSha, stagingMarkerPath } from '../services/gitops/createStagingMarker';
|
||||
import { stackManagedRoot } from '../services/gitops/directApplication';
|
||||
import type { GitOpsApplicationRow, GitOpsCreateCheckpointRow } from '../services/gitops/types';
|
||||
|
||||
const SHA = 'beef5678';
|
||||
|
||||
describe('managed-area orphan sweep', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
db.prepare('DELETE FROM gitops_create_checkpoints').run();
|
||||
db.prepare('DELETE FROM gitops_target_current').run();
|
||||
db.prepare('DELETE FROM gitops_applications').run();
|
||||
db.prepare('DELETE FROM stack_git_sources').run();
|
||||
});
|
||||
|
||||
function seedArea(stackName: string): { area: string; candidate: string; sentinel: string } {
|
||||
const area = stackManagedRoot(stackName);
|
||||
const candidate = path.join(area, candidateRelPathForSha(SHA));
|
||||
const sentinel = path.join(area, 'generations', 'applied-earlier');
|
||||
fs.mkdirSync(candidate, { recursive: true });
|
||||
fs.mkdirSync(sentinel, { recursive: true });
|
||||
return { area, candidate, sentinel };
|
||||
}
|
||||
|
||||
it('reaps an area nothing has ever claimed', async () => {
|
||||
const { area } = seedArea('orphan-none');
|
||||
|
||||
await GitSourceService.getInstance().sweepOrphans();
|
||||
|
||||
expect(fs.existsSync(area)).toBe(false);
|
||||
});
|
||||
|
||||
it('preserves an area whose marker cannot be read', async () => {
|
||||
const { area, sentinel } = seedArea('orphan-corrupt');
|
||||
fs.writeFileSync(stagingMarkerPath(area), '{ not json', 'utf8');
|
||||
|
||||
await GitSourceService.getInstance().sweepOrphans();
|
||||
|
||||
expect(fs.existsSync(sentinel)).toBe(true);
|
||||
});
|
||||
|
||||
it('removes only the staged candidate when a valid marker claims the area', async () => {
|
||||
const { area, candidate, sentinel } = seedArea('orphan-marked');
|
||||
fs.writeFileSync(stagingMarkerPath(area), JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
operationId: 'op-live',
|
||||
rootPreexisted: true,
|
||||
candidateRelPath: candidateRelPathForSha(SHA),
|
||||
createdAt: Date.now(),
|
||||
}), 'utf8');
|
||||
|
||||
await GitSourceService.getInstance().sweepOrphans();
|
||||
|
||||
expect(fs.existsSync(candidate)).toBe(false);
|
||||
expect(fs.existsSync(sentinel)).toBe(true);
|
||||
});
|
||||
|
||||
it('leaves an area claimed by an in-flight create alone', async () => {
|
||||
const { area, candidate } = seedArea('orphan-inflight');
|
||||
GitOpsStore.getInstance().insertApplication(creatingApp('app-inflight', 'orphan-inflight'));
|
||||
GitOpsStore.getInstance().insertCreateCheckpoint(checkpoint('app-inflight', 'orphan-inflight'));
|
||||
|
||||
await GitSourceService.getInstance().sweepOrphans();
|
||||
|
||||
expect(fs.existsSync(area)).toBe(true);
|
||||
expect(fs.existsSync(candidate)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
function checkpoint(applicationId: string, stackName: string): GitOpsCreateCheckpointRow {
|
||||
return {
|
||||
application_id: applicationId,
|
||||
stack_name: stackName,
|
||||
phase: 'pre_stack',
|
||||
generation_id: null,
|
||||
operation_id: `op-${applicationId}`,
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
branch: 'main',
|
||||
compose_path: 'compose.yml',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
auth_type: 'none',
|
||||
encrypted_token: null,
|
||||
auto_apply_on_webhook: 0,
|
||||
auto_deploy_on_apply: 0,
|
||||
commit_sha: SHA,
|
||||
applied_spec_json: null,
|
||||
created_managed_root: 1,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function creatingApp(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'creating',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
/**
|
||||
* Integration tests for GET /api/gitops-metrics: the Admin-only snapshot of
|
||||
* in-process GitOps transition counters.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll, beforeEach, vi } from 'vitest';
|
||||
import request from 'supertest';
|
||||
import bcrypt from 'bcrypt';
|
||||
import { setupTestDb, cleanupTestDb, loginAsTestAdmin } from './helpers/setupTestDb';
|
||||
|
||||
let tmpDir: string;
|
||||
let app: import('express').Express;
|
||||
let adminCookie: string;
|
||||
let viewerCookie: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
const { DatabaseService } = await import('../services/DatabaseService');
|
||||
({ app } = await import('../index'));
|
||||
adminCookie = await loginAsTestAdmin(app);
|
||||
|
||||
const viewerHash = await bcrypt.hash('viewerpass', 1);
|
||||
DatabaseService.getInstance().addUser({
|
||||
username: 'gitops-metrics-viewer',
|
||||
password_hash: viewerHash,
|
||||
role: 'viewer',
|
||||
});
|
||||
const viewerRes = await request(app)
|
||||
.post('/api/auth/login')
|
||||
.send({ username: 'gitops-metrics-viewer', password: 'viewerpass' });
|
||||
const cookies = viewerRes.headers['set-cookie'] as string | string[];
|
||||
viewerCookie = Array.isArray(cookies) ? cookies[0] : cookies;
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
vi.restoreAllMocks();
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
const { GitOpsMetricsService } = await import('../services/GitOpsMetricsService');
|
||||
GitOpsMetricsService.resetForTests();
|
||||
});
|
||||
|
||||
describe('GET /api/gitops-metrics', () => {
|
||||
it('returns 401 without an auth cookie', async () => {
|
||||
const res = await request(app).get('/api/gitops-metrics');
|
||||
expect(res.status).toBe(401);
|
||||
});
|
||||
|
||||
it('refuses a signed-in non-admin', async () => {
|
||||
const res = await request(app).get('/api/gitops-metrics').set('Cookie', viewerCookie);
|
||||
expect(res.status).toBe(403);
|
||||
});
|
||||
|
||||
it('returns an empty list on a fresh process', async () => {
|
||||
const res = await request(app).get('/api/gitops-metrics').set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body).toEqual({ entries: [] });
|
||||
});
|
||||
|
||||
it('returns one entry per stage and outcome pair', async () => {
|
||||
const { GitOpsMetricsService } = await import('../services/GitOpsMetricsService');
|
||||
const metrics = GitOpsMetricsService.getInstance();
|
||||
metrics.record('fetched', 'committed');
|
||||
metrics.record('fetched', 'committed');
|
||||
metrics.record('apply_failed', 'failed');
|
||||
|
||||
const res = await request(app).get('/api/gitops-metrics').set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.entries).toEqual([
|
||||
{ stage: 'apply_failed', outcome: 'failed', count: 1 },
|
||||
{ stage: 'fetched', outcome: 'committed', count: 2 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('names no stack, node, repository or actor', async () => {
|
||||
// The counters are process diagnostics, not an audit trail. Anything
|
||||
// identifying would be one with no retention policy and no per-row
|
||||
// authorization, which is what the history API exists to provide.
|
||||
const { GitOpsMetricsService } = await import('../services/GitOpsMetricsService');
|
||||
GitOpsMetricsService.getInstance().record('deploy_started', 'committed');
|
||||
|
||||
const res = await request(app).get('/api/gitops-metrics').set('Cookie', adminCookie);
|
||||
expect(Object.keys(res.body.entries[0]).sort()).toEqual(['count', 'outcome', 'stage']);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,118 @@
|
||||
/**
|
||||
* Inline Blueprint migration.
|
||||
*
|
||||
* Migration records what a Blueprint asks for. It never records agreement: a
|
||||
* Blueprint revision and a deployment's applied revision both look like
|
||||
* progress, but neither proves a node is running the intent this pass just
|
||||
* minted, and writing them as an acknowledgement would report convergence
|
||||
* nobody verified.
|
||||
*/
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService, type Blueprint } from '../services/DatabaseService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import { migrateInlineBlueprints } from '../services/gitops/migrate';
|
||||
import { commitBlueprintCreate } from '../services/gitops/blueprintProducers';
|
||||
import { decodeGitOpsEvidenceLimitations } from '../services/gitops/json';
|
||||
|
||||
describe('gitops inline blueprint migration', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('brings a pre-existing Blueprint in without claiming anyone agreed to it', () => {
|
||||
const db = DatabaseService.getInstance();
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = seedLegacy('mig-plain');
|
||||
|
||||
expect(outcomeFor(blueprint)).toBe('migrated_inline');
|
||||
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
expect(app.target_mode).toBe('inline_blueprint');
|
||||
const intent = store.getIntentRevision(app.intent_revision_id!)!;
|
||||
// Carried for display, never as an acknowledgement.
|
||||
expect(intent.blueprint_revision).toBe(blueprint.revision);
|
||||
|
||||
const candidate = store.getRolloutCandidate(app.rollout_candidate_id!)!;
|
||||
expect(candidate.provenance).toBe('legacy_inline');
|
||||
// Placement is not resolved by migration.
|
||||
expect(JSON.parse(candidate.required_targets_json)).toEqual({ nodeIds: [] });
|
||||
|
||||
// No target, so nothing claims a node is running this.
|
||||
expect(store.listTargets(app.id)).toEqual([]);
|
||||
expect(db.getBlueprint(blueprint.id)!.revision).toBe(blueprint.revision);
|
||||
});
|
||||
|
||||
it('says why an unapproved Blueprint carries no authority', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = seedLegacy('mig-unapproved');
|
||||
migrateInlineBlueprints();
|
||||
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
const limitations = decodeGitOpsEvidenceLimitations(app.evidence_limitations_json);
|
||||
// Recorded rather than left blank: an absent approval and an approval that
|
||||
// no longer authorizes this intent are otherwise indistinguishable.
|
||||
expect(limitations.map(l => l.code)).toContain('blueprint_reapproval_required');
|
||||
});
|
||||
|
||||
it('is a no-op on replay', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = seedLegacy('mig-replay');
|
||||
migrateInlineBlueprints();
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
expect(outcomeFor(blueprint)).toBe('skipped_current');
|
||||
const after = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
expect(after.intent_revision_id).toBe(app.intent_revision_id);
|
||||
expect(after.rollout_candidate_id).toBe(app.rollout_candidate_id);
|
||||
});
|
||||
|
||||
it('leaves a Blueprint the new path already described alone', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = commitBlueprintCreate({
|
||||
name: 'mig-live',
|
||||
description: null,
|
||||
compose_content: 'services:\n web:\n image: nginx:1.27\n',
|
||||
selector: { type: 'nodes', ids: [1] },
|
||||
drift_mode: 'suggest',
|
||||
classification: 'stateless',
|
||||
classification_reasons: [],
|
||||
enabled: true,
|
||||
created_by: 'tester',
|
||||
}, () => [1]);
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
expect(outcomeFor(blueprint)).toBe('skipped_live_application');
|
||||
// Its rows were written with proof this pass does not have.
|
||||
expect(store.getLiveBlueprintApplication(blueprint.id)!.intent_revision_id)
|
||||
.toBe(app.intent_revision_id);
|
||||
});
|
||||
});
|
||||
|
||||
function outcomeFor(blueprint: Blueprint): string {
|
||||
return migrateInlineBlueprints().find(r => r.stackName === blueprint.name)!.outcome;
|
||||
}
|
||||
|
||||
/** A Blueprint as an install carries it across an upgrade: no GitOps rows. */
|
||||
function seedLegacy(name: string): Blueprint {
|
||||
return DatabaseService.getInstance().createBlueprint({
|
||||
name,
|
||||
description: null,
|
||||
compose_content: `services:\n web:\n image: nginx:1.27\n# ${name}\n`,
|
||||
selector: { type: 'nodes', ids: [1] },
|
||||
drift_mode: 'suggest',
|
||||
classification: 'stateless',
|
||||
classification_reasons: [],
|
||||
enabled: true,
|
||||
created_by: null,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,382 @@
|
||||
/**
|
||||
* Migration of Git stacks that predate the revision state model.
|
||||
*
|
||||
* The rule under test is that a canonical pointer is written only when the
|
||||
* evidence proves that exact generation under the repository and ref configured
|
||||
* now. A legacy applied commit is not that proof by itself, so the interesting
|
||||
* cases are the ones where it is *not* promoted: a missing manifest, an
|
||||
* unreadable one, one stamped for a repository the stack no longer points at,
|
||||
* and one naming a commit the source row disagrees with. In each the commit
|
||||
* survives as recorded evidence and the projection asks for a fetch instead of
|
||||
* asserting something nobody verified.
|
||||
*/
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService, type StackGitSource } from '../services/DatabaseService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import { migrateDirectGitStacks, primeMigrationManifests } from '../services/gitops/migrate';
|
||||
import { directSourceIdentity, migrationDirectSourceIdentity } from '../services/gitops/directApplication';
|
||||
import { projectApplication } from '../services/gitops/derive';
|
||||
|
||||
const REPO = 'https://github.com/example/legacy.git';
|
||||
const SHA = 'legacy01';
|
||||
|
||||
type ManifestFixture =
|
||||
| { manifestVersion: number; generation: { appliedDir: string }; resolvedRevision: { commitSha: string } }
|
||||
| { corrupt: string }
|
||||
| null;
|
||||
|
||||
describe('gitops migration of pre-existing Git stacks', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
db.prepare('DELETE FROM gitops_migration_checkpoints').run();
|
||||
db.prepare('DELETE FROM gitops_target_current').run();
|
||||
db.prepare('DELETE FROM gitops_generations').run();
|
||||
db.prepare('DELETE FROM gitops_applications').run();
|
||||
db.prepare('DELETE FROM stack_git_sources').run();
|
||||
});
|
||||
|
||||
it('leaves a config-only stack asking for a fetch', () => {
|
||||
seedStack('cfg-only', { lastApplied: null });
|
||||
primeManifests({ 'cfg-only': null });
|
||||
|
||||
expect(migrateDirectGitStacks()).toEqual([{ stackName: 'cfg-only', outcome: 'migrated_unreconciled' }]);
|
||||
|
||||
const app = GitOpsStore.getInstance().getLiveDirectApplication('cfg-only')!;
|
||||
expect(app.desired_commit_sha).toBeNull();
|
||||
expect(app.accepted_generation_id).toBeNull();
|
||||
expect(app.materialization_fingerprint).not.toBeNull();
|
||||
const projection = projectOf(app.id);
|
||||
expect(projection.facets.source.status).toBe('never_reconciled');
|
||||
expect(projection.availableActions).toContain('fetch');
|
||||
expect(projection.limitations).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('accepts the applied commit only when a trusted manifest proves it', () => {
|
||||
seedStack('trusted', { lastApplied: SHA });
|
||||
primeManifests({
|
||||
trusted: {
|
||||
manifestVersion: 3,
|
||||
generation: { appliedDir: `generations/applied-${SHA}-3` },
|
||||
resolvedRevision: { commitSha: SHA },
|
||||
},
|
||||
});
|
||||
|
||||
expect(migrateDirectGitStacks()).toEqual([{ stackName: 'trusted', outcome: 'migrated_accepted' }]);
|
||||
|
||||
const store = GitOpsStore.getInstance();
|
||||
const app = store.getLiveDirectApplication('trusted')!;
|
||||
expect(app.desired_commit_sha).toBe(SHA);
|
||||
expect(app.fetched_commit_sha).toBe(SHA);
|
||||
expect(app.accepted_generation_id).not.toBeNull();
|
||||
|
||||
const generation = store.getGeneration(app.accepted_generation_id!)!;
|
||||
expect(generation.commit_sha).toBe(SHA);
|
||||
// Equal fingerprints, or the accepted generation would immediately read as
|
||||
// stale against the configuration that produced it.
|
||||
expect(generation.materialization_fingerprint).toBe(app.materialization_fingerprint);
|
||||
|
||||
const target = store.getTarget(app.id, 1)!;
|
||||
expect(target.desired_generation_id).toBe(app.accepted_generation_id);
|
||||
expect(target.applied_generation_id).toBe(app.accepted_generation_id);
|
||||
// A manifest proves what was materialized, never what is running.
|
||||
expect(target.deployed_generation_id).toBeNull();
|
||||
expect(target.healthy_generation_id).toBeNull();
|
||||
expect(target.lkg_generation_id).toBeNull();
|
||||
// Nobody approved this generation through the model.
|
||||
expect(app.source_acceptance_ref).toBeNull();
|
||||
|
||||
expect(projectOf(app.id).facets.source.status).toBe('application_generation_accepted');
|
||||
});
|
||||
|
||||
it('keeps an unprovable applied commit as evidence, never as a pointer', () => {
|
||||
const cases: Array<[string, ManifestFixture, string]> = [
|
||||
['manifest-gone', null, 'manifest_absent'],
|
||||
['manifest-broken', { corrupt: 'invalid manifest shape' }, 'manifest_corrupt'],
|
||||
['manifest-foreign', { corrupt: 'identity repository mismatch' }, 'manifest_identity_invalid'],
|
||||
];
|
||||
for (const [stackName, manifest, expectedCode] of cases) {
|
||||
seedStack(stackName, { lastApplied: SHA });
|
||||
primeManifests({ [stackName]: manifest });
|
||||
|
||||
migrateDirectGitStacks();
|
||||
|
||||
const app = GitOpsStore.getInstance().getLiveDirectApplication(stackName)!;
|
||||
expect(app.desired_commit_sha, stackName).toBeNull();
|
||||
expect(app.fetched_commit_sha, stackName).toBeNull();
|
||||
expect(app.accepted_generation_id, stackName).toBeNull();
|
||||
|
||||
const projection = projectOf(app.id);
|
||||
expect(projection.facets.source.status, stackName).toBe('never_reconciled');
|
||||
expect(projection.limitations.map((l) => l.code), stackName).toContain(expectedCode);
|
||||
// The commit is retained as the evidence behind the limitation, so an
|
||||
// operator can see what the stack used to be at.
|
||||
expect(projection.limitations.map((l) => l.evidence), stackName).toContain(SHA);
|
||||
}
|
||||
});
|
||||
|
||||
it('refuses a valid manifest that names a different commit than the source row', () => {
|
||||
// The manifest validates and belongs to this stack, repository and ref, so
|
||||
// every other check passes. Only the commits disagree, and that alone must
|
||||
// keep the canonical pointers null: the applied directory here materializes
|
||||
// MANIFEST_SHA, so accepting SHA would certify a commit whose files are not
|
||||
// the ones on disk.
|
||||
const manifestSha = 'manifest02';
|
||||
seedStack('commit-drift', { lastApplied: SHA });
|
||||
primeManifests({
|
||||
'commit-drift': {
|
||||
manifestVersion: 4,
|
||||
generation: { appliedDir: `generations/applied-${manifestSha}-4` },
|
||||
resolvedRevision: { commitSha: manifestSha },
|
||||
},
|
||||
});
|
||||
|
||||
expect(migrateDirectGitStacks()).toEqual([{ stackName: 'commit-drift', outcome: 'migrated_unreconciled' }]);
|
||||
|
||||
const app = GitOpsStore.getInstance().getLiveDirectApplication('commit-drift')!;
|
||||
expect(app.desired_commit_sha).toBeNull();
|
||||
expect(app.fetched_commit_sha).toBeNull();
|
||||
expect(app.accepted_generation_id).toBeNull();
|
||||
|
||||
const target = GitOpsStore.getInstance().getTarget(app.id, 1)!;
|
||||
expect(target.applied_generation_id).toBeNull();
|
||||
|
||||
const projection = projectOf(app.id);
|
||||
expect(projection.facets.source.status).toBe('never_reconciled');
|
||||
expect(projection.availableActions).toContain('fetch');
|
||||
// Both commits are named, so an operator can see which two records disagree
|
||||
// rather than only learning that something could not be proven.
|
||||
const mismatch = projection.limitations.find((l) => l.code === 'manifest_commit_mismatch');
|
||||
expect(mismatch).toBeDefined();
|
||||
expect(mismatch!.evidence).toContain(SHA);
|
||||
expect(mismatch!.evidence).toContain(manifestSha);
|
||||
});
|
||||
|
||||
it('separates a manifest with no commit from one that names a conflicting commit', () => {
|
||||
// A manifest adopted from an existing directory is written with an empty
|
||||
// commit and state 'migrated', which the validator permits. Folding that
|
||||
// into the mismatch case would tell an operator the manifest names a
|
||||
// different commit while naming nothing at all.
|
||||
seedStack('adopted', { lastApplied: SHA });
|
||||
primeManifests({
|
||||
adopted: {
|
||||
manifestVersion: 1,
|
||||
generation: { appliedDir: 'generations/applied-adopted-1' },
|
||||
resolvedRevision: { commitSha: '' },
|
||||
},
|
||||
});
|
||||
|
||||
expect(migrateDirectGitStacks()).toEqual([{ stackName: 'adopted', outcome: 'migrated_unreconciled' }]);
|
||||
|
||||
const app = GitOpsStore.getInstance().getLiveDirectApplication('adopted')!;
|
||||
expect(app.desired_commit_sha).toBeNull();
|
||||
expect(app.accepted_generation_id).toBeNull();
|
||||
|
||||
const codes = projectOf(app.id).limitations.map((l) => l.code);
|
||||
expect(codes).toContain('manifest_commit_unresolved');
|
||||
expect(codes).not.toContain('manifest_commit_mismatch');
|
||||
});
|
||||
|
||||
it('does not let a pending pull stand in for proof', () => {
|
||||
seedStack('pending-only', { lastApplied: null, pending: 'pending99' });
|
||||
primeManifests({ 'pending-only': null });
|
||||
|
||||
migrateDirectGitStacks();
|
||||
|
||||
const app = GitOpsStore.getInstance().getLiveDirectApplication('pending-only')!;
|
||||
expect(app.desired_commit_sha).toBeNull();
|
||||
expect(app.candidate_generation_id).toBeNull();
|
||||
const projection = projectOf(app.id);
|
||||
expect(projection.facets.source.status).toBe('never_reconciled');
|
||||
expect(projection.limitations.map((l) => l.code)).toContain('legacy_pending');
|
||||
});
|
||||
|
||||
it('is a no-op on replay and re-runs only when the configuration changes', () => {
|
||||
seedStack('replay', { lastApplied: SHA });
|
||||
primeManifests({
|
||||
replay: {
|
||||
manifestVersion: 1,
|
||||
generation: { appliedDir: `generations/applied-${SHA}-1` },
|
||||
resolvedRevision: { commitSha: SHA },
|
||||
},
|
||||
});
|
||||
|
||||
expect(migrateDirectGitStacks()[0].outcome).toBe('migrated_accepted');
|
||||
const firstId = GitOpsStore.getInstance().getLiveDirectApplication('replay')!.id;
|
||||
|
||||
expect(migrateDirectGitStacks()).toEqual([{ stackName: 'replay', outcome: 'skipped_current' }]);
|
||||
expect(GitOpsStore.getInstance().getLiveDirectApplication('replay')!.id).toBe(firstId);
|
||||
|
||||
// A material configuration change replays the matrix, and the existing
|
||||
// application is left alone rather than being rebuilt over.
|
||||
seedStack('replay', { lastApplied: SHA, composePaths: ['compose.yaml', 'compose.prod.yaml'] });
|
||||
expect(migrateDirectGitStacks()).toEqual([{ stackName: 'replay', outcome: 'skipped_live_application' }]);
|
||||
expect(GitOpsStore.getInstance().getLiveDirectApplication('replay')!.id).toBe(firstId);
|
||||
});
|
||||
|
||||
it('never touches a stack the new path already described', () => {
|
||||
seedStack('already-modelled', { lastApplied: SHA });
|
||||
primeManifests({ 'already-modelled': null });
|
||||
migrateDirectGitStacks();
|
||||
const before = GitOpsStore.getInstance().getLiveDirectApplication('already-modelled')!;
|
||||
|
||||
DatabaseService.getInstance().getDb().prepare('DELETE FROM gitops_migration_checkpoints').run();
|
||||
migrateDirectGitStacks();
|
||||
|
||||
expect(GitOpsStore.getInstance().getLiveDirectApplication('already-modelled')!.id).toBe(before.id);
|
||||
});
|
||||
|
||||
it('retires a stack whose directory is gone instead of claiming its name', () => {
|
||||
seedStack('vanished', { lastApplied: SHA, createDir: false });
|
||||
primeManifests({ vanished: null });
|
||||
|
||||
expect(migrateDirectGitStacks()).toEqual([
|
||||
{ stackName: 'vanished', outcome: 'tombstoned_missing_stack' },
|
||||
]);
|
||||
expect(GitOpsStore.getInstance().getLiveDirectApplication('vanished')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('accepts the applied commit through a trusted manifest even on a legacy URL', () => {
|
||||
// The worst real-world instance of the strict-parser bug: a stack whose
|
||||
// manifest proves its applied commit would have failed migration every
|
||||
// boot and never entered the model at all.
|
||||
const legacyUrl = `${REPO}?token=legacy-secret`;
|
||||
seedStack('legacy-trusted-url', { lastApplied: SHA, repoUrl: legacyUrl });
|
||||
primeManifests({
|
||||
'legacy-trusted-url': {
|
||||
manifestVersion: 3,
|
||||
generation: { appliedDir: `generations/applied-${SHA}-3` },
|
||||
resolvedRevision: { commitSha: SHA },
|
||||
},
|
||||
});
|
||||
|
||||
expect(migrateDirectGitStacks()).toEqual([{ stackName: 'legacy-trusted-url', outcome: 'migrated_accepted' }]);
|
||||
|
||||
const store = GitOpsStore.getInstance();
|
||||
const app = store.getLiveDirectApplication('legacy-trusted-url')!;
|
||||
expect(app.desired_commit_sha).toBe(SHA);
|
||||
expect(app.accepted_generation_id).not.toBeNull();
|
||||
expect(app.configured_repo_url).toBe(REPO);
|
||||
expect(DatabaseService.getInstance().getGitSource('legacy-trusted-url')?.repo_url).toBe(legacyUrl);
|
||||
expect(projectOf(app.id).facets.source.status).toBe('application_generation_accepted');
|
||||
});
|
||||
|
||||
it('derives the same identity as strict ingress once the legacy decoration is stripped', () => {
|
||||
const config = {
|
||||
repoUrl: REPO,
|
||||
branch: 'main',
|
||||
composePaths: ['compose.yaml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
};
|
||||
const noisy = { ...config, repoUrl: `${REPO}?token=x` };
|
||||
const lenient = migrationDirectSourceIdentity(noisy);
|
||||
const strict = directSourceIdentity(config);
|
||||
expect(lenient.repoUrl).toBe(strict.repoUrl);
|
||||
expect(lenient.identity).toEqual(strict.identity);
|
||||
// A migrated stack must be replay-recognizable against one linked fresh
|
||||
// through the user path for the same repository.
|
||||
expect(lenient.fingerprint).toBe(strict.fingerprint);
|
||||
});
|
||||
|
||||
it('migrates a legacy operational URL that still carries a query string', () => {
|
||||
const legacyUrl = `${REPO}?token=legacy-secret`;
|
||||
seedStack('legacy-query-url', { lastApplied: null, repoUrl: legacyUrl });
|
||||
primeManifests({ 'legacy-query-url': null });
|
||||
|
||||
expect(migrateDirectGitStacks()).toEqual([{ stackName: 'legacy-query-url', outcome: 'migrated_unreconciled' }]);
|
||||
|
||||
const app = GitOpsStore.getInstance().getLiveDirectApplication('legacy-query-url')!;
|
||||
expect(app.configured_repo_url).toBe(REPO);
|
||||
// The operational row keeps its query: fetch may still need it.
|
||||
expect(DatabaseService.getInstance().getGitSource('legacy-query-url')?.repo_url).toBe(legacyUrl);
|
||||
});
|
||||
|
||||
it('migrates a legacy URL carrying userinfo to the identity of its clean form', () => {
|
||||
seedStack('legacy-userinfo-url', { lastApplied: null, repoUrl: 'https://deploy:pat@github.com/example/legacy.git' });
|
||||
seedStack('clean-url', { lastApplied: null });
|
||||
primeManifests({ 'legacy-userinfo-url': null, 'clean-url': null });
|
||||
|
||||
migrateDirectGitStacks();
|
||||
|
||||
const store = GitOpsStore.getInstance();
|
||||
const legacy = store.getLiveDirectApplication('legacy-userinfo-url')!;
|
||||
const clean = store.getLiveDirectApplication('clean-url')!;
|
||||
expect(legacy.configured_repo_url).toBe(REPO);
|
||||
expect(legacy.repo_identity_json).toBe(clean.repo_identity_json);
|
||||
// The same repository under the same configuration must produce the same
|
||||
// fingerprint, or a later replay could not recognize the stack it
|
||||
// already migrated.
|
||||
expect(legacy.materialization_fingerprint).toBe(clean.materialization_fingerprint);
|
||||
expect(DatabaseService.getInstance().getGitSource('legacy-userinfo-url')?.repo_url).toContain('deploy:pat@');
|
||||
});
|
||||
});
|
||||
|
||||
function projectOf(applicationId: string) {
|
||||
const projection = projectApplication(applicationId, true);
|
||||
if (projection.targetMode === 'not_applicable') throw new Error('expected an application');
|
||||
return projection;
|
||||
}
|
||||
|
||||
function primeManifests(fixtures: Record<string, ManifestFixture>): void {
|
||||
primeMigrationManifests((stackName) => fixtures[stackName] ?? null);
|
||||
}
|
||||
|
||||
function seedStack(
|
||||
stackName: string,
|
||||
options: { lastApplied: string | null; pending?: string; composePaths?: string[]; createDir?: boolean; repoUrl?: string },
|
||||
): void {
|
||||
if (options.createDir !== false) {
|
||||
const composeDir = process.env.COMPOSE_DIR!;
|
||||
fs.mkdirSync(path.join(composeDir, stackName), { recursive: true });
|
||||
fs.writeFileSync(path.join(composeDir, stackName, 'compose.yaml'), 'services: {}\n');
|
||||
}
|
||||
const row: Parameters<DatabaseService['upsertGitSource']>[0] = {
|
||||
stack_name: stackName,
|
||||
repo_url: options.repoUrl ?? REPO,
|
||||
branch: 'main',
|
||||
compose_path: 'compose.yaml',
|
||||
compose_paths: options.composePaths ?? ['compose.yaml'],
|
||||
context_dir: null,
|
||||
sync_env: false,
|
||||
env_path: null,
|
||||
auth_type: 'none',
|
||||
encrypted_token: null,
|
||||
auto_apply_on_webhook: false,
|
||||
auto_deploy_on_apply: false,
|
||||
last_applied_commit_sha: options.lastApplied,
|
||||
last_applied_content_hash: null,
|
||||
pending_commit_sha: options.pending ?? null,
|
||||
pending_compose_content: null,
|
||||
pending_env_content: null,
|
||||
pending_fetched_at: null,
|
||||
last_debounce_at: null,
|
||||
} as StackGitSource;
|
||||
DatabaseService.getInstance().upsertGitSource(row);
|
||||
if (options.lastApplied) {
|
||||
DatabaseService.getInstance().markGitSourceApplied(stackName, options.lastApplied, '');
|
||||
}
|
||||
if (options.pending) {
|
||||
// upsertGitSource does not write the pending columns on insert, so a
|
||||
// legacy row carrying an unapplied pull is seeded directly.
|
||||
DatabaseService.getInstance().getDb()
|
||||
.prepare('UPDATE stack_git_sources SET pending_commit_sha = ? WHERE stack_name = ?')
|
||||
.run(options.pending, stackName);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
/**
|
||||
* Node-side placement recording.
|
||||
*
|
||||
* A label or a cordon is not a statement about any one Blueprint, so what
|
||||
* matters is which Blueprints the change actually moved. Reacting to the event
|
||||
* instead of comparing the resulting sets would invalidate every
|
||||
* acknowledgement in the fleet whenever someone labelled a node nothing selects
|
||||
* on.
|
||||
*/
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import type { Blueprint } from '../services/DatabaseService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import { commitBlueprintCreate } from '../services/gitops/blueprintProducers';
|
||||
import {
|
||||
recordPlacementShift,
|
||||
snapshotPlacementWith,
|
||||
type PlacementSnapshot,
|
||||
} from '../services/gitops/nodePlacementProducers';
|
||||
|
||||
describe('gitops node placement recording', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('revises only the Blueprints whose desired nodes moved', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const moved = create('np-moved');
|
||||
const still = create('np-still');
|
||||
const movedBefore = store.getLiveBlueprintApplication(moved.id)!;
|
||||
const stillBefore = store.getLiveBlueprintApplication(still.id)!;
|
||||
|
||||
const before: PlacementSnapshot = new Map([[moved.id, [1]], [still.id, [1]]]);
|
||||
const after: PlacementSnapshot = new Map([[moved.id, [1, 2]], [still.id, [1]]]);
|
||||
|
||||
expect(recordPlacementShift(before, after, 'tester', 'node_label_add')).toEqual([moved.id]);
|
||||
|
||||
expect(store.getLiveBlueprintApplication(moved.id)!.intent_revision_id)
|
||||
.not.toBe(movedBefore.intent_revision_id);
|
||||
// The Blueprint the label did not move keeps the acknowledgement it had.
|
||||
expect(store.getLiveBlueprintApplication(still.id)!.intent_revision_id)
|
||||
.toBe(stillBefore.intent_revision_id);
|
||||
});
|
||||
|
||||
it('records nothing when the same nodes come back in a different order', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('np-reorder');
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
|
||||
const before: PlacementSnapshot = new Map([[blueprint.id, [1, 2, 3]]]);
|
||||
const after: PlacementSnapshot = new Map([[blueprint.id, [3, 1, 2]]]);
|
||||
|
||||
expect(recordPlacementShift(before, after, 'tester', 'node_cordon')).toEqual([]);
|
||||
expect(store.getLiveBlueprintApplication(blueprint.id)!.intent_revision_id)
|
||||
.toBe(app.intent_revision_id);
|
||||
});
|
||||
|
||||
it('opens the revision as a roster change, not a content change', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const blueprint = create('np-provenance');
|
||||
|
||||
recordPlacementShift(
|
||||
new Map([[blueprint.id, [1]]]),
|
||||
new Map([[blueprint.id, [2]]]),
|
||||
'tester',
|
||||
'node_cordon',
|
||||
);
|
||||
|
||||
const app = store.getLiveBlueprintApplication(blueprint.id)!;
|
||||
const candidate = store.getRolloutCandidate(app.rollout_candidate_id!)!;
|
||||
expect(candidate.provenance).toBe('roster_change');
|
||||
expect(JSON.parse(candidate.required_targets_json)).toEqual({ nodeIds: [2] });
|
||||
});
|
||||
|
||||
it('leaves a Blueprint that predates the model alone', () => {
|
||||
// No application, so nothing to revise. Migration brings it in; inventing a
|
||||
// first intent here would claim a starting point nobody reconciled.
|
||||
const before: PlacementSnapshot = new Map([[99999, [1]]]);
|
||||
const after: PlacementSnapshot = new Map([[99999, [1, 2]]]);
|
||||
expect(recordPlacementShift(before, after, 'tester', 'node_label_add')).toEqual([]);
|
||||
});
|
||||
|
||||
it('snapshots every Blueprint it is given', () => {
|
||||
const a = create('np-snap-a');
|
||||
const b = create('np-snap-b');
|
||||
const snapshot = snapshotPlacementWith(
|
||||
(blueprint) => (blueprint.name === 'np-snap-a' ? [1] : [2, 3]),
|
||||
[a, b],
|
||||
);
|
||||
expect(snapshot.get(a.id)).toEqual([1]);
|
||||
expect(snapshot.get(b.id)).toEqual([2, 3]);
|
||||
});
|
||||
});
|
||||
|
||||
function create(name: string): Blueprint {
|
||||
return commitBlueprintCreate({
|
||||
name,
|
||||
description: null,
|
||||
compose_content: 'services:\n web:\n image: nginx:1.27\n',
|
||||
selector: { type: 'nodes', ids: [1] },
|
||||
drift_mode: 'suggest',
|
||||
classification: 'stateless',
|
||||
classification_reasons: [],
|
||||
enabled: true,
|
||||
created_by: 'tester',
|
||||
}, () => [1]);
|
||||
}
|
||||
@@ -0,0 +1,223 @@
|
||||
/**
|
||||
* Announcement of committed transitions: the metric increment and the
|
||||
* `state-invalidate` event that each newly inserted history row produces.
|
||||
*
|
||||
* The drain is deliberately exercised through the real `setImmediate` rather
|
||||
* than a test-only flush. The whole reason the publisher waits for a macrotask
|
||||
* is that better-sqlite3 transactions are synchronous, so a test that drained
|
||||
* by hand would prove the drain works and prove nothing about when it runs.
|
||||
*/
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { directApplicationFixture } from './helpers/gitopsFixtures';
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { GitOpsMetricsService } from '../services/GitOpsMetricsService';
|
||||
import { insertHistory } from '../services/gitops/history';
|
||||
import {
|
||||
enqueueHistoryPublication,
|
||||
resetGitOpsPublicationsForTests,
|
||||
setGitOpsEventSink,
|
||||
type GitOpsInvalidateEvent,
|
||||
} from '../services/gitops/publish';
|
||||
|
||||
/**
|
||||
* The real module, with the enqueue entry point wrapped in a spy.
|
||||
*
|
||||
* Needed because a replay is suppressed twice over: the insert declines to
|
||||
* enqueue it, and the drain would drop it anyway since the id it carries was
|
||||
* never committed. An outcome assertion therefore passes with the first
|
||||
* mechanism deleted, which is exactly the false green this suite exists to
|
||||
* avoid, so the call itself has to be observable.
|
||||
*/
|
||||
vi.mock('../services/gitops/publish', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('../services/gitops/publish')>();
|
||||
return { ...actual, enqueueHistoryPublication: vi.fn(actual.enqueueHistoryPublication) };
|
||||
});
|
||||
|
||||
/** Let the publisher's own scheduling run. */
|
||||
const settle = (): Promise<void> => new Promise((resolve) => { setImmediate(resolve); });
|
||||
|
||||
describe('gitops transition announcements', () => {
|
||||
let tmpDir: string;
|
||||
let events: GitOpsInvalidateEvent[];
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
resetGitOpsPublicationsForTests();
|
||||
GitOpsMetricsService.resetForTests();
|
||||
vi.mocked(enqueueHistoryPublication).mockClear();
|
||||
});
|
||||
|
||||
const listen = (): void => {
|
||||
events = [];
|
||||
setGitOpsEventSink((event) => { events.push(event); });
|
||||
};
|
||||
|
||||
const db = () => DatabaseService.getInstance().getDb();
|
||||
|
||||
const write = (
|
||||
operationId: string,
|
||||
stage: Parameters<typeof insertHistory>[1]['stage'],
|
||||
outcome: Parameters<typeof insertHistory>[1]['outcome'] = 'committed',
|
||||
overrides: Partial<Parameters<typeof insertHistory>[1]> = {},
|
||||
): string | null => insertHistory(db(), {
|
||||
application: directApplicationFixture(`app-${operationId}`, `stack-${operationId}`),
|
||||
nodeId: 3,
|
||||
dedupeTarget: 'app',
|
||||
operationId,
|
||||
stage,
|
||||
outcome,
|
||||
trigger: 'manual',
|
||||
actor: 'operator-1',
|
||||
before: {},
|
||||
after: {},
|
||||
at: 4242,
|
||||
...overrides,
|
||||
});
|
||||
|
||||
it('announces one event and one count per inserted row', async () => {
|
||||
listen();
|
||||
write('op-1', 'fetch_started');
|
||||
await settle();
|
||||
|
||||
expect(events).toEqual([{
|
||||
type: 'state-invalidate',
|
||||
scope: 'gitops',
|
||||
action: 'fetch_started',
|
||||
applicationId: 'app-op-1',
|
||||
targetMode: 'direct',
|
||||
stackName: 'stack-op-1',
|
||||
blueprintId: null,
|
||||
nodeId: 3,
|
||||
ts: 4242,
|
||||
}]);
|
||||
expect(GitOpsMetricsService.getInstance().snapshot()).toEqual([
|
||||
{ stage: 'fetch_started', outcome: 'committed', count: 1 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('announces rows in the order they were inserted', async () => {
|
||||
listen();
|
||||
write('op-order', 'fetch_started');
|
||||
write('op-order', 'fetched', 'committed', { dedupeTarget: 'node:3' });
|
||||
write('op-order', 'apply_failed', 'failed', { dedupeTarget: 'node:9' });
|
||||
await settle();
|
||||
|
||||
expect(events.map((e) => e.action)).toEqual(['fetch_started', 'fetched', 'apply_failed']);
|
||||
});
|
||||
|
||||
it('says nothing for a transaction that rolled back', async () => {
|
||||
listen();
|
||||
// The row is inserted and then discarded, which is what a transition
|
||||
// throwing after its history write looks like. Announcing it would tell
|
||||
// every client about a state change that never happened.
|
||||
expect(() => db().transaction(() => {
|
||||
write('op-rollback', 'applied');
|
||||
throw new Error('transition rejected');
|
||||
})()).toThrow('transition rejected');
|
||||
await settle();
|
||||
|
||||
expect(events).toEqual([]);
|
||||
expect(GitOpsMetricsService.getInstance().snapshot()).toEqual([]);
|
||||
});
|
||||
|
||||
it('does not even queue a replay of the same transition', async () => {
|
||||
listen();
|
||||
expect(write('op-replay', 'applied')).not.toBeNull();
|
||||
await settle();
|
||||
expect(events).toHaveLength(1);
|
||||
expect(vi.mocked(enqueueHistoryPublication)).toHaveBeenCalledTimes(1);
|
||||
|
||||
// Same application, operation, stage and dedupe target: the dedupe index
|
||||
// rejects it, so no row is inserted and nothing is queued.
|
||||
expect(write('op-replay', 'applied')).toBeNull();
|
||||
await settle();
|
||||
|
||||
expect(vi.mocked(enqueueHistoryPublication)).toHaveBeenCalledTimes(1);
|
||||
expect(events).toHaveLength(1);
|
||||
expect(GitOpsMetricsService.getInstance().snapshot()).toEqual([
|
||||
{ stage: 'applied', outcome: 'committed', count: 1 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('counts even when no sink is installed, and says so once', async () => {
|
||||
events = [];
|
||||
setGitOpsEventSink(null);
|
||||
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||
try {
|
||||
write('op-nosink', 'deploy_started');
|
||||
write('op-nosink', 'deploy_bound', 'committed', { dedupeTarget: 'node:4' });
|
||||
await settle();
|
||||
|
||||
expect(events).toEqual([]);
|
||||
expect(GitOpsMetricsService.getInstance().snapshot()).toEqual([
|
||||
{ stage: 'deploy_bound', outcome: 'committed', count: 1 },
|
||||
{ stage: 'deploy_started', outcome: 'committed', count: 1 },
|
||||
]);
|
||||
// Once for the batch, not once per row: an unwired sink is one fact, and
|
||||
// a boot migration would otherwise fill the log with it.
|
||||
expect(warn).toHaveBeenCalledTimes(1);
|
||||
expect(warn.mock.calls[0][0]).toContain('no event sink installed');
|
||||
} finally {
|
||||
warn.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps announcing the batch when one broadcast throws', async () => {
|
||||
const seen: string[] = [];
|
||||
setGitOpsEventSink((event) => {
|
||||
if (event.action === 'fetched') throw new Error('socket gone');
|
||||
seen.push(event.action);
|
||||
});
|
||||
write('op-throw', 'fetch_started');
|
||||
write('op-throw', 'fetched', 'committed', { dedupeTarget: 'node:1' });
|
||||
write('op-throw', 'applied', 'committed', { dedupeTarget: 'node:2' });
|
||||
await settle();
|
||||
|
||||
expect(seen).toEqual(['fetch_started', 'applied']);
|
||||
// The failed broadcast still happened as far as the model is concerned:
|
||||
// the transition committed, and the count describes the transition.
|
||||
expect(GitOpsMetricsService.getInstance().snapshot().map((e) => e.stage))
|
||||
.toEqual(['applied', 'fetch_started', 'fetched']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GitOpsMetricsService', () => {
|
||||
afterEach(() => {
|
||||
GitOpsMetricsService.resetForTests();
|
||||
});
|
||||
|
||||
it('keeps one count per stage and outcome pair', () => {
|
||||
const metrics = GitOpsMetricsService.getInstance();
|
||||
metrics.record('fetched', 'committed');
|
||||
metrics.record('fetched', 'committed');
|
||||
metrics.record('fetched', 'failed');
|
||||
metrics.record('applied', 'committed');
|
||||
|
||||
expect(metrics.snapshot()).toEqual([
|
||||
{ stage: 'applied', outcome: 'committed', count: 1 },
|
||||
{ stage: 'fetched', outcome: 'committed', count: 2 },
|
||||
{ stage: 'fetched', outcome: 'failed', count: 1 },
|
||||
]);
|
||||
});
|
||||
|
||||
it('reports nothing before anything has been recorded', () => {
|
||||
expect(GitOpsMetricsService.getInstance().snapshot()).toEqual([]);
|
||||
});
|
||||
|
||||
it('hands out copies, so a caller cannot edit the counters', () => {
|
||||
const metrics = GitOpsMetricsService.getInstance();
|
||||
metrics.record('applied', 'committed');
|
||||
const first = metrics.snapshot();
|
||||
first[0].count = 99;
|
||||
|
||||
expect(metrics.snapshot()).toEqual([{ stage: 'applied', outcome: 'committed', count: 1 }]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,191 @@
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { captureGitOpsRecoveryBinding } from '../services/gitops/recoveryCapture';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions, type EventEnvelope } from '../services/gitops/transitions';
|
||||
import type { GitOpsApplicationRow, GitOpsGenerationRow } from '../services/gitops/types';
|
||||
|
||||
describe('gitops recovery capture', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('returns nulls when no live Direct application exists', () => {
|
||||
expect(captureGitOpsRecoveryBinding('missing-stack', 1)).toEqual({
|
||||
gitops_generation_id: null,
|
||||
gitops_artifact_set_id: null,
|
||||
gitops_source_acceptance_ref: null,
|
||||
});
|
||||
});
|
||||
|
||||
it('captures deployed generation and generation-bound source acceptance', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-cap', 'cap-web'), nodeId: 1, envelope: env('op-act') });
|
||||
store.insertGeneration(gen('gen-cap', 'app-cap'));
|
||||
tx.fetchStarted('app-cap', env('op-f'));
|
||||
tx.fetched('app-cap', 'abc123', env('op-f'));
|
||||
tx.candidateReady('app-cap', 'gen-cap', false, env('op-c'));
|
||||
tx.applied({
|
||||
applicationId: 'app-cap',
|
||||
generationId: 'gen-cap',
|
||||
artifactSetId: 'art-cap',
|
||||
sourceAcceptanceId: 'acc-cap',
|
||||
authority: 'operator',
|
||||
envelope: env('op-a'),
|
||||
});
|
||||
const target = store.getTarget('app-cap', 1)!;
|
||||
store.upsertTarget({ ...target, deployed_generation_id: 'gen-cap' });
|
||||
expect(captureGitOpsRecoveryBinding('cap-web', 1)).toEqual({
|
||||
gitops_generation_id: 'gen-cap',
|
||||
gitops_artifact_set_id: 'art-cap',
|
||||
gitops_source_acceptance_ref: 'acc-cap',
|
||||
});
|
||||
});
|
||||
|
||||
it('does not capture a newer generation acceptance for an older deployed generation', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-old', 'old-web'), nodeId: 1, envelope: env('op-act-2') });
|
||||
store.insertGeneration(gen('gen-old', 'app-old'));
|
||||
store.insertGeneration(gen('gen-new', 'app-old'));
|
||||
tx.fetchStarted('app-old', env('op-f2'));
|
||||
tx.fetched('app-old', 'abc123', env('op-f2'));
|
||||
tx.candidateReady('app-old', 'gen-old', false, env('op-c2'));
|
||||
tx.applied({
|
||||
applicationId: 'app-old',
|
||||
generationId: 'gen-old',
|
||||
artifactSetId: 'art-old',
|
||||
sourceAcceptanceId: 'acc-old',
|
||||
authority: 'operator',
|
||||
envelope: env('op-a2'),
|
||||
});
|
||||
store.insertApproval({
|
||||
id: 'acc-new',
|
||||
kind: 'source_acceptance',
|
||||
authority: 'operator',
|
||||
authoritative: 1,
|
||||
application_id: 'app-old',
|
||||
generation_id: 'gen-new',
|
||||
intent_revision_id: null,
|
||||
artifact_set_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
required_targets_json: null,
|
||||
preflight_fingerprint: null,
|
||||
fingerprint: null,
|
||||
blast_json: null,
|
||||
policy_provenance_json: null,
|
||||
actor: 'tester',
|
||||
created_at: 9,
|
||||
});
|
||||
const target = store.getTarget('app-old', 1)!;
|
||||
store.upsertTarget({
|
||||
...target,
|
||||
deployed_generation_id: 'gen-old',
|
||||
source_acceptance_ref: 'acc-new',
|
||||
});
|
||||
const captured = captureGitOpsRecoveryBinding('old-web', 1);
|
||||
expect(captured.gitops_generation_id).toBe('gen-old');
|
||||
expect(captured.gitops_artifact_set_id).toBe('art-old');
|
||||
expect(captured.gitops_source_acceptance_ref).toBe('acc-old');
|
||||
});
|
||||
});
|
||||
|
||||
function env(operationId: string): EventEnvelope {
|
||||
return { operationId, actor: 'tester', trigger: 'manual', at: 1 };
|
||||
}
|
||||
|
||||
function app(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function gen(id: string, applicationId: string): GitOpsGenerationRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
commit_sha: id,
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
configured_ref: 'main',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
manifest_version: 0,
|
||||
candidate_dir: `generations/candidate-${id}`,
|
||||
applied_dir: `generations/applied-${id}-0`,
|
||||
expected_invocation_json: '{"composeFileOrder":[],"projectName":null,"projectDirectory":null,"envFileOrder":[]}',
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
validation_ok: 1,
|
||||
plan_blocked: 0,
|
||||
change_plan_fingerprint: null,
|
||||
operation_id: `op-${id}`,
|
||||
trigger: 'manual',
|
||||
actor: 'tester',
|
||||
previous_generation_id: null,
|
||||
redacted_limitations_json: '[]',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,505 @@
|
||||
/**
|
||||
* Recovery pointer rules.
|
||||
*
|
||||
* A restore moves a target back to an older generation, which is the one case
|
||||
* where the target and its application legitimately disagree about what is
|
||||
* current. These tests pin what may move with it and what may not: the
|
||||
* expectation comes from what the recovery point captured, the acceptance must
|
||||
* still prove the restored generation, and a last-known-good survives unless
|
||||
* the generation behind it is genuinely gone.
|
||||
*/
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions, type EventEnvelope } from '../services/gitops/transitions';
|
||||
import { projectApplication } from '../services/gitops/derive';
|
||||
import type { GitOpsApplicationRow, GitOpsGenerationRow } from '../services/gitops/types';
|
||||
|
||||
describe('gitops recovery', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('marks the target as recovering before anything is restored', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedTwoGenerations('app-rec-start', 'rec-start-web');
|
||||
|
||||
tx.recoveryStarted({
|
||||
applicationId: 'app-rec-start',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-1',
|
||||
recoveryGenerationId: 'gen-a-app-rec-start',
|
||||
envelope: env('op-rec-start'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-rec-start', 1)!;
|
||||
expect(target.recovery_phase).toBe('restoring');
|
||||
expect(target.recovery_ref).toBe('rec-1');
|
||||
expect(target.active_operation_stage).toBe('recovery_started');
|
||||
// Nothing has been restored, so nothing has moved.
|
||||
expect(target.desired_generation_id).toBe('gen-b-app-rec-start');
|
||||
expect(projectApplication('app-rec-start', true).targets[0]?.runtime.status).toBe('recovery_required');
|
||||
});
|
||||
|
||||
it('moves the target back to the restored generation while the application stays ahead', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedTwoGenerations('app-rec-ok', 'rec-ok-web');
|
||||
const genA = 'gen-a-app-rec-ok';
|
||||
|
||||
tx.recoveryStarted({
|
||||
applicationId: 'app-rec-ok',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-ok',
|
||||
recoveryGenerationId: genA,
|
||||
envelope: env('op-rec-ok'),
|
||||
});
|
||||
tx.recoverySucceeded({
|
||||
applicationId: 'app-rec-ok',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-ok',
|
||||
recoveryGenerationId: genA,
|
||||
proven: true,
|
||||
gitopsBinding: 'bound',
|
||||
capturedArtifactSetId: 'art-a-app-rec-ok',
|
||||
capturedSourceAcceptanceRef: 'acc-a-app-rec-ok',
|
||||
envelope: env('op-rec-ok'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-rec-ok', 1)!;
|
||||
expect(target.desired_generation_id).toBe(genA);
|
||||
expect(target.applied_generation_id).toBe(genA);
|
||||
expect(target.deployed_generation_id).toBe(genA);
|
||||
// The restored workload has not been observed healthy yet.
|
||||
expect(target.healthy_generation_id).toBeNull();
|
||||
expect(target.recovery_phase).toBe('complete');
|
||||
// The expectation and the acceptance both describe the restored generation.
|
||||
expect(target.expected_artifact_set_id).toBe('art-a-app-rec-ok');
|
||||
expect(target.source_acceptance_ref).toBe('acc-a-app-rec-ok');
|
||||
// The application is still accepted at the newer generation.
|
||||
expect(store.getApplication('app-rec-ok')?.accepted_generation_id).toBe('gen-b-app-rec-ok');
|
||||
expect(store.getApplication('app-rec-ok')?.source_acceptance_ref).toBe('acc-b-app-rec-ok');
|
||||
});
|
||||
|
||||
it('refuses to bind an acceptance that authorized a different generation', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedTwoGenerations('app-rec-xacc', 'rec-xacc-web');
|
||||
|
||||
tx.recoveryStarted({
|
||||
applicationId: 'app-rec-xacc',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-xacc',
|
||||
recoveryGenerationId: 'gen-a-app-rec-xacc',
|
||||
envelope: env('op-rec-xacc'),
|
||||
});
|
||||
tx.recoverySucceeded({
|
||||
applicationId: 'app-rec-xacc',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-xacc',
|
||||
recoveryGenerationId: 'gen-a-app-rec-xacc',
|
||||
proven: true,
|
||||
gitopsBinding: 'bound',
|
||||
capturedArtifactSetId: 'art-b-app-rec-xacc',
|
||||
// The acceptance for B cannot vouch for A.
|
||||
capturedSourceAcceptanceRef: 'acc-b-app-rec-xacc',
|
||||
envelope: env('op-rec-xacc'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-rec-xacc', 1)!;
|
||||
expect(target.source_acceptance_ref).toBeNull();
|
||||
// Nor can B's artifact set describe A.
|
||||
expect(target.expected_artifact_set_id).toBeNull();
|
||||
});
|
||||
|
||||
it('leaves every pointer alone when the restore cannot be proven', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedTwoGenerations('app-rec-unproven', 'rec-unproven-web');
|
||||
const beforeTarget = store.getTarget('app-rec-unproven', 1)!;
|
||||
|
||||
tx.recoveryStarted({
|
||||
applicationId: 'app-rec-unproven',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-unproven',
|
||||
recoveryGenerationId: null,
|
||||
envelope: env('op-rec-unproven'),
|
||||
});
|
||||
tx.recoverySucceeded({
|
||||
applicationId: 'app-rec-unproven',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-unproven',
|
||||
recoveryGenerationId: null,
|
||||
proven: false,
|
||||
gitopsBinding: 'unbound',
|
||||
capturedArtifactSetId: null,
|
||||
capturedSourceAcceptanceRef: null,
|
||||
envelope: env('op-rec-unproven'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-rec-unproven', 1)!;
|
||||
expect(target.desired_generation_id).toBe(beforeTarget.desired_generation_id);
|
||||
expect(target.applied_generation_id).toBe(beforeTarget.applied_generation_id);
|
||||
expect(target.healthy_generation_id).toBe(beforeTarget.healthy_generation_id);
|
||||
expect(target.recovery_phase).toBe('complete');
|
||||
});
|
||||
|
||||
it('keeps a still-valid last-known-good and records why one is lost', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
|
||||
// A last-known-good on the generation being restored survives intact.
|
||||
seedTwoGenerations('app-rec-lkg', 'rec-lkg-web');
|
||||
db.prepare(
|
||||
`UPDATE gitops_target_current
|
||||
SET lkg_generation_id = 'gen-a-app-rec-lkg', lkg_artifact_set_id = 'art-a-app-rec-lkg'
|
||||
WHERE application_id = 'app-rec-lkg'`,
|
||||
).run();
|
||||
recover('app-rec-lkg', 'gen-a-app-rec-lkg', 'art-a-app-rec-lkg', 'acc-a-app-rec-lkg');
|
||||
let target = store.getTarget('app-rec-lkg', 1)!;
|
||||
expect(target.lkg_generation_id).toBe('gen-a-app-rec-lkg');
|
||||
expect(target.lkg_artifact_set_id).toBe('art-a-app-rec-lkg');
|
||||
expect(target.lkg_unavailable_at).toBeNull();
|
||||
|
||||
// A last-known-good whose generation is gone becomes explicitly
|
||||
// unavailable, which is a different statement from never having had one.
|
||||
seedTwoGenerations('app-rec-lkg-gone', 'rec-lkg-gone-web');
|
||||
db.prepare(
|
||||
`UPDATE gitops_target_current
|
||||
SET lkg_generation_id = 'gen-vanished', lkg_artifact_set_id = NULL
|
||||
WHERE application_id = 'app-rec-lkg-gone'`,
|
||||
).run();
|
||||
recover('app-rec-lkg-gone', 'gen-a-app-rec-lkg-gone', 'art-a-app-rec-lkg-gone', 'acc-a-app-rec-lkg-gone');
|
||||
target = store.getTarget('app-rec-lkg-gone', 1)!;
|
||||
expect(target.lkg_generation_id).toBeNull();
|
||||
expect(target.lkg_unavailable_reason).toBe('generation_missing');
|
||||
expect(projectApplication('app-rec-lkg-gone', true).targets[0]?.lkg.status).toBe('unavailable');
|
||||
});
|
||||
|
||||
it('says why it dropped a pointer it could not prove', () => {
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedTwoGenerations('app-rec-why', 'rec-why-web');
|
||||
|
||||
tx.recoveryStarted({
|
||||
applicationId: 'app-rec-why',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-why',
|
||||
recoveryGenerationId: 'gen-a-app-rec-why',
|
||||
envelope: env('op-rec-why'),
|
||||
});
|
||||
tx.recoverySucceeded({
|
||||
applicationId: 'app-rec-why',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-why',
|
||||
recoveryGenerationId: 'gen-a-app-rec-why',
|
||||
proven: true,
|
||||
gitopsBinding: 'bound',
|
||||
// Both captured references belong to the other generation.
|
||||
capturedArtifactSetId: 'art-b-app-rec-why',
|
||||
capturedSourceAcceptanceRef: 'acc-b-app-rec-why',
|
||||
envelope: env('op-rec-why'),
|
||||
});
|
||||
|
||||
// Without these the cleared pointers are indistinguishable from pointers
|
||||
// that never existed, and the target reads healthier than it is.
|
||||
const codes = projectApplication('app-rec-why', true).limitations.map((l) => l.code);
|
||||
expect(codes).toContain('artifact_expectation_unprovable');
|
||||
expect(codes).toContain('source_acceptance_unprovable');
|
||||
});
|
||||
|
||||
it('flags an unproven restore so it cannot read as healthy', () => {
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedTwoGenerations('app-rec-flag', 'rec-flag-web');
|
||||
|
||||
tx.recoveryStarted({
|
||||
applicationId: 'app-rec-flag',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-flag',
|
||||
recoveryGenerationId: null,
|
||||
envelope: env('op-rec-flag'),
|
||||
});
|
||||
tx.recoverySucceeded({
|
||||
applicationId: 'app-rec-flag',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-flag',
|
||||
recoveryGenerationId: null,
|
||||
proven: false,
|
||||
gitopsBinding: 'unbound',
|
||||
capturedArtifactSetId: null,
|
||||
capturedSourceAcceptanceRef: null,
|
||||
envelope: env('op-rec-flag'),
|
||||
});
|
||||
|
||||
const codes = projectApplication('app-rec-flag', true).limitations.map((l) => l.code);
|
||||
expect(codes).toContain('recovery_unproven');
|
||||
});
|
||||
|
||||
it('clears a limitation once the evidence is provable again', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedTwoGenerations('app-rec-clear', 'rec-clear-web');
|
||||
const genA = 'gen-a-app-rec-clear';
|
||||
|
||||
const restore = (artifactSetId: string, acceptanceRef: string): void => {
|
||||
tx.recoveryStarted({
|
||||
applicationId: 'app-rec-clear',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-clear',
|
||||
recoveryGenerationId: genA,
|
||||
envelope: env(`op-rec-clear-${artifactSetId}`),
|
||||
});
|
||||
tx.recoverySucceeded({
|
||||
applicationId: 'app-rec-clear',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-clear',
|
||||
recoveryGenerationId: genA,
|
||||
proven: true,
|
||||
gitopsBinding: 'bound',
|
||||
capturedArtifactSetId: artifactSetId,
|
||||
capturedSourceAcceptanceRef: acceptanceRef,
|
||||
envelope: env(`op-rec-clear-${artifactSetId}`),
|
||||
});
|
||||
};
|
||||
|
||||
restore('art-b-app-rec-clear', 'acc-b-app-rec-clear');
|
||||
expect(store.getTarget('app-rec-clear', 1)?.evidence_limitations_json).not.toBeNull();
|
||||
|
||||
restore('art-a-app-rec-clear', 'acc-a-app-rec-clear');
|
||||
// A stale limitation is worse than none: it would keep reporting doubt
|
||||
// about evidence that is now proven.
|
||||
expect(store.getTarget('app-rec-clear', 1)?.evidence_limitations_json).toBeNull();
|
||||
expect(projectApplication('app-rec-clear', true).limitations).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('opens and closes a recovery from the restore path itself', async () => {
|
||||
const { StackUpdateRecoveryService } = await import('../services/StackUpdateRecoveryService');
|
||||
const store = GitOpsStore.getInstance();
|
||||
seedTwoGenerations('app-rec-wire', 'rec-wire-web');
|
||||
const genA = 'gen-a-app-rec-wire';
|
||||
|
||||
// A recovery row bound to generation A, exactly as capture writes one.
|
||||
DatabaseService.getInstance().insertStackUpdateRecoveryGeneration({
|
||||
id: 'rec-wire-1',
|
||||
node_id: 1,
|
||||
stack_name: 'rec-wire-web',
|
||||
status: 'candidate',
|
||||
phase: 'captured',
|
||||
is_current: 0,
|
||||
operation_kind: 'update',
|
||||
content_path: null,
|
||||
backup_slot_id: null,
|
||||
services_json: '[]',
|
||||
override_path: null,
|
||||
health_gate_id: null,
|
||||
gate_retain_until: null,
|
||||
artifact_expires_at: null,
|
||||
operation_lease_expires_at: Date.now() + 60_000,
|
||||
created_at: Date.now(),
|
||||
updated_at: Date.now(),
|
||||
created_by: 'tester',
|
||||
artifacts_retired: 0,
|
||||
released_at: null,
|
||||
released_by: null,
|
||||
gitops_generation_id: genA,
|
||||
gitops_artifact_set_id: 'art-a-app-rec-wire',
|
||||
gitops_source_acceptance_ref: 'acc-a-app-rec-wire',
|
||||
});
|
||||
|
||||
// The restore fails before touching files, which is the classification the
|
||||
// model has to get right: the previous workload is provably intact.
|
||||
await StackUpdateRecoveryService.getInstance().compensateWithCandidate(
|
||||
'rec-wire-1',
|
||||
async () => { throw new Error('compose unavailable'); },
|
||||
);
|
||||
|
||||
const target = store.getTarget('app-rec-wire', 1)!;
|
||||
expect(target.recovery_phase).toBe('failed');
|
||||
expect(target.failure_stage).toBe('recovery');
|
||||
expect(target.failure_class).toBe('pre_mutation');
|
||||
expect(target.active_operation_stage).toBeNull();
|
||||
// The restore never completed, so nothing moved back to generation A.
|
||||
expect(target.desired_generation_id).toBe('gen-b-app-rec-wire');
|
||||
expect(projectApplication('app-rec-wire', true).targets[0]?.runtime.status).toBe('recovery_failed');
|
||||
});
|
||||
|
||||
it('records a failed restore without moving success pointers', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedTwoGenerations('app-rec-fail', 'rec-fail-web');
|
||||
const before = store.getTarget('app-rec-fail', 1)!;
|
||||
|
||||
tx.recoveryStarted({
|
||||
applicationId: 'app-rec-fail',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-fail',
|
||||
recoveryGenerationId: 'gen-a-app-rec-fail',
|
||||
envelope: env('op-rec-fail'),
|
||||
});
|
||||
tx.recoveryFailed({
|
||||
applicationId: 'app-rec-fail',
|
||||
nodeId: 1,
|
||||
recoveryRef: 'rec-fail',
|
||||
failureClass: 'post_mutation',
|
||||
envelope: env('op-rec-fail'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-rec-fail', 1)!;
|
||||
expect(target.recovery_phase).toBe('failed');
|
||||
expect(target.failure_stage).toBe('recovery');
|
||||
expect(target.failure_class).toBe('post_mutation');
|
||||
expect(target.desired_generation_id).toBe(before.desired_generation_id);
|
||||
expect(target.active_operation_stage).toBeNull();
|
||||
|
||||
const projection = projectApplication('app-rec-fail', true);
|
||||
if (projection.targetMode === 'not_applicable') throw new Error('expected an application');
|
||||
expect(projection.targets[0]?.runtime.status).toBe('recovery_failed');
|
||||
expect(projection.facets.source.status).toBe('recovery_failed');
|
||||
});
|
||||
});
|
||||
|
||||
function recover(
|
||||
applicationId: string,
|
||||
generationId: string,
|
||||
artifactSetId: string,
|
||||
acceptanceRef: string,
|
||||
): void {
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.recoveryStarted({
|
||||
applicationId,
|
||||
nodeId: 1,
|
||||
recoveryRef: `rec-${applicationId}`,
|
||||
recoveryGenerationId: generationId,
|
||||
envelope: env(`op-${applicationId}`),
|
||||
});
|
||||
tx.recoverySucceeded({
|
||||
applicationId,
|
||||
nodeId: 1,
|
||||
recoveryRef: `rec-${applicationId}`,
|
||||
recoveryGenerationId: generationId,
|
||||
proven: true,
|
||||
gitopsBinding: 'bound',
|
||||
capturedArtifactSetId: artifactSetId,
|
||||
capturedSourceAcceptanceRef: acceptanceRef,
|
||||
envelope: env(`op-${applicationId}`),
|
||||
});
|
||||
}
|
||||
|
||||
/** Apply generation A, then B, so the target has something to fall back to. */
|
||||
function seedTwoGenerations(applicationId: string, stackName: string): void {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app(applicationId, stackName), nodeId: 1, envelope: env(`op-act-${applicationId}`) });
|
||||
for (const label of ['a', 'b'] as const) {
|
||||
const generationId = `gen-${label}-${applicationId}`;
|
||||
store.insertGeneration(gen(generationId, applicationId));
|
||||
tx.fetchStarted(applicationId, env(`op-f-${label}-${applicationId}`));
|
||||
tx.fetched(applicationId, `sha-${label}`, env(`op-f-${label}-${applicationId}`));
|
||||
tx.candidateReady(applicationId, generationId, false, env(`op-c-${label}-${applicationId}`));
|
||||
tx.applied({
|
||||
applicationId,
|
||||
generationId,
|
||||
artifactSetId: `art-${label}-${applicationId}`,
|
||||
sourceAcceptanceId: `acc-${label}-${applicationId}`,
|
||||
authority: 'operator',
|
||||
envelope: env(`op-a-${label}-${applicationId}`),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function env(operationId: string): EventEnvelope {
|
||||
return { operationId, actor: 'tester', trigger: 'manual', at: Date.now() };
|
||||
}
|
||||
|
||||
function app(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function gen(id: string, applicationId: string): GitOpsGenerationRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
commit_sha: 'abc123',
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
configured_ref: 'main',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
manifest_version: 0,
|
||||
candidate_dir: `generations/candidate-${id}`,
|
||||
applied_dir: `generations/applied-${id}-0`,
|
||||
expected_invocation_json: '{"composeFileOrder":[],"projectName":null,"projectDirectory":null,"envFileOrder":[]}',
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
validation_ok: 1,
|
||||
plan_blocked: 0,
|
||||
change_plan_fingerprint: null,
|
||||
operation_id: `op-${id}`,
|
||||
trigger: 'manual',
|
||||
actor: 'tester',
|
||||
previous_generation_id: null,
|
||||
redacted_limitations_json: '[]',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
parseHttpsRepoUrl,
|
||||
parseLegacyRepoUrl,
|
||||
secretFreeRepoUrl,
|
||||
serializeRepoIdentity,
|
||||
} from '../services/gitops/repoIdentity';
|
||||
import { canonicalMaterialConfigJson, materializationFingerprint } from '../services/gitops/fingerprint';
|
||||
|
||||
describe('secret-free repository identity', () => {
|
||||
it('rejects userinfo, query, fragment, and non-https urls', () => {
|
||||
expect(parseHttpsRepoUrl('http://github.com/org/repo.git').ok).toBe(false);
|
||||
const userinfo = parseHttpsRepoUrl('https://user:pass@github.com/org/repo.git');
|
||||
const query = parseHttpsRepoUrl('https://github.com/org/repo.git?token=1');
|
||||
const fragment = parseHttpsRepoUrl('https://github.com/org/repo.git#frag');
|
||||
expect(userinfo.ok ? null : userinfo.reason).toBe('userinfo');
|
||||
expect(query.ok ? null : query.reason).toBe('query');
|
||||
expect(fragment.ok ? null : fragment.reason).toBe('fragment');
|
||||
expect(parseHttpsRepoUrl('https://github.com/org/repo.git').ok).toBe(true);
|
||||
});
|
||||
|
||||
it('serializes host and pathname only', () => {
|
||||
const parsed = parseHttpsRepoUrl('https://github.com/org/repo.git');
|
||||
if (!parsed.ok) throw new Error('expected parse success');
|
||||
const identity = serializeRepoIdentity(parsed.url);
|
||||
expect(identity).toEqual({ host: 'github.com', pathname: '/org/repo.git' });
|
||||
expect(secretFreeRepoUrl(identity)).toBe('https://github.com/org/repo.git');
|
||||
});
|
||||
|
||||
describe('legacy operational urls (migration only)', () => {
|
||||
it('strips userinfo, query, and fragment instead of refusing the stack', () => {
|
||||
for (const raw of [
|
||||
'https://user:pass@github.com/org/repo.git',
|
||||
'https://github.com/org/repo.git?token=secret',
|
||||
'https://github.com/org/repo.git#frag',
|
||||
'https://user:pass@github.com/org/repo.git?token=secret#frag',
|
||||
]) {
|
||||
const parsed = parseLegacyRepoUrl(raw);
|
||||
if (!parsed.ok) throw new Error(`expected legacy parse success for ${raw}`);
|
||||
expect({ host: parsed.url.host, pathname: parsed.url.pathname }).toEqual({
|
||||
host: 'github.com',
|
||||
pathname: '/org/repo.git',
|
||||
});
|
||||
expect(parsed.url.username).toBe('');
|
||||
expect(parsed.url.password).toBe('');
|
||||
expect(parsed.url.search).toBe('');
|
||||
expect(parsed.url.hash).toBe('');
|
||||
expect(secretFreeRepoUrl(serializeRepoIdentity(parsed.url))).toBe('https://github.com/org/repo.git');
|
||||
}
|
||||
});
|
||||
|
||||
it('still refuses what has no storable identity', () => {
|
||||
expect(parseLegacyRepoUrl('http://github.com/org/repo.git').ok).toBe(false);
|
||||
expect(parseLegacyRepoUrl('not a url at all').ok).toBe(false);
|
||||
expect(parseLegacyRepoUrl('').ok).toBe(false);
|
||||
expect(parseLegacyRepoUrl(`https://github.com/${'x'.repeat(2100)}`).ok).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
it('fingerprints material config in the fixed key order', () => {
|
||||
const json = canonicalMaterialConfigJson({
|
||||
repoIdentity: { host: 'github.com', pathname: '/org/repo.git' },
|
||||
configuredRef: 'main',
|
||||
composePaths: ['compose.yml'],
|
||||
contextDir: ' ',
|
||||
syncEnv: false,
|
||||
envPath: '.env',
|
||||
});
|
||||
expect(json).toBe(JSON.stringify({
|
||||
composePaths: ['compose.yml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
repoIdentity: { host: 'github.com', pathname: '/org/repo.git' },
|
||||
configuredRef: 'main',
|
||||
}));
|
||||
expect(materializationFingerprint({
|
||||
repoIdentity: { host: 'github.com', pathname: '/org/repo.git' },
|
||||
configuredRef: 'main',
|
||||
composePaths: ['compose.yml'],
|
||||
contextDir: null,
|
||||
syncEnv: false,
|
||||
envPath: null,
|
||||
})).toMatch(/^[0-9a-f]{64}$/);
|
||||
const synced = canonicalMaterialConfigJson({
|
||||
repoIdentity: { host: 'github.com', pathname: '/org/repo.git' },
|
||||
configuredRef: 'main',
|
||||
composePaths: ['compose.yml'],
|
||||
contextDir: null,
|
||||
syncEnv: true,
|
||||
envPath: '.env',
|
||||
});
|
||||
expect(JSON.parse(synced).envPath).toBe('.env');
|
||||
expect(JSON.parse(synced).syncEnv).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,334 @@
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { isHubOnlyPath } from '../helpers/proxyExemptPaths';
|
||||
import { GitOpsStore, emptyTargetRow } from '../services/gitops/store';
|
||||
import { encodeArtifactEvidenceJson } from '../services/gitops/json';
|
||||
import type { GitOpsApplicationRow, GitOpsGenerationRow } from '../services/gitops/types';
|
||||
|
||||
describe('gitops schema', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('creates gitops tables, recovery columns, and the schema version', async () => {
|
||||
const { DatabaseService } = await import('../services/DatabaseService');
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
const tables = db.prepare(
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND name LIKE 'gitops_%' ORDER BY name",
|
||||
).all() as Array<{ name: string }>;
|
||||
expect(tables.map((row) => row.name)).toEqual([
|
||||
'gitops_applications',
|
||||
'gitops_approvals',
|
||||
'gitops_artifact_sets',
|
||||
'gitops_create_checkpoints',
|
||||
'gitops_generations',
|
||||
'gitops_history',
|
||||
'gitops_intent_revisions',
|
||||
'gitops_migration_checkpoints',
|
||||
'gitops_rollout_candidates',
|
||||
'gitops_target_current',
|
||||
]);
|
||||
const version = db.prepare(
|
||||
"SELECT value FROM global_settings WHERE key = 'gitops_schema_version'",
|
||||
).get() as { value: string };
|
||||
expect(version.value).toBe('1');
|
||||
const recoveryCols = new Set(
|
||||
(db.pragma('table_info(stack_update_recovery_generations)') as Array<{ name: string }>).map((c) => c.name),
|
||||
);
|
||||
expect(recoveryCols.has('gitops_generation_id')).toBe(true);
|
||||
expect(recoveryCols.has('gitops_artifact_set_id')).toBe(true);
|
||||
expect(recoveryCols.has('gitops_source_acceptance_ref')).toBe(true);
|
||||
expect(recoveryCols.has('desired_target_generation_id')).toBe(false);
|
||||
const appCols = new Set(
|
||||
(db.pragma('table_info(gitops_applications)') as Array<{ name: string }>).map((c) => c.name),
|
||||
);
|
||||
expect(appCols.has('desired_target_generation_id')).toBe(false);
|
||||
expect(appCols.has('desired_commit_sha')).toBe(true);
|
||||
const targetCols = new Set(
|
||||
(db.pragma('table_info(gitops_target_current)') as Array<{ name: string }>).map((c) => c.name),
|
||||
);
|
||||
expect(targetCols.has('desired_generation_id')).toBe(true);
|
||||
expect(targetCols.has('candidate_generation_id')).toBe(true);
|
||||
expect(targetCols.has('lkg_artifact_set_id')).toBe(true);
|
||||
expect(targetCols.has('lkg_unavailable_at')).toBe(true);
|
||||
expect(targetCols.has('lkg_unavailable_reason')).toBe(true);
|
||||
const candidateCols = new Set(
|
||||
(db.pragma('table_info(gitops_rollout_candidates)') as Array<{ name: string }>).map((c) => c.name),
|
||||
);
|
||||
expect(candidateCols.has('source_acceptance_ref')).toBe(false);
|
||||
expect(candidateCols.has('placement_approval_ref')).toBe(false);
|
||||
expect(candidateCols.has('preflight_fingerprint')).toBe(false);
|
||||
});
|
||||
|
||||
it('accepts recovery health triggers and keeps deployed_generation_id', async () => {
|
||||
const { DatabaseService } = await import('../services/DatabaseService');
|
||||
const db = DatabaseService.getInstance();
|
||||
db.insertHealthGateRun({
|
||||
id: 'rec-1',
|
||||
node_id: 1,
|
||||
stack_name: 'web',
|
||||
trigger_action: 'recovery',
|
||||
status: 'observing',
|
||||
reason: null,
|
||||
window_seconds: 90,
|
||||
containers_json: '[]',
|
||||
started_at: 1,
|
||||
ended_at: null,
|
||||
created_by: 'tester',
|
||||
target_scope: 'stack',
|
||||
service_name: null,
|
||||
failure_source: null,
|
||||
deployed_generation_id: 'gen-a',
|
||||
});
|
||||
const row = db.getHealthGateRun(1, 'web', 'rec-1');
|
||||
expect(row?.trigger_action).toBe('recovery');
|
||||
expect(row?.deployed_generation_id).toBe('gen-a');
|
||||
});
|
||||
|
||||
it('rejects invalid target recovery phases and LKG mismatches', async () => {
|
||||
const { DatabaseService } = await import('../services/DatabaseService');
|
||||
const db = DatabaseService.getInstance().getDb();
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertApplication(directApp('app-lkg', 'lkg-web'));
|
||||
store.upsertTarget(emptyTargetRow('app-lkg', 1, 1));
|
||||
expect(() => {
|
||||
db.prepare("UPDATE gitops_target_current SET recovery_phase = 'armed' WHERE application_id = 'app-lkg'").run();
|
||||
}).toThrow();
|
||||
expect(() => {
|
||||
store.upsertTarget({
|
||||
...emptyTargetRow('app-lkg', 1, 2),
|
||||
lkg_generation_id: 'gen-missing',
|
||||
lkg_artifact_set_id: 'art-missing',
|
||||
});
|
||||
}).toThrow(/lkg_artifact_set_id/);
|
||||
});
|
||||
|
||||
it('enforces one live Blueprint application across both Blueprint modes', async () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertApplication(inlineApp('bp-inline', 7));
|
||||
expect(store.assertNoLiveBlueprintApplication(7).ok).toBe(false);
|
||||
expect(() => store.insertApplication(blueprintApp('bp-git', 7))).toThrow();
|
||||
const { DatabaseService } = await import('../services/DatabaseService');
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_applications SET lifecycle_status = 'detached' WHERE id = 'bp-inline'",
|
||||
).run();
|
||||
store.insertApplication(blueprintApp('bp-git', 7));
|
||||
expect(store.getApplication('bp-git')?.target_mode).toBe('blueprint');
|
||||
});
|
||||
|
||||
it('round-trips recovery GitOps columns as null on legacy-shaped inserts', async () => {
|
||||
const { DatabaseService } = await import('../services/DatabaseService');
|
||||
const db = DatabaseService.getInstance();
|
||||
db.insertStackUpdateRecoveryGeneration({
|
||||
id: 'recov-1',
|
||||
node_id: 1,
|
||||
stack_name: 'web',
|
||||
status: 'candidate',
|
||||
phase: 'captured',
|
||||
is_current: 1,
|
||||
backup_slot_id: null,
|
||||
content_path: null,
|
||||
operation_kind: null,
|
||||
override_path: null,
|
||||
services_json: '[]',
|
||||
health_gate_id: null,
|
||||
gate_retain_until: null,
|
||||
artifact_expires_at: null,
|
||||
operation_lease_expires_at: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
created_by: 'tester',
|
||||
artifacts_retired: 0,
|
||||
released_at: null,
|
||||
released_by: null,
|
||||
});
|
||||
const row = db.getStackUpdateRecoveryGeneration('recov-1');
|
||||
expect(row?.gitops_generation_id ?? null).toBeNull();
|
||||
expect(row?.gitops_artifact_set_id ?? null).toBeNull();
|
||||
expect(row?.gitops_source_acceptance_ref ?? null).toBeNull();
|
||||
db.insertStackUpdateRecoveryGeneration({
|
||||
id: 'recov-2',
|
||||
node_id: 1,
|
||||
stack_name: 'web',
|
||||
status: 'candidate',
|
||||
phase: 'captured',
|
||||
is_current: 0,
|
||||
backup_slot_id: null,
|
||||
content_path: null,
|
||||
operation_kind: null,
|
||||
override_path: null,
|
||||
services_json: '[]',
|
||||
health_gate_id: null,
|
||||
gate_retain_until: null,
|
||||
artifact_expires_at: null,
|
||||
operation_lease_expires_at: null,
|
||||
created_at: 2,
|
||||
updated_at: 2,
|
||||
created_by: 'tester',
|
||||
artifacts_retired: 0,
|
||||
released_at: null,
|
||||
released_by: null,
|
||||
gitops_generation_id: 'gen-a',
|
||||
gitops_artifact_set_id: 'art-a',
|
||||
gitops_source_acceptance_ref: 'acc-a',
|
||||
});
|
||||
const bound = db.getStackUpdateRecoveryGeneration('recov-2');
|
||||
expect(bound?.gitops_generation_id).toBe('gen-a');
|
||||
expect(bound?.gitops_artifact_set_id).toBe('art-a');
|
||||
expect(bound?.gitops_source_acceptance_ref).toBe('acc-a');
|
||||
});
|
||||
|
||||
it('enforces one live Direct application per stack and frees the name on tombstone', async () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertApplication(directApp('dup-first', 'dup-web'));
|
||||
expect(() => store.insertApplication(directApp('dup-second', 'dup-web'))).toThrow();
|
||||
const { DatabaseService } = await import('../services/DatabaseService');
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_applications SET lifecycle_status = 'deleted' WHERE id = 'dup-first'",
|
||||
).run();
|
||||
store.insertApplication(directApp('dup-second', 'dup-web'));
|
||||
expect(store.getApplication('dup-second')?.stack_name).toBe('dup-web');
|
||||
});
|
||||
|
||||
it('keeps blueprints and node-labels hub-only and git-sources proxyable', () => {
|
||||
expect(isHubOnlyPath('/api/blueprints')).toBe(true);
|
||||
expect(isHubOnlyPath('/api/blueprints/1')).toBe(true);
|
||||
expect(isHubOnlyPath('/api/node-labels')).toBe(true);
|
||||
expect(isHubOnlyPath('/api/node-labels/1')).toBe(true);
|
||||
expect(isHubOnlyPath('/api/git-sources')).toBe(false);
|
||||
expect(isHubOnlyPath('/api/gitops/history')).toBe(false);
|
||||
});
|
||||
|
||||
it('inserts unresolved artifact evidence without advancing authority', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
store.insertApplication(directApp('app-art', 'art-web'));
|
||||
store.insertGeneration(generation('gen-art', 'app-art'));
|
||||
store.insertArtifactSet({
|
||||
id: 'art-1',
|
||||
generation_id: 'gen-art',
|
||||
evidence_version: 1,
|
||||
authoritative: 0,
|
||||
qualification: 'unresolved',
|
||||
evidence_json: encodeArtifactEvidenceJson({ kind: 'unresolved' }),
|
||||
created_at: 1,
|
||||
});
|
||||
expect(store.getArtifactSet('art-1')?.qualification).toBe('unresolved');
|
||||
});
|
||||
});
|
||||
|
||||
function directApp(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function inlineApp(id: string, blueprintId: number): GitOpsApplicationRow {
|
||||
return {
|
||||
...directApp(id, 'unused'),
|
||||
lifecycle_key: `blueprint:${blueprintId}`,
|
||||
target_mode: 'inline_blueprint',
|
||||
stack_name: null,
|
||||
blueprint_id: blueprintId,
|
||||
configured_repo_url: null,
|
||||
repo_identity_json: null,
|
||||
configured_ref: null,
|
||||
compose_paths_json: null,
|
||||
materialization_fingerprint: null,
|
||||
};
|
||||
}
|
||||
|
||||
function blueprintApp(id: string, blueprintId: number): GitOpsApplicationRow {
|
||||
return {
|
||||
...directApp(id, 'unused'),
|
||||
lifecycle_key: `blueprint:${blueprintId}`,
|
||||
target_mode: 'blueprint',
|
||||
stack_name: null,
|
||||
blueprint_id: blueprintId,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
};
|
||||
}
|
||||
|
||||
function generation(id: string, applicationId: string): GitOpsGenerationRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
commit_sha: 'abc123',
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
configured_ref: 'main',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
manifest_version: 0,
|
||||
candidate_dir: 'generations/candidate-abc123',
|
||||
applied_dir: 'generations/applied-abc123-0',
|
||||
expected_invocation_json: '{"composeFileOrder":["compose.yml"],"projectName":null,"projectDirectory":null,"envFileOrder":[]}',
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
validation_ok: 1,
|
||||
plan_blocked: 0,
|
||||
change_plan_fingerprint: null,
|
||||
operation_id: 'op-1',
|
||||
trigger: 'manual',
|
||||
actor: 'tester',
|
||||
previous_generation_id: null,
|
||||
redacted_limitations_json: '[]',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,641 @@
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { encodeArtifactEvidenceJson } from '../services/gitops/json';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions, type EventEnvelope } from '../services/gitops/transitions';
|
||||
import { projectApplication } from '../services/gitops/derive';
|
||||
import type { GitOpsApplicationRow, GitOpsGenerationRow } from '../services/gitops/types';
|
||||
|
||||
describe('gitops transitions', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
GitOpsStore.resetForTests();
|
||||
GitOpsTransitions.resetForTests();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
it('binds desired+applied and source acceptance on Direct apply', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
const env = envelope('op-apply');
|
||||
tx.activateDirect({ application: app('app-apply', 'apply-web'), nodeId: 1, envelope: env });
|
||||
store.insertGeneration(gen('gen-apply', 'app-apply'));
|
||||
tx.fetchStarted('app-apply', envelope('op-fetch'));
|
||||
tx.fetched('app-apply', 'deadbeef', envelope('op-fetch'));
|
||||
tx.candidateReady('app-apply', 'gen-apply', false, envelope('op-cand'));
|
||||
tx.applyStarted('app-apply', 'gen-apply', envelope('op-apply'));
|
||||
tx.applied({
|
||||
applicationId: 'app-apply',
|
||||
generationId: 'gen-apply',
|
||||
artifactSetId: 'art-apply',
|
||||
sourceAcceptanceId: 'acc-apply',
|
||||
authority: 'operator',
|
||||
envelope: env,
|
||||
});
|
||||
const application = store.getApplication('app-apply')!;
|
||||
const target = store.getTarget('app-apply', 1)!;
|
||||
expect(application.accepted_generation_id).toBe('gen-apply');
|
||||
expect(application.source_acceptance_ref).toBe('acc-apply');
|
||||
expect(target.desired_generation_id).toBe('gen-apply');
|
||||
expect(target.applied_generation_id).toBe('gen-apply');
|
||||
expect(target.candidate_generation_id).toBeNull();
|
||||
expect(target.source_acceptance_ref).toBe('acc-apply');
|
||||
expect(target.expected_artifact_set_id).toBe('art-apply');
|
||||
expect(store.resolveApprovalRef('acc-apply', {
|
||||
kind: 'source_acceptance',
|
||||
applicationId: 'app-apply',
|
||||
generationId: 'gen-apply',
|
||||
})?.authoritative).toBe(1);
|
||||
expect(() => tx.applied({
|
||||
applicationId: 'app-apply',
|
||||
generationId: 'gen-other',
|
||||
artifactSetId: 'art-x',
|
||||
sourceAcceptanceId: 'acc-x',
|
||||
authority: 'operator',
|
||||
envelope: envelope('op-apply-2'),
|
||||
})).toThrow(/not the current candidate/);
|
||||
});
|
||||
|
||||
it('advances expected only on first exact after unaccepted rows', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-art', 'art-web', 'gen-art', 'art-v1', 'acc-art');
|
||||
tx.recordArtifactEvidence({
|
||||
applicationId: 'app-art',
|
||||
generationId: 'gen-art',
|
||||
artifactSetId: 'art-v2',
|
||||
evidenceVersion: 2,
|
||||
qualification: 'unavailable',
|
||||
evidenceJson: encodeArtifactEvidenceJson({ kind: 'unavailable' }),
|
||||
authoritative: 0,
|
||||
envelope: envelope('op-art-2'),
|
||||
});
|
||||
tx.recordArtifactEvidence({
|
||||
applicationId: 'app-art',
|
||||
generationId: 'gen-art',
|
||||
artifactSetId: 'art-v3',
|
||||
evidenceVersion: 3,
|
||||
qualification: 'exact',
|
||||
evidenceJson: encodeArtifactEvidenceJson({ kind: 'exact', identity: 'sha256:aaa' }),
|
||||
authoritative: 0,
|
||||
envelope: envelope('op-art-3'),
|
||||
});
|
||||
const application = store.getApplication('app-art')!;
|
||||
expect(application.artifact_set_id).toBe('art-v3');
|
||||
expect(application.latest_artifact_set_id).toBe('art-v3');
|
||||
tx.recordArtifactEvidence({
|
||||
applicationId: 'app-art',
|
||||
generationId: 'gen-art',
|
||||
artifactSetId: 'art-v4',
|
||||
evidenceVersion: 4,
|
||||
qualification: 'stale',
|
||||
evidenceJson: encodeArtifactEvidenceJson({ kind: 'stale', identity: 'sha256:aaa' }),
|
||||
authoritative: 0,
|
||||
envelope: envelope('op-art-4'),
|
||||
});
|
||||
expect(store.getApplication('app-art')?.artifact_set_id).toBe('art-v3');
|
||||
expect(store.getApplication('app-art')?.latest_artifact_set_id).toBe('art-v4');
|
||||
tx.recordArtifactEvidence({
|
||||
applicationId: 'app-art',
|
||||
generationId: 'gen-art',
|
||||
artifactSetId: 'art-v5',
|
||||
evidenceVersion: 5,
|
||||
qualification: 'exact',
|
||||
evidenceJson: encodeArtifactEvidenceJson({ kind: 'exact', identity: 'sha256:bbb' }),
|
||||
authoritative: 0,
|
||||
envelope: envelope('op-art-5'),
|
||||
});
|
||||
expect(store.getApplication('app-art')?.artifact_set_id).toBe('art-v3');
|
||||
expect(store.getApplication('app-art')?.latest_artifact_set_id).toBe('art-v5');
|
||||
expect(store.getTarget('app-art', 1)?.expected_artifact_set_id).toBe('art-v3');
|
||||
expect(store.getTarget('app-art', 1)?.latest_artifact_set_id).toBe('art-v5');
|
||||
});
|
||||
|
||||
it('clears fetch failure on successful fetch and keeps accepted pointers', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-fail', 'fail-web', 'gen-fail', 'art-fail', 'acc-fail');
|
||||
tx.fetchStarted('app-fail', envelope('op-fail'));
|
||||
tx.fetchFailed('app-fail', envelope('op-fail'));
|
||||
expect(store.getApplication('app-fail')?.failure_stage).toBe('fetch');
|
||||
expect(store.getApplication('app-fail')?.accepted_generation_id).toBe('gen-fail');
|
||||
tx.fetchStarted('app-fail', envelope('op-fail-2'));
|
||||
tx.fetched('app-fail', 'cafebabe', envelope('op-fail-2'));
|
||||
const application = store.getApplication('app-fail')!;
|
||||
expect(application.failure_stage).toBeNull();
|
||||
expect(application.desired_commit_sha).toBe('cafebabe');
|
||||
expect(application.accepted_generation_id).toBe('gen-fail');
|
||||
expect(store.getTarget('app-fail', 1)?.applied_generation_id).toBe('gen-fail');
|
||||
});
|
||||
|
||||
it('rejects a candidate whose fingerprint no longer matches configuration', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-fp', 'fp-web'), nodeId: 1, envelope: envelope('op-act-fp') });
|
||||
store.insertGeneration(gen('gen-fp', 'app-fp'));
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_applications SET materialization_fingerprint = ? WHERE id = 'app-fp'",
|
||||
).run('b'.repeat(64));
|
||||
expect(() => tx.candidateReady('app-fp', 'gen-fp', false, envelope('op-c-fp'))).toThrow(/fingerprint/);
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_applications SET materialization_fingerprint = ? WHERE id = 'app-fp'",
|
||||
).run('a'.repeat(64));
|
||||
tx.fetchStarted('app-fp', envelope('op-f-fp'));
|
||||
tx.fetched('app-fp', 'abc123', envelope('op-f-fp'));
|
||||
tx.candidateReady('app-fp', 'gen-fp', false, envelope('op-c-fp2'));
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_applications SET materialization_fingerprint = ? WHERE id = 'app-fp'",
|
||||
).run('c'.repeat(64));
|
||||
expect(() => tx.applyStarted('app-fp', 'gen-fp', envelope('op-a-fp'))).toThrow(/fingerprint/);
|
||||
});
|
||||
|
||||
it('refuses to replace the candidate while an apply is in flight', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-race', 'race-web'), nodeId: 1, envelope: envelope('op-act-race') });
|
||||
store.insertGeneration(gen('gen-race-a', 'app-race'));
|
||||
store.insertGeneration(gen('gen-race-b', 'app-race'));
|
||||
tx.fetchStarted('app-race', envelope('op-f-race'));
|
||||
tx.fetched('app-race', 'abc123', envelope('op-f-race'));
|
||||
tx.candidateReady('app-race', 'gen-race-a', false, envelope('op-c-race-a'));
|
||||
tx.applyStarted('app-race', 'gen-race-a', envelope('op-a-race'));
|
||||
expect(() => tx.candidateReady('app-race', 'gen-race-b', false, envelope('op-c-race-b')))
|
||||
.toThrow(/apply is in flight/);
|
||||
expect(store.getApplication('app-race')?.candidate_generation_id).toBe('gen-race-a');
|
||||
tx.applied({
|
||||
applicationId: 'app-race',
|
||||
generationId: 'gen-race-a',
|
||||
artifactSetId: 'art-race',
|
||||
sourceAcceptanceId: 'acc-race',
|
||||
authority: 'operator',
|
||||
envelope: envelope('op-a-race'),
|
||||
});
|
||||
expect(store.getApplication('app-race')?.accepted_generation_id).toBe('gen-race-a');
|
||||
});
|
||||
|
||||
it('rejects re-accepting a generation that is already accepted', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-reapply', 'reapply-web', 'gen-reapply', 'art-reapply', 'acc-reapply');
|
||||
tx.candidateReady('app-reapply', 'gen-reapply', false, envelope('op-c-reapply-2'));
|
||||
expect(() => tx.applied({
|
||||
applicationId: 'app-reapply',
|
||||
generationId: 'gen-reapply',
|
||||
artifactSetId: 'art-reapply-2',
|
||||
sourceAcceptanceId: 'acc-reapply-2',
|
||||
authority: 'operator',
|
||||
envelope: envelope('op-a-reapply-2'),
|
||||
})).toThrow(/already accepted/);
|
||||
expect(store.getArtifactSet('art-reapply-2')).toBeUndefined();
|
||||
});
|
||||
|
||||
it('surfaces an invalid stored observation as a limitation instead of a clean unknown', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
seedApplied('app-obs', 'obs-web', 'gen-obs', 'art-obs', 'acc-obs');
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_target_current SET observed_artifact_identity_json = ? WHERE application_id = 'app-obs'",
|
||||
).run('{"kind":"nonsense"}');
|
||||
const projection = mustProject('app-obs');
|
||||
expect(projection.limitations.map((l) => l.code)).toContain('artifact_observation_invalid');
|
||||
expect(store.getTarget('app-obs', 1)?.observed_artifact_identity_json).toBe('{"kind":"nonsense"}');
|
||||
});
|
||||
|
||||
it('advances the fetched SHA on an invalid commit without minting a candidate', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-inv', 'inv-web'), nodeId: 1, envelope: envelope('op-act-inv') });
|
||||
tx.fetchStarted('app-inv', envelope('op-f-inv'));
|
||||
tx.fetchedInvalid('app-inv', 'bad1234', envelope('op-f-inv'));
|
||||
const application = store.getApplication('app-inv')!;
|
||||
expect(application.desired_commit_sha).toBe('bad1234');
|
||||
expect(application.fetched_commit_sha).toBe('bad1234');
|
||||
expect(application.candidate_generation_id).toBeNull();
|
||||
expect(application.failure_stage).toBe('validation');
|
||||
expect(mustProject('app-inv').facets.source.status).toBe('source_failed');
|
||||
});
|
||||
|
||||
it('exposes a blocked candidate without allowing it to apply', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-blk', 'blk-web'), nodeId: 1, envelope: envelope('op-act-blk') });
|
||||
store.insertGeneration({ ...gen('gen-blk', 'app-blk'), plan_blocked: 1 });
|
||||
tx.fetchStarted('app-blk', envelope('op-f-blk'));
|
||||
tx.fetched('app-blk', 'abc123', envelope('op-f-blk'));
|
||||
tx.sourceConflictBlocker('app-blk', 'gen-blk', envelope('op-b-blk'));
|
||||
expect(store.getApplication('app-blk')?.candidate_plan_blocked).toBe(1);
|
||||
expect(store.getTarget('app-blk', 1)?.candidate_generation_id).toBe('gen-blk');
|
||||
const projection = mustProject('app-blk');
|
||||
expect(projection.facets.source.status).toBe('source_conflict_blocker');
|
||||
expect(projection.availableActions).not.toContain('apply');
|
||||
expect(() => tx.applyStarted('app-blk', 'gen-blk', envelope('op-a-blk'))).toThrow(/blocked/);
|
||||
});
|
||||
|
||||
it('dismisses a candidate without touching what is already applied', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-dis', 'dis-web', 'gen-dis', 'art-dis', 'acc-dis');
|
||||
store.insertGeneration(gen('gen-dis-2', 'app-dis'));
|
||||
tx.candidateReady('app-dis', 'gen-dis-2', false, envelope('op-c-dis'));
|
||||
tx.dismissed('app-dis', envelope('op-d-dis'));
|
||||
const application = store.getApplication('app-dis')!;
|
||||
expect(application.candidate_generation_id).toBeNull();
|
||||
expect(application.accepted_generation_id).toBe('gen-dis');
|
||||
expect(store.getTarget('app-dis', 1)?.applied_generation_id).toBe('gen-dis');
|
||||
expect(store.getTarget('app-dis', 1)?.candidate_generation_id).toBeNull();
|
||||
});
|
||||
|
||||
it('refuses to dismiss while an operation is in flight', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-dis2', 'dis2-web'), nodeId: 1, envelope: envelope('op-act-dis2') });
|
||||
store.insertGeneration(gen('gen-dis2', 'app-dis2'));
|
||||
tx.fetchStarted('app-dis2', envelope('op-f-dis2'));
|
||||
tx.fetched('app-dis2', 'abc123', envelope('op-f-dis2'));
|
||||
tx.candidateReady('app-dis2', 'gen-dis2', false, envelope('op-c-dis2'));
|
||||
tx.applyStarted('app-dis2', 'gen-dis2', envelope('op-a-dis2'));
|
||||
expect(() => tx.dismissed('app-dis2', envelope('op-d-dis2'))).toThrow(/in flight/);
|
||||
expect(store.getApplication('app-dis2')?.candidate_generation_id).toBe('gen-dis2');
|
||||
});
|
||||
|
||||
it('invalidates a staged candidate when the material configuration changes', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-cfg', 'cfg-web', 'gen-cfg', 'art-cfg', 'acc-cfg');
|
||||
store.insertGeneration(gen('gen-cfg-2', 'app-cfg'));
|
||||
tx.candidateReady('app-cfg', 'gen-cfg-2', false, envelope('op-c-cfg'));
|
||||
tx.configChangedPendingCleared({
|
||||
applicationId: 'app-cfg',
|
||||
identity: {
|
||||
repoUrl: 'https://github.com/org/other.git',
|
||||
repoIdentityJson: '{"host":"github.com","pathname":"/org/other.git"}',
|
||||
configuredRef: 'release',
|
||||
},
|
||||
material: {
|
||||
composePathsJson: '["compose.yml","compose.prod.yml"]',
|
||||
contextDir: null,
|
||||
syncEnv: 0,
|
||||
envPath: null,
|
||||
fingerprint: 'd'.repeat(64),
|
||||
},
|
||||
envelope: envelope('op-cfg'),
|
||||
});
|
||||
const application = store.getApplication('app-cfg')!;
|
||||
expect(application.configured_ref).toBe('release');
|
||||
expect(application.materialization_fingerprint).toBe('d'.repeat(64));
|
||||
expect(application.desired_commit_sha).toBeNull();
|
||||
expect(application.candidate_generation_id).toBeNull();
|
||||
// The workload that is running did not change because the config did.
|
||||
expect(application.accepted_generation_id).toBe('gen-cfg');
|
||||
expect(store.getTarget('app-cfg', 1)?.applied_generation_id).toBe('gen-cfg');
|
||||
const projection = mustProject('app-cfg');
|
||||
expect(projection.facets.source.status).toBe('source_reconcile_required');
|
||||
expect(projection.availableActions).toContain('fetch');
|
||||
});
|
||||
|
||||
it('records deploy failures without moving the deployed generation', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-dep', 'dep-web', 'gen-dep', 'art-dep', 'acc-dep');
|
||||
tx.deployStarted('app-dep', 1, 'gen-dep', envelope('op-dep-1'));
|
||||
tx.deployUnbound('app-dep', 1, 'gen-dep', envelope('op-dep-1'));
|
||||
let target = store.getTarget('app-dep', 1)!;
|
||||
expect(target.deployed_generation_id).toBeNull();
|
||||
expect(target.failure_class).toBe('unbound');
|
||||
expect(mustProject('app-dep').targets[0]?.runtime.status).toBe('failed_previous_workload_intact');
|
||||
|
||||
tx.deployStarted('app-dep', 1, 'gen-dep', envelope('op-dep-2'));
|
||||
tx.deployFailed('app-dep', 1, 'post_mutation', envelope('op-dep-2'));
|
||||
target = store.getTarget('app-dep', 1)!;
|
||||
expect(target.deployed_generation_id).toBeNull();
|
||||
expect(target.failure_class).toBe('post_mutation');
|
||||
expect(mustProject('app-dep').targets[0]?.runtime.status).toBe('failed_after_mutation');
|
||||
|
||||
// A later success clears the failure in the same move as the pointer.
|
||||
tx.deployStarted('app-dep', 1, 'gen-dep', envelope('op-dep-3'));
|
||||
tx.deployBound('app-dep', 1, 'gen-dep', envelope('op-dep-3'));
|
||||
target = store.getTarget('app-dep', 1)!;
|
||||
expect(target.deployed_generation_id).toBe('gen-dep');
|
||||
expect(target.failure_stage).toBeNull();
|
||||
});
|
||||
|
||||
it('promotes healthy and last-known-good only for the generation the run watched', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-hl', 'hl-web', 'gen-hl', 'art-hl', 'acc-hl');
|
||||
tx.deployStarted('app-hl', 1, 'gen-hl', envelope('op-hl-dep'));
|
||||
tx.deployBound('app-hl', 1, 'gen-hl', envelope('op-hl-dep'));
|
||||
|
||||
// A verdict for a generation that is not the deployed one proves nothing.
|
||||
tx.healthFinalized({
|
||||
applicationId: 'app-hl',
|
||||
nodeId: 1,
|
||||
healthRunId: 'run-stale',
|
||||
healthStatus: 'passed',
|
||||
deployedGenerationId: 'gen-other',
|
||||
targetScope: 'stack',
|
||||
envelope: envelope('op-hl-stale'),
|
||||
});
|
||||
expect(store.getTarget('app-hl', 1)?.healthy_generation_id).toBeNull();
|
||||
|
||||
// Nor does a service-scoped run, which never observed the whole stack.
|
||||
tx.healthFinalized({
|
||||
applicationId: 'app-hl',
|
||||
nodeId: 1,
|
||||
healthRunId: 'run-service',
|
||||
healthStatus: 'passed',
|
||||
deployedGenerationId: 'gen-hl',
|
||||
targetScope: 'service',
|
||||
envelope: envelope('op-hl-service'),
|
||||
});
|
||||
expect(store.getTarget('app-hl', 1)?.healthy_generation_id).toBeNull();
|
||||
|
||||
// Nor does a failure.
|
||||
tx.healthFinalized({
|
||||
applicationId: 'app-hl',
|
||||
nodeId: 1,
|
||||
healthRunId: 'run-failed',
|
||||
healthStatus: 'failed',
|
||||
deployedGenerationId: 'gen-hl',
|
||||
targetScope: 'stack',
|
||||
envelope: envelope('op-hl-failed'),
|
||||
});
|
||||
expect(store.getTarget('app-hl', 1)?.healthy_generation_id).toBeNull();
|
||||
|
||||
tx.healthFinalized({
|
||||
applicationId: 'app-hl',
|
||||
nodeId: 1,
|
||||
healthRunId: 'run-pass',
|
||||
healthStatus: 'passed',
|
||||
deployedGenerationId: 'gen-hl',
|
||||
targetScope: 'stack',
|
||||
envelope: envelope('op-hl-pass'),
|
||||
});
|
||||
const target = store.getTarget('app-hl', 1)!;
|
||||
expect(target.healthy_generation_id).toBe('gen-hl');
|
||||
expect(target.lkg_generation_id).toBe('gen-hl');
|
||||
// The expected artifact belongs to this generation, so it is kept as the
|
||||
// qualification evidence for the last-known-good.
|
||||
expect(target.lkg_artifact_set_id).toBe('art-hl');
|
||||
expect(target.lkg_unavailable_at).toBeNull();
|
||||
const projection = mustProject('app-hl');
|
||||
expect(projection.targets[0]?.runtime.status).toBe('synced_and_healthy');
|
||||
expect(projection.targets[0]?.lkg.status).not.toBe('none');
|
||||
});
|
||||
|
||||
it('keeps the last-known-good generation when its artifact belongs elsewhere', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-lkg', 'lkg-web', 'gen-lkg', 'art-lkg', 'acc-lkg');
|
||||
tx.deployStarted('app-lkg', 1, 'gen-lkg', envelope('op-lkg-dep'));
|
||||
tx.deployBound('app-lkg', 1, 'gen-lkg', envelope('op-lkg-dep'));
|
||||
// Clear the expectation so the promotion has no artifact to qualify with.
|
||||
DatabaseService.getInstance().getDb().prepare(
|
||||
"UPDATE gitops_target_current SET expected_artifact_set_id = NULL WHERE application_id = 'app-lkg'",
|
||||
).run();
|
||||
|
||||
tx.healthFinalized({
|
||||
applicationId: 'app-lkg',
|
||||
nodeId: 1,
|
||||
healthRunId: 'run-lkg',
|
||||
healthStatus: 'passed',
|
||||
deployedGenerationId: 'gen-lkg',
|
||||
targetScope: 'stack',
|
||||
envelope: envelope('op-lkg-pass'),
|
||||
});
|
||||
|
||||
const target = store.getTarget('app-lkg', 1)!;
|
||||
// The generation is still good; only its executable identity is unproven.
|
||||
expect(target.lkg_generation_id).toBe('gen-lkg');
|
||||
expect(target.lkg_artifact_set_id).toBeNull();
|
||||
expect(mustProject('app-lkg').targets[0]?.lkg.status).toBe('available');
|
||||
});
|
||||
|
||||
it('tombstones an application and its target, and never reactivates it', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-tomb', 'tomb-web', 'gen-tomb', 'art-tomb', 'acc-tomb');
|
||||
tx.targetTombstoned('app-tomb', 1, envelope('op-tomb'));
|
||||
tx.applicationTombstoned('app-tomb', 'detached', envelope('op-tomb'));
|
||||
const application = store.getApplication('app-tomb')!;
|
||||
expect(application.lifecycle_status).toBe('detached');
|
||||
// Configured identity survives as a frozen fact.
|
||||
expect(application.configured_repo_url).toBe('https://github.com/org/repo.git');
|
||||
expect(application.desired_commit_sha).toBe('abc123');
|
||||
expect(store.getTarget('app-tomb', 1)?.target_status).toBe('tombstoned');
|
||||
expect(store.getLiveDirectApplication('tomb-web')).toBeUndefined();
|
||||
expect(() => tx.applicationTombstoned('app-tomb', 'deleted', envelope('op-tomb-2')))
|
||||
.toThrow(/already tombstoned/);
|
||||
expect(mustProject('app-tomb').facets.source.status).toBe('not_live');
|
||||
});
|
||||
|
||||
it('retires every live target on a node without touching its applications', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
seedApplied('app-node-a', 'node-a-web', 'gen-node-a', 'art-node-a', 'acc-node-a');
|
||||
seedApplied('app-node-b', 'node-b-web', 'gen-node-b', 'art-node-b', 'acc-node-b');
|
||||
|
||||
tx.tombstoneNodeTargets(1, envelope('op-node-del'));
|
||||
|
||||
expect(store.getTarget('app-node-a', 1)?.target_status).toBe('tombstoned');
|
||||
expect(store.getTarget('app-node-b', 1)?.target_status).toBe('tombstoned');
|
||||
// The applications still describe real stacks, so they stay live.
|
||||
expect(store.getApplication('app-node-a')?.lifecycle_status).toBe('active');
|
||||
expect(store.getApplication('app-node-b')?.lifecycle_status).toBe('active');
|
||||
// Replaying finds nothing left to retire.
|
||||
expect(tx.tombstoneNodeTargets(1, envelope('op-node-del-2')).historyIds).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('rejects terminal events with no matching operation', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-guard', 'guard-web'), nodeId: 1, envelope: envelope('op-act-guard') });
|
||||
store.insertGeneration(gen('gen-guard', 'app-guard'));
|
||||
expect(() => tx.applyFailed('app-guard', 'apply', envelope('op-g1')))
|
||||
.toThrow(/no matching apply operation/);
|
||||
expect(() => tx.deployStarted('app-guard', 1, 'gen-guard', envelope('op-g2')))
|
||||
.toThrow(/not applied/);
|
||||
expect(() => tx.deployBound('app-guard', 1, 'gen-guard', envelope('op-g3')))
|
||||
.toThrow(/no matching deploy operation/);
|
||||
expect(store.getTarget('app-guard', 1)?.deployed_generation_id).toBeNull();
|
||||
});
|
||||
|
||||
it('writes one history row per transition with the bound identity', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-hist', 'hist-web'), nodeId: 1, envelope: envelope('op-act-hist') });
|
||||
store.insertGeneration(gen('gen-hist', 'app-hist'));
|
||||
tx.fetchStarted('app-hist', envelope('op-f-hist'));
|
||||
tx.fetched('app-hist', 'abc123', envelope('op-f-hist'));
|
||||
tx.candidateReady('app-hist', 'gen-hist', false, envelope('op-c-hist'));
|
||||
const applied = tx.applied({
|
||||
applicationId: 'app-hist',
|
||||
generationId: 'gen-hist',
|
||||
artifactSetId: 'art-hist',
|
||||
sourceAcceptanceId: 'acc-hist',
|
||||
authority: 'operator',
|
||||
envelope: envelope('op-a-hist'),
|
||||
});
|
||||
expect(applied.replayed).toBe(false);
|
||||
expect(applied.historyIds).toHaveLength(1);
|
||||
const row = DatabaseService.getInstance().getDb().prepare(
|
||||
'SELECT stage, outcome, dedupe_target, generation_id, artifact_set_id, source_acceptance_ref, node_id FROM gitops_history WHERE id = ?',
|
||||
).get(applied.historyIds[0]) as Record<string, unknown>;
|
||||
expect(row.stage).toBe('applied');
|
||||
expect(row.outcome).toBe('committed');
|
||||
expect(row.dedupe_target).toBe('app');
|
||||
expect(row.generation_id).toBe('gen-hist');
|
||||
expect(row.artifact_set_id).toBe('art-hist');
|
||||
expect(row.source_acceptance_ref).toBe('acc-hist');
|
||||
expect(row.node_id).toBeNull();
|
||||
});
|
||||
|
||||
it('interrupts live apply and binds deploy after applied', () => {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app('app-int', 'int-web'), nodeId: 1, envelope: envelope('op-act-int') });
|
||||
store.insertGeneration(gen('gen-int', 'app-int'));
|
||||
tx.fetchStarted('app-int', envelope('op-f-int'));
|
||||
tx.fetched('app-int', 'abc123', envelope('op-f-int'));
|
||||
tx.candidateReady('app-int', 'gen-int', false, envelope('op-c-int'));
|
||||
tx.applyStarted('app-int', 'gen-int', envelope('op-apply-live'));
|
||||
expect(mustProject('app-int').facets.source.status).toBe('applying');
|
||||
tx.interruptActiveOperations('app-int', envelope('op-boot'));
|
||||
expect(mustProject('app-int').facets.source.status).toBe('source_unknown');
|
||||
tx.applied({
|
||||
applicationId: 'app-int',
|
||||
generationId: 'gen-int',
|
||||
artifactSetId: 'art-int',
|
||||
sourceAcceptanceId: 'acc-int',
|
||||
authority: 'operator',
|
||||
envelope: envelope('op-a-int'),
|
||||
});
|
||||
tx.deployStarted('app-int', 1, 'gen-int', envelope('op-dep'));
|
||||
tx.deployBound('app-int', 1, 'gen-int', envelope('op-dep'));
|
||||
expect(store.getTarget('app-int', 1)?.deployed_generation_id).toBe('gen-int');
|
||||
expect(store.getTarget('app-int', 1)?.failure_stage).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
function mustProject(applicationId: string) {
|
||||
const projection = projectApplication(applicationId, false);
|
||||
if (projection.targetMode === 'not_applicable') throw new Error('expected application');
|
||||
return projection;
|
||||
}
|
||||
|
||||
function seedApplied(
|
||||
applicationId: string,
|
||||
stackName: string,
|
||||
generationId: string,
|
||||
artifactSetId: string,
|
||||
sourceAcceptanceId: string,
|
||||
): void {
|
||||
const store = GitOpsStore.getInstance();
|
||||
const tx = GitOpsTransitions.getInstance();
|
||||
tx.activateDirect({ application: app(applicationId, stackName), nodeId: 1, envelope: envelope(`op-act-${applicationId}`) });
|
||||
store.insertGeneration(gen(generationId, applicationId));
|
||||
tx.fetchStarted(applicationId, envelope(`op-f-${applicationId}`));
|
||||
tx.fetched(applicationId, 'abc123', envelope(`op-f-${applicationId}`));
|
||||
tx.candidateReady(applicationId, generationId, false, envelope(`op-c-${applicationId}`));
|
||||
tx.applied({
|
||||
applicationId,
|
||||
generationId,
|
||||
artifactSetId,
|
||||
sourceAcceptanceId,
|
||||
authority: 'operator',
|
||||
envelope: envelope(`op-a-${applicationId}`),
|
||||
});
|
||||
}
|
||||
|
||||
function envelope(operationId: string): EventEnvelope {
|
||||
return { operationId, actor: 'tester', trigger: 'manual', at: Date.now() };
|
||||
}
|
||||
|
||||
function app(id: string, stackName: string): GitOpsApplicationRow {
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/org/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: 1,
|
||||
updated_at: 1,
|
||||
};
|
||||
}
|
||||
|
||||
function gen(id: string, applicationId: string): GitOpsGenerationRow {
|
||||
return {
|
||||
id,
|
||||
application_id: applicationId,
|
||||
commit_sha: 'abc123',
|
||||
repo_url: 'https://github.com/org/repo.git',
|
||||
configured_ref: 'main',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
|
||||
manifest_version: 0,
|
||||
candidate_dir: `generations/candidate-${id}`,
|
||||
applied_dir: `generations/applied-${id}-0`,
|
||||
expected_invocation_json: '{"composeFileOrder":[],"projectName":null,"projectDirectory":null,"envFileOrder":[]}',
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
validation_ok: 1,
|
||||
plan_blocked: 0,
|
||||
change_plan_fingerprint: null,
|
||||
operation_id: `op-${id}`,
|
||||
trigger: 'manual',
|
||||
actor: 'tester',
|
||||
previous_generation_id: null,
|
||||
redacted_limitations_json: '[]',
|
||||
created_at: 1,
|
||||
};
|
||||
}
|
||||
@@ -52,7 +52,7 @@ vi.mock('../services/DatabaseService', () => ({
|
||||
.sort((a, b) => b.started_at - a.started_at);
|
||||
return matches[0] ? { ...matches[0] } : undefined;
|
||||
},
|
||||
markInterruptedHealthGateRuns: () => 0,
|
||||
listObservingHealthGateRuns: () => [],
|
||||
addNotificationHistory: (_nodeId: number, item: { category?: string; message: string; level: string }) => ({ ...item, id: 1, is_read: false }),
|
||||
}),
|
||||
},
|
||||
@@ -209,7 +209,7 @@ describe('prepare / beginPrepared nullability', () => {
|
||||
});
|
||||
|
||||
it('persists an immediate unknown past the concurrency cap', async () => {
|
||||
for (let i = 0; i < 25; i++) svc().beginStack(0, `stack-${i}`, 'update', 'tester');
|
||||
for (let i = 0; i < 25; i++) svc().beginStack(0, `stack-${i}`, 'update', 'tester', { deployedGenerationId: null });
|
||||
const token = await prepareService([{ id: 'p1', name: 'web-app-1', service: 'app' }]);
|
||||
svc().attachExpectedImage(token, 'sha256:app');
|
||||
const result = svc().beginPrepared({ prepareToken: token, actor: 'tester' });
|
||||
|
||||
@@ -9,7 +9,7 @@ interface StoredRun {
|
||||
id: string;
|
||||
node_id: number;
|
||||
stack_name: string;
|
||||
trigger_action: 'update' | 'deploy' | 'service_update' | 'service_restore';
|
||||
trigger_action: 'update' | 'deploy' | 'service_update' | 'service_restore' | 'recovery';
|
||||
status: 'observing' | 'passed' | 'failed' | 'unknown';
|
||||
reason: string | null;
|
||||
window_seconds: number;
|
||||
@@ -20,13 +20,17 @@ interface StoredRun {
|
||||
target_scope: 'stack' | 'service';
|
||||
service_name: string | null;
|
||||
failure_source: 'primary' | 'collateral' | null;
|
||||
deployed_generation_id?: string | null;
|
||||
}
|
||||
|
||||
const { state } = vi.hoisted(() => ({
|
||||
state: {
|
||||
runs: new Map<string, StoredRun>(),
|
||||
recoveries: new Map<string, { id: string; health_gate_id: string | null }>(),
|
||||
activity: [] as Array<{ category?: string; message: string; level: string }>,
|
||||
settings: {} as Record<string, string>,
|
||||
/** Run id whose finalize write should fail, for the per-row sweep guard. */
|
||||
failFinalizeFor: null as string | null,
|
||||
listContainers: vi.fn(),
|
||||
inspect: vi.fn(),
|
||||
renderConfig: vi.fn(),
|
||||
@@ -39,6 +43,7 @@ vi.mock('../services/DatabaseService', () => ({
|
||||
getGlobalSettings: () => state.settings,
|
||||
insertHealthGateRun: (run: StoredRun) => { state.runs.set(run.id, { ...run }); },
|
||||
finalizeHealthGateRun: (id: string, status: StoredRun['status'], reason: string | null, endedAt: number, containersJson: string, failureSource: StoredRun['failure_source'] = null) => {
|
||||
if (state.failFinalizeFor === id) throw new Error('row is unreadable');
|
||||
const run = state.runs.get(id);
|
||||
if (run) Object.assign(run, { status, reason, ended_at: endedAt, containers_json: containersJson, failure_source: failureSource });
|
||||
},
|
||||
@@ -52,16 +57,17 @@ vi.mock('../services/DatabaseService', () => ({
|
||||
.sort((a, b) => b.started_at - a.started_at);
|
||||
return matches[0] ? { ...matches[0] } : undefined;
|
||||
},
|
||||
markInterruptedHealthGateRuns: (reason: string, endedAt: number) => {
|
||||
let n = 0;
|
||||
for (const run of state.runs.values()) {
|
||||
if (run.status === 'observing') {
|
||||
Object.assign(run, { status: 'unknown', reason, ended_at: endedAt });
|
||||
n++;
|
||||
}
|
||||
}
|
||||
return n;
|
||||
getStackUpdateRecoveryGeneration: (id: string) => {
|
||||
const row = state.recoveries.get(id);
|
||||
return row ? { ...row } : undefined;
|
||||
},
|
||||
updateStackUpdateRecoveryGeneration: (id: string, patch: { health_gate_id?: string | null }) => {
|
||||
const row = state.recoveries.get(id);
|
||||
if (row) Object.assign(row, patch);
|
||||
},
|
||||
listObservingHealthGateRuns: () => [...state.runs.values()]
|
||||
.filter(run => run.status === 'observing')
|
||||
.map(run => ({ ...run })),
|
||||
addNotificationHistory: (_nodeId: number, item: { category?: string; message: string; level: string }) => {
|
||||
state.activity.push(item);
|
||||
return { ...item, id: state.activity.length, is_read: false };
|
||||
@@ -163,6 +169,8 @@ async function ticks(n: number): Promise<void> {
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
state.runs.clear();
|
||||
state.recoveries.clear();
|
||||
state.failFinalizeFor = null;
|
||||
state.activity.length = 0;
|
||||
state.settings = { health_gate_enabled: '1', health_gate_window_seconds: '30' };
|
||||
state.listContainers.mockReset();
|
||||
@@ -181,7 +189,7 @@ afterEach(() => {
|
||||
|
||||
describe('HealthGateService verdicts', () => {
|
||||
it('passes at the window end when containers stay running', async () => {
|
||||
const id = svc().beginStack(0, 'web', 'update', 'tester');
|
||||
const id = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
expect(id).toBeTruthy();
|
||||
await ticks(3); // 15s: still observing
|
||||
expect(latest().status).toBe('observing');
|
||||
@@ -191,7 +199,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('fails fast when a container exits', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1); // baseline
|
||||
setContainers([{ id: 'aaa', name: 'web-app-1', state: 'exited', exitCode: 1, restartPolicy: 'unless-stopped' }]);
|
||||
await ticks(1);
|
||||
@@ -207,7 +215,7 @@ describe('HealthGateService verdicts', () => {
|
||||
{ id: 'app', name: 'web-app-1', service: 'app', state: 'running', restartPolicy: 'unless-stopped' },
|
||||
{ id: 'job', name: 'web-migrate-1', service: 'migrate', state: 'running', restartPolicy: 'no' },
|
||||
]);
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([
|
||||
{ id: 'app', name: 'web-app-1', service: 'app', state: 'running', restartPolicy: 'unless-stopped' },
|
||||
@@ -229,7 +237,7 @@ describe('HealthGateService verdicts', () => {
|
||||
state: 'running', restartPolicy: 'no',
|
||||
},
|
||||
]);
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([
|
||||
{
|
||||
@@ -248,7 +256,7 @@ describe('HealthGateService verdicts', () => {
|
||||
{ id: 'app', name: 'web-app-1', service: 'app', state: 'running', restartPolicy: 'unless-stopped' },
|
||||
{ id: 'job', name: 'web-migrate-1', service: 'migrate', state: 'running', restartPolicy: 'no' },
|
||||
]);
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([
|
||||
{ id: 'app', name: 'web-app-1', service: 'app', state: 'running', restartPolicy: 'unless-stopped' },
|
||||
@@ -269,7 +277,7 @@ describe('HealthGateService verdicts', () => {
|
||||
{ id: 'app', name: 'web-app-1', service: 'app', state: 'running', restartPolicy: 'unless-stopped' },
|
||||
{ id: 'job', name: 'web-migrate-1', service: 'migrate', state: 'running', restartPolicy: 'no' },
|
||||
]);
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([
|
||||
{ id: 'app', name: 'web-app-1', service: 'app', state: 'running', restartPolicy: 'unless-stopped' },
|
||||
@@ -290,7 +298,7 @@ describe('HealthGateService verdicts', () => {
|
||||
{ id: 'app', name: 'web-app-1', service: 'app', state: 'running', restartPolicy: 'unless-stopped' },
|
||||
{ id: 'job', name: 'web-migrate-1', service: 'migrate', state: 'running', restartPolicy: 'no' },
|
||||
]);
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([
|
||||
{
|
||||
@@ -310,7 +318,7 @@ describe('HealthGateService verdicts', () => {
|
||||
{ id: 'app', name: 'web-app-1', service: 'app', state: 'running', restartPolicy: 'unless-stopped' },
|
||||
{ id: 'job', name: 'web-migrate-1', service: 'migrate', state: 'running', restartPolicy: 'no' },
|
||||
]);
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([
|
||||
{
|
||||
@@ -327,7 +335,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('fails when exit 0 has unless-stopped restart policy', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([{ id: 'aaa', name: 'web-app-1', state: 'exited', exitCode: 0, restartPolicy: 'unless-stopped' }]);
|
||||
await ticks(1);
|
||||
@@ -336,7 +344,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('fails when exit 0 has always restart policy', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([{ id: 'aaa', name: 'web-app-1', state: 'exited', exitCode: 0, restartPolicy: 'always' }]);
|
||||
await ticks(1);
|
||||
@@ -345,7 +353,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('fails closed when exit code is null on an exited container', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([{ id: 'aaa', name: 'web-app-1', state: 'exited', exitCode: null, restartPolicy: 'no' }]);
|
||||
await ticks(1);
|
||||
@@ -354,7 +362,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('fails when a one-shot exits non-zero', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([{ id: 'aaa', name: 'web-app-1', state: 'exited', exitCode: 1, restartPolicy: 'no' }]);
|
||||
await ticks(1);
|
||||
@@ -363,7 +371,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('fails fast when a healthcheck reports unhealthy', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([{ id: 'aaa', name: 'web-app-1', health: 'unhealthy' }]);
|
||||
await ticks(1);
|
||||
@@ -372,7 +380,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('detects a restart loop via container replacement (new id)', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([{ id: 'bbb', name: 'web-app-1' }]);
|
||||
await ticks(1); // restart 1 observed; carried as new baseline
|
||||
@@ -388,7 +396,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('detects a restart loop via RestartCount and StartedAt movement', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([{ id: 'aaa', name: 'web-app-1', restartCount: 1 }]);
|
||||
await ticks(1);
|
||||
@@ -403,7 +411,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('tolerates a one-poll disappearance but fails on two consecutive misses', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1); // baseline
|
||||
setContainers([]); // one missed poll: tolerated
|
||||
await ticks(1);
|
||||
@@ -411,7 +419,7 @@ describe('HealthGateService verdicts', () => {
|
||||
await ticks(5); // through the 30s window
|
||||
expect(latest().status).toBe('passed');
|
||||
|
||||
const second = svc().beginStack(0, 'web', 'update', 'tester')!;
|
||||
const second = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null })!;
|
||||
await ticks(1);
|
||||
setContainers([]);
|
||||
await ticks(2); // two consecutive misses: disappeared
|
||||
@@ -421,7 +429,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('fails when a container is stuck restarting across consecutive polls', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
setContainers([{ id: 'aaa', name: 'web-app-1', state: 'restarting' }]);
|
||||
await ticks(2);
|
||||
@@ -430,7 +438,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('goes unknown after three consecutive docker errors', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(1);
|
||||
state.listContainers.mockRejectedValue(new Error('socket gone'));
|
||||
await ticks(3);
|
||||
@@ -439,7 +447,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('resolves unknown when every docker observe hangs', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
// A wedged socket never settles. The per-observe timeout turns each poll
|
||||
// into an error, and three in a row finalize the gate unknown instead of
|
||||
// observing forever on a pending promise.
|
||||
@@ -452,7 +460,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('recovers from a transient observe timeout instead of finalizing', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
// One observe wedges and times out (a single strike), then the socket
|
||||
// recovers; the gate must keep observing, not give up at one error.
|
||||
state.listContainers.mockImplementationOnce(() => new Promise<never>(() => {}));
|
||||
@@ -465,7 +473,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('runs polls single-flight: no second observe until the first settles', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
let release: (value: Array<{ Id: string; Names: string[]; State: string }>) => void = () => {};
|
||||
state.listContainers.mockImplementationOnce(() => new Promise(resolve => { release = resolve; }));
|
||||
// Advance past a second poll interval while the first observe is still
|
||||
@@ -480,7 +488,7 @@ describe('HealthGateService verdicts', () => {
|
||||
});
|
||||
|
||||
it('ends unknown when a healthcheck is still starting at the window end', async () => {
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
setContainers([{ id: 'aaa', name: 'web-app-1', health: 'starting' }]);
|
||||
await ticks(7);
|
||||
expect(latest().status).toBe('unknown');
|
||||
@@ -489,7 +497,7 @@ describe('HealthGateService verdicts', () => {
|
||||
|
||||
it('goes unknown when no containers ever appear', async () => {
|
||||
setContainers([]);
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
await ticks(4);
|
||||
expect(latest().status).toBe('unknown');
|
||||
expect(latest().reason).toContain('no containers');
|
||||
@@ -501,7 +509,7 @@ describe('HealthGateService lifecycle', () => {
|
||||
// A poll is mid-await on Docker when a newer update supersedes the gate;
|
||||
// when the await resolves with healthy containers, the superseded run
|
||||
// must keep its terminal unknown verdict.
|
||||
const first = svc().beginStack(0, 'web', 'update', 'tester')!;
|
||||
const first = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null })!;
|
||||
await ticks(2); // baseline established, healthy
|
||||
|
||||
let releasePoll: (value: Array<{ Id: string; Names: string[]; State: string }>) => void = () => {};
|
||||
@@ -510,7 +518,7 @@ describe('HealthGateService lifecycle', () => {
|
||||
);
|
||||
const straddlingPoll = vi.advanceTimersByTimeAsync(5_000); // poll now awaiting Docker
|
||||
|
||||
const second = svc().beginStack(0, 'web', 'update', 'tester')!;
|
||||
const second = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null })!;
|
||||
expect(svc().getReport(0, 'web', first).status).toBe('unknown');
|
||||
|
||||
releasePoll([{ Id: 'aaa', Names: ['/web-app-1'], State: 'running' }]);
|
||||
@@ -525,10 +533,10 @@ describe('HealthGateService lifecycle', () => {
|
||||
});
|
||||
|
||||
it('supersede finalizes the old run as unknown, clears its timer, and getRun still resolves it', async () => {
|
||||
const first = svc().beginStack(0, 'web', 'update', 'tester')!;
|
||||
const first = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null })!;
|
||||
await ticks(1);
|
||||
const timersBefore = vi.getTimerCount();
|
||||
const second = svc().beginStack(0, 'web', 'update', 'tester')!;
|
||||
const second = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null })!;
|
||||
expect(vi.getTimerCount()).toBe(timersBefore); // old interval cleared, new one added
|
||||
|
||||
const superseded = svc().getReport(0, 'web', first);
|
||||
@@ -552,9 +560,145 @@ describe('HealthGateService lifecycle', () => {
|
||||
expect(state.runs.get('stale')!.reason).toContain('restarted');
|
||||
});
|
||||
|
||||
it('start() finalizes every interrupted run even when one of them is unreadable', () => {
|
||||
state.runs.set('bad', {
|
||||
id: 'bad', node_id: 0, stack_name: 'web', trigger_action: 'update', status: 'observing',
|
||||
reason: null, window_seconds: 30, containers_json: '[]', started_at: 1, ended_at: null, created_by: null,
|
||||
target_scope: 'stack', service_name: null, failure_source: null,
|
||||
});
|
||||
state.runs.set('good', {
|
||||
id: 'good', node_id: 0, stack_name: 'other', trigger_action: 'recovery', status: 'observing',
|
||||
reason: null, window_seconds: 30, containers_json: '[]', started_at: 1, ended_at: null, created_by: null,
|
||||
target_scope: 'stack', service_name: null, failure_source: null,
|
||||
});
|
||||
// One row that cannot be written must not cost every later row its verdict.
|
||||
// A bulk sweep is what this replaced, and it told the model about none of
|
||||
// them; finalizing per row is only better if one bad row stays contained.
|
||||
state.failFinalizeFor = 'bad';
|
||||
|
||||
svc().start();
|
||||
|
||||
expect(state.runs.get('bad')!.status).toBe('observing');
|
||||
expect(state.runs.get('good')!.status).toBe('unknown');
|
||||
});
|
||||
});
|
||||
|
||||
describe('HealthGateService recovery reservations', () => {
|
||||
/** A reserved, committed recovery run as `compensateWithCandidate` leaves it. */
|
||||
function reserve(recoveryRef = 'rec-1', stackName = 'web') {
|
||||
state.recoveries.set(recoveryRef, { id: recoveryRef, health_gate_id: null });
|
||||
return svc().reserveRecoveryRun({
|
||||
recoveryRef,
|
||||
nodeId: 0,
|
||||
stackName,
|
||||
deployedGenerationId: 'gen-1',
|
||||
actor: 'system:recovery',
|
||||
});
|
||||
}
|
||||
|
||||
it('writes the run and links it to the recovery generation', () => {
|
||||
const result = reserve();
|
||||
expect(result.outcome).toBe('reserved');
|
||||
expect(result.runId).toBeTruthy();
|
||||
|
||||
const run = state.runs.get(result.runId!)!;
|
||||
expect(run.trigger_action).toBe('recovery');
|
||||
expect(run.status).toBe('observing');
|
||||
// The generation is on the row, so the verdict is attributed to what this
|
||||
// run was recorded as observing rather than to whatever is current later.
|
||||
expect(run.deployed_generation_id).toBe('gen-1');
|
||||
expect(state.recoveries.get('rec-1')!.health_gate_id).toBe(result.runId);
|
||||
// Reserving is a write, not an observation: no timer yet.
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('reuses the run a replayed recovery already owns', () => {
|
||||
const first = reserve();
|
||||
const second = svc().reserveRecoveryRun({
|
||||
recoveryRef: 'rec-1',
|
||||
nodeId: 0,
|
||||
stackName: 'web',
|
||||
deployedGenerationId: 'gen-1',
|
||||
actor: 'system:recovery',
|
||||
});
|
||||
expect(second.outcome).toBe('replayed');
|
||||
expect(second.runId).toBe(first.runId);
|
||||
expect(state.runs.size).toBe(1);
|
||||
});
|
||||
|
||||
it('reserves nothing when the gate is disabled', () => {
|
||||
state.settings.health_gate_enabled = '0';
|
||||
const result = reserve();
|
||||
expect(result).toEqual({ outcome: 'disabled', runId: null });
|
||||
expect(state.runs.size).toBe(0);
|
||||
});
|
||||
|
||||
it('arms a reserved run without inserting a second one, and is idempotent', async () => {
|
||||
const { runId } = reserve();
|
||||
svc().armReservedRun(runId!, 0, 'web');
|
||||
expect(state.runs.size).toBe(1);
|
||||
|
||||
// Arming the run that is already the active gate must not supersede it.
|
||||
svc().armReservedRun(runId!, 0, 'web');
|
||||
expect(state.runs.size).toBe(1);
|
||||
expect(state.runs.get(runId!)!.status).toBe('observing');
|
||||
|
||||
await ticks(7);
|
||||
expect(state.runs.get(runId!)!.status).toBe('passed');
|
||||
});
|
||||
|
||||
it('supersedes a conflicting stack gate rather than observing twice', async () => {
|
||||
const older = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
const { runId } = reserve();
|
||||
svc().armReservedRun(runId!, 0, 'web');
|
||||
|
||||
expect(state.runs.get(older!)!.status).toBe('unknown');
|
||||
expect(state.runs.get(older!)!.reason).toContain('superseded');
|
||||
await ticks(7);
|
||||
expect(state.runs.get(runId!)!.status).toBe('passed');
|
||||
});
|
||||
|
||||
it('refuses to arm a run that is not a reserved stack recovery', () => {
|
||||
const { runId } = reserve();
|
||||
state.runs.get(runId!)!.trigger_action = 'update';
|
||||
expect(() => svc().armReservedRun(runId!, 0, 'web')).toThrow(/reserved stack recovery/);
|
||||
|
||||
expect(() => svc().armReservedRun('no-such-run', 0, 'web')).toThrow(/not found/);
|
||||
});
|
||||
|
||||
it('refuses to arm anything once the service is stopped', () => {
|
||||
const { runId } = reserve();
|
||||
svc().stop();
|
||||
expect(() => svc().armReservedRun(runId!, 0, 'web')).toThrow(/not started/);
|
||||
svc().start();
|
||||
});
|
||||
|
||||
it('writes off a reservation nothing could arm', () => {
|
||||
const { runId } = reserve();
|
||||
svc().abandonReservedRun(runId!, 0, 'web', 'could not arm: too many concurrent observations');
|
||||
|
||||
const run = state.runs.get(runId!)!;
|
||||
expect(run.status).toBe('unknown');
|
||||
expect(run.reason).toContain('could not arm');
|
||||
// Writing it off twice must not reopen or rewrite it.
|
||||
svc().abandonReservedRun(runId!, 0, 'web', 'second attempt');
|
||||
expect(state.runs.get(runId!)!.reason).toContain('could not arm');
|
||||
});
|
||||
|
||||
it('never arms a reservation that outlived its process', () => {
|
||||
const { runId } = reserve();
|
||||
// A restart: the row is still observing, and nothing in memory owns it.
|
||||
svc().stop();
|
||||
svc().start();
|
||||
|
||||
expect(state.runs.get(runId!)!.status).toBe('unknown');
|
||||
expect(state.runs.get(runId!)!.reason).toContain('restarted');
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('no-ops when disabled but still records the update_started event', () => {
|
||||
state.settings.health_gate_enabled = '0';
|
||||
const id = svc().beginStack(0, 'web', 'update', 'tester');
|
||||
const id = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
expect(id).toBeNull();
|
||||
expect(state.runs.size).toBe(0);
|
||||
expect(state.activity.some(a => a.category === 'update_started')).toBe(true);
|
||||
@@ -562,24 +706,24 @@ describe('HealthGateService lifecycle', () => {
|
||||
});
|
||||
|
||||
it('records update_started for update triggers but not deploy triggers', () => {
|
||||
svc().beginStack(0, 'web', 'deploy', 'tester');
|
||||
svc().beginStack(0, 'web', 'deploy', 'tester', { deployedGenerationId: null });
|
||||
expect(state.activity.some(a => a.category === 'update_started')).toBe(false);
|
||||
svc().beginStack(0, 'web', 'update', 'tester');
|
||||
svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null });
|
||||
expect(state.activity.some(a => a.category === 'update_started')).toBe(true);
|
||||
});
|
||||
|
||||
it('refuses to begin before start() so shutdown cannot leak timers', () => {
|
||||
svc().stop();
|
||||
expect(svc().beginStack(0, 'web', 'update', 'tester')).toBeNull();
|
||||
expect(svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null })).toBeNull();
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
svc().start();
|
||||
});
|
||||
|
||||
it('persists an immediate unknown past the concurrency cap', () => {
|
||||
for (let i = 0; i < 25; i++) {
|
||||
svc().beginStack(0, `stack-${i}`, 'update', 'tester');
|
||||
svc().beginStack(0, `stack-${i}`, 'update', 'tester', { deployedGenerationId: null });
|
||||
}
|
||||
const overCap = svc().beginStack(0, 'one-too-many', 'update', 'tester')!;
|
||||
const overCap = svc().beginStack(0, 'one-too-many', 'update', 'tester', { deployedGenerationId: null })!;
|
||||
const report = svc().getReport(0, 'one-too-many', overCap);
|
||||
expect(report.status).toBe('unknown');
|
||||
expect(report.reason).toContain('concurrent');
|
||||
@@ -587,10 +731,10 @@ describe('HealthGateService lifecycle', () => {
|
||||
|
||||
it('clamps the configured window into its valid range and falls back on garbage', () => {
|
||||
state.settings.health_gate_window_seconds = '99999';
|
||||
const a = svc().beginStack(0, 'web', 'update', 'tester')!;
|
||||
const a = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null })!;
|
||||
expect(svc().getReport(0, 'web', a).windowSeconds).toBe(600);
|
||||
state.settings.health_gate_window_seconds = 'banana';
|
||||
const b = svc().beginStack(0, 'web', 'update', 'tester')!;
|
||||
const b = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null })!;
|
||||
expect(svc().getReport(0, 'web', b).windowSeconds).toBe(90);
|
||||
});
|
||||
|
||||
@@ -601,7 +745,7 @@ describe('HealthGateService lifecycle', () => {
|
||||
});
|
||||
|
||||
it('stop() finalizes in-flight gates as unknown with zero timers left', async () => {
|
||||
const id = svc().beginStack(0, 'web', 'update', 'tester')!;
|
||||
const id = svc().beginStack(0, 'web', 'update', 'tester', { deployedGenerationId: null })!;
|
||||
await ticks(1);
|
||||
svc().stop();
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
/**
|
||||
* Shared GitOps row fixtures for route and projection tests.
|
||||
*
|
||||
* Type-only imports, so this module pulls no service in at load time and stays
|
||||
* safe to import statically from a test file whose singletons are only wired up
|
||||
* once setupTestDb has run.
|
||||
*/
|
||||
import type { GitOpsApplicationRow } from '../../services/gitops/types';
|
||||
|
||||
/**
|
||||
* A minimal live Direct application row.
|
||||
*
|
||||
* Every column is spelled out because the row type mirrors the table, so a
|
||||
* partial object would not type-check and a cast would let a schema change land
|
||||
* without a compile error here. Only the identifiers vary between tests; the
|
||||
* rest is the quiet, freshly activated state a Direct attachment starts in.
|
||||
*/
|
||||
export function directApplicationFixture(id: string, stackName: string): GitOpsApplicationRow {
|
||||
const now = Date.now();
|
||||
return {
|
||||
id,
|
||||
lifecycle_key: `direct:${stackName}`,
|
||||
lifecycle_status: 'active',
|
||||
target_mode: 'direct',
|
||||
stack_name: stackName,
|
||||
blueprint_id: null,
|
||||
configured_repo_url: 'https://github.com/example/repo.git',
|
||||
repo_identity_json: '{"host":"github.com","pathname":"/example/repo.git"}',
|
||||
configured_ref: 'main',
|
||||
compose_paths_json: '["compose.yaml"]',
|
||||
context_dir: null,
|
||||
sync_env: 0,
|
||||
env_path: null,
|
||||
materialization_fingerprint: 'a'.repeat(64),
|
||||
desired_commit_sha: null,
|
||||
fetched_commit_sha: null,
|
||||
candidate_generation_id: null,
|
||||
accepted_generation_id: null,
|
||||
candidate_plan_blocked: 0,
|
||||
review_required: 0,
|
||||
artifact_set_id: null,
|
||||
latest_artifact_set_id: null,
|
||||
intent_revision_id: null,
|
||||
rollout_candidate_id: null,
|
||||
rollout_generation_id: null,
|
||||
source_acceptance_ref: null,
|
||||
placement_approval_ref: null,
|
||||
rollout_authorization_ref: null,
|
||||
legacy_combined_approval_ref: null,
|
||||
preflight_fingerprint: null,
|
||||
latest_operation_id: null,
|
||||
active_operation_id: null,
|
||||
active_operation_stage: null,
|
||||
active_operation_at: null,
|
||||
active_generation_id: null,
|
||||
pause_at: null,
|
||||
pause_reason: null,
|
||||
partial_json: null,
|
||||
failure_stage: null,
|
||||
failure_class: null,
|
||||
failure_at: null,
|
||||
retry_at: null,
|
||||
retry_count: 0,
|
||||
suspended_at: null,
|
||||
recovery_ref: null,
|
||||
recovery_phase: null,
|
||||
interruption_stage: null,
|
||||
interruption_at: null,
|
||||
interruption_operation_id: null,
|
||||
interruption_generation_id: null,
|
||||
evidence_fresh_at: null,
|
||||
evidence_limitations_json: null,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
};
|
||||
}
|
||||
@@ -257,4 +257,33 @@ describe('hubOnlyGuard', () => {
|
||||
|
||||
expect(res.body?.code).not.toBe('HUB_ONLY_ENDPOINT');
|
||||
});
|
||||
|
||||
it('rejects /api/blueprints with 403 when nodeId targets a remote node', async () => {
|
||||
const res = await request(app)
|
||||
.get('/api/blueprints')
|
||||
.set('Authorization', authHeader)
|
||||
.set('x-node-id', String(remoteNodeId));
|
||||
|
||||
expect(res.status).toBe(403);
|
||||
expect(res.body?.code).toBe('HUB_ONLY_ENDPOINT');
|
||||
});
|
||||
|
||||
it('rejects /api/node-labels with 403 when nodeId targets a remote node', async () => {
|
||||
const res = await request(app)
|
||||
.get('/api/node-labels')
|
||||
.set('Authorization', authHeader)
|
||||
.set('x-node-id', String(remoteNodeId));
|
||||
|
||||
expect(res.status).toBe(403);
|
||||
expect(res.body?.code).toBe('HUB_ONLY_ENDPOINT');
|
||||
});
|
||||
|
||||
it('does not treat /api/git-sources as hub-only', async () => {
|
||||
const res = await request(app)
|
||||
.get('/api/git-sources')
|
||||
.set('Authorization', authHeader)
|
||||
.set('x-node-id', String(remoteNodeId));
|
||||
|
||||
expect(res.body?.code).not.toBe('HUB_ONLY_ENDPOINT');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -761,7 +761,7 @@ describe('POST /api/auto-update/execute', () => {
|
||||
.mockResolvedValue([{ Id: 'c1', Image: 'nginx:latest' }] as never);
|
||||
const checkSpy = vi.spyOn(ImageUpdateService.getInstance(), 'checkImage')
|
||||
.mockResolvedValue({ hasUpdate: true, digestUpdate: true } as never);
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null });
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const gateSpy = vi.spyOn(PolicyEnforcement, 'enforcePolicyPreDeploy').mockResolvedValue({
|
||||
ok: false,
|
||||
bypassed: false,
|
||||
@@ -808,7 +808,7 @@ describe('POST /api/auto-update/execute', () => {
|
||||
.mockResolvedValue([{ Id: 'c1', Image: 'nginx:latest' }] as never);
|
||||
const checkSpy = vi.spyOn(ImageUpdateService.getInstance(), 'checkImage')
|
||||
.mockResolvedValue({ hasUpdate: true, digestUpdate: true } as never);
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null });
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const recheckSpy = vi.spyOn(ImageUpdateService.getInstance(), 'recheckStack')
|
||||
.mockImplementation(async () => {
|
||||
callOrder.push('recheckStack');
|
||||
@@ -826,7 +826,7 @@ describe('POST /api/auto-update/execute', () => {
|
||||
expect(res.status).toBe(200);
|
||||
expect(updateSpy).toHaveBeenCalledWith('auto-upd-gate', undefined, true);
|
||||
expect(recheckSpy).toHaveBeenCalledWith(nodeId, 'auto-upd-gate');
|
||||
expect(beginSpy).toHaveBeenCalledWith(nodeId, 'auto-upd-gate', 'update', `auto-update:${TEST_USERNAME}`);
|
||||
expect(beginSpy).toHaveBeenCalledWith(nodeId, 'auto-upd-gate', 'update', `auto-update:${TEST_USERNAME}`, { deployedGenerationId: null });
|
||||
expect(callOrder.indexOf('beginStack')).toBeLessThan(callOrder.indexOf('recheckStack'));
|
||||
} finally {
|
||||
containersSpy.mockRestore();
|
||||
@@ -848,7 +848,7 @@ describe('POST /api/auto-update/execute', () => {
|
||||
.mockResolvedValue([{ Id: 'c1', Image: 'nginx:1.2.3' }] as never);
|
||||
const checkSpy = vi.spyOn(ImageUpdateService.getInstance(), 'checkImage')
|
||||
.mockResolvedValue({ hasUpdate: true, digestUpdate: false, tagUpdate: true } as never);
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null });
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const recheckSpy = vi.spyOn(ImageUpdateService.getInstance(), 'recheckStack');
|
||||
const clearSpy = vi.spyOn(DatabaseService.getInstance(), 'clearStackUpdateStatus');
|
||||
try {
|
||||
@@ -883,7 +883,7 @@ describe('POST /api/auto-update/execute', () => {
|
||||
const checkSpy = vi.spyOn(ImageUpdateService.getInstance(), 'checkImage')
|
||||
.mockResolvedValueOnce({ hasUpdate: true, digestUpdate: true, tagUpdate: false } as never)
|
||||
.mockResolvedValueOnce({ hasUpdate: false, error: 'registry timeout', checkStatus: 'failed' } as never);
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null });
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const recheckSpy = vi.spyOn(ImageUpdateService.getInstance(), 'recheckStack');
|
||||
try {
|
||||
const res = await request(app)
|
||||
@@ -913,7 +913,7 @@ describe('POST /api/auto-update/execute', () => {
|
||||
.mockResolvedValue([{ Id: 'c1', Image: 'nginx:latest' }] as never);
|
||||
const checkSpy = vi.spyOn(ImageUpdateService.getInstance(), 'checkImage')
|
||||
.mockResolvedValue({ hasUpdate: true, digestUpdate: true, tagUpdate: false } as never);
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null });
|
||||
const updateSpy = vi.spyOn(ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const recheckSpy = vi.spyOn(ImageUpdateService.getInstance(), 'recheckStack')
|
||||
.mockResolvedValue({ outcome: 'still_present', warning: null } as never);
|
||||
const clearSpy = vi.spyOn(DatabaseService.getInstance(), 'clearStackUpdateStatus');
|
||||
|
||||
@@ -80,6 +80,26 @@ describe('pilot-agent-mode proxy role header parity', () => {
|
||||
expect(captured?.[PROXY_ROLE_HEADER]).toBe('deployer');
|
||||
});
|
||||
|
||||
it('strips conditional request headers on the gitops identity hop', async () => {
|
||||
captured = null;
|
||||
const res = await request(app)
|
||||
.get('/api/git-sources')
|
||||
.set('Authorization', personas.deployer.bearer)
|
||||
.set('x-node-id', String(pilotNodeId))
|
||||
// A remote answering this with 304 would let the client keep a cached
|
||||
// page the hub never re-filtered, so the revalidation question must
|
||||
// never reach the remote.
|
||||
.set('If-None-Match', 'W/"cached-upstream"')
|
||||
.set('Accept', 'application/json');
|
||||
expect(res.status).toBe(200);
|
||||
expect(captured).not.toBeNull();
|
||||
expect(captured?.['if-none-match']).toBeUndefined();
|
||||
// Unrelated headers still travel.
|
||||
expect(captured?.['accept']).toBe('application/json');
|
||||
// The answer itself must not be cacheable under any validator.
|
||||
expect(res.headers['cache-control']).toBe('no-store');
|
||||
});
|
||||
|
||||
it('overwrites a smuggled admin header with the real deployer role on pilot path', async () => {
|
||||
captured = null;
|
||||
const res = await request(app)
|
||||
|
||||
@@ -55,7 +55,7 @@ const {
|
||||
mockStartContainer: vi.fn().mockResolvedValue(undefined),
|
||||
mockStopContainer: vi.fn().mockResolvedValue(undefined),
|
||||
mockPruneSystem: vi.fn().mockResolvedValue({ success: true, reclaimedBytes: 0 }),
|
||||
mockUpdateStack: vi.fn().mockResolvedValue({ recoveryId: null }),
|
||||
mockUpdateStack: vi.fn().mockResolvedValue({ recoveryId: null, deployedGenerationId: null }),
|
||||
mockGetStacks: vi.fn().mockResolvedValue([]),
|
||||
mockGetStackContent: vi.fn().mockResolvedValue(''),
|
||||
mockGetEnvContent: vi.fn().mockResolvedValue(''),
|
||||
@@ -912,7 +912,7 @@ describe('SchedulerService - executeUpdate', () => {
|
||||
|
||||
await SchedulerService.getInstance().triggerTask(83);
|
||||
|
||||
expect(beginSpy).toHaveBeenCalledWith(1, 'web-app', 'update', 'system:scheduler');
|
||||
expect(beginSpy).toHaveBeenCalledWith(1, 'web-app', 'update', 'system:scheduler', { deployedGenerationId: null });
|
||||
expect(mockRecheckStack).toHaveBeenCalledWith(1, 'web-app');
|
||||
expect(callOrder.indexOf('beginStack')).toBeLessThan(callOrder.indexOf('recheckStack'));
|
||||
} finally {
|
||||
|
||||
@@ -150,7 +150,7 @@ describe('OrchestratorResult to HTTP mapping', () => {
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/web/services/app/restore')
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ recoveryId: 'rec-2' });
|
||||
.send({ deployedGenerationId: null, recoveryId: 'rec-2' });
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body).toMatchObject({ serviceName: 'app', healthGateId: 'hg-2', recoveryId: 'rec-2' });
|
||||
expect(mockExecute).toHaveBeenCalledTimes(1);
|
||||
|
||||
@@ -260,7 +260,7 @@ describe('POST /api/stacks/bulk execution', () => {
|
||||
});
|
||||
|
||||
it('handles update action (paid tier) by calling ComposeService.updateStack', async () => {
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null });
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const { LicenseService } = await import('../services/LicenseService');
|
||||
const tierSpy = vi.spyOn(LicenseService.getInstance(), 'getTier').mockReturnValue('paid');
|
||||
try {
|
||||
@@ -287,7 +287,7 @@ describe('POST /api/stacks/bulk execution', () => {
|
||||
policy: { id: 1, name: 'block-criticals', node_id: null, node_identity: '', stack_pattern: null, max_severity: 'HIGH', block_on_deploy: 1, block_on_severity: 1, block_on_kev: 0, block_on_fixable: 0, enabled: 1, replicated_from_control: 0, created_at: Date.now(), updated_at: Date.now() },
|
||||
violations: [{ imageRef: 'nginx:latest', severity: 'CRITICAL', criticalCount: 3, highCount: 0, kevCount: 0, fixableCount: 0, reasons: ['severity'], scanId: 1 }],
|
||||
});
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null });
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
try {
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/bulk')
|
||||
|
||||
@@ -98,7 +98,7 @@ afterAll(() => {
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
mockDeployStack.mockReset().mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockReset().mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
mockRunCommand.mockReset();
|
||||
mockRunDown.mockReset();
|
||||
mockUpdateStack.mockReset();
|
||||
@@ -128,7 +128,7 @@ function deferred<T>(): Deferred<T> {
|
||||
|
||||
describe('Stack lifecycle mutex', () => {
|
||||
it('returns 409 with stack_op_in_progress when a deploy is already running', async () => {
|
||||
const gate = deferred<{ recoveryId: string | null }>();
|
||||
const gate = deferred<{ recoveryId: string | null; deployedGenerationId: string | null }>();
|
||||
mockDeployStack.mockImplementationOnce(() => gate.promise);
|
||||
|
||||
const first = request(app)
|
||||
@@ -152,20 +152,20 @@ describe('Stack lifecycle mutex', () => {
|
||||
expect(second.body.error).toMatch(/already deploying/i);
|
||||
expect(typeof second.body.inProgress.startedAt).toBe('number');
|
||||
|
||||
gate.resolve({ recoveryId: null });
|
||||
gate.resolve({ recoveryId: null, deployedGenerationId: null });
|
||||
const firstRes = await first;
|
||||
expect(firstRes.status).toBe(200);
|
||||
});
|
||||
|
||||
it('releases the lock after a successful deploy so the next request acquires', async () => {
|
||||
mockDeployStack.mockResolvedValueOnce({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValueOnce({ recoveryId: null, deployedGenerationId: null });
|
||||
const first = await request(app)
|
||||
.post('/api/stacks/web/deploy')
|
||||
.set('Cookie', authCookie)
|
||||
.send({ skip_scan: true });
|
||||
expect(first.status).toBe(200);
|
||||
|
||||
mockDeployStack.mockResolvedValueOnce({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValueOnce({ recoveryId: null, deployedGenerationId: null });
|
||||
const second = await request(app)
|
||||
.post('/api/stacks/web/deploy')
|
||||
.set('Cookie', authCookie)
|
||||
@@ -181,7 +181,7 @@ describe('Stack lifecycle mutex', () => {
|
||||
.send({ skip_scan: true });
|
||||
expect(first.status).toBe(500);
|
||||
|
||||
mockDeployStack.mockResolvedValueOnce({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValueOnce({ recoveryId: null, deployedGenerationId: null });
|
||||
const second = await request(app)
|
||||
.post('/api/stacks/web/deploy')
|
||||
.set('Cookie', authCookie)
|
||||
@@ -190,7 +190,7 @@ describe('Stack lifecycle mutex', () => {
|
||||
});
|
||||
|
||||
it('blocks restart while a deploy is in flight on the same stack', async () => {
|
||||
const gate = deferred<{ recoveryId: string | null }>();
|
||||
const gate = deferred<{ recoveryId: string | null; deployedGenerationId: string | null }>();
|
||||
mockDeployStack.mockImplementationOnce(() => gate.promise);
|
||||
|
||||
const deploy = request(app)
|
||||
@@ -207,12 +207,12 @@ describe('Stack lifecycle mutex', () => {
|
||||
expect(restart.body.code).toBe('stack_op_in_progress');
|
||||
expect(restart.body.inProgress.action).toBe('deploy');
|
||||
|
||||
gate.resolve({ recoveryId: null });
|
||||
gate.resolve({ recoveryId: null, deployedGenerationId: null });
|
||||
await deploy;
|
||||
});
|
||||
|
||||
it('allows concurrent ops on different stacks', async () => {
|
||||
const gate = deferred<{ recoveryId: string | null }>();
|
||||
const gate = deferred<{ recoveryId: string | null; deployedGenerationId: string | null }>();
|
||||
mockDeployStack.mockImplementation(() => gate.promise);
|
||||
|
||||
const webDeploy = request(app)
|
||||
@@ -229,7 +229,7 @@ describe('Stack lifecycle mutex', () => {
|
||||
.then(r => r);
|
||||
await vi.waitFor(() => expect(mockDeployStack).toHaveBeenCalledTimes(2));
|
||||
|
||||
gate.resolve({ recoveryId: null });
|
||||
gate.resolve({ recoveryId: null, deployedGenerationId: null });
|
||||
const [webRes, apiRes] = await Promise.all([webDeploy, apiDeploy]);
|
||||
expect(webRes.status).toBe(200);
|
||||
expect(apiRes.status).toBe(200);
|
||||
|
||||
@@ -164,7 +164,7 @@ describe('self stack lifecycle refusal', () => {
|
||||
});
|
||||
|
||||
it('allows update on a non-self stack', async () => {
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null });
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/web/update')
|
||||
.set('Cookie', authCookie);
|
||||
@@ -177,7 +177,7 @@ describe('self stack lifecycle refusal', () => {
|
||||
describe('POST /api/stacks/bulk self stack skip', () => {
|
||||
beforeEach(() => {
|
||||
stubSelfProject('sencho');
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null });
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
mockGetContainersByStack.mockResolvedValue([{ Id: 'c1' }]);
|
||||
});
|
||||
|
||||
|
||||
@@ -35,7 +35,7 @@ function spec(name: string): EffectiveServiceSpec {
|
||||
|
||||
beforeEach(() => {
|
||||
state.updateStack.mockReset();
|
||||
state.updateStack.mockResolvedValue({ recoveryId: null });
|
||||
state.updateStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
state.model = null;
|
||||
});
|
||||
|
||||
@@ -85,7 +85,7 @@ describe('StackUpdateOrchestrator stack branch', () => {
|
||||
{ nodeId: 0, stackName: 'web', target: { scope: 'stack' }, trigger: 'manual', actor: 'tester' },
|
||||
{ atomic: true, terminalWs: null },
|
||||
);
|
||||
expect(result).toEqual({ kind: 'stack_compose_done', recoveryId: null });
|
||||
expect(result).toEqual({ kind: 'stack_compose_done', recoveryId: null, deployedGenerationId: null });
|
||||
expect(state.updateStack).toHaveBeenCalledWith('web', undefined, true);
|
||||
// recoveryId is forwarded from ComposeService.updateStack
|
||||
});
|
||||
|
||||
@@ -118,7 +118,7 @@ beforeEach(() => {
|
||||
|
||||
mockExecute.mockImplementation(async () => {
|
||||
callOrder.push('execute');
|
||||
return { kind: 'stack_compose_done', recoveryId: null };
|
||||
return { kind: 'stack_compose_done', recoveryId: null, deployedGenerationId: null };
|
||||
});
|
||||
mockBeginStack.mockImplementation(() => {
|
||||
callOrder.push('beginStack');
|
||||
|
||||
@@ -140,7 +140,7 @@ vi.mock('fs/promises', () => ({
|
||||
rm: (p: string, opts?: unknown) => mockRm(p, opts),
|
||||
}));
|
||||
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { DatabaseService, type StackUpdateRecoveryGenerationRow } from '../services/DatabaseService';
|
||||
import { StackUpdateRecoveryService } from '../services/StackUpdateRecoveryService';
|
||||
|
||||
describe('StackUpdateRecoveryService', () => {
|
||||
@@ -172,8 +172,12 @@ describe('StackUpdateRecoveryService', () => {
|
||||
mockTag.mockImplementation(async () => { order.push('tag'); });
|
||||
mockWriteFile.mockImplementation(async () => { order.push('write'); });
|
||||
|
||||
let inserted: StackUpdateRecoveryGenerationRow | undefined;
|
||||
const spyInsert = vi.spyOn(DatabaseService.prototype, 'insertStackUpdateRecoveryGeneration')
|
||||
.mockImplementation(() => { order.push('insert'); });
|
||||
.mockImplementation((row) => {
|
||||
inserted = row;
|
||||
order.push('insert');
|
||||
});
|
||||
vi.spyOn(DatabaseService.prototype, 'getGlobalSettings').mockReturnValue({});
|
||||
|
||||
await StackUpdateRecoveryService.getInstance().captureCandidate({
|
||||
@@ -185,6 +189,11 @@ describe('StackUpdateRecoveryService', () => {
|
||||
expect(order.indexOf('validate')).toBeLessThan(order.indexOf('tag'));
|
||||
expect(order.indexOf('tag')).toBeLessThan(order.indexOf('write'));
|
||||
expect(order.indexOf('write')).toBeLessThan(order.indexOf('insert'));
|
||||
expect(inserted).toMatchObject({
|
||||
gitops_generation_id: null,
|
||||
gitops_artifact_set_id: null,
|
||||
gitops_source_acceptance_ref: null,
|
||||
});
|
||||
spyInsert.mockRestore();
|
||||
});
|
||||
|
||||
|
||||
@@ -21,6 +21,15 @@ describe('classifyStackApiPath', () => {
|
||||
expect(classifyStackApiPath('GET', '/stacks/web/git-source')).toEqual({
|
||||
kind: 'named-stack', stackName: 'web', action: 'stack:read',
|
||||
});
|
||||
// Load-bearing: an unclassified named-stack path is refused before the
|
||||
// admin bypass, so a missing rule here 403s this route on every remote
|
||||
// node for every caller.
|
||||
expect(classifyStackApiPath('GET', '/stacks/web/git-source/history')).toEqual({
|
||||
kind: 'named-stack', stackName: 'web', action: 'stack:read',
|
||||
});
|
||||
expect(classifyStackApiPath('GET', '/stacks/web/git-source/manifest')).toEqual({
|
||||
kind: 'named-stack', stackName: 'web', action: 'stack:read',
|
||||
});
|
||||
expect(classifyStackApiPath('POST', '/stacks/web/drift/recheck')).toEqual({
|
||||
kind: 'named-stack', stackName: 'web', action: 'stack:read',
|
||||
});
|
||||
|
||||
@@ -142,10 +142,10 @@ afterAll(() => {
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
mockDeployStack.mockReset().mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockReset().mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
mockRunCommand.mockReset();
|
||||
mockRunDown.mockReset();
|
||||
mockUpdateStack.mockReset().mockResolvedValue({ recoveryId: null });
|
||||
mockUpdateStack.mockReset().mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
mockGetContainersByStack.mockReset();
|
||||
mockRestartContainer.mockReset();
|
||||
mockStopContainer.mockReset();
|
||||
@@ -234,7 +234,7 @@ describe('deploy_failure notification on /deploy error', () => {
|
||||
});
|
||||
|
||||
it('uses trusted proxy tier headers for remote atomic deploys', async () => {
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const token = jwt.sign({ scope: 'node_proxy' }, TEST_JWT_SECRET, { expiresIn: '1m' });
|
||||
|
||||
const res = await request(app)
|
||||
@@ -260,18 +260,18 @@ describe('health gate begin call sites', () => {
|
||||
});
|
||||
|
||||
it('begins a gate after a manual deploy and returns its id', async () => {
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/myapp/deploy')
|
||||
.set('Cookie', authCookie)
|
||||
.send({ skip_scan: true });
|
||||
expect(res.status).toBe(200);
|
||||
expect(beginSpy).toHaveBeenCalledWith(expect.any(Number), 'myapp', 'deploy', 'testadmin');
|
||||
expect(beginSpy).toHaveBeenCalledWith(expect.any(Number), 'myapp', 'deploy', 'testadmin', { deployedGenerationId: null });
|
||||
expect(res.body.healthGateId).toBe('gate-123');
|
||||
});
|
||||
|
||||
it('links the deploy recovery generation to the observing gate', async () => {
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: 'rec-deploy' });
|
||||
mockDeployStack.mockResolvedValue({ deployedGenerationId: null, recoveryId: 'rec-deploy' });
|
||||
const { StackUpdateRecoveryService } = await import('../services/StackUpdateRecoveryService');
|
||||
const linkSpy = vi.spyOn(StackUpdateRecoveryService.getInstance(), 'linkGateOrRetain');
|
||||
const res = await request(app)
|
||||
@@ -284,25 +284,25 @@ describe('health gate begin call sites', () => {
|
||||
});
|
||||
|
||||
it('begins a gate after a manual update and returns its id', async () => {
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null });
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/myapp/update')
|
||||
.set('Cookie', authCookie)
|
||||
.send({ skip_scan: true });
|
||||
expect(res.status).toBe(200);
|
||||
expect(beginSpy).toHaveBeenCalledWith(expect.any(Number), 'myapp', 'update', 'testadmin');
|
||||
expect(beginSpy).toHaveBeenCalledWith(expect.any(Number), 'myapp', 'update', 'testadmin', { deployedGenerationId: null });
|
||||
expect(res.body.healthGateId).toBe('gate-123');
|
||||
});
|
||||
|
||||
it('begins a gate per stack in a bulk update and carries ids in the results', async () => {
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null });
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/bulk')
|
||||
.set('Cookie', authCookie)
|
||||
.send({ action: 'update', stackNames: ['myapp', 'webapp'] });
|
||||
expect(res.status).toBe(200);
|
||||
expect(beginSpy).toHaveBeenCalledWith(expect.any(Number), 'myapp', 'update', 'testadmin');
|
||||
expect(beginSpy).toHaveBeenCalledWith(expect.any(Number), 'webapp', 'update', 'testadmin');
|
||||
expect(beginSpy).toHaveBeenCalledWith(expect.any(Number), 'myapp', 'update', 'testadmin', { deployedGenerationId: null });
|
||||
expect(beginSpy).toHaveBeenCalledWith(expect.any(Number), 'webapp', 'update', 'testadmin', { deployedGenerationId: null });
|
||||
const items = res.body.results as Array<{ stackName: string; ok: boolean; healthGateId?: string | null }>;
|
||||
expect(items).toHaveLength(2);
|
||||
for (const item of items) {
|
||||
@@ -318,7 +318,7 @@ describe('health gate begin call sites', () => {
|
||||
});
|
||||
|
||||
it('never begins a gate for the rollback recovery path', async () => {
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/myapp/rollback')
|
||||
.set('Cookie', authCookie);
|
||||
@@ -416,7 +416,7 @@ describe('failure classification on deploy/update error responses', () => {
|
||||
|
||||
describe('post-deploy scan opt-out', () => {
|
||||
it('does not trigger a post-deploy scan when skip_scan is true', async () => {
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
|
||||
const res = await request(app)
|
||||
.post('/api/stacks/myapp/deploy')
|
||||
@@ -530,7 +530,7 @@ describe('deploy_failure notification on /update error', () => {
|
||||
});
|
||||
|
||||
it('uses trusted proxy tier headers for remote atomic updates', async () => {
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null });
|
||||
mockUpdateStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const token = jwt.sign({ scope: 'node_proxy' }, TEST_JWT_SECRET, { expiresIn: '1m' });
|
||||
|
||||
const res = await request(app)
|
||||
|
||||
@@ -123,7 +123,7 @@ beforeEach(() => {
|
||||
envWritten: false,
|
||||
warnings: [],
|
||||
});
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null });
|
||||
mockDeployStack.mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
mockIsTrivyAvailable.mockReturnValue(true);
|
||||
mockListContainers.mockResolvedValue([{ Image: 'nginx:latest' }]);
|
||||
mockGetImageDigest.mockResolvedValue(null);
|
||||
|
||||
@@ -461,14 +461,14 @@ describe('WebhookService.execute: health gate begin call sites', () => {
|
||||
const { HealthGateService } = await import('../services/HealthGateService');
|
||||
vi.spyOn(policyGate, 'assertPolicyGateAllows').mockResolvedValue(undefined);
|
||||
vi.spyOn(fs.FileSystemService.prototype, 'getStacks').mockResolvedValue([stack]);
|
||||
vi.spyOn(compose.ComposeService.prototype, 'deployStack').mockResolvedValue({ recoveryId: 'rec-hook' });
|
||||
vi.spyOn(compose.ComposeService.prototype, 'deployStack').mockResolvedValue({ deployedGenerationId: null, recoveryId: 'rec-hook' });
|
||||
const beginSpy = vi.spyOn(HealthGateService.getInstance(), 'beginStack').mockReturnValue('gate-hook');
|
||||
const { StackUpdateRecoveryService } = await import('../services/StackUpdateRecoveryService');
|
||||
const linkSpy = vi.spyOn(StackUpdateRecoveryService.getInstance(), 'linkGateOrRetain');
|
||||
|
||||
const result = await WebhookService.getInstance().execute(webhook, 'deploy', 'test', true);
|
||||
expect(result.success).toBe(true);
|
||||
expect(beginSpy).toHaveBeenCalledWith(nodeId, stack, 'deploy', 'system:webhook');
|
||||
expect(beginSpy).toHaveBeenCalledWith(nodeId, stack, 'deploy', 'system:webhook', { deployedGenerationId: null });
|
||||
expect(linkSpy).toHaveBeenCalledWith('rec-hook', 'gate-hook');
|
||||
});
|
||||
|
||||
@@ -484,11 +484,11 @@ describe('WebhookService.execute: health gate begin call sites', () => {
|
||||
const { HealthGateService } = await import('../services/HealthGateService');
|
||||
vi.spyOn(policyGate, 'assertPolicyGateAllows').mockResolvedValue(undefined);
|
||||
vi.spyOn(fs.FileSystemService.prototype, 'getStacks').mockResolvedValue([stack]);
|
||||
vi.spyOn(compose.ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null });
|
||||
vi.spyOn(compose.ComposeService.prototype, 'updateStack').mockResolvedValue({ recoveryId: null, deployedGenerationId: null });
|
||||
const beginSpy = vi.spyOn(HealthGateService.getInstance(), 'beginStack').mockReturnValue('gate-hook');
|
||||
|
||||
const result = await WebhookService.getInstance().execute(webhook, 'pull', 'test', true);
|
||||
expect(result.success).toBe(true);
|
||||
expect(beginSpy).toHaveBeenCalledWith(nodeId, stack, 'update', 'system:webhook');
|
||||
expect(beginSpy).toHaveBeenCalledWith(nodeId, stack, 'update', 'system:webhook', { deployedGenerationId: null });
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user