Files
sencho/backend
Anso ca90588e4e fix(gitops): bound GitOps history with a retention window (#2105)
* fix(gitops): bound GitOps history with a retention window

`gitops_history` is insert-only and the reconcile tick appends to it
continuously, so it grew for the life of an installation. A new
`gitops_history_retention_days` setting (default 30, range 1 to 365)
seeds with the other GitOps settings and prunes history from the
monitor's cleanup pass. Drained outbox rows are pruned with the history
they announced; an undrained row gets a grace period past the window
because the boot repair can still drain it from its payload, and a row
still undrained after the grace is counted, logged, and dropped so it
cannot accumulate for ever. The window is instance-local, like the
artifact retry interval beside it: the history being pruned is the one
this instance holds, and the monitor that prunes it runs here.

* fix(gitops): keep each live application's newest history row through the prune

`cleanupOldGitOpsHistory` deleted `gitops_history` purely by age, but
`latestTransitionByApplication` reads `MAX(created_at)` per application and
feeds the "last activity" fact the portfolio and application sheet render. An
application with no history inside the window, because polling is off, a source
is suspended, a node is detached, or a Blueprint is idle, lost every row and
its activity column silently went empty. The prune now exempts each live
application's newest row, so the claim the JSDoc, the PR body and the settings
helper text already make is true.

Scoped to live applications rather than to every application_id: a tombstone in
`gitops_applications` is permanent, so an unscoped exemption would retain one
history row per stack ever retired and leave exactly the growth this setting
exists to stop. A retired application's rows are now pruned in full.

Also folds in three follow-ups from the audit:

- The prune is chunked to GITOPS_PRUNE_CHUNK_ROWS rows per table per pass, so
  a first prune on a long-lived install clears its backlog over several janitor
  ticks instead of holding SQLite's single writer for the whole delete. The
  method reports historyRemaining and outboxRemaining so the debug log says
  whether a pass is still catching up.
- The outbox OR-delete is two statements, so the undrained branch uses
  idx_gitops_settled_outbox_undrained instead of scanning for the drained one.
- The retention tests assert exact counts and exact surviving row ids rather
  than toBeGreaterThanOrEqual over a shared database.

New tests pin each guard: a quiet application's newest row survives every
window, its older rows are pruned, a tombstoned application clears entirely,
an unsettled attempt is still recoverable after a prune both inside the window
and outside it, and a backlog clears in bounded passes with the remainder
reported.

Removing the newest-row exemption turns 10 of the 11 tests red; removing only
the liveness clause turns the tombstone test red.

Docs: the GitOps portfolio page describes the window and its setting, and now
states that the newest entry per application is kept, which is what the code
does.

* test(gitops): drop the unused db handles from the recovery assertions

* fix(gitops): keep a settled history row ahead of its started row

A reconcile attempt stamps both rows with the same time, and the prune broke that tie on a random id, so it could delete the settlement and leave startup recovery to reopen the attempt. Recency now follows insertion order, in the exemption and in each chunk.

Remaining counts run only while developer mode is on, drained outbox deletes use an age index, and the settings copy says the kept row is the newest one for each live application.

* fix(gitops): log the undrained rows a history prune actually deletes

A pass is capped, so the warning was reporting the whole backlog. The
history-window tests now fail if the permission, the page save, the monitor
window, or the getter fallback stops matching the code.

* fix(gitops): prune a superseded unsettled attempt past the window

An unsettled reservation survives past the retention window only while it is that live application's newest row. Creating applications keep that row; detached applications do not.

* fix(gitops): skip the history prune when the retention setting cannot be read

A failed settings read used to fall back to 30 days and delete rows the operator had chosen to keep. The getter now throws, and that pass deletes nothing. A missing or unusable stored value still uses 30.

* fix(gitops): restore the history window when the retention spec fails

The retention spec restored the seeded 30 days as its last statement, not
in a finally, so a failed assertion left the shared instance on 14. The
CI retry then read 14 and failed the fresh-install assertion, turning one
failure into a persistent red.

Everything after the first save now runs inside a try whose finally puts
the window back, so the spec leaves the instance as it found it whether it
passes or fails.

* docs(settings): list the GitOps history window in the retention reference

The Data Retention table listed every other retention window and not the
GitOps one, so an operator checking retention windows did not find it. The
portfolio page described the setting in a paragraph, which is not where
retention windows are looked up.

Adds the row with its default and range, and a line under the table saying
the window is set per Sencho instance and where the control lives.

* fix(gitops): report an unmeasured prune remainder as unknown

The prune reported a remainder of 0 whenever developer mode was off, so
the diagnostic log read "none left" for a table it had never counted. It
now reports null, and the log line says the remainder was not measured.

Three smaller corrections from the same review:

- The diagnostics allowlist omitted the history window, so a support
  bundle could not explain a timeline that had already been pruned.
- The GitOps registry entry described every control as belonging to this
  node, while the retry interval and the history window are instance
  local. The sentence now names no scope it does not have.

* fix(gitops): correct what the history window is documented to govern

The copy said the window governs notifications. It does not: the prune
deletes history rows and their outbox rows, and the bell entries those
outbox rows announce age out under the Notification log window instead. An
operator setting 7 days would have expected their bell to follow.

It also promised to keep the database small, which nothing delivers. The
prune stops old rows from being written; it never reclaims the file.

- The helper now names the record behind the GitOps timeline, points bell
  entries at the Notification log, and says the window stops history from
  growing.
- The settings reference gains the GitOps section it never had, with the
  window under it rather than under a per-node table that ends in a Save
  button the control does not use.
- The portfolio page stops claiming a shorter window never changes what an
  application is doing: a webhook redelivery reads this same record, which
  the same paragraph already said.
- A fraction below one day now clamps to a day instead of flooring to zero,
  which as a cutoff is now and would delete everything not exempt. The only
  caller passes an integer, so this is a guard, not a reachable fix.

* fix(settings): snap a typed number chip to its own step on commit

The chip clamped to min and max but never rounded, so typing 1.5 wrote
"1.5" and every integer setting rejected it at the route with a bare
"Validation failed" toast. Nothing was saved, so no data was at risk, but
the control accepted input it could not store.

Rounding to the control's step fixes that and leaves a genuinely
fractional chip alone: the Docker janitor asks for 0.5 GiB and keeps it.
Clamping still runs after rounding, so a fraction below the floor cannot
round down under it.

The settings reference also listed GitOps last under Infrastructure
while the sidebar renders it third.

* docs(gitops): replace the unsourced window figure with what the code does

The helper and both docs said a window under about 3 days can let a Git
host retry a webhook. Nothing in the code picks three days, and the
dedupe the sentence describes only applies to the direct webhook path.

The text now states the condition instead of a number: a redelivery can
run the work again once the record of its first delivery is pruned. A
provider endpoint keeps its own delivery record, pruned to a fixed depth,
so the window does not affect it, and the sentence says so.

* docs(settings): say where a remote instance's window is set

The reference explained that the window belongs to an instance rather
than a node, but not how to change the window of a remote one, which is
on that instance.
2026-10-11 14:02:03 -04:00
..