fix(fleet): gate node update actions to admins and harden update tracking (#1272)

* fix(fleet): gate node update actions to admins and harden update tracking

Node update affordances now render only for admins, matching the admin-only
routes behind them. Previously a non-admin could open the Fleet view and see the
per-node Update button, Update all, retry, dismiss, and Recheck controls, then
get a 403 on click. Those controls are now hidden for non-admins, who still see
read-only update status.

Both update-status clear routes (per-node and bulk) now require admin, and the
bulk recheck throttles its forced "latest published version" lookup so a caller
cannot loop it to hammer the upstream registries; the response reports whether
the refresh actually ran so the UI can surface a "checked recently" note.

Completion detection no longer reports a node as Updated when it merely blips
offline and returns on the same version with an unchanged process start time.
That case stays in progress and is decided by the existing early-fail and
timeout heuristics, so a momentary network glitch is not mistaken for a
successful update. Failed and timed-out updates now emit an operator-visible
warning, and a periodic safety-net sweep bounds in-flight trackers when no
client is polling for status.

* fix(fleet): harden update completion and recheck failure handling

Refinements from review of the node self-update hardening:

- Completion signal 1 now requires a valid version, not merely a different one.
  A node whose /api/meta momentarily omits or mangles its version (online, same
  process) reported version=null, which compared unequal to the previous version
  and falsely marked the update completed. It now stays in progress and is
  decided by the early-fail/timeout heuristics.

- Terminal resolution is atomic: it re-reads the live tracker and transitions
  only if it is still in flight with the same start time, so two concurrent
  status polls cannot both warn or clobber each other's transition.

- The operator warning for a failed or timed-out update now redacts
  secret-shaped text (bearer/basic/token/password, credentialed URLs) from the
  underlying error before logging, in addition to stripping control characters.

- The Recheck button now surfaces an error toast when the request throws
  (network or auth failure), matching the existing non-ok-response path instead
  of only logging to the console.
This commit is contained in:
Anso
2026-06-01 17:27:25 -04:00
committed by GitHub
parent 7d7e0a6264
commit 0953025036
11 changed files with 592 additions and 34 deletions
+2 -2
View File
@@ -79,7 +79,7 @@ The gateway tracks each in-flight update in memory and watches the target node f
- The remote reports a *new* `version` value over its `/api/meta` endpoint.
- The remote's process `startedAt` timestamp moves forward, indicating a fresh container start.
- The remote went offline at any point during the watch window and has come back online.
- The remote went offline during the watch window and has come back online, used as a fallback only for nodes that do not report a process start timestamp. A node that simply blips offline and returns on the *same* version with an unchanged start time is treated as still updating, not done, so a momentary network glitch is never mistaken for a successful update.
- More than 15 seconds have elapsed and the remote's reported version is at or above the comparison target (the gateway's own version, or the published latest, whichever is appropriate).
When any signal fires, the row flips from **Updating** to a green **Updated** badge. The **Updated** state stays visible for 60 seconds, then auto-clears so the row settles back to **Up to date**.
@@ -100,7 +100,7 @@ Both **Failed** and **Timed out** badges expose two inline actions:
Hovering either badge reveals the underlying error message reported by the remote (or by the helper container, for the gateway itself).
The header of the sheet exposes a **Recheck** button that does three things in one click: it flushes the cached "latest published version" lookup (resolved against the GitHub Releases API, with a Docker Hub fallback), it clears every terminal `Failed` and `Timed out` badge across the fleet, and it re-fetches the version metadata from every node.
The header of the sheet exposes a **Recheck** button that does three things in one click: it flushes the cached "latest published version" lookup (resolved against the GitHub Releases API, with a Docker Hub fallback), it clears every terminal `Failed` and `Timed out` badge across the fleet, and it re-fetches the version metadata from every node. The published-version lookup is rate-limited to one refresh every couple of minutes, so clicking **Recheck** again right away reuses the recent result and shows a brief "checked recently" note rather than hitting the registry on every click.
<Note>
Update tracking is held entirely in the gateway's memory. Restarting the gateway clears all in-flight, failed, and timed-out states automatically, so a stuck row always resolves itself on the next gateway restart even without a manual **Dismiss**.