fix: distinguish failed image-update checks from "up to date" (#1470)

* fix: distinguish failed image-update checks from "up to date"

The image-update detector collapsed every failure (registry unreachable,
missing auth, rate limit, unresolved local digest) into hasUpdate:false and
dropped the captured reason, so a failed check was indistinguishable from a
current image and never raised a notification, even while a manual stack
update still pulled a newer image.

Detection now records a tri-state per stack (ok / partial / failed) with the
failure reason, exposed via a new GET /api/image-updates/detail (the boolean
GET / is unchanged so fleet aggregation is unaffected). A fully-failed check
preserves the last known has_update, so a transient outage neither erases a
real update nor flaps the notification state. The sidebar shows a muted
"couldn't check" indicator with the reason on hover, and the Update board
lists stacks whose check failed in a "could not be checked" advisory.

Detector hardening: the manifest digest lookup issues HEAD first (falling back
to GET) so it no longer draws down Docker Hub's anonymous pull-rate budget, and
local RepoDigest matching is normalized so official library/* images resolve
their digest instead of falling through to a silent "no update".

* fix: preserve confirmed updates through partial checks; tighten failure surfacing

Address review findings on the tri-state image-update detection:

- A partial check (some images errored) no longer erases a previously
  confirmed update; only a fully-ok check can lower has_update, so a single
  image's registry blip cannot drop the stack's update and re-fire the
  notification on recovery. Adds a regression test.
- The image-level catch stores getErrorMessage(e) rather than raw String(e),
  since that value surfaces verbatim in the sidebar tooltip and readiness
  advisory.
- useImageUpdates and the readiness detail fetch now log unexpected non-ok
  responses instead of silently leaving stale state.
- Remove an unused checkFailedCount derivation (the row indicator is driven by
  the checkStatus prop).
- Reword the recordStackCheckFailure docstring and the HEAD-first comment.
This commit is contained in:
Anso
2026-06-26 16:16:34 -04:00
committed by GitHub
parent e3b3c3b857
commit d9b7911f12
19 changed files with 820 additions and 38 deletions
+19
View File
@@ -45,6 +45,25 @@ The interval is node-scoped: each node runs its own scanner on its own cadence,
The readiness hero shows this instance's cadence at a glance: when it last checked, when the next check is due, and, right after a manual **Recheck**, how long the 2-minute cooldown has left.
## When a check cannot complete
A registry check can fail for reasons that have nothing to do with whether an update exists: the registry is unreachable, an authenticated registry has no stored credentials, or Docker Hub's anonymous pull-rate limit has been hit. Sencho treats these as a distinct **check failed** state instead of reporting "up to date", so a failed check is never mistaken for a current image.
Where it shows:
- **Sidebar.** A stack whose latest check could not be determined shows a muted indicator on its row. Hover it to read the reason (for example, "Registry unreachable for ghcr.io/acme/api:v1").
- **Update board.** Stacks whose check failed appear in a "could not be checked" advisory above the card grid, each with its reason, so a stack with no confirmed update is never silently absent.
A confirmed update from an earlier successful check is kept through a later failed check, so a momentary registry blip does not make a pending update vanish.
Common causes and how to clear them:
- **Private or custom registry with no credentials.** Add the registry under **Settings > Registries** so detection can authenticate the same way deploys do.
- **Docker Hub rate limit.** Anonymous pulls are capped per IP address. Signing in to Docker Hub under **Settings > Registries** raises the limit.
- **Blocked network egress.** The node must reach the registry host over HTTPS to read image manifests.
Once the cause is resolved, the next check (on the interval, or via **Recheck**) clears the failed state.
## Workflow
1. Open **Update** from the top nav strip.
+1 -1
View File
@@ -55,7 +55,7 @@ Each row gives you everything you need to read the stack at a glance, in a fixed
- **Status pill** on the left. Two uppercase letters in mono type, or a spinner while a lifecycle action is in flight. `UP` (green) means the stack is running with nothing crashed, `DN` (red) means the stack is stopped, and `PT` (amber) means the stack is partially running: at least one container is up and at least one has crashed (exited with an error, died, or is restart-looping). Hover the `PT` pill to see how many containers are running, such as `3/5 running`. A stack whose only stopped container finished cleanly (an init job that exited without error) stays `UP`.
- **Stack name** in mono type, truncated with an ellipsis when the row gets tight.
- **Label dots** to the right of the name. Up to three colored dots representing the stack's labels render here. If a stack carries more than three labels, a **+N** counter appears for the extras.
- **Update indicator**. When a stack has an image update pending, an extra colored dot appears alongside the label dots. If only a Git source update is pending (no image update), a small Git branch icon shows instead. The image-update dot takes priority when both apply.
- **Update indicator**. When a stack has an image update pending, an extra colored dot appears alongside the label dots. If the last registry check could not be determined (registry unreachable, missing credentials, rate limit), a muted "couldn't check" icon shows instead, with the reason on hover. If only a Git source update is pending (no image update), a small Git branch icon shows. Priority is update dot, then check-failed, then Git pending.
- **Hover kebab** on the right edge. Hover the row to reveal a vertical three-dot menu that opens the same actions as right-clicking the row.
<Frame>