gsadmin/sencho
1
0
Fork 0
mirror of https://github.com/Studio-Saelix/sencho.git synced 2026-07-26 11:49:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(fleet): verify update status before removing readiness cards (#1697)

Browse Source 
* fix(fleet): verify update status before removing readiness cards

Full-stack Apply now rechecks persisted status after the health gate starts, reloads the live preview before dropping a card, and invalidates the hub fleet aggregation so cleared updates cannot resurrect from a stale cache.

Closes #1686

* fix(fleet): align persisted update status with preview semver detection

Share digest-plus-tag detection so post-Apply sidebar status matches Fleet and Anatomy.

* fix(fleet): keep tag-only updates advisory for Compose automation

Expose digestUpdate vs tagUpdate from checkImage so scheduled and API auto-update only apply same-tag digest drift Compose can pull.

* docs: clarify scheduled auto-update applies digest drift only

Document that higher pinned tags stay advisory until Compose is changed, matching schedule and Run Now behavior.

* docs: require Compose pin edits for higher-tag advisories

Stop recommending Apply now or Update as remedies that cannot rewrite a pinned image tag.

* docs: clarify Apply now pulls pinned tags only

Align the detection-cadence bullet with digest-rebuild vs higher-tag guidance.

* fix(fleet): keep tag advisories after apply and scheduled updates

Tag-only previews were treated as cleared on Fleet reload, and scheduled/
Run Now paths wiped status without rechecking. Align post-update verification
with the manual Apply path (health gate first, recheck, no blind clear) and
block digest apply when sibling image checks failed.

* fix(fleet): clear eslint unused-arg and containers assignment
This commit is contained in:
Anso
2026-07-26 03:09:21 -04:00
committed by GitHub
parent bb7c76ba46
commit 719180f156
16 changed files with 1320 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/features/auto-update-policies.mdx
+32 -21
View File
@@ -39,8 +39,8 @@ Sencho polls your registries on a configurable schedule to detect available imag
Detection is separate from applying updates:
- **Registry detection** is this interval. It only looks for newer images and raises an "update available" notification the first time a stack goes from up to date to having an update.
- **Scheduled auto-update tasks** apply updates on their own cron schedule, independent of the detection interval. See [Scheduling auto-updates](#scheduling-auto-updates).
- **Apply now** updates a single stack on demand, regardless of either schedule.
- **Scheduled auto-update tasks** pull and recreate on their own cron schedule when the Compose-pinned tag has same-tag digest drift. A higher semver tag that is not yet written into Compose stays advisory on the readiness board and sidebar. See [Scheduling auto-updates](#scheduling-auto-updates).
- **Apply now** pulls and recreates the tags currently written in Compose (same-tag digest rebuilds). It does not rewrite a higher semver pin.
The interval is node-scoped: each node runs its own scanner on its own cadence, and across a fleet Sencho staggers the runs slightly so the nodes do not all poll at the same instant. Setting the interval requires an admin account.
@@ -68,38 +68,39 @@ Once the cause is resolved, the next check (on the interval, or via **Recheck**)
## Workflow
1. Open **Update** from the top nav strip.
2. Skim the card grid. The badge tells you the risk at a glance: `Safe · patch` is green, `Review · minor` is amber, `Blocked · major` is red, a digest-only rebuild on a non-semver tag shows the gray `Digest rebuild` badge, `Check uncertain` appears when the last preview check was incomplete or failed, and `Newer tag · edit Compose` marks a higher tag that requires editing the Compose pin (Compose pull cannot rewrite pins).
3. For a safe, Compose-actionable update (digest drift or local rebuild), click **Apply now** on the card to pull and recreate the stack immediately. **Apply now** stays disabled for tag-only advisories and for uncertain checks.
4. For a major bump, review the changelog preview and the upstream release notes. **Apply now** is disabled on the readiness board for blocked cards; to apply a major bump after review, use the stack's lifecycle **Update** action (right-click the stack in the sidebar, or open the kebab menu and choose **Update**, or click **Deploy** in the stack editor).
2. Skim the card grid. The badge tells you the risk at a glance: `Safe · patch` is green, `Review · minor` is amber, `Blocked · major` is red, a digest-only rebuild on a non-semver tag shows the gray `Digest rebuild` badge, `Check uncertain` appears when the last preview check was incomplete or failed, and `Newer tag · edit Compose` marks a higher tag that requires editing the Compose pin.
3. For a **Digest rebuild** (same Compose tag, new registry content), click **Apply now** on the card to pull and recreate the stack immediately. **Apply now** stays disabled for tag-only advisories and for uncertain checks.
4. For a higher pinned semver tag (a version diff such as `1.2.3` → `1.2.4`, including major bumps), edit the Compose `image:` reference to the next tag, then deploy the stack from the editor. **Apply now** and the lifecycle **Update** action pull the currently pinned tag only; they do not rewrite the pin.
5. For a major bump, review the changelog preview and upstream release notes before changing the pin and deploying. **Apply now** is disabled on blocked cards.
On multi-service stacks, the Updates view can also apply a single service when that service has a confirmed image update. Scheduled auto-update, webhook pull, and bulk update always refresh the full stack.
5. Use **Recheck** in the hero to force an immediate registry poll across every reachable node. A 2-minute per-node cooldown applies, and the toast tells you how many nodes were triggered, rate-limited, or failed.
On multi-service stacks, the Updates view can also apply a single service when that service has a confirmed same-tag digest update. Scheduled auto-update, webhook pull, and bulk update always refresh the full stack.
6. Use **Recheck** in the hero to force an immediate registry poll across every reachable node. A 2-minute per-node cooldown applies, and the toast tells you how many nodes were triggered, rate-limited, or failed.
**Apply now** runs the same update path as every other update trigger on a stack: Sencho takes an atomic backup of the compose and env files first, pulls the updated registry images (or rebuilds from source for services that declare `build:`, then pulls the rest), and recreates the containers. See [Controlling a running stack](/features/stack-management#controlling-a-running-stack) for exactly how a build-aware update differs from a plain pull. Once the containers are up, the [post-update health gate](/features/health-gated-updates#the-post-update-health-gate) observes them for a configurable window and records a passed, failed, or unknown verdict on the stack's timeline, the same as a manual update from the editor toolbar or sidebar.
**Apply now** runs the same update path as other Compose-pull triggers on a stack: Sencho takes an atomic backup of the compose and env files first, pulls the updated registry images for the tags already written in Compose (or rebuilds from source for services that declare `build:`, then pulls the rest), and recreates the containers. See [Controlling a running stack](/features/stack-management#controlling-a-running-stack) for exactly how a build-aware update differs from a plain pull. Once the containers are up, the [post-update health gate](/features/health-gated-updates#the-post-update-health-gate) observes them for a configurable window and records a passed, failed, or unknown verdict on the stack's timeline, the same as a manual update from the editor toolbar or sidebar. After Apply finishes, Sencho rechecks the stack's update status and keeps the readiness card when an image update is still detected or verification cannot confirm clearance.
## Risk badges
| Badge | Color | When it appears |
|-------|-------|-----------------|
| `Safe · patch` | Green (Shield icon) | Patch-level semver bump (e.g. `1.2.3` to `1.2.4`) |
| `Review · minor` | Amber (AlertTriangle icon) | Minor semver bump (e.g. `1.2.3` to `1.3.0`) |
| `Blocked · major` | Red (ShieldAlert icon) | Major semver bump (e.g. `1.2.3` to `2.0.0`). **Apply now** is disabled; the card surfaces the reason "Major version jumps require human review before applying." |
| `Digest rebuild` | Gray | Non-semver tag (e.g. `main`, `stable`) with an updated digest (tag name unchanged; content changed; see [Tags vs digests](/features/vulnerability-scanning#tags-vs-digests)) |
| `Check uncertain` | Amber | Preview check incomplete or failed; **Apply now** stays disabled until a full successful check |
| `Newer tag · edit Compose` | Amber | A higher tag exists than the Compose pin; edit Compose (do not use **Apply now**) |
| `Safe · patch` | Green (Shield icon) | Patch-level semver bump (e.g. `1.2.3` to `1.2.4`). Resolve by editing the Compose pin, then deploying. |
| `Review · minor` | Amber (AlertTriangle icon) | Minor semver bump (e.g. `1.2.3` to `1.3.0`). Resolve by editing the Compose pin, then deploying. |
| `Blocked · major` | Red (ShieldAlert icon) | Major semver bump (e.g. `1.2.3` to `2.0.0`). **Apply now** is disabled; the card surfaces the reason "Major version jumps require human review before applying." After review, edit the Compose pin, then deploy. |
| `Digest rebuild` | Gray | Non-semver tag (e.g. `main`, `stable`) with an updated digest (tag name unchanged; content changed; see [Tags vs digests](/features/vulnerability-scanning#tags-vs-digests)). **Apply now** pulls this rebuild. |
| `Check uncertain` | Amber | Preview check incomplete or failed; **Apply now** stays disabled until a full successful check. |
| `Newer tag · edit Compose` | Amber | A higher tag exists than the Compose pin; edit Compose (do not use **Apply now**). |
Blocked major bumps still surface in scheduled check runs so you stay informed, but **Apply now** stays disabled until you review them, change the Compose pin, and deploy.
A separate inline `Rebuild available` label replaces the version diff when only the digest changed (same tag, new image). The risk badge on those cards still reflects the underlying semver classification reported by the registry.
Blocked updates still surface in scheduled check runs so you stay informed, but the apply button is disabled until you review them manually.
## Per-stack control
Auto-update is opt-in per stack. A stack participates in unattended updates only when an enabled scheduled task covers it. To leave a stack out (databases, self-built images, anything pinned to a fixed tag), simply do not create a schedule for it.
- **Per-stack schedule.** Create a **Auto-update Stack** task targeting that stack alone. Only this stack is updated when the cron fires.
- **Fleet-wide schedule.** Create a **Auto-update All Stacks on Node** task targeting a node. Every stack on that node is checked and updated when new images are available. If you do not want every stack covered, create per-stack schedules instead.
- **Fleet-wide schedule.** Create a **Auto-update All Stacks on Node** task targeting a node. Every stack on that node is checked when the cron fires, and stacks with same-tag digest drift are pulled and recreated. If you do not want every stack covered, create per-stack schedules instead.
- **Stack list dot.** Image-update *detection* runs on the configured interval (every 2 hours by default) regardless of whether any schedule is configured. The sidebar dot and the readiness board still show available updates so you can decide what to do with them.
- **Manual updates are always available.** The lifecycle **Update** action on a stack applies an update on demand, independent of any scheduled task.
- **Manual updates are always available.** The lifecycle **Update** action pulls and recreates the tags currently written in Compose, independent of any scheduled task. It does not rewrite a higher-tag advisory into the Compose file.
## Cleaning up after updates
@@ -121,7 +122,9 @@ Auto-update is a first-class action in the Schedules view. To create a recurring
The task lives alongside restart, prune, snapshot, and scan tasks in the same timeline and table. Run history, notifications, and the Run Now button behave the same as for every other scheduled action. See [Scheduled Operations](/features/scheduled-operations) for details.
A scheduled run applies an update the same way **Apply now** does: an atomic backup first, then the pull or build-aware rebuild, then the [post-update health gate](/features/health-gated-updates) observing the result. A schedule never bypasses a [deploy enforcement](/features/deploy-enforcement) policy: a run that a policy would block fails with the policy violation recorded in the run history, rather than applying partway.
A scheduled run (including **Run Now**) only auto-applies when Compose already pins the tag that has new registry content: same-tag digest drift, the gray **Digest rebuild** case. Sencho then follows the same update path as **Apply now**: an atomic backup first, then the pull or build-aware rebuild, then the [post-update health gate](/features/health-gated-updates) observing the result.
When detection finds a higher semver tag while Compose still pins the older one (for example Compose says `nginx:1.2.3` and the registry also publishes `1.2.4`), the readiness board and sidebar keep showing the advisory update. The schedule does not rewrite the Compose image reference, so it skips Compose pull and recreate until you change the pin in Compose and deploy. A schedule never bypasses a [deploy enforcement](/features/deploy-enforcement) policy: a run that a policy would block fails with the policy violation recorded in the run history, rather than applying partway.
## Multi-node support
@@ -150,7 +153,7 @@ A stack that mixes registry images and `build:` services still gets a card, scor
Sencho reads changelog metadata from the registry's manifest and OCI annotations. Registries that do not publish this metadata (most private registries and many self-hosted ones) render the card without a changelog. The risk badge is still accurate because it is computed from the tag itself.
</Accordion>
<Accordion title='Apply now is disabled with a "Blocked · major" tooltip'>
The stack has a major version bump and is blocked on the readiness board by policy: major updates never auto-apply without human review. To apply after reviewing the upstream release notes, use the stack's lifecycle **Update** action from the sidebar kebab or right-click menu, or open the stack editor and click **Deploy**.
The stack has a major version bump and is blocked on the readiness board by policy: major updates never auto-apply without human review. After reviewing the upstream release notes, edit the Compose `image:` pin to the reviewed next tag, then deploy the stack from the editor. **Apply now** and lifecycle **Update** pull the currently pinned tag only; they do not bump the pin.
</Accordion>
<Accordion title='Card stays stuck on "Checking registry..."'>
The registry call is either still pending or it failed. Click **Recheck** in the hero to retry. If the stack uses private-registry credentials, confirm they are still valid in **Settings > Registries**.
@@ -159,7 +162,15 @@ A stack that mixes registry images and `build:` services still gets a card, scor
Image update detection runs on the configured interval (every 2 hours by default) on each node, and the readiness board uses the same cached status. Trigger **Recheck** to force a fresh check across every reachable node, or shorten the interval under **Settings > Automation > Image update checks**.
</Accordion>
<Accordion title='Scheduled auto-update runs are not applying to a specific stack'>
No schedule covers that stack. Open **Schedules** in the top nav, create a new **Auto-update Stack** task targeting the stack (or an **Auto-update All Stacks on Node** task on its node), and pick a cron. The next firing will include the stack, or you can trigger an immediate run from the row.
First confirm a schedule covers that stack: open **Schedules**, create or enable an **Auto-update Stack** task (or an **Auto-update All Stacks on Node** task on its node), and pick a cron. The next firing includes the stack, or you can trigger an immediate run from the row.
If a schedule already covers the stack and the run history says newer tags are available but the Compose pin is unchanged, detection found a higher tag than the image reference in Compose. Scheduled auto-update only pulls same-tag digest rebuilds; it does not bump `image:1.2.3` to `image:1.2.4` for you. Edit the Compose image pin to the next tag, then deploy.
</Accordion>
<Accordion title='Run history says newer tags are available but Compose pin unchanged'>
The registry publishes a higher pinned semver tag than the one written in Compose. The readiness board and sidebar keep that advisory visible. Scheduled auto-update leaves Compose as the source of truth, so it skips pull and recreate until the pin is updated. Edit the Compose `image:` reference to the next tag, then deploy the stack. **Apply now** and lifecycle **Update** cannot rewrite the pin.
</Accordion>
<Accordion title='Apply now finished but the readiness card still shows a newer tag'>
Compose still pins the older tag. Apply pulls and recreates that pinned tag only. To move to the next tag shown on the card, edit the Compose `image:` reference, then deploy.
</Accordion>
<Accordion title='Banner says "X of Y nodes reachable"'>
One or more nodes that are marked online in your fleet did not respond within the request timeout. Pending updates from those nodes are not shown until they come back. Check the node's status from the Fleet view and the network path between this Sencho instance and the unreachable node.
docs/openapi.yaml
+6
View File
@@ -1775,6 +1775,12 @@ paths:
Id of the post-update health gate observation started for
this update, for use with the health-gate endpoint. Null
when the health gate is disabled on the node.
recheckWarning:
type: string
description: |
Present when Compose succeeded but post-update image
verification still reports an available update or could
not fully verify clearance. Optional for older nodes.
"403":
$ref: "#/components/responses/Forbidden"
"500":