mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-07-26 11:49:16 +00:00
Anso
0daddfde00
* fix: reconcile sticky update indicators with Anatomy preview Sidebar, Updates filter, and Fleet treated retained partial/failed scanner has_update as confirmed. Keep raw state for retention/notifications, project confirmed-only to APIs, show distinct incomplete indicators, and clear sticky rows only after an authoritative-negative preview. Closes #1685 * test: align sidebar truncate E2E with failed-over-retained precedence Purple update indicators are confirmed-only; hasUpdate with a failed check correctly shows the failed trailing icon. * fix: clear confirmed update rows on authoritative-negative preview Address audit SF-1/SF-2/SF-3: observation-watermark clears for older ok+has_update rows (DB + memory gens), Fleet checkability parity with backend not_checkable, and Updates chip confirmed-only regressions. * fix: tombstone equal-generation writers on preview clear Advance the per-stack write generation when clearing at the observation watermark so a scanner reserved before preview cannot recreate the row after an authoritative-negative reconcile. * fix: clear sticky updates with digest and tag preview parity Share detection across scanner and preview, keep GET read-only with POST reconcile, gate Apply to digest and rebuild updates, and invalidate the hub fleet cache on clear. * test: set digestUpdate on auto-update checkImage mocks Scheduler and execute routes now gate Compose on digest drift; fixtures that expect an apply need digestUpdate so they exercise the update path. * fix: clear unused lint errors on sticky update branch Drop unused partial helper and fleet invalidate import; keep the CacheService inflight self-ref as let with an eslint exception so tsc stays green. * fix: use inflight holder for CacheService prefer-const Keep generation-aware ownership without a let self-reference that fights ESLint and tsc.
200 lines
20 KiB
Plaintext
200 lines
20 KiB
Plaintext
---
|
|
title: "Auto-Update Policies"
|
|
sidebarTitle: Auto-update
|
|
description: "Review pending container updates across your fleet, with risk badges, changelogs, and scheduled run times, before applying."
|
|
---
|
|
|
|
<Note>
|
|
Auto-Update is hub-only and is hidden from the nav strip when a remote node is the active selection. See [Multi-Node Management](/features/multi-node#what-top-level-views-show-when-a-remote-node-is-active).
|
|
</Note>
|
|
|
|
## Overview
|
|
|
|
Auto-Update Policies are the launchpad for every pending update across your stacks. Instead of a list of CRUD policies, the view surfaces one card per stack with an available update and tells you, at a glance, whether it is safe to apply.
|
|
|
|
Each card shows:
|
|
|
|
- The current tag and (for semver updates) the next tag, with the new version highlighted in cyan. For digest-only updates (same tag name, new registry content), the card shows the current tag with a **Rebuild available** marker instead of a version diff. See [Tags vs digests](/features/vulnerability-scanning#tags-vs-digests).
|
|
- A **risk badge** derived from the version delta: `Safe · patch` (green), `Review · minor` (amber), `Blocked · major` (red), or `Digest rebuild` (gray) for non-semver tags.
|
|
- The primary image reference, plus a count of additional services if more than one image in the stack has an update.
|
|
- A one-line changelog preview when the registry publishes one. Registries that omit changelog metadata render the card with "No changelog available from the registry yet."
|
|
- The next scheduled run for the matching auto-update task, if one exists, or "No schedule".
|
|
|
|
<Frame>
|
|
<img src="/images/auto-update/readiness-board.png" alt="Readiness board with hero counter, per-node groups, and risk badges" />
|
|
</Frame>
|
|
|
|
The hero at the top counts pending updates across every node in your fleet and tells you how many are ready to apply without human review. Its subtitle reads `X of Y ready to apply automatically across N nodes`. Stacks with a major version bump are surfaced as a separate count (`· Z blocked by major bump`) so they can be reviewed before the scheduler runs.
|
|
|
|
Cards are grouped by node, with a section header for each node that has at least one pending update. The header shows the node name, a `local` or `remote` pill, and the stack count. The local node is listed first, followed by remote nodes alphabetically. If any of your online nodes is unreachable when the page loads, a small line under the hero reads `X of Y nodes reachable. Unreachable nodes are not shown.`
|
|
|
|
## Empty state
|
|
|
|
When nothing is pending, the board renders a single Shield-icon panel with the headline "All stacks on current builds" and the sub-line "Sencho rechecks registries on the configured interval." Image update detection runs on a configurable interval (every 2 hours by default) on each node; the readiness board reflects that cached status until the next check, or until you press **Recheck**. Adjust the interval under **Settings > Automation > Image update checks**.
|
|
|
|
## Detection cadence
|
|
|
|
Sencho polls your registries on a configurable schedule to detect available image updates and raise notifications. This detection cadence is configurable under **Settings > Automation > Image update checks**: choose a fixed interval (every 15 minutes to once a day) or set a cron expression for precise scheduling (e.g. "every Monday at 03:00"). The default is every 2 hours on an interval schedule, and changing it takes effect immediately, with no restart.
|
|
|
|
Detection is separate from applying updates:
|
|
|
|
- **Registry detection** is this interval. It only looks for newer images and raises an "update available" notification the first time a stack goes from up to date to having an update.
|
|
- **Scheduled auto-update tasks** apply updates on their own cron schedule, independent of the detection interval. See [Scheduling auto-updates](#scheduling-auto-updates).
|
|
- **Apply now** updates a single stack on demand, regardless of either schedule.
|
|
|
|
The interval is node-scoped: each node runs its own scanner on its own cadence, and across a fleet Sencho staggers the runs slightly so the nodes do not all poll at the same instant. Setting the interval requires an admin account.
|
|
|
|
The readiness hero shows this instance's cadence at a glance: when it last checked, when the next check is due, and, right after a manual **Recheck**, how long the 2-minute cooldown has left.
|
|
|
|
## When a check cannot complete
|
|
|
|
A registry check can fail for reasons that have nothing to do with whether an update exists: the registry is unreachable, an authenticated registry has no stored credentials, or Docker Hub's anonymous pull-rate limit has been hit. Sencho treats these as a distinct **check failed** state instead of reporting "up to date", so a failed check is never mistaken for a current image. The recorded reason names the specific cause, such as an authentication failure, a rate limit, a missing image, or an unreachable registry, so you can tell them apart at a glance.
|
|
|
|
Where it shows:
|
|
|
|
- **Sidebar.** A stack whose latest check could not be determined shows a muted indicator on its row. Hover it to read the reason (for example, "Authentication failed for ghcr.io/acme/api:v1" or "Registry unreachable for ghcr.io/acme/api:v1").
|
|
- **Update board.** Stacks whose check failed appear in a "could not be checked" advisory above the card grid, each with its reason, so a stack with no confirmed update is never silently absent.
|
|
|
|
A confirmed update from an earlier successful check is kept through a later failed check, so a momentary registry blip does not make a pending update vanish.
|
|
|
|
Common causes and how to clear them:
|
|
|
|
- **Private or custom registry with no credentials.** Add the registry under **Settings > Registries** so detection can authenticate the same way deploys do.
|
|
- **Docker Hub rate limit.** Anonymous pulls are capped per IP address. Signing in to Docker Hub under **Settings > Registries** raises the limit.
|
|
- **Blocked network egress.** The node must reach the registry host over HTTPS to read image manifests.
|
|
|
|
Once the cause is resolved, the next check (on the interval, or via **Recheck**) clears the failed state.
|
|
|
|
## Workflow
|
|
|
|
1. Open **Update** from the top nav strip.
|
|
2. Skim the card grid. The badge tells you the risk at a glance: `Safe · patch` is green, `Review · minor` is amber, `Blocked · major` is red, a digest-only rebuild on a non-semver tag shows the gray `Digest rebuild` badge, `Check uncertain` appears when the last preview check was incomplete or failed, and `Newer tag · edit Compose` marks a higher tag that requires editing the Compose pin (Compose pull cannot rewrite pins).
|
|
3. For a safe, Compose-actionable update (digest drift or local rebuild), click **Apply now** on the card to pull and recreate the stack immediately. **Apply now** stays disabled for tag-only advisories and for uncertain checks.
|
|
4. For a major bump, review the changelog preview and the upstream release notes. **Apply now** is disabled on the readiness board for blocked cards; to apply a major bump after review, use the stack's lifecycle **Update** action (right-click the stack in the sidebar, or open the kebab menu and choose **Update**, or click **Deploy** in the stack editor).
|
|
|
|
On multi-service stacks, the Updates view can also apply a single service when that service has a confirmed image update. Scheduled auto-update, webhook pull, and bulk update always refresh the full stack.
|
|
5. Use **Recheck** in the hero to force an immediate registry poll across every reachable node. A 2-minute per-node cooldown applies, and the toast tells you how many nodes were triggered, rate-limited, or failed.
|
|
|
|
**Apply now** runs the same update path as every other update trigger on a stack: Sencho takes an atomic backup of the compose and env files first, pulls the updated registry images (or rebuilds from source for services that declare `build:`, then pulls the rest), and recreates the containers. See [Controlling a running stack](/features/stack-management#controlling-a-running-stack) for exactly how a build-aware update differs from a plain pull. Once the containers are up, the [post-update health gate](/features/health-gated-updates#the-post-update-health-gate) observes them for a configurable window and records a passed, failed, or unknown verdict on the stack's timeline, the same as a manual update from the editor toolbar or sidebar.
|
|
|
|
## Risk badges
|
|
|
|
| Badge | Color | When it appears |
|
|
|-------|-------|-----------------|
|
|
| `Safe · patch` | Green (Shield icon) | Patch-level semver bump (e.g. `1.2.3` to `1.2.4`) |
|
|
| `Review · minor` | Amber (AlertTriangle icon) | Minor semver bump (e.g. `1.2.3` to `1.3.0`) |
|
|
| `Blocked · major` | Red (ShieldAlert icon) | Major semver bump (e.g. `1.2.3` to `2.0.0`). **Apply now** is disabled; the card surfaces the reason "Major version jumps require human review before applying." |
|
|
| `Digest rebuild` | Gray | Non-semver tag (e.g. `main`, `stable`) with an updated digest (tag name unchanged; content changed; see [Tags vs digests](/features/vulnerability-scanning#tags-vs-digests)) |
|
|
| `Check uncertain` | Amber | Preview check incomplete or failed; **Apply now** stays disabled until a full successful check |
|
|
| `Newer tag · edit Compose` | Amber | A higher tag exists than the Compose pin; edit Compose (do not use **Apply now**) |
|
|
|
|
A separate inline `Rebuild available` label replaces the version diff when only the digest changed (same tag, new image). The risk badge on those cards still reflects the underlying semver classification reported by the registry.
|
|
|
|
Blocked updates still surface in scheduled check runs so you stay informed, but the apply button is disabled until you review them manually.
|
|
|
|
## Per-stack control
|
|
|
|
Auto-update is opt-in per stack. A stack participates in unattended updates only when an enabled scheduled task covers it. To leave a stack out (databases, self-built images, anything pinned to a fixed tag), simply do not create a schedule for it.
|
|
|
|
- **Per-stack schedule.** Create a **Auto-update Stack** task targeting that stack alone. Only this stack is updated when the cron fires.
|
|
- **Fleet-wide schedule.** Create a **Auto-update All Stacks on Node** task targeting a node. Every stack on that node is checked and updated when new images are available. If you do not want every stack covered, create per-stack schedules instead.
|
|
- **Stack list dot.** Image-update *detection* runs on the configured interval (every 2 hours by default) regardless of whether any schedule is configured. The sidebar dot and the readiness board still show available updates so you can decide what to do with them.
|
|
- **Manual updates are always available.** The lifecycle **Update** action on a stack applies an update on demand, independent of any scheduled task.
|
|
|
|
## Cleaning up after updates
|
|
|
|
Every update pulls a fresh image and recreates the affected containers. Once the new image is in place, the image it replaced becomes a dangling (untagged) layer that otherwise lingers on disk until you prune it by hand.
|
|
|
|
Sencho clears this for you. Under **Settings > Monitoring > Docker & Storage**, the **Prune dangling images after updates** toggle is on by default. When a stack update or a Sencho self-update finishes successfully, Sencho removes the node's dangling (untagged) image layers, including the one the update just orphaned. This is a standard dangling-image prune, so any other untagged layers already on the node are reclaimed in the same pass. Only untagged layers are touched: tagged images (including those for stopped stacks), your volumes, and your data are never removed. Changing the setting requires an admin account.
|
|
|
|
The toggle is per node. Each Sencho instance honors its own setting, so you can leave cleanup on for your hub and off for a particular remote, or the reverse. Switch the active node and open its settings to set the value for that node.
|
|
|
|
Turn it off to keep every previous image layer, for example when you roll back by re-tagging an older build. With the toggle off, reclaim space on demand from the [Resources](/features/resources) view or a scheduled prune task instead.
|
|
|
|
## Scheduling auto-updates
|
|
|
|
Auto-update is a first-class action in the Schedules view. To create a recurring check for a stack:
|
|
|
|
1. Open **Schedules** in the top nav.
|
|
2. Click **New Schedule**.
|
|
3. Set **Action** to **Auto-update Stack**, choose the target node and stack, pick a cron expression, and save.
|
|
|
|
The task lives alongside restart, prune, snapshot, and scan tasks in the same timeline and table. Run history, notifications, and the Run Now button behave the same as for every other scheduled action. See [Scheduled Operations](/features/scheduled-operations) for details.
|
|
|
|
A scheduled run applies an update the same way **Apply now** does: an atomic backup first, then the pull or build-aware rebuild, then the [post-update health gate](/features/health-gated-updates) observing the result. A schedule never bypasses a [deploy enforcement](/features/deploy-enforcement) policy: a run that a policy would block fails with the policy violation recorded in the run history, rather than applying partway.
|
|
|
|
## Multi-node support
|
|
|
|
The readiness board shows pending updates from every node in your fleet in a single view, regardless of which node is selected in the sidebar. You do not need to switch nodes to inspect what is pending elsewhere.
|
|
|
|
Each node group renders its own card grid under a header showing the node name, a `local` or `remote` pill, and a stack count. **Apply now** runs on the node that owns the stack, and **Recheck** fans out to every reachable node so registries get polled in parallel. Sencho handles the routing through the Distributed API; no additional configuration is needed.
|
|
|
|
## How readiness is computed
|
|
|
|
For each stack with a pending image update, Sencho computes a preview by:
|
|
|
|
1. Parsing the compose file to enumerate every service that pulls a registry image (a service declaring only `build:`, with no `image:` key, has no registry reference to check and is excluded from this preview).
|
|
2. Calling the registry with your configured credentials to fetch the current tag list and remote digest.
|
|
3. Picking the highest semver tag greater than the current tag (keeping the same prefix and suffix). If a higher tag exists, the card surfaces as a tag-only advisory (`Newer tag · edit Compose`); scheduled auto-update and **Apply now** do not rewrite Compose pins. If the highest available tag matches the current one but the remote digest has changed, the card surfaces as a Compose-actionable `Rebuild available` / digest update. Sencho checks both a newer semver tag and a digest change behind the same tag; see [Tags vs digests](/features/vulnerability-scanning#tags-vs-digests).
|
|
4. Scoring the overall stack by the most severe image bump. Any major bump marks the stack as blocked.
|
|
5. Normalizing Docker Hub library paths so credentials and changelog lookups resolve correctly.
|
|
|
|
The preview is recomputed each time the readiness board loads, so it reflects the live state of your registries and local images.
|
|
|
|
A stack that mixes registry images and `build:` services still gets a card, scored from its registry images. Triggering **Update** on that stack, from this board or a schedule, always rebuilds the `build:` services from source in the same run, whether or not the registry portion has a pending update. See [Controlling a running stack](/features/stack-management#controlling-a-running-stack) for what a build-aware update runs.
|
|
|
|
## Troubleshooting
|
|
|
|
<AccordionGroup>
|
|
<Accordion title='Card shows "No changelog available"'>
|
|
Sencho reads changelog metadata from the registry's manifest and OCI annotations. Registries that do not publish this metadata (most private registries and many self-hosted ones) render the card without a changelog. The risk badge is still accurate because it is computed from the tag itself.
|
|
</Accordion>
|
|
<Accordion title='Apply now is disabled with a "Blocked · major" tooltip'>
|
|
The stack has a major version bump and is blocked on the readiness board by policy: major updates never auto-apply without human review. To apply after reviewing the upstream release notes, use the stack's lifecycle **Update** action from the sidebar kebab or right-click menu, or open the stack editor and click **Deploy**.
|
|
</Accordion>
|
|
<Accordion title='Card stays stuck on "Checking registry..."'>
|
|
The registry call is either still pending or it failed. Click **Recheck** in the hero to retry. If the stack uses private-registry credentials, confirm they are still valid in **Settings > Registries**.
|
|
</Accordion>
|
|
<Accordion title='"Nothing to update" but I see an update on another view'>
|
|
Image update detection runs on the configured interval (every 2 hours by default) on each node, and the readiness board uses the same cached status. Trigger **Recheck** to force a fresh check across every reachable node, or shorten the interval under **Settings > Automation > Image update checks**.
|
|
</Accordion>
|
|
<Accordion title='Scheduled auto-update runs are not applying to a specific stack'>
|
|
No schedule covers that stack. Open **Schedules** in the top nav, create a new **Auto-update Stack** task targeting the stack (or an **Auto-update All Stacks on Node** task on its node), and pick a cron. The next firing will include the stack, or you can trigger an immediate run from the row.
|
|
</Accordion>
|
|
<Accordion title='Banner says "X of Y nodes reachable"'>
|
|
One or more nodes that are marked online in your fleet did not respond within the request timeout. Pending updates from those nodes are not shown until they come back. Check the node's status from the Fleet view and the network path between this Sencho instance and the unreachable node.
|
|
</Accordion>
|
|
<Accordion title='An old image I wanted to keep disappeared after an update'>
|
|
Automatic cleanup removes only dangling (untagged) image layers, the leftovers a pull orphans. Any image that still carries a tag, including images for stopped stacks, is left in place. If you keep a previous build to roll back to, give it a distinct tag so it is never dangling, or turn off **Prune dangling images after updates** under **Settings > Monitoring > Docker & Storage** on that node.
|
|
</Accordion>
|
|
<Accordion title='Disk usage barely changed after an update'>
|
|
Automatic cleanup reclaims dangling (untagged) image layers only. A long-running host accumulates other reclaimable data (stopped containers, unused volumes and networks, build cache) that this toggle does not touch. Run a full prune from the [Resources](/features/resources) view, or schedule a prune task, to reclaim the rest.
|
|
</Accordion>
|
|
<Accordion title='A scheduled auto-update did not apply and the run history shows a policy error'>
|
|
A [deploy enforcement](/features/deploy-enforcement) policy blocked the update, the same way it would block a manual one. The run history and the dispatched alert name the policy and the matching image findings. Review the findings and either clear them or adjust the policy, then let the next scheduled run or a manual **Apply now** retry.
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
## Related
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Health-gated updates" icon="heart-pulse" href="/features/health-gated-updates">
|
|
The readiness check before an update and the post-update health gate that every apply path, including auto-update, runs through.
|
|
</Card>
|
|
<Card title="Atomic deployments" icon="rotate" href="/features/atomic-deployments">
|
|
How Sencho backs up compose and env files before each update and rolls back automatically on failure.
|
|
</Card>
|
|
<Card title="Deploy enforcement" icon="shield-check" href="/features/deploy-enforcement">
|
|
Set risk policies that block or require review before a manual or scheduled update can apply.
|
|
</Card>
|
|
<Card title="Stack management" icon="layer-group" href="/features/stack-management#controlling-a-running-stack">
|
|
The full set of lifecycle actions on a stack, including how a build-aware update differs from a plain pull.
|
|
</Card>
|
|
<Card title="Scheduled operations" icon="calendar-clock" href="/features/scheduled-operations">
|
|
How the Auto-update Stack and Auto-update All Stacks on Node tasks fit into the shared scheduling system.
|
|
</Card>
|
|
<Card title="Vulnerability scanning" icon="shield-halved" href="/features/vulnerability-scanning#tags-vs-digests">
|
|
Why a same-tag digest change surfaces as a rebuild instead of a version diff.
|
|
</Card>
|
|
</CardGroup>
|