mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-07 17:34:23 +00:00
Anso
44e8fdfba9
* fix(scheduler): clean up stale runs on startup and auto-disable invalid cron tasks
- Add markStaleRunsAsFailed() bulk DB method with status index
- Clean up orphaned 'running' records on scheduler startup
- Auto-disable tasks when cron expression becomes invalid at execution time
- Promote CRUD debug logs to standard logs for scheduled task admin actions
- Add diagnostic logging for task pre-checks, action timing, and prune fallback
* refactor(scheduling): extract shared types and fix design system violations
- Extract ScheduledTask, TaskRun, NodeOption to shared types file
- Extract getCronDescription and formatTimestamp to shared utilities
- Fix formatTimestamp falsy-zero null check
- Tighten last_status type to 'success' | 'failure' | null
- Add strokeWidth={1.5} to all action icons per design system
- Add sr-only DialogDescription for accessibility
- Wrap Sheet run history in ScrollArea
- Fix delete button styling to match design system pattern
- Change manual trigger toast from "executed" to "triggered"
* test(scheduler): add tests for snapshot, remote update, stale cleanup, and cron invalidation
- Add stale run cleanup tests (bulk markStaleRunsAsFailed, logging)
- Add cron invalidation test (auto-disable, error message)
- Add executeSnapshot tests (fleet capture, empty stacks)
- Add executeUpdateRemote tests (proxy success, remote error)
- Document stale run cleanup and cron auto-disable in troubleshooting docs
195 lines
10 KiB
Plaintext
195 lines
10 KiB
Plaintext
---
|
|
title: "Auto-Update Policies"
|
|
description: "Automatically check for and apply container image updates on a schedule."
|
|
---
|
|
|
|
<Note>
|
|
Auto-Update Policies require a **Skipper** or **Admiral** license.
|
|
</Note>
|
|
|
|
## Overview
|
|
|
|
Auto-Update Policies let you define schedules for Sencho to automatically check your container images for updates and apply them when new versions are available. Think of it as a built-in Watchtower, integrated directly into your Sencho dashboard with full visibility into what was updated and when.
|
|
|
|
Each policy targets a specific stack (or all stacks on a node) and runs on a cron schedule. When triggered, Sencho:
|
|
|
|
1. Inspects every container in the target stack
|
|
2. Compares local image digests against the remote registry
|
|
3. If any image has a newer version, pulls the update and recreates the stack with `docker compose up -d`
|
|
4. Records the result in run history for auditability
|
|
|
|
<Frame>
|
|
<img src="/images/auto-update-policies/overview.png" alt="Auto-Update Policies view showing the policies list" />
|
|
</Frame>
|
|
|
|
## Creating a policy
|
|
|
|
Navigate to **Auto-Update** in the sidebar and click **New Policy**.
|
|
|
|
<Frame>
|
|
<img src="/images/auto-update-policies/create-dialog.png" alt="Create auto-update policy dialog" />
|
|
</Frame>
|
|
|
|
Fill in the following fields:
|
|
|
|
| Field | Description |
|
|
|-------|-------------|
|
|
| **Name** | A descriptive name for the policy (e.g., "Nightly media stack update") |
|
|
| **Node** | The node where the target stack runs |
|
|
| **Stack** | The stack to monitor and update, or **All Stacks** to update every stack on the selected node |
|
|
| **Check Frequency** | A cron preset or custom cron expression defining how often to check |
|
|
| **Enabled** | Toggle to enable or disable the policy on creation |
|
|
|
|
The Stack selector becomes available after choosing a node. Selecting **All Stacks** will check and update every stack on that node during each run.
|
|
|
|
### Schedule presets
|
|
|
|
For convenience, Sencho offers common schedule presets:
|
|
|
|
| Preset | Cron Expression | Description |
|
|
|--------|----------------|-------------|
|
|
| Every 6 hours | `0 */6 * * *` | Check four times per day |
|
|
| Every 12 hours | `0 */12 * * *` | Check twice per day |
|
|
| Daily at 3 AM | `0 3 * * *` | Low-traffic window for most users |
|
|
| Daily at midnight | `0 0 * * *` | Start of each day |
|
|
| Weekly (Sunday 3 AM) | `0 3 * * 0` | Minimal disruption for stable stacks |
|
|
| Custom | User-defined | Any valid cron expression |
|
|
|
|
When using a preset, a human-readable description of the schedule is shown below the selector. Custom cron expressions are validated and described in real time.
|
|
|
|
## Filtering by node
|
|
|
|
In a multi-node environment, you can filter the policy list to show only policies targeting a specific node. Click the **calendar icon** on any node row in **Settings > Nodes** to jump directly to a filtered view. A filter badge at the top shows the active node, with a **Clear filter** button to return to the full list.
|
|
|
|
## Managing policies
|
|
|
|
The policy list is displayed as a table with the following columns:
|
|
|
|
| Column | Description |
|
|
|--------|-------------|
|
|
| **Name** | The policy name |
|
|
| **Stack** | Target stack name, or "All Stacks" for wildcard policies |
|
|
| **Schedule** | Human-readable description with the raw cron expression below |
|
|
| **Status** | Last run result: **Success** (green), **Failed** (red), or "Never run" |
|
|
| **Last Run** | Timestamp of the most recent execution |
|
|
| **Next Run** | When the policy will next execute |
|
|
| **Enabled** | Toggle switch to enable or disable the policy |
|
|
| **Actions** | Action buttons (see below) |
|
|
|
|
### Available actions
|
|
|
|
Each policy row has four action buttons:
|
|
|
|
- **Run Now** (play icon) - Trigger an immediate check-and-update cycle without waiting for the next scheduled run
|
|
- **Execution History** (clock icon) - Open the run history panel for this policy
|
|
- **Edit** (pencil icon) - Modify the policy name, target, or schedule
|
|
- **Delete** (trash icon) - Permanently remove the policy and all its execution history after confirmation
|
|
|
|
## Run history
|
|
|
|
Click the clock icon on any policy to open the run history panel.
|
|
|
|
<Frame>
|
|
<img src="/images/auto-update-policies/run-history.png" alt="Run history panel showing execution results" />
|
|
</Frame>
|
|
|
|
The history is displayed as a table with the following columns:
|
|
|
|
| Column | Description |
|
|
|--------|-------------|
|
|
| **Time** | When the run started |
|
|
| **Source** | Whether the run was triggered by the **Scheduler** or **Manual** (via Run Now) |
|
|
| **Status** | Success, Failed, or Running |
|
|
| **Duration** | How long the run took (in seconds) |
|
|
| **Details** | Output summary or error message |
|
|
|
|
Run history is paginated at 20 entries per page. You can export the full history as CSV using the download button in the panel header.
|
|
|
|
## Multi-node support
|
|
|
|
Auto-Update Policies work seamlessly across both local and remote nodes. When a policy targets a remote node, Sencho automatically proxies the update execution to the remote Sencho instance via the Distributed API. The remote node performs all image checks and compose updates locally on its own machine, then reports the results back.
|
|
|
|
No additional configuration is required. As long as your remote node is connected and reachable, auto-update policies will execute on it just like they do on the local node.
|
|
|
|
## How it works
|
|
|
|
Under the hood, Auto-Update Policies are built on the same scheduling engine as [Scheduled Operations](/features/scheduled-operations). The key difference is that auto-update policies:
|
|
|
|
- Are available to **Skipper** tier (Scheduled Operations requires Admiral)
|
|
- Can target a single **stack** or **all stacks** on a node
|
|
- Perform a **check-then-update** flow rather than a blind restart
|
|
|
|
### The check-then-update flow
|
|
|
|
1. **Enumerate images** - Sencho lists all unique pullable images used by containers in the target stack
|
|
2. **Check digests** - For each image, Sencho compares the local digest against the remote registry manifest
|
|
3. **Conditional update** - Only if at least one image has a newer version does Sencho run `docker compose up -d` to pull and recreate
|
|
4. **Clear indicators** - After a successful update, the blue update indicator dot is automatically cleared
|
|
5. **Notify** - A notification is dispatched informing you which stack was updated and which images changed
|
|
|
|
If no updates are found, the run completes with an "all images up to date" message and no containers are restarted.
|
|
|
|
## Relationship to image update detection
|
|
|
|
Sencho has two complementary features for keeping your images current:
|
|
|
|
| Feature | Purpose | Tier |
|
|
|---------|---------|------|
|
|
| **Image Update Detection** | Passive: shows a blue dot on stacks with available updates | All tiers |
|
|
| **Auto-Update Policies** | Active: automatically applies updates on a schedule | Skipper+ |
|
|
|
|
Image Update Detection runs in the background every 6 hours and highlights stacks that have newer images available. Auto-Update Policies take this a step further by automatically applying those updates based on your defined schedule.
|
|
|
|
## Best practices
|
|
|
|
- **Start with longer intervals** - Use "Daily at 3 AM" or "Weekly" for production stacks. Reserve shorter intervals for dev/staging environments.
|
|
- **Pin critical images** - If a stack uses `image: postgres:16.2` (pinned tag), auto-update will only detect updates to that exact tag. Use floating tags like `postgres:16` if you want minor version updates.
|
|
- **Monitor run history** - Check run history periodically to ensure updates are applying cleanly. Failed runs may indicate registry authentication issues or compose file problems.
|
|
- **Use "All Stacks" carefully** - Wildcard policies update every stack on the node. This is convenient for dev environments but may be too aggressive for production.
|
|
- **Combine with notifications** - Sencho sends alert notifications when auto-updates are applied, so you stay informed even when updates happen automatically.
|
|
|
|
## Troubleshooting
|
|
|
|
### Policy reports "all images up to date" but I see an update elsewhere
|
|
|
|
This usually means the registry check couldn't determine the remote digest. Common causes:
|
|
|
|
- **Private registry without credentials** - If your images are in a private registry, make sure you've added authentication credentials in **Settings > Registries**. Without valid credentials, Sencho can't query the remote manifest.
|
|
- **Network connectivity** - The Sencho instance needs outbound HTTPS access to the registry (e.g., `registry-1.docker.io`, `ghcr.io`). Firewall rules or proxy configurations may block these requests.
|
|
- **Digest-pinned images** - Images referenced by digest (`image: nginx@sha256:abc...`) are immutable by design. Sencho strips the pin for tag-based checking, but if your compose file only uses digest refs, consider switching to tag-based refs for auto-update support.
|
|
|
|
Starting with this version, run history now distinguishes between "all images up to date" (clean check) and "image checks failed" (registry unreachable), so you can tell whether the check actually succeeded.
|
|
|
|
### Policy keeps failing with "Target node offline"
|
|
|
|
This means the remote Sencho instance was unreachable when the policy triggered. Verify:
|
|
|
|
- The remote node's Sencho instance is running
|
|
- The API URL and token in **Settings > Nodes** are correct
|
|
- The remote node has not changed IP address or port
|
|
|
|
### Policy shows "no containers found"
|
|
|
|
This warning appears when a policy targets a specific stack that has no running containers. The stack may have been:
|
|
|
|
- Stopped or removed since the policy was created
|
|
- Renamed (stack names are directory-based)
|
|
|
|
Check that the target stack exists and has at least one running container. For wildcard ("All Stacks") policies, empty stacks are silently skipped without a warning.
|
|
|
|
### "Run Now" finishes instantly
|
|
|
|
This is expected behavior. The Run Now button triggers the update check in the background and returns immediately. The actual check runs asynchronously; check the run history panel to see the result once it completes.
|
|
|
|
### Policy was automatically disabled
|
|
|
|
If a policy's cron expression becomes invalid after creation, the scheduler disables the policy and records the reason in the status column. To fix this:
|
|
|
|
1. Open the policy in edit mode.
|
|
2. Re-enter a valid cron expression (or choose a preset).
|
|
3. Re-enable the policy using the toggle switch.
|
|
|
|
### Run shows "Server restarted during execution"
|
|
|
|
This means Sencho was restarted while this policy's update check was mid-execution. The run was marked as failed automatically on startup. The policy itself is still enabled and will run at its next scheduled time. Use **Run Now** if you want to trigger it immediately.
|