gsadmin/sencho
1
0
Fork 0
mirror of https://github.com/Studio-Saelix/sencho.git synced 2026-07-26 11:49:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
38aabe70648e6c8d1252f753022b72f908819480
sencho/docs/features/deploy-progress.mdx
T
Anso 38aabe7064 feat: health-gated updates and rollback readiness (#1354) 
* feat: classify stack deploy and update failures with suggested next actions

Failed deploy and update responses now carry a failure classification
(cause category, headline, and suggested next step) derived from the
compose error output. The recovery panel and chip render the
classification and include it in copied diagnostics, and gateway-style
failures surface as a node-unreachable cause.

* feat: add update and rollback readiness reports for stacks

Before a manual update, Sencho now shows an advisory readiness verdict
computed from the stored preflight result, open drift findings, live
container health, the pending image change, the rollback backup slot,
and node disk headroom. The Stack Dossier gains a rollback readiness
section that states what a rollback can restore and explicitly
discloses that volume and bind-mounted data are not covered. Toolbar
and sidebar updates now share one update path, and admins can create a
fleet snapshot from the readiness dialog before updating. Nodes that do
not advertise the capability keep the direct update flow.

* feat: observe stack health after updates with a post-deploy health gate

After a deploy or update succeeds, Sencho now watches the stack for a
configurable observation window and records a passed, failed, or
unknown verdict: containers must stay running, healthchecks must report
healthy, and restart loops or disappearing containers fail the gate.
The deploy panel shows the observation live and holds off auto-closing
until the verdict lands, a failed gate surfaces the existing recovery
actions including rollback, and the stack timeline records update
started and gate verdict events. Scheduled, webhook, bulk, and
git-source updates are gated the same way; rollbacks and installs are
deliberately not. The gate is observational only and can be tuned or
disabled per node under host alert settings.

* docs: document health-gated updates and rollback readiness

New operator page covering the update readiness dialog, the post-update
health gate and its settings, the rollback readiness disclosure, and
classified failures, with cross-links from the atomic deployments and
deploy progress pages. The API reference gains the readiness and
health-gate endpoints, the healthGateId success field, and the failure
classification schema on deploy and update error responses.

* feat: withhold the success verdict while the health gate observes

An update used to show a green Succeeded that a failed health gate then
contradicted moments later. The deploy modal now reports Verifying
health while the gate observes, shows success only when the gate
passes, and makes a failed or unknown gate the headline result; success
toasts soften to a verifying message while a gate runs. The mobile
recovery card groups its actions behind one bottom-right Take action
menu so it stays compact on a phone, with the classified cause still
visible on the card. A successful image update now also counts as the
last known-good marker in rollback readiness, and the docs gain
screenshots of the readiness dialog, gate states, dossier section, and
settings.

* fix: harden log format strings and the env existence path check

Log calls that interpolated the stack name into the console format
string now use constant format strings with placeholder arguments, and
envExists validates path containment inline at its filesystem access,
matching the established patterns used elsewhere in the same files.

* test: adapt deploy modal success specs to the post-deploy health gate

The deploy feedback modal now withholds its success verdict while the
health gate observes the new containers, showing "Verifying health"
until the gate passes. The two success-path E2E tests waited for
"Succeeded" within the gate's 90s default window and timed out.

Shorten the observation window to the 15s minimum for these tests via
the settings API, assert the verify-then-succeed sequence the modal
actually renders, and restore the default window afterward so the test
value does not leak into later runs.

* fix: serialize health gate polling and harden gate observation

Address race conditions in the post-update health gate found in review.

Backend: the gate poller used setInterval, so a Docker observe slower
than the 5s tick could overlap the next poll and corrupt the restart and
missing-container accounting, and a wedged socket could leave a poll
pending forever. Polling is now single-flight: each cycle self-schedules
the next only after it settles, and the observe is bounded by an 8s
timeout so a hung probe counts as a poll error and resolves the gate
unknown after three in a row.

Frontend: the gate poller could overlap requests, letting a slow earlier
"observing" response overwrite an already-applied terminal verdict. It is
now single-flight with a terminal latch, so a late response can never
roll the UI back from passed or failed.

Also reject a non-digit nodeId on the snapshot coverage route instead of
letting parseInt coerce it, document that turning off the deploy progress
panel opts out of the live gate UI while the gate still runs server-side,
and add gate-coverage tests for the webhook, git source, and auto-update
apply paths plus the new single-flight, observe-timeout, and recovery
cases.
2026-06-11 00:26:26 -04:00

142 lines
13 KiB
Plaintext
Raw Blame History

---
title: Deploy Progress
description: Stream live output from stack deploy, install, update, restart, and stop operations in a structured log view.
---
When you trigger a stack action that runs through `docker compose` (Deploy, Update, Install from the App Store, Apply with deploy from a Git Source), a progress modal opens and streams the output in real time. Each line is parsed into a timestamped row with a stage badge so you can track the deployment lifecycle as it runs. The modal can be minimized to a small pill that follows you across navigation, so you can leave the App Store mid-install and still see the status from any screen. If an operation goes quiet or fails, Sencho keeps the state observable: the modal warns when output has stopped, and the stack page surfaces recovery actions you can take without leaving Sencho.
## Showing or hiding deploy progress
The modal is **on by default**. To run operations without it, open **Settings > Appearance > Display** and turn off **Deploy progress modal**. The choice is saved to the current browser only and synced across tabs in the same browser without a reload.
<Frame>
<img src="/images/deploy-progress/setting-toggle.png" alt="Settings · Personal · Appearance panel with the Deploy progress modal field and its helper text describing live output streaming for deploy, restart, update, install, and Git operations." />
</Frame>
## Using the modal
The modal opens automatically when you trigger an action. It floats centered in the viewport and does not block access to the rest of the UI.
### What the modal shows
- **Header**: the action verb (Deploying, Updating, Installing, Restarting, Stopping), the stack name in monospace truncated at 200 px, and an elapsed-time chip that appears once the connection moves past the initial **Connecting...** state.
- **Status indicator** in the upper right: one of `Connecting...` while the stream attaches, a live `<n> lines` counter while output is flowing, `Succeeded · closes in <n>s` after a clean finish, or the failure message itself when the run errors out. If the live stream cannot attach or drops, the indicator switches to `Live progress unavailable` and the operation keeps running in the background.
- **Structured log body**: one row per output line. Each row carries a timestamp, a fixed-width stage badge, and the log message. Error rows have a destructive left border and tinted background so they stand out without scanning.
- **Footer**: a `Raw output` / `Hide raw` toggle on the left, with **Minimize** and **Close** buttons on the right.
<Frame>
<img src="/images/deploy-progress/modal-streaming.png" alt="Deploy progress modal mid-flight, showing the elapsed timer, a live line count in the header, and a populated body of timestamped LOG rows from a docker compose pull" />
</Frame>
### Stage badges
Each log row is classified by content. Most lines render as **LOG** because the badges are gated on Docker Compose's `[+] Pulling` / `[+] Creating` / `[+] Starting` progress prefixes, which only appear when Compose is run in TTY mode. The full set of badges Sencho can render:
| Badge | Meaning |
|-------|---------|
| PULL | Image layer being fetched (`[+] Pulling` or `Pulling from`) |
| BUILD | Image build step (`[+] Building`) |
| CREATE | Container being created (`[+] Creating`) |
| START | Container starting (`[+] Starting`) |
| STOP | Container stopping (`[+] Stopping` / `[+] Stopped`) |
| DOWN | Container or network being removed (`[+] Removing` / `[+] Removed`) |
| WARN | Daemon warning line (prefix `WARN[`) |
| ERR | Error line (`Error response from daemon` or text starting with `error`) |
| LOG | Default for any line that does not match the patterns above |
**ERR** rows pick up the destructive left border and the tinted background. **WARN** rows use a softer warning tint without the border.
### Raw output toggle
Click **Raw output** in the footer to expand a 200 px raw terminal panel beneath the structured rows. The raw view shows the unprocessed compose stream, including the progress bars and ANSI-formatted output that the structured parser does not render. Clicking the toggle a second time (now labelled **Hide raw**) collapses the panel.
<Frame>
<img src="/images/deploy-progress/modal-raw-output.png" alt="Deploy progress modal with the Raw output panel expanded, showing structured rows above and the green-on-black raw terminal stream below" />
</Frame>
### Auto-close on success
When an action completes successfully, the status indicator switches to `Succeeded · closes in <n>s` and a 4-second countdown begins. Hover anywhere over the modal to pause the countdown; moving the cursor away restarts a fresh 4 seconds. Clicking **Close** dismisses the modal immediately.
When the [health gate](/features/health-gated-updates) is observing the stack after a deploy or update, the modal withholds the success verdict: the status indicator shows **Verifying health** instead of **Succeeded**, and the auto-close waits for the gate. A passed gate shows the success state and resumes the countdown; a failed or unknown verdict becomes the headline result and keeps the modal open with the reason. Closing the modal never stops the observation.
<Frame>
<img src="/images/deploy-progress/modal-succeeded.png" alt="Deploy progress modal in the succeeded state with a green checkmark, the text 'Succeeded' and 'closes in 2s' in the header, and the full structured log body visible beneath" />
</Frame>
### Stalled-output warning
A long pull or recreate can go quiet for a stretch. When an operation is still running but has produced no new output for a while, the modal shows a warning strip with the elapsed quiet time and the last line received (or a note that no output has arrived yet). The operation keeps running; the strip only makes the quiet visible so you are not left guessing whether anything is happening. If a step is genuinely hung, Sencho stops it after a longer idle window and shows a failure with recovery actions (see [Recovery actions](#recovery-actions)).
### On failure
If the action fails, the modal stays open. The status indicator switches to a destructive icon and shows the error message itself, truncated to 200 px in the header with the full text available on hover. Error rows in the body pick up the destructive left border so they are easy to find when scrolling. The modal stays open until you click **Close**. Close it to reveal the recovery panel on the stack page.
<Frame>
<img src="/images/deploy-progress/modal-failed.png" alt="Deploy progress modal in the failed state, with the truncated error message visible in the header and several ERR rows highlighted with a destructive border in the body" />
</Frame>
### Minimize to pill
Click **Minimize** to collapse the modal to a small status pill anchored at the bottom center of the viewport. The pill shows the action verb, the stack name in monospace, and a status dot that animates while the run is in flight (brand color, pulsing) and goes solid green or red at completion. Click the pill to expand the modal back.
The pill is mounted as a portal, so it persists across navigation: leave the App Store mid-install and the pill follows you to the dashboard, the editor, or any other view.
Minimizing, closing, or navigating away never cancels the operation. The progress view only displays output; the deploy, update, or stop runs to completion on its own. If you dismiss the view while live output is flowing, the run still finishes in the background, and its success or failure lands in your notifications.
<Frame>
<img src="/images/deploy-progress/pill.png" alt="Minimized deploy progress pill anchored at the bottom center of a stack editor view, showing the brand-colored pulsing dot and the text 'Updating docs-demo'" />
</Frame>
## Recovery actions
When a deploy or update fails, times out, or its outcome is ambiguous, the stack page offers safe next steps so you can fix the stack in place. On desktop a small **Update failed** chip appears in the stack card; click it to open a menu of recovery actions. On a phone the same actions show as an inline card on the stack detail. Either way it shows the failed action, the error, and how long the operation ran, and it works whether or not the progress modal is enabled.
The available actions are:
- **Retry** the failed operation.
- **Restart** the stack.
- **Roll back** to the previous version, shown only when a backup exists for the stack.
- **Refresh** to re-read the live container status after the failure.
- **Copy details** to put the stack name, node, action, error, elapsed time, and last output line on your clipboard for a bug report.
Retry, Restart, and Roll back require deploy permission on the stack. After a failed or stalled operation, Sencho refreshes the container state automatically so the page reflects reality, and it never leaves the action stuck in an endless spinner.
## Supported entry points
The modal opens for the following actions:
- **Deploy**, **Update**, **Restart**, **Stop** from the stack editor's action bar.
- **Install** from the App Store.
- **Apply** from a Git Source panel when the apply request includes a deploy.
Of those, **Deploy**, **Update**, **Install**, and Git **Apply** route through `docker compose up` and produce a populated structured-log body. **Restart** and **Stop** call the Docker Engine directly to act on existing containers; they bypass compose, so the modal opens, registers `0 lines`, and finishes. Useful as a confirmation surface for those actions, but with no log content.
The HTTP API also exposes a `down` action (compose-level teardown) that streams its output the same way Deploy and Update do, but no UI control currently triggers it; the `down` endpoint is reachable from automation and from Sencho's own internal cleanup paths.
## Troubleshooting
<AccordionGroup>
<Accordion title="The modal shows 'Live progress unavailable' or stays on 'Connecting...'">
The modal opens a WebSocket to the same host and port as the rest of the API to receive compose output. If that connection cannot upgrade, the modal switches to **Live progress unavailable** and the operation continues without live output, finishing in the background with its result in your notifications. To restore live streaming, check that an upstream proxy or firewall is not blocking WebSocket upgrades. For nginx, ensure `proxy_set_header Upgrade $http_upgrade;` and `proxy_set_header Connection "upgrade";` are configured. See the [configuration guide](/getting-started/configuration#reverse-proxy-setup) for a full nginx example.
</Accordion>
<Accordion title="The modal said 'Live progress unavailable' but my stack still deployed">
That is expected. The deploy, update, or stop is driven by the action you triggered, not by the progress stream. The stream is a live view layered on top, so if it cannot connect or drops partway, the operation still runs to completion. Reopen the stack or check your notifications to confirm the final result.
</Accordion>
<Accordion title="Restart or Stop opens the modal but shows 0 lines">
Restart and Stop hit the Docker Engine directly to act on the existing containers; they do not invoke `docker compose`, so there is no compose stream to render. The modal still opens to confirm the action and to surface a failure if one occurs. For a fully populated structured log, use **Deploy** or **Update** instead.
</Accordion>
<Accordion title="The structured rows are all LOG and I don't see PULL or CREATE badges">
The badges are matched on Docker Compose's `[+] Pulling`, `[+] Creating`, and `[+] Starting` progress prefixes, which Compose only emits when it detects an interactive TTY. Sencho spawns `docker compose` without a TTY, so most output falls back to the **LOG** badge. The structured row, the timestamp, the message, and the destructive styling for **ERR** rows all still apply; only the colored stage chip is missing.
</Accordion>
<Accordion title="The structured rows are empty but raw output shows content">
Some compose wrappers and Docker plugins emit non-standard output. The parser falls back to a **LOG** row for any line it cannot classify, but if a line is consumed entirely by ANSI control sequences it can drop out of the structured view. Toggle **Raw output** to see the full stream.
</Accordion>
<Accordion title="An update stalled or appears stuck">
If a pull or recreate produces no output for a while, the modal shows a stalled-output warning so you know the operation has gone quiet. The operation keeps running. If a step is genuinely hung, Sencho stops it after a longer idle window (10 minutes by default) and the stack page offers recovery actions where you can retry, restart, roll back when a backup exists, refresh the container state, or copy troubleshooting details. To change how long Sencho waits before treating a silent step as stalled, set `SENCHO_COMPOSE_STALL_TIMEOUT_MS`; raise it on slow links or for heavy local image builds.
</Accordion>
<Accordion title="I turned the setting off but the modal still appears">
The setting is stored in `localStorage` under `sencho.deploy-feedback.enabled` and applies to the current tab without a reload. The modal is on by default, so only an explicit off choice hides it. Other tabs in the same browser pick up the change through a `storage` event. If a tab still does not honour the setting, refresh that tab. The setting does not sync across browsers or devices; each one carries its own choice.
</Accordion>
</AccordionGroup>
Reference in New Issue View Git Blame Copy Permalink