From 02f98ab90ae691ab070ff9ad850f0ae8ac6aa8a9 Mon Sep 17 00:00:00 2001 From: Anso Date: Tue, 2 Jun 2026 11:43:55 -0400 Subject: [PATCH] docs: add recovery guide and link it from README, quickstart, and troubleshooting (#1283) Add a consolidated Recovery page that answers "what do I do if Sencho breaks?" in one place. It covers getting back to a working state when Sencho itself fails, a stack deploy fails, an admin is locked out of sign-in, Docker is unavailable, or a remote node is unreachable, and summarizes each path with a link to the authoritative detail page rather than duplicating it. The page opens by stating the core safety fact: Sencho keeps its state in DATA_DIR and COMPOSE_DIR and treats compose files on disk as the source of truth, so recovery is mostly file-level and running containers are unaffected by a Sencho outage. It points to Backup & Restore as the prerequisite. Link the guide from the README documentation section, the quickstart "where to next" cards, and the top of the troubleshooting page. --- README.md | 1 + docs/docs.json | 1 + docs/getting-started/quickstart.mdx | 3 + docs/operations/recovery.mdx | 118 ++++++++++++++++++++++++++++ docs/operations/troubleshooting.mdx | 2 + 5 files changed, 125 insertions(+) create mode 100644 docs/operations/recovery.mdx diff --git a/README.md b/README.md index d4e1f0ef..3df75f3d 100644 --- a/README.md +++ b/README.md @@ -186,6 +186,7 @@ Sencho does not emit telemetry, analytics, or crash reports. The only outbound t ## Documentation, community, and license - **Documentation:** [docs.sencho.io](https://docs.sencho.io) +- **If something breaks:** the [Recovery guide](https://docs.sencho.io/operations/recovery) covers getting back to a working state when Sencho, a deploy, sign-in, Docker, or a node fails. - **Community:** [GitHub Discussions](https://github.com/studio-saelix/sencho/discussions) - **Contributing:** [CONTRIBUTING.md](CONTRIBUTING.md) - **Security:** [SECURITY.md](SECURITY.md). Do not open public issues for security vulnerabilities. diff --git a/docs/docs.json b/docs/docs.json index aaf499e5..d8d9ee08 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -179,6 +179,7 @@ "operations/self-hosting", "operations/upgrade", "operations/backup", + "operations/recovery", "operations/troubleshooting", "operations/verifying-images", "operations/trivy-setup", diff --git a/docs/getting-started/quickstart.mdx b/docs/getting-started/quickstart.mdx index 3f31f1de..9757663b 100644 --- a/docs/getting-started/quickstart.mdx +++ b/docs/getting-started/quickstart.mdx @@ -114,4 +114,7 @@ Click **Create Stack** in the sidebar to deploy your first stack, or open **App Browse 190+ pre-configured templates and deploy in one click. + + What to do if Sencho, a deploy, sign-in, Docker, or a node fails, and how to get back to a working state. + diff --git a/docs/operations/recovery.mdx b/docs/operations/recovery.mdx new file mode 100644 index 00000000..1aba524f --- /dev/null +++ b/docs/operations/recovery.mdx @@ -0,0 +1,118 @@ +--- +title: Recovery +description: What to do when Sencho, a deploy, sign-in, Docker, or a remote node fails, and how to get back to a working state. +--- + +When something goes wrong, the first question is always the same: *can I get back to a working state, and how much is at risk?* For Sencho the answer is reassuring, because Sencho keeps very little state of its own. + +Everything Sencho knows lives in two directories: + +- **`DATA_DIR`** (default `/app/data`) holds `sencho.db` and `encryption.key`: your users, nodes, alert rules, labels, and settings. +- **`COMPOSE_DIR`** holds your stacks: the `compose.yaml` and `.env` files that define your actual applications. + +Your compose files on disk are always the source of truth. Sencho reads them; it does not hide your stacks inside a database. That means most recovery is file-level: if Sencho itself is unhealthy, your running containers keep running, and a fresh Sencho pointed at the same two directories comes back with everything intact. + + + The single most valuable thing you can do before relying on Sencho is to back up those two directories. See [Backup & Restore](/operations/backup) for a copy-and-restore routine and a one-line nightly cron. Do this before you start testing in earnest. + + +--- + +## Sencho itself won't start or is misbehaving + +**Symptom:** The dashboard won't load, the container restarts in a loop, or the UI is throwing errors with no useful message. + +**What to do:** + +1. Read the container's own logs first. This is where backend route and service errors surface: + ```bash + docker logs sencho + ``` +2. Check the health endpoint. A `200` confirms the backend is up: + ```bash + curl http://localhost:1852/api/health + ``` +3. If the database is corrupted (for example after an interrupted disk write or a full disk), stop the container and restore `DATA_DIR` from your most recent backup, then start it again. + +Because your containers and compose files are independent of the Sencho process, none of this stops or changes your running stacks. A reinstall pointed at the same `DATA_DIR` and `COMPOSE_DIR` resumes with your settings and stacks in place. + +See [Backup & Restore](/operations/backup#restoring) for the restore steps and [Troubleshooting](/operations/troubleshooting#getting-logs-from-the-sencho-container-itself) for reading logs and the health endpoint. + +--- + +## A stack deploy fails + +**Symptom:** You click Deploy and the containers exit immediately, never appear, or come up unhealthy. + +**What to do:** + +1. Look at the stack's own logs through the [Host Console](/features/host-console), or directly on the host: + ```bash + docker compose -f /path/to/your/stack/compose.yaml logs + ``` +2. The usual causes are a missing environment variable, a host port already in use, or a bind-mount path that does not exist yet. Each is fixed in the compose file or `.env`. +3. If a previously working stack broke after an edit, use **one-click rollback** in the editor to return to the last deployed version of the compose file. + +See [Troubleshooting: containers won't start after deploy](/operations/troubleshooting#containers-wont-start-after-deploy) for the full cause list and [Stack management](/features/stack-management) for the editor and rollback. + +--- + +## You are locked out of sign-in + +**Symptom:** An administrator has lost their second factor, or the admin password is forgotten, and no one can sign in through the UI. + +**What to do, least disruptive first:** + +1. **Another admin is still in:** any administrator can reset a locked-out user's two-factor enrolment from **Settings · Users**. The user then signs in with their password alone and re-enrols. +2. **Every admin has lost 2FA:** reset two-factor for the admin account directly on the host with the emergency command-line reset. It acts on the same database the app uses and records the action in the audit log. +3. **The admin password itself is forgotten:** Sencho has no password-reset email flow. The recovery path is to reset first-boot setup by removing `sencho.db`, which lets you create a fresh admin account. + + + Removing `sencho.db` resets all Sencho configuration: users, nodes, alert rules, notification settings, and labels. Your Docker stacks and their compose files are not affected, and Sencho re-discovers them on the next scan. Use this only when no administrator can sign in. + + +See [Managing Two-Factor Authentication](/operations/two-factor-admin) for the admin reset and the emergency CLI, and [Troubleshooting: forgotten admin password](/operations/troubleshooting#forgotten-admin-password) for the database-reset path. + +--- + +## Docker is unavailable + +**Symptom:** Sencho starts but the stack list is empty, or you see errors accessing Docker even though containers exist on the host. + +**What to do:** + +1. Confirm the Docker socket is mounted into the Sencho container: + ```yaml + volumes: + - /var/run/docker.sock:/var/run/docker.sock + ``` +2. Confirm the Docker daemon is running on the host (`docker ps` from a host shell). +3. If the daemon was down or restarting, Sencho reconnects on its own once Docker is back; no action is needed inside Sencho. + +See [Troubleshooting: permission denied on the Docker socket](/operations/troubleshooting#permission-denied-on-the-docker-socket) for the socket-mount details. + +--- + +## A remote node is unreachable + +**Symptom:** A node you added shows an offline or unknown status in the Fleet view. + +**What to do, in order:** + +1. Confirm the remote Sencho instance is running on that machine. +2. Confirm the node's API URL is correct, including protocol and port, and that the host can reach it. +3. Confirm the node's token is valid. If in doubt, generate a fresh token on the remote instance and update the node config. +4. Confirm no firewall is blocking the port between the primary and the remote. +5. Re-test connectivity from **Settings · Nodes** using the test button on the node row. + +A node going offline never affects the other nodes or the primary; each Sencho instance manages its own Docker independently. + +See [Troubleshooting: remote node shows offline](/operations/troubleshooting#remote-node-shows-offline-or-unknown) and the [Multi-node guide](/features/multi-node) for the connectivity model. + +--- + +## If none of this helps + +- Review the [known limitations](https://github.com/studio-saelix/sencho/blob/main/KNOWN_LIMITATIONS.md) to confirm you are not hitting a documented constraint. +- Browse the full [Troubleshooting](/operations/troubleshooting) page for symptom-specific entries. +- Reach the team through the channels on the [Contact & Support](/reference/contact) page. diff --git a/docs/operations/troubleshooting.mdx b/docs/operations/troubleshooting.mdx index 4e630f4f..457eb955 100644 --- a/docs/operations/troubleshooting.mdx +++ b/docs/operations/troubleshooting.mdx @@ -7,6 +7,8 @@ description: Solutions to the most common Sencho setup and runtime problems. Sencho is currently in public beta on the path to v1.0. Core workflows are actively tested, but early users should review the [known limitations](https://github.com/studio-saelix/sencho/blob/main/KNOWN_LIMITATIONS.md) and avoid deploying it blindly on critical infrastructure without testing in their own environment first. +If something has broken and you want a quick "what do I do now?" overview rather than a specific symptom, start with the [Recovery guide](/operations/recovery). It walks through getting back to a working state when Sencho, a deploy, sign-in, Docker, or a remote node fails, and links back here for the details. + ## Containers won't start after deploy **Symptom:** You click Deploy and containers immediately exit or never appear.