feat(recovery): add safe-mode recovery surface and emergency CLI (#1286)

* feat(recovery): add safe-mode recovery surface and emergency CLI

Add a read-only Recovery tab under Settings (admin-only) backed by a new
GET /api/diagnostics endpoint reporting app version, database integrity,
encryption-key status, Docker reachability, account and SSO counts, and
non-secret configuration. The endpoint loads without Docker or live metrics
so it stays available when the dashboard does not, requires a genuine admin
session, and builds its config block from a non-secret allowlist so no
credentials are ever exposed.

Expand the emergency command-line toolkit beyond the two-factor reset with
seven host-level commands: reset-password, create-emergency-admin,
clear-sessions, disable-sso, diagnostics, validate-db, and backup-data. Each
prints its result, exits with a meaningful status code, and writes an audit
entry where it changes state.

Document the toolkit in a new operator guide and link it from the recovery
and two-factor pages.

* feat(recovery): download the emergency command reference as a text file

The recovery commands are needed exactly when the dashboard is unreachable,
so reading them only in-app is a chicken-and-egg problem. Add a Download
button to the command-line section that saves the full
`docker compose exec sencho ...` reference as a text file, letting operators
keep it on hand before they need it. Reuses a shared download helper with the
existing diagnostics export.

* fix(recovery): harden diagnostics, backup, and emergency-admin against edge cases

Address findings from an independent review of the recovery toolkit:

- DiagnosticsService now degrades instead of throwing when a queried table is
  missing or corrupt: each read falls back and is folded into database.ok, so a
  broken database reports "problem detected" rather than failing the whole
  endpoint or showing a misleading healthy state with zeroed counts.
- backup-data refuses a destination that resolves to the live database, which
  would otherwise report success while producing no separate copy.
- create-emergency-admin now applies the same username rule as the user-
  management route, extracted to a shared helper so both stay in sync.

Adds tests for a missing read table, a malformed emergency-admin username, and
the backup same-target rejection.
This commit is contained in:
Anso
2026-06-02 16:11:24 -04:00
committed by GitHub
parent 06b25262cc
commit c6d1631afe
27 changed files with 1339 additions and 11 deletions
+38
View File
@@ -0,0 +1,38 @@
import { Router, type Request, type Response } from 'express';
import { requireAdmin, requireUserSession } from '../middleware/tierGates';
import { collectDiagnostics } from '../services/DiagnosticsService';
import DockerController from '../services/DockerController';
import { withTimeout } from '../utils/withTimeout';
export const diagnosticsRouter = Router();
const DOCKER_PING_TIMEOUT_MS = 2000;
// Recovery diagnostics for the local control plane. Restricted to a genuine
// signed-in admin session: requireUserSession rejects API tokens and
// node_proxy / pilot_tunnel machine credentials so a long-lived machine token
// cannot read the control plane's configuration inventory. Read-only and
// secret-free (see DiagnosticsService for the redaction allowlist). The Docker
// probe is bounded and wrapped so a down or hung daemon yields
// `docker.reachable: false` instead of failing the whole request, which is the
// exact condition an operator opens this surface to diagnose.
diagnosticsRouter.get('/', async (req: Request, res: Response): Promise<void> => {
if (!requireUserSession(req, res)) return;
if (!requireAdmin(req, res)) return;
try {
const report = await collectDiagnostics({
checkDocker: async () => {
await withTimeout(
DockerController.getInstance().getDocker().ping(),
DOCKER_PING_TIMEOUT_MS,
'docker-ping',
);
return true;
},
});
res.json(report);
} catch (err) {
console.error('[diagnostics] failed to collect report:', (err as Error).message);
res.status(500).json({ error: 'Failed to collect diagnostics.' });
}
});
+1 -7
View File
@@ -10,6 +10,7 @@ import { isDebugEnabled } from '../utils/debug';
import { getErrorMessage, isSqliteUniqueViolation } from '../utils/errors';
import { parseIntParam } from '../utils/parseIntParam';
import { sanitizeForLog } from '../utils/safeLog';
import { validateUsername } from '../helpers/validateUsername';
const USERS_SCOPE_MESSAGE = 'API tokens cannot access user management.';
const VALID_USER_ROLES: UserRole[] = ['admin', 'viewer', 'deployer', 'node-admin', 'auditor'];
@@ -23,13 +24,6 @@ function roleRequiresAdmiral(role: UserRole): boolean {
return role === 'deployer' || role === 'node-admin' || role === 'auditor';
}
function validateUsername(value: unknown): string | null {
if (typeof value !== 'string' || value.length < 3 || !/^[a-zA-Z0-9_-]+$/.test(value)) {
return 'Username must be at least 3 characters (letters, numbers, underscore, hyphen)';
}
return null;
}
// Returns a seat-limit error message if adding an account of `role` would
// exceed the current license seat caps, or null when within limits. Counts are
// read at call time so the check reflects live state. Used by both user