feat: detect stalled stack updates and add in-app recovery actions (#1347)

* feat: detect stalled stack updates and add in-app recovery actions

Add a backend idle-output backstop that stops a deploy/update compose step
that has gone silent (SENCHO_COMPOSE_STALL_TIMEOUT_MS, default 10m), so a
hung image pull surfaces a fast failure instead of spinning indefinitely.

Surface failed, timed-out, and stalled operations with recovery actions on
the stack page: a desktop chip plus popover menu and an inline mobile card
offering retry, restart, roll back (when a backup exists), refresh state,
and copy diagnostics, all gated by deploy permission. The streaming
deploy/update progress modal is now on by default and warns when output
goes quiet. Container state is refreshed after a failed or stalled
operation, and the UI never sits in an indefinite spinner.

* fix: harden rollback against policy-blocked file mutation and refine recovery

Address review findings on the stalled-update recovery work:

- The rollback route restored backup files before running the policy gate, so
  a policy-blocked rollback could leave the on-disk config rolled back while the
  deployed containers were unchanged. Snapshot the current files first and
  revert them when the gate blocks; if that revert itself fails, escalate it on
  the persistent alert feed since the 409 is already sent.
- Refresh container state after a successful manual rollback (rollback
  redeploys), without mis-recording a refetch failure as a rollback failure.
- Suppress the stalled-output warning once live progress is unavailable.

* test: mock snapshotStackFiles in the atomic-deploy rollback route tests

The rollback route now snapshots stack files before restoring a backup, so its
FileSystemService mock needs snapshotStackFiles. Without it the mocked call
threw and the route returned 500, failing the success-path rollback assertions.

frontend/src/components/EditorLayout/recovery-format.ts

@@ -0,0 +1,39 @@
+import type { Node } from '@/context/NodeContext';
+import type { StackActionResult } from './EditorView';
+
+export function capitalize(text: string): string {
+    return text.charAt(0).toUpperCase() + text.slice(1);
+}
+
+export function formatElapsed(ms: number): string {
+    const seconds = Math.max(0, Math.round(ms / 1000));
+    if (seconds >= 60) {
+        const m = Math.floor(seconds / 60);
+        const s = seconds % 60;
+        return `${m}m ${s}s`;
+    }
+    return `${seconds}s`;
+}
+
+// Plain-text troubleshooting blob for the recovery panel's "Copy details". The
+// last output line is included only when the record carries one (session-safe).
+export function buildDiagnostics(
+    stackName: string,
+    result: StackActionResult,
+    activeNode: Node | null,
+    backupInfo: { exists: boolean; timestamp: number | null },
+): string {
+    const lines = [
+        `Stack: ${stackName}`,
+        `Node: ${activeNode?.name ?? 'local'}${activeNode?.id != null ? ` (id ${activeNode.id})` : ''}`,
+        `Action: ${result.action}`,
+        `Outcome: failed${result.rolledBack ? ' (rolled back to previous version)' : ''}`,
+        `Elapsed: ${formatElapsed(result.endedAt - result.startedAt)}`,
+        `Error: ${result.errorMessage ?? 'unknown'}`,
+        `Backup: ${backupInfo.exists
+            ? `available${backupInfo.timestamp ? ` (${new Date(backupInfo.timestamp).toISOString()})` : ''}`
+            : 'none'}`,
+    ];
+    if (result.lastOutputLine) lines.push(`Last output: ${result.lastOutputLine}`);
+    return lines.join('\n');
+}