mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-31 04:38:11 +00:00
feat: detect stalled stack updates and add in-app recovery actions (#1347)
* feat: detect stalled stack updates and add in-app recovery actions Add a backend idle-output backstop that stops a deploy/update compose step that has gone silent (SENCHO_COMPOSE_STALL_TIMEOUT_MS, default 10m), so a hung image pull surfaces a fast failure instead of spinning indefinitely. Surface failed, timed-out, and stalled operations with recovery actions on the stack page: a desktop chip plus popover menu and an inline mobile card offering retry, restart, roll back (when a backup exists), refresh state, and copy diagnostics, all gated by deploy permission. The streaming deploy/update progress modal is now on by default and warns when output goes quiet. Container state is refreshed after a failed or stalled operation, and the UI never sits in an indefinite spinner. * fix: harden rollback against policy-blocked file mutation and refine recovery Address review findings on the stalled-update recovery work: - The rollback route restored backup files before running the policy gate, so a policy-blocked rollback could leave the on-disk config rolled back while the deployed containers were unchanged. Snapshot the current files first and revert them when the gate blocks; if that revert itself fails, escalate it on the persistent alert feed since the 409 is already sent. - Refresh container state after a successful manual rollback (rollback redeploys), without mis-recording a refetch failure as a rollback failure. - Suppress the stalled-output warning once live progress is unavailable. * test: mock snapshotStackFiles in the atomic-deploy rollback route tests The rollback route now snapshots stack files before restoring a backup, so its FileSystemService mock needs snapshotStackFiles. Without it the mocked call threw and the route returned 500, failing the success-path rollback assertions.
This commit is contained in:
@@ -17,10 +17,12 @@ const {
|
||||
mockDeployStack,
|
||||
mockGetBackupInfo,
|
||||
mockRestoreStackFiles,
|
||||
mockSnapshotStackFiles,
|
||||
} = vi.hoisted(() => ({
|
||||
mockDeployStack: vi.fn(),
|
||||
mockGetBackupInfo: vi.fn(),
|
||||
mockRestoreStackFiles: vi.fn(),
|
||||
mockSnapshotStackFiles: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('../services/ComposeService', async () => {
|
||||
@@ -43,6 +45,7 @@ vi.mock('../services/FileSystemService', () => ({
|
||||
hasComposeFile: vi.fn().mockResolvedValue(true),
|
||||
getBackupInfo: mockGetBackupInfo,
|
||||
restoreStackFiles: mockRestoreStackFiles,
|
||||
snapshotStackFiles: mockSnapshotStackFiles,
|
||||
}),
|
||||
},
|
||||
}));
|
||||
@@ -83,6 +86,7 @@ beforeEach(async () => {
|
||||
mockDeployStack.mockReset();
|
||||
mockGetBackupInfo.mockReset().mockResolvedValue({ exists: true, timestamp: Date.now() });
|
||||
mockRestoreStackFiles.mockReset().mockResolvedValue(undefined);
|
||||
mockSnapshotStackFiles.mockReset().mockResolvedValue(async () => {});
|
||||
const { StackOpLockService } = await import('../services/StackOpLockService');
|
||||
StackOpLockService.resetForTests();
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user