feat(scheduler): schedule container restart, stop, and start (#1526)

* feat(scheduler): schedule container restart, stop, and start

Add container as a scheduled-task target type so operators can automate lifecycle actions against standalone containers by node and name, with matching UI pickers, validation, execution on local and remote nodes, and tests.

* fix(scheduler): stack service matching and container picker hygiene

Backfill Service on smartFallback containers so per-service stack restarts work when container_name is set. Match services by compose label and container name in stack routes and scheduled restarts. Exclude Sencho from GET /api/containers lists. Hide the Restart Stack service picker when a stack has only one service.

* test(scheduler): scope service checkbox assertion to Services block

The create dialog also has a Delete after run checkbox. Count checkboxes only inside the Services section so CI does not include unrelated form controls.

* fix(scheduler): narrow closest() result to HTMLElement in schedule test

The service-checkbox assertion passed an Element from closest() into
within(), which requires an HTMLElement, failing tsc -b in the frontend
build and Docker build stages. Use the closest<HTMLElement>() type
argument so the value type-checks without an unsafe cast.

* fix(scheduler): hide Sencho container on remote node picker lists

Remote container lists are proxied from peer Sencho instances, so id-only self filtering missed peers on older builds. Await SelfIdentity init, match ImageID, and drop official saelix/sencho images. Apply the same heuristic in the scheduled-operations UI and when the hub fetches remote containers for scheduled runs.

* test(monitor): add missing DatabaseService mocks for scan history cleanup

* test(scheduler): add missing markStaleScansAsFailed mock

SchedulerService.tick() calls db.markStaleScansAsFailed() to sweep stale
vulnerability scans. The scheduler-service test was missing this method in
its DatabaseService mock, causing TypeError failures during test initialization.

Added mockMarkStaleScansAsFailed to hoisted mocks and DatabaseService mock
object, returning safe default of 0 scans marked as failed.

* test(compose): add missing FileSystemService mocks for getStackContent/getEnvContent

* test(containers-route): mock SelfIdentityService to prevent initialize() crash

The excludeSelfContainers() helper calls SelfIdentityService.initialize(), which tries to access DockerController. Without a proper SelfIdentityService mock, the initialize() call fails silently, causing a 500 error on GET /api/containers.

Added SelfIdentityService mock with initialize(), isOwnContainer(), and isOwnImage() methods to prevent the crash.
This commit is contained in:
Anso
2026-07-02 22:31:29 -04:00
committed by GitHub
parent b65daf6845
commit 10fb93dcb1
29 changed files with 932 additions and 68 deletions
+132 -1
View File
@@ -21,6 +21,8 @@ import type { ScanAllNodeImagesResult } from './TrivyService';
import TrivyInstaller from './TrivyInstaller';
import { CloudBackupService } from './CloudBackupService';
import { buildSystemPolicyGateOptions } from '../helpers/policyGate';
import { filterContainersByComposeService } from '../helpers/composeServiceMatch';
import { excludeSelfContainers } from '../helpers/excludeSelfContainers';
import { enforcePolicyPreDeploy } from './PolicyEnforcement';
import { summarizeBlockReasons } from '../utils/policy-risk';
@@ -424,6 +426,9 @@ export class SchedulerService {
}
private async executeRestart(task: ScheduledTask): Promise<string> {
if (task.target_type === 'container') {
return this.executeContainerRestart(task);
}
if (!task.target_id || task.node_id == null) {
throw new Error('Stack restart requires target_id and node_id');
}
@@ -439,7 +444,7 @@ export class SchedulerService {
let filtered = containers;
if (task.target_services) {
const serviceNames: string[] = JSON.parse(task.target_services);
filtered = containers.filter(c => c.Service && serviceNames.includes(c.Service));
filtered = serviceNames.flatMap(svc => filterContainersByComposeService(containers, svc));
if (filtered.length === 0) {
throw new Error(`No containers found matching services [${serviceNames.join(', ')}] in stack "${task.target_id}"`);
}
@@ -507,6 +512,9 @@ export class SchedulerService {
}
private async executeAutoStop(task: ScheduledTask): Promise<string> {
if (task.target_type === 'container') {
return this.executeContainerStop(task);
}
this.assertStackTarget(task, 'Auto-stop');
if (this.isRemoteNode(task.node_id)) {
await this.postToRemoteStack(task.node_id, `${encodeURIComponent(task.target_id)}/stop`);
@@ -537,6 +545,9 @@ export class SchedulerService {
}
private async executeAutoStart(task: ScheduledTask): Promise<string> {
if (task.target_type === 'container') {
return this.executeContainerStart(task);
}
this.assertStackTarget(task, 'Auto-start');
// Remote auto-start proxies to the remote's own deploy route, which runs
// that node's scan-policy gate against the images it actually holds. The
@@ -802,6 +813,126 @@ export class SchedulerService {
* `routeSuffix` is the path under `/api/stacks/`; the caller URL-encodes each
* segment.
*/
private containerNotFoundMessage(containerName: string, nodeId: number): string {
const nodeName = NodeRegistry.getInstance().getNode(nodeId)?.name ?? String(nodeId);
return `Container "${containerName}" not found on node "${nodeName}". It may have been renamed or removed.`;
}
private async resolveContainerId(task: ScheduledTask): Promise<{ id: string; name: string }> {
if (!task.target_id || task.node_id == null) {
throw new Error('Container operations require target_id and node_id');
}
const name = task.target_id;
if (this.isRemoteNode(task.node_id)) {
const containers = await this.getRemoteContainers(task.node_id);
const match = containers.find(
c => c.Names?.[0]?.replace(/^\//, '') === name,
);
if (!match) throw new Error(this.containerNotFoundMessage(name, task.node_id));
return { id: match.Id, name };
}
const found = await DockerController.getInstance(task.node_id).findContainerByName(name);
if (!found) throw new Error(this.containerNotFoundMessage(name, task.node_id));
return { id: found.id, name: found.name };
}
private async executeContainerRestart(task: ScheduledTask): Promise<string> {
const { id, name } = await this.resolveContainerId(task);
const nodeId = task.node_id!;
if (this.isRemoteNode(nodeId)) {
await this.postToRemoteContainer(nodeId, id, 'restart');
} else {
await DockerController.getInstance(nodeId).restartContainer(id);
}
const nodeName = NodeRegistry.getInstance().getNode(nodeId)?.name ?? String(nodeId);
return `Restarted container "${name}" (id ${id.slice(0, 12)}) on node "${nodeName}"`;
}
private async executeContainerStop(task: ScheduledTask): Promise<string> {
const { id, name } = await this.resolveContainerId(task);
const nodeId = task.node_id!;
if (this.isRemoteNode(nodeId)) {
await this.postToRemoteContainer(nodeId, id, 'stop');
} else {
await DockerController.getInstance(nodeId).stopContainer(id);
}
const nodeName = NodeRegistry.getInstance().getNode(nodeId)?.name ?? String(nodeId);
return `Stopped container "${name}" (id ${id.slice(0, 12)}) on node "${nodeName}"`;
}
private async executeContainerStart(task: ScheduledTask): Promise<string> {
const { id, name } = await this.resolveContainerId(task);
const nodeId = task.node_id!;
if (this.isRemoteNode(nodeId)) {
await this.postToRemoteContainer(nodeId, id, 'start');
} else {
await DockerController.getInstance(nodeId).startContainer(id);
}
const nodeName = NodeRegistry.getInstance().getNode(nodeId)?.name ?? String(nodeId);
return `Started container "${name}" (id ${id.slice(0, 12)}) on node "${nodeName}"`;
}
private async getRemoteContainers(nodeId: number): Promise<Array<{
Id: string;
Names?: string[];
State?: string;
Image?: string;
}>> {
const proxyTarget = NodeRegistry.getInstance().getProxyTarget(nodeId);
if (!proxyTarget) {
throw new Error('Remote node is not configured or missing API credentials');
}
const baseUrl = proxyTarget.apiUrl.replace(/\/$/, '');
const proxyHeaders = LicenseService.getInstance().getProxyHeaders();
const response = await fetch(`${baseUrl}/api/containers?all=true`, {
headers: {
'Authorization': `Bearer ${proxyTarget.apiToken}`,
[PROXY_TIER_HEADER]: proxyHeaders.tier,
},
signal: AbortSignal.timeout(60_000),
});
if (!response.ok) {
const body = await response.json().catch(() => ({ error: `HTTP ${response.status}` }));
throw new Error((body as { error?: string }).error || `Remote node returned ${response.status}`);
}
return excludeSelfContainers(await response.json() as Array<{
Id: string;
Names?: string[];
State?: string;
Image?: string;
ImageID?: string;
}>);
}
private async postToRemoteContainer(
nodeId: number,
containerId: string,
action: 'start' | 'stop' | 'restart',
): Promise<void> {
const proxyTarget = NodeRegistry.getInstance().getProxyTarget(nodeId);
if (!proxyTarget) {
throw new Error('Remote node is not configured or missing API credentials');
}
const baseUrl = proxyTarget.apiUrl.replace(/\/$/, '');
const proxyHeaders = LicenseService.getInstance().getProxyHeaders();
const response = await fetch(
`${baseUrl}/api/containers/${encodeURIComponent(containerId)}/${action}`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${proxyTarget.apiToken}`,
[PROXY_TIER_HEADER]: proxyHeaders.tier,
},
signal: AbortSignal.timeout(300_000),
},
);
if (!response.ok) {
const body = await response.json().catch(() => ({ error: `HTTP ${response.status}` }));
throw new Error((body as { error?: string }).error || `Remote node returned ${response.status}`);
}
}
private async postToRemoteStack(nodeId: number, routeSuffix: string): Promise<void> {
const proxyTarget = NodeRegistry.getInstance().getProxyTarget(nodeId);
if (!proxyTarget) {