feat(scheduler): schedule container restart, stop, and start (#1526)

* feat(scheduler): schedule container restart, stop, and start

Add container as a scheduled-task target type so operators can automate lifecycle actions against standalone containers by node and name, with matching UI pickers, validation, execution on local and remote nodes, and tests.

* fix(scheduler): stack service matching and container picker hygiene

Backfill Service on smartFallback containers so per-service stack restarts work when container_name is set. Match services by compose label and container name in stack routes and scheduled restarts. Exclude Sencho from GET /api/containers lists. Hide the Restart Stack service picker when a stack has only one service.

* test(scheduler): scope service checkbox assertion to Services block

The create dialog also has a Delete after run checkbox. Count checkboxes only inside the Services section so CI does not include unrelated form controls.

* fix(scheduler): narrow closest() result to HTMLElement in schedule test

The service-checkbox assertion passed an Element from closest() into
within(), which requires an HTMLElement, failing tsc -b in the frontend
build and Docker build stages. Use the closest<HTMLElement>() type
argument so the value type-checks without an unsafe cast.

* fix(scheduler): hide Sencho container on remote node picker lists

Remote container lists are proxied from peer Sencho instances, so id-only self filtering missed peers on older builds. Await SelfIdentity init, match ImageID, and drop official saelix/sencho images. Apply the same heuristic in the scheduled-operations UI and when the hub fetches remote containers for scheduled runs.

* test(monitor): add missing DatabaseService mocks for scan history cleanup

* test(scheduler): add missing markStaleScansAsFailed mock

SchedulerService.tick() calls db.markStaleScansAsFailed() to sweep stale
vulnerability scans. The scheduler-service test was missing this method in
its DatabaseService mock, causing TypeError failures during test initialization.

Added mockMarkStaleScansAsFailed to hoisted mocks and DatabaseService mock
object, returning safe default of 0 scans marked as failed.

* test(compose): add missing FileSystemService mocks for getStackContent/getEnvContent

* test(containers-route): mock SelfIdentityService to prevent initialize() crash

The excludeSelfContainers() helper calls SelfIdentityService.initialize(), which tries to access DockerController. Without a proper SelfIdentityService mock, the initialize() call fails silently, causing a 500 error on GET /api/containers.

Added SelfIdentityService mock with initialize(), isOwnContainer(), and isOwnImage() methods to prevent the crash.
This commit is contained in:
Anso
2026-07-02 22:31:29 -04:00
committed by GitHub
parent b65daf6845
commit 10fb93dcb1
29 changed files with 932 additions and 68 deletions
+1 -1
View File
@@ -537,7 +537,7 @@ export interface ApiToken {
export interface ScheduledTask {
id: number;
name: string;
target_type: 'stack' | 'fleet' | 'system';
target_type: 'stack' | 'fleet' | 'system' | 'container';
target_id: string | null;
node_id: number | null;
action: BackendScheduledAction;
+36 -1
View File
@@ -863,6 +863,31 @@ class DockerController {
return this.validateApiData<any[]>(containers);
}
/** Resolve a container by its durable name (not ephemeral ID). */
public async findContainerByName(name: string): Promise<{
id: string;
name: string;
state: string;
image: string;
stackProject: string | null;
} | null> {
const normalized = name.replace(/^\//, '');
const containers = await this.getAllContainers();
for (const c of containers) {
const containerName = c.Names?.[0]?.replace(/^\//, '');
if (containerName === normalized) {
return {
id: c.Id,
name: containerName,
state: c.State ?? 'unknown',
image: c.Image ?? '',
stackProject: c.Labels?.['com.docker.compose.project'] ?? null,
};
}
}
return null;
}
/**
* Builds topology data with 2 Docker API calls instead of N+1.
* Fetches all networks + all containers in parallel, then maps
@@ -1488,10 +1513,13 @@ class DockerController {
// 2. Extract expected container names with legacy prefix support
const expectedNames: string[] = [];
const nameToService = new Map<string, string>();
for (const [serviceName, serviceConfig] of Object.entries(parsedYaml.services)) {
const config = serviceConfig as any;
const config = serviceConfig as { container_name?: string };
nameToService.set(serviceName, serviceName);
if (config.container_name) {
expectedNames.push(config.container_name);
nameToService.set(config.container_name, serviceName);
} else {
// Standard v2 naming
expectedNames.push(serviceName);
@@ -1516,6 +1544,11 @@ class DockerController {
// 5. Map to the frontend interface
return fallbackContainers.map(c => {
const strippedName = c.Names?.[0]?.replace(/^\//, '') ?? '';
const labelService = c.Labels?.['com.docker.compose.service'];
const service = (typeof labelService === 'string' && labelService.length > 0
? labelService
: nameToService.get(strippedName)) ?? '';
let Ports: { PrivatePort: number, PublicPort: number, Type?: string }[] = [];
if (c.Ports && Array.isArray(c.Ports)) {
Ports = c.Ports
@@ -1525,8 +1558,10 @@ class DockerController {
return {
Id: c.Id,
Names: c.Names,
Service: service,
State: c.State,
Status: c.Status,
Labels: c.Labels,
Ports
};
});
+132 -1
View File
@@ -21,6 +21,8 @@ import type { ScanAllNodeImagesResult } from './TrivyService';
import TrivyInstaller from './TrivyInstaller';
import { CloudBackupService } from './CloudBackupService';
import { buildSystemPolicyGateOptions } from '../helpers/policyGate';
import { filterContainersByComposeService } from '../helpers/composeServiceMatch';
import { excludeSelfContainers } from '../helpers/excludeSelfContainers';
import { enforcePolicyPreDeploy } from './PolicyEnforcement';
import { summarizeBlockReasons } from '../utils/policy-risk';
@@ -424,6 +426,9 @@ export class SchedulerService {
}
private async executeRestart(task: ScheduledTask): Promise<string> {
if (task.target_type === 'container') {
return this.executeContainerRestart(task);
}
if (!task.target_id || task.node_id == null) {
throw new Error('Stack restart requires target_id and node_id');
}
@@ -439,7 +444,7 @@ export class SchedulerService {
let filtered = containers;
if (task.target_services) {
const serviceNames: string[] = JSON.parse(task.target_services);
filtered = containers.filter(c => c.Service && serviceNames.includes(c.Service));
filtered = serviceNames.flatMap(svc => filterContainersByComposeService(containers, svc));
if (filtered.length === 0) {
throw new Error(`No containers found matching services [${serviceNames.join(', ')}] in stack "${task.target_id}"`);
}
@@ -507,6 +512,9 @@ export class SchedulerService {
}
private async executeAutoStop(task: ScheduledTask): Promise<string> {
if (task.target_type === 'container') {
return this.executeContainerStop(task);
}
this.assertStackTarget(task, 'Auto-stop');
if (this.isRemoteNode(task.node_id)) {
await this.postToRemoteStack(task.node_id, `${encodeURIComponent(task.target_id)}/stop`);
@@ -537,6 +545,9 @@ export class SchedulerService {
}
private async executeAutoStart(task: ScheduledTask): Promise<string> {
if (task.target_type === 'container') {
return this.executeContainerStart(task);
}
this.assertStackTarget(task, 'Auto-start');
// Remote auto-start proxies to the remote's own deploy route, which runs
// that node's scan-policy gate against the images it actually holds. The
@@ -802,6 +813,126 @@ export class SchedulerService {
* `routeSuffix` is the path under `/api/stacks/`; the caller URL-encodes each
* segment.
*/
private containerNotFoundMessage(containerName: string, nodeId: number): string {
const nodeName = NodeRegistry.getInstance().getNode(nodeId)?.name ?? String(nodeId);
return `Container "${containerName}" not found on node "${nodeName}". It may have been renamed or removed.`;
}
private async resolveContainerId(task: ScheduledTask): Promise<{ id: string; name: string }> {
if (!task.target_id || task.node_id == null) {
throw new Error('Container operations require target_id and node_id');
}
const name = task.target_id;
if (this.isRemoteNode(task.node_id)) {
const containers = await this.getRemoteContainers(task.node_id);
const match = containers.find(
c => c.Names?.[0]?.replace(/^\//, '') === name,
);
if (!match) throw new Error(this.containerNotFoundMessage(name, task.node_id));
return { id: match.Id, name };
}
const found = await DockerController.getInstance(task.node_id).findContainerByName(name);
if (!found) throw new Error(this.containerNotFoundMessage(name, task.node_id));
return { id: found.id, name: found.name };
}
private async executeContainerRestart(task: ScheduledTask): Promise<string> {
const { id, name } = await this.resolveContainerId(task);
const nodeId = task.node_id!;
if (this.isRemoteNode(nodeId)) {
await this.postToRemoteContainer(nodeId, id, 'restart');
} else {
await DockerController.getInstance(nodeId).restartContainer(id);
}
const nodeName = NodeRegistry.getInstance().getNode(nodeId)?.name ?? String(nodeId);
return `Restarted container "${name}" (id ${id.slice(0, 12)}) on node "${nodeName}"`;
}
private async executeContainerStop(task: ScheduledTask): Promise<string> {
const { id, name } = await this.resolveContainerId(task);
const nodeId = task.node_id!;
if (this.isRemoteNode(nodeId)) {
await this.postToRemoteContainer(nodeId, id, 'stop');
} else {
await DockerController.getInstance(nodeId).stopContainer(id);
}
const nodeName = NodeRegistry.getInstance().getNode(nodeId)?.name ?? String(nodeId);
return `Stopped container "${name}" (id ${id.slice(0, 12)}) on node "${nodeName}"`;
}
private async executeContainerStart(task: ScheduledTask): Promise<string> {
const { id, name } = await this.resolveContainerId(task);
const nodeId = task.node_id!;
if (this.isRemoteNode(nodeId)) {
await this.postToRemoteContainer(nodeId, id, 'start');
} else {
await DockerController.getInstance(nodeId).startContainer(id);
}
const nodeName = NodeRegistry.getInstance().getNode(nodeId)?.name ?? String(nodeId);
return `Started container "${name}" (id ${id.slice(0, 12)}) on node "${nodeName}"`;
}
private async getRemoteContainers(nodeId: number): Promise<Array<{
Id: string;
Names?: string[];
State?: string;
Image?: string;
}>> {
const proxyTarget = NodeRegistry.getInstance().getProxyTarget(nodeId);
if (!proxyTarget) {
throw new Error('Remote node is not configured or missing API credentials');
}
const baseUrl = proxyTarget.apiUrl.replace(/\/$/, '');
const proxyHeaders = LicenseService.getInstance().getProxyHeaders();
const response = await fetch(`${baseUrl}/api/containers?all=true`, {
headers: {
'Authorization': `Bearer ${proxyTarget.apiToken}`,
[PROXY_TIER_HEADER]: proxyHeaders.tier,
},
signal: AbortSignal.timeout(60_000),
});
if (!response.ok) {
const body = await response.json().catch(() => ({ error: `HTTP ${response.status}` }));
throw new Error((body as { error?: string }).error || `Remote node returned ${response.status}`);
}
return excludeSelfContainers(await response.json() as Array<{
Id: string;
Names?: string[];
State?: string;
Image?: string;
ImageID?: string;
}>);
}
private async postToRemoteContainer(
nodeId: number,
containerId: string,
action: 'start' | 'stop' | 'restart',
): Promise<void> {
const proxyTarget = NodeRegistry.getInstance().getProxyTarget(nodeId);
if (!proxyTarget) {
throw new Error('Remote node is not configured or missing API credentials');
}
const baseUrl = proxyTarget.apiUrl.replace(/\/$/, '');
const proxyHeaders = LicenseService.getInstance().getProxyHeaders();
const response = await fetch(
`${baseUrl}/api/containers/${encodeURIComponent(containerId)}/${action}`,
{
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${proxyTarget.apiToken}`,
[PROXY_TIER_HEADER]: proxyHeaders.tier,
},
signal: AbortSignal.timeout(300_000),
},
);
if (!response.ok) {
const body = await response.json().catch(() => ({ error: `HTTP ${response.status}` }));
throw new Error((body as { error?: string }).error || `Remote node returned ${response.status}`);
}
}
private async postToRemoteStack(nodeId: number, routeSuffix: string): Promise<void> {
const proxyTarget = NodeRegistry.getInstance().getProxyTarget(nodeId);
if (!proxyTarget) {
@@ -10,7 +10,7 @@
* each side.
*/
export const VALID_TARGET_TYPES = ['stack', 'fleet', 'system'] as const;
export const VALID_TARGET_TYPES = ['stack', 'fleet', 'system', 'container'] as const;
export type TargetType = typeof VALID_TARGET_TYPES[number];
export interface BackendScheduledActionDefinition {
@@ -26,15 +26,15 @@ export interface BackendScheduledActionDefinition {
* in `routes/scheduledTasks.ts` ("Must be restart, snapshot, prune, ...").
*/
export const BACKEND_SCHEDULED_ACTIONS = [
{ id: 'restart', targetTypes: ['stack'], requiresNode: true },
{ id: 'restart', targetTypes: ['stack', 'container'], requiresNode: true },
{ id: 'snapshot', targetTypes: ['fleet'], requiresNode: false },
{ id: 'prune', targetTypes: ['system'], requiresNode: true, nodeScope: 'local' },
{ id: 'update', targetTypes: ['stack', 'fleet'], requiresNode: true },
{ id: 'scan', targetTypes: ['system'], requiresNode: true, nodeScope: 'local' },
{ id: 'auto_backup', targetTypes: ['stack'], requiresNode: true },
{ id: 'auto_stop', targetTypes: ['stack'], requiresNode: true },
{ id: 'auto_stop', targetTypes: ['stack', 'container'], requiresNode: true },
{ id: 'auto_down', targetTypes: ['stack'], requiresNode: true },
{ id: 'auto_start', targetTypes: ['stack'], requiresNode: true },
{ id: 'auto_start', targetTypes: ['stack', 'container'], requiresNode: true },
] as const satisfies readonly BackendScheduledActionDefinition[];
export type BackendScheduledAction = typeof BACKEND_SCHEDULED_ACTIONS[number]['id'];
@@ -58,15 +58,15 @@ const ACTION_BY_ID = new Map<BackendScheduledAction, BackendScheduledActionDefin
* the API contract, so it is kept explicit rather than templated.
*/
const TARGET_MISMATCH_MESSAGE: Record<BackendScheduledAction, string> = {
restart: 'Restart action requires target_type "stack".',
restart: 'Restart action requires target_type "stack" or "container".',
snapshot: 'Snapshot action requires target_type "fleet".',
prune: 'Prune action requires target_type "system".',
update: 'Update action requires target_type "stack" or "fleet".',
scan: 'Scan action requires target_type "system".',
auto_backup: 'auto_backup action requires target_type "stack".',
auto_stop: 'auto_stop action requires target_type "stack".',
auto_stop: 'auto_stop action requires target_type "stack" or "container".',
auto_down: 'auto_down action requires target_type "stack".',
auto_start: 'auto_start action requires target_type "stack".',
auto_start: 'auto_start action requires target_type "stack" or "container".',
};
/**