mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-19 14:56:27 +00:00
38aabe7064
* feat: classify stack deploy and update failures with suggested next actions Failed deploy and update responses now carry a failure classification (cause category, headline, and suggested next step) derived from the compose error output. The recovery panel and chip render the classification and include it in copied diagnostics, and gateway-style failures surface as a node-unreachable cause. * feat: add update and rollback readiness reports for stacks Before a manual update, Sencho now shows an advisory readiness verdict computed from the stored preflight result, open drift findings, live container health, the pending image change, the rollback backup slot, and node disk headroom. The Stack Dossier gains a rollback readiness section that states what a rollback can restore and explicitly discloses that volume and bind-mounted data are not covered. Toolbar and sidebar updates now share one update path, and admins can create a fleet snapshot from the readiness dialog before updating. Nodes that do not advertise the capability keep the direct update flow. * feat: observe stack health after updates with a post-deploy health gate After a deploy or update succeeds, Sencho now watches the stack for a configurable observation window and records a passed, failed, or unknown verdict: containers must stay running, healthchecks must report healthy, and restart loops or disappearing containers fail the gate. The deploy panel shows the observation live and holds off auto-closing until the verdict lands, a failed gate surfaces the existing recovery actions including rollback, and the stack timeline records update started and gate verdict events. Scheduled, webhook, bulk, and git-source updates are gated the same way; rollbacks and installs are deliberately not. The gate is observational only and can be tuned or disabled per node under host alert settings. * docs: document health-gated updates and rollback readiness New operator page covering the update readiness dialog, the post-update health gate and its settings, the rollback readiness disclosure, and classified failures, with cross-links from the atomic deployments and deploy progress pages. The API reference gains the readiness and health-gate endpoints, the healthGateId success field, and the failure classification schema on deploy and update error responses. * feat: withhold the success verdict while the health gate observes An update used to show a green Succeeded that a failed health gate then contradicted moments later. The deploy modal now reports Verifying health while the gate observes, shows success only when the gate passes, and makes a failed or unknown gate the headline result; success toasts soften to a verifying message while a gate runs. The mobile recovery card groups its actions behind one bottom-right Take action menu so it stays compact on a phone, with the classified cause still visible on the card. A successful image update now also counts as the last known-good marker in rollback readiness, and the docs gain screenshots of the readiness dialog, gate states, dossier section, and settings. * fix: harden log format strings and the env existence path check Log calls that interpolated the stack name into the console format string now use constant format strings with placeholder arguments, and envExists validates path containment inline at its filesystem access, matching the established patterns used elsewhere in the same files. * test: adapt deploy modal success specs to the post-deploy health gate The deploy feedback modal now withholds its success verdict while the health gate observes the new containers, showing "Verifying health" until the gate passes. The two success-path E2E tests waited for "Succeeded" within the gate's 90s default window and timed out. Shorten the observation window to the 15s minimum for these tests via the settings API, assert the verify-then-succeed sequence the modal actually renders, and restore the default window afterward so the test value does not leak into later runs. * fix: serialize health gate polling and harden gate observation Address race conditions in the post-update health gate found in review. Backend: the gate poller used setInterval, so a Docker observe slower than the 5s tick could overlap the next poll and corrupt the restart and missing-container accounting, and a wedged socket could leave a poll pending forever. Polling is now single-flight: each cycle self-schedules the next only after it settles, and the observe is bounded by an 8s timeout so a hung probe counts as a poll error and resolves the gate unknown after three in a row. Frontend: the gate poller could overlap requests, letting a slow earlier "observing" response overwrite an already-applied terminal verdict. It is now single-flight with a terminal latch, so a late response can never roll the UI back from passed or failed. Also reject a non-digit nodeId on the snapshot coverage route instead of letting parseInt coerce it, document that turning off the deploy progress panel opts out of the live gate UI while the gate still runs server-side, and add gate-coverage tests for the webhook, git source, and auto-update apply paths plus the new single-flight, observe-timeout, and recovery cases.
159 lines
7.0 KiB
TypeScript
159 lines
7.0 KiB
TypeScript
import { Router, type Request, type Response } from 'express';
|
|
import { z } from 'zod';
|
|
import { DatabaseService } from '../services/DatabaseService';
|
|
import { authMiddleware } from '../middleware/auth';
|
|
import { requireAdmin, requirePaid } from '../middleware/tierGates';
|
|
|
|
// Strict allowlist of keys readable and writable via the generic settings
|
|
// API. This is the single source of truth for what the endpoint exposes:
|
|
// reads project only these keys, so secrets written to global_settings by
|
|
// other subsystems (the cloud_backup_* credentials stored by the cloud-backup
|
|
// route, the auth_* login secrets) are never returned here; writes are
|
|
// rejected for anything outside the list.
|
|
const ALLOWED_SETTING_KEYS = new Set([
|
|
'host_cpu_limit',
|
|
'host_ram_limit',
|
|
'host_disk_limit',
|
|
'host_alert_suppression_mins',
|
|
'docker_janitor_gb',
|
|
'global_crash',
|
|
'developer_mode',
|
|
'template_registry_url',
|
|
'metrics_retention_hours',
|
|
'log_retention_days',
|
|
'audit_retention_days',
|
|
'mesh_auto_recreate',
|
|
'scan_history_per_image_limit',
|
|
'prune_on_update',
|
|
'reclaim_hero',
|
|
'snapshot_documentation',
|
|
'health_gate_enabled',
|
|
'health_gate_window_seconds',
|
|
]);
|
|
|
|
// Keys whose write requires a paid license, not just an admin role.
|
|
// audit_retention_days configures the paid audit log, so a Community admin
|
|
// must not be able to set it.
|
|
const PAID_ONLY_SETTING_KEYS = new Set(['audit_retention_days']);
|
|
|
|
// Bulk PATCH schema. All keys optional; present keys are fully validated.
|
|
const SettingsPatchSchema = z.object({
|
|
host_cpu_limit: z.coerce.number().int().min(1).max(100).transform(String),
|
|
host_ram_limit: z.coerce.number().int().min(1).max(100).transform(String),
|
|
host_disk_limit: z.coerce.number().int().min(1).max(100).transform(String),
|
|
host_alert_suppression_mins: z.coerce.number().int().min(1).max(1440).transform(String),
|
|
docker_janitor_gb: z.coerce.number().min(0).transform(String),
|
|
global_crash: z.enum(['0', '1']),
|
|
developer_mode: z.enum(['0', '1']),
|
|
template_registry_url: z.string().max(2048).refine(v => v === '' || /^https?:\/\/.+/.test(v), { message: 'Must be a valid URL or empty' }),
|
|
metrics_retention_hours: z.coerce.number().int().min(1).max(8760).transform(String),
|
|
log_retention_days: z.coerce.number().int().min(1).max(365).transform(String),
|
|
audit_retention_days: z.coerce.number().int().min(1).max(365).transform(String),
|
|
mesh_auto_recreate: z.enum(['0', '1']),
|
|
scan_history_per_image_limit: z.coerce.number().int().min(5).max(1000).transform(String),
|
|
prune_on_update: z.enum(['0', '1']),
|
|
reclaim_hero: z.enum(['0', '1']),
|
|
snapshot_documentation: z.enum(['0', '1']),
|
|
health_gate_enabled: z.enum(['0', '1']),
|
|
health_gate_window_seconds: z.coerce.number().int().min(15).max(600).transform(String),
|
|
}).partial();
|
|
|
|
export const settingsRouter = Router();
|
|
|
|
settingsRouter.get('/', authMiddleware, async (_req: Request, res: Response): Promise<void> => {
|
|
try {
|
|
const all = DatabaseService.getInstance().getGlobalSettings();
|
|
// Project only allowlisted operational keys. A denylist would leak every
|
|
// future sensitive key written to global_settings by default (e.g. the
|
|
// cloud_backup_* credentials the cloud-backup route stores here); the
|
|
// allowlist fails closed.
|
|
const settings: Record<string, string> = {};
|
|
for (const [key, value] of Object.entries(all)) {
|
|
if (ALLOWED_SETTING_KEYS.has(key)) settings[key] = value;
|
|
}
|
|
res.json(settings);
|
|
} catch (error) {
|
|
console.error('Failed to fetch settings:', error);
|
|
res.status(500).json({ error: 'Failed to fetch settings' });
|
|
}
|
|
});
|
|
|
|
settingsRouter.post('/', authMiddleware, async (req: Request, res: Response): Promise<void> => {
|
|
if (!requireAdmin(req, res)) return;
|
|
try {
|
|
const { key, value } = req.body;
|
|
if (!key || typeof key !== 'string' || !ALLOWED_SETTING_KEYS.has(key)) {
|
|
res.status(400).json({ error: `Invalid or disallowed setting key: ${key}` });
|
|
return;
|
|
}
|
|
if (PAID_ONLY_SETTING_KEYS.has(key) && !requirePaid(req, res)) return;
|
|
if (value === undefined || value === null) {
|
|
res.status(400).json({ error: 'Setting value is required' });
|
|
return;
|
|
}
|
|
// Route the single-key write through the same per-key schema used by
|
|
// the bulk PATCH so allowlisted-but-malformed values (e.g. `true`,
|
|
// `banana`, out-of-range integers) cannot bypass validation just
|
|
// because they came in via the single-key path. The schema coerces
|
|
// numeric settings to strings and rejects enum-shaped settings that
|
|
// are not one of the allowed literals.
|
|
const parsed = SettingsPatchSchema.safeParse({ [key]: value });
|
|
if (!parsed.success) {
|
|
res.status(400).json({
|
|
error: 'Validation failed',
|
|
details: parsed.error.flatten().fieldErrors,
|
|
});
|
|
return;
|
|
}
|
|
const validated = (parsed.data as Record<string, string>)[key];
|
|
if (validated === undefined) {
|
|
// Defensive: the schema is `.partial()`, so an unknown key would
|
|
// pass through silently. We already gated on ALLOWED_SETTING_KEYS,
|
|
// but reject explicitly if the key is somehow missing from the
|
|
// schema's shape (drift between the allowlist and the schema).
|
|
res.status(400).json({ error: `Setting key has no validator: ${key}` });
|
|
return;
|
|
}
|
|
DatabaseService.getInstance().updateGlobalSetting(key, validated);
|
|
res.json({ success: true });
|
|
} catch (error) {
|
|
console.error('Failed to update setting:', error);
|
|
res.status(500).json({ error: 'Failed to update setting' });
|
|
}
|
|
});
|
|
|
|
settingsRouter.patch('/', authMiddleware, async (req: Request, res: Response): Promise<void> => {
|
|
if (!requireAdmin(req, res)) return;
|
|
try {
|
|
// Reject unknown/disallowed keys outright rather than letting Zod silently
|
|
// strip them. This keeps the bulk path fail-closed and consistent with the
|
|
// single-key POST, so a client sending a stale or disallowed key (e.g. an
|
|
// auth_* secret) gets a 400, not a misleading 200.
|
|
const body = req.body;
|
|
if (body && typeof body === 'object' && !Array.isArray(body)) {
|
|
const unknownKeys = Object.keys(body).filter(k => !ALLOWED_SETTING_KEYS.has(k));
|
|
if (unknownKeys.length > 0) {
|
|
res.status(400).json({ error: `Invalid or disallowed setting key(s): ${unknownKeys.join(', ')}` });
|
|
return;
|
|
}
|
|
}
|
|
const parsed = SettingsPatchSchema.safeParse(req.body);
|
|
if (!parsed.success) {
|
|
res.status(400).json({ error: 'Validation failed', details: parsed.error.flatten().fieldErrors });
|
|
return;
|
|
}
|
|
if (Object.keys(parsed.data).some(k => PAID_ONLY_SETTING_KEYS.has(k)) && !requirePaid(req, res)) return;
|
|
const db = DatabaseService.getInstance();
|
|
const updateMany = db.getDb().transaction((entries: [string, string][]) => {
|
|
for (const [k, v] of entries) {
|
|
db.updateGlobalSetting(k, v);
|
|
}
|
|
});
|
|
updateMany(Object.entries(parsed.data) as [string, string][]);
|
|
res.json({ success: true });
|
|
} catch (error) {
|
|
console.error('Failed to bulk update settings:', error);
|
|
res.status(500).json({ error: 'Failed to update settings' });
|
|
}
|
|
});
|