feat: add service-scoped stack alert rules (#1681)

* feat: add service-scoped stack alert rules

Stack alerts can target one Compose service or all services. Breach timers
are per container and cooldowns are per service so a healthy sibling no
longer clears another container's timer or silences a different service.

* fix: gate remote scoped alert creates without losing the body

Remote hops skip JSON parsing so the proxy stream stays pipeable, which
left service_name invisible to the capability gate. Buffer POST /alerts
bodies for inspection, fail closed when the remote lacks the capability,
and rewrite the buffered bytes on forward. Restore alert-panel alt text
to match the unchanged screenshot.

* fix: bound remote alert body buffer and reject encoded JSON

Cap proxied POST /alerts buffering at the local 100KB JSON limit with
structured 413 cleanup, reject non-identity Content-Encoding with 415 so
compressed scoped bodies cannot bypass the mixed-version gate, and cover
oversized, chunked, and gzip regressions.

* fix: harden service-scoped alert delete, cooldown, and proxy gates

Reject non-digit alert ids, dual-write last_fired_at for rollback safety,
gate cooldown on persisted notification history, fail-fast oversized proxy
bodies with 413, and clarify Not in compose UI semantics.

* test: expect dispatchAlert persisted result in crash-safety cases

Update notification-routing assertions for the new { persisted } return
shape so CI matches the cooldown-gating contract.
This commit is contained in:
Anso
2026-07-23 17:57:04 -04:00
committed by GitHub
parent dd54a2e483
commit 85842cc547
32 changed files with 1736 additions and 135 deletions
+39 -4
View File
@@ -3,9 +3,21 @@ import { z } from 'zod';
import { DatabaseService } from '../services/DatabaseService';
import { authMiddleware } from '../middleware/auth';
import { requireAdmin } from '../middleware/tierGates';
import { isValidServiceName } from '../utils/validation';
import {
getActiveCapabilities,
SERVICE_SCOPED_STACK_ALERT_CAPABILITY,
} from '../services/CapabilityRegistry';
const AlertCreateSchema = z.object({
stack_name: z.string().min(1).max(255),
service_name: z.preprocess(
(val) => (val === '' ? null : val),
z.string().max(255).nullable().optional().refine(
(val) => val == null || isValidServiceName(val),
{ message: 'Invalid service name' },
),
),
metric: z.enum(['cpu_percent', 'memory_percent', 'memory_mb', 'net_rx', 'net_tx', 'restart_count']),
operator: z.enum(['>', '>=', '<', '<=', '==']),
threshold: z.number().min(0),
@@ -22,7 +34,8 @@ alertsRouter.get('/', authMiddleware, async (req: Request, res: Response) => {
const alerts = DatabaseService.getInstance().getStackAlerts(stackName);
res.json(alerts);
} catch {
} catch (error) {
console.error('Failed to fetch alerts:', error);
res.status(500).json({ error: 'Failed to fetch alerts' });
}
});
@@ -34,8 +47,23 @@ alertsRouter.post('/', authMiddleware, async (req: Request, res: Response) => {
res.status(400).json({ error: 'Invalid alert data', details: parsed.error.flatten().fieldErrors });
return;
}
const { service_name, ...alertFields } = parsed.data;
const serviceName = service_name ?? null;
if (
serviceName != null
&& !getActiveCapabilities().includes(SERVICE_SCOPED_STACK_ALERT_CAPABILITY)
) {
res.status(400).json({
error: 'This node does not support service-scoped alert rules',
code: 'capability_unavailable',
});
return;
}
try {
const created = DatabaseService.getInstance().addStackAlert(parsed.data);
const created = DatabaseService.getInstance().addStackAlert({
...alertFields,
service_name: serviceName,
});
res.status(201).json(created);
} catch (error) {
console.error('Failed to add alert:', error);
@@ -45,11 +73,18 @@ alertsRouter.post('/', authMiddleware, async (req: Request, res: Response) => {
alertsRouter.delete('/:id', authMiddleware, async (req: Request, res: Response) => {
if (!requireAdmin(req, res)) return;
// Reject leading-junk / fractional ids (parseInt("1abc") === 1, parseInt("2.5") === 2).
const rawId = String(req.params.id ?? '');
const id = /^\d+$/.test(rawId) ? Number.parseInt(rawId, 10) : NaN;
if (!Number.isInteger(id) || id <= 0) {
res.status(400).json({ error: 'Invalid alert id' });
return;
}
try {
const id = parseInt(req.params.id as string, 10);
DatabaseService.getInstance().deleteStackAlert(id);
res.json({ success: true });
} catch {
} catch (error) {
console.error('Failed to delete alert:', error);
res.status(500).json({ error: 'Failed to delete alert' });
}
});