Files
sencho/backend/src/services/GlobalLogsMetrics.ts
T
Anso 69edb0dcbb fix(observability): gate global logs to admins, scope to managed containers, harden SSE (#1254)
* fix(observability): gate global logs to admins, scope to managed containers, harden SSE

Make the Logs feed an administrator view enforced on both sides (requireAdmin on
the /api/logs/global poll and SSE routes; the Logs nav item plus a redirect guard
on the frontend), and scope the feed to Sencho-managed containers only via a
shared isManagedByComposeDir helper that /stats now reuses.

Harden the SSE stream: a stateful frame demuxer that survives chunk boundaries so
a Docker frame split across reads is reassembled instead of dropped or garbled; a
per-stream error listener so one broken follow stream cannot crash the event loop
(it posts a single degraded notice and keeps the others alive); a cap on
concurrent follow streams with a truncation notice; a bounded initial tail; and
backpressure that pauses the source streams when the client is slow and resumes on
drain. Bound the polling snapshot's per-container fan-out with a concurrency limit.

Add process-local, in-memory log-stream counters exposed at the admin-only
/api/system/log-stream-metrics endpoint (active connections, lines streamed,
attach and frame errors). Collapse the view to the local hub and remove the dead
remote-node handling.

* fix(observability): close remote-proxy bypass of the global-logs admin gate

The logs feed's requireAdmin lives in the local route handler, which the remote
proxy skips when forwarding a request whose nodeId targets a remote node. A hub
user could therefore request /api/logs/global*, /api/logs/global/stream, or
/api/system/log-stream-metrics with x-node-id (or ?nodeId= for the SSE transport)
pointing at a remote node and have it served as the node-proxy admin on the far
side, sidestepping the gate entirely.

Add these paths to HUB_ONLY_PREFIXES so hubOnlyGuard rejects a remote nodeId with
403 before the proxy runs, matching the existing protection on audit-log,
scheduled-tasks, and notification-routes. Add regression tests covering the
collection path, the SSE sub-path (both the x-node-id header and the ?nodeId=
query transport), and the stream-metrics endpoint.
2026-05-29 21:09:20 -04:00

74 lines
2.8 KiB
TypeScript

/**
* GlobalLogsMetrics: process-local counters for the Global Observability log
* path (the SSE stream and the polling snapshot). Strictly process-local and
* in-memory: Sencho does not export metrics to any external sink (privacy
* posture, process-local by design), and unlike PilotMetrics these counters
* are NOT persisted to SQLite because they describe ephemeral live-stream
* activity that is meaningless across a restart. They reset on process start.
*
* Surfaced via GET /api/system/log-stream-metrics (admin only). Purpose is
* operator support: spot a connection gauge that never drains (a leak) or a
* rising attach/frame-error count (a daemon or demux problem).
*
* No general in-process metrics facility exists in the backend today; this is
* a per-feature pattern mirroring PilotMetrics. When a shared facility lands,
* this module should be replaced by an instance of it rather than grown.
*/
export interface LogStreamCounters {
/** Live gauge: SSE connections currently open. Should drain to 0 when no tab watches. */
active_sse_connections: number;
/** Monotonic: SSE connections opened since process start. */
sse_connections_total: number;
/** Monotonic: polling-fallback snapshot requests served. */
poll_requests_total: number;
/** Monotonic: log lines pushed to clients (SSE + polling). */
lines_streamed_total: number;
/** Monotonic: follow-stream attach failures and mid-stream stream errors. */
stream_attach_errors_total: number;
/** Monotonic: malformed demux frame headers (corrupt stream, resynced). */
demux_frame_errors_total: number;
}
type MonotonicCounter = Exclude<keyof LogStreamCounters, 'active_sse_connections'>;
const ZERO: LogStreamCounters = {
active_sse_connections: 0,
sse_connections_total: 0,
poll_requests_total: 0,
lines_streamed_total: 0,
stream_attach_errors_total: 0,
demux_frame_errors_total: 0,
};
class GlobalLogsMetricsImpl {
private counters: LogStreamCounters = { ...ZERO };
/** Increment a monotonic counter by `by` (default 1). */
public increment(name: MonotonicCounter, by = 1): void {
this.counters[name] += by;
}
/** A new SSE connection opened: bump the gauge and the total. */
public openConnection(): void {
this.counters.active_sse_connections += 1;
this.counters.sse_connections_total += 1;
}
/** An SSE connection closed: drop the gauge, clamped at zero. */
public closeConnection(): void {
this.counters.active_sse_connections = Math.max(0, this.counters.active_sse_connections - 1);
}
public snapshot(): LogStreamCounters {
return { ...this.counters };
}
/** Reset all in-memory state. Tests only. */
public resetForTests(): void {
this.counters = { ...ZERO };
}
}
export const GlobalLogsMetrics = new GlobalLogsMetricsImpl();