fix(stack-activity): per-stack history integrity, attribution, sanitization (#1228)

* fix(stack-activity): per-stack history integrity, attribution, sanitization

Address the Stack Activity audit findings (PR 1 of 2):

- Per-stack history integrity: drop the per-insert 100-row prune in
  addNotificationHistory that evicted quieter stacks' history whenever
  another stack got chatty. Periodic cleanupOldNotifications now caps
  per (node, stack) at 500 rows and per-node unattached system events
  at 1000 rows, on top of the existing 30-day retention. Signature
  takes an options bag and returns a per-stage summary so MonitorService
  can log what actually ran each cycle.

- Actor attribution: thread req.user?.username through every
  notifyActionFailure call site and add synthetic actors at service
  emit sites (system:autoheal, system:scheduler, system:image-update,
  system:docker-events, system:blueprint, system:monitor, system:policy).
  The timeline renders system actors as "via <Label>" so an autoheal
  redeploy is no longer indistinguishable from a user redeploy.

- Message sanitization: new sanitizeNotificationMessage at
  NotificationService.dispatchAlert strips KEY=VALUE pairs whose key
  ends in TOKEN/KEY/PASSWORD/SECRET/CREDENTIALS/AUTH, scrubs HTTP basic
  auth in URLs and Bearer tokens, collapses COMPOSE_DIR paths, and
  truncates to 1000 chars. Applied to the stored history and to every
  downstream Discord/Slack/webhook channel. The ImageUpdateService
  recovery-path direct DB write also runs through the sanitizer.

- Composite pagination cursor: getStackActivity now accepts a
  (timestamp, id) cursor (?before=&beforeId=). The legacy timestamp-only
  form silently dropped events when a single compose up emitted many
  events sharing one millisecond. Route rejects beforeId without before.

- Frontend hardening: distinct error state with retry button (initial
  fetch failure no longer renders as the genuine empty state), strict
  positive-integer parsing on cursor params, overrequest-by-1 pagination
  so the last page does not leave a dead "Load more" click, runtime
  guard on liveEvents merge that validates the level union, per-minute
  day-bucket recompute so an open panel does not stay on "Today" past
  midnight.

No tier, role, or capability gate touched. Route permission gate
remains stack:read on the named stack.

* fix(stack-activity): sanitizer covers lowercase env vars and per-node compose dir

External review surfaced two leak paths in the message sanitizer:

- The sensitive-key regex was uppercase-only. Compose env names are
  conventionally uppercase but lowercase forms (db_password, jwt_secret,
  github_token) are valid and do leak through the same Docker and
  compose-parse error paths. Make the regex case-insensitive and tighten
  it to also catch bare TOKEN= / KEY= / PASSWORD= without a prefix word,
  while still leaving BYPASS, COMPASS, and similar non-secret keys alone.

- The compose-dir path collapse only read process.env.COMPOSE_DIR, but
  the real resolution chain is node.compose_dir (per-node DB override)
  -> process.env.COMPOSE_DIR -> /app/compose. A node with a custom
  compose_dir could still leak absolute paths into stored history and
  downstream channels. Route both the dispatchAlert call and the
  ImageUpdateService recovery-path direct write through
  NodeRegistry.getInstance().getComposeDir(localNodeId) so the
  collapse covers every resolution outcome.

Tests now assert lowercase keys are redacted and that BYPASS-style
non-secrets stay intact in both cases. notification-routing mock
extended to stub the new getComposeDir call.

* chore(stack-activity): a11y roles, visibility-aware tick, live-disconnect signal

Close three small follow-ups on the per-stack activity timeline:

- A11y: each day-group gets role="list" and each event row gets
  role="listitem" so screen readers traverse the timeline as a list
  instead of a wall of text. The day-group container also carries an
  aria-label naming the bucket.

- Visibility-aware day-bucket tick: the 60s setInterval that re-derives
  Today/Yesterday/Earlier now short-circuits when document.hidden, so a
  backgrounded panel does not re-render every minute for no visible
  effect.

- Live-disconnect signal: useNotifications dispatches a
  sencho:notifications-connection custom event on WebSocket open and
  close. The timeline listens and, when explicitly disconnected, shows
  a one-line "Live updates offline; reconnecting…" hint above the list.
  The sidebar ticker already surfaces fleet-wide connection state; this
  adds an in-context cue for users who are focused on a single stack.

Stack-name case normalization was considered and rejected: stack names
are case-permissive per the isValidStackName validator, and lowercasing
on read or write would silently rename or hide a user's "MyApp" stack.

* ci(stack-activity): drop unnecessary escape in URL_BASIC_AUTH regex

ESLint no-useless-escape errored on \- inside the character class
[a-zA-Z0-9+.\-] at notificationMessage.ts:14. Move the dash to the
end of the class so it's an unambiguous literal and the escape is no
longer required. Behavior is identical; sanitizer tests still pass.

* revert(stack-activity): drop unvalidated E2E spec from this PR

The spec was committed without ever running against a real Docker
daemon, then failed in CI when it ran for the first time: deploy
returned 200 but no notification appeared on the activity endpoint
within the polling window, suggesting either a deploy-notification
race or a node-id resolution mismatch in the CI environment.

Backend unit tests (route + composite cursor + sanitizer) and
frontend component tests cover the same logic. The E2E spec will
land in a dedicated follow-up once it has been authored against a
working CI environment.
This commit is contained in:
Anso
2026-05-25 21:09:00 -04:00
committed by GitHub
parent 117f590332
commit 2d56ea958a
24 changed files with 852 additions and 133 deletions
+58 -16
View File
@@ -2040,13 +2040,6 @@ export class DatabaseService {
notification.actor_username ?? null,
);
this.db.prepare(`
DELETE FROM notification_history
WHERE node_id = ? AND id NOT IN (
SELECT id FROM notification_history WHERE node_id = ? ORDER BY timestamp DESC LIMIT 100
)
`).run(nodeId, nodeId);
return {
id: result.lastInsertRowid as number,
level: notification.level,
@@ -2060,13 +2053,21 @@ export class DatabaseService {
};
}
public getStackActivity(nodeId: number, stackName: string, opts: { limit: number; before?: number }): NotificationHistory[] {
const sql = opts.before
? 'SELECT * FROM notification_history WHERE node_id = ? AND stack_name = ? AND timestamp < ? ORDER BY timestamp DESC LIMIT ?'
: 'SELECT * FROM notification_history WHERE node_id = ? AND stack_name = ? ORDER BY timestamp DESC LIMIT ?';
const args: (number | string)[] = opts.before
? [nodeId, stackName, opts.before, opts.limit]
: [nodeId, stackName, opts.limit];
public getStackActivity(nodeId: number, stackName: string, opts: { limit: number; before?: number; beforeId?: number }): NotificationHistory[] {
// Composite (timestamp, id) cursor: pure timestamp pagination drops rows
// on same-millisecond bursts (Docker events from one compose up).
let sql: string;
let args: (number | string)[];
if (opts.before !== undefined && opts.beforeId !== undefined) {
sql = 'SELECT * FROM notification_history WHERE node_id = ? AND stack_name = ? AND (timestamp < ? OR (timestamp = ? AND id < ?)) ORDER BY timestamp DESC, id DESC LIMIT ?';
args = [nodeId, stackName, opts.before, opts.before, opts.beforeId, opts.limit];
} else if (opts.before !== undefined) {
sql = 'SELECT * FROM notification_history WHERE node_id = ? AND stack_name = ? AND timestamp < ? ORDER BY timestamp DESC, id DESC LIMIT ?';
args = [nodeId, stackName, opts.before, opts.limit];
} else {
sql = 'SELECT * FROM notification_history WHERE node_id = ? AND stack_name = ? ORDER BY timestamp DESC, id DESC LIMIT ?';
args = [nodeId, stackName, opts.limit];
}
return (this.db.prepare(sql).all(...args) as unknown[]).map(row => this.mapNotificationRow(row as any));
}
@@ -2157,9 +2158,50 @@ export class DatabaseService {
stmt.run(cutoff);
}
public cleanupOldNotifications(daysToKeep = 30): void {
public cleanupOldNotifications(daysToKeep = 30, opts: { perStackCap?: number; perNodeUnattachedCap?: number } = {}): { ttl: number; perStack: number; perNode: number } {
const perStackCap = opts.perStackCap ?? 500;
const perNodeUnattachedCap = opts.perNodeUnattachedCap ?? 1000;
const cutoff = Date.now() - (daysToKeep * 24 * 60 * 60 * 1000);
this.db.prepare('DELETE FROM notification_history WHERE timestamp < ?').run(cutoff);
const ttlInfo = this.db.prepare('DELETE FROM notification_history WHERE timestamp < ?').run(cutoff);
const deleteById = this.db.prepare('DELETE FROM notification_history WHERE id = ?');
const deleteMany = this.db.transaction((ids: number[]) => {
for (const id of ids) deleteById.run(id);
});
// Per (node_id, stack_name) cap so a chatty stack cannot evict a quieter stack's history.
const stackOverflow = this.db.prepare(`
SELECT id FROM (
SELECT id, ROW_NUMBER() OVER (
PARTITION BY node_id, stack_name
ORDER BY timestamp DESC, id DESC
) AS rn
FROM notification_history
WHERE stack_name IS NOT NULL
)
WHERE rn > ?
`).all(perStackCap) as { id: number }[];
if (stackOverflow.length > 0) deleteMany(stackOverflow.map(r => r.id));
// Unattached system events have no stack to scope by, so they cannot share the per-stack quota; cap them per-node separately.
const unattachedOverflow = this.db.prepare(`
SELECT id FROM (
SELECT id, ROW_NUMBER() OVER (
PARTITION BY node_id
ORDER BY timestamp DESC, id DESC
) AS rn
FROM notification_history
WHERE stack_name IS NULL
)
WHERE rn > ?
`).all(perNodeUnattachedCap) as { id: number }[];
if (unattachedOverflow.length > 0) deleteMany(unattachedOverflow.map(r => r.id));
return {
ttl: Number(ttlInfo.changes ?? 0),
perStack: stackOverflow.length,
perNode: unattachedOverflow.length,
};
}
// --- Nodes ---