fix(fleet): gate node update actions to admins and harden update tracking (#1272)

* fix(fleet): gate node update actions to admins and harden update tracking

Node update affordances now render only for admins, matching the admin-only
routes behind them. Previously a non-admin could open the Fleet view and see the
per-node Update button, Update all, retry, dismiss, and Recheck controls, then
get a 403 on click. Those controls are now hidden for non-admins, who still see
read-only update status.

Both update-status clear routes (per-node and bulk) now require admin, and the
bulk recheck throttles its forced "latest published version" lookup so a caller
cannot loop it to hammer the upstream registries; the response reports whether
the refresh actually ran so the UI can surface a "checked recently" note.

Completion detection no longer reports a node as Updated when it merely blips
offline and returns on the same version with an unchanged process start time.
That case stays in progress and is decided by the existing early-fail and
timeout heuristics, so a momentary network glitch is not mistaken for a
successful update. Failed and timed-out updates now emit an operator-visible
warning, and a periodic safety-net sweep bounds in-flight trackers when no
client is polling for status.

* fix(fleet): harden update completion and recheck failure handling

Refinements from review of the node self-update hardening:

- Completion signal 1 now requires a valid version, not merely a different one.
  A node whose /api/meta momentarily omits or mangles its version (online, same
  process) reported version=null, which compared unequal to the previous version
  and falsely marked the update completed. It now stays in progress and is
  decided by the early-fail/timeout heuristics.

- Terminal resolution is atomic: it re-reads the live tracker and transitions
  only if it is still in flight with the same start time, so two concurrent
  status polls cannot both warn or clobber each other's transition.

- The operator warning for a failed or timed-out update now redacts
  secret-shaped text (bearer/basic/token/password, credentialed URLs) from the
  underlying error before logging, in addition to stripping control characters.

- The Recheck button now surfaces an error toast when the request throws
  (network or auth failure), matching the existing non-ok-response path instead
  of only logging to the console.
This commit is contained in:
Anso
2026-06-01 17:27:25 -04:00
committed by GitHub
parent 7d7e0a6264
commit 0953025036
11 changed files with 592 additions and 34 deletions
+19
View File
@@ -4,6 +4,7 @@ import DockerController from './DockerController';
import { DatabaseService, Node, StackAlert } from './DatabaseService';
import { NodeRegistry } from './NodeRegistry';
import { NotificationService } from './NotificationService';
import { FleetUpdateTrackerService } from './FleetUpdateTrackerService';
import { isValidVersion, getSenchoVersion } from './CapabilityRegistry';
import { getLatestVersion } from '../utils/version-check';
import { isDebugEnabled } from '../utils/debug';
@@ -235,6 +236,7 @@ export class MonitorService {
await this.evaluateGlobalSettings(settings);
await this.evaluateStackAlerts(db);
this.sweepStaleUpdateTrackers();
const elapsed = Date.now() - cycleStart;
if (elapsed > 25_000) {
@@ -250,6 +252,23 @@ export class MonitorService {
}
}
/**
* Safety net for the in-memory fleet update trackers. The
* /api/fleet/update-status poll is the primary resolver, but it only runs
* while a client is watching; this sweep bounds in-flight trackers (to
* timeout) and reaps stale completed badges even when nothing is polling.
* Cheap, synchronous, in-memory.
*/
private sweepStaleUpdateTrackers(): void {
const { timedOut, reaped } = FleetUpdateTrackerService.getInstance().sweepStale();
if (timedOut > 0) {
console.warn(`[Monitor] Timed out ${timedOut} stale fleet update tracker(s) (no status poll within the update window)`);
}
if (isDebugEnabled() && reaped > 0) {
console.debug(`[Monitor:diag] Reaped ${reaped} resolved fleet update tracker(s)`);
}
}
private async evaluateGlobalSettings(settings: Record<string, string>) {
// F-11 suppression window. Default 60 min; configurable via global
// settings. NaN / non-positive values fall back to the default so a