mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-24 12:42:49 +00:00
Keep alerts and backup protection honest when telemetry is unusable (#2459)
## What changed and why Monitoring should not interpret unusable telemetry as healthy protection or reliable alert evidence. - Treat NaN, infinities, invalid authoritative History samples and overflowing metric averages as unknown. Preserve the last trusted incident and notification state, then resume normal evaluation when finite observations return. Explicit rule disabling still works. - Keep malformed, future or otherwise unusable backup completion timestamps distinct from genuine absence. Existing workload rows, tooltips and guest details show cautionary uncertainty rather than healthy protection or an invented completion date. Running activity and valid freshness thresholds remain distinct. These are repairs to existing alert and backup-safety surfaces, not new features or native backup/thaw acceptance. ## Included source Complete range: `0f58485b52a59a21b944e73ed45637a8914b8a88..a1cde2ffe0bf86ea76bbd678e4bb0eb72860e72f`. - Alert candidate `1265ff10145c270f38666acfb68c13f2eb2399b6`, integrated at `f80f4b75ff46acaf7dd324208983f52a3be7171c`. - Backup timestamp candidate `a0b3c09e07bd700374a587db7908fa780d6d7631`, integrated at `1ff46c4261e11b4a8edcd573df415a217ecd34cd`. - Upstream reconciliation `01bf594d67c39ac4a63ab0b5611ee22eda790ea9` retains published main and every reviewed commit unchanged. - Additive test-fixture repair `a1cde2ffe0bf86ea76bbd678e4bb0eb72860e72f` supplies the real History response envelope to the PBS target E2E test. PR2457 run `37188438472` failed both chart assertions in three browser projects; its mock omitted range/start/end required by the reviewed response validator. Product validation, target checks and both-chart assertions are unchanged. ## Validation The combined product source passes the complete alerts package and frontend type checking plus 220 focused backup/workload tests in offline disposable source proofs. Retained backup-candidate evidence includes 548 connected tests, lint/audits, production build/budgets, 13 parent regressions and ten legitimate controls, plus twelve Chromium/WebKit production-component fixture paths. Browser receipt hashes match the final product files. The fixture correction is checked separately against the unchanged production History validator, including rejection of the old envelope and wrong-target/selection or missing-bound controls. This is a response-contract check, not a full E2E or chart-render pass. Exact-head hosted CI must verify the complete E2E journey. A cancelled earlier benchmark is not a pass. Local browser preparation failures from unavailable canonical dependencies remain explicit limitations. Provenance, clean-source, additive ancestry and diff checks pass. No accepted commit was rewritten; unchanged proofs were not repeated solely to combine candidates. ## Remaining limits No containing release, installed alert/notification relief, native backup/restore or guest thaw is established. Existing backup safety guidance and independent native acceptance remain necessary. Related to #2439 [How Pulse handles triage](https://github.com/rcourtman/Pulse/blob/main/docs/AI_TRANSPARENCY.md)
This commit is contained in:
pulse-triage[bot]
@@ -461,6 +461,20 @@ operator-visible message evidence. Window changes never change the canonical
|
||||
`metric-threshold:<metric>` identity, so start time, acknowledgement, history,
|
||||
timeline, escalation, and recovery stay one occurrence.
|
||||
|
||||
Current metric observations must be finite in both instantaneous and rolling
|
||||
evaluation, including a manager without a history provider. After selecting
|
||||
in-range observations and last-duplicate authority, any non-finite History
|
||||
sample or non-finite weighted result makes the whole window unknown. Invalid
|
||||
samples are not dropped to manufacture coverage, substituted with an older
|
||||
duplicate, clamped, or treated as zero. Both legacy and canonical evaluation
|
||||
retain the last trusted incident value, identity, acknowledgement and delivery
|
||||
state; no new lifecycle or notification evidence is emitted. Fresh finite
|
||||
evidence resumes normal evaluation, and explicitly disabling a rule still
|
||||
clears it independently of telemetry. `windowed_metric_test.go` verifies
|
||||
NaN/infinities, arithmetic overflow, duplicate authority, input isolation,
|
||||
JSON-safe incident snapshots, transition-ledger continuity and local dispatch.
|
||||
These source controls do not prove installed or external recipient acceptance.
|
||||
|
||||
Active-alert restore is opt-out at construction. `NewManagerWithDataDir` accepts
|
||||
`ManagerOption` values, and `WithoutPersistedAlertRestore` starts the manager
|
||||
with an empty active-alert set instead of reading `active-alerts.json`. Mock
|
||||
|
||||
@@ -20,6 +20,25 @@
|
||||
|
||||
## Purpose
|
||||
|
||||
### Backup age requires a usable completed timestamp
|
||||
|
||||
Existing workload indicators, age badges and drawer protection context distinguish
|
||||
an absent completed-backup timestamp from a present but unusable one. Finite,
|
||||
Date-representable positive timestamps no later than the current time may receive
|
||||
the configured freshness verdict. Future, malformed and out-of-range times are
|
||||
**Unknown**, cautionary rather than healthy or missing. The fixed explanation
|
||||
never renders the invalid raw value; future times direct the operator to check
|
||||
the Proxmox and browser clocks. An invalid current clock cannot confer freshness.
|
||||
Zero, empty and absent values retain the existing missing-backup interpretation.
|
||||
|
||||
Running activity remains separate: it neither validates the timestamp nor creates
|
||||
a completed backup. The badge, its tooltip and accessible name retain uncertainty
|
||||
while running; valid same-guest replacement restores the ordinary age verdict.
|
||||
`backupTimeEvidence.contract.test.tsx`, `format.test.ts` and
|
||||
`workloadGuestPresentation.test.ts` cover this admission and presentation. The
|
||||
real row/drawer browser fixture is `browser-tests/backup-time-evidence.cjs`.
|
||||
No backup execution, thaw, restored workload or installed acceptance is implied.
|
||||
|
||||
### Filesystem capacity is not measured usage
|
||||
|
||||
The existing filesystem list and guest Overview require finite positive capacity
|
||||
|
||||
@@ -15,6 +15,23 @@
|
||||
|
||||
## Purpose
|
||||
|
||||
### Workload backup projection preserves timestamp uncertainty
|
||||
|
||||
Both resource-API and owning-canonical-snapshot paths preserve a present malformed
|
||||
backup time for the existing shared age admission, rather than replacing it with
|
||||
the numeric zero absence sentinel. Only missing/empty values and the explicit Go
|
||||
zero-time value (including zero fractional seconds) map to absence. The drawer
|
||||
does not gate its backup presentation on timestamp truthiness: NaN must reach the
|
||||
unknown verdict, not disappear as a missing completed backup. Future dates cannot
|
||||
be clamped to Today. This adds only constant-time validation to the current mapper
|
||||
and presentation memos; it introduces no requests, timers or retained caches.
|
||||
|
||||
`useWorkloads.test.ts`, `GuestDrawer.test.tsx` and the focused backup time evidence
|
||||
controls pin the boundary. `browser-tests/backup-time-evidence.cjs` exercises real
|
||||
resource projection, rows, indicators and drawer replacement with synthetic API
|
||||
and canonical full/delta observations; those observations are not native backup
|
||||
or release proof.
|
||||
|
||||
### Guest discovery evidence and UI state have exact owners
|
||||
|
||||
The guest drawer's passive discovery lookup changes only when its explicit
|
||||
|
||||
@@ -0,0 +1,315 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const crypto = require('node:crypto');
|
||||
const { chromium, webkit } = require('playwright');
|
||||
|
||||
const root = '/workspace/frontend-modern';
|
||||
const output = '/workspace/tmp/backup-time/browser';
|
||||
const origin = 'http://127.0.0.1:5297';
|
||||
const now = '2026-10-04T08:00:00Z';
|
||||
const fresh = '2026-10-04T06:00:00Z';
|
||||
const future = '2026-10-04T08:01:00Z';
|
||||
const futureMessage =
|
||||
'Backup time unavailable: timestamp is in the future. Check the Proxmox and browser clocks.';
|
||||
const invalidMessage = 'Backup time unavailable: invalid timestamp.';
|
||||
const runtime = [
|
||||
'frontend-modern/src/utils/format.ts',
|
||||
'frontend-modern/src/utils/workloadGuestPresentation.ts',
|
||||
'frontend-modern/src/hooks/useWorkloads.ts',
|
||||
'frontend-modern/src/components/Workloads/GuestRowCells.tsx',
|
||||
'frontend-modern/src/components/Workloads/guestDrawerModel.ts',
|
||||
'frontend-modern/src/components/Workloads/useGuestDrawerState.ts',
|
||||
];
|
||||
const sha256 = (p) => crypto.createHash('sha256').update(fs.readFileSync(p)).digest('hex');
|
||||
|
||||
(async () => {
|
||||
fs.mkdirSync(output, { recursive: true });
|
||||
const playwrightVersion = require('playwright/package.json').version;
|
||||
assert.equal(
|
||||
playwrightVersion,
|
||||
JSON.parse(fs.readFileSync('/workspace/tests/integration/package-lock.json')).packages[
|
||||
'node_modules/@playwright/test'
|
||||
].version,
|
||||
);
|
||||
const hashes = Object.fromEntries(runtime.map((p) => [p, sha256(path.join('/workspace', p))]));
|
||||
process.chdir(root);
|
||||
const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js'));
|
||||
const server = await createServer({
|
||||
cacheDir: path.join(output, 'vite-cache'),
|
||||
server: { host: '127.0.0.1', port: 5297, strictPort: true },
|
||||
});
|
||||
const results = [];
|
||||
const screenshots = [];
|
||||
let browser;
|
||||
let phase = 'server';
|
||||
try {
|
||||
await server.listen();
|
||||
for (const [name, engine, width, dark] of [
|
||||
['chromium-desktop', chromium, 1365, false],
|
||||
['webkit-phone', webkit, 390, true],
|
||||
]) {
|
||||
browser = await engine.launch(
|
||||
engine === chromium
|
||||
? { headless: true, channel: 'chromium', args: ['--no-sandbox'] }
|
||||
: { headless: true },
|
||||
);
|
||||
for (const transport of ['api', 'canonical-full', 'canonical-delta']) {
|
||||
for (const kind of ['qemu', 'lxc']) {
|
||||
phase = `${name}/${transport}/${kind}`;
|
||||
const page = await browser.newPage({
|
||||
viewport: { width, height: width === 390 ? 844 : 900 },
|
||||
isMobile: width === 390,
|
||||
hasTouch: width === 390,
|
||||
locale: 'en-GB',
|
||||
timezoneId: 'UTC',
|
||||
});
|
||||
page.setDefaultTimeout(15000);
|
||||
page.setDefaultNavigationTimeout(60000);
|
||||
await page.clock.setFixedTime(new Date(now));
|
||||
let observation = { timestamp: fresh, running: false };
|
||||
let resourceReads = 0;
|
||||
const errors = [];
|
||||
const writes = [];
|
||||
const checks = [];
|
||||
page.on('pageerror', (error) => errors.push(error.message));
|
||||
await page.route('**/*', (route) => {
|
||||
const request = route.request();
|
||||
const url = new URL(request.url());
|
||||
assert.equal(url.origin, origin, 'no off-origin requests');
|
||||
if (!url.pathname.startsWith('/api/')) return route.continue();
|
||||
if (request.method() !== 'GET') writes.push(`${request.method()} ${url.pathname}`);
|
||||
let body = { success: true, data: [], alerts: [], anomalies: [], config: null };
|
||||
if (url.pathname === '/api/resources') {
|
||||
resourceReads++;
|
||||
body = {
|
||||
data: [
|
||||
{
|
||||
id: 'fixture-a-pve1-101',
|
||||
type: kind === 'lxc' ? 'system-container' : 'vm',
|
||||
name: 'backup-guest',
|
||||
status: 'running',
|
||||
sources: ['proxmox'],
|
||||
metrics: { cpu: { percent: 10 }, memory: { percent: 25 } },
|
||||
proxmox: {
|
||||
instance: 'fixture-a',
|
||||
nodeName: 'pve1',
|
||||
vmid: 101,
|
||||
runtimeStatus: 'running',
|
||||
lastBackup: observation.timestamp,
|
||||
backupInProgress: observation.running,
|
||||
lock: observation.running ? 'backup' : '',
|
||||
disks: [],
|
||||
},
|
||||
},
|
||||
],
|
||||
meta: { totalPages: 1 },
|
||||
};
|
||||
}
|
||||
return route.fulfill({
|
||||
status: 200,
|
||||
contentType: 'application/json',
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
});
|
||||
if (dark)
|
||||
await page.addInitScript(() =>
|
||||
document.addEventListener('DOMContentLoaded', () =>
|
||||
document.documentElement.classList.add('dark'),
|
||||
),
|
||||
);
|
||||
await page.goto(
|
||||
`${origin}/browser-tests/backup-time-evidence.html?transport=${transport}&kind=${kind}`,
|
||||
{ waitUntil: 'domcontentloaded' },
|
||||
);
|
||||
const row = page.getByRole('region', { name: 'Backup age row' });
|
||||
const badge = row.locator('[aria-label^="Backup status:"]');
|
||||
const indicatorRow = page.getByRole('region', { name: 'Backup indicator row' });
|
||||
const indicator = indicatorRow.locator(
|
||||
'[aria-label^="Last backup:"], [aria-label^="No backup"], [aria-label^="Backup running now"], [aria-label^="Backup time unavailable:"]',
|
||||
);
|
||||
const details = page.getByTestId('guest-technical-details');
|
||||
await details.waitFor();
|
||||
await badge.waitFor();
|
||||
const identity = await page.evaluate(() => window.__backupTimeEvidence.identity());
|
||||
await page.evaluate(() => {
|
||||
window.__originalBackupRow = document.querySelector('[aria-label="Backup age row"] tr');
|
||||
window.__originalBackupDetails = document.querySelector(
|
||||
'[data-testid="guest-technical-details"]',
|
||||
);
|
||||
});
|
||||
const observe = async (timestamp, running = false) => {
|
||||
observation = { timestamp, running };
|
||||
await page.evaluate(
|
||||
async (value) => window.__backupTimeEvidence.update(value),
|
||||
observation,
|
||||
);
|
||||
await page.evaluate(async () => {
|
||||
await document.fonts.ready;
|
||||
await new Promise((r) => requestAnimationFrame(() => requestAnimationFrame(r)));
|
||||
});
|
||||
assert.equal(
|
||||
await page.evaluate(() => window.__backupTimeEvidence.identity()),
|
||||
identity,
|
||||
);
|
||||
assert(
|
||||
await page.evaluate(
|
||||
() =>
|
||||
window.__originalBackupRow ===
|
||||
document.querySelector('[aria-label="Backup age row"] tr'),
|
||||
),
|
||||
);
|
||||
assert(
|
||||
await page.evaluate(
|
||||
() =>
|
||||
window.__originalBackupDetails ===
|
||||
document.querySelector('[data-testid="guest-technical-details"]'),
|
||||
),
|
||||
);
|
||||
};
|
||||
const capture = async (suffix) => {
|
||||
const p = path.join(output, `${name}-${kind}-${suffix}.png`);
|
||||
await page.screenshot({ path: p, fullPage: true });
|
||||
screenshots.push({ path: path.relative('/workspace', p), sha256: sha256(p) });
|
||||
};
|
||||
const checkUnknown = async (message, running) => {
|
||||
assert.equal((await badge.innerText()).trim(), running ? 'Running' : 'Unknown');
|
||||
assert((await badge.getAttribute('aria-label')).includes(message));
|
||||
assert(
|
||||
(await indicator.getAttribute('aria-label'))
|
||||
.toLowerCase()
|
||||
.includes(message.toLowerCase()),
|
||||
);
|
||||
assert(!(await badge.getAttribute('class')).includes('green'));
|
||||
assert(!(await indicator.getAttribute('class')).includes('green'));
|
||||
const reason = details.getByText(message, { exact: true });
|
||||
await reason.waitFor({ state: 'visible' });
|
||||
assert((await reason.locator('..').getAttribute('class')).includes('amber'));
|
||||
assert.equal(
|
||||
await details.getByText('No completed backup found', { exact: true }).count(),
|
||||
0,
|
||||
);
|
||||
assert.equal(await details.getByText('Today', { exact: true }).count(), 0);
|
||||
assert.equal(
|
||||
await details.getByText('Running · not completed yet', { exact: true }).count(),
|
||||
running ? 1 : 0,
|
||||
);
|
||||
assert(
|
||||
!(await details.innerText()).includes('NaN') &&
|
||||
!(await details.innerText()).includes('Invalid Date'),
|
||||
);
|
||||
checks.push(`${message} running=${running}: row, indicator, drawer and identity`);
|
||||
};
|
||||
|
||||
assert((await badge.getAttribute('class')).includes('green'));
|
||||
await details.getByText('Today', { exact: true }).waitFor();
|
||||
checks.push('valid initial completion retains fresh policy');
|
||||
await observe(future);
|
||||
await checkUnknown(futureMessage, false);
|
||||
if (transport === 'api' && kind === 'qemu') await capture('future');
|
||||
await observe(future, true);
|
||||
await checkUnknown(futureMessage, true);
|
||||
await observe('not-a-date', true);
|
||||
await checkUnknown(invalidMessage, true);
|
||||
if (name === 'chromium-desktop') {
|
||||
await badge.hover();
|
||||
const tooltip = page.locator('[data-tooltip-portal="true"]');
|
||||
await tooltip.waitFor({ state: 'visible' });
|
||||
assert((await tooltip.innerText()).includes(invalidMessage));
|
||||
assert(
|
||||
!(await tooltip.innerText()).includes('No backup has ever') &&
|
||||
!(await tooltip.innerText()).includes('Invalid Date'),
|
||||
);
|
||||
await page.locator('h1').hover();
|
||||
checks.push(
|
||||
'hover tooltip preserves timestamp uncertainty without invalid date or absence claim',
|
||||
);
|
||||
}
|
||||
if (transport === 'api' && kind === 'qemu') await capture('invalid-running');
|
||||
await observe('not-a-date');
|
||||
await checkUnknown(invalidMessage, false);
|
||||
await observe('0002-01-01T00:00:00Z');
|
||||
await checkUnknown(invalidMessage, false);
|
||||
for (const timestamp of [
|
||||
undefined,
|
||||
'',
|
||||
'0001-01-01T00:00:00Z',
|
||||
'0001-01-01T00:00:00.000000000Z',
|
||||
]) {
|
||||
await observe(timestamp);
|
||||
assert.equal((await badge.innerText()).trim(), 'None');
|
||||
assert((await badge.getAttribute('class')).includes('red'));
|
||||
await details.getByText('No completed backup found', { exact: true }).waitFor();
|
||||
checks.push(`explicit absence ${timestamp}: missing, not unknown`);
|
||||
}
|
||||
if (transport === 'api' && kind === 'lxc') await capture('missing');
|
||||
await observe('2026-09-24T08:00:00Z', true);
|
||||
await details.getByText('10d ago', { exact: true }).waitFor();
|
||||
assert.equal((await badge.innerText()).trim(), 'Running');
|
||||
assert((await badge.getAttribute('aria-label')).includes('last completed backup'));
|
||||
checks.push('valid overdue completion retained independently of running activity');
|
||||
await observe(fresh);
|
||||
assert((await badge.getAttribute('class')).includes('green'));
|
||||
assert((await indicator.getAttribute('class')).includes('green'));
|
||||
await details.getByText('Today', { exact: true }).waitFor();
|
||||
assert(!(await details.innerText()).includes('unavailable'));
|
||||
const close = page.getByRole('button', { name: 'Collapse backup-guest details' });
|
||||
if (width === 390) await close.tap();
|
||||
else {
|
||||
await close.focus();
|
||||
await page.keyboard.press('Enter');
|
||||
}
|
||||
assert.equal(await page.evaluate(() => window.__backupTimeEvidence.closed()), 1);
|
||||
assert(
|
||||
await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth),
|
||||
);
|
||||
checks.push(
|
||||
'valid recovery, same mounted surfaces, keyboard/touch collapse and no page overflow',
|
||||
);
|
||||
if (transport === 'api' && kind === 'qemu') await capture('recovered');
|
||||
assert.deepEqual(errors, []);
|
||||
assert.deepEqual(writes, []);
|
||||
assert.equal(
|
||||
resourceReads > 0,
|
||||
transport === 'api',
|
||||
'canonical path makes no parallel workload request',
|
||||
);
|
||||
results.push({
|
||||
name,
|
||||
browserVersion: browser.version(),
|
||||
transport,
|
||||
kind,
|
||||
identity,
|
||||
resourceReads,
|
||||
checks,
|
||||
errors,
|
||||
writes,
|
||||
});
|
||||
await page.close();
|
||||
}
|
||||
}
|
||||
await browser.close();
|
||||
browser = undefined;
|
||||
}
|
||||
fs.writeFileSync(
|
||||
path.join(output, 'result.json'),
|
||||
JSON.stringify(
|
||||
{ result: 'passed', playwrightVersion, runtime: hashes, results, screenshots },
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
} catch (error) {
|
||||
fs.writeFileSync(
|
||||
path.join(output, 'failure.json'),
|
||||
JSON.stringify({ phase, error: error.message, results, screenshots }, null, 2),
|
||||
);
|
||||
throw error;
|
||||
} finally {
|
||||
if (browser) await browser.close();
|
||||
await server.close();
|
||||
}
|
||||
})().catch((error) => {
|
||||
process.stderr.write(`${error.stack}\n`);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
@@ -0,0 +1,12 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Backup time evidence verification</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="./backup-time-evidence.tsx"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,119 @@
|
||||
// Production resource projection, rows and drawer; synthetic timestamps only.
|
||||
// This fixture performs no backup, guest-agent, restore or thaw operation.
|
||||
import { createSignal, Show } from 'solid-js';
|
||||
import { render } from 'solid-js/web';
|
||||
import { Router, Route } from '@solidjs/router';
|
||||
import { useWorkloads } from '../src/hooks/useWorkloads';
|
||||
import {
|
||||
mergeCanonicalResourceSnapshot,
|
||||
mergeCanonicalResourceDeltaSnapshot,
|
||||
} from '../src/utils/resourceStateAdapters';
|
||||
import { GuestRow } from '../src/components/Workloads/GuestRow';
|
||||
import { GuestDrawer } from '../src/components/Workloads/GuestDrawer';
|
||||
import { DarkModeContext } from '../src/contexts/appRuntime';
|
||||
import { syncAIRuntimeSettings } from '../src/stores/aiRuntimeState';
|
||||
import type { Resource } from '../src/types/resource';
|
||||
import '../src/index.css';
|
||||
|
||||
syncAIRuntimeSettings({ discovery_enabled: false });
|
||||
type Observation = { timestamp?: string; running?: boolean };
|
||||
const initial: Observation = { timestamp: '2026-10-04T06:00:00Z', running: false };
|
||||
const params = new URLSearchParams(window.location.search);
|
||||
const kind = params.get('kind') === 'lxc' ? 'system-container' : 'vm';
|
||||
const nativeResource = (value: Observation): Resource => ({
|
||||
id: 'fixture-a-pve1-101',
|
||||
type: kind,
|
||||
name: 'backup-guest',
|
||||
displayName: 'backup-guest',
|
||||
platformType: 'proxmox-pve',
|
||||
sourceType: 'api',
|
||||
status: 'online',
|
||||
sources: ['proxmox'],
|
||||
lastSeen: Date.UTC(2026, 9, 4, 8),
|
||||
cpu: { current: 10 },
|
||||
memory: { current: 25, used: 1024 ** 3 / 4, total: 1024 ** 3 },
|
||||
disk: { current: 50, used: 5 * 1024 ** 3, total: 10 * 1024 ** 3 },
|
||||
proxmox: {
|
||||
instance: 'fixture-a',
|
||||
nodeName: 'pve1',
|
||||
vmid: 101,
|
||||
runtimeStatus: 'running',
|
||||
template: false,
|
||||
lastBackup: value.timestamp,
|
||||
backupInProgress: value.running ?? false,
|
||||
lock: value.running ? 'backup' : '',
|
||||
disks: [],
|
||||
},
|
||||
});
|
||||
|
||||
function Fixture() {
|
||||
const transport = params.get('transport') || 'api';
|
||||
const [snapshot, setSnapshot] = createSignal<Resource[]>(
|
||||
mergeCanonicalResourceSnapshot([nativeResource(initial)], []),
|
||||
);
|
||||
const state = useWorkloads(() => true, transport === 'api' ? {} : { resourceSnapshot: snapshot });
|
||||
const guest = () => state.workloads()[0]!;
|
||||
const [closed, setClosed] = createSignal(0);
|
||||
(window as any).__backupTimeEvidence = {
|
||||
update: async (value: Observation) => {
|
||||
if (transport === 'api') {
|
||||
await state.refetch();
|
||||
} else {
|
||||
const incoming = nativeResource(value);
|
||||
setSnapshot((previous) =>
|
||||
transport === 'canonical-full'
|
||||
? mergeCanonicalResourceSnapshot([incoming], previous)
|
||||
: mergeCanonicalResourceDeltaSnapshot([incoming], previous, new Set([incoming.id])),
|
||||
);
|
||||
}
|
||||
},
|
||||
identity: () => guest().id,
|
||||
closed,
|
||||
};
|
||||
return (
|
||||
<main class="min-h-screen space-y-3 bg-surface p-3 text-base-content">
|
||||
<h1 class="text-base font-semibold">Backup time evidence</h1>
|
||||
<p class="text-xs text-muted">Synthetic {transport} timestamps; no backup command is sent.</p>
|
||||
<Show when={Boolean(guest())}>
|
||||
<section aria-label="Backup age row">
|
||||
<table class="w-full table-fixed">
|
||||
<tbody>
|
||||
<GuestRow
|
||||
guest={guest()}
|
||||
visibleColumnIds={['name', 'backup']}
|
||||
workloadTableLayoutMode={window.innerWidth <= 768 ? 'phone' : 'wide'}
|
||||
/>
|
||||
</tbody>
|
||||
</table>
|
||||
</section>
|
||||
<section aria-label="Backup indicator row">
|
||||
<table class="w-full table-fixed">
|
||||
<tbody>
|
||||
<GuestRow guest={guest()} visibleColumnIds={['name']} />
|
||||
</tbody>
|
||||
</table>
|
||||
</section>
|
||||
<section aria-label="Guest details">
|
||||
<GuestDrawer guest={guest()} onClose={() => setClosed((n) => n + 1)} />
|
||||
</section>
|
||||
</Show>
|
||||
</main>
|
||||
);
|
||||
}
|
||||
render(
|
||||
() => (
|
||||
<Router>
|
||||
<Route
|
||||
path="*"
|
||||
component={() => (
|
||||
<DarkModeContext.Provider
|
||||
value={() => document.documentElement.classList.contains('dark')}
|
||||
>
|
||||
<Fixture />
|
||||
</DarkModeContext.Provider>
|
||||
)}
|
||||
/>
|
||||
</Router>
|
||||
),
|
||||
document.getElementById('root')!,
|
||||
);
|
||||
@@ -0,0 +1,55 @@
|
||||
{
|
||||
"version": 1,
|
||||
"result": "passed",
|
||||
"base_sha": "fe42207eb4bfb04461cc780f3f32d3d7bcbe4d4b",
|
||||
"content_sha256": {
|
||||
"frontend-modern/src/utils/format.ts": "52db9924828b2d41af130ce836b9dd41e0b697600b2f1b4afed73c218046b9ea",
|
||||
"frontend-modern/src/utils/workloadGuestPresentation.ts": "d9a8dfb35b6bbdbb9fde65c724a2b8b1182e1fff3b199dfb55121fdf968b8865",
|
||||
"frontend-modern/src/hooks/useWorkloads.ts": "eb7f649920e134a9e9759324fec319c050f30baceb4c8a662640397f6489256f",
|
||||
"frontend-modern/src/components/Workloads/GuestRowCells.tsx": "43c2c631d279719c6f311ec23591c7d62792ca57bf33d3bd847bfad17f69cfd8",
|
||||
"frontend-modern/src/components/Workloads/guestDrawerModel.ts": "aaf8b1b31d228461988bead4ad3949aafac8cb51043429b69ae0908539024ffc",
|
||||
"frontend-modern/src/components/Workloads/useGuestDrawerState.ts": "63e46feb843079e27eafb84c65f6985674f576b4b5fbf331f58d77337fe24dc1"
|
||||
},
|
||||
"changed_paths": [
|
||||
"frontend-modern/src/components/Workloads/GuestRowCells.tsx",
|
||||
"frontend-modern/src/components/Workloads/guestDrawerModel.ts",
|
||||
"frontend-modern/src/components/Workloads/useGuestDrawerState.ts",
|
||||
"frontend-modern/src/hooks/useWorkloads.ts",
|
||||
"frontend-modern/src/utils/format.ts",
|
||||
"frontend-modern/src/utils/workloadGuestPresentation.ts"
|
||||
],
|
||||
"routes": [
|
||||
"Existing Proxmox VM/LXC workload row, hidden-column backup indicator and guest Overview through production-component fixture"
|
||||
],
|
||||
"states": [
|
||||
"Valid fresh/overdue completions",
|
||||
"Future/malformed/non-positive timestamps: unknown, not fresh or missing",
|
||||
"Running activity with unknown completion age",
|
||||
"Absent, empty and explicit Go zero-time timestamps",
|
||||
"Same-guest valid recovery without remount"
|
||||
],
|
||||
"interactions": [
|
||||
"Resource API replacement",
|
||||
"Canonical full/delta replacement",
|
||||
"Desktop hover tooltip",
|
||||
"Keyboard Enter collapse",
|
||||
"Touch collapse"
|
||||
],
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1365,
|
||||
"height": 900
|
||||
},
|
||||
{
|
||||
"width": 390,
|
||||
"height": 844
|
||||
}
|
||||
],
|
||||
"verified_at": "2026-10-04T08:10:06.767899Z",
|
||||
"browsers": [
|
||||
"Chromium desktop",
|
||||
"WebKit phone emulation; not native iOS"
|
||||
],
|
||||
"command": "pulse-worker-browser frontend-modern/browser-tests/backup-time-evidence.cjs",
|
||||
"proof": "12 production-component fixture paths, 150 state/identity/accessibility assertions with synthetic timestamps. No native backup, appliance, thaw, restore or shipped acceptance."
|
||||
}
|
||||
@@ -220,6 +220,23 @@ afterEach(() => {
|
||||
// ── Tests ──────────────────────────────────────────────────────────────
|
||||
|
||||
describe('GuestDrawer', () => {
|
||||
it('keeps malformed and future backup times unknown through same-guest replacement', () => {
|
||||
const [guest, setGuest] = createSignal(makeGuest({ lastBackup: NaN }));
|
||||
render(() => <GuestDrawer guest={guest()} onClose={vi.fn()} />);
|
||||
const details = technicalDetails();
|
||||
expect(details.getByText('Backup time unavailable: invalid timestamp.')).toBeVisible();
|
||||
expect(details.queryByText('No completed backup found')).not.toBeInTheDocument();
|
||||
expect(details.queryByText('Today')).not.toBeInTheDocument();
|
||||
setGuest({ ...guest(), lastBackup: Date.now() + 86_400_000, backupInProgress: true });
|
||||
expect(details.getByText(/timestamp is in the future/)).toBeVisible();
|
||||
expect(details.getByText('Running · not completed yet')).toBeVisible();
|
||||
setGuest({ ...guest(), lastBackup: Date.now() - 3600_000, backupInProgress: false });
|
||||
expect(details.getByText('Today')).toBeVisible();
|
||||
expect(details.queryByText(/unavailable/)).not.toBeInTheDocument();
|
||||
setGuest({ ...guest(), lastBackup: 0 });
|
||||
expect(details.getByText('No completed backup found')).toBeVisible();
|
||||
});
|
||||
|
||||
it('keeps deferred filesystem fallback last-known through History and fresh resumption', async () => {
|
||||
chartsApiMocks.getMetricsHistory.mockResolvedValue({
|
||||
resourceType: 'vm',
|
||||
|
||||
@@ -76,6 +76,7 @@ function getBackupAgeBadgeLabel(
|
||||
info: BackupInfo,
|
||||
): string {
|
||||
if (info.status === 'never') return 'None';
|
||||
if (info.status === 'unknown') return 'Unknown';
|
||||
const compact = formatRelativeTime(lastBackup ?? undefined, {
|
||||
compact: true,
|
||||
emptyText: 'Unknown',
|
||||
@@ -99,6 +100,7 @@ function getBackupAgeBadgeClass(status: WorkloadsGuestBackupDisplayStatus): stri
|
||||
case 'stale':
|
||||
return `${pill} border-yellow-200 bg-yellow-50 text-yellow-700 dark:border-yellow-900/70 dark:bg-yellow-950/40 dark:text-yellow-300`;
|
||||
case 'overdue':
|
||||
case 'unknown':
|
||||
return `${pill} border-yellow-200 bg-yellow-50 text-yellow-700 dark:border-yellow-900/70 dark:bg-yellow-950/40 dark:text-yellow-300`;
|
||||
case 'never':
|
||||
return `${pill} border-red-200 bg-red-50 text-red-700 dark:border-red-900/70 dark:bg-red-950/40 dark:text-red-300`;
|
||||
@@ -332,9 +334,12 @@ function BackupStatusCell(props: {
|
||||
const badgeLabel = createMemo(() =>
|
||||
displayStatus() === 'running' ? 'Running' : getBackupAgeBadgeLabel(props.lastBackup, info()),
|
||||
);
|
||||
const hasCompletedBackup = createMemo(() => info().status !== 'never');
|
||||
const hasCompletedBackup = createMemo(() => info().ageMs !== null);
|
||||
const ariaLabel = createMemo(() => {
|
||||
const currentInfo = info();
|
||||
if (currentInfo.status === 'unknown') {
|
||||
return `Backup status: ${props.backupRunning ? 'backup running now, ' : ''}${currentInfo.ageFormatted}`;
|
||||
}
|
||||
if (props.backupRunning) {
|
||||
return hasCompletedBackup()
|
||||
? `Backup status: backup running now, last completed backup ${currentInfo.ageFormatted}`
|
||||
@@ -387,7 +392,10 @@ function BackupStatusCell(props: {
|
||||
</span>
|
||||
</div>
|
||||
</Show>
|
||||
<Show when={!hasCompletedBackup()}>
|
||||
<Show when={info().status === 'unknown'}>
|
||||
<div class="py-0.5 text-amber-400">{info().ageFormatted}</div>
|
||||
</Show>
|
||||
<Show when={info().status === 'never'}>
|
||||
<div class="py-0.5 text-red-400">
|
||||
{props.backupRunning
|
||||
? 'No completed backup yet - the first backup is running now.'
|
||||
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { createSignal } from 'solid-js';
|
||||
import { cleanup, render, screen } from '@solidjs/testing-library';
|
||||
import { getBackupInfo } from '@/utils/format';
|
||||
import { BackupIndicator, BackupStatusCell } from '../GuestRowCells';
|
||||
import { getGuestDrawerBackupPresentation } from '../guestDrawerModel';
|
||||
|
||||
vi.mock('@/stores/alertsActivation', () => ({
|
||||
useAlertsActivation: () => ({ getBackupThresholds: () => undefined }),
|
||||
}));
|
||||
|
||||
const now = new Date('2026-10-04T08:00:00Z');
|
||||
const future = now.getTime() + 60_000;
|
||||
const invalidMessage = 'Backup time unavailable: invalid timestamp.';
|
||||
const futureMessage =
|
||||
'Backup time unavailable: timestamp is in the future. Check the Proxmox and browser clocks.';
|
||||
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
describe('backup time evidence', () => {
|
||||
it.each([
|
||||
['malformed', 'not-a-date', invalidMessage],
|
||||
['NaN', NaN, invalidMessage],
|
||||
['infinity', Infinity, invalidMessage],
|
||||
['negative', -1, invalidMessage],
|
||||
['outside Date range', 8.64e15 + 1, invalidMessage],
|
||||
['future numeric', future, futureMessage],
|
||||
['future ISO', new Date(future).toISOString(), futureMessage],
|
||||
] as const)(
|
||||
'regression: %s is uncertainty, never freshness or absence',
|
||||
(_name, value, message) => {
|
||||
const info = getBackupInfo(value, undefined, now);
|
||||
expect(info).toEqual({ status: 'unknown', ageMs: null, ageFormatted: message });
|
||||
const drawer = getGuestDrawerBackupPresentation(value, undefined, now);
|
||||
expect(drawer.ageLabel).toBe(message);
|
||||
expect(drawer.ageClass).not.toContain('green');
|
||||
expect(drawer.ageClass).not.toContain('red');
|
||||
expect(drawer.dateLabel).toBe('Unknown');
|
||||
},
|
||||
);
|
||||
|
||||
it.each([NaN, Infinity, new Date(NaN), 8.64e15 + 1])(
|
||||
'regression: an unusable current time %s cannot confer freshness',
|
||||
(clock) => {
|
||||
expect(getBackupInfo(now.getTime() - 3600_000, undefined, clock)).toEqual({
|
||||
status: 'unknown',
|
||||
ageMs: null,
|
||||
ageFormatted: 'Backup age unavailable: invalid current time.',
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it.each([null, undefined, '', 0])('control: absence sentinel %s stays missing', (value) => {
|
||||
expect(getBackupInfo(value, undefined, now)).toEqual({
|
||||
status: 'never',
|
||||
ageMs: null,
|
||||
ageFormatted: 'Never',
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
[0, 'fresh'],
|
||||
[24 * 3600_000, 'fresh'],
|
||||
[24 * 3600_000 + 1, 'stale'],
|
||||
[72 * 3600_000, 'stale'],
|
||||
[72 * 3600_000 + 1, 'overdue'],
|
||||
] as const)('control: a valid age of %s retains %s', (age, status) => {
|
||||
const value = now.getTime() - age;
|
||||
expect(getBackupInfo(value, undefined, now).status).toBe(status);
|
||||
expect(getBackupInfo(new Date(value).toISOString(), undefined, now).status).toBe(status);
|
||||
});
|
||||
|
||||
it('control: valid policy-relative thresholds stay in effect', () => {
|
||||
expect(
|
||||
getBackupInfo(now.getTime() - 15 * 24 * 3600_000, { freshHours: 16 * 24 }, now).status,
|
||||
).toBe('fresh');
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
'regression: row and indicator keep uncertainty while running=%s',
|
||||
(running) => {
|
||||
vi.useFakeTimers();
|
||||
vi.setSystemTime(now);
|
||||
const [timestamp, setTimestamp] = createSignal<string | number>(future);
|
||||
render(() => (
|
||||
<div>
|
||||
<BackupStatusCell lastBackup={timestamp()} backupRunning={running} />
|
||||
<BackupIndicator lastBackup={timestamp()} backupRunning={running} isTemplate={false} />
|
||||
</div>
|
||||
));
|
||||
const badge = screen.getByLabelText(/^Backup status:/);
|
||||
expect(badge).toHaveTextContent(running ? 'Running' : 'Unknown');
|
||||
expect(badge).toHaveAccessibleName(new RegExp('timestamp is in the future'));
|
||||
expect(badge.className).not.toContain('green');
|
||||
const indicator = screen.getByLabelText(
|
||||
running ? /^Backup running now · backup time unavailable/ : /^Backup time unavailable/,
|
||||
);
|
||||
expect(indicator.className).not.toContain('green');
|
||||
|
||||
setTimestamp('invalid');
|
||||
expect(badge).toHaveAccessibleName(new RegExp('invalid timestamp'));
|
||||
expect(badge).not.toHaveAccessibleName(/no completed backup/);
|
||||
setTimestamp(now.getTime() - 3600_000);
|
||||
expect(badge).not.toHaveAccessibleName(/unavailable/);
|
||||
expect(badge.className).toContain(running ? 'blue' : 'green');
|
||||
setTimestamp(0);
|
||||
expect(badge).toHaveTextContent(running ? 'Running' : 'None');
|
||||
expect(badge).toHaveAccessibleName(/no (completed )?backup/);
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -430,17 +430,22 @@ export const normalizeGuestDrawerTags = (tags: Guest['tags']): string[] => {
|
||||
};
|
||||
|
||||
export const getGuestDrawerBackupPresentation = (
|
||||
lastBackup: string | number | Date,
|
||||
lastBackup: string | number | Date | null | undefined,
|
||||
thresholds?: BackupThresholds,
|
||||
now: Date = new Date(),
|
||||
): GuestDrawerBackupPresentation => {
|
||||
const backupDate = new Date(lastBackup);
|
||||
const daysSince = Math.max(
|
||||
0,
|
||||
Math.floor((now.getTime() - backupDate.getTime()) / (1000 * 60 * 60 * 24)),
|
||||
);
|
||||
const info = getBackupInfo(backupDate.getTime(), thresholds, now);
|
||||
const timestamp = lastBackup instanceof Date ? lastBackup.getTime() : lastBackup;
|
||||
const info = getBackupInfo(timestamp, thresholds, now);
|
||||
const statusPresentation = getWorkloadsGuestBackupStatusPresentation(info.status);
|
||||
if (info.ageMs === null) {
|
||||
return {
|
||||
ageClass: statusPresentation.color,
|
||||
ageLabel: info.status === 'never' ? 'No completed backup found' : info.ageFormatted,
|
||||
dateLabel: 'Unknown',
|
||||
};
|
||||
}
|
||||
const backupDate = new Date(timestamp!);
|
||||
const daysSince = Math.floor(info.ageMs / (1000 * 60 * 60 * 24));
|
||||
|
||||
return {
|
||||
ageClass: statusPresentation.color,
|
||||
|
||||
@@ -99,12 +99,10 @@ export function useGuestDrawerState(props: GuestDrawerProps) {
|
||||
const hasNetworkInterfaces = createMemo(() => networkInterfaces().length > 0);
|
||||
const normalizedTags = createMemo(() => normalizeGuestDrawerTags(props.guest.tags));
|
||||
const backupPresentation = createMemo(() =>
|
||||
props.guest.lastBackup
|
||||
? getGuestDrawerBackupPresentation(
|
||||
props.guest.lastBackup,
|
||||
alertsActivation.getBackupThresholds(),
|
||||
)
|
||||
: null,
|
||||
getGuestDrawerBackupPresentation(
|
||||
props.guest.lastBackup,
|
||||
alertsActivation.getBackupThresholds(),
|
||||
),
|
||||
);
|
||||
const hasDiscoverySupport = createMemo(
|
||||
() => discoveryFeatureEnabled() && hasDiscoverySupportForWorkload(props.guest),
|
||||
|
||||
@@ -80,6 +80,25 @@ describe('useWorkloads', () => {
|
||||
resetWorkloadsCacheForTests();
|
||||
});
|
||||
|
||||
it.each(['invalid', '0002-01-01T00:00:00Z'])(
|
||||
'does not turn a present unusable backup time %s into absence',
|
||||
async (lastBackup) => {
|
||||
const [snapshot] = createSignal([{ ...sampleResource, proxmox: { lastBackup } }] as any);
|
||||
let dispose = () => {};
|
||||
let result: ReturnType<UseWorkloadsModule['useWorkloads']> | undefined;
|
||||
createRoot((d) => {
|
||||
dispose = d;
|
||||
result = useWorkloads(() => true, { resourceSnapshot: snapshot });
|
||||
});
|
||||
try {
|
||||
await flushAsync();
|
||||
expect(result!.workloads()[0]?.lastBackup).not.toBe(0);
|
||||
} finally {
|
||||
dispose();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('preserves linked-agent RAID arrays from the VM resource for guest detail', async () => {
|
||||
const [snapshot] = createSignal([
|
||||
{
|
||||
|
||||
@@ -601,10 +601,11 @@ const mapResourceToWorkload = (resource: APIResource): WorkloadGuest | null => {
|
||||
uptime: uptime ?? 0,
|
||||
template: resource.proxmox?.template ?? false,
|
||||
lastBackup: (() => {
|
||||
if (!resource.proxmox?.lastBackup) return 0;
|
||||
const parsed = Date.parse(resource.proxmox.lastBackup);
|
||||
// Go zero time "0001-01-01T00:00:00Z" parses to a large negative number
|
||||
return parsed > 0 ? parsed : 0;
|
||||
const timestamp = resource.proxmox?.lastBackup?.trim();
|
||||
// Go zero time is an explicit absence sentinel. Keep other invalid
|
||||
// readings (NaN included) distinct so the UI can report uncertainty.
|
||||
if (!timestamp || /^0001-01-01T00:00:00(?:\.0+)?Z$/.test(timestamp)) return 0;
|
||||
return Date.parse(timestamp);
|
||||
})(),
|
||||
backupInProgress: resource.proxmox?.backupInProgress ?? false,
|
||||
// vSphere's flat `resource.tags` is a mixed keyword set: the adapter keeps
|
||||
|
||||
@@ -242,11 +242,11 @@ describe('getBackupInfo', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('returns never status for invalid timestamp', () => {
|
||||
it('keeps invalid timestamps distinct from an absent backup', () => {
|
||||
expect(getBackupInfo('invalid')).toEqual({
|
||||
status: 'never',
|
||||
status: 'unknown',
|
||||
ageMs: null,
|
||||
ageFormatted: 'Never',
|
||||
ageFormatted: 'Backup time unavailable: invalid timestamp.',
|
||||
});
|
||||
expect(getBackupInfo(0)).toEqual({
|
||||
status: 'never',
|
||||
|
||||
@@ -44,6 +44,19 @@ describe('workloadGuestPresentation', () => {
|
||||
expect(getWorkloadsGuestBackupTooltip('stale', '3d')).toBe('Last backup: 3d');
|
||||
});
|
||||
|
||||
it('keeps an unknown backup age cautionary and distinct from no backup', () => {
|
||||
expect(getWorkloadsGuestBackupStatusPresentation('unknown')).toEqual({
|
||||
color: 'text-yellow-600 dark:text-yellow-400',
|
||||
bgColor: 'bg-yellow-100 dark:bg-yellow-900/25',
|
||||
icon: 'warning',
|
||||
});
|
||||
const reason = 'Backup time unavailable: invalid timestamp.';
|
||||
expect(getWorkloadsGuestBackupTooltip('unknown', reason)).toBe(reason);
|
||||
expect(getWorkloadsGuestBackupTooltip('unknown', reason, true)).toBe(
|
||||
`Backup running now · ${reason.toLowerCase()}`,
|
||||
);
|
||||
});
|
||||
|
||||
it('returns canonical compact protection context for object drawers', () => {
|
||||
expect(getWorkloadsGuestProtectionPresentation({})).toEqual({
|
||||
label: 'No completed backup found',
|
||||
|
||||
@@ -198,7 +198,7 @@ function formatTimeDiff(diffMs: number, compact?: boolean): string {
|
||||
}
|
||||
}
|
||||
|
||||
export type BackupStatus = 'fresh' | 'stale' | 'overdue' | 'never';
|
||||
export type BackupStatus = 'fresh' | 'stale' | 'overdue' | 'never' | 'unknown';
|
||||
|
||||
export interface BackupInfo {
|
||||
status: BackupStatus;
|
||||
@@ -227,7 +227,9 @@ export function getBackupInfo(
|
||||
thresholds?: BackupThresholds,
|
||||
now: number | Date = Date.now(),
|
||||
): BackupInfo {
|
||||
if (!lastBackup) {
|
||||
// Only the explicit absence sentinels mean no completed backup was found.
|
||||
// A present but unusable timestamp is uncertainty, not absence or freshness.
|
||||
if (lastBackup === null || lastBackup === undefined || lastBackup === '' || lastBackup === 0) {
|
||||
return { status: 'never', ageMs: null, ageFormatted: 'Never' };
|
||||
}
|
||||
|
||||
@@ -238,11 +240,34 @@ export function getBackupInfo(
|
||||
timestamp = lastBackup;
|
||||
}
|
||||
|
||||
if (isNaN(timestamp) || timestamp <= 0) {
|
||||
return { status: 'never', ageMs: null, ageFormatted: 'Never' };
|
||||
if (
|
||||
!Number.isFinite(timestamp) ||
|
||||
timestamp <= 0 ||
|
||||
!Number.isFinite(new Date(timestamp).getTime())
|
||||
) {
|
||||
return {
|
||||
status: 'unknown',
|
||||
ageMs: null,
|
||||
ageFormatted: 'Backup time unavailable: invalid timestamp.',
|
||||
};
|
||||
}
|
||||
|
||||
const nowMs = typeof now === 'number' ? now : now.getTime();
|
||||
if (!Number.isFinite(nowMs) || !Number.isFinite(new Date(nowMs).getTime())) {
|
||||
return {
|
||||
status: 'unknown',
|
||||
ageMs: null,
|
||||
ageFormatted: 'Backup age unavailable: invalid current time.',
|
||||
};
|
||||
}
|
||||
if (timestamp > nowMs) {
|
||||
return {
|
||||
status: 'unknown',
|
||||
ageMs: null,
|
||||
ageFormatted:
|
||||
'Backup time unavailable: timestamp is in the future. Check the Proxmox and browser clocks.',
|
||||
};
|
||||
}
|
||||
const ageMs = nowMs - timestamp;
|
||||
|
||||
// Use provided thresholds or fall back to defaults
|
||||
|
||||
@@ -40,6 +40,11 @@ const BACKUP_STATUS_PRESENTATION: Record<
|
||||
bgColor: 'bg-red-100 dark:bg-red-900/25',
|
||||
icon: 'x',
|
||||
},
|
||||
unknown: {
|
||||
color: 'text-yellow-600 dark:text-yellow-400',
|
||||
bgColor: 'bg-yellow-100 dark:bg-yellow-900/25',
|
||||
icon: 'warning',
|
||||
},
|
||||
running: {
|
||||
color: 'text-blue-600 dark:text-blue-400',
|
||||
bgColor: 'bg-blue-100 dark:bg-blue-900/25',
|
||||
@@ -59,7 +64,11 @@ export function getWorkloadsGuestBackupTooltip(
|
||||
backupRunning?: boolean,
|
||||
): string {
|
||||
const base =
|
||||
status === 'never' ? 'No completed backup found' : `Last backup: ${ageFormatted || 'Unknown'}`;
|
||||
status === 'never'
|
||||
? 'No completed backup found'
|
||||
: status === 'unknown'
|
||||
? ageFormatted || 'Backup age unavailable'
|
||||
: `Last backup: ${ageFormatted || 'Unknown'}`;
|
||||
if (backupRunning) {
|
||||
return `Backup running now · ${base.toLowerCase()}`;
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ package alerts
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"math"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -66,6 +67,12 @@ func (m *Manager) evaluateMetricWindow(resourceID, resourceType, metricType stri
|
||||
provider := m.metricWindowProvider
|
||||
m.mu.RUnlock()
|
||||
|
||||
// Unknown telemetry is neither a trigger nor recovery evidence, including
|
||||
// instantaneous evaluation and embedders without a history provider.
|
||||
if !finiteMetricValue(current) {
|
||||
return metricWindowObservation{CurrentValue: current, WindowSeconds: windowSeconds}
|
||||
}
|
||||
|
||||
instant := metricWindowObservation{Value: current, CurrentValue: current, Ready: true}
|
||||
if windowSeconds <= 0 {
|
||||
return instant
|
||||
@@ -92,6 +99,9 @@ func (m *Manager) evaluateMetricWindow(resourceID, resourceType, metricType stri
|
||||
|
||||
func calculateMetricWindow(points []MetricWindowPoint, start, end time.Time, current float64, windowSeconds int) metricWindowObservation {
|
||||
result := metricWindowObservation{CurrentValue: current, WindowSeconds: windowSeconds}
|
||||
if !finiteMetricValue(current) {
|
||||
return result
|
||||
}
|
||||
filtered := make([]MetricWindowPoint, 0, len(points))
|
||||
for _, point := range points {
|
||||
if point.Timestamp.Before(start) || point.Timestamp.After(end) {
|
||||
@@ -112,6 +122,14 @@ func calculateMetricWindow(points []MetricWindowPoint, start, end time.Time, cur
|
||||
unique = append(unique, point)
|
||||
}
|
||||
result.SampleCount = len(unique)
|
||||
// Validate after duplicate authority is resolved. Dropping bad values would
|
||||
// invent continuous coverage across unavailable observations; using an older
|
||||
// finite duplicate would override the latest evidence.
|
||||
for _, point := range unique {
|
||||
if !finiteMetricValue(point.Value) {
|
||||
return result
|
||||
}
|
||||
}
|
||||
if len(unique) < minimumMetricWindowSamples {
|
||||
return result
|
||||
}
|
||||
@@ -141,11 +159,19 @@ func calculateMetricWindow(points []MetricWindowPoint, start, end time.Time, cur
|
||||
if weightedDuration <= 0 {
|
||||
return result
|
||||
}
|
||||
result.Value = weightedTotal / weightedDuration.Seconds()
|
||||
value := weightedTotal / weightedDuration.Seconds()
|
||||
if !finiteMetricValue(value) {
|
||||
return result
|
||||
}
|
||||
result.Value = value
|
||||
result.Ready = true
|
||||
return result
|
||||
}
|
||||
|
||||
func finiteMetricValue(value float64) bool {
|
||||
return !math.IsNaN(value) && !math.IsInf(value, 0)
|
||||
}
|
||||
|
||||
func metricWindowOptions(opts *metricOptions, metricType, resourceType string, observation metricWindowObservation) *metricOptions {
|
||||
if observation.WindowSeconds <= 0 {
|
||||
return opts
|
||||
|
||||
@@ -1,9 +1,15 @@
|
||||
package alerts
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"math"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts/eventlog"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
|
||||
)
|
||||
|
||||
func TestCalculateMetricWindowUsesTimeWeightedAverage(t *testing.T) {
|
||||
@@ -133,3 +139,237 @@ func TestWindowedMetricUnknownHistoryPreservesActiveIncident(t *testing.T) {
|
||||
t.Fatalf("expected unknown history to preserve the incident, got %+v", active)
|
||||
}
|
||||
}
|
||||
|
||||
func nonFiniteMetricSamples() map[string]float64 {
|
||||
return map[string]float64{"NaN": math.NaN(), "positive infinity": math.Inf(1), "negative infinity": math.Inf(-1)}
|
||||
}
|
||||
|
||||
func TestMetricEvaluationRejectsNonFiniteCurrent(t *testing.T) {
|
||||
for _, mode := range []string{"instant", "window without provider", "window with provider"} {
|
||||
for name, value := range nonFiniteMetricSamples() {
|
||||
t.Run(mode+"/"+name, func(t *testing.T) {
|
||||
m := NewManagerWithDataDir(t.TempDir(), WithoutPersistedAlertRestore())
|
||||
t.Cleanup(m.Stop)
|
||||
window := 0
|
||||
if mode != "instant" {
|
||||
window = 300
|
||||
}
|
||||
m.mu.Lock()
|
||||
m.config.MetricEvaluationWindows = map[string]map[string]int{"all": {"cpu": window}}
|
||||
m.mu.Unlock()
|
||||
calls := 0
|
||||
if mode == "window with provider" {
|
||||
m.SetMetricWindowProvider(func(MetricWindowRequest) ([]MetricWindowPoint, error) {
|
||||
calls++
|
||||
return nil, nil
|
||||
})
|
||||
}
|
||||
got := m.evaluateMetricWindow("vm-1", "vm", "cpu", value, time.Now())
|
||||
if got.Ready || calls != 0 {
|
||||
t.Fatalf("invalid current is not trustworthy or a reason to query history: ready=%v calls=%d", got.Ready, calls)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMetricWindowRejectsNonFiniteEvidence(t *testing.T) {
|
||||
end := time.Date(2026, 10, 4, 8, 0, 0, 0, time.UTC)
|
||||
start := end.Add(-4 * time.Minute)
|
||||
for name, value := range nonFiniteMetricSamples() {
|
||||
for _, position := range []string{"first", "middle", "last", "current"} {
|
||||
t.Run(name+"/"+position, func(t *testing.T) {
|
||||
points := []MetricWindowPoint{{start, 90}, {start.Add(2 * time.Minute), 90}, {end, 90}}
|
||||
current := 90.0
|
||||
switch position {
|
||||
case "first":
|
||||
points[0].Value = value
|
||||
case "middle":
|
||||
points[1].Value = value
|
||||
case "last":
|
||||
points[2].Value = value
|
||||
case "current":
|
||||
current = value
|
||||
}
|
||||
if got := calculateMetricWindow(points, start, end, current, 240); got.Ready {
|
||||
t.Fatalf("non-finite %s was marked ready: %+v", position, got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
for name, value := range map[string]float64{"positive overflow": math.MaxFloat64, "negative overflow": -math.MaxFloat64} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
points := []MetricWindowPoint{{start, value}, {start.Add(2 * time.Minute), value}, {end, value}}
|
||||
if got := calculateMetricWindow(points, start, end, value, 240); got.Ready {
|
||||
t.Fatalf("non-finite weighted result was marked ready: %+v", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestMetricWindowFiniteAuthorityAndInputIsolation(t *testing.T) {
|
||||
end := time.Date(2026, 10, 4, 8, 0, 0, 0, time.UTC)
|
||||
start := end.Add(-4 * time.Minute)
|
||||
for name, invalid := range nonFiniteMetricSamples() {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
// Unordered input and obsolete invalid duplicates are not evidence.
|
||||
points := []MetricWindowPoint{{end, invalid}, {start, 90}, {start.Add(2 * time.Minute), 90}, {end, 90}, {start.Add(-time.Second), invalid}, {end.Add(time.Second), invalid}}
|
||||
before := append([]MetricWindowPoint(nil), points...)
|
||||
got := calculateMetricWindow(points, start, end, 90, 240)
|
||||
if !got.Ready || got.Value != 90 || got.SampleCount != 3 || got.CoverageSeconds != 240 {
|
||||
t.Fatalf("last in-range duplicate must remain authoritative: %+v", got)
|
||||
}
|
||||
for i := range points {
|
||||
if points[i].Timestamp != before[i].Timestamp || math.Float64bits(points[i].Value) != math.Float64bits(before[i].Value) {
|
||||
t.Fatal("window evaluation mutated provider evidence")
|
||||
}
|
||||
}
|
||||
points = append(points, MetricWindowPoint{end, invalid})
|
||||
if got := calculateMetricWindow(points, start, end, 90, 240); got.Ready {
|
||||
t.Fatal("invalid latest duplicate must not resurrect the older finite value")
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, value := range []float64{-20, 0, 90, 1e300} {
|
||||
points := []MetricWindowPoint{{start, value}, {start.Add(2 * time.Minute), value}, {end, value}}
|
||||
got := calculateMetricWindow(points, start, end, value, 240)
|
||||
if !got.Ready || math.Abs(got.Value-value) > math.Abs(value)*1e-14 {
|
||||
t.Fatalf("finite values must not be clipped or made unknown: value=%v result=%+v", value, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNonFiniteMetricPreservesIncidentAndDeliveryEvidence(t *testing.T) {
|
||||
for _, mode := range []string{"legacy instant", "canonical instant", "legacy history", "canonical history"} {
|
||||
for name, invalid := range nonFiniteMetricSamples() {
|
||||
t.Run(mode+"/"+name, func(t *testing.T) {
|
||||
m := newEventLogManager(t)
|
||||
m.mu.Lock()
|
||||
m.config.TimeThresholds = map[string]int{}
|
||||
m.config.MinimumDelta = 0
|
||||
m.config.SuppressionWindow = 0
|
||||
m.config.FlappingEnabled = false
|
||||
window := 0
|
||||
withHistory := mode == "legacy history" || mode == "canonical history"
|
||||
if withHistory {
|
||||
window = 300
|
||||
}
|
||||
m.config.MetricEvaluationWindows = map[string]map[string]int{"all": {"cpu": window}}
|
||||
m.mu.Unlock()
|
||||
now := time.Date(2026, 10, 4, 8, 0, 0, 0, time.UTC)
|
||||
m.now = func() time.Time { return now }
|
||||
historyValue := 85.0
|
||||
m.SetMetricWindowProvider(func(request MetricWindowRequest) ([]MetricWindowPoint, error) {
|
||||
return []MetricWindowPoint{{request.Start, historyValue}, {request.Start.Add(2 * time.Minute), historyValue}, {request.Start.Add(4 * time.Minute), historyValue}}, nil
|
||||
})
|
||||
threshold := &HysteresisThreshold{Trigger: 80, Clear: 75}
|
||||
spec, err := buildCanonicalMetricSpec("vm-1", "web-1", unifiedresources.ResourceTypeVM, "cpu", threshold)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var deliveries atomic.Int64
|
||||
fired := make(chan struct{}, 8)
|
||||
m.SetAlertCallback(func(*Alert) { deliveries.Add(1); fired <- struct{}{} })
|
||||
evaluate := func(value float64) {
|
||||
if mode == "canonical instant" || mode == "canonical history" {
|
||||
m.evaluateCanonicalMetricAlert(spec, "web-1", "node-1", "pve-1", "vm", value, threshold, nil)
|
||||
} else {
|
||||
m.checkMetric("vm-1", "web-1", "node-1", "pve-1", "vm", "cpu", value, threshold, nil)
|
||||
}
|
||||
}
|
||||
evaluate(85)
|
||||
select {
|
||||
case <-fired:
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("finite trigger did not reach the asynchronous dispatch callback")
|
||||
}
|
||||
active := m.GetActiveAlerts()
|
||||
if len(active) != 1 || deliveries.Load() != 1 {
|
||||
t.Fatalf("finite trigger must fire and dispatch exactly once: active=%d deliveries=%d", len(active), deliveries.Load())
|
||||
}
|
||||
if err := m.AcknowledgeAlert(active[0].ID, "operator"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before, err := json.Marshal(m.GetActiveAlerts())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
eventsBefore := len(queryAlertEvents(t, m, eventlog.Filter{}))
|
||||
now = now.Add(time.Minute)
|
||||
if withHistory {
|
||||
historyValue = invalid
|
||||
evaluate(10)
|
||||
} else {
|
||||
evaluate(invalid)
|
||||
}
|
||||
after, err := json.Marshal(m.GetActiveAlerts())
|
||||
if err != nil || !bytes.Equal(before, after) || deliveries.Load() != 1 {
|
||||
t.Fatalf("unknown evidence changed acknowledged incident/delivery or broke JSON: error=%v before=%s after=%s deliveries=%d", err, before, after, deliveries.Load())
|
||||
}
|
||||
if got := len(queryAlertEvents(t, m, eventlog.Filter{})); got != eventsBefore {
|
||||
t.Fatalf("unknown evidence changed durable transition ledger: before=%d after=%d", eventsBefore, got)
|
||||
}
|
||||
historyValue = 10
|
||||
now = now.Add(time.Minute)
|
||||
evaluate(10)
|
||||
evaluate(10)
|
||||
if len(m.GetActiveAlerts()) != 0 || len(queryAlertEvents(t, m, eventlog.Filter{Types: []string{eventlog.TypeResolved}})) != 1 {
|
||||
t.Fatal("fresh finite evidence must recover the same incident exactly once")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNonFiniteMetricCannotActivateIncident(t *testing.T) {
|
||||
for _, window := range []int{0, 300} {
|
||||
for name, invalid := range nonFiniteMetricSamples() {
|
||||
t.Run((time.Duration(window)*time.Second).String()+"/"+name, func(t *testing.T) {
|
||||
m := newEventLogManager(t)
|
||||
m.mu.Lock()
|
||||
m.config.TimeThresholds = map[string]int{}
|
||||
m.config.MetricEvaluationWindows = map[string]map[string]int{"all": {"cpu": window}}
|
||||
m.mu.Unlock()
|
||||
m.SetMetricWindowProvider(func(request MetricWindowRequest) ([]MetricWindowPoint, error) {
|
||||
return []MetricWindowPoint{{request.Start, invalid}, {request.Start.Add(2 * time.Minute), invalid}, {request.Start.Add(4 * time.Minute), invalid}}, nil
|
||||
})
|
||||
var deliveries atomic.Int64
|
||||
m.SetAlertCallback(func(*Alert) { deliveries.Add(1) })
|
||||
value := invalid
|
||||
if window > 0 {
|
||||
value = 85
|
||||
}
|
||||
m.checkMetric("vm-1", "web-1", "node-1", "pve-1", "vm", "cpu", value, &HysteresisThreshold{Trigger: 80, Clear: 75}, nil)
|
||||
if len(m.GetActiveAlerts()) != 0 || deliveries.Load() != 0 || len(queryAlertEvents(t, m, eventlog.Filter{})) != 0 {
|
||||
t.Fatal("invalid evidence opened an incident or dispatched a notification")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNonFiniteMetricDoesNotPreventExplicitDisable(t *testing.T) {
|
||||
for _, canonical := range []bool{false, true} {
|
||||
m := newEventLogManager(t)
|
||||
m.mu.Lock()
|
||||
m.config.TimeThresholds = map[string]int{}
|
||||
m.config.MetricEvaluationWindows = map[string]map[string]int{"all": {"cpu": 0}}
|
||||
m.mu.Unlock()
|
||||
threshold := &HysteresisThreshold{Trigger: 80, Clear: 75}
|
||||
spec, err := buildCanonicalMetricSpec("vm-1", "web-1", unifiedresources.ResourceTypeVM, "cpu", threshold)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if canonical {
|
||||
m.evaluateCanonicalMetricAlert(spec, "web-1", "node-1", "pve-1", "vm", 85, threshold, nil)
|
||||
spec.Disabled = true
|
||||
m.evaluateCanonicalMetricAlert(spec, "web-1", "node-1", "pve-1", "vm", math.NaN(), threshold, nil)
|
||||
} else {
|
||||
m.checkMetric("vm-1", "web-1", "node-1", "pve-1", "vm", "cpu", 85, threshold, nil)
|
||||
m.checkMetric("vm-1", "web-1", "node-1", "pve-1", "vm", "cpu", math.NaN(), nil, nil)
|
||||
}
|
||||
if len(m.GetActiveAlerts()) != 0 || len(queryAlertEvents(t, m, eventlog.Filter{Types: []string{eventlog.TypeFired}})) != 1 || len(queryAlertEvents(t, m, eventlog.Filter{Types: []string{eventlog.TypeResolved}})) != 1 {
|
||||
t.Fatal("explicit disable must still resolve the active rule independently of telemetry")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -116,7 +116,18 @@ for (const width of [1280, 390]) {
|
||||
await page.route('**/api/metrics-store/history?**', async route => {
|
||||
const q = new URL(route.request().url()).searchParams;
|
||||
targets.push(`${q.get('resourceType')}/${q.get('resourceId')}`);
|
||||
await route.fulfill({ json: { metrics: { cpu: [0, 1, 2].map(i => ({ timestamp: Date.now() - (2-i)*60000, value: 10+i, min: 10+i, max: 10+i, count: 1 })), memory: [0, 1, 2].map(i => ({ timestamp: Date.now() - (2-i)*60000, value: 20+i, min: 20+i, max: 20+i, count: 1 })) }, resourceId: q.get('resourceId'), resourceType: q.get('resourceType') } });
|
||||
// Match the real history envelope. Response admission deliberately rejects
|
||||
// missing selection/range bounds, even when the metric points are valid.
|
||||
const end = Date.now();
|
||||
const start = end - 120_000;
|
||||
await route.fulfill({ json: {
|
||||
resourceId: q.get('resourceId'), resourceType: q.get('resourceType'),
|
||||
range: q.get('range') || '24h', start, end, source: 'store',
|
||||
metrics: {
|
||||
cpu: [0, 1, 2].map(i => ({ timestamp: start + i*60000, value: 10+i, min: 10+i, max: 10+i, count: 1 })),
|
||||
memory: [0, 1, 2].map(i => ({ timestamp: start + i*60000, value: 20+i, min: 20+i, max: 20+i, count: 1 })),
|
||||
},
|
||||
} });
|
||||
});
|
||||
await page.goto('/proxmox/backups', { waitUntil: 'domcontentloaded' });
|
||||
for (const type of ['vm', 'agent']) {
|
||||
|
||||
Reference in New Issue
Block a user