diff --git a/docs/release-control/v6/internal/subsystems/alerts.md b/docs/release-control/v6/internal/subsystems/alerts.md index fefabf9828..20fd82c066 100644 --- a/docs/release-control/v6/internal/subsystems/alerts.md +++ b/docs/release-control/v6/internal/subsystems/alerts.md @@ -3236,3 +3236,25 @@ The monitor delegates recovery eligibility to persisted occurrence/destination receipts even when an old retry remains or restart has lost RAM cooldown state. The partial-destination monitor control establishes a real local HTTP 200/503 split and preserves the newer firing and the old recipient's recovery. + +### Notification activity reports each evidence source independently + +The existing Notifications activity card distinguishes an initial or pending +read from a confirmed empty window. Held/deferred event reads have their own +loading and unavailable state; a failed read withdraws their prior rows and +keeps the warning until a later successful read. Only the current refresh owns +those states, and disposed readers cannot publish late completions. A slow +held-event read does not delay delivery attempts or disable their refresh. + +A delivery-attempt failure withdraws only attempt rows, not independently +readable held events. Conversely, missing held evidence does not hide readable +attempts or imply that no notifications were held. The combined empty state +requires both sources to be readable and no refresh pending. Retention, event +limits, queue actions, permissions and notification delivery policy are unchanged. + +`useNotificationDeliveryLog.test.tsx`, `useAlertDestinationsTabState.test.tsx` +and `AlertDeliveryLogCard.test.tsx` exercise independent failure, recovery, +initial loading and current-request ownership; the production Notifications +tab browser fixture verifies the connected read/retry states at desktop and +phone widths. These are synthetic frontend controls, not recipient delivery +or containing-release acceptance. diff --git a/docs/release-control/v6/internal/subsystems/deployment-installability.md b/docs/release-control/v6/internal/subsystems/deployment-installability.md index f1b2c505e5..39ff236c60 100644 --- a/docs/release-control/v6/internal/subsystems/deployment-installability.md +++ b/docs/release-control/v6/internal/subsystems/deployment-installability.md @@ -15,6 +15,23 @@ ## Purpose +### Update History consent does not infer the backup's restore scope + +The confirmation names the observed running server separately from the selected +entry's pre-/post-update versions. No version threshold or free-form notes stand +in for actual scope: the current History API does not describe the manifest or +active-store layout. Explain that installation-only recovery leaves active stores +in place but may restore independent installation files; legacy recovery can +also replace install-local configuration/data. Do not promise a settings/alerts +rewind. Full-state recovery still requires a stopped service and a consistent +backup of every active store, via the existing shipped recovery instructions. + +Opening/cancelling consent sends no rollback request, pending consent rejects +duplicate actions and cancellation, and a rejected request retains the warning +and refreshes history. Mounted UpdateHistory tests and the production-CSS offline +browser fixture exercise those states; neither proves a binary/data rollback. + + ### Demo transactions share the service host, not just a workflow queue Stable demo update and recovery keep their existing publication/activation, diff --git a/docs/release-control/v6/internal/subsystems/frontend-primitives.md b/docs/release-control/v6/internal/subsystems/frontend-primitives.md index 7f1fbe6186..23f91dd104 100644 --- a/docs/release-control/v6/internal/subsystems/frontend-primitives.md +++ b/docs/release-control/v6/internal/subsystems/frontend-primitives.md @@ -20,6 +20,61 @@ ## Purpose +### Guest disk deferral and retained observations + +The existing workload row, disk breakdown tooltip/list and guest Overview use +one presentation for the fixed guest-read reason vocabulary. Operation locks, +unverified lock clearance, an in-flight request, cooldown, incomplete response +and capacity deferrals are not evidence that QGA is missing. Invalid VM identity +remains distinct. A timeout describes uncertain completion and must not recommend +restarting QGA during a backup or changing freeze settings. + +`prev-` marks last-known disk values, not a fresh sample. Keep those values and +their identity visible, with the explanation beside the disk evidence, including +the breakdown tooltip and the phone-readable Overview. Show readable status even +before the first filesystem sample; do not hide it behind an empty disk list or +expose only a raw reason. A same-VM fresh observation withdraws the notice and +updates values without remounting the row. Permission, disabled-agent and actual +unavailable cases retain their own explanations. + +Verification: `workloadGuestPresentation.test.ts`, the mounted `GuestRow`, +`DiskList`, `StackedDiskBar` and `GuestDrawerOverview.filesystems` regressions, +and `browser-tests/guest-disk-deferral.cjs` cover the fixed reasons, retained/no +previous data and fresh resumption with production components and CSS on desktop +Chromium and touch WebKit. Synthetic observations do not establish the native +command-ID cause, guest thaw, fresh History collection or containing release. + + +### Composed operator tables and conservative rollback consent + +Phone storage tables use the canonical weighted-width helper and matching +header/cell gutters. Compact labels shorten disk verdicts and capacity values, +not their underlying health or measured units; full reasons stay in the real +disk header and tooltips. Structured ZFS scan activity supplies a compact badge +only during reported rebuild activity; its complete provider summary remains +available. Stable disk History catalog/organisation/access ownership is unchanged. + +Controller expansions carry the supplied kind-specific targets, absolute times, +duration and cluster/namespace when the phone row omits those columns. Proxmox +coverage distinguishes independent backups from guest-local snapshots; By date +retains archive identifiers in hover/search while showing the format as detail. +Phone identity weights leave every configured metric column visible. + +UpdateHistory consent names both the requested target and the observed running +version, separately from the version on the selected history entry. The API does +not expose backup restore scope: do not infer it from a version cutoff, notes or +a retained path. Explain installation-only versus legacy recovery and stopped, +consistent full-state recovery without promising later settings will be reverted +or preserved. Opening/cancelling never starts a write, rejection retains consent, +and a pending request cannot be submitted again or cancelled. + +Verification: the mounted UpdateHistory regression tests, existing kind-specific +controller/storage/backup/workload tests and production-CSS browser fixture +`frontend-modern/browser-tests/operator-tables-consent.cjs` exercise the composed +source at desktop, intermediate and phone widths. Synthetic HTTP/snapshots do +not establish native collectors, tenancy, installation recovery or release delivery. + + ### Organisation-scoped single-metric History Fetched HistoryChart observations belong to the active organisation as well as @@ -775,6 +830,28 @@ that manual-channel consequence must stay visible at desktop and narrow widths. ## Shared Boundaries +### Transport-independent operator rows + +Proxmox/Storage and Docker presenters retain the shared table/detail, responsive +column weighting, relative-time and governed action primitives. Display names +for Proxmox storage plugins preserve their operator vocabulary (LVM-Thin, NFS, +SMB/CIFS and the other supported acronyms); only explicitly shared storage on a +synthetic cluster/shared location gets the shared label. That label does not +alter canonical identity, parent, History target or pool-to-disk ownership. + +The Docker image phone projection pairs hidden headers and cells and gives +measured size and update verdict room without hiding identity or dropping the +existing detail disclosure. The container projection retains its shared 30% +phone identity reservation and five scan signals; off-row restart/host/consumer +evidence remains in detail. Swarm rollout tokens read as words, with compact +Running headings and wrapping rollout states on phones; raw rollout and +placement evidence remain reachable through keyboard/touch row disclosure. +`platformOverviewLayout.guardrails.test.ts`, mounted +Docker table tests and the desktop/touch-phone production-component browser +fixture check these priorities, rather than treating an old percentage as an +immutable primitive contract. No new public route or navigation entry is added. + + ### Maintenance schedule copy Operator maintenance banners show local absolute start and end timestamps, @@ -7856,3 +7933,21 @@ kind normalisation, the kind switch, the email-only payload, and the plain cover the gating, the reduced form, the pinned fields, and the saved payload. This is a presentation and form-state boundary; the schedule API contract, report rendering, and email delivery are unchanged. + +### Notifications activity has independent evidence-read states + +The Notifications tab composes the delivery-log card with both attempt-read +and held/deferred-event loading and unavailable state from its owned reader. +Pending or failed evidence is never rendered as a confirmed empty activity +window. Each failed source withdraws its own rows without hiding independently +readable rows from the other source. Source-specific warnings have `role="alert"` +and loading text has `role="status"`; a slow held-event read does not block the +existing attempt-refresh button. Recovery clears a warning only after a +successful read, without remounting the destination editors or changing their +unfinished input. + +The connected Notifications browser fixture uses the real tab/state/API/card +chain at desktop and phone widths with keyboard/touch refresh, class-dark CSS, +403/503 controls and pending recovery. Unit controls separately cover current +request ownership and disposal. No mutation, route, navigation entry, queue +policy, permission or recipient-delivery contract is added by these states. diff --git a/docs/release-control/v6/internal/subsystems/monitoring.md b/docs/release-control/v6/internal/subsystems/monitoring.md index 9660e69b59..24cf2bb889 100644 --- a/docs/release-control/v6/internal/subsystems/monitoring.md +++ b/docs/release-control/v6/internal/subsystems/monitoring.md @@ -17,6 +17,47 @@ ## Purpose +### Guest-agent coordination around backups — issue #2439 + +All PVE QEMU guest-agent reads, including disk diagnostics and legacy meminfo +callers, share fail-fast per-guest admission within this Pulse process. Clients +for the same endpoint and configured cluster aliases coordinate across methods; +VMIDs remain serial through node migration. Other VMs and independent endpoints +remain independent. No guest payload or credentials enter coordination state. +Before each command and before returning its fully received bounded payload, +Pulse reads the operation lock from VM config. Any nonempty lock (not just +backup), missing/malformed config or an unverifiable lock defers the command. +Known locks in node inventory, cluster resources or detailed status suppress +polling enrichment even when status fails and recent agent evidence exists. + +Admission is held through the complete response, not just headers. A transport +failure, timeout, wrong-command-ID response or incomplete body defers every +command for that guest for a minute; callers neither immediately retry nor +replay through another cluster endpoint. Coordination retains at most 4,096 +active/cooling endpoint-guest entries and expires cooldowns. Explicit API +permission refusals and unsupported commands remain errors, not successes. + +Deferrals retain the original metadata/memory cache timestamps and supported-OS +failure evidence. Locked guests retain labelled last-known disks/identity and +previous-snapshot memory, or unavailable memory without trustworthy evidence. +Current non-QGA memory, CPU and I/O may still be observed. History never writes +retained disk values or deferred cached memory as newly observed telemetry; +normal unlocked polling resumes and replaces them with fresh data. + +`TestGuestAgentBackupMonitoringContract` exercises healthy → status lock → +resource lock with failed status → node fallback → unlocked resumption through +the real client/builders and History. The client controls cover every read, +unknown lock state, diagnostic overlap, aliases/source/VM isolation, in-flight +lock changes, incomplete bodies, cooldown and absence of failover replay. +These reproduce missing Pulse safety controls, **not** the reported native +PVE/QGA command-ID collision or successful backup thaw. HTTP lock checks cannot +atomically reserve PVE's serial channel against a backup beginning afterwards, +cancel work already running in pvedaemon, detect a guest left frozen after its +lock clears, or coordinate separate Pulse processes. Native freeze/thaw acceptance +on a containing build remains required. Until then, stopping Pulse before a +freeze-enabled backup and restarting only after verified thaw remains the safe +workaround, with monitoring and alerts unavailable during that window. + ### Exact TrueNAS subscription termination — issue #2396 JSON-RPC stream readers and the subscription-acknowledgement wait recognise diff --git a/docs/release-control/v6/internal/subsystems/performance-and-scalability.md b/docs/release-control/v6/internal/subsystems/performance-and-scalability.md index 80cd898e5a..9d9e0731fd 100644 --- a/docs/release-control/v6/internal/subsystems/performance-and-scalability.md +++ b/docs/release-control/v6/internal/subsystems/performance-and-scalability.md @@ -15,6 +15,26 @@ ## Purpose +### Guest disk status updates retain row ownership + +Disk deferral copy follows reactive same-VM props in the existing row and drawer, +without new guest commands, client polling, History writes or row remounts. Retained +values are labelled rather than replaced with invented fresh percentages; fresh +observations update values and remove the status. Mounted row/list/drawer controls +and `browser-tests/guest-disk-deferral.cjs` verify same-row ownership in bar and +sparkline modes with desktop and phone CSS. This is client behaviour proof, not +native backup coordination, new History samples or a fleet performance result. + +### Phone workload identity weights + +The existing phone guest projection allocates 38 percent to the name cell, +which includes disclosure, status and backup controls, while normalising every +remaining visible metric weight. Narrow/mobile/desktop modes and windowing +ownership remain unchanged. Guest column regressions verify the sums and the +production composed-table browser fixture checks identity and all configured +metric tracks at narrow width; this is readability, not a throughput claim. + + ### Bounded confidence/priority work in broadcast grouping — issue #2199 The general identity matcher retains every confidence, reason, ordering and @@ -301,6 +321,21 @@ the existing bounded HTTP client and artifact-size checks. ## Shared Boundaries +### Multi-disk Bars summary + +The existing vertical-bars workload/node disk cell shows the fullest measured +disk's percentage beside equal-weight per-disk micro-bars. Its title retains +the actual mount and percentage. Unknown/invalid usage cannot produce a new +summary; an aggregate fallback must not masquerade as measured member usage. +The displayed maximum uses the existing shared disk measurement and threshold +logic; no collector, sorting key, metric units, History request, resampling or +performance threshold changes. `StackedDiskBar.test.tsx` and the model branch +controls cover real/unknown measurements and unchanged per-member bars. +`browser-tests/proxmox-docker-rows.cjs` checks the actual label, bars and title +at desktop and phone widths; this is presentation proof, not native collection +or a measured fleet performance improvement. + + 1. `frontend-modern/src/components/Infrastructure/infrastructureSelectors.ts` shared with `unified-resources`: the infrastructure selector pipeline is both a canonical unified-resource consumer surface and a fleet-scale performance hot-path boundary. 2. `frontend-modern/src/components/Infrastructure/resourceDetailMappers.ts` shared with `unified-resources`: resource detail mappers are both a canonical unified-resource consumer surface and a fleet-scale performance hot-path boundary. 3. `frontend-modern/src/components/Infrastructure/UnifiedResourceHostTableCard.tsx` shared with `unified-resources`: the unified resource host table card is both a canonical unified-resource consumer surface and a fleet-scale performance hot-path boundary. diff --git a/docs/release-control/v6/internal/subsystems/storage-recovery.md b/docs/release-control/v6/internal/subsystems/storage-recovery.md index c0a85bd7b9..ef5654ebc6 100644 --- a/docs/release-control/v6/internal/subsystems/storage-recovery.md +++ b/docs/release-control/v6/internal/subsystems/storage-recovery.md @@ -21,6 +21,28 @@ ## Purpose +### Backup identity, phone storage and History consent composition + +The Proxmox Coverage last-backup pointer selects completed PBS snapshots or PVE +backup files, never guest-local snapshots. Guest snapshots remain listed as +restore evidence but cannot make an old independent backup look current. +Archive detail uses its format (or filename fallback), preserving the full +provider volid for hover and search in both Coverage and By date. + +Phone pool/disk rows preserve health and identity while using weighted widths, +consistent gutters and compact capacity/health labels. Full disk risk reasons +move into the existing expanded header; the ZFS activity badge uses supplied +scan data only when rebuild is reported and keeps the full summary as its title. +Collection-independent, organisation-owned disk History remains intact. + +The UpdateHistory confirmation is a version-rollback consent, not a full-state +recovery promise. Its current API exposes no verified restore-scope metadata; +copy must retain installation-only and legacy differences and direct full-state +recovery to the stopped-service procedure. Regressions cover these distinctions, +archive/snapshot attribution and storage values; the composed browser fixture +checks real table/drawer rendering, not native or installed recovery. + + ### Disk History is independent of current collection The existing physical-disk History catalog is selected by disk family, not by @@ -378,6 +400,20 @@ command-capable profile. ## Shared Boundaries +### Operator storage vocabulary without identity changes + +Storage topology names preserve Proxmox plugin vocabulary, including LVM-Thin, +NFS, SMB/CIFS, iSCSI, ZFS and BTRFS. Explicit topology overrides still win. +Only a storage explicitly marked shared whose retained location is the synthetic +cluster/shared marker reads Shared, with the supplied reporting-node count when +more than one node is present. Node-local storage and shared storage with a +real native location keep that location. This is host-label presentation, not +resource consolidation: IDs, parent/node hints, ownership evidence, metrics +targets, alerts, health and recovery authority are unchanged. Adapter/presentation +regressions and the production storage table browser fixture verify labels +without treating shared capacity as a backup or demonstrating native recovery. + + - After password configuration has been persisted and the runtime local-admin identity changes, authorizer synchronisation must also occur if subsequent API-token persistence fails. Development first-run reset must clear the configured-admin bypass only on successful auth reset, not on its persistence-failure rollback path. No role-store deletion or migration is part of this recovery. ### PBS host history correlation diff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.md index 975c62f408..cf80b93be1 100644 --- a/docs/release-control/v6/internal/subsystems/unified-resources.md +++ b/docs/release-control/v6/internal/subsystems/unified-resources.md @@ -23,6 +23,36 @@ and sort the complete canonical change table while startup and ingestion wait. ## Purpose +### Guest disk freshness presentation + +Existing workload disk evidence retains its source reason through the row, +breakdown and Overview. `prev-` values are explicitly last-known, while a +deferral with no previous disks explains the missing observation without a +measured percentage. Lock/uncertain-completion deferrals must not become an +install/restart recommendation. Human-readable Overview status remains visible +without filesystem rows and is cleared by a same-identity fresh observation. +The shared presentation and mounted regressions plus the production-component +`browser-tests/guest-disk-deferral.cjs` fixture verify this client behaviour; +they do not prove native QGA coordination, History freshness or delivery. + + +### Narrow controller and backup summaries retain their source facts + +Kubernetes controller row expansions expose supplied kind-specific targets, +counts, absolute timestamps, valid elapsed durations and namespace/cluster when +responsive rows hide those columns. Missing facts are omitted and the ordinary +pluralisation model handles singular targets. Job/CronJob timestamp regressions +and the production composed-table browser fixture verify this existing summary. + +Proxmox Coverage uses latestBackup for completed independent backup evidence, +with guest-local snapshots retained separately. Responsive backup-server, host +and guest rows allocate space through existing canonical weighted-width helpers; +archive format is presentation only, with the provider identifier retained in +hover and search. The recovery-model, table and guest regressions plus desktop/ +phone browser rendering verify attribution, visible facts and source composition. +These frontend improvements change no collector, API authority or native status. + + ### Resource evidence fallback access boundaries When the existing resource-facet reader receives a final HTTP 401/403, its @@ -600,6 +630,29 @@ about the same disk cannot diverge. ## Shared Boundaries +### REST facets and container table projection + +The unified-resource REST mapper carries PMG metadata, per-source status, +server action-readiness refusals and technology at the same top-level paths +used by canonical websocket rows. Existing platformData compatibility paths +remain; transport order must not decide whether mail counters or lifecycle +refusal explanations are present. A replaced REST snapshot clears removed +facets instead of retaining old counters or refusal reasons. Mapping does not +invent capabilities or bypass planning, review, server authorisation or execution. +`useUnifiedResources.test.ts` checks actual lifecycle refusal consumers for stale, +failed and server-denied observations alongside a no-invented-grant control. + +The Docker phone container projection keeps identity, state when applicable, +CPU, memory and update; restart counts remain in the existing detail expansion +and return at mobile width. Images retain image, size and update verdict on +phones; host and consumers stay reachable in detail. Hidden header/body columns +are symmetric. Swarm rollout tokens render as words with raw state retained in +the title; task starts use the shared relative-time primitive with their exact +timestamp retained. Native table/model/guardrail tests and the production +`browser-tests/proxmox-docker-rows.cjs` exercise these surfaces without executing +a container action or claiming native appliance or release acceptance. + + ### PBS host history correlation PBS presentation correlation accepts standalone agents and VM/system-container resources with an agent facet. Preserve unique identity matching and canonical metricsTarget type and ID. Guests without telemetry and mixed ambiguous agent/guest matches must not replace the PBS target. diff --git a/frontend-modern/browser-tests/guest-disk-deferral.cjs b/frontend-modern/browser-tests/guest-disk-deferral.cjs new file mode 100644 index 0000000000..349adae709 --- /dev/null +++ b/frontend-modern/browser-tests/guest-disk-deferral.cjs @@ -0,0 +1,273 @@ +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const crypto = require('node:crypto'); +const { chromium, webkit } = require('playwright'); +const root = '/workspace/frontend-modern'; +const output = '/workspace/tmp/guest-disk-deferral-proof'; +const runtime = [ + 'frontend-modern/src/utils/workloadGuestPresentation.ts', + 'frontend-modern/src/components/Workloads/GuestRow.tsx', + 'frontend-modern/src/components/Workloads/stackedDiskBarModel.ts', + 'frontend-modern/src/components/Workloads/StackedDiskBar.tsx', + 'frontend-modern/src/components/Workloads/DiskList.tsx', + 'frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx', +]; +const deferrals = [ + ['vm-locked', 'VM operation lock'], + ['lock-unverified', 'cannot verify that the VM is unlocked'], + ['agent-busy', 'earlier guest request is still in progress'], + ['agent-cooldown', 'after the cooldown'], + ['agent-response-incomplete', 'previous response was incomplete'], + ['agent-capacity', 'guest-read capacity'], + ['invalid-guest-key', 'VM identity is invalid'], + ['agent-timeout', 'Do not restart the guest agent during a backup'], +]; +(async () => { + fs.mkdirSync(output, { recursive: true }); + const playwrightVersion = require('playwright/package.json').version; + assert.equal( + playwrightVersion, + JSON.parse(fs.readFileSync('/workspace/tests/integration/package-lock.json')).packages[ + 'node_modules/@playwright/test' + ].version, + ); + const hashes = Object.fromEntries( + runtime.map((p) => [ + p, + crypto + .createHash('sha256') + .update(fs.readFileSync(path.join('/workspace', p))) + .digest('hex'), + ]), + ); + process.chdir(root); + const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js')); + const server = await createServer({ + root, + configFile: path.join(root, 'vite.config.ts'), + cacheDir: path.join(output, 'vite-cache'), + server: { host: '127.0.0.1', port: 5294, strictPort: true }, + }); + const results = [], + screenshots = []; + let browser; + try { + await server.listen(); + for (const [name, engine, width, dark] of [ + ['desktop', chromium, 1365, false], + ['phone', webkit, 390, true], + ]) { + browser = await engine.launch( + engine === chromium + ? { headless: true, channel: 'chromium', args: ['--no-sandbox'] } + : { headless: true }, + ); + const page = await browser.newPage({ + viewport: { width, height: width === 390 ? 844 : 900 }, + isMobile: width === 390, + hasTouch: width === 390, + locale: 'en-GB', + timezoneId: 'UTC', + }); + page.setDefaultTimeout(15000); + page.setDefaultNavigationTimeout(60000); + const errors = [], + checks = []; + page.on('pageerror', (e) => { + errors.push(e.message); + process.stderr.write(`BROWSER_PAGE_ERROR ${e.message}\n`); + }); + page.on('console', (m) => { + if (m.type() === 'error') + process.stderr.write(`BROWSER_CONSOLE ${m.text().slice(0, 1500)}\n`); + }); + page.on('response', (r) => { + if (r.status() >= 400) + process.stderr.write(`BROWSER_HTTP ${r.status()} ${new URL(r.url()).pathname}\n`); + }); + await page.route('http://127.0.0.1:5294/api/**', (route) => + route.fulfill({ + status: 200, + contentType: 'application/json', + body: JSON.stringify({ + success: true, + data: [], + alerts: [], + anomalies: [], + config: null, + }), + }), + ); + await page.goto('http://127.0.0.1:5294/browser-tests/guest-disk-deferral.html', { + waitUntil: 'domcontentloaded', + }); + try { + await page.waitForFunction(() => window.__guestDiskEvidence, null, { timeout: 15000 }); + } catch (error) { + fs.writeFileSync( + path.join(output, 'failure.json'), + JSON.stringify({ phase: 'fixture-readiness', errors }, null, 2), + ); + throw error; + } + if (dark) await page.evaluate(() => document.documentElement.classList.add('dark')); + const row = page.locator('[data-guest-id="fixture-pve:pve-a:101"]'); + const cell = row.locator('[data-workload-col="disk"]'); + await row.click(); + const drawer = page.getByRole('region', { name: 'Guest Overview' }); + await drawer.waitFor({ state: 'visible' }); + await page.evaluate( + () => + (window.__originalDiskRow = document.querySelector( + '[data-guest-id="fixture-pve:pve-a:101"]', + )), + ); + const update = async (reason, data, usage = 50) => { + await page.evaluate((value) => window.__guestDiskEvidence.update(value), { + reason, + data, + usage, + }); + await page.waitForFunction( + () => + document.querySelector('[data-guest-id="fixture-pve:pve-a:101"]') === + window.__originalDiskRow, + ); + assert.equal( + await page.evaluate(() => window.__guestDiskEvidence.identity()), + 'fixture-pve:pve-a:101', + ); + }; + const capture = async (suffix) => { + const p = path.join(output, `${name}-${suffix}.png`); + await page.screenshot({ path: p, fullPage: true }); + screenshots.push({ + path: path.relative('/workspace', p), + sha256: crypto.createHash('sha256').update(fs.readFileSync(p)).digest('hex'), + }); + }; + for (const [reason, expected] of deferrals) { + await update(`prev-${reason}`, true); + await page.waitForFunction( + (text) => document.querySelector('[data-workload-col="disk"]').title.includes(text), + expected, + ); + const title = await cell.getAttribute('title'); + assert(title.startsWith('Using last known disk stats. ')); + assert(title.includes(expected)); + assert( + !title.includes('may not be installed') && !title.includes('may need to be restarted'), + ); + assert((await drawer.innerText()).includes(title)); + const statusBox = await drawer.getByText(title, { exact: true }).boundingBox(); + assert( + statusBox.width >= 100 && statusBox.height < 220, + 'status explanation fits its card', + ); + assert((await drawer.innerText()).includes('/data')); + assert( + (await page.getByRole('region', { name: 'Guest disk list' }).innerText()).includes(title), + ); + if (name === 'desktop') { + // The disclosure click can land on this bar and dismiss its hover + // overlay on pointerdown. Leave it before exercising a new hover. + await page.locator('h1').hover(); + await cell.locator('[data-stacked-disk-trigger]').hover(); + await page.locator('[data-tooltip-portal="true"]').waitFor({ state: 'visible' }); + assert((await page.locator('[data-tooltip-portal="true"]').innerText()).includes(title)); + await page.locator('h1').hover(); + } + checks.push(`${reason}: retained row, tooltip, drawer, list and identity`); + if (reason === 'vm-locked') await capture('retained-backup'); + await update(reason, false); + await page.waitForFunction( + (text) => document.querySelector('[data-workload-col="disk"]').title.includes(text), + expected, + ); + assert(!(await cell.getAttribute('title')).includes('Using last known')); + assert((await drawer.innerText()).includes(expected)); + assert(!(await drawer.innerText()).includes('/data')); + assert.equal(await cell.locator('[data-stacked-disk-trigger]').count(), 0); + checks.push(`${reason}: no previous disk values and no fresh percentage`); + if (reason === 'agent-timeout') await capture('uncertain-timeout'); + } + for (const [reason, expected] of [ + ['permission-denied', 'Permission denied'], + ['agent-disabled', 'disabled in VM configuration'], + ['agent-not-running', 'Guest agent not running'], + ['no-data', 'No disk data available'], + ['vm-stopped', 'VM is stopped'], + ['no-status', 'could not read the VM status'], + ['agent-error', 'Error communicating with guest agent'], + ]) { + await update(reason, false); + await page.waitForFunction( + (text) => document.querySelector('[data-workload-col="disk"]').title.includes(text), + expected, + ); + assert((await drawer.innerText()).includes(expected)); + checks.push(`${reason}: existing unavailable case remains distinct`); + } + await update('prev-agent-cooldown', true); + await page.evaluate(() => window.__guestDiskEvidence.mode('sparklines')); + assert((await cell.getAttribute('title')).includes('Using last known')); + await update('', true, 75); + await page.waitForFunction( + () => !document.querySelector('[data-workload-col="disk"]').hasAttribute('title'), + ); + assert(!(await drawer.innerText()).includes('Using last known')); + assert(!(await drawer.innerText()).includes('Guest reads')); + assert.equal(await drawer.getByRole('progressbar').getAttribute('aria-valuenow'), '75'); + await page.evaluate(() => window.__guestDiskEvidence.mode('bars')); + assert.equal( + await page.evaluate( + () => + document.querySelector('[data-guest-id="fixture-pve:pve-a:101"]') === + window.__originalDiskRow, + ), + true, + ); + checks.push( + 'same-VM fresh resumption removes notice and updates values in both row modes without remount', + ); + await capture('fresh-resumption'); + assert.equal( + await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth), + true, + ); + assert.deepEqual(errors, []); + results.push({ + name, + browser_version: browser.version(), + viewport: { width, height: width === 390 ? 844 : 900 }, + checks, + page_errors: errors, + }); + await browser.close(); + browser = null; + } + const receipt = { + version: 1, + result: 'passed', + verified_at: new Date().toISOString(), + content_sha256: hashes, + playwright_version: playwrightVersion, + results, + screenshots, + limitation: + 'Synthetic same-VM observations through production components and CSS; no native QGA/backup, History sample, installation or release acceptance.', + }; + fs.writeFileSync( + path.join(output, 'browser-result.json'), + JSON.stringify(receipt, null, 2) + '\n', + ); + process.stdout.write(JSON.stringify(receipt) + '\n'); + } finally { + if (browser) await browser.close(); + await server.close(); + } +})().catch((e) => { + process.stderr.write(e.stack + '\n'); + process.exitCode = 1; +}); diff --git a/frontend-modern/browser-tests/guest-disk-deferral.html b/frontend-modern/browser-tests/guest-disk-deferral.html new file mode 100644 index 0000000000..16926a6671 --- /dev/null +++ b/frontend-modern/browser-tests/guest-disk-deferral.html @@ -0,0 +1,12 @@ + + + + + + Guest disk deferral verification + + +
+ + + diff --git a/frontend-modern/browser-tests/guest-disk-deferral.tsx b/frontend-modern/browser-tests/guest-disk-deferral.tsx new file mode 100644 index 0000000000..d1ad8dfdd2 --- /dev/null +++ b/frontend-modern/browser-tests/guest-disk-deferral.tsx @@ -0,0 +1,127 @@ +// Production row, disk tooltip, list and disclosed Overview with synthetic +// same-VM observations. No collector, QGA, native backup or release proof. +import { createSignal, Show } from 'solid-js'; +import { render } from 'solid-js/web'; +import { Router, Route } from '@solidjs/router'; +import { GuestRow } from '../src/components/Workloads/GuestRow'; +import { GuestDrawerOverview } from '../src/components/Workloads/GuestDrawerOverview'; +import { DiskList } from '../src/components/Workloads/DiskList'; +import type { VM } from '../src/types/api'; +import { DarkModeContext } from '../src/contexts/appRuntime'; +import '../src/index.css'; + +function Fixture() { + const [observation, setObservation] = createSignal({ + reason: 'prev-vm-locked', + data: true, + usage: 50, + }); + const [expanded, setExpanded] = createSignal(false); + const [mode, setMode] = createSignal<'bars' | 'sparklines'>('bars'); + const guest = (): VM => ({ + id: 'fixture-pve:pve-a:101', + vmid: 101, + name: 'backup-guest', + node: 'pve-a', + instance: 'fixture-pve', + status: 'running', + type: 'qemu', + cpu: 0.1, + cpus: 2, + memory: { total: 1024 ** 3, used: 256 * 1024 ** 2, free: 768 * 1024 ** 2, usage: 25 }, + disk: { + total: 10 * 1024 ** 3, + used: observation().data ? (observation().usage / 100) * 10 * 1024 ** 3 : 0, + usage: observation().data ? observation().usage : -1, + }, + disks: observation().data + ? [ + { + mountpoint: '/data', + type: 'ext4', + total: 10 * 1024 ** 3, + used: (observation().usage / 100) * 10 * 1024 ** 3, + usage: observation().usage, + }, + ] + : [], + diskStatusReason: observation().reason, + networkIn: 0, + networkOut: 0, + diskRead: 0, + diskWrite: 0, + uptime: 3600, + template: false, + lastBackup: 0, + tags: [], + lock: observation().reason.endsWith('vm-locked') ? 'backup' : '', + backupInProgress: observation().reason.endsWith('vm-locked'), + lastSeen: '2026-10-03T14:00:00Z', + }); + (window as any).__guestDiskEvidence = { + update: setObservation, + mode: setMode, + identity: () => guest().id, + }; + return ( +
+

Guest disk observation

+ + + setExpanded(!expanded())} + isExpanded={expanded()} + metricDisplayMode={mode()} + workloadTableLayoutMode={window.innerWidth <= 768 ? 'phone' : 'wide'} + /> + +
+ +
+ 0} + hasNetworkInterfaces={false} + hasOsInfo={false} + hasWorkloadActionAgent={false} + showInGuestAgentInstallCue={false} + ipAddresses={[]} + networkInterfaces={[]} + normalizedTags={[]} + backupPresentation={null} + workloadActionAgentTitle="" + /> +
+
+
+

Filesystems

+ +
+
+ ); +} + +render( + () => ( + + ( + document.documentElement.classList.contains('dark')} + > + + + )} + /> + + ), + document.getElementById('root')!, +); diff --git a/frontend-modern/browser-tests/notification-evidence.cjs b/frontend-modern/browser-tests/notification-evidence.cjs new file mode 100644 index 0000000000..8a6fef7319 --- /dev/null +++ b/frontend-modern/browser-tests/notification-evidence.cjs @@ -0,0 +1,383 @@ +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const crypto = require('node:crypto'); +const { chromium, webkit } = require('playwright'); +const output = '/workspace/tmp/notification-evidence-proof'; +const root = '/workspace/frontend-modern'; +const runtime = [ + 'frontend-modern/src/features/alerts/AlertDeliveryLogCard.tsx', + 'frontend-modern/src/features/alerts/useNotificationDeliveryLog.ts', + 'frontend-modern/src/features/alerts/useAlertDestinationsTabState.ts', + 'frontend-modern/src/features/alerts/tabs/DestinationsTab.tsx', + 'frontend-modern/src/utils/alertDestinationsPresentation.ts', +]; +const log = (entries = []) => ({ + entries, + window_days: 30, + completed_retention_days: 7, + dead_letter_retention_days: 30, +}); +const sent = { + notificationId: 'synthetic-attempt', + type: 'email', + outcome: 'sent', + alertIds: ['synthetic-disk-alert'], + alertCount: 1, + attempts: 1, + success: true, + timestamp: '2026-10-03T10:00:00Z', +}; +const held = [ + { + id: 42, + type: 'notification_deferred', + alertId: 'synthetic-held-alert', + resourceName: 'synthetic-nas', + alertType: 'usage', + reason: 'quiet_hours:performance', + occurredAt: '2026-10-03T10:01:00Z', + }, +]; +(async () => { + fs.mkdirSync(output, { recursive: true }); + const playwrightVersion = require('playwright/package.json').version; + assert.equal( + playwrightVersion, + JSON.parse(fs.readFileSync('/workspace/tests/integration/package-lock.json')).packages[ + 'node_modules/@playwright/test' + ].version, + ); + const hashes = Object.fromEntries( + runtime.map((p) => [ + p, + crypto + .createHash('sha256') + .update(fs.readFileSync(path.join('/workspace', p))) + .digest('hex'), + ]), + ); + process.chdir(root); + const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js')); + const server = await createServer({ + root, + configFile: path.join(root, 'vite.config.ts'), + cacheDir: path.join(output, 'vite-cache'), + server: { host: '127.0.0.1', port: 5291, strictPort: true }, + }); + const results = [], + screens = []; + let browser; + try { + await server.listen(); + for (const [name, engine, width, dark] of [ + ['desktop', chromium, 1365, false], + ['phone', webkit, 390, true], + ]) { + browser = await engine.launch( + engine === chromium + ? { headless: true, channel: 'chromium', args: ['--no-sandbox'] } + : { headless: true }, + ); + const page = await browser.newPage({ + viewport: { width, height: width === 390 ? 844 : 900 }, + isMobile: width === 390, + hasTouch: width === 390, + locale: 'en-GB', + timezoneId: 'UTC', + }); + page.setDefaultTimeout(20000); + const errors = [], + offOrigin = [], + writes = [], + requests = [], + checks = [], + releases = []; + let attemptMode = 'hold-empty', + heldMode = 'hold-empty'; + page.on('pageerror', (error) => errors.push(error.message)); + await page.route('**/*', async (route) => { + const request = route.request(), + url = new URL(request.url()); + if (url.origin !== 'http://127.0.0.1:5291') { + offOrigin.push(url.origin); + return route.abort(); + } + if (!url.pathname.startsWith('/api/')) return route.continue(); + if (request.method() !== 'GET') { + writes.push({ method: request.method(), path: url.pathname }); + return route.abort(); + } + if ( + url.pathname === '/api/notifications/delivery-log' || + url.pathname === '/api/alerts/events' + ) { + const isHeld = url.pathname === '/api/alerts/events'; + const mode = isHeld ? heldMode : attemptMode; + requests.push({ path: url.pathname, query: url.search, mode }); + if (mode.startsWith('hold-')) + await new Promise((resolve) => releases.push({ isHeld, resolve })); + if (mode.endsWith('denied')) + return route.fulfill({ + status: isHeld ? 403 : 503, + json: { error: 'Synthetic unavailable evidence' }, + }); + return route.fulfill({ + json: isHeld + ? mode.endsWith('known') + ? held + : [] + : log(mode.endsWith('known') ? [sent] : []), + }); + } + if (url.pathname === '/api/notifications/health') + return route.fulfill({ + json: { + overall_healthy: true, + queue: { + pending: 0, + sending: 0, + sent: 1, + failed: 0, + dlq: 0, + healthy: true, + status: 'healthy', + attention_required: 0, + reason_codes: [], + completed_retention_days: 7, + dead_letter_retention_days: 30, + counts_are_retention_bounded: true, + }, + }, + }); + if (url.pathname === '/api/license/runtime-capabilities') + return route.fulfill({ + json: { + capabilities: [], + limits: [], + max_history_days: 7, + hosted_mode: false, + runtime: { build: 'community' }, + blocked_capabilities: [], + }, + }); + if (url.pathname === '/api/license/entitlements') + return route.fulfill({ json: { entitlements: [] } }); + if (url.pathname === '/api/security/status') + return route.fulfill({ json: { hasAuthentication: true, requiresAuth: false } }); + return route.fulfill({ json: [] }); + }); + const check = async (label, action) => { + await action(); + checks.push({ label, passed: true }); + }; + const activity = page.locator('#notification-delivery-activity'); + const empty = activity.getByText(/No alert deliveries were attempted/); + const heldWarning = activity.getByText(/could not read held or deferred notifications/); + const attemptWarning = activity.getByText(/could not read the delivery log/); + const refresh = activity.getByRole('button', { name: 'Refresh delivery status' }); + const release = (isHeld) => { + for (const pending of releases.splice(0).filter((item) => { + if (item.isHeld === isHeld) { + item.resolve(); + return false; + } + return true; + })) + releases.push(pending); + }; + const capture = async (state) => { + const filename = `${name}-${state}.png`; + await activity.screenshot({ path: path.join(output, filename) }); + screens.push({ + file: filename, + state, + viewport: { width, height: width === 390 ? 844 : 900 }, + sha256: crypto + .createHash('sha256') + .update(fs.readFileSync(path.join(output, filename))) + .digest('hex'), + }); + }; + try { + await page.goto('http://127.0.0.1:5291/browser-tests/notification-evidence.html'); + await page.evaluate( + (dark) => document.documentElement.classList.toggle('dark', dark), + dark, + ); + await check('Initial reads are loading, not empty', async () => { + await activity.getByText('Loading delivery attempts...').waitFor(); + await activity.getByText('Loading held and deferred notifications...').waitFor(); + assert.equal(await empty.count(), 0); + }); + await check( + 'A slow held read does not block the successful attempts read or its refresh', + async () => { + release(false); + await activity.getByText('Loading delivery attempts...').waitFor({ state: 'hidden' }); + assert.equal(await refresh.isEnabled(), true); + assert.equal(await empty.count(), 0); + await activity.getByText('Loading held and deferred notifications...').waitFor(); + }, + ); + await capture('held-pending'); + await check('Only two completed empty reads establish the empty window', async () => { + release(true); + await empty.waitFor(); + assert.equal(await activity.getByRole('status').count(), 0); + }); + await check( + 'Keyboard or touch refresh shows attempts and held events from their real API adapters', + async () => { + attemptMode = heldMode = 'known'; + if (width === 390) await refresh.tap(); + else { + await refresh.focus(); + await page.keyboard.press('Enter'); + } + await activity.getByText('Delivered', { exact: true }).waitFor(); + await activity.getByText('synthetic-nas (usage)', { exact: true }).waitFor(); + assert.equal(await empty.count(), 0); + }, + ); + await check( + 'Held-event 403 withdraws held rows and warns without hiding delivered evidence', + async () => { + heldMode = 'denied'; + await refresh.click(); + await heldWarning.waitFor(); + assert.equal( + await activity.getByText('synthetic-nas (usage)', { exact: true }).count(), + 0, + ); + assert.equal(await activity.getByText('Delivered', { exact: true }).count(), 1); + assert.equal(await empty.count(), 0); + }, + ); + await capture('held-unavailable'); + await check('The held warning persists during a pending retry', async () => { + heldMode = 'hold-empty'; + await refresh.click(); + await activity.getByText('Loading held and deferred notifications...').waitFor(); + assert.equal(await heldWarning.count(), 1); + await refresh.waitFor({ state: 'visible' }); + await page.waitForFunction( + () => !document.querySelector('#notification-delivery-activity button').disabled, + ); + assert.equal(await empty.count(), 0); + release(true); + await heldWarning.waitFor({ state: 'hidden' }); + }); + await check( + 'An attempt-read failure keeps independently readable held evidence', + async () => { + attemptMode = 'denied'; + heldMode = 'known'; + await refresh.click(); + await attemptWarning.waitFor(); + await activity.getByText('synthetic-nas (usage)', { exact: true }).waitFor(); + assert.equal(await activity.getByText('Delivered', { exact: true }).count(), 0); + assert.equal(await empty.count(), 0); + }, + ); + await capture('attempts-unavailable'); + await check('Both failed sources are reported, not called empty', async () => { + heldMode = 'denied'; + await refresh.click(); + await heldWarning.waitFor(); + assert.equal(await attemptWarning.count(), 1); + assert.equal(await activity.getByRole('listitem').count(), 0); + assert.equal(await empty.count(), 0); + }); + await check( + 'Both successful reads clear the warnings and restore a genuine empty state', + async () => { + attemptMode = heldMode = 'empty'; + await refresh.click(); + await empty.waitFor(); + assert.equal(await activity.getByRole('alert').count(), 0); + assert.equal(await activity.getByRole('status').count(), 0); + }, + ); + await check( + 'No horizontal clipping, JavaScript page error, off-origin request or mutation', + async () => { + assert.equal( + await activity.evaluate((element) => element.scrollWidth <= element.clientWidth + 1), + true, + ); + assert.deepEqual(errors, []); + assert.deepEqual(offOrigin, []); + assert.deepEqual(writes, []); + assert( + requests + .filter((r) => r.path === '/api/alerts/events') + .every( + (r) => r.query.includes('limit=100') && r.query.includes('notification_deferred'), + ), + ); + assert( + requests + .filter((r) => r.path === '/api/notifications/delivery-log') + .every((r) => r.query === '?limit=200'), + ); + }, + ); + results.push({ + name, + browser_version: browser.version(), + dark, + checks, + requests, + errors, + offOrigin, + writes, + }); + } finally { + for (const pending of releases) pending.resolve(); + await browser.close(); + browser = null; + } + } + fs.writeFileSync( + path.join(output, 'result.json'), + JSON.stringify( + { + passed: true, + playwright_version: playwrightVersion, + content_sha256: hashes, + changed_paths: runtime, + results, + screenshots: screens, + }, + null, + 2, + ) + '\n', + ); + console.log( + JSON.stringify({ + passed: true, + checks: results.reduce((n, r) => n + r.checks.length, 0), + screenshots: screens.length, + playwright_version: playwrightVersion, + results: results.map((r) => ({ name: r.name, browser_version: r.browser_version })), + }), + ); + } catch (error) { + fs.writeFileSync( + path.join(output, 'failed.json'), + JSON.stringify( + { passed: false, error: error.message, results, screenshots: screens }, + null, + 2, + ) + '\n', + ); + throw error; + } finally { + if (browser) await browser.close(); + await server.close(); + } +})().catch((error) => { + console.error(error); + process.exitCode = 1; +}); diff --git a/frontend-modern/browser-tests/notification-evidence.html b/frontend-modern/browser-tests/notification-evidence.html new file mode 100644 index 0000000000..71c910e8fe --- /dev/null +++ b/frontend-modern/browser-tests/notification-evidence.html @@ -0,0 +1,12 @@ + + + + + + Notification evidence verification + + +
+ + + diff --git a/frontend-modern/browser-tests/notification-evidence.tsx b/frontend-modern/browser-tests/notification-evidence.tsx new file mode 100644 index 0000000000..cd358e54b1 --- /dev/null +++ b/frontend-modern/browser-tests/notification-evidence.tsx @@ -0,0 +1,69 @@ +// The production Notifications tab and API readers with synthetic read responses. +// No notification is sent, retried or dismissed by this fixture. +import { createSignal } from 'solid-js'; +import { render } from 'solid-js/web'; +import { Router, Route } from '@solidjs/router'; +import { DestinationsTab } from '../src/features/alerts/tabs/DestinationsTab'; +import type { UIEmailConfig, UIAppriseConfig } from '../src/features/alerts/types'; +import '../src/index.css'; + +function Fixture() { + const [emailConfig, setEmailConfig] = createSignal({ + enabled: false, + provider: 'smtp', + server: '', + port: 587, + username: '', + password: '', + from: '', + to: [], + tls: true, + startTLS: true, + replyTo: '', + maxRetries: 3, + retryDelay: 5, + rateLimit: 60, + }); + const [appriseConfig, setAppriseConfig] = createSignal({ + enabled: false, + mode: 'cli', + targetsText: '', + cliPath: '', + timeoutSeconds: 20, + serverUrl: '', + configKey: '', + apiKey: '', + apiKeyHeader: 'X-API-KEY', + skipTlsVerify: false, + hasApiKey: false, + }); + return ( +
+

Notifications

+ {}} + configLoadError={() => null} + isRetrying={() => false} + isLoadingDestinations={() => false} + onRetryLoad={() => {}} + webhooks={() => []} + deadManPingUrl={() => ''} + setDeadManPingUrl={() => {}} + pushMinimumSeverity={() => 'all'} + setPushMinimumSeverity={() => {}} + /> +
+ ); +} +render( + () => ( + + + + ), + document.getElementById('root')!, +); diff --git a/frontend-modern/browser-tests/operator-tables-consent.cjs b/frontend-modern/browser-tests/operator-tables-consent.cjs new file mode 100644 index 0000000000..606dafe4be --- /dev/null +++ b/frontend-modern/browser-tests/operator-tables-consent.cjs @@ -0,0 +1,419 @@ +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const crypto = require('node:crypto'); +const { chromium, webkit } = require('playwright'); +(async () => { + const root = '/workspace/frontend-modern'; + const output = '/workspace/tmp/operator-tables-proof'; + fs.mkdirSync(output, { recursive: true }); + process.chdir(root); + const playwright = require('playwright/package.json').version; + assert.equal( + playwright, + JSON.parse(fs.readFileSync('/workspace/tests/integration/package-lock.json')).packages[ + 'node_modules/@playwright/test' + ].version, + ); + const paths = JSON.parse(fs.readFileSync(path.join(output, 'runtime-paths.json'))); + const hashes = Object.fromEntries( + paths.map((p) => [ + p, + crypto + .createHash('sha256') + .update(fs.readFileSync(path.join('/workspace', p))) + .digest('hex'), + ]), + ); + const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js')); + const server = await createServer({ + root, + configFile: path.join(root, 'vite.config.ts'), + cacheDir: path.join(output, 'vite-cache'), + server: { host: '127.0.0.1', port: 5271, strictPort: true }, + }); + const results = [], + screens = []; + let browser; + try { + await server.listen(); + for (const [name, engine, width] of [ + ['desktop', chromium, 1365], + ['phone', webkit, 390], + ['intermediate', chromium, 768], + ]) { + browser = await engine.launch( + engine === chromium + ? { headless: true, channel: 'chromium', args: ['--no-sandbox'] } + : { headless: true }, + ); + const page = await browser.newPage({ + viewport: { width, height: width === 390 ? 844 : 900 }, + isMobile: width === 390, + hasTouch: width === 390, + locale: 'en-GB', + timezoneId: 'UTC', + }); + await page.clock.setFixedTime(new Date('2026-10-03T12:00:00Z')); + const errors = [], + offOrigin = [], + checks = [], + requests = []; + let rollbackMode = 'reject', + finishRollback; + page.on('pageerror', (e) => errors.push(e.message)); + page.setDefaultTimeout(20000); + await page.route('**/*', async (route) => { + const url = new URL(route.request().url()); + if (url.origin !== 'http://127.0.0.1:5271') { + offOrigin.push(url.origin); + return route.abort(); + } + if (!url.pathname.startsWith('/api/')) return route.continue(); + if (url.pathname === '/api/updates/history') + return route.fulfill({ + json: [ + { + event_id: 'fixture-update-1', + timestamp: '2026-10-02T12:00:00Z', + action: 'update', + channel: 'stable', + version_from: '6.4.5', + version_to: '6.4.6', + deployment_type: 'systemd', + initiated_by: 'user', + initiated_via: 'ui', + status: 'success', + duration_ms: 30000, + backup_path: '/private/fixture-backup', + notes: 'untrusted restore-everything note', + }, + ], + }); + if (url.pathname === '/api/version') + return route.fulfill({ + json: { + version: '6.6.0-dev', + build: 'synthetic', + runtime: 'community', + isDocker: false, + isSourceBuild: true, + isDevelopment: false, + }, + }); + if (url.pathname === '/api/updates/rollback') { + requests.push({ method: route.request().method(), body: route.request().postDataJSON() }); + if (rollbackMode === 'hold') + await new Promise((resolve) => { + finishRollback = resolve; + }); + return rollbackMode === 'accept' + ? route.fulfill({ + json: { status: 'accepted', message: 'Synthetic acceptance, not recovery' }, + }) + : route.fulfill({ + status: 409, + json: { error: 'Synthetic incompatible backup scope' }, + }); + } + if (url.pathname === '/api/license/runtime-capabilities') + return route.fulfill({ + json: { + capabilities: [], + limits: [], + max_history_days: 7, + hosted_mode: false, + runtime: { build: 'community' }, + blocked_capabilities: [], + }, + }); + if (url.pathname === '/api/license/entitlements') + return route.fulfill({ json: { entitlements: [] } }); + if (url.pathname.includes('/facets')) + return route.fulfill({ + json: { facets: [], capabilities: [], relationships: [], recentChanges: [] }, + }); + if (url.pathname === '/api/security/status') + return route.fulfill({ + json: { + hasAuthentication: true, + requiresAuth: true, + settingsCapabilities: { authenticationWrite: true }, + }, + }); + if (url.pathname === '/api/metrics-store/history') + return route.fulfill({ json: { points: [], metrics: {}, source: 'store' } }); + return route.fulfill({ + json: { + enabled: false, + data: [], + alerts: {}, + facets: [], + capabilities: [], + relationships: [], + recentChanges: [], + }, + }); + }); + const check = async (label, run) => { + try { + await run(); + checks.push({ label, passed: true }); + } catch (e) { + checks.push({ label, passed: false, error: e.message }); + } + }; + const select = async (name) => { + await page + .getByRole('navigation', { name: 'Fixture views' }) + .getByRole('button', { name, exact: true }) + .click(); + }; + const capture = async (state) => { + // Await the production entrance animation instead of capturing translucent frames. + if (await page.getByRole('dialog').count()) { + await page.waitForFunction(() => { + const panel = document.querySelector('[role=dialog]'); + return panel && Number(getComputedStyle(panel).opacity) >= 0.999; + }); + } + await page.waitForTimeout(100); // ResizeObserver layout, not freshness proof. + const p = `${name}-${state}.png`; + await page.screenshot({ path: path.join(output, p), fullPage: true }); + const dimensions = await page.evaluate(() => ({ + width: innerWidth, + scroll: document.documentElement.scrollWidth, + })); + assert.ok(dimensions.scroll <= dimensions.width + 1, JSON.stringify(dimensions)); + screens.push({ file: p, state, browser: name, width, dimensions }); + }; + await page.goto('http://127.0.0.1:5271/browser-tests/operator-tables-consent.html', { + waitUntil: 'domcontentloaded', + timeout: 120000, + }); + if (width === 390) await page.evaluate(() => document.documentElement.classList.add('dark')); + await check('Pool state and measured capacity survive the phone projection', async () => { + const pool = page.getByRole('region', { name: 'Pool table' }); + await pool.getByText('archive-tank', { exact: true }).waitFor(); + await pool.getByText('Degraded', { exact: true }).waitFor(); + const state = pool.locator('tbody td[data-storage-column="state"]').first(); + const bounds = await state.evaluate((e) => ({ + width: e.clientWidth, + scroll: e.scrollWidth, + text: e.textContent, + })); + assert.ok(bounds.scroll <= bounds.width + 1, JSON.stringify(bounds)); + if (width >= 544) { + await pool.getByText('Resilvering 45%', { exact: true }).waitFor(); + await pool.getByText('Scrubbing 45%', { exact: true }).waitFor(); + assert.ok( + await pool + .locator('[title="ZFS pool archive-tank resilver is running (45.2%)"]') + .count(), + ); + } else { + assert.ok((await pool.getByText('59%', { exact: true }).count()) >= 2); + } + await capture('pools'); + }); + if (width !== 768) { + await select('Disks'); + await check('Disk row and expanded header retain the full SMART reason', async () => { + await page.getByText('Archive HDD', { exact: true }).waitFor(); + await page + .locator('[data-row-id="disk-sda"]') + .getByText('Archive HDD', { exact: true }) + .click(); + const health = page.getByTestId('disk-detail-health'); + await health.getByText('Replace Now', { exact: true }).waitFor(); + assert.ok( + (await health.innerText()).includes('SMART failed. The full reason remains readable'), + ); + const bounds = await health.evaluate((e) => ({ + width: e.clientWidth, + scroll: e.scrollWidth, + })); + assert.ok(bounds.scroll <= bounds.width + 1, JSON.stringify(bounds)); + await capture('disk-reason'); + }); + await select('Controllers'); + await check( + 'Job target and absolute started/completed times are reachable from a narrow row', + async () => { + await page.getByText('nightly-import', { exact: true }).click(); + const details = page.getByTestId('resource-kubernetes-controller-section'); + await details.getByText('Started', { exact: true }).waitFor(); + await details.getByText('Completed', { exact: true }).waitFor(); + await details.getByText('5m', { exact: true }).waitFor(); + await details.getByText('1 completion', { exact: true }).waitFor(); + assert.ok((await details.innerText()).includes('2026')); + await capture('job-times'); + }, + ); + await check( + 'CronJob retains last-run, last-success, schedule and suspension in its expansion', + async () => { + await page.getByText('billing-rollup', { exact: true }).click(); + const details = page.getByTestId('resource-kubernetes-controller-section'); + await details.getByText('Last run', { exact: true }).waitFor(); + await details.getByText('Last success', { exact: true }).waitFor(); + await details.getByText('*/5 * * * *', { exact: true }).waitFor(); + await details.getByText('Suspended', { exact: true }).waitFor(); + await capture('cron-times'); + }, + ); + } + await select('Backups'); + await check( + 'Coverage last backup remains older than a fresh local guest snapshot', + async () => { + const coverage = page.locator('[data-proxmox-backups-table="coverage"]'); + await coverage.waitFor(); + const columns = await coverage + .locator('colgroup col') + .evaluateAll((els) => els.map((e) => e.dataset.proxmoxBackupsColumn)); + const row = coverage.locator('tr[data-proxmox-backup-row=coverage]'); + const backupAge = await row.locator('td').nth(columns.indexOf('latest')).innerText(); + assert.ok(backupAge.includes('13d'), backupAge); + assert.ok(!backupAge.includes('30m'), backupAge); + if (columns.includes('snapshot')) { + assert.ok( + (await row.locator('td').nth(columns.indexOf('snapshot')).innerText()).includes( + '30m', + ), + ); + } + await capture('backup-attribution'); + }, + ); + await check( + 'By date keeps archive format visible and full identifier available at wide widths', + async () => { + const byDate = page.locator('[data-proxmox-backups-table="recoverable"]'); + await byDate.waitFor(); + if (width >= 1120) { + await byDate.getByText('tar.zst', { exact: true }).waitFor(); + assert.ok( + await byDate + .locator('[title="local:backup/vzdump-lxc-310-2026_09_20-02_00_00.tar.zst"]') + .count(), + ); + } + assert.ok((await page.getByText('artifact-cache-310', { exact: true }).count()) >= 2); + }, + ); + if (width !== 768) { + await select('Consent'); + await check( + 'Consent separates running/backup versions and does not promise active-data restoration', + async () => { + const action = page.getByRole('button', { name: 'Roll back', exact: true }); + await action.focus(); + await page.keyboard.press('Enter'); + const dialog = page.getByRole('dialog', { name: 'Confirm rollback' }); + await dialog.getByText('Roll back to Pulse v6.4.5?', { exact: true }).waitFor(); + await dialog.getByText('Running now: Pulse v6.6.0-dev.', { exact: true }).waitFor(); + const text = await dialog.innerText(); + assert.ok(text.includes('Restore scope is not reported for this backup.')); + assert.ok(text.includes('Older recovery can also replace install-local data')); + assert.ok(text.includes('Do not assume this will undo settings or alerts.')); + assert.ok(!text.includes('/private/fixture-backup')); + assert.ok(!text.includes('untrusted restore-everything note')); + assert.equal( + await dialog + .getByRole('link', { name: 'Read the recovery instructions' }) + .getAttribute('href'), + '/docs/AUTO_UPDATE#manual-rollback', + ); + assert.equal(requests.length, 0); + await capture('consent'); + await dialog.getByRole('button', { name: 'Cancel', exact: true }).click(); + assert.equal(await page.getByRole('dialog').count(), 0); + assert.equal(requests.length, 0); + assert.equal(await action.evaluate((e) => e === document.activeElement), true); + }, + ); + await check( + 'Rejection retains consent and pending submission cannot be duplicated or cancelled', + async () => { + await page.getByRole('button', { name: 'Roll back', exact: true }).click(); + await page.getByRole('button', { name: 'Roll back to v6.4.5', exact: true }).click(); + await page + .getByText('Synthetic incompatible backup scope', { exact: true }) + .first() + .waitFor(); + assert.equal(requests.length, 1); + assert.ok( + (await page.getByRole('dialog').innerText()).includes( + 'Restore scope is not reported', + ), + ); + await capture('consent-rejected'); + rollbackMode = 'hold'; + await page.getByRole('button', { name: 'Roll back to v6.4.5', exact: true }).click(); + await page.waitForFunction( + () => document.querySelector('button:disabled')?.textContent, + ); + assert.ok( + await page + .getByRole('button', { name: 'Starting rollback...', exact: true }) + .isDisabled(), + ); + assert.ok(await page.getByRole('button', { name: 'Cancel', exact: true }).isDisabled()); + await page.keyboard.press('Escape'); + assert.equal(await page.getByRole('dialog').count(), 1); + while (!finishRollback) await new Promise((resolve) => setTimeout(resolve, 10)); + assert.equal(requests.length, 2); + rollbackMode = 'accept'; + finishRollback(); + await page.getByRole('dialog').waitFor({ state: 'hidden' }); + assert.deepEqual(requests, [ + { method: 'POST', body: { eventId: 'fixture-update-1' } }, + { method: 'POST', body: { eventId: 'fixture-update-1' } }, + ]); + }, + ); + } + await check('No uncaught application errors or external requests', async () => { + assert.deepEqual(errors, []); + assert.deepEqual(offOrigin, []); + }); + results.push({ + name, + width, + browser: browser.version(), + checks, + errors, + offOrigin, + rollbackRequests: requests, + }); + await page.close(); + await browser.close(); + browser = undefined; + } + } finally { + if (browser) await browser.close(); + await server.close(); + } + const result = { + result: results.every((r) => r.checks.every((c) => c.passed)) ? 'passed' : 'failed', + playwright, + base_sha: '697ad83e19fa5e8afee634c212c945600d2292b4', + source_content_sha256: hashes, + results, + screens, + }; + fs.writeFileSync(path.join(output, 'result.json'), JSON.stringify(result, null, 2)); + console.log( + JSON.stringify({ + result: result.result, + checks: results.flatMap((r) => r.checks).length, + failed: results.flatMap((r) => r.checks.filter((c) => !c.passed)), + screens: screens.length, + }), + ); + if (result.result !== 'passed') process.exitCode = 1; +})().catch((e) => { + console.error(e); + process.exitCode = 1; +}); diff --git a/frontend-modern/browser-tests/operator-tables-consent.html b/frontend-modern/browser-tests/operator-tables-consent.html new file mode 100644 index 0000000000..4aabdd1bcb --- /dev/null +++ b/frontend-modern/browser-tests/operator-tables-consent.html @@ -0,0 +1,12 @@ + + + + + + Operator tables and rollback consent verification + + +
+ + + diff --git a/frontend-modern/browser-tests/operator-tables-consent.tsx b/frontend-modern/browser-tests/operator-tables-consent.tsx new file mode 100644 index 0000000000..17b99932c1 --- /dev/null +++ b/frontend-modern/browser-tests/operator-tables-consent.tsx @@ -0,0 +1,399 @@ +// Production components and CSS with synthetic observations. No native, released or +// installed recovery claim; the current history API does not describe restore scope. +import { createSignal, Show } from 'solid-js'; +import { render } from 'solid-js/web'; +import { Router, Route } from '@solidjs/router'; +import { StoragePoolsTable } from '../src/components/Storage/StoragePoolsTable'; +import { DiskList } from '../src/components/Storage/DiskList'; +import { KubernetesControllersTable } from '../src/features/kubernetes/KubernetesControllersTable'; +import { ProxmoxCoverageTable } from '../src/features/proxmox/ProxmoxCoverageTable'; +import { ProxmoxRecoverableTable } from '../src/features/proxmox/ProxmoxRecoverableTable'; +import { ProxmoxBackupServersTable } from '../src/features/proxmox/ProxmoxBackupServersTable'; +import { ProxmoxNodesTable } from '../src/features/proxmox/ProxmoxNodesTable'; +import { GuestRow } from '../src/components/Workloads/GuestRow'; +import { Table, TableBody } from '../src/components/shared/Table'; +import { ToastContainer } from '../src/components/Toast/Toast'; +import { UpdateHistorySection } from '../src/components/Settings/UpdateHistorySection'; +import { buildProxmoxBackupRecoveryModel } from '../src/features/proxmox/proxmoxBackupRecoveryModel'; +import { groupStorageRecords } from '../src/features/storageBackups/storageModelCore'; +import { EMPTY_STORAGE_ALERT_STATE } from '../src/features/storageBackups/storageAlertState'; +import { DarkModeContext, WebSocketContext } from '../src/contexts/appRuntime'; +import { updateStore } from '../src/stores/updates'; +import type { Resource } from '../src/types/resource'; +import type { StorageRecord } from '../src/features/storageBackups/models'; +import '../src/index.css'; + +const now = Date.parse('2026-10-03T12:00:00Z'); +const resource = (id: string, type: Resource['type'], extra: Partial = {}): Resource => ({ + id, + type, + name: id, + displayName: id, + platformId: 'fixture-estate', + platformType: 'proxmox-pve', + sourceType: 'api', + status: 'online', + lastSeen: now, + ...extra, +}); +const pools: StorageRecord[] = [ + ['archive-tank', 'DEGRADED', 'resilver'], + ['backup-pool', 'ONLINE', 'scrub'], +].map(([name, state, scan], index) => ({ + id: name, + name, + category: 'pool', + health: index ? 'healthy' : 'warning', + location: { label: index ? 'backup-nas' : 'archive-nas', scope: 'host' }, + source: { + platform: 'truenas', + family: 'onprem', + origin: 'resource', + adapterId: 'resource-storage', + }, + capacity: { totalBytes: 2e12, usedBytes: 1.18e12, freeBytes: 0.82e12, usagePercent: 59 }, + capabilities: ['capacity', 'health'], + observedAt: now, + protectionLabel: `ZFS pool ${name} ${scan} is running (45.2%)`, + protectionSummary: `ZFS pool ${name} ${scan} is running (45.2%)`, + rebuildInProgress: true, + details: { + zfsPool: { + state, + scan: `${scan} in progress`, + scanDetails: { + function: scan.toUpperCase(), + state: 'SCANNING', + percentage: 45.2, + }, + devices: [], + }, + }, +})); +const disks = [ + resource('disk-sda', 'physical_disk', { + identity: { hostname: 'archive-nas' }, + canonicalIdentity: { hostname: 'archive-nas' }, + physicalDisk: { + devPath: '/dev/sda', + model: 'Archive HDD', + serial: 'FIXTURE-1', + diskType: 'hdd', + sizeBytes: 2e12, + health: 'FAILED', + temperature: 42, + storageRole: 'data', + risk: { + level: 'critical', + reasons: [ + { + code: 'smart-failed', + severity: 'critical', + summary: 'SMART failed. The full reason remains readable in the expanded disk header.', + }, + ], + }, + }, + }), + resource('disk-nvme', 'physical_disk', { + identity: { hostname: 'backup-nas' }, + canonicalIdentity: { hostname: 'backup-nas' }, + physicalDisk: { + devPath: '/dev/nvme0n1', + model: 'Backup SSD', + serial: 'FIXTURE-2', + diskType: 'nvme', + sizeBytes: 1e12, + health: 'PASSED', + temperature: 44, + wearout: 2, + storageRole: 'cache', + }, + }), +]; +const controllers = [ + resource('nightly-import', 'k8s-job', { + platformType: 'kubernetes', + kubernetes: { + resourceKind: 'Job', + clusterName: 'prod', + namespace: 'batch', + startTime: '2026-10-03T11:00:00Z', + completionTime: '2026-10-03T11:05:00Z', + desiredReplicas: 1, + succeeded: 1, + active: 0, + failed: 0, + }, + }), + resource('billing-rollup', 'k8s-cronjob', { + platformType: 'kubernetes', + kubernetes: { + resourceKind: 'CronJob', + clusterName: 'prod', + namespace: 'batch', + schedule: '*/5 * * * *', + lastScheduleTime: '2026-10-03T11:00:00Z', + lastSuccessfulTime: '2026-10-03T11:05:00Z', + suspend: true, + active: 0, + }, + }), + resource('node-exporter', 'k8s-daemonset', { + platformType: 'kubernetes', + kubernetes: { + resourceKind: 'DaemonSet', + clusterName: 'prod', + namespace: 'observability', + desiredNumberScheduled: 3, + currentNumberScheduled: 3, + numberReady: 2, + numberUnavailable: 1, + }, + }), + resource('database', 'k8s-statefulset', { + platformType: 'kubernetes', + kubernetes: { + resourceKind: 'StatefulSet', + clusterName: 'prod', + namespace: 'apps', + desiredReplicas: 1, + readyReplicas: 1, + serviceName: 'database-headless', + }, + }), +]; +const workload = resource('ct-310', 'system-container', { + name: 'artifact-cache-310', + displayName: 'artifact-cache-310', + status: 'running', + proxmox: { vmid: 310, node: 'pve-edge', instance: 'homelab' }, +}); +const model = buildProxmoxBackupRecoveryModel({ + workloads: [workload], + pbsBackups: [], + tasks: [], + nowMs: now, + archives: [ + { + id: 'file-310', + storage: 'local', + node: 'pve-edge', + instance: 'homelab', + type: 'ct', + vmid: 310, + time: '2026-09-20T02:00:00Z', + ctime: Date.parse('2026-09-20T02:00:00Z') / 1000, + size: 1048576, + format: 'tar.zst', + protected: false, + volid: 'local:backup/vzdump-lxc-310-2026_09_20-02_00_00.tar.zst', + isPBS: false, + verified: false, + }, + ], + snapshots: [ + { + id: 'snap-310', + name: 'fresh-local-snapshot', + node: 'pve-edge', + instance: 'homelab', + type: 'ct', + vmid: 310, + time: '2026-10-03T11:30:00Z', + vmstate: false, + }, + ], +}); +const pbs = resource('pbs-archive', 'pbs', { + platformType: 'proxmox-pbs', + cpu: { current: 14 }, + memory: { current: 32, total: 8e9, used: 2.56e9, free: 5.44e9 }, + pbs: { + instance: 'pbs-archive', + datastores: [ + { + name: 'archive', + total: 2e12, + used: 1.18e12, + free: 0.82e12, + usage: 59, + status: 'available', + }, + ], + }, +}); +const node = resource('pve-edge', 'node', { + cpu: { current: 12, cores: 4 }, + memory: { current: 32, total: 8e9, used: 2.56e9, free: 5.44e9 }, + disk: { current: 30, total: 1e12, used: 0.3e12, free: 0.7e12 }, + proxmox: { node: 'pve-edge', instance: 'homelab', uptime: 86400 }, +}); +function Fixture() { + const [view, setView] = createSignal('Pools'); + const [disk, setDisk] = createSignal(null); + const [pool, setPool] = createSignal(null); + const [expanded, setExpanded] = createSignal(new Set()); + const ws = { + state: { pmg: [], pbs: [], nodes: [], resources: [], vms: [], containers: [] }, + activeAlerts: () => ({}), + connected: () => true, + initialDataReceived: () => true, + }; + return ( + + document.documentElement.classList.contains('dark')}> + +
+

Operator tables and rollback consent

+ + +
+ {}} + expandedPoolId={pool()} + setExpandedPoolId={setPool} + storageGrowthBySeriesId={new Map()} + storageGrowthColumnLabel="Growth" + physicalDisks={[]} + nodeOnlineByLabel={new Map()} + highlightedRecordId={null} + getRecordAlertState={() => EMPTY_STORAGE_ALERT_STATE} + isLoading={false} + /> +
+
+ +
+ +
+
+ +
+ } + emptyTitle="No controllers" + emptyDescription="No controllers" + showToolbar={false} + /> +
+
+ +
+

Coverage

+ } + emptyTitle="No backups" + emptyDescription="No backups" + sortKey={() => 'posture'} + sortDirection={() => 'asc'} + onSort={() => {}} + expandedKeys={expanded()} + onToggleExpand={(key) => + setExpanded((current) => + current.has(key) + ? new Set([...current].filter((item) => item !== key)) + : new Set([...current, key]), + ) + } + showPbsColumn + showArchiveColumn + showSnapshotColumn + showTaskColumn={false} + /> +

By date

+ } + emptyTitle="No artifacts" + emptyDescription="No artifacts" + sortKey={() => 'created'} + sortDirection={() => 'desc'} + onSort={() => {}} + sizeMaxBytes={1048576} + /> +

Backup servers

+ +

Proxmox nodes

+ } + emptyTitle="No nodes" + emptyDescription="No nodes" + /> +

Workload row

+ + + + +
+
+
+ +
+ +
+
+
+
+
+ ); +} +void updateStore.checkForUpdates(); +render( + () => ( + + + + ), + document.getElementById('root')!, +); diff --git a/frontend-modern/browser-tests/proxmox-docker-rows.cjs b/frontend-modern/browser-tests/proxmox-docker-rows.cjs new file mode 100644 index 0000000000..bcc376eded --- /dev/null +++ b/frontend-modern/browser-tests/proxmox-docker-rows.cjs @@ -0,0 +1,497 @@ +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const crypto = require('node:crypto'); +const { chromium, webkit } = require('playwright'); + +const now = '2026-10-03T12:00:00Z'; +const base = (id, type, extra = {}) => ({ + id, + type, + name: id, + status: 'online', + lastSeen: now, + sources: ['docker'], + ...extra, +}); +const observations = (mode) => [ + base('mail-gateway-eu', 'pmg', { + sources: ['pmg'], + technology: 'pmg', + uptime: 86400, + sourceStatus: { pmg: { status: 'online', lastSeen: now } }, + ...(mode === 'missing' + ? {} + : { + pmg: { + instanceId: 'pmg-eu', + hostname: 'mail-gateway-eu', + version: '8.1-2', + nodeCount: 2, + mailCountTotal: 2740, + spamIn: 321, + virusIn: 14, + quarantine: 3, + queueTotal: mode === 'fresh' ? 19 : 11, + queueDeferred: 5, + }, + }), + }), + base('nfs-iso', 'storage', { + sources: ['proxmox'], + parentName: 'cluster', + parentId: 'pve-cluster-a', + technology: 'nfs', + metrics: { disk: { total: 1e12, used: 3e11, percent: 30 } }, + storage: { type: 'nfs', shared: true, nodes: ['pve1', 'pve2', 'pve3'] }, + }), + base('local-lvm', 'storage', { + sources: ['proxmox'], + parentName: 'pve2', + parentId: 'pve2', + metrics: { disk: { total: 1e12, used: 6e11, percent: 60 } }, + storage: { type: 'lvmthin', shared: false }, + }), + base('edge-web', 'app-container', { + status: 'running', + technology: 'docker', + sourceStatus: { docker: { status: 'online', lastSeen: now } }, + actionReadiness: [{ name: 'restart', available: false, reason: 'Command agent disconnected' }], + metrics: { cpu: { percent: 42 }, memory: { percent: 50, used: 5e8, total: 1e9 } }, + docker: { + agentId: 'agent-edge', + hostSourceId: 'agent-edge', + containerId: 'native-edge-web', + hostname: 'edge-long-hostname', + runtime: 'docker', + runtimeVersion: '27.5.1', + image: 'nginx:latest', + containerState: 'running', + restartCount: 7, + updateStatus: { + updateAvailable: true, + currentDigest: 'sha256:current', + latestDigest: 'sha256:latest', + lastChecked: now, + }, + }, + }), + base('edge-cache', 'app-container', { + status: 'offline', + metrics: { cpu: { percent: 0 }, memory: { percent: 0 } }, + docker: { + agentId: 'agent-edge', + hostname: 'edge-long-hostname', + runtime: 'podman', + runtimeVersion: '5.2.1', + image: 'redis:7.4', + containerState: 'exited', + restartCount: 0, + }, + }), + base('nginx:latest', 'docker-image', { + docker: { + hostname: 'edge-long-hostname', + imageId: 'image-nginx', + repoTags: ['nginx:latest'], + sizeBytes: 2e8, + containers: ['edge-web'], + containerCount: 1, + }, + }), + base('web-service', 'docker-service', { + docker: { + hostname: 'edge-long-hostname', + image: 'nginx:latest', + serviceMode: 'replicated', + desiredTasks: 4, + runningTasks: 2, + mode: 'replicated', + serviceUpdate: { state: 'rollback_started', message: 'Synthetic rollout observation' }, + }, + }), + base('web.2', 'docker-task', { + docker: { + hostname: 'edge-long-hostname', + serviceName: 'web-service', + nodeName: 'worker-1', + currentState: 'running', + desiredState: 'running', + startedAt: '2026-10-03T11:30:00.123456Z', + }, + }), +]; + +(async () => { + const root = '/workspace/frontend-modern'; + const output = '/workspace/tmp/proxmox-docker-rows-final-source'; + fs.mkdirSync(output, { recursive: true }); + process.chdir(root); + const playwright = require('playwright/package.json').version; + assert.equal( + playwright, + JSON.parse(fs.readFileSync('/workspace/tests/integration/package-lock.json')).packages[ + 'node_modules/@playwright/test' + ].version, + ); + // Keep the fixture runnable from clean committed source, without requiring + // an invocation-specific untracked runtime-paths input. + const paths = [ + 'frontend-modern/src/components/Workloads/StackedDiskBar.tsx', + 'frontend-modern/src/components/Workloads/stackedDiskBarModel.ts', + 'frontend-modern/src/features/docker/DockerImagesTable.tsx', + 'frontend-modern/src/features/docker/DockerServicesTable.tsx', + 'frontend-modern/src/features/docker/DockerTasksTable.tsx', + 'frontend-modern/src/features/docker/dockerContainerTableModel.ts', + 'frontend-modern/src/features/storageBackups/resourceStoragePresentation.ts', + 'frontend-modern/src/features/storageBackups/storageAdapters.ts', + 'frontend-modern/src/hooks/useUnifiedResources.ts', + ]; + const hashes = Object.fromEntries( + paths.map((p) => [ + p, + crypto + .createHash('sha256') + .update(fs.readFileSync(path.join('/workspace', p))) + .digest('hex'), + ]), + ); + const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js')); + const server = await createServer({ + root, + configFile: path.join(root, 'vite.config.ts'), + cacheDir: path.join(output, 'vite-cache'), + server: { host: '127.0.0.1', port: 5272, strictPort: true }, + }); + const results = [], + screenshots = []; + let browser; + try { + await server.listen(); + for (const [name, engine, width] of [ + ['desktop', chromium, 1440], + ['phone', webkit, 390], + ]) { + browser = await engine.launch( + engine === chromium + ? { headless: true, channel: 'chromium', args: ['--no-sandbox'] } + : { headless: true }, + ); + const page = await browser.newPage({ + viewport: { width, height: width === 390 ? 844 : 900 }, + isMobile: width === 390, + hasTouch: width === 390, + locale: 'en-GB', + timezoneId: 'UTC', + }); + await page.clock.setFixedTime(new Date(now)); + page.setDefaultTimeout(15000); + const checks = [], + errors = [], + offOrigin = [], + mutations = []; + let mode = 'initial'; + page.on('pageerror', (e) => errors.push(e.message)); + await page.route('**/*', async (route) => { + const request = route.request(), + url = new URL(request.url()); + if (url.origin !== 'http://127.0.0.1:5272') { + offOrigin.push(url.origin); + return route.abort(); + } + if (!url.pathname.startsWith('/api/')) return route.continue(); + if (request.method() !== 'GET') { + mutations.push({ path: url.pathname, method: request.method() }); + return route.fulfill({ status: 409, json: { error: 'No fixture mutation' } }); + } + if (url.pathname === '/api/resources') + return route.fulfill({ json: { data: observations(mode) } }); + if (url.pathname === '/api/license/runtime-capabilities') + return route.fulfill({ + json: { + capabilities: [], + limits: [], + max_history_days: 7, + hosted_mode: false, + blocked_capabilities: [], + runtime: { build: 'community' }, + }, + }); + if (url.pathname === '/api/security/status') + return route.fulfill({ json: { hasAuthentication: true, requiresAuth: true } }); + return route.fulfill({ + json: { + enabled: false, + data: [], + capabilities: [], + facets: [], + relationships: [], + recentChanges: [], + alerts: {}, + }, + }); + }); + const check = async (label, run) => { + try { + await run(); + checks.push({ label, passed: true }); + } catch (e) { + checks.push({ label, passed: false, error: e.message }); + } + }; + const select = (view) => + page + .getByRole('navigation', { name: 'Fixture views' }) + .getByRole('button', { name: view, exact: true }) + .click(); + const capture = async (state) => { + await page.waitForTimeout(180); // ResizeObserver and entrance layout, not freshness evidence. + const dimensions = await page.evaluate(() => ({ + width: innerWidth, + scroll: document.documentElement.scrollWidth, + })); + checks.push({ + label: `No horizontal page overflow: ${state}`, + passed: dimensions.scroll <= dimensions.width + 1, + dimensions, + }); + const file = `${name}-${state}.png`; + await page.screenshot({ path: path.join(output, file), fullPage: true }); + screenshots.push({ file, state, engine: name, width, dimensions }); + }; + await page.goto('http://127.0.0.1:5272/browser-tests/proxmox-docker-rows.html', { + waitUntil: 'domcontentloaded', + timeout: 120000, + }); + if (width === 390) await page.evaluate(() => document.documentElement.classList.add('dark')); + await page.getByText('8 REST-mapped rows', { exact: true }).waitFor(); + await check( + 'REST-first PMG counters and facets are present before any websocket hydration', + async () => { + const table = page.getByRole('region', { name: 'Mail table' }); + await table.getByText('2,740', { exact: true }).waitFor(); + await table.getByText('11', { exact: true }).waitFor(); + const row = await page.evaluate(() => + window.__operatorRows.rows().find((r) => r.type === 'pmg'), + ); + assert.equal(row.technology, 'pmg'); + assert.equal(row.sourceStatus.pmg.status, 'online'); + assert.equal(row.pmg.queueTotal, 11); + assert.deepEqual(row.pmg, row.platformData.pmg); + if (width > 390) await table.getByText('8.1-2', { exact: true }).waitFor(); + }, + ); + await capture('rest-mail'); + await check( + 'Canonical store hydration followed by REST refresh keeps the new PMG counters', + async () => { + await page.evaluate(() => window.__operatorRows.seedWebsocket()); + mode = 'fresh'; + await page.evaluate(() => window.__operatorRows.refresh()); + await page + .getByRole('region', { name: 'Mail table' }) + .getByText('19', { exact: true }) + .waitFor(); + }, + ); + await capture('refreshed-mail'); + await check('A removed REST PMG facet does not retain obsolete queue evidence', async () => { + mode = 'missing'; + await page.evaluate(() => window.__operatorRows.refresh()); + assert.equal( + await page + .getByRole('region', { name: 'Mail table' }) + .getByText('19', { exact: true }) + .count(), + 0, + ); + const row = await page.evaluate(() => + window.__operatorRows.rows().find((r) => r.type === 'pmg'), + ); + assert.equal(row.pmg, undefined); + mode = 'fresh'; + await page.evaluate(() => window.__operatorRows.refresh()); + }); + await select('Storage'); + await check('Shared storage vocabulary preserves native locations and IDs', async () => { + const table = page.getByRole('region', { name: 'Storage table' }); + await table.getByText('NFS', { exact: true }).waitFor({ state: 'attached' }); + await table.getByText('LVM-Thin', { exact: true }).waitFor({ state: 'attached' }); + await table.getByText('Shared · 3 nodes', { exact: true }).waitFor({ state: 'attached' }); + await table.getByText('pve2', { exact: true }).waitFor({ state: 'attached' }); + const rows = await page.evaluate(() => + window.__operatorRows.rows().filter((r) => r.type === 'storage'), + ); + assert.deepEqual( + rows.map((r) => [r.id, r.parentId]), + [ + ['nfs-iso', 'pve-cluster-a'], + ['local-lvm', 'pve2'], + ], + ); + }); + await capture('storage'); + await select('Containers'); + await check( + 'REST lifecycle refusal remains available; phone row keeps update and moves restarts to detail', + async () => { + await page + .locator('[data-lifecycle-refusal]') + .getByText('Command agent disconnected', { exact: true }) + .waitFor(); + const table = page.getByRole('region', { name: 'Container table' }); + const update = table.locator('.docker-container-update-cell button'); + await update.waitFor(); + assert.match(await update.innerText(), /Update/); + const bounds = await update.evaluate((e) => ({ + w: e.getBoundingClientRect().width, + cell: e.closest('td').getBoundingClientRect().width, + scroll: e.scrollWidth, + client: e.clientWidth, + })); + assert.ok( + bounds.w <= bounds.cell + 1 && bounds.scroll <= bounds.client + 1, + JSON.stringify(bounds), + ); + const headers = await table.locator('th:visible').allTextContents(); + if (width === 390) { + assert.ok(!headers.some((h) => h.includes('Restarts'))); + assert.ok(headers.some((h) => h.includes('State'))); + } + }, + ); + await capture('containers'); + await check( + 'Restart count and native identity remain reachable via row disclosure', + async () => { + const row = page.locator('[data-docker-container-row="edge-web"]'); + if (width === 390) await row.locator('td').first().tap(); + else await row.getByRole('button', { name: /details/i }).click(); + const detail = page.getByTestId('resource-docker-container-section'); + await detail.getByText('Restarts', { exact: true }).waitFor(); + await detail.getByText('7', { exact: true }).waitFor(); + }, + ); + await capture('container-detail'); + await select('Images'); + await check('Image verdict fits; phone hidden headers and cells are symmetric', async () => { + const table = page.getByRole('region', { name: 'Image table' }); + await table.getByText('nginx:latest', { exact: true }).waitFor(); + const headers = await table.locator('th:visible').allTextContents(); + const row = table.locator('[data-docker-image-row]'); + assert.equal(await row.locator('td:visible').count(), headers.length); + if (width === 390) { + assert.ok(!headers.some((h) => h.includes('Host') || h.includes('Used by'))); + assert.ok(headers.some((h) => h.includes('Size'))); + } + }); + await capture('images'); + await select('Swarm'); + await check( + 'Swarm state and started time use operator wording with raw evidence retained', + async () => { + const state = page.getByText('Rollback started', { exact: true }); + await state.waitFor({ state: 'attached' }); + assert.match( + await state.locator('xpath=ancestor-or-self::*[@title][1]').getAttribute('title'), + /rollback_started/, + ); + const visibleState = await state.evaluate((element) => { + const cell = element.closest('td').getBoundingClientRect(); + const range = document.createRange(); + range.selectNodeContents(element); + return [...range.getClientRects()].every( + (line) => + line.left >= cell.left && + line.right <= cell.right && + line.top >= cell.top && + line.bottom <= cell.bottom, + ); + }); + assert.equal(visibleState, true, 'Every word of the rollout state must fit its cell'); + if (width === 390) { + await page.getByText('Run', { exact: true }).waitFor({ state: 'visible' }); + } + const started = page.locator('[title="2026-10-03T11:30:00.123456Z"]'); + await started.waitFor({ state: 'attached' }); + assert.match(await started.innerText(), /ago/); + await page + .getByRole('region', { name: 'Swarm tasks' }) + .getByTitle('running', { exact: true }) + .waitFor(); + }, + ); + await capture('swarm'); + await check( + 'Swarm touch or keyboard disclosure exposes raw rollout and placement evidence', + async () => { + const row = page.locator('[data-docker-service-row="web-service"]'); + if (width === 390) await row.locator('td').first().tap(); + else { + const toggle = row.getByRole('button', { + name: 'Expand details for web-service', + exact: true, + }); + await toggle.focus(); + await page.keyboard.press('Enter'); + } + const detail = page.locator('[data-inline-docker-service-detail-for="web-service"]'); + await detail + .getByText('rollback_started | Synthetic rollout observation', { exact: true }) + .waitFor(); + await detail.getByText('edge-long-hostname', { exact: true }).waitFor(); + await detail.getByText('4', { exact: true }).waitFor(); + await detail.getByText('2', { exact: true }).waitFor(); + }, + ); + await capture('swarm-detail'); + await select('Disk'); + await check( + 'Multi-disk Bars show the measured fullest disk and unchanged equal-weight bars', + async () => { + const label = page.locator('[data-stacked-disk-max-label]'); + assert.equal(await label.innerText(), '92%'); + assert.equal(await label.getAttribute('title'), 'Highest usage: /data 92%'); + assert.equal(await page.locator('[data-stacked-disk-fill="vertical"]').count(), 2); + }, + ); + await capture('disks'); + checks.push({ + label: 'No page errors, external requests or mutation requests', + passed: !errors.length && !offOrigin.length && !mutations.length, + errors, + offOrigin, + mutations, + }); + results.push({ engine: name, version: browser.version(), width, checks }); + await browser.close(); + browser = undefined; + } + } finally { + if (browser) await browser.close(); + await server.close(); + } + const result = { + playwright, + verified_at: new Date().toISOString(), + paths, + hashes, + results, + screenshots, + }; + fs.writeFileSync(path.join(output, 'result.json'), JSON.stringify(result, null, 2)); + const failures = results.flatMap((r) => + r.checks.filter((c) => !c.passed).map((c) => ({ engine: r.engine, ...c })), + ); + console.log( + JSON.stringify({ + checked: results.reduce((n, r) => n + r.checks.length, 0), + screenshots: screenshots.length, + failures, + }), + ); + assert.equal(failures.length, 0, JSON.stringify(failures)); +})().catch((error) => { + console.error(error); + process.exitCode = 1; +}); diff --git a/frontend-modern/browser-tests/proxmox-docker-rows.html b/frontend-modern/browser-tests/proxmox-docker-rows.html new file mode 100644 index 0000000000..5e96968d98 --- /dev/null +++ b/frontend-modern/browser-tests/proxmox-docker-rows.html @@ -0,0 +1,12 @@ + + + + + + Transport facets and operator rows verification + + +
+ + + diff --git a/frontend-modern/browser-tests/proxmox-docker-rows.tsx b/frontend-modern/browser-tests/proxmox-docker-rows.tsx new file mode 100644 index 0000000000..2f0285c0be --- /dev/null +++ b/frontend-modern/browser-tests/proxmox-docker-rows.tsx @@ -0,0 +1,182 @@ +// Synthetic REST observations through the real hook, tables, detail disclosures +// and CSS. No real websocket, appliance or installed/released acceptance claim. +import { createSignal, Show } from 'solid-js'; +import { createStore, unwrap } from 'solid-js/store'; +import { render } from 'solid-js/web'; +import { Router, Route } from '@solidjs/router'; +import { useUnifiedResources } from '../src/hooks/useUnifiedResources'; +import { ProxmoxMailGatewayTable } from '../src/features/proxmox/ProxmoxMailGatewayTable'; +import { DockerContainersTable } from '../src/features/docker/DockerContainersTable'; +import { DockerImagesTable } from '../src/features/docker/DockerImagesTable'; +import { DockerServicesTable } from '../src/features/docker/DockerServicesTable'; +import { DockerTasksTable } from '../src/features/docker/DockerTasksTable'; +import { StackedDiskBar } from '../src/components/Workloads/StackedDiskBar'; +import { StoragePoolsTable } from '../src/components/Storage/StoragePoolsTable'; +import { Table, TableBody, TableRow, TableCell } from '../src/components/shared/Table'; +import { getDockerContainerLifecycleDisabledReason } from '../src/features/docker/dockerContainerLifecycleActions'; +import { buildStorageRecords } from '../src/features/storageBackups/storageAdapters'; +import { groupStorageRecords } from '../src/features/storageBackups/storageModelCore'; +import { EMPTY_STORAGE_ALERT_STATE } from '../src/features/storageBackups/storageAlertState'; +import { DarkModeContext, WebSocketContext } from '../src/contexts/appRuntime'; +import type { Resource } from '../src/types/resource'; +import '../src/index.css'; + +const [wsState, setWsState] = createStore({ + resources: [] as Resource[], + lastUpdate: 0, + nodes: [], + pbs: [], + pmg: [], + vms: [], + containers: [], +}); +const [connected, setConnected] = createSignal(false); +const ws = { + state: wsState, + connected, + initialDataReceived: () => true, + resourceChange: () => ({ version: wsState.lastUpdate, changedIds: null, changedKeys: null }), + activeAlerts: () => ({}), + shutdown: () => {}, +}; +window.__pulseWsStore = ws as any; + +function Fixture() { + const [view, setView] = createSignal('Mail'); + const [expandedPool, setExpandedPool] = createSignal(null); + const rows = useUnifiedResources({ query: '', cacheKey: 'operator-rows-browser' }); + const ofType = (type: string) => rows.resources().filter((row) => row.type === type); + const container = () => ofType('app-container')[0]; + const records = () => + buildStorageRecords({ state: wsState as any, resources: ofType('storage') }); + const grouped = () => groupStorageRecords(records(), 'none'); + (window as any).__operatorRows = { + rows: () => JSON.parse(JSON.stringify(unwrap(rows.resources()))), + refresh: () => rows.refetch(), + seedWebsocket: () => { + setWsState('resources', JSON.parse(JSON.stringify(unwrap(rows.resources())))); + setWsState('lastUpdate', Date.now()); + setConnected(true); + }, + }; + const nativeProps = { + emptyIcon: , + emptyTitle: 'No fixture rows', + emptyDescription: 'No fixture rows', + showToolbar: false, + }; + return ( + + document.documentElement.classList.contains('dark')}> +
+

Transport facets and operator rows

+ +

{rows.resources().length} REST-mapped rows

+ +
+ +
+
+ +
+ {}} + expandedPoolId={expandedPool()} + setExpandedPoolId={setExpandedPool} + storageGrowthBySeriesId={new Map()} + storageGrowthColumnLabel="Growth" + physicalDisks={[]} + nodeOnlineByLabel={new Map()} + highlightedRecordId={null} + getRecordAlertState={() => EMPTY_STORAGE_ALERT_STATE} + isLoading={false} + /> +
+
+ +
+

+ {container() && getDockerContainerLifecycleDisabledReason(container(), 'restart')} +

+ +
+
+ +
+ +
+
+ +
+ +
+
+ +
+
+ +
+ + + + pve-edge + + + + + +
+
+
+
+
+
+ ); +} +render( + () => ( + + + + ), + document.getElementById('root')!, +); diff --git a/frontend-modern/src/components/Infrastructure/ResourceDetailSummary.tsx b/frontend-modern/src/components/Infrastructure/ResourceDetailSummary.tsx index 50e454e147..2cdbf67ff4 100644 --- a/frontend-modern/src/components/Infrastructure/ResourceDetailSummary.tsx +++ b/frontend-modern/src/components/Infrastructure/ResourceDetailSummary.tsx @@ -13,6 +13,7 @@ import { TagBadges } from '@/components/shared/TagBadges'; import { formatBytes, formatRelativeTime, formatUptime } from '@/utils/format'; import { getDockerImageRegistryLink } from '@/features/docker/dockerImageReference'; import { formatInteger } from './resourceDetailMappers'; +import { buildKubernetesControllerSection } from './resourceDetailDrawerKubernetesModel'; import type { UseResourceDetailDrawerStateResult } from './useResourceDetailDrawerState'; interface ResourceSummaryPresentationProps { @@ -285,6 +286,7 @@ export const InlineResourceSummaryTables: Component { ).toBe('https://prod.k8s.local:6443'); }); }); + +describe('buildKubernetesControllerSection', () => { + afterEach(() => { + vi.useRealTimers(); + }); + + const absolute = (timestamp: string): string => + formatPlatformTableDateTimeValue(timestamp, { dateTimeFormat: { year: 'numeric' } }); + + it('returns nothing for resources that are not workload controllers', () => { + expect( + buildKubernetesControllerSection( + makeResource({ id: 'pod-1', type: 'pod', kubernetes: { namespace: 'apps' } }), + ), + ).toBeNull(); + expect(buildKubernetesControllerSection(makeResource({ id: 'vm-1', type: 'vm' }))).toBeNull(); + expect( + buildKubernetesControllerSection(makeResource({ id: 'job-1', type: 'k8s-job' })), + ).toBeNull(); + }); + + it('leads a Job with its absolute start and completion times, then the counts', () => { + vi.useFakeTimers(); + vi.setSystemTime(new Date('2026-05-24T13:31:00Z')); + + const section = buildKubernetesControllerSection( + makeResource({ + id: 'nightly-import', + type: 'k8s-job', + kubernetes: { + clusterName: 'prod', + namespace: 'batch', + desiredReplicas: 10, + active: 1, + succeeded: 8, + failed: 2, + startTime: '2026-05-24T12:55:00Z', + completionTime: '2026-05-24T13:00:00Z', + }, + }), + ); + + expect(section?.label).toBe('Job'); + expect(section?.testId).toBe('resource-kubernetes-controller-section'); + expect(section?.rows).toEqual([ + { label: 'Started', value: `${absolute('2026-05-24T12:55:00Z')} (36m ago)`, wrap: true }, + { label: 'Completed', value: `${absolute('2026-05-24T13:00:00Z')} (31m ago)`, wrap: true }, + { label: 'Duration', value: '5m' }, + { label: 'Target', value: '10 completions' }, + { label: 'Active', value: '1' }, + { label: 'Succeeded', value: '8' }, + { label: 'Failed', value: '2', tone: 'danger' }, + { label: 'Namespace', value: 'batch' }, + { label: 'Cluster', value: 'prod' }, + ]); + }); + + it('keeps a running Job on its start time without a completion or duration row', () => { + vi.useFakeTimers(); + vi.setSystemTime(new Date('2026-05-24T13:31:00Z')); + + const section = buildKubernetesControllerSection( + makeResource({ + id: 'nightly-import', + type: 'k8s-job', + kubernetes: { + namespace: 'batch', + desiredReplicas: 1, + active: 1, + failed: 0, + startTime: '2026-05-24T13:29:30Z', + }, + }), + ); + + expect(section?.rows.map((row) => row.label)).toEqual([ + 'Started', + 'Target', + 'Active', + 'Failed', + 'Namespace', + ]); + expect(section?.rows[0]).toEqual({ + label: 'Started', + value: `${absolute('2026-05-24T13:29:30Z')} (1m ago)`, + wrap: true, + }); + expect(section?.rows[3]).toEqual({ label: 'Failed', value: '0', tone: 'default' }); + }); + + it('leads a CronJob with its schedule and the last run and last success times', () => { + vi.useFakeTimers(); + vi.setSystemTime(new Date('2026-05-24T13:31:00Z')); + + const section = buildKubernetesControllerSection( + makeResource({ + id: 'billing-rollup', + type: 'k8s-cronjob', + kubernetes: { + clusterName: 'prod', + namespace: 'batch', + schedule: '0 2 * * *', + suspend: true, + active: 0, + lastScheduleTime: '2026-05-24T02:00:00Z', + lastSuccessfulTime: '2026-05-23T02:00:00Z', + }, + }), + ); + + expect(section?.label).toBe('CronJob'); + expect(section?.rows).toEqual([ + { label: 'Schedule', value: '0 2 * * *', valueClass: 'font-mono' }, + { label: 'Last run', value: `${absolute('2026-05-24T02:00:00Z')} (11h ago)`, wrap: true }, + { label: 'Last success', value: `${absolute('2026-05-23T02:00:00Z')} (1d ago)`, wrap: true }, + { label: 'Suspended', value: 'Yes', tone: 'warning' }, + { label: 'Active', value: '0' }, + { label: 'Namespace', value: 'batch' }, + { label: 'Cluster', value: 'prod' }, + ]); + }); + + it('carries the service name and counts for StatefulSets and DaemonSets', () => { + const statefulSet = buildKubernetesControllerSection( + makeResource({ + id: 'checkout-api-stateful', + type: 'k8s-statefulset', + kubernetes: { + namespace: 'apps', + desiredReplicas: 3, + currentReplicas: 3, + readyReplicas: 2, + availableReplicas: 2, + updatedReplicas: 2, + serviceName: 'checkout-headless', + }, + }), + ); + expect(statefulSet?.label).toBe('StatefulSet'); + expect(statefulSet?.rows).toEqual([ + { label: 'Service', value: 'checkout-headless' }, + { label: 'Target', value: '3 pods' }, + { label: 'Current', value: '3' }, + { label: 'Ready', value: '2' }, + { label: 'Available', value: '2' }, + { label: 'Updated', value: '2' }, + { label: 'Namespace', value: 'apps' }, + ]); + + const daemonSet = buildKubernetesControllerSection( + makeResource({ + id: 'node-exporter', + type: 'k8s-daemonset', + kubernetes: { + namespace: 'observability', + desiredNumberScheduled: 6, + currentNumberScheduled: 6, + numberReady: 5, + numberAvailable: 5, + numberUnavailable: 1, + numberMisscheduled: 0, + updatedReplicas: 5, + }, + }), + ); + expect(daemonSet?.label).toBe('DaemonSet'); + expect(daemonSet?.rows).toEqual([ + { label: 'Target', value: '6 nodes' }, + { label: 'Current', value: '6' }, + { label: 'Ready', value: '5' }, + { label: 'Available', value: '5' }, + { label: 'Updated', value: '5' }, + { label: 'Unavailable', value: '1', tone: 'warning' }, + { label: 'Misscheduled', value: '0', tone: 'default' }, + { label: 'Namespace', value: 'observability' }, + ]); + }); + + it('shows the ReplicaSet labelling and generation fields the Detail column carries', () => { + const section = buildKubernetesControllerSection( + makeResource({ + id: 'checkout-api-replicaset', + type: 'k8s-replicaset', + kubernetes: { + desiredReplicas: 4, + readyReplicas: 3, + fullyLabeledReplicas: 4, + observedGeneration: 7, + }, + }), + ); + expect(section?.label).toBe('ReplicaSet'); + expect(section?.rows).toEqual([ + { label: 'Target', value: '4 pods' }, + { label: 'Ready', value: '3' }, + { label: 'Fully labeled', value: '4' }, + { label: 'Observed generation', value: '7' }, + ]); + }); +}); diff --git a/frontend-modern/src/components/Infrastructure/resourceDetailDrawerKubernetesModel.ts b/frontend-modern/src/components/Infrastructure/resourceDetailDrawerKubernetesModel.ts index 5d3d5e9275..2d44225f6b 100644 --- a/frontend-modern/src/components/Infrastructure/resourceDetailDrawerKubernetesModel.ts +++ b/frontend-modern/src/components/Infrastructure/resourceDetailDrawerKubernetesModel.ts @@ -3,9 +3,18 @@ import { compactDetailRows as compactRows, compactDetailSections as compactSections, formatDetailBytesValue, + formatDetailCountValue, + formatDetailIntegerValue, makeDetailRow as makeRow, + type DetailRow, type DetailSection, + type DetailValueTone, } from '@/components/shared/detailSectionModel'; +import { + formatPlatformTableDateTimeValue, + formatPlatformTableDurationValue, + formatPlatformTableRelativeTimeValue, +} from '@/features/platformPage/sharedPlatformPage'; export type ResourceDetailDrawerKubernetesSection = DetailSection; @@ -120,6 +129,146 @@ export const buildKubernetesDetailSections = ( return compactSections(sections); }; +const countRow = ( + label: string, + value: number | undefined, + options: { noun?: string; tone?: (count: number) => DetailValueTone } = {}, +): DetailRow | null => { + if (typeof value !== 'number' || !Number.isFinite(value)) return null; + const text = options.noun + ? formatDetailCountValue(value, options.noun) + : formatDetailIntegerValue(value); + return makeRow(label, text, options.tone ? { tone: options.tone(value) } : {}); +}; + +const warnWhenPositive = (count: number): DetailValueTone => (count > 0 ? 'warning' : 'default'); +const dangerWhenPositive = (count: number): DetailValueTone => (count > 0 ? 'danger' : 'default'); + +// Kubernetes reports controller timestamps as RFC 3339 strings. The table row +// only has room for an age, so the expansion carries the absolute time first +// and the age after it: "Oct 2, 2026, 09:13 PM (1h ago)". The value wraps +// because a phone-width detail cell is too narrow for both on one line. +const timestampRow = (label: string, timestamp?: string | null): DetailRow | null => { + const raw = asString(timestamp); + if (!raw) return null; + const absolute = formatPlatformTableDateTimeValue(raw, { + emptyText: '', + dateTimeFormat: { year: 'numeric' }, + }); + if (!absolute) return makeRow(label, raw, { wrap: true }); + const relative = formatPlatformTableRelativeTimeValue(raw, { emptyText: '' }); + return makeRow(label, relative ? `${absolute} (${relative})` : absolute, { wrap: true }); +}; + +const elapsedSeconds = (start?: string | null, end?: string | null): number | undefined => { + const startedAt = Date.parse(asString(start) ?? ''); + const endedAt = Date.parse(asString(end) ?? ''); + if (!Number.isFinite(startedAt) || !Number.isFinite(endedAt) || endedAt < startedAt) { + return undefined; + } + return (endedAt - startedAt) / 1000; +}; + +type ControllerRows = { label: string; rows: Array }; + +// Each kind leads with what its narrow table row drops: the Detail column +// (service name, timestamps) below the large layout and Target on a phone. +const controllerRows = (resource: Resource): ControllerRows | null => { + const k = resource.kubernetes; + if (!k) return null; + switch (resource.type) { + case 'k8s-replicaset': + return { + label: 'ReplicaSet', + rows: [ + countRow('Target', k.desiredReplicas, { noun: 'pod' }), + countRow('Current', k.currentReplicas), + countRow('Ready', k.readyReplicas), + countRow('Available', k.availableReplicas), + countRow('Fully labeled', k.fullyLabeledReplicas), + countRow('Observed generation', k.observedGeneration), + ], + }; + case 'k8s-statefulset': + return { + label: 'StatefulSet', + rows: [ + makeRow('Service', k.serviceName), + countRow('Target', k.desiredReplicas, { noun: 'pod' }), + countRow('Current', k.currentReplicas), + countRow('Ready', k.readyReplicas), + countRow('Available', k.availableReplicas), + countRow('Updated', k.updatedReplicas), + ], + }; + case 'k8s-daemonset': + return { + label: 'DaemonSet', + rows: [ + countRow('Target', k.desiredNumberScheduled, { noun: 'node' }), + countRow('Current', k.currentNumberScheduled), + countRow('Ready', k.numberReady), + countRow('Available', k.numberAvailable), + countRow('Updated', k.updatedReplicas), + countRow('Unavailable', k.numberUnavailable, { tone: warnWhenPositive }), + countRow('Misscheduled', k.numberMisscheduled, { tone: warnWhenPositive }), + ], + }; + case 'k8s-job': + return { + label: 'Job', + rows: [ + timestampRow('Started', k.startTime), + timestampRow('Completed', k.completionTime), + makeRow( + 'Duration', + formatPlatformTableDurationValue(elapsedSeconds(k.startTime, k.completionTime), { + emptyText: '', + }), + ), + countRow('Target', k.desiredReplicas, { noun: 'completion' }), + countRow('Active', k.active), + countRow('Succeeded', k.succeeded), + countRow('Failed', k.failed, { tone: dangerWhenPositive }), + ], + }; + case 'k8s-cronjob': + return { + label: 'CronJob', + rows: [ + makeRow('Schedule', k.schedule, { valueClass: 'font-mono' }), + timestampRow('Last run', k.lastScheduleTime), + timestampRow('Last success', k.lastSuccessfulTime), + makeRow('Suspended', k.suspend === true ? 'Yes' : null, { tone: 'warning' }), + countRow('Active', k.active), + ], + }; + default: + return null; + } +}; + +// Workload-controller facts for the always-visible summary of the resource +// drawer, the same slot Docker containers use for their timestamps. The +// controllers table hides Scope and Detail below its large layout and Target +// on a phone, so without this a Job's completion time or a CronJob's last +// success is unreachable from a narrow row. +export const buildKubernetesControllerSection = (resource: Resource): DetailSection | null => { + const controller = controllerRows(resource); + if (!controller) return null; + const rows = compactRows([ + ...controller.rows, + makeRow('Namespace', resource.kubernetes?.namespace), + makeRow('Cluster', resource.kubernetes?.clusterName), + ]); + if (rows.length === 0) return null; + return { + label: controller.label, + rows, + testId: 'resource-kubernetes-controller-section', + }; +}; + export const buildKubernetesDetailsSummary = (resource: Resource): string | null => { const k = resource.kubernetes; if (!k) return null; diff --git a/frontend-modern/src/components/Settings/UpdateHistorySection.tsx b/frontend-modern/src/components/Settings/UpdateHistorySection.tsx index b8dd29d332..4fee194fd4 100644 --- a/frontend-modern/src/components/Settings/UpdateHistorySection.tsx +++ b/frontend-modern/src/components/Settings/UpdateHistorySection.tsx @@ -5,6 +5,7 @@ import { updateStore } from '@/stores/updates'; import { Button } from '@/components/shared/Button'; import { Dialog } from '@/components/shared/Dialog'; import { LoadingSpinner } from '@/components/shared/LoadingSpinner'; +import { getShippedDocUrl } from '@/utils/docsLinks'; const HISTORY_LIMIT = 20; @@ -45,7 +46,7 @@ const statusPresentation = (status: string): { label: string; className: string } }; -// A rollback restores the backup taken before this update was applied, so it +// A rollback uses the backup taken before this update was applied, so it // is only offered where that backup is still retained on disk (the backend // clears backup_path when retention prunes it) and the update actually landed. const canRollBack = (entry: UpdateHistoryEntry): boolean => @@ -89,7 +90,7 @@ export const UpdateHistorySection: Component = () => { Update History

- Updates applied through Pulse, with rollback to the backup taken before each one. + Updates applied through Pulse, with version rollback using a retained backup.

@@ -187,14 +188,30 @@ export const UpdateHistorySection: Component = () => {
-

- This restores the binary, configuration, and data from the backup taken before the - update to {formatVersionLabel(entry().version_to)} on{' '} - {formatTimestamp(entry().timestamp)}. +

+ Running now: Pulse {formatVersionLabel(updateStore.versionInfo()?.version || '')}.

- Settings and alert changes made since that backup will be reverted, and Pulse will - restart to complete the rollback. + This requests an installation rollback using the backup taken before the update to{' '} + {formatVersionLabel(entry().version_to)} on {formatTimestamp(entry().timestamp)}. + Pulse will restart if the rollback succeeds. +

+

+ Restore scope is not reported for this backup. A scope-aware server keeps active + runtime data in place and restores only the binary and independent installation + files. Older recovery can also replace install-local data and configuration, + reverting later changes. +

+

+ Do not assume this will undo settings or alerts. Full-state recovery requires + Pulse to be stopped and a consistent backup of every active store.{' '} + + Read the recovery instructions + + .

diff --git a/frontend-modern/src/components/Settings/__tests__/UpdateHistorySection.test.tsx b/frontend-modern/src/components/Settings/__tests__/UpdateHistorySection.test.tsx index 4f6ab77192..20a5b255e5 100644 --- a/frontend-modern/src/components/Settings/__tests__/UpdateHistorySection.test.tsx +++ b/frontend-modern/src/components/Settings/__tests__/UpdateHistorySection.test.tsx @@ -4,6 +4,7 @@ import type { UpdateHistoryEntry } from '@/api/updates'; const mockListUpdateHistory = vi.fn(); const mockStoreRollbackUpdate = vi.fn(); +let runningVersion: string | undefined = '6.0.5'; vi.mock('@/api/updates', () => ({ UpdatesAPI: { @@ -13,7 +14,7 @@ vi.mock('@/api/updates', () => ({ vi.mock('@/stores/updates', () => ({ updateStore: { - versionInfo: () => ({ version: '6.0.5' }), + versionInfo: () => (runningVersion ? { version: runningVersion } : null), rollbackUpdate: (...args: unknown[]) => mockStoreRollbackUpdate(...args), }, })); @@ -39,6 +40,7 @@ describe('UpdateHistorySection', () => { beforeEach(() => { mockListUpdateHistory.mockReset(); mockStoreRollbackUpdate.mockReset(); + runningVersion = '6.0.5'; }); afterEach(() => { @@ -102,6 +104,24 @@ describe('UpdateHistorySection', () => { fireEvent.click(await screen.findByRole('button', { name: 'Roll back' })); expect(await screen.findByText('Roll back to Pulse v6.0.4?')).toBeInTheDocument(); + expect(screen.getByText('Running now: Pulse v6.0.5.')).toBeInTheDocument(); + expect(screen.getByText(/Restore scope is not reported for this backup/)).toHaveTextContent( + 'A scope-aware server keeps active runtime data in place', + ); + expect(screen.getByText(/Older recovery can also/)).toHaveTextContent( + 'reverting later changes', + ); + expect(screen.getByText(/Do not assume this will undo settings or alerts/)).toHaveTextContent( + 'Full-state recovery requires Pulse to be stopped', + ); + expect(screen.getByRole('link', { name: 'Read the recovery instructions' })).toHaveAttribute( + 'href', + '/docs/AUTO_UPDATE#manual-rollback', + ); + expect( + screen.queryByText(/Settings and alert changes made since that backup will be reverted/), + ).not.toBeInTheDocument(); + expect(mockStoreRollbackUpdate).not.toHaveBeenCalled(); fireEvent.click(screen.getByRole('button', { name: 'Roll back to v6.0.4' })); @@ -116,4 +136,62 @@ describe('UpdateHistorySection', () => { expect(screen.queryByText('Roll back to Pulse v6.0.4?')).not.toBeInTheDocument(), ); }); + + it('distinguishes the running version from an older history entry without guessing restore scope', async () => { + runningVersion = 'v6.6.0-rc.1'; + mockListUpdateHistory.mockResolvedValue([{ ...baseEntry, notes: 'restore everything' }]); + render(() => ); + fireEvent.click(await screen.findByRole('button', { name: 'Roll back' })); + expect(screen.getByText('Running now: Pulse v6.6.0-rc.1.')).toBeInTheDocument(); + expect(screen.getByText(/This requests an installation rollback/)).toHaveTextContent( + 'before the update to v6.0.5', + ); + expect(screen.getByText(/Restore scope is not reported/)).toBeInTheDocument(); + expect(screen.queryByText('restore everything')).not.toBeInTheDocument(); + expect(screen.queryByText(baseEntry.backup_path!)).not.toBeInTheDocument(); + fireEvent.click(screen.getByRole('button', { name: 'Cancel' })); + expect(screen.queryByRole('dialog')).not.toBeInTheDocument(); + expect(mockStoreRollbackUpdate).not.toHaveBeenCalled(); + }); + + it('does not infer the running version from the selected update when version information is missing', async () => { + runningVersion = undefined; + mockListUpdateHistory.mockResolvedValue([baseEntry]); + render(() => ); + fireEvent.click(await screen.findByRole('button', { name: 'Roll back' })); + expect(screen.getByText('Running now: Pulse unknown.')).toBeInTheDocument(); + expect(screen.getByText(/Restore scope is not reported/)).toBeInTheDocument(); + }); + + it('retains the consent and refreshes history after a rejected rollback', async () => { + mockListUpdateHistory.mockResolvedValue([baseEntry]); + mockStoreRollbackUpdate.mockResolvedValue(false); + render(() => ); + fireEvent.click(await screen.findByRole('button', { name: 'Roll back' })); + fireEvent.click(screen.getByRole('button', { name: 'Roll back to v6.0.4' })); + await waitFor(() => expect(mockListUpdateHistory).toHaveBeenCalledTimes(2)); + expect(screen.getByRole('dialog')).toHaveTextContent('Restore scope is not reported'); + expect(screen.getByRole('button', { name: 'Roll back to v6.0.4' })).not.toBeDisabled(); + }); + + it('prevents duplicate requests and cancellation while the rollback request is pending', async () => { + mockListUpdateHistory.mockResolvedValue([baseEntry]); + let finish!: (accepted: boolean) => void; + mockStoreRollbackUpdate.mockReturnValue( + new Promise((resolve) => { + finish = resolve; + }), + ); + render(() => ); + fireEvent.click(await screen.findByRole('button', { name: 'Roll back' })); + fireEvent.click(screen.getByRole('button', { name: 'Roll back to v6.0.4' })); + expect(screen.getByRole('button', { name: 'Starting rollback...' })).toBeDisabled(); + expect(screen.getByRole('button', { name: 'Cancel' })).toBeDisabled(); + fireEvent.click(screen.getByRole('button', { name: 'Starting rollback...' })); + fireEvent.keyDown(document, { key: 'Escape' }); + expect(mockStoreRollbackUpdate).toHaveBeenCalledTimes(1); + expect(screen.getByRole('dialog')).toBeInTheDocument(); + finish(true); + await waitFor(() => expect(screen.queryByRole('dialog')).not.toBeInTheDocument()); + }); }); diff --git a/frontend-modern/src/components/Storage/DiskDetail.tsx b/frontend-modern/src/components/Storage/DiskDetail.tsx index 71c6cedb3a..3588836cc0 100644 --- a/frontend-modern/src/components/Storage/DiskDetail.tsx +++ b/frontend-modern/src/components/Storage/DiskDetail.tsx @@ -11,6 +11,10 @@ import { resolveHistoryRangeWithinLimit, } from '@/components/Storage/historyRangeAccess'; import { + DISK_DETAIL_HEADER_STACK_CLASS, + DISK_DETAIL_HEALTH_LABEL_CLASS, + DISK_DETAIL_HEALTH_ROW_CLASS, + DISK_DETAIL_HEALTH_SUMMARY_CLASS, DISK_DETAIL_HISTORY_RANGE_OPTIONS, DISK_DETAIL_LIVE_CHARTS, getDiskAttributeValueTextClass, @@ -55,6 +59,7 @@ export const DiskDetail: Component = (props) => { historyResourceId, attributeCards, historyCharts, + healthPresentation, metricResourceId, collectionMessages, liveIOAvailable, @@ -91,19 +96,32 @@ export const DiskDetail: Component = (props) => { return (
-
- {diskData().model || 'Unknown Disk'} - {diskData().devPath} - {diskData().node} - - {getPhysicalDiskSourceBadgePresentation(props.disk).label} - - - S/N: {diskData().serial} - +
+
+ + {diskData().model || 'Unknown Disk'} + + {diskData().devPath} + {diskData().node} + + {getPhysicalDiskSourceBadgePresentation(props.disk).label} + + + S/N: {diskData().serial} + +
+ {/* The verdict the row shows as a single word, with its reason in full. */} +
+ + {healthPresentation().label} + + + {healthPresentation().summary} + +
diff --git a/frontend-modern/src/components/Storage/DiskList.tsx b/frontend-modern/src/components/Storage/DiskList.tsx index 5715390316..fdff3b3d36 100644 --- a/frontend-modern/src/components/Storage/DiskList.tsx +++ b/frontend-modern/src/components/Storage/DiskList.tsx @@ -55,7 +55,6 @@ import { PHYSICAL_DISK_HEADER_SIZE_CLASS, PHYSICAL_DISK_HEADER_TEMP_CLASS, PHYSICAL_DISK_HEALTH_LABEL_CLASS, - PHYSICAL_DISK_HEALTH_SUMMARY_CLASS, PHYSICAL_DISK_HEALTH_WRAP_CLASS, PHYSICAL_DISK_DEVICE_TEXT_CLASS, PHYSICAL_DISK_LIFE_CLASS, @@ -72,8 +71,10 @@ import { PHYSICAL_DISK_TABLE_ROW_HOVER_CLASS, PHYSICAL_DISK_TABLE_ROW_SELECTED_CLASS, PHYSICAL_DISK_TABLE_ROW_STYLE, - getPhysicalDiskColumnWidthPercent, + getPhysicalDiskCellPaddingClass, + getPhysicalDiskColumnWidthStyle, getPhysicalDiskEmptyStatePresentation, + getPhysicalDiskHealthCompactLabel, getPhysicalDiskHealthStatus, getPhysicalDiskHealthSummary, getPhysicalDiskHostLabel, @@ -117,11 +118,18 @@ export const DiskList: Component = (props) => { baseClass: string, columnId: PhysicalDiskTableColumnId, visibleClass: 'table-column' | 'table-cell', - ) => - `${baseClass} ${isPhysicalDiskColumnVisible(layoutMode(), columnId) ? visibleClass : 'hidden'}`.trim(); - const columnStyle = (columnId: PhysicalDiskTableColumnId) => ({ - width: `${getPhysicalDiskColumnWidthPercent(layoutMode(), columnId)}%`, - }); + ) => { + const visibility = isPhysicalDiskColumnVisible(layoutMode(), columnId) + ? visibleClass + : 'hidden'; + // Rendered cells and headers shed the shared primitives' desktop padding + // on the phone layouts; `` elements carry no padding. + const padding = + visibleClass === 'table-cell' ? getPhysicalDiskCellPaddingClass(layoutMode()) : ''; + return `${baseClass} ${padding} ${visibility}`.replace(/\s+/g, ' ').trim(); + }; + const columnStyle = (columnId: PhysicalDiskTableColumnId) => + getPhysicalDiskColumnWidthStyle(layoutMode(), columnId); const { getDiskTemperatureThresholds } = useAlertsActivation(); const model = useDiskListModel({ disks: () => props.disks, @@ -327,6 +335,11 @@ export const DiskList: Component = (props) => { const status = createMemo(() => getPhysicalDiskHealthStatus(data())); const hostLabel = createMemo(() => getPhysicalDiskHostLabel(data(), disk)); const healthSummary = createMemo(() => getPhysicalDiskHealthSummary(status())); + const healthCompactLabel = createMemo(() => + getPhysicalDiskHealthCompactLabel(status().label), + ); + const healthLabelClass = () => + `${PHYSICAL_DISK_HEALTH_LABEL_CLASS} ${status().tone}`; const isSelected = () => model.selectedDisk()?.id === disk.id; const summarySeriesId = createMemo(() => resolvePhysicalDiskMetricResourceId(disk)); const isSummaryHighlighted = () => @@ -446,15 +459,30 @@ export const DiskList: Component = (props) => { data-storage-column="health" >
- - {status().label} - - + {/* The row shows the verdict word only; the reason is the + cell's title here and is spelled out in the disk drawer. + The phone projection of the table container shows the + shorter word; the tone stays on each span so the rendered + label is the styled element in either projection. */} + + {status().label} + + } + > - {healthSummary()} + {healthCompactLabel()} + + + {status().label}
diff --git a/frontend-modern/src/components/Storage/EnhancedStorageBar.tsx b/frontend-modern/src/components/Storage/EnhancedStorageBar.tsx index 4df116d888..c75475a3da 100644 --- a/frontend-modern/src/components/Storage/EnhancedStorageBar.tsx +++ b/frontend-modern/src/components/Storage/EnhancedStorageBar.tsx @@ -25,6 +25,7 @@ import { getStorageBarTooltipTitle, getStorageBarZfsHeadingLabel, } from '@/features/storageBackups/storageBarPresentation'; +import { PlatformResponsiveTableLabel } from '@/features/platformPage/sharedPlatformPage'; import { useTooltip } from '@/hooks/useTooltip'; import { ProgressBar } from '@/components/shared/ProgressBar'; import { TooltipPortal } from '@/components/shared/TooltipPortal'; @@ -42,13 +43,14 @@ interface EnhancedStorageBarProps { export function EnhancedStorageBar(props: EnhancedStorageBarProps) { const tip = useTooltip(); const alertsActivation = useAlertsActivation(); - const { usagePercent, barColor, label, tooltipRows, zfsSummary } = useEnhancedStorageBarModel({ - used: () => props.used, - total: () => props.total, - free: () => props.free, - zfsPool: () => props.zfsPool, - thresholds: () => alertsActivation.getMetricThresholds('storage', 'disk'), - }); + const { usagePercent, barColor, label, compactLabel, tooltipRows, zfsSummary } = + useEnhancedStorageBarModel({ + used: () => props.used, + total: () => props.total, + free: () => props.free, + zfsPool: () => props.zfsPool, + thresholds: () => alertsActivation.getMetricThresholds('storage', 'disk'), + }); return (
@@ -71,7 +73,12 @@ export function EnhancedStorageBar(props: EnhancedStorageBarProps) { } label={ - {label()} + + {/* The phone projection of the table container keeps the + percentage only; the used/total sublabel returns with the + wider container and always lives in the tooltip. */} + + } /> diff --git a/frontend-modern/src/components/Storage/StoragePoolRow.tsx b/frontend-modern/src/components/Storage/StoragePoolRow.tsx index 58e8e6d2a8..1d29643412 100644 --- a/frontend-modern/src/components/Storage/StoragePoolRow.tsx +++ b/frontend-modern/src/components/Storage/StoragePoolRow.tsx @@ -26,6 +26,7 @@ import { import type { StorageCapacityDeltaPresentation } from '@/features/storageBackups/storageCapacityDeltaPresentation'; import type { Resource } from '@/types/resource'; import { + getStoragePoolCellPaddingClass, isStoragePoolColumnVisible, type StoragePoolTableColumnId, type StoragePoolTableLayoutMode, @@ -62,7 +63,11 @@ interface StoragePoolRowProps { export const StoragePoolRow: Component = (props) => { const columnClass = (baseClass: string, columnId: StoragePoolTableColumnId) => - `${baseClass} ${isStoragePoolColumnVisible(props.layoutMode, columnId) ? 'table-cell' : 'hidden'}`.trim(); + `${baseClass} ${getStoragePoolCellPaddingClass(props.layoutMode)} ${ + isStoragePoolColumnVisible(props.layoutMode, columnId) ? 'table-cell' : 'hidden' + }` + .replace(/\s+/g, ' ') + .trim(); const row = createMemo(() => buildStoragePoolRowModel(props.record, props.growthDelta ?? null)); const detailControlsId = createMemo(() => buildSummaryDisclosureControlsId(props.summarySeriesId), diff --git a/frontend-modern/src/components/Storage/StoragePoolsTable.tsx b/frontend-modern/src/components/Storage/StoragePoolsTable.tsx index e05cf67137..2b81545e57 100644 --- a/frontend-modern/src/components/Storage/StoragePoolsTable.tsx +++ b/frontend-modern/src/components/Storage/StoragePoolsTable.tsx @@ -9,7 +9,7 @@ import { } from '@/components/shared/Table'; import { getPlatformTableHeadClassForKind } from '@/features/platformPage/sharedPlatformPage'; import { - getStoragePoolColumnWidthPercent, + getStoragePoolColumnWidthStyle, getStoragePoolTableColumns, getStoragePoolTableLayoutModeForContainer, getStorageEmptyStateMessage, @@ -106,9 +106,7 @@ export const StoragePoolsTable: Component = (props) => { {(column) => ( )} diff --git a/frontend-modern/src/components/Storage/__tests__/DiskDetail.test.tsx b/frontend-modern/src/components/Storage/__tests__/DiskDetail.test.tsx index cfa8a0f5d0..e1630ffa95 100644 --- a/frontend-modern/src/components/Storage/__tests__/DiskDetail.test.tsx +++ b/frontend-modern/src/components/Storage/__tests__/DiskDetail.test.tsx @@ -1,4 +1,4 @@ -import { fireEvent, render, screen } from '@solidjs/testing-library'; +import { fireEvent, render, screen, within } from '@solidjs/testing-library'; import { createSignal, type JSX } from 'solid-js'; import { describe, expect, it, vi } from 'vitest'; import { DiskDetail } from '@/components/Storage/DiskDetail'; @@ -135,6 +135,33 @@ describe('DiskDetail', () => { expect(screen.getByText('Reallocated Sectors')).toBeInTheDocument(); }); + it('shows the health verdict and its reason in the drawer header', () => { + const disk = buildDisk(); + disk.physicalDisk!.health = 'FAILED'; + + render(() => ); + + const health = screen.getByTestId('disk-detail-health'); + expect(within(health).getByText('Replace Now')).toHaveClass('text-red-700'); + expect(within(health).getByText('Disk health has degraded to a critical state.')).toHaveClass( + 'min-w-0', + 'max-w-full', + 'whitespace-normal', + 'wrap-break-word', + ); + }); + + it('keeps the drawer header to the verdict alone when the disk is healthy', () => { + const disk = buildDisk(); + disk.physicalDisk!.health = 'PASSED'; + + render(() => ); + + const health = screen.getByTestId('disk-detail-health'); + expect(within(health).getByText('Healthy')).toBeInTheDocument(); + expect(within(health).queryByText('No active disk-health issues.')).not.toBeInTheDocument(); + }); + it('shows an explicit overview fallback when no detail readings are available', () => { const disk = buildDisk(); delete disk.physicalDisk!.smart; diff --git a/frontend-modern/src/components/Storage/__tests__/DiskList.test.tsx b/frontend-modern/src/components/Storage/__tests__/DiskList.test.tsx index c8b0ad9432..ae75eb0693 100644 --- a/frontend-modern/src/components/Storage/__tests__/DiskList.test.tsx +++ b/frontend-modern/src/components/Storage/__tests__/DiskList.test.tsx @@ -134,12 +134,18 @@ describe('DiskList', () => { 'archive', 'Cache Pool', 'Archive Pool', - 'SMART failed.', '4%', '63°C', ]) { expect(within(row as HTMLElement).getByText(text)).toBeInTheDocument(); } + // The reason is not a second truncating string in the 32px row: it rides + // on the verdict as its title and is spelled out in the disk drawer. + expect(within(row as HTMLElement).queryByText('SMART failed.')).not.toBeInTheDocument(); + expect(within(row as HTMLElement).getByText('Replace Now')).toHaveAttribute( + 'title', + 'SMART failed.', + ); expect(within(row as HTMLElement).getByText('3.64 TB')).toBeInTheDocument(); expect(screen.queryByText('Healthy')).not.toBeInTheDocument(); expect(screen.queryByText('96%')).not.toBeInTheDocument(); @@ -275,7 +281,13 @@ describe('DiskList', () => { expect(screen.getByText('Parity')).toBeInTheDocument(); expect(screen.getByText('Tower Array')).toBeInTheDocument(); expect(screen.getByText('Needs Attention')).toBeInTheDocument(); - expect(screen.getByText('Pending sectors detected.')).toBeInTheDocument(); + // The reason stays one hover away on the verdict rather than as a second + // truncating string in the row; the drawer spells it out. + expect(screen.queryByText('Pending sectors detected.')).not.toBeInTheDocument(); + expect(screen.getByText('Needs Attention')).toHaveAttribute( + 'title', + 'Pending sectors detected.', + ); }); it('renders SSD life and falls back to the Proxmox usage string for Belongs', () => { diff --git a/frontend-modern/src/components/Storage/__tests__/Storage.test.tsx b/frontend-modern/src/components/Storage/__tests__/Storage.test.tsx index e77acd0a3d..7c917328be 100644 --- a/frontend-modern/src/components/Storage/__tests__/Storage.test.tsx +++ b/frontend-modern/src/components/Storage/__tests__/Storage.test.tsx @@ -1154,10 +1154,11 @@ describe('Storage', () => { render(() => ); + // zpool's upper-case state is presented titleized, like every other state. await waitFor(() => { - expect(screen.getAllByText('DEGRADED').length).toBeGreaterThan(0); + expect(screen.getAllByText('Degraded').length).toBeGreaterThan(0); }); - expect(screen.getAllByText('DEGRADED')[0]).toHaveAttribute( + expect(screen.getAllByText('Degraded')[0]).toHaveAttribute( 'title', '1 read, 2 checksum errors', ); diff --git a/frontend-modern/src/components/Storage/__tests__/useEnhancedStorageBarModel.test.ts b/frontend-modern/src/components/Storage/__tests__/useEnhancedStorageBarModel.test.ts index b3fc3721a0..18d67ca221 100644 --- a/frontend-modern/src/components/Storage/__tests__/useEnhancedStorageBarModel.test.ts +++ b/frontend-modern/src/components/Storage/__tests__/useEnhancedStorageBarModel.test.ts @@ -28,6 +28,7 @@ describe('useEnhancedStorageBarModel', () => { expect(result.usagePercent()).toBe(40); expect(result.label()).toBe('40% (400 B/1000 B)'); + expect(result.compactLabel()).toBe('40%'); expect(result.tooltipRows()).toEqual([ { label: 'Used', value: '400 B' }, { label: 'Free', value: '600 B' }, diff --git a/frontend-modern/src/components/Storage/useDiskDetailModel.ts b/frontend-modern/src/components/Storage/useDiskDetailModel.ts index 7ea36e2a61..2038b38433 100644 --- a/frontend-modern/src/components/Storage/useDiskDetailModel.ts +++ b/frontend-modern/src/components/Storage/useDiskDetailModel.ts @@ -7,6 +7,7 @@ import { } from '@/features/storageBackups/diskPresentation'; import { getDiskDetailAttributeCards, + getDiskDetailHealthPresentation, getDiskDetailHistoryCharts, } from '@/features/storageBackups/diskDetailPresentation'; import { useAlertsActivation } from '@/stores/alertsActivation'; @@ -29,6 +30,7 @@ export const useDiskDetailModel = (options: UseDiskDetailModelOptions) => { getDiskDetailAttributeCards(diskData(), getDiskTemperatureThresholds(diskData().type)), ); const historyCharts = createMemo(() => getDiskDetailHistoryCharts(diskData())); + const healthPresentation = createMemo(() => getDiskDetailHealthPresentation(diskData())); const metricResourceId = createMemo(() => historyResourceId()); const collectionMessages = createMemo(() => getPhysicalDiskCollectionMessages(diskData())); const liveIOAvailable = createMemo(() => { @@ -43,6 +45,7 @@ export const useDiskDetailModel = (options: UseDiskDetailModelOptions) => { historyResourceId, attributeCards, historyCharts, + healthPresentation, metricResourceId, collectionMessages, liveIOAvailable, diff --git a/frontend-modern/src/components/Storage/useEnhancedStorageBarModel.ts b/frontend-modern/src/components/Storage/useEnhancedStorageBarModel.ts index ddb6f141e7..06cf0aa3d7 100644 --- a/frontend-modern/src/components/Storage/useEnhancedStorageBarModel.ts +++ b/frontend-modern/src/components/Storage/useEnhancedStorageBarModel.ts @@ -3,6 +3,7 @@ import type { ZFSPool } from '@/types/api'; import { getMetricColorClass } from '@/utils/metricThresholds'; import type { MetricDisplayThresholds } from '@/utils/metricThresholds'; import { + getStorageBarCompactLabel, getStorageBarLabel, getStorageBarTooltipRows, getStorageBarUsagePercent, @@ -23,6 +24,7 @@ export const useEnhancedStorageBarModel = (options: UseEnhancedStorageBarModelOp getMetricColorClass(usagePercent(), 'disk', options.thresholds?.()), ); const label = createMemo(() => getStorageBarLabel(options.used(), options.total())); + const compactLabel = createMemo(() => getStorageBarCompactLabel(options.used(), options.total())); const tooltipRows = createMemo(() => getStorageBarTooltipRows(options.used(), options.free(), options.total()), ); @@ -32,6 +34,7 @@ export const useEnhancedStorageBarModel = (options: UseEnhancedStorageBarModelOp usagePercent, barColor, label, + compactLabel, tooltipRows, zfsSummary, }; diff --git a/frontend-modern/src/components/Workloads/DiskList.tsx b/frontend-modern/src/components/Workloads/DiskList.tsx index ae74147396..ab122415c2 100644 --- a/frontend-modern/src/components/Workloads/DiskList.tsx +++ b/frontend-modern/src/components/Workloads/DiskList.tsx @@ -15,6 +15,9 @@ export function DiskList(props: DiskListProps) { } >
+ +

{diskStatusTooltip()}

+
{(disk) => { return ( diff --git a/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx b/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx index c7d95a9fae..2d843f3a35 100644 --- a/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx +++ b/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx @@ -13,7 +13,10 @@ import { AvailabilityProbeStatusCards } from '@/components/Infrastructure/Availa import type { DiscoveryIdentifiedSummary } from '@/utils/discoveryPresentation'; import { formatBytes } from '@/utils/format'; import type { MetricDisplayThresholds } from '@/utils/metricThresholds'; -import { getWorkloadsGuestProtectionPresentation } from '@/utils/workloadGuestPresentation'; +import { + getWorkloadGuestDiskStatusMessage, + getWorkloadsGuestProtectionPresentation, +} from '@/utils/workloadGuestPresentation'; import { AvailabilityProbeSuggestionCard } from './AvailabilityProbeSuggestionCard'; import { buildWorkloadsDiskPresentation } from './diskListModel'; @@ -119,6 +122,10 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) { const diskRows = (props.guest.disks ?? []).map((disk, index) => buildWorkloadsDiskPresentation(disk, index, props.diskThresholds), ); + const diskStatusMessage = + isGuestDrawerVM(props.guest) && props.guest.diskStatusReason + ? getWorkloadGuestDiskStatusMessage(props.guest.diskStatusReason) + : null; return compactDetailSections([ discovery @@ -187,7 +194,7 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) { .map((row) => makeDetailRow(row.label, row.value)), ), }, - props.hasFilesystemDetails && diskRows.length > 0 + (props.hasFilesystemDetails && diskRows.length > 0) || diskStatusMessage ? { label: 'Filesystems', rows: compactDetailRows([ @@ -212,11 +219,7 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) { }, ), ), - makeDetailRow( - 'Status', - isGuestDrawerVM(props.guest) ? props.guest.diskStatusReason : null, - { wrap: true }, - ), + makeDetailRow('Status', diskStatusMessage, { layout: 'stacked', wrap: true }), ]), } : null, diff --git a/frontend-modern/src/components/Workloads/GuestRow.tsx b/frontend-modern/src/components/Workloads/GuestRow.tsx index 16df105c4e..2622930599 100644 --- a/frontend-modern/src/components/Workloads/GuestRow.tsx +++ b/frontend-modern/src/components/Workloads/GuestRow.tsx @@ -553,7 +553,13 @@ export function GuestRow(props: GuestRowProps) { {/* Disk */} - + {renderMetricSparkline( 'disk', @@ -585,6 +591,11 @@ export function GuestRow(props: GuestRowProps) { aggregateDisk={props.guest.disk} anomaly={diskAnomaly()} thresholds={diskThresholds()} + statusMessage={ + isVM(props.guest) && props.guest.diskStatusReason + ? getDiskStatusTooltip() + : undefined + } /> } > diff --git a/frontend-modern/src/components/Workloads/StackedDiskBar.tsx b/frontend-modern/src/components/Workloads/StackedDiskBar.tsx index d50486f26b..90913e7afa 100644 --- a/frontend-modern/src/components/Workloads/StackedDiskBar.tsx +++ b/frontend-modern/src/components/Workloads/StackedDiskBar.tsx @@ -189,44 +189,57 @@ export function StackedDiskBar(props: StackedDiskBarProps) { } > - {/* Vertical micro-bars: one per disk, fill height = utilization, color = threshold */} + {/* Vertical micro-bars: one per disk, fill height = utilization, color = threshold. + The fullest disk's percentage sits beside them so the cell still reads + as a number at a glance; every bar keeps equal weight. */}
-
- - {(bar) => ( -
+
+ + + {presentation().verticalBarsLabel} + + +
+ + {(bar) => (
- + +
-
- )} - + )} + +
@@ -242,6 +255,9 @@ export function StackedDiskBar(props: StackedDiskBarProps) {
{presentation().tooltipTitle}
+ +

{props.statusMessage}

+
{(item, idx) => (
= {}): Disk { return { @@ -46,7 +48,10 @@ describe('DiskList', () => { 'agent-not-running', 'Guest agent not running. Install and start qemu-guest-agent in the VM.', ], - ['agent-timeout', 'Guest agent timeout. Agent may need to be restarted.'], + [ + 'agent-timeout', + 'Guest request timed out. Completion is uncertain. Do not restart the guest agent during a backup.', + ], [ 'permission-denied', 'Permission denied. Check that your Pulse user/token has VM.Monitor permission (PVE 8) or VM.GuestAgent.Audit permission (PVE 9).', @@ -59,6 +64,11 @@ describe('DiskList', () => { ], ['agent-error', 'Error communicating with guest agent.'], ['no-data', 'No disk data available from Proxmox API.'], + ['vm-stopped', 'Guest filesystem stats unavailable while the VM is stopped.'], + [ + 'no-status', + 'Guest filesystem stats unavailable because Pulse could not read the VM status from Proxmox.', + ], [ 'prev-no-filesystems', 'Using last known disk stats. No filesystems found. VM may be booting or using a Live ISO.', @@ -78,6 +88,30 @@ describe('DiskList', () => { }); describe('rendering disks', () => { + it.each(guestDiskDeferrals)( + 'keeps retained values and explains %s until fresh data resumes', + (reason, message) => { + const [disks, setDisks] = createSignal([makeDisk()]); + const [status, setStatus] = createSignal(`prev-${reason}`); + render(() => ); + expect(screen.getByText(`Using last known disk stats. ${message}`)).toBeInTheDocument(); + expect(screen.getByText('50%')).toBeInTheDocument(); + setDisks([makeDisk({ used: 80530636800, usage: 75 })]); + setStatus(undefined); + expect(screen.queryByText(/Using last known/)).not.toBeInTheDocument(); + expect(screen.getByText('75%')).toBeInTheDocument(); + }, + ); + + it.each(guestDiskDeferrals)( + 'explains %s without claiming previous values when none exist', + (reason, message) => { + render(() => ); + expect(screen.getByText('-')).toHaveAttribute('title', message); + expect(screen.queryByText(/Using last known/)).not.toBeInTheDocument(); + }, + ); + it('renders a single disk with correct label, usage, and type', () => { const disk = makeDisk({ mountpoint: '/data', type: 'xfs' }); render(() => ); diff --git a/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.filesystems.test.tsx b/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.filesystems.test.tsx index 69a89b4926..820a69d0c4 100644 --- a/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.filesystems.test.tsx +++ b/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.filesystems.test.tsx @@ -1,7 +1,8 @@ import { cleanup, render, screen } from '@solidjs/testing-library'; -import type { ComponentProps } from 'solid-js'; +import { createSignal, type ComponentProps } from 'solid-js'; import { afterEach, describe, expect, it } from 'vitest'; import { GuestDrawerOverview } from '../GuestDrawerOverview'; +import { guestDiskDeferrals } from '../__fixtures__/guestDiskDeferrals'; const paths = ['/mnt/Plex/Media/Animation/Films', '/mnt/Plex/Media/Animation/Series']; const props: ComponentProps = { @@ -55,6 +56,47 @@ const props: ComponentProps = { describe('GuestDrawerOverview filesystem labels', () => { afterEach(cleanup); + it.each(guestDiskDeferrals)( + 'explains %s in the existing drawer even before any disk sample', + (reason, message) => { + render(() => ( + + )); + expect(screen.getByText('Filesystems')).toBeInTheDocument(); + expect(screen.getByText(message)).toBeInTheDocument(); + expect(screen.queryByRole('progressbar')).not.toBeInTheDocument(); + expect(screen.queryByText(/Using last known/)).not.toBeInTheDocument(); + }, + ); + + it('retains disk rows with readable status, then withdraws the notice on fresh resumption', () => { + const [guest, setGuest] = createSignal({ + ...props.guest, + type: 'qemu' as const, + diskStatusReason: 'prev-vm-locked', + }); + render(() => ); + expect( + screen.getByText(/^Using last known disk stats\. Guest reads paused/).closest('td'), + ).toHaveAttribute('colspan', '2'); + expect(screen.queryByText('prev-vm-locked')).not.toBeInTheDocument(); + for (const path of paths) expect(screen.getByText(path)).toBeInTheDocument(); + setGuest({ + ...guest(), + diskStatusReason: '', + disks: guest().disks!.map((disk) => ({ ...disk, used: (disk.total ?? 0) * 0.75, usage: 75 })), + }); + expect(screen.queryByText(/Using last known/)).not.toBeInTheDocument(); + expect(screen.queryByText('Status')).not.toBeInTheDocument(); + expect( + screen.getByRole('progressbar', { name: `Filesystem ${paths[0]} utilization` }), + ).toHaveAttribute('aria-valuenow', '75'); + }); + it('shows linked-agent RAID arrays beside the guest storage evidence', () => { render(() => ( ({ })); vi.mock('../StackedDiskBar', () => ({ - StackedDiskBar: () =>
, + StackedDiskBar: (props: { statusMessage?: string }) => ( +
+ ), })); vi.mock('../StackedMemoryBar', () => ({ @@ -668,6 +671,39 @@ describe('GuestRow', () => { }); describe('lock label', () => { + it.each(guestDiskDeferrals)( + 'annotates retained row and disk tooltip for %s', + (reason, message) => { + const { container } = renderGuestRow({ + guest: makeGuest({ diskStatusReason: `prev-${reason}` }), + }); + const expected = `Using last known disk stats. ${message}`; + expect(container.querySelector('[data-workload-col="disk"]')).toHaveAttribute( + 'title', + expected, + ); + expect(screen.getByTestId('disk-bar')).toHaveAttribute('data-status-message', expected); + }, + ); + + it('clears retained status for a same-VM fresh update in sparkline mode without remounting', () => { + const [guest, setGuest] = createSignal( + makeGuest({ diskStatusReason: 'prev-agent-cooldown' }), + ); + const { container } = render(() => ( + + + + +
+ )); + const cell = container.querySelector('[data-workload-col="disk"]'); + expect(cell).toHaveAttribute('title', expect.stringContaining('Using last known')); + setGuest(makeGuest({ diskStatusReason: undefined, disk: makeDisk({ usage: 75 }) })); + expect(container.querySelector('[data-workload-col="disk"]')).toBe(cell); + expect(cell).not.toHaveAttribute('title'); + }); + it('shows lock label when guest is locked', () => { renderGuestRow({ guest: makeGuest({ lock: 'migrate' }) }); expect(screen.getByText(/Lock:.*migrate/)).toBeTruthy(); @@ -1308,23 +1344,26 @@ describe('GUEST_COLUMNS', () => { }); it('derives mobile overrides from the canonical guest column model', () => { + // The phone name cell carries the toggle, status dot, and backup badge, so + // it takes a wider anchor than the shared 30% and the metric chips share + // the rest. expect(getGuestColumnStyle('name', true)).toEqual({ - width: '30%', - 'max-width': '30%', + width: '38%', + 'max-width': '38%', }); expect(getGuestColumnStyle('cpu', true)).toEqual({ - width: '11.3235%', - 'max-width': '11.3235%', + width: '10.0294%', + 'max-width': '10.0294%', }); expect(getGuestColumnStyle('availability', true)).toEqual({ - width: '7.2059%', - 'max-width': '7.2059%', + width: '6.3824%', + 'max-width': '6.3824%', }); expect(getGuestColumnStyle('type', true)).toEqual({ - width: '9.2647%', - 'max-width': '9.2647%', + width: '8.2059%', + 'max-width': '8.2059%', }); - expect(getGuestColumnWidthStyle('name', true)).toEqual({ width: '30%' }); + expect(getGuestColumnWidthStyle('name', true)).toEqual({ width: '38%' }); expect(getGuestColumnWidthStyle('diskIo', true)).toEqual({ width: '170px' }); }); diff --git a/frontend-modern/src/components/Workloads/__tests__/StackedDiskBar.test.tsx b/frontend-modern/src/components/Workloads/__tests__/StackedDiskBar.test.tsx index e550f069a0..9cd200c640 100644 --- a/frontend-modern/src/components/Workloads/__tests__/StackedDiskBar.test.tsx +++ b/frontend-modern/src/components/Workloads/__tests__/StackedDiskBar.test.tsx @@ -343,6 +343,17 @@ describe('StackedDiskBar', () => { // ── Tooltip ────────────────────────────────────────────────────────────── describe('tooltip', () => { + it('includes caller-owned freshness context beside unchanged disk evidence', async () => { + const message = 'Using last known disk stats. Guest reads paused while a backup is running.'; + const { container } = render(() => ( + + )); + await fireEvent.mouseEnter(getBarTrigger(container)); + expect(screen.getByText(message)).toBeInTheDocument(); + expect(screen.getByText('/')).toBeInTheDocument(); + expect(screen.getByText('Disk Usage')).toBeInTheDocument(); + }); + it('shows tooltip on mouse enter with disk details', async () => { const disk = makeDisk({ used: 53687091200, @@ -587,4 +598,36 @@ describe('StackedDiskBar', () => { expect(bar?.getAttribute('fill')).toContain('239, 68, 68'); }); }); + + describe('vertical-bars mode', () => { + it('labels the micro-bars with the fullest disk so the cell reads as a number', () => { + const { container } = render(() => ( + + )); + + const label = container.querySelector('[data-stacked-disk-max-label]'); + expect(label).toHaveTextContent('52%'); + expect(label).toHaveAttribute('title', 'Highest usage: /data 52%'); + // Every disk keeps its own bar with equal weight; the label is derived. + expect(container.querySelectorAll('[data-stacked-disk-fill="vertical"]')).toHaveLength(2); + }); + + it('omits the label when no disk has measured usage', () => { + const { container } = render(() => ( + + )); + + expect(container.querySelector('[data-stacked-disk-max-label]')).toBeNull(); + }); + }); }); diff --git a/frontend-modern/src/components/Workloads/__tests__/WorkloadsSurface.performance.contract.test.tsx b/frontend-modern/src/components/Workloads/__tests__/WorkloadsSurface.performance.contract.test.tsx index 961906663b..fc3877ae3a 100644 --- a/frontend-modern/src/components/Workloads/__tests__/WorkloadsSurface.performance.contract.test.tsx +++ b/frontend-modern/src/components/Workloads/__tests__/WorkloadsSurface.performance.contract.test.tsx @@ -1661,7 +1661,11 @@ describe('Workloads performance contract', () => { ); expect(filesystemSectionSource).toContain("layout: 'stacked'"); expect(filesystemSectionSource).toContain('wrap: true'); - expect(guestDrawerOverviewSource.match(/layout: 'stacked'/g)).toHaveLength(1); + // Both filesystem paths and their longer freshness explanation need + // the card's full width. Do not couple that layout to a global count. + expect(guestDrawerOverviewSource).toContain( + "makeDetailRow('Status', diskStatusMessage, { layout: 'stacked', wrap: true })", + ); expect(guestDrawerOverviewSource).toContain('TechnicalDetailsSection'); expect(nodeDrawerOverviewSource).toContain('TechnicalDetailsSection'); diff --git a/frontend-modern/src/components/Workloads/__tests__/stackedDiskBarModel.branchcov0712b.test.ts b/frontend-modern/src/components/Workloads/__tests__/stackedDiskBarModel.branchcov0712b.test.ts index 054068a04c..b56964dfe4 100644 --- a/frontend-modern/src/components/Workloads/__tests__/stackedDiskBarModel.branchcov0712b.test.ts +++ b/frontend-modern/src/components/Workloads/__tests__/stackedDiskBarModel.branchcov0712b.test.ts @@ -372,6 +372,10 @@ describe('stackedDiskBarModel (branch coverage 2)', () => { title: '/x: 40% (40.0 B/100 B)', }); expect(p.verticalBars[1].color).toBe(CRITICAL); // 120% -> critical + // The fullest disk's usage labels the cell so a multi-disk host still + // reads as a number without a hover. + expect(p.verticalBarsLabel).toBe('120%'); + expect(p.verticalBarsLabelTitle).toBe('Highest usage: /y 120%'); expect(p.tooltipContent).toHaveLength(3); expect(p.tooltipContent[2]).toMatchObject({ label: '/z', percent: '—', used: '?' }); }); diff --git a/frontend-modern/src/components/Workloads/guestRowModel.tsx b/frontend-modern/src/components/Workloads/guestRowModel.tsx index 40b81bf47e..0c7c67ea17 100644 --- a/frontend-modern/src/components/Workloads/guestRowModel.tsx +++ b/frontend-modern/src/components/Workloads/guestRowModel.tsx @@ -469,6 +469,13 @@ const percentageColumn = (width: string): GuestColumnWidthOverride => ({ maxWidth: width, }); +// The phone name cell carries the expansion toggle, status dot, and backup +// badge before the name, so the shared 30% identity anchor left a 390px phone +// 66px for the name and clipped "artifact-cache-310" to "artifact-ca…". The +// four metric chips beside it need far less than an even split, so the name +// takes the same wider phone anchor the Proxmox host table uses. +const GUEST_PHONE_IDENTITY_WIDTH_PERCENT = 38; + // Responsive weights are normalized against the currently visible column set. // That keeps each workload view mode full-width without assuming one fixed set. const GUEST_COLUMN_RESPONSIVE_WEIGHTS: Record< @@ -548,9 +555,11 @@ const getResponsiveColumnOverride = ( activeIds, layoutMode === 'narrow' ? { columnId: 'name', widthPercent: PLATFORM_TABLE_NARROW_IDENTITY_WIDTH_PERCENT } - : layoutMode === 'phone' || layoutMode === 'mobile' - ? { columnId: 'name', widthPercent: PLATFORM_TABLE_PHONE_IDENTITY_WIDTH_PERCENT } - : undefined, + : layoutMode === 'phone' + ? { columnId: 'name', widthPercent: GUEST_PHONE_IDENTITY_WIDTH_PERCENT } + : layoutMode === 'mobile' + ? { columnId: 'name', widthPercent: PLATFORM_TABLE_PHONE_IDENTITY_WIDTH_PERCENT } + : undefined, ); return typeof style.width === 'string' ? percentageColumn(style.width) : undefined; }; diff --git a/frontend-modern/src/components/Workloads/stackedDiskBarModel.ts b/frontend-modern/src/components/Workloads/stackedDiskBarModel.ts index 7b2adc3f65..3dbf3531b6 100644 --- a/frontend-modern/src/components/Workloads/stackedDiskBarModel.ts +++ b/frontend-modern/src/components/Workloads/stackedDiskBarModel.ts @@ -18,6 +18,7 @@ export interface StackedDiskBarProps { showDiskCount?: boolean; anomaly?: AnomalyReport | null; thresholds?: MetricDisplayThresholds | null; + statusMessage?: string; } export interface StackedDiskSegment { @@ -87,6 +88,10 @@ export interface StackedDiskBarPresentation { tooltipTitle: string; useStackedSegments: boolean; verticalBars: StackedDiskVerticalBar[]; + // The fullest disk's usage, shown beside the micro-bars so a multi-disk + // host answers "how close is this host to running out?" without a hover. + verticalBarsLabel: string; + verticalBarsLabelTitle: string; verticalBarsMode: boolean; } @@ -359,6 +364,8 @@ export function buildStackedDiskBarPresentation( }; }) : []; + const verticalBarsLabel = verticalBarsMode && maxInfo ? formatPercent(maxInfo.percent) : ''; + const verticalBarsLabelTitle = verticalBarsMode && maxInfo ? maxLabelFull : ''; const tooltipContent = buildTooltipContent(allDisks, { aggregateDisk: props.aggregateDisk, aggregateMode, @@ -402,6 +409,8 @@ export function buildStackedDiskBarPresentation( tooltipTitle: allDisks.length > 1 ? 'Disk Breakdown' : 'Disk Usage', useStackedSegments, verticalBars, + verticalBarsLabel, + verticalBarsLabelTitle, verticalBarsMode, }; } diff --git a/frontend-modern/src/features/alerts/AlertDeliveryLogCard.test.tsx b/frontend-modern/src/features/alerts/AlertDeliveryLogCard.test.tsx index 312340f152..4fb6b60f9a 100644 --- a/frontend-modern/src/features/alerts/AlertDeliveryLogCard.test.tsx +++ b/frontend-modern/src/features/alerts/AlertDeliveryLogCard.test.tsx @@ -273,7 +273,7 @@ describe('AlertDeliveryLogCard', () => { expect(screen.queryByRole('list')).not.toBeInTheDocument(); }); - it('hides held rows during an outage and restores their current state on recovery', () => { + it('hides held rows when their own read fails and restores them on recovery', () => { const [unavailable, setUnavailable] = createSignal(false); const [heldEvents, setHeldEvents] = createSignal([ { @@ -290,7 +290,8 @@ describe('AlertDeliveryLogCard', () => { render(() => ( { expect(screen.getByText('Deferred')).toBeInTheDocument(); setUnavailable(true); - expect(screen.getByRole('alert')).toHaveTextContent(/could not read the delivery log/); + expect(screen.getByRole('alert')).toHaveTextContent( + /could not read held or deferred notifications/, + ); expect(screen.queryByRole('list')).not.toBeInTheDocument(); expect(screen.queryByText('nas-quiet (usage)')).not.toBeInTheDocument(); expect(screen.queryByText(/No alert deliveries were attempted/)).not.toBeInTheDocument(); @@ -332,4 +335,106 @@ describe('AlertDeliveryLogCard', () => { expect(screen.getByRole('alert')).toHaveTextContent(/could not read the delivery log/); expect(screen.queryByText(/No alert deliveries were attempted/)).not.toBeInTheDocument(); }); + it('keeps independently readable held evidence when delivery attempts are unavailable', () => { + render(() => ( + + )); + expect(screen.getByRole('alert')).toHaveTextContent(/could not read the delivery log/); + expect(screen.getByText('held-known')).toBeInTheDocument(); + expect(screen.getByText('Quiet hours')).toBeInTheDocument(); + expect(screen.queryByText('Delivered')).not.toBeInTheDocument(); + expect(screen.queryByText(/No alert deliveries were attempted/)).not.toBeInTheDocument(); + }); + + it('reports missing held evidence without hiding readable attempts', () => { + render(() => ( + + )); + expect(screen.getByRole('alert')).toHaveTextContent( + /could not read held or deferred notifications/, + ); + expect(screen.getByText('Delivered')).toBeInTheDocument(); + expect(screen.getByText('Ops Discord')).toBeInTheDocument(); + expect(screen.queryByText('stale-held')).not.toBeInTheDocument(); + }); + + it('does not claim an empty activity window while held evidence is pending', () => { + render(() => ( + + )); + expect(screen.getByRole('status')).toHaveTextContent( + 'Loading held and deferred notifications...', + ); + expect(screen.queryByText(/No alert deliveries were attempted/)).not.toBeInTheDocument(); + // A slow held read must not prevent a new delivery-attempt refresh. + expect(screen.getByRole('button', { name: 'Refresh delivery status' })).toBeEnabled(); + }); + + it('does not claim an empty activity window when held evidence is unavailable', () => { + render(() => ( + + )); + expect(screen.getByRole('alert')).toHaveTextContent( + /could not read held or deferred notifications/, + ); + expect(screen.queryByText(/No alert deliveries were attempted/)).not.toBeInTheDocument(); + }); + + it('labels the initial attempt read as loading rather than empty', () => { + render(() => ( + + )); + expect(screen.getByRole('status')).toHaveTextContent('Loading delivery attempts...'); + expect(screen.queryByText(/No alert deliveries were attempted/)).not.toBeInTheDocument(); + }); }); diff --git a/frontend-modern/src/features/alerts/AlertDeliveryLogCard.tsx b/frontend-modern/src/features/alerts/AlertDeliveryLogCard.tsx index 1b7bfb411b..cb1be99e27 100644 --- a/frontend-modern/src/features/alerts/AlertDeliveryLogCard.tsx +++ b/frontend-modern/src/features/alerts/AlertDeliveryLogCard.tsx @@ -15,6 +15,9 @@ import { getAlertDestinationsDeliveryLogEmpty, getAlertDestinationsDeliveryLogTitle, getAlertDestinationsDeliveryLogUnavailable, + getAlertDestinationsDeliveryLogLoading, + getAlertDestinationsHeldEventsLoading, + getAlertDestinationsHeldEventsUnavailable, getAlertDestinationsDeliveryRefreshLabel, } from '@/utils/alertDestinationsPresentation'; import { formatRelativeTime } from '@/utils/format'; @@ -28,6 +31,8 @@ interface AlertDeliveryLogCardProps { onRefresh: () => void; webhooks: Webhook[]; heldEvents?: AlertEvent[]; + heldEventsUnavailable?: boolean; + refreshingHeldEvents?: boolean; } type DeliveryLogRow = @@ -99,10 +104,18 @@ export function AlertDeliveryLogCard(props: AlertDeliveryLogCardProps) { return rest.length > 0 ? `${first} +${rest.length} more` : first; }; - const entries = () => props.log?.entries ?? []; + const entries = () => (props.unavailable ? [] : (props.log?.entries ?? [])); const completedRetentionDays = () => props.log?.completedRetentionDays ?? 7; const deadLetterRetentionDays = () => props.log?.deadLetterRetentionDays ?? 30; - const rows = () => mergeDeliveryLogRows(entries(), props.heldEvents ?? []); + const rows = () => + mergeDeliveryLogRows(entries(), props.heldEventsUnavailable ? [] : (props.heldEvents ?? [])); + const attemptsLoading = () => props.refreshing || (!props.log && !props.unavailable); + const confirmedEmpty = () => + props.log !== null && + !props.unavailable && + !attemptsLoading() && + !props.heldEventsUnavailable && + !props.refreshingHeldEvents; const absoluteTimestamp = (value: string): string => { const parsed = new Date(value); @@ -149,100 +162,113 @@ export function AlertDeliveryLogCard(props: AlertDeliveryLogCardProps) {
+ + + + + + + +

+ {getAlertDestinationsDeliveryLogLoading()} +

+
+ +

+ {getAlertDestinationsHeldEventsLoading()} +

+
0} fallback={ - - } - > - 0} - fallback={ +

{getAlertDestinationsDeliveryLogEmpty()}

- } - > -
    - - {(row) => - row.kind === 'attempt' ? ( -
  • -
    - - {getAlertDeliveryLogOutcomeLabel(row.entry.outcome)} - - - {destinationLabel(row.entry)} - - - {alertSummary(row.entry)} - - -
    - + } + > +
      + + {(row) => + row.kind === 'attempt' ? ( +
    • +
      + -

      - - {(failureClass) => ( - - {getAlertDeliveryLogFailureClassLabel(failureClass())} - {row.entry.errorMessage ? ': ' : ''} - - )} - - {row.entry.errorMessage} -

      - -
    • - ) : ( -
    • -
      - - {row.event.type === 'notification_deferred' ? 'Deferred' : 'Held'} - - - {heldResourceLabel(row.event)} - - - {describeAlertEventReason(row.event.reason)} - - -
      -
    • - ) - } -
      -
    -
    + {getAlertDeliveryLogOutcomeLabel(row.entry.outcome)} + + + {destinationLabel(row.entry)} + + + {alertSummary(row.entry)} + + +
+ +

+ + {(failureClass) => ( + + {getAlertDeliveryLogFailureClassLabel(failureClass())} + {row.entry.errorMessage ? ': ' : ''} + + )} + + {row.entry.errorMessage} +

+
+ + ) : ( +
  • +
    + + {row.event.type === 'notification_deferred' ? 'Deferred' : 'Held'} + + + {heldResourceLabel(row.event)} + + + {describeAlertEventReason(row.event.reason)} + + +
    +
  • + ) + } + +
    diff --git a/frontend-modern/src/features/alerts/__tests__/DestinationsTab.recoveryediting.test.tsx b/frontend-modern/src/features/alerts/__tests__/DestinationsTab.recoveryediting.test.tsx index 2c07536cad..2d3589247e 100644 --- a/frontend-modern/src/features/alerts/__tests__/DestinationsTab.recoveryediting.test.tsx +++ b/frontend-modern/src/features/alerts/__tests__/DestinationsTab.recoveryediting.test.tsx @@ -259,7 +259,7 @@ describe('DestinationsTab recovery while editing SMTP', () => { ).toBeInTheDocument(); expect( await screen.findByText( - 'Pulse could not read the delivery log, so recent delivery activity cannot be shown.', + 'Pulse could not read the delivery log. Refresh to check delivery attempts.', ), ).toBeInTheDocument(); expect(screen.queryByText('SMTP fixture rejected')).not.toBeInTheDocument(); diff --git a/frontend-modern/src/features/alerts/__tests__/useAlertDestinationsTabState.test.tsx b/frontend-modern/src/features/alerts/__tests__/useAlertDestinationsTabState.test.tsx index b01d8e43b0..2515c6f393 100644 --- a/frontend-modern/src/features/alerts/__tests__/useAlertDestinationsTabState.test.tsx +++ b/frontend-modern/src/features/alerts/__tests__/useAlertDestinationsTabState.test.tsx @@ -156,6 +156,17 @@ describe('useAlertDestinationsTabState', () => { await Promise.resolve(); expect(result.deliveryLog()).toEqual(current); expect(result.deliveryLogUnavailable()).toBe(false); + expect(result.heldEventsUnavailable()).toBe(false); + expect(result.refreshingHeldEvents()).toBe(false); + vi.mocked(AlertsAPI.getEvents).mockRejectedValueOnce(new Error('held events denied')); + vi.mocked(NotificationsAPI.getDeliveryLog).mockResolvedValueOnce(current); + await result.loadDeliveryLog(); + expect(result.heldEvents()).toEqual([]); + expect(result.heldEventsUnavailable()).toBe(true); + expect(result.refreshingHeldEvents()).toBe(false); + expect(result.deliveryLogUnavailable()).toBe(false); + expect(result.deliveryLog()).toEqual(current); + expect(result.refreshingDeliveryLog()).toBe(false); } finally { cleanup(); diff --git a/frontend-modern/src/features/alerts/__tests__/useNotificationDeliveryLog.test.tsx b/frontend-modern/src/features/alerts/__tests__/useNotificationDeliveryLog.test.tsx index e4398987af..ad497217a0 100644 --- a/frontend-modern/src/features/alerts/__tests__/useNotificationDeliveryLog.test.tsx +++ b/frontend-modern/src/features/alerts/__tests__/useNotificationDeliveryLog.test.tsx @@ -107,6 +107,8 @@ describe('useNotificationDeliveryLog', () => { expect(state.deliveryLog()).toEqual(emptyLog); expect(state.refreshingDeliveryLog()).toBe(false); expect(state.heldEvents()).toEqual([]); + expect(state.refreshingHeldEvents()).toBe(true); + expect(state.heldEventsUnavailable()).toBe(false); } finally { held.resolve([]); dispose(); @@ -132,6 +134,8 @@ describe('useNotificationDeliveryLog', () => { expect(state.heldEvents()).toHaveLength(1); await state.loadDeliveryLog(); expect(state.heldEvents()).toEqual([]); + expect(state.heldEventsUnavailable()).toBe(true); + expect(state.refreshingHeldEvents()).toBe(false); expect(state.deliveryLog()).toEqual(emptyLog); expect(state.deliveryLogUnavailable()).toBe(false); expect(state.refreshingDeliveryLog()).toBe(false); @@ -222,6 +226,8 @@ describe('useNotificationDeliveryLog', () => { else older.reject(new Error('stale held read')); await Promise.resolve(); expect(state.heldEvents()).toEqual(current); + expect(state.heldEventsUnavailable()).toBe(false); + expect(state.refreshingHeldEvents()).toBe(false); } finally { dispose(); } @@ -264,4 +270,56 @@ describe('useNotificationDeliveryLog', () => { expect(AlertsAPI.getEvents).toHaveBeenCalledTimes(1); }), ); + it('retains the held-read warning during retry until that read succeeds', () => + createRoot(async (dispose) => { + const recovery = deferred>>(); + vi.mocked(NotificationsAPI.getDeliveryLog).mockResolvedValue(emptyLog); + vi.mocked(AlertsAPI.getEvents) + .mockRejectedValueOnce(new Error('permission denied')) + .mockReturnValueOnce(recovery.promise); + const state = useNotificationDeliveryLog(); + try { + await state.loadDeliveryLog(); + expect(state.heldEventsUnavailable()).toBe(true); + await state.loadDeliveryLog(); + expect(state.heldEventsUnavailable()).toBe(true); + expect(state.refreshingHeldEvents()).toBe(true); + expect(state.refreshingDeliveryLog()).toBe(false); + recovery.resolve([]); + await recovery.promise; + expect(state.heldEventsUnavailable()).toBe(false); + expect(state.refreshingHeldEvents()).toBe(false); + } finally { + dispose(); + } + })); + + it.each(['success', 'failure'])( + 'keeps the newest held read pending after an older %s', + (outcome) => + createRoot(async (dispose) => { + const older = deferred>>(); + const newer = deferred>>(); + vi.mocked(NotificationsAPI.getDeliveryLog).mockResolvedValue(emptyLog); + vi.mocked(AlertsAPI.getEvents) + .mockReturnValueOnce(older.promise) + .mockReturnValueOnce(newer.promise); + const state = useNotificationDeliveryLog(); + try { + await state.loadDeliveryLog(); + await state.loadDeliveryLog(); + if (outcome === 'success') older.resolve([]); + else older.reject(new Error('stale denied request')); + await Promise.resolve(); + expect(state.refreshingHeldEvents()).toBe(true); + expect(state.heldEventsUnavailable()).toBe(false); + newer.resolve([]); + await newer.promise; + expect(state.refreshingHeldEvents()).toBe(false); + expect(state.heldEventsUnavailable()).toBe(false); + } finally { + dispose(); + } + }), + ); }); diff --git a/frontend-modern/src/features/alerts/tabs/DestinationsTab.tsx b/frontend-modern/src/features/alerts/tabs/DestinationsTab.tsx index 3f2e0f7718..d6462cefe9 100644 --- a/frontend-modern/src/features/alerts/tabs/DestinationsTab.tsx +++ b/frontend-modern/src/features/alerts/tabs/DestinationsTab.tsx @@ -102,6 +102,8 @@ export function DestinationsTab(props: DestinationsTabProps) { onRefresh={() => void state.loadDeliveryLog()} webhooks={state.webhooks()} heldEvents={state.heldEvents()} + heldEventsUnavailable={state.heldEventsUnavailable()} + refreshingHeldEvents={state.refreshingHeldEvents()} /> diff --git a/frontend-modern/src/features/alerts/useAlertDestinationsTabState.ts b/frontend-modern/src/features/alerts/useAlertDestinationsTabState.ts index 26d815d52c..86c90f37f1 100644 --- a/frontend-modern/src/features/alerts/useAlertDestinationsTabState.ts +++ b/frontend-modern/src/features/alerts/useAlertDestinationsTabState.ts @@ -53,6 +53,8 @@ export function useAlertDestinationsTabState(props: AlertDestinationsTabStatePro deliveryLogUnavailable, refreshingDeliveryLog, heldEvents, + heldEventsUnavailable, + refreshingHeldEvents, loadDeliveryLog, } = useNotificationDeliveryLog(); const webhookState = useAlertWebhookDestinationsState({ @@ -173,6 +175,8 @@ export function useAlertDestinationsTabState(props: AlertDestinationsTabStatePro deliveryLogUnavailable, deliveryNeedsAttention, heldEvents, + heldEventsUnavailable, + refreshingHeldEvents, dismissTerminalFailures, dismissingTerminalFailures, handleRetry, diff --git a/frontend-modern/src/features/alerts/useNotificationDeliveryLog.ts b/frontend-modern/src/features/alerts/useNotificationDeliveryLog.ts index 1ddceb830f..e36c6036df 100644 --- a/frontend-modern/src/features/alerts/useNotificationDeliveryLog.ts +++ b/frontend-modern/src/features/alerts/useNotificationDeliveryLog.ts @@ -19,12 +19,15 @@ const DELIVERY_LOG_LIMIT = 200; // Held events are the third half of that story: notifications that were // deliberately not attempted, with the mechanism that held them. They come // from the alert event log and degrade independently — a failed events read -// hides the held rows without marking delivery attempts unavailable. +// withdraws the held rows and reports that gap without hiding readable attempts. export function useNotificationDeliveryLog() { const [deliveryLog, setDeliveryLog] = createSignal(null); const [deliveryLogUnavailable, setDeliveryLogUnavailable] = createSignal(false); const [refreshingDeliveryLog, setRefreshingDeliveryLog] = createSignal(false); const [heldEvents, setHeldEvents] = createSignal([]); + const [heldEventsUnavailable, setHeldEventsUnavailable] = createSignal(false); + // Before the first read, an empty array means unknown, not no held alerts. + const [refreshingHeldEvents, setRefreshingHeldEvents] = createSignal(true); // Both reads belong to the same refresh, but held events never block the log. let latestRequest = 0; @@ -35,6 +38,7 @@ export function useNotificationDeliveryLog() { const ownsRequest = (request: number) => !disposed && request === latestRequest; const loadHeldEvents = async (request: number) => { + setRefreshingHeldEvents(true); try { const since = new Date( Date.now() - HELD_EVENT_WINDOW_DAYS * 24 * 60 * 60 * 1000, @@ -44,11 +48,16 @@ export function useNotificationDeliveryLog() { since, limit: HELD_EVENT_LIMIT, }); - if (ownsRequest(request)) setHeldEvents(events); + if (!ownsRequest(request)) return; + setHeldEvents(events); + setHeldEventsUnavailable(false); } catch (error) { if (!ownsRequest(request)) return; logger.error('Failed to load held alert notification events', error); setHeldEvents([]); + setHeldEventsUnavailable(true); + } finally { + if (ownsRequest(request)) setRefreshingHeldEvents(false); } }; @@ -82,6 +91,8 @@ export function useNotificationDeliveryLog() { deliveryLogUnavailable, refreshingDeliveryLog, heldEvents, + heldEventsUnavailable, + refreshingHeldEvents, loadDeliveryLog, }; } diff --git a/frontend-modern/src/features/docker/DockerImagesTable.tsx b/frontend-modern/src/features/docker/DockerImagesTable.tsx index 88345be440..744bc18b71 100644 --- a/frontend-modern/src/features/docker/DockerImagesTable.tsx +++ b/frontend-modern/src/features/docker/DockerImagesTable.tsx @@ -152,7 +152,7 @@ export const DockerImagesTable: Component< kind="text" sort={sort} sortKey="host" - class="platform-table-mobile-w-15 md:w-[18%]" + class="platform-table-phone-hidden md:w-[18%]" > Host @@ -160,7 +160,7 @@ export const DockerImagesTable: Component< kind="text" sort={sort} sortKey="usedBy" - class="platform-table-mobile-w-15 md:w-[24%]" + class="platform-table-phone-hidden md:w-[24%]" > Used by @@ -168,7 +168,7 @@ export const DockerImagesTable: Component< kind="numeric-value" sort={sort} sortKey="size" - class="platform-table-mobile-w-15 md:w-[12%]" + class="platform-table-mobile-w-20 md:w-[12%]" > Size @@ -176,7 +176,7 @@ export const DockerImagesTable: Component< kind="badge" sort={sort} sortKey="update" - class="platform-table-mobile-w-15 md:w-[16%]" + class="platform-table-mobile-w-25 md:w-[16%]" > @@ -211,13 +211,15 @@ export const DockerImagesTable: Component< /> } /> + {/* Host and consumers fit the row expansion on a phone; the + row keeps the image, its size and the update verdict. */} {dockerHostName(resource)} c.toUpperCase()) + : 'Updating'; const title = [state, message, completedAt].filter(Boolean).join(' | ') || label; return { label, title }; }; @@ -256,7 +262,7 @@ export const DockerServicesTable: Component<{ sortKey="running" class="platform-table-mobile-w-10 w-[10%] md:w-[8%]" > - Running + - + {update().label} diff --git a/frontend-modern/src/features/docker/DockerTasksTable.tsx b/frontend-modern/src/features/docker/DockerTasksTable.tsx index e8fabdeaa0..e45f4a182b 100644 --- a/frontend-modern/src/features/docker/DockerTasksTable.tsx +++ b/frontend-modern/src/features/docker/DockerTasksTable.tsx @@ -12,6 +12,7 @@ import { createPlatformTableSortState, getPlatformTableCellClassForKind, getPlatformTableDateTimeSortValue, + PlatformTableRelativeTimeValue, PlatformTableShell, type PlatformTableSortValue, withPlatformStatusCounts, @@ -255,11 +256,10 @@ export const DockerTasksTable: Component = (props) => { diff --git a/frontend-modern/src/features/docker/__tests__/DockerNativeTables.test.tsx b/frontend-modern/src/features/docker/__tests__/DockerNativeTables.test.tsx index 1695ae6c76..15bb7087d2 100644 --- a/frontend-modern/src/features/docker/__tests__/DockerNativeTables.test.tsx +++ b/frontend-modern/src/features/docker/__tests__/DockerNativeTables.test.tsx @@ -1505,6 +1505,8 @@ describe('Docker native tables', () => { )); expect(screen.getByText('Update')).toBeInTheDocument(); + expect(screen.getByText('Run')).toHaveClass('platform-table-label-compact'); + expect(screen.getByText('Running')).toHaveClass('platform-table-label-full'); expect(screen.getByText('Stack')).toBeInTheDocument(); expect(screen.getByText('shop')).toBeInTheDocument(); expect(screen.getByText('checkout-api')).toBeInTheDocument(); @@ -1512,7 +1514,15 @@ describe('Docker native tables', () => { expect(screen.getByText('replicated')).toBeInTheDocument(); expect(screen.getByText('4')).toBeInTheDocument(); expect(screen.getByText('2')).toBeInTheDocument(); - expect(screen.getByText('rollback_started')).toBeInTheDocument(); + // The snake_case rollout token reads as words; the raw token stays in the title. + expect(screen.getByText('Rollback started')).toBeInTheDocument(); + expect(screen.getByText('Rollback started')).toHaveClass( + 'whitespace-normal', + 'wrap-break-word', + ); + expect( + screen.getByText('Rollback started').closest('[title]')?.getAttribute('title'), + ).toContain('rollback_started'); expect(screen.getByText('18080:8080/tcp')).toBeInTheDocument(); expect(screen.getByText('manager-1')).toBeInTheDocument(); expect(document.querySelector('[data-docker-service-row="service-1"]')).not.toBeNull(); @@ -1651,7 +1661,10 @@ describe('Docker native tables', () => { expect(screen.getByText('web.2')).toBeInTheDocument(); expect(screen.getByText('web')).toBeInTheDocument(); expect(screen.getByText('running 2 minutes')).toBeInTheDocument(); - expect(screen.getByText('2026-05-24T13:05:00Z')).toBeInTheDocument(); + // The raw RFC3339 stamp moves to the title; the cell shows an age. + const started = screen.getByTitle('2026-05-24T13:05:00Z'); + expect(started).toBeInTheDocument(); + expect(started.textContent).toMatch(/ago$/); }); it('renders Docker Swarm secret API metadata without secret data', () => { diff --git a/frontend-modern/src/features/docker/__tests__/dockerContainerTableModel.test.ts b/frontend-modern/src/features/docker/__tests__/dockerContainerTableModel.test.ts index f8a8fc36f4..41f10a0869 100644 --- a/frontend-modern/src/features/docker/__tests__/dockerContainerTableModel.test.ts +++ b/frontend-modern/src/features/docker/__tests__/dockerContainerTableModel.test.ts @@ -30,7 +30,9 @@ describe('dockerContainerTableModel', () => { const columns = getDockerContainerVisibleColumnsForLayout('phone', true, true, true); const ids = columns.map((column) => column.id); - expect(ids).toEqual(['container', 'state', 'cpu', 'memory', 'restarts', 'updates']); + // Restarts waits for mobile width: a phone row could not fit its header + // beside the Update control, and the count stays in the row expansion. + expect(ids).toEqual(['container', 'state', 'cpu', 'memory', 'updates']); expect(getDockerContainerColumnWidthStyle('container', 'phone', ids)).toEqual({ width: '30%', }); diff --git a/frontend-modern/src/features/docker/dockerContainerTableModel.ts b/frontend-modern/src/features/docker/dockerContainerTableModel.ts index b573f3aba2..2961092c89 100644 --- a/frontend-modern/src/features/docker/dockerContainerTableModel.ts +++ b/frontend-modern/src/features/docker/dockerContainerTableModel.ts @@ -70,7 +70,9 @@ const DOCKER_CONTAINER_COLUMN_MIN_LAYOUT: Record< state: 'narrow', cpu: 'narrow', memory: 'narrow', - restarts: 'phone', + // A phone row cannot fit the Restarts header beside the Update control; + // the count stays in the row expansion there and returns at mobile width. + restarts: 'mobile', updates: 'narrow', actions: 'mobile', host: 'tablet', @@ -126,11 +128,10 @@ const DOCKER_CONTAINER_RESPONSIVE_WIDTHS: Record< }, phone: { container: 32, - state: 13, - cpu: 15, - memory: 17, - restarts: 10, - updates: 13, + state: 14, + cpu: 14, + memory: 14, + updates: 16, }, mobile: { container: 30, @@ -151,17 +152,20 @@ const DOCKER_CONTAINER_RESPONSIVE_WIDTHS: Record< updates: 9, actions: 6, }, + // Compact is what a 1280-1536px laptop window gets. Host, Engine and the + // Update control are short values that were clipped ("docker 2…") while the + // percentage-only CPU and memory bars had room to spare. compact: { container: 18, - host: 10, - runtime: 8, - image: 18, + host: 12, + runtime: 9.5, + image: 16.5, state: 7, - cpu: 11, - memory: 12, - restarts: 9, + cpu: 9.5, + memory: 10, + restarts: 8.5, ports: 12, - updates: 9, + updates: 10, actions: 8, }, }; diff --git a/frontend-modern/src/features/kubernetes/KubernetesControllersTable.tsx b/frontend-modern/src/features/kubernetes/KubernetesControllersTable.tsx index 189041e54d..47f82717ce 100644 --- a/frontend-modern/src/features/kubernetes/KubernetesControllersTable.tsx +++ b/frontend-modern/src/features/kubernetes/KubernetesControllersTable.tsx @@ -16,6 +16,7 @@ import { formatPlatformTableDateTimeValue, formatPlatformTableRelativeTimeValue, formatPlatformTableTextValue, + getPlatformResourceCountNoun, getPlatformTableCellClassForKind, PlatformTableShell, type PlatformTableSortValue, @@ -43,15 +44,20 @@ const controllerName = (resource: Resource): string => const controllerKind = (resource: Resource): string => resource.kubernetes?.resourceKind || getResourceTypeLabel(resource.type) || resource.type; +const countedTarget = (count: number | undefined, noun: string): string => { + const value = count ?? 0; + return `${value} ${getPlatformResourceCountNoun(noun, value)}`; +}; + const targetValue = (resource: Resource): string => { switch (resource.type) { case 'k8s-replicaset': case 'k8s-statefulset': - return `${resource.kubernetes?.desiredReplicas ?? 0} pods`; + return countedTarget(resource.kubernetes?.desiredReplicas, 'pods'); case 'k8s-daemonset': - return `${resource.kubernetes?.desiredNumberScheduled ?? 0} nodes`; + return countedTarget(resource.kubernetes?.desiredNumberScheduled, 'nodes'); case 'k8s-job': - return `${resource.kubernetes?.desiredReplicas ?? 0} completions`; + return countedTarget(resource.kubernetes?.desiredReplicas, 'completions'); case 'k8s-cronjob': return formatPlatformTableTextValue(resource.kubernetes?.schedule); default: @@ -317,7 +323,7 @@ export const KubernetesControllersTable: Component<{ kind="text" sort={sort} sortKey="kind" - class="platform-table-mobile-w-15 md:w-[8%]" + class="platform-table-mobile-w-25 md:w-[8%]" > Kind @@ -329,11 +335,16 @@ export const KubernetesControllersTable: Component<{ > Scope + {/* The phone row keeps controller, kind, ready and issues. A + 15% kind track clipped "DaemonSet" and "StatefulSet" and the + target track clipped "1 completions", so target moves to the + row expansion, which also carries the schedule and job + timestamps the narrower layouts drop. */} Target @@ -365,7 +376,7 @@ export const KubernetesControllersTable: Component<{ kind="text" sort={sort} sortKey="exceptions" - class="platform-table-mobile-w-20 md:w-[11%]" + class="platform-table-mobile-w-30 md:w-[11%]" > @@ -431,7 +442,7 @@ export const KubernetesControllersTable: Component<{ {target()} diff --git a/frontend-modern/src/features/kubernetes/__tests__/KubernetesControllersTable.test.tsx b/frontend-modern/src/features/kubernetes/__tests__/KubernetesControllersTable.test.tsx index cf201f66f5..1823b37427 100644 --- a/frontend-modern/src/features/kubernetes/__tests__/KubernetesControllersTable.test.tsx +++ b/frontend-modern/src/features/kubernetes/__tests__/KubernetesControllersTable.test.tsx @@ -1,6 +1,16 @@ -import { cleanup, render, screen } from '@solidjs/testing-library'; +import { cleanup, fireEvent, render, screen, within } from '@solidjs/testing-library'; import { afterEach, describe, expect, it, vi } from 'vitest'; +vi.mock('@/api/resources', () => ({ + ResourceAPI: { + getFacetBundle: vi.fn().mockResolvedValue({ + capabilities: [], + relationships: [], + recentChanges: [], + }), + }, +})); + import { formatPlatformTableDateTimeValue } from '@/features/platformPage/sharedPlatformPage'; import type { Resource } from '@/types/resource'; import { KubernetesControllersTable } from '../KubernetesControllersTable'; @@ -160,6 +170,117 @@ describe('KubernetesControllersTable', () => { expect(screen.queryByText(/2026-05-24T1[23]/)).not.toBeInTheDocument(); }); + it('keeps controller, kind, ready and issues on the phone row and moves target to the expansion', () => { + render(() => ( + } + emptyTitle="No controllers" + emptyDescription="No controllers" + showToolbar={false} + /> + )); + + // "DaemonSet" and "StatefulSet" clipped in a 15% kind track at 390px, and + // "1 completions" clipped in the target track, so the phone row widens kind + // and issues and demotes target to the row expansion. + expect(screen.getByText('Kind').closest('th')).toHaveClass('platform-table-mobile-w-25'); + expect(screen.getByText('Target').closest('th')).toHaveClass('platform-table-phone-hidden'); + expect(screen.getByText('6 nodes').closest('td')).toHaveClass('platform-table-phone-hidden'); + expect(screen.getByText('Exceptions').closest('th')).toHaveClass('platform-table-mobile-w-30'); + expect(screen.getByText('Controller').closest('th')).toHaveClass('platform-table-mobile-w-30'); + expect(screen.getByText('Ready/Done').closest('th')).toHaveClass('platform-table-mobile-w-15'); + expect(screen.getByText('Kind').closest('th')).not.toHaveClass('platform-table-phone-hidden'); + }); + + it('shows the absolute Job and CronJob times in the row expansion', () => { + vi.useFakeTimers(); + vi.setSystemTime(new Date('2026-05-24T13:31:00Z')); + + render(() => ( + } + emptyTitle="No controllers" + emptyDescription="No controllers" + showToolbar={false} + /> + )); + + // Tapping the row opens the expansion; the Detail column is hidden below a + // large container, so this is where a phone or half-width pane reads the + // completion time. + fireEvent.click(screen.getByText('nightly-import')); + const job = screen.getByTestId('resource-kubernetes-controller-section'); + expect(within(job).getByText('Started')).toBeInTheDocument(); + expect( + within(job).getByText(`${absoluteTime('2026-05-24T12:55:00Z')} (36m ago)`), + ).toBeInTheDocument(); + expect(within(job).getByText('Completed')).toBeInTheDocument(); + expect( + within(job).getByText(`${absoluteTime('2026-05-24T13:00:00Z')} (31m ago)`), + ).toBeInTheDocument(); + expect(within(job).getByText('Duration')).toBeInTheDocument(); + expect(within(job).getByText('5m')).toBeInTheDocument(); + expect(within(job).getByText('Target')).toBeInTheDocument(); + expect(within(job).getByText('10 completions')).toBeInTheDocument(); + expect(within(job).getByText('Failed').closest('tr')).toHaveTextContent('2'); + + fireEvent.click(screen.getByText('billing-rollup')); + const cron = screen.getByTestId('resource-kubernetes-controller-section'); + expect(within(cron).getByText('Schedule')).toBeInTheDocument(); + expect(within(cron).getByText('*/5 * * * *')).toBeInTheDocument(); + expect(within(cron).getByText('Last run')).toBeInTheDocument(); + expect( + within(cron).getByText(`${absoluteTime('2026-05-24T13:25:00Z')} (6m ago)`), + ).toBeInTheDocument(); + expect(within(cron).getByText('Last success')).toBeInTheDocument(); + expect( + within(cron).getByText(`${absoluteTime('2026-05-24T12:55:00Z')} (36m ago)`), + ).toBeInTheDocument(); + expect(within(cron).getByText('Namespace').closest('tr')).toHaveTextContent('batch'); + }); + it('falls back to start and last-run times when a Job or CronJob has not completed', () => { vi.useFakeTimers(); vi.setSystemTime(new Date('2026-05-24T13:31:00Z')); @@ -198,6 +319,9 @@ describe('KubernetesControllersTable', () => { /> )); + // A single desired completion reads as "1 completion", matching the + // expansion's Target row. + expect(screen.getByText('1 completion')).toBeInTheDocument(); expect(screen.getByText('Started 1m ago')).toHaveAttribute( 'title', `Started: ${absoluteTime('2026-05-24T13:29:30Z')}`, diff --git a/frontend-modern/src/features/platformPage/__tests__/platformOverviewLayout.guardrails.test.ts b/frontend-modern/src/features/platformPage/__tests__/platformOverviewLayout.guardrails.test.ts index b811259023..8a63862653 100644 --- a/frontend-modern/src/features/platformPage/__tests__/platformOverviewLayout.guardrails.test.ts +++ b/frontend-modern/src/features/platformPage/__tests__/platformOverviewLayout.guardrails.test.ts @@ -322,7 +322,7 @@ describe('platform overview layout guardrails', () => { expect(source).toContain('platform-table-mobile-w-30'); } expect(dockerImagesTableSource).toMatch( - /sortKey="size"[\s\S]{0,120}?class="platform-table-mobile-w-15 md:w-\[12%\]"/, + /sortKey="size"[\s\S]{0,120}?class="platform-table-mobile-w-20 md:w-\[12%\]"/, ); expect(dockerServicesTableSource).toMatch( /sortKey="mode"[\s\S]{0,120}?class="platform-table-phone-hidden md:w-\[8%\]"/, diff --git a/frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx b/frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx index 0ed2264d9a..ec407b7b8c 100644 --- a/frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx +++ b/frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx @@ -422,14 +422,10 @@ export function ProxmoxBackupServersTable(props: { } header={ <> - + - + @@ -449,20 +445,16 @@ export function ProxmoxBackupServersTable(props: { - + - + Used @@ -597,12 +589,14 @@ export function ProxmoxBackupServersTable(props: { > {(datastore) => (
    - + + + diff --git a/frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx b/frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx index ef84f0d194..f90874b0d5 100644 --- a/frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx +++ b/frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx @@ -43,6 +43,7 @@ import { import { buildProxmoxBackupRecoveryModel, coverageRowMatchesSearch, + isBackupArtifact, recoverableArtifactMatchesSearch, type RecoverableArtifact, type WorkloadCoverageRow, @@ -347,17 +348,20 @@ export const ProxmoxBackupsTable: Component<{ return recoveryModel().coverageRows.flatMap((row) => { const artifacts = row.artifacts.filter(locationMatches); if (artifacts.length === 0) return []; + // Same rule as the model: running or failed artifacts stay listed but + // never become a "latest" pointer, and snapshots never become the backup. + const completed = artifacts.filter((artifact) => !artifact.running && !artifact.failed); return [ { ...row, artifacts, - latestRecovery: newestArtifact(artifacts), - latestPBS: newestArtifact(artifacts.filter((artifact) => artifact.sourceKind === 'pbs')), + latestBackup: newestArtifact(completed.filter(isBackupArtifact)), + latestPBS: newestArtifact(completed.filter((artifact) => artifact.sourceKind === 'pbs')), latestArchive: newestArtifact( - artifacts.filter((artifact) => artifact.sourceKind === 'archive'), + completed.filter((artifact) => artifact.sourceKind === 'archive'), ), latestSnapshot: newestArtifact( - artifacts.filter((artifact) => artifact.sourceKind === 'snapshot'), + completed.filter((artifact) => artifact.sourceKind === 'snapshot'), ), pbsCount: artifacts.filter((artifact) => artifact.sourceKind === 'pbs').length, archiveCount: artifacts.filter((artifact) => artifact.sourceKind === 'archive').length, @@ -385,7 +389,7 @@ export const ProxmoxBackupsTable: Component<{ case 'workload': return cmpString(a.workload.label, b.workload.label, direction); case 'latest': - return cmpNumber(a.latestRecovery?.createdMs, b.latestRecovery?.createdMs, direction); + return cmpNumber(a.latestBackup?.createdMs, b.latestBackup?.createdMs, direction); case 'pbs': return cmpNumber(a.latestPBS?.createdMs, b.latestPBS?.createdMs, direction); case 'archive': diff --git a/frontend-modern/src/features/proxmox/ProxmoxCoverageTable.tsx b/frontend-modern/src/features/proxmox/ProxmoxCoverageTable.tsx index dcafa1e9ae..ce0c4e3d86 100644 --- a/frontend-modern/src/features/proxmox/ProxmoxCoverageTable.tsx +++ b/frontend-modern/src/features/proxmox/ProxmoxCoverageTable.tsx @@ -50,6 +50,10 @@ import { useProxmoxBackupTableWindowing } from './useProxmoxBackupTableWindowing // "None" instead of restating the source ("No PVE backup file"). The long form // truncated at most widths and stays available as the hover title. const COVERAGE_EMPTY_LABEL = 'None'; +const COVERAGE_NO_BACKUP_TITLE = 'No PBS snapshot or PVE backup file'; +// Matches the posture engine's own wording for a snapshot-only subject. +const COVERAGE_SNAPSHOT_ONLY_TITLE = + 'No PBS snapshot or PVE backup file. Guest snapshots alone do not prove independent recovery.'; const coveragePostureVariant = ( posture: WorkloadCoverageRow['posture'], @@ -240,7 +244,13 @@ export function ProxmoxCoverageTable(props: { headClass={getPlatformTableHeadClassForKind('text')} /> + {COVERAGE_EMPTY_LABEL} } > - {(artifact) => } + {(artifact) => ( + + )} @@ -429,7 +451,12 @@ export function ProxmoxCoverageTable(props: { } > - {(artifact) => } + {(artifact) => ( + + )} @@ -703,7 +730,7 @@ export function ProxmoxCoverageTable(props: { > {artifact.detail || '—'} diff --git a/frontend-modern/src/features/proxmox/ProxmoxRecoverableTable.tsx b/frontend-modern/src/features/proxmox/ProxmoxRecoverableTable.tsx index 367d630696..f5a514a569 100644 --- a/frontend-modern/src/features/proxmox/ProxmoxRecoverableTable.tsx +++ b/frontend-modern/src/features/proxmox/ProxmoxRecoverableTable.tsx @@ -123,6 +123,9 @@ export function ProxmoxRecoverableTable(props: { artifact.workload.name || artifact.workload.label, artifact.workload.typeLabel, artifact.workload.vmid, + // The phone projection has no location column, so the row's + // hover title carries the repository instead. + columnVisible('location') ? undefined : artifact.location, ] .filter(Boolean) .join(' · ')} @@ -163,7 +166,7 @@ export function ProxmoxRecoverableTable(props: { - + @@ -189,7 +192,10 @@ export function ProxmoxRecoverableTable(props: { - + {artifact.detail || '—'} diff --git a/frontend-modern/src/features/proxmox/__tests__/ProxmoxCoverageTable.test.tsx b/frontend-modern/src/features/proxmox/__tests__/ProxmoxCoverageTable.test.tsx index 50adeb3dd8..b9b7e75eaf 100644 --- a/frontend-modern/src/features/proxmox/__tests__/ProxmoxCoverageTable.test.tsx +++ b/frontend-modern/src/features/proxmox/__tests__/ProxmoxCoverageTable.test.tsx @@ -80,7 +80,7 @@ describe('ProxmoxCoverageTable column visibility', () => { expect(headers).toContain('Target ID'); expect(headers).toContain('Node'); expect(headers).toContain('Posture▲'); - expect(headers).toContain('Restore'); + expect(headers).toContain('Last backup'); expect(headers).toContain('PBS snapshot'); expect(headers).toContain('Guest snapshot'); expect(headers).not.toContain('PVE file'); @@ -239,7 +239,7 @@ describe('ProxmoxCoverageTable column visibility', () => { // cannot truncate to "No PVE backup f…". const empty = (title: string) => document.querySelector(`td span[title="${title}"]`); for (const title of [ - 'No restore point', + 'No PBS snapshot or PVE backup file', 'No PBS snapshot', 'No PVE backup file', 'No guest snapshot', diff --git a/frontend-modern/src/features/proxmox/__tests__/ProxmoxRecoverableTable.test.tsx b/frontend-modern/src/features/proxmox/__tests__/ProxmoxRecoverableTable.test.tsx index 0dea167612..a8c0248ff5 100644 --- a/frontend-modern/src/features/proxmox/__tests__/ProxmoxRecoverableTable.test.tsx +++ b/frontend-modern/src/features/proxmox/__tests__/ProxmoxRecoverableTable.test.tsx @@ -55,12 +55,13 @@ describe('ProxmoxRecoverableTable responsive columns', () => { expect([...document.querySelectorAll('thead th')].map((th) => th.textContent?.trim())).toEqual([ 'Workload', 'Via', - 'Loc', 'Age▼', 'State', ]); expect(document.body.textContent).toContain('VM 100'); - expect(document.body.textContent).toContain('main / pve1'); + // The phone projection has no location column; the row title carries it. + expect(document.body.textContent).not.toContain('main / pve1'); + expect(document.querySelector('td [title*="main / pve1"]')).not.toBeNull(); expect(document.body.textContent).not.toContain('5 PBS files'); }); diff --git a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupRecoveryModel.branchcov.test.ts b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupRecoveryModel.branchcov.test.ts index 8ae3acda04..6ab7884a78 100644 --- a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupRecoveryModel.branchcov.test.ts +++ b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupRecoveryModel.branchcov.test.ts @@ -522,7 +522,7 @@ describe('buildProxmoxBackupRecoveryModel summary, sort, and empty edges', () => expect(model.coverageSummary.attention).toBe(0); }); - it('sorts rows by postureRank asc, then latestRecovery createdMs desc', () => { + it('sorts rows by postureRank asc, then latestBackup createdMs desc', () => { const postures = new Map([ ['vm-830', posture('vm-830', 'unprotected')], ['vm-840', posture('vm-840', 'protected')], diff --git a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupRecoveryModel.test.ts b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupRecoveryModel.test.ts index a7e222ac29..43fc7e5a41 100644 --- a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupRecoveryModel.test.ts +++ b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupRecoveryModel.test.ts @@ -162,6 +162,35 @@ describe('proxmoxBackupRecoveryModel', () => { ); }); + it('shows the archive format as its detail and keeps the volid on hover and in search', () => { + const model = buildProxmoxBackupRecoveryModel({ + workloads: [workload({})], + pbsBackups: [], + archives: [archive({ format: 'tar.zst' })], + snapshots: [], + tasks: [], + nowMs: Date.parse('2026-05-26T08:00:00Z'), + }); + + const [artifact] = model.recoverableArtifacts; + expect(artifact.detail).toBe('tar.zst'); + expect(artifact.detailTitle).toBe('local:backup/vzdump-lxc-112-2026_05_24-02_00_00.tar.zst'); + expect(recoverableArtifactMatchesSearch(artifact, 'vzdump-lxc-112')).toBe(true); + }); + + it('falls back to the archive file name when the provider reports no format', () => { + const model = buildProxmoxBackupRecoveryModel({ + workloads: [workload({})], + pbsBackups: [], + archives: [archive({ format: '' })], + snapshots: [], + tasks: [], + nowMs: Date.parse('2026-05-26T08:00:00Z'), + }); + + expect(model.recoverableArtifacts[0].detail).toBe('vzdump-lxc-112-2026_05_24-02_00_00.tar.zst'); + }); + it('keeps a terminally incomplete PBS artifact out of latest recovery', () => { const model = buildProxmoxBackupRecoveryModel({ workloads: [workload({})], @@ -186,10 +215,49 @@ describe('proxmoxBackupRecoveryModel', () => { const failed = model.recoverableArtifacts.find((artifact) => artifact.id.includes('failed')); expect(failed).toMatchObject({ failed: true, running: false, verified: undefined }); - expect(model.coverageRows[0].latestRecovery?.createdAt).toBe('2026-05-25T01:34:25Z'); + expect(model.coverageRows[0].latestBackup?.createdAt).toBe('2026-05-25T01:34:25Z'); expect(recoverableArtifactMatchesSearch(failed!, 'failed incomplete')).toBe(true); }); + it('keeps a fresh guest snapshot from standing in for the last backup', () => { + const model = buildProxmoxBackupRecoveryModel({ + workloads: [workload({})], + pbsBackups: [pbsBackup({ backupTime: '2026-05-05T01:34:25Z' })], + archives: [], + snapshots: [snapshot({ time: '2026-05-26T03:00:00Z' })], + tasks: [], + nowMs: Date.parse('2026-05-26T08:00:00Z'), + }); + + const row = model.coverageRows[0]; + expect(row.latestSnapshot?.createdAt).toBe('2026-05-26T03:00:00Z'); + expect(row.latestBackup?.sourceKind).toBe('pbs'); + expect(row.latestBackup?.createdAt).toBe('2026-05-05T01:34:25Z'); + expect( + getRecoveryAgeBand( + row.latestBackup?.createdMs, + model.coverageRows.length && Date.parse('2026-05-26T08:00:00Z'), + ), + ).toBe('aging'); + }); + + it('reports no last backup for a snapshot-only workload', () => { + const model = buildProxmoxBackupRecoveryModel({ + workloads: [workload({})], + pbsBackups: [], + archives: [], + snapshots: [snapshot()], + tasks: [], + nowMs: Date.parse('2026-05-26T08:00:00Z'), + }); + + const row = model.coverageRows[0]; + expect(row.latestBackup).toBeUndefined(); + expect(row.latestSnapshot?.sourceKind).toBe('snapshot'); + expect(row.artifacts).toHaveLength(1); + expect(model.coverageSummary.recoverableArtifacts).toBe(1); + }); + it('uses canonical workload attention while retaining the failed task as evidence', () => { const model = buildProxmoxBackupRecoveryModel({ workloads: [workload({})], diff --git a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTablePresentation.test.ts b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTablePresentation.test.ts index 8817787792..a52bd7c075 100644 --- a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTablePresentation.test.ts +++ b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTablePresentation.test.ts @@ -44,10 +44,10 @@ describe('Proxmox backups responsive table presentation', () => { 'status', 'datastore', 'used', - 'backups', ]); expect(getBackupServerColumnWidthStyle('server', 'compact')).toEqual({ width: '40%' }); - expect(getBackupServerColumnWidthStyle('used', 'compact')).toEqual({ width: '20%' }); + expect(getBackupServerColumnWidthStyle('status', 'compact')).toEqual({ width: '23%' }); + expect(getBackupServerColumnWidthStyle('used', 'compact')).toEqual({ width: '15%' }); expect(ids(getBackupServerColumns('basic'))).toEqual([ 'server', 'status', @@ -124,12 +124,11 @@ describe('Proxmox backups responsive table presentation', () => { expect(ids(getRecoverableColumns('compact'))).toEqual([ 'workload', 'source', - 'location', 'created', 'state', ]); expect(getRecoverableColumnWidthStyle('workload', 'compact')).toEqual({ width: '40%' }); - expect(getRecoverableColumnWidthStyle('state', 'compact')).toEqual({ width: '16%' }); + expect(getRecoverableColumnWidthStyle('state', 'compact')).toEqual({ width: '30%' }); expect(ids(getRecoverableColumns('basic'))).toEqual([ 'workload', 'source', diff --git a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTableShared.test.tsx b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTableShared.test.tsx index 64f5895ce9..4ffe9073bd 100644 --- a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTableShared.test.tsx +++ b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTableShared.test.tsx @@ -8,8 +8,10 @@ import proxmoxBackupsTableSharedSource from '../proxmoxBackupsTableShared.tsx?ra import { ArtifactSourceBadge, ArtifactStateBadge, + ProxmoxBackupAgeText, ProxmoxBackupWorkloadTypeBadge, SortableHead, + formatCompactBackupAge, } from '../proxmoxBackupsTableShared'; import type { RecoverableArtifact } from '../proxmoxBackupRecoveryModel'; @@ -86,6 +88,28 @@ describe('proxmoxBackupsTableShared', () => { } }); + it('drops the age suffix in the phone projection and keeps the timestamp on hover', () => { + const createdAt = new Date(Date.now() - 18 * 60 * 60 * 1000).toISOString(); + expect(formatCompactBackupAge(createdAt)).toBe('18h'); + expect(formatCompactBackupAge(new Date().toISOString())).toBe('now'); + + const artifact = { + id: 'pbs:1', + nativeId: '1', + sourceKind: 'pbs', + sourceLabel: 'PBS', + workload: { key: 'w', type: 'vm', typeLabel: 'VM', vmid: '100', label: 'VM 100' }, + createdAt, + createdMs: Date.parse(createdAt), + location: 'main', + detail: '', + protected: false, + } as RecoverableArtifact; + render(() => ); + const age = screen.getByText('18h'); + expect(age.closest('[title]')?.getAttribute('title')).toContain(createdAt); + }); + it('keeps backup age text on the shared relative-time primitive', () => { expect(proxmoxBackupsTableSharedSource).toContain('PlatformTableRelativeTimeValue'); expect(proxmoxBackupsTableSharedSource).not.toContain('formatRelativeTime('); diff --git a/frontend-modern/src/features/proxmox/__tests__/proxmoxHostTableModel.test.ts b/frontend-modern/src/features/proxmox/__tests__/proxmoxHostTableModel.test.ts index 66ee4d61d0..1b587b374a 100644 --- a/frontend-modern/src/features/proxmox/__tests__/proxmoxHostTableModel.test.ts +++ b/frontend-modern/src/features/proxmox/__tests__/proxmoxHostTableModel.test.ts @@ -19,16 +19,16 @@ describe('proxmoxHostTableModel', () => { expect(getProxmoxHostColumnWidthStyle('uptime', 'narrow', ids)).toEqual({ width: '15%' }); }); - it('shows the full operational metric track at phone width', () => { + it('matches the guest table columns and keeps node names whole at phone width', () => { const columns = getProxmoxHostVisibleColumnsForLayout('phone'); const ids = columns.map((column) => column.id); - expect(ids).toEqual(['node', 'cpu', 'memory', 'disk', 'temp', 'uptime']); + expect(ids).toEqual(['node', 'cpu', 'memory', 'disk', 'uptime']); expect(getProxmoxHostTableMinWidthClass('phone')).toBe('min-w-0'); - expect(getProxmoxHostColumnWidthStyle('node', 'phone', ids)).toEqual({ width: '30%' }); - expect(getProxmoxHostColumnWidthStyle('disk', 'phone', ids)).toEqual({ width: '13.3824%' }); - expect(getProxmoxHostColumnWidthStyle('temp', 'phone', ids)).toEqual({ width: '15.4412%' }); - expect(getProxmoxHostColumnWidthStyle('uptime', 'phone', ids)).toEqual({ width: '14.4118%' }); + expect(getProxmoxHostColumnWidthStyle('node', 'phone', ids)).toEqual({ width: '38%' }); + expect(getProxmoxHostColumnWidthStyle('cpu', 'phone', ids)).toEqual({ width: '15.5%' }); + expect(getProxmoxHostColumnWidthStyle('disk', 'phone', ids)).toEqual({ width: '15.5%' }); + expect(getProxmoxHostColumnWidthStyle('uptime', 'phone', ids)).toEqual({ width: '15.5%' }); }); it('prioritizes live utilization columns in the mobile host table', () => { diff --git a/frontend-modern/src/features/proxmox/proxmoxBackupRecoveryModel.ts b/frontend-modern/src/features/proxmox/proxmoxBackupRecoveryModel.ts index c71846a6ae..b4288cf9f6 100644 --- a/frontend-modern/src/features/proxmox/proxmoxBackupRecoveryModel.ts +++ b/frontend-modern/src/features/proxmox/proxmoxBackupRecoveryModel.ts @@ -47,6 +47,8 @@ export interface RecoverableArtifact { /** Operator-facing source/repository label used by the backup-location filter. */ locationLabel?: string; detail: string; + /** Full provider identifier behind a shortened `detail`, shown on hover. */ + detailTitle?: string; protected: boolean; verified?: boolean; /** A still-running backup is writing this artifact; it is not recoverable yet. */ @@ -70,7 +72,13 @@ export interface WorkloadCoverageRow { key: string; workload: WorkloadReference; artifacts: RecoverableArtifact[]; - latestRecovery?: RecoverableArtifact; + /** + * Newest completed PBS snapshot or PVE backup file. Guest snapshots never + * fill this slot: they live on the guest's own storage and the protection + * posture engine does not count them as independent recovery, so a fresh + * snapshot must not read as a fresh backup beside a stale posture. + */ + latestBackup?: RecoverableArtifact; latestPBS?: RecoverableArtifact; latestArchive?: RecoverableArtifact; latestSnapshot?: RecoverableArtifact; @@ -138,12 +146,25 @@ export function getRecoveryAgeBand( return 'stale'; } +// A guest snapshot shares the guest's storage, so it is listed as restore +// evidence but never counted as a backup. Mirrors the server posture rule that +// "snapshots alone do not prove independent recovery". +export function isBackupArtifact(artifact: RecoverableArtifact): boolean { + return artifact.sourceKind !== 'snapshot'; +} + function parseTimestampMs(value: string | undefined): number | undefined { if (!value) return undefined; const ms = Date.parse(value); return Number.isFinite(ms) ? ms : undefined; } +function archiveFileName(volid: string | undefined): string { + const trimmed = volid?.trim() ?? ''; + if (!trimmed) return ''; + return trimmed.slice(trimmed.lastIndexOf('/') + 1); +} + function pbsBackupFileDetailLabel(fileCount: number): string { if (fileCount <= 0) return 'PBS files not listed'; if (fileCount === 1) return '1 PBS file'; @@ -572,7 +593,12 @@ export function buildProxmoxBackupRecoveryModel( size: archive.size, location: archive.storage || archive.node || '—', ...artifactLocationScope('archive', archive.instance, archive.storage || archive.node), - detail: archive.volid || archive.format || archiveSource.detailFallbackLabel, + // The volid repeats the storage, guest type, VMID, and timestamp the row + // already shows in their own columns and clips at every width; the + // archive format is the one fact it adds, so that is the cell and the + // volid stays on hover and in search. + detail: archive.format || archiveFileName(archive.volid) || archiveSource.detailFallbackLabel, + detailTitle: archive.volid || undefined, protected: archive.protected, verified: archive.inProgress ? undefined : archive.isPBS ? archive.verified : undefined, running: archive.inProgress === true, @@ -626,6 +652,7 @@ export function buildProxmoxBackupRecoveryModel( // the "latest" pointers (which answer "what can I recover to?") skip them. // The artifacts themselves stay listed with an honest state. const completed = row.artifacts.filter((artifact) => !artifact.running && !artifact.failed); + row.latestBackup = newest(completed.filter(isBackupArtifact), (artifact) => artifact.createdMs); row.latestPBS = newest( completed.filter((artifact) => artifact.sourceKind === 'pbs'), (artifact) => artifact.createdMs, @@ -638,7 +665,6 @@ export function buildProxmoxBackupRecoveryModel( completed.filter((artifact) => artifact.sourceKind === 'snapshot'), (artifact) => artifact.createdMs, ); - row.latestRecovery = newest(completed, (artifact) => artifact.createdMs); } const coverageRows = Array.from(rows.values()).map((row) => { @@ -665,7 +691,7 @@ export function buildProxmoxBackupRecoveryModel( coverageRows.sort((left, right) => { if (left.postureRank !== right.postureRank) return left.postureRank - right.postureRank; - return (right.latestRecovery?.createdMs ?? 0) - (left.latestRecovery?.createdMs ?? 0); + return (right.latestBackup?.createdMs ?? 0) - (left.latestBackup?.createdMs ?? 0); }); artifacts.sort((left, right) => (right.createdMs ?? 0) - (left.createdMs ?? 0)); @@ -730,6 +756,7 @@ export function recoverableArtifactMatchesSearch( artifact.sourceTitle, artifact.location, artifact.detail, + artifact.detailTitle, artifact.verified === true ? 'verified' : artifact.verified === false diff --git a/frontend-modern/src/features/proxmox/proxmoxBackupsTablePresentation.ts b/frontend-modern/src/features/proxmox/proxmoxBackupsTablePresentation.ts index b1342d3396..92e3e2f3a0 100644 --- a/frontend-modern/src/features/proxmox/proxmoxBackupsTablePresentation.ts +++ b/frontend-modern/src/features/proxmox/proxmoxBackupsTablePresentation.ts @@ -93,8 +93,10 @@ const BACKUP_SERVER_VISIBLE: Record< > = { // Reachability and datastore exhaustion are the two risks that can stop all // future backups. Host telemetry and space-efficiency context return only as - // the table gains enough room to keep those headline answers readable. - compact: ['server', 'status', 'datastore', 'used', 'backups'], + // the table gains enough room to keep those headline answers readable. The + // phone projection keeps those two answers whole ("Healthy", "33.6%") and + // lets the backup count return with the first wider layout. + compact: ['server', 'status', 'datastore', 'used'], basic: ['server', 'status', 'datastore', 'used', 'backups'], operational: ['server', 'status', 'cpu', 'memory', 'datastore', 'used', 'backups'], expanded: ['server', 'status', 'cpu', 'memory', 'uptime', 'datastore', 'used', 'backups'], @@ -105,8 +107,10 @@ const BACKUP_SERVER_WEIGHTS: Record< ProxmoxBackupsTableLayoutMode, Partial> > = { - compact: { server: 40, status: 15, datastore: 15, used: 20, backups: 10 }, - basic: { server: 24, status: 14, datastore: 20, used: 28, backups: 14 }, + compact: { server: 40, status: 23, datastore: 22, used: 15 }, + // Just above the phone projection the used cell carries its used/total + // pair again, the longest value in the row, so it takes the most room. + basic: { server: 24, status: 13.6, datastore: 18.2, used: 28.4, backups: 9.8 }, operational: { server: 20, status: 12, @@ -175,7 +179,7 @@ export const COVERAGE_COLUMNS: readonly BackupTableColumn[] = { id: 'targetId', label: 'Target ID', kind: 'text' }, { id: 'node', label: 'Node', kind: 'text' }, { id: 'posture', label: 'Posture', kind: 'text' }, - { id: 'latest', label: 'Restore', kind: 'numeric-value' }, + { id: 'latest', label: 'Last backup', kind: 'numeric-value' }, { id: 'pbs', label: 'PBS snapshot', kind: 'text' }, { id: 'archive', label: 'PVE file', kind: 'text' }, { id: 'snapshot', label: 'Guest snapshot', kind: 'text' }, @@ -184,7 +188,7 @@ export const COVERAGE_COLUMNS: readonly BackupTableColumn[] = const COVERAGE_VISIBLE: Record = { // On narrow surfaces answer: which workload, what posture, how recent is the - // newest restore point, and did the latest task succeed? Provider-by-provider + // newest independent backup, and did the latest task succeed? Provider-by-provider // evidence is progressive detail in the expansion row; target identity folds // beneath the name so the scan stays legible at 320px. compact: ['workload', 'posture', 'latest', 'pbs', 'task'], @@ -198,11 +202,14 @@ const COVERAGE_WEIGHTS: Record< ProxmoxBackupsTableLayoutMode, Partial> > = { - compact: { workload: 40, posture: 18, latest: 16, pbs: 16, task: 10 }, + // Posture keeps its whole compact word ("Unknown"), the two age cells hold + // the suffix-free compact age, and the job dot takes what a "Job" header + // needs. + compact: { workload: 40, posture: 21, latest: 14.5, pbs: 14.5, task: 10 }, // Above the phone projection the identity columns (workload, node) take the // slack that the short type badge, target id, and age cells cannot use, so // names stay whole instead of truncating beside empty space. - basic: { workload: 31, node: 20, posture: 20, latest: 14, task: 15 }, + basic: { workload: 31, node: 18, posture: 22, latest: 15, task: 14 }, operational: { workload: 27, type: 8, node: 17, posture: 17, latest: 15, task: 16 }, expanded: { workload: 19, @@ -287,7 +294,10 @@ const RECOVERABLE_VISIBLE: Record getRecoveryAgeBand(props.artifact.createdMs); const title = () => { const parts = [recoveryAgeTitleByBand[band()], props.artifact.createdAt].filter(Boolean); @@ -228,7 +239,12 @@ export function ProxmoxBackupAgeText(props: { artifact: RecoverableArtifact }) { return ( - + } + > + {formatCompactBackupAge(props.artifact.createdAt)} + ); } diff --git a/frontend-modern/src/features/proxmox/proxmoxHostTableModel.ts b/frontend-modern/src/features/proxmox/proxmoxHostTableModel.ts index 76ce153aa8..be6aaafe31 100644 --- a/frontend-modern/src/features/proxmox/proxmoxHostTableModel.ts +++ b/frontend-modern/src/features/proxmox/proxmoxHostTableModel.ts @@ -31,7 +31,7 @@ const HOST_COLUMN_MIN_LAYOUT: Record = { - node: 18, - version: 7, + node: 20, + version: 6, uptime: 7, cpu: 13, memory: 13, @@ -55,7 +55,7 @@ const HOST_COLUMN_DESKTOP_WIDTHS: Record = { temp: 6, vms: 5, cts: 5, - cluster: 13, + cluster: 12, }; const HOST_COLUMN_RESPONSIVE_WEIGHTS: Record< @@ -69,13 +69,15 @@ const HOST_COLUMN_RESPONSIVE_WEIGHTS: Record< disk: 15, uptime: 15, }, + // Phones carry the same five columns as the guest table below them. A sixth + // (temperature) left the node name with four characters, which made a + // cluster of "Analytics A".."F" rows unreadable. phone: { node: 32, - cpu: 13, - memory: 13, - disk: 13, - temp: 15, - uptime: 14, + cpu: 15, + memory: 15, + disk: 15, + uptime: 15, }, mobile: { node: 32, @@ -146,6 +148,11 @@ export const getProxmoxHostVisibleColumnsForLayout = ( ); }; +// The node cell carries the expansion toggle and status dot before the name, +// so it needs more than the shared 30% phone identity anchor to show a +// typical node name whole. +const PROXMOX_HOST_PHONE_IDENTITY_WIDTH_PERCENT = 38; + export const getProxmoxHostColumnWidthStyle = ( columnId: ProxmoxHostTableColumnId, layoutMode: WorkloadTableLayoutMode, @@ -159,9 +166,11 @@ export const getProxmoxHostColumnWidthStyle = ( visibleColumnIds, layoutMode === 'narrow' ? { columnId: 'node', widthPercent: PLATFORM_TABLE_NARROW_IDENTITY_WIDTH_PERCENT } - : layoutMode === 'phone' || layoutMode === 'mobile' - ? { columnId: 'node', widthPercent: PLATFORM_TABLE_PHONE_IDENTITY_WIDTH_PERCENT } - : undefined, + : layoutMode === 'phone' + ? { columnId: 'node', widthPercent: PROXMOX_HOST_PHONE_IDENTITY_WIDTH_PERCENT } + : layoutMode === 'mobile' + ? { columnId: 'node', widthPercent: PLATFORM_TABLE_PHONE_IDENTITY_WIDTH_PERCENT } + : undefined, ); }; diff --git a/frontend-modern/src/features/storageBackups/__tests__/diskDetailPresentation.test.ts b/frontend-modern/src/features/storageBackups/__tests__/diskDetailPresentation.test.ts index 8edf3ca711..89a8caacb3 100644 --- a/frontend-modern/src/features/storageBackups/__tests__/diskDetailPresentation.test.ts +++ b/frontend-modern/src/features/storageBackups/__tests__/diskDetailPresentation.test.ts @@ -4,6 +4,7 @@ import { DISK_DETAIL_LIVE_CHARTS, getDiskAttributeValueTextClass, getDiskDetailAttributeCards, + getDiskDetailHealthPresentation, getDiskDetailHistoryCharts, getDiskDetailHistoryFallbackMessage, getDiskDetailLiveBadgeLabel, @@ -34,6 +35,41 @@ describe('diskDetailPresentation', () => { ); }); + it('carries the row verdict and its reason into the drawer header', () => { + const disk = { + node: 'tower', + instance: 'cluster-main', + devPath: '/dev/sda', + model: 'Archive HDD', + serial: 'SERIAL-1', + wwn: '', + size: 1, + health: 'PASSED', + wearout: -1, + type: 'hdd', + temperature: 42, + rpm: 7200, + used: '', + riskReasons: [] as string[], + }; + expect(getDiskDetailHealthPresentation(disk)).toEqual({ + label: 'Healthy', + summary: '', + tone: 'text-emerald-700 dark:text-emerald-300', + }); + expect( + getDiskDetailHealthPresentation({ + ...disk, + health: 'FAILED', + riskReasons: ['SMART failed.'], + }), + ).toEqual({ + label: 'Replace Now', + summary: 'SMART failed.', + tone: 'text-red-700 dark:text-red-300', + }); + }); + it('builds canonical SATA and NVMe attribute cards', () => { expect( getDiskDetailAttributeCards({ diff --git a/frontend-modern/src/features/storageBackups/__tests__/diskPresentation.test.ts b/frontend-modern/src/features/storageBackups/__tests__/diskPresentation.test.ts index 4d177523c1..935bcce339 100644 --- a/frontend-modern/src/features/storageBackups/__tests__/diskPresentation.test.ts +++ b/frontend-modern/src/features/storageBackups/__tests__/diskPresentation.test.ts @@ -19,9 +19,11 @@ import { PHYSICAL_DISK_TABLE_CLASS, PHYSICAL_DISK_TABLE_ROW_HOVER_CLASS, getPhysicalDiskEmptyStatePresentation, - getPhysicalDiskColumnWidthPercent, + getPhysicalDiskCellPaddingClass, + getPhysicalDiskColumnWidthStyle, getPhysicalDiskCollectionMessages, getPhysicalDiskFieldStatusMessage, + getPhysicalDiskHealthCompactLabel, getPhysicalDiskHealthStatus, getPhysicalDiskHealthSummary, getPhysicalDiskHostLabel, @@ -158,14 +160,55 @@ describe('diskPresentation', () => { ), ), ).toBe(true); - expect(getPhysicalDiskColumnWidthPercent('compact', 'disk')).toBe(33); - expect(getPhysicalDiskColumnWidthPercent('compact', 'size')).toBe(14); - expect(getPhysicalDiskColumnWidthPercent('operational', 'life')).toBe(9); - expect(getPhysicalDiskColumnWidthPercent('compact', 'device')).toBe(0); + expect(getPhysicalDiskColumnWidthStyle('compact', 'disk')).toEqual({ width: '34%' }); + expect(getPhysicalDiskColumnWidthStyle('compact', 'temp')).toEqual({ width: '12%' }); + expect(getPhysicalDiskColumnWidthStyle('compact', 'size')).toEqual({ width: '15%' }); + expect(getPhysicalDiskColumnWidthStyle('operational', 'life')).toEqual({ width: '9%' }); + expect(getPhysicalDiskColumnWidthStyle('compact', 'device')).toEqual({ width: '0%' }); expect(isPhysicalDiskColumnVisible('narrow', 'life')).toBe(false); expect(isPhysicalDiskColumnVisible('narrow', 'temp')).toBe(true); - expect(getPhysicalDiskColumnWidthPercent('narrow', 'disk')).toBe(40); - expect(getPhysicalDiskColumnWidthPercent('narrow', 'size')).toBe(15); + expect(getPhysicalDiskColumnWidthStyle('narrow', 'disk')).toEqual({ width: '41%' }); + expect(getPhysicalDiskColumnWidthStyle('narrow', 'temp')).toEqual({ width: '12%' }); + expect(getPhysicalDiskColumnWidthStyle('narrow', 'size')).toEqual({ width: '16%' }); + }); + + it('keeps the desktop disk column widths on the canonical weighted helper', () => { + const columns = [ + 'disk', + 'device', + 'host', + 'role', + 'parent', + 'health', + 'life', + 'temp', + 'size', + ] as const; + expect(columns.map((column) => getPhysicalDiskColumnWidthStyle('full', column).width)).toEqual([ + '19%', + '9%', + '10%', + '8%', + '12%', + '17%', + '6%', + '7%', + '12%', + ]); + }); + + it('sheds the shared cell padding only on the phone disk layouts', () => { + expect(getPhysicalDiskCellPaddingClass('narrow')).toBe('px-1!'); + expect(getPhysicalDiskCellPaddingClass('compact')).toBe('px-1!'); + expect(getPhysicalDiskCellPaddingClass('basic')).toBe(''); + expect(getPhysicalDiskCellPaddingClass('full')).toBe(''); + }); + + it('shortens only the health words that cannot fit a phone health column', () => { + expect(getPhysicalDiskHealthCompactLabel('Needs Attention')).toBe('Attention'); + expect(getPhysicalDiskHealthCompactLabel('Replace Now')).toBe('Replace'); + expect(getPhysicalDiskHealthCompactLabel('Healthy')).toBe('Healthy'); + expect(getPhysicalDiskHealthCompactLabel('Unknown')).toBe('Unknown'); }); it('detects SMART warnings from counters', () => { diff --git a/frontend-modern/src/features/storageBackups/__tests__/resourceStoragePresentation.test.ts b/frontend-modern/src/features/storageBackups/__tests__/resourceStoragePresentation.test.ts index 14cd9deebd..5185db68c3 100644 --- a/frontend-modern/src/features/storageBackups/__tests__/resourceStoragePresentation.test.ts +++ b/frontend-modern/src/features/storageBackups/__tests__/resourceStoragePresentation.test.ts @@ -62,6 +62,14 @@ describe('resourceStoragePresentation', () => { ), ).toBe('Datastore'); expect(getResourceStorageTopologyLabel(makeResource(), 'rbd')).toBe('Cluster Storage'); + // Proxmox plugin names are acronyms or compound words; title-casing them + // produced "Lvmthin" for every default local-lvm storage and "Nfs". + expect(getResourceStorageTopologyLabel(makeResource(), 'lvmthin')).toBe('LVM-Thin'); + expect(getResourceStorageTopologyLabel(makeResource(), 'lvm')).toBe('LVM'); + expect(getResourceStorageTopologyLabel(makeResource(), 'nfs')).toBe('NFS'); + expect(getResourceStorageTopologyLabel(makeResource(), 'cifs')).toBe('SMB/CIFS'); + expect(getResourceStorageTopologyLabel(makeResource(), 'iscsi')).toBe('iSCSI'); + expect(getResourceStorageTopologyLabel(makeResource(), 'btrfs')).toBe('BTRFS'); expect(getResourceStorageTopologyLabel(makeResource(), 'ignored', 'rebuild target')).toBe( 'Rebuild Target', ); diff --git a/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.branchcov0712b.test.ts b/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.branchcov0712b.test.ts index 99ee1f17b7..a23aa645a0 100644 --- a/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.branchcov0712b.test.ts +++ b/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.branchcov0712b.test.ts @@ -219,12 +219,12 @@ describe('getStoragePoolStateLabel branch coverage', () => { expect(getStoragePoolStateLabel(record)).toBe('Online'); }); - it('returns the raw pool state when it is anything other than "ONLINE"', () => { + it('titleizes a non-ONLINE zfs pool state like every other state path', () => { const record = { ...baseRecord(), details: { zfsPool: { state: 'FAULTED', devices: [] } }, } as unknown as StorageRecord; - expect(getStoragePoolStateLabel(record)).toBe('FAULTED'); + expect(getStoragePoolStateLabel(record)).toBe('Faulted'); }); it('skips a zfs pool whose state is the empty string (pool?.state truthiness guard false)', () => { diff --git a/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.branchcov2.test.ts b/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.branchcov2.test.ts index 0c2c8f0b59..27d8fff8e6 100644 --- a/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.branchcov2.test.ts +++ b/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.branchcov2.test.ts @@ -239,7 +239,7 @@ describe('getStoragePoolStateLabel branch coverage', () => { expect(getStoragePoolStateLabel(record)).toBe('Online'); }); - it('returns the raw pool state when it is anything other than "ONLINE"', () => { + it('titleizes a non-ONLINE zfs pool state like every other state path', () => { const record = { ...baseRecord(), details: { @@ -249,7 +249,7 @@ describe('getStoragePoolStateLabel branch coverage', () => { }, }, } as unknown as StorageRecord; - expect(getStoragePoolStateLabel(record)).toBe('FAULTED'); + expect(getStoragePoolStateLabel(record)).toBe('Faulted'); }); it('titleizes the derived status when arrayState and pool are both absent', () => { diff --git a/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.test.ts b/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.test.ts index 1afe8dff92..8787f8637d 100644 --- a/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.test.ts +++ b/frontend-modern/src/features/storageBackups/__tests__/rowPresentation.test.ts @@ -101,6 +101,47 @@ describe('storage row presentation', () => { expect(getCompactStoragePoolProtectionTitle(record)).not.toContain('without parity'); }); + it('shows a zfs rebuild as its activity word and keeps the sentence as the title', () => { + const resilvering = { + ...baseRecord(), + rebuildInProgress: true, + protectionLabel: 'ZFS pool tank is resilvering', + details: { + zfsPool: { state: 'DEGRADED', scan: 'resilver in progress', devices: [] }, + }, + } as unknown as StorageRecord; + expect(getCompactStoragePoolProtectionLabel(resilvering)).toBe('Resilvering'); + expect(getCompactStoragePoolProtectionTitle(resilvering)).toBe('ZFS pool tank is resilvering'); + + const scrubbing = { + ...baseRecord(), + rebuildInProgress: true, + protectionLabel: 'ZFS pool tank scrub is running (45.2%)', + protectionSummary: 'ZFS pool tank scrub is running (45.2%)', + details: { + zfsPool: { + state: 'ONLINE', + scan: '', + scanDetails: { function: 'SCRUB', state: 'SCANNING', percentage: 45.2 }, + devices: [], + }, + }, + } as unknown as StorageRecord; + expect(getCompactStoragePoolProtectionLabel(scrubbing)).toBe('Scrubbing 45%'); + expect(getCompactStoragePoolProtectionTitle(scrubbing)).toBe( + 'ZFS pool tank scrub is running (45.2%)', + ); + + // A rebuild without zfs scan data keeps the label its platform supplied. + const parity = { + ...baseRecord(), + rebuildInProgress: true, + protectionLabel: 'Parity sync 12%', + } as unknown as StorageRecord; + expect(getCompactStoragePoolProtectionLabel(parity)).toBe('Parity sync 12%'); + expect(getCompactStoragePoolProtectionTitle(parity)).toBe('Parity sync 12%'); + }); + it('derives zfs issue fallback from pool state and errors', () => { const record = { ...baseRecord(), diff --git a/frontend-modern/src/features/storageBackups/__tests__/storageAdapters.test.ts b/frontend-modern/src/features/storageBackups/__tests__/storageAdapters.test.ts index 96866fbeda..7e5917d601 100644 --- a/frontend-modern/src/features/storageBackups/__tests__/storageAdapters.test.ts +++ b/frontend-modern/src/features/storageBackups/__tests__/storageAdapters.test.ts @@ -467,6 +467,37 @@ describe('storageAdapters', () => { expect(records[0].hostLabel).toBe('pve1'); }); + it('labels a poller-merged shared storage as shared with its node count', () => { + const records = buildStorageRecords({ + state: baseState(), + resources: [ + makeResourceStorage({ + id: 'cluster-a-cluster-nfs-iso', + name: 'nfs-iso', + parentName: 'cluster', + platformData: { type: 'nfs', node: 'cluster', instance: 'cluster-a', shared: true }, + storage: { + type: 'nfs', + shared: true, + nodes: ['pve1', 'pve2', 'pve3'], + } as Resource['storage'], + }), + makeResourceStorage({ + id: 'pve2-local-lvm', + name: 'local-lvm', + parentName: 'pve2', + platformData: { type: 'lvmthin', node: 'pve2', instance: 'cluster-a', shared: false }, + storage: { type: 'lvmthin', shared: false } as Resource['storage'], + }), + ], + }); + + // The poller folds a shared storage into one entry on the synthetic + // "cluster" node; a node-local storage keeps its node name. + expect(records.map((record) => record.hostLabel)).toEqual(['Shared · 3 nodes', 'pve2']); + expect(records.map((record) => record.topologyLabel)).toEqual(['NFS', 'LVM-Thin']); + }); + it('keeps dependency impact separate from healthy storage primary issues', () => { const records = buildStorageRecords({ state: baseState(), diff --git a/frontend-modern/src/features/storageBackups/__tests__/storageBarPresentation.test.ts b/frontend-modern/src/features/storageBackups/__tests__/storageBarPresentation.test.ts index a2b1c04bed..86ec88080b 100644 --- a/frontend-modern/src/features/storageBackups/__tests__/storageBarPresentation.test.ts +++ b/frontend-modern/src/features/storageBackups/__tests__/storageBarPresentation.test.ts @@ -13,6 +13,7 @@ import { STORAGE_BAR_ZFS_SECTION_CLASS, STORAGE_BAR_ZFS_STATE_LABEL_CLASS, STORAGE_BAR_ZFS_STATE_ROW_CLASS, + getStorageBarCompactLabel, getStorageBarLabel, getStorageBarTooltipRowClass, getStorageBarTooltipRows, @@ -39,6 +40,7 @@ describe('storageBarPresentation', () => { expect(STORAGE_BAR_ZFS_STATE_LABEL_CLASS).toBe('text-slate-400'); expect(getStorageBarUsagePercent(40, 100)).toBe(40); expect(getStorageBarLabel(40, 100)).toBe('40% (40.0 B/100 B)'); + expect(getStorageBarCompactLabel(40, 100)).toBe('40%'); expect(getStorageBarTooltipTitle()).toBe('Storage Details'); expect(getStorageBarTooltipRowClass()).toBe('flex justify-between gap-3 py-0.5 '); expect(getStorageBarTooltipRowClass(true)).toBe( diff --git a/frontend-modern/src/features/storageBackups/__tests__/storagePagePresentation.test.ts b/frontend-modern/src/features/storageBackups/__tests__/storagePagePresentation.test.ts index a8971d42b9..1a6f807595 100644 --- a/frontend-modern/src/features/storageBackups/__tests__/storagePagePresentation.test.ts +++ b/frontend-modern/src/features/storageBackups/__tests__/storagePagePresentation.test.ts @@ -1,6 +1,7 @@ import { describe, expect, it } from 'vitest'; import { - getStoragePoolColumnWidthPercent, + getStoragePoolCellPaddingClass, + getStoragePoolColumnWidthStyle, getStoragePoolTableColumns, getStoragePoolTableLayoutModeForContainer, getStorageTableHeading, @@ -46,22 +47,44 @@ describe('storagePagePresentation', () => { expect(getStoragePoolTableLayoutModeForContainer(0)).toBe('compact'); expect(getStoragePoolTableLayoutModeForContainer(359)).toBe('narrow'); expect(getStoragePoolTableLayoutModeForContainer(360)).toBe('compact'); - expect(getStoragePoolTableLayoutModeForContainer(559)).toBe('compact'); - expect(getStoragePoolTableLayoutModeForContainer(560)).toBe('operational'); + // The compact/operational boundary is the shared phone container query + // (34rem), so the usage bar label and the column set change together. + expect(getStoragePoolTableLayoutModeForContainer(543)).toBe('compact'); + expect(getStoragePoolTableLayoutModeForContainer(544)).toBe('operational'); expect(getStoragePoolTableLayoutModeForContainer(1_039)).toBe('operational'); expect(getStoragePoolTableLayoutModeForContainer(1_040)).toBe('full'); expect(isStoragePoolColumnVisible('operational', 'host')).toBe(true); expect(isStoragePoolColumnVisible('operational', 'growth')).toBe(false); - expect(getStoragePoolColumnWidthPercent('operational', 'usage')).toBe(21); + expect(getStoragePoolColumnWidthStyle('operational', 'usage')).toEqual({ width: '25%' }); + expect(getStoragePoolColumnWidthStyle('operational', 'name')).toEqual({ width: '27%' }); expect(isStoragePoolColumnVisible('compact', 'host')).toBe(true); expect(isStoragePoolColumnVisible('compact', 'type')).toBe(false); expect(isStoragePoolColumnVisible('compact', 'protection')).toBe(false); - expect(getStoragePoolColumnWidthPercent('compact', 'name')).toBe(35); - expect(getStoragePoolColumnWidthPercent('compact', 'usage')).toBe(25); - expect(getStoragePoolColumnWidthPercent('compact', 'growth')).toBe(0); + expect(getStoragePoolColumnWidthStyle('compact', 'name')).toEqual({ width: '37%' }); + expect(getStoragePoolColumnWidthStyle('compact', 'host')).toEqual({ width: '31.5%' }); + expect(getStoragePoolColumnWidthStyle('compact', 'usage')).toEqual({ width: '13%' }); + expect(getStoragePoolColumnWidthStyle('compact', 'growth')).toEqual({ width: '0%' }); expect(isStoragePoolColumnVisible('narrow', 'type')).toBe(false); expect(isStoragePoolColumnVisible('narrow', 'protection')).toBe(false); - expect(getStoragePoolColumnWidthPercent('narrow', 'name')).toBe(40); - expect(getStoragePoolColumnWidthPercent('narrow', 'usage')).toBe(20); + expect(getStoragePoolColumnWidthStyle('narrow', 'name')).toEqual({ width: '35%' }); + expect(getStoragePoolColumnWidthStyle('narrow', 'usage')).toEqual({ width: '13%' }); + }); + + it('sheds the desktop cell gutter only on the phone pool layouts', () => { + expect(getStoragePoolCellPaddingClass('narrow')).toBe('px-1!'); + expect(getStoragePoolCellPaddingClass('compact')).toBe('px-1!'); + expect(getStoragePoolCellPaddingClass('operational')).toBe(''); + expect(getStoragePoolCellPaddingClass('full')).toBe(''); + }); + + it('keeps the desktop pool column widths on the canonical weighted helper', () => { + const fullWidths = ['name', 'state', 'type', 'host', 'protection', 'usage', 'growth'].map( + (column) => + getStoragePoolColumnWidthStyle( + 'full', + column as 'name' | 'state' | 'type' | 'host' | 'protection' | 'usage' | 'growth', + ).width, + ); + expect(fullWidths).toEqual(['20%', '14%', '10%', '12%', '13%', '20%', '11%']); }); }); diff --git a/frontend-modern/src/features/storageBackups/diskDetailPresentation.ts b/frontend-modern/src/features/storageBackups/diskDetailPresentation.ts index 12cc6a8f12..810daab8d7 100644 --- a/frontend-modern/src/features/storageBackups/diskDetailPresentation.ts +++ b/frontend-modern/src/features/storageBackups/diskDetailPresentation.ts @@ -1,4 +1,8 @@ -import type { PhysicalDiskPresentationData } from '@/features/storageBackups/diskPresentation'; +import { + getPhysicalDiskHealthStatus, + getPhysicalDiskHealthSummary, + type PhysicalDiskPresentationData, +} from '@/features/storageBackups/diskPresentation'; import type { HistoryTimeRange } from '@/api/charts'; import { formatPowerOnHours } from '@/utils/format'; import { getMetricSeverity, type MetricDisplayThresholds } from '@/utils/metricThresholds'; @@ -78,6 +82,32 @@ export const getDiskDetailLiveBadgeLabel = (): string => 'Real-time'; export const getDiskDetailHistoryFallbackMessage = (): string => 'Historical disk charts are unavailable until Pulse can resolve a stable identity for this disk.'; +export type DiskDetailHealthPresentation = { + label: string; + summary: string; + tone: string; +}; + +// The drawer header carries the disk's verdict and its reason. The table row +// shows only the verdict word, so this is where the reason is read in full. +export const DISK_DETAIL_HEADER_STACK_CLASS = 'min-w-0 space-y-1'; +export const DISK_DETAIL_HEALTH_ROW_CLASS = + 'flex flex-wrap items-baseline gap-x-2 gap-y-0.5 text-[11px]'; +export const DISK_DETAIL_HEALTH_LABEL_CLASS = 'font-semibold'; +export const DISK_DETAIL_HEALTH_SUMMARY_CLASS = + 'min-w-0 max-w-full whitespace-normal wrap-break-word text-muted'; + +export function getDiskDetailHealthPresentation( + disk: PhysicalDiskPresentationData, +): DiskDetailHealthPresentation { + const status = getPhysicalDiskHealthStatus(disk); + return { + label: status.label, + summary: getPhysicalDiskHealthSummary(status), + tone: status.tone, + }; +} + export function getDiskDetailAttributeCards( disk: PhysicalDiskPresentationData, diskTempThresholds?: MetricDisplayThresholds | null, diff --git a/frontend-modern/src/features/storageBackups/diskPresentation.ts b/frontend-modern/src/features/storageBackups/diskPresentation.ts index 38d3516395..563bc8c740 100644 --- a/frontend-modern/src/features/storageBackups/diskPresentation.ts +++ b/frontend-modern/src/features/storageBackups/diskPresentation.ts @@ -1,8 +1,10 @@ +import type { JSX } from 'solid-js'; import type { PhysicalDiskCollectionStatus, PhysicalDiskFieldStatus, Resource, } from '@/types/resource'; +import { getPlatformTableWeightedColumnWidthStyle } from '@/features/platformPage/sharedPlatformPage'; import { getSourcePlatformLabel, getSourcePlatformPresentation, @@ -141,12 +143,16 @@ const PHYSICAL_DISK_VISIBLE_COLUMNS: Record< full: ['disk', 'device', 'host', 'role', 'parent', 'health', 'life', 'temp', 'size'], }; -const PHYSICAL_DISK_COLUMN_WIDTHS: Record< +// Relative weights per layout, resolved through the canonical weighted-width +// helper. The phone layouts are sized from the measured values they show: +// a temperature ("100°C"), a capacity ("5.46 TB"), a compact health word and +// a short node name must each fit whole, and the disk model takes the rest. +const PHYSICAL_DISK_COLUMN_WEIGHTS: Record< PhysicalDiskTableLayoutMode, Partial> > = { - narrow: { disk: 40, host: 15, health: 20, temp: 10, size: 15 }, - compact: { disk: 33, host: 15, health: 18, life: 10, temp: 10, size: 14 }, + narrow: { disk: 41, host: 12, health: 19, temp: 12, size: 16 }, + compact: { disk: 34, host: 11, health: 17, life: 11, temp: 12, size: 15 }, basic: { disk: 33, host: 17, health: 22, temp: 12, size: 16 }, operational: { disk: 25, host: 12, parent: 15, health: 17, life: 9, temp: 9, size: 13 }, expanded: { disk: 22, host: 10, role: 9, parent: 14, health: 15, life: 8, temp: 8, size: 14 }, @@ -168,10 +174,31 @@ export const isPhysicalDiskColumnVisible = ( columnId: PhysicalDiskTableColumnId, ): boolean => PHYSICAL_DISK_VISIBLE_COLUMNS[layout].includes(columnId); -export const getPhysicalDiskColumnWidthPercent = ( +export const getPhysicalDiskColumnWidthStyle = ( layout: PhysicalDiskTableLayoutMode, columnId: PhysicalDiskTableColumnId, -): number => PHYSICAL_DISK_COLUMN_WIDTHS[layout][columnId] ?? 0; +): JSX.CSSProperties => + getPlatformTableWeightedColumnWidthStyle( + columnId, + PHYSICAL_DISK_COLUMN_WEIGHTS[layout], + PHYSICAL_DISK_VISIBLE_COLUMNS[layout], + ); + +// The shared table cell primitives pad every cell for desktop density. On the +// phone layouts that padding alone is a fifth of the row, so rendered cells +// and headers shed it the same way the Proxmox replication table does. +export const PHYSICAL_DISK_PHONE_CELL_PADDING_CLASS = 'px-1!'; + +export const getPhysicalDiskCellPaddingClass = (layout: PhysicalDiskTableLayoutMode): string => + layout === 'narrow' || layout === 'compact' ? PHYSICAL_DISK_PHONE_CELL_PADDING_CLASS : ''; + +// Health words that do not fit a phone-width health column fall back to a +// shorter form; the full label stays in the wider projections and the detail. +export const getPhysicalDiskHealthCompactLabel = (label: string): string => { + if (label === 'Needs Attention') return 'Attention'; + if (label === 'Replace Now') return 'Replace'; + return label; +}; export const PHYSICAL_DISK_COL_DISK_CLASS = ''; export const PHYSICAL_DISK_COL_DEVICE_CLASS = ''; @@ -221,8 +248,10 @@ export const PHYSICAL_DISK_MUTED_PLACEHOLDER_CLASS = 'text-[11px] text-muted'; export const PHYSICAL_DISK_LIFE_CLASS = 'text-[11px] font-medium'; export const PHYSICAL_DISK_HEALTH_WRAP_CLASS = 'flex min-w-0 items-center gap-1.5 whitespace-nowrap'; -export const PHYSICAL_DISK_HEALTH_LABEL_CLASS = 'min-w-0 truncate text-[11px] font-semibold'; -export const PHYSICAL_DISK_HEALTH_SUMMARY_CLASS = 'hidden xl:block truncate text-[11px] text-muted'; +// The health word is the row's verdict and always renders whole. Its reason +// lives in the cell title and in the disk drawer, not in a truncating sibling +// that would compete with it for the track. +export const PHYSICAL_DISK_HEALTH_LABEL_CLASS = 'shrink-0 text-[11px] font-semibold'; export const PHYSICAL_DISK_TEMPERATURE_CLASS = 'text-[11px] font-medium'; export const PHYSICAL_DISK_SIZE_VALUE_CLASS = 'text-[11px] text-base-content'; diff --git a/frontend-modern/src/features/storageBackups/resourceStoragePresentation.ts b/frontend-modern/src/features/storageBackups/resourceStoragePresentation.ts index 0c3b5d618a..9302b84d66 100644 --- a/frontend-modern/src/features/storageBackups/resourceStoragePresentation.ts +++ b/frontend-modern/src/features/storageBackups/resourceStoragePresentation.ts @@ -244,6 +244,30 @@ export const getResourceStorageTopologyLabel = ( case 'rbd': case 'cephfs': return 'Cluster Storage'; + // Proxmox storage plugin names are acronyms or compound words that + // title-casing mangles ("Lvmthin" for the default local-lvm, "Nfs"); use + // the names the Proxmox UI shows for them. + case 'nfs': + return 'NFS'; + case 'cifs': + case 'smb': + return 'SMB/CIFS'; + case 'lvm': + return 'LVM'; + case 'lvmthin': + case 'lvm-thin': + return 'LVM-Thin'; + case 'iscsi': + case 'iscsidirect': + return 'iSCSI'; + case 'zfs': + return 'ZFS'; + case 'btrfs': + return 'BTRFS'; + case 'glusterfs': + return 'GlusterFS'; + case 'esxi': + return 'ESXi'; default: return titleize(storageType) || titleize(resource.type) || 'Storage'; } diff --git a/frontend-modern/src/features/storageBackups/rowPresentation.ts b/frontend-modern/src/features/storageBackups/rowPresentation.ts index 10e4c68f7a..f9bacbab41 100644 --- a/frontend-modern/src/features/storageBackups/rowPresentation.ts +++ b/frontend-modern/src/features/storageBackups/rowPresentation.ts @@ -66,9 +66,32 @@ export function getStoragePoolStateTextClass(record: StorageRecord): string { return 'text-base-content'; } +// The rebuild summary the backend promotes into the protection label is a +// sentence ("ZFS pool tank is resilvering (45.2%)"). The protection column is +// a badge, so a ZFS scan reads as the activity word, derived from the pool's +// own scan state (the same source the backend keys its risk reason on); the +// sentence stays available as the cell title. Non-ZFS rebuilds keep the label +// their platform supplies. +export function getStoragePoolRebuildLabel(record: StorageRecord): string { + const pool = getStorageRecordZfsPool(record); + if (!pool) return ''; + const scan = `${pool.scanDetails?.function || ''} ${pool.scan || ''}`.toLowerCase(); + const percentage = pool.scanDetails?.percentage; + const progress = + typeof percentage === 'number' && Number.isFinite(percentage) && percentage > 0 + ? ` ${Math.round(percentage)}%` + : ''; + if (scan.includes('resilver')) return `Resilvering${progress}`; + if (scan.includes('scrub')) return `Scrubbing${progress}`; + return ''; +} + export function getCompactStoragePoolProtectionLabel(record: StorageRecord): string { const label = (record.protectionLabel || '').trim(); - if (record.rebuildInProgress || record.protectionReduced) { + if (record.rebuildInProgress) { + return getStoragePoolRebuildLabel(record) || label || '—'; + } + if (record.protectionReduced) { return label || '—'; } if (label && label.toLowerCase() !== 'healthy') { @@ -87,7 +110,9 @@ export function getStoragePoolStateLabel(record: StorageRecord): string { } const pool = getStorageRecordZfsPool(record); if (pool?.state) { - return pool.state === 'ONLINE' ? 'Online' : pool.state; + // zpool reports upper-case states; present them like every other state + // path so a DEGRADED pool reads the same as a degraded array. + return titleize(pool.state); } const status = getStorageRecordStatus(record); return status ? titleize(status) : '—'; @@ -122,7 +147,9 @@ export function getCompactStoragePoolProtectionTitle(record: StorageRecord): str return issueSummary; } } - return label; + // When the badge shows a derived word, the platform's full label is the title. + const fullLabel = (record.protectionLabel || '').trim(); + return fullLabel && fullLabel.toLowerCase() !== label.toLowerCase() ? fullLabel : label; } export function getCompactStoragePoolImpactLabel(record: StorageRecord): string { diff --git a/frontend-modern/src/features/storageBackups/storageAdapters.ts b/frontend-modern/src/features/storageBackups/storageAdapters.ts index 19b8ac6598..7e22fdd6d7 100644 --- a/frontend-modern/src/features/storageBackups/storageAdapters.ts +++ b/frontend-modern/src/features/storageBackups/storageAdapters.ts @@ -129,7 +129,16 @@ const mapResourceStorageRecord = (resource: Resource, adapterId: string): Storag const usedBytes = asNumberOrNull(resource.disk?.used); const freeBytes = asNumberOrNull(resource.disk?.free); const canonicalPlatform = getCanonicalStoragePlatformKey(resource, resource.storage?.platform); - const hostLabel = locationLabel; + // The poller folds a shared storage into one entry whose node is the + // synthetic "cluster" marker (the mock uses "shared"); show that as a + // shared storage with its reporting node count rather than a node name. + const sharedLocationMarkers = new Set(['cluster', 'shared']); + const hostLabel = + shared && sharedLocationMarkers.has(locationLabel.trim().toLowerCase()) + ? storageNodes.length > 1 + ? `Shared · ${storageNodes.length} nodes` + : 'Shared' + : locationLabel; const platformLabel = getResourceStoragePlatformLabel(canonicalPlatform); const issueLabel = getResourceStorageIssueLabel(resource); const issueSummary = getResourceStorageIssueSummary(resource); diff --git a/frontend-modern/src/features/storageBackups/storageBarPresentation.ts b/frontend-modern/src/features/storageBackups/storageBarPresentation.ts index 02b7fda274..533641e90a 100644 --- a/frontend-modern/src/features/storageBackups/storageBarPresentation.ts +++ b/frontend-modern/src/features/storageBackups/storageBarPresentation.ts @@ -38,8 +38,13 @@ export const getStorageBarUsagePercent = (used: number, total: number): number = return (used / total) * 100; }; +// Phone-width tables only have room for the percentage inside the bar; the +// used/total sublabel stays in the tooltip and in the wider projections. +export const getStorageBarCompactLabel = (used: number, total: number): string => + formatPercent(getStorageBarUsagePercent(used, total)); + export const getStorageBarLabel = (used: number, total: number): string => - `${formatPercent(getStorageBarUsagePercent(used, total))} (${formatBytes(used)}/${formatBytes(total)})`; + `${getStorageBarCompactLabel(used, total)} (${formatBytes(used)}/${formatBytes(total)})`; export const getStorageBarTooltipTitle = (): string => 'Storage Details'; diff --git a/frontend-modern/src/features/storageBackups/storagePagePresentation.ts b/frontend-modern/src/features/storageBackups/storagePagePresentation.ts index 87652660c1..8fe7d87546 100644 --- a/frontend-modern/src/features/storageBackups/storagePagePresentation.ts +++ b/frontend-modern/src/features/storageBackups/storagePagePresentation.ts @@ -1,5 +1,7 @@ +import type { JSX } from 'solid-js'; import type { StorageSortKey } from './storageModelCore'; import type { PlatformTableColumnKind } from '@/features/platformPage/columnAlignment'; +import { getPlatformTableWeightedColumnWidthStyle } from '@/features/platformPage/sharedPlatformPage'; export type StorageViewOption = { value: 'pools' | 'disks'; @@ -21,11 +23,14 @@ export type StoragePoolTableColumn = { export type StoragePoolTableLayoutMode = 'narrow' | 'compact' | 'operational' | 'full'; +// The compact/operational boundary matches the shared phone projection of the +// table container (`@container (max-width: 33.999rem)` = 544px), so the +// columns and the in-cell labels always change projection together. export const getStoragePoolTableLayoutModeForContainer = ( containerWidth: number, ): StoragePoolTableLayoutMode => { if (containerWidth >= 1_040) return 'full'; - if (containerWidth >= 560) return 'operational'; + if (containerWidth >= 544) return 'operational'; if (containerWidth > 0 && containerWidth < 360) return 'narrow'; return 'compact'; }; @@ -43,13 +48,20 @@ const STORAGE_POOL_VISIBLE_COLUMNS: Record< full: ['name', 'state', 'type', 'host', 'protection', 'usage', 'growth'], }; -const STORAGE_POOL_COLUMN_WIDTHS: Record< +// Relative weights per layout, resolved through the canonical weighted-width +// helper. On phones the usage bar carries only its percentage, so most of the +// row goes to the two identity strings (pool name and host); a state word such +// as "Degraded" must still fit whole. The shared 30% identity anchor is not +// used here because a four-column row cannot keep the host readable with it. +const STORAGE_POOL_COLUMN_WEIGHTS: Record< StoragePoolTableLayoutMode, Partial> > = { - narrow: { name: 40, state: 20, host: 20, usage: 20 }, - compact: { name: 35, state: 20, host: 20, usage: 25 }, - operational: { name: 29, state: 20, host: 15, protection: 15, usage: 21 }, + narrow: { name: 35, state: 21, host: 31, usage: 13 }, + compact: { name: 37, state: 18.5, host: 31.5, usage: 13 }, + // The full "59% (1.18 TB/2.00 TB)" bar label returns at 544px, so the usage + // track must hold ~120px of text from the first operational width. + operational: { name: 27, state: 18, host: 15, protection: 15, usage: 25 }, full: { name: 20, state: 14, type: 10, host: 12, protection: 13, usage: 20, growth: 11 }, }; @@ -58,10 +70,24 @@ export const isStoragePoolColumnVisible = ( columnId: StoragePoolTableColumnId, ): boolean => STORAGE_POOL_VISIBLE_COLUMNS[layout].includes(columnId); -export const getStoragePoolColumnWidthPercent = ( +// On phones the pool row sheds the desktop cell gutter the same way its +// headers (via the shared container query) and the physical-disk rows do: +// four cells at 12px each were 48px of a 361px row, and the measured values +// fit the tracks with only a pixel or two to spare. +export const STORAGE_POOL_PHONE_CELL_PADDING_CLASS = 'px-1!'; + +export const getStoragePoolCellPaddingClass = (layout: StoragePoolTableLayoutMode): string => + layout === 'narrow' || layout === 'compact' ? STORAGE_POOL_PHONE_CELL_PADDING_CLASS : ''; + +export const getStoragePoolColumnWidthStyle = ( layout: StoragePoolTableLayoutMode, columnId: StoragePoolTableColumnId, -): number => STORAGE_POOL_COLUMN_WIDTHS[layout][columnId] ?? 0; +): JSX.CSSProperties => + getPlatformTableWeightedColumnWidthStyle( + columnId, + STORAGE_POOL_COLUMN_WEIGHTS[layout], + STORAGE_POOL_VISIBLE_COLUMNS[layout], + ); const STORAGE_POOL_TABLE_HEADER_CLASS = 'overflow-hidden text-ellipsis whitespace-nowrap text-[10px] lg:text-xs uppercase tracking-wider'; diff --git a/frontend-modern/src/hooks/__tests__/useUnifiedResources.test.ts b/frontend-modern/src/hooks/__tests__/useUnifiedResources.test.ts index 3408c0079d..ef2dfae357 100644 --- a/frontend-modern/src/hooks/__tests__/useUnifiedResources.test.ts +++ b/frontend-modern/src/hooks/__tests__/useUnifiedResources.test.ts @@ -2,6 +2,7 @@ import { batch, createRoot, createSignal } from 'solid-js'; import { createStore, reconcile, type SetStoreFunction } from 'solid-js/store'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { buildProxmoxPageModel } from '@/features/proxmox/proxmoxPageModel'; +import { getDockerContainerLifecycleDisabledReason } from '@/features/docker/dockerContainerLifecycleActions'; import type { State } from '@/types/api'; import type { Resource } from '@/types/resource'; import useUnifiedResourcesSource from '../useUnifiedResources.ts?raw'; @@ -1626,6 +1627,173 @@ describe('useUnifiedResources', () => { dispose(); }); + it('keeps PMG metadata on the canonical resource for REST-first paints', async () => { + // The Mail Gateway table reads `resource.pmg`. The mapper used to keep it + // only under platformData, so a REST-sourced row rendered every PMG column + // as a dash until a websocket row happened to replace it. + apiFetchMock.mockResolvedValueOnce({ + ok: true, + json: async () => ({ + data: [ + { + id: 'pmg-1', + type: 'pmg', + name: 'mail-gateway-eu', + status: 'online', + lastSeen: '2026-02-06T12:00:00Z', + sources: ['pmg'], + uptime: 1_555_222, + technology: 'pmg', + sourceStatus: { pmg: { status: 'online', lastSeen: '2026-02-06T12:00:00Z' } }, + actionReadiness: [{ name: 'restart', available: true }], + pmg: { + instanceId: 'pmg-eu', + hostname: 'pmg.example.lan', + version: '8.1-2', + nodeCount: 1, + mailCountTotal: 2_740, + spamIn: 321, + virusIn: 14, + queueTotal: 11, + queueDeferred: 5, + }, + }, + ], + }), + }); + + let dispose = () => {}; + let result: ReturnType | undefined; + createRoot((d) => { + dispose = d; + result = useUnifiedResources({ query: 'type=pmg&source=pmg', cacheKey: 'pmg-first-paint' }); + }); + + await result!.refetch(); + const row = result!.resources().find((resource) => resource.id === 'pmg-1'); + expect(row?.pmg).toEqual( + expect.objectContaining({ version: '8.1-2', spamIn: 321, queueDeferred: 5 }), + ); + expect(row?.platformData?.pmg).toEqual(row?.pmg); + // The per-source status, action readiness, and technology facets ride the + // same race: Docker lifecycle actions and storage freshness read them at + // the top level, where only websocket rows used to carry them. + expect(row?.sourceStatus?.pmg?.status).toBe('online'); + expect(row?.actionReadiness?.[0]?.name).toBe('restart'); + expect(row?.technology).toBe('pmg'); + + dispose(); + }); + + it.each([ + { + label: 'stale inventory', + sourceStatus: { docker: { status: 'stale', lastSeen: '2026-02-06T12:00:00Z' } }, + actionReadiness: [], + expected: 'Docker inventory is stale. Refresh inventory before running lifecycle actions.', + }, + { + label: 'failed inventory', + sourceStatus: { + docker: { status: 'online', lastSeen: '2026-02-06T12:00:00Z', error: 'Collector refused' }, + }, + actionReadiness: [], + expected: 'Docker inventory is not healthy: Collector refused', + }, + { + label: 'server action refusal', + sourceStatus: { docker: { status: 'online', lastSeen: '2026-02-06T12:00:00Z' } }, + actionReadiness: [ + { name: 'restart', available: false, reason: 'Command agent disconnected' }, + ], + expected: 'Command agent disconnected', + }, + { + label: 'no invented action grant', + sourceStatus: { docker: { status: 'online', lastSeen: '2026-02-06T12:00:00Z' } }, + actionReadiness: [], + expected: + 'Pulse does not currently advertise a fresh restart command capability for this container.', + }, + ])('preserves $label through REST mapping into lifecycle controls', async (testCase) => { + apiFetchMock.mockResolvedValue( + resourceResponse([ + { + id: 'docker-container-1', + type: 'app-container', + name: 'edge-web', + status: 'running', + lastSeen: '2026-02-06T12:00:00Z', + sources: ['docker'], + technology: 'docker', + docker: { runtime: 'docker', agentId: 'agent-edge', containerState: 'running' }, + sourceStatus: testCase.sourceStatus, + actionReadiness: testCase.actionReadiness, + }, + ]), + ); + let dispose = () => {}; + let result: ReturnType | undefined; + createRoot((d) => { + dispose = d; + result = useUnifiedResources({ query: 'type=app-container', cacheKey: testCase.label }); + }); + try { + await result!.refetch(); + const row = result!.resources()[0]; + expect(row.sourceStatus).toEqual(testCase.sourceStatus); + expect(row.actionReadiness).toEqual(testCase.actionReadiness); + expect(row.technology).toBe('docker'); + expect(getDockerContainerLifecycleDisabledReason(row, 'restart')).toBe(testCase.expected); + } finally { + dispose(); + } + }); + + it('replaces REST facets without retaining a removed PMG counter or action refusal', async () => { + const base = { + id: 'pmg-1', + type: 'pmg', + name: 'mail-gateway', + status: 'online', + sources: ['pmg'], + lastSeen: '2026-02-06T12:00:00Z', + }; + apiFetchMock.mockResolvedValue( + resourceResponse([ + { + ...base, + technology: 'pmg', + pmg: { version: '8.1-2', queueTotal: 11 }, + sourceStatus: { pmg: { status: 'stale', lastSeen: base.lastSeen } }, + actionReadiness: [{ name: 'restart', available: false, reason: 'Old refusal' }], + }, + ]), + ); + let dispose = () => {}; + let result: ReturnType | undefined; + createRoot((d) => { + dispose = d; + result = useUnifiedResources({ query: 'type=pmg', cacheKey: 'removed-rest-facets' }); + }); + try { + await result!.refetch(); + expect(result!.resources()[0].pmg?.queueTotal).toBe(11); + apiFetchMock.mockResolvedValue( + resourceResponse([{ ...base, actionReadiness: [], sourceStatus: {} }]), + ); + await result!.refetch(); + const row = result!.resources()[0]; + expect(row.pmg).toBeUndefined(); + expect(row.platformData?.pmg).toBeUndefined(); + expect(row.actionReadiness).toEqual([]); + expect(row.sourceStatus).toEqual({}); + expect(row.technology).toBeUndefined(); + } finally { + dispose(); + } + }); + it('falls back to canonical resource.uptime when no platform-specific field is set (vSphere)', async () => { // vSphere ESXi hosts and VMs populate only the canonical // Resource.Uptime field — there's no vmware-specific UptimeSeconds diff --git a/frontend-modern/src/hooks/useUnifiedResources.ts b/frontend-modern/src/hooks/useUnifiedResources.ts index 9093092194..dbb5d9b3d8 100644 --- a/frontend-modern/src/hooks/useUnifiedResources.ts +++ b/frontend-modern/src/hooks/useUnifiedResources.ts @@ -18,8 +18,10 @@ import type { ResourceDockerMeta, ResourceHealth, ResourceMetricsTarget, + ResourceActionReadiness, ResourceAvailabilityMeta, ResourcePBSMeta, + ResourcePMGMeta, ResourcePolicyPostureSummary, ResourcePhysicalDiskMeta, ResourcePoolHealth, @@ -210,6 +212,8 @@ type APIResource = { sources?: string[]; platformScopes?: string[]; sourceStatus?: Record; + actionReadiness?: ResourceActionReadiness[]; + technology?: string; health?: ResourceHealth; identity?: { machineId?: string; @@ -868,6 +872,17 @@ const toResource = (v2: APIResource): Resource => { truenas: v2.truenas as ResourceTrueNASMeta | undefined, vmware: v2.vmware as ResourceVMwareMeta | undefined, pbs: v2.pbs as ResourcePBSMeta | undefined, + // Mail Gateway rows read `resource.pmg` directly. Leaving it only under + // platformData meant a REST-first paint rendered every PMG column as a + // dash until a websocket row happened to replace it, and never did when + // the REST snapshot landed last. + pmg: v2.pmg as ResourcePMGMeta | undefined, + // Same race for the per-source status, action readiness, and technology + // facets: Docker lifecycle actions and storage freshness read them from + // the top level, and the websocket rows carry them there. + sourceStatus: v2.sourceStatus, + actionReadiness: v2.actionReadiness, + technology: v2.technology, availability: v2.availability as ResourceAvailabilityMeta | undefined, availabilityChecks: v2.availabilityChecks as ResourceAvailabilityMeta[] | undefined, physicalDisk: v2.physicalDisk, diff --git a/frontend-modern/src/utils/__tests__/alertDestinationsPresentation.test.ts b/frontend-modern/src/utils/__tests__/alertDestinationsPresentation.test.ts index 839f7bc178..8cf26ffba5 100644 --- a/frontend-modern/src/utils/__tests__/alertDestinationsPresentation.test.ts +++ b/frontend-modern/src/utils/__tests__/alertDestinationsPresentation.test.ts @@ -35,6 +35,9 @@ import { getAlertDestinationsDeliveryLogEmpty, getAlertDestinationsDeliveryLogTitle, getAlertDestinationsDeliveryLogUnavailable, + getAlertDestinationsDeliveryLogLoading, + getAlertDestinationsHeldEventsLoading, + getAlertDestinationsHeldEventsUnavailable, getAlertDestinationsTestPausedWarning, getAlertDestinationsAppriseTargetsHelp, getAlertDestinationsAppriseTestLabel, @@ -267,6 +270,28 @@ describe('alert destinations delivery log copy', () => { ); }); + it('distinguishes pending and unavailable activity sources without implying delivery failure', () => { + expect(getAlertDestinationsDeliveryLogLoading()).toBe('Loading delivery attempts...'); + expect(getAlertDestinationsHeldEventsLoading()).toBe( + 'Loading held and deferred notifications...', + ); + expect(getAlertDestinationsHeldEventsUnavailable()).toBe( + 'Pulse could not read held or deferred notifications. Refresh to try again.', + ); + expect(getAlertDestinationsDeliveryLogUnavailable()).toBe( + 'Pulse could not read the delivery log. Refresh to check delivery attempts.', + ); + for (const message of [ + getAlertDestinationsDeliveryLogLoading(), + getAlertDestinationsHeldEventsLoading(), + getAlertDestinationsHeldEventsUnavailable(), + getAlertDestinationsDeliveryLogUnavailable(), + ]) { + expect(message).not.toContain('No alert deliveries'); + expect(message).not.toContain('not delivered'); + } + }); + it('warns that a passing test does not mean live alerts flow while delivery is paused', () => { const warning = getAlertDestinationsTestPausedWarning(); expect(warning).toContain('Test sent'); diff --git a/frontend-modern/src/utils/__tests__/workloadGuestPresentation.test.ts b/frontend-modern/src/utils/__tests__/workloadGuestPresentation.test.ts index 40445442f9..e1194c4f5e 100644 --- a/frontend-modern/src/utils/__tests__/workloadGuestPresentation.test.ts +++ b/frontend-modern/src/utils/__tests__/workloadGuestPresentation.test.ts @@ -1,4 +1,5 @@ import { describe, expect, it } from 'vitest'; +import { guestDiskDeferrals } from '@/components/Workloads/__fixtures__/guestDiskDeferrals'; import { getWorkloadsGuestBackupStatusPresentation, getWorkloadsGuestBackupTooltip, @@ -8,6 +9,18 @@ import { } from '@/utils/workloadGuestPresentation'; describe('workloadGuestPresentation', () => { + it.each(guestDiskDeferrals)( + 'explains %s with and without retained disk evidence', + (reason, message) => { + expect(getWorkloadGuestDiskStatusMessage(reason)).toBe(message); + expect(getWorkloadGuestDiskStatusMessage(`prev-${reason}`)).toBe( + `Using last known disk stats. ${message}`, + ); + expect(message).not.toContain('may not be installed'); + expect(message).not.toContain('may need to be restarted'); + }, + ); + it('returns canonical guest backup status presentation', () => { expect(getWorkloadsGuestBackupStatusPresentation('fresh')).toEqual({ color: 'text-green-600 dark:text-green-400', diff --git a/frontend-modern/src/utils/alertDestinationsPresentation.ts b/frontend-modern/src/utils/alertDestinationsPresentation.ts index 92ad797dc7..1fc3454103 100644 --- a/frontend-modern/src/utils/alertDestinationsPresentation.ts +++ b/frontend-modern/src/utils/alertDestinationsPresentation.ts @@ -277,7 +277,19 @@ export const ALERT_DESTINATIONS_DELIVERY_LOG_TITLE = 'Recent delivery activity'; export const ALERT_DESTINATIONS_DELIVERY_LOG_EMPTY = 'No alert deliveries were attempted in this window.'; export const ALERT_DESTINATIONS_DELIVERY_LOG_UNAVAILABLE = - 'Pulse could not read the delivery log, so recent delivery activity cannot be shown.'; + 'Pulse could not read the delivery log. Refresh to check delivery attempts.'; + +export function getAlertDestinationsHeldEventsUnavailable() { + return 'Pulse could not read held or deferred notifications. Refresh to try again.'; +} + +export function getAlertDestinationsDeliveryLogLoading() { + return 'Loading delivery attempts...'; +} + +export function getAlertDestinationsHeldEventsLoading() { + return 'Loading held and deferred notifications...'; +} export function getAlertDestinationsDeliveryLogTitle() { return ALERT_DESTINATIONS_DELIVERY_LOG_TITLE; diff --git a/frontend-modern/src/utils/workloadGuestPresentation.ts b/frontend-modern/src/utils/workloadGuestPresentation.ts index 09d3b9cd4a..00d1fe68cf 100644 --- a/frontend-modern/src/utils/workloadGuestPresentation.ts +++ b/frontend-modern/src/utils/workloadGuestPresentation.ts @@ -103,7 +103,21 @@ export function getWorkloadGuestDiskStatusMessage(reason?: string): string { case 'agent-not-running': return 'Guest agent not running. Install and start qemu-guest-agent in the VM.'; case 'agent-timeout': - return 'Guest agent timeout. Agent may need to be restarted.'; + return 'Guest request timed out. Completion is uncertain. Do not restart the guest agent during a backup.'; + case 'vm-locked': + return 'Guest reads paused while Proxmox reports a VM operation lock, such as a backup. Pulse will check again on a later poll.'; + case 'lock-unverified': + return 'Guest reads deferred because Pulse cannot verify that the VM is unlocked. Pulse will check again on a later poll.'; + case 'agent-busy': + return 'Guest reads deferred while an earlier guest request is still in progress. Pulse will check again on a later poll.'; + case 'agent-cooldown': + return 'Guest reads paused after an earlier request did not complete reliably. Pulse will check again after the cooldown.'; + case 'agent-response-incomplete': + return 'Guest reads paused because the previous response was incomplete. Pulse will check again on a later poll.'; + case 'agent-capacity': + return 'Guest reads deferred because Pulse has reached its guest-read capacity. Pulse will check again on a later poll.'; + case 'invalid-guest-key': + return 'Guest reads unavailable because the VM identity is invalid.'; case 'permission-denied': return 'Permission denied. Check that your Pulse user/token has VM.Monitor permission (PVE 8) or VM.GuestAgent.Audit permission (PVE 9).'; case 'agent-disabled': @@ -116,6 +130,10 @@ export function getWorkloadGuestDiskStatusMessage(reason?: string): string { return 'Error communicating with guest agent.'; case 'no-data': return 'No disk data available from Proxmox API.'; + case 'vm-stopped': + return 'Guest filesystem stats unavailable while the VM is stopped.'; + case 'no-status': + return 'Guest filesystem stats unavailable because Pulse could not read the VM status from Proxmox.'; default: return 'Disk stats unavailable. Guest agent may not be installed.'; } diff --git a/internal/monitoring/canonical_guardrails_test.go b/internal/monitoring/canonical_guardrails_test.go index abb02e0309..30ab89b6c5 100644 --- a/internal/monitoring/canonical_guardrails_test.go +++ b/internal/monitoring/canonical_guardrails_test.go @@ -1913,7 +1913,7 @@ func TestProxmoxGuestAgentContinuityUsesCanonicalEvidenceAndRetryPaths(t *testin "monitor_pve_guest_builders.go": { "guestAgentAvailable := shouldQueryGuestAgent(state.detailedStatus, prevVM, now) ||", "m.hasRecentGuestMetadataEvidence(instanceName, res.Node, res.VMID, now)", - "if guestAgentAvailable && state.detailedStatus == nil {", + `if guestAgentAvailable && res.Lock == "" && state.detailedStatus == nil {`, }, "monitor_polling_vm.go": { "prevVMByID := prevGuests.vmsByID", @@ -2969,3 +2969,8 @@ func TestBroadcastProjectionMatchesPreviousPipeline(t *testing.T) { t.Fatal("broadcast decoration mutated registry") } } + +// The runtime contract includes both poll builders, protocol admission and truthful History. +func TestGuestAgentBackupMonitoringContract(t *testing.T) { + testGuestAgentBackupMonitoringLifecycle(t) +} diff --git a/internal/monitoring/guest_agent_backup_integration_test.go b/internal/monitoring/guest_agent_backup_integration_test.go new file mode 100644 index 0000000000..3afe1c8a27 --- /dev/null +++ b/internal/monitoring/guest_agent_backup_integration_test.go @@ -0,0 +1,190 @@ +package monitoring + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "reflect" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/rcourtman/pulse-go-rewrite/internal/alerts" + "github.com/rcourtman/pulse-go-rewrite/internal/config" + "github.com/rcourtman/pulse-go-rewrite/internal/models" + "github.com/rcourtman/pulse-go-rewrite/pkg/proxmox" +) + +func testGuestAgentBackupMonitoringLifecycle(t *testing.T) { + const mib = uint64(1024 * 1024) + var phase, calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + p := phase.Load() + w.Header().Set("Content-Type", "application/json") + switch { + case strings.HasSuffix(r.URL.Path, "/config"): + if p == 1 || p == 2 { + fmt.Fprint(w, `{"data":{"lock":"backup"}}`) + } else { + fmt.Fprint(w, `{"data":{}}`) + } + case strings.HasSuffix(r.URL.Path, "/status/current"): + if p == 2 { + http.Error(w, "unavailable", 503) + return + } + lock := "" + if p == 1 { + lock = `,"lock":"backup"` + } + fmt.Fprintf(w, `{"data":{"status":"running","agent":1,"maxmem":%d,"mem":%d%s}}`, 8*mib, 8*mib, lock) + case strings.Contains(r.URL.Path, "/agent/"): + calls.Add(1) + switch { + case strings.HasSuffix(r.URL.Path, "file-read"): + available := 5 * 1024 + if p == 3 { + available = 4 * 1024 + } + json.NewEncoder(w).Encode(map[string]any{"data": map[string]any{"content": fmt.Sprintf("MemTotal: 8192 kB\nMemAvailable: %d kB\n", available)}}) + case strings.HasSuffix(r.URL.Path, "get-fsinfo"): + used := 300 * mib + if p == 3 { + used = 400 * mib + } + fmt.Fprintf(w, `{"data":{"result":[{"mountpoint":"/","type":"ext4","total-bytes":%d,"used-bytes":%d}]}}`, 1000*mib, used) + case strings.HasSuffix(r.URL.Path, "network-get-interfaces"): + fmt.Fprint(w, `{"data":{"result":[{"name":"eth0","hardware-address":"02:00:00:00:00:01","ip-addresses":[{"ip-address":"192.0.2.10","ip-address-type":"ipv4","prefix":24}]}]}}`) + case strings.HasSuffix(r.URL.Path, "get-osinfo"): + fmt.Fprint(w, `{"data":{"name":"Linux","version":"fixture"}}`) + default: + fmt.Fprint(w, `{"data":{"result":{"version":"1.0"}}}`) + } + default: + http.NotFound(w, r) + } + })) + defer server.Close() + client, err := proxmox.NewClient(proxmox.ClientConfig{Host: server.URL, TokenName: "fixture@pve!pulse", TokenValue: "fixture", Timeout: time.Second}) + if err != nil { + t.Fatal(err) + } + m := &Monitor{config: &config.Config{}, rateTracker: NewRateTracker(), metricsHistory: NewMetricsHistory(32, time.Hour), guestMetadataLimiter: make(map[string]time.Time)} + res := proxmox.ClusterResource{Type: "qemu", Node: "node", Name: "guest", VMID: 105, Status: "running", MaxMem: 8 * mib, Mem: 8 * mib, MaxDisk: 1000 * mib, CPU: 0.1} + var previous *models.VM + identity := "fixture:node:105" + build := func() (models.VM, string) { + vm, raw, source, notes, at, ok := m.buildVMFromClusterResource(context.Background(), "fixture", res, client, identity, nil, previous) + if !ok { + t.Fatal("VM disappeared") + } + m.recordGuestSnapshot("fixture", "qemu", "node", 105, GuestMemorySnapshot{Name: vm.Name, Status: vm.Status, RetrievedAt: at, MemorySource: source, Memory: vm.Memory, Raw: raw, Notes: notes}) + m.recordGuestMetrics([]models.VM{vm}, nil, time.Now().Add(-time.Second)) + previous = &vm + return vm, source + } + initial, source := build() + if source != "guest-agent-meminfo" || initial.Memory.Used != int64(3*mib) || initial.Disk.Used != int64(300*mib) { + t.Fatalf("healthy readings absent: %s %#v %#v", source, initial.Memory, initial.Disk) + } + before := calls.Load() + if before != 5 { + t.Fatalf("healthy agent command count = %d, want five", before) + } + metadataKey := guestMetadataCacheKey("fixture", "node", 105) + memKey := guestMemoryCacheKey("fixture", "node", 105) + originalMetadata := m.guestMetadataCache[metadataKey] + originalMemory := m.vmAgentMemCache[memKey] + phase.Store(1) + res.CPU = 0.2 + locked, source := build() + if locked.ID != initial.ID || locked.GuestAgentStatus != "deferred" || locked.DiskStatusReason != "prev-vm-locked" { + t.Fatalf("lock lost continuity/truth: %#v", locked) + } + if locked.Memory.Used != initial.Memory.Used || source != "previous-snapshot" || locked.Disk.Used != initial.Disk.Used || !reflect.DeepEqual(locked.NetworkInterfaces, initial.NetworkInterfaces) { + t.Fatalf("last known evidence lost: %s %#v", source, locked) + } + if calls.Load() != before { + t.Fatalf("status-lock sent agent commands: %d -> %d", before, calls.Load()) + } + if !reflect.DeepEqual(m.guestMetadataCache[metadataKey], originalMetadata) || !reflect.DeepEqual(m.vmAgentMemCache[memKey], originalMemory) { + t.Fatal("lock renewed/replaced cached observations") + } + for metric, want := range map[string]int{"cpu": 2, "memory": 1, "memoryused": 1, "disk": 1} { + if got := len(m.metricsHistory.GetGuestMetrics(identity, metric, time.Hour)); got != want { + t.Errorf("%s history points = %d, want %d", metric, got, want) + } + } + // Cluster lock must survive a failed status call and the recent-evidence fallback. + phase.Store(2) + if err := json.Unmarshal([]byte(`{"type":"qemu","node":"node","name":"guest","vmid":105,"status":"running","lock":"backup","maxmem":8388608,"mem":8388608,"maxdisk":1048576000}`), &res); err != nil { + t.Fatal(err) + } + locked, _ = build() + if calls.Load() != before || locked.GuestAgentStatus != "deferred" || locked.AgentVersion != initial.AgentVersion { + t.Fatalf("resource-lock fallback queried/lost identity: calls=%d %#v", calls.Load(), locked) + } + // The per-node inventory fallback must propagate its lock to the same builder. + manager := alerts.NewManagerWithDataDir(t.TempDir()) + defer manager.Stop() + m.alertManager = manager + nodeVMs, _ := m.pollNodeVMsWithClusterResourceBuilder(context.Background(), "fixture", "node", []proxmox.VM{{VMID: 105, Name: "guest", Status: "running", Lock: "backup", MaxMem: 8 * mib, Mem: 8 * mib, MaxDisk: 1000 * mib}}, client, map[string]models.VM{makeGuestID("fixture", "node", 105): locked}, nil) + if len(nodeVMs) != 1 || nodeVMs[0].GuestAgentStatus != "deferred" || calls.Load() != before { + t.Fatalf("node fallback lost lock or sent guest commands: %#v calls=%d", nodeVMs, calls.Load()) + } + // Resume from expired, unrenewed caches; new data must replace the retained view. + phase.Store(3) + res.Lock = "" + cached := m.vmAgentMemCache[memKey] + cached.fetchedAt = time.Now().Add(-2 * vmAgentMemCacheTTL) + m.vmAgentMemCache[memKey] = cached + metadata := m.guestMetadataCache[metadataKey] + metadata.fetchedAt = time.Now().Add(-2 * guestMetadataCacheTTL) + m.guestMetadataCache[metadataKey] = metadata + m.guestMetadataLimiter = make(map[string]time.Time) + resumed, source := build() + if resumed.ID != initial.ID || resumed.GuestAgentStatus != "available" || resumed.DiskStatusReason != "" || resumed.Disk.Used != int64(400*mib) || resumed.Memory.Used != int64(4*mib) || source != "guest-agent-meminfo" { + t.Fatalf("healthy resumption failed: %s %#v", source, resumed) + } + if calls.Load() != before+5 { + t.Errorf("fresh resume commands = %d, want %d", calls.Load(), before+5) + } + if got := len(m.metricsHistory.GetGuestMetrics(identity, "disk", time.Hour)); got != 2 { + t.Errorf("cached disk was written as new telemetry: %d points", got) + } +} + +func TestGuestAgentDeferralPreservesMetadataAndMemoryCache(t *testing.T) { + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } + calls.Add(1) + http.Error(w, "QEMU command timed out", 500) + })) + defer server.Close() + client, err := proxmox.NewClient(proxmox.ClientConfig{Host: server.URL, TokenName: "fixture@pve!pulse", TokenValue: "fixture", Timeout: time.Second}) + if err != nil { + t.Fatal(err) + } + now := time.Now().Add(-10 * time.Minute) + metadata := guestMetadataCacheEntry{ipAddresses: []string{"192.0.2.10"}, osName: "Linux", agentVersion: "1.0", fetchedAt: now, osInfoFailureCount: 2} + memory := agentMemCacheEntry{available: 1024, fetchedAt: now, info: proxmox.LinuxMemoryAvailability{Source: "meminfo-available", EffectiveAvailable: 1024}} + mk, gk := guestMemoryCacheKey("fixture", "node", 105), guestMetadataCacheKey("fixture", "node", 105) + m := &Monitor{guestMetadataCache: map[string]guestMetadataCacheEntry{gk: metadata}, vmAgentMemCache: map[string]agentMemCacheEntry{mk: memory}, guestMetadataLimiter: make(map[string]time.Time)} + ips, _, os, _, version := m.fetchGuestAgentMetadata(context.Background(), client, "fixture", "node", "guest", 105, &proxmox.VMStatus{Agent: proxmox.VMAgentField{Value: 1}}, false) + if len(ips) != 1 || os != "Linux" || version != "1.0" { + t.Fatal("deferral lost last known metadata") + } + if _, err := m.getVMAgentMemoryAvailability(context.Background(), client, "fixture", "node", 105); proxmox.GuestAgentDeferredReason(err) != "agent-cooldown" { + t.Fatalf("memory retried deferred VM: %v", err) + } + if calls.Load() != 1 || !reflect.DeepEqual(m.guestMetadataCache[gk], metadata) || !reflect.DeepEqual(m.vmAgentMemCache[mk], memory) { + t.Fatal("deferral queued work, renewed cache or poisoned supported-OS evidence") + } +} diff --git a/internal/monitoring/guest_agent_evidence.go b/internal/monitoring/guest_agent_evidence.go index caf19fd1ce..001c33d176 100644 --- a/internal/monitoring/guest_agent_evidence.go +++ b/internal/monitoring/guest_agent_evidence.go @@ -39,7 +39,7 @@ func hasRecentGuestAgentEvidence(prev *models.VM, now time.Time) bool { func shouldQueryGuestAgent(vmStatus *proxmox.VMStatus, prev *models.VM, now time.Time) bool { if vmStatus != nil { - return vmStatus.Agent.IsAvailable() + return vmStatus.Lock == "" && vmStatus.Agent.IsAvailable() } return hasRecentGuestAgentEvidence(prev, now) } diff --git a/internal/monitoring/guest_disk_stability.go b/internal/monitoring/guest_disk_stability.go index 87cb04f5b0..53a99390d5 100644 --- a/internal/monitoring/guest_disk_stability.go +++ b/internal/monitoring/guest_disk_stability.go @@ -4,6 +4,8 @@ import ( "strings" "time" + "github.com/rcourtman/pulse-go-rewrite/pkg/proxmox" + "github.com/rcourtman/pulse-go-rewrite/internal/models" ) @@ -19,6 +21,10 @@ func classifyGuestAgentDiskStatusError(err error) string { return "" } + if reason := proxmox.GuestAgentDeferredReason(err); reason != "" { + return reason + } + errStr := err.Error() errStrLower := strings.ToLower(errStr) diff --git a/internal/monitoring/guest_memory_agent.go b/internal/monitoring/guest_memory_agent.go index 8b29b23279..3439899636 100644 --- a/internal/monitoring/guest_memory_agent.go +++ b/internal/monitoring/guest_memory_agent.go @@ -2,6 +2,7 @@ package monitoring import ( "context" + "errors" "fmt" "strings" "time" @@ -93,6 +94,12 @@ func (m *Monitor) getVMAgentMemoryAvailability(ctx context.Context, client PVECl return proxmox.LinuxMemoryAvailability{}, fmt.Errorf("guest agent meminfo fallback unsupported") } + if errors.Is(err, proxmox.ErrGuestAgentDeferred) { + // A coordination pause is not a failed guest observation. Preserve the + // last successful cache and its original timestamp for normal resumption. + return proxmox.LinuxMemoryAvailability{}, err + } + m.rrdCacheMu.Lock() defer m.rrdCacheMu.Unlock() if m.vmAgentMemCache == nil { diff --git a/internal/monitoring/guest_memory_sources.go b/internal/monitoring/guest_memory_sources.go index d26d7c48f0..709038169c 100644 --- a/internal/monitoring/guest_memory_sources.go +++ b/internal/monitoring/guest_memory_sources.go @@ -173,7 +173,7 @@ func selectGuestLowTrustUsedMemory(memTotal uint64, status *proxmox.VMStatus) (u } func shouldPreferGuestAgentMemAvailable(status *proxmox.VMStatus, memTotal uint64) bool { - if status == nil || !status.Agent.IsAvailable() || status.Mem == 0 { + if status == nil || status.Lock != "" || !status.Agent.IsAvailable() || status.Mem == 0 { return false } if guestStatusFreeMem(status) > 0 { @@ -311,7 +311,7 @@ func (m *Monitor) resolveGuestStatusMemory( // only in node RRD, so a guest RRD lookup can never yield memory evidence // (#1634). - if !hasMemAvailable && !hasSelectedUsed && status.Agent.IsAvailable() && !triedGuestAgentMemAvailable { + if !hasMemAvailable && !hasSelectedUsed && status.Lock == "" && status.Agent.IsAvailable() && !triedGuestAgentMemAvailable { if agentAvailable, agentSource, ok := m.tryGuestAgentMemAvailable(ctx, client, instanceName, guestName, node, vmid, memTotal, guestRaw); ok { memAvailable = agentAvailable hasMemAvailable = true diff --git a/internal/monitoring/guest_metadata.go b/internal/monitoring/guest_metadata.go index ec955ed3cc..ce22b3636c 100644 --- a/internal/monitoring/guest_metadata.go +++ b/internal/monitoring/guest_metadata.go @@ -3,6 +3,7 @@ package monitoring import ( "context" "encoding/json" + "errors" "fmt" "math" "sort" @@ -206,7 +207,7 @@ func (m *Monitor) retryGuestAgentCall(ctx context.Context, timeout time.Duration lastErr = err // Don't retry non-timeout errors or if this was the last attempt - if attempt >= maxRetries || !strings.Contains(err.Error(), "timeout") { + if errors.Is(err, proxmox.ErrGuestAgentDeferred) || attempt >= maxRetries || !strings.Contains(err.Error(), "timeout") { break } @@ -229,6 +230,11 @@ func (m *Monitor) fetchGuestAgentMetadata(ctx context.Context, client PVEClientI cached, ok := m.guestMetadataCache[key] m.guestMetadataMu.RUnlock() + if vmStatus != nil && vmStatus.Lock != "" { + // Deliberately retained identity, not a newly fetched observation. + return cloneStringSlice(cached.ipAddresses), cloneGuestNetworkInterfaces(cached.networkInterfaces), cached.osName, cached.osVersion, cached.agentVersion + } + agentAvailable := client != nil && ((vmStatus != nil && vmStatus.Agent.IsAvailable()) || allowWithoutStatus) if !agentAvailable { if ok && now.Sub(cached.fetchedAt) < guestMetadataCacheEntryTTL(cached) { @@ -274,6 +280,10 @@ func (m *Monitor) fetchGuestAgentMetadata(ctx context.Context, client PVEClientI interfaces, err := m.retryGuestAgentCall(ctx, m.guestAgentNetworkTimeout, m.guestAgentRetries, func(ctx context.Context) (interface{}, error) { return client.GetVMNetworkInterfaces(ctx, nodeName, vmid) }) + if errors.Is(err, proxmox.ErrGuestAgentDeferred) { + m.deferGuestMetadataRetry(key, time.Now()) + return cloneStringSlice(cached.ipAddresses), cloneGuestNetworkInterfaces(cached.networkInterfaces), cached.osName, cached.osVersion, cached.agentVersion + } if err != nil { log.Debug(). Str("instance", instanceName). @@ -311,6 +321,10 @@ func (m *Monitor) fetchGuestAgentMetadata(ctx context.Context, client PVEClientI agentInfoRaw, err := m.retryGuestAgentCall(ctx, m.guestAgentOSInfoTimeout, m.guestAgentRetries, func(ctx context.Context) (interface{}, error) { return client.GetVMAgentInfo(ctx, nodeName, vmid) }) + if errors.Is(err, proxmox.ErrGuestAgentDeferred) { + m.deferGuestMetadataRetry(key, time.Now()) + return cloneStringSlice(cached.ipAddresses), cloneGuestNetworkInterfaces(cached.networkInterfaces), cached.osName, cached.osVersion, cached.agentVersion + } if err != nil { if isGuestAgentOSInfoUnsupportedError(err) { osInfoSkip = true @@ -365,6 +379,10 @@ func (m *Monitor) fetchGuestAgentMetadata(ctx context.Context, client PVEClientI versionRaw, err := m.retryGuestAgentCall(ctx, m.guestAgentVersionTimeout, m.guestAgentRetries, func(ctx context.Context) (interface{}, error) { return client.GetVMAgentVersion(ctx, nodeName, vmid) }) + if errors.Is(err, proxmox.ErrGuestAgentDeferred) { + m.deferGuestMetadataRetry(key, time.Now()) + return cloneStringSlice(cached.ipAddresses), cloneGuestNetworkInterfaces(cached.networkInterfaces), cached.osName, cached.osVersion, cached.agentVersion + } if err != nil { log.Debug(). Str("instance", instanceName). diff --git a/internal/monitoring/memory_trust_characterization_test.go b/internal/monitoring/memory_trust_characterization_test.go index 2d12f5091a..60c583c3a1 100644 --- a/internal/monitoring/memory_trust_characterization_test.go +++ b/internal/monitoring/memory_trust_characterization_test.go @@ -2,6 +2,8 @@ package monitoring import ( "context" + "fmt" + "strings" "testing" "time" @@ -1171,3 +1173,30 @@ func TestGracePeriodGuestContributesNoMemorySample(t *testing.T) { t.Fatalf("grace-period guest recorded %d memory sample(s): %+v", len(got), got) } } + +func TestBackupLockMemoryTrustKeepsNativeEvidenceWithoutAgentReads(t *testing.T) { + const gib = uint64(1024 * 1024 * 1024) + for _, native := range []bool{false, true} { + t.Run(fmt.Sprintf("native-%t", native), func(t *testing.T) { + m := &Monitor{} + client := &vmMemoryTrustStubClient{stubPVEClient: &stubPVEClient{}, vmAgentMemAvailable: 5 * gib} + status := &proxmox.VMStatus{Status: "running", Agent: proxmox.VMAgentField{Value: 1}, Lock: "backup", MaxMem: 8 * gib, Mem: 8 * gib} + if native { + status.MemInfo = &proxmox.VMMemInfo{Total: 8 * gib, Available: 5 * gib} + } + _, used, source := m.resolveGuestStatusMemory(context.Background(), client, "fixture", "guest", "node", 105, "fixture:node:105", status, nil, 8*gib, "", &VMMemoryRaw{}) + if client.vmAgentMemCalls != 0 || strings.HasPrefix(source, "guest-agent-meminfo") { + t.Fatalf("backup used fresh/cached guest-agent evidence: calls=%d source=%s", client.vmAgentMemCalls, source) + } + if native && (used != 3*gib || source != "available-field") { + t.Fatalf("current native memory lost: %d %s", used, source) + } + status.Lock = "" + status.MemInfo = nil + _, _, source = m.resolveGuestStatusMemory(context.Background(), client, "fixture", "guest", "node", 105, "fixture:node:105", status, nil, 8*gib, "", &VMMemoryRaw{}) + if client.vmAgentMemCalls != 1 || source != "guest-agent-meminfo" { + t.Fatalf("unlocked memory did not resume: %d %s", client.vmAgentMemCalls, source) + } + }) + } +} diff --git a/internal/monitoring/monitor_pve_guest_builders.go b/internal/monitoring/monitor_pve_guest_builders.go index af16aa7dbc..3f1423fa2c 100644 --- a/internal/monitoring/monitor_pve_guest_builders.go +++ b/internal/monitoring/monitor_pve_guest_builders.go @@ -58,6 +58,12 @@ func (m *Monitor) applyVMStatusDetails( return } + if res.Lock != "" { + copy := *status + copy.Lock = res.Lock + status = © + } + res.Lock = status.Lock state.detailedStatus = status state.guestAgentStatus, state.guestAgentExpected = vmGuestAgentRuntimeState(status, recentGuestAgentEvidence) state.memTotal, state.memUsed, state.memorySource = m.resolveGuestStatusMemory( @@ -111,6 +117,9 @@ func (m *Monitor) applyVMStatusDetails( if len(fsDisks) > 0 { state.individualDisks = fsDisks } + if guestAgentDiskDeferred(state.diskStatusReason) { + state.guestAgentStatus = "deferred" + } } else { // Agent disabled - show allocated disk size if state.diskTotal > 0 { @@ -172,6 +181,9 @@ func vmGuestAgentRuntimeState(status *proxmox.VMStatus, recentGuestAgentEvidence if status == nil { return "", false } + if status.Lock != "" { + return "deferred", status.Agent.IsEnabled() + } if status.Agent.IsAvailable() { return "available", true } @@ -265,10 +277,13 @@ func (m *Monitor) buildVMFromClusterResource( Msg("Could not get VM status, using cluster/resources disk data") } + if state.detailedStatus != nil && state.detailedStatus.Lock != "" { + res.Lock = state.detailedStatus.Lock + } now := time.Now() guestAgentAvailable := shouldQueryGuestAgent(state.detailedStatus, prevVM, now) || m.hasRecentGuestMetadataEvidence(instanceName, res.Node, res.VMID, now) - if guestAgentAvailable && state.detailedStatus == nil { + if guestAgentAvailable && res.Lock == "" && state.detailedStatus == nil { guestIPs, guestIfaces, guestOSName, guestOSVersion, guestAgentVersion := m.fetchGuestAgentMetadata( ctx, client, @@ -308,12 +323,31 @@ func (m *Monitor) buildVMFromClusterResource( if len(fsDisks) > 0 { state.individualDisks = fsDisks } + if guestAgentDiskDeferred(state.diskStatusReason) { + state.guestAgentStatus = "deferred" + } state.guestAgentExpected = true - if len(guestIPs) > 0 || len(guestIfaces) > 0 || guestOSName != "" || guestOSVersion != "" || guestAgentVersion != "" || state.diskFromAgent { + if state.guestAgentStatus != "deferred" && (len(guestIPs) > 0 || len(guestIfaces) > 0 || guestOSName != "" || guestOSVersion != "" || guestAgentVersion != "" || state.diskFromAgent) { state.guestAgentStatus = "available" } } + if res.Lock != "" { + state.guestAgentStatus = "deferred" + state.diskStatusReason = "vm-locked" + state.diskFromAgent = false + state.diskUsage = -1 + if guestMemorySourceReliability(state.memorySource) == guestMemoryReliabilityLow { + state.memorySource = "unavailable" + state.memUsed = 0 + } + if prevVM != nil { + state.ipAddresses = cloneStringSlice(prevVM.IPAddresses) + state.networkInterfaces = cloneGuestNetworkInterfaces(prevVM.NetworkInterfaces) + state.osName, state.osVersion, state.agentVersion = prevVM.OSName, prevVM.OSVersion, prevVM.AgentVersion + } + } + var preferred bool state.diskTotal, state.diskUsed, state.diskFree, state.diskUsage, state.individualDisks, state.diskStatusReason, preferred = preferLinkedHostAgentDiskInventory( guestID, @@ -449,6 +483,7 @@ func (m *Monitor) buildVMFromClusterResource( Disks: state.individualDisks, DiskStatusReason: state.diskStatusReason, GuestAgentStatus: state.guestAgentStatus, + Lock: res.Lock, GuestAgentExpected: state.guestAgentExpected, IPAddresses: state.ipAddresses, OSName: state.osName, @@ -531,6 +566,9 @@ type vmFSInfoSummary struct { } func (m *Monitor) fetchVMFSInfo(ctx context.Context, instanceName string, res proxmox.ClusterResource, client PVEClientInterface) ([]proxmox.VMFileSystem, string, bool) { + if res.Lock != "" { + return nil, "vm-locked", false + } // Use retry logic for guest agent calls to handle transient timeouts (refs #630) fsInfoRaw, err := m.retryGuestAgentCall(ctx, m.guestAgentFSInfoTimeout, m.guestAgentRetries, func(ctx context.Context) (interface{}, error) { return client.GetVMFSInfo(ctx, res.Node, res.VMID) @@ -542,6 +580,9 @@ func (m *Monitor) fetchVMFSInfo(ctx context.Context, instanceName string, res pr } } if err != nil { + if reason := proxmox.GuestAgentDeferredReason(err); reason != "" { + return nil, reason, false + } // Log more helpful error messages based on the error type errMsg := err.Error() if strings.Contains(errMsg, "500") || strings.Contains(errMsg, "QEMU guest agent is not running") { @@ -847,3 +888,11 @@ func (m *Monitor) updateVMDisksFromGuestAgentFSInfo( return diskTotal, diskUsed, diskTotal - diskUsed, diskUsage, nil, false, "special-filesystems-only" } + +func guestAgentDiskDeferred(reason string) bool { + switch strings.TrimPrefix(reason, "prev-") { + case "vm-locked", "lock-unverified", "agent-busy", "agent-cooldown", "agent-capacity", "agent-timeout", "agent-response-incomplete": + return true + } + return false +} diff --git a/internal/monitoring/monitor_pve_guest_helpers.go b/internal/monitoring/monitor_pve_guest_helpers.go index 65885359ff..86fe424d2f 100644 --- a/internal/monitoring/monitor_pve_guest_helpers.go +++ b/internal/monitoring/monitor_pve_guest_helpers.go @@ -1,6 +1,7 @@ package monitoring import ( + "strings" "time" "github.com/rcourtman/pulse-go-rewrite/internal/models" @@ -134,7 +135,17 @@ func (m *Monitor) recordGuestMetrics(allVMs []models.VM, allContainers []models. } if vm.Status == "running" { diskRead, diskWrite, networkIn, networkOut := guestHistoryRates(vm.DiskRead, vm.DiskWrite, vm.NetworkIn, vm.NetworkOut, vm.IORateValidity) - m.recordGuestMetric("vm", vm.ID, unifiedresources.ProxmoxGuestCPUPercent(vm.CPU), historyMemoryUsage(vm.Memory), historyMemoryUsed(vm.Memory), vm.Disk.Usage, diskRead, diskWrite, networkIn, networkOut, now) + memoryUsage, memoryUsed, diskUsage := historyMemoryUsage(vm.Memory), historyMemoryUsed(vm.Memory), vm.Disk.Usage + if strings.HasPrefix(vm.DiskStatusReason, "prev-") { + diskUsage = -1 + } + if vm.GuestAgentStatus == "deferred" { + snapshot := m.previousGuestSnapshot(vm.Instance, "qemu", vm.Node, vm.VMID) + if snapshot == nil || snapshot.MemorySource == "previous-snapshot" || strings.HasPrefix(CanonicalMemorySource(snapshot.MemorySource), "guest-agent-meminfo") { + memoryUsage, memoryUsed = -1, -1 + } + } + m.recordGuestMetric("vm", vm.ID, unifiedresources.ProxmoxGuestCPUPercent(vm.CPU), memoryUsage, memoryUsed, diskUsage, diskRead, diskWrite, networkIn, networkOut, now) } } for _, ct := range allContainers { diff --git a/internal/monitoring/monitor_pve_node_vm_builder.go b/internal/monitoring/monitor_pve_node_vm_builder.go index 755c249e48..c5182072be 100644 --- a/internal/monitoring/monitor_pve_node_vm_builder.go +++ b/internal/monitoring/monitor_pve_node_vm_builder.go @@ -51,6 +51,7 @@ func (m *Monitor) pollNodeVMsWithClusterResourceBuilder( Uptime: vm.Uptime, Template: vm.Template, Tags: vm.Tags, + Lock: vm.Lock, IOCounters: vm.IOCounters, ObservedAt: vm.ObservedAt, }, diff --git a/pkg/proxmox/client.go b/pkg/proxmox/client.go index 5dc74c8d6f..dec68ba461 100644 --- a/pkg/proxmox/client.go +++ b/pkg/proxmox/client.go @@ -52,13 +52,20 @@ func classifyAPIErrorLog(status int, path string) (apiErrorLogLevel, string) { return apiErrorLogNone, "" } +// Preserve an incomplete body through the HTTP error-response wrapper too. +// An error status alone is not a completed refusal if its body was truncated. +type responseBodyReadError struct{ cause error } + +func (e *responseBodyReadError) Error() string { return e.cause.Error() } +func (e *responseBodyReadError) Unwrap() error { return e.cause } + func readResponseBodyLimited(r io.Reader) ([]byte, error) { body, err := io.ReadAll(io.LimitReader(r, maxResponseBodyBytes+1)) if err != nil { - return nil, err + return nil, &responseBodyReadError{cause: err} } if int64(len(body)) > maxResponseBodyBytes { - return nil, fmt.Errorf("response body exceeds %d bytes", maxResponseBodyBytes) + return nil, &responseBodyReadError{cause: fmt.Errorf("response body exceeds %d bytes", maxResponseBodyBytes)} } return body, nil } @@ -640,6 +647,12 @@ func (c *Client) requestWithRetry(ctx context.Context, method, path string, data // get performs a GET request func (c *Client) get(ctx context.Context, path string) (*http.Response, error) { + if node, vmid, ok := guestAgentPath(path); ok { + return c.getGuestAgent(ctx, path, node, vmid) + } + if strings.Contains(path, "/agent/") { + return nil, &guestAgentDeferredError{reason: "invalid-guest-key"} + } return c.request(ctx, "GET", path, nil) } @@ -2432,6 +2445,7 @@ type ClusterResource struct { Tags string `json:"tags,omitempty"` IOCounters IOCounterPresence `json:"-"` ObservedAt time.Time `json:"-"` + Lock string `json:"lock,omitempty"` } // GetClusterResources returns all resources (VMs, containers) across the cluster @@ -2715,6 +2729,7 @@ type VMStatus struct { Agent VMAgentField `json:"agent"` IOCounters IOCounterPresence `json:"-"` ObservedAt time.Time `json:"-"` + Lock string `json:"lock,omitempty"` } // GetZFSPoolStatus gets the status of ZFS pools on a node diff --git a/pkg/proxmox/client_api_more3_test.go b/pkg/proxmox/client_api_more3_test.go index 12a4aafc58..240dce4cef 100644 --- a/pkg/proxmox/client_api_more3_test.go +++ b/pkg/proxmox/client_api_more3_test.go @@ -2,12 +2,19 @@ package proxmox import ( "context" + "fmt" "net/http" + "strings" "testing" ) func TestClientVMFSInfoParsing(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/node1/qemu/100/agent/get-fsinfo": writeJSON(t, w, map[string]interface{}{ @@ -56,6 +63,11 @@ func TestClientVMFSInfoParsing(t *testing.T) { func TestClientVMFSInfoParsingCentOSXFSResult(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/pve1/qemu/1376/agent/get-fsinfo": w.Header().Set("Content-Type", "application/json") @@ -87,6 +99,11 @@ func TestClientVMFSInfoParsingCentOSXFSResult(t *testing.T) { func TestClientVMFSInfoParsingPrivilegedCapacityFallback(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/node1/qemu/100/agent/get-fsinfo": writeJSON(t, w, map[string]interface{}{ @@ -126,6 +143,11 @@ func TestClientVMFSInfoParsingPrivilegedCapacityFallback(t *testing.T) { func TestClientVMFSInfoParsingWindowsVolumeGUIDMountpointFallback(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/node1/qemu/100/agent/get-fsinfo": writeJSON(t, w, map[string]interface{}{ @@ -164,6 +186,11 @@ func TestClientVMFSInfoParsingWindowsVolumeGUIDMountpointFallback(t *testing.T) func TestClientVMFSInfoParsingIssue1319WindowsVolumePayload(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/pve7/qemu/116/agent/get-fsinfo": w.Header().Set("Content-Type", "application/json") @@ -200,6 +227,11 @@ func TestClientVMFSInfoParsingIssue1319WindowsVolumePayload(t *testing.T) { func TestClientVMFSInfoSkipsMalformedEntries(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/node1/qemu/100/agent/get-fsinfo": writeJSON(t, w, map[string]interface{}{ @@ -242,6 +274,11 @@ func TestClientVMFSInfoSkipsMalformedEntries(t *testing.T) { func TestClientVMFSInfoObjectResult(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/node1/qemu/100/agent/get-fsinfo": writeJSON(t, w, map[string]interface{}{ @@ -266,6 +303,11 @@ func TestClientVMFSInfoObjectResult(t *testing.T) { func TestClientVMFSInfoObjectFilesystemResult(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/node1/qemu/100/agent/get-fsinfo": writeJSON(t, w, map[string]interface{}{ diff --git a/pkg/proxmox/client_api_more_test.go b/pkg/proxmox/client_api_more_test.go index 095d6cc9a3..bf66d6c96f 100644 --- a/pkg/proxmox/client_api_more_test.go +++ b/pkg/proxmox/client_api_more_test.go @@ -3,8 +3,10 @@ package proxmox import ( "context" "encoding/json" + "fmt" "net/http" "net/http/httptest" + "strings" "testing" "time" ) @@ -217,6 +219,11 @@ func TestClientClusterAndAgentInfo(t *testing.T) { func TestClientGetVMNetworkInterfaces_ObjectAndPartialPayloads(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/node1/qemu/100/agent/network-get-interfaces": writeJSON(t, w, map[string]interface{}{ @@ -340,6 +347,11 @@ func TestClientStatusAndResources(t *testing.T) { func TestClientGetVMMemAvailableFromAgent(t *testing.T) { client := newTestClient(t, func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes/node1/qemu/100/agent/file-read": if got := r.URL.Query().Get("file"); got != "/proc/meminfo" { diff --git a/pkg/proxmox/client_vma_version_test.go b/pkg/proxmox/client_vma_version_test.go index cc8a57c18f..4d957a5f03 100644 --- a/pkg/proxmox/client_vma_version_test.go +++ b/pkg/proxmox/client_vma_version_test.go @@ -5,11 +5,17 @@ import ( "fmt" "net/http" "net/http/httptest" + "strings" "testing" ) func TestClient_GetVMAgentVersion_MapResult(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } w.Header().Set("Content-Type", "application/json") if r.URL.Path == "/api2/json/nodes/node1/qemu/100/agent/info" { fmt.Fprint(w, `{"data":{"result":{"version":{"version":"3.1"}}}}`) @@ -40,6 +46,11 @@ func TestClient_GetVMAgentVersion_MapResult(t *testing.T) { func TestClient_GetVMAgentVersion_NoVersion(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } w.Header().Set("Content-Type", "application/json") if r.URL.Path == "/api2/json/nodes/node1/qemu/100/agent/info" { fmt.Fprint(w, `{"data":{"result":{"foo":"bar"}}}`) diff --git a/pkg/proxmox/cluster_client.go b/pkg/proxmox/cluster_client.go index be6d44c381..64338a616d 100644 --- a/pkg/proxmox/cluster_client.go +++ b/pkg/proxmox/cluster_client.go @@ -172,6 +172,7 @@ func sanitizeEndpointError(errMsg string) string { // endpointFingerprints is an optional map of endpoint URL -> TLS fingerprint for per-node certificate verification. // This enables TOFU (Trust On First Use) for clusters with unique self-signed certs per node. func NewClusterClient(name string, config ClientConfig, endpoints []string, endpointFingerprints map[string]string) *ClusterClient { + registerGuestAgentEndpoints(config.Host, endpoints) if endpointFingerprints == nil { endpointFingerprints = make(map[string]string) } @@ -1373,7 +1374,7 @@ func (cc *ClusterClient) GetVMConfig(ctx context.Context, node string, vmid int) func (cc *ClusterClient) GetVMAgentInfo(ctx context.Context, node string, vmid int) (map[string]interface{}, error) { var result map[string]interface{} - err := cc.executeWithFailover(ctx, func(client *Client) error { + err := cc.executeGuestAgent(ctx, func(client *Client) error { info, err := client.GetVMAgentInfo(ctx, node, vmid) if err != nil { return err @@ -1387,7 +1388,7 @@ func (cc *ClusterClient) GetVMAgentInfo(ctx context.Context, node string, vmid i // GetVMAgentVersion returns the guest agent version for the VM. func (cc *ClusterClient) GetVMAgentVersion(ctx context.Context, node string, vmid int) (string, error) { var version string - err := cc.executeWithFailover(ctx, func(client *Client) error { + err := cc.executeGuestAgent(ctx, func(client *Client) error { v, err := client.GetVMAgentVersion(ctx, node, vmid) if err != nil { return err @@ -1401,7 +1402,7 @@ func (cc *ClusterClient) GetVMAgentVersion(ctx context.Context, node string, vmi // GetVMFSInfo returns filesystem information from QEMU guest agent func (cc *ClusterClient) GetVMFSInfo(ctx context.Context, node string, vmid int) ([]VMFileSystem, error) { var result []VMFileSystem - err := cc.executeWithFailover(ctx, func(client *Client) error { + err := cc.executeGuestAgent(ctx, func(client *Client) error { info, err := client.GetVMFSInfo(ctx, node, vmid) if err != nil { return err @@ -1415,7 +1416,7 @@ func (cc *ClusterClient) GetVMFSInfo(ctx context.Context, node string, vmid int) // GetVMNetworkInterfaces returns guest network interfaces from the QEMU agent func (cc *ClusterClient) GetVMNetworkInterfaces(ctx context.Context, node string, vmid int) ([]VMNetworkInterface, error) { var result []VMNetworkInterface - err := cc.executeWithFailover(ctx, func(client *Client) error { + err := cc.executeGuestAgent(ctx, func(client *Client) error { interfaces, err := client.GetVMNetworkInterfaces(ctx, node, vmid) if err != nil { return err @@ -1429,7 +1430,7 @@ func (cc *ClusterClient) GetVMNetworkInterfaces(ctx context.Context, node string // GetVMMemAvailableFromAgent reads /proc/meminfo via the QEMU guest agent to get MemAvailable. func (cc *ClusterClient) GetVMMemAvailableFromAgent(ctx context.Context, node string, vmid int) (uint64, error) { var result uint64 - err := cc.executeWithFailover(ctx, func(client *Client) error { + err := cc.executeGuestAgent(ctx, func(client *Client) error { available, err := client.GetVMMemAvailableFromAgent(ctx, node, vmid) if err != nil { return err @@ -1444,7 +1445,7 @@ func (cc *ClusterClient) GetVMMemAvailableFromAgent(ctx context.Context, node st // through the first healthy cluster client. func (cc *ClusterClient) GetVMMemoryAvailabilityFromAgent(ctx context.Context, node string, vmid int) (LinuxMemoryAvailability, error) { var result LinuxMemoryAvailability - err := cc.executeWithFailover(ctx, func(client *Client) error { + err := cc.executeGuestAgent(ctx, func(client *Client) error { availability, err := client.GetVMMemoryAvailabilityFromAgent(ctx, node, vmid) if err != nil { return err diff --git a/pkg/proxmox/cluster_client_additional_api6_test.go b/pkg/proxmox/cluster_client_additional_api6_test.go index d969ed0461..ca085c0376 100644 --- a/pkg/proxmox/cluster_client_additional_api6_test.go +++ b/pkg/proxmox/cluster_client_additional_api6_test.go @@ -5,6 +5,7 @@ import ( "fmt" "net/http" "net/http/httptest" + "strings" "testing" "time" ) @@ -33,6 +34,11 @@ func TestClusterClient_GetNodePendingUpdates_NoHealthyNodes(t *testing.T) { func TestClusterClient_GetVMAgentVersion_QemuGA(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } w.Header().Set("Content-Type", "application/json") if r.URL.Path == "/api2/json/nodes" { fmt.Fprint(w, `{"data":[{"node":"node1","status":"online"}]}`) diff --git a/pkg/proxmox/cluster_client_additional_api_test.go b/pkg/proxmox/cluster_client_additional_api_test.go index 17d825eeb3..0c6d65ccff 100644 --- a/pkg/proxmox/cluster_client_additional_api_test.go +++ b/pkg/proxmox/cluster_client_additional_api_test.go @@ -146,6 +146,11 @@ func TestClusterClient_GetVMConfig(t *testing.T) { func TestClusterClient_GetVMAgentInfo(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } w.Header().Set("Content-Type", "application/json") if r.URL.Path == "/api2/json/nodes" { fmt.Fprint(w, `{"data":[{"node":"node1","status":"online"}]}`) @@ -173,6 +178,11 @@ func TestClusterClient_GetVMAgentInfo(t *testing.T) { func TestClusterClient_GetVMAgentVersion(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } w.Header().Set("Content-Type", "application/json") if r.URL.Path == "/api2/json/nodes" { fmt.Fprint(w, `{"data":[{"node":"node1","status":"online"}]}`) @@ -200,6 +210,11 @@ func TestClusterClient_GetVMAgentVersion(t *testing.T) { func TestClusterClient_GetVMNetworkInterfaces(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } w.Header().Set("Content-Type", "application/json") if r.URL.Path == "/api2/json/nodes" { fmt.Fprint(w, `{"data":[{"node":"node1","status":"online"}]}`) diff --git a/pkg/proxmox/cluster_client_api_test.go b/pkg/proxmox/cluster_client_api_test.go index 516f373a48..bbf27e5310 100644 --- a/pkg/proxmox/cluster_client_api_test.go +++ b/pkg/proxmox/cluster_client_api_test.go @@ -6,6 +6,7 @@ import ( "fmt" "net/http" "net/http/httptest" + "strings" "sync/atomic" "testing" "time" @@ -37,6 +38,11 @@ func TestClusterClient_GetCephStatus(t *testing.T) { func TestClusterClient_GetVMMemoryAvailabilityFromAgent(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } w.Header().Set("Content-Type", "application/json") switch r.URL.Path { case "/api2/json/nodes": diff --git a/pkg/proxmox/cluster_client_test.go b/pkg/proxmox/cluster_client_test.go index e5eaf068fe..19ebf4f59c 100644 --- a/pkg/proxmox/cluster_client_test.go +++ b/pkg/proxmox/cluster_client_test.go @@ -6,6 +6,7 @@ import ( "fmt" "net/http" "net/http/httptest" + "strings" "sync/atomic" "testing" "time" @@ -67,6 +68,11 @@ func TestClusterClientHandlesRateLimitWithoutMarkingUnhealthy(t *testing.T) { func TestClusterClientIgnoresGuestAgentTimeoutForHealth(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes": w.Header().Set("Content-Type", "application/json") @@ -295,6 +301,11 @@ func TestIsEndpointConnectivityError(t *testing.T) { func TestClusterClientIgnoresPlainGuestAgent500ForHealth(t *testing.T) { server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + // Guest reads require fresh operation-lock evidence, independently of their payload fixture. + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } switch r.URL.Path { case "/api2/json/nodes": w.Header().Set("Content-Type", "application/json") diff --git a/pkg/proxmox/guest_agent_backup_test.go b/pkg/proxmox/guest_agent_backup_test.go new file mode 100644 index 0000000000..f4b1d3ff1a --- /dev/null +++ b/pkg/proxmox/guest_agent_backup_test.go @@ -0,0 +1,206 @@ +package proxmox + +import ( + "context" + "fmt" + "net/http" + "net/http/httptest" + "strings" + "sync" + "sync/atomic" + "testing" + "time" +) + +func backupTestClient(t *testing.T, host string) *Client { + t.Helper() + c, err := NewClient(ClientConfig{Host: host, TokenName: "fixture@pve!pulse", TokenValue: "fixture", Timeout: time.Second}) + if err != nil { + t.Fatal(err) + } + return c +} + +func backupAgentReads() map[string]func(context.Context, *Client, int) error { + return map[string]func(context.Context, *Client, int) error{ + "os": func(ctx context.Context, c *Client, id int) error { + _, err := c.GetVMAgentInfo(ctx, "node", id) + return err + }, + "version": func(ctx context.Context, c *Client, id int) error { + _, err := c.GetVMAgentVersion(ctx, "node", id) + return err + }, + "filesystem": func(ctx context.Context, c *Client, id int) error { + _, err := c.GetVMFSInfo(ctx, "node", id) + return err + }, + "network": func(ctx context.Context, c *Client, id int) error { + _, err := c.GetVMNetworkInterfaces(ctx, "node", id) + return err + }, + "meminfo": func(ctx context.Context, c *Client, id int) error { + _, err := c.GetVMMemoryAvailabilityFromAgent(ctx, "node", id) + return err + }, + "legacy-memory": func(ctx context.Context, c *Client, id int) error { + _, err := c.GetVMMemAvailableFromAgent(ctx, "node", id) + return err + }, + } +} + +func backupAgentPayload(w http.ResponseWriter, r *http.Request) { + switch { + case strings.HasSuffix(r.URL.Path, "file-read"): + fmt.Fprint(w, `{"data":{"content":"MemTotal: 1024 kB\nMemAvailable: 512 kB\n"}}`) + case strings.HasSuffix(r.URL.Path, "info") && !strings.HasSuffix(r.URL.Path, "fsinfo") && !strings.HasSuffix(r.URL.Path, "osinfo"): + fmt.Fprint(w, `{"data":{"result":{"version":"1.0"}}}`) + case strings.HasSuffix(r.URL.Path, "get-osinfo"): + fmt.Fprint(w, `{"data":{"id":"linux"}}`) + default: + fmt.Fprint(w, `{"data":{"result":[]}}`) + } +} + +func TestGuestAgentBackupLockBlocksEveryReadAndResumes(t *testing.T) { + var locked atomic.Bool + locked.Store(true) + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + if strings.HasSuffix(r.URL.Path, "/config") { + if locked.Load() { + fmt.Fprint(w, `{"data":{"lock":"backup"}}`) + } else { + fmt.Fprint(w, `{"data":{}}`) + } + return + } + if strings.Contains(r.URL.Path, "/agent/") { + calls.Add(1) + backupAgentPayload(w, r) + return + } + http.NotFound(w, r) + })) + defer server.Close() + c := backupTestClient(t, server.URL) + for name, read := range backupAgentReads() { + t.Run(name, func(t *testing.T) { + if err := read(context.Background(), c, 105); err == nil { + t.Fatal("backup-locked guest was queried") + } + }) + } + if got := calls.Load(); got != 0 { + t.Errorf("agent requests while locked = %d, want zero", got) + } + locked.Store(false) + for name, read := range backupAgentReads() { + t.Run("resumed-"+name, func(t *testing.T) { + if err := read(context.Background(), c, 105); err != nil { + t.Fatal(err) + } + }) + } + if got := calls.Load(); got != 6 { + t.Errorf("agent requests after unlock = %d, want six", got) + } +} + +func TestGuestAgentUnknownLockStateFailsClosed(t *testing.T) { + for _, body := range []string{`{"data":null}`, `{}`, `{"data":{"lock":true}}`, `{"data":{"lock":"snapshot"}}`} { + t.Run(body, func(t *testing.T) { + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, body) + return + } + calls.Add(1) + backupAgentPayload(w, r) + })) + defer server.Close() + _, err := backupTestClient(t, server.URL).GetVMFSInfo(context.Background(), "node", 105) + if err == nil || calls.Load() != 0 { + t.Fatalf("unverified lock state queried agent: err=%v calls=%d", err, calls.Load()) + } + }) + } +} + +func TestGuestAgentConcurrentDiagnosticsDoNotOverlap(t *testing.T) { + var active, peak atomic.Int32 + entered := make(chan struct{}, 10) + release := make(chan struct{}) + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } + n := active.Add(1) + for old := peak.Load(); n > old; old = peak.Load() { + if peak.CompareAndSwap(old, n) { + break + } + } + entered <- struct{}{} + <-release + active.Add(-1) + backupAgentPayload(w, r) + })) + defer server.Close() + // Separate Client instances model a diagnostic alongside ordinary polling. + c1, c2 := backupTestClient(t, server.URL), backupTestClient(t, server.URL) + var wg sync.WaitGroup + wg.Add(1) + go func() { defer wg.Done(); _, _ = c1.GetVMFSInfo(context.Background(), "node", 105) }() + <-entered + wg.Add(1) + done := make(chan struct{}) + go func() { + defer wg.Done() + _, _ = c2.GetVMNetworkInterfaces(context.Background(), "node", 105) + close(done) + }() + select { + case <-done: + case <-entered: + case <-time.After(100 * time.Millisecond): + } + close(release) + wg.Wait() + if got := peak.Load(); got != 1 { + t.Fatalf("simultaneous guest-agent requests = %d, want one", got) + } +} + +func TestGuestAgentTimeoutDefersOtherCommands(t *testing.T) { + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } + calls.Add(1) + if strings.HasSuffix(r.URL.Path, "get-fsinfo") { + <-r.Context().Done() + return + } + backupAgentPayload(w, r) + })) + defer server.Close() + c := backupTestClient(t, server.URL) + ctx, cancel := context.WithTimeout(context.Background(), 40*time.Millisecond) + defer cancel() + if _, err := c.GetVMFSInfo(ctx, "node", 105); err == nil { + t.Fatal("missing timeout") + } + if _, err := c.GetVMAgentInfo(context.Background(), "node", 105); err == nil { + t.Error("another command was sent immediately after timeout") + } + if got := calls.Load(); got != 1 { + t.Fatalf("guest-agent requests = %d, want one", got) + } +} diff --git a/pkg/proxmox/guest_agent_guard.go b/pkg/proxmox/guest_agent_guard.go new file mode 100644 index 0000000000..fce6be56b2 --- /dev/null +++ b/pkg/proxmox/guest_agent_guard.go @@ -0,0 +1,218 @@ +package proxmox + +import ( + "bytes" + "context" + "errors" + "fmt" + "io" + "net/http" + "strconv" + "strings" + "sync" + "time" + + "github.com/rcourtman/pulse-go-rewrite/internal/securityutil" +) + +// ErrGuestAgentDeferred means no further guest-agent commands should be queued +// by the caller. In particular, a timed-out HTTP request may still be running +// in pvedaemon; another command is not a safe way to recover it. +var ErrGuestAgentDeferred = errors.New("guest agent deferred") + +type guestAgentDeferredError struct { + reason string + cause error +} + +func (e *guestAgentDeferredError) Error() string { return "guest agent deferred: " + e.reason } +func (e *guestAgentDeferredError) Is(target error) bool { return target == ErrGuestAgentDeferred } +func (e *guestAgentDeferredError) Unwrap() error { return e.cause } + +// GuestAgentDeferredReason returns a fixed reason, never provider response text. +func GuestAgentDeferredReason(err error) string { + var deferred *guestAgentDeferredError + if errors.As(err, &deferred) { + return deferred.reason + } + return "" +} + +const guestAgentUncertainCooldown = time.Minute +const maxGuestAgentGuardEntries = 4096 + +type guestAgentGuardKey struct { + endpoint string + vmid int // VMIDs are cluster-wide, including while a VM migrates nodes. +} +type guestAgentGuardEntry struct { + busy bool + until time.Time +} + +// Share admission with independently constructed diagnostic clients, not just +// the polling client. Only coordination is shared: no credentials or guest data. +var guestAgentGuards = struct { + sync.Mutex + aliases map[string][]string + entries map[guestAgentGuardKey]guestAgentGuardEntry + nextCleanup time.Time +}{aliases: make(map[string][]string), entries: make(map[guestAgentGuardKey]guestAgentGuardEntry)} + +func registerGuestAgentEndpoints(host string, endpoints []string) { + guestAgentGuards.Lock() + defer guestAgentGuards.Unlock() + group := make(map[string]bool) + for _, endpoint := range append([]string{host}, endpoints...) { + u, err := securityutil.NormalizeHTTPBaseURL(endpoint, "https") + if err != nil { + continue + } + key := securityutil.AppendURLPath(u, "api2", "json").String() + group[key] = true + for _, alias := range guestAgentGuards.aliases[key] { + group[alias] = true + } + } + keys := make([]string, 0, len(group)) + for key := range group { + keys = append(keys, key) + } + for key := range group { + guestAgentGuards.aliases[key] = keys + } +} + +func acquireGuestAgent(endpoint string, vmid int) (func(bool), error) { + guestAgentGuards.Lock() + defer guestAgentGuards.Unlock() + now := time.Now() + if !now.Before(guestAgentGuards.nextCleanup) || len(guestAgentGuards.entries) >= maxGuestAgentGuardEntries { + for key, entry := range guestAgentGuards.entries { + if !entry.busy && !now.Before(entry.until) { + delete(guestAgentGuards.entries, key) + } + } + guestAgentGuards.nextCleanup = now.Add(time.Minute) + } + endpoints := guestAgentGuards.aliases[endpoint] + if len(endpoints) == 0 { + endpoints = []string{endpoint} + } + keys := make([]guestAgentGuardKey, 0, len(endpoints)) + for _, endpoint := range endpoints { + key := guestAgentGuardKey{endpoint: endpoint, vmid: vmid} + entry := guestAgentGuards.entries[key] + if entry.busy { + return nil, &guestAgentDeferredError{reason: "agent-busy"} + } + if now.Before(entry.until) { + return nil, &guestAgentDeferredError{reason: "agent-cooldown"} + } + keys = append(keys, key) + } + if len(guestAgentGuards.entries)+len(keys) > maxGuestAgentGuardEntries { + return nil, &guestAgentDeferredError{reason: "agent-capacity"} + } + for _, key := range keys { + guestAgentGuards.entries[key] = guestAgentGuardEntry{busy: true} + } + return func(uncertain bool) { + guestAgentGuards.Lock() + defer guestAgentGuards.Unlock() + for _, key := range keys { + if uncertain { + guestAgentGuards.entries[key] = guestAgentGuardEntry{until: time.Now().Add(guestAgentUncertainCooldown)} + } else { + delete(guestAgentGuards.entries, key) + } + } + }, nil +} + +func guestAgentPath(path string) (string, int, bool) { + parts := strings.Split(strings.SplitN(path, "?", 2)[0], "/") + if len(parts) != 7 || parts[1] != "nodes" || parts[3] != "qemu" || parts[5] != "agent" { + return "", 0, false + } + vmid, err := strconv.Atoi(parts[4]) + return parts[2], vmid, err == nil && vmid > 0 +} + +func (c *Client) verifyGuestAgentUnlocked(ctx context.Context, node string, vmid int) error { + // The config endpoint contains the authoritative PVE operation lock even + // when a PVE version omits it from status/current or cluster/resources. + config, err := c.GetVMConfig(ctx, node, vmid) + if err != nil || config == nil { + return &guestAgentDeferredError{reason: "lock-unverified", cause: err} + } + if value, exists := config["lock"]; exists { + lock, ok := value.(string) + if !ok { + return &guestAgentDeferredError{reason: "lock-unverified"} + } + if strings.TrimSpace(lock) != "" { + return &guestAgentDeferredError{reason: "vm-locked"} + } + } + return nil +} + +func (c *Client) getGuestAgent(ctx context.Context, path, node string, vmid int) (*http.Response, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + release, err := acquireGuestAgent(c.baseURL, vmid) + if err != nil { + return nil, err + } + uncertain := false + defer func() { release(uncertain) }() + if err := c.verifyGuestAgentUnlocked(ctx, node, vmid); err != nil { + return nil, err + } + // A password-session 401 must not replay this command after re-authentication: + // the operation lock could change between attempts. Ordinary API reads still + // own their usual session recovery; this guest admission is single-attempt. + resp, err := c.requestWithRetry(ctx, http.MethodGet, path, nil, true) + if err != nil { + var incomplete *responseBodyReadError + if errors.As(err, &incomplete) { + uncertain = true + return nil, &guestAgentDeferredError{reason: "agent-response-incomplete", cause: err} + } + // Explicit refusals/unsupported commands remain ordinary errors. An + // uncertain completion blocks every command, not just this method. + lower := strings.ToLower(err.Error()) + uncertain = !strings.Contains(lower, "api error") || ctx.Err() != nil || strings.Contains(lower, "timeout") || strings.Contains(lower, "timed out") || strings.Contains(lower, "wrong command id") + if uncertain { + return nil, &guestAgentDeferredError{reason: "agent-timeout", cause: err} + } + return nil, err + } + // Hold admission until the entire bounded response has arrived. Returning + // at response headers would allow another command while the first stalls. + body, readErr := readResponseBodyLimited(resp.Body) + closeErr := resp.Body.Close() + if readErr != nil || closeErr != nil { + uncertain = true + return nil, &guestAgentDeferredError{reason: "agent-response-incomplete", cause: errors.Join(readErr, closeErr)} + } + // A backup may have started while the command was in flight. Do not publish + // its payload as fresh telemetry if lock clearance cannot still be verified. + if err := c.verifyGuestAgentUnlocked(ctx, node, vmid); err != nil { + return nil, err + } + resp.Body = io.NopCloser(bytes.NewReader(body)) + return resp, nil +} + +// Guest calls do not fail over or replay: another HTTP worker still addresses +// the same serial QGA channel. Transport recovery belongs to ordinary API reads. +func (cc *ClusterClient) executeGuestAgent(ctx context.Context, fn func(*Client) error) error { + client, err := cc.getHealthyClient(ctx) + if err != nil { + return fmt.Errorf("select guest agent endpoint: %w", err) + } + return fn(client) +} diff --git a/pkg/proxmox/guest_agent_guard_test.go b/pkg/proxmox/guest_agent_guard_test.go new file mode 100644 index 0000000000..7fcdc823d4 --- /dev/null +++ b/pkg/proxmox/guest_agent_guard_test.go @@ -0,0 +1,279 @@ +package proxmox + +import ( + "context" + "errors" + "fmt" + "net/http" + "net/http/httptest" + "strings" + "sync/atomic" + "testing" + "time" +) + +func TestGuestAgentClusterAliasesAndVMIsolation(t *testing.T) { + entered, release := make(chan struct{}), make(chan struct{}) + first := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch { + case strings.HasSuffix(r.URL.Path, "/nodes"): + fmt.Fprint(w, `{"data":[]}`) + case strings.HasSuffix(r.URL.Path, "/config"): + fmt.Fprint(w, `{"data":{}}`) + case strings.Contains(r.URL.Path, "/agent/"): + close(entered) + <-release + backupAgentPayload(w, r) + default: + http.NotFound(w, r) + } + })) + defer first.Close() + var calls atomic.Int32 + handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.HasSuffix(r.URL.Path, "/nodes") { + fmt.Fprint(w, `{"data":[]}`) + return + } + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } + calls.Add(1) + backupAgentPayload(w, r) + }) + second := httptest.NewServer(handler) + defer second.Close() + independent := httptest.NewServer(handler) + defer independent.Close() + NewClusterClient("aliases", ClientConfig{Host: first.URL, TokenName: "fixture@pve!pulse", TokenValue: "fixture", Timeout: time.Second}, []string{first.URL, second.URL}, nil) + c1, c2, c3 := backupTestClient(t, first.URL), backupTestClient(t, second.URL), backupTestClient(t, independent.URL) + done := make(chan error, 1) + go func() { _, err := c1.GetVMFSInfo(context.Background(), "node", 105); done <- err }() + <-entered + defer func() { + close(release) + if err := <-done; err != nil { + t.Error(err) + } + }() + if _, err := c2.GetVMAgentInfo(context.Background(), "migrated-node", 105); !errors.Is(err, ErrGuestAgentDeferred) { + t.Errorf("alias bypassed busy VM: %v", err) + } + if _, err := c2.GetVMAgentInfo(context.Background(), "node", 106); err != nil { + t.Errorf("different VM blocked: %v", err) + } + if _, err := c3.GetVMAgentInfo(context.Background(), "node", 105); err != nil { + t.Errorf("independent source blocked: %v", err) + } + if calls.Load() != 2 { + t.Errorf("healthy isolated calls = %d", calls.Load()) + } +} + +func TestGuestAgentLockAppearingInFlightDiscardsPayload(t *testing.T) { + var locked atomic.Bool + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.HasSuffix(r.URL.Path, "/config") { + if locked.Load() { + fmt.Fprint(w, `{"data":{"lock":"backup"}}`) + } else { + fmt.Fprint(w, `{"data":{}}`) + } + return + } + calls.Add(1) + locked.Store(true) + backupAgentPayload(w, r) + })) + defer server.Close() + c := backupTestClient(t, server.URL) + if value, err := c.GetVMFSInfo(context.Background(), "node", 105); value != nil || GuestAgentDeferredReason(err) != "vm-locked" { + t.Fatalf("in-flight backup payload published: %#v %v", value, err) + } + if _, err := c.GetVMAgentInfo(context.Background(), "node", 105); GuestAgentDeferredReason(err) != "vm-locked" { + t.Fatal(err) + } + if calls.Load() != 1 { + t.Fatalf("another command queued: %d", calls.Load()) + } +} + +func TestGuestAgentIncompleteBodyBlocksWholeVM(t *testing.T) { + for _, status := range []int{http.StatusOK, http.StatusInternalServerError} { + t.Run(fmt.Sprintf("status-%d", status), func(t *testing.T) { + testGuestAgentIncompleteBodyBlocksWholeVM(t, status) + }) + } +} + +func testGuestAgentIncompleteBodyBlocksWholeVM(t *testing.T, status int) { + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } + calls.Add(1) + w.Header().Set("Content-Length", "1000") + w.WriteHeader(status) + fmt.Fprint(w, `{"data":{"result":[]}}`) + })) + defer server.Close() + c := backupTestClient(t, server.URL) + if _, err := c.GetVMFSInfo(context.Background(), "node", 105); GuestAgentDeferredReason(err) != "agent-response-incomplete" { + t.Errorf("incomplete body not deferred: %v", err) + } + for name, read := range backupAgentReads() { + t.Run(name, func(t *testing.T) { + if err := read(context.Background(), c, 105); GuestAgentDeferredReason(err) != "agent-cooldown" { + t.Fatal(err) + } + }) + } + if calls.Load() != 1 { + t.Fatalf("commands after incomplete body = %d", calls.Load()) + } +} + +func TestGuestAgentDeferralDoesNotHideCompletedRefusal(t *testing.T) { + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if strings.HasSuffix(r.URL.Path, "/config") { + fmt.Fprint(w, `{"data":{}}`) + return + } + calls.Add(1) + if strings.HasSuffix(r.URL.Path, "/get-fsinfo") { + http.Error(w, "permission denied", http.StatusForbidden) + return + } + backupAgentPayload(w, r) + })) + defer server.Close() + c := backupTestClient(t, server.URL) + if _, err := c.GetVMFSInfo(context.Background(), "node", 105); err == nil || errors.Is(err, ErrGuestAgentDeferred) || !strings.Contains(err.Error(), "403") { + t.Fatalf("completed refusal hidden or lost: %v", err) + } + if _, err := c.GetVMAgentInfo(context.Background(), "node", 105); err != nil { + t.Fatalf("completed refusal poisoned other healthy reads: %v", err) + } + if calls.Load() != 2 { + t.Fatalf("refused/healthy command count = %d", calls.Load()) + } +} + +func TestGuestAgentClusterDoesNotReplayUncertainCall(t *testing.T) { + var calls atomic.Int32 + handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch { + case strings.HasSuffix(r.URL.Path, "/nodes"): + fmt.Fprint(w, `{"data":[]}`) + case strings.HasSuffix(r.URL.Path, "/config"): + fmt.Fprint(w, `{"data":{}}`) + default: + calls.Add(1) + http.Error(w, "QEMU guest command timed out", 500) + } + }) + a, b := httptest.NewServer(handler), httptest.NewServer(handler) + defer a.Close() + defer b.Close() + cc := NewClusterClient("timeouts", ClientConfig{Host: a.URL, TokenName: "fixture@pve!pulse", TokenValue: "fixture", Timeout: time.Second}, []string{a.URL, b.URL}, nil) + if _, err := cc.GetVMFSInfo(context.Background(), "node", 105); !errors.Is(err, ErrGuestAgentDeferred) { + t.Fatalf("uncertain completion lost: %v", err) + } + if _, err := cc.GetVMAgentInfo(context.Background(), "node", 105); GuestAgentDeferredReason(err) != "agent-cooldown" { + t.Fatalf("another cluster command sent: %v", err) + } + if calls.Load() != 1 { + t.Fatalf("uncertain call replayed: %d", calls.Load()) + } + for endpoint, healthy := range cc.GetHealthStatus() { + if !healthy { + t.Errorf("VM timeout poisoned endpoint %s", endpoint) + } + } +} + +func TestGuestAgentCooldownExpiresWithoutSleepOrReplay(t *testing.T) { + endpoint := "http://fixture.invalid/api2/json" + release, err := acquireGuestAgent(endpoint, 105) + if err != nil { + t.Fatal(err) + } + release(true) + if _, err := acquireGuestAgent(endpoint, 105); GuestAgentDeferredReason(err) != "agent-cooldown" { + t.Fatal(err) + } + guestAgentGuards.Lock() + key := guestAgentGuardKey{endpoint: endpoint, vmid: 105} + entry := guestAgentGuards.entries[key] + entry.until = time.Now().Add(-time.Second) + guestAgentGuards.entries[key] = entry + guestAgentGuards.Unlock() + release, err = acquireGuestAgent(endpoint, 105) + if err != nil { + t.Fatal(err) + } + release(false) + guestAgentGuards.Lock() + _, retained := guestAgentGuards.entries[key] + guestAgentGuards.Unlock() + if retained { + t.Fatal("healthy completed operation retained estate state") + } +} + +func TestGuestAgentPasswordSessionDoesNotReplayRefusedCommand(t *testing.T) { + for _, refused := range []bool{false, true} { + t.Run(fmt.Sprintf("refused-%t", refused), func(t *testing.T) { + var authCalls, agentCalls, nodeCalls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch { + case strings.HasSuffix(r.URL.Path, "/access/ticket"): + authCalls.Add(1) + fmt.Fprint(w, `{"data":{"ticket":"fixture-ticket","CSRFPreventionToken":"fixture-csrf"}}`) + case strings.HasSuffix(r.URL.Path, "/config"): + fmt.Fprint(w, `{"data":{}}`) + case strings.Contains(r.URL.Path, "/agent/"): + if agentCalls.Add(1) == 1 && refused { + http.Error(w, "session refused", http.StatusUnauthorized) + return + } + backupAgentPayload(w, r) + case strings.HasSuffix(r.URL.Path, "/nodes"): + if nodeCalls.Add(1) == 1 { + http.Error(w, "expired ordinary API session", http.StatusUnauthorized) + return + } + fmt.Fprint(w, `{"data":[]}`) + default: + http.NotFound(w, r) + } + })) + defer server.Close() + client, err := NewClient(ClientConfig{Host: server.URL, User: "fixture@pam", Password: "fixture-password", Timeout: time.Second}) + if err != nil { + t.Fatal(err) + } + _, err = client.GetVMFSInfo(context.Background(), "node", 105) + if refused && (err == nil || !strings.Contains(err.Error(), "401")) { + t.Fatalf("guest refusal lost: %v", err) + } + if !refused && err != nil { + t.Fatal(err) + } + if agentCalls.Load() != 1 || authCalls.Load() != 1 { + t.Fatalf("guest replay: commands=%d authentications=%d", agentCalls.Load(), authCalls.Load()) + } + if _, err := client.GetNodes(context.Background()); err != nil { + t.Fatalf("ordinary API session recovery broken: %v", err) + } + if agentCalls.Load() != 1 || nodeCalls.Load() != 2 || authCalls.Load() != 2 { + t.Fatalf("ordinary recovery replayed guest or did not recover: guest=%d nodes=%d auth=%d", agentCalls.Load(), nodeCalls.Load(), authCalls.Load()) + } + }) + } +} diff --git a/pkg/proxmox/io_counters_test.go b/pkg/proxmox/io_counters_test.go index 8259242b34..fc210c76ab 100644 --- a/pkg/proxmox/io_counters_test.go +++ b/pkg/proxmox/io_counters_test.go @@ -115,3 +115,26 @@ func TestInternalCounterMetadataNeverChangesProxmoxWireShape(t *testing.T) { } } } + +func TestBackupLocksRetainCumulativeCounterPresence(t *testing.T) { + payload := []byte(`{"lock":"backup","diskread":0,"diskwrite":null,"netin":42}`) + var listing VM + var resource ClusterResource + var status VMStatus + for _, dst := range []any{&listing, &resource, &status} { + if err := json.Unmarshal(payload, dst); err != nil { + t.Fatal(err) + } + } + for name, value := range map[string]struct { + lock string + presence IOCounterPresence + }{"listing": {listing.Lock, listing.IOCounters}, "resource": {resource.Lock, resource.IOCounters}, "status": {status.Lock, status.IOCounters}} { + t.Run(name, func(t *testing.T) { + p := value.presence.Effective() + if value.lock != "backup" || !p.DiskRead || !p.NetworkIn || p.DiskWrite || p.NetworkOut { + t.Fatalf("lock decoding corrupted presence: %#v", value) + } + }) + } +}