mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-24 12:42:49 +00:00
Keep storage History and TrueNAS polling responsive and correctly attributed (#2420)
## What changes and why - Storage History readings can be inspected by touch without a later compatibility-focus event replacing the selected sample. Scrolling, gesture cancellation, keyboard inspection and measured zero readings are preserved. - Storage pool details use the owning host's physical disks, temperatures, SMART facts and topology counts. Repeated device paths and UnRAID group names no longer mix facts from different hosts; unknown ownership does not invent membership. - TrueNAS skips app-stat subscriptions for empty app inventories, disposes of failed retried streams, and preserves successful retried log-tail completion. Session-close diagnostics use classified local causes rather than private wire data. This repairs the empty-inventory path reported in #2396. A later reviewed repair now handles matching subscription-termination notifications without stalled acknowledgement or stream reads. The association is source-derived, without a retained native capture; #2400's reported 60-second close remains unexplained. PBS History guidance also now recognises the drawer's batched `metrics` response rather than asking readers to find a separate CPU request that the drawer does not send. This follows the empty-batch response supplied in #1723 and preserves privacy-safe diagnostics. These are repairs to existing monitoring and inspection surfaces, not new collectors or features. The storage defects were reproduced in the existing desktop/phone presentation. ## Included source Complete unpublished range: `f26182f98ccd0e9ee1089bf651d41e0003871ed2..90c436c83d616eccbe544bf3737b13c3b6251e53`. Reviewed candidates: - `7c48430d5a3ed0eefc677b6a5ccc50ee4bc0920a` — History touch inspection. - `34f73590bbb72c01a3e16e6085227849556a2b1d` — pool disk ownership. - `b8cc3b44f8b7e1bd927b0368b10d59f37ede2187` — TrueNAS empty-app/stream lifecycle. - `7b370c7db9941bba4037a888230cb513c02927c0` — PBS batch-aware diagnostic guidance. - `f0ea29480e172631e71ed8da54de95e9cfabda22` — exact subscription termination and polling recovery. Integration merges preserve their original identities and incorporate the published mock-replication, Kubernetes job-time, audit and release-source-mapping frontiers. Additive merge `90c436c83d616eccbe544bf3737b13c3b6251e53` incorporates current main's Patrol weekly-summary scheduling (#2423). The conflict was limited to appending both frontend contracts and retaining the unchanged upstream receipt pointer; runtime source is byte-identical to the respective verified candidates. No accepted commit was rewritten. ## Validation - Retained storage candidate proof: 21,743 tests passed, three skipped; typecheck, lint, production/embedded builds and budgets passed, with complete output and guest shutdown. - Retained Chromium/WebKit parent/final controls cover touch inspection and 12 pool-ownership presentation states, including mounted host replacement and isolated History requests. - Retained TrueNAS proofs cover correctness, races, vet, five focused race repeats, connected monitoring/API checks, backend compilation and 132 completion-guard tests. Initial missing-object setup and a later governance-import exit 1 remain adverse evidence; corrected discovery passed separately. - Retained PBS guide proof `source-h1cvqpr3`: eleven tests and inspected production Docs in desktop Chromium and phone WebKit. - Retained termination-repair evidence: full TrueNAS correctness/races, five focused race repeats, connected three-poll recovery, persisted exact PBS batches and identity controls, vet, server/agent builds and 132 governance tests. Eight completed attempts retain three adverse fixture exits; their corrected controls are separate results. - Publication composition: browser range guard confirms all seven changed frontend runtime files across the original common-base range match in-range receipts; completion guard passes on the composed published-base range without overrides. Tested runtime blobs remain identical, provenance checks and diff whitespace checks pass, and both remote/main ancestry and a clean checkout were verified. A redundant confined-host guide check failed to open localhost sockets (eight errors); this is an execution-context failure, not a passing check. The unchanged guide retains its successful disposable-VM proof. Limits: local Go proof used 1.26.7 rather than declared 1.26.8, and backend compilation used an embed fixture. Exact-head CI remains required. Synthetic presentation and source proofs do not establish native appliance acceptance, whole-report resolution, release availability or promotion readiness. This does not alter the frozen published v6.4.6 RC. [How Pulse handles triage](https://github.com/rcourtman/Pulse/blob/main/docs/AI_TRANSPARENCY.md) [How Pulse handles triage](https://github.com/rcourtman/Pulse/blob/main/docs/AI_TRANSPARENCY.md)
This commit is contained in:
+21
-10
@@ -242,14 +242,24 @@ Start with one affected PBS and a recent, unlocked time range:
|
||||
collecting the identities again. In your signed-in browser, open developer
|
||||
tools → **Network**, filter for `metrics-store/history`, then open the
|
||||
affected **History** tab. Do not refresh or restart the server. Inspect one
|
||||
completed request with `metric=cpu` and its `resourceType`, `resourceId` and
|
||||
`range` query fields. Do not edit or replay the request.
|
||||
3. Check its HTTP status and **Response** locally. A successful JSON response
|
||||
with an empty `points` array means this request returned no samples for that
|
||||
target and range; it does not prove there is no data under another target.
|
||||
Non-empty `points` with an empty chart points to a different display problem.
|
||||
An error, a cancelled request, or an HTML sign-in page is not an empty series.
|
||||
Record a redacted error if present rather than changing permissions.
|
||||
completed request for the drawer's `resourceType`, `resourceId` and `range`.
|
||||
The drawer normally fetches all chart metrics together: a request **without
|
||||
`metric`** is expected, so do not wait for a separate `metric=cpu` request.
|
||||
Do not edit or replay the request.
|
||||
3. Check its HTTP status and **Response** locally. The query determines which
|
||||
response field to inspect:
|
||||
- **No `metric` query field:** inspect the `metrics` object. An empty
|
||||
`metrics: {}` means this request returned no series. If it has a `cpu`
|
||||
entry, note whether the `metrics.cpu` array is empty or non-empty; other
|
||||
metrics can be present even when CPU is absent.
|
||||
- **`metric=cpu`:** inspect the `points` array for that single metric.
|
||||
An empty array means this request returned no CPU samples.
|
||||
A successful empty result applies only to this target and range; it does
|
||||
not prove there is no data under another target. Non-empty samples with an
|
||||
empty chart point to a different display problem. A measured zero is a
|
||||
sample, not an empty result. An error, a cancelled request, or an HTML
|
||||
sign-in page is not an empty series. Record a redacted error if present
|
||||
rather than changing permissions.
|
||||
|
||||
If no matching request appears, record that fact and the visible chart message;
|
||||
do not paste scripts into the browser console to force one. If opening the
|
||||
@@ -257,8 +267,9 @@ page makes the browser unresponsive, stop and retain the observations already
|
||||
available. No repeated reproduction is needed.
|
||||
|
||||
When reporting this, share only the failing route, time range, whether the
|
||||
request target agrees with the drawer, HTTP status, and whether `points` is
|
||||
empty or non-empty. Use consistent placeholders for private hostnames and IDs.
|
||||
request target agrees with the drawer, HTTP status, and the response shape:
|
||||
`metrics` empty, CPU absent, CPU empty/non-empty, or single-metric `points`
|
||||
empty/non-empty. Use consistent placeholders for private hostnames and IDs.
|
||||
Do not share a HAR export, **Copy as cURL** output, request headers, cookies,
|
||||
tokens, or a full response. These checks use the existing browser session;
|
||||
they need no new API token or command-execution permission.
|
||||
|
||||
@@ -7730,6 +7730,46 @@ the production chart and API transport with synthetic stored samples. This
|
||||
does not establish native collection, whole-report resolution or availability
|
||||
in a release. Existing request, source, units and access boundaries are unchanged.
|
||||
|
||||
### Shared History touch inspection preserves native scrolling
|
||||
|
||||
A primary touch tap inside a Storage History plot selects the nearest actual
|
||||
observation at that horizontal position. Compatibility mouse focus must not
|
||||
replace that reading with the latest sample or announce it as keyboard input.
|
||||
The existing hover group and timestamp reconciliation remain shared with mouse
|
||||
and keyboard inspection. Single measured-zero observations remain inspectable;
|
||||
empty data and a changed selection cannot inherit an old touch.
|
||||
|
||||
Movement beyond ten CSS pixels, cancellation, multiple contacts and release
|
||||
outside the plot are not taps. Inspection neither prevents native touch defaults
|
||||
nor captures the pointer or changes touch-action: page scrolling and pinch zoom
|
||||
remain browser-owned. Blur and Escape clear inspection, and supported keyboard
|
||||
keys resume the existing dated, politely announced sample navigation.
|
||||
|
||||
Mounted `HistoryChart.test.tsx` regressions cover focus ordering, gesture
|
||||
rejection, selection ownership, refresh, empty/zero and input coexistence.
|
||||
`browser-tests/history-touch.cjs` exercises actual touchscreen taps through the
|
||||
production disk/pool details and API transport in phone Chromium/WebKit, native
|
||||
vertical panning in Chromium, and desktop mouse/keyboard controls. Synthetic
|
||||
sample presentation is not native collector, physical device, assistive-device
|
||||
or installed whole-report acceptance.
|
||||
|
||||
### Shared Storage presenters preserve disk ownership
|
||||
|
||||
Pool-detail presenters under `features/storageBackups/` must not use repeated
|
||||
device paths, UnRAID group labels or host display names as fleet-wide identity.
|
||||
Direct pool children are retained; inferred membership requires the same
|
||||
canonical host parent, or a complete native Proxmox instance/node pair when
|
||||
parents are missing. Known parent conflicts win over native names. Full ZFS
|
||||
device identifiers may normalize an optional `/dev/` prefix but must not match
|
||||
arbitrary suffixes. Missing ownership produces no invented linked-disk facts.
|
||||
The storage adapter preserves the instance needed for that presentation scope;
|
||||
this does not rewrite canonical resources or History targets. Existing keyed
|
||||
pool details must update linked disks and local health through target/snapshot
|
||||
replacement without retaining peer-host rows. The mounted
|
||||
`StoragePoolDetail.ownership.test.tsx` and production-component browser fixture
|
||||
`browser-tests/pool-ownership.cjs` verify desktop/phone presentation and History
|
||||
coexistence; they are not field collection or native recovery evidence.
|
||||
|
||||
### Scheduled report kinds
|
||||
|
||||
The Settings > Reporting schedule form composes the shared `FormSelect` for a
|
||||
|
||||
@@ -17,6 +17,32 @@
|
||||
|
||||
## Purpose
|
||||
|
||||
### Exact TrueNAS subscription termination — issue #2396
|
||||
|
||||
JSON-RPC stream readers and the subscription-acknowledgement wait recognise
|
||||
`notify_unsubscribed` only for the exact requested collection, including its
|
||||
arguments. An unrelated interval or event source cannot reject the current
|
||||
subscription. A matching rejection ends the operation promptly, discards the
|
||||
stream session and is not retried or downgraded to REST. A subsequent ordinary
|
||||
read may authenticate a fresh session within the existing operation budget.
|
||||
The notification uses middleware errno fields, not a JSON-RPC response error;
|
||||
only its numeric errno and a fixed message survive, never provider reason,
|
||||
trace, extra fields or private collection arguments. Malformed termination
|
||||
cannot leave a stream reusable. A clean end without telemetry remains
|
||||
unavailable, not a zero sample; a clean log end returns the bounded collected
|
||||
tail and retains the socket without cancelling an already-ended subscription.
|
||||
|
||||
`TestJSONRPCSubscriptionTerminationRejectsWithoutWaitingOrRetry` covers all
|
||||
three readers before/after acknowledgement, cleanup, confidentiality and fresh
|
||||
read recovery. The adjacent termination controls pin exact-collection
|
||||
isolation, malformed events, absent samples and clean log completion.
|
||||
`TestTrueNASPollerSubscriptionRejectionKeepsPollingAndRecovers` exercises
|
||||
three connected protocol/poller snapshots: usable inventory and poll-ledger
|
||||
progress during rejected nonempty Apps enrichment, native alert disappearance,
|
||||
then ordinary stats recovery with stable identity. These are source controls
|
||||
using the reporter's source-derived envelope, not a retained wire capture,
|
||||
native appliance/incident acceptance or release availability.
|
||||
|
||||
### RAID count provenance across ingestion — issue #2369
|
||||
|
||||
Agent ingestion, host snapshots and canonical read-state projection retain
|
||||
@@ -196,6 +222,47 @@ server, opening-context cancellation, concurrent controls/RPCs and sender dispos
|
||||
cycles, completion timestamps and unchanged failure backoff. These are synthetic
|
||||
runtime proofs, not native firmware timeout or reporter-resolution evidence.
|
||||
|
||||
### TrueNAS empty app inventories and session-disposal evidence
|
||||
|
||||
An empty or null `app.query`/legacy `/app` result remains a successful empty
|
||||
app inventory. It must not request `app.stats`: there is no workload to enrich,
|
||||
and a stopped Apps service must not consume the operation budget or dispose of
|
||||
an otherwise usable monitoring session just to enrich no rows. Nonempty app
|
||||
inventories retain best-effort stats enrichment. This removes a needless
|
||||
subscription path reported in #2396; it does not establish the exact
|
||||
`notify_unsubscribed` envelope or resolve rejection with nonempty inventories.
|
||||
Unrecognised stream notifications remain bounded by the existing operation
|
||||
timeout, not classified as a known subscription's rejection without evidence.
|
||||
|
||||
The one permitted stream transport retry uses the same disposal rules as the
|
||||
initial exchange. Malformed events, empty subscription IDs and failed
|
||||
unsubscribe cleanup discard the second session and report disconnected status;
|
||||
normal app-log idle completion returns the collected tail and discards its
|
||||
timed-out socket rather than surfacing an internal consumed-session sentinel.
|
||||
No extra retry, action replay or REST downgrade is introduced.
|
||||
|
||||
Each authenticated socket's first local disposal records an INFO event with
|
||||
`component=truenas_rpc`, `action=close_session`, a fixed `reason`, authenticated
|
||||
`session_age`, and optional request-method/error-category classifications. Causes
|
||||
distinguish client closure, stream completion/error, transport error and failed
|
||||
keepalive writes. Error categories distinguish timeout, cancellation, method,
|
||||
transport and protocol errors. No error text, params, subscription ID, endpoint,
|
||||
username or credential is logged. Keepalive failure may precede a subsequent
|
||||
poll's disposal, so that first cause must be preserved and logged only once.
|
||||
These are local close-path observations, not appliance/proxy diagnoses or
|
||||
evidence that #2400's minute-by-minute reauthentication has been resolved. There
|
||||
is still no fixed session-age or per-poll closure policy.
|
||||
|
||||
Verification in `internal/truenas/transport_test.go`:
|
||||
`TestJSONRPCEmptyAppsDoNotSubscribeToStats`,
|
||||
`TestLegacyRESTEmptyAppsDoNotNegotiateStats`, the snapshot inventory contract,
|
||||
`TestJSONRPCStreamRetryDiscardsUnusableSecondSession`,
|
||||
`TestJSONRPCStreamRetryReturnsCompletedLogTail`, `TestRPCSessionCloseDetails`,
|
||||
`TestRPCSessionCloseLogIsBoundedAndSecretFree` and
|
||||
`TestRPCSessionCloseLogKeepsFirstKeepaliveCause`. These controlled runtime tests
|
||||
preserve inventory, retry limits, fresh-session recovery and log confidentiality;
|
||||
native SCALE behaviour, published delivery and field acceptance remain separate.
|
||||
|
||||
|
||||
### TrueNAS reporting summaries preserve raw History — issue #2346
|
||||
|
||||
|
||||
@@ -6162,6 +6162,7 @@
|
||||
"internal/truenas/disk_health.go",
|
||||
"internal/truenas/fixtures.go",
|
||||
"internal/truenas/provider.go",
|
||||
"internal/truenas/subscription.go",
|
||||
"internal/truenas/telemetry.go",
|
||||
"internal/truenas/testdata/core13_reporting.json",
|
||||
"internal/truenas/transport.go",
|
||||
|
||||
@@ -6302,3 +6302,35 @@ This changes no collector, health thresholds, history policy or server identity.
|
||||
`frontend-modern/src/components/Storage/__tests__/DiskList.test.tsx` exercises
|
||||
snapshot replacement, in-place reconciliation, missing evidence, target changes
|
||||
and fault/recovery using the real keyed renderer.
|
||||
|
||||
Touch inspection in the existing pool and physical-disk History charts selects
|
||||
an actual stored reading rather than substituting the latest sample on focus.
|
||||
Page pan/cancel and multi-contact gestures do not imply a selected observation;
|
||||
browser-owned scrolling remains available. This changes no source target, API,
|
||||
units, collection, retention, access or recovery boundary. The production
|
||||
disk/pool fixture `frontend-modern/browser-tests/history-touch.cjs` and mounted
|
||||
`HistoryChart.test.tsx` regressions verify presentation/input behaviour only,
|
||||
not live appliance collection or recovery success.
|
||||
|
||||
### Pool-to-physical-disk ownership in Storage details
|
||||
|
||||
`storagePoolDetailPresentation.ts` must scope inferred ZFS device and UnRAID
|
||||
array/cache-group membership to the owning host. An explicit disk parent equal
|
||||
to the canonical pool ID or its retained resource reference remains a direct
|
||||
association. Otherwise, matching canonical host parents establishes scope;
|
||||
conflicting parents cannot be overridden by names, paths or group labels.
|
||||
When canonical parents are unavailable, only a complete matching native
|
||||
Proxmox instance/node pair establishes scope. `storageAdapters.ts` retains that
|
||||
instance alongside the native node. Display names and unscoped hostnames alone
|
||||
are insufficient. Unknown ownership must not present another host's SMART,
|
||||
temperature, counters or topology counts as this pool's evidence.
|
||||
|
||||
ZFS device matching compares full identifiers with only the optional `/dev/`
|
||||
prefix normalized, not arbitrary suffixes or invented partition associations.
|
||||
This is read-only presentation: no collector, canonical server identity,
|
||||
metrics target, health threshold, persistence or recovery authority changes.
|
||||
`storagePoolDiskOwnership.test.ts`, the mounted
|
||||
`StoragePoolDetail.ownership.test.tsx` and `browser-tests/pool-ownership.cjs`
|
||||
cover repeated paths/groups, direct children, scoped legacy snapshots, missing
|
||||
ownership and live target replacement. Synthetic presentation proof does not
|
||||
establish native appliance, installation or recovery acceptance.
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const { chromium, webkit } = require('playwright');
|
||||
|
||||
(async () => {
|
||||
const root = '/workspace/frontend-modern';
|
||||
process.chdir(root);
|
||||
const artifacts = path.join(root, 'node_modules/history-touch-proof');
|
||||
fs.mkdirSync(artifacts, { recursive: true });
|
||||
const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js'));
|
||||
const server = await createServer({
|
||||
root, configFile: path.join(root, 'vite.config.ts'),
|
||||
cacheDir: fs.mkdtempSync(path.join(artifacts, 'vite-')),
|
||||
server: { host: '127.0.0.1', port: 5227, strictPort: true },
|
||||
});
|
||||
const parent = false;
|
||||
let browser;
|
||||
const observations = [];
|
||||
const timestamp = Date.parse('2026-10-02T10:00:00Z');
|
||||
const point = (value, index = 0) => ({ timestamp: timestamp + index * 60000, value, min: value, max: value });
|
||||
const playwright = require('playwright/package.json').version;
|
||||
const expectedPlaywright = JSON.parse(fs.readFileSync('/workspace/tests/integration/package-lock.json', 'utf8')).packages['node_modules/@playwright/test'].version;
|
||||
assert.equal(playwright, expectedPlaywright);
|
||||
try {
|
||||
await server.listen();
|
||||
const configs = [
|
||||
['chromium-desktop', chromium, false, false],
|
||||
['chromium-phone', chromium, true, false],
|
||||
['webkit-phone', webkit, true, true],
|
||||
];
|
||||
for (const [name, engine, touch, dark] of configs) {
|
||||
if (parent && !touch) continue;
|
||||
browser = await engine.launch(engine === chromium ? { headless: true, channel: 'chromium', args: ['--no-sandbox'] } : { headless: true });
|
||||
const page = await browser.newPage({
|
||||
viewport: touch ? { width: 390, height: 844 } : { width: 1365, height: 900 },
|
||||
hasTouch: touch, isMobile: touch, timezoneId: 'Europe/London',
|
||||
});
|
||||
page.setDefaultTimeout(15000);
|
||||
const errors = [];
|
||||
const requests = [];
|
||||
page.on('pageerror', error => errors.push(error.message));
|
||||
let samples = [point(10), point(20, 1), point(30, 2)];
|
||||
await page.route('**/*', route => {
|
||||
const url = new URL(route.request().url());
|
||||
if (url.origin !== 'http://127.0.0.1:5227') return route.abort();
|
||||
if (!url.pathname.startsWith('/api/')) return route.continue();
|
||||
if (url.pathname === '/api/metrics-store/history') {
|
||||
requests.push(Object.fromEntries(url.searchParams));
|
||||
return route.fulfill({ json: { points: samples, source: 'store' } });
|
||||
}
|
||||
return route.fulfill({ json: url.pathname === '/api/license/runtime-capabilities'
|
||||
? { capabilities: [], limits: [], max_history_days: 7, hosted_mode: false, runtime: { build: 'community' }, blocked_capabilities: [] }
|
||||
: { data: [], enabled: false } });
|
||||
});
|
||||
await page.goto('http://127.0.0.1:5227/browser-tests/history-touch.html');
|
||||
if (dark) await page.evaluate(() => document.documentElement.classList.add('dark'));
|
||||
const disk = page.getByRole('region', { name: 'Disk fixture' });
|
||||
const pool = page.getByRole('region', { name: 'Pool fixture' });
|
||||
await disk.getByRole('tab', { name: 'History', exact: true }).click();
|
||||
const busy = disk.getByRole('img', { name: 'Busy chart', exact: true });
|
||||
const description = chart => chart.evaluate(el => document.getElementById(el.getAttribute('aria-describedby')).textContent);
|
||||
await page.waitForFunction(() => [...document.querySelectorAll('canvas')].some(el => el.getAttribute('aria-label') === 'Busy chart' && document.getElementById(el.getAttribute('aria-describedby')).textContent.includes('3 data points')));
|
||||
const tooltip = chart => chart.locator('..').locator('[data-history-chart-tooltip]');
|
||||
const live = chart => chart.locator('..').locator('[aria-live="polite"]');
|
||||
async function inspect(chart, fraction) {
|
||||
await chart.scrollIntoViewIfNeeded();
|
||||
const box = await chart.boundingBox();
|
||||
// Percent labels reserve 40px at left; the time label reserves less than
|
||||
// 25px at right. These interior positions are nearest to each real sample.
|
||||
const x = box.x + 45 + (box.width - 70) * fraction;
|
||||
const y = box.y + box.height / 2;
|
||||
if (touch) await page.touchscreen.tap(x, y);
|
||||
else await page.mouse.move(x, y);
|
||||
await page.waitForTimeout(80);
|
||||
return (await tooltip(chart).count()) ? await tooltip(chart).textContent() : null;
|
||||
}
|
||||
async function record(phase, chart, text) {
|
||||
const dimensions = await page.evaluate(() => ({ scroll: document.documentElement.scrollWidth, width: innerWidth }));
|
||||
assert.ok(dimensions.scroll <= dimensions.width + 1, JSON.stringify(dimensions));
|
||||
observations.push({ name, browser: browser.version(), touch, dark, phase, text, description: await description(chart), dimensions });
|
||||
await page.screenshot({ path: path.join(artifacts, `${parent ? 'parent-' : ''}${name}-${phase}.png`) });
|
||||
}
|
||||
if (parent) {
|
||||
for (const [phase, fraction, expected] of [['first', 0.04, '10.0%'], ['middle', 0.5, '20.0%']]) {
|
||||
const text = await inspect(busy, fraction);
|
||||
if (phase === 'first') assert.ok(!text?.includes(expected), `predecessor unexpectedly inspected the first tap: ${text}`);
|
||||
await record(phase, busy, text);
|
||||
}
|
||||
} else {
|
||||
for (const [phase, fraction, expected] of [['first', 0.04, '10.0%'], ['middle', 0.5, '20.0%'], ['last', 0.96, '30.0%']]) {
|
||||
const text = await inspect(busy, fraction);
|
||||
assert.ok(text?.includes(expected), `${name} ${phase}: ${text}`);
|
||||
assert.equal(await live(busy).textContent(), '', 'pointer inspection must not speak keyboard announcements');
|
||||
assert.equal(await disk.locator('[data-history-chart-tooltip]').count(), 3, 'live I/O group shares inspection');
|
||||
await record(phase, busy, text);
|
||||
}
|
||||
if (touch) {
|
||||
const read = disk.getByRole('img', { name: 'Read chart', exact: true });
|
||||
assert.ok((await inspect(read, 0.5))?.includes('20.0 B/s'));
|
||||
assert.ok((await tooltip(busy).textContent()).includes('20.0%'), 'a peer blur must not clear the new group tap');
|
||||
assert.equal(await live(read).textContent(), '');
|
||||
await inspect(busy, 0.5);
|
||||
// A hybrid device may switch back to a real mouse after a tap.
|
||||
await busy.scrollIntoViewIfNeeded();
|
||||
const box = await busy.boundingBox();
|
||||
await page.mouse.move(box.x + box.width / 2, box.y + box.height / 2);
|
||||
assert.ok((await tooltip(busy).textContent()).includes('20.0%'));
|
||||
await page.mouse.move(0, 0);
|
||||
assert.equal(await tooltip(busy).count(), 0);
|
||||
}
|
||||
await busy.focus();
|
||||
for (const [key, expected] of [['Home', '10.0%'], ['ArrowRight', '20.0%'], ['End', '30.0%']]) {
|
||||
await page.keyboard.press(key);
|
||||
assert.ok((await live(busy).textContent()).includes(expected));
|
||||
}
|
||||
await page.keyboard.press('Escape');
|
||||
assert.equal(await tooltip(busy).count(), 0);
|
||||
await page.getByRole('button', { name: 'After charts' }).focus();
|
||||
assert.equal(await live(busy).textContent(), '');
|
||||
|
||||
if (touch && engine === chromium) {
|
||||
await busy.scrollIntoViewIfNeeded();
|
||||
await inspect(busy, 0.5);
|
||||
const box = await busy.boundingBox();
|
||||
const session = await page.context().newCDPSession(page);
|
||||
await busy.evaluate(el => {
|
||||
el.dataset.cancelCount = '0';
|
||||
el.addEventListener('pointercancel', () => el.dataset.cancelCount = String(Number(el.dataset.cancelCount) + 1));
|
||||
});
|
||||
const scrollMetrics = await page.evaluate(() => ({ before: scrollY, maximum: document.documentElement.scrollHeight - innerHeight }));
|
||||
assert.ok(scrollMetrics.maximum > scrollMetrics.before + 200, 'the fixture must have room to scroll');
|
||||
const before = scrollMetrics.before;
|
||||
const x = box.x + box.width / 2, y = box.y + box.height / 2;
|
||||
await session.send('Input.dispatchTouchEvent', { type: 'touchStart', touchPoints: [{ x, y, id: 1 }] });
|
||||
for (let step = 1; step <= 5; step++) {
|
||||
await session.send('Input.dispatchTouchEvent', { type: 'touchMove', touchPoints: [{ x, y: y - step * 20, id: 1 }] });
|
||||
await page.waitForTimeout(50);
|
||||
}
|
||||
await session.send('Input.dispatchTouchEvent', { type: 'touchEnd', touchPoints: [] });
|
||||
await page.waitForTimeout(300);
|
||||
const after = await page.evaluate(() => scrollY);
|
||||
fs.writeFileSync(path.join(artifacts, 'pan-control.json'), JSON.stringify({before, after, pointerCancels: await busy.getAttribute('data-cancel-count'), touchAction: await busy.evaluate(el => getComputedStyle(el).touchAction)},null,2));
|
||||
assert.ok(after > before + 40, `native pan must scroll: ${before} -> ${after}`);
|
||||
assert.ok(Number(await busy.getAttribute('data-cancel-count')) > 0);
|
||||
assert.equal(await tooltip(busy).count(), 0, 'native scrolling must not select a reading');
|
||||
observations.push({ name, phase: 'native-vertical-pan', before, after, maximum: scrollMetrics.maximum, pointerCancels: Number(await busy.getAttribute('data-cancel-count')), touchAction: await busy.evaluate(el => getComputedStyle(el).touchAction) });
|
||||
await session.detach();
|
||||
}
|
||||
|
||||
await pool.getByRole('tab', { name: 'History', exact: true }).click();
|
||||
const usage = pool.getByRole('img', { name: 'Usage chart', exact: true });
|
||||
await usage.waitFor();
|
||||
await page.waitForFunction(() => [...document.querySelectorAll('canvas')].some(el => el.getAttribute('aria-label') === 'Usage chart' && document.getElementById(el.getAttribute('aria-describedby')).textContent.includes('3 data points')));
|
||||
assert.ok((await inspect(usage, 0.04))?.includes('10.0%'));
|
||||
await record('pool-first', usage, await tooltip(usage).textContent());
|
||||
assert.equal(await pool.getByLabel('Capacity history range').inputValue(), '7d');
|
||||
samples = [point(0)];
|
||||
await pool.getByLabel('Capacity history range').selectOption('24h');
|
||||
await page.waitForFunction(() => [...document.querySelectorAll('canvas')].some(el => el.getAttribute('aria-label') === 'Usage chart' && document.getElementById(el.getAttribute('aria-describedby')).textContent.includes('1 data point')));
|
||||
assert.ok((await inspect(usage, 0.5))?.includes('0.0%'));
|
||||
await record('pool-zero-singleton', usage, await tooltip(usage).textContent());
|
||||
samples = [];
|
||||
await pool.getByLabel('Capacity history range').selectOption('7d');
|
||||
await pool.getByText('No history samples in this time range.', { exact: true }).waitFor();
|
||||
assert.equal(await inspect(usage, 0.5), null);
|
||||
assert.equal(await live(usage).textContent(), '');
|
||||
await record('pool-empty', usage, null);
|
||||
assert.ok(requests.some(request => request.resourceType === 'disk' && request.resourceId === 'fixture-disk'));
|
||||
assert.ok(requests.some(request => request.resourceType === 'storage' && request.resourceId === 'fixture-pool'));
|
||||
}
|
||||
assert.deepEqual(errors, []);
|
||||
await browser.close(); browser = null;
|
||||
}
|
||||
fs.writeFileSync(path.join(artifacts, parent ? 'parent.json' : 'result.json'), JSON.stringify({ playwright, expectedPlaywright, parent, observations }, null, 2));
|
||||
console.log(JSON.stringify({ result: 'passed', parent, observations: observations.length }));
|
||||
} finally {
|
||||
if (browser) await browser.close();
|
||||
await server.close();
|
||||
}
|
||||
})().catch(error => { console.error(error); process.exitCode = 1; });
|
||||
@@ -0,0 +1 @@
|
||||
<!doctype html><html lang="en"><head><meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><title>Storage History touch verification</title></head><body><div id="root"></div><script type="module" src="./history-touch.tsx"></script></body></html>
|
||||
@@ -0,0 +1,60 @@
|
||||
import { render } from 'solid-js/web';
|
||||
import { DiskDetail } from '../src/components/Storage/DiskDetail';
|
||||
import { StoragePoolDetail } from '../src/components/Storage/StoragePoolDetail';
|
||||
import type { StorageRecord } from '../src/features/storageBackups/models';
|
||||
import type { Resource } from '../src/types/resource';
|
||||
import '../src/index.css';
|
||||
|
||||
const disk = {
|
||||
id: 'fixture-disk',
|
||||
type: 'physical_disk',
|
||||
name: 'Archive HDD',
|
||||
displayName: 'Archive HDD',
|
||||
platformType: 'proxmox-pve',
|
||||
platformId: 'fixture',
|
||||
sourceType: 'api',
|
||||
status: 'online',
|
||||
lastSeen: Date.now(),
|
||||
identity: { hostname: 'storage-host' },
|
||||
metricsTarget: { resourceType: 'disk', resourceId: 'fixture-disk' },
|
||||
physicalDisk: { devPath: '/dev/sda', model: 'Archive HDD', diskType: 'hdd', temperature: 42 },
|
||||
} as Resource;
|
||||
|
||||
const pool: StorageRecord = {
|
||||
id: 'fixture-pool',
|
||||
name: 'Archive',
|
||||
category: 'datastore',
|
||||
health: 'unknown',
|
||||
location: { label: 'archive-host', scope: 'host' },
|
||||
source: {
|
||||
platform: 'proxmox-pbs',
|
||||
family: 'onprem',
|
||||
origin: 'resource',
|
||||
adapterId: 'resource-storage',
|
||||
},
|
||||
capacity: { totalBytes: 1024, usedBytes: 512, freeBytes: 512, usagePercent: 50 },
|
||||
capabilities: ['capacity'],
|
||||
observedAt: Date.now(),
|
||||
};
|
||||
|
||||
render(
|
||||
() => (
|
||||
<main class="min-h-screen space-y-4 bg-surface p-4 text-base-content">
|
||||
<h1>Storage History touch verification</h1>
|
||||
<p>Synthetic stored samples in production disk and pool details.</p>
|
||||
<section aria-label="Disk fixture" class="max-w-3xl">
|
||||
<DiskDetail disk={disk} nodes={[]} />
|
||||
</section>
|
||||
<section aria-label="Pool fixture" class="max-w-3xl">
|
||||
<table class="w-full table-fixed">
|
||||
<tbody>
|
||||
<StoragePoolDetail record={pool} physicalDisks={[]} summarySeriesId="fixture-pool" />
|
||||
</tbody>
|
||||
</table>
|
||||
</section>
|
||||
<button class="min-h-11 border px-3">After charts</button>
|
||||
<div style={{ height: '1000px' }} aria-hidden="true" />
|
||||
</main>
|
||||
),
|
||||
document.getElementById('root')!,
|
||||
);
|
||||
@@ -0,0 +1,177 @@
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const { chromium, webkit } = require('playwright');
|
||||
const parent = false;
|
||||
(async () => {
|
||||
const root = '/workspace/frontend-modern';
|
||||
process.chdir(root);
|
||||
const artifacts = path.join(root, 'node_modules/pool-ownership-proof');
|
||||
fs.mkdirSync(artifacts, { recursive: true });
|
||||
const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js'));
|
||||
const server = await createServer({
|
||||
root,
|
||||
configFile: path.join(root, 'vite.config.ts'),
|
||||
cacheDir: fs.mkdtempSync(path.join(artifacts, 'vite-')),
|
||||
server: { host: '127.0.0.1', port: 5234, strictPort: true },
|
||||
});
|
||||
const playwright = require('playwright/package.json').version;
|
||||
const expected = JSON.parse(
|
||||
fs.readFileSync('/workspace/tests/integration/package-lock.json', 'utf8'),
|
||||
).packages['node_modules/@playwright/test'].version;
|
||||
assert.equal(playwright, expected);
|
||||
const observations = [];
|
||||
let browser;
|
||||
try {
|
||||
await server.listen();
|
||||
for (const [name, engine, phone] of [
|
||||
['chromium-desktop', chromium, false],
|
||||
['webkit-phone', webkit, true],
|
||||
]) {
|
||||
browser = await engine.launch(
|
||||
engine === chromium
|
||||
? { headless: true, channel: 'chromium', args: ['--no-sandbox'] }
|
||||
: { headless: true },
|
||||
);
|
||||
const page = await browser.newPage({
|
||||
viewport: phone ? { width: 390, height: 844 } : { width: 1365, height: 900 },
|
||||
hasTouch: phone,
|
||||
isMobile: phone,
|
||||
});
|
||||
const requests = [],
|
||||
errors = [];
|
||||
page.on('pageerror', (error) => errors.push(error.message));
|
||||
await page.route('**/*', (route) => {
|
||||
const url = new URL(route.request().url());
|
||||
if (url.origin !== 'http://127.0.0.1:5234') return route.abort();
|
||||
if (!url.pathname.startsWith('/api/')) return route.continue();
|
||||
if (url.pathname === '/api/metrics-store/history') {
|
||||
requests.push(Object.fromEntries(url.searchParams));
|
||||
return route.fulfill({
|
||||
json: {
|
||||
points: [{ timestamp: 1790935200000, value: 25, min: 25, max: 25 }],
|
||||
source: 'store',
|
||||
},
|
||||
});
|
||||
}
|
||||
return route.fulfill({
|
||||
json:
|
||||
url.pathname === '/api/license/runtime-capabilities'
|
||||
? {
|
||||
capabilities: [],
|
||||
limits: [],
|
||||
max_history_days: 7,
|
||||
hosted_mode: false,
|
||||
runtime: { build: 'community' },
|
||||
blocked_capabilities: [],
|
||||
}
|
||||
: { data: [], enabled: false },
|
||||
});
|
||||
});
|
||||
await page.goto('http://127.0.0.1:5234/browser-tests/pool-ownership.html');
|
||||
if (phone) await page.evaluate(() => document.documentElement.classList.add('dark'));
|
||||
const detail = page.getByRole('region', { name: 'Pool detail' });
|
||||
const phases = parent
|
||||
? [
|
||||
['ZFS host A', 4, null],
|
||||
['UnRAID host A', 3, null],
|
||||
]
|
||||
: [
|
||||
['ZFS host A', 1, 'Host A disk'],
|
||||
['ZFS host B', 1, 'Host B disk'],
|
||||
['Missing ownership', 0, null],
|
||||
['Direct child', 1, 'Host A disk'],
|
||||
['UnRAID host A', 1, 'Host A disk'],
|
||||
['UnRAID host B', 1, 'Host B disk'],
|
||||
];
|
||||
for (const [phase, count, model] of phases) {
|
||||
await page.getByRole('button', { name: phase, exact: true }).click();
|
||||
const text = await detail.textContent();
|
||||
if (count) {
|
||||
await detail.getByText(`Physical Disks (${count})`, { exact: true }).waitFor();
|
||||
if (model) {
|
||||
assert.ok(text.includes(model));
|
||||
assert.ok(!text.includes(model === 'Host A disk' ? 'Host B disk' : 'Host A disk'));
|
||||
assert.ok(!text.includes('Unowned disk') && !text.includes('Suffix collision disk'));
|
||||
assert.equal(text.includes('99 errors'), model === 'Host B disk');
|
||||
assert.ok(text.includes(model === 'Host B disk' ? '82°C' : '34°C'));
|
||||
} else
|
||||
assert.ok(
|
||||
text.includes('Host B disk') && text.includes('99 errors'),
|
||||
'parent must expose foreign health',
|
||||
);
|
||||
} else assert.equal(await detail.getByText(/Physical Disks/).count(), 0);
|
||||
const dimensions = await page.evaluate(() => ({
|
||||
scroll: document.documentElement.scrollWidth,
|
||||
width: innerWidth,
|
||||
}));
|
||||
assert.ok(dimensions.scroll <= dimensions.width + 1, JSON.stringify(dimensions));
|
||||
const screenshot = `${parent ? 'parent-' : ''}${name}-${phase.toLowerCase().replaceAll(' ', '-')}.png`;
|
||||
await page.screenshot({ path: path.join(artifacts, screenshot), fullPage: true });
|
||||
observations.push({
|
||||
name,
|
||||
phase,
|
||||
browser: browser.version(),
|
||||
text,
|
||||
dimensions,
|
||||
screenshot,
|
||||
});
|
||||
}
|
||||
if (!parent) {
|
||||
await page.getByRole('button', { name: 'ZFS host A', exact: true }).click();
|
||||
await detail.getByRole('tab', { name: 'History', exact: true }).click();
|
||||
await detail.getByRole('img', { name: 'Usage chart', exact: true }).waitFor();
|
||||
await page.waitForFunction(() =>
|
||||
[...document.querySelectorAll('canvas')].some((el) =>
|
||||
document
|
||||
.getElementById(el.getAttribute('aria-describedby'))
|
||||
?.textContent.includes('1 data point'),
|
||||
),
|
||||
);
|
||||
assert.ok(
|
||||
requests.some(
|
||||
(request) =>
|
||||
request.resourceId === 'history-a' &&
|
||||
request.metric === 'usage' &&
|
||||
request.resourceType === 'storage',
|
||||
),
|
||||
);
|
||||
await page.getByRole('button', { name: 'ZFS host B', exact: true }).click();
|
||||
await page.waitForFunction(() =>
|
||||
[...document.querySelectorAll('canvas')].some((el) =>
|
||||
document
|
||||
.getElementById(el.getAttribute('aria-describedby'))
|
||||
?.textContent.includes('1 data point'),
|
||||
),
|
||||
);
|
||||
assert.ok(
|
||||
requests.some(
|
||||
(request) =>
|
||||
request.resourceId === 'history-b' &&
|
||||
request.metric === 'usage' &&
|
||||
request.resourceType === 'storage',
|
||||
),
|
||||
);
|
||||
await detail.getByRole('tab', { name: 'Overview', exact: true }).click();
|
||||
await detail.getByText('Physical Disks (1)', { exact: true }).waitFor();
|
||||
assert.equal(await detail.getByText('Host A disk', { exact: true }).count(), 0);
|
||||
assert.equal(await detail.getByText('Host B disk', { exact: true }).count(), 1);
|
||||
}
|
||||
assert.deepEqual(errors, []);
|
||||
observations.push({ name, phase: 'history-requests', requests, errors });
|
||||
await browser.close();
|
||||
browser = null;
|
||||
}
|
||||
fs.writeFileSync(
|
||||
path.join(artifacts, parent ? 'parent.json' : 'result.json'),
|
||||
JSON.stringify({ playwright, expected, parent, observations }, null, 2),
|
||||
);
|
||||
console.log(JSON.stringify({ result: 'passed', parent, observations: observations.length }));
|
||||
} finally {
|
||||
if (browser) await browser.close();
|
||||
await server.close();
|
||||
}
|
||||
})().catch((error) => {
|
||||
console.error(error);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
@@ -0,0 +1,12 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Pool ownership verification</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="./pool-ownership.tsx"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,123 @@
|
||||
import { createMemo, createSignal, For } from 'solid-js';
|
||||
import { render } from 'solid-js/web';
|
||||
import { StoragePoolDetail } from '../src/components/Storage/StoragePoolDetail';
|
||||
import { buildStorageRecords } from '../src/features/storageBackups/storageAdapters';
|
||||
import type { State } from '../src/types/api';
|
||||
import type { Resource } from '../src/types/resource';
|
||||
import '../src/index.css';
|
||||
|
||||
const modes = [
|
||||
'ZFS host A',
|
||||
'ZFS host B',
|
||||
'UnRAID host A',
|
||||
'UnRAID host B',
|
||||
'Missing ownership',
|
||||
'Direct child',
|
||||
];
|
||||
const [mode, setMode] = createSignal(modes[0]);
|
||||
const resource = createMemo(() => {
|
||||
const unraid = mode().startsWith('UnRAID');
|
||||
const b = mode().endsWith('B');
|
||||
const unknown = mode() === 'Missing ownership';
|
||||
return {
|
||||
id: b ? 'pool-b' : 'pool-a',
|
||||
type: 'storage',
|
||||
name: unraid ? 'Array' : 'tank',
|
||||
displayName: 'tank',
|
||||
platformType: unraid ? 'unraid' : 'truenas',
|
||||
platformId: 'fixture',
|
||||
sourceType: 'api',
|
||||
parentId: unknown ? undefined : b ? 'host-b' : 'host-a',
|
||||
parentName: 'Storage host',
|
||||
status: 'online',
|
||||
lastSeen: Date.now(),
|
||||
metricsTarget: { resourceType: 'storage', resourceId: b ? 'history-b' : 'history-a' },
|
||||
disk: { current: 25, total: 1024, used: 256, free: 768 },
|
||||
storage: unraid
|
||||
? { type: 'unraid-array', platform: 'unraid' }
|
||||
: {
|
||||
type: 'zfs-pool',
|
||||
isZfs: true,
|
||||
zfsPool: {
|
||||
name: 'tank',
|
||||
state: 'ONLINE',
|
||||
readErrors: 0,
|
||||
writeErrors: 0,
|
||||
checksumErrors: 0,
|
||||
devices: [{ name: 'sda', type: 'disk', state: 'ONLINE' }],
|
||||
},
|
||||
},
|
||||
} as Resource;
|
||||
});
|
||||
const record = createMemo(
|
||||
() => buildStorageRecords({ state: {} as State, resources: [resource()] })[0],
|
||||
);
|
||||
const disks = createMemo(() =>
|
||||
[
|
||||
['a', 'host-a', 'Host A disk', '/dev/sda', 34, 0],
|
||||
['b', 'host-b', 'Host B disk', '/dev/sda', 82, 99],
|
||||
['unknown', undefined, 'Unowned disk', '/dev/sda', 70, 0],
|
||||
['suffix', 'host-a', 'Suffix collision disk', '/dev/not-sda', 66, 0],
|
||||
].map(
|
||||
([id, owner, model, devPath, temperature, errorCount]) =>
|
||||
({
|
||||
id: `disk-${id}`,
|
||||
type: 'physical_disk',
|
||||
name: model,
|
||||
displayName: model,
|
||||
platformType: 'truenas',
|
||||
platformId: 'fixture',
|
||||
sourceType: 'api',
|
||||
status: 'online',
|
||||
lastSeen: Date.now(),
|
||||
parentId: mode() === 'Direct child' && id === 'a' ? 'pool-a' : owner,
|
||||
parentName: 'Storage host',
|
||||
physicalDisk: {
|
||||
devPath,
|
||||
model,
|
||||
temperature,
|
||||
errorCount,
|
||||
storageGroup: id === 'suffix' ? 'other-group' : 'unraid-array',
|
||||
storageRole: 'data',
|
||||
sizeBytes: 1024,
|
||||
storageState: 'online',
|
||||
},
|
||||
}) as Resource,
|
||||
),
|
||||
);
|
||||
|
||||
render(
|
||||
() => (
|
||||
<main class="min-h-screen space-y-4 bg-surface p-4 text-base-content">
|
||||
<h1>Pool ownership verification</h1>
|
||||
<p>
|
||||
Repeated disk names and groups across synthetic hosts, using the production Storage detail.
|
||||
</p>
|
||||
<nav aria-label="Fixture selection" class="flex flex-wrap gap-2">
|
||||
<For each={modes}>
|
||||
{(value) => (
|
||||
<button
|
||||
class="min-h-11 rounded border border-border px-3"
|
||||
onClick={() => setMode(value)}
|
||||
>
|
||||
{value}
|
||||
</button>
|
||||
)}
|
||||
</For>
|
||||
</nav>
|
||||
<section aria-label="Pool detail">
|
||||
<h2>{mode()}</h2>
|
||||
<table class="w-full table-fixed">
|
||||
<tbody>
|
||||
<StoragePoolDetail
|
||||
record={record()}
|
||||
physicalDisks={disks()}
|
||||
summarySeriesId="ownership-pool"
|
||||
/>
|
||||
</tbody>
|
||||
</table>
|
||||
</section>
|
||||
</main>
|
||||
),
|
||||
document.getElementById('root')!,
|
||||
);
|
||||
@@ -242,14 +242,24 @@ Start with one affected PBS and a recent, unlocked time range:
|
||||
collecting the identities again. In your signed-in browser, open developer
|
||||
tools → **Network**, filter for `metrics-store/history`, then open the
|
||||
affected **History** tab. Do not refresh or restart the server. Inspect one
|
||||
completed request with `metric=cpu` and its `resourceType`, `resourceId` and
|
||||
`range` query fields. Do not edit or replay the request.
|
||||
3. Check its HTTP status and **Response** locally. A successful JSON response
|
||||
with an empty `points` array means this request returned no samples for that
|
||||
target and range; it does not prove there is no data under another target.
|
||||
Non-empty `points` with an empty chart points to a different display problem.
|
||||
An error, a cancelled request, or an HTML sign-in page is not an empty series.
|
||||
Record a redacted error if present rather than changing permissions.
|
||||
completed request for the drawer's `resourceType`, `resourceId` and `range`.
|
||||
The drawer normally fetches all chart metrics together: a request **without
|
||||
`metric`** is expected, so do not wait for a separate `metric=cpu` request.
|
||||
Do not edit or replay the request.
|
||||
3. Check its HTTP status and **Response** locally. The query determines which
|
||||
response field to inspect:
|
||||
- **No `metric` query field:** inspect the `metrics` object. An empty
|
||||
`metrics: {}` means this request returned no series. If it has a `cpu`
|
||||
entry, note whether the `metrics.cpu` array is empty or non-empty; other
|
||||
metrics can be present even when CPU is absent.
|
||||
- **`metric=cpu`:** inspect the `points` array for that single metric.
|
||||
An empty array means this request returned no CPU samples.
|
||||
A successful empty result applies only to this target and range; it does
|
||||
not prove there is no data under another target. Non-empty samples with an
|
||||
empty chart point to a different display problem. A measured zero is a
|
||||
sample, not an empty result. An error, a cancelled request, or an HTML
|
||||
sign-in page is not an empty series. Record a redacted error if present
|
||||
rather than changing permissions.
|
||||
|
||||
If no matching request appears, record that fact and the visible chart message;
|
||||
do not paste scripts into the browser console to force one. If opening the
|
||||
@@ -257,8 +267,9 @@ page makes the browser unresponsive, stop and retain the observations already
|
||||
available. No repeated reproduction is needed.
|
||||
|
||||
When reporting this, share only the failing route, time range, whether the
|
||||
request target agrees with the drawer, HTTP status, and whether `points` is
|
||||
empty or non-empty. Use consistent placeholders for private hostnames and IDs.
|
||||
request target agrees with the drawer, HTTP status, and the response shape:
|
||||
`metrics` empty, CPU absent, CPU empty/non-empty, or single-metric `points`
|
||||
empty/non-empty. Use consistent placeholders for private hostnames and IDs.
|
||||
Do not share a HAR export, **Copy as cURL** output, request headers, cookies,
|
||||
tokens, or a full response. These checks use the existing browser session;
|
||||
they need no new API token or command-execution permission.
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
import { createSignal } from 'solid-js';
|
||||
import { fireEvent, render, screen } from '@solidjs/testing-library';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { StoragePoolDetail } from '../StoragePoolDetail';
|
||||
import { buildStorageRecords } from '@/features/storageBackups/storageAdapters';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import type { State } from '@/types/api';
|
||||
|
||||
vi.mock('@/components/shared/HistoryChart', () => ({
|
||||
HistoryChart: () => <div>History fixture</div>,
|
||||
}));
|
||||
|
||||
const pool = (owner: string | undefined): Resource =>
|
||||
({
|
||||
id: 'pool',
|
||||
name: 'tank',
|
||||
type: 'storage',
|
||||
platformType: 'truenas',
|
||||
parentId: owner,
|
||||
parentName: 'Identical label',
|
||||
storage: {
|
||||
type: 'zfs-pool',
|
||||
isZfs: true,
|
||||
zfsPool: {
|
||||
name: 'tank',
|
||||
state: 'ONLINE',
|
||||
readErrors: 0,
|
||||
writeErrors: 0,
|
||||
checksumErrors: 0,
|
||||
devices: [{ name: 'sda', type: 'disk', state: 'ONLINE' }],
|
||||
},
|
||||
},
|
||||
}) as Resource;
|
||||
|
||||
const disk = (id: string, owner: string, model: string, errors = 0): Resource =>
|
||||
({
|
||||
id,
|
||||
name: model,
|
||||
type: 'physical_disk',
|
||||
platformType: 'truenas',
|
||||
parentId: owner,
|
||||
parentName: 'Identical label',
|
||||
physicalDisk: { devPath: '/dev/sda', model, errorCount: errors, temperature: errors ? 82 : 34 },
|
||||
}) as Resource;
|
||||
|
||||
describe('Storage pool ownership through live snapshots', () => {
|
||||
it('updates mounted ownership, local warnings and removal without leaking peer disks', () => {
|
||||
const [resource, setResource] = createSignal(pool('host-a'));
|
||||
const [disks, setDisks] = createSignal([
|
||||
disk('a', 'host-a', 'Local disk'),
|
||||
disk('b', 'host-b', 'Peer disk', 99),
|
||||
]);
|
||||
const record = () => buildStorageRecords({ state: {} as State, resources: [resource()] })[0];
|
||||
render(() => (
|
||||
<table>
|
||||
<tbody>
|
||||
<StoragePoolDetail record={record()} physicalDisks={disks()} summarySeriesId="tank" />
|
||||
</tbody>
|
||||
</table>
|
||||
));
|
||||
|
||||
expect(screen.getByText('Physical Disks (1)')).toBeVisible();
|
||||
expect(screen.getByText('Local disk')).toBeVisible();
|
||||
expect(screen.queryByText('Peer disk')).not.toBeInTheDocument();
|
||||
expect(screen.queryByText('99 errors')).not.toBeInTheDocument();
|
||||
|
||||
setResource(pool('host-b'));
|
||||
expect(screen.queryByText('Local disk')).not.toBeInTheDocument();
|
||||
expect(screen.getByText('Peer disk')).toBeVisible();
|
||||
expect(screen.getByText('99 errors')).toBeVisible();
|
||||
expect(screen.getByText('82°C')).toBeVisible();
|
||||
fireEvent.click(screen.getByRole('tab', { name: 'History' }));
|
||||
fireEvent.click(screen.getByRole('tab', { name: 'Overview' }));
|
||||
expect(screen.getByText('Peer disk')).toBeVisible();
|
||||
|
||||
setResource(pool(undefined));
|
||||
expect(screen.queryByText(/Physical Disks/)).not.toBeInTheDocument();
|
||||
expect(screen.getByText('ZFS Pool')).toBeVisible();
|
||||
setResource(pool('host-a'));
|
||||
setDisks([disk('a', 'host-a', 'Local disk', 4)]);
|
||||
expect(screen.getByText('Local disk')).toBeVisible();
|
||||
expect(screen.getByText('4 errors')).toBeVisible();
|
||||
setDisks([]);
|
||||
expect(screen.queryByText(/Physical Disks/)).not.toBeInTheDocument();
|
||||
expect(screen.queryByText('4 errors')).not.toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
@@ -238,6 +238,7 @@ describe('StoragePoolDetail', () => {
|
||||
[
|
||||
{
|
||||
id: 'disk1',
|
||||
parentId: 'storage-1',
|
||||
type: 'physical_disk',
|
||||
name: 'Disk 1',
|
||||
displayName: 'Disk 1',
|
||||
|
||||
@@ -23,6 +23,7 @@ const buildRecord = (): StorageRecord => ({
|
||||
metricsTarget: { resourceType: 'storage', resourceId: 'pool:tank' },
|
||||
details: {
|
||||
node: 'truenas01',
|
||||
parentId: 'host-truenas01',
|
||||
type: 'pool',
|
||||
zfsPool: {
|
||||
state: 'ONLINE',
|
||||
@@ -38,6 +39,7 @@ const buildRecord = (): StorageRecord => ({
|
||||
const buildDisk = (): Resource =>
|
||||
({
|
||||
id: 'disk-1',
|
||||
parentId: 'host-truenas01',
|
||||
type: 'physical_disk',
|
||||
name: 'disk-1',
|
||||
displayName: 'disk-1',
|
||||
|
||||
@@ -78,10 +78,15 @@ export const HistoryChart: Component<HistoryChartProps> = (props) => {
|
||||
aria-describedby={descriptionId}
|
||||
onMouseMove={chart.handleMouseMove}
|
||||
onMouseLeave={chart.handleMouseLeave}
|
||||
onPointerDown={chart.handlePointerDown}
|
||||
onPointerMove={chart.handlePointerMove}
|
||||
onPointerUp={chart.handlePointerUp}
|
||||
onPointerCancel={chart.handlePointerCancel}
|
||||
/>
|
||||
<p id={descriptionId} class="sr-only">
|
||||
{refreshStatus()} {accessibleDescription()} Use Left and Right arrow keys to inspect
|
||||
readings, Home and End for the first and last reading, and Escape to clear inspection.
|
||||
{refreshStatus()} {accessibleDescription()} Tap the chart to inspect a reading, or use
|
||||
Left and Right arrow keys, Home and End for the first and last reading, and Escape to
|
||||
clear inspection.
|
||||
</p>
|
||||
<p class="sr-only" aria-live="polite" aria-atomic="true">
|
||||
{chart.keyboardInspecting() && chart.hoveredPoint()
|
||||
|
||||
@@ -15,6 +15,63 @@ import {
|
||||
HISTORY_CHART_RANGES,
|
||||
} from '@/components/shared/historyChartModel';
|
||||
|
||||
function touchPointer(canvas: Element, type: string, fields: Partial<PointerEvent> = {}) {
|
||||
const event = Object.assign(new Event(type, { bubbles: true, cancelable: true }), {
|
||||
pointerId: 1,
|
||||
pointerType: 'touch',
|
||||
isPrimary: true,
|
||||
clientX: 60,
|
||||
clientY: 60,
|
||||
...fields,
|
||||
});
|
||||
fireEvent(canvas, event);
|
||||
// Inspection must never consume native page panning or pinch zoom.
|
||||
expect(event.defaultPrevented).toBe(false);
|
||||
}
|
||||
|
||||
function mountTouchHistory() {
|
||||
const [target, setTarget] = createSignal('disk-a');
|
||||
const [data, setData] = createSignal(
|
||||
[10, 20, 30].map((value, index) => ({
|
||||
timestamp: (index + 1) * 1000,
|
||||
value,
|
||||
min: value,
|
||||
max: value,
|
||||
})),
|
||||
);
|
||||
const view = render(() => (
|
||||
<HistoryChart
|
||||
resourceType="disk"
|
||||
resourceId={target()}
|
||||
metric="disk"
|
||||
unit="%"
|
||||
data={data()}
|
||||
hideSelector
|
||||
/>
|
||||
));
|
||||
const canvas = view.container.querySelector('canvas')!;
|
||||
vi.spyOn(canvas, 'getBoundingClientRect').mockReturnValue({
|
||||
x: 0,
|
||||
y: 0,
|
||||
left: 0,
|
||||
top: 0,
|
||||
right: 400,
|
||||
bottom: 200,
|
||||
width: 400,
|
||||
height: 200,
|
||||
toJSON: () => ({}),
|
||||
});
|
||||
return {
|
||||
...view,
|
||||
canvas,
|
||||
setTarget,
|
||||
data,
|
||||
setData,
|
||||
tooltip: () => view.container.querySelector('[data-history-chart-tooltip]'),
|
||||
announcement: view.container.querySelector('[aria-live="polite"]')!,
|
||||
};
|
||||
}
|
||||
|
||||
if (typeof globalThis.ResizeObserver === 'undefined') {
|
||||
globalThis.ResizeObserver = class ResizeObserver {
|
||||
observe() {}
|
||||
@@ -181,6 +238,142 @@ describe('HistoryChart', () => {
|
||||
expect(announcement.textContent).toBe('');
|
||||
});
|
||||
|
||||
it('inspects the tapped sample without compatibility focus selecting the latest reading', () => {
|
||||
const view = mountTouchHistory();
|
||||
touchPointer(view.canvas, 'pointerdown');
|
||||
touchPointer(view.canvas, 'pointerup');
|
||||
fireEvent.focus(view.canvas);
|
||||
expect(view.tooltip()).toHaveTextContent('10.0%');
|
||||
expect(view.announcement.textContent).toBe('');
|
||||
fireEvent.mouseMove(view.canvas, { clientX: 370 });
|
||||
fireEvent.mouseLeave(view.canvas);
|
||||
expect(view.tooltip()).toHaveTextContent('10.0%');
|
||||
touchPointer(view.canvas, 'pointerdown', { clientX: 200 });
|
||||
touchPointer(view.canvas, 'pointerup', { clientX: 200 });
|
||||
expect(view.tooltip()).toHaveTextContent('20.0%');
|
||||
fireEvent.keyDown(view.canvas, { key: 'ArrowRight' });
|
||||
expect(view.announcement).toHaveTextContent('30.0%');
|
||||
fireEvent.keyDown(view.canvas, { key: 'Escape' });
|
||||
expect(view.tooltip()).toBeNull();
|
||||
fireEvent.blur(view.canvas);
|
||||
fireEvent.focus(view.canvas);
|
||||
expect(view.announcement).toHaveTextContent('30.0%');
|
||||
});
|
||||
|
||||
it.each(['moved', 'release-displaced', 'cancelled', 'second-finger', 'different-pointer'])(
|
||||
'does not turn a %s touch into inspection',
|
||||
(gesture) => {
|
||||
const view = mountTouchHistory();
|
||||
touchPointer(view.canvas, 'pointerdown');
|
||||
if (gesture === 'moved') {
|
||||
touchPointer(view.canvas, 'pointermove', { clientY: 90 });
|
||||
// Returning to the origin is still a drag, not a new tap.
|
||||
touchPointer(view.canvas, 'pointermove');
|
||||
} else if (gesture === 'cancelled') {
|
||||
touchPointer(view.canvas, 'pointercancel');
|
||||
} else if (gesture === 'second-finger') {
|
||||
touchPointer(view.canvas, 'pointerdown', { pointerId: 2, isPrimary: false });
|
||||
}
|
||||
touchPointer(view.canvas, 'pointerup', {
|
||||
...(gesture === 'release-displaced' ? { clientY: 90 } : {}),
|
||||
...(gesture === 'different-pointer' ? { pointerId: 2 } : {}),
|
||||
});
|
||||
fireEvent.focus(view.canvas);
|
||||
expect(view.tooltip()).toBeNull();
|
||||
expect(view.announcement.textContent).toBe('');
|
||||
},
|
||||
);
|
||||
|
||||
it.each([{ clientX: 0 }, { clientX: 400 }, { clientY: -1 }, { clientY: 201 }])(
|
||||
'does not inspect a tap outside the plot: %j',
|
||||
(position) => {
|
||||
const view = mountTouchHistory();
|
||||
touchPointer(view.canvas, 'pointerdown', position);
|
||||
touchPointer(view.canvas, 'pointerup', position);
|
||||
expect(view.tooltip()).toBeNull();
|
||||
},
|
||||
);
|
||||
|
||||
it('invalidates an in-flight touch when the resource changes', () => {
|
||||
const view = mountTouchHistory();
|
||||
touchPointer(view.canvas, 'pointerdown');
|
||||
view.setTarget('disk-b');
|
||||
touchPointer(view.canvas, 'pointerup');
|
||||
fireEvent.focus(view.canvas);
|
||||
expect(view.tooltip()).toBeNull();
|
||||
expect(view.announcement.textContent).toBe('');
|
||||
touchPointer(view.canvas, 'pointerdown', { clientX: 200 });
|
||||
touchPointer(view.canvas, 'pointerup', { clientX: 200 });
|
||||
expect(view.tooltip()).toHaveTextContent('20.0%');
|
||||
});
|
||||
|
||||
it('keeps the tapped time on refresh and distinguishes a lone zero from empty history', () => {
|
||||
const view = mountTouchHistory();
|
||||
touchPointer(view.canvas, 'pointerdown');
|
||||
touchPointer(view.canvas, 'pointerup');
|
||||
view.setData(view.data().map((point) => ({ ...point, value: point.value + 1 })));
|
||||
expect(view.tooltip()).toHaveTextContent('11.0%');
|
||||
view.setData([{ timestamp: 1000, value: 0, min: 0, max: 0 }]);
|
||||
touchPointer(view.canvas, 'pointerdown', { clientX: 200 });
|
||||
touchPointer(view.canvas, 'pointerup', { clientX: 200 });
|
||||
expect(view.tooltip()).toHaveTextContent('0.0%');
|
||||
view.setData([]);
|
||||
touchPointer(view.canvas, 'pointerdown');
|
||||
touchPointer(view.canvas, 'pointerup');
|
||||
fireEvent.focus(view.canvas);
|
||||
expect(view.tooltip()).toBeNull();
|
||||
expect(view.announcement.textContent).toBe('');
|
||||
});
|
||||
|
||||
it('leaves mouse hover available after touch inspection', () => {
|
||||
const view = mountTouchHistory();
|
||||
touchPointer(view.canvas, 'pointerdown');
|
||||
touchPointer(view.canvas, 'pointerup');
|
||||
touchPointer(view.canvas, 'pointermove', { pointerType: 'mouse', clientX: 200 });
|
||||
fireEvent.mouseMove(view.canvas, { clientX: 200 });
|
||||
expect(view.tooltip()).toHaveTextContent('20.0%');
|
||||
fireEvent.mouseLeave(view.canvas);
|
||||
expect(view.tooltip()).toBeNull();
|
||||
});
|
||||
|
||||
it('keeps a new group chart tap when the previously focused chart blurs', () => {
|
||||
const points = [10, 20, 30].map((value, index) => ({
|
||||
timestamp: (index + 1) * 1000,
|
||||
value,
|
||||
min: value,
|
||||
max: value,
|
||||
}));
|
||||
const { container } = render(() => (
|
||||
<HistoryChartHoverGroup>
|
||||
<HistoryChart resourceType="disk" resourceId="a" metric="disk" unit="%" data={points} />
|
||||
<HistoryChart resourceType="disk" resourceId="a" metric="usage" unit="%" data={points} />
|
||||
</HistoryChartHoverGroup>
|
||||
));
|
||||
const [previous, next] = container.querySelectorAll('canvas');
|
||||
vi.spyOn(next, 'getBoundingClientRect').mockReturnValue({
|
||||
x: 0,
|
||||
y: 0,
|
||||
left: 0,
|
||||
top: 0,
|
||||
right: 400,
|
||||
bottom: 200,
|
||||
width: 400,
|
||||
height: 200,
|
||||
toJSON: () => ({}),
|
||||
});
|
||||
fireEvent.focus(previous);
|
||||
touchPointer(next, 'pointerdown');
|
||||
touchPointer(next, 'pointerup');
|
||||
fireEvent.blur(previous);
|
||||
fireEvent.focus(next);
|
||||
const tooltips = container.querySelectorAll('[data-history-chart-tooltip]');
|
||||
expect(tooltips).toHaveLength(2);
|
||||
for (const tooltip of tooltips) expect(tooltip).toHaveTextContent('10.0%');
|
||||
for (const live of container.querySelectorAll('[aria-live="polite"]')) {
|
||||
expect(live.textContent).toBe('');
|
||||
}
|
||||
});
|
||||
|
||||
it('renders the default history label', () => {
|
||||
render(() => <HistoryChart resourceType="agent" resourceId="node-1" metric="cpu" />);
|
||||
|
||||
|
||||
@@ -48,6 +48,9 @@ export function useHistoryChartState(
|
||||
const chartHeight = createMemo(() => props.height || 200);
|
||||
let chartLeftInset = HISTORY_CHART_MIN_LEFT_INSET;
|
||||
let chartRightInset = 0;
|
||||
let touchFocus = false;
|
||||
let touchTimestamp: number | null = null;
|
||||
let pendingTouch: { pointerId: number; x: number; y: number; moved: boolean } | undefined;
|
||||
const hoveredTimestamp = hoverGroup?.hoveredTimestamp ?? localHoveredTimestamp;
|
||||
const setHoveredTimestamp = hoverGroup?.setHoveredTimestamp ?? setLocalHoveredTimestamp;
|
||||
|
||||
@@ -132,6 +135,8 @@ export function useHistoryChartState(
|
||||
suppliedData !== undefined,
|
||||
]);
|
||||
if (selection !== previousSelection) {
|
||||
pendingTouch = undefined;
|
||||
touchTimestamp = null;
|
||||
setHoveredPoint(null);
|
||||
setHoveredTimestamp(null);
|
||||
}
|
||||
@@ -376,12 +381,21 @@ export function useHistoryChartState(
|
||||
});
|
||||
|
||||
const handleFocus = () => {
|
||||
// Touch's compatibility mouse events can focus the canvas after pointerup.
|
||||
// That focus must not replace the tapped reading with the latest sample.
|
||||
if (touchFocus) {
|
||||
setHoveredTimestamp(touchTimestamp);
|
||||
return;
|
||||
}
|
||||
setKeyboardInspecting(true);
|
||||
const points = data();
|
||||
setHoveredTimestamp(points.length ? points[points.length - 1].timestamp : null);
|
||||
};
|
||||
|
||||
const handleBlur = () => {
|
||||
touchFocus = false;
|
||||
touchTimestamp = null;
|
||||
pendingTouch = undefined;
|
||||
setKeyboardInspecting(false);
|
||||
setHoveredTimestamp(null);
|
||||
};
|
||||
@@ -389,12 +403,18 @@ export function useHistoryChartState(
|
||||
const handleKeyDown = (event: KeyboardEvent) => {
|
||||
if (event.altKey || event.ctrlKey || event.metaKey) return;
|
||||
if (event.key === 'Escape') {
|
||||
touchFocus = false;
|
||||
touchTimestamp = null;
|
||||
pendingTouch = undefined;
|
||||
setHoveredTimestamp(null);
|
||||
return;
|
||||
}
|
||||
const points = data();
|
||||
if (!points.length || !['ArrowLeft', 'ArrowRight', 'Home', 'End'].includes(event.key)) return;
|
||||
event.preventDefault();
|
||||
touchFocus = false;
|
||||
touchTimestamp = null;
|
||||
pendingTouch = undefined;
|
||||
setKeyboardInspecting(true);
|
||||
const timestamp = hoveredTimestamp();
|
||||
const index =
|
||||
@@ -410,14 +430,14 @@ export function useHistoryChartState(
|
||||
setHoveredTimestamp(points[next].timestamp);
|
||||
};
|
||||
|
||||
const handleMouseMove = (event: MouseEvent) => {
|
||||
const inspectAtClientX = (clientX: number, snapToSample = false) => {
|
||||
const canvas = refs.getCanvas();
|
||||
const points = data();
|
||||
if (!canvas || points.length === 0) return;
|
||||
|
||||
setKeyboardInspecting(false);
|
||||
const rect = canvas.getBoundingClientRect();
|
||||
const x = event.clientX - rect.left;
|
||||
const x = clientX - rect.left;
|
||||
const width = rect.width;
|
||||
if (x < chartLeftInset || x > width - chartRightInset) {
|
||||
setHoveredTimestamp(null);
|
||||
@@ -434,11 +454,58 @@ export function useHistoryChartState(
|
||||
leftInset: chartLeftInset,
|
||||
rightInset: chartRightInset,
|
||||
});
|
||||
setHoveredTimestamp(geometry.getTimestamp(x));
|
||||
const timestamp = geometry.getTimestamp(x);
|
||||
const inspectedTimestamp = snapToSample
|
||||
? findHistoryChartClosestPoint(points, timestamp).timestamp
|
||||
: timestamp;
|
||||
setHoveredTimestamp(inspectedTimestamp);
|
||||
return inspectedTimestamp;
|
||||
};
|
||||
|
||||
const handleMouseMove = (event: MouseEvent) => {
|
||||
if (!touchFocus) inspectAtClientX(event.clientX);
|
||||
};
|
||||
|
||||
const handlePointerDown = (event: PointerEvent) => {
|
||||
if (event.pointerType === 'mouse') touchFocus = false;
|
||||
if (event.pointerType !== 'touch') return;
|
||||
touchFocus = true;
|
||||
touchTimestamp = null;
|
||||
setKeyboardInspecting(false);
|
||||
setHoveredTimestamp(null);
|
||||
pendingTouch = event.isPrimary
|
||||
? { pointerId: event.pointerId, x: event.clientX, y: event.clientY, moved: false }
|
||||
: undefined;
|
||||
};
|
||||
|
||||
const handlePointerMove = (event: PointerEvent) => {
|
||||
if (event.pointerType === 'mouse') touchFocus = false;
|
||||
if (event.pointerType !== 'touch' || pendingTouch?.pointerId !== event.pointerId) return;
|
||||
if (Math.hypot(event.clientX - pendingTouch.x, event.clientY - pendingTouch.y) > 10) {
|
||||
pendingTouch.moved = true;
|
||||
}
|
||||
};
|
||||
|
||||
const handlePointerUp = (event: PointerEvent) => {
|
||||
if (event.pointerType !== 'touch') return;
|
||||
const touch = pendingTouch;
|
||||
pendingTouch = undefined;
|
||||
if (!touch || !event.isPrimary || touch.pointerId !== event.pointerId || touch.moved) return;
|
||||
if (Math.hypot(event.clientX - touch.x, event.clientY - touch.y) > 10) return;
|
||||
const rect = refs.getCanvas()?.getBoundingClientRect();
|
||||
if (!rect || event.clientY < rect.top || event.clientY > rect.bottom) return;
|
||||
touchTimestamp = inspectAtClientX(event.clientX, true) ?? null;
|
||||
};
|
||||
|
||||
const handlePointerCancel = (event: PointerEvent) => {
|
||||
if (event.pointerType !== 'touch') return;
|
||||
pendingTouch = undefined;
|
||||
touchTimestamp = null;
|
||||
setHoveredTimestamp(null);
|
||||
};
|
||||
|
||||
const handleMouseLeave = () => {
|
||||
if (keyboardInspecting()) return;
|
||||
if (keyboardInspecting() || touchFocus) return;
|
||||
setHoveredTimestamp(null);
|
||||
};
|
||||
|
||||
@@ -454,6 +521,10 @@ export function useHistoryChartState(
|
||||
keyboardInspecting,
|
||||
handleMouseLeave,
|
||||
handleMouseMove,
|
||||
handlePointerDown,
|
||||
handlePointerMove,
|
||||
handlePointerUp,
|
||||
handlePointerCancel,
|
||||
chartHeight,
|
||||
chartWidth,
|
||||
hoveredPoint,
|
||||
|
||||
+6
-2
@@ -453,16 +453,18 @@ describe('getStoragePoolLinkedDisks branch coverage', () => {
|
||||
id: 'cache-pool-1',
|
||||
name: ' My Cache ',
|
||||
source: unraidSource(),
|
||||
details: { type: 'unraid-cache-pool', platform: 'unraid' },
|
||||
details: { type: 'unraid-cache-pool', platform: 'unraid', parentId: 'host-1' },
|
||||
});
|
||||
const disks = [
|
||||
{
|
||||
id: 'in',
|
||||
parentId: 'host-1',
|
||||
name: 'in',
|
||||
physicalDisk: { devPath: '/dev/sda', storageRole: 'cache', storageGroup: 'My Cache' },
|
||||
},
|
||||
{
|
||||
id: 'out',
|
||||
parentId: 'host-1',
|
||||
name: 'out',
|
||||
physicalDisk: { devPath: '/dev/sdb', storageRole: 'cache', storageGroup: 'other' },
|
||||
},
|
||||
@@ -476,16 +478,18 @@ describe('getStoragePoolLinkedDisks branch coverage', () => {
|
||||
id: 'arr-1',
|
||||
name: 'Tower',
|
||||
source: unraidSource(),
|
||||
details: { type: 'pool', platform: 'unraid', topology: 'array' },
|
||||
details: { type: 'pool', platform: 'unraid', topology: 'array', parentId: 'host-1' },
|
||||
});
|
||||
const disks = [
|
||||
{
|
||||
id: 'in',
|
||||
parentId: 'host-1',
|
||||
name: 'in',
|
||||
physicalDisk: { devPath: '/dev/sda', storageRole: 'data', storageGroup: 'unraid-array' },
|
||||
},
|
||||
{
|
||||
id: 'out',
|
||||
parentId: 'host-1',
|
||||
name: 'out',
|
||||
physicalDisk: { devPath: '/dev/sdb', storageRole: 'data', storageGroup: 'nope' },
|
||||
},
|
||||
|
||||
+5
@@ -30,6 +30,7 @@ const buildRecord = (overrides: Partial<StorageRecord> = {}): StorageRecord => (
|
||||
observedAt: Date.now(),
|
||||
details: {
|
||||
node: 'truenas01',
|
||||
parentId: 'host-truenas01',
|
||||
type: 'pool',
|
||||
zfsPool: {
|
||||
state: 'ONLINE',
|
||||
@@ -66,6 +67,7 @@ const buildRecord = (overrides: Partial<StorageRecord> = {}): StorageRecord => (
|
||||
const buildDisk = (): Resource =>
|
||||
({
|
||||
id: 'disk-1',
|
||||
parentId: 'host-truenas01',
|
||||
type: 'physical_disk',
|
||||
name: 'disk-1',
|
||||
displayName: 'disk-1',
|
||||
@@ -249,6 +251,7 @@ describe('storagePoolDetailPresentation', () => {
|
||||
},
|
||||
details: {
|
||||
type: 'unraid-array',
|
||||
parentId: 'host-tower',
|
||||
platform: 'unraid',
|
||||
topology: 'array',
|
||||
arrayState: 'STARTED',
|
||||
@@ -257,6 +260,7 @@ describe('storagePoolDetailPresentation', () => {
|
||||
const disks = [
|
||||
{
|
||||
id: 'parity',
|
||||
parentId: 'host-tower',
|
||||
type: 'physical_disk',
|
||||
name: 'Parity',
|
||||
displayName: 'Parity',
|
||||
@@ -277,6 +281,7 @@ describe('storagePoolDetailPresentation', () => {
|
||||
},
|
||||
{
|
||||
id: 'disk1',
|
||||
parentId: 'host-tower',
|
||||
type: 'physical_disk',
|
||||
name: 'Disk 1',
|
||||
displayName: 'Disk 1',
|
||||
|
||||
+170
@@ -0,0 +1,170 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { State } from '@/types/api';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import { buildStorageRecords } from '../storageAdapters';
|
||||
import { getStoragePoolLinkedDisks } from '../storagePoolDetailPresentation';
|
||||
|
||||
const storage = (overrides: Partial<Resource> = {}): Resource =>
|
||||
({
|
||||
id: 'pool-a',
|
||||
type: 'storage',
|
||||
name: 'tank',
|
||||
displayName: 'tank',
|
||||
platformId: 'provider-a',
|
||||
platformType: 'proxmox-pve',
|
||||
sourceType: 'api',
|
||||
status: 'online',
|
||||
lastSeen: 1,
|
||||
parentId: 'host-a',
|
||||
parentName: 'Same display name',
|
||||
storage: {
|
||||
type: 'zfspool',
|
||||
isZfs: true,
|
||||
zfsPool: {
|
||||
name: 'tank',
|
||||
state: 'ONLINE',
|
||||
readErrors: 0,
|
||||
writeErrors: 0,
|
||||
checksumErrors: 0,
|
||||
devices: [{ name: 'sda', type: 'disk', state: 'ONLINE' }],
|
||||
},
|
||||
},
|
||||
proxmox: { nodeName: 'pve1', instance: 'cluster-a' },
|
||||
...overrides,
|
||||
}) as Resource;
|
||||
|
||||
const disk = (id: string, overrides: Partial<Resource> = {}): Resource =>
|
||||
({
|
||||
id,
|
||||
type: 'physical_disk',
|
||||
name: id,
|
||||
displayName: id,
|
||||
platformId: 'provider-a',
|
||||
platformType: 'proxmox-pve',
|
||||
sourceType: 'api',
|
||||
status: 'online',
|
||||
lastSeen: 1,
|
||||
parentId: 'host-a',
|
||||
parentName: 'Same display name',
|
||||
proxmox: { nodeName: 'pve1', instance: 'cluster-a' },
|
||||
physicalDisk: { devPath: '/dev/sda', model: id, temperature: 35 },
|
||||
...overrides,
|
||||
}) as Resource;
|
||||
|
||||
const record = (resource = storage()) =>
|
||||
buildStorageRecords({ state: {} as State, resources: [resource] })[0];
|
||||
const linked = (resource: Resource, disks: Resource[]) =>
|
||||
getStoragePoolLinkedDisks(record(resource), disks).map((item) => item.id);
|
||||
|
||||
describe('pool physical disk ownership', () => {
|
||||
it('excludes other hosts and unknown owners despite identical paths and labels', () => {
|
||||
expect(
|
||||
linked(storage(), [
|
||||
disk('local'),
|
||||
disk('remote', { parentId: 'host-b' }),
|
||||
disk('unknown', { parentId: undefined, proxmox: undefined }),
|
||||
]),
|
||||
).toEqual(['local']);
|
||||
});
|
||||
|
||||
it('never lets matching provider names override mismatched canonical parents', () => {
|
||||
expect(linked(storage(), [disk('remote', { parentId: 'host-b' })])).toEqual([]);
|
||||
});
|
||||
|
||||
it('preserves direct canonical pool and legacy resource-reference children', () => {
|
||||
const target = record();
|
||||
target.refs = { resourceId: 'legacy-pool' };
|
||||
expect(
|
||||
getStoragePoolLinkedDisks(target, [
|
||||
disk('direct', { parentId: 'pool-a', physicalDisk: { devPath: '/dev/other' } }),
|
||||
disk('legacy', { parentId: 'legacy-pool', physicalDisk: { devPath: '/dev/legacy' } }),
|
||||
]).map((item) => item.id),
|
||||
).toEqual(['legacy', 'direct']);
|
||||
});
|
||||
|
||||
it.each(['top-level', 'legacy'] as const)(
|
||||
'retains exact native instance/node ownership: %s',
|
||||
(shape) => {
|
||||
const pool = storage({ parentId: undefined, proxmox: undefined, platformData: undefined });
|
||||
const local = disk('local', { parentId: undefined, proxmox: undefined });
|
||||
const native = { nodeName: 'pve1', instance: 'cluster-a' };
|
||||
if (shape === 'top-level') {
|
||||
pool.proxmox = native;
|
||||
local.proxmox = native;
|
||||
} else {
|
||||
pool.platformData = { proxmox: native };
|
||||
local.platformData = { proxmox: native };
|
||||
}
|
||||
expect(record(pool).details?.instance).toBe('cluster-a');
|
||||
expect(
|
||||
linked(pool, [
|
||||
local,
|
||||
disk('other-cluster', {
|
||||
parentId: undefined,
|
||||
proxmox: { ...native, instance: 'cluster-b' },
|
||||
}),
|
||||
disk('other-node', { parentId: undefined, proxmox: { ...native, nodeName: 'pve2' } }),
|
||||
disk('unscoped', { parentId: undefined, proxmox: { nodeName: 'pve1' } }),
|
||||
]),
|
||||
).toEqual(['local']);
|
||||
},
|
||||
);
|
||||
|
||||
it('requires both native instance and node when canonical ownership is missing', () => {
|
||||
expect(
|
||||
linked(storage({ parentId: undefined, proxmox: { nodeName: 'pve1' } }), [
|
||||
disk('unscoped', { parentId: undefined, proxmox: { nodeName: 'pve1' } }),
|
||||
]),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it.each(['sda', '/dev/sda'])('matches whole device identifiers, not a suffix: %s', (name) => {
|
||||
const pool = storage();
|
||||
pool.storage!.zfsPool!.devices = [
|
||||
{ name, type: 'disk', state: 'ONLINE', readErrors: 0, writeErrors: 0, checksumErrors: 0 },
|
||||
];
|
||||
expect(
|
||||
linked(pool, [
|
||||
disk('local'),
|
||||
disk('suffix-collision', { physicalDisk: { devPath: '/dev/not-sda' } }),
|
||||
disk('partition', { physicalDisk: { devPath: '/dev/sda1' } }),
|
||||
]),
|
||||
).toEqual(['local']);
|
||||
});
|
||||
|
||||
it('does not confuse nvme device numbers with a shorter vdev name', () => {
|
||||
const pool = storage();
|
||||
pool.storage!.zfsPool!.devices = [
|
||||
{
|
||||
name: 'n1',
|
||||
type: 'disk',
|
||||
state: 'ONLINE',
|
||||
readErrors: 0,
|
||||
writeErrors: 0,
|
||||
checksumErrors: 0,
|
||||
},
|
||||
];
|
||||
expect(linked(pool, [disk('nvme', { physicalDisk: { devPath: '/dev/nvme0n1' } })])).toEqual([]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['unraid-array', 'Array', 'unraid-array'],
|
||||
['unraid-cache-pool', 'cache', 'cache'],
|
||||
])('keeps repeated UnRAID groups on their own host: %s', (type, name, group) => {
|
||||
const pool = storage({
|
||||
name,
|
||||
platformType: 'unraid',
|
||||
proxmox: undefined,
|
||||
storage: { type, platform: 'unraid' },
|
||||
});
|
||||
const member = { devPath: '/dev/sda', storageGroup: group, storageRole: 'data' };
|
||||
expect(
|
||||
linked(pool, [
|
||||
disk('local', { physicalDisk: member }),
|
||||
disk('remote', { parentId: 'host-b', physicalDisk: member }),
|
||||
disk('unknown', { parentId: undefined, proxmox: undefined, physicalDisk: member }),
|
||||
disk('different-group', { physicalDisk: { ...member, storageGroup: 'other' } }),
|
||||
]),
|
||||
).toEqual(['local']);
|
||||
});
|
||||
});
|
||||
@@ -224,6 +224,10 @@ const mapResourceStorageRecord = (resource: Resource, adapterId: string): Storag
|
||||
parentId: resource.parentId,
|
||||
parentName: resource.parentName,
|
||||
node: proxmoxNativeNode || (platformData.node as string | undefined) || storageNodes[0],
|
||||
instance:
|
||||
resource.proxmox?.instance ||
|
||||
(proxmoxPlatform?.instance as string | undefined) ||
|
||||
(platformData.instance as string | undefined),
|
||||
nodeHints,
|
||||
hostLabel,
|
||||
platformLabel,
|
||||
|
||||
@@ -368,6 +368,28 @@ const diskRoleRank = (role: string): number => {
|
||||
}
|
||||
};
|
||||
|
||||
const diskBelongsToStorageHost = (record: StorageRecord, disk: Resource): boolean => {
|
||||
const hostId = readRecordDetailString(record, 'parentId');
|
||||
const diskParentId = disk.parentId?.trim();
|
||||
// Canonical ownership outranks labels, paths and source-native names.
|
||||
// In particular, two hosts may both be called pve1 and both report /dev/sda.
|
||||
if (hostId && diskParentId) return hostId === diskParentId;
|
||||
|
||||
// Older Proxmox snapshots may lack canonical parents. Only the complete
|
||||
// native instance/node pair is a scope; a hostname alone is not one.
|
||||
const platformData = (disk.platformData || {}) as Record<string, unknown>;
|
||||
const proxmox = platformData.proxmox as Resource['proxmox'] | undefined;
|
||||
const node = readRecordDetailString(record, 'node').toLowerCase();
|
||||
const instance = readRecordDetailString(record, 'instance');
|
||||
const diskNode = (disk.proxmox?.nodeName || disk.proxmox?.node || proxmox?.nodeName || '')
|
||||
.trim()
|
||||
.toLowerCase();
|
||||
const diskInstance = (disk.proxmox?.instance || proxmox?.instance || '').trim();
|
||||
return Boolean(node && instance && node === diskNode && instance === diskInstance);
|
||||
};
|
||||
|
||||
const normalizeDevicePath = (value: string): string => value.trim().replace(/^\/dev\//, '');
|
||||
|
||||
export function getStoragePoolLinkedDisks(
|
||||
record: StorageRecord,
|
||||
physicalDisks: Resource[],
|
||||
@@ -382,9 +404,14 @@ export function getStoragePoolLinkedDisks(
|
||||
.filter((disk) => {
|
||||
const devPath = readDiskDevPath(disk);
|
||||
const group = readDiskStorageGroup(disk).toLowerCase();
|
||||
if (directParentIds.has(disk.parentId || '')) return true;
|
||||
if (directParentIds.has(disk.parentId?.trim() || '')) return true;
|
||||
if (!diskBelongsToStorageHost(record, disk)) return false;
|
||||
if (pool?.devices?.length && devPath) {
|
||||
return pool.devices.some((device) => devPath.endsWith(device.name));
|
||||
return pool.devices.some(
|
||||
(device) =>
|
||||
Boolean(device.name.trim()) &&
|
||||
normalizeDevicePath(devPath) === normalizeDevicePath(device.name),
|
||||
);
|
||||
}
|
||||
return Boolean(unraidStorageGroup && group === unraidStorageGroup);
|
||||
})
|
||||
|
||||
@@ -2015,6 +2015,133 @@ func TestTrueNASPollerUnresponsiveRPCDoesNotFreezeOtherConnections(t *testing.T)
|
||||
}
|
||||
}
|
||||
|
||||
// #2396's nonempty Apps path: an accepted subscription is then rejected by
|
||||
// middlewared. Keep the inventory, advance the poll ledger, observe native
|
||||
// alert disappearance and recover stats on the next ordinary poll. This is a
|
||||
// protocol/poller control, not a native appliance or incident-delivery proof.
|
||||
func TestTrueNASPollerSubscriptionRejectionKeepsPollingAndRecovers(t *testing.T) {
|
||||
var reject atomic.Bool
|
||||
var sessions, statsSubscriptions atomic.Int32
|
||||
upgrader := websocket.Upgrader{}
|
||||
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path != "/api/current" {
|
||||
t.Error("modern polling fell back to REST")
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
conn, err := upgrader.Upgrade(w, r, nil)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
defer conn.Close()
|
||||
sessions.Add(1)
|
||||
for {
|
||||
var request struct {
|
||||
ID int64 `json:"id"`
|
||||
Method string `json:"method"`
|
||||
Params []json.RawMessage `json:"params"`
|
||||
}
|
||||
if conn.ReadJSON(&request) != nil {
|
||||
return
|
||||
}
|
||||
var result any = []any{}
|
||||
var collection string
|
||||
switch request.Method {
|
||||
case "auth.login_ex":
|
||||
result = map[string]any{"response_type": "SUCCESS"}
|
||||
case "system.info":
|
||||
result = map[string]any{"hostname": "reject-nas", "version": "TrueNAS-SCALE-25.04.2.6", "system_serial": "REJECT-FIXTURE"}
|
||||
case "pool.query":
|
||||
result = []map[string]any{{"id": 1, "name": "tank", "status": "ONLINE", "size": 1000, "allocated": 400}}
|
||||
case "app.query":
|
||||
result = []map[string]any{{"id": "fixture-app", "name": "fixture-app", "state": "RUNNING"}}
|
||||
case "alert.list":
|
||||
if !reject.Load() {
|
||||
result = []map[string]any{{"id": "fixture-alert", "level": "WARNING", "formatted": "fixture warning", "source": "fixture"}}
|
||||
}
|
||||
case "core.subscribe":
|
||||
if len(request.Params) != 1 || json.Unmarshal(request.Params[0], &collection) != nil {
|
||||
t.Error("invalid subscription")
|
||||
return
|
||||
}
|
||||
result = "fixture-sub"
|
||||
case "core.unsubscribe":
|
||||
result = nil
|
||||
}
|
||||
if conn.WriteJSON(map[string]any{"jsonrpc": "2.0", "id": request.ID, "result": result}) != nil {
|
||||
return
|
||||
}
|
||||
if collection == "" {
|
||||
continue
|
||||
}
|
||||
var params any
|
||||
method := "collection_update"
|
||||
if strings.HasPrefix(collection, "app.stats:") {
|
||||
statsSubscriptions.Add(1)
|
||||
if reject.Load() {
|
||||
method = "notify_unsubscribed"
|
||||
params = map[string]any{"collection": collection, "error": map[string]any{"error": 14, "errname": "EFAULT", "reason": "[EFAULT] Apps are not available", "trace": nil, "extra": nil}}
|
||||
} else {
|
||||
params = map[string]any{"collection": collection, "fields": []any{map[string]any{"app_name": "fixture-app", "cpu_usage": 12}}}
|
||||
}
|
||||
} else {
|
||||
params = map[string]any{"collection": collection, "fields": map[string]any{"cpu": map[string]any{"usage": 10}}}
|
||||
}
|
||||
if conn.WriteJSON(map[string]any{"jsonrpc": "2.0", "method": method, "params": params}) != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}))
|
||||
t.Cleanup(server.Close)
|
||||
client, err := truenas.NewClient(truenas.ClientConfig{
|
||||
Host: server.URL, APIKey: "fixture-key", Username: "fixture-user", Timeout: 2 * time.Second,
|
||||
InsecureSkipVerify: true, Fingerprint: fmt.Sprintf("%x", sha256.Sum256(server.Certificate().Raw)),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(client.Close)
|
||||
instance := config.TrueNASInstance{ID: "reject-connection", Host: server.URL, Enabled: true}
|
||||
provider := truenas.NewLiveProviderForConnection(&truenas.APIFetcher{Client: client}, instance.ID)
|
||||
poller := NewTrueNASPoller(nil, 0, nil)
|
||||
poller.providersByOrg["default"] = map[string]*truenas.Provider{instance.ID: provider}
|
||||
poller.configsByOrg["default"] = map[string]config.TrueNASInstance{instance.ID: instance}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
var lastSuccess time.Time
|
||||
for poll := 0; poll < 3; poll++ {
|
||||
reject.Store(poll == 1)
|
||||
poller.ensureConnectionRuntimeStatusLocked("default", instance.ID).nextPollAt = time.Now().Add(-time.Second)
|
||||
started := time.Now()
|
||||
poller.pollAll(ctx)
|
||||
if elapsed := time.Since(started); elapsed >= time.Second {
|
||||
t.Fatalf("rejection stalled poll %d for %s", poll, elapsed)
|
||||
}
|
||||
summary := poller.ConnectionSummaries("default", []config.TrueNASInstance{instance})[instance.ID]
|
||||
if summary.Poll.LastError != nil || summary.Poll.LastSuccessAt == nil || !summary.Poll.LastSuccessAt.After(lastSuccess) || summary.Poll.LastAttemptAt == nil {
|
||||
t.Fatalf("optional stats rejection prevented poll progress: %+v", summary)
|
||||
}
|
||||
lastSuccess = *summary.Poll.LastSuccessAt
|
||||
snapshot := provider.Snapshot()
|
||||
if snapshot == nil || snapshot.System.Hostname != "reject-nas" || len(snapshot.Pools) != 1 || len(snapshot.Apps) != 1 || snapshot.System.CPUPercent != 10 {
|
||||
t.Fatalf("poll %d lost usable inventory/telemetry: %+v", poll, snapshot)
|
||||
}
|
||||
if poll == 1 {
|
||||
if snapshot.Apps[0].Stats != nil || len(snapshot.Alerts) != 0 {
|
||||
t.Fatal("rejected stats fabricated data or left the former native alert")
|
||||
}
|
||||
} else if snapshot.Apps[0].Stats == nil || snapshot.Apps[0].Stats.CPUPercent != 12 || len(snapshot.Alerts) != 1 {
|
||||
t.Fatal("healthy stats/native alerts did not return")
|
||||
}
|
||||
if !hasTrueNASHostForOrg(poller, "default", "reject-nas") {
|
||||
t.Fatal("poll lost the appliance identity")
|
||||
}
|
||||
}
|
||||
if statsSubscriptions.Load() != 3 || sessions.Load() != 2 {
|
||||
t.Fatalf("rejection replayed or churned later polls: subscriptions=%d sessions=%d", statsSubscriptions.Load(), sessions.Load())
|
||||
}
|
||||
}
|
||||
|
||||
type trueNASMockServer struct {
|
||||
server *httptest.Server
|
||||
requests atomic.Int64
|
||||
|
||||
+56
-12
@@ -258,9 +258,9 @@ func (c *Client) GetSystemTelemetry(ctx context.Context) (*SystemInfo, error) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
telemetry, err = rpc.readSystemTelemetryEvent(ctx, defaultRealtimeIntervalSeconds)
|
||||
telemetry, err = rpc.readSystemTelemetryEvent(ctx, defaultRealtimeIntervalSeconds, subscriptionName)
|
||||
if err != nil {
|
||||
return discardRPCSessionForStreamError(err)
|
||||
return discardRPCSessionForStreamError("reporting.realtime", err)
|
||||
}
|
||||
return rpc.unsubscribe(ctx, subscriptionID)
|
||||
})
|
||||
@@ -1743,12 +1743,18 @@ func (c *Client) getAppsREST(ctx context.Context) ([]App, error) {
|
||||
}
|
||||
|
||||
func (c *Client) parseAppsWithStats(ctx context.Context, response []map[string]any) []App {
|
||||
apps := make([]App, 0, len(response))
|
||||
// There is nothing to enrich when app.query returns an empty inventory.
|
||||
// Subscribing anyway can wait on a stopped Apps service and needlessly
|
||||
// discard an otherwise healthy, persistent monitoring session.
|
||||
if len(response) == 0 {
|
||||
return apps
|
||||
}
|
||||
statsByApp, err := c.GetAppStats(ctx)
|
||||
if err != nil {
|
||||
statsByApp = nil
|
||||
}
|
||||
|
||||
apps := make([]App, 0, len(response))
|
||||
for _, item := range response {
|
||||
activeWorkloads := readMapAny(item, "active_workloads", "activeWorkloads")
|
||||
|
||||
@@ -1811,9 +1817,9 @@ func (c *Client) GetAppStats(ctx context.Context) (map[string]AppStats, error) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
stats, err = rpc.readAppStatsEvent(ctx, defaultAppStatsIntervalSeconds)
|
||||
stats, err = rpc.readAppStatsEvent(ctx, defaultAppStatsIntervalSeconds, subscriptionName)
|
||||
if err != nil {
|
||||
return discardRPCSessionForStreamError(err)
|
||||
return discardRPCSessionForStreamError("app.stats", err)
|
||||
}
|
||||
return rpc.unsubscribe(ctx, subscriptionID)
|
||||
})
|
||||
@@ -1867,9 +1873,12 @@ func (c *Client) GetAppLogs(ctx context.Context, appName, containerID string, ta
|
||||
return err
|
||||
}
|
||||
var reusable bool
|
||||
lines, reusable, err = rpc.readAppLogEvents(ctx, tailLines)
|
||||
lines, reusable, err = rpc.readAppLogEvents(ctx, tailLines, subscriptionName)
|
||||
if errors.Is(err, errRPCSubscriptionComplete) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return discardRPCSessionForStreamError(err)
|
||||
return discardRPCSessionForStreamError("app.container_log_follow", err)
|
||||
}
|
||||
if !reusable {
|
||||
return errRPCStreamSessionConsumed
|
||||
@@ -2335,6 +2344,8 @@ func appendDiskTemperature(out map[string]int, diskName string, value any) {
|
||||
type trueNASRPCClient struct {
|
||||
conn *websocket.Conn
|
||||
nextID int64
|
||||
openedAt time.Time
|
||||
closeOnce sync.Once
|
||||
keepaliveStop chan struct{}
|
||||
keepaliveDone chan struct{}
|
||||
}
|
||||
@@ -2346,7 +2357,7 @@ func (c *trueNASRPCClient) subscribe(ctx context.Context, event string) (string,
|
||||
}
|
||||
subscriptionID = strings.TrimSpace(subscriptionID)
|
||||
if subscriptionID == "" {
|
||||
return "", &discardRPCSessionError{err: fmt.Errorf("truenas rpc core.subscribe returned an empty subscription id")}
|
||||
return "", &discardRPCSessionError{method: "core.subscribe", err: fmt.Errorf("truenas rpc core.subscribe returned an empty subscription id")}
|
||||
}
|
||||
return subscriptionID, nil
|
||||
}
|
||||
@@ -2357,7 +2368,7 @@ func (c *trueNASRPCClient) unsubscribe(ctx context.Context, subscriptionID strin
|
||||
return fmt.Errorf("truenas rpc subscription id is required")
|
||||
}
|
||||
if err := c.call(ctx, "core.unsubscribe", []any{subscriptionID}, nil); err != nil {
|
||||
return &discardRPCSessionError{err: fmt.Errorf("unsubscribe %q: %w", subscriptionID, err)}
|
||||
return &discardRPCSessionError{method: "core.unsubscribe", err: fmt.Errorf("unsubscribe %q: %w", subscriptionID, err)}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -2648,6 +2659,21 @@ func (c *trueNASRPCClient) call(ctx context.Context, method string, params any,
|
||||
return &RPCTransportError{Method: method, Phase: "read", Err: err}
|
||||
}
|
||||
if message.Method != "" {
|
||||
// A rejection can race with the subscription acknowledgement.
|
||||
// Do not drop it while waiting for the core.subscribe response.
|
||||
if method == "core.subscribe" {
|
||||
if args, ok := params.([]any); ok && len(args) == 1 {
|
||||
if collection, ok := args[0].(string); ok {
|
||||
ended, err := rpcSubscriptionTermination(message, collection, method)
|
||||
if ended || err != nil {
|
||||
if err == nil {
|
||||
err = rpcSubscriptionEndedBeforeData(method)
|
||||
}
|
||||
return discardRPCSessionForStreamError(method, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if message.ID != request.ID {
|
||||
@@ -2666,7 +2692,7 @@ func (c *trueNASRPCClient) call(ctx context.Context, method string, params any,
|
||||
}
|
||||
}
|
||||
|
||||
func (c *trueNASRPCClient) readAppStatsEvent(ctx context.Context, intervalSeconds int) (map[string]AppStats, error) {
|
||||
func (c *trueNASRPCClient) readAppStatsEvent(ctx context.Context, intervalSeconds int, collection string) (map[string]AppStats, error) {
|
||||
if c == nil || c.conn == nil {
|
||||
return nil, fmt.Errorf("truenas rpc connection is nil")
|
||||
}
|
||||
@@ -2682,6 +2708,12 @@ func (c *trueNASRPCClient) readAppStatsEvent(ctx context.Context, intervalSecond
|
||||
if err := c.conn.ReadJSON(&message); err != nil {
|
||||
return nil, &RPCTransportError{Method: "app.stats", Phase: "read", Err: err}
|
||||
}
|
||||
if ended, err := rpcSubscriptionTermination(message, collection, "app.stats"); ended || err != nil {
|
||||
if err == nil {
|
||||
err = rpcSubscriptionEndedBeforeData("app.stats")
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if message.Method == "" {
|
||||
if message.Error != nil {
|
||||
return nil, rpcErrorFromWire("app.stats", message.Error)
|
||||
@@ -2734,7 +2766,7 @@ func (c *trueNASRPCClient) readAppStatsEvent(ctx context.Context, intervalSecond
|
||||
}
|
||||
}
|
||||
|
||||
func (c *trueNASRPCClient) readAppLogEvents(ctx context.Context, tailLines int) ([]AppLogLine, bool, error) {
|
||||
func (c *trueNASRPCClient) readAppLogEvents(ctx context.Context, tailLines int, collection string) ([]AppLogLine, bool, error) {
|
||||
if c == nil || c.conn == nil {
|
||||
return nil, false, fmt.Errorf("truenas rpc connection is nil")
|
||||
}
|
||||
@@ -2780,6 +2812,12 @@ func (c *trueNASRPCClient) readAppLogEvents(ctx context.Context, tailLines int)
|
||||
}
|
||||
return nil, false, &RPCTransportError{Method: "app.container_log_follow", Phase: "read", Err: err}
|
||||
}
|
||||
if ended, err := rpcSubscriptionTermination(message, collection, "app.container_log_follow"); ended || err != nil {
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
return trimAppLogLines(lines, tailLines), true, errRPCSubscriptionComplete
|
||||
}
|
||||
if message.Method == "" {
|
||||
if message.Error != nil {
|
||||
return nil, false, rpcErrorFromWire("app.container_log_follow", message.Error)
|
||||
@@ -3088,7 +3126,7 @@ func (c *trueNASRPCClient) getReportingDataWithQuery(ctx context.Context, graphs
|
||||
return response, nil
|
||||
}
|
||||
|
||||
func (c *trueNASRPCClient) readSystemTelemetryEvent(ctx context.Context, intervalSeconds int) (*SystemInfo, error) {
|
||||
func (c *trueNASRPCClient) readSystemTelemetryEvent(ctx context.Context, intervalSeconds int, collection string) (*SystemInfo, error) {
|
||||
if c == nil || c.conn == nil {
|
||||
return nil, fmt.Errorf("truenas rpc connection is nil")
|
||||
}
|
||||
@@ -3104,6 +3142,12 @@ func (c *trueNASRPCClient) readSystemTelemetryEvent(ctx context.Context, interva
|
||||
if err := c.conn.ReadJSON(&message); err != nil {
|
||||
return nil, &RPCTransportError{Method: "reporting.realtime", Phase: "read", Err: err}
|
||||
}
|
||||
if ended, err := rpcSubscriptionTermination(message, collection, "reporting.realtime"); ended || err != nil {
|
||||
if err == nil {
|
||||
err = rpcSubscriptionEndedBeforeData("reporting.realtime")
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if message.Method == "" {
|
||||
if message.Error != nil {
|
||||
return nil, rpcErrorFromWire("reporting.realtime", message.Error)
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
package truenas
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// A clean terminal event completes a log tail without leaving a subscription
|
||||
// to cancel. It is not a sample for a telemetry stream.
|
||||
var errRPCSubscriptionComplete = errors.New("truenas rpc subscription completed")
|
||||
|
||||
// rpcSubscriptionTermination recognises only the exact collection we subscribed
|
||||
// to, including its arguments. Other subscriptions (even another interval for
|
||||
// the same event source) must not terminate the current reader. middlewared's
|
||||
// notification error is not a JSON-RPC response error: it uses errno/reason
|
||||
// fields, and may contain private names and traces. Retain only its numeric
|
||||
// errno and a fixed message, never that wire text or the collection arguments.
|
||||
func rpcSubscriptionTermination(message trueNASRPCResponse, collection, method string) (bool, error) {
|
||||
if message.Method != "notify_unsubscribed" {
|
||||
return false, nil
|
||||
}
|
||||
var notification struct {
|
||||
Collection string `json:"collection"`
|
||||
Error json.RawMessage `json:"error"`
|
||||
}
|
||||
if err := json.Unmarshal(message.Params, ¬ification); err != nil {
|
||||
return false, fmt.Errorf("invalid truenas %s termination notification", method)
|
||||
}
|
||||
if notification.Collection != collection {
|
||||
return false, nil
|
||||
}
|
||||
if len(notification.Error) == 0 {
|
||||
return true, fmt.Errorf("invalid truenas %s termination notification", method)
|
||||
}
|
||||
if string(notification.Error) == "null" {
|
||||
return true, nil
|
||||
}
|
||||
var failure struct {
|
||||
Errno int `json:"error"`
|
||||
}
|
||||
if err := json.Unmarshal(notification.Error, &failure); err != nil {
|
||||
return true, fmt.Errorf("invalid truenas %s termination error", method)
|
||||
}
|
||||
return true, &RPCError{Method: method, Code: failure.Errno, Message: "subscription rejected"}
|
||||
}
|
||||
|
||||
func rpcSubscriptionEndedBeforeData(method string) error {
|
||||
return &RPCError{Method: method, Message: "subscription ended before a sample"}
|
||||
}
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/gorilla/websocket"
|
||||
"github.com/rs/zerolog/log"
|
||||
)
|
||||
|
||||
var errRPCStreamSessionConsumed = errors.New("truenas rpc stream session cannot be reused")
|
||||
@@ -60,7 +61,8 @@ func (c *Client) rpcOperationContext(ctx context.Context) (context.Context, cont
|
||||
}
|
||||
|
||||
type discardRPCSessionError struct {
|
||||
err error
|
||||
method string
|
||||
err error
|
||||
}
|
||||
|
||||
func (e *discardRPCSessionError) Error() string {
|
||||
@@ -69,7 +71,7 @@ func (e *discardRPCSessionError) Error() string {
|
||||
|
||||
func (e *discardRPCSessionError) Unwrap() error { return e.err }
|
||||
|
||||
func discardRPCSessionForStreamError(err error) error {
|
||||
func discardRPCSessionForStreamError(method string, err error) error {
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
@@ -77,7 +79,7 @@ func discardRPCSessionForStreamError(err error) error {
|
||||
if errors.As(err, &transportErr) {
|
||||
return err
|
||||
}
|
||||
return &discardRPCSessionError{err: err}
|
||||
return &discardRPCSessionError{method: method, err: err}
|
||||
}
|
||||
|
||||
// TransportMode is the connection-local API transport selected for an
|
||||
@@ -326,6 +328,7 @@ func (c *Client) openAuthenticatedRPC(ctx context.Context) (*trueNASRPCClient, s
|
||||
_ = conn.Close()
|
||||
return nil, "", err
|
||||
}
|
||||
rpc.openedAt = time.Now()
|
||||
rpc.startKeepalive(25 * time.Second)
|
||||
return rpc, authMechanism, nil
|
||||
}
|
||||
@@ -354,13 +357,13 @@ func (c *Client) withRPC(ctx context.Context, operation func(context.Context, *t
|
||||
}
|
||||
if err := operation(ctx, c.rpc); err != nil {
|
||||
if errors.Is(err, errRPCStreamSessionConsumed) {
|
||||
c.closeRPCLocked()
|
||||
c.discardRPCLocked("stream_complete", err)
|
||||
c.reconnect = 0
|
||||
return nil
|
||||
}
|
||||
var discardErr *discardRPCSessionError
|
||||
if errors.As(err, &discardErr) {
|
||||
c.closeRPCLocked()
|
||||
c.discardRPCLocked("stream_error", err)
|
||||
c.recordTransportError(err)
|
||||
return err
|
||||
}
|
||||
@@ -369,7 +372,7 @@ func (c *Client) withRPC(ctx context.Context, operation func(context.Context, *t
|
||||
c.recordRPCOperationError(err)
|
||||
return err
|
||||
}
|
||||
c.closeRPCLocked()
|
||||
c.discardRPCLocked("transport_error", err)
|
||||
c.recordTransportError(err)
|
||||
if err := c.waitReconnectBackoff(ctx); err != nil {
|
||||
return err
|
||||
@@ -390,9 +393,18 @@ func (c *Client) withRPC(ctx context.Context, operation func(context.Context, *t
|
||||
status.LastError = ""
|
||||
})
|
||||
if err := operation(ctx, c.rpc); err != nil {
|
||||
if errors.Is(err, errRPCStreamSessionConsumed) {
|
||||
c.discardRPCLocked("stream_complete", err)
|
||||
c.reconnect = 0
|
||||
return nil
|
||||
}
|
||||
var secondDiscardErr *discardRPCSessionError
|
||||
var secondTransportErr *RPCTransportError
|
||||
if errors.As(err, &secondTransportErr) {
|
||||
c.closeRPCLocked()
|
||||
if errors.As(err, &secondDiscardErr) {
|
||||
c.discardRPCLocked("stream_error", err)
|
||||
c.recordTransportError(err)
|
||||
} else if errors.As(err, &secondTransportErr) {
|
||||
c.discardRPCLocked("transport_error", err)
|
||||
c.recordTransportError(err)
|
||||
} else {
|
||||
c.recordRPCOperationError(err)
|
||||
@@ -442,7 +454,7 @@ func (c *Client) callRPCWithRetry(ctx context.Context, method string, params any
|
||||
return err
|
||||
}
|
||||
|
||||
c.closeRPCLocked()
|
||||
c.discardRPCLocked("transport_error", err)
|
||||
c.recordTransportError(err)
|
||||
if !retryRead {
|
||||
return fmt.Errorf("truenas action %s transport failed after dispatch; outcome is unknown and Pulse will not replay it: %w", method, err)
|
||||
@@ -470,7 +482,7 @@ func (c *Client) callRPCWithRetry(ctx context.Context, method string, params any
|
||||
if err := c.rpc.call(ctx, method, params, result); err != nil {
|
||||
var secondTransportErr *RPCTransportError
|
||||
if errors.As(err, &secondTransportErr) {
|
||||
c.closeRPCLocked()
|
||||
c.discardRPCLocked("transport_error", err)
|
||||
c.recordTransportError(err)
|
||||
} else {
|
||||
c.recordRPCOperationError(err)
|
||||
@@ -502,7 +514,13 @@ func (c *Client) waitReconnectBackoff(ctx context.Context) error {
|
||||
}
|
||||
|
||||
func (c *Client) closeRPCLocked() {
|
||||
c.discardRPCLocked("client_closed", nil)
|
||||
}
|
||||
|
||||
func (c *Client) discardRPCLocked(reason string, err error) {
|
||||
if c.rpc != nil && c.rpc.conn != nil {
|
||||
method, category := rpcSessionCloseDetails(err)
|
||||
c.rpc.closeSocket(reason, method, category)
|
||||
c.rpc.close()
|
||||
}
|
||||
c.rpc = nil
|
||||
@@ -531,7 +549,7 @@ func (c *trueNASRPCClient) startKeepalive(interval time.Duration) {
|
||||
if err := c.conn.WriteControl(websocket.PingMessage, nil, time.Now().Add(5*time.Second)); err != nil {
|
||||
// Unblock any in-flight reader; ordinary transport handling owns retry
|
||||
// and status, including the no-replay rule for actions.
|
||||
_ = c.conn.Close()
|
||||
c.closeSocket("keepalive_write_error", "", "transport_error")
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -544,12 +562,68 @@ func (c *trueNASRPCClient) close() {
|
||||
if c.keepaliveStop != nil {
|
||||
close(c.keepaliveStop)
|
||||
}
|
||||
_ = c.conn.Close()
|
||||
c.closeSocket("client_closed", "", "")
|
||||
if c.keepaliveDone != nil {
|
||||
<-c.keepaliveDone
|
||||
}
|
||||
}
|
||||
|
||||
// Record the first local disposal cause, including an idle keepalive failure
|
||||
// which may precede the next poll. Never log wire errors, params, subscription
|
||||
// IDs, endpoints or credentials. These fixed classifications explain a close;
|
||||
// they do not diagnose the appliance or proxy that caused an exchange to fail.
|
||||
func (c *trueNASRPCClient) closeSocket(reason, method, category string) {
|
||||
c.closeOnce.Do(func() {
|
||||
event := log.Info().
|
||||
Str("component", "truenas_rpc").
|
||||
Str("action", "close_session").
|
||||
Str("reason", reason)
|
||||
if !c.openedAt.IsZero() {
|
||||
event.Dur("session_age", time.Since(c.openedAt))
|
||||
}
|
||||
if method != "" {
|
||||
event.Str("method", method)
|
||||
}
|
||||
if category != "" {
|
||||
event.Str("category", category)
|
||||
}
|
||||
event.Msg("Closing TrueNAS RPC session")
|
||||
_ = c.conn.Close()
|
||||
})
|
||||
}
|
||||
|
||||
func rpcSessionCloseDetails(err error) (method, category string) {
|
||||
if err == nil {
|
||||
return "", ""
|
||||
}
|
||||
if errors.Is(err, errRPCStreamSessionConsumed) {
|
||||
return "app.container_log_follow", "idle_timeout"
|
||||
}
|
||||
var discardErr *discardRPCSessionError
|
||||
var transportErr *RPCTransportError
|
||||
var rpcErr *RPCError
|
||||
if errors.As(err, &discardErr) {
|
||||
method = discardErr.method
|
||||
} else if errors.As(err, &transportErr) {
|
||||
method = transportErr.Method
|
||||
} else if errors.As(err, &rpcErr) {
|
||||
method = rpcErr.Method
|
||||
}
|
||||
switch {
|
||||
case errors.Is(err, context.Canceled):
|
||||
category = "cancelled"
|
||||
case errors.Is(err, context.DeadlineExceeded) || isTimeoutError(err):
|
||||
category = "timeout"
|
||||
case errors.As(err, &rpcErr):
|
||||
category = "method_error"
|
||||
case errors.As(err, &transportErr):
|
||||
category = "transport_error"
|
||||
default:
|
||||
category = "protocol_error"
|
||||
}
|
||||
return method, category
|
||||
}
|
||||
|
||||
func (c *Client) recordTransportError(err error) {
|
||||
message := c.sanitizeTransportError(err)
|
||||
c.updateTransportStatus(func(status *TransportStatus) {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package truenas
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
@@ -16,12 +17,196 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/gorilla/websocket"
|
||||
"github.com/rs/zerolog"
|
||||
"github.com/rs/zerolog/log"
|
||||
)
|
||||
|
||||
// Subscription terminal-message regressions for #2396. The envelope is from
|
||||
// the reporter's source-derived 25.04.2.6 example, not a retained wire capture.
|
||||
func TestJSONRPCSubscriptionTerminationRejectsWithoutWaitingOrRetry(t *testing.T) {
|
||||
for _, stream := range []string{"app.stats", "reporting.realtime", "app.container_log_follow"} {
|
||||
for _, beforeReply := range []bool{false, true} {
|
||||
t.Run(fmt.Sprintf("%s/before_reply=%t", stream, beforeReply), func(t *testing.T) {
|
||||
var subscribes, unsubscribes atomic.Int32
|
||||
fixture := newProtocolFixture(t, func(_ int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
switch request.Method {
|
||||
case "auth.login_ex":
|
||||
return protocolFixtureReply{result: map[string]any{"response_type": "SUCCESS"}}
|
||||
case "reporting.get_data":
|
||||
return protocolFixtureReply{result: []any{}}
|
||||
case "core.subscribe":
|
||||
subscribes.Add(1)
|
||||
collection := request.Params.([]any)[0].(string)
|
||||
return protocolFixtureReply{result: "private-sub-id", beforeReply: beforeReply, notifications: []protocolFixtureNotification{{
|
||||
method: "notify_unsubscribed", params: map[string]any{"collection": collection, "error": map[string]any{
|
||||
"error": 14, "errname": "EFAULT", "reason": "private-app-name fixture-key fixture-user https://private.invalid",
|
||||
"trace": "private-trace", "extra": "private-extra",
|
||||
}},
|
||||
}}}
|
||||
case "core.unsubscribe":
|
||||
unsubscribes.Add(1)
|
||||
return protocolFixtureReply{result: nil}
|
||||
case "pool.query":
|
||||
return protocolFixtureReply{result: []any{}}
|
||||
default:
|
||||
t.Errorf("unexpected method %s", request.Method)
|
||||
return protocolFixtureReply{close: true}
|
||||
}
|
||||
}, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "fixture-key", Username: "fixture-user", Timeout: 2 * time.Second})
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
started := time.Now()
|
||||
var err error
|
||||
switch stream {
|
||||
case "app.stats":
|
||||
var stats map[string]AppStats
|
||||
stats, err = client.GetAppStats(ctx)
|
||||
if stats != nil {
|
||||
t.Error("rejected stats fabricated a sample")
|
||||
}
|
||||
case "reporting.realtime":
|
||||
var telemetry *SystemInfo
|
||||
telemetry, err = client.GetSystemTelemetry(ctx)
|
||||
if telemetry != nil {
|
||||
t.Error("rejected telemetry fabricated a sample")
|
||||
}
|
||||
default:
|
||||
var lines []AppLogLine
|
||||
lines, err = client.GetAppLogs(ctx, "private-app-name", "private-container", 100)
|
||||
if lines != nil {
|
||||
t.Error("rejected logs appeared successful")
|
||||
}
|
||||
}
|
||||
var rpcErr *RPCError
|
||||
if !errors.As(err, &rpcErr) || rpcErr.Code != 14 || rpcErr.Message != "subscription rejected" {
|
||||
t.Fatalf("termination = %v, want structured errno=14 rejection", err)
|
||||
}
|
||||
if elapsed := time.Since(started); elapsed >= time.Second {
|
||||
t.Fatalf("terminal event waited for the operation budget: %s", elapsed)
|
||||
}
|
||||
for _, private := range []string{"private-app-name", "fixture-key", "fixture-user", "private.invalid", "private-trace", "private-extra", "private-sub-id", "private-container"} {
|
||||
if strings.Contains(err.Error(), private) || strings.Contains(client.TransportStatus().LastError, private) {
|
||||
t.Fatalf("termination exposed %s", private)
|
||||
}
|
||||
}
|
||||
if client.rpc != nil || client.TransportStatus().Connected || subscribes.Load() != 1 || unsubscribes.Load() != 0 || fixture.sessions.Load() != 1 {
|
||||
t.Fatal("terminal rejection retried, reused the stream or tried to cancel its ended subscription")
|
||||
}
|
||||
if _, err := client.GetPools(ctx); err != nil || fixture.sessions.Load() != 2 || fixture.restRequests.Load() != 0 {
|
||||
t.Fatalf("next read did not recover on a fresh RPC session: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONRPCSubscriptionTerminationIgnoresOtherCollections(t *testing.T) {
|
||||
var unsubscribes atomic.Int32
|
||||
fixture := newProtocolFixture(t, func(_ int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
switch request.Method {
|
||||
case "auth.login_ex":
|
||||
return protocolFixtureReply{result: map[string]any{"response_type": "SUCCESS"}}
|
||||
case "core.subscribe":
|
||||
collection := request.Params.([]any)[0].(string)
|
||||
return protocolFixtureReply{result: "stats-sub", notifications: []protocolFixtureNotification{
|
||||
{method: "notify_unsubscribed", params: map[string]any{"collection": "app.stats:{\"interval\":99}", "error": map[string]any{"error": 14}}},
|
||||
{method: "notify_unsubscribed", params: map[string]any{"collection": "app.stats.extra:{\"interval\":2}", "error": "malformed-unrelated-error"}},
|
||||
{method: "notify_unsubscribed", params: map[string]any{"collection": "reporting.realtime:{\"interval\":2}", "error": nil}},
|
||||
{method: "collection_update", params: map[string]any{"collection": collection, "fields": []any{map[string]any{"app_name": "fixture-app", "cpu_usage": 7}}}},
|
||||
}}
|
||||
case "core.unsubscribe":
|
||||
unsubscribes.Add(1)
|
||||
return protocolFixtureReply{result: nil}
|
||||
default:
|
||||
return protocolFixtureReply{result: []any{}}
|
||||
}
|
||||
}, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "fixture-key", Username: "fixture-user"})
|
||||
for poll := 0; poll < 2; poll++ {
|
||||
stats, err := client.GetAppStats(context.Background())
|
||||
if err != nil || stats["fixture-app"].CPUPercent != 7 {
|
||||
t.Fatalf("unrelated terminal event interrupted stats: %v, %+v", err, stats)
|
||||
}
|
||||
}
|
||||
if fixture.sessions.Load() != 1 || unsubscribes.Load() != 2 || !client.TransportStatus().Connected {
|
||||
t.Fatal("unrelated event prevented ordinary cleanup and session reuse")
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONRPCSubscriptionCleanTerminationDoesNotInventStats(t *testing.T) {
|
||||
fixture := newProtocolFixture(t, func(_ int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
if request.Method == "auth.login_ex" {
|
||||
return protocolFixtureReply{result: map[string]any{"response_type": "SUCCESS"}}
|
||||
}
|
||||
if request.Method == "core.subscribe" {
|
||||
return protocolFixtureReply{result: "stats-sub", notifications: []protocolFixtureNotification{{method: "notify_unsubscribed", params: map[string]any{"collection": request.Params.([]any)[0], "error": nil}}}}
|
||||
}
|
||||
return protocolFixtureReply{result: []any{}}
|
||||
}, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "fixture-key", Username: "fixture-user", Timeout: 2 * time.Second})
|
||||
stats, err := client.GetAppStats(context.Background())
|
||||
var rpcErr *RPCError
|
||||
if stats != nil || !errors.As(err, &rpcErr) || rpcErr.Message != "subscription ended before a sample" || client.TransportStatus().Connected {
|
||||
t.Fatalf("clean termination appeared sampled or left an unresolved stream: %v, %+v", err, stats)
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONRPCSubscriptionCleanLogEndKeepsSession(t *testing.T) {
|
||||
var unsubscribes atomic.Int32
|
||||
fixture := newProtocolFixture(t, func(_ int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
if request.Method == "auth.login_ex" {
|
||||
return protocolFixtureReply{result: map[string]any{"response_type": "SUCCESS"}}
|
||||
}
|
||||
if request.Method == "core.subscribe" {
|
||||
collection := request.Params.([]any)[0]
|
||||
return protocolFixtureReply{result: "log-sub", notifications: []protocolFixtureNotification{
|
||||
{method: "collection_update", params: map[string]any{"collection": collection, "fields": []any{map[string]any{"data": "fixture-line"}}}},
|
||||
{method: "notify_unsubscribed", params: map[string]any{"collection": "app.container_log_follow:{}", "error": map[string]any{"error": 14}}},
|
||||
{method: "notify_unsubscribed", params: map[string]any{"collection": collection, "error": nil}},
|
||||
}}
|
||||
}
|
||||
if request.Method == "core.unsubscribe" {
|
||||
unsubscribes.Add(1)
|
||||
}
|
||||
return protocolFixtureReply{result: []any{}}
|
||||
}, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "fixture-key", Username: "fixture-user"})
|
||||
lines, err := client.GetAppLogs(context.Background(), "fixture-app", "fixture-container", 100)
|
||||
if err != nil || len(lines) != 1 || lines[0].Data != "fixture-line" {
|
||||
t.Fatalf("clean log end lost the bounded tail: %v, %+v", err, lines)
|
||||
}
|
||||
if _, err := client.GetPools(context.Background()); err != nil || fixture.sessions.Load() != 1 || unsubscribes.Load() != 0 || !client.TransportStatus().Connected {
|
||||
t.Fatalf("clean log end discarded or unsubscribed an already-ended stream: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONRPCSubscriptionMalformedTerminalDiscardsWithoutWireText(t *testing.T) {
|
||||
for _, terminal := range []any{map[string]any{"error": "private-invalid-errno", "reason": "private-error"}, "private-error", true} {
|
||||
t.Run(fmt.Sprintf("%T", terminal), func(t *testing.T) {
|
||||
fixture := newProtocolFixture(t, func(_ int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
if request.Method == "auth.login_ex" {
|
||||
return protocolFixtureReply{result: map[string]any{"response_type": "SUCCESS"}}
|
||||
}
|
||||
if request.Method == "core.subscribe" {
|
||||
return protocolFixtureReply{result: "stats-sub", notifications: []protocolFixtureNotification{{method: "notify_unsubscribed", params: map[string]any{"collection": request.Params.([]any)[0], "error": terminal}}}}
|
||||
}
|
||||
return protocolFixtureReply{result: []any{}}
|
||||
}, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "fixture-key", Username: "fixture-user", Timeout: 2 * time.Second})
|
||||
_, err := client.GetAppStats(context.Background())
|
||||
if err == nil || strings.Contains(err.Error(), "private-") || !strings.Contains(err.Error(), "invalid truenas app.stats termination error") || client.rpc != nil || fixture.sessions.Load() != 1 {
|
||||
t.Fatalf("malformed termination was lost, leaked or reused: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type protocolFixtureReply struct {
|
||||
result any
|
||||
err *trueNASRPCError
|
||||
notifications []protocolFixtureNotification
|
||||
beforeReply bool
|
||||
close bool
|
||||
}
|
||||
|
||||
@@ -86,8 +271,10 @@ func newProtocolFixture(
|
||||
}
|
||||
response.Result = raw
|
||||
}
|
||||
if err := conn.WriteJSON(response); err != nil {
|
||||
return
|
||||
if !reply.beforeReply {
|
||||
if err := conn.WriteJSON(response); err != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
for _, notification := range reply.notifications {
|
||||
params, err := json.Marshal(notification.params)
|
||||
@@ -103,6 +290,11 @@ func newProtocolFixture(
|
||||
return
|
||||
}
|
||||
}
|
||||
if reply.beforeReply {
|
||||
if err := conn.WriteJSON(response); err != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}))
|
||||
t.Cleanup(fixture.server.Close)
|
||||
@@ -786,8 +978,8 @@ func TestJSONRPCFetchSnapshotUsesPublishedTrueNAS26MethodsWithoutREST(t *testing
|
||||
subscribeCalls := methodCalls["core.subscribe"]
|
||||
unsubscribeCalls := methodCalls["core.unsubscribe"]
|
||||
methodCallsMu.Unlock()
|
||||
if subscribeCalls != 2 || unsubscribeCalls != 2 {
|
||||
t.Fatalf("subscription lifecycle calls: subscribe=%d unsubscribe=%d, want 2/2", subscribeCalls, unsubscribeCalls)
|
||||
if subscribeCalls != 1 || unsubscribeCalls != 1 {
|
||||
t.Fatalf("subscription lifecycle calls: subscribe=%d unsubscribe=%d, want 1/1 (realtime only for empty apps)", subscribeCalls, unsubscribeCalls)
|
||||
}
|
||||
if fixture.sessions.Load() != 1 || fixture.restRequests.Load() != 0 {
|
||||
t.Fatalf("snapshot sessions=%d REST=%d, want one modern session and no REST",
|
||||
@@ -1419,3 +1611,272 @@ func TestJSONRPCSnapshotContinuesAfterTelemetryTimeout(t *testing.T) {
|
||||
t.Fatalf("inventory did not recover on RPC after discarding the stream: sessions=%d rest=%d", fixture.sessions.Load(), fixture.restRequests.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONRPCEmptyAppsDoNotSubscribeToStats(t *testing.T) {
|
||||
var appQueries, statsSubscriptions atomic.Int32
|
||||
fixture := newProtocolFixture(t, func(_ int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
switch request.Method {
|
||||
case "auth.login_ex":
|
||||
return protocolFixtureReply{result: map[string]any{"response_type": "SUCCESS"}}
|
||||
case "app.query":
|
||||
if appQueries.Add(1) == 1 {
|
||||
return protocolFixtureReply{result: nil}
|
||||
}
|
||||
return protocolFixtureReply{result: []any{}}
|
||||
case "core.subscribe":
|
||||
statsSubscriptions.Add(1)
|
||||
// Deliberately silent stats on an appliance with no apps. The final
|
||||
// client must never issue this request, even with a cancellation-only
|
||||
// caller context like the production poller.
|
||||
return protocolFixtureReply{result: "unavailable-apps"}
|
||||
case "system.info":
|
||||
return protocolFixtureReply{result: map[string]any{"hostname": "empty-app-nas", "version": "TrueNAS-SCALE-25.04.2.6"}}
|
||||
default:
|
||||
t.Errorf("unexpected method %s", request.Method)
|
||||
return protocolFixtureReply{close: true}
|
||||
}
|
||||
}, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "fixture-key", Username: "fixture-user", Timeout: 100 * time.Millisecond})
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
for i := 0; i < 2; i++ {
|
||||
apps, err := client.GetApps(ctx)
|
||||
if err != nil || apps == nil || len(apps) != 0 {
|
||||
t.Fatalf("empty app query %d did not preserve an empty successful inventory", i)
|
||||
}
|
||||
}
|
||||
if statsSubscriptions.Load() != 0 {
|
||||
t.Fatal("empty inventory still issued a stats subscription")
|
||||
}
|
||||
if _, err := client.GetSystemInfo(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if fixture.sessions.Load() != 1 || fixture.restRequests.Load() != 0 || !client.TransportStatus().Connected {
|
||||
t.Fatal("empty apps discarded the healthy authenticated session or changed transport")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyRESTEmptyAppsDoNotNegotiateStats(t *testing.T) {
|
||||
var appQueries atomic.Int32
|
||||
fixture := newProtocolFixture(t, func(_ int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
t.Errorf("empty legacy app inventory issued RPC method %s", request.Method)
|
||||
return protocolFixtureReply{close: true}
|
||||
}, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path != "/api/v2.0/app" {
|
||||
t.Errorf("unexpected legacy path %s", r.URL.Path)
|
||||
return
|
||||
}
|
||||
appQueries.Add(1)
|
||||
_ = json.NewEncoder(w).Encode([]any{})
|
||||
})
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "fixture-key"})
|
||||
// Negotiation is covered separately; isolate the already-selected legacy
|
||||
// inventory path here, so no attempted JSON-RPC stats lookup is hidden.
|
||||
client.mode = TransportLegacyREST
|
||||
apps, err := client.GetApps(context.Background())
|
||||
if err != nil || apps == nil || len(apps) != 0 || appQueries.Load() != 1 || fixture.sessions.Load() != 0 {
|
||||
t.Fatal("empty legacy app inventory did not stay a single REST read")
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONRPCStreamRetryDiscardsUnusableSecondSession(t *testing.T) {
|
||||
for _, failure := range []string{"malformed_event", "empty_subscription", "unsubscribe_error"} {
|
||||
t.Run(failure, func(t *testing.T) {
|
||||
var subscribes atomic.Int32
|
||||
fixture := newProtocolFixture(t, func(session int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
switch request.Method {
|
||||
case "auth.login_ex":
|
||||
return protocolFixtureReply{result: map[string]any{"response_type": "SUCCESS"}}
|
||||
case "core.subscribe":
|
||||
subscribes.Add(1)
|
||||
if session == 1 {
|
||||
return protocolFixtureReply{close: true}
|
||||
}
|
||||
if failure == "empty_subscription" {
|
||||
return protocolFixtureReply{result: ""}
|
||||
}
|
||||
var fields any = []any{map[string]any{"app_name": "fixture-app", "cpu_usage": 3}}
|
||||
if failure == "malformed_event" {
|
||||
fields = "invalid-fields"
|
||||
}
|
||||
return protocolFixtureReply{result: "private-subscription-id", notifications: []protocolFixtureNotification{{method: "collection_update", params: map[string]any{"collection": "app.stats:{\"interval\":2}", "fields": fields}}}}
|
||||
case "core.unsubscribe":
|
||||
return protocolFixtureReply{err: &trueNASRPCError{Code: -32001, Message: "fixture-method-error"}}
|
||||
case "pool.query":
|
||||
if session != 3 {
|
||||
t.Error("next read reused an unusable second stream session")
|
||||
}
|
||||
return protocolFixtureReply{result: []any{}}
|
||||
default:
|
||||
t.Errorf("unexpected method %s", request.Method)
|
||||
return protocolFixtureReply{close: true}
|
||||
}
|
||||
}, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "fixture-key", Username: "fixture-user"})
|
||||
if _, err := client.GetAppStats(context.Background()); err == nil {
|
||||
t.Fatal("bad second stream session appeared successful")
|
||||
}
|
||||
if client.TransportStatus().Connected {
|
||||
t.Fatal("unusable second stream session remained connected")
|
||||
}
|
||||
if client.rpc != nil {
|
||||
t.Fatal("unusable second stream session was retained")
|
||||
}
|
||||
if _, err := client.GetPools(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if subscribes.Load() != 2 || fixture.sessions.Load() != 3 || fixture.restRequests.Load() != 0 {
|
||||
t.Fatal("stream retry/disposal changed its retry limit or downgraded transport")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONRPCStreamRetryReturnsCompletedLogTail(t *testing.T) {
|
||||
fixture := newProtocolFixture(t, func(session int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
switch request.Method {
|
||||
case "auth.login_ex":
|
||||
return protocolFixtureReply{result: map[string]any{"response_type": "SUCCESS"}}
|
||||
case "core.subscribe":
|
||||
if session == 1 {
|
||||
return protocolFixtureReply{close: true}
|
||||
}
|
||||
return protocolFixtureReply{result: "fixture-log-sub", notifications: []protocolFixtureNotification{{method: "collection_update", params: map[string]any{"collection": "app.container_log_follow:{}", "fields": []any{map[string]any{"data": "fixture-line"}}}}}}
|
||||
case "pool.query":
|
||||
if session != 3 {
|
||||
t.Error("timed-out log socket was reused")
|
||||
}
|
||||
return protocolFixtureReply{result: []any{}}
|
||||
default:
|
||||
t.Errorf("unexpected method %s", request.Method)
|
||||
return protocolFixtureReply{close: true}
|
||||
}
|
||||
}, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "fixture-key", Username: "fixture-user"})
|
||||
lines, err := client.GetAppLogs(context.Background(), "fixture-app", "fixture-container", 100)
|
||||
if err != nil || len(lines) != 1 || lines[0].Data != "fixture-line" {
|
||||
t.Fatal("completed retry lost its log tail or returned a consumed-session sentinel")
|
||||
}
|
||||
if client.TransportStatus().Connected || client.rpc != nil {
|
||||
t.Fatal("completed retry kept a timed-out log socket")
|
||||
}
|
||||
if _, err := client.GetPools(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if fixture.sessions.Load() != 3 || fixture.restRequests.Load() != 0 {
|
||||
t.Fatal("next read did not open exactly one fresh modern session")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRPCSessionCloseDetails(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
err error
|
||||
method, category string
|
||||
}{
|
||||
{"explicit", nil, "", ""},
|
||||
{"idle tail", errRPCStreamSessionConsumed, "app.container_log_follow", "idle_timeout"},
|
||||
{"timeout", &RPCTransportError{Method: "pool.query", Phase: "read", Err: context.DeadlineExceeded}, "pool.query", "timeout"},
|
||||
{"cancel", &RPCTransportError{Method: "app.start", Phase: "read", Err: context.Canceled}, "app.start", "cancelled"},
|
||||
{"transport", &RPCTransportError{Method: "system.info", Phase: "read", Err: errors.New("private-error-text")}, "system.info", "transport_error"},
|
||||
{"stream decode", &discardRPCSessionError{method: "app.stats", err: errors.New("private-event-text")}, "app.stats", "protocol_error"},
|
||||
{"unsubscribe refusal", &discardRPCSessionError{method: "core.unsubscribe", err: &RPCError{Method: "core.unsubscribe", Message: "private-wire-message"}}, "core.unsubscribe", "method_error"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
method, category := rpcSessionCloseDetails(tc.err)
|
||||
if method != tc.method || category != tc.category {
|
||||
t.Fatalf("close classification = %s/%s, want %s/%s", method, category, tc.method, tc.category)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func captureRPCCloseLogs(t *testing.T) *bytes.Buffer {
|
||||
t.Helper()
|
||||
var output bytes.Buffer
|
||||
previous := log.Logger
|
||||
log.Logger = zerolog.New(&output)
|
||||
t.Cleanup(func() { log.Logger = previous })
|
||||
return &output
|
||||
}
|
||||
|
||||
func rpcCloseLogRecords(t *testing.T, output *bytes.Buffer) []map[string]any {
|
||||
t.Helper()
|
||||
var records []map[string]any
|
||||
decoder := json.NewDecoder(bytes.NewReader(output.Bytes()))
|
||||
for decoder.More() {
|
||||
var record map[string]any
|
||||
if err := decoder.Decode(&record); err != nil {
|
||||
t.Fatal("close log is not structured JSON")
|
||||
}
|
||||
records = append(records, record)
|
||||
}
|
||||
return records
|
||||
}
|
||||
|
||||
func TestRPCSessionCloseLogIsBoundedAndSecretFree(t *testing.T) {
|
||||
output := captureRPCCloseLogs(t)
|
||||
fixture := newProtocolFixture(t, func(_ int, request trueNASRPCRequest) protocolFixtureReply {
|
||||
switch request.Method {
|
||||
case "auth.login_ex":
|
||||
return protocolFixtureReply{result: map[string]any{"response_type": "SUCCESS"}}
|
||||
case "core.subscribe":
|
||||
return protocolFixtureReply{result: "private-subscription-id", notifications: []protocolFixtureNotification{{method: "collection_update", params: map[string]any{"collection": "app.stats:{\"interval\":2}", "fields": []any{}}}}}
|
||||
case "core.unsubscribe":
|
||||
return protocolFixtureReply{err: &trueNASRPCError{Code: -32001, Message: "private-wire-message private-api-key private-password private-subscription-id"}}
|
||||
default:
|
||||
return protocolFixtureReply{close: true}
|
||||
}
|
||||
}, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{APIKey: "private-api-key", Password: "private-password", Username: "private-owner"})
|
||||
if _, err := client.GetAppStats(context.Background()); err == nil {
|
||||
t.Fatal("fixture did not fail its unsubscribe")
|
||||
}
|
||||
client.Close()
|
||||
records := rpcCloseLogRecords(t, output)
|
||||
if len(records) != 1 {
|
||||
t.Fatalf("close records = %d, want exactly one for disposal and subsequent Close", len(records))
|
||||
}
|
||||
record := records[0]
|
||||
if record["reason"] != "stream_error" || record["method"] != "core.unsubscribe" || record["category"] != "method_error" || record["component"] != "truenas_rpc" || record["action"] != "close_session" {
|
||||
t.Fatal("close log lost its fixed local cause")
|
||||
}
|
||||
if _, ok := record["session_age"].(float64); !ok {
|
||||
t.Fatal("close log lost the authenticated session age")
|
||||
}
|
||||
for _, private := range []string{"private-api-key", "private-password", "private-owner", "private-wire-message", "private-subscription-id", fixture.server.URL} {
|
||||
if bytes.Contains(output.Bytes(), []byte(private)) {
|
||||
t.Fatal("close log contains private wire/configuration data")
|
||||
}
|
||||
}
|
||||
for key := range record {
|
||||
switch key {
|
||||
case "component", "action", "reason", "method", "category", "session_age", "level", "message":
|
||||
default:
|
||||
t.Fatalf("unexpected close log field %s", key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRPCSessionCloseLogKeepsFirstKeepaliveCause(t *testing.T) {
|
||||
output := captureRPCCloseLogs(t)
|
||||
fixture := newProtocolFixture(t, func(_ int, _ trueNASRPCRequest) protocolFixtureReply { return protocolFixtureReply{result: true} }, nil)
|
||||
client := protocolFixtureClient(t, fixture.server.URL, ClientConfig{})
|
||||
conn, err := client.dialRPC(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rpc := &trueNASRPCClient{conn: conn, nextID: 1, openedAt: time.Now()}
|
||||
rpc.startKeepalive(time.Millisecond)
|
||||
_ = conn.Close() // Inject a control-write failure, without an RPC payload.
|
||||
select {
|
||||
case <-rpc.keepaliveDone:
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("failed keepalive did not stop")
|
||||
}
|
||||
rpc.close()
|
||||
records := rpcCloseLogRecords(t, output)
|
||||
if len(records) != 1 || records[0]["reason"] != "keepalive_write_error" || records[0]["category"] != "transport_error" {
|
||||
t.Fatal("later client disposal overwrote or duplicated the first keepalive close cause")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -125,6 +125,17 @@ class PBSDocsTest(unittest.TestCase):
|
||||
"Do not delete backups", "Do not clear History"):
|
||||
self.assertIn(phrase, text)
|
||||
|
||||
def test_history_checks_cover_batched_and_single_metric_requests(self):
|
||||
history = DOC.read_text().split("### PBS is connected but History stays empty\n")[1].split("\n### ")[0]
|
||||
for phrase in ("without\n `metric`", "do not wait for a separate `metric=cpu` request",
|
||||
"`metrics: {}`", "`metrics.cpu` array", "CPU is absent",
|
||||
"`points` array for that single metric", "A measured zero is a\n sample",
|
||||
"Do not edit or replay", "Do not share a HAR", "enable\nDebug logging"):
|
||||
with self.subTest(phrase=phrase):
|
||||
self.assertIn(phrase, history)
|
||||
self.assertNotIn("completed request with `metric=cpu`", history)
|
||||
self.assertIn("does\n not prove there is no data under another target", history)
|
||||
|
||||
def test_every_documented_shell_recipe_parses(self):
|
||||
for command in blocks():
|
||||
with self.subTest(command=command):
|
||||
|
||||
Reference in New Issue
Block a user