mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-26 13:39:47 +00:00
Distinguish Active PBS connections from missing monitoring data
Add a safe missing-PBS path to the existing guide and troubleshooting entry. Separate connection polling, the server row, direct inventory and History without requesting credential changes, an agent install or repeated diagnostics. Contract-Neutral: Documentation and consuming proof only; no API, monitoring, permissions or runtime behaviour change. Change-source: pulse-maintainer
This commit is contained in:
+31
@@ -203,6 +203,37 @@ If you have multiple PBS servers, add each one separately in Settings. Pulse wil
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### PBS is Active but missing from monitoring
|
||||
|
||||
**Active** in **Settings → Infrastructure** describes the configured
|
||||
connection's polling health. It does not establish that Pulse has a matching
|
||||
PBS resource, every datastore or fresh readings. A direct PBS API connection
|
||||
does not require a Pulse Agent; adding one does not prove that the API path
|
||||
recovered.
|
||||
|
||||
Use the existing views to distinguish what is missing:
|
||||
|
||||
| Observation | Where to look next |
|
||||
| --- | --- |
|
||||
| The PBS server itself is missing | Open **Proxmox → Backups** and look for its **Backup server** row. Note any existing search or view filter. The PBS VM/LXC in **Overview** is a different resource. If Proxmox navigation is absent too, retain that fact. |
|
||||
| The server row exists, but datastores or backups are missing | Compare the affected view with the connection's existing collection scope. PVE passthrough backups do not establish that direct PBS inventory is available. Do not widen permissions or change scope just to diagnose the gap. |
|
||||
| Current readings exist, but a chart is empty | Use the [PBS History checks](#pbs-is-connected-but-history-stays-empty) for that server and time range. An empty chart is not the same as a missing server. |
|
||||
|
||||
When reporting a missing server, keep the Pulse server version separate from
|
||||
the PBS version and whether a Pulse Agent is installed. Say which existing
|
||||
views lack it, whether Infrastructure says Active, and whether it was visible
|
||||
before. A last known working version is useful only if already known; unknown
|
||||
is valid. Use consistent aliases for private server names. Existing error text
|
||||
or screenshots are enough to start; do not run Diagnostics, enable Debug or
|
||||
collect a full response just to fill the report.
|
||||
|
||||
Do not delete and re-add the connection, replace tokens, install an agent,
|
||||
restart PBS or Pulse, clear History or run a backup to make the server appear.
|
||||
These actions can lose evidence or change the affected path. Keep certificate
|
||||
verification and existing access restrictions intact. If the original incident
|
||||
also involved an unresponsive guest or uncertain freeze/thaw, follow
|
||||
[Backup safety](VM_DISK_MONITORING.md#backup-safety) before any live checks.
|
||||
|
||||
### "Connection Failed" Error
|
||||
|
||||
Check from the Pulse host or container's network, if possible. A request from
|
||||
|
||||
@@ -39,6 +39,10 @@ establish safe recovery.
|
||||
- **TrueNAS readings are stale or absent:** use
|
||||
[TrueNAS data checks](TRUENAS.md#stale-truenas-data), keeping the platform API
|
||||
and Pulse Agent paths separate.
|
||||
- **PBS is Active but missing from monitoring:** distinguish the
|
||||
[server, backup inventory and History](PBS.md#pbs-is-active-but-missing-from-monitoring).
|
||||
An Active API connection does not establish a visible PBS resource; installing
|
||||
an agent does not prove API recovery.
|
||||
- **Pulse itself uses more memory, CPU or disk writes:** use
|
||||
[memory baseline and growth checks](TROUBLESHOOTING.md#memory-use-keeps-growing)
|
||||
or [CPU and write-rate checks](TROUBLESHOOTING.md#excessive-cpu-writes-or-database-growth).
|
||||
|
||||
@@ -55,7 +55,7 @@ try {
|
||||
for (const file of ['package.json', 'package-lock.json', 'tsconfig.json', 'index.html'])
|
||||
binding.sources[file] = hash(fs.readFileSync(path.join(root, file)));
|
||||
fs.mkdirSync(path.join(output, 'docs'), { recursive: true });
|
||||
for (const name of ['FAQ', 'UNIFIED_AGENT', 'TROUBLESHOOTING', 'DOCKER']) {
|
||||
for (const name of ['FAQ', 'UNIFIED_AGENT', 'TROUBLESHOOTING', 'DOCKER', 'PBS']) {
|
||||
const file = `public/docs/${name}.md`;
|
||||
binding.sources[file] = hash(fs.readFileSync(path.join(root, file)));
|
||||
fs.copyFileSync(path.join(root, file), path.join(output, 'docs', `${name}.md`));
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
// Production Docs renderer/router/CSS and exact shipped Markdown, no PBS call.
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const http = require('node:http');
|
||||
const crypto = require('node:crypto');
|
||||
const assert = require('node:assert/strict');
|
||||
const { chromium, webkit } = require('playwright');
|
||||
|
||||
const root = '/workspace';
|
||||
const frontend = path.join(root, 'frontend-modern');
|
||||
const evidence = path.join(root, 'tmp/pbs-missing-help');
|
||||
const preview = path.join(root, 'tmp/doc-command-help/static');
|
||||
const hash = bytes => crypto.createHash('sha256').update(bytes).digest('hex');
|
||||
const bindingBytes = fs.readFileSync(path.join(preview, 'binding.json'));
|
||||
const binding = JSON.parse(bindingBytes);
|
||||
const fragment = 'pbs-is-active-but-missing-from-monitoring';
|
||||
const result = { source: binding.source_sha, binding_sha256: hash(bindingBytes),
|
||||
cases: [], captures: [], errors: [], backendRequests: [], mutations: [], served: [] };
|
||||
|
||||
(async () => {
|
||||
fs.mkdirSync(evidence, { recursive: true });
|
||||
let server, browser;
|
||||
try {
|
||||
for (const [file, expected] of Object.entries(binding.sources))
|
||||
assert.equal(hash(fs.readFileSync(path.join(frontend, file))), expected, file);
|
||||
for (const [file, expected] of Object.entries(binding.artifacts))
|
||||
assert.equal(hash(fs.readFileSync(path.join(preview, file))), expected, file);
|
||||
for (const name of ['PBS', 'TROUBLESHOOTING'])
|
||||
assert.deepEqual(fs.readFileSync(path.join(preview, 'docs', `${name}.md`)),
|
||||
fs.readFileSync(path.join(root, 'docs', `${name}.md`)));
|
||||
result.source_inputs = Object.keys(binding.sources).length;
|
||||
server = http.createServer((request, response) => {
|
||||
const route = new URL(request.url, 'http://fixture.invalid').pathname;
|
||||
let asset = route.slice(1);
|
||||
if (route.startsWith('/docs/') && !route.endsWith('.md'))
|
||||
asset = 'browser-tests/doc-command-help.html';
|
||||
if (!Object.hasOwn(binding.artifacts, asset)) {
|
||||
response.writeHead(404); response.end(); return;
|
||||
}
|
||||
const bytes = fs.readFileSync(path.join(preview, asset));
|
||||
assert.equal(hash(bytes), binding.artifacts[asset]);
|
||||
result.served.push({ route, asset, sha256: hash(bytes) });
|
||||
const type = asset.endsWith('.md') ? 'text/markdown; charset=utf-8' :
|
||||
asset.endsWith('.js') ? 'text/javascript' : asset.endsWith('.css') ? 'text/css' : 'text/html';
|
||||
response.writeHead(200, { 'Content-Type': type, 'Cache-Control': 'no-store' });
|
||||
response.end(bytes);
|
||||
});
|
||||
await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
|
||||
const origin = `http://127.0.0.1:${server.address().port}`;
|
||||
for (const state of [
|
||||
{ engine: 'chromium', name: 'desktop-light', width: 1280, height: 900, dark: false },
|
||||
{ engine: 'webkit', name: 'phone-dark', width: 390, height: 900, dark: true },
|
||||
]) {
|
||||
browser = await { chromium, webkit }[state.engine].launch(state.engine === 'chromium'
|
||||
? { headless: true, channel: 'chromium', args: ['--no-sandbox'] } : { headless: true });
|
||||
const context = await browser.newContext({ viewport: { width: state.width, height: state.height },
|
||||
isMobile: state.width < 500, hasTouch: state.width < 500 });
|
||||
const page = await context.newPage();
|
||||
page.on('pageerror', error => result.errors.push(error.message));
|
||||
page.on('request', request => {
|
||||
if (!['GET', 'HEAD'].includes(request.method())) result.mutations.push(request.method());
|
||||
});
|
||||
await page.route('**/api/**', async route => {
|
||||
result.backendRequests.push(new URL(route.request().url()).pathname); await route.abort();
|
||||
});
|
||||
await page.addInitScript(dark => {
|
||||
document.addEventListener('DOMContentLoaded', () => document.documentElement.classList.toggle('dark', dark));
|
||||
const fetch = window.fetch.bind(window);
|
||||
window.__helpDocuments = [];
|
||||
window.fetch = async (...args) => {
|
||||
const response = await fetch(...args);
|
||||
if (/\/docs\/(PBS|TROUBLESHOOTING)\.md$/.test(new URL(response.url).pathname))
|
||||
window.__helpDocuments.push({ path: new URL(response.url).pathname, status: response.status,
|
||||
bytes: Array.from(new Uint8Array(await response.clone().arrayBuffer())) });
|
||||
return response;
|
||||
};
|
||||
}, state.dark);
|
||||
const activate = async link => {
|
||||
if (state.width < 500) await link.tap();
|
||||
else { await link.focus(); await page.keyboard.press('Enter'); }
|
||||
};
|
||||
const capture = async (label, locator) => {
|
||||
await locator.evaluate(element => window.scrollBy(0, element.getBoundingClientRect().top - 64));
|
||||
await page.evaluate(() => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve))));
|
||||
assert.ok(await page.evaluate(() => document.documentElement.scrollWidth <= innerWidth + 1));
|
||||
const file = `${state.name}-${label}.png`;
|
||||
await page.screenshot({ path: path.join(evidence, file) });
|
||||
result.captures.push({ file, sha256: hash(fs.readFileSync(path.join(evidence, file))), state });
|
||||
};
|
||||
await page.goto(`${origin}/docs/TROUBLESHOOTING#choose-the-affected-workflow`);
|
||||
const entry = page.locator(`article a[href="/docs/PBS#${fragment}"]`);
|
||||
await entry.waitFor();
|
||||
await capture('entry', entry.locator('..'));
|
||||
await activate(entry);
|
||||
await page.waitForURL(`**/docs/PBS#${fragment}`);
|
||||
const heading = page.locator(`#${fragment}`);
|
||||
await page.waitForFunction(id => document.activeElement?.id === id, fragment);
|
||||
await capture('missing-server', heading);
|
||||
const section = await heading.evaluate(element => {
|
||||
const text = [];
|
||||
for (let next = element.nextElementSibling; next && !/^H[1-3]$/.test(next.tagName); next = next.nextElementSibling)
|
||||
text.push(next.textContent);
|
||||
return text.join(' ').replace(/\s+/g, ' ');
|
||||
});
|
||||
for (const phrase of ["connection's polling health", 'does not require a Pulse Agent',
|
||||
'The PBS VM/LXC in Overview is a different resource', 'If Proxmox navigation is absent too',
|
||||
'PVE passthrough backups do not establish', 'unknown is valid',
|
||||
'Do not delete and re-add the connection, replace tokens, install an agent',
|
||||
'Keep certificate verification and existing access restrictions intact'])
|
||||
assert.ok(section.includes(phrase), `${state.name}: ${phrase}`);
|
||||
const safety = page.locator('article p').filter({ hasText: 'Do not delete and re-add the connection, replace tokens, install an agent' });
|
||||
await capture('preserve-state', safety);
|
||||
await activate(page.locator(`article #${fragment} ~ table a[href="#pbs-is-connected-but-history-stays-empty"]`).first());
|
||||
await page.waitForURL('**#pbs-is-connected-but-history-stays-empty');
|
||||
await page.waitForFunction(() => document.activeElement?.id === 'pbs-is-connected-but-history-stays-empty');
|
||||
const consumed = await page.evaluate(() => window.__helpDocuments);
|
||||
assert.ok(consumed.some(item => item.path === '/docs/PBS.md'));
|
||||
assert.ok(consumed.some(item => item.path === '/docs/TROUBLESHOOTING.md'));
|
||||
for (const item of consumed) {
|
||||
assert.equal(item.status, 200);
|
||||
assert.equal(hash(Buffer.from(item.bytes)), binding.artifacts[item.path.slice(1)]);
|
||||
}
|
||||
result.cases.push({ ...state, browser: browser.version(), focusedFragment: true,
|
||||
historyLink: true, noHorizontalPageOverflow: true,
|
||||
consumed: consumed.map(item => ({ path: item.path, sha256: hash(Buffer.from(item.bytes)) })) });
|
||||
await browser.close(); browser = undefined;
|
||||
}
|
||||
assert.deepEqual(result.errors, []);
|
||||
assert.deepEqual(result.backendRequests, []);
|
||||
assert.deepEqual(result.mutations, []);
|
||||
result.playwright = require('playwright/package.json').version;
|
||||
assert.equal(result.playwright, '1.56.1');
|
||||
result.result = 'passed';
|
||||
} finally {
|
||||
if (browser) await browser.close();
|
||||
if (server) await new Promise(resolve => server.close(resolve));
|
||||
result.browserAndServerClosed = true;
|
||||
fs.writeFileSync(path.join(evidence, 'result.json'), JSON.stringify(result, null, 2) + '\n');
|
||||
}
|
||||
console.log(JSON.stringify(result));
|
||||
})().catch(error => { console.error(error); process.exitCode = 1; });
|
||||
@@ -203,6 +203,37 @@ If you have multiple PBS servers, add each one separately in Settings. Pulse wil
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### PBS is Active but missing from monitoring
|
||||
|
||||
**Active** in **Settings → Infrastructure** describes the configured
|
||||
connection's polling health. It does not establish that Pulse has a matching
|
||||
PBS resource, every datastore or fresh readings. A direct PBS API connection
|
||||
does not require a Pulse Agent; adding one does not prove that the API path
|
||||
recovered.
|
||||
|
||||
Use the existing views to distinguish what is missing:
|
||||
|
||||
| Observation | Where to look next |
|
||||
| --- | --- |
|
||||
| The PBS server itself is missing | Open **Proxmox → Backups** and look for its **Backup server** row. Note any existing search or view filter. The PBS VM/LXC in **Overview** is a different resource. If Proxmox navigation is absent too, retain that fact. |
|
||||
| The server row exists, but datastores or backups are missing | Compare the affected view with the connection's existing collection scope. PVE passthrough backups do not establish that direct PBS inventory is available. Do not widen permissions or change scope just to diagnose the gap. |
|
||||
| Current readings exist, but a chart is empty | Use the [PBS History checks](#pbs-is-connected-but-history-stays-empty) for that server and time range. An empty chart is not the same as a missing server. |
|
||||
|
||||
When reporting a missing server, keep the Pulse server version separate from
|
||||
the PBS version and whether a Pulse Agent is installed. Say which existing
|
||||
views lack it, whether Infrastructure says Active, and whether it was visible
|
||||
before. A last known working version is useful only if already known; unknown
|
||||
is valid. Use consistent aliases for private server names. Existing error text
|
||||
or screenshots are enough to start; do not run Diagnostics, enable Debug or
|
||||
collect a full response just to fill the report.
|
||||
|
||||
Do not delete and re-add the connection, replace tokens, install an agent,
|
||||
restart PBS or Pulse, clear History or run a backup to make the server appear.
|
||||
These actions can lose evidence or change the affected path. Keep certificate
|
||||
verification and existing access restrictions intact. If the original incident
|
||||
also involved an unresponsive guest or uncertain freeze/thaw, follow
|
||||
[Backup safety](VM_DISK_MONITORING.md#backup-safety) before any live checks.
|
||||
|
||||
### "Connection Failed" Error
|
||||
|
||||
Check from the Pulse host or container's network, if possible. A request from
|
||||
|
||||
@@ -39,6 +39,10 @@ establish safe recovery.
|
||||
- **TrueNAS readings are stale or absent:** use
|
||||
[TrueNAS data checks](TRUENAS.md#stale-truenas-data), keeping the platform API
|
||||
and Pulse Agent paths separate.
|
||||
- **PBS is Active but missing from monitoring:** distinguish the
|
||||
[server, backup inventory and History](PBS.md#pbs-is-active-but-missing-from-monitoring).
|
||||
An Active API connection does not establish a visible PBS resource; installing
|
||||
an agent does not prove API recovery.
|
||||
- **Pulse itself uses more memory, CPU or disk writes:** use
|
||||
[memory baseline and growth checks](TROUBLESHOOTING.md#memory-use-keeps-growing)
|
||||
or [CPU and write-rate checks](TROUBLESHOOTING.md#excessive-cpu-writes-or-database-growth).
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
import { readFileSync } from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { createGuideFixture } from './guideFixture';
|
||||
|
||||
const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../../..');
|
||||
const shipped = (name: string) =>
|
||||
readFileSync(path.join(root, 'frontend-modern/public/docs', `${name}.md`), 'utf8');
|
||||
const render = createGuideFixture(shipped);
|
||||
const fragment = 'pbs-is-active-but-missing-from-monitoring';
|
||||
|
||||
function section(): HTMLElement {
|
||||
const heading = render('PBS').querySelector(`#${fragment}`);
|
||||
expect(heading).not.toBeNull();
|
||||
const element = document.createElement('section');
|
||||
for (let next = heading!.nextElementSibling; next; next = next.nextElementSibling) {
|
||||
if (/^H[1-3]$/.test(next.tagName)) break;
|
||||
element.appendChild(next.cloneNode(true));
|
||||
}
|
||||
return element;
|
||||
}
|
||||
|
||||
const prose = (element: HTMLElement) =>
|
||||
(element.textContent ?? '').replace(/\s+/g, ' ').trim();
|
||||
|
||||
describe('missing API-only PBS help', () => {
|
||||
it('does not mistake connection polling health or an agent view for API recovery', () => {
|
||||
const text = prose(section());
|
||||
expect(text).toContain("connection's polling health");
|
||||
expect(text).toContain('does not establish that Pulse has a matching PBS resource');
|
||||
expect(text).toContain('A direct PBS API connection does not require a Pulse Agent');
|
||||
expect(text).toContain('adding one does not prove that the API path recovered');
|
||||
});
|
||||
|
||||
it('separates the missing server, partial inventory and empty History without probes', () => {
|
||||
const element = section();
|
||||
const rows = element.querySelectorAll('tbody tr');
|
||||
expect(rows).toHaveLength(3);
|
||||
expect(prose(rows[0] as HTMLElement)).toContain('The PBS VM/LXC in Overview is a different resource');
|
||||
expect(prose(rows[0] as HTMLElement)).toContain('If Proxmox navigation is absent too');
|
||||
expect(prose(rows[1] as HTMLElement)).toContain('PVE passthrough backups do not establish');
|
||||
expect(prose(rows[2] as HTMLElement)).toContain('not the same as a missing server');
|
||||
expect(element.querySelector('pre')).toBeNull();
|
||||
});
|
||||
|
||||
it('accepts existing evidence and unknown versions without transferring diagnosis', () => {
|
||||
const text = prose(section());
|
||||
expect(text).toContain('keep the Pulse server version separate from the PBS version');
|
||||
expect(text).toContain('unknown is valid');
|
||||
expect(text).toContain('Use consistent aliases for private server names');
|
||||
expect(text).toContain('Existing error text or screenshots are enough to start');
|
||||
expect(text).toContain('do not run Diagnostics, enable Debug or collect a full response');
|
||||
expect(text).toContain('Do not delete and re-add the connection, replace tokens, install an agent');
|
||||
expect(text).toContain('Keep certificate verification and existing access restrictions intact');
|
||||
});
|
||||
|
||||
it('links the existing troubleshooting entry and safety/history guides to real headings', () => {
|
||||
const entry = render('TROUBLESHOOTING').querySelector(`a[href="/docs/PBS#${fragment}"]`);
|
||||
expect(entry).not.toBeNull();
|
||||
expect(entry?.closest('li')?.textContent).toContain('installing an agent does not prove API recovery');
|
||||
for (const link of section().querySelectorAll('a[href]')) {
|
||||
const href = link.getAttribute('href')!;
|
||||
const target = /^#(.+)$/.exec(href);
|
||||
const cross = /^\/docs\/([^#]+)#(.+)$/.exec(href);
|
||||
expect(target || cross, href).not.toBeNull();
|
||||
const article = render(cross ? cross[1] : 'PBS');
|
||||
expect(article.querySelector(`[id="${cross ? cross[2] : target![1]}"]`), href).not.toBeNull();
|
||||
}
|
||||
for (const name of ['PBS', 'TROUBLESHOOTING']) {
|
||||
expect(shipped(name)).toBe(readFileSync(path.join(root, 'docs', `${name}.md`), 'utf8'));
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -44,7 +44,7 @@ describe('troubleshooting workflow entry', () => {
|
||||
const section = entry();
|
||||
const link = section.querySelector('a');
|
||||
expect(link?.getAttribute('href')).toBe('/docs/VM_DISK_MONITORING#backup-safety');
|
||||
expect(section.querySelectorAll('li')).toHaveLength(10);
|
||||
expect(section.querySelectorAll('li')).toHaveLength(11);
|
||||
expect(section.querySelector('pre')).toBeNull();
|
||||
const beforeList = [...section.children]
|
||||
.slice(
|
||||
@@ -87,7 +87,7 @@ describe('troubleshooting workflow entry', () => {
|
||||
|
||||
it('routes every entry to an existing shipped heading in the production viewer', () => {
|
||||
const links = [...entry().querySelectorAll('a[href]')];
|
||||
expect(links).toHaveLength(17);
|
||||
expect(links).toHaveLength(18);
|
||||
for (const link of links) {
|
||||
const href = link.getAttribute('href')!;
|
||||
const match = /^\/docs\/([^#]+)#(.+)$/.exec(href);
|
||||
|
||||
Reference in New Issue
Block a user