mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-24 04:33:17 +00:00
Integrate reviewed external-probe safety guidance
Accept exact PI candidateb27387d8fcand ancestorad4fbff6be. Existing help now matches observation-location selection, server-dependent notification delivery, bounded buffering and Mobile retirement. Retain source-_7_dmsnc and inspected desktop/phone-emulated Docs render evidence; no runtime behaviour or native delivery acceptance is claimed. Contract-Neutral: Integrate existing help, shipped mirrors and their verification without changing runtime, API semantics, entitlement or alert policy. Change-source: pulse-maintainer
This commit is contained in:
+6
-1
@@ -1663,7 +1663,12 @@ Target payload fields:
|
||||
- `linkedResourceId` - Optional resource id hint for attaching the probe facet to an existing resource.
|
||||
- `certificateMonitoringDisabled` - Explicit HTTPS-only opt-out; certificate validity monitoring is enabled by default.
|
||||
- `certificateExpiryWarningDays` - HTTPS certificate expiry warning window; defaults to 30 days.
|
||||
- `probeAgentId` - Optional registered host-agent ID that runs the check instead of the Pulse server; requires the Pro `external_probe` entitlement. Send an explicit empty string on update to return the check to local execution.
|
||||
- `observationLocationIds` - Observation locations for this one logical check: `pulse:local` for this Pulse server and `agent:<host-agent-id>` for a connected agent. Agent locations require the Pro `external_probe` entitlement. Include `pulse:local` only when a local observation is wanted; agent-only sets do not also run locally. Each location retains separate evidence rather than treating one path failure as a universal outage.
|
||||
- `probeAgentId` - Compatibility field for single-location clients. A non-empty registered host-agent ID selects that agent; an explicit empty string selects local execution. When updating only this field, omit `observationLocationIds`. For multi-location updates, send the complete desired `observationLocationIds` set and clear `probeAgentId` to `""` so a legacy value cannot override a single-location edit.
|
||||
|
||||
External agents collect observations, but alert evaluation and notification
|
||||
delivery still depend on the Pulse server. Missing reports are not proof of a
|
||||
target outage. See [probe coverage and outage limits](CONFIGURATION.md#external-probes-pro).
|
||||
|
||||
An explicit `linkedResourceId` is authoritative and fails closed when it
|
||||
cannot resolve. Without it, Pulse correlates only on one exact normalized IP
|
||||
|
||||
+45
-24
@@ -577,30 +577,51 @@ alerts, and resource projections keep one canonical protocol value.
|
||||
|
||||
### External probes (Pro)
|
||||
|
||||
By default every availability check runs from the Pulse server itself, which
|
||||
means a check cannot report the one failure that matters most: the site
|
||||
running Pulse losing its connectivity. With the Pro `external_probe`
|
||||
entitlement, a check can instead be assigned to any connected Pulse agent
|
||||
(`probeAgentId` in the API, "Run from" in the UI) — for example an agent on a
|
||||
cloud VM or at another site. The assigned agent receives the check through
|
||||
its signed agent configuration, runs it on the configured interval, and
|
||||
reports results back with its reports; results are only accepted from the
|
||||
currently assigned agent. An assigned check is not also run locally. If no
|
||||
report arrives for several intervals the check shows as indeterminate
|
||||
("no recent report from probe agent"). Freshness uses the Pulse server's receipt
|
||||
time, so clock drift on the probe cannot create or hide a disconnect. After at
|
||||
least five minutes without a report, Pulse raises one warning per disconnected
|
||||
probe through the normal notification routes, even when that probe owns several
|
||||
checks. The incident is keyed to the probe agent rather than an arbitrary check,
|
||||
so changing assignments does not reopen it. If the agent heartbeat is also
|
||||
offline, the existing
|
||||
host-offline alert owns the incident instead of producing a duplicate probe
|
||||
warning. A paired Pulse Mobile client receives a privacy-safe
|
||||
`external_probe_offline` Relay push linked to that canonical alert. If the whole
|
||||
Pulse instance becomes unreachable, Relay's independent five-minute
|
||||
instance-offline push covers the dark-site case. If the entitlement lapses, the
|
||||
check automatically resumes running from the Pulse server. Checks without an
|
||||
assignment are unaffected and remain available in every edition.
|
||||
External probes change **where a check runs**, not where alerts are sent.
|
||||
By default, checks run on the Pulse server. With the Pro `external_probe`
|
||||
entitlement, select one or more connected agents under **Observation locations**
|
||||
in the check editor. An agent on a cloud VM or at another site can observe a
|
||||
service from outside its local network. Keep **This Pulse server** selected if
|
||||
you also want a local observation; deselect it for agent-only checks. Pulse keeps
|
||||
each location's evidence separate, so a failure on one path is not a universal
|
||||
outage. The API uses `observationLocationIds`; see the
|
||||
[target fields](API.md#availability-checks).
|
||||
|
||||
The assigned agent receives signed configuration, runs the check and reports
|
||||
results back to Pulse. Only agents assigned to that target may submit results;
|
||||
the server runs a local check only when **This Pulse server** is selected.
|
||||
**The agent does not send
|
||||
notifications directly.** The Pulse server must be running and able to reach
|
||||
the notification destination to evaluate results and send alerts.
|
||||
|
||||
| Situation | What the check can tell you |
|
||||
| --- | --- |
|
||||
| Pulse is running and receives probe results | Target observations are evaluated through the normal alert and notification policies. |
|
||||
| Pulse is running but probe results stop arriving | The affected location becomes indeterminate: "no recent report from probe agent". This is missing evidence, not proof the target is down. Other locations retain their own evidence. |
|
||||
| Pulse is stopped or cannot reach its notification destination | An external agent cannot deliver Pulse alerts in its place. Use an independent external watchdog for this failure. |
|
||||
|
||||
The missing-report window is the longer of **five minutes or three check
|
||||
intervals**, measured from server receipt time, not the agent's clock. During
|
||||
normal server evaluation, eligible stale probes raise one warning per agent,
|
||||
not per assigned check. Alert and connectivity policies still apply. If the
|
||||
agent heartbeat is also offline, the host-offline alert owns the incident
|
||||
instead of a duplicate probe warning. See the
|
||||
[agent probe guide](UNIFIED_AGENT.md#external-probes-pro) for buffering limits.
|
||||
|
||||
For an outage of Pulse itself, configure an
|
||||
[external watchdog](TROUBLESHOOTING.md#no-alert-when-pulse-power-or-internet-goes-down)
|
||||
outside the power and network failure you need to detect, with an independently
|
||||
reachable notification destination. Verify actual receipt in an authorised test
|
||||
environment; a successful destination test does not prove outage coverage.
|
||||
|
||||
Existing paired Pulse Mobile/Relay users retain their current push path until
|
||||
**31 March 2027**. Relay is no longer sold. Its instance-disconnect push does not
|
||||
evaluate individual targets while Pulse is offline, and is not a permanent
|
||||
substitute for the watchdog. See [Mobile retirement](RELAY.md).
|
||||
|
||||
If the entitlement lapses, the check resumes running from the Pulse server:
|
||||
this changes its observation location and may change its result. Checks without
|
||||
an agent assignment remain available in every edition.
|
||||
|
||||
### ICMP probe privileges
|
||||
|
||||
|
||||
+39
-24
@@ -303,42 +303,57 @@ storage nodes to avoid duplicate or ambiguously correlated readings.
|
||||
## External Probes (Pro)
|
||||
|
||||
With the Pro `external_probe` entitlement, availability checks configured in
|
||||
Pulse can be assigned to run from a specific agent instead of the Pulse
|
||||
server (Settings -> Monitoring -> Availability checks -> "Run from"). This is
|
||||
how you monitor a site from the outside: deploy the agent on a machine
|
||||
elsewhere — a cloud VM, a Docker host at another location — and assign checks
|
||||
to it. Target failures are evaluated on the Pulse server through your normal
|
||||
alert routes.
|
||||
Pulse can run from one or more connected agents. In **Settings → Monitoring →
|
||||
Availability checks**, select the agents under **Observation locations** in the
|
||||
check editor. Keep **This Pulse server** selected for a local observation too,
|
||||
or deselect it for agent-only checks. Each location keeps its own evidence; a
|
||||
failure on one path is not a universal outage.
|
||||
|
||||
To observe a service from outside its local network, deploy the agent on a
|
||||
machine elsewhere — a cloud VM, a Docker host at another location — and assign
|
||||
checks to it. **The agent does not send notifications directly.** The
|
||||
Pulse server must be running and able to reach the notification destination;
|
||||
target failures are evaluated there through your normal alert routes.
|
||||
|
||||
There is nothing to configure on the agent itself. Assignments arrive through
|
||||
the agent's signed remote configuration, the agent runs each check on its
|
||||
configured interval, and results are delivered with its regular reports.
|
||||
Results survive temporary connectivity loss to the Pulse server in a bounded
|
||||
in-memory queue; if the agent cannot deliver for several check intervals the
|
||||
check shows as indeterminate in Pulse until reports resume. After the
|
||||
five-minute minimum grace window, Pulse raises one
|
||||
Results waiting for delivery use a bounded **in-memory** queue of up to 200
|
||||
observations across the agent's checks. Oldest pending observations are dropped
|
||||
when it fills, and an agent restart loses the queue; it is not a complete outage
|
||||
record. If reports stop arriving, the affected location becomes indeterminate ("no recent
|
||||
report from probe agent"). This is missing evidence, not proof the target is
|
||||
down. The missing-report window is the
|
||||
longer of **five minutes or three check intervals**. During normal server
|
||||
evaluation, eligible stale probes raise one
|
||||
`availability_probe_unavailable` warning per disconnected probe, regardless of
|
||||
how many checks it owns. Pulse measures that reporting window from server receipt
|
||||
time rather than the agent's clock, so clock skew cannot create or conceal the
|
||||
disconnect. That warning uses the normal email, webhook, Apprise, and
|
||||
recovery-notification pipeline. When Pulse Mobile is paired through Relay, Pulse
|
||||
also sends a privacy-safe `external_probe_offline` push linked to the canonical
|
||||
mobile attention item without exposing target names or addresses. The alert
|
||||
identity belongs to the probe agent, so adding or removing an assigned check
|
||||
does not resolve and reopen it.
|
||||
recovery-notification pipeline, subject to alert and connectivity policies.
|
||||
The alert identity belongs to the probe agent, so adding or removing an assigned
|
||||
check does not resolve and reopen it.
|
||||
|
||||
When the host heartbeat itself is offline, Pulse keeps the existing
|
||||
host-offline alert as the single canonical incident and suppresses the
|
||||
probe-results warning. Assigned probe hosts still receive the external-probe
|
||||
mobile push, but operators do not get two normal alerts for the same agent
|
||||
failure.
|
||||
probe-results warning, rather than producing two normal alerts for the same
|
||||
agent failure.
|
||||
|
||||
This has a complementary dark-site path: if the entire Pulse instance or its
|
||||
site goes offline, Pulse Relay independently sends its existing instance
|
||||
offline push after five minutes. Together, probe-loss alerts while Pulse is
|
||||
online and Relay's instance-loss alert while Pulse is dark ensure the
|
||||
outside-monitoring path cannot disappear silently. Relay does not evaluate
|
||||
individual target results while the Pulse server is offline.
|
||||
If Pulse itself stops or loses outbound connectivity, the external agent cannot
|
||||
deliver Pulse alerts in its place. Configure an
|
||||
[external watchdog](TROUBLESHOOTING.md#no-alert-when-pulse-power-or-internet-goes-down)
|
||||
outside the shared power/network failure, with an independently reachable
|
||||
notification destination. Verify recipient delivery in an authorised test
|
||||
environment, not by interrupting a production site.
|
||||
|
||||
Existing paired Pulse Mobile/Relay users retain their current push path until
|
||||
**31 March 2027**. Relay is no longer sold. Its instance-disconnect push does not
|
||||
evaluate individual targets while Pulse is offline, and is not a permanent
|
||||
substitute for the watchdog. See [Mobile retirement](RELAY.md).
|
||||
|
||||
If the external-probe entitlement lapses, checks resume on the Pulse server,
|
||||
not the agent. That changes the observation location and may change the result;
|
||||
see the [configuration guide](CONFIGURATION.md#external-probes-pro).
|
||||
|
||||
The module appears as `availability` in the agent's module status when at
|
||||
least one check is assigned.
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
// Render the existing probe help and its outage/retirement links with the real
|
||||
// Docs page. No agent, destination, watchdog or Mobile operation is performed.
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const crypto = require('node:crypto');
|
||||
const { chromium, webkit } = require('playwright');
|
||||
|
||||
(async () => {
|
||||
const root = '/workspace/frontend-modern';
|
||||
const artifacts = path.join(root, 'node_modules', 'probe-guidance-browser');
|
||||
fs.mkdirSync(artifacts, { recursive: true });
|
||||
const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js'));
|
||||
const server = await createServer({
|
||||
root,
|
||||
configFile: path.join(root, 'vite.config.ts'),
|
||||
cacheDir: path.join(root, 'node_modules', '.vite-probe-guidance'),
|
||||
server: { host: '127.0.0.1', port: 5253, strictPort: true },
|
||||
optimizeDeps: { noDiscovery: true, include: [] },
|
||||
});
|
||||
const results = [];
|
||||
let browser;
|
||||
try {
|
||||
await server.listen();
|
||||
for (const [engine, width] of [['chromium', 1280], ['webkit', 390]]) {
|
||||
browser = await { chromium, webkit }[engine].launch(engine === 'chromium'
|
||||
? { headless: true, channel: 'chromium', args: ['--no-sandbox'] }
|
||||
: { headless: true });
|
||||
const page = await browser.newPage({ viewport: { width, height: 1000 },
|
||||
...(engine === 'webkit' ? { isMobile: true, hasTouch: true } : {}) });
|
||||
const errors = [];
|
||||
page.on('pageerror', (error) => errors.push(error.message));
|
||||
page.on('console', (message) => {
|
||||
if (message.type() === 'error') errors.push(message.text());
|
||||
});
|
||||
await page.goto('http://127.0.0.1:5253/browser-tests/docs-fragment-navigation.html?scenario=configuration',
|
||||
{ waitUntil: 'domcontentloaded', timeout: 60_000 });
|
||||
await page.getByRole('heading', { name: 'External probes (Pro)', exact: true }).waitFor();
|
||||
if (engine === 'webkit') await page.evaluate(() => document.documentElement.classList.add('dark'));
|
||||
const table = page.getByRole('table').filter({ has: page.getByRole('columnheader',
|
||||
{ name: 'Situation', exact: true }) });
|
||||
assert.equal(await table.getByRole('row').count(), 4);
|
||||
assert.ok(await table.evaluate((element) => {
|
||||
const parent = element.parentElement;
|
||||
return parent.dataset.docTableScroll !== undefined &&
|
||||
getComputedStyle(parent).overflowX === 'auto' &&
|
||||
[...element.querySelectorAll('thead th')].every((header) => header.scope === 'col');
|
||||
}));
|
||||
const assertBoundaries = async () => {
|
||||
const body = (await page.locator('article').innerText()).replace(/\s+/g, ' ');
|
||||
for (const text of ['The agent does not send notifications directly.',
|
||||
'Pulse server must be running and able to reach the notification destination',
|
||||
'Observation locations', 'This Pulse server', 'not a universal outage',
|
||||
'not proof the target is down', 'five minutes or three check intervals',
|
||||
'31 March 2027', 'Relay is no longer sold', 'not a permanent substitute']) {
|
||||
assert.ok(body.includes(text), text);
|
||||
}
|
||||
assert.ok(!body.includes('cannot disappear silently'));
|
||||
assert.ok(await page.evaluate(() => document.documentElement.scrollWidth <= innerWidth + 1));
|
||||
};
|
||||
const screenshot = async (name, heading) => {
|
||||
await page.getByRole('heading', { name: heading, exact: true })
|
||||
.evaluate((element) => element.scrollIntoView({ block: 'start' }));
|
||||
await page.evaluate(async () => {
|
||||
await new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve)));
|
||||
await Promise.all(document.getAnimations().map((animation) => animation.finished.catch(() => {})));
|
||||
});
|
||||
await page.screenshot({ path: path.join(artifacts, `${engine}-${name}.png`) });
|
||||
};
|
||||
const follow = async (name, href, heading, id) => {
|
||||
const link = page.getByRole('link', { name, exact: true });
|
||||
assert.equal(await link.getAttribute('href'), href);
|
||||
await link.focus();
|
||||
await page.keyboard.press('Enter');
|
||||
await page.waitForURL(`**${href}`);
|
||||
await page.getByRole('heading', { name: heading, exact: true }).waitFor();
|
||||
if (id) await page.waitForFunction((id) => document.activeElement?.id === id, id);
|
||||
};
|
||||
await assertBoundaries();
|
||||
await screenshot('configuration', 'External probes (Pro)');
|
||||
await follow('agent probe guide', '/docs/UNIFIED_AGENT#external-probes-pro',
|
||||
'External Probes (Pro)', 'external-probes-pro');
|
||||
await assertBoundaries();
|
||||
const agentBody = (await page.locator('article').innerText()).replace(/\s+/g, ' ');
|
||||
for (const text of ['up to 200 observations', 'Oldest pending observations are dropped',
|
||||
'agent restart loses the queue', 'not a complete outage record']) assert.ok(agentBody.includes(text), text);
|
||||
await screenshot('agent', 'External Probes (Pro)');
|
||||
await follow('external watchdog', '/docs/TROUBLESHOOTING#no-alert-when-pulse-power-or-internet-goes-down',
|
||||
'No alert when Pulse, power or internet goes down', 'no-alert-when-pulse-power-or-internet-goes-down');
|
||||
assert.ok((await page.locator('article').innerText()).includes('period plus grace'));
|
||||
await screenshot('watchdog', 'No alert when Pulse, power or internet goes down');
|
||||
await page.goBack();
|
||||
await page.getByRole('heading', { name: 'External Probes (Pro)', exact: true }).waitFor();
|
||||
await follow('Mobile retirement', '/docs/RELAY', 'Relay / Pulse Mobile', '');
|
||||
assert.ok((await page.locator('article').innerText()).includes('31 March 2027'));
|
||||
await page.goBack();
|
||||
await page.getByRole('heading', { name: 'External Probes (Pro)', exact: true }).waitFor();
|
||||
await follow('configuration guide', '/docs/CONFIGURATION#external-probes-pro',
|
||||
'External probes (Pro)', 'external-probes-pro');
|
||||
await follow('target fields', '/docs/API#availability-checks',
|
||||
'Availability Checks', 'availability-checks');
|
||||
assert.ok((await page.locator('article').innerText()).includes('clear probeAgentId to ""'));
|
||||
await page.goBack();
|
||||
await page.getByRole('heading', { name: 'External probes (Pro)', exact: true }).waitFor();
|
||||
await follow('external watchdog', '/docs/TROUBLESHOOTING#no-alert-when-pulse-power-or-internet-goes-down',
|
||||
'No alert when Pulse, power or internet goes down', 'no-alert-when-pulse-power-or-internet-goes-down');
|
||||
assert.deepEqual(errors, []);
|
||||
results.push({ engine, version: browser.version(), width,
|
||||
scope: engine === 'webkit' ? 'phone-emulated, not a native device' : 'desktop',
|
||||
serverDependencyVisible: true, semanticContainedCoverageTable: true,
|
||||
noDocumentOverflow: true, queueAndRetirementLimitsVisible: true,
|
||||
keyboardAgentGuide: true, keyboardConfigurationGuide: true,
|
||||
keyboardWatchdogLinks: 2, keyboardRetirementLink: true, keyboardCanonicalApiFields: true, errors });
|
||||
await browser.close();
|
||||
browser = undefined;
|
||||
}
|
||||
const files = ['public/docs/CONFIGURATION.md', 'public/docs/UNIFIED_AGENT.md', 'public/docs/API.md',
|
||||
'browser-tests/probe-guidance.cjs', 'browser-tests/docs-fragment-navigation.tsx',
|
||||
'src/features/docs/docMarkdown.ts', 'src/pages/Docs.tsx'].map((file) => ({ file,
|
||||
sha256: crypto.createHash('sha256').update(fs.readFileSync(path.join(root, file))).digest('hex') }));
|
||||
const result = { result: 'passed', playwright: require('playwright/package.json').version,
|
||||
scope: 'existing shipped Docs rendering and keyboard links only; no native outage or delivery acceptance',
|
||||
files, results };
|
||||
fs.writeFileSync(path.join(artifacts, 'result.json'), JSON.stringify(result, null, 2) + '\n');
|
||||
console.log(JSON.stringify(result));
|
||||
} finally {
|
||||
if (browser) await browser.close();
|
||||
await server.close();
|
||||
}
|
||||
})().catch((error) => { console.error(error); process.exitCode = 1; });
|
||||
@@ -1663,7 +1663,12 @@ Target payload fields:
|
||||
- `linkedResourceId` - Optional resource id hint for attaching the probe facet to an existing resource.
|
||||
- `certificateMonitoringDisabled` - Explicit HTTPS-only opt-out; certificate validity monitoring is enabled by default.
|
||||
- `certificateExpiryWarningDays` - HTTPS certificate expiry warning window; defaults to 30 days.
|
||||
- `probeAgentId` - Optional registered host-agent ID that runs the check instead of the Pulse server; requires the Pro `external_probe` entitlement. Send an explicit empty string on update to return the check to local execution.
|
||||
- `observationLocationIds` - Observation locations for this one logical check: `pulse:local` for this Pulse server and `agent:<host-agent-id>` for a connected agent. Agent locations require the Pro `external_probe` entitlement. Include `pulse:local` only when a local observation is wanted; agent-only sets do not also run locally. Each location retains separate evidence rather than treating one path failure as a universal outage.
|
||||
- `probeAgentId` - Compatibility field for single-location clients. A non-empty registered host-agent ID selects that agent; an explicit empty string selects local execution. When updating only this field, omit `observationLocationIds`. For multi-location updates, send the complete desired `observationLocationIds` set and clear `probeAgentId` to `""` so a legacy value cannot override a single-location edit.
|
||||
|
||||
External agents collect observations, but alert evaluation and notification
|
||||
delivery still depend on the Pulse server. Missing reports are not proof of a
|
||||
target outage. See [probe coverage and outage limits](CONFIGURATION.md#external-probes-pro).
|
||||
|
||||
An explicit `linkedResourceId` is authoritative and fails closed when it
|
||||
cannot resolve. Without it, Pulse correlates only on one exact normalized IP
|
||||
|
||||
@@ -577,30 +577,51 @@ alerts, and resource projections keep one canonical protocol value.
|
||||
|
||||
### External probes (Pro)
|
||||
|
||||
By default every availability check runs from the Pulse server itself, which
|
||||
means a check cannot report the one failure that matters most: the site
|
||||
running Pulse losing its connectivity. With the Pro `external_probe`
|
||||
entitlement, a check can instead be assigned to any connected Pulse agent
|
||||
(`probeAgentId` in the API, "Run from" in the UI) — for example an agent on a
|
||||
cloud VM or at another site. The assigned agent receives the check through
|
||||
its signed agent configuration, runs it on the configured interval, and
|
||||
reports results back with its reports; results are only accepted from the
|
||||
currently assigned agent. An assigned check is not also run locally. If no
|
||||
report arrives for several intervals the check shows as indeterminate
|
||||
("no recent report from probe agent"). Freshness uses the Pulse server's receipt
|
||||
time, so clock drift on the probe cannot create or hide a disconnect. After at
|
||||
least five minutes without a report, Pulse raises one warning per disconnected
|
||||
probe through the normal notification routes, even when that probe owns several
|
||||
checks. The incident is keyed to the probe agent rather than an arbitrary check,
|
||||
so changing assignments does not reopen it. If the agent heartbeat is also
|
||||
offline, the existing
|
||||
host-offline alert owns the incident instead of producing a duplicate probe
|
||||
warning. A paired Pulse Mobile client receives a privacy-safe
|
||||
`external_probe_offline` Relay push linked to that canonical alert. If the whole
|
||||
Pulse instance becomes unreachable, Relay's independent five-minute
|
||||
instance-offline push covers the dark-site case. If the entitlement lapses, the
|
||||
check automatically resumes running from the Pulse server. Checks without an
|
||||
assignment are unaffected and remain available in every edition.
|
||||
External probes change **where a check runs**, not where alerts are sent.
|
||||
By default, checks run on the Pulse server. With the Pro `external_probe`
|
||||
entitlement, select one or more connected agents under **Observation locations**
|
||||
in the check editor. An agent on a cloud VM or at another site can observe a
|
||||
service from outside its local network. Keep **This Pulse server** selected if
|
||||
you also want a local observation; deselect it for agent-only checks. Pulse keeps
|
||||
each location's evidence separate, so a failure on one path is not a universal
|
||||
outage. The API uses `observationLocationIds`; see the
|
||||
[target fields](API.md#availability-checks).
|
||||
|
||||
The assigned agent receives signed configuration, runs the check and reports
|
||||
results back to Pulse. Only agents assigned to that target may submit results;
|
||||
the server runs a local check only when **This Pulse server** is selected.
|
||||
**The agent does not send
|
||||
notifications directly.** The Pulse server must be running and able to reach
|
||||
the notification destination to evaluate results and send alerts.
|
||||
|
||||
| Situation | What the check can tell you |
|
||||
| --- | --- |
|
||||
| Pulse is running and receives probe results | Target observations are evaluated through the normal alert and notification policies. |
|
||||
| Pulse is running but probe results stop arriving | The affected location becomes indeterminate: "no recent report from probe agent". This is missing evidence, not proof the target is down. Other locations retain their own evidence. |
|
||||
| Pulse is stopped or cannot reach its notification destination | An external agent cannot deliver Pulse alerts in its place. Use an independent external watchdog for this failure. |
|
||||
|
||||
The missing-report window is the longer of **five minutes or three check
|
||||
intervals**, measured from server receipt time, not the agent's clock. During
|
||||
normal server evaluation, eligible stale probes raise one warning per agent,
|
||||
not per assigned check. Alert and connectivity policies still apply. If the
|
||||
agent heartbeat is also offline, the host-offline alert owns the incident
|
||||
instead of a duplicate probe warning. See the
|
||||
[agent probe guide](UNIFIED_AGENT.md#external-probes-pro) for buffering limits.
|
||||
|
||||
For an outage of Pulse itself, configure an
|
||||
[external watchdog](TROUBLESHOOTING.md#no-alert-when-pulse-power-or-internet-goes-down)
|
||||
outside the power and network failure you need to detect, with an independently
|
||||
reachable notification destination. Verify actual receipt in an authorised test
|
||||
environment; a successful destination test does not prove outage coverage.
|
||||
|
||||
Existing paired Pulse Mobile/Relay users retain their current push path until
|
||||
**31 March 2027**. Relay is no longer sold. Its instance-disconnect push does not
|
||||
evaluate individual targets while Pulse is offline, and is not a permanent
|
||||
substitute for the watchdog. See [Mobile retirement](RELAY.md).
|
||||
|
||||
If the entitlement lapses, the check resumes running from the Pulse server:
|
||||
this changes its observation location and may change its result. Checks without
|
||||
an agent assignment remain available in every edition.
|
||||
|
||||
### ICMP probe privileges
|
||||
|
||||
|
||||
@@ -303,42 +303,57 @@ storage nodes to avoid duplicate or ambiguously correlated readings.
|
||||
## External Probes (Pro)
|
||||
|
||||
With the Pro `external_probe` entitlement, availability checks configured in
|
||||
Pulse can be assigned to run from a specific agent instead of the Pulse
|
||||
server (Settings -> Monitoring -> Availability checks -> "Run from"). This is
|
||||
how you monitor a site from the outside: deploy the agent on a machine
|
||||
elsewhere — a cloud VM, a Docker host at another location — and assign checks
|
||||
to it. Target failures are evaluated on the Pulse server through your normal
|
||||
alert routes.
|
||||
Pulse can run from one or more connected agents. In **Settings → Monitoring →
|
||||
Availability checks**, select the agents under **Observation locations** in the
|
||||
check editor. Keep **This Pulse server** selected for a local observation too,
|
||||
or deselect it for agent-only checks. Each location keeps its own evidence; a
|
||||
failure on one path is not a universal outage.
|
||||
|
||||
To observe a service from outside its local network, deploy the agent on a
|
||||
machine elsewhere — a cloud VM, a Docker host at another location — and assign
|
||||
checks to it. **The agent does not send notifications directly.** The
|
||||
Pulse server must be running and able to reach the notification destination;
|
||||
target failures are evaluated there through your normal alert routes.
|
||||
|
||||
There is nothing to configure on the agent itself. Assignments arrive through
|
||||
the agent's signed remote configuration, the agent runs each check on its
|
||||
configured interval, and results are delivered with its regular reports.
|
||||
Results survive temporary connectivity loss to the Pulse server in a bounded
|
||||
in-memory queue; if the agent cannot deliver for several check intervals the
|
||||
check shows as indeterminate in Pulse until reports resume. After the
|
||||
five-minute minimum grace window, Pulse raises one
|
||||
Results waiting for delivery use a bounded **in-memory** queue of up to 200
|
||||
observations across the agent's checks. Oldest pending observations are dropped
|
||||
when it fills, and an agent restart loses the queue; it is not a complete outage
|
||||
record. If reports stop arriving, the affected location becomes indeterminate ("no recent
|
||||
report from probe agent"). This is missing evidence, not proof the target is
|
||||
down. The missing-report window is the
|
||||
longer of **five minutes or three check intervals**. During normal server
|
||||
evaluation, eligible stale probes raise one
|
||||
`availability_probe_unavailable` warning per disconnected probe, regardless of
|
||||
how many checks it owns. Pulse measures that reporting window from server receipt
|
||||
time rather than the agent's clock, so clock skew cannot create or conceal the
|
||||
disconnect. That warning uses the normal email, webhook, Apprise, and
|
||||
recovery-notification pipeline. When Pulse Mobile is paired through Relay, Pulse
|
||||
also sends a privacy-safe `external_probe_offline` push linked to the canonical
|
||||
mobile attention item without exposing target names or addresses. The alert
|
||||
identity belongs to the probe agent, so adding or removing an assigned check
|
||||
does not resolve and reopen it.
|
||||
recovery-notification pipeline, subject to alert and connectivity policies.
|
||||
The alert identity belongs to the probe agent, so adding or removing an assigned
|
||||
check does not resolve and reopen it.
|
||||
|
||||
When the host heartbeat itself is offline, Pulse keeps the existing
|
||||
host-offline alert as the single canonical incident and suppresses the
|
||||
probe-results warning. Assigned probe hosts still receive the external-probe
|
||||
mobile push, but operators do not get two normal alerts for the same agent
|
||||
failure.
|
||||
probe-results warning, rather than producing two normal alerts for the same
|
||||
agent failure.
|
||||
|
||||
This has a complementary dark-site path: if the entire Pulse instance or its
|
||||
site goes offline, Pulse Relay independently sends its existing instance
|
||||
offline push after five minutes. Together, probe-loss alerts while Pulse is
|
||||
online and Relay's instance-loss alert while Pulse is dark ensure the
|
||||
outside-monitoring path cannot disappear silently. Relay does not evaluate
|
||||
individual target results while the Pulse server is offline.
|
||||
If Pulse itself stops or loses outbound connectivity, the external agent cannot
|
||||
deliver Pulse alerts in its place. Configure an
|
||||
[external watchdog](TROUBLESHOOTING.md#no-alert-when-pulse-power-or-internet-goes-down)
|
||||
outside the shared power/network failure, with an independently reachable
|
||||
notification destination. Verify recipient delivery in an authorised test
|
||||
environment, not by interrupting a production site.
|
||||
|
||||
Existing paired Pulse Mobile/Relay users retain their current push path until
|
||||
**31 March 2027**. Relay is no longer sold. Its instance-disconnect push does not
|
||||
evaluate individual targets while Pulse is offline, and is not a permanent
|
||||
substitute for the watchdog. See [Mobile retirement](RELAY.md).
|
||||
|
||||
If the external-probe entitlement lapses, checks resume on the Pulse server,
|
||||
not the agent. That changes the observation location and may change the result;
|
||||
see the [configuration guide](CONFIGURATION.md#external-probes-pro).
|
||||
|
||||
The module appears as `availability` in the agent's module status when at
|
||||
least one check is assigned.
|
||||
|
||||
@@ -61,6 +61,87 @@ function getRuntimeSourceFiles(dir: string): string[] {
|
||||
}
|
||||
|
||||
describe('docsLinks', () => {
|
||||
const probeGuides = () => [
|
||||
readFileSync(path.join(repoRoot, 'docs', 'CONFIGURATION.md'), 'utf8')
|
||||
.split('### External probes (Pro)')[1].split('### ICMP probe privileges')[0],
|
||||
readFileSync(path.join(repoRoot, 'docs', 'UNIFIED_AGENT.md'), 'utf8')
|
||||
.split('## External Probes (Pro)')[1].split('## Custom metrics')[0],
|
||||
].map((section) => section.replace(/\s+/g, ' '));
|
||||
|
||||
it('keeps external probes dependent on server-side alert delivery', () => {
|
||||
for (const guide of probeGuides()) {
|
||||
expect(guide).toContain('The agent does not send notifications directly.');
|
||||
expect(guide).toContain('Pulse server must be running and able to reach the notification destination');
|
||||
expect(guide).toContain('agent cannot deliver Pulse alerts in its place');
|
||||
expect(guide).toContain('not proof');
|
||||
expect(guide).not.toContain('cannot disappear silently');
|
||||
}
|
||||
});
|
||||
|
||||
it('explains the existing missing-report and bounded-buffer limits', () => {
|
||||
for (const guide of probeGuides()) {
|
||||
expect(guide).toContain('five minutes or three check intervals');
|
||||
expect(guide).toContain('server receipt time');
|
||||
expect(guide).toContain('host-offline alert');
|
||||
expect(guide).toContain('policies');
|
||||
expect(guide).toMatch(/changes (?:its|the) observation location/);
|
||||
}
|
||||
const agentGuide = probeGuides()[1];
|
||||
expect(agentGuide).toContain('up to 200 observations');
|
||||
expect(agentGuide).toContain('Oldest pending observations are dropped');
|
||||
expect(agentGuide).toContain('agent restart loses the queue');
|
||||
expect(agentGuide).toContain('not a complete outage record');
|
||||
// These limits come from the current implementation, not a new policy.
|
||||
const monitor = readFileSync(
|
||||
path.join(repoRoot, 'internal', 'monitoring', 'availability_probe_agent.go'), 'utf8',
|
||||
);
|
||||
expect(monitor).toContain('availabilityProbeStaleFloor = 5 * time.Minute');
|
||||
expect(monitor).toContain('target.EffectivePollIntervalSecs()) * 3 * time.Second');
|
||||
const agent = readFileSync(
|
||||
path.join(repoRoot, 'internal', 'hostagent', 'availability.go'), 'utf8',
|
||||
);
|
||||
expect(agent).toContain('availabilityPendingCapacity = 200');
|
||||
});
|
||||
|
||||
it('links independent outage coverage without a permanent Mobile guarantee', () => {
|
||||
for (const guide of probeGuides()) {
|
||||
expect(guide).toContain('TROUBLESHOOTING.md#no-alert-when-pulse-power-or-internet-goes-down');
|
||||
expect(guide).toContain('independently reachable notification destination');
|
||||
expect(guide).toContain('authorised test environment');
|
||||
expect(guide).toContain('Existing paired Pulse Mobile/Relay users');
|
||||
expect(guide).toContain('31 March 2027');
|
||||
expect(guide).toContain('Relay is no longer sold');
|
||||
expect(guide).toContain('not a permanent substitute');
|
||||
expect(guide).toContain('[Mobile retirement](RELAY.md)');
|
||||
expect(guide).not.toContain('push covers the dark-site case');
|
||||
}
|
||||
});
|
||||
|
||||
it('matches the existing observation-location editor and compatibility fields', () => {
|
||||
for (const guide of probeGuides()) {
|
||||
expect(guide).toContain('Observation locations');
|
||||
expect(guide).toContain('This Pulse server');
|
||||
expect(guide).toContain('not a universal outage');
|
||||
expect(guide).not.toContain('Run from');
|
||||
expect(guide).not.toContain('an assigned check is not also run locally');
|
||||
}
|
||||
const api = readFileSync(path.join(repoRoot, 'docs', 'API.md'), 'utf8');
|
||||
expect(api).toContain('`observationLocationIds` - Observation locations');
|
||||
expect(api).toContain('`pulse:local` for this Pulse server');
|
||||
expect(api).toContain('`agent:<host-agent-id>` for a connected agent');
|
||||
expect(api).toContain('omit `observationLocationIds`');
|
||||
expect(api).toContain('clear `probeAgentId` to `""`');
|
||||
const editor = readFileSync(path.join(frontendRoot, 'src', 'components', 'Settings',
|
||||
'ConnectionEditor', 'CredentialSlots', 'AvailabilityTargetSlot.tsx'), 'utf8');
|
||||
expect(editor).toContain('Observation locations</legend>');
|
||||
expect(editor).toContain('observationLocationIds: [...form.observationLocationIds]');
|
||||
const handler = readFileSync(
|
||||
path.join(repoRoot, 'internal', 'api', 'availability_handlers.go'), 'utf8',
|
||||
);
|
||||
expect(handler).toContain('compatibility.ObservationLocationIDs == nil && compatibility.ProbeAgentID != nil');
|
||||
expect(handler).toContain('agentID != "" && len(*compatibility.ObservationLocationIDs) == 1');
|
||||
});
|
||||
|
||||
it('separates server removal from persistent-data erasure', () => {
|
||||
const installation = readFileSync(path.join(repoRoot, 'docs', 'INSTALL.md'), 'utf8');
|
||||
const removal = installation.split('## 🗑️ Uninstall')[1];
|
||||
|
||||
Reference in New Issue
Block a user