From 25a2e56d5ef62b2674a4deba655a121ecb7c8761 Mon Sep 17 00:00:00 2001 From: "pulse-triage[bot]" <249995291+pulse-triage[bot]@users.noreply.github.com> Date: Wed, 30 Sep 2026 02:29:54 +0100 Subject: [PATCH] Make cloned-host identity guidance safe and persistent The agent prefers its saved identity before deriving one from the OS. The old live machine-ID deletion recipe could disrupt unrelated services while leaving a Pulse collision unchanged. Explain managed overrides, precedence, private evidence and distinct-host verification in both shipped guides; cover the safe path with documentation and real-browser checks. Contract-Neutral: Guidance and verification for existing agent identity options only; no runtime, authentication, schema or frontend behaviour changes. Change-source: pulse-maintainer --- docs/TROUBLESHOOTING.md | 18 ++- docs/UNIFIED_AGENT.md | 59 +++++++--- .../public/docs/TROUBLESHOOTING.md | 18 ++- frontend-modern/public/docs/UNIFIED_AGENT.md | 59 +++++++--- .../src/utils/__tests__/docsLinks.test.ts | 31 +++++ scripts/check-clone-identity-guidance.cjs | 107 ++++++++++++++++++ 6 files changed, 256 insertions(+), 36 deletions(-) create mode 100644 scripts/check-clone-identity-guidance.cjs diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index 0882611df1..eaa7e5cebb 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -155,11 +155,19 @@ repair an older generated unit rather than adding a JSON-parsing wrapper. - See [Temperature Monitoring](TEMPERATURE_MONITORING.md). #### Docker hosts appearing/disappearing -- **Duplicate IDs**: Cloned VMs often share `/etc/machine-id`. -- **Fix**: Run `rm /etc/machine-id && systemd-machine-id-setup` on the clone. -- **Identity note**: The displayed IP is not the durable identity. Pulse uses - the machine ID or an explicit agent ID, so two clones with the same value can - collapse into one record even when their hostnames or IP addresses differ. + +Cloned hosts can share a **saved Pulse agent ID**, not just an OS machine ID. +The agent uses an explicit ID first, then its saved `agent-id` file, and derives +one from the machine only when neither is available. Changing a hostname, IP or +`/etc/machine-id` therefore does not necessarily change its Pulse identity. + +Compare the affected hosts in **Agent Doctor** and inspect only their configured +`agent-id` files locally. Do not delete the OS machine ID, agent state or Pulse +history as a troubleshooting step. If a duplicate is confirmed, give only the +clone a stable, unique ID in its managed service or container configuration; +leave the original host unchanged. Follow +[Clone identity recovery](UNIFIED_AGENT.md#duplicate-agents) for the configuration +precedence, systemd example and checks after restart. ### Notifications diff --git a/docs/UNIFIED_AGENT.md b/docs/UNIFIED_AGENT.md index 7207a29560..ff7435ca9d 100644 --- a/docs/UNIFIED_AGENT.md +++ b/docs/UNIFIED_AGENT.md @@ -412,7 +412,7 @@ sudo chmod 0700 /usr/local/libexec/pulse-queue-depth | `--insecure` | `PULSE_INSECURE_SKIP_VERIFY` | Skip TLS verification | `false` | | `--allow-plaintext-http` | `PULSE_AGENT_ALLOW_PLAINTEXT_HTTP` | Allow plain HTTP to a Pulse server that does not look local (private IP, single-label, `.local`/`.lan`/`.home`/`.home.arpa`/`.internal`, or resolves to private addresses). Sends the API token in cleartext; only for networks you fully control, e.g. internal networks numbered from public IP space | `false` | | `--hostname` | `PULSE_HOSTNAME` | Override hostname | *(OS hostname)* | -| `--agent-id` | `PULSE_AGENT_ID` | Unique agent identifier | *(machine-id)* | +| `--agent-id` | `PULSE_AGENT_ID` | Unique agent identifier | *(saved ID, then machine ID)* | | `--report-ip` | `PULSE_REPORT_IP` | Override reported IP (multi-NIC) | *(auto)* | | `--disable-ceph` | `PULSE_DISABLE_CEPH` | Disable local Ceph status polling | `false` | | `--tag` | `PULSE_TAGS` | Apply tags (repeatable or CSV) | *(none)* | @@ -834,20 +834,53 @@ file and re-run the installer to pick up the rotating configuration. identity evidence. The endpoint reports repair handoffs but does not run them. ### Duplicate Agents -If cloned VMs appear as the same agent: -```bash -sudo rm /etc/machine-id && sudo systemd-machine-id-setup -``` -Or set a unique agent ID: -```bash ---agent-id my-unique-agent-id -``` +Cloning an installed agent can copy its saved identity and credentials. Different +hostnames, MAC addresses or IPs do not make those agents distinct in Pulse. +Identity selection is: -The displayed or reported IP is not the durable agent identity. Pulse normally -uses the machine ID (or an explicit `--agent-id`), so cloned systems must have -unique machine and agent IDs even when their hostnames, MAC addresses, and IPs -differ. +1. An explicit `--agent-id` argument (which overrides `PULSE_AGENT_ID`). +2. `PULSE_AGENT_ID` when no explicit argument is supplied. +3. The saved `agent-id` file, when no override is set. +4. The machine-derived ID, with hostname as a fallback, when no saved ID exists. + +First compare the affected hosts in **Agent Doctor**. Inspect the identity file +locally: the default Linux path is `/var/lib/pulse-agent/agent-id`; `--state-dir` +changes that directory and `--agent-id-file` or `PULSE_AGENT_ID_FILE` can select +another file. Do not upload `connection.env`, token files or whole service and +environment dumps; older installations can contain credentials there. + +**Do not delete `/etc/machine-id`, agent state or Pulse history to repair a +duplicate.** A saved agent ID survives an OS machine-ID change, and changing the +OS identity affects more than Pulse. Prepare OS identities when provisioning a +clone, using the distribution's instructions, not during live Pulse diagnosis. + +For a confirmed duplicate, keep the original host unchanged and choose one +stable, unique override for the clone. Persist it in the configuration that +actually starts that agent, rather than launching a second agent process: + +- **Systemd, without an existing `--agent-id` argument**: use + `sudo systemctl edit pulse-agent.service` and add the following drop-in, + replacing the example ID with the clone's unique ID: + + ```ini + [Service] + Environment="PULSE_AGENT_ID=vm-clone-02" + ``` + + Then restart only that agent with `sudo systemctl restart pulse-agent.service`. + If its managed launch command already supplies `--agent-id`, change that + existing override instead: the argument wins over the environment variable. +- **Docker Compose**: set `PULSE_AGENT_ID` for the clone's agent service, keeping + its existing volumes, collector options and token-file configuration. An + existing `--agent-id` in `command` takes precedence; update it instead if set. + +Keep the override across restarts and upgrades. After restart, verify that both +hosts report fresh, distinct IDs in Agent Doctor and that each row's metrics +belong to the correct host. Changing an ID does not split or recover historical +data already combined under the old ID. If authentication or reporting fails, +keep the bounded, redacted error for diagnosis; do not delete tokens, re-enrol +blindly or remove the original host's record. ### Permission Denied (Docker) Ensure the agent can access the Docker socket: diff --git a/frontend-modern/public/docs/TROUBLESHOOTING.md b/frontend-modern/public/docs/TROUBLESHOOTING.md index 0882611df1..eaa7e5cebb 100644 --- a/frontend-modern/public/docs/TROUBLESHOOTING.md +++ b/frontend-modern/public/docs/TROUBLESHOOTING.md @@ -155,11 +155,19 @@ repair an older generated unit rather than adding a JSON-parsing wrapper. - See [Temperature Monitoring](TEMPERATURE_MONITORING.md). #### Docker hosts appearing/disappearing -- **Duplicate IDs**: Cloned VMs often share `/etc/machine-id`. -- **Fix**: Run `rm /etc/machine-id && systemd-machine-id-setup` on the clone. -- **Identity note**: The displayed IP is not the durable identity. Pulse uses - the machine ID or an explicit agent ID, so two clones with the same value can - collapse into one record even when their hostnames or IP addresses differ. + +Cloned hosts can share a **saved Pulse agent ID**, not just an OS machine ID. +The agent uses an explicit ID first, then its saved `agent-id` file, and derives +one from the machine only when neither is available. Changing a hostname, IP or +`/etc/machine-id` therefore does not necessarily change its Pulse identity. + +Compare the affected hosts in **Agent Doctor** and inspect only their configured +`agent-id` files locally. Do not delete the OS machine ID, agent state or Pulse +history as a troubleshooting step. If a duplicate is confirmed, give only the +clone a stable, unique ID in its managed service or container configuration; +leave the original host unchanged. Follow +[Clone identity recovery](UNIFIED_AGENT.md#duplicate-agents) for the configuration +precedence, systemd example and checks after restart. ### Notifications diff --git a/frontend-modern/public/docs/UNIFIED_AGENT.md b/frontend-modern/public/docs/UNIFIED_AGENT.md index 7207a29560..ff7435ca9d 100644 --- a/frontend-modern/public/docs/UNIFIED_AGENT.md +++ b/frontend-modern/public/docs/UNIFIED_AGENT.md @@ -412,7 +412,7 @@ sudo chmod 0700 /usr/local/libexec/pulse-queue-depth | `--insecure` | `PULSE_INSECURE_SKIP_VERIFY` | Skip TLS verification | `false` | | `--allow-plaintext-http` | `PULSE_AGENT_ALLOW_PLAINTEXT_HTTP` | Allow plain HTTP to a Pulse server that does not look local (private IP, single-label, `.local`/`.lan`/`.home`/`.home.arpa`/`.internal`, or resolves to private addresses). Sends the API token in cleartext; only for networks you fully control, e.g. internal networks numbered from public IP space | `false` | | `--hostname` | `PULSE_HOSTNAME` | Override hostname | *(OS hostname)* | -| `--agent-id` | `PULSE_AGENT_ID` | Unique agent identifier | *(machine-id)* | +| `--agent-id` | `PULSE_AGENT_ID` | Unique agent identifier | *(saved ID, then machine ID)* | | `--report-ip` | `PULSE_REPORT_IP` | Override reported IP (multi-NIC) | *(auto)* | | `--disable-ceph` | `PULSE_DISABLE_CEPH` | Disable local Ceph status polling | `false` | | `--tag` | `PULSE_TAGS` | Apply tags (repeatable or CSV) | *(none)* | @@ -834,20 +834,53 @@ file and re-run the installer to pick up the rotating configuration. identity evidence. The endpoint reports repair handoffs but does not run them. ### Duplicate Agents -If cloned VMs appear as the same agent: -```bash -sudo rm /etc/machine-id && sudo systemd-machine-id-setup -``` -Or set a unique agent ID: -```bash ---agent-id my-unique-agent-id -``` +Cloning an installed agent can copy its saved identity and credentials. Different +hostnames, MAC addresses or IPs do not make those agents distinct in Pulse. +Identity selection is: -The displayed or reported IP is not the durable agent identity. Pulse normally -uses the machine ID (or an explicit `--agent-id`), so cloned systems must have -unique machine and agent IDs even when their hostnames, MAC addresses, and IPs -differ. +1. An explicit `--agent-id` argument (which overrides `PULSE_AGENT_ID`). +2. `PULSE_AGENT_ID` when no explicit argument is supplied. +3. The saved `agent-id` file, when no override is set. +4. The machine-derived ID, with hostname as a fallback, when no saved ID exists. + +First compare the affected hosts in **Agent Doctor**. Inspect the identity file +locally: the default Linux path is `/var/lib/pulse-agent/agent-id`; `--state-dir` +changes that directory and `--agent-id-file` or `PULSE_AGENT_ID_FILE` can select +another file. Do not upload `connection.env`, token files or whole service and +environment dumps; older installations can contain credentials there. + +**Do not delete `/etc/machine-id`, agent state or Pulse history to repair a +duplicate.** A saved agent ID survives an OS machine-ID change, and changing the +OS identity affects more than Pulse. Prepare OS identities when provisioning a +clone, using the distribution's instructions, not during live Pulse diagnosis. + +For a confirmed duplicate, keep the original host unchanged and choose one +stable, unique override for the clone. Persist it in the configuration that +actually starts that agent, rather than launching a second agent process: + +- **Systemd, without an existing `--agent-id` argument**: use + `sudo systemctl edit pulse-agent.service` and add the following drop-in, + replacing the example ID with the clone's unique ID: + + ```ini + [Service] + Environment="PULSE_AGENT_ID=vm-clone-02" + ``` + + Then restart only that agent with `sudo systemctl restart pulse-agent.service`. + If its managed launch command already supplies `--agent-id`, change that + existing override instead: the argument wins over the environment variable. +- **Docker Compose**: set `PULSE_AGENT_ID` for the clone's agent service, keeping + its existing volumes, collector options and token-file configuration. An + existing `--agent-id` in `command` takes precedence; update it instead if set. + +Keep the override across restarts and upgrades. After restart, verify that both +hosts report fresh, distinct IDs in Agent Doctor and that each row's metrics +belong to the correct host. Changing an ID does not split or recover historical +data already combined under the old ID. If authentication or reporting fails, +keep the bounded, redacted error for diagnosis; do not delete tokens, re-enrol +blindly or remove the original host's record. ### Permission Denied (Docker) Ensure the agent can access the Docker socket: diff --git a/frontend-modern/src/utils/__tests__/docsLinks.test.ts b/frontend-modern/src/utils/__tests__/docsLinks.test.ts index ffe039997f..6e7cfdc2f9 100644 --- a/frontend-modern/src/utils/__tests__/docsLinks.test.ts +++ b/frontend-modern/src/utils/__tests__/docsLinks.test.ts @@ -152,6 +152,37 @@ describe('docsLinks', () => { expect(help).toContain('If installation never started Pulse'); }); + it('keeps clone identity recovery separate from destructive OS or credential resets', () => { + const troubleshooting = readFileSync( + path.join(repoRoot, 'docs', 'TROUBLESHOOTING.md'), + 'utf8', + ); + const agentGuide = readFileSync( + path.join(repoRoot, 'docs', 'UNIFIED_AGENT.md'), + 'utf8', + ); + const shortGuide = troubleshooting + .split('#### Docker hosts appearing/disappearing')[1] + .split('\n### ')[0]; + const cloneGuide = agentGuide.split('### Duplicate Agents')[1].split('\n### ')[0]; + + expect(shortGuide).toContain('UNIFIED_AGENT.md#duplicate-agents'); + for (const section of [shortGuide, cloneGuide]) { + expect(section).toContain('saved'); + expect(section).toContain('Agent Doctor'); + expect(section).toContain('original host unchanged'); + expect(section).not.toContain('systemd-machine-id-setup'); + expect(section).not.toMatch(/(?:sudo\s+)?rm\s+(?:-\S+\s+)*\/etc\/machine-id/); + } + expect(cloneGuide).toContain('argument wins over the environment variable'); + expect(cloneGuide).toContain('PULSE_AGENT_ID_FILE'); + expect(cloneGuide).toContain('Environment="PULSE_AGENT_ID=vm-clone-02"'); + expect(cloneGuide).toContain('fresh, distinct IDs'); + expect(cloneGuide).toContain('does not split or recover historical'); + expect(cloneGuide).toContain('Do not upload `connection.env`, token files'); + expect(cloneGuide).toContain('do not delete tokens, re-enrol'); + }); + it('ships a bounded maintenance example separately from incident snoozing', () => { const apiReference = readFileSync(path.join(repoRoot, 'docs', 'API.md'), 'utf8'); const maintenance = apiReference diff --git a/scripts/check-clone-identity-guidance.cjs b/scripts/check-clone-identity-guidance.cjs new file mode 100644 index 0000000000..d99f4539cc --- /dev/null +++ b/scripts/check-clone-identity-guidance.cjs @@ -0,0 +1,107 @@ +// Real shipped documents, production Docs renderer/router; no live host changes. +// Run from the assigned workspace root: +// pulse-worker-browser scripts/check-clone-identity-guidance.cjs +const fs = require('node:fs'); +const path = require('node:path'); +const crypto = require('node:crypto'); +const { chromium } = require('playwright'); + +const root = process.cwd(); +const frontend = path.join(root, 'frontend-modern'); +const output = path.join(root, 'tmp', 'clone-identity-guidance'); +const port = 5198; +const observations = []; +const errors = []; + +function check(condition, message) { + observations.push({ passed: condition, message }); + if (!condition) throw new Error(message); +} + +async function sectionText(page, id) { + return page.locator(`#${id}`).evaluate((heading) => { + const parts = [heading.textContent]; + for (let node = heading.nextElementSibling; node; node = node.nextElementSibling) { + if (/^H[1-3]$/.test(node.tagName)) break; + parts.push(node.textContent); + } + return parts.join('\n').replace(/\s+/g, ' '); + }); +} + +(async () => { + fs.mkdirSync(output, { recursive: true }); + process.chdir(frontend); + const { createServer } = await import( + path.join(frontend, 'node_modules', 'vite', 'dist', 'node', 'index.js') + ); + const server = await createServer({ + root: frontend, + configFile: path.join(frontend, 'vite.config.ts'), + server: { host: '127.0.0.1', port, strictPort: true }, + }); + let browser; + let result; + try { + await server.listen(); + browser = await chromium.launch({ + headless: true, channel: 'chromium', args: ['--no-sandbox'], + }); + const page = await browser.newPage(); + page.on('pageerror', (error) => errors.push(error.message)); + page.on('response', (response) => { + if (response.status() >= 400) errors.push(`${response.status()} ${response.url()}`); + }); + // Authentication probing only; both Markdown documents are real assets. + await page.route(new RegExp(`^http://127\\.0\\.0\\.1:${port}/api/`), (route) => + route.fulfill({ + status: 200, contentType: 'application/json', + body: JSON.stringify({ hasAuthentication: false }), + }), + ); + + for (const [size, width, height] of [['desktop', 1440, 900], ['phone', 390, 844]]) { + await page.setViewportSize({ width, height }); + await page.goto(`http://127.0.0.1:${port}/browser-tests/docs-fragment-navigation.html?scenario=troubleshooting`); + await page.waitForFunction(() => document.activeElement?.id === 'recovery-mode'); + await page.locator('#docker-hosts-appearingdisappearing').scrollIntoViewIfNeeded(); + const shortGuide = await sectionText(page, 'docker-hosts-appearingdisappearing'); + check(shortGuide.includes('saved Pulse agent ID'), `${size}: saved clone identity is identified`); + check(shortGuide.includes('Do not delete the OS machine ID'), `${size}: no destructive OS reset is prescribed`); + const link = page.getByRole('link', { name: 'Clone identity recovery', exact: true }); + check((await link.getAttribute('href')) === '/docs/UNIFIED_AGENT#duplicate-agents', `${size}: link targets the shipped agent guide`); + check(!(await page.evaluate(() => document.documentElement.scrollWidth > document.documentElement.clientWidth)), `${size}: troubleshooting does not overflow`); + await page.screenshot({ path: path.join(output, `${size}-troubleshooting.png`) }); + + await link.focus(); + await page.keyboard.press('Enter'); + await page.waitForFunction(() => document.activeElement?.id === 'duplicate-agents'); + const cloneGuide = await sectionText(page, 'duplicate-agents'); + check(cloneGuide.includes('argument wins over the environment variable'), `${size}: argument precedence is explicit`); + check(cloneGuide.includes('PULSE_AGENT_ID_FILE'), `${size}: custom identity files are covered`); + check(cloneGuide.includes('Environment="PULSE_AGENT_ID=vm-clone-02"'), `${size}: persistent systemd configuration is rendered`); + check(cloneGuide.includes('fresh, distinct IDs') && cloneGuide.includes('does not split or recover historical'), `${size}: verification and history limitations are rendered`); + check(cloneGuide.includes('Do not upload connection.env, token files'), `${size}: credential-bearing evidence is excluded`); + check(!(await page.evaluate(() => document.documentElement.scrollWidth > document.documentElement.clientWidth)), `${size}: agent guide does not overflow`); + await page.locator('article pre').filter({ hasText: 'PULSE_AGENT_ID=vm-clone-02' }).scrollIntoViewIfNeeded(); + await page.screenshot({ path: path.join(output, `${size}-agent-configuration.png`) }); + } + check(errors.length === 0, 'no page errors or failed HTTP responses'); + result = { + result: 'passed', observations, errors, chromium: browser.version(), + playwright: require('playwright/package.json').version, + sourceAssets: ['TROUBLESHOOTING.md', 'UNIFIED_AGENT.md'].map((name) => ({ + path: `frontend-modern/public/docs/${name}`, + sha256: crypto.createHash('sha256').update(fs.readFileSync(path.join(frontend, 'public', 'docs', name))).digest('hex'), + })), + }; + } catch (error) { + result = { result: 'failed', observations, errors, error: error.message }; + } finally { + if (browser) await browser.close(); + await server.close(); + } + fs.writeFileSync(path.join(output, 'result.json'), JSON.stringify(result, null, 2)); + console.log(JSON.stringify(result, null, 2)); + process.exitCode = result.result === 'passed' ? 0 : 1; +})();