mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-24 04:33:17 +00:00
rcourtman
bc15c3500b
## Problem
Nested Proxmox clusters (each PVE node is itself a QEMU VM) lose
physical disks from inventory. QEMU gives a virtual disk configured
without a serial a default one built from its drive ID or a per-VM
counter, so every VM built the same way reports the same value:
- Proxmox's `/nodes/{node}/disks/list` serial is udev's
`ID_SERIAL_SHORT` (`PVE/Diskmanage.pm`). A virtio-scsi disk on a Proxmox
VM reports `drive-scsi0` with no `ID_WWN`; on a libvirt VM it is
`drive-scsi0-0-0-0`. A `-drive` given no ID falls back to QEMU's
automatic ID, `scsi0-hd0` for `if=scsi` and `none0` for `if=none`.
- An emulated ATA disk's serial is `QM` plus a five-digit counter
(`QM00001`).
`diskinventory.IsUsableHardwareID` accepted all of these, so
`resourceFromPhysicalDisk` set `MachineID = "drive-scsi0"` on every node
and the registry minted one `machine:drive-scsi0` canonical disk and one
SMART series for the whole cluster. Three nodes with two such disks each
produced 2 canonical disks instead of 6.
## Change
- `IsUsableHardwareID` rejects exactly QEMU's default shapes:
`drive-scsi` plus digits with at most three `-digits` groups,
`scsi<n>-hd<n>`, `none<n>`, and `QM` plus five digits. Real serials that
only share a prefix (`DRIVE-ASSET123`, `drive-sata0`, `QM0001`) stay
identity. It also strips `naa.`/`eui.`/`wwn-`/`0x` framing before its
all-zero and all-F test, so a `0x0000000000000000` WWN is a placeholder
too.
- TrueNAS provider: a disk took its canonical identity from any
non-empty serial, so a placeholder merged disks across appliances. It
now uses only a usable serial, and lists superseded IDs for every disk
without one.
- TrueNAS history keys: for a serial that was already rejected, such as
`UNKNOWN`, native TrueNAS history used TrueNAS's own disk identifier,
and the unified SMART writer (`syncUnifiedPhysicalDiskMetrics`)
substituted the metrics target only for an empty serial, so the samples
went to the canonical resource ID. Neither matched the chart reader's
key, and rejecting QEMU serials would have put TrueNAS VMs on that path.
Native history now falls back to the disk's source ID, and the writer
always writes under the resolved metrics target, so writer, native
history and reader agree. History lookup keys also drop placeholder
serials.
## Upgrade effect
A disk reporting one of these values (nested PVE nodes, TrueNAS or agent
hosts running as QEMU VMs with virtual disks, TrueNAS disks reporting
`UNKNOWN`) changes canonical ID and SMART metric key once. Samples
recorded under the old key are not carried over, because that key mixed
every disk that shared the value. Path-shaped alert references (Proxmox
disk health and wearout, host-agent SMART) are unaffected; anything
keyed on the old canonical ID or metric key starts again under the new
one. This is recorded in the monitoring and agent-lifecycle contracts.
## Tests
- `TestIsUsableHardwareIDRejectsQEMUDefaultDiskSerials`: each default
shape is rejected, falls back to the scoped topology ID and never
matches itself; prefix-sharing real serials stay usable.
`TestPreferredIDRejectsPlaceholderHardwareIdentity` gains the framed
all-zero/all-F WWNs.
- `TestNestedProxmoxDefaultQEMUSerialsStayPerNode`: 3 nodes ×
(`drive-scsi0`, `QM00005`, `drive-scsi0-0-0-1`) give 9 canonical disks,
9 slots and 9 metric keys, every SMART writer key is a key readers
resolve, and agent SMART source IDs stay per host.
-
`TestTrueNASPlaceholderDiskSerialsStayPerApplianceAndShareOneHistoryKey`:
two appliances with `UNKNOWN` and `drive-scsiN` serials keep one
canonical disk each, and the SMART writer's samples are readable under
each disk's metrics target.
- `TestProviderPlaceholderDiskSerialsKeyNeitherIdentityNorHistory`:
placeholder serials key no identity, list superseded canonical IDs, drop
out of native history lookup keys, and native history lands under the
registry's metrics target.
- Negative controls via `go test -overlay`, one runtime file reverted at
a time: base `identity.go` fails all three round trips (3 canonical
disks instead of 9, 9 TrueNAS disks instead of 16); base `provider.go`
fails the identity and history assertions; base `monitor.go` fails the
SMART-history-under-reader-key assertion; reverting only the native
history key, the superseded-ID gate or the lookup-key filter fails its
own provider assertion.
- `pkg/diskinventory`, `internal/truenas/...`,
`internal/unifiedresources/...` and `internal/monitoring` pass in full;
`internal/api/...` disk, hardware, serial and TrueNAS tests pass.
Stacked on #2648 (`fix/pve-disk-serial-carry`), whose commit this branch
carries until it merges. Not in this change: the registry's merge of
disks across parents on a canonical-ID collision (separate change), and
a pre-existing writer/reader key mismatch for controller-member disks
without a usable serial or WWN (`PhysicalDiskMetaMetricID` appends
topology to a source ID that already carries it), which is tracked
separately.
Pulse documentation
Start here for installation, platform setup, security, operations, and Pulse Intelligence. Commands, configuration keys, image names, API fields, and product identifiers remain untranslated in localized guides.
Start here
- Install Pulse — signed Proxmox/Linux installation, Docker, Docker Compose, Kubernetes, and first-run setup.
- Production deployment and security — least-privilege Proxmox setup, root-agent boundaries, discovery, supply-chain verification, plan limits, scale evidence, and a rollout checklist.
- Upgrade from Pulse v5 — migration prerequisites, rollback, agent continuity, and post-upgrade checks.
- Releases and update channels: choosing Stable or Preview, beta and RC expectations, and how releases are tested and promoted.
- Configure Pulse — authentication, notifications, discovery, retention, and system settings.
- Deployment models — data locations, lifecycle, and differences between supported deployment paths.
- Troubleshooting and FAQ — common failures, diagnostics, and operator questions.
Localized getting started guides: Deutsch · Español
Platforms and agents
- Proxmox Backup Server
- Proxmox Mail Gateway
- Docker and Podman
- Kubernetes and Helm
- TrueNAS SCALE and CORE
- Unified Agent
- Agent security
- VM disk monitoring
- ZFS monitoring
- Temperature monitoring
VMware vSphere support is early access. Current builds expose dedicated vSphere inventory and recovery context, but operators should validate the integration against their own vCenter before production use.
Monitoring and operations
- Metrics history
- Recovery data
- Webhooks
- Automatic updates
- Centralized agent management (Pro)
- Operational trust model
- Current product screenshots
Pulse Intelligence
- Assistant, Patrol, and external-agent overview
- Patrol modes and safety
- Assistant safety model
- External agent HTTP and MCP substrate
Patrol watch-only analysis is available on Community with a local model or the operator's own provider. Investigation and governed fixes require the relevant Pulse Pro capabilities.
Security, privacy, and access
- Production deployment and security
- Security review scope
- Authentication and credential storage review packet
- Security guide
- Privacy and telemetry disclosure
- OIDC and SSO
- Proxy authentication
- Role-based access control (Pro)
- Audit logging (Pro)
- Reverse proxy configuration
- Code-signing policy
Plans and managed access
- Community, Relay, and Pro capabilities
- Relay and Pulse Mobile handoff
- Multi-tenant organizations (Enterprise/custom)
- Provider-hosted MSP operations (request-assisted)
Pulse Cloud is not generally available. Ordinary self-hosted Pulse remains the primary installation path; MSP and Enterprise access are explicit commercial paths rather than defaults in self-hosted setup.
Development and reference
Detailed design notes and dated migration specifications may remain in this directory for maintainers, but they are not operator setup guides unless they are linked from the sections above.
Previous versions and migrations
- Upgrade from v4 to v5
- Retired unified-navigation migration — historical context only; current Pulse uses platform-shaped navigation.
- Move a Pulse installation
Found a bug? Use the issue forms. For setup questions, use GitHub Discussions.