Review finding on #97: `statusReached` was sticky, so a guest that reached the
expected status once and then drifted away from it would still be reported as a
success at the deadline, contradicting the fallback's own meaning.
Track the most recent observation instead. A guest that drifts away and never
re-matches now raises PveTaskTimeoutException as it should. A failed poll leaves
the previous observation standing, so an API blip is not read as divergence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ci: test against the PVE 9.2 ISO
The nested nodes were pinned to proxmox-ve_9.1-1.iso while production runs 9.2.
Validated by runs 183/184; the cluster suite passed in both, and run 184 was
fully green at 148 passed / 0 failed / 5 skipped.
PVE publishes a guest's new status while the operation still holds
/var/lock/qemu-server/lock-<vmid>.conf, so WaitForStatusTransition could return
while the guest was still locked and the caller's next request would fail with
"can't lock file ... got timeout".
Integration run 183 failed four tests from this one cause: Restart-PveVm -Wait
returned after 4.1s having seen "running", the following Stop-PveVm spent exactly
10.0s failing to take the lock, and that cascaded into the template convert,
clone, and remove tests. Run 184 - same commit, re-run - passed because its status
poll happened to take 10.1s, by which point the lock had cleared. The same
settling happens either way; the only variable is whether the wait absorbs it or
the next caller does.
The check goes in WaitForStatusTransition because all nine lifecycle call sites
(Start/Stop/Restart/Reset/Resume across VMs and containers) route through it.
`lock` comes from the status/current response the poll already fetches - present
on both qemu and lxc since PVE 5.4, below the module's 7.0 floor - so it costs no
extra request.
If the status is reached but the lock outlasts -Timeout the cmdlet still returns
success, so a call that succeeded before this change cannot become an exception
after it.
Recorded as D015, the guest-lock sibling of D014.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fix: New-PveCluster -Wait blocks until quorate; make cluster failures diagnosable
Adds ClusterConfigService.WaitForQuorum with a -Timeout parameter (D014), so
Add-PveClusterMember no longer hits "cluster not ready - no quorum?" after a
completed create task. Adds diagnose-cluster.sh, which captures corosync and
pmxcfs state plus PVE task logs from both nodes before cleanup destroys them.
The nested nodes were pinned to proxmox-ve_9.1-1.iso while production runs
9.2, so the suite was validating a PVE nobody uses. run-integration.sh already
reads PVE9_ISO with the 9.1 name only as a fallback, so this needs no script
change; setting it at the workflow level keeps provision and cleanup agreeing
on the ISO name, which force-cleanup needs to find the right file to delete.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`apt-get upgrade` holds back any package whose upgrade needs new dependencies,
so it neither pins nor updates — it produces whatever partial set the
dependency graph allows that day. On these nodes it upgraded pve-cluster to
9.1.6 while leaving libpve-cluster-api-perl at 9.1.0.
Those two ship the halves of the join: cfs_backup_database() in
PVE/Cluster.pm (pve-cluster) and finish_join() in PVE/Cluster/Setup.pm
(libpve-cluster-api-perl). Upstream removed `return $dbfile` from the former
and stopped relying on it in the latter, both at 9.1.1 — 9.1.6's finish_join
calls cfs_unlink_db_unsafe() instead. The 9.1.0 caller against the 9.1.6
callee unlinks an empty string, so the standalone config.db survives the join,
pmxcfs restarts in local mode, and the node reports online=0 forever while
corosync forms a healthy 2-node membership. That is the "2 nodes online"
failure, and it is not reachable on any coherent install.
The ISO is the pin, so drop the upgrade and install only what the harness
needs. Upgrades belong in a separate currency lane that records the package
set it tested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 173 left node B with healthy corosync (2-member primary component, both
links connected) but no /etc/pve/corosync.conf, no dcdb/status journal lines,
and pvecm status reporting it is not part of a cluster. That file is
database-backed: pmxcfs creates it only when it starts with no config.db and
imports /etc/corosync/corosync.conf, so a surviving standalone config.db would
mean silent local mode.
Capture the package versions, pmxcfs command line, /etc/pve mount, .members,
the config.db and its backup dir, whether the database holds a corosync.conf
row, and the CPG group membership. Read-only; the sqlite3 CLI is not guaranteed
on a PVE node, so fall back to strings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Also documents the two distinct timeout conventions in the module so the
-Wait vs HTTP-client distinction does not have to be re-derived.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
New-PveCluster -Wait now guarantees quorum, so the Start-Sleep -Seconds 5
that lost the race is replaced by an assertion on the new contract.
The link0 pin is dropped: the join-abort and the never-a-member modes both
occurred with and without it, so it was never implicated, and -Links is
already covered at the service level.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Returns once quorate, throws TimeoutException when quorum never arrives,
and keeps polling through a PveApiException from the restarting API.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Polls GET /cluster/status for the cluster entry with quorate = 1, bounded
by a timeout (60 s default) and tolerating PveApiException while pmxcfs
and corosync restart during cluster formation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PVE's cluster-create task completes before corosync converges. Until the
node is quorate it rejects a join with "cluster not ready - no quorum?",
so New-PveCluster -Wait followed by Add-PveClusterMember failed for every
caller. Wait for quorum after the task, bounded by -Timeout (default 60 s,
following the -Wait timeout convention used by Stop-PveContainer and
Reset-PveVm). See DECISIONS.md D014.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
goodolclint-claude[bot]