`apt-get upgrade` holds back any package whose upgrade needs new dependencies,
so it neither pins nor updates — it produces whatever partial set the
dependency graph allows that day. On these nodes it upgraded pve-cluster to
9.1.6 while leaving libpve-cluster-api-perl at 9.1.0.
Those two ship the halves of the join: cfs_backup_database() in
PVE/Cluster.pm (pve-cluster) and finish_join() in PVE/Cluster/Setup.pm
(libpve-cluster-api-perl). Upstream removed `return $dbfile` from the former
and stopped relying on it in the latter, both at 9.1.1 — 9.1.6's finish_join
calls cfs_unlink_db_unsafe() instead. The 9.1.0 caller against the 9.1.6
callee unlinks an empty string, so the standalone config.db survives the join,
pmxcfs restarts in local mode, and the node reports online=0 forever while
corosync forms a healthy 2-node membership. That is the "2 nodes online"
failure, and it is not reachable on any coherent install.
The ISO is the pin, so drop the upgrade and install only what the harness
needs. Upgrades belong in a separate currency lane that records the package
set it tested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ensures nested PVE nodes are fully patched before integration tests
run. Adds ~5-10min to first provision but gives more realistic test
results against current PVE releases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Provision two PVE nodes per version (a/b) for future cluster testing,
plus Docker-based iSCSI target and NFS server for shared storage tests.
Multi-node changes:
- Each PVE version gets two nodes: 9a/9b and 8a/8b (4 VMs total)
- Parameterized answer.toml FQDN for unique hostnames per node
- Per-node auto-install ISOs with unique answer files
- Node name discovered from FQDN and included in test config
- API token creation handles pre-existing tokens (delete + recreate)
- New test env vars: PVETEST_HOST_B, PVETEST_APITOKEN_B
- Removed preflight cleanup from provision (use explicit cleanup instead)
Docker storage services:
- New docker-compose.storage.yml with iSCSI (tgt) and NFS containers
- Host networking so PVE nodes can reach storage services
- Docker socket mounted into dev-infra container for host Docker access
- Docker CLI added to dev-infra Dockerfile stage
- New test env vars: PVETEST_STORAGE_VM_IP, PVETEST_ISCSI_IQN, PVETEST_NFS_EXPORT
Other fixes:
- first-boot.sh installs open-iscsi on PVE nodes
- preflight-cleanup.sh handles empty ISO filename gracefully
- TMPDIR set to work dir to avoid /tmp overflow during ISO uploads
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PVE 8 (Bookworm) may not have /etc/apt/sources.list.d/debian.sources,
causing the grep fallback to use 'trixie' which is invalid for PVE 8.
Use /etc/os-release VERSION_CODENAME which works on both PVE 8 and 9.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- first-boot.sh: Remove .sources files (PVE 9 DEB822 format) not just
.list, and detect suite (trixie vs bookworm) from debian.sources
- create-api-token.sh: Accept full endpoint URL (fixes missing port),
use /access/domains for API check (no auth needed on PVE 9),
avoid pipefail on curl|python3 pipe
- preflight-cleanup.sh: New script that runs at start and end of every
provisioning run — cleans orphaned VMs, ISOs, and stale terraform state
- Workflow: Add pre-flight and final cleanup steps so failed runs can't
leave the runner/host in a broken state
All changes validated manually on the self-hosted runner.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Workflow now provisions a throwaway nested PVE VM via Terraform, runs
integration tests against it, then destroys it. Uses proxmox-auto-install-assistant
to bake the answer file and a first-boot script (installs qemu-guest-agent)
directly into the ISO. IP is discovered via the QEMU guest agent on the
parent PVE, eliminating the need for static IP configuration.
Supports both PVE 8.x and 9.x via workflow_dispatch version selector.
Skip provisioning with skip_provision=true to test against a pre-existing PVE.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
goodolclint-claude[bot]