refactor(ci): parallel provisioning, file caching, provision/test/cleanup split

Restructures the integration test workflow from a monolithic sequential
job into separate provision → test → cleanup stages:

- provision: creates ALL nested PVE VMs in a single terraform apply
  (parallel), waits for APIs, creates tokens, passes outputs to tests
- test: matrix [pve9, pve8] consumes provision outputs, no provisioning
- cleanup: always runs, API-only teardown for all VMs

Terraform refactored to for_each with pve_instances map variable,
enabling parallel ISO upload and VM creation.

New caching scripts reduce redundant downloads:
- ensure-base-iso.sh: downloads PVE ISOs to /opt/pve-isos if missing
- ensure-cloud-images.sh: caches cloud image + OVA with 7-day TTL
- prepare-auto-iso.sh: --cache-dir flag with hash-based skip

Runner no longer needs manual ISO provisioning (zero-touch setup).
cleanup-images bumped to min-versions-to-keep: 3 to survive overlapping
runs. All jobs gated against dependabot.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Clint Branham
2026-03-23 16:05:20 -05:00
parent d19c6f0c4b
commit a424dd18ac
9 changed files with 561 additions and 221 deletions
@@ -1,26 +1,17 @@
#!/usr/bin/env bash
# Prepares the test environment on the nested PVE node.
# Only performs operations that have no PVE API equivalent, plus
# downloads test artifacts for the integration tests to upload.
# Only performs operations that have no PVE API equivalent.
#
# Usage: prepare-test-environment.sh <nested-pve-ip> <root-password> <output-dir>
# Usage: prepare-test-environment.sh <nested-pve-ip> <root-password>
#
# Operations:
# - Enable snippets+import content types on local storage (pvesm set)
# - Upload cloud-init user-data snippet (SCP — no snippet upload API)
# - Download Ubuntu cloud image to <output-dir> for upload tests
set -euo pipefail
NESTED_IP="${1:?Usage: prepare-test-environment.sh <ip> <password> <output-dir>}"
NESTED_IP="${1:?Usage: prepare-test-environment.sh <ip> <password>}"
ROOT_PASS="$2"
OUTPUT_DIR="${3:?Output directory required}"
CLOUD_IMAGE_URL="https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img"
# PVE upload API validates extensions per content type — content=import
# does not accept .img. The Ubuntu cloud image is qcow2 format, so we
# rename it to .qcow2 for compatibility with the upload endpoint.
CLOUD_IMAGE_FILENAME="noble-server-cloudimg-amd64.qcow2"
SSH_OPTS="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR"
SSH_CMD="sshpass -p ${ROOT_PASS} ssh ${SSH_OPTS} root@${NESTED_IP}"
@@ -47,27 +38,4 @@ YAML
${SCP_CMD} "${USERDATA}" "root@${NESTED_IP}:/var/lib/vz/snippets/test-vm-userdata.yml"
rm -f "${USERDATA}"
# Download cloud image for integration tests to upload via Send-PveFile
CLOUD_IMAGE_PATH="${OUTPUT_DIR}/${CLOUD_IMAGE_FILENAME}"
if [ ! -f "${CLOUD_IMAGE_PATH}" ]; then
echo "Downloading Ubuntu cloud image..."
curl -fSL -o "${CLOUD_IMAGE_PATH}" "${CLOUD_IMAGE_URL}"
else
echo "Cloud image already cached at ${CLOUD_IMAGE_PATH}"
fi
# Download Ubuntu cloud OVA for Import-PveOva testing
OVA_URL="https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-amd64.ova"
OVA_FILENAME="ubuntu-24.04-server-cloudimg-amd64.ova"
OVA_PATH="${OUTPUT_DIR}/${OVA_FILENAME}"
if [ ! -f "${OVA_PATH}" ]; then
echo "Downloading Ubuntu cloud OVA (this may take a few minutes)..."
curl -fSL -o "${OVA_PATH}" "${OVA_URL}"
echo "Downloaded OVA ($(du -h "${OVA_PATH}" | cut -f1))"
else
echo "OVA already cached at ${OVA_PATH}"
fi
echo "CLOUD_IMAGE_PATH=${CLOUD_IMAGE_PATH}"
echo "OVA_PATH=${OVA_PATH}"
echo "Environment preparation complete."