Run 168 burned 13 minutes to report 'unable to locate package tgt' when
the real failure was dead egress from the CI VLAN (apt lists never
updated). The setup script now probes gateway/internet/DNS from inside
the VM and prints the verdicts, fails apt fast with the real error, and
marks each phase.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
run-integration.sh: two-phase apply (storage VM configured before the
PVE guests boot), Pester PassThru with explicit failed/zero-test checks,
auto-ISO cache names embed the answer host, force-cleanup covers the
storage VM and imported cloud image, docker usage removed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The nested PVE guests live on an isolated VLAN with no route to the
runner network (ADR 0032, homelab repo), and the runner-hosted Docker
services died with each ephemeral ARC pod anyway. A small cloud-image VM
in the ci pool now serves NFS, iSCSI, and the auto-install answer files
from inside the sandbox. Includes fixes from Codex and in-house review:
bpg provider >= 0.79.0, serial console for the resized cloud image,
key-based SSH (cloud images refuse password auth), CIDR validation,
storage VM in the headroom check, docker socket mounts removed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PVE_TARGET_NODE becomes optional: unset (or 'auto') queries
/cluster/resources for the online node with the most free memory and
refuses to provision when no node has vm_count x 8GiB + 8GiB headroom
free. The chosen node persists on the shared mount so cleanup and
force-cleanup aim at the node provision picked.
2026-08-31: a provision onto the pinned node OOM-killed a 16GiB
services-plane VM on the parent hypervisor; the pin predated a memory
bump that consumed the headroom it silently relied on. Needs PVEAuditor
on /nodes for the token's user (a non-privsep token ignores token ACLs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- VmService.ExecuteGuestCommand: guard against null elements in -Args.
The old JSON-serialization tolerated nulls (as "null"); the repeated-key
path would NRE in EncodeFormValue. Throw a clear ArgumentException instead.
- findings.json: refresh the stale counters block (untouched since F085) to
the actual ledger state — next_id 92, resolved 83 — and bump last_updated
to 2026-05-22. last_scan_date stays 2026-03-26 (F086–F091 came from issue
triage, not a formal review scan).
- VmServiceTests: add empty-array (single command entry) and null-element
(throws) cases.
Note: F091 is the correct next ID — F086–F090 already exist from prior
merged PRs (#60/#61/#66/#67); only the counters were lagging.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ExecuteGuestCommand JSON-serialized the args array into the agent/exec
'input-data' field — which is the process's STDIN, not its arguments. So
guest commands ran with no argv: cmd.exe started interactively and the
JSON blob ['/c','echo',...] arrived at its prompt.
PVE's agent/exec 'command' parameter is itself an array (element 0 = the
executable, the rest = argv) sent as repeated form keys. The low-level
client couldn't express repeated keys (Dictionary<string,string> only),
so:
- Add PostAsync(string, IEnumerable<KeyValuePair<string,string>>) to
IPveHttpClient/PveHttpClient; BuildFormContent now emits one key=value
field per pair, so a key may repeat.
- ExecuteGuestCommand builds command = [exe] + args as repeated 'command'
fields and no longer touches input-data.
Tests: form-encoder repeated-key + per-value encoding cases; VmService
tests asserting the command array, order, and absence of input-data; an
integration regression guard that echoes an arg and checks it round-trips
as stdout.
Tracked as F091. Closes#68.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- HasDiskOptions(): include -ScsiHardware so the "disk options ignored"
warning fires when -ScsiHardware is passed without -DiskStorage/-DiskSize.
- PveVmConfig.AdditionalProperties: lazy-init a backing field so the native
dictionary is built once rather than reallocated on every property access
(matters when iterating many configs in a pipeline). Safe because the model
is effectively immutable after deserialization.
- New-PveVm.Tests.ps1: add a case asserting -DiskIoThread on scsi with a
wrong -ScsiHardware (virtio-scsi-pci) is rejected, covering the validator's
"!= virtio-scsi-single" branch (not just the null case).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New-PveVm (F089):
- Add -DiskBus (virtio/scsi/sata/ide, default virtio), -ScsiHardware (scsihw),
-DiskIoThread, -DiskAio, -DiskSsd, -DiskDiscard, -DiskCache so a tuned disk
(e.g. virtio-scsi-single + scsi0,iothread=1,aio=native,ssd=1,discard=on) can
be created in one call instead of diskless + a hand-built Set-PveVmConfig string.
- Disk spec built via BuildDiskSpec; ValidateDiskOptions runs before ShouldProcess
and rejects ssd on virtio and iothread on sata/ide or scsi-without-virtio-scsi-single
with clear errors, instead of letting PVE fail at VM start.
Get-PveVmConfig (F090):
- PveVmConfig was a fixed allow-list, silently dropping keys like scsihw, efidisk0,
tpmstate0, hostpci0. Add typed scsihw/efidisk0/tpmstate0 plus a [JsonExtensionData]
catch-all exposed as AdditionalProperties (native types via JsonHelper.ToNative,
per D013 — no JToken leakage). Makes the disk tuning above verifiable by reading
the config back.
Closes#65.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PveHttpClient.EncodeFormValue encoded &, =, +, space, and % but left ';'
literal. PVE's application/x-www-form-urlencoded parser treats a raw ';'
as a field separator (the historical alternative to '&'), so a value like
boot=order=scsi0;ide2
was split into 'boot=order=scsi0' plus an empty 'ide2' field, and PVE
rejected the PUT with "ide2: unable to parse drive options". This broke
any multi-device boot order set via Set-PveVmConfig -AdditionalConfig,
and any other value containing ';'.
Encode ';' as %3B. Safe under the existing minimal-encoding policy that
keeps ':' and '!' literal for cluster-join: cluster-join payloads never
contain ';', and PVE url-decodes config form values.
Tracked as F088. Closes#64.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When HttpClient.Timeout elapses, .NET throws TaskCanceledException — not
HttpRequestException — so the existing catch in PveHttpClient.SendAsync
missed it and callers got a raw stack trace. With -TimeoutSeconds now
configurable and documented, this gap became user-visible.
In .NET 5+ HttpClient surfaces transport timeouts as TaskCanceledException
with a TimeoutException inner; user-driven token cancellation does not.
Catch by that inner-type signature and rethrow as PveApiException with
HttpStatusCode.RequestTimeout, the resource path, and a message that
reports the configured timeout.
Adds SendAsync_TimeoutFires_ThrowsPveApiExceptionWithRequestTimeout which
swaps in a delaying HttpMessageHandler with a 50ms timeout to exercise
the path deterministically. Drops the redundant
DefaultSessionTimeoutIs100Seconds test (covered by
PveSessionTests.Timeout_DefaultIs100Seconds and the existing flow-through
test).
Addresses PR #61 review feedback.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolves findings.json conflict — F086 (from #60, merged into main) and
F087 (this branch) both append to the trailing findings array. Kept both
entries, in numeric order.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PveHttpClient was constructed without setting HttpClient.Timeout, so
.NET's 100s default applied to every request. Multi-GB ISO uploads via
Send-PveFile on a real LAN reliably tripped this with TaskCanceledException
after 100 seconds, and there was no way to override it.
- PveSession gains a Timeout (TimeSpan) property, defaulting to 100s.
- PveHttpClient accepts an optional per-instance timeout override that
takes precedence over the session timeout.
- Connect-PveServer exposes -TimeoutSeconds to set the session default.
- Send-PveFile and Invoke-PveStorageDownload expose -TimeoutSeconds with
a 30-minute implicit default so large uploads/downloads do not trip
the 100s default. -TimeoutSeconds 0 means Timeout.InfiniteTimeSpan.
Tracked as F087. Closes#59.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- SizeParser: wrap TB-suffix overflow in try/catch so callers get
ArgumentException with the parameter name rather than OverflowException.
- New-PveVm/New-PveContainer: validate -DiskSize/-RootFsSize before
ShouldProcess so typos like "512M" are rejected even with -WhatIf and
even when the matching -DiskStorage/-RootFsStorage is omitted.
- Add Pester tests for the new DiskSize and RootFsSize validation paths,
including a new New-PveContainer.Tests.ps1.
- Add SizeParserTests coverage for the TB overflow path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Disk and rootfs size strings were interpolated directly into the disk
spec as "<storage>:<size>", so "60G" produced "local-lvm:60G". On
LVM/LVM-thin storages PVE parses the value after the colon as a volume
name unless it is a bare integer, returning "unable to parse lvm volume
name '60G'". File-backed storages mask this by accepting either form.
SizeParser.NormalizeToGibibytes() now strips G/GB/T/TB suffixes and
returns a bare GiB integer string, so the documented "32G" call shape
works on every storage type. Sub-GB units are rejected with a clear
error rather than being silently truncated.
Tracked as F086. Closes#58.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace single-version fixture with per-version specs from pve-api.
Tests now validate enum values against PVE 7 (best-effort), 8, and 9.
Key findings from version-specific specs:
- VM.Monitor: valid in PVE 7+8, removed in PVE 9
- VM.Replicate: added in PVE 9 only
- VM.GuestAgent.*: added in PVE 9 only
- Mapping.*: added in PVE 8+
- glusterfs: not in any version (removed before PVE 7)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 70 xUnit tests that validate every ValidateSet in the module
against the PVE OpenAPI spec. Three bugs found and fixed:
- Storage: remove `glusterfs` (dropped in PVE 9), add `btrfs`, `esxi`
- Backup compression: `none` → `0` (PVE uses "0" not "none")
- Cluster resources: remove `lxc` filter (PVE uses `vm` for both)
The pve-api-enums.json fixture (199KB) is extracted from the full
OpenAPI spec and contains parameter enum values for 302 API paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The ValidateSet attribute was intentionally removed from the Type
parameter during merge review to allow future rule types without
code changes. Updated test to only assert Type is mandatory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PVE 9 API requires 'type' on PUT /cluster/ha/rules/{rule} even for
updates. Added mandatory Type parameter with ValidateSet for
node-affinity and resource-affinity.
Updated integration test and Pester unit tests to pass -Type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Rename ZZ.ClusterConfig.Integration.Tests.ps1 → 16_Cluster.Tests.ps1
- Replace manual env var setup with _IntegrationHelper.ps1 + Connect-TestPve
- Use credential auth (root@pam) throughout — no API tokens
- Use $script:PasswordB for node B auth (from helper)
- Access JoinInfo.Nodelist as List<Dictionary> (native types from D013)
- Keep cluster-specific skip helpers (Skip-IfNoCluster, Skip-IfPve9HaGroups)
- Remove redundant Connection context (Connect-TestPve handles it)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PVE 9 rejects rules referencing unmanaged resources. Test now creates
a minimal VM, registers it as a disabled HA resource, then creates
the node-affinity rule. Cleanup removes rule, resource, and VM.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PVE 9 rejects 'affinity' as an unexpected property on POST
cluster/ha/rules — it's implied by the node-affinity type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full lifecycle: create node-affinity rule, list, get by ID, update
comment, delete, and verify deletion. Uses vm:99999 as a synthetic
resource SID. Tests skip on PVE 8 (rules not available).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cluster tests create a 2-node cluster that cannot be torn down via API
(quorum loss). Prefix with ZZ so they execute after all other
integration tests that assume standalone nodes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removing a node from a 2-node PVE cluster via REST API is not supported
— the remaining node loses quorum mid-operation, causing "no quorum!"
errors. PVE requires stopping corosync on the departing node first
(pvecm expected 1), which is not available via the REST API.
Replace removal tests with a final cluster state verification.
Test infrastructure handles cleanup via reprovisioning.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove-PveClusterConfigNode requires root@pam (not API token).
Fixed both the test context and AfterAll cleanup to connect with
root@pam credentials before attempting node removal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Extract re-auth retry into ReauthenticateWithRetry helper (fixes
cognitive complexity warning)
- Retry auth up to 10x with 3s delay — node B's auth services need
time to restart after joining the cluster
- Wrap join in try/finally so the test always reconnects to node A,
even if the join fails
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All cluster operations (create, status, join, options, cleanup) require
root@pam ticket auth — API tokens lack Sys.Audit and Sys.Modify on /.
Changed test to connect as root@pam once during cluster creation and
stay on that session for the entire lifecycle.
Removed redundant Connect-PveServer calls that switched between
root@pam and API token between contexts.
Added 5s stabilization sleep after cluster creation for corosync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Convert JoinInfo.Nodelist to native PS array via ConvertFrom-Json
(JArray .Item() still hits IEnumerator error in PowerShell)
- Use root@pam for cluster options context (delete requires Sys.Modify)
- Reconnect with API token after options restore
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
xUnit (47 new tests, 429 total):
- ClusterConfigServiceTests: 25 tests covering all 14 service methods
including URL encoding, null guards, and error responses
- HaServiceTests: 22 tests covering resources, groups, status, and rules
with URI encoding verification (vm:100 → vm%3A100)
Pester (187 new tests, 1525 total):
- ClusterConfigCmdlets.Tests.ps1: 11 cmdlets tested
- HaCmdlets.Tests.ps1: 14 cmdlets tested
Integration (ClusterConfig.Integration.Tests.ps1):
- Full 2-node cluster lifecycle with -Wait for task completion
- Uses root@pam ticket auth for cluster create/join operations
- HA group tests skip on PVE 9.0+ (groups migrated to rules)
- JArray indexing uses .Item() for PowerShell compatibility
Cmdlet improvements:
- New-PveCluster, Add-PveClusterConfigNode, Add-PveClusterMember now
support -Wait switch to block until task completes (via TaskService)
- GetClusterConfig returns JToken to handle array responses on standalone
- OutputType updated to PveTask for task-returning cmdlets
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RUNNER_TEMP (/__w/_temp in CI) is container-local and invisible to
the Docker host. Files written there can't be bind-mounted into
sibling containers (answer server, storage). This was the root cause
of the "not a directory" mount failures in CI.
Removed RUNNER_TEMP from the WORK_DIR fallback chain. WORK_DIR now
always defaults to CACHE_DIR/work (/opt/pve-integration/work/) which
is on the shared mount visible to both the CI container and Docker host.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mounting the whole directory to /app overwrites server.py inside the
container, causing "can't open file '/app/server.py'" errors.
Mount the two paths individually instead:
- answer_server_dir/answers → /app/answers
- answer_server_dir/default.toml → /app/default.toml
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Docker bind mount fails when mounting a file that doesn't exist on
the host (creates a directory instead). In CI, the container's
$WORK_DIR path differs from the host path, so the file mount failed.
Changed to mount a single answer-server/ directory containing both
default.toml and answers/ subdirectory. Replaced two TF variables
(answer_files_dir, default_answer_file) with one (answer_server_dir).
Layout: $WORK_DIR/answer-server/default.toml + answers/<mac>.toml
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Terraform state was stored in the container's working directory
(fresh checkout), so it was lost between CI jobs. The cleanup job
couldn't destroy resources because it had no state.
Now stores state at /opt/pve-integration/work/terraform.tfstate via
-state flag on all terraform commands. This persists across the
provision → test → cleanup job chain in GitHub Actions.
Also:
- Force cleanup now removes state from both local and shared paths
- Added -reconfigure to terraform init (avoids backend mismatch errors)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Terraform creates resources in parallel. Without depends_on, PVE VMs
could boot and start the auto-installer before the HTTP answer server
container is running, causing "could not find answer file" errors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix bare Skip-IfNoTarget calls in 13_Firewall and 14_Backup
(missing if/return pattern caused tests to run when they should skip)
- Validate modifier-only switches in dev.ps1 (-Force/-Reprovision
without an action switch now errors instead of defaulting to -Shell)
- Add force-cleanup to usage text in run-integration.sh
- Add --connect-timeout/--max-time to guest agent curl in wait-for-pve.sh
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
goodolclint-claude[bot]