- VmService.ExecuteGuestCommand: guard against null elements in -Args.
The old JSON-serialization tolerated nulls (as "null"); the repeated-key
path would NRE in EncodeFormValue. Throw a clear ArgumentException instead.
- findings.json: refresh the stale counters block (untouched since F085) to
the actual ledger state — next_id 92, resolved 83 — and bump last_updated
to 2026-05-22. last_scan_date stays 2026-03-26 (F086–F091 came from issue
triage, not a formal review scan).
- VmServiceTests: add empty-array (single command entry) and null-element
(throws) cases.
Note: F091 is the correct next ID — F086–F090 already exist from prior
merged PRs (#60/#61/#66/#67); only the counters were lagging.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ExecuteGuestCommand JSON-serialized the args array into the agent/exec
'input-data' field — which is the process's STDIN, not its arguments. So
guest commands ran with no argv: cmd.exe started interactively and the
JSON blob ['/c','echo',...] arrived at its prompt.
PVE's agent/exec 'command' parameter is itself an array (element 0 = the
executable, the rest = argv) sent as repeated form keys. The low-level
client couldn't express repeated keys (Dictionary<string,string> only),
so:
- Add PostAsync(string, IEnumerable<KeyValuePair<string,string>>) to
IPveHttpClient/PveHttpClient; BuildFormContent now emits one key=value
field per pair, so a key may repeat.
- ExecuteGuestCommand builds command = [exe] + args as repeated 'command'
fields and no longer touches input-data.
Tests: form-encoder repeated-key + per-value encoding cases; VmService
tests asserting the command array, order, and absence of input-data; an
integration regression guard that echoes an arg and checks it round-trips
as stdout.
Tracked as F091. Closes#68.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- HasDiskOptions(): include -ScsiHardware so the "disk options ignored"
warning fires when -ScsiHardware is passed without -DiskStorage/-DiskSize.
- PveVmConfig.AdditionalProperties: lazy-init a backing field so the native
dictionary is built once rather than reallocated on every property access
(matters when iterating many configs in a pipeline). Safe because the model
is effectively immutable after deserialization.
- New-PveVm.Tests.ps1: add a case asserting -DiskIoThread on scsi with a
wrong -ScsiHardware (virtio-scsi-pci) is rejected, covering the validator's
"!= virtio-scsi-single" branch (not just the null case).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New-PveVm (F089):
- Add -DiskBus (virtio/scsi/sata/ide, default virtio), -ScsiHardware (scsihw),
-DiskIoThread, -DiskAio, -DiskSsd, -DiskDiscard, -DiskCache so a tuned disk
(e.g. virtio-scsi-single + scsi0,iothread=1,aio=native,ssd=1,discard=on) can
be created in one call instead of diskless + a hand-built Set-PveVmConfig string.
- Disk spec built via BuildDiskSpec; ValidateDiskOptions runs before ShouldProcess
and rejects ssd on virtio and iothread on sata/ide or scsi-without-virtio-scsi-single
with clear errors, instead of letting PVE fail at VM start.
Get-PveVmConfig (F090):
- PveVmConfig was a fixed allow-list, silently dropping keys like scsihw, efidisk0,
tpmstate0, hostpci0. Add typed scsihw/efidisk0/tpmstate0 plus a [JsonExtensionData]
catch-all exposed as AdditionalProperties (native types via JsonHelper.ToNative,
per D013 — no JToken leakage). Makes the disk tuning above verifiable by reading
the config back.
Closes#65.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PveHttpClient.EncodeFormValue encoded &, =, +, space, and % but left ';'
literal. PVE's application/x-www-form-urlencoded parser treats a raw ';'
as a field separator (the historical alternative to '&'), so a value like
boot=order=scsi0;ide2
was split into 'boot=order=scsi0' plus an empty 'ide2' field, and PVE
rejected the PUT with "ide2: unable to parse drive options". This broke
any multi-device boot order set via Set-PveVmConfig -AdditionalConfig,
and any other value containing ';'.
Encode ';' as %3B. Safe under the existing minimal-encoding policy that
keeps ':' and '!' literal for cluster-join: cluster-join payloads never
contain ';', and PVE url-decodes config form values.
Tracked as F088. Closes#64.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When HttpClient.Timeout elapses, .NET throws TaskCanceledException — not
HttpRequestException — so the existing catch in PveHttpClient.SendAsync
missed it and callers got a raw stack trace. With -TimeoutSeconds now
configurable and documented, this gap became user-visible.
In .NET 5+ HttpClient surfaces transport timeouts as TaskCanceledException
with a TimeoutException inner; user-driven token cancellation does not.
Catch by that inner-type signature and rethrow as PveApiException with
HttpStatusCode.RequestTimeout, the resource path, and a message that
reports the configured timeout.
Adds SendAsync_TimeoutFires_ThrowsPveApiExceptionWithRequestTimeout which
swaps in a delaying HttpMessageHandler with a 50ms timeout to exercise
the path deterministically. Drops the redundant
DefaultSessionTimeoutIs100Seconds test (covered by
PveSessionTests.Timeout_DefaultIs100Seconds and the existing flow-through
test).
Addresses PR #61 review feedback.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolves findings.json conflict — F086 (from #60, merged into main) and
F087 (this branch) both append to the trailing findings array. Kept both
entries, in numeric order.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PveHttpClient was constructed without setting HttpClient.Timeout, so
.NET's 100s default applied to every request. Multi-GB ISO uploads via
Send-PveFile on a real LAN reliably tripped this with TaskCanceledException
after 100 seconds, and there was no way to override it.
- PveSession gains a Timeout (TimeSpan) property, defaulting to 100s.
- PveHttpClient accepts an optional per-instance timeout override that
takes precedence over the session timeout.
- Connect-PveServer exposes -TimeoutSeconds to set the session default.
- Send-PveFile and Invoke-PveStorageDownload expose -TimeoutSeconds with
a 30-minute implicit default so large uploads/downloads do not trip
the 100s default. -TimeoutSeconds 0 means Timeout.InfiniteTimeSpan.
Tracked as F087. Closes#59.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- SizeParser: wrap TB-suffix overflow in try/catch so callers get
ArgumentException with the parameter name rather than OverflowException.
- New-PveVm/New-PveContainer: validate -DiskSize/-RootFsSize before
ShouldProcess so typos like "512M" are rejected even with -WhatIf and
even when the matching -DiskStorage/-RootFsStorage is omitted.
- Add Pester tests for the new DiskSize and RootFsSize validation paths,
including a new New-PveContainer.Tests.ps1.
- Add SizeParserTests coverage for the TB overflow path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Disk and rootfs size strings were interpolated directly into the disk
spec as "<storage>:<size>", so "60G" produced "local-lvm:60G". On
LVM/LVM-thin storages PVE parses the value after the colon as a volume
name unless it is a bare integer, returning "unable to parse lvm volume
name '60G'". File-backed storages mask this by accepting either form.
SizeParser.NormalizeToGibibytes() now strips G/GB/T/TB suffixes and
returns a bare GiB integer string, so the documented "32G" call shape
works on every storage type. Sub-GB units are rejected with a clear
error rather than being silently truncated.
Tracked as F086. Closes#58.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace single-version fixture with per-version specs from pve-api.
Tests now validate enum values against PVE 7 (best-effort), 8, and 9.
Key findings from version-specific specs:
- VM.Monitor: valid in PVE 7+8, removed in PVE 9
- VM.Replicate: added in PVE 9 only
- VM.GuestAgent.*: added in PVE 9 only
- Mapping.*: added in PVE 8+
- glusterfs: not in any version (removed before PVE 7)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 70 xUnit tests that validate every ValidateSet in the module
against the PVE OpenAPI spec. Three bugs found and fixed:
- Storage: remove `glusterfs` (dropped in PVE 9), add `btrfs`, `esxi`
- Backup compression: `none` → `0` (PVE uses "0" not "none")
- Cluster resources: remove `lxc` filter (PVE uses `vm` for both)
The pve-api-enums.json fixture (199KB) is extracted from the full
OpenAPI spec and contains parameter enum values for 302 API paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The ValidateSet attribute was intentionally removed from the Type
parameter during merge review to allow future rule types without
code changes. Updated test to only assert Type is mandatory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PVE 9 API requires 'type' on PUT /cluster/ha/rules/{rule} even for
updates. Added mandatory Type parameter with ValidateSet for
node-affinity and resource-affinity.
Updated integration test and Pester unit tests to pass -Type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Rename ZZ.ClusterConfig.Integration.Tests.ps1 → 16_Cluster.Tests.ps1
- Replace manual env var setup with _IntegrationHelper.ps1 + Connect-TestPve
- Use credential auth (root@pam) throughout — no API tokens
- Use $script:PasswordB for node B auth (from helper)
- Access JoinInfo.Nodelist as List<Dictionary> (native types from D013)
- Keep cluster-specific skip helpers (Skip-IfNoCluster, Skip-IfPve9HaGroups)
- Remove redundant Connection context (Connect-TestPve handles it)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PVE 9 rejects rules referencing unmanaged resources. Test now creates
a minimal VM, registers it as a disabled HA resource, then creates
the node-affinity rule. Cleanup removes rule, resource, and VM.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PVE 9 rejects 'affinity' as an unexpected property on POST
cluster/ha/rules — it's implied by the node-affinity type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full lifecycle: create node-affinity rule, list, get by ID, update
comment, delete, and verify deletion. Uses vm:99999 as a synthetic
resource SID. Tests skip on PVE 8 (rules not available).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cluster tests create a 2-node cluster that cannot be torn down via API
(quorum loss). Prefix with ZZ so they execute after all other
integration tests that assume standalone nodes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removing a node from a 2-node PVE cluster via REST API is not supported
— the remaining node loses quorum mid-operation, causing "no quorum!"
errors. PVE requires stopping corosync on the departing node first
(pvecm expected 1), which is not available via the REST API.
Replace removal tests with a final cluster state verification.
Test infrastructure handles cleanup via reprovisioning.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove-PveClusterConfigNode requires root@pam (not API token).
Fixed both the test context and AfterAll cleanup to connect with
root@pam credentials before attempting node removal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Extract re-auth retry into ReauthenticateWithRetry helper (fixes
cognitive complexity warning)
- Retry auth up to 10x with 3s delay — node B's auth services need
time to restart after joining the cluster
- Wrap join in try/finally so the test always reconnects to node A,
even if the join fails
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All cluster operations (create, status, join, options, cleanup) require
root@pam ticket auth — API tokens lack Sys.Audit and Sys.Modify on /.
Changed test to connect as root@pam once during cluster creation and
stay on that session for the entire lifecycle.
Removed redundant Connect-PveServer calls that switched between
root@pam and API token between contexts.
Added 5s stabilization sleep after cluster creation for corosync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Convert JoinInfo.Nodelist to native PS array via ConvertFrom-Json
(JArray .Item() still hits IEnumerator error in PowerShell)
- Use root@pam for cluster options context (delete requires Sys.Modify)
- Reconnect with API token after options restore
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
xUnit (47 new tests, 429 total):
- ClusterConfigServiceTests: 25 tests covering all 14 service methods
including URL encoding, null guards, and error responses
- HaServiceTests: 22 tests covering resources, groups, status, and rules
with URI encoding verification (vm:100 → vm%3A100)
Pester (187 new tests, 1525 total):
- ClusterConfigCmdlets.Tests.ps1: 11 cmdlets tested
- HaCmdlets.Tests.ps1: 14 cmdlets tested
Integration (ClusterConfig.Integration.Tests.ps1):
- Full 2-node cluster lifecycle with -Wait for task completion
- Uses root@pam ticket auth for cluster create/join operations
- HA group tests skip on PVE 9.0+ (groups migrated to rules)
- JArray indexing uses .Item() for PowerShell compatibility
Cmdlet improvements:
- New-PveCluster, Add-PveClusterConfigNode, Add-PveClusterMember now
support -Wait switch to block until task completes (via TaskService)
- GetClusterConfig returns JToken to handle array responses on standalone
- OutputType updated to PveTask for task-returning cmdlets
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RUNNER_TEMP (/__w/_temp in CI) is container-local and invisible to
the Docker host. Files written there can't be bind-mounted into
sibling containers (answer server, storage). This was the root cause
of the "not a directory" mount failures in CI.
Removed RUNNER_TEMP from the WORK_DIR fallback chain. WORK_DIR now
always defaults to CACHE_DIR/work (/opt/pve-integration/work/) which
is on the shared mount visible to both the CI container and Docker host.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mounting the whole directory to /app overwrites server.py inside the
container, causing "can't open file '/app/server.py'" errors.
Mount the two paths individually instead:
- answer_server_dir/answers → /app/answers
- answer_server_dir/default.toml → /app/default.toml
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Docker bind mount fails when mounting a file that doesn't exist on
the host (creates a directory instead). In CI, the container's
$WORK_DIR path differs from the host path, so the file mount failed.
Changed to mount a single answer-server/ directory containing both
default.toml and answers/ subdirectory. Replaced two TF variables
(answer_files_dir, default_answer_file) with one (answer_server_dir).
Layout: $WORK_DIR/answer-server/default.toml + answers/<mac>.toml
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Terraform state was stored in the container's working directory
(fresh checkout), so it was lost between CI jobs. The cleanup job
couldn't destroy resources because it had no state.
Now stores state at /opt/pve-integration/work/terraform.tfstate via
-state flag on all terraform commands. This persists across the
provision → test → cleanup job chain in GitHub Actions.
Also:
- Force cleanup now removes state from both local and shared paths
- Added -reconfigure to terraform init (avoids backend mismatch errors)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Terraform creates resources in parallel. Without depends_on, PVE VMs
could boot and start the auto-installer before the HTTP answer server
container is running, causing "could not find answer file" errors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix bare Skip-IfNoTarget calls in 13_Firewall and 14_Backup
(missing if/return pattern caused tests to run when they should skip)
- Validate modifier-only switches in dev.ps1 (-Force/-Reprovision
without an action switch now errors instead of defaulting to -Shell)
- Add force-cleanup to usage text in run-integration.sh
- Add --connect-timeout/--max-time to guest agent curl in wait-for-pve.sh
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Update header comments: PVETEST_APITOKEN → PVETEST_PASSWORD,
CACHE_DIR default → /opt/pve-integration
- Pin ubuntu and NFS server Docker images to SHA256 digests
- Fix cmd_all to pass version to provision and cleanup
- Keep .terraform.lock.hcl in force cleanup (provider reproducibility)
- Remove || true from terraform destroy in cleanup (propagate errors;
use -Force for best-effort recovery)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add ValidateSet('8','9','all') to dev.ps1 -Version parameter
- Fix Shell warning to reference $DevContainer not $InfraContainer
- Fix Skip-IfNoNodeB to check $PasswordB not $Password
- Pass PVE_TARGET_NODE to wait-for-pve.sh instead of auto-discovering
- Add error default cases to all pve_* helper functions
- Lowercase MAC addresses for answer server matching
- Create answer file paths before terraform destroy in cleanup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot review fixes:
- Pin answer server Docker image to SHA256 digest instead of :latest
for reproducible builds
- Fix cmd_taint: ISO resources are keyed by version ("9") not node
("9a"), so taint was no-op. Now taints ISOs by version and VMs
by node separately.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Test was still expecting GetAsync on cloudinit/dump but the service
now calls PutAsync on cloudinit (regenerate endpoint). Updated mock
setup and assertions to match.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Ensures nested PVE nodes are fully patched before integration tests
run. Adds ~5-10min to first provision but gives more realistic test
results against current PVE releases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WORK_DIR now defaults to /opt/pve-integration/work (under the shared
host mount) instead of /tmp/pve-integration (container-local).
This fixes the answer server seeing empty answer files — the dev-infra
container writes answer files to WORK_DIR, and the answer server
container (a sibling) needs to read them from the same host path.
Also:
- Renamed mount from /opt/pve-isos to /opt/pve-integration
- Added must_run=true, start=true to answer server container
- Updated docker-compose.test.yml and dev.ps1 remote override
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Changed Terraform ISO resource from for_each=pve_instances (per-node)
to for_each=pve_isos (per-version). With HTTP answer server, both
nodes of the same version share the same generic ISO.
New pve_isos variable maps version to ISO path. pve_instances now
has pve_version field instead of iso_local_path. VMs reference
their version's ISO via auto_iso[each.value.pve_version].
Updated run-integration.sh tfvars generation and -target flags
for both provision and cleanup to use the new schema.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Force cleanup wipes all Terraform state, so filtering by version
would leave an inconsistent state where subsequent provisions fail.
Error early with a clear message instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Force cleanup must remove pvetest-* Docker containers unconditionally,
not just when cleaning all versions. Stale containers cause Terraform
to fail on next provision ("container already exists") since the state
was also wiped.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace 4 per-node auto-install ISOs (~4GB) with 2 generic ISOs
(~2GB) served by an HTTP answer server that routes per-node configs
by MAC address.
New flow:
1. Deterministic MAC addresses assigned per node (AA:BB:CC:00:VV:NN)
2. Per-MAC answer.toml files generated in answers/ directory
3. HTTP answer server (slothcroissant/proxmox-auto-installer-server)
managed by Terraform, serves answer files on port 8000
4. Generic ISOs prepared with --fetch-from http --url pointing to
the answer server
5. PVE installer POSTs system info, server matches MAC to answer file
Benefits:
- 50% reduction in ISO disk/tmp usage (2 ISOs instead of 4)
- Faster ISO preparation (2 builds instead of 4)
- Answer files can be updated without rebuilding ISOs
- First-boot script embedded in generic ISO via --on-first-boot
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When Terraform state is corrupted (e.g. interrupted provision),
-Cleanup -Force bypasses Terraform and:
1. Destroys VMs via direct PVE API calls (preflight-cleanup.sh)
2. Force-removes Docker storage containers and volumes
3. Deletes Terraform state files so next provision starts clean
Usage: dev.ps1 -Cleanup -Force -DockerHost 172.16.40.113
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When provisioning a subset of versions (-Version 9), the -target flags
only included PVE VM resources. Docker storage containers (iSCSI, NFS)
were skipped because they weren't targeted. Now always includes all
Docker resources in the target list since storage is shared across
all PVE versions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Major provisioning pipeline changes:
Terraform:
- Add kreuzwerker/docker provider to manage iSCSI and NFS storage
containers alongside PVE VMs in a single Terraform config
- New storage.tf with Docker container, image, and volume resources
- docker_host_ip variable for PVE nodes to reach storage services
Provisioning:
- Replace docker-compose storage management with Terraform
- Replace create-api-token.sh with wait-for-pve.sh (IP discovery +
API readiness + auth verification only — no token creation)
- Tests use root@pam credentials, not API tokens
Cleanup:
- Replace preflight-cleanup.sh loop with terraform destroy
- Supports version filtering: cleanup 9 destroys only PVE 9 resources
- Full cleanup also removes config.json and tfvars
New commands:
- taint [8|9|all]: marks VMs for recreation on next provision
- dev.ps1 -Reprovision: runs taint before provision to force VM rebuild
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Clint Branham