* ci(nightly): persist the nightly deb on Cloudflare R2
Upload the deb to artifacts/rustfs/packages/nightly/ (dated name plus a
rustfs-nightly-latest.deb alias) through the same R2 channel package.yml
uses, so the nightly package can be downloaded later with a stable URL.
The step is skipped when the R2 secrets are not configured, keeping the
artifact-only mode intact.
* test: add pool expansion / decommission E2E script and workflow
Add the admin-API based pool expansion, rebalance and decommission test
script (scripts/test/rustfs_pool_expand.sh) plus a workflow_dispatch /
nightly workflow that runs it on a self-hosted runner against real nodes.
The workflow accepts a release tag or a direct .deb URL (e.g. nightly/R2
package) via the package_url input.
* ci(pool-test): run the pool expansion test on the smoke-testing runner
The nightly GNU build now also packages the release binary as
rustfs-nightly-<YYYY-MM-DD>.deb (Asia/Shanghai date, matching the schedule
timezone) and uploads it as a workflow artifact. Packaging mirrors
package.yml: DEBIAN control/conffiles and the systemd service from
deploy/build/, built with fakeroot dpkg-deb.
No workflow ever set RUSTFS_KMS_VAULT_TOKEN, so live_vault_backends() returned an empty set in every CI run and behavior_rotation.rs never asserted the working half of rotate/versioning; the #[ignore] live-Vault tests had never executed in CI either. nightly-gnu.yml gains a kms-vault-lane job (vault server -dev with KV2 + Transit, full rustfs-kms suite with the lane on, the dev-Vault ignored tests, and the AppRole live script) plus a separate kms-vault-ha-failover job for the three-node Raft failover script, isolated so an election-timing flake cannot mask the main lane's verdict. GitHub-hosted ubuntu-latest rather than the self-hosted fleet: the HA script needs Docker, and e2e-s3tests.yml's banner records how the heterogeneous sm-standard pods burned the last docker-dependent workflow.
The behavior harness now records every key TestKms::create_key mints and deletes them after each Vault-backed for_each_backend case, on a fresh manager over the same configuration with the immediate-deletion gate enabled for cleanup only. Transit needs the deletion issued twice (first call parks the key in PendingDeletion, the second destroys it); KV2 destroys on the first call. Verified against a real dev Vault: after a full suite run the server holds zero behavior-* keys.
Also fixes test_vault_cancel_key_deletion_persists_state, which was broken by construction — Default::default() never picks up the insecure-dev-defaults env override, so the HTTP dev Vault the test requires was always refused. It now declares development mode on the config, and passes.
Refs rustfs/backlog#1774, rustfs/backlog#1562.
hector