Zhengchao An
846aa95c32
test(security): add GHSA-named regression tests for 3p3x and r5qv (backlog#1151 sec-6) Anchor the two fixed advisories to discoverable, named regression tests so `rg -i "ghsa|3p3x|r5qv"` finds a guard for each, and future fixes are forced to update pinned behavior (red -> green). GHSA-3p3x-734c-h5vx (constant-time WebDAV/FTPS secret comparison, rustfs#4403): - ftps_core.rs: new `assert_ftps_ghsa_3p3x_wrong_credentials_rejected` drives the `ct_eq` reject branch in FtpsAuthenticator::authenticate; asserts wrong password and unknown user are both rejected (530) and indistinguishable. - webdav_core.rs: the auth-failure block now sends a valid access key with a wrong secret (exercising the `ct_eq` branch, not just the unknown-access-key path) plus an unknown user, asserting both 401 and indistinguishable. - Module doc comments map advisory -> tests -> fix PR on both files. GHSA-r5qv-rc46-hv8q (internode RPC fail-closed, rustfs#4402): - http_auth.rs: renamed the default-fallback rejection test to `ghsa_r5qv_resolve_shared_secret_rejects_default_fallback` (and broadened it to cover default env secret + blank secrets), and added `ghsa_r5qv_verify_rpc_signature_fails_closed_on_missing_or_invalid_auth` pinning the exact advisory scenario (missing/forged/cross-URL signature is rejected; a correctly signed request still passes). File-level doc maps the advisory. Docs: new docs/testing/security-regressions.md with the advisory -> test mapping table and where each layer runs; linked from docs/testing/README.md. sec-14 will formalize the written policy in AGENTS.md. The unit-level ghsa_r5qv_* tests run in the default CI pass. The WebDAV/FTPS e2e live in the protocols suite (fixed ports, --test-threads=1); they cannot join the e2e-smoke profile and are wired into CI by sec-5. Refs: rustfs/backlog#1151 (sec-6), rustfs/backlog#1155
Protocol E2E Tests
FTPS, WebDAV, and SFTP protocol end-to-end tests for RustFS.
Prerequisites
No external SSH tooling is required. The test framework generates ed25519 host keys in-process via russh::keys under the per-test temp directory before each SFTP server spawn, and russh-sftp drives the protocol from the test process directly.
Running Tests
RUSTFS_BUILD_FEATURES=ftps,webdav,sftp cargo test --package e2e_test test_protocol_core_suite -- --test-threads=1 --nocapture
RUSTFS_BUILD_FEATURES controls which features the test rustfs binary is
built with. When this variable is set, the protocol test runner schedules
only entries whose protocol is present in the requested feature list. Leave
it unset to run every protocol entry.
--test-threads=1 is required because every entry spawns a rustfs server
on fixed bind ports.
Test Coverage
FTPS Tests
- mkdir bucket
- cd to bucket
- put file
- ls list objects
- cd . (stay in current directory)
- cd / (return to root)
- cd nonexistent bucket (should fail)
- delete object
- cdup
- rmdir delete bucket
WebDAV Tests
- PROPFIND at root (list buckets)
- MKCOL (create bucket)
- PUT (upload file)
- GET (download file)
- PROPFIND on bucket (list objects)
- DELETE file
- MOVE file (rename object)
- DELETE bucket
- Authentication failure test
SFTP Tests
The SFTP suite lives in three entries plus a standalone idle-timeout case.
Every assertion runs against a freshly spawned rustfs binary with
RUSTFS_SFTP_ENABLE=true; the test framework also pins
RUSTFS_SFTP_PART_SIZE=5242880 so the multipart boundary is deterministic.
sftp_core (test_sftp_core_operations)
Bind ports 9022 (SFTP) and 9200 (S3). 22 in-suite assertions covering the core protocol surface plus cross-protocol consistency:
- Subsystem canary: SFTPv3 version exchange completes after password auth
- Bucket lifecycle: mkdir, root listing, rmdir, post-delete listing
- Small-file round-trip with SHA256 compare
- Stat on a file (size + file type) and on a bucket (directory)
- SETSTAT on a path returns ok
- Rename within bucket, listing reflects the rename
- Multipart-sized round-trip (just over 2 × part_size) with SHA256 compare
- Negative cases: symlink rejected, open of nonexistent file rejected, read_dir of nonexistent bucket rejected, path traversal rejected
- Spec-letter assertions: APPEND open returns an error, CREATE+EXCLUDE on an existing path returns an error, bad-password authentication is rejected
- Cross-protocol via aws-sdk-s3: SFTP write then S3 read with SHA256 match, S3 write then SFTP read with SHA256 match
- Cross-API directory visibility: SFTP-created sub-directory visible via S3
ListObjectsV2, S3-created
__XLDIR__marker visible via SFTP readdir as a directory entry
sftp_compliance (test_sftp_compliance_suite)
Bind ports 9024 (SFTP) and 9300 (S3). 14 compliance regression cases against one shared server spawn. Each case carries a stable CMPTST-NN identifier:
- CMPTST-01: medium-binary upload then download with SHA256 compare (single-shot PutObject path below the multipart boundary)
- CMPTST-02: zero-byte upload, download, and stat-size match
- CMPTST-03: rm against a bucket path is rejected; the bucket is preserved
- CMPTST-04: rmdir against a non-empty bucket is rejected; the contained object survives
- CMPTST-05: rmdir against a non-empty sub-directory is rejected; the inner object survives
- CMPTST-06: open with a path-traversal pattern cannot leak a host file via SFTP read
- CMPTST-07: read_dir of
/..either errors or returns a listing that contains no host system entries - CMPTST-08: rename across buckets preserves payload and removes the source object
- CMPTST-09: paths with embedded spaces round-trip through the russh-sftp client
- CMPTST-10: read_link is rejected (S3 storage has no symlinks)
- CMPTST-11: SETSTAT on a path and FSETSTAT on a separate open handle both return ok (rsync, WinSCP transfer-success contract)
- CMPTST-12: rename to the same path is a no-op; the file persists with the original payload
- CMPTST-13: implicit-directory round-trip; uploading to a nested key creates the parent directory implicitly and three listing forms surface the inner file
- CMPTST-14: OPEN, WRITE, FSETSTAT, CLOSE on the same write handle all return ok (WinSCP wire shape)
sftp_compliance_readonly (test_sftp_compliance_readonly)
Bind ports 9025 (SFTP) and 9301 (S3). Spawns a second rustfs binary with
RUSTFS_SFTP_READ_ONLY=true; the S3 endpoint stays writable so the suite
can seed a bucket and a fixture object via aws-sdk-s3 before opening the
SFTP session. 7 compliance cases:
- CMPTST-15: put through SFTP is rejected
- CMPTST-16: rm through SFTP is rejected
- CMPTST-17: mkdir through SFTP is rejected
- CMPTST-18: rmdir through SFTP is rejected
- CMPTST-19: rename through SFTP is rejected
- CMPTST-20: ls through SFTP is allowed and lists the seeded bucket
- CMPTST-21: get through SFTP is allowed and returns the seeded payload byte-for-byte
The full case index lives at the top of sftp_compliance.rs; each helper's
log lines name its CMPTST-NN code so a failure in CI points at one named
property without consulting any external doc.
sftp_idle_timeout (test_sftp_idle_timeout_disconnects)
Bind ports 9023 (SFTP) and 9100 (S3). Spawns rustfs with
RUSTFS_SFTP_IDLE_TIMEOUT=5, sleeps 10 s past the timeout, then issues an
SFTP request and asserts the server has closed the session.