Files
rustfs/docs/testing/README.md
T
Zhengchao An 846aa95c32 test(security): GHSA-named regression tests for 3p3x and r5qv (backlog#1151 sec-6) (#4707)
test(security): add GHSA-named regression tests for 3p3x and r5qv (backlog#1151 sec-6)

Anchor the two fixed advisories to discoverable, named regression tests so
`rg -i "ghsa|3p3x|r5qv"` finds a guard for each, and future fixes are forced
to update pinned behavior (red -> green).

GHSA-3p3x-734c-h5vx (constant-time WebDAV/FTPS secret comparison, rustfs#4403):
- ftps_core.rs: new `assert_ftps_ghsa_3p3x_wrong_credentials_rejected` drives
  the `ct_eq` reject branch in FtpsAuthenticator::authenticate; asserts wrong
  password and unknown user are both rejected (530) and indistinguishable.
- webdav_core.rs: the auth-failure block now sends a valid access key with a
  wrong secret (exercising the `ct_eq` branch, not just the unknown-access-key
  path) plus an unknown user, asserting both 401 and indistinguishable.
- Module doc comments map advisory -> tests -> fix PR on both files.

GHSA-r5qv-rc46-hv8q (internode RPC fail-closed, rustfs#4402):
- http_auth.rs: renamed the default-fallback rejection test to
  `ghsa_r5qv_resolve_shared_secret_rejects_default_fallback` (and broadened it
  to cover default env secret + blank secrets), and added
  `ghsa_r5qv_verify_rpc_signature_fails_closed_on_missing_or_invalid_auth`
  pinning the exact advisory scenario (missing/forged/cross-URL signature is
  rejected; a correctly signed request still passes). File-level doc maps the
  advisory.

Docs: new docs/testing/security-regressions.md with the advisory -> test
mapping table and where each layer runs; linked from docs/testing/README.md.
sec-14 will formalize the written policy in AGENTS.md.

The unit-level ghsa_r5qv_* tests run in the default CI pass. The WebDAV/FTPS
e2e live in the protocols suite (fixed ports, --test-threads=1); they cannot
join the e2e-smoke profile and are wired into CI by sec-5.

Refs: rustfs/backlog#1151 (sec-6), rustfs/backlog#1155
2026-07-11 05:18:27 +00:00

2.8 KiB
Raw Blame History

RustFS Testing

Owner: backlog#1153 (infra-11). This file is the authoritative home for test taxonomy, naming conventions, and serial/quarantine rules. It is currently a skeleton — infra-11 fills in the remaining sections. The event × budget × required matrix lives in docs/testing/ci-gates.md (ci-15); this file links to it rather than duplicating counts.

Test taxonomy

TODO (infra-11): unit / e2e-smoke / e2e-full / e2e-nightly / protocols / s3-tests / fuzz / perf, with naming conventions.

Security advisory regression tests

Fixed GHSA advisories map to named, discoverable regression tests. The advisory -> test map lives in docs/testing/security-regressions.md. sec-14 (backlog#1151) formalizes the written admission policy in AGENTS.md.

Serial groups & CI profiles

TODO (infra-11): document the [test-groups] mechanism and the default vs ci nextest profiles. See .config/nextest.toml.

Flake policy

A flaky test is one that fails non-deterministically without a corresponding code change. Flakes erode trust in the gate and block tightening required checks, so they are handled on a strict, time-boxed loop.

Retry semantics (source of truth: .config/nextest.toml):

  • The local default profile never retries. A red test on your machine is a real failure to investigate, not noise to paper over.
  • The CI ci profile runs with global retries = 0. A new race must fail on its first occurrence so the first crime scene is never masked.
  • Only tests on the quarantine list get retries = 2, and only under the ci profile. Each quarantine entry MUST link exactly one OPEN issue.
  • JUnit flaky markers are the observable. A quarantined test that passes only after a retry is marked flaky in target/nextest/ci/junit.xml (uploaded as a CI artifact). That marker — not a green check — is how we see a flake is still live.

Lifecycle of a flake:

  1. Discover — a test fails non-deterministically (CI or local), or shows a flaky marker in the JUnit report.
  2. Open an issue within 24h — file/track an issue describing the flake (symptom, suspected cause, affected suite). No silent re-runs.
  3. Quarantine — add the test to the quarantine override block in .config/nextest.toml with a comment linking that OPEN issue. This grants retries = 2 under CI so the flake stops reddening unrelated PRs, while the flaky marker keeps it visible.
  4. Fix or delete within 30 days — make the test robust (then remove the quarantine entry) or delete the test. A quarantine entry may not outlive its fix window; an entry without a live OPEN issue link is a policy violation.

First quarantine members: the two backlog#937 ecstore groups (concurrent_resend_same_part_commits_one_generation and store::bucket::tests::bucket_delete_*).