test(ci): strict nextest ci profile with quarantine + flake policy (#4666)

test(ci): add strict nextest ci profile with quarantine + flake policy

Formalize the existing ecstore-serial-flaky mechanism into a strict CI gate
(ci-10, absorbs infra-15; backlog#1149).

- .config/nextest.toml: add [profile.ci] with global retries=0 (never mask a
  new race's first occurrence), fail-fast=false, and JUnit output at
  target/nextest/ci/junit.xml. Add a quarantine section where flaky tests get
  retries=2 under the ci profile only; each entry links one OPEN issue. First
  members are the two backlog#937 ecstore groups
  (concurrent_resend_same_part_commits_one_generation and
  store::bucket::tests::bucket_delete_*), which keep their existing
  ecstore-serial-flaky test-group serialization. Local default profile still
  never retries.
- .github/workflows/ci.yml: run the main test step with --profile ci and upload
  the JUnit report (if: always(), 3-day retention, run-number in name). The
  migration-proof step stays on the default profile to avoid clobbering the ci
  JUnit artifact (its tests are not quarantined).
- docs/testing/README.md: new skeleton (owned by backlog#1153 infra-11) holding
  the flake policy: discover -> open issue within 24h -> quarantine with issue
  link -> fix or delete within 30 days. AGENTS.md points to it.

Refs: rustfs/backlog#1149, rustfs/backlog#937, rustfs/backlog#1155
This commit is contained in:
Zhengchao An
2026-07-10 20:08:31 +08:00
committed by GitHub
parent f0b1202b65
commit 89ea931ee1
5 changed files with 128 additions and 1 deletions
+13 -1
View File
@@ -155,12 +155,24 @@ jobs:
- name: Run tests
run: |
cargo nextest run --all --exclude e2e_test
cargo nextest run --profile ci --all --exclude e2e_test
cargo test --all --doc
- name: Upload test junit report
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: junit-test-and-lint-${{ github.run_number }}
path: target/nextest/ci/junit.xml
retention-days: 3
if-no-files-found: ignore
# Explicit gate for migration-critical suites. These tests already ran in
# the full nextest pass above; a single filtered nextest invocation keeps
# the named gate without rebuilding or re-running them one package at a time.
# Kept on the default profile (no --profile ci): a second --profile ci run
# would clobber target/nextest/ci/junit.xml, and none of these tests are
# quarantined so they gain nothing from the ci profile's retry overrides.
- name: Run rebalance/decommission migration proofs
run: |
cargo nextest run -p rustfs-ecstore --lib \