ci: harden test selection and nightly coverage (#6341)

This commit is contained in:
Zhengchao An
2026-08-21 23:04:08 +08:00
committed by GitHub
parent bce5922aef
commit bc07cfd115
31 changed files with 1146 additions and 259 deletions
+11 -5
View File
@@ -253,6 +253,9 @@ Test results are saved in the `artifacts/s3tests-${TEST_MODE}/` directory (defau
- `junit.xml`: Test results in JUnit format (compatible with CI/CD systems)
- `pytest.log`: Detailed pytest logs with full test output
- `all-collected-nodeids.txt`: Exact node IDs in the pinned upstream suite
- `selected-nodeids.txt`: Exact node IDs expected in this run
- `unsharded-selected-nodeids.txt`: Exact node IDs before deterministic sharding
- `compat-report.md`: Classification report generated by `report_compat.py`
regressions against `implemented_tests.txt`, promotion candidates (tests
that pass but are still listed as unimplemented/excluded), and tests missing
@@ -449,9 +452,11 @@ RustFS. Two GitHub Actions workflows delegate to it:
- **Full sweep** (`.github/workflows/e2e-s3tests.yml`): weekly scheduled (and
manually dispatchable) run of the ENTIRE upstream suite (`TEST_SCOPE=all`)
against a Docker deployment — single node or a real 4-node distributed
cluster behind HAProxy. The sweep fails only on regressions in the
implemented whitelist; everything else is reported by `report_compat.py`
as promotion candidates or unclassified tests.
cluster behind HAProxy. Regressions, unclassified tests, incomplete
execution, and infrastructure errors fail the sweep; classified unsupported
behavior remains informational. Scheduled topology runs are split into four
deterministic exact-node-ID shards, and every case has a five-minute timeout,
so one stalled case cannot erase the entire sweep's evidence.
Keeping both workflows on this script means local runs, the PR gate, and the
scheduled sweep always execute tests the same way (same pinned s3-tests
@@ -466,8 +471,9 @@ pass/fail table in the job summary.
## Companion Tools
- `report_compat.py` — diffs a junit.xml result against the classification
lists; run automatically at the end of `run.sh`, and used by the weekly
sweep to gate on whitelist regressions only (`--fail-on-regression`).
lists and the exact pytest collection; run before execution to reject stale
or missing classifications, then after execution to detect regressions and
incomplete parameterized cases.
- `api_coverage.py` — quantifies S3 API surface coverage by comparing the
s3s `S3` trait (at the revision pinned in Cargo.toml) against the methods
RustFS overrides in `impl S3 for FS`: