ci(coverage): add weekly cargo-llvm-cov baseline workflow (#4820)

ci(coverage): weekly cargo-llvm-cov workspace baseline (non-blocking)

Add the weekly line-coverage report (backlog#1153 infra-5):

- .github/workflows/coverage.yml — Sunday 07:00 UTC + workflow_dispatch;
  runs `cargo llvm-cov nextest --workspace --exclude e2e_test` under
  NEXTEST_PROFILE=ci (same scope and profile as the ci.yml test gate),
  writes a per-crate line-coverage table to the job summary, uploads
  lcov + JSON as a 90-day artifact, and routes scheduled failures
  through the shared schedule-failure-issue action (ci-8). Runs only on
  schedule/dispatch, so it can never become a required PR check.
- scripts/coverage_per_crate.py — stdlib-only aggregation of the
  llvm-cov JSON export into the per-crate markdown table (worst-first,
  TOTAL row), shared by the workflow and the local target.
- make coverage (.config/make/coverage.mak) — local equivalent with the
  same command sequence; fails with install hints when cargo-llvm-cov
  or cargo-nextest are missing. Listed in make help.
- docs/testing/README.md — Coverage section: cadence, where the table
  and artifacts live, the trend-comparison method, and what is not
  measured (doctests, e2e_test).
This commit is contained in:
Zhengchao An
2026-07-15 11:10:22 +08:00
committed by GitHub
parent 04616e32c8
commit 5fd2e8b6a1
4 changed files with 269 additions and 0 deletions
+27
View File
@@ -152,6 +152,33 @@ and `advance`) or explicit event synchronization over fixed `sleep` race
windows. The written convention and the `docs/testing/time-control.md` guide are
added by backlog#1153 infra-4.
## Coverage
Line coverage is measured **weekly, not per-PR**, and is non-blocking: it
exists for visibility and trend, never as a required check. Per-crate ratchets
for the security-critical crates (iam / kms / policy / crypto) build on this
baseline later (backlog#1153 infra-6, report-only first).
- **CI**: `.github/workflows/coverage.yml` runs every Sunday and on manual
dispatch: `cargo llvm-cov nextest --workspace --exclude e2e_test` under the
`ci` nextest profile — the same scope and profile as the PR test gate. The
per-crate line-coverage table lands in the run's job summary; the lcov +
JSON exports are uploaded as a `coverage-lcov-<run>` artifact kept for
90 days. Scheduled failures open/append the `[scheduled-failure] coverage`
issue via the shared alert action (ci-8).
- **Local**: `make coverage` is the equivalent (slow — instrumented rebuild
plus the full suite). It prints the same per-crate table via
`scripts/coverage_per_crate.py` and writes `target/llvm-cov/lcov.info` and
`coverage.json`.
- **Trend comparison**: each run's job summary is the weekly per-crate
snapshot — open two runs from the Actions history (workflow "coverage") and
compare their tables. For line-level diffs, download the two runs'
`coverage-lcov-*` artifacts and compare the `lcov.info` files with your lcov
tooling of choice.
- **Not measured**: doctests (ci.yml runs them uninstrumented; covering them
would require a nightly toolchain) and the `e2e_test` crate (excluded from
the unit gate; its lanes are described in the taxonomy above).
## Flake policy
A flaky test is one that fails non-deterministically without a corresponding