ci: fit the Rust caches back inside the 10GB quota (#5532)

The repository has 18 rust-cache families of 1.2-3.1GB each against GitHub's
fixed 10GB per-repo quota. Measured usage sat at 9.72GB, 9.96GB and 11.63GB on
three samples, so LRU eviction is continuous and the main lanes lose: ci-test
and ci-e2e were repeatedly absent from the surviving entries. That is what
makes "Setup Rust environment" bimodal — 0.7-3.4 minutes warm against
11.8-20.9 minutes cold.

Four changes, all cache-only. No job builds or tests anything different.

Collapse ci.yml's nine cache sites into four keys, each with exactly one
writer: ci-dev (test-and-lint writes; ILM, debug-binary, e2e-tests and e2e-full
read), ci-feat-rio (rio-v2 lint writes, its debug-binary reads), ci-feat-proto
(the swift leg writes for both protocol legs), and ci-uring, which stays alone.
e2e-full is easy to miss here: it already shared ci-e2e with e2e-tests and both
saved, so without an explicit 'false' it would have become a second unnamed
writer of ci-dev. rio/swift/sftp are deliberately NOT merged — measured at
2365/2307/1200MB they are not near-identical, and none is a superset of the
others.

Give each writer a superset warm-up on main. A writer's own steps are not
automatically a superset of its readers': clippy emits metadata only, the
nextest pass excludes e2e_test, and no lint lane enables e2e-test-hooks, whose
different feature resolution yields a different -Cmetadata. Without these
builds the readers would restore a cache missing precisely what they need.
Guarded to main, so the PR critical path is unaffected.

Stop writing tag-scoped caches in build.yml. A cache saved on refs/tags/X can
only be restored by a re-run of that same tag, so each release cycle wrote up
to 12 unreadable 1-2GB entries that evicted the hot lanes. Tag builds still
restore the main-scoped cache. The one real cost is that re-running a failed
leg of the same tag now falls back to main's cache.

Flip the composite action's cache-save-if default to 'false' and make the input
mandatory in practice. audit.yml was relying on the old "true" default: every
PR touching Cargo.toml or Cargo.lock saved a second, PR-scoped copy (~843MB
measured, job 91048468127) that pushed main-scoped lanes out of the quota. The
fail-safe direction is a cold cache, not a stolen quota slice.
scripts/security/check_cache_save_if.sh now asserts every call site states it,
wired into audit.yml next to the existing pin check.

While there, cargo-deny stops pulling the full setup composite. It compiles
nothing, so apt, protoc, flatc and nextest were pure overhead — but it does run
cargo metadata, and Cargo.toml pins datafusion and s3s as git dependencies that
must be materialised into ~/.cargo/git, so the cache itself stays.

Refs: rustfs/backlog#1598, rustfs/backlog#1600
This commit is contained in:
Zhengchao An
2026-08-01 11:06:31 +08:00
committed by GitHub
parent 790bdc0e63
commit 7354a5663d
5 changed files with 153 additions and 20 deletions
+23 -3
View File
@@ -75,10 +75,27 @@ jobs:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- name: Setup Rust environment
uses: ./.github/actions/setup
# cargo-deny compiles nothing, so the full setup composite (apt packages,
# protoc, flatc, nextest, rustfmt/clippy) was pure overhead here. It does
# still need a real cargo: `cargo deny check` runs `cargo metadata`, and
# Cargo.toml pins datafusion and s3s as git dependencies, which must be
# materialised into ~/.cargo/git — a cold clone is hundreds of MB, so the
# cache stays.
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
# Was relying on the composite's default, which used to be "true": every
# PR touching Cargo.toml/Cargo.lock saved a second, PR-scoped copy of this
# cache and pushed the main-scoped lanes out of the 10GB quota. The
# default is now "false", but state it explicitly — see
# scripts/security/check_cache_save_if.sh.
- name: Setup Rust cache
uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
with:
cache-shared-key: rustfs-cargo-deny
cache-all-crates: true
cache-on-failure: true
shared-key: rustfs-cargo-deny
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Install cargo-deny
uses: taiki-e/install-action@bffeee26d4db9be238a4ea78d8826604ebcb594d # v2
@@ -100,6 +117,9 @@ jobs:
- name: Report unpinned GitHub Actions
run: ./scripts/security/check_workflow_pins.sh --enforce
- name: Check setup cache-save-if is explicit
run: ./scripts/security/check_cache_save_if.sh
- name: Check preview release workflow policy
run: ./scripts/security/check_preview_release_workflow.sh