mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-10 23:26:53 +00:00
ci: stop archiving every dependency's unpacked source in each cache (#5566)
The four consolidated ci keys plus the build keys still do not fit the repository's fixed 10GB Actions cache quota, so LRU keeps evicting them: measured demand is ci-dev 2331MB + ci-feat-proto 2310MB + ci-feat-rio 1951MB + ci-uring 1317MB + two build legs at ~1429MB + cargo-deny 844MB, and main pushes add two more build legs. That is roughly 14.4GB against 10.24GB. The symptom is misleading: Cache Warm reports every job successful and the restores log "full match: true", yet ci-feat-rio and ci-uring disappear from the cache list between runs. cache-all-crates was the wrong default for this repository. With it set to true, rust-cache's cleanup returns before pruning ~/.cargo/registry/src and its config archives the whole registry, so every cache carried the unpacked source tree of every dependency — not, as the name suggests, just a few extra crates. Setting it to false is rust-cache's own default and loses no coverage: the package set comes from `cargo metadata --all-features`, a strict superset of any single lane's feature closure; -sys crates are explicitly exempt from pruning, since their source timestamps would otherwise trigger rebuilds; and everything pruned is re-unpacked from the .crate files still in registry/cache, whose mtimes crates.io normalises, so cargo fingerprints stay valid. Applied to the setup composite and to audit.yml's own rust-cache. Cache Warm now also reports the sizes of registry/src, registry/cache, registry/index, ~/.cargo/git and target/ to the step summary, immediately before rust-cache's post step archives them, so the size of the effect is measured rather than assumed. The new sizes only appear once the cache key next rotates, since rust-cache skips the save entirely on an exact key hit. Deliberately not forcing that by bumping prefix-key: it would invalidate every family at once and produce a repository-wide cold build. Refs: rustfs/backlog#1598, rustfs/backlog#1600
This commit is contained in:
@@ -107,10 +107,6 @@ jobs:
|
||||
cache-save-if: 'true'
|
||||
install-build-packaging-tools: 'false'
|
||||
|
||||
# --all-targets covers the test binaries nextest builds, including
|
||||
# e2e_test, which test-and-lint's own run excludes. The second build adds
|
||||
# the e2e-test-hooks feature resolution that build-rustfs-debug-binary uses
|
||||
# and that no lint lane enables.
|
||||
# rustfs/backlog#1601 gate. sccache can only cache compilation units whose
|
||||
# --emit includes link, so it covers workspace rlibs and nothing else:
|
||||
# clippy is metadata-only, and the ~100 test binaries, the rustfs bin and
|
||||
@@ -138,6 +134,10 @@ jobs:
|
||||
retention-days: 30
|
||||
if-no-files-found: error
|
||||
|
||||
# --all-targets covers the test binaries nextest builds, including
|
||||
# e2e_test, which test-and-lint's own run excludes. The second build adds
|
||||
# the e2e-test-hooks feature resolution that build-rustfs-debug-binary uses
|
||||
# and that no lint lane enables.
|
||||
- name: Build ci-dev superset
|
||||
env:
|
||||
# Same limit ci.yml puts on its nextest step: this builds the same
|
||||
@@ -148,6 +148,21 @@ jobs:
|
||||
cargo build --workspace --all-targets
|
||||
cargo build -p rustfs --bins --features e2e-test-hooks
|
||||
|
||||
# Runs before rust-cache's post step, so these are the sizes it is about
|
||||
# to archive. Reported so the cache-all-crates decision stays evidence-led:
|
||||
# registry/src is what that flag prunes, registry/cache is what the pruned
|
||||
# sources are re-unpacked from. See rustfs/backlog#1600.
|
||||
- name: Report cache input sizes
|
||||
if: always()
|
||||
run: |
|
||||
{
|
||||
echo "### Cache input sizes (ci-dev)"
|
||||
echo '```'
|
||||
du -sh ~/.cargo/registry/src ~/.cargo/registry/cache ~/.cargo/registry/index \
|
||||
~/.cargo/git target 2>/dev/null || true
|
||||
echo '```'
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# Readers: test-and-lint-rio-v2, build-rustfs-debug-binary-rio-v2.
|
||||
warm-ci-feat-rio:
|
||||
name: Warm ci-feat-rio
|
||||
|
||||
Reference in New Issue
Block a user