mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-04 04:17:44 +00:00
Zhengchao An
7354a5663d
The repository has 18 rust-cache families of 1.2-3.1GB each against GitHub's fixed 10GB per-repo quota. Measured usage sat at 9.72GB, 9.96GB and 11.63GB on three samples, so LRU eviction is continuous and the main lanes lose: ci-test and ci-e2e were repeatedly absent from the surviving entries. That is what makes "Setup Rust environment" bimodal — 0.7-3.4 minutes warm against 11.8-20.9 minutes cold. Four changes, all cache-only. No job builds or tests anything different. Collapse ci.yml's nine cache sites into four keys, each with exactly one writer: ci-dev (test-and-lint writes; ILM, debug-binary, e2e-tests and e2e-full read), ci-feat-rio (rio-v2 lint writes, its debug-binary reads), ci-feat-proto (the swift leg writes for both protocol legs), and ci-uring, which stays alone. e2e-full is easy to miss here: it already shared ci-e2e with e2e-tests and both saved, so without an explicit 'false' it would have become a second unnamed writer of ci-dev. rio/swift/sftp are deliberately NOT merged — measured at 2365/2307/1200MB they are not near-identical, and none is a superset of the others. Give each writer a superset warm-up on main. A writer's own steps are not automatically a superset of its readers': clippy emits metadata only, the nextest pass excludes e2e_test, and no lint lane enables e2e-test-hooks, whose different feature resolution yields a different -Cmetadata. Without these builds the readers would restore a cache missing precisely what they need. Guarded to main, so the PR critical path is unaffected. Stop writing tag-scoped caches in build.yml. A cache saved on refs/tags/X can only be restored by a re-run of that same tag, so each release cycle wrote up to 12 unreadable 1-2GB entries that evicted the hot lanes. Tag builds still restore the main-scoped cache. The one real cost is that re-running a failed leg of the same tag now falls back to main's cache. Flip the composite action's cache-save-if default to 'false' and make the input mandatory in practice. audit.yml was relying on the old "true" default: every PR touching Cargo.toml or Cargo.lock saved a second, PR-scoped copy (~843MB measured, job 91048468127) that pushed main-scoped lanes out of the quota. The fail-safe direction is a cold cache, not a stolen quota slice. scripts/security/check_cache_save_if.sh now asserts every call site states it, wired into audit.yml next to the existing pin check. While there, cargo-deny stops pulling the full setup composite. It compiles nothing, so apt, protoc, flatc and nextest were pure overhead — but it does run cargo metadata, and Cargo.toml pins datafusion and s3s as git dependencies that must be materialised into ~/.cargo/git, so the cache itself stays. Refs: rustfs/backlog#1598, rustfs/backlog#1600
102 lines
3.2 KiB
YAML
102 lines
3.2 KiB
YAML
# Copyright 2024 RustFS Team
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
name: "Setup Rust Environment"
|
|
description: "Setup Rust development environment with caching for RustFS"
|
|
|
|
inputs:
|
|
rust-version:
|
|
description: "Rust version to install"
|
|
required: false
|
|
default: "stable"
|
|
cache-shared-key:
|
|
description: "Shared cache key for Rust dependencies"
|
|
required: false
|
|
default: "rustfs-deps"
|
|
cache-save-if:
|
|
description: >-
|
|
Whether to save the cache. The fail-safe default is 'false': a caller that
|
|
wants to populate a cache must opt in explicitly, so a forgotten input
|
|
costs a cold cache (minutes) rather than silently consuming the
|
|
repository-wide 10GB Actions cache quota and evicting other lanes.
|
|
required: false
|
|
default: "false"
|
|
install-cross-tools:
|
|
description: "Install cross-compilation tools"
|
|
required: false
|
|
default: "false"
|
|
target:
|
|
description: "Target architecture to add"
|
|
required: false
|
|
default: ""
|
|
github-token:
|
|
description: "GitHub token for API access"
|
|
required: false
|
|
default: ""
|
|
|
|
runs:
|
|
using: "composite"
|
|
steps:
|
|
- name: Install system dependencies (Ubuntu)
|
|
if: runner.os == 'Linux'
|
|
shell: bash
|
|
run: |
|
|
sudo apt-get update
|
|
sudo apt-get install -y \
|
|
musl-tools \
|
|
build-essential \
|
|
pkg-config \
|
|
libssl-dev \
|
|
ripgrep \
|
|
unzip \
|
|
zip \
|
|
protobuf-compiler
|
|
|
|
- name: Install protoc
|
|
uses: rustfs/setup-protoc@a3705324d8f9bf5b6c3573fb6cf8ae421db55dd6 # v3.0.1
|
|
with:
|
|
version: "34.1"
|
|
repo-token: ${{ github.token }}
|
|
|
|
- name: Install flatc
|
|
uses: Nugine/setup-flatc@e7855e994773ce90094a3f1626d4afc9080c23ae # v1
|
|
with:
|
|
version: "25.12.19"
|
|
|
|
- name: Install Rust toolchain
|
|
uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
|
|
with:
|
|
toolchain: ${{ inputs.rust-version }}
|
|
targets: ${{ inputs.target }}
|
|
components: rustfmt, clippy
|
|
|
|
- name: Install Zig
|
|
if: inputs.install-cross-tools == 'true'
|
|
uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2
|
|
|
|
- name: Install cargo-zigbuild
|
|
if: inputs.install-cross-tools == 'true'
|
|
uses: taiki-e/install-action@a21ae4029b089b9ddc45704028756f51ab8abe48 # cargo-zigbuild
|
|
|
|
- name: Install cargo-nextest
|
|
uses: taiki-e/install-action@96c7780c1d8a2b8723e12031def873a434d39d8d # nextest
|
|
|
|
- name: Setup Rust cache
|
|
uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
|
|
with:
|
|
cache-all-crates: true
|
|
cache-on-failure: true
|
|
shared-key: ${{ inputs.cache-shared-key }}
|
|
save-if: ${{ inputs.cache-save-if }}
|