mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-05 12:57:42 +00:00
Zhengchao An
428fde069d
Two cleanups, neither changing what is built or published. docker.yml loses its type=gha layer cache. The image build compiles nothing — it downloads a release zip and runs apk/apt — so the cache could only save the minute or two those take, against a real correctness problem: with RELEASE=latest the binary URL is resolved by curl inside a RUN layer, and the layer key does not include what it resolved to. A rebuild at the same RELEASE value (a dispatch with version=latest, or a re-run of the same version) would hit the old layer and ship the previous release's binary. mode=max also drew on the same repo-wide 10GB Actions cache quota the Rust lanes are contending for. Only RELEASE is passed as a build-arg now; it is the sole one the Dockerfiles declare besides TARGETARCH. BUILDTIME, VERSION, BUILD_TYPE, REVISION and CHANNEL were read by no stage, and BUILDTIME's $(date ...) was a literal string in the YAML block rather than a substitution. The DOCKER_CHANNEL computation that fed CHANNEL evaluated to "release" down every branch and is gone with it. BUILD_DATE and VCS_REF stay unset even though the Dockerfiles declare them: supplying them would change the published image labels. The setup composite stops installing tools its callers do not use. protobuf- compiler comes out of the apt list entirely — setup-protoc installs 34.1 into the tool cache and prepends it to PATH, so the apt build was shadowed on every run and never used; the same duplicate install is removed from the io_uring lane. musl-tools, zip and unzip move behind install-build-packaging-tools, off for the CI and coverage lanes and left on for build.yml, whose native musl leg needs musl-gcc and whose packaging steps need zip. cargo-nextest and the rustfmt/clippy components move behind install-test-tools, off only for build.yml, which runs no tests and no lints; coverage and the nightly replication lane keep them because both invoke nextest. The action's github-token input is deleted along with its 20 call sites. Its runs block never referenced it — setup-protoc uses github.token directly — so it was 20 places handing a token to something that ignored it. Refs: rustfs/backlog#1598, rustfs/backlog#1600, rustfs/backlog#1603
131 lines
5.3 KiB
YAML
131 lines
5.3 KiB
YAML
# Copyright 2024 RustFS Team
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
# Nightly full replication e2e lane (backlog#1147 repl-1, deps: ci-4).
|
|
#
|
|
# The per-PR gate (ci.yml `e2e-tests` job, `--profile e2e-smoke`) runs the 20
|
|
# FAST replication tests. This scheduled lane runs the remaining 27
|
|
# heavier replication e2e tests that are unfit for a per-PR gate:
|
|
#
|
|
# * 2 remote-target TLS validation tests.
|
|
# * 12 bucket-replication data-plane/helper tests (PUT/delete + poll for
|
|
# convergence; two replicate over HTTPS, two pin active SSE failure
|
|
# contracts, and one guards event/history observers). The SSE-S3 contract
|
|
# remains ignored under backlog#1291.
|
|
# * 11 `_real_dual_node` site-replication tests (each spawns TWO rustfs
|
|
# servers and drives the cross-process site-replication control plane).
|
|
# * 1 `_real_three_node` site-replication test.
|
|
# * 1 `_real_single_node` service-account round-trip test.
|
|
#
|
|
# The selection is the [profile.e2e-repl-nightly] default-filter in
|
|
# .config/nextest.toml — the single wiring mechanism (repl-1 / ci-4). Do NOT
|
|
# add ad-hoc cargo-test steps here; change the filterset instead.
|
|
#
|
|
# Explicit division of labor: these 27 tests run ONLY here, never double-run
|
|
# in ci-5's future e2e-full merge gate. TODO(ci-7): once the ci domain's
|
|
# consolidated scheduled e2e workflow exists, fold this interim repl-owned lane
|
|
# into it rather than growing a second scheduled entrypoint.
|
|
|
|
name: e2e-replication-nightly
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
schedule:
|
|
# 04:00 UTC nightly — staggered clear of fuzz/e2e-s3tests (02:00),
|
|
# stale (01:30) and performance-ab (06:00).
|
|
- cron: "0 4 * * *"
|
|
|
|
# Only alert-on-failure needs more than read access; it declares its own
|
|
# job-level `issues: write`.
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
repl-nightly:
|
|
name: Replication e2e (nightly)
|
|
# dual-node tests spawn two full rustfs servers each; the extra headroom
|
|
# over the per-PR sm-standard-2 e2e job keeps the parallel process fan-out
|
|
# from starving the runner. Mirrors the ILM serial lane's runner class.
|
|
runs-on: sm-standard-4
|
|
timeout-minutes: 45
|
|
env:
|
|
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
|
|
|
|
- name: Setup Rust environment
|
|
uses: ./.github/actions/setup
|
|
with:
|
|
rust-version: stable
|
|
cache-shared-key: ci-e2e-repl
|
|
cache-save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
install-build-packaging-tools: 'false'
|
|
|
|
# awscurl lets the STS dual-node test actually exercise its path. Without
|
|
# it the test skips gracefully with a visible log line
|
|
# (`awscurl_available()` in crates/e2e_test/src/common.rs), so the lane
|
|
# still passes — installing it just upgrades that one test from skip to
|
|
# real coverage.
|
|
- name: Set up Python
|
|
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
|
|
with:
|
|
python-version: "3.12"
|
|
|
|
- name: Install awscurl
|
|
run: |
|
|
python3 -m pip install --user --upgrade pip awscurl
|
|
echo "AWSCURL_PATH=$HOME/.local/bin/awscurl" >> "$GITHUB_ENV"
|
|
|
|
- name: Verify awscurl
|
|
run: test -x "$AWSCURL_PATH"
|
|
|
|
# Build the rustfs binary once up front. The e2e tests spawn it as a
|
|
# child process (crates/e2e_test/src/common.rs) and will build it on
|
|
# demand otherwise, but a single explicit build avoids several parallel
|
|
# nextest test processes racing to build it at once.
|
|
- name: Build rustfs binary
|
|
run: cargo build -p rustfs --bins
|
|
|
|
- name: Run replication e2e nightly suite
|
|
run: cargo nextest run --profile e2e-repl-nightly -p e2e_test
|
|
|
|
- name: Upload nextest junit report
|
|
if: always()
|
|
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
|
|
with:
|
|
name: e2e-replication-nightly-junit-${{ github.run_number }}
|
|
path: target/nextest/e2e-repl-nightly/junit.xml
|
|
retention-days: 7
|
|
if-no-files-found: ignore
|
|
|
|
alert-on-failure:
|
|
name: Alert on scheduled failure
|
|
needs: [repl-nightly]
|
|
# Only scheduled runs open/append the tracking issue (backlog#1149 ci-8);
|
|
# manual workflow_dispatch runs stay quiet so a debugging run never files a
|
|
# spurious alert.
|
|
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
permissions:
|
|
contents: read
|
|
issues: write
|
|
steps:
|
|
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
|
|
- name: Open or update failure-tracking issue
|
|
uses: ./.github/actions/schedule-failure-issue
|
|
with:
|
|
github-token: ${{ secrets.GITHUB_TOKEN }}
|