Files
rustfs/.github/workflows/fuzz.yml
T
Zhengchao An 6c99d4fe22 ci: stop the weekly flake.lock bot from running the whole pipeline (#5539)
nix-flake-update opens a PR every Sunday that changes flake.lock and nothing
else. flake.lock is consumed only by Nix packaging — cargo never reads it — yet
it was in no paths filter, so each of those PRs ran the full Continuous
Integration pipeline and the merge then ran Build and Release too. Added to all
four lists that have to agree: ci.yml's push and pull_request paths-ignore,
build.yml's push paths-ignore, and ci-docs-only.yml's paths.

The cron stays. flake.lock still has consumers — anyone running `nix build` or
`nix develop` — so stopping the updates would remove the workflow's output, not
just its CI cost.

Those four lists drifting is a silent failure, so scripts/check_ci_paths_sync.sh
now asserts the pair that matters: ci.yml's pull_request paths-ignore must equal
ci-docs-only.yml's paths. An entry present only in the first means a PR touching
those files triggers neither workflow, nobody reports "Test and Lint" or "Quick
Checks", and the PR waits on a required check forever. It also asserts both
companion job names still exist, since renaming one produces the same hang. The
push list is not compared: no required check is reported for push events.

Also in this cleanup:

- nix-flake-update's GITHUB_TOKEN drops to contents: read. The branch push and
  the pull request are both made by update-flake-lock with the
  FLAKE_UPDATE_TOKEN PAT, so the write scopes were an unused repo-write
  credential on an unattended weekly job.
- build.yml's build_docker dispatch input is documented as advisory. docker.yml
  triggers on workflow_run and requires the triggering event to be a tag push,
  so a manual dispatch never reaches it whatever this input says.
- The nine disabled workflow files get a banner saying so. Their
  disabled_manually state lives in GitHub's UI and is invisible when reading the
  file, which has already misled one audit into treating dead workflows as live.

Refs: rustfs/backlog#1598, rustfs/backlog#1603
2026-08-01 11:28:14 +08:00

262 lines
11 KiB
YAML

# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# DISABLED. This workflow is switched off in the repository's Actions settings
# (state: disabled_manually) and does not run on any trigger, including its cron
# and workflow_dispatch. That state lives in GitHub's UI and is invisible when
# reading this file, which has already misled at least one audit — hence this
# banner. Re-enabling is a UI action; anyone doing so should first check that the
# workflow still matches the current CI layout. See rustfs/backlog#1603.
#
name: Fuzz
on:
pull_request:
types: [ opened, synchronize, reopened, closed ]
# PR trigger is intentionally narrow: only changes to the fuzz harness
# itself gate a PR. Broad crate paths (ecstore/filemeta/utils/policy/…)
# are covered by the nightly `schedule` run below, which fuzzes against
# whatever landed on main. Widening these paths previously queued a
# ~45min fuzz-build on nearly every PR and is why this workflow was
# disabled; do not re-add crate paths here.
paths:
- "fuzz/**"
- "scripts/fuzz/**"
- ".github/workflows/fuzz.yml"
schedule:
- cron: "0 2 * * *"
workflow_dispatch:
inputs:
profile:
description: "Which fuzz profile to run"
required: false
default: "both"
type: choice
options:
- smoke
- nightly
- both
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env:
CARGO_TERM_COLOR: always
CARGO_BUILD_TARGET: x86_64-unknown-linux-gnu
# Fuzz targets build without the dial9 feature, so tokio_unstable is not needed.
RUSTFLAGS: "-C target-feature=-crt-static"
jobs:
cancel-closed-pr-runs:
name: Cancel Closed PR Runs
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Explain cancellation run
run: echo "PR closed; this run only cancels older runs in the same concurrency group."
# ──────────────────────────────────────────────────────────────
# Phase 1: Build all fuzz harness binaries once.
# ──────────────────────────────────────────────────────────────
fuzz-build:
name: Build Fuzz Harness
if: >
(github.event_name == 'pull_request' && github.event.action != 'closed') ||
github.event_name == 'schedule' ||
github.event_name == 'workflow_dispatch'
runs-on: sm-standard-4
timeout-minutes: 45
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- name: Setup Rust environment
uses: ./.github/actions/setup
with:
rust-version: nightly
cache-shared-key: fuzz-${{ hashFiles('fuzz/Cargo.lock') }}
github-token: ${{ secrets.GITHUB_TOKEN }}
cache-save-if: ${{ github.ref == 'refs/heads/main' || github.event_name == 'schedule' }}
- name: Install cargo-fuzz
uses: taiki-e/install-action@bffeee26d4db9be238a4ea78d8826604ebcb594d # v2
with:
tool: cargo-fuzz
- name: Build all fuzz targets
run: BUILD_ONLY=1 ./scripts/fuzz/run.sh
- name: Stage prebuilt fuzz binaries
run: |
binary_root="fuzz/prebuilt/${CARGO_BUILD_TARGET}/release"
mkdir -p "${binary_root}"
# Keep this list in sync with the smoke/nightly matrices and
# scripts/fuzz/run.sh default target set (all 5 buildable bins in
# fuzz/Cargo.toml). The *_storage_api.rs files are `mod` submodules
# of their parent targets, not standalone bins.
for target in archive_extract bucket_validation local_metadata path_containment policy_ingress; do
install -Dm755 "fuzz/target/${CARGO_BUILD_TARGET}/release/${target}" "${binary_root}/${target}"
done
- name: Upload prebuilt fuzz binaries
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: fuzz-prebuilt-binaries-${{ github.run_number }}
path: |
fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/archive_extract
fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/bucket_validation
fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/local_metadata
fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/path_containment
fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/policy_ingress
if-no-files-found: error
retention-days: 1
compression-level: 0
# ──────────────────────────────────────────────────────────────
# Phase 2 (PR): Run smoke targets in parallel via matrix.
# ──────────────────────────────────────────────────────────────
pr-fuzz-smoke:
name: "Smoke / ${{ matrix.target }}"
needs: fuzz-build
if: >
(github.event_name == 'pull_request' && github.event.action != 'closed') ||
(github.event_name == 'workflow_dispatch' &&
(github.event.inputs.profile == 'smoke' || github.event.inputs.profile == 'both'))
runs-on: sm-standard-4
timeout-minutes: 30
strategy:
fail-fast: false
# One job per target runs in parallel, so wall-clock time is per-target
# (~60s fuzz + download/chmod overhead), not the sum across the matrix.
matrix:
target: [archive_extract, bucket_validation, local_metadata, path_containment, policy_ingress]
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- name: Download prebuilt fuzz binaries
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
with:
name: fuzz-prebuilt-binaries-${{ github.run_number }}
path: fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release
- name: Restore executable bit
run: chmod +x "fuzz/prebuilt/${CARGO_BUILD_TARGET}/release/${{ matrix.target }}"
- name: Run ${{ matrix.target }}
env:
FUZZ_TARGET: ${{ matrix.target }}
MAX_TOTAL_TIME: 60
SKIP_BUILD: 1
USE_PREBUILT_BINARY: 1
PREBUILT_BINARY_DIR: ${{ github.workspace }}/fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release
run: ./scripts/fuzz/run.sh
- name: Upload fuzz smoke artifacts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: fuzz-smoke-${{ matrix.target }}-${{ github.run_number }}
path: |
fuzz/artifacts/**
fuzz/corpus/${{ matrix.target }}/**
if-no-files-found: ignore
retention-days: 7
# ──────────────────────────────────────────────────────────────
# Phase 2 (Nightly): Run all targets in parallel via matrix.
# ──────────────────────────────────────────────────────────────
nightly-fuzz-corpus:
name: "Nightly / ${{ matrix.target }}"
needs: fuzz-build
# TODO(ci-8): when the schedule-failure-issue composite action lands,
# add a step here (or a dependent job) that opens/updates a GitHub issue
# on nightly failure. ci-8 is the single alerting mechanism for all
# scheduled workflows; do not self-roll alerting in this workflow.
if: >
github.event_name == 'schedule' ||
(github.event_name == 'workflow_dispatch' &&
(github.event.inputs.profile == 'nightly' || github.event.inputs.profile == 'both'))
runs-on: sm-standard-4
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
target: [archive_extract, bucket_validation, local_metadata, path_containment, policy_ingress]
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- name: Download prebuilt fuzz binaries
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
with:
name: fuzz-prebuilt-binaries-${{ github.run_number }}
path: fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release
- name: Restore executable bit
run: chmod +x "fuzz/prebuilt/${CARGO_BUILD_TARGET}/release/${{ matrix.target }}"
- name: Run ${{ matrix.target }}
env:
FUZZ_TARGET: ${{ matrix.target }}
MAX_TOTAL_TIME: 300
SKIP_BUILD: 1
USE_PREBUILT_BINARY: 1
PREBUILT_BINARY_DIR: ${{ github.workspace }}/fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release
run: ./scripts/fuzz/run.sh
- name: Upload nightly fuzz artifacts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: fuzz-nightly-${{ matrix.target }}-${{ github.run_number }}
path: |
fuzz/artifacts/**
fuzz/corpus/${{ matrix.target }}/**
if-no-files-found: ignore
retention-days: 30
# ──────────────────────────────────────────────────────────────
# Nightly alerting: open/update a tracking issue on failure.
# ──────────────────────────────────────────────────────────────
alert-on-failure:
name: Alert on scheduled failure
needs: [fuzz-build, nightly-fuzz-corpus]
# `always()` is required: without it this job is skipped when a needed
# job fails. Alerts only for scheduled (nightly) runs (backlog#1149
# ci-8); PR and manual dispatch failures are already watched by a human.
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- name: Open or update failure-tracking issue
uses: ./.github/actions/schedule-failure-issue
with:
github-token: ${{ secrets.GITHUB_TOKEN }}