mirror of
https://github.com/rustfs/rustfs.git
synced 2026-09-06 03:59:14 +00:00
ci(functional): reliable chain driver, heal-once, backlog issues, clone retry (#7013)
Problem: the nightly functional chain has not completed end-to-end. Evidence from recent runs: - workflow_run events are fire-and-forget: after KMS finished at 17:09Z on 8/31 no tier run was created; rustfs-storage-test.yml has never run. - 'if: conclusion == success' gates skip downstream suites on any failure (security was skipped after pool failed on 9/1 01:48Z). - rustfs-pool-expand-test.yml embedded a heal pass without continue-on-error, so a heal failure failed the whole workflow. Fixes: - Add rustfs-functional-chain.yml: entry point that dispatches the first suite via repository_dispatch; each suite hands off to the next with an explicit, re-drivable API call instead of workflow_run triggers. - Split heal out of the pool workflow (renamed to RustFS Pool Expansion Test): heal now runs exactly once per chain, in rustfs-heal-test.yml (storage -> heal -> pool). - Every suite job gets continue-on-error so a failing test never fails the workflow; failures are filed as issues in rustfs/backlog (report + redacted log tail) and the chain moves on. - Clone rustfs/auto-testing with the PF token via 'gh repo clone' plus a 5-attempt retry loop (transient clone failures aborted whole suites). - Stop rewriting functional/index.html from every suite (divergent copies raced each other with stale SHAs); the canonical index now lives in the dashboard repo. - Standalone workflow_dispatch runs are unchanged and never forward the chain; performance runs on its own runner, dispatched in parallel.
This commit is contained in:
@@ -0,0 +1,74 @@
|
||||
# Copyright 2024 RustFS Team
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# Functional chain driver: runs the nine functional suites in a fixed order
|
||||
# (upgrade -> s3 -> kms -> tier -> storage -> heal -> pool -> security, with
|
||||
# performance on its own runner in parallel) and guarantees the chain keeps
|
||||
# moving even when individual suites fail.
|
||||
#
|
||||
# Each suite workflow can still be dispatched standalone (workflow_dispatch);
|
||||
# only chain-triggered runs forward to the next suite via repository_dispatch,
|
||||
# so a standalone run never drags the rest of the chain behind it.
|
||||
#
|
||||
# Why not workflow_run chaining: GitHub does not guarantee delivery of
|
||||
# workflow_run events (they are fire-and-forget), and the head-SHA filter made
|
||||
# newly added suites (storage) unable to trigger at all. Explicit
|
||||
# repository_dispatch handoffs are verifiable and re-drivable.
|
||||
|
||||
name: RustFS Functional Chain
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_run:
|
||||
# Entry point: start the chain after the nightly build completes. The
|
||||
# build's own conclusion does not gate the chain; each suite reports its
|
||||
# own result to rustfs/backlog and the dashboard.
|
||||
workflows: ["Nightly GNU Build"]
|
||||
types: [completed]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
start-chain:
|
||||
name: Start functional chain (upgrade first)
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
if: ${{ github.event_name == 'workflow_dispatch' || (github.event_name == 'workflow_run' && github.event.workflow_run.event == 'schedule') }}
|
||||
steps:
|
||||
- name: Dispatch first suite (upgrade)
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.PF_TESTING_GH_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${GH_TOKEN:-}" ]; then
|
||||
echo "PF_TESTING_GH_TOKEN is not configured; cannot start the functional chain" >&2
|
||||
exit 1
|
||||
fi
|
||||
gh api --method POST repos/rustfs/rustfs/dispatches \
|
||||
-f event_type='rustfs-chain-upgrade' \
|
||||
-F 'client_payload[from_suite]=nightly-build'
|
||||
|
||||
- name: Dispatch performance suite (parallel, own runner)
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.PF_TESTING_GH_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${GH_TOKEN:-}" ]; then
|
||||
echo "PF_TESTING_GH_TOKEN is not configured; cannot dispatch performance" >&2
|
||||
exit 1
|
||||
fi
|
||||
gh api --method POST repos/rustfs/rustfs/dispatches \
|
||||
-f event_type='rustfs-chain-performance' \
|
||||
-F 'client_payload[from_suite]=nightly-build'
|
||||
Reference in New Issue
Block a user