Compare commits

...

6 Commits

Author SHA1 Message Date
hector ccc25bce07 ci(validation): install the validation Python client requirements
The pool runner dies with 'boto3 is required for pool validation'
before its assertions: the managed interpreter alone is not enough.
Mirror the chain validation job exactly - build a venv from
validation_ilm/requirements.txt and put its bin on PATH so every
module runner (pool/ilm/replication) gets its client stack.
2026-11-02 16:38:53 +08:00
hector 6199e30ffc Merge branch 'main' into ci/validation-sweep-schedule-fix 2026-11-02 15:01:41 +08:00
hector 835b84c57a ci(validation): install sweep Python client dependencies
Scheduled sweeps failed pool.status before any assertion because
pool_behavior.py exits when boto3 is missing. Install the auto-testing
validation requirements into a per-run virtualenv and put it on PATH,
matching the functional chain, so case scripts can import the client.

Co-authored-by: hector <majinghe@users.noreply.github.com>
2026-11-02 06:56:18 +00:00
hector 14a2fd9569 ci(validation): sweep harness fixes from the first live run
- ssh -n in the inter-case cleanup loop: plain ssh consumed the case
  list from the loop's stdin, so the first dispatch executed only one
  of the twelve selected cases.
- Provide the managed Python 3.12 runtime for pool/ilm/replication
  cases, mirroring the chain validation job (pool.status reported
  INFRA_ERROR after its assertions passed; the missing runtime is the
  prime suspect).
- Record the auto-testing sha in the report step environment.
- Retain the last case's evidence tree in the sweep artifact so gate
  verdicts like INFRA_ERROR can be diagnosed from the run itself.
2026-11-02 09:27:20 +08:00
hector bf201caf34 ci(validation): read the resolver chain envelope correctly
CHAIN_OUTPUT is the chain envelope {schema, run_id, attempt,
workflow_sha, testing_sha, candidate}; the manifest lives under
candidate.manifest and the testing sha at the top level.
2026-11-02 09:17:09 +08:00
hector 4d81330479 ci(validation): resolve the newest nightly build on scheduled sweeps
The first scheduled sweep failed at candidate resolution: a schedule
carries no workflow inputs and resolve_functional_candidate.py only
derives a build from a workflow_run payload, so the empty
BUILD_RUN_ID crashed int(). Look up the newest successful
nightly-gnu run explicitly when the inputs are empty and feed it to
the resolver. Also downgrade the evidence artifact to
if-no-files-found: warn so an early infra failure does not cascade
into a second red step.
2026-11-02 09:09:20 +08:00
+39 -5
View File
@@ -92,6 +92,17 @@ jobs:
CHAIN_OUTPUT: ${{ runner.temp }}/validation-sweep-candidate.json
run: |
set -euo pipefail
# Scheduled runs carry no inputs and the resolver only derives a
# build from a workflow_run payload (which a schedule cannot
# provide), so resolve the newest successful nightly build
# explicitly before invoking it.
if [ -z "${BUILD_RUN_ID:-}" ]; then
LATEST="$(gh api "repos/rustfs/rustfs/actions/workflows/nightly-gnu.yml/runs?branch=main&status=success&per_page=1" --jq '.workflow_runs[0]')"
BUILD_RUN_ID="$(printf '%s' "${LATEST}" | jq -r '.id')"
BUILD_RUN_ATTEMPT="$(printf '%s' "${LATEST}" | jq -r '.run_attempt')"
export BUILD_RUN_ID BUILD_RUN_ATTEMPT
echo "scheduled sweep pinned to nightly ${BUILD_RUN_ID} attempt ${BUILD_RUN_ATTEMPT}"
fi
python3 scripts/resolve_functional_candidate.py
python3 - <<'PY'
import json
@@ -99,15 +110,15 @@ jobs:
from pathlib import Path
root = Path(os.environ["RUNNER_TEMP"]) / "validation-sweep"
root.mkdir(parents=True, exist_ok=True)
candidate = json.loads((Path(os.environ["RUNNER_TEMP"]) / "validation-sweep-candidate.json").read_text())
manifest = candidate["manifest"]
chain = json.loads((Path(os.environ["RUNNER_TEMP"]) / "validation-sweep-candidate.json").read_text())
manifest = chain["candidate"]["manifest"]
(root / "candidate.json").write_text(json.dumps({
"package_url": manifest["package_url"],
"package_sha256": manifest["package_sha256"],
"source_sha": manifest["source_sha"],
}, sort_keys=True) + "\n")
with open(os.environ["GITHUB_OUTPUT"], "a") as output:
output.write("testing_sha=" + candidate["testing_sha"] + "\n")
output.write("testing_sha=" + chain["testing_sha"] + "\n")
output.write("package_url=" + manifest["package_url"] + "\n")
output.write("source_sha=" + manifest["source_sha"] + "\n")
PY
@@ -121,6 +132,24 @@ jobs:
path: auto-testing
persist-credentials: false
- name: Prepare validation Python runtime
# The pool / ilm / replication validation runners require a managed
# Python with their client requirements, exactly like the chain's
# validation job provides (boto3 et al).
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Install validation Python client
run: |
set -euo pipefail
validation_python="${RUNNER_TEMP}/rustfs-validation-python-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
test ! -e "${validation_python}"
python3 -m venv "${validation_python}"
"${validation_python}/bin/python" -m pip install \
-r auto-testing/scripts/validation_ilm/requirements.txt
printf '%s\n' "${validation_python}/bin" >> "${GITHUB_PATH}"
- name: Select sweep cases
id: select
env:
@@ -224,7 +253,8 @@ jobs:
# topology, this purge guarantees the next case starts clean even
# when the previous one died mid-run.
for node in ${RUSTFS_NODES}; do
ssh -o BatchMode=yes -o ConnectTimeout=10 "${RUSTFS_SSH_USER}@${node}" '
# -n keeps ssh from consuming the case list this loop reads from stdin
ssh -n -o BatchMode=yes -o ConnectTimeout=10 "${RUSTFS_SSH_USER}@${node}" '
SUDO=""; [ "$(id -u)" -ne 0 ] && SUDO="sudo -n"
${SUDO} systemctl stop rustfs 2>/dev/null || true
if ${SUDO} dpkg -l rustfs 2>/dev/null | grep -q "^ii"; then
@@ -243,6 +273,7 @@ jobs:
SWEEP_ROOT: ${{ runner.temp }}/validation-sweep
PACKAGE_URL: ${{ steps.candidate.outputs.package_url }}
SOURCE_SHA: ${{ steps.candidate.outputs.source_sha }}
AUTO_TESTING_SHA: ${{ steps.candidate.outputs.testing_sha }}
run: |
set -euo pipefail
python3 - <<'PY'
@@ -351,7 +382,10 @@ jobs:
${{ runner.temp }}/validation-sweep/sweep-cases.txt
${{ runner.temp }}/validation-sweep/candidate.json
${{ runner.temp }}/validation-sweep/results/
if-no-files-found: error
${{ runner.temp }}/validation-sweep/run/
# warn (not error): an early infra failure legitimately leaves no
# per-case evidence, and the failing step is already red.
if-no-files-found: warn
retention-days: 30
- name: Cleanup environment (after)