Files
rustfs/docs/testing/validation-candidates.md
T
cxymds 4a63e53161 test(validation): run canonical cases and build pinned Heal tools (#8455)
* test(validation): add pinned auto-testing manual lane

* Install pinned ILM client for canonical validation lane

* Prepare Python client for pool validation cases

* Install validation client for replication cases

* test(validation): provide the Python client for Heal cases

* test(validation): build the candidate-bound Heal inspector artifact

* test(validation): expose tool artifacts through the existing manual lane

* test(validation): verify pinned Heal tools before fleet execution

* test(validation): verify Heal tools against released shard captures

* ci: audit retained validation environments before finalization

* test(validation): add optional native Heal regression tooling

* test(validation): route native tools through the existing driver

* Run validation tool resolution on hosted Linux without acquiring the fleet lock

* test(validation): bind C07 to pinned native authority evidence

* test(validation): build optional C10 fixture codec on hosted Linux

* test(ci): bind C10 execution to the verified shared codec

* feat(ci): expose read-only validation fleet inventory

* docs(ci): describe supported validation recovery

* test(ci): add isolated manager native validation builds

* test(ci): expose manager regressions through tools-only dispatch

* ci(validation): consume pinned manager native prerequisites

* fix(heal): preserve typed bucket errors for bounded retries

* fix(heal): preserve typed errors and SSE-C response headers

* ci(validation): publish isolated immutable candidate packages

* test(validation): enforce private candidate manifest schema

* test(validation): verify isolated package publication failures

* ci(validation): dispatch isolated packages through the driver

* fix(heal): deeply verify read repair payload failures

* test(ecstore): verify incarnation fence at repair commit

* ci(validation): verify C11 native storage evidence

* fix(ci): isolate validation package publication credentials

* test(ci): archive scoped F07 directory fsync evidence

* fix(ecstore): retain typed quorum errors for missing parts

* chore(ecstore): shrink typed-error ratchet baseline

* fix(ecstore): retain below-quorum data during no-remove heal

* fix(ci): allow recovery of completed cancelled validation runs

* fix(heal): retain exhausted admin batch repair receipts

* fix(observability): retain typed object error diagnostics

* fix(s3): map typed RPC unavailability to service refusal

* fix(filemeta): classify supported inline framing damage as corrupt

* test(ecstore): exercise heal during committed overwrite cleanup

* ci(validation): admit source-bound G01 native evidence before fleet execution

* test(ecstore): fence version races and bind cleanup slots

* ci(validation): require complete race native runtime preflight

* test(ecstore): fence marker and batch deletes during heal

* ci(validation): require G04 native and runtime admission

* test(ecstore): fence multipart continue complete and abort during heal

* test(ecstore): cover default heal owners and copy metadata races

* ci(validation): admit source-bound owner and copy race proofs

* test(scanner): assert typed corruption after metadata truncation

* fix(ci): require complete native admission for multipart races

* test(validation): wire I05 queued-successor native prerequisites

* fix(scanner): bind object heals to the scanning disk scope

* test(scanner): reuse storage boundary for scope regression

* style: format RPC digest precedence assertion

* fix(heal): query automatic tasks on their owning node

* test(admin): reuse error facade in heal query regression

* fix(heal): retain cancellation after successful bucket storage I/O

* fix(ci): retain validation evidence outside runner temp

* fix(heal): retain resolved object identity on terminal failures

* test(scanner): adapt scope regression to admission receipts

* ci(validation): inspect shared runner without fleet access

* ci(validation): prepare Python before creating client environments

* ci(validation): isolate each client virtual environment

* ci(validation): retain interrupted evidence without fleet recovery

* ci(validation): bind recovery to retained runner evidence

* ci(validation): wire persistence and lock source plans

* fix(validation): align read-only dispatch defaults

---------

Co-authored-by: Chris <anzhengchao@gmail.com>
2026-10-29 08:50:42 +08:00

53 lines
2.6 KiB
Markdown

# Immutable validation candidates
Canonical validation on the current auto-testing fleet consumes one exact
package, source commit and candidate-bound tool build. A successful package
build is provenance evidence, not a canonical test PASS.
The scheduled Functional Chain continues to resolve successful Nightly GNU
Build attempts from `main`. Optional manual validation can also resolve a
validation package built without publishing nightly aliases or release assets.
## Build a validation package
Dispatch **RustFS Functional Chain** from the desired driver revision with
only `validation_package_sha` set to the full 40-character RustFS source SHA.
Keep candidate, case, helper, tools and cleanup inputs empty or false. The
request validator rejects mixed package/fleet execution before building.
The package job calls `rustfs-validation-package.yml`, checks out the requested
source, confirms its Git SHA and builds the ordinary release binary and DEB.
This job uses build runners and does not acquire or mutate the Heal fleet.
The same workflow can be dispatched directly with its `source_sha` input when
it is available on the repository's default branch.
Publication requires complete R2 configuration, conditional `PutObject` with
`If-None-Match: *`, and a public download whose SHA-256 matches the local DEB.
Failure emits no candidate manifest. An existing object is never overwritten.
The only object path is:
```text
artifacts/rustfs/packages/validation/runs/<run>/<attempt>/<package-sha256>/rustfs.deb
```
The schema-3 artifact `validation-candidate-<run>-<attempt>` records the
workflow SHA independently of the exact source SHA and source ref, the
`validation` channel, build run/attempt, immutable URL and package checksum.
The resolver verifies the successful manual run, repository/workflow identity,
exact artifact name and digest, archive members, schema and immutable URL.
The ordinary scheduled chain does not enable validation package resolution.
## Validate the candidate
After the package run succeeds, dispatch Functional Chain with its exact
`build_run_id` and `build_run_attempt`, a canonical `validation_case`, and an
exact `auto_testing_sha`. Heal also requires a new tool build verified against
that source and helper revision. Reassess source-bound contracts and native
prerequisites when the source changes; an older tool artifact does not certify
a new candidate.
The fleet lane retains the shared environment lock, per-case evidence and
cleanup gate. Record new results separately. A package build, native test,
diagnostic replay, clean recovery or partial topology result cannot replace
complete original source assertions or rewrite a retained canonical FAIL.