fix(ci): align FT evidence contract with scenario E (44 cases) (#8052)

The fault-tolerance suite now registers and runs scenario E
(rustfs/auto-testing#100): --all covers A, B, C, C2, D, E with 38 + 6
= 44 expected cases. The chain-evidence assertion still hardcoded
[A, B, C, C2, D] and 38 everywhere, which would fail every green FT
run's evidence validation once E is registered.

Bump the scenario list to include E and all count assertions from 38
to 44; step name updated to match. No other lanes reference 38.
This commit is contained in:
hector
2026-09-21 20:13:56 +08:00
committed by GitHub
parent 8ba3309624
commit bdea44f332
@@ -149,7 +149,7 @@ jobs:
CHAIN_MANIFEST: ${{ inputs.chain_manifest }}
run: python3 scripts/prepare_functional_package.py prepare
- name: Run fault-tolerance scenarios (A, B, C, C2, D)
- name: Run fault-tolerance scenarios (A, B, C, C2, D, E)
timeout-minutes: 45
id: test
# Standalone case failures are reported by the backlog manager.
@@ -194,21 +194,21 @@ jobs:
jq -e --arg testing_sha "${AUTO_TESTING_SHA}" '
.schema_version == 1 and
.auto_testing_sha == $testing_sha and
.selected_scenarios == ["A", "B", "C", "C2", "D"] and
.selected_scenarios == ["A", "B", "C", "C2", "D", "E"] and
.complete == true and
.counts.expected == 38 and
.counts.seen == 38 and
.counts.expected == 44 and
.counts.seen == 44 and
.counts.harness_errors == 0 and
(.expected_cases | length) == 38 and
(.expected_cases | unique | length) == 38 and
(.seen_cases | length) == 38 and
(.seen_cases | unique | length) == 38 and
(.expected_cases | length) == 44 and
(.expected_cases | unique | length) == 44 and
(.seen_cases | length) == 44 and
(.seen_cases | unique | length) == 44 and
(.seen_cases | sort) == (.expected_cases | sort) and
(.missing_cases | length) == 0 and
(.harness_errors | length) == 0 and
(.cases | length) == 38 and
(.cases | length) == 44 and
([.cases[].case_id] | sort) == (.expected_cases | sort) and
([.cases[].case_id] | unique | length) == 38 and
([.cases[].case_id] | unique | length) == 44 and
all(.cases[]; .verdict == "pass" or .verdict == "known-divergence" or .verdict == "UNEXPECTED") and
.counts.unexpected == ([.cases[] | select(.verdict == "UNEXPECTED")] | length) and
.counts.known_divergence == ([.cases[] | select(.verdict == "known-divergence")] | length)