Compare commits

...

13 Commits

Author SHA1 Message Date
Chris An 55d931353f ci(s3-tests): add reference parity lane against pinned 1.0.5 (#9288) 2026-11-10 22:50:17 -05:00
Chris An fc09148f08 chore(gateway): record ignored microbenchmarks merged from main (#9290)
chore(gateway): record the ignored microbenchmarks merged from main

main added three throughput microbenchmarks that are ignored by design
(#9217): one each in rustfs-checksums, rustfs-rio and rustfs-utils. The
test-count ratchet treats a higher ignored count as a failure until the
row is raised, so raise those three rows to 1 and tighten the test rows
to the merged tree with --update.
2026-11-10 21:30:34 -05:00
rustfs-cla-assistant[bot] c9e1464a2a Merge remote-tracking branch 'origin/main' into gateway/integration 2026-11-11 02:06:08 +00:00
Chris An 0e3028d29e ci(gateway): make the test-count ratchet a floor, not exact (#9278)
ci(gateway): make the test-count ratchet a floor instead of an exact baseline

Tests below a row still fail; tests above it, a member without a row and ignores below the row now pass with a note. Ignores above a row still fail. Adds contract tests wired into make script-tests.
2026-11-10 20:53:06 -05:00
Chris An 365014a385 feat(scripts): add junit_diff.py for dual-stack JUnit comparison (#9275) 2026-11-10 19:55:24 -05:00
Ramakrishna Chilaka 89d01aef4f perf(ecstore): reduce syscalls and directory re-opens in PUT and DELETE (#9264)
Co-authored-by: RustFS <hello@rustfs.com>
Co-authored-by: Hauser Zhi <housemecn@gmail.com>
2026-11-11 00:06:11 +00:00
dependabot[bot] 86c3395d07 chore(deps): bump the dependencies group across 1 directory with 12 updates (#9251)
Bumps the dependencies group with 12 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [async-nats](https://github.com/nats-io/nats.rs) | `0.51.0` | `0.52.0` |
| [tar-codec](https://github.com/astral-sh/astral-codec) | `0.0.15` | `0.0.16` |
| [tar-framing](https://github.com/astral-sh/astral-codec) | `0.0.15` | `0.0.16` |
| [flate2](https://github.com/rust-lang/flate2-rs) | `1.1.10` | `1.1.11` |
| [smallvec](https://github.com/servo/rust-smallvec) | `1.16.4` | `1.16.5` |
| [starshard](https://github.com/houseme/starshard) | `2.4.0` | `2.4.1` |
| [zip](https://github.com/zip-rs/zip2) | `9.3.0` | `9.4.1` |
| [dial9](https://github.com/dial9-rs/dial9) | `0.5.7` | `0.5.8` |
| [rustfs-mimalloc](https://github.com/houseme/rustfs-mimalloc) | `0.7.3` | `0.7.4` |
| [hotpath](https://github.com/hotpath-rs/hotpath) | `0.30.3` | `0.31.1` |
| [rustfs-mimalloc-sys](https://github.com/houseme/rustfs-mimalloc) | `0.7.3` | `0.7.4` |
| [toml](https://github.com/toml-rs/toml) | `1.2.0+spec-1.1.0` | `1.2.1+spec-1.1.0` |



Updates `async-nats` from 0.51.0 to 0.52.0
- [Release notes](https://github.com/nats-io/nats.rs/releases)
- [Commits](https://github.com/nats-io/nats.rs/compare/async-nats/v0.51.0...async-nats/v0.52.0)

Updates `tar-codec` from 0.0.15 to 0.0.16
- [Release notes](https://github.com/astral-sh/astral-codec/releases)
- [Commits](https://github.com/astral-sh/astral-codec/compare/v0.0.15...v0.0.16)

Updates `tar-framing` from 0.0.15 to 0.0.16
- [Release notes](https://github.com/astral-sh/astral-codec/releases)
- [Commits](https://github.com/astral-sh/astral-codec/compare/v0.0.15...v0.0.16)

Updates `flate2` from 1.1.10 to 1.1.11
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Changelog](https://github.com/rust-lang/flate2-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.1.10...v1.1.11)

Updates `smallvec` from 1.16.4 to 1.16.5
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.16.4...v1.16.5)

Updates `starshard` from 2.4.0 to 2.4.1
- [Changelog](https://github.com/houseme/starshard/blob/main/CHANGELOG.md)
- [Commits](https://github.com/houseme/starshard/compare/2.4.0...2.4.1)

Updates `zip` from 9.3.0 to 9.4.1
- [Release notes](https://github.com/zip-rs/zip2/releases)
- [Changelog](https://github.com/zip-rs/zip2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/zip-rs/zip2/compare/v9.3.0...v9.4.1)

Updates `dial9` from 0.5.7 to 0.5.8
- [Release notes](https://github.com/dial9-rs/dial9/releases)
- [Changelog](https://github.com/dial9-rs/dial9/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dial9-rs/dial9/compare/dial9-v0.5.7...dial9-v0.5.8)

Updates `rustfs-mimalloc` from 0.7.3 to 0.7.4
- [Release notes](https://github.com/houseme/rustfs-mimalloc/releases)
- [Changelog](https://github.com/houseme/rustfs-mimalloc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/houseme/rustfs-mimalloc/compare/v0.7.3...v0.7.4)

Updates `hotpath` from 0.30.3 to 0.31.1
- [Release notes](https://github.com/hotpath-rs/hotpath/releases)
- [Changelog](https://github.com/hotpath-rs/hotpath/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hotpath-rs/hotpath/compare/v0.30.3...v0.31.1)

Updates `rustfs-mimalloc-sys` from 0.7.3 to 0.7.4
- [Release notes](https://github.com/houseme/rustfs-mimalloc/releases)
- [Changelog](https://github.com/houseme/rustfs-mimalloc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/houseme/rustfs-mimalloc/compare/v0.7.3...v0.7.4)

Updates `toml` from 1.2.0+spec-1.1.0 to 1.2.1+spec-1.1.0
- [Commits](https://github.com/toml-rs/toml/compare/toml-v1.2.0...toml-v1.2.1)

---
updated-dependencies:
- dependency-name: async-nats
  dependency-version: 0.52.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: tar-codec
  dependency-version: 0.0.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: tar-framing
  dependency-version: 0.0.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: flate2
  dependency-version: 1.1.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: smallvec
  dependency-version: 1.16.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: starshard
  dependency-version: 2.4.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: zip
  dependency-version: 9.4.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: dial9
  dependency-version: 0.5.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: rustfs-mimalloc
  dependency-version: 0.7.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: hotpath
  dependency-version: 0.31.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: rustfs-mimalloc-sys
  dependency-version: 0.7.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: toml
  dependency-version: 1.2.1+spec-1.1.0
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Hauser Zhi <housemecn@gmail.com>
2026-11-11 00:04:47 +00:00
cxymds c9743f166e fix(s3): normalize leading-slash keys in bulk deletes (#9258)
* fix(s3): normalize leading-slash keys in bulk deletes

* test(s3): align bulk delete reserved-prefix guard expectations
2026-11-10 17:56:00 -05:00
Boyi (Elliot) Li 50707a610c test(utils,rio,checksums): add checksum SIMD equivalence and bench UTs (#9217)
Add unaligned-boundary SIMD vs scalar equivalence unit tests and on-demand (#[ignore]) utility-level throughput microbenchmarks across rustfs-utils, rustfs-rio, and rustfs-checksums:
- crates/utils/src/hash.rs: verify HighwayHasher SIMD (AVX2/SSE4.1/NEON) against highway::PortableHash scalar reference and contiguous hash_encode vs odd-chunked hash_encode_slices across unaligned tail boundaries; provide on-demand throughput microbenchmark.
- crates/rio/src/checksum.rs: verify SIMD-accelerated CRC32, CRC32C, and CRC64_NVME against bitwise scalar reference implementations across vector tail boundaries and measure utility-level throughput across all 10 BASE_CHECKSUM_TYPES.
- crates/checksums/src/lib.rs: verify streaming chunked update + finalize + headers() against one-shot digests across all 9 ChecksumAlgorithm variants for unaligned chunk and payload sizes, plus on-demand throughput microbenchmark.
2026-11-10 17:54:11 -05:00
Patrick Wicki ba3e0e1546 fix(admin): keep measured bucket usage when the overlay is unconverged (#9243)
The single-process usage overlay clears usage_snapshot_complete when it
holds counts it cannot reconcile with the persisted snapshot, for
example writes into a bucket that has no verified baseline. The admin
datausageinfo path then judged the overlaid document as structurally
incomplete and replaced it with an empty one, hiding every bucket the
scanner had measured. Because each write restarts that bucket's scanner
fence, a continuously written bucket kept the whole response empty
indefinitely.

Decide whether the persisted snapshot is usable before the overlay is
applied, and report overlay-induced incompleteness as a non-converged
snapshot instead, as is already done for buckets the scan never reached
(rustfs#5806).
2026-11-10 17:52:06 -05:00
Chris An 95e128336d ci: narrow checks for stable installation metadata (#9269)
* ci: narrow checks for stable installation metadata

* fix: reject release metadata marker collisions
2026-11-10 16:10:04 -05:00
Chris An 8b7bbdb17d chore(release): align installation references for 1.0.5 (#9268) 2026-11-10 14:48:02 -05:00
Chris An 2780bba67b ci: stage release builds and promote verified artifacts (#9265)
* ci: stage release builds and promote verified artifacts

* fix: preserve release package and channel retry contracts

* fix: align installation chart versions with sealed candidates
2026-11-10 13:15:05 -05:00
99 changed files with 7344 additions and 3906 deletions
+21 -26
View File
@@ -1,43 +1,38 @@
---
name: rustfs-release-publish
description: Publish a RustFS version through Console, release-branch CI, preview acceptance, human confirmation, and final publication. Use only for an explicit release request.
description: Publish a RustFS version through frozen Linux candidate acceptance, desktop completion, manifest approval, byte-preserving publication and post-release follow-up. Use only for an explicit release request.
---
# RustFS Release
Require the exact final SemVer target before version edits or publication. If unspecified, inspect current releases/tags and ask with concrete candidates; do not guess the channel or bump. A named bump type resolves to an exact version that still needs confirmation. Report the confirmed target verbatim; resolve any later mismatch before proceeding. Independent read-only preflight may continue.
Require the exact final SemVer target before version edits or publication. If unspecified, inspect releases/tags and ask with concrete candidates; do not guess the channel or bump. Reuse the user's confirmed target and authorized delivery scope. Independent read-only preflight may continue.
## Invariants
- Cargo stores `<target>`, never `-preview.N`. Before publication, installation defaults retain the previous available deliverable. Final and preview binaries embed their own tag names, so version/asset names differ despite identical source.
- Use annotated tags without `v`. Preview names are exactly `<target>-preview.<digits>`; choose the next unused iteration after fetching tags. Preview Releases are prereleases, never Latest; publish only versioned assets/exact Docker tags, never latest assets, R2, Helm, moving Docker channels, or installation/banner updates.
- `release/<target>` owns the candidate. Retain its history, backport only reviewed fixes, and never merge main wholesale or force-push it. Pin `PREVIEW_HASH` to its exact commit with complete passing CI. Preview and final tags both point there; never tag a later main/branch HEAD or insert another bump.
- Phase failures block dependents. Keep preview Releases until final publication and notes generation finish; CI removes Releases while retaining tags. Never move an existing preview or final tag.
- Cargo and every candidate executable contain the final product version. `<target>-preview.<run_number>` is an immutable candidate identifier, not a public tag or displayed binary version.
- Freeze the exact source, reviewed workflow recipe, toolchain, Console, CLI, dependency lock, previous stable version and validation plan. Newer unrelated main or Console releases do not change those inputs.
- Retain `release/<target>` history when a release branch is used; backport only reviewed fixes. Pin `CANDIDATE_SHA` to its exact CI-validated commit. Never force-push a release branch or move a public tag.
- Build Linux first. Complete field acceptance before macOS/Windows and packaging. Full certification and final acceptance are required before publication; a partial Linux PASS grants no release authority.
- Promote the accepted bytes. Final publication never recompiles binaries or rebuilds packages/images. Infrastructure retries reuse successful artifacts; changed source or inputs require a new candidate and affected acceptance.
- Phase failures block dependents. No latest/default-channel update before versioned delivery. Use the [pipeline runbook](../../../docs/operations/release-pipeline.md) for executable entry points, locks and recovery.
## Read by Phase
Read only the current phase's procedure; retain completed evidence across turns. Follow root verification, Git/PR, and authorization rules.
## Read by phase
| Phase | Procedure |
|---|---|
| 0 | Clean tree, fetch main/tags, verify GitHub access, then complete the mandatory [Console release gate](references/console-gate.md). A green Console build alone is insufficient. |
| 1–2 | [Prepare/reuse the release candidate and run exact-SHA branch CI](references/release-candidate.md). Source version changes use `rustfs-release-version-bump` with authorized delivery scope. |
| 3 | [Verify the preview build, assets, channels, and notes](references/preview-release.md). |
| 4–5 | [Run the downloaded binary, Console CRUD, and latest-rc matrix](references/preview-acceptance.md). Every check must pass. |
| Confirmation | Complete the gate below and end the turn. |
| 6 | After fresh confirmation, [publish and verify the final tag](references/final-release.md) at `PREVIEW_HASH`. |
| 7 | [Reconcile Security Advisories, milestones, installation references, and the website; reconcile main and remove the completed release branch](references/post-release-updates.md) only after complete publication. |
| 0 | Clean tree, fetch main/tags, verify access, then complete the mandatory [Console gate](references/console-gate.md). |
| 1–2 | [Prepare the exact source and run branch CI](references/release-candidate.md). Version edits use `rustfs-release-version-bump`. |
| 3 | [Build and verify the Linux candidate](references/preview-release.md). |
| 4–5 | [Run artifact, field, Console and rc acceptance](references/preview-acceptance.md), then complete desktop/package/image/chart validation. |
| Confirmation | Report the complete manifest and obtain the confirmation below. |
| 6 | [Publish the accepted manifest](references/final-release.md). |
| 7 | [Reconcile advisories, milestones, installation references, website and release branch](references/post-release-updates.md). |
## Human Confirmation
## Human confirmation
After Phases 3–5 pass, recheck that the release branch still equals `PREVIEW_HASH` and that the current Console asset tag/digest match the preview's recorded build inputs. Report the target, branch, preview tag/hash, branch CI run, preview Release URL, accepted Console asset identity/result, and rc matrix. Explicitly ask whether to publish the final tag and end the turn without creating/pushing it.
After all acceptance passes, report the target, source SHA, source CI run, Linux and completion run URLs, final manifest SHA-256, accepted Console/CLI identities, compatibility/performance results and backlog report. Explicitly ask whether to publish this exact manifest and end the turn without dispatching publication.
Only a new affirmative user reply to that exact report authorizes Phase 6. The original release request, earlier confirmation, silence, or automated follow-up does not count. Confirmation binds the target, preview tag, hash, and accepted build inputs; a changed value or repeated/failed acceptance cycle invalidates it. Revalidate and obtain fresh confirmation before continuing.
Only a new affirmative reply to that exact report authorizes Phase 6. An earlier general release request, silence or automatic continuation does not count. Reuse confirmation for retries of that same manifest; do not ask again merely because a destination failed. A changed source, input, manifest or acceptance result invalidates it and requires renewed acceptance/confirmation.
## Recovery
## Recovery and reporting
- Before the final tag exists, land fixes on main, backport to the candidate, verify target versions, and restart branch CI/acceptance. Use a new preview iteration once a preview was tagged; main may advance independently.
- If only an external build input changed, record it, create a new preview iteration even when the source SHA is unchanged, and repeat the affected build and artifact acceptance before fresh confirmation. Reuse source checks whose inputs remain valid.
- Once the final tag exists, retry publication jobs for that exact tag. A source fix requires a newly confirmed target, not retagging or a newer preview. Phase 7 retries recheck artifact availability and superseding releases.
- If abandoned after the bump merged, report the unpublished version on main and record whether to revert or let the next release overwrite it. Deleting unpublished candidate branches or preview tags requires an explicit decision. Completed release branches follow Phase 7 cleanup; keep release tags.
Report completed phase results and incomplete work with evidence: Console previous/latest tags, release need, hash and run/Release URLs; target, branch CI/SHA, previews and hash, confirmation state; preview/final assets, latest-channel state, Console/rc results and cleanup; advisory reconciliation results and pending disclosure decisions; both milestone states, installation PR checks/merge state, and website text/link/PR/deployment state. Keep artifact publication separate from follow-up delivery; do not call an open PR a live update. Report deviations and their authorization.
Resume failed infrastructure jobs without rebuilding successful units. Fix product defects on main, backport when needed, and start a new candidate. After a final tag exists, retry the same bytes or use a new patch version; never move the tag. Do not conflate implemented, CI-passed, field-accepted, approved, published and post-release-complete states. Report incomplete work and deviations with evidence; a tag or green build alone is not a completed release.
@@ -4,7 +4,7 @@ Read during Phase 0, before changing RustFS version files or tags. Follow the pa
### Console release gate
Complete this gate before changing any RustFS version file or creating any RustFS tag. RustFS `build.yml` downloads the asset returned by `repos/rustfs/console/releases/latest`, so a successful Console build alone is insufficient.
Complete this gate before changing any RustFS version file or creating any RustFS tag. Candidate creation resolves `repos/rustfs/console/releases/latest` once and freezes the exact asset for every build, so a successful Console build alone is insufficient. After freezing, later Console releases do not silently change or invalidate the accepted inputs.
1. Read the latest published Console tag and compare it with Console `main`:
@@ -1,22 +1,13 @@
# Final Release Publication
# Final release publication
Read only after fresh user confirmation of the preview acceptance report. The [parent confirmation gate](../SKILL.md#human-confirmation) applies.
Read only after fresh confirmation of the complete candidate report. The [parent confirmation gate](../SKILL.md#human-confirmation) applies.
## Phase 6 — Publish the final tag on the validated commit
Dispatch `release-publish.yml` on main with the successful completion run ID and approved `release-manifest.json` SHA-256. Do not create the tag manually. The workflow revalidates source, bytes and trusted certification, then creates the annotated final tag at `CANDIDATE_SHA`. A recipe change fails closed and requires a new candidate; unrelated main changes do not require a rebuild.
No second source version bump. Before creating the final tag, recheck the remote `release/<target>` HEAD equals `PREVIEW_HASH` and the current Console release tag/asset digest still match the accepted preview inputs. If either changed after confirmation, the approval no longer applies; validate the changed inputs with a new preview and obtain fresh confirmation. The final tag goes on the exact commit the preview validated:
Require both versioned publication and channel publication jobs to succeed. Verify the final tag's commit and manifest identity; all six platform ZIPs and compatibility aliases; DEB/RPM, checksums, SBOM, provenance and evidence bundle; both multi-architecture image variants in Docker Hub, GHCR and Quay; and the accepted Helm chart in the chart repository. Download a host-compatible final artifact and confirm its digest is exactly the accepted candidate digest and its version/source output agrees.
```bash
git fetch origin --tags
git rev-parse "<preview-tag>^{commit}" # must equal PREVIEW_HASH — abort if not
git tag -a "<target>" -m "Release <target>" "$PREVIEW_HASH"
git push origin "<target>"
```
For stable releases verify GitHub Latest, R2 download aliases and `latest.json` unless a newer stable release already owns those channels. Prereleases must remain GitHub prereleases and leave stable pointers unchanged. Verify generated release notes compare against the frozen previous stable release, excluding internal previews.
- CI rebuilds from the same source with a different tag name, so the binary self-reports `<target>`. Other inputs can change: `build.yml` reads Console `releases/latest` during each build. Compare the actual Console tag, URL, and verified digest in every final build job with the accepted preview record, and inspect available build provenance for other relevant changes. A pre-tag check cannot prevent a later input change. On a mismatch or missing evidence, repeat affected acceptance against the final artifacts and report the limitation; failed acceptance leaves release validation incomplete even if publication already succeeded. Never move the final tag to conceal a failure.
- Verify the final tag's complete publication path: all matrix and release jobs green; `gh release view "<target>"` shows the full versioned and `-latest` asset set plus checksums, SBOM, and provenance; Docker and Helm workflows succeed; `latest.json` points to `<target>`. A stable target must have `isPrerelease=false` and `isLatest=true`. An alpha/beta/rc target must have `isPrerelease=true`; GitHub does not permit prereleases to be Latest, but the project `latest.json` still advances to the final non-preview target.
- Verify the final Release body contains `## What's Changed` and a Full Changelog link. When `PREVIOUS_DELIVERABLE` exists, the link MUST be `https://github.com/rustfs/rustfs/compare/<PREVIOUS_DELIVERABLE>...<target>` and the baseline MUST equal the preview Release baseline; for example, both `1.0.0-beta.12-preview.1` and `1.0.0-beta.12` compare from `1.0.0-beta.11`.
- Verify the preview cleanup: `cleanup-preview-releases` must succeed and `gh release view "<preview-tag>"` must then report `release not found` for every preview iteration of this target. Each surviving preview tag must still resolve to its own recorded commit; the accepted preview resolves to `PREVIEW_HASH`. If the job failed, delete the leftover Releases manually with `gh release delete "<preview-tag>" --yes` and report it.
- Verify a host-compatible final artifact's checksum and `./rustfs --version`; it must report `<target>` and the expected source SHA. This identity check does not replace affected acceptance when build inputs changed.
On infrastructure failure retry the same workflow/seal; matching destinations are reused. Never clobber an asset, force a tag, rebuild the image or silently accept a changed digest. A source fix after the final tag requires a new patch version. Cross-registry writes are not atomic: report the unfinished destination rather than claiming publication complete.
Never pass `--cleanup-tag` when removing preview Releases. Remove them manually only if `cleanup-preview-releases` failed, after final publication and notes generation. Record deleted Releases and surviving tags.
New candidates create no preview Releases to clean up. For a legacy preview release, preserve its tag and remove its Release only after final delivery and notes are verified. Do not pass `--cleanup-tag`. Complete the [post-release follow-up](post-release-updates.md) separately.
@@ -24,7 +24,7 @@ Reconcile advisories after every completed final release, including Console-only
1. Re-read the published deliverable state before edits or a retry. If a newer release has superseded this target, do not downgrade installation references or the website announcement. Report the superseding release and leave those defaults intact.
2. Invoke `rustfs-release-version-bump` with stage `post-release`, the exact target, and the authorized delivery scope. Prepare the follow-up on current main; do not edit or move either validated release tag. Cargo versions may already be preparing the next target and must remain untouched.
3. Verify the exact Docker image manifest, Release download links, RPM source archive, and Helm repository entry before publishing the installation PR. The chart workflow overrides chart/app versions from the final tag, pins the source checkout to that release, and waits for its default Docker image before packaging.
3. Verify the exact Docker image manifest, Release download links, RPM source archive, and Helm repository entry before publishing the installation PR. The chart is packaged and tested before sealing; publication uploads those same bytes only after versioned Docker images are available.
4. Report the installation PR and its merge state separately from the completed release. Reuse existing merge authorization and repository checks; do not infer new merge authority from this reference.
## rustfs.com top banner
@@ -54,4 +54,4 @@ Use the existing announcement configuration from [website PR #112](https://githu
1. Compare the exact published tag with current main. Audit every release-side commit since their merge base, including inherited backports. An ahead count is a history difference, not proof of missing changes. Use ancestry and patch equivalence, then inspect current code and regression evidence for squashed, adapted, superseded, or reverted changes. Record each commit as present, superseded, release-only, or missing; a positive `git cherry` result alone does not prove a missing fix.
2. Bring missing shared changes to main through a reviewed PR and the required checks before closing the follow-up. Preserve main's later changes; decide version bumps and pinned Console configuration separately. If a history merge is needed, use a merge commit and verify the release commit is an ancestor of main; squash does not establish that ancestry. Do not merge main into the published release branch or move its tags. Report unresolved changes instead of claiming reconciliation is complete.
3. After publication and authorized follow-ups are complete, delete the remote `release/<target>` branch. Immediately before deletion, verify its HEAD still equals the published tag's commit and no open PR targets or originates from it. Preserve a branch with newer work or open PRs and report the blocker. Use a deletion with an expected-old-SHA check to avoid removing a concurrently advanced branch.
4. Verify the remote branch is absent and the final and preview tags still resolve to their recorded commits. Remove a local release branch only if it is task-owned, unchecked out, and has no unpreserved work. Record cleanup separately from artifact publication; an explicitly authorized deletion before reconciliation does not make unresolved main changes complete.
4. Verify the remote branch is absent and the final tag and any legacy preview tags still resolve to their recorded commits. Remove a local release branch only if it is task-owned, unchecked out, and has no unpreserved work. Record cleanup separately from artifact publication; an explicitly authorized deletion before reconciliation does not make unresolved main changes complete.
@@ -1,10 +1,10 @@
# Preview Artifact Acceptance
# Candidate Artifact Acceptance
Read after Phase 3 succeeds. Complete every check below before the parent skill's manual confirmation gate. These checks cover the downloaded artifact, embedded Console, and latest rc client.
Read after Phase 3 succeeds. Complete every check below before the parent skill's manual confirmation gate. These checks cover the downloaded artifact, embedded Console, and the rc release frozen in the candidate inputs.
## Phase 4 — Run the artifact locally, verify the console
Use a unique task-owned scratch directory. Select the versioned archive matching the host OS, architecture, and runtime, download its checksum file, and verify the archive before extraction. Do not substitute a different build when a matching artifact is unavailable; report that acceptance gap. The shell example below assumes the verified executable is at `RUSTFS_BINARY` (`rustfs.exe` on Windows), `SCRATCH` is the task directory, and `S3_PORT` and `CONSOLE_PORT` are distinct available ports. Use equivalent native commands when the shell differs.
Use a unique task-owned scratch directory. Select the versioned archive matching the acceptance host, and verify it against the sealed manifest before extraction. Before desktop completion, use the Linux acceptance environment rather than rebuilding for the local desktop. Do not substitute a different build when a matching artifact is unavailable; report that acceptance gap. The shell example below assumes the verified executable is at `RUSTFS_BINARY` (`rustfs.exe` on Windows), `SCRATCH` is the task directory, and `S3_PORT` and `CONSOLE_PORT` are distinct available ports. Use equivalent native commands when the shell differs.
Bind both listeners to loopback. Choose task-only credentials, set loopback proxy exclusions for the server/client processes, and record the server PID. A port probe does not reserve a port; retry startup with fresh ports on a bind conflict rather than connecting to an existing service.
@@ -24,16 +24,16 @@ PREVIEW_SERVER_PID=$!
Checks (all must pass):
- `"$RUSTFS_BINARY" --version` reports the preview tag name and the short SHA of `PREVIEW_HASH`. Reporting `<target>` without the `-preview.N` suffix means the build did not embed the tag — treat as FAIL and investigate before proceeding.
- `"$RUSTFS_BINARY" --version` reports the exact final `<target>` and full `CANDIDATE_SHA`. A preview suffix, another version or another source is a failure.
- Wait with a bounded readiness deadline while checking the recorded server process remains alive; `curl --noproxy '*' -fsS "$S3_ENDPOINT/health/ready"` must return ready from this process's listener.
- Startup log shows the embedded console being served (this was the regression that `fix(release): require embedded console assets` guards).
- Open `CONSOLE_ENDPOINT` in the browser and log in with the task credentials. The dashboard must render without JS console errors; create a bucket, upload a file, download it back (byte-identical), and delete the object and bucket. Keep the server running for Phase 5.
## Phase 5 — Validate with the latest rc client
## Phase 5 — Validate with the pinned rc client
`rc` is the RustFS CLI client from <https://github.com/rustfs/cli>.
- Resolve the latest release with `gh api repos/rustfs/cli/releases/latest --jq .tag_name`; verify the local client's version or download and verify a host-compatible client into the task directory. Set `RC_BIN` to that executable and record its version/digest. Avoid changing the user's installed client for this check.
- Read the CLI release and asset digests from `inputs.json`; verify the local client's version or download a host-compatible asset from that exact release into the task directory. If the host is a desktop, resolve its asset within the same pinned CLI release and record the digest; do not independently follow `latest`. Set `RC_BIN` to that executable and record its version/digest. Avoid changing the user's installed client for this check.
- Inspect the client's current help for an isolated configuration option and use it when supported. Otherwise choose a unique `PREVIEW_ALIAS` and confirm it is absent before creating it; never overwrite an existing alias. Use task-owned bucket/user names and input/output files. The names below are safe only inside the newly started isolated server.
- Point the client at `S3_ENDPOINT` with loopback proxy exclusions and run the command matrix, recording PASS/FAIL per command:
@@ -60,3 +60,9 @@ cmp <local-file> "$SCRATCH/out"
```
- Any FAIL blocks acceptance. On success or failure, stop only the recorded task process and remove only task-created aliases/data after retaining the needed failure evidence. Do not erase an existing client configuration or unrelated server data.
## Field acceptance and desktop completion
Before accepting the Linux manifest, record Scanner/Heal, multi-pool, memory, major features, high-concurrency 503, performance and previous-version integrity/upgrade results in backlog. Preserve the release-specific agreed limits and baseline; a PASS label without artifact identities or test results does not satisfy acceptance.
Dispatch `release-complete.yml` only after that report passes, using the exact Linux manifest digest and backlog URL. Require native macOS/Windows checks, package extraction, image startup/scans, chart tests and complete certification to pass. Complete any desktop Console/client checks applicable to the release against these same archives. Record the completion run and final manifest SHA-256 before entering the parent confirmation gate. Partial Linux acceptance does not authorize final publication.
@@ -1,25 +1,9 @@
# Preview Release Verification
# Linux candidate verification
Read for Phase 3 after the candidate passed exact-SHA CI. The [parent release invariants](../SKILL.md) apply.
Read for Phase 3 after the source passed exact-SHA CI. The [parent invariants](../SKILL.md) apply; commands and recovery live in the [pipeline runbook](../../../../docs/operations/release-pipeline.md).
## Preview channels
Dispatch `release-candidate.yml` on main with the confirmed final version and `CANDIDATE_SHA`. Record the run ID, frozen `inputs.json`, every Linux archive digest and `linux-manifest.json` SHA-256. Require the four Linux builds, native startup/S3 checks, previous-release upgrade matrix and Linux seal to succeed. Missing, skipped, cancelled or failed required work is incomplete acceptance.
- Use `<target>-preview.N` for every target, e.g. `1.0.0-beta.10-preview.3` or `1.1.0-preview.1`.
- The canonical suffix is exactly `-preview.<digits>`. `build.yml` recognizes it before alpha/beta/rc classification and routes it to the preview-only path; any other tag containing `-preview` fails closed instead of being treated as a release.
- A preview Release MUST be published with `isPrerelease=true` and `isLatest=false`. Any `*-latest` preview asset or preview-triggered `latest.json`, R2, Helm, or moving Docker channel tag is a pipeline failure. Docker images may use the exact preview version tag, including its existing variant suffixes.
- Preview Releases are cleaned up by the `cleanup-preview-releases` job after `publish-release` succeeds for the deliverable tag. It deletes every Release whose tag is exactly `<target>-preview.<digits>` and never passes `--cleanup-tag`, so the tags survive.
Verify the frozen Console identity equals the one accepted in Phase 0. Every matrix job consumes that same digest and exact toolchain. The binary must report the final product version and source commit; its candidate ID is separate. No public tag, Release, registry tag, R2 release path, Helm repository or latest pointer should change during this stage.
## Phase 3 — CI and preview Release verification
- Find and watch the tag build: `gh run list --workflow build.yml --branch "<preview-tag>" --limit 1` then `gh run watch <run-id>`. Every build matrix target must succeed (linux x86_64/aarch64 × musl/gnu, macos-aarch64, windows-x86_64).
- Confirm the preview tag resolves to the Phase 2 CI-validated `PREVIEW_HASH`. If the release branch advances during acceptance, repeat Phases 2–5 with a new preview tag.
- Confirm the Release publication jobs (`create-release`, `upload-release-assets`, and `publish-release`) succeed while `update-latest-version` is skipped.
- Record the inputs actually consumed by each build job: source SHA and the embedded Console release tag, asset URL, and verified SHA256 from the download logs. The workflow reads Console `releases/latest` separately for each build; confirm the matrix used the Console asset accepted in Phase 0. A missing identity or differing asset blocks acceptance until resolved. Retain available toolchain and build provenance with the artifact checksums; the same source SHA alone does not establish equivalent builds.
- Verify `gh release view "<preview-tag>" --json isPrerelease,assets,url`: `isPrerelease` must be `true`, and the Release must contain all 6 versioned platform zips, checksums, SBOM, and provenance with no `-latest` assets. Confirm `gh api repos/{owner}/{repo}/releases/latest --jq .tag_name` does not return `<preview-tag>`.
- Record `PREVIOUS_DELIVERABLE`, selected from published Releases by `publishedAt` after excluding the current tag and every `-preview.N` tag. Verify `gh release view "<preview-tag>" --json body --jq .body` contains `## What's Changed` and, when `PREVIOUS_DELIVERABLE` exists, `**Full Changelog**: https://github.com/rustfs/rustfs/compare/<PREVIOUS_DELIVERABLE>...<preview-tag>`. For a repository with no previous deliverable, verify a Full Changelog link exists and record the GitHub baseline fallback.
- Confirm Helm is skipped. If preview Docker images are published, confirm they use only exact preview tags (and variant suffixes), with no `latest`, `alpha`, `beta`, or `rc` channel updates. Preview validation covers the built RustFS binaries, embedded console, and rc compatibility; it does not authorize advancing any installation default.
## Release notes
- When a previous deliverable exists, GitHub Release notes for the preview and final tags MUST use it as their shared comparison baseline: the most recently published non-preview Release before the target. Internal `-preview.N` Releases are explicitly excluded from that selection, even when they point at the same commit as the final tag — cleanup runs after the notes are generated, so the preview Release is still present and would otherwise be picked as the baseline. If no previous deliverable exists, omit `previous_tag_name` and record that GitHub's default baseline fallback was used.
- Generated Release notes carry a workflow-management marker so retries can repair them. Before manually curating a generated body, remove that marker; unmarked non-placeholder notes are preserved by later workflow runs.
A green Linux run is ready for field acceptance, not publication. The partial certification must have `publish_allowed: false`. Retain the exact Linux manifest for the next stage; a new source/build-input candidate must not inherit the old approval. A newer unrelated main commit does not invalidate this evidence.
@@ -16,22 +16,13 @@ Read for Phases 1–2. The [parent release invariants](../SKILL.md) and Console
- Only when neither the release branch, an unfinished preview tag, nor current main already has `<target>`: if main is at a later version, select and verify a historical commit containing `<target>` rather than downgrading main; block if none exists. Otherwise invoke the `rustfs-release-version-bump` skill with stage `prepare`, the final `<target>` (NOT a preview version), and full GitHub flow (commit/push/PR).
- If a bump PR was needed, get it merged into main and record its merge commit. For a new release branch when main already has `<target>`, identify the main commit that introduced those versions. Phase 2 chooses a release candidate containing that commit; an existing release branch keeps its own history and a newer main HEAD is not required.
Do not wait for the moving latest main HEAD to become green. The release-branch CI in Phase 2 validates the chosen commit before a preview tag is created.
Do not wait for the moving latest main HEAD to become green. The release-branch CI in Phase 2 validates the chosen commit before a candidate build is dispatched.
## Phase 2 — Pin and validate the release candidate, then publish the preview tag
## Phase 2 — Pin and validate the release source
- Use `release/<target>`. On the first attempt, create it from a selected main commit containing the version bump and intended changes; it need not be the latest main HEAD. On a restart, retain the existing branch and backport only reviewed fixes from main. Verify the branch's Cargo versions still equal `<target>`. If adopting an existing preview, run this branch CI before continuing its acceptance; reuse the tag only when it points to the validated SHA.
- The current `ci.yml` runs automatically on main pushes, not release-branch pushes. Dispatch it explicitly with `gh workflow run ci.yml --ref "release/<target>"`. Locate that dispatch run, verify its `head_sha` equals the release branch's remote HEAD, and require the full expected CI job set to pass, including `Quick Checks`, `Test and Lint`, and `End-to-End Tests (full merge gate)`. Treat a missing or skipped required lane as incomplete. A successful PR check or CI run for a different SHA does not qualify. If the branch advances or a run is cancelled, validate the new SHA again. Do not tag a candidate with failing or incomplete CI.
- Use `release/<target>`. On the first attempt, create it from a selected main commit containing the version bump and intended changes; it need not be the latest main HEAD. On a restart, retain the existing branch and backport only reviewed fixes from main. Verify the branch's Cargo versions still equal `<target>`. A legacy preview can anchor source selection, but its tag and old artifacts do not constitute a candidate in the new protocol; freeze and verify a new candidate after the release recipe is aligned.
- The current `ci.yml` runs automatically on main pushes, not release-branch pushes. Dispatch it explicitly with `gh workflow run ci.yml --ref "release/<target>"`. Locate that dispatch run, verify its `head_sha` equals the release branch's remote HEAD, and require the full expected CI job set to pass, including `Quick Checks`, `Test and Lint`, and `End-to-End Tests (full merge gate)`. Treat a missing or skipped required lane as incomplete. A successful PR check or CI run for a different SHA does not qualify. If the branch advances or a run is cancelled, validate the new SHA again. Do not dispatch a candidate with failing or incomplete CI.
- Prefer protecting `release/*` against force pushes and deletion and requiring reviewed backport PRs. Check live rulesets rather than assuming the main ruleset covers release branches; report a missing release-branch rule as a gap.
- Set `PREVIEW_HASH` to the CI-validated release-branch commit and recheck the remote branch HEAD immediately before tagging. Report the branch, commit, and CI run URL; do not derive `PREVIEW_HASH` from a later `origin/main` fetch. When adopting an existing preview tag at this SHA, skip tag creation and continue Phase 3.
```bash
git tag -a "<preview-tag>" -m "Release <preview-tag>" "$PREVIEW_HASH"
git push origin "<preview-tag>"
```
Pushing the tag triggers `.github/workflows/build.yml` ("Build and Release"); `docker.yml` chains off it via `workflow_run`.
The preview run builds versioned artifacts and publishes them in a GitHub prerelease. Docker may publish exact preview image tags. Latest-channel, R2, and Helm publication must be skipped; installation references and the website announcement stay unchanged.
On a restart after a backport, re-run release-branch CI before setting the new `PREVIEW_HASH`. Use the next unused preview iteration when the validated commit differs from an existing preview tag; never move an existing tag.
- Set `CANDIDATE_SHA` to the CI-validated release-branch commit. Record the branch, exact commit and CI run URL. The new candidate protocol starts from a commit whose release recipe matches the reviewed main dispatcher; backport the recipe before freezing an older release branch if needed.
- Dispatch the Linux candidate as described in [candidate verification](preview-release.md). Do not create a preview or final tag here. The final executable already contains `<target>`; publication later promotes those same bytes.
- After a source or build-input change, run the affected source checks and create a new candidate. Reuse the frozen candidate after an infrastructure failure. Do not restart acceptance for unrelated main advances.
@@ -4,18 +4,18 @@ description: Prepare an exact RustFS Cargo version or align installation referen
---
# Release Version Files
Require an exact target; ask if missing or ambiguous. Reject targets containing `-preview`: those are tag-only. Infer `post-release` only from an explicit request or parent workflow; otherwise use `prepare`. Delivery defaults to local edit/verify; commit, push, PR, and merge follow existing authorization and [Git rules](../../references/pull-requests.md).
Require an exact target; ask if missing or ambiguous. Reject targets containing `-preview`: those identify candidates, not product versions or new public tags. Infer `post-release` only from an explicit request or parent workflow; otherwise use `prepare`. Delivery defaults to local edit/verify; commit, push, PR, and merge follow existing authorization and [Git rules](../../references/pull-requests.md).
| Stage | Files and constraints |
|---|---|
| `prepare` | Only `Cargo.toml` and `Cargo.lock`: workspace package, internal dependency, and workspace member versions become `<target>`. External dependencies stay unchanged. |
| `post-release` | Only `README.md`, `README_ZH.md`, `flake.nix`, `helm/rustfs/Chart.yaml`, and `rustfs.spec`. Cargo may already target the next release; leave it untouched. |
Before installation edits, verify the exact non-preview Release, downloadable assets/source archive, pullable `rustfs/rustfs:<target>` image manifest, and published Helm chart. Block on missing artifacts. Do not downgrade defaults if a newer deliverable superseded the target. Preview preparation retains the previous published installation references; follow-up commits never replace either validated tag.
Before installation edits, verify the exact non-preview Release, downloadable assets/source archive, pullable `rustfs/rustfs:<target>` image manifest, and published Helm chart. Block on missing artifacts. Do not downgrade defaults if a newer deliverable superseded the target. Candidate preparation retains the previous published installation references; follow-up commits never replace validated tags or sealed artifacts.
Packaging policy:
- Docker image tags have no `v` prefix. Derive chart/app versions with `scripts/helm_chart_version.sh <target>` (`beta.N -> 0.N.0`; other versions unchanged). Helm CI derives versions from the final tag, so no early chart bump is required.
- Docker image tags have no `v` prefix. Derive chart/app versions with `scripts/helm_chart_version.sh <target>`; both equal the final product SemVer, including prerelease suffixes. Helm CI uses the frozen candidate version before sealing, so no early installation chart bump is required. Previously published chart versions remain unchanged.
- RPM `Version` is numeric; `Release` is the prerelease suffix or `1` for stable. Expanded `Source0` and the unpack directory must resolve to the exact target including its suffix. Add the changelog entry using current git identity, date, and target; never copy an example identity/date.
- Changing these mappings requires explicit confirmation.
+1 -1
View File
@@ -71,7 +71,7 @@ s3s-footprint-check: ## Check the s3s dependency footprint ratchet stays frozen
./scripts/check_s3s_footprint.sh
.PHONY: test-count-ratchet-check
test-count-ratchet-check: ## Check no workspace crate lost tests or gained #[ignore] against the committed baseline
test-count-ratchet-check: ## Check no workspace crate fell below its test floor or rose above its #[ignore] ceiling in the committed baseline
@echo "🧪 Checking per-crate test-count ratchet..."
./scripts/check_test_count_ratchet.sh
+13
View File
@@ -22,6 +22,14 @@ TEST_THREADS ?= 1
# differ from CI and .config/nextest.toml test-groups will NOT apply.
.PHONY: script-tests
.PHONY: release-package-tests
release-package-tests: ## Run release package integration on Linux with fpm, dpkg and rpm installed
$(RUSTFS_PYTHON_BIN) ./scripts/test_release_packages.py
.PHONY: release-image-tests
release-image-tests: ## Run OCI integration with Docker, Go and skopeo; set RELEASE_IMAGE_VARIANT and RELEASE_IMAGE_OUTPUT
$(RUSTFS_PYTHON_BIN) ./scripts/test_release_images.py
script-tests: ## Run shell script tests
@echo "Running script tests..."
trap 'status=$$?; printf "ERROR: script-tests failed (exit %s): %s\n" "$$status" "$$BASH_COMMAND" >&2; exit "$$status"' ERR
@@ -53,12 +61,15 @@ script-tests: ## Run shell script tests
$(RUSTFS_PYTHON_BIN) ./scripts/test_check_doc_paths.py
./scripts/check_embedded_secrets.sh --self-test
./scripts/check_s3s_footprint.sh --self-test
./scripts/test_check_test_count_ratchet.sh
$(RUSTFS_PYTHON_BIN) ./scripts/check_test_wiring.py --self-test
$(RUSTFS_PYTHON_BIN) ./scripts/codemods/s3_error_to_s3_types.py --self-test
$(RUSTFS_PYTHON_BIN) ./scripts/test_e2e_binary.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_download_release_asset.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_reference_binary.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_inspect_btf_prototypes.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_release_build_workflow.py
$(RUSTFS_PYTHON_BIN) -m unittest discover -s scripts/release -p "test_*.py"
$(RUSTFS_PYTHON_BIN) ./scripts/test_release_glibc.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_upgrade_workflow.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_migration_gate_evidence.py
@@ -79,8 +90,10 @@ script-tests: ## Run shell script tests
$(RUSTFS_PYTHON_BIN) ./scripts/test_inspect_validation_runner.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_retain_validation_evidence.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_ci_timing_report.py
$(RUSTFS_PYTHON_BIN) ./scripts/test_junit_diff.py
$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_report_compat.py
$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_runner_tools.py
$(RUSTFS_PYTHON_BIN) ./scripts/s3-tests/test_reference_parity.py
bash -n ./scripts/validate_object_data_cache_cold_stampede.sh
$(RUSTFS_PYTHON_BIN) ./scripts/check_object_data_cache_follower_samples.py --self-test
./scripts/validate_object_data_cache_cold_stampede.sh --self-test
+20
View File
@@ -0,0 +1,20 @@
# Pinned RustFS release used as the "reference" target of parity lanes
# (.github/workflows/reference-parity.yml). This is the single place that names
# the reference release; .github/actions/reference-binary resolves, downloads and
# verifies it, and fails on a missing pin, a missing asset or a digest mismatch.
#
# Bumping the reference is a reviewed event: every per-test difference between
# the candidate and the reference is measured against this exact build.
#
# Each sha256 is the value published in the release's SHA256SUMS asset, which
# matches the digest GitHub reports for the same asset. The action checks both.
schema = 1
repository = "rustfs/rustfs"
version = "1.0.5"
tag = "1.0.5"
[platforms.linux-x86_64-gnu]
asset = "rustfs-linux-x86_64-gnu-v1.0.5.zip"
sha256 = "5addae2f74756bf80104d0975e6347cda843bc2daffdfcd1f0cb80118b95cbd0"
executable = "rustfs"
+24
View File
@@ -0,0 +1,24 @@
# Targets of the s3-tests reference parity lane (.github/workflows/reference-parity.yml).
#
# Exactly one target uses `binary = "reference"` (the release pinned in
# .config/reference-binary.toml). Every other target uses `binary = "candidate"`
# (the binary built from the branch under test) and is compared per node id
# against the reference. All targets run the same harness, test list and server
# configuration; `env` adds only the target's own RUSTFS_* settings.
#
# Adding a target is one more [[targets]] table, for example:
#
# [[targets]]
# name = "candidate-gateway"
# binary = "candidate"
# env = { RUSTFS_S3_STACK = "gateway" }
schema = 1
[[targets]]
name = "reference"
binary = "reference"
[[targets]]
name = "candidate-legacy"
binary = "candidate"
+12
View File
@@ -0,0 +1,12 @@
# Tolerated per-test differences of the s3-tests reference parity lane.
#
# One entry per line, four fields separated by " | ":
#
# <target> | <pytest node id> | <tracking issue URL> | <reason>
#
# <target> is a candidate name from .config/reference-parity-targets.toml, the
# node id is exactly as the lane prints it (s3tests/functional/test_s3.py::...),
# and the issue URL is https://github.com/rustfs/<repo>/issues/<number>.
#
# Same semantics as scripts/junit_diff.py --allow: an entry whose difference no
# longer occurs is stale and fails the lane, so remove it when it is fixed.
+82 -24
View File
@@ -1,44 +1,93 @@
{
"schema_version": 1,
"plan_id": "rustfs-release-validation",
"description": "Versioned certification plan for every public RustFS release exit (rustfs/backlog#2662, R01). preview, prerelease and stable all require this explicit plan. Scripts/release/release_gate.py fail-closes publication on any missing, duplicate, stale, substituted or non-PASS required item.",
"release_types": ["preview", "prerelease", "stable"],
"description": "Certification for immutable candidates. All shipped platforms require native identity and startup; stable and prerelease require the pinned upgrade matrix. Linux-only acceptance never authorizes publication.",
"release_types": [
"preview",
"prerelease",
"stable"
],
"targets": [
{"os": "linux", "arch": "x86_64", "abi": "musl"},
{"os": "linux", "arch": "aarch64", "abi": "musl"},
{"os": "linux", "arch": "x86_64", "abi": "gnu"},
{"os": "linux", "arch": "aarch64", "abi": "gnu"},
{"os": "macos", "arch": "aarch64", "abi": "none"},
{"os": "windows", "arch": "x86_64", "abi": "msvc"}
{
"os": "linux",
"arch": "x86_64",
"abi": "musl"
},
{
"os": "linux",
"arch": "aarch64",
"abi": "musl"
},
{
"os": "linux",
"arch": "x86_64",
"abi": "gnu"
},
{
"os": "linux",
"arch": "aarch64",
"abi": "gnu"
},
{
"os": "macos",
"arch": "aarch64",
"abi": "none"
},
{
"os": "windows",
"arch": "x86_64",
"abi": "msvc"
}
],
"targets_basis": "Platform matrix of .github/workflows/build.yml (prepare-platform-matrix). gnu and macOS targets build with the pyroscope feature; musl and Windows targets build with no extra features.",
"features": ["pyroscope"],
"trusted_workflows": [
{"repository": "rustfs/s3chaos", "workflow_id": "release-gate.yml"}
"features": [
"pyroscope"
],
"trusted_workflows_basis": "The shared evidence envelope producer lives in rustfs/s3chaos (G01, fault/release_gate.rs) and its plan/live split workflow is .github/workflows/release-gate.yml there. Only results from these identities may enter a release certification; PR-uploaded JSON never gains release authority. R02 must extend this list when additional verifier workflows land, never widen it to untrusted sources.",
"trusted_workflows": [
{
"repository": "rustfs/rustfs",
"workflow_id": "release-candidate.yml"
},
{
"repository": "rustfs/rustfs",
"workflow_id": "release-complete.yml"
}
],
"trusted_workflows_basis": "Receipts are produced by the reviewed main workflows and fetched by exact run ID with GitHub artifact digest verification. Candidate inputs freeze the harness commit separately from product source.",
"lanes": [
{
"lane": "identity",
"harness_repository": "rustfs/s3chaos",
"harness_sha": "9a48f55c1d6d58ce63a8fb1ea7317e024f32f11d",
"harness_basis": "Fixed G01 evidence-envelope producer revision (rustfs/s3chaos PR #102 head recorded in rustfs/backlog#2658). R02 re-pins this when the live verifier ships.",
"harness_repository": "rustfs/rustfs",
"harness_sha": "candidate.workflow_sha",
"harness_basis": "The exact workflow commit is frozen in candidate inputs; no mutable branch is accepted as a harness identity.",
"timeout_seconds": 1800,
"environment_capabilities": ["process-exec"],
"cases": ["artifact-identity", "startup"],
"environment_capabilities": [
"process-exec"
],
"cases": [
"artifact-identity",
"startup"
],
"applies_to": {
"release_types": ["preview", "prerelease", "stable"],
"release_types": [
"preview",
"prerelease",
"stable"
],
"platforms": "all"
},
"applies_basis": "Artifact identity and startup checks are mandatory for every release type, preview included, on every shipped target (rustfs/backlog#2662 frozen design item 5)."
},
{
"lane": "upgrade",
"harness_repository": "rustfs/s3chaos",
"harness_sha": "9a48f55c1d6d58ce63a8fb1ea7317e024f32f11d",
"harness_basis": "Same fixed envelope producer as the identity lane; the workflow SHA and the product build SHA are bound separately and need not be equal.",
"harness_repository": "rustfs/rustfs",
"harness_sha": "candidate.workflow_sha",
"harness_basis": "The exact workflow commit is frozen in candidate inputs; no mutable branch is accepted as a harness identity.",
"timeout_seconds": 5400,
"environment_capabilities": ["process-exec", "previous-release-artifact"],
"environment_capabilities": [
"process-exec",
"previous-release-artifact"
],
"cases": [
"direct-upgrade",
"mixed-version-upgrade",
@@ -49,8 +98,17 @@
],
"cases_basis": "Case IDs of the pinned previous-release matrix in .github/workflows/e2e-upgrade.yml (source.version/asset/sha256 and the case list mapped there).",
"applies_to": {
"release_types": ["prerelease", "stable"],
"platforms": [{"os": "linux", "arch": "x86_64", "abi": "gnu"}]
"release_types": [
"prerelease",
"stable"
],
"platforms": [
{
"os": "linux",
"arch": "x86_64",
"abi": "gnu"
}
]
},
"applies_basis": "preview releases use the pre-declared smaller set (identity lane only). The upgrade lane is only exercised on linux-x86_64-gnu, the platform of the pinned source assets."
}
+10 -8
View File
@@ -4,15 +4,17 @@
# One row per workspace member: <package>, tests, ignored, tab-separated.
# tests counts #[test], #[tokio::test], #[tokio::test(...)] and #[test_case(...)]
# attribute lines under the package directory; ignored counts #[ignore] and
# #[ignore = "..."]. Both columns are exact: CI fails when a crate's tests fall
# below or rise above its row, or when its ignored count changes. Raise a row in
# the PR that adds tests; lowering one is a reviewed statement that tests were
# intentionally removed. Regenerate with:
# #[ignore = "..."]. The tests column is a floor and the ignored column is a
# ceiling: CI fails when a crate's tests fall below its row or its ignored count
# rises above it. Extra tests and fewer ignores pass; --update tightens the rows.
# Lowering a tests row or raising an ignored row is a reviewed statement that
# tests were intentionally removed or ignored; justify it in the PR description.
# Regenerate with:
# scripts/check_test_count_ratchet.sh --update
e2e_test 1026 20
rustfs 6438 64
rustfs 6440 64
rustfs-audit 70 0
rustfs-checksums 26 0
rustfs-checksums 28 1
rustfs-common 36 0
rustfs-concurrency 87 0
rustfs-config 43 0
@@ -43,7 +45,7 @@ rustfs-policy 758 0
rustfs-protocols 770 1
rustfs-protos 75 0
rustfs-replication 209 0
rustfs-rio 223 0
rustfs-rio 225 1
rustfs-rio-v2 34 6
rustfs-s3-client 71 0
rustfs-s3-ops 8 0
@@ -59,6 +61,6 @@ rustfs-targets 612 26
rustfs-test-utils 1 0
rustfs-tls-runtime 44 0
rustfs-trusted-proxies 72 0
rustfs-utils 251 0
rustfs-utils 253 1
rustfs-zip 30 0
zstd 11 0
@@ -0,0 +1,96 @@
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
name: "Reference RustFS binary"
description: "Download the pinned reference RustFS release, verify its sha256 and expose the executable path. A missing pin, missing asset or digest mismatch fails the step."
inputs:
pin-file:
description: "Pin file naming the reference release (repository-relative)"
required: false
default: ".config/reference-binary.toml"
platform:
description: "Platform table in the pin file"
required: false
default: "linux-x86_64-gnu"
destination:
description: "Directory that receives the executable; defaults to $RUNNER_TEMP/reference-binary"
required: false
default: ""
outputs:
path:
description: "Absolute path of the verified executable"
value: ${{ steps.unpack.outputs.path }}
version:
description: "Pinned reference version"
value: ${{ steps.pin.outputs.version }}
sha256:
description: "Pinned and verified archive sha256"
value: ${{ steps.pin.outputs.sha256 }}
runs:
using: "composite"
steps:
- name: Resolve reference pin
id: pin
shell: bash
env:
PIN_TOOL: ${{ github.action_path }}/reference_binary.py
PIN_FILE: ${{ inputs.pin-file }}
PIN_PLATFORM: ${{ inputs.platform }}
run: |
set -euo pipefail
resolved="$(python3 "$PIN_TOOL" resolve --pin "$PIN_FILE" --platform "$PIN_PLATFORM")"
printf '%s\n' "$resolved"
printf '%s\n' "$resolved" >> "$GITHUB_OUTPUT"
- name: Download and verify reference archive
id: download
shell: bash
env:
GH_TOKEN: ${{ github.token }}
ASSET_DOWNLOADER: ${{ github.action_path }}/../setup/download_release_asset.py
REF_REPOSITORY: ${{ steps.pin.outputs.repository }}
REF_TAG: ${{ steps.pin.outputs.tag }}
REF_ASSET: ${{ steps.pin.outputs.asset }}
REF_SHA256: ${{ steps.pin.outputs.sha256 }}
run: |
set -euo pipefail
archive="$RUNNER_TEMP/reference-archive/$REF_ASSET"
# Fails unless the release lists exactly one asset of this name whose
# published digest equals the pin and whose downloaded bytes match it.
python3 "$ASSET_DOWNLOADER" --repository "$REF_REPOSITORY" --tag "$REF_TAG" \
--asset "$REF_ASSET" --output "$archive" --sha256 "$REF_SHA256"
echo "archive=$archive" >> "$GITHUB_OUTPUT"
- name: Unpack reference executable
id: unpack
shell: bash
env:
PIN_TOOL: ${{ github.action_path }}/reference_binary.py
REF_ARCHIVE: ${{ steps.download.outputs.archive }}
REF_SHA256: ${{ steps.pin.outputs.sha256 }}
REF_EXECUTABLE: ${{ steps.pin.outputs.executable }}
REF_DESTINATION: ${{ inputs.destination }}
run: |
set -euo pipefail
destination="${REF_DESTINATION:-$RUNNER_TEMP/reference-binary}"
unpacked="$(python3 "$PIN_TOOL" unpack --archive "$REF_ARCHIVE" --sha256 "$REF_SHA256" \
--executable "$REF_EXECUTABLE" --destination "$destination")"
path="${unpacked#path=}"
test -x "$path"
echo "Reference executable: $path"
"$path" --version
echo "path=$path" >> "$GITHUB_OUTPUT"
+144
View File
@@ -0,0 +1,144 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Resolve and unpack the pinned reference RustFS release binary.
`resolve` reads the pin file and prints the selected platform's identity as
`key=value` lines (GitHub step-output format). `unpack` re-verifies the
downloaded archive digest against the pin and extracts exactly one executable.
Every missing or malformed input is a failure; nothing is skipped.
"""
import argparse
import hashlib
from pathlib import Path
import re
import sys
import tomllib
import zipfile
REPOSITORY = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]*/[A-Za-z0-9][A-Za-z0-9_.-]*")
TAG = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.+-]*")
PLATFORM = re.compile(r"[a-z0-9][a-z0-9_-]*")
ASSET = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.+-]*\.zip")
SHA256 = re.compile(r"[0-9a-f]{64}")
EXECUTABLE = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]*")
MAX_EXECUTABLE_BYTES = 1024 * 1024 * 1024
class PinError(Exception):
"""A pin, archive or argument that cannot be trusted."""
def field(table, key, pattern, where):
value = table.get(key)
if not isinstance(value, str) or not pattern.fullmatch(value):
raise PinError(f"{where}: missing or invalid '{key}'")
return value
def resolve(pin_path, platform):
if not PLATFORM.fullmatch(platform or ""):
raise PinError("invalid platform name")
try:
data = tomllib.loads(Path(pin_path).read_text(encoding="utf-8"))
except FileNotFoundError as exc:
raise PinError(f"{pin_path}: reference pin file is missing") from exc
except (OSError, UnicodeDecodeError, tomllib.TOMLDecodeError) as exc:
raise PinError(f"{pin_path}: cannot read reference pin: {exc}") from exc
if data.get("schema") != 1 or type(data.get("schema")) is not int:
raise PinError(f"{pin_path}: unsupported pin schema")
identity = {
"repository": field(data, "repository", REPOSITORY, pin_path),
"version": field(data, "version", TAG, pin_path),
"tag": field(data, "tag", TAG, pin_path),
}
platforms = data.get("platforms")
if not isinstance(platforms, dict) or not platforms:
raise PinError(f"{pin_path}: no pinned platforms")
entry = platforms.get(platform)
if not isinstance(entry, dict):
raise PinError(f"{pin_path}: no pin for platform '{platform}'")
where = f"{pin_path} [platforms.{platform}]"
identity["asset"] = field(entry, "asset", ASSET, where)
identity["sha256"] = field(entry, "sha256", SHA256, where)
identity["executable"] = field(entry, "executable", EXECUTABLE, where)
if identity["version"] not in identity["asset"]:
raise PinError(f"{where}: asset name does not carry version {identity['version']}")
unknown = set(entry) - {"asset", "sha256", "executable"}
if unknown:
raise PinError(f"{where}: unknown keys {sorted(unknown)}")
return identity
def file_sha256(path):
with open(path, "rb") as stream:
return hashlib.file_digest(stream, "sha256").hexdigest()
def unpack(archive, sha256, executable, destination):
if not SHA256.fullmatch(sha256 or "") or not EXECUTABLE.fullmatch(executable or ""):
raise PinError("invalid expected digest or executable name")
try:
actual = file_sha256(archive)
except OSError as exc:
raise PinError(f"{archive}: reference archive is missing or unreadable") from exc
if actual != sha256:
raise PinError(f"{archive}: sha256 {actual} does not match the pinned digest")
try:
with zipfile.ZipFile(archive) as bundle:
matches = [info for info in bundle.infolist() if info.filename == executable]
if len(matches) != 1:
raise PinError(f"{archive}: must contain exactly one top-level '{executable}'")
info = matches[0]
if info.is_dir() or not 0 < info.file_size <= MAX_EXECUTABLE_BYTES:
raise PinError(f"{archive}: '{executable}' is not a plausible executable")
destination = Path(destination)
destination.mkdir(parents=True, exist_ok=True)
target = destination / executable
with bundle.open(info) as source, open(target, "wb") as sink:
while chunk := source.read(1024 * 1024):
sink.write(chunk)
except zipfile.BadZipFile as exc:
raise PinError(f"{archive}: not a valid zip archive") from exc
target.chmod(0o755)
return target
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = parser.add_subparsers(dest="mode", required=True)
res = sub.add_parser("resolve")
res.add_argument("--pin", required=True)
res.add_argument("--platform", required=True)
unp = sub.add_parser("unpack")
unp.add_argument("--archive", required=True)
unp.add_argument("--sha256", required=True)
unp.add_argument("--executable", required=True)
unp.add_argument("--destination", required=True)
args = parser.parse_args(argv)
try:
if args.mode == "resolve":
for key, value in resolve(args.pin, args.platform).items():
print(f"{key}={value}")
else:
print(f"path={unpack(args.archive, args.sha256, args.executable, args.destination)}")
except PinError as exc:
print(f"reference binary: {exc}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+108 -21
View File
@@ -350,10 +350,10 @@
"roles": [
"periodic",
"manual",
"release"
"reusable"
],
"events": [
"push",
"workflow_call",
"schedule",
"workflow_dispatch"
],
@@ -367,7 +367,7 @@
"monitor": "workflow",
"max_age_hours": 36,
"result_owner": "build.yml",
"disposition": "retained independent entry",
"disposition": "Builds artifacts only; public release writes belong to release-publish.yml.",
"failure_owner": "workflow",
"origin": "baseline",
"alert_contract": "scheduled-validation"
@@ -745,14 +745,13 @@
},
{
"workflow": ".github/workflows/docker.yml",
"name": "Docker Images",
"name": "Build candidate images",
"roles": [
"manual",
"release"
"release",
"reusable"
],
"events": [
"workflow_run",
"workflow_dispatch"
"workflow_call"
],
"cadence": "event",
"timezone": "UTC",
@@ -761,7 +760,7 @@
"policy": "active",
"monitor": "none",
"result_owner": "docker.yml",
"disposition": "retained independent entry",
"disposition": "Builds artifacts only; public release writes belong to release-publish.yml.",
"monitor_reason": "Event-driven or operator-requested work has no periodic freshness obligation.",
"origin": "baseline"
},
@@ -870,6 +869,33 @@
"failure_owner": "watchdog",
"origin": "baseline"
},
{
"workflow": ".github/workflows/reference-parity.yml",
"name": "Reference Parity",
"roles": [
"pr",
"periodic",
"manual"
],
"events": [
"pull_request",
"schedule",
"workflow_dispatch"
],
"cadence": "nightly",
"timezone": "UTC",
"schedule": [
"37 1 * * *"
],
"owner": "@overtrue",
"policy": "active",
"monitor": "workflow",
"max_age_hours": 36,
"result_owner": "reference-parity.yml",
"disposition": "s3-tests per-node-id parity of branch candidates against the pinned reference release",
"failure_owner": "watchdog",
"origin": "added"
},
{
"workflow": ".github/workflows/functional-chain-health.yml",
"name": "Functional chain health",
@@ -954,14 +980,13 @@
},
{
"workflow": ".github/workflows/helm-package.yml",
"name": "Publish helm chart to artifacthub",
"name": "Build candidate Helm chart",
"roles": [
"manual",
"release"
"release",
"reusable"
],
"events": [
"workflow_run",
"workflow_dispatch"
"workflow_call"
],
"cadence": "event",
"timezone": "UTC",
@@ -970,7 +995,7 @@
"policy": "active",
"monitor": "none",
"result_owner": "helm-package.yml",
"disposition": "retained independent entry",
"disposition": "Builds artifacts only; public release writes belong to release-publish.yml.",
"monitor_reason": "Event-driven or operator-requested work has no periodic freshness obligation.",
"origin": "baseline"
},
@@ -1182,14 +1207,13 @@
},
{
"workflow": ".github/workflows/package.yml",
"name": "Package DEB/RPM",
"name": "Build candidate packages",
"roles": [
"manual",
"release"
"release",
"reusable"
],
"events": [
"workflow_run",
"workflow_dispatch"
"workflow_call"
],
"cadence": "event",
"timezone": "UTC",
@@ -1198,7 +1222,7 @@
"policy": "active",
"monitor": "none",
"result_owner": "package.yml",
"disposition": "retained independent entry",
"disposition": "Builds artifacts only; public release writes belong to release-publish.yml.",
"monitor_reason": "Event-driven or operator-requested work has no periodic freshness obligation.",
"origin": "baseline"
},
@@ -1727,6 +1751,69 @@
"disposition": "retained independent entry",
"monitor_reason": "Event-driven or operator-requested work has no periodic freshness obligation.",
"origin": "baseline"
},
{
"workflow": ".github/workflows/release-candidate.yml",
"name": "Release candidate (Linux)",
"roles": [
"manual",
"release"
],
"events": [
"workflow_dispatch"
],
"cadence": "event",
"timezone": "UTC",
"schedule": [],
"owner": "@overtrue",
"policy": "active",
"monitor": "none",
"result_owner": "release-candidate.yml",
"disposition": "Explicit immutable candidate stage",
"monitor_reason": "Operator-requested release stage has no periodic freshness obligation.",
"origin": "release-candidate-migration"
},
{
"workflow": ".github/workflows/release-complete.yml",
"name": "Release candidate (complete)",
"roles": [
"manual",
"release"
],
"events": [
"workflow_dispatch"
],
"cadence": "event",
"timezone": "UTC",
"schedule": [],
"owner": "@overtrue",
"policy": "active",
"monitor": "none",
"result_owner": "release-complete.yml",
"disposition": "Explicit immutable candidate stage",
"monitor_reason": "Operator-requested release stage has no periodic freshness obligation.",
"origin": "release-candidate-migration"
},
{
"workflow": ".github/workflows/release-publish.yml",
"name": "Publish accepted release",
"roles": [
"manual",
"release"
],
"events": [
"workflow_dispatch"
],
"cadence": "event",
"timezone": "UTC",
"schedule": [],
"owner": "@overtrue",
"policy": "active",
"monitor": "none",
"result_owner": "release-publish.yml",
"disposition": "Explicit immutable candidate stage",
"monitor_reason": "Operator-requested release stage has no periodic freshness obligation.",
"origin": "release-candidate-migration"
}
]
}
+11 -4
View File
@@ -5,7 +5,7 @@
## Billing facts
This repository is public. For a public repository, GitHub's standard hosted runners (`ubuntu-latest`, `windows-latest`, `macos-latest`) cost nothing and have no minute cap, whatever plan the organisation is on. GitHub bills only two things: larger hosted runners, which have custom labels and are not used here, and standard runners in private repositories.
This repository is public. For a public repository, GitHub's standard hosted runners (`ubuntu-latest`, `windows-latest`, `macos-latest`) cost nothing and have no minute cap, whatever plan the organisation is on. GitHub bills only two things: larger hosted runners, including the `macos-latest-xlarge` release lane used here, and standard runners in private repositories.
Every other label in this repository is capacity RustFS pays for. ARC pods and lab machines accrue cost while the machine exists. A Ubicloud label is metered for the minutes a job holds its VM:
@@ -13,6 +13,7 @@ Every other label in this repository is capacity RustFS pays for. ARC pods and l
|---|---|---|---|
| `ubuntu-latest` | nobody | GitHub-hosted, Ubuntu 24.04, 4 vCPU, 16 GB RAM, about 14 GB of free disk. Ships a C toolchain, `docker`, `gh`, `curl`, `wget`, `jq`, Python. Can `mount` tmpfs and loop devices; io_uring works | the default for every job |
| `windows-latest`, `macos-latest` | nobody | GitHub-hosted standard runners | Windows and macOS targets |
| `macos-latest-xlarge` | RustFS | GitHub-hosted larger Apple Silicon runner | macOS release builds, deferred until Linux acceptance for candidates |
| `sm-standard-2` | RustFS | ARC pod, Ubuntu 24.04, minimal image: no `gh`, no C toolchain; Docker only on the `dind-` label | only a job that a hosted runner cannot run (see the placement rule) |
| `sm-standard-4` | RustFS | ARC pod, Ubuntu 24.04, 14 CPUs and 28 GB (the label is not the core count), large disk | workspace-scale Cargo builds |
| `ubicloud-standard-16-arm` | RustFS | Ubicloud VM, alias of `ubicloud-standard-16-arm-ubuntu-2404`: Ubuntu 24.04 arm64, 16 vCPU, 48 GB, 300 GB disk. Reached through the `build.yml` matrix, so `actionlint.yaml` does not list it | native aarch64 release builds (`linux-aarch64-musl`, `linux-aarch64-gnu`) |
@@ -32,7 +33,7 @@ A job runs on `ubuntu-latest` unless it has one of these reasons, and the reason
Jobs that do only the following have no reason to be self-hosted: shell or Python scripts, `cargo fmt`, `cargo deny`, typos, actionlint, API calls (`gh`, dependency review, issue and stale bots, CLA checks), downloading an artifact and running a prebuilt binary, or a `cargo build -p rustfs` with its own cache key. `e2e-distributed.yml` and `e2e-upgrade.yml` build the server and the `e2e_test` harness on `ubuntu-latest` inside their budgets, so "it compiles something" alone is not a reason.
The functional chain's `prepare` and `complete-chain` jobs, the performance runner availability check, and runner hygiene use `ubuntu-latest`: they inspect APIs or artifacts, not the lab hosts. The actual functional and performance suites remain on their lab runners. Functional-chain health collection uses `ubuntu-latest` so a lab outage cannot also stop its independent monitor. Docker image packaging also uses `ubuntu-latest` for both musl and glibc variants; it downloads release binaries rather than compiling the workspace, and still sets up QEMU and Buildx explicitly.
The functional chain's `prepare` and `complete-chain` jobs, the performance runner availability check, and runner hygiene use `ubuntu-latest`: they inspect APIs or artifacts, not the lab hosts. The actual functional and performance suites remain on their lab runners. Functional-chain health collection uses `ubuntu-latest` so a lab outage cannot also stop its independent monitor. Docker image packaging also uses `ubuntu-latest` for both musl and glibc variants; it wraps verified candidate binaries rather than compiling the workspace, and still sets up QEMU and Buildx explicitly.
Moving these jobs off AKS reduces capacity demand, not necessarily the Azure bill immediately. VM costs fall only when nodes scale down or smaller capacity replaces them.
@@ -52,7 +53,7 @@ A hosted runner is also the only choice for a few things the pods cannot do: `mo
These are the patterns that have cost real money here without validating anything more:
- An `if: always()` summary or alert job on a self-hosted label inside a `workflow_run`-triggered workflow. The upstream workflow completes many times a day, the real jobs skip, and the summary still starts a pod each time. Gate it on the upstream job having run, and keep it on `ubuntu-latest`.
- Per-push workflows on `main`. A merge triggers the full `ci.yml` matrix, `cache-warm.yml` and the `build.yml` development build; with tens of merges a day that is as much self-hosted time as all pull requests together. Anything that does not verify the merged tree itself belongs on a schedule or behind a `paths` filter.
- Per-push workflows on `main`. A merge triggers the applicable `ci.yml` matrix and cache warming; `build.yml` development builds are scheduled or manual; with tens of merges a day that is as much self-hosted time as all pull requests together. Anything that does not verify the merged tree itself belongs on a schedule or behind a `paths` filter.
- `apt-get install` of tools the hosted image already ships. It is only needed on the minimal pod images.
- A throttled `CARGO_BUILD_JOBS` on a 14-CPU pod. The limit exists for link-storm reasons documented in `ci.yml`; measure before raising it, and measure before lowering it further.
- Tiny jobs on ARC pods. Each one pays roughly a minute of pod start-up before its first step; a compile-free job that takes ten seconds costs more to schedule than to run.
@@ -66,7 +67,7 @@ gh api "repos/rustfs/rustfs/actions/runs/<run_id>/jobs?per_page=100" \
--jq '.jobs[] | "\(.conclusion)\t\(.labels | join(","))\t\(.started_at)\t\(.completed_at)\t\(.name)"'
```
Only rows whose label starts with `sm-`, `dind-`, `ubicloud-`, `smoke-` or `pf-` cost anything. At the October 2026 review, a full pull-request run of `ci.yml` spent about 150 self-hosted minutes, a third of it in jobs that never compiled; treat that as the scale of what a placement mistake costs, not as a current figure.
Rows using `macos-latest-xlarge` or a label starting with `sm-`, `dind-`, `ubicloud-`, `smoke-` or `pf-` use paid capacity. At the October 2026 review, a full pull-request run of `ci.yml` spent about 150 self-hosted minutes, a third of it in jobs that never compiled; treat that as the scale of what a placement mistake costs, not as a current figure.
## Checklist for a new or moved job
@@ -76,3 +77,9 @@ Only rows whose label starts with `sm-`, `dind-`, `ubicloud-`, `smoke-` or `pf-`
- A self-hosted `runs-on` carries a comment naming its reason from the placement rule.
- Every `./.github/actions/setup` call states `cache-save-if`, and a key read from one pool is not written from the other.
- Moving a job does not change its check name, so the `main` ruleset and `docs/testing/ci-gates.md` stay as they are; renaming it does, and then that document's change checklist applies.
## Release stages
The [candidate release pipeline](../../docs/operations/release-pipeline.md) builds Linux first and defers paid desktop capacity until acceptance. Daily development builds also select Linux; use manual platform selection for desktop development artifacts. All six platforms remain required before publication. The pipeline records and promotes the exact tested bytes. Builder workflows have no public-release credentials. Keep periodic development builds, PR verification and field-test chains separate when their contracts differ; do not re-enable disabled workflows as part of a source-file migration.
Larger runners remain billed for public repositories; see [GitHub runner billing](https://docs.github.com/en/actions/concepts/runners/larger-runners#billing). Actual spending comes from organization billing, not runner labels or an estimated build duration.
+149 -8
View File
@@ -23,9 +23,16 @@ on:
- 'deny.toml'
- '.github/actions/**'
- '.github/workflows/**'
- 'scripts/release/create_or_update_release.sh'
- 'scripts/release/package_service_scripts.sh'
- 'scripts/release/package_versions.sh'
- 'scripts/release/**'
- 'scripts/security/generate_release_supply_chain_assets.py'
- '.config/release-validation-plan.json'
- '.github/scheduled-validations.json'
- 'scripts/test_helm_release_workflow.py'
- 'scripts/test_release_packages.py'
- 'scripts/test_release_images.py'
- 'Dockerfile*'
- 'entrypoint.sh'
- 'bootstrap/**'
- 'scripts/test_package_service_scripts.sh'
- 'scripts/test_package_versions.sh'
- 'scripts/test_docker_workflow.py'
@@ -41,9 +48,16 @@ on:
- 'deny.toml'
- '.github/actions/**'
- '.github/workflows/**'
- 'scripts/release/create_or_update_release.sh'
- 'scripts/release/package_service_scripts.sh'
- 'scripts/release/package_versions.sh'
- 'scripts/release/**'
- 'scripts/security/generate_release_supply_chain_assets.py'
- '.config/release-validation-plan.json'
- '.github/scheduled-validations.json'
- 'scripts/test_helm_release_workflow.py'
- 'scripts/test_release_packages.py'
- 'scripts/test_release_images.py'
- 'Dockerfile*'
- 'entrypoint.sh'
- 'bootstrap/**'
- 'scripts/test_package_service_scripts.sh'
- 'scripts/test_package_versions.sh'
- 'scripts/test_docker_workflow.py'
@@ -132,12 +146,38 @@ jobs:
if: github.repository == 'rustfs/rustfs' && (github.event_name != 'pull_request' || github.event.action != 'closed')
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
actions: read
outputs:
release_changed: ${{ steps.scope.outputs.release_changed }}
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Scope native packaging checks to release changes
id: scope
env:
BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}
run: |
python3 - <<'PY'
import os, re, subprocess
event, base = os.environ['GITHUB_EVENT_NAME'], os.environ.get('BASE_SHA', '')
changed = event == 'workflow_dispatch'
if event in ('push', 'pull_request'):
if not re.fullmatch(r'[0-9a-f]{40}', base) or base == '0' * 40:
changed = True
else:
paths = subprocess.check_output(['git', 'diff', '--name-only', base, os.environ['GITHUB_SHA']], text=True).splitlines()
pattern = r'^(scripts/release/|scripts/test_release_|Dockerfile|entrypoint.sh$|bootstrap/|\.dockerignore$|\.config/release-validation-plan.json$|\.github/workflows/(audit|build|docker|package|helm-package|release-[^.]+)\.yml$)'
changed = any(re.match(pattern, path) for path in paths)
with open(os.environ['GITHUB_OUTPUT'], 'a') as output:
output.write(f'release_changed={str(changed).lower()}\n')
PY
- name: Report unpinned GitHub Actions
run: ./scripts/security/check_workflow_pins.sh --enforce
@@ -150,8 +190,35 @@ jobs:
- name: Check checkouts clear their credentials
run: ./scripts/security/check_persist_credentials.sh
- name: Check preview release workflow policy
run: ./scripts/security/check_preview_release_workflow.sh
- name: Check release candidate and promotion contracts
run: python3 -m unittest discover -s scripts/release -p 'test_*.py'
- name: Prepare artifact transport rehearsal
run: |
mkdir -p "$RUNNER_TEMP/release-transport"
printf '{"purpose":"transport contract, not release evidence"}\n' > "$RUNNER_TEMP/release-transport/transport.json"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-transport-contract-${{ github.run_attempt }}
path: ${{ runner.temp }}/release-transport/transport.json
if-no-files-found: error
retention-days: 1
- name: Verify actual GitHub artifact metadata and downloaded bytes
env:
GH_TOKEN: ${{ github.token }}
run: |
python3 - <<'PY'
import os, sys
from pathlib import Path
sys.path.insert(0, 'scripts/release')
import candidate as c
run = c.github(f"repos/{c.REPOSITORY}/actions/runs/{os.environ['GITHUB_RUN_ID']}")
destination = Path(os.environ['RUNNER_TEMP']) / 'release-transport-downloaded'
c.download_artifact(run, f"release-transport-contract-{os.environ['GITHUB_RUN_ATTEMPT']}", destination)
c.require(c.read_json(destination / 'transport.json') == {'purpose': 'transport contract, not release evidence'}, 'artifact round trip changed')
PY
- name: Check tier evidence workflow isolation
run: ./scripts/security/check_tier_artifact_workflow.sh
@@ -162,6 +229,80 @@ jobs:
- name: Check package service lifecycle contract
run: ./scripts/test_package_service_scripts.sh
release-package-contract:
name: Release package contract
needs: workflow-pin-report
if: needs.workflow-pin-report.outputs.release_changed == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Install real package managers
run: |
sudo apt-get update
sudo apt-get install -y ruby ruby-dev build-essential rpm fakeroot cpio
sudo gem install fpm --version 1.17.0 --no-document
- name: Exercise accepted and substituted payloads through both recipes
run: python3 scripts/test_release_packages.py
release-image-contract:
name: Release image contract (${{ matrix.variant }})
needs: workflow-pin-report
if: needs.workflow-pin-report.outputs.release_changed == 'true'
runs-on: ubuntu-latest
timeout-minutes: 35
strategy:
fail-fast: false
matrix:
variant: [musl, gnu]
env:
RELEASE_IMAGE_VARIANT: ${{ matrix.variant }}
NO_PROXY: 127.0.0.1,localhost,::1
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Install OCI tools
run: |
echo "RELEASE_IMAGE_OUTPUT=$RUNNER_TEMP/image-contract" >> "$GITHUB_ENV"
echo 'no_proxy=127.0.0.1,localhost,::1' >> "$GITHUB_ENV"
sudo apt-get update
sudo apt-get install -y skopeo
go version
- uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
- uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
- name: Exercise real OCI packaging, startup, registry retry and conflicts
run: python3 scripts/test_release_images.py
- name: Exercise amd64 OCI archive scanning
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
env:
TRIVY_PLATFORM: linux/amd64
with:
input: ${{ runner.temp }}/image-contract-scan/image-${{ matrix.variant }}-amd64.tar
format: sarif
output: ${{ runner.temp }}/image-contract/scan-${{ matrix.variant }}-amd64.sarif
exit-code: "0"
- name: Exercise arm64 OCI archive scanning
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
env:
TRIVY_PLATFORM: linux/arm64
with:
input: ${{ runner.temp }}/image-contract-scan/image-${{ matrix.variant }}-arm64.tar
format: sarif
output: ${{ runner.temp }}/image-contract/scan-${{ matrix.variant }}-arm64.sarif
exit-code: "0"
- name: Verify scan receipts before recording the archive
run: |
python3 - <<'PY'
import os, sys
from pathlib import Path
sys.path.insert(0, 'scripts/release')
import deliver, test_candidate
deliver.image_record(test_candidate.inputs(), os.environ['RELEASE_IMAGE_VARIANT'], Path(os.environ['RELEASE_IMAGE_OUTPUT']))
PY
dependency-review:
name: Dependency Review
runs-on: ubuntu-latest
File diff suppressed because it is too large Load Diff
+71 -751
View File
@@ -1,779 +1,99 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Docker Images Workflow
#
# This workflow builds Docker images using pre-built binaries from the build workflow.
#
# Trigger Types:
# 1. workflow_run: Automatically triggered when "Build and Release" workflow completes
# 2. workflow_dispatch: Manual trigger for standalone Docker builds
#
# Key Features:
# - Only triggers when Linux builds (x86_64 + aarch64) are successful
# - Independent of macOS/Windows build status
# - Uses workflow_run event for precise control
# - Only builds Docker images for releases and prereleases (development builds are skipped)
name: Docker Images
# Permissions needed for workflow_run event and Docker registry access
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
name: Build candidate images
on:
workflow_call:
inputs:
candidate_manifest:
type: string
required: true
permissions:
contents: read
packages: write
on:
# Automatically triggered when build workflow completes
workflow_run:
workflows: [ "Build and Release" ]
types: [ completed ]
# Manual trigger with same parameters for consistency
workflow_dispatch:
inputs:
push_images:
description: "Push images to registries"
required: false
default: true
type: boolean
version:
description: "Version to build (latest, v1.0.0, v1.0.0-alpha1, or v1.0.1-preview.1)"
required: false
default: "latest"
type: string
force_rebuild:
description: "Force rebuild even if binary exists (useful for testing)"
required: false
default: false
type: boolean
env:
CONCLUSION: ${{ github.event.workflow_run.conclusion }}
HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
TRIGGERING_EVENT: ${{ github.event.workflow_run.event }}
DOCKERHUB_USERNAME: rustfs
CARGO_TERM_COLOR: always
REGISTRY_DOCKERHUB: rustfs/rustfs
REGISTRY_GHCR: ghcr.io/${{ github.repository }}
REGISTRY_QUAY: quay.io/rustfs/rustfs
DOCKER_PLATFORMS: linux/amd64,linux/arm64
actions: read
jobs:
# Check if we should build Docker images.
# Short-circuit at job level so per-merge development builds (workflow_run
# from a push to main) skip the whole run without spawning a checkout job:
# images are only published for tag builds — build.yml push triggers cover
# only main and tags, so a non-main head_branch is a tag name — and for
# manual dispatch.
build-check:
name: Docker Build Check
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push' &&
github.event.workflow_run.head_branch != 'main')
runs-on: ubuntu-latest
timeout-minutes: 30
outputs:
should_build: ${{ steps.check.outputs.should_build }}
should_push: ${{ steps.check.outputs.should_push }}
build_type: ${{ steps.check.outputs.build_type }}
version: ${{ steps.check.outputs.version }}
short_sha: ${{ steps.check.outputs.short_sha }}
is_prerelease: ${{ steps.check.outputs.is_prerelease }}
create_latest: ${{ steps.check.outputs.create_latest }}
source_ref: ${{ steps.check.outputs.source_ref }}
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
# For workflow_run events, checkout the specific commit that triggered the workflow
ref: ${{ github.event.workflow_run.head_sha || github.sha }}
- name: Check build conditions
id: check
env:
# dispatch inputs via env, not `${{ }}` interpolation: they are
# free-form strings and would otherwise be evaluated by bash.
INPUT_VERSION: ${{ github.event.inputs.version }}
INPUT_PUSH_IMAGES: ${{ github.event.inputs.push_images }}
INPUT_FORCE_REBUILD: ${{ github.event.inputs.force_rebuild }}
run: |
should_build=false
should_push=false
build_type="none"
version=""
short_sha=""
is_prerelease=false
create_latest=false
source_ref="$GITHUB_SHA"
# Pre-GA policy: until the first stable (vX.Y.Z) tag exists, every
# prerelease (alpha/beta/rc) also moves `latest`, so users pulling
# `latest` get the newest test build. Once a stable tag is published
# this returns false and `latest` follows stable releases only.
prerelease_moves_latest() {
local stable_tags
stable_tags=$(git ls-remote --tags --refs origin 2>/dev/null \
| awk '{print $2}' \
| grep -E '^refs/tags/v?[0-9]+\.[0-9]+\.[0-9]+$' || true)
if [[ -z "$stable_tags" ]]; then
return 0
fi
echo "ℹ️ Stable release tag(s) already exist; prereleases no longer update latest"
return 1
}
if [[ "${{ github.event_name }}" == "workflow_run" ]]; then
# Triggered by build workflow completion
echo "🔗 Triggered by build workflow completion"
# Check if the triggering workflow was successful
# If the workflow succeeded, it means ALL builds (including Linux x86_64 and aarch64) succeeded
if [[ "$CONCLUSION" == "success" ]]; then
echo "✅ Build workflow succeeded, all builds including Linux are successful"
should_build=true
should_push=true
else
echo "❌ Build workflow failed (conclusion: $CONCLUSION), skipping Docker build"
should_build=false
fi
# Extract version info from commit message or use commit SHA
# Use Git to generate consistent short SHA (ensures uniqueness like build.yml)
short_sha=$(git rev-parse --short "$HEAD_SHA")
source_ref="$HEAD_SHA"
# Determine build type based on triggering workflow event and ref
triggering_event="$TRIGGERING_EVENT"
head_branch="$HEAD_BRANCH"
echo "🔍 Analyzing triggering workflow:"
echo " 📋 Event: $triggering_event"
echo " 🌿 Head branch: $head_branch"
echo " 📎 Head SHA: $HEAD_SHA"
# Check if this was triggered by a tag push
if [[ "$triggering_event" == "push" ]]; then
# For tag pushes, head_branch will be like "refs/tags/v1.0.0" or just "v1.0.0"
if [[ "$head_branch" == refs/tags/* ]]; then
# Extract tag name from refs/tags/TAG_NAME
tag_name="${head_branch#refs/tags/}"
version="$tag_name"
elif [[ "$head_branch" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+ ]]; then
# Direct tag name like "v1.0.0" or "1.0.0-alpha.1"
version="$head_branch"
elif [[ "$head_branch" == "main" ]]; then
# Regular branch push to main
build_type="development"
version="dev-${short_sha}"
should_build=false
echo "⏭️ Skipping Docker build for development version (main branch push)"
else
# Other branch push
build_type="development"
version="dev-${short_sha}"
should_build=false
echo "⏭️ Skipping Docker build for development version (branch: $head_branch)"
fi
# If we extracted a version (tag), determine release type
if [[ -n "$version" ]] && [[ "$version" != "dev-${short_sha}" ]]; then
# Remove 'v' prefix if present for consistent version format
if [[ "$version" == v* ]]; then
version="${version#v}"
fi
if [[ "$version" =~ -preview\.[0-9]+$ ]]; then
build_type="preview"
is_prerelease=true
echo "🔍 Building Docker image for preview: $version (version tags only)"
elif [[ "$version" == *"-preview"* ]]; then
echo "❌ Invalid preview tag: $version (expected suffix: -preview.<number>)" >&2
exit 1
elif [[ "$version" == *"alpha"* ]] || [[ "$version" == *"beta"* ]] || [[ "$version" == *"rc"* ]]; then
build_type="prerelease"
is_prerelease=true
# Pre-GA policy: prereleases update latest until the first stable tag exists.
if prerelease_moves_latest; then
create_latest=true
echo "🧪 Building Docker image for prerelease: $version (creating latest tag)"
else
echo "🧪 Building Docker image for prerelease: $version"
fi
else
build_type="release"
create_latest=true
echo "🚀 Building Docker image for release: $version"
fi
fi
else
# Non-push events
build_type="development"
version="dev-${short_sha}"
should_build=false
echo "⏭️ Skipping Docker build for development version (event: $triggering_event)"
fi
echo "🔄 Build triggered by workflow_run:"
echo " 📋 Conclusion: $CONCLUSION"
echo " 🌿 Branch: $HEAD_BRANCH"
echo " 📎 SHA: $HEAD_SHA"
echo " 🎯 Event: $TRIGGERING_EVENT"
elif [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
# Manual trigger
input_version="$INPUT_VERSION"
version="${input_version}"
should_push="$INPUT_PUSH_IMAGES"
should_build=true
# Get short SHA
short_sha=$(git rev-parse --short HEAD)
echo "🎯 Manual Docker build triggered:"
echo " 📋 Requested version: $input_version"
echo " 🔧 Force rebuild: $INPUT_FORCE_REBUILD"
echo " 🚀 Push images: $should_push"
case "$input_version" in
"latest")
build_type="release"
create_latest=true
echo "🚀 Building with latest stable release version"
;;
*-preview*)
if [[ ! "$input_version" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?-preview\.[0-9]+$ ]]; then
echo "❌ Invalid preview version: $input_version" >&2
exit 1
fi
build_type="preview"
is_prerelease=true
version="${input_version#v}"
echo "🔍 Building with preview version: $version (version tags only)"
;;
# Prerelease versions (must match first, more specific)
v*alpha*|v*beta*|v*rc*|*alpha*|*beta*|*rc*)
build_type="prerelease"
is_prerelease=true
# Pre-GA policy: prereleases update latest until the first stable tag exists.
if prerelease_moves_latest; then
create_latest=true
echo "🧪 Building with prerelease version: $input_version (creating latest tag)"
else
echo "🧪 Building with prerelease version: $input_version"
fi
;;
# Release versions (match after prereleases, more general)
v[0-9]*|[0-9]*.*.*)
build_type="release"
create_latest=true
echo "📦 Building with specific release version: $input_version"
;;
*)
# Invalid version for Docker build
should_build=false
echo "❌ Invalid version for Docker build: $input_version"
echo "⚠️ Only release versions (latest, v1.0.0, 1.0.0) and prereleases (v1.0.0-alpha1, 1.0.0-beta2) are supported"
;;
esac
if [[ "$should_build" == true && "$input_version" != "latest" ]]; then
tag_ref="refs/tags/$input_version"
if ! git ls-remote --exit-code origin "$tag_ref" >/dev/null 2>&1; then
if [[ "$input_version" == v* ]]; then
tag_ref="refs/tags/${input_version#v}"
else
tag_ref="refs/tags/v$input_version"
fi
fi
if ! git ls-remote --exit-code origin "$tag_ref" >/dev/null 2>&1; then
echo "❌ Release tag not found for Docker build: $input_version"
exit 1
fi
source_ref="$tag_ref"
fi
fi
{
echo "should_build=$should_build"
echo "should_push=$should_push"
echo "build_type=$build_type"
echo "version=$version"
echo "short_sha=$short_sha"
echo "is_prerelease=$is_prerelease"
echo "create_latest=$create_latest"
echo "source_ref=$source_ref"
} >> "$GITHUB_OUTPUT"
echo "🐳 Docker Build Summary:"
echo " - Should build: $should_build"
echo " - Should push: $should_push"
echo " - Build type: $build_type"
echo " - Version: $version"
echo " - Short SHA: $short_sha"
echo " - Is prerelease: $is_prerelease"
echo " - Create latest: $create_latest"
echo " - Source ref: $source_ref"
# Consume the trusted release certification for the exact version this
# workflow wraps and verify the certified input digests. A manual dispatch
# without that certification fails here and pushes no official tag; only
# results from the trusted verification sources carry release authority.
release-certification:
name: Release Certification
needs: build-check
if: needs.build-check.outputs.should_build == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
ref: ${{ needs.build-check.outputs.source_ref }}
- name: Download and verify release certification
env:
GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
TRIGGERING_RUN_ID: ${{ github.event.workflow_run.id }}
VERSION: ${{ needs.build-check.outputs.version }}
shell: bash
run: |
set -euo pipefail
mkdir -p certification
run_id="$TRIGGERING_RUN_ID"
if [[ -z "$run_id" ]]; then
run_id="$(gh api --method GET \
"repos/${GITHUB_REPOSITORY}/actions/workflows/build.yml/runs" \
-f branch="$VERSION" -f status=success -F per_page=1 \
--jq '.workflow_runs[0].id // empty' || true)"
fi
if [[ -z "$run_id" ]]; then
echo "No certified Build and Release run found for ${VERSION}; failing closed" >&2
exit 1
fi
if ! gh run download "$run_id" -n release-certification -D certification; then
echo "Build run ${run_id} published no release-certification artifact; failing closed" >&2
exit 1
fi
python3 scripts/release/release_gate.py verify-promotion \
--plan .config/release-validation-plan.json \
--candidate certification/candidate.json \
--summary certification/release-gate-output \
--artifact-sha256 "$(jq -r '.deliverables[0].sha256' certification/candidate.json)" \
--artifact-name "$(jq -r '.deliverables[0].name' certification/candidate.json)"
# Input digest binding: every zip this image wraps must be the
# certified byte-for-byte content, never a mutable latest pointer.
while IFS=$'\t' read -r name location expected; do
echo "Verifying certified input ${name}"
actual="$(curl -fsSL "$location" | sha256sum | awk '{print $1}')"
if [[ "$actual" != "$expected" ]]; then
echo "Certified input ${name} does not match its recorded digest" >&2
exit 1
fi
done < <(jq -r '.deliverables[] | select(.name | contains("linux")) | [.name, .location, .sha256] | @tsv' certification/candidate.json)
# Build multi-arch Docker images
# Strategy: Build images using pre-built binaries from dl.rustfs.com
# Supports both release and dev channel binaries based on build context
# Only runs when should_build is true (which includes workflow success check)
build-docker:
name: Build Docker Images
needs: [ build-check, release-certification ]
if: needs.build-check.outputs.should_build == 'true'
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
include:
- variant: musl
file: Dockerfile
suffix: ""
- variant: glibc
- variant: gnu
file: Dockerfile.glibc
suffix: "-glibc"
env:
CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
VARIANT: ${{ matrix.variant }}
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
ref: ${{ fromJson(inputs.candidate_manifest).source_sha }}
persist-credentials: false
ref: ${{ needs.build-check.outputs.source_ref }}
- name: Login to Docker Hub
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
username: ${{ env.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
password: ${{ secrets.GHCR_PASSWORD }}
- name: Login to Quay.io
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
registry: quay.io
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_PASSWORD }}
- name: Set up QEMU
uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
- name: Extract metadata and generate tags
id: meta
- name: Reuse completed image
id: reuse
env:
GH_TOKEN: ${{ github.token }}
run: python3 scripts/release/deliver.py reuse --kind image --target "$VARIANT"
- name: Install OCI tools
if: steps.reuse.outputs.reused != 'true'
run: |
BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
VERSION="${{ needs.build-check.outputs.version }}"
CREATE_LATEST="${{ needs.build-check.outputs.create_latest }}"
VARIANT_SUFFIX="${{ matrix.suffix }}"
# Convert version format for Dockerfile compatibility. The former
# DOCKER_CHANNEL was "release" down every branch and was passed as a
# build-arg no Dockerfile declares, so it is gone.
case "$VERSION" in
"latest")
DOCKER_RELEASE="latest"
;;
v*)
# For versioned releases (v1.0.0), remove 'v' prefix for Dockerfile
DOCKER_RELEASE="${VERSION#v}"
;;
*)
# For other versions, pass as-is
DOCKER_RELEASE="${VERSION}"
;;
esac
echo "docker_release=$DOCKER_RELEASE" >> "$GITHUB_OUTPUT"
echo "🐳 Docker build parameters:"
echo " - Original version: $VERSION"
echo " - Docker RELEASE: $DOCKER_RELEASE"
# Generate tags based on build type
# Only support release and prerelease builds (no development builds)
TAG_BASE="${VERSION}${VARIANT_SUFFIX}"
TAGS="${{ env.REGISTRY_DOCKERHUB }}:$TAG_BASE,${{ env.REGISTRY_GHCR }}:$TAG_BASE,${{ env.REGISTRY_QUAY }}:$TAG_BASE"
# Add latest when requested (stable releases, and prereleases before GA)
if [[ "$CREATE_LATEST" == "true" ]]; then
TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:latest${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:latest${VARIANT_SUFFIX}"
fi
# Always add the channel tag for prereleases, independent of latest
if [[ "$BUILD_TYPE" == "prerelease" ]]; then
# Prerelease channel tags (alpha, beta, rc)
if [[ "$VERSION" == *"alpha"* ]]; then
CHANNEL="alpha"
elif [[ "$VERSION" == *"beta"* ]]; then
CHANNEL="beta"
elif [[ "$VERSION" == *"rc"* ]]; then
CHANNEL="rc"
fi
if [[ -n "$CHANNEL" ]]; then
TAGS="$TAGS,${{ env.REGISTRY_DOCKERHUB }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_GHCR }}:${CHANNEL}${VARIANT_SUFFIX},${{ env.REGISTRY_QUAY }}:${CHANNEL}${VARIANT_SUFFIX}"
fi
fi
# Output tags
echo "tags=$TAGS" >> "$GITHUB_OUTPUT"
# Staging identity for this run's promotion: official tags are only
# created after the staged image verifies against the certification,
# and they point at this same digest (no rebuild-after-verify).
STAGING_TAG="staging-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}${VARIANT_SUFFIX}"
STAGING_TAGS="${{ env.REGISTRY_DOCKERHUB }}:${STAGING_TAG},${{ env.REGISTRY_GHCR }}:${STAGING_TAG},${{ env.REGISTRY_QUAY }}:${STAGING_TAG}"
echo "staging_tags=$STAGING_TAGS" >> "$GITHUB_OUTPUT"
echo "staging_tag=$STAGING_TAG" >> "$GITHUB_OUTPUT"
# Generate labels
LABELS="org.opencontainers.image.title=RustFS"
LABELS="$LABELS,org.opencontainers.image.description=RustFS distributed object storage system"
LABELS="$LABELS,org.opencontainers.image.version=$VERSION"
SOURCE_REVISION="$(git rev-parse HEAD)"
LABELS="$LABELS,org.opencontainers.image.revision=$SOURCE_REVISION"
LABELS="$LABELS,org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}"
LABELS="$LABELS,org.opencontainers.image.created=$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
LABELS="$LABELS,org.opencontainers.image.build-type=$BUILD_TYPE"
echo "labels=$LABELS" >> "$GITHUB_OUTPUT"
echo "🐳 Generated Docker tags:"
echo "$TAGS" | tr ',' '\n' | sed 's/^/ - /'
echo "📋 Build type: $BUILD_TYPE"
echo "🔖 Version: $VERSION"
- name: Build and push Docker image
sudo apt-get update
sudo apt-get install -y skopeo
- uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
if: steps.reuse.outputs.reused != 'true'
- uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
if: steps.reuse.outputs.reused != 'true'
- name: Supply accepted binaries and pinned CLI
if: steps.reuse.outputs.reused != 'true'
env:
GH_TOKEN: ${{ github.token }}
run: python3 scripts/release/deliver.py image-prepare --target "$VARIANT"
- name: Build OCI archive without registry writes
if: steps.reuse.outputs.reused != 'true'
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
with:
context: .
file: ${{ matrix.file }}
platforms: ${{ env.DOCKER_PLATFORMS }}
push: ${{ needs.build-check.outputs.should_push == 'true' }}
# Push to staging tags only: official tags are created by the
# certification-gated promotion step below from this same digest.
tags: ${{ steps.meta.outputs.staging_tags }}
labels: ${{ steps.meta.outputs.labels }}
# No layer cache. This build compiles nothing — it downloads a
# release zip and runs apk/apt — so the cache could only save the
# minute or two those take, while creating a correctness problem: with
# RELEASE=latest the binary URL is resolved by curl *inside* a RUN
# layer, and the layer key does not include what that resolved to. A
# rebuild at the same RELEASE value (dispatch with version=latest, or
# a re-run of the same version) would hit the old layer and ship the
# previous release's binary. mode=max also consumed the same 10GB
# Actions cache quota the Rust lanes are fighting over.
#
# Only RELEASE is passed: it is the sole build-arg the Dockerfiles
# declare besides TARGETARCH. BUILDTIME, VERSION, BUILD_TYPE, REVISION
# and CHANNEL were never read by any stage (and BUILDTIME's $(date ...)
# was a literal here, not a shell substitution). BUILD_DATE and VCS_REF
# are declared by the Dockerfiles but deliberately left unset —
# supplying them would change the published image labels.
platforms: linux/amd64,linux/arm64
push: false
provenance: false
outputs: type=oci,dest=delivery/image-${{ matrix.variant }}.tar
build-args: |
RELEASE=${{ steps.meta.outputs.docker_release }}
provenance: true
sbom: true
# Add retry mechanism by splitting the build process
no-cache: false
pull: true
# A later-built wrapper: the staged image must complete this platform's
# own binary identity verification before any official tag is written.
# It may not inherit an unverified package's conclusion.
- name: Verify staged image against certification
if: needs.build-check.outputs.should_push == 'true'
env:
GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
TRIGGERING_RUN_ID: ${{ github.event.workflow_run.id }}
VERSION: ${{ needs.build-check.outputs.version }}
shell: bash
run: |
set -euo pipefail
mkdir -p certification
run_id="$TRIGGERING_RUN_ID"
if [[ -z "$run_id" ]]; then
run_id="$(gh api --method GET \
"repos/${GITHUB_REPOSITORY}/actions/workflows/build.yml/runs" \
-f branch="$VERSION" -f status=success -F per_page=1 \
--jq '.workflow_runs[0].id // empty' || true)"
fi
gh run download "$run_id" -n release-certification -D certification
STAGING_REF="${{ env.REGISTRY_DOCKERHUB }}:staging-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}${{ matrix.suffix }}"
case "${{ matrix.variant }}" in
musl) libc="musl" ;;
glibc) libc="gnu" ;;
esac
for platform in linux/amd64 linux/arm64; do
case "$platform" in
linux/amd64) arch_sub="x86_64" ;;
linux/arm64) arch_sub="aarch64" ;;
esac
docker pull --platform "$platform" "$STAGING_REF"
container="$(docker create --platform "$platform" "$STAGING_REF")"
docker cp "$container:/usr/bin/rustfs" "$RUNNER_TEMP/staged-rustfs"
docker rm "$container" >/dev/null
actual="$(sha256sum "$RUNNER_TEMP/staged-rustfs" | awk '{print $1}')"
expected="$(jq -r --arg needle "linux-${arch_sub}-${libc}" \
'[.deliverables[] | select(.name | contains($needle)) | .binary_sha256][0] // empty' \
certification/candidate.json)"
if [[ -z "$expected" || "$actual" != "$expected" ]]; then
echo "Staged image binary for ${platform} does not match the certified binary digest" >&2
exit 1
fi
python3 scripts/release/release_gate.py verify-promotion \
--plan .config/release-validation-plan.json \
--candidate certification/candidate.json \
--summary certification/release-gate-output \
--artifact "$RUNNER_TEMP/staged-rustfs" \
--artifact-role binary
done
- name: Promote staged image to official tags
if: needs.build-check.outputs.should_push == 'true'
shell: bash
run: |
set -euo pipefail
STAGING_REF="${{ env.REGISTRY_DOCKERHUB }}:staging-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}${{ matrix.suffix }}"
IFS=',' read -r -a official_tags <<< "${{ steps.meta.outputs.tags }}"
for tag in "${official_tags[@]}"; do
docker buildx imagetools create -t "$tag" "$STAGING_REF"
echo "Promoted $tag -> $STAGING_REF"
done
# Downstream consumers (Helm) verify this record instead of inheriting
# an unverified wrapper's conclusion.
- name: Record image promotion
if: needs.build-check.outputs.should_push == 'true'
shell: bash
env:
GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
TRIGGERING_RUN_ID: ${{ github.event.workflow_run.id }}
VERSION: ${{ needs.build-check.outputs.version }}
run: |
set -euo pipefail
mkdir -p certification
run_id="$TRIGGERING_RUN_ID"
if [[ -z "$run_id" ]]; then
run_id="$(gh api --method GET \
"repos/${GITHUB_REPOSITORY}/actions/workflows/build.yml/runs" \
-f branch="$VERSION" -f status=success -F per_page=1 \
--jq '.workflow_runs[0].id // empty' || true)"
fi
gh run download "$run_id" -n release-certification -D certification
STAGING_REF="${{ env.REGISTRY_DOCKERHUB }}:staging-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}${{ matrix.suffix }}"
digest="$(docker buildx imagetools inspect "$STAGING_REF" --format '{{json .Manifest}}' | jq -r '.digest')"
candidate_sha="$(python3 -c "import json,hashlib,sys; sys.path.insert(0,'scripts/release'); import release_gate; print(release_gate.canonical_sha256(json.load(open('certification/candidate.json'))))")"
jq -n \
--arg version "$VERSION" \
--arg digest "$digest" \
--arg staging_ref "$STAGING_REF" \
--arg candidate_sha256 "$candidate_sha" \
--arg tags "${{ steps.meta.outputs.tags }}" \
--slurpfile candidate certification/candidate.json \
'{schema_version: 1, version: $version, image_digest: $digest, staging_ref: $staging_ref,
promoted_tags: ($tags | split(",")), candidate_sha256: $candidate_sha256,
binary_sha256s: ($candidate[0].deliverables | map(.binary_sha256) | unique)}' \
> image-promotion-record.json
- name: Upload image promotion record
if: needs.build-check.outputs.should_push == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: image-promotion-record${{ matrix.suffix }}
path: image-promotion-record.json
if-no-files-found: error
retention-days: 30
# Note: Manifest creation is no longer needed as we only build one variant
# Multi-arch manifests are automatically created by docker/build-push-action
# Report-only container image vulnerability scan
scan-docker-image:
name: Scan Docker Images
needs: [ build-check, build-docker ]
if: needs.build-check.outputs.should_build == 'true' && needs.build-check.outputs.should_push == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
include:
- variant: musl
suffix: ""
- variant: glibc
suffix: "-glibc"
steps:
- name: Login to GitHub Container Registry
# docker/login-action v3
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
password: ${{ secrets.GHCR_PASSWORD }}
- name: Scan image with Trivy
# aquasecurity/trivy-action v0.36.0
BINARY_SOURCE=supplied
RELEASE=${{ fromJson(inputs.candidate_manifest).version }}
VCS_REF=${{ fromJson(inputs.candidate_manifest).source_sha }}
- name: Verify wrapped binaries and startup on both architectures
if: steps.reuse.outputs.reused != 'true'
run: python3 scripts/release/deliver.py image-smoke --target "$VARIANT"
- name: Scan amd64 image before sealing
if: steps.reuse.outputs.reused != 'true'
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
env:
TRIVY_PLATFORM: linux/amd64
with:
image-ref: ${{ env.REGISTRY_GHCR }}:${{ needs.build-check.outputs.version }}${{ matrix.suffix }}
input: delivery-scan/image-${{ matrix.variant }}-amd64.tar
format: sarif
output: trivy-${{ matrix.variant }}.sarif
ignore-unfixed: true
vuln-type: os,library
severity: CRITICAL,HIGH
output: delivery/scan-${{ matrix.variant }}-amd64.sarif
exit-code: "0" # Keep the repository's report-only vulnerability policy.
- name: Scan arm64 image before sealing
if: steps.reuse.outputs.reused != 'true'
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25
env:
TRIVY_PLATFORM: linux/arm64
with:
input: delivery-scan/image-${{ matrix.variant }}-arm64.tar
format: sarif
output: delivery/scan-${{ matrix.variant }}-arm64.sarif
exit-code: "0"
# Surface findings in the Security tab; an artifact alone is write-only
# reporting nobody reviews.
- name: Upload scan results to code scanning
uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
- name: Record verified OCI identity
if: steps.reuse.outputs.reused != 'true'
run: python3 scripts/release/deliver.py image-record --target "$VARIANT"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: steps.reuse.outputs.reused != 'true'
with:
sarif_file: trivy-${{ matrix.variant }}.sarif
category: container-image-${{ matrix.variant }}
- name: Upload container scan report
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: container-image-scan-${{ matrix.variant }}
path: trivy-${{ matrix.variant }}.sarif
name: image-${{ matrix.variant }}
path: delivery/*
if-no-files-found: error
retention-days: 30
# Docker build summary
docker-summary:
name: Docker Build Summary
needs: [ build-check, build-docker ]
if: always() && needs.build-check.outputs.should_build == 'true'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Docker build completion summary
run: |
BUILD_TYPE="${{ needs.build-check.outputs.build_type }}"
VERSION="${{ needs.build-check.outputs.version }}"
CREATE_LATEST="${{ needs.build-check.outputs.create_latest }}"
echo "🐳 Docker build completed successfully!"
echo "📦 Build type: $BUILD_TYPE"
echo "🔢 Version: $VERSION"
echo "🚀 Strategy: Images using pre-built binaries (release channel only)"
echo ""
case "$BUILD_TYPE" in
"preview")
echo "🔍 Preview Docker image has been built with ${VERSION} tags"
echo "⏭️ Preview images do not update latest or prerelease channels"
;;
"release")
echo "🚀 Release Docker image has been built with ${VERSION} tags"
echo "✅ This image is ready for production use"
if [[ "$CREATE_LATEST" == "true" ]]; then
echo "🏷️ Latest tag has been created for stable release"
fi
;;
"prerelease")
echo "🧪 Prerelease Docker image has been built with ${VERSION} tags"
echo "⚠️ This is a prerelease image - use with caution"
# Prereleases move latest until the first stable tag exists (pre-GA policy).
if [[ "$CREATE_LATEST" == "true" ]]; then
echo "🏷️ Latest tag has been created for prerelease: $VERSION"
else
echo "🚫 Latest tag NOT created for prerelease"
fi
;;
*)
echo "❌ Unexpected build type: $BUILD_TYPE"
;;
esac
compression-level: 0
+51 -277
View File
@@ -1,229 +1,26 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
name: Publish helm chart to artifacthub
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
name: Build candidate Helm chart
on:
workflow_run:
workflows: [ "Build and Release" ]
types: [ completed ]
workflow_dispatch:
workflow_call:
inputs:
version:
description: "Release version to publish, e.g. 1.0.0-beta.1 or v1.0.0-beta.1"
required: true
default: "1.0.0-beta.1"
candidate_manifest:
type: string
required: true
permissions:
contents: read
actions: read
jobs:
# Consume the same trusted release certification as every other publication
# exit and verify the image promotion record of the Docker workflow. The
# chart wraps no binary of its own, so it may only reference an image whose
# platform verification already passed; it must not inherit an unverified
# package's conclusion. A run without that evidence fails closed here and
# never reaches the chart repository.
release-certification:
package:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
actions: read
outputs:
version: ${{ steps.version.outputs.raw_tag }}
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
ref: ${{ fromJson(inputs.candidate_manifest).source_sha }}
persist-credentials: false
- name: Normalize release version
id: version
env:
RAW_INPUT: ${{ github.event.inputs.version }}
RAW_BRANCH: ${{ github.event.workflow_run.head_branch }}
run: |
set -euo pipefail
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
RAW="$RAW_INPUT"
else
RAW="$RAW_BRANCH"
fi
case "$RAW" in
refs/tags/*) RAW_TAG="${RAW#refs/tags/}" ;;
*) RAW_TAG="$RAW" ;;
esac
echo "raw_tag=$RAW_TAG" >> "$GITHUB_OUTPUT"
- name: Download and verify release certification and image record
env:
GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
TRIGGERING_RUN_ID: ${{ github.event.workflow_run.id }}
RAW_TAG: ${{ steps.version.outputs.raw_tag }}
shell: bash
run: |
set -euo pipefail
mkdir -p certification image-record
build_run="$TRIGGERING_RUN_ID"
if [[ -z "$build_run" ]]; then
build_run="$(gh api --method GET \
"repos/${GITHUB_REPOSITORY}/actions/workflows/build.yml/runs" \
-f branch="$RAW_TAG" -f status=success -F per_page=1 \
--jq '.workflow_runs[0].id // empty' || true)"
fi
if [[ -z "$build_run" ]]; then
echo "No certified Build and Release run found for ${RAW_TAG}; failing closed" >&2
exit 1
fi
if ! gh run download "$build_run" -n release-certification -D certification; then
echo "Build run ${build_run} published no release-certification artifact; failing closed" >&2
exit 1
fi
python3 scripts/release/release_gate.py verify-promotion \
--plan .config/release-validation-plan.json \
--candidate certification/candidate.json \
--summary certification/release-gate-output \
--artifact-sha256 "$(jq -r '.deliverables[0].sha256' certification/candidate.json)" \
--artifact-name "$(jq -r '.deliverables[0].name' certification/candidate.json)"
docker_run="$(gh api --method GET \
"repos/${GITHUB_REPOSITORY}/actions/workflows/docker.yml/runs" \
-f branch="$RAW_TAG" -f status=success -F per_page=1 \
--jq '.workflow_runs[0].id // empty' || true)"
if [[ -z "$docker_run" ]] || ! gh run download "$docker_run" -n image-promotion-record -D image-record; then
echo "No image promotion record for ${RAW_TAG}; a chart may not wrap an unverified image" >&2
exit 1
fi
candidate_sha="$(python3 -c "import json,hashlib,sys; sys.path.insert(0,'scripts/release'); import release_gate; print(release_gate.canonical_sha256(json.load(open('certification/candidate.json'))))")"
record_sha="$(jq -r '.candidate_sha256 // empty' image-record/image-promotion-record.json)"
if [[ "$record_sha" != "$candidate_sha" ]]; then
echo "Image promotion record does not belong to this certification" >&2
exit 1
fi
overlap="$(jq -r --slurpfile candidate certification/candidate.json \
'[.binary_sha256s[]] - ($candidate[0].deliverables | map(.binary_sha256)) | length' \
image-record/image-promotion-record.json)"
if [[ "$overlap" -ne 0 ]]; then
echo "Image promotion record carries binary digests outside the certification" >&2
exit 1
fi
build-helm-package:
runs-on: ubuntu-latest
timeout-minutes: 30
needs: [ release-certification ]
if: |
(github.event_name == 'workflow_dispatch' && !contains(github.event.inputs.version, '-preview')) ||
(
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push' &&
contains(github.event.workflow_run.head_branch, '.') &&
!contains(github.event.workflow_run.head_branch, '-preview')
)
outputs:
raw_tag: ${{ steps.version.outputs.raw_tag }}
app_version: ${{ steps.version.outputs.app_version }}
chart_version: ${{ steps.version.outputs.chart_version }}
steps:
- name: Checkout release source
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
ref: ${{ github.event.workflow_run.head_sha || (startsWith(inputs.version, 'refs/tags/') && inputs.version || format('refs/tags/{0}', inputs.version)) }}
persist-credentials: false
# Both inputs reach the shell through env rather than `${{ }}`
# interpolation. A git ref name may contain `$(...)` — anything without a
# space is a legal tag — and interpolation pastes it into the script
# verbatim, where bash would run it. Reading "$RAW_INPUT" instead makes it
# data.
- name: Normalize release version
id: version
env:
RAW_INPUT: ${{ github.event.inputs.version }}
RAW_BRANCH: ${{ github.event.workflow_run.head_branch }}
run: |
set -eux
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
RAW="$RAW_INPUT"
else
RAW="$RAW_BRANCH"
fi
case "$RAW" in
refs/tags/*)
RAW_TAG="${RAW#refs/tags/}"
;;
*)
RAW_TAG="$RAW"
;;
esac
./scripts/helm_chart_version.sh "$RAW_TAG"
# Docker and Helm share the binary-build trigger. Do not publish a chart
# until the image selected by its default appVersion is pullable.
- name: Wait for release image
timeout-minutes: 20
env:
APP_VERSION: ${{ steps.version.outputs.app_version }}
run: |
set -euo pipefail
if [[ ! "$APP_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] ||
[[ "$APP_VERSION" == *-preview* ]]; then
echo "Invalid deliverable image version: $APP_VERSION" >&2
exit 1
fi
for attempt in {1..60}; do
if token=$(curl --fail --silent --show-error --connect-timeout 10 --max-time 20 \
'https://auth.docker.io/token?service=registry.docker.io&scope=repository:rustfs/rustfs:pull' \
| jq -er '.token // empty') &&
curl --fail --silent --show-error --head --connect-timeout 10 --max-time 20 \
--header "Authorization: Bearer $token" \
--header 'Accept: application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json' \
"https://registry-1.docker.io/v2/rustfs/rustfs/manifests/$APP_VERSION" > /dev/null; then
echo "Release image is available: rustfs/rustfs:$APP_VERSION"
exit 0
fi
echo "Waiting for rustfs/rustfs:$APP_VERSION ($attempt/60)"
if (( attempt < 60 )); then
sleep 15
fi
done
echo "Release image is unavailable; refusing to publish Helm chart" >&2
exit 1
- name: Replace chart version and app version
env:
CHART_VERSION: ${{ steps.version.outputs.chart_version }}
APP_VERSION: ${{ steps.version.outputs.app_version }}
run: |
set -eux
sed -i -E "s/^version:.*/version: \"${CHART_VERSION}\"/" helm/rustfs/Chart.yaml
sed -i -E "s/^appVersion:.*/appVersion: \"${APP_VERSION}\"/" helm/rustfs/Chart.yaml
- name: Set up Helm
uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
- name: Install yq
- uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
- name: Install pinned yq
run: |
set -euo pipefail
yq_version="4.54.1"
@@ -238,66 +35,43 @@ jobs:
echo "$tools_dir" >> "$GITHUB_PATH"
"$tools_dir/yq" --version
- name: Test Helm Chart Templates
run: ./scripts/test_helm_templates.sh
- name: Package Helm Chart
run: |
set -eux
cp helm/README.md helm/rustfs/
helm package ./helm/rustfs \
--destination helm/rustfs/ \
--version "${{ steps.version.outputs.chart_version }}"
- name: Upload helm package as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: helm-package
path: |
helm/rustfs/*.tgz
helm/rustfs/README.md
retention-days: 1
publish-helm-package:
runs-on: ubuntu-latest
timeout-minutes: 30
needs: [ build-helm-package ]
if: needs.build-helm-package.result == 'success'
steps:
- name: Checkout helm package repo
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
# persist-credentials-exempt: this checkout's token IS the push credential —
# the job git-pushes to rustfs/helm below. Clearing it breaks chart publishing.
repository: rustfs/helm
token: ${{ secrets.RUSTFS_HELM_PACKAGE }}
- name: Download helm package
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: helm-package
path: ./
- name: Set up helm
uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
- name: Generate index
run: helm repo index . --url https://charts.rustfs.com
# app_version is derived from the triggering tag name, and this job holds
# the cross-repository push token with rustfs/helm already checked out —
# the worst place in the repo to paste an attacker-influenced string into
# a shell line. Passed through env so bash treats it as data.
- name: Push helm package and index file
- name: Test and package the exact application version
id: chart
env:
GIT_USERNAME: ${{ secrets.USERNAME }}
GIT_EMAIL: ${{ secrets.EMAIL_ADDRESS }}
APP_VERSION: ${{ needs.build-helm-package.outputs.app_version }}
GH_TOKEN: ${{ github.token }}
CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
run: |
set -eux
git config --global user.name "${GIT_USERNAME}"
git config --global user.email "${GIT_EMAIL}"
git add .
git commit -m "Update rustfs helm package with ${APP_VERSION}." || echo "No changes to commit"
git push origin main
python3 - <<'PYCHART'
import sys, subprocess, shutil
from pathlib import Path
sys.path.insert(0, 'scripts/release')
import candidate as c, pipeline
inputs = pipeline.manifest_env()
current = pipeline.current_run('release-complete.yml')
if any(a['name'] == 'helm-package' for a in c.artifact_list(current['id'])):
c.download_artifact(current, 'helm-package', 'chart')
record = c.read_json('chart/chart.json')
c.require(record['inputs_sha256'] == c.digest(inputs), 'chart belongs to another candidate')
c.emit_output('reused', 'true')
else:
chart = Path('helm/rustfs/Chart.yaml')
import re
value = chart.read_text()
for key in ('version', 'appVersion'):
value, count = re.subn(rf'(?m)^{key}:.*$', f'{key}: "{inputs["version"]}"', value)
c.require(count == 1, 'ambiguous chart metadata')
chart.write_text(value)
subprocess.run(['bash', 'scripts/test_helm_templates.sh'], check=True)
shutil.copyfile('helm/README.md', 'helm/rustfs/README.md')
subprocess.run(['helm', 'package', 'helm/rustfs', '--destination', 'chart'], check=True)
path = Path('chart') / f"rustfs-{inputs['version']}.tgz"
c.write_json('chart/chart.json', {'version': inputs['version'], 'inputs_sha256': c.digest(inputs),
'name': path.name, 'sha256': c.file_digest(path), 'size': path.stat().st_size})
PYCHART
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: steps.chart.outputs.reused != 'true'
with:
name: helm-package
path: chart/*
if-no-files-found: error
retention-days: 30
+2 -2
View File
@@ -47,8 +47,8 @@ permissions:
contents: read
concurrency:
group: oidc-keycloak-live-${{ github.ref }}
cancel-in-progress: true
group: oidc-keycloak-live-${{ github.event_name }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name != 'schedule' }}
jobs:
oidc-keycloak-live:
+38 -825
View File
@@ -1,840 +1,53 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Package Workflow - Build DEB/RPM packages
#
# This workflow builds DEB and RPM packages from pre-built Linux binaries
# and uploads them to Cloudflare R2 and the GitHub release.
#
# Trigger:
# - workflow_run: automatically package after "Build and Release" completes
# for a release tag (the mac/windows/linux binaries are already uploaded
# to the GitHub release before packaging starts)
# - workflow_dispatch: manual fallback with a release tag and/or exact build run ID
#
# Flow:
# 1. Resolve and validate the selected Build workflow run and source identity
# 2. Download Linux binaries (x86_64-gnu, aarch64-gnu, x86_64-musl,
# aarch64-musl) from build artifacts
# 3. Build DEB packages for amd64 and arm64, gnu and musl variants
# 4. Build RPM packages for x86_64 and aarch64, gnu and musl variants
# 5. Upload all packages to Cloudflare R2 and the GitHub release
name: Package DEB/RPM
permissions:
# contents: write is required to upload packages to the GitHub release
contents: write
actions: read
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
name: Build candidate packages
on:
# Follows the same pattern as docker.yml: run after the release build
# workflow completes, so packaging is triggered only by release tags
# (e.g. 1.0.0-rc.2, 1.0.0-rc.3), never by development builds.
workflow_run:
workflows: [ "Build and Release" ]
types: [ completed ]
workflow_dispatch:
workflow_call:
inputs:
tag:
description: "Release tag to package (e.g. 1.0.0-beta.12). Leave empty for latest main build."
required: false
candidate_manifest:
type: string
build_run_id:
description: "Build workflow run ID (when combined with tag, both must identify the same release commit)"
required: false
type: string
concurrency:
group: ${{ github.workflow }}-${{ github.event.workflow_run.head_branch || github.event.inputs.tag || github.run_id }}
cancel-in-progress: true
env:
HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }}
WORKFLOW_RUN_ID: ${{ github.event.workflow_run.id }}
required: true
permissions:
contents: read
actions: read
jobs:
# Resolve which build run to use and extract version info
resolve:
name: Resolve Build
# Auto-trigger only from successful tag builds of "Build and Release".
# Tag pushes arrive as event == push with head_branch != main (a
# non-main push head_branch is the release tag name). Manual dispatch
# stays available as a fallback for backfills and re-runs.
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push' &&
github.event.workflow_run.head_branch != 'main')
runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
timeout-minutes: 10
outputs:
version: ${{ steps.resolve.outputs.version }}
build_type: ${{ steps.resolve.outputs.build_type }}
build_run_id: ${{ steps.resolve.outputs.build_run_id }}
build_run_number: ${{ steps.resolve.outputs.build_run_number }}
head_sha: ${{ steps.resolve.outputs.head_sha }}
dev_sequence: ${{ steps.resolve.outputs.dev_sequence }}
tag: ${{ steps.resolve.outputs.tag }}
steps:
- name: Resolve build run
id: resolve
shell: bash
env:
GH_TOKEN: ${{ github.token }}
EVENT_NAME: ${{ github.event_name }}
REPOSITORY: ${{ github.repository }}
INPUT_TAG: ${{ github.event.inputs.tag }}
INPUT_RUN_ID: ${{ github.event.inputs.build_run_id }}
run: |
set -euo pipefail
fail() {
echo "❌ $1" >&2
exit 1
}
TAG=""
BUILD_RUN_ID=""
case "$EVENT_NAME" in
workflow_run)
TAG="$HEAD_BRANCH"
BUILD_RUN_ID="$WORKFLOW_RUN_ID"
;;
workflow_dispatch)
TAG="$INPUT_TAG"
BUILD_RUN_ID="$INPUT_RUN_ID"
;;
*) fail "unsupported event: $EVENT_NAME" ;;
esac
# Validate and classify tags before using them in API paths or logs.
semver_core='(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)'
prerelease_id='(alpha|beta|rc)\.(0|[1-9][0-9]*)'
if [[ -n "$TAG" ]]; then
if [[ "$TAG" =~ ^${semver_core}(-${prerelease_id})?-preview\.(0|[1-9][0-9]*)$ ]]; then
BUILD_TYPE=preview
elif [[ "$TAG" =~ ^${semver_core}-${prerelease_id}$ ]]; then
BUILD_TYPE=prerelease
elif [[ "$TAG" =~ ^${semver_core}$ ]]; then
BUILD_TYPE=release
else
fail "tag is not a supported strict package version"
fi
else
BUILD_TYPE=development
fi
if [[ -n "$BUILD_RUN_ID" ]]; then
[[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ ]] || fail "build run ID must be a positive decimal integer"
echo "Using selected build run: $BUILD_RUN_ID"
elif [[ -n "$TAG" ]]; then
echo "Looking for build run for tag: $TAG"
BUILD_RUN_ID=$(gh api --method GET \
"repos/${REPOSITORY}/actions/workflows/build.yml/runs" \
-f branch="$TAG" -f status=success -F per_page=1 \
--jq '.workflow_runs[0].id // empty' 2>/dev/null || true)
if [[ -z "$BUILD_RUN_ID" ]]; then
BUILD_RUN_ID=$(gh api --method GET \
"repos/${REPOSITORY}/actions/workflows/build.yml/runs" \
-f event=push -f status=success -F per_page=100 2>/dev/null |
jq -r --arg tag "$TAG" \
'[.workflow_runs[] | select(.head_branch == $tag)][0].id // empty' || true)
fi
[[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ ]] || fail "no successful build run found for tag"
echo "Found build run: $BUILD_RUN_ID"
else
echo "No tag specified, looking for latest main build"
BUILD_RUN_ID=$(gh api --method GET \
"repos/${REPOSITORY}/actions/workflows/build.yml/runs" \
-f branch=main -f status=success -F per_page=1 \
--jq '.workflow_runs[0].id // empty' 2>/dev/null || true)
[[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ ]] || fail "no successful main build found"
echo "Latest main build: $BUILD_RUN_ID"
fi
# Fetch once and use the same immutable run metadata for identity,
# ordering, workflow provenance, and release-channel validation.
RUN_JSON=$(gh api "repos/${REPOSITORY}/actions/runs/${BUILD_RUN_ID}") ||
fail "cannot read selected build run"
RUN_ID=$(jq -r '.id // empty' <<<"$RUN_JSON")
RUN_NUMBER=$(jq -r '.run_number // empty' <<<"$RUN_JSON")
RUN_STATUS=$(jq -r '.status // empty' <<<"$RUN_JSON")
RUN_CONCLUSION=$(jq -r '.conclusion // empty' <<<"$RUN_JSON")
RUN_PATH=$(jq -r '.path // empty' <<<"$RUN_JSON")
HEAD_SHA=$(jq -r '.head_sha // empty' <<<"$RUN_JSON")
RUN_HEAD_BRANCH=$(jq -r '.head_branch // empty' <<<"$RUN_JSON")
[[ "$RUN_ID" == "$BUILD_RUN_ID" ]] || fail "run metadata ID mismatch"
[[ "$RUN_NUMBER" =~ ^[1-9][0-9]*$ ]] || fail "build run number must be a positive decimal integer"
[[ "$RUN_STATUS" == completed && "$RUN_CONCLUSION" == success ]] || fail "selected build run is not successful"
[[ "$RUN_PATH" == .github/workflows/build.yml ]] || fail "selected run is not Build and Release"
[[ "$HEAD_SHA" =~ ^[0-9a-f]{40}$ ]] || fail "selected build run has an invalid head SHA"
[[ "$RUN_HEAD_BRANCH" != *$'\n'* && -n "$RUN_HEAD_BRANCH" ]] || fail "selected build run has an invalid head branch"
if [[ -n "$TAG" ]]; then
[[ "$RUN_HEAD_BRANCH" == "$TAG" ]] || fail "tag and build run head branch do not match"
TAG_REF_JSON=$(gh api "repos/${REPOSITORY}/git/ref/tags/${TAG}") ||
fail "cannot resolve release tag ref"
TAG_OBJECT_TYPE=$(jq -r '.object.type // empty' <<<"$TAG_REF_JSON")
TAG_OBJECT_SHA=$(jq -r '.object.sha // empty' <<<"$TAG_REF_JSON")
depth=0
while [[ "$TAG_OBJECT_TYPE" == tag && $depth -lt 5 ]]; do
TAG_OBJECT_JSON=$(gh api "repos/${REPOSITORY}/git/tags/${TAG_OBJECT_SHA}") ||
fail "cannot peel annotated release tag"
TAG_OBJECT_TYPE=$(jq -r '.object.type // empty' <<<"$TAG_OBJECT_JSON")
TAG_OBJECT_SHA=$(jq -r '.object.sha // empty' <<<"$TAG_OBJECT_JSON")
depth=$((depth + 1))
done
[[ "$TAG_OBJECT_TYPE" == commit && "$TAG_OBJECT_SHA" =~ ^[0-9a-f]{40}$ ]] ||
fail "release tag does not resolve to a commit"
[[ "$TAG_OBJECT_SHA" == "$HEAD_SHA" ]] || fail "release tag commit and build run head SHA do not match"
VERSION="$TAG"
DEV_SEQUENCE=""
else
VERSION="dev-${HEAD_SHA}"
DEV_SEQUENCE="$RUN_NUMBER"
fi
{
echo "version=$VERSION"
echo "build_type=$BUILD_TYPE"
echo "build_run_id=$BUILD_RUN_ID"
echo "build_run_number=$RUN_NUMBER"
echo "head_sha=$HEAD_SHA"
echo "dev_sequence=$DEV_SEQUENCE"
echo "tag=${TAG}"
} >> "$GITHUB_OUTPUT"
echo "📊 Resolved:"
echo " Version: $VERSION"
echo " Build type: $BUILD_TYPE"
echo " Build run ID: $BUILD_RUN_ID"
echo " Build run number: $RUN_NUMBER"
# Consume the trusted release certification of the resolved build run and
# verify the certified input digests before any package is wrapped. A run
# without that certification fails closed here; packages built later must
# complete their own verification and cannot inherit an unverified
# conclusion.
release-certification:
name: Release Certification
needs: resolve
if: needs.resolve.result == 'success'
runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
timeout-minutes: 30
permissions:
contents: read
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Download and verify release certification
env:
GH_TOKEN: ${{ secrets.RELEASE_VALIDATION_TOKEN || github.token }}
BUILD_RUN_ID: ${{ needs.resolve.outputs.build_run_id }}
BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
shell: bash
run: |
set -euo pipefail
mkdir -p certification
if [[ "$BUILD_TYPE" == "development" ]]; then
# Development artifacts are explicitly uncertified and never touch
# official or latest channels; only the staging/dev paths remain.
jq -n '{schema_version: 1, certified: false, reason: "development artifacts are explicitly uncertified"}' \
> certification/development-uncertified.json
exit 0
fi
if ! gh run download "$BUILD_RUN_ID" -n release-certification -D certification; then
echo "Build run ${BUILD_RUN_ID} published no release-certification artifact; failing closed" >&2
exit 1
fi
python3 scripts/release/release_gate.py verify-promotion \
--plan .config/release-validation-plan.json \
--candidate certification/candidate.json \
--summary certification/release-gate-output \
--artifact-sha256 "$(jq -r '.deliverables[0].sha256' certification/candidate.json)" \
--artifact-name "$(jq -r '.deliverables[0].name' certification/candidate.json)"
# Build DEB and RPM packages for each architecture and libc variant
package:
name: Package (${{ matrix.arch }}-${{ matrix.libc }})
needs: [ resolve, release-certification ]
runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
timeout-minutes: 30
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
max-parallel: 1
matrix:
include:
- arch: x86_64
libc: gnu
deb_arch: amd64
rpm_arch: x86_64
artifact_name: "rustfs-linux-x86_64-gnu"
# gnu binaries dynamically link glibc; musl binaries are static
# and run without a libc dependency.
deb_depends: "libc6 (>= 2.31)"
rpm_depends: "glibc >= 2.31"
- arch: aarch64
libc: gnu
deb_arch: arm64
rpm_arch: aarch64
artifact_name: "rustfs-linux-aarch64-gnu"
deb_depends: "libc6 (>= 2.31)"
rpm_depends: "glibc >= 2.31"
- arch: x86_64
libc: musl
deb_arch: amd64
rpm_arch: x86_64
artifact_name: "rustfs-linux-x86_64-musl"
deb_depends: ""
rpm_depends: ""
- arch: aarch64
libc: musl
deb_arch: arm64
rpm_arch: aarch64
artifact_name: "rustfs-linux-aarch64-musl"
deb_depends: ""
rpm_depends: ""
target: [linux-x86_64-musl, linux-aarch64-musl, linux-x86_64-gnu, linux-aarch64-gnu]
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
ref: ${{ fromJson(inputs.candidate_manifest).source_sha }}
persist-credentials: false
- name: Normalize package metadata
id: versions
shell: bash
env:
BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
SOURCE_VERSION: ${{ needs.resolve.outputs.version }}
DEV_SEQUENCE: ${{ needs.resolve.outputs.dev_sequence }}
ARCH: ${{ matrix.rpm_arch }}
LIBC: ${{ matrix.libc }}
run: |
set -euo pipefail
normalized=$(./scripts/release/package_versions.sh \
"$BUILD_TYPE" "$SOURCE_VERSION" "$DEV_SEQUENCE" "$ARCH" "$LIBC")
printf '%s\n' "$normalized" >> "$GITHUB_OUTPUT"
- name: Download binary artifact from build run
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: ${{ matrix.artifact_name }}*
path: ./binary-artifact
run-id: ${{ needs.resolve.outputs.build_run_id }}
github-token: ${{ github.token }}
merge-multiple: true
- name: Extract binary
id: binary
shell: bash
run: |
set -euo pipefail
ZIP_FILE=$(find ./binary-artifact -name "*.zip" -type f | head -1)
if [[ -z "$ZIP_FILE" ]]; then
echo "❌ No binary artifact found"
find ./binary-artifact -mindepth 1 -maxdepth 1 -print 2>/dev/null || true
exit 1
fi
echo "Found artifact: $ZIP_FILE"
mkdir -p ./bin
unzip -o "$ZIP_FILE" -d ./bin
if [[ ! -f ./bin/rustfs ]]; then
echo "❌ rustfs binary not found in archive"
exit 1
fi
chmod +x ./bin/rustfs
stat --printf='%n %s bytes\n' ./bin/rustfs
echo "✅ Binary extracted"
- name: Build DEB package
id: deb
shell: bash
env:
DEB_VERSION: ${{ steps.versions.outputs.deb_version }}
DEB_ARCH: ${{ matrix.deb_arch }}
DEB_FILE: ${{ steps.versions.outputs.deb_file }}
DEB_DEPENDS: ${{ matrix.deb_depends }}
LIBC: ${{ matrix.libc }}
run: |
set -euo pipefail
PKG_DIR="${DEB_FILE%.deb}"
echo "Building DEB: ${DEB_FILE}"
mkdir -p "${PKG_DIR}/DEBIAN"
mkdir -p "${PKG_DIR}/usr/bin"
mkdir -p "${PKG_DIR}/etc/default"
mkdir -p "${PKG_DIR}/lib/systemd/system"
mkdir -p "${PKG_DIR}/usr/share/doc/rustfs"
cp ./bin/rustfs "${PKG_DIR}/usr/bin/"
chmod 755 "${PKG_DIR}/usr/bin/rustfs"
cp deploy/build/rustfs.service "${PKG_DIR}/lib/systemd/system/"
cat > "${PKG_DIR}/etc/default/rustfs" << 'ENVEOF'
# RustFS Environment Configuration
# See https://docs.rustfs.com for more information
# RUSTFS_ACCESS_KEY=<changeme>
# RUSTFS_SECRET_KEY=<changeme>
# RUSTFS_VOLUMES=""
# RUSTFS_ADDRESS=":9000"
# RUSTFS_CONSOLE_ADDRESS=":9001"
# RUSTFS_CONSOLE_ENABLE=true
# RUSTFS_OBS_LOGGER_LEVEL=error
# RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/"
ENVEOF
cat > "${PKG_DIR}/DEBIAN/control" << EOF
Package: rustfs
Version: ${DEB_VERSION}
Section: utils
Priority: optional
Architecture: ${DEB_ARCH}
EOF
# gnu binaries dynamically link glibc and must declare it; musl
# binaries are statically linked and need no libc dependency.
if [[ -n "$DEB_DEPENDS" ]]; then
echo "Depends: ${DEB_DEPENDS}" >> "${PKG_DIR}/DEBIAN/control"
fi
cat >> "${PKG_DIR}/DEBIAN/control" << EOF
Maintainer: RustFS Team <support@rustfs.com>
Description: High-performance distributed object storage (${LIBC} build)
RustFS is a high-performance distributed object storage software
built using Rust. It is compatible with MinIO and S3 API.
Homepage: https://rustfs.com
EOF
# Declare /etc/default/rustfs as a conffile so dpkg preserves user
# modifications on upgrade instead of silently overwriting them.
cat > "${PKG_DIR}/DEBIAN/conffiles" << 'CONFFILES'
/etc/default/rustfs
CONFFILES
./scripts/release/package_service_scripts.sh after-install > "${PKG_DIR}/DEBIAN/postinst"
chmod 755 "${PKG_DIR}/DEBIAN/postinst"
./scripts/release/package_service_scripts.sh before-remove > "${PKG_DIR}/DEBIAN/prerm"
chmod 755 "${PKG_DIR}/DEBIAN/prerm"
./scripts/release/package_service_scripts.sh after-remove > "${PKG_DIR}/DEBIAN/postrm"
chmod 755 "${PKG_DIR}/DEBIAN/postrm"
cp LICENSE "${PKG_DIR}/usr/share/doc/rustfs/"
cp README.md "${PKG_DIR}/usr/share/doc/rustfs/"
fakeroot dpkg-deb --build "${PKG_DIR}" "$DEB_FILE"
[[ $(dpkg-deb -f "$DEB_FILE" Package) == rustfs ]]
[[ $(dpkg-deb -f "$DEB_FILE" Version) == "$DEB_VERSION" ]]
[[ $(dpkg-deb -f "$DEB_FILE" Architecture) == "$DEB_ARCH" ]]
dpkg-deb --fsys-tarfile "$DEB_FILE" | tar -tf - | grep -Fx './usr/bin/rustfs' >/dev/null
stat --printf='%n %s bytes\n' "$DEB_FILE"
echo "deb_file=$DEB_FILE" >> "$GITHUB_OUTPUT"
echo "✅ DEB built: $DEB_FILE"
- name: Build RPM package
id: rpm
shell: bash
env:
RPM_VERSION: ${{ steps.versions.outputs.rpm_version }}
RPM_RELEASE: ${{ steps.versions.outputs.rpm_release }}
RPM_ARCH: ${{ matrix.rpm_arch }}
RPM_FILE: ${{ steps.versions.outputs.rpm_file }}
RPM_DEPENDS: ${{ matrix.rpm_depends }}
LIBC: ${{ matrix.libc }}
run: |
set -euo pipefail
echo "Building RPM for ${RPM_ARCH} (${LIBC})"
sudo apt-get update && sudo apt-get install -y ruby ruby-dev build-essential rpm
sudo gem install fpm
./scripts/test_package_versions.sh --require-package-managers
# gnu binaries dynamically link glibc and must declare it; musl
# binaries are statically linked and need no libc dependency.
depends_args=()
if [[ -n "$RPM_DEPENDS" ]]; then
depends_args+=(--depends "$RPM_DEPENDS")
fi
# Create config file for fpm (DEB build creates it in its package dir structure,
# but fpm needs the file to exist before packaging)
mkdir -p ./tmp-pkg/etc/default
cat > ./tmp-pkg/etc/default/rustfs << 'ENVEOF'
# RustFS Environment Configuration
# See https://docs.rustfs.com for more information
# RUSTFS_ACCESS_KEY=<changeme>
# RUSTFS_SECRET_KEY=<changeme>
# RUSTFS_VOLUMES=""
# RUSTFS_ADDRESS=":9000"
# RUSTFS_CONSOLE_ADDRESS=":9001"
# RUSTFS_CONSOLE_ENABLE=true
# RUSTFS_OBS_LOGGER_LEVEL=error
# RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/"
ENVEOF
fpm -s dir -t rpm \
--name rustfs \
--version "$RPM_VERSION" \
--iteration "$RPM_RELEASE" \
--architecture "$RPM_ARCH" \
--package "$RPM_FILE" \
"${depends_args[@]}" \
--description "High-performance distributed object storage (${LIBC} build)" \
--maintainer "RustFS Team <support@rustfs.com>" \
--url "https://rustfs.com" \
--license "Apache-2.0" \
--before-install <(./scripts/release/package_service_scripts.sh rpm-before-install) \
--before-upgrade <(./scripts/release/package_service_scripts.sh rpm-before-upgrade) \
--after-install <(./scripts/release/package_service_scripts.sh after-install) \
--after-upgrade <(./scripts/release/package_service_scripts.sh after-install) \
--before-remove <(./scripts/release/package_service_scripts.sh before-remove) \
--after-remove <(./scripts/release/package_service_scripts.sh after-remove) \
--rpm-posttrans <(./scripts/release/package_service_scripts.sh rpm-posttrans) \
--config-files /etc/default/rustfs \
./bin/rustfs=/usr/bin/rustfs \
./tmp-pkg/etc/default/rustfs=/etc/default/rustfs \
deploy/build/rustfs.service=/lib/systemd/system/rustfs.service \
LICENSE=/usr/share/doc/rustfs/LICENSE \
README.md=/usr/share/doc/rustfs/README.md
if [[ ! -f "$RPM_FILE" ]]; then
echo "❌ RPM build failed"
exit 1
fi
RPM_METADATA=$(rpm -qp --qf '%{NAME}\n%{VERSION}\n%{RELEASE}\n%{ARCH}\n' "$RPM_FILE")
EXPECTED_METADATA=$(printf 'rustfs\n%s\n%s\n%s' "$RPM_VERSION" "$RPM_RELEASE" "$RPM_ARCH")
[[ "$RPM_METADATA" == "$EXPECTED_METADATA" ]]
rpm -qpl "$RPM_FILE" | grep -Fx '/usr/bin/rustfs' >/dev/null
stat --printf='%n %s bytes\n' "$RPM_FILE"
echo "rpm_file=$RPM_FILE" >> "$GITHUB_OUTPUT"
echo "✅ RPM built: $RPM_FILE"
- name: Download certification evidence
if: needs.resolve.outputs.build_type != 'development'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: release-certification
path: ./certification
# Later-built wrapper verification: the binary inside each package must
# match the certified binary digest for this platform before anything is
# uploaded, and the package itself stays on staging until promotion.
- name: Verify wrapped binary against certification
if: needs.resolve.outputs.build_type != 'development'
shell: bash
env:
DEB_FILE: ${{ steps.deb.outputs.deb_file }}
RPM_FILE: ${{ steps.rpm.outputs.rpm_file }}
ARTIFACT_NAME: ${{ matrix.artifact_name }}
run: |
set -euo pipefail
expected="$(jq -r --arg needle "$ARTIFACT_NAME" \
'[.deliverables[] | select(.name | contains($needle)) | .binary_sha256][0] // empty' \
certification/candidate.json)"
if [[ -z "$expected" ]]; then
echo "No certified binary digest recorded for ${ARTIFACT_NAME}" >&2
exit 1
fi
if ! command -v rpm2cpio >/dev/null 2>&1; then
sudo apt-get update && sudo apt-get install -y rpm2cpio
fi
if ! command -v cpio >/dev/null 2>&1; then
sudo apt-get update && sudo apt-get install -y cpio
fi
for package_file in "$DEB_FILE" "$RPM_FILE"; do
case "$package_file" in
*.deb)
dpkg-deb --fsys-tarfile "$package_file" | tar -xOf - ./usr/bin/rustfs > "$RUNNER_TEMP/wrapped-rustfs"
;;
*.rpm)
rpm2cpio "$package_file" | cpio -i --to-stdout ./usr/bin/rustfs > "$RUNNER_TEMP/wrapped-rustfs"
;;
esac
python3 scripts/release/release_gate.py verify-promotion \
--plan .config/release-validation-plan.json \
--candidate certification/candidate.json \
--summary certification/release-gate-output \
--artifact "$RUNNER_TEMP/wrapped-rustfs" \
--artifact-role binary
done
- name: Upload packages to artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: packages-${{ matrix.arch }}-${{ matrix.libc }}
path: |
*.deb
*.rpm
retention-days: 30
- name: Upload packages to Cloudflare R2
if: needs.resolve.outputs.build_type != 'preview' && env.R2_ACCESS_KEY_ID != ''
env:
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
R2_BUCKET: ${{ secrets.R2_BUCKET }}
AWS_EC2_METADATA_DISABLED: true
BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
DEB_FILE: ${{ steps.deb.outputs.deb_file }}
RPM_FILE: ${{ steps.rpm.outputs.rpm_file }}
shell: bash
run: |
set -euo pipefail
if [[ -z "$R2_ACCESS_KEY_ID" || -z "$R2_SECRET_ACCESS_KEY" || -z "$R2_ENDPOINT" || -z "$R2_BUCKET" ]]; then
echo "⚠️ R2 credentials missing, skipping upload"
exit 0
fi
if ! command -v aws >/dev/null 2>&1; then
sudo apt-get update && sudo apt-get install -y awscli
fi
export AWS_ACCESS_KEY_ID="$R2_ACCESS_KEY_ID"
export AWS_SECRET_ACCESS_KEY="$R2_SECRET_ACCESS_KEY"
export AWS_DEFAULT_REGION="auto"
# Build-phase writes stay on staging for release types: the official
# packages/release and packages/latest paths are written only by the
# certification-gated promote-packages job, from these same staged
# objects. Development packages keep the uncertified dev channel.
if [[ "$BUILD_TYPE" == "development" ]]; then
R2_PREFIX="artifacts/rustfs/packages/dev"
else
R2_PREFIX="artifacts/rustfs/packages/staging/${GITHUB_RUN_ID}/${GITHUB_RUN_ATTEMPT}"
fi
R2_PATH="s3://${R2_BUCKET}/${R2_PREFIX}/"
echo "📤 Uploading to $R2_PATH"
for f in "$DEB_FILE" "$RPM_FILE"; do
if [[ -n "$f" && -f "$f" ]]; then
echo "Uploading: $f"
aws s3 cp "$f" "$R2_PATH" --endpoint-url "$R2_ENDPOINT" --only-show-errors
fi
done
echo "✅ Upload complete"
- name: Upload packages to GitHub Release
if: needs.resolve.outputs.tag != ''
- name: Reuse completed package set
id: reuse
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ needs.resolve.outputs.tag }}
DEB_FILE: ${{ steps.deb.outputs.deb_file }}
RPM_FILE: ${{ steps.rpm.outputs.rpm_file }}
shell: bash
CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
TARGET: ${{ matrix.target }}
run: python3 scripts/release/deliver.py reuse --kind packages --target "$TARGET"
- name: Install packaging tools
if: steps.reuse.outputs.reused != 'true'
run: |
set -euo pipefail
# Upload the packages, then refresh the release checksums so the new
# assets are covered, matching the binary release flow.
for f in "$DEB_FILE" "$RPM_FILE"; do
if [[ -n "$f" && -f "$f" ]]; then
echo "📤 Uploading $(basename "$f") to GitHub release ${TAG}..."
gh release upload "$TAG" "$f" --clobber
fi
done
CHECKSUM_DIR="$(mktemp -d)"
gh release download "$TAG" -p 'SHA256SUMS' -p 'SHA512SUMS' \
-D "$CHECKSUM_DIR" --clobber
for spec in "SHA256SUMS:sha256sum" "SHA512SUMS:sha512sum"; do
asset="${spec%%:*}"
checksum_cmd="${spec##*:}"
checksum_file="${CHECKSUM_DIR}/${asset}"
for f in "$DEB_FILE" "$RPM_FILE"; do
if [[ -n "$f" && -f "$f" ]]; then
base="$(basename "$f")"
# GitHub stores release asset names with '~' normalized to '.'.
# Current package file names embed the raw tag and contain no
# '~', but older releases may still ship such assets, so the
# checksum entries must reference the name as stored on the
# release.
github_base="${base//\~/.}"
# Remove any stale entry (both naming variants), then append
grep -Fv -- "$base" "$checksum_file" > "${checksum_file}.tmp" || true
grep -Fv -- "$github_base" "${checksum_file}.tmp" > "${checksum_file}.tmp2" || true
mv "${checksum_file}.tmp2" "$checksum_file"
digest=$("$checksum_cmd" -- "$f" | awk '{print $1}')
printf '%s %s\n' "$digest" "$github_base" >> "$checksum_file"
fi
done
echo "📤 Updating ${asset} for release ${TAG}..."
gh release upload "$TAG" "$checksum_file" --clobber
done
echo "✅ GitHub release assets updated"
# The only writer of the official packages/release and packages/latest R2
# paths. Promotion re-checks each package's wrapped binary against the
# passed certification list and copies the staged objects byte-for-byte:
# no rebuild-after-verify, no mutable latest resolution downstream.
promote-packages:
name: Promote Packages
needs: [ resolve, package, release-certification ]
# Target-channel lock for the official package paths.
concurrency:
group: release-channel-r2-packages
cancel-in-progress: false
if: needs.resolve.outputs.tag != '' && (needs.resolve.outputs.build_type == 'release' || needs.resolve.outputs.build_type == 'prerelease')
runs-on: ubuntu-latest # gh CLI required; the sm-standard-2 fleet does not ship it
timeout-minutes: 30
permissions:
contents: read
actions: read
env:
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
R2_BUCKET: ${{ secrets.R2_BUCKET }}
AWS_EC2_METADATA_DISABLED: true
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Download packages and certification
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: ./packages
pattern: packages-*
merge-multiple: true
- name: Download certification evidence
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: release-certification
path: ./certification
- name: Verify and promote staged packages
shell: bash
run: |
set -euo pipefail
if [[ -z "$R2_ACCESS_KEY_ID" || -z "$R2_SECRET_ACCESS_KEY" || -z "$R2_ENDPOINT" || -z "$R2_BUCKET" ]]; then
echo "❌ R2 credentials missing; refusing to promote to the official channel" >&2
exit 1
fi
STAGING_PREFIX="s3://${R2_BUCKET}/artifacts/rustfs/packages/staging/${GITHUB_RUN_ID}/${GITHUB_RUN_ATTEMPT}/"
RELEASE_PATH="s3://${R2_BUCKET}/artifacts/rustfs/packages/release/"
LATEST_PATH="s3://${R2_BUCKET}/artifacts/rustfs/packages/latest/"
if ! command -v aws >/dev/null 2>&1; then
sudo apt-get update && sudo apt-get install -y awscli
fi
export AWS_ACCESS_KEY_ID="$R2_ACCESS_KEY_ID"
export AWS_SECRET_ACCESS_KEY="$R2_SECRET_ACCESS_KEY"
export AWS_DEFAULT_REGION="auto"
if ! command -v rpm2cpio >/dev/null 2>&1; then
sudo apt-get install -y rpm2cpio
fi
if ! command -v cpio >/dev/null 2>&1; then
sudo apt-get install -y cpio
fi
promoted=0
for package_file in packages/*.deb packages/*.rpm; do
[[ -f "$package_file" ]] || continue
name="$(basename "$package_file")"
case "$package_file" in
*.deb)
dpkg-deb --fsys-tarfile "$package_file" | tar -xOf - ./usr/bin/rustfs > "$RUNNER_TEMP/wrapped-rustfs"
;;
*.rpm)
rpm2cpio "$package_file" | cpio -i --to-stdout ./usr/bin/rustfs > "$RUNNER_TEMP/wrapped-rustfs"
;;
esac
python3 scripts/release/release_gate.py verify-promotion \
--plan .config/release-validation-plan.json \
--candidate certification/candidate.json \
--summary certification/release-gate-output \
--artifact "$RUNNER_TEMP/wrapped-rustfs" \
--artifact-role binary
staged="$(mktemp)"
aws s3 cp "${STAGING_PREFIX}${name}" "$staged" --endpoint-url "$R2_ENDPOINT" --only-show-errors
staged_sha="$(sha256sum "$staged" | awk '{print $1}')"
built_sha="$(sha256sum "$package_file" | awk '{print $1}')"
if [[ "$staged_sha" != "$built_sha" ]]; then
echo "❌ Staged package ${name} does not match the verified bytes" >&2
exit 1
fi
aws s3 cp "$staged" "${RELEASE_PATH}${name}" --endpoint-url "$R2_ENDPOINT" --only-show-errors
aws s3 cp "$staged" "${LATEST_PATH}${name}" --endpoint-url "$R2_ENDPOINT" --only-show-errors
rm -f "$staged"
promoted=$((promoted + 1))
echo "✅ Promoted ${name} (${built_sha})"
done
if [[ "$promoted" -eq 0 ]]; then
echo "No staged packages found to promote" >&2
exit 1
fi
# Summary
summary:
name: Summary
needs: [ resolve, package ]
# Every completed Build and Release run triggers this workflow; when
# resolve declined it (not a release tag), there is nothing to summarise.
if: always() && needs.resolve.result != 'skipped'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Print summary
shell: bash
sudo apt-get update
sudo apt-get install -y ruby ruby-dev build-essential rpm fakeroot cpio
sudo gem install fpm --version 1.17.0 --no-document
- name: Package accepted executable and verify contents
if: steps.reuse.outputs.reused != 'true'
env:
SUMMARY_VERSION: ${{ needs.resolve.outputs.version }}
SUMMARY_BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
SUMMARY_BUILD_RUN_ID: ${{ needs.resolve.outputs.build_run_id }}
SUMMARY_PACKAGE_STATUS: ${{ needs.package.result }}
run: |
{
echo "## 📦 Package Summary"
echo ""
echo "| Item | Value |"
echo "|------|-------|"
echo "| Version | \`${SUMMARY_VERSION}\` |"
echo "| Build Type | ${SUMMARY_BUILD_TYPE} |"
echo "| Build Run | #${SUMMARY_BUILD_RUN_ID} |"
echo "| Package Status | ${SUMMARY_PACKAGE_STATUS} |"
} >> "$GITHUB_STEP_SUMMARY"
GH_TOKEN: ${{ github.token }}
CANDIDATE_MANIFEST: ${{ inputs.candidate_manifest }}
TARGET: ${{ matrix.target }}
run: python3 scripts/release/deliver.py package --target "$TARGET" --output delivery
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: steps.reuse.outputs.reused != 'true'
with:
name: packages-${{ matrix.target }}
path: delivery/*
if-no-files-found: error
compression-level: 0
retention-days: 30
+308
View File
@@ -0,0 +1,308 @@
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# s3-tests reference parity lane (rustfs/backlog#2773).
#
# Runs the ceph/s3-tests implemented list (scripts/s3-tests/implemented_tests.txt)
# against every target in .config/reference-parity-targets.toml and fails on
# any per-node-id difference between a candidate and the reference:
#
# - reference: the release pinned in .config/reference-binary.toml, downloaded
# and sha256-verified by .github/actions/reference-binary;
# - candidate targets: the debug binary built from this checkout exactly as
# ci.yml's build-rustfs-debug-binary job builds it for S3 Implemented Tests.
#
# Every target runs the same harness (scripts/s3-tests/run.sh from this
# checkout), test list and server configuration on its own hosted runner. A
# target's failures get one rerun against a fresh server; a case that passes it
# is flaky, not a difference. scripts/s3-tests/reference_parity.py judges the
# result and honours .config/reference-parity-tolerated.txt, whose entries must
# each match a real difference.
#
# Report-only: not part of the Test and Lint aggregate. Scheduled failures are
# owned by the scheduled-validation watchdog (.github/scheduled-validations.json).
name: Reference Parity
on:
pull_request:
paths:
- ".github/workflows/reference-parity.yml"
- ".github/actions/reference-binary/**"
- ".config/reference-binary.toml"
- ".config/reference-parity-targets.toml"
- ".config/reference-parity-tolerated.txt"
- "scripts/s3-tests/reference_parity.py"
- "scripts/junit_diff.py"
schedule:
- cron: "37 1 * * *"
workflow_dispatch:
# Matches ci.yml: these variables are hashed into the rust-cache key, so the
# candidate build reads the same ci-dev cache as build-rustfs-debug-binary.
env:
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
defaults:
run:
shell: bash
jobs:
plan:
name: Plan parity targets
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
targets: ${{ steps.targets.outputs.targets }}
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Validate target list
id: targets
run: |
set -euo pipefail
targets="$(python3 scripts/s3-tests/reference_parity.py targets --config .config/reference-parity-targets.toml)"
echo "targets=${targets}" >> "$GITHUB_OUTPUT"
echo "Parity targets: ${targets}"
build-candidate:
name: Build candidate binary
# Critical-path CPU and cache ownership: the same debug build as ci.yml's
# build-rustfs-debug-binary, reading the ci-dev cache written on sm-standard-4.
runs-on: sm-standard-4
timeout-minutes: 60
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Setup Rust environment
uses: ./.github/actions/setup
with:
rust-version: stable
cache-shared-key: ci-dev
cache-save-if: 'false'
install-build-packaging-tools: 'false'
- name: Build debug binary
run: python3 scripts/e2e_binary.py build --bins --features e2e-test-hooks
- name: Upload candidate binary
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: reference-parity-candidate-binary
overwrite: true
path: |
target/debug/rustfs
target/debug/rustfs.e2e.json
if-no-files-found: error
retention-days: 1
s3tests:
name: s3-tests (${{ matrix.target.name }})
needs: [ plan, build-candidate ]
runs-on: ubuntu-latest
timeout-minutes: 75
strategy:
fail-fast: false
matrix:
target: ${{ fromJSON(needs.plan.outputs.targets) }}
env:
TARGET_NAME: ${{ matrix.target.name }}
TARGET_KIND: ${{ matrix.target.binary }}
# Above this many first-run failures the target is systemically broken;
# a rerun would only repeat it, so the first run is judged as is.
RERUN_CAP: "50"
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Fetch pinned reference binary
id: reference
if: matrix.target.binary == 'reference'
uses: ./.github/actions/reference-binary
- name: Download candidate binary
if: matrix.target.binary == 'candidate'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: reference-parity-candidate-binary
path: target/debug
- name: Select target binary
env:
REFERENCE_PATH: ${{ steps.reference.outputs.path }}
run: |
set -euo pipefail
case "$TARGET_KIND" in
reference) binary="$REFERENCE_PATH" ;;
candidate) chmod +x target/debug/rustfs; binary="$PWD/target/debug/rustfs" ;;
*) echo "unknown target binary kind: $TARGET_KIND" >&2; exit 1 ;;
esac
test -x "$binary"
echo "TARGET_BINARY=$binary" >> "$GITHUB_ENV"
mkdir -p "parity/$TARGET_NAME"
{
echo "target=$TARGET_NAME"
echo "binary=$TARGET_KIND"
echo "sha256=$(sha256sum "$binary" | cut -d' ' -f1)"
echo "version=$("$binary" --version | tr '\n' ' ')"
} | tee "parity/$TARGET_NAME/target.txt"
- name: Apply target environment
env:
TARGET_JSON: ${{ toJSON(matrix.target) }}
run: |
set -euo pipefail
python3 scripts/s3-tests/reference_parity.py export-env --target-json "$TARGET_JSON" \
| tee -a "$GITHUB_ENV" "parity/$TARGET_NAME/target-env.txt"
- name: Run implemented s3-tests
run: |
set -uo pipefail
report="parity/$TARGET_NAME"
harness="artifacts/s3tests-single"
# Same invocation as ci.yml's S3 Implemented Tests, with a fresh data
# root and port per run. $1 optionally replaces the test list.
run_suite() {
local list="$1" label="$2" root port
root="${RUNNER_TEMP}/rustfs-s3tests-${label}"
rm -rf "$harness" "$root"
mkdir -p "$root"
port="$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1]); s.close()')"
env DEPLOY_MODE=binary \
RUSTFS_BINARY="$TARGET_BINARY" \
TEST_MODE=single \
MAXFAIL=0 \
XDIST=4 \
S3_PORT="$port" \
DATA_ROOT="$root" \
S3TESTS_CONF="$harness/s3tests.conf" \
${list:+IMPLEMENTED_TESTS_FILE="$list"} \
./scripts/s3-tests/run.sh
}
# A run is usable only if it produced a JUnit report; test failures
# are data for the comparison, a missing report is a lane failure.
keep() {
local prefix="$1"
for file in pytest.log compat-report.md; do
if [[ -f "$harness/$file" ]]; then cp "$harness/$file" "$report/${prefix}${file}"; fi
done
if [[ -f "$harness/rustfs-single/rustfs.log" ]]; then cp "$harness/rustfs-single/rustfs.log" "$report/${prefix}rustfs.log"; fi
test -s "$harness/junit.xml" || { echo "::error::$TARGET_NAME produced no JUnit report (${prefix:-first} run)"; return 1; }
cp "$harness/junit.xml" "$report/${prefix}junit.xml"
}
run_suite "" first
echo "first_run_exit=$?" >> "$report/target.txt"
keep "" || exit 1
cp "$harness/selected-nodeids.txt" "$report/selected-nodeids.txt" || exit 1
failures="$(python3 scripts/s3-tests/reference_parity.py failed --report "$report")" || exit 1
count="$(printf '%s' "$failures" | grep -c . || true)"
echo "first_run_failures=$count" >> "$report/target.txt"
if (( count == 0 )); then
exit 0
fi
if (( count > RERUN_CAP )); then
echo "::warning::$TARGET_NAME has $count failures (cap $RERUN_CAP); judging the first run without a rerun"
echo "rerun=skipped-over-cap" >> "$report/target.txt"
exit 0
fi
printf '%s\n' "$failures" > "$RUNNER_TEMP/rerun-tests.txt"
run_suite "$RUNNER_TEMP/rerun-tests.txt" rerun
echo "rerun_exit=$?" >> "$report/target.txt"
keep rerun- || exit 1
- name: Upload target report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: reference-parity-report-${{ matrix.target.name }}
overwrite: true
path: parity/${{ matrix.target.name }}/
if-no-files-found: error
retention-days: 14
compare:
name: Compare targets
needs: [ plan, s3tests ]
if: always() && needs.plan.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Download target reports
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: reference-parity-report-*
path: parity-download
- name: Compare per node id
env:
TARGETS: ${{ needs.plan.outputs.targets }}
S3TESTS_RESULT: ${{ needs.s3tests.result }}
run: |
set -uo pipefail
mkdir -p parity
for dir in parity-download/reference-parity-report-*; do
if [[ -d "$dir" ]]; then mv "$dir" "parity/${dir#parity-download/reference-parity-report-}"; fi
done
python3 scripts/s3-tests/reference_parity.py compare \
--targets-json "$TARGETS" \
--reports parity \
--tolerated .config/reference-parity-tolerated.txt \
--markdown "$GITHUB_STEP_SUMMARY" \
--json parity/diff.json > parity/diff.md
status=$?
cat parity/diff.md
if [[ "$S3TESTS_RESULT" != "success" ]]; then
echo "::error::s3-tests target jobs finished with '$S3TESTS_RESULT'"
status=1
fi
exit "$status"
- name: Upload parity diff
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: reference-parity-diff
overwrite: true
path: |
parity/diff.md
parity/diff.json
if-no-files-found: warn
retention-days: 14
+133
View File
@@ -0,0 +1,133 @@
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
name: Release candidate (Linux)
on:
workflow_dispatch:
inputs:
version:
description: Final product version (no preview suffix)
type: string
required: true
source_sha:
description: Full reviewed source commit
type: string
required: true
permissions:
contents: read
actions: read
concurrency:
group: release-candidate-${{ inputs.version }}
cancel-in-progress: false
jobs:
freeze:
if: github.repository == 'rustfs/rustfs' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
timeout-minutes: 15
outputs:
manifest: ${{ steps.freeze.outputs.manifest }}
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Freeze or recover original inputs
id: freeze
env:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ inputs.version }}
SOURCE_SHA: ${{ inputs.source_sha }}
run: python3 scripts/release/pipeline.py prepare --version "$VERSION" --source "$SOURCE_SHA" --directory frozen
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: steps.freeze.outputs.reused != 'true'
with:
name: release-inputs
path: frozen/inputs.json
if-no-files-found: error
retention-days: 30
linux:
needs: freeze
uses: ./.github/workflows/build.yml
with:
candidate_manifest: ${{ needs.freeze.outputs.manifest }}
platforms: linux
upgrade:
needs: [freeze, linux]
runs-on: ubuntu-latest
timeout-minutes: 150
env:
RUSTUP_TOOLCHAIN: ${{ fromJson(needs.freeze.outputs.manifest).toolchain }}
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
ref: ${{ fromJson(needs.freeze.outputs.manifest).workflow_sha }}
persist-credentials: false
- uses: ./.github/actions/setup
with:
rust-version: ${{ fromJson(needs.freeze.outputs.manifest).toolchain }}
cache-shared-key: release-upgrade-harness-hosted
cache-save-if: ${{ github.ref == 'refs/heads/main' }}
install-build-packaging-tools: "false"
install-test-tools: "false"
- name: Verify archived server against previous releases
env:
GH_TOKEN: ${{ github.token }}
CANDIDATE_MANIFEST: ${{ needs.freeze.outputs.manifest }}
run: |
python3 - <<'PYVERIFY'
import sys
from pathlib import Path
sys.path.insert(0, 'scripts/release')
import candidate as c, pipeline, verify
inputs = pipeline.manifest_env()
run = pipeline.current_run('release-candidate.yml')
if any(a['name'] == 'release-upgrade' for a in c.artifact_list(run['id'])):
c.download_artifact(run, 'release-upgrade', 'receipts')
pipeline.verify_evidence(inputs, Path('receipts'), {run['id']: run})
c.emit_output('reused', 'true')
else:
target = 'linux-x86_64-gnu'
c.download_artifact(run, f"candidate-{inputs['candidate_id']}-{target}", 'binary')
verify.upgrade(inputs, Path('binary') / f"rustfs-{target}-v{inputs['version']}.zip", Path('receipts'))
PYVERIFY
id: verify
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: steps.verify.outputs.reused != 'true'
with:
name: release-upgrade
path: receipts/*
if-no-files-found: error
retention-days: 30
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: release-upgrade-failure-${{ github.run_attempt }}
path: receipts/*
retention-days: 7
seal-linux:
needs: [freeze, linux, upgrade]
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Collect trusted receipts and seal Linux bytes
id: seal
env:
GH_TOKEN: ${{ github.token }}
run: |
python3 scripts/release/pipeline.py linux-seal --directory sealed
{
echo '## Linux acceptance'
echo 'Use this run ID and manifest SHA-256 when starting Release candidate (complete).'
sha256sum sealed/linux-manifest.json
echo 'Record cluster, performance, integrity and upgrade acceptance in backlog before continuing.'
} >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: steps.seal.outputs.reused != 'true'
with:
name: release-linux
path: sealed/*
if-no-files-found: error
compression-level: 0
retention-days: 30
+97
View File
@@ -0,0 +1,97 @@
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
name: Release candidate (complete)
on:
workflow_dispatch:
inputs:
candidate_run_id:
description: Successful Linux candidate run ID
type: string
required: true
linux_manifest_sha256:
description: Exact accepted Linux manifest SHA-256
type: string
required: true
acceptance_report:
description: Backlog report accepting cluster, performance, integrity and upgrade results
type: string
required: true
permissions:
contents: read
actions: read
concurrency:
group: release-complete-${{ inputs.candidate_run_id }}
cancel-in-progress: false
jobs:
acceptance:
if: github.repository == 'rustfs/rustfs' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
timeout-minutes: 30
environment: release-linux-acceptance
outputs:
manifest: ${{ steps.accept.outputs.manifest }}
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Bind acceptance to the exact Linux manifest
id: accept
env:
GH_TOKEN: ${{ github.token }}
CANDIDATE_RUN: ${{ inputs.candidate_run_id }}
MANIFEST_SHA: ${{ inputs.linux_manifest_sha256 }}
REPORT: ${{ inputs.acceptance_report }}
run: python3 scripts/release/pipeline.py complete-prepare --run-id "$CANDIDATE_RUN" --digest "$MANIFEST_SHA" --report "$REPORT" --directory linux
desktop:
needs: acceptance
uses: ./.github/workflows/build.yml
with:
candidate_manifest: ${{ needs.acceptance.outputs.manifest }}
platforms: desktop
packages:
needs: acceptance
uses: ./.github/workflows/package.yml
with:
candidate_manifest: ${{ needs.acceptance.outputs.manifest }}
images:
needs: acceptance
uses: ./.github/workflows/docker.yml
with:
candidate_manifest: ${{ needs.acceptance.outputs.manifest }}
chart:
needs: acceptance
uses: ./.github/workflows/helm-package.yml
with:
candidate_manifest: ${{ needs.acceptance.outputs.manifest }}
seal:
needs: [acceptance, desktop, packages, images, chart]
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Verify every deliverable and seal the candidate
id: seal
env:
GH_TOKEN: ${{ github.token }}
CANDIDATE_RUN: ${{ inputs.candidate_run_id }}
MANIFEST_SHA: ${{ inputs.linux_manifest_sha256 }}
REPORT: ${{ inputs.acceptance_report }}
run: |
python3 scripts/release/pipeline.py complete-seal --run-id "$CANDIDATE_RUN" --digest "$MANIFEST_SHA" --report "$REPORT" --directory sealed
{
echo '## Final publication approval'
echo 'Review this manifest, then explicitly dispatch Publish accepted release with its SHA-256.'
sha256sum sealed/release-manifest.json
} >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: steps.seal.outputs.reused != 'true'
with:
name: release-seal
path: sealed/*
if-no-files-found: error
compression-level: 0
retention-days: 30
+193
View File
@@ -0,0 +1,193 @@
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
name: Publish accepted release
on:
workflow_dispatch:
inputs:
complete_run_id:
description: Successful completed candidate run ID
type: string
required: true
manifest_sha256:
description: Exact approved final manifest SHA-256 (authorizes publication)
type: string
required: true
permissions:
contents: read
actions: read
concurrency:
group: release-publish-${{ inputs.complete_run_id }}
cancel-in-progress: false
jobs:
verify:
if: github.repository == 'rustfs/rustfs' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
timeout-minutes: 45
outputs:
version: ${{ steps.verify.outputs.version }}
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Recheck approved bytes and trusted certification
id: verify
env:
GH_TOKEN: ${{ github.token }}
COMPLETE_RUN: ${{ inputs.complete_run_id }}
MANIFEST_SHA: ${{ inputs.manifest_sha256 }}
run: python3 scripts/release/publish.py verify --run-id "$COMPLETE_RUN" --digest "$MANIFEST_SHA" --directory sealed
version:
needs: verify
runs-on: ubuntu-latest
timeout-minutes: 90
concurrency:
group: release-version-${{ needs.verify.outputs.version }}
cancel-in-progress: false
environment: release-publication
permissions:
contents: write
actions: read
packages: write
security-events: write
env:
GH_TOKEN: ${{ github.token }}
COMPLETE_RUN: ${{ inputs.complete_run_id }}
MANIFEST_SHA: ${{ inputs.manifest_sha256 }}
AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: auto
AWS_REQUEST_CHECKSUM_CALCULATION: when_required
AWS_RESPONSE_CHECKSUM_VALIDATION: when_required
AWS_EC2_METADATA_DISABLED: "true"
R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
R2_BUCKET: ${{ secrets.R2_BUCKET }}
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Install promotion tools
run: |
sudo apt-get update
sudo apt-get install -y skopeo
aws --version
- uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
username: rustfs
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
password: ${{ secrets.GHCR_PASSWORD }}
- uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
registry: quay.io
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_PASSWORD }}
- name: Share registry credentials with the OCI copier
run: |
install -m 600 "$HOME/.docker/config.json" "$RUNNER_TEMP/registry-auth.json"
echo "REGISTRY_AUTH_FILE=$RUNNER_TEMP/registry-auth.json" >> "$GITHUB_ENV"
- name: Promote verified version
run: python3 scripts/release/publish.py version --run-id "$COMPLETE_RUN" --digest "$MANIFEST_SHA" --directory sealed
- name: Publish accepted musl amd64 scan
uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4
with:
sarif_file: sealed/scan-musl-amd64.sarif
category: container-image-musl-amd64
- name: Publish accepted musl arm64 scan
uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4
with:
sarif_file: sealed/scan-musl-arm64.sarif
category: container-image-musl-arm64
- name: Publish accepted GNU amd64 scan
uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4
with:
sarif_file: sealed/scan-gnu-amd64.sarif
category: container-image-gnu-amd64
- name: Publish accepted GNU arm64 scan
uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4
with:
sarif_file: sealed/scan-gnu-arm64.sarif
category: container-image-gnu-arm64
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-version-receipt-${{ github.run_attempt }}
path: publication.json
if-no-files-found: error
retention-days: 90
channels:
needs: [verify, version]
runs-on: ubuntu-latest
timeout-minutes: 90
concurrency:
group: rustfs-release-channels
cancel-in-progress: false
permissions:
contents: write
actions: read
packages: write
env:
GH_TOKEN: ${{ github.token }}
COMPLETE_RUN: ${{ inputs.complete_run_id }}
MANIFEST_SHA: ${{ inputs.manifest_sha256 }}
AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: auto
AWS_REQUEST_CHECKSUM_CALCULATION: when_required
AWS_RESPONSE_CHECKSUM_VALIDATION: when_required
AWS_EC2_METADATA_DISABLED: "true"
R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
R2_BUCKET: ${{ secrets.R2_BUCKET }}
steps:
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
fetch-depth: 0
persist-credentials: false
- name: Install promotion tools
run: |
sudo apt-get update
sudo apt-get install -y skopeo
aws --version
- uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
username: rustfs
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
password: ${{ secrets.GHCR_PASSWORD }}
- uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
registry: quay.io
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_PASSWORD }}
- name: Share registry credentials with the OCI copier
run: |
install -m 600 "$HOME/.docker/config.json" "$RUNNER_TEMP/registry-auth.json"
echo "REGISTRY_AUTH_FILE=$RUNNER_TEMP/registry-auth.json" >> "$GITHUB_ENV"
- uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
- name: Install pinned OSS client
run: |
curl --fail --location --retry 3 --output "$RUNNER_TEMP/ossutil.zip" https://gosspublic.alicdn.com/ossutil/v2/2.1.1/ossutil-2.1.1-linux-amd64.zip
printf '%s %s\n' 60f3a808cbbdfd4b5269b8f967c6a66f564c9dacff52d93a5d21a588976ab5a2 "$RUNNER_TEMP/ossutil.zip" | sha256sum --check -
unzip -q "$RUNNER_TEMP/ossutil.zip" -d "$RUNNER_TEMP/ossutil-extract"
install -m 755 "$RUNNER_TEMP/ossutil-extract/ossutil-2.1.1-linux-amd64/ossutil" "$RUNNER_TEMP/ossutil"
echo "OSSUTIL_BIN=$RUNNER_TEMP/ossutil" >> "$GITHUB_ENV"
- name: Promote verified channels
env:
HELM_TOKEN: ${{ secrets.RUSTFS_HELM_PACKAGE }}
OSS_ACCESS_KEY_ID: ${{ secrets.ALICLOUDOSS_KEY_ID }}
OSS_ACCESS_KEY_SECRET: ${{ secrets.ALICLOUDOSS_KEY_SECRET }}
OSS_REGION: cn-beijing
OSS_ENDPOINT: https://oss-cn-beijing.aliyuncs.com
run: python3 scripts/release/publish.py channels --run-id "$COMPLETE_RUN" --digest "$MANIFEST_SHA" --directory sealed
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-channels-receipt-${{ github.run_attempt }}
path: publication.json
if-no-files-found: error
retention-days: 90
@@ -26,7 +26,13 @@ permissions:
jobs:
alert-on-incomplete-run:
name: Alert on incomplete scheduled run
if: github.repository == 'rustfs/rustfs'
# The registry still owns responsibility and cadence selection. Successful
# and PR runs can never produce an incomplete scheduled-run alert.
if: >-
github.repository == 'rustfs/rustfs' &&
github.event.workflow_run.conclusion != 'success' &&
(github.event.workflow_run.event == 'schedule' ||
github.event.workflow_run.event == 'workflow_run')
# Stay on GitHub-hosted runners: this monitor must keep running when the
# self-hosted sm-standard-* pool is down, which is exactly when the
# scheduled validations it watches stall.
Generated
+40 -49
View File
@@ -322,9 +322,9 @@ dependencies = [
[[package]]
name = "archive-trait"
version = "0.0.15"
version = "0.0.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "999bf88cc483664cab77700ca2fdfb0415a73412abe284ef5bc2319d00938e00"
checksum = "0b5040ca935c09888e7dfb04383b4d62fa3571e25bc66b6f345b591e97958524"
dependencies = [
"cap-std",
"thiserror 2.0.21",
@@ -811,9 +811,9 @@ dependencies = [
[[package]]
name = "async-nats"
version = "0.51.0"
version = "0.52.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4f92759f418774b78c36ef07885849150ce950c67d9bcdeff8a9c253f57d9a1"
checksum = "015f0a7a3b0b29701b78c76a4d4a122624c97c71fa30363381664eb729d619c7"
dependencies = [
"aws-lc-rs",
"base64 0.22.1",
@@ -3906,9 +3906,9 @@ dependencies = [
[[package]]
name = "dial9"
version = "0.5.7"
version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f3298a54fd82af243c9f122595762f5b0e48bd3d1ea8fe3f3eb5e95c130e881"
checksum = "c83757d61dc75b3e9d3579a3004c7fe0c42aff2f779ca9eeec7f0c8e1ae3fc10"
dependencies = [
"bon",
"dial9-core",
@@ -3990,9 +3990,9 @@ dependencies = [
[[package]]
name = "dial9-tokio-telemetry"
version = "0.5.7"
version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0428cba60fbcf37ab82666f3a7ef3b6d0671e90dfb6a352376813b471319f061"
checksum = "6f0eb05627c12c57e40510e63f9a3d5f628ab4b32fec9ccdb0ae98544c13f15e"
dependencies = [
"bon",
"bytes",
@@ -4525,9 +4525,9 @@ dependencies = [
[[package]]
name = "flate2"
version = "1.1.10"
version = "1.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
checksum = "626d4928326b646e70f65ed36fc7cf9f206bdb68a0019ee7c6b4709da4e084c4"
dependencies = [
"crc32fast",
"miniz_oxide",
@@ -4594,7 +4594,7 @@ checksum = "94e7099f6313ecacbe1256e8ff9d617b75d1bcb16a6fddef94866d225a01a14a"
dependencies = [
"io-lifetimes 2.0.4",
"rustix",
"windows-sys 0.59.0",
"windows-sys 0.52.0",
]
[[package]]
@@ -5385,9 +5385,9 @@ dependencies = [
[[package]]
name = "hotpath"
version = "0.30.3"
version = "0.31.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9632c1fd213e63c1b88d8f3e59543da4999b17f859cc1e1d67cf8f2207e8935"
checksum = "4039758c8bca06a34c475d17aaafc17ac1073ce2cecd8dba8c82e69edb300bbe"
dependencies = [
"arc-swap",
"async-channel",
@@ -5420,15 +5420,15 @@ dependencies = [
[[package]]
name = "hotpath-drain"
version = "0.30.3"
version = "0.31.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "738324f75aec847dea984f6806402f4f1bdfab437b4700fc8c3ebcad2dff6424"
checksum = "bbba0a3badfb9ebf95f22dc6cdb4afcd0ff07b91a85fa3196fd1060576ea3065"
[[package]]
name = "hotpath-macros"
version = "0.30.3"
version = "0.31.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d92fdfec803932e5764ea93c201cf7c6c2d8cff1871da44f64f5094c9c3ef8a"
checksum = "7d980014875338771b00ac44f1c9e2ea1a10650aa68ce54226233bc767714361"
dependencies = [
"proc-macro2",
"quote",
@@ -6412,9 +6412,9 @@ dependencies = [
[[package]]
name = "lzma-rust2"
version = "0.24.0"
version = "0.25.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "490750560fa7851fbbf5d3ab1a8750f09350ed361ed8aed85b1e1e40587812ee"
checksum = "ef1d73c59a56771343712b22ea5615092c612c16ba291c0979e3105dfeec4890"
dependencies = [
"sha2 0.11.1",
]
@@ -10388,18 +10388,18 @@ dependencies = [
[[package]]
name = "rustfs-mimalloc"
version = "0.7.3"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "216ecfc9341b5d6e2506d47ce57c9232ae1b8020bb10264ea05286fb234286e7"
checksum = "fc7914232a854336c62058c24a8362c553ac8e7adba2e2395ce9097db24d1510"
dependencies = [
"rustfs-mimalloc-sys",
]
[[package]]
name = "rustfs-mimalloc-sys"
version = "0.7.3"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb16ad68f96883407a75cdb88252b43dc2ea592428f0fc54ff7885756de4b814"
checksum = "8073624242781448512682f1931624ee67108b6b7873b22fb29c1a1df689ac0c"
dependencies = [
"cc",
]
@@ -11773,9 +11773,9 @@ dependencies = [
[[package]]
name = "serde_spanned"
version = "1.1.2"
version = "1.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4e7523beb55eece201a2356bee0bbca0d1ab466c14c07703b2e0ee6d42cb0c2c"
checksum = "9c187e0a369471c550af028cba56d4f4fa2a489ad28151ae95a4d340a2a8b782"
dependencies = [
"serde_core",
]
@@ -12105,9 +12105,9 @@ dependencies = [
[[package]]
name = "smallvec"
version = "1.16.4"
version = "1.16.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0d5d4b1981b8bbbb7aed204cae5726d683d5d6ac7e74ec54122ea28f315e7c8"
checksum = "149309498a5f7334b12ddae0cad8aba81ce785fe91755068867e017cf98fee46"
dependencies = [
"serde",
]
@@ -12302,9 +12302,9 @@ dependencies = [
[[package]]
name = "starshard"
version = "2.4.0"
version = "2.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc45de5060fab0de00d7a42230877543ae00c37caf6f49975e69851089b023ac"
checksum = "b0883700b50fd0b62120ebc4c07230e02f0cf1ead474c3b3b44eaa13e0a999ea"
dependencies = [
"async-trait",
"hashbrown 0.17.1",
@@ -12553,9 +12553,9 @@ dependencies = [
[[package]]
name = "tar-codec"
version = "0.0.15"
version = "0.0.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9dad615e50d181afea4bafea23dacb8e33ed52dde06503813570634b932d467"
checksum = "6875ba507bbec6022500f46477159a03f07ebdffdb4ea043254974531df1dac2"
dependencies = [
"archive-trait",
"tar-framing",
@@ -12565,9 +12565,9 @@ dependencies = [
[[package]]
name = "tar-framing"
version = "0.0.15"
version = "0.0.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "563e8181b47c9cdcce569eae3ad431eec71543d6269dd60c7bd2350f70bcd233"
checksum = "49727b1a300514960d425d0e741a7da3e693170f3db39de3180fdbbc07e542c5"
dependencies = [
"thiserror 2.0.21",
"tokio",
@@ -12941,9 +12941,9 @@ dependencies = [
[[package]]
name = "toml"
version = "1.2.0+spec-1.1.0"
version = "1.2.1+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c5ffcdf8f50436dbbfd675c1fe1b8e0854f42b7defea8584bdebe1aee900e561"
checksum = "d5755c51863cdac711f1956baf8eb9588e060755f7d3c4c979e70ea1e1f32cd7"
dependencies = [
"indexmap 2.14.2",
"serde_core",
@@ -12956,9 +12956,9 @@ dependencies = [
[[package]]
name = "toml_datetime"
version = "1.1.2+spec-1.1.0"
version = "1.1.3+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b86d767906c6c42421dcba507eb9d203e779497710a47782a224bb871653053"
checksum = "173f25c883cd2176a2d8212155638c0a60d484a9d84546d270866f40f56b6e49"
dependencies = [
"serde_core",
]
@@ -13851,15 +13851,6 @@ dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-sys"
version = "0.59.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b"
dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-sys"
version = "0.60.2"
@@ -14032,7 +14023,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f3fd376f71958b862e7afb20cfe5a22830e1963462f3a17f49d82a6c1d1f42d"
dependencies = [
"bitflags 2.13.2",
"windows-sys 0.59.0",
"windows-sys 0.52.0",
]
[[package]]
@@ -14282,9 +14273,9 @@ dependencies = [
[[package]]
name = "zip"
version = "9.3.0"
version = "9.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eaeb7f16d2605b49aef586996ded29e65642fab7259a3077e537351df6d80426"
checksum = "0700e6c427aff64acdbf02e85fbfd946c2ee12319d1af013092f3a2178914ff8"
dependencies = [
"aes 0.9.4",
"bzip2",
+4 -4
View File
@@ -156,7 +156,7 @@ async_zip = { default-features = false, version = "0.0.19" }
mysql_async = { default-features = false, version = "0.37.1" }
async-compression = { version = "0.4.52" }
async-trait = "0.1.92"
async-nats = { version = "0.51.0", default-features = false }
async-nats = { version = "0.52.0", default-features = false }
axum = "0.8.9"
futures = "0.3.34"
futures-core = "0.3.34"
@@ -256,8 +256,8 @@ anyhow = "1.0.104"
arc-swap = "1.9.3"
rustfs-tokio-tar = { version = "0.7.1" }
# Candidate Snowball parser versions exercised by rustfs-zip compatibility fixtures.
tar-codec = "0.0.15"
tar-framing = "0.0.15"
tar-codec = "0.0.16"
tar-framing = "0.0.16"
atoi = "3.1.1"
atomic_enum = "0.3.0"
aws-config = { version = "1.13.0" }
@@ -401,7 +401,7 @@ dav-server = "0.11.0"
# Performance Analysis and Memory Profiling
rustfs-mimalloc = { version = "0.7.3" }
# Preserve Unicode focus filters until rustfs/backlog#2302 is resolved.
hotpath = { version = "0.30.3", default-features = false }
hotpath = { version = "0.31.1", default-features = false }
# Snapshot testing for output format regression detection
insta = { version = "1.49" }
+11 -2
View File
@@ -12,7 +12,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM alpine:3.24.2 AS build
ARG BINARY_SOURCE=download
FROM alpine:3.24.2 AS download
ARG TARGETARCH
ARG RELEASE=latest
@@ -74,6 +76,14 @@ COPY scripts/download_rc.sh /usr/local/bin/download_rc.sh
RUN sh /usr/local/bin/download_rc.sh "$TARGETARCH" musl "$RC_RELEASE" /build/rc
# Candidate packaging supplies the accepted executables from verified archives.
# The default download stage remains available for local image builds.
FROM scratch AS supplied
ARG TARGETARCH
COPY .release-inputs/${TARGETARCH}/ /build/
FROM ${BINARY_SOURCE} AS build
FROM alpine:3.24.2
ARG RELEASE=latest
@@ -103,7 +113,6 @@ RUN apk upgrade --no-cache && \
tzdata \
&& test "$(TZ=Asia/Kolkata date +%z)" = "+0530"
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /build/rustfs /usr/bin/rustfs
COPY --from=build /build/rc /usr/bin/rc
COPY entrypoint.sh /etc/rustfs/entrypoint.sh
+11 -1
View File
@@ -12,7 +12,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM ubuntu:26.04 AS build
ARG BINARY_SOURCE=download
FROM ubuntu:26.04 AS download
ARG TARGETARCH
ARG RELEASE=latest
@@ -80,6 +82,14 @@ RUN set -eux; \
COPY scripts/download_rc.sh /usr/local/bin/download_rc.sh
RUN sh /usr/local/bin/download_rc.sh "$TARGETARCH" gnu "$RC_RELEASE" /build/rc
# Candidate packaging supplies the accepted executables from verified archives.
# The default download stage remains available for local image builds.
FROM scratch AS supplied
ARG TARGETARCH
COPY .release-inputs/${TARGETARCH}/ /build/
FROM ${BINARY_SOURCE} AS build
FROM ubuntu:26.04
ARG RELEASE=latest
+3 -3
View File
@@ -145,7 +145,7 @@ chown -R 10001:10001 data logs
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
# Using specific version
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.4
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.5
```
If you use [podman](https://github.com/containers/podman) instead of docker, you can install the RustFS with the below command
@@ -224,7 +224,7 @@ For developers who want to build RustFS Docker images from source with multi-arc
./docker-buildx.sh --push
# Build specific version
./docker-buildx.sh --release 1.0.4 --push
./docker-buildx.sh --release 1.0.5 --push
# Build for custom registry
./docker-buildx.sh --registry your-registry.com --namespace yourname --push
@@ -242,7 +242,7 @@ You can also use Make targets for convenience:
```bash
make docker-buildx # Build locally
make docker-buildx-push # Build and push
make docker-buildx-version VERSION=1.0.4 # Build specific version
make docker-buildx-version VERSION=1.0.5 # Build specific version
make help-docker # Show all Docker-related commands
```
+3 -3
View File
@@ -138,7 +138,7 @@ chown -R 10001:10001 data logs
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
# 使用指定版本运行
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.4
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.5
```
如果您通过绑定挂载启用 TLS 证书目录,也请用同样方式准备该目录:
@@ -176,7 +176,7 @@ docker compose -f docker-compose-simple.yml up -d
./docker-buildx.sh --push
# 构建指定版本
./docker-buildx.sh --release 1.0.4 --push
./docker-buildx.sh --release 1.0.5 --push
# 构建并推送到自定义仓库
./docker-buildx.sh --registry your-registry.com --namespace yourname --push
@@ -193,7 +193,7 @@ docker compose -f docker-compose-simple.yml up -d
```bash
make docker-buildx # 本地构建
make docker-buildx-push # 构建并推送
make docker-buildx-version VERSION=1.0.4 # 构建指定版本
make docker-buildx-version VERSION=1.0.5 # 构建指定版本
make help-docker # 显示所有 Docker 相关命令
```
+100
View File
@@ -902,4 +902,104 @@ mod tests {
assert_eq!(raw.len(), 8);
assert_eq!(&raw[..], reference.digest().to_be_bytes().as_slice());
}
#[test]
fn checksums_streaming_chunks_and_header_encoding_match_oneshot_across_unaligned_boundaries() {
use crate::http::HttpChecksum;
let all = [
ChecksumAlgorithm::Crc32,
ChecksumAlgorithm::Crc32c,
ChecksumAlgorithm::Crc64Nvme,
ChecksumAlgorithm::Sha1,
ChecksumAlgorithm::Sha256,
ChecksumAlgorithm::Sha512,
ChecksumAlgorithm::Xxhash3,
ChecksumAlgorithm::Xxhash64,
ChecksumAlgorithm::Xxhash128,
];
for &payload_len in &[0usize, 1, 31, 33, 1024, 4096 + 37, 64 * 1024 + 19] {
let payload: Vec<u8> = (0..payload_len).map(|i| ((i * 19 + 5) % 253) as u8).collect();
for algorithm in all {
let mut one_shot = algorithm.into_impl();
one_shot.update(&payload);
let expected_raw = one_shot.finalize();
assert_eq!(expected_raw.len(), algorithm.raw_len());
let mut one_shot_hdr = algorithm.into_impl();
one_shot_hdr.update(&payload);
let expected_headers = one_shot_hdr.headers();
for &chunk_size in &[1usize, 31, 1021, 4096] {
if payload_len > 4096 && chunk_size == 1 {
continue;
}
let mut streaming: Box<dyn HttpChecksum> = algorithm.into_impl();
for chunk in payload.chunks(chunk_size) {
streaming.update(chunk);
}
let actual_headers = streaming.headers();
assert_eq!(
actual_headers.get(algorithm.http_header_name()),
expected_headers.get(algorithm.http_header_name()),
"{algorithm:?} streaming mismatch at payload_len={payload_len}, chunk_size={chunk_size}"
);
}
}
}
}
#[test]
#[ignore = "utility throughput microbenchmark; run via cargo test -p rustfs-checksums --lib --release -- --ignored --nocapture"]
fn microbench_checksums_streaming_chunks_and_header_encoding() {
use crate::http::HttpChecksum;
use std::hint::black_box;
use std::time::Instant;
let all = [
ChecksumAlgorithm::Crc32,
ChecksumAlgorithm::Crc32c,
ChecksumAlgorithm::Crc64Nvme,
ChecksumAlgorithm::Sha1,
ChecksumAlgorithm::Sha256,
ChecksumAlgorithm::Sha512,
ChecksumAlgorithm::Xxhash3,
ChecksumAlgorithm::Xxhash64,
ChecksumAlgorithm::Xxhash128,
];
let payload: Vec<u8> = (0..(256 * 1024)).map(|i| ((i * 19 + 5) % 253) as u8).collect();
for &chunk_size in &[4 * 1024usize, 64 * 1024, 256 * 1024] {
for algorithm in all {
let mut one_shot = algorithm.into_impl();
one_shot.update(&payload);
let expected_headers = one_shot.headers();
let iters = 8usize;
let t0 = Instant::now();
let mut last_headers = http::HeaderMap::new();
for _ in 0..iters {
let mut streaming: Box<dyn HttpChecksum> = algorithm.into_impl();
for chunk in payload.chunks(chunk_size) {
streaming.update(black_box(chunk));
}
last_headers = streaming.headers();
black_box(&last_headers);
}
let elapsed = t0.elapsed();
assert_eq!(
last_headers.get(algorithm.http_header_name()),
expected_headers.get(algorithm.http_header_name()),
"{algorithm:?} streaming chunk_size={chunk_size} header mismatch"
);
let total_mib = (payload.len() * iters) as f64 / (1024.0 * 1024.0);
let mibs = total_mib / elapsed.as_secs_f64().max(1e-9);
println!(
"[microbench][checksums] algo={algorithm:?} chunk_size={chunk_size}B throughput={mibs:.1} MiB/s (size={}B)",
algorithm.raw_len()
);
}
}
}
}
+255 -2
View File
@@ -21,12 +21,15 @@
#[cfg(test)]
mod tests {
use crate::common::{RustFSTestEnvironment, init_logging};
use crate::common::{RustFSTestEnvironment, admin_create_user, init_logging};
use aws_sdk_s3::Client;
use aws_sdk_s3::primitives::ByteStream;
use aws_sdk_s3::types::{BucketVersioningStatus, Delete, ObjectIdentifier, VersioningConfiguration};
use std::error::Error;
use tracing::info;
type TestResult = Result<(), Box<dyn Error + Send + Sync>>;
async fn create_bucket(client: &Client, bucket: &str) -> Result<(), Box<dyn Error + Send + Sync>> {
client.create_bucket().bucket(bucket).send().await?;
Ok(())
@@ -90,7 +93,7 @@ mod tests {
Ok(())
}
/// Duplicate and repeated slashes after a leading slash collapse MinIO-style.
/// Duplicate and repeated slashes after a leading slash collapse MinIO-style across single and bulk operations.
#[tokio::test]
async fn test_put_object_with_duplicate_slashes_normalized() -> Result<(), Box<dyn Error + Send + Sync>> {
init_logging();
@@ -144,8 +147,258 @@ mod tests {
"GET after DELETE with raw key must return HTTP 404, got {error:?}"
);
assert_bulk_delete_normalizes_keys(&client).await?;
assert_bulk_delete_authorizes_normalized_keys(&env, &client).await?;
assert_bulk_delete_preserves_version_identity(&client).await?;
env.stop_server();
info!("Test completed successfully");
Ok(())
}
async fn assert_bulk_delete_normalizes_keys(client: &Client) -> TestResult {
let bucket = "test-bulk-leading-slash";
create_bucket(client, bucket).await?;
let cases = [
("/foo/bar", "foo/bar"),
("//keyname", "keyname"),
("/dir///sub//file", "dir/sub/file"),
("/folder///", "folder/"),
("/literal%2F+名", "literal%2F+名"),
("plain", "plain"),
];
let invalid = ["///", "/dir/../keep", "dir//keep"];
client
.put_object()
.bucket(bucket)
.key("keep")
.body(ByteStream::from_static(b"keep"))
.send()
.await?;
client
.put_object()
.bucket(bucket)
.key("dir/keep")
.body(ByteStream::from_static(b"keep"))
.send()
.await?;
for quiet in [false, true] {
for (raw, _) in cases {
client
.put_object()
.bucket(bucket)
.key(raw)
.body(ByteStream::from_static(b""))
.send()
.await?;
}
let listing = client.list_objects_v2().bucket(bucket).send().await?;
let stored: Vec<_> = listing.contents().iter().filter_map(|object| object.key()).collect();
let mut expected: Vec<_> = cases
.iter()
.map(|(_, normalized)| *normalized)
.chain(["dir/keep", "keep"])
.collect();
expected.sort_unstable();
assert_eq!(stored, expected, "PUT must store the exact normalized targets before bulk deletion");
let objects = cases
.iter()
.map(|(raw, _)| *raw)
.chain(invalid)
.map(|key| ObjectIdentifier::builder().key(key).build())
.collect::<Result<Vec<_>, _>>()?;
let delete = Delete::builder().set_objects(Some(objects)).quiet(quiet).build()?;
// A second identical batch also pins idempotent deletion of absent normalized keys.
for _ in 0..2 {
let output = client.delete_objects().bucket(bucket).delete(delete.clone()).send().await?;
let errors: Vec<_> = output.errors().iter().map(|error| (error.key(), error.code())).collect();
assert_eq!(
errors,
invalid.map(|key| (Some(key), Some("InvalidArgument"))),
"unexpected per-key errors: {output:?}"
);
let deleted: Vec<_> = output.deleted().iter().filter_map(|object| object.key()).collect();
let expected = if quiet {
vec![]
} else {
cases.iter().map(|(raw, _)| *raw).collect()
};
assert_eq!(deleted, expected, "responses must retain request keys, including directory slashes");
}
let listing = client.list_objects_v2().bucket(bucket).send().await?;
let remaining: Vec<_> = listing.contents().iter().filter_map(|object| object.key()).collect();
assert_eq!(
remaining,
["dir/keep", "keep"],
"bulk delete must remove normalized objects without resolving dot segments"
);
}
Ok(())
}
async fn assert_bulk_delete_authorizes_normalized_keys(env: &RustFSTestEnvironment, admin: &Client) -> TestResult {
let bucket = "test-bulk-slash-policy";
let user = "bulk-slash-user";
admin_create_user(env, user, "bulkSlashTestSecret123!").await?;
create_bucket(admin, bucket).await?;
for key in ["allowed/item", "protected/item"] {
admin
.put_object()
.bucket(bucket)
.key(key)
.body(ByteStream::from_static(b"keep"))
.send()
.await?;
}
let policy = serde_json::json!({
"Version": "2012-10-17",
"Statement": [
{"Effect": "Allow", "Principal": {"AWS": [user]}, "Action": ["s3:DeleteObject"],
"Resource": [format!("arn:aws:s3:::{bucket}/*")]},
{"Effect": "Deny", "Principal": {"AWS": [user]}, "Action": ["s3:DeleteObject"],
"Resource": [format!("arn:aws:s3:::{bucket}/protected/*")]}
]
});
admin
.put_bucket_policy()
.bucket(bucket)
.policy(policy.to_string())
.send()
.await?;
let client = env.create_s3_client_with_credentials(user, "bulkSlashTestSecret123!");
let output = client
.delete_objects()
.bucket(bucket)
.delete(
Delete::builder()
.objects(ObjectIdentifier::builder().key("/allowed///item").build()?)
.objects(ObjectIdentifier::builder().key("/protected///item").build()?)
.build()?,
)
.send()
.await?;
assert_eq!(output.deleted().len(), 1, "allowed normalized key must be deleted: {output:?}");
assert_eq!(output.deleted()[0].key(), Some("/allowed///item"));
assert_eq!(output.errors().len(), 1, "normalized resource denial must apply: {output:?}");
assert_eq!(output.errors()[0].key(), Some("/protected///item"));
assert_eq!(output.errors()[0].code(), Some("AccessDenied"));
let listing = admin.list_objects_v2().bucket(bucket).send().await?;
let remaining: Vec<_> = listing.contents().iter().filter_map(|object| object.key()).collect();
assert_eq!(remaining, ["protected/item"], "only the authorized object may be removed");
let protected = admin.get_object().bucket(bucket).key("protected/item").send().await?;
assert_eq!(protected.body.collect().await?.into_bytes().as_ref(), b"keep");
Ok(())
}
async fn assert_bulk_delete_preserves_version_identity(client: &Client) -> TestResult {
let bucket = "test-bulk-slash-versions";
let raw = "/history///file";
let normalized = "history/file";
create_bucket(client, bucket).await?;
client
.put_bucket_versioning()
.bucket(bucket)
.versioning_configuration(
VersioningConfiguration::builder()
.status(BucketVersioningStatus::Enabled)
.build(),
)
.send()
.await?;
let first = client
.put_object()
.bucket(bucket)
.key(raw)
.body(ByteStream::from_static(b"old"))
.send()
.await?;
let first_id = first.version_id().ok_or("first PUT omitted version ID")?;
let latest = client
.put_object()
.bucket(bucket)
.key(raw)
.body(ByteStream::from_static(b"latest"))
.send()
.await?;
let latest_id = latest.version_id().ok_or("latest PUT omitted version ID")?;
let invalid = client
.delete_objects()
.bucket(bucket)
.delete(
Delete::builder()
.objects(ObjectIdentifier::builder().key("///").build()?)
.build()?,
)
.send()
.await?;
assert!(invalid.deleted().is_empty(), "slash-only keys must not delete a versioned object");
assert_eq!(
invalid.errors().len(),
1,
"slash-only keys must remain invalid in versioned buckets: {invalid:?}"
);
assert_eq!(invalid.errors()[0].key(), Some("///"));
assert_eq!(invalid.errors()[0].code(), Some("InvalidArgument"));
let deleted = client
.delete_objects()
.bucket(bucket)
.delete(
Delete::builder()
.objects(ObjectIdentifier::builder().key(raw).version_id(first_id).build()?)
.build()?,
)
.send()
.await?;
assert!(deleted.errors().is_empty(), "historical delete failed: {deleted:?}");
assert_eq!(deleted.deleted().len(), 1);
assert_eq!(deleted.deleted()[0].key(), Some(raw));
assert_eq!(deleted.deleted()[0].version_id(), Some(first_id));
let listing = client.list_object_versions().bucket(bucket).send().await?;
assert_eq!(listing.versions().len(), 1, "only the latest data version should remain: {listing:?}");
assert_eq!(listing.versions()[0].version_id(), Some(latest_id));
let deleted = client
.delete_objects()
.bucket(bucket)
.delete(
Delete::builder()
.objects(ObjectIdentifier::builder().key(raw).build()?)
.build()?,
)
.send()
.await?;
assert!(deleted.errors().is_empty(), "delete marker creation failed: {deleted:?}");
assert_eq!(deleted.deleted().len(), 1);
assert_eq!(deleted.deleted()[0].key(), Some(raw));
assert_eq!(deleted.deleted()[0].delete_marker(), Some(true));
let marker = deleted.deleted()[0]
.delete_marker_version_id()
.ok_or("missing delete marker ID")?;
let listing = client.list_object_versions().bucket(bucket).send().await?;
assert_eq!(listing.delete_markers().len(), 1);
assert_eq!(listing.delete_markers()[0].key(), Some(normalized));
assert_eq!(listing.delete_markers()[0].version_id(), Some(marker));
let deleted = client
.delete_objects()
.bucket(bucket)
.delete(
Delete::builder()
.objects(ObjectIdentifier::builder().key(raw).version_id(marker).build()?)
.build()?,
)
.send()
.await?;
assert!(deleted.errors().is_empty(), "explicit marker removal failed: {deleted:?}");
assert_eq!(deleted.deleted().len(), 1);
assert_eq!(deleted.deleted()[0].key(), Some(raw));
assert_eq!(deleted.deleted()[0].version_id(), Some(marker));
let listing = client.list_object_versions().bucket(bucket).send().await?;
assert!(listing.delete_markers().is_empty());
let restored = client.get_object().bucket(bucket).key(normalized).send().await?;
assert_eq!(restored.version_id(), Some(latest_id));
assert_eq!(restored.body.collect().await?.into_bytes().as_ref(), b"latest");
Ok(())
}
}
+33 -16
View File
@@ -4325,20 +4325,28 @@ fn take_presynced_tmp_file(file: &Path) -> bool {
PRESYNCED_TMP_FILES.remove(file).is_some()
}
fn open_lazy_local_file_std(file_path: &Path, volume_dir: &Path, _reclaim_on_shutdown: bool) -> std::io::Result<std::fs::File> {
if let Some(parent) = file_path.parent()
&& (volume_dir.as_os_str().is_empty() || !volume_dir.starts_with(parent))
&& let Err(err) = super::fs::create_dir_all_std(parent)
&& err.kind() != std::io::ErrorKind::AlreadyExists
{
return Err(to_file_error(err));
}
let file = std::fs::OpenOptions::new()
.write(true)
.create(true)
.truncate(true)
.open(file_path)
.map_err(to_file_error)?;
fn open_lazy_local_file_std(
file_path: &Path,
volume_dir: &Path,
_reclaim_on_shutdown: bool,
parent_exists: bool,
) -> std::io::Result<std::fs::File> {
let mut open_opts = std::fs::OpenOptions::new();
open_opts.write(true).create(true).truncate(true);
let file = match parent_exists.then(|| open_opts.open(file_path)) {
Some(Ok(file)) => file,
Some(Err(err)) if err.kind() != std::io::ErrorKind::NotFound => return Err(to_file_error(err)),
_ => {
if let Some(parent) = file_path.parent()
&& (volume_dir.as_os_str().is_empty() || !volume_dir.starts_with(parent))
&& let Err(err) = super::fs::create_dir_all_std(parent)
&& err.kind() != std::io::ErrorKind::AlreadyExists
{
return Err(to_file_error(err));
}
open_opts.open(file_path).map_err(to_file_error)?
}
};
#[cfg(target_os = "macos")]
if _reclaim_on_shutdown {
let _ = set_std_fd_nocache(&file);
@@ -4353,6 +4361,7 @@ struct LazyLocalFileWriter {
reclaim_on_shutdown: bool,
sync_on_close: bool,
mark_dir_presynced: bool,
parent_exists: bool,
bytes_written: usize,
state: LazyLocalFileWriterState,
}
@@ -4373,6 +4382,7 @@ impl LazyLocalFileWriter {
reclaim_on_shutdown: bool,
sync_on_close: bool,
mark_dir_presynced: bool,
parent_exists: bool,
) -> Self {
Self {
file_path,
@@ -4381,6 +4391,7 @@ impl LazyLocalFileWriter {
reclaim_on_shutdown,
sync_on_close,
mark_dir_presynced,
parent_exists,
bytes_written: 0,
state: LazyLocalFileWriterState::Unopened,
}
@@ -4393,12 +4404,13 @@ impl LazyLocalFileWriter {
let reclaim_on_shutdown = self.reclaim_on_shutdown;
let sync_on_close = self.sync_on_close;
let mark_dir_presynced = self.mark_dir_presynced;
let parent_exists = self.parent_exists;
match std::mem::replace(&mut self.state, LazyLocalFileWriterState::Closed) {
LazyLocalFileWriterState::Unopened => {
let file_path = self.file_path.clone();
let volume_dir = self.volume_dir.clone();
self.state = LazyLocalFileWriterState::Writing(tokio::task::spawn_blocking(move || {
let mut file = open_lazy_local_file_std(&file_path, &volume_dir, reclaim_on_shutdown)?;
let mut file = open_lazy_local_file_std(&file_path, &volume_dir, reclaim_on_shutdown, parent_exists)?;
std::io::Write::write_all(&mut file, &payload).map_err(to_file_error)?;
if close_after {
if sync_on_close {
@@ -4540,8 +4552,9 @@ impl AsyncWrite for LazyLocalFileWriter {
let file_path = self.file_path.clone();
let volume_dir = self.volume_dir.clone();
let reclaim_on_shutdown = self.reclaim_on_shutdown;
let parent_exists = self.parent_exists;
self.state = LazyLocalFileWriterState::Closing(tokio::task::spawn_blocking(move || {
open_lazy_local_file_std(&file_path, &volume_dir, reclaim_on_shutdown).map(drop)
open_lazy_local_file_std(&file_path, &volume_dir, reclaim_on_shutdown, parent_exists).map(drop)
}));
}
LazyLocalFileWriterState::Open(file) => {
@@ -5593,11 +5606,14 @@ impl LocalIoBackend for StdBackend {
let file_path = local_disk_object_path(self.io_root(), volume, path)?;
check_path_length(file_path.to_string_lossy().as_ref())?;
let mut mark_dir_presynced = false;
let mut parent_exists = false;
if let Some(parent) = file_path.parent() {
check_path_length(parent.to_string_lossy().as_ref())?;
parent_exists = parent == volume_dir;
if volume == RUSTFS_META_TMP_BUCKET {
unmark_presynced_tmp_file(&file_path);
let parent_was_presynced = unmark_presynced_tmp_data_dir(parent);
parent_exists |= parent_was_presynced;
let file_name = file_path.file_name().and_then(|n| n.to_str()).unwrap_or_default();
mark_dir_presynced = sync_on_close
&& (file_name == "part.1"
@@ -5651,6 +5667,7 @@ impl LocalIoBackend for StdBackend {
reclaim_on_shutdown,
sync_on_close,
mark_dir_presynced,
parent_exists,
)))
}
WriteMode::Append => {
+5 -1
View File
@@ -286,12 +286,15 @@ pub(super) fn fsync_inline_commit_std(
if destination_was_absent {
let mut ancestor = dst.parent().and_then(Path::parent);
let mut ancestor_step = 0usize;
while let Some(ancestor_dir) = ancestor {
if !ancestor_dir.starts_with(dst_volume_dir) {
break;
}
let fsync_started = rustfs_io_metrics::put_stage_timer();
if let Err(err) = os::fsync_dir_limited_std(ancestor_dir) {
if let Err(err) =
os::fsync_commit_ancestor_directory_limited_std(ancestor_dir, dst_volume_dir, ancestor_step, commit_guard)
{
rustfs_io_metrics::record_put_object_stage_duration_from(
rustfs_io_metrics::PUT_STAGE_SET_DISK_RENAME_ANCESTOR_DIR_FSYNC,
fsync_started,
@@ -310,6 +313,7 @@ pub(super) fn fsync_inline_commit_std(
break;
}
ancestor = ancestor_dir.parent();
ancestor_step += 1;
}
}
Ok(())
+138 -86
View File
@@ -1128,6 +1128,8 @@ struct PreparedDeleteReceiptEntry {
journal_raw: Vec<u8>,
journal: DeleteRecoveryJournal,
sole_transaction: bool,
had_old_data_dir: bool,
rollback_dir_fd: Option<Arc<std::fs::File>>,
}
#[derive(Clone)]
@@ -1137,6 +1139,8 @@ struct PublishedDeleteReceiptEntry {
file_path: PathBuf,
staging_root: PathBuf,
journal_raw: Vec<u8>,
had_old_data_dir: bool,
rollback_dir_fd: Option<Arc<std::fs::File>>,
}
const PREPARED_DELETE_RECEIPTS_PER_SHARD: usize = 64;
@@ -2495,8 +2499,9 @@ impl LocalDisk {
Err(err) if err.kind() == ErrorKind::NotFound => return Ok(Ok(None)),
Err(err) => return Err(err),
};
type PreparedDeleteTuple = (Option<Uuid>, Option<Vec<u8>>, Vec<u8>, DeleteRecoveryJournal, bool, bool, bool);
let (old_dir, target, journal_raw, journal, sole_transaction, destination_missing, old_data_dir_present) =
type PreparedDeleteTuple =
(Option<Uuid>, Option<Vec<u8>>, Vec<u8>, DeleteRecoveryJournal, bool, bool, Option<bool>);
let (old_dir, target, journal_raw, journal, sole_transaction, destination_missing, old_data_dir_state) =
match (|| -> Result<Option<PreparedDeleteTuple>> {
let mut meta = FileMeta::load(&buf)?;
let mut deleted_headers: Vec<_> = meta
@@ -2513,13 +2518,12 @@ impl LocalDisk {
if old_dir.is_some_and(|dir| dir == transaction || dir.is_nil()) {
return Err(DiskError::FileCorrupt);
}
let mut old_path_to_verify = None;
if let Some(uuid) = old_dir {
meta.data.remove(vec![fi_clone.version_id.unwrap_or_default(), uuid])?;
let old_path = path_join(&[file_path.as_path(), Path::new(uuid.to_string().as_str())]);
check_path_length(old_path.to_string_lossy().as_ref())?;
if old_path.join(STORAGE_FORMAT_FILE).exists() {
return Ok(None);
}
old_path_to_verify = Some(old_path);
}
let surviving: HashSet<_> = meta
.versions
@@ -2601,6 +2605,12 @@ impl LocalDisk {
}
}
}
if let Some(old_path) = old_path_to_verify
&& (!scanned_file_path || old_data_dir_exists)
&& old_path.join(STORAGE_FORMAT_FILE).exists()
{
return Ok(None);
}
sole_transaction &= predecessors.is_empty();
let mut sparse = fi_clone;
sparse.data = None;
@@ -2626,15 +2636,21 @@ impl LocalDisk {
journal,
sole_transaction,
scanned_file_path && !destination_exists,
scanned_file_path && old_data_dir_exists,
scanned_file_path.then_some(old_data_dir_exists),
)))
})() {
Ok(Some(tuple)) => tuple,
Ok(None) => return Ok(Ok(None)),
Err(err) => return Ok(Err(err)),
};
std::fs::create_dir_all(&staging_root)?;
let staged = tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root)?;
let staged = match tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root) {
Ok(dir) => dir,
Err(err) if err.kind() == ErrorKind::NotFound => {
std::fs::create_dir_all(&staging_root)?;
tempfile::Builder::new().prefix("delete-prep-").tempdir_in(&staging_root)?
}
Err(err) => return Err(err),
};
let staged_dir = staged.path().to_path_buf();
let backup_path = staged_dir.join(STORAGE_FORMAT_FILE_BACKUP);
if std::fs::hard_link(&xl_path, &backup_path).is_err() {
@@ -2658,8 +2674,12 @@ impl LocalDisk {
for file in written_files {
file.sync_data()?;
}
#[cfg(unix)]
let rollback_dir_fd = sync.then(|| std::fs::File::open(&staged_dir).ok().map(Arc::new)).flatten();
#[cfg(not(unix))]
let rollback_dir_fd: Option<Arc<std::fs::File>> = None;
if sync {
os::fsync_dir_std(&staged_dir)?;
os::fsync_directory_handle_std(&staged_dir, rollback_dir_fd.as_deref())?;
}
if !destination_missing {
match std::fs::symlink_metadata(&destination) {
@@ -2682,8 +2702,12 @@ impl LocalDisk {
}
let old_data_path = old_dir
.map(|uuid| file_path.join(uuid.to_string()))
.filter(|p| old_data_dir_present || p.is_dir());
if sync && let Err(err) = os::fsync_dir_std(&file_path) {
.filter(|p| old_data_dir_state.unwrap_or_else(|| p.is_dir()));
#[cfg(unix)]
let file_path_dir = sync.then(|| std::fs::File::open(&file_path).ok()).flatten();
#[cfg(not(unix))]
let file_path_dir: Option<std::fs::File> = None;
if sync && let Err(err) = os::fsync_directory_handle_std(&file_path, file_path_dir.as_ref()) {
let _ = std::fs::remove_dir_all(&destination);
return Err(err);
}
@@ -2692,11 +2716,15 @@ impl LocalDisk {
let mut reserved_data_path = None;
if let Some(old_path) = old_data_path {
let reserved_marker = old_path.join(format!("{RESERVED_DELETE_DATA_DIR_MARKER_PREFIX}{transaction}"));
#[cfg(unix)]
let old_dir_file = sync.then(|| std::fs::File::open(&old_path).ok()).flatten();
#[cfg(not(unix))]
let old_dir_file: Option<std::fs::File> = None;
let reserve_res = (|| -> std::io::Result<()> {
let marker = std::fs::File::create(&reserved_marker)?;
if sync {
marker.sync_all()?;
os::fsync_dir_std(&old_path)?;
os::fsync_directory_handle_std(&old_path, old_dir_file.as_ref())?;
}
Ok(())
})();
@@ -2705,7 +2733,7 @@ impl LocalDisk {
let _ = std::fs::remove_dir_all(&destination);
return Err(err);
}
reserved_data_path = Some((old_path, reserved_marker));
reserved_data_path = Some((old_path, reserved_marker, old_dir_file));
}
if let Some(ref target_bytes) = target {
// A provisional head must stay outside the published recovery transaction.
@@ -2737,7 +2765,7 @@ impl LocalDisk {
})()
};
if let Err(err) = write_res {
if let Some((_, reserved_marker)) = &reserved_data_path {
if let Some((_, reserved_marker, _)) = &reserved_data_path {
let _ = std::fs::remove_file(reserved_marker);
}
let _ = std::fs::remove_dir_all(&destination);
@@ -2746,19 +2774,20 @@ impl LocalDisk {
} else if let Err(err) = std::fs::remove_file(&xl_path)
&& err.kind() != ErrorKind::NotFound
{
if let Some((_, reserved_marker)) = &reserved_data_path {
if let Some((_, reserved_marker, _)) = &reserved_data_path {
let _ = std::fs::remove_file(reserved_marker);
}
let _ = std::fs::remove_dir_all(&destination);
return Err(err);
}
if sync {
os::fsync_dir_std(&file_path)?;
os::fsync_directory_handle_std(&file_path, file_path_dir.as_ref())?;
}
if let Some((old_path, reserved_marker)) = reserved_data_path {
let had_old_data_dir = reserved_data_path.is_some();
if let Some((old_path, reserved_marker, old_dir_file)) = reserved_data_path {
let recovery_marker = old_path.join(format!("{RECOVERY_DELETE_DATA_DIR_MARKER_PREFIX}{transaction}"));
match std::fs::rename(&reserved_marker, &recovery_marker) {
Ok(()) if sync => os::fsync_dir_std(&old_path)?,
Ok(()) if sync => os::fsync_directory_handle_std(&old_path, old_dir_file.as_ref())?,
Ok(()) => {}
Err(err) if err.kind() == ErrorKind::NotFound => {}
Err(err) => return Err(err),
@@ -2774,9 +2803,11 @@ impl LocalDisk {
journal_raw,
journal,
sole_transaction,
had_old_data_dir,
rollback_dir_fd,
},
);
Ok(Ok(Some(old_dir)))
Ok(Ok(Some(old_dir.filter(|_| had_old_data_dir))))
},
)
.await
@@ -2829,7 +2860,13 @@ impl LocalDisk {
Some(entry.bucket_path),
entry.file_path,
entry.staging_root,
Some((entry.journal_raw, entry.journal, entry.sole_transaction)),
Some((
entry.journal_raw,
entry.journal,
entry.sole_transaction,
entry.had_old_data_dir,
entry.rollback_dir_fd,
)),
),
None => {
self.reject_multipart_delete_recovery_mutation(volume, object)?;
@@ -2860,7 +2897,7 @@ impl LocalDisk {
lease,
sync,
move || -> std::io::Result<Result<Option<(DeleteRecoveryJournal, bool, bool)>>> {
if let Some((journal_raw, journal, sole_transaction)) = cached_receipt
if let Some((journal_raw, journal, sole_transaction, had_old_data_dir, rollback_dir_fd)) = cached_receipt
&& rollback.join(PREPARED).is_file()
{
let tmp_receipt = staging_root.join(format!("receipt-{transaction}.tmp"));
@@ -2874,7 +2911,7 @@ impl LocalDisk {
os::record_native_publication(&receipt_path);
std::fs::rename(&tmp_receipt, &receipt_path)?;
if sync {
os::fsync_dir_std(&rollback)?;
os::fsync_directory_handle_std(&rollback, rollback_dir_fd.as_deref())?;
}
Ok(())
})();
@@ -2891,6 +2928,8 @@ impl LocalDisk {
file_path: root,
staging_root,
journal_raw,
had_old_data_dir,
rollback_dir_fd,
},
);
}
@@ -3027,29 +3066,34 @@ impl LocalDisk {
None
};
let cached_published = rollback_open_path.as_deref().and_then(take_published_delete_receipt);
let (object_path, bucket_path, staging_root, root, cached_journal_raw) = match cached_published {
Some(entry) => (
entry.object_path,
entry.bucket_path,
entry.staging_root,
entry.file_path,
Some(entry.journal_raw),
),
None => {
self.reject_multipart_delete_recovery_mutation(volume, object)?;
if !safe_path(object, false) {
return Err(DiskError::InvalidPath);
let (object_path, bucket_path, staging_root, root, cached_journal_raw, cached_had_old_data_dir, mut rollback_dir_fd) =
match cached_published {
Some(entry) => (
entry.object_path,
entry.bucket_path,
entry.staging_root,
entry.file_path,
Some(entry.journal_raw),
Some(entry.had_old_data_dir),
entry.rollback_dir_fd,
),
None => {
self.reject_multipart_delete_recovery_mutation(volume, object)?;
if !safe_path(object, false) {
return Err(DiskError::InvalidPath);
}
let obj_path = self.get_object_path(volume, object)?;
(
obj_path,
self.io_get_bucket_path(volume)?,
self.io_get_bucket_path(RUSTFS_META_TMP_BUCKET)?,
self.io_get_object_path(volume, object)?,
None,
None,
None,
)
}
let obj_path = self.get_object_path(volume, object)?;
(
obj_path,
self.io_get_bucket_path(volume)?,
self.io_get_bucket_path(RUSTFS_META_TMP_BUCKET)?,
self.io_get_object_path(volume, object)?,
None,
)
}
};
};
let metadata_owner = os::acquire_metadata_mutation_lease(&object_path, namespace_owner).await;
let volume_guard = os::disk_volume_mutation_lock(&self.root, volume).read_owned().await;
let owner: Option<Arc<dyn Send + Sync>> = Some(Arc::new((metadata_owner, volume_guard)));
@@ -3123,56 +3167,61 @@ impl LocalDisk {
let recovery_committed = format!("{RECOVERY_DELETE_DATA_DIR_MARKER_PREFIX}{transaction}");
let reserved = format!("{RESERVED_DELETE_DATA_DIR_MARKER_PREFIX}{transaction}");
let mut had_physical_data_dirs = false;
for data_dir in &data_dirs {
for base in [&root, &rollback] {
let dir_path = base.join(data_dir.to_string());
match std::fs::read_dir(&dir_path) {
Ok(entries) => {
had_physical_data_dirs = true;
let mut removable = Vec::new();
let mut preserve_dir = false;
for entry in recovery_entries(entries, work_budget.as_deref()) {
let entry = entry?;
let name = entry.file_name();
let Some(name) = name.to_str() else {
preserve_dir = true;
continue;
};
if entry.file_type()?.is_file()
&& (delete_recovery_part_file(name)
|| name == committed
|| name == recovery_committed
|| name == reserved
|| is_delete_transaction_marker(name, RESERVED_DELETE_DATA_DIR_MARKER_PREFIX))
{
removable.push(entry.path());
} else {
preserve_dir = true;
}
}
for path in removable {
ignore_not_found(std::fs::remove_file(path))?;
}
if sync {
os::fsync_dir_std(&dir_path)?;
}
if !preserve_dir {
match std::fs::remove_dir(&dir_path) {
Ok(()) => {}
Err(err) if err.kind() == ErrorKind::NotFound || is_dir_not_empty_error(&err) => {}
Err(err) => return Err(err),
let fast_cleanup = cached_journal_raw.is_some();
if cached_had_old_data_dir != Some(false) {
let bases: &[&Path] = if fast_cleanup { &[&root] } else { &[&root, &rollback] };
for data_dir in &data_dirs {
for base in bases {
let dir_path = base.join(data_dir.to_string());
match std::fs::read_dir(&dir_path) {
Ok(entries) => {
had_physical_data_dirs = true;
let mut removable = Vec::new();
let mut preserve_dir = false;
for entry in recovery_entries(entries, work_budget.as_deref()) {
let entry = entry?;
let name = entry.file_name();
let Some(name) = name.to_str() else {
preserve_dir = true;
continue;
};
if entry.file_type()?.is_file()
&& (delete_recovery_part_file(name)
|| name == committed
|| name == recovery_committed
|| name == reserved
|| is_delete_transaction_marker(name, RESERVED_DELETE_DATA_DIR_MARKER_PREFIX))
{
removable.push(entry.path());
} else {
preserve_dir = true;
}
}
for path in removable {
ignore_not_found(std::fs::remove_file(path))?;
}
let mut dir_removed = false;
if !preserve_dir {
match std::fs::remove_dir(&dir_path) {
Ok(()) => dir_removed = true,
Err(err) if err.kind() == ErrorKind::NotFound => dir_removed = true,
Err(err) if is_dir_not_empty_error(&err) => {}
Err(err) => return Err(err),
}
}
if !dir_removed && sync {
os::fsync_dir_std(&dir_path)?;
}
}
Err(err) if err.kind() == ErrorKind::NotFound => {}
Err(err) => return Err(err),
}
Err(err) if err.kind() == ErrorKind::NotFound => {}
Err(err) => return Err(err),
}
}
}
if had_physical_data_dirs && sync && root.is_dir() {
os::fsync_dir_std(&root)?;
}
let fast_cleanup = cached_journal_raw.is_some();
if fast_cleanup {
let backup_or_marker = if has_original {
Some(STORAGE_FORMAT_FILE_BACKUP)
@@ -3221,12 +3270,15 @@ impl LocalDisk {
}
}
}
if sync && let Err(err) = os::fsync_dir_std(&rollback) {
let rollback_sync = rollback_dir_fd.take();
if sync && let Err(err) = os::fsync_directory_handle_std(&rollback, rollback_sync.as_deref()) {
drop(rollback_sync);
if has_target || had_physical_data_dirs {
let _ = persist_receipt(true);
}
return Err(err);
}
drop(rollback_sync);
#[cfg(all(any(test, feature = "test-util"), not(windows)))]
os::prepared_publication_test_hooks::run(os::prepared_publication_test_hooks::Stage::Remove, &receipt_path);
if let Err(err) = ignore_not_found(std::fs::remove_file(&receipt_path)) {
+36 -40
View File
@@ -619,10 +619,6 @@ pub(crate) mod prepare_rename_test_hooks {
}
}
pub(crate) fn has_after_component_opened() -> bool {
!AFTER_COMPONENT_OPENED.lock().is_empty()
}
/// Hooks keyed by a directory component name, run before a Windows
/// directory pin retries a transient sharing conflict on it.
#[cfg(windows)]
@@ -1292,6 +1288,28 @@ pub(crate) fn fsync_dir_limited_std(dir: &Path) -> io::Result<()> {
fsync_dir_std(dir)
}
pub(crate) fn fsync_commit_ancestor_directory_limited_std(
dir: &Path,
base_dir: &Path,
step_from_destination: usize,
guard: &RenameCommitGuard,
) -> io::Result<()> {
#[cfg(target_os = "linux")]
if base_dir == guard.directories.destination_base
&& guard.directories.destination_path.ancestors().nth(1 + step_from_destination) == Some(dir)
&& let Some(ancestors) = guard.directories.destination_ancestors.get()
&& let Some(idx) = ancestors.len().checked_sub(1 + step_from_destination)
&& let Some(file) = ancestors.get(idx)
{
#[cfg(test)]
fsync_dir_recorder::record_limited(dir);
return fsync_directory_handle_std(dir, Some(file));
}
#[cfg(not(target_os = "linux"))]
let _ = (base_dir, step_from_destination, guard);
fsync_dir_limited_std(dir)
}
/// Reuse the published directory without bypassing group commit or sync admission.
pub(crate) async fn fsync_commit_directory(
dir: &Path,
@@ -5996,6 +6014,7 @@ struct LinuxCommitDirectories {
destination_path: PathBuf,
source: LinuxCommitDirectory,
destination: std::sync::OnceLock<LinuxCommitDirectory>,
destination_ancestors: std::sync::OnceLock<Vec<std::fs::File>>,
destination_base: PathBuf,
}
@@ -6012,6 +6031,8 @@ impl LinuxCommitDirectories {
.ok_or_else(|| io::Error::other("missing destination directory guard"))?,
);
let directory = LinuxCommitDirectory::new(directory)?;
let ancestors = parents.into_iter().map(std::fs::File::from).collect();
let _ = self.destination_ancestors.set(ancestors);
Ok(self.destination.get_or_init(|| directory))
}
@@ -6021,7 +6042,7 @@ impl LinuxCommitDirectories {
.ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "file parent does not match its commit guard"))
}
fn validate(&self) -> io::Result<()> {
fn validate(&self, check_destination: bool) -> io::Result<()> {
use std::os::unix::fs::MetadataExt;
// A pinned directory may have been unlinked or replaced. Do not report a
// successful publication into a detached object tree.
@@ -6030,8 +6051,9 @@ impl LinuxCommitDirectories {
return Err(io::Error::new(io::ErrorKind::InvalidData, "commit directory identity changed"));
}
let destination = self.prepare_destination()?;
if (self.source.device, self.source.inode) != (destination.device, destination.inode)
|| self.source_path.as_os_str() != self.destination_path.as_os_str()
if check_destination
&& ((self.source.device, self.source.inode) != (destination.device, destination.inode)
|| self.source_path.as_os_str() != self.destination_path.as_os_str())
{
let current = self.root.open_directory(&self.destination_path)?.metadata()?;
if (current.dev(), current.ino()) != (destination.device, destination.inode) {
@@ -6045,8 +6067,9 @@ impl LinuxCommitDirectories {
record_native_publication(destination);
let source_name = self.file_name(source, true)?;
let destination_name = self.file_name(destination, false)?;
let destination_was_prepared = self.destination.get().is_some();
let destination_parent = self.prepare_destination()?;
self.validate()?;
self.validate(destination_was_prepared)?;
let mut attempt = 0;
loop {
if let Some(prepared) = prepared {
@@ -6268,7 +6291,7 @@ pub(crate) fn multipart_payload_commit_guard_std(
source_parent: &Path,
) -> io::Result<RenameCommitGuard> {
let original = &metadata_guard.directories;
original.validate()?;
original.validate(true)?;
let source = LinuxCommitDirectory::new(original.root.open_directory(source_parent)?)?;
let destination = original.prepare_destination()?;
// Clone the pinned file identity, never recapture the target pathname
@@ -6285,6 +6308,7 @@ pub(crate) fn multipart_payload_commit_guard_std(
destination_path: original.destination_path.clone(),
source,
destination: std::sync::OnceLock::from(destination),
destination_ancestors: std::sync::OnceLock::new(),
destination_base: original.destination_base.clone(),
}),
})
@@ -6366,6 +6390,7 @@ pub(crate) fn prepare_rename_commit_guard(
destination_path: destination_parent.to_path_buf(),
source,
destination,
destination_ancestors: std::sync::OnceLock::new(),
destination_base: destination_base.to_path_buf(),
}),
})
@@ -7475,6 +7500,7 @@ fn same_parent_commit_guard(
device: dir.device,
inode: dir.inode,
}),
destination_ancestors: std::sync::OnceLock::new(),
destination_base: base.to_path_buf(),
}),
});
@@ -8514,37 +8540,7 @@ pub(crate) fn mkdir_all_below_existing_base_std(
let flags = OFlags::RDONLY | OFlags::DIRECTORY | OFlags::NOFOLLOW | OFlags::CLOEXEC;
let mode = Mode::RWXU | Mode::RWXG | Mode::RWXO;
#[cfg(target_os = "linux")]
let base = {
use rustix::fs::{ResolveFlags, openat2};
use std::os::fd::AsFd;
let owned_base = (base_dir != publication_root.io_path.as_path() && base_dir != publication_root.path.as_path())
.then(|| publication_root.open_directory(base_dir).map(std::os::fd::OwnedFd::from))
.transpose()?;
#[cfg(test)]
let has_hook = prepare_rename_test_hooks::has_after_component_opened();
#[cfg(not(test))]
let has_hook = false;
if !has_hook {
let base_fd = owned_base
.as_ref()
.map_or_else(|| publication_root.directory.as_fd(), AsFd::as_fd);
let target = if relative.as_os_str().is_empty() {
Path::new(".")
} else {
relative
};
match openat2(base_fd, target, flags, Mode::empty(), ResolveFlags::BENEATH | ResolveFlags::NO_SYMLINKS) {
Ok(leaf) => return Ok(vec![leaf]),
Err(Errno::NOENT | Errno::NOSYS) => {}
Err(err) => return Err(err.into()),
}
}
match owned_base {
Some(base) => base,
None => publication_root.open_directory(base_dir)?.into(),
}
};
let base = publication_root.open_directory(base_dir)?.into();
#[cfg(not(target_os = "linux"))]
let base = {
// The configured endpoint is trusted; every component beneath it
+119
View File
@@ -1873,4 +1873,123 @@ mod tests {
assert_eq!(err.kind(), std::io::ErrorKind::InvalidData);
assert!(err.to_string().contains("invalid checksum value"), "unexpected error: {err}");
}
fn software_crc32_bitwise(reflected_poly: u32, data: &[u8]) -> [u8; 4] {
let mut crc = !0u32;
for &b in data {
crc ^= u32::from(b);
for _ in 0..8 {
let mask = (crc & 1).wrapping_neg();
crc = (crc >> 1) ^ (reflected_poly & mask);
}
}
(!crc).to_be_bytes()
}
fn software_crc64_nvme_bitwise(data: &[u8]) -> [u8; 8] {
let reflected_poly = super::CRC64_NVME_POLYNOMIAL.reverse_bits();
let mut crc = !0u64;
for &b in data {
crc ^= u64::from(b);
for _ in 0..8 {
let mask = (crc & 1).wrapping_neg();
crc = (crc >> 1) ^ (reflected_poly & mask);
}
}
(!crc).to_be_bytes()
}
#[test]
fn rio_crc_simd_matches_bitwise_scalar_across_tail_and_vector_boundaries() {
for &size in &[
0usize,
1,
7,
8,
15,
16,
17,
31,
32,
33,
63,
64,
65,
127,
128,
129,
255,
256,
257,
1024,
4096 + 17,
64 * 1024,
] {
let payload: Vec<u8> = (0..size).map(|i| ((i * 31 + 17) % 251) as u8).collect();
let crc32_simd = Checksum::new_from_data(ChecksumType::CRC32, &payload).expect("crc32");
let crc32c_simd = Checksum::new_from_data(ChecksumType::CRC32C, &payload).expect("crc32c");
let crc64_simd = Checksum::new_from_data(ChecksumType::CRC64_NVME, &payload).expect("crc64nvme");
let crc32_sw = software_crc32_bitwise(0xEDB88320, &payload);
let crc32c_sw = software_crc32_bitwise(0x82F63B78, &payload);
let crc64_sw = software_crc64_nvme_bitwise(&payload);
assert_eq!(crc32_simd.raw.as_slice(), &crc32_sw, "CRC32 SIMD vs scalar mismatch at {size}B");
assert_eq!(crc32c_simd.raw.as_slice(), &crc32c_sw, "CRC32C SIMD vs scalar mismatch at {size}B");
assert_eq!(crc64_simd.raw.as_slice(), &crc64_sw, "CRC64_NVME SIMD vs scalar mismatch at {size}B");
}
}
#[test]
#[ignore = "utility throughput microbenchmark; run via cargo test -p rustfs-rio --lib --release -- --ignored --nocapture"]
fn microbench_rio_checksum_simd_vs_scalar_and_all_base_algorithms() {
use std::hint::black_box;
use std::io::Write as _;
use std::time::Instant;
for &(size, iters) in &[(1024usize, 100usize), (64 * 1024, 20), (1024 * 1024, 4)] {
let payload: Vec<u8> = (0..size).map(|i| ((i * 31 + 17) % 251) as u8).collect();
let total_mib = (size * iters) as f64 / (1024.0 * 1024.0);
// Verify SIMD-accelerated CRC32 / CRC32C / CRC64_NVME against bitwise scalar reference at 1 KiB & 64 KiB.
if size <= 64 * 1024 {
let crc32_simd = Checksum::new_from_data(ChecksumType::CRC32, &payload).expect("crc32");
let crc32c_simd = Checksum::new_from_data(ChecksumType::CRC32C, &payload).expect("crc32c");
let crc64_simd = Checksum::new_from_data(ChecksumType::CRC64_NVME, &payload).expect("crc64nvme");
let t_sw = Instant::now();
let crc32_sw = software_crc32_bitwise(0xEDB88320, black_box(&payload));
let crc32c_sw = software_crc32_bitwise(0x82F63B78, black_box(&payload));
let crc64_sw = software_crc64_nvme_bitwise(black_box(&payload));
let sw_elapsed = t_sw.elapsed();
assert_eq!(crc32_simd.raw.as_slice(), &crc32_sw, "CRC32 SIMD vs scalar mismatch at {size}B");
assert_eq!(crc32c_simd.raw.as_slice(), &crc32c_sw, "CRC32C SIMD vs scalar mismatch at {size}B");
assert_eq!(crc64_simd.raw.as_slice(), &crc64_sw, "CRC64_NVME SIMD vs scalar mismatch at {size}B");
println!(
"[microbench][rio::checksum] size={size}B bitwise scalar reference check passed in {:?}",
sw_elapsed
);
}
for &checksum_type in super::BASE_CHECKSUM_TYPES {
let mut hasher = checksum_type.hasher().expect("base checksum type must have hasher");
let t0 = Instant::now();
let mut digest = Vec::new();
for _ in 0..iters {
hasher.reset();
hasher.write_all(black_box(&payload)).expect("hasher write_all");
digest = hasher.finalize();
black_box(&digest);
}
let elapsed = t0.elapsed();
assert_eq!(digest.len(), checksum_type.raw_byte_len());
let mibs = total_mib / elapsed.as_secs_f64().max(1e-9);
println!(
"[microbench][rio::checksum] size={size}B algo={} throughput={mibs:.1} MiB/s",
checksum_type
);
}
}
}
}
+162
View File
@@ -735,4 +735,166 @@ mod tests {
}
}
}
#[test]
fn highwayhash_simd_matches_portable_scalar_and_chunked_slices_across_unaligned_boundaries() {
use highway::PortableHash;
for &size in &[
0usize,
1,
7,
15,
31,
32,
33,
63,
64,
65,
127,
128,
129,
255,
257,
1024,
4096 + 17,
64 * 1024,
] {
let payload = generate_compat_test_data(size);
let mut h_simd = HighwayHasher::new(MAGIC_HIGHWAY_HASH256_PARSED_KEY);
h_simd.append(&payload);
let simd_digest = u8x32_from_u64x4(h_simd.finalize256());
let mut h_scalar = PortableHash::new(MAGIC_HIGHWAY_HASH256_PARSED_KEY);
h_scalar.append(&payload);
let scalar_digest = u8x32_from_u64x4(h_scalar.finalize256());
assert_eq!(
simd_digest, scalar_digest,
"HighwayHasher SIMD must match PortableHash scalar byte-for-byte at {size}B"
);
for &chunk_size in &[17usize, 61, 4096] {
if size > 4096 && chunk_size < 61 {
continue;
}
let slices: Vec<&[u8]> = payload.chunks(chunk_size).collect();
for algo in [
HashAlgorithm::HighwayHash256S,
HashAlgorithm::HighwayHash256SLegacy,
HashAlgorithm::SHA256,
HashAlgorithm::BLAKE2b512,
HashAlgorithm::Md5,
] {
let contig_out = algo.hash_encode(&payload);
let sliced_out = algo.hash_encode_slices(slices.iter().copied());
assert_eq!(
contig_out.as_ref(),
sliced_out.as_ref(),
"{algo:?} contiguous vs {chunk_size}B sliced digest mismatch at {size}B"
);
}
}
}
}
#[test]
#[ignore = "utility throughput microbenchmark; run via cargo test -p rustfs-utils --lib --features hash --release -- --ignored --nocapture"]
fn microbench_highwayhash_simd_vs_portable_scalar_and_hash_algorithms() {
use highway::PortableHash;
use std::hint::black_box;
use std::time::Instant;
for &(size, iters) in &[(1024usize, 200usize), (64 * 1024, 40), (1024 * 1024, 8)] {
let payload = generate_compat_test_data(size);
// 1. Compare HighwayHasher (runtime SIMD: AVX2/SSE4.1/NEON) against PortableHash scalar reference.
let t_simd = Instant::now();
let mut simd_digest = [0u8; 32];
for _ in 0..iters {
let mut h = HighwayHasher::new(MAGIC_HIGHWAY_HASH256_PARSED_KEY);
h.append(black_box(&payload));
simd_digest = u8x32_from_u64x4(h.finalize256());
black_box(&simd_digest);
}
let simd_elapsed = t_simd.elapsed();
let t_scalar = Instant::now();
let mut scalar_digest = [0u8; 32];
for _ in 0..iters {
let mut h = PortableHash::new(MAGIC_HIGHWAY_HASH256_PARSED_KEY);
h.append(black_box(&payload));
scalar_digest = u8x32_from_u64x4(h.finalize256());
black_box(&scalar_digest);
}
let scalar_elapsed = t_scalar.elapsed();
assert_eq!(
simd_digest, scalar_digest,
"HighwayHasher SIMD must match PortableHash scalar byte-for-byte at {size}B"
);
let total_mib = (size * iters) as f64 / (1024.0 * 1024.0);
let simd_mibs = total_mib / simd_elapsed.as_secs_f64().max(1e-9);
let scalar_mibs = total_mib / scalar_elapsed.as_secs_f64().max(1e-9);
println!(
"[microbench][utils::hash] size={size}B HighwayHash256S SIMD={simd_mibs:.1} MiB/s vs PortableScalar={scalar_mibs:.1} MiB/s (speedup={:.2}x)",
simd_mibs / scalar_mibs.max(1e-9)
);
// 2. Compare contiguous hash_encode vs 4 KiB chunked hash_encode_slices across all bitrot algorithms.
let slices: Vec<&[u8]> = payload.chunks(4096).collect();
for algo in [
HashAlgorithm::HighwayHash256S,
HashAlgorithm::HighwayHash256SLegacy,
HashAlgorithm::SHA256,
HashAlgorithm::BLAKE2b512,
HashAlgorithm::Md5,
] {
let t_contig = Instant::now();
let mut contig_out = Vec::new();
for _ in 0..iters {
contig_out = algo.hash_encode(black_box(&payload)).as_ref().to_vec();
black_box(&contig_out);
}
let contig_elapsed = t_contig.elapsed();
let t_sliced = Instant::now();
let mut sliced_out = Vec::new();
for _ in 0..iters {
sliced_out = algo.hash_encode_slices(black_box(slices.iter().copied())).as_ref().to_vec();
black_box(&sliced_out);
}
let sliced_elapsed = t_sliced.elapsed();
assert_eq!(contig_out, sliced_out, "{algo:?} contiguous vs 4KiB sliced digest mismatch at {size}B");
let contig_mibs = total_mib / contig_elapsed.as_secs_f64().max(1e-9);
let sliced_mibs = total_mib / sliced_elapsed.as_secs_f64().max(1e-9);
println!(
"[microbench][utils::hash] size={size}B algo={algo:?} contiguous={contig_mibs:.1} MiB/s vs sliced(4KiB)={sliced_mibs:.1} MiB/s"
);
}
}
// 3. Key-routing microbenchmark: crc_hash (CRC32) vs sip_hash (SipHash-1-3).
let keys: Vec<String> = (0..1024)
.map(|i| format!("bucket/prefix-{i}/object-key-{i}.parquet"))
.collect();
let t_crc = Instant::now();
let mut crc_acc = 0usize;
for key in &keys {
crc_acc ^= crc_hash(black_box(key), 16);
}
let crc_ns = t_crc.elapsed().as_nanos();
let t_sip = Instant::now();
let mut sip_acc = 0usize;
for key in &keys {
sip_acc ^= sip_hash(black_box(key), 16, &[0u8; 16]);
}
black_box(sip_acc);
let sip_ns = t_sip.elapsed().as_nanos();
assert!(black_box(crc_acc) < 16 && black_box(sip_acc) < 16);
println!("[microbench][utils::hash] 1024 object keys: crc_hash={crc_ns}ns total, sip_hash={sip_ns}ns total");
}
}
+2
View File
@@ -8,6 +8,8 @@ collection:
## Operations
- [Candidate release pipeline](operations/release-pipeline.md): frozen inputs, staged acceptance, immutable publication and recovery.
- [Sampled local batch CPU diagnostics](operations/local-batch-cpu-diagnostics.md) — default-off request-correlated allocation and worker wall/CPU diagnosis.
- [Retained MRF responsibilities](operations/mrf-responsibilities.md) — unknown bucket identities, bounded probes, and unreadable checkpoint recovery.
+79
View File
@@ -0,0 +1,79 @@
# Candidate acceptance and release publication
RustFS builds the final product version before publication. A candidate is identified by its Actions run and `<version>-preview.<run_number>`; this identifier is not a Git tag or the executable's displayed version. Failed candidates consume no public version. A public tag is created only after the complete artifact manifest is approved.
## Entry points
| Workflow | Input | Result | Public writes |
|---|---|---|---|
| `release-candidate.yml` | Exact final version and full source SHA | Four Linux archives, native checks, upgrade results, Linux manifest | None |
| `release-complete.yml` | Successful candidate run, accepted Linux manifest SHA-256, backlog acceptance report | Desktop archives, DEB/RPM, OCI images, Helm chart, full certification and sealed manifest | None |
| `release-publish.yml` | Successful completion run and approved final manifest SHA-256 | Annotated tag, GitHub Release, R2 files, registry images, stable pointers and Helm index | Explicit publication |
The builder workflows (`build.yml`, `package.yml`, `docker.yml`, `helm-package.yml`) are reusable stages. Only `release-publish.yml` has release write credentials. Development builds retain their artifact names and download channel. Daily development builds now select Linux only; macOS/Windows development artifacts are refreshed by an explicit manual build, and both remain mandatory for a completed release candidate. Tag pushes no longer start a second compilation.
## Freeze and build
Select a reviewed source commit whose Cargo version equals the intended final version. Finish exact-source CI before dispatching the candidate. The candidate workflow itself must run from upstream main. Unrelated main commits do not require rebuilding an accepted source.
```bash
gh workflow run release-candidate.yml --ref main \
-f version='<version>' -f source_sha='<full-source-sha>'
```
The initial job records the source and workflow SHAs, release recipe, dependency lock, exact stable Rust version, Console tag/asset/SHA-256, CLI assets, previous stable release and validation plan. All platforms consume that record; none independently follows `latest`. The source commit must contain the same reviewed release recipe as the dispatcher. Relevant recipe changes require a new candidate; unrelated source changes on main do not invalidate the frozen record.
Linux is built first. Each archived executable must display the final version, expected target, source commit and compiler version, then start natively, serve the Console, and pass an authenticated S3 bucket/object round trip. The upgrade lane runs the existing pinned compatibility matrix plus the stable version frozen at candidate creation. It compiles the harness and runs the accepted executable; it does not rebuild a substitute server. A zero-test invocation fails.
The Linux manifest is partial evidence and cannot authorize publication. Download the exact artifacts for field acceptance. Record the source, manifest digest, run URL and actual tested binary digests in backlog. Required field acceptance includes Scanner/Heal and multi-pool behavior, major features, memory, high-concurrency 503s, performance against the agreed baseline, previous-version data integrity and upgrade recovery. The agreed stability duration is a release-specific requirement, not an implicit workflow default.
## Complete the accepted candidate
After the Linux report passes, dispatch `release-complete.yml` with the exact digest printed by the candidate run. The `release-linux-acceptance` environment is the operator gate. Configure its protection and allowed main branch in repository settings; a typed digest and report identify what is being accepted, while the workflow independently revalidates automated evidence.
```bash
gh workflow run release-complete.yml --ref main \
-f candidate_run_id='<run-id>' \
-f linux_manifest_sha256='<accepted-linux-manifest-sha256>' \
-f acceptance_report='https://github.com/rustfs/backlog/issues/<issue>#issuecomment-<comment>'
```
macOS and Windows now build and run the same native checks. Packages and images wrap the accepted Linux binaries. DEB/RPM extraction must reproduce the recorded binary digest. Both OCI variants are checked on amd64 and arm64 and scanned before sealing. Vulnerability scans retain the repository's report-only policy; scanner execution errors still fail the stage. Image OS packages are resolved during packaging, then the actual resulting OCI bytes are accepted and preserved; this is not a promise of reproducible package-manager output.
The Helm chart is tested and packaged before sealing, with chart and application versions equal to the final product version. Full certification requires every platform and upgrade case from the original plan. The seal includes artifact hashes, raw evidence, source/input identities, acceptance report, checksums, SBOM and provenance. Successful Linux receipts remain valid while desktop results are added because each receipt binds its own exact archive and executable, the frozen plan, candidate identity and trusted producer.
Native smoke checks are not a substitute for Console UI and rc acceptance. Complete the release skill's UI/client checks against these exact artifacts, including desktop checks applicable to the release, before asking for final publication approval.
## Approve and publish
Review the complete run and final `release-manifest.json`. Human approval binds its SHA-256, source and version. A changed manifest requires renewed approval. No source version bump, compilation, package generation or image build occurs during publication.
```bash
gh workflow run release-publish.yml --ref main \
-f complete_run_id='<successful-completion-run-id>' \
-f manifest_sha256='<approved-final-manifest-sha256>'
```
The publisher downloads the artifact from the exact successful main workflow run, checks GitHub's artifact digest, recomputes file hashes and reruns complete certification. It creates an annotated tag whose message binds the source and manifest. An existing tag or asset with different bytes is a hard failure; tags are never moved and uploads never clobber assets.
Versioned GitHub assets, R2 objects and Docker Hub/GHCR/Quay manifests are uploaded and read back before the Release becomes public. OCI promotion uses `--preserve-digests`; registry conversion fails rather than silently accepting a new image. R2 creation uses atomic `If-None-Match: *`. The small evidence bundle persists with the public Release after Actions artifacts expire. Provenance is descriptive, not a claim of cryptographic attestation.
Publication uses a per-version lock and a shared channel lock, both without cancelling an active run. Stable channels advance only after versioned delivery succeeds. GitHub Latest acts as a monotonic marker during partial channel updates, so an older retry cannot downgrade Docker tags, R2 aliases or installer pointers. Prereleases do not advance stable channels. Helm publishes the already accepted chart after versioned images exist. GitHub's legacy `*-latest.zip` download names remain aliases of the accepted bytes; they become default downloads only when GitHub Latest advances.
Cross-registry publication is not atomic. A channel job can fail after some verified destinations advance. Resume the same run/seal: matching destinations are reused and conflicting ones stop the workflow. Retrying never recompiles. Do not claim complete publication until the channel receipt and all public endpoints are verified.
## Failure and retention
- Infrastructure failure: rerun failed jobs in the same run. Frozen inputs, successful platform/package/image artifacts and completed seals are reused. Missing or expired artifacts fail closed.
- Product or build-input change before tagging: create a new candidate run and repeat affected acceptance. Do not mutate the old candidate.
- Failure after tagging: resume publication of exactly that seal. A code fix requires a new patch version; never move the tag.
- New main commits: retain evidence for unchanged source, recipe and inputs. Recheck only affected boundaries; do not restart the entire matrix because an unrelated PR merged.
- Artifacts are retained for 30 days while awaiting acceptance. Public-repository Actions artifacts and staging identifiers are not confidential storage. Preserve important failure evidence in task-owned storage and backlog.
## Migration and workflow maintenance
Finish any in-flight legacy tag release, including its downstream Docker/package/Helm jobs, before enabling this cutover on main. Existing tags and Releases retain their identities. Do not feed a legacy public preview binary into the new promotion path: it embeds a different version and was not built from the frozen-input contract.
All workflow responsibilities, triggers and schedule ownership remain registered in `.github/scheduled-validations.json`. The migration does not re-enable workflows disabled in GitHub. PR checks, cache warming, nightly development builds, functional chains, lab tests and monitoring retain their distinct purposes; shared source or cache does not make different feature/platform tests redundant. The watchdog allocates a runner only for relevant unsuccessful scheduled/chained runs, while its registry still determines ownership.
Run `actionlint`, the release contract suite and the workflow registry validator after changes. The Security Audit workflow also exercises actual GitHub artifact transport, DEB/RPM package extraction, both multi-platform OCI builders, local registry promotion and archive scanning. Native packaging jobs run only for relevant changes or a manual audit. Their payloads are explicit transport fixtures, not product release acceptance. See the [release skill](../../.agents/skills/rustfs-release-publish/SKILL.md) for field acceptance, human confirmation and post-release follow-up.
+1
View File
@@ -101,6 +101,7 @@ Scheduled lanes never block a PR. Their workflow-local gate fails the run. The v
| `coverage.yml` (weekly) | `coverage` | report-only trend; lcov and JSON artifact | yes (disabled) | `make coverage` |
| `runner-hygiene.yml` (monthly) | `check-ephemerality` | runner ephemerality | yes | dispatch |
| `e2e-upgrade.yml` (weekly, and on version tags) | `upgrade` (serial cases sharing one harness build) | upgrade and rollback gate; server logs | yes | `python3 scripts/e2e_binary.py build`, then the job's `python3 scripts/e2e_binary.py run -- cargo test --locked -p e2e_test` commands with `RUSTFS_UPGRADE_SOURCE_BINARY` pointing at the pinned previous release (`UPGRADE_SOURCE_VERSION`) |
| `reference-parity.yml` (nightly, and on PRs that change the lane's own files) | `plan`, `build-candidate`, `s3tests` (one job per target in `.config/reference-parity-targets.toml`), `compare` | report-only parity gate: fails on any per-node-id s3-tests difference between a branch candidate and the release pinned in `.config/reference-binary.toml` that `.config/reference-parity-tolerated.txt` does not list, and on a stale tolerated entry; per-target JUnit, logs and `diff.md`/`diff.json` | yes | `DEPLOY_MODE=binary RUSTFS_BINARY=<binary> MAXFAIL=0 XDIST=4 ./scripts/s3-tests/run.sh` once per target, then `python3 scripts/s3-tests/reference_parity.py compare` as in the `compare` job |
| `oidc-keycloak.yml` (weekly) | `oidc-keycloak-live` | live OIDC gate | yes | see the PR row |
| `targets-integration.yml` (nightly) | `targets-live` | live target gate; container logs | yes (disabled) | see the PR row |
| `gateway-integration-sync.yml` (weekdays) | `sync` | merges `main` into `gateway/integration` as a `--no-ff` merge commit; a conflicting merge is aborted and opens a `sync/<source>-<date>` pull request listing the conflicting files, which keeps the run green; a rejected push or missing pull request fails it | yes | dispatch, optionally with `source_ref` |
+2
View File
@@ -8,6 +8,8 @@ The classifier reads the pull request base revision's planner and policy. It com
The committed policy is `enforced`: ordinary Rust changes select their reverse dependency closure plus the fixed core inventory, and unrelated changes can omit the Connect specialization. Package-owned fixtures, non-Rust inputs, build scripts, manifests, unknown paths and non-PR events retain full validation. Existing documentation and isolated E2E scopes remain conservative. Changing a PR's own policy cannot authorize skipping its base revision's requirements. A base planner predating the current workflow contract clears all narrower outputs and selects the full matrix. Set the policy back to `shadow` to retain proposed selections while restoring full ordinary-code execution; do not remove core inventories or negative checks as a rollback.
Stable installation-version alignment has one content-based exception in enforced mode: literal version fields in `flake.nix`, `helm/rustfs/Chart.yaml` and `rustfs.spec`, plus a standard prepended RPM release note, can use the documentation scope. Quick Checks, Typos, CLA and the required aggregate remain. The planner compares both complete Git blobs and requires ordinary non-executable files; changes to build logic, CLI pins, chart settings, RPM macros or older changelog entries retain the full gate. New/deleted files, mode changes, prerelease syntax, unavailable history, shadow rollout and explicit full-path policy overrides also retain full validation. Product-code changes in the same PR retain their normal dependency and core coverage, and main pushes, merge groups, scheduled and manual runs still validate the complete candidate. This exception does not establish artifact availability or authorize a release; post-release checks still verify the published target before updating defaults.
`Test and Lint` remains the required aggregate name. Every verification producer and consumer is a direct dependency. The verifier accepts only the effective plan's exact expected success/skipped outcomes; failure, cancellation, missing results, extra jobs or a plan for a different candidate fail. Quick Checks and CLA retain their separate required names.
## Build and execute
+1 -1
View File
@@ -77,7 +77,7 @@
rustfs = rustPlatform.buildRustPackage {
pname = "rustfs";
version = "1.0.4";
version = "1.0.5";
src = ./.;
+2 -2
View File
@@ -2,8 +2,8 @@ apiVersion: v2
name: rustfs
description: RustFS helm chart to deploy RustFS on kubernetes cluster.
type: application
version: "1.0.4"
appVersion: "1.0.4"
version: "1.0.5"
appVersion: "1.0.5"
home: https://rustfs.com
icon: https://media.sys.truenas.net/apps/rustfs/icons/icon.svg
maintainers:
+4 -1
View File
@@ -1,7 +1,7 @@
%global _enable_debug_packages 0
%global _empty_manifest_terminate_build 0
Name: rustfs
Version: 1.0.4
Version: 1.0.5
Release: 1
Summary: High-performance distributed object storage for MinIO alternative
@@ -57,6 +57,9 @@ install %_builddir/%{name}-%{version}/target/%_arch/%_arch-unknown-linux-gnu/rel
%_bindir/rustfs
%changelog
* Wed Nov 11 2026 overtrue <anzhengchao@gmail.com>
- Update RPM package to RustFS 1.0.5
* Mon Oct 26 2026 overtrue <anzhengchao@gmail.com>
- Update RPM package to RustFS 1.0.4
+96 -27
View File
@@ -254,11 +254,15 @@ impl DefaultAdminUsecase {
/// absent from `buckets_usage` — absent already means "unknown" on the
/// wire, distinct from a present zero — and the snapshot is marked
/// non-converged so clients can tell it is not the whole namespace. A
/// snapshot that is structurally incomplete, or that measured nothing the
/// namespace still contains, carries no usable information and is dropped
/// exactly as before.
fn narrow_data_usage_snapshot_to_measured_buckets(info: &mut DataUsageInfo, buckets: impl IntoIterator<Item = String>) {
if !info.is_complete_bucket_usage_snapshot() && !info.is_valid_partial_snapshot() {
/// snapshot that is structurally incomplete (`snapshot_measured` is false),
/// or that measured nothing the namespace still contains, carries no usable
/// information and is dropped exactly as before.
fn narrow_data_usage_snapshot_to_measured_buckets(
info: &mut DataUsageInfo,
snapshot_measured: bool,
buckets: impl IntoIterator<Item = String>,
) {
if !snapshot_measured {
*info = DataUsageInfo::default();
return;
}
@@ -275,13 +279,31 @@ impl DefaultAdminUsecase {
let covers_namespace = current_buckets.len() == info.buckets_usage.len();
info.buckets_count = info.buckets_usage.len() as u64;
if !covers_namespace {
// Some bucket in the namespace was never measured, so the totals
// below are a floor rather than the converged truth.
if !covers_namespace || !info.is_complete_bucket_usage_snapshot() {
// Some bucket in the namespace was never measured, or holds
// mutations no scan has reconciled yet, so the totals below are
// not the converged truth.
info.usage_snapshot_converged = Some(false);
}
}
/// Overlay single-process counts on a persisted snapshot and narrow it
/// for display.
///
/// Whether the snapshot measured anything is decided before the overlay.
/// The overlay clears `usage_snapshot_complete` when it holds counts it
/// cannot reconcile with the snapshot, such as writes into a bucket that
/// has no verified baseline. That makes the totals unconverged, not the
/// scanned buckets unusable; treating it as structural incompleteness
/// blanked usage for every bucket for as long as one such bucket kept
/// receiving writes, because each write restarts its scanner fence.
async fn present_data_usage_snapshot(mut info: DataUsageInfo, buckets: impl IntoIterator<Item = String>) -> DataUsageInfo {
let snapshot_measured = info.is_complete_bucket_usage_snapshot() || info.is_valid_partial_snapshot();
apply_bucket_usage_memory_overlay(&mut info).await;
Self::narrow_data_usage_snapshot_to_measured_buckets(&mut info, snapshot_measured, buckets);
info
}
async fn rebalance_status_snapshot(store: &ECStore) -> AdminUsecaseResult<Option<RebalanceMeta>> {
store.rebalance_status_meta_snapshot().await.map_err(|err| {
error!("refresh rebalance metadata for pool status failed: {:?}", err);
@@ -323,8 +345,7 @@ impl DefaultAdminUsecase {
/// This path never triggers a live full-version listing
/// (rustfs/backlog#1306); freshness is owned by the scanner.
pub(crate) async fn query_data_usage_info_with_store(store: Arc<ECStore>) -> AdminUsecaseResult<DataUsageInfo> {
let mut info = Self::map_data_usage_load_result(load_admin_data_usage_from_backend_cached(store.clone()).await)?;
apply_bucket_usage_memory_overlay(&mut info).await;
let info = Self::map_data_usage_load_result(load_admin_data_usage_from_backend_cached(store.clone()).await)?;
let buckets = store
.list_bucket(&BucketOptions {
cached: true,
@@ -333,11 +354,7 @@ impl DefaultAdminUsecase {
})
.await
.map_err(|_| Self::app_error(S3ErrorCode::InternalError, "list_bucket failed"))?;
// Serve what the scan measured; buckets it never reached stay absent
// (absent means unknown on the wire, unlike a present zero) and the
// snapshot is flagged non-converged. See rustfs#5806 for why blanking
// the whole document instead was worse than the rule it enforced.
Self::narrow_data_usage_snapshot_to_measured_buckets(&mut info, buckets.into_iter().map(|bucket| bucket.name));
let mut info = Self::present_data_usage_snapshot(info, buckets.into_iter().map(|bucket| bucket.name)).await;
let storage_info = StorageAdminApi::storage_info(store.as_ref()).await;
@@ -876,6 +893,12 @@ mod tests {
});
}
/// Narrow a snapshot no overlay has touched, judging it as persisted.
fn narrow_persisted_snapshot(info: &mut DataUsageInfo, buckets: impl IntoIterator<Item = String>) {
let snapshot_measured = info.is_complete_bucket_usage_snapshot() || info.is_valid_partial_snapshot();
DefaultAdminUsecase::narrow_data_usage_snapshot_to_measured_buckets(info, snapshot_measured, buckets);
}
/// rustfs#5806: a bucket the last scan never reached must not blank the
/// usage the scan *did* measure — that turned every bucket creation into a
/// deployment-wide usage blackout until the next cycle.
@@ -897,10 +920,7 @@ mod tests {
// A brand-new bucket the scan never saw: keep bucket-a, omit bucket-new,
// and say so via the convergence flag.
let mut info = measured("bucket-a");
DefaultAdminUsecase::narrow_data_usage_snapshot_to_measured_buckets(
&mut info,
["bucket-a".to_string(), "bucket-new".to_string(), ".rustfs.sys".to_string()],
);
narrow_persisted_snapshot(&mut info, ["bucket-a".to_string(), "bucket-new".to_string(), ".rustfs.sys".to_string()]);
assert!(info.buckets_usage.contains_key("bucket-a"), "measured usage must survive");
assert!(!info.buckets_usage.contains_key("bucket-new"), "unmeasured bucket stays absent");
assert_eq!(info.usage_snapshot_converged, Some(false));
@@ -909,22 +929,19 @@ mod tests {
// Full coverage leaves the snapshot untouched.
let mut info = measured("bucket-a");
DefaultAdminUsecase::narrow_data_usage_snapshot_to_measured_buckets(
&mut info,
["bucket-a".to_string(), ".rustfs.sys".to_string()],
);
narrow_persisted_snapshot(&mut info, ["bucket-a".to_string(), ".rustfs.sys".to_string()]);
assert_eq!(info.usage_snapshot_converged, None, "a covering snapshot is not downgraded");
assert_eq!(info.buckets_count, 1);
// A bucket deleted since the scan must not linger in the response.
let mut info = measured("bucket-gone");
DefaultAdminUsecase::narrow_data_usage_snapshot_to_measured_buckets(&mut info, ["bucket-live".to_string()]);
narrow_persisted_snapshot(&mut info, ["bucket-live".to_string()]);
assert_eq!(info, DataUsageInfo::default(), "nothing measured is still in the namespace");
// Structurally incomplete snapshots carry no usable information.
let mut info = measured("bucket-a");
info.usage_snapshot_complete = false;
DefaultAdminUsecase::narrow_data_usage_snapshot_to_measured_buckets(&mut info, ["bucket-a".to_string()]);
narrow_persisted_snapshot(&mut info, ["bucket-a".to_string()]);
assert_eq!(info, DataUsageInfo::default());
// A structurally valid partial admin view can still carry useful
@@ -945,7 +962,7 @@ mod tests {
tombstone: false,
segment_invalidation_proof: None,
}];
DefaultAdminUsecase::narrow_data_usage_snapshot_to_measured_buckets(&mut info, ["bucket-a".to_string()]);
narrow_persisted_snapshot(&mut info, ["bucket-a".to_string()]);
assert_eq!(info.usage_snapshot_converged, Some(false));
assert_eq!(info.buckets_count, 1);
assert_eq!(info.objects_total_count, 7);
@@ -957,11 +974,63 @@ mod tests {
buckets_count: 0,
..Default::default()
};
DefaultAdminUsecase::narrow_data_usage_snapshot_to_measured_buckets(&mut info, [".rustfs.sys".to_string()]);
narrow_persisted_snapshot(&mut info, [".rustfs.sys".to_string()]);
assert_eq!(info.usage_snapshot_converged, None);
assert_eq!(info.buckets_count, 0);
}
/// A write into a bucket without a verified baseline leaves process-local
/// counts the overlay cannot reconcile, so it clears
/// `usage_snapshot_complete`. That must flag the response non-converged,
/// not erase every bucket the persisted scan measured: under continuous
/// writes the scanner fence never clears and the blackout never ends.
#[tokio::test]
async fn unverified_bucket_writes_do_not_blank_measured_usage() {
let measured_bucket = "admin-usage-measured";
let unverified_bucket = "admin-usage-unverified";
let mut persisted = DataUsageInfo {
last_update: Some(std::time::SystemTime::now() - std::time::Duration::from_secs(60)),
usage_snapshot_complete: true,
buckets_count: 1,
..Default::default()
};
persisted.buckets_usage.insert(
measured_bucket.to_string(),
rustfs_data_usage::BucketUsageInfo {
objects_count: 4_000,
versions_count: 4_000,
size: 8_192_000,
..Default::default()
},
);
persisted.bucket_sizes.insert(measured_bucket.to_string(), 8_192_000);
persisted.calculate_totals();
assert!(persisted.is_complete_bucket_usage_snapshot());
crate::app::storage_api::test::data_usage::record_bucket_object_write_memory(unverified_bucket, None, 1_024).await;
let info = DefaultAdminUsecase::present_data_usage_snapshot(
persisted,
[measured_bucket.to_string(), unverified_bucket.to_string()],
)
.await;
assert_eq!(
info.buckets_usage
.get(measured_bucket)
.map(|usage| (usage.objects_count, usage.size)),
Some((4_000, 8_192_000)),
"usage the scanner measured must survive an unrelated unverified bucket"
);
assert!(
!info.buckets_usage.contains_key(unverified_bucket),
"a bucket without a verified baseline stays absent (unknown)"
);
assert_eq!(info.usage_snapshot_converged, Some(false));
assert!(!info.is_complete_bucket_usage_snapshot());
assert_eq!(info.objects_total_count, 4_000);
}
#[tokio::test]
async fn execute_query_storage_info_returns_internal_error_when_store_uninitialized() {
let usecase = DefaultAdminUsecase::without_context();
+83 -12
View File
@@ -17,6 +17,7 @@
use super::*;
use crate::app::storage_api::object_usecase::DeleteObjectCondition;
use s3s::dto::ObjectIdentifier;
use std::borrow::Cow;
use std::collections::HashSet;
async fn authorize_recursive_delete<T>(
@@ -81,6 +82,19 @@ async fn authorize_recursive_delete<T>(
result
}
// XML body keys bypass s3s's URL-path normalization. Match its private slash
// normalizer without URL-decoding or resolving dot segments in client keys.
fn normalize_delete_object_key(key: &str) -> Cow<'_, str> {
if !key.starts_with('/') {
return Cow::Borrowed(key);
}
let mut normalized = key.split('/').filter(|part| !part.is_empty()).collect::<Vec<_>>().join("/");
if key.ends_with('/') {
normalized.push('/');
}
Cow::Owned(normalized)
}
fn successful_delete_audit_objects(
delete: &s3s::dto::Delete,
successful_results: impl IntoIterator<Item = bool>,
@@ -90,7 +104,9 @@ fn successful_delete_audit_objects(
.iter()
.zip(successful_results)
.filter(|(_, successful)| *successful)
.map(|(requested, _)| AuditObjectVersion::new(requested.key.clone(), requested.version_id.clone()))
.map(|(requested, _)| {
AuditObjectVersion::new(normalize_delete_object_key(&requested.key).into_owned(), requested.version_id.clone())
})
.collect()
}
@@ -590,7 +606,10 @@ impl DefaultObjectUsecase {
// Per-key denials are audited once per denied action, in aggregate, after the loop.
let mut bulk_denials = crate::storage::access::BulkDenialAudit::start(&mut req)?;
for (idx, obj_id) in delete.objects.iter().enumerate() {
if let Err(err) = validate_object_key(&obj_id.key, "DeleteObjects") {
// Authorization and every storage side effect must address the same
// normalized object; retain the request key only for response entries.
let key = normalize_delete_object_key(&obj_id.key);
if let Err(err) = validate_object_key(&key, "DeleteObjects") {
delete_results[idx].error = Some(s3s::dto::Error {
code: Some(err.code().as_str().to_string()),
key: Some(obj_id.key.clone()),
@@ -623,7 +642,7 @@ impl DefaultObjectUsecase {
{
let req_info = req_info_mut(&mut req)?;
req_info.bucket = Some(bucket.clone());
req_info.object = Some(obj_id.key.clone());
req_info.object = Some(key.to_string());
req_info.version_id = version_id.clone();
}
@@ -670,7 +689,7 @@ impl DefaultObjectUsecase {
}
}
if let Err(err) = validate_table_catalog_object_mutation(&bucket, &obj_id.key).await {
if let Err(err) = validate_table_catalog_object_mutation(&bucket, &key).await {
delete_results[idx].error = Some(s3s::dto::Error {
code: Some("InvalidRequest".to_string()),
key: Some(obj_id.key.clone()),
@@ -680,9 +699,9 @@ impl DefaultObjectUsecase {
continue;
}
let synthetic_version_id = delete_condition.is_none() && version_id.is_none() && is_dir_object(&obj_id.key);
let synthetic_version_id = delete_condition.is_none() && version_id.is_none() && is_dir_object(&key);
let object = ObjectToDelete {
object_name: obj_id.key.clone(),
object_name: key.into_owned(),
version_id: version_uuid,
delete_condition,
synthetic_version_id,
@@ -910,15 +929,16 @@ impl DefaultObjectUsecase {
let deleted = (!delete.quiet.unwrap_or_default()).then(|| {
delete_results
.iter()
.filter_map(|result| result.delete_object.as_ref().map(|object| (result, object)))
.map(|(result, object)| DeletedObject {
.enumerate()
.filter_map(|(idx, result)| result.delete_object.as_ref().map(|object| (idx, result, object)))
.map(|(idx, result, object)| DeletedObject {
delete_marker: { if object.delete_marker { Some(true) } else { None } },
delete_marker_version_id: delete_marker_response_version_id(
object.delete_marker_version_id,
object.delete_marker && result.version_suspended && result.requested_version_id.is_none(),
result.synthetic_version_id,
),
key: Some(object.object_name.clone()),
key: Some(delete.objects[idx].key.clone()),
version_id: delete_entry_response_version_id(
object.version_id,
object.delete_marker,
@@ -936,7 +956,12 @@ impl DefaultObjectUsecase {
let errors = delete_results
.iter()
.filter_map(|v| v.error.clone())
.enumerate()
.filter_map(|(idx, result)| {
let mut error = result.error.clone()?;
error.key = Some(delete.objects[idx].key.clone());
Some(error)
})
.collect::<Vec<s3s::dto::Error>>();
let output = DeleteObjectsOutput {
deleted,
@@ -2819,11 +2844,57 @@ mod tests {
assert_eq!(err.message(), Some("Not init"));
}
#[test]
fn delete_objects_key_normalization_matches_s3s_path_keys() {
use s3s::path::{
parse_path_style_with_validation_and_normalization, parse_virtual_hosted_style_with_validation_and_normalization,
};
use s3s::validation::AwsNameValidation;
for key in [
"plain",
"/a/b",
"//a/b",
"/dir///sub//file",
"/dir///",
"/",
"///",
"dir//file",
"/literal%2F+名",
"/dir/../file",
"/dir/./file",
"/dir\\file",
"/a\0b",
] {
let normalized = normalize_delete_object_key(key);
let path = parse_path_style_with_validation_and_normalization(
&format!("/test-bucket/{key}"),
&AwsNameValidation::new(),
true,
)
.expect("path-style key should parse before storage validation");
let virtual_host = parse_virtual_hosted_style_with_validation_and_normalization(
Some("test-bucket"),
&format!("/{key}"),
&AwsNameValidation::new(),
true,
)
.expect("virtual-hosted key should parse before storage validation");
assert_eq!(Some(normalized.as_ref()), path.get_object_key(), "path-style mismatch for {key:?}");
assert_eq!(
Some(normalized.as_ref()),
virtual_host.get_object_key(),
"virtual-hosted mismatch for {key:?}"
);
}
assert_eq!(normalize_delete_object_key(""), "");
}
#[test]
fn delete_objects_audit_details_include_only_successful_request_entries() {
let requested = vec![
ObjectIdentifier {
key: "first-key".to_string(),
key: "/first-key".to_string(),
version_id: None,
..Default::default()
},
@@ -2833,7 +2904,7 @@ mod tests {
..Default::default()
},
ObjectIdentifier {
key: "versioned-key".to_string(),
key: "//versioned-key".to_string(),
version_id: Some("requested-version".to_string()),
..Default::default()
},
+2 -2
View File
@@ -14460,8 +14460,8 @@ fn object_mutation_entrypoints_call_reserved_prefix_guard() {
assert_ordered(
method(delete, "fn execute_delete_objects_owned("),
&[
"validate_object_key(&obj_id.key, \"DeleteObjects\")",
"if let Err(err) = validate_table_catalog_object_mutation(&bucket, &obj_id.key).await {",
"validate_object_key(&key, \"DeleteObjects\")",
"if let Err(err) = validate_table_catalog_object_mutation(&bucket, &key).await {",
"continue;\n }",
"authorized_deletes.push(AuthorizedDelete { idx, object });",
],
+7 -2
View File
@@ -37,11 +37,11 @@ their issue closes.
| `check_no_planning_docs.sh` | ci-gate | Blocks committed planning-type documents | ci.yml Quick Checks; `make pre-commit` |
| `check_no_tokio_io_uring.sh` | ci-gate | Keeps tokio's io-uring backend disabled | ci.yml Quick Checks |
| `check_s3s_footprint.sh` | ci-gate | Exact, lower-only baseline on the direct s3s surface ahead of the gateway migration (reads `.config/s3s-footprint-baseline.txt`; `s3_error_lines` leaves out lines whose file imports the macro from `rustfs-s3-types`, rule in the script header); `--mode allowlist` fails on any Rust file referencing s3s outside `.config/s3s-edge-allowlist.txt`, `--dry-run` lists them | ci.yml Quick Checks; `make pre-commit`; `make script-tests` (`--self-test`) |
| `check_test_count_ratchet.sh` | ci-gate | Exact per-crate test-count and `#[ignore]` ratchet against `.config/test-count-baseline.txt`; no crate may lose tests or gain ignores | ci.yml Quick Checks; `make pre-commit` |
| `check_test_count_ratchet.sh` | ci-gate | Per-crate test-count floor and `#[ignore]` ceiling against `.config/test-count-baseline.txt`; no crate may fall below its tests row or rise above its ignored row (extra tests and new crates pass with a note) | ci.yml Quick Checks; `make pre-commit` |
| `check_unsafe_code_allowances.sh` | ci-gate | Unsafe-code allowance ledger guard | ci.yml Quick Checks |
| `layer-dependency-baseline.txt` | ci-gate (data) | Committed baseline consumed by `check_layer_dependencies.sh` | [Architecture guard troubleshooting](../docs/operations/architecture-guard-troubleshooting.md) |
| `static.sh` | ci-gate | Static-build helper executed inside image builds | `Dockerfile.source`, `Dockerfile.decommission-local` |
| `helm_chart_version.sh` | ci-gate | Keeps the Helm chart version in sync with the release | helm-package.yml |
| `helm_chart_version.sh` | dev-tool | Normalizes chart/app versions for installation references | rustfs-release-version-bump skill |
| `test_helm_templates.sh` | ci-gate | Helm template rendering test | helm-package.yml |
## Test & e2e runners
@@ -55,6 +55,11 @@ their issue closes.
| `test_prepare_replacement_migration.py` | dev-tool | Verifies maintenance approval scope, publication, and stopped-writer assertion | Python unittest; same runbook |
| `test_diagnose_scanner_enumeration_restart.py` | dev-tool | Driver report validation and positive convergence oracle tests | Python unittest; same guide |
| `test_object_generation_protocol_model.py` | dev-tool | Bounded single-slot, four-voter promise/accept design model and recovery/retirement examples; no production runtime evidence | `python3 scripts/test_object_generation_protocol_model.py`; [Generation contract](../docs/architecture/unified-object-generation.md#executable-design-model-and-acceptance-mapping) |
| `junit_diff.py` | dev-tool | Case-by-case pass/fail/skip diff of two JUnit XML reports (reference vs candidate stack); exit 1 on any difference or stale `--allow` entry, 2 on unreadable input | `python3 scripts/junit_diff.py a.xml b.xml`; backlog T3.1 |
| `test_junit_diff.py` | ci-gate | Contract tests and fixtures (`scripts/tests/junit_diff/`) for `junit_diff.py` | `make script-tests` |
| `s3-tests/reference_parity.py` | ci-gate | Plans the reference parity targets, lists a run's failures for the single rerun, and compares every candidate's s3-tests JUnit against the pinned reference per node id with a tolerated-difference file | `.github/workflows/reference-parity.yml`; backlog T3.2 |
| `s3-tests/test_reference_parity.py` | ci-gate | Contract tests for `reference_parity.py` | `make script-tests` |
| `test_reference_binary.py` | ci-gate | Contract tests for `.github/actions/reference-binary/reference_binary.py` (pin resolution and verified unpacking) | `make script-tests` |
| `e2e-run.sh` | ci-gate | Boots a rustfs server and runs the `s3s-e2e` black-box conformance tool against it | ci.yml `e2e-tests` jobs; `docs/testing/README.md` |
| `run_ecstore_validation_suite.sh` | dev-tool | ecstore black-box validation suite (`quick`/`full`/`destructive`/`fuzz` profiles) | `docs/testing/README.md`, `docs/testing/ecstore-validation-suite-design.md` |
| `run_e2e_tests.sh` | dev-tool | Local `e2e_test` crate runner (starts a server, applies filters, cleans up) | `crates/e2e_test/README.md` |
+44 -24
View File
@@ -3,7 +3,8 @@
# section 1 rule 3; task T0.3 in rustfs/backlog#2738).
#
# No PR may lower the number of tests in any workspace crate, and no PR may
# add #[ignore]: "delete the test to get green" is rejected in CI. The counter
# add #[ignore] without a reviewed baseline change: "delete the test to get
# green" is rejected in CI. The counter
# is deliberately per crate, never a workspace total — a drop in one crate
# must not hide behind an addition in another.
#
@@ -25,26 +26,37 @@
# doc comments that mention `#[test]` out of the count.
#
# The counts are compared row by row with .config/test-count-baseline.txt
# (`<package>\t<tests>\t<ignored>`, one row per member). The baseline is an
# EXACT value, not a floor, in both columns:
# (`<package>\t<tests>\t<ignored>`, one row per member). The baseline is a
# FLOOR on tests and a CEILING on ignored. It used to be an exact value in both
# columns, which made every PR that adds a test edit the same file; on a branch
# that takes many commits a week those PRs conflict with one another. What the
# ratchet protects is that nobody deletes or ignores tests to get green without
# it showing in the diff, and that only needs a one-sided bound:
#
# - tests below the row fails: `crate <name>: <actual> < baseline <n>`.
# Restore the tests. If the removal is intentional, lower the row in the
# same PR and justify it in the PR description, so the reduction is visible
# in the diff instead of disappearing into a green check.
# - tests above the row fails as well, telling the author to raise the row in
# the same PR. An exact baseline cannot drift upward silently, so the next
# deletion is always measured against the true count.
# - ignored above the row fails (the line contains `ignored`); adding
# #[ignore] is not accepted on gateway/integration. Ignored below the row
# fails too, asking for the row to be lowered so that ratchet stays tight.
# - tests above the row passes and prints a note on stdout. Rows are raised
# only by `--update`, in whichever PR chooses to tighten them.
# - ignored above the row fails (the line contains `ignored >`): raise the
# row in the same PR and justify the new #[ignore] in the PR description.
# Ignored below the row passes with a note.
# - a row whose crate is no longer a member fails (deleting a crate deletes
# its tests), and a member without a row fails.
# its tests; remove the row in the same PR).
# - a member without a row passes with a note, so adding a crate does not
# block on this file; the next `--update` picks it up. Until then the new
# crate has no floor.
#
# Failures go to stderr and exit 1. Notes go to stdout and never use the
# failure shapes above (` < baseline`, `ignored >`, `not a workspace member`).
#
# `--update` rewrites the baseline from the current tree. Review the diff it
# produces before committing it; it is the only sanctioned way to change the
# file other than editing the single row a PR touched.
#
# Contract tests: scripts/test_check_test_count_ratchet.sh (make script-tests).
#
# Usage:
# scripts/check_test_count_ratchet.sh # compare against the baseline
# scripts/check_test_count_ratchet.sh --update # regenerate the baseline
@@ -132,10 +144,12 @@ if [[ "$mode" == "update" ]]; then
echo "# One row per workspace member: <package>, tests, ignored, tab-separated."
echo "# tests counts #[test], #[tokio::test], #[tokio::test(...)] and #[test_case(...)]"
echo "# attribute lines under the package directory; ignored counts #[ignore] and"
echo "# #[ignore = \"...\"]. Both columns are exact: CI fails when a crate's tests fall"
echo "# below or rise above its row, or when its ignored count changes. Raise a row in"
echo "# the PR that adds tests; lowering one is a reviewed statement that tests were"
echo "# intentionally removed. Regenerate with:"
echo "# #[ignore = \"...\"]. The tests column is a floor and the ignored column is a"
echo "# ceiling: CI fails when a crate's tests fall below its row or its ignored count"
echo "# rises above it. Extra tests and fewer ignores pass; --update tightens the rows."
echo "# Lowering a tests row or raising an ignored row is a reviewed statement that"
echo "# tests were intentionally removed or ignored; justify it in the PR description."
echo "# Regenerate with:"
echo "# scripts/check_test_count_ratchet.sh --update"
cat "$TMP_DIR/actual"
} >"$BASELINE_FILE"
@@ -165,9 +179,10 @@ if cut -f1 "$TMP_DIR/baseline" | LC_ALL=C sort | uniq -d | grep . >"$TMP_DIR/dup
exit 1
fi
# Every violation is one line. The `<actual> < baseline <n>` and `ignored`
# shapes are the acceptance contract of rustfs/backlog#2738; keep them.
awk -F'\t' '
# Every violation is one line on stderr; every note is one line on stdout. The
# `<actual> < baseline <n>` and `ignored >` shapes are the acceptance contract
# of rustfs/backlog#2738 and the contract tests; notes must never reuse them.
awk -F'\t' -v notes="$TMP_DIR/notes" '
NR == FNR {
baseline_tests[$1] = $2
baseline_ignored[$1] = $3
@@ -176,18 +191,18 @@ NR == FNR {
{
seen[$1] = 1
if (!($1 in baseline_tests)) {
printf "crate %s: no baseline row (%d tests, %d ignored); add it in this PR\n", $1, $2, $3
printf "crate %s: no baseline row (%d tests, %d ignore attributes); run --update to add it\n", $1, $2, $3 >notes
next
}
if ($2 + 0 < baseline_tests[$1] + 0) {
printf "crate %s: %d < baseline %d\n", $1, $2, baseline_tests[$1]
} else if ($2 + 0 > baseline_tests[$1] + 0) {
printf "crate %s: %d > baseline %d; raise the baseline row to %d in this PR\n", $1, $2, baseline_tests[$1], $2
printf "crate %s: %d above baseline %d; run --update to tighten\n", $1, $2, baseline_tests[$1] >notes
}
if ($3 + 0 > baseline_ignored[$1] + 0) {
printf "crate %s: %d ignored > baseline %d; adding #[ignore] is not accepted\n", $1, $3, baseline_ignored[$1]
printf "crate %s: %d ignored > baseline %d; raise the baseline row to %d in this PR and justify the new #[ignore] in the PR description\n", $1, $3, baseline_ignored[$1], $3
} else if ($3 + 0 < baseline_ignored[$1] + 0) {
printf "crate %s: %d ignored < baseline %d; lower the baseline row to %d in this PR\n", $1, $3, baseline_ignored[$1], $3
printf "crate %s: %d ignore attributes below baseline %d; run --update to tighten\n", $1, $3, baseline_ignored[$1] >notes
}
}
END {
@@ -199,14 +214,19 @@ END {
}
' "$TMP_DIR/baseline" "$TMP_DIR/actual" | LC_ALL=C sort >"$TMP_DIR/violations"
if [[ -f "$TMP_DIR/notes" ]]; then
LC_ALL=C sort "$TMP_DIR/notes" | sed 's/^/note: /'
fi
if grep -q . "$TMP_DIR/violations"; then
cat "$TMP_DIR/violations" >&2
echo "error: test-count ratchet violated (rustfs/backlog#2734 section 1 rule 3)." >&2
echo " A crate below its baseline lost tests: restore them. If the removal is" >&2
echo " intentional, lower that row in $BASELINE_FILE in the same PR and" >&2
echo " justify it in the PR description. A crate above its baseline raises its" >&2
echo " row in the same PR ('$0 --update' rewrites the file; review the diff)." >&2
echo " justify it in the PR description. A crate above its ignored baseline" >&2
echo " raises that row in the same PR with a justification in the PR description" >&2
echo " ('$0 --update' rewrites the file; review the diff)." >&2
exit 1
fi
echo "test-count ratchet OK: $crate_count crates, $total_tests tests, $total_ignored ignored match $BASELINE_FILE"
echo "test-count ratchet OK: $crate_count crates, $total_tests tests, $total_ignored ignored; every crate at or above its tests floor and at or below its ignored ceiling in $BASELINE_FILE"
+62 -3
View File
@@ -98,6 +98,58 @@ def git_file(root: Path, revision: str, path: str) -> str:
return git(root, "show", f"{revision}:{path}")
def release_metadata_only(root: Path, base: str, head: str, path: str) -> bool:
"""Recognize literal stable-version edits, never general packaging changes."""
stable = r"(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)"
marker = "<version>"
patterns = {
"flake.nix": [rf'(?m)^([ \t]*pname = "rustfs";\n[ \t]*version = ")({stable})(";)$'],
"helm/rustfs/Chart.yaml": [rf'(?m)^({key}: ")({stable})(")$' for key in ("version", "appVersion")],
"rustfs.spec": [rf'(?m)^(Version:[ \t]+)({stable})()$'],
}
if path not in patterns or not SHA.fullmatch(base) or not SHA.fullmatch(head):
return False
try:
normalized, versions = [], []
for revision in (base, head):
# A symlink, added/deleted file or executable-mode change is a
# build input even if its apparent text resembles metadata.
entry = git(root, "ls-tree", "-z", revision, "--", path)
if not re.fullmatch(rf"100644 blob [0-9a-f]{{40}}\t{re.escape(path)}\x00", entry):
return False
content, values = git_file(root, revision, path), []
# Existing markers could make edits to distinct fields collapse
# to identical normalized text.
if marker in content:
return False
for pattern in patterns[path]:
matches = list(re.finditer(pattern, content))
if len(matches) != 1:
return False
values.append(matches[0][2])
content = re.sub(pattern, lambda match: match[1] + marker + match[3], content)
if len(set(values)) != 1:
return False
normalized.append(content)
versions.append(values[0])
if path == "rustfs.spec":
old, new = (content.split("%changelog\n") for content in normalized)
if len(old) != 2 or len(new) != 2 or old[0] != new[0]:
return False
if old[1] == new[1]:
return True
# Only the standard prepended release note is non-executable;
# RPM macro syntax and edits to older history retain the full gate.
note = (r"\* (?:Mon|Tue|Wed|Thu|Fri|Sat|Sun) "
r"(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) [0-9]{2} [0-9]{4} "
r"[A-Za-z0-9 ._-]+ <[A-Za-z0-9._+-]+@[A-Za-z0-9.-]+>\n"
rf"- Update RPM package to RustFS {re.escape(versions[1])}\n\n")
return re.fullmatch(note + re.escape(old[1]), new[1]) is not None
return normalized[0] == normalized[1]
except (OSError, subprocess.CalledProcessError, UnicodeError, ValueError):
return False
def dependency_tables(document: dict):
for key, value in document.items():
if key in ("dependencies", "dev-dependencies", "build-dependencies"):
@@ -283,10 +335,17 @@ def select_plan(event: str, base: str, head: str, ref: str, root: Path, action:
base_packages, base_graph = head_packages, head_graph
reasons.append(f"conservative fallback: {type(error).__name__}")
graph = {name: head_graph.get(name, set()) | base_graph.get(name, set()) for name in set(head_graph) | set(base_graph)}
affected = set()
affected, non_runtime = set(), set()
if not full and policy is not None:
for path in changed:
if documentation_path(path):
non_runtime.add(path)
continue
if (policy["rollout"] == "enforced"
and not any(fnmatch.fnmatchcase(path, pattern) for pattern in policy["full_paths"])
and release_metadata_only(root, base, head, path)):
non_runtime.add(path)
reasons.append(f"literal stable release metadata: {path}")
continue
if path in E2E_SELECTION_FILES:
affected.add("e2e_test")
@@ -308,9 +367,9 @@ def select_plan(event: str, base: str, head: str, ref: str, root: Path, action:
affected.update(owners)
if not affected and not full:
mode = "docs"
reasons.append("all changes are documentation")
reasons.append("all changes are documentation or literal stable release metadata")
elif affected == {"e2e_test"} and not full and not (closure({"e2e_test"}, graph, reverse=True) - {"e2e_test"}) and all(
documentation_path(path) or path in E2E_SELECTION_FILES or
path in non_runtime or path in E2E_SELECTION_FILES or
(path.startswith("crates/e2e_test/src/") and path.endswith(".rs")) for path in changed):
mode = "e2e"
reasons.append("isolated E2E harness changes")
+1 -6
View File
@@ -17,12 +17,7 @@ case "$RAW" in
esac
APP_VERSION="${RAW_TAG#v}"
BETA_NUM=$(printf '%s\n' "$APP_VERSION" | sed -n -E 's/^.*-beta\.([0-9]+)$/\1/p')
if [ -n "$BETA_NUM" ]; then
CHART_VERSION="0.${BETA_NUM}.0"
else
CHART_VERSION="$APP_VERSION"
fi
CHART_VERSION="$APP_VERSION"
if [ -n "${GITHUB_OUTPUT:-}" ]; then
{
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Compare two JUnit XML reports case by case.
Cases are aligned on classname + name and reduced to pass / fail / skip. A case
present in only one report is reported as `missing` on the other side. A
`<flakyFailure>` case that finally passed is a pass and is listed separately.
Exit codes: 0 no unallowed differences and no stale allow-list entry; 1 at least
one difference or stale allow-list entry; 2 unreadable or malformed input.
"""
import argparse
import json
import sys
import xml.etree.ElementTree as ET
PASS, FAIL, SKIP, MISSING = "pass", "fail", "skip", "missing"
_RETRY_TAGS = {"flakyFailure", "flakyError", "rerunFailure", "rerunError"}
class InputError(Exception):
"""Input that cannot be trusted as a report; maps to exit code 2."""
def case_id(classname, name):
return f"{classname}::{name}" if classname else name
def parse_report(path):
"""Return ({case_id: state}, [flaky case ids]) for one JUnit file."""
try:
with open(path, "rb") as handle:
data = handle.read()
except OSError as exc:
raise InputError(f"{path}: cannot read: {exc.strerror or exc}") from exc
if not data.strip():
raise InputError(f"{path}: file is empty")
try:
root = ET.fromstring(data)
except ET.ParseError as exc:
raise InputError(f"{path}: malformed XML: {exc}") from exc
if root.tag not in ("testsuites", "testsuite"):
raise InputError(f"{path}: unexpected root element <{root.tag}>")
states, flaky = {}, []
for case in root.iter("testcase"):
name = case.get("name")
if not name:
raise InputError(f"{path}: <testcase> without a name")
cid = case_id(case.get("classname", ""), name)
if cid in states:
raise InputError(f"{path}: duplicate case {cid}")
tags = {child.tag for child in case}
if tags & {"failure", "error"}:
state = FAIL
elif "skipped" in tags:
state = SKIP
else:
state = PASS
states[cid] = state
if state == PASS and tags & _RETRY_TAGS:
flaky.append(cid)
if not states:
raise InputError(f"{path}: report contains no test cases")
return states, sorted(flaky)
def read_allow(path):
try:
with open(path, encoding="utf-8") as handle:
lines = handle.read().splitlines()
except (OSError, UnicodeDecodeError) as exc:
raise InputError(f"{path}: cannot read allow-list: {exc}") from exc
return {line.strip() for line in lines if line.strip() and not line.lstrip().startswith("#")}
def counts(states):
result = {PASS: 0, FAIL: 0, SKIP: 0}
for state in states.values():
result[state] += 1
return result
def compare(a, b, allow):
diffs = []
for cid in sorted(set(a) | set(b)):
sa, sb = a.get(cid, MISSING), b.get(cid, MISSING)
if sa != sb:
diffs.append({"case": cid, "a": sa, "b": sb, "allowed": cid in allow})
stale = sorted(allow - {d["case"] for d in diffs})
return diffs, stale
def render(args, diffs, stale, flaky_a, flaky_b, counts_a, counts_b):
out = []
for label, c in ((args.label_a, counts_a), (args.label_b, counts_b)):
out.append(f"{label}: pass={c[PASS]} fail={c[FAIL]} skip={c[SKIP]}")
for label, flaky in ((args.label_a, flaky_a), (args.label_b, flaky_b)):
if flaky:
out.append(f"\nFlaky in {label} (passed after retry, not a difference):")
out.extend(f" {cid}" for cid in flaky)
if diffs:
width = max(len("case"), *(len(d["case"]) for d in diffs))
wa = max(len(args.label_a), 7)
out.append("")
out.append(f"{'case':<{width}} {args.label_a:<{wa}} {args.label_b}")
for d in diffs:
note = " (allowed)" if d["allowed"] else ""
out.append(f"{d['case']:<{width}} {d['a']:<{wa}} {d['b']}{note}")
if stale:
out.append("\nStale allow-list entries (no matching difference; remove them):")
out.extend(f" {cid}" for cid in stale)
if not diffs and not stale:
out.append("\nNo differences.")
return "\n".join(out)
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("a", help="first JUnit XML report")
parser.add_argument("b", help="second JUnit XML report")
parser.add_argument("--label-a", default="a")
parser.add_argument("--label-b", default="b")
parser.add_argument("--json", metavar="PATH", help="write a machine-readable summary")
parser.add_argument("--allow", metavar="FILE", help="case ids (one per line) whose difference is tolerated")
args = parser.parse_args(argv)
if args.label_a == args.label_b:
print("error: --label-a and --label-b must differ", file=sys.stderr)
return 2
try:
states_a, flaky_a = parse_report(args.a)
states_b, flaky_b = parse_report(args.b)
allow = read_allow(args.allow) if args.allow else set()
diffs, stale = compare(states_a, states_b, allow)
counts_a, counts_b = counts(states_a), counts(states_b)
if args.json:
summary = {
"labels": {"a": args.label_a, "b": args.label_b},
"counts": {args.label_a: counts_a, args.label_b: counts_b},
"differences": diffs,
"flaky": {args.label_a: flaky_a, args.label_b: flaky_b},
"stale_allow": stale,
}
with open(args.json, "w", encoding="utf-8") as handle:
json.dump(summary, handle, indent=2, sort_keys=True)
handle.write("\n")
except (InputError, OSError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 2
print(render(args, diffs, stale, flaky_a, flaky_b, counts_a, counts_b))
return 1 if stale or any(not d["allowed"] for d in diffs) else 0
if __name__ == "__main__":
sys.exit(main())
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/env bash
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
set -euo pipefail
PKG_DIR="${DEB_FILE%.deb}"
echo "Building DEB: ${DEB_FILE}"
mkdir -p "${PKG_DIR}/DEBIAN"
mkdir -p "${PKG_DIR}/usr/bin"
mkdir -p "${PKG_DIR}/etc/default"
mkdir -p "${PKG_DIR}/lib/systemd/system"
mkdir -p "${PKG_DIR}/usr/share/doc/rustfs"
cp ./bin/rustfs "${PKG_DIR}/usr/bin/"
chmod 755 "${PKG_DIR}/usr/bin/rustfs"
cp deploy/build/rustfs.service "${PKG_DIR}/lib/systemd/system/"
cat > "${PKG_DIR}/etc/default/rustfs" << 'ENVEOF'
# RustFS Environment Configuration
# See https://docs.rustfs.com for more information
# RUSTFS_ACCESS_KEY=<changeme>
# RUSTFS_SECRET_KEY=<changeme>
# RUSTFS_VOLUMES=""
# RUSTFS_ADDRESS=":9000"
# RUSTFS_CONSOLE_ADDRESS=":9001"
# RUSTFS_CONSOLE_ENABLE=true
# RUSTFS_OBS_LOGGER_LEVEL=error
# RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/"
ENVEOF
cat > "${PKG_DIR}/DEBIAN/control" << EOF
Package: rustfs
Version: ${DEB_VERSION}
Section: utils
Priority: optional
Architecture: ${DEB_ARCH}
EOF
# gnu binaries dynamically link glibc and must declare it; musl
# binaries are statically linked and need no libc dependency.
if [[ -n "$DEB_DEPENDS" ]]; then
echo "Depends: ${DEB_DEPENDS}" >> "${PKG_DIR}/DEBIAN/control"
fi
cat >> "${PKG_DIR}/DEBIAN/control" << EOF
Maintainer: RustFS Team <support@rustfs.com>
Description: High-performance distributed object storage (${LIBC} build)
RustFS is a high-performance distributed object storage software
built using Rust. It is compatible with MinIO and S3 API.
Homepage: https://rustfs.com
EOF
# Declare /etc/default/rustfs as a conffile so dpkg preserves user
# modifications on upgrade instead of silently overwriting them.
cat > "${PKG_DIR}/DEBIAN/conffiles" << 'CONFFILES'
/etc/default/rustfs
CONFFILES
./scripts/release/package_service_scripts.sh after-install > "${PKG_DIR}/DEBIAN/postinst"
chmod 755 "${PKG_DIR}/DEBIAN/postinst"
./scripts/release/package_service_scripts.sh before-remove > "${PKG_DIR}/DEBIAN/prerm"
chmod 755 "${PKG_DIR}/DEBIAN/prerm"
./scripts/release/package_service_scripts.sh after-remove > "${PKG_DIR}/DEBIAN/postrm"
chmod 755 "${PKG_DIR}/DEBIAN/postrm"
cp LICENSE "${PKG_DIR}/usr/share/doc/rustfs/"
cp README.md "${PKG_DIR}/usr/share/doc/rustfs/"
fakeroot dpkg-deb --build "${PKG_DIR}" "$DEB_FILE"
[[ $(dpkg-deb -f "$DEB_FILE" Package) == rustfs ]]
[[ $(dpkg-deb -f "$DEB_FILE" Version) == "$DEB_VERSION" ]]
[[ $(dpkg-deb -f "$DEB_FILE" Architecture) == "$DEB_ARCH" ]]
dpkg-deb --fsys-tarfile "$DEB_FILE" | tar -tf - | grep -Fx './usr/bin/rustfs' >/dev/null
stat --printf='%n %s bytes\n' "$DEB_FILE"
echo "deb_file=$DEB_FILE" >> "$GITHUB_OUTPUT"
echo "✅ DEB built: $DEB_FILE"
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env bash
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
set -euo pipefail
echo "Building RPM for ${RPM_ARCH} (${LIBC})"
./scripts/test_package_versions.sh --require-package-managers
# gnu binaries dynamically link glibc and must declare it; musl
# binaries are statically linked and need no libc dependency.
depends_args=()
if [[ -n "$RPM_DEPENDS" ]]; then
depends_args+=(--depends "$RPM_DEPENDS")
fi
# Create config file for fpm (DEB build creates it in its package dir structure,
# but fpm needs the file to exist before packaging)
mkdir -p ./tmp-pkg/etc/default
cat > ./tmp-pkg/etc/default/rustfs << 'ENVEOF'
# RustFS Environment Configuration
# See https://docs.rustfs.com for more information
# RUSTFS_ACCESS_KEY=<changeme>
# RUSTFS_SECRET_KEY=<changeme>
# RUSTFS_VOLUMES=""
# RUSTFS_ADDRESS=":9000"
# RUSTFS_CONSOLE_ADDRESS=":9001"
# RUSTFS_CONSOLE_ENABLE=true
# RUSTFS_OBS_LOGGER_LEVEL=error
# RUSTFS_OBS_LOG_DIRECTORY="/var/log/rustfs/"
ENVEOF
fpm -s dir -t rpm \
--name rustfs \
--version "$RPM_VERSION" \
--iteration "$RPM_RELEASE" \
--architecture "$RPM_ARCH" \
--package "$RPM_FILE" \
"${depends_args[@]}" \
--description "High-performance distributed object storage (${LIBC} build)" \
--maintainer "RustFS Team <support@rustfs.com>" \
--url "https://rustfs.com" \
--license "Apache-2.0" \
--before-install <(./scripts/release/package_service_scripts.sh rpm-before-install) \
--before-upgrade <(./scripts/release/package_service_scripts.sh rpm-before-upgrade) \
--after-install <(./scripts/release/package_service_scripts.sh after-install) \
--after-upgrade <(./scripts/release/package_service_scripts.sh after-install) \
--before-remove <(./scripts/release/package_service_scripts.sh before-remove) \
--after-remove <(./scripts/release/package_service_scripts.sh after-remove) \
--rpm-posttrans <(./scripts/release/package_service_scripts.sh rpm-posttrans) \
--config-files /etc/default/rustfs \
./bin/rustfs=/usr/bin/rustfs \
./tmp-pkg/etc/default/rustfs=/etc/default/rustfs \
deploy/build/rustfs.service=/lib/systemd/system/rustfs.service \
LICENSE=/usr/share/doc/rustfs/LICENSE \
README.md=/usr/share/doc/rustfs/README.md
if [[ ! -f "$RPM_FILE" ]]; then
echo "❌ RPM build failed"
exit 1
fi
RPM_METADATA=$(rpm -qp --qf '%{NAME}\n%{VERSION}\n%{RELEASE}\n%{ARCH}\n' "$RPM_FILE")
EXPECTED_METADATA=$(printf 'rustfs\n%s\n%s\n%s' "$RPM_VERSION" "$RPM_RELEASE" "$RPM_ARCH")
[[ "$RPM_METADATA" == "$EXPECTED_METADATA" ]]
rpm -qpl "$RPM_FILE" | grep -Fx '/usr/bin/rustfs' >/dev/null
stat --printf='%n %s bytes\n' "$RPM_FILE"
echo "rpm_file=$RPM_FILE" >> "$GITHUB_OUTPUT"
echo "✅ RPM built: $RPM_FILE"
+428
View File
@@ -0,0 +1,428 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
"""Frozen release inputs and artifact identity; publication never builds binaries."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import subprocess
import sys
import tempfile
import tomllib
import urllib.request
from urllib.parse import urlsplit
import zipfile
ROOT = Path(__file__).resolve().parents[2]
REPOSITORY = "rustfs/rustfs"
SCHEMA = 1
HEX40 = re.compile(r"[0-9a-f]{40}\Z")
HEX64 = re.compile(r"[0-9a-f]{64}\Z")
VERSION = re.compile(r"(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(?:-(?:alpha|beta|rc)\.(0|[1-9][0-9]*))?\Z")
TARGETS = ("linux-x86_64-musl", "linux-aarch64-musl", "linux-x86_64-gnu", "linux-aarch64-gnu", "macos-aarch64", "windows-x86_64")
LINUX = TARGETS[:4]
DESKTOP = TARGETS[4:]
MAX_MEMBER = 2 * 1024**3
RECIPE_PATHS = (
".github/workflows/build.yml",
".github/workflows/release-candidate.yml",
".github/workflows/release-complete.yml",
".github/workflows/release-publish.yml",
".github/workflows/docker.yml",
".github/workflows/package.yml",
".github/actions/setup/action.yml",
".github/actions/setup/download_release_asset.py",
"scripts/release/candidate.py",
"scripts/release/pipeline.py",
"scripts/release/verify.py",
"scripts/release/deliver.py",
"scripts/release/publish.py",
"scripts/release/release_gate.py",
"scripts/release/package_versions.sh",
"scripts/release/package_service_scripts.sh",
"scripts/release/build_deb.sh",
"scripts/release/build_rpm.sh",
"scripts/release/check_glibc.py",
".config/release-validation-plan.json",
"Dockerfile", "Dockerfile.glibc", "entrypoint.sh", "deploy/build/rustfs.service",
".github/workflows/helm-package.yml", "scripts/test_helm_templates.sh",
"scripts/security/generate_release_supply_chain_assets.py",
".github/workflows/ci.yml", "scripts/ci_gate.py",
)
def require(condition, message):
if not condition:
raise ValueError(message)
def digest(value):
return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()).hexdigest()
def stream_digest(stream):
result = hashlib.sha256()
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
result.update(chunk)
return result.hexdigest()
def file_digest(path):
with Path(path).open("rb") as stream:
return stream_digest(stream)
def read_json(path):
path = Path(path)
require(path.is_file() and path.stat().st_size <= 4 * 1024**2, "missing or oversized JSON record")
def unique(pairs):
value = {}
for name, item in pairs:
require(name not in value, "duplicate JSON field")
value[name] = item
return value
return json.loads(path.read_text(encoding="utf-8"), object_pairs_hook=unique)
def write_json(path, value):
"""Existing identity records are immutable, including across job retries."""
path = Path(path)
if path.exists():
require(read_json(path) == value, "refusing to replace an existing identity record")
return
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("x", encoding="utf-8") as stream:
json.dump(value, stream, indent=2, sort_keys=True, ensure_ascii=False)
stream.write("\n")
def command(args, *, input_text=None, cwd=None):
return subprocess.run(args, input=input_text, text=True, capture_output=True, check=True, cwd=cwd, timeout=120).stdout.strip()
class ArtifactRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, request, fp, code, message, headers, url):
require(urlsplit(url).scheme == "https", "artifact redirect must use HTTPS")
redirected = super().redirect_request(request, fp, code, message, headers, url)
if redirected is not None and urlsplit(request.full_url).netloc != urlsplit(url).netloc:
redirected.remove_header("Authorization")
return redirected
def github_stream(path):
require(path.startswith("repos/") and ":" not in path and not path.startswith("/"), "invalid GitHub API path")
token = os.environ.get("GH_TOKEN") or command(["gh", "auth", "token"])
request = urllib.request.Request("https://api.github.com/" + path,
headers={"Authorization": "Bearer " + token, "Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28", "User-Agent": "rustfs-release"})
return urllib.request.build_opener(ArtifactRedirect()).open(request, timeout=600)
def github(path):
with github_stream(path) as response:
return json.load(response)
def git_file(revision, path, root=ROOT):
require(HEX40.fullmatch(revision) is not None, "source must be a full commit SHA")
return subprocess.check_output(["git", "show", f"{revision}:{path}"], cwd=root, timeout=60)
def recipe_digest(revision, root=ROOT):
files = {path: hashlib.sha256(git_file(revision, path, root)).hexdigest() for path in RECIPE_PATHS}
for directory in ("bootstrap", "helm"):
paths = command(["git", "ls-tree", "-r", "--name-only", revision, "--", directory], cwd=root).splitlines()
require(bool(paths), "missing release recipe directory")
files.update({path: hashlib.sha256(git_file(revision, path, root)).hexdigest() for path in paths})
return digest(files)
def console_asset(release):
require(isinstance(release, dict) and release.get("draft") is False and release.get("prerelease") is False,
"Console must be a published stable release")
tag = release.get("tag_name")
require(isinstance(tag, str) and re.fullmatch(r"v?[0-9]+\.[0-9]+\.[0-9]+", tag), "invalid Console version")
assets = [asset for asset in release.get("assets", [])
if re.fullmatch(r"rustfs-console-[A-Za-z0-9._-]+\.zip", asset.get("name", ""))]
require(len(assets) == 1, "Console must have exactly one unambiguous ZIP asset")
asset = assets[0]
checksum = asset.get("digest", "")
require(isinstance(checksum, str) and checksum.startswith("sha256:") and HEX64.fullmatch(checksum[7:]),
"Console asset must have a SHA-256 digest")
expected = f"https://github.com/rustfs/console/releases/download/{tag}/{asset['name']}"
require(asset.get("browser_download_url") == expected, "Console URL does not match its release identity")
return {"tag": tag, "name": asset["name"], "url": expected, "sha256": checksum[7:]}
def release_asset(release, repository, name):
require(release.get("draft") is False and release.get("prerelease") is False, "dependency must be a published stable release")
tag = release.get("tag_name", "")
require(re.fullmatch(r"v?[0-9]+\.[0-9]+\.[0-9]+", tag), "invalid dependency tag")
matches = [a for a in release.get("assets", []) if a.get("name") == name]
require(len(matches) == 1, "missing or ambiguous pinned dependency")
asset = matches[0]
url = f"https://github.com/{repository}/releases/download/{tag}/{name}"
require(asset.get("browser_download_url") == url and re.fullmatch(r"sha256:[0-9a-f]{64}", asset.get("digest", "")), "invalid dependency identity")
return {"version": tag, "name": name, "url": url, "sha256": asset["digest"][7:]}
def validate_dependency(asset, repository):
require(isinstance(asset, dict) and set(asset) == {"version", "name", "url", "sha256"}, "invalid dependency record")
require(re.fullmatch(r"[A-Za-z0-9._-]+", asset["name"]), "unsafe dependency name")
normalized = release_asset({"draft": False, "prerelease": False, "tag_name": asset["version"],
"assets": [{"name": asset["name"], "browser_download_url": asset["url"],
"digest": "sha256:" + asset["sha256"]}]}, repository, asset["name"])
require(normalized == asset, "dependency record changed")
def validate_inputs(value):
require(isinstance(value, dict), "candidate inputs must be an object")
fields = {"schema", "candidate_id", "version", "source_sha", "workflow_sha", "recipe_sha256", "lock_sha256",
"plan_sha256", "toolchain", "console", "cli", "previous", "ci", "origin_run_id", "origin_run_number"}
require(set(value) == fields and type(value["schema"]) is int and value["schema"] == SCHEMA,
"unsupported candidate input schema")
require(isinstance(value["version"], str) and VERSION.fullmatch(value["version"]), "invalid release version")
for name in ("source_sha", "workflow_sha"):
require(isinstance(value[name], str) and HEX40.fullmatch(value[name]), f"invalid {name}")
for name in ("recipe_sha256", "lock_sha256", "plan_sha256"):
require(isinstance(value[name], str) and HEX64.fullmatch(value[name]), f"invalid {name}")
for name in ("origin_run_id", "origin_run_number"):
require(type(value[name]) is int and value[name] > 0, f"invalid {name}")
require(value["candidate_id"] == f"{value['version']}-preview.{value['origin_run_number']}", "candidate name differs from its immutable run identity")
require(isinstance(value["toolchain"], str) and re.fullmatch(r"[1-9][0-9]*\.[0-9]+\.[0-9]+", value["toolchain"]),
"candidate toolchain must be an exact stable Rust version")
console = value["console"]
require(isinstance(console, dict) and set(console) == {"tag", "name", "url", "sha256"}, "invalid Console identity")
normalized = console_asset({"draft": False, "prerelease": False, "tag_name": console["tag"],
"assets": [{"name": console["name"], "browser_download_url": console["url"],
"digest": "sha256:" + str(console["sha256"])}]})
require(normalized == console, "invalid pinned Console asset")
require(isinstance(value["cli"], dict) and set(value["cli"]) == set(LINUX), "missing CLI platform")
for asset in value["cli"].values():
validate_dependency(asset, "rustfs/cli")
validate_dependency(value["previous"], REPOSITORY)
require(value["previous"]["version"].lstrip("v") != value["version"], "previous release must differ from the candidate")
require(isinstance(value["ci"], dict) and set(value["ci"]) == {"run_id", "run_attempt"}
and all(type(item) is int and item > 0 for item in value["ci"].values()), "invalid exact-source CI identity")
return value
def freeze(version, source_sha, workflow_sha, toolchain, release, run_id, run_number, cli, previous, ci, root=ROOT):
require(VERSION.fullmatch(version) is not None, "invalid release version")
source = tomllib.loads(git_file(source_sha, "Cargo.toml", root).decode())
require(source["workspace"]["package"]["version"] == version, "Cargo workspace version differs from the target")
recipe = recipe_digest(workflow_sha, root)
require(recipe_digest(source_sha, root) == recipe, "candidate must contain the reviewed release recipe")
plan = json.loads(git_file(workflow_sha, ".config/release-validation-plan.json", root))
value = {"schema": SCHEMA, "version": version, "candidate_id": f"{version}-preview.{run_number}",
"source_sha": source_sha, "workflow_sha": workflow_sha, "recipe_sha256": recipe,
"lock_sha256": hashlib.sha256(git_file(source_sha, "Cargo.lock", root)).hexdigest(),
"plan_sha256": digest(plan), "toolchain": toolchain, "console": console_asset(release),
"origin_run_id": run_id, "origin_run_number": run_number, "ci": ci,
"cli": {target: release_asset(cli, "rustfs/cli", f"rustfs-cli-linux-{'amd64' if 'x86_64' in target else 'arm64'}{'-gnu' if target.endswith('gnu') else ''}-{cli['tag_name']}.tar.gz") for target in LINUX},
"previous": release_asset(previous, REPOSITORY, f"rustfs-linux-x86_64-gnu-v{previous['tag_name'].lstrip('v')}.zip")}
return validate_inputs(value)
def targets(selection):
if selection == "linux":
return LINUX
if selection == "desktop":
return DESKTOP
if selection == "all":
return TARGETS
selected = tuple(selection.split(","))
require(selected and len(set(selected)) == len(selected) and all(item in TARGETS for item in selected),
"unknown or duplicate build targets")
return selected
def archive_record(path, target, inputs):
validate_inputs(inputs)
require(target in TARGETS, "unknown binary platform")
path = Path(path)
name = f"rustfs-{target}-v{inputs['version']}.zip"
require(path.name == name and path.is_file() and not path.is_symlink(), "archive name does not match the target version")
binary = "rustfs.exe" if target == "windows-x86_64" else "rustfs"
expected = {binary}
if target.endswith("-gnu"):
expected.update(("rustfs.cpu-symbol-catalog.json", "rustfs.cpu-symbol-catalog.sha256"))
with zipfile.ZipFile(path) as archive:
members = archive.infolist()
require(len(members) == len(expected) and {m.filename for m in members} == expected,
"unexpected or duplicate release archive member")
for member in members:
require(not member.is_dir() and 0 < member.file_size <= MAX_MEMBER and (member.external_attr >> 16) & 0o170000 != 0o120000,
"invalid release archive member")
with archive.open(binary) as stream:
binary_sha = stream_digest(stream)
return {"target": target, "name": name, "kind": "zip", "sha256": file_digest(path), "size": path.stat().st_size,
"binary_sha256": binary_sha, "inputs_sha256": digest(inputs)}
def verify_records(inputs, records, directory, required):
validate_inputs(inputs)
require(isinstance(records, list) and len(records) == len(required), "candidate is missing required platform artifacts")
seen = set()
for record in records:
require(isinstance(record, dict) and record.get("target") in required, "unexpected artifact platform")
target = record["target"]
require(target not in seen, "duplicate artifact platform")
seen.add(target)
expected = archive_record(Path(directory) / f"rustfs-{target}-v{inputs['version']}.zip", target, inputs)
require(record == expected, "candidate artifact or input identity changed")
require(seen == set(required), "incomplete candidate platform set")
return sorted(records, key=lambda record: record["target"])
def certification_candidate(inputs, records, directory, required=TARGETS):
"""Candidate identity survives adding desktop files; test receipts bind bytes."""
records = verify_records(inputs, records, directory, required)
return {"schema_version": 1, "product_commit": inputs["source_sha"], "version": inputs["version"],
"release_profile": "prerelease" if "-" in inputs["version"] else "stable", "features": ["pyroscope"],
"plan_sha256": inputs["plan_sha256"], "workflow_sha": inputs["workflow_sha"],
"validation_batch": {"run_id": inputs["candidate_id"], "run_attempt": 1},
"deliverables": [{"kind": "zip", "name": r["name"], "sha256": r["sha256"], "size": r["size"],
"binary_sha256": r["binary_sha256"],
"location": f"https://github.com/{REPOSITORY}/actions/runs/{inputs['origin_run_id']}#artifact-{r['sha256']}"}
for r in records]}
def validate_run(run, workflow, *, completed=True):
require(isinstance(run, dict) and (run.get("repository") or {}).get("full_name") == REPOSITORY,
"build run belongs to another repository")
require(run.get("path") == f".github/workflows/{workflow}" and run.get("event") == "workflow_dispatch"
and run.get("head_branch") == "main", "candidate must originate from the trusted main workflow")
require(type(run.get("id")) is int and run["id"] > 0 and type(run.get("run_attempt")) is int and run["run_attempt"] > 0,
"invalid workflow run identity")
require(isinstance(run.get("head_sha"), str) and HEX40.fullmatch(run["head_sha"]), "invalid workflow source SHA")
if completed:
require(run.get("status") == "completed" and run.get("conclusion") == "success", "candidate run did not succeed")
return run
def validate_artifact(artifact, run, name):
require(artifact.get("name") == name and artifact.get("expired") is False, "missing or expired candidate artifact")
origin = artifact.get("workflow_run") or {}
require(origin.get("id") == run["id"] and origin.get("head_sha") == run["head_sha"], "artifact belongs to another build run")
require(type(artifact.get("id")) is int and artifact["id"] > 0 and type(artifact.get("size_in_bytes")) is int
and 0 < artifact["size_in_bytes"] <= 8 * MAX_MEMBER, "invalid artifact identity or size")
require(isinstance(artifact.get("digest"), str) and re.fullmatch(r"sha256:[0-9a-f]{64}", artifact["digest"]),
"artifact has no immutable digest")
return artifact
def artifact_list(run_id):
require(type(run_id) is int and run_id > 0, "invalid run ID")
result = []
for page in range(1, 101):
batch = github(f"repos/{REPOSITORY}/actions/runs/{run_id}/artifacts?per_page=100&page={page}")["artifacts"]
result.extend(batch)
if len(batch) < 100:
return result
raise ValueError("too many candidate artifacts")
def download_artifact(run, name, destination):
matches = [item for item in artifact_list(run["id"]) if item.get("name") == name]
require(len(matches) == 1, "candidate artifact is absent or ambiguous")
artifact = validate_artifact(matches[0], run, name)
destination = Path(destination)
destination.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory() as temp:
archive_path = Path(temp) / "artifact.zip"
with archive_path.open("wb") as stream, github_stream(f"repos/{REPOSITORY}/actions/artifacts/{artifact['id']}/zip") as response:
received = 0
for chunk in iter(lambda: response.read(1024 * 1024), b""):
received += len(chunk)
require(received <= artifact["size_in_bytes"], "artifact response exceeds its declared size")
stream.write(chunk)
require(archive_path.stat().st_size == artifact["size_in_bytes"] and
"sha256:" + file_digest(archive_path) == artifact["digest"], "downloaded artifact digest or size mismatch")
with zipfile.ZipFile(archive_path) as archive:
members = archive.infolist()
require(0 < len(members) <= 256 and len({m.filename for m in members}) == len(members), "invalid artifact members")
require(sum(member.file_size for member in members) <= 8 * MAX_MEMBER, "expanded artifact exceeds the candidate size limit")
for member in members:
require(re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._-]{0,200}", member.filename)
and not member.is_dir() and 0 < member.file_size <= MAX_MEMBER
and (member.external_attr >> 16) & 0o170000 != 0o120000, "unsafe artifact member")
target = destination / member.filename
require(not target.exists(), "artifact extraction would overwrite a file")
with archive.open(member) as source, target.open("xb") as sink:
import shutil
shutil.copyfileobj(source, sink)
return artifact
def emit_output(name, value):
require("\n" not in str(value) and "\r" not in str(value), "invalid workflow output")
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as stream:
stream.write(f"{name}={value}\n")
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest="command", required=True)
p = sub.add_parser("freeze")
for name in ("version", "source-sha", "workflow-sha", "toolchain", "console", "cli", "previous", "output"):
p.add_argument("--" + name, required=True)
p.add_argument("--run-id", type=int, required=True)
p.add_argument("--run-number", type=int, required=True)
p.add_argument("--ci-run-id", type=int, required=True)
p.add_argument("--ci-run-attempt", type=int, required=True)
p = sub.add_parser("validate-inputs")
p.add_argument("--inputs", required=True)
p.add_argument("--check-checkout", action="store_true")
p = sub.add_parser("record")
p.add_argument("--inputs", required=True)
p.add_argument("--target", required=True)
p.add_argument("--archive", required=True)
p.add_argument("--output", required=True)
p = sub.add_parser("assemble")
p.add_argument("--inputs", required=True)
p.add_argument("--directory", required=True)
p.add_argument("--output", required=True)
p.add_argument("--platforms", choices=("linux", "all"), default="all")
p = sub.add_parser("download")
p.add_argument("--run-id", type=int, required=True)
p.add_argument("--workflow", choices=("release-candidate.yml", "release-complete.yml"), required=True)
p.add_argument("--name", required=True)
p.add_argument("--destination", required=True)
args = parser.parse_args(argv)
if args.command == "freeze":
write_json(args.output, freeze(args.version, args.source_sha, args.workflow_sha, args.toolchain,
read_json(args.console), args.run_id, args.run_number, read_json(args.cli), read_json(args.previous),
{"run_id": args.ci_run_id, "run_attempt": args.ci_run_attempt}))
elif args.command == "validate-inputs":
inputs = validate_inputs(read_json(args.inputs))
if args.check_checkout:
head = command(["git", "rev-parse", "HEAD"], cwd=ROOT)
require(head == inputs["source_sha"], "checkout differs from the frozen source")
require(recipe_digest(head) == inputs["recipe_sha256"], "release recipe differs from frozen inputs")
require(file_digest(ROOT / "Cargo.lock") == inputs["lock_sha256"], "dependency lock differs from frozen inputs")
print(digest(inputs))
elif args.command == "record":
write_json(args.output, archive_record(args.archive, args.target, read_json(args.inputs)))
elif args.command == "assemble":
inputs = validate_inputs(read_json(args.inputs))
directory = Path(args.directory)
records = [read_json(path) for path in sorted(directory.glob("record-*.json"))]
write_json(args.output, certification_candidate(inputs, records, directory, targets(args.platforms)))
elif args.command == "download":
run = validate_run(github(f"repos/{REPOSITORY}/actions/runs/{args.run_id}"), args.workflow)
download_artifact(run, args.name, args.destination)
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except (ValueError, OSError, KeyError, TypeError, zipfile.BadZipFile, subprocess.CalledProcessError) as error:
print(f"candidate rejected: {error}", file=sys.stderr)
raise SystemExit(1) from None
+200
View File
@@ -0,0 +1,200 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
"""Wrap accepted binaries, then inspect the actual packages and OCI images."""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import shutil
import subprocess
import tarfile
import tempfile
import urllib.request
import time
import candidate as c
import pipeline
def run(args, **kwargs):
return subprocess.run(args, check=True, timeout=1800, **kwargs)
def binary(inputs, target, output):
origin = pipeline.get_run(inputs["origin_run_id"], "release-candidate.yml")
c.download_artifact(origin, f"candidate-{inputs['candidate_id']}-{target}", output)
record = c.read_json(output / f"record-{target}.json")
c.verify_records(inputs, [record], output, (target,))
pipeline.verify_evidence(inputs, output, {origin["id"]: origin})
import verify
path, _ = verify.unpack(output / record["name"], target, inputs, output / "bin")
return path, record
def reuse(kind, target):
inputs = pipeline.manifest_env()
current = pipeline.current_run("release-complete.yml")
name = f"{kind}-{target}"
exists = any(a["name"] == name for a in c.artifact_list(current["id"]))
if exists:
with tempfile.TemporaryDirectory() as temp:
c.download_artifact(current, name, temp)
record = c.read_json(Path(temp) / f"{name}.json")
c.require(record.get("inputs_sha256") == c.digest(inputs), "existing delivery belongs to another candidate")
# Successful jobs are immutable. The final seal independently checks
# all sizes and digests before they can be approved for publication.
c.emit_output("reused", str(exists).lower())
def package(inputs, target, output):
output.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory() as temp:
temp = Path(temp)
executable, record = binary(inputs, target, temp / "source")
arch, libc = target.split("-")[1:]
build_type = "prerelease" if "-" in inputs["version"] else "release"
values = c.command(["bash", "scripts/release/package_versions.sh", build_type, inputs["version"], "", arch, libc], cwd=c.ROOT)
versions = dict(line.split("=", 1) for line in values.splitlines())
# Existing package recipes operate in a private working directory.
for name in ("scripts", "deploy"):
(temp / name).symlink_to(c.ROOT / name, target_is_directory=True)
for name in ("LICENSE", "README.md"):
shutil.copyfile(c.ROOT / name, temp / name)
(temp / "bin").mkdir()
shutil.copyfile(executable, temp / "bin/rustfs")
(temp / "bin/rustfs").chmod(0o755)
env = dict(os.environ, **{key.upper(): value for key, value in versions.items()}, LIBC=libc,
DEB_ARCH="amd64" if arch == "x86_64" else "arm64", RPM_ARCH=arch,
DEB_DEPENDS="libc6 (>= 2.31)" if libc == "gnu" else "",
RPM_DEPENDS="glibc >= 2.31" if libc == "gnu" else "", GITHUB_OUTPUT=str(temp / "outputs"))
for kind in ("deb", "rpm"):
run(["bash", str(c.ROOT / f"scripts/release/build_{kind}.sh")], cwd=temp, env=env)
path = temp / versions[f"{kind}_file"]
unpacked = temp / f"unpacked-{kind}"
unpacked.mkdir()
if kind == "deb":
run(["dpkg-deb", "--extract", str(path), str(unpacked)])
else:
with subprocess.Popen(["rpm2cpio", str(path)], stdout=subprocess.PIPE) as source:
run(["cpio", "--extract", "--make-directories", "--quiet", "./usr/bin/rustfs"], stdin=source.stdout, cwd=unpacked)
source.stdout.close()
c.require(source.wait(timeout=60) == 0, "RPM extraction failed")
c.require(c.file_digest(unpacked / "usr/bin/rustfs") == record["binary_sha256"], "package binary differs from accepted ZIP")
shutil.copyfile(path, output / path.name)
files = pipeline.file_records(output)
c.write_json(output / f"packages-{target}.json", {"inputs_sha256": c.digest(inputs), "binary_sha256": record["binary_sha256"], "files": files})
def image_prepare(inputs, variant, output):
output.mkdir(parents=True, exist_ok=True)
binaries = {}
for arch, docker_arch in (("x86_64", "amd64"), ("aarch64", "arm64")):
target = f"linux-{arch}-{variant}"
with tempfile.TemporaryDirectory() as temp:
temp = Path(temp)
executable, record = binary(inputs, target, temp / "source")
directory = c.ROOT / ".release-inputs" / docker_arch
directory.mkdir(parents=True, exist_ok=True)
shutil.copyfile(executable, directory / "rustfs")
(directory / "rustfs").chmod(0o755)
cli = inputs["cli"][target]
archive = temp / "cli.tar.gz"
run(["curl", "--fail", "--location", "--retry", "3", "--output", str(archive), cli["url"]])
c.require(c.file_digest(archive) == cli["sha256"], "CLI dependency changed")
with tarfile.open(archive, "r:gz") as package:
members = [m for m in package.getmembers() if m.name == "rc"]
c.require(len(members) == 1 and members[0].isfile() and 0 < members[0].size <= c.MAX_MEMBER, "invalid CLI archive")
with package.extractfile(members[0]) as source, (directory / "rc").open("wb") as dest:
shutil.copyfileobj(source, dest)
(directory / "rc").chmod(0o755)
binaries[arch] = record["binary_sha256"]
c.write_json(output / f"image-{variant}-inputs.json", {"inputs_sha256": c.digest(inputs), "binaries": binaries})
def image_smoke(inputs, variant, output):
archive = output / f"image-{variant}.tar"
record = c.read_json(output / f"image-{variant}-inputs.json")
scan_directory = Path(str(output) + "-scan")
scan_directory.mkdir(parents=True, exist_ok=True)
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
for arch, docker_arch in (("x86_64", "amd64"), ("aarch64", "arm64")):
image = f"rustfs-acceptance:{variant}-{docker_arch}"
run(["skopeo", "copy", "--override-os", "linux", "--override-arch", docker_arch, f"oci-archive:{archive}", f"docker-daemon:{image}"])
container = c.command(["docker", "create", "--platform", f"linux/{docker_arch}", image])
try:
with tempfile.TemporaryDirectory() as temp:
path = Path(temp) / "rustfs"
run(["docker", "cp", f"{container}:/usr/bin/rustfs", str(path)])
c.require(c.file_digest(path) == record["binaries"][arch], "image wraps a different executable")
finally:
run(["docker", "rm", "-v", container])
version = c.command(["docker", "run", "--rm", "--platform", f"linux/{docker_arch}", "--entrypoint", "/usr/bin/rustfs", image, "--version"])
c.require(version.splitlines()[0] == f"rustfs {inputs['version']}", "container displays wrong version")
container = c.command(["docker", "run", "-d", "--platform", f"linux/{docker_arch}", "-p", "127.0.0.1::9001",
"-e", "RUSTFS_ACCESS_KEY=release-smoke", "-e", "RUSTFS_SECRET_KEY=release-smoke-test-secret",
"-e", "RUSTFS_CONSOLE_ENABLE=true", image])
log = output / f"image-{variant}-{docker_arch}.log"
try:
address = c.command(["docker", "port", container, "9001/tcp"]).strip()
deadline = time.monotonic() + 60
while time.monotonic() < deadline:
try:
with opener.open(f"http://{address}/rustfs/console/", timeout=2) as response:
c.require(response.status == 200 and "text/html" in response.headers.get("Content-Type", ""), "image Console did not return HTML")
break
except OSError:
time.sleep(0.5)
else:
raise ValueError("container startup timed out")
finally:
with log.open("w") as stream:
subprocess.run(["docker", "logs", container], stdout=stream, stderr=subprocess.STDOUT, check=True, timeout=60)
run(["docker", "rm", "-f", "-v", container])
with log.open("a") as stream:
stream.write("\nConsole: HTTP 200 text/html; wrapped executable SHA-256 and version verified\n")
# Trivy's local archive reader does not reliably select a platform from
# a multi-architecture OCI index. Export the image just verified above.
scan_archive = scan_directory / f"image-{variant}-{docker_arch}.tar"
run(["docker", "save", "--output", str(scan_archive), image])
record.setdefault("scan_inputs", {})[docker_arch] = c.file_digest(scan_archive)
raw = subprocess.check_output(["skopeo", "inspect", "--raw", f"oci-archive:{archive}"], timeout=60)
import hashlib
record.update(digest="sha256:" + hashlib.sha256(raw).hexdigest(), sha256=c.file_digest(archive), size=archive.stat().st_size, smoke="pass")
c.write_json(output / f"image-{variant}-smoke.json", record)
def image_record(inputs, variant, output):
record = c.read_json(output / f"image-{variant}-smoke.json")
c.require(record["inputs_sha256"] == c.digest(inputs), "image inputs changed")
for arch in ("amd64", "arm64"):
scan_archive = Path(str(output) + "-scan") / f"image-{variant}-{arch}.tar"
c.require(record.get("scan_inputs", {}).get(arch) == c.file_digest(scan_archive), "scan input changed after image verification")
report = c.read_json(output / f"scan-{variant}-{arch}.sarif")
c.require(report.get("version") == "2.1.0" and report.get("runs"), "container scan did not produce a SARIF report")
record["scan"] = "completed"
c.write_json(output / f"image-{variant}.json", record)
def main():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("mode", choices=("reuse", "package", "image-prepare", "image-smoke", "image-record"))
p.add_argument("--kind", choices=("packages", "image"))
p.add_argument("--target", required=True)
p.add_argument("--output", type=Path, default=Path("delivery"))
args = p.parse_args()
inputs = pipeline.manifest_env()
if args.mode == "reuse":
reuse(args.kind, args.target)
elif args.mode == "package":
c.require(args.target in c.LINUX, "unknown package platform")
package(inputs, args.target, args.output)
else:
c.require(args.target in ("musl", "gnu"), "unknown image variant")
{"image-prepare": image_prepare, "image-smoke": image_smoke, "image-record": image_record}[args.mode](inputs, args.target, args.output)
if __name__ == "__main__":
main()
+361
View File
@@ -0,0 +1,361 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
"""Candidate orchestration. GitHub run identity is checked before trusting bytes."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import subprocess
import tempfile
import urllib.error
import zipfile
import candidate as c
import release_gate
def current_run(workflow):
c.require(os.environ.get("GITHUB_REPOSITORY") == c.REPOSITORY and os.environ.get("GITHUB_REF") == "refs/heads/main",
"release orchestration must run on the upstream main branch")
run = c.validate_run(c.github(f"repos/{c.REPOSITORY}/actions/runs/{int(os.environ['GITHUB_RUN_ID'])}"), workflow, completed=False)
c.require(run["head_sha"] == os.environ["GITHUB_SHA"], "workflow source changed")
return run
def get_run(run_id, workflow):
run = c.github(f"repos/{c.REPOSITORY}/actions/runs/{int(run_id)}")
active = str(run_id) == os.environ.get("GITHUB_RUN_ID")
return c.validate_run(run, workflow, completed=not active)
def manifest_env():
return c.validate_inputs(json.loads(os.environ["CANDIDATE_MANIFEST"]))
def output_inputs(inputs):
c.emit_output("manifest", json.dumps(inputs, separators=(",", ":")))
for field in ("candidate_id", "source_sha", "workflow_sha", "version"):
c.emit_output(field, inputs[field])
def source_ci(source, version):
runs = c.github(f"repos/{c.REPOSITORY}/actions/workflows/ci.yml/runs?head_sha={source}&status=success&per_page=10")["workflow_runs"]
required = {"Quick Checks", "Test and Lint", "End-to-End Tests (full merge gate)"}
for run in runs:
if (run.get("head_sha") != source or run.get("path") != ".github/workflows/ci.yml"
or run.get("head_branch") not in ("main", f"release/{version}")
or run.get("event") not in ("push", "workflow_dispatch")
or run.get("conclusion") != "success" or run.get("status") != "completed"
or (run.get("repository") or {}).get("full_name") != c.REPOSITORY):
continue
jobs = []
for page in range(1, 11):
batch = c.github(f"repos/{c.REPOSITORY}/actions/runs/{run['id']}/attempts/{run['run_attempt']}/jobs?per_page=100&page={page}")["jobs"]
jobs.extend(batch)
if len(batch) < 100:
break
selected = {name: [job for job in jobs if job.get("name") == name] for name in required}
if all(len(found) == 1 and found[0].get("status") == "completed" and found[0].get("conclusion") == "success" for found in selected.values()):
return {"run_id": run["id"], "run_attempt": run["run_attempt"]}
raise ValueError("no complete exact-source CI found; run ci.yml on main or the release branch before creating a candidate")
def unused_version(version):
# Reject a spent version before allocating native builders. A partial
# publication resumes its existing seal instead of making a new candidate.
for tag in (version, "v" + version):
for endpoint in ("git/ref/tags", "releases/tags"):
try:
c.github(f"repos/{c.REPOSITORY}/{endpoint}/{tag}")
except urllib.error.HTTPError as error:
if error.code == 404:
continue
raise
raise ValueError("version already has a tag or release; resume its approved seal or choose a new version")
def prepare(version, source, directory):
run = current_run("release-candidate.yml")
c.require(c.HEX40.fullmatch(source) and c.VERSION.fullmatch(version), "use a full source commit and exact version")
directory.mkdir(parents=True, exist_ok=True)
name = "release-inputs"
exists = any(a["name"] == name for a in c.artifact_list(run["id"]))
if exists:
c.download_artifact(run, name, directory)
inputs = c.validate_inputs(c.read_json(directory / "inputs.json"))
c.require(inputs["source_sha"] == source and inputs["version"] == version and inputs["workflow_sha"] == run["head_sha"], "retry changed candidate inputs")
else:
unused_version(version)
c.command(["git", "fetch", "--no-tags", "origin", source], cwd=c.ROOT)
ci = source_ci(source, version)
c.command(["rustup", "update", "stable", "--no-self-update"])
toolchain = c.command(["rustc", "+stable", "--version"]).split()[1]
inputs = c.freeze(version, source, run["head_sha"], toolchain, c.github("repos/rustfs/console/releases/latest"),
run["id"], int(os.environ["GITHUB_RUN_NUMBER"]), c.github("repos/rustfs/cli/releases/latest"),
c.github("repos/rustfs/rustfs/releases/latest"), ci)
c.write_json(directory / "inputs.json", inputs)
output_inputs(inputs)
c.emit_output("reused", str(exists).lower())
def reuse_platform(target):
inputs = manifest_env()
workflow = os.environ["GITHUB_WORKFLOW_REF"].split("@", 1)[0].rsplit("/", 1)[-1]
run = current_run(workflow)
name = f"candidate-{inputs['candidate_id']}-{target}"
exists = any(a["name"] == name for a in c.artifact_list(run["id"]))
if exists:
with tempfile.TemporaryDirectory() as directory:
c.download_artifact(run, name, directory)
records = [c.read_json(Path(directory) / f"record-{target}.json")]
c.verify_records(inputs, records, directory, (target,))
verify_evidence(inputs, Path(directory), {run["id"]: run})
c.emit_output("reused", str(exists).lower())
def load_linux(run_id, expected, directory):
run = get_run(run_id, "release-candidate.yml")
c.download_artifact(run, "release-linux", directory)
inputs = c.validate_inputs(c.read_json(directory / "inputs.json"))
c.require(inputs["origin_run_id"] == run["id"] and inputs["workflow_sha"] == run["head_sha"], "candidate inputs belong to another run")
c.require(c.HEX64.fullmatch(expected or "") and c.file_digest(directory / "linux-manifest.json") == expected,
"Linux approval digest does not match the downloaded manifest")
verify_seal_files(c.read_json(directory / "linux-manifest.json"), directory)
check_recipe(inputs)
return inputs, run
def check_recipe(inputs):
# Unrelated main changes do not invalidate a frozen candidate. A changed
# producer/verifier/publisher does, even when the product source is unchanged.
c.require(c.recipe_digest(os.environ["GITHUB_SHA"]) == inputs["recipe_sha256"], "release recipe changed; create a new candidate")
def evidence_files(directory):
return sorted(list(directory.glob("identity-*.json")) + list(directory.glob("upgrade-*.json")))
def verify_evidence(inputs, directory, runs):
for path in evidence_files(directory):
value = c.read_json(path)
workflow = value.get("workflow", {})
run = runs.get(int(workflow.get("run_id", 0)))
c.require(run is not None and workflow.get("repository") == c.REPOSITORY
and workflow.get("workflow_id") == run["path"].rsplit("/", 1)[-1]
and workflow.get("workflow_sha") == run["head_sha"]
and type(workflow.get("run_attempt")) is int and 0 < workflow["run_attempt"] <= run["run_attempt"],
"receipt was not produced by the downloaded trusted run")
c.require(value.get("harness_commit") == inputs["workflow_sha"], "receipt harness changed")
for item in value.get("evidence", []):
name = item.get("path", "")
c.require(re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._-]{0,200}", name), "unsafe evidence path")
log = directory / name
c.require(log.is_file() and not log.is_symlink() and log.stat().st_size == item.get("size")
and c.file_digest(log) == item.get("sha256"), "receipt evidence was removed or substituted")
def certify(inputs, directory, runs, scope):
records = [c.read_json(directory / f"record-{target}.json") for target in c.targets("linux" if scope == "linux" else "all")]
candidate = c.certification_candidate(inputs, records, directory, c.targets("linux" if scope == "linux" else "all"))
plan = c.read_json(c.ROOT / ".config/release-validation-plan.json")
c.require(c.digest(plan) == inputs["plan_sha256"], "validation plan changed")
verify_evidence(inputs, directory, runs)
with tempfile.TemporaryDirectory() as temp:
temp = Path(temp)
results = temp / "results"
results.mkdir()
for path in evidence_files(directory):
shutil.copyfile(path, results / path.name)
c.write_json(temp / "candidate.json", candidate)
c.write_json(temp / "plan.json", plan)
status = release_gate.run_validate(temp / "plan.json", temp / "candidate.json", results, temp / "gate", scope)
summary = c.read_json(temp / "gate/release-gate-summary.json")
c.require(status == 0, "certification failed: " + json.dumps(summary["findings"]))
return candidate, summary
def file_records(directory, excluded=()):
result = []
for path in sorted(directory.iterdir()):
if path.name in excluded:
continue
c.require(path.is_file() and not path.is_symlink() and re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._-]{0,200}", path.name), "unexpected staged file")
result.append({"name": path.name, "sha256": c.file_digest(path), "size": path.stat().st_size})
return result
def verify_seal_files(seal, directory):
c.require(isinstance(seal, dict) and seal.get("schema") == 1 and seal.get("stage") in ("linux", "complete"), "invalid release seal")
filename = "linux-manifest.json" if seal["stage"] == "linux" else "release-manifest.json"
c.require(seal.get("files") == file_records(directory, (filename,)), "sealed files changed, are missing, or contain extras")
def aliases(inputs):
return {f"rustfs-{target}-latest.zip": f"rustfs-{target}-v{inputs['version']}.zip" for target in c.TARGETS}
def linux_seal(directory):
run = current_run("release-candidate.yml")
if any(a["name"] == "release-linux" for a in c.artifact_list(run["id"])):
c.download_artifact(run, "release-linux", directory)
inputs = c.validate_inputs(c.read_json(directory / "inputs.json"))
verify_seal_files(c.read_json(directory / "linux-manifest.json"), directory)
certify(inputs, directory, {run["id"]: run}, "linux")
c.emit_output("reused", "true")
c.emit_output("manifest_sha256", c.file_digest(directory / "linux-manifest.json"))
return
c.download_artifact(run, "release-inputs", directory)
inputs = c.validate_inputs(c.read_json(directory / "inputs.json"))
for target in c.LINUX:
c.download_artifact(run, f"candidate-{inputs['candidate_id']}-{target}", directory)
c.download_artifact(run, "release-upgrade", directory)
_, summary = certify(inputs, directory, {run["id"]: run}, "linux")
c.require(not summary["publish_allowed"], "partial certification must not grant publication authority")
c.write_json(directory / "linux-manifest.json", {"schema": 1, "stage": "linux", "inputs_sha256": c.digest(inputs),
"files": file_records(directory), "summary": summary})
c.emit_output("manifest_sha256", c.file_digest(directory / "linux-manifest.json"))
def check_acceptance(url):
c.require(re.fullmatch(r"https://github\.com/rustfs/backlog/issues/[1-9][0-9]*(?:#issuecomment-[1-9][0-9]*)?", url),
"acceptance must link to a RustFS backlog report")
def complete_prepare(run_id, expected, report, directory):
current_run("release-complete.yml")
check_acceptance(report)
inputs, run = load_linux(run_id, expected, directory)
certify(inputs, directory, {run["id"]: run}, "linux")
output_inputs(inputs)
def complete_seal(run_id, expected, report, directory):
current = current_run("release-complete.yml")
if any(a["name"] == "release-seal" for a in c.artifact_list(current["id"])):
c.download_artifact(current, "release-seal", directory)
inputs = c.validate_inputs(c.read_json(directory / "inputs.json"))
seal = c.read_json(directory / "release-manifest.json")
verify_seal_files(seal, directory)
c.require(inputs["origin_run_id"] == run_id and seal["acceptance"] == {"linux_manifest_sha256": expected, "report": report}, "completion retry changed acceptance")
origin = get_run(run_id, "release-candidate.yml")
certify(inputs, directory, {origin["id"]: origin, current["id"]: current}, "full")
verify_deliveries(inputs, directory)
c.emit_output("reused", "true")
c.emit_output("manifest_sha256", c.file_digest(directory / "release-manifest.json"))
return
inputs, origin = load_linux(run_id, expected, directory)
check_acceptance(report)
(directory / "linux-manifest.json").unlink()
for target in c.DESKTOP:
c.download_artifact(current, f"candidate-{inputs['candidate_id']}-{target}", directory)
for target in c.LINUX:
c.download_artifact(current, f"packages-{target}", directory)
for variant in ("musl", "gnu"):
c.download_artifact(current, f"image-{variant}", directory)
c.download_artifact(current, "helm-package", directory)
candidate, summary = certify(inputs, directory, {origin["id"]: origin, current["id"]: current}, "full")
verify_deliveries(inputs, directory)
# Persist the small proof bundle with the public release; Actions retention
# must not be the only way to audit an already published version.
evidence = directory / "release-evidence.zip"
with zipfile.ZipFile(evidence, "x", compression=zipfile.ZIP_DEFLATED) as archive:
for path in sorted(directory.iterdir()):
if path.suffix in (".json", ".log", ".sarif"):
archive.write(path, path.name)
archive.writestr("certification.json", json.dumps(summary, indent=2, sort_keys=True))
archive.writestr("validation-plan.json", (c.ROOT / ".config/release-validation-plan.json").read_bytes())
with tempfile.TemporaryDirectory() as temp:
lock = Path(temp) / "Cargo.lock"
lock.write_bytes(c.git_file(inputs["source_sha"], "Cargo.lock"))
c.require(c.file_digest(lock) == inputs["lock_sha256"], "SBOM dependency lock changed")
subprocess.run(["python3", str(c.ROOT / "scripts/security/generate_release_supply_chain_assets.py"),
"--asset-dir", str(directory), "--version", inputs["version"], "--tag", inputs["version"],
"--build-type", candidate["release_profile"], "--repository", c.REPOSITORY,
"--ref", "refs/heads/main", "--sha", inputs["source_sha"], "--run-id", str(current["id"]),
"--run-attempt", str(current["run_attempt"]), "--cargo-lock", str(lock),
"--workflow", "release-complete.yml"], check=True, timeout=60)
provenance = directory / f"rustfs-{inputs['version']}.provenance.json"
value = c.read_json(provenance)
value["predicate"]["buildDefinition"]["externalParameters"]["candidateInputs"] = inputs
provenance.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n")
assets = {p.name: p for p in directory.iterdir() if p.suffix in (".zip", ".deb", ".rpm", ".tgz") or p.name.endswith((".sbom.cdx.json", ".provenance.json"))}
assets.update({name: directory / source for name, source in aliases(inputs).items()})
for algorithm in ("sha256", "sha512"):
checksums = []
for name, path in sorted(assets.items()):
with path.open("rb") as stream:
checksum = hashlib.file_digest(stream, algorithm).hexdigest()
checksums.append(f"{checksum} {name}")
(directory / f"{algorithm.upper()}SUMS").write_text("\n".join(checksums) + "\n")
c.write_json(directory / "release-manifest.json", {
"schema": 1, "stage": "complete", "inputs_sha256": c.digest(inputs), "candidate": candidate,
"summary": summary, "files": file_records(directory), "complete_run_id": current["id"],
"aliases": aliases(inputs),
"acceptance": {"linux_manifest_sha256": expected, "report": report},
})
c.emit_output("manifest_sha256", c.file_digest(directory / "release-manifest.json"))
def verify_deliveries(inputs, directory):
chart = c.read_json(directory / "chart.json")
c.require(chart.get("version") == inputs["version"] and chart.get("inputs_sha256") == c.digest(inputs)
and chart.get("name") == f"rustfs-{inputs['version']}.tgz", "chart identity changed")
path = directory / chart["name"]
c.require(path.stat().st_size == chart["size"] and c.file_digest(path) == chart["sha256"], "chart bytes changed")
for target in c.LINUX:
binary = c.read_json(directory / f"record-{target}.json")["binary_sha256"]
record = c.read_json(directory / f"packages-{target}.json")
c.require(record.get("inputs_sha256") == c.digest(inputs) and record.get("binary_sha256") == binary, "package wraps a different candidate")
c.require(len(record.get("files", [])) == 2 and {Path(f["name"]).suffix for f in record["files"]} == {".deb", ".rpm"}, "missing package format")
for item in record["files"]:
path = directory / item["name"]
c.require(path.name == item["name"] and path.is_file() and path.stat().st_size == item["size"] and c.file_digest(path) == item["sha256"], "package changed")
for variant in ("musl", "gnu"):
record = c.read_json(directory / f"image-{variant}.json")
c.require(record.get("inputs_sha256") == c.digest(inputs), "image inputs changed")
expected = {arch: c.read_json(directory / f"record-linux-{arch}-{variant}.json")["binary_sha256"] for arch in ("x86_64", "aarch64")}
c.require(record.get("binaries") == expected, "image contains an unverified binary")
path = directory / f"image-{variant}.tar"
c.require(c.file_digest(path) == record.get("sha256") and path.stat().st_size == record.get("size")
and re.fullmatch(r"sha256:[0-9a-f]{64}", record.get("digest", "")), "image archive changed")
c.require(record.get("smoke") == "pass" and record.get("scan") == "completed", "image verification did not finish")
def main():
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest="mode", required=True)
p = sub.add_parser("prepare")
p.add_argument("--version", required=True)
p.add_argument("--source", required=True)
p.add_argument("--directory", type=Path, required=True)
p = sub.add_parser("reuse-platform")
p.add_argument("--target", choices=c.TARGETS, required=True)
p = sub.add_parser("linux-seal")
p.add_argument("--directory", type=Path, required=True)
for name in ("complete-prepare", "complete-seal"):
p = sub.add_parser(name)
p.add_argument("--run-id", type=int, required=True)
p.add_argument("--digest", required=True)
p.add_argument("--report", required=True)
p.add_argument("--directory", type=Path, required=True)
args = parser.parse_args()
if args.mode == "prepare":
prepare(args.version, args.source, args.directory)
elif args.mode == "reuse-platform":
reuse_platform(args.target)
elif args.mode == "linux-seal":
linux_seal(args.directory)
elif args.mode == "complete-prepare":
complete_prepare(args.run_id, args.digest, args.report, args.directory)
else:
complete_seal(args.run_id, args.digest, args.report, args.directory)
if __name__ == "__main__":
main()
+280
View File
@@ -0,0 +1,280 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
"""Promote an approved seal. This module never compiles or packages a deliverable."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import subprocess
import tempfile
import candidate as c
import pipeline
REGISTRIES = ("docker.io/rustfs/rustfs", "ghcr.io/rustfs/rustfs", "quay.io/rustfs/rustfs")
def api(path, method="GET", body=None, missing=False):
args = ["gh", "api", "--method", method, path]
if body is not None:
args += ["--input", "-"]
result = subprocess.run(args, input=json.dumps(body) if body is not None else None, text=True,
capture_output=True, timeout=120)
if result.returncode:
if missing and "(HTTP 404)" in result.stderr:
return None
raise ValueError(f"GitHub {method} failed for {path}: {result.stderr.strip()}")
return json.loads(result.stdout) if result.stdout.strip() else None
def verify(run_id, expected, directory):
current = pipeline.current_run("release-publish.yml")
complete = pipeline.get_run(run_id, "release-complete.yml")
c.download_artifact(complete, "release-seal", directory)
manifest_path = directory / "release-manifest.json"
c.require(c.HEX64.fullmatch(expected or "") and c.file_digest(manifest_path) == expected, "publication approval does not match the sealed manifest")
seal = c.read_json(manifest_path)
pipeline.verify_seal_files(seal, directory)
c.require(seal["stage"] == "complete" and seal.get("complete_run_id") == complete["id"], "seal belongs to another completion run")
inputs = c.validate_inputs(c.read_json(directory / "inputs.json"))
c.require(seal.get("inputs_sha256") == c.digest(inputs), "seal inputs changed")
c.require(seal.get("aliases") == pipeline.aliases(inputs), "release aliases differ from the approved versioned archives")
pipeline.check_recipe(inputs)
origin = pipeline.get_run(inputs["origin_run_id"], "release-candidate.yml")
c.require(origin["head_sha"] == inputs["workflow_sha"], "candidate producer changed")
candidate, summary = pipeline.certify(inputs, directory, {origin["id"]: origin, complete["id"]: complete}, "full")
c.require(candidate == seal.get("candidate") and summary == seal.get("summary") and summary["publish_allowed"], "full certification does not match the seal")
pipeline.verify_deliveries(inputs, directory)
pipeline.check_acceptance(seal["acceptance"]["report"])
c.require(c.HEX64.fullmatch(seal["acceptance"].get("linux_manifest_sha256", "")), "missing Linux acceptance digest")
c.emit_output("version", inputs["version"])
c.emit_output("source_sha", inputs["source_sha"])
return inputs, seal, current
def ensure_tag(inputs, manifest_sha):
version, source = inputs["version"], inputs["source_sha"]
message = f"RustFS {version}\nRelease manifest SHA-256: {manifest_sha}\n"
path = f"repos/{c.REPOSITORY}/git/ref/tags/{version}"
reference = api(path, missing=True)
if reference is None:
tag = api(f"repos/{c.REPOSITORY}/git/tags", "POST", {"tag": version, "message": message, "object": source, "type": "commit"})
api(f"repos/{c.REPOSITORY}/git/refs", "POST", {"ref": f"refs/tags/{version}", "sha": tag["sha"]})
reference = api(path)
c.require(reference["object"]["type"] == "tag", "existing release tag is not an annotated candidate tag")
tag = api(f"repos/{c.REPOSITORY}/git/tags/{reference['object']['sha']}")
c.require(tag.get("tag") == version and tag.get("object", {}).get("sha") == source and tag.get("object", {}).get("type") == "commit"
and tag.get("message", "").rstrip() == message.rstrip(), "public tag already identifies different source or bytes; never move it")
def public_files(directory):
names = {"release-manifest.json", "SHA256SUMS", "SHA512SUMS", "release-evidence.zip"}
return sorted(p for p in directory.iterdir() if p.name in names or p.suffix in (".zip", ".deb", ".rpm", ".tgz") or p.name.endswith((".sbom.cdx.json", ".provenance.json")))
def release(inputs, manifest_sha):
endpoint = f"repos/{c.REPOSITORY}/releases"
value = api(endpoint + "/tags/" + inputs["version"], missing=True)
marker = f"Release manifest SHA-256: {manifest_sha}"
if value is None:
notes = api(endpoint + "/generate-notes", "POST", {"tag_name": inputs["version"], "target_commitish": inputs["source_sha"],
"previous_tag_name": inputs["previous"]["version"]})
value = api(endpoint, "POST", {"tag_name": inputs["version"], "target_commitish": inputs["source_sha"],
"name": f"RustFS {inputs['version']}", "body": marker + "\n\n" + notes["body"],
"draft": True, "prerelease": "-" in inputs["version"], "make_latest": "false"})
c.require(marker in value.get("body", ""), "existing release belongs to another sealed candidate")
return value
def github_assets(inputs, value, paths):
assets = []
for page in range(1, 101):
batch = api(f"repos/{c.REPOSITORY}/releases/{value['id']}/assets?per_page=100&page={page}")
assets.extend(batch)
if len(batch) < 100:
break
known = {a["name"]: a for a in assets}
c.require(len(known) == len(assets) and set(known) <= {p.name for p in paths}, "release has unsealed or duplicate assets")
for path in paths:
old = known.get(path.name)
if old is None:
# No --clobber: a race or interrupted upload must fail closed.
subprocess.run(["gh", "release", "upload", inputs["version"], str(path), "--repo", c.REPOSITORY], check=True, timeout=600)
else:
c.require(old.get("digest") == "sha256:" + c.file_digest(path) and old.get("size") == path.stat().st_size,
"existing release asset differs; never overwrite accepted bytes")
actual = api(f"repos/{c.REPOSITORY}/releases/{value['id']}/assets?per_page=100")
c.require(len(actual) == len(paths), "release asset set incomplete")
for path in paths:
match = [a for a in actual if a["name"] == path.name]
c.require(len(match) == 1 and match[0].get("digest") == "sha256:" + c.file_digest(path)
and match[0].get("size") == path.stat().st_size, "uploaded release asset verification failed")
def aws(*args, capture=True):
return subprocess.run(["aws", "--endpoint-url", os.environ["R2_ENDPOINT"], *args],
text=True, capture_output=capture, check=False, timeout=600)
def r2_file(path, key, *, mutable=False):
bucket = os.environ["R2_BUCKET"]
expected = c.file_digest(path)
head = aws("s3api", "head-object", "--bucket", bucket, "--key", key)
if head.returncode and "(404)" not in head.stderr:
raise ValueError("R2 lookup failed; refusing to treat an unavailable destination as empty")
if head.returncode == 0:
with tempfile.TemporaryDirectory() as temp:
old = Path(temp) / "object"
result = aws("s3", "cp", f"s3://{bucket}/{key}", str(old), "--only-show-errors")
c.require(result.returncode == 0, "could not verify existing R2 object")
if c.file_digest(old) == expected:
return
c.require(mutable, "existing versioned R2 object differs; never overwrite it")
args = ["s3api", "put-object", "--bucket", bucket, "--key", key, "--body", str(path), "--metadata", json.dumps({"sha256": expected})]
if not mutable:
args += ["--if-none-match", "*"]
result = aws(*args)
c.require(result.returncode == 0, "R2 upload failed; retry the same seal after checking destination state")
with tempfile.TemporaryDirectory() as temp:
actual = Path(temp) / "object"
result = aws("s3", "cp", f"s3://{bucket}/{key}", str(actual), "--only-show-errors")
c.require(result.returncode == 0 and c.file_digest(actual) == expected, "R2 read-back digest mismatch")
def image_digest(reference, *, missing=False):
result = subprocess.run(["skopeo", "inspect", "--authfile", os.environ["REGISTRY_AUTH_FILE"], "--raw", f"docker://{reference}"],
capture_output=True, timeout=120)
if result.returncode:
error = result.stderr.decode(errors="replace").lower()
if missing and ("manifest unknown" in error or "manifest_unknown" in error or "name unknown" in error):
return None
raise ValueError(f"cannot verify registry manifest for {reference}")
return "sha256:" + hashlib.sha256(result.stdout).hexdigest()
def copy_image(archive, reference, expected, *, mutable=False):
old = image_digest(reference, missing=True)
if old == expected:
return
c.require(old is None or mutable, "existing image tag identifies different bytes; never replace it")
subprocess.run(["skopeo", "copy", "--authfile", os.environ["REGISTRY_AUTH_FILE"], "--all", "--preserve-digests",
f"oci-archive:{archive}", f"docker://{reference}"], check=True, timeout=1800)
c.require(image_digest(reference) == expected, "registry changed the accepted image digest")
def images(inputs, directory, *, latest=False):
for variant, suffix in (("musl", ""), ("gnu", "-glibc")):
record = c.read_json(directory / f"image-{variant}.json")
for registry in REGISTRIES:
tag = ("latest" if latest else inputs["version"]) + suffix
copy_image(directory / f"image-{variant}.tar", f"{registry}:{tag}", record["digest"], mutable=latest)
def publish_version(inputs, seal, directory, manifest_sha):
ensure_tag(inputs, manifest_sha)
value = release(inputs, manifest_sha)
paths = public_files(directory)
with tempfile.TemporaryDirectory() as temp:
alias_paths = []
for name, source in seal["aliases"].items():
path = Path(temp) / name
shutil.copyfile(directory / source, path)
alias_paths.append(path)
github_assets(inputs, value, paths + alias_paths)
for path in paths:
prefix = "artifacts/rustfs/release/"
if path.name in ("release-manifest.json", "release-evidence.zip", "SHA256SUMS", "SHA512SUMS"):
prefix += inputs["version"] + "/"
r2_file(path, prefix + path.name)
images(inputs, directory)
# Every versioned destination is verified before the draft is made public.
if value["draft"]:
api(f"repos/{c.REPOSITORY}/releases/{value['id']}", "PATCH",
{"draft": False, "prerelease": "-" in inputs["version"], "make_latest": "false"})
def stable_key(version):
c.require(isinstance(version, str) and re.fullmatch(r"v?(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)", version), "invalid stable channel version")
return tuple(map(int, version.lstrip("v").split(".")))
def helm(inputs, directory):
record = c.read_json(directory / "chart.json")
package = directory / record["name"]
c.require(c.file_digest(package) == record["sha256"], "Helm package changed")
env = dict(os.environ, GH_TOKEN=os.environ["HELM_TOKEN"])
subprocess.run(["gh", "auth", "setup-git", "--hostname", "github.com"], env=env, check=True, timeout=60)
with tempfile.TemporaryDirectory() as temp:
subprocess.run(["git", "clone", "--depth", "1", "--branch", "main", "https://github.com/rustfs/helm.git", temp], env=env, check=True, timeout=120)
path = Path(temp) / package.name
if path.exists():
c.require(c.file_digest(path) == record["sha256"], "published Helm version already has different bytes")
c.require((Path(temp) / "index.yaml").is_file(), "existing Helm chart has no index")
return
shutil.copyfile(package, path)
c.command(["helm", "repo", "index", ".", "--url", "https://charts.rustfs.com"], cwd=temp)
c.command(["git", "config", "user.name", "RustFS Release"], cwd=temp)
c.command(["git", "config", "user.email", "dev@rustfs.com"], cwd=temp)
c.command(["git", "add", package.name, "index.yaml"], cwd=temp)
c.command(["git", "commit", "-m", f"chore: publish chart {inputs['version']}"], cwd=temp)
subprocess.run(["git", "push", "origin", "HEAD:main"], cwd=temp, env=env, check=True, timeout=120)
def channels(inputs, seal, directory, manifest_sha):
# Recheck all immutable destinations, including when resuming a failed job.
publish_version(inputs, seal, directory, manifest_sha)
# The versioned Helm catalog entry must still be completed when a newer
# release already owns the mutable channels.
helm(inputs, directory)
if "-" in inputs["version"]:
return
latest = api(f"repos/{c.REPOSITORY}/releases/latest", missing=True)
if latest and stable_key(latest["tag_name"]) > stable_key(inputs["version"]):
print("A newer stable release owns the channels; keeping its pointers.")
return
value = release(inputs, manifest_sha)
# GitHub Latest also acts as a monotonic marker during partial channel
# failures. Versioned assets and images are already fully available here.
api(f"repos/{c.REPOSITORY}/releases/{value['id']}", "PATCH", {"make_latest": "true"})
images(inputs, directory, latest=True)
for target in c.TARGETS:
path = directory / f"rustfs-{target}-v{inputs['version']}.zip"
r2_file(path, f"artifacts/rustfs/release/rustfs-{target}-latest.zip", mutable=True)
pointer = {"version": inputs["version"], "tag": inputs["version"], "release_date": value.get("published_at"),
"release_type": "stable", "download_url": f"https://github.com/{c.REPOSITORY}/releases/tag/{inputs['version']}"}
with tempfile.TemporaryDirectory() as temp:
path = Path(temp) / "latest.json"
c.write_json(path, pointer)
subprocess.run([os.environ["OSSUTIL_BIN"], "cp", str(path), "oss://rustfs-version/latest.json", "--force"], check=True, timeout=120)
actual = Path(temp) / "actual.json"
subprocess.run([os.environ["OSSUTIL_BIN"], "cp", "oss://rustfs-version/latest.json", str(actual)], check=True, timeout=120)
c.require(c.read_json(actual) == pointer, "installer pointer did not match the published release")
def main():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("mode", choices=("verify", "version", "channels"))
p.add_argument("--run-id", type=int, required=True)
p.add_argument("--digest", required=True)
p.add_argument("--directory", type=Path, required=True)
args = p.parse_args()
inputs, seal, current = verify(args.run_id, args.digest, args.directory)
if args.mode != "verify":
for name in ("R2_ENDPOINT", "R2_BUCKET", "AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "REGISTRY_AUTH_FILE"):
c.require(bool(os.environ.get(name)), f"missing publication setting: {name}")
if args.mode == "channels":
c.require(os.environ.get("HELM_TOKEN") and os.environ.get("OSSUTIL_BIN"), "missing channel publication settings")
{"version": publish_version, "channels": channels}[args.mode](inputs, seal, args.directory, args.digest)
c.write_json("publication.json", {"schema": 1, "version": inputs["version"], "manifest_sha256": args.digest,
"source_sha": inputs["source_sha"], "complete_run_id": args.run_id,
"publish_run_id": current["id"], "stage": args.mode})
if __name__ == "__main__":
main()
+35 -9
View File
@@ -263,9 +263,12 @@ def validate_plan(plan):
harness_repository = require_string(
entry, "harness_repository", code, f"plan lane {lane}"
)
harness_sha = require_hex(
entry.get("harness_sha"), HEX40, code, "harness_sha", f"plan lane {lane}"
)
harness_sha = entry.get("harness_sha")
if harness_sha == "candidate.workflow_sha":
if harness_repository != "rustfs/rustfs":
raise GateError(code, "dynamic harness identity is limited to the product repository")
else:
harness_sha = require_hex(harness_sha, HEX40, code, "harness_sha", f"plan lane {lane}")
timeout_seconds = entry.get("timeout_seconds")
if not isinstance(timeout_seconds, int) or timeout_seconds <= 0:
raise GateError(
@@ -500,7 +503,12 @@ def validate_candidate(candidate, plan_info, plan_sha256):
)
image = {"ref": ref, "digest": digest, "binary_sha256": image_binary}
workflow_sha = candidate.get("workflow_sha")
if any(lane["harness_sha"] == "candidate.workflow_sha" for lane in plan_info["lanes"].values()):
workflow_sha = require_hex(workflow_sha, HEX40, code, "workflow_sha", "candidate")
return {
"workflow_sha": workflow_sha,
"product_commit": product_commit,
"version": version,
"release_profile": release_profile,
@@ -616,7 +624,10 @@ def validate_envelope(envelope, plan_info, candidate_info, source_name):
lane_info = plan_info["lanes"].get(lane)
if lane_info is not None:
if harness_commit != lane_info["harness_sha"]:
expected_harness = lane_info["harness_sha"]
if expected_harness == "candidate.workflow_sha":
expected_harness = candidate_info["workflow_sha"]
if harness_commit != expected_harness:
raise GateError(
"harness_mismatch",
f"{context}: harness_commit does not match the fixed harness SHA for lane {lane}",
@@ -658,6 +669,14 @@ def validate_envelope(envelope, plan_info, candidate_info, source_name):
)
artifact = envelope.get("artifact")
if state == "pass" and candidate_info.get("workflow_sha"):
if not isinstance(artifact, dict) or not artifact.get("archive_sha256") or not artifact.get("binary_sha256"):
raise GateError(code, f"{context}: candidate PASS must bind both archive and executable bytes")
target = "-".join(part for part in platform_key(platform) if part not in ("none", "msvc"))
name = f"rustfs-{target}-v{candidate_info['version']}.zip"
matches = [d for d in candidate_info["deliverables"] if d["name"] == name]
if len(matches) != 1 or matches[0]["sha256"] != artifact["archive_sha256"] or matches[0]["binary_sha256"] != artifact["binary_sha256"]:
raise GateError("digest_mismatch", f"{context}: evidence belongs to a different platform artifact")
if artifact is not None:
if not isinstance(artifact, dict):
raise GateError(code, f"{context}: artifact must be an object")
@@ -734,7 +753,7 @@ def _validate_artifact_binding(artifact, candidate_info, context, case_id, lane)
# ---------------------------------------------------------------------------
def run_validate(plan_path, candidate_path, results_dir, output_dir):
def run_validate(plan_path, candidate_path, results_dir, output_dir, scope="full"):
findings = []
plan_info = None
candidate_info = None
@@ -755,6 +774,10 @@ def run_validate(plan_path, candidate_path, results_dir, output_dir):
return 1
required = expand_required(plan_info["lanes"], candidate_info["release_profile"])
if scope == "linux":
required = [case for case in required if case["platform"]["os"] == "linux"]
elif scope != "full":
raise GateError("plan_invalid", "unknown validation scope")
expansion = {
"schema_version": SCHEMA_VERSION,
"plan_sha256": plan_info["plan_sha256"],
@@ -915,7 +938,7 @@ def run_validate(plan_path, candidate_path, results_dir, output_dir):
)
)
_write_outputs(output, findings, plan_info, candidate_info, required, envelopes)
_write_outputs(output, findings, plan_info, candidate_info, required, envelopes, scope=scope)
return 1 if findings else 0
@@ -939,12 +962,13 @@ def _validate_batch(document, candidate_info, source_name):
return {"run_id": run_id, "run_attempt": run_attempt, "status": status}
def _write_outputs(output, findings, plan_info, candidate_info, required, envelopes):
def _write_outputs(output, findings, plan_info, candidate_info, required, envelopes, scope="full"):
state = "blocked" if findings else "pass"
summary = {
"schema_version": SCHEMA_VERSION,
"state": state,
"publish_allowed": not findings,
"scope": scope,
"publish_allowed": not findings and scope == "full",
"plan_sha256": plan_info["plan_sha256"] if plan_info else None,
"candidate_sha256": candidate_info["candidate_sha256"] if candidate_info else None,
"product_commit": candidate_info["product_commit"] if candidate_info else None,
@@ -972,7 +996,7 @@ def _write_outputs(output, findings, plan_info, candidate_info, required, envelo
"# Release gate summary",
"",
f"- state: {state}",
f"- publish_allowed: {str(not findings).lower()}",
f"- publish_allowed: {str(summary['publish_allowed']).lower()}",
f"- plan_sha256: {summary['plan_sha256']}",
f"- candidate_sha256: {summary['candidate_sha256']}",
f"- required cases: {len(required)}",
@@ -1190,6 +1214,7 @@ def main(argv=None):
validate_parser.add_argument("--candidate", required=True)
validate_parser.add_argument("--results", required=True)
validate_parser.add_argument("--output", required=True)
validate_parser.add_argument("--scope", choices=("full", "linux"), default="full")
promote_parser = subparsers.add_parser(
"verify-promotion", help="re-check to-be-published bytes against the passed list"
@@ -1216,6 +1241,7 @@ def main(argv=None):
arguments.candidate,
arguments.results,
arguments.output,
scope=arguments.scope,
)
except GateError as error:
print(f"{error.code}: {error.message}", file=sys.stderr)
+279
View File
@@ -0,0 +1,279 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
"""Exercise release identity, substitution and retry boundaries without publishing."""
from __future__ import annotations
import copy
import io
import os
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
import urllib.request
import zipfile
import candidate
class TransportTests(unittest.TestCase):
def test_signed_download_redirect_does_not_leak_the_github_token(self):
request = urllib.request.Request("https://api.github.com/repos/rustfs/rustfs/actions/artifacts/1/zip", headers={"Authorization": "Bearer test-token"})
redirected = candidate.ArtifactRedirect().redirect_request(request, None, 302, "Found", {}, "https://example.blob.core.windows.net/artifact.zip?signature=fixture")
self.assertIsNone(redirected.get_header("Authorization"))
same_host = candidate.ArtifactRedirect().redirect_request(request, None, 302, "Found", {}, "https://api.github.com/another-path")
self.assertEqual(same_host.get_header("Authorization"), "Bearer test-token")
with self.assertRaises(ValueError):
candidate.ArtifactRedirect().redirect_request(request, None, 302, "Found", {}, "http://example.invalid/artifact.zip")
def test_download_checks_real_bytes_and_rejects_traversal(self):
import hashlib
run = {"id": 70, "head_sha": "a" * 40}
def blob(name):
stream = io.BytesIO()
with zipfile.ZipFile(stream, "w") as package:
package.writestr(name, b"accepted")
return stream.getvalue()
for name, valid in (("record.json", True), ("../escape", False)):
content = blob(name)
artifact = {"id": 90, "name": "candidate", "expired": False, "workflow_run": run,
"size_in_bytes": len(content), "digest": "sha256:" + hashlib.sha256(content).hexdigest()}
with tempfile.TemporaryDirectory() as temp, patch.object(candidate, "artifact_list", return_value=[artifact]), patch.object(candidate, "github_stream", side_effect=lambda _: io.BytesIO(content)):
if valid:
candidate.download_artifact(run, "candidate", temp)
self.assertEqual((Path(temp) / name).read_bytes(), b"accepted")
artifact["digest"] = "sha256:" + "0" * 64
with self.assertRaisesRegex(ValueError, "digest"):
candidate.download_artifact(run, "candidate", Path(temp) / "changed")
else:
with self.assertRaisesRegex(ValueError, "unsafe"):
candidate.download_artifact(run, "candidate", temp)
def test_duplicate_json_fields_are_not_silently_accepted(self):
with tempfile.TemporaryDirectory() as temp:
path = Path(temp) / "manifest.json"
path.write_text('{"schema": 1, "schema": 2}')
with self.assertRaisesRegex(ValueError, "duplicate"):
candidate.read_json(path)
class FreezeRepositoryTests(unittest.TestCase):
def test_freeze_reads_real_commit_and_rejects_recipe_or_version_drift(self):
with tempfile.TemporaryDirectory() as temp:
root = Path(temp)
for name in candidate.RECIPE_PATHS:
path = root / name; path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("recipe\n" if name != ".config/release-validation-plan.json" else "{}\n")
for directory in ("bootstrap", "helm"):
(root / directory).mkdir()
(root / directory / "input.txt").write_text("runtime input\n")
(root / "Cargo.toml").write_text('[workspace.package]\nversion = "1.2.3"\n')
(root / "Cargo.lock").write_text("version = 4\n")
env = dict(os.environ, GIT_AUTHOR_NAME="Fixture", GIT_AUTHOR_EMAIL="fixture@example.invalid", GIT_COMMITTER_NAME="Fixture", GIT_COMMITTER_EMAIL="fixture@example.invalid")
def git(*args):
return subprocess.check_output(["git", *args], cwd=root, env=env, stderr=subprocess.DEVNULL, text=True).strip()
git("init", "-q"); git("add", "."); git("commit", "-qm", "fixture")
original = git("rev-parse", "HEAD")
value = inputs()
console = {"draft": False, "prerelease": False, "tag_name": value["console"]["tag"],
"assets": [{"name": value["console"]["name"], "browser_download_url": value["console"]["url"], "digest": "sha256:" + value["console"]["sha256"]}]}
cli = {"draft": False, "prerelease": False, "tag_name": "v0.1.0", "assets": []}
for arch in ("amd64", "arm64"):
for abi in ("", "-gnu"):
name = f"rustfs-cli-linux-{arch}{abi}-v0.1.0.tar.gz"
cli["assets"].append({"name": name, "browser_download_url": f"https://github.com/rustfs/cli/releases/download/v0.1.0/{name}", "digest": "sha256:" + "f" * 64})
previous = {"draft": False, "prerelease": False, "tag_name": "1.2.2", "assets": [{"name": value["previous"]["name"], "browser_download_url": value["previous"]["url"], "digest": "sha256:" + value["previous"]["sha256"]}]}
def freeze(source, workflow, version="1.2.3"):
return candidate.freeze(version, source, workflow, "1.92.0", console, 70, 7, cli, previous, {"run_id": 60, "run_attempt": 1}, root)
frozen = freeze(original, original)
(root / "unrelated.txt").write_text("main may advance\n")
git("add", "."); git("commit", "-qm", "unrelated")
advanced = git("rev-parse", "HEAD")
self.assertEqual(freeze(original, advanced)["recipe_sha256"], frozen["recipe_sha256"])
(root / ".github/workflows/build.yml").write_text("different recipe\n")
git("add", "."); git("commit", "-qm", "recipe change")
with self.assertRaisesRegex(ValueError, "recipe"):
freeze(original, git("rev-parse", "HEAD"))
with self.assertRaisesRegex(ValueError, "version"):
freeze(original, original, "1.2.4")
def inputs():
value = {
"schema": 1, "candidate_id": "1.2.3-preview.7", "version": "1.2.3", "ci": {"run_id": 60, "run_attempt": 1},
"source_sha": "a" * 40, "workflow_sha": "b" * 40, "recipe_sha256": "c" * 64,
"lock_sha256": "d" * 64, "plan_sha256": "e" * 64, "toolchain": "1.92.0",
"console": {"tag": "v0.1.41", "name": "rustfs-console-v0.1.41.zip",
"url": "https://github.com/rustfs/console/releases/download/v0.1.41/rustfs-console-v0.1.41.zip",
"sha256": "f" * 64}, "origin_run_id": 70, "origin_run_number": 7,
}
def asset(repo, version, name):
return {"version": version, "name": name, "url": f"https://github.com/{repo}/releases/download/{version}/{name}", "sha256": "f" * 64}
value["previous"] = asset("rustfs/rustfs", "1.2.2", "rustfs-linux-x86_64-gnu-v1.2.2.zip")
value["cli"] = {target: asset("rustfs/cli", "v0.1.0", "rustfs-cli.tar.gz") for target in candidate.LINUX}
return value
def archive(directory, target, value=None, *, extra=None, binary=b"exact candidate bytes"):
value = value or inputs()
path = Path(directory) / f"rustfs-{target}-v{value['version']}.zip"
members = {"rustfs.exe" if target == "windows-x86_64" else "rustfs": binary}
if target.endswith("-gnu"):
members.update({"rustfs.cpu-symbol-catalog.json": b"{}", "rustfs.cpu-symbol-catalog.sha256": b"catalog"})
if extra:
members.update(extra)
with zipfile.ZipFile(path, "w") as output:
for name, content in members.items():
output.writestr(name, content)
return path
class InputIdentityTests(unittest.TestCase):
def test_valid_frozen_candidate(self):
self.assertEqual(candidate.validate_inputs(inputs()), inputs())
def test_moving_or_injected_inputs_rejected(self):
changes = [("source_sha", "main"), ("version", "1.2.3\nTAG=bad"), ("version", "01.2.3"),
("candidate_id", "1.2.3-preview.8"), ("toolchain", "stable"), ("origin_run_id", True),
("schema", True), ("recipe_sha256", "c" * 63)]
for key, value in changes:
with self.subTest(key=key, value=value):
record = inputs(); record[key] = value
with self.assertRaises(ValueError):
candidate.validate_inputs(record)
def test_console_asset_cannot_redirect_or_be_ambiguous(self):
record = inputs()
record["console"]["url"] = "https://example.invalid/console.zip"
with self.assertRaises(ValueError):
candidate.validate_inputs(record)
asset = {"name": inputs()["console"]["name"], "browser_download_url": inputs()["console"]["url"], "digest": "sha256:" + "f" * 64}
for release in ({"draft": True, "prerelease": False, "tag_name": "v0.1.41", "assets": [asset]},
{"draft": False, "prerelease": False, "tag_name": "v0.1.41", "assets": [asset, asset]}):
with self.assertRaises(ValueError):
candidate.console_asset(release)
def test_frozen_record_is_retryable_but_not_replaceable(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "inputs.json"
candidate.write_json(path, inputs())
before = path.read_bytes()
candidate.write_json(path, inputs())
self.assertEqual(before, path.read_bytes())
changed = inputs(); changed["source_sha"] = "f" * 40
with self.assertRaisesRegex(ValueError, "replace"):
candidate.write_json(path, changed)
self.assertEqual(before, path.read_bytes())
def test_platform_selection_keeps_desktop_out_of_linux(self):
self.assertEqual(candidate.targets("linux"), candidate.LINUX)
self.assertEqual(candidate.targets("desktop"), candidate.DESKTOP)
for value in ("", "linux-x86_64-gnu,linux-x86_64-gnu", "linux,windows", "all,linux"):
with self.subTest(value=value), self.assertRaises(ValueError):
candidate.targets(value)
class ArtifactIdentityTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory(); self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.inputs = inputs()
self.records = [candidate.archive_record(archive(self.root, target), target, self.inputs) for target in candidate.TARGETS]
def test_all_six_final_artifacts_bind_exact_bytes(self):
value = candidate.certification_candidate(self.inputs, self.records, self.root)
self.assertEqual(len(value["deliverables"]), 6)
self.assertEqual(value["version"], "1.2.3")
self.assertEqual(value["release_profile"], "stable")
self.assertEqual(value["validation_batch"], {"run_id": "1.2.3-preview.7", "run_attempt": 1})
self.assertNotIn("latest", json.dumps(value))
def test_partial_linux_bundle_cannot_be_published_as_complete(self):
linux = self.records[:4]
self.assertEqual(len(candidate.certification_candidate(self.inputs, linux, self.root, candidate.LINUX)["deliverables"]), 4)
with self.assertRaisesRegex(ValueError, "missing"):
candidate.certification_candidate(self.inputs, linux, self.root)
def test_same_version_different_bytes_rejected(self):
archive(self.root, candidate.TARGETS[0], binary=b"replacement")
with self.assertRaisesRegex(ValueError, "changed"):
candidate.verify_records(self.inputs, self.records, self.root, candidate.TARGETS)
def test_different_inputs_cannot_share_receipts(self):
changed = copy.deepcopy(self.inputs); changed["console"]["sha256"] = "0" * 64
with self.assertRaisesRegex(ValueError, "changed"):
candidate.verify_records(changed, self.records, self.root, candidate.TARGETS)
def test_duplicate_or_mislabeled_platform_rejected(self):
for replacement in (self.records[:-1] + [self.records[0]], self.records + [self.records[0]]):
with self.assertRaises(ValueError):
candidate.verify_records(self.inputs, replacement, self.root, candidate.TARGETS)
wrong = copy.deepcopy(self.records); wrong[0]["name"] = "../replacement.zip"
with self.assertRaises(ValueError):
candidate.verify_records(self.inputs, wrong, self.root, candidate.TARGETS)
def test_archive_members_are_strict(self):
for name in ("../escape", "rustfs-cli", "nested/rustfs"):
with self.subTest(name=name):
path = archive(self.root, candidate.TARGETS[0], extra={name: b"extra"})
with self.assertRaises(ValueError):
candidate.archive_record(path, candidate.TARGETS[0], self.inputs)
def test_binary_symlink_rejected(self):
path = self.root / f"rustfs-{candidate.TARGETS[0]}-v1.2.3.zip"
with zipfile.ZipFile(path, "w") as output:
entry = zipfile.ZipInfo("rustfs"); entry.external_attr = 0o120777 << 16
output.writestr(entry, "../../untrusted")
with self.assertRaises(ValueError):
candidate.archive_record(path, candidate.TARGETS[0], self.inputs)
def test_real_cli_fails_without_overwriting_existing_output(self):
candidate.write_json(self.root / "inputs.json", self.inputs)
for item in self.records:
candidate.write_json(self.root / f"record-{item['target']}.json", item)
output = self.root / "candidate.json"
command = [sys.executable, str(Path(candidate.__file__)), "assemble", "--inputs", str(self.root / "inputs.json"),
"--directory", str(self.root), "--output", str(output)]
good = subprocess.run(command, text=True, capture_output=True)
self.assertEqual(good.returncode, 0, good.stderr)
original = output.read_bytes()
archive(self.root, candidate.TARGETS[1], binary=b"bad retry")
bad = subprocess.run(command, text=True, capture_output=True)
self.assertNotEqual(bad.returncode, 0)
self.assertEqual(output.read_bytes(), original)
class ProvenanceTests(unittest.TestCase):
def run_record(self):
return {"repository": {"full_name": "rustfs/rustfs"}, "path": ".github/workflows/release-candidate.yml",
"event": "workflow_dispatch", "head_branch": "main", "id": 70, "run_attempt": 1,
"head_sha": "b" * 40, "status": "completed", "conclusion": "success"}
def test_trusted_source_and_completion_required(self):
run = self.run_record()
self.assertEqual(candidate.validate_run(run, "release-candidate.yml"), run)
for field, value in (("head_branch", "unreviewed"), ("event", "pull_request"), ("conclusion", "failure"),
("status", "in_progress"), ("repository", {"full_name": "fork/rustfs"}),
("path", ".github/workflows/other.yml")):
with self.subTest(field=field):
changed = {**run, field: value}
with self.assertRaises(ValueError):
candidate.validate_run(changed, "release-candidate.yml")
def test_artifact_must_match_exact_origin(self):
run = self.run_record()
artifact = {"name": "candidate", "expired": False, "id": 1, "size_in_bytes": 100,
"digest": "sha256:" + "c" * 64, "workflow_run": {"id": 70, "head_sha": "b" * 40}}
candidate.validate_artifact(artifact, run, "candidate")
for field, value in (("expired", True), ("name", "other"), ("digest", ""), ("size_in_bytes", True),
("workflow_run", {"id": 71, "head_sha": "b" * 40})):
with self.subTest(field=field), self.assertRaises(ValueError):
candidate.validate_artifact({**artifact, field: value}, run, "candidate")
if __name__ == "__main__":
unittest.main()
+453
View File
@@ -0,0 +1,453 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
"""Offline contract rehearsals; these are not product acceptance receipts."""
from __future__ import annotations
import copy
import hashlib
import json
import os
from pathlib import Path
import re
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
import zipfile
import candidate as c
import pipeline
import publish
import release_gate
import verify
from test_candidate import archive, inputs
ROOT = c.ROOT
def workflow(name):
# Ruby/Psych is available on the developer host and hosted CI image. Tests
# parse the actual workflow rather than a separately maintained job graph.
result = subprocess.check_output(["ruby", "-ryaml", "-rjson", "-e",
"puts JSON.generate(YAML.load_file(ARGV[0]))", str(ROOT / ".github/workflows" / name)], text=True)
value = json.loads(result)
value["on"] = value.pop("true", value.get("on"))
return value
def run_metadata(run_id, file, sha):
return {"id": run_id, "run_attempt": 2, "head_sha": sha, "head_branch": "main", "event": "workflow_dispatch",
"status": "completed", "conclusion": "success", "path": f".github/workflows/{file}",
"repository": {"full_name": c.REPOSITORY}}
def fixture(directory, selection="all"):
value = inputs()
value["plan_sha256"] = c.digest(c.read_json(ROOT / ".config/release-validation-plan.json"))
runs = {70: run_metadata(70, "release-candidate.yml", value["workflow_sha"]),
80: run_metadata(80, "release-complete.yml", "c" * 40)}
for target in c.targets(selection):
record = c.archive_record(archive(directory, target, value, binary=target.encode()), target, value)
c.write_json(directory / f"record-{target}.json", record)
run = runs[70 if target in c.LINUX else 80]
env = {"GITHUB_WORKFLOW_REF": f"{c.REPOSITORY}/{run['path']}@refs/heads/main", "GITHUB_SHA": run["head_sha"],
"GITHUB_RUN_ID": str(run["id"]), "GITHUB_RUN_ATTEMPT": "1"}
with patch.dict(os.environ, env):
for case in ("artifact-identity", "startup"):
log = directory / f"{target}-{case}.log"
log.write_text("contract fixture only\n")
verify.envelope(value, record, "identity", case, [log], verify.now(), directory)
if target == "linux-x86_64-gnu":
for case in verify.UPGRADES:
log = directory / f"upgrade-{case}.log"
log.write_text("contract fixture only\n")
verify.envelope(value, record, "upgrade", case, [log], verify.now(), directory)
return value, runs
class CertificationTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.path = Path(self.temp.name)
self.value, self.runs = fixture(self.path)
def test_complete_original_bytes_certify(self):
_, summary = pipeline.certify(self.value, self.path, self.runs, "full")
self.assertTrue(summary["publish_allowed"])
self.assertEqual(len(summary["required_cases"]), 18)
def test_linux_only_cannot_authorize_release(self):
with tempfile.TemporaryDirectory() as temp:
value, runs = fixture(Path(temp), "linux")
candidate, summary = pipeline.certify(value, Path(temp), runs, "linux")
self.assertEqual(summary["state"], "pass")
self.assertFalse(summary["publish_allowed"])
self.assertEqual(len(summary["required_cases"]), 14)
self.assertEqual(len(candidate["deliverables"]), 4)
with self.assertRaises((ValueError, OSError)):
pipeline.certify(value, Path(temp), runs, "full")
def mutate_envelope(self, edit):
path = self.path / "identity-linux-x86_64-gnu-startup.json"
value = c.read_json(path)
edit(value)
path.write_text(json.dumps(value))
def test_digest_binding_is_required_and_platform_specific(self):
original = (self.path / "identity-linux-x86_64-gnu-startup.json").read_bytes()
arm = c.read_json(self.path / "record-linux-aarch64-gnu.json")
edits = [lambda v: v.pop("artifact"), lambda v: v["artifact"].update(archive_sha256="0" * 64),
lambda v: v["artifact"].update(archive_sha256=arm["sha256"], binary_sha256=arm["binary_sha256"])]
for edit in edits:
with self.subTest(edit=edit):
(self.path / "identity-linux-x86_64-gnu-startup.json").write_bytes(original)
self.mutate_envelope(edit)
with self.assertRaises(ValueError):
pipeline.certify(self.value, self.path, self.runs, "full")
def test_missing_changed_and_duplicate_evidence_are_rejected(self):
path = self.path / "linux-x86_64-gnu-startup.log"
path.write_text("substituted")
with self.assertRaisesRegex(ValueError, "evidence"):
pipeline.certify(self.value, self.path, self.runs, "full")
def test_receipt_cannot_claim_another_run_or_new_attempt(self):
self.mutate_envelope(lambda v: v["workflow"].update(run_attempt=3))
with self.assertRaisesRegex(ValueError, "trusted run"):
pipeline.certify(self.value, self.path, self.runs, "full")
def test_old_successful_unit_survives_failed_job_retry(self):
# The candidate batch is immutable; successful unit receipts from an
# earlier attempt still bind the exact archive, plan and producer.
_, summary = pipeline.certify(self.value, self.path, self.runs, "full")
self.assertEqual(summary["state"], "pass")
self.mutate_envelope(lambda v: v.update(run_id="different-candidate"))
with self.assertRaises(ValueError):
pipeline.certify(self.value, self.path, self.runs, "full")
def test_new_verifier_or_plan_cannot_reuse_old_acceptance(self):
self.value["workflow_sha"] = "d" * 40
with self.assertRaises(ValueError):
pipeline.certify(self.value, self.path, self.runs, "full")
class SourceCITests(unittest.TestCase):
def test_spent_versions_and_unavailable_lookups_fail_before_building(self):
import urllib.error
missing = urllib.error.HTTPError('https://api.github.com/fixture', 404, 'Not Found', {}, None)
with patch.object(c, "github", side_effect=missing) as api:
pipeline.unused_version("1.2.3")
self.assertEqual(api.call_count, 4)
with patch.object(c, "github", return_value={"id": 1}):
with self.assertRaisesRegex(ValueError, "already has a tag or release"):
pipeline.unused_version("1.2.3")
unavailable = urllib.error.HTTPError('https://api.github.com/fixture', 503, 'Unavailable', {}, None)
with patch.object(c, "github", side_effect=unavailable):
with self.assertRaises(urllib.error.HTTPError):
pipeline.unused_version("1.2.3")
def test_only_complete_exact_source_ci_is_accepted(self):
run = run_metadata(60, "ci.yml", "a" * 40)
jobs = [{"name": name, "status": "completed", "conclusion": "success"} for name in
("Quick Checks", "Test and Lint", "End-to-End Tests (full merge gate)")]
def api(path):
return {"jobs": jobs} if "/jobs?" in path else {"workflow_runs": [run]}
with patch.object(c, "github", side_effect=api):
self.assertEqual(pipeline.source_ci("a" * 40, "1.2.3"), {"run_id": 60, "run_attempt": 2})
for key, wrong in (("head_sha", "b" * 40), ("head_branch", "unreviewed"), ("event", "pull_request"), ("conclusion", "failure")):
old = run[key]; run[key] = wrong
with self.subTest(key=key), self.assertRaises(ValueError):
pipeline.source_ci("a" * 40, "1.2.3")
run[key] = old
for conclusion in ("skipped", "failure", None):
jobs[2]["conclusion"] = conclusion
with self.subTest(conclusion=conclusion), self.assertRaises(ValueError):
pipeline.source_ci("a" * 40, "1.2.3")
def test_duplicate_required_ci_job_is_rejected(self):
run = run_metadata(60, "ci.yml", "a" * 40)
jobs = [{"name": name, "status": "completed", "conclusion": "success"} for name in
("Quick Checks", "Quick Checks", "Test and Lint", "End-to-End Tests (full merge gate)")]
with patch.object(c, "github", side_effect=lambda path: {"jobs": jobs} if "/jobs?" in path else {"workflow_runs": [run]}):
with self.assertRaises(ValueError):
pipeline.source_ci("a" * 40, "1.2.3")
class SupplyChainTests(unittest.TestCase):
def test_actual_generator_uses_frozen_lock_and_all_package_formats(self):
with tempfile.TemporaryDirectory() as temp:
root = Path(temp)
assets = root / "assets"; assets.mkdir()
for name in ("server.zip", "package.deb", "package.rpm", "chart.tgz"):
(assets / name).write_bytes(name.encode())
lock = root / "Cargo.lock"
lock.write_text('version = 4\n[[package]]\nname = "frozen-fixture"\nversion = "1.2.3"\n')
args = [sys.executable, str(ROOT / "scripts/security/generate_release_supply_chain_assets.py"), "--asset-dir", str(assets),
"--version", "1.2.3", "--cargo-lock", str(lock), "--workflow", "release-complete.yml", "--sha", "a" * 40]
result = subprocess.run(args, capture_output=True, text=True, timeout=10)
self.assertEqual(result.returncode, 0, result.stderr)
provenance = c.read_json(assets / "rustfs-1.2.3.provenance.json")
self.assertEqual({s["name"] for s in provenance["subject"]}, {"server.zip", "package.deb", "package.rpm", "chart.tgz"})
self.assertTrue(provenance["predicate"]["buildDefinition"]["externalParameters"]["workflow"].endswith("/release-complete.yml"))
sbom = c.read_json(assets / "rustfs-1.2.3.sbom.cdx.json")
self.assertIn("frozen-fixture", {item["name"] for item in sbom["components"]})
lock.unlink()
result = subprocess.run(args, capture_output=True, text=True, timeout=10)
self.assertNotEqual(result.returncode, 0)
class NativeEntryTests(unittest.TestCase):
def test_real_native_entry_point_version_startup_and_failure(self):
# A small child process exercises the verifier entry point and lifecycle;
# it deliberately does not pretend to be a RustFS product test.
target = "macos-aarch64"
value = inputs()
version = f"rustfs {value['version']}\nbuild target : {verify.TRIPLES[target]}\ngit commit : {value['source_sha']}\nrust version : {value['toolchain']}\n"
source = f'''#!/usr/bin/env python3
import sys, threading
from http.server import BaseHTTPRequestHandler, HTTPServer
store = {{}}
if '--version' in sys.argv:
print({version!r}, end='')
else:
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header('Content-Type', 'text/html')
self.end_headers()
self.wfile.write(store.get(self.path, b'<html>fixture</html>'))
def do_PUT(self):
store[self.path] = self.rfile.read(int(self.headers.get('Content-Length', '0')))
self.send_response(200); self.end_headers()
def do_DELETE(self):
store.pop(self.path, None)
self.send_response(204); self.end_headers()
api = sys.argv[sys.argv.index('--address')+1].split(':')
server = HTTPServer((api[0], int(api[1])), Handler)
threading.Thread(target=server.serve_forever, daemon=True).start()
address=sys.argv[sys.argv.index('--console-address')+1]
host, port=address.split(':')
HTTPServer((host,int(port)),Handler).serve_forever()
'''
with tempfile.TemporaryDirectory() as temp:
path = Path(temp)
c.write_json(path / "inputs.json", value)
binary = archive(path, target, value, binary=source.encode())
env = dict(os.environ, GITHUB_WORKFLOW_REF=f"{c.REPOSITORY}/.github/workflows/release-complete.yml@refs/heads/main",
GITHUB_RUN_ID="80", GITHUB_RUN_ATTEMPT="1", GITHUB_SHA="c" * 40)
args = [sys.executable, str(ROOT / "scripts/release/verify.py"), "native", "--inputs", str(path / "inputs.json"),
"--archive", str(binary), "--target", target, "--output", str(path / "receipts")]
result = subprocess.run(args, env=env, capture_output=True, text=True, timeout=30)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(len(list((path / "receipts").glob("*.json"))), 2)
archive(path, target, value, binary=source.replace("rustfs 1.2.3", "rustfs 1.2.4").encode())
args[-1] = str(path / "rejected")
result = subprocess.run(args, env=env, capture_output=True, text=True, timeout=30)
self.assertNotEqual(result.returncode, 0)
self.assertFalse(list((path / "rejected").glob("*.json")))
class PromotionTests(unittest.TestCase):
def test_tag_is_created_once_and_conflicting_tag_never_moved(self):
value = inputs()
state = {}
calls = []
def api(path, method="GET", body=None, missing=False):
calls.append((path, method, body))
if path.endswith("/git/tags"):
state["tag"] = dict(body, object={"sha": body["object"], "type": body["type"]})
return {"sha": "f" * 40}
if path.endswith("/git/refs"):
state["ref"] = {"object": {"type": "tag", "sha": body["sha"]}}
return state["ref"]
if "/git/ref/tags/" in path:
return state.get("ref")
return state["tag"]
with patch.object(publish, "api", side_effect=api):
publish.ensure_tag(value, "a" * 64)
count = len([call for call in calls if call[1] == "POST"])
publish.ensure_tag(value, "a" * 64)
self.assertEqual(len([call for call in calls if call[1] == "POST"]), count)
with self.assertRaisesRegex(ValueError, "never move"):
publish.ensure_tag(value, "b" * 64)
state["tag"]["object"]["sha"] = "e" * 40
with self.assertRaises(ValueError):
publish.ensure_tag(value, "a" * 64)
self.assertFalse(any(method in ("PATCH", "DELETE") for _, method, _ in calls))
def test_image_promotion_preserves_digest_and_refuses_overwrite(self):
with patch.dict(os.environ, REGISTRY_AUTH_FILE="auth.json"), patch.object(publish, "image_digest", return_value="sha256:" + "a" * 64), patch.object(subprocess, "run") as command:
publish.copy_image(Path("image.tar"), "registry/image:1.2.3", "sha256:" + "a" * 64)
command.assert_not_called()
with self.assertRaisesRegex(ValueError, "never replace"):
publish.copy_image(Path("image.tar"), "registry/image:1.2.3", "sha256:" + "b" * 64)
command.assert_not_called()
with patch.dict(os.environ, REGISTRY_AUTH_FILE="auth.json"), patch.object(publish, "image_digest", side_effect=[None, "sha256:" + "a" * 64]), patch.object(subprocess, "run") as command:
publish.copy_image(Path("image.tar"), "registry/image:1.2.3", "sha256:" + "a" * 64)
self.assertIn("--preserve-digests", command.call_args.args[0])
self.assertIn("--all", command.call_args.args[0])
def test_newer_channel_cannot_be_downgraded_after_old_job_retry(self):
with patch.object(publish, "publish_version") as versions, patch.object(publish, "api", return_value={"tag_name": "1.2.4"}) as api, patch.object(publish, "images") as images, patch.object(publish, "helm") as helm:
publish.channels(inputs(), {}, Path("candidate"), "a" * 64)
versions.assert_called_once()
images.assert_not_called()
helm.assert_called_once()
self.assertTrue(all(call.args[1:] == () for call in api.call_args_list))
def test_prerelease_never_changes_stable_pointers(self):
value = inputs(); value["version"] = "1.3.0-rc.1"
with patch.object(publish, "publish_version"), patch.object(publish, "api") as api, patch.object(publish, "images") as images, patch.object(publish, "helm") as helm:
publish.channels(value, {}, Path("candidate"), "a" * 64)
api.assert_not_called(); images.assert_not_called(); helm.assert_called_once()
def test_interrupted_version_publish_never_advances_channels(self):
with patch.object(publish, "publish_version", side_effect=ValueError("registry unavailable")), patch.object(publish, "api") as api:
with self.assertRaises(ValueError):
publish.channels(inputs(), {}, Path("candidate"), "a" * 64)
api.assert_not_called()
def test_incomplete_helm_publication_does_not_advance_stable_channels(self):
with patch.object(publish, "publish_version"), patch.object(publish, "helm", side_effect=ValueError("chart repository unavailable")), patch.object(publish, "api") as api, patch.object(publish, "images") as images:
with self.assertRaises(ValueError):
publish.channels(inputs(), {}, Path("candidate"), "a" * 64)
api.assert_not_called()
images.assert_not_called()
def test_r2_retry_checks_bytes_and_uses_conditional_creation(self):
with tempfile.TemporaryDirectory() as temp, patch.dict(os.environ, R2_BUCKET="releases"):
path = Path(temp) / "asset.zip"; path.write_bytes(b"accepted")
objects = {}
calls = []
def aws(*args, **kwargs):
calls.append(args)
if args[:2] == ("s3api", "head-object"):
exists = args[args.index("--key") + 1] in objects
return subprocess.CompletedProcess(args, 0 if exists else 1, "{}", "" if exists else "(404)")
if args[:2] == ("s3api", "put-object"):
self.assertIn("--if-none-match", args)
objects[args[args.index("--key") + 1]] = Path(args[args.index("--body") + 1]).read_bytes()
else:
Path(args[3]).write_bytes(objects[args[2].split("/", 3)[3]])
return subprocess.CompletedProcess(args, 0, "{}", "")
with patch.object(publish, "aws", side_effect=aws):
publish.r2_file(path, "version/asset.zip")
publish.r2_file(path, "version/asset.zip")
self.assertEqual(sum(call[:2] == ("s3api", "put-object") for call in calls), 1)
objects["version/asset.zip"] = b"changed"
with self.assertRaisesRegex(ValueError, "never overwrite"):
publish.r2_file(path, "version/asset.zip")
class WorkflowContractTests(unittest.TestCase):
def test_nightly_avoids_desktop_and_manual_builds_keep_platform_selection(self):
value = workflow("build.yml")
selection = value["jobs"]["prepare-platform-matrix"]["steps"][-1]
self.assertEqual(selection["env"]["RAW_PLATFORMS"], "${{ inputs.platforms || 'linux' }}")
self.assertEqual(value["on"]["workflow_dispatch"]["inputs"]["platforms"]["default"], "all")
def test_main_push_cannot_cancel_the_scheduled_oidc_gate(self):
value = workflow("oidc-keycloak.yml")
self.assertIn("github.event_name", value["concurrency"]["group"])
self.assertIn("!= 'schedule'", value["concurrency"]["cancel-in-progress"])
def test_linux_is_the_only_initial_build(self):
value = workflow("release-candidate.yml")
self.assertEqual(set(value["on"]), {"workflow_dispatch"})
self.assertEqual(value["jobs"]["linux"]["with"]["platforms"], "linux")
self.assertNotIn("desktop", value["jobs"])
self.assertEqual(set(value["jobs"]["seal-linux"]["needs"]), {"freeze", "linux", "upgrade"})
def test_desktop_and_deliveries_wait_for_linux_acceptance(self):
value = workflow("release-complete.yml")
self.assertEqual(value["jobs"]["acceptance"]["environment"], "release-linux-acceptance")
for name in ("desktop", "packages", "images", "chart"):
self.assertEqual(value["jobs"][name]["needs"], "acceptance")
self.assertNotIn("secrets", value["jobs"][name])
self.assertEqual(set(value["jobs"]["seal"]["needs"]), {"acceptance", "desktop", "packages", "images", "chart"})
def test_only_publisher_has_release_writes_and_never_builds(self):
value = workflow("release-publish.yml")
self.assertEqual(set(value["on"]), {"workflow_dispatch"})
self.assertEqual(value["jobs"]["version"]["environment"], "release-publication")
self.assertEqual(value["jobs"]["channels"]["needs"], ["verify", "version"])
self.assertEqual(value["jobs"]["channels"]["concurrency"]["group"], "rustfs-release-channels")
for job in ("version", "channels"):
self.assertFalse(value["jobs"][job]["concurrency"]["cancel-in-progress"])
for name in ("build.yml", "package.yml", "docker.yml", "helm-package.yml"):
parsed = workflow(name)
self.assertEqual(parsed["permissions"]["actions"], "read")
self.assertNotIn("push", parsed["on"])
self.assertNotIn("workflow_run", parsed["on"])
self.assertNotIn("contents: write", (ROOT / ".github/workflows" / name).read_text())
source = (ROOT / "scripts/release/publish.py").read_text()
for forbidden in ("cargo build", "build-push", "helm package", "--clobber", "force=true"):
self.assertNotIn(forbidden, source.replace("# No --clobber:", "# No replacement:"))
def test_stage_failures_do_not_have_a_continue_on_error_bypass(self):
for name in ("release-candidate.yml", "release-complete.yml", "release-publish.yml", "docker.yml", "package.yml", "helm-package.yml"):
value = workflow(name)
for job in value["jobs"].values():
self.assertNotIn("continue-on-error", job)
for step in job.get("steps", []):
self.assertNotIn("continue-on-error", step)
class ImageWorkflowTests(unittest.TestCase):
def test_substituting_a_single_platform_scan_archive_is_rejected(self):
import deliver
with tempfile.TemporaryDirectory() as temp:
output = Path(temp) / "delivery"
scans = Path(str(output) + "-scan")
output.mkdir(); scans.mkdir()
value = inputs()
record = {"inputs_sha256": c.digest(value), "scan_inputs": {}}
for arch in ("amd64", "arm64"):
path = scans / f"image-musl-{arch}.tar"
path.write_bytes(arch.encode())
record["scan_inputs"][arch] = c.file_digest(path)
c.write_json(output / f"scan-musl-{arch}.sarif", {"version": "2.1.0", "runs": [{}]})
c.write_json(output / "image-musl-smoke.json", record)
deliver.image_record(value, "musl", output)
self.assertEqual(c.read_json(output / "image-musl.json")["scan"], "completed")
(scans / "image-musl-arm64.tar").write_bytes(b"another architecture")
with self.assertRaisesRegex(ValueError, "scan input changed"):
deliver.image_record(value, "musl", output)
def test_both_architectures_scan_before_seal_without_registry_credentials(self):
value = workflow("docker.yml")
self.assertEqual(value["permissions"], {"contents": "read", "actions": "read"})
steps = value["jobs"]["build-docker"]["steps"]
build = next(s for s in steps if "docker/build-push-action@" in s.get("uses", ""))
self.assertFalse(build["with"]["push"])
self.assertIn("type=oci", build["with"]["outputs"])
scans = [s for s in steps if "aquasecurity/trivy-action@" in s.get("uses", "")]
self.assertEqual({s["env"]["TRIVY_PLATFORM"] for s in scans}, {"linux/amd64", "linux/arm64"})
for scan in scans:
self.assertEqual(scan["with"]["exit-code"], "0")
self.assertIn("input", scan["with"])
architecture = scan["env"]["TRIVY_PLATFORM"].split("/")[-1]
self.assertTrue(scan["with"]["input"].endswith(f"-{architecture}.tar"))
self.assertLess(steps.index(scan), next(i for i, s in enumerate(steps) if s.get("name") == "Record verified OCI identity"))
class HelmWorkflowTests(unittest.TestCase):
def test_installation_versions_match_candidate_chart_versions(self):
subprocess.run(["bash", "scripts/test_helm_chart_version.sh"], cwd=ROOT,
check=True, capture_output=True, text=True, timeout=15)
def test_chart_is_sealed_before_publish_and_does_not_push(self):
value = workflow("helm-package.yml")
self.assertEqual(set(value["on"]), {"workflow_call"})
source = (ROOT / ".github/workflows/helm-package.yml").read_text()
self.assertIn("scripts/test_helm_templates.sh", source)
self.assertIn("('version', 'appVersion')", source)
self.assertNotIn("git push", source)
self.assertIn("inputs_sha256", source)
if __name__ == "__main__":
unittest.main()
+31 -63
View File
@@ -111,7 +111,22 @@ class GateFixtureTests(unittest.TestCase):
self.assertEqual(summary["findings"], [])
expansion = json.loads((self.output / "pass" / "plan-expansion.json").read_text())
self.assertEqual(len(expansion["required_cases"]), 5)
promote = run_promote(self.output / "pass")
import hashlib
artifact = self.output / ARTIFACT_NAME
artifact.write_bytes(b"release gate contract payload\n")
candidate = json.loads(CANDIDATE.read_text())
old_sha = candidate["deliverables"][0]["sha256"]
new_sha = hashlib.sha256(artifact.read_bytes()).hexdigest()
candidate["deliverables"][0].update(sha256=new_sha, size=artifact.stat().st_size)
candidate_path = self.output / "candidate.json"
candidate_path.write_text(json.dumps(candidate))
results = self.output / "actual-results"
results.mkdir()
for path in (FIXTURES / "pass").glob("*.json"):
(results / path.name).write_text(path.read_text().replace(old_sha, new_sha))
result = run_validate(results, self.output / "actual-certification", candidate=candidate_path)
self.assertEqual(result.returncode, 0, result.stderr)
promote = run_promote(self.output / "actual-certification", candidate=candidate_path, artifact=artifact)
self.assertEqual(promote.returncode, 0, promote.stderr)
def test_cross_platform_same_case_and_split_shas_accepted(self):
@@ -306,70 +321,23 @@ class PlanContractTests(unittest.TestCase):
class PublicationWiringTests(unittest.TestCase):
"""Static checks that the real publication exits consume the gate."""
"""The publisher re-runs full certification over trusted downloaded receipts."""
def read(self, relative):
return (ROOT / relative).read_text()
def test_single_publisher_revalidates_the_complete_seal(self):
source = (ROOT / "scripts/release/publish.py").read_text()
self.assertIn('pipeline.certify(inputs, directory,', source)
self.assertIn('"full")', source)
self.assertIn('summary["publish_allowed"]', source)
self.assertIn('pipeline.verify_deliveries(inputs, directory)', source)
workflow = (ROOT / ".github/workflows/release-publish.yml").read_text()
self.assertIn('environment: release-publication', workflow)
self.assertIn('needs: [verify, version]', workflow)
def job_block(self, workflow, job):
match = re.search(
rf"^ {re.escape(job)}:\n(.*?)(?=^ [A-Za-z#])",
workflow,
re.S | re.M,
)
self.assertIsNotNone(match, f"job {job} not found")
return match.group(1)
def test_build_publish_requires_certification(self):
workflow = self.read(".github/workflows/build.yml")
self.assertIn("\n release-certification:", workflow)
publish_block = self.job_block(workflow, "publish-release")
self.assertIn("release-certification", publish_block)
self.assertIn("release_gate.py validate", workflow)
self.assertIn("release_gate.py verify-promotion", workflow)
def test_build_time_official_r2_writes_are_staged(self):
workflow = self.read(".github/workflows/build.yml")
self.assertIn("artifacts/rustfs/staging/", workflow)
release_writes = [
line for line in workflow.splitlines()
if "artifacts/rustfs/release/" in line
]
# Only the certification-gated promotion job may write the official path.
self.assertTrue(release_writes, "the promotion job must write the official R2 path")
promotion_block = self.job_block(workflow, "promote-release-artifacts")
self.assertIn("verify-promotion", promotion_block)
self.assertIn("artifacts/rustfs/release/", promotion_block)
build_block = self.job_block(workflow, "build-rustfs")
self.assertNotIn('R2_PATH="s3://${R2_BUCKET}/artifacts/rustfs/release/"', build_block)
def test_downstream_workflows_verify_certification(self):
docker = self.read(".github/workflows/docker.yml")
self.assertIn("\n release-certification:", docker)
build_docker_block = self.job_block(docker, "build-docker")
self.assertIn("release-certification", build_docker_block)
self.assertIn("verify-promotion", docker)
package = self.read(".github/workflows/package.yml")
self.assertIn("\n release-certification:", package)
package_block = self.job_block(package, "package")
self.assertIn("release-certification", package_block)
helm = self.read(".github/workflows/helm-package.yml")
self.assertIn("\n release-certification:", helm)
helm_block = self.job_block(helm, "build-helm-package")
self.assertIn("release-certification", helm_block)
def test_manual_downstream_dispatch_without_certification_fails(self):
# Every downstream certification job must fail closed when the trusted
# certification artifact is absent (e.g. a manual dispatch without one).
for relative in (
".github/workflows/docker.yml",
".github/workflows/package.yml",
".github/workflows/helm-package.yml",
):
workflow = self.read(relative)
certification_block = self.job_block(workflow, "release-certification")
self.assertIn("fail", certification_block.lower(), relative)
self.assertIn("release-certification", certification_block, relative)
def test_builders_do_not_have_publication_permissions(self):
for name in ("build.yml", "docker.yml", "package.yml", "helm-package.yml"):
source = (ROOT / ".github/workflows" / name).read_text()
self.assertNotIn("contents: write", source)
self.assertNotIn("packages: write", source)
if __name__ == "__main__":
+230
View File
@@ -0,0 +1,230 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
"""Run release checks against the archived executable, never a rebuilt server."""
from __future__ import annotations
import argparse
from datetime import datetime, timezone
import json
import hashlib
import hmac
import os
from pathlib import Path
import re
import socket
import subprocess
import tempfile
import time
import urllib.request
import zipfile
import candidate as c
TRIPLES = {
"linux-x86_64-musl": "x86_64-unknown-linux-musl", "linux-aarch64-musl": "aarch64-unknown-linux-musl",
"linux-x86_64-gnu": "x86_64-unknown-linux-gnu", "linux-aarch64-gnu": "aarch64-unknown-linux-gnu",
"macos-aarch64": "aarch64-apple-darwin", "windows-x86_64": "x86_64-pc-windows-msvc",
}
UPGRADES = {
"direct-upgrade": "direct_upgrade_from_previous_release_preserves_object_contracts",
"mixed-version-upgrade": "rolling_upgrade_from_previous_release_preserves_mixed_version_contracts",
"bucket-config-upgrade": "direct_upgrade_from_previous_release_preserves_bucket_configuration",
"bucket-config-rollback": "rollback_to_previous_release_reads_current_bucket_metadata",
"multipart-layout-upgrade": "direct_upgrade_from_previous_release_preserves_multipart_layouts",
"multipart-layout-baseline": "previous_release_baseline_replicates_multipart_layouts",
}
def now():
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def envelope(inputs, record, lane, case, logs, started, output):
"""Only called after the real check succeeds; logs remain part of the seal."""
target = record["target"].split("-")
platform = dict(zip(("os", "arch", "abi"), target))
platform.setdefault("abi", "msvc" if platform["os"] == "windows" else "none")
workflow = os.environ["GITHUB_WORKFLOW_REF"].split("@", 1)[0].rsplit("/", 1)[-1]
c.require(workflow in ("release-candidate.yml", "release-complete.yml"), "untrusted verifier workflow")
value = {
"schema_version": 1, "evidence_profile": "release-candidate", "tier": "release",
"run_id": inputs["candidate_id"], "run_attempt": 1, "plan_sha256": inputs["plan_sha256"],
"lane": lane, "case_id": case, "product_commit": inputs["source_sha"],
"harness_commit": inputs["workflow_sha"], "target_fingerprint": record["binary_sha256"],
"platform": platform, "started_at": started, "finished_at": now(), "state": "pass",
"reason_code": "CHECK_PASSED", "reason_class": "NOT_APPLICABLE",
"artifact": {"archive_sha256": record["sha256"], "binary_sha256": record["binary_sha256"]},
"evidence": [{"path": p.name, "sha256": c.file_digest(p), "size": p.stat().st_size} for p in logs],
"workflow": {"repository": c.REPOSITORY, "workflow_id": workflow, "run_id": os.environ["GITHUB_RUN_ID"],
"run_attempt": int(os.environ["GITHUB_RUN_ATTEMPT"]), "workflow_sha": os.environ["GITHUB_SHA"],
"status": "completed"},
}
c.write_json(output / f"{lane}-{record['target']}-{case}.json", value)
def unpack(archive, target, inputs, directory):
record = c.archive_record(archive, target, inputs)
name = "rustfs.exe" if target.startswith("windows") else "rustfs"
with zipfile.ZipFile(archive) as package:
package.extract(name, directory)
binary = (Path(directory) / name).resolve()
binary.chmod(0o755)
return binary, record
def port():
with socket.socket() as sock:
sock.bind(("127.0.0.1", 0))
return sock.getsockname()[1]
def s3_request(opener, endpoint, method, path, access, secret, body=b""):
timestamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
date = timestamp[:8]
host = endpoint.removeprefix("http://")
payload = hashlib.sha256(body).hexdigest()
headers = f"host:{host}\nx-amz-content-sha256:{payload}\nx-amz-date:{timestamp}\n"
signed = "host;x-amz-content-sha256;x-amz-date"
canonical = f"{method}\n{path}\n\n{headers}\n{signed}\n{payload}"
scope = f"{date}/us-east-1/s3/aws4_request"
to_sign = f"AWS4-HMAC-SHA256\n{timestamp}\n{scope}\n{hashlib.sha256(canonical.encode()).hexdigest()}"
key = ("AWS4" + secret).encode()
for part in (date, "us-east-1", "s3", "aws4_request"):
key = hmac.digest(key, part.encode(), "sha256")
signature = hmac.new(key, to_sign.encode(), hashlib.sha256).hexdigest()
request = urllib.request.Request(endpoint + path, data=body if method == "PUT" else None, method=method,
headers={"Host": host, "X-Amz-Content-Sha256": payload, "X-Amz-Date": timestamp,
"Authorization": f"AWS4-HMAC-SHA256 Credential={access}/{scope}, SignedHeaders={signed}, Signature={signature}"})
with opener.open(request, timeout=10) as response:
c.require(response.status in (200, 204), "S3 smoke request failed")
return response.read(1024 * 1024)
def native(inputs, archive, target, output):
started = now()
output.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory() as temp:
binary, record = unpack(archive, target, inputs, temp)
version = subprocess.check_output([str(binary), "--version"], text=True, timeout=60)
c.require(version.splitlines()[0] == f"rustfs {inputs['version']}", "binary displays a different product version")
for label, expected in (("build target", TRIPLES[target]), ("git commit", inputs["source_sha"])):
c.require(re.search(rf"(?m)^{label}\s*:\s*{re.escape(expected)}\s*$", version), f"binary {label} mismatch")
c.require(re.search(rf"(?m)^rust version\s*:\s*(?:rustc )?{re.escape(inputs['toolchain'])}(?:\s|$)", version), "binary toolchain mismatch")
log = output / f"identity-{target}.log"
log.write_text(version, encoding="utf-8")
envelope(inputs, record, "identity", "artifact-identity", [log], started, output)
data = Path(temp) / "data"
data.mkdir()
api, console = port(), port()
while console == api:
console = port()
log = output / f"startup-{target}.log"
access, secret = "release-" + os.urandom(8).hex(), os.urandom(24).hex()
env = dict(os.environ, NO_PROXY="127.0.0.1,localhost,::1", no_proxy="127.0.0.1,localhost,::1")
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
with log.open("w", encoding="utf-8") as stream:
process = subprocess.Popen([str(binary), "server", "--address", f"127.0.0.1:{api}", "--console-enable",
"--console-address", f"127.0.0.1:{console}", "--access-key", access,
"--secret-key", secret, str(data)],
stdout=stream, stderr=subprocess.STDOUT, env=env)
try:
deadline = time.monotonic() + 60
while time.monotonic() < deadline:
c.require(process.poll() is None, "candidate exited before becoming ready")
try:
with opener.open(f"http://127.0.0.1:{api}/health/ready", timeout=2) as response:
c.require(response.status == 200, "S3 listener is not ready")
with opener.open(f"http://127.0.0.1:{console}/rustfs/console/", timeout=2) as response:
c.require(response.status == 200 and "text/html" in response.headers.get("Content-Type", ""), "Console did not return HTML")
break
except OSError:
time.sleep(0.5)
else:
raise ValueError("candidate did not become ready within 60 seconds")
bucket = "/release-" + os.urandom(8).hex()
endpoint = f"http://127.0.0.1:{api}"
content = os.urandom(1024)
s3_request(opener, endpoint, "PUT", bucket, access, secret)
s3_request(opener, endpoint, "PUT", bucket + "/roundtrip", access, secret, content)
c.require(s3_request(opener, endpoint, "GET", bucket + "/roundtrip", access, secret) == content, "S3 round-trip bytes differ")
s3_request(opener, endpoint, "DELETE", bucket + "/roundtrip", access, secret)
s3_request(opener, endpoint, "DELETE", bucket, access, secret)
c.require(process.poll() is None, "candidate exited during startup verification")
finally:
process.terminate()
try:
process.wait(timeout=15)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=15)
c.require(c.file_digest(binary) == record["binary_sha256"], "server bytes changed during native verification")
with log.open("a", encoding="utf-8") as stream:
stream.write("\nConsole: HTTP 200 text/html; S3: ready, bucket/object CRUD and byte-identical round trip\n")
envelope(inputs, record, "identity", "startup", [log], started, output)
def upgrade(inputs, archive, output):
output.mkdir(parents=True, exist_ok=True)
plan = c.read_json(c.ROOT / ".config/release-validation-plan.json")
c.require(c.digest(plan) == inputs["plan_sha256"], "upgrade plan changed")
with tempfile.TemporaryDirectory() as temp:
root = Path(temp)
binary, record = unpack(archive, "linux-x86_64-gnu", inputs, root)
metadata = binary.stat()
receipt = root / "receipt.json"
receipt.write_text(json.dumps({"schema": 1, "workspace": str(c.ROOT.resolve()), "binary": str(binary),
"size": metadata.st_size, "modified_ns": metadata.st_mtime_ns, "features": ["pyroscope"]}))
env = dict(os.environ, CARGO_BIN_EXE_rustfs=str(binary), RUSTFS_E2E_BINARY_RECEIPT=str(receipt),
RUSTFS_BUILD_FEATURES="pyroscope", NO_PROXY="127.0.0.1,localhost,::1", no_proxy="127.0.0.1,localhost,::1",
HTTP_PROXY="", HTTPS_PROXY="", ALL_PROXY="", http_proxy="", https_proxy="", all_proxy="")
logs = {case: [] for case in UPGRADES}
started = now()
sources = list(plan["upgrade_sources"]["sources"])
previous = inputs["previous"]
if previous["version"] not in {s["version"] for s in sources}:
sources.append({"version": previous["version"], "artifact_name": previous["name"], "artifact_sha256": previous["sha256"]})
for source in sources:
directory = root / source["version"]
directory.mkdir()
package = directory / source["artifact_name"]
url = f"https://github.com/{c.REPOSITORY}/releases/download/{source['version']}/{source['artifact_name']}"
subprocess.run(["curl", "--fail", "--location", "--retry", "3", "--output", str(package), url], check=True, timeout=600)
c.require(c.file_digest(package) == source["artifact_sha256"], "previous release digest mismatch")
with zipfile.ZipFile(package) as previous:
c.require(previous.namelist().count("rustfs") == 1, "ambiguous previous server")
previous.extract("rustfs", directory)
(directory / "rustfs").chmod(0o755)
env["RUSTFS_UPGRADE_SOURCE_BINARY"] = str(directory / "rustfs")
for case, test in UPGRADES.items():
log = output / f"upgrade-{source['version']}-{case}.log"
env["RUSTFS_E2E_LOG_DIR"] = str(root / f"logs-{source['version']}-{case}")
with log.open("w", encoding="utf-8") as stream:
subprocess.run(["cargo", "test", "--locked", "-p", "e2e_test", f"upgrade_compatibility_test::{test}",
"--", "--ignored", "--exact", "--nocapture"], cwd=c.ROOT, env=env,
stdout=stream, stderr=subprocess.STDOUT, check=True, timeout=1800)
c.require(re.search(r"test result: ok\. 1 passed; 0 failed;", log.read_text()), "upgrade command did not run exactly one passing test")
c.require(c.file_digest(binary) == record["binary_sha256"], "candidate changed during upgrade tests")
logs[case].append(log)
for case, files in logs.items():
envelope(inputs, record, "upgrade", case, files, started, output)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("mode", choices=("native", "upgrade"))
parser.add_argument("--inputs", required=True)
parser.add_argument("--archive", required=True)
parser.add_argument("--target", choices=c.TARGETS)
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
inputs = c.validate_inputs(c.read_json(args.inputs))
if args.mode == "native":
c.require(args.target is not None, "native verification needs a platform")
native(inputs, args.archive, args.target, args.output)
else:
upgrade(inputs, args.archive, args.output)
if __name__ == "__main__":
main()
+329
View File
@@ -0,0 +1,329 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Plan and judge the s3-tests reference parity lane.
Subcommands:
targets validate the target list and print it as a JSON matrix
export-env validate one target's extra environment and print KEY=VALUE lines
failed print the failed test names of one run, in implemented_tests.txt form
compare compare every candidate against the reference, per pytest node id
`compare` reads <reports>/<target>/selected-nodeids.txt, junit.xml and the
optional rerun-junit.xml of every target. Each node id is reduced to pass /
fail / skip / missing; a case that failed and then passed its single rerun is a
pass and is listed as flaky. A tolerated entry must match a real difference.
Exit codes: 0 no untolerated difference and no stale entry; 1 a difference, a
stale tolerated entry or an incomplete run; 2 missing or malformed input.
"""
import argparse
import json
from pathlib import Path
import re
import sys
import tomllib
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
import junit_diff # noqa: E402 (sibling script, not a package)
PASS, FAIL, SKIP, MISSING = junit_diff.PASS, junit_diff.FAIL, junit_diff.SKIP, junit_diff.MISSING
NAME = re.compile(r"[a-z][a-z0-9-]{0,39}")
ENV_KEY = re.compile(r"RUSTFS_[A-Z0-9_]+")
ENV_VALUE = re.compile(r"[A-Za-z0-9._,:/=+-]*")
ISSUE = re.compile(r"https://github\.com/rustfs/[A-Za-z0-9_.-]+/issues/[1-9][0-9]*")
NODEID = re.compile(r"[A-Za-z0-9_/]+\.py(::[A-Za-z0-9_]+)+(\[[^\]\s]*\])?")
TEST_FILE_PREFIX = "s3tests/functional/test_s3.py::"
BINARIES = ("reference", "candidate")
class InputError(Exception):
"""Input that cannot be trusted; maps to exit code 2."""
# --- target list -----------------------------------------------------------
def load_targets(path):
try:
data = tomllib.loads(Path(path).read_text(encoding="utf-8"))
except FileNotFoundError as exc:
raise InputError(f"{path}: target list is missing") from exc
except (OSError, UnicodeDecodeError, tomllib.TOMLDecodeError) as exc:
raise InputError(f"{path}: cannot read target list: {exc}") from exc
if data.get("schema") != 1 or type(data.get("schema")) is not int or set(data) != {"schema", "targets"}:
raise InputError(f"{path}: unsupported target list schema")
return validate_targets(data["targets"], str(path))
def validate_targets(targets, where="targets"):
if not isinstance(targets, list) or len(targets) < 2:
raise InputError(f"{where}: need one reference and at least one candidate")
result, names = [], set()
for item in targets:
if not isinstance(item, dict) or not set(item) <= {"name", "binary", "env"}:
raise InputError(f"{where}: invalid target entry {item!r}")
name, binary, env = item.get("name"), item.get("binary"), item.get("env", {})
if not isinstance(name, str) or not NAME.fullmatch(name) or name in names:
raise InputError(f"{where}: invalid or duplicate target name {name!r}")
if binary not in BINARIES:
raise InputError(f"{where}: target {name} has unknown binary {binary!r}")
validate_env(env, f"{where}: target {name}")
names.add(name)
result.append({"name": name, "binary": binary, "env": dict(env)})
if [t["binary"] for t in result].count("reference") != 1:
raise InputError(f"{where}: exactly one target must use the reference binary")
return result
def validate_env(env, where):
if not isinstance(env, dict):
raise InputError(f"{where}: env must be a table")
for key, value in env.items():
if not isinstance(key, str) or not ENV_KEY.fullmatch(key):
raise InputError(f"{where}: env key {key!r} must match RUSTFS_[A-Z0-9_]+")
if not isinstance(value, str) or not ENV_VALUE.fullmatch(value):
raise InputError(f"{where}: env value for {key} has unsupported characters")
return env
# --- node ids and reports --------------------------------------------------
def nodeid_to_case(nodeid):
"""Map a pytest node id to the classname::name id that junit_diff uses."""
if not NODEID.fullmatch(nodeid):
raise InputError(f"invalid pytest node id {nodeid!r}")
path, *parts = nodeid.split("::")
module = path[: -len(".py")].replace("/", ".")
classname = ".".join([module, *parts[:-1]])
return junit_diff.case_id(classname, parts[-1])
def read_selected(path):
try:
lines = Path(path).read_text(encoding="utf-8").splitlines()
except FileNotFoundError as exc:
raise InputError(f"{path}: selected node id list is missing") from exc
except (OSError, UnicodeDecodeError) as exc:
raise InputError(f"{path}: cannot read selected node ids: {exc}") from exc
nodeids = [line.strip() for line in lines if line.strip()]
if not nodeids:
raise InputError(f"{path}: selected node id list is empty")
if len(set(nodeids)) != len(nodeids):
raise InputError(f"{path}: duplicate selected node id")
by_case = {}
for nodeid in nodeids:
case = nodeid_to_case(nodeid)
if case in by_case:
raise InputError(f"{path}: node ids {by_case[case]} and {nodeid} collide")
by_case[case] = nodeid
return nodeids, by_case
def target_states(directory):
"""Return (nodeids, {nodeid: state}, [flaky nodeids]) for one target run."""
directory = Path(directory)
if not directory.is_dir():
raise InputError(f"{directory}: report directory is missing")
nodeids, by_case = read_selected(directory / "selected-nodeids.txt")
try:
first, _ = junit_diff.parse_report(str(directory / "junit.xml"))
except junit_diff.InputError as exc:
raise InputError(str(exc)) from exc
unexpected = sorted(set(first) - set(by_case))
if unexpected:
raise InputError(f"{directory}: junit.xml has unselected cases: {', '.join(unexpected[:5])}")
states = {by_case[case]: state for case, state in first.items()}
flaky = []
rerun_path = directory / "rerun-junit.xml"
if rerun_path.exists():
try:
rerun, _ = junit_diff.parse_report(str(rerun_path))
except junit_diff.InputError as exc:
raise InputError(str(exc)) from exc
for case, state in sorted(rerun.items()):
nodeid = by_case.get(case)
if nodeid is None or states.get(nodeid) != FAIL:
raise InputError(f"{rerun_path}: {case} was rerun without failing first")
if state == PASS:
flaky.append(nodeid)
states[nodeid] = PASS if state == PASS else FAIL
for nodeid in nodeids:
states.setdefault(nodeid, MISSING)
return nodeids, states, flaky
def failed_names(directory):
_, states, _ = target_states(directory)
names = []
for nodeid, state in states.items():
if state == FAIL:
if not nodeid.startswith(TEST_FILE_PREFIX):
raise InputError(f"cannot express {nodeid} as an implemented_tests.txt entry")
names.append(nodeid[len(TEST_FILE_PREFIX):])
return sorted(names)
# --- tolerated differences ---------------------------------------------------
def read_tolerated(path, candidates):
try:
lines = Path(path).read_text(encoding="utf-8").splitlines()
except FileNotFoundError as exc:
raise InputError(f"{path}: tolerated-difference file is missing") from exc
except (OSError, UnicodeDecodeError) as exc:
raise InputError(f"{path}: cannot read tolerated differences: {exc}") from exc
entries = {}
for number, line in enumerate(lines, 1):
if not line.strip() or line.lstrip().startswith("#"):
continue
fields = [part.strip() for part in line.split(" | ")]
if len(fields) != 4 or not all(fields):
raise InputError(f"{path}:{number}: expected '<target> | <node id> | <issue URL> | <reason>'")
target, nodeid, issue, _reason = fields
if target not in candidates:
raise InputError(f"{path}:{number}: {target!r} is not a candidate target")
nodeid_to_case(nodeid)
if not ISSUE.fullmatch(issue):
raise InputError(f"{path}:{number}: tracking issue must be a https://github.com/rustfs/<repo>/issues/<n> URL")
if (target, nodeid) in entries:
raise InputError(f"{path}:{number}: duplicate entry for {target} {nodeid}")
entries[(target, nodeid)] = issue
return entries
# --- comparison ----------------------------------------------------------------
def count(states):
result = {PASS: 0, FAIL: 0, SKIP: 0, MISSING: 0}
for state in states.values():
result[state] += 1
return result
def compare(targets, reports, tolerated_path):
reference = next(t["name"] for t in targets if t["binary"] == "reference")
candidates = [t["name"] for t in targets if t["binary"] == "candidate"]
tolerated = read_tolerated(tolerated_path, set(candidates))
runs = {t["name"]: target_states(Path(reports) / t["name"]) for t in targets}
selected = runs[reference][0]
for name, (nodeids, _, _) in runs.items():
if nodeids != selected:
raise InputError(f"{name}: selected node ids differ from the {reference} run")
ref_states = runs[reference][1]
differences = []
for name in candidates:
states = runs[name][1]
for nodeid in selected:
if ref_states[nodeid] != states[nodeid]:
issue = tolerated.get((name, nodeid))
differences.append({"target": name, "nodeid": nodeid, "reference": ref_states[nodeid],
"candidate": states[nodeid], "tolerated": issue})
seen = {(d["target"], d["nodeid"]) for d in differences}
stale = [{"target": t, "nodeid": n, "issue": i} for (t, n), i in sorted(tolerated.items()) if (t, n) not in seen]
incomplete = {name: [n for n in selected if run[1][n] == MISSING] for name, run in runs.items()}
return {
"reference": reference,
"candidates": candidates,
"selected": len(selected),
"counts": {name: count(run[1]) for name, run in runs.items()},
"flaky": {name: run[2] for name, run in runs.items()},
"differences": differences,
"stale_tolerated": stale,
"incomplete": {name: ids for name, ids in incomplete.items() if ids},
}
def failed(summary):
return bool(summary["stale_tolerated"] or summary["incomplete"]
or any(not d["tolerated"] for d in summary["differences"]))
def render(summary):
out = ["## s3-tests reference parity", ""]
out.append(f"Reference target `{summary['reference']}`; {summary['selected']} selected node ids per target.")
out += ["", "| Target | pass | fail | skip | missing | flaky (passed on rerun) |", "|---|---:|---:|---:|---:|---:|"]
for name, c in summary["counts"].items():
out.append(f"| `{name}` | {c[PASS]} | {c[FAIL]} | {c[SKIP]} | {c[MISSING]} | {len(summary['flaky'][name])} |")
diffs = summary["differences"]
out += ["", f"### Differences ({len(diffs)})", ""]
if diffs:
out += ["| Target | Node id | reference | candidate | Tolerated |", "|---|---|---|---|---|"]
for d in diffs:
out.append(f"| `{d['target']}` | `{d['nodeid']}` | {d['reference']} | {d['candidate']} | {d['tolerated'] or 'no'} |")
else:
out.append("No differences.")
if summary["stale_tolerated"]:
out += ["", "### Stale tolerated entries (no matching difference; remove them)", ""]
out += [f"- `{s['target']}` `{s['nodeid']}` ({s['issue']})" for s in summary["stale_tolerated"]]
if summary["incomplete"]:
out += ["", "### Incomplete runs (selected node ids without a result)", ""]
out += [f"- `{name}`: {len(ids)} missing, first `{ids[0]}`" for name, ids in summary["incomplete"].items()]
flaky = [(name, ids) for name, ids in summary["flaky"].items() if ids]
if flaky:
out += ["", "### Flaky (failed, then passed the single rerun; not a difference)", ""]
out += [f"- `{name}`: " + ", ".join(f"`{i}`" for i in ids) for name, ids in flaky]
out += ["", "Result: **" + ("FAIL" if failed(summary) else "PASS") + "**", ""]
return "\n".join(out)
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = parser.add_subparsers(dest="mode", required=True)
tgt = sub.add_parser("targets")
tgt.add_argument("--config", required=True)
env = sub.add_parser("export-env")
env.add_argument("--target-json", required=True)
fail = sub.add_parser("failed")
fail.add_argument("--report", required=True, help="one target's report directory")
cmp_ = sub.add_parser("compare")
cmp_.add_argument("--targets-json", required=True)
cmp_.add_argument("--reports", required=True)
cmp_.add_argument("--tolerated", required=True)
cmp_.add_argument("--markdown", help="write the summary table here (appended)")
cmp_.add_argument("--json", help="write the machine-readable summary here")
args = parser.parse_args(argv)
try:
if args.mode == "targets":
print(json.dumps(load_targets(args.config), separators=(",", ":")))
elif args.mode == "export-env":
target = json.loads(args.target_json)
if not isinstance(target, dict):
raise InputError("target must be a JSON object")
for key, value in sorted(validate_env(target.get("env", {}), "target").items()):
print(f"{key}={value}")
elif args.mode == "failed":
for name in failed_names(args.report):
print(name)
else:
targets = validate_targets(json.loads(args.targets_json))
summary = compare(targets, args.reports, args.tolerated)
text = render(summary)
print(text)
if args.markdown:
with open(args.markdown, "a", encoding="utf-8") as handle:
handle.write(text + "\n")
if args.json:
with open(args.json, "w", encoding="utf-8") as handle:
json.dump(summary, handle, indent=2, sort_keys=True)
handle.write("\n")
return 1 if failed(summary) else 0
except (InputError, json.JSONDecodeError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 2
return 0
if __name__ == "__main__":
sys.exit(main())
+332
View File
@@ -0,0 +1,332 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Contract tests for reference_parity.py: target list, rerun merge, tolerated entries and exit codes."""
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
SCRIPT = Path(__file__).resolve().parent / "reference_parity.py"
REPO = Path(__file__).resolve().parents[2]
PREFIX = "s3tests/functional/test_s3.py::"
CLASSNAME = "s3tests.functional.test_s3"
TARGETS = [{"name": "reference", "binary": "reference", "env": {}},
{"name": "candidate-legacy", "binary": "candidate", "env": {}}]
ISSUE = "https://github.com/rustfs/rustfs/issues/1"
def junit(states):
cases = []
for name, state in states.items():
body = {"pass": "", "fail": "<failure message='x'/>", "error": "<error message='x'/>",
"skip": "<skipped/>"}[state]
cases.append(f'<testcase classname="{CLASSNAME}" name="{name}">{body}</testcase>')
return f'<testsuites><testsuite name="pytest">{"".join(cases)}</testsuite></testsuites>'
def run(*args):
proc = subprocess.run([sys.executable, str(SCRIPT), *args], capture_output=True, text=True, check=False)
return proc.returncode, proc.stdout, proc.stderr
class Lane:
"""A temporary reports directory with one sub-directory per target."""
def __init__(self, case):
self.temp = tempfile.TemporaryDirectory()
case.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.reports = self.root / "reports"
self.tolerated = self.root / "tolerated.txt"
self.tolerated.write_text("# none\n")
def target(self, name, states, selected=None, rerun=None):
directory = self.reports / name
directory.mkdir(parents=True)
names = selected if selected is not None else list(states)
(directory / "selected-nodeids.txt").write_text("".join(PREFIX + n + "\n" for n in names))
(directory / "junit.xml").write_text(junit(states))
if rerun is not None:
(directory / "rerun-junit.xml").write_text(junit(rerun))
return directory
def compare(self, targets=None):
summary = self.root / "summary.json"
code, out, err = run("compare", "--targets-json", json.dumps(targets or TARGETS),
"--reports", str(self.reports), "--tolerated", str(self.tolerated),
"--json", str(summary))
data = json.loads(summary.read_text()) if summary.exists() else None
return code, out, err, data
BASE = {"test_a": "pass", "test_b": "pass", "test_c": "skip"}
class CompareTests(unittest.TestCase):
# --- negative cases ---
def test_pass_versus_fail_is_a_difference(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_a": "fail"})
code, out, _, data = lane.compare()
self.assertEqual(code, 1)
self.assertEqual(data["differences"], [{"target": "candidate-legacy", "nodeid": PREFIX + "test_a",
"reference": "pass", "candidate": "fail", "tolerated": None}])
self.assertIn("Result: **FAIL**", out)
def test_error_counts_as_fail(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_b": "error"})
code, _, _, data = lane.compare()
self.assertEqual(code, 1)
self.assertEqual(data["differences"][0]["candidate"], "fail")
def test_reference_fail_candidate_pass_is_also_a_difference(self):
lane = Lane(self)
lane.target("reference", {**BASE, "test_a": "fail"})
lane.target("candidate-legacy", BASE)
code, _, _, data = lane.compare()
self.assertEqual(code, 1)
self.assertEqual(data["differences"][0]["reference"], "fail")
def test_skip_versus_pass_is_a_difference(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_c": "pass"})
code, _, _, data = lane.compare()
self.assertEqual(code, 1)
self.assertEqual(data["differences"][0]["nodeid"], PREFIX + "test_c")
def test_case_absent_from_candidate_report_is_missing_and_incomplete(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {"test_a": "pass", "test_c": "skip"}, selected=list(BASE))
code, out, _, data = lane.compare()
self.assertEqual(code, 1)
self.assertEqual(data["differences"][0]["candidate"], "missing")
self.assertEqual(data["incomplete"], {"candidate-legacy": [PREFIX + "test_b"]})
self.assertIn("Incomplete runs", out)
def test_case_missing_from_both_reports_still_fails(self):
lane = Lane(self)
partial = {"test_a": "pass", "test_c": "skip"}
lane.target("reference", partial, selected=list(BASE))
lane.target("candidate-legacy", partial, selected=list(BASE))
code, _, _, data = lane.compare()
self.assertEqual(code, 1)
self.assertEqual(data["differences"], [])
self.assertEqual(set(data["incomplete"]), {"reference", "candidate-legacy"})
def test_failure_that_also_fails_the_rerun_stays_a_difference(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_a": "fail"}, rerun={"test_a": "fail"})
code, _, _, data = lane.compare()
self.assertEqual(code, 1)
self.assertEqual(data["flaky"]["candidate-legacy"], [])
def test_rerun_of_a_case_that_did_not_fail_is_rejected(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_a": "fail"}, rerun={"test_b": "pass"})
code, _, err, _ = lane.compare()
self.assertEqual(code, 2)
self.assertIn("rerun without failing first", err)
def test_missing_target_report_fails_rather_than_skips(self):
lane = Lane(self)
lane.target("reference", BASE)
code, _, err, data = lane.compare()
self.assertEqual(code, 2)
self.assertIsNone(data)
self.assertIn("candidate-legacy: report directory is missing", err)
def test_missing_junit_fails(self):
lane = Lane(self)
lane.target("reference", BASE)
(lane.target("candidate-legacy", BASE) / "junit.xml").unlink()
code, _, err, _ = lane.compare()
self.assertEqual(code, 2)
self.assertIn("cannot read", err)
def test_different_selected_lists_are_rejected(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {"test_a": "pass", "test_b": "pass"})
code, _, err, _ = lane.compare()
self.assertEqual(code, 2)
self.assertIn("selected node ids differ", err)
def test_unselected_case_in_junit_is_rejected(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_extra": "pass"}, selected=list(BASE))
code, _, err, _ = lane.compare()
self.assertEqual(code, 2)
self.assertIn("unselected cases", err)
def test_stale_tolerated_entry_fails(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", BASE)
lane.tolerated.write_text(f"candidate-legacy | {PREFIX}test_a | {ISSUE} | known drift\n")
code, out, _, data = lane.compare()
self.assertEqual(code, 1)
self.assertEqual(data["stale_tolerated"][0]["nodeid"], PREFIX + "test_a")
self.assertIn("Stale tolerated entries", out)
def test_tolerated_entry_for_another_target_does_not_cover_the_difference(self):
lane = Lane(self)
targets = TARGETS + [{"name": "candidate-gateway", "binary": "candidate", "env": {}}]
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_a": "fail"})
lane.target("candidate-gateway", {**BASE, "test_a": "fail"})
lane.tolerated.write_text(f"candidate-gateway | {PREFIX}test_a | {ISSUE} | gateway gap\n")
code, _, _, data = lane.compare(targets)
self.assertEqual(code, 1)
untolerated = [d["target"] for d in data["differences"] if not d["tolerated"]]
self.assertEqual(untolerated, ["candidate-legacy"])
def test_tolerated_entry_without_issue_url_is_rejected(self):
for line in (f"candidate-legacy | {PREFIX}test_a | rustfs#1 | reason",
f"candidate-legacy | {PREFIX}test_a | {ISSUE} | ",
f"candidate-legacy | {PREFIX}test_a | {ISSUE}",
f"reference | {PREFIX}test_a | {ISSUE} | reason",
f"candidate-legacy | test_a | {ISSUE} | reason"):
with self.subTest(line=line):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_a": "fail"})
lane.tolerated.write_text(line + "\n")
code, _, err, _ = lane.compare()
self.assertEqual(code, 2, err)
def test_duplicate_tolerated_entry_is_rejected(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_a": "fail"})
entry = f"candidate-legacy | {PREFIX}test_a | {ISSUE} | reason\n"
lane.tolerated.write_text(entry * 2)
code, _, err, _ = lane.compare()
self.assertEqual(code, 2)
self.assertIn("duplicate entry", err)
def test_missing_tolerated_file_fails(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", BASE)
lane.tolerated.unlink()
code, _, err, _ = lane.compare()
self.assertEqual(code, 2)
self.assertIn("tolerated-difference file is missing", err)
# --- positive cases ---
def test_identical_runs_pass(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", BASE)
code, out, _, data = lane.compare()
self.assertEqual(code, 0)
self.assertEqual(data["counts"]["candidate-legacy"], {"pass": 2, "fail": 0, "skip": 1, "missing": 0})
self.assertIn("No differences.", out)
self.assertIn("Result: **PASS**", out)
def test_failure_that_passes_the_rerun_is_flaky_not_a_difference(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_a": "fail"}, rerun={"test_a": "pass"})
code, out, _, data = lane.compare()
self.assertEqual(code, 0)
self.assertEqual(data["flaky"]["candidate-legacy"], [PREFIX + "test_a"])
self.assertIn("Flaky", out)
def test_tolerated_difference_passes_and_is_labelled(self):
lane = Lane(self)
lane.target("reference", BASE)
lane.target("candidate-legacy", {**BASE, "test_a": "fail"})
lane.tolerated.write_text(f"# comment\ncandidate-legacy | {PREFIX}test_a | {ISSUE} | known drift\n")
code, out, _, data = lane.compare()
self.assertEqual(code, 0)
self.assertEqual(data["differences"][0]["tolerated"], ISSUE)
self.assertIn(ISSUE, out)
class TargetTests(unittest.TestCase):
def write(self, text):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
path = Path(temp.name) / "targets.toml"
path.write_text(text)
return path
def test_invalid_target_lists_are_rejected(self):
ref = '[[targets]]\nname = "reference"\nbinary = "reference"\n'
cand = '[[targets]]\nname = "candidate-legacy"\nbinary = "candidate"\n'
for label, text in {
"no schema": ref + cand,
"only reference": "schema = 1\n" + ref,
"two references": "schema = 1\n" + ref + ref.replace('"reference"\nbinary', '"other"\nbinary'),
"no reference": "schema = 1\n" + cand + cand.replace("candidate-legacy", "candidate-two"),
"duplicate name": "schema = 1\n" + ref + cand + cand,
"unknown binary": "schema = 1\n" + ref + cand.replace('binary = "candidate"', 'binary = "main"'),
"bad name": "schema = 1\n" + ref + cand.replace("candidate-legacy", "Candidate Legacy"),
"foreign env key": "schema = 1\n" + ref + cand + 'env = { PATH = "/tmp" }\n',
"env injection": "schema = 1\n" + ref + cand + 'env = { RUSTFS_X = "a\\nB=c" }\n',
"unknown key": "schema = 1\n" + ref + cand + 'stack = "gateway"\n',
}.items():
with self.subTest(label):
code, _, _ = run("targets", "--config", str(self.write(text)))
self.assertEqual(code, 2)
def test_missing_target_file_fails(self):
code, _, err = run("targets", "--config", "/nonexistent/targets.toml")
self.assertEqual(code, 2)
self.assertIn("target list is missing", err)
def test_export_env_rejects_foreign_keys(self):
code, _, _ = run("export-env", "--target-json", json.dumps({"env": {"LD_PRELOAD": "x"}}))
self.assertEqual(code, 2)
def test_repository_target_list_is_valid(self):
code, out, err = run("targets", "--config", str(REPO / ".config/reference-parity-targets.toml"))
self.assertEqual(code, 0, err)
self.assertEqual(json.loads(out), TARGETS)
def test_export_env_prints_validated_lines(self):
code, out, _ = run("export-env", "--target-json", json.dumps({"env": {"RUSTFS_S3_STACK": "gateway"}}))
self.assertEqual((code, out), (0, "RUSTFS_S3_STACK=gateway\n"))
class FailedTests(unittest.TestCase):
def test_failed_lists_only_failures_in_test_list_form(self):
lane = Lane(self)
directory = lane.target("reference", {**BASE, "test_a": "fail", "test_d": "error"})
code, out, _ = run("failed", "--report", str(directory))
self.assertEqual((code, out), (0, "test_a\ntest_d\n"))
def test_failed_rejects_a_missing_report(self):
code, _, err = run("failed", "--report", "/nonexistent/report")
self.assertEqual(code, 2)
self.assertIn("report directory is missing", err)
if __name__ == "__main__":
unittest.main()
@@ -1,27 +1,4 @@
#!/usr/bin/env bash
set -euo pipefail
workflow=".github/workflows/docker.yml"
require_literal() {
local needle="$1"
local description="$2"
if ! grep -Fq "$needle" "$workflow"; then
echo "missing container scan workflow contract: $description" >&2
exit 1
fi
}
require_literal "scan-docker-image:" "scan job"
require_literal "needs: [ build-check, build-docker ]" "build dependency"
require_literal "needs.build-check.outputs.should_build == 'true' && needs.build-check.outputs.should_push == 'true'" "release image push guard"
require_literal "docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9" "pinned GHCR login action"
require_literal "aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25" "pinned Trivy action"
require_literal 'image-ref: ${{ env.REGISTRY_GHCR }}:${{ needs.build-check.outputs.version }}${{ matrix.suffix }}' "GHCR image reference"
require_literal "format: sarif" "SARIF report format"
require_literal 'exit-code: "0"' "report-only failure policy"
require_literal "actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a" "pinned report upload action"
require_literal "container-image-scan-" "scan report artifact name"
echo "Container scan workflow contract ok."
cd "$(dirname "${BASH_SOURCE[0]}")/../.."
python3 scripts/release/test_pipeline.py ImageWorkflowTests
@@ -1,420 +1,4 @@
#!/usr/bin/env bash
set -euo pipefail
build_workflow=".github/workflows/build.yml"
docker_workflow=".github/workflows/docker.yml"
helm_workflow=".github/workflows/helm-package.yml"
package_workflow=".github/workflows/package.yml"
release_script="scripts/release/create_or_update_release.sh"
# shellcheck source=scripts/release/create_or_update_release.sh
source "$release_script"
require_line() {
local file="$1"
local line="$2"
local description="$3"
if ! grep -Fxq "$line" "$file"; then
echo "missing preview release workflow contract: $description" >&2
exit 1
fi
}
require_absent() {
local file="$1"
local text="$2"
local description="$3"
if grep -Fq -- "$text" "$file"; then
echo "invalid preview release workflow contract: $description" >&2
exit 1
fi
}
require_line "$build_workflow" " cargo zigbuild --release --locked --target \"\$TARGET_ARG\" \"\${FEATURE_ARGS[@]}\" \"\${PROFILE_ARGS[@]}\" -p rustfs --bin rustfs" "cross builds must keep matrix features"
require_line "$build_workflow" " cargo build --release --locked --target \"\$TARGET_ARG\" \"\${FEATURE_ARGS[@]}\" \"\${PROFILE_ARGS[@]}\" -p rustfs --bin rustfs" "native builds must keep matrix features"
require_absent "$build_workflow" " cargo zigbuild --release --locked --target \"\$TARGET_ARG\" -p rustfs --bin rustfs" "cross builds must not drop matrix features"
require_absent "$build_workflow" " cargo build --release --locked --target \"\$TARGET_ARG\" -p rustfs --bin rustfs" "native builds must not drop matrix features"
extract_job_if() {
local file="$1"
local job="$2"
awk -v job="$job" '
$0 == " " job ":" { in_job = 1 }
in_job && /^ if:/ { in_if = 1 }
in_if && /^ [A-Za-z0-9_-]+:/ && $0 !~ /^ if:/ { exit }
in_if { print }
in_job && $0 ~ /^ [A-Za-z0-9_-]+:$/ && $0 != " " job ":" { exit }
' "$file"
}
require_job_if() {
local file="$1"
local job="$2"
local expected="$3"
local actual
actual=$(extract_job_if "$file" "$job")
if [[ "$actual" != "$expected" ]]; then
echo "missing preview release workflow contract: $job condition" >&2
exit 1
fi
}
require_assignment() {
local block="$1"
local name="$2"
local expected="$3"
local count
count=$(printf '%s\n' "$block" | grep -Ec "^[[:space:]]+${name}=")
if [[ "$count" -ne 1 || "$block" != *"${name}=${expected}"* ]]; then
echo "missing preview release workflow contract: $name=$expected" >&2
exit 1
fi
}
require_no_assignment() {
local block="$1"
local name="$2"
if printf '%s\n' "$block" | grep -Eq "^[[:space:]]+${name}="; then
echo "invalid preview release workflow contract: unexpected $name override" >&2
exit 1
fi
}
tag_classification=$(awk '
/if \[\[ "\$tag_name" =~ -preview\\\.\[0-9\]\+\$ \]\]; then/ { in_preview = 1 }
in_preview { print }
in_preview && $0 == " fi" { exit }
' "$build_workflow")
invalid_preview_condition="elif [[ \"\$tag_name\" == *\"-preview\"* ]]; then"
prerelease_condition="elif [[ \"\$tag_name\" == *\"alpha\"* ]] || [[ \"\$tag_name\" == *\"beta\"* ]] || [[ \"\$tag_name\" == *\"rc\"* ]]; then"
preview_branch=$(printf '%s\n' "$tag_classification" | awk -v boundary="$invalid_preview_condition" 'index($0, boundary) { exit } { print }')
invalid_preview_branch=$(printf '%s\n' "$tag_classification" | awk -v start="$invalid_preview_condition" -v boundary="$prerelease_condition" 'index($0, start) { in_branch = 1 } in_branch && index($0, boundary) { exit } in_branch { print }')
prerelease_branch=$(printf '%s\n' "$tag_classification" | awk -v start="$prerelease_condition" 'index($0, start) { in_branch = 1 } in_branch && $0 == " else" { exit } in_branch { print }')
release_branch=$(printf '%s\n' "$tag_classification" | awk '$0 == " else" { in_branch = 1; next } in_branch && $0 == " fi" { exit } in_branch { print }')
if [[ -z "$preview_branch" || -z "$invalid_preview_branch" || -z "$prerelease_branch" || -z "$release_branch" ]]; then
echo "missing preview release workflow contract: ordered preview, invalid-preview, prerelease, and release branches" >&2
exit 1
fi
require_assignment "$preview_branch" "build_type" '"preview"'
require_assignment "$preview_branch" "is_prerelease" "true"
require_no_assignment "$invalid_preview_branch" "build_type"
if [[ "$invalid_preview_branch" != *"exit 1"* ]]; then
echo "missing preview release workflow contract: invalid preview tags must fail closed" >&2
exit 1
fi
require_assignment "$prerelease_branch" "build_type" '"prerelease"'
require_assignment "$prerelease_branch" "is_prerelease" "true"
require_assignment "$release_branch" "build_type" '"release"'
require_no_assignment "$release_branch" "is_prerelease"
initial_prerelease_line=$(grep -Fn " is_prerelease=false" "$build_workflow")
preview_condition_line=$(grep -Fn " if [[ \"\$tag_name\" =~ -preview\\.[0-9]+\$ ]]; then" "$build_workflow")
if [[ $(printf '%s\n' "$initial_prerelease_line" | wc -l) -ne 1 || "${initial_prerelease_line%%:*}" -ge "${preview_condition_line%%:*}" ]]; then
echo "missing preview release workflow contract: is_prerelease=false must initialize before tag classification" >&2
exit 1
fi
tag_branch_tail=$(awk '
/if \[\[ "\$tag_name" =~ -preview\\\.\[0-9\]\+\$ \]\]; then/ { in_classification = 1 }
in_classification && $0 == " fi" { after_classification = 1; next }
after_classification && /^ elif / { exit }
after_classification { print }
' "$build_workflow")
post_strategy=$(awk '
/# Determine build type based on trigger/ { in_strategy = 1 }
in_strategy && $0 == " fi" { after_strategy = 1; next }
after_strategy && $0 == " {" { exit }
after_strategy { print }
' "$build_workflow")
for block in "$tag_branch_tail" "$post_strategy"; do
require_no_assignment "$block" "build_type"
require_no_assignment "$block" "is_prerelease"
done
require_line "$build_workflow" " if [[ \"\$BUILD_TYPE\" == \"release\" ]] || [[ \"\$BUILD_TYPE\" == \"prerelease\" ]]; then" "latest artifact guard"
require_line "$build_workflow" " if: needs.build-check.outputs.should_publish == 'true' && env.R2_ACCESS_KEY_ID != '' && (needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease' || needs.build-check.outputs.build_type == 'development')" "R2 publication guard"
require_line "$package_workflow" " if [[ \"\$TAG\" =~ ^\${semver_core}(-\${prerelease_id})?-preview\\.(0|[1-9][0-9]*)$ ]]; then" "stable and prerelease preview package classification"
require_line "$package_workflow" " if: needs.resolve.outputs.build_type != 'preview' && env.R2_ACCESS_KEY_ID != ''" "preview packages must not publish to R2"
release_guard="needs.build-check.outputs.should_publish == 'true' && startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'preview' || needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')"
for job in create-release upload-release-assets publish-release; do
require_job_if "$build_workflow" "$job" " if: $release_guard"
done
latest_guard="needs.build-check.outputs.should_publish == 'true' && startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')"
require_job_if "$build_workflow" "update-latest-version" " if: $latest_guard"
require_line "$build_workflow" " needs: [ build-check, publish-release ]" "latest update must follow release publication"
# Preview releases are internal validation artifacts: once the deliverable
# release is published they are deleted, while their tags stay behind.
require_job_if "$build_workflow" "cleanup-preview-releases" " if: $latest_guard"
require_line "$build_workflow" " gh release delete \"\$preview_tag\" --repo \"\${GITHUB_REPOSITORY}\" --yes" "preview release cleanup after publication"
require_line "$build_workflow" " | select(.tag_name | startswith(\$tag + \"-preview.\"))" "cleanup must match the target's own preview tags"
require_line "$build_workflow" " | select(.tag_name | ltrimstr(\$tag + \"-preview.\") | test(\"^[0-9]+\$\"))" "cleanup must match a numeric preview iteration"
require_absent "$build_workflow" "--cleanup-tag" "preview tags must survive their release cleanup"
require_line "$build_workflow" " TARGET_COMMITISH=\$(git rev-parse --verify \"refs/tags/\${TAG}^{commit}\")" "release target commit resolution"
require_line "$build_workflow" " ./scripts/release/create_or_update_release.sh \\" "managed release creation"
require_absent "$build_workflow" "git tag -l --format='%(contents)'" "annotated tag messages must not become release notes"
require_absent "$build_workflow" "--generate-notes" "GitHub must not choose the release notes baseline implicitly"
require_line "$release_script" " gh api --method POST \"repos/\${GITHUB_REPOSITORY}/releases/generate-notes\" \\" "Generate Release Notes API"
require_line "$release_script" " local create_args=(release create \"\$tag\" --title \"\$title\" --notes-file \"\$notes_file\" --latest=false --draft)" "draft release creation with a notes file"
release_channel_block=$(awk '
$0 == " if [[ \"$BUILD_TYPE\" == \"release\" ]]; then" { in_block = 1 }
in_block { print }
in_block && $0 == " fi" { exit }
' "$build_workflow")
IFS= read -r -d '' expected_release_channel_block <<'EOF' || true
if [[ "$BUILD_TYPE" == "release" ]]; then
gh api --method PATCH "repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}" \
-F draft=false \
-F prerelease=false \
-f make_latest=true >/dev/null
else
gh api --method PATCH "repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}" \
-F draft=false \
-F prerelease=true \
-f make_latest=false >/dev/null
fi
EOF
expected_release_channel_block=${expected_release_channel_block%$'\n'}
if [[ "$release_channel_block" != "$expected_release_channel_block" ]]; then
echo "missing preview release workflow contract: only stable releases may become GitHub Latest" >&2
exit 1
fi
publish_release_block=$(awk '
$0 == " publish-release:" { in_job = 1 }
in_job { print }
in_job && /^ [A-Za-z0-9_-]+:$/ && $0 != " publish-release:" { exit }
' "$build_workflow")
if [[ "$publish_release_block" != *" - name: Publish release"* ]]; then
echo "missing preview release workflow contract: publish job must only publish the prepared release" >&2
exit 1
fi
if grep -Eq -- '--notes|generate-notes|create_or_update_release' <<<"$publish_release_block"; then
echo "invalid preview release workflow contract: publish job must not replace release notes" >&2
exit 1
fi
IFS= read -r -d '' expected_docker_automatic_guard <<'EOF' || true
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push' &&
github.event.workflow_run.head_branch != 'main')
EOF
expected_docker_automatic_guard=${expected_docker_automatic_guard%$'\n'}
require_job_if "$docker_workflow" "build-check" "$expected_docker_automatic_guard"
require_line "$docker_workflow" ' source_ref: ${{ steps.check.outputs.source_ref }}' "Docker source ref output"
require_line "$docker_workflow" ' source_ref="$HEAD_SHA"' "automatic Docker source ref"
require_line "$docker_workflow" ' source_ref="$tag_ref"' "manual Docker source ref"
require_line "$docker_workflow" ' ref: ${{ needs.build-check.outputs.source_ref }}' "Docker release source checkout"
require_line "$docker_workflow" ' SOURCE_REVISION="$(git rev-parse HEAD)"' "Docker source revision resolution"
require_line "$docker_workflow" ' LABELS="$LABELS,org.opencontainers.image.revision=$SOURCE_REVISION"' "Docker revision label"
require_absent "$docker_workflow" 'org.opencontainers.image.revision=${{ github.sha }}' "Docker revision must not use the workflow branch SHA"
python3 scripts/test_docker_workflow.py
IFS= read -r -d '' expected_helm_guard <<'EOF' || true
if: |
(github.event_name == 'workflow_dispatch' && !contains(github.event.inputs.version, '-preview')) ||
(
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push' &&
contains(github.event.workflow_run.head_branch, '.') &&
!contains(github.event.workflow_run.head_branch, '-preview')
)
EOF
expected_helm_guard=${expected_helm_guard%$'\n'}
require_job_if "$helm_workflow" "build-helm-package" "$expected_helm_guard"
require_line "$helm_workflow" " ref: \${{ github.event.workflow_run.head_sha || (startsWith(inputs.version, 'refs/tags/') && inputs.version || format('refs/tags/{0}', inputs.version)) }}" "Helm release source checkout"
python3 scripts/test_helm_release_workflow.py
assert_equal() {
local expected="$1"
local actual="$2"
local description="$3"
if [[ "$actual" != "$expected" ]]; then
echo "invalid preview release workflow contract: $description (expected '$expected', got '$actual')" >&2
exit 1
fi
}
IFS= read -r -d '' release_fixture <<'EOF' || true
[
[
{"tag_name":"1.0.0-beta.12-preview.1","created_at":"2026-07-30T02:30:00Z","published_at":"2026-07-30T02:36:43Z","draft":false},
{"tag_name":"1.0.0-beta.12","created_at":"2026-07-30T04:00:00Z","published_at":"2026-07-30T04:13:04Z","draft":false},
{"tag_name":"1.0.0-beta.12-preview.0","created_at":"2026-07-29T10:00:00Z","published_at":"2026-07-29T10:05:00Z","draft":false}
],
[
{"tag_name":"1.0.0-beta.10","created_at":"2026-07-17T05:50:00Z","published_at":"2026-07-17T05:58:19Z","draft":false},
{"tag_name":"1.0.0-beta.11","created_at":"2026-07-23T04:00:00Z","published_at":"2026-07-23T04:07:31Z","draft":false},
{"tag_name":"9.0.0","created_at":"2025-01-01T00:00:00Z","published_at":"2025-01-01T00:00:00Z","draft":false},
{"tag_name":"1.0.0-beta.13","created_at":"2026-08-01T00:00:00Z","published_at":null,"draft":true}
]
]
EOF
preview_previous=$(select_previous_release_tag "1.0.0-beta.12-preview.1" "2026-07-30T02:36:43Z" <<<"$release_fixture")
assert_equal "1.0.0-beta.11" "$preview_previous" "preview notes baseline must exclude preview releases and its final deliverable"
final_previous=$(select_previous_release_tag "1.0.0-beta.12" "2026-07-30T04:13:04Z" <<<"$release_fixture")
assert_equal "1.0.0-beta.11" "$final_previous" "final notes baseline must exclude the same-commit preview release"
no_previous=$(select_previous_release_tag "1.0.0-beta.1-preview.1" "2026-01-01T00:00:00Z" <<'EOF'
[[{"tag_name":"1.0.0-beta.1-preview.1","created_at":"2026-01-01T00:00:00Z","published_at":"2026-01-01T00:00:00Z","draft":false}]]
EOF
)
assert_equal "" "$no_previous" "repositories without a previous deliverable must use GitHub's fallback baseline"
delayed_draft_previous=$(select_previous_release_tag "1.0.0-beta.12" "2026-07-30T04:00:00Z" <<'EOF'
[[
{"tag_name":"1.0.0-beta.12","created_at":"2026-07-30T04:00:00Z","published_at":null,"draft":true},
{"tag_name":"1.0.0-beta.13","created_at":"2026-08-01T00:00:00Z","published_at":"2026-08-01T00:05:00Z","draft":false},
{"tag_name":"1.0.0-beta.11","created_at":"2026-07-23T04:00:00Z","published_at":"2026-07-23T04:07:31Z","draft":false}
]]
EOF
)
assert_equal "1.0.0-beta.11" "$delayed_draft_previous" "draft rerun must exclude deliverables published after the draft was created"
release_test_dir=$(mktemp -d "${TMPDIR:-/tmp}/rustfs-release-workflow-test.XXXXXX")
trap 'rm -rf -- "$release_test_dir"' EXIT
request_json="${release_test_dir}/request.json"
write_generate_notes_request "1.0.0-beta.12" "0123456789abcdef" "1.0.0-beta.11" "$request_json"
assert_equal "1.0.0-beta.11" "$(jq -r .previous_tag_name "$request_json")" "explicit previous_tag_name request field"
write_generate_notes_request "1.0.0-beta.1" "0123456789abcdef" "" "$request_json"
assert_equal "false" "$(jq 'has("previous_tag_name")' "$request_json")" "fallback request must omit previous_tag_name"
existing_release_json="${release_test_dir}/existing-release.json"
jq -n --arg body "Pre-release 1.0.0-beta.12 (beta)" '{body: $body}' > "$existing_release_json"
assert_equal "update" "$(release_notes_action "$existing_release_json" "1.0.0-beta.12")" "rerun must repair legacy prerelease placeholder notes"
jq -n --arg body "${MANAGED_NOTES_MARKER}
## What's Changed" '{body: $body}' > "$existing_release_json"
assert_equal "update" "$(release_notes_action "$existing_release_json" "1.0.0-beta.12")" "rerun must refresh managed notes"
generated_body="## What's Changed
* Fix release notes by @alice in #123
* First contribution by @bob in #124
## New Contributors
* @bob made their first contribution in #124
**Full Changelog**: https://github.com/rustfs/rustfs/compare/1.0.0-beta.11...1.0.0-beta.12"
response_json="${release_test_dir}/response.json"
notes_file="${release_test_dir}/notes.md"
jq -n --arg body "$generated_body" '{body: $body}' > "$response_json"
write_generated_release_notes "$response_json" "$notes_file" "rustfs/rustfs" "1.0.0-beta.12" "1.0.0-beta.11"
expected_notes="${MANAGED_NOTES_MARKER}
${generated_body}"
assert_equal "$expected_notes" "$(<"$notes_file")" "generated multiline Markdown must remain intact"
mock_log="${release_test_dir}/gh.log"
mock_notes="${release_test_dir}/captured-notes.md"
mock_generate_calls="${release_test_dir}/generate-calls"
MOCK_RELEASES_JSON="$release_fixture"
MOCK_GENERATED_BODY="$generated_body"
MOCK_RELEASE_EXISTS=true
MOCK_RELEASE_BODY="Release 1.0.0-beta.12"
MOCK_RELEASE_VIEW_ERROR=""
gh() {
local input_file=""
local notes_path=""
if [[ "$1" == "api" && "$*" == *"--paginate --slurp"* ]]; then
printf '%s\n' "$MOCK_RELEASES_JSON"
return
fi
if [[ "$1" == "api" && "$*" == *"releases/generate-notes"* ]]; then
while [[ "$#" -gt 0 ]]; do
if [[ "$1" == "--input" ]]; then
input_file="$2"
break
fi
shift
done
jq -e '.tag_name == "1.0.0-beta.12" and .target_commitish == "0123456789abcdef" and .previous_tag_name == "1.0.0-beta.11"' "$input_file" >/dev/null
printf '%s\n' called >> "$mock_generate_calls"
jq -n --arg body "$MOCK_GENERATED_BODY" '{body: $body}'
return
fi
if [[ "$1" == "release" && "$2" == "view" ]]; then
if [[ -n "$MOCK_RELEASE_VIEW_ERROR" ]]; then
printf '%s\n' "$MOCK_RELEASE_VIEW_ERROR" >&2
return 1
fi
if [[ "$MOCK_RELEASE_EXISTS" != "true" ]]; then
echo "release not found" >&2
return 1
fi
jq -n --arg body "$MOCK_RELEASE_BODY" '{databaseId: 123, url: "https://github.com/rustfs/rustfs/releases/tag/test", body: $body, createdAt: "2026-07-30T04:00:00Z", publishedAt: "2026-07-30T04:13:04Z"}'
return
fi
if [[ "$1" == "release" && ("$2" == "edit" || "$2" == "create") ]]; then
local action="$2"
shift 2
while [[ "$#" -gt 0 ]]; do
if [[ "$1" == "--notes-file" ]]; then
notes_path="$2"
break
fi
shift
done
cp "$notes_path" "$mock_notes"
MOCK_RELEASE_BODY=$(<"$mock_notes")
MOCK_RELEASE_EXISTS=true
printf '%s\n' "$action" >> "$mock_log"
return
fi
echo "unexpected mock gh invocation: $*" >&2
return 1
}
GITHUB_REPOSITORY="rustfs/rustfs"
GITHUB_OUTPUT="${release_test_dir}/github-output"
RUNNER_TEMP="$release_test_dir"
create_or_update_release "1.0.0-beta.12" "0123456789abcdef" "RustFS 1.0.0-beta.12 (beta)" true
assert_equal "edit" "$(<"$mock_log")" "existing placeholder release must be updated on rerun"
assert_equal "$expected_notes" "$(<"$mock_notes")" "rerun must update notes through --notes-file"
: > "$mock_log"
MOCK_RELEASE_EXISTS=false
MOCK_RELEASE_BODY=""
create_or_update_release "1.0.0-beta.12" "0123456789abcdef" "RustFS 1.0.0-beta.12 (beta)" true
assert_equal "create" "$(<"$mock_log")" "first run must create the draft release"
: > "$mock_log"
: > "$mock_generate_calls"
MOCK_RELEASE_EXISTS=true
MOCK_RELEASE_BODY="$generated_body"
create_or_update_release "1.0.0-beta.12" "0123456789abcdef" "RustFS 1.0.0-beta.12 (beta)" true
assert_equal "" "$(<"$mock_log")" "manual notes must not trigger a release edit"
assert_equal "" "$(<"$mock_generate_calls")" "manual notes must be preserved without regeneration"
MOCK_RELEASE_BODY="Manually curated release guidance"
if create_or_update_release "1.0.0-beta.12" "0123456789abcdef" "RustFS 1.0.0-beta.12 (beta)" true; then
echo "invalid preview release workflow contract: incomplete manual notes must block publication" >&2
exit 1
fi
assert_equal "" "$(<"$mock_log")" "incomplete manual notes must not be overwritten"
MOCK_RELEASE_VIEW_ERROR="gh: service unavailable (HTTP 503)"
if create_or_update_release "1.0.0-beta.12" "0123456789abcdef" "RustFS 1.0.0-beta.12 (beta)" true; then
echo "invalid preview release workflow contract: transient release lookup failures must fail closed" >&2
exit 1
fi
assert_equal "" "$(<"$mock_log")" "release lookup failures must not attempt create or edit"
echo "Preview release workflow contract ok."
cd "$(dirname "${BASH_SOURCE[0]}")/../.."
python3 scripts/release/test_pipeline.py WorkflowContractTests
@@ -25,6 +25,7 @@ def parse_args() -> argparse.Namespace:
parser.add_argument("--run-id", default=os.getenv("GITHUB_RUN_ID", ""))
parser.add_argument("--run-attempt", default=os.getenv("GITHUB_RUN_ATTEMPT", ""))
parser.add_argument("--cargo-lock", default="Cargo.lock", help="Cargo.lock path.")
parser.add_argument("--workflow", default="build.yml", help="Workflow that assembled these assets.")
parser.add_argument("--metadata-file", help="Use an existing cargo metadata JSON file.")
return parser.parse_args()
@@ -118,7 +119,7 @@ def build_sbom(packages: list[dict], args: argparse.Namespace, timestamp: str) -
def build_provenance(asset_dir: Path, args: argparse.Namespace, timestamp: str) -> dict:
subjects = []
for asset in sorted(asset_dir.glob("*.zip")):
for asset in sorted(path for path in asset_dir.iterdir() if path.suffix in (".zip", ".deb", ".rpm", ".tgz")):
subjects.append(
{
"name": asset.name,
@@ -129,7 +130,7 @@ def build_provenance(asset_dir: Path, args: argparse.Namespace, timestamp: str)
if not subjects:
raise RuntimeError(f"no .zip release assets found in {asset_dir}")
workflow_ref = f"{args.repository}/.github/workflows/build.yml"
workflow_ref = f"{args.repository}/.github/workflows/{args.workflow}"
invocation = args.run_id
if args.run_attempt:
invocation = f"{invocation}/attempts/{args.run_attempt}" if invocation else args.run_attempt
+260
View File
@@ -0,0 +1,260 @@
#!/usr/bin/env bash
# Contract tests for scripts/check_test_count_ratchet.sh.
#
# Each case builds a throwaway two-crate cargo workspace (crates `alpha` and
# `beta`), copies the script under test into it, writes a baseline, and runs
# the script there, so the real workspace and baseline are never touched. Only
# `cargo metadata --no-deps` runs; nothing is compiled.
#
# Contract under test: the baseline is a floor on tests and a ceiling on
# #[ignore]; see the header of check_test_count_ratchet.sh.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
SCRIPT_UNDER_TEST="${RATCHET_SCRIPT:-$SCRIPT_DIR/check_test_count_ratchet.sh}"
TMP_ROOT="$(mktemp -d)"
trap 'rm -rf "$TMP_ROOT"' EXIT
passed=0
case_no=0
FX=""
OUT=""
ERR=""
STATUS=0
fail() {
echo "FAIL: $1" >&2
exit 1
}
# new_case <name>: fresh workspace with alpha and beta (members, no tests yet).
new_case() {
case_no=$((case_no + 1))
FX="$TMP_ROOT/case$case_no"
mkdir -p "$FX/scripts" "$FX/.config"
cp "$SCRIPT_UNDER_TEST" "$FX/scripts/check_test_count_ratchet.sh"
chmod +x "$FX/scripts/check_test_count_ratchet.sh"
printf '[workspace]\nresolver = "2"\nmembers = ["alpha", "beta"]\n' >"$FX/Cargo.toml"
make_crate alpha 1 0
make_crate beta 0 0
: >"$FX/.config/test-count-baseline.txt"
}
# make_crate <name> <tests> <ignored>: (re)write a member with that many
# attribute lines.
make_crate() {
local name="$1" tests="$2" ignored="$3" i
mkdir -p "$FX/$name/src"
printf '[package]\nname = "%s"\nversion = "0.0.0"\nedition = "2021"\n' "$name" >"$FX/$name/Cargo.toml"
{
for ((i = 0; i < tests; i++)); do printf '#[test]\nfn t%d() {}\n' "$i"; done
for ((i = 0; i < ignored; i++)); do printf '#[ignore]\nfn i%d() {}\n' "$i"; done
printf 'pub fn anchor() {}\n'
} >"$FX/$name/src/lib.rs"
}
# baseline <package> <tests> <ignored>...: append rows (tab-separated).
baseline() {
printf '%s\t%s\t%s\n' "$1" "$2" "$3" >>"$FX/.config/test-count-baseline.txt"
}
run() {
STATUS=0
(cd "$FX" && ./scripts/check_test_count_ratchet.sh "$@") >"$TMP_ROOT/out" 2>"$TMP_ROOT/err" || STATUS=$?
OUT="$(cat "$TMP_ROOT/out")"
ERR="$(cat "$TMP_ROOT/err")"
}
expect_pass() {
((STATUS == 0)) || fail "$1: expected exit 0, got $STATUS; stderr: $ERR"
}
expect_fail() {
((STATUS == 1)) || fail "$1: expected exit 1, got $STATUS; stderr: $ERR"
}
# stream_text <stdout|stderr>: captured output of the last run.
stream_text() {
if [[ "$1" == stdout ]]; then printf '%s' "$OUT"; else printf '%s' "$ERR"; fi
}
# expect_in <label> <stdout|stderr> <needle>
expect_in() {
local text
text="$(stream_text "$2")"
[[ "$text" == *"$3"* ]] || fail "$1: $2 lacks '$3'; got: $text"
}
expect_not_in() {
local text
text="$(stream_text "$2")"
[[ "$text" != *"$3"* ]] || fail "$1: $2 must not contain '$3'; got: $text"
}
ok() {
passed=$((passed + 1))
echo "ok: $1"
}
# --- positive cases -------------------------------------------------------
name="exact match passes silently"
new_case
make_crate alpha 3 1
baseline alpha 3 1
baseline beta 0 0
run
expect_pass "$name"
expect_not_in "$name" stdout "above baseline"
ok "$name"
name="tests above baseline pass with an informational line"
new_case
make_crate alpha 5 0
baseline alpha 3 0
baseline beta 0 0
run
expect_pass "$name"
expect_in "$name" stdout "note: crate alpha: 5 above baseline 3; run --update to tighten"
expect_not_in "$name" stderr "alpha"
ok "$name"
name="ignored below baseline passes with an informational line"
new_case
make_crate alpha 3 0
baseline alpha 3 2
baseline beta 0 0
run
expect_pass "$name"
expect_in "$name" stdout "note: crate alpha: 0 ignore attributes below baseline 2; run --update to tighten"
ok "$name"
name="member without a row passes with a notice"
new_case
make_crate alpha 3 0
make_crate beta 2 1
baseline alpha 3 0
run
expect_pass "$name"
expect_in "$name" stdout "note: crate beta: no baseline row (2 tests, 1 ignore attributes); run --update to add it"
ok "$name"
name="informational lines never carry a failure shape"
new_case
make_crate alpha 5 0
make_crate beta 1 0
baseline alpha 3 2
run
expect_pass "$name"
for shape in " < baseline" "ignored >" "not a workspace member" "violated"; do
expect_not_in "$name" stdout "$shape"
done
ok "$name"
name="--update round-trips: rewrites rows from the tree and the check then passes"
new_case
make_crate alpha 4 1
make_crate beta 2 0
baseline alpha 9 9
run --update
expect_pass "$name"
grep -qxF "$(printf 'alpha\t4\t1')" "$FX/.config/test-count-baseline.txt" || fail "$name: alpha row not rewritten"
grep -qxF "$(printf 'beta\t2\t0')" "$FX/.config/test-count-baseline.txt" || fail "$name: beta row not added"
run
expect_pass "$name (check after update)"
expect_not_in "$name" stdout "above baseline"
ok "$name"
# --- negative cases -------------------------------------------------------
name="tests below baseline fail"
new_case
make_crate alpha 1 0
baseline alpha 3 0
baseline beta 0 0
run
expect_fail "$name"
expect_in "$name" stderr "crate alpha: 1 < baseline 3"
ok "$name"
name="a drop in one crate is not hidden by an addition in another"
new_case
make_crate alpha 1 0
make_crate beta 9 0
baseline alpha 3 0
baseline beta 2 0
run
expect_fail "$name"
expect_in "$name" stderr "crate alpha: 1 < baseline 3"
expect_in "$name" stdout "note: crate beta: 9 above baseline 2"
ok "$name"
name="ignored above baseline fails with the main-branch wording"
new_case
make_crate alpha 3 2
baseline alpha 3 0
baseline beta 0 0
run
expect_fail "$name"
expect_in "$name" stderr "crate alpha: 2 ignored > baseline 0"
expect_in "$name" stderr "raise the baseline row"
expect_in "$name" stderr "PR description"
expect_not_in "$name" stderr "gateway/integration"
ok "$name"
name="ignored above fails even when tests are above baseline too"
new_case
make_crate alpha 8 1
baseline alpha 3 0
baseline beta 0 0
run
expect_fail "$name"
expect_in "$name" stderr "crate alpha: 1 ignored > baseline 0"
ok "$name"
name="a row for a crate that is no longer a member fails"
new_case
make_crate alpha 3 0
baseline alpha 3 0
baseline beta 0 0
baseline gamma 4 0
run
expect_fail "$name"
expect_in "$name" stderr "crate gamma: not a workspace member"
ok "$name"
name="malformed row fails"
new_case
printf 'alpha\tmany\t0\n' >>"$FX/.config/test-count-baseline.txt"
run
expect_fail "$name"
expect_in "$name" stderr "malformed rows"
ok "$name"
name="duplicate row fails"
new_case
baseline alpha 0 0
baseline alpha 0 0
baseline beta 0 0
run
expect_fail "$name"
expect_in "$name" stderr "duplicate rows"
ok "$name"
name="missing baseline file fails"
new_case
make_crate alpha 1 0
rm "$FX/.config/test-count-baseline.txt"
run
expect_fail "$name"
expect_in "$name" stderr "is missing"
ok "$name"
name="unknown argument is a usage error"
new_case
run --bogus
((STATUS == 2)) || fail "$name: expected exit 2, got $STATUS"
ok "$name"
echo "check_test_count_ratchet tests passed: $passed cases"
+102
View File
@@ -152,6 +152,108 @@ class PlannerTests(unittest.TestCase):
self.assertEqual(code['mode'], 'full')
self.assertIn('leaf', code['proposed']['packages'])
def release_metadata_base(self):
self.repo.write(gate.POLICY_PATH, json.dumps(dict(self.repo.policy, rollout='enforced')))
files = {
'flake.nix': '{ pkgs }: {\n rustfs = pkgs.buildRustPackage {\n pname = "rustfs";\n version = "1.0.4";\n src = ./.;\n };\n clientVersion = "0.1.36";\n}\n',
'helm/rustfs/Chart.yaml': 'apiVersion: v2\nname: rustfs\nversion: "1.0.4"\nappVersion: "1.0.4"\n',
'rustfs.spec': 'Name: rustfs\nVersion: 1.0.4\nRelease: 1\n%build\ncargo build --release\n%changelog\n* Mon Oct 26 2026 Maintainer <maintainer@example.invalid>\n- Update RPM package to RustFS 1.0.4\n',
}
for path, content in files.items():
self.repo.write(path, content)
return self.repo.commit(), files
def test_stable_installation_metadata_keeps_quick_checks_without_rust_rebuild(self):
baseline, files = self.release_metadata_base()
for path, content in files.items():
updated = content.replace('1.0.4', '1.0.5') if path != 'rustfs.spec' else content.replace('Version: 1.0.4', 'Version: 1.0.5')
if path == 'rustfs.spec':
updated = updated.replace('%changelog\n', '%changelog\n* Wed Nov 11 2026 Maintainer <maintainer@example.invalid>\n- Update RPM package to RustFS 1.0.5\n\n')
self.repo.write(path, updated)
self.repo.write('README.md', '# Install 1.0.5\n')
head = self.repo.commit()
plan = self.repo.plan(head, base=baseline)
self.assertEqual(plan['mode'], 'docs')
self.assertEqual(plan['packages'], [])
for job in gate.ALWAYS_JOBS:
self.assertEqual(plan['jobs'][job], 'success')
self.assertTrue(all(plan['jobs'][job] == 'skipped' for job in gate.CODE_JOBS))
self.assertEqual(gate.validate_plan(plan, 'pull_request', 'refs/heads/main', head, plan['tree']), plan['jobs'])
for event in ('push', 'merge_group', 'schedule', 'workflow_dispatch'):
self.assertTrue(self.repo.plan(head, event=event, base=baseline)['full_validation'])
def test_individual_stable_version_fields_without_new_rpm_history_are_metadata(self):
baseline, files = self.release_metadata_base()
for path, content in files.items():
with self.subTest(path=path):
self.repo.git('reset', '--hard', baseline)
updated = content.replace('1.0.4', '1.0.5') if path != 'rustfs.spec' else content.replace('Version: 1.0.4', 'Version: 1.0.5')
self.repo.write(path, updated)
self.assertEqual(self.repo.plan(self.repo.commit(), base=baseline)['mode'], 'docs')
def test_installation_metadata_does_not_hide_build_or_malformed_changes(self):
baseline, files = self.release_metadata_base()
cases = [
('flake.nix', files['flake.nix'].replace('src = ./.;', 'src = ./other;')),
('flake.nix', files['flake.nix'].replace('0.1.36', '0.1.37')),
('flake.nix', files['flake.nix'].replace('1.0.4', '${builtins.readFile ./version}')),
('helm/rustfs/Chart.yaml', files['helm/rustfs/Chart.yaml'] + 'dependencies: []\n'),
('helm/rustfs/Chart.yaml', files['helm/rustfs/Chart.yaml'].replace('appVersion: "1.0.4"', 'appVersion: "1.0.5"')),
('helm/rustfs/Chart.yaml', files['helm/rustfs/Chart.yaml'] + 'version: "1.0.5"\n'),
('rustfs.spec', files['rustfs.spec'].replace('cargo build --release', 'cargo build')),
('rustfs.spec', files['rustfs.spec'].replace('Release: 1', 'Release: 2')),
('rustfs.spec', files['rustfs.spec'].replace('%changelog\n', '%changelog\n%(echo injected)\n')),
('rustfs.spec', files['rustfs.spec'].replace('Update RPM package', 'Rewrite existing history')),
]
cases.extend((path, content.replace('1.0.4', '1.0.5-rc.1')) for path, content in files.items())
for path, content in cases:
with self.subTest(path=path, content=content):
self.repo.git('reset', '--hard', baseline)
self.repo.write(path, content)
self.assertTrue(self.repo.plan(self.repo.commit(), base=baseline)['full_validation'])
def test_installation_metadata_rejects_normalization_marker_collisions(self):
_, files = self.release_metadata_base()
old = files['flake.nix'].replace(' clientVersion =', ' second = pkgs.buildRustPackage {\n pname = "rustfs";\n version = "<version>";\n src = ./other;\n };\n clientVersion =')
self.repo.write('flake.nix', old)
baseline = self.repo.commit()
new = old.replace('version = "1.0.4"', 'version = "<version>"').replace('version = "<version>";\n src = ./other;', 'version = "1.0.5";\n src = ./other;')
self.repo.write('flake.nix', new)
self.assertTrue(self.repo.plan(self.repo.commit(), base=baseline)['full_validation'])
def test_installation_metadata_rejects_missing_files_symlinks_and_mode_changes(self):
baseline, files = self.release_metadata_base()
for path in files:
for mutation in ('delete', 'symlink', 'executable'):
with self.subTest(path=path, mutation=mutation):
self.repo.git('reset', '--hard', baseline)
target = self.repo.root / path
if mutation == 'executable':
target.chmod(0o755)
else:
target.unlink()
if mutation == 'symlink':
target.symlink_to('/tmp/release-metadata')
self.assertTrue(self.repo.plan(self.repo.commit(), base=baseline)['full_validation'])
self.repo.git('reset', '--hard', baseline)
self.assertTrue(self.repo.plan(baseline, base=self.repo.base)['full_validation'])
def test_installation_metadata_cannot_override_code_or_explicit_full_policy(self):
baseline, files = self.release_metadata_base()
self.repo.write('flake.nix', files['flake.nix'].replace('1.0.4', '1.0.5'))
self.repo.write('leaf/src/lib.rs', '// changed implementation\n')
plan = self.repo.plan(self.repo.commit(), base=baseline)
self.assertEqual(plan['mode'], 'full')
self.assertEqual(plan['packages'], ['consumer', 'core', 'leaf', 'rustfs'])
self.repo.git('reset', '--hard', baseline)
self.repo.write('flake.nix', files['flake.nix'].replace('1.0.4', '1.0.5'))
head = self.repo.commit()
policy = dict(self.repo.policy, rollout='enforced', full_paths=['flake.nix'])
with patch.object(gate, 'read_policy', return_value=policy):
self.assertTrue(self.repo.plan(head, base=baseline)['full_validation'])
with patch.object(gate, 'read_policy', return_value=self.repo.policy):
self.assertTrue(self.repo.plan(head, base=baseline)['full_validation'])
def test_shadow_retains_full_mode_for_harness_build_and_fixture_changes(self):
for path in ('crates/e2e_test/build.rs', 'crates/e2e_test/src/fixture.json'):
self.repo.write(path, 'changed\n')
+7 -194
View File
@@ -1,199 +1,12 @@
#!/usr/bin/env python3
"""Exercise the Docker workflow's version selection and tags without publishing."""
import os
import re
import subprocess
import tempfile
import unittest
"""Exercise the candidate-based replacement for the former release trigger."""
import importlib
import sys
from pathlib import Path
import unittest
ROOT = Path(__file__).resolve().parents[1]
WORKFLOW = ROOT / ".github/workflows/docker.yml"
REGISTRIES = ("rustfs/rustfs", "ghcr.io/rustfs/rustfs", "quay.io/rustfs/rustfs")
def run_body(name):
lines = WORKFLOW.read_text().splitlines()
start = lines.index(f" - name: {name}")
start = lines.index(" run: |", start) + 1
body = []
for line in lines[start:]:
if line.strip() and not line.startswith(" "):
break
body.append(line[10:])
return "\n".join(body)
class DockerWorkflowTests(unittest.TestCase):
def run_step(self, name, context, **env):
script = re.sub(
r"\$\{\{\s*(.*?)\s*\}\}", lambda m: context[m[1]], run_body(name)
)
# Fail on unexpected Git calls so these tests cannot contact a remote.
mock_git = """
git() {
if [[ "$1" == rev-parse ]]; then
echo 0123456789abcdef
elif [[ "$1 $2 $3" == "ls-remote --tags --refs" ]]; then
if [[ "$STABLE_EXISTS" == true ]]; then
echo "0123456789abcdef refs/tags/v1.0.0"
fi
elif [[ "$1 $2" == "ls-remote --exit-code" ]]; then
[[ "$4" == "$EXISTING_TAG" ]]
else
return 99
fi
}
"""
with tempfile.TemporaryDirectory() as directory:
output = Path(directory) / "output"
result = subprocess.run(
["bash", "-e", "-o", "pipefail", "-c", mock_git + script],
env={**os.environ, "GITHUB_OUTPUT": str(output), **env},
text=True,
capture_output=True,
check=False,
)
values = (
dict(line.split("=", 1) for line in output.read_text().splitlines())
if output.exists()
else {}
)
return result, values
def classify(
self, version, event="workflow_run", push="true", stable="true", tag=None, **env
):
return self.run_step(
"Check build conditions",
{"github.event_name": event},
GITHUB_SHA="workflow-sha",
HEAD_SHA="release-sha",
HEAD_BRANCH=version,
INPUT_VERSION=version,
INPUT_PUSH_IMAGES=push,
INPUT_FORCE_REBUILD="false",
STABLE_EXISTS=stable,
EXISTING_TAG=tag or f"refs/tags/{version}",
**{"CONCLUSION": "success", "TRIGGERING_EVENT": "push", **env},
)
def assert_tags(self, values, suffix="", channels=()):
context = {
f"needs.build-check.outputs.{key}": value for key, value in values.items()
}
context.update(
{
"matrix.suffix": suffix,
"env.REGISTRY_DOCKERHUB": REGISTRIES[0],
"env.REGISTRY_GHCR": REGISTRIES[1],
"env.REGISTRY_QUAY": REGISTRIES[2],
"github.server_url": "https://github.com",
"github.repository": "rustfs/rustfs",
}
)
result, metadata = self.run_step("Extract metadata and generate tags", context)
self.assertEqual(result.returncode, 0, result.stderr)
expected = {
f"{registry}:{tag}{suffix}"
for registry in REGISTRIES
for tag in (values["version"], *channels)
}
self.assertEqual(set(metadata["tags"].split(",")), expected)
def test_preview_version_tags_only(self):
versions = (
"1.0.1-preview.8",
"v1.0.1-preview.8",
"1.0.0-alpha.1-preview.2",
"1.0.0-beta.1-preview.2",
"v1.0.0-rc.1-preview.2",
)
for event in ("workflow_run", "workflow_dispatch"):
for version in versions:
for stable in ("true", "false"):
with self.subTest(event=event, version=version, stable=stable):
result, values = self.classify(version, event, stable=stable)
self.assertEqual(result.returncode, 0, result.stderr)
for key, expected in {
"should_build": "true",
"should_push": "true",
"build_type": "preview",
"is_prerelease": "true",
"create_latest": "false",
"version": version.removeprefix("v"),
}.items():
self.assertEqual(values[key], expected, key)
expected_ref = (
"release-sha"
if event == "workflow_run"
else f"refs/tags/{version}"
)
self.assertEqual(values["source_ref"], expected_ref)
for suffix in ("", "-glibc"):
self.assert_tags(values, suffix)
def test_manual_preview_dry_run_and_tag_fallback(self):
result, values = self.classify(
"1.0.1-preview.8",
"workflow_dispatch",
push="false",
tag="refs/tags/v1.0.1-preview.8",
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(values["should_build"], "true")
self.assertEqual(values["should_push"], "false")
self.assertEqual(values["source_ref"], "refs/tags/v1.0.1-preview.8")
def test_full_tag_ref(self):
result, values = self.classify("refs/tags/v1.0.1-preview.8")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(values["build_type"], "preview")
self.assertEqual(values["version"], "1.0.1-preview.8")
self.assert_tags(values)
def test_invalid_preview_and_missing_tag_fail(self):
for event in ("workflow_run", "workflow_dispatch"):
for version in (
"1.0.1-preview",
"1.0.1-preview.x",
"1.0.1-preview.8-extra",
):
with self.subTest(event=event, version=version):
result, _ = self.classify(version, event)
self.assertNotEqual(result.returncode, 0)
result, _ = self.classify(
"1.0.1-preview.8", "workflow_dispatch", tag="refs/tags/missing"
)
self.assertNotEqual(result.returncode, 0)
def test_existing_release_channels(self):
for event in ("workflow_run", "workflow_dispatch"):
for stable in ("true", "false"):
for channel in (None, "alpha", "beta", "rc"):
version = f"1.0.1-{channel}.1" if channel else "1.0.1"
with self.subTest(event=event, stable=stable, channel=channel):
result, values = self.classify(version, event, stable=stable)
self.assertEqual(result.returncode, 0, result.stderr)
latest = channel is None or stable == "false"
self.assertEqual(values["create_latest"], str(latest).lower())
channels = ([channel] if channel else []) + (
["latest"] if latest else []
)
self.assert_tags(values, channels=channels)
def test_non_release_builds_are_skipped(self):
for version, env in (
("main", {}),
("feature/test", {}),
("1.0.1-preview.8", {"CONCLUSION": "failure"}),
("1.0.1-preview.8", {"TRIGGERING_EVENT": "workflow_dispatch"}),
):
result, values = self.classify(version, **env)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(values["should_build"], "false")
sys.path.insert(0, str(Path(__file__).resolve().parent / "release"))
module = importlib.import_module("test_pipeline")
if __name__ == "__main__":
unittest.main()
unittest.main(module=module, defaultTest=('PromotionTests', 'ImageWorkflowTests'))
+3 -1
View File
@@ -21,6 +21,8 @@ assert_version() {
rm -f "$output"
}
assert_version "refs/tags/v1.0.0-beta.12" "v1.0.0-beta.12" "1.0.0-beta.12" "0.12.0"
assert_version "refs/tags/v1.0.0-beta.12" "v1.0.0-beta.12" "1.0.0-beta.12" "1.0.0-beta.12"
assert_version "2.0.0-beta.12" "2.0.0-beta.12" "2.0.0-beta.12" "2.0.0-beta.12"
assert_version "1.0.6-rc.1" "1.0.6-rc.1" "1.0.6-rc.1" "1.0.6-rc.1"
assert_version "v1.0.0" "v1.0.0" "1.0.0" "1.0.0"
assert_version "refs/tags/1.0.0" "1.0.0" "1.0.0" "1.0.0"
+7 -130
View File
@@ -1,135 +1,12 @@
#!/usr/bin/env python3
"""Exercise the Helm publication image gate without contacting a registry."""
import os
import subprocess
import tempfile
import unittest
"""Exercise the candidate-based replacement for the former release trigger."""
import importlib
import sys
from pathlib import Path
import unittest
ROOT = Path(__file__).resolve().parents[1]
WORKFLOW = ROOT / ".github/workflows/helm-package.yml"
def image_gate():
lines = WORKFLOW.read_text().splitlines()
start = lines.index(" - name: Wait for release image")
start = lines.index(" run: |", start) + 1
body = []
for line in lines[start:]:
if line.strip() and not line.startswith(" "):
break
body.append(line[10:])
return "\n".join(body)
class HelmReleaseWorkflowTests(unittest.TestCase):
def run_gate(self, version="1.0.1", failures=0, auth="ready"):
# Unexpected URLs fail closed; these functions cannot access the network.
mocks = r"""
curl() {
case "${*: -1}" in
'https://auth.docker.io/token?service=registry.docker.io&scope=repository:rustfs/rustfs:pull')
echo auth >> "$CALLS"
case "$AUTH_STATE" in
unavailable) return 22 ;;
empty) echo '{}' ;;
ready) echo '{"token":"registry-test-token"}' ;;
*) return 99 ;;
esac
;;
"https://registry-1.docker.io/v2/rustfs/rustfs/manifests/$APP_VERSION")
[[ " $* " == *" --head "* ]] || return 99
[[ "$*" == *"Authorization: Bearer registry-test-token"* ]] || return 99
echo image >> "$CALLS"
[[ $(grep -c '^image$' "$CALLS") -gt "$IMAGE_FAILURES" ]]
;;
*) return 99 ;;
esac
}
sleep() {
[[ "$1" == 15 ]] || return 99
echo sleep >> "$CALLS"
}
"""
with tempfile.TemporaryDirectory() as directory:
calls = Path(directory) / "calls"
result = subprocess.run(
["bash", "-e", "-o", "pipefail", "-c", mocks + image_gate()],
env={
**os.environ,
"APP_VERSION": version,
"IMAGE_FAILURES": str(failures),
"AUTH_STATE": auth,
"CALLS": str(calls),
},
capture_output=True,
text=True,
check=False,
timeout=30,
)
events = calls.read_text().splitlines() if calls.exists() else []
return result, events
def test_published_stable_and_prerelease_images(self):
for version in ("1.0.1", "1.1.0-beta.1", "1.1.0-rc.1"):
with self.subTest(version=version):
result, events = self.run_gate(version)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(events, ["auth", "image"])
def test_image_published_after_binary_build(self):
result, events = self.run_gate(failures=2)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(events.count("image"), 3)
self.assertEqual(events.count("sleep"), 2)
def test_missing_image_blocks_publication_after_bounded_retries(self):
result, events = self.run_gate(failures=60)
self.assertNotEqual(result.returncode, 0)
self.assertIn("refusing to publish Helm chart", result.stderr)
self.assertEqual(events.count("image"), 60)
self.assertEqual(events.count("sleep"), 59)
def test_auth_failure_is_not_image_availability(self):
for auth in ("unavailable", "empty"):
with self.subTest(auth=auth):
result, events = self.run_gate(auth=auth)
self.assertNotEqual(result.returncode, 0)
self.assertEqual(events.count("auth"), 60)
self.assertNotIn("image", events)
def test_preview_and_invalid_versions_never_reach_registry(self):
for version in ("1.0.1-preview.1", "1.1.0-beta.1-preview.2", "latest", "", "../tags"):
with self.subTest(version=version):
result, events = self.run_gate(version)
self.assertNotEqual(result.returncode, 0)
self.assertEqual(events, [])
def test_image_gate_precedes_packaging(self):
workflow = WORKFLOW.read_text()
gate = workflow.index(" - name: Wait for release image")
package = workflow.index(" - name: Package Helm Chart")
self.assertLess(gate, package)
self.assertNotIn("continue-on-error", workflow[gate:package])
def test_upload_artifact_includes_chart_and_readme(self):
workflow = WORKFLOW.read_text()
copy_readme = workflow.index("cp helm/README.md helm/rustfs/")
upload = workflow.index(" - name: Upload helm package as artifact")
self.assertLess(copy_readme, upload)
publish = workflow.index(" publish-helm-package:", upload)
upload_step = workflow[upload:publish]
expected_path = "path: |\n helm/rustfs/*.tgz\n helm/rustfs/README.md\n"
self.assertIn(expected_path, upload_step)
download = workflow.index(" - name: Download helm package", publish)
push_step = workflow.index(" - name: Push helm package and index file", download)
download_step = workflow[download:push_step]
self.assertIn("name: helm-package", download_step)
self.assertIn("path: ./", download_step)
sys.path.insert(0, str(Path(__file__).resolve().parent / "release"))
module = importlib.import_module("test_pipeline")
if __name__ == "__main__":
unittest.main()
unittest.main(module=module, defaultTest=('HelmWorkflowTests',))
+172
View File
@@ -0,0 +1,172 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Contract tests for junit_diff.py: exit codes, the difference table and the JSON summary."""
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
SCRIPT = Path(__file__).resolve().parent / "junit_diff.py"
FIXTURES = Path(__file__).resolve().parent / "tests" / "junit_diff"
def run(a, b, *extra):
"""Run the script on two fixture names; return (exit code, stdout, stderr)."""
proc = subprocess.run(
[sys.executable, str(SCRIPT), str(FIXTURES / a), str(FIXTURES / b), *extra],
capture_output=True, text=True, check=False)
return proc.returncode, proc.stdout, proc.stderr
class JunitDiffTests(unittest.TestCase):
# --- negative cases: differences and bad input ---
def test_pass_vs_fail_is_difference(self):
code, out, _ = run("base.xml", "pass_fail.xml")
self.assertEqual(code, 1)
self.assertIn("rustfs::s3::alpha", out)
self.assertRegex(out, r"rustfs::s3::alpha\s+pass\s+fail")
def test_pass_vs_error_counts_as_fail(self):
code, out, _ = run("base.xml", "pass_error.xml")
self.assertEqual(code, 1)
self.assertRegex(out, r"rustfs::s3::alpha\s+pass\s+fail")
def test_pass_vs_skip_is_difference(self):
code, out, _ = run("base.xml", "pass_skip.xml")
self.assertEqual(code, 1)
self.assertRegex(out, r"rustfs::s3::alpha\s+pass\s+skip")
def test_case_missing_on_b_is_difference(self):
code, out, _ = run("base.xml", "missing_case.xml")
self.assertEqual(code, 1)
self.assertRegex(out, r"rustfs::s3::beta\s+pass\s+missing")
def test_case_missing_on_a_is_difference(self):
code, out, _ = run("missing_case.xml", "base.xml")
self.assertEqual(code, 1)
self.assertRegex(out, r"rustfs::s3::beta\s+missing\s+pass")
def test_rerun_failure_final_fail_is_difference(self):
code, out, _ = run("base.xml", "rerun_fail.xml")
self.assertEqual(code, 1)
self.assertRegex(out, r"rustfs::s3::alpha\s+pass\s+fail")
def test_malformed_xml_exits_2(self):
for a, b in (("malformed.xml", "base.xml"), ("base.xml", "malformed.xml")):
code, out, err = run(a, b)
self.assertEqual(code, 2, (out, err))
self.assertNotIn("No differences", out)
def test_empty_file_exits_2(self):
code, _, err = run("base.xml", "empty.xml")
self.assertEqual(code, 2)
self.assertIn("empty.xml", err)
def test_report_without_cases_exits_2(self):
code, _, _ = run("no_cases.xml", "no_cases.xml")
self.assertEqual(code, 2)
def test_unexpected_root_exits_2(self):
code, _, _ = run("wrong_root.xml", "base.xml")
self.assertEqual(code, 2)
def test_nonexistent_file_exits_2(self):
code, _, _ = run("base.xml", "does_not_exist.xml")
self.assertEqual(code, 2)
def test_stale_allow_entry_fails_run(self):
code, out, _ = run("base.xml", "pass_fail.xml", "--allow", str(FIXTURES / "allow_stale.txt"))
self.assertEqual(code, 1)
self.assertIn("rustfs::s3::nonexistent", out)
self.assertIn("stale", out.lower())
def test_allow_entry_without_any_difference_is_stale(self):
code, out, _ = run("base.xml", "base.xml", "--allow", str(FIXTURES / "allow_ok.txt"))
self.assertEqual(code, 1)
self.assertIn("stale", out.lower())
def test_allow_does_not_cover_other_differences(self):
code, out, _ = run("base.xml", "missing_case.xml", "--allow", str(FIXTURES / "allow_ok.txt"))
self.assertEqual(code, 1)
def test_unreadable_allow_file_exits_2(self):
code, _, _ = run("base.xml", "base.xml", "--allow", str(FIXTURES / "nope.txt"))
self.assertEqual(code, 2)
# --- positive cases ---
def test_identical_files_exit_0(self):
code, out, _ = run("base.xml", "base.xml")
self.assertEqual(code, 0)
self.assertIn("No differences", out)
def test_bare_testsuite_root_matches_nested(self):
code, _, _ = run("base.xml", "bare_suite.xml")
self.assertEqual(code, 0)
def test_allowed_difference_exits_0(self):
code, out, _ = run("base.xml", "pass_fail.xml", "--allow", str(FIXTURES / "allow_ok.txt"))
self.assertEqual(code, 0)
self.assertIn("allowed", out.lower())
def test_allowed_missing_case_exits_0(self):
code, _, _ = run("base.xml", "missing_case.xml", "--allow", str(FIXTURES / "allow_missing_case.txt"))
self.assertEqual(code, 0)
def test_flaky_pass_listed_not_a_difference(self):
code, out, _ = run("base.xml", "flaky_pass.xml")
self.assertEqual(code, 0)
self.assertIn("rustfs::s3::alpha", out)
self.assertIn("flaky", out.lower())
# --- options and summary ---
def test_labels_appear_in_table(self):
code, out, _ = run("base.xml", "pass_skip.xml", "--label-a", "reference", "--label-b", "candidate")
self.assertEqual(code, 1)
self.assertIn("reference", out)
self.assertIn("candidate", out)
def test_json_summary(self):
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "out.json"
code, _, _ = run("base.xml", "flaky_pass.xml", "--json", str(path),
"--label-a", "reference", "--label-b", "candidate")
self.assertEqual(code, 0)
data = json.loads(path.read_text())
self.assertEqual(data["counts"]["reference"], {"pass": 3, "fail": 0, "skip": 1})
self.assertEqual(data["counts"]["candidate"], {"pass": 3, "fail": 0, "skip": 1})
self.assertEqual(data["differences"], [])
self.assertEqual(data["flaky"]["candidate"], ["rustfs::s3::alpha"])
self.assertEqual(data["flaky"]["reference"], [])
self.assertEqual(data["stale_allow"], [])
def test_json_records_differences(self):
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "out.json"
code, _, _ = run("base.xml", "missing_case.xml", "--json", str(path))
self.assertEqual(code, 1)
data = json.loads(path.read_text())
self.assertEqual(data["differences"],
[{"case": "rustfs::s3::beta", "a": "pass", "b": "missing", "allowed": False}])
self.assertEqual(data["counts"]["b"], {"pass": 2, "fail": 0, "skip": 1})
if __name__ == "__main__":
unittest.main()
+157
View File
@@ -0,0 +1,157 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Exercise the reference-binary pin resolver and unpacker without network access."""
import hashlib
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
import zipfile
ROOT = Path(__file__).resolve().parents[1]
SCRIPT = ROOT / ".github/actions/reference-binary/reference_binary.py"
PIN = '''schema = 1
repository = "rustfs/rustfs"
version = "1.0.5"
tag = "1.0.5"
[platforms.linux-x86_64-gnu]
asset = "rustfs-linux-x86_64-gnu-v1.0.5.zip"
sha256 = "{sha}"
executable = "rustfs"
'''
SHA = "a" * 64
def run(*args):
proc = subprocess.run([sys.executable, str(SCRIPT), *args], capture_output=True, text=True, check=False)
return proc.returncode, proc.stdout, proc.stderr
class Base(unittest.TestCase):
def setUp(self):
temp = tempfile.TemporaryDirectory()
self.addCleanup(temp.cleanup)
self.root = Path(temp.name)
def pin(self, text):
path = self.root / "pin.toml"
path.write_text(text)
return path
def archive(self, members):
path = self.root / "rustfs.zip"
with zipfile.ZipFile(path, "w") as bundle:
for name, data in members.items():
bundle.writestr(name, data)
return path, hashlib.sha256(path.read_bytes()).hexdigest()
class ResolveTests(Base):
def test_missing_pin_file_fails(self):
code, _, err = run("resolve", "--pin", str(self.root / "absent.toml"), "--platform", "linux-x86_64-gnu")
self.assertEqual(code, 1)
self.assertIn("reference pin file is missing", err)
def test_unpinned_platform_fails(self):
code, _, err = run("resolve", "--pin", str(self.pin(PIN.format(sha=SHA))), "--platform", "linux-aarch64-gnu")
self.assertEqual(code, 1)
self.assertIn("no pin for platform", err)
def test_malformed_pins_fail(self):
good = PIN.format(sha=SHA)
for label, text in {
"short digest": good.replace(SHA, "a" * 63),
"uppercase digest": good.replace(SHA, "A" * 64),
"no digest": good.replace(f'sha256 = "{SHA}"\n', ""),
"path in asset": good.replace('asset = "rustfs-', 'asset = "../rustfs-'),
"non-zip asset": good.replace("v1.0.5.zip", "v1.0.5.deb"),
"asset of another version": good.replace("v1.0.5.zip", "v1.0.4.zip"),
"path in executable": good.replace('executable = "rustfs"', 'executable = "bin/rustfs"'),
"wrong schema": good.replace("schema = 1", "schema = 2"),
"boolean schema": good.replace("schema = 1", "schema = true"),
"no repository": good.replace('repository = "rustfs/rustfs"\n', ""),
"unknown key": good + 'url = "https://example.invalid"\n',
"not toml": "schema = \n",
}.items():
with self.subTest(label):
code, _, _ = run("resolve", "--pin", str(self.pin(text)), "--platform", "linux-x86_64-gnu")
self.assertEqual(code, 1)
def test_resolves_identity_as_step_outputs(self):
code, out, err = run("resolve", "--pin", str(self.pin(PIN.format(sha=SHA))), "--platform", "linux-x86_64-gnu")
self.assertEqual(code, 0, err)
self.assertEqual(out.splitlines(), ["repository=rustfs/rustfs", "version=1.0.5", "tag=1.0.5",
"asset=rustfs-linux-x86_64-gnu-v1.0.5.zip", f"sha256={SHA}",
"executable=rustfs"])
def test_repository_pin_is_valid(self):
code, out, err = run("resolve", "--pin", str(ROOT / ".config/reference-binary.toml"),
"--platform", "linux-x86_64-gnu")
self.assertEqual(code, 0, err)
self.assertIn("version=1.0.5", out.splitlines())
class UnpackTests(Base):
def unpack(self, archive, sha):
return run("unpack", "--archive", str(archive), "--sha256", sha, "--executable", "rustfs",
"--destination", str(self.root / "out"))
def test_digest_mismatch_fails_and_extracts_nothing(self):
archive, _ = self.archive({"rustfs": b"binary"})
code, _, err = self.unpack(archive, SHA)
self.assertEqual(code, 1)
self.assertIn("does not match the pinned digest", err)
self.assertFalse((self.root / "out").exists())
def test_missing_archive_fails(self):
code, _, err = self.unpack(self.root / "absent.zip", SHA)
self.assertEqual(code, 1)
self.assertIn("missing or unreadable", err)
def test_archive_without_top_level_executable_fails(self):
archive, sha = self.archive({"bin/rustfs": b"binary"})
code, _, err = self.unpack(archive, sha)
self.assertEqual(code, 1)
self.assertIn("exactly one top-level", err)
def test_empty_executable_fails(self):
archive, sha = self.archive({"rustfs": b""})
code, _, _ = self.unpack(archive, sha)
self.assertEqual(code, 1)
def test_non_zip_archive_fails(self):
archive = self.root / "rustfs.zip"
archive.write_bytes(b"not a zip")
code, _, err = self.unpack(archive, hashlib.sha256(b"not a zip").hexdigest())
self.assertEqual(code, 1)
self.assertIn("not a valid zip", err)
def test_extracts_only_the_executable(self):
archive, sha = self.archive({"rustfs": b"binary", "../escape": b"x", "README": b"r"})
code, out, err = self.unpack(archive, sha)
self.assertEqual(code, 0, err)
target = self.root / "out" / "rustfs"
self.assertEqual(out.strip(), f"path={target}")
self.assertEqual(target.read_bytes(), b"binary")
self.assertTrue(target.stat().st_mode & 0o111)
self.assertEqual(sorted(p.name for p in (self.root / "out").iterdir()), ["rustfs"])
self.assertFalse((self.root / "escape").exists())
if __name__ == "__main__":
unittest.main()
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
"""Exercise real OCI builders and promotion with explicit non-product fixtures."""
import importlib
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import time
import unittest
import urllib.request
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parent / "release"))
c = importlib.import_module("candidate")
deliver = importlib.import_module("deliver")
publish = importlib.import_module("publish")
fixtures = importlib.import_module("test_candidate")
class ImageIntegrationTests(unittest.TestCase):
def test_native_oci_smoke_and_digest_preserving_registry_promotion(self):
value = fixtures.inputs()
variant = os.environ["RELEASE_IMAGE_VARIANT"]
self.assertIn(variant, ("musl", "gnu"))
output = Path(os.environ["RELEASE_IMAGE_OUTPUT"]).resolve()
output.mkdir(parents=True, exist_ok=True)
run = subprocess.run
with tempfile.TemporaryDirectory() as temp:
root = Path(temp)
for name in ("Dockerfile", "Dockerfile.glibc", "entrypoint.sh"):
shutil.copyfile(c.ROOT / name, root / name)
shutil.copytree(c.ROOT / "bootstrap", root / "bootstrap")
(root / "scripts").mkdir()
shutil.copyfile(c.ROOT / "scripts/download_rc.sh", root / "scripts/download_rc.sh")
source = root / "fixture.go"
source.write_text('''package main
import ("fmt"; "net/http"; "os")
func main() {
if len(os.Args) > 1 && os.Args[1] == "--version" { fmt.Println("rustfs 1.2.3"); return }
http.HandleFunc("/rustfs/console/", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html"); fmt.Fprintln(w, "<html>transport fixture</html>")
})
if err := http.ListenAndServe(":9001", nil); err != nil { panic(err) }
}
''')
binaries = {}
for arch, docker_arch in (("x86_64", "amd64"), ("aarch64", "arm64")):
directory = root / ".release-inputs" / docker_arch
directory.mkdir(parents=True)
env = dict(os.environ, GOOS="linux", GOARCH=docker_arch, CGO_ENABLED="0")
run(["go", "build", "-o", str(directory / "rustfs"), str(source)], env=env, check=True, timeout=180)
(directory / "rc").write_text("#!/bin/sh\nexit 0\n")
(directory / "rc").chmod(0o755)
binaries[arch] = c.file_digest(directory / "rustfs")
archive = output / f"image-{variant}.tar"
dockerfile = "Dockerfile" if variant == "musl" else "Dockerfile.glibc"
run(["docker", "buildx", "build", "--platform", "linux/amd64,linux/arm64", "--provenance=false",
"--build-arg", "BINARY_SOURCE=supplied", "--build-arg", "RELEASE=1.2.3", "-f", str(root / dockerfile),
"--output", f"type=oci,dest={archive}", str(root)], check=True, timeout=1800)
c.write_json(output / f"image-{variant}-inputs.json", {"inputs_sha256": c.digest(value), "binaries": binaries})
deliver.image_smoke(value, variant, output)
with self.assertRaisesRegex(ValueError, "wrong version"):
deliver.image_smoke(dict(value, version="0.0.0"), variant, output)
expected = c.read_json(output / f"image-{variant}-smoke.json")["digest"]
registry = c.command(["docker", "run", "-d", "-p", "127.0.0.1::5000", "registry:3"])
try:
address = c.command(["docker", "port", registry, "5000/tcp"])
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
deadline = time.monotonic() + 15
while time.monotonic() < deadline:
try:
with opener.open(f"http://{address}/v2/", timeout=2) as response:
self.assertEqual(response.status, 200)
break
except OSError:
time.sleep(0.1)
else:
self.fail("local registry did not become ready")
auth = root / "auth.json"
c.write_json(auth, {"auths": {}})
reference = f"{address}/fixture:1.2.3"
def local_registry(args, **kwargs):
# Only the ephemeral loopback registry uses HTTP. All OCI
# copying, digest checks and conflict handling are real.
if args[:2] == ["skopeo", "inspect"]:
args = args[:2] + ["--tls-verify=false"] + args[2:]
elif args[:2] == ["skopeo", "copy"]:
args = args[:2] + ["--dest-tls-verify=false"] + args[2:]
return run(args, **kwargs)
with patch.dict(os.environ, REGISTRY_AUTH_FILE=str(auth)), patch.object(subprocess, "run", side_effect=local_registry):
publish.copy_image(archive, reference, expected)
publish.copy_image(archive, reference, expected)
self.assertEqual(publish.image_digest(reference), expected)
with self.assertRaisesRegex(ValueError, "never replace"):
publish.copy_image(archive, reference, "sha256:" + "0" * 64)
self.assertEqual(publish.image_digest(reference), expected)
finally:
run(["docker", "rm", "-f", "-v", registry], check=True, timeout=60)
if __name__ == "__main__":
unittest.main()
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env python3
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
"""Linux integration tests using the real DEB/RPM recipes and package managers."""
import hashlib
import importlib
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parent / "release"))
deliver = importlib.import_module("deliver")
c = importlib.import_module("candidate")
fixtures = importlib.import_module("test_candidate")
class PackageIntegrationTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
missing = [tool for tool in ("dpkg-deb", "fakeroot", "rpm", "rpm2cpio", "cpio", "fpm") if not shutil.which(tool)]
if sys.platform != "linux" or missing:
raise RuntimeError(f"Run on Linux with real packaging tools; missing: {missing}")
def test_each_target_wraps_its_accepted_bytes(self):
for target in c.LINUX:
with self.subTest(target=target), tempfile.TemporaryDirectory() as temp:
root = Path(temp)
executable = root / "rustfs"
executable.write_bytes(b"package contract fixture only: " + target.encode())
expected = hashlib.sha256(executable.read_bytes()).hexdigest()
value = fixtures.inputs()
with patch.object(deliver, "binary", return_value=(executable, {"binary_sha256": expected})):
deliver.package(value, target, root / "output")
record = c.read_json(root / "output" / f"packages-{target}.json")
self.assertEqual(record["binary_sha256"], expected)
self.assertEqual(record["inputs_sha256"], c.digest(value))
self.assertEqual({Path(item["name"]).suffix for item in record["files"]}, {".deb", ".rpm"})
def test_real_package_with_substituted_payload_is_rejected(self):
with tempfile.TemporaryDirectory() as temp:
root = Path(temp)
executable = root / "rustfs"
executable.write_bytes(b"accepted fixture")
expected = c.file_digest(executable)
real_run = deliver.run
def mutate_deb(args, **kwargs):
result = real_run(args, **kwargs)
if any(str(arg).endswith("build_deb.sh") for arg in args):
work = Path(kwargs["cwd"])
package = work / kwargs["env"]["DEB_FILE"]
unpacked = work / "corrupted-package"
subprocess.run(["dpkg-deb", "--raw-extract", str(package), str(unpacked)], check=True)
(unpacked / "usr/bin/rustfs").write_bytes(b"wrong server")
subprocess.run(["dpkg-deb", "--build", str(unpacked), str(package)], check=True)
return result
with patch.object(deliver, "binary", return_value=(executable, {"binary_sha256": expected})), patch.object(deliver, "run", side_effect=mutate_deb):
with self.assertRaisesRegex(ValueError, "package binary differs"):
deliver.package(fixtures.inputs(), "linux-x86_64-musl", root / "output")
self.assertFalse(list((root / "output").glob("packages-*.json")))
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1 @@
rustfs::s3::beta
+3
View File
@@ -0,0 +1,3 @@
# tolerated while the gateway lane stabilises
rustfs::s3::alpha
+3
View File
@@ -0,0 +1,3 @@
rustfs::s3::alpha
rustfs::s3::nonexistent
+6
View File
@@ -0,0 +1,6 @@
<testsuite name="rustfs" tests="4">
<testcase name="alpha" classname="rustfs::s3"/>
<testcase name="beta" classname="rustfs::s3"/>
<testcase name="gamma" classname="rustfs::s3"><skipped/></testcase>
<testcase name="delta" classname="rustfs::iam"/>
</testsuite>
+11
View File
@@ -0,0 +1,11 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="nextest-run">
<testsuite name="rustfs" tests="3">
<testcase name="alpha" classname="rustfs::s3"/>
<testcase name="beta" classname="rustfs::s3"/>
<testcase name="gamma" classname="rustfs::s3"><skipped/></testcase>
</testsuite>
<testsuite name="other" tests="1">
<testcase name="delta" classname="rustfs::iam"/>
</testsuite>
</testsuites>
View File
+6
View File
@@ -0,0 +1,6 @@
<testsuites><testsuite name="rustfs">
<testcase name="alpha" classname="rustfs::s3"><flakyFailure message="first try"/></testcase>
<testcase name="beta" classname="rustfs::s3"/>
<testcase name="gamma" classname="rustfs::s3"><skipped/></testcase>
<testcase name="delta" classname="rustfs::iam"/>
</testsuite></testsuites>
+1
View File
@@ -0,0 +1 @@
<testsuites><testsuite><testcase name="alpha" classname="x">
@@ -0,0 +1,5 @@
<testsuites><testsuite name="rustfs">
<testcase name="alpha" classname="rustfs::s3"/>
<testcase name="gamma" classname="rustfs::s3"><skipped/></testcase>
<testcase name="delta" classname="rustfs::iam"/>
</testsuite></testsuites>
+1
View File
@@ -0,0 +1 @@
<testsuites><testsuite name="rustfs"/></testsuites>
+6
View File
@@ -0,0 +1,6 @@
<testsuites><testsuite name="rustfs">
<testcase name="alpha" classname="rustfs::s3"><error message="crash"/></testcase>
<testcase name="beta" classname="rustfs::s3"/>
<testcase name="gamma" classname="rustfs::s3"><skipped/></testcase>
<testcase name="delta" classname="rustfs::iam"/>
</testsuite></testsuites>
+6
View File
@@ -0,0 +1,6 @@
<testsuites><testsuite name="rustfs">
<testcase name="alpha" classname="rustfs::s3"><failure message="boom"/></testcase>
<testcase name="beta" classname="rustfs::s3"/>
<testcase name="gamma" classname="rustfs::s3"><skipped/></testcase>
<testcase name="delta" classname="rustfs::iam"/>
</testsuite></testsuites>
+6
View File
@@ -0,0 +1,6 @@
<testsuites><testsuite name="rustfs">
<testcase name="alpha" classname="rustfs::s3"><skipped/></testcase>
<testcase name="beta" classname="rustfs::s3"/>
<testcase name="gamma" classname="rustfs::s3"><skipped/></testcase>
<testcase name="delta" classname="rustfs::iam"/>
</testsuite></testsuites>
+6
View File
@@ -0,0 +1,6 @@
<testsuites><testsuite name="rustfs">
<testcase name="alpha" classname="rustfs::s3"><failure message="boom"/><rerunFailure message="again"/></testcase>
<testcase name="beta" classname="rustfs::s3"/>
<testcase name="gamma" classname="rustfs::s3"><skipped/></testcase>
<testcase name="delta" classname="rustfs::iam"/>
</testsuite></testsuites>
+1
View File
@@ -0,0 +1 @@
<html><body/></html>