Compare commits

...

27 Commits

Author SHA1 Message Date
Chris 8741bb77ea fix(storage): weight automatic multipart admission by part size (#8118)
* fix(storage): weight automatic multipart admission by part size

* fix(ci): restore filesystem runner capabilities and typos dependency

* fix(ci): make release guard portable and spell out part variables

* ci: restore sm-standard-2 runners for io_uring and distributed e2e jobs

---------

Co-authored-by: hector <42570491+majinghe@users.noreply.github.com>
Co-authored-by: Hauser <housemecn@gmail.com>
2026-09-26 22:35:39 +08:00
cxymds 212b20f890 fix(ecstore): admit cold bucket metadata at read quorum (#8120)
* fix(ecstore): admit cold bucket metadata at read quorum

Encode a durable creation-commit state in bucket metadata so committed Object Lock buckets can use read quorum for cold loads while pre-physical creation intents remain fail-closed. Drain commit-write fan-out, keep write quorum for uncommitted intents, and add regression coverage for exact, below, migrated, and partial-create quorum boundaries.

* fix(ecstore): fence bucket creation-commit persistence

Address review findings on the creation-commit proof.

Persist the proof only under the bucket metadata transaction fence: at bucket creation, and through a fenced migration that re-reads the authoritative metadata and revalidates physical presence at write quorum before writing. This stops a stale snapshot from reverting an acknowledged configuration update or outliving a delete/recreate.

Establish commitment when Object Lock is enabled on an existing bucket, inside the same configuration mutation, so a healthy cluster no longer rejects object operations with ErasureWriteQuorum.

* fix(ecstore): keep commit fence error message stable

The error(format!) ratchet requires a stable Display for quorum bucketing; use a fixed message instead of embedding the bucket name.

* test(ecstore): reuse canonical Object Lock fixture in regression tests

The s3s footprint ratchet is shrink-only. Use the existing ENABLED_OBJECT_LOCK_CONFIG static instead of naming s3s DTO types in store tests.

---------

Co-authored-by: hector <42570491+majinghe@users.noreply.github.com>
Co-authored-by: Hauser <housemecn@gmail.com>
2026-09-26 21:40:40 +08:00
hector 7d3d0b6c2b ci: align the matrix contract with the 36-combination sweep (#8136)
Companion to rustfs/auto-testing#109: the 4x1 topology is removed from
MATRIX_TOPOS because the product rejects a 1-drive-per-node distributed
pool at startup (FATAL BelowMinimum). The unfiltered sweep is now 36
combinations / 300 cases; update the contract totals and comments so an
unfiltered sweep is still held to an exact expected case count.
2026-09-26 20:31:32 +08:00
hector d317bb3274 ci: run the KMS Vault lanes on ubuntu-latest again (#8133)
#8112 replaced ubuntu-latest with sm-standard-2 across workflows. Both nightly
KMS Vault lanes need a working Docker daemon; the sm-standard-2 pool does not
provide one, so both lanes die in seconds on 'Docker is not available' while
the package compiles and publishes fine - and the functional chain never fires
because its gate requires a successful build run. Restore the lanes to
ubuntu-latest, as documented in the lane comment before #8112.
2026-09-26 19:12:18 +08:00
Hauser 5cac685e9a feat(ecstore): integrate io_uring backend improvements (#8105) 2026-09-26 17:14:17 +08:00
Hauser d8bf268885 fix(scanner): bound SNSD deep scans and checkpoint cloning (#8126)
* fix(scanner): bound SNSD deep scans and checkpoint cloning

* ci: run mount-dependent jobs on hosted VMs

---------

Co-authored-by: hector <42570491+majinghe@users.noreply.github.com>
2026-09-26 06:06:50 +00:00
hector fdd753a1dd ci: run distributed e2e on ubuntu-latest to restore tmpfs mounts (#8128) 2026-09-26 12:00:26 +08:00
Chris de80e8a673 fix(ci): make shared checks portable across runners (#8124)
* fix(ci): isolate monitor argument checks from runner tools

* fix(ci): install the Typos action download dependency

* fix(ci): make release policy matching portable across awk variants
2026-09-26 10:55:06 +08:00
RustFS 4cf45e9ed2 Update bug report storage requirements: allow SAN/JBOD, recommend XFS (#8125)
docs(github): clarify supported storage in bug reports

State that local disks, SAN/Fibre Channel volumes, and JBOD are supported,
recommend XFS, and keep the existing unsupported remote/shared filesystem rule.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-09-26 10:43:33 +08:00
Chris 9f8acf688c feat(github): add issue forms with storage support checks (#8123) 2026-09-26 10:07:49 +08:00
Chris 96fc26cd6a fix(release): defer installation updates until artifacts are live (#8122) 2026-09-26 09:17:15 +08:00
hector edcc81a8fd ci: replace ubuntu-latest runner with sm-standard-2 across workflows (#8112)
* ci: replace ubuntu-latest runner with sm-standard-2 across workflows

* ci: keep scheduled-validation monitors on hosted runners

The freshness and watchdog jobs report stalled scheduled validations.
Running them on the same sm-standard-2 pool means a pool outage stalls
the monitors too, so nothing reports it.

---------

Co-authored-by: overtrue <anzhengchao@gmail.com>
2026-09-25 10:02:45 +08:00
Hauser 286bbb8e1f chore: upgrade workspace dependency releases (#8107)
* chore: upgrade workspace dependencies

* chore: refresh dependency lockfile

* fix(rebalance): avoid waiting on active entry gate during stop preparation
2026-09-24 21:24:14 +08:00
cxymds 0913686b12 fix(heal): detect stale delete-marker metadata (#8102) 2026-09-24 11:16:02 +08:00
George Melikov df64242f64 fix(ecstore): resume scan_dir past a dash-suffixed sibling directory (#8071)
A flat (recursive) listing resumed inside "s/" re-emitted everything in
its sibling "s-x/". scan_dir dropped the entries before forward_to by
comparing directory names without their trailing slash, where "s" sorts
before "s-x", while the keys they stand for sort the other way round:
'-' (0x2d) is below '/' (0x2f), so all of "s-x/..." precedes "s/...".
The drain stopped at "s" and kept "s-x".

The next page then started with keys at or before the marker; the
listing layer filtered all of them out, found no more candidates and
answered IsTruncated=false. A bucket of 104,137 objects with backup
directories named "<id>" and "<id>-rollbacks" listed as 5,000; two such
directories of 1,200 keys each listed as 2,000.

Compare every remaining entry as the key prefix it stands for, slash
included, and keep it only when it sorts at or after forward_to or
contains it. The remainder of forward_to is taken before `current` is
trimmed, so this also holds below the bucket root; retain also drops
entries when all of them precede forward_to, which the drain never did.
2026-09-24 07:13:30 +08:00
Chris d3b75e695e test(connect): add scheduler receipt acceptance workflow (#8078)
* test(connect): add scheduler receipt acceptance workflow

* chore(deps): upgrade crates and fix faster-hex advisory (#8098)

* test(connect): isolate scheduler acceptance workflow

---------

Co-authored-by: Hauser <housemecn@gmail.com>
2026-09-24 03:34:27 +08:00
Chris dde1aa77a1 fix(connect): preserve diagnostic schedules across restart (#8092) 2026-09-24 03:28:37 +08:00
Chris be01e513be feat(connect): sample memory within the running service (#8091)
* feat(connect): sample memory within the running service

* test(connect): add official memory service acceptance

* test(connect): consume the final memory job without cloning
2026-09-24 03:25:28 +08:00
cxymds 9ab034df62 fix(heal): preserve retryable writer failures (#8089)
Co-authored-by: Hauser <housemecn@gmail.com>
2026-09-24 02:41:16 +08:00
Tyler Hillery 624463c5ba fix(s3): default CopyObject content-type to binary/octet-stream on REPLACE (#8073)
fix: default CopyObject content-type to binary/octet-stream on REPLACE
2026-09-23 18:04:05 +00:00
Chris 713920dcd6 docs: update license copyright to RustFS, Inc. (#8097) 2026-09-24 00:56:08 +08:00
GatewayJ f219a0aba8 perf(ecstore): parallelize multipart I/O setup and metadata reads (#8085)
* perf(ecstore): parallelize multipart I/O setup and metadata reads

* perf(ecstore): share multipart paths and increase read concurrency

* fix(ecstore): route multipart benchmark through storage API facade
2026-09-23 22:19:23 +08:00
Hauser 311e4306e6 chore(ci): complete action pin and HAProxy upgrades (#8090) 2026-09-23 19:20:39 +08:00
Hauser 7f0f4941cf chore(ci): refresh checkout pin in validated workflows (#8083)
* chore(ci): update checkout pin in validated workflows

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ci): pin migrated checkout alerts to requested commit

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* chore(ci): pin additional checkout workflows to verified commit (#8084)

* chore(ci): pin more checkout workflows to requested commit

Update six additional workflows to the verified upstream checkout commit without changing their permissions, inputs, or triggers.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* chore(ci): pin functional and OIDC checkout uses

Extend the verified checkout commit pin to functional-chain and OIDC workflows without changing their behavior.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>

---------

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-09-23 09:27:54 +00:00
cxymds 655bb470b8 fix(heal): retry faulty storage disk errors (#8086) 2026-09-23 07:48:39 +00:00
cxymds 56a4099f99 fix(heal): recreate missing bucket volumes (#8082)
* fix(heal): recreate missing bucket volumes

* fix(heal): use typed error for empty targets
2026-09-23 14:51:24 +08:00
hector ee6de7d786 ci(package): build gnu and musl DEB/RPM variants with distinct file names (#8079)
* ci(package): build gnu and musl DEB/RPM variants with distinct file names

The Build and Release workflow produces four Linux binaries
(x86_64-gnu, aarch64-gnu, x86_64-musl, aarch64-musl), but packaging
only consumed the two gnu artifacts. Add matrix entries for the two
musl artifacts so every release ships all four DEB/RPM variants.

The libc variant is now part of the package file names, which would
otherwise collide between gnu and musl builds of the same version:

- deb: rustfs_<version>_<libc>_<arch>.deb
- rpm: rustfs-<libc>-<version>-<release>.<arch>.rpm

The dpkg Package and rpm Name stay plain "rustfs", so gnu and musl
remain mutually exclusive upgrades of one package rather than
co-installable packages fighting over /usr/bin/rustfs.

Dependency declarations now follow the linkage: gnu binaries
dynamically link glibc and keep Depends: libc6 (>= 2.31) /
glibc >= 2.31; musl binaries are statically linked and declare no
libc dependency. The libc variant is also visible in the package
description.

scripts/release/package_versions.sh gains a LIBC argument and its
contract tests cover both variants plus the invalid-libc cases.

* ci(package): align deb/rpm file names with the zip artifact naming

Rename the package file names so every release asset of one build
shares the same stem as its binary artifact, differing only by
extension:

- before: rustfs_<deb_version>_<libc>_<deb_arch>.deb
          rustfs-<libc>-<rpm_version>-<rpm_release>.<rpm_arch>.rpm
- after:  rustfs-linux-<arch>-<libc>-v<version>.deb / .rpm

e.g. rustfs-linux-x86_64-gnu-v1.0.0.zip,
     rustfs-linux-x86_64-gnu-v1.0.0.deb,
     rustfs-linux-x86_64-gnu-v1.0.0.rpm.

Non-development builds embed the raw release tag (with 'v'), like the
zips; development builds embed dev-<full sha>. The dpkg/rpm versions
(including the '~' prerelease ordering) are unchanged - they live in
the package metadata, and a side effect is that release asset names no
longer contain '~' (which GitHub normalizes to '.').

package_versions.sh now takes the target arch (x86_64|aarch64) instead
of the deb/rpm arch pair; the deb Architecture (amd64/arm64) in the
control metadata still comes from the workflow matrix. The two test
workflows that assemble deb download URLs from a release tag
(rustfs-table-test, rustfs-upgrade-test) are updated to the new name,
which also removes their '~'-to-'.' asset name workaround.
2026-09-23 03:17:47 +00:00
146 changed files with 6987 additions and 1289 deletions
+22 -13
View File
@@ -1,6 +1,6 @@
---
name: rustfs-release-publish
description: "Run the end-to-end RustFS console gate, version bump, preview validation, human confirmation, and final-tag publication pipeline. Use only when the user explicitly asks to release or publish a RustFS version (发版/发布)."
description: "Run the RustFS console gate, source version bump, preview validation, human confirmation, final-tag publication, and post-release installation and website updates. Use only when the user explicitly asks to release or publish a RustFS version (发版/发布)."
---
# RustFS Release Publish (preview-validated pipeline)
@@ -18,16 +18,18 @@ Pipeline shape:
```
check console main against its latest Release
-> if ahead: publish console -> wait for Release asset + latest API
-> bump RustFS version files to <target> (final version, ONE commit) -> merge
-> bump Cargo.toml and Cargo.lock to <target> -> merge
-> tag <preview-tag> at that commit -> CI green
-> verify preview Release assets -> run binary locally + console checks
-> validate with latest rc client
-> report preview acceptance results -> STOP for explicit human confirmation
-> tag <target> at the SAME commit (zero delta) -> re-verify CI/release
-> CI deletes the <target>-preview.N Releases (tags kept)
-> verify published images -> publish Helm chart from the validated source
-> update installation references on main -> update rustfs.com announcement
```
On validation failure: fix lands on main via normal PR (version files are already at `<target>`, no new bump PR), then tag `<preview-tag N+1>` at the new main commit and restart from Phase 2.
On preview validation failure: fix lands on main via normal PR (Cargo versions are already at `<target>`, no new source bump PR), then tag `<preview-tag N+1>` at the new main commit and restart from Phase 2. Installation references remain on the previous published deliverable throughout preview validation.
## Required inputs
@@ -56,19 +58,21 @@ Rules:
- Use `<target>-preview.N` for every target, e.g. `1.0.0-beta.10-preview.3` or `1.1.0-preview.1`.
- The canonical suffix is exactly `-preview.<digits>`. `build.yml` recognizes it before alpha/beta/rc classification and routes it to the preview-only path; any other tag containing `-preview` fails closed instead of being treated as a release.
- A preview Release MUST be published with `isPrerelease=true` and `isLatest=false`. Any `*-latest` preview asset or preview-triggered `latest.json`, R2, Docker, or Helm publication is a pipeline failure.
- A preview Release MUST be published with `isPrerelease=true` and `isLatest=false`. Any `*-latest` preview asset or preview-triggered `latest.json`, R2, Helm, or moving Docker channel tag is a pipeline failure. Docker images may use the exact preview version tag, including its existing variant suffixes.
- Preview Releases are cleaned up by the `cleanup-preview-releases` job after `publish-release` succeeds for the deliverable tag. It deletes every Release whose tag is exactly `<target>-preview.<digits>` and never passes `--cleanup-tag`, so the tags survive.
## Hard rules
- Version files (Cargo.toml, Cargo.lock, README, flake.nix, Chart.yaml, rustfs.spec) are bumped ONCE, directly to `<target>`. Never write a `-preview.N` suffix into any version file. If `rustfs-release-version-bump` is ever asked for a `-preview` version, that is a pipeline bug — stop.
- Before preview, bump only Cargo.toml (workspace package and internal dependency versions) and the corresponding workspace members in Cargo.lock, directly to `<target>`. Keep README installation examples, flake.nix, Chart.yaml, and rustfs.spec on the previous published deliverable until Phase 7. Never write a `-preview.N` suffix into these files; preview identifiers belong to validation tags and artifacts.
- Publishing a tag alone does not make an installation target available. Verify the exact image manifest, Release assets, and source archive before advancing references that consume them. Helm packaging derives its versions from the final tag and waits for that image; it does not require an early Chart.yaml bump on main.
- Preview Release assets are versioned and intentionally visible on the Releases page for the duration of validation. Do not label them Latest or use them to update any latest distribution channel.
- Never delete a preview Release by hand before Phase 6 finishes — Phase 4 downloads its assets and the final Release notes are generated while it still exists. Cleanup is CI's job; only step in manually (`gh release delete "<preview-tag>" --yes`, never `--cleanup-tag`) if `cleanup-preview-releases` failed.
- Tags have no `v` prefix. Always annotated: `git tag -a <tag> -m "Release <tag>"`.
- The final tag MUST point at exactly `PREVIEW_HASH` — the commit the validated preview tag points at. Never tag current `main` HEAD (commits merged after validation are unvalidated), and never create an extra version-bump commit between preview and final.
- The final tag MUST point at exactly `PREVIEW_HASH` — the commit the validated preview tag points at. Never tag current `main` HEAD (commits merged after validation are unvalidated), and never create an extra version-bump commit between preview and final. Phase 7 updates installation references on main after publication; neither tag moves to that follow-up commit.
- When a previous deliverable exists, GitHub Release notes for the preview and final tags MUST use it as their shared comparison baseline: the most recently published non-preview Release before the target. Internal `-preview.N` Releases are explicitly excluded from that selection, even when they point at the same commit as the final tag — cleanup runs after the notes are generated, so the preview Release is still present and would otherwise be picked as the baseline. If no previous deliverable exists, omit `previous_tag_name` and record that GitHub's default baseline fallback was used.
- Generated Release notes carry a workflow-management marker so retries can repair them. Before manually curating a generated body, remove that marker; unmarked non-placeholder notes are preserved by later workflow runs.
- Phases run in order; a failure in any phase blocks everything after it. After the fix lands on main, restart from Phase 2 with the next preview iteration against the new `origin/main` hash — do not resume mid-pipeline against a stale hash.
- Phases run in order; a failure blocks dependent steps. For a preview/source acceptance failure before the final tag exists, land the fix, verify Cargo still matches the confirmed target, and restart from Phase 2 with the next preview at the fixed commit. If main has advanced to another target, resolve that source/version mismatch before another preview.
- Once the final tag exists, preserve its commit. Retry failed publication jobs for that exact tag; do not recreate the tag or restart preview on newer main. If a source change is required after final-tag publication, report the failure and obtain a new target version. Phase 7 failures resume only the failed follow-up after rechecking artifact availability and whether a newer release has superseded it.
- Completing preview acceptance does not authorize the final tag. After Phases 3–5 pass, report the acceptance evidence and stop until the user explicitly confirms continuation. The original release request, an earlier confirmation, silence, or an automated follow-up does not satisfy this gate.
- Confirmation is scoped to the reported `<target>`, `<preview-tag>`, and `PREVIEW_HASH`. A failed or repeated acceptance cycle, including any new preview iteration, invalidates prior confirmation and requires a new one.
- If the release is abandoned after Phase 1 merged, main's version files claim a version that was never tagged. Either revert the bump PR or leave it to be overwritten by the next release — but tell the user explicitly and record the decision.
@@ -84,10 +88,10 @@ Rules:
Read and complete [the Console gate](references/console-gate.md) before Phase 1. Verify the latest published Console asset and exact commit; if Console main is ahead, complete its release and asset verification first. A successful build alone does not satisfy this gate.
## Phase 1 — Version bump to the final target (once)
## Phase 1 — Source version bump to the final target (once)
- If main's version files already read `<target>` (e.g. this is a restart after a failed preview), verify with `rg -n "<target>" Cargo.toml rustfs.spec helm/rustfs/Chart.yaml` and skip to Phase 2.
- Otherwise invoke the `rustfs-release-version-bump` skill with the final `<target>` (NOT a preview version), full GitHub flow (commit/push/PR).
- If Cargo.toml and all workspace members in Cargo.lock already read `<target>` (e.g. this is a restart after a failed preview), verify both and skip the source bump. Installation versions intentionally differ during preview; do not treat them as leftovers. If a previous preparation advanced installation references to an unavailable target, restore those references to the verified published baseline before selecting the next preview commit.
- Otherwise invoke the `rustfs-release-version-bump` skill with stage `prepare`, the final `<target>` (NOT a preview version), and full GitHub flow (commit/push/PR).
- Get the PR merged into main. Record the resulting main commit:
```bash
@@ -106,7 +110,7 @@ git push origin "<preview-tag>"
Pushing the tag triggers `.github/workflows/build.yml` ("Build and Release"); `docker.yml` chains off it via `workflow_run`.
The preview run builds versioned artifacts and publishes them in a GitHub prerelease. Its latest-channel, R2, Docker, and Helm jobs must be skipped. Those publication paths run only after the final tag is pushed.
The preview run builds versioned artifacts and publishes them in a GitHub prerelease. Docker may publish exact preview image tags. Latest-channel, R2, and Helm publication must be skipped; installation references and the website announcement stay unchanged.
On a restart (N+1), refresh `PREVIEW_HASH=$(git rev-parse origin/main)` first — it must contain the fix — and re-report it.
@@ -116,7 +120,7 @@ On a restart (N+1), refresh `PREVIEW_HASH=$(git rev-parse origin/main)` first
- Confirm the Release publication jobs (`create-release`, `upload-release-assets`, and `publish-release`) succeed while `update-latest-version` is skipped.
- Verify `gh release view "<preview-tag>" --json isPrerelease,assets,url`: `isPrerelease` must be `true`, and the Release must contain all 6 versioned platform zips, checksums, SBOM, and provenance with no `-latest` assets. Confirm `gh api repos/{owner}/{repo}/releases/latest --jq .tag_name` does not return `<preview-tag>`.
- Record `PREVIOUS_DELIVERABLE`, selected from published Releases by `publishedAt` after excluding the current tag and every `-preview.N` tag. Verify `gh release view "<preview-tag>" --json body --jq .body` contains `## What's Changed` and, when `PREVIOUS_DELIVERABLE` exists, `**Full Changelog**: https://github.com/rustfs/rustfs/compare/<PREVIOUS_DELIVERABLE>...<preview-tag>`. For a repository with no previous deliverable, verify a Full Changelog link exists and record the GitHub baseline fallback.
- Confirm preview-triggered Docker and Helm jobs are skipped. Preview validation covers the built RustFS binaries, embedded console, and rc compatibility; Docker image construction and Helm publication are deferred to the final tag because the Dockerfiles consume GitHub Release assets.
- Confirm Helm is skipped. If preview Docker images are published, confirm they use only exact preview tags (and variant suffixes), with no `latest`, `alpha`, `beta`, or `rc` channel updates. Preview validation covers the built RustFS binaries, embedded console, and rc compatibility; it does not authorize advancing any installation default.
## Phases 4–5 — Local artifact, Console, and rc acceptance
@@ -130,7 +134,7 @@ Continue to Phase 6 only after a new user reply explicitly confirms the reported
## Phase 6 — Publish the final tag on the validated commit
No second version bump, no release branch. The final tag goes on the exact commit the preview validated:
No second source version bump. The final tag goes on the exact commit the preview validated:
```bash
git fetch origin --tags
@@ -145,6 +149,10 @@ git push origin "<target>"
- Verify the preview cleanup: `cleanup-preview-releases` must succeed, `gh release view "<preview-tag>"` must then report `release not found` for every preview iteration of this target, and `git rev-parse "<preview-tag>^{commit}"` must still resolve to `PREVIEW_HASH` (the tag is kept). If the job failed, delete the leftover Releases manually with `gh release delete "<preview-tag>" --yes` and report it.
- Optionally spot-check `./rustfs --version` from a final-tag artifact — it must report `<target>`.
## Phase 7 — Installation references and website announcement
Only after Phase 6 succeeds, complete [post-release updates](references/post-release-updates.md): align installation references on main, then update the existing top banner in `rustfs/rustfs.com`. A failed or incomplete publication leaves both on the previous available version. These follow-up commits never change `PREVIEW_HASH` or either release tag. Track their PR, merge, and deployment states separately from artifact publication; an open PR is not a live website update.
## Output contract
Always report:
@@ -153,4 +161,5 @@ Always report:
- Target version, preview tag(s) used, `PREVIEW_HASH` (which both tags point at).
- Manual confirmation gate status (`WAITING_FOR_CONFIRMATION` or `CONFIRMED`) and its exact target, preview tag, and `PREVIEW_HASH`.
- Per-phase result (PASS/FAIL/BLOCKED) with key evidence: preview and final Release URLs, preview `isPrerelease`/`isLatest` state, final latest-channel state, console check results, the rc command matrix, and the preview-Release cleanup result (deleted Releases plus surviving tags).
- Post-release installation PR and verification results; website banner target, text, link, PR, and observed deployment state. Report any remaining merge authorization or failed deployment explicitly rather than claiming the banner is live.
- Any deviation from this pipeline and why the user approved it.
@@ -0,0 +1,32 @@
# Post-release Installation and Website Updates
Read only after the final non-preview tag has passed Phase 6. Tag creation or a green binary build alone is insufficient: the release assets, source archive, container images, Helm package, and applicable latest channels must be available. Preview tags never enter this phase.
## Installation references
1. Re-read the published deliverable state before edits or a retry. If a newer release has superseded this target, do not downgrade installation references or the website announcement. Report the superseding release and leave those defaults intact.
2. Invoke `rustfs-release-version-bump` with stage `post-release`, the exact target, and the authorized delivery scope. Prepare the follow-up on current main; do not edit or move either validated release tag. Cargo versions may already be preparing the next target and must remain untouched.
3. Verify the exact Docker image manifest, Release download links, RPM source archive, and Helm repository entry before publishing the installation PR. The chart workflow overrides chart/app versions from the final tag, pins the source checkout to that release, and waits for its default Docker image before packaging.
4. Report the installation PR and its merge state separately from the completed release. Reuse existing merge authorization and repository checks; do not infer new merge authority from this reference.
## rustfs.com top banner
Use the existing announcement configuration from [website PR #112](https://github.com/rustfs/rustfs.com/pull/112). Do not add a second banner or redesign the header.
1. In an isolated checkout of `rustfs/rustfs.com`, read its current `AGENTS.md`, PR template if present, `data/announcement.ts`, and the consuming header component. Search for an existing announcement PR for this target before creating another.
2. Update `homeAnnouncement` in `data/announcement.ts` after the release and installation availability checks pass. Keep `enabled: true`, `badge: 'New'`, and the existing component. For a stable release, use:
```ts
export const homeAnnouncement = {
enabled: true,
badge: 'New',
message: 'RustFS <target> is now available! Please',
linkText: 'upgrade and try',
href: '/download',
} as const
```
Replace `<target>` with the exact published version. Reserve GA wording for the actual first stable release. For a final alpha/beta/rc tag, explicitly call it a prerelease; use its GitHub Release URL if `/download` does not expose that target. Never announce a `-preview.N` build here.
3. Use the site's current package manager and required checks. For this config-only change, type-check and lint the changed file, build the site, and inspect the generated homepage for the version, visible link text, and destination. Confirm `/download` actually offers the announced version before using that link.
4. Commit, push, and create or update the website PR when included in the authorized release scope. Include the published RustFS Release URL and checks run. Keep merge and production-deployment authority separate; reuse permission already given, but this reference grants none by itself.
5. After an authorized merge/deployment, verify the live `https://rustfs.com/` banner and its destination. If only the PR is ready, report that state and URL; do not report the banner as deployed. A website failure does not invalidate or move the published release tag: retry this phase after resolving the failure.
@@ -1,16 +1,15 @@
---
name: rustfs-release-version-bump
description: "Prepare the version-file and release-asset bump for an exact RustFS alpha/beta/stable target, with verification and optional commit/push/PR delivery. Use for an explicit version bump or when invoked by the release-publish workflow."
description: "Prepare Cargo versions for an exact RustFS target, or align installation references after its artifacts are published, with verification and optional commit/push/PR delivery. Use for an explicit version bump or when invoked by the release-publish workflow."
---
# RustFS Release Version Bump
Use this skill to prepare and verify release version files. Commit, push, and PR steps apply only when included in the user's delivery scope; publishing release tags belongs to `rustfs-release-publish`.
Validated baseline: release pattern used in PR `#2957`.
## Required inputs
- Exact target version, for example `1.0.0-beta.4`.
- Stage: `prepare` (default) or `post-release`. Infer `post-release` only when the user or parent release workflow explicitly requests installation updates after publication; a normal version bump means `prepare`.
- Delivery scope: local (`edit/verify`), git (`commit/push`), or GitHub
(`commit/push/PR`). Derive it from the conversation; when unspecified, prepare
and verify locally without blocking on a delivery question.
@@ -25,25 +24,32 @@ Reject any target version containing `-preview`: preview identifiers are tag-onl
- `.github/pull_request_template.md` only when preparing a PR.
- Current branch status and diff against `origin/main`.
## Default release file scope
## Stage boundaries
Treat the following file list as the default checklist for each release bump:
`prepare` updates only:
- `Cargo.toml`
- `Cargo.lock`
`post-release` updates installation and packaging references only:
- `README.md`
- `README_ZH.md`
- `flake.nix`
- `helm/rustfs/Chart.yaml`
- `rustfs.spec`
Only drop a file when the current repository release process clearly no longer requires it.
During preview, installation references intentionally retain the previous published deliverable. Do not sweep them into a source bump to make every version string match. `flake.nix` builds local source, but its package label is aligned with the other packaging references after publication.
Before `post-release` edits, verify that the exact non-preview target has a published GitHub Release, downloadable assets and source archive, and a pullable `rustfs/rustfs:<target>` image manifest. If any prerequisite is missing, report `BLOCKED` and leave installation references unchanged. If a newer deliverable has already superseded this target, do not downgrade installation defaults during a retry.
Helm CI derives chart versions from the triggering tag, so the final tag can retain the previous Chart.yaml values. Update the repository copy only after the target image and published chart are available. Neither post-release changes nor their merge commit may replace the preview-validated final tag.
## Hard release policy
- Docker doc tags use `<version>` (for example `rustfs/rustfs:1.0.0-beta.4`), not `v<version>`.
- Helm chart version mapping follows `beta.N -> 0.N.0`.
- `rustfs.spec` `Release` uses prerelease suffix only (for example `beta.4`).
- Derive Helm chart and app versions with `scripts/helm_chart_version.sh <target>`; the existing mapping is `beta.N -> 0.N.0`, with other target versions retained.
- `rustfs.spec` `Release` uses the prerelease suffix (for example `beta.4`), or `1` for a stable release.
- Do not change these rules without explicit confirmation.
## Step-by-step workflow
@@ -52,22 +58,18 @@ Only drop a file when the current repository release process clearly no longer r
- Use the exact target and delivery scope already supplied; ask only for a missing or ambiguous target or a material release-policy choice.
- Inspect current branch and ensure only release-related files are touched for this task.
2. Update workspace versions
2. Update workspace versions (`prepare` only)
- Bump `[workspace.package].version` in `Cargo.toml`.
- Bump internal workspace crate dependency versions in `Cargo.toml`.
- Update `Cargo.lock` so workspace package versions match target version.
- Re-scan for partial leftovers.
- Re-scan Cargo.toml and workspace members in Cargo.lock for partial leftovers; leave external dependency versions unchanged.
3. Update release assets
3. Update installation references (`post-release` only)
- `README.md` and `README_ZH.md`: update versioned Docker examples to target version.
- `flake.nix`: update package version to target version.
- `helm/rustfs/Chart.yaml`:
- `appVersion` = target version.
- `version` follows chart mapping rule, for example:
- `1.0.0-beta.3` -> `0.3.0`
- `1.0.0-beta.4` -> `0.4.0`
- `helm/rustfs/Chart.yaml`: use the app and chart versions returned by `scripts/helm_chart_version.sh <target>`.
- `rustfs.spec`:
- Set `Release` to prerelease suffix (example `beta.4`).
- Set `Version` to the numeric version and `Release` to the prerelease suffix (example `beta.4`), or `1` for a stable release. Verify that `Source0` and the unpacked source directory resolve to the exact published target, including its prerelease suffix when present.
- Add/update top changelog entry with exact format:
- `* Thu May 20 2026 houseme <housemecn@gmail.com>`
- `- Update RPM package to RustFS 1.0.0-beta.4`
@@ -78,15 +80,14 @@ Only drop a file when the current repository release process clearly no longer r
- Changelog version text must match target release version exactly.
4. Verify before shipping
- Run:
- `make pre-commit`
- If `make pre-commit` fails, fix task-attributable failures and rerun affected checks. Report unresolved required checks as `BLOCKED`; do not silently widen scope to fix unrelated issues.
- Follow the root verification tiers for the final diff instead of running a full-workspace gate for version strings.
- For `prepare`, validate Cargo metadata with the updated lockfile and confirm workspace package/internal dependency versions agree. Do not accept a lockfile containing unrelated dependency updates.
- For `post-release`, render the Helm chart and confirm its default image is the verified target; run `scripts/test_helm_chart_version.sh` when chart versions change. Check the README image tags, Nix package version, and expanded RPM source URL against that same target.
- Run `git diff --check` for either stage. Report unresolved required checks as `BLOCKED`; do not silently widen scope to fix unrelated issues.
5. Commit strategy (only when committing is authorized)
- Preferred split when both parts changed:
- `chore(release): prepare <version>` for `Cargo.toml` and `Cargo.lock`.
- `chore(release): align release assets for <version>` for docs and packaging files.
- If user asks for one commit, use one commit.
- Use `chore(release): prepare <version>` for `prepare` and `chore(release): align installation references for <version>` for `post-release`.
- These stages happen on opposite sides of publication; do not combine them in a preview-preparation commit or PR.
- Stage only intended release files; do not include unrelated working tree changes.
6. Push and PR (only for the authorized delivery scope)
@@ -104,13 +105,12 @@ Only drop a file when the current repository release process clearly no longer r
- `git diff --name-only origin/main...HEAD`
- `git diff --stat origin/main...HEAD`
- `rg -n "<old_version>|<new_version>" Cargo.toml Cargo.lock README.md README_ZH.md flake.nix helm/rustfs/Chart.yaml rustfs.spec`
- `make pre-commit`
## Output contract
When using this skill, always report:
- Target version.
- Target version and stage.
- Files changed.
- Any assumptions or uncertainties requiring confirmation.
- Verification result (`PASSED` or `BLOCKED`) with key evidence.
+1
View File
@@ -24,6 +24,7 @@ TEST_THREADS ?= 1
.PHONY: script-tests
script-tests: ## Run shell script tests
@echo "Running script tests..."
trap 'status=$$?; printf "ERROR: script-tests failed (exit %s): %s\n" "$$status" "$$BASH_COMMAND" >&2; exit "$$status"' ERR
./scripts/test_build_rustfs_options.sh
./scripts/test_docker_runtime_timezone.sh
./scripts/test_entrypoint_credentials.sh
-38
View File
@@ -1,38 +0,0 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.
+110
View File
@@ -0,0 +1,110 @@
name: Bug report
description: Create a report to help us improve
body:
- type: markdown
attributes:
value: |
### Storage support requirements
**Supported:** local disks, SAN (Fibre Channel) attached volumes, or JBOD-mounted disks. We recommend formatting them with the **XFS** file system.
**Not supported:** NFS, SMB/Samba/CIFS, and other remote or shared filesystems as RustFS data directories. This also applies to the underlying host storage for Docker volumes and Kubernetes PersistentVolumes.
Reports that only reproduce on unsupported storage will be closed without investigation. Please reproduce the issue on supported storage before submitting a bug report.
See [Disk Planning and File System Selection](https://docs.rustfs.com/en/installation/linux/prerequisites-and-service) for details.
- type: checkboxes
id: storage-support
attributes:
label: Supported storage confirmation
options:
- label: I have reproduced this issue with all RustFS data directories on supported storage (local disks, SAN/Fibre Channel, or JBOD; XFS recommended), without NFS, SMB/Samba/CIFS, or other remote or shared filesystem mounts, including storage backing Docker volumes and Kubernetes PersistentVolumes.
required: true
- type: textarea
id: bug-description
attributes:
label: Describe the bug
description: A clear and concise description of what happened.
validations:
required: true
- type: textarea
id: reproduction-steps
attributes:
label: Steps to reproduce
description: Include the commands, configuration, or actions needed to reproduce the problem.
placeholder: |
1. Start RustFS with ...
2. Run ...
3. Observe ...
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: Expected behavior
description: A clear and concise description of what you expected to happen.
validations:
required: true
- type: input
id: rustfs-version
attributes:
label: RustFS version
description: Provide the release version, container image tag, or commit hash.
validations:
required: true
- type: dropdown
id: deployment-method
attributes:
label: Deployment method
description: How are you running RustFS? Add any relevant details in Additional context.
options:
- Binary
- Docker / Docker Compose
- Kubernetes / Helm
- Built from source
- Other
- type: textarea
id: storage-mounts
attributes:
label: Data directory filesystems and mounts
description: |
Identify the filesystem type and mount source for every RustFS data directory.
On Linux, include output from `findmnt -T /path/to/data -o TARGET,SOURCE,FSTYPE,OPTIONS` for each data path; on other systems, provide equivalent mount information.
For Docker, identify the backing host path and filesystem, not just the container path or volume name.
For Kubernetes, identify the PV, StorageClass/CSI driver, and actual backing filesystem; a PVC name alone is not sufficient.
Remove credentials and other sensitive information before submitting.
placeholder: |
RustFS data paths:
Filesystem types and mount sources:
Mount command output:
Docker host paths or Kubernetes volume backing storage (if applicable):
validations:
required: true
- type: input
id: operating-system
attributes:
label: Server operating system and architecture
description: Include the operating system version and CPU architecture where RustFS is running.
placeholder: "For example: Ubuntu 24.04, x86_64"
validations:
required: true
- type: textarea
id: client-environment
attributes:
label: Client environment
description: Include the S3 client or SDK and version. For console issues, include the browser, operating system, and desktop or mobile device details.
- type: textarea
id: logs
attributes:
label: Relevant logs
description: Paste relevant log output. Remove credentials and other sensitive information before submitting.
render: shell
- type: textarea
id: screenshots
attributes:
label: Screenshots
description: If applicable, drag and drop screenshots here to help explain the problem.
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context, such as deployment topology or relevant configuration. Remove credentials and other sensitive information before submitting.
+1
View File
@@ -0,0 +1 @@
blank_issues_enabled: false
-20
View File
@@ -1,20 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
@@ -0,0 +1,38 @@
name: Feature request
description: Suggest an idea for this project
body:
- type: checkboxes
id: request-scope
attributes:
label: Request scope
description: |
RustFS data directories backed by NFS, SMB/Samba/CIFS, or other remote or shared filesystems are unsupported, including those behind Docker volumes or Kubernetes PersistentVolumes.
Troubleshooting requests for these deployments will be closed without investigation. Use this form for new or improved functionality.
options:
- label: This is a feature request and does not ask for troubleshooting an unsupported remote or shared filesystem deployment.
required: true
- type: textarea
id: problem-description
attributes:
label: Problem or use case
description: Describe the problem or use case that motivates this feature request.
placeholder: "For example: I am trying to ... but ..."
validations:
required: true
- type: textarea
id: proposed-solution
attributes:
label: Proposed solution
description: A clear and concise description of what you would like to happen.
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered
description: Describe any alternative solutions or features you have considered.
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or drag and drop screenshots about the feature request here.
+1 -1
View File
@@ -19,7 +19,7 @@ runs:
using: composite
steps:
- name: Install quality tools
uses: taiki-e/install-action@bffeee26d4db9be238a4ea78d8826604ebcb594d # v2
uses: taiki-e/install-action@7623a79cdfecb99d681017af368ca353d9f49bb5 # v2
with:
tool: |
ripgrep@15.2.0
+1 -1
View File
@@ -109,7 +109,7 @@ runs:
uses: taiki-e/install-action@96c7780c1d8a2b8723e12031def873a434d39d8d # nextest
- name: Setup Rust cache
uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
# false is rust-cache's own default. With true, cleanup.ts returns
# *before* pruning ~/.cargo/registry/src, and config.ts archives the
@@ -36,7 +36,7 @@ jobs:
cancel-closed-pr-runs:
name: Cancel Closed PR Runs
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Explain cancellation run
@@ -45,15 +45,15 @@ jobs:
architecture-migration-rules:
name: Architecture Migration Rules
if: github.event_name != 'pull_request' || github.event.action != 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Install ripgrep
uses: taiki-e/install-action@bffeee26d4db9be238a4ea78d8826604ebcb594d # v2
uses: taiki-e/install-action@7623a79cdfecb99d681017af368ca353d9f49bb5 # v2
with:
tool: ripgrep@15.2.0
+11 -11
View File
@@ -77,7 +77,7 @@ jobs:
cancel-closed-pr-runs:
name: Cancel Closed PR Runs
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Explain cancellation run
@@ -89,11 +89,11 @@ jobs:
cargo-deny:
name: Cargo Deny
if: github.event_name != 'pull_request' || github.event.action != 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -112,7 +112,7 @@ jobs:
# default is now "false", but state it explicitly — see
# scripts/security/check_cache_save_if.sh.
- name: Setup Rust cache
uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2
with:
# Same reasoning as the setup composite: true archives every
# dependency's unpacked source tree.
@@ -122,7 +122,7 @@ jobs:
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Install cargo-deny
uses: taiki-e/install-action@bffeee26d4db9be238a4ea78d8826604ebcb594d # v2
uses: taiki-e/install-action@7623a79cdfecb99d681017af368ca353d9f49bb5 # v2
with:
tool: cargo-deny
@@ -132,11 +132,11 @@ jobs:
workflow-pin-report:
name: Workflow Pin Report
if: github.event_name != 'pull_request' || github.event.action != 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 5
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -169,7 +169,7 @@ jobs:
dependency-review:
name: Dependency Review
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
if: github.event_name == 'pull_request' && github.event.action != 'closed'
permissions:
@@ -177,7 +177,7 @@ jobs:
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -204,13 +204,13 @@ jobs:
# open the Actions tab. Same ci-8 mechanism coverage.yml and
# e2e-replication-nightly.yml already use.
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+16 -16
View File
@@ -88,7 +88,7 @@ jobs:
# Build strategy check - determine build type based on trigger
build-check:
name: Build Strategy Check
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
outputs:
should_build: ${{ steps.check.outputs.should_build }}
@@ -98,7 +98,7 @@ jobs:
is_prerelease: ${{ steps.check.outputs.is_prerelease }}
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -172,7 +172,7 @@ jobs:
prepare-platform-matrix:
name: Prepare Platform Matrix
needs: build-check
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
outputs:
matrix: ${{ steps.select.outputs.matrix }}
@@ -257,7 +257,7 @@ jobs:
matrix: ${{ fromJson(needs.prepare-platform-matrix.outputs.matrix) }}
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
fetch-depth: 0
@@ -702,7 +702,7 @@ jobs:
exit 1
- name: Upload to GitHub artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ steps.package.outputs.package_name }}
path: "rustfs-*.zip"
@@ -770,7 +770,7 @@ jobs:
name: Build Summary
needs: [ build-check, build-rustfs ]
if: always() && needs.build-check.outputs.should_build == 'true'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Build completion summary
@@ -831,7 +831,7 @@ jobs:
name: Create GitHub Release
needs: [ build-check, build-rustfs ]
if: startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'preview' || needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
permissions:
contents: write
@@ -840,7 +840,7 @@ jobs:
release_url: ${{ steps.create.outputs.release_url }}
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
fetch-depth: 0
@@ -892,19 +892,19 @@ jobs:
name: Upload Release Assets
needs: [ build-check, build-rustfs, create-release ]
if: startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'preview' || needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
permissions:
contents: write
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Download all build artifacts
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: ./artifacts
pattern: rustfs-*
@@ -986,7 +986,7 @@ jobs:
name: Update Latest Version
needs: [ build-check, publish-release ]
if: startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
steps:
- name: Update latest.json
@@ -1046,7 +1046,7 @@ jobs:
name: Publish Release
needs: [ build-check, create-release, upload-release-assets ]
if: startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'preview' || needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
permissions:
contents: write
@@ -1084,7 +1084,7 @@ jobs:
name: Cleanup Preview Releases
needs: [ build-check, publish-release ]
if: startsWith(github.ref, 'refs/tags/') && (needs.build-check.outputs.build_type == 'release' || needs.build-check.outputs.build_type == 'prerelease')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
permissions:
contents: write
@@ -1132,13 +1132,13 @@ jobs:
if: >-
always() && github.event_name == 'schedule' &&
(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+6 -6
View File
@@ -109,7 +109,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -141,7 +141,7 @@ jobs:
- name: Upload cargo timings report
if: inputs.emit_timings
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: cargo-timings-ci-dev
path: target/cargo-timings/
@@ -199,7 +199,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -230,7 +230,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -256,11 +256,11 @@ jobs:
# belonged to it.
warm-ci-uring:
name: Warm ci-uring
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 60
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
+49 -45
View File
@@ -63,7 +63,7 @@ jobs:
cancel-closed-pr-runs:
name: Cancel Closed PR Runs
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Explain cancellation run
@@ -72,12 +72,12 @@ jobs:
classify-changes:
name: Select CI scope
if: github.event_name != 'pull_request' || github.event.action != 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
outputs:
mode: ${{ steps.scope.outputs.mode }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
fetch-depth: 2
persist-credentials: false
@@ -98,24 +98,28 @@ jobs:
typos:
name: Typos
if: github.event_name != 'pull_request' || github.event.action != 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Install typos download dependency
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends wget
- name: Typos check with custom config file
uses: crate-ci/typos@37bb98842b0d8c4ffebdb75301a13db0267cef89 # master
uses: crate-ci/typos@512fc24f32f44ab01972217aaaf3dc86ec234d53 # v1.50.2
# Fail early with compile-free checks for every pull request.
quick-checks:
name: Quick Checks
if: github.event_name != 'pull_request' || github.event.action != 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -132,7 +136,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
# Checkout otherwise writes the token into .git/config, where a PR's
# own build.rs or proc-macro could read it back out.
@@ -261,7 +265,7 @@ jobs:
- name: Upload test reports and diagnostics
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: junit-test-and-lint-${{ github.run_number }}
path: |
@@ -305,7 +309,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -344,7 +348,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -391,7 +395,7 @@ jobs:
- name: Upload ILM test diagnostics
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ilm-integration-${{ github.run_number }}-${{ github.run_attempt }}
path: |
@@ -408,7 +412,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -449,7 +453,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -521,7 +525,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -561,7 +565,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -621,7 +625,7 @@ jobs:
PYBUILD
- name: Upload debug binary
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-debug-binary
path: |
@@ -646,7 +650,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -662,7 +666,7 @@ jobs:
run: python3 scripts/e2e_binary.py build --bins --features rio-v2,e2e-test-hooks
- name: Upload debug binary
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-debug-binary-rio-v2
path: |
@@ -690,7 +694,7 @@ jobs:
timeout-minutes: 30
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -754,7 +758,7 @@ jobs:
timeout-minutes: 30
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -770,7 +774,7 @@ jobs:
install-build-packaging-tools: 'false'
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
@@ -785,7 +789,7 @@ jobs:
# Download after the cache restore so the freshly built binary from the
# build job always wins over anything restored into target/debug.
- name: Download debug binary
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: rustfs-debug-binary
path: target/debug
@@ -821,7 +825,7 @@ jobs:
- name: Upload e2e smoke diagnostics
if: failure()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-smoke-diagnostics-${{ github.run_number }}
path: |
@@ -831,7 +835,7 @@ jobs:
- name: Upload e2e smoke JUnit report
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-smoke-junit-${{ github.run_number }}
path: target/nextest/e2e-smoke/junit.xml
@@ -856,7 +860,7 @@ jobs:
- name: Upload test logs
if: failure()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-test-logs-${{ github.run_number }}
path: ${{ runner.temp }}/rustfs-e2e-*/rustfs.log
@@ -879,7 +883,7 @@ jobs:
timeout-minutes: 55
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -903,7 +907,7 @@ jobs:
fi
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
@@ -945,7 +949,7 @@ jobs:
# Download after the cache restore so the freshly built binary from the
# build job always wins over anything restored into target/debug.
- name: Download debug binary
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: rustfs-debug-binary
path: target/debug
@@ -1005,7 +1009,7 @@ jobs:
- name: Upload junit
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-full-junit-${{ github.run_number }}
path: |
@@ -1015,7 +1019,7 @@ jobs:
- name: Upload startup CAS evidence
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fresh-startup-cas-evidence-${{ github.run_number }}
path: |
@@ -1034,7 +1038,7 @@ jobs:
timeout-minutes: 30
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -1044,7 +1048,7 @@ jobs:
rm -f /tmp/rustfs.log
- name: Download debug binary
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: rustfs-debug-binary-rio-v2
path: target/debug
@@ -1077,7 +1081,7 @@ jobs:
- name: Upload test logs
if: failure()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-test-logs-rio-v2-${{ github.run_number }}
path: /tmp/rustfs.log
@@ -1090,12 +1094,12 @@ jobs:
timeout-minutes: 60
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Download debug binary
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: rustfs-debug-binary
path: target/debug
@@ -1120,7 +1124,7 @@ jobs:
- name: Upload s3 test artifacts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: s3tests-implemented-${{ github.run_number }}
path: artifacts/s3tests-single/**
@@ -1164,12 +1168,12 @@ jobs:
timeout-minutes: 30
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Download debug binary
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: rustfs-debug-binary
path: target/debug
@@ -1202,7 +1206,7 @@ jobs:
- name: Upload s3 test artifacts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: s3tests-lifecycle-behavior-${{ github.run_number }}
path: artifacts/s3tests-single/**
@@ -1230,10 +1234,10 @@ jobs:
- build-rustfs-debug-binary-rio-v2
- e2e-tests-rio-v2
- e2e-full
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Require the expected result of every CI lane
@@ -1266,13 +1270,13 @@ jobs:
if: >-
always() && github.event_name == 'schedule' &&
(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+3 -3
View File
@@ -46,7 +46,7 @@ jobs:
# Echoes one line; the run exists only so the concurrency group cancels the
# in-flight run of a closed PR.
permissions: {}
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Explain cancellation run
@@ -61,12 +61,12 @@ jobs:
checks: write
issues: write
pull-requests: write
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
steps:
- name: Report CLA result for merge queue
if: github.event_name == 'merge_group'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
await github.rest.checks.create({
@@ -0,0 +1,198 @@
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
name: Connect diagnostic scheduler receipt acceptance
on:
workflow_dispatch:
inputs:
build_run_id:
description: Main-branch Build and Release run ID
required: true
type: string
artifact_id:
description: Linux x86_64 GNU artifact from that run
required: true
type: string
source_sha:
description: Exact RustFS source commit
required: true
type: string
artifact_digest:
description: GitHub artifact digest including sha256 prefix
required: true
type: string
binary_sha256:
description: Independently verified rustfs binary SHA-256
required: true
type: string
connect_sha:
description: Exact Connect scheduler acceptance harness commit
required: true
type: string
permissions:
actions: read
contents: read
jobs:
diagnostic-schedule:
name: Verify production scheduler receipts over mTLS
runs-on: sm-standard-2
timeout-minutes: 45
steps:
- name: Checkout exact RustFS source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
path: rustfs-source
persist-credentials: false
ref: ${{ inputs.source_sha }}
- name: Checkout exact Connect harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
repository: rustfs/connect
path: connect-harness
persist-credentials: false
ref: ${{ inputs.connect_sha }}
token: ${{ secrets.PF_TESTING_GH_TOKEN }}
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 22.22.2
cache: npm
cache-dependency-path: connect-harness/web/package-lock.json
- name: Verify official source and artifact identity
shell: bash
env:
GH_TOKEN: ${{ github.token }}
BUILD_RUN_ID: ${{ inputs.build_run_id }}
ARTIFACT_ID: ${{ inputs.artifact_id }}
SOURCE_SHA: ${{ inputs.source_sha }}
ARTIFACT_DIGEST: ${{ inputs.artifact_digest }}
BINARY_SHA256: ${{ inputs.binary_sha256 }}
CONNECT_SHA: ${{ inputs.connect_sha }}
run: |
set -euo pipefail
[[ "$BUILD_RUN_ID" =~ ^[0-9]+$ && "$ARTIFACT_ID" =~ ^[0-9]+$ ]]
[[ "$SOURCE_SHA" =~ ^[0-9a-f]{40}$ && "$CONNECT_SHA" =~ ^[0-9a-f]{40}$ ]]
[[ "$ARTIFACT_DIGEST" =~ ^sha256:[0-9a-f]{64}$ ]]
[[ "$BINARY_SHA256" =~ ^[0-9a-f]{64}$ ]]
[[ $(git -C rustfs-source rev-parse HEAD) == "$SOURCE_SHA" ]]
[[ $(git -C connect-harness rev-parse HEAD) == "$CONNECT_SHA" ]]
[[ $(git -C rustfs-source remote get-url origin) == https://github.com/rustfs/rustfs ]]
[[ $(git -C connect-harness remote get-url origin) == https://github.com/rustfs/connect ]]
run=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs/${BUILD_RUN_ID}")
jq -e --arg source "$SOURCE_SHA" --arg repository "$GITHUB_REPOSITORY" '
.head_sha == $source and .head_branch == "main"
and .head_repository.full_name == $repository
and .name == "Build and Release" and .path == ".github/workflows/build.yml"
and .status == "completed" and .conclusion == "success"
' <<<"$run" >/dev/null
jobs=$(gh api --paginate --slurp "repos/${GITHUB_REPOSITORY}/actions/runs/${BUILD_RUN_ID}/jobs?per_page=100")
jq -e '
[.[].jobs[] | select(.name | test("^Build RustFS \\(linux-x86_64-gnu, [a-z0-9-]+, x86_64-unknown-linux-gnu, false, linux, pyroscope\\)$"))] as $matches
| ($matches | length) == 1 and $matches[0].conclusion == "success"
' <<<"$jobs" >/dev/null
artifact=$(gh api "repos/${GITHUB_REPOSITORY}/actions/artifacts/${ARTIFACT_ID}")
jq -e --argjson run "$BUILD_RUN_ID" --arg source "$SOURCE_SHA" --arg digest "$ARTIFACT_DIGEST" '
.workflow_run.id == $run and .workflow_run.head_sha == $source
and .name == ("rustfs-linux-x86_64-gnu-dev-" + $source[0:7])
and .digest == $digest and .expired == false
and (.expires_at | fromdateiso8601) > now
and .size_in_bytes > 0 and .size_in_bytes <= 2147483648
' <<<"$artifact" >/dev/null
gh api "repos/${GITHUB_REPOSITORY}/actions/artifacts/${ARTIFACT_ID}/zip" > official-artifact.zip
printf '%s official-artifact.zip\n' "${ARTIFACT_DIGEST#sha256:}" | sha256sum --check --strict
- name: Extract the verified official binary
shell: bash
env:
SOURCE_SHA: ${{ inputs.source_sha }}
BINARY_SHA256: ${{ inputs.binary_sha256 }}
run: |
set -euo pipefail
python3 - <<'PY'
import io
import os
import pathlib
import stat
import zipfile
def regular(entry):
mode = entry.external_attr >> 16
return not entry.is_dir() and (stat.S_IFMT(mode) in (0, stat.S_IFREG))
package = f"rustfs-linux-x86_64-gnu-dev-{os.environ['SOURCE_SHA'][:7]}.zip"
with zipfile.ZipFile('official-artifact.zip') as outer:
entries = outer.infolist()
assert 1 <= len(entries) <= 16
assert len({item.filename for item in entries}) == len(entries)
assert all(regular(item) and pathlib.PurePosixPath(item.filename).name == item.filename for item in entries)
assert sum(item.file_size for item in entries) <= 2147483648
assert outer.testzip() is None
with zipfile.ZipFile(io.BytesIO(outer.read(package))) as inner:
entries = inner.infolist()
assert {item.filename for item in entries} == {'rustfs', 'rustfs-cli'} and len(entries) == 2
assert all(regular(item) and 0 < item.file_size <= 1073741824 for item in entries)
assert inner.testzip() is None
pathlib.Path('binary').mkdir()
with inner.open('rustfs') as source, open('binary/rustfs', 'xb') as destination:
import shutil
shutil.copyfileobj(source, destination)
PY
chmod 0755 binary/rustfs
printf '%s binary/rustfs\n' "$BINARY_SHA256" | sha256sum --check --strict
file binary/rustfs | grep -Eq 'ELF 64-bit LSB.*x86-64'
- name: Run the real scheduler and receipt delivery scenario
shell: bash
env:
SOURCE_SHA: ${{ inputs.source_sha }}
run: |
set -euo pipefail
git -C connect-harness diff --exit-code
printf '%s\n' "$SOURCE_SHA" >connect-harness/tests/e2e/connected/rustfs-ref
[[ $(git -C connect-harness diff --name-only) == tests/e2e/connected/rustfs-ref ]]
npm --prefix connect-harness/web ci
connect-harness/web/node_modules/.bin/playwright install --with-deps chromium
make -C connect-harness e2e-connected-dispatch-check
(cd connect-harness && ./tests/connectivity/diagnostic-schedule-evidence.test.sh)
RUSTFS_BINARY="$GITHUB_WORKSPACE/binary/rustfs" \
RUSTFS_WORKTREE="$GITHUB_WORKSPACE/rustfs-source" \
CONNECT_E2E_DIAGNOSTIC_SCHEDULE_EVIDENCE="$GITHUB_WORKSPACE/diagnostic-schedule-evidence.json" \
make -C connect-harness e2e-connected E2E_SCENARIO=diagnostic-schedule
- name: Bind and validate sanitized evidence
shell: bash
env:
BUILD_RUN_ID: ${{ inputs.build_run_id }}
ARTIFACT_ID: ${{ inputs.artifact_id }}
ARTIFACT_DIGEST: ${{ inputs.artifact_digest }}
BINARY_SHA256: ${{ inputs.binary_sha256 }}
CONNECT_SHA: ${{ inputs.connect_sha }}
SOURCE_SHA: ${{ inputs.source_sha }}
run: |
set -euo pipefail
jq -e --arg connect "$CONNECT_SHA" --arg source "$SOURCE_SHA" --arg binary "$BINARY_SHA256" '
.provenance.connectSha == $connect and .provenance.sourceSha == $source
and .provenance.binarySha256 == $binary
' diagnostic-schedule-evidence.json >/dev/null
jq --arg run "$GITHUB_RUN_ID" --arg build "$BUILD_RUN_ID" \
--arg artifact "$ARTIFACT_ID" --arg digest "$ARTIFACT_DIGEST" '
.provenance += {workflowRunId:$run,buildRunId:$build,artifactId:$artifact,artifactDigest:$digest}
' diagnostic-schedule-evidence.json > diagnostic-schedule-evidence.bound.json
mv diagnostic-schedule-evidence.bound.json diagnostic-schedule-evidence.json
node connect-harness/tests/connectivity/verify-diagnostic-schedule-evidence.mjs diagnostic-schedule-evidence.json --bound
- name: Upload verified receipt evidence
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: connect-diagnostic-schedule-evidence-${{ github.run_id }}
path: diagnostic-schedule-evidence.json
retention-days: 14
if-no-files-found: error
@@ -49,23 +49,23 @@ permissions:
jobs:
performance-drive:
name: Verify native Linux x86_64 performance.drive service job
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 45
steps:
- name: Checkout acceptance harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Checkout exact RustFS source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
path: rustfs-source
persist-credentials: false
ref: ${{ inputs.source_sha }}
- name: Checkout exact Connect harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/connect
path: connect-harness
@@ -122,7 +122,7 @@ jobs:
[[ $(jq -r '.expired' <<<"$artifact") == false ]]
- name: Download exact build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
artifact-ids: ${{ inputs.artifact_id }}
path: artifact
@@ -183,7 +183,7 @@ jobs:
- name: Upload acceptance evidence
if: ${{ always() && hashFiles('performance-drive-service-job-evidence.json') != '' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: connect-performance-drive-evidence-${{ github.run_id }}
path: |
@@ -49,23 +49,23 @@ permissions:
jobs:
performance-network:
name: Verify native Linux x86_64 performance.network service job
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 45
steps:
- name: Checkout acceptance harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Checkout exact RustFS source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
path: rustfs-source
persist-credentials: false
ref: ${{ inputs.source_sha }}
- name: Checkout exact Connect harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/connect
path: connect-harness
@@ -122,7 +122,7 @@ jobs:
[[ $(jq -r '.expired' <<<"$artifact") == false ]]
- name: Download exact build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
artifact-ids: ${{ inputs.artifact_id }}
path: artifact
@@ -183,7 +183,7 @@ jobs:
- name: Upload acceptance evidence
if: ${{ always() && hashFiles('performance-network-service-job-evidence.json') != '' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: connect-performance-network-evidence-${{ github.run_id }}
path: |
@@ -49,23 +49,23 @@ permissions:
jobs:
profile-cpu:
name: Verify native Linux x86_64 profile.cpu service job
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 45
steps:
- name: Checkout acceptance harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Checkout exact RustFS source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
path: rustfs-source
persist-credentials: false
ref: ${{ inputs.source_sha }}
- name: Checkout exact Connect harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/connect
path: connect-harness
@@ -122,7 +122,7 @@ jobs:
[[ $(jq -r '.expired' <<<"$artifact") == false ]]
- name: Download exact build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
artifact-ids: ${{ inputs.artifact_id }}
path: artifact
@@ -183,7 +183,7 @@ jobs:
- name: Upload acceptance evidence
if: ${{ always() && hashFiles('profile-service-job-evidence.json') != '' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: connect-profile-cpu-evidence-${{ github.run_id }}
path: |
@@ -0,0 +1,201 @@
# Copyright 2026 RustFS Team
# SPDX-License-Identifier: Apache-2.0
name: Connect service memory profile acceptance
on:
workflow_dispatch:
inputs:
build_run_id:
description: Successful main-branch Build and Release run ID
required: true
type: string
artifact_id:
description: Linux x86_64 GNU artifact from that run
required: true
type: string
source_sha:
description: Exact RustFS source commit
required: true
type: string
artifact_digest:
description: GitHub artifact digest including sha256 prefix
required: true
type: string
binary_sha256:
description: Independently verified rustfs binary SHA-256
required: true
type: string
connect_sha:
description: Exact Connect memory acceptance harness commit
required: true
type: string
permissions:
actions: read
contents: read
jobs:
profile-memory:
name: Verify signed service memory profile over mTLS
runs-on: sm-standard-2
timeout-minutes: 45
steps:
- name: Checkout exact RustFS source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
path: rustfs-source
persist-credentials: false
ref: ${{ inputs.source_sha }}
- name: Checkout exact Connect harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
repository: rustfs/connect
path: connect-harness
persist-credentials: false
ref: ${{ inputs.connect_sha }}
token: ${{ secrets.PF_TESTING_GH_TOKEN }}
- name: Set up Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 22.22.2
cache: npm
cache-dependency-path: connect-harness/web/package-lock.json
- name: Verify official source and artifact identity
shell: bash
env:
GH_TOKEN: ${{ github.token }}
BUILD_RUN_ID: ${{ inputs.build_run_id }}
ARTIFACT_ID: ${{ inputs.artifact_id }}
SOURCE_SHA: ${{ inputs.source_sha }}
ARTIFACT_DIGEST: ${{ inputs.artifact_digest }}
BINARY_SHA256: ${{ inputs.binary_sha256 }}
CONNECT_SHA: ${{ inputs.connect_sha }}
run: |
set -euo pipefail
[[ "$GITHUB_REPOSITORY" == rustfs/rustfs ]]
[[ "$BUILD_RUN_ID" =~ ^[1-9][0-9]*$ && "$ARTIFACT_ID" =~ ^[1-9][0-9]*$ ]]
[[ "$SOURCE_SHA" =~ ^[0-9a-f]{40}$ && "$CONNECT_SHA" =~ ^[0-9a-f]{40}$ ]]
[[ "$ARTIFACT_DIGEST" =~ ^sha256:[0-9a-f]{64}$ ]]
[[ "$BINARY_SHA256" =~ ^[0-9a-f]{64}$ ]]
[[ $(git -C rustfs-source rev-parse HEAD) == "$SOURCE_SHA" ]]
[[ $(git -C connect-harness rev-parse HEAD) == "$CONNECT_SHA" ]]
[[ $(git -C rustfs-source remote get-url origin) == https://github.com/rustfs/rustfs ]]
[[ $(git -C connect-harness remote get-url origin) == https://github.com/rustfs/connect ]]
run=$(gh api "repos/rustfs/rustfs/actions/runs/${BUILD_RUN_ID}")
jq -e --arg source "$SOURCE_SHA" '
.head_sha == $source and .head_branch == "main"
and .head_repository.full_name == "rustfs/rustfs"
and .name == "Build and Release" and .path == ".github/workflows/build.yml"
and .status == "completed" and .conclusion == "success"
' <<<"$run" >/dev/null
jobs=$(gh api --paginate --slurp "repos/rustfs/rustfs/actions/runs/${BUILD_RUN_ID}/jobs?per_page=100")
jq -e '
[.[].jobs[] | select(.name | test("^Build RustFS \\(linux-x86_64-gnu, [a-z0-9-]+, x86_64-unknown-linux-gnu, false, linux, pyroscope\\)$"))] as $matches
| ($matches | length) == 1 and $matches[0].conclusion == "success"
' <<<"$jobs" >/dev/null
artifact=$(gh api "repos/rustfs/rustfs/actions/artifacts/${ARTIFACT_ID}")
jq -e --argjson run "$BUILD_RUN_ID" --arg source "$SOURCE_SHA" --arg digest "$ARTIFACT_DIGEST" '
.workflow_run.id == $run and .workflow_run.head_sha == $source
and .name == ("rustfs-linux-x86_64-gnu-dev-" + $source[0:7])
and .digest == $digest and .expired == false
and (.expires_at | fromdateiso8601) > now
and .size_in_bytes > 0 and .size_in_bytes <= 2147483648
' <<<"$artifact" >/dev/null
jq -r '.expires_at' <<<"$artifact" > artifact-expires-at
gh api "repos/rustfs/rustfs/actions/artifacts/${ARTIFACT_ID}/zip" > official-artifact.zip
[[ $(stat --format=%s official-artifact.zip) == $(jq -r '.size_in_bytes' <<<"$artifact") ]]
printf '%s official-artifact.zip\n' "${ARTIFACT_DIGEST#sha256:}" | sha256sum --check --strict
- name: Extract only the verified official service binary
shell: bash
env:
SOURCE_SHA: ${{ inputs.source_sha }}
BINARY_SHA256: ${{ inputs.binary_sha256 }}
run: |
set -euo pipefail
python3 - <<'PY'
import io
import os
import pathlib
import shutil
import stat
import zipfile
def regular(entry):
mode = entry.external_attr >> 16
return not entry.is_dir() and stat.S_IFMT(mode) in (0, stat.S_IFREG)
package = f"rustfs-linux-x86_64-gnu-dev-{os.environ['SOURCE_SHA'][:7]}.zip"
with zipfile.ZipFile('official-artifact.zip') as outer:
entries = outer.infolist()
assert 1 <= len(entries) <= 16
assert len({entry.filename for entry in entries}) == len(entries)
assert all(regular(entry) and pathlib.PurePosixPath(entry.filename).name == entry.filename for entry in entries)
assert sum(entry.file_size for entry in entries) <= 2147483648
assert outer.testzip() is None
with zipfile.ZipFile(io.BytesIO(outer.read(package))) as inner:
entries = inner.infolist()
assert {entry.filename for entry in entries} == {'rustfs', 'rustfs-cli'} and len(entries) == 2
assert all(regular(entry) and 0 < entry.file_size <= 1073741824 for entry in entries)
assert inner.testzip() is None
pathlib.Path('binary').mkdir()
with inner.open('rustfs') as source, open('binary/rustfs', 'xb') as destination:
shutil.copyfileobj(source, destination)
PY
chmod 0755 binary/rustfs
printf '%s binary/rustfs\n' "$BINARY_SHA256" | sha256sum --check --strict
file binary/rustfs | grep -Eq 'ELF 64-bit LSB.*x86-64'
- name: Run the real service job under S3 workload
shell: bash
env:
SOURCE_SHA: ${{ inputs.source_sha }}
run: |
set -euo pipefail
git -C connect-harness diff --exit-code
printf '%s\n' "$SOURCE_SHA" >connect-harness/tests/e2e/connected/rustfs-ref
changed=$(git -C connect-harness diff --name-only)
[[ -z "$changed" || "$changed" == tests/e2e/connected/rustfs-ref ]]
npm --prefix connect-harness/web ci
connect-harness/web/node_modules/.bin/playwright install --with-deps chromium
make -C connect-harness e2e-connected-dispatch-check
(cd connect-harness && ./tests/connectivity/profile-memory-evidence.test.sh)
RUSTFS_BINARY="$GITHUB_WORKSPACE/binary/rustfs" \
RUSTFS_WORKTREE="$GITHUB_WORKSPACE/rustfs-source" \
CONNECT_E2E_PROFILE_MEMORY_EVIDENCE="$GITHUB_WORKSPACE/profile-memory-evidence.json" \
make -C connect-harness e2e-connected E2E_SCENARIO=profile-memory
- name: Bind and validate sanitized evidence
shell: bash
env:
BUILD_RUN_ID: ${{ inputs.build_run_id }}
ARTIFACT_ID: ${{ inputs.artifact_id }}
ARTIFACT_DIGEST: ${{ inputs.artifact_digest }}
BINARY_SHA256: ${{ inputs.binary_sha256 }}
CONNECT_SHA: ${{ inputs.connect_sha }}
SOURCE_SHA: ${{ inputs.source_sha }}
run: |
set -euo pipefail
jq -e --arg connect "$CONNECT_SHA" --arg source "$SOURCE_SHA" --arg binary "$BINARY_SHA256" '
.provenance.connectSha == $connect and .provenance.sourceSha == $source
and .provenance.binarySha256 == $binary
' profile-memory-evidence.json >/dev/null
jq --arg run "$GITHUB_RUN_ID" --arg build "$BUILD_RUN_ID" \
--arg artifact "$ARTIFACT_ID" --arg digest "$ARTIFACT_DIGEST" \
--arg expiry "$(cat artifact-expires-at)" '
.provenance += {workflowRunId:$run,buildRunId:$build,artifactId:$artifact,artifactDigest:$digest,artifactExpiresAt:$expiry}
' profile-memory-evidence.json > profile-memory-evidence.bound.json
mv profile-memory-evidence.bound.json profile-memory-evidence.json
node connect-harness/tests/connectivity/profile-memory-evidence.mjs profile-memory-evidence.json --bound
- name: Upload only verified sanitized evidence
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: connect-profile-memory-evidence-${{ github.run_id }}
path: profile-memory-evidence.json
retention-days: 14
if-no-files-found: error
@@ -49,23 +49,23 @@ permissions:
jobs:
profile-threads:
name: Verify native Linux x86_64 profile.threads service job
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 45
steps:
- name: Checkout acceptance harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Checkout exact RustFS source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
path: rustfs-source
persist-credentials: false
ref: ${{ inputs.source_sha }}
- name: Checkout exact Connect harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/connect
path: connect-harness
@@ -122,7 +122,7 @@ jobs:
[[ $(jq -r '.expired' <<<"$artifact") == false ]]
- name: Download exact build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
artifact-ids: ${{ inputs.artifact_id }}
path: artifact
@@ -183,7 +183,7 @@ jobs:
- name: Upload acceptance evidence
if: ${{ always() && hashFiles('profile-threads-service-job-evidence.json') != '' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: connect-profile-threads-evidence-${{ github.run_id }}
path: |
@@ -49,23 +49,23 @@ permissions:
jobs:
top-api:
name: Verify native Linux x86_64 top.api service job
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 45
steps:
- name: Checkout acceptance harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Checkout exact RustFS source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
path: rustfs-source
persist-credentials: false
ref: ${{ inputs.source_sha }}
- name: Checkout exact Connect harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/connect
path: connect-harness
@@ -122,7 +122,7 @@ jobs:
[[ $(jq -r '.expired' <<<"$artifact") == false ]]
- name: Download exact build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
artifact-ids: ${{ inputs.artifact_id }}
path: artifact
@@ -183,7 +183,7 @@ jobs:
- name: Upload acceptance evidence
if: ${{ always() && hashFiles('top-api-service-job-evidence.json') != '' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: connect-top-api-evidence-${{ github.run_id }}
path: |
@@ -45,11 +45,11 @@ permissions:
jobs:
top-disk:
name: Verify native Linux x86_64 top.disk artifact
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 15
steps:
- name: Checkout acceptance harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -83,7 +83,7 @@ jobs:
[[ $(jq -r '.expired' <<<"$artifact") == false ]]
- name: Download exact build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
artifact-ids: ${{ inputs.artifact_id }}
path: artifact
@@ -132,7 +132,7 @@ jobs:
mv top-disk-runtime-evidence.bound.json top-disk-runtime-evidence.json
- name: Upload acceptance evidence
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: connect-top-disk-evidence-${{ github.run_id }}
path: |
@@ -49,23 +49,23 @@ permissions:
jobs:
top-locks:
name: Verify native Linux x86_64 top.locks service job
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 45
steps:
- name: Checkout acceptance harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Checkout exact RustFS source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
path: rustfs-source
persist-credentials: false
ref: ${{ inputs.source_sha }}
- name: Checkout exact Connect harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/connect
path: connect-harness
@@ -122,7 +122,7 @@ jobs:
[[ $(jq -r '.expired' <<<"$artifact") == false ]]
- name: Download exact build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
artifact-ids: ${{ inputs.artifact_id }}
path: artifact
@@ -183,7 +183,7 @@ jobs:
- name: Upload acceptance evidence
if: ${{ always() && hashFiles('top-locks-service-job-evidence.json') != '' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: connect-top-locks-evidence-${{ github.run_id }}
path: |
@@ -49,23 +49,23 @@ permissions:
jobs:
top-rpc:
name: Verify native Linux x86_64 top.rpc service job
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 45
steps:
- name: Checkout acceptance harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Checkout exact RustFS source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
path: rustfs-source
persist-credentials: false
ref: ${{ inputs.source_sha }}
- name: Checkout exact Connect harness
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/connect
path: connect-harness
@@ -122,7 +122,7 @@ jobs:
[[ $(jq -r '.expired' <<<"$artifact") == false ]]
- name: Download exact build artifact
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
artifact-ids: ${{ inputs.artifact_id }}
path: artifact
@@ -183,7 +183,7 @@ jobs:
- name: Upload acceptance evidence
if: ${{ always() && hashFiles('top-rpc-service-job-evidence.json') != '' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: connect-top-rpc-evidence-${{ github.run_id }}
path: |
+5 -5
View File
@@ -76,7 +76,7 @@ jobs:
NEXTEST_PROFILE: ci
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -89,7 +89,7 @@ jobs:
install-build-packaging-tools: 'false'
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@bffeee26d4db9be238a4ea78d8826604ebcb594d # v2
uses: taiki-e/install-action@7623a79cdfecb99d681017af368ca353d9f49bb5 # v2
with:
tool: cargo-llvm-cov
@@ -112,7 +112,7 @@ jobs:
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: coverage-lcov-${{ github.run_number }}
path: |
@@ -128,13 +128,13 @@ jobs:
# manual workflow_dispatch runs stay quiet so a debugging run never files a
# spurious alert.
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+8 -8
View File
@@ -83,7 +83,7 @@ jobs:
(github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push' &&
github.event.workflow_run.head_branch != 'main')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
outputs:
should_build: ${{ steps.check.outputs.should_build }}
@@ -96,7 +96,7 @@ jobs:
source_ref: ${{ steps.check.outputs.source_ref }}
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
# For workflow_run events, checkout the specific commit that triggered the workflow
@@ -352,7 +352,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
ref: ${{ needs.build-check.outputs.source_ref }}
@@ -501,7 +501,7 @@ jobs:
name: Scan Docker Images
needs: [ build-check, build-docker ]
if: needs.build-check.outputs.should_build == 'true' && needs.build-check.outputs.should_push == 'true'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
permissions:
contents: read
@@ -544,7 +544,7 @@ jobs:
category: container-image-${{ matrix.variant }}
- name: Upload container scan report
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: container-image-scan-${{ matrix.variant }}
path: trivy-${{ matrix.variant }}.sarif
@@ -559,11 +559,11 @@ jobs:
name: Sync Docker Hub Description
needs: [ build-check, build-docker ]
if: needs.build-check.outputs.should_build == 'true' && needs.build-check.outputs.should_push == 'true'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
ref: ${{ needs.build-check.outputs.source_ref }}
@@ -604,7 +604,7 @@ jobs:
name: Docker Build Summary
needs: [ build-check, build-docker ]
if: always() && needs.build-check.outputs.should_build == 'true'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Docker build completion summary
+4 -4
View File
@@ -93,7 +93,7 @@ jobs:
UPGRADE_SOURCE_SHA256: 7c789386bf85278f865b8e0d359bf4edb84d5aa408cc3fa54a18c25ca74cd6e7
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -174,7 +174,7 @@ jobs:
- name: Upload distributed e2e diagnostics
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-distributed-${{ github.run_number }}
path: |
@@ -200,13 +200,13 @@ jobs:
name: Alert on scheduled failure
needs: [distributed]
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
@@ -63,7 +63,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -77,7 +77,7 @@ jobs:
# The STS dual-node test requires awscurl and fails if it is unavailable.
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
@@ -108,7 +108,7 @@ jobs:
- name: Upload nextest junit report
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-replication-nightly-junit-${{ github.run_number }}
path: |
@@ -126,7 +126,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -157,7 +157,7 @@ jobs:
- name: Upload cluster fault diagnostics
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-cluster-nightly-${{ github.run_number }}
path: |
@@ -176,7 +176,7 @@ jobs:
RUSTFS_BUILD_FEATURES: ftps,webdav,sftp
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -213,7 +213,7 @@ jobs:
- name: Upload protocol diagnostics
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-protocol-nightly-${{ github.run_number }}
path: |
@@ -230,13 +230,13 @@ jobs:
# manual workflow_dispatch runs stay quiet so a debugging run never files a
# spurious alert.
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+15 -10
View File
@@ -159,7 +159,7 @@ jobs:
TEST_MODE: ${{ matrix.test-mode }}
S3_SHARD_INDEX: ${{ matrix.shard-index }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -167,7 +167,7 @@ jobs:
# ship a working pip (ci-1: a bare python3 without pip is what broke the
# scheduled sweep). Keeps the workflow correct across runner drift.
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
@@ -264,7 +264,7 @@ jobs:
hostname: rustfs4
volumes: [rustfs4-data:/data]
lb:
image: haproxy:2.9
image: haproxy:3.4.4
hostname: lb
networks: [rustfs-net]
ports:
@@ -308,6 +308,11 @@ jobs:
docker compose -f compose.yml up -d
# Validate the pinned HAProxy image parses the exact config before
# the S3 compatibility sweep starts.
docker compose -f compose.yml exec -T lb \
haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg
- name: Wait for RustFS ready
run: |
for _ in {1..120}; do
@@ -357,7 +362,7 @@ jobs:
- name: Upload artifacts
if: always() && env.ACT != 'true'
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: s3tests-${{ env.TEST_MODE }}-shard-${{ matrix.shard-index }}
path: artifacts/**
@@ -366,15 +371,15 @@ jobs:
name: Upstream HEAD classification canary
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
continue-on-error: true
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 20
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
@@ -435,7 +440,7 @@ jobs:
- name: Upload canary artifacts
if: always() && env.ACT != 'true'
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: s3tests-upstream-head
path: artifacts/s3tests-upstream-head/**
@@ -448,13 +453,13 @@ jobs:
# job fails. Alerts only for scheduled runs (backlog#1149 ci-8) — manual
# dispatch failures are already watched by a human.
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+4 -4
View File
@@ -90,13 +90,13 @@ jobs:
cache_key: e2e-multipart-layout-baseline
test: rc5_baseline_replicates_multipart_layouts
artifact: multipart-layout-baseline
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 60
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -137,7 +137,7 @@ jobs:
- name: Upload scanner/heal G09 evidence
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ matrix.artifact }}-g09-evidence-${{ github.run_number }}
path: ${{ runner.temp }}/rustfs-upgrade-g09-evidence/${{ matrix.artifact }}
@@ -146,7 +146,7 @@ jobs:
- name: Upload server logs
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ matrix.artifact }}-server-logs-${{ github.run_number }}
path: ${{ runner.temp }}/rustfs-upgrade-logs
@@ -11,10 +11,10 @@ concurrency:
cancel-in-progress: false
jobs:
collect:
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 15
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Collect and publish verified chain health
@@ -24,7 +24,7 @@ jobs:
run: python3 scripts/functional_chain_health.py --source-ref "${NIGHTLY_SOURCE_REF}" --publish --output "${RUNNER_TEMP}/chain-health.json"
- name: Retain health observation
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-health-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-health.json
+12 -12
View File
@@ -60,7 +60,7 @@ jobs:
cancel-closed-pr-runs:
name: Cancel Closed PR Runs
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Explain cancellation run
@@ -81,7 +81,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -93,7 +93,7 @@ jobs:
cache-save-if: ${{ github.ref == 'refs/heads/main' || github.event_name == 'schedule' }}
- name: Install cargo-fuzz
uses: taiki-e/install-action@bffeee26d4db9be238a4ea78d8826604ebcb594d # v2
uses: taiki-e/install-action@7623a79cdfecb99d681017af368ca353d9f49bb5 # v2
with:
tool: cargo-fuzz
@@ -113,7 +113,7 @@ jobs:
done
- name: Upload prebuilt fuzz binaries
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fuzz-prebuilt-binaries-${{ github.run_number }}
path: fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release/
@@ -143,12 +143,12 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Download prebuilt fuzz binaries
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: fuzz-prebuilt-binaries-${{ github.run_number }}
path: fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release
@@ -167,7 +167,7 @@ jobs:
- name: Upload fuzz smoke artifacts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fuzz-smoke-${{ matrix.target }}-${{ github.run_number }}
path: |
@@ -197,12 +197,12 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Download prebuilt fuzz binaries
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: fuzz-prebuilt-binaries-${{ github.run_number }}
path: fuzz/prebuilt/${{ env.CARGO_BUILD_TARGET }}/release
@@ -221,7 +221,7 @@ jobs:
- name: Upload nightly fuzz artifacts
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: fuzz-nightly-${{ matrix.target }}-${{ github.run_number }}
path: |
@@ -240,13 +240,13 @@ jobs:
# job fails. Alerts only for scheduled (nightly) runs (backlog#1149
# ci-8); PR and manual dispatch failures are already watched by a human.
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+42 -7
View File
@@ -31,7 +31,7 @@ permissions:
jobs:
build-helm-package:
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
if: |
(github.event_name == 'workflow_dispatch' && !contains(github.event.inputs.version, '-preview')) ||
@@ -48,9 +48,10 @@ jobs:
chart_version: ${{ steps.version.outputs.chart_version }}
steps:
- name: Checkout helm chart repo
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- name: Checkout release source
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
ref: ${{ github.event.workflow_run.head_sha || (startsWith(inputs.version, 'refs/tags/') && inputs.version || format('refs/tags/{0}', inputs.version)) }}
persist-credentials: false
# Both inputs reach the shell through env rather than `${{ }}`
@@ -83,6 +84,40 @@ jobs:
./scripts/helm_chart_version.sh "$RAW_TAG"
# Docker and Helm share the binary-build trigger. Do not publish a chart
# until the image selected by its default appVersion is pullable.
- name: Wait for release image
timeout-minutes: 20
env:
APP_VERSION: ${{ steps.version.outputs.app_version }}
run: |
set -euo pipefail
if [[ ! "$APP_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] ||
[[ "$APP_VERSION" == *-preview* ]]; then
echo "Invalid deliverable image version: $APP_VERSION" >&2
exit 1
fi
for attempt in {1..60}; do
if token=$(curl --fail --silent --show-error --connect-timeout 10 --max-time 20 \
'https://auth.docker.io/token?service=registry.docker.io&scope=repository:rustfs/rustfs:pull' \
| jq -er '.token // empty') &&
curl --fail --silent --show-error --head --connect-timeout 10 --max-time 20 \
--header "Authorization: Bearer $token" \
--header 'Accept: application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json' \
"https://registry-1.docker.io/v2/rustfs/rustfs/manifests/$APP_VERSION" > /dev/null; then
echo "Release image is available: rustfs/rustfs:$APP_VERSION"
exit 0
fi
echo "Waiting for rustfs/rustfs:$APP_VERSION ($attempt/60)"
if (( attempt < 60 )); then
sleep 15
fi
done
echo "Release image is unavailable; refusing to publish Helm chart" >&2
exit 1
- name: Replace chart version and app version
env:
CHART_VERSION: ${{ steps.version.outputs.chart_version }}
@@ -107,21 +142,21 @@ jobs:
--version "${{ steps.version.outputs.chart_version }}"
- name: Upload helm package as artifact
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: helm-package
path: helm/rustfs/*.tgz
retention-days: 1
publish-helm-package:
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
needs: [ build-helm-package ]
if: needs.build-helm-package.result == 'success'
steps:
- name: Checkout helm package repo
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
# persist-credentials-exempt: this checkout's token IS the push credential —
# the job git-pushes to rustfs/helm below. Clearing it breaks chart publishing.
@@ -129,7 +164,7 @@ jobs:
token: ${{ secrets.RUSTFS_HELM_PACKAGE }}
- name: Download helm package
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: helm-package
path: ./
+1 -1
View File
@@ -32,7 +32,7 @@ permissions:
jobs:
build:
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- uses: usthe/issues-translate-action@b41f55ddc81d7d54bd542a4f289fe28ec081898e # v2.7
+4 -4
View File
@@ -62,7 +62,7 @@ jobs:
name: MinIO interop (EC + SSE read parity)
# Skip on forks: needs the repo's runners and is not a contributor gate.
if: github.repository == 'rustfs/rustfs'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 40
env:
# Fixed 32-byte test KMS key baked into the fixture lab; not a secret.
@@ -83,7 +83,7 @@ jobs:
INTEROP_REQUIRED_TESTS: '["reads_minio_generated_sse_s3_multipart_fixture", "reads_minio_generated_sse_kms_multipart_fixture", "rejects_minio_generated_sse_s3_fixture_with_wrong_kms_key", "rejects_minio_generated_sse_s3_fixture_with_truncated_ciphertext"]'
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -131,13 +131,13 @@ jobs:
name: Alert on scheduled failure
needs: [minio-interop]
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+5 -5
View File
@@ -114,10 +114,10 @@ jobs:
# label was abandoned (no docker.sock on the pod its only run landed on).
# TODO(ci-8): scheduled-failure alerting (auto-open issue on a red weekly
# run) is added by the ci-8 composite action; do not implement it here.
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 120
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -246,7 +246,7 @@ jobs:
- name: Upload artifacts
if: always() && env.ACT != 'true'
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: mint
path: artifacts/**
@@ -258,13 +258,13 @@ jobs:
# job fails. Alerts only for scheduled runs (backlog#1149 ci-8) — manual
# dispatch failures are already watched by a human.
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+16 -13
View File
@@ -44,14 +44,14 @@ env:
jobs:
resolve-source:
name: Resolve nightly source
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
outputs:
source_sha: ${{ steps.source.outputs.sha }}
source_ref: ${{ steps.source.outputs.ref }}
steps:
- name: Checkout selected source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
ref: ${{ env.NIGHTLY_BUILD_REF }}
@@ -71,7 +71,7 @@ jobs:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
ref: ${{ needs.resolve-source.outputs.source_sha }}
@@ -237,12 +237,12 @@ jobs:
echo "rpm_file=$RPM_FILE" >> "$GITHUB_OUTPUT"
- name: Upload DEB artifact
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ steps.deb.outputs.deb_file }}
path: ${{ steps.deb.outputs.deb_file }}
- name: Upload RPM artifact
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ steps.rpm.outputs.rpm_file }}
path: ${{ steps.rpm.outputs.rpm_file }}
@@ -328,7 +328,7 @@ jobs:
- name: Upload nightly candidate manifest
if: ${{ steps.publish.outputs.candidate_file != '' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: nightly-candidate-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ steps.publish.outputs.candidate_file }}
@@ -360,9 +360,12 @@ jobs:
# live scripts (AppRole login, three-node Raft leader failover).
#
# GitHub-hosted ubuntu-latest, deliberately not the self-hosted sm-standard
# fleet: the HA failover script needs a working Docker daemon, and the
# self-hosted fleet is heterogeneous — a docker-dependent workflow has been
# burned by it before (see the banner in e2e-s3tests.yml, rustfs/backlog#1149).
# fleet: both lanes need a working Docker daemon, and the sm-standard-2 pool
# does not provide one. #8112 rerouted these lanes there and both died in
# seconds on "Docker is not available" while the package itself built and
# published fine, which kept the nightly chain from firing (its gate
# requires a successful build run). See also the banner in e2e-s3tests.yml
# and rustfs/backlog#1149.
kms-vault-lane:
needs: resolve-source
name: KMS live Vault lane
@@ -383,7 +386,7 @@ jobs:
NO_PROXY: 127.0.0.1,localhost
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
ref: ${{ needs.resolve-source.outputs.source_sha }}
@@ -473,7 +476,7 @@ jobs:
NO_PROXY: 127.0.0.1,localhost
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
ref: ${{ needs.resolve-source.outputs.source_sha }}
@@ -495,13 +498,13 @@ jobs:
if: >-
always() && github.event_name == 'schedule' &&
(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+6 -6
View File
@@ -33,30 +33,30 @@ concurrency:
jobs:
update-flake:
name: Update flake.lock
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 90
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
# persist-credentials-exempt: update-flake-lock pushes the branch and opens
# the PR. It passes FLAKE_UPDATE_TOKEN to create-pull-request itself rather
# than reusing .git/config, but that is unverified — exempt until a
# workflow_dispatch run confirms it (rustfs/backlog#1602).
- name: Install Nix
uses: DeterminateSystems/determinate-nix-action@629b284231c2a82554b724e357e47fc6020833c8 # v3
uses: DeterminateSystems/determinate-nix-action@8d87e8d5e5b8a8309d4281094560f127d9a265f1 # v3.22.5
- name: Cache Nix
uses: DeterminateSystems/flakehub-cache-action@1f9a51a2959d3e26c7838c6f3bf9f48acae525ea # v3.20.0
uses: DeterminateSystems/flakehub-cache-action@83282a8aef353db1d659d3bacf4a28b6683de0b3 # v3.22.5
- name: Check Nix flake inputs
uses: DeterminateSystems/flake-checker-action@3164002371bc90729c68af0e24d5aacf20d7c9f6 # v12
uses: DeterminateSystems/flake-checker-action@786422608c7bded2bbc9741ad9f91356842bf520 # v14
- name: Update flake.lock
id: update
uses: DeterminateSystems/update-flake-lock@fd9359ac79d0e912f1b4b947a48470b3e2799b56 # main
uses: DeterminateSystems/update-flake-lock@da03c0f078bc4b2c37ee4f7e072d34bf8f188bb3 # v29
with:
git-author-name: houseme
git-author-email: housemecn@gmail.com
+6 -6
View File
@@ -54,7 +54,7 @@ jobs:
cancel-closed-pr-runs:
name: Cancel Closed PR Runs
if: github.event_name == 'pull_request' && github.event.action == 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- name: Explain cancellation run
@@ -63,7 +63,7 @@ jobs:
nix-validation:
name: Nix Build & Check
if: github.event_name != 'pull_request' || github.event.action != 'closed'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 60
permissions:
contents: read
@@ -72,12 +72,12 @@ jobs:
NIX_CURL_FLAGS: -A cargo/stable
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Install Nix
uses: DeterminateSystems/determinate-nix-action@4eea0b33e3d1f02ecfe37cf16e7204c424009606 # v3.21.0
uses: DeterminateSystems/determinate-nix-action@8d87e8d5e5b8a8309d4281094560f127d9a265f1 # v3.22.5
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
extra-conf: |
@@ -86,10 +86,10 @@ jobs:
max-jobs = 1
- name: Cache Nix
uses: DeterminateSystems/flakehub-cache-action@c01e819d047464c3edf6ba778f075952af5a3aa7 # v3.21.0
uses: DeterminateSystems/flakehub-cache-action@83282a8aef353db1d659d3bacf4a28b6683de0b3 # v3.22.5
- name: Check Nix Flake Inputs
uses: DeterminateSystems/flake-checker-action@3164002371bc90729c68af0e24d5aacf20d7c9f6 # v12
uses: DeterminateSystems/flake-checker-action@786422608c7bded2bbc9741ad9f91356842bf520 # v14
with:
fail-mode: true
ignore-missing-flake-lock: false
+5 -5
View File
@@ -51,13 +51,13 @@ concurrency:
jobs:
oidc-keycloak-live:
name: OIDC Keycloak live gate
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 60
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -82,7 +82,7 @@ jobs:
- name: Upload service logs
if: failure()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: oidc-keycloak-live-${{ github.run_number }}
path: ${{ runner.temp }}/rustfs-keycloak-live-*/**/*.log
@@ -95,13 +95,13 @@ jobs:
if: >-
always() && github.event_name == 'schedule' &&
(needs.oidc-keycloak-live.result == 'failure' || needs.oidc-keycloak-live.result == 'cancelled')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
@@ -72,7 +72,7 @@ jobs:
# Skip on forks: needs this repository's runners and is not a contributor
# gate.
if: github.repository == 'rustfs/rustfs'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 90
env:
NO_PROXY: 127.0.0.1,localhost
@@ -93,7 +93,7 @@ jobs:
NEXTEST_LISTING: ${{ github.workspace }}/artifacts/odm-interop/minio/selection.json
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -166,7 +166,7 @@ jobs:
- name: Upload the MinIO interop report
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: odm-interop-minio-${{ github.run_number }}-${{ github.run_attempt }}
path: |
@@ -183,7 +183,7 @@ jobs:
cloud-source:
name: ${{ matrix.provider }} source (three-case minimum)
if: github.repository == 'rustfs/rustfs'
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 45
strategy:
fail-fast: false
@@ -237,7 +237,7 @@ jobs:
- name: Checkout repository
if: steps.credentials.outputs.present == 'true'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -286,7 +286,7 @@ jobs:
- name: Upload the ${{ matrix.provider }} interop report
if: always() && steps.credentials.outputs.present == 'true'
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: odm-interop-${{ matrix.provider }}-${{ github.run_number }}-${{ github.run_attempt }}
path: |
@@ -300,13 +300,13 @@ jobs:
if: >-
always() && github.event_name == 'schedule' &&
(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+67 -24
View File
@@ -25,9 +25,10 @@
#
# Flow:
# 1. Resolve and validate the selected Build workflow run and source identity
# 2. Download Linux binaries (x86_64-gnu, aarch64-gnu) from build artifacts
# 3. Build DEB packages for amd64 and arm64
# 4. Build RPM packages for x86_64 and aarch64
# 2. Download Linux binaries (x86_64-gnu, aarch64-gnu, x86_64-musl,
# aarch64-musl) from build artifacts
# 3. Build DEB packages for amd64 and arm64, gnu and musl variants
# 4. Build RPM packages for x86_64 and aarch64, gnu and musl variants
# 5. Upload all packages to Cloudflare R2 and the GitHub release
name: Package DEB/RPM
@@ -76,7 +77,7 @@ jobs:
(github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'push' &&
github.event.workflow_run.head_branch != 'main')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
outputs:
version: ${{ steps.resolve.outputs.version }}
@@ -224,11 +225,11 @@ jobs:
echo " Build run ID: $BUILD_RUN_ID"
echo " Build run number: $RUN_NUMBER"
# Build DEB and RPM packages for each architecture
# Build DEB and RPM packages for each architecture and libc variant
package:
name: Package (${{ matrix.arch }})
name: Package (${{ matrix.arch }}-${{ matrix.libc }})
needs: resolve
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
strategy:
fail-fast: false
@@ -236,16 +237,38 @@ jobs:
matrix:
include:
- arch: x86_64
libc: gnu
deb_arch: amd64
rpm_arch: x86_64
artifact_name: "rustfs-linux-x86_64-gnu"
# gnu binaries dynamically link glibc; musl binaries are static
# and run without a libc dependency.
deb_depends: "libc6 (>= 2.31)"
rpm_depends: "glibc >= 2.31"
- arch: aarch64
libc: gnu
deb_arch: arm64
rpm_arch: aarch64
artifact_name: "rustfs-linux-aarch64-gnu"
deb_depends: "libc6 (>= 2.31)"
rpm_depends: "glibc >= 2.31"
- arch: x86_64
libc: musl
deb_arch: amd64
rpm_arch: x86_64
artifact_name: "rustfs-linux-x86_64-musl"
deb_depends: ""
rpm_depends: ""
- arch: aarch64
libc: musl
deb_arch: arm64
rpm_arch: aarch64
artifact_name: "rustfs-linux-aarch64-musl"
deb_depends: ""
rpm_depends: ""
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -256,17 +279,17 @@ jobs:
BUILD_TYPE: ${{ needs.resolve.outputs.build_type }}
SOURCE_VERSION: ${{ needs.resolve.outputs.version }}
DEV_SEQUENCE: ${{ needs.resolve.outputs.dev_sequence }}
DEB_ARCH: ${{ matrix.deb_arch }}
RPM_ARCH: ${{ matrix.rpm_arch }}
ARCH: ${{ matrix.rpm_arch }}
LIBC: ${{ matrix.libc }}
run: |
set -euo pipefail
normalized=$(./scripts/release/package_versions.sh \
"$BUILD_TYPE" "$SOURCE_VERSION" "$DEV_SEQUENCE" "$DEB_ARCH" "$RPM_ARCH")
"$BUILD_TYPE" "$SOURCE_VERSION" "$DEV_SEQUENCE" "$ARCH" "$LIBC")
printf '%s\n' "$normalized" >> "$GITHUB_OUTPUT"
- name: Download binary artifact from build run
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: ${{ matrix.artifact_name }}*
path: ./binary-artifact
@@ -308,6 +331,8 @@ jobs:
DEB_VERSION: ${{ steps.versions.outputs.deb_version }}
DEB_ARCH: ${{ matrix.deb_arch }}
DEB_FILE: ${{ steps.versions.outputs.deb_file }}
DEB_DEPENDS: ${{ matrix.deb_depends }}
LIBC: ${{ matrix.libc }}
run: |
set -euo pipefail
@@ -345,9 +370,17 @@ jobs:
Section: utils
Priority: optional
Architecture: ${DEB_ARCH}
Depends: libc6 (>= 2.31)
EOF
# gnu binaries dynamically link glibc and must declare it; musl
# binaries are statically linked and need no libc dependency.
if [[ -n "$DEB_DEPENDS" ]]; then
echo "Depends: ${DEB_DEPENDS}" >> "${PKG_DIR}/DEBIAN/control"
fi
cat >> "${PKG_DIR}/DEBIAN/control" << EOF
Maintainer: RustFS Team <support@rustfs.com>
Description: High-performance distributed object storage
Description: High-performance distributed object storage (${LIBC} build)
RustFS is a high-performance distributed object storage software
built using Rust. It is compatible with MinIO and S3 API.
Homepage: https://rustfs.com
@@ -389,15 +422,24 @@ jobs:
RPM_RELEASE: ${{ steps.versions.outputs.rpm_release }}
RPM_ARCH: ${{ matrix.rpm_arch }}
RPM_FILE: ${{ steps.versions.outputs.rpm_file }}
RPM_DEPENDS: ${{ matrix.rpm_depends }}
LIBC: ${{ matrix.libc }}
run: |
set -euo pipefail
echo "Building RPM for ${RPM_ARCH}"
echo "Building RPM for ${RPM_ARCH} (${LIBC})"
sudo apt-get update && sudo apt-get install -y ruby ruby-dev build-essential rpm
sudo gem install fpm
./scripts/test_package_versions.sh --require-package-managers
# gnu binaries dynamically link glibc and must declare it; musl
# binaries are statically linked and need no libc dependency.
depends_args=()
if [[ -n "$RPM_DEPENDS" ]]; then
depends_args+=(--depends "$RPM_DEPENDS")
fi
# Create config file for fpm (DEB build creates it in its package dir structure,
# but fpm needs the file to exist before packaging)
mkdir -p ./tmp-pkg/etc/default
@@ -420,9 +462,9 @@ jobs:
--iteration "$RPM_RELEASE" \
--architecture "$RPM_ARCH" \
--package "$RPM_FILE" \
--depends "glibc >= 2.31" \
"${depends_args[@]}" \
--description "High-performance distributed object storage (${LIBC} build)" \
--maintainer "RustFS Team <support@rustfs.com>" \
--description "High-performance distributed object storage" \
--url "https://rustfs.com" \
--license "Apache-2.0" \
--before-install <(./scripts/release/package_service_scripts.sh rpm-before-install) \
@@ -453,9 +495,9 @@ jobs:
echo "✅ RPM built: $RPM_FILE"
- name: Upload packages to artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: packages-${{ matrix.arch }}
name: packages-${{ matrix.arch }}-${{ matrix.libc }}
path: |
*.deb
*.rpm
@@ -551,10 +593,11 @@ jobs:
for f in "$DEB_FILE" "$RPM_FILE"; do
if [[ -n "$f" && -f "$f" ]]; then
base="$(basename "$f")"
# GitHub stores release asset names with '~' normalized to '.'
# (e.g. rustfs_1.0.0~rc.2_amd64.deb is stored as
# rustfs_1.0.0.rc.2_amd64.deb), so checksum entries must
# reference the name as stored on the release.
# GitHub stores release asset names with '~' normalized to '.'.
# Current package file names embed the raw tag and contain no
# '~', but older releases may still ship such assets, so the
# checksum entries must reference the name as stored on the
# release.
github_base="${base//\~/.}"
# Remove any stale entry (both naming variants), then append
grep -Fv -- "$base" "$checksum_file" > "${checksum_file}.tmp" || true
@@ -576,7 +619,7 @@ jobs:
name: Summary
needs: [ resolve, package ]
if: always()
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 5
steps:
- name: Print summary
+5 -5
View File
@@ -57,7 +57,7 @@ jobs:
timeout-minutes: 180
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
fetch-depth: 0 # baseline may be an earlier successful scheduled head
@@ -94,7 +94,7 @@ jobs:
- name: Find last successful scheduled baseline
id: scheduled_baseline
if: github.event_name == 'schedule'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
result-encoding: string
script: |
@@ -271,7 +271,7 @@ jobs:
- name: Upload A/B results
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: hotpath-warp-ab-${{ github.run_number }}
# Includes per-cell median_summary.csv / baseline_compare.csv, both gates,
@@ -366,13 +366,13 @@ jobs:
if: >-
always() && github.event_name == 'schedule' &&
(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+4 -4
View File
@@ -43,11 +43,11 @@ concurrency:
jobs:
check-ephemerality:
name: Check runner ephemerality
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -66,13 +66,13 @@ jobs:
# scheduled runs file a tracking issue, manual dispatch stays quiet so
# debugging never produces a spurious alert.
if: always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
@@ -12,7 +12,7 @@
# assert the expected outage
# M-<topo>EC<P>-rec after restoring every node, reads/writes succeed
#
# 10 topologies x EC levels = 38 combinations, 320 cases. The full sweep runs
# 9 topologies x EC levels = 36 combinations, 300 cases. The full sweep runs
# 3-5 hours and is dispatched on demand (e.g. after a nightly chain goes
# green), not nightly. Case ids and expectations come from product source
# (default_parity_count, erasure set sizing) via rustfs/auto-testing.
@@ -32,7 +32,7 @@ on:
default: main
type: string
matrix_topo:
description: 'Topology filter like 4x2; empty sweeps all ten topologies'
description: 'Topology filter like 4x2; empty sweeps all nine topologies'
required: false
default: ''
type: string
@@ -95,7 +95,7 @@ jobs:
- name: Checkout auto-testing scripts
timeout-minutes: 5
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ inputs.auto_testing_ref }}
@@ -174,12 +174,12 @@ jobs:
} > "${REPORT_FILE}"
cat "${REPORT_FILE}" >> "${GITHUB_STEP_SUMMARY}"
# An unfiltered sweep must land exactly 320 cases (38 topology x EC
# An unfiltered sweep must land exactly 300 cases (36 topology x EC
# combinations at 2*(nodes+1) cases each); filtered sweeps are held
# to internal consistency. Missing/harness-error rows always fail.
EXPECTED_TOTAL=""
if [ -z "${{ inputs.matrix_topo }}" ] && [ -z "${MATRIX_EC}" ]; then
EXPECTED_TOTAL=320
EXPECTED_TOTAL=300
fi
jq -e --arg testing_sha "${AUTO_TESTING_SHA}" --argjson expected_total "${EXPECTED_TOTAL:-null}" '
.schema_version == 1 and
@@ -204,7 +204,7 @@ jobs:
- name: Upload test logs & evidence
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-fault-tolerance-matrix-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -83,7 +83,7 @@ jobs:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
steps:
- name: Checkout repository (for report parser)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -113,7 +113,7 @@ jobs:
- name: Checkout auto-testing scripts
timeout-minutes: 5
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || inputs.auto_testing_ref || 'main' }}
@@ -294,7 +294,7 @@ jobs:
- name: Upload test logs & evidence
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-fault-tolerance-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -386,7 +386,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-fault-tolerance-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-fault-tolerance-${{ github.run_id }}-${{ github.run_attempt }}/fault-tolerance.json
@@ -42,13 +42,13 @@ concurrency:
jobs:
prepare:
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.event == 'schedule' }}
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
outputs:
manifest: ${{ steps.candidate.outputs.manifest }}
performance_ready: ${{ steps.perf_probe.outputs.performance_ready }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Resolve published candidate
@@ -63,7 +63,7 @@ jobs:
CHAIN_OUTPUT: ${{ runner.temp }}/chain-candidate.json
run: python3 scripts/resolve_functional_candidate.py
- name: Retain candidate identity
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-candidate-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-candidate.json
@@ -188,16 +188,16 @@ jobs:
complete-chain:
needs: [prepare, upgrade, s3, kms, tier, storage, heal, pool, security, replication, fault-tolerance, table, performance]
if: ${{ always() && needs.prepare.result != 'skipped' }}
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Download suite evidence
id: download
continue-on-error: true
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: functional-chain-*-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-evidence
@@ -213,7 +213,7 @@ jobs:
--output "${RUNNER_TEMP}/chain-report.json"
- name: Retain chain report regardless of test verdict
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-report-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -231,7 +231,7 @@ jobs:
--directory "${RUNNER_TEMP}/chain-evidence"
--output "${RUNNER_TEMP}/chain-complete.json"
- name: Upload complete-chain evidence
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-complete-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-complete.json
+4 -4
View File
@@ -66,7 +66,7 @@ jobs:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
steps:
- name: Checkout chain tooling
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -93,7 +93,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -366,7 +366,7 @@ jobs:
- name: Upload test logs
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-heal-test-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -467,7 +467,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-heal-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-heal-${{ github.run_id }}-${{ github.run_attempt }}/heal.json
+4 -4
View File
@@ -59,7 +59,7 @@ jobs:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
steps:
- name: Checkout repository (for report parser)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -85,7 +85,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -315,7 +315,7 @@ jobs:
- name: Upload report and logs
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-kms-test-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -414,7 +414,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-kms-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-kms-${{ github.run_id }}-${{ github.run_attempt }}/kms.json
@@ -90,10 +90,10 @@ env:
jobs:
runner-check:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 5
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Check performance runner before entering its queue
@@ -116,7 +116,7 @@ jobs:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
steps:
- name: Checkout chain tooling
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -144,7 +144,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -307,7 +307,7 @@ jobs:
- name: Upload test logs & results
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-perf-test-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -349,7 +349,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-performance-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-performance-${{ github.run_id }}-${{ github.run_attempt }}/performance.json
@@ -92,7 +92,7 @@ jobs:
RUSTFS_POOL_NODE_ENDPOINTS: ${{ secrets.RUSTFS_POOL_NODE_ENDPOINTS || vars.RUSTFS_POOL_NODE_ENDPOINTS || 'http://rustfs-node1:9000 http://rustfs-node2:9000 http://rustfs-node3:9000 http://rustfs-node4:9000' }}
steps:
- name: Checkout chain tooling
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -104,7 +104,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -656,7 +656,7 @@ jobs:
- name: Upload test logs
timeout-minutes: 2
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-pool-test-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/rustfs-pool-${{ github.run_id }}-${{ github.run_attempt }}
@@ -764,7 +764,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-pool-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-pool-${{ github.run_id }}-${{ github.run_attempt }}/pool.json
@@ -71,7 +71,7 @@ jobs:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
steps:
- name: Checkout repository (for report parser)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -97,7 +97,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -323,7 +323,7 @@ jobs:
- name: Upload report and logs
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-replication-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -419,7 +419,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-replication-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-replication-${{ github.run_id }}-${{ github.run_attempt }}/replication.json
+4 -4
View File
@@ -47,7 +47,7 @@ jobs:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
steps:
- name: Checkout repository (for report parser)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -73,7 +73,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -292,7 +292,7 @@ jobs:
- name: Upload report and logs
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-s3-compat-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -391,7 +391,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-s3-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-s3-${{ github.run_id }}-${{ github.run_attempt }}/s3.json
+4 -4
View File
@@ -87,7 +87,7 @@ jobs:
# the workspace root would wipe the auto-testing clone above (that is
# exactly how run 33934141181 lost rustfs-security-test.sh).
- name: Checkout repository (for the OIDC live gate script)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
path: rustfs-repo
@@ -109,7 +109,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -325,7 +325,7 @@ jobs:
- name: Upload report and logs
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-security-test-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -395,7 +395,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-security-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-security-${{ github.run_id }}-${{ github.run_attempt }}/security.json
+4 -4
View File
@@ -56,7 +56,7 @@ jobs:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
steps:
- name: Checkout repository (for report parser)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -82,7 +82,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -307,7 +307,7 @@ jobs:
- name: Upload report and logs
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-storage-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -406,7 +406,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-storage-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-storage-${{ github.run_id }}-${{ github.run_attempt }}/storage.json
+5 -5
View File
@@ -46,7 +46,7 @@ jobs:
timeout-minutes: 60
steps:
- name: Checkout repository (for report parser)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -73,7 +73,7 @@ jobs:
- name: Checkout auto-testing scripts
timeout-minutes: 5
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -133,7 +133,7 @@ jobs:
if [ -n "${PACKAGE_URL}" ]; then
ARGS+=(--package-url "${PACKAGE_URL}")
elif [ -n "${RUSTFS_VERSION}" ] && [ "${RUSTFS_VERSION}" != "null" ]; then
ARGS+=(--package-url "https://github.com/rustfs/rustfs/releases/download/${RUSTFS_VERSION}/rustfs_${RUSTFS_VERSION}_amd64.deb")
ARGS+=(--package-url "https://github.com/rustfs/rustfs/releases/download/${RUSTFS_VERSION}/rustfs-linux-x86_64-gnu-v${RUSTFS_VERSION}.deb")
else
ARGS+=(--package-url "${RUSTFS_NIGHTLY_PACKAGE_URL}")
fi
@@ -281,7 +281,7 @@ jobs:
- name: Upload report and logs
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-table-test-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -353,7 +353,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-table-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-table-${{ github.run_id }}-${{ github.run_attempt }}/table.json
+4 -4
View File
@@ -71,7 +71,7 @@ jobs:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
steps:
- name: Checkout chain tooling
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -95,7 +95,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -460,7 +460,7 @@ jobs:
- name: Upload report and logs
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-tier-test-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ env.TIER_ARTIFACTS_DIR }}/
@@ -734,7 +734,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-tier-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-tier-${{ github.run_id }}-${{ github.run_attempt }}/tier.json
+5 -5
View File
@@ -89,7 +89,7 @@ jobs:
if: ${{ inputs.chain_manifest != '' || github.event_name == 'workflow_dispatch' || github.event_name == 'repository_dispatch' }}
steps:
- name: Checkout repository (for report parser)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -115,7 +115,7 @@ jobs:
run: python3 scripts/functional_chain_evidence.py consume
- name: Checkout auto-testing scripts
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
repository: rustfs/auto-testing
ref: ${{ steps.chain.outputs.testing_sha || 'main' }}
@@ -207,7 +207,7 @@ jobs:
local version="$1" tag asset url
[ -n "${version}" ] && [ "${version}" != "null" ] || return 0
tag="${version#v}"
asset="rustfs_${tag//-/.}_amd64.deb"
asset="rustfs-linux-x86_64-gnu-v${tag}.deb"
url="https://github.com/rustfs/rustfs/releases/download/${tag}/${asset}"
if ! gh api "repos/rustfs/rustfs/releases/tags/${tag}" --jq '.assets[].name' 2>/dev/null | grep -qxF "${asset}"; then
echo "ERROR: release ${tag} has no downloadable asset ${asset}:" >&2
@@ -400,7 +400,7 @@ jobs:
- name: Upload report and logs
timeout-minutes: 2
if: ${{ always() && steps.evidence.outcome == 'success' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: rustfs-upgrade-test-${{ github.run_id }}-${{ github.run_attempt }}
path: |
@@ -503,7 +503,7 @@ jobs:
- name: Upload chain evidence
if: ${{ always() && steps.chain_record.outputs.written == 'true' }}
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: functional-chain-upgrade-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/chain-upgrade-${{ github.run_id }}-${{ github.run_attempt }}/upgrade.json
@@ -44,7 +44,7 @@ permissions:
jobs:
forced-failure:
name: Forced failure
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 5
steps:
- name: Fail on purpose
@@ -58,13 +58,13 @@ jobs:
# Mirrors the consumer wiring, minus the schedule-event guard (this
# workflow is dispatch-only by design).
if: always() && contains(needs.*.result, 'failure')
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
@@ -29,6 +29,9 @@ concurrency:
jobs:
check-freshness:
name: Check scheduled validation freshness
# Stay on GitHub-hosted runners: this monitor must keep running when the
# self-hosted sm-standard-* pool is down, which is exactly when the
# scheduled validations it watches stall.
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
@@ -36,7 +39,7 @@ jobs:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Check latest scheduled runs
@@ -42,6 +42,9 @@ jobs:
github.event.workflow_run.event == 'schedule' &&
github.event.workflow_run.conclusion != 'success' &&
github.event.workflow_run.conclusion != 'failure'
# Stay on GitHub-hosted runners: this monitor must keep running when the
# self-hosted sm-standard-* pool is down, which is exactly when the
# scheduled validations it watches stall.
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
@@ -49,7 +52,7 @@ jobs:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update incomplete-run issue
+2 -2
View File
@@ -26,10 +26,10 @@ on:
jobs:
stale:
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 30
steps:
- uses: actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9
- uses: actions/stale@4391f3da665fdf50b6810c1a66712fb9ba21aa93 # v11.0.0
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: 'This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.'
+1 -1
View File
@@ -14,7 +14,7 @@ concurrency:
jobs:
update:
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
steps:
- uses: overtrue/repo-visuals-action@ee2c632f6ce617e851fb46ea935ee8af762ebb93 # v1.4.0
+5 -5
View File
@@ -41,7 +41,7 @@ env:
jobs:
targets-live:
name: PostgreSQL, MySQL, AMQP, and NATS
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 90
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
@@ -52,7 +52,7 @@ jobs:
RUSTFS_TEST_NATS_URL: nats://127.0.0.1:4222
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
@@ -166,7 +166,7 @@ jobs:
- name: Upload target integration diagnostics
if: always()
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: targets-integration-${{ github.run_number }}-${{ github.run_attempt }}
path: artifacts/targets-live
@@ -177,13 +177,13 @@ jobs:
if: >-
always() && github.event_name == 'schedule' &&
(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled'))
runs-on: ubuntu-latest
runs-on: sm-standard-2
timeout-minutes: 10
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
- name: Open or update failure-tracking issue
+1 -1
View File
@@ -56,7 +56,7 @@ jobs:
timeout-minutes: 60
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
uses: actions/checkout@f548e57e544e1ff5a4c46bf1e1b8685f8e4a348a # v7
with:
persist-credentials: false
Generated
+242 -271
View File
File diff suppressed because it is too large Load Diff
+16 -16
View File
@@ -156,9 +156,9 @@ futures-lite = "2.6.1"
futures-util = "0.3.34"
pollster = "1.0.1"
pulsar = { default-features = false, version = "6.9.0" }
lapin = { default-features = false, version = "4.11.0" }
lapin = { default-features = false, version = "4.12.0" }
hyper = { version = "1.11.1" }
hyper-rustls = { default-features = false, version = "0.27.9" }
hyper-rustls = { default-features = false, version = "0.27.10" }
hyper-util = { version = "0.1.20" }
http = "1.5.0"
http-body = "1.1.0"
@@ -244,13 +244,13 @@ atoi = "3.1.0"
atomic_enum = "0.3.0"
aws-config = { version = "1.12.0" }
aws-credential-types = { version = "1.3.0" }
aws-sdk-kms = { default-features = false, version = "1.120.0" }
aws-sdk-s3 = { default-features = false, version = "1.148.0" }
aws-sdk-sts = { default-features = false, version = "1.116.0" }
aws-sdk-kms = { default-features = false, version = "1.121.0" }
aws-sdk-s3 = { default-features = false, version = "1.149.0" }
aws-sdk-sts = { default-features = false, version = "1.117.0" }
aws-smithy-async = { version = "1.3.0" }
aws-smithy-http-client = { default-features = false, version = "1.4.2" }
aws-smithy-runtime-api = { version = "1.16.2" }
aws-smithy-types = { version = "1.6.4" }
aws-smithy-runtime-api = { version = "1.17.0" }
aws-smithy-types = { version = "1.8.0" }
base64-simd = "0.8.0"
brotli = "9.0.0"
clap = { version = "4.6.7" }
@@ -265,7 +265,7 @@ datafusion = { default-features = false, version = "55.1.0" }
derive_builder = "0.20.2"
ed25519-dalek = "3.0.0"
enumset = "1.1.14"
faster-hex = "0.10.0"
faster-hex = "1.0.0"
flate2 = "1.1.10"
glob = "0.3.4"
google-cloud-storage = "1.18.0"
@@ -301,7 +301,7 @@ percent-encoding = "2.3.2"
qrcode-rs = { version = "2.1.1", default-features = false, features = ["std", "svg"] }
pin-project-lite = "0.2.17"
pretty_assertions = "1.4.1"
rand = { version = "0.10.2" }
rand = { version = "0.10.3" }
ratelimit = "2.0.0"
rayon = "1.12.0"
rustfs-erasure-codec = { version = "9.0.0" }
@@ -313,8 +313,8 @@ rustify = { version = "0.7", default-features = false }
rustix = { version = "1.1.5" }
rust-embed = { version = "8.12.0" }
rustc-hash = { version = "2.1.3" }
s3s = { git = "https://github.com/s3s-project/s3s.git", rev = "50c94aeb1e5ea9ef8e7393ef6bbc87b72f6891c6", features = ["minio"] }
s3s-sigv4 = { git = "https://github.com/s3s-project/s3s.git", rev = "50c94aeb1e5ea9ef8e7393ef6bbc87b72f6891c6" }
s3s = { version = "0.17.0", features = ["minio"] }
s3s-sigv4 = "0.17.0"
serial_test = "4.0.1"
shadow-rs = { default-features = false, version = "2.0.0" }
siphasher = "1.0.3"
@@ -326,8 +326,8 @@ strum = { version = "0.28.0" }
sysinfo = "0.39.6"
temp-env = "0.3.6"
tempfile = "3.27.0"
test-case = "3.3.1"
thiserror = "2.0.20"
test-case = "3.4.0"
thiserror = "2.0.21"
tracing = { version = "0.1.44" }
tracing-appender = "0.2.5"
tracing-core = "0.1.36"
@@ -351,7 +351,7 @@ zstd = "0.14.0"
# Observability and Metrics
metrics = "0.24.6"
metrics-util = "0.20"
dial9 = { version = "0.5.1", default-features = false }
dial9 = { version = "0.5.2", default-features = false }
opentelemetry = { version = "0.32.0" }
opentelemetry-appender-tracing = { version = "0.32.0" }
opentelemetry-otlp = { version = "0.32.0" }
@@ -364,7 +364,7 @@ pyroscope = { version = "2.1.1" }
# FTP and SFTP
libunftp = { version = "0.23.0" }
unftp-core = "0.1.0"
suppaftp = { version = "12.0.0" }
suppaftp = { version = "12.0.1" }
rcgen = { version = "0.14.10", default-features = false, features = ["aws_lc_rs", "crypto", "pem"] }
russh = { version = "0.63.3" }
russh-sftp = "3.0.0"
@@ -373,7 +373,7 @@ russh-sftp = "3.0.0"
dav-server = "0.11.0"
# Performance Analysis and Memory Profiling
rustfs-mimalloc = { version = "0.5.4" }
rustfs-mimalloc = { version = "0.5.6" }
# Preserve Unicode focus filters until rustfs/backlog#2302 is resolved.
hotpath = { version = "0.26.0", default-features = false }
# Snapshot testing for output format regression detection
+1 -1
View File
@@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2024 Beijing Henghesha Technology Co., Ltd.
Copyright 2024-2026 RustFS, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
+1 -1
View File
@@ -141,7 +141,7 @@ chown -R 10001:10001 data logs
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
# Using specific version
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.1
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0
```
If you use [podman](https://github.com/containers/podman) instead of docker, you can install the RustFS with the below command
+1 -1
View File
@@ -138,7 +138,7 @@ chown -R 10001:10001 data logs
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
# 使用指定版本运行
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.1
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0
```
如果您通过绑定挂载启用 TLS 证书目录,也请用同样方式准备该目录:
+2 -2
View File
@@ -96,11 +96,11 @@ pub struct WorkloadAdmissionSnapshot {
pub class: WorkloadClass,
/// Current admission state.
pub state: AdmissionState,
/// Active work count when the owner exposes it.
/// Active work count or allocated/reserved permit units, as defined by the owner.
pub active: Option<usize>,
/// Queued work count when the owner exposes it.
pub queued: Option<usize>,
/// Admission limit when the owner exposes it.
/// Admission limit in the same units as `active`, when the owner exposes it.
pub limit: Option<usize>,
/// Optional state reason for disabled, throttled, saturated, or unknown states.
pub reason: Option<String>,
+11 -9
View File
@@ -169,15 +169,17 @@ Scanner cycle budget controls:
## Foreground write admission environment variables
Large direct `PutObject` requests and multipart `UploadPart` requests share one
per-process permit pool that bounds how many bodies are ingested and written
concurrently. Small direct PUTs stay on the legacy path.
per-process permit pool that bounds concurrent body ingest and storage writes.
Small direct PUTs stay on the legacy path.
- `RUSTFS_PUT_LARGE_FOREGROUND_ADMISSION_ENABLE`
- enables the default-on pool; `false` keeps only the soft request counter.
- default is `true`.
- `RUSTFS_PUT_LARGE_FOREGROUND_ADMISSION_LIMIT`
- permits in the pool; `0` derives half of `RUSTFS_OBJECT_MAX_CONCURRENT_DISK_READS`, clamped to `32`.
- default is `0` (32 permits at stock settings).
- a positive value is an exact request-count limit shared by all gated writes.
- default is `0`: derive half of `RUSTFS_OBJECT_MAX_CONCURRENT_DISK_READS`, clamped to `32`, as large-write slots. Each slot has eight units. A gated direct PUT or unknown-size multipart part uses eight units; known-size parts use one unit per 8 MiB, rounded up, with a minimum of one and maximum of eight.
- stock settings therefore share 256 units across at most 32 large/unknown writes or 256 parts of up to 8 MiB. Mixed sizes consume the same budget. Admitted known parts of up to 64 MiB total at most 2 GiB of declared payload; larger streaming writes and queued bodies are separate. This is not a bound on total process or kernel memory.
- admission snapshots report allocated or reserved units in `active` and the unit budget in `limit` under automatic sizing. Explicit and strict limits still report request-count permits.
- `RUSTFS_PUT_LARGE_FOREGROUND_ADMISSION_MIN_SIZE_BYTES`
- smallest direct `PutObject` that takes a permit; unknown-size requests always do.
- default is `33554432` (32 MiB).
@@ -193,8 +195,8 @@ concurrently. Small direct PUTs stay on the legacy path.
- RustFS does not read the request body while a part is queued, so the client's socket write stalls for the whole wait and whatever timeout the client or an intermediary has configured competes with this value. Keep it with margin below the shortest such timeout in use (botocore applies its 60 s `connect_timeout` to the body write; the AWS SDK for Java v2 has a 30 s socket write timeout; reverse proxies add their own body timeouts); a wait that outlives the client timeout surfaces as a dropped connection instead of `SlowDown`.
- `RUSTFS_PUT_MULTIPART_FOREGROUND_ADMISSION_MAX_PENDING`
- maximum `UploadPart` requests waiting for a permit at once; parts beyond it return `SlowDown` without waiting.
- default is `0`, which derives 16 times the permit limit (512 at stock settings).
- each queued HTTP/1 part holds whatever unread body the client already pushed into the connection's kernel receive buffer (an HTTP/2 part holds up to its flow-control window in process memory), so this depth also bounds that memory. RustFS leaves the receive buffer to kernel autotuning (see `RUSTFS_HTTP_SOCKET_RECV_BUFFER_BYTES` below), which keeps an unread connection at the kernel's initial size (128 KiB on current Linux).
- default is `0`, which derives 16 times the large-write slot count, or the explicit request-count limit (512 at stock settings). Automatic subdivision does not enlarge this queue.
- each queued HTTP/1 part holds whatever unread body the client already pushed into the connection's kernel receive buffer (an HTTP/2 part holds up to its flow-control window in process memory). Autotuned receive buffers can remain large on reused connections; queue depth does not imply a fixed per-connection memory cost.
- `RUSTFS_PUT_FOREGROUND_ADMISSION_ENABLE`, `RUSTFS_PUT_FOREGROUND_ADMISSION_LIMIT`, `RUSTFS_PUT_FOREGROUND_ADMISSION_WAIT_TIMEOUT_MS`
- experimental strict gate that applies to every foreground write regardless of size and replaces the pool above when enabled.
- default is disabled; enabling it with limit `0` disables foreground write admission entirely.
@@ -203,9 +205,9 @@ concurrently. Small direct PUTs stay on the legacy path.
- `RUSTFS_HTTP_SOCKET_RECV_BUFFER_BYTES`
- fixed `SO_RCVBUF` for the API listener, inherited by every accepted socket; `0` leaves the receive buffer to kernel autotuning.
- default is `0`. Earlier releases hard-coded 4 MiB, which Linux doubles to 8 MiB and which disables autotuning, so every connection whose body was not being read yet (a multipart part queued for a foreground write permit) could accumulate up to 8 MiB of unread body in kernel memory; at SDK-default multipart concurrency that was enough to push a node into TCP memory pressure.
- with autotuning the per-connection receive ceiling is the kernel's (`net.ipv4.tcp_rmem` max, 6 MiB on stock Linux) instead of the former fixed 8 MiB, so a single very high-bandwidth-delay connection may see a somewhat lower ceiling; raise `net.ipv4.tcp_rmem` first, and set this variable only on kernels without receive-buffer autotuning (illumos/Solaris) or where the sysctl cannot be changed.
- the send buffer stays fixed at 4 MiB because the stock Linux send autotuning ceiling (`net.ipv4.tcp_wmem` max, 4 MiB) is lower than a GB-level response stream needs.
- default is `0`. A fixed buffer disables receive autotuning. Linux doubles the requested value for socket-memory accounting, subject to kernel limits; that capacity is not a measurement of queued payload or allocated memory.
- with autotuning the receive ceiling is controlled by the kernel (`net.ipv4.tcp_rmem` on Linux); its defaults vary with kernel version and memory. A connection can retain a buffer enlarged by previous requests while its next request is queued. Tune the kernel ceiling first, and set this variable only on kernels without receive-buffer autotuning (illumos/Solaris) or where the sysctl cannot be changed.
- the send buffer remains fixed at 4 MiB; this setting only controls the receive side.
## Remote tier timeout environment variables
+7 -4
View File
@@ -364,11 +364,14 @@ const _: () = assert!(!DEFAULT_PUT_FOREGROUND_ADMISSION_ENABLE);
pub const ENV_PUT_LARGE_FOREGROUND_ADMISSION_ENABLE: &str = "RUSTFS_PUT_LARGE_FOREGROUND_ADMISSION_ENABLE";
pub const DEFAULT_PUT_LARGE_FOREGROUND_ADMISSION_ENABLE: bool = true;
/// Maximum automatic foreground write requests admitted concurrently per process.
/// Explicit maximum foreground write requests admitted concurrently per process.
///
/// `0` derives a conservative default from the local disk-read scheduler cap,
/// currently clamped to protect the commit path without making ordinary high
/// throughput uploads single-file.
/// `0` derives up to 32 large-write slots from the local disk-read scheduler.
/// Each automatic slot has eight units: a gated direct PUT or unknown-size
/// part uses all eight, while a known-size part uses one unit per 8 MiB,
/// rounded up and capped at eight. Thus stock settings share one budget across
/// at most 32 large/unknown writes or 256 parts of up to 8 MiB. A positive
/// override retains request-count semantics for every gated write.
pub const ENV_PUT_LARGE_FOREGROUND_ADMISSION_LIMIT: &str = "RUSTFS_PUT_LARGE_FOREGROUND_ADMISSION_LIMIT";
pub const DEFAULT_PUT_LARGE_FOREGROUND_ADMISSION_LIMIT: usize = 0;
+4 -7
View File
@@ -164,13 +164,10 @@ pub const DEFAULT_HTTP1_MAX_BUF_SIZE: usize = 64 * 1024; // 64 KB
/// autotuning.
///
/// A fixed `SO_RCVBUF` is inherited by every accepted socket and disables
/// receive-buffer autotuning, so a connection whose request body is not being
/// read yet (a multipart part queued for a foreground write permit) lets up to
/// the fixed size of unread body accumulate in kernel memory — Linux doubles
/// the requested value, so the former hard-coded 4 MiB held up to 8 MiB per
/// queued connection (issue #7385). Autotuning keeps an unread connection at
/// the kernel's initial size and grows only connections that are being
/// drained. Set this only on kernels without receive-buffer autotuning
/// receive-buffer autotuning. Queued multipart bodies can accumulate in kernel
/// memory with either policy: a reused autotuned connection may already have
/// a large buffer. Socket capacity is not the same as queued payload or actual
/// memory allocation. Set this only on kernels without receive-buffer autotuning
/// (illumos/Solaris) or on very high-bandwidth-delay links where the kernel's
/// autotuning ceiling (`net.ipv4.tcp_rmem` on Linux) is too low and cannot be
/// raised.
@@ -298,7 +298,11 @@ mod tests {
assert_eq!(empty_head_resp.content_disposition(), None);
assert_eq!(empty_head_resp.content_encoding(), None);
assert_eq!(empty_head_resp.content_language(), None);
assert_eq!(empty_head_resp.content_type(), None);
assert_eq!(
empty_head_resp.content_type(),
Some("binary/octet-stream"),
"REPLACE without an explicit content type must default to binary/octet-stream, matching S3"
);
assert_eq!(empty_head_resp.expires_string(), None);
assert_eq!(empty_head_resp.website_redirect_location(), None);
@@ -394,7 +398,15 @@ mod tests {
assert_eq!(head.content_disposition(), (field == "content-disposition").then_some("attachment"));
assert_eq!(head.content_encoding(), (field == "content-encoding").then_some("gzip"));
assert_eq!(head.content_language(), (field == "content-language").then_some("de"));
assert_eq!(head.content_type(), (field == "content-type").then_some("text/field"));
assert_eq!(
head.content_type(),
Some(if field == "content-type" {
"text/field"
} else {
"binary/octet-stream"
}),
"REPLACE without an explicit content type must default to binary/octet-stream, matching S3"
);
assert_eq!(
head.expires_string(),
(field == "expires").then_some(replacement_expires_http_date.as_str())
@@ -420,7 +432,11 @@ mod tests {
.send()
.await
.expect("HEAD failed for metadata collision case");
assert_eq!(collision_head.content_type(), None);
assert_eq!(
collision_head.content_type(),
Some("binary/octet-stream"),
"user-metadata under the reserved 'content-type' key must not satisfy the system content-type default"
);
assert_eq!(collision_head.content_encoding(), None);
assert_eq!(
collision_head.metadata().and_then(|metadata| metadata.get("content-type")),
+9 -3
View File
@@ -223,11 +223,13 @@ aws-smithy-http-client = { workspace = true, default-features = false, features
# Observability and Metrics
metrics = { workspace = true }
# Runtime-probed io_uring read backend (backlog#1104). Linux-only, from
# crates.io. The guard scripts/check_no_tokio_io_uring.sh allows an explicit
# Runtime-probed io_uring read backend (backlog#1104). Linux-only.
# The guard scripts/check_no_tokio_io_uring.sh allows an explicit
# io-uring integration; only the tokio "io-uring" runtime feature is banned.
[target.'cfg(target_os = "linux")'.dependencies]
rustfs-uring = "0.2.2"
# SharedReadBudget and the bounded read APIs are available in the 0.3.0
# registry release; keep the application on the published contract.
rustfs-uring = "0.3.0"
[target.'cfg(windows)'.dependencies]
winapi-util.workspace = true
@@ -282,5 +284,9 @@ harness = false
name = "single_block_non_inline_benchmark"
harness = false
[[bench]]
name = "multipart_read_parts_benchmark"
harness = false
[lib]
doctest = false
@@ -0,0 +1,92 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use criterion::{BenchmarkId, Criterion, Throughput, criterion_group, criterion_main};
use rustfs_filemeta::ObjectPartInfo;
use std::hint::black_box;
use std::time::Duration;
mod storage_api;
use storage_api::multipart::{DiskAPI, DiskOption, Endpoint, new_disk};
fn bench_multipart_read_parts(c: &mut Criterion) {
let runtime = tokio::runtime::Builder::new_multi_thread()
.worker_threads(4)
.enable_all()
.build()
.expect("benchmark runtime");
let root = tempfile::tempdir().expect("benchmark disk");
let mut endpoint = Endpoint::try_from(root.path().to_str().expect("UTF-8 path")).expect("endpoint");
endpoint.set_pool_index(0);
endpoint.set_set_index(0);
endpoint.set_disk_index(0);
let disk = runtime
.block_on(new_disk(
&endpoint,
&DiskOption {
cleanup: false,
health_check: false,
},
))
.expect("local disk");
let bucket = "multipart-bench";
runtime.block_on(disk.make_volume(bucket)).expect("benchmark volume");
let upload = root.path().join(bucket).join("upload");
std::fs::create_dir_all(&upload).expect("upload directory");
let mut paths = Vec::with_capacity(1024);
for number in 1..=1024 {
let part = ObjectPartInfo {
number,
etag: format!("{number:032x}"),
size: 1024,
actual_size: 1024,
..Default::default()
};
std::fs::write(upload.join(format!("part.{number}")), b"data").expect("part data");
std::fs::write(upload.join(format!("part.{number}.meta")), part.marshal_msg().expect("metadata")).expect("part metadata");
paths.push(format!("upload/part.{number}.meta"));
}
// Measures the real DiskAPI path against warm local files. Fixture creation
// and correctness checks stay outside the timed region; this is not a cold-disk benchmark.
let mut group = c.benchmark_group("multipart_read_parts");
group.sample_size(20);
group.warm_up_time(Duration::from_secs(1));
group.measurement_time(Duration::from_secs(2));
for count in [1, 32, 256, 1024] {
let paths = &paths[..count];
let parts = runtime.block_on(disk.read_parts(bucket, paths)).expect("read parts");
assert_eq!(parts.len(), count);
assert!(
parts
.iter()
.enumerate()
.all(|(i, part)| part.number == i + 1 && part.error.is_none())
);
group.throughput(Throughput::Elements(u64::try_from(count).expect("part count")));
group.bench_with_input(BenchmarkId::from_parameter(count), &paths, |b, paths| {
b.iter(|| {
black_box(
runtime
.block_on(disk.read_parts(bucket, black_box(paths)))
.expect("read parts"),
)
});
});
}
group.finish();
}
criterion_group!(benches, bench_multipart_read_parts);
criterion_main!(benches);
@@ -29,3 +29,7 @@ pub(crate) mod erasure {
pub(crate) mod single_block_non_inline {
pub(crate) use super::{BitrotWriterWrapper, CustomWriter, Erasure};
}
pub(crate) mod multipart {
pub(crate) use rustfs_ecstore::api::disk::{DiskAPI, DiskOption, Endpoint, new_disk};
}
+66 -6
View File
@@ -18,9 +18,10 @@ use super::versioning::VersioningApi;
use super::{quota::BucketQuota, target::BucketTargets};
use crate::bucket::replication::invalid_replication_config_status_field;
use crate::bucket::utils::deserialize;
use crate::config::com::{read_config, read_config_preserve_empty, save_config};
use crate::config::com::{read_config, read_config_preserve_empty, save_config, save_config_with_opts};
use crate::disk::BUCKET_META_PREFIX;
use crate::error::{Error, Result};
use crate::object_api::WriteCompletion;
use crate::runtime::sources as runtime_sources;
use crate::store::ECStore;
use byteorder::{BigEndian, ByteOrder, LittleEndian};
@@ -437,6 +438,7 @@ pub struct BucketMetadata {
pub lock_enabled: bool, // While marked as unused, it may need to be retained
pub bucket_incarnation_id: Uuid,
pub(crate) bucket_incarnation_sidecar: bool,
pub(crate) bucket_creation_committed: bool,
pub policy_config_json: Vec<u8>,
pub notification_config_xml: Vec<u8>,
pub lifecycle_config_xml: Vec<u8>,
@@ -511,6 +513,7 @@ impl Default for BucketMetadata {
lock_enabled: Default::default(),
bucket_incarnation_id: Uuid::nil(),
bucket_incarnation_sidecar: false,
bucket_creation_committed: false,
policy_config_json: Default::default(),
notification_config_xml: Default::default(),
lifecycle_config_xml: Default::default(),
@@ -594,6 +597,22 @@ impl BucketMetadata {
metadata
}
/// Whether this generation still needs its creation-commit proof.
///
/// The proof is only ever written while the caller holds the bucket
/// metadata transaction fence; see
/// `BucketMetadataSys::migrate_bucket_creation_commit`.
///
/// Metadata without an authoritative incarnation sidecar is owned by the
/// legacy migration path, which mints the sidecar first; it must not be
/// rewritten by this migration.
pub(crate) fn needs_bucket_creation_commit(&self) -> bool {
self.lock_enabled
&& !self.bucket_creation_committed
&& self.bucket_incarnation_sidecar
&& !self.bucket_incarnation_id.is_nil()
}
pub fn save_file_path(&self) -> String {
format!("{}/{}/{}", BUCKET_META_PREFIX, self.name.as_str(), BUCKET_METADATA_FILE)
}
@@ -733,6 +752,7 @@ impl BucketMetadata {
"TableBucketConfigUpdatedAt" => self.table_bucket_config_updated_at = read_msgp_time_value(rd)?,
"DurabilityConfigUpdatedAt" => self.durability_config_updated_at = read_msgp_time_value(rd)?,
"OnDemandMigrationConfigUpdatedAt" => self.on_demand_migration_config_updated_at = read_msgp_time_value(rd)?,
"BucketCreationCommitted" => self.bucket_creation_committed = read_msgp_bool(rd)?,
other => {
tracing::debug!(field = %other, "BucketMetadata decode_from: skipping unknown field");
skip_msgp_value(rd)?;
@@ -745,8 +765,8 @@ impl BucketMetadata {
/// Encode to msgp bytes. Field order follows MinIO BucketMetadata for compatibility.
pub fn encode_to<W: Write>(&self, wr: &mut W) -> Result<()> {
// Map size: MinIO fields (25) + RustFS extensions (21)
let map_len: u32 = 46;
// Map size: MinIO fields (25) + RustFS extensions (22)
let map_len: u32 = 47;
rmp::encode::write_map_len(wr, map_len)?;
// MinIO field order (same as Go struct)
@@ -827,6 +847,8 @@ impl BucketMetadata {
write_msgp_time(wr, self.durability_config_updated_at)?;
rmp::encode::write_str(wr, "OnDemandMigrationConfigUpdatedAt")?;
write_msgp_time(wr, self.on_demand_migration_config_updated_at)?;
rmp::encode::write_str(wr, "BucketCreationCommitted")?;
rmp::encode::write_bool(wr, self.bucket_creation_committed)?;
Ok(())
}
@@ -982,6 +1004,11 @@ impl BucketMetadata {
self.object_lock_config = None;
if !data.is_empty() {
self.lock_enabled = true;
// Enabling Object Lock is only reachable for a bucket that
// already exists: the caller validated physical presence and
// holds the incarnation fence, so this generation is committed
// and must never be mistaken for a pre-physical creation intent.
self.bucket_creation_committed = true;
}
self.object_lock_config_xml = data;
self.object_lock_config_updated_at = updated;
@@ -1086,6 +1113,21 @@ impl BucketMetadata {
/// server's bucket metadata lands in that server's `.rustfs.sys`, not the
/// ambient (first) one. [`BucketMetadata::save`] keeps the ambient default.
pub async fn save_with_store(&mut self, store: std::sync::Arc<crate::store::ECStore>) -> Result<()> {
self.save_with_store_completion(store, WriteCompletion::Quorum).await
}
/// Persist a committed creation record and drain the rename fan-out before
/// returning. A quorum ACK is insufficient here: later degraded reads may
/// have to reconstruct this record from any read quorum of surviving shards.
pub(crate) async fn save_with_store_committed(&mut self, store: std::sync::Arc<crate::store::ECStore>) -> Result<()> {
self.save_with_store_completion(store, WriteCompletion::TailDrained).await
}
async fn save_with_store_completion(
&mut self,
store: std::sync::Arc<crate::store::ECStore>,
write_completion: WriteCompletion,
) -> Result<()> {
self.parse_all_configs()?;
let mut buf: Vec<u8> = vec![0; 4];
@@ -1099,7 +1141,17 @@ impl BucketMetadata {
buf.extend_from_slice(&data);
save_config(store, self.save_file_path().as_str(), buf).await?;
save_config_with_opts(
store,
self.save_file_path().as_str(),
buf,
&crate::object_api::ObjectOptions {
max_parity: true,
write_completion,
..Default::default()
},
)
.await?;
Ok(())
}
@@ -1451,8 +1503,13 @@ pub(crate) async fn load_bucket_metadata_parse_with_presence(
}
};
let incarnation = load_bucket_incarnation(api, bucket).await?;
let incarnation = load_bucket_incarnation(api.clone(), bucket).await?;
if persisted {
if bm.name.is_empty() {
bm.name = bucket.to_string();
} else if bm.name != bucket {
return Err(Error::FileCorrupt);
}
if let Some(incarnation) = incarnation {
if !bm.bucket_incarnation_id.is_nil() && bm.bucket_incarnation_id != incarnation {
return Err(Error::other("bucket incarnation sidecar does not match bucket metadata"));
@@ -1622,6 +1679,7 @@ mod test {
BucketMetadata::check_header(&body).expect("recovered body is a valid .metadata.bin");
let mut bm = BucketMetadata::unmarshal(&body[4..]).expect("unmarshal recovered blob");
assert_eq!(bm.name, "interop");
assert!(!bm.bucket_creation_committed, "legacy metadata must default to uncommitted");
bm.parse_all_configs().expect("parse recovered configs");
assert!(bm.lifecycle_config.is_some());
}
@@ -1630,13 +1688,15 @@ mod test {
async fn marshal_msg() {
// write_time(OffsetDateTime::UNIX_EPOCH).unwrap();
let bm = BucketMetadata::new("dada");
let mut bm = BucketMetadata::new("dada");
bm.bucket_creation_committed = true;
let buf = bm.marshal_msg().unwrap();
let new = BucketMetadata::unmarshal(&buf).unwrap();
assert_eq!(bm.name, new.name);
assert!(new.bucket_creation_committed);
assert!(!bm.bucket_incarnation_id.is_nil());
assert_eq!(bm.bucket_incarnation_id, new.bucket_incarnation_id);
}
+131 -14
View File
@@ -527,11 +527,19 @@ pub(crate) async fn set_new_bucket_metadata_in(
lock.persist_new_and_set(bm).await
}
pub(crate) async fn cache_bucket_metadata_in(ctx: &crate::runtime::instance::InstanceContext, bm: BucketMetadata) -> Result<()> {
pub(crate) async fn set_new_bucket_metadata_intent_in(
ctx: &crate::runtime::instance::InstanceContext,
bm: BucketMetadata,
) -> Result<()> {
let sys = bucket_metadata_sys_of(ctx)?;
let lock = sys.read().await;
lock.set(bm.name.clone(), Arc::new(bm)).await;
Ok(())
lock.persist_new_bucket_metadata_intent(bm).await
}
pub(crate) async fn commit_bucket_metadata_in(ctx: &crate::runtime::instance::InstanceContext, bm: BucketMetadata) -> Result<()> {
let sys = bucket_metadata_sys_of(ctx)?;
let lock = sys.read().await;
lock.commit_bucket_metadata(bm).await
}
pub(crate) async fn remove_bucket_metadata_in(ctx: &crate::runtime::instance::InstanceContext, bucket: &str) -> Result<bool> {
@@ -2074,9 +2082,32 @@ impl BucketMetadataSys {
}
async fn persist_new_and_set(&self, mut bm: BucketMetadata) -> Result<()> {
bm.bucket_creation_committed = true;
bm.save_with_store_committed(self.object_store()).await?;
save_bucket_incarnation(self.object_store(), &bm.name, bm.bucket_incarnation_id).await?;
bm.bucket_incarnation_sidecar = true;
self.set(bm.name.clone(), Arc::new(bm)).await;
Ok(())
}
async fn persist_new_bucket_metadata_intent(&self, mut bm: BucketMetadata) -> Result<()> {
bm.bucket_creation_committed = false;
bm.save_with_store(self.object_store()).await?;
save_bucket_incarnation(self.object_store(), &bm.name, bm.bucket_incarnation_id).await?;
bm.bucket_incarnation_sidecar = true;
// A pending creation intent must not enter the authoritative cache. The
// caller publishes it only after physical creation and commit-marker write.
Ok(())
}
async fn commit_bucket_metadata(&self, mut bm: BucketMetadata) -> Result<()> {
bm.parse_all_configs()?;
bm.bucket_incarnation_sidecar = true;
// The caller holds the full bucket creation fence (lifecycle, metadata
// transaction, and namespace write) and physical creation already
// returned, so this generation is committed.
bm.bucket_creation_committed = true;
bm.save_with_store_committed(self.object_store()).await?;
self.set(bm.name.clone(), Arc::new(bm)).await;
Ok(())
}
@@ -2192,18 +2223,25 @@ impl BucketMetadataSys {
)
.await?;
let bm = Arc::new(bm);
if persisted {
// Object Lock metadata is persisted before physical bucket creation as a
// creation intent. The commit marker distinguishes a committed bucket
// from an intent left behind by an interrupted create.
let require_write_quorum = bm.needs_bucket_creation_commit();
await_bucket_namespace_operation(
Some(&guard),
bucket,
"lazy bucket metadata existence check",
Box::pin(async {
self.object_store()
.get_bucket_info_from_sets(bucket, &crate::storage_api_contracts::bucket::BucketOptions::default())
.await
.map(|_| ())
let store = self.object_store();
let options = crate::storage_api_contracts::bucket::BucketOptions::default();
let info = if require_write_quorum {
store.get_bucket_info_from_sets(bucket, &options).await
} else {
store.get_bucket_info_from_sets_at_read_quorum(bucket, &options).await
};
info.map(|_| ())
}),
)
.await?;
@@ -2212,6 +2250,7 @@ impl BucketMetadataSys {
"bucket namespace lock was lost before lazy bucket metadata publish: {bucket}"
)));
}
let bm = Arc::new(bm);
let _publish_guard = self
.lock_metadata_publish(bucket, &guard, "lazy bucket metadata publish")
.await?;
@@ -2226,6 +2265,7 @@ impl BucketMetadataSys {
sync_bucket_target_sys(bucket, &bm).await;
sync_bucket_durability(bucket, &bm);
sync_on_demand_migration(bucket, &bm);
return Ok((bm, true));
} else {
let exists = self
.bucket_exists(bucket, &guard, "lazy bucket metadata existence check")
@@ -2245,7 +2285,7 @@ impl BucketMetadataSys {
}
}
Ok((bm, true))
Ok((Arc::new(bm), true))
}
}
@@ -2430,7 +2470,13 @@ impl BucketMetadataSys {
}
async fn get_metadata_authority(&self, bucket: &str) -> Result<BucketMetadataAuthority> {
if let Some(bm) = self.metadata_map.read().await.get(bucket).cloned() {
// Bind before matching: holding the map guard across the migration would
// deadlock on the publish write.
let cached = self.metadata_map.read().await.get(bucket).cloned();
if let Some(bm) = cached {
if bm.needs_bucket_creation_commit() {
return self.migrate_bucket_creation_commit(bucket).await;
}
return Ok(BucketMetadataAuthority::Authoritative(bm));
}
if self.fabricated_metadata.read().await.contains(bucket) {
@@ -2442,8 +2488,13 @@ impl BucketMetadataSys {
self.get_config(bucket).await?;
if let Some(bm) = self.metadata_map.read().await.get(bucket).cloned() {
Ok(BucketMetadataAuthority::Authoritative(bm))
let loaded = self.metadata_map.read().await.get(bucket).cloned();
if let Some(bm) = loaded {
if bm.needs_bucket_creation_commit() {
self.migrate_bucket_creation_commit(bucket).await
} else {
Ok(BucketMetadataAuthority::Authoritative(bm))
}
} else if self.fabricated_metadata.read().await.contains(bucket) {
Ok(BucketMetadataAuthority::Fabricated)
} else if self.missing_buckets.get(bucket).await.is_some() {
@@ -2453,6 +2504,70 @@ impl BucketMetadataSys {
}
}
/// Persist the creation-commit proof for an Object Lock bucket created
/// before this field existed.
///
/// Lock order matches the established config-write order: metadata
/// transaction (write), then bucket namespace (read). The authoritative
/// metadata is re-read under both fences and the physical namespace is
/// revalidated at write quorum, so a stale snapshot can neither revert an
/// acknowledged configuration update nor outlive a delete/recreate.
async fn migrate_bucket_creation_commit(&self, bucket: &str) -> Result<BucketMetadataAuthority> {
let transaction_lock = self
.object_store()
.new_ns_lock(RUSTFS_META_BUCKET, &bucket_metadata_transaction_lock_key(bucket))
.await?;
let transaction_guard = transaction_lock
.get_write_lock(crate::set_disk::get_lock_acquire_timeout())
.await?;
let namespace_lock = self.object_store().new_ns_lock(bucket, bucket).await?;
let namespace_guard = namespace_lock
.get_read_lock(crate::set_disk::get_lock_acquire_timeout())
.await?;
// Write quorum must still prove that the physical namespace committed.
await_bucket_namespace_operation(
Some(&namespace_guard),
bucket,
"bucket creation commitment existence check",
self.object_store()
.get_bucket_info_from_sets(bucket, &crate::storage_api_contracts::bucket::BucketOptions::default()),
)
.await?;
let (mut metadata, persisted) = await_bucket_namespace_operation(
Some(&namespace_guard),
bucket,
"bucket creation commitment metadata reload",
load_bucket_metadata_parse_with_presence(self.object_store(), bucket, true),
)
.await?;
if !persisted {
return Ok(BucketMetadataAuthority::Fabricated);
}
if metadata.needs_bucket_creation_commit() {
metadata.bucket_creation_committed = true;
metadata.save_with_store_committed(self.object_store()).await?;
}
if transaction_guard.is_lock_lost() || namespace_guard.is_lock_lost() {
// Stable message: formatted per-bucket detail would split quorum
// error buckets (backlog#1845).
return Err(Error::other("bucket creation commitment fence was lost before publish"));
}
let metadata = Arc::new(metadata);
let _publish_guard = self
.lock_metadata_publish(bucket, &namespace_guard, "bucket creation commitment publish")
.await?;
self.metadata_map
.write()
.await
.insert(bucket.to_string(), Arc::clone(&metadata));
self.fabricated_metadata.write().await.remove(bucket);
self.missing_buckets.invalidate(bucket).await;
Ok(BucketMetadataAuthority::Authoritative(metadata))
}
async fn migrate_legacy_metadata(&self, bucket: &str) -> Result<BucketMetadataAuthority> {
let transaction_lock = self
.object_store()
@@ -2564,7 +2679,6 @@ impl BucketMetadataSys {
.await?;
}
}
if namespace_guard.is_lock_lost() {
return Err(Error::other(format!(
"bucket namespace lock was lost before legacy metadata publish: {bucket}"
@@ -2626,6 +2740,9 @@ impl BucketMetadataSys {
load_bucket_metadata_parse_with_presence(self.object_store(), bucket, true),
)
.await?;
if persisted && metadata.lock_enabled && !metadata.bucket_creation_committed {
return Err(Error::ErasureWriteQuorum);
}
if persisted {
Ok(BucketMetadataAuthority::Authoritative(Arc::new(metadata)))
} else {
+11 -7
View File
@@ -1185,14 +1185,18 @@ mod decommission_lock_order_tests {
barrier.wait_until_paused().await;
let probe_store = Arc::clone(&other_store);
tokio::time::timeout(std::time::Duration::from_secs(1), async move {
probe_store
.save_current_pool_meta_for_test(&[0])
.await
.expect("pool metadata mutation probe should commit before UploadPart admission");
})
let pool_meta_lock = probe_store
.new_ns_lock(RUSTFS_META_BUCKET, POOL_META_NAME)
.await
.expect("create the public UploadPart capacity guard probe");
let pool_meta_guard = tokio::time::timeout(
std::time::Duration::from_secs(1),
pool_meta_lock.get_write_lock(std::time::Duration::from_secs(30)),
)
.await
.expect("public UploadPart must not hold a capacity read guard during staging");
.expect("public UploadPart must not hold a capacity read guard during staging")
.expect("public UploadPart capacity guard probe should acquire");
drop(pool_meta_guard);
barrier.release();
let part = tokio::time::timeout(std::time::Duration::from_secs(30), put)
+179
View File
@@ -0,0 +1,179 @@
// Copyright 2024 RustFS Team
// SPDX-License-Identifier: Apache-2.0
use super::{FdCache, FdCacheEntry, FdKey};
use std::fs::File;
use std::os::unix::fs::FileExt;
use std::sync::Arc;
fn key(volume: &str, path: &str, direct: bool) -> FdKey {
FdKey {
volume: volume.to_owned(),
path: path.to_owned(),
direct,
}
}
fn fixture() -> (tempfile::TempDir, Arc<File>) {
let directory = tempfile::tempdir().expect("create cache fixture directory");
let path = directory.path().join("part");
std::fs::write(&path, b"old").expect("write original cache fixture");
let file = Arc::new(File::open(path).expect("open original cache fixture"));
(directory, file)
}
// Both key variants intentionally use normal files. These tests exercise the
// real Moka cache's identity/invalidation contract, not native O_DIRECT I/O.
#[tokio::test]
async fn uring_fd_cache_exact_invalidation_removes_both_modes_and_preserves_other_paths() {
let (_directory, file) = fixture();
let cache = FdCache::new();
let paths = [("v", "a/b"), ("v", "a/b/part.1"), ("v", "a/bc"), ("other", "a/b")];
for (volume, path) in paths {
for direct in [false, true] {
cache.insert(key(volume, path, direct), Arc::clone(&file)).await;
}
}
assert_eq!(cache.entry_count().await, 8, "all mode/scope fixtures must exist before invalidation");
cache.invalidate_exact("v", "a/b").await;
for (volume, path) in paths {
for direct in [false, true] {
let hit = cache.get(&key(volume, path, direct)).await.is_some();
assert_eq!(
hit,
(volume, path) != ("v", "a/b"),
"wrong invalidation scope: {volume}/{path}, direct={direct}"
);
}
}
}
#[tokio::test]
async fn uring_fd_cache_exact_invalidation_bumps_one_generation_and_rejects_old_epoch_inserts() {
let (_directory, file) = fixture();
let cache = FdCache::new();
let entry = Arc::new(FdCacheEntry { file, len: 3 });
let old_generation = cache.generation();
cache.invalidate_exact("v", "part").await;
let current_generation = cache.generation();
assert_eq!(current_generation, old_generation + 1, "one logical invalidation must advance once");
for direct in [false, true] {
let key = key("v", "part", direct);
cache.insert_if_fresh(key.clone(), Arc::clone(&entry), old_generation).await;
assert!(cache.get(&key).await.is_none(), "old-epoch insert resurrected direct={direct}");
cache
.insert_if_fresh(key.clone(), Arc::clone(&entry), current_generation)
.await;
assert!(cache.get(&key).await.is_some(), "current epoch must allow refill, direct={direct}");
}
}
#[tokio::test]
async fn uring_fd_cache_exact_invalidation_accepts_missing_or_single_mode_entries() {
let (_directory, file) = fixture();
for present in [None, Some(false), Some(true)] {
let cache = FdCache::new();
if let Some(direct) = present {
cache.insert(key("v", "part", direct), Arc::clone(&file)).await;
assert!(cache.get(&key("v", "part", direct)).await.is_some());
}
cache.invalidate_exact("v", "part").await;
for direct in [false, true] {
assert!(cache.get(&key("v", "part", direct)).await.is_none(), "mode survived: {present:?}");
}
assert_eq!(cache.generation(), 1);
cache.invalidate_exact("v", "part").await;
assert_eq!(cache.generation(), 2, "repeated invalidation still fences concurrent miss opens");
}
}
#[tokio::test]
async fn uring_fd_cache_prefix_invalidation_covers_both_modes_without_crossing_components() {
let (_directory, file) = fixture();
let cache = FdCache::new();
let paths = [("v", "a/b"), ("v", "a/b/part.1"), ("v", "a/bc"), ("other", "a/b")];
for (volume, path) in paths {
for direct in [false, true] {
cache.insert(key(volume, path, direct), Arc::clone(&file)).await;
assert!(cache.get(&key(volume, path, direct)).await.is_some());
}
}
cache.invalidate_under("v", "a/b/");
for (volume, path) in paths {
for direct in [false, true] {
let hit = cache.get(&key(volume, path, direct)).await.is_some();
let removed = volume == "v" && matches!(path, "a/b" | "a/b/part.1");
assert_eq!(hit, !removed, "wrong prefix scope: {volume}/{path}, direct={direct}");
}
}
}
#[tokio::test]
async fn uring_fd_cache_volume_and_clear_invalidation_cover_both_modes() {
let (_directory, file) = fixture();
let cache = FdCache::new();
for volume in ["v", "other"] {
for direct in [false, true] {
cache.insert(key(volume, "part", direct), Arc::clone(&file)).await;
assert!(cache.get(&key(volume, "part", direct)).await.is_some());
}
}
cache.invalidate_volume("v");
for direct in [false, true] {
assert!(cache.get(&key("v", "part", direct)).await.is_none());
assert!(cache.get(&key("other", "part", direct)).await.is_some());
cache.insert(key("v", "part", direct), Arc::clone(&file)).await;
assert!(
cache.get(&key("v", "part", direct)).await.is_some(),
"new entries must survive the earlier volume predicate"
);
}
cache.clear();
for volume in ["v", "other"] {
for direct in [false, true] {
assert!(cache.get(&key(volume, "part", direct)).await.is_none());
}
}
}
#[tokio::test]
async fn uring_fd_cache_invalidation_keeps_borrowed_old_inode_alive_and_allows_replacement_refill() {
let (directory, file) = fixture();
let cache = FdCache::new();
let mut borrowed = Vec::new();
for direct in [false, true] {
let key = key("v", "part", direct);
cache.insert(key.clone(), Arc::clone(&file)).await;
borrowed.push(cache.get(&key).await.expect("borrow original inode before replacement"));
}
drop(file);
let replacement = directory.path().join("replacement");
std::fs::write(&replacement, b"replacement").expect("write replacement inode");
std::fs::rename(&replacement, directory.path().join("part")).expect("replace cached path with new inode");
cache.invalidate_exact("v", "part").await;
for (entry, direct) in borrowed.iter().zip([false, true]) {
assert!(cache.get(&key("v", "part", direct)).await.is_none());
let mut bytes = [0; 3];
entry
.file
.read_exact_at(&mut bytes, 0)
.expect("borrowed descriptor remains readable after invalidation");
assert_eq!(&bytes, b"old");
assert_eq!(entry.len, 3, "borrowed size snapshot belongs to the old inode");
}
let replacement = Arc::new(FdCacheEntry {
file: Arc::new(File::open(directory.path().join("part")).expect("open replacement inode")),
len: 11,
});
for direct in [false, true] {
let key = key("v", "part", direct);
cache
.insert_if_fresh(key.clone(), Arc::clone(&replacement), cache.generation())
.await;
let fresh = cache.get(&key).await.expect("replacement should populate the current epoch");
let mut bytes = [0; 11];
fresh.file.read_exact_at(&mut bytes, 0).expect("read replacement inode");
assert_eq!(&bytes, b"replacement");
assert_eq!(fresh.len, 11);
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,340 @@
// Copyright 2024 RustFS Team
// SPDX-License-Identifier: Apache-2.0
use std::ffi::OsStr;
use std::sync::Arc;
use tokio::sync::{OwnedSemaphorePermit, Semaphore, TryAcquireError};
/// Optional process budget for the dedicated driver threads (one per shard).
/// It does not account for io-wq workers, blocking tasks, or buffer allocation.
pub(super) enum DriverThreadBudget {
Unlimited,
Limited(Arc<Semaphore>),
}
#[derive(Debug)]
pub(super) struct InvalidBudget;
impl std::fmt::Display for InvalidBudget {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str("expected an unsigned decimal integer within Semaphore::MAX_PERMITS")
}
}
impl std::error::Error for InvalidBudget {}
impl DriverThreadBudget {
/// Parse a supplied value so tests do not mutate process-wide environment.
/// Missing or zero is the legacy unlimited policy; malformed input fails.
pub(super) fn from_env_value(value: Option<&OsStr>) -> Result<Self, InvalidBudget> {
let Some(value) = value else { return Ok(Self::Unlimited) };
let text = value.to_str().ok_or(InvalidBudget)?;
if text.is_empty() || !text.bytes().all(|byte| byte.is_ascii_digit()) {
return Err(InvalidBudget);
}
let limit = text.parse::<usize>().map_err(|_| InvalidBudget)?;
if limit > Semaphore::MAX_PERMITS {
return Err(InvalidBudget);
}
Ok(if limit == 0 {
Self::Unlimited
} else {
Self::Limited(Arc::new(Semaphore::new(limit)))
})
}
/// Reserve the full configured shard count without waiting for a live
/// driver to retire. Insufficient capacity selects the caller's std backend.
pub(super) fn try_reserve(&self, shards: usize) -> Result<Option<OwnedSemaphorePermit>, TryAcquireError> {
// Production shards are clamped to 1..=16 before this call. Retain a
// checked conversion at the semaphore's u32 API boundary.
let count = u32::try_from(shards).map_err(|_| TryAcquireError::NoPermits)?;
if count == 0 {
return Err(TryAcquireError::NoPermits);
}
match self {
Self::Unlimited => Ok(None),
Self::Limited(permits) => Arc::clone(permits).try_acquire_many_owned(count).map(Some),
}
}
}
/// Driver and its lifetime reservation must share the same Arc, including
/// transient references held by the stats exporter.
pub(super) struct BudgetedDriver<T> {
// Field drop order is intentional: joining all driver threads finishes
// before the reservation is returned. Do not move the permit above driver.
driver: T,
_threads: Option<OwnedSemaphorePermit>,
}
impl<T> BudgetedDriver<T> {
pub(super) fn new(driver: T, threads: Option<OwnedSemaphorePermit>) -> Self {
Self {
driver,
_threads: threads,
}
}
}
impl<T> std::ops::Deref for BudgetedDriver<T> {
type Target = T;
fn deref(&self) -> &T {
&self.driver
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicBool, Ordering};
use std::task::{Context, Poll, Waker};
use tokio::sync::oneshot;
fn budget(limit: usize) -> DriverThreadBudget {
DriverThreadBudget::from_env_value(Some(OsStr::new(&limit.to_string()))).expect("valid budget")
}
fn poll_once<T>(future: std::pin::Pin<&mut impl Future<Output = T>>) -> Poll<T> {
future.poll(&mut Context::from_waker(Waker::noop()))
}
#[test]
fn configuration_accepts_only_supported_decimal_limits() {
for value in [None, Some(OsStr::new("0")), Some(OsStr::new("000"))] {
assert!(matches!(DriverThreadBudget::from_env_value(value), Ok(DriverThreadBudget::Unlimited)));
}
for limit in [1, 16, Semaphore::MAX_PERMITS] {
let DriverThreadBudget::Limited(permits) = budget(limit) else {
panic!("nonzero budget must be limited")
};
assert_eq!(permits.available_permits(), limit);
}
for value in ["", " ", " 4", "4 ", "+4", "-1", "bad", "1.5", "184467440737095516160"] {
assert!(
DriverThreadBudget::from_env_value(Some(OsStr::new(value))).is_err(),
"invalid config accepted: {value}"
);
}
for limit in [Semaphore::MAX_PERMITS + 1, usize::MAX] {
assert!(DriverThreadBudget::from_env_value(Some(OsStr::new(&limit.to_string()))).is_err());
}
}
#[cfg(unix)]
#[test]
fn non_utf8_configuration_fails_closed() {
use std::os::unix::ffi::OsStrExt;
assert!(DriverThreadBudget::from_env_value(Some(OsStr::from_bytes(&[0xff]))).is_err());
}
#[test]
fn weighted_reservations_reject_excess_without_partial_acquisition() {
let budget = budget(5);
let first = budget.try_reserve(3).expect("first driver fits");
assert!(matches!(budget.try_reserve(3), Err(TryAcquireError::NoPermits)));
let second = budget.try_reserve(2).expect("failed reserve must not consume capacity");
assert!(matches!(budget.try_reserve(1), Err(TryAcquireError::NoPermits)));
drop(first);
let replacement = budget.try_reserve(3).expect("retirement returns exact weight");
drop(second);
drop(replacement);
assert!(budget.try_reserve(5).is_ok());
}
#[test]
fn unlimited_policy_preserves_legacy_and_invalid_shard_counts_do_not_panic() {
let budget = DriverThreadBudget::Unlimited;
assert!(budget.try_reserve(16).expect("legacy admission").is_none());
assert!(budget.try_reserve(0).is_err());
if usize::BITS > u32::BITS {
assert!(budget.try_reserve(usize::MAX).is_err());
}
}
#[test]
fn stats_arc_keeps_budget_until_final_driver_drop() {
let budget = budget(2);
let driver = Arc::new(BudgetedDriver::new((), budget.try_reserve(2).expect("reserve driver")));
let stats = Arc::downgrade(&driver);
let exporter_reference = stats.upgrade().expect("stats borrows a live driver");
drop(driver);
assert!(budget.try_reserve(1).is_err(), "backend drop must not bypass the exporter Arc");
drop(exporter_reference);
assert!(stats.upgrade().is_none());
assert!(budget.try_reserve(2).is_ok(), "a Weak alone must not hold the reservation");
}
#[tokio::test]
async fn blocking_driver_drop_retains_budget_until_join_finishes() {
struct JoiningDriver {
started: Option<oneshot::Sender<()>>,
finish: std::sync::mpsc::Receiver<()>,
}
impl Drop for JoiningDriver {
fn drop(&mut self) {
self.started
.take()
.expect("single driver drop")
.send(())
.expect("observe drop entry");
self.finish
.recv_timeout(std::time::Duration::from_secs(10))
.expect("release simulated join");
}
}
let budget = budget(2);
let (started_tx, started_rx) = oneshot::channel();
let (finish_tx, finish_rx) = std::sync::mpsc::channel();
let driver = BudgetedDriver::new(
JoiningDriver {
started: Some(started_tx),
finish: finish_rx,
},
budget.try_reserve(2).expect("reserve driver"),
);
let cleanup = tokio::task::spawn_blocking(move || drop(driver));
started_rx.await.expect("driver drop started");
let unavailable_during_join = budget.try_reserve(1).is_err();
finish_tx.send(()).expect("finish driver join");
cleanup.await.expect("cleanup completes");
assert!(unavailable_during_join);
assert!(budget.try_reserve(2).is_ok());
}
#[tokio::test]
async fn failed_and_panicking_probes_return_their_thread_reservation() {
let budget = budget(2);
let threads = budget.try_reserve(2).expect("reserve failed probe");
let failure = super::super::uring_probe::run(move || {
let _threads = threads;
Err::<(), _>(std::io::Error::from_raw_os_error(5))
})
.await
.expect("probe work returns normally");
assert_eq!(failure.expect_err("probe error preserved").raw_os_error(), Some(5));
let threads = budget.try_reserve(2).expect("failed probe returned reservation");
assert!(
super::super::uring_probe::run(move || {
let _threads = threads;
panic!("injected probe panic");
})
.await
.is_err()
);
assert!(budget.try_reserve(2).is_ok());
}
#[tokio::test]
async fn started_probe_cancellation_retains_thread_reservation_until_completion() {
let budget = Arc::new(budget(2));
let threads = budget.try_reserve(2).expect("reserve initializing driver");
let (entered_tx, entered_rx) = oneshot::channel();
let (release_tx, release_rx) = std::sync::mpsc::channel();
let (dropped_tx, dropped_rx) = oneshot::channel();
struct Driver(Option<oneshot::Sender<()>>);
impl Drop for Driver {
fn drop(&mut self) {
self.0.take().expect("single drop").send(()).expect("observe cleanup");
}
}
let mut probe = Box::pin(super::super::uring_probe::run(move || {
entered_tx.send(()).expect("probe entered");
release_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("finish probe");
BudgetedDriver::new(Driver(Some(dropped_tx)), threads)
}));
assert!(poll_once(probe.as_mut()).is_pending());
entered_rx.await.expect("probe started");
drop(probe);
let still_reserved = budget.try_reserve(1).is_err();
release_tx.send(()).expect("complete detached probe");
dropped_rx.await.expect("abandoned driver is cleaned up");
// The Drop notification precedes the permit field's destructor. Wait on
// the test semaphore only to observe final cleanup, never in production.
let DriverThreadBudget::Limited(permits) = budget.as_ref() else { panic!("limited budget") };
let returned = permits
.clone()
.acquire_many_owned(2)
.await
.expect("cleanup returns all thread slots");
assert!(still_reserved);
drop(returned);
}
#[tokio::test]
async fn canceling_initialization_waiter_releases_reserved_driver_slots() {
let budget = budget(2);
let threads = budget.try_reserve(2).expect("reserve pending initialization");
let initialization = Arc::new(Semaphore::new(0));
let started = Arc::new(AtomicBool::new(false));
let mark_started = started.clone();
let mut probe = Box::pin(super::super::uring_probe::run_with_permits(initialization, move || {
mark_started.store(true, Ordering::SeqCst);
BudgetedDriver::new((), threads)
}));
assert!(poll_once(probe.as_mut()).is_pending());
assert!(budget.try_reserve(1).is_err());
drop(probe);
assert!(!started.load(Ordering::SeqCst));
assert!(budget.try_reserve(2).is_ok());
}
#[tokio::test]
async fn completed_unconsumed_driver_keeps_slots_through_offloaded_cleanup() {
struct WakeFinished(Arc<tokio::sync::Notify>);
impl std::task::Wake for WakeFinished {
fn wake(self: Arc<Self>) {
self.0.notify_one();
}
}
struct Driver {
budget: Arc<DriverThreadBudget>,
dropped: Option<oneshot::Sender<(bool, std::thread::ThreadId)>>,
}
impl Drop for Driver {
fn drop(&mut self) {
let reservation_retained = self.budget.try_reserve(1).is_err();
self.dropped
.take()
.expect("single driver drop")
.send((reservation_retained, std::thread::current().id()))
.expect("observe cleanup");
}
}
let budget = Arc::new(budget(2));
let threads = budget.try_reserve(2).expect("reserve driver");
let (drop_tx, drop_rx) = oneshot::channel();
let driver = Driver {
budget: budget.clone(),
dropped: Some(drop_tx),
};
let (finish_tx, finish_rx) = std::sync::mpsc::channel();
let initialization = Arc::new(Semaphore::new(1));
let mut probe = Box::pin(super::super::uring_probe::run_with_permits(initialization, move || {
finish_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("allow result publication");
BudgetedDriver::new(driver, threads)
}));
let finished = Arc::new(tokio::sync::Notify::new());
let waker = Waker::from(Arc::new(WakeFinished(finished.clone())));
assert!(probe.as_mut().poll(&mut Context::from_waker(&waker)).is_pending());
finish_tx.send(()).expect("complete driver initialization");
finished.notified().await;
assert!(budget.try_reserve(1).is_err(), "ready but unconsumed driver remains charged");
drop(probe);
let (retained_during_drop, dropped_on) = drop_rx.await.expect("detached result cleanup");
let DriverThreadBudget::Limited(permits) = budget.as_ref() else { panic!("limited budget") };
let returned = permits
.clone()
.acquire_many_owned(2)
.await
.expect("completed cleanup returns thread slots");
assert!(retained_during_drop);
assert_ne!(dropped_on, std::thread::current().id());
drop(returned);
}
}
+261
View File
@@ -0,0 +1,261 @@
// Copyright 2024 RustFS Team
// SPDX-License-Identifier: Apache-2.0
use std::io;
use std::sync::{Arc, LazyLock};
use tokio::sync::{OwnedSemaphorePermit, Semaphore};
// This caps concurrent initialization, not live drivers, io-wq threads or bytes.
const MAX_CONCURRENT_PROBES: usize = 4;
static PROBE_PERMITS: LazyLock<Arc<Semaphore>> = LazyLock::new(|| Arc::new(Semaphore::new(MAX_CONCURRENT_PROBES)));
/// Run only the root-independent driver probe here. Backend construction may
/// depend on a mount-lease fd owned by the async caller and must stay with it.
pub(super) async fn run<T: Send + 'static>(probe: impl FnOnce() -> T + Send + 'static) -> io::Result<T> {
run_with_permits(Arc::clone(&PROBE_PERMITS), probe).await
}
pub(super) async fn run_with_permits<T: Send + 'static>(
permits: Arc<Semaphore>,
probe: impl FnOnce() -> T + Send + 'static,
) -> io::Result<T> {
// Acquisition precedes spawn: canceling a waiter cannot enqueue probe work.
// The closure owns its permit even after its async JoinHandle is abandoned.
let permit = permits.acquire_owned().await.map_err(io::Error::other)?;
let mut result = tokio::task::spawn_blocking(move || ProbeResult {
value: Some(probe()),
permit: Some(permit),
})
.await
.map_err(io::Error::other)?;
// Consuming the result ends initialization admission. Abandoned results
// retain the permit until their blocking cleanup finishes instead.
result
.value
.take()
.ok_or_else(|| io::Error::other("uring probe result was already consumed"))
}
struct ProbeResult<T: Send + 'static> {
value: Option<T>,
permit: Option<OwnedSemaphorePermit>,
}
impl<T: Send + 'static> Drop for ProbeResult<T> {
fn drop(&mut self) {
if let Some(value) = self.value.take() {
let permit = self.permit.take();
let cleanup = move || {
// Driver destruction can join threads. In particular, dropping
// a completed-but-unconsumed JoinHandle may run here on a Tokio
// worker, so move that destruction back to blocking work.
drop(value);
drop(permit);
};
match tokio::runtime::Handle::try_current() {
Ok(runtime) => {
runtime.spawn_blocking(cleanup);
}
Err(_) => cleanup(),
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::task::{Context, Poll, Wake, Waker};
use tokio::sync::oneshot;
fn poll_once<T>(future: std::pin::Pin<&mut impl Future<Output = T>>) -> Poll<T> {
future.poll(&mut Context::from_waker(Waker::noop()))
}
#[tokio::test]
async fn probe_runs_outside_async_caller_thread() {
let caller = std::thread::current().id();
let worker = run(std::thread::current).await.expect("probe should finish").id();
assert_ne!(worker, caller);
}
#[tokio::test]
async fn concurrent_probes_never_exceed_the_initialization_limit() {
let permits = Arc::new(Semaphore::new(MAX_CONCURRENT_PROBES));
let active = Arc::new(AtomicUsize::new(0));
let peak = Arc::new(AtomicUsize::new(0));
let (entered_tx, mut entered_rx) = tokio::sync::mpsc::unbounded_channel();
let mut releases = Vec::new();
let mut probes = Vec::new();
for _ in 0..MAX_CONCURRENT_PROBES {
let (release_tx, release_rx) = std::sync::mpsc::channel();
releases.push(release_tx);
let active = active.clone();
let peak = peak.clone();
let entered = entered_tx.clone();
probes.push(tokio::spawn(run_with_permits(permits.clone(), move || {
let running = active.fetch_add(1, Ordering::SeqCst) + 1;
peak.fetch_max(running, Ordering::SeqCst);
entered.send(()).expect("observe started probe");
release_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("release blocked probe");
active.fetch_sub(1, Ordering::SeqCst);
})));
}
for _ in 0..MAX_CONCURRENT_PROBES {
entered_rx.recv().await.expect("each admitted probe should start");
}
let extra_started = Arc::new(AtomicBool::new(false));
let mark_started = extra_started.clone();
let mut extra = Box::pin(run_with_permits(permits.clone(), move || mark_started.store(true, Ordering::SeqCst)));
let was_pending = poll_once(extra.as_mut()).is_pending();
let started_early = extra_started.load(Ordering::SeqCst);
for release in releases {
release.send(()).expect("release active probes");
}
for probe in probes {
probe.await.expect("probe task join").expect("probe execution");
}
extra.await.expect("queued probe runs after release");
assert!(was_pending && !started_early);
assert_eq!(peak.load(Ordering::SeqCst), MAX_CONCURRENT_PROBES);
assert_eq!(permits.available_permits(), MAX_CONCURRENT_PROBES);
}
#[tokio::test]
async fn canceling_before_admission_never_starts_probe() {
let permits = Arc::new(Semaphore::new(1));
let held = permits.clone().acquire_owned().await.expect("occupy admission");
let started = Arc::new(AtomicBool::new(false));
let mark_started = started.clone();
let mut waiting = Box::pin(run_with_permits(permits.clone(), move || mark_started.store(true, Ordering::SeqCst)));
assert!(poll_once(waiting.as_mut()).is_pending());
drop(waiting);
drop(held);
run_with_permits(permits.clone(), || ()).await.expect("next probe should run");
assert!(!started.load(Ordering::SeqCst));
assert_eq!(permits.available_permits(), 1);
}
#[tokio::test]
async fn canceling_started_probe_holds_capacity_until_worker_finishes() {
let permits = Arc::new(Semaphore::new(1));
let (entered_tx, entered_rx) = oneshot::channel();
let (release_tx, release_rx) = std::sync::mpsc::channel();
let mut first = Box::pin(run_with_permits(permits.clone(), move || {
entered_tx.send(()).expect("notify probe entry");
release_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("release running probe");
}));
assert!(poll_once(first.as_mut()).is_pending());
entered_rx.await.expect("probe started");
drop(first);
let held_after_cancel = permits.available_permits();
let mut second = Box::pin(run_with_permits(permits.clone(), || ()));
let second_pending = poll_once(second.as_mut()).is_pending();
release_tx.send(()).expect("finish detached probe");
second.await.expect("capacity returns after real completion");
assert_eq!(held_after_cancel, 0);
assert!(second_pending);
assert_eq!(permits.available_permits(), 1);
}
#[tokio::test]
async fn ordinary_probe_errors_remain_unwrapped_and_release_capacity() {
let permits = Arc::new(Semaphore::new(1));
// Model both restriction and unexpected probe results. The offload layer
// must pass them through; only its own Acquire/Join errors are wrapped.
for errno in [1, 5] {
let result = run_with_permits(permits.clone(), move || Err::<(), _>(io::Error::from_raw_os_error(errno)))
.await
.expect("blocking work itself should complete");
assert_eq!(result.expect_err("probe returned an error").raw_os_error(), Some(errno));
assert_eq!(permits.available_permits(), 1);
}
}
#[tokio::test]
async fn panic_and_closed_admission_are_not_os_restriction_errors() {
let permits = Arc::new(Semaphore::new(1));
let panic_error = run_with_permits(permits.clone(), || panic!("injected probe panic"))
.await
.expect_err("worker panic must be returned");
assert!(
panic_error
.get_ref()
.and_then(|error| error.downcast_ref::<tokio::task::JoinError>())
.is_some()
);
assert_eq!(panic_error.raw_os_error(), None);
assert_eq!(permits.available_permits(), 1);
permits.close();
let closed_error = run_with_permits(permits, || panic!("closed admission must not start"))
.await
.expect_err("closed admission must return an error");
assert_eq!(closed_error.raw_os_error(), None);
assert!(
closed_error
.get_ref()
.and_then(|error| error.downcast_ref::<tokio::sync::AcquireError>())
.is_some()
);
}
#[tokio::test]
async fn completed_unconsumed_result_drops_outside_async_caller_with_capacity_held() {
struct WakeFinished(Arc<tokio::sync::Notify>);
impl Wake for WakeFinished {
fn wake(self: Arc<Self>) {
self.0.notify_one();
}
}
struct BlockingDrop {
entered: Option<oneshot::Sender<std::thread::ThreadId>>,
release: std::sync::mpsc::Receiver<()>,
}
impl Drop for BlockingDrop {
fn drop(&mut self) {
self.entered
.take()
.expect("single drop")
.send(std::thread::current().id())
.expect("observe cleanup");
self.release
.recv_timeout(std::time::Duration::from_secs(10))
.expect("release cleanup");
}
}
let permits = Arc::new(Semaphore::new(1));
let (drop_tx, drop_rx) = oneshot::channel();
let (release_tx, release_rx) = std::sync::mpsc::channel();
let (produce_tx, produce_rx) = std::sync::mpsc::channel();
let result = BlockingDrop {
entered: Some(drop_tx),
release: release_rx,
};
let mut probe = Box::pin(run_with_permits(permits.clone(), move || {
produce_rx
.recv_timeout(std::time::Duration::from_secs(10))
.expect("allow probe completion after waiter registration");
result
}));
let finished = Arc::new(tokio::sync::Notify::new());
let waker = Waker::from(Arc::new(WakeFinished(finished.clone())));
assert!(probe.as_mut().poll(&mut Context::from_waker(&waker)).is_pending());
produce_tx.send(()).expect("complete probe");
// JoinHandle wakes only after its result is ready. Do not poll the
// outer future again: cancel precisely before it consumes that result.
finished.notified().await;
drop(probe);
let dropped_on = drop_rx.await.expect("abandoned result cleanup should run");
let capacity_during_drop = permits.available_permits();
release_tx.send(()).expect("allow cleanup to finish");
let released = permits.clone().acquire_owned().await.expect("cleanup returns permit");
assert_ne!(dropped_on, std::thread::current().id());
assert_eq!(capacity_during_drop, 0);
drop(released);
}
}
@@ -0,0 +1,156 @@
// Copyright 2024 RustFS Team
// SPDX-License-Identifier: Apache-2.0
use std::ffi::OsStr;
#[cfg(target_os = "linux")]
pub(super) const ENV_TOTAL: &str = "RUSTFS_IO_URING_READ_BUDGET_TOTAL_BYTES";
#[cfg(target_os = "linux")]
pub(super) const ENV_DRIVER: &str = "RUSTFS_IO_URING_READ_BUDGET_DRIVER_BYTES";
/// Parsed independently of Linux driver construction so configuration checks
/// never require a kernel probe or process-wide environment mutation.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(super) enum ReadBudgetConfig {
Disabled,
Limited { total: usize, per_driver: usize },
}
#[derive(Debug)]
pub(super) struct InvalidReadBudget;
impl std::fmt::Display for InvalidReadBudget {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str(
"expected both read-budget settings as positive decimal byte counts, with driver quota no greater than total capacity or Semaphore::MAX_PERMITS",
)
}
}
impl std::error::Error for InvalidReadBudget {}
impl ReadBudgetConfig {
pub(super) fn from_env_values(total: Option<&OsStr>, per_driver: Option<&OsStr>) -> Result<Self, InvalidReadBudget> {
let (total, per_driver) = match (total, per_driver) {
(None, None) => return Ok(Self::Disabled),
(Some(total), Some(per_driver)) => (total, per_driver),
_ => return Err(InvalidReadBudget),
};
let parse = |raw: &OsStr| {
let text = raw.to_str().ok_or(InvalidReadBudget)?;
if text.is_empty() || !text.bytes().all(|byte| byte.is_ascii_digit()) {
return Err(InvalidReadBudget);
}
text.parse::<usize>().map_err(|_| InvalidReadBudget)
};
let total = parse(total)?;
let per_driver = parse(per_driver)?;
if total == 0 || per_driver == 0 || per_driver > total || per_driver > tokio::sync::Semaphore::MAX_PERMITS {
return Err(InvalidReadBudget);
}
Ok(Self::Limited { total, per_driver })
}
}
/// Clones retain the same process pool, including across disk reconstruction.
/// The library, rather than the backend wrapper, retains leaked reservations.
#[cfg(target_os = "linux")]
#[derive(Clone)]
pub(super) enum DriverReadBudget {
Disabled,
Limited {
pool: rustfs_uring::SharedReadBudget,
per_driver: usize,
},
}
#[cfg(target_os = "linux")]
impl DriverReadBudget {
pub(super) fn from_config(config: ReadBudgetConfig) -> Result<Self, InvalidReadBudget> {
match config {
ReadBudgetConfig::Disabled => Ok(Self::Disabled),
ReadBudgetConfig::Limited { total, per_driver } => Ok(Self::Limited {
pool: rustfs_uring::SharedReadBudget::new(total).map_err(|_| InvalidReadBudget)?,
per_driver,
}),
}
}
pub(super) fn start_driver(
&self,
entries: u32,
shards: usize,
) -> Result<rustfs_uring::UringDriver, rustfs_uring::ProbeFailure> {
match self {
Self::Disabled => rustfs_uring::UringDriver::probe_and_start_sharded(entries, shards),
Self::Limited { pool, per_driver } => rustfs_uring::UringDriver::probe_and_start_with_shared_budget(
entries,
shards,
rustfs_uring::ReadLimits {
max_read_len: None,
max_in_flight_bytes: Some(*per_driver),
},
pool,
),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn parse(total: Option<&str>, per_driver: Option<&str>) -> Result<ReadBudgetConfig, InvalidReadBudget> {
ReadBudgetConfig::from_env_values(total.map(OsStr::new), per_driver.map(OsStr::new))
}
#[test]
fn both_unset_preserve_legacy_but_partial_configuration_fails() {
assert_eq!(parse(None, None).expect("unset settings"), ReadBudgetConfig::Disabled);
assert!(parse(Some("8"), None).is_err());
assert!(parse(None, Some("8")).is_err());
assert!(parse(Some("0"), Some("0")).is_err(), "zero is not an implicit unlimited mode");
}
#[test]
fn valid_pair_keeps_usize_total_and_exact_driver_quota() {
for (total, per_driver) in [(1, 1), (16, 8), (usize::MAX, tokio::sync::Semaphore::MAX_PERMITS)] {
assert_eq!(
parse(Some(&total.to_string()), Some(&per_driver.to_string())).expect("valid quota pair"),
ReadBudgetConfig::Limited { total, per_driver }
);
}
assert_eq!(
parse(Some("00016"), Some("00008")).expect("leading zeros remain decimal"),
ReadBudgetConfig::Limited {
total: 16,
per_driver: 8
}
);
}
#[test]
fn malformed_or_impossible_pair_never_removes_the_limit() {
for value in ["", " ", "0", " 8", "8 ", "+8", "-1", "8MiB", "1.5", "184467440737095516160"] {
assert!(parse(Some(value), Some("1")).is_err(), "invalid total accepted");
assert!(parse(Some("16"), Some(value)).is_err(), "invalid driver quota accepted");
}
assert!(parse(Some("7"), Some("8")).is_err());
assert!(
parse(
Some(&usize::MAX.to_string()),
Some(&(tokio::sync::Semaphore::MAX_PERMITS + 1).to_string())
)
.is_err()
);
}
#[cfg(unix)]
#[test]
fn non_unicode_configuration_is_rejected_on_either_side() {
use std::os::unix::ffi::OsStrExt;
let invalid = OsStr::from_bytes(&[0xff]);
assert!(ReadBudgetConfig::from_env_values(Some(invalid), Some(OsStr::new("1"))).is_err());
assert!(ReadBudgetConfig::from_env_values(Some(OsStr::new("8")), Some(invalid)).is_err());
}
}
@@ -0,0 +1,95 @@
// Copyright 2024 RustFS Team
// SPDX-License-Identifier: Apache-2.0
use std::ffi::OsStr;
const MIN_READ_CHUNK_BYTES: usize = 4096;
const DEFAULT_READ_CHUNK_BYTES: usize = 128 << 20;
/// Validated logical length cap for one io_uring read operation. Direct-I/O
/// padding, parallel reads and application result allocations are not included.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(super) struct ReadChunkSize(usize);
#[derive(Debug)]
pub(super) struct InvalidReadChunkSize;
impl std::fmt::Display for InvalidReadChunkSize {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str("expected an unsigned decimal integer between 4096 and 134217728 bytes")
}
}
impl std::error::Error for InvalidReadChunkSize {}
impl ReadChunkSize {
pub(super) fn from_env_value(value: Option<&OsStr>) -> Result<Self, InvalidReadChunkSize> {
let Some(value) = value else { return Ok(Self(DEFAULT_READ_CHUNK_BYTES)) };
let text = value.to_str().ok_or(InvalidReadChunkSize)?;
if text.is_empty() || !text.bytes().all(|byte| byte.is_ascii_digit()) {
return Err(InvalidReadChunkSize);
}
let size = text.parse::<usize>().map_err(|_| InvalidReadChunkSize)?;
if !(MIN_READ_CHUNK_BYTES..=DEFAULT_READ_CHUNK_BYTES).contains(&size) {
return Err(InvalidReadChunkSize);
}
Ok(Self(size))
}
pub(super) fn get(self) -> usize {
self.0
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn missing_configuration_preserves_128_mib_default() {
assert_eq!(ReadChunkSize::from_env_value(None).expect("default config").get(), 128 << 20);
}
#[test]
fn accepts_range_boundaries_and_non_block_multiple_caps() {
for size in [4096, 4097, 65536, 128 << 20] {
assert_eq!(
ReadChunkSize::from_env_value(Some(OsStr::new(&size.to_string())))
.expect("valid cap")
.get(),
size
);
}
}
#[test]
fn invalid_configuration_never_becomes_the_default() {
for value in [
"",
" ",
"0",
"4095",
"134217729",
"-4096",
"+4096",
" 4096",
"4096 ",
"4KiB",
"4096.0",
"184467440737095516160",
] {
assert!(
ReadChunkSize::from_env_value(Some(OsStr::new(value))).is_err(),
"invalid cap accepted: {value}"
);
}
assert!(ReadChunkSize::from_env_value(Some(OsStr::new(&usize::MAX.to_string()))).is_err());
}
#[cfg(unix)]
#[test]
fn non_utf8_configuration_is_rejected() {
use std::os::unix::ffi::OsStrExt;
assert!(ReadChunkSize::from_env_value(Some(OsStr::from_bytes(&[0xff]))).is_err());
}
}
@@ -0,0 +1,183 @@
// Copyright 2024 RustFS Team
// SPDX-License-Identifier: Apache-2.0
use bytes::Bytes;
use std::ffi::OsStr;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
#[cfg(target_os = "linux")]
pub(super) const ENV_RESULT: &str = "RUSTFS_IO_URING_READ_RESULT_BUDGET_BYTES";
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) enum ResultBudgetConfig {
Disabled,
Limited { bytes: usize },
}
#[derive(Debug)]
pub(super) struct InvalidResultBudget;
impl std::fmt::Display for InvalidResultBudget {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str("expected a positive decimal result-budget byte count")
}
}
impl std::error::Error for InvalidResultBudget {}
impl ResultBudgetConfig {
pub(super) fn from_env_value(value: Option<&OsStr>) -> Result<Self, InvalidResultBudget> {
let Some(value) = value else {
return Ok(Self::Disabled);
};
let value = value.to_str().ok_or(InvalidResultBudget)?;
if value.is_empty() || !value.bytes().all(|byte| byte.is_ascii_digit()) {
return Err(InvalidResultBudget);
}
let bytes = value.parse::<usize>().map_err(|_| InvalidResultBudget)?;
if bytes == 0 {
return Err(InvalidResultBudget);
}
Ok(Self::Limited { bytes })
}
}
#[derive(Clone, Debug)]
pub(super) enum ProcessResultBudget {
Disabled,
Limited(Arc<ResultBudgetInner>),
}
#[derive(Debug)]
pub(super) struct ResultBudgetInner {
available: AtomicUsize,
}
impl ProcessResultBudget {
pub(super) fn from_config(config: ResultBudgetConfig) -> Self {
match config {
ResultBudgetConfig::Disabled => Self::Disabled,
ResultBudgetConfig::Limited { bytes } => Self::Limited(Arc::new(ResultBudgetInner {
available: AtomicUsize::new(bytes),
})),
}
}
pub(super) fn reserve(&self, bytes: usize) -> Result<Option<ResultBudgetReservation>, std::io::Error> {
let Self::Limited(inner) = self else {
return Ok(None);
};
if bytes == 0 {
return Ok(None);
}
inner
.available
.fetch_update(Ordering::AcqRel, Ordering::Acquire, |available| available.checked_sub(bytes))
.map(|_| {
Some(ResultBudgetReservation {
inner: Arc::clone(inner),
bytes,
})
})
.map_err(|_| std::io::Error::new(std::io::ErrorKind::WouldBlock, ResultBudgetExhausted))
}
#[cfg(test)]
pub(super) fn available(&self) -> Option<usize> {
match self {
Self::Disabled => None,
Self::Limited(inner) => Some(inner.available.load(Ordering::Acquire)),
}
}
}
#[derive(Debug)]
pub(super) struct ResultBudgetReservation {
inner: Arc<ResultBudgetInner>,
bytes: usize,
}
impl Drop for ResultBudgetReservation {
fn drop(&mut self) {
self.inner.available.fetch_add(self.bytes, Ordering::Release);
}
}
/// Keeps the result-budget reservation alive until every returned `Bytes` clone
/// is dropped. `Bytes::from_owner` retains this owner without copying the data.
pub(super) struct BudgetedBytes {
bytes: Bytes,
_reservation: Option<ResultBudgetReservation>,
}
impl BudgetedBytes {
pub(super) fn wrap(bytes: Bytes, reservation: Option<ResultBudgetReservation>) -> Bytes {
match reservation {
Some(reservation) => Bytes::from_owner(Self {
bytes,
_reservation: Some(reservation),
}),
None => bytes,
}
}
}
impl AsRef<[u8]> for BudgetedBytes {
fn as_ref(&self) -> &[u8] {
self.bytes.as_ref()
}
}
/// Marker carried by `WouldBlock` so the io_uring backend does not route a
/// process-budget denial through the unbounded std fallback.
#[derive(Debug)]
pub(super) struct ResultBudgetExhausted;
impl std::fmt::Display for ResultBudgetExhausted {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str("io_uring result budget exhausted")
}
}
impl std::error::Error for ResultBudgetExhausted {}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn configuration_is_strict_and_unset_is_disabled() {
assert_eq!(ResultBudgetConfig::from_env_value(None).unwrap(), ResultBudgetConfig::Disabled);
assert_eq!(
ResultBudgetConfig::from_env_value(Some(OsStr::new("4096"))).unwrap(),
ResultBudgetConfig::Limited { bytes: 4096 }
);
for value in ["", "0", " 8", "+8", "8 ", "8MiB"] {
assert!(ResultBudgetConfig::from_env_value(Some(OsStr::new(value))).is_err());
}
}
#[test]
fn reservation_is_refunded_only_after_owner_drop() {
let budget = ProcessResultBudget::from_config(ResultBudgetConfig::Limited { bytes: 8 });
let reservation = budget.reserve(8).unwrap().unwrap();
assert_eq!(budget.available(), Some(0));
assert!(budget.reserve(1).is_err());
drop(reservation);
assert_eq!(budget.available(), Some(8));
}
#[test]
fn bytes_owner_keeps_reservation_through_clones() {
let budget = ProcessResultBudget::from_config(ResultBudgetConfig::Limited { bytes: 4 });
let reservation = budget.reserve(4).unwrap().unwrap();
let bytes = BudgetedBytes::wrap(Bytes::from_static(b"data"), Some(reservation));
let clone = bytes.clone();
assert_eq!(budget.available(), Some(0));
drop(bytes);
assert_eq!(budget.available(), Some(0));
drop(clone);
assert_eq!(budget.available(), Some(4));
}
}
@@ -23,7 +23,6 @@ use crate::error::{Error, Result};
use crate::layout::endpoints::EndpointServerPools;
use crate::runtime::sources as runtime_sources;
use crate::services::metrics_realtime::{CollectMetricsOpts, MetricType};
use crate::services::rebalance::RebalSaveOpt;
use crate::storage_api_contracts::admin::StorageAdminApi;
use bytes::Bytes;
use futures::future::join_all;
@@ -2455,30 +2454,11 @@ impl NotificationSys {
None => store.current_rebalance_id().await,
};
match store.stop_rebalance_for_id(local_rebalance_id.as_deref()).await {
Ok(_) => {
let save_result = match local_rebalance_id.as_deref() {
Some(expected_id) => {
store
.save_rebalance_stats_for_id(usize::MAX, RebalSaveOpt::StoppedAt, expected_id)
.await
}
None => Ok(()),
};
if let Err(err) = save_result {
error!(
event = EVENT_NOTIFICATION_PEER_PROPAGATION,
component = LOG_COMPONENT_ECSTORE,
subsystem = LOG_SUBSYSTEM_NOTIFICATION,
action = "stop_rebalance",
result = "local_save_failed",
error = %err,
"notification peer propagation"
);
return Err(Error::other(format!(
"local stop_rebalance save_rebalance_stats(stopped_at) failed: {err}"
)));
}
}
// `stop_rebalance_for_id` durably saves the terminal snapshot after
// activation readers drain. Do not acquire the same metadata lock
// again for an equivalent `StoppedAt` save while cleanup is still
// releasing its own metadata work.
Ok(_) => {}
Err(err) => {
error!(
event = EVENT_NOTIFICATION_PEER_PROPAGATION,
@@ -630,7 +630,7 @@ impl ECStore {
pub async fn prepare_rebalance_stop(&self) -> Result<Option<String>> {
let _start_guard = self.start_gate.lock().await;
{
let (local_stop_id, activation_gate) = {
let mut rebalance_meta = self.rebalance_meta.write().await;
if let Some(meta) = rebalance_meta.as_mut()
&& is_rebalance_conflicting_with_decommission(meta)
@@ -641,7 +641,22 @@ impl ECStore {
.cancel();
#[cfg(any(test, feature = "test-util"))]
observe_rebalance_stop_wait_attempt(Some(meta.id.as_str()));
(Some(meta.id.clone()), Some(Arc::clone(&meta.activation_gate)))
} else {
(None, None)
}
};
// An in-flight entry already owns the activation read gate. Admission
// has been cancelled above, so do not wait on that same gate merely to
// refresh a stop target: the eventual stop operation will take the
// write gate after the entry drains. If the gate is currently free,
// retain the historical reload behaviour (which observes a worker that
// reached a terminal state just before this request).
if let Some(gate) = activation_gate
&& gate.try_write_owned().is_err()
{
return Ok(local_stop_id);
}
self.load_rebalance_meta_under_start_gate().await?;
@@ -2490,15 +2490,15 @@ impl SetDisks {
part_numbers: &[usize],
read_quorum: usize,
) -> disk::error::Result<Vec<ObjectPartInfo>> {
let bucket = bucket.to_string();
let part_meta_paths = part_meta_paths.to_vec();
let bucket: Arc<str> = Arc::from(bucket);
let part_meta_paths: Arc<[String]> = Arc::from(part_meta_paths);
let tasks: Vec<_> = disks
.iter()
.map(|disk| {
let disk = disk.clone();
let bucket = bucket.clone();
let part_meta_paths = part_meta_paths.clone();
let bucket = Arc::clone(&bucket);
let part_meta_paths = Arc::clone(&part_meta_paths);
async move {
if let Some(disk) = disk {
+44 -2
View File
@@ -6809,7 +6809,10 @@ pub fn should_heal_object_on_disk(
latest_meta: &FileInfo,
) -> (bool, bool, Option<DiskError>) {
if let Some(err) = err
&& (err == &DiskError::FileNotFound || err == &DiskError::FileVersionNotFound || err == &DiskError::FileCorrupt)
&& (err == &DiskError::FileNotFound
|| err == &DiskError::FileVersionNotFound
|| err == &DiskError::FileCorrupt
|| err == &DiskError::VolumeNotFound)
{
return (true, true, Some(err.clone()));
}
@@ -6818,7 +6821,12 @@ pub fn should_heal_object_on_disk(
return (false, false, err.clone());
}
if !meta.equals(latest_meta) {
// `FileInfo::equals` intentionally compares the erasure payload shape and
// modification time, but it does not compare the selected version or the
// delete-marker bit. Heal uses this decision for versioned metadata, so a
// stale historical version that is still marked latest must be treated as
// outdated even when those storage-level fields happen to match.
if !meta.equals(latest_meta) || !heal_metadata_identity_matches(meta, latest_meta) {
debug!(
event = EVENT_SET_DISK_HEAL,
component = LOG_COMPONENT_ECSTORE,
@@ -6841,6 +6849,11 @@ pub fn should_heal_object_on_disk(
(false, false, None)
}
fn heal_metadata_identity_matches(meta: &FileInfo, latest_meta: &FileInfo) -> bool {
meta.deleted == latest_meta.deleted
&& meta.version_id.filter(|version| !version.is_nil()) == latest_meta.version_id.filter(|version| !version.is_nil())
}
/// Probe every drive of the set at once. Each live probe is bounded by the
/// drive `disk_info` timeout, and the admin peer probe budget only covers one
/// such timeout; a sequential walk over several stalled drives after a power
@@ -11263,6 +11276,12 @@ mod tests {
let (should_heal, _, _) = should_heal_object_on_disk(&err, &[], &meta, &latest_meta);
assert!(should_heal);
let err = Some(DiskError::VolumeNotFound);
let (should_heal, is_meta, reason) = should_heal_object_on_disk(&err, &[], &meta, &latest_meta);
assert!(should_heal);
assert!(is_meta);
assert_eq!(reason, Some(DiskError::VolumeNotFound));
let err = Some(DiskError::FileCorrupt);
let (should_heal, is_meta, reason) = should_heal_object_on_disk(&err, &[], &meta, &latest_meta);
assert!(should_heal);
@@ -11279,6 +11298,29 @@ mod tests {
assert_eq!(reason, Some(DiskError::FileCorrupt));
}
#[test]
fn stale_delete_marker_identity_requires_metadata_heal() {
let historical_version = Uuid::new_v4();
let marker_version = Uuid::new_v4();
let mut stale = FileInfo {
version_id: Some(historical_version),
..FileInfo::default()
};
let mut latest = stale.clone();
stale.deleted = false;
latest.deleted = true;
latest.version_id = Some(marker_version);
// The generic equality helper intentionally considers these records
// equal when their erasure shape and mod-time match. Heal must still
// repair the version identity and delete-marker state.
assert!(stale.equals(&latest));
let (should_heal, metadata, reason) = should_heal_object_on_disk(&None, &[], &stale, &latest);
assert!(should_heal);
assert!(metadata);
assert_eq!(reason, Some(DiskError::OutdatedXLMeta));
}
#[test]
fn metadata_io_failures_never_authorize_heal_overwrite() {
let meta = FileInfo::default();
+203 -7
View File
@@ -502,6 +502,22 @@ fn heal_writer_error_summary(error: &DiskError) -> String {
}
}
/// Whether a writer failure can be retried after the target disk or peer
/// becomes available again. Keep this decision typed: the aggregate
/// `all_targets_unavailable` path must not turn permanent errors such as
/// `DiskFull` into retryable failures.
fn is_retryable_heal_writer_error(error: &DiskError) -> bool {
matches!(
error,
DiskError::DiskNotFound
| DiskError::Timeout
| DiskError::SourceStalled
| DiskError::FaultyRemoteDisk
| DiskError::FaultyDisk
| DiskError::RemoteClientUnavailable(_)
) || error.is_retryable_internode_write_failure()
}
fn warn_heal_writer_failures(
bucket: &str,
object: &str,
@@ -1398,6 +1414,26 @@ impl SetDisks {
));
}
// `errs` is reported in physical-disk order while the
// reconstruction loop below uses the shard order from
// the selected metadata. Preserve which targets are
// missing their bucket volume across that permutation.
let mut missing_bucket_volumes = vec![false; out_dated_disks.len()];
for (physical_index, error) in errs.iter().enumerate() {
if error.as_ref() != Some(&DiskError::VolumeNotFound) {
continue;
}
let shard_index = latest_meta
.erasure
.distribution
.get(physical_index)
.and_then(|index| index.checked_sub(1))
.unwrap_or(physical_index);
if let Some(target) = missing_bucket_volumes.get_mut(shard_index) {
*target = true;
}
}
out_dated_disks = Self::shuffle_disks(&out_dated_disks, &latest_meta.erasure.distribution);
let mut parts_metadata = Self::shuffle_parts_metadata(&parts_metadata, &latest_meta.erasure.distribution);
let mut copy_parts_metadata = vec![None; parts_metadata.len()];
@@ -1424,6 +1460,58 @@ impl SetDisks {
}
}
// A missing bucket volume is a repairable object target,
// but the final rename cannot create its parent volume.
// Recreate only the affected volumes after quorum and
// lifecycle checks have passed. Keep failures visible so
// a partial repair cannot be reported as successful.
let mut volume_creation_error = None;
for (index, outdated_disk) in out_dated_disks.iter_mut().enumerate() {
if !missing_bucket_volumes.get(index).copied().unwrap_or(false) || outdated_disk.is_none() {
continue;
}
if let Some(scope) = &bucket_heal_scope {
scope.check()?;
}
let disk = outdated_disk.as_ref().expect("checked above").clone();
let endpoint = disk.endpoint().to_string();
match disk.make_volume(bucket).await {
Ok(()) | Err(DiskError::VolumeExists) => {}
Err(error) => {
warn!(
event = EVENT_SET_DISK_HEAL,
component = LOG_COMPONENT_ECSTORE,
subsystem = LOG_SUBSYSTEM_HEAL,
bucket,
object,
version_id,
disk_index = index,
endpoint = %endpoint,
error = %error,
state = "bucket_volume_recreate_failed",
"Heal object could not recreate the target bucket volume"
);
if volume_creation_error.is_none() {
volume_creation_error = Some(error.clone());
}
*outdated_disk = None;
disks_to_heal_count = disks_to_heal_count.saturating_sub(1);
for drive in &mut result.after.drives {
if drive.endpoint == endpoint {
drive.state = heal_drive_state_for_error(&error).to_string();
}
}
}
}
}
if let Some(scope) = &bucket_heal_scope {
scope.check()?;
}
if disks_to_heal_count == 0 {
return Ok((result, volume_creation_error.or(Some(DiskError::ErasureWriteQuorum))));
}
// We write at temporary location and then rename to final location.
let tmp_id = Uuid::new_v4().to_string();
// Delete markers and remote (transitioned) objects carry no data_dir and
@@ -1458,6 +1546,8 @@ impl SetDisks {
let erasure_info = latest_meta.erasure.clone();
let mut writer_failure_count = 0usize;
let mut first_writer_failure = None;
let mut first_writer_error = None;
let mut all_writer_failures_retryable = true;
let mut writer_failure_warned = false;
for (part_index, part) in latest_meta.parts.iter().enumerate() {
@@ -1584,9 +1674,11 @@ impl SetDisks {
Ok(writer) => writer,
Err(err) => {
writer_failure_count += 1;
all_writer_failures_retryable &= is_retryable_heal_writer_error(&err);
if first_writer_failure.is_none() {
first_writer_failure =
Some((part.number, index, heal_writer_error_summary(&err)));
first_writer_error = Some(err);
}
writers.push(None);
continue;
@@ -1714,12 +1806,12 @@ impl SetDisks {
}
// Clean up healed shards written to .rustfs/tmp before bailing (B20).
let _ = self.delete_all(RUSTFS_META_TMP_BUCKET, &tmp_id).await;
return Ok((
result,
Some(DiskError::other(format!(
"all drives had write errors, unable to heal {bucket}/{object}"
))),
));
let error = if all_writer_failures_retryable {
first_writer_error.take().unwrap_or(DiskError::FaultyDisk)
} else {
DiskError::other(format!("all drives had write errors, unable to heal {bucket}/{object}"))
};
return Ok((result, Some(error)));
}
}
@@ -1865,6 +1957,10 @@ impl SetDisks {
));
}
if let Some(error) = volume_creation_error {
return Ok((result, Some(error)));
}
if result.integrity_verified
&& let Err(error) =
crate::io_support::shard_integrity::restore_proof_replicas(&latest_meta, &disks, bucket, object)
@@ -3430,7 +3526,7 @@ impl SetDisks {
mod heal_result_report_tests {
use super::{
AbsenceProofRequest, DanglingCheckPartsFailure, DanglingDeleteFailure, DanglingDeleteSafety, ReadRepairCommitFingerprint,
ReadRepairPauseScope, SetDisks, heal_writer_error_summary,
ReadRepairPauseScope, SetDisks, heal_writer_error_summary, is_retryable_heal_writer_error,
};
use super::{HEAL_RENAME_INCOMPLETE, HealRenameFailureScope, HealWriterFailureScope};
use crate::disk::endpoint::Endpoint;
@@ -3663,6 +3759,24 @@ mod heal_result_report_tests {
assert!(!summary.contains("sensitive"));
}
#[test]
fn retryable_heal_writer_error_keeps_permanent_errors_out() {
for error in [
DiskError::DiskNotFound,
DiskError::Timeout,
DiskError::SourceStalled,
DiskError::FaultyRemoteDisk,
DiskError::FaultyDisk,
DiskError::RemoteClientUnavailable("peer restarting".to_string()),
] {
assert!(is_retryable_heal_writer_error(&error), "{error:?} should be retryable");
}
for error in [DiskError::DiskFull, DiskError::DiskAccessDenied, DiskError::FileCorrupt] {
assert!(!is_retryable_heal_writer_error(&error), "{error:?} should remain permanent");
}
}
#[tokio::test(flavor = "current_thread")]
#[serial_test::serial]
async fn heal_writer_failures_emit_one_aggregate_warning_per_object() {
@@ -4223,6 +4337,88 @@ mod heal_result_report_tests {
);
}
#[tokio::test]
async fn deep_heal_recreates_missing_bucket_volume_before_rebuilding_shard() {
let (temp_dirs, disks, set) = hermetic_set_disks_isolated(4).await;
let bucket = "deep-heal-missing-bucket-volume";
let object = "object.bin";
for disk in &disks {
disk.make_volume(bucket).await.expect("bucket volume should be created");
}
let mut reader = PutObjReader::from_vec(vec![0x5c; 1024 * 1024]);
set.put_object(
bucket,
object,
&mut reader,
&ObjectOptions {
no_lock: true,
..Default::default()
},
)
.await
.expect("source object should be written before volume loss");
let source = disks[2]
.read_version("", bucket, object, "", &ReadOptions::default())
.await
.expect("source metadata should be readable");
let data_dir = source.data_dir.expect("non-inline source should have a data directory");
disks[1]
.delete_volume(bucket, true)
.await
.expect("target bucket volume should be removable for the regression setup");
assert!(matches!(disks[1].stat_volume(bucket).await, Err(DiskError::VolumeNotFound)));
let (result, error) = set
.heal_object(
bucket,
object,
"",
&HealOpts {
no_lock: true,
scan_mode: HealScanMode::Deep,
..Default::default()
},
)
.await
.expect("deep heal should finish after bucket volume loss");
assert!(error.is_none(), "deep heal should recreate the missing bucket volume: {error:?}");
assert!(disks[1].stat_volume(bucket).await.is_ok(), "target bucket volume should be restored");
let healed = disks[1]
.read_version("", bucket, object, "", &ReadOptions::default())
.await
.expect("healed target metadata should be readable");
assert_eq!(healed.data_dir, Some(data_dir));
assert_eq!(result.after.drives[1].state, DriveState::Ok.to_string());
assert!(
temp_dirs[1]
.path()
.join(bucket)
.join(object)
.join(data_dir.to_string())
.join("part.1")
.exists(),
"deep heal should rebuild the object shard after recreating its volume"
);
let (_, retry_error) = set
.heal_object(
bucket,
object,
"",
&HealOpts {
no_lock: true,
scan_mode: HealScanMode::Deep,
..Default::default()
},
)
.await
.expect("repeat heal should complete");
assert!(retry_error.is_none(), "repeat heal should be idempotent: {retry_error:?}");
}
#[tokio::test]
async fn deep_heal_rebuilds_a_stale_current_delete_marker_metadata_replica() {
let (_temp_dirs, disks, set) = hermetic_set_disks_isolated(4).await;
+128 -40
View File
@@ -66,6 +66,7 @@ use crate::disk::DiskOption;
use crate::disk::STORAGE_FORMAT_FILE;
#[cfg(test)]
use crate::disk::new_disk;
use crate::erasure::coding::BitrotWriterWrapper;
use crate::multipart_listing::paginate_multipart_listing;
#[cfg(test)]
use crate::object_api::ObjectLockConfigSnapshot;
@@ -77,7 +78,7 @@ use crate::storage_api_contracts::multipart::MultipartOperations;
#[cfg(test)]
use crate::storage_api_contracts::object::HTTPPreconditions;
use crate::storage_api_contracts::object::ObjectOperations;
use futures::{StreamExt, stream};
use futures::{StreamExt, future::join_all, stream};
#[cfg(test)]
use http::HeaderMap;
use rustfs_filemeta::metadata_keys;
@@ -100,6 +101,49 @@ use std::time::Duration;
use tokio::io::AsyncReadExt;
use tokio::task::JoinSet;
// The erasure-set width bounds fan-out. Await every opener so errors retain
// their disk slots and every successful writer remains owned until quorum is checked.
async fn create_part_writers(
disks: &[Option<DiskStore>],
path: &str,
length: i64,
shard_size: usize,
) -> (Vec<Option<BitrotWriterWrapper>>, Vec<Option<DiskError>>) {
join_all(disks.iter().map(|disk| async move {
let Some(disk) = disk else {
return (None, Some(DiskError::DiskNotFound));
};
match create_bitrot_writer(
false,
Some(disk),
RUSTFS_META_TMP_BUCKET,
path,
length,
shard_size,
HashAlgorithm::HighwayHash256S,
)
.await
{
Ok(writer) => (Some(writer), None),
Err(err) => {
warn!(
event = EVENT_SET_DISK_MULTIPART,
component = LOG_COMPONENT_ECSTORE,
subsystem = LOG_SUBSYSTEM_SET_DISK,
disk = ?disk,
state = "bitrot_writer_skipped",
error = ?err,
"Set disk multipart bitrot writer skipped"
);
(None, Some(err))
}
}
}))
.await
.into_iter()
.unzip()
}
const MULTIPART_LIST_IO_CONCURRENCY: usize = 16;
static CAPPED_MULTIPART_STAGING: OnceLock<Mutex<HashMap<String, Arc<tokio::sync::Semaphore>>>> = OnceLock::new();
@@ -1542,45 +1586,13 @@ impl crate::storage_api_contracts::multipart::MultipartOperations for SetDisks {
.map_err(Error::from)?);
let writer_setup_stage_start = rustfs_io_metrics::put_stage_metrics_enabled().then(Instant::now);
let mut writers = Vec::with_capacity(shuffle_disks.len());
let mut errors = Vec::with_capacity(shuffle_disks.len());
for disk_op in shuffle_disks.iter() {
if let Some(disk) = disk_op {
let writer = match create_bitrot_writer(
false,
Some(disk),
RUSTFS_META_TMP_BUCKET,
&tmp_part_path,
erasure.shard_file_size(data.size()),
erasure.shard_size(),
HashAlgorithm::HighwayHash256S,
)
.await
{
Ok(writer) => writer,
Err(err) => {
warn!(
event = EVENT_SET_DISK_MULTIPART,
component = LOG_COMPONENT_ECSTORE,
subsystem = LOG_SUBSYSTEM_SET_DISK,
disk = ?disk,
state = "bitrot_writer_skipped",
error = ?err,
"Set disk multipart bitrot writer skipped"
);
errors.push(Some(err));
writers.push(None);
continue;
}
};
writers.push(Some(writer));
errors.push(None);
} else {
errors.push(Some(DiskError::DiskNotFound));
writers.push(None);
}
}
let (mut writers, errors) = create_part_writers(
&shuffle_disks,
&tmp_part_path,
erasure.shard_file_size(data.size()),
erasure.shard_size(),
)
.await;
if let Some(stage_start) = writer_setup_stage_start {
rustfs_io_metrics::record_put_object_stage_duration(
@@ -3594,6 +3606,82 @@ mod tests {
use tempfile::TempDir;
use tokio::sync::{Notify, RwLock};
#[tokio::test]
async fn multipart_writer_setup_opens_disks_concurrently_and_preserves_error_slots() {
use crate::cluster::rpc::internode_data_transport::{
InternodeDataTransport, InternodeDataTransportCapabilities, ReadStreamRequest, WalkDirStreamRequest,
WriteStreamRequest,
};
use crate::cluster::rpc::remote_disk::RemoteDisk;
use crate::disk::{Disk, FileReader, FileWriter};
#[derive(Debug)]
struct BarrierTransport {
barrier: Arc<tokio::sync::Barrier>,
fail: bool,
}
#[async_trait::async_trait]
impl InternodeDataTransport for BarrierTransport {
async fn open_read(&self, _: ReadStreamRequest) -> disk::error::Result<FileReader> {
unreachable!("writer setup must not read")
}
async fn open_walk_dir(&self, _: WalkDirStreamRequest) -> disk::error::Result<FileReader> {
unreachable!("writer setup must not list")
}
async fn open_write(&self, request: WriteStreamRequest) -> disk::error::Result<FileWriter> {
assert_eq!(request.volume, RUSTFS_META_TMP_BUCKET);
assert_eq!(request.path, "upload/part.1");
self.barrier.wait().await;
if self.fail {
Err(DiskError::FileAccessDenied)
} else {
Ok(Box::new(tokio::io::sink()))
}
}
fn name(&self) -> &'static str {
"multipart-writer-test"
}
fn capabilities(&self) -> InternodeDataTransportCapabilities {
InternodeDataTransportCapabilities::tcp_http()
}
}
let barrier = Arc::new(tokio::sync::Barrier::new(3));
let mut disks = Vec::new();
for i in 0..3 {
let endpoint = Endpoint {
url: url::Url::parse(&format!("http://multipart-test.invalid:9000/disk{i}")).expect("endpoint"),
is_local: false,
pool_idx: 0,
set_idx: 0,
disk_idx: i,
};
let disk = RemoteDisk::new(
&endpoint,
&DiskOption {
cleanup: false,
health_check: false,
},
Arc::new(BarrierTransport {
barrier: Arc::clone(&barrier),
fail: i == 1,
}),
)
.await
.expect("remote disk");
disks.push(Some(Arc::new(Disk::Remote(Box::new(disk)))));
}
disks.insert(1, None);
// A serial opener cannot cross the barrier. The timeout only bounds failures;
// the assertion depends on all three independent openers making progress.
let (writers, errors) =
tokio::time::timeout(Duration::from_secs(10), create_part_writers(&disks, "upload/part.1", 1024, 256))
.await
.expect("all disk openers must be polled concurrently");
assert_eq!(writers.iter().map(Option::is_some).collect::<Vec<_>>(), [true, false, false, true]);
assert_eq!(errors, [None, Some(DiskError::DiskNotFound), Some(DiskError::FileAccessDenied), None]);
}
#[test]
fn multipart_bucket_incarnation_metadata_is_consistent_and_non_nil() {
let incarnation = Uuid::new_v4();
+343 -5
View File
@@ -694,14 +694,13 @@ impl ECStore {
let lock_newly_enabled = opts.lock_enabled && !meta.lock_enabled;
if opts.lock_enabled {
meta.lock_enabled = true;
meta.object_lock_config_xml =
crate::bucket::utils::serialize::<ObjectLockConfiguration>(&ENABLED_OBJECT_LOCK_CONFIG)?;
meta.object_lock_config_xml = crate::bucket::utils::serialize(&*crate::store::ENABLED_OBJECT_LOCK_CONFIG)?;
meta.versioning_config_xml = crate::bucket::utils::serialize::<VersioningConfiguration>(&ENABLED_VERSIONING_CONFIG)?;
}
let metadata_persisted_before_physical = confirmed_missing && !is_meta_bucketname(bucket) && opts.lock_enabled;
if metadata_persisted_before_physical {
metadata_sys::set_new_bucket_metadata_in(&self.ctx, meta.clone()).await?;
metadata_sys::set_new_bucket_metadata_intent_in(&self.ctx, meta.clone()).await?;
if bucket_lifecycle_guard.as_ref().is_some_and(|guard| guard.is_lock_lost())
|| metadata_transaction_guard.as_ref().is_some_and(|guard| guard.is_lock_lost())
|| ns_guard.as_ref().is_some_and(|guard| guard.is_lock_lost())
@@ -759,12 +758,12 @@ impl ECStore {
let metadata_result = async {
if metadata_persisted_before_physical {
return Ok(());
return metadata_sys::commit_bucket_metadata_in(&self.ctx, meta).await;
}
if is_meta_bucketname(bucket) {
metadata_sys::set_bucket_metadata_in(&self.ctx, meta).await
} else if existing_incarnation_is_authoritative && !lock_newly_enabled {
metadata_sys::cache_bucket_metadata_in(&self.ctx, meta).await
metadata_sys::commit_bucket_metadata_in(&self.ctx, meta).await
} else {
metadata_sys::set_new_bucket_metadata_in(&self.ctx, meta).await
}
@@ -910,6 +909,9 @@ impl ECStore {
// permission to serve degraded reads.
return Err(Error::ErasureReadQuorum);
}
if metadata.lock_enabled && !metadata.bucket_creation_committed {
return Err(Error::ErasureWriteQuorum);
}
if metadata.name != bucket {
return Err(Error::FileCorrupt);
}
@@ -2393,6 +2395,342 @@ mod tests {
}
}
#[tokio::test]
#[serial]
async fn lazy_metadata_load_uses_read_quorum_for_non_lock_bucket() {
let (_temp_dir, store) = setup_bucket_quorum_test_env(&[4], Some(2)).await;
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let bucket = format!("lazy-read-quorum-{}", Uuid::new_v4().simple());
store
.make_bucket(&bucket, &MakeBucketOptions::default())
.await
.expect("healthy namespace should accept bucket creation");
// Replace the instance metadata system with a cold cache while keeping the
// persisted metadata and physical bucket in place.
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let set = &store.pools[0].disk_set[0];
let offline = take_set_disks_offline(&store, set, &[0, 1]).await;
metadata_sys::get_on_demand_migration_config_in(&store, &bucket)
.await
.expect("cold metadata load should admit the exact namespace read quorum");
assert!(
metadata_sys::get_in(&store.ctx, &bucket).await.is_ok(),
"the lazy load should publish authoritative metadata"
);
assert_eq!(
store
.get_bucket_info_from_sets(&bucket, &BucketOptions::default())
.await
.expect_err("bucket mutations must retain their majority namespace check"),
StorageError::ErasureWriteQuorum
);
restore_set_disks(&store, set, offline).await;
}
#[tokio::test]
#[serial]
async fn lazy_metadata_load_rejects_below_read_quorum() {
let (_temp_dir, store) = setup_bucket_quorum_test_env(&[4], Some(2)).await;
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let bucket = format!("lazy-below-quorum-{}", Uuid::new_v4().simple());
store
.make_bucket(&bucket, &MakeBucketOptions::default())
.await
.expect("healthy namespace should accept bucket creation");
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let set = &store.pools[0].disk_set[0];
let offline = take_set_disks_offline(&store, set, &[0, 1, 2]).await;
let error = metadata_sys::get_on_demand_migration_config_in(&store, &bucket)
.await
.expect_err("cold metadata load must reject read quorum minus one");
assert!(
matches!(error, StorageError::ErasureReadQuorum | StorageError::InsufficientReadQuorum(_, _)),
"below read quorum must fail with a read-quorum error, got {error}"
);
assert!(
metadata_sys::get_in(&store.ctx, &bucket).await.is_err(),
"a below-quorum load must not publish metadata"
);
restore_set_disks(&store, set, offline).await;
}
#[tokio::test]
#[serial]
async fn lazy_metadata_load_keeps_lock_creation_intent_fail_closed() {
let (temp_dir, store) = setup_bucket_quorum_test_env(&[4], Some(2)).await;
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let bucket = format!("lazy-lock-intent-{}", Uuid::new_v4().simple());
let mut intent = BucketMetadata::new(&bucket);
intent.lock_enabled = true;
metadata_sys::set_new_bucket_metadata_intent_in(&store.ctx, intent)
.await
.expect("Object Lock creation intent should persist metadata and its incarnation sidecar");
let set = &store.pools[0].disk_set[0];
let offline = take_set_disks_offline(&store, set, &[0, 1]).await;
assert_eq!(
store
.make_bucket_on_sets(&bucket, &MakeBucketOptions::default())
.await
.expect_err("physical creation must not commit below write quorum"),
StorageError::ErasureWriteQuorum
);
assert!(!temp_dir.path().join("pool0-disk0").join(&bucket).exists());
assert!(!temp_dir.path().join("pool0-disk1").join(&bucket).exists());
assert!(temp_dir.path().join("pool0-disk2").join(&bucket).is_dir());
assert!(temp_dir.path().join("pool0-disk3").join(&bucket).is_dir());
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let error = metadata_sys::get_on_demand_migration_config_in(&store, &bucket)
.await
.expect_err("Object Lock intent must not become visible at read quorum");
assert_eq!(error, StorageError::ErasureWriteQuorum);
assert!(
metadata_sys::get_in(&store.ctx, &bucket).await.is_err(),
"the rejected Object Lock intent must not enter the metadata cache"
);
assert_eq!(
store
.get_bucket_info(&bucket, &BucketOptions::default())
.await
.expect_err("bucket-info read fallback must also reject a pending Object Lock intent"),
StorageError::ErasureWriteQuorum
);
restore_set_disks(&store, set, offline).await;
}
#[tokio::test]
#[serial]
async fn lazy_metadata_load_admits_committed_lock_bucket_at_read_quorum() {
let (_temp_dir, store) = setup_bucket_quorum_test_env(&[4], Some(2)).await;
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let bucket = format!("lazy-committed-lock-{}", Uuid::new_v4().simple());
store
.make_bucket(
&bucket,
&MakeBucketOptions {
lock_enabled: true,
..Default::default()
},
)
.await
.expect("healthy namespace should accept Object Lock bucket creation");
let (metadata, persisted) = metadata_sys::get_config_from_disk_with_presence_in(&store.ctx, &bucket)
.await
.expect("read committed Object Lock metadata");
assert!(persisted && metadata.bucket_creation_committed);
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let set = &store.pools[0].disk_set[0];
let offline = take_set_disks_offline(&store, set, &[0, 1]).await;
metadata_sys::get_on_demand_migration_config_in(&store, &bucket)
.await
.expect("committed Object Lock metadata should load at read quorum");
assert!(
metadata_sys::get_in(&store.ctx, &bucket).await.is_ok(),
"committed Object Lock metadata should be published"
);
restore_set_disks(&store, set, offline).await;
}
#[tokio::test]
#[serial]
async fn creation_commit_backfill_runs_under_the_config_write_fence() {
let (_temp_dir, store) = setup_bucket_quorum_test_env(&[4], Some(2)).await;
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let bucket = format!("lazy-commit-backfill-{}", Uuid::new_v4().simple());
let mut intent = BucketMetadata::new(&bucket);
intent.lock_enabled = true;
metadata_sys::set_new_bucket_metadata_intent_in(&store.ctx, intent)
.await
.expect("persist pending Object Lock intent");
store
.make_bucket_on_sets(&bucket, &MakeBucketOptions::default())
.await
.expect("physical creation should commit at full write quorum");
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
metadata_sys::get_object_lock_config_state_in(&store.ctx, &bucket)
.await
.expect("the fenced authority path should backfill the creation commit");
let (metadata, persisted) = metadata_sys::get_config_from_disk_with_presence_in(&store.ctx, &bucket)
.await
.expect("read backfilled creation commit");
assert!(persisted && metadata.bucket_creation_committed);
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let set = &store.pools[0].disk_set[0];
let offline = take_set_disks_offline(&store, set, &[0, 1]).await;
metadata_sys::get_on_demand_migration_config_in(&store, &bucket)
.await
.expect("backfilled Object Lock bucket should load at read quorum");
restore_set_disks(&store, set, offline).await;
}
#[tokio::test]
#[serial]
async fn enabling_object_lock_on_existing_bucket_establishes_commitment() {
let (_temp_dir, store) = setup_bucket_quorum_test_env(&[4], Some(2)).await;
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let bucket = format!("enable-lock-legacy-{}", Uuid::new_v4().simple());
store
.make_bucket(&bucket, &MakeBucketOptions::default())
.await
.expect("create a versioned non-lock bucket");
// Simulate a bucket persisted by a build before the commit field existed.
let mut legacy = metadata_sys::get_in(&store.ctx, &bucket)
.await
.expect("cached metadata")
.as_ref()
.clone();
legacy.lock_enabled = false;
legacy.object_lock_config_xml = Vec::new();
legacy.object_lock_config = None;
legacy.bucket_creation_committed = false;
metadata_sys::set_new_bucket_metadata_intent_in(&store.ctx, legacy)
.await
.expect("persist legacy-shaped metadata");
let lock_xml = crate::bucket::utils::serialize(&*crate::store::ENABLED_OBJECT_LOCK_CONFIG)
.expect("serialize Object Lock configuration");
metadata_sys::update_in(&store.ctx, &bucket, "object-lock.xml", lock_xml)
.await
.expect("enabling Object Lock on an existing bucket must succeed");
assert!(
matches!(
metadata_sys::get_object_lock_config_state_in(&store.ctx, &bucket)
.await
.expect("read the freshly enabled Object Lock state"),
metadata_sys::ObjectLockConfigState::Configured { .. }
),
"a healthy cluster must not report ErasureWriteQuorum after enabling Object Lock"
);
let (metadata, persisted) = metadata_sys::get_config_from_disk_with_presence_in(&store.ctx, &bucket)
.await
.expect("read persisted metadata");
assert!(persisted && metadata.bucket_creation_committed);
}
#[tokio::test]
#[serial]
async fn creation_commit_backfill_preserves_concurrent_config_update() {
let (_temp_dir, store) = setup_bucket_quorum_test_env(&[4], Some(2)).await;
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let bucket = format!("commit-backfill-pack-{}", Uuid::new_v4().simple());
store
.make_bucket(&bucket, &MakeBucketOptions::default())
.await
.expect("create bucket");
// The cached snapshot on the migrating node still predates another
// node's acknowledged configuration update.
let mut stale = metadata_sys::get_in(&store.ctx, &bucket)
.await
.expect("cached metadata")
.as_ref()
.clone();
stale.bucket_creation_committed = false;
metadata_sys::set_new_bucket_metadata_intent_in(&store.ctx, stale.clone())
.await
.expect("persist the stale pre-upgrade snapshot");
// A newer, still-uncommitted copy carrying an acknowledged Object Lock
// update lands on disk after the migrating node read its snapshot.
let mut newer = stale.clone();
newer.lock_enabled = true;
newer.object_lock_config_xml = crate::bucket::utils::serialize(&*crate::store::ENABLED_OBJECT_LOCK_CONFIG)
.expect("serialize Object Lock configuration");
newer
.save_with_store(store.clone())
.await
.expect("persist the newer configuration");
// A cold node must re-read the newest authoritative metadata under the
// fence instead of rewriting its stale snapshot.
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
metadata_sys::get_object_lock_config_state_in(&store.ctx, &bucket)
.await
.expect("fenced commit migration");
let (metadata, persisted) = metadata_sys::get_config_from_disk_with_presence_in(&store.ctx, &bucket)
.await
.expect("read persisted metadata");
assert!(persisted && metadata.bucket_creation_committed);
assert!(
metadata.lock_enabled && !metadata.object_lock_config_xml.is_empty(),
"the acknowledged Object Lock update must survive the backfill"
);
}
#[tokio::test]
#[serial]
async fn creation_commit_backfill_keeps_recreated_generation() {
let (_temp_dir, store) = setup_bucket_quorum_test_env(&[4], Some(2)).await;
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
let bucket = format!("commit-backfill-recreate-{}", Uuid::new_v4().simple());
store
.make_bucket(&bucket, &MakeBucketOptions::default())
.await
.expect("create bucket");
let old_incarnation = store.bucket_incarnation_id_from_disk(&bucket).await.expect("old incarnation");
// A node that still holds the pre-recreation snapshot must never let it
// outlive the replacement generation.
let mut stale = metadata_sys::get_in(&store.ctx, &bucket)
.await
.expect("cached metadata")
.as_ref()
.clone();
stale.lock_enabled = true;
stale.object_lock_config_xml = crate::bucket::utils::serialize(&*crate::store::ENABLED_OBJECT_LOCK_CONFIG)
.expect("serialize Object Lock configuration");
stale.bucket_creation_committed = false;
stale.save_with_store(store.clone()).await.expect("persist stale metadata");
store
.delete_bucket(&bucket, &crate::storage_api_contracts::bucket::DeleteBucketOptions::default())
.await
.expect("delete bucket");
store
.make_bucket(
&bucket,
&MakeBucketOptions {
lock_enabled: true,
..Default::default()
},
)
.await
.expect("recreate bucket with Object Lock");
let new_incarnation = store.bucket_incarnation_id_from_disk(&bucket).await.expect("new incarnation");
assert_ne!(old_incarnation, new_incarnation);
metadata_sys::init_bucket_metadata_sys(store.clone(), Vec::new()).await;
metadata_sys::get_object_lock_config_state_in(&store.ctx, &bucket)
.await
.expect("fenced backfill on the replacement generation");
let (metadata, persisted) = metadata_sys::get_config_from_disk_with_presence_in(&store.ctx, &bucket)
.await
.expect("read persisted metadata");
assert!(persisted);
assert_eq!(
metadata.bucket_incarnation_id, new_incarnation,
"the replacement generation must stay authoritative"
);
assert!(metadata.bucket_creation_committed);
}
#[tokio::test]
#[serial]
async fn bucket_info_read_quorum_is_scoped_to_each_erasure_set() {
+4 -5
View File
@@ -857,11 +857,10 @@ impl ECStore {
opts: &HealOpts,
allow_unversioned_absence: bool,
) -> Result<HealObjectStorageResult> {
if opts.dry_run
|| opts.no_lock
|| (!allow_unversioned_absence && version_id.is_empty())
|| super::utils::is_reserved_or_invalid_bucket(bucket, false)
{
// Object heal may recreate a missing bucket volume before committing a
// shard. Keep the current bucket generation fenced for the whole
// operation, including the ordinary unversioned path.
if opts.dry_run || opts.no_lock || super::utils::is_reserved_or_invalid_bucket(bucket, false) {
let (item, error) = self.handle_heal_object(bucket, object, version_id, opts).await?;
return Ok(HealObjectStorageResult {
item,
+4
View File
@@ -143,6 +143,8 @@ impl Error {
err,
EcstoreError::DiskNotFound
| EcstoreError::VolumeNotFound
| EcstoreError::FaultyDisk
| EcstoreError::FaultyRemoteDisk
| EcstoreError::SlowDown
| EcstoreError::OperationCanceled
| EcstoreError::RemoteClientUnavailable(_)
@@ -348,6 +350,8 @@ mod tests {
assert!(Error::Disk(DiskError::DiskNotFound).is_recoverable_heal());
assert!(Error::Storage(EcstoreError::DiskNotFound).is_recoverable_heal());
assert!(Error::Storage(EcstoreError::VolumeNotFound).is_recoverable_heal());
assert!(Error::Storage(EcstoreError::FaultyDisk).is_recoverable_heal());
assert!(Error::Storage(EcstoreError::FaultyRemoteDisk).is_recoverable_heal());
}
#[test]

Some files were not shown because too many files have changed in this diff Show More