gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-06 03:59:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

test(odm): add the scheduled provider interop lane (#7114)

Browse Source 
* test(odm): drive the migration cases from an env-named source

The ODM e2e suite only ever migrates from the in-process fake source, so
path-style addressing, region handling, ETag shape and list pagination on
real implementations stay untested. OdmInteropEnv resolves the source from
RUSTFS_ODM_INTEROP_*, seeding into a per-run source_prefix so a shared real
bucket can host concurrent runs and every seeded key is removed afterwards.
A named provider with a missing variable is an error, never a silent
fallback to the fake source.

interop_test holds the four cases that run against either source, and the
e2e-odm-interop profile is the lane that selects them; e2e-full excludes
them, so its committed selection is unchanged. wait_until_odm_engaged
replaces the fake source's journal probe for the readiness wait, since a
real source keeps no journal.

* ci(odm): add the scheduled provider interop lane

on-demand-migration-interop.yml runs the interop cases against a pinned
MinIO container with a 5,000-object backfill - past the fake source's 4,096
version and journal caps - and the three-case minimum against AWS, R2 and
GCS when their ODM_INTEROP_* secrets exist, skipping with a summary note
when they do not. Each provider gets one JSON report merging the per-case
entries with the nextest JUnit, which stays authoritative for what ran.

Report-only and never required: it depends on third-party endpoints and on
secrets a fork does not have.
This commit is contained in:
Zhengchao An
2026-09-04 10:05:40 +08:00
committed by GitHub
parent 9863f4848d
commit 9d10d69a6d
9 changed files with 1138 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operations/on-demand-migration.md
+10 -8
View File
@@ -125,17 +125,19 @@ Validation also rejects two shapes outright: a source whose endpoint and bucket
## Provider presets and source permissions
| Provider | Endpoint | Addressing | `region` | Notes |
|---|---|---|---|---|
| `aws` | Optional; derived as `https://s3.<region>.amazonaws.com` | Virtual-host | Real region required (`auto` rejected) | The derived form only accepts `[A-Za-z0-9-]` in `region` |
| `s3` | Required | Path-style | Real region | Generic S3-compatible endpoint (Wasabi, Backblaze B2 S3 API, Ceph RGW, …) |
| `minio` | Required | Path-style | `auto` allowed | |
| `rustfs` | Required | Path-style | `auto` allowed | A RustFS source answers the migration request locally thanks to the anti-loop marker |
| `r2` | `https://<account-id>.r2.cloudflarestorage.com` | Virtual-host | `auto` allowed (signed as `us-east-1`) | |
| `gcs` | `https://storage.googleapis.com` | Virtual-host | Real region required | Uses the GCS XML interoperability API with an HMAC key pair, not a service-account JSON key |
| Provider | Endpoint | Addressing | `region` | Notes | Interop evidence |
|---|---|---|---|---|---|
| `aws` | Optional; derived as `https://s3.<region>.amazonaws.com` | Virtual-host | Real region required (`auto` rejected) | The derived form only accepts `[A-Za-z0-9-]` in `region` | `cloud-source (aws)`, only while `ODM_INTEROP_AWS_*` are configured; no difference recorded yet |
| `s3` | Required | Path-style | Real region | Generic S3-compatible endpoint (Wasabi, Backblaze B2 S3 API, Ceph RGW, …) | No lane of its own; the preset is the same code path the `minio` job exercises |
| `minio` | Required | Path-style | `auto` allowed | | `minio-source`, nightly: read-through, HEAD passthrough, merged list pagination and a 5,000-object backfill; no difference recorded yet |
| `rustfs` | Required | Path-style | `auto` allowed | A RustFS source answers the migration request locally thanks to the anti-loop marker | `real_source_test.rs` in the `e2e-nightly` lane |
| `r2` | `https://<account-id>.r2.cloudflarestorage.com` | Virtual-host | `auto` allowed (signed as `us-east-1`) | | `cloud-source (r2)`, only while `ODM_INTEROP_R2_*` are configured; no difference recorded yet |
| `gcs` | `https://storage.googleapis.com` | Virtual-host | Real region required | Uses the GCS XML interoperability API with an HMAC key pair, not a service-account JSON key | `cloud-source (gcs)`, only while `ODM_INTEROP_GCS_HMAC_*` are configured; no difference recorded yet |
Azure Blob has no preset; a native provider is deferred (rustfs/backlog#2166).
The "Interop evidence" column names the job in `.github/workflows/on-demand-migration-interop.yml` (rustfs/backlog#2167) that last exercised the preset against a real implementation, and is where a provider difference belongs once the lane finds one. That lane is report-only and scheduled: it runs `crates/e2e_test/src/on_demand_migration/interop_test.rs` — the same case bodies as the merge-gate suite, with the source injected through `RUSTFS_ODM_INTEROP_*` — against a pinned MinIO container, and against each cloud provider whose repository secrets are configured. A provider without secrets is skipped with a note in the run summary rather than failing, so "no difference recorded yet" means exactly that and not "verified clean"; see [ci-gates.md](../testing/ci-gates.md) for the row.
The credentials only ever need read access to the source bucket:
- `s3:ListBucket` on the bucket — used by the admin probe, by the backfill listing, and by every merged listing under `policy.list_through`.
docs/testing/ci-gates.md
+1
View File
@@ -77,6 +77,7 @@ Scheduled lanes never block a PR. Their workflow-local gate fails the run, sched
| `e2e-s3tests.yml` (weekly) | `s3tests` (single and distributed, four shards each), `upstream-head-canary` | compatibility gate; report, JUnit, node IDs, server logs | yes | `scripts/s3-tests/run.sh` against an existing single or distributed target |
| `fuzz.yml` (nightly) | `nightly-fuzz-corpus` per target | gate; corpus and crash artifacts | yes | `MAX_TOTAL_TIME=<seconds> ./scripts/fuzz/run.sh` |
| `minio-interop.yml` (nightly) | `minio-interop` | EC + SSE read-parity gate | yes, with `never_ran_grace_until` | pinned Docker fixture steps in the workflow |
| `on-demand-migration-interop.yml` (nightly) | `minio-source`, `cloud-source` (`aws`, `r2`, `gcs`) | report-only provider interop; one JSON report per provider naming cases, timings and source request counts, plus JUnit and MinIO logs. A cloud provider whose `ODM_INTEROP_*` secrets are absent is skipped with a summary note, not failed | no | start the pinned MinIO container as in the job, export the `RUSTFS_ODM_INTEROP_*` variables, then `cargo nextest run --profile e2e-odm-interop -p e2e_test` |
| `performance-ab.yml` (nightly) | `warp-ab` | regression-budget gate; A/B summaries and server logs | yes | `bash scripts/run_hotpath_warp_abba.sh --help` |
| `nightly-gnu.yml` (nightly) | `build`, `kms-vault-lane`, `kms-vault-ha-failover` | build, live Vault, and HA failover gates | yes | commands and pinned Vault images in the workflow |
| `audit.yml` (nightly) | `cargo-deny`, `workflow-pin-report` | dependency and workflow-pin gates | yes | `cargo deny check`; `scripts/security/check_workflow_pins.sh` |