mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 16:28:15 +00:00

Files

T

History

houseme ea2e24ac13 test/ci(ecstore): fix MinIO SSE interop size assertion + nightly dockerized interop check (#4809 )

* test(ecstore): assert decrypted_size for MinIO SSE interop round-trip

The ignored MinIO interop round-trip tests asserted `ObjectInfo.size`
against the plaintext length. For SSE objects `size` is the on-disk
DARE-encrypted size (plaintext + 32 bytes per 64 KiB block), so the
assertion can never hold once real fixtures are present — the two
`#[ignore]` tests failed the moment a real MinIO-written fixture was fed
in, even though the decoded data was byte-identical.

The client-visible object size comes from `decrypted_size()` /
`get_actual_size()`, which correctly reads MinIO's
`x-*-internal-actual-size` metadata (verified: both SSE-S3 and SSE-KMS
8 MiB multipart fixtures now report 8388608). Assert against that
instead and keep the plaintext length and SHA-256 data checks.

With real 4-drive MinIO fixtures (RELEASE.2025-09-07) all four tests
pass, confirming RustFS reads MinIO erasure-coded SSE objects with
byte-identical data and correct logical size.

Co-Authored-By: heihutu <heihutu@gmail.com>

* ci(ecstore): nightly MinIO interop check + dockerized fixture capture

Wire the ignored MinIO on-disk interop reader tests into a nightly,
non-required CI job, and make their fixtures reproducible without a host
MinIO install.

- Dockerfile + capture_via_docker.sh: build a throwaway image carrying
  the official MinIO server binary (pinned RELEASE.2025-09-07) plus the
  fixture lab on a small Python base, then run `lab.py capture-matrix` to
  write the SSE-S3 / SSE-KMS multipart fixtures the tests consume. lab.py
  drives MinIO's S3 API directly, so no `mc` is needed.
- .github/workflows/minio-interop.yml: nightly + manual workflow on
  GitHub-hosted ubuntu-latest (reliable Docker + Python, unlike the
  self-hosted fleet — see e2e-s3tests.yml infra note). Regenerates the
  gitignored fixtures each run and executes the #[ignore] reader tests.
  Not a PR gate.
- README: document the Docker capture path.

Validated end to end: the script builds the image, captures the two
multipart cases, and `cargo nextest run --run-ignored ignored-only`
passes all four interop tests.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>

2026-07-14 15:08:14 +00:00

fixtures

feat(rio): rio_v2 is compatible with minio for storing data. (#3115 )

2026-06-08 11:59:14 +00:00

minio_fixture_lab

test/ci(ecstore): fix MinIO SSE interop size assertion + nightly dockerized interop check (#4809 )

2026-07-14 15:08:14 +00:00

minio_generated_fixtures.rs

feat(rio): rio_v2 is compatible with minio for storing data. (#3115 )

2026-06-08 11:59:14 +00:00

README.md

build: upgrade Rust baseline to 1.96.0 (#3291 )

2026-06-08 20:32:37 +00:00

README.md

rio-v2 Fixture Tests

This directory holds interoperability tests for rustfs-rio-v2.

Generated MinIO fixtures

The integration test minio_generated_fixtures.rs reads raw MinIO backend data generated by crates/rio-v2/tests/minio_fixture_lab/lab.py.

Default fixture root:

crates/rio-v2/tests/fixtures/minio-generated

The generated MinIO fixture data is intentionally not checked in and these tests are ignored by default, so they are not part of automated CI runs.

To generate or expand fixtures locally, use:

uv run python .\minio_fixture_lab\lab.py capture-matrix `
  --root .\rustfs\crates\rio-v2\tests\fixtures\minio-generated `
  --minio-binary .\rustfs\tmp\minio.darwin-arm64.RELEASE.2025-09-07T16-13-09Z `
  --endpoint https://127.0.0.1:19000 `
  --disk-count 1

Or point the tests at another generated root:

$env:RUSTFS_MINIO_FIXTURE_ROOT = '.\rustfs\tmp\minio-fixture-lab-smoke'
cargo +1.96.0 test -p rustfs-rio-v2 --test minio_generated_fixtures -- --ignored

Scope

The current fixture test covers the full default capture matrix:

sse-s3-singlepart-64k
sse-s3-multipart-8m
sse-kms-singlepart-64k
sse-kms-multipart-8m
sse-c-singlepart-64k
sse-c-multipart-8m

It checks that:

raw xl.meta can be parsed through rustfs-filemeta
singlepart and multipart fixture structures are recognized
SSE-S3, SSE-KMS, and SSE-C metadata markers survive capture
KMS key ids are derived from each fixture's own manifest.json, so local static-KMS runs are not tied to one hard-coded key name
SSE-C HEAD responses round-trip the expected customer algorithm and customer-key MD5

These tests do not yet validate full plaintext reconstruction from MinIO-written encrypted data.