mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 08:18:18 +00:00
test/ci(ecstore): fix MinIO SSE interop size assertion + nightly dockerized interop check (#4809)
* test(ecstore): assert decrypted_size for MinIO SSE interop round-trip The ignored MinIO interop round-trip tests asserted `ObjectInfo.size` against the plaintext length. For SSE objects `size` is the on-disk DARE-encrypted size (plaintext + 32 bytes per 64 KiB block), so the assertion can never hold once real fixtures are present — the two `#[ignore]` tests failed the moment a real MinIO-written fixture was fed in, even though the decoded data was byte-identical. The client-visible object size comes from `decrypted_size()` / `get_actual_size()`, which correctly reads MinIO's `x-*-internal-actual-size` metadata (verified: both SSE-S3 and SSE-KMS 8 MiB multipart fixtures now report 8388608). Assert against that instead and keep the plaintext length and SHA-256 data checks. With real 4-drive MinIO fixtures (RELEASE.2025-09-07) all four tests pass, confirming RustFS reads MinIO erasure-coded SSE objects with byte-identical data and correct logical size. Co-Authored-By: heihutu <heihutu@gmail.com> * ci(ecstore): nightly MinIO interop check + dockerized fixture capture Wire the ignored MinIO on-disk interop reader tests into a nightly, non-required CI job, and make their fixtures reproducible without a host MinIO install. - Dockerfile + capture_via_docker.sh: build a throwaway image carrying the official MinIO server binary (pinned RELEASE.2025-09-07) plus the fixture lab on a small Python base, then run `lab.py capture-matrix` to write the SSE-S3 / SSE-KMS multipart fixtures the tests consume. lab.py drives MinIO's S3 API directly, so no `mc` is needed. - .github/workflows/minio-interop.yml: nightly + manual workflow on GitHub-hosted ubuntu-latest (reliable Docker + Python, unlike the self-hosted fleet — see e2e-s3tests.yml infra note). Regenerates the gitignored fixtures each run and executes the #[ignore] reader tests. Not a PR gate. - README: document the Docker capture path. Validated end to end: the script builds the image, captures the two multipart cases, and `cargo nextest run --run-ignored ignored-only` passes all four interop tests. Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
# Copyright 2024 RustFS Team
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# MinIO on-disk interop: prove RustFS reads MinIO-written erasure-coded SSE
|
||||
# objects with byte-identical data and correct logical size.
|
||||
#
|
||||
# This is NOT a PR gate. The fixtures are real MinIO backend trees generated on
|
||||
# the fly (they are gitignored, never committed), so the job regenerates them
|
||||
# each run with Docker and then runs the `#[ignore]` reader tests in
|
||||
# crates/ecstore/tests/minio_generated_read_test.rs.
|
||||
#
|
||||
# Runner: GitHub-hosted `ubuntu-latest`. It reliably ships Docker + Python,
|
||||
# unlike the self-hosted fleet, whose pods drift in Docker/pip availability
|
||||
# (see the infra note in e2e-s3tests.yml). Nightly + manual only.
|
||||
name: minio-interop
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
schedule:
|
||||
# Nightly at 03:17 UTC (offset from other nightly jobs).
|
||||
- cron: "17 3 * * *"
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
minio-interop:
|
||||
name: MinIO interop (EC + SSE read parity)
|
||||
# Skip on forks: needs the repo's runners and is not a contributor gate.
|
||||
if: github.repository == 'rustfs/rustfs'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 40
|
||||
env:
|
||||
# Fixed 32-byte test KMS key baked into the fixture lab; not a secret.
|
||||
RUSTFS_MINIO_STATIC_KMS_KEY_B64: IyqsU3kMFloCNup4BsZtf/rmfHVcTgznO2F25CkEH1g=
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
|
||||
|
||||
- name: Setup Rust environment
|
||||
uses: ./.github/actions/setup
|
||||
with:
|
||||
rust-version: stable
|
||||
cache-shared-key: ci-minio-interop
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
cache-save-if: ${{ github.ref == 'refs/heads/main' }}
|
||||
|
||||
- name: Generate real MinIO fixtures via Docker
|
||||
run: bash crates/rio-v2/tests/minio_fixture_lab/capture_via_docker.sh
|
||||
|
||||
- name: Run MinIO interop reader tests
|
||||
run: |
|
||||
cargo nextest run --run-ignored ignored-only \
|
||||
-p rustfs-ecstore --features rio-v2 \
|
||||
-E 'binary(minio_generated_read_test)'
|
||||
Reference in New Issue
Block a user