mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 08:18:18 +00:00
houseme
ea2e24ac13
* test(ecstore): assert decrypted_size for MinIO SSE interop round-trip The ignored MinIO interop round-trip tests asserted `ObjectInfo.size` against the plaintext length. For SSE objects `size` is the on-disk DARE-encrypted size (plaintext + 32 bytes per 64 KiB block), so the assertion can never hold once real fixtures are present — the two `#[ignore]` tests failed the moment a real MinIO-written fixture was fed in, even though the decoded data was byte-identical. The client-visible object size comes from `decrypted_size()` / `get_actual_size()`, which correctly reads MinIO's `x-*-internal-actual-size` metadata (verified: both SSE-S3 and SSE-KMS 8 MiB multipart fixtures now report 8388608). Assert against that instead and keep the plaintext length and SHA-256 data checks. With real 4-drive MinIO fixtures (RELEASE.2025-09-07) all four tests pass, confirming RustFS reads MinIO erasure-coded SSE objects with byte-identical data and correct logical size. Co-Authored-By: heihutu <heihutu@gmail.com> * ci(ecstore): nightly MinIO interop check + dockerized fixture capture Wire the ignored MinIO on-disk interop reader tests into a nightly, non-required CI job, and make their fixtures reproducible without a host MinIO install. - Dockerfile + capture_via_docker.sh: build a throwaway image carrying the official MinIO server binary (pinned RELEASE.2025-09-07) plus the fixture lab on a small Python base, then run `lab.py capture-matrix` to write the SSE-S3 / SSE-KMS multipart fixtures the tests consume. lab.py drives MinIO's S3 API directly, so no `mc` is needed. - .github/workflows/minio-interop.yml: nightly + manual workflow on GitHub-hosted ubuntu-latest (reliable Docker + Python, unlike the self-hosted fleet — see e2e-s3tests.yml infra note). Regenerates the gitignored fixtures each run and executes the #[ignore] reader tests. Not a PR gate. - README: document the Docker capture path. Validated end to end: the script builds the image, captures the two multipart cases, and `cargo nextest run --run-ignored ignored-only` passes all four interop tests. Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com>
71 lines
2.6 KiB
YAML
71 lines
2.6 KiB
YAML
# Copyright 2024 RustFS Team
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
# MinIO on-disk interop: prove RustFS reads MinIO-written erasure-coded SSE
|
|
# objects with byte-identical data and correct logical size.
|
|
#
|
|
# This is NOT a PR gate. The fixtures are real MinIO backend trees generated on
|
|
# the fly (they are gitignored, never committed), so the job regenerates them
|
|
# each run with Docker and then runs the `#[ignore]` reader tests in
|
|
# crates/ecstore/tests/minio_generated_read_test.rs.
|
|
#
|
|
# Runner: GitHub-hosted `ubuntu-latest`. It reliably ships Docker + Python,
|
|
# unlike the self-hosted fleet, whose pods drift in Docker/pip availability
|
|
# (see the infra note in e2e-s3tests.yml). Nightly + manual only.
|
|
name: minio-interop
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
schedule:
|
|
# Nightly at 03:17 UTC (offset from other nightly jobs).
|
|
- cron: "17 3 * * *"
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
minio-interop:
|
|
name: MinIO interop (EC + SSE read parity)
|
|
# Skip on forks: needs the repo's runners and is not a contributor gate.
|
|
if: github.repository == 'rustfs/rustfs'
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 40
|
|
env:
|
|
# Fixed 32-byte test KMS key baked into the fixture lab; not a secret.
|
|
RUSTFS_MINIO_STATIC_KMS_KEY_B64: IyqsU3kMFloCNup4BsZtf/rmfHVcTgznO2F25CkEH1g=
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
|
|
|
|
- name: Setup Rust environment
|
|
uses: ./.github/actions/setup
|
|
with:
|
|
rust-version: stable
|
|
cache-shared-key: ci-minio-interop
|
|
github-token: ${{ secrets.GITHUB_TOKEN }}
|
|
cache-save-if: ${{ github.ref == 'refs/heads/main' }}
|
|
|
|
- name: Generate real MinIO fixtures via Docker
|
|
run: bash crates/rio-v2/tests/minio_fixture_lab/capture_via_docker.sh
|
|
|
|
- name: Run MinIO interop reader tests
|
|
run: |
|
|
cargo nextest run --run-ignored ignored-only \
|
|
-p rustfs-ecstore --features rio-v2 \
|
|
-E 'binary(minio_generated_read_test)'
|