mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-09 06:39:25 +00:00
houseme
ea2e24ac13
* test(ecstore): assert decrypted_size for MinIO SSE interop round-trip The ignored MinIO interop round-trip tests asserted `ObjectInfo.size` against the plaintext length. For SSE objects `size` is the on-disk DARE-encrypted size (plaintext + 32 bytes per 64 KiB block), so the assertion can never hold once real fixtures are present — the two `#[ignore]` tests failed the moment a real MinIO-written fixture was fed in, even though the decoded data was byte-identical. The client-visible object size comes from `decrypted_size()` / `get_actual_size()`, which correctly reads MinIO's `x-*-internal-actual-size` metadata (verified: both SSE-S3 and SSE-KMS 8 MiB multipart fixtures now report 8388608). Assert against that instead and keep the plaintext length and SHA-256 data checks. With real 4-drive MinIO fixtures (RELEASE.2025-09-07) all four tests pass, confirming RustFS reads MinIO erasure-coded SSE objects with byte-identical data and correct logical size. Co-Authored-By: heihutu <heihutu@gmail.com> * ci(ecstore): nightly MinIO interop check + dockerized fixture capture Wire the ignored MinIO on-disk interop reader tests into a nightly, non-required CI job, and make their fixtures reproducible without a host MinIO install. - Dockerfile + capture_via_docker.sh: build a throwaway image carrying the official MinIO server binary (pinned RELEASE.2025-09-07) plus the fixture lab on a small Python base, then run `lab.py capture-matrix` to write the SSE-S3 / SSE-KMS multipart fixtures the tests consume. lab.py drives MinIO's S3 API directly, so no `mc` is needed. - .github/workflows/minio-interop.yml: nightly + manual workflow on GitHub-hosted ubuntu-latest (reliable Docker + Python, unlike the self-hosted fleet — see e2e-s3tests.yml infra note). Regenerates the gitignored fixtures each run and executes the #[ignore] reader tests. Not a PR gate. - README: document the Docker capture path. Validated end to end: the script builds the image, captures the two multipart cases, and `cargo nextest run --run-ignored ignored-only` passes all four interop tests. Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com>
49 lines
2.0 KiB
Bash
Executable File
49 lines
2.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Generate real MinIO on-disk fixtures via Docker — no host `minio`/`mc` install.
|
|
#
|
|
# Builds a throwaway image carrying the official MinIO server binary plus the
|
|
# fixture lab, runs `lab.py capture-matrix` inside it, and writes the captured
|
|
# backend fixtures under crates/rio-v2/tests/fixtures/minio-generated (which is
|
|
# gitignored — regenerate as needed). The same script is used locally and by the
|
|
# nightly `minio-interop` CI workflow so both paths stay in sync.
|
|
#
|
|
# Usage:
|
|
# ./capture_via_docker.sh # the two multipart cases the
|
|
# # ignored interop tests consume
|
|
# ./capture_via_docker.sh sse-s3-singlepart-64k # specific case id(s)
|
|
# ./capture_via_docker.sh all # full SSE/size matrix
|
|
set -euo pipefail
|
|
|
|
IMAGE="${MINIO_LAB_IMAGE:-rustfs-minio-lab:latest}"
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
# repo root is four levels up: crates/rio-v2/tests/minio_fixture_lab -> repo
|
|
REPO_ROOT="$(cd "$SCRIPT_DIR/../../../.." && pwd)"
|
|
LAB_REL="crates/rio-v2/tests/minio_fixture_lab"
|
|
FIXTURE_REL="crates/rio-v2/tests/fixtures/minio-generated"
|
|
|
|
# Default to the two multipart cases the ignored round-trip tests consume; pass
|
|
# case ids to override, or "all" for the full default matrix.
|
|
cases=("$@")
|
|
if [ "${#cases[@]}" -eq 0 ]; then
|
|
cases=(sse-s3-multipart-8m sse-kms-multipart-8m)
|
|
fi
|
|
case_args=()
|
|
if [ "${cases[0]}" != "all" ]; then
|
|
for c in "${cases[@]}"; do
|
|
case_args+=(--case-id "$c")
|
|
done
|
|
fi
|
|
|
|
echo ">> building ${IMAGE}"
|
|
docker build -f "${SCRIPT_DIR}/Dockerfile" -t "${IMAGE}" "${SCRIPT_DIR}"
|
|
|
|
echo ">> capturing fixtures into ${FIXTURE_REL}"
|
|
docker run --rm -v "${REPO_ROOT}:/repo" "${IMAGE}" \
|
|
python3 "/repo/${LAB_REL}/lab.py" capture-matrix \
|
|
--root "/repo/${FIXTURE_REL}" \
|
|
--work-root /tmp/minio-lab-work \
|
|
--minio-binary /usr/local/bin/minio \
|
|
"${case_args[@]}"
|
|
|
|
echo ">> done — fixtures under ${REPO_ROOT}/${FIXTURE_REL}/cases/"
|