Compare commits

...

2 Commits

Author SHA1 Message Date
overtrue 2aa4e411e5 test(interop): fix the fixture lab's all-cases and SSE-C capture paths
Two defects kept most of the fixture matrix uncapturable, both found by actually running the lab rather than reading it.

`./capture_via_docker.sh all` never worked. With no case ids to forward, `case_args` is an empty array, and `${arr[@]}` under `set -u` aborts on bash 3.2 — still the default /bin/bash on macOS — with "case_args[@]: unbound variable". The build_args array this branch added has the same shape and would have hit it on the CI path, where no mirror overrides are set. Both now use ${arr[@]+"${arr[@]}"}, which is empty-safe on 3.2.

The SSE-C cases could not be captured at all: MinIO refuses SSE-C over a plain-HTTP connection, and the script pinned the lab to its default http endpoint with no way to override. lab.py already provisions a self-signed certificate and speaks https end to end, so this only needed an endpoint passthrough — MINIO_LAB_ENDPOINT, documented alongside the requirement.

With both fixed, one command captures the full six-case matrix (SSE-S3 / SSE-KMS / SSE-C x singlepart / multipart), which is what backlog#1638's D4 matrix has to be measured against.

Refs rustfs/backlog#1638.
2026-08-16 22:25:27 +08:00
overtrue 5571d4830b test(interop): let the MinIO fixture lab build from registry mirrors
The fixture lab's throwaway image hardcoded two Docker Hub base images, so on a network that cannot reach registry-1.docker.io the capture script fails at build time and no MinIO fixtures can be generated at all. That matters more than it looks: the interop reader tests are #[ignore]d precisely because the fixtures are gitignored and must be generated locally, so an unreachable registry silently keeps the whole MinIO SSE interop lane unverifiable.

Both base images become build args with the current Docker Hub values as defaults, and the capture script forwards MINIO_LAB_MINIO_IMAGE / MINIO_LAB_PYTHON_IMAGE when set. CI keeps the defaults; a restricted network can point at quay.io (which publishes the MinIO releases) and public.ecr.aws (which mirrors the official Python images), both verified to produce a working lab image and real fixtures.

The README documents the override and warns against an unpinned :latest MinIO tag, since the captured on-disk format is exactly what the interop tests assert against.

Refs rustfs/backlog#1638.
2026-08-16 21:23:17 +08:00
3 changed files with 55 additions and 4 deletions
@@ -6,9 +6,18 @@
#
# The MinIO release is pinned so the captured fixture format is reproducible;
# this is the release the interop tests were validated against.
FROM minio/minio:RELEASE.2025-09-07T16-13-09Z AS minio
#
# Both base images are build args so a network that cannot reach Docker Hub can
# point them at a mirror carrying the same content — quay.io publishes the MinIO
# releases, and public.ecr.aws mirrors the official Python images. CI keeps the
# Docker Hub defaults. Override with:
# --build-arg MINIO_IMAGE=quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z \
# --build-arg PYTHON_IMAGE=public.ecr.aws/docker/library/python:3.12-slim
ARG MINIO_IMAGE=minio/minio:RELEASE.2025-09-07T16-13-09Z
ARG PYTHON_IMAGE=python:3.12-slim
FROM ${MINIO_IMAGE} AS minio
FROM python:3.12-slim
FROM ${PYTHON_IMAGE}
RUN apt-get update \
&& apt-get install -y --no-install-recommends openssl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
@@ -22,6 +22,26 @@ Use the automated path when you want the lab to:
- upload a predefined SSE fixture case
- export the generated backend tree into the lab layout
## Networks without Docker Hub access
`capture_via_docker.sh` pulls its two base images from Docker Hub by default. Where that registry is unreachable, point the build at mirrors carrying the same content — quay.io publishes the MinIO releases and public.ecr.aws mirrors the official Python images:
```bash
MINIO_LAB_MINIO_IMAGE=quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z \
MINIO_LAB_PYTHON_IMAGE=public.ecr.aws/docker/library/python:3.12-slim \
./capture_via_docker.sh
```
Pin the MinIO tag to the same release the Dockerfile names; an unpinned `:latest` captures whatever format that day's build writes, which is not what the interop tests were validated against.
## Capturing the SSE-C cases
MinIO refuses SSE-C over a plain-HTTP connection, so the `sse-c-*` cases cannot be captured against the default endpoint — `./capture_via_docker.sh all` fails on the first SSE-C upload with `InvalidRequest ... must be made over a secure connection`. The lab provisions its own self-signed certificate; point it at the HTTPS endpoint to capture them:
```bash
MINIO_LAB_ENDPOINT=https://127.0.0.1:9000 ./capture_via_docker.sh all
```
## Layout
The default root is `artifacts/minio-fixture-lab`, which is already ignored by the repository.
@@ -12,6 +12,13 @@
# # ignored interop tests consume
# ./capture_via_docker.sh sse-s3-singlepart-64k # specific case id(s)
# ./capture_via_docker.sh all # full SSE/size matrix
#
# The SSE-C cases are not reachable over the default plain-HTTP endpoint: MinIO
# refuses SSE-C unless the connection is secure ("Requests specifying Server
# Side Encryption with Customer provided keys must be made over a secure
# connection"). Capture those by pointing the lab at its self-signed HTTPS
# endpoint, which it provisions itself:
# MINIO_LAB_ENDPOINT=https://127.0.0.1:9000 ./capture_via_docker.sh all
set -euo pipefail
IMAGE="${MINIO_LAB_IMAGE:-rustfs-minio-lab:latest}"
@@ -34,8 +41,22 @@ if [ "${cases[0]}" != "all" ]; then
done
fi
# Base images are overridable so a network without Docker Hub access can point
# them at a mirror (see the Dockerfile header). Unset by default, which keeps the
# Dockerfile's Docker Hub defaults for CI.
build_args=()
if [ -n "${MINIO_LAB_MINIO_IMAGE:-}" ]; then
build_args+=(--build-arg "MINIO_IMAGE=${MINIO_LAB_MINIO_IMAGE}")
fi
if [ -n "${MINIO_LAB_PYTHON_IMAGE:-}" ]; then
build_args+=(--build-arg "PYTHON_IMAGE=${MINIO_LAB_PYTHON_IMAGE}")
fi
echo ">> building ${IMAGE}"
docker build -f "${SCRIPT_DIR}/Dockerfile" -t "${IMAGE}" "${SCRIPT_DIR}"
# ${arr[@]+"${arr[@]}"} rather than "${arr[@]}": under `set -u`, bash 3.2 —
# still the default /bin/bash on macOS — treats an empty array expansion as an
# unbound variable and aborts.
docker build -f "${SCRIPT_DIR}/Dockerfile" -t "${IMAGE}" ${build_args[@]+"${build_args[@]}"} "${SCRIPT_DIR}"
echo ">> capturing fixtures into ${FIXTURE_REL}"
docker run --rm -v "${REPO_ROOT}:/repo" "${IMAGE}" \
@@ -43,6 +64,7 @@ docker run --rm -v "${REPO_ROOT}:/repo" "${IMAGE}" \
--root "/repo/${FIXTURE_REL}" \
--work-root /tmp/minio-lab-work \
--minio-binary /usr/local/bin/minio \
"${case_args[@]}"
--endpoint "${MINIO_LAB_ENDPOINT:-http://127.0.0.1:9000}" \
${case_args[@]+"${case_args[@]}"}
echo ">> done — fixtures under ${REPO_ROOT}/${FIXTURE_REL}/cases/"