Two defects kept most of the fixture matrix uncapturable, both found by actually running the lab rather than reading it.
`./capture_via_docker.sh all` never worked. With no case ids to forward, `case_args` is an empty array, and `${arr[@]}` under `set -u` aborts on bash 3.2 — still the default /bin/bash on macOS — with "case_args[@]: unbound variable". The build_args array this branch added has the same shape and would have hit it on the CI path, where no mirror overrides are set. Both now use ${arr[@]+"${arr[@]}"}, which is empty-safe on 3.2.
The SSE-C cases could not be captured at all: MinIO refuses SSE-C over a plain-HTTP connection, and the script pinned the lab to its default http endpoint with no way to override. lab.py already provisions a self-signed certificate and speaks https end to end, so this only needed an endpoint passthrough — MINIO_LAB_ENDPOINT, documented alongside the requirement.
With both fixed, one command captures the full six-case matrix (SSE-S3 / SSE-KMS / SSE-C x singlepart / multipart), which is what backlog#1638's D4 matrix has to be measured against.
Refs rustfs/backlog#1638.
The fixture lab's throwaway image hardcoded two Docker Hub base images, so on a network that cannot reach registry-1.docker.io the capture script fails at build time and no MinIO fixtures can be generated at all. That matters more than it looks: the interop reader tests are #[ignore]d precisely because the fixtures are gitignored and must be generated locally, so an unreachable registry silently keeps the whole MinIO SSE interop lane unverifiable.
Both base images become build args with the current Docker Hub values as defaults, and the capture script forwards MINIO_LAB_MINIO_IMAGE / MINIO_LAB_PYTHON_IMAGE when set. CI keeps the defaults; a restricted network can point at quay.io (which publishes the MinIO releases) and public.ecr.aws (which mirrors the official Python images), both verified to produce a working lab image and real fixtures.
The README documents the override and warns against an unpinned :latest MinIO tag, since the captured on-disk format is exactly what the interop tests assert against.
Refs rustfs/backlog#1638.
Add opt-in hotpath feature surfaces to every workspace crate and wire the root rustfs feature passthrough for function, allocation, and CPU profiling.
Add a focused set of function-level measurements for scanner, heal, lock, target replay, IAM, KMS, Keystone, trusted proxy, and capacity paths without adding request-scoped primitive wrappers.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(sse): decouple encryption from ecstore
* feat(kms): enhance KMS service manager with runtime state and persistence support
* feat(kms): add local key export functionality for SSE-S3 migration tests
* fix(kms): keep local key export narrowly scoped
* fix(sse): validate copy source customer algorithm
---------
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
Move workspace-level dependency feature lists into the member crates that consume each dependency while keeping required default-features flags at the workspace root.
Also refresh starshard to 2.2.2 via cargo update and cargo upgrade --exclude ratelimit.
Co-authored-by: heihutu <heihutu@gmail.com>
* test(ecstore): assert decrypted_size for MinIO SSE interop round-trip
The ignored MinIO interop round-trip tests asserted `ObjectInfo.size`
against the plaintext length. For SSE objects `size` is the on-disk
DARE-encrypted size (plaintext + 32 bytes per 64 KiB block), so the
assertion can never hold once real fixtures are present — the two
`#[ignore]` tests failed the moment a real MinIO-written fixture was fed
in, even though the decoded data was byte-identical.
The client-visible object size comes from `decrypted_size()` /
`get_actual_size()`, which correctly reads MinIO's
`x-*-internal-actual-size` metadata (verified: both SSE-S3 and SSE-KMS
8 MiB multipart fixtures now report 8388608). Assert against that
instead and keep the plaintext length and SHA-256 data checks.
With real 4-drive MinIO fixtures (RELEASE.2025-09-07) all four tests
pass, confirming RustFS reads MinIO erasure-coded SSE objects with
byte-identical data and correct logical size.
Co-Authored-By: heihutu <heihutu@gmail.com>
* ci(ecstore): nightly MinIO interop check + dockerized fixture capture
Wire the ignored MinIO on-disk interop reader tests into a nightly,
non-required CI job, and make their fixtures reproducible without a host
MinIO install.
- Dockerfile + capture_via_docker.sh: build a throwaway image carrying
the official MinIO server binary (pinned RELEASE.2025-09-07) plus the
fixture lab on a small Python base, then run `lab.py capture-matrix` to
write the SSE-S3 / SSE-KMS multipart fixtures the tests consume. lab.py
drives MinIO's S3 API directly, so no `mc` is needed.
- .github/workflows/minio-interop.yml: nightly + manual workflow on
GitHub-hosted ubuntu-latest (reliable Docker + Python, unlike the
self-hosted fleet — see e2e-s3tests.yml infra note). Regenerates the
gitignored fixtures each run and executes the #[ignore] reader tests.
Not a PR gate.
- README: document the Docker capture path.
Validated end to end: the script builds the image, captures the two
multipart cases, and `cargo nextest run --run-ignored ignored-only`
passes all four interop tests.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
overtrue