test(interop): add real MinIO read and migration parity tests (#4377)

test(interop): real-MinIO read + migration parity, Phase 1/2 (backlog#580)

Capture authentic on-disk fixtures from MinIO RELEASE.2025-07-23 (a bucket with
versioning, object-lock, lifecycle, tagging, quota, a public policy, SSE-S3
encryption, a webhook notification target, and a replication rule, plus inline /
versioned / multipart objects and a delete marker) and prove RustFS reads and
migrates them losslessly:

- filemeta parses_real_minio_object_xlmeta: small inline, two-object-version +
  delete marker, and multipart object xl.meta parse to the expected FileInfo.
- ecstore parses_real_minio_bucket_metadata_blob_without_loss: the MinIO
  .metadata.bin msgpack decodes via the PascalCase field names and
  parse_all_configs loads all ten config types present (policy, lifecycle incl.
  <ExpiryUpdatedAt>, object-lock, versioning, tagging, quota, notification,
  encryption/SSE-S3, replication incl. DeleteMarkerReplication /
  ExistingObjectReplication) without loss.
- ecstore reads_minio_inline_bucket_metadata_via_bitrot: MinIO inlines an object
  body as [HighwayHash256 32B][body]; RustFS's BitrotReader with HighwayHash256S
  verifies and yields the exact blob (the "inline_data 前缀不同" is that prefix).
- ecstore migrates_real_minio_bucket_metadata_end_to_end: on a throwaway 4-drive
  local ECStore, a real MinIO .metadata.bin seeded under .minio.sys is migrated
  into .rustfs.sys byte-identically for every config, exercising the Phase 2
  source adapter (MIGRATING_META_BUCKET = ".minio.sys") through the object layer.

All four run as ordinary crate tests (nextest CI). Phase 4 (MinIO re-reading a
RustFS drive) is documented as out of scope for one-way migration.

Refs rustfs/backlog#580
This commit is contained in:
Zhengchao An
2026-07-08 01:12:49 +08:00
committed by GitHub
parent 62a31e4ec4
commit a91d9cefc6
11 changed files with 367 additions and 7 deletions
+20
View File
@@ -151,6 +151,26 @@ pub fn create_issue_2434_legacy_meta_v2_pool_xlmeta() -> Result<Vec<u8>> {
decode_hex_fixture(include_str!("../tests/fixtures/issue_2434_legacy_meta_v2_pool.hex"))
}
/// Real MinIO-written xl.meta for a small inlined object (`small.txt`, 19 bytes),
/// captured from MinIO `RELEASE.2025-07-23` — see `tests/fixtures/minio/README.md`
/// (backlog#580).
pub fn create_minio_small_object_xlmeta() -> Result<Vec<u8>> {
decode_hex_fixture(include_str!("../tests/fixtures/minio/object_small_txt.xlmeta.hex"))
}
/// Real MinIO-written xl.meta for a versioned object (`versioned.txt`, two
/// versions) — see `tests/fixtures/minio/README.md` (backlog#580).
pub fn create_minio_versioned_object_xlmeta() -> Result<Vec<u8>> {
decode_hex_fixture(include_str!("../tests/fixtures/minio/object_versioned_txt.xlmeta.hex"))
}
/// Real MinIO-written xl.meta for a larger, non-inlined object (`large.bin`,
/// 300 KiB, stored as `part.1`) — see `tests/fixtures/minio/README.md`
/// (backlog#580).
pub fn create_minio_large_object_xlmeta() -> Result<Vec<u8>> {
decode_hex_fixture(include_str!("../tests/fixtures/minio/object_large_bin.xlmeta.hex"))
}
fn write_legacy_time(wr: &mut Vec<u8>, ts: OffsetDateTime) {
wr.push(MSGPACK_EXT8);
wr.push(12);