mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 16:28:15 +00:00
docs: add MinIO compatibility router + file-format docs (#4328)
Add two durable architecture references grounded in current code: - minio-rustfs-router-compatibility.md: MinIO cmd/api-router.go (S3 object/bucket) and cmd/admin-router.go (admin /v3, /v4) vs RustFS implementation status, with per-row landing points and a gaps-only checklist of still-missing admin endpoints (profiling, healthinfo, LDAP IDP config, replication diff, MRF, batch, locks, speedtest, log stream, top, trace). - minio-file-format-compat.md: xl.meta (meta_ver 3 write, <=3 read, XL2 magic, rs-vandermonde, HighwayHash256 bitrot, inline data) and .metadata.bin bucket-metadata interop matrix for the backlog#580 items, plus old-RustFS -> new migration and a phased plan. Cross-links s3-compatibility-matrix.md and admin-route-action-snapshot.md and indexes both docs in docs/architecture/README.md. Refs rustfs/backlog#596 rustfs/backlog#603 rustfs/backlog#580
This commit is contained in:
@@ -40,6 +40,8 @@ Two rules keep this directory healthy:
|
||||
|
||||
- [s3-compatibility-matrix.md](s3-compatibility-matrix.md)
|
||||
- [s3-tables-support-matrix.md](s3-tables-support-matrix.md)
|
||||
- [minio-rustfs-router-compatibility.md](minio-rustfs-router-compatibility.md)
|
||||
- [minio-file-format-compat.md](minio-file-format-compat.md)
|
||||
|
||||
## Inventories & baselines (snapshots that feed migration work)
|
||||
|
||||
|
||||
@@ -0,0 +1,279 @@
|
||||
# MinIO File-Format Interoperability — Gap Analysis & Phased Plan
|
||||
|
||||
Assesses how closely the RustFS on-disk format matches MinIO's, so that a
|
||||
MinIO drive set can be read (and eventually served) by RustFS and vice versa.
|
||||
This is a **plan and analysis document**. It changes no storage code. Every
|
||||
claim below cites the code that backs it.
|
||||
|
||||
Scope: the two on-disk artifacts that matter for interop are the per-object
|
||||
`xl.meta` (object metadata + inline data) and the per-bucket `.metadata.bin`
|
||||
(bucket configuration blob). IAM/config layout is noted where it affects
|
||||
bucket-metadata migration.
|
||||
|
||||
Refs rustfs/backlog#580.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
- **`xl.meta`**: RustFS writes `XL_META_VERSION = 3` and reads meta_ver ≤ 3,
|
||||
including legacy meta_ver 2 objects with legacy checksums. Magic `XL2 `,
|
||||
erasure algorithm `rs-vandermonde` (Reed-Solomon), and HighwayHash256 bitrot
|
||||
all match MinIO. `xl.meta` interop is the **strong** part of the story.
|
||||
- **`.metadata.bin`**: RustFS uses the same filename, the same 4-byte
|
||||
`format|version` header, the same MessagePack blob layout, and the same
|
||||
per-config field encodings (XML/JSON) as MinIO's `bucketMetadata`. The
|
||||
divergence is a small set of RustFS-only fields (table-bucket support,
|
||||
bucket-targets meta) — not a format mismatch.
|
||||
- **Migration**: RustFS already ships a one-way importer that reads a legacy
|
||||
meta bucket and rewrites bucket-metadata + IAM config into the RustFS meta
|
||||
bucket (`crates/ecstore/src/bucket/migration.rs`).
|
||||
|
||||
The remaining work is verification breadth and closing per-config parsing gaps,
|
||||
not a format rewrite.
|
||||
|
||||
---
|
||||
|
||||
## Part A — `xl.meta` Object Format
|
||||
|
||||
### Version support
|
||||
|
||||
| Aspect | Value | Evidence |
|
||||
|---|---|---|
|
||||
| Write version (`meta_ver`) | 3 | `crates/filemeta/src/filemeta.rs:54` (`XL_META_VERSION = 3`), written in `FileMeta::new` at `crates/filemeta/src/filemeta.rs:121` |
|
||||
| Read versions accepted | ≤ 3 (1, 2, 3) | Decode rejects only `meta_ver > XL_META_VERSION` — see `crates/filemeta/src/filemeta/codec.rs` (`decode_xl_headers`); `load_or_convert` doc at `crates/filemeta/src/filemeta.rs:864` |
|
||||
| Legacy meta_ver 2 read | Supported (with legacy checksum) | Regression fixtures `test_issue_2265_legacy_meta_v2_object_compatibility` / `test_issue_2288_legacy_xlmeta_compatibility` at `crates/filemeta/src/filemeta.rs:1130`, `:1152`; `uses_legacy_checksum` asserted at `:1174` |
|
||||
|
||||
RustFS is a **read-forward-compatible** consumer of MinIO's `xl.meta`: it can
|
||||
parse older MinIO objects and normalizes them to meta_ver 3 on rewrite. It does
|
||||
not write MinIO's older versions.
|
||||
|
||||
### Container header
|
||||
|
||||
| Field | RustFS value | Evidence |
|
||||
|---|---|---|
|
||||
| Magic | `XL2 ` (`[b'X', b'L', b'2', b' ']`) | `crates/filemeta/src/filemeta.rs:46` |
|
||||
| File version major / minor | 1 / 3 | `crates/filemeta/src/filemeta.rs:51-52` |
|
||||
| Header version | 3 | `crates/filemeta/src/filemeta.rs:53` |
|
||||
| Magic + version check (decode entry) | `check_xl2_v1` validates magic and rejects `major > 1` | `crates/filemeta/src/filemeta/codec.rs:45-61` |
|
||||
| Version-only probe (no full parse) | `read_format_versions` returns `(major, minor, header_ver, meta_ver)` | `crates/filemeta/src/filemeta/codec.rs:30-43` |
|
||||
|
||||
The layout after the 8-byte header is `bin-length-prefixed msgpack header block`
|
||||
followed by a CRC trailer and optional inline data — matching MinIO's XL2 v1
|
||||
container.
|
||||
|
||||
### Erasure coding
|
||||
|
||||
| Aspect | Value | Evidence |
|
||||
|---|---|---|
|
||||
| Algorithm enum | `ErasureAlgo::ReedSolomon = 1` | `crates/filemeta/src/fileinfo.rs:83-106` |
|
||||
| Algorithm string | `rs-vandermonde` | `crates/filemeta/src/fileinfo.rs:31` (`ERASURE_ALGORITHM`); also `crates/ecstore/src/object_api/mod.rs:52` |
|
||||
| Codec crate | `rustfs-erasure-codec` (Reed-Solomon, SIMD) | `Cargo.toml:277` |
|
||||
|
||||
Same Reed-Solomon Vandermonde scheme and identifier string as MinIO.
|
||||
|
||||
### Bitrot / shard integrity
|
||||
|
||||
| Aspect | Value | Evidence |
|
||||
|---|---|---|
|
||||
| Default hash | `HashAlgorithm::HighwayHash256S` | Bitrot read/write paths in `crates/ecstore/src/io_support/bitrot.rs` (e.g. `:564`, `:767`) |
|
||||
| Legacy variant | `HighwayHash256SLegacy` (fixed key) for old objects | referenced from `rustfs_utils::HashAlgorithm` (imported at `crates/ecstore/src/io_support/bitrot.rs:26`) |
|
||||
| HighwayHash crate | `highway` 1.3.0 | `Cargo.toml:252` |
|
||||
| Legacy bitrot read coverage | dedicated test | `crates/ecstore/tests/legacy_bitrot_read_test.rs` |
|
||||
|
||||
MinIO uses HighwayHash256 for bitrot; RustFS's default `HighwayHash256S` is
|
||||
compatible, with a legacy-key variant retained for older shards.
|
||||
|
||||
### Inline data
|
||||
|
||||
Small objects are inlined into the `xl.meta` container after the CRC trailer
|
||||
rather than written as a separate `part.1`. Handling lives in
|
||||
`crates/filemeta/src/filemeta/inline_data.rs` (e.g. `physical_data_dir` and the
|
||||
shared-data-dir accounting), and the inline block is appended/consumed by the
|
||||
codec in `crates/filemeta/src/filemeta/codec.rs`. This mirrors MinIO's inline
|
||||
data feature and the `null`/version-id keying used for the inline map
|
||||
(`data_key_for_version` at `crates/filemeta/src/filemeta.rs:69`, legacy key at
|
||||
`:77`).
|
||||
|
||||
### `xl.meta` interop verdict
|
||||
|
||||
| Item | Done | Partial | Todo |
|
||||
|---|:--:|:--:|:--:|
|
||||
| Read MinIO meta_ver ≤ 3 | ✅ | | |
|
||||
| Legacy meta_ver 2 + legacy checksum read | ✅ | | |
|
||||
| XL2 container magic/version parity | ✅ | | |
|
||||
| Reed-Solomon `rs-vandermonde` parity | ✅ | | |
|
||||
| HighwayHash256 bitrot parity | ✅ | | |
|
||||
| Inline data parity | ✅ | | |
|
||||
| Broad fixture corpus from real MinIO writers | | ⚠️ | |
|
||||
| Write-back parity for round-trip (RustFS→MinIO read) | | ⚠️ | |
|
||||
|
||||
The two ⚠️ items are verification breadth, not known incompatibilities: the
|
||||
current fixtures are targeted regressions (issues #2265, #2288), and there is no
|
||||
CI job proving a MinIO binary can re-read a RustFS-written `xl.meta`.
|
||||
|
||||
---
|
||||
|
||||
## Part B — Bucket Metadata (`.metadata.bin`)
|
||||
|
||||
### On-disk layout
|
||||
|
||||
| Aspect | RustFS value | Evidence |
|
||||
|---|---|---|
|
||||
| Meta bucket | `.rustfs.sys` | `crates/ecstore/src/disk/mod.rs:29` (`RUSTFS_META_BUCKET`) |
|
||||
| Bucket-config prefix | `buckets` | `crates/ecstore/src/disk/mod.rs:34` (`BUCKET_META_PREFIX`) |
|
||||
| Blob file | `.metadata.bin` | `crates/ecstore/src/bucket/metadata.rs:227` (`BUCKET_METADATA_FILE`) |
|
||||
| Full path | `buckets/{bucket}/.metadata.bin` | `crates/ecstore/src/bucket/metadata.rs:415-416` (`save_file_path`) |
|
||||
| Header | `format: u16 LE` + `version: u16 LE`, both `= 1` | `crates/ecstore/src/bucket/metadata.rs:228-229`, checked in `check_header` at `:595-614` |
|
||||
| Body | MessagePack-encoded `BucketMetadata` | `marshal_msg`/`unmarshal` at `crates/ecstore/src/bucket/metadata.rs:582-593`; read strips the 4-byte header (`unmarshal(&data[4..])` at `:1079`) |
|
||||
|
||||
This is the same design as MinIO's bucket metadata: a single
|
||||
`.minio.sys/buckets/<bucket>/.metadata.bin` blob with a 4-byte
|
||||
`bucketMetadataFormat|bucketMetadataVersion` header and a msgpack body. The
|
||||
filename, header shape, and format/version values (`1`/`1`) all match. The
|
||||
`BucketMetadata` field names correspond one-to-one to MinIO's `bucketMetadata`
|
||||
struct (`policyConfigJSON`, `lifecycleConfigXML`, `objectLockConfigXML`, …).
|
||||
|
||||
> Correction to a common misconception: modern MinIO does **not** store each
|
||||
> bucket config as a separate loose `versioning.json` / `lifecycle.json` file —
|
||||
> it embeds them in the same `.metadata.bin` blob, with XML for the S3-XML
|
||||
> configs and JSON for policy/quota/targets. The per-config filename constants
|
||||
> in RustFS (`policy.json`, `lifecycle.xml`, …) are the **keys used by
|
||||
> `update_config`** to select a field, not separate on-disk files.
|
||||
|
||||
### Interop matrix (backlog#580 items)
|
||||
|
||||
Field/constant references are in `crates/ecstore/src/bucket/metadata.rs`.
|
||||
"Encoding" is the payload RustFS stores in that field and must match MinIO's for
|
||||
byte-level interop. Getter functions live in
|
||||
`crates/ecstore/src/bucket/metadata_sys.rs`.
|
||||
|
||||
| Config item | RustFS field / constant | Encoding | MinIO field | Status |
|
||||
|---|---|---|---|---|
|
||||
| versioning | `versioning_config_xml` / `BUCKET_VERSIONING_CONFIG` = `versioning.xml` | XML | versioningConfigXML | Done |
|
||||
| quota | `quota_config_json` / `BUCKET_QUOTA_CONFIG_FILE` = `quota.json` | JSON | quotaConfigJSON | Done |
|
||||
| object_lock | `object_lock_config_xml` / `OBJECT_LOCK_CONFIG` = `object-lock.xml` | XML | objectLockConfigXML | Done |
|
||||
| replication | `replication_config_xml` / `BUCKET_REPLICATION_CONFIG` = `replication.xml` | XML | replicationConfigXML | Done |
|
||||
| policy | `policy_config_json` / `BUCKET_POLICY_CONFIG` = `policy.json` | JSON | policyConfigJSON | Done |
|
||||
| lifecycle | `lifecycle_config_xml` / `BUCKET_LIFECYCLE_CONFIG` = `lifecycle.xml` | XML | lifecycleConfigXML | Done |
|
||||
| tagging | `tagging_config_xml` / `BUCKET_TAGGING_CONFIG` = `tagging.xml` | XML | taggingConfigXML | Done |
|
||||
| bucket_targets | `bucket_targets_config_json` + `bucket_targets_config_meta_json` / `BUCKET_TARGETS_FILE` = `bucket-targets.json` | JSON | bucketTargetsConfigJSON (+ meta variant) | Partial |
|
||||
| notification | `notification_config_xml` / `BUCKET_NOTIFICATION_CONFIG` = `notification.xml` | XML | notificationConfigXML | Done |
|
||||
| encryption | `encryption_config_xml` / `BUCKET_SSECONFIG` = `bucket-encryption.xml` | XML | encryptionConfigXML | Done |
|
||||
| cors | `cors_config_xml` / `BUCKET_CORS_CONFIG` = `cors.xml` | XML | corsConfigXML | Done |
|
||||
| public_access | `public_access_block_config_xml` / `BUCKET_PUBLIC_ACCESS_BLOCK_CONFIG` = `public-access-block.xml` | XML | publicAccessBlockConfigXML | Done |
|
||||
| bucket_acl | `bucket_acl_config_json` / `BUCKET_ACL_CONFIG` = `bucket-acl.json` | JSON | bucketACLConfigJSON | Partial |
|
||||
|
||||
Field definitions: `crates/ecstore/src/bucket/metadata.rs:274-336`. Constants:
|
||||
`:227-247`. `update_config` field routing: `:678-761`. `parse_all_configs` is
|
||||
invoked on load (`load_bucket_metadata_parse` at `:1043`).
|
||||
|
||||
Notes on the two "Partial" rows:
|
||||
|
||||
- **bucket_targets** — RustFS carries an extra `bucket_targets_config_meta_json`
|
||||
field (`:288`) beyond MinIO's single targets blob. The primary
|
||||
`bucket-targets.json` payload is interoperable; the meta side-channel is
|
||||
RustFS-specific and a MinIO reader would ignore it. ACL enforcement itself is
|
||||
bounded (S3 `PutBucketAcl`/`PutObjectAcl` accept canned ACLs only — see
|
||||
[minio-rustfs-router-compatibility.md](minio-rustfs-router-compatibility.md)).
|
||||
- **bucket_acl** — stored and round-tripped in the blob, but ACL grant
|
||||
semantics are intentionally limited at the S3 layer.
|
||||
|
||||
RustFS also defines fields with no interop requirement from backlog#580 but
|
||||
worth noting so a migration tool does not choke on them: `logging_config_xml`,
|
||||
`website_config_xml`, `accelerate_config_xml`, `request_payment_config_xml`
|
||||
(`:242-245`), and the RustFS-only `table_bucket_config_json`
|
||||
(`BUCKET_TABLE_CONFIG` = `table-bucket.json`, `:248`). A MinIO reader that does
|
||||
not know `table_bucket_config_json` will ignore the unknown msgpack field.
|
||||
|
||||
### Old-RustFS → new-RustFS migration
|
||||
|
||||
RustFS ships a one-way importer that reads a legacy meta bucket
|
||||
(`MIGRATING_META_BUCKET`) and rewrites both bucket metadata and IAM config into
|
||||
the current RustFS meta bucket, skipping entries that already exist
|
||||
(idempotent). See `crates/ecstore/src/bucket/migration.rs`:
|
||||
|
||||
- `try_migrate_bucket_metadata` copies `buckets/{bucket}/.metadata.bin` and the
|
||||
replication resync blob for each bucket (`crates/ecstore/src/bucket/migration.rs:193`).
|
||||
- `try_migrate_iam_config` walks `config/iam/` and normalizes legacy IAM
|
||||
records — legacy timestamp fields (`update_at` → `updatedAt`) and legacy
|
||||
policy-mapping field aliases (`policies` → `policy`) are rewritten
|
||||
(`normalize_iam_config_blob` at `:97`; regression test at `:428`).
|
||||
- Bucket resync metadata is re-encoded through `ReplicationMigrationBridge`
|
||||
(`normalize_bucket_meta_blob` at `:178`).
|
||||
|
||||
This importer is the practical basis for a MinIO → RustFS bucket-metadata
|
||||
migration: because the blob layout and field encodings already match, the
|
||||
missing piece is a source adapter that points the importer at a MinIO
|
||||
`.minio.sys` layout rather than the RustFS legacy layout.
|
||||
|
||||
### Bucket-metadata interop verdict
|
||||
|
||||
| Item | Done | Partial | Todo |
|
||||
|---|:--:|:--:|:--:|
|
||||
| `.metadata.bin` filename + header + msgpack layout parity | ✅ | | |
|
||||
| Per-config field encodings (XML/JSON) match MinIO | ✅ | | |
|
||||
| versioning/quota/object_lock/replication/policy/lifecycle/tagging/notification/encryption/cors/public_access round-trip | ✅ | | |
|
||||
| bucket_targets primary blob | ✅ | | |
|
||||
| bucket_targets meta side-channel + ACL grant semantics | | ⚠️ | |
|
||||
| Old-RustFS → new-RustFS importer | ✅ | | |
|
||||
| MinIO `.minio.sys` source adapter for the importer | | | ❌ |
|
||||
| CI proof a MinIO-written `.metadata.bin` loads unchanged | | | ❌ |
|
||||
|
||||
---
|
||||
|
||||
## Phased Plan
|
||||
|
||||
The format is already close; the plan is verification, a source adapter, and
|
||||
closing the two partial encodings — not a rewrite.
|
||||
|
||||
### Phase 1 — Read parity, proven (verification)
|
||||
|
||||
- Add a MinIO-writer fixture corpus for `xl.meta` (inline + multipart +
|
||||
versioned + delete-marker + transitioned) and assert RustFS parses each to a
|
||||
`FileInfo` equivalent to MinIO's, alongside the existing issue #2265 / #2288
|
||||
fixtures in `crates/filemeta/src/filemeta.rs`.
|
||||
- Add a fixture `.metadata.bin` written by MinIO and assert
|
||||
`BucketMetadata::unmarshal` + `parse_all_configs` load every field without
|
||||
loss (`crates/ecstore/src/bucket/metadata.rs`).
|
||||
- Exit criterion: a CI job that fails if a real MinIO-written object or bucket
|
||||
blob cannot be read.
|
||||
|
||||
### Phase 2 — MinIO source adapter for migration
|
||||
|
||||
- Generalize the importer in `crates/ecstore/src/bucket/migration.rs` so the
|
||||
source can be a MinIO `.minio.sys/buckets/<bucket>/.metadata.bin` layout, not
|
||||
only the RustFS legacy meta bucket. Because the blob format matches, this is
|
||||
mostly source-path plumbing plus IAM record normalization reuse.
|
||||
- Exit criterion: importing a MinIO backup reproduces all backlog#580
|
||||
bucket-config items with byte-identical config payloads.
|
||||
|
||||
### Phase 3 — Close the two partial encodings
|
||||
|
||||
- bucket_targets: document/normalize the RustFS-only
|
||||
`bucket_targets_config_meta_json` so a round-trip through MinIO and back does
|
||||
not silently drop it; or fold its content into a MinIO-compatible
|
||||
representation.
|
||||
- bucket_acl: decide whether ACL grant semantics beyond canned ACLs are in
|
||||
scope; if not, keep the blob round-trippable but document the enforcement
|
||||
limit (already reflected in the router compatibility matrix).
|
||||
|
||||
### Phase 4 — Round-trip / write-back parity (stretch)
|
||||
|
||||
- Prove a MinIO binary can re-read a RustFS-written `xl.meta` and
|
||||
`.metadata.bin` (the reverse direction). This is a stretch goal because
|
||||
RustFS writes meta_ver 3 and MinIO's acceptance of specific v3 features must
|
||||
be validated per feature.
|
||||
- Exit criterion: a MinIO instance mounted on a RustFS-written drive set serves
|
||||
objects and bucket configs without repair.
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- This document is analysis only. Any change to `crates/filemeta` or
|
||||
`crates/ecstore/src/bucket` metadata encoding is a storage-format change and
|
||||
must follow the migration and readiness contracts in
|
||||
[README.md](README.md) and the ecstore layout boundary rules.
|
||||
- The version constants (`XL_META_VERSION`,
|
||||
`BUCKET_METADATA_FORMAT`/`BUCKET_METADATA_VERSION`) are compatibility anchors.
|
||||
Bumping any of them requires a read-compat path for the prior value and a
|
||||
migration story, exactly as the current meta_ver 2 → 3 read path provides.
|
||||
@@ -0,0 +1,213 @@
|
||||
# MinIO ↔ RustFS Router Compatibility Matrix
|
||||
|
||||
Tracks how RustFS covers the MinIO HTTP router surface, split into the S3
|
||||
data-plane router (`cmd/api-router.go` in MinIO: object + bucket APIs) and the
|
||||
admin control-plane router (`cmd/admin-router.go`: admin `/v3/` and `/v4/`
|
||||
APIs). Each row records the current RustFS implementation status and the
|
||||
landing point in the code so the matrix can be re-verified after refactors.
|
||||
|
||||
This complements two neighbouring documents and does not duplicate them:
|
||||
|
||||
- [s3-compatibility-matrix.md](s3-compatibility-matrix.md) — the release-facing
|
||||
S3 compatibility claim and the Ceph s3tests lists that gate it.
|
||||
- [admin-route-action-snapshot.md](admin-route-action-snapshot.md) — the admin
|
||||
route/handler/authorization-action migration guardrail (the source of truth
|
||||
for exact route patterns and auth contracts).
|
||||
|
||||
Refs rustfs/backlog#596 rustfs/backlog#603.
|
||||
|
||||
## Status Legend
|
||||
|
||||
| Status | Meaning |
|
||||
|---|---|
|
||||
| 已实现 (implemented) | Handler is registered and performs the real operation. |
|
||||
| 部分兼容 (partial) | Registered and functional, but a documented subset of the MinIO behavior is rejected or unsupported. |
|
||||
| 已注册未完成 (registered, incomplete) | Route is registered but the handler returns `NotImplemented` (a behavior contract, not a real implementation). |
|
||||
| 缺失 (missing) | No RustFS route/handler for the MinIO endpoint. |
|
||||
| 行为不一致 (behavior differs) | Implemented but intentionally diverges from MinIO's response contract. |
|
||||
|
||||
Prefixes: RustFS registers admin routes under the canonical `/rustfs/admin`
|
||||
prefix and accepts `/minio/admin` as a compatibility alias via router
|
||||
canonicalization (see
|
||||
[admin-route-action-snapshot.md](admin-route-action-snapshot.md)). Admin paths
|
||||
below are shown relative to that prefix (e.g. `/v3/info`).
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — S3 Data Plane (MinIO `cmd/api-router.go`)
|
||||
|
||||
RustFS implements the S3 surface through the `s3s` service trait in
|
||||
`rustfs/src/storage/ecfs.rs`, delegating to use-case layers under
|
||||
`rustfs/src/app/`. Line numbers are indicative landing points on the branch
|
||||
this matrix was written against and may drift; the file paths are stable.
|
||||
|
||||
### Bucket-level operations
|
||||
|
||||
| MinIO / S3 operation | Status | RustFS landing point |
|
||||
|---|---|---|
|
||||
| CreateBucket | 已实现 | `rustfs/src/storage/ecfs.rs` (`create_bucket`) |
|
||||
| DeleteBucket | 已实现 | `rustfs/src/storage/ecfs.rs` (`delete_bucket`) |
|
||||
| HeadBucket | 已实现 | `rustfs/src/storage/ecfs.rs` (`head_bucket`) |
|
||||
| ListBuckets | 已实现 | `rustfs/src/storage/ecfs.rs` (`list_buckets`) |
|
||||
| GetBucketLocation | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_location`) |
|
||||
| ListObjects (v1) | 已实现 | `rustfs/src/storage/ecfs.rs` (`list_objects`) |
|
||||
| ListObjectsV2 | 已实现 | `rustfs/src/storage/ecfs.rs` (`list_objects_v2`) |
|
||||
| ListObjectVersions | 已实现 | `rustfs/src/storage/ecfs.rs` (`list_object_versions`) |
|
||||
| ListMultipartUploads | 已实现 | `rustfs/src/storage/ecfs.rs` (`list_multipart_uploads`) |
|
||||
| Get/PutBucketVersioning | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_versioning`, `put_bucket_versioning`) |
|
||||
| Get/Put/DeleteBucketPolicy | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_policy`, `put_bucket_policy`, `delete_bucket_policy`) |
|
||||
| GetBucketPolicyStatus | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_policy_status`) |
|
||||
| Get/Put/DeleteBucketTagging | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_tagging`, `put_bucket_tagging`, `delete_bucket_tagging`) |
|
||||
| Get/Put/DeleteBucketLifecycle | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_lifecycle_configuration`, `put_bucket_lifecycle_configuration`, `delete_bucket_lifecycle`) |
|
||||
| Get/Put/DeleteBucketReplication | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_replication`, `put_bucket_replication`, `delete_bucket_replication`) |
|
||||
| Get/Put/DeleteBucketEncryption | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_encryption`, `put_bucket_encryption`, `delete_bucket_encryption`) |
|
||||
| Get/PutObjectLockConfiguration | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_object_lock_configuration`, `put_object_lock_configuration`) |
|
||||
| Get/Put/DeletePublicAccessBlock | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_public_access_block`, `put_public_access_block`, `delete_public_access_block`) |
|
||||
| Get/Put/DeleteBucketCors | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_cors`, `put_bucket_cors`, `delete_bucket_cors`) |
|
||||
| GetBucketNotificationConfiguration | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_notification_configuration`) |
|
||||
| PutBucketNotificationConfiguration | 已实现 | `rustfs/src/storage/ecfs.rs` (`put_bucket_notification_configuration`) |
|
||||
| Get/PutBucketRequestPayment | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_request_payment`, `put_bucket_request_payment`) |
|
||||
| Get/PutBucketLogging | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_logging`, `put_bucket_logging`) |
|
||||
| Get/Put/DeleteBucketWebsite | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_website`, `put_bucket_website`, `delete_bucket_website`) |
|
||||
| Get/PutBucketAccelerateConfiguration | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_accelerate_configuration`, `put_bucket_accelerate_configuration`) |
|
||||
| GetBucketAcl | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_bucket_acl`) |
|
||||
| PutBucketAcl | 部分兼容 | `rustfs/src/storage/ecfs.rs` (`put_bucket_acl`) — canned-ACL headers only; XML grant policies return `NotImplemented`. |
|
||||
| GetBucketReplicationMetrics | 缺失 | No S3-path handler; replication metrics are exposed via the admin API `/v3/replicationmetrics` instead. |
|
||||
| GetBucketOwnershipControls | 缺失 | Not implemented (matches the "bucket ownership controls: planned" note in `s3-compatibility-matrix.md`). |
|
||||
| Put/DeleteBucketOwnershipControls | 缺失 | Not implemented. |
|
||||
|
||||
Note on delete verbs: several S3 sub-resource DELETE operations
|
||||
(DeleteBucketNotification, DeleteBucketLogging, DeleteBucketRequestPayment,
|
||||
DeleteBucketAccelerate) are not exposed as distinct handlers; the corresponding
|
||||
config is cleared by writing an empty configuration through the PUT path. Treat
|
||||
these as 部分兼容 at the client level.
|
||||
|
||||
### Object-level operations
|
||||
|
||||
| MinIO / S3 operation | Status | RustFS landing point |
|
||||
|---|---|---|
|
||||
| GetObject | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_object`) |
|
||||
| PutObject | 已实现 | `rustfs/src/storage/ecfs.rs` (`put_object`) |
|
||||
| DeleteObject | 已实现 | `rustfs/src/storage/ecfs.rs` (`delete_object`) |
|
||||
| DeleteObjects (multi-delete) | 已实现 | `rustfs/src/storage/ecfs.rs` (`delete_objects`) |
|
||||
| HeadObject | 已实现 | `rustfs/src/storage/ecfs.rs` (`head_object`) |
|
||||
| CopyObject | 已实现 | `rustfs/src/storage/ecfs.rs` (`copy_object`) |
|
||||
| GetObjectAcl | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_object_acl`) |
|
||||
| PutObjectAcl | 部分兼容 | `rustfs/src/storage/ecfs.rs` (`put_object_acl`) — canned-ACL headers only; XML grants return `NotImplemented`. |
|
||||
| Get/Put/DeleteObjectTagging | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_object_tagging`, `put_object_tagging`, `delete_object_tagging`) |
|
||||
| GetObjectAttributes | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_object_attributes`) |
|
||||
| Get/PutObjectLegalHold | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_object_legal_hold`, `put_object_legal_hold`) |
|
||||
| Get/PutObjectRetention | 已实现 | `rustfs/src/storage/ecfs.rs` (`get_object_retention`, `put_object_retention`) |
|
||||
| RestoreObject (POST restore) | 已实现 | `rustfs/src/storage/ecfs.rs` (`restore_object`) |
|
||||
| SelectObjectContent | 已实现 | `rustfs/src/storage/ecfs.rs` (`select_object_content`) → `rustfs/src/app/select_object.rs` |
|
||||
| CreateMultipartUpload | 已实现 | `rustfs/src/storage/ecfs.rs` (`create_multipart_upload`) |
|
||||
| UploadPart / UploadPartCopy | 已实现 | `rustfs/src/storage/ecfs.rs` (`upload_part`, `upload_part_copy`) |
|
||||
| CompleteMultipartUpload | 已实现 | `rustfs/src/storage/ecfs.rs` (`complete_multipart_upload`) |
|
||||
| AbortMultipartUpload | 已实现 | `rustfs/src/storage/ecfs.rs` (`abort_multipart_upload`) |
|
||||
| ListParts | 已实现 | `rustfs/src/storage/ecfs.rs` (`list_parts`) |
|
||||
| PostObject (POST form upload) | 已实现 | Routed via the POST-object marker into the put-object path (`rustfs/src/app/object_usecase.rs`). See the "POST Object form upload checksum handling: planned" note in `s3-compatibility-matrix.md`. |
|
||||
| GetObjectTorrent | 行为不一致 | `rustfs/src/storage/ecfs.rs` (`get_object_torrent`) — returns `404 NoSuchKey` by design (not `501 NotImplemented`) so clients degrade gracefully. |
|
||||
|
||||
For the gate-level view of which of these are covered by executable s3tests,
|
||||
defer to [s3-compatibility-matrix.md](s3-compatibility-matrix.md); this table is
|
||||
the router/handler view, not the test-list view.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Admin Control Plane (MinIO `cmd/admin-router.go`)
|
||||
|
||||
Router assembly is `rustfs/src/admin/mod.rs::register_admin_routes`; the exact
|
||||
route patterns, handler ownership, and authorization actions are the guardrail
|
||||
in [admin-route-action-snapshot.md](admin-route-action-snapshot.md). This table
|
||||
maps MinIO admin route families to RustFS status.
|
||||
|
||||
### Implemented / registered families
|
||||
|
||||
| MinIO admin family | Status | RustFS landing point |
|
||||
|---|---|---|
|
||||
| STS / is-admin probe | 已实现 | `rustfs/src/admin/handlers/sts.rs`, `is_admin.rs` |
|
||||
| User lifecycle (list/add/info/remove/status) | 已实现 | `rustfs/src/admin/handlers/user_lifecycle.rs`, `user.rs` |
|
||||
| Groups | 已实现 | `rustfs/src/admin/handlers/group.rs` |
|
||||
| Service accounts / access keys | 已实现 | `rustfs/src/admin/handlers/service_account.rs` |
|
||||
| Canned policies + builtin policy attach/detach + policy-entities | 已实现 | `rustfs/src/admin/handlers/policies.rs` |
|
||||
| IAM import/export | 已实现 | `rustfs/src/admin/handlers/user_iam.rs`, `user.rs` |
|
||||
| Account info | 已实现 | `rustfs/src/admin/handlers/account_info.rs` |
|
||||
| Config KV (get/set/del/help/history/restore + `/v3/config`) | 已实现 | `rustfs/src/admin/handlers/config_admin.rs` |
|
||||
| Server info / storageinfo / datausageinfo | 已实现 | `rustfs/src/admin/handlers/system.rs` |
|
||||
| Metrics stream (`/v3/metrics`) | 已实现 | `rustfs/src/admin/handlers/metrics.rs` via `system.rs` |
|
||||
| Runtime capabilities (`/v4/runtime/capabilities`) | 已实现 | `rustfs/src/admin/handlers/system.rs` |
|
||||
| Pools list/status | 已实现 | `rustfs/src/admin/handlers/pools.rs` |
|
||||
| Pools decommission/cancel/clear | 部分兼容 | `rustfs/src/admin/handlers/pools.rs` — returns `NotImplemented` when endpoints are not initialized (single-pool / uninitialized clusters). |
|
||||
| Rebalance start/status/stop | 已实现 | `rustfs/src/admin/handlers/rebalance.rs` |
|
||||
| Heal + background-heal status | 已实现 | `rustfs/src/admin/handlers/heal.rs` |
|
||||
| Tier (list/stats/verify/add/edit/remove/clear) | 已实现 | `rustfs/src/admin/handlers/tier.rs` |
|
||||
| Quota (legacy + bucket-scoped + stats/check) | 已实现 | `rustfs/src/admin/handlers/quota.rs` |
|
||||
| Bucket metadata export/import | 已实现 | `rustfs/src/admin/handlers/bucket_meta.rs` |
|
||||
| Scanner status | 已实现 | `rustfs/src/admin/handlers/scanner.rs` |
|
||||
| Notification targets (list/arns/put/reset) | 已实现 | `rustfs/src/admin/handlers/event.rs` |
|
||||
| Audit targets (list/put/reset) | 已实现 | `rustfs/src/admin/handlers/audit.rs` |
|
||||
| Module switches | 已实现 | `rustfs/src/admin/handlers/module_switch.rs` |
|
||||
| Plugin catalog + instances (`/v4/plugins/*`) | 已实现 | `rustfs/src/admin/handlers/plugins_catalog.rs`, `plugins_instances.rs` |
|
||||
| Extension catalog + instances (`/v4/extensions/*`) | 已实现 | `rustfs/src/admin/handlers/extensions.rs` |
|
||||
| Object ZIP download (`/v3/zip-downloads`) | 已实现 | `rustfs/src/admin/handlers/object_zip_download.rs` |
|
||||
| Cluster snapshot (`/v4/cluster/snapshot`) | 已实现 | `rustfs/src/admin/handlers/cluster_snapshot.rs` |
|
||||
| Bucket-level remote targets (list/metrics/set/remove) | 已实现 | `rustfs/src/admin/handlers/replication.rs` |
|
||||
| Site replication (add/remove/info/status/peer/resync + devnull/netperf) | 已实现 | `rustfs/src/admin/handlers/site_replication.rs` |
|
||||
| Admin profiling (`/debug/pprof/profile`, `/debug/pprof/status`) | 已实现 | `rustfs/src/admin/handlers/profile_admin.rs`, `profile.rs` |
|
||||
| TLS debug (`/debug/tls/status`) | 已实现 | `rustfs/src/admin/handlers/tls_debug.rs`, `profile.rs` |
|
||||
| KMS management / dynamic / keys | 已实现 | `rustfs/src/admin/handlers/kms_management.rs`, `kms_dynamic.rs`, `kms_keys.rs` |
|
||||
| OIDC public + config | 已实现 | `rustfs/src/admin/handlers/oidc.rs` |
|
||||
| Table catalog (Iceberg) | 已实现 | `rustfs/src/admin/handlers/table_catalog.rs` |
|
||||
|
||||
### Registered-but-incomplete
|
||||
|
||||
| MinIO admin family | Status | RustFS landing point |
|
||||
|---|---|---|
|
||||
| Service restart/stop (`POST /v3/service`) | 已注册未完成 | `rustfs/src/admin/handlers/system.rs` — handler returns `NotImplemented`. |
|
||||
| Inspect data (`GET|POST /v3/inspect-data`) | 已注册未完成 | `rustfs/src/admin/handlers/system.rs` — handler returns `NotImplemented`. |
|
||||
|
||||
These registered-but-`NotImplemented` routes are behavior contracts; per the
|
||||
migration rules in [admin-route-action-snapshot.md](admin-route-action-snapshot.md),
|
||||
implementing or removing them is a behavior-change PR.
|
||||
|
||||
---
|
||||
|
||||
## Gaps Only — Missing Admin Endpoints (follow-up checklist)
|
||||
|
||||
The following MinIO admin `/v3/` route families have **no** RustFS registration
|
||||
today. This is the actionable checklist for closing admin-API parity. Verified
|
||||
against `rustfs/src/admin/mod.rs` and `rustfs/src/admin/handlers/` on the branch
|
||||
this doc was written on.
|
||||
|
||||
- [ ] **Server profiling start/stop** — MinIO `/v3/profile` (bulk profiling
|
||||
session). RustFS only exposes `/debug/pprof/profile` and
|
||||
`/debug/pprof/status`, which are a different, single-shot pprof surface.
|
||||
- [ ] **Health info** — MinIO `/v3/healthinfo` (cluster health report / subnet
|
||||
diagnostics). No RustFS route.
|
||||
- [ ] **LDAP / generic IDP config CRUD** — MinIO `/v3/idp/{ldap|openid}/...`
|
||||
config management. RustFS exposes OIDC config under `/v3/oidc/*` only; there
|
||||
is no LDAP IDP config route.
|
||||
- [ ] **Bucket / site replication diff** — MinIO replication-diff endpoints.
|
||||
RustFS exposes `/v3/replicationmetrics` (metrics) and site-replication
|
||||
status, but no per-object diff.
|
||||
- [ ] **MRF metrics** — MinIO's most-recent-failures replication metrics
|
||||
breakdown. RustFS has only the generic `/v3/metrics` stream.
|
||||
- [ ] **Batch jobs** — MinIO `/v3/batch`, `/v3/list-batch-jobs`, job
|
||||
describe/cancel. No RustFS batch API.
|
||||
- [ ] **Distributed locks introspection** — MinIO `/v3/force-unlock` and
|
||||
`/v3/top/locks`. No RustFS locks-management API.
|
||||
- [ ] **Speedtest / perf** — MinIO `/v3/speedtest` (object/drive/net perf).
|
||||
RustFS has `netperf`/`devnull` **only** inside the site-replication family,
|
||||
not as standalone admin speedtest endpoints.
|
||||
- [ ] **Console log stream** — MinIO `/v3/log` (kstream / log search). No RustFS
|
||||
route.
|
||||
- [ ] **Top introspection** — MinIO `/v3/top/locks`, `/v3/top/drives`,
|
||||
`/v3/top/net`. No RustFS unified `top` family.
|
||||
- [ ] **Trace stream** — MinIO `/v3/trace`. A `trace.rs` handler skeleton
|
||||
exists under `rustfs/src/admin/handlers/` but its registration function is
|
||||
**not** called from `register_admin_routes`, so no route is live.
|
||||
|
||||
When one of these lands, register it in `rustfs/src/admin/mod.rs`, extend
|
||||
`rustfs/src/admin/route_registration_test.rs`, update
|
||||
[admin-route-action-snapshot.md](admin-route-action-snapshot.md) with the
|
||||
route/handler/action rows, and move the item out of this checklist.
|
||||
Reference in New Issue
Block a user