mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 08:18:18 +00:00
Zhengchao An
4b83efaf36
* ci(ilm): move first batch of 5 s3-tests lifecycle expiration cases into a gated behavior lane (backlog#1148 ilm-10)
The 20 lifecycle cases in excluded_tests.txt were labeled "vendor-specific"
but the real blocker was the absence of a Ceph lc_debug_interval equivalent.
RUSTFS_ILM_DEBUG_DAY_SECS (ilm-5) now provides that, so Days>=1 expiration
behavior is testable in seconds.
These cases assert that objects/versions/uploads are actually removed by the
background scanner and the stale-multipart cleanup loop. They cannot join the
default single-server s3-implemented-tests gate: it disables the scanner, and a
global RUSTFS_ILM_DEBUG_DAY_SECS also shrinks the x-amz-expiration header that
the already-passing test_lifecycle_expiration_header_* cases assert on. So this
adds an isolated lane instead of putting them in implemented_tests.txt.
First batch (5), each verified against the pinned upstream source and the
RustFS evaluator/scanner/stale-multipart execution paths:
- test_lifecycle_expiration (prefix Days=1/Days=5, scanner delete)
- test_lifecyclev2_expiration (same via ListObjectsV2)
- test_lifecycle_noncur_expiration (NoncurrentVersionExpiration)
- test_lifecycle_deletemarker_expiration (ExpiredObjectDeleteMarker cascade)
- test_lifecycle_multipart_expiration (AbortIncompleteMultipartUpload)
Dropped from the batch, kept excluded with reason:
- test_lifecycle_expiration_days0: RustFS accepts Expiration{Days:0} (returns
200; only Days<0 is rejected in crates/lifecycle/src/core.rs validate()),
while AWS/this test expect InvalidArgument. Real validation gap.
Changes:
- scripts/s3-tests/lifecycle_behavior_tests.txt: new exact-node-id run set.
- scripts/s3-tests/run.sh: honor an IMPLEMENTED_TESTS_FILE override so a lane
can point at an alternate whitelist.
- .github/workflows/ci.yml: new s3-lifecycle-behavior-tests PR-gate job that
starts rustfs with RUSTFS_ILM_DEBUG_DAY_SECS=10, scanner enabled (cycle 2s,
no start delay) and a 2s stale-multipart cleanup interval, running the new
list serially.
- scripts/s3-tests/report_compat.py: recognize the behavior list so the weekly
scope=all sweep (plain server) does not misclassify expected failures.
- scripts/s3-tests/excluded_tests.txt: remove the 5 enabled cases; re-annotate
the remaining 15 lifecycle exclusions with real reasons + batch-2 plan.
- docs/architecture/s3-compatibility-matrix.md: sync counts (implemented 451,
excluded 274, behavior 5) and document the lane.
Refs backlog#1148 (ilm-10), master plan backlog#1155.
* fix(lifecycle): reject zero-day expiration/noncurrent/abort rules (backlog#1148 ilm-10) (#4722)
80 lines
4.1 KiB
Markdown
80 lines
4.1 KiB
Markdown
# S3 Compatibility Matrix
|
|
|
|
This matrix records the user-facing S3 compatibility claim for RustFS and ties
|
|
it to the executable Ceph s3tests lists under `scripts/s3-tests/`.
|
|
|
|
## Current Claim
|
|
|
|
RustFS provides broad S3 API compatibility for supported features. It does not
|
|
claim complete coverage of every standard or vendor-specific S3 behavior.
|
|
|
|
The root README should use the same wording: supported S3-compatible clients and
|
|
features are covered by the compatibility matrix and test lists.
|
|
|
|
## Test List Sources
|
|
|
|
| List | Purpose | Current count | Source |
|
|
|---|---:|---:|---|
|
|
| Implemented tests | Standard S3 tests expected to pass and used by the default local s3tests run. | 452 | `scripts/s3-tests/implemented_tests.txt` |
|
|
| Lifecycle behavior tests | Expiration behavior cases gated by the dedicated `s3-lifecycle-behavior-tests` lane (debug-accelerated day + scanner enabled). | 5 | `scripts/s3-tests/lifecycle_behavior_tests.txt` |
|
|
| Unimplemented tests | Standard S3 features planned but not yet implemented. | 17 | `scripts/s3-tests/unimplemented_tests.txt` |
|
|
| Excluded tests | Vendor-specific or intentionally unsupported behavior excluded from RustFS compatibility gating. | 273 | `scripts/s3-tests/excluded_tests.txt` |
|
|
|
|
Counts ignore blank lines and comments.
|
|
|
|
The lifecycle behavior lane runs real Days-based expiration cases that need
|
|
`RUSTFS_ILM_DEBUG_DAY_SECS` (Ceph `lc_debug_interval` equivalent) and an enabled
|
|
background scanner; it cannot share the default single-server gate because a
|
|
global debug day would also shrink the `x-amz-expiration` header asserted by the
|
|
`test_lifecycle_expiration_header_*` cases. See `scripts/s3-tests/run.sh`
|
|
(`IMPLEMENTED_TESTS_FILE` override) and the `s3-lifecycle-behavior-tests` job in
|
|
`.github/workflows/ci.yml`.
|
|
|
|
## Supported Coverage
|
|
|
|
The implemented test list currently covers the common object-storage surface:
|
|
|
|
| Area | Status | Evidence |
|
|
|---|---|---|
|
|
| Bucket create/delete/list/head | Supported | `implemented_tests.txt` |
|
|
| Object put/get/delete/copy/head | Supported | `implemented_tests.txt` |
|
|
| ListObjects/ListObjectsV2 prefix, delimiter, marker, max-keys | Supported | `implemented_tests.txt` |
|
|
| Multipart upload create/upload/complete/abort and selected multipart copy/checksum/object-attribute behavior | Supported | `implemented_tests.txt` |
|
|
| Bucket and object tagging | Supported | `implemented_tests.txt` |
|
|
| Bucket policy put/get/delete | Supported | `implemented_tests.txt` |
|
|
| Public access block put/get/delete | Supported | `implemented_tests.txt` |
|
|
| Presigned GET and PUT URLs | Supported | `implemented_tests.txt` |
|
|
| Range and conditional reads | Supported | `implemented_tests.txt` |
|
|
| User metadata | Supported | `implemented_tests.txt` |
|
|
| SSE-C and selected SSE-KMS edge cases | Supported | `implemented_tests.txt` |
|
|
| Selected versioning, object-lock, checksum, CORS, raw request, and conditional write behavior | Supported | `implemented_tests.txt` |
|
|
|
|
## Planned Standard Coverage
|
|
|
|
These are standard S3 areas that remain planned work and must not be described
|
|
as already complete:
|
|
|
|
| Area | Status | Evidence |
|
|
|---|---|---|
|
|
| Bucket access logging | Planned | `unimplemented_tests.txt` |
|
|
| POST Object form upload checksum handling | Planned | `unimplemented_tests.txt` |
|
|
| Bucket ownership controls | Planned | `unimplemented_tests.txt` |
|
|
| Multipart upload listing and part lookup compatibility edge cases | Not part of default gate | `excluded_tests.txt` |
|
|
| IAM-account or multi-storage-class dependent cases | Not part of default gate | `unimplemented_tests.txt` |
|
|
| Tenanted bucket policy edge cases | Needs investigation | `unimplemented_tests.txt` |
|
|
|
|
## Intentional Exclusions
|
|
|
|
`excluded_tests.txt` contains tests that should not block the RustFS
|
|
compatibility gate. They fall into two classes:
|
|
|
|
- vendor-specific or non-portable behavior not required for RustFS S3
|
|
compatibility;
|
|
- intentionally unsupported product behavior, such as ACL authorization.
|
|
|
|
## Update Rule
|
|
|
|
When a planned S3 feature is implemented, move its passing test entries from
|
|
`unimplemented_tests.txt` to `implemented_tests.txt`, update this matrix, and
|
|
avoid changing README wording beyond the supported coverage.
|