mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 08:18:18 +00:00
ci(ilm): add gated s3-tests lane for lifecycle expiration cases (#4715)
* ci(ilm): move first batch of 5 s3-tests lifecycle expiration cases into a gated behavior lane (backlog#1148 ilm-10)
The 20 lifecycle cases in excluded_tests.txt were labeled "vendor-specific"
but the real blocker was the absence of a Ceph lc_debug_interval equivalent.
RUSTFS_ILM_DEBUG_DAY_SECS (ilm-5) now provides that, so Days>=1 expiration
behavior is testable in seconds.
These cases assert that objects/versions/uploads are actually removed by the
background scanner and the stale-multipart cleanup loop. They cannot join the
default single-server s3-implemented-tests gate: it disables the scanner, and a
global RUSTFS_ILM_DEBUG_DAY_SECS also shrinks the x-amz-expiration header that
the already-passing test_lifecycle_expiration_header_* cases assert on. So this
adds an isolated lane instead of putting them in implemented_tests.txt.
First batch (5), each verified against the pinned upstream source and the
RustFS evaluator/scanner/stale-multipart execution paths:
- test_lifecycle_expiration (prefix Days=1/Days=5, scanner delete)
- test_lifecyclev2_expiration (same via ListObjectsV2)
- test_lifecycle_noncur_expiration (NoncurrentVersionExpiration)
- test_lifecycle_deletemarker_expiration (ExpiredObjectDeleteMarker cascade)
- test_lifecycle_multipart_expiration (AbortIncompleteMultipartUpload)
Dropped from the batch, kept excluded with reason:
- test_lifecycle_expiration_days0: RustFS accepts Expiration{Days:0} (returns
200; only Days<0 is rejected in crates/lifecycle/src/core.rs validate()),
while AWS/this test expect InvalidArgument. Real validation gap.
Changes:
- scripts/s3-tests/lifecycle_behavior_tests.txt: new exact-node-id run set.
- scripts/s3-tests/run.sh: honor an IMPLEMENTED_TESTS_FILE override so a lane
can point at an alternate whitelist.
- .github/workflows/ci.yml: new s3-lifecycle-behavior-tests PR-gate job that
starts rustfs with RUSTFS_ILM_DEBUG_DAY_SECS=10, scanner enabled (cycle 2s,
no start delay) and a 2s stale-multipart cleanup interval, running the new
list serially.
- scripts/s3-tests/report_compat.py: recognize the behavior list so the weekly
scope=all sweep (plain server) does not misclassify expected failures.
- scripts/s3-tests/excluded_tests.txt: remove the 5 enabled cases; re-annotate
the remaining 15 lifecycle exclusions with real reasons + batch-2 plan.
- docs/architecture/s3-compatibility-matrix.md: sync counts (implemented 451,
excluded 274, behavior 5) and document the lane.
Refs backlog#1148 (ilm-10), master plan backlog#1155.
* fix(lifecycle): reject zero-day expiration/noncurrent/abort rules (backlog#1148 ilm-10) (#4722)
This commit is contained in:
@@ -495,3 +495,61 @@ jobs:
|
||||
path: artifacts/s3tests-single/**
|
||||
if-no-files-found: ignore
|
||||
retention-days: 3
|
||||
|
||||
# Dedicated lifecycle behavior lane (backlog#1148 ilm-10, master plan #1155).
|
||||
#
|
||||
# These ceph/s3-tests cases assert that objects/versions/uploads are actually
|
||||
# removed by the background scanner and the stale-multipart cleanup loop, so
|
||||
# they need a debug-accelerated day plus an enabled scanner. That configuration
|
||||
# cannot be applied to the s3-implemented-tests lane above because a global
|
||||
# RUSTFS_ILM_DEBUG_DAY_SECS also shrinks the x-amz-expiration header those tests
|
||||
# assert on (test_lifecycle_expiration_header_*). Hence a separate server here.
|
||||
s3-lifecycle-behavior-tests:
|
||||
name: S3 Lifecycle Behavior Tests
|
||||
needs: [ build-rustfs-debug-binary ]
|
||||
runs-on: sm-standard-4
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
|
||||
|
||||
- name: Download debug binary
|
||||
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
|
||||
with:
|
||||
name: rustfs-debug-binary
|
||||
path: target/debug
|
||||
|
||||
- name: Make binary executable
|
||||
run: chmod +x ./target/debug/rustfs
|
||||
|
||||
- name: Run lifecycle behavior s3-tests
|
||||
run: |
|
||||
RUN_ROOT="${RUNNER_TEMP}/rustfs-s3tests-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${GITHUB_JOB}"
|
||||
mkdir -p "${RUN_ROOT}"
|
||||
S3_PORT="$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1]); s.close()')"
|
||||
# Serial (XDIST=0) for timing determinism: the cases sleep on multiples
|
||||
# of the debug day and assert scanner-driven deletion.
|
||||
DEPLOY_MODE=binary \
|
||||
RUSTFS_BINARY=./target/debug/rustfs \
|
||||
TEST_MODE=single \
|
||||
MAXFAIL=0 \
|
||||
XDIST=0 \
|
||||
IMPLEMENTED_TESTS_FILE=scripts/s3-tests/lifecycle_behavior_tests.txt \
|
||||
RUSTFS_ILM_DEBUG_DAY_SECS=10 \
|
||||
RUSTFS_SCANNER_ENABLED=true \
|
||||
RUSTFS_SCANNER_CYCLE=2 \
|
||||
RUSTFS_SCANNER_START_DELAY_SECS=0 \
|
||||
RUSTFS_API_STALE_UPLOADS_CLEANUP_INTERVAL=2s \
|
||||
S3_PORT="${S3_PORT}" \
|
||||
DATA_ROOT="${RUN_ROOT}" \
|
||||
S3TESTS_CONF=artifacts/s3tests-single/s3tests.conf \
|
||||
./scripts/s3-tests/run.sh
|
||||
|
||||
- name: Upload s3 test artifacts
|
||||
if: always()
|
||||
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
|
||||
with:
|
||||
name: s3tests-lifecycle-behavior-${{ github.run_number }}
|
||||
path: artifacts/s3tests-single/**
|
||||
if-no-files-found: ignore
|
||||
retention-days: 3
|
||||
|
||||
Reference in New Issue
Block a user