gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-28 17:18:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

test(lifecycle): regression test for expire/GET race (#3491) (#4706)

Browse Source 
Adds a serial-lane ILM integration regression test that pins the
local-first ordering contract established by rustfs#3491, where
`expire_transitioned_object` deletes local metadata BEFORE any remote
tier cleanup so a concurrent GET can never observe live local metadata
pointing at an already-removed remote tier version (user-visible
`NoSuchVersion`).

`serial_tests::test_expire_transitioned_object_never_races_concurrent_get`
in `crates/scanner/tests/lifecycle_integration_test.rs`:

- Transitions an object to a mock warm tier, then runs a tight
  concurrent GET loop while calling `expire_transitioned_object`; every
  GET must return a full, correct body or a clean object/version-not-found
  -- never a tier-fetch failure.
- Deterministic ordering assertion (revert-proof): immediately after
  expiry the remote tier object is still present and the mock recorded
  zero remote `remove` calls, proving remote cleanup is deferred to
  free-version recovery. Reverting to remote-first ordering turns this
  assertion red.

Supporting changes:
- Re-export `expire_transitioned_object` from `api::bucket::lifecycle`.
- Record remote-tier mutating ops in the scanner test `MockWarmBackend`.
- Update tier-ilm-debugging.md to reference the new regression test.

Runs in the CI ILM Integration (serial) lane (ci.yml
test-ilm-integration-serial), picked up by the existing
binary(lifecycle_integration_test) filter.

Refs: rustfs/backlog#1148 (ilm-2), rustfs/backlog#1155
This commit is contained in:
Zhengchao An
2026-07-11 09:53:24 +08:00
committed by GitHub
parent 5c7c757a30
commit ce53feee87
4 changed files with 203 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operations/tier-ilm-debugging.md
+5 -1
View File
@@ -83,7 +83,11 @@ was transitioned to a non-versioned tier bucket and no versionId must be sent.
remote-tier cleanup is driven by persisted free-version recovery
(`crates/ecstore/src/bucket/lifecycle/tier_free_version_recovery.rs`).
**Invariant — keep local-first ordering**: never remove a remote tier
version while live local metadata still points at it.
version while live local metadata still points at it. Regression test:
`serial_tests::test_expire_transitioned_object_never_races_concurrent_get`
in `crates/scanner/tests/lifecycle_integration_test.rs` (runs in the CI
ILM Integration serial lane) pins both the local-first ordering and the
"concurrent GET never sees `NoSuchVersion`" contract.
- Nil-UUID versionId sent to tier (`NoSuchVersion`): reading code used
`Uuid::from_slice(..).unwrap_or_default()`, converting an empty metadata
value into `Uuid::nil()`. Fixed by the `and_then`/`filter` pattern above.