gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-11 23:56:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

test(lifecycle): regression test for expire/GET race (#3491) (#4706)

Browse Source 
Adds a serial-lane ILM integration regression test that pins the
local-first ordering contract established by rustfs#3491, where
`expire_transitioned_object` deletes local metadata BEFORE any remote
tier cleanup so a concurrent GET can never observe live local metadata
pointing at an already-removed remote tier version (user-visible
`NoSuchVersion`).

`serial_tests::test_expire_transitioned_object_never_races_concurrent_get`
in `crates/scanner/tests/lifecycle_integration_test.rs`:

- Transitions an object to a mock warm tier, then runs a tight
  concurrent GET loop while calling `expire_transitioned_object`; every
  GET must return a full, correct body or a clean object/version-not-found
  -- never a tier-fetch failure.
- Deterministic ordering assertion (revert-proof): immediately after
  expiry the remote tier object is still present and the mock recorded
  zero remote `remove` calls, proving remote cleanup is deferred to
  free-version recovery. Reverting to remote-first ordering turns this
  assertion red.

Supporting changes:
- Re-export `expire_transitioned_object` from `api::bucket::lifecycle`.
- Record remote-tier mutating ops in the scanner test `MockWarmBackend`.
- Update tier-ilm-debugging.md to reference the new regression test.

Runs in the CI ILM Integration (serial) lane (ci.yml
test-ilm-integration-serial), picked up by the existing
binary(lifecycle_integration_test) filter.

Refs: rustfs/backlog#1148 (ilm-2), rustfs/backlog#1155
This commit is contained in:
Zhengchao An
2026-07-11 09:53:24 +08:00
committed by GitHub
parent 5c7c757a30
commit ce53feee87
4 changed files with 203 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/scanner/tests/storage_api/mod.rs
+9 -6
View File
@@ -13,8 +13,9 @@
// limitations under the License.
pub(crate) use rustfs_ecstore::api::bucket::lifecycle::{
bucket_lifecycle_ops::{enqueue_transition_for_existing_objects, init_background_expiry},
lifecycle::TransitionOptions,
bucket_lifecycle_audit::LcEventSrc,
bucket_lifecycle_ops::{enqueue_transition_for_existing_objects, expire_transitioned_object, init_background_expiry},
lifecycle::{Event as LcEvent, IlmAction, TransitionOptions},
};
pub(crate) use rustfs_ecstore::api::bucket::metadata::BUCKET_LIFECYCLE_CONFIG;
pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{
@@ -24,6 +25,7 @@ pub(crate) use rustfs_ecstore::api::bucket::versioning_sys::BucketVersioningSys;
pub(crate) use rustfs_ecstore::api::capacity::path2_bucket_object_with_base_path;
pub(crate) use rustfs_ecstore::api::client::transition_api::{ReadCloser, ReaderImpl};
pub(crate) use rustfs_ecstore::api::disk::{DiskAPI, DiskOption, STORAGE_FORMAT_FILE, endpoint::Endpoint, new_disk};
pub(crate) use rustfs_ecstore::api::error::{Error as EcstoreError, is_err_object_not_found, is_err_version_not_found};
pub(crate) use rustfs_ecstore::api::layout::{EndpointServerPools, Endpoints, PoolEndpoints};
pub(crate) use rustfs_ecstore::api::runtime::global_tier_config_mgr as get_global_tier_config_mgr;
pub(crate) use rustfs_ecstore::api::storage::{ECStore, init_local_disks};
@@ -40,10 +42,11 @@ pub(crate) mod lifecycle {
};
pub(crate) use super::{
BUCKET_LIFECYCLE_CONFIG, BucketVersioningSys, DiskAPI, DiskOption, ECStore, Endpoint, EndpointServerPools, Endpoints,
PoolEndpoints, ReadCloser, ReaderImpl, STORAGE_FORMAT_FILE, ScannerWarmBackend, TierConfig, TierMinIO, TierType,
TransitionOptions, WarmBackendGetOpts, build_transition_put_options, enqueue_transition_for_existing_objects,
get_bucket_metadata, get_global_tier_config_mgr, init_background_expiry, init_bucket_metadata_sys, init_local_disks,
BUCKET_LIFECYCLE_CONFIG, BucketVersioningSys, DiskAPI, DiskOption, ECStore, EcstoreError, Endpoint, EndpointServerPools,
Endpoints, IlmAction, LcEvent, LcEventSrc, PoolEndpoints, ReadCloser, ReaderImpl, STORAGE_FORMAT_FILE,
ScannerWarmBackend, TierConfig, TierMinIO, TierType, TransitionOptions, WarmBackendGetOpts, build_transition_put_options,
enqueue_transition_for_existing_objects, expire_transitioned_object, get_bucket_metadata, get_global_tier_config_mgr,
init_background_expiry, init_bucket_metadata_sys, init_local_disks, is_err_object_not_found, is_err_version_not_found,
new_disk, path2_bucket_object_with_base_path, update_bucket_metadata,
};
}