gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-07 12:35:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(odm): merge the source listing into ListObjectsV2 (#7112)

Browse Source 
* feat(odm): merge the source listing into ListObjectsV2

Adds policy.list_through: ListObjectsV2 merges the local and source
listings into one ordered page so clients see the whole namespace during
an on-demand migration. Local entries win a key both sides hold,
CommonPrefixes are unioned under a delimiter, and the continuation token
is an opaque versioned envelope carrying both cursors.

A source listing failure or an open breaker follows policy.source_error:
propagate answers 424, not_found answers from local state and marks the
response x-rustfs-on-demand-migration-list: local_only. Source listings
are capped at 10 per second per bucket.

* test(odm): refresh the e2e-full darwin selection digest

The list-through e2e module adds seven cases to the merge lane.
This commit is contained in:
Zhengchao An
2026-09-04 09:08:35 +08:00
committed by GitHub
parent 80c88a9031
commit 7dfc2ee5f0
22 changed files with 1914 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files
rustfs/src/app/object/shared.rs
+12
View File
@@ -799,6 +799,12 @@ pub(super) fn object_lock_checks_required_for_state(state: &metadata_sys::Object
pub(crate) const ON_DEMAND_MIGRATION_HEADER: http::HeaderName = http::HeaderName::from_static("x-rustfs-on-demand-migration");
pub(crate) const ON_DEMAND_MIGRATION_SOURCE: HeaderValue = HeaderValue::from_static("source");
/// Response header marking a `ListObjectsV2` that could not consult the source
/// and was answered from local state alone (rustfs/backlog#2164).
pub(crate) const ON_DEMAND_MIGRATION_LIST_HEADER: http::HeaderName =
http::HeaderName::from_static("x-rustfs-on-demand-migration-list");
pub(crate) const ON_DEMAND_MIGRATION_LIST_LOCAL_ONLY: HeaderValue = HeaderValue::from_static("local_only");
/// Custom S3 error code for a source failure surfaced under
/// `policy.source_error = propagate`; carried on HTTP 424.
pub(crate) const ODM_SOURCE_UNAVAILABLE_CODE: &str = "SourceUnavailable";
@@ -872,6 +878,12 @@ pub(crate) fn mark_on_demand_migration_response(headers: &mut HeaderMap) {
headers.insert(ON_DEMAND_MIGRATION_HEADER, ON_DEMAND_MIGRATION_SOURCE);
}
/// Marks a merged listing that fell back to local state because the source
/// could not be consulted (rustfs/backlog#2164).
pub(crate) fn mark_on_demand_migration_list_local_only(headers: &mut HeaderMap) {
headers.insert(ON_DEMAND_MIGRATION_LIST_HEADER, ON_DEMAND_MIGRATION_LIST_LOCAL_ONLY);
}
/// Evaluates the request's conditional headers (`If-Match`, `If-None-Match`,
/// `If-Modified-Since`, `If-Unmodified-Since`) against the source's view of
/// the object, with the same S3 semantics the local path applies to