feat(ecstore): add on-demand migration write-back pipeline (#7079)

* feat(ecstore): add on-demand migration pull queue and write-back pipeline

Background pull queue per bucket (bounded by pull_queue_capacity, concurrency via the state's pull slot), OdmWriteBack/PullSource traits, single-part and multipart write-back with a pumped body that enforces idle timeout, cancel and content length, retry policy for retryable source errors, inline commit helper, and stats accounting (rustfs/backlog#2153).

* feat(object): implement on-demand migration write-back over internal put

OdmWriteBack impl mapping source heads onto InternalPutContext (content-header allowlist, x-amz-meta copy, tags, dual-prefix odm-* provenance, ETag policy), injected into OnDemandMigrationSys at startup; removes the dead-code gates left by ODM-06a (rustfs/backlog#2153).
This commit is contained in:
Zhengchao An
2026-09-03 02:45:24 +08:00
committed by GitHub
parent 5be9d255e6
commit 0fe6cc3641
10 changed files with 2630 additions and 12 deletions
+5
View File
@@ -159,6 +159,11 @@ pub mod bucket {
OnDemandMigrationConfig, OnDemandMigrationConfigError, PathStyle, PolicyConfig, Provider, RangeGetPolicy,
SourceConfig, SourceCredentials, SourceErrorPolicy, SourceTimeout, TlsConfig, ValidationContext,
};
pub use crate::bucket::on_demand_migration::{
EnqueueOutcome, LocalObject, MAX_MULTIPART_PARTS, OdmWriteBack, PULL_MAX_RETRIES, PULL_RETRY_BASE_DELAYS,
PullCompletion, PullQueue, PullReason, PullSource, SourceBody, WriteBackBody, WriteBackError, WriteBackOutcome,
WriteBackPart, WriteBackRequest, commit_inline, commit_inline_with,
};
pub mod source_client {
pub use crate::bucket::on_demand_migration::source_client::{
SourceClient, SourceClientSpec, SourceError, SourceGet, SourceHead, SourceObject, SourcePage, SourceProbe,