refactor(ecstore): extract the embedded S3 client into rustfs-s3-client (#6627)

The storage engine embedded a ~8.4K-line hand-written S3 HTTP client under crates/ecstore/src/client (rustfs/backlog#1842). That client is a legitimate engine capability — it consumes remote S3-compatible endpoints for ILM tier warm backends and transition targets — but it was misfiled inside the engine, dragging s3s/hyper wire types into ecstore and blocking ARCHITECTURE.md invariant 4.

This PR is the pure-move step: 21 modules move verbatim to the new crates/s3-client crate (rustfs-s3-client), and crates/ecstore/src/client/mod.rs becomes a re-export shim so every in-crate crate::client:: path keeps working. The two server-side modules that were historically misfiled under client/ — object_api_utils.rs and object_handlers_common.rs — stay in ecstore.

Three reverse dependencies from the client into engine internals are severed so the move can be pure:

- transition_api::ReaderImpl::ObjectBody held ecstore's GetObjectReader; the client only ever reads the body, so the variant now holds an ObjectReader newtype over Box<dyn AsyncRead + Send + Sync + Unpin> with the same read_all() surface. The single production construction site (set_disk transition upload) and the two engine-side consumers were adjusted.
- api_list/api_remove used ecstore's storage_api_contracts / object_api types; api_list now imports BucketInfo from rustfs-storage-api directly, and api_remove uses the client's own transition_api::ObjectInfo (only .name/.version_id were read; the error-path bucket name is now threaded as a parameter instead of read from the deleted objects).
- the api_put_object_streaming regression tests built a GetObjectReader by hand; they now wrap the duplex stream in ObjectReader::new.

Guard updates: the s3s footprint ratchet gains an ecstore-scoped counter (42 files, shrink-only, per rustfs/backlog#1842), the ecstore module-lint-blanket register follows the moved files into crates/s3-client so the blanket ratchet keeps covering them, the logging guardrail path pin follows transition_api.rs, and the ::other(format!) baseline is regenerated (moved call sites left ecstore).

Verification: cargo check -p rustfs-s3-client -p rustfs-ecstore; cargo nextest run -p rustfs-s3-client (43 passed) and -p rustfs-ecstore (4515/4523; the 8 failures reproduce identically on pristine origin/main on the same machine); cargo clippy --all-targets; scripts/check_layer_dependencies.sh, check_architecture_migration_rules.sh, check_s3s_footprint.sh, check_logging_guardrails.sh, check_error_other_format_ratchet.sh, check_doc_paths.sh, check_ci_paths_sync.sh all pass.
This commit is contained in:
Zhengchao An
2026-08-26 12:38:52 +08:00
committed by GitHub
parent 65a7cc9cd4
commit 8f0d4a20d1
35 changed files with 352 additions and 182 deletions
Generated
+46
View File
@@ -9552,6 +9552,7 @@ dependencies = [
"rustfs-replication",
"rustfs-rio",
"rustfs-rio-v2",
"rustfs-s3-client",
"rustfs-s3-types",
"rustfs-signer",
"rustfs-storage-api",
@@ -10316,6 +10317,51 @@ dependencies = [
"walkdir",
]
[[package]]
name = "rustfs-s3-client"
version = "1.0.0-rc.3"
dependencies = [
"base64-simd",
"bytes",
"enumset",
"futures",
"futures-util",
"hex-simd",
"http 1.5.0",
"http-body 1.1.0",
"http-body-util",
"hyper",
"hyper-rustls",
"hyper-util",
"lazy_static",
"md-5 0.11.0",
"quick-xml 0.42.0",
"rand 0.10.2",
"rustfs-checksums",
"rustfs-config",
"rustfs-rio",
"rustfs-signer",
"rustfs-storage-api",
"rustfs-tls-runtime",
"rustfs-utils",
"rustls",
"rustls-pki-types",
"s3s",
"serde",
"serde_json",
"sha1 0.11.0",
"sha2 0.11.0",
"thiserror 2.0.20",
"time",
"tokio",
"tokio-util",
"tower",
"tracing",
"url",
"urlencoding",
"uuid",
]
[[package]]
name = "rustfs-s3-ops"
version = "1.0.0-rc.3"