gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-27 08:38:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: prefer app context object store in ecfs (#3433)

Browse Source
This commit is contained in:
安正超
2026-06-14 15:52:26 +08:00
committed by GitHub
parent 046d5386ba
commit 5e68cf8a29
2 changed files with 45 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/migration-progress.md
+21 -14
View File
@@ -5,16 +5,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-admin-object-store-context`
- Baseline: `upstream/main` at `6fcd62ba5648f90f6a78a0819f54e4a881cc2a87`
- Branch: `overtrue/arch-storage-ecfs-object-store-context`
- Baseline: `upstream/main` at `046d5386ba9ffe6a0df39ffd78322c62e9f358d2`
- PR type for this branch: `consumer-migration`
- Runtime behavior changes: no external behavior change expected; admin
- Runtime behavior changes: no external behavior change expected; S3 ECFS
object-store lookups prefer the AppContext-owned object store and keep the
existing global fallback.
- Rust code changes: add global fallback to the shared object-store resolver and
migrate admin handlers, services, and router helpers to that resolver.
- Rust code changes: migrate `rustfs/src/storage/ecfs.rs` object-store lookups
to the shared resolver without touching lower-layer storage infra modules.
- CI/script changes: none.
- Docs changes: record `CTX-005` consumer migration scope and verification.
- Docs changes: record `CTX-006` consumer migration scope and verification.
## Phase 0 Tasks
@@ -174,6 +174,14 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
preserve the existing global object-layer fallback when absent.
- Verification: focused resolver test, formatting, compile check, migration
guards, diff hygiene, Rust risk scan, and full `make pre-commit`.
- [x] `CTX-006` Migrate ECFS object-store consumers.
- Do: migrate S3 ECFS object operations to the shared object-store resolver.
- Acceptance: ECFS object-store lookups use AppContext when present and
preserve the existing global object-layer fallback when absent.
- Must preserve: S3 object/bucket API behavior, object-lock/tagging/metadata
semantics, and existing storage error paths.
- Verification: formatting, compile check, migration guards, diff hygiene,
Rust risk scan, and full `make pre-commit`.
## Phase 1 Security Governance Tasks
@@ -481,7 +489,7 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Next PRs
1. `consumer-migration`: migrate the next low-risk non-admin AppContext
consumer group while preserving global fallback.
consumer group that does not introduce a new reverse layer dependency.
2. `pure-move`: start `R-009` boot wrapper with the IAM degraded readiness
contract covered.
@@ -489,17 +497,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | pass | Single `consumer-migration` slice; the shared object-store resolver owns fallback behavior and admin callers do not add service containers, registries, or cross-layer dependencies. |
| Migration preservation | pass | AppContext object-store resolution is preferred when present, existing global object-layer fallback and admin error paths are preserved, and method bodies keep their storage behavior. |
| Testing/verification | pass | Focused resolver test, formatting, compile check, diff hygiene, migration guards, added-line Rust risk scan, and full `make pre-commit` passed. |
| Quality/architecture | pass | Single `consumer-migration` slice; `ecfs.rs` is an interface-layer module already using app usecases, and the layer guard reports no new reverse dependencies. |
| Migration preservation | pass | AppContext object-store resolution is preferred when present, existing global object-layer fallback and S3 storage error paths are preserved, and ECFS method bodies keep their behavior. |
| Testing/verification | pass | Formatting, compile check, diff hygiene, migration/layer guards, added-line Rust risk scan, and full `make pre-commit` passed. |
## Verification Notes
Passed on `6fcd62ba5648f90f6a78a0819f54e4a881cc2a87`:
Passed on `046d5386ba9ffe6a0df39ffd78322c62e9f358d2`:
- `cargo fmt --all`.
- `cargo fmt --all --check`.
- `cargo check -p rustfs --lib`.
- `cargo test -p rustfs app::context::compat::tests::resolver_helpers_are_context_first_and_fallback_when_context_is_absent --lib`.
- `git diff --check`.
- `./scripts/check_architecture_migration_rules.sh`.
- `./scripts/check_layer_dependencies.sh`.
@@ -510,12 +517,12 @@ Passed on `6fcd62ba5648f90f6a78a0819f54e4a881cc2a87`:
111 skipped, plus doctests.
Notes:
- This slice migrates one coherent admin object-store consumer group.
- This slice migrates one coherent ECFS object-store consumer group.
- Global object-layer fallback remains available until the planned cleanup
phase.
## Handoff Notes
- CTX-005 is complete.
- CTX-006 is complete.
- Keep the next consumer or boot wrapper PR scoped to one PR type and preserve
global fallback until the planned cleanup phase.
rustfs/src/storage/ecfs.rs
+24 -24
View File
@@ -13,6 +13,7 @@
// limitations under the License.
use crate::app::bucket_usecase::DefaultBucketUsecase;
use crate::app::context::resolve_object_store_handle;
use crate::app::multipart_usecase::DefaultMultipartUsecase;
use crate::app::object_usecase::DefaultObjectUsecase;
use crate::error::ApiError;
@@ -39,7 +40,6 @@ use rustfs_ecstore::{
versioning_sys::BucketVersioningSys,
},
error::{StorageError, is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found},
new_object_layer_fn,
store_api::{BucketOperations, ObjectLockRetentionOptions, ObjectOperations, ObjectOptions},
};
use rustfs_io_metrics::record_s3_op;
@@ -105,7 +105,7 @@ impl FS {
object: &str,
version_id: Option<&str>,
) -> S3Result<std::collections::HashMap<String, Vec<String>>> {
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Ok(std::collections::HashMap::new());
};
let opts = ObjectOptions {
@@ -349,7 +349,7 @@ impl S3 for FS {
&self,
req: S3Request<DeleteBucketWebsiteInput>,
) -> S3Result<S3Response<DeleteBucketWebsiteOutput>> {
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(s3_error!(InternalError, "Not init"));
};
@@ -398,7 +398,7 @@ impl S3 for FS {
validate_table_catalog_object_mutation(&bucket, &object).await?;
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
error!(
component = LOG_COMPONENT_STORAGE,
subsystem = LOG_SUBSYSTEM_TAGGING,
@@ -479,7 +479,7 @@ impl S3 for FS {
record_s3_op(S3Operation::GetBucketAcl, &req.input.bucket);
let GetBucketAclInput { bucket, .. } = req.input;
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -495,7 +495,7 @@ impl S3 for FS {
&self,
req: S3Request<GetBucketAccelerateConfigurationInput>,
) -> S3Result<S3Response<GetBucketAccelerateConfigurationOutput>> {
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(s3_error!(InternalError, "Not init"));
};
@@ -585,7 +585,7 @@ impl S3 for FS {
&self,
req: S3Request<GetBucketRequestPaymentInput>,
) -> S3Result<S3Response<GetBucketRequestPaymentOutput>> {
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(s3_error!(InternalError, "Not init"));
};
@@ -633,7 +633,7 @@ impl S3 for FS {
}
async fn get_bucket_website(&self, req: S3Request<GetBucketWebsiteInput>) -> S3Result<S3Response<GetBucketWebsiteOutput>> {
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(s3_error!(InternalError, "Not init"));
};
@@ -671,7 +671,7 @@ impl S3 for FS {
bucket, key, version_id, ..
} = req.input;
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -701,7 +701,7 @@ impl S3 for FS {
bucket, key, version_id, ..
} = req.input.clone();
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -762,7 +762,7 @@ impl S3 for FS {
record_s3_op(S3Operation::GetObjectLockConfiguration, &req.input.bucket);
let GetObjectLockConfigurationInput { bucket, .. } = req.input;
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -810,7 +810,7 @@ impl S3 for FS {
bucket, key, version_id, ..
} = req.input.clone();
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -862,7 +862,7 @@ impl S3 for FS {
let bucket = req.input.bucket.as_str();
let object = req.input.key.as_str();
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
error!(
component = LOG_COMPONENT_STORAGE,
subsystem = LOG_SUBSYSTEM_TAGGING,
@@ -1020,7 +1020,7 @@ impl S3 for FS {
} = req.input;
record_s3_op(S3Operation::PutBucketAcl, &bucket);
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -1043,7 +1043,7 @@ impl S3 for FS {
&self,
req: S3Request<PutBucketAccelerateConfigurationInput>,
) -> S3Result<S3Response<PutBucketAccelerateConfigurationOutput>> {
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(s3_error!(InternalError, "Not init"));
};
store
@@ -1068,7 +1068,7 @@ impl S3 for FS {
async fn get_bucket_logging(&self, req: S3Request<GetBucketLoggingInput>) -> S3Result<S3Response<GetBucketLoggingOutput>> {
record_s3_op(S3Operation::GetBucketLogging, &req.input.bucket);
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(s3_error!(InternalError, "Not init"));
};
store
@@ -1087,7 +1087,7 @@ impl S3 for FS {
async fn put_bucket_logging(&self, req: S3Request<PutBucketLoggingInput>) -> S3Result<S3Response<PutBucketLoggingOutput>> {
record_s3_op(S3Operation::PutBucketLogging, &req.input.bucket);
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(s3_error!(InternalError, "Not init"));
};
store
@@ -1146,7 +1146,7 @@ impl S3 for FS {
&self,
req: S3Request<PutBucketRequestPaymentInput>,
) -> S3Result<S3Response<PutBucketRequestPaymentOutput>> {
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(s3_error!(InternalError, "Not init"));
};
store
@@ -1188,7 +1188,7 @@ impl S3 for FS {
}
async fn put_bucket_website(&self, req: S3Request<PutBucketWebsiteInput>) -> S3Result<S3Response<PutBucketWebsiteOutput>> {
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(s3_error!(InternalError, "Not init"));
};
store
@@ -1218,7 +1218,7 @@ impl S3 for FS {
let key = &req.input.key;
let version_id = req.input.version_id.clone();
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -1260,7 +1260,7 @@ impl S3 for FS {
validate_table_catalog_object_mutation(&bucket, &key).await?;
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -1313,7 +1313,7 @@ impl S3 for FS {
let Some(input_cfg) = object_lock_configuration else { return Err(s3_error!(InvalidArgument)) };
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -1384,7 +1384,7 @@ impl S3 for FS {
validate_table_catalog_object_mutation(&bucket, &key).await?;
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
@@ -1462,7 +1462,7 @@ impl S3 for FS {
crate::storage::s3_api::tagging::validate_object_tag_set(&tagging.tag_set)?;
let Some(store) = new_object_layer_fn() else {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};