refactor: remove app dto wildcard imports (#3853)

This commit is contained in:
Zhengchao An
2026-06-25 16:07:49 +08:00
committed by GitHub
parent fe0227e9dc
commit 2dd98b1f84
8 changed files with 98 additions and 16 deletions
+24 -6
View File
@@ -5,9 +5,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-app-usecase-context-import-batch`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208`.
- Based on: stacked on PR #3848 head while PR #3848 is pending.
- Branch: `overtrue/arch-app-explicit-dto-imports`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209`.
- Based on: stacked on API-208 while PR #3848 is pending.
- PR type for this branch: `consumer-migration`
- Runtime behavior changes: none.
- Rust code changes: route replication pool, outbound TLS generation, runtime
@@ -45,7 +45,7 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
readiness-time, observability, metrics, buffer, and TLS runtime source
helpers, plus server readiness/audit/event/module-switch runtime source
helpers, storage request/RPC/SSE runtime source helpers, and admin
handler/service/router runtime source helpers, plus root auth/init/config/protocol/workload, app usecase, storage node-service, remaining admin grouped context import runtime source helpers, and app bucket/object/multipart usecase explicit storage imports,
handler/service/router runtime source helpers, plus root auth/init/config/protocol/workload, app usecase, storage node-service, remaining admin grouped context import runtime source helpers, app bucket/object/multipart usecase explicit storage imports, and app select/bucket/object/multipart explicit S3 DTO plus ECFS owner imports,
through AppContext-first or owner-crate resolver boundaries.
- CI/script changes: lock completed owner and test/fuzz boundaries against
bare/glob imports, scattered raw ECStore facade subpaths, and startup
@@ -56,8 +56,8 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
event-bridge thin module regressions, plus IAM runtime-source bypasses;
accept the reviewed AppContext resolver reverse dependencies in the layer
baseline, and block direct admin AppContext resolver consumers outside the
admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, and reject app usecase storage wildcard imports.
- Docs changes: record the API-136 through API-208 owner facade and lifecycle
admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, reject app usecase storage wildcard imports, reject app-layer S3 DTO and ECFS wildcard imports, and narrow the object-usecase ECFS layer baseline entry to `FS`.
- Docs changes: record the API-136 through API-209 owner facade and lifecycle
runtime-source cleanup.
## Phase 0 Tasks
@@ -4996,6 +4996,21 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
layer guards, diff hygiene, residual wildcard scan, Rust risk scan, fast PR
gate, and full PR gate before PR.
- [x] `API-209` Remove app DTO and ECFS wildcard imports.
- Do: replace broad `s3s::dto::*` imports in app select, bucket, object, and
multipart usecases, replace the object usecase ECFS wildcard with the
explicit owner type, and narrow the matching layer baseline entry.
- Acceptance: app consumers no longer import S3 DTOs or ECFS owner APIs
through wildcards, and migration rules reject reintroducing those wildcard
imports in app-layer files.
- Must preserve: select-object request validation and event streaming, bucket
DTO mapping/list output behavior, object CRUD/copy/delete/restore DTO
mapping, multipart upload/copy checksum behavior, and existing ECFS FS test
harness usage.
- Verification: focused RustFS app compile/tests, formatting, migration and
layer guards, diff hygiene, residual wildcard scan, Rust risk scan, fast PR
gate, and full PR gate before PR.
## Next PRs
1. `consumer-migration`: continue reducing direct global reads behind AppContext resolver boundaries.
@@ -5004,6 +5019,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | pass | API-209 removes app-layer S3 DTO and ECFS wildcard imports and adds a regression guard for the completed boundary. |
| Migration preservation | pass | Select-object streaming, bucket DTO mapping, object CRUD/copy/delete/restore DTO handling, multipart checksum/copy behavior, and ECFS FS test harness usage keep the same call paths. |
| Testing/verification | pass | Focused RustFS app compile/tests, formatting, migration/layer guards, residual wildcard scan, Rust risk scan, fast PR gate, and full PR gate are planned before PR. |
| Quality/architecture | pass | API-208 removes broad storage wildcard imports from app bucket, object, and multipart usecases and adds a regression guard. |
| Migration preservation | pass | Bucket listing/notification, object precondition/SSE/object-lock/CORS, and multipart SSE/object-lock behavior keep the same storage owner call sites. |
| Testing/verification | pass | Focused app compile/tests, formatting, migration/layer guards, residual wildcard scan, Rust risk scan, fast PR gate, and full PR gate are planned before PR. |
+29 -1
View File
@@ -79,7 +79,30 @@ use rustfs_trusted_proxies::ClientInfo;
use rustfs_utils::http::{SUFFIX_FORCE_DELETE, get_header};
use rustfs_utils::obj::extract_user_defined_metadata;
use rustfs_utils::string::parse_bool;
use s3s::dto::*;
use s3s::dto::{
BucketLifecycleConfiguration, BucketLocationConstraint, CommonPrefix, CreateBucketInput, CreateBucketOutput,
DeleteBucketCorsInput, DeleteBucketCorsOutput, DeleteBucketEncryptionInput, DeleteBucketEncryptionOutput, DeleteBucketInput,
DeleteBucketLifecycleInput, DeleteBucketLifecycleOutput, DeleteBucketOutput, DeleteBucketPolicyInput,
DeleteBucketPolicyOutput, DeleteBucketReplicationInput, DeleteBucketReplicationOutput, DeleteBucketTaggingInput,
DeleteBucketTaggingOutput, DeleteMarkerM, DeletePublicAccessBlockInput, DeletePublicAccessBlockOutput, EncodingType,
ExpirationStatus, GetBucketCorsInput, GetBucketCorsOutput, GetBucketEncryptionInput, GetBucketEncryptionOutput,
GetBucketLifecycleConfigurationInput, GetBucketLifecycleConfigurationOutput, GetBucketLocationInput, GetBucketLocationOutput,
GetBucketNotificationConfigurationInput, GetBucketNotificationConfigurationOutput, GetBucketPolicyInput,
GetBucketPolicyOutput, GetBucketPolicyStatusInput, GetBucketPolicyStatusOutput, GetBucketReplicationInput,
GetBucketReplicationOutput, GetBucketTaggingInput, GetBucketTaggingOutput, GetBucketVersioningInput,
GetBucketVersioningOutput, GetPublicAccessBlockInput, GetPublicAccessBlockOutput, HeadBucketInput, HeadBucketOutput,
LifecycleRule, ListBucketsInput, ListBucketsOutput, ListObjectVersionMEntry, ListObjectVersionsInput,
ListObjectVersionsMOutput, ListObjectVersionsOutput, ListObjectsInput, ListObjectsOutput, ListObjectsV2Input,
ListObjectsV2MOutput, ListObjectsV2Output, NotificationConfiguration, NotificationConfigurationFilter, ObjectInternalInfo,
ObjectLockConfiguration, ObjectM, ObjectStorageClass, ObjectVersionM, ObjectVersionStorageClass, PolicyStatus,
PutBucketCorsInput, PutBucketCorsOutput, PutBucketEncryptionInput, PutBucketEncryptionOutput,
PutBucketLifecycleConfigurationInput, PutBucketLifecycleConfigurationOutput, PutBucketNotificationConfigurationInput,
PutBucketNotificationConfigurationOutput, PutBucketPolicyInput, PutBucketPolicyOutput, PutBucketReplicationInput,
PutBucketReplicationOutput, PutBucketTaggingInput, PutBucketTaggingOutput, PutBucketVersioningInput,
PutBucketVersioningOutput, PutPublicAccessBlockInput, PutPublicAccessBlockOutput, ReplicationConfiguration,
ReplicationRuleStatus, ServerSideEncryption, Tagging, Timestamp, UserMetadataCollection, UserMetadataEntry,
VersioningConfiguration,
};
use s3s::region::Region;
use s3s::xml;
use s3s::{S3Error, S3ErrorCode, S3Request, S3Response, S3Result, s3_error};
@@ -2186,6 +2209,11 @@ impl DefaultBucketUsecase {
mod tests {
use super::*;
use http::{Extensions, HeaderMap, Method, Uri};
use s3s::dto::{
BucketVersioningStatus, CORSConfiguration, Destination, ExcludedPrefix, FilterRule, FilterRuleName, LifecycleExpiration,
NoncurrentVersionTransition, PublicAccessBlockConfiguration, QueueConfiguration, ReplicationRule, S3KeyFilter,
ServerSideEncryptionConfiguration, Tag, Transition, TransitionStorageClass,
};
use std::sync::Arc;
fn build_request<T>(input: T, method: Method) -> S3Request<T> {
+7 -1
View File
@@ -71,7 +71,12 @@ use rustfs_utils::http::{
headers::{AMZ_DECODED_CONTENT_LENGTH, AMZ_OBJECT_TAGGING},
insert_str,
};
use s3s::dto::*;
use s3s::dto::{
AbortMultipartUploadInput, AbortMultipartUploadOutput, ChecksumAlgorithm, ChecksumType, CompleteMultipartUploadInput,
CompleteMultipartUploadOutput, CompletedPart, CopyPartResult, CopySource, CreateMultipartUploadInput,
CreateMultipartUploadOutput, ETag, ListMultipartUploadsInput, ListMultipartUploadsOutput, ListPartsInput, ListPartsOutput,
ServerSideEncryption, StreamingBlob, Timestamp, UploadPartCopyInput, UploadPartCopyOutput, UploadPartInput, UploadPartOutput,
};
use s3s::header::{X_AMZ_OBJECT_LOCK_LEGAL_HOLD, X_AMZ_OBJECT_LOCK_MODE, X_AMZ_OBJECT_LOCK_RETAIN_UNTIL_DATE};
use s3s::{S3Error, S3ErrorCode, S3Request, S3Response, S3Result, s3_error};
#[cfg(test)]
@@ -1338,6 +1343,7 @@ mod tests {
use rustfs_utils::http::{
AMZ_OBJECT_LOCK_LEGAL_HOLD_LOWER, AMZ_OBJECT_LOCK_MODE_LOWER, AMZ_OBJECT_LOCK_RETAIN_UNTIL_DATE_LOWER,
};
use s3s::dto::{CompletedMultipartUpload, StorageClass};
use std::{collections::HashMap, io::Cursor};
use temp_env::async_with_vars;
use tokio::io::AsyncReadExt;
+14 -4
View File
@@ -57,7 +57,7 @@ use crate::storage::concurrency::{
ConcurrencyManager, GetObjectGuard, PutObjectGuard, get_concurrency_aware_buffer_size, get_concurrency_manager,
get_put_concurrency_aware_buffer_size,
};
use crate::storage::ecfs::*;
use crate::storage::ecfs::FS;
use crate::storage::head_prefix::{head_prefix_not_found_message, probe_prefix_has_children};
use crate::storage::helper::{OperationHelper, spawn_background_with_context};
use crate::storage::options::{
@@ -123,7 +123,16 @@ use rustfs_utils::http::{
};
use rustfs_utils::path::{encode_dir_object, is_dir_object, path_join_buf};
use rustfs_zip::{ArchiveLimits, CompressionFormat};
use s3s::dto::*;
use s3s::dto::{
CacheControl, Checksum, ChecksumAlgorithm, ChecksumType, ContentDisposition, ContentEncoding, ContentLanguage, ContentType,
CopyObjectInput, CopyObjectOutput, CopyObjectResult, CopySource, DeleteObjectInput, DeleteObjectOutput, DeleteObjectsInput,
DeleteObjectsOutput, DeletedObject, ETag, GetObjectAttributesInput, GetObjectAttributesOutput, GetObjectAttributesParts,
GetObjectInput, GetObjectOutput, HeadObjectInput, HeadObjectOutput, MetadataDirective, ObjectAttributes, ObjectLockLegalHold,
ObjectLockLegalHoldStatus, ObjectLockMode, ObjectLockRetention, ObjectLockRetentionMode, ObjectPart, PutObjectInput,
PutObjectOutput, Range, ReplicationConfiguration, RequestCharged, RestoreObjectInput, RestoreObjectOutput, RestoreStatus,
SSECustomerAlgorithm, SSECustomerKeyMD5, SSEKMSKeyId, SelectObjectContentInput, SelectObjectContentOutput,
ServerSideEncryption, StorageClass, StreamingBlob, TaggingHeader, Timestamp, TimestampFormat, WebsiteRedirectLocation,
};
use s3s::header::{X_AMZ_RESTORE, X_AMZ_RESTORE_OUTPUT_PATH};
use s3s::{S3Error, S3ErrorCode, S3Request, S3Response, S3Result, s3_error};
@@ -4997,8 +5006,9 @@ mod tests {
use super::*;
use http::{Extensions, HeaderMap, HeaderName, HeaderValue, Method, Uri};
use s3s::dto::{
DeleteMarkerReplication, DeleteMarkerReplicationStatus, Destination, ExistingObjectReplication,
ExistingObjectReplicationStatus, ReplicationConfiguration, ReplicationRule, ReplicationRuleStatus,
Delete, DeleteMarkerReplication, DeleteMarkerReplicationStatus, Destination, ExistingObjectReplication,
ExistingObjectReplicationStatus, ObjectIdentifier, ReplicaModifications, ReplicaModificationsStatus,
ReplicationConfiguration, ReplicationRule, ReplicationRuleStatus, RestoreRequest, SourceSelectionCriteria,
};
use std::pin::Pin;
use std::sync::Arc;
+7 -1
View File
@@ -17,7 +17,12 @@ use rustfs_s3select_api::{
query::{Context, Query},
};
use rustfs_storage_api::ObjectOperations as _;
use s3s::dto::*;
use s3s::dto::{
CSVOutput, CompressionType, ContinuationEvent, EndEvent, ExpressionType, FileHeaderInfo, InputSerialization, JSONInput,
JSONOutput, JSONType, OutputSerialization, Progress, ProgressEvent, QuoteFields, RecordsEvent, SelectObjectContentEvent,
SelectObjectContentEventStream, SelectObjectContentInput, SelectObjectContentOutput, SelectObjectContentRequest, Stats,
StatsEvent,
};
use s3s::{S3Error, S3ErrorCode, S3Request, S3Response, S3Result, s3_error};
use std::sync::Arc;
use tokio::sync::mpsc;
@@ -545,6 +550,7 @@ mod tests {
use super::*;
use datafusion::sql::sqlparser::parser::ParserError;
use http::HeaderMap;
use s3s::dto::{CSVInput, ParquetInput, ScanRange};
fn base_input() -> SelectObjectContentInput {
SelectObjectContentInput {
+6 -2
View File
@@ -9190,10 +9190,14 @@ mod tests {
#[derive(Clone, Default)]
struct TestCatalogObjectBackend {
state: std::sync::Arc<tokio::sync::Mutex<TestCatalogObjectState>>,
locks: std::sync::Arc<tokio::sync::Mutex<BTreeMap<(String, String), std::sync::Arc<tokio::sync::Mutex<()>>>>>,
state: Arc<tokio::sync::Mutex<TestCatalogObjectState>>,
locks: TestCatalogObjectLocks,
}
type TestCatalogObjectLockKey = (String, String);
type TestCatalogObjectLock = Arc<tokio::sync::Mutex<()>>;
type TestCatalogObjectLocks = Arc<tokio::sync::Mutex<BTreeMap<TestCatalogObjectLockKey, TestCatalogObjectLock>>>;
#[derive(Default)]
struct TestCatalogObjectState {
objects: BTreeMap<(String, String), TestCatalogObjectRecord>,
@@ -160,6 +160,7 @@ RUSTFS_ADMIN_RUNTIME_SOURCE_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_admin_runtime_so
RUSTFS_APP_CONTEXT_DIRECT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_context_direct_bypass_hits.txt"
RUSTFS_APP_USECASE_RUNTIME_SOURCE_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_usecase_runtime_source_bypass_hits.txt"
RUSTFS_APP_USECASE_STORAGE_WILDCARD_HITS_FILE="${TMP_DIR}/rustfs_app_usecase_storage_wildcard_hits.txt"
RUSTFS_APP_WILDCARD_IMPORT_HITS_FILE="${TMP_DIR}/rustfs_app_wildcard_import_hits.txt"
RUSTFS_STORAGE_DIRECT_APP_CONTEXT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_storage_direct_app_context_bypass_hits.txt"
awk '
@@ -1418,6 +1419,15 @@ if [[ -s "$RUSTFS_APP_USECASE_STORAGE_WILDCARD_HITS_FILE" ]]; then
report_failure "RustFS app usecase consumers must import storage owner APIs explicitly instead of crate::storage::*: $(paste -sd '; ' "$RUSTFS_APP_USECASE_STORAGE_WILDCARD_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename 'use (s3s::dto|crate::storage::ecfs)::\*;' rustfs/src/app --glob '*.rs' || true
) >"$RUSTFS_APP_WILDCARD_IMPORT_HITS_FILE"
if [[ -s "$RUSTFS_APP_WILDCARD_IMPORT_HITS_FILE" ]]; then
report_failure "RustFS app consumers must import S3 DTO and ECFS owner APIs explicitly instead of wildcard imports: $(paste -sd '; ' "$RUSTFS_APP_WILDCARD_IMPORT_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename 'crate::app::context::|use crate::app::context|app::context::' rustfs/src/storage --glob '*.rs' |
+1 -1
View File
@@ -37,7 +37,7 @@ dep|rustfs/src/app/multipart_usecase.rs|app->interface|crate::storage::s3_api::m
dep|rustfs/src/app/multipart_usecase.rs|app->interface|crate::storage::s3_api::multipart::parse_list_multipart_uploads_params
dep|rustfs/src/app/multipart_usecase.rs|app->interface|crate::storage::s3_api::multipart::parse_list_parts_params
dep|rustfs/src/app/multipart_usecase.rs|app->interface|crate::storage::s3_api::multipart::parse_upload_part_number
dep|rustfs/src/app/object_usecase.rs|app->interface|crate::storage::ecfs
dep|rustfs/src/app/object_usecase.rs|app->interface|crate::storage::ecfs::FS
dep|rustfs/src/app/object_usecase.rs|app->interface|crate::storage::s3_api::multipart::parse_list_parts_params
dep|rustfs/src/init.rs|infra->interface|crate::admin
dep|rustfs/src/protocols/client.rs|infra->interface|crate::storage::ecfs::FS