From 6bfed0bd437f15709faca2383d012ac556e26c68 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=AE=89=E6=AD=A3=E8=B6=85?= Date: Sun, 21 Jun 2026 14:08:05 +0800 Subject: [PATCH] refactor: prune root bucket compat modules (#3686) --- docs/architecture/crate-boundaries.md | 4 ++ docs/architecture/migration-progress.md | 48 ++++++++++++++----- rustfs/src/error.rs | 3 +- rustfs/src/init.rs | 4 +- rustfs/src/storage_compat.rs | 10 +++- rustfs/src/table_catalog.rs | 10 ++-- scripts/check_architecture_migration_rules.sh | 11 +++++ 7 files changed, 64 insertions(+), 26 deletions(-) diff --git a/docs/architecture/crate-boundaries.md b/docs/architecture/crate-boundaries.md index 78babb290..8178fd6c8 100644 --- a/docs/architecture/crate-boundaries.md +++ b/docs/architecture/crate-boundaries.md @@ -142,6 +142,10 @@ well as remaining bitrot, erasure coding, object DTO/reader, event, list, and batch processor root modules once their facade groups exist. ECStore root `global` re-exports must also stay removed once consumers use `rustfs_ecstore::api::global` or crate-internal `crate::global` paths. +RustFS root `storage_compat.rs` must expose bucket metadata and quota contracts +as explicit aliases only. Broad `metadata`, `metadata_sys`, and `quota` module +passthroughs are reserved to narrower app/admin/storage compatibility +boundaries that still need module-local owner cleanup. ECStore ClusterControlPlane read models must stay owned by the crate-private `cluster` module. Public access goes through `rustfs_ecstore::api::cluster` so diff --git a/docs/architecture/migration-progress.md b/docs/architecture/migration-progress.md index 2c91c049c..25ff981d7 100644 --- a/docs/architecture/migration-progress.md +++ b/docs/architecture/migration-progress.md @@ -5,17 +5,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ## Current Context - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) -- Branch: `overtrue/arch-workload-admission-provider-compose` -- Baseline: completed `C-011/C-012/API-055/API-059`. -- Stacked on: storage concurrency policy consumers. -- PR type for this branch: `api-extraction` +- Branch: `overtrue/arch-root-runtime-compat-boundaries` +- Baseline: completed `C-011/C-012/C-013/API-055/API-059`. +- Stacked on: main after workload admission provider composition. +- PR type for this branch: `pure-move` - Runtime behavior changes: none. -- Rust code changes: route storage object backpressure and request hang/deadlock - runtime config consumption through shared concurrency facade policies, then - compose RustFS workload admission through provider registries. -- CI/script changes: none. -- Docs changes: record the C-013 admission provider composition precondition - for later controller status work. +- Rust code changes: prune broad root bucket metadata/quota compatibility module + passthroughs into explicit aliases for the current RustFS root consumers. +- CI/script changes: guard against restoring broad root bucket compatibility + module passthroughs. +- Docs changes: record the API-079 root runtime bucket compatibility boundary. ## Phase 0 Tasks @@ -111,6 +110,20 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block - Verification: workload contract tests, RustFS workload admission tests, compile coverage, formatting, diff hygiene, risk scan, architecture guard, pre-commit quality gate, and three-expert review. +- [x] `API-079` Prune root runtime bucket compatibility modules. + - Completed slice: collapse RustFS root `storage_compat.rs` bucket + metadata/quota module passthroughs into explicit notification-config, + table-catalog metadata, and quota-error aliases, and guard the boundary + against broad module restores. + - Acceptance: root runtime consumers use direct compatibility aliases for + bucket notification loading, table-catalog metadata checks, and quota error + mapping, while app/admin/storage owner-local compatibility modules keep + their narrower module paths until their own cleanup slices. + - Must preserve: bucket notification loading, notifier event registration, + table-bucket mutation guards, quota error to S3 error mapping, ECStore + bucket metadata ownership, and all app/admin/storage compatibility paths. + - Verification: RustFS compile coverage, formatting, diff hygiene, risk + scan, architecture guard, pre-commit quality gate, and three-expert review. - [x] `G-012` Inventory placement and repair invariants. - Acceptance: [`placement-repair-invariants.md`](placement-repair-invariants.md) records @@ -3144,9 +3157,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block | Expert | Status | Notes | |---|---|---| -| Quality/architecture | passed | C-013 composes workload admission providers through the shared registry contract without adding ownership cycles or ECStore dependencies. | -| Migration preservation | passed | Provider composition preserves storage foreground-read ownership, scanner activity, heal counters, replication stats, metadata runtime checks, and object/queue behavior. | -| Testing/verification | passed | Focused workload admission tests, RustFS/concurrency compile coverage, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. | +| Quality/architecture | passed | API-079 narrows the root RustFS bucket compatibility boundary to explicit aliases without adding new ECStore ownership cycles. | +| Migration preservation | passed | Bucket notification loading, table-catalog metadata checks, quota error mapping, and app/admin/storage compatibility paths remain preserved. | +| Testing/verification | passed | RustFS compile coverage, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. | ## Verification Notes @@ -3165,6 +3178,15 @@ Passed before push: - Rust added-line risk scan on changed Rust files: passed. - `make pre-commit`: passed. +- Issue #660 API-079 current slice: + - `cargo check -p rustfs --lib`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - Rust added-line risk scan on changed Rust files and guard script: passed. + - `make pre-commit`: passed. + - Issue #660 C-012 current slice: - `cargo test -p rustfs --lib storage::backpressure::tests:: -- --nocapture`: passed. - `cargo test -p rustfs --lib storage::deadlock_detector::tests:: -- --nocapture`: passed. diff --git a/rustfs/src/error.rs b/rustfs/src/error.rs index a2bf53eba..46f62435b 100644 --- a/rustfs/src/error.rs +++ b/rustfs/src/error.rs @@ -12,8 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_compat::StorageError; -use crate::storage_compat::quota::QuotaError; +use crate::storage_compat::{QuotaError, StorageError}; use rustfs_storage_api::HTTPRangeError; use s3s::{S3Error, S3ErrorCode}; diff --git a/rustfs/src/init.rs b/rustfs/src/init.rs index e1bc0a171..e9bacf04e 100644 --- a/rustfs/src/init.rs +++ b/rustfs/src/init.rs @@ -14,7 +14,7 @@ use crate::server::ShutdownHandle; use crate::storage::{process_lambda_configurations, process_queue_configurations, process_topic_configurations}; -use crate::storage_compat::metadata_sys; +use crate::storage_compat::get_notification_config; use crate::{admin, config, version}; use rustfs_config::{ DEFAULT_BUFFER_MAX_SIZE, DEFAULT_BUFFER_MIN_SIZE, DEFAULT_BUFFER_PROFILE, DEFAULT_BUFFER_UNKNOWN_SIZE, DEFAULT_UPDATE_CHECK, @@ -174,7 +174,7 @@ pub async fn add_bucket_notification_configuration(buckets: Vec) { RUSTFS_REGION }); for bucket in buckets.iter() { - let has_notification_config = metadata_sys::get_notification_config(bucket).await.unwrap_or_else(|err| { + let has_notification_config = get_notification_config(bucket).await.unwrap_or_else(|err| { warn!( target: "rustfs::init", event = "notification_config_load_failed", diff --git a/rustfs/src/storage_compat.rs b/rustfs/src/storage_compat.rs index 92002095a..76e97cdc4 100644 --- a/rustfs/src/storage_compat.rs +++ b/rustfs/src/storage_compat.rs @@ -12,12 +12,18 @@ // See the License for the specific language governing permissions and // limitations under the License. -pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{get_global_bucket_metadata_sys, init_bucket_metadata_sys}; +pub(crate) use rustfs_ecstore::api::bucket::metadata::{ + BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG, + BUCKET_TABLE_RESERVED_PREFIX, table_catalog_path_hash, +}; +pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{ + get as get_bucket_metadata, get_global_bucket_metadata_sys, get_notification_config, init_bucket_metadata_sys, +}; pub(crate) use rustfs_ecstore::api::bucket::migration::{try_migrate_bucket_metadata, try_migrate_iam_config}; +pub(crate) use rustfs_ecstore::api::bucket::quota::QuotaError; pub(crate) use rustfs_ecstore::api::bucket::replication::{ GLOBAL_REPLICATION_STATS, get_global_replication_pool, init_background_replication, }; -pub(crate) use rustfs_ecstore::api::bucket::{metadata, metadata_sys, quota}; pub(crate) use rustfs_ecstore::api::cluster::topology_snapshot_from_endpoint_pools_with_capabilities; pub(crate) use rustfs_ecstore::api::config::{com, init, init_global_config_sys, try_migrate_server_config}; pub(crate) use rustfs_ecstore::api::disk::{DiskAPI, RUSTFS_META_BUCKET, endpoint::Endpoint}; diff --git a/rustfs/src/table_catalog.rs b/rustfs/src/table_catalog.rs index b16f3672e..124ea9fab 100644 --- a/rustfs/src/table_catalog.rs +++ b/rustfs/src/table_catalog.rs @@ -29,13 +29,9 @@ use std::{ use crate::storage_compat::RUSTFS_META_BUCKET; use crate::storage_compat::get_lock_acquire_timeout; -use crate::storage_compat::{EcstoreError, StorageError}; use crate::storage_compat::{ - metadata::{ - BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG, - BUCKET_TABLE_RESERVED_PREFIX, table_catalog_path_hash, - }, - metadata_sys, + BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG, + BUCKET_TABLE_RESERVED_PREFIX, EcstoreError, StorageError, get_bucket_metadata, table_catalog_path_hash, }; use bytes::Bytes; use datafusion::{ @@ -7007,7 +7003,7 @@ pub fn validate_object_mutation(table_bucket_enabled: bool, object_key: &str) -> } pub(crate) async fn validate_bucket_object_mutation(bucket: &str, object_key: &str) -> Result<(), TableObjectMutationError> { - let table_bucket_enabled = metadata_sys::get(bucket) + let table_bucket_enabled = get_bucket_metadata(bucket) .await .is_ok_and(|metadata| metadata.table_bucket_enabled()); diff --git a/scripts/check_architecture_migration_rules.sh b/scripts/check_architecture_migration_rules.sh index e6c1b825e..1f759a645 100755 --- a/scripts/check_architecture_migration_rules.sh +++ b/scripts/check_architecture_migration_rules.sh @@ -78,6 +78,7 @@ TEST_HARNESS_NESTED_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/test_harness_nested_sto RUSTFS_NESTED_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_nested_storage_compat_hits.txt" RUSTFS_RUNTIME_SCALAR_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_runtime_scalar_storage_compat_hits.txt" RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_runtime_secondary_storage_compat_hits.txt" +RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_bucket_storage_compat_module_hits.txt" PRODUCTION_UNUSED_COMPAT_ALLOW_HITS_FILE="${TMP_DIR}/production_unused_compat_allow_hits.txt" BROAD_STORE_API_COMPAT_REEXPORT_HITS_FILE="${TMP_DIR}/broad_store_api_compat_reexport_hits.txt" NESTED_STORE_API_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/nested_store_api_compat_module_hits.txt" @@ -765,6 +766,16 @@ if [[ -s "$RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE" ]]; then report_failure "RustFS runtime secondary storage compatibility paths must use direct aliases instead of bucket/config/rio/client/tier/compress/disk/rebalance modules: $(paste -sd '; ' "$RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE")" fi +( + cd "$ROOT_DIR" + rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::bucket::\{[^}]*\b(?:metadata|metadata_sys|quota)\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::bucket::(?:metadata|metadata_sys|quota)\s*;' \ + rustfs/src/storage_compat.rs || true +) >"$RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" + +if [[ -s "$RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then + report_failure "RustFS root storage compatibility must expose bucket metadata/quota contracts as explicit aliases: $(paste -sd '; ' "$RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE")" +fi + ( cd "$ROOT_DIR" rg -n --no-heading '#!\[allow\(unused_imports\)\]' \