gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 08:18:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: prune root bucket compat modules (#3686)

Browse Source
This commit is contained in:
安正超
2026-06-21 14:08:05 +08:00
committed by GitHub
parent 32d7e55564
commit 6bfed0bd43
7 changed files with 64 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/crate-boundaries.md
+4
View File
@@ -142,6 +142,10 @@ well as remaining bitrot, erasure coding, object DTO/reader, event, list, and
batch processor root modules once their facade groups exist.
ECStore root `global` re-exports must also stay removed once consumers use
`rustfs_ecstore::api::global` or crate-internal `crate::global` paths.
RustFS root `storage_compat.rs` must expose bucket metadata and quota contracts
as explicit aliases only. Broad `metadata`, `metadata_sys`, and `quota` module
passthroughs are reserved to narrower app/admin/storage compatibility
boundaries that still need module-local owner cleanup.
ECStore ClusterControlPlane read models must stay owned by the crate-private
`cluster` module. Public access goes through `rustfs_ecstore::api::cluster` so
docs/architecture/migration-progress.md
+35 -13
View File
@@ -5,17 +5,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-workload-admission-provider-compose`
- Baseline: completed `C-011/C-012/API-055/API-059`.
- Stacked on: storage concurrency policy consumers.
- PR type for this branch: `api-extraction`
- Branch: `overtrue/arch-root-runtime-compat-boundaries`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059`.
- Stacked on: main after workload admission provider composition.
- PR type for this branch: `pure-move`
- Runtime behavior changes: none.
- Rust code changes: route storage object backpressure and request hang/deadlock
runtime config consumption through shared concurrency facade policies, then
compose RustFS workload admission through provider registries.
- CI/script changes: none.
- Docs changes: record the C-013 admission provider composition precondition
for later controller status work.
- Rust code changes: prune broad root bucket metadata/quota compatibility module
passthroughs into explicit aliases for the current RustFS root consumers.
- CI/script changes: guard against restoring broad root bucket compatibility
module passthroughs.
- Docs changes: record the API-079 root runtime bucket compatibility boundary.
## Phase 0 Tasks
@@ -111,6 +110,20 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
- Verification: workload contract tests, RustFS workload admission tests,
compile coverage, formatting, diff hygiene, risk scan, architecture guard,
pre-commit quality gate, and three-expert review.
- [x] `API-079` Prune root runtime bucket compatibility modules.
- Completed slice: collapse RustFS root `storage_compat.rs` bucket
metadata/quota module passthroughs into explicit notification-config,
table-catalog metadata, and quota-error aliases, and guard the boundary
against broad module restores.
- Acceptance: root runtime consumers use direct compatibility aliases for
bucket notification loading, table-catalog metadata checks, and quota error
mapping, while app/admin/storage owner-local compatibility modules keep
their narrower module paths until their own cleanup slices.
- Must preserve: bucket notification loading, notifier event registration,
table-bucket mutation guards, quota error to S3 error mapping, ECStore
bucket metadata ownership, and all app/admin/storage compatibility paths.
- Verification: RustFS compile coverage, formatting, diff hygiene, risk
scan, architecture guard, pre-commit quality gate, and three-expert review.
- [x] `G-012` Inventory placement and repair invariants.
- Acceptance:
[`placement-repair-invariants.md`](placement-repair-invariants.md) records
@@ -3144,9 +3157,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | passed | C-013 composes workload admission providers through the shared registry contract without adding ownership cycles or ECStore dependencies. |
| Migration preservation | passed | Provider composition preserves storage foreground-read ownership, scanner activity, heal counters, replication stats, metadata runtime checks, and object/queue behavior. |
| Testing/verification | passed | Focused workload admission tests, RustFS/concurrency compile coverage, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. |
| Quality/architecture | passed | API-079 narrows the root RustFS bucket compatibility boundary to explicit aliases without adding new ECStore ownership cycles. |
| Migration preservation | passed | Bucket notification loading, table-catalog metadata checks, quota error mapping, and app/admin/storage compatibility paths remain preserved. |
| Testing/verification | passed | RustFS compile coverage, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. |
## Verification Notes
@@ -3165,6 +3178,15 @@ Passed before push:
- Rust added-line risk scan on changed Rust files: passed.
- `make pre-commit`: passed.
- Issue #660 API-079 current slice:
- `cargo check -p rustfs --lib`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- Rust added-line risk scan on changed Rust files and guard script: passed.
- `make pre-commit`: passed.
- Issue #660 C-012 current slice:
- `cargo test -p rustfs --lib storage::backpressure::tests:: -- --nocapture`: passed.
- `cargo test -p rustfs --lib storage::deadlock_detector::tests:: -- --nocapture`: passed.
rustfs/src/error.rs
+1 -2
View File
@@ -12,8 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage_compat::StorageError;
use crate::storage_compat::quota::QuotaError;
use crate::storage_compat::{QuotaError, StorageError};
use rustfs_storage_api::HTTPRangeError;
use s3s::{S3Error, S3ErrorCode};
rustfs/src/init.rs
+2 -2
View File
@@ -14,7 +14,7 @@
use crate::server::ShutdownHandle;
use crate::storage::{process_lambda_configurations, process_queue_configurations, process_topic_configurations};
use crate::storage_compat::metadata_sys;
use crate::storage_compat::get_notification_config;
use crate::{admin, config, version};
use rustfs_config::{
DEFAULT_BUFFER_MAX_SIZE, DEFAULT_BUFFER_MIN_SIZE, DEFAULT_BUFFER_PROFILE, DEFAULT_BUFFER_UNKNOWN_SIZE, DEFAULT_UPDATE_CHECK,
@@ -174,7 +174,7 @@ pub async fn add_bucket_notification_configuration(buckets: Vec<String>) {
RUSTFS_REGION
});
for bucket in buckets.iter() {
let has_notification_config = metadata_sys::get_notification_config(bucket).await.unwrap_or_else(|err| {
let has_notification_config = get_notification_config(bucket).await.unwrap_or_else(|err| {
warn!(
target: "rustfs::init",
event = "notification_config_load_failed",
rustfs/src/storage_compat.rs
+8 -2
View File
@@ -12,12 +12,18 @@
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{get_global_bucket_metadata_sys, init_bucket_metadata_sys};
pub(crate) use rustfs_ecstore::api::bucket::metadata::{
BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG,
BUCKET_TABLE_RESERVED_PREFIX, table_catalog_path_hash,
};
pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{
get as get_bucket_metadata, get_global_bucket_metadata_sys, get_notification_config, init_bucket_metadata_sys,
};
pub(crate) use rustfs_ecstore::api::bucket::migration::{try_migrate_bucket_metadata, try_migrate_iam_config};
pub(crate) use rustfs_ecstore::api::bucket::quota::QuotaError;
pub(crate) use rustfs_ecstore::api::bucket::replication::{
GLOBAL_REPLICATION_STATS, get_global_replication_pool, init_background_replication,
};
pub(crate) use rustfs_ecstore::api::bucket::{metadata, metadata_sys, quota};
pub(crate) use rustfs_ecstore::api::cluster::topology_snapshot_from_endpoint_pools_with_capabilities;
pub(crate) use rustfs_ecstore::api::config::{com, init, init_global_config_sys, try_migrate_server_config};
pub(crate) use rustfs_ecstore::api::disk::{DiskAPI, RUSTFS_META_BUCKET, endpoint::Endpoint};
rustfs/src/table_catalog.rs
+3 -7
View File
@@ -29,13 +29,9 @@ use std::{
use crate::storage_compat::RUSTFS_META_BUCKET;
use crate::storage_compat::get_lock_acquire_timeout;
use crate::storage_compat::{EcstoreError, StorageError};
use crate::storage_compat::{
metadata::{
BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG,
BUCKET_TABLE_RESERVED_PREFIX, table_catalog_path_hash,
},
metadata_sys,
BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG,
BUCKET_TABLE_RESERVED_PREFIX, EcstoreError, StorageError, get_bucket_metadata, table_catalog_path_hash,
};
use bytes::Bytes;
use datafusion::{
@@ -7007,7 +7003,7 @@ pub fn validate_object_mutation(table_bucket_enabled: bool, object_key: &str) ->
}
pub(crate) async fn validate_bucket_object_mutation(bucket: &str, object_key: &str) -> Result<(), TableObjectMutationError> {
let table_bucket_enabled = metadata_sys::get(bucket)
let table_bucket_enabled = get_bucket_metadata(bucket)
.await
.is_ok_and(|metadata| metadata.table_bucket_enabled());
scripts/check_architecture_migration_rules.sh
+11
View File
@@ -78,6 +78,7 @@ TEST_HARNESS_NESTED_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/test_harness_nested_sto
RUSTFS_NESTED_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_nested_storage_compat_hits.txt"
RUSTFS_RUNTIME_SCALAR_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_runtime_scalar_storage_compat_hits.txt"
RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_runtime_secondary_storage_compat_hits.txt"
RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_bucket_storage_compat_module_hits.txt"
PRODUCTION_UNUSED_COMPAT_ALLOW_HITS_FILE="${TMP_DIR}/production_unused_compat_allow_hits.txt"
BROAD_STORE_API_COMPAT_REEXPORT_HITS_FILE="${TMP_DIR}/broad_store_api_compat_reexport_hits.txt"
NESTED_STORE_API_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/nested_store_api_compat_module_hits.txt"
@@ -765,6 +766,16 @@ if [[ -s "$RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE" ]]; then
report_failure "RustFS runtime secondary storage compatibility paths must use direct aliases instead of bucket/config/rio/client/tier/compress/disk/rebalance modules: $(paste -sd '; ' "$RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::bucket::\{[^}]*\b(?:metadata|metadata_sys|quota)\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::bucket::(?:metadata|metadata_sys|quota)\s*;' \
rustfs/src/storage_compat.rs || true
) >"$RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE"
if [[ -s "$RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then
report_failure "RustFS root storage compatibility must expose bucket metadata/quota contracts as explicit aliases: $(paste -sd '; ' "$RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --no-heading '#!\[allow\(unused_imports\)\]' \