refactor: prune root runtime compat re-exports (#3694)

This commit is contained in:
安正超
2026-06-21 18:42:25 +08:00
committed by GitHub
parent 0cf9d07e40
commit 48ed331fc8
6 changed files with 255 additions and 52 deletions
+1 -1
View File
@@ -116,7 +116,7 @@ pub mod global {
pub mod layout {
pub use crate::disks_layout::DisksLayout;
pub use crate::endpoints::{EndpointServerPools, Endpoints, PoolEndpoints};
pub use crate::endpoints::{EndpointServerPools, Endpoints, PoolEndpoints, SetupType};
}
pub mod metrics {
+4
View File
@@ -149,6 +149,10 @@ boundaries that still need module-local owner cleanup.
Root runtime storage config initialization and disk endpoint contracts must also
stay explicit aliases. The root compatibility boundary must not restore `com`,
bare `init`, or grouped `endpoint::Endpoint` passthroughs.
RustFS root `storage_compat.rs` must not re-export ECStore API symbols directly;
remaining root runtime compatibility symbols must be local type aliases,
constants, traits, or wrapper functions so ownership stays visible at the
boundary.
RustFS admin `storage_compat.rs` must expose config IO and default
initialization through explicit aliases. The admin compatibility boundary must
not restore broad `com` or bare `init` passthroughs.
+36 -11
View File
@@ -5,16 +5,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-test-protocol-compat-prune`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084`.
- Stacked on: API-084 edge compatibility alias pruning.
- Branch: `overtrue/arch-root-runtime-compat-wrappers`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085`.
- Stacked on: API-085 test and fuzz compatibility alias pruning.
- PR type for this branch: `pure-move`
- Runtime behavior changes: none.
- Rust code changes: prune heal/scanner test and fuzz compatibility
passthroughs into explicit aliases and local wrappers.
- CI/script changes: guard against restoring grouped test and fuzz
compatibility passthroughs.
- Docs changes: record the API-085 test/fuzz compatibility boundary.
- Rust code changes: prune root RustFS runtime compatibility re-exports into
local aliases, traits, and wrapper functions.
- CI/script changes: guard against restoring root RustFS runtime ECStore API
re-exports.
- Docs changes: record the API-086 root runtime compatibility boundary.
## Phase 0 Tasks
@@ -214,6 +214,21 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
- Verification: focused heal/scanner compile coverage, test/fuzz
compatibility residual scans, formatting, diff hygiene, architecture guard,
pre-commit quality gate, and three-expert review.
- [x] `API-086` Prune root runtime compatibility re-exports.
- Completed slice: replace root RustFS runtime `storage_compat.rs` ECStore API
re-exports with local constants, type aliases, a minimal disk trait, and
wrapper functions.
- Acceptance: root runtime startup, metadata, replication admission,
topology, notification, RPC, capacity, table-catalog, and shutdown call
sites keep their existing local compatibility names while the root boundary
no longer re-exports ECStore API symbols directly.
- Must preserve: startup storage initialization order, bucket metadata
migration/init, replication runtime startup and admission counts,
notification init, RPC signature checks, capacity disk references,
topology snapshots, table-catalog metadata access, and shutdown behavior.
- Verification: RustFS compile coverage, root compatibility re-export
residual scan, formatting, diff hygiene, architecture guard, pre-commit
quality gate, and three-expert review.
- [x] `G-012` Inventory placement and repair invariants.
- Acceptance:
[`placement-repair-invariants.md`](placement-repair-invariants.md) records
@@ -3247,14 +3262,24 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | passed | API-085 narrows heal/scanner test and fuzz compatibility passthroughs without adding ECStore ownership cycles or runtime dependencies. |
| Migration preservation | passed | Heal test setup, scanner lifecycle integration helpers, local disk initialization, bucket metadata updates, transition enqueue behavior, and fuzz validation semantics remain preserved behind aliases/wrappers. |
| Testing/verification | passed | Focused heal/scanner compile coverage, test/fuzz compatibility residual scans, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. |
| Quality/architecture | passed | API-086 narrows root RustFS runtime compatibility with local aliases/wrappers and adds an ECStore API re-export guard without adding runtime ownership cycles. |
| Migration preservation | passed | Startup storage, bucket metadata migration/init, replication admission counts, notification init, RPC signature checks, capacity disk references, topology snapshots, table-catalog metadata access, and shutdown behavior remain behind existing local compatibility names. |
| Testing/verification | passed | RustFS compile coverage, root re-export residual scan, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. |
## Verification Notes
Passed before push:
- Issue #660 API-086 current slice:
- `cargo check -p rustfs`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- Root runtime ECStore API re-export residual scan: passed.
- Rust added-line risk scan on changed Rust files and guard script: passed.
- `make pre-commit`: passed.
- Issue #660 API-085 current slice:
- `cargo check --tests -p rustfs-heal -p rustfs-scanner`: passed.
- `cargo fmt --all --check`: passed.
+201 -32
View File
@@ -12,37 +12,206 @@
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) use rustfs_ecstore::api::bucket::metadata::{
BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG,
BUCKET_TABLE_RESERVED_PREFIX, table_catalog_path_hash,
};
pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{
get as get_bucket_metadata, get_global_bucket_metadata_sys, get_notification_config, init_bucket_metadata_sys,
};
pub(crate) use rustfs_ecstore::api::bucket::migration::{try_migrate_bucket_metadata, try_migrate_iam_config};
pub(crate) use rustfs_ecstore::api::bucket::quota::QuotaError;
pub(crate) use rustfs_ecstore::api::bucket::replication::{
GLOBAL_REPLICATION_STATS, get_global_replication_pool, init_background_replication,
};
pub(crate) use rustfs_ecstore::api::cluster::topology_snapshot_from_endpoint_pools_with_capabilities;
pub(crate) use rustfs_ecstore::api::config::com::{read_config as read_ecstore_config, save_config as save_ecstore_config};
pub(crate) use rustfs_ecstore::api::config::{init as init_ecstore_config, init_global_config_sys, try_migrate_server_config};
pub(crate) use rustfs_ecstore::api::disk::endpoint::Endpoint;
pub(crate) use rustfs_ecstore::api::disk::{DiskAPI, RUSTFS_META_BUCKET};
pub(crate) use rustfs_ecstore::api::error::{Error as EcstoreError, Result as EcstoreResult, StorageError};
pub(crate) use rustfs_ecstore::api::event::{EventArgs as EcstoreEventArgs, register_event_dispatch_hook};
pub(crate) use rustfs_ecstore::api::global::{
get_global_endpoints_opt, get_global_lock_clients, get_global_region, is_dist_erasure, resolve_object_store_handle,
set_global_endpoints, set_global_region, set_global_rustfs_port, shutdown_background_services, update_erasure_type,
};
use std::{collections::HashMap, sync::Arc, time::Duration};
pub(crate) const BUCKET_TABLE_CATALOG_META_PREFIX: &str = rustfs_ecstore::api::bucket::metadata::BUCKET_TABLE_CATALOG_META_PREFIX;
pub(crate) const BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX: &str =
rustfs_ecstore::api::bucket::metadata::BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX;
pub(crate) const BUCKET_TABLE_CONFIG: &str = rustfs_ecstore::api::bucket::metadata::BUCKET_TABLE_CONFIG;
pub(crate) const BUCKET_TABLE_RESERVED_PREFIX: &str = rustfs_ecstore::api::bucket::metadata::BUCKET_TABLE_RESERVED_PREFIX;
pub(crate) const RUSTFS_META_BUCKET: &str = rustfs_ecstore::api::disk::RUSTFS_META_BUCKET;
pub(crate) const TONIC_RPC_PREFIX: &str = rustfs_ecstore::api::rpc::TONIC_RPC_PREFIX;
pub(crate) type ECStore = rustfs_ecstore::api::storage::ECStore;
pub(crate) type EcstoreError = rustfs_ecstore::api::error::Error;
pub(crate) type EcstoreEventArgs = rustfs_ecstore::api::event::EventArgs;
pub(crate) type EcstoreResult<T> = rustfs_ecstore::api::error::Result<T>;
pub(crate) type Endpoint = rustfs_ecstore::api::disk::endpoint::Endpoint;
pub(crate) type EndpointServerPools = rustfs_ecstore::api::layout::EndpointServerPools;
pub(crate) type QuotaError = rustfs_ecstore::api::bucket::quota::QuotaError;
pub(crate) type StorageError = rustfs_ecstore::api::error::StorageError;
#[cfg(test)]
pub(crate) use rustfs_ecstore::api::layout::DisksLayout;
pub(crate) use rustfs_ecstore::api::layout::EndpointServerPools;
pub(crate) type DisksLayout = rustfs_ecstore::api::layout::DisksLayout;
#[cfg(test)]
pub(crate) use rustfs_ecstore::api::layout::{Endpoints, PoolEndpoints};
pub(crate) use rustfs_ecstore::api::notification::new_global_notification_sys;
pub(crate) use rustfs_ecstore::api::rpc::{TONIC_RPC_PREFIX, verify_rpc_signature};
pub(crate) use rustfs_ecstore::api::set_disk::get_lock_acquire_timeout;
pub(crate) use rustfs_ecstore::api::storage::{
ECStore, all_local_disk, init_local_disks, init_lock_clients, prewarm_local_disk_id_map,
};
pub(crate) type Endpoints = rustfs_ecstore::api::layout::Endpoints;
#[cfg(test)]
pub(crate) type PoolEndpoints = rustfs_ecstore::api::layout::PoolEndpoints;
pub(crate) trait DiskAPI {
fn endpoint(&self) -> Endpoint;
fn to_string(&self) -> String;
}
impl<T> DiskAPI for T
where
T: rustfs_ecstore::api::disk::DiskAPI,
{
fn endpoint(&self) -> Endpoint {
rustfs_ecstore::api::disk::DiskAPI::endpoint(self)
}
fn to_string(&self) -> String {
rustfs_ecstore::api::disk::DiskAPI::to_string(self)
}
}
pub(crate) fn table_catalog_path_hash(value: &str) -> String {
rustfs_ecstore::api::bucket::metadata::table_catalog_path_hash(value)
}
pub(crate) async fn get_bucket_metadata(
bucket: &str,
) -> rustfs_ecstore::api::error::Result<Arc<rustfs_ecstore::api::bucket::metadata::BucketMetadata>> {
rustfs_ecstore::api::bucket::metadata_sys::get(bucket).await
}
pub(crate) fn get_global_bucket_metadata_sys()
-> Option<Arc<tokio::sync::RwLock<rustfs_ecstore::api::bucket::metadata_sys::BucketMetadataSys>>> {
rustfs_ecstore::api::bucket::metadata_sys::get_global_bucket_metadata_sys()
}
pub(crate) async fn get_notification_config(
bucket: &str,
) -> rustfs_ecstore::api::error::Result<Option<s3s::dto::NotificationConfiguration>> {
rustfs_ecstore::api::bucket::metadata_sys::get_notification_config(bucket).await
}
pub(crate) async fn init_bucket_metadata_sys(api: Arc<ECStore>, buckets: Vec<String>) {
rustfs_ecstore::api::bucket::metadata_sys::init_bucket_metadata_sys(api, buckets).await;
}
pub(crate) async fn try_migrate_bucket_metadata(store: Arc<ECStore>) {
rustfs_ecstore::api::bucket::migration::try_migrate_bucket_metadata(store).await;
}
pub(crate) async fn try_migrate_iam_config(store: Arc<ECStore>) {
rustfs_ecstore::api::bucket::migration::try_migrate_iam_config(store).await;
}
pub(crate) fn get_global_replication_pool() -> Option<Arc<rustfs_ecstore::api::bucket::replication::DynReplicationPool>> {
rustfs_ecstore::api::bucket::replication::get_global_replication_pool()
}
pub(crate) fn replication_queue_current_count() -> Option<i64> {
rustfs_ecstore::api::bucket::replication::GLOBAL_REPLICATION_STATS
.get()
.and_then(|stats| {
stats
.q_cache
.try_lock()
.ok()
.map(|cache| cache.sr_queue_stats.curr.get_current_count())
})
}
pub(crate) async fn init_background_replication(storage: Arc<ECStore>) {
rustfs_ecstore::api::bucket::replication::init_background_replication(storage).await;
}
pub(crate) fn topology_snapshot_from_endpoint_pools_with_capabilities(
endpoint_pools: &EndpointServerPools,
capabilities: rustfs_storage_api::TopologyCapabilities,
disk_capabilities: rustfs_storage_api::DiskCapabilities,
) -> rustfs_storage_api::TopologySnapshot {
rustfs_ecstore::api::cluster::topology_snapshot_from_endpoint_pools_with_capabilities(
endpoint_pools,
capabilities,
disk_capabilities,
)
}
pub(crate) async fn read_ecstore_config(api: Arc<ECStore>, file: &str) -> EcstoreResult<Vec<u8>> {
rustfs_ecstore::api::config::com::read_config(api, file).await
}
pub(crate) async fn save_ecstore_config(api: Arc<ECStore>, file: &str, data: Vec<u8>) -> EcstoreResult<()> {
rustfs_ecstore::api::config::com::save_config(api, file, data).await
}
pub(crate) fn init_ecstore_config() {
rustfs_ecstore::api::config::init();
}
pub(crate) async fn init_global_config_sys(api: Arc<ECStore>) -> EcstoreResult<()> {
rustfs_ecstore::api::config::init_global_config_sys(api).await
}
pub(crate) async fn try_migrate_server_config(api: Arc<ECStore>) {
rustfs_ecstore::api::config::try_migrate_server_config(api).await;
}
pub(crate) fn register_event_dispatch_hook<F>(hook: F) -> bool
where
F: Fn(EcstoreEventArgs) + Send + Sync + 'static,
{
rustfs_ecstore::api::event::register_event_dispatch_hook(hook)
}
pub(crate) fn get_global_endpoints_opt() -> Option<EndpointServerPools> {
rustfs_ecstore::api::global::get_global_endpoints_opt()
}
pub(crate) fn get_global_lock_clients() -> Option<&'static HashMap<String, Arc<dyn rustfs_lock::client::LockClient>>> {
rustfs_ecstore::api::global::get_global_lock_clients()
}
pub(crate) fn get_global_region() -> Option<s3s::region::Region> {
rustfs_ecstore::api::global::get_global_region()
}
pub(crate) async fn is_dist_erasure() -> bool {
rustfs_ecstore::api::global::is_dist_erasure().await
}
pub(crate) fn resolve_object_store_handle() -> Option<Arc<ECStore>> {
rustfs_ecstore::api::global::resolve_object_store_handle()
}
pub(crate) fn set_global_endpoints(eps: Vec<rustfs_ecstore::api::layout::PoolEndpoints>) {
rustfs_ecstore::api::global::set_global_endpoints(eps);
}
pub(crate) fn set_global_region(region: s3s::region::Region) {
rustfs_ecstore::api::global::set_global_region(region);
}
pub(crate) fn set_global_rustfs_port(value: u16) {
rustfs_ecstore::api::global::set_global_rustfs_port(value);
}
pub(crate) fn shutdown_background_services() {
rustfs_ecstore::api::global::shutdown_background_services();
}
pub(crate) async fn update_erasure_type(setup_type: rustfs_ecstore::api::layout::SetupType) {
rustfs_ecstore::api::global::update_erasure_type(setup_type).await;
}
pub(crate) async fn new_global_notification_sys(eps: EndpointServerPools) -> EcstoreResult<()> {
rustfs_ecstore::api::notification::new_global_notification_sys(eps).await
}
pub(crate) fn verify_rpc_signature(url: &str, method: &http::Method, headers: &http::HeaderMap) -> std::io::Result<()> {
rustfs_ecstore::api::rpc::verify_rpc_signature(url, method, headers)
}
pub(crate) fn get_lock_acquire_timeout() -> Duration {
rustfs_ecstore::api::set_disk::get_lock_acquire_timeout()
}
pub(crate) async fn all_local_disk() -> Vec<rustfs_ecstore::api::disk::DiskStore> {
rustfs_ecstore::api::storage::all_local_disk().await
}
pub(crate) async fn init_local_disks(endpoint_pools: EndpointServerPools) -> EcstoreResult<()> {
rustfs_ecstore::api::storage::init_local_disks(endpoint_pools).await
}
pub(crate) fn init_lock_clients(endpoint_pools: EndpointServerPools) {
rustfs_ecstore::api::storage::init_lock_clients(endpoint_pools);
}
pub(crate) async fn prewarm_local_disk_id_map() {
rustfs_ecstore::api::storage::prewarm_local_disk_id_map().await;
}
+2 -8
View File
@@ -18,7 +18,7 @@ use rustfs_concurrency::{
};
use crate::storage::concurrency::get_concurrency_manager;
use crate::storage_compat::{GLOBAL_REPLICATION_STATS, get_global_bucket_metadata_sys, get_global_replication_pool};
use crate::storage_compat::{get_global_bucket_metadata_sys, get_global_replication_pool, replication_queue_current_count};
const BUCKET_METADATA_RUNTIME_NOT_INITIALIZED: &str = "bucket metadata runtime not initialized";
const HEAL_MANAGER_NOT_INITIALIZED: &str = "heal manager not initialized";
@@ -159,13 +159,7 @@ pub fn replication_workload_admission_snapshot() -> WorkloadAdmissionSnapshot {
.active_workers()
.saturating_add(pool.active_lrg_workers())
.saturating_add(pool.active_mrf_workers());
let queued = GLOBAL_REPLICATION_STATS.get().and_then(|stats| {
stats
.q_cache
.try_lock()
.ok()
.map(|cache| i64_to_usize_saturated(cache.sr_queue_stats.curr.get_current_count()))
});
let queued = replication_queue_current_count().map(i64_to_usize_saturated);
replication_workload_admission_snapshot_from_counts(true, Some(i32_to_usize_saturated(active)), queued)
}
@@ -78,6 +78,7 @@ TEST_HARNESS_NESTED_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/test_harness_nested_sto
RUSTFS_NESTED_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_nested_storage_compat_hits.txt"
RUSTFS_RUNTIME_SCALAR_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_runtime_scalar_storage_compat_hits.txt"
RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_runtime_secondary_storage_compat_hits.txt"
RUSTFS_ROOT_STORAGE_COMPAT_REEXPORT_HITS_FILE="${TMP_DIR}/rustfs_root_storage_compat_reexport_hits.txt"
RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_bucket_storage_compat_module_hits.txt"
RUSTFS_ROOT_RUNTIME_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_runtime_storage_compat_module_hits.txt"
RUSTFS_ADMIN_CONFIG_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_admin_config_storage_compat_module_hits.txt"
@@ -776,6 +777,16 @@ if [[ -s "$RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE" ]]; then
report_failure "RustFS runtime secondary storage compatibility paths must use direct aliases instead of bucket/config/rio/client/tier/compress/disk/rebalance modules: $(paste -sd '; ' "$RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --no-heading 'pub\(crate\)\s+use\s+rustfs_ecstore::api::' \
rustfs/src/storage_compat.rs || true
) >"$RUSTFS_ROOT_STORAGE_COMPAT_REEXPORT_HITS_FILE"
if [[ -s "$RUSTFS_ROOT_STORAGE_COMPAT_REEXPORT_HITS_FILE" ]]; then
report_failure "RustFS root storage compatibility must use local aliases or wrappers instead of re-exporting ECStore API symbols: $(paste -sd '; ' "$RUSTFS_ROOT_STORAGE_COMPAT_REEXPORT_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::bucket::\{[^}]*\b(?:metadata|metadata_sys|quota)\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::bucket::(?:metadata|metadata_sys|quota)\s*;' \