diff --git a/docs/architecture/migration-progress.md b/docs/architecture/migration-progress.md index aa3c29e5e..b08d5a1e3 100644 --- a/docs/architecture/migration-progress.md +++ b/docs/architecture/migration-progress.md @@ -5,17 +5,17 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ## Current Context - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) -- Branch: `overtrue/arch-disk-rpc-method-wrappers` -- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096`. -- Stacked on: `overtrue/arch-compat-trait-method-wrappers` pending API-096 merge. +- Branch: `overtrue/arch-runtime-compat-consumer-batch` +- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100`. +- Stacked on: `overtrue/arch-disk-rpc-method-wrappers` pending API-097 merge. - PR type for this branch: `pure-move` - Runtime behavior changes: none. -- Rust code changes: move remaining disk RPC, peer S3 RPC, heal/scanner disk, - and warm-backend test method access behind local compatibility traits or - aliases. -- CI/script changes: remove the final direct ECStore method-resolution import - exceptions outside compatibility boundaries. -- Docs changes: record the API-097 disk/RPC/warm-backend wrapper cleanup. +- Rust code changes: move capacity, server, startup, table catalog, + runtime-capability, workload-admission, error, and config-test ECStore + compatibility consumers from the root runtime facade into local boundaries. +- CI/script changes: guard against restoring consumer wrappers in the root + runtime compatibility facade and tolerate retiring the empty root facade. +- Docs changes: record the API-098/API-099/API-100 runtime consumer boundary cleanup. ## Phase 0 Tasks @@ -377,7 +377,7 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block import residual scan, migration guard, formatting, diff hygiene, Rust risk scan, pre-commit quality gate, and three-expert review. - [x] `API-097` Prune disk/RPC/warm-backend method imports. - - Current slice: move disk RPC, peer S3 RPC, heal/scanner disk, and + - Completed slice: move disk RPC, peer S3 RPC, heal/scanner disk, and warm-backend test method access behind local compatibility traits or aliases in the owning boundaries. - Acceptance: non-compat RustFS, scanner, heal, and test sources no longer @@ -390,6 +390,51 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block disk/RPC/warm-backend trait import residual scan, migration guard, formatting, diff hygiene, Rust risk scan, pre-commit quality gate, and three-expert review. +- [x] `API-098` Prune root runtime capacity/server compat consumers. + - Completed slice: move capacity disk access, HTTP RPC signature verification, + event dispatch bridging, module-switch config persistence, and readiness + storage/lock quorum lookups into local `capacity` and `server` + compatibility boundaries. + - Acceptance: root runtime `storage_compat.rs` no longer owns + capacity/server-only ECStore wrapper functions, trait shims, or constants; + migration rules reject restoring those wrappers to the root facade. + - Must preserve: capacity background refresh disk discovery, internode RPC + signature verification, live event dispatch, module-switch persistence, + storage readiness, and distributed lock quorum behavior. + - Verification: RustFS test-target compile coverage, capacity/server residual + scan, migration and layer guards, formatting, diff hygiene, Rust risk scan, + pre-commit quality gate, and three-expert review. +- [x] `API-099` Prune root runtime startup compat consumers. + - Completed slice: move startup storage bootstrap, bucket metadata migration, + notification initialization, global region/port setup, background shutdown, + and startup service ECStore aliases into a dedicated startup compatibility + boundary. + - Acceptance: startup and init modules no longer consume root + `storage_compat.rs`; root runtime `storage_compat.rs` no longer owns + startup-only ECStore wrapper functions or aliases; migration rules reject + restoring those wrappers to the root facade. + - Must preserve: endpoint parsing, unsupported filesystem policy, + local-disk and lock-client initialization, global config migration, + bucket metadata migration, IAM migration, notification registration, + default-region fallback, background replication, and shutdown behavior. + - Verification: RustFS test-target compile coverage, startup residual scan, + migration and layer guards, formatting, diff hygiene, Rust risk scan, + pre-commit quality gate, and three-expert review. +- [x] `API-100` Retire root runtime storage compatibility consumers. + - Completed slice: move table catalog metadata constants and bucket metadata + reads, runtime topology capability mapping, workload admission runtime + state probes, S3 error mapping aliases, and config test disk-layout aliases + into local compatibility boundaries, then remove the root + `storage_compat.rs` module. + - Acceptance: no RustFS source consumes `crate::storage_compat`; root + runtime compatibility file is removed; migration rules still reject direct + ECStore imports outside `*storage_compat.rs` boundaries. + - Must preserve: table catalog internal metadata paths, lock timeout lookup, + runtime topology snapshots, workload admission status reporting, quota and + storage error mapping, and config disk-layout parsing tests. + - Verification: RustFS test-target compile coverage, direct root compatibility + consumer residual scan, migration and layer guards, formatting, diff + hygiene, Rust risk scan, pre-commit quality gate, and three-expert review. - [x] `G-012` Inventory placement and repair invariants. - Acceptance: [`placement-repair-invariants.md`](placement-repair-invariants.md) records @@ -3431,6 +3476,43 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block Passed before push: +- Issue #660 API-098 current slice: + - `cargo check -p rustfs --tests`: passed. + - `cargo fmt --all`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - Direct root capacity/server compatibility consumer residual scan: passed. + - Rust risk scan on changed Rust files and guard script: passed. + - `make pre-commit`: passed. + +- Issue #660 API-099 current slice: + - `cargo check -p rustfs --tests`: passed. + - `cargo fmt --all`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - Startup/init root compatibility consumer residual scan: passed. + - Root startup consumer wrapper residual scan: passed. + - Rust risk scan on changed Rust files and guard script: passed. + - `make pre-commit`: passed. + +- Issue #660 API-100 current slice: + - `cargo check -p rustfs --tests`: passed. + - `cargo fmt --all`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - Direct root compatibility consumer residual scan: passed. + - Rust risk scan on changed Rust files and guard script: passed. + - `make pre-commit`: passed. + - Issue #660 API-097 current slice: - `cargo check -p rustfs -p rustfs-scanner -p rustfs-heal -p e2e_test --tests`: passed. - `cargo fmt --all --check`: passed. diff --git a/rustfs/src/capacity/mod.rs b/rustfs/src/capacity/mod.rs index cf8dc9a29..e796027c7 100644 --- a/rustfs/src/capacity/mod.rs +++ b/rustfs/src/capacity/mod.rs @@ -52,6 +52,7 @@ pub mod capacity_integration; pub mod service; +mod storage_compat; pub use service::{ capacity_disk_ref, get_cached_capacity_with_metrics, init_capacity_management_for_local_disks, record_capacity_write, diff --git a/rustfs/src/capacity/service.rs b/rustfs/src/capacity/service.rs index 12f76cc1a..a9c922a34 100644 --- a/rustfs/src/capacity/service.rs +++ b/rustfs/src/capacity/service.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_compat::DiskAPI; +use super::storage_compat::CapacityDiskExt as _; use rustfs_io_metrics::capacity_metrics::{ record_capacity_cache_hit, record_capacity_cache_miss, record_capacity_cache_served, record_capacity_refresh_request, record_capacity_scan_mode, @@ -263,7 +263,7 @@ pub async fn init_capacity_management_for_local_disks() { "Capacity manager state changed" ); - let disks = crate::storage_compat::all_local_disk().await; + let disks = super::storage_compat::all_local_disk().await; if disks.is_empty() { warn!( component = LOG_COMPONENT_CAPACITY, diff --git a/rustfs/src/capacity/storage_compat.rs b/rustfs/src/capacity/storage_compat.rs new file mode 100644 index 000000000..ebff531d0 --- /dev/null +++ b/rustfs/src/capacity/storage_compat.rs @@ -0,0 +1,39 @@ +// Copyright 2024 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use rustfs_ecstore::api::{disk as ecstore_disk, storage as ecstore_storage}; + +pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint; + +pub(crate) trait CapacityDiskExt { + fn endpoint(&self) -> Endpoint; + fn to_string(&self) -> String; +} + +impl CapacityDiskExt for T +where + T: ecstore_disk::DiskAPI, +{ + fn endpoint(&self) -> Endpoint { + ecstore_disk::DiskAPI::endpoint(self) + } + + fn to_string(&self) -> String { + ecstore_disk::DiskAPI::to_string(self) + } +} + +pub(crate) async fn all_local_disk() -> Vec { + ecstore_storage::all_local_disk().await +} diff --git a/rustfs/src/config/config_test.rs b/rustfs/src/config/config_test.rs index 2c1487a5e..5c80060ba 100644 --- a/rustfs/src/config/config_test.rs +++ b/rustfs/src/config/config_test.rs @@ -16,7 +16,7 @@ #[allow(unsafe_op_in_unsafe_fn)] mod tests { use crate::config::{CommandResult, Config, Opt, TlsCommands}; - use crate::storage_compat::DisksLayout; + use crate::config_storage_compat::DisksLayout; use rustfs_config::{DEFAULT_CONSOLE_ADDRESS, DEFAULT_CONSOLE_ENABLE, DEFAULT_OBS_ENDPOINT, RUSTFS_REGION}; use rustfs_credentials::{DEFAULT_ACCESS_KEY, DEFAULT_SECRET_KEY}; use serial_test::serial; @@ -262,7 +262,7 @@ mod tests { #[test] #[serial] fn test_volumes_and_disk_layout_parsing() { - use crate::storage_compat::DisksLayout; + use crate::config_storage_compat::DisksLayout; // Test case 1: Single volume path let args = vec!["rustfs", "/data/vol1"]; diff --git a/rustfs/src/config_storage_compat.rs b/rustfs/src/config_storage_compat.rs new file mode 100644 index 000000000..ab5880427 --- /dev/null +++ b/rustfs/src/config_storage_compat.rs @@ -0,0 +1,19 @@ +// Copyright 2024 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#[cfg(test)] +use rustfs_ecstore::api::layout as ecstore_layout; + +#[cfg(test)] +pub(crate) type DisksLayout = ecstore_layout::DisksLayout; diff --git a/rustfs/src/error.rs b/rustfs/src/error.rs index 46f62435b..e323a93d1 100644 --- a/rustfs/src/error.rs +++ b/rustfs/src/error.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_compat::{QuotaError, StorageError}; +use crate::error_storage_compat::{QuotaError, StorageError}; use rustfs_storage_api::HTTPRangeError; use s3s::{S3Error, S3ErrorCode}; diff --git a/rustfs/src/error_storage_compat.rs b/rustfs/src/error_storage_compat.rs new file mode 100644 index 000000000..040619550 --- /dev/null +++ b/rustfs/src/error_storage_compat.rs @@ -0,0 +1,18 @@ +// Copyright 2024 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use rustfs_ecstore::api::{bucket as ecstore_bucket, error as ecstore_error}; + +pub(crate) type QuotaError = ecstore_bucket::quota::QuotaError; +pub(crate) type StorageError = ecstore_error::StorageError; diff --git a/rustfs/src/init.rs b/rustfs/src/init.rs index e9bacf04e..89c67377a 100644 --- a/rustfs/src/init.rs +++ b/rustfs/src/init.rs @@ -13,8 +13,8 @@ // limitations under the License. use crate::server::ShutdownHandle; +use crate::startup_storage_compat::{get_global_region, get_notification_config}; use crate::storage::{process_lambda_configurations, process_queue_configurations, process_topic_configurations}; -use crate::storage_compat::get_notification_config; use crate::{admin, config, version}; use rustfs_config::{ DEFAULT_BUFFER_MAX_SIZE, DEFAULT_BUFFER_MIN_SIZE, DEFAULT_BUFFER_PROFILE, DEFAULT_BUFFER_UNKNOWN_SIZE, DEFAULT_UPDATE_CHECK, @@ -157,7 +157,7 @@ fn arn_to_target_id(arn_str: &str) -> Result) { - let global_region = crate::storage_compat::get_global_region(); + let global_region = get_global_region(); let region = global_region .as_ref() .filter(|r| !r.as_str().is_empty()) diff --git a/rustfs/src/lib.rs b/rustfs/src/lib.rs index e497ad1c7..0909a5a66 100644 --- a/rustfs/src/lib.rs +++ b/rustfs/src/lib.rs @@ -57,9 +57,11 @@ pub mod auth; pub mod auth_keystone; pub mod capacity; pub mod config; +pub(crate) mod config_storage_compat; pub mod delete_tail_activity; pub mod embedded; pub mod error; +pub(crate) mod error_storage_compat; pub mod init; pub mod license; pub mod memory_observability; @@ -67,6 +69,7 @@ pub mod profiling; #[cfg(any(feature = "ftps", feature = "webdav", feature = "sftp"))] pub mod protocols; pub mod runtime_capabilities; +pub(crate) mod runtime_capabilities_storage_compat; pub mod server; pub(crate) mod startup_audit; pub(crate) mod startup_auth; @@ -90,14 +93,16 @@ pub(crate) mod startup_server; pub(crate) mod startup_services; pub(crate) mod startup_shutdown; pub(crate) mod startup_storage; +pub(crate) mod startup_storage_compat; pub(crate) mod startup_tls_material; pub mod storage; -pub(crate) mod storage_compat; pub(crate) mod table_catalog; +pub(crate) mod table_catalog_storage_compat; pub mod tls; pub mod update; pub mod version; pub mod workload_admission; +pub(crate) mod workload_admission_storage_compat; // Re-export from rustfs_utils so that config sub-modules can use // `crate::apply_external_env_compat` without breaking. diff --git a/rustfs/src/runtime_capabilities.rs b/rustfs/src/runtime_capabilities.rs index 1ad90a5d0..724e0f772 100644 --- a/rustfs/src/runtime_capabilities.rs +++ b/rustfs/src/runtime_capabilities.rs @@ -18,7 +18,7 @@ use rustfs_storage_api::{ UserspaceProfilingCapability, }; -use crate::storage_compat::EndpointServerPools; +use crate::runtime_capabilities_storage_compat::{EndpointServerPools, topology_snapshot_from_endpoint_pools_with_capabilities}; const NOT_WIRED_INTO_RUNTIME: &str = "not wired into runtime"; const STORAGE_MEDIA_NOT_REPORTED: &str = "storage media not reported by endpoints"; @@ -80,7 +80,7 @@ impl TopologySnapshotProvider for EndpointTopologySnapshotProvider { } pub fn topology_snapshot_from_endpoint_pools(endpoint_pools: &EndpointServerPools) -> TopologySnapshot { - crate::storage_compat::topology_snapshot_from_endpoint_pools_with_capabilities( + topology_snapshot_from_endpoint_pools_with_capabilities( endpoint_pools, TopologyCapabilities { profiling: cpu_profiling_status(), @@ -132,7 +132,7 @@ fn cgroup_memory_status() -> CapabilityStatus { #[cfg(test)] mod tests { use super::*; - use crate::storage_compat::{Endpoint, Endpoints, PoolEndpoints}; + use crate::runtime_capabilities_storage_compat::{Endpoint, Endpoints, PoolEndpoints}; use rustfs_storage_api::{CapabilityState, ObservabilitySnapshotProvider, TopologySnapshotProvider}; #[tokio::test] diff --git a/rustfs/src/runtime_capabilities_storage_compat.rs b/rustfs/src/runtime_capabilities_storage_compat.rs new file mode 100644 index 000000000..1f441251e --- /dev/null +++ b/rustfs/src/runtime_capabilities_storage_compat.rs @@ -0,0 +1,33 @@ +// Copyright 2024 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#[cfg(test)] +use rustfs_ecstore::api::disk as ecstore_disk; +use rustfs_ecstore::api::{cluster as ecstore_cluster, layout as ecstore_layout}; + +#[cfg(test)] +pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint; +pub(crate) type EndpointServerPools = ecstore_layout::EndpointServerPools; +#[cfg(test)] +pub(crate) type Endpoints = ecstore_layout::Endpoints; +#[cfg(test)] +pub(crate) type PoolEndpoints = ecstore_layout::PoolEndpoints; + +pub(crate) fn topology_snapshot_from_endpoint_pools_with_capabilities( + endpoint_pools: &EndpointServerPools, + capabilities: rustfs_storage_api::TopologyCapabilities, + disk_capabilities: rustfs_storage_api::DiskCapabilities, +) -> rustfs_storage_api::TopologySnapshot { + ecstore_cluster::topology_snapshot_from_endpoint_pools_with_capabilities(endpoint_pools, capabilities, disk_capabilities) +} diff --git a/rustfs/src/server/event.rs b/rustfs/src/server/event.rs index 27bb75e30..495067c76 100644 --- a/rustfs/src/server/event.rs +++ b/rustfs/src/server/event.rs @@ -12,10 +12,10 @@ // See the License for the specific language governing permissions and // limitations under the License. +use super::storage_compat::{EcstoreEventArgs, register_event_dispatch_hook}; use super::{module_switch::resolve_notify_module_state, refresh_persisted_module_switches_from_store}; use crate::app::context::resolve_server_config; use crate::storage::StorageObjectInfo; -use crate::storage_compat::{EcstoreEventArgs, register_event_dispatch_hook}; use chrono::{DateTime, Utc}; use rustfs_notify::{EventArgs as NotifyEventArgs, NotifyObjectInfo}; use rustfs_s3_types::EventName; diff --git a/rustfs/src/server/http.rs b/rustfs/src/server/http.rs index 63fe4efee..d84c38e00 100644 --- a/rustfs/src/server/http.rs +++ b/rustfs/src/server/http.rs @@ -13,6 +13,7 @@ // limitations under the License. // Import HTTP server components and compression configuration +use super::storage_compat::{TONIC_RPC_PREFIX, verify_rpc_signature}; use crate::admin; use crate::auth::IAMAuth; use crate::auth_keystone; @@ -33,7 +34,6 @@ use crate::server::{ use crate::storage; use crate::storage::rpc::InternodeRpcService; use crate::storage::tonic_service::make_server; -use crate::storage_compat::{TONIC_RPC_PREFIX, verify_rpc_signature}; use bytes::Bytes; use http::{HeaderMap, Method, Request as HttpRequest, Response}; use hyper_util::{ diff --git a/rustfs/src/server/mod.rs b/rustfs/src/server/mod.rs index 17691426d..f085c0d13 100644 --- a/rustfs/src/server/mod.rs +++ b/rustfs/src/server/mod.rs @@ -24,6 +24,7 @@ mod prefix; mod readiness; mod runtime; mod service_state; +mod storage_compat; pub mod tls_material; use tracing::warn; diff --git a/rustfs/src/server/module_switch.rs b/rustfs/src/server/module_switch.rs index 680bce14a..dac07725e 100644 --- a/rustfs/src/server/module_switch.rs +++ b/rustfs/src/server/module_switch.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_compat::{ +use super::storage_compat::{ EcstoreError as StorageError, read_ecstore_config, resolve_object_store_handle, save_ecstore_config, }; use serde::{Deserialize, Serialize}; diff --git a/rustfs/src/server/readiness.rs b/rustfs/src/server/readiness.rs index 28039ced7..220b42327 100644 --- a/rustfs/src/server/readiness.rs +++ b/rustfs/src/server/readiness.rs @@ -12,11 +12,12 @@ // See the License for the specific language governing permissions and // limitations under the License. +use super::storage_compat::{ + Endpoint, EndpointServerPools, get_global_endpoints_opt, get_global_lock_clients, is_dist_erasure, + resolve_object_store_handle, +}; use crate::server::{ServiceState, ServiceStateManager}; use crate::server::{has_path_prefix, is_table_catalog_path}; -use crate::storage_compat::is_dist_erasure; -use crate::storage_compat::resolve_object_store_handle; -use crate::storage_compat::{get_global_endpoints_opt, get_global_lock_clients}; use bytes::Bytes; use http::{Request as HttpRequest, Response, StatusCode}; use http_body::Body; @@ -496,10 +497,10 @@ async fn collect_storage_readiness_uncached() -> bool { } } -fn set_lock_quorum_status(online_hosts: &HashSet, set_endpoints: &[crate::storage_compat::Endpoint]) -> LockQuorumStatus { +fn set_lock_quorum_status(online_hosts: &HashSet, set_endpoints: &[Endpoint]) -> LockQuorumStatus { let total_clients = set_endpoints .iter() - .map(crate::storage_compat::Endpoint::host_port) + .map(Endpoint::host_port) .filter(|host| !host.is_empty()) .collect::>(); let total_clients_len = total_clients.len(); @@ -522,10 +523,7 @@ fn set_lock_quorum_status(online_hosts: &HashSet, set_endpoints: &[crate } } -fn aggregate_lock_quorum_status( - pool_endpoints: &crate::storage_compat::EndpointServerPools, - online_hosts: &HashSet, -) -> LockQuorumStatus { +fn aggregate_lock_quorum_status(pool_endpoints: &EndpointServerPools, online_hosts: &HashSet) -> LockQuorumStatus { let mut connected_clients = 0usize; let mut total_clients = 0usize; let mut required_quorum = 0usize; @@ -839,8 +837,7 @@ mod tests { #[test] fn aggregate_lock_quorum_status_requires_each_set_to_meet_quorum() { - use crate::storage_compat::Endpoint; - use crate::storage_compat::{EndpointServerPools, Endpoints, PoolEndpoints}; + use crate::server::storage_compat::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints}; let endpoints = vec![ Endpoint { @@ -897,8 +894,7 @@ mod tests { #[test] fn aggregate_lock_quorum_status_fails_when_any_set_loses_quorum() { - use crate::storage_compat::Endpoint; - use crate::storage_compat::{EndpointServerPools, Endpoints, PoolEndpoints}; + use crate::server::storage_compat::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints}; let endpoints = vec![ Endpoint { diff --git a/rustfs/src/server/storage_compat.rs b/rustfs/src/server/storage_compat.rs new file mode 100644 index 000000000..dceca3172 --- /dev/null +++ b/rustfs/src/server/storage_compat.rs @@ -0,0 +1,69 @@ +// Copyright 2024 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use std::{collections::HashMap, sync::Arc}; + +use rustfs_ecstore::api::{ + config as ecstore_config, disk as ecstore_disk, error as ecstore_error, event as ecstore_event, global as ecstore_global, + layout as ecstore_layout, rpc as ecstore_rpc, storage as ecstore_storage, +}; + +pub(crate) const TONIC_RPC_PREFIX: &str = ecstore_rpc::TONIC_RPC_PREFIX; + +pub(crate) type ECStore = ecstore_storage::ECStore; +pub(crate) type EcstoreError = ecstore_error::Error; +pub(crate) type EcstoreEventArgs = ecstore_event::EventArgs; +pub(crate) type EcstoreResult = ecstore_error::Result; +pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint; +pub(crate) type EndpointServerPools = ecstore_layout::EndpointServerPools; + +#[cfg(test)] +pub(crate) type Endpoints = ecstore_layout::Endpoints; +#[cfg(test)] +pub(crate) type PoolEndpoints = ecstore_layout::PoolEndpoints; + +pub(crate) async fn read_ecstore_config(api: Arc, file: &str) -> EcstoreResult> { + ecstore_config::com::read_config(api, file).await +} + +pub(crate) async fn save_ecstore_config(api: Arc, file: &str, data: Vec) -> EcstoreResult<()> { + ecstore_config::com::save_config(api, file, data).await +} + +pub(crate) fn register_event_dispatch_hook(hook: F) -> bool +where + F: Fn(EcstoreEventArgs) + Send + Sync + 'static, +{ + ecstore_event::register_event_dispatch_hook(hook) +} + +pub(crate) fn get_global_endpoints_opt() -> Option { + ecstore_global::get_global_endpoints_opt() +} + +pub(crate) fn get_global_lock_clients() -> Option<&'static HashMap>> { + ecstore_global::get_global_lock_clients() +} + +pub(crate) async fn is_dist_erasure() -> bool { + ecstore_global::is_dist_erasure().await +} + +pub(crate) fn resolve_object_store_handle() -> Option> { + ecstore_global::resolve_object_store_handle() +} + +pub(crate) fn verify_rpc_signature(url: &str, method: &http::Method, headers: &http::HeaderMap) -> std::io::Result<()> { + ecstore_rpc::verify_rpc_signature(url, method, headers) +} diff --git a/rustfs/src/startup_background.rs b/rustfs/src/startup_background.rs index d12c2d8fb..279ec538f 100644 --- a/rustfs/src/startup_background.rs +++ b/rustfs/src/startup_background.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_compat::ECStore; +use crate::startup_storage_compat::ECStore; use rustfs_heal::{create_ahm_services_cancel_token, heal::storage::ECStoreHealStorage, init_heal_manager}; use rustfs_utils::get_env_bool_with_aliases; use std::{io::Result, sync::Arc}; diff --git a/rustfs/src/startup_bucket_metadata.rs b/rustfs/src/startup_bucket_metadata.rs index 88549cfaf..d8b7937f8 100644 --- a/rustfs/src/startup_bucket_metadata.rs +++ b/rustfs/src/startup_bucket_metadata.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_compat::{ +use crate::startup_storage_compat::{ ECStore, get_global_replication_pool, init_bucket_metadata_sys, try_migrate_bucket_metadata, try_migrate_iam_config, }; use rustfs_storage_api::{BucketOperations, BucketOptions}; diff --git a/rustfs/src/startup_fs_guard.rs b/rustfs/src/startup_fs_guard.rs index 5ff2f7295..bef46caa0 100644 --- a/rustfs/src/startup_fs_guard.rs +++ b/rustfs/src/startup_fs_guard.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_compat::EndpointServerPools; +use crate::startup_storage_compat::EndpointServerPools; use rustfs_config::{ DEFAULT_RUSTFS_UNSUPPORTED_FS_POLICY, ENV_RUSTFS_UNSUPPORTED_FS_POLICY, RUSTFS_UNSUPPORTED_FS_POLICY_FAIL, RUSTFS_UNSUPPORTED_FS_POLICY_WARN, diff --git a/rustfs/src/startup_iam.rs b/rustfs/src/startup_iam.rs index fe32143a3..53bc68f2c 100644 --- a/rustfs/src/startup_iam.rs +++ b/rustfs/src/startup_iam.rs @@ -14,7 +14,7 @@ use crate::app::context::AppContext; use crate::server::{ServiceStateManager, publish_ready_when_runtime_ready}; -use crate::storage_compat::ECStore; +use crate::startup_storage_compat::ECStore; use rustfs_common::{GlobalReadiness, SystemStage}; use rustfs_iam::init_iam_sys; use rustfs_kms::KmsServiceManager; diff --git a/rustfs/src/startup_lifecycle.rs b/rustfs/src/startup_lifecycle.rs index e3dca2ac7..93ecef737 100644 --- a/rustfs/src/startup_lifecycle.rs +++ b/rustfs/src/startup_lifecycle.rs @@ -17,7 +17,7 @@ use crate::{ startup_iam::{IamBootstrapDisposition, publish_ready_for_iam_bootstrap}, startup_services::StartupServiceRuntime, startup_shutdown::run_startup_shutdown_sequence, - storage_compat::ECStore, + startup_storage_compat::ECStore, }; use rustfs_common::GlobalReadiness; use rustfs_scanner::init_data_scanner; diff --git a/rustfs/src/startup_notification.rs b/rustfs/src/startup_notification.rs index 1d475a789..dbb18d628 100644 --- a/rustfs/src/startup_notification.rs +++ b/rustfs/src/startup_notification.rs @@ -13,7 +13,7 @@ // limitations under the License. use crate::init::add_bucket_notification_configuration; -use crate::storage_compat::{EndpointServerPools, new_global_notification_sys}; +use crate::startup_storage_compat::{EcstoreResult, EndpointServerPools, new_global_notification_sys}; use std::{ future::Future, io::{Error, Result}, @@ -50,14 +50,14 @@ pub(crate) async fn init_notification_runtime(endpoint_pools: EndpointServerPool }) } -pub(crate) async fn init_notification_system(endpoint_pools: EndpointServerPools) -> crate::storage_compat::EcstoreResult<()> { +pub(crate) async fn init_notification_system(endpoint_pools: EndpointServerPools) -> EcstoreResult<()> { init_notification_system_with(|| new_global_notification_sys(endpoint_pools)).await } -async fn init_notification_system_with(init_notification: InitFn) -> crate::storage_compat::EcstoreResult<()> +async fn init_notification_system_with(init_notification: InitFn) -> EcstoreResult<()> where InitFn: FnOnce() -> InitFuture, - InitFuture: Future>, + InitFuture: Future>, { init_notification().await } @@ -79,7 +79,8 @@ mod tests { #[tokio::test] async fn notification_system_returns_source_error() { - let result = init_notification_system_with(|| async { Err(crate::storage_compat::EcstoreError::FaultyDisk) }).await; + let result = + init_notification_system_with(|| async { Err(crate::startup_storage_compat::EcstoreError::FaultyDisk) }).await; assert!(result.is_err()); } diff --git a/rustfs/src/startup_server.rs b/rustfs/src/startup_server.rs index f5bac7961..b8124c833 100644 --- a/rustfs/src/startup_server.rs +++ b/rustfs/src/startup_server.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_compat::set_global_rustfs_port; +use crate::startup_storage_compat::{set_global_region, set_global_rustfs_port}; use crate::{ capacity::capacity_integration::init_capacity_management, config::Config, @@ -82,7 +82,7 @@ pub(crate) async fn init_startup_listen_context(config: &Config) -> Result() - .map(crate::storage_compat::set_global_region) + .map(set_global_region) .map_err(|err| Error::other(format!("invalid region '{}': {}", region_str, err)))?; } @@ -190,7 +190,7 @@ pub(crate) async fn init_embedded_startup_listen_context(config: &Config) -> Res if let Some(region_str) = &config.region { region_str .parse::() - .map(crate::storage_compat::set_global_region) + .map(set_global_region) .map_err(|err| Error::other(format!("invalid region '{region_str}': {err}")))?; } diff --git a/rustfs/src/startup_services.rs b/rustfs/src/startup_services.rs index 8389b4061..f48c7e7f7 100644 --- a/rustfs/src/startup_services.rs +++ b/rustfs/src/startup_services.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_compat::{ECStore, EndpointServerPools}; +use crate::startup_storage_compat::{ECStore, EndpointServerPools}; use crate::{ config::Config, init::{init_buffer_profile_system, init_kms_system}, diff --git a/rustfs/src/startup_shutdown.rs b/rustfs/src/startup_shutdown.rs index eb9415dc7..4412600b3 100644 --- a/rustfs/src/startup_shutdown.rs +++ b/rustfs/src/startup_shutdown.rs @@ -17,7 +17,7 @@ use crate::{ startup_optional_runtime_sidecars::{ OptionalRuntimeServices, prepare_optional_runtime_shutdowns, shutdown_optional_runtime_services, }, - storage_compat::shutdown_background_services, + startup_storage_compat::shutdown_background_services, }; use rustfs_heal::shutdown_ahm_services; use rustfs_utils::get_env_bool_with_aliases; diff --git a/rustfs/src/startup_storage.rs b/rustfs/src/startup_storage.rs index 364c3e60e..059bd1aa3 100644 --- a/rustfs/src/startup_storage.rs +++ b/rustfs/src/startup_storage.rs @@ -13,7 +13,7 @@ // limitations under the License. use crate::startup_fs_guard::enforce_unsupported_fs_policy; -use crate::storage_compat::{ +use crate::startup_storage_compat::{ ECStore, EndpointServerPools, init_background_replication, init_ecstore_config, init_global_config_sys, init_local_disks, init_lock_clients, prewarm_local_disk_id_map, set_global_endpoints, try_migrate_server_config, update_erasure_type, }; diff --git a/rustfs/src/startup_storage_compat.rs b/rustfs/src/startup_storage_compat.rs new file mode 100644 index 000000000..0b37f5b85 --- /dev/null +++ b/rustfs/src/startup_storage_compat.rs @@ -0,0 +1,102 @@ +// Copyright 2024 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use std::sync::Arc; + +use rustfs_ecstore::api::{ + bucket as ecstore_bucket, config as ecstore_config, error as ecstore_error, global as ecstore_global, + layout as ecstore_layout, notification as ecstore_notification, storage as ecstore_storage, +}; + +pub(crate) type ECStore = ecstore_storage::ECStore; +#[cfg(test)] +pub(crate) type EcstoreError = ecstore_error::Error; +pub(crate) type EcstoreResult = ecstore_error::Result; +pub(crate) type EndpointServerPools = ecstore_layout::EndpointServerPools; + +pub(crate) async fn get_notification_config(bucket: &str) -> ecstore_error::Result> { + ecstore_bucket::metadata_sys::get_notification_config(bucket).await +} + +pub(crate) async fn init_bucket_metadata_sys(api: Arc, buckets: Vec) { + ecstore_bucket::metadata_sys::init_bucket_metadata_sys(api, buckets).await; +} + +pub(crate) async fn try_migrate_bucket_metadata(store: Arc) { + ecstore_bucket::migration::try_migrate_bucket_metadata(store).await; +} + +pub(crate) async fn try_migrate_iam_config(store: Arc) { + ecstore_bucket::migration::try_migrate_iam_config(store).await; +} + +pub(crate) fn get_global_replication_pool() -> Option> { + ecstore_bucket::replication::get_global_replication_pool() +} + +pub(crate) async fn init_background_replication(storage: Arc) { + ecstore_bucket::replication::init_background_replication(storage).await; +} + +pub(crate) fn init_ecstore_config() { + ecstore_config::init(); +} + +pub(crate) async fn init_global_config_sys(api: Arc) -> EcstoreResult<()> { + ecstore_config::init_global_config_sys(api).await +} + +pub(crate) async fn try_migrate_server_config(api: Arc) { + ecstore_config::try_migrate_server_config(api).await; +} + +pub(crate) fn get_global_region() -> Option { + ecstore_global::get_global_region() +} + +pub(crate) fn set_global_endpoints(eps: Vec) { + ecstore_global::set_global_endpoints(eps); +} + +pub(crate) fn set_global_region(region: s3s::region::Region) { + ecstore_global::set_global_region(region); +} + +pub(crate) fn set_global_rustfs_port(value: u16) { + ecstore_global::set_global_rustfs_port(value); +} + +pub(crate) fn shutdown_background_services() { + ecstore_global::shutdown_background_services(); +} + +pub(crate) async fn update_erasure_type(setup_type: ecstore_layout::SetupType) { + ecstore_global::update_erasure_type(setup_type).await +} + +pub(crate) async fn new_global_notification_sys(eps: EndpointServerPools) -> EcstoreResult<()> { + ecstore_notification::new_global_notification_sys(eps).await +} + +pub(crate) async fn init_local_disks(endpoint_pools: EndpointServerPools) -> EcstoreResult<()> { + ecstore_storage::init_local_disks(endpoint_pools).await +} + +pub(crate) fn init_lock_clients(endpoint_pools: EndpointServerPools) { + ecstore_storage::init_lock_clients(endpoint_pools); +} + +pub(crate) async fn prewarm_local_disk_id_map() { + ecstore_storage::prewarm_local_disk_id_map().await; +} diff --git a/rustfs/src/storage_compat.rs b/rustfs/src/storage_compat.rs deleted file mode 100644 index c7af4f74b..000000000 --- a/rustfs/src/storage_compat.rs +++ /dev/null @@ -1,213 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -use std::{collections::HashMap, sync::Arc, time::Duration}; - -use rustfs_ecstore::api::{ - bucket as ecstore_bucket, cluster as ecstore_cluster, config as ecstore_config, disk as ecstore_disk, error as ecstore_error, - event as ecstore_event, global as ecstore_global, layout as ecstore_layout, notification as ecstore_notification, - rpc as ecstore_rpc, set_disk as ecstore_set_disk, storage as ecstore_storage, -}; - -pub(crate) const BUCKET_TABLE_CATALOG_META_PREFIX: &str = ecstore_bucket::metadata::BUCKET_TABLE_CATALOG_META_PREFIX; -pub(crate) const BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX: &str = - ecstore_bucket::metadata::BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX; -pub(crate) const BUCKET_TABLE_CONFIG: &str = ecstore_bucket::metadata::BUCKET_TABLE_CONFIG; -pub(crate) const BUCKET_TABLE_RESERVED_PREFIX: &str = ecstore_bucket::metadata::BUCKET_TABLE_RESERVED_PREFIX; -pub(crate) const RUSTFS_META_BUCKET: &str = ecstore_disk::RUSTFS_META_BUCKET; -pub(crate) const TONIC_RPC_PREFIX: &str = ecstore_rpc::TONIC_RPC_PREFIX; - -pub(crate) type ECStore = ecstore_storage::ECStore; -pub(crate) type EcstoreError = ecstore_error::Error; -pub(crate) type EcstoreEventArgs = ecstore_event::EventArgs; -pub(crate) type EcstoreResult = ecstore_error::Result; -pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint; -pub(crate) type EndpointServerPools = ecstore_layout::EndpointServerPools; -pub(crate) type QuotaError = ecstore_bucket::quota::QuotaError; -pub(crate) type StorageError = ecstore_error::StorageError; - -#[cfg(test)] -pub(crate) type DisksLayout = ecstore_layout::DisksLayout; -#[cfg(test)] -pub(crate) type Endpoints = ecstore_layout::Endpoints; -#[cfg(test)] -pub(crate) type PoolEndpoints = ecstore_layout::PoolEndpoints; - -pub(crate) trait DiskAPI { - fn endpoint(&self) -> Endpoint; - fn to_string(&self) -> String; -} - -impl DiskAPI for T -where - T: ecstore_disk::DiskAPI, -{ - fn endpoint(&self) -> Endpoint { - ecstore_disk::DiskAPI::endpoint(self) - } - - fn to_string(&self) -> String { - ecstore_disk::DiskAPI::to_string(self) - } -} - -pub(crate) fn table_catalog_path_hash(value: &str) -> String { - ecstore_bucket::metadata::table_catalog_path_hash(value) -} - -pub(crate) async fn get_bucket_metadata(bucket: &str) -> ecstore_error::Result> { - ecstore_bucket::metadata_sys::get(bucket).await -} - -pub(crate) fn get_global_bucket_metadata_sys() -> Option>> -{ - ecstore_bucket::metadata_sys::get_global_bucket_metadata_sys() -} - -pub(crate) async fn get_notification_config(bucket: &str) -> ecstore_error::Result> { - ecstore_bucket::metadata_sys::get_notification_config(bucket).await -} - -pub(crate) async fn init_bucket_metadata_sys(api: Arc, buckets: Vec) { - ecstore_bucket::metadata_sys::init_bucket_metadata_sys(api, buckets).await; -} - -pub(crate) async fn try_migrate_bucket_metadata(store: Arc) { - ecstore_bucket::migration::try_migrate_bucket_metadata(store).await; -} - -pub(crate) async fn try_migrate_iam_config(store: Arc) { - ecstore_bucket::migration::try_migrate_iam_config(store).await; -} - -pub(crate) fn get_global_replication_pool() -> Option> { - ecstore_bucket::replication::get_global_replication_pool() -} - -pub(crate) fn replication_queue_current_count() -> Option { - ecstore_bucket::replication::GLOBAL_REPLICATION_STATS.get().and_then(|stats| { - stats - .q_cache - .try_lock() - .ok() - .map(|cache| cache.sr_queue_stats.curr.get_current_count()) - }) -} - -pub(crate) async fn init_background_replication(storage: Arc) { - ecstore_bucket::replication::init_background_replication(storage).await; -} - -pub(crate) fn topology_snapshot_from_endpoint_pools_with_capabilities( - endpoint_pools: &EndpointServerPools, - capabilities: rustfs_storage_api::TopologyCapabilities, - disk_capabilities: rustfs_storage_api::DiskCapabilities, -) -> rustfs_storage_api::TopologySnapshot { - ecstore_cluster::topology_snapshot_from_endpoint_pools_with_capabilities(endpoint_pools, capabilities, disk_capabilities) -} - -pub(crate) async fn read_ecstore_config(api: Arc, file: &str) -> EcstoreResult> { - ecstore_config::com::read_config(api, file).await -} - -pub(crate) async fn save_ecstore_config(api: Arc, file: &str, data: Vec) -> EcstoreResult<()> { - ecstore_config::com::save_config(api, file, data).await -} - -pub(crate) fn init_ecstore_config() { - ecstore_config::init(); -} - -pub(crate) async fn init_global_config_sys(api: Arc) -> EcstoreResult<()> { - ecstore_config::init_global_config_sys(api).await -} - -pub(crate) async fn try_migrate_server_config(api: Arc) { - ecstore_config::try_migrate_server_config(api).await; -} - -pub(crate) fn register_event_dispatch_hook(hook: F) -> bool -where - F: Fn(EcstoreEventArgs) + Send + Sync + 'static, -{ - ecstore_event::register_event_dispatch_hook(hook) -} - -pub(crate) fn get_global_endpoints_opt() -> Option { - ecstore_global::get_global_endpoints_opt() -} - -pub(crate) fn get_global_lock_clients() -> Option<&'static HashMap>> { - ecstore_global::get_global_lock_clients() -} - -pub(crate) fn get_global_region() -> Option { - ecstore_global::get_global_region() -} - -pub(crate) async fn is_dist_erasure() -> bool { - ecstore_global::is_dist_erasure().await -} - -pub(crate) fn resolve_object_store_handle() -> Option> { - ecstore_global::resolve_object_store_handle() -} - -pub(crate) fn set_global_endpoints(eps: Vec) { - ecstore_global::set_global_endpoints(eps); -} - -pub(crate) fn set_global_region(region: s3s::region::Region) { - ecstore_global::set_global_region(region); -} - -pub(crate) fn set_global_rustfs_port(value: u16) { - ecstore_global::set_global_rustfs_port(value); -} - -pub(crate) fn shutdown_background_services() { - ecstore_global::shutdown_background_services(); -} - -pub(crate) async fn update_erasure_type(setup_type: ecstore_layout::SetupType) { - ecstore_global::update_erasure_type(setup_type).await; -} - -pub(crate) async fn new_global_notification_sys(eps: EndpointServerPools) -> EcstoreResult<()> { - ecstore_notification::new_global_notification_sys(eps).await -} - -pub(crate) fn verify_rpc_signature(url: &str, method: &http::Method, headers: &http::HeaderMap) -> std::io::Result<()> { - ecstore_rpc::verify_rpc_signature(url, method, headers) -} - -pub(crate) fn get_lock_acquire_timeout() -> Duration { - ecstore_set_disk::get_lock_acquire_timeout() -} - -pub(crate) async fn all_local_disk() -> Vec { - ecstore_storage::all_local_disk().await -} - -pub(crate) async fn init_local_disks(endpoint_pools: EndpointServerPools) -> EcstoreResult<()> { - ecstore_storage::init_local_disks(endpoint_pools).await -} - -pub(crate) fn init_lock_clients(endpoint_pools: EndpointServerPools) { - ecstore_storage::init_lock_clients(endpoint_pools); -} - -pub(crate) async fn prewarm_local_disk_id_map() { - ecstore_storage::prewarm_local_disk_id_map().await; -} diff --git a/rustfs/src/table_catalog.rs b/rustfs/src/table_catalog.rs index 124ea9fab..3b20c5ed2 100644 --- a/rustfs/src/table_catalog.rs +++ b/rustfs/src/table_catalog.rs @@ -27,11 +27,10 @@ use std::{ time::{Duration as StdDuration, Instant}, }; -use crate::storage_compat::RUSTFS_META_BUCKET; -use crate::storage_compat::get_lock_acquire_timeout; -use crate::storage_compat::{ +use crate::table_catalog_storage_compat::{ BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG, - BUCKET_TABLE_RESERVED_PREFIX, EcstoreError, StorageError, get_bucket_metadata, table_catalog_path_hash, + BUCKET_TABLE_RESERVED_PREFIX, EcstoreError, RUSTFS_META_BUCKET, StorageError, get_bucket_metadata, get_lock_acquire_timeout, + table_catalog_path_hash, }; use bytes::Bytes; use datafusion::{ @@ -7886,7 +7885,7 @@ mod tests { .map(|(bucket, _)| bucket.as_str()) .collect::>(); - assert_eq!(object_buckets, BTreeSet::from([crate::storage_compat::RUSTFS_META_BUCKET])); + assert_eq!(object_buckets, BTreeSet::from([RUSTFS_META_BUCKET])); } #[tokio::test] @@ -10931,9 +10930,7 @@ mod tests { .await .unwrap(); backend.seed_object(bucket, &new_metadata, b"{}".to_vec()).await; - backend - .fail_put_attempt(crate::storage_compat::RUSTFS_META_BUCKET, &idempotency_path, 1) - .await; + backend.fail_put_attempt(RUSTFS_META_BUCKET, &idempotency_path, 1).await; let err = store .commit_table(TableCommitRequest { @@ -10981,9 +10978,7 @@ mod tests { .await .unwrap(); backend.seed_object(bucket, &new_metadata, b"{}".to_vec()).await; - backend - .fail_put_attempt(crate::storage_compat::RUSTFS_META_BUCKET, &commit_path, 2) - .await; + backend.fail_put_attempt(RUSTFS_META_BUCKET, &commit_path, 2).await; let request = TableCommitRequest { table_bucket: bucket.to_string(), @@ -11036,9 +11031,7 @@ mod tests { .await .unwrap(); backend.seed_object(bucket, &new_metadata, b"{}".to_vec()).await; - backend - .fail_put_attempt(crate::storage_compat::RUSTFS_META_BUCKET, &commit_path, 2) - .await; + backend.fail_put_attempt(RUSTFS_META_BUCKET, &commit_path, 2).await; let result = store .commit_table(TableCommitRequest { @@ -11090,9 +11083,7 @@ mod tests { .await .unwrap(); backend.seed_object(bucket, &new_metadata, b"{}".to_vec()).await; - backend - .fail_put_attempt(crate::storage_compat::RUSTFS_META_BUCKET, &commit_path, 2) - .await; + backend.fail_put_attempt(RUSTFS_META_BUCKET, &commit_path, 2).await; store .commit_table(TableCommitRequest { @@ -11144,9 +11135,7 @@ mod tests { .await .unwrap(); backend.seed_object(bucket, &new_metadata, b"{}".to_vec()).await; - backend - .fail_put_attempt(crate::storage_compat::RUSTFS_META_BUCKET, &commit_path, 2) - .await; + backend.fail_put_attempt(RUSTFS_META_BUCKET, &commit_path, 2).await; store .commit_table(TableCommitRequest { @@ -11195,9 +11184,7 @@ mod tests { .unwrap(); backend.seed_object(bucket, ¤t_metadata, b"{}".to_vec()).await; backend.seed_object(bucket, &new_metadata, b"{}".to_vec()).await; - backend - .fail_put_attempt(crate::storage_compat::RUSTFS_META_BUCKET, &table_path, 2) - .await; + backend.fail_put_attempt(RUSTFS_META_BUCKET, &table_path, 2).await; let err = store .commit_table(TableCommitRequest { @@ -11252,9 +11239,7 @@ mod tests { .await .unwrap(); backend.seed_object(bucket, &new_metadata, b"{}".to_vec()).await; - backend - .fail_put_attempt(crate::storage_compat::RUSTFS_META_BUCKET, &idempotency_path, 2) - .await; + backend.fail_put_attempt(RUSTFS_META_BUCKET, &idempotency_path, 2).await; let result = store .commit_table(TableCommitRequest { diff --git a/rustfs/src/table_catalog_storage_compat.rs b/rustfs/src/table_catalog_storage_compat.rs new file mode 100644 index 000000000..8322ca159 --- /dev/null +++ b/rustfs/src/table_catalog_storage_compat.rs @@ -0,0 +1,38 @@ +// Copyright 2024 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use std::{sync::Arc, time::Duration}; + +use rustfs_ecstore::api::{bucket as ecstore_bucket, disk as ecstore_disk, error as ecstore_error, set_disk as ecstore_set_disk}; + +pub(crate) const BUCKET_TABLE_CATALOG_META_PREFIX: &str = ecstore_bucket::metadata::BUCKET_TABLE_CATALOG_META_PREFIX; +pub(crate) const BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX: &str = + ecstore_bucket::metadata::BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX; +pub(crate) const BUCKET_TABLE_CONFIG: &str = ecstore_bucket::metadata::BUCKET_TABLE_CONFIG; +pub(crate) const BUCKET_TABLE_RESERVED_PREFIX: &str = ecstore_bucket::metadata::BUCKET_TABLE_RESERVED_PREFIX; +pub(crate) const RUSTFS_META_BUCKET: &str = ecstore_disk::RUSTFS_META_BUCKET; +pub(crate) type EcstoreError = ecstore_error::Error; +pub(crate) type StorageError = ecstore_error::StorageError; + +pub(crate) fn table_catalog_path_hash(value: &str) -> String { + ecstore_bucket::metadata::table_catalog_path_hash(value) +} + +pub(crate) async fn get_bucket_metadata(bucket: &str) -> ecstore_error::Result> { + ecstore_bucket::metadata_sys::get(bucket).await +} + +pub(crate) fn get_lock_acquire_timeout() -> Duration { + ecstore_set_disk::get_lock_acquire_timeout() +} diff --git a/rustfs/src/workload_admission.rs b/rustfs/src/workload_admission.rs index 9d761e529..bdd1ac3a9 100644 --- a/rustfs/src/workload_admission.rs +++ b/rustfs/src/workload_admission.rs @@ -18,7 +18,9 @@ use rustfs_concurrency::{ }; use crate::storage::concurrency::get_concurrency_manager; -use crate::storage_compat::{get_global_bucket_metadata_sys, get_global_replication_pool, replication_queue_current_count}; +use crate::workload_admission_storage_compat::{ + get_global_bucket_metadata_sys, get_global_replication_pool, replication_queue_current_count, +}; const BUCKET_METADATA_RUNTIME_NOT_INITIALIZED: &str = "bucket metadata runtime not initialized"; const HEAL_MANAGER_NOT_INITIALIZED: &str = "heal manager not initialized"; diff --git a/rustfs/src/workload_admission_storage_compat.rs b/rustfs/src/workload_admission_storage_compat.rs new file mode 100644 index 000000000..20a983e22 --- /dev/null +++ b/rustfs/src/workload_admission_storage_compat.rs @@ -0,0 +1,36 @@ +// Copyright 2024 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use std::sync::Arc; + +use rustfs_ecstore::api::bucket as ecstore_bucket; + +pub(crate) fn get_global_bucket_metadata_sys() -> Option>> +{ + ecstore_bucket::metadata_sys::get_global_bucket_metadata_sys() +} + +pub(crate) fn get_global_replication_pool() -> Option> { + ecstore_bucket::replication::get_global_replication_pool() +} + +pub(crate) fn replication_queue_current_count() -> Option { + ecstore_bucket::replication::GLOBAL_REPLICATION_STATS.get().and_then(|stats| { + stats + .q_cache + .try_lock() + .ok() + .map(|cache| cache.sr_queue_stats.curr.get_current_count()) + }) +} diff --git a/scripts/check_architecture_migration_rules.sh b/scripts/check_architecture_migration_rules.sh index 5a237f32f..ba6499892 100755 --- a/scripts/check_architecture_migration_rules.sh +++ b/scripts/check_architecture_migration_rules.sh @@ -81,6 +81,7 @@ RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_runtime_sec RUSTFS_ROOT_STORAGE_COMPAT_REEXPORT_HITS_FILE="${TMP_DIR}/rustfs_root_storage_compat_reexport_hits.txt" RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_bucket_storage_compat_module_hits.txt" RUSTFS_ROOT_RUNTIME_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_runtime_storage_compat_module_hits.txt" +RUSTFS_ROOT_CONSUMER_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_root_consumer_compat_hits.txt" RUSTFS_ADMIN_CONFIG_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_admin_config_storage_compat_module_hits.txt" RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_storage_bucket_storage_compat_module_hits.txt" RUSTFS_STORAGE_OWNER_COMPAT_REEXPORT_HITS_FILE="${TMP_DIR}/rustfs_storage_owner_compat_reexport_hits.txt" @@ -689,6 +690,7 @@ fi rustfs/src crates fuzz/fuzz_targets \ --glob '!crates/ecstore/**' \ --glob '!**/storage_compat.rs' \ + --glob '!**/*storage_compat.rs' \ --glob '!target/**' || true ) | cat >"$DIRECT_ECSTORE_IMPORT_HITS_FILE" @@ -788,7 +790,7 @@ fi ( cd "$ROOT_DIR" rg -n --no-heading 'pub\(crate\)\s+use\s+rustfs_ecstore::api::' \ - rustfs/src/storage_compat.rs || true + rustfs/src/storage_compat.rs 2>/dev/null || true ) >"$RUSTFS_ROOT_STORAGE_COMPAT_REEXPORT_HITS_FILE" if [[ -s "$RUSTFS_ROOT_STORAGE_COMPAT_REEXPORT_HITS_FILE" ]]; then @@ -798,7 +800,7 @@ fi ( cd "$ROOT_DIR" rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::bucket::\{[^}]*\b(?:metadata|metadata_sys|quota)\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::bucket::(?:metadata|metadata_sys|quota)\s*;' \ - rustfs/src/storage_compat.rs || true + rustfs/src/storage_compat.rs 2>/dev/null || true ) >"$RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" if [[ -s "$RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then @@ -808,13 +810,23 @@ fi ( cd "$ROOT_DIR" rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::config::\{[^}]*\bcom\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::config::\{[^}]*\binit\s*(?:,|})[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::disk::\{[^}]*\bendpoint::Endpoint\b[^}]*\}\s*;' \ - rustfs/src/storage_compat.rs || true + rustfs/src/storage_compat.rs 2>/dev/null || true ) >"$RUSTFS_ROOT_RUNTIME_STORAGE_COMPAT_MODULE_HITS_FILE" if [[ -s "$RUSTFS_ROOT_RUNTIME_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then report_failure "RustFS root storage compatibility must expose config init and disk endpoint contracts as explicit aliases: $(paste -sd '; ' "$RUSTFS_ROOT_RUNTIME_STORAGE_COMPAT_MODULE_HITS_FILE")" fi +( + cd "$ROOT_DIR" + rg -n --no-heading 'pub\(crate\)\s+(?:async\s+)?fn\s+(?:all_local_disk|read_ecstore_config|save_ecstore_config|get_global_endpoints_opt|get_global_lock_clients|is_dist_erasure|resolve_object_store_handle|register_event_dispatch_hook|verify_rpc_signature|get_notification_config|init_bucket_metadata_sys|try_migrate_bucket_metadata|try_migrate_iam_config|init_background_replication|init_ecstore_config|init_global_config_sys|try_migrate_server_config|get_global_region|set_global_endpoints|set_global_region|set_global_rustfs_port|shutdown_background_services|update_erasure_type|new_global_notification_sys|init_local_disks|init_lock_clients|prewarm_local_disk_id_map)\b|pub\(crate\)\s+(?:const|type)\s+(?:TONIC_RPC_PREFIX|EcstoreEventArgs|ECStore|EcstoreResult)\b|pub\(crate\)\s+trait\s+DiskAPI\b' \ + rustfs/src/storage_compat.rs 2>/dev/null || true +) >"$RUSTFS_ROOT_CONSUMER_COMPAT_HITS_FILE" + +if [[ -s "$RUSTFS_ROOT_CONSUMER_COMPAT_HITS_FILE" ]]; then + report_failure "RustFS root storage compatibility must not own capacity/server/startup consumer wrappers: $(paste -sd '; ' "$RUSTFS_ROOT_CONSUMER_COMPAT_HITS_FILE")" +fi + ( cd "$ROOT_DIR" rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::config::\{[^}]*\bcom\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::config::\{[^}]*\binit\s*(?:,|})[^}]*\}\s*;' \ @@ -942,7 +954,7 @@ fi ( cd "$ROOT_DIR" - rg -n --no-heading 'rustfs_ecstore::api::' rustfs/src/storage_compat.rs crates/e2e_test/src/storage_compat.rs \ + rg -n --with-filename --no-heading 'rustfs_ecstore::api::' crates/e2e_test/src/storage_compat.rs \ | rg -v '^[^:]+:[0-9]+:use rustfs_ecstore::api::\{' || true ) >"$RUSTFS_ROOT_E2E_COMPAT_RAW_FACADE_PATH_HITS_FILE"