refactor: route root storage facades (#3867)

This commit is contained in:
Zhengchao An
2026-06-25 23:23:45 +08:00
committed by GitHub
parent 987ea4919f
commit 33b7f48f37
32 changed files with 169 additions and 57 deletions
+31 -7
View File
@@ -5,9 +5,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-admin-facade-storage-api-cleanup`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215/API-216`.
- Based on: API-216 merged; branch replays the admin storage_api cleanup on latest `main`.
- Branch: `overtrue/arch-root-server-storage-facade-cleanup`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215/API-216/API-217`.
- Based on: API-217 merged; branch routes root/server/startup storage facade consumers through a root-local `storage_api` boundary on latest `main`.
- PR type for this branch: `consumer-migration`
- Runtime behavior changes: none.
- Rust code changes: route replication pool, outbound TLS generation, runtime
@@ -49,7 +49,11 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
admin root storage facade consumers for config, bucket metadata, replication,
rebalancing, tier, quota, metrics, object zip, site replication, and admin
service config paths, through AppContext-first or owner-crate resolver
boundaries.
boundaries, plus root/server/startup storage facade consumers for startup
storage, bucket metadata, notification, services, readiness, HTTP/layer
request context, module switches, event dispatch, cluster/runtime snapshots,
capacity, workload admission, table catalog, init, protocol clients, and
config tests through a root-local storage_api boundary.
- CI/script changes: lock completed owner and test/fuzz boundaries against
bare/glob imports, scattered raw ECStore facade subpaths, and startup
runtime/root-server/table/S3/app shared/app bucket/app ECStore/admin facade
@@ -59,8 +63,8 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
event-bridge thin module regressions, plus IAM runtime-source bypasses;
accept the reviewed AppContext resolver reverse dependencies in the layer
baseline, and block direct admin AppContext resolver consumers outside the
admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, reject app usecase storage wildcard imports, reject app-layer S3 DTO and ECFS wildcard imports, narrow the object-usecase ECFS layer baseline entry to `FS`, reject direct storage S3 API helper imports from app usecase files, reject direct storage helper imports from app select/usecase files, reject completed app/admin storage helper bypasses, reject app usecase bypasses for migrated storage IO/compression/set-disk helpers, reject app usecase/test bypasses for migrated storage error, ETag, and storage-class helpers, reject app root bucket owner facade bypasses from migrated app consumers, reject app/admin runtime/data-usage root facade regressions, and reject admin root storage facade regressions from migrated admin consumers.
- Docs changes: record the API-136 through API-217 owner facade and lifecycle
admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, reject app usecase storage wildcard imports, reject app-layer S3 DTO and ECFS wildcard imports, narrow the object-usecase ECFS layer baseline entry to `FS`, reject direct storage S3 API helper imports from app usecase files, reject direct storage helper imports from app select/usecase files, reject completed app/admin storage helper bypasses, reject app usecase bypasses for migrated storage IO/compression/set-disk helpers, reject app usecase/test bypasses for migrated storage error, ETag, and storage-class helpers, reject app root bucket owner facade bypasses from migrated app consumers, reject app/admin runtime/data-usage root facade regressions, reject admin root storage facade regressions from migrated admin consumers, and reject root/server/startup direct storage facade regressions from migrated outer consumers.
- Docs changes: record the API-136 through API-218 owner facade and lifecycle
runtime-source cleanup.
## Phase 0 Tasks
@@ -5140,14 +5144,34 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
layer guards, diff hygiene, residual migrated facade scan, Rust risk scan,
fast PR gate, and full PR gate before PR.
- [x] `API-218` Route root/server/startup storage facades through storage_api boundary.
- Do: add a root-local `storage_api` boundary for storage owner symbols used
by startup, server, capacity, cluster/runtime snapshot, table catalog,
protocol client, init, workload admission, and config-test consumers, then
route those outer runtime modules through it.
- Acceptance: root/server/startup consumers no longer import storage owner
facades directly, and migration rules reject direct `crate::storage`
regressions outside storage/app/admin owner boundaries.
- Must preserve: startup storage initialization, bucket metadata migration,
notification initialization, readiness gating, RPC request context,
module switch persistence, event dispatch, cluster/runtime snapshots,
capacity disk refresh, workload admission snapshots, table catalog object
contracts, and config test layout coverage.
- Verification: focused RustFS compile/tests, formatting, migration and
layer guards, diff hygiene, residual migrated facade scan, Rust risk scan,
fast PR gate, and full PR gate before PR.
## Next PRs
1. `consumer-migration`: continue larger app/admin/storage facade batches after API-217.
1. `consumer-migration`: continue larger outer/owner facade batches after API-218.
## Pre-Push Review Log
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | pass | API-218 adds a root-local storage_api boundary and routes root/server/startup storage facade consumers through it. |
| Migration preservation | pass | Startup storage, metadata migration, notification, readiness, HTTP request context, module switches, event dispatch, snapshots, capacity, workload admission, table catalog, and config tests keep the same storage implementations. |
| Testing/verification | pass | Focused RustFS compile/tests, formatting, migration/layer guards, residual migrated facade scan, Rust risk scan, fast PR gate, and full PR gate are planned before PR. |
| Quality/architecture | pass | API-217 moves admin storage facade ownership behind admin storage_api and removes the admin root compatibility surface. |
| Migration preservation | pass | Config IO/defaults, bucket metadata/targets, replication/resync, rebalance, quota, tier, metrics, object zip, and site replication keep the same storage implementations. |
| Testing/verification | pass | Focused RustFS admin compile/tests, formatting, migration/layer guards, residual migrated facade scan, Rust risk scan, fast PR gate, and full PR gate are planned before PR. |
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::{all_local_disk, disk_drive_path, disk_endpoint};
use crate::storage_api::{all_local_disk, disk_drive_path, disk_endpoint};
use rustfs_io_metrics::capacity_metrics::{
record_capacity_cache_hit, record_capacity_cache_miss, record_capacity_cache_served, record_capacity_refresh_request,
record_capacity_scan_mode,
+3 -3
View File
@@ -16,8 +16,8 @@ use crate::runtime_capabilities::runtime_observability_snapshot;
use crate::server::{
DependencyReadiness, DependencyReadinessReport, ReadinessDegradedReason, snapshot_dependency_readiness_report,
};
use crate::storage::EndpointServerPools;
use crate::storage::ecstore_cluster::{
use crate::storage_api::EndpointServerPools;
use crate::storage_api::ecstore_cluster::{
ClusterControlPlane, ClusterControlPlaneSnapshot, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot,
ClusterPeerHealthSnapshot, ClusterPoolStateSnapshot,
};
@@ -116,7 +116,7 @@ pub fn cluster_has_actionable_pressure(snapshot: &ClusterReadOnlySnapshot) -> bo
#[cfg(test)]
mod tests {
use super::*;
use crate::storage::{Endpoint, Endpoints, PoolEndpoints};
use crate::storage_api::{Endpoint, Endpoints, PoolEndpoints};
use rustfs_concurrency::{WorkloadAdmissionSnapshot, WorkloadClass};
use rustfs_storage_api::{CapabilityState, CapabilityStatus, DiskCapabilities, TopologyCapabilities};
+2 -2
View File
@@ -17,7 +17,7 @@
mod tests {
use crate::config::cli::default_server_opts;
use crate::config::{CommandResult, Config, Opt, TlsCommands};
use crate::storage::DisksLayout;
use crate::storage_api::DisksLayout;
use rustfs_config::{DEFAULT_CONSOLE_ADDRESS, DEFAULT_CONSOLE_ENABLE, DEFAULT_OBS_ENDPOINT, RUSTFS_REGION};
use rustfs_credentials::{DEFAULT_ACCESS_KEY, DEFAULT_SECRET_KEY};
use serial_test::serial;
@@ -275,7 +275,7 @@ mod tests {
#[test]
#[serial]
fn test_volumes_and_disk_layout_parsing() {
use crate::storage::DisksLayout;
use crate::storage_api::DisksLayout;
// Test case 1: Single volume path
let args = vec!["rustfs", "/data/vol1"];
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::{QuotaError, StorageError};
use crate::storage_api::{QuotaError, StorageError};
use rustfs_storage_api::HTTPRangeError;
use s3s::{S3Error, S3ErrorCode};
+6 -6
View File
@@ -14,7 +14,7 @@
use crate::runtime_sources::{resolve_notify_interface, resolve_region};
use crate::server::ShutdownHandle;
use crate::storage::{
use crate::storage_api::{
get_bucket_notification_config, process_lambda_configurations, process_queue_configurations, process_topic_configurations,
};
use crate::{admin, config, startup_runtime_sources, version};
@@ -717,7 +717,7 @@ where
/// When enabled, it spawns a background task that tunes concurrency settings
/// every 60 seconds.
pub async fn init_auto_tuner(ctx: tokio_util::sync::CancellationToken) {
use crate::storage::concurrency::get_concurrency_manager;
use crate::storage_api::concurrency::get_concurrency_manager;
use rustfs_io_metrics::AutoTuner;
use rustfs_io_metrics::TunerConfig;
use tracing::{debug, error, info};
@@ -850,7 +850,7 @@ pub async fn init_ftp_system() -> Result<Option<ShutdownHandle>, Box<dyn std::er
config.validate().await?;
// Create FTP server with protocol storage client
let fs = crate::storage::ecfs::FS::new();
let fs = crate::storage_api::ecfs::FS::new();
let storage_client = ProtocolStorageClient::new(fs);
let server: FtpsServer<ProtocolStorageClient> = FtpsServer::new(config, storage_client).await?;
let bind_addr = server.config().bind_addr;
@@ -970,7 +970,7 @@ pub async fn init_ftps_system() -> Result<Option<ShutdownHandle>, Box<dyn std::e
config.validate().await?;
// Create FTPS server with protocol storage client
let fs = crate::storage::ecfs::FS::new();
let fs = crate::storage_api::ecfs::FS::new();
let storage_client = ProtocolStorageClient::new(fs);
let server: FtpsServer<ProtocolStorageClient> = FtpsServer::new(config, storage_client).await?;
let bind_addr = server.config().bind_addr;
@@ -1088,7 +1088,7 @@ pub async fn init_webdav_system() -> Result<Option<ShutdownHandle>, Box<dyn std:
};
// Create WebDAV server with protocol storage client
let fs = crate::storage::ecfs::FS::new();
let fs = crate::storage_api::ecfs::FS::new();
let storage_client = ProtocolStorageClient::new(fs);
let server: WebDavServer<crate::protocols::ProtocolStorageClient> = WebDavServer::new(config, storage_client).await?;
let bind_addr = server.config().bind_addr;
@@ -1222,7 +1222,7 @@ pub async fn init_sftp_system() -> Result<Option<ShutdownHandle>, Box<dyn std::e
// file has insecure permissions.
let host_keys = SftpConfig::load_host_keys(&config.host_key_dir).await?;
let fs = crate::storage::ecfs::FS::new();
let fs = crate::storage_api::ecfs::FS::new();
let storage_client = ProtocolStorageClient::new(fs);
let server = SftpServer::new(config.clone(), storage_client, host_keys)?;
+1
View File
@@ -95,6 +95,7 @@ pub(crate) mod startup_shutdown;
pub(crate) mod startup_storage;
pub(crate) mod startup_tls_material;
pub mod storage;
pub(crate) mod storage_api;
pub(crate) mod table_catalog;
pub mod tls;
pub mod update;
+3 -3
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::runtime_sources::resolve_action_credentials;
use crate::storage::ecfs::FS;
use crate::storage_api::ecfs::FS;
use http::{HeaderMap, Method};
use percent_encoding::{AsciiSet, CONTROLS, utf8_percent_encode};
use rustfs_credentials;
@@ -151,7 +151,7 @@ impl ProtocolStorageClient {
secret_key: params.secret_key.to_string().into(),
});
extensions.insert(crate::storage::access::ReqInfo {
extensions.insert(crate::storage_api::access::ReqInfo {
cred: Some(rustfs_credentials::Credentials {
access_key: params.access_key.to_string(),
secret_key: params.secret_key.to_string(),
@@ -169,7 +169,7 @@ impl ProtocolStorageClient {
object: params.object,
version_id: None,
region: None,
request_context: Some(crate::storage::request_context::RequestContext::fallback()),
request_context: Some(crate::storage_api::request_context::RequestContext::fallback()),
});
let req = S3Request {
+2 -2
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::{EndpointServerPools, topology_snapshot_from_endpoint_pools_with_capabilities};
use crate::storage_api::{EndpointServerPools, topology_snapshot_from_endpoint_pools_with_capabilities};
use rustfs_storage_api::{
CapabilitySnapshotError, CapabilityStatus, DiskCapabilities, MemorySamplingState, ObservabilitySnapshot,
ObservabilitySnapshotProvider, PlatformSupport, TopologyCapabilities, TopologySnapshot, TopologySnapshotProvider,
@@ -185,7 +185,7 @@ fn cgroup_memory_status() -> CapabilityStatus {
#[cfg(test)]
mod tests {
use super::*;
use crate::storage::{Endpoint, Endpoints, PoolEndpoints};
use crate::storage_api::{Endpoint, Endpoints, PoolEndpoints};
use rustfs_storage_api::{CapabilityState, ObservabilitySnapshotProvider, TopologySnapshotProvider};
#[tokio::test]
+2 -2
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use super::{module_switch::resolve_notify_module_state, refresh_persisted_module_switches_from_store, runtime_sources};
use crate::storage::{EventArgs as EcstoreEventArgs, StorageObjectInfo, register_event_dispatch_hook};
use crate::storage_api::{EventArgs as EcstoreEventArgs, StorageObjectInfo, register_event_dispatch_hook};
use chrono::{DateTime, Utc};
use rustfs_notify::{EventArgs as NotifyEventArgs, NotifyObjectInfo};
use rustfs_s3_types::EventName;
@@ -234,7 +234,7 @@ pub async fn init_event_notifier() {
#[cfg(test)]
mod tests {
use super::{convert_ecstore_object_info, parse_host_and_port};
use crate::storage::StorageObjectInfo;
use crate::storage_api::StorageObjectInfo;
use chrono::{DateTime, Utc};
use rustfs_storage_api::TransitionedObject;
use std::{collections::HashMap, sync::Arc};
+7 -7
View File
@@ -30,11 +30,11 @@ use crate::server::{
TlsAcceptorHolder, TlsHandshakeFailureKind, build_acceptor_from_loaded, load_tls_material, spawn_reload_loop,
},
};
use crate::storage;
use crate::storage::request_context::{RequestContext, extract_request_id_from_headers};
use crate::storage::rpc::InternodeRpcService;
use crate::storage::tonic_service::make_server;
use crate::storage::{TONIC_RPC_PREFIX, verify_rpc_signature};
use crate::storage_api as storage;
use crate::storage_api::request_context::{RequestContext, extract_request_id_from_headers};
use crate::storage_api::rpc::InternodeRpcService;
use crate::storage_api::tonic_service::make_server;
use crate::storage_api::{TONIC_RPC_PREFIX, verify_rpc_signature};
use bytes::Bytes;
use http::{HeaderMap, Method, Request as HttpRequest, Response};
use hyper::body::Incoming;
@@ -1104,7 +1104,7 @@ fn process_connection(
.layer(
TraceLayer::new_for_http()
.make_span_with(|request: &HttpRequest<_>| {
let request_context = request.extensions().get::<crate::storage::request_context::RequestContext>();
let request_context = request.extensions().get::<crate::storage_api::request_context::RequestContext>();
let request_id = request_context
.map(|ctx| ctx.request_id.as_str())
.unwrap_or("unknown");
@@ -1253,7 +1253,7 @@ fn process_connection(
.layer(
TraceLayer::new_for_http()
.make_span_with(|request: &HttpRequest<_>| {
let request_context = request.extensions().get::<crate::storage::request_context::RequestContext>();
let request_context = request.extensions().get::<crate::storage_api::request_context::RequestContext>();
let request_id = request_context
.map(|ctx| ctx.request_id.as_str())
.unwrap_or("unknown");
+2 -2
View File
@@ -25,8 +25,8 @@ use crate::server::{
RUSTFS_ADMIN_PREFIX, active_http_requests, collect_dependency_readiness_report, has_path_prefix, is_admin_path,
is_table_catalog_path,
};
use crate::storage::apply_cors_headers;
use crate::storage::request_context::{
use crate::storage_api::apply_cors_headers;
use crate::storage_api::request_context::{
RequestContext, extract_request_id_from_headers, extract_trace_context_ids_from_headers, spawn_traced,
};
use bytes::Bytes;
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use super::runtime_sources;
use crate::storage::{Error as StorageError, read_config, save_config};
use crate::storage_api::{Error as StorageError, read_config, save_config};
use serde::{Deserialize, Serialize};
use std::sync::atomic::{AtomicBool, Ordering};
+2 -2
View File
@@ -15,9 +15,9 @@
use crate::server::runtime_sources;
use crate::server::{ServiceState, ServiceStateManager};
use crate::server::{has_path_prefix, is_table_catalog_path};
use crate::storage::{Endpoint, EndpointServerPools, is_dist_erasure};
use crate::storage_api::{Endpoint, EndpointServerPools, is_dist_erasure};
#[cfg(test)]
use crate::storage::{Endpoints, PoolEndpoints};
use crate::storage_api::{Endpoints, PoolEndpoints};
use bytes::Bytes;
use http::{Request as HttpRequest, Response, StatusCode};
use http_body::Body;
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::app::context;
use crate::storage::{ECStore, EndpointServerPools};
use crate::storage_api::{ECStore, EndpointServerPools};
use rustfs_kms::KmsServiceManager;
use rustfs_lock::LockClient;
use std::collections::HashMap;
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::ECStore;
use crate::storage_api::ECStore;
use rustfs_heal::{create_ahm_services_cancel_token, heal::storage::ECStoreHealStorage, init_heal_manager};
use rustfs_utils::get_env_bool_with_aliases;
use std::{io::Result, sync::Arc};
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::startup_runtime_sources;
use crate::storage::{ECStore, init_bucket_metadata_sys, try_migrate_bucket_metadata, try_migrate_iam_config};
use crate::storage_api::{ECStore, init_bucket_metadata_sys, try_migrate_bucket_metadata, try_migrate_iam_config};
use rustfs_storage_api::{BucketOperations, BucketOptions};
use std::{
io::{Error, Result},
+1 -1
View File
@@ -19,7 +19,7 @@ const LOG_SUBSYSTEM_STARTUP: &str = "startup";
const EVENT_DEADLOCK_DETECTOR_STATE: &str = "deadlock_detector_state";
pub(crate) fn init_deadlock_detector_runtime() {
let detector = crate::storage::deadlock_detector::get_deadlock_detector();
let detector = crate::storage_api::deadlock_detector::get_deadlock_detector();
if detector.is_enabled() {
detector.start();
info!(
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::EndpointServerPools;
use crate::storage_api::EndpointServerPools;
use rustfs_config::{
DEFAULT_RUSTFS_UNSUPPORTED_FS_POLICY, ENV_RUSTFS_UNSUPPORTED_FS_POLICY, RUSTFS_UNSUPPORTED_FS_POLICY_FAIL,
RUSTFS_UNSUPPORTED_FS_POLICY_WARN,
+1 -1
View File
@@ -14,7 +14,7 @@
use crate::runtime_sources::AppContext;
use crate::server::{ServiceStateManager, publish_ready_when_runtime_ready};
use crate::storage::ECStore;
use crate::storage_api::ECStore;
use rustfs_common::{GlobalReadiness, SystemStage};
use rustfs_iam::init_iam_sys;
use rustfs_kms::KmsServiceManager;
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::ECStore;
use crate::storage_api::ECStore;
use crate::{
server::{ServiceStateManager, ShutdownHandle, wait_for_shutdown},
startup_iam::{IamBootstrapDisposition, publish_ready_for_iam_bootstrap},
+2 -2
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::init::add_bucket_notification_configuration;
use crate::storage::{EndpointServerPools, Result as StorageResult, new_global_notification_sys};
use crate::storage_api::{EndpointServerPools, Result as StorageResult, new_global_notification_sys};
use std::{
future::Future,
io::{Error, Result},
@@ -76,7 +76,7 @@ fn log_embedded_optional_service_skipped(service: &str, err: impl std::fmt::Disp
#[cfg(test)]
mod tests {
use super::init_notification_system_with;
use crate::storage::Error as EcstoreError;
use crate::storage_api::Error as EcstoreError;
#[tokio::test]
async fn notification_system_returns_source_error() {
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::config::RustFSBufferConfig;
use crate::storage::{DynReplicationPool, set_global_region, set_global_rustfs_port};
use crate::storage_api::{DynReplicationPool, set_global_region, set_global_rustfs_port};
use rustfs_kms::KmsServiceManager;
use rustfs_obs::{GlobalError as ObservabilityError, OtelGuard};
use rustfs_tls_runtime::{OutboundTlsMaterial, TlsGeneration};
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::{ECStore, EndpointServerPools};
use crate::storage_api::{ECStore, EndpointServerPools};
use crate::{
config::Config,
init::{init_buffer_profile_system, init_kms_system},
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::shutdown_background_services;
use crate::storage_api::shutdown_background_services;
use crate::{
server::{ServiceState, ServiceStateManager, ShutdownHandle, ShutdownSignal, shutdown_event_notifier, stop_audit_system},
startup_optional_runtime_sidecars::{
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::startup_fs_guard::enforce_unsupported_fs_policy;
use crate::storage::{
use crate::storage_api::{
ECStore, EndpointServerPools, init_background_replication, init_ecstore_config, init_global_config_sys, init_local_disks,
init_lock_clients, prewarm_local_disk_id_map, set_global_endpoints, try_migrate_server_config, update_erasure_type,
};
+1
View File
@@ -244,6 +244,7 @@ pub(crate) type ExpiryState = ecstore_bucket::lifecycle::bucket_lifecycle_ops::E
pub(crate) type FileInfoVersions = ecstore_disk::FileInfoVersions;
pub(crate) type FileReader = ecstore_disk::FileReader;
pub(crate) type FileWriter = ecstore_disk::FileWriter;
pub(crate) type FS = ecfs::FS;
pub(crate) type HashReader = ecstore_rio::HashReader;
pub(crate) type LocalPeerS3Client = ecstore_rpc::LocalPeerS3Client;
pub(crate) type MetricType = ecstore_metrics::MetricType;
+71
View File
@@ -0,0 +1,71 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Root-local boundary for storage owner APIs used by startup, server, and
//! other RustFS outer runtime modules.
pub(crate) use crate::storage::{
BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG,
BUCKET_TABLE_RESERVED_PREFIX, DynReplicationPool, ECStore, Endpoint, EndpointServerPools, Error, EventArgs, QuotaError,
RUSTFS_META_BUCKET, Result, StorageDeletedObject, StorageError, StorageGetObjectReader, StorageObjectInfo,
StorageObjectOptions, StorageObjectToDelete, StoragePutObjReader, TONIC_RPC_PREFIX, all_local_disk, apply_cors_headers,
bucket_metadata_runtime_initialized, disk_drive_path, disk_endpoint, get_bucket_metadata, get_bucket_notification_config,
get_lock_acquire_timeout, init_background_replication, init_bucket_metadata_sys, init_ecstore_config, init_global_config_sys,
init_local_disks, init_lock_clients, is_dist_erasure, new_global_notification_sys, prewarm_local_disk_id_map,
process_lambda_configurations, process_queue_configurations, process_topic_configurations, read_config,
register_event_dispatch_hook, replication_queue_current_count, save_config, set_global_endpoints, set_global_region,
set_global_rustfs_port, shutdown_background_services, table_catalog_path_hash,
topology_snapshot_from_endpoint_pools_with_capabilities, try_migrate_bucket_metadata, try_migrate_iam_config,
try_migrate_server_config, update_erasure_type, verify_rpc_signature,
};
#[cfg(test)]
pub(crate) use crate::storage::{DisksLayout, Endpoints, PoolEndpoints};
pub(crate) mod access {
pub(crate) use crate::storage::access::ReqInfo;
}
pub(crate) mod concurrency {
pub(crate) use crate::storage::concurrency::get_concurrency_manager;
}
pub(crate) mod deadlock_detector {
pub(crate) use crate::storage::deadlock_detector::get_deadlock_detector;
}
pub(crate) mod ecfs {
pub(crate) type FS = crate::storage::FS;
}
pub(crate) mod ecstore_cluster {
pub(crate) use crate::storage::ecstore_cluster::{
ClusterControlPlane, ClusterControlPlaneSnapshot, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot,
ClusterPeerHealthSnapshot, ClusterPoolStateSnapshot,
};
}
pub(crate) mod request_context {
pub(crate) use crate::storage::request_context::{
RequestContext, extract_request_id_from_headers, extract_trace_context_ids_from_headers, spawn_traced,
};
}
pub(crate) mod rpc {
pub(crate) use crate::storage::rpc::InternodeRpcService;
}
pub(crate) mod tonic_service {
pub(crate) use crate::storage::tonic_service::make_server;
}
+2 -2
View File
@@ -27,7 +27,7 @@ use std::{
time::{Duration as StdDuration, Instant},
};
use crate::storage::{
use crate::storage_api::{
BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG,
BUCKET_TABLE_RESERVED_PREFIX, Error as EcstoreError, RUSTFS_META_BUCKET, StorageError, get_bucket_metadata,
get_lock_acquire_timeout, table_catalog_path_hash,
@@ -51,7 +51,7 @@ use time::{Duration, OffsetDateTime};
use tokio::io::AsyncReadExt;
use uuid::Uuid;
use crate::storage::{
use crate::storage_api::{
StorageDeletedObject as DeletedObject, StorageGetObjectReader as GetObjectReader, StorageObjectInfo as ObjectInfo,
StorageObjectOptions as ObjectOptions, StorageObjectToDelete as ObjectToDelete, StoragePutObjReader as PutObjReader,
};
+2 -2
View File
@@ -13,13 +13,13 @@
// limitations under the License.
use crate::runtime_sources::resolve_replication_pool_handle;
use crate::storage::{bucket_metadata_runtime_initialized, replication_queue_current_count};
use crate::storage_api::{bucket_metadata_runtime_initialized, replication_queue_current_count};
use rustfs_concurrency::{
AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadAdmissionSnapshotProvider,
WorkloadClass,
};
use crate::storage::concurrency::get_concurrency_manager;
use crate::storage_api::concurrency::get_concurrency_manager;
const BUCKET_METADATA_RUNTIME_NOT_INITIALIZED: &str = "bucket metadata runtime not initialized";
const FOREGROUND_WRITE_NOT_EXPOSED_BY_PROVIDER: &str = "foreground write admission not yet exposed by RustFS runtime";
@@ -166,6 +166,7 @@ RUSTFS_APP_USECASE_S3_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_usecase_s3_api
RUSTFS_APP_USECASE_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_usecase_storage_api_bypass_hits.txt"
RUSTFS_APP_ADMIN_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_admin_storage_api_bypass_hits.txt"
RUSTFS_ADMIN_STORAGE_API_ROOT_FACADE_HITS_FILE="${TMP_DIR}/rustfs_admin_storage_api_root_facade_hits.txt"
RUSTFS_ROOT_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_root_storage_api_bypass_hits.txt"
RUSTFS_STORAGE_DIRECT_APP_CONTEXT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_storage_direct_app_context_bypass_hits.txt"
awk '
@@ -1519,6 +1520,21 @@ if [[ -s "$RUSTFS_ADMIN_STORAGE_API_ROOT_FACADE_HITS_FILE" ]]; then
report_failure "RustFS admin storage facades must stay behind rustfs/src/admin/storage_api.rs: $(paste -sd '; ' "$RUSTFS_ADMIN_STORAGE_API_ROOT_FACADE_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename 'use crate::storage(?:;|::|\s*\{)|crate::storage::' \
rustfs/src \
--glob '*.rs' \
--glob '!rustfs/src/admin/**' \
--glob '!rustfs/src/app/**' \
--glob '!rustfs/src/storage/**' \
--glob '!rustfs/src/storage_api.rs' || true
) >"$RUSTFS_ROOT_STORAGE_API_BYPASS_HITS_FILE"
if [[ -s "$RUSTFS_ROOT_STORAGE_API_BYPASS_HITS_FILE" ]]; then
report_failure "RustFS root/server/startup storage facades must stay behind rustfs/src/storage_api.rs: $(paste -sd '; ' "$RUSTFS_ROOT_STORAGE_API_BYPASS_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename 'crate::app::context::|use crate::app::context|app::context::' rustfs/src/storage --glob '*.rs' |
-1
View File
@@ -38,7 +38,6 @@ dep|rustfs/src/app/s3_api.rs|app->interface|crate::storage::s3_api::multipart::p
dep|rustfs/src/app/s3_api.rs|app->interface|crate::storage::s3_api::multipart::parse_list_parts_params
dep|rustfs/src/app/s3_api.rs|app->interface|crate::storage::s3_api::multipart::parse_upload_part_number
dep|rustfs/src/init.rs|infra->interface|crate::admin
dep|rustfs/src/protocols/client.rs|infra->interface|crate::storage::ecfs::FS
dep|rustfs/src/runtime_sources.rs|infra->app|crate::app::context
dep|rustfs/src/server/http.rs|infra->interface|crate::admin
dep|rustfs/src/server/layer.rs|infra->interface|crate::admin::console::is_console_path