refactor: route RustFS ECStore consumers via owner boundary (#3745)

This commit is contained in:
Zhengchao An
2026-06-22 19:29:01 +08:00
committed by GitHub
parent a53ccb2407
commit 89805ffb4d
26 changed files with 102 additions and 69 deletions
+41 -12
View File
@@ -5,17 +5,17 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context ## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-rustfs-owner-compat-cleanup` - Branch: `overtrue/arch-rustfs-ecstore-boundary-cleanup`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127`. - Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128`.
- Based on: API-127 slice. - Based on: API-128 slice.
- PR type for this branch: `pure-move` - PR type for this branch: `pure-move`
- Runtime behavior changes: none. - Runtime behavior changes: none.
- Rust code changes: remove RustFS runtime owner storage compatibility bridges - Rust code changes: route RustFS crate startup, server, capacity, config,
from app, admin, and storage, then route consumers directly through owner API table-catalog, workload, and S3 API consumers through the storage owner
definitions. ECStore boundary instead of direct ECStore facade imports.
- CI/script changes: allow migrated RustFS owner roots to import ECStore facade - CI/script changes: narrow RustFS direct ECStore import allowance to the
APIs directly and reject reintroduced app/admin/storage bridge modules. app/admin/storage owner modules.
- Docs changes: record the API-128 RustFS owner bridge cleanup. - Docs changes: record the API-129 RustFS internal ECStore boundary cleanup.
## Phase 0 Tasks ## Phase 0 Tasks
@@ -3829,6 +3829,22 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
formatting, diff hygiene, Rust risk scan, branch freshness check, formatting, diff hygiene, Rust risk scan, branch freshness check,
pre-commit quality gate, and three-expert review. pre-commit quality gate, and three-expert review.
- [x] `API-129` Route RustFS internal ECStore consumers through owner boundary.
- Do: expose crate-local ECStore facade module aliases from
`rustfs/src/storage/mod.rs` and migrate RustFS startup, server, capacity,
config, table-catalog, workload admission, and S3 API helper consumers to
import those aliases from `crate::storage`.
- Acceptance: non-owner RustFS files no longer import `rustfs_ecstore::api`
directly, while `app`, `admin`, and `storage` owner modules remain the only
RustFS crate direct ECStore facade import points.
- Must preserve: startup sequencing, global endpoint/config side effects,
readiness checks, RPC signature verification, notification event dispatch,
capacity refresh behavior, table-catalog constants, workload admission
snapshots, and S3 ETag conversion behavior.
- Verification: focused RustFS compile, direct import residual scan,
migration/layer guards, formatting, diff hygiene, Rust risk scan, branch
freshness check, pre-commit quality gate, and three-expert review.
## Next PRs ## Next PRs
1. `pure-move`: continue pruning remaining facade compatibility and owner boundaries. 1. `pure-move`: continue pruning remaining facade compatibility and owner boundaries.
@@ -3837,14 +3853,27 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
| Expert | Status | Notes | | Expert | Status | Notes |
|---|---|---| |---|---|---|
| Quality/architecture | pass | API-128 removes only RustFS owner bridge modules and keeps call sites on explicit owner APIs. | | Quality/architecture | pass | API-129 keeps ECStore facade exposure in the storage owner boundary and migrates RustFS internal consumers without new wrappers. |
| Migration preservation | pass | The migration guard now rejects deleted app, admin, and storage bridge files, module declarations, and bridge consumers. | | Migration preservation | pass | Startup, readiness, server, capacity, table-catalog, workload, and S3 helper call paths keep the same ECStore symbols through `crate::storage`. |
| Testing/verification | pass | Focused RustFS compile, residual scan, migration guard, layer guard, formatting, diff hygiene, and diff-only Rust risk scan passed. | | Testing/verification | pass | Full pre-commit, focused RustFS compile, residual scan, migration/layer guards, formatting, diff hygiene, and diff-only Rust risk scan passed. |
## Verification Notes ## Verification Notes
Passed before push: Passed before push:
- Issue #660 API-129 current slice:
- `cargo check --tests -p rustfs`: passed.
- `cargo fmt --all`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- `make pre-commit`: passed, including clippy, script tests, nextest
`6509 passed, 111 skipped`, and doc-tests.
- RustFS direct ECStore facade residual scan outside owner modules: passed.
- Rust risk scan: diff-only scan found no new unwrap/expect, panic/todo,
debug prints, relaxed ordering, or integer casts.
- Issue #660 API-128 current slice: - Issue #660 API-128 current slice:
- `cargo check --tests -p rustfs`: passed. - `cargo check --tests -p rustfs`: passed.
- `cargo fmt --all`: passed. - `cargo fmt --all`: passed.
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use rustfs_ecstore::api::{disk as ecstore_disk, storage as ecstore_storage}; use crate::storage::{ecstore_disk, ecstore_storage};
use rustfs_io_metrics::capacity_metrics::{ use rustfs_io_metrics::capacity_metrics::{
record_capacity_cache_hit, record_capacity_cache_miss, record_capacity_cache_served, record_capacity_refresh_request, record_capacity_cache_hit, record_capacity_cache_miss, record_capacity_cache_served, record_capacity_refresh_request,
record_capacity_scan_mode, record_capacity_scan_mode,
+2 -2
View File
@@ -16,9 +16,9 @@
#[allow(unsafe_op_in_unsafe_fn)] #[allow(unsafe_op_in_unsafe_fn)]
mod tests { mod tests {
use crate::config::{CommandResult, Config, Opt, TlsCommands}; use crate::config::{CommandResult, Config, Opt, TlsCommands};
use crate::storage::ecstore_layout::DisksLayout;
use rustfs_config::{DEFAULT_CONSOLE_ADDRESS, DEFAULT_CONSOLE_ENABLE, DEFAULT_OBS_ENDPOINT, RUSTFS_REGION}; use rustfs_config::{DEFAULT_CONSOLE_ADDRESS, DEFAULT_CONSOLE_ENABLE, DEFAULT_OBS_ENDPOINT, RUSTFS_REGION};
use rustfs_credentials::{DEFAULT_ACCESS_KEY, DEFAULT_SECRET_KEY}; use rustfs_credentials::{DEFAULT_ACCESS_KEY, DEFAULT_SECRET_KEY};
use rustfs_ecstore::api::layout::DisksLayout;
use serial_test::serial; use serial_test::serial;
use std::env; use std::env;
@@ -262,7 +262,7 @@ mod tests {
#[test] #[test]
#[serial] #[serial]
fn test_volumes_and_disk_layout_parsing() { fn test_volumes_and_disk_layout_parsing() {
use rustfs_ecstore::api::layout::DisksLayout; use crate::storage::ecstore_layout::DisksLayout;
// Test case 1: Single volume path // Test case 1: Single volume path
let args = vec!["rustfs", "/data/vol1"]; let args = vec!["rustfs", "/data/vol1"];
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use rustfs_ecstore::api::{bucket::quota::QuotaError, error::StorageError}; use crate::storage::{ecstore_bucket::quota::QuotaError, ecstore_error::StorageError};
use rustfs_storage_api::HTTPRangeError; use rustfs_storage_api::HTTPRangeError;
use s3s::{S3Error, S3ErrorCode}; use s3s::{S3Error, S3ErrorCode};
+1 -1
View File
@@ -13,13 +13,13 @@
// limitations under the License. // limitations under the License.
use crate::server::ShutdownHandle; use crate::server::ShutdownHandle;
use crate::storage::{ecstore_bucket::metadata_sys as ecstore_metadata_sys, ecstore_global};
use crate::storage::{process_lambda_configurations, process_queue_configurations, process_topic_configurations}; use crate::storage::{process_lambda_configurations, process_queue_configurations, process_topic_configurations};
use crate::{admin, config, version}; use crate::{admin, config, version};
use rustfs_config::{ use rustfs_config::{
DEFAULT_BUFFER_MAX_SIZE, DEFAULT_BUFFER_MIN_SIZE, DEFAULT_BUFFER_PROFILE, DEFAULT_BUFFER_UNKNOWN_SIZE, DEFAULT_UPDATE_CHECK, DEFAULT_BUFFER_MAX_SIZE, DEFAULT_BUFFER_MIN_SIZE, DEFAULT_BUFFER_PROFILE, DEFAULT_BUFFER_UNKNOWN_SIZE, DEFAULT_UPDATE_CHECK,
ENV_RUSTFS_BUFFER_DEFAULT_SIZE, ENV_RUSTFS_BUFFER_MAX_SIZE, ENV_RUSTFS_BUFFER_MIN_SIZE, ENV_UPDATE_CHECK, RUSTFS_REGION, ENV_RUSTFS_BUFFER_DEFAULT_SIZE, ENV_RUSTFS_BUFFER_MAX_SIZE, ENV_RUSTFS_BUFFER_MIN_SIZE, ENV_UPDATE_CHECK, RUSTFS_REGION,
}; };
use rustfs_ecstore::api::{bucket::metadata_sys as ecstore_metadata_sys, global as ecstore_global};
use rustfs_notify::notifier_global; use rustfs_notify::notifier_global;
use rustfs_targets::arn::{ARN, TargetIDError}; use rustfs_targets::arn::{ARN, TargetIDError};
use rustfs_utils::get_env_usize; use rustfs_utils::get_env_usize;
+4 -4
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use rustfs_ecstore::api::{cluster as ecstore_cluster, layout::EndpointServerPools}; use crate::storage::{ecstore_cluster, ecstore_layout::EndpointServerPools};
use rustfs_storage_api::{ use rustfs_storage_api::{
CapabilitySnapshotError, CapabilityStatus, DiskCapabilities, MemorySamplingState, ObservabilitySnapshot, CapabilitySnapshotError, CapabilityStatus, DiskCapabilities, MemorySamplingState, ObservabilitySnapshot,
ObservabilitySnapshotProvider, PlatformSupport, TopologyCapabilities, TopologySnapshot, TopologySnapshotProvider, ObservabilitySnapshotProvider, PlatformSupport, TopologyCapabilities, TopologySnapshot, TopologySnapshotProvider,
@@ -131,9 +131,9 @@ fn cgroup_memory_status() -> CapabilityStatus {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use rustfs_ecstore::api::{ use crate::storage::{
disk::endpoint::Endpoint, ecstore_disk::endpoint::Endpoint,
layout::{Endpoints, PoolEndpoints}, ecstore_layout::{Endpoints, PoolEndpoints},
}; };
use rustfs_storage_api::{CapabilityState, ObservabilitySnapshotProvider, TopologySnapshotProvider}; use rustfs_storage_api::{CapabilityState, ObservabilitySnapshotProvider, TopologySnapshotProvider};
+1 -1
View File
@@ -15,8 +15,8 @@
use super::{module_switch::resolve_notify_module_state, refresh_persisted_module_switches_from_store}; use super::{module_switch::resolve_notify_module_state, refresh_persisted_module_switches_from_store};
use crate::app::context::resolve_server_config; use crate::app::context::resolve_server_config;
use crate::storage::StorageObjectInfo; use crate::storage::StorageObjectInfo;
use crate::storage::ecstore_event::{EventArgs as EcstoreEventArgs, register_event_dispatch_hook};
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use rustfs_ecstore::api::event::{EventArgs as EcstoreEventArgs, register_event_dispatch_hook};
use rustfs_notify::{EventArgs as NotifyEventArgs, NotifyObjectInfo}; use rustfs_notify::{EventArgs as NotifyEventArgs, NotifyObjectInfo};
use rustfs_s3_types::EventName; use rustfs_s3_types::EventName;
use std::net::SocketAddr; use std::net::SocketAddr;
+1 -1
View File
@@ -31,6 +31,7 @@ use crate::server::{
}, },
}; };
use crate::storage; use crate::storage;
use crate::storage::ecstore_rpc::{TONIC_RPC_PREFIX, verify_rpc_signature};
use crate::storage::rpc::InternodeRpcService; use crate::storage::rpc::InternodeRpcService;
use crate::storage::tonic_service::make_server; use crate::storage::tonic_service::make_server;
use bytes::Bytes; use bytes::Bytes;
@@ -45,7 +46,6 @@ use metrics::{counter, gauge, histogram};
use opentelemetry::global; use opentelemetry::global;
use opentelemetry::trace::TraceContextExt; use opentelemetry::trace::TraceContextExt;
use rustfs_common::GlobalReadiness; use rustfs_common::GlobalReadiness;
use rustfs_ecstore::api::rpc::{TONIC_RPC_PREFIX, verify_rpc_signature};
use rustfs_keystone::KeystoneAuthLayer; use rustfs_keystone::KeystoneAuthLayer;
#[cfg(feature = "swift")] #[cfg(feature = "swift")]
use rustfs_protocols::SwiftService; use rustfs_protocols::SwiftService;
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use rustfs_ecstore::api::{config as ecstore_config, error::Error as StorageError, global::resolve_object_store_handle}; use crate::storage::{ecstore_config, ecstore_error::Error as StorageError, ecstore_global::resolve_object_store_handle};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
+7 -7
View File
@@ -14,6 +14,13 @@
use crate::server::{ServiceState, ServiceStateManager}; use crate::server::{ServiceState, ServiceStateManager};
use crate::server::{has_path_prefix, is_table_catalog_path}; use crate::server::{has_path_prefix, is_table_catalog_path};
#[cfg(test)]
use crate::storage::ecstore_layout::{Endpoints, PoolEndpoints};
use crate::storage::{
ecstore_disk::endpoint::Endpoint,
ecstore_global::{get_global_endpoints_opt, get_global_lock_clients, is_dist_erasure, resolve_object_store_handle},
ecstore_layout::EndpointServerPools,
};
use bytes::Bytes; use bytes::Bytes;
use http::{Request as HttpRequest, Response, StatusCode}; use http::{Request as HttpRequest, Response, StatusCode};
use http_body::Body; use http_body::Body;
@@ -21,13 +28,6 @@ use http_body_util::{BodyExt, Full};
use hyper::body::Incoming; use hyper::body::Incoming;
use metrics::{counter, gauge}; use metrics::{counter, gauge};
use rustfs_common::GlobalReadiness; use rustfs_common::GlobalReadiness;
#[cfg(test)]
use rustfs_ecstore::api::layout::{Endpoints, PoolEndpoints};
use rustfs_ecstore::api::{
disk::endpoint::Endpoint,
global::{get_global_endpoints_opt, get_global_lock_clients, is_dist_erasure, resolve_object_store_handle},
layout::EndpointServerPools,
};
use rustfs_iam::get_global_iam_sys; use rustfs_iam::get_global_iam_sys;
use rustfs_madmin::{Disk, StorageInfo}; use rustfs_madmin::{Disk, StorageInfo};
use rustfs_storage_api::StorageAdminApi; use rustfs_storage_api::StorageAdminApi;
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use rustfs_ecstore::api::storage::ECStore; use crate::storage::ECStore;
use rustfs_heal::{create_ahm_services_cancel_token, heal::storage::ECStoreHealStorage, init_heal_manager}; use rustfs_heal::{create_ahm_services_cancel_token, heal::storage::ECStoreHealStorage, init_heal_manager};
use rustfs_utils::get_env_bool_with_aliases; use rustfs_utils::get_env_bool_with_aliases;
use std::{io::Result, sync::Arc}; use std::{io::Result, sync::Arc};
+3 -3
View File
@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use rustfs_ecstore::api::{ use crate::storage::{
bucket::{metadata_sys as ecstore_metadata_sys, migration as ecstore_migration, replication as ecstore_replication}, ECStore,
storage::ECStore, ecstore_bucket::{metadata_sys as ecstore_metadata_sys, migration as ecstore_migration, replication as ecstore_replication},
}; };
use rustfs_storage_api::{BucketOperations, BucketOptions}; use rustfs_storage_api::{BucketOperations, BucketOptions};
use std::{ use std::{
+1 -1
View File
@@ -12,11 +12,11 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use crate::storage::ecstore_layout::EndpointServerPools;
use rustfs_config::{ use rustfs_config::{
DEFAULT_RUSTFS_UNSUPPORTED_FS_POLICY, ENV_RUSTFS_UNSUPPORTED_FS_POLICY, RUSTFS_UNSUPPORTED_FS_POLICY_FAIL, DEFAULT_RUSTFS_UNSUPPORTED_FS_POLICY, ENV_RUSTFS_UNSUPPORTED_FS_POLICY, RUSTFS_UNSUPPORTED_FS_POLICY_FAIL,
RUSTFS_UNSUPPORTED_FS_POLICY_WARN, RUSTFS_UNSUPPORTED_FS_POLICY_WARN,
}; };
use rustfs_ecstore::api::layout::EndpointServerPools;
use std::collections::BTreeSet; use std::collections::BTreeSet;
use std::io::{Error, Result}; use std::io::{Error, Result};
use tracing::warn; use tracing::warn;
+1 -1
View File
@@ -14,8 +14,8 @@
use crate::app::context::AppContext; use crate::app::context::AppContext;
use crate::server::{ServiceStateManager, publish_ready_when_runtime_ready}; use crate::server::{ServiceStateManager, publish_ready_when_runtime_ready};
use crate::storage::ECStore;
use rustfs_common::{GlobalReadiness, SystemStage}; use rustfs_common::{GlobalReadiness, SystemStage};
use rustfs_ecstore::api::storage::ECStore;
use rustfs_iam::init_iam_sys; use rustfs_iam::init_iam_sys;
use rustfs_kms::KmsServiceManager; use rustfs_kms::KmsServiceManager;
use std::future::Future; use std::future::Future;
+1 -1
View File
@@ -12,6 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use crate::storage::ECStore;
use crate::{ use crate::{
server::{ServiceStateManager, ShutdownHandle, wait_for_shutdown}, server::{ServiceStateManager, ShutdownHandle, wait_for_shutdown},
startup_iam::{IamBootstrapDisposition, publish_ready_for_iam_bootstrap}, startup_iam::{IamBootstrapDisposition, publish_ready_for_iam_bootstrap},
@@ -19,7 +20,6 @@ use crate::{
startup_shutdown::run_startup_shutdown_sequence, startup_shutdown::run_startup_shutdown_sequence,
}; };
use rustfs_common::GlobalReadiness; use rustfs_common::GlobalReadiness;
use rustfs_ecstore::api::storage::ECStore;
use rustfs_scanner::init_data_scanner; use rustfs_scanner::init_data_scanner;
use std::{ use std::{
io::Result, io::Result,
+2 -2
View File
@@ -13,7 +13,7 @@
// limitations under the License. // limitations under the License.
use crate::init::add_bucket_notification_configuration; use crate::init::add_bucket_notification_configuration;
use rustfs_ecstore::api::{error as ecstore_error, layout::EndpointServerPools, notification as ecstore_notification}; use crate::storage::{ecstore_error, ecstore_layout::EndpointServerPools, ecstore_notification};
use std::{ use std::{
future::Future, future::Future,
io::{Error, Result}, io::{Error, Result},
@@ -76,7 +76,7 @@ fn log_embedded_optional_service_skipped(service: &str, err: impl std::fmt::Disp
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::init_notification_system_with; use super::init_notification_system_with;
use rustfs_ecstore::api::error::Error as EcstoreError; use crate::storage::ecstore_error::Error as EcstoreError;
#[tokio::test] #[tokio::test]
async fn notification_system_returns_source_error() { async fn notification_system_returns_source_error() {
+1 -1
View File
@@ -12,6 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use crate::storage::ecstore_global::{set_global_region, set_global_rustfs_port};
use crate::{ use crate::{
capacity::capacity_integration::init_capacity_management, capacity::capacity_integration::init_capacity_management,
config::Config, config::Config,
@@ -19,7 +20,6 @@ use crate::{
}; };
use rustfs_common::{GlobalReadiness, set_global_addr}; use rustfs_common::{GlobalReadiness, set_global_addr};
use rustfs_credentials::init_global_action_credentials; use rustfs_credentials::init_global_action_credentials;
use rustfs_ecstore::api::global::{set_global_region, set_global_rustfs_port};
use rustfs_utils::net::parse_and_resolve_address; use rustfs_utils::net::parse_and_resolve_address;
use std::{ use std::{
io::{Error, Result}, io::{Error, Result},
+1 -1
View File
@@ -12,6 +12,7 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use crate::storage::{ECStore, ecstore_layout::EndpointServerPools};
use crate::{ use crate::{
config::Config, config::Config,
init::{init_buffer_profile_system, init_kms_system}, init::{init_buffer_profile_system, init_kms_system},
@@ -28,7 +29,6 @@ use crate::{
startup_optional_runtime_sidecars::{OptionalRuntimeServices, init_optional_runtime_services}, startup_optional_runtime_sidecars::{OptionalRuntimeServices, init_optional_runtime_services},
}; };
use rustfs_common::GlobalReadiness; use rustfs_common::GlobalReadiness;
use rustfs_ecstore::api::{layout::EndpointServerPools, storage::ECStore};
use std::{io::Result, sync::Arc}; use std::{io::Result, sync::Arc};
use tokio_util::sync::CancellationToken; use tokio_util::sync::CancellationToken;
+1 -1
View File
@@ -12,13 +12,13 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use crate::storage::ecstore_global::shutdown_background_services;
use crate::{ use crate::{
server::{ServiceState, ServiceStateManager, ShutdownHandle, ShutdownSignal, shutdown_event_notifier, stop_audit_system}, server::{ServiceState, ServiceStateManager, ShutdownHandle, ShutdownSignal, shutdown_event_notifier, stop_audit_system},
startup_optional_runtime_sidecars::{ startup_optional_runtime_sidecars::{
OptionalRuntimeServices, prepare_optional_runtime_shutdowns, shutdown_optional_runtime_services, OptionalRuntimeServices, prepare_optional_runtime_shutdowns, shutdown_optional_runtime_services,
}, },
}; };
use rustfs_ecstore::api::global::shutdown_background_services;
use rustfs_heal::shutdown_ahm_services; use rustfs_heal::shutdown_ahm_services;
use rustfs_utils::get_env_bool_with_aliases; use rustfs_utils::get_env_bool_with_aliases;
use std::path::Path; use std::path::Path;
+5 -5
View File
@@ -13,12 +13,12 @@
// limitations under the License. // limitations under the License.
use crate::startup_fs_guard::enforce_unsupported_fs_policy; use crate::startup_fs_guard::enforce_unsupported_fs_policy;
use rustfs_common::{GlobalReadiness, SystemStage}; use crate::storage::ECStore;
use rustfs_ecstore::api::storage::ECStore; use crate::storage::{
use rustfs_ecstore::api::{ ecstore_bucket::replication as ecstore_replication, ecstore_config, ecstore_global, ecstore_layout::EndpointServerPools,
bucket::replication as ecstore_replication, config as ecstore_config, global as ecstore_global, layout::EndpointServerPools, ecstore_storage,
storage as ecstore_storage,
}; };
use rustfs_common::{GlobalReadiness, SystemStage};
use std::{ use std::{
io::{Error, Result}, io::{Error, Result},
net::SocketAddr, net::SocketAddr,
+16 -12
View File
@@ -70,18 +70,22 @@ pub(crate) use sse::{
use std::sync::Arc; use std::sync::Arc;
use rustfs_ecstore::api::admin as ecstore_admin; pub(crate) use rustfs_ecstore::api::admin as ecstore_admin;
use rustfs_ecstore::api::bucket as ecstore_bucket; pub(crate) use rustfs_ecstore::api::bucket as ecstore_bucket;
#[cfg(test)] pub(crate) use rustfs_ecstore::api::client as ecstore_client;
use rustfs_ecstore::api::config as ecstore_config; pub(crate) use rustfs_ecstore::api::cluster as ecstore_cluster;
use rustfs_ecstore::api::disk as ecstore_disk; pub(crate) use rustfs_ecstore::api::config as ecstore_config;
use rustfs_ecstore::api::error as ecstore_error; pub(crate) use rustfs_ecstore::api::disk as ecstore_disk;
use rustfs_ecstore::api::global as ecstore_global; pub(crate) use rustfs_ecstore::api::error as ecstore_error;
use rustfs_ecstore::api::metrics as ecstore_metrics; pub(crate) use rustfs_ecstore::api::event as ecstore_event;
use rustfs_ecstore::api::rio as ecstore_rio; pub(crate) use rustfs_ecstore::api::global as ecstore_global;
use rustfs_ecstore::api::rpc as ecstore_rpc; pub(crate) use rustfs_ecstore::api::layout as ecstore_layout;
use rustfs_ecstore::api::set_disk as ecstore_set_disk; pub(crate) use rustfs_ecstore::api::metrics as ecstore_metrics;
use rustfs_ecstore::api::storage as ecstore_storage; pub(crate) use rustfs_ecstore::api::notification as ecstore_notification;
pub(crate) use rustfs_ecstore::api::rio as ecstore_rio;
pub(crate) use rustfs_ecstore::api::rpc as ecstore_rpc;
pub(crate) use rustfs_ecstore::api::set_disk as ecstore_set_disk;
pub(crate) use rustfs_ecstore::api::storage as ecstore_storage;
pub(crate) const BUCKET_ACCELERATE_CONFIG: &str = ecstore_bucket::metadata::BUCKET_ACCELERATE_CONFIG; pub(crate) const BUCKET_ACCELERATE_CONFIG: &str = ecstore_bucket::metadata::BUCKET_ACCELERATE_CONFIG;
pub(crate) const BUCKET_LOGGING_CONFIG: &str = ecstore_bucket::metadata::BUCKET_LOGGING_CONFIG; pub(crate) const BUCKET_LOGGING_CONFIG: &str = ecstore_bucket::metadata::BUCKET_LOGGING_CONFIG;
+1 -1
View File
@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use crate::storage::ecstore_client::object_api_utils::to_s3s_etag;
use crate::storage::s3_api::common::rustfs_owner; use crate::storage::s3_api::common::rustfs_owner;
use percent_encoding::percent_decode_str; use percent_encoding::percent_decode_str;
use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag;
use rustfs_storage_api::{ use rustfs_storage_api::{
BucketInfo, ListObjectVersionsInfo as StorageListObjectVersionsInfo, ListObjectsV2Info as StorageListObjectsV2Info, BucketInfo, ListObjectVersionsInfo as StorageListObjectVersionsInfo, ListObjectsV2Info as StorageListObjectsV2Info,
}; };
+2 -2
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use crate::storage::ecstore_client::object_api_utils::to_s3s_etag;
use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner}; use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner};
use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag;
use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo}; use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo};
use s3s::dto::{CommonPrefix, ListMultipartUploadsOutput, ListPartsOutput, MultipartUpload, Part, Timestamp}; use s3s::dto::{CommonPrefix, ListMultipartUploadsOutput, ListPartsOutput, MultipartUpload, Part, Timestamp};
use s3s::{S3Error, S3ErrorCode}; use s3s::{S3Error, S3ErrorCode};
@@ -195,8 +195,8 @@ mod tests {
MAX_MULTIPART_UPLOADS_LIST, build_list_multipart_uploads_output, build_list_parts_output, MAX_MULTIPART_UPLOADS_LIST, build_list_multipart_uploads_output, build_list_parts_output,
parse_list_multipart_uploads_params, parse_list_parts_params, parse_upload_part_number, parse_list_multipart_uploads_params, parse_list_parts_params, parse_upload_part_number,
}; };
use crate::storage::ecstore_client::object_api_utils::to_s3s_etag;
use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner}; use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner};
use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag;
use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo, MultipartInfo, PartInfo}; use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo, MultipartInfo, PartInfo};
use s3s::S3ErrorCode; use s3s::S3ErrorCode;
use s3s::dto::Timestamp; use s3s::dto::Timestamp;
+4 -4
View File
@@ -27,6 +27,10 @@ use std::{
time::{Duration as StdDuration, Instant}, time::{Duration as StdDuration, Instant},
}; };
use crate::storage::{
ecstore_bucket::{metadata as ecstore_metadata, metadata_sys as ecstore_metadata_sys},
ecstore_disk, ecstore_error, ecstore_set_disk,
};
use bytes::Bytes; use bytes::Bytes;
use datafusion::{ use datafusion::{
arrow::datatypes::SchemaRef, arrow::datatypes::SchemaRef,
@@ -34,10 +38,6 @@ use datafusion::{
}; };
use http::HeaderMap; use http::HeaderMap;
use metrics::{counter, histogram}; use metrics::{counter, histogram};
use rustfs_ecstore::api::{
bucket::{metadata as ecstore_metadata, metadata_sys as ecstore_metadata_sys},
disk as ecstore_disk, error as ecstore_error, set_disk as ecstore_set_disk,
};
use rustfs_filemeta::FileInfo; use rustfs_filemeta::FileInfo;
use rustfs_storage_api::{ use rustfs_storage_api::{
HTTPPreconditions, HTTPRangeSpec, ListObjectVersionsInfo as StorageListObjectVersionsInfo, HTTPPreconditions, HTTPRangeSpec, ListObjectVersionsInfo as StorageListObjectVersionsInfo,
+1 -1
View File
@@ -12,11 +12,11 @@
// See the License for the specific language governing permissions and // See the License for the specific language governing permissions and
// limitations under the License. // limitations under the License.
use crate::storage::ecstore_bucket::{metadata_sys, replication};
use rustfs_concurrency::{ use rustfs_concurrency::{
AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadAdmissionSnapshotProvider, AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadAdmissionSnapshotProvider,
WorkloadClass, WorkloadClass,
}; };
use rustfs_ecstore::api::bucket::{metadata_sys, replication};
use crate::storage::concurrency::get_concurrency_manager; use crate::storage::concurrency::get_concurrency_manager;
@@ -715,7 +715,7 @@ fi
--glob '!**/storage_compat.rs' \ --glob '!**/storage_compat.rs' \
--glob '!**/*storage_compat.rs' \ --glob '!**/*storage_compat.rs' \
--glob '!target/**' \ --glob '!target/**' \
| rg -v '^(rustfs/src/(admin/mod|app/mod|storage/mod|capacity/service|config/config_test|error|init|runtime_capabilities|startup_(background|bucket_metadata|fs_guard|iam|lifecycle|notification|server|services|shutdown|storage)|table_catalog|workload_admission)\.rs|rustfs/src/server/(event|http|module_switch|readiness)\.rs|rustfs/src/storage/s3_api/(bucket|multipart)\.rs|crates/e2e_test/src/(replication_extension_test|reliant/(grpc_lock_client|node_interact_test))\.rs|crates/heal/src/heal/mod\.rs|crates/heal/tests/(endpoint_index_test|heal_bug_fixes_test|heal_integration_test)\.rs|crates/iam/src/lib\.rs|crates/notify/src/config_manager\.rs|crates/obs/src/metrics/(scheduler|stats_collector)\.rs|crates/protocols/src/swift/mod\.rs|crates/s3select-api/src/object_store\.rs|crates/scanner/src/lib\.rs|crates/scanner/tests/lifecycle_integration_test\.rs|fuzz/fuzz_targets/(bucket_validation|path_containment)\.rs):' || true | rg -v '^(rustfs/src/(admin/mod|app/mod|storage/mod)\.rs|crates/e2e_test/src/(replication_extension_test|reliant/(grpc_lock_client|node_interact_test))\.rs|crates/heal/src/heal/mod\.rs|crates/heal/tests/(endpoint_index_test|heal_bug_fixes_test|heal_integration_test)\.rs|crates/iam/src/lib\.rs|crates/notify/src/config_manager\.rs|crates/obs/src/metrics/(scheduler|stats_collector)\.rs|crates/protocols/src/swift/mod\.rs|crates/s3select-api/src/object_store\.rs|crates/scanner/src/lib\.rs|crates/scanner/tests/lifecycle_integration_test\.rs|fuzz/fuzz_targets/(bucket_validation|path_containment)\.rs):' || true
) | ) |
cat >"$DIRECT_ECSTORE_IMPORT_HITS_FILE" cat >"$DIRECT_ECSTORE_IMPORT_HITS_FILE"