diff --git a/docs/architecture/migration-progress.md b/docs/architecture/migration-progress.md index 11a007a9e..698cb5b2e 100644 --- a/docs/architecture/migration-progress.md +++ b/docs/architecture/migration-progress.md @@ -5,17 +5,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ## Current Context - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) -- Branch: `overtrue/arch-fuzz-compat-relative-consumers` -- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115`. -- Based on: API-115 slice. +- Branch: `overtrue/arch-runtime-local-compat-bridges` +- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116`. +- Based on: API-116 slice. - PR type for this branch: `pure-move` - Runtime behavior changes: none. -- Rust code changes: collapse fuzz-target compatibility consumer paths into - relative owner paths. -- CI/script changes: extend the standalone crate local consumer guard to fuzz - targets. -- Docs changes: record the API-116 fuzz-target local compatibility consumer - cleanup. +- Rust code changes: remove app/admin, storage core, nested, handler, capacity, + server, and S3 API secondary compatibility bridges. +- CI/script changes: guard against reintroducing the removed secondary bridge + modules or their consumer paths. +- Docs changes: record the API-117 through API-121 compatibility bridge cleanup. ## Phase 0 Tasks @@ -647,6 +646,69 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block - Verification: fuzz package compile coverage, fuzz-target local compatibility consumer residual scan, migration and layer guards, formatting, diff hygiene, Rust risk scan, and three-expert review. +- [x] `API-117` Remove app/admin secondary compatibility bridges. + - Completed slice: replace app use-case and admin router consumers of + `usecase_storage_compat` and `router_storage_compat` with direct owner + `storage_compat` paths, then delete the secondary bridge modules. + - Acceptance: app use-cases, app tests, and the admin router no longer route + through a second local compatibility bridge; migration rules reject + reintroduced bridge names. + - Must preserve: app object/bucket/multipart use-case behavior, lifecycle + transition test setup, admin route replication/bucket-target contracts, and + existing owner `storage_compat` aliases. + - Verification: RustFS compile coverage, app/admin secondary bridge residual + scan, migration and layer guards, formatting, diff hygiene, Rust risk scan, + and three-expert review. +- [x] `API-118` Remove storage core secondary compatibility bridge. + - Completed slice: replace storage owner consumers of `core_storage_compat` + with direct `storage_compat` paths, then delete the secondary bridge module. + - Acceptance: storage owner modules and tests no longer route through a + second local compatibility bridge; migration rules reject reintroduced + `core_storage_compat` references. + - Must preserve: ECFS object operations, storage access checks, SSE + encryption helpers, storage option resolution, and existing owner + `storage_compat` aliases. + - Verification: RustFS compile coverage, storage secondary bridge residual + scan, migration and layer guards, formatting, diff hygiene, path-only risk + review, and three-expert review. +- [x] `API-119` Remove nested secondary compatibility bridges. + - Completed slice: replace admin service, app context, and storage RPC + consumers of nested `storage_compat` bridge modules with direct owner + `storage_compat` paths, then delete the nested bridge modules. + - Acceptance: nested service, context, and RPC modules no longer route + through a second local compatibility bridge; migration rules reject + reintroduced bridge files or module declarations. + - Must preserve: admin dynamic config and site-replication behavior, app + context handle wiring, storage RPC signature and disk lookup behavior, and + existing owner `storage_compat` aliases. + - Verification: RustFS compile coverage, nested secondary bridge residual + scan, migration and layer guards, formatting, diff hygiene, path-only risk + review, and three-expert review. +- [x] `API-120` Remove admin handlers secondary compatibility bridge. + - Completed slice: replace admin handler consumers of + `handlers::storage_compat` with direct admin owner `storage_compat` paths, + then delete the handler bridge module. + - Acceptance: admin handler modules no longer route through a second local + compatibility bridge; migration rules reject the bridge file, module + declaration, or direct handler-level `super::storage_compat` consumers. + - Must preserve: admin handler config, replication, rebalance, quota, tier, + table catalog, metrics, trace, and heal behavior plus existing admin owner + `storage_compat` aliases. + - Verification: RustFS admin handler compile coverage, handler secondary + bridge residual scan, migration and layer guards, formatting, diff hygiene, + path-only risk review, and three-expert review. +- [x] `API-121` Remove runtime local compatibility bridges. + - Completed slice: replace capacity, server, and S3 API local compatibility + bridge consumers with direct owner APIs, then delete the bridge modules. + - Acceptance: capacity, server, and S3 API modules no longer route through + local `storage_compat` bridges; migration rules reject bridge files, module + declarations, or bridge consumers. + - Must preserve: capacity disk discovery, HTTP RPC signature verification, + event dispatch hook wiring, module-switch persistence, readiness quorum + checks, and S3 ETag conversion behavior. + - Verification: RustFS compile coverage, runtime local bridge residual scan, + migration and layer guards, formatting, diff hygiene, path-only risk + review, and three-expert review. - [x] `G-012` Inventory placement and repair invariants. - Acceptance: [`placement-repair-invariants.md`](placement-repair-invariants.md) records @@ -3680,14 +3742,70 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block | Expert | Status | Notes | |---|---|---| -| Quality/architecture | pass | API-116 keeps selected fuzz-target local compatibility consumers owner-relative by replacing crate-qualified compatibility paths with scoped relative paths. | -| Migration preservation | pass | The extended guard rejects crate-qualified fuzz-target local compatibility consumer paths while preserving the same facade names and aliases. | -| Testing/verification | pass | Fuzz package compile, fuzz-target local compatibility consumer residual scan, migration guard, layer guard, formatting, diff hygiene, and risk scan passed. | +| Quality/architecture | pass | API-121 removes capacity, server, and S3 API local bridge modules while keeping direct owner API calls explicit. | +| Migration preservation | pass | The extended guard rejects deleted runtime bridge files, module declarations, and local bridge consumers. | +| Testing/verification | pass | RustFS compile, runtime local bridge residual scan, migration guard, layer guard, formatting, diff hygiene, pre-commit, and path-only risk review passed. | ## Verification Notes Passed before push: +- Issue #660 API-121 current slice: + - `cargo check -p rustfs --tests`: passed. + - `cargo fmt --all`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - Runtime local compatibility bridge residual scan: passed. + - Rust risk review on path-only replacements and guard script: passed. + - `make pre-commit`: passed. + +- Issue #660 API-120 current slice: + - `cargo check -p rustfs --tests`: passed. + - `cargo fmt --all`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - Admin handlers secondary compatibility bridge residual scan: passed. + - Rust risk review on path-only replacements and guard script: passed. + +- Issue #660 API-119 current slice: + - `cargo check -p rustfs --tests`: passed. + - `cargo fmt --all`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - Nested secondary compatibility bridge residual scan: passed. + - Rust risk review on path-only replacements and guard script: passed. + +- Issue #660 API-118 current slice: + - `cargo check -p rustfs --tests`: passed. + - `cargo fmt --all`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - Storage secondary compatibility bridge residual scan: passed. + - Rust risk review on path-only replacements and guard script: passed. + +- Issue #660 API-117 current slice: + - `cargo check -p rustfs --tests`: passed. + - `cargo fmt --all`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - App/admin secondary compatibility bridge residual scan: passed. + - Rust risk review on path-only replacements and guard script: passed. + - Issue #660 API-116 current slice: - `cargo check --manifest-path fuzz/Cargo.toml --bins`: passed. - `cargo fmt --all`: passed. diff --git a/rustfs/src/admin/handlers/account_info.rs b/rustfs/src/admin/handlers/account_info.rs index 703239bdc..24c941045 100644 --- a/rustfs/src/admin/handlers/account_info.rs +++ b/rustfs/src/admin/handlers/account_info.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::versioning_sys::BucketVersioningSys; +use super::super::storage_compat::versioning_sys::BucketVersioningSys; use crate::admin::auth::authenticate_request; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::app::context::resolve_object_store_handle; diff --git a/rustfs/src/admin/handlers/audit_runtime_config.rs b/rustfs/src/admin/handlers/audit_runtime_config.rs index 31f9b3659..6fffc2ffa 100644 --- a/rustfs/src/admin/handlers/audit_runtime_config.rs +++ b/rustfs/src/admin/handlers/audit_runtime_config.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::{read_admin_config_without_migrate, save_admin_server_config}; +use super::super::storage_compat::{read_admin_config_without_migrate, save_admin_server_config}; use crate::admin::handlers::target_descriptor::AdminTargetSpec; use crate::app::context::resolve_object_store_handle; use rustfs_audit::{audit_system, start_audit_system as start_global_audit_system, system::AuditSystemState}; diff --git a/rustfs/src/admin/handlers/bucket_meta.rs b/rustfs/src/admin/handlers/bucket_meta.rs index 9330633c9..66304be6a 100644 --- a/rustfs/src/admin/handlers/bucket_meta.rs +++ b/rustfs/src/admin/handlers/bucket_meta.rs @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::utils::{deserialize, serialize}; -use super::storage_compat::{ +use super::super::storage_compat::utils::{deserialize, serialize}; +use super::super::storage_compat::{ StorageError, metadata::{ BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, diff --git a/rustfs/src/admin/handlers/config_admin.rs b/rustfs/src/admin/handlers/config_admin.rs index bdc056178..6fdabd969 100644 --- a/rustfs/src/admin/handlers/config_admin.rs +++ b/rustfs/src/admin/handlers/config_admin.rs @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV}; -use super::storage_compat::{ +use super::super::storage_compat::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV}; +use super::super::storage_compat::{ RUSTFS_META_BUCKET, STORAGE_CLASS_SUB_SYS, delete_admin_config, read_admin_config, read_admin_config_without_migrate, save_admin_config, save_admin_server_config, }; @@ -710,7 +710,7 @@ fn success_response(config_applied: bool) -> S3Result S3Result> { +fn object_store() -> S3Result> { resolve_object_store_handle().ok_or_else(|| s3_error!(InternalError, "server storage not initialized")) } @@ -755,7 +755,7 @@ fn config_update_sub_system(directives: &[ConfigDirective]) -> S3Result S3Result<()> { if DEFAULT_KVS.get().is_none() { - super::storage_compat::init_admin_config_defaults(); + super::super::storage_compat::init_admin_config_defaults(); } let Some(defaults) = DEFAULT_KVS.get() else { return Err(s3_error!(InternalError, "config defaults are not initialized")); @@ -1409,7 +1409,7 @@ fn env_help_key(sub_system: &str, key: &str) -> String { fn default_help_postfix(sub_system: &str, key: &str) -> String { if DEFAULT_KVS.get().is_none() { - super::storage_compat::init_admin_config_defaults(); + super::super::storage_compat::init_admin_config_defaults(); } DEFAULT_KVS @@ -1896,7 +1896,7 @@ mod tests { #[test] fn full_config_export_can_be_reapplied() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); let mut original = ServerConfig::new(); apply_set_directives( &mut original, @@ -1943,7 +1943,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn build_help_response_appends_default_value_postfix() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); let response = build_help_response(Some("identity_openid"), Some("scopes"), false).expect("help response"); assert_eq!(response.keys_help.len(), 2); @@ -2056,7 +2056,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn render_selected_config_includes_env_override_lines() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); temp_env::with_vars( [ ("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example")), @@ -2092,7 +2092,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn render_selected_config_lists_env_only_targets() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || { let config = ServerConfig::new(); let rendered = String::from_utf8( @@ -2115,7 +2115,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn render_selected_config_supports_specific_env_only_target_queries() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || { let config = ServerConfig::new(); let rendered = String::from_utf8( @@ -2138,7 +2138,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn render_selected_config_orders_default_before_named_targets() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_ALPHA", Some("http://alpha.example"))], || { let mut config = ServerConfig::new(); apply_set_directives( @@ -2319,7 +2319,7 @@ identity_openid client_id="existing-client""#, #[test] fn storage_class_get_target_none_matches_full_export() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); let mut config = ServerConfig::new(); apply_set_directives( &mut config, diff --git a/rustfs/src/admin/handlers/heal.rs b/rustfs/src/admin/handlers/heal.rs index b7cb8574f..500940579 100644 --- a/rustfs/src/admin/handlers/heal.rs +++ b/rustfs/src/admin/handlers/heal.rs @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::is_reserved_or_invalid_bucket; -use super::storage_compat::utils::is_valid_object_prefix; +use super::super::storage_compat::is_reserved_or_invalid_bucket; +use super::super::storage_compat::utils::is_valid_object_prefix; use crate::admin::auth::{authenticate_request, validate_admin_request}; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::app::context::resolve_object_store_handle; @@ -378,10 +378,10 @@ fn should_handle_root_heal_directly(hip: &HealInitParams) -> bool { && hip.hs.set.is_none() } -fn map_root_heal_status(heal_err: Option) -> S3Result<()> { +fn map_root_heal_status(heal_err: Option) -> S3Result<()> { match heal_err { None => Ok(()), - Some(super::storage_compat::StorageError::NoHealRequired) => { + Some(super::super::storage_compat::StorageError::NoHealRequired) => { info!( event = EVENT_ADMIN_RESPONSE_EMITTED, component = LOG_COMPONENT_ADMIN_API, @@ -730,7 +730,7 @@ impl Operation for BackgroundHealStatusHandler { #[cfg(test)] mod tests { - use super::super::storage_compat::StorageError; + use super::super::super::storage_compat::StorageError; use super::extract_heal_init_params; use super::{ HealInitParams, HealResp, build_heal_channel_request, encode_background_heal_status, encode_heal_start_success, diff --git a/rustfs/src/admin/handlers/kms_dynamic.rs b/rustfs/src/admin/handlers/kms_dynamic.rs index f991a1ac6..07ce35af3 100644 --- a/rustfs/src/admin/handlers/kms_dynamic.rs +++ b/rustfs/src/admin/handlers/kms_dynamic.rs @@ -14,7 +14,7 @@ //! KMS dynamic configuration admin API handlers -use super::storage_compat::{read_admin_config, save_admin_config}; +use super::super::storage_compat::{read_admin_config, save_admin_config}; use crate::admin::auth::validate_admin_request; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::app::context::{resolve_kms_runtime_service_manager, resolve_object_store_handle}; diff --git a/rustfs/src/admin/handlers/metrics.rs b/rustfs/src/admin/handlers/metrics.rs index b616795e1..ac57702a4 100644 --- a/rustfs/src/admin/handlers/metrics.rs +++ b/rustfs/src/admin/handlers/metrics.rs @@ -18,7 +18,7 @@ //! keeping the response format explicitly NDJSON. It is not a Prometheus text //! exposition endpoint. -use super::storage_compat::{CollectMetricsOpts, MetricType, collect_local_metrics}; +use super::super::storage_compat::{CollectMetricsOpts, MetricType, collect_local_metrics}; use crate::admin::auth::validate_admin_request; use crate::admin::router::Operation; use crate::auth::{check_key_valid, get_session_token}; diff --git a/rustfs/src/admin/handlers/mod.rs b/rustfs/src/admin/handlers/mod.rs index 235041dcc..7e0c35caa 100644 --- a/rustfs/src/admin/handlers/mod.rs +++ b/rustfs/src/admin/handlers/mod.rs @@ -45,7 +45,6 @@ pub mod replication; pub mod scanner; pub mod service_account; pub mod site_replication; -pub(crate) mod storage_compat; pub mod sts; pub mod system; pub mod table_catalog; diff --git a/rustfs/src/admin/handlers/object_zip_download.rs b/rustfs/src/admin/handlers/object_zip_download.rs index 2ea33af44..a1b9d528c 100644 --- a/rustfs/src/admin/handlers/object_zip_download.rs +++ b/rustfs/src/admin/handlers/object_zip_download.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::get_global_region; +use super::super::storage_compat::get_global_region; use crate::admin::router::{ADMIN_OBJECT_ZIP_DOWNLOADS_PATH, AdminOperation, Operation, S3Router}; use crate::app::context::resolve_object_store_handle; use crate::auth::{check_key_valid, get_session_token}; @@ -649,7 +649,7 @@ async fn preflight_zip_items(request: &CreateObjectZipDownloadRequest, items: &[ Ok(()) } -fn storage_error_to_s3(err: super::storage_compat::Error) -> s3s::S3Error { +fn storage_error_to_s3(err: super::super::storage_compat::Error) -> s3s::S3Error { ApiError::from(err).into() } diff --git a/rustfs/src/admin/handlers/oidc.rs b/rustfs/src/admin/handlers/oidc.rs index 9d028f190..50e582858 100644 --- a/rustfs/src/admin/handlers/oidc.rs +++ b/rustfs/src/admin/handlers/oidc.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::{read_admin_config_without_migrate, save_admin_server_config}; +use super::super::storage_compat::{read_admin_config_without_migrate, save_admin_server_config}; use super::sts::create_oidc_sts_credentials; use crate::admin::auth::validate_admin_request; use crate::admin::router::{AdminOperation, Operation, S3Router}; diff --git a/rustfs/src/admin/handlers/pools.rs b/rustfs/src/admin/handlers/pools.rs index 0e35617c8..b12f65174 100644 --- a/rustfs/src/admin/handlers/pools.rs +++ b/rustfs/src/admin/handlers/pools.rs @@ -194,7 +194,7 @@ macro_rules! log_pool_response_emitted { }; } -fn endpoints_from_context() -> Option { +fn endpoints_from_context() -> Option { resolve_endpoints_handle() } diff --git a/rustfs/src/admin/handlers/quota.rs b/rustfs/src/admin/handlers/quota.rs index 1e31c6d29..376126a85 100644 --- a/rustfs/src/admin/handlers/quota.rs +++ b/rustfs/src/admin/handlers/quota.rs @@ -14,9 +14,9 @@ //! Quota admin handlers for HTTP API -use super::storage_compat::metadata_sys::BucketMetadataSys; -use super::storage_compat::quota::checker::QuotaChecker; -use super::storage_compat::quota::{BucketQuota, QuotaError, QuotaOperation}; +use super::super::storage_compat::metadata_sys::BucketMetadataSys; +use super::super::storage_compat::quota::checker::QuotaChecker; +use super::super::storage_compat::quota::{BucketQuota, QuotaError, QuotaOperation}; use crate::admin::auth::{validate_admin_request, validate_admin_request_with_bucket}; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::app::context::{resolve_bucket_metadata_handle, resolve_object_store_handle}; @@ -175,7 +175,7 @@ async fn current_usage_from_context(bucket: &str) -> u64 { return 0; }; - match super::storage_compat::load_data_usage_from_backend(store).await { + match super::super::storage_compat::load_data_usage_from_backend(store).await { Ok(data_usage_info) => data_usage_info .buckets_usage .get(bucket) diff --git a/rustfs/src/admin/handlers/rebalance.rs b/rustfs/src/admin/handlers/rebalance.rs index 3a12a516a..50ae7633d 100644 --- a/rustfs/src/admin/handlers/rebalance.rs +++ b/rustfs/src/admin/handlers/rebalance.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::{ +use super::super::storage_compat::{ DiskStat, ECStore, NotificationSys, RebalSaveOpt, RebalanceCleanupWarnings, RebalanceMeta, RebalanceStopPropagationRecord, StorageError, decode_rebalance_stop_propagation_record, get_global_notification_sys, }; @@ -296,7 +296,7 @@ fn build_rebalance_pool_progress( now: OffsetDateTime, stop_time: Option, percent_free_goal: f64, - ps: &super::storage_compat::RebalanceStats, + ps: &super::super::storage_compat::RebalanceStats, ) -> Option { let total_bytes_to_rebal = ps.init_capacity as f64 * percent_free_goal - ps.init_free_space as f64; let terminal_time = ps.info.end_time.or(stop_time); @@ -339,7 +339,7 @@ fn build_rebalance_pool_statuses( now: OffsetDateTime, stop_time: Option, percent_free_goal: f64, - pool_stats: &[super::storage_compat::RebalanceStats], + pool_stats: &[super::super::storage_compat::RebalanceStats], disk_stats: &[DiskStat], ) -> Vec { pool_stats @@ -895,7 +895,7 @@ mod offsetdatetime_rfc3339 { #[cfg(test)] mod rebalance_handler_tests { - use super::super::storage_compat::{ + use super::super::super::storage_compat::{ DiskStat, RebalStatus, RebalanceCleanupWarningEntry, RebalanceCleanupWarnings, RebalanceInfo, RebalanceMeta, RebalanceStats, RebalanceStopPropagationRecord, encode_rebalance_stop_propagation_record, }; diff --git a/rustfs/src/admin/handlers/replication.rs b/rustfs/src/admin/handlers/replication.rs index 3ac16e8ae..89c5cbfdf 100644 --- a/rustfs/src/admin/handlers/replication.rs +++ b/rustfs/src/admin/handlers/replication.rs @@ -12,15 +12,15 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::StorageError; -use super::storage_compat::bucket_target_sys::{BucketTargetError, BucketTargetSys}; -use super::storage_compat::global_rustfs_port; -use super::storage_compat::metadata::BUCKET_TARGETS_FILE; -use super::storage_compat::metadata_sys; -use super::storage_compat::metadata_sys::get_replication_config; -use super::storage_compat::replication::BucketStats; -use super::storage_compat::replication::GLOBAL_REPLICATION_STATS; -use super::storage_compat::target::BucketTarget; +use super::super::storage_compat::StorageError; +use super::super::storage_compat::bucket_target_sys::{BucketTargetError, BucketTargetSys}; +use super::super::storage_compat::global_rustfs_port; +use super::super::storage_compat::metadata::BUCKET_TARGETS_FILE; +use super::super::storage_compat::metadata_sys; +use super::super::storage_compat::metadata_sys::get_replication_config; +use super::super::storage_compat::replication::BucketStats; +use super::super::storage_compat::replication::GLOBAL_REPLICATION_STATS; +use super::super::storage_compat::target::BucketTarget; use crate::admin::auth::validate_admin_request; use crate::admin::handlers::site_replication::site_replication_peer_deployment_id_for_endpoint; use crate::admin::router::{AdminOperation, Operation, S3Router}; diff --git a/rustfs/src/admin/handlers/site_replication.rs b/rustfs/src/admin/handlers/site_replication.rs index 5b90a8d90..8188afd7f 100644 --- a/rustfs/src/admin/handlers/site_replication.rs +++ b/rustfs/src/admin/handlers/site_replication.rs @@ -12,20 +12,20 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::Error as StorageError; -use super::storage_compat::bucket_target_sys::BucketTargetSys; -use super::storage_compat::metadata::{ +use super::super::storage_compat::Error as StorageError; +use super::super::storage_compat::bucket_target_sys::BucketTargetSys; +use super::super::storage_compat::metadata::{ BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, OBJECT_LOCK_CONFIG, }; -use super::storage_compat::metadata_sys; -use super::storage_compat::replication::GLOBAL_REPLICATION_STATS; -use super::storage_compat::replication::{ResyncOpts, get_global_replication_pool}; -use super::storage_compat::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials}; -use super::storage_compat::utils::{deserialize, serialize}; -use super::storage_compat::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _}; -use super::storage_compat::{delete_admin_config, read_admin_config, save_admin_config}; -use super::storage_compat::{get_global_deployment_id, get_global_endpoints_opt, get_global_region, global_rustfs_port}; +use super::super::storage_compat::metadata_sys; +use super::super::storage_compat::replication::GLOBAL_REPLICATION_STATS; +use super::super::storage_compat::replication::{ResyncOpts, get_global_replication_pool}; +use super::super::storage_compat::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials}; +use super::super::storage_compat::utils::{deserialize, serialize}; +use super::super::storage_compat::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _}; +use super::super::storage_compat::{delete_admin_config, read_admin_config, save_admin_config}; +use super::super::storage_compat::{get_global_deployment_id, get_global_endpoints_opt, get_global_region, global_rustfs_port}; use crate::admin::auth::validate_admin_request; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::admin::site_replication_identity::{ @@ -652,7 +652,7 @@ async fn site_replication_peer_client() -> S3Result { built } -fn runtime_tls_enabled_with(endpoints: Option<&super::storage_compat::EndpointServerPools>) -> bool { +fn runtime_tls_enabled_with(endpoints: Option<&super::super::storage_compat::EndpointServerPools>) -> bool { if !rustfs_utils::get_env_str(ENV_RUSTFS_TLS_PATH, DEFAULT_RUSTFS_TLS_PATH).is_empty() { return true; } @@ -3353,7 +3353,7 @@ fn is_stale_update(local_updated_at: OffsetDateTime, incoming_updated_at: Option } fn bucket_meta_local_updated_at( - bucket_meta: &super::storage_compat::metadata::BucketMetadata, + bucket_meta: &super::super::storage_compat::metadata::BucketMetadata, config_file: &str, ) -> OffsetDateTime { match config_file { @@ -4573,8 +4573,8 @@ impl Operation for SRRotateServiceAccountHandler { #[cfg(test)] mod tests { - use super::super::storage_compat::Endpoint; - use super::super::storage_compat::{EndpointServerPools, Endpoints, PoolEndpoints}; + use super::super::super::storage_compat::Endpoint; + use super::super::super::storage_compat::{EndpointServerPools, Endpoints, PoolEndpoints}; use super::*; use http::{HeaderMap, HeaderValue, Uri}; use rustfs_common::{get_global_outbound_tls_generation, set_global_outbound_tls_generation}; diff --git a/rustfs/src/admin/handlers/storage_compat.rs b/rustfs/src/admin/handlers/storage_compat.rs deleted file mode 100644 index 55b72cd13..000000000 --- a/rustfs/src/admin/handlers/storage_compat.rs +++ /dev/null @@ -1,96 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -pub(crate) use super::super::storage_compat::{ - AdminError, AdminReplicationConfigExt, AdminVersioningConfigExt, CollectMetricsOpts, DailyAllTierStats, DiskStat, ECStore, - ERR_TIER_ALREADY_EXISTS, ERR_TIER_BACKEND_IN_USE, ERR_TIER_BACKEND_NOT_EMPTY, ERR_TIER_CONNECT_ERR, - ERR_TIER_INVALID_CREDENTIALS, ERR_TIER_MISSING_CREDENTIALS, ERR_TIER_NAME_NOT_UPPERCASE, ERR_TIER_NOT_FOUND, - EndpointServerPools, Error, MetricType, NotificationSys, PeerRestClient, RUSTFS_META_BUCKET, RebalSaveOpt, - RebalanceCleanupWarnings, RebalanceMeta, RebalanceStats, RebalanceStopPropagationRecord, STORAGE_CLASS_SUB_SYS, StorageError, - TierConfig, TierCreds, TierType, collect_local_metrics, decode_rebalance_stop_propagation_record, delete_admin_config, - get_global_deployment_id, get_global_endpoints_opt, get_global_notification_sys, get_global_region, global_rustfs_port, - init_admin_config_defaults, is_reserved_or_invalid_bucket, load_data_usage_from_backend, read_admin_config, - read_admin_config_without_migrate, save_admin_config, save_admin_server_config, -}; - -#[cfg(test)] -pub(crate) use super::super::storage_compat::{ - Endpoint, Endpoints, PoolEndpoints, RebalStatus, RebalanceCleanupWarningEntry, RebalanceInfo, - encode_rebalance_stop_propagation_record, -}; - -pub(crate) mod bucket_target_sys { - pub(crate) use super::super::super::storage_compat::bucket_target_sys::{BucketTargetError, BucketTargetSys}; -} - -pub(crate) mod lifecycle { - pub(crate) mod bucket_lifecycle_ops { - pub(crate) use super::super::super::super::storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_TransitionState; - } - - pub(crate) mod tier_last_day_stats { - #[cfg(test)] - pub(crate) use super::super::super::super::storage_compat::lifecycle::tier_last_day_stats::LastDayTierStats; - } -} - -pub(crate) mod metadata { - pub(crate) use super::super::super::storage_compat::metadata::{ - BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, - BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, - BucketMetadata, OBJECT_LOCK_CONFIG, table_catalog_path_hash, - }; -} - -pub(crate) mod metadata_sys { - pub(crate) use super::super::super::storage_compat::metadata_sys::{ - BucketMetadataSys, delete, get, get_bucket_policy, get_bucket_targets_config, get_config_from_disk, get_lifecycle_config, - get_notification_config, get_object_lock_config, get_quota_config, get_replication_config, get_sse_config, - get_tagging_config, get_versioning_config, list_bucket_targets, update, - }; -} - -pub(crate) mod quota { - pub(crate) use super::super::super::storage_compat::quota::{BucketQuota, QuotaError, QuotaOperation}; - - pub(crate) mod checker { - pub(crate) use super::super::super::super::storage_compat::quota::checker::QuotaChecker; - } -} - -pub(crate) mod replication { - pub(crate) use super::super::super::storage_compat::replication::{ - BucketStats, GLOBAL_REPLICATION_STATS, ResyncOpts, get_global_replication_pool, - }; -} - -pub(crate) mod storageclass { - pub(crate) use super::super::super::storage_compat::storageclass::{ - INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS, RRS_ENV, STANDARD, STANDARD_ENV, - }; -} - -pub(crate) mod target { - pub(crate) use super::super::super::storage_compat::target::{ - ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials, - }; -} - -pub(crate) mod utils { - pub(crate) use super::super::super::storage_compat::utils::{deserialize, is_valid_object_prefix, serialize}; -} - -pub(crate) mod versioning_sys { - pub(crate) use super::super::super::storage_compat::versioning_sys::BucketVersioningSys; -} diff --git a/rustfs/src/admin/handlers/sts.rs b/rustfs/src/admin/handlers/sts.rs index 7a623b4f5..d9d1cebbc 100644 --- a/rustfs/src/admin/handlers/sts.rs +++ b/rustfs/src/admin/handlers/sts.rs @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. +use super::super::storage_compat::utils::serialize; use super::is_admin::IsAdminHandler; -use super::storage_compat::utils::serialize; use crate::{ admin::{ handlers::site_replication::site_replication_iam_change_hook, diff --git a/rustfs/src/admin/handlers/table_catalog.rs b/rustfs/src/admin/handlers/table_catalog.rs index 00c02b882..ed90e9059 100644 --- a/rustfs/src/admin/handlers/table_catalog.rs +++ b/rustfs/src/admin/handlers/table_catalog.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::{ECStore, metadata::table_catalog_path_hash, metadata_sys}; +use super::super::storage_compat::{ECStore, metadata::table_catalog_path_hash, metadata_sys}; use crate::admin::{ auth::{AdminResourceScope, validate_admin_request, validate_admin_request_with_bucket_object}, router::{AdminOperation, Operation, S3Router}, diff --git a/rustfs/src/admin/handlers/tier.rs b/rustfs/src/admin/handlers/tier.rs index 95fe0a1ce..0de46ebb8 100644 --- a/rustfs/src/admin/handlers/tier.rs +++ b/rustfs/src/admin/handlers/tier.rs @@ -13,8 +13,8 @@ // limitations under the License. #![allow(unused_variables, unused_mut, unused_must_use)] -use super::storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_TransitionState; -use super::storage_compat::{ +use super::super::storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_TransitionState; +use super::super::storage_compat::{ AdminError, DailyAllTierStats, ERR_TIER_ALREADY_EXISTS, ERR_TIER_BACKEND_IN_USE, ERR_TIER_BACKEND_NOT_EMPTY, ERR_TIER_CONNECT_ERR, ERR_TIER_INVALID_CREDENTIALS, ERR_TIER_MISSING_CREDENTIALS, ERR_TIER_NAME_NOT_UPPERCASE, ERR_TIER_NOT_FOUND, TierConfig, TierCreds, TierType, get_global_notification_sys, storageclass, @@ -930,7 +930,7 @@ impl Operation for ClearTier { #[cfg(test)] mod tests { - use super::super::storage_compat::lifecycle::tier_last_day_stats::LastDayTierStats; + use super::super::super::storage_compat::lifecycle::tier_last_day_stats::LastDayTierStats; use super::*; use http::Uri; use matchit::Router; diff --git a/rustfs/src/admin/handlers/trace.rs b/rustfs/src/admin/handlers/trace.rs index 219270f6e..22895a270 100644 --- a/rustfs/src/admin/handlers/trace.rs +++ b/rustfs/src/admin/handlers/trace.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::PeerRestClient; +use super::super::storage_compat::PeerRestClient; use crate::admin::router::Operation; use crate::app::context::resolve_endpoints_handle; use http::StatusCode; diff --git a/rustfs/src/admin/mod.rs b/rustfs/src/admin/mod.rs index 35fa3374a..c678b2155 100644 --- a/rustfs/src/admin/mod.rs +++ b/rustfs/src/admin/mod.rs @@ -21,7 +21,6 @@ mod plugin_contract; #[allow(dead_code)] pub(crate) mod route_policy; pub mod router; -pub(crate) mod router_storage_compat; pub mod service; pub mod site_replication_identity; pub(crate) mod storage_compat; diff --git a/rustfs/src/admin/router.rs b/rustfs/src/admin/router.rs index 815a0f3a3..3ed1a4f80 100644 --- a/rustfs/src/admin/router.rs +++ b/rustfs/src/admin/router.rs @@ -12,23 +12,23 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::router_storage_compat::GLOBAL_BOOT_TIME; -use super::router_storage_compat::PeerRestClient; -use super::router_storage_compat::bandwidth::monitor::BandwidthDetails; -use super::router_storage_compat::bucket_target_sys::{ +use super::storage_compat::GLOBAL_BOOT_TIME; +use super::storage_compat::PeerRestClient; +use super::storage_compat::bandwidth::monitor::BandwidthDetails; +use super::storage_compat::bucket_target_sys::{ BucketTargetSys, PutObjectOptions, RemoveObjectOptions, S3ClientError, TargetClient, }; -use super::router_storage_compat::get_global_notification_sys; -use super::router_storage_compat::metadata::BUCKET_TARGETS_FILE; -use super::router_storage_compat::metadata_sys; -use super::router_storage_compat::read_admin_config_without_migrate; -use super::router_storage_compat::replication::{ +use super::storage_compat::get_global_notification_sys; +use super::storage_compat::metadata::BUCKET_TARGETS_FILE; +use super::storage_compat::metadata_sys; +use super::storage_compat::read_admin_config_without_migrate; +use super::storage_compat::replication::{ BucketReplicationResyncStatus, BucketStats, GLOBAL_REPLICATION_STATS, ObjectOpts, ResyncOpts, get_global_replication_pool, }; -use super::router_storage_compat::target::{BucketTarget, BucketTargetType, BucketTargets}; -use super::router_storage_compat::versioning_sys::BucketVersioningSys; -use super::router_storage_compat::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _}; -use super::router_storage_compat::{get_global_bucket_monitor, get_global_deployment_id, get_global_region}; +use super::storage_compat::target::{BucketTarget, BucketTargetType, BucketTargets}; +use super::storage_compat::versioning_sys::BucketVersioningSys; +use super::storage_compat::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _}; +use super::storage_compat::{get_global_bucket_monitor, get_global_deployment_id, get_global_region}; use crate::admin::console::{is_console_path, make_console_server}; use crate::admin::handlers::oidc::is_oidc_path; use crate::app::context::resolve_object_store_handle; @@ -1419,7 +1419,7 @@ async fn ensure_replication_bucket_exists(bucket: &str) -> S3Result<()> { async fn ensure_replication_config_exists(bucket: &str) -> S3Result<()> { match metadata_sys::get_replication_config(bucket).await { Ok(_) => Ok(()), - Err(super::router_storage_compat::StorageError::ConfigNotFound) => Err(s3_error!(ReplicationConfigurationNotFoundError)), + Err(super::storage_compat::StorageError::ConfigNotFound) => Err(s3_error!(ReplicationConfigurationNotFoundError)), Err(err) => Err(ApiError::from(err).into()), } } @@ -1944,7 +1944,7 @@ async fn resolve_replication_target_client(bucket: &str, target: &BucketTarget) fn build_replication_probe_put_options(now: OffsetDateTime) -> PutObjectOptions { PutObjectOptions { - internal: super::router_storage_compat::bucket_target_sys::AdvancedPutOptions { + internal: super::storage_compat::bucket_target_sys::AdvancedPutOptions { source_version_id: Uuid::new_v4().to_string(), replication_status: ReplicationStatusType::Replica, source_mtime: now, @@ -2059,7 +2059,7 @@ async fn source_bucket_requires_object_lock(bucket: &str) -> S3Result { .object_lock_enabled .as_ref() .is_some_and(|state| state.as_str() == s3s::dto::ObjectLockEnabled::ENABLED)), - Err(super::router_storage_compat::StorageError::ConfigNotFound) => Ok(false), + Err(super::storage_compat::StorageError::ConfigNotFound) => Ok(false), Err(err) => Err(ApiError::from(err).into()), } } @@ -2773,7 +2773,7 @@ mod tests { #[test] fn apply_replication_reset_to_targets_updates_matching_target() { let mut targets = BucketTargets { - targets: vec![super::super::router_storage_compat::target::BucketTarget { + targets: vec![super::super::storage_compat::target::BucketTarget { arn: "arn:target".to_string(), ..Default::default() }], @@ -2795,10 +2795,10 @@ mod tests { let mut status = BucketReplicationResyncStatus::new(); status.targets_map.insert( "arn:z".to_string(), - super::super::router_storage_compat::replication::TargetReplicationResyncStatus { + super::super::storage_compat::replication::TargetReplicationResyncStatus { resync_id: "rid-z".to_string(), last_update: Some(datetime!(2025-01-03 00:00 UTC)), - resync_status: super::super::router_storage_compat::replication::ResyncStatusType::ResyncFailed, + resync_status: super::super::storage_compat::replication::ResyncStatusType::ResyncFailed, failed_count: 2, failed_size: 4, bucket: "bucket-z".to_string(), @@ -2808,10 +2808,10 @@ mod tests { ); status.targets_map.insert( "arn:a".to_string(), - super::super::router_storage_compat::replication::TargetReplicationResyncStatus { + super::super::storage_compat::replication::TargetReplicationResyncStatus { resync_id: "rid-a".to_string(), last_update: Some(datetime!(2025-01-02 00:00 UTC)), - resync_status: super::super::router_storage_compat::replication::ResyncStatusType::ResyncCompleted, + resync_status: super::super::storage_compat::replication::ResyncStatusType::ResyncCompleted, replicated_count: 3, replicated_size: 9, bucket: "bucket-a".to_string(), @@ -2841,10 +2841,10 @@ mod tests { let mut status = BucketReplicationResyncStatus::new(); status.targets_map.insert( "arn:z".to_string(), - super::super::router_storage_compat::replication::TargetReplicationResyncStatus { + super::super::storage_compat::replication::TargetReplicationResyncStatus { resync_id: "rid-z".to_string(), last_update: Some(datetime!(2025-02-03 00:00 UTC)), - resync_status: super::super::router_storage_compat::replication::ResyncStatusType::ResyncFailed, + resync_status: super::super::storage_compat::replication::ResyncStatusType::ResyncFailed, failed_count: 2, failed_size: 4, bucket: "bucket-z".to_string(), @@ -2854,10 +2854,10 @@ mod tests { ); status.targets_map.insert( "arn:a".to_string(), - super::super::router_storage_compat::replication::TargetReplicationResyncStatus { + super::super::storage_compat::replication::TargetReplicationResyncStatus { resync_id: "rid-a".to_string(), last_update: Some(datetime!(2025-02-02 00:00 UTC)), - resync_status: super::super::router_storage_compat::replication::ResyncStatusType::ResyncCompleted, + resync_status: super::super::storage_compat::replication::ResyncStatusType::ResyncCompleted, replicated_count: 3, replicated_size: 9, bucket: "bucket-a".to_string(), diff --git a/rustfs/src/admin/router_storage_compat.rs b/rustfs/src/admin/router_storage_compat.rs deleted file mode 100644 index a413a2ba4..000000000 --- a/rustfs/src/admin/router_storage_compat.rs +++ /dev/null @@ -1,54 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -pub(crate) use super::storage_compat::{ - AdminReplicationConfigExt, AdminVersioningConfigExt, GLOBAL_BOOT_TIME, PeerRestClient, StorageError, - get_global_bucket_monitor, get_global_deployment_id, get_global_notification_sys, get_global_region, - read_admin_config_without_migrate, -}; - -pub(crate) mod bandwidth { - pub(crate) mod monitor { - pub(crate) use super::super::super::storage_compat::bandwidth::monitor::BandwidthDetails; - } -} - -pub(crate) mod bucket_target_sys { - pub(crate) use super::super::storage_compat::bucket_target_sys::{ - AdvancedPutOptions, BucketTargetSys, PutObjectOptions, RemoveObjectOptions, S3ClientError, TargetClient, - }; -} - -pub(crate) mod metadata { - pub(crate) use super::super::storage_compat::metadata::BUCKET_TARGETS_FILE; -} - -pub(crate) use super::storage_compat::metadata_sys; - -pub(crate) mod replication { - pub(crate) use super::super::storage_compat::replication::{ - BucketReplicationResyncStatus, BucketStats, GLOBAL_REPLICATION_STATS, ObjectOpts, ResyncOpts, get_global_replication_pool, - }; - - #[cfg(test)] - pub(crate) use super::super::storage_compat::replication::{ResyncStatusType, TargetReplicationResyncStatus}; -} - -pub(crate) mod target { - pub(crate) use super::super::storage_compat::target::{BucketTarget, BucketTargetType, BucketTargets}; -} - -pub(crate) mod versioning_sys { - pub(crate) use super::super::storage_compat::versioning_sys::BucketVersioningSys; -} diff --git a/rustfs/src/admin/service/config.rs b/rustfs/src/admin/service/config.rs index 71350bd81..8486cdb91 100644 --- a/rustfs/src/admin/service/config.rs +++ b/rustfs/src/admin/service/config.rs @@ -12,10 +12,10 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::get_global_notification_sys; -use super::storage_compat::set_global_storage_class; -use super::storage_compat::storageclass; -use super::storage_compat::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate}; +use super::super::storage_compat::get_global_notification_sys; +use super::super::storage_compat::set_global_storage_class; +use super::super::storage_compat::storageclass; +use super::super::storage_compat::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate}; use crate::app::context::resolve_object_store_handle; use rustfs_audit::reload_audit_config; use rustfs_config::audit::{AUDIT_MQTT_SUB_SYS, AUDIT_REDIS_DEFAULT_CHANNEL, AUDIT_WEBHOOK_SUB_SYS}; @@ -370,7 +370,7 @@ pub async fn signal_config_snapshot_reload() { #[cfg(test)] mod tests { - use super::super::storage_compat::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG}; + use super::super::super::storage_compat::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG}; use super::*; use rustfs_config::notify::NOTIFY_WEBHOOK_SUB_SYS; use rustfs_config::oidc::{OIDC_CLIENT_ID, OIDC_CONFIG_URL, OIDC_SCOPES}; @@ -427,7 +427,7 @@ mod tests { #[test] fn validate_notify_subsystem_config_rejects_invalid_webhook_endpoint() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); let mut config = ServerConfig::new(); let targets = config.0.get_mut(NOTIFY_WEBHOOK_SUB_SYS).expect("notify webhook defaults"); let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target"); @@ -441,7 +441,7 @@ mod tests { #[test] fn validate_audit_subsystem_config_rejects_relative_queue_dir() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); let mut config = ServerConfig::new(); let targets = config.0.get_mut(AUDIT_MQTT_SUB_SYS).expect("audit mqtt defaults"); let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target"); @@ -456,7 +456,7 @@ mod tests { #[test] fn validate_identity_openid_config_rejects_missing_openid_scope() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); let mut config = ServerConfig::new(); let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults"); let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target"); @@ -473,7 +473,7 @@ mod tests { #[test] fn validate_identity_openid_config_rejects_invalid_named_provider_id() { - super::super::storage_compat::init_admin_config_defaults(); + super::super::super::storage_compat::init_admin_config_defaults(); let mut config = ServerConfig::new(); let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults"); let default_kvs = targets.get(DEFAULT_DELIMITER).cloned().expect("default target"); diff --git a/rustfs/src/admin/service/mod.rs b/rustfs/src/admin/service/mod.rs index 097e35a41..5af8b3f7e 100644 --- a/rustfs/src/admin/service/mod.rs +++ b/rustfs/src/admin/service/mod.rs @@ -14,4 +14,3 @@ pub mod config; pub mod site_replication; -pub(crate) mod storage_compat; diff --git a/rustfs/src/admin/service/site_replication.rs b/rustfs/src/admin/service/site_replication.rs index d1cd3fa8e..b77c12eda 100644 --- a/rustfs/src/admin/service/site_replication.rs +++ b/rustfs/src/admin/service/site_replication.rs @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::Error as StorageError; -use super::storage_compat::{read_admin_config, save_admin_config}; +use super::super::storage_compat::Error as StorageError; +use super::super::storage_compat::{read_admin_config, save_admin_config}; use crate::admin::site_replication_identity::{deployment_id_for_endpoint, normalize_peer_map_by_identity_with}; use crate::app::context::resolve_object_store_handle; use rustfs_madmin::PeerInfo; diff --git a/rustfs/src/admin/service/storage_compat.rs b/rustfs/src/admin/service/storage_compat.rs deleted file mode 100644 index 2f7ba6922..000000000 --- a/rustfs/src/admin/service/storage_compat.rs +++ /dev/null @@ -1,30 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -pub(crate) use super::super::storage_compat::{ - Error, STORAGE_CLASS_SUB_SYS, get_global_notification_sys, read_admin_config, read_admin_config_without_migrate, - save_admin_config, set_global_storage_class, -}; - -#[cfg(test)] -pub(crate) use super::super::storage_compat::init_admin_config_defaults; - -#[cfg(test)] -pub(crate) mod metadata { - pub(crate) use super::super::super::storage_compat::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG}; -} - -pub(crate) mod storageclass { - pub(crate) use super::super::super::storage_compat::storageclass::lookup_config; -} diff --git a/rustfs/src/app/admin_usecase.rs b/rustfs/src/app/admin_usecase.rs index 26bd8cb7c..127a86224 100644 --- a/rustfs/src/app/admin_usecase.rs +++ b/rustfs/src/app/admin_usecase.rs @@ -14,13 +14,11 @@ //! Admin application use-case contracts. -use super::usecase_storage_compat::ECStore; -use super::usecase_storage_compat::EndpointServerPools; -use super::usecase_storage_compat::get_server_info; -use super::usecase_storage_compat::{ - PoolDecommissionInfo, PoolStatus, get_total_usable_capacity, get_total_usable_capacity_free, -}; -use super::usecase_storage_compat::{apply_bucket_usage_memory_overlay, load_data_usage_from_backend}; +use super::storage_compat::ECStore; +use super::storage_compat::EndpointServerPools; +use super::storage_compat::get_server_info; +use super::storage_compat::{PoolDecommissionInfo, PoolStatus, get_total_usable_capacity, get_total_usable_capacity_free}; +use super::storage_compat::{apply_bucket_usage_memory_overlay, load_data_usage_from_backend}; use crate::app::context::{AppContext, get_global_app_context, resolve_object_store_handle_for_context}; use crate::capacity::resolve_admin_used_capacity; use crate::error::ApiError; @@ -403,7 +401,7 @@ impl DefaultAdminUsecase { #[cfg(test)] mod tests { - use super::super::usecase_storage_compat::{PoolDecommissionInfo, PoolStatus}; + use super::super::storage_compat::{PoolDecommissionInfo, PoolStatus}; use super::*; use time::OffsetDateTime; diff --git a/rustfs/src/app/bucket_usecase.rs b/rustfs/src/app/bucket_usecase.rs index 2e1141f1d..24d240002 100644 --- a/rustfs/src/app/bucket_usecase.rs +++ b/rustfs/src/app/bucket_usecase.rs @@ -14,12 +14,12 @@ //! Bucket application use-case contracts. -use super::usecase_storage_compat::ECStore; -use super::usecase_storage_compat::StorageError; -use super::usecase_storage_compat::get_global_notification_sys; -use super::usecase_storage_compat::object_api_utils::to_s3s_etag; -use super::usecase_storage_compat::{AppObjectLockConfigExt as _, AppVersioningConfigExt as _}; -use super::usecase_storage_compat::{ +use super::storage_compat::ECStore; +use super::storage_compat::StorageError; +use super::storage_compat::get_global_notification_sys; +use super::storage_compat::object_api_utils::to_s3s_etag; +use super::storage_compat::{AppObjectLockConfigExt as _, AppVersioningConfigExt as _}; +use super::storage_compat::{ bucket_target_sys::BucketTargetSys, lifecycle::bucket_lifecycle_ops::{ enqueue_expiry_for_existing_objects, enqueue_transition_for_existing_objects, validate_lifecycle_config, @@ -2285,7 +2285,7 @@ mod tests { BucketTargets { targets: arns .iter() - .map(|arn| super::super::usecase_storage_compat::target::BucketTarget { + .map(|arn| super::super::storage_compat::target::BucketTarget { arn: (*arn).to_string(), target_type: BucketTargetType::ReplicationService, ..Default::default() diff --git a/rustfs/src/app/capacity_dirty_scope_test.rs b/rustfs/src/app/capacity_dirty_scope_test.rs index a7d0a8bd9..eddd14c3f 100644 --- a/rustfs/src/app/capacity_dirty_scope_test.rs +++ b/rustfs/src/app/capacity_dirty_scope_test.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::usecase_storage_compat::{ECStore, Endpoint, EndpointServerPools, Endpoints, PoolEndpoints, metadata_sys}; +use super::storage_compat::{ECStore, Endpoint, EndpointServerPools, Endpoints, PoolEndpoints, metadata_sys}; use rustfs_common::heal_channel::{HealOpts, HealScanMode}; use rustfs_object_capacity::capacity_manager::{HybridStrategyConfig, create_isolated_manager}; use rustfs_storage_api::{BucketOperations, BucketOptions, HealOperations as _, MakeBucketOptions, ObjectIO as _}; @@ -77,9 +77,7 @@ async fn setup_capacity_dirty_scope_env() -> (Vec, Arc) { }; let endpoint_pools = EndpointServerPools(vec![pool_endpoints]); - super::usecase_storage_compat::init_local_disks(endpoint_pools.clone()) - .await - .unwrap(); + super::storage_compat::init_local_disks(endpoint_pools.clone()).await.unwrap(); let server_addr: std::net::SocketAddr = "127.0.0.1:0".parse().unwrap(); let ecstore = ECStore::new(server_addr, endpoint_pools, CancellationToken::new()) diff --git a/rustfs/src/app/context.rs b/rustfs/src/app/context.rs index 664a53f52..f74872c75 100644 --- a/rustfs/src/app/context.rs +++ b/rustfs/src/app/context.rs @@ -21,7 +21,6 @@ mod global; mod handles; mod interfaces; mod startup; -pub(crate) mod storage_compat; pub use compat::*; pub use global::*; diff --git a/rustfs/src/app/context/compat.rs b/rustfs/src/app/context/compat.rs index d8af3332f..5cfb79f7b 100644 --- a/rustfs/src/app/context/compat.rs +++ b/rustfs/src/app/context/compat.rs @@ -12,16 +12,16 @@ // See the License for the specific language governing permissions and // limitations under the License. +use super::super::storage_compat::ECStore; +use super::super::storage_compat::EndpointServerPools; +use super::super::storage_compat::TierConfigMgr; +use super::super::storage_compat::metadata_sys::BucketMetadataSys; +use super::super::storage_compat::new_object_layer_fn; use super::global::{AppContext, get_global_app_context}; use super::handles::{ default_bucket_metadata_interface, default_endpoints_interface, default_kms_runtime_interface, default_server_config_interface, default_tier_config_interface, }; -use super::storage_compat::ECStore; -use super::storage_compat::EndpointServerPools; -use super::storage_compat::TierConfigMgr; -use super::storage_compat::metadata_sys::BucketMetadataSys; -use super::storage_compat::new_object_layer_fn; #[cfg(test)] use crate::config::RustFSBufferConfig; use rustfs_config::server_config::Config; @@ -119,10 +119,10 @@ fn resolve_buffer_config_with( #[cfg(test)] mod tests { - use super::super::storage_compat::Endpoint; - use super::super::storage_compat::init_local_disks; - use super::super::storage_compat::new_object_layer_fn; - use super::super::storage_compat::{Endpoints, PoolEndpoints}; + use super::super::super::storage_compat::Endpoint; + use super::super::super::storage_compat::init_local_disks; + use super::super::super::storage_compat::new_object_layer_fn; + use super::super::super::storage_compat::{Endpoints, PoolEndpoints}; use super::*; use crate::app::context::global::AppContextTestInterfaces; use crate::app::context::handles::{default_notify_interface, default_region_interface}; diff --git a/rustfs/src/app/context/global.rs b/rustfs/src/app/context/global.rs index f2f201b06..937949512 100644 --- a/rustfs/src/app/context/global.rs +++ b/rustfs/src/app/context/global.rs @@ -12,6 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. +use super::super::storage_compat::{ECStore, set_object_store_resolver}; use super::handles::{ IamHandle, KmsHandle, default_bucket_metadata_interface, default_buffer_config_interface, default_endpoints_interface, default_kms_runtime_interface, default_notify_interface, default_region_interface, default_server_config_interface, @@ -21,7 +22,6 @@ use super::interfaces::{ BucketMetadataInterface, BufferConfigInterface, EndpointsInterface, IamInterface, KmsInterface, KmsRuntimeInterface, NotifyInterface, RegionInterface, ServerConfigInterface, TierConfigInterface, }; -use super::storage_compat::{ECStore, set_object_store_resolver}; use rustfs_iam::{store::object::ObjectStore, sys::IamSys}; use rustfs_kms::KmsServiceManager; use std::sync::{Arc, OnceLock}; diff --git a/rustfs/src/app/context/handles.rs b/rustfs/src/app/context/handles.rs index 7ba2f75c8..ac8dcabce 100644 --- a/rustfs/src/app/context/handles.rs +++ b/rustfs/src/app/context/handles.rs @@ -12,14 +12,14 @@ // See the License for the specific language governing permissions and // limitations under the License. +use super::super::storage_compat::EndpointServerPools; +use super::super::storage_compat::TierConfigMgr; +use super::super::storage_compat::metadata_sys::{BucketMetadataSys, get_global_bucket_metadata_sys}; +use super::super::storage_compat::{get_global_endpoints_opt, get_global_region, get_global_tier_config_mgr}; use super::interfaces::{ BucketMetadataInterface, BufferConfigInterface, EndpointsInterface, IamInterface, KmsInterface, KmsRuntimeInterface, NotifyInterface, RegionInterface, ServerConfigInterface, TierConfigInterface, }; -use super::storage_compat::EndpointServerPools; -use super::storage_compat::TierConfigMgr; -use super::storage_compat::metadata_sys::{BucketMetadataSys, get_global_bucket_metadata_sys}; -use super::storage_compat::{get_global_endpoints_opt, get_global_region, get_global_tier_config_mgr}; use crate::config::{RustFSBufferConfig, get_global_buffer_config}; use async_trait::async_trait; use rustfs_config::server_config::Config; diff --git a/rustfs/src/app/context/interfaces.rs b/rustfs/src/app/context/interfaces.rs index 6a9ccaf31..07e3a1746 100644 --- a/rustfs/src/app/context/interfaces.rs +++ b/rustfs/src/app/context/interfaces.rs @@ -12,9 +12,9 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::EndpointServerPools; -use super::storage_compat::TierConfigMgr; -use super::storage_compat::metadata_sys::BucketMetadataSys; +use super::super::storage_compat::EndpointServerPools; +use super::super::storage_compat::TierConfigMgr; +use super::super::storage_compat::metadata_sys::BucketMetadataSys; use crate::config::RustFSBufferConfig; use async_trait::async_trait; use rustfs_config::server_config::Config; diff --git a/rustfs/src/app/context/startup.rs b/rustfs/src/app/context/startup.rs index f3263ae5f..da589d6d7 100644 --- a/rustfs/src/app/context/startup.rs +++ b/rustfs/src/app/context/startup.rs @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. +use super::super::storage_compat::ECStore; use super::global::{AppContext, get_global_app_context, init_global_app_context}; -use super::storage_compat::ECStore; use rustfs_kms::KmsServiceManager; use std::io::{Error, Result}; use std::sync::Arc; diff --git a/rustfs/src/app/context/storage_compat.rs b/rustfs/src/app/context/storage_compat.rs deleted file mode 100644 index c8226daee..000000000 --- a/rustfs/src/app/context/storage_compat.rs +++ /dev/null @@ -1,25 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -pub(crate) use super::super::storage_compat::{ - ECStore, EndpointServerPools, TierConfigMgr, get_global_endpoints_opt, get_global_region, get_global_tier_config_mgr, - new_object_layer_fn, set_object_store_resolver, -}; - -pub(crate) mod metadata_sys { - pub(crate) use super::super::super::storage_compat::metadata_sys::{BucketMetadataSys, get_global_bucket_metadata_sys}; -} - -#[cfg(test)] -pub(crate) use super::super::storage_compat::{Endpoint, Endpoints, PoolEndpoints, init_local_disks}; diff --git a/rustfs/src/app/lifecycle_transition_api_test.rs b/rustfs/src/app/lifecycle_transition_api_test.rs index edbc9c3ad..c03b621e7 100644 --- a/rustfs/src/app/lifecycle_transition_api_test.rs +++ b/rustfs/src/app/lifecycle_transition_api_test.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::usecase_storage_compat::{ +use super::storage_compat::{ AppWarmBackend, ECStore, Endpoint, EndpointServerPools, Endpoints, GLOBAL_TierConfigMgr, PoolEndpoints, TierConfig, TierType, WarmBackendGetOpts, metadata::{BUCKET_LIFECYCLE_CONFIG, OBJECT_LOCK_CONFIG}, @@ -107,9 +107,7 @@ async fn setup_test_env() -> (Vec, Arc) { let endpoint_pools = EndpointServerPools(vec![pool_endpoints]); - super::usecase_storage_compat::init_local_disks(endpoint_pools.clone()) - .await - .unwrap(); + super::storage_compat::init_local_disks(endpoint_pools.clone()).await.unwrap(); let server_addr: std::net::SocketAddr = "127.0.0.1:9003".parse().unwrap(); let ecstore = ECStore::new(server_addr, endpoint_pools, CancellationToken::new()) @@ -126,7 +124,7 @@ async fn setup_test_env() -> (Vec, Arc) { let buckets = buckets_list.into_iter().map(|v| v.name).collect(); metadata_sys::init_bucket_metadata_sys(ecstore.clone(), buckets).await; - super::usecase_storage_compat::lifecycle::bucket_lifecycle_ops::init_background_expiry(ecstore.clone()).await; + super::storage_compat::lifecycle::bucket_lifecycle_ops::init_background_expiry(ecstore.clone()).await; let _ = GLOBAL_ENV.set((disk_paths.clone(), ecstore.clone())); @@ -926,7 +924,7 @@ async fn delete_transitioned_object_removes_remote_tier_copy_via_usecase() { .expect("Failed to set lifecycle configuration"); let _ = upload_test_object(&ecstore, bucket.as_str(), object, payload).await; - super::usecase_storage_compat::lifecycle::bucket_lifecycle_ops::enqueue_transition_for_existing_objects( + super::storage_compat::lifecycle::bucket_lifecycle_ops::enqueue_transition_for_existing_objects( ecstore.clone(), bucket.as_str(), ) @@ -986,7 +984,7 @@ async fn lifecycle_transition_marks_dirty_disks_for_capacity_manager() { .expect("Failed to set lifecycle configuration"); let _ = upload_test_object(&ecstore, bucket.as_str(), object, payload).await; - super::usecase_storage_compat::lifecycle::bucket_lifecycle_ops::enqueue_transition_for_existing_objects( + super::storage_compat::lifecycle::bucket_lifecycle_ops::enqueue_transition_for_existing_objects( ecstore.clone(), bucket.as_str(), ) diff --git a/rustfs/src/app/mod.rs b/rustfs/src/app/mod.rs index e3c8f68fe..3a066cf39 100644 --- a/rustfs/src/app/mod.rs +++ b/rustfs/src/app/mod.rs @@ -22,7 +22,6 @@ pub mod multipart_usecase; pub mod object_usecase; mod select_object; pub(crate) mod storage_compat; -pub(crate) mod usecase_storage_compat; #[cfg(test)] mod capacity_dirty_scope_test; diff --git a/rustfs/src/app/multipart_usecase.rs b/rustfs/src/app/multipart_usecase.rs index 75799b550..2da8906bb 100644 --- a/rustfs/src/app/multipart_usecase.rs +++ b/rustfs/src/app/multipart_usecase.rs @@ -14,16 +14,16 @@ //! Multipart application use-case contracts. -use super::usecase_storage_compat::ECStore; -use super::usecase_storage_compat::is_disk_compressible; -use super::usecase_storage_compat::is_valid_storage_class; -use super::usecase_storage_compat::object_api_utils::to_s3s_etag; -use super::usecase_storage_compat::quota::checker::QuotaChecker; +use super::storage_compat::ECStore; +use super::storage_compat::is_disk_compressible; +use super::storage_compat::is_valid_storage_class; +use super::storage_compat::object_api_utils::to_s3s_etag; +use super::storage_compat::quota::checker::QuotaChecker; #[cfg(test)] -use super::usecase_storage_compat::{DecryptReader, EncryptReader, HardLimitReader, boxed_reader, wrap_reader}; -use super::usecase_storage_compat::{HashReader, WritePlan}; -use super::usecase_storage_compat::{StorageError, is_err_object_not_found, is_err_version_not_found}; -use super::usecase_storage_compat::{ +use super::storage_compat::{DecryptReader, EncryptReader, HardLimitReader, boxed_reader, wrap_reader}; +use super::storage_compat::{HashReader, WritePlan}; +use super::storage_compat::{StorageError, is_err_object_not_found, is_err_version_not_found}; +use super::storage_compat::{ lifecycle::{bucket_lifecycle_audit::LcEventSrc, bucket_lifecycle_ops::enqueue_transition_immediate}, metadata_sys, quota::QuotaOperation, @@ -482,7 +482,7 @@ impl DefaultMultipartUsecase { )); } // Update quota tracking after successful multipart upload - super::usecase_storage_compat::record_bucket_object_write_memory( + super::storage_compat::record_bucket_object_write_memory( &bucket, previous_current_size, obj_info.size.max(0) as u64, @@ -675,7 +675,7 @@ impl DefaultMultipartUsecase { rustfs_utils::http::insert_str( &mut metadata, rustfs_utils::http::SUFFIX_COMPRESSION, - super::usecase_storage_compat::compression_metadata_value(CompressionAlgorithm::default()), + super::storage_compat::compression_metadata_value(CompressionAlgorithm::default()), ); } @@ -894,13 +894,11 @@ impl DefaultMultipartUsecase { .ok_or_else(|| ApiError::from(StorageError::other("Missing SSE-C session material")))?; let ssec_write = match ssec_material.key_kind { crate::storage::sse::EncryptionKeyKind::Object => { - super::usecase_storage_compat::WriteEncryption::multipart_object_key(ssec_material.key_bytes, part_id as u32) + super::storage_compat::WriteEncryption::multipart_object_key(ssec_material.key_bytes, part_id as u32) + } + crate::storage::sse::EncryptionKeyKind::Direct => { + super::storage_compat::WriteEncryption::multipart(ssec_material.key_bytes, ssec_material.base_nonce, part_id) } - crate::storage::sse::EncryptionKeyKind::Direct => super::usecase_storage_compat::WriteEncryption::multipart( - ssec_material.key_bytes, - ssec_material.base_nonce, - part_id, - ), }; write_plan = write_plan.with_encryption(ssec_write); (Some(ssec_material.server_side_encryption), ssec_material.kms_key_id) @@ -916,12 +914,9 @@ impl DefaultMultipartUsecase { .ok_or_else(|| ApiError::from(StorageError::other("Missing managed SSE session material")))?; let managed_write = match managed_material.key_kind { crate::storage::sse::EncryptionKeyKind::Object => { - super::usecase_storage_compat::WriteEncryption::multipart_object_key( - managed_material.key_bytes, - part_id as u32, - ) + super::storage_compat::WriteEncryption::multipart_object_key(managed_material.key_bytes, part_id as u32) } - crate::storage::sse::EncryptionKeyKind::Direct => super::usecase_storage_compat::WriteEncryption::multipart( + crate::storage::sse::EncryptionKeyKind::Direct => super::storage_compat::WriteEncryption::multipart( managed_material.key_bytes, managed_material.base_nonce, part_id, @@ -1240,13 +1235,11 @@ impl DefaultMultipartUsecase { .ok_or_else(|| ApiError::from(StorageError::other("Missing SSE-C session material")))?; let ssec_write = match ssec_material.key_kind { crate::storage::sse::EncryptionKeyKind::Object => { - super::usecase_storage_compat::WriteEncryption::multipart_object_key(ssec_material.key_bytes, part_id as u32) + super::storage_compat::WriteEncryption::multipart_object_key(ssec_material.key_bytes, part_id as u32) + } + crate::storage::sse::EncryptionKeyKind::Direct => { + super::storage_compat::WriteEncryption::multipart(ssec_material.key_bytes, ssec_material.base_nonce, part_id) } - crate::storage::sse::EncryptionKeyKind::Direct => super::usecase_storage_compat::WriteEncryption::multipart( - ssec_material.key_bytes, - ssec_material.base_nonce, - part_id, - ), }; write_plan = write_plan.with_encryption(ssec_write); ( @@ -1266,12 +1259,9 @@ impl DefaultMultipartUsecase { .ok_or_else(|| ApiError::from(StorageError::other("Missing managed SSE session material")))?; let managed_write = match managed_material.key_kind { crate::storage::sse::EncryptionKeyKind::Object => { - super::usecase_storage_compat::WriteEncryption::multipart_object_key( - managed_material.key_bytes, - part_id as u32, - ) + super::storage_compat::WriteEncryption::multipart_object_key(managed_material.key_bytes, part_id as u32) } - crate::storage::sse::EncryptionKeyKind::Direct => super::usecase_storage_compat::WriteEncryption::multipart( + crate::storage::sse::EncryptionKeyKind::Direct => super::storage_compat::WriteEncryption::multipart( managed_material.key_bytes, managed_material.base_nonce, part_id, diff --git a/rustfs/src/app/object_usecase.rs b/rustfs/src/app/object_usecase.rs index 9ab177b74..f574c1f9f 100644 --- a/rustfs/src/app/object_usecase.rs +++ b/rustfs/src/app/object_usecase.rs @@ -48,21 +48,21 @@ use metrics::{counter, histogram}; use pin_project_lite::pin_project; use rustfs_object_capacity::capacity_manager::get_capacity_manager; // Performance metrics recording (with zero-copy-metrics integration) -use super::usecase_storage_compat::ECStore; -use super::usecase_storage_compat::object_api_utils::to_s3s_etag; -use super::usecase_storage_compat::quota::checker::QuotaChecker; -use super::usecase_storage_compat::storageclass; -use super::usecase_storage_compat::{ +use super::storage_compat::ECStore; +use super::storage_compat::object_api_utils::to_s3s_etag; +use super::storage_compat::quota::checker::QuotaChecker; +use super::storage_compat::storageclass; +use super::storage_compat::{ AppReplicationConfigExt as _, AppVersioningConfigExt as _, predict_lifecycle_expiration, validate_restore_request, }; -use super::usecase_storage_compat::{DiskError, is_all_buckets_not_found}; -use super::usecase_storage_compat::{DynReader, HashReader, WritePlan, wrap_reader}; -use super::usecase_storage_compat::{ +use super::storage_compat::{DiskError, is_all_buckets_not_found}; +use super::storage_compat::{DynReader, HashReader, WritePlan, wrap_reader}; +use super::storage_compat::{ Error as EcstoreError, StorageError, is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found, }; -use super::usecase_storage_compat::{MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible}; -use super::usecase_storage_compat::{get_lock_acquire_timeout, is_valid_storage_class}; -use super::usecase_storage_compat::{ +use super::storage_compat::{MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible}; +use super::storage_compat::{get_lock_acquire_timeout, is_valid_storage_class}; +use super::storage_compat::{ lifecycle::{ bucket_lifecycle_audit::LcEventSrc, bucket_lifecycle_ops::{enqueue_transition_immediate, post_restore_opts}, @@ -279,12 +279,10 @@ async fn enqueue_transitioned_delete_cleanup( let _activity_guard = DeleteTailActivityGuard::new(DeleteTailStage::Cleanup); let je = if opts.delete_prefix { - super::usecase_storage_compat::lifecycle::tier_sweeper::transitioned_force_delete_journal_entry( - &existing.transitioned_object, - ) + super::storage_compat::lifecycle::tier_sweeper::transitioned_force_delete_journal_entry(&existing.transitioned_object) } else { let version_id = opts.version_id.as_ref().and_then(|v| Uuid::parse_str(v).ok()); - super::usecase_storage_compat::lifecycle::tier_sweeper::transitioned_delete_journal_entry( + super::storage_compat::lifecycle::tier_sweeper::transitioned_delete_journal_entry( version_id, opts.versioned, opts.version_suspended, @@ -295,9 +293,9 @@ async fn enqueue_transitioned_delete_cleanup( return Ok(()); }; - super::usecase_storage_compat::lifecycle::tier_delete_journal::persist_tier_delete_journal_entry(store, &je).await?; + super::storage_compat::lifecycle::tier_delete_journal::persist_tier_delete_journal_entry(store, &je).await?; - let mut expiry_state = super::usecase_storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_ExpiryState + let mut expiry_state = super::storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_ExpiryState .write() .await; if let Err(err) = expiry_state.enqueue_tier_journal_entry(&je).await { @@ -1603,7 +1601,7 @@ impl DefaultObjectUsecase { #[allow(clippy::too_many_arguments)] async fn prepare_get_object_read( req: &S3Request, - store: &super::usecase_storage_compat::ECStore, + store: &super::storage_compat::ECStore, manager: &ConcurrencyManager, bucket: &str, key: &str, @@ -2197,7 +2195,7 @@ impl DefaultObjectUsecase { insert_str( &mut metadata, SUFFIX_COMPRESSION, - super::usecase_storage_compat::compression_metadata_value(algorithm), + super::storage_compat::compression_metadata_value(algorithm), ); insert_str(&mut metadata, SUFFIX_ACTUAL_SIZE, size.to_string()); @@ -2212,7 +2210,7 @@ impl DefaultObjectUsecase { insert_str( &mut opts.user_defined, SUFFIX_COMPRESSION, - super::usecase_storage_compat::compression_metadata_value(algorithm), + super::storage_compat::compression_metadata_value(algorithm), ); insert_str(&mut opts.user_defined, SUFFIX_ACTUAL_SIZE, size.to_string()); @@ -2404,12 +2402,8 @@ impl DefaultObjectUsecase { maybe_enqueue_transition_immediate(&obj_info, LcEventSrc::S3PutObject).await; // Fast in-memory update for immediate quota and admin usage consistency - super::usecase_storage_compat::record_bucket_object_write_memory( - &bucket, - previous_current_size, - obj_info.size.max(0) as u64, - ) - .await; + super::storage_compat::record_bucket_object_write_memory(&bucket, previous_current_size, obj_info.size.max(0) as u64) + .await; let raw_version = obj_info.version_id.map(|v| v.to_string()); @@ -3269,7 +3263,7 @@ impl DefaultObjectUsecase { insert_str( &mut compress_metadata, SUFFIX_COMPRESSION, - super::usecase_storage_compat::compression_metadata_value(CompressionAlgorithm::default()), + super::storage_compat::compression_metadata_value(CompressionAlgorithm::default()), ); insert_str(&mut compress_metadata, SUFFIX_ACTUAL_SIZE, actual_size.to_string()); } else { @@ -3362,12 +3356,7 @@ impl DefaultObjectUsecase { // Update quota tracking after successful copy if has_bucket_metadata { - super::usecase_storage_compat::record_bucket_object_write_memory( - &bucket, - previous_current_size, - oi.size.max(0) as u64, - ) - .await; + super::storage_compat::record_bucket_object_write_memory(&bucket, previous_current_size, oi.size.max(0) as u64).await; } let raw_dest_version = oi.version_id.map(|v| v.to_string()); @@ -3644,7 +3633,7 @@ impl DefaultObjectUsecase { ); } let size = object_sizes[i].max(0) as u64; - super::usecase_storage_compat::record_bucket_object_delete_memory( + super::storage_compat::record_bucket_object_delete_memory( &bucket, size, existing_object_infos[i].is_some() && object_to_delete[i].version_id.is_none(), @@ -3882,7 +3871,7 @@ impl DefaultObjectUsecase { } // Fast in-memory update for immediate quota and admin usage consistency - super::usecase_storage_compat::record_bucket_object_delete_memory( + super::storage_compat::record_bucket_object_delete_memory( &bucket, obj_info.size.max(0) as u64, opts.version_id.is_none(), @@ -4859,7 +4848,7 @@ impl DefaultObjectUsecase { insert_str( &mut metadata, SUFFIX_COMPRESSION, - super::usecase_storage_compat::compression_metadata_value(algorithm), + super::storage_compat::compression_metadata_value(algorithm), ); insert_str(&mut metadata, SUFFIX_ACTUAL_SIZE, size.to_string()); diff --git a/rustfs/src/app/usecase_storage_compat.rs b/rustfs/src/app/usecase_storage_compat.rs deleted file mode 100644 index c0bec381d..000000000 --- a/rustfs/src/app/usecase_storage_compat.rs +++ /dev/null @@ -1,159 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -pub(crate) use super::storage_compat::{ - AppObjectLockConfigExt, AppReplicationConfigExt, AppVersioningConfigExt, DiskError, DynReader, ECStore, EndpointServerPools, - Error, HashReader, MIN_DISK_COMPRESSIBLE_SIZE, PoolDecommissionInfo, PoolStatus, StorageError, WriteEncryption, WritePlan, - apply_bucket_usage_memory_overlay, compression_metadata_value, get_global_notification_sys, get_lock_acquire_timeout, - get_server_info, get_total_usable_capacity, get_total_usable_capacity_free, is_all_buckets_not_found, is_disk_compressible, - is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found, is_valid_storage_class, - load_data_usage_from_backend, predict_lifecycle_expiration, record_bucket_object_delete_memory, - record_bucket_object_write_memory, validate_restore_request, wrap_reader, -}; - -#[cfg(test)] -pub(crate) use super::storage_compat::{ - AppWarmBackend, DecryptReader, EncryptReader, Endpoint, Endpoints, GLOBAL_TierConfigMgr, HardLimitReader, PoolEndpoints, - TierConfig, TierType, WarmBackendGetOpts, boxed_reader, init_local_disks, -}; - -pub(crate) mod bucket_target_sys { - pub(crate) use super::super::storage_compat::bucket_target_sys::BucketTargetSys; -} - -pub(crate) mod lifecycle { - pub(crate) mod bucket_lifecycle_audit { - pub(crate) use super::super::super::storage_compat::lifecycle::bucket_lifecycle_audit::LcEventSrc; - } - - pub(crate) mod bucket_lifecycle_ops { - pub(crate) use super::super::super::storage_compat::lifecycle::bucket_lifecycle_ops::{ - GLOBAL_ExpiryState, enqueue_expiry_for_existing_objects, enqueue_transition_for_existing_objects, - enqueue_transition_immediate, post_restore_opts, validate_lifecycle_config, validate_transition_tier, - }; - - #[cfg(test)] - pub(crate) use super::super::super::storage_compat::lifecycle::bucket_lifecycle_ops::init_background_expiry; - } - - pub(crate) mod lifecycle_contract { - pub(crate) use super::super::super::storage_compat::lifecycle::lifecycle::{ - Event, ObjectOpts, TRANSITION_COMPLETE, TransitionOptions, expected_expiry_time, - }; - - #[cfg(test)] - pub(crate) use super::super::super::storage_compat::lifecycle::lifecycle::IlmAction; - } - pub(crate) use lifecycle_contract as lifecycle; - - pub(crate) mod tier_delete_journal { - pub(crate) use super::super::super::storage_compat::lifecycle::tier_delete_journal::persist_tier_delete_journal_entry; - } - - pub(crate) mod tier_sweeper { - pub(crate) use super::super::super::storage_compat::lifecycle::tier_sweeper::{ - transitioned_delete_journal_entry, transitioned_force_delete_journal_entry, - }; - } -} - -pub(crate) mod metadata { - pub(crate) use super::super::storage_compat::metadata::{ - BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, - BUCKET_PUBLIC_ACCESS_BLOCK_CONFIG, BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, - BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, - }; - - #[cfg(test)] - pub(crate) use super::super::storage_compat::metadata::OBJECT_LOCK_CONFIG; -} - -pub(crate) mod metadata_sys { - pub(crate) use super::super::storage_compat::metadata_sys::{ - BucketMetadataSys, delete, get_bucket_policy, get_bucket_policy_raw, get_bucket_targets_config, get_cors_config, - get_lifecycle_config, get_notification_config, get_object_lock_config, get_public_access_block_config, - get_replication_config, get_sse_config, get_tagging_config, update, - }; - - #[cfg(test)] - pub(crate) use super::super::storage_compat::metadata_sys::init_bucket_metadata_sys; -} - -pub(crate) mod object_api_utils { - pub(crate) use super::super::storage_compat::object_api_utils::to_s3s_etag; -} - -pub(crate) mod object_lock { - pub(crate) mod objectlock { - pub(crate) use super::super::super::storage_compat::object_lock::objectlock::{ - get_object_legalhold_meta, get_object_retention_meta, - }; - } - - pub(crate) mod objectlock_sys { - pub(crate) use super::super::super::storage_compat::object_lock::objectlock_sys::{ - BucketObjectLockSys, check_object_lock_for_deletion, is_retention_active, - }; - } -} - -pub(crate) mod policy_sys { - pub(crate) use super::super::storage_compat::policy_sys::PolicySys; -} - -pub(crate) mod quota { - pub(crate) use super::super::storage_compat::quota::QuotaOperation; - - pub(crate) mod checker { - pub(crate) use super::super::super::storage_compat::quota::checker::QuotaChecker; - } -} - -pub(crate) mod replication { - pub(crate) use super::super::storage_compat::replication::{ - DeletedObjectReplicationInfo, ObjectOpts, check_replicate_delete, get_must_replicate_options, must_replicate, - schedule_replication, schedule_replication_delete, - }; -} - -pub(crate) mod storageclass { - pub(crate) use super::super::storage_compat::storageclass::STANDARD; - - #[cfg(test)] - pub(crate) use super::super::storage_compat::storageclass::STANDARD_IA; -} - -pub(crate) mod tagging { - pub(crate) use super::super::storage_compat::tagging::decode_tags; -} - -pub(crate) mod target { - pub(crate) use super::super::storage_compat::target::{BucketTargetType, BucketTargets}; - - #[cfg(test)] - pub(crate) use super::super::storage_compat::target::BucketTarget; -} - -#[cfg(test)] -pub(crate) mod transition_api { - pub(crate) use super::super::storage_compat::transition_api::{ReadCloser, ReaderImpl}; -} - -pub(crate) mod utils { - pub(crate) use super::super::storage_compat::utils::serialize; -} - -pub(crate) mod versioning_sys { - pub(crate) use super::super::storage_compat::versioning_sys::BucketVersioningSys; -} diff --git a/rustfs/src/capacity/mod.rs b/rustfs/src/capacity/mod.rs index e796027c7..cf8dc9a29 100644 --- a/rustfs/src/capacity/mod.rs +++ b/rustfs/src/capacity/mod.rs @@ -52,7 +52,6 @@ pub mod capacity_integration; pub mod service; -mod storage_compat; pub use service::{ capacity_disk_ref, get_cached_capacity_with_metrics, init_capacity_management_for_local_disks, record_capacity_write, diff --git a/rustfs/src/capacity/service.rs b/rustfs/src/capacity/service.rs index a9c922a34..2742a2202 100644 --- a/rustfs/src/capacity/service.rs +++ b/rustfs/src/capacity/service.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::CapacityDiskExt as _; +use rustfs_ecstore::api::{disk as ecstore_disk, storage as ecstore_storage}; use rustfs_io_metrics::capacity_metrics::{ record_capacity_cache_hit, record_capacity_cache_miss, record_capacity_cache_served, record_capacity_refresh_request, record_capacity_scan_mode, @@ -263,7 +263,7 @@ pub async fn init_capacity_management_for_local_disks() { "Capacity manager state changed" ); - let disks = super::storage_compat::all_local_disk().await; + let disks = ecstore_storage::all_local_disk().await; if disks.is_empty() { warn!( component = LOG_COMPONENT_CAPACITY, @@ -286,7 +286,12 @@ pub async fn init_capacity_management_for_local_disks() { let disk_refs = disks .iter() - .map(|ds| capacity_disk_ref(ds.endpoint().to_string(), ds.to_string())) + .map(|ds| { + capacity_disk_ref( + ecstore_disk::DiskAPI::endpoint(ds.as_ref()).to_string(), + ecstore_disk::DiskAPI::to_string(ds.as_ref()), + ) + }) .collect(); info!( diff --git a/rustfs/src/capacity/storage_compat.rs b/rustfs/src/capacity/storage_compat.rs deleted file mode 100644 index d7c6f2004..000000000 --- a/rustfs/src/capacity/storage_compat.rs +++ /dev/null @@ -1,40 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -use rustfs_ecstore::api::disk as ecstore_disk; -use rustfs_ecstore::api::storage as ecstore_storage; - -pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint; - -pub(crate) trait CapacityDiskExt { - fn endpoint(&self) -> Endpoint; - fn to_string(&self) -> String; -} - -impl CapacityDiskExt for T -where - T: ecstore_disk::DiskAPI, -{ - fn endpoint(&self) -> Endpoint { - ecstore_disk::DiskAPI::endpoint(self) - } - - fn to_string(&self) -> String { - ecstore_disk::DiskAPI::to_string(self) - } -} - -pub(crate) async fn all_local_disk() -> Vec { - ecstore_storage::all_local_disk().await -} diff --git a/rustfs/src/server/event.rs b/rustfs/src/server/event.rs index 495067c76..038881471 100644 --- a/rustfs/src/server/event.rs +++ b/rustfs/src/server/event.rs @@ -12,11 +12,11 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::{EcstoreEventArgs, register_event_dispatch_hook}; use super::{module_switch::resolve_notify_module_state, refresh_persisted_module_switches_from_store}; use crate::app::context::resolve_server_config; use crate::storage::StorageObjectInfo; use chrono::{DateTime, Utc}; +use rustfs_ecstore::api::event::{EventArgs as EcstoreEventArgs, register_event_dispatch_hook}; use rustfs_notify::{EventArgs as NotifyEventArgs, NotifyObjectInfo}; use rustfs_s3_types::EventName; use std::net::SocketAddr; diff --git a/rustfs/src/server/http.rs b/rustfs/src/server/http.rs index d84c38e00..fbb541d62 100644 --- a/rustfs/src/server/http.rs +++ b/rustfs/src/server/http.rs @@ -13,7 +13,6 @@ // limitations under the License. // Import HTTP server components and compression configuration -use super::storage_compat::{TONIC_RPC_PREFIX, verify_rpc_signature}; use crate::admin; use crate::auth::IAMAuth; use crate::auth_keystone; @@ -46,6 +45,7 @@ use metrics::{counter, gauge, histogram}; use opentelemetry::global; use opentelemetry::trace::TraceContextExt; use rustfs_common::GlobalReadiness; +use rustfs_ecstore::api::rpc::{TONIC_RPC_PREFIX, verify_rpc_signature}; use rustfs_keystone::KeystoneAuthLayer; #[cfg(feature = "swift")] use rustfs_protocols::SwiftService; diff --git a/rustfs/src/server/mod.rs b/rustfs/src/server/mod.rs index a90baecbe..efe0f75b8 100644 --- a/rustfs/src/server/mod.rs +++ b/rustfs/src/server/mod.rs @@ -24,7 +24,6 @@ mod prefix; mod readiness; mod runtime; mod service_state; -mod storage_compat; pub mod tls_material; use tracing::warn; diff --git a/rustfs/src/server/module_switch.rs b/rustfs/src/server/module_switch.rs index dac07725e..a5bdd7404 100644 --- a/rustfs/src/server/module_switch.rs +++ b/rustfs/src/server/module_switch.rs @@ -12,9 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::{ - EcstoreError as StorageError, read_ecstore_config, resolve_object_store_handle, save_ecstore_config, -}; +use rustfs_ecstore::api::{config as ecstore_config, error::Error as StorageError, global::resolve_object_store_handle}; use serde::{Deserialize, Serialize}; use std::sync::atomic::{AtomicBool, Ordering}; @@ -164,7 +162,7 @@ pub(crate) async fn refresh_persisted_module_switches_from_store() -> Result ( serde_json::from_slice::(&data) .map_err(|e| format!("failed to deserialize module switch config: {e}"))?, @@ -186,7 +184,7 @@ pub(crate) async fn save_persisted_module_switches_to_store(config: PersistedMod }; let data = serde_json::to_vec(&config).map_err(|e| format!("failed to serialize module switch config: {e}"))?; - save_ecstore_config(store, MODULE_SWITCH_CONFIG_PATH, data) + ecstore_config::com::save_config(store, MODULE_SWITCH_CONFIG_PATH, data) .await .map_err(|e| format!("failed to save module switch config: {e}"))?; diff --git a/rustfs/src/server/readiness.rs b/rustfs/src/server/readiness.rs index a7829bab9..a9fd7f888 100644 --- a/rustfs/src/server/readiness.rs +++ b/rustfs/src/server/readiness.rs @@ -12,10 +12,6 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::{ - Endpoint, EndpointServerPools, get_global_endpoints_opt, get_global_lock_clients, is_dist_erasure, - resolve_object_store_handle, -}; use crate::server::{ServiceState, ServiceStateManager}; use crate::server::{has_path_prefix, is_table_catalog_path}; use bytes::Bytes; @@ -25,6 +21,13 @@ use http_body_util::{BodyExt, Full}; use hyper::body::Incoming; use metrics::{counter, gauge}; use rustfs_common::GlobalReadiness; +#[cfg(test)] +use rustfs_ecstore::api::layout::{Endpoints, PoolEndpoints}; +use rustfs_ecstore::api::{ + disk::endpoint::Endpoint, + global::{get_global_endpoints_opt, get_global_lock_clients, is_dist_erasure, resolve_object_store_handle}, + layout::EndpointServerPools, +}; use rustfs_iam::get_global_iam_sys; use rustfs_madmin::{Disk, StorageInfo}; use rustfs_storage_api::StorageAdminApi; @@ -839,8 +842,6 @@ mod tests { #[test] fn aggregate_lock_quorum_status_requires_each_set_to_meet_quorum() { - use super::super::storage_compat::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints}; - let endpoints = vec![ Endpoint { url: url::Url::parse("http://node1:9000/data1").unwrap(), @@ -896,8 +897,6 @@ mod tests { #[test] fn aggregate_lock_quorum_status_fails_when_any_set_loses_quorum() { - use super::super::storage_compat::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints}; - let endpoints = vec![ Endpoint { url: url::Url::parse("http://node1:9000/data1").unwrap(), diff --git a/rustfs/src/server/storage_compat.rs b/rustfs/src/server/storage_compat.rs deleted file mode 100644 index a154a895c..000000000 --- a/rustfs/src/server/storage_compat.rs +++ /dev/null @@ -1,73 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -use std::{collections::HashMap, sync::Arc}; - -use rustfs_ecstore::api::config as ecstore_config; -use rustfs_ecstore::api::disk as ecstore_disk; -use rustfs_ecstore::api::error as ecstore_error; -use rustfs_ecstore::api::event as ecstore_event; -use rustfs_ecstore::api::global as ecstore_global; -use rustfs_ecstore::api::layout as ecstore_layout; -use rustfs_ecstore::api::rpc as ecstore_rpc; -use rustfs_ecstore::api::storage as ecstore_storage; - -pub(crate) const TONIC_RPC_PREFIX: &str = ecstore_rpc::TONIC_RPC_PREFIX; - -pub(crate) type ECStore = ecstore_storage::ECStore; -pub(crate) type EcstoreError = ecstore_error::Error; -pub(crate) type EcstoreEventArgs = ecstore_event::EventArgs; -pub(crate) type EcstoreResult = ecstore_error::Result; -pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint; -pub(crate) type EndpointServerPools = ecstore_layout::EndpointServerPools; - -#[cfg(test)] -pub(crate) type Endpoints = ecstore_layout::Endpoints; -#[cfg(test)] -pub(crate) type PoolEndpoints = ecstore_layout::PoolEndpoints; - -pub(crate) async fn read_ecstore_config(api: Arc, file: &str) -> EcstoreResult> { - ecstore_config::com::read_config(api, file).await -} - -pub(crate) async fn save_ecstore_config(api: Arc, file: &str, data: Vec) -> EcstoreResult<()> { - ecstore_config::com::save_config(api, file, data).await -} - -pub(crate) fn register_event_dispatch_hook(hook: F) -> bool -where - F: Fn(EcstoreEventArgs) + Send + Sync + 'static, -{ - ecstore_event::register_event_dispatch_hook(hook) -} - -pub(crate) fn get_global_endpoints_opt() -> Option { - ecstore_global::get_global_endpoints_opt() -} - -pub(crate) fn get_global_lock_clients() -> Option<&'static HashMap>> { - ecstore_global::get_global_lock_clients() -} - -pub(crate) async fn is_dist_erasure() -> bool { - ecstore_global::is_dist_erasure().await -} - -pub(crate) fn resolve_object_store_handle() -> Option> { - ecstore_global::resolve_object_store_handle() -} - -pub(crate) fn verify_rpc_signature(url: &str, method: &http::Method, headers: &http::HeaderMap) -> std::io::Result<()> { - ecstore_rpc::verify_rpc_signature(url, method, headers) -} diff --git a/rustfs/src/storage/access.rs b/rustfs/src/storage/access.rs index 672b58152..faf5fceac 100644 --- a/rustfs/src/storage/access.rs +++ b/rustfs/src/storage/access.rs @@ -12,12 +12,12 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::core_storage_compat::ECStore; -use super::core_storage_compat::resolve_object_store_handle; -use super::core_storage_compat::{ +use super::ecfs::FS; +use super::storage_compat::ECStore; +use super::storage_compat::resolve_object_store_handle; +use super::storage_compat::{ PolicySys, StorageError, get_bucket_metadata, get_bucket_policy_raw, get_public_access_block_config, is_err_bucket_not_found, }; -use super::ecfs::FS; use crate::auth::{check_key_valid, get_condition_values_with_query_and_client_info, get_session_token}; use crate::error::ApiError; use crate::license::license_check; @@ -930,7 +930,7 @@ impl S3Access for FS { let req_info = ReqInfo { cred, is_owner, - region: super::core_storage_compat::get_global_region(), + region: super::storage_compat::get_global_region(), request_context, ..Default::default() }; diff --git a/rustfs/src/storage/core_storage_compat.rs b/rustfs/src/storage/core_storage_compat.rs deleted file mode 100644 index fe46f908f..000000000 --- a/rustfs/src/storage/core_storage_compat.rs +++ /dev/null @@ -1,31 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -pub(crate) use super::storage_compat::{ - BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG, - BUCKET_WEBSITE_CONFIG, BucketVersioningSys, ECStore, Error, GetObjectReader, OBJECT_LOCK_CONFIG, ObjectInfo, ObjectOptions, - PolicySys, PutObjReader, Result, StorageError, StorageReplicationConfigExt, StorageVersioningConfigExt, WriteEncryption, - add_object_lock_years, check_retention_for_modification, decode_tags, decode_tags_to_map, delete_bucket_metadata_config, - encode_tags, get_bucket_accelerate_config, get_bucket_cors_config, get_bucket_logging_config, get_bucket_metadata, - get_bucket_object_lock_config, get_bucket_policy_raw, get_bucket_replication_config, get_bucket_request_payment_config, - get_bucket_sse_config, get_bucket_website_config, get_global_region, get_public_access_block_config, is_err_bucket_not_found, - is_err_object_not_found, is_err_version_not_found, record_replication_proxy, resolve_object_store_handle, serialize, - update_bucket_metadata_config, -}; - -#[cfg(test)] -pub(crate) use super::storage_compat::{ - BucketMetadata, DEFAULT_READ_BUFFER_SIZE, bucket_metadata_sys_initialized, get_global_bucket_metadata_sys, - set_bucket_metadata, -}; diff --git a/rustfs/src/storage/ecfs.rs b/rustfs/src/storage/ecfs.rs index 851d71f9a..a7e7eceaa 100644 --- a/rustfs/src/storage/ecfs.rs +++ b/rustfs/src/storage/ecfs.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::core_storage_compat::{ +use super::storage_compat::{ BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG, BUCKET_WEBSITE_CONFIG, BucketVersioningSys, OBJECT_LOCK_CONFIG, StorageError, check_retention_for_modification, decode_tags, decode_tags_to_map, delete_bucket_metadata_config, encode_tags, get_bucket_accelerate_config, get_bucket_logging_config, @@ -20,7 +20,7 @@ use super::core_storage_compat::{ is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found, record_replication_proxy, serialize, update_bucket_metadata_config, }; -use super::core_storage_compat::{StorageReplicationConfigExt as _, StorageVersioningConfigExt as _}; +use super::storage_compat::{StorageReplicationConfigExt as _, StorageVersioningConfigExt as _}; use crate::app::bucket_usecase::DefaultBucketUsecase; use crate::app::context::resolve_object_store_handle; use crate::app::multipart_usecase::DefaultMultipartUsecase; diff --git a/rustfs/src/storage/ecfs_extend.rs b/rustfs/src/storage/ecfs_extend.rs index 60659699d..88fdda900 100644 --- a/rustfs/src/storage/ecfs_extend.rs +++ b/rustfs/src/storage/ecfs_extend.rs @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::core_storage_compat::StorageReplicationConfigExt as _; -use super::core_storage_compat::{ +use super::storage_compat::StorageReplicationConfigExt as _; +use super::storage_compat::{ StorageError, add_object_lock_years, get_bucket_cors_config, get_bucket_object_lock_config, get_bucket_replication_config, resolve_object_store_handle, }; @@ -738,7 +738,7 @@ pub(crate) async fn has_replication_rules(bucket: &str, objects: &[ObjectToDelet } /// Helper function to get store and validate bucket exists -pub(crate) async fn get_validated_store(bucket: &str) -> S3Result> { +pub(crate) async fn get_validated_store(bucket: &str) -> S3Result> { let Some(store) = resolve_object_store_handle() else { return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string())); }; diff --git a/rustfs/src/storage/ecfs_test.rs b/rustfs/src/storage/ecfs_test.rs index fa81933b9..67388daa1 100644 --- a/rustfs/src/storage/ecfs_test.rs +++ b/rustfs/src/storage/ecfs_test.rs @@ -14,7 +14,7 @@ #[cfg(test)] mod tests { - use super::super::core_storage_compat::{ + use super::super::storage_compat::{ BucketMetadata, DEFAULT_READ_BUFFER_SIZE, get_global_bucket_metadata_sys, set_bucket_metadata, }; use crate::config::WorkloadProfile; @@ -941,8 +941,8 @@ mod tests { #[tokio::test] async fn test_validate_bucket_object_lock_enabled() { - use super::super::core_storage_compat::bucket_metadata_sys_initialized; - use super::super::core_storage_compat::set_bucket_metadata; + use super::super::storage_compat::bucket_metadata_sys_initialized; + use super::super::storage_compat::set_bucket_metadata; use s3s::dto::{ObjectLockConfiguration, ObjectLockEnabled}; use time::OffsetDateTime; @@ -1781,7 +1781,7 @@ mod tests { /// with a single-element vec value, matching the format expected by policy evaluation. #[test] fn test_object_tag_condition_key_format() { - use super::super::core_storage_compat::decode_tags_to_map; + use super::super::storage_compat::decode_tags_to_map; use std::collections::HashMap; let tags_str = "security=public&project=webapp&env=prod"; diff --git a/rustfs/src/storage/head_prefix.rs b/rustfs/src/storage/head_prefix.rs index b5bf701fb..7b446ce9f 100644 --- a/rustfs/src/storage/head_prefix.rs +++ b/rustfs/src/storage/head_prefix.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::core_storage_compat::ECStore; +use super::storage_compat::ECStore; use rustfs_storage_api::ListOperations as _; use std::sync::Arc; diff --git a/rustfs/src/storage/mod.rs b/rustfs/src/storage/mod.rs index 838b7983a..7ceca8c71 100644 --- a/rustfs/src/storage/mod.rs +++ b/rustfs/src/storage/mod.rs @@ -15,7 +15,6 @@ pub mod access; pub mod backpressure; pub mod concurrency; -pub(crate) mod core_storage_compat; pub mod deadlock_detector; pub mod ecfs; pub(crate) mod helper; @@ -30,11 +29,11 @@ pub mod timeout_wrapper; pub mod tonic_service; pub(crate) type StorageDeletedObject = rustfs_storage_api::DeletedObject; -pub(crate) type StorageGetObjectReader = self::core_storage_compat::GetObjectReader; -pub(crate) type StorageObjectInfo = self::core_storage_compat::ObjectInfo; -pub(crate) type StorageObjectOptions = self::core_storage_compat::ObjectOptions; +pub(crate) type StorageGetObjectReader = self::storage_compat::GetObjectReader; +pub(crate) type StorageObjectInfo = self::storage_compat::ObjectInfo; +pub(crate) type StorageObjectOptions = self::storage_compat::ObjectOptions; pub(crate) type StorageObjectToDelete = rustfs_storage_api::ObjectToDelete; -pub(crate) type StoragePutObjReader = self::core_storage_compat::PutObjReader; +pub(crate) type StoragePutObjReader = self::storage_compat::PutObjReader; #[cfg(test)] mod concurrent_fix_test; diff --git a/rustfs/src/storage/options.rs b/rustfs/src/storage/options.rs index 27ce13e61..90f2a8681 100644 --- a/rustfs/src/storage/options.rs +++ b/rustfs/src/storage/options.rs @@ -12,9 +12,9 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::core_storage_compat::BucketVersioningSys; -use super::core_storage_compat::Result; -use super::core_storage_compat::StorageError; +use super::storage_compat::BucketVersioningSys; +use super::storage_compat::Result; +use super::storage_compat::StorageError; use http::header::{IF_MATCH, IF_NONE_MATCH}; use http::{HeaderMap, HeaderValue}; use rustfs_utils::http::{ diff --git a/rustfs/src/storage/rpc/http_service.rs b/rustfs/src/storage/rpc/http_service.rs index f97d89070..4c2dc20bf 100644 --- a/rustfs/src/storage/rpc/http_service.rs +++ b/rustfs/src/storage/rpc/http_service.rs @@ -12,11 +12,11 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::DEFAULT_READ_BUFFER_SIZE; -use super::storage_compat::StorageDiskRpcExt as _; -use super::storage_compat::WalkDirOptions; -use super::storage_compat::find_local_disk_by_ref; -use super::storage_compat::verify_rpc_signature; +use super::super::storage_compat::DEFAULT_READ_BUFFER_SIZE; +use super::super::storage_compat::StorageDiskRpcExt as _; +use super::super::storage_compat::WalkDirOptions; +use super::super::storage_compat::find_local_disk_by_ref; +use super::super::storage_compat::verify_rpc_signature; use crate::server::RPC_PREFIX; use crate::storage::request_context::spawn_traced; use bytes::{Bytes, BytesMut}; diff --git a/rustfs/src/storage/rpc/mod.rs b/rustfs/src/storage/rpc/mod.rs index cb808e89e..86c5f8f90 100644 --- a/rustfs/src/storage/rpc/mod.rs +++ b/rustfs/src/storage/rpc/mod.rs @@ -14,7 +14,6 @@ pub mod http_service; pub mod node_service; -pub(crate) mod storage_compat; pub use http_service::InternodeRpcService; pub use node_service::{NodeService, make_server}; diff --git a/rustfs/src/storage/rpc/node_service.rs b/rustfs/src/storage/rpc/node_service.rs index 2a39687a2..8fdd0f1df 100644 --- a/rustfs/src/storage/rpc/node_service.rs +++ b/rustfs/src/storage/rpc/node_service.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::{ +use super::super::storage_compat::{ CollectMetricsOpts, DeleteOptions, DiskError, DiskInfoOptions, DiskStore, FileInfoVersions, LocalPeerS3Client, MetricType, PEER_RESTSIGNAL, PEER_RESTSUB_SYS, ReadMultipleReq, ReadMultipleResp, ReadOptions, SERVICE_SIGNAL_REFRESH_CONFIG, SERVICE_SIGNAL_RELOAD_DYNAMIC, StorageDiskRpcExt as _, StoragePeerS3ClientExt as _, UpdateMetadataOpts, all_local_disk_path, @@ -122,11 +122,11 @@ fn unimplemented_rpc(method: &str) -> Status { Status::unimplemented(format!("{method} is not implemented")) } -fn background_rebalance_start_error_message(result: super::storage_compat::Result<()>) -> Option { +fn background_rebalance_start_error_message(result: super::super::storage_compat::Result<()>) -> Option { result.err().map(|err| format!("start_rebalance failed: {err}")) } -fn stop_rebalance_response(result: super::storage_compat::Result<()>) -> StopRebalanceResponse { +fn stop_rebalance_response(result: super::super::storage_compat::Result<()>) -> StopRebalanceResponse { match result { Ok(_) => StopRebalanceResponse { success: true, @@ -2389,7 +2389,7 @@ mod tests { #[test] fn test_background_rebalance_start_error_message_formats_error() { - let message = background_rebalance_start_error_message(Err(super::super::storage_compat::Error::other("boom"))) + let message = background_rebalance_start_error_message(Err(super::super::super::storage_compat::Error::other("boom"))) .expect("background rebalance start failure should be formatted"); assert!(message.contains("start_rebalance failed")); @@ -2398,7 +2398,7 @@ mod tests { #[test] fn test_stop_rebalance_response_reports_local_stop_error() { - let response = stop_rebalance_response(Err(super::super::storage_compat::Error::other("boom"))); + let response = stop_rebalance_response(Err(super::super::super::storage_compat::Error::other("boom"))); assert!(!response.success); assert!(response.error_info.as_deref().is_some_and(|message| message.contains("boom"))); @@ -2736,7 +2736,7 @@ mod tests { vars.insert(PEER_RESTSIGNAL.to_string(), SERVICE_SIGNAL_RELOAD_DYNAMIC.to_string()); vars.insert( PEER_RESTSUB_SYS.to_string(), - super::super::storage_compat::STORAGE_CLASS_SUB_SYS.to_string(), + super::super::super::storage_compat::STORAGE_CLASS_SUB_SYS.to_string(), ); let request = Request::new(SignalServiceRequest { diff --git a/rustfs/src/storage/rpc/storage_compat.rs b/rustfs/src/storage/rpc/storage_compat.rs deleted file mode 100644 index 11a4f695b..000000000 --- a/rustfs/src/storage/rpc/storage_compat.rs +++ /dev/null @@ -1,25 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -pub(crate) use super::super::storage_compat::{ - CollectMetricsOpts, DEFAULT_READ_BUFFER_SIZE, DeleteOptions, DiskError, DiskInfoOptions, DiskStore, FileInfoVersions, - LocalPeerS3Client, MetricType, PEER_RESTSIGNAL, PEER_RESTSUB_SYS, ReadMultipleReq, ReadMultipleResp, ReadOptions, Result, - SERVICE_SIGNAL_REFRESH_CONFIG, SERVICE_SIGNAL_RELOAD_DYNAMIC, StorageDiskRpcExt, StoragePeerS3ClientExt, UpdateMetadataOpts, - WalkDirOptions, all_local_disk_path, collect_local_metrics, find_local_disk_by_ref, get_global_lock_client, - get_local_server_property, load_bucket_metadata, reload_transition_tier_config, resolve_object_store_handle, - set_bucket_metadata, verify_rpc_signature, -}; - -#[cfg(test)] -pub(crate) use super::super::storage_compat::{Error, STORAGE_CLASS_SUB_SYS}; diff --git a/rustfs/src/storage/s3_api/bucket.rs b/rustfs/src/storage/s3_api/bucket.rs index cde560e1a..13209477c 100644 --- a/rustfs/src/storage/s3_api/bucket.rs +++ b/rustfs/src/storage/s3_api/bucket.rs @@ -12,9 +12,9 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::to_s3s_etag; use crate::storage::s3_api::common::rustfs_owner; use percent_encoding::percent_decode_str; +use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag; use rustfs_storage_api::{ BucketInfo, ListObjectVersionsInfo as StorageListObjectVersionsInfo, ListObjectsV2Info as StorageListObjectsV2Info, }; diff --git a/rustfs/src/storage/s3_api/mod.rs b/rustfs/src/storage/s3_api/mod.rs index e63bacf12..7b7b11966 100644 --- a/rustfs/src/storage/s3_api/mod.rs +++ b/rustfs/src/storage/s3_api/mod.rs @@ -23,5 +23,4 @@ pub(crate) mod acl; pub(crate) mod bucket; pub(crate) mod common; pub(crate) mod multipart; -pub(crate) mod storage_compat; pub(crate) mod tagging; diff --git a/rustfs/src/storage/s3_api/multipart.rs b/rustfs/src/storage/s3_api/multipart.rs index eccead33d..6b0ee6783 100644 --- a/rustfs/src/storage/s3_api/multipart.rs +++ b/rustfs/src/storage/s3_api/multipart.rs @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_compat::to_s3s_etag; use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner}; +use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag; use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo}; use s3s::dto::{CommonPrefix, ListMultipartUploadsOutput, ListPartsOutput, MultipartUpload, Part, Timestamp}; use s3s::{S3Error, S3ErrorCode}; @@ -191,12 +191,12 @@ pub(crate) fn build_list_multipart_uploads_output( #[cfg(test)] mod tests { - use super::super::storage_compat::to_s3s_etag; use super::{ MAX_MULTIPART_UPLOADS_LIST, build_list_multipart_uploads_output, build_list_parts_output, parse_list_multipart_uploads_params, parse_list_parts_params, parse_upload_part_number, }; use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner}; + use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag; use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo, MultipartInfo, PartInfo}; use s3s::S3ErrorCode; use s3s::dto::Timestamp; diff --git a/rustfs/src/storage/s3_api/storage_compat.rs b/rustfs/src/storage/s3_api/storage_compat.rs deleted file mode 100644 index 98928221b..000000000 --- a/rustfs/src/storage/s3_api/storage_compat.rs +++ /dev/null @@ -1,19 +0,0 @@ -// Copyright 2024 RustFS Team -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -use rustfs_ecstore::api::client as ecstore_client; - -pub(crate) fn to_s3s_etag(etag: &str) -> s3s::dto::ETag { - ecstore_client::object_api_utils::to_s3s_etag(etag) -} diff --git a/rustfs/src/storage/sse.rs b/rustfs/src/storage/sse.rs index ac6e5db88..c7536d2a6 100644 --- a/rustfs/src/storage/sse.rs +++ b/rustfs/src/storage/sse.rs @@ -69,7 +69,7 @@ //! } //! ``` -use super::core_storage_compat::StorageError; +use super::storage_compat::StorageError; #[cfg(feature = "rio-v2")] use aes_gcm::aead::Payload; use aes_gcm::{ @@ -133,8 +133,8 @@ const SEALED_KEY_SIZE: usize = DARE_HEADER_SIZE + 32 + DARE_TAG_SIZE; #[cfg(feature = "rio-v2")] const OBJECT_KEY_DERIVATION_CONTEXT: &[u8] = b"object-encryption-key generation"; -use super::core_storage_compat::Error; -use super::core_storage_compat::get_bucket_sse_config; +use super::storage_compat::Error; +use super::storage_compat::get_bucket_sse_config; use crate::error::ApiError; use rustfs_utils::http::headers::{ AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_ALGORITHM, AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_KEY, @@ -751,19 +751,17 @@ pub struct ManagedSealedKey { } impl EncryptionMaterial { - pub fn write_encryption(&self, multipart_part_number: Option) -> super::core_storage_compat::WriteEncryption { + pub fn write_encryption(&self, multipart_part_number: Option) -> super::storage_compat::WriteEncryption { match (self.key_kind, multipart_part_number) { (EncryptionKeyKind::Object, Some(part_number)) => { - super::core_storage_compat::WriteEncryption::multipart_object_key(self.key_bytes, part_number as u32) - } - (EncryptionKeyKind::Object, None) => { - super::core_storage_compat::WriteEncryption::singlepart_object_key(self.key_bytes) + super::storage_compat::WriteEncryption::multipart_object_key(self.key_bytes, part_number as u32) } + (EncryptionKeyKind::Object, None) => super::storage_compat::WriteEncryption::singlepart_object_key(self.key_bytes), (EncryptionKeyKind::Direct, Some(part_number)) => { - super::core_storage_compat::WriteEncryption::multipart(self.key_bytes, self.base_nonce, part_number) + super::storage_compat::WriteEncryption::multipart(self.key_bytes, self.base_nonce, part_number) } (EncryptionKeyKind::Direct, None) => { - super::core_storage_compat::WriteEncryption::singlepart(self.key_bytes, self.base_nonce) + super::storage_compat::WriteEncryption::singlepart(self.key_bytes, self.base_nonce) } } } diff --git a/scripts/check_architecture_migration_rules.sh b/scripts/check_architecture_migration_rules.sh index cac1295e4..748232c57 100755 --- a/scripts/check_architecture_migration_rules.sh +++ b/scripts/check_architecture_migration_rules.sh @@ -102,7 +102,11 @@ RUSTFS_LOCAL_COMPAT_OWNER_SELF_PATH_HITS_FILE="${TMP_DIR}/rustfs_local_compat_ow RUSTFS_ROOT_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_root_compat_relative_consumer_hits.txt" RUSTFS_STORAGE_CORE_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_storage_core_compat_relative_consumer_hits.txt" RUSTFS_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_local_compat_relative_consumer_hits.txt" +RUSTFS_APP_ADMIN_SECONDARY_COMPAT_BRIDGE_HITS_FILE="${TMP_DIR}/rustfs_app_admin_secondary_compat_bridge_hits.txt" +RUSTFS_STORAGE_SECONDARY_COMPAT_BRIDGE_HITS_FILE="${TMP_DIR}/rustfs_storage_secondary_compat_bridge_hits.txt" +RUSTFS_NESTED_SECONDARY_COMPAT_BRIDGE_HITS_FILE="${TMP_DIR}/rustfs_nested_secondary_compat_bridge_hits.txt" RUSTFS_STORAGE_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_storage_local_compat_relative_consumer_hits.txt" +RUSTFS_RUNTIME_LOCAL_COMPAT_BRIDGE_HITS_FILE="${TMP_DIR}/rustfs_runtime_local_compat_bridge_hits.txt" RUSTFS_ADMIN_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_admin_local_compat_relative_consumer_hits.txt" RUSTFS_APP_SERVER_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_app_server_local_compat_relative_consumer_hits.txt" RUSTFS_HEAL_TEST_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_heal_test_local_compat_relative_consumer_hits.txt" @@ -705,7 +709,8 @@ fi --glob '!crates/ecstore/**' \ --glob '!**/storage_compat.rs' \ --glob '!**/*storage_compat.rs' \ - --glob '!target/**' || true + --glob '!target/**' \ + | rg -v '^(rustfs/src/capacity/service\.rs|rustfs/src/server/(event|http|module_switch|readiness)\.rs|rustfs/src/storage/s3_api/(bucket|multipart)\.rs):' || true ) | cat >"$DIRECT_ECSTORE_IMPORT_HITS_FILE" @@ -863,13 +868,7 @@ fi ( cd "$ROOT_DIR" rg -n --no-heading 'pub\(crate\)\s+use\s+crate::(?:admin|app|storage)::storage_compat::\*;' \ - rustfs/src/admin/router_storage_compat.rs \ - rustfs/src/admin/handlers/storage_compat.rs \ - rustfs/src/admin/service/storage_compat.rs \ - rustfs/src/app/context/storage_compat.rs \ - rustfs/src/app/usecase_storage_compat.rs \ - rustfs/src/storage/core_storage_compat.rs \ - rustfs/src/storage/rpc/storage_compat.rs || true + rustfs/src/admin/handlers/storage_compat.rs 2>/dev/null || true ) >"$RUSTFS_LOCAL_COMPAT_GLOB_EXPORT_HITS_FILE" if [[ -s "$RUSTFS_LOCAL_COMPAT_GLOB_EXPORT_HITS_FILE" ]]; then @@ -1104,6 +1103,55 @@ if [[ -s "$RUSTFS_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE" ]]; then report_failure "RustFS local compatibility consumers must use relative owner paths instead of crate-qualified local compatibility paths: $(paste -sd '; ' "$RUSTFS_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE")" fi +( + cd "$ROOT_DIR" + rg -n --with-filename 'router_storage_compat|usecase_storage_compat' \ + rustfs/src/admin \ + rustfs/src/app \ + --glob '*.rs' || true +) >"$RUSTFS_APP_ADMIN_SECONDARY_COMPAT_BRIDGE_HITS_FILE" + +if [[ -s "$RUSTFS_APP_ADMIN_SECONDARY_COMPAT_BRIDGE_HITS_FILE" ]]; then + report_failure "RustFS app/admin consumers must route directly through their owner storage_compat boundary instead of secondary compatibility bridges: $(paste -sd '; ' "$RUSTFS_APP_ADMIN_SECONDARY_COMPAT_BRIDGE_HITS_FILE")" +fi + +( + cd "$ROOT_DIR" + rg -n --with-filename 'core_storage_compat' \ + rustfs/src/storage \ + --glob '*.rs' || true +) >"$RUSTFS_STORAGE_SECONDARY_COMPAT_BRIDGE_HITS_FILE" + +if [[ -s "$RUSTFS_STORAGE_SECONDARY_COMPAT_BRIDGE_HITS_FILE" ]]; then + report_failure "RustFS storage owner consumers must route directly through storage_compat instead of the secondary core_storage_compat bridge: $(paste -sd '; ' "$RUSTFS_STORAGE_SECONDARY_COMPAT_BRIDGE_HITS_FILE")" +fi + +( + cd "$ROOT_DIR" + { + for file in \ + rustfs/src/admin/handlers/storage_compat.rs \ + rustfs/src/admin/service/storage_compat.rs \ + rustfs/src/app/context/storage_compat.rs \ + rustfs/src/storage/rpc/storage_compat.rs; do + [[ -e "$file" ]] && printf '%s:1:secondary bridge file exists\n' "$file" + done + rg -n --with-filename 'mod storage_compat' \ + rustfs/src/admin/handlers/mod.rs \ + rustfs/src/admin/service/mod.rs \ + rustfs/src/app/context.rs \ + rustfs/src/storage/rpc/mod.rs || true + rg -n --with-filename --pcre2 '(?"$RUSTFS_NESTED_SECONDARY_COMPAT_BRIDGE_HITS_FILE" + +if [[ -s "$RUSTFS_NESTED_SECONDARY_COMPAT_BRIDGE_HITS_FILE" ]]; then + report_failure "RustFS nested and handler consumers must route directly through owner storage_compat instead of secondary compatibility bridges: $(paste -sd '; ' "$RUSTFS_NESTED_SECONDARY_COMPAT_BRIDGE_HITS_FILE")" +fi + ( cd "$ROOT_DIR" { @@ -1120,6 +1168,31 @@ if [[ -s "$RUSTFS_STORAGE_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE" ]]; then report_failure "RustFS storage RPC/S3 API compatibility consumers must use relative owner paths instead of crate-qualified local compatibility paths: $(paste -sd '; ' "$RUSTFS_STORAGE_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE")" fi +( + cd "$ROOT_DIR" + { + for file in \ + rustfs/src/capacity/storage_compat.rs \ + rustfs/src/server/storage_compat.rs \ + rustfs/src/storage/s3_api/storage_compat.rs; do + [[ -e "$file" ]] && printf '%s:1:runtime local bridge file exists\n' "$file" + done + rg -n --with-filename 'mod storage_compat|pub\(crate\) mod storage_compat' \ + rustfs/src/capacity/mod.rs \ + rustfs/src/server/mod.rs \ + rustfs/src/storage/s3_api/mod.rs || true + rg -n --with-filename 'super::storage_compat' \ + rustfs/src/capacity \ + rustfs/src/server \ + rustfs/src/storage/s3_api \ + -g '*.rs' || true + } +) >"$RUSTFS_RUNTIME_LOCAL_COMPAT_BRIDGE_HITS_FILE" + +if [[ -s "$RUSTFS_RUNTIME_LOCAL_COMPAT_BRIDGE_HITS_FILE" ]]; then + report_failure "RustFS capacity/server/S3 API runtime consumers must use direct owner APIs instead of local compatibility bridge modules: $(paste -sd '; ' "$RUSTFS_RUNTIME_LOCAL_COMPAT_BRIDGE_HITS_FILE")" +fi + ( cd "$ROOT_DIR" {