diff --git a/docs/architecture/migration-progress.md b/docs/architecture/migration-progress.md index be8bb2370..a447796a9 100644 --- a/docs/architecture/migration-progress.md +++ b/docs/architecture/migration-progress.md @@ -5,15 +5,15 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ## Current Context - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) -- Branch: `overtrue/arch-storage-ecfs-usecase-boundary` -- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215/API-216/API-217/API-218/API-219/API-220/API-221/API-222/API-223/API-224/API-225/API-226/API-227/API-228/API-229/CTX-002`. -- Based on: stacked on API-232 local branch while API-230 PR #3894 is pending; - branch routes storage ECFS S3 route app usecase construction through the - storage S3 API boundary. +- Branch: `overtrue/arch-root-storage-api-domain-boundary` +- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215/API-216/API-217/API-218/API-219/API-220/API-221/API-222/API-223/API-224/API-225/API-226/API-227/API-228/API-229/API-230/API-231/API-232/API-233/API-234/API-235/API-236/CTX-002`. +- Based on: PR #3899 has merged; branch segments the root-local and admin-local + storage API boundaries by consumer domain. - PR type for this branch: `consumer-migration` -- Runtime behavior changes: none expected for API-233; storage ECFS S3 routes - still construct the same AppContext-backed bucket, multipart, and object - usecases. +- Runtime behavior changes: none expected for API-237/API-238; root, server, + startup, table, protocol, cluster, capacity, workload, config-test, error, + admin handler, admin service, and admin router consumers still use the same + owner symbols through narrower domain modules. - Rust code changes: route replication pool, outbound TLS generation, runtime region, KMS encryption service, runtime support handles, S3 Select DB, internode RPC metrics, IAM authorization/handler reads, notification @@ -70,8 +70,13 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ECStore internal storage contract imports through the owner-local `storage_api_contracts` boundary, admin system, pool, cluster snapshot, plugin catalog, table catalog, module-switch, and console admin discovery - `DefaultAdminUsecase` construction through `admin::runtime_sources`, and - storage ECFS S3 route app usecase construction through `storage::s3_api`. + `DefaultAdminUsecase` construction through `admin::runtime_sources`, storage + ECFS S3 route app usecase construction through `storage::s3_api`, root + storage API consumers through domain modules for startup, server, cluster, + table, protocols, capacity, workload, config tests, and error mapping, and + admin storage API consumers through admin domain modules for access, bucket, + cluster, config, contract, error, metrics, object, rebalance, runtime, and + tier boundaries. - CI/script changes: lock completed owner and test/fuzz boundaries against bare/glob imports, scattered raw ECStore facade subpaths, and startup runtime/root-server/table/S3/app shared/app bucket/app ECStore/admin facade @@ -81,10 +86,10 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block event-bridge thin module regressions, plus IAM runtime-source bypasses; accept the reviewed AppContext resolver reverse dependencies in the layer baseline, and block direct admin AppContext resolver consumers outside the - admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, reject app usecase storage wildcard imports, reject app-layer S3 DTO and ECFS wildcard imports, narrow the object-usecase ECFS layer baseline entry to `FS`, reject direct storage S3 API helper imports from app usecase files, reject direct storage helper imports from app select/usecase files, reject completed app/admin storage helper bypasses, reject app usecase bypasses for migrated storage IO/compression/set-disk helpers, reject app usecase/test bypasses for migrated storage error, ETag, and storage-class helpers, reject app root bucket owner facade bypasses from migrated app consumers, reject app/admin runtime/data-usage root facade regressions, reject admin root storage facade regressions from migrated admin consumers, reject root/server/startup direct storage facade regressions from migrated outer consumers, reject root/server/startup direct storage contract imports from migrated outer consumers, reject app/admin direct storage contract imports from migrated owner consumers, keep app S3 helper imports routed through `app::storage_api`, reject scanner/heal direct ECStore or storage contract imports outside their local `storage_api` boundaries, reject external runtime/test/fuzz ECStore or storage contract imports outside their local `storage_api` boundaries, reject storage owner direct ECStore/storage-api imports outside the owner-local `storage_api` boundary, reject ECStore internal direct storage-api imports outside the owner-local `storage_api_contracts` boundary, and reject direct storage ECFS app usecase construction outside the storage S3 API boundary. -- Docs changes: record the API-136 through API-226 owner facade and lifecycle - runtime-source cleanup plus API-233 storage ECFS usecase construction - boundary. + admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, reject app usecase storage wildcard imports, reject app-layer S3 DTO and ECFS wildcard imports, narrow the object-usecase ECFS layer baseline entry to `FS`, reject direct storage S3 API helper imports from app usecase files, reject direct storage helper imports from app select/usecase files, reject completed app/admin storage helper bypasses, reject app usecase bypasses for migrated storage IO/compression/set-disk helpers, reject app usecase/test bypasses for migrated storage error, ETag, and storage-class helpers, reject app root bucket owner facade bypasses from migrated app consumers, reject app/admin runtime/data-usage root facade regressions, reject admin root storage facade regressions from migrated admin consumers, reject root/server/startup direct storage facade regressions from migrated outer consumers, reject root/server/startup direct storage contract imports from migrated outer consumers, reject app/admin direct storage contract imports from migrated owner consumers, keep app S3 helper imports routed through `app::storage_api`, reject scanner/heal direct ECStore or storage contract imports outside their local `storage_api` boundaries, reject external runtime/test/fuzz ECStore or storage contract imports outside their local `storage_api` boundaries, reject storage owner direct ECStore/storage-api imports outside the owner-local `storage_api` boundary, reject ECStore internal direct storage-api imports outside the owner-local `storage_api_contracts` boundary, reject direct storage ECFS app usecase construction outside the storage S3 API boundary, reject flat root `storage_api` imports outside the new root-local domain modules, and reject flat admin `storage_api` imports outside the new admin domain modules. +- Docs changes: record the API-136 through API-238 owner facade, + runtime-source, ECFS usecase, root storage API domain-boundary, and admin + storage API domain-boundary cleanup. ## Phase 0 Tasks @@ -5454,15 +5459,56 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block layer guards, diff hygiene, residual owner runtime-source AppContext scan, Rust risk scan, and full PR gate before PR. +- [x] `API-237` Segment root storage API boundary by outer runtime domain. + - Do: replace the flat root-local `storage_api` re-export surface with + domain modules for startup, server, cluster, table, protocols, capacity, + workload, config tests, and error mapping, then migrate root/server/startup + consumers to those modules. + - Acceptance: root consumers no longer import flat `crate::storage_api` + symbols or legacy nested helper modules directly, storage contracts are + still imported once through the root boundary, and migration rules reject + flat root `storage_api` bypasses. + - Must preserve: startup storage/bootstrap behavior, notification config + wiring, HTTP/gRPC request context handling, readiness checks, cluster and + topology snapshots, table catalog object I/O contracts, workload admission, + capacity reporting, protocol request setup, config tests, and error mapping. + - Verification: focused RustFS compile, formatting, migration and layer + guards, diff hygiene, residual flat root storage-api scan, Rust risk scan, + and full PR gate passed before PR. + +- [x] `API-238` Segment admin storage API boundary by admin consumer domain. + - Do: add admin-local domain modules for access, bucket, cluster, config, + contract, error, metrics, object, rebalance, runtime, and tier symbols, + then migrate admin handlers, services, router, and console consumers away + from flat `admin::storage_api` imports. + - Acceptance: admin consumers no longer import flat storage facade symbols, + storage contracts, config helpers, bucket helper modules, runtime handles, + rebalance types, tier errors, or request helpers directly from the admin + storage API root; migration rules reject the old flat paths. + - Must preserve: admin route contracts, authorization flow, config + persistence, bucket metadata operations, site replication serialization, + remote-target validation, quota/heal/object-zip behavior, metrics + collection, rebalance response mapping, tier error behavior, and admin test + fixtures. + - Verification: focused RustFS compile, formatting, migration and layer + guards, diff hygiene, residual flat admin storage-api scan, Rust risk scan, + and full PR gate passed before PR. + ## Next PRs -1. `consumer-migration`: continue larger app/runtime global-source batches - after API-236. +1. `consumer-migration`: continue larger root and owner boundary batches after + API-238. ## Pre-Push Review Log | Expert | Status | Notes | |---|---|---| +| Quality/architecture | pass | API-238 segments the admin-local storage API boundary into admin consumer domain modules instead of a flat re-export surface. | +| Migration preservation | pass | Admin handlers, services, router, console paths, config persistence, bucket metadata, replication, quota, heal, metrics, rebalance, tier, and object zip consumers keep the same owner symbols and call paths. | +| Testing/verification | pass | RustFS focused compile, formatting, migration/layer guards, residual flat admin storage-api scan, diff hygiene, diff-added Rust risk scan, and full PR gate passed before PR. | +| Quality/architecture | pass | API-237 segments the root-local storage API boundary into domain modules instead of a flat re-export surface. | +| Migration preservation | pass | Startup, server, cluster, table catalog, protocol, capacity, workload, config-test, and error-mapping consumers keep the same owner symbols and call paths. | +| Testing/verification | pass | RustFS focused compile, formatting, migration/layer guards, residual flat root storage-api scan, diff hygiene, diff-added Rust risk scan, and full PR gate passed before PR. | | Quality/architecture | pass | API-236 makes admin, app, and storage runtime sources consume root runtime-source entrypoints instead of importing app context directly. | | Migration preservation | pass | Admin/object usecase construction, app explicit-context fallback, storage runtime reads, IAM/KMS/TLS resolver behavior, notification dispatch, and test TLS hooks keep the same semantics. | | Testing/verification | pass | Focused admin/app/storage checks, formatting, migration/layer guards, residual owner runtime-source AppContext scan, diff hygiene, Rust risk scan, and full PR gate passed before PR. | @@ -5712,6 +5758,27 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block Passed before push: +- Issue #660 API-238 current slice: + - Branch freshness check: rebased onto current `origin/main` after PR #3899 merged. + - `cargo check -p rustfs`: passed. + - `make pre-pr`: passed. + +- Issue #660 API-237 current slice: + - Branch freshness check: rebased onto current `origin/main` after PR #3899 merged. + - `cargo check -p rustfs`: passed. + - `cargo fmt --all`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_layer_dependencies.sh`: passed. + - Flat root storage-api residual scan: passed; root consumers now use + `storage_api` domain modules instead of flat imports or legacy helper + modules. + - Diff-added Rust risk scan: passed; matches were import aliases only, with + no new production unwrap/expect, numeric cast, String error, Box dyn Error, + print macro, or relaxed atomic ordering lines. + - `make pre-pr`: passed. + - Issue #660 API-236 current slice: - Branch freshness check: rebased onto current `origin/main` after API-232 merged. diff --git a/rustfs/src/admin/console.rs b/rustfs/src/admin/console.rs index 86d12863e..c9721cc2b 100644 --- a/rustfs/src/admin/console.rs +++ b/rustfs/src/admin/console.rs @@ -14,7 +14,7 @@ use crate::admin::handlers::health::{HealthProbe, build_health_response_parts, collect_dependency_readiness}; use crate::admin::runtime_sources::{default_admin_usecase, resolve_oidc_handle}; -use crate::admin::storage_api::RequestContext; +use crate::admin::storage_api::access::RequestContext; use crate::license::has_valid_license; use crate::server::has_path_prefix; use crate::server::{ diff --git a/rustfs/src/admin/handlers/account_info.rs b/rustfs/src/admin/handlers/account_info.rs index 455777684..5ab3b3317 100644 --- a/rustfs/src/admin/handlers/account_info.rs +++ b/rustfs/src/admin/handlers/account_info.rs @@ -15,8 +15,8 @@ use crate::admin::auth::authenticate_request; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::admin::runtime_sources::{resolve_action_credentials, resolve_object_store_handle}; -use crate::admin::storage_api::versioning_sys::BucketVersioningSys; -use crate::admin::storage_api::{BucketOperations, BucketOptions, StorageAdminApi}; +use crate::admin::storage_api::bucket::versioning_sys::BucketVersioningSys; +use crate::admin::storage_api::contract::{BucketOperations, BucketOptions, StorageAdminApi}; use crate::auth::get_condition_values; use crate::server::{ADMIN_PREFIX, RemoteAddr}; use http::{HeaderMap, HeaderValue}; diff --git a/rustfs/src/admin/handlers/audit_runtime_config.rs b/rustfs/src/admin/handlers/audit_runtime_config.rs index 0920882fe..7c7c94cdf 100644 --- a/rustfs/src/admin/handlers/audit_runtime_config.rs +++ b/rustfs/src/admin/handlers/audit_runtime_config.rs @@ -14,7 +14,7 @@ use crate::admin::handlers::target_descriptor::AdminTargetSpec; use crate::admin::runtime_sources::resolve_object_store_handle; -use crate::admin::storage_api::{read_admin_config_without_migrate, save_admin_server_config}; +use crate::admin::storage_api::config::{read_admin_config_without_migrate, save_admin_server_config}; use rustfs_audit::{audit_system, start_audit_system as start_global_audit_system, system::AuditSystemState}; use rustfs_config::DEFAULT_DELIMITER; use rustfs_config::server_config::Config; diff --git a/rustfs/src/admin/handlers/bucket_meta.rs b/rustfs/src/admin/handlers/bucket_meta.rs index ff5318317..a4ed995a0 100644 --- a/rustfs/src/admin/handlers/bucket_meta.rs +++ b/rustfs/src/admin/handlers/bucket_meta.rs @@ -12,10 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::admin::storage_api::ecstore_utils::{deserialize, serialize}; -use crate::admin::storage_api::{BucketOperations, BucketOptions, MakeBucketOptions}; -use crate::admin::storage_api::{ - StorageError, +use crate::admin::storage_api::bucket::utils::{deserialize, serialize}; +use crate::admin::storage_api::bucket::{ metadata::{ BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, @@ -25,6 +23,8 @@ use crate::admin::storage_api::{ quota::BucketQuota, target::BucketTargets, }; +use crate::admin::storage_api::contract::{BucketOperations, BucketOptions, MakeBucketOptions}; +use crate::admin::storage_api::error::StorageError; use crate::{ admin::runtime_sources::resolve_object_store_handle, admin::{ diff --git a/rustfs/src/admin/handlers/cluster_snapshot.rs b/rustfs/src/admin/handlers/cluster_snapshot.rs index 0648cf764..f4a1eac04 100644 --- a/rustfs/src/admin/handlers/cluster_snapshot.rs +++ b/rustfs/src/admin/handlers/cluster_snapshot.rs @@ -12,12 +12,12 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::admin::storage_api::{CapabilityState, CapabilityStatus, ObservabilitySnapshot, TopologySnapshot}; +use crate::admin::storage_api::cluster::{CapabilityState, CapabilityStatus, ObservabilitySnapshot, TopologySnapshot}; use crate::admin::{ auth::validate_admin_request, router::{AdminOperation, Operation, S3Router}, runtime_sources::default_admin_usecase, - storage_api::ecstore_cluster::{ + storage_api::cluster::{ ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState, ClusterPoolStateSnapshot, @@ -590,13 +590,13 @@ fn summarize_named_capability_statuses( #[cfg(test)] mod tests { use super::{ClusterSnapshotResponse, ClusterSnapshotSummary, ClusterSnapshotView}; - use crate::admin::storage_api::CapabilityState; - use crate::admin::storage_api::ecstore_cluster::{ + use crate::admin::storage_api::cluster::CapabilityState; + use crate::admin::storage_api::cluster::{CapabilityStatus, ObservabilitySnapshot, TopologySnapshot}; + use crate::admin::storage_api::cluster::{ ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState, ClusterPoolStateSnapshot, }; - use crate::admin::storage_api::{CapabilityStatus, ObservabilitySnapshot, TopologySnapshot}; use crate::cluster_snapshot::{ClusterReadOnlySnapshot, ClusterRuntimeReadinessState, ClusterRuntimeStatusSnapshot}; use crate::server::{DependencyReadiness, ReadinessDegradedReason}; use rustfs_concurrency::{AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadClass}; diff --git a/rustfs/src/admin/handlers/config_admin.rs b/rustfs/src/admin/handlers/config_admin.rs index 8060bdac4..d2cda02f9 100644 --- a/rustfs/src/admin/handlers/config_admin.rs +++ b/rustfs/src/admin/handlers/config_admin.rs @@ -19,12 +19,12 @@ use crate::admin::service::config::{ apply_dynamic_config_for_subsystem, is_dynamic_config_subsystem, signal_config_snapshot_reload, signal_dynamic_config_reload, validate_server_config, }; -use crate::admin::storage_api::ListOperations as _; -use crate::admin::storage_api::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV}; -use crate::admin::storage_api::{ +use crate::admin::storage_api::config::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV}; +use crate::admin::storage_api::config::{ RUSTFS_META_BUCKET, STORAGE_CLASS_SUB_SYS, delete_admin_config, read_admin_config, read_admin_config_without_migrate, save_admin_config, save_admin_server_config, }; +use crate::admin::storage_api::contract::ListOperations as _; use crate::admin::utils::{encode_compatible_admin_payload, is_compat_admin_request, read_compatible_admin_body}; use crate::auth::{check_key_valid, get_session_token}; use crate::error::ApiError; @@ -708,7 +708,7 @@ fn success_response(config_applied: bool) -> S3Result S3Result> { +fn object_store() -> S3Result> { resolve_object_store_handle().ok_or_else(|| s3_error!(InternalError, "server storage not initialized")) } @@ -753,7 +753,7 @@ fn config_update_sub_system(directives: &[ConfigDirective]) -> S3Result S3Result<()> { if DEFAULT_KVS.get().is_none() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); } let Some(defaults) = DEFAULT_KVS.get() else { return Err(s3_error!(InternalError, "config defaults are not initialized")); @@ -1407,7 +1407,7 @@ fn env_help_key(sub_system: &str, key: &str) -> String { fn default_help_postfix(sub_system: &str, key: &str) -> String { if DEFAULT_KVS.get().is_none() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); } DEFAULT_KVS @@ -1894,7 +1894,7 @@ mod tests { #[test] fn full_config_export_can_be_reapplied() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); let mut original = ServerConfig::new(); apply_set_directives( &mut original, @@ -1941,7 +1941,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn build_help_response_appends_default_value_postfix() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); let response = build_help_response(Some("identity_openid"), Some("scopes"), false).expect("help response"); assert_eq!(response.keys_help.len(), 2); @@ -2054,7 +2054,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn render_selected_config_includes_env_override_lines() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); temp_env::with_vars( [ ("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example")), @@ -2090,7 +2090,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn render_selected_config_lists_env_only_targets() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || { let config = ServerConfig::new(); let rendered = String::from_utf8( @@ -2113,7 +2113,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn render_selected_config_supports_specific_env_only_target_queries() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || { let config = ServerConfig::new(); let rendered = String::from_utf8( @@ -2136,7 +2136,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#, #[test] fn render_selected_config_orders_default_before_named_targets() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_ALPHA", Some("http://alpha.example"))], || { let mut config = ServerConfig::new(); apply_set_directives( @@ -2317,7 +2317,7 @@ identity_openid client_id="existing-client""#, #[test] fn storage_class_get_target_none_matches_full_export() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); let mut config = ServerConfig::new(); apply_set_directives( &mut config, diff --git a/rustfs/src/admin/handlers/extensions.rs b/rustfs/src/admin/handlers/extensions.rs index 1f4baae49..09199594a 100644 --- a/rustfs/src/admin/handlers/extensions.rs +++ b/rustfs/src/admin/handlers/extensions.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::admin::storage_api::CapabilityStatus; +use crate::admin::storage_api::cluster::CapabilityStatus; use crate::admin::{ auth::validate_admin_request, handlers::{cluster_snapshot, plugins_instances, system}, @@ -118,8 +118,8 @@ pub(crate) struct ExtensionInstancesResponse { pub next_marker: Option, } -async fn build_extension_catalog_response() -> Result -{ +async fn build_extension_catalog_response() +-> Result { let mut extensions = builtin_extension_schemas(); let example = example_external_webhook_plugin(); extensions.push(target_marketplace_extension_schema(&example.manifest)); diff --git a/rustfs/src/admin/handlers/heal.rs b/rustfs/src/admin/handlers/heal.rs index 258a18e0d..211fe51aa 100644 --- a/rustfs/src/admin/handlers/heal.rs +++ b/rustfs/src/admin/handlers/heal.rs @@ -15,10 +15,10 @@ use crate::admin::auth::{authenticate_request, validate_admin_request}; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::admin::runtime_sources::resolve_object_store_handle; -use crate::admin::storage_api::HealOperations as _; -use crate::admin::storage_api::ecstore_utils::is_valid_object_prefix; -use crate::admin::storage_api::is_reserved_or_invalid_bucket; -use crate::admin::storage_api::spawn_traced; +use crate::admin::storage_api::access::spawn_traced; +use crate::admin::storage_api::bucket::is_reserved_or_invalid_bucket; +use crate::admin::storage_api::bucket::utils::is_valid_object_prefix; +use crate::admin::storage_api::contract::HealOperations as _; use crate::server::ADMIN_PREFIX; use crate::server::RemoteAddr; use bytes::Bytes; @@ -386,10 +386,10 @@ fn should_handle_root_heal_directly(_hip: &HealInitParams) -> bool { false } -fn map_root_heal_status(heal_err: Option) -> S3Result<()> { +fn map_root_heal_status(heal_err: Option) -> S3Result<()> { match heal_err { None => Ok(()), - Some(crate::admin::storage_api::StorageError::NoHealRequired) => { + Some(crate::admin::storage_api::error::StorageError::NoHealRequired) => { info!( event = EVENT_ADMIN_RESPONSE_EMITTED, component = LOG_COMPONENT_ADMIN_API, @@ -750,7 +750,7 @@ mod tests { json_response, map_heal_response, map_root_heal_status, should_handle_root_heal_directly, validate_heal_request_mode, validate_heal_target, }; - use crate::admin::storage_api::StorageError; + use crate::admin::storage_api::error::StorageError; use bytes::Bytes; use http::StatusCode; use http::Uri; diff --git a/rustfs/src/admin/handlers/kms_dynamic.rs b/rustfs/src/admin/handlers/kms_dynamic.rs index 9faef0e5d..659453e7c 100644 --- a/rustfs/src/admin/handlers/kms_dynamic.rs +++ b/rustfs/src/admin/handlers/kms_dynamic.rs @@ -19,7 +19,7 @@ use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::admin::runtime_sources::{ resolve_kms_runtime_service_manager, resolve_object_store_handle, resolve_or_init_kms_runtime_service_manager, }; -use crate::admin::storage_api::{read_admin_config, save_admin_config}; +use crate::admin::storage_api::config::{read_admin_config, save_admin_config}; use crate::auth::{check_key_valid, get_session_token}; use crate::server::{ADMIN_PREFIX, RemoteAddr}; use hyper::{Method, StatusCode}; diff --git a/rustfs/src/admin/handlers/metrics.rs b/rustfs/src/admin/handlers/metrics.rs index 1f91a2e36..54e723166 100644 --- a/rustfs/src/admin/handlers/metrics.rs +++ b/rustfs/src/admin/handlers/metrics.rs @@ -20,8 +20,8 @@ use crate::admin::auth::validate_admin_request; use crate::admin::router::Operation; -use crate::admin::storage_api::spawn_traced; -use crate::admin::storage_api::{CollectMetricsOpts, MetricType, collect_local_metrics}; +use crate::admin::storage_api::access::spawn_traced; +use crate::admin::storage_api::metrics::{CollectMetricsOpts, MetricType, collect_local_metrics}; use crate::auth::{check_key_valid, get_session_token}; use crate::server::RemoteAddr; use bytes::Bytes; diff --git a/rustfs/src/admin/handlers/object_zip_download.rs b/rustfs/src/admin/handlers/object_zip_download.rs index b8f8337c9..dee116c9a 100644 --- a/rustfs/src/admin/handlers/object_zip_download.rs +++ b/rustfs/src/admin/handlers/object_zip_download.rs @@ -14,8 +14,11 @@ use crate::admin::router::{ADMIN_OBJECT_ZIP_DOWNLOADS_PATH, AdminOperation, Operation, S3Router}; use crate::admin::runtime_sources::{resolve_action_credentials, resolve_object_store_handle, resolve_region}; -use crate::admin::storage_api::{BucketOperations, BucketOptions, ListOperations as _, ObjectIO as _, ObjectOperations as _}; -use crate::admin::storage_api::{ReqInfo, StorageObjectOptions as ObjectOptions, authorize_request}; +use crate::admin::storage_api::access::{ReqInfo, authorize_request}; +use crate::admin::storage_api::contract::{ + BucketOperations, BucketOptions, ListOperations as _, ObjectIO as _, ObjectOperations as _, +}; +use crate::admin::storage_api::object::StorageObjectOptions as ObjectOptions; use crate::auth::{check_key_valid, get_session_token}; use crate::error::ApiError; use crate::license::license_check; @@ -645,7 +648,7 @@ async fn preflight_zip_items(request: &CreateObjectZipDownloadRequest, items: &[ Ok(()) } -fn storage_error_to_s3(err: crate::admin::storage_api::Error) -> s3s::S3Error { +fn storage_error_to_s3(err: crate::admin::storage_api::error::Error) -> s3s::S3Error { ApiError::from(err).into() } diff --git a/rustfs/src/admin/handlers/oidc.rs b/rustfs/src/admin/handlers/oidc.rs index a0fd28d9c..19d0b9fdc 100644 --- a/rustfs/src/admin/handlers/oidc.rs +++ b/rustfs/src/admin/handlers/oidc.rs @@ -16,7 +16,7 @@ use super::sts::create_oidc_sts_credentials; use crate::admin::auth::validate_admin_request; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::admin::runtime_sources::{resolve_object_store_handle, resolve_oidc_handle, resolve_server_config}; -use crate::admin::storage_api::{read_admin_config_without_migrate, save_admin_server_config}; +use crate::admin::storage_api::config::{read_admin_config_without_migrate, save_admin_server_config}; use crate::auth::{check_key_valid, get_session_token}; use crate::server::{ADMIN_PREFIX, MINIO_ADMIN_PREFIX, RemoteAddr}; use http::StatusCode; diff --git a/rustfs/src/admin/handlers/pools.rs b/rustfs/src/admin/handlers/pools.rs index 789bbe155..ebd0bb3c4 100644 --- a/rustfs/src/admin/handlers/pools.rs +++ b/rustfs/src/admin/handlers/pools.rs @@ -32,7 +32,7 @@ use crate::{ admin::{ auth::validate_admin_request, router::{AdminOperation, Operation, S3Router}, - storage_api::{EndpointServerPools, PeerRestClient}, + storage_api::runtime::{EndpointServerPools, PeerRestClient}, }, auth::{check_key_valid, get_session_token}, error::ApiError, @@ -213,7 +213,7 @@ macro_rules! log_pool_response_emitted { }; } -fn endpoints_from_context() -> Option { +fn endpoints_from_context() -> Option { resolve_endpoints_handle() } @@ -1080,7 +1080,7 @@ mod pools_handler_tests { pool_admin_pool_not_found_error_with_audit, pool_admin_pool_parse_error_with_audit, pool_admin_query_parse_error, pool_admin_query_parse_error_with_audit, validate_pool_mutation_leader, validate_start_decommission_guards, }; - use crate::admin::storage_api::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints}; + use crate::admin::storage_api::runtime::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints}; fn test_pool_endpoints(is_local: bool) -> EndpointServerPools { let mut endpoint = Endpoint::try_from("http://127.0.0.1:9000/disk").expect("test endpoint should parse"); diff --git a/rustfs/src/admin/handlers/quota.rs b/rustfs/src/admin/handlers/quota.rs index c5f81b938..a75a36bb3 100644 --- a/rustfs/src/admin/handlers/quota.rs +++ b/rustfs/src/admin/handlers/quota.rs @@ -17,9 +17,9 @@ use crate::admin::auth::{validate_admin_request, validate_admin_request_with_bucket}; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::admin::runtime_sources::{resolve_bucket_metadata_handle, resolve_object_store_handle}; -use crate::admin::storage_api::metadata_sys::BucketMetadataSys; -use crate::admin::storage_api::quota::checker::QuotaChecker; -use crate::admin::storage_api::quota::{BucketQuota, QuotaError, QuotaOperation}; +use crate::admin::storage_api::bucket::metadata_sys::BucketMetadataSys; +use crate::admin::storage_api::bucket::quota::checker::QuotaChecker; +use crate::admin::storage_api::bucket::quota::{BucketQuota, QuotaError, QuotaOperation}; use crate::auth::{check_key_valid, get_session_token}; use crate::server::ADMIN_PREFIX; use hyper::{Method, StatusCode}; diff --git a/rustfs/src/admin/handlers/rebalance.rs b/rustfs/src/admin/handlers/rebalance.rs index f8271250b..1b12dfef7 100644 --- a/rustfs/src/admin/handlers/rebalance.rs +++ b/rustfs/src/admin/handlers/rebalance.rs @@ -12,11 +12,13 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::admin::storage_api::{BucketOperations, BucketOptions, StorageAdminApi}; -use crate::admin::storage_api::{ - DiskStat, ECStore, NotificationSys, RebalSaveOpt, RebalanceCleanupWarnings, RebalanceMeta, RebalanceStopPropagationRecord, - StorageError, decode_rebalance_stop_propagation_record, +use crate::admin::storage_api::contract::{BucketOperations, BucketOptions, StorageAdminApi}; +use crate::admin::storage_api::error::StorageError; +use crate::admin::storage_api::rebalance::{ + DiskStat, RebalSaveOpt, RebalanceCleanupWarnings, RebalanceMeta, RebalanceStopPropagationRecord, + decode_rebalance_stop_propagation_record, }; +use crate::admin::storage_api::runtime::{ECStore, NotificationSys}; use crate::{ admin::runtime_sources::{resolve_notification_system, resolve_object_store_handle}, admin::{ @@ -296,7 +298,7 @@ fn build_rebalance_pool_progress( now: OffsetDateTime, stop_time: Option, percent_free_goal: f64, - ps: &crate::admin::storage_api::RebalanceStats, + ps: &crate::admin::storage_api::rebalance::RebalanceStats, ) -> Option { let total_bytes_to_rebal = ps.init_capacity as f64 * percent_free_goal - ps.init_free_space as f64; let terminal_time = ps.info.end_time.or(stop_time); @@ -339,7 +341,7 @@ fn build_rebalance_pool_statuses( now: OffsetDateTime, stop_time: Option, percent_free_goal: f64, - pool_stats: &[crate::admin::storage_api::RebalanceStats], + pool_stats: &[crate::admin::storage_api::rebalance::RebalanceStats], disk_stats: &[DiskStat], ) -> Vec { pool_stats @@ -903,7 +905,7 @@ mod rebalance_handler_tests { rebalance_pool_used, rebalance_query_present, rebalance_remaining_buckets, rebalance_rollback_failure_message, rebalance_rollback_stop_failure_message, rebalance_start_rollback_error, rebalance_used_pct, rollback_result_label, }; - use crate::admin::storage_api::{ + use crate::admin::storage_api::rebalance::{ DiskStat, RebalStatus, RebalanceCleanupWarningEntry, RebalanceCleanupWarnings, RebalanceInfo, RebalanceMeta, RebalanceStats, RebalanceStopPropagationRecord, encode_rebalance_stop_propagation_record, }; diff --git a/rustfs/src/admin/handlers/replication.rs b/rustfs/src/admin/handlers/replication.rs index 191a7282c..426282b74 100644 --- a/rustfs/src/admin/handlers/replication.rs +++ b/rustfs/src/admin/handlers/replication.rs @@ -16,14 +16,14 @@ use crate::admin::auth::validate_admin_request; use crate::admin::handlers::site_replication::site_replication_peer_deployment_id_for_endpoint; use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::admin::runtime_sources::{resolve_object_store_handle, resolve_replication_stats_handle, resolve_runtime_port}; -use crate::admin::storage_api::StorageError; -use crate::admin::storage_api::bucket_target_sys::{BucketTargetError, BucketTargetSys}; -use crate::admin::storage_api::metadata::BUCKET_TARGETS_FILE; -use crate::admin::storage_api::metadata_sys; -use crate::admin::storage_api::metadata_sys::get_replication_config; -use crate::admin::storage_api::replication::BucketStats; -use crate::admin::storage_api::target::BucketTarget; -use crate::admin::storage_api::{BucketOperations, BucketOptions}; +use crate::admin::storage_api::bucket::metadata::BUCKET_TARGETS_FILE; +use crate::admin::storage_api::bucket::metadata_sys; +use crate::admin::storage_api::bucket::metadata_sys::get_replication_config; +use crate::admin::storage_api::bucket::replication::BucketStats; +use crate::admin::storage_api::bucket::target::BucketTarget; +use crate::admin::storage_api::bucket::target_sys::{BucketTargetError, BucketTargetSys}; +use crate::admin::storage_api::contract::{BucketOperations, BucketOptions}; +use crate::admin::storage_api::error::StorageError; use crate::admin::utils::read_compatible_admin_body; use crate::auth::{check_key_valid, get_session_token}; use crate::error::ApiError; @@ -448,7 +448,7 @@ impl Operation for RemoveRemoteTargetHandler { #[cfg(test)] mod tests { use super::{extract_query_params, validate_remote_target_tls_settings}; - use crate::admin::storage_api::target::BucketTarget; + use crate::admin::storage_api::bucket::target::BucketTarget; use http::Uri; #[test] diff --git a/rustfs/src/admin/handlers/site_replication.rs b/rustfs/src/admin/handlers/site_replication.rs index 9464467e4..1b9e038d2 100644 --- a/rustfs/src/admin/handlers/site_replication.rs +++ b/rustfs/src/admin/handlers/site_replication.rs @@ -23,19 +23,21 @@ use crate::admin::site_replication_identity::{ canonical_endpoint, deployment_id_for_endpoint, normalize_peer_map_by_identity_with, same_identity_endpoint, site_identity_key, }; -use crate::admin::storage_api::Error as StorageError; -use crate::admin::storage_api::bucket_target_sys::BucketTargetSys; -use crate::admin::storage_api::ecstore_utils::{deserialize, serialize}; -use crate::admin::storage_api::metadata::{ +use crate::admin::storage_api::bucket::metadata::{ BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, OBJECT_LOCK_CONFIG, }; -use crate::admin::storage_api::metadata_sys; -use crate::admin::storage_api::replication::ResyncOpts; -use crate::admin::storage_api::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials}; -use crate::admin::storage_api::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _}; -use crate::admin::storage_api::{BucketOperations, BucketOptions, DeleteBucketOptions, MakeBucketOptions, SRBucketDeleteOp}; -use crate::admin::storage_api::{delete_admin_config, read_admin_config, save_admin_config}; +use crate::admin::storage_api::bucket::metadata_sys; +use crate::admin::storage_api::bucket::replication::ResyncOpts; +use crate::admin::storage_api::bucket::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials}; +use crate::admin::storage_api::bucket::target_sys::BucketTargetSys; +use crate::admin::storage_api::bucket::utils::{deserialize, serialize}; +use crate::admin::storage_api::bucket::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _}; +use crate::admin::storage_api::config::{delete_admin_config, read_admin_config, save_admin_config}; +use crate::admin::storage_api::contract::{ + BucketOperations, BucketOptions, DeleteBucketOptions, MakeBucketOptions, SRBucketDeleteOp, +}; +use crate::admin::storage_api::error::Error as StorageError; use crate::admin::utils::{encode_compatible_admin_payload, read_compatible_admin_body}; use crate::auth::{check_key_valid, get_session_token}; use crate::config::get_config_snapshot; @@ -652,7 +654,7 @@ async fn site_replication_peer_client() -> S3Result { built } -fn runtime_tls_enabled_with(endpoints: Option<&crate::admin::storage_api::EndpointServerPools>) -> bool { +fn runtime_tls_enabled_with(endpoints: Option<&crate::admin::storage_api::runtime::EndpointServerPools>) -> bool { if !rustfs_utils::get_env_str(ENV_RUSTFS_TLS_PATH, DEFAULT_RUSTFS_TLS_PATH).is_empty() { return true; } @@ -3358,7 +3360,7 @@ fn is_stale_update(local_updated_at: OffsetDateTime, incoming_updated_at: Option } fn bucket_meta_local_updated_at( - bucket_meta: &crate::admin::storage_api::metadata::BucketMetadata, + bucket_meta: &crate::admin::storage_api::bucket::metadata::BucketMetadata, config_file: &str, ) -> OffsetDateTime { match config_file { @@ -4605,8 +4607,8 @@ impl Operation for SRRotateServiceAccountHandler { mod tests { use super::*; use crate::admin::runtime_sources::{resolve_outbound_tls_generation, set_test_outbound_tls_generation}; - use crate::admin::storage_api::Endpoint; - use crate::admin::storage_api::{EndpointServerPools, Endpoints, PoolEndpoints}; + use crate::admin::storage_api::runtime::Endpoint; + use crate::admin::storage_api::runtime::{EndpointServerPools, Endpoints, PoolEndpoints}; use http::{HeaderMap, HeaderValue, Uri}; use rustfs_policy::policy::action::S3Action; use serial_test::serial; diff --git a/rustfs/src/admin/handlers/sts.rs b/rustfs/src/admin/handlers/sts.rs index f8437629a..1d67ccfad 100644 --- a/rustfs/src/admin/handlers/sts.rs +++ b/rustfs/src/admin/handlers/sts.rs @@ -13,7 +13,7 @@ // limitations under the License. use super::is_admin::IsAdminHandler; -use crate::admin::storage_api::ecstore_utils::serialize; +use crate::admin::storage_api::bucket::utils::serialize; use crate::{ admin::runtime_sources::{resolve_action_credentials, resolve_oidc_handle, resolve_token_signing_key}, admin::{ diff --git a/rustfs/src/admin/handlers/system.rs b/rustfs/src/admin/handlers/system.rs index ecf537bcb..806e09209 100644 --- a/rustfs/src/admin/handlers/system.rs +++ b/rustfs/src/admin/handlers/system.rs @@ -18,7 +18,7 @@ use crate::admin::router::{AdminOperation, Operation, S3Router}; use crate::admin::runtime_sources::{ DefaultAdminUsecase, QueryServerInfoRequest, default_admin_usecase, resolve_endpoints_handle, }; -use crate::admin::storage_api::{ +use crate::admin::storage_api::cluster::{ CapabilityState, CapabilityStatus, ObservabilitySnapshotProvider, TopologySnapshot, TopologySnapshotProvider, }; use crate::auth::{check_key_valid, get_session_token}; @@ -298,7 +298,7 @@ pub struct RuntimeCapabilitiesResponse { pub summary: RuntimeCapabilitiesSummary, pub cluster_snapshot_path: String, pub cluster_snapshot_summary: Option, - pub observability: crate::admin::storage_api::ObservabilitySnapshot, + pub observability: crate::admin::storage_api::cluster::ObservabilitySnapshot, pub workload_admission: WorkloadAdmissionRegistrySnapshot, pub topology: Option, pub topology_status: CapabilityStatus, @@ -307,7 +307,7 @@ pub struct RuntimeCapabilitiesResponse { pub struct RuntimeCapabilitiesHandler {} pub(crate) async fn build_runtime_capabilities_response() --> Result { +-> Result { let usecase = default_admin_usecase(); let observability_provider = RustFsObservabilitySnapshotProvider; let observability = observability_provider.observability_snapshot().await?; @@ -340,7 +340,7 @@ pub(crate) async fn build_runtime_capabilities_response() } fn build_runtime_capabilities_summary( - observability: &crate::admin::storage_api::ObservabilitySnapshot, + observability: &crate::admin::storage_api::cluster::ObservabilitySnapshot, topology: Option<&TopologySnapshot>, topology_status: &CapabilityStatus, cluster_snapshot_summary: Option<&CapabilityStatus>, @@ -517,7 +517,7 @@ mod tests { build_runtime_capabilities_response, build_runtime_capabilities_summary, system_admin_discovery, }; use crate::admin::runtime_sources::DefaultAdminUsecase; - use crate::admin::storage_api::{ + use crate::admin::storage_api::cluster::{ CapabilityState, CapabilityStatus, MemorySamplingState, ObservabilitySnapshot, PlatformSupport, TopologyCapabilities, TopologySnapshot, UserspaceProfilingCapability, }; diff --git a/rustfs/src/admin/handlers/table_catalog.rs b/rustfs/src/admin/handlers/table_catalog.rs index 467e4c5ea..8b0f88c9f 100644 --- a/rustfs/src/admin/handlers/table_catalog.rs +++ b/rustfs/src/admin/handlers/table_catalog.rs @@ -14,7 +14,8 @@ use crate::admin::runtime_sources::default_admin_usecase; use crate::admin::runtime_sources::{resolve_object_store_handle, resolve_token_signing_key}; -use crate::admin::storage_api::{ECStore, metadata::table_catalog_path_hash, metadata_sys}; +use crate::admin::storage_api::bucket::{metadata::table_catalog_path_hash, metadata_sys}; +use crate::admin::storage_api::runtime::ECStore; use crate::admin::{ auth::{AdminResourceScope, validate_admin_request, validate_admin_request_with_bucket_object}, router::{AdminOperation, Operation, S3Router}, diff --git a/rustfs/src/admin/handlers/tier.rs b/rustfs/src/admin/handlers/tier.rs index ac33a941f..dcd8c1012 100644 --- a/rustfs/src/admin/handlers/tier.rs +++ b/rustfs/src/admin/handlers/tier.rs @@ -13,7 +13,7 @@ // limitations under the License. #![allow(unused_variables, unused_mut, unused_must_use)] -use crate::admin::storage_api::{ +use crate::admin::storage_api::tier::{ AdminError, DailyAllTierStats, ERR_TIER_ALREADY_EXISTS, ERR_TIER_BACKEND_IN_USE, ERR_TIER_BACKEND_NOT_EMPTY, ERR_TIER_CONNECT_ERR, ERR_TIER_INVALID_CREDENTIALS, ERR_TIER_MISSING_CREDENTIALS, ERR_TIER_NAME_NOT_UPPERCASE, ERR_TIER_NOT_FOUND, TierConfig, TierCreds, TierType, storageclass, @@ -932,7 +932,7 @@ impl Operation for ClearTier { #[cfg(test)] mod tests { use super::*; - use crate::admin::storage_api::lifecycle::tier_last_day_stats::LastDayTierStats; + use crate::admin::storage_api::bucket::lifecycle::tier_last_day_stats::LastDayTierStats; use http::Uri; use matchit::Router; diff --git a/rustfs/src/admin/handlers/trace.rs b/rustfs/src/admin/handlers/trace.rs index db0a8ae9e..3af6ac7fa 100644 --- a/rustfs/src/admin/handlers/trace.rs +++ b/rustfs/src/admin/handlers/trace.rs @@ -14,7 +14,7 @@ use crate::admin::router::Operation; use crate::admin::runtime_sources::resolve_endpoints_handle; -use crate::admin::storage_api::PeerRestClient; +use crate::admin::storage_api::runtime::PeerRestClient; use http::StatusCode; use hyper::Uri; use matchit::Params; diff --git a/rustfs/src/admin/router.rs b/rustfs/src/admin/router.rs index 0e580f134..93f9c70bd 100644 --- a/rustfs/src/admin/router.rs +++ b/rustfs/src/admin/router.rs @@ -12,18 +12,19 @@ // See the License for the specific language governing permissions and // limitations under the License. -use super::storage_api::PeerRestClient; -use super::storage_api::bandwidth::monitor::BandwidthDetails; -use super::storage_api::bucket_target_sys::{ +use super::storage_api::bucket::bandwidth::monitor::BandwidthDetails; +use super::storage_api::bucket::metadata::BUCKET_TARGETS_FILE; +use super::storage_api::bucket::metadata_sys; +use super::storage_api::bucket::replication::{BucketReplicationResyncStatus, BucketStats, ObjectOpts, ResyncOpts}; +use super::storage_api::bucket::target::{BucketTarget, BucketTargetType, BucketTargets}; +use super::storage_api::bucket::target_sys::{ BucketTargetSys, PutObjectOptions, RemoveObjectOptions, S3ClientError, TargetClient, }; -use super::storage_api::metadata::BUCKET_TARGETS_FILE; -use super::storage_api::metadata_sys; -use super::storage_api::read_admin_config_without_migrate; -use super::storage_api::replication::{BucketReplicationResyncStatus, BucketStats, ObjectOpts, ResyncOpts}; -use super::storage_api::target::{BucketTarget, BucketTargetType, BucketTargets}; -use super::storage_api::versioning_sys::BucketVersioningSys; -use super::storage_api::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _, StorageError}; +use super::storage_api::bucket::versioning_sys::BucketVersioningSys; +use super::storage_api::bucket::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _}; +use super::storage_api::config::read_admin_config_without_migrate; +use super::storage_api::error::StorageError; +use super::storage_api::runtime::PeerRestClient; use crate::admin::console::{is_console_path, make_console_server}; use crate::admin::handlers::oidc::is_oidc_path; use crate::admin::runtime_sources::{ @@ -31,8 +32,8 @@ use crate::admin::runtime_sources::{ resolve_object_store_handle, resolve_region, resolve_replication_pool_handle, resolve_replication_stats_handle, resolve_server_config, }; -use crate::admin::storage_api::{BucketOperations, BucketOptions}; -use crate::admin::storage_api::{ReqInfo, authorize_request, spawn_traced}; +use crate::admin::storage_api::access::{ReqInfo, authorize_request, spawn_traced}; +use crate::admin::storage_api::contract::{BucketOperations, BucketOptions}; use crate::auth::{check_key_valid, get_session_token}; use crate::error::ApiError; use crate::license::license_check; @@ -1939,7 +1940,7 @@ async fn resolve_replication_target_client(bucket: &str, target: &BucketTarget) fn build_replication_probe_put_options(now: OffsetDateTime) -> PutObjectOptions { PutObjectOptions { - internal: super::storage_api::bucket_target_sys::AdvancedPutOptions { + internal: super::storage_api::bucket::target_sys::AdvancedPutOptions { source_version_id: Uuid::new_v4().to_string(), replication_status: ReplicationStatusType::Replica, source_mtime: now, @@ -2768,7 +2769,7 @@ mod tests { #[test] fn apply_replication_reset_to_targets_updates_matching_target() { let mut targets = BucketTargets { - targets: vec![crate::admin::storage_api::target::BucketTarget { + targets: vec![crate::admin::storage_api::bucket::target::BucketTarget { arn: "arn:target".to_string(), ..Default::default() }], @@ -2790,10 +2791,10 @@ mod tests { let mut status = BucketReplicationResyncStatus::new(); status.targets_map.insert( "arn:z".to_string(), - crate::admin::storage_api::replication::TargetReplicationResyncStatus { + crate::admin::storage_api::bucket::replication::TargetReplicationResyncStatus { resync_id: "rid-z".to_string(), last_update: Some(datetime!(2025-01-03 00:00 UTC)), - resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncFailed, + resync_status: crate::admin::storage_api::bucket::replication::ResyncStatusType::ResyncFailed, failed_count: 2, failed_size: 4, bucket: "bucket-z".to_string(), @@ -2803,10 +2804,10 @@ mod tests { ); status.targets_map.insert( "arn:a".to_string(), - crate::admin::storage_api::replication::TargetReplicationResyncStatus { + crate::admin::storage_api::bucket::replication::TargetReplicationResyncStatus { resync_id: "rid-a".to_string(), last_update: Some(datetime!(2025-01-02 00:00 UTC)), - resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncCompleted, + resync_status: crate::admin::storage_api::bucket::replication::ResyncStatusType::ResyncCompleted, replicated_count: 3, replicated_size: 9, bucket: "bucket-a".to_string(), @@ -2836,10 +2837,10 @@ mod tests { let mut status = BucketReplicationResyncStatus::new(); status.targets_map.insert( "arn:z".to_string(), - crate::admin::storage_api::replication::TargetReplicationResyncStatus { + crate::admin::storage_api::bucket::replication::TargetReplicationResyncStatus { resync_id: "rid-z".to_string(), last_update: Some(datetime!(2025-02-03 00:00 UTC)), - resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncFailed, + resync_status: crate::admin::storage_api::bucket::replication::ResyncStatusType::ResyncFailed, failed_count: 2, failed_size: 4, bucket: "bucket-z".to_string(), @@ -2849,10 +2850,10 @@ mod tests { ); status.targets_map.insert( "arn:a".to_string(), - crate::admin::storage_api::replication::TargetReplicationResyncStatus { + crate::admin::storage_api::bucket::replication::TargetReplicationResyncStatus { resync_id: "rid-a".to_string(), last_update: Some(datetime!(2025-02-02 00:00 UTC)), - resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncCompleted, + resync_status: crate::admin::storage_api::bucket::replication::ResyncStatusType::ResyncCompleted, replicated_count: 3, replicated_size: 9, bucket: "bucket-a".to_string(), diff --git a/rustfs/src/admin/service/config.rs b/rustfs/src/admin/service/config.rs index 49f76dcbe..18e76e918 100644 --- a/rustfs/src/admin/service/config.rs +++ b/rustfs/src/admin/service/config.rs @@ -16,9 +16,8 @@ use crate::admin::runtime_sources::{ AppContext, current_app_context, publish_server_config, publish_storage_class_config, resolve_notification_system, resolve_object_store_handle, resolve_object_store_handle_for_context, }; -use crate::admin::storage_api::StorageAdminApi; -use crate::admin::storage_api::storageclass; -use crate::admin::storage_api::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate}; +use crate::admin::storage_api::config::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate, storageclass}; +use crate::admin::storage_api::contract::StorageAdminApi; use rustfs_audit::reload_audit_config; use rustfs_config::audit::{AUDIT_MQTT_SUB_SYS, AUDIT_REDIS_DEFAULT_CHANNEL, AUDIT_WEBHOOK_SUB_SYS}; use rustfs_config::notify::{NOTIFY_MQTT_SUB_SYS, NOTIFY_REDIS_DEFAULT_CHANNEL, NOTIFY_WEBHOOK_SUB_SYS}; @@ -384,7 +383,7 @@ pub async fn signal_config_snapshot_reload() { #[cfg(test)] mod tests { use super::*; - use crate::admin::storage_api::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG}; + use crate::admin::storage_api::bucket::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG}; use rustfs_config::notify::NOTIFY_WEBHOOK_SUB_SYS; use rustfs_config::oidc::{OIDC_CLIENT_ID, OIDC_CONFIG_URL, OIDC_SCOPES}; use rustfs_config::{HEAL_SUB_SYS, SCANNER_SUB_SYS}; @@ -440,7 +439,7 @@ mod tests { #[test] fn validate_notify_subsystem_config_rejects_invalid_webhook_endpoint() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); let mut config = ServerConfig::new(); let targets = config.0.get_mut(NOTIFY_WEBHOOK_SUB_SYS).expect("notify webhook defaults"); let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target"); @@ -454,7 +453,7 @@ mod tests { #[test] fn validate_audit_subsystem_config_rejects_relative_queue_dir() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); let mut config = ServerConfig::new(); let targets = config.0.get_mut(AUDIT_MQTT_SUB_SYS).expect("audit mqtt defaults"); let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target"); @@ -469,7 +468,7 @@ mod tests { #[test] fn validate_identity_openid_config_rejects_missing_openid_scope() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); let mut config = ServerConfig::new(); let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults"); let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target"); @@ -486,7 +485,7 @@ mod tests { #[test] fn validate_identity_openid_config_rejects_invalid_named_provider_id() { - crate::admin::storage_api::init_admin_config_defaults(); + crate::admin::storage_api::config::init_admin_config_defaults(); let mut config = ServerConfig::new(); let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults"); let default_kvs = targets.get(DEFAULT_DELIMITER).cloned().expect("default target"); diff --git a/rustfs/src/admin/service/site_replication.rs b/rustfs/src/admin/service/site_replication.rs index 7c19e9eb2..0fbc6afa9 100644 --- a/rustfs/src/admin/service/site_replication.rs +++ b/rustfs/src/admin/service/site_replication.rs @@ -14,8 +14,8 @@ use crate::admin::runtime_sources::{AppContext, current_app_context, resolve_object_store_handle_for_context}; use crate::admin::site_replication_identity::{deployment_id_for_endpoint, normalize_peer_map_by_identity_with}; -use crate::admin::storage_api::Error as StorageError; -use crate::admin::storage_api::{read_admin_config, save_admin_config}; +use crate::admin::storage_api::config::{read_admin_config, save_admin_config}; +use crate::admin::storage_api::error::Error as StorageError; use rustfs_madmin::PeerInfo; use s3s::{S3Error, S3ErrorCode, S3Result}; use serde_json::{Map, Value}; diff --git a/rustfs/src/admin/storage_api.rs b/rustfs/src/admin/storage_api.rs index 43a78db8b..c2f8dba2a 100644 --- a/rustfs/src/admin/storage_api.rs +++ b/rustfs/src/admin/storage_api.rs @@ -428,3 +428,93 @@ pub(crate) mod data_usage { crate::storage::ecstore_data_usage::load_data_usage_from_backend(store).await } } + +pub(crate) mod access { + pub(crate) use super::{ReqInfo, RequestContext, authorize_request, spawn_traced}; +} + +pub(crate) mod bucket { + pub(crate) use super::bandwidth; + pub(crate) use super::bucket_target_sys as target_sys; + #[cfg(test)] + pub(crate) use super::lifecycle; + pub(crate) use super::metadata; + pub(crate) use super::metadata_sys; + pub(crate) use super::quota; + pub(crate) use super::replication; + pub(crate) use super::target; + pub(crate) use super::versioning_sys; + pub(crate) use super::{AdminReplicationConfigExt, AdminVersioningConfigExt, is_reserved_or_invalid_bucket}; + + pub(crate) mod utils { + pub(crate) use super::super::ecstore_utils::{deserialize, is_valid_object_prefix, serialize}; + } +} + +pub(crate) mod cluster { + pub(crate) use super::ecstore_cluster::{ + ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot, + ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState, + ClusterPoolStateSnapshot, + }; + pub(crate) use super::{ + CapabilitySnapshotError, CapabilityState, CapabilityStatus, ObservabilitySnapshot, ObservabilitySnapshotProvider, + TopologySnapshot, TopologySnapshotProvider, + }; + + #[cfg(test)] + pub(crate) use super::{MemorySamplingState, PlatformSupport, TopologyCapabilities, UserspaceProfilingCapability}; +} + +pub(crate) mod config { + pub(crate) use super::storageclass; + pub(crate) use super::{ + RUSTFS_META_BUCKET, STORAGE_CLASS_SUB_SYS, delete_admin_config, init_admin_config_defaults, read_admin_config, + read_admin_config_without_migrate, save_admin_config, save_admin_server_config, + }; +} + +pub(crate) mod contract { + pub(crate) use super::{ + BucketOperations, BucketOptions, DeleteBucketOptions, HealOperations, ListOperations, MakeBucketOptions, ObjectIO, + ObjectOperations, SRBucketDeleteOp, StorageAdminApi, + }; +} + +pub(crate) mod error { + pub(crate) use super::{Error, StorageError}; +} + +pub(crate) mod metrics { + pub(crate) use super::{CollectMetricsOpts, MetricType, collect_local_metrics}; +} + +pub(crate) mod object { + pub(crate) use super::StorageObjectOptions; +} + +pub(crate) mod rebalance { + pub(crate) use super::{ + DiskStat, RebalSaveOpt, RebalanceCleanupWarnings, RebalanceMeta, RebalanceStats, RebalanceStopPropagationRecord, + decode_rebalance_stop_propagation_record, + }; + + #[cfg(test)] + pub(crate) use super::{RebalStatus, RebalanceCleanupWarningEntry, RebalanceInfo, encode_rebalance_stop_propagation_record}; +} + +pub(crate) mod runtime { + pub(crate) use super::{ECStore, EndpointServerPools, NotificationSys, PeerRestClient}; + + #[cfg(test)] + pub(crate) use super::{Endpoint, Endpoints, PoolEndpoints}; +} + +pub(crate) mod tier { + pub(crate) use super::storageclass; + pub(crate) use super::{ + AdminError, DailyAllTierStats, ERR_TIER_ALREADY_EXISTS, ERR_TIER_BACKEND_IN_USE, ERR_TIER_BACKEND_NOT_EMPTY, + ERR_TIER_CONNECT_ERR, ERR_TIER_INVALID_CREDENTIALS, ERR_TIER_MISSING_CREDENTIALS, ERR_TIER_NAME_NOT_UPPERCASE, + ERR_TIER_NOT_FOUND, TierConfig, TierCreds, TierType, + }; +} diff --git a/rustfs/src/capacity/service.rs b/rustfs/src/capacity/service.rs index 1757373d1..73fb8deac 100644 --- a/rustfs/src/capacity/service.rs +++ b/rustfs/src/capacity/service.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_api::{all_local_disk, disk_drive_path, disk_endpoint}; +use crate::storage_api::capacity::{all_local_disk, disk_drive_path, disk_endpoint}; use rustfs_io_metrics::capacity_metrics::{ record_capacity_cache_hit, record_capacity_cache_miss, record_capacity_cache_served, record_capacity_refresh_request, record_capacity_scan_mode, diff --git a/rustfs/src/cluster_snapshot.rs b/rustfs/src/cluster_snapshot.rs index 2b7de0fd5..4f9624197 100644 --- a/rustfs/src/cluster_snapshot.rs +++ b/rustfs/src/cluster_snapshot.rs @@ -16,11 +16,11 @@ use crate::runtime_capabilities::runtime_observability_snapshot; use crate::server::{ DependencyReadiness, DependencyReadinessReport, ReadinessDegradedReason, snapshot_dependency_readiness_report, }; -use crate::storage_api::ecstore_cluster::{ +use crate::storage_api::cluster::control_plane::{ ClusterControlPlane, ClusterControlPlaneSnapshot, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot, ClusterPeerHealthSnapshot, ClusterPoolStateSnapshot, }; -use crate::storage_api::{EndpointServerPools, ObservabilitySnapshot, TopologySnapshot}; +use crate::storage_api::cluster::{EndpointServerPools, ObservabilitySnapshot, TopologySnapshot}; use crate::workload_admission::workload_admission_registry_snapshot; use rustfs_concurrency::{AdmissionState, WorkloadAdmissionRegistrySnapshot}; @@ -115,7 +115,7 @@ pub fn cluster_has_actionable_pressure(snapshot: &ClusterReadOnlySnapshot) -> bo #[cfg(test)] mod tests { use super::*; - use crate::storage_api::{ + use crate::storage_api::cluster::{ CapabilityState, CapabilityStatus, DiskCapabilities, Endpoint, Endpoints, PoolEndpoints, TopologyCapabilities, }; use rustfs_concurrency::{WorkloadAdmissionSnapshot, WorkloadClass}; diff --git a/rustfs/src/config/config_test.rs b/rustfs/src/config/config_test.rs index 7350f00b3..71b81b26c 100644 --- a/rustfs/src/config/config_test.rs +++ b/rustfs/src/config/config_test.rs @@ -17,7 +17,7 @@ mod tests { use crate::config::cli::default_server_opts; use crate::config::{CommandResult, Config, Opt, TlsCommands}; - use crate::storage_api::DisksLayout; + use crate::storage_api::config_test::DisksLayout; use rustfs_config::{DEFAULT_CONSOLE_ADDRESS, DEFAULT_CONSOLE_ENABLE, DEFAULT_OBS_ENDPOINT, RUSTFS_REGION}; use rustfs_credentials::{DEFAULT_ACCESS_KEY, DEFAULT_SECRET_KEY}; use serial_test::serial; @@ -275,7 +275,7 @@ mod tests { #[test] #[serial] fn test_volumes_and_disk_layout_parsing() { - use crate::storage_api::DisksLayout; + use crate::storage_api::config_test::DisksLayout; // Test case 1: Single volume path let args = vec!["rustfs", "/data/vol1"]; diff --git a/rustfs/src/error.rs b/rustfs/src/error.rs index bd20be6c0..3310ef78e 100644 --- a/rustfs/src/error.rs +++ b/rustfs/src/error.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_api::{HTTPRangeError, QuotaError, StorageError}; +use crate::storage_api::error::{HTTPRangeError, QuotaError, StorageError}; use s3s::{S3Error, S3ErrorCode}; #[derive(Debug)] diff --git a/rustfs/src/init.rs b/rustfs/src/init.rs index 93743d706..e0d0e6a26 100644 --- a/rustfs/src/init.rs +++ b/rustfs/src/init.rs @@ -14,7 +14,7 @@ use crate::runtime_sources::{resolve_notify_interface, resolve_region}; use crate::server::ShutdownHandle; -use crate::storage_api::{ +use crate::storage_api::startup::{ get_bucket_notification_config, process_lambda_configurations, process_queue_configurations, process_topic_configurations, }; use crate::{admin, config, startup_runtime_sources, version}; @@ -717,7 +717,7 @@ where /// When enabled, it spawns a background task that tunes concurrency settings /// every 60 seconds. pub async fn init_auto_tuner(ctx: tokio_util::sync::CancellationToken) { - use crate::storage_api::concurrency::get_concurrency_manager; + use crate::storage_api::startup::concurrency::get_concurrency_manager; use rustfs_io_metrics::AutoTuner; use rustfs_io_metrics::TunerConfig; use tracing::{debug, error, info}; @@ -850,7 +850,7 @@ pub async fn init_ftp_system() -> Result, Box = FtpsServer::new(config, storage_client).await?; let bind_addr = server.config().bind_addr; @@ -970,7 +970,7 @@ pub async fn init_ftps_system() -> Result, Box = FtpsServer::new(config, storage_client).await?; let bind_addr = server.config().bind_addr; @@ -1088,7 +1088,7 @@ pub async fn init_webdav_system() -> Result, Box = WebDavServer::new(config, storage_client).await?; let bind_addr = server.config().bind_addr; @@ -1222,7 +1222,7 @@ pub async fn init_sftp_system() -> Result, Box CapabilityStatus { #[cfg(test)] mod tests { use super::*; - use crate::storage_api::{ + use crate::storage_api::cluster::{ CapabilityState, Endpoint, Endpoints, ObservabilitySnapshotProvider, PoolEndpoints, TopologySnapshotProvider, }; diff --git a/rustfs/src/server/event.rs b/rustfs/src/server/event.rs index 6c5f02d18..b9b54e3fe 100644 --- a/rustfs/src/server/event.rs +++ b/rustfs/src/server/event.rs @@ -13,7 +13,7 @@ // limitations under the License. use super::{module_switch::resolve_notify_module_state, refresh_persisted_module_switches_from_store, runtime_sources}; -use crate::storage_api::{EventArgs as EcstoreEventArgs, StorageObjectInfo, register_event_dispatch_hook}; +use crate::storage_api::server::{EventArgs as EcstoreEventArgs, StorageObjectInfo, register_event_dispatch_hook}; use chrono::{DateTime, Utc}; use rustfs_notify::{EventArgs as NotifyEventArgs, NotifyObjectInfo}; use rustfs_s3_types::EventName; @@ -234,7 +234,7 @@ pub async fn init_event_notifier() { #[cfg(test)] mod tests { use super::{convert_ecstore_object_info, parse_host_and_port}; - use crate::storage_api::{StorageObjectInfo, TransitionedObject}; + use crate::storage_api::server::{StorageObjectInfo, TransitionedObject}; use chrono::{DateTime, Utc}; use std::{collections::HashMap, sync::Arc}; use time::{Duration, OffsetDateTime}; diff --git a/rustfs/src/server/http.rs b/rustfs/src/server/http.rs index d1c42d5cb..5028f9dda 100644 --- a/rustfs/src/server/http.rs +++ b/rustfs/src/server/http.rs @@ -30,11 +30,11 @@ use crate::server::{ TlsAcceptorHolder, TlsHandshakeFailureKind, build_acceptor_from_loaded, load_tls_material, spawn_reload_loop, }, }; -use crate::storage_api as storage; -use crate::storage_api::request_context::{RequestContext, extract_request_id_from_headers}; -use crate::storage_api::rpc::InternodeRpcService; -use crate::storage_api::tonic_service::make_server; -use crate::storage_api::{TONIC_RPC_PREFIX, verify_rpc_signature}; +use crate::storage_api::server as storage; +use crate::storage_api::server::request_context::{RequestContext, extract_request_id_from_headers}; +use crate::storage_api::server::rpc::InternodeRpcService; +use crate::storage_api::server::tonic_service::make_server; +use crate::storage_api::server::{TONIC_RPC_PREFIX, verify_rpc_signature}; use bytes::Bytes; use http::{HeaderMap, Method, Request as HttpRequest, Response}; use hyper::body::Incoming; @@ -1104,7 +1104,7 @@ fn process_connection( .layer( TraceLayer::new_for_http() .make_span_with(|request: &HttpRequest<_>| { - let request_context = request.extensions().get::(); + let request_context = request.extensions().get::(); let request_id = request_context .map(|ctx| ctx.request_id.as_str()) .unwrap_or("unknown"); @@ -1253,7 +1253,7 @@ fn process_connection( .layer( TraceLayer::new_for_http() .make_span_with(|request: &HttpRequest<_>| { - let request_context = request.extensions().get::(); + let request_context = request.extensions().get::(); let request_id = request_context .map(|ctx| ctx.request_id.as_str()) .unwrap_or("unknown"); diff --git a/rustfs/src/server/layer.rs b/rustfs/src/server/layer.rs index 43995203b..54abd4a7b 100644 --- a/rustfs/src/server/layer.rs +++ b/rustfs/src/server/layer.rs @@ -25,8 +25,8 @@ use crate::server::{ RUSTFS_ADMIN_PREFIX, active_http_requests, collect_dependency_readiness_report, has_path_prefix, is_admin_path, is_table_catalog_path, }; -use crate::storage_api::apply_cors_headers; -use crate::storage_api::request_context::{ +use crate::storage_api::server::apply_cors_headers; +use crate::storage_api::server::request_context::{ RequestContext, extract_request_id_from_headers, extract_trace_context_ids_from_headers, spawn_traced, }; use bytes::Bytes; diff --git a/rustfs/src/server/module_switch.rs b/rustfs/src/server/module_switch.rs index 22618f5a5..d5fccca39 100644 --- a/rustfs/src/server/module_switch.rs +++ b/rustfs/src/server/module_switch.rs @@ -13,7 +13,7 @@ // limitations under the License. use super::runtime_sources; -use crate::storage_api::{Error as StorageError, read_config, save_config}; +use crate::storage_api::server::{Error as StorageError, read_config, save_config}; use serde::{Deserialize, Serialize}; use std::sync::atomic::{AtomicBool, Ordering}; diff --git a/rustfs/src/server/readiness.rs b/rustfs/src/server/readiness.rs index 9bcbb4066..e737caf07 100644 --- a/rustfs/src/server/readiness.rs +++ b/rustfs/src/server/readiness.rs @@ -15,9 +15,9 @@ use crate::server::runtime_sources; use crate::server::{ServiceState, ServiceStateManager}; use crate::server::{has_path_prefix, is_table_catalog_path}; -use crate::storage_api::{Endpoint, EndpointServerPools, StorageAdminApi, is_dist_erasure}; +use crate::storage_api::server::{Endpoint, EndpointServerPools, StorageAdminApi, is_dist_erasure}; #[cfg(test)] -use crate::storage_api::{Endpoints, PoolEndpoints}; +use crate::storage_api::server::{Endpoints, PoolEndpoints}; use bytes::Bytes; use http::{Request as HttpRequest, Response, StatusCode}; use http_body::Body; diff --git a/rustfs/src/server/runtime_sources.rs b/rustfs/src/server/runtime_sources.rs index 47f9f50d3..651e44e1b 100644 --- a/rustfs/src/server/runtime_sources.rs +++ b/rustfs/src/server/runtime_sources.rs @@ -16,7 +16,7 @@ use crate::runtime_sources::{ NotifyInterface, resolve_endpoints_handle, resolve_iam_ready, resolve_kms_runtime_service_manager, resolve_lock_clients_handle, resolve_notify_interface, resolve_object_store_handle, resolve_server_config, }; -use crate::storage_api::{ECStore, EndpointServerPools}; +use crate::storage_api::server::{ECStore, EndpointServerPools}; use rustfs_kms::KmsServiceManager; use rustfs_lock::LockClient; use std::collections::HashMap; diff --git a/rustfs/src/startup_background.rs b/rustfs/src/startup_background.rs index ba2514d34..8f855b117 100644 --- a/rustfs/src/startup_background.rs +++ b/rustfs/src/startup_background.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_api::ECStore; +use crate::storage_api::startup::ECStore; use rustfs_heal::{create_ahm_services_cancel_token, heal::storage::ECStoreHealStorage, init_heal_manager}; use rustfs_utils::get_env_bool_with_aliases; use std::{io::Result, sync::Arc}; diff --git a/rustfs/src/startup_bucket_metadata.rs b/rustfs/src/startup_bucket_metadata.rs index d62fe1c26..b17a68314 100644 --- a/rustfs/src/startup_bucket_metadata.rs +++ b/rustfs/src/startup_bucket_metadata.rs @@ -13,7 +13,7 @@ // limitations under the License. use crate::startup_runtime_sources; -use crate::storage_api::{ +use crate::storage_api::startup::{ BucketOperations, BucketOptions, ECStore, init_bucket_metadata_sys, try_migrate_bucket_metadata, try_migrate_iam_config, }; use std::{ diff --git a/rustfs/src/startup_deadlock.rs b/rustfs/src/startup_deadlock.rs index ef633d3c3..2ba2820cc 100644 --- a/rustfs/src/startup_deadlock.rs +++ b/rustfs/src/startup_deadlock.rs @@ -19,7 +19,7 @@ const LOG_SUBSYSTEM_STARTUP: &str = "startup"; const EVENT_DEADLOCK_DETECTOR_STATE: &str = "deadlock_detector_state"; pub(crate) fn init_deadlock_detector_runtime() { - let detector = crate::storage_api::deadlock_detector::get_deadlock_detector(); + let detector = crate::storage_api::startup::deadlock_detector::get_deadlock_detector(); if detector.is_enabled() { detector.start(); info!( diff --git a/rustfs/src/startup_fs_guard.rs b/rustfs/src/startup_fs_guard.rs index a6d549184..4440d7efc 100644 --- a/rustfs/src/startup_fs_guard.rs +++ b/rustfs/src/startup_fs_guard.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_api::EndpointServerPools; +use crate::storage_api::startup::EndpointServerPools; use rustfs_config::{ DEFAULT_RUSTFS_UNSUPPORTED_FS_POLICY, ENV_RUSTFS_UNSUPPORTED_FS_POLICY, RUSTFS_UNSUPPORTED_FS_POLICY_FAIL, RUSTFS_UNSUPPORTED_FS_POLICY_WARN, diff --git a/rustfs/src/startup_iam.rs b/rustfs/src/startup_iam.rs index ddbe91598..4ea5e1ffb 100644 --- a/rustfs/src/startup_iam.rs +++ b/rustfs/src/startup_iam.rs @@ -14,7 +14,7 @@ use crate::runtime_sources::AppContext; use crate::server::{ServiceStateManager, publish_ready_when_runtime_ready}; -use crate::storage_api::ECStore; +use crate::storage_api::startup::ECStore; use rustfs_common::{GlobalReadiness, SystemStage}; use rustfs_iam::init_iam_sys; use rustfs_kms::KmsServiceManager; diff --git a/rustfs/src/startup_lifecycle.rs b/rustfs/src/startup_lifecycle.rs index fd81df0c7..b23746ab6 100644 --- a/rustfs/src/startup_lifecycle.rs +++ b/rustfs/src/startup_lifecycle.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_api::ECStore; +use crate::storage_api::startup::ECStore; use crate::{ server::{ServiceStateManager, ShutdownHandle, wait_for_shutdown}, startup_iam::{IamBootstrapDisposition, publish_ready_for_iam_bootstrap}, diff --git a/rustfs/src/startup_notification.rs b/rustfs/src/startup_notification.rs index 46aafee0d..143e596e9 100644 --- a/rustfs/src/startup_notification.rs +++ b/rustfs/src/startup_notification.rs @@ -13,7 +13,7 @@ // limitations under the License. use crate::init::add_bucket_notification_configuration; -use crate::storage_api::{EndpointServerPools, Result as StorageResult, new_global_notification_sys}; +use crate::storage_api::startup::{EndpointServerPools, Result as StorageResult, new_global_notification_sys}; use std::{ future::Future, io::{Error, Result}, @@ -76,7 +76,7 @@ fn log_embedded_optional_service_skipped(service: &str, err: impl std::fmt::Disp #[cfg(test)] mod tests { use super::init_notification_system_with; - use crate::storage_api::Error as EcstoreError; + use crate::storage_api::startup::Error as EcstoreError; #[tokio::test] async fn notification_system_returns_source_error() { diff --git a/rustfs/src/startup_runtime_sources.rs b/rustfs/src/startup_runtime_sources.rs index e72da65c5..76dcfc46f 100644 --- a/rustfs/src/startup_runtime_sources.rs +++ b/rustfs/src/startup_runtime_sources.rs @@ -14,7 +14,7 @@ use crate::config::RustFSBufferConfig; use crate::runtime_sources::{resolve_outbound_tls_generation, resolve_replication_pool_handle}; -use crate::storage_api::{DynReplicationPool, set_global_region, set_global_rustfs_port}; +use crate::storage_api::startup::{DynReplicationPool, set_global_region, set_global_rustfs_port}; use rustfs_kms::KmsServiceManager; use rustfs_obs::{GlobalError as ObservabilityError, OtelGuard}; use rustfs_tls_runtime::{OutboundTlsMaterial, TlsGeneration}; diff --git a/rustfs/src/startup_services.rs b/rustfs/src/startup_services.rs index 046f00117..c879b9715 100644 --- a/rustfs/src/startup_services.rs +++ b/rustfs/src/startup_services.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_api::{ECStore, EndpointServerPools}; +use crate::storage_api::startup::{ECStore, EndpointServerPools}; use crate::{ config::Config, init::{init_buffer_profile_system, init_kms_system}, diff --git a/rustfs/src/startup_shutdown.rs b/rustfs/src/startup_shutdown.rs index 248812af3..d8466b019 100644 --- a/rustfs/src/startup_shutdown.rs +++ b/rustfs/src/startup_shutdown.rs @@ -12,7 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. -use crate::storage_api::shutdown_background_services; +use crate::storage_api::startup::shutdown_background_services; use crate::{ server::{ServiceState, ServiceStateManager, ShutdownHandle, ShutdownSignal, shutdown_event_notifier, stop_audit_system}, startup_optional_runtime_sidecars::{ diff --git a/rustfs/src/startup_storage.rs b/rustfs/src/startup_storage.rs index 4833c9c00..8c3f11775 100644 --- a/rustfs/src/startup_storage.rs +++ b/rustfs/src/startup_storage.rs @@ -13,7 +13,7 @@ // limitations under the License. use crate::startup_fs_guard::enforce_unsupported_fs_policy; -use crate::storage_api::{ +use crate::storage_api::startup::{ ECStore, EndpointServerPools, init_background_replication, init_ecstore_config, init_global_config_sys, init_local_disks, init_lock_clients, prewarm_local_disk_id_map, set_global_endpoints, try_migrate_server_config, update_erasure_type, }; diff --git a/rustfs/src/storage_api.rs b/rustfs/src/storage_api.rs index 71976fb8f..d73a6e827 100644 --- a/rustfs/src/storage_api.rs +++ b/rustfs/src/storage_api.rs @@ -12,69 +12,135 @@ // See the License for the specific language governing permissions and // limitations under the License. -//! Root-local boundary for storage owner APIs used by startup, server, and -//! other RustFS outer runtime modules. +//! Root-local domain boundaries for storage owner APIs used by RustFS outer +//! runtime modules. -pub(crate) use crate::storage::{ - BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG, - BUCKET_TABLE_RESERVED_PREFIX, DynReplicationPool, ECStore, Endpoint, EndpointServerPools, Error, EventArgs, QuotaError, - RUSTFS_META_BUCKET, Result, StorageDeletedObject, StorageError, StorageGetObjectReader, StorageObjectInfo, - StorageObjectOptions, StorageObjectToDelete, StoragePutObjReader, TONIC_RPC_PREFIX, all_local_disk, apply_cors_headers, - bucket_metadata_runtime_initialized, disk_drive_path, disk_endpoint, get_bucket_metadata, get_bucket_notification_config, - get_lock_acquire_timeout, init_background_replication, init_bucket_metadata_sys, init_ecstore_config, init_global_config_sys, - init_local_disks, init_lock_clients, is_dist_erasure, new_global_notification_sys, prewarm_local_disk_id_map, - process_lambda_configurations, process_queue_configurations, process_topic_configurations, read_config, - register_event_dispatch_hook, replication_queue_current_count, save_config, set_global_endpoints, set_global_region, - set_global_rustfs_port, shutdown_background_services, table_catalog_path_hash, - topology_snapshot_from_endpoint_pools_with_capabilities, try_migrate_bucket_metadata, try_migrate_iam_config, - try_migrate_server_config, update_erasure_type, verify_rpc_signature, -}; -pub(crate) use rustfs_storage_api::{ - BucketOperations, BucketOptions, CapabilitySnapshotError, CapabilityStatus, DiskCapabilities, HTTPPreconditions, - HTTPRangeError, HTTPRangeSpec, ListObjectVersionsInfo, ListObjectsV2Info, ListOperations, MemorySamplingState, - NamespaceLocking, ObjectIO, ObjectInfoOrErr, ObjectOperations, ObservabilitySnapshot, ObservabilitySnapshotProvider, - PlatformSupport, StorageAdminApi, TopologyCapabilities, TopologySnapshot, TopologySnapshotProvider, - UserspaceProfilingCapability, WalkOptions, -}; +use rustfs_storage_api as storage_contracts; + +pub(crate) mod capacity { + pub(crate) use crate::storage::{all_local_disk, disk_drive_path, disk_endpoint}; +} + +pub(crate) mod cluster { + pub(crate) use super::storage_contracts::{ + CapabilitySnapshotError, CapabilityStatus, DiskCapabilities, MemorySamplingState, ObservabilitySnapshot, + ObservabilitySnapshotProvider, PlatformSupport, TopologyCapabilities, TopologySnapshot, TopologySnapshotProvider, + UserspaceProfilingCapability, + }; + pub(crate) use crate::storage::{EndpointServerPools, topology_snapshot_from_endpoint_pools_with_capabilities}; + + #[cfg(test)] + pub(crate) use super::storage_contracts::CapabilityState; + #[cfg(test)] + pub(crate) use crate::storage::{Endpoint, Endpoints, PoolEndpoints}; + + pub(crate) mod control_plane { + pub(crate) use crate::storage::ecstore_cluster::{ + ClusterControlPlane, ClusterControlPlaneSnapshot, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot, + ClusterPeerHealthSnapshot, ClusterPoolStateSnapshot, + }; + } +} #[cfg(test)] -pub(crate) use crate::storage::{DisksLayout, Endpoints, PoolEndpoints}; -#[cfg(test)] -pub(crate) use rustfs_storage_api::{CapabilityState, TransitionedObject}; - -pub(crate) mod access { - pub(crate) use crate::storage::access::ReqInfo; +pub(crate) mod config_test { + pub(crate) use crate::storage::DisksLayout; } -pub(crate) mod concurrency { - pub(crate) use crate::storage::concurrency::get_concurrency_manager; +pub(crate) mod error { + pub(crate) use super::storage_contracts::HTTPRangeError; + pub(crate) use crate::storage::{QuotaError, StorageError}; } -pub(crate) mod deadlock_detector { - pub(crate) use crate::storage::deadlock_detector::get_deadlock_detector; +pub(crate) mod protocols { + pub(crate) mod access { + pub(crate) use crate::storage::access::ReqInfo; + } + + pub(crate) mod ecfs { + pub(crate) type FS = crate::storage::FS; + } + + pub(crate) mod request_context { + pub(crate) use crate::storage::request_context::RequestContext; + } } -pub(crate) mod ecfs { - pub(crate) type FS = crate::storage::FS; +pub(crate) mod server { + pub(crate) use super::storage_contracts::StorageAdminApi; + pub(crate) use crate::storage::{ + ECStore, Endpoint, EndpointServerPools, Error, EventArgs, StorageObjectInfo, TONIC_RPC_PREFIX, apply_cors_headers, + is_dist_erasure, read_config, register_event_dispatch_hook, save_config, verify_rpc_signature, + }; + + #[cfg(test)] + pub(crate) use super::storage_contracts::TransitionedObject; + #[cfg(test)] + pub(crate) use crate::storage::{Endpoints, PoolEndpoints}; + + pub(crate) mod ecfs { + pub(crate) type FS = crate::storage::FS; + } + + pub(crate) mod request_context { + pub(crate) use crate::storage::request_context::{ + RequestContext, extract_request_id_from_headers, extract_trace_context_ids_from_headers, spawn_traced, + }; + } + + pub(crate) mod rpc { + pub(crate) use crate::storage::rpc::InternodeRpcService; + } + + pub(crate) mod tonic_service { + pub(crate) use crate::storage::tonic_service::make_server; + } } -pub(crate) mod ecstore_cluster { - pub(crate) use crate::storage::ecstore_cluster::{ - ClusterControlPlane, ClusterControlPlaneSnapshot, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot, - ClusterPeerHealthSnapshot, ClusterPoolStateSnapshot, +pub(crate) mod startup { + pub(crate) use super::storage_contracts::{BucketOperations, BucketOptions}; + pub(crate) use crate::storage::{ + DynReplicationPool, ECStore, EndpointServerPools, Result, get_bucket_notification_config, init_background_replication, + init_bucket_metadata_sys, init_ecstore_config, init_global_config_sys, init_local_disks, init_lock_clients, + new_global_notification_sys, prewarm_local_disk_id_map, process_lambda_configurations, process_queue_configurations, + process_topic_configurations, set_global_endpoints, set_global_region, set_global_rustfs_port, + shutdown_background_services, try_migrate_bucket_metadata, try_migrate_iam_config, try_migrate_server_config, + update_erasure_type, + }; + + #[cfg(test)] + pub(crate) use crate::storage::Error; + + pub(crate) mod concurrency { + pub(crate) use crate::storage::concurrency::get_concurrency_manager; + } + + pub(crate) mod deadlock_detector { + pub(crate) use crate::storage::deadlock_detector::get_deadlock_detector; + } + + pub(crate) mod ecfs { + pub(crate) type FS = crate::storage::FS; + } +} + +pub(crate) mod table { + pub(crate) use super::storage_contracts::{ + HTTPPreconditions, HTTPRangeSpec, ListObjectVersionsInfo, ListObjectsV2Info, ListOperations, NamespaceLocking, ObjectIO, + ObjectInfoOrErr, ObjectOperations, WalkOptions, + }; + pub(crate) use crate::storage::{ + BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG, + BUCKET_TABLE_RESERVED_PREFIX, Error, RUSTFS_META_BUCKET, StorageDeletedObject, StorageError, StorageGetObjectReader, + StorageObjectInfo, StorageObjectOptions, StorageObjectToDelete, StoragePutObjReader, get_bucket_metadata, + get_lock_acquire_timeout, table_catalog_path_hash, }; } -pub(crate) mod request_context { - pub(crate) use crate::storage::request_context::{ - RequestContext, extract_request_id_from_headers, extract_trace_context_ids_from_headers, spawn_traced, - }; -} +pub(crate) mod workload { + pub(crate) use crate::storage::{bucket_metadata_runtime_initialized, replication_queue_current_count}; -pub(crate) mod rpc { - pub(crate) use crate::storage::rpc::InternodeRpcService; -} - -pub(crate) mod tonic_service { - pub(crate) use crate::storage::tonic_service::make_server; + pub(crate) mod concurrency { + pub(crate) use crate::storage::concurrency::get_concurrency_manager; + } } diff --git a/rustfs/src/table_catalog.rs b/rustfs/src/table_catalog.rs index 2c344fb22..537a15841 100644 --- a/rustfs/src/table_catalog.rs +++ b/rustfs/src/table_catalog.rs @@ -27,7 +27,7 @@ use std::{ time::{Duration as StdDuration, Instant}, }; -use crate::storage_api::{ +use crate::storage_api::table::{ BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG, BUCKET_TABLE_RESERVED_PREFIX, Error as EcstoreError, HTTPPreconditions, HTTPRangeSpec, ListObjectVersionsInfo as StorageListObjectVersionsInfo, ListObjectsV2Info as StorageListObjectsV2Info, @@ -48,7 +48,7 @@ use time::{Duration, OffsetDateTime}; use tokio::io::AsyncReadExt; use uuid::Uuid; -use crate::storage_api::{ +use crate::storage_api::table::{ StorageDeletedObject as DeletedObject, StorageGetObjectReader as GetObjectReader, StorageObjectInfo as ObjectInfo, StorageObjectOptions as ObjectOptions, StorageObjectToDelete as ObjectToDelete, StoragePutObjReader as PutObjReader, }; diff --git a/rustfs/src/workload_admission.rs b/rustfs/src/workload_admission.rs index c88472d0e..eadc3ce17 100644 --- a/rustfs/src/workload_admission.rs +++ b/rustfs/src/workload_admission.rs @@ -13,13 +13,13 @@ // limitations under the License. use crate::runtime_sources::resolve_replication_pool_handle; -use crate::storage_api::{bucket_metadata_runtime_initialized, replication_queue_current_count}; +use crate::storage_api::workload::{bucket_metadata_runtime_initialized, replication_queue_current_count}; use rustfs_concurrency::{ AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadAdmissionSnapshotProvider, WorkloadClass, }; -use crate::storage_api::concurrency::get_concurrency_manager; +use crate::storage_api::workload::concurrency::get_concurrency_manager; const BUCKET_METADATA_RUNTIME_NOT_INITIALIZED: &str = "bucket metadata runtime not initialized"; const FOREGROUND_WRITE_NOT_EXPOSED_BY_PROVIDER: &str = "foreground write admission not yet exposed by RustFS runtime"; diff --git a/scripts/check_architecture_migration_rules.sh b/scripts/check_architecture_migration_rules.sh index b3856a756..e0e0e3f04 100755 --- a/scripts/check_architecture_migration_rules.sh +++ b/scripts/check_architecture_migration_rules.sh @@ -183,6 +183,8 @@ RUSTFS_APP_ADMIN_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_admin_stora RUSTFS_ADMIN_STORAGE_API_ROOT_FACADE_HITS_FILE="${TMP_DIR}/rustfs_admin_storage_api_root_facade_hits.txt" RUSTFS_ROOT_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_root_storage_api_bypass_hits.txt" RUSTFS_ROOT_STORAGE_API_CONTRACT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_root_storage_api_contract_bypass_hits.txt" +RUSTFS_ROOT_STORAGE_API_DOMAIN_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_root_storage_api_domain_bypass_hits.txt" +RUSTFS_ADMIN_STORAGE_API_DOMAIN_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_admin_storage_api_domain_bypass_hits.txt" RUSTFS_APP_STORAGE_API_CONTRACT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_storage_api_contract_bypass_hits.txt" RUSTFS_ADMIN_STORAGE_API_CONTRACT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_admin_storage_api_contract_bypass_hits.txt" RUSTFS_STORAGE_DIRECT_APP_CONTEXT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_storage_direct_app_context_bypass_hits.txt" @@ -1708,6 +1710,32 @@ if [[ -s "$RUSTFS_ROOT_STORAGE_API_CONTRACT_BYPASS_HITS_FILE" ]]; then report_failure "RustFS root/server/startup storage contracts must stay behind rustfs/src/storage_api.rs: $(paste -sd '; ' "$RUSTFS_ROOT_STORAGE_API_CONTRACT_BYPASS_HITS_FILE")" fi +( + cd "$ROOT_DIR" + rg -n --with-filename \ + 'use crate::storage_api(?:\s*(?:;|as\b)|::\{)|crate::storage_api::(?:access|concurrency|deadlock_detector|ecfs|ecstore_cluster|request_context|rpc|tonic_service)\b' \ + rustfs/src \ + --glob '*.rs' \ + --glob '!rustfs/src/storage_api.rs' || true +) >"$RUSTFS_ROOT_STORAGE_API_DOMAIN_BYPASS_HITS_FILE" + +if [[ -s "$RUSTFS_ROOT_STORAGE_API_DOMAIN_BYPASS_HITS_FILE" ]]; then + report_failure "RustFS root storage-api consumers must use domain modules from rustfs/src/storage_api.rs: $(paste -sd '; ' "$RUSTFS_ROOT_STORAGE_API_DOMAIN_BYPASS_HITS_FILE")" +fi + +( + cd "$ROOT_DIR" + rg -n --with-filename \ + '(?:crate::admin|super)::storage_api::(?:ecstore_cluster|ecstore_utils|Error|StorageError|ECStore|Endpoint|EndpointServerPools|Endpoints|PoolEndpoints|PeerRestClient|RequestContext|ReqInfo|authorize_request|spawn_traced|CollectMetricsOpts|MetricType|collect_local_metrics|BucketOperations|BucketOptions|DeleteBucketOptions|HealOperations|ListOperations|MakeBucketOptions|ObjectIO|ObjectOperations|StorageAdminApi|CapabilitySnapshotError|CapabilityState|CapabilityStatus|ObservabilitySnapshot|ObservabilitySnapshotProvider|TopologySnapshot|TopologySnapshotProvider|read_admin_config|read_admin_config_without_migrate|save_admin_config|save_admin_server_config|delete_admin_config|init_admin_config_defaults|STORAGE_CLASS_SUB_SYS|RebalanceStats|RebalanceMeta|RebalanceStopPropagationRecord|StorageObjectOptions|is_reserved_or_invalid_bucket)\b|use (?:crate::admin|super)::storage_api::\{' \ + rustfs/src/admin \ + --glob '*.rs' \ + --glob '!rustfs/src/admin/storage_api.rs' || true +) >"$RUSTFS_ADMIN_STORAGE_API_DOMAIN_BYPASS_HITS_FILE" + +if [[ -s "$RUSTFS_ADMIN_STORAGE_API_DOMAIN_BYPASS_HITS_FILE" ]]; then + report_failure "RustFS admin storage-api consumers must use admin domain modules from rustfs/src/admin/storage_api.rs: $(paste -sd '; ' "$RUSTFS_ADMIN_STORAGE_API_DOMAIN_BYPASS_HITS_FILE")" +fi + ( cd "$ROOT_DIR" rg -n --with-filename '^use rustfs_storage_api|rustfs_storage_api::' \