gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-27 16:48:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: segment storage api domain boundaries (#3901)

Browse Source
This commit is contained in:
Zhengchao An
2026-06-26 14:04:28 +08:00
committed by GitHub
parent 92c50156a3
commit 2a1bddfcca
55 changed files with 495 additions and 236 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/migration-progress.md
+83 -16
View File
@@ -5,15 +5,15 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-storage-ecfs-usecase-boundary`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215/API-216/API-217/API-218/API-219/API-220/API-221/API-222/API-223/API-224/API-225/API-226/API-227/API-228/API-229/CTX-002`.
- Based on: stacked on API-232 local branch while API-230 PR #3894 is pending;
branch routes storage ECFS S3 route app usecase construction through the
storage S3 API boundary.
- Branch: `overtrue/arch-root-storage-api-domain-boundary`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215/API-216/API-217/API-218/API-219/API-220/API-221/API-222/API-223/API-224/API-225/API-226/API-227/API-228/API-229/API-230/API-231/API-232/API-233/API-234/API-235/API-236/CTX-002`.
- Based on: PR #3899 has merged; branch segments the root-local and admin-local
storage API boundaries by consumer domain.
- PR type for this branch: `consumer-migration`
- Runtime behavior changes: none expected for API-233; storage ECFS S3 routes
still construct the same AppContext-backed bucket, multipart, and object
usecases.
- Runtime behavior changes: none expected for API-237/API-238; root, server,
startup, table, protocol, cluster, capacity, workload, config-test, error,
admin handler, admin service, and admin router consumers still use the same
owner symbols through narrower domain modules.
- Rust code changes: route replication pool, outbound TLS generation, runtime
region, KMS encryption service, runtime support handles, S3 Select DB,
internode RPC metrics, IAM authorization/handler reads, notification
@@ -70,8 +70,13 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
ECStore internal storage contract imports through the owner-local
`storage_api_contracts` boundary, admin system, pool, cluster snapshot,
plugin catalog, table catalog, module-switch, and console admin discovery
`DefaultAdminUsecase` construction through `admin::runtime_sources`, and
storage ECFS S3 route app usecase construction through `storage::s3_api`.
`DefaultAdminUsecase` construction through `admin::runtime_sources`, storage
ECFS S3 route app usecase construction through `storage::s3_api`, root
storage API consumers through domain modules for startup, server, cluster,
table, protocols, capacity, workload, config tests, and error mapping, and
admin storage API consumers through admin domain modules for access, bucket,
cluster, config, contract, error, metrics, object, rebalance, runtime, and
tier boundaries.
- CI/script changes: lock completed owner and test/fuzz boundaries against
bare/glob imports, scattered raw ECStore facade subpaths, and startup
runtime/root-server/table/S3/app shared/app bucket/app ECStore/admin facade
@@ -81,10 +86,10 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
event-bridge thin module regressions, plus IAM runtime-source bypasses;
accept the reviewed AppContext resolver reverse dependencies in the layer
baseline, and block direct admin AppContext resolver consumers outside the
admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, reject app usecase storage wildcard imports, reject app-layer S3 DTO and ECFS wildcard imports, narrow the object-usecase ECFS layer baseline entry to `FS`, reject direct storage S3 API helper imports from app usecase files, reject direct storage helper imports from app select/usecase files, reject completed app/admin storage helper bypasses, reject app usecase bypasses for migrated storage IO/compression/set-disk helpers, reject app usecase/test bypasses for migrated storage error, ETag, and storage-class helpers, reject app root bucket owner facade bypasses from migrated app consumers, reject app/admin runtime/data-usage root facade regressions, reject admin root storage facade regressions from migrated admin consumers, reject root/server/startup direct storage facade regressions from migrated outer consumers, reject root/server/startup direct storage contract imports from migrated outer consumers, reject app/admin direct storage contract imports from migrated owner consumers, keep app S3 helper imports routed through `app::storage_api`, reject scanner/heal direct ECStore or storage contract imports outside their local `storage_api` boundaries, reject external runtime/test/fuzz ECStore or storage contract imports outside their local `storage_api` boundaries, reject storage owner direct ECStore/storage-api imports outside the owner-local `storage_api` boundary, reject ECStore internal direct storage-api imports outside the owner-local `storage_api_contracts` boundary, and reject direct storage ECFS app usecase construction outside the storage S3 API boundary.
- Docs changes: record the API-136 through API-226 owner facade and lifecycle
runtime-source cleanup plus API-233 storage ECFS usecase construction
boundary.
admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, reject app usecase storage wildcard imports, reject app-layer S3 DTO and ECFS wildcard imports, narrow the object-usecase ECFS layer baseline entry to `FS`, reject direct storage S3 API helper imports from app usecase files, reject direct storage helper imports from app select/usecase files, reject completed app/admin storage helper bypasses, reject app usecase bypasses for migrated storage IO/compression/set-disk helpers, reject app usecase/test bypasses for migrated storage error, ETag, and storage-class helpers, reject app root bucket owner facade bypasses from migrated app consumers, reject app/admin runtime/data-usage root facade regressions, reject admin root storage facade regressions from migrated admin consumers, reject root/server/startup direct storage facade regressions from migrated outer consumers, reject root/server/startup direct storage contract imports from migrated outer consumers, reject app/admin direct storage contract imports from migrated owner consumers, keep app S3 helper imports routed through `app::storage_api`, reject scanner/heal direct ECStore or storage contract imports outside their local `storage_api` boundaries, reject external runtime/test/fuzz ECStore or storage contract imports outside their local `storage_api` boundaries, reject storage owner direct ECStore/storage-api imports outside the owner-local `storage_api` boundary, reject ECStore internal direct storage-api imports outside the owner-local `storage_api_contracts` boundary, reject direct storage ECFS app usecase construction outside the storage S3 API boundary, reject flat root `storage_api` imports outside the new root-local domain modules, and reject flat admin `storage_api` imports outside the new admin domain modules.
- Docs changes: record the API-136 through API-238 owner facade,
runtime-source, ECFS usecase, root storage API domain-boundary, and admin
storage API domain-boundary cleanup.
## Phase 0 Tasks
@@ -5454,15 +5459,56 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
layer guards, diff hygiene, residual owner runtime-source AppContext scan,
Rust risk scan, and full PR gate before PR.
- [x] `API-237` Segment root storage API boundary by outer runtime domain.
- Do: replace the flat root-local `storage_api` re-export surface with
domain modules for startup, server, cluster, table, protocols, capacity,
workload, config tests, and error mapping, then migrate root/server/startup
consumers to those modules.
- Acceptance: root consumers no longer import flat `crate::storage_api`
symbols or legacy nested helper modules directly, storage contracts are
still imported once through the root boundary, and migration rules reject
flat root `storage_api` bypasses.
- Must preserve: startup storage/bootstrap behavior, notification config
wiring, HTTP/gRPC request context handling, readiness checks, cluster and
topology snapshots, table catalog object I/O contracts, workload admission,
capacity reporting, protocol request setup, config tests, and error mapping.
- Verification: focused RustFS compile, formatting, migration and layer
guards, diff hygiene, residual flat root storage-api scan, Rust risk scan,
and full PR gate passed before PR.
- [x] `API-238` Segment admin storage API boundary by admin consumer domain.
- Do: add admin-local domain modules for access, bucket, cluster, config,
contract, error, metrics, object, rebalance, runtime, and tier symbols,
then migrate admin handlers, services, router, and console consumers away
from flat `admin::storage_api` imports.
- Acceptance: admin consumers no longer import flat storage facade symbols,
storage contracts, config helpers, bucket helper modules, runtime handles,
rebalance types, tier errors, or request helpers directly from the admin
storage API root; migration rules reject the old flat paths.
- Must preserve: admin route contracts, authorization flow, config
persistence, bucket metadata operations, site replication serialization,
remote-target validation, quota/heal/object-zip behavior, metrics
collection, rebalance response mapping, tier error behavior, and admin test
fixtures.
- Verification: focused RustFS compile, formatting, migration and layer
guards, diff hygiene, residual flat admin storage-api scan, Rust risk scan,
and full PR gate passed before PR.
## Next PRs
1. `consumer-migration`: continue larger app/runtime global-source batches
after API-236.
1. `consumer-migration`: continue larger root and owner boundary batches after
API-238.
## Pre-Push Review Log
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | pass | API-238 segments the admin-local storage API boundary into admin consumer domain modules instead of a flat re-export surface. |
| Migration preservation | pass | Admin handlers, services, router, console paths, config persistence, bucket metadata, replication, quota, heal, metrics, rebalance, tier, and object zip consumers keep the same owner symbols and call paths. |
| Testing/verification | pass | RustFS focused compile, formatting, migration/layer guards, residual flat admin storage-api scan, diff hygiene, diff-added Rust risk scan, and full PR gate passed before PR. |
| Quality/architecture | pass | API-237 segments the root-local storage API boundary into domain modules instead of a flat re-export surface. |
| Migration preservation | pass | Startup, server, cluster, table catalog, protocol, capacity, workload, config-test, and error-mapping consumers keep the same owner symbols and call paths. |
| Testing/verification | pass | RustFS focused compile, formatting, migration/layer guards, residual flat root storage-api scan, diff hygiene, diff-added Rust risk scan, and full PR gate passed before PR. |
| Quality/architecture | pass | API-236 makes admin, app, and storage runtime sources consume root runtime-source entrypoints instead of importing app context directly. |
| Migration preservation | pass | Admin/object usecase construction, app explicit-context fallback, storage runtime reads, IAM/KMS/TLS resolver behavior, notification dispatch, and test TLS hooks keep the same semantics. |
| Testing/verification | pass | Focused admin/app/storage checks, formatting, migration/layer guards, residual owner runtime-source AppContext scan, diff hygiene, Rust risk scan, and full PR gate passed before PR. |
@@ -5712,6 +5758,27 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
Passed before push:
- Issue #660 API-238 current slice:
- Branch freshness check: rebased onto current `origin/main` after PR #3899 merged.
- `cargo check -p rustfs`: passed.
- `make pre-pr`: passed.
- Issue #660 API-237 current slice:
- Branch freshness check: rebased onto current `origin/main` after PR #3899 merged.
- `cargo check -p rustfs`: passed.
- `cargo fmt --all`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- Flat root storage-api residual scan: passed; root consumers now use
`storage_api` domain modules instead of flat imports or legacy helper
modules.
- Diff-added Rust risk scan: passed; matches were import aliases only, with
no new production unwrap/expect, numeric cast, String error, Box dyn Error,
print macro, or relaxed atomic ordering lines.
- `make pre-pr`: passed.
- Issue #660 API-236 current slice:
- Branch freshness check: rebased onto current `origin/main` after API-232
merged.
rustfs/src/admin/console.rs
+1 -1
View File
@@ -14,7 +14,7 @@
use crate::admin::handlers::health::{HealthProbe, build_health_response_parts, collect_dependency_readiness};
use crate::admin::runtime_sources::{default_admin_usecase, resolve_oidc_handle};
use crate::admin::storage_api::RequestContext;
use crate::admin::storage_api::access::RequestContext;
use crate::license::has_valid_license;
use crate::server::has_path_prefix;
use crate::server::{
rustfs/src/admin/handlers/account_info.rs
+2 -2
View File
@@ -15,8 +15,8 @@
use crate::admin::auth::authenticate_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{resolve_action_credentials, resolve_object_store_handle};
use crate::admin::storage_api::versioning_sys::BucketVersioningSys;
use crate::admin::storage_api::{BucketOperations, BucketOptions, StorageAdminApi};
use crate::admin::storage_api::bucket::versioning_sys::BucketVersioningSys;
use crate::admin::storage_api::contract::{BucketOperations, BucketOptions, StorageAdminApi};
use crate::auth::get_condition_values;
use crate::server::{ADMIN_PREFIX, RemoteAddr};
use http::{HeaderMap, HeaderValue};
rustfs/src/admin/handlers/audit_runtime_config.rs
+1 -1
View File
@@ -14,7 +14,7 @@
use crate::admin::handlers::target_descriptor::AdminTargetSpec;
use crate::admin::runtime_sources::resolve_object_store_handle;
use crate::admin::storage_api::{read_admin_config_without_migrate, save_admin_server_config};
use crate::admin::storage_api::config::{read_admin_config_without_migrate, save_admin_server_config};
use rustfs_audit::{audit_system, start_audit_system as start_global_audit_system, system::AuditSystemState};
use rustfs_config::DEFAULT_DELIMITER;
use rustfs_config::server_config::Config;
rustfs/src/admin/handlers/bucket_meta.rs
+4 -4
View File
@@ -12,10 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::admin::storage_api::ecstore_utils::{deserialize, serialize};
use crate::admin::storage_api::{BucketOperations, BucketOptions, MakeBucketOptions};
use crate::admin::storage_api::{
StorageError,
use crate::admin::storage_api::bucket::utils::{deserialize, serialize};
use crate::admin::storage_api::bucket::{
metadata::{
BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE,
BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG,
@@ -25,6 +23,8 @@ use crate::admin::storage_api::{
quota::BucketQuota,
target::BucketTargets,
};
use crate::admin::storage_api::contract::{BucketOperations, BucketOptions, MakeBucketOptions};
use crate::admin::storage_api::error::StorageError;
use crate::{
admin::runtime_sources::resolve_object_store_handle,
admin::{
rustfs/src/admin/handlers/cluster_snapshot.rs
+5 -5
View File
@@ -12,12 +12,12 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::admin::storage_api::{CapabilityState, CapabilityStatus, ObservabilitySnapshot, TopologySnapshot};
use crate::admin::storage_api::cluster::{CapabilityState, CapabilityStatus, ObservabilitySnapshot, TopologySnapshot};
use crate::admin::{
auth::validate_admin_request,
router::{AdminOperation, Operation, S3Router},
runtime_sources::default_admin_usecase,
storage_api::ecstore_cluster::{
storage_api::cluster::{
ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot,
ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState,
ClusterPoolStateSnapshot,
@@ -590,13 +590,13 @@ fn summarize_named_capability_statuses<const N: usize>(
#[cfg(test)]
mod tests {
use super::{ClusterSnapshotResponse, ClusterSnapshotSummary, ClusterSnapshotView};
use crate::admin::storage_api::CapabilityState;
use crate::admin::storage_api::ecstore_cluster::{
use crate::admin::storage_api::cluster::CapabilityState;
use crate::admin::storage_api::cluster::{CapabilityStatus, ObservabilitySnapshot, TopologySnapshot};
use crate::admin::storage_api::cluster::{
ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot,
ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState,
ClusterPoolStateSnapshot,
};
use crate::admin::storage_api::{CapabilityStatus, ObservabilitySnapshot, TopologySnapshot};
use crate::cluster_snapshot::{ClusterReadOnlySnapshot, ClusterRuntimeReadinessState, ClusterRuntimeStatusSnapshot};
use crate::server::{DependencyReadiness, ReadinessDegradedReason};
use rustfs_concurrency::{AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadClass};
rustfs/src/admin/handlers/config_admin.rs
+13 -13
View File
@@ -19,12 +19,12 @@ use crate::admin::service::config::{
apply_dynamic_config_for_subsystem, is_dynamic_config_subsystem, signal_config_snapshot_reload, signal_dynamic_config_reload,
validate_server_config,
};
use crate::admin::storage_api::ListOperations as _;
use crate::admin::storage_api::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV};
use crate::admin::storage_api::{
use crate::admin::storage_api::config::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV};
use crate::admin::storage_api::config::{
RUSTFS_META_BUCKET, STORAGE_CLASS_SUB_SYS, delete_admin_config, read_admin_config, read_admin_config_without_migrate,
save_admin_config, save_admin_server_config,
};
use crate::admin::storage_api::contract::ListOperations as _;
use crate::admin::utils::{encode_compatible_admin_payload, is_compat_admin_request, read_compatible_admin_body};
use crate::auth::{check_key_valid, get_session_token};
use crate::error::ApiError;
@@ -708,7 +708,7 @@ fn success_response(config_applied: bool) -> S3Result<S3Response<(StatusCode, Bo
Ok(S3Response::with_headers((StatusCode::OK, Body::default()), headers))
}
fn object_store() -> S3Result<std::sync::Arc<crate::admin::storage_api::ECStore>> {
fn object_store() -> S3Result<std::sync::Arc<crate::admin::storage_api::runtime::ECStore>> {
resolve_object_store_handle().ok_or_else(|| s3_error!(InternalError, "server storage not initialized"))
}
@@ -753,7 +753,7 @@ fn config_update_sub_system(directives: &[ConfigDirective]) -> S3Result<Option<&
fn validate_config_directives(directives: &[ConfigDirective]) -> S3Result<()> {
if DEFAULT_KVS.get().is_none() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
}
let Some(defaults) = DEFAULT_KVS.get() else {
return Err(s3_error!(InternalError, "config defaults are not initialized"));
@@ -1407,7 +1407,7 @@ fn env_help_key(sub_system: &str, key: &str) -> String {
fn default_help_postfix(sub_system: &str, key: &str) -> String {
if DEFAULT_KVS.get().is_none() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
}
DEFAULT_KVS
@@ -1894,7 +1894,7 @@ mod tests {
#[test]
fn full_config_export_can_be_reapplied() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
let mut original = ServerConfig::new();
apply_set_directives(
&mut original,
@@ -1941,7 +1941,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn build_help_response_appends_default_value_postfix() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
let response = build_help_response(Some("identity_openid"), Some("scopes"), false).expect("help response");
assert_eq!(response.keys_help.len(), 2);
@@ -2054,7 +2054,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_includes_env_override_lines() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
temp_env::with_vars(
[
("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example")),
@@ -2090,7 +2090,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_lists_env_only_targets() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || {
let config = ServerConfig::new();
let rendered = String::from_utf8(
@@ -2113,7 +2113,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_supports_specific_env_only_target_queries() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || {
let config = ServerConfig::new();
let rendered = String::from_utf8(
@@ -2136,7 +2136,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_orders_default_before_named_targets() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_ALPHA", Some("http://alpha.example"))], || {
let mut config = ServerConfig::new();
apply_set_directives(
@@ -2317,7 +2317,7 @@ identity_openid client_id="existing-client""#,
#[test]
fn storage_class_get_target_none_matches_full_export() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
let mut config = ServerConfig::new();
apply_set_directives(
&mut config,
rustfs/src/admin/handlers/extensions.rs
+3 -3
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::admin::storage_api::CapabilityStatus;
use crate::admin::storage_api::cluster::CapabilityStatus;
use crate::admin::{
auth::validate_admin_request,
handlers::{cluster_snapshot, plugins_instances, system},
@@ -118,8 +118,8 @@ pub(crate) struct ExtensionInstancesResponse {
pub next_marker: Option<String>,
}
async fn build_extension_catalog_response() -> Result<ExtensionCatalogResponse, crate::admin::storage_api::CapabilitySnapshotError>
{
async fn build_extension_catalog_response()
-> Result<ExtensionCatalogResponse, crate::admin::storage_api::cluster::CapabilitySnapshotError> {
let mut extensions = builtin_extension_schemas();
let example = example_external_webhook_plugin();
extensions.push(target_marketplace_extension_schema(&example.manifest));
rustfs/src/admin/handlers/heal.rs
+7 -7
View File
@@ -15,10 +15,10 @@
use crate::admin::auth::{authenticate_request, validate_admin_request};
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::resolve_object_store_handle;
use crate::admin::storage_api::HealOperations as _;
use crate::admin::storage_api::ecstore_utils::is_valid_object_prefix;
use crate::admin::storage_api::is_reserved_or_invalid_bucket;
use crate::admin::storage_api::spawn_traced;
use crate::admin::storage_api::access::spawn_traced;
use crate::admin::storage_api::bucket::is_reserved_or_invalid_bucket;
use crate::admin::storage_api::bucket::utils::is_valid_object_prefix;
use crate::admin::storage_api::contract::HealOperations as _;
use crate::server::ADMIN_PREFIX;
use crate::server::RemoteAddr;
use bytes::Bytes;
@@ -386,10 +386,10 @@ fn should_handle_root_heal_directly(_hip: &HealInitParams) -> bool {
false
}
fn map_root_heal_status(heal_err: Option<crate::admin::storage_api::Error>) -> S3Result<()> {
fn map_root_heal_status(heal_err: Option<crate::admin::storage_api::error::Error>) -> S3Result<()> {
match heal_err {
None => Ok(()),
Some(crate::admin::storage_api::StorageError::NoHealRequired) => {
Some(crate::admin::storage_api::error::StorageError::NoHealRequired) => {
info!(
event = EVENT_ADMIN_RESPONSE_EMITTED,
component = LOG_COMPONENT_ADMIN_API,
@@ -750,7 +750,7 @@ mod tests {
json_response, map_heal_response, map_root_heal_status, should_handle_root_heal_directly, validate_heal_request_mode,
validate_heal_target,
};
use crate::admin::storage_api::StorageError;
use crate::admin::storage_api::error::StorageError;
use bytes::Bytes;
use http::StatusCode;
use http::Uri;
rustfs/src/admin/handlers/kms_dynamic.rs
+1 -1
View File
@@ -19,7 +19,7 @@ use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{
resolve_kms_runtime_service_manager, resolve_object_store_handle, resolve_or_init_kms_runtime_service_manager,
};
use crate::admin::storage_api::{read_admin_config, save_admin_config};
use crate::admin::storage_api::config::{read_admin_config, save_admin_config};
use crate::auth::{check_key_valid, get_session_token};
use crate::server::{ADMIN_PREFIX, RemoteAddr};
use hyper::{Method, StatusCode};
rustfs/src/admin/handlers/metrics.rs
+2 -2
View File
@@ -20,8 +20,8 @@
use crate::admin::auth::validate_admin_request;
use crate::admin::router::Operation;
use crate::admin::storage_api::spawn_traced;
use crate::admin::storage_api::{CollectMetricsOpts, MetricType, collect_local_metrics};
use crate::admin::storage_api::access::spawn_traced;
use crate::admin::storage_api::metrics::{CollectMetricsOpts, MetricType, collect_local_metrics};
use crate::auth::{check_key_valid, get_session_token};
use crate::server::RemoteAddr;
use bytes::Bytes;
rustfs/src/admin/handlers/object_zip_download.rs
+6 -3
View File
@@ -14,8 +14,11 @@
use crate::admin::router::{ADMIN_OBJECT_ZIP_DOWNLOADS_PATH, AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{resolve_action_credentials, resolve_object_store_handle, resolve_region};
use crate::admin::storage_api::{BucketOperations, BucketOptions, ListOperations as _, ObjectIO as _, ObjectOperations as _};
use crate::admin::storage_api::{ReqInfo, StorageObjectOptions as ObjectOptions, authorize_request};
use crate::admin::storage_api::access::{ReqInfo, authorize_request};
use crate::admin::storage_api::contract::{
BucketOperations, BucketOptions, ListOperations as _, ObjectIO as _, ObjectOperations as _,
};
use crate::admin::storage_api::object::StorageObjectOptions as ObjectOptions;
use crate::auth::{check_key_valid, get_session_token};
use crate::error::ApiError;
use crate::license::license_check;
@@ -645,7 +648,7 @@ async fn preflight_zip_items(request: &CreateObjectZipDownloadRequest, items: &[
Ok(())
}
fn storage_error_to_s3(err: crate::admin::storage_api::Error) -> s3s::S3Error {
fn storage_error_to_s3(err: crate::admin::storage_api::error::Error) -> s3s::S3Error {
ApiError::from(err).into()
}
rustfs/src/admin/handlers/oidc.rs
+1 -1
View File
@@ -16,7 +16,7 @@ use super::sts::create_oidc_sts_credentials;
use crate::admin::auth::validate_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{resolve_object_store_handle, resolve_oidc_handle, resolve_server_config};
use crate::admin::storage_api::{read_admin_config_without_migrate, save_admin_server_config};
use crate::admin::storage_api::config::{read_admin_config_without_migrate, save_admin_server_config};
use crate::auth::{check_key_valid, get_session_token};
use crate::server::{ADMIN_PREFIX, MINIO_ADMIN_PREFIX, RemoteAddr};
use http::StatusCode;
rustfs/src/admin/handlers/pools.rs
+3 -3
View File
@@ -32,7 +32,7 @@ use crate::{
admin::{
auth::validate_admin_request,
router::{AdminOperation, Operation, S3Router},
storage_api::{EndpointServerPools, PeerRestClient},
storage_api::runtime::{EndpointServerPools, PeerRestClient},
},
auth::{check_key_valid, get_session_token},
error::ApiError,
@@ -213,7 +213,7 @@ macro_rules! log_pool_response_emitted {
};
}
fn endpoints_from_context() -> Option<crate::admin::storage_api::EndpointServerPools> {
fn endpoints_from_context() -> Option<crate::admin::storage_api::runtime::EndpointServerPools> {
resolve_endpoints_handle()
}
@@ -1080,7 +1080,7 @@ mod pools_handler_tests {
pool_admin_pool_not_found_error_with_audit, pool_admin_pool_parse_error_with_audit, pool_admin_query_parse_error,
pool_admin_query_parse_error_with_audit, validate_pool_mutation_leader, validate_start_decommission_guards,
};
use crate::admin::storage_api::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints};
use crate::admin::storage_api::runtime::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints};
fn test_pool_endpoints(is_local: bool) -> EndpointServerPools {
let mut endpoint = Endpoint::try_from("http://127.0.0.1:9000/disk").expect("test endpoint should parse");
rustfs/src/admin/handlers/quota.rs
+3 -3
View File
@@ -17,9 +17,9 @@
use crate::admin::auth::{validate_admin_request, validate_admin_request_with_bucket};
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{resolve_bucket_metadata_handle, resolve_object_store_handle};
use crate::admin::storage_api::metadata_sys::BucketMetadataSys;
use crate::admin::storage_api::quota::checker::QuotaChecker;
use crate::admin::storage_api::quota::{BucketQuota, QuotaError, QuotaOperation};
use crate::admin::storage_api::bucket::metadata_sys::BucketMetadataSys;
use crate::admin::storage_api::bucket::quota::checker::QuotaChecker;
use crate::admin::storage_api::bucket::quota::{BucketQuota, QuotaError, QuotaOperation};
use crate::auth::{check_key_valid, get_session_token};
use crate::server::ADMIN_PREFIX;
use hyper::{Method, StatusCode};
rustfs/src/admin/handlers/rebalance.rs
+9 -7
View File
@@ -12,11 +12,13 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::admin::storage_api::{BucketOperations, BucketOptions, StorageAdminApi};
use crate::admin::storage_api::{
DiskStat, ECStore, NotificationSys, RebalSaveOpt, RebalanceCleanupWarnings, RebalanceMeta, RebalanceStopPropagationRecord,
StorageError, decode_rebalance_stop_propagation_record,
use crate::admin::storage_api::contract::{BucketOperations, BucketOptions, StorageAdminApi};
use crate::admin::storage_api::error::StorageError;
use crate::admin::storage_api::rebalance::{
DiskStat, RebalSaveOpt, RebalanceCleanupWarnings, RebalanceMeta, RebalanceStopPropagationRecord,
decode_rebalance_stop_propagation_record,
};
use crate::admin::storage_api::runtime::{ECStore, NotificationSys};
use crate::{
admin::runtime_sources::{resolve_notification_system, resolve_object_store_handle},
admin::{
@@ -296,7 +298,7 @@ fn build_rebalance_pool_progress(
now: OffsetDateTime,
stop_time: Option<OffsetDateTime>,
percent_free_goal: f64,
ps: &crate::admin::storage_api::RebalanceStats,
ps: &crate::admin::storage_api::rebalance::RebalanceStats,
) -> Option<RebalPoolProgress> {
let total_bytes_to_rebal = ps.init_capacity as f64 * percent_free_goal - ps.init_free_space as f64;
let terminal_time = ps.info.end_time.or(stop_time);
@@ -339,7 +341,7 @@ fn build_rebalance_pool_statuses(
now: OffsetDateTime,
stop_time: Option<OffsetDateTime>,
percent_free_goal: f64,
pool_stats: &[crate::admin::storage_api::RebalanceStats],
pool_stats: &[crate::admin::storage_api::rebalance::RebalanceStats],
disk_stats: &[DiskStat],
) -> Vec<RebalancePoolStatus> {
pool_stats
@@ -903,7 +905,7 @@ mod rebalance_handler_tests {
rebalance_pool_used, rebalance_query_present, rebalance_remaining_buckets, rebalance_rollback_failure_message,
rebalance_rollback_stop_failure_message, rebalance_start_rollback_error, rebalance_used_pct, rollback_result_label,
};
use crate::admin::storage_api::{
use crate::admin::storage_api::rebalance::{
DiskStat, RebalStatus, RebalanceCleanupWarningEntry, RebalanceCleanupWarnings, RebalanceInfo, RebalanceMeta,
RebalanceStats, RebalanceStopPropagationRecord, encode_rebalance_stop_propagation_record,
};
rustfs/src/admin/handlers/replication.rs
+9 -9
View File
@@ -16,14 +16,14 @@ use crate::admin::auth::validate_admin_request;
use crate::admin::handlers::site_replication::site_replication_peer_deployment_id_for_endpoint;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{resolve_object_store_handle, resolve_replication_stats_handle, resolve_runtime_port};
use crate::admin::storage_api::StorageError;
use crate::admin::storage_api::bucket_target_sys::{BucketTargetError, BucketTargetSys};
use crate::admin::storage_api::metadata::BUCKET_TARGETS_FILE;
use crate::admin::storage_api::metadata_sys;
use crate::admin::storage_api::metadata_sys::get_replication_config;
use crate::admin::storage_api::replication::BucketStats;
use crate::admin::storage_api::target::BucketTarget;
use crate::admin::storage_api::{BucketOperations, BucketOptions};
use crate::admin::storage_api::bucket::metadata::BUCKET_TARGETS_FILE;
use crate::admin::storage_api::bucket::metadata_sys;
use crate::admin::storage_api::bucket::metadata_sys::get_replication_config;
use crate::admin::storage_api::bucket::replication::BucketStats;
use crate::admin::storage_api::bucket::target::BucketTarget;
use crate::admin::storage_api::bucket::target_sys::{BucketTargetError, BucketTargetSys};
use crate::admin::storage_api::contract::{BucketOperations, BucketOptions};
use crate::admin::storage_api::error::StorageError;
use crate::admin::utils::read_compatible_admin_body;
use crate::auth::{check_key_valid, get_session_token};
use crate::error::ApiError;
@@ -448,7 +448,7 @@ impl Operation for RemoveRemoteTargetHandler {
#[cfg(test)]
mod tests {
use super::{extract_query_params, validate_remote_target_tls_settings};
use crate::admin::storage_api::target::BucketTarget;
use crate::admin::storage_api::bucket::target::BucketTarget;
use http::Uri;
#[test]
rustfs/src/admin/handlers/site_replication.rs
+16 -14
View File
@@ -23,19 +23,21 @@ use crate::admin::site_replication_identity::{
canonical_endpoint, deployment_id_for_endpoint, normalize_peer_map_by_identity_with, same_identity_endpoint,
site_identity_key,
};
use crate::admin::storage_api::Error as StorageError;
use crate::admin::storage_api::bucket_target_sys::BucketTargetSys;
use crate::admin::storage_api::ecstore_utils::{deserialize, serialize};
use crate::admin::storage_api::metadata::{
use crate::admin::storage_api::bucket::metadata::{
BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, BUCKET_REPLICATION_CONFIG,
BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, OBJECT_LOCK_CONFIG,
};
use crate::admin::storage_api::metadata_sys;
use crate::admin::storage_api::replication::ResyncOpts;
use crate::admin::storage_api::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials};
use crate::admin::storage_api::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use crate::admin::storage_api::{BucketOperations, BucketOptions, DeleteBucketOptions, MakeBucketOptions, SRBucketDeleteOp};
use crate::admin::storage_api::{delete_admin_config, read_admin_config, save_admin_config};
use crate::admin::storage_api::bucket::metadata_sys;
use crate::admin::storage_api::bucket::replication::ResyncOpts;
use crate::admin::storage_api::bucket::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials};
use crate::admin::storage_api::bucket::target_sys::BucketTargetSys;
use crate::admin::storage_api::bucket::utils::{deserialize, serialize};
use crate::admin::storage_api::bucket::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use crate::admin::storage_api::config::{delete_admin_config, read_admin_config, save_admin_config};
use crate::admin::storage_api::contract::{
BucketOperations, BucketOptions, DeleteBucketOptions, MakeBucketOptions, SRBucketDeleteOp,
};
use crate::admin::storage_api::error::Error as StorageError;
use crate::admin::utils::{encode_compatible_admin_payload, read_compatible_admin_body};
use crate::auth::{check_key_valid, get_session_token};
use crate::config::get_config_snapshot;
@@ -652,7 +654,7 @@ async fn site_replication_peer_client() -> S3Result<reqwest::Client> {
built
}
fn runtime_tls_enabled_with(endpoints: Option<&crate::admin::storage_api::EndpointServerPools>) -> bool {
fn runtime_tls_enabled_with(endpoints: Option<&crate::admin::storage_api::runtime::EndpointServerPools>) -> bool {
if !rustfs_utils::get_env_str(ENV_RUSTFS_TLS_PATH, DEFAULT_RUSTFS_TLS_PATH).is_empty() {
return true;
}
@@ -3358,7 +3360,7 @@ fn is_stale_update(local_updated_at: OffsetDateTime, incoming_updated_at: Option
}
fn bucket_meta_local_updated_at(
bucket_meta: &crate::admin::storage_api::metadata::BucketMetadata,
bucket_meta: &crate::admin::storage_api::bucket::metadata::BucketMetadata,
config_file: &str,
) -> OffsetDateTime {
match config_file {
@@ -4605,8 +4607,8 @@ impl Operation for SRRotateServiceAccountHandler {
mod tests {
use super::*;
use crate::admin::runtime_sources::{resolve_outbound_tls_generation, set_test_outbound_tls_generation};
use crate::admin::storage_api::Endpoint;
use crate::admin::storage_api::{EndpointServerPools, Endpoints, PoolEndpoints};
use crate::admin::storage_api::runtime::Endpoint;
use crate::admin::storage_api::runtime::{EndpointServerPools, Endpoints, PoolEndpoints};
use http::{HeaderMap, HeaderValue, Uri};
use rustfs_policy::policy::action::S3Action;
use serial_test::serial;
rustfs/src/admin/handlers/sts.rs
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use super::is_admin::IsAdminHandler;
use crate::admin::storage_api::ecstore_utils::serialize;
use crate::admin::storage_api::bucket::utils::serialize;
use crate::{
admin::runtime_sources::{resolve_action_credentials, resolve_oidc_handle, resolve_token_signing_key},
admin::{
rustfs/src/admin/handlers/system.rs
+5 -5
View File
@@ -18,7 +18,7 @@ use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{
DefaultAdminUsecase, QueryServerInfoRequest, default_admin_usecase, resolve_endpoints_handle,
};
use crate::admin::storage_api::{
use crate::admin::storage_api::cluster::{
CapabilityState, CapabilityStatus, ObservabilitySnapshotProvider, TopologySnapshot, TopologySnapshotProvider,
};
use crate::auth::{check_key_valid, get_session_token};
@@ -298,7 +298,7 @@ pub struct RuntimeCapabilitiesResponse {
pub summary: RuntimeCapabilitiesSummary,
pub cluster_snapshot_path: String,
pub cluster_snapshot_summary: Option<CapabilityStatus>,
pub observability: crate::admin::storage_api::ObservabilitySnapshot,
pub observability: crate::admin::storage_api::cluster::ObservabilitySnapshot,
pub workload_admission: WorkloadAdmissionRegistrySnapshot,
pub topology: Option<TopologySnapshot>,
pub topology_status: CapabilityStatus,
@@ -307,7 +307,7 @@ pub struct RuntimeCapabilitiesResponse {
pub struct RuntimeCapabilitiesHandler {}
pub(crate) async fn build_runtime_capabilities_response()
-> Result<RuntimeCapabilitiesResponse, crate::admin::storage_api::CapabilitySnapshotError> {
-> Result<RuntimeCapabilitiesResponse, crate::admin::storage_api::cluster::CapabilitySnapshotError> {
let usecase = default_admin_usecase();
let observability_provider = RustFsObservabilitySnapshotProvider;
let observability = observability_provider.observability_snapshot().await?;
@@ -340,7 +340,7 @@ pub(crate) async fn build_runtime_capabilities_response()
}
fn build_runtime_capabilities_summary(
observability: &crate::admin::storage_api::ObservabilitySnapshot,
observability: &crate::admin::storage_api::cluster::ObservabilitySnapshot,
topology: Option<&TopologySnapshot>,
topology_status: &CapabilityStatus,
cluster_snapshot_summary: Option<&CapabilityStatus>,
@@ -517,7 +517,7 @@ mod tests {
build_runtime_capabilities_response, build_runtime_capabilities_summary, system_admin_discovery,
};
use crate::admin::runtime_sources::DefaultAdminUsecase;
use crate::admin::storage_api::{
use crate::admin::storage_api::cluster::{
CapabilityState, CapabilityStatus, MemorySamplingState, ObservabilitySnapshot, PlatformSupport, TopologyCapabilities,
TopologySnapshot, UserspaceProfilingCapability,
};
rustfs/src/admin/handlers/table_catalog.rs
+2 -1
View File
@@ -14,7 +14,8 @@
use crate::admin::runtime_sources::default_admin_usecase;
use crate::admin::runtime_sources::{resolve_object_store_handle, resolve_token_signing_key};
use crate::admin::storage_api::{ECStore, metadata::table_catalog_path_hash, metadata_sys};
use crate::admin::storage_api::bucket::{metadata::table_catalog_path_hash, metadata_sys};
use crate::admin::storage_api::runtime::ECStore;
use crate::admin::{
auth::{AdminResourceScope, validate_admin_request, validate_admin_request_with_bucket_object},
router::{AdminOperation, Operation, S3Router},
rustfs/src/admin/handlers/tier.rs
+2 -2
View File
@@ -13,7 +13,7 @@
// limitations under the License.
#![allow(unused_variables, unused_mut, unused_must_use)]
use crate::admin::storage_api::{
use crate::admin::storage_api::tier::{
AdminError, DailyAllTierStats, ERR_TIER_ALREADY_EXISTS, ERR_TIER_BACKEND_IN_USE, ERR_TIER_BACKEND_NOT_EMPTY,
ERR_TIER_CONNECT_ERR, ERR_TIER_INVALID_CREDENTIALS, ERR_TIER_MISSING_CREDENTIALS, ERR_TIER_NAME_NOT_UPPERCASE,
ERR_TIER_NOT_FOUND, TierConfig, TierCreds, TierType, storageclass,
@@ -932,7 +932,7 @@ impl Operation for ClearTier {
#[cfg(test)]
mod tests {
use super::*;
use crate::admin::storage_api::lifecycle::tier_last_day_stats::LastDayTierStats;
use crate::admin::storage_api::bucket::lifecycle::tier_last_day_stats::LastDayTierStats;
use http::Uri;
use matchit::Router;
rustfs/src/admin/handlers/trace.rs
+1 -1
View File
@@ -14,7 +14,7 @@
use crate::admin::router::Operation;
use crate::admin::runtime_sources::resolve_endpoints_handle;
use crate::admin::storage_api::PeerRestClient;
use crate::admin::storage_api::runtime::PeerRestClient;
use http::StatusCode;
use hyper::Uri;
use matchit::Params;
rustfs/src/admin/router.rs
+23 -22
View File
@@ -12,18 +12,19 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_api::PeerRestClient;
use super::storage_api::bandwidth::monitor::BandwidthDetails;
use super::storage_api::bucket_target_sys::{
use super::storage_api::bucket::bandwidth::monitor::BandwidthDetails;
use super::storage_api::bucket::metadata::BUCKET_TARGETS_FILE;
use super::storage_api::bucket::metadata_sys;
use super::storage_api::bucket::replication::{BucketReplicationResyncStatus, BucketStats, ObjectOpts, ResyncOpts};
use super::storage_api::bucket::target::{BucketTarget, BucketTargetType, BucketTargets};
use super::storage_api::bucket::target_sys::{
BucketTargetSys, PutObjectOptions, RemoveObjectOptions, S3ClientError, TargetClient,
};
use super::storage_api::metadata::BUCKET_TARGETS_FILE;
use super::storage_api::metadata_sys;
use super::storage_api::read_admin_config_without_migrate;
use super::storage_api::replication::{BucketReplicationResyncStatus, BucketStats, ObjectOpts, ResyncOpts};
use super::storage_api::target::{BucketTarget, BucketTargetType, BucketTargets};
use super::storage_api::versioning_sys::BucketVersioningSys;
use super::storage_api::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _, StorageError};
use super::storage_api::bucket::versioning_sys::BucketVersioningSys;
use super::storage_api::bucket::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use super::storage_api::config::read_admin_config_without_migrate;
use super::storage_api::error::StorageError;
use super::storage_api::runtime::PeerRestClient;
use crate::admin::console::{is_console_path, make_console_server};
use crate::admin::handlers::oidc::is_oidc_path;
use crate::admin::runtime_sources::{
@@ -31,8 +32,8 @@ use crate::admin::runtime_sources::{
resolve_object_store_handle, resolve_region, resolve_replication_pool_handle, resolve_replication_stats_handle,
resolve_server_config,
};
use crate::admin::storage_api::{BucketOperations, BucketOptions};
use crate::admin::storage_api::{ReqInfo, authorize_request, spawn_traced};
use crate::admin::storage_api::access::{ReqInfo, authorize_request, spawn_traced};
use crate::admin::storage_api::contract::{BucketOperations, BucketOptions};
use crate::auth::{check_key_valid, get_session_token};
use crate::error::ApiError;
use crate::license::license_check;
@@ -1939,7 +1940,7 @@ async fn resolve_replication_target_client(bucket: &str, target: &BucketTarget)
fn build_replication_probe_put_options(now: OffsetDateTime) -> PutObjectOptions {
PutObjectOptions {
internal: super::storage_api::bucket_target_sys::AdvancedPutOptions {
internal: super::storage_api::bucket::target_sys::AdvancedPutOptions {
source_version_id: Uuid::new_v4().to_string(),
replication_status: ReplicationStatusType::Replica,
source_mtime: now,
@@ -2768,7 +2769,7 @@ mod tests {
#[test]
fn apply_replication_reset_to_targets_updates_matching_target() {
let mut targets = BucketTargets {
targets: vec![crate::admin::storage_api::target::BucketTarget {
targets: vec![crate::admin::storage_api::bucket::target::BucketTarget {
arn: "arn:target".to_string(),
..Default::default()
}],
@@ -2790,10 +2791,10 @@ mod tests {
let mut status = BucketReplicationResyncStatus::new();
status.targets_map.insert(
"arn:z".to_string(),
crate::admin::storage_api::replication::TargetReplicationResyncStatus {
crate::admin::storage_api::bucket::replication::TargetReplicationResyncStatus {
resync_id: "rid-z".to_string(),
last_update: Some(datetime!(2025-01-03 00:00 UTC)),
resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncFailed,
resync_status: crate::admin::storage_api::bucket::replication::ResyncStatusType::ResyncFailed,
failed_count: 2,
failed_size: 4,
bucket: "bucket-z".to_string(),
@@ -2803,10 +2804,10 @@ mod tests {
);
status.targets_map.insert(
"arn:a".to_string(),
crate::admin::storage_api::replication::TargetReplicationResyncStatus {
crate::admin::storage_api::bucket::replication::TargetReplicationResyncStatus {
resync_id: "rid-a".to_string(),
last_update: Some(datetime!(2025-01-02 00:00 UTC)),
resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncCompleted,
resync_status: crate::admin::storage_api::bucket::replication::ResyncStatusType::ResyncCompleted,
replicated_count: 3,
replicated_size: 9,
bucket: "bucket-a".to_string(),
@@ -2836,10 +2837,10 @@ mod tests {
let mut status = BucketReplicationResyncStatus::new();
status.targets_map.insert(
"arn:z".to_string(),
crate::admin::storage_api::replication::TargetReplicationResyncStatus {
crate::admin::storage_api::bucket::replication::TargetReplicationResyncStatus {
resync_id: "rid-z".to_string(),
last_update: Some(datetime!(2025-02-03 00:00 UTC)),
resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncFailed,
resync_status: crate::admin::storage_api::bucket::replication::ResyncStatusType::ResyncFailed,
failed_count: 2,
failed_size: 4,
bucket: "bucket-z".to_string(),
@@ -2849,10 +2850,10 @@ mod tests {
);
status.targets_map.insert(
"arn:a".to_string(),
crate::admin::storage_api::replication::TargetReplicationResyncStatus {
crate::admin::storage_api::bucket::replication::TargetReplicationResyncStatus {
resync_id: "rid-a".to_string(),
last_update: Some(datetime!(2025-02-02 00:00 UTC)),
resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncCompleted,
resync_status: crate::admin::storage_api::bucket::replication::ResyncStatusType::ResyncCompleted,
replicated_count: 3,
replicated_size: 9,
bucket: "bucket-a".to_string(),
rustfs/src/admin/service/config.rs
+7 -8
View File
@@ -16,9 +16,8 @@ use crate::admin::runtime_sources::{
AppContext, current_app_context, publish_server_config, publish_storage_class_config, resolve_notification_system,
resolve_object_store_handle, resolve_object_store_handle_for_context,
};
use crate::admin::storage_api::StorageAdminApi;
use crate::admin::storage_api::storageclass;
use crate::admin::storage_api::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate};
use crate::admin::storage_api::config::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate, storageclass};
use crate::admin::storage_api::contract::StorageAdminApi;
use rustfs_audit::reload_audit_config;
use rustfs_config::audit::{AUDIT_MQTT_SUB_SYS, AUDIT_REDIS_DEFAULT_CHANNEL, AUDIT_WEBHOOK_SUB_SYS};
use rustfs_config::notify::{NOTIFY_MQTT_SUB_SYS, NOTIFY_REDIS_DEFAULT_CHANNEL, NOTIFY_WEBHOOK_SUB_SYS};
@@ -384,7 +383,7 @@ pub async fn signal_config_snapshot_reload() {
#[cfg(test)]
mod tests {
use super::*;
use crate::admin::storage_api::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG};
use crate::admin::storage_api::bucket::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG};
use rustfs_config::notify::NOTIFY_WEBHOOK_SUB_SYS;
use rustfs_config::oidc::{OIDC_CLIENT_ID, OIDC_CONFIG_URL, OIDC_SCOPES};
use rustfs_config::{HEAL_SUB_SYS, SCANNER_SUB_SYS};
@@ -440,7 +439,7 @@ mod tests {
#[test]
fn validate_notify_subsystem_config_rejects_invalid_webhook_endpoint() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(NOTIFY_WEBHOOK_SUB_SYS).expect("notify webhook defaults");
let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target");
@@ -454,7 +453,7 @@ mod tests {
#[test]
fn validate_audit_subsystem_config_rejects_relative_queue_dir() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(AUDIT_MQTT_SUB_SYS).expect("audit mqtt defaults");
let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target");
@@ -469,7 +468,7 @@ mod tests {
#[test]
fn validate_identity_openid_config_rejects_missing_openid_scope() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults");
let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target");
@@ -486,7 +485,7 @@ mod tests {
#[test]
fn validate_identity_openid_config_rejects_invalid_named_provider_id() {
crate::admin::storage_api::init_admin_config_defaults();
crate::admin::storage_api::config::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults");
let default_kvs = targets.get(DEFAULT_DELIMITER).cloned().expect("default target");
rustfs/src/admin/service/site_replication.rs
+2 -2
View File
@@ -14,8 +14,8 @@
use crate::admin::runtime_sources::{AppContext, current_app_context, resolve_object_store_handle_for_context};
use crate::admin::site_replication_identity::{deployment_id_for_endpoint, normalize_peer_map_by_identity_with};
use crate::admin::storage_api::Error as StorageError;
use crate::admin::storage_api::{read_admin_config, save_admin_config};
use crate::admin::storage_api::config::{read_admin_config, save_admin_config};
use crate::admin::storage_api::error::Error as StorageError;
use rustfs_madmin::PeerInfo;
use s3s::{S3Error, S3ErrorCode, S3Result};
use serde_json::{Map, Value};
rustfs/src/admin/storage_api.rs
+90
View File
@@ -428,3 +428,93 @@ pub(crate) mod data_usage {
crate::storage::ecstore_data_usage::load_data_usage_from_backend(store).await
}
}
pub(crate) mod access {
pub(crate) use super::{ReqInfo, RequestContext, authorize_request, spawn_traced};
}
pub(crate) mod bucket {
pub(crate) use super::bandwidth;
pub(crate) use super::bucket_target_sys as target_sys;
#[cfg(test)]
pub(crate) use super::lifecycle;
pub(crate) use super::metadata;
pub(crate) use super::metadata_sys;
pub(crate) use super::quota;
pub(crate) use super::replication;
pub(crate) use super::target;
pub(crate) use super::versioning_sys;
pub(crate) use super::{AdminReplicationConfigExt, AdminVersioningConfigExt, is_reserved_or_invalid_bucket};
pub(crate) mod utils {
pub(crate) use super::super::ecstore_utils::{deserialize, is_valid_object_prefix, serialize};
}
}
pub(crate) mod cluster {
pub(crate) use super::ecstore_cluster::{
ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot,
ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState,
ClusterPoolStateSnapshot,
};
pub(crate) use super::{
CapabilitySnapshotError, CapabilityState, CapabilityStatus, ObservabilitySnapshot, ObservabilitySnapshotProvider,
TopologySnapshot, TopologySnapshotProvider,
};
#[cfg(test)]
pub(crate) use super::{MemorySamplingState, PlatformSupport, TopologyCapabilities, UserspaceProfilingCapability};
}
pub(crate) mod config {
pub(crate) use super::storageclass;
pub(crate) use super::{
RUSTFS_META_BUCKET, STORAGE_CLASS_SUB_SYS, delete_admin_config, init_admin_config_defaults, read_admin_config,
read_admin_config_without_migrate, save_admin_config, save_admin_server_config,
};
}
pub(crate) mod contract {
pub(crate) use super::{
BucketOperations, BucketOptions, DeleteBucketOptions, HealOperations, ListOperations, MakeBucketOptions, ObjectIO,
ObjectOperations, SRBucketDeleteOp, StorageAdminApi,
};
}
pub(crate) mod error {
pub(crate) use super::{Error, StorageError};
}
pub(crate) mod metrics {
pub(crate) use super::{CollectMetricsOpts, MetricType, collect_local_metrics};
}
pub(crate) mod object {
pub(crate) use super::StorageObjectOptions;
}
pub(crate) mod rebalance {
pub(crate) use super::{
DiskStat, RebalSaveOpt, RebalanceCleanupWarnings, RebalanceMeta, RebalanceStats, RebalanceStopPropagationRecord,
decode_rebalance_stop_propagation_record,
};
#[cfg(test)]
pub(crate) use super::{RebalStatus, RebalanceCleanupWarningEntry, RebalanceInfo, encode_rebalance_stop_propagation_record};
}
pub(crate) mod runtime {
pub(crate) use super::{ECStore, EndpointServerPools, NotificationSys, PeerRestClient};
#[cfg(test)]
pub(crate) use super::{Endpoint, Endpoints, PoolEndpoints};
}
pub(crate) mod tier {
pub(crate) use super::storageclass;
pub(crate) use super::{
AdminError, DailyAllTierStats, ERR_TIER_ALREADY_EXISTS, ERR_TIER_BACKEND_IN_USE, ERR_TIER_BACKEND_NOT_EMPTY,
ERR_TIER_CONNECT_ERR, ERR_TIER_INVALID_CREDENTIALS, ERR_TIER_MISSING_CREDENTIALS, ERR_TIER_NAME_NOT_UPPERCASE,
ERR_TIER_NOT_FOUND, TierConfig, TierCreds, TierType,
};
}
rustfs/src/capacity/service.rs
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage_api::{all_local_disk, disk_drive_path, disk_endpoint};
use crate::storage_api::capacity::{all_local_disk, disk_drive_path, disk_endpoint};
use rustfs_io_metrics::capacity_metrics::{
record_capacity_cache_hit, record_capacity_cache_miss, record_capacity_cache_served, record_capacity_refresh_request,
record_capacity_scan_mode,
rustfs/src/cluster_snapshot.rs
+3 -3
View File
@@ -16,11 +16,11 @@ use crate::runtime_capabilities::runtime_observability_snapshot;
use crate::server::{
DependencyReadiness, DependencyReadinessReport, ReadinessDegradedReason, snapshot_dependency_readiness_report,
};
use crate::storage_api::ecstore_cluster::{
use crate::storage_api::cluster::control_plane::{
ClusterControlPlane, ClusterControlPlaneSnapshot, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot,
ClusterPeerHealthSnapshot, ClusterPoolStateSnapshot,
};
use crate::storage_api::{EndpointServerPools, ObservabilitySnapshot, TopologySnapshot};
use crate::storage_api::cluster::{EndpointServerPools, ObservabilitySnapshot, TopologySnapshot};
use crate::workload_admission::workload_admission_registry_snapshot;
use rustfs_concurrency::{AdmissionState, WorkloadAdmissionRegistrySnapshot};
@@ -115,7 +115,7 @@ pub fn cluster_has_actionable_pressure(snapshot: &ClusterReadOnlySnapshot) -> bo
#[cfg(test)]
mod tests {
use super::*;
use crate::storage_api::{
use crate::storage_api::cluster::{
CapabilityState, CapabilityStatus, DiskCapabilities, Endpoint, Endpoints, PoolEndpoints, TopologyCapabilities,
};
use rustfs_concurrency::{WorkloadAdmissionSnapshot, WorkloadClass};
rustfs/src/config/config_test.rs
+2 -2
View File
@@ -17,7 +17,7 @@
mod tests {
use crate::config::cli::default_server_opts;
use crate::config::{CommandResult, Config, Opt, TlsCommands};
use crate::storage_api::DisksLayout;
use crate::storage_api::config_test::DisksLayout;
use rustfs_config::{DEFAULT_CONSOLE_ADDRESS, DEFAULT_CONSOLE_ENABLE, DEFAULT_OBS_ENDPOINT, RUSTFS_REGION};
use rustfs_credentials::{DEFAULT_ACCESS_KEY, DEFAULT_SECRET_KEY};
use serial_test::serial;
@@ -275,7 +275,7 @@ mod tests {
#[test]
#[serial]
fn test_volumes_and_disk_layout_parsing() {
use crate::storage_api::DisksLayout;
use crate::storage_api::config_test::DisksLayout;
// Test case 1: Single volume path
let args = vec!["rustfs", "/data/vol1"];
rustfs/src/error.rs
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage_api::{HTTPRangeError, QuotaError, StorageError};
use crate::storage_api::error::{HTTPRangeError, QuotaError, StorageError};
use s3s::{S3Error, S3ErrorCode};
#[derive(Debug)]
rustfs/src/init.rs
+6 -6
View File
@@ -14,7 +14,7 @@
use crate::runtime_sources::{resolve_notify_interface, resolve_region};
use crate::server::ShutdownHandle;
use crate::storage_api::{
use crate::storage_api::startup::{
get_bucket_notification_config, process_lambda_configurations, process_queue_configurations, process_topic_configurations,
};
use crate::{admin, config, startup_runtime_sources, version};
@@ -717,7 +717,7 @@ where
/// When enabled, it spawns a background task that tunes concurrency settings
/// every 60 seconds.
pub async fn init_auto_tuner(ctx: tokio_util::sync::CancellationToken) {
use crate::storage_api::concurrency::get_concurrency_manager;
use crate::storage_api::startup::concurrency::get_concurrency_manager;
use rustfs_io_metrics::AutoTuner;
use rustfs_io_metrics::TunerConfig;
use tracing::{debug, error, info};
@@ -850,7 +850,7 @@ pub async fn init_ftp_system() -> Result<Option<ShutdownHandle>, Box<dyn std::er
config.validate().await?;
// Create FTP server with protocol storage client
let fs = crate::storage_api::ecfs::FS::new();
let fs = crate::storage_api::startup::ecfs::FS::new();
let storage_client = ProtocolStorageClient::new(fs);
let server: FtpsServer<ProtocolStorageClient> = FtpsServer::new(config, storage_client).await?;
let bind_addr = server.config().bind_addr;
@@ -970,7 +970,7 @@ pub async fn init_ftps_system() -> Result<Option<ShutdownHandle>, Box<dyn std::e
config.validate().await?;
// Create FTPS server with protocol storage client
let fs = crate::storage_api::ecfs::FS::new();
let fs = crate::storage_api::startup::ecfs::FS::new();
let storage_client = ProtocolStorageClient::new(fs);
let server: FtpsServer<ProtocolStorageClient> = FtpsServer::new(config, storage_client).await?;
let bind_addr = server.config().bind_addr;
@@ -1088,7 +1088,7 @@ pub async fn init_webdav_system() -> Result<Option<ShutdownHandle>, Box<dyn std:
};
// Create WebDAV server with protocol storage client
let fs = crate::storage_api::ecfs::FS::new();
let fs = crate::storage_api::startup::ecfs::FS::new();
let storage_client = ProtocolStorageClient::new(fs);
let server: WebDavServer<crate::protocols::ProtocolStorageClient> = WebDavServer::new(config, storage_client).await?;
let bind_addr = server.config().bind_addr;
@@ -1222,7 +1222,7 @@ pub async fn init_sftp_system() -> Result<Option<ShutdownHandle>, Box<dyn std::e
// file has insecure permissions.
let host_keys = SftpConfig::load_host_keys(&config.host_key_dir).await?;
let fs = crate::storage_api::ecfs::FS::new();
let fs = crate::storage_api::startup::ecfs::FS::new();
let storage_client = ProtocolStorageClient::new(fs);
let server = SftpServer::new(config.clone(), storage_client, host_keys)?;
rustfs/src/protocols/client.rs
+3 -3
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::runtime_sources::resolve_action_credentials;
use crate::storage_api::ecfs::FS;
use crate::storage_api::protocols::ecfs::FS;
use http::{HeaderMap, Method};
use percent_encoding::{AsciiSet, CONTROLS, utf8_percent_encode};
use rustfs_credentials;
@@ -151,7 +151,7 @@ impl ProtocolStorageClient {
secret_key: params.secret_key.to_string().into(),
});
extensions.insert(crate::storage_api::access::ReqInfo {
extensions.insert(crate::storage_api::protocols::access::ReqInfo {
cred: Some(rustfs_credentials::Credentials {
access_key: params.access_key.to_string(),
secret_key: params.secret_key.to_string(),
@@ -169,7 +169,7 @@ impl ProtocolStorageClient {
object: params.object,
version_id: None,
region: None,
request_context: Some(crate::storage_api::request_context::RequestContext::fallback()),
request_context: Some(crate::storage_api::protocols::request_context::RequestContext::fallback()),
});
let req = S3Request {
rustfs/src/runtime_capabilities.rs
+2 -2
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage_api::{
use crate::storage_api::cluster::{
CapabilitySnapshotError, CapabilityStatus, DiskCapabilities, EndpointServerPools, MemorySamplingState, ObservabilitySnapshot,
ObservabilitySnapshotProvider, PlatformSupport, TopologyCapabilities, TopologySnapshot, TopologySnapshotProvider,
UserspaceProfilingCapability, topology_snapshot_from_endpoint_pools_with_capabilities,
@@ -184,7 +184,7 @@ fn cgroup_memory_status() -> CapabilityStatus {
#[cfg(test)]
mod tests {
use super::*;
use crate::storage_api::{
use crate::storage_api::cluster::{
CapabilityState, Endpoint, Endpoints, ObservabilitySnapshotProvider, PoolEndpoints, TopologySnapshotProvider,
};
rustfs/src/server/event.rs
+2 -2
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use super::{module_switch::resolve_notify_module_state, refresh_persisted_module_switches_from_store, runtime_sources};
use crate::storage_api::{EventArgs as EcstoreEventArgs, StorageObjectInfo, register_event_dispatch_hook};
use crate::storage_api::server::{EventArgs as EcstoreEventArgs, StorageObjectInfo, register_event_dispatch_hook};
use chrono::{DateTime, Utc};
use rustfs_notify::{EventArgs as NotifyEventArgs, NotifyObjectInfo};
use rustfs_s3_types::EventName;
@@ -234,7 +234,7 @@ pub async fn init_event_notifier() {
#[cfg(test)]
mod tests {
use super::{convert_ecstore_object_info, parse_host_and_port};
use crate::storage_api::{StorageObjectInfo, TransitionedObject};
use crate::storage_api::server::{StorageObjectInfo, TransitionedObject};
use chrono::{DateTime, Utc};
use std::{collections::HashMap, sync::Arc};
use time::{Duration, OffsetDateTime};
rustfs/src/server/http.rs
+7 -7
View File
@@ -30,11 +30,11 @@ use crate::server::{
TlsAcceptorHolder, TlsHandshakeFailureKind, build_acceptor_from_loaded, load_tls_material, spawn_reload_loop,
},
};
use crate::storage_api as storage;
use crate::storage_api::request_context::{RequestContext, extract_request_id_from_headers};
use crate::storage_api::rpc::InternodeRpcService;
use crate::storage_api::tonic_service::make_server;
use crate::storage_api::{TONIC_RPC_PREFIX, verify_rpc_signature};
use crate::storage_api::server as storage;
use crate::storage_api::server::request_context::{RequestContext, extract_request_id_from_headers};
use crate::storage_api::server::rpc::InternodeRpcService;
use crate::storage_api::server::tonic_service::make_server;
use crate::storage_api::server::{TONIC_RPC_PREFIX, verify_rpc_signature};
use bytes::Bytes;
use http::{HeaderMap, Method, Request as HttpRequest, Response};
use hyper::body::Incoming;
@@ -1104,7 +1104,7 @@ fn process_connection(
.layer(
TraceLayer::new_for_http()
.make_span_with(|request: &HttpRequest<_>| {
let request_context = request.extensions().get::<crate::storage_api::request_context::RequestContext>();
let request_context = request.extensions().get::<crate::storage_api::server::request_context::RequestContext>();
let request_id = request_context
.map(|ctx| ctx.request_id.as_str())
.unwrap_or("unknown");
@@ -1253,7 +1253,7 @@ fn process_connection(
.layer(
TraceLayer::new_for_http()
.make_span_with(|request: &HttpRequest<_>| {
let request_context = request.extensions().get::<crate::storage_api::request_context::RequestContext>();
let request_context = request.extensions().get::<crate::storage_api::server::request_context::RequestContext>();
let request_id = request_context
.map(|ctx| ctx.request_id.as_str())
.unwrap_or("unknown");
rustfs/src/server/layer.rs
+2 -2
View File
@@ -25,8 +25,8 @@ use crate::server::{
RUSTFS_ADMIN_PREFIX, active_http_requests, collect_dependency_readiness_report, has_path_prefix, is_admin_path,
is_table_catalog_path,
};
use crate::storage_api::apply_cors_headers;
use crate::storage_api::request_context::{
use crate::storage_api::server::apply_cors_headers;
use crate::storage_api::server::request_context::{
RequestContext, extract_request_id_from_headers, extract_trace_context_ids_from_headers, spawn_traced,
};
use bytes::Bytes;
rustfs/src/server/module_switch.rs
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use super::runtime_sources;
use crate::storage_api::{Error as StorageError, read_config, save_config};
use crate::storage_api::server::{Error as StorageError, read_config, save_config};
use serde::{Deserialize, Serialize};
use std::sync::atomic::{AtomicBool, Ordering};
rustfs/src/server/readiness.rs
+2 -2
View File
@@ -15,9 +15,9 @@
use crate::server::runtime_sources;
use crate::server::{ServiceState, ServiceStateManager};
use crate::server::{has_path_prefix, is_table_catalog_path};
use crate::storage_api::{Endpoint, EndpointServerPools, StorageAdminApi, is_dist_erasure};
use crate::storage_api::server::{Endpoint, EndpointServerPools, StorageAdminApi, is_dist_erasure};
#[cfg(test)]
use crate::storage_api::{Endpoints, PoolEndpoints};
use crate::storage_api::server::{Endpoints, PoolEndpoints};
use bytes::Bytes;
use http::{Request as HttpRequest, Response, StatusCode};
use http_body::Body;
rustfs/src/server/runtime_sources.rs
+1 -1
View File
@@ -16,7 +16,7 @@ use crate::runtime_sources::{
NotifyInterface, resolve_endpoints_handle, resolve_iam_ready, resolve_kms_runtime_service_manager,
resolve_lock_clients_handle, resolve_notify_interface, resolve_object_store_handle, resolve_server_config,
};
use crate::storage_api::{ECStore, EndpointServerPools};
use crate::storage_api::server::{ECStore, EndpointServerPools};
use rustfs_kms::KmsServiceManager;
use rustfs_lock::LockClient;
use std::collections::HashMap;
rustfs/src/startup_background.rs
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage_api::ECStore;
use crate::storage_api::startup::ECStore;
use rustfs_heal::{create_ahm_services_cancel_token, heal::storage::ECStoreHealStorage, init_heal_manager};
use rustfs_utils::get_env_bool_with_aliases;
use std::{io::Result, sync::Arc};
rustfs/src/startup_bucket_metadata.rs
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::startup_runtime_sources;
use crate::storage_api::{
use crate::storage_api::startup::{
BucketOperations, BucketOptions, ECStore, init_bucket_metadata_sys, try_migrate_bucket_metadata, try_migrate_iam_config,
};
use std::{
rustfs/src/startup_deadlock.rs
+1 -1
View File
@@ -19,7 +19,7 @@ const LOG_SUBSYSTEM_STARTUP: &str = "startup";
const EVENT_DEADLOCK_DETECTOR_STATE: &str = "deadlock_detector_state";
pub(crate) fn init_deadlock_detector_runtime() {
let detector = crate::storage_api::deadlock_detector::get_deadlock_detector();
let detector = crate::storage_api::startup::deadlock_detector::get_deadlock_detector();
if detector.is_enabled() {
detector.start();
info!(
rustfs/src/startup_fs_guard.rs
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage_api::EndpointServerPools;
use crate::storage_api::startup::EndpointServerPools;
use rustfs_config::{
DEFAULT_RUSTFS_UNSUPPORTED_FS_POLICY, ENV_RUSTFS_UNSUPPORTED_FS_POLICY, RUSTFS_UNSUPPORTED_FS_POLICY_FAIL,
RUSTFS_UNSUPPORTED_FS_POLICY_WARN,
rustfs/src/startup_iam.rs
+1 -1
View File
@@ -14,7 +14,7 @@
use crate::runtime_sources::AppContext;
use crate::server::{ServiceStateManager, publish_ready_when_runtime_ready};
use crate::storage_api::ECStore;
use crate::storage_api::startup::ECStore;
use rustfs_common::{GlobalReadiness, SystemStage};
use rustfs_iam::init_iam_sys;
use rustfs_kms::KmsServiceManager;
rustfs/src/startup_lifecycle.rs
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage_api::ECStore;
use crate::storage_api::startup::ECStore;
use crate::{
server::{ServiceStateManager, ShutdownHandle, wait_for_shutdown},
startup_iam::{IamBootstrapDisposition, publish_ready_for_iam_bootstrap},
rustfs/src/startup_notification.rs
+2 -2
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::init::add_bucket_notification_configuration;
use crate::storage_api::{EndpointServerPools, Result as StorageResult, new_global_notification_sys};
use crate::storage_api::startup::{EndpointServerPools, Result as StorageResult, new_global_notification_sys};
use std::{
future::Future,
io::{Error, Result},
@@ -76,7 +76,7 @@ fn log_embedded_optional_service_skipped(service: &str, err: impl std::fmt::Disp
#[cfg(test)]
mod tests {
use super::init_notification_system_with;
use crate::storage_api::Error as EcstoreError;
use crate::storage_api::startup::Error as EcstoreError;
#[tokio::test]
async fn notification_system_returns_source_error() {
rustfs/src/startup_runtime_sources.rs
+1 -1
View File
@@ -14,7 +14,7 @@
use crate::config::RustFSBufferConfig;
use crate::runtime_sources::{resolve_outbound_tls_generation, resolve_replication_pool_handle};
use crate::storage_api::{DynReplicationPool, set_global_region, set_global_rustfs_port};
use crate::storage_api::startup::{DynReplicationPool, set_global_region, set_global_rustfs_port};
use rustfs_kms::KmsServiceManager;
use rustfs_obs::{GlobalError as ObservabilityError, OtelGuard};
use rustfs_tls_runtime::{OutboundTlsMaterial, TlsGeneration};
rustfs/src/startup_services.rs
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage_api::{ECStore, EndpointServerPools};
use crate::storage_api::startup::{ECStore, EndpointServerPools};
use crate::{
config::Config,
init::{init_buffer_profile_system, init_kms_system},
rustfs/src/startup_shutdown.rs
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage_api::shutdown_background_services;
use crate::storage_api::startup::shutdown_background_services;
use crate::{
server::{ServiceState, ServiceStateManager, ShutdownHandle, ShutdownSignal, shutdown_event_notifier, stop_audit_system},
startup_optional_runtime_sidecars::{
rustfs/src/startup_storage.rs
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::startup_fs_guard::enforce_unsupported_fs_policy;
use crate::storage_api::{
use crate::storage_api::startup::{
ECStore, EndpointServerPools, init_background_replication, init_ecstore_config, init_global_config_sys, init_local_disks,
init_lock_clients, prewarm_local_disk_id_map, set_global_endpoints, try_migrate_server_config, update_erasure_type,
};
rustfs/src/storage_api.rs
+116 -50
View File
@@ -12,69 +12,135 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//! Root-local boundary for storage owner APIs used by startup, server, and
//! other RustFS outer runtime modules.
//! Root-local domain boundaries for storage owner APIs used by RustFS outer
//! runtime modules.
pub(crate) use crate::storage::{
BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG,
BUCKET_TABLE_RESERVED_PREFIX, DynReplicationPool, ECStore, Endpoint, EndpointServerPools, Error, EventArgs, QuotaError,
RUSTFS_META_BUCKET, Result, StorageDeletedObject, StorageError, StorageGetObjectReader, StorageObjectInfo,
StorageObjectOptions, StorageObjectToDelete, StoragePutObjReader, TONIC_RPC_PREFIX, all_local_disk, apply_cors_headers,
bucket_metadata_runtime_initialized, disk_drive_path, disk_endpoint, get_bucket_metadata, get_bucket_notification_config,
get_lock_acquire_timeout, init_background_replication, init_bucket_metadata_sys, init_ecstore_config, init_global_config_sys,
init_local_disks, init_lock_clients, is_dist_erasure, new_global_notification_sys, prewarm_local_disk_id_map,
process_lambda_configurations, process_queue_configurations, process_topic_configurations, read_config,
register_event_dispatch_hook, replication_queue_current_count, save_config, set_global_endpoints, set_global_region,
set_global_rustfs_port, shutdown_background_services, table_catalog_path_hash,
topology_snapshot_from_endpoint_pools_with_capabilities, try_migrate_bucket_metadata, try_migrate_iam_config,
try_migrate_server_config, update_erasure_type, verify_rpc_signature,
};
pub(crate) use rustfs_storage_api::{
BucketOperations, BucketOptions, CapabilitySnapshotError, CapabilityStatus, DiskCapabilities, HTTPPreconditions,
HTTPRangeError, HTTPRangeSpec, ListObjectVersionsInfo, ListObjectsV2Info, ListOperations, MemorySamplingState,
NamespaceLocking, ObjectIO, ObjectInfoOrErr, ObjectOperations, ObservabilitySnapshot, ObservabilitySnapshotProvider,
PlatformSupport, StorageAdminApi, TopologyCapabilities, TopologySnapshot, TopologySnapshotProvider,
UserspaceProfilingCapability, WalkOptions,
};
use rustfs_storage_api as storage_contracts;
pub(crate) mod capacity {
pub(crate) use crate::storage::{all_local_disk, disk_drive_path, disk_endpoint};
}
pub(crate) mod cluster {
pub(crate) use super::storage_contracts::{
CapabilitySnapshotError, CapabilityStatus, DiskCapabilities, MemorySamplingState, ObservabilitySnapshot,
ObservabilitySnapshotProvider, PlatformSupport, TopologyCapabilities, TopologySnapshot, TopologySnapshotProvider,
UserspaceProfilingCapability,
};
pub(crate) use crate::storage::{EndpointServerPools, topology_snapshot_from_endpoint_pools_with_capabilities};
#[cfg(test)]
pub(crate) use super::storage_contracts::CapabilityState;
#[cfg(test)]
pub(crate) use crate::storage::{Endpoint, Endpoints, PoolEndpoints};
pub(crate) mod control_plane {
pub(crate) use crate::storage::ecstore_cluster::{
ClusterControlPlane, ClusterControlPlaneSnapshot, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot,
ClusterPeerHealthSnapshot, ClusterPoolStateSnapshot,
};
}
}
#[cfg(test)]
pub(crate) use crate::storage::{DisksLayout, Endpoints, PoolEndpoints};
#[cfg(test)]
pub(crate) use rustfs_storage_api::{CapabilityState, TransitionedObject};
pub(crate) mod access {
pub(crate) use crate::storage::access::ReqInfo;
pub(crate) mod config_test {
pub(crate) use crate::storage::DisksLayout;
}
pub(crate) mod concurrency {
pub(crate) use crate::storage::concurrency::get_concurrency_manager;
pub(crate) mod error {
pub(crate) use super::storage_contracts::HTTPRangeError;
pub(crate) use crate::storage::{QuotaError, StorageError};
}
pub(crate) mod deadlock_detector {
pub(crate) use crate::storage::deadlock_detector::get_deadlock_detector;
pub(crate) mod protocols {
pub(crate) mod access {
pub(crate) use crate::storage::access::ReqInfo;
}
pub(crate) mod ecfs {
pub(crate) type FS = crate::storage::FS;
}
pub(crate) mod request_context {
pub(crate) use crate::storage::request_context::RequestContext;
}
}
pub(crate) mod ecfs {
pub(crate) type FS = crate::storage::FS;
pub(crate) mod server {
pub(crate) use super::storage_contracts::StorageAdminApi;
pub(crate) use crate::storage::{
ECStore, Endpoint, EndpointServerPools, Error, EventArgs, StorageObjectInfo, TONIC_RPC_PREFIX, apply_cors_headers,
is_dist_erasure, read_config, register_event_dispatch_hook, save_config, verify_rpc_signature,
};
#[cfg(test)]
pub(crate) use super::storage_contracts::TransitionedObject;
#[cfg(test)]
pub(crate) use crate::storage::{Endpoints, PoolEndpoints};
pub(crate) mod ecfs {
pub(crate) type FS = crate::storage::FS;
}
pub(crate) mod request_context {
pub(crate) use crate::storage::request_context::{
RequestContext, extract_request_id_from_headers, extract_trace_context_ids_from_headers, spawn_traced,
};
}
pub(crate) mod rpc {
pub(crate) use crate::storage::rpc::InternodeRpcService;
}
pub(crate) mod tonic_service {
pub(crate) use crate::storage::tonic_service::make_server;
}
}
pub(crate) mod ecstore_cluster {
pub(crate) use crate::storage::ecstore_cluster::{
ClusterControlPlane, ClusterControlPlaneSnapshot, ClusterLocalNodeStorageSnapshot, ClusterMembershipSnapshot,
ClusterPeerHealthSnapshot, ClusterPoolStateSnapshot,
pub(crate) mod startup {
pub(crate) use super::storage_contracts::{BucketOperations, BucketOptions};
pub(crate) use crate::storage::{
DynReplicationPool, ECStore, EndpointServerPools, Result, get_bucket_notification_config, init_background_replication,
init_bucket_metadata_sys, init_ecstore_config, init_global_config_sys, init_local_disks, init_lock_clients,
new_global_notification_sys, prewarm_local_disk_id_map, process_lambda_configurations, process_queue_configurations,
process_topic_configurations, set_global_endpoints, set_global_region, set_global_rustfs_port,
shutdown_background_services, try_migrate_bucket_metadata, try_migrate_iam_config, try_migrate_server_config,
update_erasure_type,
};
#[cfg(test)]
pub(crate) use crate::storage::Error;
pub(crate) mod concurrency {
pub(crate) use crate::storage::concurrency::get_concurrency_manager;
}
pub(crate) mod deadlock_detector {
pub(crate) use crate::storage::deadlock_detector::get_deadlock_detector;
}
pub(crate) mod ecfs {
pub(crate) type FS = crate::storage::FS;
}
}
pub(crate) mod table {
pub(crate) use super::storage_contracts::{
HTTPPreconditions, HTTPRangeSpec, ListObjectVersionsInfo, ListObjectsV2Info, ListOperations, NamespaceLocking, ObjectIO,
ObjectInfoOrErr, ObjectOperations, WalkOptions,
};
pub(crate) use crate::storage::{
BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG,
BUCKET_TABLE_RESERVED_PREFIX, Error, RUSTFS_META_BUCKET, StorageDeletedObject, StorageError, StorageGetObjectReader,
StorageObjectInfo, StorageObjectOptions, StorageObjectToDelete, StoragePutObjReader, get_bucket_metadata,
get_lock_acquire_timeout, table_catalog_path_hash,
};
}
pub(crate) mod request_context {
pub(crate) use crate::storage::request_context::{
RequestContext, extract_request_id_from_headers, extract_trace_context_ids_from_headers, spawn_traced,
};
}
pub(crate) mod workload {
pub(crate) use crate::storage::{bucket_metadata_runtime_initialized, replication_queue_current_count};
pub(crate) mod rpc {
pub(crate) use crate::storage::rpc::InternodeRpcService;
}
pub(crate) mod tonic_service {
pub(crate) use crate::storage::tonic_service::make_server;
pub(crate) mod concurrency {
pub(crate) use crate::storage::concurrency::get_concurrency_manager;
}
}
rustfs/src/table_catalog.rs
+2 -2
View File
@@ -27,7 +27,7 @@ use std::{
time::{Duration as StdDuration, Instant},
};
use crate::storage_api::{
use crate::storage_api::table::{
BUCKET_TABLE_CATALOG_META_PREFIX, BUCKET_TABLE_CATALOG_TABLE_BUCKETS_PREFIX, BUCKET_TABLE_CONFIG,
BUCKET_TABLE_RESERVED_PREFIX, Error as EcstoreError, HTTPPreconditions, HTTPRangeSpec,
ListObjectVersionsInfo as StorageListObjectVersionsInfo, ListObjectsV2Info as StorageListObjectsV2Info,
@@ -48,7 +48,7 @@ use time::{Duration, OffsetDateTime};
use tokio::io::AsyncReadExt;
use uuid::Uuid;
use crate::storage_api::{
use crate::storage_api::table::{
StorageDeletedObject as DeletedObject, StorageGetObjectReader as GetObjectReader, StorageObjectInfo as ObjectInfo,
StorageObjectOptions as ObjectOptions, StorageObjectToDelete as ObjectToDelete, StoragePutObjReader as PutObjReader,
};
rustfs/src/workload_admission.rs
+2 -2
View File
@@ -13,13 +13,13 @@
// limitations under the License.
use crate::runtime_sources::resolve_replication_pool_handle;
use crate::storage_api::{bucket_metadata_runtime_initialized, replication_queue_current_count};
use crate::storage_api::workload::{bucket_metadata_runtime_initialized, replication_queue_current_count};
use rustfs_concurrency::{
AdmissionState, WorkloadAdmissionRegistrySnapshot, WorkloadAdmissionSnapshot, WorkloadAdmissionSnapshotProvider,
WorkloadClass,
};
use crate::storage_api::concurrency::get_concurrency_manager;
use crate::storage_api::workload::concurrency::get_concurrency_manager;
const BUCKET_METADATA_RUNTIME_NOT_INITIALIZED: &str = "bucket metadata runtime not initialized";
const FOREGROUND_WRITE_NOT_EXPOSED_BY_PROVIDER: &str = "foreground write admission not yet exposed by RustFS runtime";
scripts/check_architecture_migration_rules.sh
+28
View File
@@ -183,6 +183,8 @@ RUSTFS_APP_ADMIN_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_admin_stora
RUSTFS_ADMIN_STORAGE_API_ROOT_FACADE_HITS_FILE="${TMP_DIR}/rustfs_admin_storage_api_root_facade_hits.txt"
RUSTFS_ROOT_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_root_storage_api_bypass_hits.txt"
RUSTFS_ROOT_STORAGE_API_CONTRACT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_root_storage_api_contract_bypass_hits.txt"
RUSTFS_ROOT_STORAGE_API_DOMAIN_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_root_storage_api_domain_bypass_hits.txt"
RUSTFS_ADMIN_STORAGE_API_DOMAIN_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_admin_storage_api_domain_bypass_hits.txt"
RUSTFS_APP_STORAGE_API_CONTRACT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_storage_api_contract_bypass_hits.txt"
RUSTFS_ADMIN_STORAGE_API_CONTRACT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_admin_storage_api_contract_bypass_hits.txt"
RUSTFS_STORAGE_DIRECT_APP_CONTEXT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_storage_direct_app_context_bypass_hits.txt"
@@ -1708,6 +1710,32 @@ if [[ -s "$RUSTFS_ROOT_STORAGE_API_CONTRACT_BYPASS_HITS_FILE" ]]; then
report_failure "RustFS root/server/startup storage contracts must stay behind rustfs/src/storage_api.rs: $(paste -sd '; ' "$RUSTFS_ROOT_STORAGE_API_CONTRACT_BYPASS_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename \
'use crate::storage_api(?:\s*(?:;|as\b)|::\{)|crate::storage_api::(?:access|concurrency|deadlock_detector|ecfs|ecstore_cluster|request_context|rpc|tonic_service)\b' \
rustfs/src \
--glob '*.rs' \
--glob '!rustfs/src/storage_api.rs' || true
) >"$RUSTFS_ROOT_STORAGE_API_DOMAIN_BYPASS_HITS_FILE"
if [[ -s "$RUSTFS_ROOT_STORAGE_API_DOMAIN_BYPASS_HITS_FILE" ]]; then
report_failure "RustFS root storage-api consumers must use domain modules from rustfs/src/storage_api.rs: $(paste -sd '; ' "$RUSTFS_ROOT_STORAGE_API_DOMAIN_BYPASS_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename \
'(?:crate::admin|super)::storage_api::(?:ecstore_cluster|ecstore_utils|Error|StorageError|ECStore|Endpoint|EndpointServerPools|Endpoints|PoolEndpoints|PeerRestClient|RequestContext|ReqInfo|authorize_request|spawn_traced|CollectMetricsOpts|MetricType|collect_local_metrics|BucketOperations|BucketOptions|DeleteBucketOptions|HealOperations|ListOperations|MakeBucketOptions|ObjectIO|ObjectOperations|StorageAdminApi|CapabilitySnapshotError|CapabilityState|CapabilityStatus|ObservabilitySnapshot|ObservabilitySnapshotProvider|TopologySnapshot|TopologySnapshotProvider|read_admin_config|read_admin_config_without_migrate|save_admin_config|save_admin_server_config|delete_admin_config|init_admin_config_defaults|STORAGE_CLASS_SUB_SYS|RebalanceStats|RebalanceMeta|RebalanceStopPropagationRecord|StorageObjectOptions|is_reserved_or_invalid_bucket)\b|use (?:crate::admin|super)::storage_api::\{' \
rustfs/src/admin \
--glob '*.rs' \
--glob '!rustfs/src/admin/storage_api.rs' || true
) >"$RUSTFS_ADMIN_STORAGE_API_DOMAIN_BYPASS_HITS_FILE"
if [[ -s "$RUSTFS_ADMIN_STORAGE_API_DOMAIN_BYPASS_HITS_FILE" ]]; then
report_failure "RustFS admin storage-api consumers must use admin domain modules from rustfs/src/admin/storage_api.rs: $(paste -sd '; ' "$RUSTFS_ADMIN_STORAGE_API_DOMAIN_BYPASS_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename '^use rustfs_storage_api|rustfs_storage_api::' \