gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 08:18:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: route admin storage facades (#3866)

Browse Source
This commit is contained in:
Zhengchao An
2026-06-25 22:37:54 +08:00
committed by GitHub
parent 86e9362459
commit 987ea4919f
26 changed files with 554 additions and 522 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/migration-progress.md
+29 -7
View File
@@ -5,9 +5,9 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-app-runtime-global-facade-cleanup`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215`.
- Based on: stacked on API-215 while PR #3856 is pending.
- Branch: `overtrue/arch-admin-facade-storage-api-cleanup`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116/API-117/API-118/API-119/API-120/API-121/API-122/API-123/API-124/API-125/API-126/API-127/API-128/API-129/API-130/API-131/API-132/API-133/API-134/API-135/API-136/API-137/API-138/API-139/API-140/API-141/API-142/API-143/API-144/API-145/API-146/API-147/API-148/API-149/API-150/API-151/API-152/API-153/API-154/API-155/API-156/API-157/API-158/API-159/API-160/API-161/API-162/API-163/API-164/API-165/API-166/API-167/API-168/API-169/API-170/API-171/API-172/API-173/API-174/API-175/API-176/API-177/API-178/API-179/API-180/API-181/API-182/API-183/API-184/API-185/API-186/API-187/API-188/API-189/API-190/API-191/API-192/API-193/API-194/API-195/API-196/API-197/API-198/API-199/API-200/API-201/API-202/API-203/API-204/API-205/API-206/API-207/API-208/API-209/API-210/API-211/API-212/API-213/API-214/API-215/API-216`.
- Based on: API-216 merged; branch replays the admin storage_api cleanup on latest `main`.
- PR type for this branch: `consumer-migration`
- Runtime behavior changes: none.
- Rust code changes: route replication pool, outbound TLS generation, runtime
@@ -46,7 +46,10 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
helpers, plus server readiness/audit/event/module-switch runtime source
helpers, storage request/RPC/SSE runtime source helpers, and admin
handler/service/router runtime source helpers, plus root auth/init/config/protocol/workload, app usecase, storage node-service, remaining admin grouped context import runtime source helpers, app bucket/object/multipart usecase explicit storage imports, app select/bucket/object/multipart explicit S3 DTO plus ECFS owner imports, app-local S3 API response helper imports for bucket/object/multipart usecases, app-local storage helper boundary imports for select/bucket/object/multipart usecases, admin/app test storage helper boundary imports for request context, authorization, object options, and ECFS test harness consumers, app storage IO/compression/set-disk helper boundary imports for object and multipart usecases, app storage error, ETag, and storage-class helper boundary imports for bucket/object/multipart usecases and lifecycle transition tests, app bucket owner facade imports for lifecycle, metadata, object-lock, quota, replication, tagging, target, versioning, and transition test helpers, and app/admin runtime, capacity, data-usage, endpoint, and global facade imports,
through AppContext-first or owner-crate resolver boundaries.
admin root storage facade consumers for config, bucket metadata, replication,
rebalancing, tier, quota, metrics, object zip, site replication, and admin
service config paths, through AppContext-first or owner-crate resolver
boundaries.
- CI/script changes: lock completed owner and test/fuzz boundaries against
bare/glob imports, scattered raw ECStore facade subpaths, and startup
runtime/root-server/table/S3/app shared/app bucket/app ECStore/admin facade
@@ -56,8 +59,8 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
event-bridge thin module regressions, plus IAM runtime-source bypasses;
accept the reviewed AppContext resolver reverse dependencies in the layer
baseline, and block direct admin AppContext resolver consumers outside the
admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, reject app usecase storage wildcard imports, reject app-layer S3 DTO and ECFS wildcard imports, narrow the object-usecase ECFS layer baseline entry to `FS`, reject direct storage S3 API helper imports from app usecase files, reject direct storage helper imports from app select/usecase files, reject completed app/admin storage helper bypasses, reject app usecase bypasses for migrated storage IO/compression/set-disk helpers, reject app usecase/test bypasses for migrated storage error, ETag, and storage-class helpers, reject app root bucket owner facade bypasses from migrated app consumers, and reject app/admin runtime/data-usage root facade regressions.
- Docs changes: record the API-136 through API-216 owner facade and lifecycle
admin runtime-source boundary, block root, app usecase, and storage direct AppContext resolver consumers outside their runtime-source boundaries, catch grouped AppContext imports, reject app usecase storage wildcard imports, reject app-layer S3 DTO and ECFS wildcard imports, narrow the object-usecase ECFS layer baseline entry to `FS`, reject direct storage S3 API helper imports from app usecase files, reject direct storage helper imports from app select/usecase files, reject completed app/admin storage helper bypasses, reject app usecase bypasses for migrated storage IO/compression/set-disk helpers, reject app usecase/test bypasses for migrated storage error, ETag, and storage-class helpers, reject app root bucket owner facade bypasses from migrated app consumers, reject app/admin runtime/data-usage root facade regressions, and reject admin root storage facade regressions from migrated admin consumers.
- Docs changes: record the API-136 through API-217 owner facade and lifecycle
runtime-source cleanup.
## Phase 0 Tasks
@@ -5121,14 +5124,33 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
layer guards, diff hygiene, residual migrated facade scan, Rust risk scan,
fast PR gate, and full PR gate before PR.
- [x] `API-217` Route admin storage facades through storage_api boundary.
- Do: move admin-facing ECStore bucket/config/rebalance/tier/metrics/error
facades from the admin root module into `admin::storage_api`, then route
admin handlers, router, and config/site-replication services through that
boundary.
- Acceptance: admin root no longer exposes the migrated storage facade
modules, aliases, constants, or wrappers, and migration rules reject
root-level regressions for completed admin storage facade consumers.
- Must preserve: admin config IO/default initialization, bucket metadata and
target management, replication/resync DTO handling, rebalance status
projection, quota usage reads, tier admin contracts, metrics collection,
object-zip error mapping, and site-replication metadata serialization.
- Verification: focused RustFS admin compile/tests, formatting, migration and
layer guards, diff hygiene, residual migrated facade scan, Rust risk scan,
fast PR gate, and full PR gate before PR.
## Next PRs
1. `consumer-migration`: batch remaining app/admin facade cleanup after API-216.
1. `consumer-migration`: continue larger app/admin/storage facade batches after API-217.
## Pre-Push Review Log
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | pass | API-217 moves admin storage facade ownership behind admin storage_api and removes the admin root compatibility surface. |
| Migration preservation | pass | Config IO/defaults, bucket metadata/targets, replication/resync, rebalance, quota, tier, metrics, object zip, and site replication keep the same storage implementations. |
| Testing/verification | pass | Focused RustFS admin compile/tests, formatting, migration/layer guards, residual migrated facade scan, Rust risk scan, fast PR gate, and full PR gate are planned before PR. |
| Quality/architecture | pass | API-216 removes app root runtime/global facades and routes app/admin data-usage and runtime consumers through local storage_api boundaries. |
| Migration preservation | pass | Admin info/capacity, data-usage accounting, AppContext fallback runtime sources, lifecycle test disk initialization, tier warm-backend tests, and scanner metrics keep the same storage implementations. |
| Testing/verification | pass | Focused RustFS app compile/tests, formatting, migration/layer guards, residual migrated facade scan, Rust risk scan, fast PR gate, and full PR gate are planned before PR. |
rustfs/src/admin/handlers/account_info.rs
+1 -1
View File
@@ -12,10 +12,10 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::versioning_sys::BucketVersioningSys;
use crate::admin::auth::authenticate_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{resolve_action_credentials, resolve_object_store_handle};
use crate::admin::storage_api::versioning_sys::BucketVersioningSys;
use crate::auth::get_condition_values;
use crate::server::{ADMIN_PREFIX, RemoteAddr};
use http::{HeaderMap, HeaderValue};
rustfs/src/admin/handlers/audit_runtime_config.rs
+1 -1
View File
@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::{read_admin_config_without_migrate, save_admin_server_config};
use crate::admin::handlers::target_descriptor::AdminTargetSpec;
use crate::admin::runtime_sources::resolve_object_store_handle;
use crate::admin::storage_api::{read_admin_config_without_migrate, save_admin_server_config};
use rustfs_audit::{audit_system, start_audit_system as start_global_audit_system, system::AuditSystemState};
use rustfs_config::DEFAULT_DELIMITER;
use rustfs_config::server_config::Config;
rustfs/src/admin/handlers/bucket_meta.rs
+2 -2
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::ecstore_utils::{deserialize, serialize};
use super::super::{
use crate::admin::storage_api::ecstore_utils::{deserialize, serialize};
use crate::admin::storage_api::{
StorageError,
metadata::{
BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE,
rustfs/src/admin/handlers/cluster_snapshot.rs
+3 -3
View File
@@ -14,12 +14,12 @@
use crate::admin::{
auth::validate_admin_request,
ecstore_cluster::{
router::{AdminOperation, Operation, S3Router},
storage_api::ecstore_cluster::{
ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot,
ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState,
ClusterPoolStateSnapshot,
},
router::{AdminOperation, Operation, S3Router},
system,
};
use crate::app::admin_usecase::DefaultAdminUsecase;
@@ -590,7 +590,7 @@ fn summarize_named_capability_statuses<const N: usize>(
#[cfg(test)]
mod tests {
use super::{ClusterSnapshotResponse, ClusterSnapshotSummary, ClusterSnapshotView};
use crate::admin::ecstore_cluster::{
use crate::admin::storage_api::ecstore_cluster::{
ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot,
ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState,
ClusterPoolStateSnapshot,
rustfs/src/admin/handlers/config_admin.rs
+15 -15
View File
@@ -12,11 +12,6 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV};
use super::super::{
RUSTFS_META_BUCKET, STORAGE_CLASS_SUB_SYS, delete_admin_config, read_admin_config, read_admin_config_without_migrate,
save_admin_config, save_admin_server_config,
};
use crate::admin::auth::validate_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{publish_server_config, resolve_object_store_handle, resolve_server_config};
@@ -24,6 +19,11 @@ use crate::admin::service::config::{
apply_dynamic_config_for_subsystem, is_dynamic_config_subsystem, signal_config_snapshot_reload, signal_dynamic_config_reload,
validate_server_config,
};
use crate::admin::storage_api::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV};
use crate::admin::storage_api::{
RUSTFS_META_BUCKET, STORAGE_CLASS_SUB_SYS, delete_admin_config, read_admin_config, read_admin_config_without_migrate,
save_admin_config, save_admin_server_config,
};
use crate::admin::utils::{encode_compatible_admin_payload, is_compat_admin_request, read_compatible_admin_body};
use crate::auth::{check_key_valid, get_session_token};
use crate::error::ApiError;
@@ -708,7 +708,7 @@ fn success_response(config_applied: bool) -> S3Result<S3Response<(StatusCode, Bo
Ok(S3Response::with_headers((StatusCode::OK, Body::default()), headers))
}
fn object_store() -> S3Result<std::sync::Arc<super::super::ECStore>> {
fn object_store() -> S3Result<std::sync::Arc<crate::admin::storage_api::ECStore>> {
resolve_object_store_handle().ok_or_else(|| s3_error!(InternalError, "server storage not initialized"))
}
@@ -753,7 +753,7 @@ fn config_update_sub_system(directives: &[ConfigDirective]) -> S3Result<Option<&
fn validate_config_directives(directives: &[ConfigDirective]) -> S3Result<()> {
if DEFAULT_KVS.get().is_none() {
super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
}
let Some(defaults) = DEFAULT_KVS.get() else {
return Err(s3_error!(InternalError, "config defaults are not initialized"));
@@ -1407,7 +1407,7 @@ fn env_help_key(sub_system: &str, key: &str) -> String {
fn default_help_postfix(sub_system: &str, key: &str) -> String {
if DEFAULT_KVS.get().is_none() {
super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
}
DEFAULT_KVS
@@ -1894,7 +1894,7 @@ mod tests {
#[test]
fn full_config_export_can_be_reapplied() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
let mut original = ServerConfig::new();
apply_set_directives(
&mut original,
@@ -1941,7 +1941,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn build_help_response_appends_default_value_postfix() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
let response = build_help_response(Some("identity_openid"), Some("scopes"), false).expect("help response");
assert_eq!(response.keys_help.len(), 2);
@@ -2054,7 +2054,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_includes_env_override_lines() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
temp_env::with_vars(
[
("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example")),
@@ -2090,7 +2090,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_lists_env_only_targets() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || {
let config = ServerConfig::new();
let rendered = String::from_utf8(
@@ -2113,7 +2113,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_supports_specific_env_only_target_queries() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || {
let config = ServerConfig::new();
let rendered = String::from_utf8(
@@ -2136,7 +2136,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_orders_default_before_named_targets() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_ALPHA", Some("http://alpha.example"))], || {
let mut config = ServerConfig::new();
apply_set_directives(
@@ -2317,7 +2317,7 @@ identity_openid client_id="existing-client""#,
#[test]
fn storage_class_get_target_none_matches_full_export() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
let mut config = ServerConfig::new();
apply_set_directives(
&mut config,
rustfs/src/admin/handlers/heal.rs
+5 -5
View File
@@ -12,11 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::ecstore_utils::is_valid_object_prefix;
use super::super::is_reserved_or_invalid_bucket;
use crate::admin::auth::{authenticate_request, validate_admin_request};
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::resolve_object_store_handle;
use crate::admin::storage_api::ecstore_utils::is_valid_object_prefix;
use crate::admin::storage_api::is_reserved_or_invalid_bucket;
use crate::admin::storage_api::spawn_traced;
use crate::server::ADMIN_PREFIX;
use crate::server::RemoteAddr;
@@ -386,10 +386,10 @@ fn should_handle_root_heal_directly(_hip: &HealInitParams) -> bool {
false
}
fn map_root_heal_status(heal_err: Option<super::super::Error>) -> S3Result<()> {
fn map_root_heal_status(heal_err: Option<crate::admin::storage_api::Error>) -> S3Result<()> {
match heal_err {
None => Ok(()),
Some(super::super::StorageError::NoHealRequired) => {
Some(crate::admin::storage_api::StorageError::NoHealRequired) => {
info!(
event = EVENT_ADMIN_RESPONSE_EMITTED,
component = LOG_COMPONENT_ADMIN_API,
@@ -743,7 +743,6 @@ impl Operation for BackgroundHealStatusHandler {
#[cfg(test)]
mod tests {
use super::super::super::StorageError;
use super::extract_heal_init_params;
use super::{
HealInitParams, HealResp, build_heal_channel_request, encode_background_heal_status, encode_heal_start_success,
@@ -751,6 +750,7 @@ mod tests {
json_response, map_heal_response, map_root_heal_status, should_handle_root_heal_directly, validate_heal_request_mode,
validate_heal_target,
};
use crate::admin::storage_api::StorageError;
use bytes::Bytes;
use http::StatusCode;
use http::Uri;
rustfs/src/admin/handlers/kms_dynamic.rs
+1 -1
View File
@@ -14,12 +14,12 @@
//! KMS dynamic configuration admin API handlers
use super::super::{read_admin_config, save_admin_config};
use crate::admin::auth::validate_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{
resolve_kms_runtime_service_manager, resolve_object_store_handle, resolve_or_init_kms_runtime_service_manager,
};
use crate::admin::storage_api::{read_admin_config, save_admin_config};
use crate::auth::{check_key_valid, get_session_token};
use crate::server::{ADMIN_PREFIX, RemoteAddr};
use hyper::{Method, StatusCode};
rustfs/src/admin/handlers/metrics.rs
+1 -1
View File
@@ -18,10 +18,10 @@
//! keeping the response format explicitly NDJSON. It is not a Prometheus text
//! exposition endpoint.
use super::super::{CollectMetricsOpts, MetricType, collect_local_metrics};
use crate::admin::auth::validate_admin_request;
use crate::admin::router::Operation;
use crate::admin::storage_api::spawn_traced;
use crate::admin::storage_api::{CollectMetricsOpts, MetricType, collect_local_metrics};
use crate::auth::{check_key_valid, get_session_token};
use crate::server::RemoteAddr;
use bytes::Bytes;
rustfs/src/admin/handlers/object_zip_download.rs
+1 -1
View File
@@ -645,7 +645,7 @@ async fn preflight_zip_items(request: &CreateObjectZipDownloadRequest, items: &[
Ok(())
}
fn storage_error_to_s3(err: super::super::Error) -> s3s::S3Error {
fn storage_error_to_s3(err: crate::admin::storage_api::Error) -> s3s::S3Error {
ApiError::from(err).into()
}
rustfs/src/admin/handlers/oidc.rs
+1 -1
View File
@@ -12,11 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::{read_admin_config_without_migrate, save_admin_server_config};
use super::sts::create_oidc_sts_credentials;
use crate::admin::auth::validate_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{resolve_object_store_handle, resolve_oidc_handle, resolve_server_config};
use crate::admin::storage_api::{read_admin_config_without_migrate, save_admin_server_config};
use crate::auth::{check_key_valid, get_session_token};
use crate::server::{ADMIN_PREFIX, MINIO_ADMIN_PREFIX, RemoteAddr};
use http::StatusCode;
rustfs/src/admin/handlers/pools.rs
+3 -3
View File
@@ -27,9 +27,9 @@ use tracing::{error, info, warn};
use crate::{
admin::runtime_sources::{resolve_endpoints_handle, resolve_notification_system, resolve_object_store_handle},
admin::{
EndpointServerPools, PeerRestClient,
auth::validate_admin_request,
router::{AdminOperation, Operation, S3Router},
storage_api::{EndpointServerPools, PeerRestClient},
},
app::admin_usecase::{DefaultAdminUsecase, QueryPoolStatusRequest},
auth::{check_key_valid, get_session_token},
@@ -211,7 +211,7 @@ macro_rules! log_pool_response_emitted {
};
}
fn endpoints_from_context() -> Option<super::super::EndpointServerPools> {
fn endpoints_from_context() -> Option<crate::admin::storage_api::EndpointServerPools> {
resolve_endpoints_handle()
}
@@ -1078,7 +1078,7 @@ mod pools_handler_tests {
pool_admin_pool_parse_error_with_audit, pool_admin_query_parse_error, pool_admin_query_parse_error_with_audit,
validate_pool_mutation_leader, validate_start_decommission_guards,
};
use crate::admin::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints};
use crate::admin::storage_api::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints};
fn test_pool_endpoints(is_local: bool) -> EndpointServerPools {
let mut endpoint = Endpoint::try_from("http://127.0.0.1:9000/disk").expect("test endpoint should parse");
rustfs/src/admin/handlers/quota.rs
+4 -4
View File
@@ -14,12 +14,12 @@
//! Quota admin handlers for HTTP API
use super::super::metadata_sys::BucketMetadataSys;
use super::super::quota::checker::QuotaChecker;
use super::super::quota::{BucketQuota, QuotaError, QuotaOperation};
use crate::admin::auth::{validate_admin_request, validate_admin_request_with_bucket};
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{resolve_bucket_metadata_handle, resolve_object_store_handle};
use crate::admin::storage_api::metadata_sys::BucketMetadataSys;
use crate::admin::storage_api::quota::checker::QuotaChecker;
use crate::admin::storage_api::quota::{BucketQuota, QuotaError, QuotaOperation};
use crate::auth::{check_key_valid, get_session_token};
use crate::server::ADMIN_PREFIX;
use hyper::{Method, StatusCode};
@@ -175,7 +175,7 @@ async fn current_usage_from_context(bucket: &str) -> u64 {
return 0;
};
match super::super::storage_api::data_usage::load_data_usage_from_backend(store).await {
match crate::admin::storage_api::data_usage::load_data_usage_from_backend(store).await {
Ok(data_usage_info) => data_usage_info
.buckets_usage
.get(bucket)
rustfs/src/admin/handlers/rebalance.rs
+7 -7
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::{
use crate::admin::storage_api::{
DiskStat, ECStore, NotificationSys, RebalSaveOpt, RebalanceCleanupWarnings, RebalanceMeta, RebalanceStopPropagationRecord,
StorageError, decode_rebalance_stop_propagation_record,
};
@@ -296,7 +296,7 @@ fn build_rebalance_pool_progress(
now: OffsetDateTime,
stop_time: Option<OffsetDateTime>,
percent_free_goal: f64,
ps: &super::super::RebalanceStats,
ps: &crate::admin::storage_api::RebalanceStats,
) -> Option<RebalPoolProgress> {
let total_bytes_to_rebal = ps.init_capacity as f64 * percent_free_goal - ps.init_free_space as f64;
let terminal_time = ps.info.end_time.or(stop_time);
@@ -339,7 +339,7 @@ fn build_rebalance_pool_statuses(
now: OffsetDateTime,
stop_time: Option<OffsetDateTime>,
percent_free_goal: f64,
pool_stats: &[super::super::RebalanceStats],
pool_stats: &[crate::admin::storage_api::RebalanceStats],
disk_stats: &[DiskStat],
) -> Vec<RebalancePoolStatus> {
pool_stats
@@ -895,10 +895,6 @@ mod offsetdatetime_rfc3339 {
#[cfg(test)]
mod rebalance_handler_tests {
use super::super::super::{
DiskStat, RebalStatus, RebalanceCleanupWarningEntry, RebalanceCleanupWarnings, RebalanceInfo, RebalanceMeta,
RebalanceStats, RebalanceStopPropagationRecord, encode_rebalance_stop_propagation_record,
};
use super::build_rebalance_pool_progress;
use super::calculate_rebalance_progress;
use super::{
@@ -907,6 +903,10 @@ mod rebalance_handler_tests {
rebalance_pool_used, rebalance_query_present, rebalance_remaining_buckets, rebalance_rollback_failure_message,
rebalance_rollback_stop_failure_message, rebalance_start_rollback_error, rebalance_used_pct, rollback_result_label,
};
use crate::admin::storage_api::{
DiskStat, RebalStatus, RebalanceCleanupWarningEntry, RebalanceCleanupWarnings, RebalanceInfo, RebalanceMeta,
RebalanceStats, RebalanceStopPropagationRecord, encode_rebalance_stop_propagation_record,
};
use time::OffsetDateTime;
#[test]
rustfs/src/admin/handlers/replication.rs
+8 -8
View File
@@ -12,17 +12,17 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::StorageError;
use super::super::bucket_target_sys::{BucketTargetError, BucketTargetSys};
use super::super::metadata::BUCKET_TARGETS_FILE;
use super::super::metadata_sys;
use super::super::metadata_sys::get_replication_config;
use super::super::replication::BucketStats;
use super::super::target::BucketTarget;
use crate::admin::auth::validate_admin_request;
use crate::admin::handlers::site_replication::site_replication_peer_deployment_id_for_endpoint;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{resolve_object_store_handle, resolve_replication_stats_handle, resolve_runtime_port};
use crate::admin::storage_api::StorageError;
use crate::admin::storage_api::bucket_target_sys::{BucketTargetError, BucketTargetSys};
use crate::admin::storage_api::metadata::BUCKET_TARGETS_FILE;
use crate::admin::storage_api::metadata_sys;
use crate::admin::storage_api::metadata_sys::get_replication_config;
use crate::admin::storage_api::replication::BucketStats;
use crate::admin::storage_api::target::BucketTarget;
use crate::admin::utils::read_compatible_admin_body;
use crate::auth::{check_key_valid, get_session_token};
use crate::error::ApiError;
@@ -448,7 +448,7 @@ impl Operation for RemoveRemoteTargetHandler {
#[cfg(test)]
mod tests {
use super::{extract_query_params, validate_remote_target_tls_settings};
use crate::admin::target::BucketTarget;
use crate::admin::storage_api::target::BucketTarget;
use http::Uri;
#[test]
rustfs/src/admin/handlers/site_replication.rs
+19 -16
View File
@@ -12,18 +12,6 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::Error as StorageError;
use super::super::bucket_target_sys::BucketTargetSys;
use super::super::ecstore_utils::{deserialize, serialize};
use super::super::metadata::{
BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, BUCKET_REPLICATION_CONFIG,
BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, OBJECT_LOCK_CONFIG,
};
use super::super::metadata_sys;
use super::super::replication::ResyncOpts;
use super::super::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials};
use super::super::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use super::super::{delete_admin_config, read_admin_config, save_admin_config};
use crate::admin::auth::validate_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::runtime_sources::{
@@ -35,6 +23,18 @@ use crate::admin::site_replication_identity::{
canonical_endpoint, deployment_id_for_endpoint, normalize_peer_map_by_identity_with, same_identity_endpoint,
site_identity_key,
};
use crate::admin::storage_api::Error as StorageError;
use crate::admin::storage_api::bucket_target_sys::BucketTargetSys;
use crate::admin::storage_api::ecstore_utils::{deserialize, serialize};
use crate::admin::storage_api::metadata::{
BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, BUCKET_REPLICATION_CONFIG,
BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, OBJECT_LOCK_CONFIG,
};
use crate::admin::storage_api::metadata_sys;
use crate::admin::storage_api::replication::ResyncOpts;
use crate::admin::storage_api::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials};
use crate::admin::storage_api::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use crate::admin::storage_api::{delete_admin_config, read_admin_config, save_admin_config};
use crate::admin::utils::{encode_compatible_admin_payload, read_compatible_admin_body};
use crate::auth::{check_key_valid, get_session_token};
use crate::config::get_config_snapshot;
@@ -652,7 +652,7 @@ async fn site_replication_peer_client() -> S3Result<reqwest::Client> {
built
}
fn runtime_tls_enabled_with(endpoints: Option<&super::super::EndpointServerPools>) -> bool {
fn runtime_tls_enabled_with(endpoints: Option<&crate::admin::storage_api::EndpointServerPools>) -> bool {
if !rustfs_utils::get_env_str(ENV_RUSTFS_TLS_PATH, DEFAULT_RUSTFS_TLS_PATH).is_empty() {
return true;
}
@@ -3357,7 +3357,10 @@ fn is_stale_update(local_updated_at: OffsetDateTime, incoming_updated_at: Option
incoming_updated_at.is_some_and(|incoming_updated_at| incoming_updated_at < local_updated_at)
}
fn bucket_meta_local_updated_at(bucket_meta: &super::super::metadata::BucketMetadata, config_file: &str) -> OffsetDateTime {
fn bucket_meta_local_updated_at(
bucket_meta: &crate::admin::storage_api::metadata::BucketMetadata,
config_file: &str,
) -> OffsetDateTime {
match config_file {
BUCKET_POLICY_CONFIG => bucket_meta.policy_config_updated_at,
BUCKET_TAGGING_CONFIG => bucket_meta.tagging_config_updated_at,
@@ -4600,10 +4603,10 @@ impl Operation for SRRotateServiceAccountHandler {
#[cfg(test)]
mod tests {
use super::super::super::Endpoint;
use super::super::super::{EndpointServerPools, Endpoints, PoolEndpoints};
use super::*;
use crate::admin::runtime_sources::{resolve_outbound_tls_generation, set_test_outbound_tls_generation};
use crate::admin::storage_api::Endpoint;
use crate::admin::storage_api::{EndpointServerPools, Endpoints, PoolEndpoints};
use http::{HeaderMap, HeaderValue, Uri};
use rustfs_policy::policy::action::S3Action;
use serial_test::serial;
rustfs/src/admin/handlers/sts.rs
+1 -1
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::ecstore_utils::serialize;
use super::is_admin::IsAdminHandler;
use crate::admin::storage_api::ecstore_utils::serialize;
use crate::{
admin::runtime_sources::{resolve_action_credentials, resolve_oidc_handle, resolve_token_signing_key},
admin::{
rustfs/src/admin/handlers/table_catalog.rs
+1 -1
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::{ECStore, metadata::table_catalog_path_hash, metadata_sys};
use crate::admin::runtime_sources::{resolve_object_store_handle, resolve_token_signing_key};
use crate::admin::storage_api::{ECStore, metadata::table_catalog_path_hash, metadata_sys};
use crate::admin::{
auth::{AdminResourceScope, validate_admin_request, validate_admin_request_with_bucket_object},
router::{AdminOperation, Operation, S3Router},
rustfs/src/admin/handlers/tier.rs
+2 -2
View File
@@ -13,7 +13,7 @@
// limitations under the License.
#![allow(unused_variables, unused_mut, unused_must_use)]
use super::super::{
use crate::admin::storage_api::{
AdminError, DailyAllTierStats, ERR_TIER_ALREADY_EXISTS, ERR_TIER_BACKEND_IN_USE, ERR_TIER_BACKEND_NOT_EMPTY,
ERR_TIER_CONNECT_ERR, ERR_TIER_INVALID_CREDENTIALS, ERR_TIER_MISSING_CREDENTIALS, ERR_TIER_NAME_NOT_UPPERCASE,
ERR_TIER_NOT_FOUND, TierConfig, TierCreds, TierType, storageclass,
@@ -931,8 +931,8 @@ impl Operation for ClearTier {
#[cfg(test)]
mod tests {
use super::super::super::lifecycle::tier_last_day_stats::LastDayTierStats;
use super::*;
use crate::admin::storage_api::lifecycle::tier_last_day_stats::LastDayTierStats;
use http::Uri;
use matchit::Router;
rustfs/src/admin/handlers/trace.rs
+1 -1
View File
@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::PeerRestClient;
use crate::admin::router::Operation;
use crate::admin::runtime_sources::resolve_endpoints_handle;
use crate::admin::storage_api::PeerRestClient;
use http::StatusCode;
use hyper::Uri;
use matchit::Params;
rustfs/src/admin/mod.rs
-410
View File
@@ -88,413 +88,3 @@ fn register_admin_routes(r: &mut S3Router<AdminOperation>) -> std::io::Result<()
Ok(())
}
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::ops::Deref;
use std::sync::Arc;
mod ecstore_bucket {
pub(crate) use crate::storage::ecstore_bucket::{
bandwidth, bucket_target_sys, lifecycle, metadata, metadata_sys, quota, replication, target, utils, versioning,
versioning_sys,
};
}
mod ecstore_capacity {
pub(crate) use crate::storage::ecstore_capacity::is_reserved_or_invalid_bucket;
}
mod ecstore_client {
pub(crate) use crate::storage::ecstore_client::admin_handler_utils;
}
mod ecstore_cluster {
pub(crate) use crate::storage::ecstore_cluster::{
ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot,
ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState,
ClusterPoolStateSnapshot,
};
}
mod ecstore_config {
pub(crate) use crate::storage::ecstore_config::{com, init, storageclass};
}
#[allow(unused_imports)]
mod ecstore_disk {
pub(crate) use crate::storage::ecstore_disk::{RUSTFS_META_BUCKET, endpoint};
}
mod ecstore_error {
pub(crate) use crate::storage::ecstore_error::StorageError;
}
#[allow(unused_imports)]
mod ecstore_layout {
pub(crate) use crate::storage::ecstore_layout::{EndpointServerPools, Endpoints, PoolEndpoints};
}
mod ecstore_metrics {
pub(crate) use crate::storage::ecstore_metrics::{CollectMetricsOpts, MetricType, collect_local_metrics};
}
mod ecstore_notification {
pub(crate) use crate::storage::ecstore_notification::NotificationSys;
}
#[allow(unused_imports)]
mod ecstore_rebalance {
pub(crate) use crate::storage::ecstore_rebalance::{
DiskStat, RebalSaveOpt, RebalStatus, RebalanceCleanupWarningEntry, RebalanceCleanupWarnings, RebalanceInfo,
RebalanceMeta, RebalanceStats, RebalanceStopPropagationRecord, decode_rebalance_stop_propagation_record,
encode_rebalance_stop_propagation_record,
};
}
mod ecstore_rpc {
pub(crate) use crate::storage::ecstore_rpc::PeerRestClient;
}
mod ecstore_storage {
pub(crate) use crate::storage::ecstore_storage::ECStore;
}
mod ecstore_tier {
pub(crate) use crate::storage::ecstore_tier::{tier, tier_admin, tier_config, tier_handlers};
}
pub(crate) const RUSTFS_META_BUCKET: &str = ecstore_disk::RUSTFS_META_BUCKET;
pub(crate) const STORAGE_CLASS_SUB_SYS: &str = ecstore_config::com::STORAGE_CLASS_SUB_SYS;
pub(crate) type AdminError = ecstore_client::admin_handler_utils::AdminError;
pub(crate) type CollectMetricsOpts = ecstore_metrics::CollectMetricsOpts;
pub(crate) type DiskStat = ecstore_rebalance::DiskStat;
pub(crate) type ECStore = ecstore_storage::ECStore;
pub(crate) type EndpointServerPools = ecstore_layout::EndpointServerPools;
pub(crate) type MetricType = ecstore_metrics::MetricType;
pub(crate) type NotificationSys = ecstore_notification::NotificationSys;
pub(crate) type PeerRestClient = ecstore_rpc::PeerRestClient;
pub(crate) type RebalSaveOpt = ecstore_rebalance::RebalSaveOpt;
pub(crate) type RebalanceCleanupWarnings = ecstore_rebalance::RebalanceCleanupWarnings;
pub(crate) type RebalanceMeta = ecstore_rebalance::RebalanceMeta;
pub(crate) type RebalanceStats = ecstore_rebalance::RebalanceStats;
pub(crate) type RebalanceStopPropagationRecord = ecstore_rebalance::RebalanceStopPropagationRecord;
pub(crate) type StorageError = ecstore_error::StorageError;
pub(crate) type Error = StorageError;
pub(crate) type Result<T> = core::result::Result<T, Error>;
pub(crate) type TierConfig = ecstore_tier::tier_config::TierConfig;
pub(crate) type TierCreds = ecstore_tier::tier_admin::TierCreds;
pub(crate) type TierType = ecstore_tier::tier_config::TierType;
#[cfg(test)]
pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint;
#[cfg(test)]
pub(crate) type Endpoints = ecstore_layout::Endpoints;
#[cfg(test)]
pub(crate) type PoolEndpoints = ecstore_layout::PoolEndpoints;
#[cfg(test)]
pub(crate) type RebalStatus = ecstore_rebalance::RebalStatus;
#[cfg(test)]
pub(crate) type RebalanceCleanupWarningEntry = ecstore_rebalance::RebalanceCleanupWarningEntry;
#[cfg(test)]
pub(crate) type RebalanceInfo = ecstore_rebalance::RebalanceInfo;
pub(crate) fn decode_rebalance_stop_propagation_record(message: &str) -> Option<RebalanceStopPropagationRecord> {
ecstore_rebalance::decode_rebalance_stop_propagation_record(message)
}
#[cfg(test)]
pub(crate) fn encode_rebalance_stop_propagation_record(record: &RebalanceStopPropagationRecord) -> String {
ecstore_rebalance::encode_rebalance_stop_propagation_record(record)
}
pub(crate) trait AdminReplicationConfigExt {
fn filter_target_arns(&self, obj: &replication::ObjectOpts) -> Vec<String>;
fn has_existing_object_replication(&self, arn: &str) -> (bool, bool);
}
impl AdminReplicationConfigExt for s3s::dto::ReplicationConfiguration {
fn filter_target_arns(&self, obj: &replication::ObjectOpts) -> Vec<String> {
<s3s::dto::ReplicationConfiguration as ecstore_bucket::replication::ReplicationConfigurationExt>::filter_target_arns(
self, obj,
)
}
fn has_existing_object_replication(&self, arn: &str) -> (bool, bool) {
<s3s::dto::ReplicationConfiguration as ecstore_bucket::replication::ReplicationConfigurationExt>::has_existing_object_replication(
self, arn,
)
}
}
pub(crate) trait AdminVersioningConfigExt {
fn enabled(&self) -> bool;
}
impl AdminVersioningConfigExt for s3s::dto::VersioningConfiguration {
fn enabled(&self) -> bool {
<s3s::dto::VersioningConfiguration as ecstore_bucket::versioning::VersioningApi>::enabled(self)
}
}
pub(crate) mod bandwidth {
pub(crate) mod monitor {
pub(crate) type BandwidthDetails = super::super::ecstore_bucket::bandwidth::monitor::BandwidthDetails;
}
}
pub(crate) mod bucket_target_sys {
pub(crate) type AdvancedPutOptions = super::ecstore_bucket::bucket_target_sys::AdvancedPutOptions;
pub(crate) type BucketTargetError = super::ecstore_bucket::bucket_target_sys::BucketTargetError;
pub(crate) type BucketTargetSys = super::ecstore_bucket::bucket_target_sys::BucketTargetSys;
pub(crate) type PutObjectOptions = super::ecstore_bucket::bucket_target_sys::PutObjectOptions;
pub(crate) type RemoveObjectOptions = super::ecstore_bucket::bucket_target_sys::RemoveObjectOptions;
pub(crate) type S3ClientError = super::ecstore_bucket::bucket_target_sys::S3ClientError;
pub(crate) type TargetClient = super::ecstore_bucket::bucket_target_sys::TargetClient;
}
pub(crate) mod lifecycle {
pub(crate) mod tier_last_day_stats {
#[cfg(test)]
pub(crate) type LastDayTierStats = super::super::ecstore_bucket::lifecycle::tier_last_day_stats::LastDayTierStats;
}
}
pub(crate) mod metadata {
pub(crate) const BUCKET_CORS_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_CORS_CONFIG;
pub(crate) const BUCKET_LIFECYCLE_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_LIFECYCLE_CONFIG;
pub(crate) const BUCKET_NOTIFICATION_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_NOTIFICATION_CONFIG;
pub(crate) const BUCKET_POLICY_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_POLICY_CONFIG;
pub(crate) const BUCKET_QUOTA_CONFIG_FILE: &str = super::ecstore_bucket::metadata::BUCKET_QUOTA_CONFIG_FILE;
pub(crate) const BUCKET_REPLICATION_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_REPLICATION_CONFIG;
pub(crate) const BUCKET_SSECONFIG: &str = super::ecstore_bucket::metadata::BUCKET_SSECONFIG;
pub(crate) const BUCKET_TAGGING_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_TAGGING_CONFIG;
pub(crate) const BUCKET_TARGETS_FILE: &str = super::ecstore_bucket::metadata::BUCKET_TARGETS_FILE;
pub(crate) const BUCKET_VERSIONING_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_VERSIONING_CONFIG;
pub(crate) const OBJECT_LOCK_CONFIG: &str = super::ecstore_bucket::metadata::OBJECT_LOCK_CONFIG;
pub(crate) type BucketMetadata = super::ecstore_bucket::metadata::BucketMetadata;
pub(crate) fn table_catalog_path_hash(value: &str) -> String {
super::ecstore_bucket::metadata::table_catalog_path_hash(value)
}
}
pub(crate) mod metadata_sys {
use std::sync::Arc;
use rustfs_policy::policy::BucketPolicy;
use s3s::dto::{
BucketLifecycleConfiguration, NotificationConfiguration, ObjectLockConfiguration, ServerSideEncryptionConfiguration,
Tagging, VersioningConfiguration,
};
use time::OffsetDateTime;
use super::Result;
use super::metadata::BucketMetadata;
use super::quota::BucketQuota;
use super::target::BucketTargets;
pub(crate) type BucketMetadataSys = super::ecstore_bucket::metadata_sys::BucketMetadataSys;
pub(crate) async fn get(bucket: &str) -> Result<Arc<BucketMetadata>> {
super::ecstore_bucket::metadata_sys::get(bucket).await
}
pub(crate) async fn update(bucket: &str, config_file: &str, data: Vec<u8>) -> Result<OffsetDateTime> {
super::ecstore_bucket::metadata_sys::update(bucket, config_file, data).await
}
pub(crate) async fn delete(bucket: &str, config_file: &str) -> Result<OffsetDateTime> {
super::ecstore_bucket::metadata_sys::delete(bucket, config_file).await
}
pub(crate) async fn get_bucket_policy(bucket: &str) -> Result<(BucketPolicy, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_bucket_policy(bucket).await
}
pub(crate) async fn get_bucket_targets_config(bucket: &str) -> Result<BucketTargets> {
super::ecstore_bucket::metadata_sys::get_bucket_targets_config(bucket).await
}
pub(crate) async fn get_config_from_disk(bucket: &str) -> Result<BucketMetadata> {
super::ecstore_bucket::metadata_sys::get_config_from_disk(bucket).await
}
pub(crate) async fn get_lifecycle_config(bucket: &str) -> Result<(BucketLifecycleConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_lifecycle_config(bucket).await
}
pub(crate) async fn get_notification_config(bucket: &str) -> Result<Option<NotificationConfiguration>> {
super::ecstore_bucket::metadata_sys::get_notification_config(bucket).await
}
pub(crate) async fn get_object_lock_config(bucket: &str) -> Result<(ObjectLockConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_object_lock_config(bucket).await
}
pub(crate) async fn get_quota_config(bucket: &str) -> Result<(BucketQuota, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_quota_config(bucket).await
}
pub(crate) async fn get_replication_config(bucket: &str) -> Result<(s3s::dto::ReplicationConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_replication_config(bucket).await
}
pub(crate) async fn get_sse_config(bucket: &str) -> Result<(ServerSideEncryptionConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_sse_config(bucket).await
}
pub(crate) async fn get_tagging_config(bucket: &str) -> Result<(Tagging, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_tagging_config(bucket).await
}
pub(crate) async fn get_versioning_config(bucket: &str) -> Result<(VersioningConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_versioning_config(bucket).await
}
pub(crate) async fn list_bucket_targets(bucket: &str) -> Result<BucketTargets> {
super::ecstore_bucket::metadata_sys::list_bucket_targets(bucket).await
}
}
pub(crate) mod quota {
pub(crate) mod checker {
pub(crate) type QuotaChecker = super::super::ecstore_bucket::quota::checker::QuotaChecker;
}
pub(crate) type BucketQuota = super::ecstore_bucket::quota::BucketQuota;
pub(crate) type QuotaError = super::ecstore_bucket::quota::QuotaError;
pub(crate) type QuotaOperation = super::ecstore_bucket::quota::QuotaOperation;
}
pub(crate) mod replication {
pub(crate) type BucketReplicationResyncStatus = super::ecstore_bucket::replication::BucketReplicationResyncStatus;
pub(crate) type BucketStats = super::ecstore_bucket::replication::BucketStats;
pub(crate) type ObjectOpts = super::ecstore_bucket::replication::ObjectOpts;
pub(crate) type ResyncOpts = super::ecstore_bucket::replication::ResyncOpts;
#[cfg(test)]
pub(crate) type ResyncStatusType = super::ecstore_bucket::replication::ResyncStatusType;
#[cfg(test)]
pub(crate) type TargetReplicationResyncStatus = super::ecstore_bucket::replication::TargetReplicationResyncStatus;
}
pub(crate) mod target {
#[allow(clippy::upper_case_acronyms)]
pub(crate) type ARN = super::ecstore_bucket::target::ARN;
pub(crate) type BucketTarget = super::ecstore_bucket::target::BucketTarget;
pub(crate) type BucketTargetType = super::ecstore_bucket::target::BucketTargetType;
pub(crate) type BucketTargets = super::ecstore_bucket::target::BucketTargets;
pub(crate) type Credentials = super::ecstore_bucket::target::Credentials;
}
pub(crate) mod ecstore_utils {
pub(crate) fn deserialize<T>(input: &[u8]) -> s3s::xml::DeResult<T>
where
T: for<'xml> s3s::xml::Deserialize<'xml>,
{
super::ecstore_bucket::utils::deserialize(input)
}
pub(crate) fn is_valid_object_prefix(object: &str) -> bool {
super::ecstore_bucket::utils::is_valid_object_prefix(object)
}
pub(crate) fn serialize<T: s3s::xml::Serialize>(val: &T) -> s3s::xml::SerResult<Vec<u8>> {
super::ecstore_bucket::utils::serialize(val)
}
}
pub(crate) mod versioning {}
pub(crate) mod versioning_sys {
pub(crate) type BucketVersioningSys = super::ecstore_bucket::versioning_sys::BucketVersioningSys;
}
pub(crate) mod storageclass {
pub(crate) const INLINE_BLOCK_ENV: &str = super::ecstore_config::storageclass::INLINE_BLOCK_ENV;
pub(crate) const OPTIMIZE_ENV: &str = super::ecstore_config::storageclass::OPTIMIZE_ENV;
pub(crate) const RRS: &str = super::ecstore_config::storageclass::RRS;
pub(crate) const RRS_ENV: &str = super::ecstore_config::storageclass::RRS_ENV;
pub(crate) const STANDARD: &str = super::ecstore_config::storageclass::STANDARD;
pub(crate) const STANDARD_ENV: &str = super::ecstore_config::storageclass::STANDARD_ENV;
pub(crate) type Config = super::ecstore_config::storageclass::Config;
pub(crate) fn lookup_config(kvs: &rustfs_config::server_config::KVS, set_drive_count: usize) -> super::Result<Config> {
super::ecstore_config::storageclass::lookup_config(kvs, set_drive_count)
}
}
pub(crate) type DailyAllTierStats = ecstore_bucket::lifecycle::tier_last_day_stats::DailyAllTierStats;
pub(crate) fn is_reserved_or_invalid_bucket(bucket_entry: &str, strict: bool) -> bool {
ecstore_capacity::is_reserved_or_invalid_bucket(bucket_entry, strict)
}
pub(crate) async fn read_admin_config(api: Arc<ECStore>, file: &str) -> Result<Vec<u8>> {
ecstore_config::com::read_config(api, file).await
}
pub(crate) async fn read_admin_config_without_migrate(api: Arc<ECStore>) -> Result<rustfs_config::server_config::Config> {
ecstore_config::com::read_config_without_migrate(api).await
}
pub(crate) async fn save_admin_config(api: Arc<ECStore>, file: &str, data: Vec<u8>) -> Result<()> {
ecstore_config::com::save_config(api, file, data).await
}
pub(crate) async fn delete_admin_config(api: Arc<ECStore>, file: &str) -> Result<()> {
ecstore_config::com::delete_config(api, file).await
}
pub(crate) async fn save_admin_server_config(api: Arc<ECStore>, cfg: &rustfs_config::server_config::Config) -> Result<()> {
ecstore_config::com::save_server_config(api, cfg).await
}
pub(crate) fn init_admin_config_defaults() {
ecstore_config::init();
}
pub(crate) async fn collect_local_metrics(
types: MetricType,
opts: &CollectMetricsOpts,
) -> rustfs_madmin::metrics::RealtimeMetrics {
ecstore_metrics::collect_local_metrics(types, opts).await
}
pub(crate) struct AdminErrorRef(fn() -> &'static AdminError);
impl Deref for AdminErrorRef {
type Target = AdminError;
fn deref(&self) -> &Self::Target {
(self.0)()
}
}
pub(crate) static ERR_TIER_BACKEND_IN_USE: AdminErrorRef = AdminErrorRef(|| &ecstore_tier::tier::ERR_TIER_BACKEND_IN_USE);
pub(crate) static ERR_TIER_BACKEND_NOT_EMPTY: AdminErrorRef = AdminErrorRef(|| &ecstore_tier::tier::ERR_TIER_BACKEND_NOT_EMPTY);
pub(crate) static ERR_TIER_MISSING_CREDENTIALS: AdminErrorRef =
AdminErrorRef(|| &ecstore_tier::tier::ERR_TIER_MISSING_CREDENTIALS);
pub(crate) static ERR_TIER_ALREADY_EXISTS: AdminErrorRef =
AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_ALREADY_EXISTS);
pub(crate) static ERR_TIER_CONNECT_ERR: AdminErrorRef = AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_CONNECT_ERR);
pub(crate) static ERR_TIER_INVALID_CREDENTIALS: AdminErrorRef =
AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_INVALID_CREDENTIALS);
pub(crate) static ERR_TIER_NAME_NOT_UPPERCASE: AdminErrorRef =
AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_NAME_NOT_UPPERCASE);
pub(crate) static ERR_TIER_NOT_FOUND: AdminErrorRef = AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_NOT_FOUND);
rustfs/src/admin/router.rs
+24 -22
View File
@@ -12,16 +12,18 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::PeerRestClient;
use super::bandwidth::monitor::BandwidthDetails;
use super::bucket_target_sys::{BucketTargetSys, PutObjectOptions, RemoveObjectOptions, S3ClientError, TargetClient};
use super::metadata::BUCKET_TARGETS_FILE;
use super::metadata_sys;
use super::read_admin_config_without_migrate;
use super::replication::{BucketReplicationResyncStatus, BucketStats, ObjectOpts, ResyncOpts};
use super::target::{BucketTarget, BucketTargetType, BucketTargets};
use super::versioning_sys::BucketVersioningSys;
use super::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use super::storage_api::PeerRestClient;
use super::storage_api::bandwidth::monitor::BandwidthDetails;
use super::storage_api::bucket_target_sys::{
BucketTargetSys, PutObjectOptions, RemoveObjectOptions, S3ClientError, TargetClient,
};
use super::storage_api::metadata::BUCKET_TARGETS_FILE;
use super::storage_api::metadata_sys;
use super::storage_api::read_admin_config_without_migrate;
use super::storage_api::replication::{BucketReplicationResyncStatus, BucketStats, ObjectOpts, ResyncOpts};
use super::storage_api::target::{BucketTarget, BucketTargetType, BucketTargets};
use super::storage_api::versioning_sys::BucketVersioningSys;
use super::storage_api::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _, StorageError};
use crate::admin::console::{is_console_path, make_console_server};
use crate::admin::handlers::oidc::is_oidc_path;
use crate::admin::runtime_sources::{
@@ -1414,7 +1416,7 @@ async fn ensure_replication_bucket_exists(bucket: &str) -> S3Result<()> {
async fn ensure_replication_config_exists(bucket: &str) -> S3Result<()> {
match metadata_sys::get_replication_config(bucket).await {
Ok(_) => Ok(()),
Err(super::StorageError::ConfigNotFound) => Err(s3_error!(ReplicationConfigurationNotFoundError)),
Err(StorageError::ConfigNotFound) => Err(s3_error!(ReplicationConfigurationNotFoundError)),
Err(err) => Err(ApiError::from(err).into()),
}
}
@@ -1938,7 +1940,7 @@ async fn resolve_replication_target_client(bucket: &str, target: &BucketTarget)
fn build_replication_probe_put_options(now: OffsetDateTime) -> PutObjectOptions {
PutObjectOptions {
internal: super::bucket_target_sys::AdvancedPutOptions {
internal: super::storage_api::bucket_target_sys::AdvancedPutOptions {
source_version_id: Uuid::new_v4().to_string(),
replication_status: ReplicationStatusType::Replica,
source_mtime: now,
@@ -2053,7 +2055,7 @@ async fn source_bucket_requires_object_lock(bucket: &str) -> S3Result<bool> {
.object_lock_enabled
.as_ref()
.is_some_and(|state| state.as_str() == s3s::dto::ObjectLockEnabled::ENABLED)),
Err(super::StorageError::ConfigNotFound) => Ok(false),
Err(StorageError::ConfigNotFound) => Ok(false),
Err(err) => Err(ApiError::from(err).into()),
}
}
@@ -2767,7 +2769,7 @@ mod tests {
#[test]
fn apply_replication_reset_to_targets_updates_matching_target() {
let mut targets = BucketTargets {
targets: vec![super::super::target::BucketTarget {
targets: vec![crate::admin::storage_api::target::BucketTarget {
arn: "arn:target".to_string(),
..Default::default()
}],
@@ -2789,10 +2791,10 @@ mod tests {
let mut status = BucketReplicationResyncStatus::new();
status.targets_map.insert(
"arn:z".to_string(),
super::super::replication::TargetReplicationResyncStatus {
crate::admin::storage_api::replication::TargetReplicationResyncStatus {
resync_id: "rid-z".to_string(),
last_update: Some(datetime!(2025-01-03 00:00 UTC)),
resync_status: super::super::replication::ResyncStatusType::ResyncFailed,
resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncFailed,
failed_count: 2,
failed_size: 4,
bucket: "bucket-z".to_string(),
@@ -2802,10 +2804,10 @@ mod tests {
);
status.targets_map.insert(
"arn:a".to_string(),
super::super::replication::TargetReplicationResyncStatus {
crate::admin::storage_api::replication::TargetReplicationResyncStatus {
resync_id: "rid-a".to_string(),
last_update: Some(datetime!(2025-01-02 00:00 UTC)),
resync_status: super::super::replication::ResyncStatusType::ResyncCompleted,
resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncCompleted,
replicated_count: 3,
replicated_size: 9,
bucket: "bucket-a".to_string(),
@@ -2835,10 +2837,10 @@ mod tests {
let mut status = BucketReplicationResyncStatus::new();
status.targets_map.insert(
"arn:z".to_string(),
super::super::replication::TargetReplicationResyncStatus {
crate::admin::storage_api::replication::TargetReplicationResyncStatus {
resync_id: "rid-z".to_string(),
last_update: Some(datetime!(2025-02-03 00:00 UTC)),
resync_status: super::super::replication::ResyncStatusType::ResyncFailed,
resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncFailed,
failed_count: 2,
failed_size: 4,
bucket: "bucket-z".to_string(),
@@ -2848,10 +2850,10 @@ mod tests {
);
status.targets_map.insert(
"arn:a".to_string(),
super::super::replication::TargetReplicationResyncStatus {
crate::admin::storage_api::replication::TargetReplicationResyncStatus {
resync_id: "rid-a".to_string(),
last_update: Some(datetime!(2025-02-02 00:00 UTC)),
resync_status: super::super::replication::ResyncStatusType::ResyncCompleted,
resync_status: crate::admin::storage_api::replication::ResyncStatusType::ResyncCompleted,
replicated_count: 3,
replicated_size: 9,
bucket: "bucket-a".to_string(),
rustfs/src/admin/service/config.rs
+7 -7
View File
@@ -12,12 +12,12 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::storageclass;
use super::super::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate};
use crate::admin::runtime_sources::{
AppContext, get_global_app_context, publish_server_config, publish_storage_class_config, resolve_notification_system,
resolve_object_store_handle, resolve_object_store_handle_for_context,
};
use crate::admin::storage_api::storageclass;
use crate::admin::storage_api::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate};
use rustfs_audit::reload_audit_config;
use rustfs_config::audit::{AUDIT_MQTT_SUB_SYS, AUDIT_REDIS_DEFAULT_CHANNEL, AUDIT_WEBHOOK_SUB_SYS};
use rustfs_config::notify::{NOTIFY_MQTT_SUB_SYS, NOTIFY_REDIS_DEFAULT_CHANNEL, NOTIFY_WEBHOOK_SUB_SYS};
@@ -383,8 +383,8 @@ pub async fn signal_config_snapshot_reload() {
#[cfg(test)]
mod tests {
use super::super::super::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG};
use super::*;
use crate::admin::storage_api::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG};
use rustfs_config::notify::NOTIFY_WEBHOOK_SUB_SYS;
use rustfs_config::oidc::{OIDC_CLIENT_ID, OIDC_CONFIG_URL, OIDC_SCOPES};
use rustfs_config::{HEAL_SUB_SYS, SCANNER_SUB_SYS};
@@ -440,7 +440,7 @@ mod tests {
#[test]
fn validate_notify_subsystem_config_rejects_invalid_webhook_endpoint() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(NOTIFY_WEBHOOK_SUB_SYS).expect("notify webhook defaults");
let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target");
@@ -454,7 +454,7 @@ mod tests {
#[test]
fn validate_audit_subsystem_config_rejects_relative_queue_dir() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(AUDIT_MQTT_SUB_SYS).expect("audit mqtt defaults");
let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target");
@@ -469,7 +469,7 @@ mod tests {
#[test]
fn validate_identity_openid_config_rejects_missing_openid_scope() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults");
let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target");
@@ -486,7 +486,7 @@ mod tests {
#[test]
fn validate_identity_openid_config_rejects_invalid_named_provider_id() {
super::super::super::init_admin_config_defaults();
crate::admin::storage_api::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults");
let default_kvs = targets.get(DEFAULT_DELIMITER).cloned().expect("default target");
rustfs/src/admin/service/site_replication.rs
+2 -2
View File
@@ -12,10 +12,10 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::Error as StorageError;
use super::super::{read_admin_config, save_admin_config};
use crate::admin::runtime_sources::{AppContext, get_global_app_context, resolve_object_store_handle_for_context};
use crate::admin::site_replication_identity::{deployment_id_for_endpoint, normalize_peer_map_by_identity_with};
use crate::admin::storage_api::Error as StorageError;
use crate::admin::storage_api::{read_admin_config, save_admin_config};
use rustfs_madmin::PeerInfo;
use s3s::{S3Error, S3ErrorCode, S3Result};
use serde_json::{Map, Value};
rustfs/src/admin/storage_api.rs
+396
View File
@@ -12,6 +12,402 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use std::ops::Deref;
use std::sync::Arc;
mod ecstore_bucket {
pub(crate) use crate::storage::ecstore_bucket::{
bandwidth, bucket_target_sys, lifecycle, metadata, metadata_sys, quota, replication, target, utils, versioning,
versioning_sys,
};
}
mod ecstore_capacity {
pub(crate) use crate::storage::ecstore_capacity::is_reserved_or_invalid_bucket;
}
mod ecstore_client {
pub(crate) use crate::storage::ecstore_client::admin_handler_utils;
}
pub(crate) mod ecstore_cluster {
pub(crate) use crate::storage::ecstore_cluster::{
ClusterDriveMembership, ClusterEndpointType, ClusterLocalNodeStorage, ClusterLocalNodeStorageSnapshot,
ClusterMembershipSnapshot, ClusterNodeMembership, ClusterPeerHealth, ClusterPeerHealthSnapshot, ClusterPoolState,
ClusterPoolStateSnapshot,
};
}
mod ecstore_config {
pub(crate) use crate::storage::ecstore_config::{com, init, storageclass};
}
#[allow(unused_imports)]
mod ecstore_disk {
pub(crate) use crate::storage::ecstore_disk::{RUSTFS_META_BUCKET, endpoint};
}
mod ecstore_error {
pub(crate) use crate::storage::ecstore_error::StorageError;
}
#[allow(unused_imports)]
mod ecstore_layout {
pub(crate) use crate::storage::ecstore_layout::{EndpointServerPools, Endpoints, PoolEndpoints};
}
mod ecstore_metrics {
pub(crate) use crate::storage::ecstore_metrics::{CollectMetricsOpts, MetricType, collect_local_metrics};
}
mod ecstore_notification {
pub(crate) use crate::storage::ecstore_notification::NotificationSys;
}
#[allow(unused_imports)]
mod ecstore_rebalance {
pub(crate) use crate::storage::ecstore_rebalance::{
DiskStat, RebalSaveOpt, RebalStatus, RebalanceCleanupWarningEntry, RebalanceCleanupWarnings, RebalanceInfo,
RebalanceMeta, RebalanceStats, RebalanceStopPropagationRecord, decode_rebalance_stop_propagation_record,
encode_rebalance_stop_propagation_record,
};
}
mod ecstore_rpc {
pub(crate) use crate::storage::ecstore_rpc::PeerRestClient;
}
mod ecstore_storage {
pub(crate) use crate::storage::ecstore_storage::ECStore;
}
mod ecstore_tier {
pub(crate) use crate::storage::ecstore_tier::{tier, tier_admin, tier_config, tier_handlers};
}
pub(crate) const RUSTFS_META_BUCKET: &str = ecstore_disk::RUSTFS_META_BUCKET;
pub(crate) const STORAGE_CLASS_SUB_SYS: &str = ecstore_config::com::STORAGE_CLASS_SUB_SYS;
pub(crate) type AdminError = ecstore_client::admin_handler_utils::AdminError;
pub(crate) type CollectMetricsOpts = ecstore_metrics::CollectMetricsOpts;
pub(crate) type DiskStat = ecstore_rebalance::DiskStat;
pub(crate) type ECStore = ecstore_storage::ECStore;
pub(crate) type EndpointServerPools = ecstore_layout::EndpointServerPools;
pub(crate) type MetricType = ecstore_metrics::MetricType;
pub(crate) type NotificationSys = ecstore_notification::NotificationSys;
pub(crate) type PeerRestClient = ecstore_rpc::PeerRestClient;
pub(crate) type RebalSaveOpt = ecstore_rebalance::RebalSaveOpt;
pub(crate) type RebalanceCleanupWarnings = ecstore_rebalance::RebalanceCleanupWarnings;
pub(crate) type RebalanceMeta = ecstore_rebalance::RebalanceMeta;
pub(crate) type RebalanceStats = ecstore_rebalance::RebalanceStats;
pub(crate) type RebalanceStopPropagationRecord = ecstore_rebalance::RebalanceStopPropagationRecord;
pub(crate) type StorageError = ecstore_error::StorageError;
pub(crate) type Error = StorageError;
pub(crate) type Result<T> = core::result::Result<T, Error>;
pub(crate) type TierConfig = ecstore_tier::tier_config::TierConfig;
pub(crate) type TierCreds = ecstore_tier::tier_admin::TierCreds;
pub(crate) type TierType = ecstore_tier::tier_config::TierType;
#[cfg(test)]
pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint;
#[cfg(test)]
pub(crate) type Endpoints = ecstore_layout::Endpoints;
#[cfg(test)]
pub(crate) type PoolEndpoints = ecstore_layout::PoolEndpoints;
#[cfg(test)]
pub(crate) type RebalStatus = ecstore_rebalance::RebalStatus;
#[cfg(test)]
pub(crate) type RebalanceCleanupWarningEntry = ecstore_rebalance::RebalanceCleanupWarningEntry;
#[cfg(test)]
pub(crate) type RebalanceInfo = ecstore_rebalance::RebalanceInfo;
pub(crate) fn decode_rebalance_stop_propagation_record(message: &str) -> Option<RebalanceStopPropagationRecord> {
ecstore_rebalance::decode_rebalance_stop_propagation_record(message)
}
#[cfg(test)]
pub(crate) fn encode_rebalance_stop_propagation_record(record: &RebalanceStopPropagationRecord) -> String {
ecstore_rebalance::encode_rebalance_stop_propagation_record(record)
}
pub(crate) trait AdminReplicationConfigExt {
fn filter_target_arns(&self, obj: &replication::ObjectOpts) -> Vec<String>;
fn has_existing_object_replication(&self, arn: &str) -> (bool, bool);
}
impl AdminReplicationConfigExt for s3s::dto::ReplicationConfiguration {
fn filter_target_arns(&self, obj: &replication::ObjectOpts) -> Vec<String> {
<s3s::dto::ReplicationConfiguration as ecstore_bucket::replication::ReplicationConfigurationExt>::filter_target_arns(
self, obj,
)
}
fn has_existing_object_replication(&self, arn: &str) -> (bool, bool) {
<s3s::dto::ReplicationConfiguration as ecstore_bucket::replication::ReplicationConfigurationExt>::has_existing_object_replication(
self, arn,
)
}
}
pub(crate) trait AdminVersioningConfigExt {
fn enabled(&self) -> bool;
}
impl AdminVersioningConfigExt for s3s::dto::VersioningConfiguration {
fn enabled(&self) -> bool {
<s3s::dto::VersioningConfiguration as ecstore_bucket::versioning::VersioningApi>::enabled(self)
}
}
pub(crate) mod bandwidth {
pub(crate) mod monitor {
pub(crate) type BandwidthDetails = super::super::ecstore_bucket::bandwidth::monitor::BandwidthDetails;
}
}
pub(crate) mod bucket_target_sys {
pub(crate) type AdvancedPutOptions = super::ecstore_bucket::bucket_target_sys::AdvancedPutOptions;
pub(crate) type BucketTargetError = super::ecstore_bucket::bucket_target_sys::BucketTargetError;
pub(crate) type BucketTargetSys = super::ecstore_bucket::bucket_target_sys::BucketTargetSys;
pub(crate) type PutObjectOptions = super::ecstore_bucket::bucket_target_sys::PutObjectOptions;
pub(crate) type RemoveObjectOptions = super::ecstore_bucket::bucket_target_sys::RemoveObjectOptions;
pub(crate) type S3ClientError = super::ecstore_bucket::bucket_target_sys::S3ClientError;
pub(crate) type TargetClient = super::ecstore_bucket::bucket_target_sys::TargetClient;
}
pub(crate) mod lifecycle {
pub(crate) mod tier_last_day_stats {
#[cfg(test)]
pub(crate) type LastDayTierStats = super::super::ecstore_bucket::lifecycle::tier_last_day_stats::LastDayTierStats;
}
}
pub(crate) mod metadata {
pub(crate) const BUCKET_CORS_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_CORS_CONFIG;
pub(crate) const BUCKET_LIFECYCLE_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_LIFECYCLE_CONFIG;
pub(crate) const BUCKET_NOTIFICATION_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_NOTIFICATION_CONFIG;
pub(crate) const BUCKET_POLICY_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_POLICY_CONFIG;
pub(crate) const BUCKET_QUOTA_CONFIG_FILE: &str = super::ecstore_bucket::metadata::BUCKET_QUOTA_CONFIG_FILE;
pub(crate) const BUCKET_REPLICATION_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_REPLICATION_CONFIG;
pub(crate) const BUCKET_SSECONFIG: &str = super::ecstore_bucket::metadata::BUCKET_SSECONFIG;
pub(crate) const BUCKET_TAGGING_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_TAGGING_CONFIG;
pub(crate) const BUCKET_TARGETS_FILE: &str = super::ecstore_bucket::metadata::BUCKET_TARGETS_FILE;
pub(crate) const BUCKET_VERSIONING_CONFIG: &str = super::ecstore_bucket::metadata::BUCKET_VERSIONING_CONFIG;
pub(crate) const OBJECT_LOCK_CONFIG: &str = super::ecstore_bucket::metadata::OBJECT_LOCK_CONFIG;
pub(crate) type BucketMetadata = super::ecstore_bucket::metadata::BucketMetadata;
pub(crate) fn table_catalog_path_hash(value: &str) -> String {
super::ecstore_bucket::metadata::table_catalog_path_hash(value)
}
}
pub(crate) mod metadata_sys {
use std::sync::Arc;
use rustfs_policy::policy::BucketPolicy;
use s3s::dto::{
BucketLifecycleConfiguration, NotificationConfiguration, ObjectLockConfiguration, ServerSideEncryptionConfiguration,
Tagging, VersioningConfiguration,
};
use time::OffsetDateTime;
use super::Result;
use super::metadata::BucketMetadata;
use super::quota::BucketQuota;
use super::target::BucketTargets;
pub(crate) type BucketMetadataSys = super::ecstore_bucket::metadata_sys::BucketMetadataSys;
pub(crate) async fn get(bucket: &str) -> Result<Arc<BucketMetadata>> {
super::ecstore_bucket::metadata_sys::get(bucket).await
}
pub(crate) async fn update(bucket: &str, config_file: &str, data: Vec<u8>) -> Result<OffsetDateTime> {
super::ecstore_bucket::metadata_sys::update(bucket, config_file, data).await
}
pub(crate) async fn delete(bucket: &str, config_file: &str) -> Result<OffsetDateTime> {
super::ecstore_bucket::metadata_sys::delete(bucket, config_file).await
}
pub(crate) async fn get_bucket_policy(bucket: &str) -> Result<(BucketPolicy, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_bucket_policy(bucket).await
}
pub(crate) async fn get_bucket_targets_config(bucket: &str) -> Result<BucketTargets> {
super::ecstore_bucket::metadata_sys::get_bucket_targets_config(bucket).await
}
pub(crate) async fn get_config_from_disk(bucket: &str) -> Result<BucketMetadata> {
super::ecstore_bucket::metadata_sys::get_config_from_disk(bucket).await
}
pub(crate) async fn get_lifecycle_config(bucket: &str) -> Result<(BucketLifecycleConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_lifecycle_config(bucket).await
}
pub(crate) async fn get_notification_config(bucket: &str) -> Result<Option<NotificationConfiguration>> {
super::ecstore_bucket::metadata_sys::get_notification_config(bucket).await
}
pub(crate) async fn get_object_lock_config(bucket: &str) -> Result<(ObjectLockConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_object_lock_config(bucket).await
}
pub(crate) async fn get_quota_config(bucket: &str) -> Result<(BucketQuota, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_quota_config(bucket).await
}
pub(crate) async fn get_replication_config(bucket: &str) -> Result<(s3s::dto::ReplicationConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_replication_config(bucket).await
}
pub(crate) async fn get_sse_config(bucket: &str) -> Result<(ServerSideEncryptionConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_sse_config(bucket).await
}
pub(crate) async fn get_tagging_config(bucket: &str) -> Result<(Tagging, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_tagging_config(bucket).await
}
pub(crate) async fn get_versioning_config(bucket: &str) -> Result<(VersioningConfiguration, OffsetDateTime)> {
super::ecstore_bucket::metadata_sys::get_versioning_config(bucket).await
}
pub(crate) async fn list_bucket_targets(bucket: &str) -> Result<BucketTargets> {
super::ecstore_bucket::metadata_sys::list_bucket_targets(bucket).await
}
}
pub(crate) mod quota {
pub(crate) mod checker {
pub(crate) type QuotaChecker = super::super::ecstore_bucket::quota::checker::QuotaChecker;
}
pub(crate) type BucketQuota = super::ecstore_bucket::quota::BucketQuota;
pub(crate) type QuotaError = super::ecstore_bucket::quota::QuotaError;
pub(crate) type QuotaOperation = super::ecstore_bucket::quota::QuotaOperation;
}
pub(crate) mod replication {
pub(crate) type BucketReplicationResyncStatus = super::ecstore_bucket::replication::BucketReplicationResyncStatus;
pub(crate) type BucketStats = super::ecstore_bucket::replication::BucketStats;
pub(crate) type ObjectOpts = super::ecstore_bucket::replication::ObjectOpts;
pub(crate) type ResyncOpts = super::ecstore_bucket::replication::ResyncOpts;
#[cfg(test)]
pub(crate) type ResyncStatusType = super::ecstore_bucket::replication::ResyncStatusType;
#[cfg(test)]
pub(crate) type TargetReplicationResyncStatus = super::ecstore_bucket::replication::TargetReplicationResyncStatus;
}
pub(crate) mod target {
#[allow(clippy::upper_case_acronyms)]
pub(crate) type ARN = super::ecstore_bucket::target::ARN;
pub(crate) type BucketTarget = super::ecstore_bucket::target::BucketTarget;
pub(crate) type BucketTargetType = super::ecstore_bucket::target::BucketTargetType;
pub(crate) type BucketTargets = super::ecstore_bucket::target::BucketTargets;
pub(crate) type Credentials = super::ecstore_bucket::target::Credentials;
}
pub(crate) mod ecstore_utils {
pub(crate) fn deserialize<T>(input: &[u8]) -> s3s::xml::DeResult<T>
where
T: for<'xml> s3s::xml::Deserialize<'xml>,
{
super::ecstore_bucket::utils::deserialize(input)
}
pub(crate) fn is_valid_object_prefix(object: &str) -> bool {
super::ecstore_bucket::utils::is_valid_object_prefix(object)
}
pub(crate) fn serialize<T: s3s::xml::Serialize>(val: &T) -> s3s::xml::SerResult<Vec<u8>> {
super::ecstore_bucket::utils::serialize(val)
}
}
pub(crate) mod versioning {}
pub(crate) mod versioning_sys {
pub(crate) type BucketVersioningSys = super::ecstore_bucket::versioning_sys::BucketVersioningSys;
}
pub(crate) mod storageclass {
pub(crate) const INLINE_BLOCK_ENV: &str = super::ecstore_config::storageclass::INLINE_BLOCK_ENV;
pub(crate) const OPTIMIZE_ENV: &str = super::ecstore_config::storageclass::OPTIMIZE_ENV;
pub(crate) const RRS: &str = super::ecstore_config::storageclass::RRS;
pub(crate) const RRS_ENV: &str = super::ecstore_config::storageclass::RRS_ENV;
pub(crate) const STANDARD: &str = super::ecstore_config::storageclass::STANDARD;
pub(crate) const STANDARD_ENV: &str = super::ecstore_config::storageclass::STANDARD_ENV;
pub(crate) type Config = super::ecstore_config::storageclass::Config;
pub(crate) fn lookup_config(kvs: &rustfs_config::server_config::KVS, set_drive_count: usize) -> super::Result<Config> {
super::ecstore_config::storageclass::lookup_config(kvs, set_drive_count)
}
}
pub(crate) type DailyAllTierStats = ecstore_bucket::lifecycle::tier_last_day_stats::DailyAllTierStats;
pub(crate) fn is_reserved_or_invalid_bucket(bucket_entry: &str, strict: bool) -> bool {
ecstore_capacity::is_reserved_or_invalid_bucket(bucket_entry, strict)
}
pub(crate) async fn read_admin_config(api: Arc<ECStore>, file: &str) -> Result<Vec<u8>> {
ecstore_config::com::read_config(api, file).await
}
pub(crate) async fn read_admin_config_without_migrate(api: Arc<ECStore>) -> Result<rustfs_config::server_config::Config> {
ecstore_config::com::read_config_without_migrate(api).await
}
pub(crate) async fn save_admin_config(api: Arc<ECStore>, file: &str, data: Vec<u8>) -> Result<()> {
ecstore_config::com::save_config(api, file, data).await
}
pub(crate) async fn delete_admin_config(api: Arc<ECStore>, file: &str) -> Result<()> {
ecstore_config::com::delete_config(api, file).await
}
pub(crate) async fn save_admin_server_config(api: Arc<ECStore>, cfg: &rustfs_config::server_config::Config) -> Result<()> {
ecstore_config::com::save_server_config(api, cfg).await
}
pub(crate) fn init_admin_config_defaults() {
ecstore_config::init();
}
pub(crate) async fn collect_local_metrics(
types: MetricType,
opts: &CollectMetricsOpts,
) -> rustfs_madmin::metrics::RealtimeMetrics {
ecstore_metrics::collect_local_metrics(types, opts).await
}
pub(crate) struct AdminErrorRef(fn() -> &'static AdminError);
impl Deref for AdminErrorRef {
type Target = AdminError;
fn deref(&self) -> &Self::Target {
(self.0)()
}
}
pub(crate) static ERR_TIER_BACKEND_IN_USE: AdminErrorRef = AdminErrorRef(|| &ecstore_tier::tier::ERR_TIER_BACKEND_IN_USE);
pub(crate) static ERR_TIER_BACKEND_NOT_EMPTY: AdminErrorRef = AdminErrorRef(|| &ecstore_tier::tier::ERR_TIER_BACKEND_NOT_EMPTY);
pub(crate) static ERR_TIER_MISSING_CREDENTIALS: AdminErrorRef =
AdminErrorRef(|| &ecstore_tier::tier::ERR_TIER_MISSING_CREDENTIALS);
pub(crate) static ERR_TIER_ALREADY_EXISTS: AdminErrorRef =
AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_ALREADY_EXISTS);
pub(crate) static ERR_TIER_CONNECT_ERR: AdminErrorRef = AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_CONNECT_ERR);
pub(crate) static ERR_TIER_INVALID_CREDENTIALS: AdminErrorRef =
AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_INVALID_CREDENTIALS);
pub(crate) static ERR_TIER_NAME_NOT_UPPERCASE: AdminErrorRef =
AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_NAME_NOT_UPPERCASE);
pub(crate) static ERR_TIER_NOT_FOUND: AdminErrorRef = AdminErrorRef(|| &ecstore_tier::tier_handlers::ERR_TIER_NOT_FOUND);
pub(crate) use crate::storage::StorageObjectOptions;
pub(crate) use crate::storage::access::{ReqInfo, authorize_request};
pub(crate) use crate::storage::request_context::{RequestContext, spawn_traced};
scripts/check_architecture_migration_rules.sh
+19
View File
@@ -165,6 +165,7 @@ RUSTFS_APP_WILDCARD_IMPORT_HITS_FILE="${TMP_DIR}/rustfs_app_wildcard_import_hits
RUSTFS_APP_USECASE_S3_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_usecase_s3_api_bypass_hits.txt"
RUSTFS_APP_USECASE_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_usecase_storage_api_bypass_hits.txt"
RUSTFS_APP_ADMIN_STORAGE_API_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_app_admin_storage_api_bypass_hits.txt"
RUSTFS_ADMIN_STORAGE_API_ROOT_FACADE_HITS_FILE="${TMP_DIR}/rustfs_admin_storage_api_root_facade_hits.txt"
RUSTFS_STORAGE_DIRECT_APP_CONTEXT_BYPASS_HITS_FILE="${TMP_DIR}/rustfs_storage_direct_app_context_bypass_hits.txt"
awk '
@@ -1500,6 +1501,24 @@ if [[ -s "$RUSTFS_APP_RUNTIME_STORAGE_API_BYPASS_HITS_FILE" ]]; then
report_failure "RustFS app/admin runtime and data-usage facades must stay behind local storage_api boundaries: $(paste -sd '; ' "$RUSTFS_APP_RUNTIME_STORAGE_API_BYPASS_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
{
rg -n --with-filename \
'\b(?:ecstore_(?:bucket|capacity|client|cluster|config|disk|error|layout|metrics|notification|rebalance|rpc|storage|tier)|AdminReplicationConfigExt|AdminVersioningConfigExt|RUSTFS_META_BUCKET|STORAGE_CLASS_SUB_SYS|read_admin_config|read_admin_config_without_migrate|save_admin_config|save_admin_server_config|delete_admin_config|init_admin_config_defaults|StorageError|ECStore|Endpoint|Endpoints|EndpointServerPools|PoolEndpoints|PeerRestClient|RebalanceStats|RebalanceMeta|RebalanceStopPropagationRecord)\b' \
rustfs/src/admin/mod.rs || true
rg -n --with-filename \
'(?:crate::admin|super(?::super)+)::(?:bandwidth|bucket_target_sys|lifecycle|metadata|metadata_sys|quota|replication|target|versioning|versioning_sys|storageclass|tier|AdminReplicationConfigExt|AdminVersioningConfigExt|RUSTFS_META_BUCKET|STORAGE_CLASS_SUB_SYS|read_admin_config|read_admin_config_without_migrate|save_admin_config|save_admin_server_config|delete_admin_config|init_admin_config_defaults|StorageError|Error|ECStore|Endpoint|Endpoints|EndpointServerPools|PoolEndpoints|PeerRestClient|RebalanceStats|RebalanceMeta|RebalanceStopPropagationRecord|collect_local_metrics|is_reserved_or_invalid_bucket)\b' \
rustfs/src/admin \
-g '*.rs' \
-g '!storage_api.rs' || true
}
) >"$RUSTFS_ADMIN_STORAGE_API_ROOT_FACADE_HITS_FILE"
if [[ -s "$RUSTFS_ADMIN_STORAGE_API_ROOT_FACADE_HITS_FILE" ]]; then
report_failure "RustFS admin storage facades must stay behind rustfs/src/admin/storage_api.rs: $(paste -sd '; ' "$RUSTFS_ADMIN_STORAGE_API_ROOT_FACADE_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename 'crate::app::context::|use crate::app::context|app::context::' rustfs/src/storage --glob '*.rs' |