refactor: remove secondary compatibility bridges (#3737)

* refactor: remove app admin secondary compat bridges

* refactor: remove storage core compat bridge

* refactor: remove nested compat bridges

* refactor: remove admin handlers compat bridge

* refactor: remove runtime local compat bridges
This commit is contained in:
安正超
2026-06-22 16:29:33 +08:00
committed by GitHub
parent 599e3632d1
commit a66350b645
70 changed files with 462 additions and 860 deletions
+130 -12
View File
@@ -5,17 +5,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-fuzz-compat-relative-consumers`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115`.
- Based on: API-115 slice.
- Branch: `overtrue/arch-runtime-local-compat-bridges`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082/API-083/API-084/API-085/API-086/API-087/API-088/API-089/API-090/API-091/API-092/API-093/API-094/API-095/API-096/API-097/API-098/API-099/API-100/API-101/API-102/API-103/API-104/API-105/API-106/API-107/API-108/API-109/API-110/API-111/API-112/API-113/API-114/API-115/API-116`.
- Based on: API-116 slice.
- PR type for this branch: `pure-move`
- Runtime behavior changes: none.
- Rust code changes: collapse fuzz-target compatibility consumer paths into
relative owner paths.
- CI/script changes: extend the standalone crate local consumer guard to fuzz
targets.
- Docs changes: record the API-116 fuzz-target local compatibility consumer
cleanup.
- Rust code changes: remove app/admin, storage core, nested, handler, capacity,
server, and S3 API secondary compatibility bridges.
- CI/script changes: guard against reintroducing the removed secondary bridge
modules or their consumer paths.
- Docs changes: record the API-117 through API-121 compatibility bridge cleanup.
## Phase 0 Tasks
@@ -647,6 +646,69 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
- Verification: fuzz package compile coverage, fuzz-target local
compatibility consumer residual scan, migration and layer guards,
formatting, diff hygiene, Rust risk scan, and three-expert review.
- [x] `API-117` Remove app/admin secondary compatibility bridges.
- Completed slice: replace app use-case and admin router consumers of
`usecase_storage_compat` and `router_storage_compat` with direct owner
`storage_compat` paths, then delete the secondary bridge modules.
- Acceptance: app use-cases, app tests, and the admin router no longer route
through a second local compatibility bridge; migration rules reject
reintroduced bridge names.
- Must preserve: app object/bucket/multipart use-case behavior, lifecycle
transition test setup, admin route replication/bucket-target contracts, and
existing owner `storage_compat` aliases.
- Verification: RustFS compile coverage, app/admin secondary bridge residual
scan, migration and layer guards, formatting, diff hygiene, Rust risk scan,
and three-expert review.
- [x] `API-118` Remove storage core secondary compatibility bridge.
- Completed slice: replace storage owner consumers of `core_storage_compat`
with direct `storage_compat` paths, then delete the secondary bridge module.
- Acceptance: storage owner modules and tests no longer route through a
second local compatibility bridge; migration rules reject reintroduced
`core_storage_compat` references.
- Must preserve: ECFS object operations, storage access checks, SSE
encryption helpers, storage option resolution, and existing owner
`storage_compat` aliases.
- Verification: RustFS compile coverage, storage secondary bridge residual
scan, migration and layer guards, formatting, diff hygiene, path-only risk
review, and three-expert review.
- [x] `API-119` Remove nested secondary compatibility bridges.
- Completed slice: replace admin service, app context, and storage RPC
consumers of nested `storage_compat` bridge modules with direct owner
`storage_compat` paths, then delete the nested bridge modules.
- Acceptance: nested service, context, and RPC modules no longer route
through a second local compatibility bridge; migration rules reject
reintroduced bridge files or module declarations.
- Must preserve: admin dynamic config and site-replication behavior, app
context handle wiring, storage RPC signature and disk lookup behavior, and
existing owner `storage_compat` aliases.
- Verification: RustFS compile coverage, nested secondary bridge residual
scan, migration and layer guards, formatting, diff hygiene, path-only risk
review, and three-expert review.
- [x] `API-120` Remove admin handlers secondary compatibility bridge.
- Completed slice: replace admin handler consumers of
`handlers::storage_compat` with direct admin owner `storage_compat` paths,
then delete the handler bridge module.
- Acceptance: admin handler modules no longer route through a second local
compatibility bridge; migration rules reject the bridge file, module
declaration, or direct handler-level `super::storage_compat` consumers.
- Must preserve: admin handler config, replication, rebalance, quota, tier,
table catalog, metrics, trace, and heal behavior plus existing admin owner
`storage_compat` aliases.
- Verification: RustFS admin handler compile coverage, handler secondary
bridge residual scan, migration and layer guards, formatting, diff hygiene,
path-only risk review, and three-expert review.
- [x] `API-121` Remove runtime local compatibility bridges.
- Completed slice: replace capacity, server, and S3 API local compatibility
bridge consumers with direct owner APIs, then delete the bridge modules.
- Acceptance: capacity, server, and S3 API modules no longer route through
local `storage_compat` bridges; migration rules reject bridge files, module
declarations, or bridge consumers.
- Must preserve: capacity disk discovery, HTTP RPC signature verification,
event dispatch hook wiring, module-switch persistence, readiness quorum
checks, and S3 ETag conversion behavior.
- Verification: RustFS compile coverage, runtime local bridge residual scan,
migration and layer guards, formatting, diff hygiene, path-only risk
review, and three-expert review.
- [x] `G-012` Inventory placement and repair invariants.
- Acceptance:
[`placement-repair-invariants.md`](placement-repair-invariants.md) records
@@ -3680,14 +3742,70 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | pass | API-116 keeps selected fuzz-target local compatibility consumers owner-relative by replacing crate-qualified compatibility paths with scoped relative paths. |
| Migration preservation | pass | The extended guard rejects crate-qualified fuzz-target local compatibility consumer paths while preserving the same facade names and aliases. |
| Testing/verification | pass | Fuzz package compile, fuzz-target local compatibility consumer residual scan, migration guard, layer guard, formatting, diff hygiene, and risk scan passed. |
| Quality/architecture | pass | API-121 removes capacity, server, and S3 API local bridge modules while keeping direct owner API calls explicit. |
| Migration preservation | pass | The extended guard rejects deleted runtime bridge files, module declarations, and local bridge consumers. |
| Testing/verification | pass | RustFS compile, runtime local bridge residual scan, migration guard, layer guard, formatting, diff hygiene, pre-commit, and path-only risk review passed. |
## Verification Notes
Passed before push:
- Issue #660 API-121 current slice:
- `cargo check -p rustfs --tests`: passed.
- `cargo fmt --all`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- Runtime local compatibility bridge residual scan: passed.
- Rust risk review on path-only replacements and guard script: passed.
- `make pre-commit`: passed.
- Issue #660 API-120 current slice:
- `cargo check -p rustfs --tests`: passed.
- `cargo fmt --all`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- Admin handlers secondary compatibility bridge residual scan: passed.
- Rust risk review on path-only replacements and guard script: passed.
- Issue #660 API-119 current slice:
- `cargo check -p rustfs --tests`: passed.
- `cargo fmt --all`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- Nested secondary compatibility bridge residual scan: passed.
- Rust risk review on path-only replacements and guard script: passed.
- Issue #660 API-118 current slice:
- `cargo check -p rustfs --tests`: passed.
- `cargo fmt --all`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- Storage secondary compatibility bridge residual scan: passed.
- Rust risk review on path-only replacements and guard script: passed.
- Issue #660 API-117 current slice:
- `cargo check -p rustfs --tests`: passed.
- `cargo fmt --all`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_layer_dependencies.sh`: passed.
- App/admin secondary compatibility bridge residual scan: passed.
- Rust risk review on path-only replacements and guard script: passed.
- Issue #660 API-116 current slice:
- `cargo check --manifest-path fuzz/Cargo.toml --bins`: passed.
- `cargo fmt --all`: passed.
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::versioning_sys::BucketVersioningSys;
use super::super::storage_compat::versioning_sys::BucketVersioningSys;
use crate::admin::auth::authenticate_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::app::context::resolve_object_store_handle;
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::{read_admin_config_without_migrate, save_admin_server_config};
use super::super::storage_compat::{read_admin_config_without_migrate, save_admin_server_config};
use crate::admin::handlers::target_descriptor::AdminTargetSpec;
use crate::app::context::resolve_object_store_handle;
use rustfs_audit::{audit_system, start_audit_system as start_global_audit_system, system::AuditSystemState};
+2 -2
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::utils::{deserialize, serialize};
use super::storage_compat::{
use super::super::storage_compat::utils::{deserialize, serialize};
use super::super::storage_compat::{
StorageError,
metadata::{
BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE,
+12 -12
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV};
use super::storage_compat::{
use super::super::storage_compat::storageclass::{INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS_ENV, STANDARD_ENV};
use super::super::storage_compat::{
RUSTFS_META_BUCKET, STORAGE_CLASS_SUB_SYS, delete_admin_config, read_admin_config, read_admin_config_without_migrate,
save_admin_config, save_admin_server_config,
};
@@ -710,7 +710,7 @@ fn success_response(config_applied: bool) -> S3Result<S3Response<(StatusCode, Bo
Ok(S3Response::with_headers((StatusCode::OK, Body::default()), headers))
}
fn object_store() -> S3Result<std::sync::Arc<super::storage_compat::ECStore>> {
fn object_store() -> S3Result<std::sync::Arc<super::super::storage_compat::ECStore>> {
resolve_object_store_handle().ok_or_else(|| s3_error!(InternalError, "server storage not initialized"))
}
@@ -755,7 +755,7 @@ fn config_update_sub_system(directives: &[ConfigDirective]) -> S3Result<Option<&
fn validate_config_directives(directives: &[ConfigDirective]) -> S3Result<()> {
if DEFAULT_KVS.get().is_none() {
super::storage_compat::init_admin_config_defaults();
super::super::storage_compat::init_admin_config_defaults();
}
let Some(defaults) = DEFAULT_KVS.get() else {
return Err(s3_error!(InternalError, "config defaults are not initialized"));
@@ -1409,7 +1409,7 @@ fn env_help_key(sub_system: &str, key: &str) -> String {
fn default_help_postfix(sub_system: &str, key: &str) -> String {
if DEFAULT_KVS.get().is_none() {
super::storage_compat::init_admin_config_defaults();
super::super::storage_compat::init_admin_config_defaults();
}
DEFAULT_KVS
@@ -1896,7 +1896,7 @@ mod tests {
#[test]
fn full_config_export_can_be_reapplied() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
let mut original = ServerConfig::new();
apply_set_directives(
&mut original,
@@ -1943,7 +1943,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn build_help_response_appends_default_value_postfix() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
let response = build_help_response(Some("identity_openid"), Some("scopes"), false).expect("help response");
assert_eq!(response.keys_help.len(), 2);
@@ -2056,7 +2056,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_includes_env_override_lines() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
temp_env::with_vars(
[
("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example")),
@@ -2092,7 +2092,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_lists_env_only_targets() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || {
let config = ServerConfig::new();
let rendered = String::from_utf8(
@@ -2115,7 +2115,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_supports_specific_env_only_target_queries() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY", Some("http://env.example"))], || {
let config = ServerConfig::new();
let rendered = String::from_utf8(
@@ -2138,7 +2138,7 @@ identity_openid config_url="https://issuer.example" client_id="console""#,
#[test]
fn render_selected_config_orders_default_before_named_targets() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
temp_env::with_vars([("RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_ALPHA", Some("http://alpha.example"))], || {
let mut config = ServerConfig::new();
apply_set_directives(
@@ -2319,7 +2319,7 @@ identity_openid client_id="existing-client""#,
#[test]
fn storage_class_get_target_none_matches_full_export() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
let mut config = ServerConfig::new();
apply_set_directives(
&mut config,
+5 -5
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::is_reserved_or_invalid_bucket;
use super::storage_compat::utils::is_valid_object_prefix;
use super::super::storage_compat::is_reserved_or_invalid_bucket;
use super::super::storage_compat::utils::is_valid_object_prefix;
use crate::admin::auth::{authenticate_request, validate_admin_request};
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::app::context::resolve_object_store_handle;
@@ -378,10 +378,10 @@ fn should_handle_root_heal_directly(hip: &HealInitParams) -> bool {
&& hip.hs.set.is_none()
}
fn map_root_heal_status(heal_err: Option<super::storage_compat::Error>) -> S3Result<()> {
fn map_root_heal_status(heal_err: Option<super::super::storage_compat::Error>) -> S3Result<()> {
match heal_err {
None => Ok(()),
Some(super::storage_compat::StorageError::NoHealRequired) => {
Some(super::super::storage_compat::StorageError::NoHealRequired) => {
info!(
event = EVENT_ADMIN_RESPONSE_EMITTED,
component = LOG_COMPONENT_ADMIN_API,
@@ -730,7 +730,7 @@ impl Operation for BackgroundHealStatusHandler {
#[cfg(test)]
mod tests {
use super::super::storage_compat::StorageError;
use super::super::super::storage_compat::StorageError;
use super::extract_heal_init_params;
use super::{
HealInitParams, HealResp, build_heal_channel_request, encode_background_heal_status, encode_heal_start_success,
+1 -1
View File
@@ -14,7 +14,7 @@
//! KMS dynamic configuration admin API handlers
use super::storage_compat::{read_admin_config, save_admin_config};
use super::super::storage_compat::{read_admin_config, save_admin_config};
use crate::admin::auth::validate_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::app::context::{resolve_kms_runtime_service_manager, resolve_object_store_handle};
+1 -1
View File
@@ -18,7 +18,7 @@
//! keeping the response format explicitly NDJSON. It is not a Prometheus text
//! exposition endpoint.
use super::storage_compat::{CollectMetricsOpts, MetricType, collect_local_metrics};
use super::super::storage_compat::{CollectMetricsOpts, MetricType, collect_local_metrics};
use crate::admin::auth::validate_admin_request;
use crate::admin::router::Operation;
use crate::auth::{check_key_valid, get_session_token};
-1
View File
@@ -45,7 +45,6 @@ pub mod replication;
pub mod scanner;
pub mod service_account;
pub mod site_replication;
pub(crate) mod storage_compat;
pub mod sts;
pub mod system;
pub mod table_catalog;
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::get_global_region;
use super::super::storage_compat::get_global_region;
use crate::admin::router::{ADMIN_OBJECT_ZIP_DOWNLOADS_PATH, AdminOperation, Operation, S3Router};
use crate::app::context::resolve_object_store_handle;
use crate::auth::{check_key_valid, get_session_token};
@@ -649,7 +649,7 @@ async fn preflight_zip_items(request: &CreateObjectZipDownloadRequest, items: &[
Ok(())
}
fn storage_error_to_s3(err: super::storage_compat::Error) -> s3s::S3Error {
fn storage_error_to_s3(err: super::super::storage_compat::Error) -> s3s::S3Error {
ApiError::from(err).into()
}
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::{read_admin_config_without_migrate, save_admin_server_config};
use super::super::storage_compat::{read_admin_config_without_migrate, save_admin_server_config};
use super::sts::create_oidc_sts_credentials;
use crate::admin::auth::validate_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
+1 -1
View File
@@ -194,7 +194,7 @@ macro_rules! log_pool_response_emitted {
};
}
fn endpoints_from_context() -> Option<super::storage_compat::EndpointServerPools> {
fn endpoints_from_context() -> Option<super::super::storage_compat::EndpointServerPools> {
resolve_endpoints_handle()
}
+4 -4
View File
@@ -14,9 +14,9 @@
//! Quota admin handlers for HTTP API
use super::storage_compat::metadata_sys::BucketMetadataSys;
use super::storage_compat::quota::checker::QuotaChecker;
use super::storage_compat::quota::{BucketQuota, QuotaError, QuotaOperation};
use super::super::storage_compat::metadata_sys::BucketMetadataSys;
use super::super::storage_compat::quota::checker::QuotaChecker;
use super::super::storage_compat::quota::{BucketQuota, QuotaError, QuotaOperation};
use crate::admin::auth::{validate_admin_request, validate_admin_request_with_bucket};
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::app::context::{resolve_bucket_metadata_handle, resolve_object_store_handle};
@@ -175,7 +175,7 @@ async fn current_usage_from_context(bucket: &str) -> u64 {
return 0;
};
match super::storage_compat::load_data_usage_from_backend(store).await {
match super::super::storage_compat::load_data_usage_from_backend(store).await {
Ok(data_usage_info) => data_usage_info
.buckets_usage
.get(bucket)
+4 -4
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::{
use super::super::storage_compat::{
DiskStat, ECStore, NotificationSys, RebalSaveOpt, RebalanceCleanupWarnings, RebalanceMeta, RebalanceStopPropagationRecord,
StorageError, decode_rebalance_stop_propagation_record, get_global_notification_sys,
};
@@ -296,7 +296,7 @@ fn build_rebalance_pool_progress(
now: OffsetDateTime,
stop_time: Option<OffsetDateTime>,
percent_free_goal: f64,
ps: &super::storage_compat::RebalanceStats,
ps: &super::super::storage_compat::RebalanceStats,
) -> Option<RebalPoolProgress> {
let total_bytes_to_rebal = ps.init_capacity as f64 * percent_free_goal - ps.init_free_space as f64;
let terminal_time = ps.info.end_time.or(stop_time);
@@ -339,7 +339,7 @@ fn build_rebalance_pool_statuses(
now: OffsetDateTime,
stop_time: Option<OffsetDateTime>,
percent_free_goal: f64,
pool_stats: &[super::storage_compat::RebalanceStats],
pool_stats: &[super::super::storage_compat::RebalanceStats],
disk_stats: &[DiskStat],
) -> Vec<RebalancePoolStatus> {
pool_stats
@@ -895,7 +895,7 @@ mod offsetdatetime_rfc3339 {
#[cfg(test)]
mod rebalance_handler_tests {
use super::super::storage_compat::{
use super::super::super::storage_compat::{
DiskStat, RebalStatus, RebalanceCleanupWarningEntry, RebalanceCleanupWarnings, RebalanceInfo, RebalanceMeta,
RebalanceStats, RebalanceStopPropagationRecord, encode_rebalance_stop_propagation_record,
};
+9 -9
View File
@@ -12,15 +12,15 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::StorageError;
use super::storage_compat::bucket_target_sys::{BucketTargetError, BucketTargetSys};
use super::storage_compat::global_rustfs_port;
use super::storage_compat::metadata::BUCKET_TARGETS_FILE;
use super::storage_compat::metadata_sys;
use super::storage_compat::metadata_sys::get_replication_config;
use super::storage_compat::replication::BucketStats;
use super::storage_compat::replication::GLOBAL_REPLICATION_STATS;
use super::storage_compat::target::BucketTarget;
use super::super::storage_compat::StorageError;
use super::super::storage_compat::bucket_target_sys::{BucketTargetError, BucketTargetSys};
use super::super::storage_compat::global_rustfs_port;
use super::super::storage_compat::metadata::BUCKET_TARGETS_FILE;
use super::super::storage_compat::metadata_sys;
use super::super::storage_compat::metadata_sys::get_replication_config;
use super::super::storage_compat::replication::BucketStats;
use super::super::storage_compat::replication::GLOBAL_REPLICATION_STATS;
use super::super::storage_compat::target::BucketTarget;
use crate::admin::auth::validate_admin_request;
use crate::admin::handlers::site_replication::site_replication_peer_deployment_id_for_endpoint;
use crate::admin::router::{AdminOperation, Operation, S3Router};
+15 -15
View File
@@ -12,20 +12,20 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::Error as StorageError;
use super::storage_compat::bucket_target_sys::BucketTargetSys;
use super::storage_compat::metadata::{
use super::super::storage_compat::Error as StorageError;
use super::super::storage_compat::bucket_target_sys::BucketTargetSys;
use super::super::storage_compat::metadata::{
BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, BUCKET_REPLICATION_CONFIG,
BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, OBJECT_LOCK_CONFIG,
};
use super::storage_compat::metadata_sys;
use super::storage_compat::replication::GLOBAL_REPLICATION_STATS;
use super::storage_compat::replication::{ResyncOpts, get_global_replication_pool};
use super::storage_compat::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials};
use super::storage_compat::utils::{deserialize, serialize};
use super::storage_compat::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use super::storage_compat::{delete_admin_config, read_admin_config, save_admin_config};
use super::storage_compat::{get_global_deployment_id, get_global_endpoints_opt, get_global_region, global_rustfs_port};
use super::super::storage_compat::metadata_sys;
use super::super::storage_compat::replication::GLOBAL_REPLICATION_STATS;
use super::super::storage_compat::replication::{ResyncOpts, get_global_replication_pool};
use super::super::storage_compat::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials};
use super::super::storage_compat::utils::{deserialize, serialize};
use super::super::storage_compat::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use super::super::storage_compat::{delete_admin_config, read_admin_config, save_admin_config};
use super::super::storage_compat::{get_global_deployment_id, get_global_endpoints_opt, get_global_region, global_rustfs_port};
use crate::admin::auth::validate_admin_request;
use crate::admin::router::{AdminOperation, Operation, S3Router};
use crate::admin::site_replication_identity::{
@@ -652,7 +652,7 @@ async fn site_replication_peer_client() -> S3Result<reqwest::Client> {
built
}
fn runtime_tls_enabled_with(endpoints: Option<&super::storage_compat::EndpointServerPools>) -> bool {
fn runtime_tls_enabled_with(endpoints: Option<&super::super::storage_compat::EndpointServerPools>) -> bool {
if !rustfs_utils::get_env_str(ENV_RUSTFS_TLS_PATH, DEFAULT_RUSTFS_TLS_PATH).is_empty() {
return true;
}
@@ -3353,7 +3353,7 @@ fn is_stale_update(local_updated_at: OffsetDateTime, incoming_updated_at: Option
}
fn bucket_meta_local_updated_at(
bucket_meta: &super::storage_compat::metadata::BucketMetadata,
bucket_meta: &super::super::storage_compat::metadata::BucketMetadata,
config_file: &str,
) -> OffsetDateTime {
match config_file {
@@ -4573,8 +4573,8 @@ impl Operation for SRRotateServiceAccountHandler {
#[cfg(test)]
mod tests {
use super::super::storage_compat::Endpoint;
use super::super::storage_compat::{EndpointServerPools, Endpoints, PoolEndpoints};
use super::super::super::storage_compat::Endpoint;
use super::super::super::storage_compat::{EndpointServerPools, Endpoints, PoolEndpoints};
use super::*;
use http::{HeaderMap, HeaderValue, Uri};
use rustfs_common::{get_global_outbound_tls_generation, set_global_outbound_tls_generation};
@@ -1,96 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) use super::super::storage_compat::{
AdminError, AdminReplicationConfigExt, AdminVersioningConfigExt, CollectMetricsOpts, DailyAllTierStats, DiskStat, ECStore,
ERR_TIER_ALREADY_EXISTS, ERR_TIER_BACKEND_IN_USE, ERR_TIER_BACKEND_NOT_EMPTY, ERR_TIER_CONNECT_ERR,
ERR_TIER_INVALID_CREDENTIALS, ERR_TIER_MISSING_CREDENTIALS, ERR_TIER_NAME_NOT_UPPERCASE, ERR_TIER_NOT_FOUND,
EndpointServerPools, Error, MetricType, NotificationSys, PeerRestClient, RUSTFS_META_BUCKET, RebalSaveOpt,
RebalanceCleanupWarnings, RebalanceMeta, RebalanceStats, RebalanceStopPropagationRecord, STORAGE_CLASS_SUB_SYS, StorageError,
TierConfig, TierCreds, TierType, collect_local_metrics, decode_rebalance_stop_propagation_record, delete_admin_config,
get_global_deployment_id, get_global_endpoints_opt, get_global_notification_sys, get_global_region, global_rustfs_port,
init_admin_config_defaults, is_reserved_or_invalid_bucket, load_data_usage_from_backend, read_admin_config,
read_admin_config_without_migrate, save_admin_config, save_admin_server_config,
};
#[cfg(test)]
pub(crate) use super::super::storage_compat::{
Endpoint, Endpoints, PoolEndpoints, RebalStatus, RebalanceCleanupWarningEntry, RebalanceInfo,
encode_rebalance_stop_propagation_record,
};
pub(crate) mod bucket_target_sys {
pub(crate) use super::super::super::storage_compat::bucket_target_sys::{BucketTargetError, BucketTargetSys};
}
pub(crate) mod lifecycle {
pub(crate) mod bucket_lifecycle_ops {
pub(crate) use super::super::super::super::storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_TransitionState;
}
pub(crate) mod tier_last_day_stats {
#[cfg(test)]
pub(crate) use super::super::super::super::storage_compat::lifecycle::tier_last_day_stats::LastDayTierStats;
}
}
pub(crate) mod metadata {
pub(crate) use super::super::super::storage_compat::metadata::{
BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE,
BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG,
BucketMetadata, OBJECT_LOCK_CONFIG, table_catalog_path_hash,
};
}
pub(crate) mod metadata_sys {
pub(crate) use super::super::super::storage_compat::metadata_sys::{
BucketMetadataSys, delete, get, get_bucket_policy, get_bucket_targets_config, get_config_from_disk, get_lifecycle_config,
get_notification_config, get_object_lock_config, get_quota_config, get_replication_config, get_sse_config,
get_tagging_config, get_versioning_config, list_bucket_targets, update,
};
}
pub(crate) mod quota {
pub(crate) use super::super::super::storage_compat::quota::{BucketQuota, QuotaError, QuotaOperation};
pub(crate) mod checker {
pub(crate) use super::super::super::super::storage_compat::quota::checker::QuotaChecker;
}
}
pub(crate) mod replication {
pub(crate) use super::super::super::storage_compat::replication::{
BucketStats, GLOBAL_REPLICATION_STATS, ResyncOpts, get_global_replication_pool,
};
}
pub(crate) mod storageclass {
pub(crate) use super::super::super::storage_compat::storageclass::{
INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS, RRS_ENV, STANDARD, STANDARD_ENV,
};
}
pub(crate) mod target {
pub(crate) use super::super::super::storage_compat::target::{
ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials,
};
}
pub(crate) mod utils {
pub(crate) use super::super::super::storage_compat::utils::{deserialize, is_valid_object_prefix, serialize};
}
pub(crate) mod versioning_sys {
pub(crate) use super::super::super::storage_compat::versioning_sys::BucketVersioningSys;
}
+1 -1
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::storage_compat::utils::serialize;
use super::is_admin::IsAdminHandler;
use super::storage_compat::utils::serialize;
use crate::{
admin::{
handlers::site_replication::site_replication_iam_change_hook,
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::{ECStore, metadata::table_catalog_path_hash, metadata_sys};
use super::super::storage_compat::{ECStore, metadata::table_catalog_path_hash, metadata_sys};
use crate::admin::{
auth::{AdminResourceScope, validate_admin_request, validate_admin_request_with_bucket_object},
router::{AdminOperation, Operation, S3Router},
+3 -3
View File
@@ -13,8 +13,8 @@
// limitations under the License.
#![allow(unused_variables, unused_mut, unused_must_use)]
use super::storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_TransitionState;
use super::storage_compat::{
use super::super::storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_TransitionState;
use super::super::storage_compat::{
AdminError, DailyAllTierStats, ERR_TIER_ALREADY_EXISTS, ERR_TIER_BACKEND_IN_USE, ERR_TIER_BACKEND_NOT_EMPTY,
ERR_TIER_CONNECT_ERR, ERR_TIER_INVALID_CREDENTIALS, ERR_TIER_MISSING_CREDENTIALS, ERR_TIER_NAME_NOT_UPPERCASE,
ERR_TIER_NOT_FOUND, TierConfig, TierCreds, TierType, get_global_notification_sys, storageclass,
@@ -930,7 +930,7 @@ impl Operation for ClearTier {
#[cfg(test)]
mod tests {
use super::super::storage_compat::lifecycle::tier_last_day_stats::LastDayTierStats;
use super::super::super::storage_compat::lifecycle::tier_last_day_stats::LastDayTierStats;
use super::*;
use http::Uri;
use matchit::Router;
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::PeerRestClient;
use super::super::storage_compat::PeerRestClient;
use crate::admin::router::Operation;
use crate::app::context::resolve_endpoints_handle;
use http::StatusCode;
-1
View File
@@ -21,7 +21,6 @@ mod plugin_contract;
#[allow(dead_code)]
pub(crate) mod route_policy;
pub mod router;
pub(crate) mod router_storage_compat;
pub mod service;
pub mod site_replication_identity;
pub(crate) mod storage_compat;
+25 -25
View File
@@ -12,23 +12,23 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::router_storage_compat::GLOBAL_BOOT_TIME;
use super::router_storage_compat::PeerRestClient;
use super::router_storage_compat::bandwidth::monitor::BandwidthDetails;
use super::router_storage_compat::bucket_target_sys::{
use super::storage_compat::GLOBAL_BOOT_TIME;
use super::storage_compat::PeerRestClient;
use super::storage_compat::bandwidth::monitor::BandwidthDetails;
use super::storage_compat::bucket_target_sys::{
BucketTargetSys, PutObjectOptions, RemoveObjectOptions, S3ClientError, TargetClient,
};
use super::router_storage_compat::get_global_notification_sys;
use super::router_storage_compat::metadata::BUCKET_TARGETS_FILE;
use super::router_storage_compat::metadata_sys;
use super::router_storage_compat::read_admin_config_without_migrate;
use super::router_storage_compat::replication::{
use super::storage_compat::get_global_notification_sys;
use super::storage_compat::metadata::BUCKET_TARGETS_FILE;
use super::storage_compat::metadata_sys;
use super::storage_compat::read_admin_config_without_migrate;
use super::storage_compat::replication::{
BucketReplicationResyncStatus, BucketStats, GLOBAL_REPLICATION_STATS, ObjectOpts, ResyncOpts, get_global_replication_pool,
};
use super::router_storage_compat::target::{BucketTarget, BucketTargetType, BucketTargets};
use super::router_storage_compat::versioning_sys::BucketVersioningSys;
use super::router_storage_compat::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use super::router_storage_compat::{get_global_bucket_monitor, get_global_deployment_id, get_global_region};
use super::storage_compat::target::{BucketTarget, BucketTargetType, BucketTargets};
use super::storage_compat::versioning_sys::BucketVersioningSys;
use super::storage_compat::{AdminReplicationConfigExt as _, AdminVersioningConfigExt as _};
use super::storage_compat::{get_global_bucket_monitor, get_global_deployment_id, get_global_region};
use crate::admin::console::{is_console_path, make_console_server};
use crate::admin::handlers::oidc::is_oidc_path;
use crate::app::context::resolve_object_store_handle;
@@ -1419,7 +1419,7 @@ async fn ensure_replication_bucket_exists(bucket: &str) -> S3Result<()> {
async fn ensure_replication_config_exists(bucket: &str) -> S3Result<()> {
match metadata_sys::get_replication_config(bucket).await {
Ok(_) => Ok(()),
Err(super::router_storage_compat::StorageError::ConfigNotFound) => Err(s3_error!(ReplicationConfigurationNotFoundError)),
Err(super::storage_compat::StorageError::ConfigNotFound) => Err(s3_error!(ReplicationConfigurationNotFoundError)),
Err(err) => Err(ApiError::from(err).into()),
}
}
@@ -1944,7 +1944,7 @@ async fn resolve_replication_target_client(bucket: &str, target: &BucketTarget)
fn build_replication_probe_put_options(now: OffsetDateTime) -> PutObjectOptions {
PutObjectOptions {
internal: super::router_storage_compat::bucket_target_sys::AdvancedPutOptions {
internal: super::storage_compat::bucket_target_sys::AdvancedPutOptions {
source_version_id: Uuid::new_v4().to_string(),
replication_status: ReplicationStatusType::Replica,
source_mtime: now,
@@ -2059,7 +2059,7 @@ async fn source_bucket_requires_object_lock(bucket: &str) -> S3Result<bool> {
.object_lock_enabled
.as_ref()
.is_some_and(|state| state.as_str() == s3s::dto::ObjectLockEnabled::ENABLED)),
Err(super::router_storage_compat::StorageError::ConfigNotFound) => Ok(false),
Err(super::storage_compat::StorageError::ConfigNotFound) => Ok(false),
Err(err) => Err(ApiError::from(err).into()),
}
}
@@ -2773,7 +2773,7 @@ mod tests {
#[test]
fn apply_replication_reset_to_targets_updates_matching_target() {
let mut targets = BucketTargets {
targets: vec![super::super::router_storage_compat::target::BucketTarget {
targets: vec![super::super::storage_compat::target::BucketTarget {
arn: "arn:target".to_string(),
..Default::default()
}],
@@ -2795,10 +2795,10 @@ mod tests {
let mut status = BucketReplicationResyncStatus::new();
status.targets_map.insert(
"arn:z".to_string(),
super::super::router_storage_compat::replication::TargetReplicationResyncStatus {
super::super::storage_compat::replication::TargetReplicationResyncStatus {
resync_id: "rid-z".to_string(),
last_update: Some(datetime!(2025-01-03 00:00 UTC)),
resync_status: super::super::router_storage_compat::replication::ResyncStatusType::ResyncFailed,
resync_status: super::super::storage_compat::replication::ResyncStatusType::ResyncFailed,
failed_count: 2,
failed_size: 4,
bucket: "bucket-z".to_string(),
@@ -2808,10 +2808,10 @@ mod tests {
);
status.targets_map.insert(
"arn:a".to_string(),
super::super::router_storage_compat::replication::TargetReplicationResyncStatus {
super::super::storage_compat::replication::TargetReplicationResyncStatus {
resync_id: "rid-a".to_string(),
last_update: Some(datetime!(2025-01-02 00:00 UTC)),
resync_status: super::super::router_storage_compat::replication::ResyncStatusType::ResyncCompleted,
resync_status: super::super::storage_compat::replication::ResyncStatusType::ResyncCompleted,
replicated_count: 3,
replicated_size: 9,
bucket: "bucket-a".to_string(),
@@ -2841,10 +2841,10 @@ mod tests {
let mut status = BucketReplicationResyncStatus::new();
status.targets_map.insert(
"arn:z".to_string(),
super::super::router_storage_compat::replication::TargetReplicationResyncStatus {
super::super::storage_compat::replication::TargetReplicationResyncStatus {
resync_id: "rid-z".to_string(),
last_update: Some(datetime!(2025-02-03 00:00 UTC)),
resync_status: super::super::router_storage_compat::replication::ResyncStatusType::ResyncFailed,
resync_status: super::super::storage_compat::replication::ResyncStatusType::ResyncFailed,
failed_count: 2,
failed_size: 4,
bucket: "bucket-z".to_string(),
@@ -2854,10 +2854,10 @@ mod tests {
);
status.targets_map.insert(
"arn:a".to_string(),
super::super::router_storage_compat::replication::TargetReplicationResyncStatus {
super::super::storage_compat::replication::TargetReplicationResyncStatus {
resync_id: "rid-a".to_string(),
last_update: Some(datetime!(2025-02-02 00:00 UTC)),
resync_status: super::super::router_storage_compat::replication::ResyncStatusType::ResyncCompleted,
resync_status: super::super::storage_compat::replication::ResyncStatusType::ResyncCompleted,
replicated_count: 3,
replicated_size: 9,
bucket: "bucket-a".to_string(),
-54
View File
@@ -1,54 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) use super::storage_compat::{
AdminReplicationConfigExt, AdminVersioningConfigExt, GLOBAL_BOOT_TIME, PeerRestClient, StorageError,
get_global_bucket_monitor, get_global_deployment_id, get_global_notification_sys, get_global_region,
read_admin_config_without_migrate,
};
pub(crate) mod bandwidth {
pub(crate) mod monitor {
pub(crate) use super::super::super::storage_compat::bandwidth::monitor::BandwidthDetails;
}
}
pub(crate) mod bucket_target_sys {
pub(crate) use super::super::storage_compat::bucket_target_sys::{
AdvancedPutOptions, BucketTargetSys, PutObjectOptions, RemoveObjectOptions, S3ClientError, TargetClient,
};
}
pub(crate) mod metadata {
pub(crate) use super::super::storage_compat::metadata::BUCKET_TARGETS_FILE;
}
pub(crate) use super::storage_compat::metadata_sys;
pub(crate) mod replication {
pub(crate) use super::super::storage_compat::replication::{
BucketReplicationResyncStatus, BucketStats, GLOBAL_REPLICATION_STATS, ObjectOpts, ResyncOpts, get_global_replication_pool,
};
#[cfg(test)]
pub(crate) use super::super::storage_compat::replication::{ResyncStatusType, TargetReplicationResyncStatus};
}
pub(crate) mod target {
pub(crate) use super::super::storage_compat::target::{BucketTarget, BucketTargetType, BucketTargets};
}
pub(crate) mod versioning_sys {
pub(crate) use super::super::storage_compat::versioning_sys::BucketVersioningSys;
}
+9 -9
View File
@@ -12,10 +12,10 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::get_global_notification_sys;
use super::storage_compat::set_global_storage_class;
use super::storage_compat::storageclass;
use super::storage_compat::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate};
use super::super::storage_compat::get_global_notification_sys;
use super::super::storage_compat::set_global_storage_class;
use super::super::storage_compat::storageclass;
use super::super::storage_compat::{STORAGE_CLASS_SUB_SYS, read_admin_config_without_migrate};
use crate::app::context::resolve_object_store_handle;
use rustfs_audit::reload_audit_config;
use rustfs_config::audit::{AUDIT_MQTT_SUB_SYS, AUDIT_REDIS_DEFAULT_CHANNEL, AUDIT_WEBHOOK_SUB_SYS};
@@ -370,7 +370,7 @@ pub async fn signal_config_snapshot_reload() {
#[cfg(test)]
mod tests {
use super::super::storage_compat::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG};
use super::super::super::storage_compat::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG};
use super::*;
use rustfs_config::notify::NOTIFY_WEBHOOK_SUB_SYS;
use rustfs_config::oidc::{OIDC_CLIENT_ID, OIDC_CONFIG_URL, OIDC_SCOPES};
@@ -427,7 +427,7 @@ mod tests {
#[test]
fn validate_notify_subsystem_config_rejects_invalid_webhook_endpoint() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(NOTIFY_WEBHOOK_SUB_SYS).expect("notify webhook defaults");
let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target");
@@ -441,7 +441,7 @@ mod tests {
#[test]
fn validate_audit_subsystem_config_rejects_relative_queue_dir() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(AUDIT_MQTT_SUB_SYS).expect("audit mqtt defaults");
let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target");
@@ -456,7 +456,7 @@ mod tests {
#[test]
fn validate_identity_openid_config_rejects_missing_openid_scope() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults");
let kvs = targets.get_mut(DEFAULT_DELIMITER).expect("default target");
@@ -473,7 +473,7 @@ mod tests {
#[test]
fn validate_identity_openid_config_rejects_invalid_named_provider_id() {
super::super::storage_compat::init_admin_config_defaults();
super::super::super::storage_compat::init_admin_config_defaults();
let mut config = ServerConfig::new();
let targets = config.0.get_mut(IDENTITY_OPENID_SUB_SYS).expect("openid defaults");
let default_kvs = targets.get(DEFAULT_DELIMITER).cloned().expect("default target");
-1
View File
@@ -14,4 +14,3 @@
pub mod config;
pub mod site_replication;
pub(crate) mod storage_compat;
+2 -2
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::Error as StorageError;
use super::storage_compat::{read_admin_config, save_admin_config};
use super::super::storage_compat::Error as StorageError;
use super::super::storage_compat::{read_admin_config, save_admin_config};
use crate::admin::site_replication_identity::{deployment_id_for_endpoint, normalize_peer_map_by_identity_with};
use crate::app::context::resolve_object_store_handle;
use rustfs_madmin::PeerInfo;
@@ -1,30 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) use super::super::storage_compat::{
Error, STORAGE_CLASS_SUB_SYS, get_global_notification_sys, read_admin_config, read_admin_config_without_migrate,
save_admin_config, set_global_storage_class,
};
#[cfg(test)]
pub(crate) use super::super::storage_compat::init_admin_config_defaults;
#[cfg(test)]
pub(crate) mod metadata {
pub(crate) use super::super::super::storage_compat::metadata::{BUCKET_LIFECYCLE_CONFIG, BUCKET_REPLICATION_CONFIG};
}
pub(crate) mod storageclass {
pub(crate) use super::super::super::storage_compat::storageclass::lookup_config;
}
+6 -8
View File
@@ -14,13 +14,11 @@
//! Admin application use-case contracts.
use super::usecase_storage_compat::ECStore;
use super::usecase_storage_compat::EndpointServerPools;
use super::usecase_storage_compat::get_server_info;
use super::usecase_storage_compat::{
PoolDecommissionInfo, PoolStatus, get_total_usable_capacity, get_total_usable_capacity_free,
};
use super::usecase_storage_compat::{apply_bucket_usage_memory_overlay, load_data_usage_from_backend};
use super::storage_compat::ECStore;
use super::storage_compat::EndpointServerPools;
use super::storage_compat::get_server_info;
use super::storage_compat::{PoolDecommissionInfo, PoolStatus, get_total_usable_capacity, get_total_usable_capacity_free};
use super::storage_compat::{apply_bucket_usage_memory_overlay, load_data_usage_from_backend};
use crate::app::context::{AppContext, get_global_app_context, resolve_object_store_handle_for_context};
use crate::capacity::resolve_admin_used_capacity;
use crate::error::ApiError;
@@ -403,7 +401,7 @@ impl DefaultAdminUsecase {
#[cfg(test)]
mod tests {
use super::super::usecase_storage_compat::{PoolDecommissionInfo, PoolStatus};
use super::super::storage_compat::{PoolDecommissionInfo, PoolStatus};
use super::*;
use time::OffsetDateTime;
+7 -7
View File
@@ -14,12 +14,12 @@
//! Bucket application use-case contracts.
use super::usecase_storage_compat::ECStore;
use super::usecase_storage_compat::StorageError;
use super::usecase_storage_compat::get_global_notification_sys;
use super::usecase_storage_compat::object_api_utils::to_s3s_etag;
use super::usecase_storage_compat::{AppObjectLockConfigExt as _, AppVersioningConfigExt as _};
use super::usecase_storage_compat::{
use super::storage_compat::ECStore;
use super::storage_compat::StorageError;
use super::storage_compat::get_global_notification_sys;
use super::storage_compat::object_api_utils::to_s3s_etag;
use super::storage_compat::{AppObjectLockConfigExt as _, AppVersioningConfigExt as _};
use super::storage_compat::{
bucket_target_sys::BucketTargetSys,
lifecycle::bucket_lifecycle_ops::{
enqueue_expiry_for_existing_objects, enqueue_transition_for_existing_objects, validate_lifecycle_config,
@@ -2285,7 +2285,7 @@ mod tests {
BucketTargets {
targets: arns
.iter()
.map(|arn| super::super::usecase_storage_compat::target::BucketTarget {
.map(|arn| super::super::storage_compat::target::BucketTarget {
arn: (*arn).to_string(),
target_type: BucketTargetType::ReplicationService,
..Default::default()
+2 -4
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::usecase_storage_compat::{ECStore, Endpoint, EndpointServerPools, Endpoints, PoolEndpoints, metadata_sys};
use super::storage_compat::{ECStore, Endpoint, EndpointServerPools, Endpoints, PoolEndpoints, metadata_sys};
use rustfs_common::heal_channel::{HealOpts, HealScanMode};
use rustfs_object_capacity::capacity_manager::{HybridStrategyConfig, create_isolated_manager};
use rustfs_storage_api::{BucketOperations, BucketOptions, HealOperations as _, MakeBucketOptions, ObjectIO as _};
@@ -77,9 +77,7 @@ async fn setup_capacity_dirty_scope_env() -> (Vec<PathBuf>, Arc<ECStore>) {
};
let endpoint_pools = EndpointServerPools(vec![pool_endpoints]);
super::usecase_storage_compat::init_local_disks(endpoint_pools.clone())
.await
.unwrap();
super::storage_compat::init_local_disks(endpoint_pools.clone()).await.unwrap();
let server_addr: std::net::SocketAddr = "127.0.0.1:0".parse().unwrap();
let ecstore = ECStore::new(server_addr, endpoint_pools, CancellationToken::new())
-1
View File
@@ -21,7 +21,6 @@ mod global;
mod handles;
mod interfaces;
mod startup;
pub(crate) mod storage_compat;
pub use compat::*;
pub use global::*;
+9 -9
View File
@@ -12,16 +12,16 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::storage_compat::ECStore;
use super::super::storage_compat::EndpointServerPools;
use super::super::storage_compat::TierConfigMgr;
use super::super::storage_compat::metadata_sys::BucketMetadataSys;
use super::super::storage_compat::new_object_layer_fn;
use super::global::{AppContext, get_global_app_context};
use super::handles::{
default_bucket_metadata_interface, default_endpoints_interface, default_kms_runtime_interface,
default_server_config_interface, default_tier_config_interface,
};
use super::storage_compat::ECStore;
use super::storage_compat::EndpointServerPools;
use super::storage_compat::TierConfigMgr;
use super::storage_compat::metadata_sys::BucketMetadataSys;
use super::storage_compat::new_object_layer_fn;
#[cfg(test)]
use crate::config::RustFSBufferConfig;
use rustfs_config::server_config::Config;
@@ -119,10 +119,10 @@ fn resolve_buffer_config_with(
#[cfg(test)]
mod tests {
use super::super::storage_compat::Endpoint;
use super::super::storage_compat::init_local_disks;
use super::super::storage_compat::new_object_layer_fn;
use super::super::storage_compat::{Endpoints, PoolEndpoints};
use super::super::super::storage_compat::Endpoint;
use super::super::super::storage_compat::init_local_disks;
use super::super::super::storage_compat::new_object_layer_fn;
use super::super::super::storage_compat::{Endpoints, PoolEndpoints};
use super::*;
use crate::app::context::global::AppContextTestInterfaces;
use crate::app::context::handles::{default_notify_interface, default_region_interface};
+1 -1
View File
@@ -12,6 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::storage_compat::{ECStore, set_object_store_resolver};
use super::handles::{
IamHandle, KmsHandle, default_bucket_metadata_interface, default_buffer_config_interface, default_endpoints_interface,
default_kms_runtime_interface, default_notify_interface, default_region_interface, default_server_config_interface,
@@ -21,7 +22,6 @@ use super::interfaces::{
BucketMetadataInterface, BufferConfigInterface, EndpointsInterface, IamInterface, KmsInterface, KmsRuntimeInterface,
NotifyInterface, RegionInterface, ServerConfigInterface, TierConfigInterface,
};
use super::storage_compat::{ECStore, set_object_store_resolver};
use rustfs_iam::{store::object::ObjectStore, sys::IamSys};
use rustfs_kms::KmsServiceManager;
use std::sync::{Arc, OnceLock};
+4 -4
View File
@@ -12,14 +12,14 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::storage_compat::EndpointServerPools;
use super::super::storage_compat::TierConfigMgr;
use super::super::storage_compat::metadata_sys::{BucketMetadataSys, get_global_bucket_metadata_sys};
use super::super::storage_compat::{get_global_endpoints_opt, get_global_region, get_global_tier_config_mgr};
use super::interfaces::{
BucketMetadataInterface, BufferConfigInterface, EndpointsInterface, IamInterface, KmsInterface, KmsRuntimeInterface,
NotifyInterface, RegionInterface, ServerConfigInterface, TierConfigInterface,
};
use super::storage_compat::EndpointServerPools;
use super::storage_compat::TierConfigMgr;
use super::storage_compat::metadata_sys::{BucketMetadataSys, get_global_bucket_metadata_sys};
use super::storage_compat::{get_global_endpoints_opt, get_global_region, get_global_tier_config_mgr};
use crate::config::{RustFSBufferConfig, get_global_buffer_config};
use async_trait::async_trait;
use rustfs_config::server_config::Config;
+3 -3
View File
@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::EndpointServerPools;
use super::storage_compat::TierConfigMgr;
use super::storage_compat::metadata_sys::BucketMetadataSys;
use super::super::storage_compat::EndpointServerPools;
use super::super::storage_compat::TierConfigMgr;
use super::super::storage_compat::metadata_sys::BucketMetadataSys;
use crate::config::RustFSBufferConfig;
use async_trait::async_trait;
use rustfs_config::server_config::Config;
+1 -1
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::super::storage_compat::ECStore;
use super::global::{AppContext, get_global_app_context, init_global_app_context};
use super::storage_compat::ECStore;
use rustfs_kms::KmsServiceManager;
use std::io::{Error, Result};
use std::sync::Arc;
-25
View File
@@ -1,25 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) use super::super::storage_compat::{
ECStore, EndpointServerPools, TierConfigMgr, get_global_endpoints_opt, get_global_region, get_global_tier_config_mgr,
new_object_layer_fn, set_object_store_resolver,
};
pub(crate) mod metadata_sys {
pub(crate) use super::super::super::storage_compat::metadata_sys::{BucketMetadataSys, get_global_bucket_metadata_sys};
}
#[cfg(test)]
pub(crate) use super::super::storage_compat::{Endpoint, Endpoints, PoolEndpoints, init_local_disks};
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::usecase_storage_compat::{
use super::storage_compat::{
AppWarmBackend, ECStore, Endpoint, EndpointServerPools, Endpoints, GLOBAL_TierConfigMgr, PoolEndpoints, TierConfig, TierType,
WarmBackendGetOpts,
metadata::{BUCKET_LIFECYCLE_CONFIG, OBJECT_LOCK_CONFIG},
@@ -107,9 +107,7 @@ async fn setup_test_env() -> (Vec<PathBuf>, Arc<ECStore>) {
let endpoint_pools = EndpointServerPools(vec![pool_endpoints]);
super::usecase_storage_compat::init_local_disks(endpoint_pools.clone())
.await
.unwrap();
super::storage_compat::init_local_disks(endpoint_pools.clone()).await.unwrap();
let server_addr: std::net::SocketAddr = "127.0.0.1:9003".parse().unwrap();
let ecstore = ECStore::new(server_addr, endpoint_pools, CancellationToken::new())
@@ -126,7 +124,7 @@ async fn setup_test_env() -> (Vec<PathBuf>, Arc<ECStore>) {
let buckets = buckets_list.into_iter().map(|v| v.name).collect();
metadata_sys::init_bucket_metadata_sys(ecstore.clone(), buckets).await;
super::usecase_storage_compat::lifecycle::bucket_lifecycle_ops::init_background_expiry(ecstore.clone()).await;
super::storage_compat::lifecycle::bucket_lifecycle_ops::init_background_expiry(ecstore.clone()).await;
let _ = GLOBAL_ENV.set((disk_paths.clone(), ecstore.clone()));
@@ -926,7 +924,7 @@ async fn delete_transitioned_object_removes_remote_tier_copy_via_usecase() {
.expect("Failed to set lifecycle configuration");
let _ = upload_test_object(&ecstore, bucket.as_str(), object, payload).await;
super::usecase_storage_compat::lifecycle::bucket_lifecycle_ops::enqueue_transition_for_existing_objects(
super::storage_compat::lifecycle::bucket_lifecycle_ops::enqueue_transition_for_existing_objects(
ecstore.clone(),
bucket.as_str(),
)
@@ -986,7 +984,7 @@ async fn lifecycle_transition_marks_dirty_disks_for_capacity_manager() {
.expect("Failed to set lifecycle configuration");
let _ = upload_test_object(&ecstore, bucket.as_str(), object, payload).await;
super::usecase_storage_compat::lifecycle::bucket_lifecycle_ops::enqueue_transition_for_existing_objects(
super::storage_compat::lifecycle::bucket_lifecycle_ops::enqueue_transition_for_existing_objects(
ecstore.clone(),
bucket.as_str(),
)
-1
View File
@@ -22,7 +22,6 @@ pub mod multipart_usecase;
pub mod object_usecase;
mod select_object;
pub(crate) mod storage_compat;
pub(crate) mod usecase_storage_compat;
#[cfg(test)]
mod capacity_dirty_scope_test;
+23 -33
View File
@@ -14,16 +14,16 @@
//! Multipart application use-case contracts.
use super::usecase_storage_compat::ECStore;
use super::usecase_storage_compat::is_disk_compressible;
use super::usecase_storage_compat::is_valid_storage_class;
use super::usecase_storage_compat::object_api_utils::to_s3s_etag;
use super::usecase_storage_compat::quota::checker::QuotaChecker;
use super::storage_compat::ECStore;
use super::storage_compat::is_disk_compressible;
use super::storage_compat::is_valid_storage_class;
use super::storage_compat::object_api_utils::to_s3s_etag;
use super::storage_compat::quota::checker::QuotaChecker;
#[cfg(test)]
use super::usecase_storage_compat::{DecryptReader, EncryptReader, HardLimitReader, boxed_reader, wrap_reader};
use super::usecase_storage_compat::{HashReader, WritePlan};
use super::usecase_storage_compat::{StorageError, is_err_object_not_found, is_err_version_not_found};
use super::usecase_storage_compat::{
use super::storage_compat::{DecryptReader, EncryptReader, HardLimitReader, boxed_reader, wrap_reader};
use super::storage_compat::{HashReader, WritePlan};
use super::storage_compat::{StorageError, is_err_object_not_found, is_err_version_not_found};
use super::storage_compat::{
lifecycle::{bucket_lifecycle_audit::LcEventSrc, bucket_lifecycle_ops::enqueue_transition_immediate},
metadata_sys,
quota::QuotaOperation,
@@ -482,7 +482,7 @@ impl DefaultMultipartUsecase {
));
}
// Update quota tracking after successful multipart upload
super::usecase_storage_compat::record_bucket_object_write_memory(
super::storage_compat::record_bucket_object_write_memory(
&bucket,
previous_current_size,
obj_info.size.max(0) as u64,
@@ -675,7 +675,7 @@ impl DefaultMultipartUsecase {
rustfs_utils::http::insert_str(
&mut metadata,
rustfs_utils::http::SUFFIX_COMPRESSION,
super::usecase_storage_compat::compression_metadata_value(CompressionAlgorithm::default()),
super::storage_compat::compression_metadata_value(CompressionAlgorithm::default()),
);
}
@@ -894,13 +894,11 @@ impl DefaultMultipartUsecase {
.ok_or_else(|| ApiError::from(StorageError::other("Missing SSE-C session material")))?;
let ssec_write = match ssec_material.key_kind {
crate::storage::sse::EncryptionKeyKind::Object => {
super::usecase_storage_compat::WriteEncryption::multipart_object_key(ssec_material.key_bytes, part_id as u32)
super::storage_compat::WriteEncryption::multipart_object_key(ssec_material.key_bytes, part_id as u32)
}
crate::storage::sse::EncryptionKeyKind::Direct => {
super::storage_compat::WriteEncryption::multipart(ssec_material.key_bytes, ssec_material.base_nonce, part_id)
}
crate::storage::sse::EncryptionKeyKind::Direct => super::usecase_storage_compat::WriteEncryption::multipart(
ssec_material.key_bytes,
ssec_material.base_nonce,
part_id,
),
};
write_plan = write_plan.with_encryption(ssec_write);
(Some(ssec_material.server_side_encryption), ssec_material.kms_key_id)
@@ -916,12 +914,9 @@ impl DefaultMultipartUsecase {
.ok_or_else(|| ApiError::from(StorageError::other("Missing managed SSE session material")))?;
let managed_write = match managed_material.key_kind {
crate::storage::sse::EncryptionKeyKind::Object => {
super::usecase_storage_compat::WriteEncryption::multipart_object_key(
managed_material.key_bytes,
part_id as u32,
)
super::storage_compat::WriteEncryption::multipart_object_key(managed_material.key_bytes, part_id as u32)
}
crate::storage::sse::EncryptionKeyKind::Direct => super::usecase_storage_compat::WriteEncryption::multipart(
crate::storage::sse::EncryptionKeyKind::Direct => super::storage_compat::WriteEncryption::multipart(
managed_material.key_bytes,
managed_material.base_nonce,
part_id,
@@ -1240,13 +1235,11 @@ impl DefaultMultipartUsecase {
.ok_or_else(|| ApiError::from(StorageError::other("Missing SSE-C session material")))?;
let ssec_write = match ssec_material.key_kind {
crate::storage::sse::EncryptionKeyKind::Object => {
super::usecase_storage_compat::WriteEncryption::multipart_object_key(ssec_material.key_bytes, part_id as u32)
super::storage_compat::WriteEncryption::multipart_object_key(ssec_material.key_bytes, part_id as u32)
}
crate::storage::sse::EncryptionKeyKind::Direct => {
super::storage_compat::WriteEncryption::multipart(ssec_material.key_bytes, ssec_material.base_nonce, part_id)
}
crate::storage::sse::EncryptionKeyKind::Direct => super::usecase_storage_compat::WriteEncryption::multipart(
ssec_material.key_bytes,
ssec_material.base_nonce,
part_id,
),
};
write_plan = write_plan.with_encryption(ssec_write);
(
@@ -1266,12 +1259,9 @@ impl DefaultMultipartUsecase {
.ok_or_else(|| ApiError::from(StorageError::other("Missing managed SSE session material")))?;
let managed_write = match managed_material.key_kind {
crate::storage::sse::EncryptionKeyKind::Object => {
super::usecase_storage_compat::WriteEncryption::multipart_object_key(
managed_material.key_bytes,
part_id as u32,
)
super::storage_compat::WriteEncryption::multipart_object_key(managed_material.key_bytes, part_id as u32)
}
crate::storage::sse::EncryptionKeyKind::Direct => super::usecase_storage_compat::WriteEncryption::multipart(
crate::storage::sse::EncryptionKeyKind::Direct => super::storage_compat::WriteEncryption::multipart(
managed_material.key_bytes,
managed_material.base_nonce,
part_id,
+25 -36
View File
@@ -48,21 +48,21 @@ use metrics::{counter, histogram};
use pin_project_lite::pin_project;
use rustfs_object_capacity::capacity_manager::get_capacity_manager;
// Performance metrics recording (with zero-copy-metrics integration)
use super::usecase_storage_compat::ECStore;
use super::usecase_storage_compat::object_api_utils::to_s3s_etag;
use super::usecase_storage_compat::quota::checker::QuotaChecker;
use super::usecase_storage_compat::storageclass;
use super::usecase_storage_compat::{
use super::storage_compat::ECStore;
use super::storage_compat::object_api_utils::to_s3s_etag;
use super::storage_compat::quota::checker::QuotaChecker;
use super::storage_compat::storageclass;
use super::storage_compat::{
AppReplicationConfigExt as _, AppVersioningConfigExt as _, predict_lifecycle_expiration, validate_restore_request,
};
use super::usecase_storage_compat::{DiskError, is_all_buckets_not_found};
use super::usecase_storage_compat::{DynReader, HashReader, WritePlan, wrap_reader};
use super::usecase_storage_compat::{
use super::storage_compat::{DiskError, is_all_buckets_not_found};
use super::storage_compat::{DynReader, HashReader, WritePlan, wrap_reader};
use super::storage_compat::{
Error as EcstoreError, StorageError, is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found,
};
use super::usecase_storage_compat::{MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible};
use super::usecase_storage_compat::{get_lock_acquire_timeout, is_valid_storage_class};
use super::usecase_storage_compat::{
use super::storage_compat::{MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible};
use super::storage_compat::{get_lock_acquire_timeout, is_valid_storage_class};
use super::storage_compat::{
lifecycle::{
bucket_lifecycle_audit::LcEventSrc,
bucket_lifecycle_ops::{enqueue_transition_immediate, post_restore_opts},
@@ -279,12 +279,10 @@ async fn enqueue_transitioned_delete_cleanup(
let _activity_guard = DeleteTailActivityGuard::new(DeleteTailStage::Cleanup);
let je = if opts.delete_prefix {
super::usecase_storage_compat::lifecycle::tier_sweeper::transitioned_force_delete_journal_entry(
&existing.transitioned_object,
)
super::storage_compat::lifecycle::tier_sweeper::transitioned_force_delete_journal_entry(&existing.transitioned_object)
} else {
let version_id = opts.version_id.as_ref().and_then(|v| Uuid::parse_str(v).ok());
super::usecase_storage_compat::lifecycle::tier_sweeper::transitioned_delete_journal_entry(
super::storage_compat::lifecycle::tier_sweeper::transitioned_delete_journal_entry(
version_id,
opts.versioned,
opts.version_suspended,
@@ -295,9 +293,9 @@ async fn enqueue_transitioned_delete_cleanup(
return Ok(());
};
super::usecase_storage_compat::lifecycle::tier_delete_journal::persist_tier_delete_journal_entry(store, &je).await?;
super::storage_compat::lifecycle::tier_delete_journal::persist_tier_delete_journal_entry(store, &je).await?;
let mut expiry_state = super::usecase_storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_ExpiryState
let mut expiry_state = super::storage_compat::lifecycle::bucket_lifecycle_ops::GLOBAL_ExpiryState
.write()
.await;
if let Err(err) = expiry_state.enqueue_tier_journal_entry(&je).await {
@@ -1603,7 +1601,7 @@ impl DefaultObjectUsecase {
#[allow(clippy::too_many_arguments)]
async fn prepare_get_object_read(
req: &S3Request<GetObjectInput>,
store: &super::usecase_storage_compat::ECStore,
store: &super::storage_compat::ECStore,
manager: &ConcurrencyManager,
bucket: &str,
key: &str,
@@ -2197,7 +2195,7 @@ impl DefaultObjectUsecase {
insert_str(
&mut metadata,
SUFFIX_COMPRESSION,
super::usecase_storage_compat::compression_metadata_value(algorithm),
super::storage_compat::compression_metadata_value(algorithm),
);
insert_str(&mut metadata, SUFFIX_ACTUAL_SIZE, size.to_string());
@@ -2212,7 +2210,7 @@ impl DefaultObjectUsecase {
insert_str(
&mut opts.user_defined,
SUFFIX_COMPRESSION,
super::usecase_storage_compat::compression_metadata_value(algorithm),
super::storage_compat::compression_metadata_value(algorithm),
);
insert_str(&mut opts.user_defined, SUFFIX_ACTUAL_SIZE, size.to_string());
@@ -2404,12 +2402,8 @@ impl DefaultObjectUsecase {
maybe_enqueue_transition_immediate(&obj_info, LcEventSrc::S3PutObject).await;
// Fast in-memory update for immediate quota and admin usage consistency
super::usecase_storage_compat::record_bucket_object_write_memory(
&bucket,
previous_current_size,
obj_info.size.max(0) as u64,
)
.await;
super::storage_compat::record_bucket_object_write_memory(&bucket, previous_current_size, obj_info.size.max(0) as u64)
.await;
let raw_version = obj_info.version_id.map(|v| v.to_string());
@@ -3269,7 +3263,7 @@ impl DefaultObjectUsecase {
insert_str(
&mut compress_metadata,
SUFFIX_COMPRESSION,
super::usecase_storage_compat::compression_metadata_value(CompressionAlgorithm::default()),
super::storage_compat::compression_metadata_value(CompressionAlgorithm::default()),
);
insert_str(&mut compress_metadata, SUFFIX_ACTUAL_SIZE, actual_size.to_string());
} else {
@@ -3362,12 +3356,7 @@ impl DefaultObjectUsecase {
// Update quota tracking after successful copy
if has_bucket_metadata {
super::usecase_storage_compat::record_bucket_object_write_memory(
&bucket,
previous_current_size,
oi.size.max(0) as u64,
)
.await;
super::storage_compat::record_bucket_object_write_memory(&bucket, previous_current_size, oi.size.max(0) as u64).await;
}
let raw_dest_version = oi.version_id.map(|v| v.to_string());
@@ -3644,7 +3633,7 @@ impl DefaultObjectUsecase {
);
}
let size = object_sizes[i].max(0) as u64;
super::usecase_storage_compat::record_bucket_object_delete_memory(
super::storage_compat::record_bucket_object_delete_memory(
&bucket,
size,
existing_object_infos[i].is_some() && object_to_delete[i].version_id.is_none(),
@@ -3882,7 +3871,7 @@ impl DefaultObjectUsecase {
}
// Fast in-memory update for immediate quota and admin usage consistency
super::usecase_storage_compat::record_bucket_object_delete_memory(
super::storage_compat::record_bucket_object_delete_memory(
&bucket,
obj_info.size.max(0) as u64,
opts.version_id.is_none(),
@@ -4859,7 +4848,7 @@ impl DefaultObjectUsecase {
insert_str(
&mut metadata,
SUFFIX_COMPRESSION,
super::usecase_storage_compat::compression_metadata_value(algorithm),
super::storage_compat::compression_metadata_value(algorithm),
);
insert_str(&mut metadata, SUFFIX_ACTUAL_SIZE, size.to_string());
-159
View File
@@ -1,159 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) use super::storage_compat::{
AppObjectLockConfigExt, AppReplicationConfigExt, AppVersioningConfigExt, DiskError, DynReader, ECStore, EndpointServerPools,
Error, HashReader, MIN_DISK_COMPRESSIBLE_SIZE, PoolDecommissionInfo, PoolStatus, StorageError, WriteEncryption, WritePlan,
apply_bucket_usage_memory_overlay, compression_metadata_value, get_global_notification_sys, get_lock_acquire_timeout,
get_server_info, get_total_usable_capacity, get_total_usable_capacity_free, is_all_buckets_not_found, is_disk_compressible,
is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found, is_valid_storage_class,
load_data_usage_from_backend, predict_lifecycle_expiration, record_bucket_object_delete_memory,
record_bucket_object_write_memory, validate_restore_request, wrap_reader,
};
#[cfg(test)]
pub(crate) use super::storage_compat::{
AppWarmBackend, DecryptReader, EncryptReader, Endpoint, Endpoints, GLOBAL_TierConfigMgr, HardLimitReader, PoolEndpoints,
TierConfig, TierType, WarmBackendGetOpts, boxed_reader, init_local_disks,
};
pub(crate) mod bucket_target_sys {
pub(crate) use super::super::storage_compat::bucket_target_sys::BucketTargetSys;
}
pub(crate) mod lifecycle {
pub(crate) mod bucket_lifecycle_audit {
pub(crate) use super::super::super::storage_compat::lifecycle::bucket_lifecycle_audit::LcEventSrc;
}
pub(crate) mod bucket_lifecycle_ops {
pub(crate) use super::super::super::storage_compat::lifecycle::bucket_lifecycle_ops::{
GLOBAL_ExpiryState, enqueue_expiry_for_existing_objects, enqueue_transition_for_existing_objects,
enqueue_transition_immediate, post_restore_opts, validate_lifecycle_config, validate_transition_tier,
};
#[cfg(test)]
pub(crate) use super::super::super::storage_compat::lifecycle::bucket_lifecycle_ops::init_background_expiry;
}
pub(crate) mod lifecycle_contract {
pub(crate) use super::super::super::storage_compat::lifecycle::lifecycle::{
Event, ObjectOpts, TRANSITION_COMPLETE, TransitionOptions, expected_expiry_time,
};
#[cfg(test)]
pub(crate) use super::super::super::storage_compat::lifecycle::lifecycle::IlmAction;
}
pub(crate) use lifecycle_contract as lifecycle;
pub(crate) mod tier_delete_journal {
pub(crate) use super::super::super::storage_compat::lifecycle::tier_delete_journal::persist_tier_delete_journal_entry;
}
pub(crate) mod tier_sweeper {
pub(crate) use super::super::super::storage_compat::lifecycle::tier_sweeper::{
transitioned_delete_journal_entry, transitioned_force_delete_journal_entry,
};
}
}
pub(crate) mod metadata {
pub(crate) use super::super::storage_compat::metadata::{
BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG,
BUCKET_PUBLIC_ACCESS_BLOCK_CONFIG, BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG,
BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG,
};
#[cfg(test)]
pub(crate) use super::super::storage_compat::metadata::OBJECT_LOCK_CONFIG;
}
pub(crate) mod metadata_sys {
pub(crate) use super::super::storage_compat::metadata_sys::{
BucketMetadataSys, delete, get_bucket_policy, get_bucket_policy_raw, get_bucket_targets_config, get_cors_config,
get_lifecycle_config, get_notification_config, get_object_lock_config, get_public_access_block_config,
get_replication_config, get_sse_config, get_tagging_config, update,
};
#[cfg(test)]
pub(crate) use super::super::storage_compat::metadata_sys::init_bucket_metadata_sys;
}
pub(crate) mod object_api_utils {
pub(crate) use super::super::storage_compat::object_api_utils::to_s3s_etag;
}
pub(crate) mod object_lock {
pub(crate) mod objectlock {
pub(crate) use super::super::super::storage_compat::object_lock::objectlock::{
get_object_legalhold_meta, get_object_retention_meta,
};
}
pub(crate) mod objectlock_sys {
pub(crate) use super::super::super::storage_compat::object_lock::objectlock_sys::{
BucketObjectLockSys, check_object_lock_for_deletion, is_retention_active,
};
}
}
pub(crate) mod policy_sys {
pub(crate) use super::super::storage_compat::policy_sys::PolicySys;
}
pub(crate) mod quota {
pub(crate) use super::super::storage_compat::quota::QuotaOperation;
pub(crate) mod checker {
pub(crate) use super::super::super::storage_compat::quota::checker::QuotaChecker;
}
}
pub(crate) mod replication {
pub(crate) use super::super::storage_compat::replication::{
DeletedObjectReplicationInfo, ObjectOpts, check_replicate_delete, get_must_replicate_options, must_replicate,
schedule_replication, schedule_replication_delete,
};
}
pub(crate) mod storageclass {
pub(crate) use super::super::storage_compat::storageclass::STANDARD;
#[cfg(test)]
pub(crate) use super::super::storage_compat::storageclass::STANDARD_IA;
}
pub(crate) mod tagging {
pub(crate) use super::super::storage_compat::tagging::decode_tags;
}
pub(crate) mod target {
pub(crate) use super::super::storage_compat::target::{BucketTargetType, BucketTargets};
#[cfg(test)]
pub(crate) use super::super::storage_compat::target::BucketTarget;
}
#[cfg(test)]
pub(crate) mod transition_api {
pub(crate) use super::super::storage_compat::transition_api::{ReadCloser, ReaderImpl};
}
pub(crate) mod utils {
pub(crate) use super::super::storage_compat::utils::serialize;
}
pub(crate) mod versioning_sys {
pub(crate) use super::super::storage_compat::versioning_sys::BucketVersioningSys;
}
-1
View File
@@ -52,7 +52,6 @@
pub mod capacity_integration;
pub mod service;
mod storage_compat;
pub use service::{
capacity_disk_ref, get_cached_capacity_with_metrics, init_capacity_management_for_local_disks, record_capacity_write,
+8 -3
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::CapacityDiskExt as _;
use rustfs_ecstore::api::{disk as ecstore_disk, storage as ecstore_storage};
use rustfs_io_metrics::capacity_metrics::{
record_capacity_cache_hit, record_capacity_cache_miss, record_capacity_cache_served, record_capacity_refresh_request,
record_capacity_scan_mode,
@@ -263,7 +263,7 @@ pub async fn init_capacity_management_for_local_disks() {
"Capacity manager state changed"
);
let disks = super::storage_compat::all_local_disk().await;
let disks = ecstore_storage::all_local_disk().await;
if disks.is_empty() {
warn!(
component = LOG_COMPONENT_CAPACITY,
@@ -286,7 +286,12 @@ pub async fn init_capacity_management_for_local_disks() {
let disk_refs = disks
.iter()
.map(|ds| capacity_disk_ref(ds.endpoint().to_string(), ds.to_string()))
.map(|ds| {
capacity_disk_ref(
ecstore_disk::DiskAPI::endpoint(ds.as_ref()).to_string(),
ecstore_disk::DiskAPI::to_string(ds.as_ref()),
)
})
.collect();
info!(
-40
View File
@@ -1,40 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use rustfs_ecstore::api::disk as ecstore_disk;
use rustfs_ecstore::api::storage as ecstore_storage;
pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint;
pub(crate) trait CapacityDiskExt {
fn endpoint(&self) -> Endpoint;
fn to_string(&self) -> String;
}
impl<T> CapacityDiskExt for T
where
T: ecstore_disk::DiskAPI,
{
fn endpoint(&self) -> Endpoint {
ecstore_disk::DiskAPI::endpoint(self)
}
fn to_string(&self) -> String {
ecstore_disk::DiskAPI::to_string(self)
}
}
pub(crate) async fn all_local_disk() -> Vec<ecstore_disk::DiskStore> {
ecstore_storage::all_local_disk().await
}
+1 -1
View File
@@ -12,11 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::{EcstoreEventArgs, register_event_dispatch_hook};
use super::{module_switch::resolve_notify_module_state, refresh_persisted_module_switches_from_store};
use crate::app::context::resolve_server_config;
use crate::storage::StorageObjectInfo;
use chrono::{DateTime, Utc};
use rustfs_ecstore::api::event::{EventArgs as EcstoreEventArgs, register_event_dispatch_hook};
use rustfs_notify::{EventArgs as NotifyEventArgs, NotifyObjectInfo};
use rustfs_s3_types::EventName;
use std::net::SocketAddr;
+1 -1
View File
@@ -13,7 +13,6 @@
// limitations under the License.
// Import HTTP server components and compression configuration
use super::storage_compat::{TONIC_RPC_PREFIX, verify_rpc_signature};
use crate::admin;
use crate::auth::IAMAuth;
use crate::auth_keystone;
@@ -46,6 +45,7 @@ use metrics::{counter, gauge, histogram};
use opentelemetry::global;
use opentelemetry::trace::TraceContextExt;
use rustfs_common::GlobalReadiness;
use rustfs_ecstore::api::rpc::{TONIC_RPC_PREFIX, verify_rpc_signature};
use rustfs_keystone::KeystoneAuthLayer;
#[cfg(feature = "swift")]
use rustfs_protocols::SwiftService;
-1
View File
@@ -24,7 +24,6 @@ mod prefix;
mod readiness;
mod runtime;
mod service_state;
mod storage_compat;
pub mod tls_material;
use tracing::warn;
+3 -5
View File
@@ -12,9 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::{
EcstoreError as StorageError, read_ecstore_config, resolve_object_store_handle, save_ecstore_config,
};
use rustfs_ecstore::api::{config as ecstore_config, error::Error as StorageError, global::resolve_object_store_handle};
use serde::{Deserialize, Serialize};
use std::sync::atomic::{AtomicBool, Ordering};
@@ -164,7 +162,7 @@ pub(crate) async fn refresh_persisted_module_switches_from_store() -> Result<Per
return Err("storage layer not initialized".to_string());
};
let (config, configured) = match read_ecstore_config(store, MODULE_SWITCH_CONFIG_PATH).await {
let (config, configured) = match ecstore_config::com::read_config(store, MODULE_SWITCH_CONFIG_PATH).await {
Ok(data) => (
serde_json::from_slice::<PersistedModuleSwitches>(&data)
.map_err(|e| format!("failed to deserialize module switch config: {e}"))?,
@@ -186,7 +184,7 @@ pub(crate) async fn save_persisted_module_switches_to_store(config: PersistedMod
};
let data = serde_json::to_vec(&config).map_err(|e| format!("failed to serialize module switch config: {e}"))?;
save_ecstore_config(store, MODULE_SWITCH_CONFIG_PATH, data)
ecstore_config::com::save_config(store, MODULE_SWITCH_CONFIG_PATH, data)
.await
.map_err(|e| format!("failed to save module switch config: {e}"))?;
+7 -8
View File
@@ -12,10 +12,6 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::{
Endpoint, EndpointServerPools, get_global_endpoints_opt, get_global_lock_clients, is_dist_erasure,
resolve_object_store_handle,
};
use crate::server::{ServiceState, ServiceStateManager};
use crate::server::{has_path_prefix, is_table_catalog_path};
use bytes::Bytes;
@@ -25,6 +21,13 @@ use http_body_util::{BodyExt, Full};
use hyper::body::Incoming;
use metrics::{counter, gauge};
use rustfs_common::GlobalReadiness;
#[cfg(test)]
use rustfs_ecstore::api::layout::{Endpoints, PoolEndpoints};
use rustfs_ecstore::api::{
disk::endpoint::Endpoint,
global::{get_global_endpoints_opt, get_global_lock_clients, is_dist_erasure, resolve_object_store_handle},
layout::EndpointServerPools,
};
use rustfs_iam::get_global_iam_sys;
use rustfs_madmin::{Disk, StorageInfo};
use rustfs_storage_api::StorageAdminApi;
@@ -839,8 +842,6 @@ mod tests {
#[test]
fn aggregate_lock_quorum_status_requires_each_set_to_meet_quorum() {
use super::super::storage_compat::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints};
let endpoints = vec![
Endpoint {
url: url::Url::parse("http://node1:9000/data1").unwrap(),
@@ -896,8 +897,6 @@ mod tests {
#[test]
fn aggregate_lock_quorum_status_fails_when_any_set_loses_quorum() {
use super::super::storage_compat::{Endpoint, EndpointServerPools, Endpoints, PoolEndpoints};
let endpoints = vec![
Endpoint {
url: url::Url::parse("http://node1:9000/data1").unwrap(),
-73
View File
@@ -1,73 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use std::{collections::HashMap, sync::Arc};
use rustfs_ecstore::api::config as ecstore_config;
use rustfs_ecstore::api::disk as ecstore_disk;
use rustfs_ecstore::api::error as ecstore_error;
use rustfs_ecstore::api::event as ecstore_event;
use rustfs_ecstore::api::global as ecstore_global;
use rustfs_ecstore::api::layout as ecstore_layout;
use rustfs_ecstore::api::rpc as ecstore_rpc;
use rustfs_ecstore::api::storage as ecstore_storage;
pub(crate) const TONIC_RPC_PREFIX: &str = ecstore_rpc::TONIC_RPC_PREFIX;
pub(crate) type ECStore = ecstore_storage::ECStore;
pub(crate) type EcstoreError = ecstore_error::Error;
pub(crate) type EcstoreEventArgs = ecstore_event::EventArgs;
pub(crate) type EcstoreResult<T> = ecstore_error::Result<T>;
pub(crate) type Endpoint = ecstore_disk::endpoint::Endpoint;
pub(crate) type EndpointServerPools = ecstore_layout::EndpointServerPools;
#[cfg(test)]
pub(crate) type Endpoints = ecstore_layout::Endpoints;
#[cfg(test)]
pub(crate) type PoolEndpoints = ecstore_layout::PoolEndpoints;
pub(crate) async fn read_ecstore_config(api: Arc<ECStore>, file: &str) -> EcstoreResult<Vec<u8>> {
ecstore_config::com::read_config(api, file).await
}
pub(crate) async fn save_ecstore_config(api: Arc<ECStore>, file: &str, data: Vec<u8>) -> EcstoreResult<()> {
ecstore_config::com::save_config(api, file, data).await
}
pub(crate) fn register_event_dispatch_hook<F>(hook: F) -> bool
where
F: Fn(EcstoreEventArgs) + Send + Sync + 'static,
{
ecstore_event::register_event_dispatch_hook(hook)
}
pub(crate) fn get_global_endpoints_opt() -> Option<EndpointServerPools> {
ecstore_global::get_global_endpoints_opt()
}
pub(crate) fn get_global_lock_clients() -> Option<&'static HashMap<String, Arc<dyn rustfs_lock::client::LockClient>>> {
ecstore_global::get_global_lock_clients()
}
pub(crate) async fn is_dist_erasure() -> bool {
ecstore_global::is_dist_erasure().await
}
pub(crate) fn resolve_object_store_handle() -> Option<Arc<ECStore>> {
ecstore_global::resolve_object_store_handle()
}
pub(crate) fn verify_rpc_signature(url: &str, method: &http::Method, headers: &http::HeaderMap) -> std::io::Result<()> {
ecstore_rpc::verify_rpc_signature(url, method, headers)
}
+5 -5
View File
@@ -12,12 +12,12 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::core_storage_compat::ECStore;
use super::core_storage_compat::resolve_object_store_handle;
use super::core_storage_compat::{
use super::ecfs::FS;
use super::storage_compat::ECStore;
use super::storage_compat::resolve_object_store_handle;
use super::storage_compat::{
PolicySys, StorageError, get_bucket_metadata, get_bucket_policy_raw, get_public_access_block_config, is_err_bucket_not_found,
};
use super::ecfs::FS;
use crate::auth::{check_key_valid, get_condition_values_with_query_and_client_info, get_session_token};
use crate::error::ApiError;
use crate::license::license_check;
@@ -930,7 +930,7 @@ impl S3Access for FS {
let req_info = ReqInfo {
cred,
is_owner,
region: super::core_storage_compat::get_global_region(),
region: super::storage_compat::get_global_region(),
request_context,
..Default::default()
};
-31
View File
@@ -1,31 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) use super::storage_compat::{
BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG,
BUCKET_WEBSITE_CONFIG, BucketVersioningSys, ECStore, Error, GetObjectReader, OBJECT_LOCK_CONFIG, ObjectInfo, ObjectOptions,
PolicySys, PutObjReader, Result, StorageError, StorageReplicationConfigExt, StorageVersioningConfigExt, WriteEncryption,
add_object_lock_years, check_retention_for_modification, decode_tags, decode_tags_to_map, delete_bucket_metadata_config,
encode_tags, get_bucket_accelerate_config, get_bucket_cors_config, get_bucket_logging_config, get_bucket_metadata,
get_bucket_object_lock_config, get_bucket_policy_raw, get_bucket_replication_config, get_bucket_request_payment_config,
get_bucket_sse_config, get_bucket_website_config, get_global_region, get_public_access_block_config, is_err_bucket_not_found,
is_err_object_not_found, is_err_version_not_found, record_replication_proxy, resolve_object_store_handle, serialize,
update_bucket_metadata_config,
};
#[cfg(test)]
pub(crate) use super::storage_compat::{
BucketMetadata, DEFAULT_READ_BUFFER_SIZE, bucket_metadata_sys_initialized, get_global_bucket_metadata_sys,
set_bucket_metadata,
};
+2 -2
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::core_storage_compat::{
use super::storage_compat::{
BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG,
BUCKET_WEBSITE_CONFIG, BucketVersioningSys, OBJECT_LOCK_CONFIG, StorageError, check_retention_for_modification, decode_tags,
decode_tags_to_map, delete_bucket_metadata_config, encode_tags, get_bucket_accelerate_config, get_bucket_logging_config,
@@ -20,7 +20,7 @@ use super::core_storage_compat::{
is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found, record_replication_proxy, serialize,
update_bucket_metadata_config,
};
use super::core_storage_compat::{StorageReplicationConfigExt as _, StorageVersioningConfigExt as _};
use super::storage_compat::{StorageReplicationConfigExt as _, StorageVersioningConfigExt as _};
use crate::app::bucket_usecase::DefaultBucketUsecase;
use crate::app::context::resolve_object_store_handle;
use crate::app::multipart_usecase::DefaultMultipartUsecase;
+3 -3
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::core_storage_compat::StorageReplicationConfigExt as _;
use super::core_storage_compat::{
use super::storage_compat::StorageReplicationConfigExt as _;
use super::storage_compat::{
StorageError, add_object_lock_years, get_bucket_cors_config, get_bucket_object_lock_config, get_bucket_replication_config,
resolve_object_store_handle,
};
@@ -738,7 +738,7 @@ pub(crate) async fn has_replication_rules(bucket: &str, objects: &[ObjectToDelet
}
/// Helper function to get store and validate bucket exists
pub(crate) async fn get_validated_store(bucket: &str) -> S3Result<Arc<super::core_storage_compat::ECStore>> {
pub(crate) async fn get_validated_store(bucket: &str) -> S3Result<Arc<super::storage_compat::ECStore>> {
let Some(store) = resolve_object_store_handle() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
+4 -4
View File
@@ -14,7 +14,7 @@
#[cfg(test)]
mod tests {
use super::super::core_storage_compat::{
use super::super::storage_compat::{
BucketMetadata, DEFAULT_READ_BUFFER_SIZE, get_global_bucket_metadata_sys, set_bucket_metadata,
};
use crate::config::WorkloadProfile;
@@ -941,8 +941,8 @@ mod tests {
#[tokio::test]
async fn test_validate_bucket_object_lock_enabled() {
use super::super::core_storage_compat::bucket_metadata_sys_initialized;
use super::super::core_storage_compat::set_bucket_metadata;
use super::super::storage_compat::bucket_metadata_sys_initialized;
use super::super::storage_compat::set_bucket_metadata;
use s3s::dto::{ObjectLockConfiguration, ObjectLockEnabled};
use time::OffsetDateTime;
@@ -1781,7 +1781,7 @@ mod tests {
/// with a single-element vec value, matching the format expected by policy evaluation.
#[test]
fn test_object_tag_condition_key_format() {
use super::super::core_storage_compat::decode_tags_to_map;
use super::super::storage_compat::decode_tags_to_map;
use std::collections::HashMap;
let tags_str = "security=public&project=webapp&env=prod";
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::core_storage_compat::ECStore;
use super::storage_compat::ECStore;
use rustfs_storage_api::ListOperations as _;
use std::sync::Arc;
+4 -5
View File
@@ -15,7 +15,6 @@
pub mod access;
pub mod backpressure;
pub mod concurrency;
pub(crate) mod core_storage_compat;
pub mod deadlock_detector;
pub mod ecfs;
pub(crate) mod helper;
@@ -30,11 +29,11 @@ pub mod timeout_wrapper;
pub mod tonic_service;
pub(crate) type StorageDeletedObject = rustfs_storage_api::DeletedObject;
pub(crate) type StorageGetObjectReader = self::core_storage_compat::GetObjectReader;
pub(crate) type StorageObjectInfo = self::core_storage_compat::ObjectInfo;
pub(crate) type StorageObjectOptions = self::core_storage_compat::ObjectOptions;
pub(crate) type StorageGetObjectReader = self::storage_compat::GetObjectReader;
pub(crate) type StorageObjectInfo = self::storage_compat::ObjectInfo;
pub(crate) type StorageObjectOptions = self::storage_compat::ObjectOptions;
pub(crate) type StorageObjectToDelete = rustfs_storage_api::ObjectToDelete;
pub(crate) type StoragePutObjReader = self::core_storage_compat::PutObjReader;
pub(crate) type StoragePutObjReader = self::storage_compat::PutObjReader;
#[cfg(test)]
mod concurrent_fix_test;
+3 -3
View File
@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::core_storage_compat::BucketVersioningSys;
use super::core_storage_compat::Result;
use super::core_storage_compat::StorageError;
use super::storage_compat::BucketVersioningSys;
use super::storage_compat::Result;
use super::storage_compat::StorageError;
use http::header::{IF_MATCH, IF_NONE_MATCH};
use http::{HeaderMap, HeaderValue};
use rustfs_utils::http::{
+5 -5
View File
@@ -12,11 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::DEFAULT_READ_BUFFER_SIZE;
use super::storage_compat::StorageDiskRpcExt as _;
use super::storage_compat::WalkDirOptions;
use super::storage_compat::find_local_disk_by_ref;
use super::storage_compat::verify_rpc_signature;
use super::super::storage_compat::DEFAULT_READ_BUFFER_SIZE;
use super::super::storage_compat::StorageDiskRpcExt as _;
use super::super::storage_compat::WalkDirOptions;
use super::super::storage_compat::find_local_disk_by_ref;
use super::super::storage_compat::verify_rpc_signature;
use crate::server::RPC_PREFIX;
use crate::storage::request_context::spawn_traced;
use bytes::{Bytes, BytesMut};
-1
View File
@@ -14,7 +14,6 @@
pub mod http_service;
pub mod node_service;
pub(crate) mod storage_compat;
pub use http_service::InternodeRpcService;
pub use node_service::{NodeService, make_server};
+6 -6
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::{
use super::super::storage_compat::{
CollectMetricsOpts, DeleteOptions, DiskError, DiskInfoOptions, DiskStore, FileInfoVersions, LocalPeerS3Client, MetricType,
PEER_RESTSIGNAL, PEER_RESTSUB_SYS, ReadMultipleReq, ReadMultipleResp, ReadOptions, SERVICE_SIGNAL_REFRESH_CONFIG,
SERVICE_SIGNAL_RELOAD_DYNAMIC, StorageDiskRpcExt as _, StoragePeerS3ClientExt as _, UpdateMetadataOpts, all_local_disk_path,
@@ -122,11 +122,11 @@ fn unimplemented_rpc(method: &str) -> Status {
Status::unimplemented(format!("{method} is not implemented"))
}
fn background_rebalance_start_error_message(result: super::storage_compat::Result<()>) -> Option<String> {
fn background_rebalance_start_error_message(result: super::super::storage_compat::Result<()>) -> Option<String> {
result.err().map(|err| format!("start_rebalance failed: {err}"))
}
fn stop_rebalance_response(result: super::storage_compat::Result<()>) -> StopRebalanceResponse {
fn stop_rebalance_response(result: super::super::storage_compat::Result<()>) -> StopRebalanceResponse {
match result {
Ok(_) => StopRebalanceResponse {
success: true,
@@ -2389,7 +2389,7 @@ mod tests {
#[test]
fn test_background_rebalance_start_error_message_formats_error() {
let message = background_rebalance_start_error_message(Err(super::super::storage_compat::Error::other("boom")))
let message = background_rebalance_start_error_message(Err(super::super::super::storage_compat::Error::other("boom")))
.expect("background rebalance start failure should be formatted");
assert!(message.contains("start_rebalance failed"));
@@ -2398,7 +2398,7 @@ mod tests {
#[test]
fn test_stop_rebalance_response_reports_local_stop_error() {
let response = stop_rebalance_response(Err(super::super::storage_compat::Error::other("boom")));
let response = stop_rebalance_response(Err(super::super::super::storage_compat::Error::other("boom")));
assert!(!response.success);
assert!(response.error_info.as_deref().is_some_and(|message| message.contains("boom")));
@@ -2736,7 +2736,7 @@ mod tests {
vars.insert(PEER_RESTSIGNAL.to_string(), SERVICE_SIGNAL_RELOAD_DYNAMIC.to_string());
vars.insert(
PEER_RESTSUB_SYS.to_string(),
super::super::storage_compat::STORAGE_CLASS_SUB_SYS.to_string(),
super::super::super::storage_compat::STORAGE_CLASS_SUB_SYS.to_string(),
);
let request = Request::new(SignalServiceRequest {
-25
View File
@@ -1,25 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) use super::super::storage_compat::{
CollectMetricsOpts, DEFAULT_READ_BUFFER_SIZE, DeleteOptions, DiskError, DiskInfoOptions, DiskStore, FileInfoVersions,
LocalPeerS3Client, MetricType, PEER_RESTSIGNAL, PEER_RESTSUB_SYS, ReadMultipleReq, ReadMultipleResp, ReadOptions, Result,
SERVICE_SIGNAL_REFRESH_CONFIG, SERVICE_SIGNAL_RELOAD_DYNAMIC, StorageDiskRpcExt, StoragePeerS3ClientExt, UpdateMetadataOpts,
WalkDirOptions, all_local_disk_path, collect_local_metrics, find_local_disk_by_ref, get_global_lock_client,
get_local_server_property, load_bucket_metadata, reload_transition_tier_config, resolve_object_store_handle,
set_bucket_metadata, verify_rpc_signature,
};
#[cfg(test)]
pub(crate) use super::super::storage_compat::{Error, STORAGE_CLASS_SUB_SYS};
+1 -1
View File
@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::to_s3s_etag;
use crate::storage::s3_api::common::rustfs_owner;
use percent_encoding::percent_decode_str;
use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag;
use rustfs_storage_api::{
BucketInfo, ListObjectVersionsInfo as StorageListObjectVersionsInfo, ListObjectsV2Info as StorageListObjectsV2Info,
};
-1
View File
@@ -23,5 +23,4 @@ pub(crate) mod acl;
pub(crate) mod bucket;
pub(crate) mod common;
pub(crate) mod multipart;
pub(crate) mod storage_compat;
pub(crate) mod tagging;
+2 -2
View File
@@ -12,8 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::storage_compat::to_s3s_etag;
use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner};
use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag;
use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo};
use s3s::dto::{CommonPrefix, ListMultipartUploadsOutput, ListPartsOutput, MultipartUpload, Part, Timestamp};
use s3s::{S3Error, S3ErrorCode};
@@ -191,12 +191,12 @@ pub(crate) fn build_list_multipart_uploads_output(
#[cfg(test)]
mod tests {
use super::super::storage_compat::to_s3s_etag;
use super::{
MAX_MULTIPART_UPLOADS_LIST, build_list_multipart_uploads_output, build_list_parts_output,
parse_list_multipart_uploads_params, parse_list_parts_params, parse_upload_part_number,
};
use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner};
use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag;
use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo, MultipartInfo, PartInfo};
use s3s::S3ErrorCode;
use s3s::dto::Timestamp;
@@ -1,19 +0,0 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use rustfs_ecstore::api::client as ecstore_client;
pub(crate) fn to_s3s_etag(etag: &str) -> s3s::dto::ETag {
ecstore_client::object_api_utils::to_s3s_etag(etag)
}
+8 -10
View File
@@ -69,7 +69,7 @@
//! }
//! ```
use super::core_storage_compat::StorageError;
use super::storage_compat::StorageError;
#[cfg(feature = "rio-v2")]
use aes_gcm::aead::Payload;
use aes_gcm::{
@@ -133,8 +133,8 @@ const SEALED_KEY_SIZE: usize = DARE_HEADER_SIZE + 32 + DARE_TAG_SIZE;
#[cfg(feature = "rio-v2")]
const OBJECT_KEY_DERIVATION_CONTEXT: &[u8] = b"object-encryption-key generation";
use super::core_storage_compat::Error;
use super::core_storage_compat::get_bucket_sse_config;
use super::storage_compat::Error;
use super::storage_compat::get_bucket_sse_config;
use crate::error::ApiError;
use rustfs_utils::http::headers::{
AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_ALGORITHM, AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_KEY,
@@ -751,19 +751,17 @@ pub struct ManagedSealedKey {
}
impl EncryptionMaterial {
pub fn write_encryption(&self, multipart_part_number: Option<usize>) -> super::core_storage_compat::WriteEncryption {
pub fn write_encryption(&self, multipart_part_number: Option<usize>) -> super::storage_compat::WriteEncryption {
match (self.key_kind, multipart_part_number) {
(EncryptionKeyKind::Object, Some(part_number)) => {
super::core_storage_compat::WriteEncryption::multipart_object_key(self.key_bytes, part_number as u32)
}
(EncryptionKeyKind::Object, None) => {
super::core_storage_compat::WriteEncryption::singlepart_object_key(self.key_bytes)
super::storage_compat::WriteEncryption::multipart_object_key(self.key_bytes, part_number as u32)
}
(EncryptionKeyKind::Object, None) => super::storage_compat::WriteEncryption::singlepart_object_key(self.key_bytes),
(EncryptionKeyKind::Direct, Some(part_number)) => {
super::core_storage_compat::WriteEncryption::multipart(self.key_bytes, self.base_nonce, part_number)
super::storage_compat::WriteEncryption::multipart(self.key_bytes, self.base_nonce, part_number)
}
(EncryptionKeyKind::Direct, None) => {
super::core_storage_compat::WriteEncryption::singlepart(self.key_bytes, self.base_nonce)
super::storage_compat::WriteEncryption::singlepart(self.key_bytes, self.base_nonce)
}
}
}
+81 -8
View File
@@ -102,7 +102,11 @@ RUSTFS_LOCAL_COMPAT_OWNER_SELF_PATH_HITS_FILE="${TMP_DIR}/rustfs_local_compat_ow
RUSTFS_ROOT_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_root_compat_relative_consumer_hits.txt"
RUSTFS_STORAGE_CORE_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_storage_core_compat_relative_consumer_hits.txt"
RUSTFS_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_local_compat_relative_consumer_hits.txt"
RUSTFS_APP_ADMIN_SECONDARY_COMPAT_BRIDGE_HITS_FILE="${TMP_DIR}/rustfs_app_admin_secondary_compat_bridge_hits.txt"
RUSTFS_STORAGE_SECONDARY_COMPAT_BRIDGE_HITS_FILE="${TMP_DIR}/rustfs_storage_secondary_compat_bridge_hits.txt"
RUSTFS_NESTED_SECONDARY_COMPAT_BRIDGE_HITS_FILE="${TMP_DIR}/rustfs_nested_secondary_compat_bridge_hits.txt"
RUSTFS_STORAGE_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_storage_local_compat_relative_consumer_hits.txt"
RUSTFS_RUNTIME_LOCAL_COMPAT_BRIDGE_HITS_FILE="${TMP_DIR}/rustfs_runtime_local_compat_bridge_hits.txt"
RUSTFS_ADMIN_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_admin_local_compat_relative_consumer_hits.txt"
RUSTFS_APP_SERVER_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_app_server_local_compat_relative_consumer_hits.txt"
RUSTFS_HEAL_TEST_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE="${TMP_DIR}/rustfs_heal_test_local_compat_relative_consumer_hits.txt"
@@ -705,7 +709,8 @@ fi
--glob '!crates/ecstore/**' \
--glob '!**/storage_compat.rs' \
--glob '!**/*storage_compat.rs' \
--glob '!target/**' || true
--glob '!target/**' \
| rg -v '^(rustfs/src/capacity/service\.rs|rustfs/src/server/(event|http|module_switch|readiness)\.rs|rustfs/src/storage/s3_api/(bucket|multipart)\.rs):' || true
) |
cat >"$DIRECT_ECSTORE_IMPORT_HITS_FILE"
@@ -863,13 +868,7 @@ fi
(
cd "$ROOT_DIR"
rg -n --no-heading 'pub\(crate\)\s+use\s+crate::(?:admin|app|storage)::storage_compat::\*;' \
rustfs/src/admin/router_storage_compat.rs \
rustfs/src/admin/handlers/storage_compat.rs \
rustfs/src/admin/service/storage_compat.rs \
rustfs/src/app/context/storage_compat.rs \
rustfs/src/app/usecase_storage_compat.rs \
rustfs/src/storage/core_storage_compat.rs \
rustfs/src/storage/rpc/storage_compat.rs || true
rustfs/src/admin/handlers/storage_compat.rs 2>/dev/null || true
) >"$RUSTFS_LOCAL_COMPAT_GLOB_EXPORT_HITS_FILE"
if [[ -s "$RUSTFS_LOCAL_COMPAT_GLOB_EXPORT_HITS_FILE" ]]; then
@@ -1104,6 +1103,55 @@ if [[ -s "$RUSTFS_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE" ]]; then
report_failure "RustFS local compatibility consumers must use relative owner paths instead of crate-qualified local compatibility paths: $(paste -sd '; ' "$RUSTFS_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename 'router_storage_compat|usecase_storage_compat' \
rustfs/src/admin \
rustfs/src/app \
--glob '*.rs' || true
) >"$RUSTFS_APP_ADMIN_SECONDARY_COMPAT_BRIDGE_HITS_FILE"
if [[ -s "$RUSTFS_APP_ADMIN_SECONDARY_COMPAT_BRIDGE_HITS_FILE" ]]; then
report_failure "RustFS app/admin consumers must route directly through their owner storage_compat boundary instead of secondary compatibility bridges: $(paste -sd '; ' "$RUSTFS_APP_ADMIN_SECONDARY_COMPAT_BRIDGE_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --with-filename 'core_storage_compat' \
rustfs/src/storage \
--glob '*.rs' || true
) >"$RUSTFS_STORAGE_SECONDARY_COMPAT_BRIDGE_HITS_FILE"
if [[ -s "$RUSTFS_STORAGE_SECONDARY_COMPAT_BRIDGE_HITS_FILE" ]]; then
report_failure "RustFS storage owner consumers must route directly through storage_compat instead of the secondary core_storage_compat bridge: $(paste -sd '; ' "$RUSTFS_STORAGE_SECONDARY_COMPAT_BRIDGE_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
{
for file in \
rustfs/src/admin/handlers/storage_compat.rs \
rustfs/src/admin/service/storage_compat.rs \
rustfs/src/app/context/storage_compat.rs \
rustfs/src/storage/rpc/storage_compat.rs; do
[[ -e "$file" ]] && printf '%s:1:secondary bridge file exists\n' "$file"
done
rg -n --with-filename 'mod storage_compat' \
rustfs/src/admin/handlers/mod.rs \
rustfs/src/admin/service/mod.rs \
rustfs/src/app/context.rs \
rustfs/src/storage/rpc/mod.rs || true
rg -n --with-filename --pcre2 '(?<!super::)super::storage_compat' \
rustfs/src/admin/handlers \
-g '*.rs' \
-g '!storage_compat.rs' || true
}
) >"$RUSTFS_NESTED_SECONDARY_COMPAT_BRIDGE_HITS_FILE"
if [[ -s "$RUSTFS_NESTED_SECONDARY_COMPAT_BRIDGE_HITS_FILE" ]]; then
report_failure "RustFS nested and handler consumers must route directly through owner storage_compat instead of secondary compatibility bridges: $(paste -sd '; ' "$RUSTFS_NESTED_SECONDARY_COMPAT_BRIDGE_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
{
@@ -1120,6 +1168,31 @@ if [[ -s "$RUSTFS_STORAGE_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE" ]]; then
report_failure "RustFS storage RPC/S3 API compatibility consumers must use relative owner paths instead of crate-qualified local compatibility paths: $(paste -sd '; ' "$RUSTFS_STORAGE_LOCAL_COMPAT_RELATIVE_CONSUMER_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
{
for file in \
rustfs/src/capacity/storage_compat.rs \
rustfs/src/server/storage_compat.rs \
rustfs/src/storage/s3_api/storage_compat.rs; do
[[ -e "$file" ]] && printf '%s:1:runtime local bridge file exists\n' "$file"
done
rg -n --with-filename 'mod storage_compat|pub\(crate\) mod storage_compat' \
rustfs/src/capacity/mod.rs \
rustfs/src/server/mod.rs \
rustfs/src/storage/s3_api/mod.rs || true
rg -n --with-filename 'super::storage_compat' \
rustfs/src/capacity \
rustfs/src/server \
rustfs/src/storage/s3_api \
-g '*.rs' || true
}
) >"$RUSTFS_RUNTIME_LOCAL_COMPAT_BRIDGE_HITS_FILE"
if [[ -s "$RUSTFS_RUNTIME_LOCAL_COMPAT_BRIDGE_HITS_FILE" ]]; then
report_failure "RustFS capacity/server/S3 API runtime consumers must use direct owner APIs instead of local compatibility bridge modules: $(paste -sd '; ' "$RUSTFS_RUNTIME_LOCAL_COMPAT_BRIDGE_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
{