diff --git a/docs/architecture/crate-boundaries.md b/docs/architecture/crate-boundaries.md index 268c92274..6911eda28 100644 --- a/docs/architecture/crate-boundaries.md +++ b/docs/architecture/crate-boundaries.md @@ -152,6 +152,12 @@ bare `init`, or grouped `endpoint::Endpoint` passthroughs. RustFS admin `storage_compat.rs` must expose config IO and default initialization through explicit aliases. The admin compatibility boundary must not restore broad `com` or bare `init` passthroughs. +RustFS storage `storage_compat.rs` must expose bucket metadata, object-lock, +policy, replication, tagging, versioning, object API, and test-only +storage-class config contracts through explicit aliases. The storage +compatibility boundary must not restore broad `metadata`, `metadata_sys`, +`object_lock`, `policy_sys`, `replication`, `tagging`, `utils`, `versioning`, +`versioning_sys`, `object_api_utils`, or `com` passthroughs. ECStore ClusterControlPlane read models must stay owned by the crate-private `cluster` module. Public access goes through `rustfs_ecstore::api::cluster` so diff --git a/docs/architecture/migration-progress.md b/docs/architecture/migration-progress.md index 67f551e9f..7c9215138 100644 --- a/docs/architecture/migration-progress.md +++ b/docs/architecture/migration-progress.md @@ -5,17 +5,17 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ## Current Context - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) -- Branch: `overtrue/arch-admin-config-compat-aliases` -- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080`. -- Stacked on: API-080 root runtime compatibility alias pruning. +- Branch: `overtrue/arch-storage-runtime-compat-aliases` +- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081`. +- Stacked on: API-081 admin config compatibility alias pruning. - PR type for this branch: `pure-move` - Runtime behavior changes: none. -- Rust code changes: prune admin config read/write/delete, server-config read, - server-config save, and config-default initialization passthroughs into - explicit aliases for admin handlers and services. -- CI/script changes: guard against restoring broad admin config compatibility - module passthroughs. -- Docs changes: record the API-081 admin config compatibility alias boundary. +- Rust code changes: prune storage bucket metadata, object-lock, policy, + replication, tagging, versioning, object API, and test-only config + passthroughs into explicit aliases for storage runtime modules and tests. +- CI/script changes: guard against restoring broad storage bucket/object-api + compatibility module passthroughs. +- Docs changes: record the API-082 storage compatibility alias boundary. ## Phase 0 Tasks @@ -157,6 +157,22 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block - Verification: RustFS compile coverage, admin focused compile coverage, formatting, diff hygiene, risk scan, architecture guard, pre-commit quality gate, and three-expert review. +- [x] `API-082` Prune storage bucket compatibility aliases. + - Completed slice: replace storage `metadata`, `metadata_sys`, + `object_lock`, `policy_sys`, `replication`, `tagging`, `utils`, + `versioning`, `versioning_sys`, `object_api_utils`, and test-only `com` + passthroughs with explicit storage compatibility aliases. + - Acceptance: storage S3 handlers, access checks, SSE resolution, RPC + metadata loading, CORS/object-lock helpers, list-output ETag helpers, and + storage tests use direct compatibility aliases while preserving the same + bucket metadata keys and ECStore facade APIs. + - Must preserve: bucket metadata update/delete/read semantics, object-lock + retention/default behavior, bucket policy/public-access checks, SSE bucket + defaults, replication stats lookups, object tag encoding/decoding, S3 list + ETag formatting, and test-only storage-class signal constants. + - Verification: RustFS compile coverage, storage compatibility residual + scan, formatting, diff hygiene, risk scan, architecture guard, pre-commit + quality gate, and three-expert review. - [x] `G-012` Inventory placement and repair invariants. - Acceptance: [`placement-repair-invariants.md`](placement-repair-invariants.md) records @@ -3190,14 +3206,25 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block | Expert | Status | Notes | |---|---|---| -| Quality/architecture | passed | API-081 narrows admin config compatibility to explicit aliases without adding new ECStore ownership cycles. | -| Migration preservation | passed | Admin config history, dynamic KMS/OIDC/audit config, site-replication state, router notification reads, storage-class defaults, and route contracts remain preserved. | -| Testing/verification | passed | RustFS compile coverage, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. | +| Quality/architecture | passed | API-082 narrows RustFS storage bucket, object API, and config compatibility to explicit aliases without adding new ECStore ownership cycles. | +| Migration preservation | passed | Bucket metadata read/update/delete, object-lock checks, policy/public-access reads, SSE defaults, replication/tagging/versioning access, S3 ETag conversion, RPC metadata load, and test-only storage-class access remain preserved. | +| Testing/verification | passed | RustFS compile coverage, storage compatibility residual scans, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. | ## Verification Notes Passed before push: +- Issue #660 API-082 current slice: + - `cargo check -p rustfs --lib`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - Storage compatibility residual scan excluding `storage_compat.rs`: passed. + - Broad storage compatibility export scan: passed. + - Rust added-line risk scan on changed Rust files and guard script: passed. + - `make pre-commit`: passed. + - Issue #660 API-081 current slice: - `cargo check -p rustfs --lib`: passed. - `cargo fmt --all --check`: passed. diff --git a/rustfs/src/storage/access.rs b/rustfs/src/storage/access.rs index 065a195df..a4e4a2c9e 100644 --- a/rustfs/src/storage/access.rs +++ b/rustfs/src/storage/access.rs @@ -19,10 +19,10 @@ use crate::license::license_check; use crate::server::RemoteAddr; use crate::storage::request_context::RequestContext; use crate::storage::storage_compat::ECStore; -use crate::storage::storage_compat::metadata_sys; -use crate::storage::storage_compat::policy_sys::PolicySys; use crate::storage::storage_compat::resolve_object_store_handle; -use crate::storage::storage_compat::{StorageError, is_err_bucket_not_found}; +use crate::storage::storage_compat::{ + PolicySys, StorageError, get_bucket_metadata, get_bucket_policy_raw, get_public_access_block_config, is_err_bucket_not_found, +}; use metrics::counter; use rustfs_iam::error::Error as IamError; use rustfs_policy::policy::action::{Action, AdminAction, S3Action}; @@ -136,7 +136,7 @@ fn classify_bucket_policy_raw_load_error(err: &StorageError) -> BucketPolicyRawL /// Load and parse bucket policy once for ExistingObjectTag hint checks. async fn load_bucket_policy_existing_object_tag_hint(bucket: &str, action: Action) -> BucketPolicyExistingObjectTagHint { - let (policy_str, _) = match metadata_sys::get_bucket_policy_raw(bucket).await { + let (policy_str, _) = match get_bucket_policy_raw(bucket).await { Ok(v) => v, Err(err) => match classify_bucket_policy_raw_load_error(&err) { BucketPolicyRawLoadErrorKind::PolicyMissing => { @@ -671,7 +671,7 @@ pub async fn authorize_request(req: &mut S3Request, action: Action) -> S3R if policy_allowed { deny_anonymous_table_data_plane_if_needed(action, bucket.as_str(), object.as_str()).await?; // RestrictPublicBuckets: when true, deny public access even if bucket policy allows it. - match metadata_sys::get_public_access_block_config(bucket_name).await { + match get_public_access_block_config(bucket_name).await { Ok((config, _)) => { if config.restrict_public_buckets.unwrap_or(false) { return Err(s3_error!(AccessDenied, "Access Denied")); @@ -797,7 +797,7 @@ async fn table_data_plane_resource_for_request( return Ok(None); } - match metadata_sys::get(bucket).await { + match get_bucket_metadata(bucket).await { Ok(metadata) if metadata.table_bucket_enabled() => {} Ok(_) | Err(StorageError::ConfigNotFound) => return Ok(None), Err(err) if is_err_bucket_not_found(&err) => return Ok(None), diff --git a/rustfs/src/storage/ecfs.rs b/rustfs/src/storage/ecfs.rs index b6ffc3e12..5838fc07b 100644 --- a/rustfs/src/storage/ecfs.rs +++ b/rustfs/src/storage/ecfs.rs @@ -22,18 +22,12 @@ use crate::storage::helper::OperationHelper; use crate::storage::options::get_opts; use crate::storage::s3_api::acl; use crate::storage::storage_compat::{ - StorageError, is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found, - metadata::{ - BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG, - BUCKET_WEBSITE_CONFIG, OBJECT_LOCK_CONFIG, - }, - metadata_sys, - object_lock::objectlock_sys::check_retention_for_modification, - replication::{GLOBAL_REPLICATION_STATS, ReplicationConfigurationExt}, - tagging::{decode_tags, decode_tags_to_map, encode_tags}, - utils::serialize, - versioning::VersioningApi, - versioning_sys::BucketVersioningSys, + BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG, + BUCKET_WEBSITE_CONFIG, BucketVersioningSys, GLOBAL_REPLICATION_STATS, OBJECT_LOCK_CONFIG, ReplicationConfigurationExt, + StorageError, VersioningApi, check_retention_for_modification, decode_tags, decode_tags_to_map, + delete_bucket_metadata_config, encode_tags, get_bucket_accelerate_config, get_bucket_logging_config, + get_bucket_object_lock_config, get_bucket_replication_config, get_bucket_request_payment_config, get_bucket_website_config, + is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found, serialize, update_bucket_metadata_config, }; use crate::storage::{parse_object_lock_legal_hold, parse_object_lock_retention, validate_bucket_object_lock_enabled}; use crate::table_catalog; @@ -83,7 +77,7 @@ impl FS { } async fn replication_tagging_enabled(bucket: &str, object: &str) -> bool { - metadata_sys::get_replication_config(bucket) + get_bucket_replication_config(bucket) .await .map(|(cfg, _)| cfg.has_active_rules(object, true)) .unwrap_or(false) @@ -357,7 +351,7 @@ impl S3 for FS { .await .map_err(crate::error::ApiError::from)?; - metadata_sys::delete(&req.input.bucket, BUCKET_WEBSITE_CONFIG) + delete_bucket_metadata_config(&req.input.bucket, BUCKET_WEBSITE_CONFIG) .await .map_err(crate::error::ApiError::from)?; @@ -504,7 +498,7 @@ impl S3 for FS { .await .map_err(crate::error::ApiError::from)?; - match metadata_sys::get_accelerate_config(&req.input.bucket).await { + match get_bucket_accelerate_config(&req.input.bucket).await { Ok((accelerate, _)) => Ok(S3Response::new(GetBucketAccelerateConfigurationOutput { status: accelerate.status, ..Default::default() @@ -594,7 +588,7 @@ impl S3 for FS { .await .map_err(crate::error::ApiError::from)?; - match metadata_sys::get_request_payment_config(&req.input.bucket).await { + match get_bucket_request_payment_config(&req.input.bucket).await { Ok((payment, _)) => Ok(S3Response::new(GetBucketRequestPaymentOutput { payer: Some(payment.payer), })), @@ -642,7 +636,7 @@ impl S3 for FS { .await .map_err(crate::error::ApiError::from)?; - match metadata_sys::get_website_config(&req.input.bucket).await { + match get_bucket_website_config(&req.input.bucket).await { Ok((website, _)) => Ok(S3Response::new(GetBucketWebsiteOutput { error_document: website.error_document, index_document: website.index_document, @@ -771,7 +765,7 @@ impl S3 for FS { .await .map_err(ApiError::from)?; - let object_lock_configuration = match metadata_sys::get_object_lock_config(&bucket).await { + let object_lock_configuration = match get_bucket_object_lock_config(&bucket).await { Ok((cfg, _created)) => Some(cfg), Err(err) => { if err == StorageError::ConfigNotFound { @@ -1053,7 +1047,7 @@ impl S3 for FS { let accelerate_config = serialize(&req.input.accelerate_configuration) .map_err(|err| S3Error::with_message(S3ErrorCode::MalformedXML, format!("{err}")))?; - metadata_sys::update(&req.input.bucket, BUCKET_ACCELERATE_CONFIG, accelerate_config) + update_bucket_metadata_config(&req.input.bucket, BUCKET_ACCELERATE_CONFIG, accelerate_config) .await .map_err(crate::error::ApiError::from)?; @@ -1076,7 +1070,7 @@ impl S3 for FS { .await .map_err(crate::error::ApiError::from)?; - match metadata_sys::get_logging_config(&req.input.bucket).await { + match get_bucket_logging_config(&req.input.bucket).await { Ok((logging, _)) => Ok(S3Response::new(GetBucketLoggingOutput { logging_enabled: logging.logging_enabled, })), @@ -1097,7 +1091,7 @@ impl S3 for FS { let logging_config = serialize(&req.input.bucket_logging_status) .map_err(|err| S3Error::with_message(S3ErrorCode::MalformedXML, format!("{err}")))?; - metadata_sys::update(&req.input.bucket, BUCKET_LOGGING_CONFIG, logging_config) + update_bucket_metadata_config(&req.input.bucket, BUCKET_LOGGING_CONFIG, logging_config) .await .map_err(crate::error::ApiError::from)?; @@ -1156,7 +1150,7 @@ impl S3 for FS { let payment_config = serialize(&req.input.request_payment_configuration) .map_err(|err| S3Error::with_message(S3ErrorCode::MalformedXML, format!("{err}")))?; - metadata_sys::update(&req.input.bucket, BUCKET_REQUEST_PAYMENT_CONFIG, payment_config) + update_bucket_metadata_config(&req.input.bucket, BUCKET_REQUEST_PAYMENT_CONFIG, payment_config) .await .map_err(crate::error::ApiError::from)?; @@ -1198,7 +1192,7 @@ impl S3 for FS { let website_config = serialize(&req.input.website_configuration) .map_err(|err| S3Error::with_message(S3ErrorCode::MalformedXML, format!("{err}")))?; - metadata_sys::update(&req.input.bucket, BUCKET_WEBSITE_CONFIG, website_config) + update_bucket_metadata_config(&req.input.bucket, BUCKET_WEBSITE_CONFIG, website_config) .await .map_err(crate::error::ApiError::from)?; @@ -1325,7 +1319,7 @@ impl S3 for FS { validate_object_lock_configuration_input(&input_cfg)?; - match metadata_sys::get_object_lock_config(&bucket).await { + match get_bucket_object_lock_config(&bucket).await { Ok(_) => {} Err(err) => { if err == StorageError::ConfigNotFound { @@ -1347,7 +1341,7 @@ impl S3 for FS { let data = serialize(&input_cfg).map_err(|err| S3Error::with_message(S3ErrorCode::InternalError, format!("{err}")))?; - metadata_sys::update(&bucket, OBJECT_LOCK_CONFIG, data) + update_bucket_metadata_config(&bucket, OBJECT_LOCK_CONFIG, data) .await .map_err(ApiError::from)?; @@ -1361,7 +1355,7 @@ impl S3 for FS { }; let versioning_data = serialize(&enable_versioning_config) .map_err(|err| S3Error::with_message(S3ErrorCode::InternalError, format!("{err}")))?; - metadata_sys::update(&bucket, BUCKET_VERSIONING_CONFIG, versioning_data) + update_bucket_metadata_config(&bucket, BUCKET_VERSIONING_CONFIG, versioning_data) .await .map_err(ApiError::from)?; } diff --git a/rustfs/src/storage/ecfs_extend.rs b/rustfs/src/storage/ecfs_extend.rs index 681460b7a..826bdcf58 100644 --- a/rustfs/src/storage/ecfs_extend.rs +++ b/rustfs/src/storage/ecfs_extend.rs @@ -16,12 +16,10 @@ use crate::config::{RustFSBufferConfig, WorkloadProfile, get_global_buffer_confi use crate::error::ApiError; use crate::server::cors; use crate::storage::ecfs::ListObjectUnorderedQuery; -use crate::storage::storage_compat::StorageError; -use crate::storage::storage_compat::metadata_sys; -use crate::storage::storage_compat::metadata_sys::get_replication_config; -use crate::storage::storage_compat::object_lock::objectlock_sys; -use crate::storage::storage_compat::replication::ReplicationConfigurationExt; -use crate::storage::storage_compat::resolve_object_store_handle; +use crate::storage::storage_compat::{ + ReplicationConfigurationExt, StorageError, add_object_lock_years, get_bucket_cors_config, get_bucket_object_lock_config, + get_bucket_replication_config, resolve_object_store_handle, +}; use http::header::{IF_MATCH, IF_MODIFIED_SINCE, IF_NONE_MATCH, IF_UNMODIFIED_SINCE}; use http::{HeaderMap, HeaderValue, StatusCode}; use metrics::counter; @@ -106,7 +104,7 @@ pub(crate) fn apply_lock_retention(object_lock_config: Option now.saturating_add(time::Duration::days(days as i64)), - (None, Some(years)) => objectlock_sys::add_years(now, years), + (None, Some(years)) => add_object_lock_years(now, years), _ => return, }; @@ -148,7 +146,7 @@ pub(crate) async fn apply_bucket_default_lock_retention( return Ok(()); } - let object_lock_configuration = match metadata_sys::get_object_lock_config(bucket).await { + let object_lock_configuration = match get_bucket_object_lock_config(bucket).await { Ok((cfg, _created)) => Some(cfg), Err(err) => { if err == StorageError::ConfigNotFound { @@ -426,7 +424,7 @@ pub(crate) fn parse_object_lock_legal_hold(legal_hold: Option S3Result<()> { - match metadata_sys::get_object_lock_config(bucket).await { + match get_bucket_object_lock_config(bucket).await { Ok((cfg, _created)) => { if cfg.object_lock_enabled != Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)) { return Err(S3Error::with_message( @@ -723,7 +721,7 @@ where } pub(crate) async fn has_replication_rules(bucket: &str, objects: &[ObjectToDelete]) -> bool { - let (cfg, _created) = match get_replication_config(bucket).await { + let (cfg, _created) = match get_bucket_replication_config(bucket).await { Ok(replication_config) => replication_config, Err(_err) => { return false; @@ -788,7 +786,7 @@ pub(crate) async fn apply_cors_headers(bucket: &str, method: &http::Method, head let origin = headers.get(cors::standard::ORIGIN)?.to_str().ok()?; // Get CORS configuration for the bucket - let cors_config = match metadata_sys::get_cors_config(bucket).await { + let cors_config = match get_bucket_cors_config(bucket).await { Ok((config, _)) => config, Err(_) => return None, // No CORS config, no headers to add }; diff --git a/rustfs/src/storage/ecfs_test.rs b/rustfs/src/storage/ecfs_test.rs index 9a807ea0c..5aa9ff563 100644 --- a/rustfs/src/storage/ecfs_test.rs +++ b/rustfs/src/storage/ecfs_test.rs @@ -18,8 +18,9 @@ mod tests { use crate::server::cors; use crate::storage::ecfs::{FS, validate_object_lock_configuration_input}; use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner}; - use crate::storage::storage_compat::DEFAULT_READ_BUFFER_SIZE; - use crate::storage::storage_compat::{metadata::BucketMetadata, metadata_sys}; + use crate::storage::storage_compat::{ + BucketMetadata, DEFAULT_READ_BUFFER_SIZE, get_global_bucket_metadata_sys, set_bucket_metadata, + }; use crate::storage::{ StorageObjectInfo as ObjectInfo, apply_cors_headers, apply_default_lock_retention_metadata, check_preconditions, get_adaptive_buffer_size_with_profile, get_buffer_size_opt_in, is_etag_equal, matches_origin_pattern, parse_etag, @@ -940,15 +941,12 @@ mod tests { #[tokio::test] async fn test_validate_bucket_object_lock_enabled() { - use crate::storage::storage_compat::metadata::BucketMetadata; - use crate::storage::storage_compat::metadata_sys::set_bucket_metadata; + use crate::storage::storage_compat::GLOBAL_BUCKET_METADATA_SYS; + use crate::storage::storage_compat::set_bucket_metadata; use s3s::dto::{ObjectLockConfiguration, ObjectLockEnabled}; use time::OffsetDateTime; - if crate::storage::storage_compat::metadata_sys::GLOBAL_BucketMetadataSys - .get() - .is_none() - { + if GLOBAL_BUCKET_METADATA_SYS.get().is_none() { eprintln!("Skipping test: GLOBAL_BucketMetadataSys not initialized"); return; } @@ -1529,7 +1527,7 @@ mod tests { #[tokio::test] async fn test_apply_cors_headers_unmatched_origin_with_cors_config() { - if metadata_sys::get_global_bucket_metadata_sys().is_none() { + if get_global_bucket_metadata_sys().is_none() { eprintln!("Skipping test: GLOBAL_BucketMetadataSys not initialized"); return; } @@ -1546,7 +1544,7 @@ mod tests { max_age_seconds: None, }], }); - metadata_sys::set_bucket_metadata(bucket.to_string(), bm).await.unwrap(); + set_bucket_metadata(bucket.to_string(), bm).await.unwrap(); let mut headers = HeaderMap::new(); headers.insert(cors::standard::ORIGIN, "https://disallowed.example.com".parse().unwrap()); @@ -1560,14 +1558,14 @@ mod tests { assert!(result.get(cors::response::ACCESS_CONTROL_ALLOW_ORIGIN).is_none()); assert!(result.get(cors::response::ACCESS_CONTROL_ALLOW_METHODS).is_none()); - metadata_sys::set_bucket_metadata(bucket.to_string(), BucketMetadata::new(bucket)) + set_bucket_metadata(bucket.to_string(), BucketMetadata::new(bucket)) .await .unwrap(); } #[tokio::test] async fn test_apply_cors_headers_credentialed_request_with_wildcard_origin() { - if metadata_sys::get_global_bucket_metadata_sys().is_none() { + if get_global_bucket_metadata_sys().is_none() { eprintln!("Skipping test: GLOBAL_BucketMetadataSys not initialized"); return; } @@ -1584,7 +1582,7 @@ mod tests { max_age_seconds: None, }], }); - metadata_sys::set_bucket_metadata(bucket.to_string(), bm).await.unwrap(); + set_bucket_metadata(bucket.to_string(), bm).await.unwrap(); let mut headers = HeaderMap::new(); headers.insert(cors::standard::ORIGIN, "https://console.localhost".parse().unwrap()); @@ -1600,7 +1598,7 @@ mod tests { assert_eq!(result.get(cors::response::ACCESS_CONTROL_ALLOW_CREDENTIALS).unwrap(), "true"); assert_eq!(result.get(cors::standard::VARY).unwrap(), "Origin"); - metadata_sys::set_bucket_metadata(bucket.to_string(), BucketMetadata::new(bucket)) + set_bucket_metadata(bucket.to_string(), BucketMetadata::new(bucket)) .await .unwrap(); } @@ -1783,7 +1781,7 @@ mod tests { /// with a single-element vec value, matching the format expected by policy evaluation. #[test] fn test_object_tag_condition_key_format() { - use crate::storage::storage_compat::tagging::decode_tags_to_map; + use crate::storage::storage_compat::decode_tags_to_map; use std::collections::HashMap; let tags_str = "security=public&project=webapp&env=prod"; diff --git a/rustfs/src/storage/options.rs b/rustfs/src/storage/options.rs index cfb3db7af..bf5b98393 100644 --- a/rustfs/src/storage/options.rs +++ b/rustfs/src/storage/options.rs @@ -12,9 +12,9 @@ // See the License for the specific language governing permissions and // limitations under the License. +use crate::storage::storage_compat::BucketVersioningSys; use crate::storage::storage_compat::Result; use crate::storage::storage_compat::StorageError; -use crate::storage::storage_compat::versioning_sys::BucketVersioningSys; use http::header::{IF_MATCH, IF_NONE_MATCH}; use http::{HeaderMap, HeaderValue}; use rustfs_utils::http::{ diff --git a/rustfs/src/storage/rpc/bucket.rs b/rustfs/src/storage/rpc/bucket.rs index 2320d7f11..6083157ae 100644 --- a/rustfs/src/storage/rpc/bucket.rs +++ b/rustfs/src/storage/rpc/bucket.rs @@ -51,7 +51,7 @@ impl NodeService { match load_bucket_metadata(store, &bucket).await { Ok(meta) => { - if let Err(err) = metadata_sys::set_bucket_metadata(bucket, meta).await { + if let Err(err) = set_bucket_metadata(bucket, meta).await { return Ok(Response::new(LoadBucketMetadataResponse { success: false, error_info: Some(err.to_string()), diff --git a/rustfs/src/storage/rpc/node_service.rs b/rustfs/src/storage/rpc/node_service.rs index 922fa25f7..efb44cfc6 100644 --- a/rustfs/src/storage/rpc/node_service.rs +++ b/rustfs/src/storage/rpc/node_service.rs @@ -20,8 +20,8 @@ use crate::storage::storage_compat::{ CollectMetricsOpts, DeleteOptions, DiskAPI, DiskError, DiskInfoOptions, DiskStore, FileInfoVersions, GLOBAL_TierConfigMgr, LocalPeerS3Client, MetricType, PEER_RESTSIGNAL, PEER_RESTSUB_SYS, PeerS3Client, ReadMultipleReq, ReadMultipleResp, ReadOptions, SERVICE_SIGNAL_REFRESH_CONFIG, SERVICE_SIGNAL_RELOAD_DYNAMIC, UpdateMetadataOpts, all_local_disk_path, - collect_local_metrics, find_local_disk_by_ref, get_global_lock_client, get_local_server_property, - metadata::load_bucket_metadata, metadata_sys, resolve_object_store_handle, + collect_local_metrics, find_local_disk_by_ref, get_global_lock_client, get_local_server_property, load_bucket_metadata, + resolve_object_store_handle, set_bucket_metadata, }; use bytes::Bytes; use futures::Stream; @@ -2697,7 +2697,7 @@ mod tests { vars.insert(PEER_RESTSIGNAL.to_string(), SERVICE_SIGNAL_RELOAD_DYNAMIC.to_string()); vars.insert( PEER_RESTSUB_SYS.to_string(), - crate::storage::storage_compat::com::STORAGE_CLASS_SUB_SYS.to_string(), + crate::storage::storage_compat::STORAGE_CLASS_SUB_SYS.to_string(), ); let request = Request::new(SignalServiceRequest { diff --git a/rustfs/src/storage/s3_api/bucket.rs b/rustfs/src/storage/s3_api/bucket.rs index b0ce08ed3..509028636 100644 --- a/rustfs/src/storage/s3_api/bucket.rs +++ b/rustfs/src/storage/s3_api/bucket.rs @@ -13,7 +13,7 @@ // limitations under the License. use crate::storage::s3_api::common::rustfs_owner; -use crate::storage::storage_compat::object_api_utils::to_s3s_etag; +use crate::storage::storage_compat::to_s3s_etag; use percent_encoding::percent_decode_str; use rustfs_storage_api::{ BucketInfo, ListObjectVersionsInfo as StorageListObjectVersionsInfo, ListObjectsV2Info as StorageListObjectsV2Info, diff --git a/rustfs/src/storage/s3_api/multipart.rs b/rustfs/src/storage/s3_api/multipart.rs index c36c18fd7..ab4523dfe 100644 --- a/rustfs/src/storage/s3_api/multipart.rs +++ b/rustfs/src/storage/s3_api/multipart.rs @@ -13,7 +13,7 @@ // limitations under the License. use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner}; -use crate::storage::storage_compat::object_api_utils::to_s3s_etag; +use crate::storage::storage_compat::to_s3s_etag; use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo}; use s3s::dto::{CommonPrefix, ListMultipartUploadsOutput, ListPartsOutput, MultipartUpload, Part, Timestamp}; use s3s::{S3Error, S3ErrorCode}; @@ -191,7 +191,7 @@ mod tests { parse_list_multipart_uploads_params, parse_list_parts_params, parse_upload_part_number, }; use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner}; - use crate::storage::storage_compat::object_api_utils::to_s3s_etag; + use crate::storage::storage_compat::to_s3s_etag; use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo, MultipartInfo, PartInfo}; use s3s::S3ErrorCode; use s3s::dto::Timestamp; diff --git a/rustfs/src/storage/sse.rs b/rustfs/src/storage/sse.rs index 830863e52..6c54b9d75 100644 --- a/rustfs/src/storage/sse.rs +++ b/rustfs/src/storage/sse.rs @@ -135,7 +135,7 @@ const OBJECT_KEY_DERIVATION_CONTEXT: &[u8] = b"object-encryption-key generation" use crate::error::ApiError; use crate::storage::storage_compat::Error; -use crate::storage::storage_compat::metadata_sys; +use crate::storage::storage_compat::get_bucket_sse_config; use rustfs_utils::http::headers::{ AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_ALGORITHM, AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_KEY, AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_KEY_MD5, AMZ_SERVER_SIDE_ENCRYPTION_KMS_CONTEXT, @@ -219,7 +219,7 @@ async fn prepare_sse_configuration( } // Get bucket default encryption configuration. - let bucket_sse_config_result = metadata_sys::get_sse_config(bucket).await; + let bucket_sse_config_result = get_bucket_sse_config(bucket).await; debug!( component = LOG_COMPONENT_STORAGE, subsystem = LOG_SUBSYSTEM_SSE, diff --git a/rustfs/src/storage/storage_compat.rs b/rustfs/src/storage/storage_compat.rs index 416c0d741..d5840f47f 100644 --- a/rustfs/src/storage/storage_compat.rs +++ b/rustfs/src/storage/storage_compat.rs @@ -13,12 +13,36 @@ // limitations under the License. pub(crate) use rustfs_ecstore::api::admin::get_local_server_property; -pub(crate) use rustfs_ecstore::api::bucket::{ - metadata, metadata_sys, object_lock, policy_sys, replication, tagging, utils, versioning, versioning_sys, -}; -pub(crate) use rustfs_ecstore::api::client::object_api_utils; #[cfg(test)] -pub(crate) use rustfs_ecstore::api::config::com; +pub(crate) use rustfs_ecstore::api::bucket::metadata::BucketMetadata; +pub(crate) use rustfs_ecstore::api::bucket::metadata::{ + BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG, + BUCKET_WEBSITE_CONFIG, OBJECT_LOCK_CONFIG, load_bucket_metadata, +}; +#[cfg(test)] +pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{ + GLOBAL_BucketMetadataSys as GLOBAL_BUCKET_METADATA_SYS, get_global_bucket_metadata_sys, +}; +pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{ + delete as delete_bucket_metadata_config, get as get_bucket_metadata, get_accelerate_config as get_bucket_accelerate_config, + get_bucket_policy_raw, get_cors_config as get_bucket_cors_config, get_logging_config as get_bucket_logging_config, + get_object_lock_config as get_bucket_object_lock_config, get_public_access_block_config, + get_replication_config as get_bucket_replication_config, get_request_payment_config as get_bucket_request_payment_config, + get_sse_config as get_bucket_sse_config, get_website_config as get_bucket_website_config, set_bucket_metadata, + update as update_bucket_metadata_config, +}; +pub(crate) use rustfs_ecstore::api::bucket::object_lock::objectlock_sys::{ + add_years as add_object_lock_years, check_retention_for_modification, +}; +pub(crate) use rustfs_ecstore::api::bucket::policy_sys::PolicySys; +pub(crate) use rustfs_ecstore::api::bucket::replication::{GLOBAL_REPLICATION_STATS, ReplicationConfigurationExt}; +pub(crate) use rustfs_ecstore::api::bucket::tagging::{decode_tags, decode_tags_to_map, encode_tags}; +pub(crate) use rustfs_ecstore::api::bucket::utils::serialize; +pub(crate) use rustfs_ecstore::api::bucket::versioning::VersioningApi; +pub(crate) use rustfs_ecstore::api::bucket::versioning_sys::BucketVersioningSys; +pub(crate) use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag; +#[cfg(test)] +pub(crate) use rustfs_ecstore::api::config::com::STORAGE_CLASS_SUB_SYS; pub(crate) use rustfs_ecstore::api::disk::error::DiskError; pub(crate) use rustfs_ecstore::api::disk::{ DeleteOptions, DiskAPI, DiskInfoOptions, DiskStore, FileInfoVersions, ReadMultipleReq, ReadMultipleResp, ReadOptions, diff --git a/scripts/check_architecture_migration_rules.sh b/scripts/check_architecture_migration_rules.sh index 8f4a536b1..74fef66bc 100755 --- a/scripts/check_architecture_migration_rules.sh +++ b/scripts/check_architecture_migration_rules.sh @@ -81,6 +81,7 @@ RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_runtime_sec RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_bucket_storage_compat_module_hits.txt" RUSTFS_ROOT_RUNTIME_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_runtime_storage_compat_module_hits.txt" RUSTFS_ADMIN_CONFIG_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_admin_config_storage_compat_module_hits.txt" +RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_storage_bucket_storage_compat_module_hits.txt" PRODUCTION_UNUSED_COMPAT_ALLOW_HITS_FILE="${TMP_DIR}/production_unused_compat_allow_hits.txt" BROAD_STORE_API_COMPAT_REEXPORT_HITS_FILE="${TMP_DIR}/broad_store_api_compat_reexport_hits.txt" NESTED_STORE_API_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/nested_store_api_compat_module_hits.txt" @@ -798,6 +799,16 @@ if [[ -s "$RUSTFS_ADMIN_CONFIG_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then report_failure "RustFS admin storage compatibility must expose config IO and default initialization as explicit aliases: $(paste -sd '; ' "$RUSTFS_ADMIN_CONFIG_STORAGE_COMPAT_MODULE_HITS_FILE")" fi +( + cd "$ROOT_DIR" + rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::bucket::\{[^}]*\b(?:metadata|metadata_sys|object_lock|policy_sys|replication|tagging|utils|versioning|versioning_sys)\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::bucket::(?:metadata|metadata_sys|object_lock|policy_sys|replication|tagging|utils|versioning|versioning_sys)\s*;|pub\(crate\)\s+use rustfs_ecstore::api::client::object_api_utils\s*;|pub\(crate\)\s+use rustfs_ecstore::api::config::com\s*;' \ + rustfs/src/storage/storage_compat.rs || true +) >"$RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" + +if [[ -s "$RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then + report_failure "RustFS storage compatibility must expose bucket/object-api/config contracts as explicit aliases: $(paste -sd '; ' "$RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE")" +fi + ( cd "$ROOT_DIR" rg -n --no-heading '#!\[allow\(unused_imports\)\]' \