gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 16:28:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: prune storage compat bucket aliases (#3689)

Browse Source
This commit is contained in:
安正超
2026-06-21 15:56:13 +08:00
committed by GitHub
parent 6e42afcea5
commit 84d662c312
14 changed files with 143 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/crate-boundaries.md
+6
View File
@@ -152,6 +152,12 @@ bare `init`, or grouped `endpoint::Endpoint` passthroughs.
RustFS admin `storage_compat.rs` must expose config IO and default
initialization through explicit aliases. The admin compatibility boundary must
not restore broad `com` or bare `init` passthroughs.
RustFS storage `storage_compat.rs` must expose bucket metadata, object-lock,
policy, replication, tagging, versioning, object API, and test-only
storage-class config contracts through explicit aliases. The storage
compatibility boundary must not restore broad `metadata`, `metadata_sys`,
`object_lock`, `policy_sys`, `replication`, `tagging`, `utils`, `versioning`,
`versioning_sys`, `object_api_utils`, or `com` passthroughs.
ECStore ClusterControlPlane read models must stay owned by the crate-private
`cluster` module. Public access goes through `rustfs_ecstore::api::cluster` so
docs/architecture/migration-progress.md
+39 -12
View File
@@ -5,17 +5,17 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
## Current Context
- Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660)
- Branch: `overtrue/arch-admin-config-compat-aliases`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080`.
- Stacked on: API-080 root runtime compatibility alias pruning.
- Branch: `overtrue/arch-storage-runtime-compat-aliases`
- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081`.
- Stacked on: API-081 admin config compatibility alias pruning.
- PR type for this branch: `pure-move`
- Runtime behavior changes: none.
- Rust code changes: prune admin config read/write/delete, server-config read,
server-config save, and config-default initialization passthroughs into
explicit aliases for admin handlers and services.
- CI/script changes: guard against restoring broad admin config compatibility
module passthroughs.
- Docs changes: record the API-081 admin config compatibility alias boundary.
- Rust code changes: prune storage bucket metadata, object-lock, policy,
replication, tagging, versioning, object API, and test-only config
passthroughs into explicit aliases for storage runtime modules and tests.
- CI/script changes: guard against restoring broad storage bucket/object-api
compatibility module passthroughs.
- Docs changes: record the API-082 storage compatibility alias boundary.
## Phase 0 Tasks
@@ -157,6 +157,22 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
- Verification: RustFS compile coverage, admin focused compile coverage,
formatting, diff hygiene, risk scan, architecture guard, pre-commit quality
gate, and three-expert review.
- [x] `API-082` Prune storage bucket compatibility aliases.
- Completed slice: replace storage `metadata`, `metadata_sys`,
`object_lock`, `policy_sys`, `replication`, `tagging`, `utils`,
`versioning`, `versioning_sys`, `object_api_utils`, and test-only `com`
passthroughs with explicit storage compatibility aliases.
- Acceptance: storage S3 handlers, access checks, SSE resolution, RPC
metadata loading, CORS/object-lock helpers, list-output ETag helpers, and
storage tests use direct compatibility aliases while preserving the same
bucket metadata keys and ECStore facade APIs.
- Must preserve: bucket metadata update/delete/read semantics, object-lock
retention/default behavior, bucket policy/public-access checks, SSE bucket
defaults, replication stats lookups, object tag encoding/decoding, S3 list
ETag formatting, and test-only storage-class signal constants.
- Verification: RustFS compile coverage, storage compatibility residual
scan, formatting, diff hygiene, risk scan, architecture guard, pre-commit
quality gate, and three-expert review.
- [x] `G-012` Inventory placement and repair invariants.
- Acceptance:
[`placement-repair-invariants.md`](placement-repair-invariants.md) records
@@ -3190,14 +3206,25 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block
| Expert | Status | Notes |
|---|---|---|
| Quality/architecture | passed | API-081 narrows admin config compatibility to explicit aliases without adding new ECStore ownership cycles. |
| Migration preservation | passed | Admin config history, dynamic KMS/OIDC/audit config, site-replication state, router notification reads, storage-class defaults, and route contracts remain preserved. |
| Testing/verification | passed | RustFS compile coverage, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. |
| Quality/architecture | passed | API-082 narrows RustFS storage bucket, object API, and config compatibility to explicit aliases without adding new ECStore ownership cycles. |
| Migration preservation | passed | Bucket metadata read/update/delete, object-lock checks, policy/public-access reads, SSE defaults, replication/tagging/versioning access, S3 ETag conversion, RPC metadata load, and test-only storage-class access remain preserved. |
| Testing/verification | passed | RustFS compile coverage, storage compatibility residual scans, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. |
## Verification Notes
Passed before push:
- Issue #660 API-082 current slice:
- `cargo check -p rustfs --lib`: passed.
- `cargo fmt --all --check`: passed.
- `git diff --check`: passed.
- `bash -n scripts/check_architecture_migration_rules.sh`: passed.
- `./scripts/check_architecture_migration_rules.sh`: passed.
- Storage compatibility residual scan excluding `storage_compat.rs`: passed.
- Broad storage compatibility export scan: passed.
- Rust added-line risk scan on changed Rust files and guard script: passed.
- `make pre-commit`: passed.
- Issue #660 API-081 current slice:
- `cargo check -p rustfs --lib`: passed.
- `cargo fmt --all --check`: passed.
rustfs/src/storage/access.rs
+6 -6
View File
@@ -19,10 +19,10 @@ use crate::license::license_check;
use crate::server::RemoteAddr;
use crate::storage::request_context::RequestContext;
use crate::storage::storage_compat::ECStore;
use crate::storage::storage_compat::metadata_sys;
use crate::storage::storage_compat::policy_sys::PolicySys;
use crate::storage::storage_compat::resolve_object_store_handle;
use crate::storage::storage_compat::{StorageError, is_err_bucket_not_found};
use crate::storage::storage_compat::{
PolicySys, StorageError, get_bucket_metadata, get_bucket_policy_raw, get_public_access_block_config, is_err_bucket_not_found,
};
use metrics::counter;
use rustfs_iam::error::Error as IamError;
use rustfs_policy::policy::action::{Action, AdminAction, S3Action};
@@ -136,7 +136,7 @@ fn classify_bucket_policy_raw_load_error(err: &StorageError) -> BucketPolicyRawL
/// Load and parse bucket policy once for ExistingObjectTag hint checks.
async fn load_bucket_policy_existing_object_tag_hint(bucket: &str, action: Action) -> BucketPolicyExistingObjectTagHint {
let (policy_str, _) = match metadata_sys::get_bucket_policy_raw(bucket).await {
let (policy_str, _) = match get_bucket_policy_raw(bucket).await {
Ok(v) => v,
Err(err) => match classify_bucket_policy_raw_load_error(&err) {
BucketPolicyRawLoadErrorKind::PolicyMissing => {
@@ -671,7 +671,7 @@ pub async fn authorize_request<T>(req: &mut S3Request<T>, action: Action) -> S3R
if policy_allowed {
deny_anonymous_table_data_plane_if_needed(action, bucket.as_str(), object.as_str()).await?;
// RestrictPublicBuckets: when true, deny public access even if bucket policy allows it.
match metadata_sys::get_public_access_block_config(bucket_name).await {
match get_public_access_block_config(bucket_name).await {
Ok((config, _)) => {
if config.restrict_public_buckets.unwrap_or(false) {
return Err(s3_error!(AccessDenied, "Access Denied"));
@@ -797,7 +797,7 @@ async fn table_data_plane_resource_for_request(
return Ok(None);
}
match metadata_sys::get(bucket).await {
match get_bucket_metadata(bucket).await {
Ok(metadata) if metadata.table_bucket_enabled() => {}
Ok(_) | Err(StorageError::ConfigNotFound) => return Ok(None),
Err(err) if is_err_bucket_not_found(&err) => return Ok(None),
rustfs/src/storage/ecfs.rs
+20 -26
View File
@@ -22,18 +22,12 @@ use crate::storage::helper::OperationHelper;
use crate::storage::options::get_opts;
use crate::storage::s3_api::acl;
use crate::storage::storage_compat::{
StorageError, is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found,
metadata::{
BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG,
BUCKET_WEBSITE_CONFIG, OBJECT_LOCK_CONFIG,
},
metadata_sys,
object_lock::objectlock_sys::check_retention_for_modification,
replication::{GLOBAL_REPLICATION_STATS, ReplicationConfigurationExt},
tagging::{decode_tags, decode_tags_to_map, encode_tags},
utils::serialize,
versioning::VersioningApi,
versioning_sys::BucketVersioningSys,
BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG,
BUCKET_WEBSITE_CONFIG, BucketVersioningSys, GLOBAL_REPLICATION_STATS, OBJECT_LOCK_CONFIG, ReplicationConfigurationExt,
StorageError, VersioningApi, check_retention_for_modification, decode_tags, decode_tags_to_map,
delete_bucket_metadata_config, encode_tags, get_bucket_accelerate_config, get_bucket_logging_config,
get_bucket_object_lock_config, get_bucket_replication_config, get_bucket_request_payment_config, get_bucket_website_config,
is_err_bucket_not_found, is_err_object_not_found, is_err_version_not_found, serialize, update_bucket_metadata_config,
};
use crate::storage::{parse_object_lock_legal_hold, parse_object_lock_retention, validate_bucket_object_lock_enabled};
use crate::table_catalog;
@@ -83,7 +77,7 @@ impl FS {
}
async fn replication_tagging_enabled(bucket: &str, object: &str) -> bool {
metadata_sys::get_replication_config(bucket)
get_bucket_replication_config(bucket)
.await
.map(|(cfg, _)| cfg.has_active_rules(object, true))
.unwrap_or(false)
@@ -357,7 +351,7 @@ impl S3 for FS {
.await
.map_err(crate::error::ApiError::from)?;
metadata_sys::delete(&req.input.bucket, BUCKET_WEBSITE_CONFIG)
delete_bucket_metadata_config(&req.input.bucket, BUCKET_WEBSITE_CONFIG)
.await
.map_err(crate::error::ApiError::from)?;
@@ -504,7 +498,7 @@ impl S3 for FS {
.await
.map_err(crate::error::ApiError::from)?;
match metadata_sys::get_accelerate_config(&req.input.bucket).await {
match get_bucket_accelerate_config(&req.input.bucket).await {
Ok((accelerate, _)) => Ok(S3Response::new(GetBucketAccelerateConfigurationOutput {
status: accelerate.status,
..Default::default()
@@ -594,7 +588,7 @@ impl S3 for FS {
.await
.map_err(crate::error::ApiError::from)?;
match metadata_sys::get_request_payment_config(&req.input.bucket).await {
match get_bucket_request_payment_config(&req.input.bucket).await {
Ok((payment, _)) => Ok(S3Response::new(GetBucketRequestPaymentOutput {
payer: Some(payment.payer),
})),
@@ -642,7 +636,7 @@ impl S3 for FS {
.await
.map_err(crate::error::ApiError::from)?;
match metadata_sys::get_website_config(&req.input.bucket).await {
match get_bucket_website_config(&req.input.bucket).await {
Ok((website, _)) => Ok(S3Response::new(GetBucketWebsiteOutput {
error_document: website.error_document,
index_document: website.index_document,
@@ -771,7 +765,7 @@ impl S3 for FS {
.await
.map_err(ApiError::from)?;
let object_lock_configuration = match metadata_sys::get_object_lock_config(&bucket).await {
let object_lock_configuration = match get_bucket_object_lock_config(&bucket).await {
Ok((cfg, _created)) => Some(cfg),
Err(err) => {
if err == StorageError::ConfigNotFound {
@@ -1053,7 +1047,7 @@ impl S3 for FS {
let accelerate_config = serialize(&req.input.accelerate_configuration)
.map_err(|err| S3Error::with_message(S3ErrorCode::MalformedXML, format!("{err}")))?;
metadata_sys::update(&req.input.bucket, BUCKET_ACCELERATE_CONFIG, accelerate_config)
update_bucket_metadata_config(&req.input.bucket, BUCKET_ACCELERATE_CONFIG, accelerate_config)
.await
.map_err(crate::error::ApiError::from)?;
@@ -1076,7 +1070,7 @@ impl S3 for FS {
.await
.map_err(crate::error::ApiError::from)?;
match metadata_sys::get_logging_config(&req.input.bucket).await {
match get_bucket_logging_config(&req.input.bucket).await {
Ok((logging, _)) => Ok(S3Response::new(GetBucketLoggingOutput {
logging_enabled: logging.logging_enabled,
})),
@@ -1097,7 +1091,7 @@ impl S3 for FS {
let logging_config = serialize(&req.input.bucket_logging_status)
.map_err(|err| S3Error::with_message(S3ErrorCode::MalformedXML, format!("{err}")))?;
metadata_sys::update(&req.input.bucket, BUCKET_LOGGING_CONFIG, logging_config)
update_bucket_metadata_config(&req.input.bucket, BUCKET_LOGGING_CONFIG, logging_config)
.await
.map_err(crate::error::ApiError::from)?;
@@ -1156,7 +1150,7 @@ impl S3 for FS {
let payment_config = serialize(&req.input.request_payment_configuration)
.map_err(|err| S3Error::with_message(S3ErrorCode::MalformedXML, format!("{err}")))?;
metadata_sys::update(&req.input.bucket, BUCKET_REQUEST_PAYMENT_CONFIG, payment_config)
update_bucket_metadata_config(&req.input.bucket, BUCKET_REQUEST_PAYMENT_CONFIG, payment_config)
.await
.map_err(crate::error::ApiError::from)?;
@@ -1198,7 +1192,7 @@ impl S3 for FS {
let website_config = serialize(&req.input.website_configuration)
.map_err(|err| S3Error::with_message(S3ErrorCode::MalformedXML, format!("{err}")))?;
metadata_sys::update(&req.input.bucket, BUCKET_WEBSITE_CONFIG, website_config)
update_bucket_metadata_config(&req.input.bucket, BUCKET_WEBSITE_CONFIG, website_config)
.await
.map_err(crate::error::ApiError::from)?;
@@ -1325,7 +1319,7 @@ impl S3 for FS {
validate_object_lock_configuration_input(&input_cfg)?;
match metadata_sys::get_object_lock_config(&bucket).await {
match get_bucket_object_lock_config(&bucket).await {
Ok(_) => {}
Err(err) => {
if err == StorageError::ConfigNotFound {
@@ -1347,7 +1341,7 @@ impl S3 for FS {
let data = serialize(&input_cfg).map_err(|err| S3Error::with_message(S3ErrorCode::InternalError, format!("{err}")))?;
metadata_sys::update(&bucket, OBJECT_LOCK_CONFIG, data)
update_bucket_metadata_config(&bucket, OBJECT_LOCK_CONFIG, data)
.await
.map_err(ApiError::from)?;
@@ -1361,7 +1355,7 @@ impl S3 for FS {
};
let versioning_data = serialize(&enable_versioning_config)
.map_err(|err| S3Error::with_message(S3ErrorCode::InternalError, format!("{err}")))?;
metadata_sys::update(&bucket, BUCKET_VERSIONING_CONFIG, versioning_data)
update_bucket_metadata_config(&bucket, BUCKET_VERSIONING_CONFIG, versioning_data)
.await
.map_err(ApiError::from)?;
}
rustfs/src/storage/ecfs_extend.rs
+9 -11
View File
@@ -16,12 +16,10 @@ use crate::config::{RustFSBufferConfig, WorkloadProfile, get_global_buffer_confi
use crate::error::ApiError;
use crate::server::cors;
use crate::storage::ecfs::ListObjectUnorderedQuery;
use crate::storage::storage_compat::StorageError;
use crate::storage::storage_compat::metadata_sys;
use crate::storage::storage_compat::metadata_sys::get_replication_config;
use crate::storage::storage_compat::object_lock::objectlock_sys;
use crate::storage::storage_compat::replication::ReplicationConfigurationExt;
use crate::storage::storage_compat::resolve_object_store_handle;
use crate::storage::storage_compat::{
ReplicationConfigurationExt, StorageError, add_object_lock_years, get_bucket_cors_config, get_bucket_object_lock_config,
get_bucket_replication_config, resolve_object_store_handle,
};
use http::header::{IF_MATCH, IF_MODIFIED_SINCE, IF_NONE_MATCH, IF_UNMODIFIED_SINCE};
use http::{HeaderMap, HeaderValue, StatusCode};
use metrics::counter;
@@ -106,7 +104,7 @@ pub(crate) fn apply_lock_retention(object_lock_config: Option<ObjectLockConfigur
let now = OffsetDateTime::now_utc();
let retain_until = match (default_retention.days, default_retention.years) {
(Some(days), _) => now.saturating_add(time::Duration::days(days as i64)),
(None, Some(years)) => objectlock_sys::add_years(now, years),
(None, Some(years)) => add_object_lock_years(now, years),
_ => return,
};
@@ -148,7 +146,7 @@ pub(crate) async fn apply_bucket_default_lock_retention(
return Ok(());
}
let object_lock_configuration = match metadata_sys::get_object_lock_config(bucket).await {
let object_lock_configuration = match get_bucket_object_lock_config(bucket).await {
Ok((cfg, _created)) => Some(cfg),
Err(err) => {
if err == StorageError::ConfigNotFound {
@@ -426,7 +424,7 @@ pub(crate) fn parse_object_lock_legal_hold(legal_hold: Option<ObjectLockLegalHol
}
pub(crate) async fn validate_bucket_object_lock_enabled(bucket: &str) -> S3Result<()> {
match metadata_sys::get_object_lock_config(bucket).await {
match get_bucket_object_lock_config(bucket).await {
Ok((cfg, _created)) => {
if cfg.object_lock_enabled != Some(ObjectLockEnabled::from_static(ObjectLockEnabled::ENABLED)) {
return Err(S3Error::with_message(
@@ -723,7 +721,7 @@ where
}
pub(crate) async fn has_replication_rules(bucket: &str, objects: &[ObjectToDelete]) -> bool {
let (cfg, _created) = match get_replication_config(bucket).await {
let (cfg, _created) = match get_bucket_replication_config(bucket).await {
Ok(replication_config) => replication_config,
Err(_err) => {
return false;
@@ -788,7 +786,7 @@ pub(crate) async fn apply_cors_headers(bucket: &str, method: &http::Method, head
let origin = headers.get(cors::standard::ORIGIN)?.to_str().ok()?;
// Get CORS configuration for the bucket
let cors_config = match metadata_sys::get_cors_config(bucket).await {
let cors_config = match get_bucket_cors_config(bucket).await {
Ok((config, _)) => config,
Err(_) => return None, // No CORS config, no headers to add
};
rustfs/src/storage/ecfs_test.rs
+13 -15
View File
@@ -18,8 +18,9 @@ mod tests {
use crate::server::cors;
use crate::storage::ecfs::{FS, validate_object_lock_configuration_input};
use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner};
use crate::storage::storage_compat::DEFAULT_READ_BUFFER_SIZE;
use crate::storage::storage_compat::{metadata::BucketMetadata, metadata_sys};
use crate::storage::storage_compat::{
BucketMetadata, DEFAULT_READ_BUFFER_SIZE, get_global_bucket_metadata_sys, set_bucket_metadata,
};
use crate::storage::{
StorageObjectInfo as ObjectInfo, apply_cors_headers, apply_default_lock_retention_metadata, check_preconditions,
get_adaptive_buffer_size_with_profile, get_buffer_size_opt_in, is_etag_equal, matches_origin_pattern, parse_etag,
@@ -940,15 +941,12 @@ mod tests {
#[tokio::test]
async fn test_validate_bucket_object_lock_enabled() {
use crate::storage::storage_compat::metadata::BucketMetadata;
use crate::storage::storage_compat::metadata_sys::set_bucket_metadata;
use crate::storage::storage_compat::GLOBAL_BUCKET_METADATA_SYS;
use crate::storage::storage_compat::set_bucket_metadata;
use s3s::dto::{ObjectLockConfiguration, ObjectLockEnabled};
use time::OffsetDateTime;
if crate::storage::storage_compat::metadata_sys::GLOBAL_BucketMetadataSys
.get()
.is_none()
{
if GLOBAL_BUCKET_METADATA_SYS.get().is_none() {
eprintln!("Skipping test: GLOBAL_BucketMetadataSys not initialized");
return;
}
@@ -1529,7 +1527,7 @@ mod tests {
#[tokio::test]
async fn test_apply_cors_headers_unmatched_origin_with_cors_config() {
if metadata_sys::get_global_bucket_metadata_sys().is_none() {
if get_global_bucket_metadata_sys().is_none() {
eprintln!("Skipping test: GLOBAL_BucketMetadataSys not initialized");
return;
}
@@ -1546,7 +1544,7 @@ mod tests {
max_age_seconds: None,
}],
});
metadata_sys::set_bucket_metadata(bucket.to_string(), bm).await.unwrap();
set_bucket_metadata(bucket.to_string(), bm).await.unwrap();
let mut headers = HeaderMap::new();
headers.insert(cors::standard::ORIGIN, "https://disallowed.example.com".parse().unwrap());
@@ -1560,14 +1558,14 @@ mod tests {
assert!(result.get(cors::response::ACCESS_CONTROL_ALLOW_ORIGIN).is_none());
assert!(result.get(cors::response::ACCESS_CONTROL_ALLOW_METHODS).is_none());
metadata_sys::set_bucket_metadata(bucket.to_string(), BucketMetadata::new(bucket))
set_bucket_metadata(bucket.to_string(), BucketMetadata::new(bucket))
.await
.unwrap();
}
#[tokio::test]
async fn test_apply_cors_headers_credentialed_request_with_wildcard_origin() {
if metadata_sys::get_global_bucket_metadata_sys().is_none() {
if get_global_bucket_metadata_sys().is_none() {
eprintln!("Skipping test: GLOBAL_BucketMetadataSys not initialized");
return;
}
@@ -1584,7 +1582,7 @@ mod tests {
max_age_seconds: None,
}],
});
metadata_sys::set_bucket_metadata(bucket.to_string(), bm).await.unwrap();
set_bucket_metadata(bucket.to_string(), bm).await.unwrap();
let mut headers = HeaderMap::new();
headers.insert(cors::standard::ORIGIN, "https://console.localhost".parse().unwrap());
@@ -1600,7 +1598,7 @@ mod tests {
assert_eq!(result.get(cors::response::ACCESS_CONTROL_ALLOW_CREDENTIALS).unwrap(), "true");
assert_eq!(result.get(cors::standard::VARY).unwrap(), "Origin");
metadata_sys::set_bucket_metadata(bucket.to_string(), BucketMetadata::new(bucket))
set_bucket_metadata(bucket.to_string(), BucketMetadata::new(bucket))
.await
.unwrap();
}
@@ -1783,7 +1781,7 @@ mod tests {
/// with a single-element vec value, matching the format expected by policy evaluation.
#[test]
fn test_object_tag_condition_key_format() {
use crate::storage::storage_compat::tagging::decode_tags_to_map;
use crate::storage::storage_compat::decode_tags_to_map;
use std::collections::HashMap;
let tags_str = "security=public&project=webapp&env=prod";
rustfs/src/storage/options.rs
+1 -1
View File
@@ -12,9 +12,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::storage::storage_compat::BucketVersioningSys;
use crate::storage::storage_compat::Result;
use crate::storage::storage_compat::StorageError;
use crate::storage::storage_compat::versioning_sys::BucketVersioningSys;
use http::header::{IF_MATCH, IF_NONE_MATCH};
use http::{HeaderMap, HeaderValue};
use rustfs_utils::http::{
rustfs/src/storage/rpc/bucket.rs
+1 -1
View File
@@ -51,7 +51,7 @@ impl NodeService {
match load_bucket_metadata(store, &bucket).await {
Ok(meta) => {
if let Err(err) = metadata_sys::set_bucket_metadata(bucket, meta).await {
if let Err(err) = set_bucket_metadata(bucket, meta).await {
return Ok(Response::new(LoadBucketMetadataResponse {
success: false,
error_info: Some(err.to_string()),
rustfs/src/storage/rpc/node_service.rs
+3 -3
View File
@@ -20,8 +20,8 @@ use crate::storage::storage_compat::{
CollectMetricsOpts, DeleteOptions, DiskAPI, DiskError, DiskInfoOptions, DiskStore, FileInfoVersions, GLOBAL_TierConfigMgr,
LocalPeerS3Client, MetricType, PEER_RESTSIGNAL, PEER_RESTSUB_SYS, PeerS3Client, ReadMultipleReq, ReadMultipleResp,
ReadOptions, SERVICE_SIGNAL_REFRESH_CONFIG, SERVICE_SIGNAL_RELOAD_DYNAMIC, UpdateMetadataOpts, all_local_disk_path,
collect_local_metrics, find_local_disk_by_ref, get_global_lock_client, get_local_server_property,
metadata::load_bucket_metadata, metadata_sys, resolve_object_store_handle,
collect_local_metrics, find_local_disk_by_ref, get_global_lock_client, get_local_server_property, load_bucket_metadata,
resolve_object_store_handle, set_bucket_metadata,
};
use bytes::Bytes;
use futures::Stream;
@@ -2697,7 +2697,7 @@ mod tests {
vars.insert(PEER_RESTSIGNAL.to_string(), SERVICE_SIGNAL_RELOAD_DYNAMIC.to_string());
vars.insert(
PEER_RESTSUB_SYS.to_string(),
crate::storage::storage_compat::com::STORAGE_CLASS_SUB_SYS.to_string(),
crate::storage::storage_compat::STORAGE_CLASS_SUB_SYS.to_string(),
);
let request = Request::new(SignalServiceRequest {
rustfs/src/storage/s3_api/bucket.rs
+1 -1
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::storage::s3_api::common::rustfs_owner;
use crate::storage::storage_compat::object_api_utils::to_s3s_etag;
use crate::storage::storage_compat::to_s3s_etag;
use percent_encoding::percent_decode_str;
use rustfs_storage_api::{
BucketInfo, ListObjectVersionsInfo as StorageListObjectVersionsInfo, ListObjectsV2Info as StorageListObjectsV2Info,
rustfs/src/storage/s3_api/multipart.rs
+2 -2
View File
@@ -13,7 +13,7 @@
// limitations under the License.
use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner};
use crate::storage::storage_compat::object_api_utils::to_s3s_etag;
use crate::storage::storage_compat::to_s3s_etag;
use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo};
use s3s::dto::{CommonPrefix, ListMultipartUploadsOutput, ListPartsOutput, MultipartUpload, Part, Timestamp};
use s3s::{S3Error, S3ErrorCode};
@@ -191,7 +191,7 @@ mod tests {
parse_list_multipart_uploads_params, parse_list_parts_params, parse_upload_part_number,
};
use crate::storage::s3_api::common::{rustfs_initiator, rustfs_owner};
use crate::storage::storage_compat::object_api_utils::to_s3s_etag;
use crate::storage::storage_compat::to_s3s_etag;
use rustfs_storage_api::{ListMultipartsInfo, ListPartsInfo, MultipartInfo, PartInfo};
use s3s::S3ErrorCode;
use s3s::dto::Timestamp;
rustfs/src/storage/sse.rs
+2 -2
View File
@@ -135,7 +135,7 @@ const OBJECT_KEY_DERIVATION_CONTEXT: &[u8] = b"object-encryption-key generation"
use crate::error::ApiError;
use crate::storage::storage_compat::Error;
use crate::storage::storage_compat::metadata_sys;
use crate::storage::storage_compat::get_bucket_sse_config;
use rustfs_utils::http::headers::{
AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_ALGORITHM, AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_KEY,
AMZ_SERVER_SIDE_ENCRYPTION_CUSTOMER_KEY_MD5, AMZ_SERVER_SIDE_ENCRYPTION_KMS_CONTEXT,
@@ -219,7 +219,7 @@ async fn prepare_sse_configuration(
}
// Get bucket default encryption configuration.
let bucket_sse_config_result = metadata_sys::get_sse_config(bucket).await;
let bucket_sse_config_result = get_bucket_sse_config(bucket).await;
debug!(
component = LOG_COMPONENT_STORAGE,
subsystem = LOG_SUBSYSTEM_SSE,
rustfs/src/storage/storage_compat.rs
+29 -5
View File
@@ -13,12 +13,36 @@
// limitations under the License.
pub(crate) use rustfs_ecstore::api::admin::get_local_server_property;
pub(crate) use rustfs_ecstore::api::bucket::{
metadata, metadata_sys, object_lock, policy_sys, replication, tagging, utils, versioning, versioning_sys,
};
pub(crate) use rustfs_ecstore::api::client::object_api_utils;
#[cfg(test)]
pub(crate) use rustfs_ecstore::api::config::com;
pub(crate) use rustfs_ecstore::api::bucket::metadata::BucketMetadata;
pub(crate) use rustfs_ecstore::api::bucket::metadata::{
BUCKET_ACCELERATE_CONFIG, BUCKET_LOGGING_CONFIG, BUCKET_REQUEST_PAYMENT_CONFIG, BUCKET_VERSIONING_CONFIG,
BUCKET_WEBSITE_CONFIG, OBJECT_LOCK_CONFIG, load_bucket_metadata,
};
#[cfg(test)]
pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{
GLOBAL_BucketMetadataSys as GLOBAL_BUCKET_METADATA_SYS, get_global_bucket_metadata_sys,
};
pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{
delete as delete_bucket_metadata_config, get as get_bucket_metadata, get_accelerate_config as get_bucket_accelerate_config,
get_bucket_policy_raw, get_cors_config as get_bucket_cors_config, get_logging_config as get_bucket_logging_config,
get_object_lock_config as get_bucket_object_lock_config, get_public_access_block_config,
get_replication_config as get_bucket_replication_config, get_request_payment_config as get_bucket_request_payment_config,
get_sse_config as get_bucket_sse_config, get_website_config as get_bucket_website_config, set_bucket_metadata,
update as update_bucket_metadata_config,
};
pub(crate) use rustfs_ecstore::api::bucket::object_lock::objectlock_sys::{
add_years as add_object_lock_years, check_retention_for_modification,
};
pub(crate) use rustfs_ecstore::api::bucket::policy_sys::PolicySys;
pub(crate) use rustfs_ecstore::api::bucket::replication::{GLOBAL_REPLICATION_STATS, ReplicationConfigurationExt};
pub(crate) use rustfs_ecstore::api::bucket::tagging::{decode_tags, decode_tags_to_map, encode_tags};
pub(crate) use rustfs_ecstore::api::bucket::utils::serialize;
pub(crate) use rustfs_ecstore::api::bucket::versioning::VersioningApi;
pub(crate) use rustfs_ecstore::api::bucket::versioning_sys::BucketVersioningSys;
pub(crate) use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag;
#[cfg(test)]
pub(crate) use rustfs_ecstore::api::config::com::STORAGE_CLASS_SUB_SYS;
pub(crate) use rustfs_ecstore::api::disk::error::DiskError;
pub(crate) use rustfs_ecstore::api::disk::{
DeleteOptions, DiskAPI, DiskInfoOptions, DiskStore, FileInfoVersions, ReadMultipleReq, ReadMultipleResp, ReadOptions,
scripts/check_architecture_migration_rules.sh
+11
View File
@@ -81,6 +81,7 @@ RUSTFS_RUNTIME_SECONDARY_STORAGE_COMPAT_HITS_FILE="${TMP_DIR}/rustfs_runtime_sec
RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_bucket_storage_compat_module_hits.txt"
RUSTFS_ROOT_RUNTIME_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_runtime_storage_compat_module_hits.txt"
RUSTFS_ADMIN_CONFIG_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_admin_config_storage_compat_module_hits.txt"
RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_storage_bucket_storage_compat_module_hits.txt"
PRODUCTION_UNUSED_COMPAT_ALLOW_HITS_FILE="${TMP_DIR}/production_unused_compat_allow_hits.txt"
BROAD_STORE_API_COMPAT_REEXPORT_HITS_FILE="${TMP_DIR}/broad_store_api_compat_reexport_hits.txt"
NESTED_STORE_API_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/nested_store_api_compat_module_hits.txt"
@@ -798,6 +799,16 @@ if [[ -s "$RUSTFS_ADMIN_CONFIG_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then
report_failure "RustFS admin storage compatibility must expose config IO and default initialization as explicit aliases: $(paste -sd '; ' "$RUSTFS_ADMIN_CONFIG_STORAGE_COMPAT_MODULE_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::bucket::\{[^}]*\b(?:metadata|metadata_sys|object_lock|policy_sys|replication|tagging|utils|versioning|versioning_sys)\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::bucket::(?:metadata|metadata_sys|object_lock|policy_sys|replication|tagging|utils|versioning|versioning_sys)\s*;|pub\(crate\)\s+use rustfs_ecstore::api::client::object_api_utils\s*;|pub\(crate\)\s+use rustfs_ecstore::api::config::com\s*;' \
rustfs/src/storage/storage_compat.rs || true
) >"$RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE"
if [[ -s "$RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then
report_failure "RustFS storage compatibility must expose bucket/object-api/config contracts as explicit aliases: $(paste -sd '; ' "$RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE")"
fi
(
cd "$ROOT_DIR"
rg -n --no-heading '#!\[allow\(unused_imports\)\]' \