From 9d510e9d5b2facf80cf8a8055cb148f962729058 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=AE=89=E6=AD=A3=E8=B6=85?= Date: Sun, 21 Jun 2026 16:40:40 +0800 Subject: [PATCH] refactor: prune admin app compat aliases (#3690) --- docs/architecture/crate-boundaries.md | 4 + docs/architecture/migration-progress.md | 48 +++++-- rustfs/src/admin/storage_compat.rs | 73 ++++++++++- rustfs/src/app/storage_compat.rs | 120 +++++++++++++++++- scripts/check_architecture_migration_rules.sh | 22 ++++ 5 files changed, 243 insertions(+), 24 deletions(-) diff --git a/docs/architecture/crate-boundaries.md b/docs/architecture/crate-boundaries.md index 6911eda28..41f62bee4 100644 --- a/docs/architecture/crate-boundaries.md +++ b/docs/architecture/crate-boundaries.md @@ -152,6 +152,10 @@ bare `init`, or grouped `endpoint::Endpoint` passthroughs. RustFS admin `storage_compat.rs` must expose config IO and default initialization through explicit aliases. The admin compatibility boundary must not restore broad `com` or bare `init` passthroughs. +RustFS admin and app `storage_compat.rs` bucket-facing compatibility contracts +must stay explicitly whitelisted. They must not restore broad bucket module, +client object API, client transition API, or storage-class module passthroughs +once a local compatibility boundary has narrowed them to specific aliases. RustFS storage `storage_compat.rs` must expose bucket metadata, object-lock, policy, replication, tagging, versioning, object API, and test-only storage-class config contracts through explicit aliases. The storage diff --git a/docs/architecture/migration-progress.md b/docs/architecture/migration-progress.md index 7c9215138..78d4d6752 100644 --- a/docs/architecture/migration-progress.md +++ b/docs/architecture/migration-progress.md @@ -5,17 +5,16 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block ## Current Context - Issue: [`rustfs/backlog#660`](https://github.com/rustfs/backlog/issues/660) -- Branch: `overtrue/arch-storage-runtime-compat-aliases` -- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081`. -- Stacked on: API-081 admin config compatibility alias pruning. +- Branch: `overtrue/arch-admin-app-compat-aliases` +- Baseline: completed `C-011/C-012/C-013/API-055/API-059/API-079/API-080/API-081/API-082`. +- Stacked on: API-082 storage compatibility alias pruning. - PR type for this branch: `pure-move` - Runtime behavior changes: none. -- Rust code changes: prune storage bucket metadata, object-lock, policy, - replication, tagging, versioning, object API, and test-only config - passthroughs into explicit aliases for storage runtime modules and tests. -- CI/script changes: guard against restoring broad storage bucket/object-api - compatibility module passthroughs. -- Docs changes: record the API-082 storage compatibility alias boundary. +- Rust code changes: prune admin/app bucket-facing compatibility passthroughs + into explicit local compatibility modules and aliases. +- CI/script changes: guard against restoring broad admin/app bucket, client, + and storage-class compatibility passthroughs. +- Docs changes: record the API-083 admin/app compatibility alias boundary. ## Phase 0 Tasks @@ -173,6 +172,21 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block - Verification: RustFS compile coverage, storage compatibility residual scan, formatting, diff hygiene, risk scan, architecture guard, pre-commit quality gate, and three-expert review. +- [x] `API-083` Prune admin/app bucket compatibility aliases. + - Completed slice: replace admin and app broad bucket/client/storage-class + compatibility passthroughs with explicit local compatibility modules and + symbol whitelists. + - Acceptance: admin replication, bucket metadata, quota, tier, site + replication, router, and app bucket/object/multipart/lifecycle consumers + keep their existing call paths while `storage_compat.rs` no longer exposes + broad upstream modules. + - Must preserve: admin bucket target updates, replication status and resync + DTOs, site-replication metadata serialization, quota checks, lifecycle + transition hooks, bucket metadata IO, object ETag conversion, object-lock + checks, and app storage-class behavior. + - Verification: RustFS compile coverage, admin/app compatibility residual + scans, formatting, diff hygiene, risk scan, architecture guard, pre-commit + quality gate, and three-expert review. - [x] `G-012` Inventory placement and repair invariants. - Acceptance: [`placement-repair-invariants.md`](placement-repair-invariants.md) records @@ -3206,14 +3220,24 @@ Status values: `[ ]` not started, `[~]` in progress, `[x]` complete, `[!]` block | Expert | Status | Notes | |---|---|---| -| Quality/architecture | passed | API-082 narrows RustFS storage bucket, object API, and config compatibility to explicit aliases without adding new ECStore ownership cycles. | -| Migration preservation | passed | Bucket metadata read/update/delete, object-lock checks, policy/public-access reads, SSE defaults, replication/tagging/versioning access, S3 ETag conversion, RPC metadata load, and test-only storage-class access remain preserved. | -| Testing/verification | passed | RustFS compile coverage, storage compatibility residual scans, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. | +| Quality/architecture | passed | API-083 narrows admin/app bucket-facing compatibility passthroughs into explicit local aliases without adding new ECStore ownership cycles. | +| Migration preservation | passed | Admin replication, site-replication metadata, bucket targets, quota, tier stats, app bucket/object/multipart/lifecycle, ETag conversion, and storage-class behavior remain preserved. | +| Testing/verification | passed | RustFS compile coverage, admin/app compatibility residual scans, migration guard, formatting, diff hygiene, added-line risk scan, full pre-commit, and three-expert review passed. | ## Verification Notes Passed before push: +- Issue #660 API-083 current slice: + - `cargo check -p rustfs --lib`: passed. + - `cargo fmt --all --check`: passed. + - `git diff --check`: passed. + - `bash -n scripts/check_architecture_migration_rules.sh`: passed. + - `./scripts/check_architecture_migration_rules.sh`: passed. + - Admin/app broad compatibility export scans: passed. + - Rust added-line risk scan on changed Rust files and guard script: passed. + - `make pre-commit`: passed. + - Issue #660 API-082 current slice: - `cargo check -p rustfs --lib`: passed. - `cargo fmt --all --check`: passed. diff --git a/rustfs/src/admin/storage_compat.rs b/rustfs/src/admin/storage_compat.rs index 84790f6a6..9fb9b8ed2 100644 --- a/rustfs/src/admin/storage_compat.rs +++ b/rustfs/src/admin/storage_compat.rs @@ -12,11 +12,69 @@ // See the License for the specific language governing permissions and // limitations under the License. +pub(crate) mod bandwidth { + pub(crate) mod monitor { + pub(crate) use rustfs_ecstore::api::bucket::bandwidth::monitor::BandwidthDetails; + } +} +pub(crate) mod bucket_target_sys { + pub(crate) use rustfs_ecstore::api::bucket::bucket_target_sys::{ + AdvancedPutOptions, BucketTargetError, BucketTargetSys, PutObjectOptions, RemoveObjectOptions, S3ClientError, + TargetClient, + }; +} +pub(crate) mod lifecycle { + pub(crate) mod bucket_lifecycle_ops { + pub(crate) use rustfs_ecstore::api::bucket::lifecycle::bucket_lifecycle_ops::GLOBAL_TransitionState; + } + + pub(crate) mod tier_last_day_stats { + #[cfg(test)] + pub(crate) use rustfs_ecstore::api::bucket::lifecycle::tier_last_day_stats::LastDayTierStats; + } +} +pub(crate) mod metadata { + pub(crate) use rustfs_ecstore::api::bucket::metadata::{ + BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, BUCKET_QUOTA_CONFIG_FILE, + BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, + BucketMetadata, OBJECT_LOCK_CONFIG, table_catalog_path_hash, + }; +} +pub(crate) mod metadata_sys { + pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{ + BucketMetadataSys, delete, get, get_bucket_policy, get_bucket_targets_config, get_config_from_disk, get_lifecycle_config, + get_notification_config, get_object_lock_config, get_quota_config, get_replication_config, get_sse_config, + get_tagging_config, get_versioning_config, list_bucket_targets, update, + }; +} +pub(crate) mod quota { + pub(crate) mod checker { + pub(crate) use rustfs_ecstore::api::bucket::quota::checker::QuotaChecker; + } + + pub(crate) use rustfs_ecstore::api::bucket::quota::{BucketQuota, QuotaError, QuotaOperation}; +} +pub(crate) mod replication { + pub(crate) use rustfs_ecstore::api::bucket::replication::{ + BucketReplicationResyncStatus, BucketStats, GLOBAL_REPLICATION_STATS, ObjectOpts, ReplicationConfigurationExt, + ResyncOpts, get_global_replication_pool, + }; + #[cfg(test)] + pub(crate) use rustfs_ecstore::api::bucket::replication::{ResyncStatusType, TargetReplicationResyncStatus}; +} +pub(crate) mod target { + pub(crate) use rustfs_ecstore::api::bucket::target::{ARN, BucketTarget, BucketTargetType, BucketTargets, Credentials}; +} +pub(crate) mod utils { + pub(crate) use rustfs_ecstore::api::bucket::utils::{deserialize, is_valid_object_prefix, serialize}; +} +pub(crate) mod versioning { + pub(crate) use rustfs_ecstore::api::bucket::versioning::VersioningApi; +} +pub(crate) mod versioning_sys { + pub(crate) use rustfs_ecstore::api::bucket::versioning_sys::BucketVersioningSys; +} pub(crate) use rustfs_ecstore::api::bucket::lifecycle::tier_last_day_stats::DailyAllTierStats; -pub(crate) use rustfs_ecstore::api::bucket::{ - bandwidth, bucket_target_sys, lifecycle, metadata, metadata_sys, quota, replication, target, utils, versioning, - versioning_sys, -}; pub(crate) use rustfs_ecstore::api::capacity::is_reserved_or_invalid_bucket; pub(crate) use rustfs_ecstore::api::client::admin_handler_utils::AdminError; pub(crate) use rustfs_ecstore::api::config::com::{ @@ -24,7 +82,12 @@ pub(crate) use rustfs_ecstore::api::config::com::{ read_config_without_migrate as read_admin_config_without_migrate, save_config as save_admin_config, save_server_config as save_admin_server_config, }; -pub(crate) use rustfs_ecstore::api::config::{init as init_admin_config_defaults, set_global_storage_class, storageclass}; +pub(crate) mod storageclass { + pub(crate) use rustfs_ecstore::api::config::storageclass::{ + INLINE_BLOCK_ENV, OPTIMIZE_ENV, RRS, RRS_ENV, STANDARD, STANDARD_ENV, lookup_config, + }; +} +pub(crate) use rustfs_ecstore::api::config::{init as init_admin_config_defaults, set_global_storage_class}; pub(crate) use rustfs_ecstore::api::data_usage::load_data_usage_from_backend; pub(crate) use rustfs_ecstore::api::disk::RUSTFS_META_BUCKET; #[cfg(test)] diff --git a/rustfs/src/app/storage_compat.rs b/rustfs/src/app/storage_compat.rs index 52befa42e..3adb87564 100644 --- a/rustfs/src/app/storage_compat.rs +++ b/rustfs/src/app/storage_compat.rs @@ -13,18 +13,124 @@ // limitations under the License. pub(crate) use rustfs_ecstore::api::admin::get_server_info; -pub(crate) use rustfs_ecstore::api::bucket::{ - bucket_target_sys, lifecycle, metadata, metadata_sys, object_lock, policy_sys, quota, replication, tagging, target, utils, - versioning, versioning_sys, -}; +pub(crate) mod bucket_target_sys { + pub(crate) use rustfs_ecstore::api::bucket::bucket_target_sys::BucketTargetSys; +} +pub(crate) mod lifecycle { + pub(crate) mod bucket_lifecycle_audit { + pub(crate) use rustfs_ecstore::api::bucket::lifecycle::bucket_lifecycle_audit::LcEventSrc; + } + + pub(crate) mod bucket_lifecycle_ops { + #[cfg(test)] + pub(crate) use rustfs_ecstore::api::bucket::lifecycle::bucket_lifecycle_ops::init_background_expiry; + pub(crate) use rustfs_ecstore::api::bucket::lifecycle::bucket_lifecycle_ops::{ + GLOBAL_ExpiryState, RestoreRequestOps, enqueue_expiry_for_existing_objects, enqueue_transition_for_existing_objects, + enqueue_transition_immediate, post_restore_opts, validate_transition_tier, + }; + } + + pub(crate) mod lifecycle_contract { + #[cfg(test)] + pub(crate) use rustfs_ecstore::api::bucket::lifecycle::lifecycle::IlmAction; + pub(crate) use rustfs_ecstore::api::bucket::lifecycle::lifecycle::{ + Event, Lifecycle, ObjectOpts, TRANSITION_COMPLETE, TransitionOptions, expected_expiry_time, + }; + } + pub(crate) use lifecycle_contract as lifecycle; + + pub(crate) mod tier_delete_journal { + pub(crate) use rustfs_ecstore::api::bucket::lifecycle::tier_delete_journal::persist_tier_delete_journal_entry; + } + + pub(crate) mod tier_sweeper { + pub(crate) use rustfs_ecstore::api::bucket::lifecycle::tier_sweeper::{ + transitioned_delete_journal_entry, transitioned_force_delete_journal_entry, + }; + } +} +pub(crate) mod metadata { + #[cfg(test)] + pub(crate) use rustfs_ecstore::api::bucket::metadata::OBJECT_LOCK_CONFIG; + pub(crate) use rustfs_ecstore::api::bucket::metadata::{ + BUCKET_CORS_CONFIG, BUCKET_LIFECYCLE_CONFIG, BUCKET_NOTIFICATION_CONFIG, BUCKET_POLICY_CONFIG, + BUCKET_PUBLIC_ACCESS_BLOCK_CONFIG, BUCKET_REPLICATION_CONFIG, BUCKET_SSECONFIG, BUCKET_TAGGING_CONFIG, + BUCKET_TARGETS_FILE, BUCKET_VERSIONING_CONFIG, + }; +} +pub(crate) mod metadata_sys { + #[cfg(test)] + pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::init_bucket_metadata_sys; + pub(crate) use rustfs_ecstore::api::bucket::metadata_sys::{ + BucketMetadataSys, delete, get_bucket_policy, get_bucket_policy_raw, get_bucket_targets_config, get_cors_config, + get_global_bucket_metadata_sys, get_lifecycle_config, get_notification_config, get_object_lock_config, + get_public_access_block_config, get_replication_config, get_sse_config, get_tagging_config, update, + }; +} +pub(crate) mod object_api_utils { + pub(crate) use rustfs_ecstore::api::client::object_api_utils::to_s3s_etag; +} +pub(crate) mod object_lock { + pub(crate) mod objectlock { + pub(crate) use rustfs_ecstore::api::bucket::object_lock::objectlock::{ + get_object_legalhold_meta, get_object_retention_meta, + }; + } + + pub(crate) mod objectlock_sys { + pub(crate) use rustfs_ecstore::api::bucket::object_lock::objectlock_sys::{ + BucketObjectLockSys, check_object_lock_for_deletion, is_retention_active, + }; + } + + pub(crate) use rustfs_ecstore::api::bucket::object_lock::ObjectLockApi; +} +pub(crate) mod policy_sys { + pub(crate) use rustfs_ecstore::api::bucket::policy_sys::PolicySys; +} +pub(crate) mod quota { + pub(crate) mod checker { + pub(crate) use rustfs_ecstore::api::bucket::quota::checker::QuotaChecker; + } + + pub(crate) use rustfs_ecstore::api::bucket::quota::QuotaOperation; +} +pub(crate) mod replication { + pub(crate) use rustfs_ecstore::api::bucket::replication::{ + DeletedObjectReplicationInfo, ObjectOpts, ReplicationConfigurationExt, check_replicate_delete, + get_must_replicate_options, must_replicate, schedule_replication, schedule_replication_delete, + }; +} +pub(crate) mod tagging { + pub(crate) use rustfs_ecstore::api::bucket::tagging::decode_tags; +} +pub(crate) mod target { + #[cfg(test)] + pub(crate) use rustfs_ecstore::api::bucket::target::BucketTarget; + pub(crate) use rustfs_ecstore::api::bucket::target::{BucketTargetType, BucketTargets}; +} +pub(crate) mod utils { + pub(crate) use rustfs_ecstore::api::bucket::utils::serialize; +} +pub(crate) mod versioning { + pub(crate) use rustfs_ecstore::api::bucket::versioning::VersioningApi; +} +pub(crate) mod versioning_sys { + pub(crate) use rustfs_ecstore::api::bucket::versioning_sys::BucketVersioningSys; +} pub(crate) use rustfs_ecstore::api::capacity::{ PoolDecommissionInfo, PoolStatus, get_total_usable_capacity, get_total_usable_capacity_free, }; -pub(crate) use rustfs_ecstore::api::client::object_api_utils; #[cfg(test)] -pub(crate) use rustfs_ecstore::api::client::transition_api; +pub(crate) mod transition_api { + pub(crate) use rustfs_ecstore::api::client::transition_api::{ReadCloser, ReaderImpl}; +} pub(crate) use rustfs_ecstore::api::compression::{MIN_DISK_COMPRESSIBLE_SIZE, is_disk_compressible}; -pub(crate) use rustfs_ecstore::api::config::storageclass; +pub(crate) mod storageclass { + pub(crate) use rustfs_ecstore::api::config::storageclass::STANDARD; + #[cfg(test)] + pub(crate) use rustfs_ecstore::api::config::storageclass::STANDARD_IA; +} pub(crate) use rustfs_ecstore::api::data_usage::{ apply_bucket_usage_memory_overlay, load_data_usage_from_backend, record_bucket_object_delete_memory, record_bucket_object_write_memory, diff --git a/scripts/check_architecture_migration_rules.sh b/scripts/check_architecture_migration_rules.sh index 74fef66bc..0ebcd5956 100755 --- a/scripts/check_architecture_migration_rules.sh +++ b/scripts/check_architecture_migration_rules.sh @@ -82,6 +82,8 @@ RUSTFS_ROOT_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_bucke RUSTFS_ROOT_RUNTIME_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_root_runtime_storage_compat_module_hits.txt" RUSTFS_ADMIN_CONFIG_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_admin_config_storage_compat_module_hits.txt" RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_storage_bucket_storage_compat_module_hits.txt" +RUSTFS_ADMIN_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_admin_bucket_storage_compat_module_hits.txt" +RUSTFS_APP_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/rustfs_app_bucket_storage_compat_module_hits.txt" PRODUCTION_UNUSED_COMPAT_ALLOW_HITS_FILE="${TMP_DIR}/production_unused_compat_allow_hits.txt" BROAD_STORE_API_COMPAT_REEXPORT_HITS_FILE="${TMP_DIR}/broad_store_api_compat_reexport_hits.txt" NESTED_STORE_API_COMPAT_MODULE_HITS_FILE="${TMP_DIR}/nested_store_api_compat_module_hits.txt" @@ -809,6 +811,26 @@ if [[ -s "$RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then report_failure "RustFS storage compatibility must expose bucket/object-api/config contracts as explicit aliases: $(paste -sd '; ' "$RUSTFS_STORAGE_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE")" fi +( + cd "$ROOT_DIR" + rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::bucket::\{[^}]*\b(?:bandwidth|bucket_target_sys|lifecycle|metadata|metadata_sys|quota|replication|target|utils|versioning|versioning_sys)\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::bucket::(?:bandwidth|bucket_target_sys|lifecycle|metadata|metadata_sys|quota|replication|target|utils|versioning|versioning_sys)\s*;|pub\(crate\)\s+use rustfs_ecstore::api::config::\{[^}]*\bstorageclass\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::config::storageclass\s*;' \ + rustfs/src/admin/storage_compat.rs || true +) >"$RUSTFS_ADMIN_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" + +if [[ -s "$RUSTFS_ADMIN_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then + report_failure "RustFS admin storage compatibility must expose bucket/storageclass contracts as explicit aliases: $(paste -sd '; ' "$RUSTFS_ADMIN_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE")" +fi + +( + cd "$ROOT_DIR" + rg -n --no-heading 'pub\(crate\)\s+use rustfs_ecstore::api::bucket::\{[^}]*\b(?:bucket_target_sys|lifecycle|metadata|metadata_sys|object_lock|policy_sys|quota|replication|tagging|target|utils|versioning|versioning_sys)\b[^}]*\}\s*;|pub\(crate\)\s+use rustfs_ecstore::api::bucket::(?:bucket_target_sys|lifecycle|metadata|metadata_sys|object_lock|policy_sys|quota|replication|tagging|target|utils|versioning|versioning_sys)\s*;|pub\(crate\)\s+use rustfs_ecstore::api::client::(?:object_api_utils|transition_api)\s*;|pub\(crate\)\s+use rustfs_ecstore::api::config::storageclass\s*;' \ + rustfs/src/app/storage_compat.rs || true +) >"$RUSTFS_APP_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" + +if [[ -s "$RUSTFS_APP_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE" ]]; then + report_failure "RustFS app storage compatibility must expose bucket/client/storageclass contracts as explicit aliases: $(paste -sd '; ' "$RUSTFS_APP_BUCKET_STORAGE_COMPAT_MODULE_HITS_FILE")" +fi + ( cd "$ROOT_DIR" rg -n --no-heading '#!\[allow\(unused_imports\)\]' \