gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-28 07:57:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(replication): own filemeta wire contracts (#4254)

Browse Source
This commit is contained in:
Zhengchao An
2026-07-04 06:39:54 +08:00
committed by GitHub
parent 09fcacbe2b
commit 3d4fb532e5
22 changed files with 1196 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/ecstore/src/bucket/replication/README.md
+10 -11
View File
@@ -37,11 +37,11 @@ paths.
| `EcstoreReplicationBoundaryImports` | ECStore-side imports from `rustfs-replication`. | Direct `rustfs-replication` imports under `crates/ecstore/src/bucket/replication` stay in `*_boundary.rs` modules, including config and resync facade re-exports. |
| `RuntimeReplicationFacadeConsumers` | Runtime owner consumers of replication DTOs and status types. | Scanner, admin, and storage owner facades import replication DTOs/status types through `rustfs-ecstore`; app storage keeps the remaining direct object/delete helper calls behind its local storage API boundary. |
| `ReplicationResyncContracts` | Resync options, target status, bucket status, status classifiers, and persisted resync/MRF status wire format. | Owned by `crates/replication`; ECStore imports them through `replication_resync_boundary.rs`, which maps crate errors to ECStore errors. |
| `ReplicationCrateFileMetaFacade` | Replication facade compatibility symbols that still originate in filemeta wire contracts. | `crates/replication/src/filemeta.rs` is the only direct `rustfs-filemeta` import boundary inside `rustfs-replication`. |
| `ReplicationCrateFileMetaIndependence` | Replication status, decision, MRF, resync, and target-reset wire contracts owned by `rustfs-replication`. | `crates/replication/src/filemeta.rs` owns these contracts; `rustfs-replication` must not import or depend on `rustfs-filemeta`. |
| `ReplicationConfigStore` | Replication config persistence and config-derived labels used by target options. | Config read/save helpers and storage class labels are exposed through the contract type in `replication_config_store.rs`. |
| `ReplicationFileMeta` | Replication status, decisions, MRF entries, resync decisions, and target reset helpers. | `rustfs_filemeta` replication contracts are concentrated in `replication_filemeta_boundary.rs`; `FileInfo` remains in the storage boundary for storage trait bindings and walk options. |
| `StorageApiReplicationContracts` | Storage-api delete DTO replication state/status helpers. | The temporary `rustfs-filemeta` dependency is isolated in `crates/storage-api/src/replication.rs` until the wire contracts can move without creating a `rustfs-replication` / `rustfs-storage-api` cycle. |
| `ReplicationCrateStorageApiBoundary` | Storage API delete DTOs consumed by `rustfs-replication`. | `crates/replication/src/storage_api.rs` is the only direct `rustfs-storage-api` import boundary inside `rustfs-replication`. |
| `ReplicationFileMeta` | ECStore compatibility conversions for filemeta replication state/status. | `rustfs_filemeta` to `rustfs_replication` conversions are concentrated in `replication_filemeta_boundary.rs`; `FileInfo` remains in the storage boundary for storage trait bindings and walk options. |
| `StorageApiReplicationContracts` | Storage-api delete DTO replication state/status helpers. | Storage-api owner DTOs keep their local replication boundary; ECStore converts them in `replication_storage_boundary.rs` before queueing replication work. |
| `ReplicationCrateStorageApiIndependence` | Delete work DTOs consumed by `rustfs-replication`. | `crates/replication/src/storage_api.rs` owns these DTOs; `rustfs-replication` must not import or depend on `rustfs-storage-api`. |
| `ReplicationObjectDecisionContracts` | Object replication options, delete replication decisions, resync target projection, multipart planning, and delete-marker retry classifiers. | Owned by `crates/replication`; ECStore imports them through `replication_object_decision_boundary.rs`. |
| `ReplicationQueueContracts` | Queue admission, heal queue results/actions, worker operations, worker sizing, and backpressure decisions. | Owned by `crates/replication`; ECStore imports them through `replication_queue_boundary.rs`. |
| `ReplicationStatsContracts` | Bucket stats, replication target stats, queue/proxy metrics, and worker metric snapshots. | Owned by `crates/replication`; ECStore imports them through `replication_stats_boundary.rs`. |
@@ -88,13 +88,12 @@ paths.
10. Keep ECStore owner modules outside `bucket/replication` behind bridge
contracts when they need replication codec or config helper behavior.
11. Keep storage-api replication status/state helpers behind
`crates/storage-api/src/replication.rs` until the underlying wire contracts
can move without a `rustfs-replication` / `rustfs-storage-api` dependency
cycle.
12. Keep direct `rustfs-filemeta` imports inside `rustfs-replication`
concentrated in `crates/replication/src/filemeta.rs`.
13. Keep direct `rustfs-storage-api` imports inside `rustfs-replication`
concentrated in `crates/replication/src/storage_api.rs`.
`crates/storage-api/src/replication.rs`; ECStore converts owner DTOs at the
replication storage boundary.
12. Keep `rustfs-replication` independent from `rustfs-filemeta`; ECStore
compatibility conversions live in `replication_filemeta_boundary.rs`.
13. Keep `rustfs-replication` independent from `rustfs-storage-api`; ECStore
compatibility conversions live in `replication_storage_boundary.rs`.
14. Keep direct `rustfs-replication` imports inside ECStore replication
concentrated in `*_boundary.rs` modules.
15. Keep scanner, admin, and storage-owner replication status/DTO consumers
crates/ecstore/src/bucket/replication/mod.rs
+5 -1
View File
@@ -52,7 +52,11 @@ pub(crate) use replication_filemeta_boundary::ReplicateTargetDecision;
pub(crate) use replication_filemeta_boundary::version_purge_statuses_map;
pub use replication_filemeta_boundary::{
REPLICATE_INCOMING_DELETE, ReplicateDecision, ReplicateObjectInfo, ReplicationState, ReplicationStatusType, ReplicationType,
VersionPurgeStatusType, replication_statuses_map,
VersionPurgeStatusType, replication_state_to_filemeta, replication_status_to_filemeta, replication_statuses_map,
version_purge_status_to_filemeta,
};
pub(crate) use replication_filemeta_boundary::{
replication_state_from_filemeta, replication_status_from_filemeta, version_purge_status_from_filemeta,
};
pub(crate) use replication_lifecycle_bridge::{ReplicationLifecycleBridge, ReplicationLifecycleConfig};
pub(crate) use replication_migration_bridge::ReplicationMigrationBridge;
crates/ecstore/src/bucket/replication/replication_filemeta_boundary.rs
+62
View File
@@ -22,3 +22,65 @@ pub use rustfs_replication::{
REPLICATE_INCOMING_DELETE, ReplicateDecision, ReplicateObjectInfo, ReplicationState, ReplicationStatusType, ReplicationType,
VersionPurgeStatusType, replication_statuses_map,
};
pub(crate) fn replication_status_from_filemeta(status: rustfs_filemeta::ReplicationStatusType) -> ReplicationStatusType {
ReplicationStatusType::from(status.as_str())
}
pub(crate) fn version_purge_status_from_filemeta(status: rustfs_filemeta::VersionPurgeStatusType) -> VersionPurgeStatusType {
VersionPurgeStatusType::from(status.as_str())
}
pub(crate) fn replication_state_from_filemeta(state: &rustfs_filemeta::ReplicationState) -> ReplicationState {
ReplicationState {
replica_timestamp: state.replica_timestamp,
replica_status: replication_status_from_filemeta(state.replica_status.clone()),
delete_marker: state.delete_marker,
replication_timestamp: state.replication_timestamp,
replication_status_internal: state.replication_status_internal.clone(),
version_purge_status_internal: state.version_purge_status_internal.clone(),
replicate_decision_str: state.replicate_decision_str.clone(),
targets: state
.targets
.iter()
.map(|(arn, status)| (arn.clone(), replication_status_from_filemeta(status.clone())))
.collect(),
purge_targets: state
.purge_targets
.iter()
.map(|(arn, status)| (arn.clone(), version_purge_status_from_filemeta(status.clone())))
.collect(),
reset_statuses_map: state.reset_statuses_map.clone(),
}
}
pub fn replication_status_to_filemeta(status: ReplicationStatusType) -> rustfs_filemeta::ReplicationStatusType {
rustfs_filemeta::ReplicationStatusType::from(status.as_str())
}
pub fn version_purge_status_to_filemeta(status: VersionPurgeStatusType) -> rustfs_filemeta::VersionPurgeStatusType {
rustfs_filemeta::VersionPurgeStatusType::from(status.as_str())
}
pub fn replication_state_to_filemeta(state: &ReplicationState) -> rustfs_filemeta::ReplicationState {
rustfs_filemeta::ReplicationState {
replica_timestamp: state.replica_timestamp,
replica_status: replication_status_to_filemeta(state.replica_status.clone()),
delete_marker: state.delete_marker,
replication_timestamp: state.replication_timestamp,
replication_status_internal: state.replication_status_internal.clone(),
version_purge_status_internal: state.version_purge_status_internal.clone(),
replicate_decision_str: state.replicate_decision_str.clone(),
targets: state
.targets
.iter()
.map(|(arn, status)| (arn.clone(), replication_status_to_filemeta(status.clone())))
.collect(),
purge_targets: state
.purge_targets
.iter()
.map(|(arn, status)| (arn.clone(), version_purge_status_to_filemeta(status.clone())))
.collect(),
reset_statuses_map: state.reset_statuses_map.clone(),
}
}
crates/ecstore/src/bucket/replication/replication_storage_boundary.rs
+3 -2
View File
@@ -16,6 +16,7 @@ use rustfs_filemeta::FileInfo;
use tokio_util::sync::CancellationToken;
use super::replication_error_boundary::Error;
use super::replication_filemeta_boundary::{replication_state_from_filemeta, version_purge_status_from_filemeta};
pub(crate) type ReplicationObjectStore = crate::store::ECStore;
pub(crate) use crate::client::api_get_options::{AdvancedGetOptions, StatObjectOptions};
pub(crate) use crate::object_api::{GetObjectReader, ObjectInfo, ObjectOptions, PutObjReader};
@@ -101,7 +102,7 @@ pub(crate) fn deleted_object_for_replication(delete_object: DeletedObject) -> Re
object_name: delete_object.object_name,
version_id: delete_object.version_id,
delete_marker_mtime: delete_object.delete_marker_mtime,
replication_state: delete_object.replication_state,
replication_state: delete_object.replication_state.as_ref().map(replication_state_from_filemeta),
found: delete_object.found,
force_delete: delete_object.force_delete,
}
@@ -112,7 +113,7 @@ pub(crate) fn object_to_delete_for_replication(object: &ObjectToDelete) -> Repli
object_name: object.object_name.clone(),
version_id: object.version_id,
delete_marker_replication_status: object.delete_marker_replication_status.clone(),
version_purge_status: object.version_purge_status.clone(),
version_purge_status: object.version_purge_status.clone().map(version_purge_status_from_filemeta),
version_purge_statuses: object.version_purge_statuses.clone(),
replicate_decision_str: object.replicate_decision_str.clone(),
}