mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-19 02:56:18 +00:00
refactor(replication): move resyncer pure decision logic into rustfs-replication (M2) (#6180)
* refactor(replication): move resyncer pure decision logic into rustfs-replication (M2) Pure-move milestone M2 of the ECStore replication split (backlog#1675 P1-17): relocate the resyncer's IO-free decision helpers, with their unit tests, into the crates they already belong to by type ownership. No behavior change. Moved into crates/replication: - resync.rs: resync_status_duration - delete.rs: resync_existing_delete_replication_info, replicate_delete_outcome, target_delete_version_id, delete_marker_purge_version_id, delete_marker_purge_mrf_entry - object.rs: version_identity_drifted, is_replication_target_offline_error, SsecPassthroughCapability, SsecPassthroughGate, ssec_passthrough_gate, ssec_passthrough_evidence_present (param-demoted to the echoed customer-algorithm string; ECStore keeps the HeadObjectOutput adapter) - filemeta.rs: NULL_VERSION_ID wire literal (crate-owned copy per the filemeta-independence contract) ECStore rewiring (Rule #14: imports stay in *_boundary.rs): - resync/object-decision/target boundaries re-export the moved symbols; resyncer call sites are unchanged - bucket_target_sys keeps only the verdict cache + TTL and re-exports the capability enum so existing consumer paths keep compiling Not moved (signatures carry ECStore or aws-sdk types): verify_resync_head_result, resync_target_error_detail, the SdkError classifiers, the replicate_all_* option/info builders, and the env-coupled bounded_resync_max_jobs admission clamp. README milestone table updated. * chore(replication): retire the datatypes.rs relay early README sanctions retiring datatypes.rs ahead of M4. The module was a pure relay (resync boundary -> datatypes -> mod.rs facade) with no external consumer importing it directly, so the facade now re-exports ResyncStatusType from replication_resync_boundary and the relay file is deleted. Consumers stay behind the ECStore facade, keeping Migration Rule #15 intact — the original retirement wording ("consumers import through rustfs-replication directly") conflicted with that rule and is corrected in the README. * chore(arch): extend migration guards to the M2-moved decision contracts The adversarial review of the M2 move found the per-symbol ratchet in check_architecture_migration_rules.sh was not extended for the moved symbols, leaving them free to be redefined in ECStore or imported past their boundary without CI noticing: - resync definition pin + boundary fences gain resync_status_duration; - the object-decision boundary fences gain the five delete-family helpers (delete_marker_purge_mrf_entry, delete_marker_purge_version_id, replicate_delete_outcome, resync_existing_delete_replication_info, target_delete_version_id); - the target-boundary fence gains the SSE-C gate family, the offline classifier, and version_identity_drifted; - a new definition pin rejects ECStore redefinitions of the M2-moved fns/enums (ssec_passthrough_evidence_present deliberately excluded: ECStore keeps a thin HeadObjectOutput adapter under that name). Mutation-verified: a probe fn ssec_passthrough_gate under crates/ecstore/src/bucket/replication trips the new pin. Also anchors the intentionally-duplicated NULL_VERSION_ID wire literal from the filemeta side and tightens the M2 README note on bounded_resync_max_jobs.
This commit is contained in:
唐小鸭
@@ -242,6 +242,7 @@ REPLICATION_RESYNC_CONTRACT_BACKSLIDE_HITS_FILE="${TMP_DIR}/replication_resync_c
|
||||
REPLICATION_RESYNC_BOUNDARY_BYPASS_HITS_FILE="${TMP_DIR}/replication_resync_boundary_bypass_hits.txt"
|
||||
REPLICATION_OBJECT_DECISION_BOUNDARY_BYPASS_HITS_FILE="${TMP_DIR}/replication_object_decision_boundary_bypass_hits.txt"
|
||||
REPLICATION_OBJECT_COMPARE_CONTRACT_BACKSLIDE_HITS_FILE="${TMP_DIR}/replication_object_compare_contract_backslide_hits.txt"
|
||||
REPLICATION_M2_DECISION_CONTRACT_BACKSLIDE_HITS_FILE="${TMP_DIR}/replication_m2_decision_contract_backslide_hits.txt"
|
||||
REPLICATION_MRF_WIRE_FORMAT_BACKSLIDE_HITS_FILE="${TMP_DIR}/replication_mrf_wire_format_backslide_hits.txt"
|
||||
STORAGE_REPLICATION_HANDLE_BOUNDARY_BYPASS_HITS_FILE="${TMP_DIR}/storage_replication_handle_boundary_bypass_hits.txt"
|
||||
STORAGE_REPLICATION_CRATE_BYPASS_HITS_FILE="${TMP_DIR}/storage_replication_crate_bypass_hits.txt"
|
||||
@@ -2981,7 +2982,7 @@ fi
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
replication_resync_status=0
|
||||
rg -n --with-filename '^\s*(?:pub(?:\([^)]*\))?\s+)?(?:(?:struct|enum)\s+(?:ResyncOpts|TargetReplicationResyncStatus|BucketReplicationResyncStatus|ResyncStatusType)|fn\s+(?:resync_state_accepts_update|should_count_head_proxy_failure|should_auto_resume_resync|is_version_id_mismatch))\b' \
|
||||
rg -n --with-filename '^\s*(?:pub(?:\([^)]*\))?\s+)?(?:(?:struct|enum)\s+(?:ResyncOpts|TargetReplicationResyncStatus|BucketReplicationResyncStatus|ResyncStatusType)|fn\s+(?:resync_state_accepts_update|resync_status_duration|should_count_head_proxy_failure|should_auto_resume_resync|is_version_id_mismatch))\b' \
|
||||
crates/ecstore/src/bucket/replication \
|
||||
--glob '*.rs' >"$REPLICATION_RESYNC_CONTRACT_BACKSLIDE_HITS_FILE" || replication_resync_status=$?
|
||||
if [[ "$replication_resync_status" -ne 0 && "$replication_resync_status" -ne 1 ]]; then
|
||||
@@ -2996,7 +2997,7 @@ fi
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
replication_resync_boundary_status=0
|
||||
rg -n --with-filename 'rustfs_replication::(resync::(RESYNC_META_FORMAT|RESYNC_META_VERSION|WIRE_ZERO_TIME_UNIX)|mrf::(MRF_META_FORMAT|MRF_META_VERSION)|(encode_resync_file|decode_resync_file|encode_mrf_file|decode_mrf_file|BucketReplicationResyncStatus|ResyncOpts|TargetReplicationResyncStatus|resync_state_accepts_update|should_count_head_proxy_failure|should_auto_resume_resync|is_version_id_mismatch)\b)' \
|
||||
rg -n --with-filename 'rustfs_replication::(resync::(RESYNC_META_FORMAT|RESYNC_META_VERSION|WIRE_ZERO_TIME_UNIX)|mrf::(MRF_META_FORMAT|MRF_META_VERSION)|(encode_resync_file|decode_resync_file|encode_mrf_file|decode_mrf_file|BucketReplicationResyncStatus|ResyncOpts|TargetReplicationResyncStatus|resync_state_accepts_update|resync_status_duration|should_count_head_proxy_failure|should_auto_resume_resync|is_version_id_mismatch)\b)' \
|
||||
crates/ecstore/src/bucket/replication \
|
||||
--glob '*.rs' \
|
||||
--glob '!replication_resync_boundary.rs' >"$REPLICATION_RESYNC_BOUNDARY_BYPASS_HITS_FILE" || replication_resync_boundary_status=$?
|
||||
@@ -3005,7 +3006,7 @@ fi
|
||||
fi
|
||||
|
||||
replication_resync_boundary_grouped_status=0
|
||||
rg -n -U --with-filename 'use\s+rustfs_replication::\{[^}]*\b(encode_resync_file|decode_resync_file|encode_mrf_file|decode_mrf_file|BucketReplicationResyncStatus|ResyncOpts|TargetReplicationResyncStatus|resync_state_accepts_update|should_count_head_proxy_failure|should_auto_resume_resync|is_version_id_mismatch)\b' \
|
||||
rg -n -U --with-filename 'use\s+rustfs_replication::\{[^}]*\b(encode_resync_file|decode_resync_file|encode_mrf_file|decode_mrf_file|BucketReplicationResyncStatus|ResyncOpts|TargetReplicationResyncStatus|resync_state_accepts_update|resync_status_duration|should_count_head_proxy_failure|should_auto_resume_resync|is_version_id_mismatch)\b' \
|
||||
crates/ecstore/src/bucket/replication \
|
||||
--glob '*.rs' \
|
||||
--glob '!replication_resync_boundary.rs' >>"$REPLICATION_RESYNC_BOUNDARY_BYPASS_HITS_FILE" || replication_resync_boundary_grouped_status=$?
|
||||
@@ -3021,7 +3022,7 @@ fi
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
replication_object_decision_boundary_status=0
|
||||
rg -n --with-filename 'rustfs_replication::(MustReplicateOptions|ReplicationDeleteSource|ReplicationMultipartPartInput|ReplicationResyncTargetObject|delete_replication_missing_source_decision|delete_replication_object_opts|heal_uses_delete_replication_path|is_retryable_delete_replication_head_error|is_version_delete_replication|replication_etags_match|replication_multipart_complete_actual_size|replication_multipart_part_plan|resync_target_for_object|should_retry_delete_marker_purge)\b' \
|
||||
rg -n --with-filename 'rustfs_replication::(MustReplicateOptions|ReplicationDeleteSource|ReplicationMultipartPartInput|ReplicationResyncTargetObject|delete_marker_purge_mrf_entry|delete_marker_purge_version_id|delete_replication_missing_source_decision|delete_replication_object_opts|heal_uses_delete_replication_path|is_retryable_delete_replication_head_error|is_version_delete_replication|replicate_delete_outcome|replication_etags_match|replication_multipart_complete_actual_size|replication_multipart_part_plan|resync_existing_delete_replication_info|resync_target_for_object|should_retry_delete_marker_purge|target_delete_version_id)\b' \
|
||||
crates/ecstore/src/bucket/replication \
|
||||
--glob '*.rs' \
|
||||
--glob '!replication_object_decision_boundary.rs' >"$REPLICATION_OBJECT_DECISION_BOUNDARY_BYPASS_HITS_FILE" || replication_object_decision_boundary_status=$?
|
||||
@@ -3030,7 +3031,7 @@ fi
|
||||
fi
|
||||
|
||||
replication_object_decision_boundary_grouped_status=0
|
||||
rg -n -U --with-filename 'use\s+rustfs_replication::\{[^}]*\b(MustReplicateOptions|ReplicationDeleteSource|ReplicationMultipartPartInput|ReplicationResyncTargetObject|delete_replication_missing_source_decision|delete_replication_object_opts|heal_uses_delete_replication_path|is_retryable_delete_replication_head_error|is_version_delete_replication|replication_etags_match|replication_multipart_complete_actual_size|replication_multipart_part_plan|resync_target_for_object|should_retry_delete_marker_purge)\b' \
|
||||
rg -n -U --with-filename 'use\s+rustfs_replication::\{[^}]*\b(MustReplicateOptions|ReplicationDeleteSource|ReplicationMultipartPartInput|ReplicationResyncTargetObject|delete_marker_purge_mrf_entry|delete_marker_purge_version_id|delete_replication_missing_source_decision|delete_replication_object_opts|heal_uses_delete_replication_path|is_retryable_delete_replication_head_error|is_version_delete_replication|replicate_delete_outcome|replication_etags_match|replication_multipart_complete_actual_size|replication_multipart_part_plan|resync_existing_delete_replication_info|resync_target_for_object|should_retry_delete_marker_purge|target_delete_version_id)\b' \
|
||||
crates/ecstore/src/bucket/replication \
|
||||
--glob '*.rs' \
|
||||
--glob '!replication_object_decision_boundary.rs' >>"$REPLICATION_OBJECT_DECISION_BOUNDARY_BYPASS_HITS_FILE" || replication_object_decision_boundary_grouped_status=$?
|
||||
@@ -3058,6 +3059,25 @@ if [[ -s "$REPLICATION_OBJECT_COMPARE_CONTRACT_BACKSLIDE_HITS_FILE" ]]; then
|
||||
report_failure "replication object comparison contracts must stay in crates/replication: $(paste -sd '; ' "$REPLICATION_OBJECT_COMPARE_CONTRACT_BACKSLIDE_HITS_FILE")"
|
||||
fi
|
||||
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
replication_m2_decision_status=0
|
||||
# M2-moved pure decision contracts must not be redefined in ECStore. The
|
||||
# ssec_passthrough_evidence_present name is deliberately absent: ECStore
|
||||
# keeps a thin HeadObjectOutput adapter under that name in
|
||||
# replication_target_boundary.rs which delegates to the crate-owned pure fn.
|
||||
rg -n --with-filename '^\s*(?:pub(?:\([^)]*\))?\s+)?(?:enum\s+(?:SsecPassthroughCapability|SsecPassthroughGate)|fn\s+(?:replicate_delete_outcome|target_delete_version_id|delete_marker_purge_version_id|delete_marker_purge_mrf_entry|resync_existing_delete_replication_info|version_identity_drifted|is_replication_target_offline_error|ssec_passthrough_gate))\b' \
|
||||
crates/ecstore/src/bucket/replication \
|
||||
--glob '*.rs' >"$REPLICATION_M2_DECISION_CONTRACT_BACKSLIDE_HITS_FILE" || replication_m2_decision_status=$?
|
||||
if [[ "$replication_m2_decision_status" -ne 0 && "$replication_m2_decision_status" -ne 1 ]]; then
|
||||
exit "$replication_m2_decision_status"
|
||||
fi
|
||||
)
|
||||
|
||||
if [[ -s "$REPLICATION_M2_DECISION_CONTRACT_BACKSLIDE_HITS_FILE" ]]; then
|
||||
report_failure "M2-moved replication decision contracts must stay in crates/replication: $(paste -sd '; ' "$REPLICATION_M2_DECISION_CONTRACT_BACKSLIDE_HITS_FILE")"
|
||||
fi
|
||||
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
rg -n --with-filename 'rmp_serde::(to_vec_named|from_slice)|LittleEndian::(write_u16|read_u16)|const\s+MRF_META_(FORMAT|VERSION):\s+u16\s*=\s*1\b' \
|
||||
@@ -3728,6 +3748,12 @@ fi
|
||||
crates/ecstore/src/bucket/replication \
|
||||
--glob '*.rs' |
|
||||
rg -v '^crates/ecstore/src/bucket/replication/replication_target_boundary\.rs:' || true
|
||||
# M2-moved target decision contracts (pure gate family, offline classifier,
|
||||
# version identity drift) route through the target boundary only.
|
||||
rg -n -U --with-filename 'rustfs_replication::\{[^;]*\b(?:SsecPassthroughCapability|SsecPassthroughGate|is_replication_target_offline_error|ssec_passthrough_gate|ssec_passthrough_evidence_present|version_identity_drifted)\b|rustfs_replication::(?:SsecPassthroughCapability|SsecPassthroughGate|is_replication_target_offline_error|ssec_passthrough_gate|ssec_passthrough_evidence_present|version_identity_drifted)\b' \
|
||||
crates/ecstore/src/bucket/replication \
|
||||
--glob '*.rs' |
|
||||
rg -v '^crates/ecstore/src/bucket/replication/replication_target_boundary\.rs:' || true
|
||||
) >"$REPLICATION_TARGET_BOUNDARY_BYPASS_HITS_FILE"
|
||||
|
||||
if [[ -s "$REPLICATION_TARGET_BOUNDARY_BYPASS_HITS_FILE" ]]; then
|
||||
|
||||
Reference in New Issue
Block a user