gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-10 07:06:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(replication): make bucket replication rules editable from clients (#5715)

Browse Source 
* fix(replication): accept explicit STANDARD destination storage class

The replication engine never reads Rule.Destination.StorageClass (replica
placement comes from the bucket-target config or the source object), yet the
validator rejected any config carrying the field. The console's add-rule form
always sends StorageClass=STANDARD, so every rule created through it failed
with InvalidRequest.

Tolerate exactly STANDARD as a no-op — semantically identical to omitting
the field — and keep rejecting every other value, which would be silently
ignored rather than honored. Document the deliberate omission from the
replication capability contract.

* feat(admin): support MinIO-style partial updates for set-remote-target

set-remote-target?update=true previously replaced every stored field and
required complete credentials in the body, so flipping a target's sync mode
from the console forced operators to re-enter the secret key, and real
mc replicate update bodies (madmin Clone() strips the secret) failed to
deserialize at all.

Adopt MinIO's TargetUpdateType contract: query params creds/sync/bandwidth/
path name the field groups to overlay onto the stored target, everything
else keeps its persisted value, and unsupported groups (proxy, healthcheck,
edge, edgeSyncBeforeExpiry) fail loudly. Credentials updates are skipped for
site-replication peer targets — probed by both scheme derivations of the
stored endpoint and the stored deployment id — because an operator never
knows the site replicator's credentials, and a body-supplied deployment id
is ignored on update since it anchors peer identity. madmin JSON aliases
(bandwidthlimit, storageclass, resetID, deploymentID, sessionToken) let mc
bodies parse under deny_unknown_fields.

e2e: cover a credential-free sync-only update preserving the stored
connection and the zero-ops no-op contract; align the missing-arn assertion
with the earlier validation error.

* chore(scripts): add two-site replication lab manager

site_replication_smoke.py spawns and manages two local rustfs processes,
pairs them via the site-replication admin API (idempotent), and verifies
bidirectional object replication. Subcommands: up/down/restart/status/logs/
smoke/info/remove/clean. Stdlib-only; requests are SigV4-signed the same
way as crates/e2e_test.

* chore(scripts): rename direction-suffixed payload variables for typos check

The typos linter reads the _ba suffix in payload_ba as a misspelling of
"by"; use payload_a_to_b / payload_b_to_a instead.

---------

Co-authored-by: overtrue <anzhengchao@gmail.com>
This commit is contained in:
唐小鸭
2026-08-05 09:50:31 +08:00
committed by GitHub
parent 4042bc0a5e
commit 15b9c1f4e3
4 changed files with 992 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/replication/src/config.rs
+58 -2
View File
@@ -20,7 +20,7 @@ use s3s::dto::DeleteReplicationStatus;
use s3s::dto::Destination;
use s3s::dto::{
ExistingObjectReplicationStatus, ReplicaModificationsStatus, ReplicationConfiguration, ReplicationRule,
ReplicationRuleStatus, ReplicationRules,
ReplicationRuleStatus, ReplicationRules, StorageClass,
};
use serde::{Deserialize, Serialize};
use std::collections::{BTreeMap, HashMap, HashSet};
@@ -28,6 +28,10 @@ use uuid::Uuid;
pub const REPLICATION_CAPABILITY_CONTRACT_VERSION: u32 = 1;
// `Rule.Destination.StorageClass` is deliberately absent from both lists below:
// clients should keep omitting it, but the validator tolerates an explicit
// `STANDARD` as a no-op (see `unsupported_replication_config_field`) because the
// console's rule form always sends it.
pub const REPLICATION_WRITABLE_FIELDS: &[&str] = &[
"Role",
"Rule.ID",
@@ -164,7 +168,18 @@ pub fn unsupported_replication_config_field(config: &ReplicationConfiguration) -
if rule.destination.replication_time.is_some() {
return Some("Destination.ReplicationTime");
}
if rule.destination.storage_class.is_some() {
// The replication engine never reads this field (replica placement comes from
// the bucket-target config or the source object), so an explicit STANDARD —
// which the console's rule form always sends — is indistinguishable from
// omitting it and is tolerated as a no-op. Any other value would be silently
// ignored rather than honored, so it stays rejected. Exact match: S3 storage
// class enums are case-sensitive.
if rule
.destination
.storage_class
.as_ref()
.is_some_and(|class| class.as_str() != StorageClass::STANDARD)
{
return Some("Destination.StorageClass");
}
}
@@ -952,6 +967,47 @@ mod tests {
config.rules[0].destination.storage_class =
Some(s3s::dto::StorageClass::from_static(s3s::dto::StorageClass::STANDARD_IA));
assert_eq!(unsupported_replication_config_field(&config), Some("Destination.StorageClass"));
// The exact-match contract is deliberate: S3 storage class enums are
// case-sensitive, so a lowercase variant must stay rejected.
config.rules[0].destination.storage_class = Some(StorageClass::from("standard".to_string()));
assert_eq!(unsupported_replication_config_field(&config), Some("Destination.StorageClass"));
// Explicit STANDARD is a no-op (the engine never reads the field) and must
// pass: the console's rule form always sends it, and rejecting it makes the
// form unusable.
config.rules[0].destination.storage_class = Some(StorageClass::from_static(StorageClass::STANDARD));
assert_eq!(unsupported_replication_config_field(&config), None);
}
#[test]
fn explicit_standard_storage_class_is_accepted_from_wire_xml() {
// The exact request shape the console's add-replication-rule form sends:
// a rule whose Destination carries <StorageClass>STANDARD</StorageClass>.
let xml = br#"
<ReplicationConfiguration>
<Role></Role>
<Rule>
<ID>console-rule</ID>
<Status>Enabled</Status>
<Priority>1</Priority>
<DeleteMarkerReplication><Status>Enabled</Status></DeleteMarkerReplication>
<Destination>
<Bucket>arn:aws:s3:::destination</Bucket>
<StorageClass>STANDARD</StorageClass>
</Destination>
</Rule>
</ReplicationConfiguration>
"#;
let mut deserializer = Deserializer::new(xml);
let config = <ReplicationConfiguration as s3s::xml::Deserialize>::deserialize(&mut deserializer)
.expect("console-shaped config should parse");
deserializer
.expect_eof()
.expect("console-shaped config should consume the whole body");
assert_eq!(config.rules[0].destination.storage_class.as_ref().map(|c| c.as_str()), Some("STANDARD"));
assert_eq!(unsupported_replication_config_field(&config), None);
}
#[test]