gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-30 08:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(ecstore,rustfs): reuse canonical starts_with_ignore_ascii_case (#6759)

Browse Source 
* refactor(ecstore,rustfs): reuse canonical starts_with_ignore_ascii_case

`crates/utils/src/http/metadata_compat.rs` owns the internal metadata key helpers, including `starts_with_ignore_ascii_case`. Two files carried their own byte-identical copies of that predicate: `SetDisks::starts_with_ignore_ascii_case` in ecstore and a free function in the S3 options layer. Both drive internal metadata key classification (`internal_metadata_suffix` and quorum hashing on one side, `should_skip_object_metadata_key` and `is_reserved_user_metadata_key` on the other), so keeping three implementations of one predicate is an avoidable drift risk on a path that decides whether an internal key is treated as user metadata.

Delete both local copies and call the canonical implementation. Every prefix used at these call sites is an ASCII constant or literal, where the canonical byte-slice comparison and the removed `str::get(..n)` form are equivalent; that equivalence was checked differentially over 4.6M (key, prefix) pairs, including keys with multi-byte characters straddling the prefix boundary. No other logic in `internal_metadata_suffix` or `should_skip_object_metadata_key` changed.

Add regression tests on both sides pinning the two properties the switch depends on: internal prefixes match case-insensitively (a mixed-case `X-RustFS-Internal-*` key stays internal), and keys shorter than a prefix never match (they stay ordinary user metadata).

Refs rustfs/backlog#2051

* fix(rustfs): avoid typos-checker false positive in prefix-length test

The test literal "x-rustfs-encryptio" (a deliberate truncation of the
x-rustfs-encryption- prefix, used to assert that a key shorter than every
internal prefix falls through to user metadata) reads as a likely typo of
"encryption" to the repo's typos CI check. Derive it from
RUSTFS_ENCRYPTION_PREFIX via slicing instead of a hand-typed literal, which
both satisfies the linter and ties the truncation to the real constant
instead of a copy-typed guess.

Refs rustfs/backlog#2051
This commit is contained in:
Zhengchao An
2026-08-28 07:35:09 +08:00
committed by GitHub
parent 18068eb7e5
commit a6a04b5faa
2 changed files with 95 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/ecstore/src/set_disk/metadata.rs
+39 -10
View File
@@ -575,18 +575,12 @@ impl SetDisks {
meta.metadata.keys().any(|name| http::is_object_encryption_marker(name))
}
fn starts_with_ignore_ascii_case(value: &str, prefix: &str) -> bool {
value
.get(..prefix.len())
.is_some_and(|value_prefix| value_prefix.eq_ignore_ascii_case(prefix))
}
fn internal_metadata_suffix(name: &str) -> Option<&str> {
name.get(http::RUSTFS_INTERNAL_PREFIX.len()..)
.filter(|_| Self::starts_with_ignore_ascii_case(name, http::RUSTFS_INTERNAL_PREFIX))
.filter(|_| http::starts_with_ignore_ascii_case(name, http::RUSTFS_INTERNAL_PREFIX))
.or_else(|| {
name.get(http::MINIO_INTERNAL_PREFIX.len()..)
.filter(|_| Self::starts_with_ignore_ascii_case(name, http::MINIO_INTERNAL_PREFIX))
.filter(|_| http::starts_with_ignore_ascii_case(name, http::MINIO_INTERNAL_PREFIX))
})
}
@@ -604,9 +598,9 @@ impl SetDisks {
|| suffix.eq_ignore_ascii_case(http::SUFFIX_REPLICATION_STATUS)
|| suffix.eq_ignore_ascii_case(http::SUFFIX_REPLICATION_TIMESTAMP)
|| suffix.eq_ignore_ascii_case(http::SUFFIX_PURGESTATUS)
|| Self::starts_with_ignore_ascii_case(suffix, http::SUFFIX_REPLICATION_RESET_ARN_PREFIX)
|| http::starts_with_ignore_ascii_case(suffix, http::SUFFIX_REPLICATION_RESET_ARN_PREFIX)
// Raw compatibility keys are normalized and hashed separately below.
|| Self::starts_with_ignore_ascii_case(suffix, http::SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX)
|| http::starts_with_ignore_ascii_case(suffix, http::SUFFIX_REPLICATION_DELETE_MARKER_VERSION_ARN_PREFIX)
}
fn update_hash_quorum_metadata_map(hasher: &mut Sha256, entries: &HashMap<String, String>) {
@@ -1590,6 +1584,41 @@ mod tests {
);
}
/// Guards the switch to `rustfs_utils::http::starts_with_ignore_ascii_case`:
/// internal prefixes must keep matching case-insensitively, and keys shorter
/// than the prefix must keep being rejected. Misclassifying either way leaks
/// internal metadata into the quorum hash (or drops it out of it).
#[test]
fn internal_metadata_suffix_is_prefix_case_insensitive_and_rejects_short_keys() {
assert_eq!(
SetDisks::internal_metadata_suffix("X-RustFS-Internal-Replica-Status"),
Some("Replica-Status"),
"mixed-case RustFS prefix must match and preserve the suffix casing"
);
assert_eq!(
SetDisks::internal_metadata_suffix("X-MINIO-INTERNAL-replica-status"),
Some("replica-status"),
"mixed-case MinIO prefix must match"
);
assert_eq!(SetDisks::internal_metadata_suffix(http::RUSTFS_INTERNAL_PREFIX), Some(""));
// Keys shorter than either prefix, and non-internal keys, stay unmatched.
assert_eq!(SetDisks::internal_metadata_suffix(""), None);
assert_eq!(SetDisks::internal_metadata_suffix("x-rustfs-interna"), None);
assert_eq!(SetDisks::internal_metadata_suffix("x-minio-interna"), None);
assert_eq!(SetDisks::internal_metadata_suffix("x-amz-meta-custom"), None);
// The suffix-prefix comparisons behind the classifier follow the same rules.
assert!(SetDisks::is_replication_quorum_metadata_key(
"X-RustFS-Internal-Replication-Reset-arn:rustfs:replication::target:bucket"
));
assert!(SetDisks::is_replication_quorum_metadata_key(
"X-Minio-Internal-Replication-Delete-Marker-Version-arn:rustfs:replication::target:bucket"
));
assert!(!SetDisks::is_replication_quorum_metadata_key("x-rustfs-interna"));
assert!(!SetDisks::is_replication_quorum_metadata_key("x-rustfs-internal-replication-res"));
}
/// rustfs#5801: parity counts outside [0, total_shards] come from corrupt
/// or foreign metadata and must be treated as invalid entries instead of
/// clamped values that poison `common_parity`'s occurrence counting.