fix(s3): degrade multipart listings per upload instead of failing the bucket (#5721)

The multipart staging namespace is one flat set of sha256(bucket/object) directories shared by every bucket, and the cross-set listing rewrite reads every upload's metadata. Two shapes poisoned the whole ListMultipartUploads response with InternalError: Corrupted format: a healthy in-flight upload belonging to another bucket (its stored owner bucket fails the guard and fell into the corrupted-format arm), and a single upload directory whose xl.meta was torn by an unclean shutdown. Docker Distribution calls ListMultipartUploads on every PATCH/commit, so either shape broke OCI registry pushes entirely (issue #5716).

Foreign-bucket uploads are now skipped silently, and directories whose metadata is affirmatively corrupt at quorum are skipped with a debug log, while every other decode failure (quorum loss from offline disks, timeouts, transport errors) keeps failing the listing so clients retry instead of silently losing entries. The degrade-vs-propagate decision is a named corrupt-family classifier with a unit test pinning both sides. FileMeta::check_xl2_v1 now classifies a missing or wrong XL2 magic as FileCorrupt instead of an anonymous io error so damage is distinguishable from transient IO faults.

Refs #5716
This commit is contained in:
Zhengchao An
2026-08-05 11:49:52 +08:00
committed by GitHub
parent 8c9e884cf2
commit f73054f6ad
2 changed files with 280 additions and 18 deletions
+7 -2
View File
@@ -44,12 +44,17 @@ impl FileMeta {
}
pub fn check_xl2_v1(buf: &[u8]) -> Result<(&[u8], u16, u16)> {
// A file too short to hold the XL2 magic, or one that carries the
// wrong magic, is not merely unreadable — it is affirmative evidence
// of a torn or foreign write. Classify it as FileCorrupt so quorum
// and listing code can distinguish deterministic damage from
// transient IO faults (issue #5716).
if buf.len() < 8 {
return Err(Error::other("xl file header not exists"));
return Err(Error::FileCorrupt);
}
if buf[0..4] != XL_FILE_HEADER {
return Err(Error::other("xl file header err"));
return Err(Error::FileCorrupt);
}
let major = byteorder::LittleEndian::read_u16(&buf[4..6]);