fix(replication): keep multipart objects on the multipart transport (#7047)

mirror of https://github.com/rustfs/rustfs.git synced 2026-09-05 19:55:37 +00:00

A 6 GiB object uploaded to the source as a 768-part multipart upload was
replicated to a generic S3 target with a single PutObject, and the target
rejected the body with EntityTooLarge. No CreateMultipartUpload was ever
issued, so the multipart replication transport never ran for the object
it exists for.

`replication_put_object_options` seeded the transport from
`object_info.is_multipart()` and then overwrote it with the second
return value of `decrypt_checksums`. Those two booleans do not mean the
same thing: the first is the object's storage shape, read from the ETag,
while the second reports whether the stored *checksum record* carries
per-part data. A full-object checksum -- what `aws s3 cp` writes by
default for a CRC algorithm -- is serialized with no MULTIPART flag even
on a multipart upload, so the record reports false and the object was
routed as a single PUT. `decrypt_checksums` documents this in
object_api/types.rs: callers that need routing must consult
`is_multipart()`. Replication did the opposite.

Route on the object's own shape, and let the checksum record only add
multipart-ness, never take it away. Objects already stored with such a
record are fixed too: the ETag was always right.

This also repairs the diagnosis of rustfs#6825, where the single-PUT
5 GiB guard fired against an object that was multipart all along and
told the operator to re-upload it as multipart.

Tests cover the three shapes the router has to separate: a multipart
object with a full-object checksum record (the regression, which fails
without this change), a multipart object with a composite record, and a
single-part object that must not be promoted onto multipart.

This commit is contained in:

唐小鸭

2026-09-02 18:22:32 +08:00

committed by

GitHub

parent 32eb116cbc

commit 922552083f

2 changed files with 117 additions and 2 deletions

									
										docs/operations/replication-object-size-limits.md
									
		+5
		
												View File
												
				@@ -24,6 +24,11 @@ because heal and delete convergence address the replica by that identity.

				This is why object size alone does not tell you whether an object is

				replicable — how it was uploaded does.

				The shape is read from the object's own ETag: a multipart ETag carries a

				`-<part count>` suffix. Nothing else selects the transport — in particular the

				checksum algorithm and checksum type (`COMPOSITE` or `FULL_OBJECT`) an object

				was uploaded with have no bearing on it.

				## Limits

				### Single-`PutObject` objects: 5 GiB