gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-07 04:25:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(storage): complete Snowball and decommission follow-ups (#7039)

Browse Source 
* feat(storage): complete Snowball and capacity follow-ups

* fix(ecstore): clarify V3 capacity gate guidance

* fix(ecstore): keep target contention retryable

* fix(ecstore): preserve typed target lock errors

* fix(ecstore): harden decommission recovery

* fix(ecstore): close decommission recovery races

* fix(ecstore): fail closed on multipart cleanup gaps

* fix(ecstore): model capacity mutation parameters

* fix(ecstore): settle checkpoint capacity retries
This commit is contained in:
cxymds
2026-09-03 11:58:59 +08:00
committed by GitHub
parent d011ec9952
commit df30dff1a7
19 changed files with 4817 additions and 449 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operations/snowball-auto-extract.md
+36
View File
@@ -0,0 +1,36 @@
# Snowball Auto-Extract Limits
RustFS accepts MinIO-compatible Snowball auto-extract uploads. Archive
members are streamed into objects while RustFS enforces entry-count, path,
PAX metadata, per-object, cumulative unpacked-size, and decoded-stream
limits.
## Size limits
The defaults remain compatible with the existing safety policy:
| Environment variable | Default | Hard maximum | Meaning |
| --- | ---: | ---: | --- |
| `RUSTFS_SNOWBALL_MAX_ENTRY_BYTES` | 1 GiB | 1 TiB | Maximum unpacked size of one archive member |
| `RUSTFS_SNOWBALL_MAX_UNPACKED_BYTES` | 10 GiB | 10 TiB | Maximum cumulative unpacked object bytes in one request |
Invalid values use the default. Zero is treated as one byte, values above the
hard maximum are clamped, and the per-entry limit is never allowed to exceed
the cumulative request limit. RustFS derives a separate decoded-stream limit
with bounded room for tar headers and PAX metadata; it cannot be disabled.
Increasing either limit raises the maximum work performed by one admitted
request. Snowball archive decoder admission remains globally bounded, so a
larger archive cannot create an unbounded number of concurrent decoders.
Restart RustFS after changing these environment variables.
## Small-member concurrency
For requests that set Snowball ignore-errors and do not use bucket quota
accounting, RustFS stages members up to 128 KiB and commits at most 16 at a
time. Requests that must stop on the first write error and quota-enabled
requests remain serial so their observable error and accounting behavior does
not change.
Set `RUSTFS_SNOWBALL_EXTRACT_MAX_INFLIGHT=1` to restore fully serial member
commits. Values are clamped to the range 1 through 16.