* perf(erasure): remove UUID from clone + increase encode inflight budget
Two targeted optimizations for the erasure encoding hot path:
1. Erasure::clone() no longer generates Uuid::new_v4() per clone.
The _id field is unused in the hot path; reusing the original ID
eliminates a CSPRNG call per block encode (100 calls for a 100MB
object with 1MB blocks).
2. Default RUSTFS_ERASURE_ENCODE_MAX_INFLIGHT_BYTES raised from 8MB
to 32MB. This increases the encode pipeline depth from ~5 to ~20
blocks, allowing more read-ahead between the encoder and disk
writer stages. The per-request memory bound is still controlled
by the 8-block hard cap and the env var override.
3. Added encode_data_owned() utility method for zero-copy encoding
when the caller already owns a heap buffer (Vec<u8> → BytesMut
via Bytes::try_into_mut). Not used in the hot path yet but
available for future callers.
All 1157 ecstore tests pass. Criterion micro-benchmarks show no
regression (< 2% variance). Single-machine warp E2E tests were
inconclusive due to high variance; a dedicated multi-disk test
environment is needed for reliable E2E comparison.
Ref: https://github.com/rustfs/backlog/issues/659
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore: update Cargo.lock
* fix(erasure): align encode inflight cap and tests
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(ecstore): remove dead _buf field from Erasure struct
_buf was allocated in new(), clone(), and Default but never read or written.
Each clone() was allocating block_size bytes (typically 10MB) for nothing.
- Remove _buf from struct, Default, Clone, new_with_options
- 51 erasure tests pass
* fix: remove empty line after doc comment
---------
Co-authored-by: houseme <housemecn@gmail.com>
* fix(rio): surface incomplete put bodies
Propagate incomplete PUT request bodies as IncompleteBody instead of allowing erasure encode to treat truncated input as a normal EOF.\n\n- mark premature EOFs in HardLimitReader with an explicit IncompleteBody error\n- preserve EOF error chains through read_full and map them to S3 IncompleteBody\n- stop erasure encode from swallowing UnexpectedEof on truncated input\n- add regression tests for reader, erasure encode, and API error mapping\n\nRefs: rustfs/backlog#654
* fix(s3): honor decoded length for aws chunked put
Use x-amz-decoded-content-length for aws-chunked PutObject requests so trailer-checksum uploads are sized against the decoded payload instead of the wire-encoded content-length.\n\n- prefer decoded content length for aws-chunked put bodies\n- add a regression test covering the size selection logic\n- keeps the incomplete body fix working for truly truncated uploads while restoring checksum trailer compatibility\n\nRefs: rustfs/backlog#654
* fix(io): follow up review comments on incompletebody handling
Address PR review feedback by restoring read_full's existing EOF contract, adding a dedicated read_full_or_eof helper for erasure encoding, covering nested incomplete-body error chains, and documenting plus hardening aws-chunked size selection.\n\n- keep read_full returning early EOF on empty reads\n- use read_full_or_eof only in erasure encoding paths\n- detect aws-chunked via content-encoding or transfer-encoding\n- add nested error-chain and aws-chunked regression tests\n\nRefs: rustfs/backlog#654
* fix(rio): surface incomplete put bodies
Propagate incomplete PUT request bodies as IncompleteBody instead of allowing erasure encode to treat truncated input as a normal EOF.\n\n- mark premature EOFs in HardLimitReader with an explicit IncompleteBody error\n- preserve EOF error chains through read_full and map them to S3 IncompleteBody\n- stop erasure encode from swallowing UnexpectedEof on truncated input\n- add regression tests for reader, erasure encode, and API error mapping\n\nRefs: rustfs/backlog#654
* fix(s3): honor decoded length for aws chunked put
Use x-amz-decoded-content-length for aws-chunked PutObject requests so trailer-checksum uploads are sized against the decoded payload instead of the wire-encoded content-length.\n\n- prefer decoded content length for aws-chunked put bodies\n- add a regression test covering the size selection logic\n- keeps the incomplete body fix working for truly truncated uploads while restoring checksum trailer compatibility\n\nRefs: rustfs/backlog#654
* fix(io): follow up review comments on incompletebody handling
Address PR review feedback by restoring read_full's existing EOF contract, adding a dedicated read_full_or_eof helper for erasure encoding, covering nested incomplete-body error chains, and documenting plus hardening aws-chunked size selection.\n\n- keep read_full returning early EOF on empty reads\n- use read_full_or_eof only in erasure encoding paths\n- detect aws-chunked via content-encoding or transfer-encoding\n- add nested error-chain and aws-chunked regression tests\n\nRefs: rustfs/backlog#654
* fix(rio): reject bytes beyond hard limit
* fix(ecstore): reject zero-sized erasure blocks
* perf: reduce spawn_blocking contention in PUT path (~23% throughput gain)
Flame graph profiling identified tokio blocking pool mutex contention
as the #1 bottleneck (17.3% of CPU time). Each spawn_blocking call must
acquire parking_lot::raw_mutex to enqueue work. With 16 concurrent PUTs
× 4 disks × 3+ spawn_blocking per disk, this became a serialization point.
Optimizations applied:
- Merge make_dir_all + file write into single spawn_blocking
- Merge read_file + parse + write + rename into single spawn_blocking
for inline objects (small files)
- Optimize reliable_rename to try rename first, mkdir only on ENOENT
- Optimize remove/remove_std to try remove_file first, EISDIR fallback
- Add encode_inline_small fast path for small objects
- Parallelize bitrot writer creation with join_all
Benchmark (4KiB PUT, 4-disk EC, 16 concurrent, 8 rounds, randomized A/B):
Baseline: ~950 obj/s → Optimized: ~1173 obj/s (+23%)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: address review comments
- Return old_data_dir for non-inline rename_data path (was incorrectly None)
- Restore delete_all cleanup of PUT temp data on failure paths
- Fix cargo fmt formatting
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* style: apply rustfmt from stable 1.96.0
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: collapse nested if-let chains for clippy compliance
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: address Copilot review comments
- fs.rs: handle macOS EPERM from remove_file on directories
- os.rs: restore NotFound=Ok(()) semantics on first rename attempt
- local.rs: use try-rename-then-mkdir pattern for inline rename_data
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test: add unit tests for encode_inline_small fast path
* test: fix comment and line length in encode_inline_small tests
* fix: revert reliable_rename and write_all_internal to match original
Restore the original reliable_rename logic (check parent exists, then
rename in loop) and the original write_all_internal (make_dir_all outside
spawn_blocking). The optimization changes caused a CI-only test failure
in capacity_dirty_scope_test that could not be reproduced locally.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: update comment and fix formatting for CI
- Fix macOS/BSD comment to accurately say macOS only
- Fix encode_inline_small test formatting to match rustfmt 1.96.0
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: propagate inline rename errors
* fix: retry rename when parent missing
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Object repair rebuilt missing data shards but left missing parity shards unrecreated. A parity-only repair could therefore complete without restoring the shard contents.
Add a repair-specific reconstruction path that regenerates parity after data shards are available, while keeping normal read decoding data-only. Also shut down repair writers after all blocks are written.
Constraint: Preserve read-path decode behavior and limit parity regeneration to repair.
Confidence: high
Scope-risk: moderate
Directive: Do not route read decoding through parity regeneration without measuring the cost.
Tested: cargo test -p rustfs-ecstore decode_data_and_parity -- --nocapture
Tested: cargo test -p rustfs-ecstore heal_reconstructs_missing_parity_shard -- --nocapture
Tested: cargo test -p rustfs-ecstore erasure_coding -- --nocapture
Tested: cargo test -p rustfs heal_object_marks_missing_shard_disk_dirty_for_capacity_manager -- --nocapture
Tested: cargo test -p rustfs-heal -- --nocapture
Tested: cargo clippy -p rustfs-ecstore --all-targets -- -D warnings
Tested: cargo fmt --all --check
Tested: make pre-commit
Not-tested: Live multi-node disk replacement outside local test harness
* feat(append): implement object append operations with state tracking
Signed-off-by: junxiang Mu <1948535941@qq.com>
* chore: rebase
Signed-off-by: junxiang Mu <1948535941@qq.com>
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Refactor: reimplement lock
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Fix: fix test case failed
Signed-off-by: junxiang Mu <1948535941@qq.com>
* Improve: lock pref
Signed-off-by: junxiang Mu <1948535941@qq.com>
* fix(lock): Fix resource cleanup issue when batch lock acquisition fails
Ensure that the locks already acquired are properly released when batch lock acquisition fails to avoid memory leaks
Improve the lock protection mechanism to prevent double release issues
Add complete Apache license declarations to all files
Signed-off-by: junxiang Mu <1948535941@qq.com>
---------
Signed-off-by: junxiang Mu <1948535941@qq.com>
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.
Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
安正超