* fix(ecstore): skip hidden metadata in walk limit
* fix(ecstore): match walk limit to visible versions
* fix(ecstore): avoid decoding all versions for limit
* fix: return 503 on lock contention instead of 500
When concurrent PUTs to the same key contend for the namespace write
lock, lock timeout and conflict errors were wrapped as
StorageError::other(...) → StorageError::Io(...), which fell through
to S3ErrorCode::InternalError (500) in the error mapping.
Only QuorumNotReached was correctly mapped to ServiceUnavailable (503);
all other lock errors (Timeout, AlreadyLocked, etc.) became 500.
Fix: map_namespace_lock_error now returns StorageError::Lock(err) for
non-quorum lock errors, and StorageError::Lock is mapped to
S3ErrorCode::ServiceUnavailable in the HTTP error conversion.
Affected paths:
- crates/ecstore/src/set_disk/lock.rs
- crates/ecstore/src/store/object.rs
- crates/ecstore/src/store/bucket.rs (make_bucket, delete_bucket)
- rustfs/src/app/object_usecase.rs (copy_object)
Regression test: test_concurrent_put_same_key_never_returns_500
* test: fail on unexpected lock contention errors
---------
Co-authored-by: houseme <housemecn@gmail.com>
* fix(lifecycle): harden scanner ILM expiry accounting
* fix(scanner): gate ILM action accounting on enqueue
* fix(metrics): avoid scanner source work argument list
* fix(scanner): gate local ILM accounting on enqueue
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Log local endpoint disk-topology details during physical disk independence checks.
Include canonical paths, st_dev major:minor values, and resolved device ids in validation output to make Docker and bind-mount startup failures easier to diagnose.
Also assert the new diagnostics are present in the shared-disk regression test.
* fix(tier): stop sending nil/garbage versionId to warm backend S3
Three bugs caused NoSuchVersion errors when reading tiered objects:
1. warm_backend_s3sdk: GET and DELETE ignored rv/range opts entirely —
fixed to forward version_id and byte-range to the SDK request.
2. version.rs (MetaObject + MetaDeleteMarker): transition_version_id was
parsed with unwrap_or_default(), turning invalid/wrong-length bytes
into Uuid::nil(). The nil UUID was then serialized and sent as
?versionId=00000000-... to the tier backend -> NoSuchVersion.
Fixed: .and_then(.ok()).filter(!is_nil()) so only valid non-nil UUIDs
are forwarded as versionId.
3. bucket_lifecycle_ops: add debug/error logs in
get_transitioned_object_reader to record tier, tier_object, and
tier_version_id before and on failure of the tier GET.
Also adds tier transition fields to dump_fileinfo example for offline
xl.meta inspection, and fixes Docker build (cargo path + entrypoint).
Adds CLAUDE.md with tier architecture and debugging notes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* more fixes for versionId
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Marcelo Bartsch <marcelo@bartsch.cl>
* remove branch
* Add tests and fix cargo path, add load to build-docker
* update documentation (CLAUDE.md)
* more fixes for recover
* More fixes to ILM recover
* final fix
* chore: add missing-shard first-scene diagnostics (#3213)
chore(ecstore): add missing-shard first-scene diagnostics
Log rename_data quorum context behind RUSTFS_ISSUE3031_DIAG_ENABLE so partial-disk success can be correlated with later missing shard reads.
Also log put_object commit success and tmp cleanup boundaries to capture when successful quorum writes are followed by tmp_dir cleanup.
* fix test anmd fmt
* fix cargo path
fix test
* fix(tier): format copy_object self-copy guard
---------
Signed-off-by: Marcelo Bartsch <marcelo@bartsch.cl>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
* fix(admin): format policy JSON and improve error messages in service account API
- Use serde_json::to_string_pretty for policy serialization to match MinIO behavior
- Replace technical error messages with user-friendly ones for policy validation
- Fixesrustfs/rustfs#3233 (policy JSON not formatted)
- Fixesrustfs/rustfs#3232 (error messages not shown to users)
* Delete scripts/tempfile.zip
---------
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
* feat(scanner): expose checkpoint and source work status
* fix(scanner): count ignored checkpoints once per scan
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
* fix(allocator): restore validated jemalloc target gating
Restrict the global allocator and jemalloc profiling paths to linux-gnu-x86_64 so Linux ARM64 builds fall back to mimalloc again.
Update the runtime profiling, allocator reclaim, and admin profiling handlers to use the same target gating and avoid exposing jemalloc-only code on unsupported targets.
Verification:
- cargo fmt --all
- cargo fmt --all --check
- cargo check -p rustfs
- make pre-commit
* fix(profiling): restore non-jemalloc platform behavior
Restore CPU profiling support on macOS while keeping jemalloc-backed memory profiling restricted to validated linux-gnu-x86_64 targets.
Also restore Unix log directory permission hardening so the allocator regression fix does not roll back unrelated observability behavior on other supported platforms.
Verification in progress:
- cargo fmt --all
- cargo check -p rustfs
- make pre-commit (running)
* fix(profiling): qualify periodic memory log macros
Use explicit tracing macro paths in periodic jemalloc memory profiling logging so builds do not fail when the local target configuration excludes those branches from the current import set.
Verification:
- cargo fmt --all
- cargo check -p rustfs
* fix(profiling): gate unsupported memory helper
Restrict the unsupported memory profiling helper to non-linux-gnu-x86_64 targets so Linux builds do not emit dead_code warnings for an unreachable fallback.
Verification:
- cargo fmt --all
- cargo check -p rustfs
* fix(bucket-encryption): populate default KMS key for SSE-KMS without key ID
When PutBucketEncryption receives SSE-KMS configuration without a specific
KMS key ID, query the KMS service for the default key and populate
KMSMasterKeyID before storing. This ensures GetBucketEncryption responses
include the key ID, which S3 clients like mc rely on to distinguish SSE-KMS
from SSE-S3 in their display logic.
Fixes#3039
* test(kms): cover empty bucket encryption key ID
* fix(bucket-encryption): require default KMS key
* refactor(ecstore): add accessor methods for service globals
Phase 3 of global singleton consolidation. Add ECStore accessor
methods that delegate to process-global service singletons,
providing a unified API through ECStore.
New methods:
- notification_system() — delegates to notification_sys global
- bucket_metadata_sys() — delegates to bucket metadata global
- endpoints() — delegates to endpoints global
- region() — delegates to region global
- tier_config_mgr() — delegates to tier config global
- server_config() — delegates to server config global
- storage_class() — delegates to storage class global
1161 tests pass, clippy clean, formatting clean.
* fix: use imported functions instead of fully-qualified paths
Address review comments: use imported functions instead of
crate:: paths to avoid unused import warnings.
1160 tests pass, clippy clean.
* fix(ecstore): remove unused service imports
* fix(ecstore): skip hidden metadata in walk limit
* fix(ecstore): match walk limit to visible versions
* fix(ecstore): avoid decoding all versions for limit
* refactor(ecstore): migrate config globals into ECStore struct fields
Phase 2 of global singleton consolidation. Add server_config and
storage_class fields to ECStore, sharing the same underlying data
as the process-global LazyLock statics.
New ECStore fields:
- server_config: RwLock<Option<Config>>
- storage_class: RwLock<storageclass::Config>
New async accessor methods:
- get_server_config() / set_server_config()
- get_storage_class() / set_storage_class()
Fields initialized with defaults in ECStore::new(), synced from
globals after init() completes. Global functions preserved for
backward compatibility.
1160 tests pass, clippy clean, full workspace compiles.
* fix: address PR #3219 review comments - delegate to globals
Accessors now delegate to process-global statics to avoid state drift:
- get_server_config() delegates to get_global_server_config()
- set_server_config() updates both global and local field
- get_storage_class() delegates to get_global_storage_class()
- set_storage_class() updates both global and local field
- Removed stale sync code from init()
- Changed fields to std::sync::RwLock for sync access
1160 tests pass.
* fix: correct import order for cargo fmt
* fix: simplify config accessors - delegate to globals without local state
Remove local server_config and storage_class fields from ECStore.
Accessors now purely delegate to process-global statics, eliminating
the state drift risk identified in review.
1160 tests pass, formatting clean.
* chore(ecstore): add missing-shard first-scene diagnostics
Log rename_data quorum context behind RUSTFS_ISSUE3031_DIAG_ENABLE so partial-disk success can be correlated with later missing shard reads.
Also log put_object commit success and tmp cleanup boundaries to capture when successful quorum writes are followed by tmp_dir cleanup.
* chore(ecstore): add delete-objects lock batch diagnostics
Log DeleteObjects batch-lock request size, distributed lock quorum summary, and failed object keys behind RUSTFS_ISSUE3031_DIAG_ENABLE.
Include a Chinese operator checklist under docs/ that explains which logs to capture and how to interpret DeleteObjects lock_batch timeout incidents.
* chore: remove lock batch checklist from repo
Keep docs/issue-658-deleteobjects-lock-batch-checklist-zh.md only in the local checkout and drop it from the PR history.
* perf(erasure): remove UUID from clone + increase encode inflight budget
Two targeted optimizations for the erasure encoding hot path:
1. Erasure::clone() no longer generates Uuid::new_v4() per clone.
The _id field is unused in the hot path; reusing the original ID
eliminates a CSPRNG call per block encode (100 calls for a 100MB
object with 1MB blocks).
2. Default RUSTFS_ERASURE_ENCODE_MAX_INFLIGHT_BYTES raised from 8MB
to 32MB. This increases the encode pipeline depth from ~5 to ~20
blocks, allowing more read-ahead between the encoder and disk
writer stages. The per-request memory bound is still controlled
by the 8-block hard cap and the env var override.
3. Added encode_data_owned() utility method for zero-copy encoding
when the caller already owns a heap buffer (Vec<u8> → BytesMut
via Bytes::try_into_mut). Not used in the hot path yet but
available for future callers.
All 1157 ecstore tests pass. Criterion micro-benchmarks show no
regression (< 2% variance). Single-machine warp E2E tests were
inconclusive due to high variance; a dedicated multi-disk test
environment is needed for reliable E2E comparison.
Ref: https://github.com/rustfs/backlog/issues/659
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore: update Cargo.lock
* fix(erasure): align encode inflight cap and tests
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
houseme