Commit Graph

17 Commits

Author SHA1 Message Date
houseme cb9edd59f5 test(ecstore): serialize bucket_delete tests to fix cargo-test global race (#4595)
The three store::bucket::tests::bucket_delete_* tests drive make_bucket /
delete_bucket through the process-global local-disk registry
(GLOBAL_LOCAL_DISK_MAP) and lock client, and through Sets::new which reads the
process-global erasure mode. Under `cargo test` (single process, many threads)
they race other global-touching tests and fail deterministically with
InsufficientWriteQuorum.

serial_test's #[serial] with the crate default key serializes them across the
in-process suite. Measured at --test-threads=16: baseline fails all three every
round; with #[serial] all three pass every round. nextest's per-test processes
are covered separately by the ecstore-serial-flaky test-group in
.config/nextest.toml (#4558). Full instance-level isolation remains a backlog
#939 (InstanceContext) concern and is not attempted here.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-09 01:37:53 +00:00
Zhengchao An ed81d2f6b8 test(ecstore): complete EC validation coverage gate
* test(ecstore): complete EC validation coverage gate

* test(ecstore): stabilize validation suite after rebase

* test(ecstore): fix rio-v2 clippy lint
2026-07-09 03:12:48 +08:00
Zhengchao An 3ed414cdb4 fix(storage): complete pending metadata and quorum fixes (#4375) 2026-07-07 23:22:54 +08:00
Zhengchao An 0a5b1b1b3a refactor: consolidate ecstore owner module layout (#3934)
* refactor: shrink ecstore root owner facades

* refactor: remove ecstore core store root shims

* refactor: move ecstore erasure owner modules

* refactor: remove ecstore root rpc facade

* refactor: move ecstore services domain modules
2026-06-27 09:03:20 +08:00
Zhengchao An b38976d5ee refactor: segment ECStore storage contracts by domain (#3910) 2026-06-26 17:47:36 +08:00
Zhengchao An c9614eb7cb refactor: route ecstore storage api boundaries (#3892) 2026-06-26 09:35:18 +08:00
Zhengchao An 7e60432588 refactor: centralize ecstore runtime owner sources (#3798) 2026-06-24 06:34:06 +08:00
安正超 57403525ee refactor: move heal and namespace contracts (#3560) 2026-06-18 11:00:21 +08:00
cxymds 339a192a09 fix(ecstore): preserve bucket visibility during rebalance (#3418)
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-14 03:37:54 +08:00
Henry Guo 0cdcd1eb7b feat(table-catalog): internalize catalog backing paths (#3295)
* feat(table-catalog): internalize catalog backing paths

* fix(table-catalog): clean internal catalog on bucket delete

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-09 12:01:25 +00:00
安正超 78df276d25 fix: return 503 on lock contention instead of 500 (#3274)
* fix(ecstore): skip hidden metadata in walk limit

* fix(ecstore): match walk limit to visible versions

* fix(ecstore): avoid decoding all versions for limit

* fix: return 503 on lock contention instead of 500

When concurrent PUTs to the same key contend for the namespace write
lock, lock timeout and conflict errors were wrapped as
StorageError::other(...) → StorageError::Io(...), which fell through
to S3ErrorCode::InternalError (500) in the error mapping.

Only QuorumNotReached was correctly mapped to ServiceUnavailable (503);
all other lock errors (Timeout, AlreadyLocked, etc.) became 500.

Fix: map_namespace_lock_error now returns StorageError::Lock(err) for
non-quorum lock errors, and StorageError::Lock is mapped to
S3ErrorCode::ServiceUnavailable in the HTTP error conversion.

Affected paths:
- crates/ecstore/src/set_disk/lock.rs
- crates/ecstore/src/store/object.rs
- crates/ecstore/src/store/bucket.rs (make_bucket, delete_bucket)
- rustfs/src/app/object_usecase.rs (copy_object)

Regression test: test_concurrent_put_same_key_never_returns_500

* test: fail on unexpected lock contention errors

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 02:20:21 +00:00
安正超 d6120f5788 refactor(ecstore): migrate mutable globals into ECStore struct fields (#3214)
* refactor(ecstore): migrate mutable globals into ECStore struct fields

Phase 1 of global singleton consolidation. Move mutable globals from
lazy_static into ECStore struct fields as the first step toward
dependency injection and multi-instance support.

New ECStore fields:
- is_erasure, is_dist_erasure, is_erasure_sd (erasure type flags)
- local_disk_map, local_disk_id_map, local_disk_set_drives
- root_disk_threshold
- tier_config_mgr, event_notifier, bucket_monitor

New accessor methods:
- is_erasure(), is_dist_erasure(), is_erasure_sd()
- update_erasure_type()
- tier_config_mgr(), event_notifier(), bucket_monitor()

Global functions in global.rs preserved for backward compatibility.
1151 ecstore tests pass.

* fix: address PR #3214 review comments

- Sync ECStore fields from globals after init()
- Enforce DistErasure => is_erasure invariant in update_erasure_type()
- Change bucket_monitor from Option to OnceLock for deferred initialization
- Restrict TypeLocalDiskSetDrives to pub(crate)
- 1151 tests pass

* fix: address PR #3150 review comments

- Restore get_host_addr as best-effort wrapper (String return type)
- Replace bare expect("err") with descriptive messages in unsigned trailer
- Simplify aws-chunked header construction
- 20 signer + 96 io-core tests pass

* fix(ecstore): format store imports

* fix(ecstore): keep migrated accessors in sync

* fix(signer): preserve host fallback and unsigned trailer errors

* fix(ecstore): defer migrated global accessors

* fix(signer): box unsigned trailer signing errors

* fix(ecstore): avoid lock awaits during sync

* fix(ecstore): narrow phase one globals

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-04 17:27:21 +00:00
Henry Guo be98c1f86a feat(table): add catalog boundary primitives (#3173)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-03 13:16:59 +00:00
安正超 97dd825468 fix(ecstore): optimize ObjectInfo clone and fix critical TODOs (#3149)
* fix(ecstore): optimize ObjectInfo clone and fix critical TODOs

## ObjectInfo hot-path clone optimization (issue #653 item 2)
- Wrap user_defined (HashMap), user_tags (String), parts (Vec) in Arc
- Clone cost reduced from ~20 heap allocations to O(1) ref count bump
- Updated 11 downstream access sites with explicit deref where needed

## Critical TODO/FIXME fixes (issue #653 item 7)
- rpc/peer_s3_client.rs: descriptive error message for empty peer response
- set_disk/write.rs: concurrent rollback deletes via tokio::spawn + join_all
- store_list_objects.rs: resolved FIXME with explanation + 4 regression tests
- store/bucket.rs: namespace write locks for make_bucket and delete_bucket

Skipped TODOs (too risky without broader context):
- multipart.rs:30 nslock — causes lock timeout in existing tests
- bucket.rs:88 cached list_bucket — needs cache invalidation strategy
- bucket.rs:149 replication delete — needs replication subsystem integration

1134 tests pass (1 flaky under parallel execution, passes individually).

* fix: address PR #3149 review comments

- Preserve NamespaceLockQuorumUnavailable variant in bucket create/delete
- Fix cancellation test to actually test cancellation (keep senders open)
- Update ObjectInfo consumers in app/ for Arc-backed fields
- Update bucket_usecase.rs user_tags to Arc<String>

* fix: update ObjectInfo Arc consumers

* fix: address ecstore merge review comments

* fix: cancel blocked merge output sends

* fix(ecstore): satisfy ObjectInfo clone clippy
2026-06-01 00:44:58 +00:00
cxymds 236142a682 fix(ecstore): repair lifecycle transition and restore flows (#2240)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: weisd <im@weisd.in>
2026-03-23 12:29:13 +08:00
weisd c6715259b1 fix(ecstore): keep bucket created time when metadata is epoch (#2220)
Co-authored-by: momoda693 <momoda693@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-19 01:02:34 +08:00
安正超 095b77795d refactor(ecstore): split store.rs into store submodules (#1942) 2026-02-25 06:35:27 +08:00