Commit Graph

20 Commits

Author SHA1 Message Date
cxymds 6e0640444e fix(multipart): list uploads across all sets (#5362) 2026-07-28 17:04:40 +08:00
cxymds bd978bed2d fix(admin): stabilize cluster capacity usage reporting (#5053)
* fix(admin): stabilize cluster capacity usage reporting

* test(admin): strengthen capacity usage regressions
2026-07-20 17:42:03 +08:00
houseme 15f4e75870 fix(cache): harden object data cache coordination (#5004)
* fix(cache): enforce projected entry capacity

Refs: rustfs/backlog#1335

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(cache): fence identity budget eviction by generation

Refs rustfs/backlog#1334.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(cache): fence clear against concurrent fills

Refs rustfs/backlog#1333

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(cache): linearize memory reservation claims

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(cache): retain allocation memory claims

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(cache): publish memory snapshots by epoch

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(cache): coordinate cold object fills

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ecstore): fence metadata cache transition races

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-18 14:37:10 +00:00
Zhengchao An 91a5c87132 refactor(startup): thread explicit InstanceContext through the storage startup path (#4611) 2026-07-09 16:53:41 +08:00
Zhengchao An 4f999bb6b8 perf(ecstore): backfill rename_data old size and gate the PUT prelookup (#4598) 2026-07-09 12:08:01 +08:00
houseme e3e5693e60 fix(ecstore): correct heal result drive-record reporting (#4555)
Two report-only defects on the heal result-reporting surface (they do not
affect the data path or heal decisions):

- default_heal_result (set_disk/ops/heal.rs): the offline-disk branch pushed
  an Offline record but fell through into the unconditional push, emitting a
  second (Corrupt) record for the same disk. This grew before/after.drives to
  disk_count + offline_count and misaligned every entry after the first offline
  slot. Add `continue` after the offline push, drive `disk_len` and the loop
  from a single `self.disks` snapshot, and assert `errs.len() == disk_len`.

- Sets::heal_format (core/sets.rs): the before/after drive lists were
  pre-filled with N default placeholders and then N real entries were pushed,
  yielding a 2N list whose healed status updates (indexed 0..N) landed on the
  blank placeholder half. Assign the lists directly from formats_to_drives_info
  (mirroring the set-level heal_format) so the healed updates hit the real
  entries.

Add regression tests covering offline/online record alignment and the
NoHealRequired and heal paths of the pool-level heal_format.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-08 18:29:54 +00:00
Zhengchao An 91dec123d9 refactor(ecstore): add per-instance InstanceContext, migrate erasure setup type (#4413)
* refactor(ecstore): add per-instance InstanceContext, migrate erasure setup type

Phase 5 of the global-singleton consolidation (backlog#939): begin moving
runtime identity state out of process globals so multiple ECStore instances
can coexist in one process. Isolation is carried by the object graph
(ECStore -> Sets -> SetDisks holding an Arc<InstanceContext>), not a
task-local, which does not propagate across the many internal tokio::spawn
boundaries in the data/background paths.

This first slice migrates the erasure setup type -- previously three
independent process-global bools -- into a single per-instance
RwLock<SetupType> that derives is_erasure / is_dist_erasure / is_erasure_sd,
removing a triple source of truth that could drift out of sync.

- New runtime::instance module: InstanceContext + process bootstrap context.
- The legacy free-function facade (is_erasure/update_erasure_type/...) keeps
  its signatures and forwards to the current instance's context, falling back
  to the bootstrap context before a store is published.
- ECStore gains a pub(crate) ctx field and setup_is_* accessors; its
  constructors adopt the bootstrap context (never mint a fresh one) so startup
  writes and post-construction reads share one cell -- single-instance
  behavior is byte-for-byte unchanged.

Tests: erasure predicate derivation vs the legacy behavior, object-graph
carrier isolation across two ECStore instances, and bootstrap adoption.

Refs: backlog#939 (Phase 5, Slice 1), backlog#653 (item 8)

* refactor(ecstore): thread InstanceContext down the object graph (Phase 5 Slice 2) (#4415)

* refactor(ecstore): source the namespace lock manager per-instance (#4417)

refactor(ecstore): source the namespace lock manager per-instance (Phase 5 Slice 3)

Phase 5 Slice 3 (backlog#939): give each instance its own lock namespace by
sourcing SetDisks' lock manager from the instance context instead of the
process singleton. This removes the false cross-instance mutual exclusion (and
attendant ABBA risk) that a shared GlobalLockManager would cause once multiple
instances coexist.

- InstanceContext gains a `lock_manager: Arc<GlobalLockManager>`. `new()` mints
  a fresh manager (independent per-instance); `bootstrap_ctx()` aliases the
  process singleton via get_global_lock_manager(), so a single-instance
  deployment keeps exactly one shared namespace.
- SetDisks::new sources `local_lock_manager` from `ctx.lock_manager()` (the ctx
  it already adopts), not `runtime_sources::global_lock_manager()`. Single
  instance: same Arc as before, so behavior is unchanged.
- Remove the now-unused `runtime_sources::global_lock_manager()` wrapper.

Tests: bootstrap lock manager aliases the process singleton; two fresh contexts
own distinct managers; a SetDisks' lock manager is the one from its context and
aliases the global singleton in a single-instance build.

Verification: cargo test -p rustfs-ecstore (10 Phase 5 + set_disk locking
regressions green), cargo clippy -p rustfs-ecstore --all-targets (clean),
make pre-commit (pass).

Refs: backlog#939 (Phase 5, Slice 3). Stacked on #4415 (Slice 2).
2026-07-08 15:01:40 +08:00
Zhengchao An 31c6859965 chore: converge stale TODOs and apply safe fills (backlog#646) (#4322)
Second TODO-convergence round over the current tree (backlog#646). All
line numbers in the old inventory had gone stale after the set_disk /
diagnostics / cluster refactors, so this re-scans and reduces the marker
count from 144 to 99.

STALE removals (comment describes already-implemented behavior, or dead
commented-out blocks) across ecstore (set_disk ops/core, store,
cluster/rpc, bucket/metadata_sys, services), iam, filemeta, s3select and
rustfs auth/object_usecase. No behavior change.

Safe fills, each verified:
- filemeta: replication_info_equals now also compares
  replication_state_internal (function currently has no callers; adds a
  regression test).
- bitrot: drop the confirmed-unused `_want` parameter from bitrot_verify
  and the now-unused `sum` on LocalDisk::bitrot_verify, removing a
  Bytes::copy_from_slice allocation. Streaming verify uses the file's
  embedded per-shard hash, never the passed sum.
- signer: rename v4_ignored_headers -> V4_IGNORED_HEADERS and drop the
  non_upper_case_globals allow.
- admin/heal: test_decode was #[ignore]d and used serde_urlencoded on a
  JSON body (would panic); rewire to serde_json::from_slice to match the
  production decode path, add assertions, un-ignore.

Verified: cargo fmt; cargo check on touched crates; tests pass
(filemeta, signer, bitrot, heal::test_decode); arch guardrail scripts
pass.
2026-07-06 22:43:32 +08:00
Zhengchao An 3d4fb532e5 refactor(replication): own filemeta wire contracts (#4254) 2026-07-04 06:39:54 +08:00
Zhengchao An 6a81445a96 refactor(replication): isolate ecstore owner contracts (#4239) 2026-07-03 21:42:45 +08:00
cxymds 5404ab155b fix(ecstore): precheck decommission capacity (#4196) 2026-07-02 22:36:29 +08:00
cxymds d1a6033191 fix(ecstore): recheck source cleanup under lock (#4193) 2026-07-02 22:35:31 +08:00
Zhengchao An 18b79ccc2a refactor(replication): route filemeta paths through crate (#4161) 2026-07-02 09:08:47 +08:00
GatewayJ a60b6310a7 fix(lifecycle): block actions on replication state (#4125) 2026-07-01 22:31:00 +08:00
Zhengchao An 055d3ad825 refactor(ecstore): scope dead code allowances (#4098) 2026-06-30 06:07:45 +08:00
GatewayJ 14e9fa2c8d fix(ecstore): avoid startup pool meta locks (#4056) 2026-06-29 20:19:38 +08:00
Zhengchao An e1a4b9e0b6 refactor: batch cluster lock and health readiness (#3936) 2026-06-27 10:51:06 +08:00
Zhengchao An 0a5b1b1b3a refactor: consolidate ecstore owner module layout (#3934)
* refactor: shrink ecstore root owner facades

* refactor: remove ecstore core store root shims

* refactor: move ecstore erasure owner modules

* refactor: remove ecstore root rpc facade

* refactor: move ecstore services domain modules
2026-06-27 09:03:20 +08:00
Zhengchao An c6ecfae39e refactor: move ecstore owner layout modules (#3932) 2026-06-27 05:54:25 +08:00
安正超 88dfdda4a0 refactor: establish ecstore layout foundation (#3645) 2026-06-20 09:54:36 +08:00