mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 16:28:15 +00:00
13bdca67624d5c0bf15e47945edeaa10cf292f2c
977 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 houseme
|
13bdca6762 |
build(toolchain): switch Rust channel to stable (#4775)
* Change Rust toolchain channel to stable Signed-off-by: houseme <housemecn@gmail.com> * style: apply clippy --fix and cargo fix lint suggestions Run `cargo clippy --fix --all-targets --all-features` and `cargo fix --lib --all-targets` across the workspace, then resolve the remaining warnings by hand: - collapse needless borrows in `format!` args, prefer `?` over explicit early returns, and use `.values()` / `.flatten()` iterator adapters - rewrite the `Md5` scan loop via `manual_flatten` and re-indent the `select!` macro body (rustfmt skips macro interiors) - annotate the intentional dead-code `Md5` inherent methods (constructed only by the test factory) with `#[allow(dead_code)]` Behavior is unchanged. Co-Authored-By: heihutu <heihutu@gmail.com> --------- Signed-off-by: houseme <housemecn@gmail.com> Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
cxymds
|
d8e69a3adf |
fix(logging): enforce single-writer sinks and bound tracing (#4765)
* fix(obs): prevent rolling log stdout aliasing * fix(ecstore): bound hot-path tracing payloads * test(logging): guard service and disk log invariants * fix(obs): silence useless_conversion on st_dev for Linux clippy rustix's Stat.st_dev is u64 on Linux/glibc, making u64::try_from a no-op that trips clippy::useless_conversion under -D warnings. The conversion is still needed on macOS/BSD where st_dev is a signed dev_t, so suppress the lint on that line rather than dropping the portable fallible conversion. * fix(logging): keep stdout sink validation portable (#4769) * fix(obs): keep stdout device conversion portable * docs(logging): update single-writer plan status --------- Co-authored-by: overtrue <anzhengchao@gmail.com> Co-authored-by: houseme <housemecn@gmail.com> |
||
|
houseme
|
2ddafb4ed9 |
test(ecstore): bound file sync probe waits (#4767)
Co-authored-by: Zhengchao An <anzhengchao@gmail.com> |
||
|
Zhengchao An
|
4c9431704c | fix(ecstore): cancel orphaned listing walks (#4773) | ||
|
Zhengchao An
|
b540c7e2d0 | test(ecstore): cover list marker key stripping (#4757) | ||
|
Zhengchao An
|
a5765274fc |
fix: auto-repair test_rename_data_shares_file_sync_limit hang on macOS (#4758)
fix(test): use canonicalized disk root for file_sync_probe in rename_data test On macOS, tempfile::tempdir returns /var/folders/... while LocalDisk resolves the root to /private/var/folders/... via dunce::canonicalize. The file_sync_probe::enter() path check uses starts_with(), so passing the non-canonical tempdir path caused the probe to never activate, making wait_for_active() hang indefinitely. Use disk.root (already canonicalized) for the probe instead. |
||
|
houseme
|
6886dca7d1 |
feat(ecstore): make GET codec-streaming a single rollout switch (backlog#1183) (#4752)
backlog#1183, staged rollout step. Simplify enabling the zero-duplex GET
codec-streaming fast path to a single switch — RUSTFS_GET_CODEC_STREAMING_ROLLOUT
(default "off") — now that body/header parity is proven (parity e2e net + bench
A/B on backlog#1183).
- Add a clean production rollout token "on" (aliases "full"/"production"); the
legacy "internal"/"benchmark" tokens remain accepted.
- Flip DEFAULT_RUSTFS_GET_CODEC_STREAMING_ENABLE and the two ..._COMPAT_CONFIRMED
defaults to true. They are retained as emergency kill-switches (set any to
false to force the fast path off) but no longer gate enablement — the rollout
switch does.
No production behavior change: with no env set the rollout switch defaults to
"off", so GET stays on the legacy duplex path exactly as before. Flipping the
hard default to on is deferred to a follow-up after a production soak.
Note (intentional semantics change): with the rollout switch opted in
("on"/"internal"/"benchmark"), codec streaming now activates without also
setting the two ..._COMPAT_CONFIRMED vars — compatibility is confirmed, so those
confirmations are baked in.
Co-authored-by: heihutu <heihutu@gmail.com>
|
||
|
Ramakrishna Chilaka
|
028ba6a675 |
perf(ecstore): parallelize multipart shard syncing (#4734)
Bound large shard-directory syncs per disk and process while preserving small-directory and durability behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> |
||
|
houseme
|
89557a7ffe |
perf(ecstore): cache io_uring fallback root label (#4747)
`record_uring_fallback` is called at multiple sites in the io_uring read path whenever a read falls back to `StdBackend`. It formatted `self.root.display().to_string()` on every call, heap-allocating a `String` from a `Path` that never changes after construction — pure per-read waste when io_uring is degraded. Cache the label once in `UringBackend::try_new` as a `String` field (`root_label`) and clone it per emission. The metric name and the `"root"` label value are unchanged; only the redundant `Path` formatting is removed. The clone is a single alloc of an already-short string. Refs: rustfs/backlog#1185 Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
Zhengchao An
|
763f246f8a |
test(ecstore): add shared MockWarmBackend test utility for lifecycle and tier tests (#4716)
* test(ecstore): extract shared MockWarmBackend into a test-util feature (backlog#1148 ilm-6) The tier/lifecycle integration tests carried two byte-for-byte copies of an in-memory WarmBackend mock — one in crates/scanner/tests and one in rustfs/src/app — plus duplicated register_mock_tier and polling helpers. Both implemented the same ecstore WarmBackend trait. Consolidate them into ecstore behind a new `test-util` feature, exposed via the `rustfs_ecstore::api::tier::test_util` facade: - MockWarmBackend: in-memory WarmBackend with an operation log (for ordering assertions such as "local delete precedes remote remove") and fault injection (FaultConfig): unreachable, HTTP 5xx, credential rejection, injected latency, plus external_remove to simulate an out-of-band remote deletion. - register_mock_tier / register_mock_tier_backend: register the mock into any TierConfigMgr handle (the global manager used by scanner tests or a per-instance one used by the app tests). - xl.meta transition assertion helpers: read_transition_meta, assert_transition_meta_consistent (cross-shard consistency of the status/tier/remote-key/remote-version-id tuple plus free-version count), and free_version_count. - polling helpers: wait_for_remote_absence, wait_for_object_count, wait_for_free_version_absence. Both existing copies now consume this single definition; `rg 'struct MockWarmBackend'` collapses to one. The feature is enabled only from [dev-dependencies], so it never links into the production binary (resolver 3). Designed for downstream ilm-8 (restore lifecycle) and ilm-11 (tier fault injection matrix). Coordinates with #4706 (ilm-2), which adds op-logging to the scanner mock — that op-logging is now part of this shared surface, so #4706 should rebase onto it. Refs rustfs/backlog#1148 (ilm-6), rustfs/backlog#1155. * test(ecstore): fix shared MockWarmBackend usage after main merge - Access stored objects via MockWarmBackend::contains() instead of the now private inner objects map (fixes E0609 after the shared test-util refactor). - Drop dead ReadCloser/ReaderImpl/DiskAPI imports and the unused transition_api test re-exports the mock extraction left behind. - Reword the scanner/rustfs test-util dependency comments so they no longer embed the literal rustfs_ecstore:: path that trips the ECStore architecture-migration guard. |
||
|
houseme
|
264b2dd480 |
perf(metrics): drop needless per-emission work on hot metric paths (#4743)
* perf(metrics): drop needless per-emission work on hot metric paths Audit of the metrics hot paths surfaced four low-risk wins where emission did work it did not need to: - `record_file_cache_reclaim_success/error` (disk/local.rs) called `.to_string()` on `kind` (already `&'static str`) and on the `"ok"`/`"err"` literals, heap- allocating up to four `String`s per page-cache reclaim window — which runs per read-stream reclaim. The `metrics` macros accept `&'static str` label values directly, so pass them as-is. - `record_read_repair_dedup` (set_disk/core/io_primitives.rs) likewise `.to_string()`-ed an already-`&'static str` `reason`. - `SetDisks::get_object_reader` (set_disk/ops/object.rs) captured `Instant::now()` and emitted the `rustfs.lock.acquire.*` counter and histogram unconditionally on every GET, right beside an already-gated stage timer. Gate them behind `get_stage_metrics_enabled()` too, so an inactive observability config pays no per-GET clock read or recorder lookups. - The per-response-body-chunk counter in server/http.rs re-ran the `counter!` registry lookup on every chunk (a streamed GET emits many). Resolve the label-less handle once into a `LazyLock<metrics::Counter>`; the global recorder is installed at startup before any response streams, so the cached handle binds to the final recorder. No metric names or label values change. The only behavior change is that the `rustfs.lock.acquire.*` GET-path metrics now follow the GET stage-metrics flag, consistent with the neighbouring stage timings. Co-Authored-By: heihutu <heihutu@gmail.com> * perf(metrics): gate page-cache reclaim metrics behind metrics_enabled() `record_file_cache_reclaim_success/error` run per read-stream reclaim window on large-object reads and emitted unconditionally. When general metrics are disabled the `counter!`/`histogram!` macros still construct three metric keys per call for nothing. Skip the emission behind `rustfs_io_metrics::metrics_enabled()`, matching how the io-metrics free functions self-gate. The serial reclaim-metrics test now enables the flag (save/restore) alongside the existing stage gate. Left ungated deliberately: `record_read_repair_dedup` (rare read-repair path, and its non-serial test would need a global-flag toggle), and the HTTP body-chunk counter (its cached handle already makes the disabled case a no-op increment). Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
e742a540a4 |
test(cache): guard the body-cache eligibility gate against deny-list regressions (#1146) (#4742)
`full_object_plaintext_len` decides whether a body-cache hook hit may serve bytes in place of the erasure read. It is a fail-closed allow-list: it excludes every read whose `ReadPlan::build` applies some other transform (ranged/part, raw/data-movement, restore, encrypted, remote) with an early `return None`, then returns a `Some(..)` length only for the whole-plaintext cases. A newly added `ReadPlan` branch that nobody teaches this gate about falls through to `None` and safely bypasses the cache. Flip it to a deny-list and the same new branch silently serves bytes in the wrong representation — the backlog#1108 / #1109 / #1146 class of bug. The existing unit and e2e tests only cover the branches that exist today. This adds `scripts/check_body_cache_whitelist.sh`, a structural guard wired into pre-commit / pre-pr / dev-check and CI, that asserts every exclusion predicate and a `return None` still precede the first `Some(..)`. Reordering a predicate, dropping one, moving the positive return ahead of the gate, or renaming/removing the function all fail; wording, formatting, and adding a new exclusion in the same gate do not. Mutation-tested against all four regression shapes. This machine-enforces the structural invariant that backlog#1146 was kept open to guard by hand. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
1ab66e124c |
test(ecstore): pin the remaining io_uring fd-cache invariants (#1180) (#4739)
Close the three test-completeness items in rustfs/backlog#1180 that the earlier hardening (rustfs/rustfs#4726, #4729) left unpinned; the other two of the five (sharded cancel routing, bailout-handle error) already landed in rustfs/uring. - rename_data end-to-end invalidation: drive the real `LocalDisk::rename_data` commit (non-inline part, so `invalidate_part_paths` is non-empty) and assert the destination part descriptor is dropped — not merely `rename_file`/`delete`. Both production `rename_data` call sites share this invalidation, so a "fix one copy, miss the other" regression is now caught. The test is non-vacuous: it seeds the cache, removes the on-disk data dir out of band (the cached fd keeps the old inode alive and clears the path for the directory rename `rename_data` performs), and asserts a read still returns the OLD bytes before the commit — which fails outright if the cache is off, so it cannot pass without a live cache. - FD_CACHE_TTL backstop: an injected short TTL proves the cache self-evicts a descriptor with no explicit invalidation; a static check pins the 5s value. - zero-length read bounds parity on the cache-HIT path: a `length == 0` read past EOF must be rejected identically to the miss path and StdBackend, pinning the #1173 fix against regression. Refactors `FdCache::new` to delegate to a private `with_ttl(ttl)` helper so the TTL backstop can be exercised with a short TTL instead of a multi-second wait. Verified: `cargo test -p rustfs-ecstore` on Linux with real io_uring (seccomp=unconfined, RLIMIT_NOFILE raised, RUSTFS_URING_TESTS_MUST_RUN=1 so a degraded skip fails rather than passing vacuously). Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
793b2a06e2 | perf(ecstore): snapshot per-IO env config on local disk read/write paths (#4736) | ||
|
houseme
|
9d64d71bd7 | fix(ecstore): reduce GET reader setup shard fanout (#4735) | ||
|
houseme
|
c79366d42c |
refactor(ecstore): tidy io_uring read backend (docs + fd-cache handle) (#4732)
Two behavior-preserving cleanups to the io_uring local read backend that accumulated as later work layered onto it: - Reunite the `UringBackend` struct doc comment. The backlog#1145 fd-cache constants were inserted into the middle of the struct's doc block, so its opening sentences were orphaned onto `ENV_RUSTFS_IO_URING_FD_CACHE` and the struct itself was documented by a sentence fragment starting mid-clause with "through rustfs-uring's...". Move the opening back onto the struct and give the constant its own one-line doc. - Fold the fd-cache handle and its lookup key into a single `Option<(&FdCache, FdKey)>` in `pread_uring`, so the get and the insert_if_fresh sites stop re-deriving `self.fd_cache.as_ref()` and re-matching presence. Semantics are identical, including the backlog#1176 generation guard (`gen_at_open` snapshot before open, insert only when the generation is unchanged). No functional change. The borrow/move pattern was validated against a host-compilable reduction (the cfg(linux) path cannot be built from macOS); CI compiles the Linux backend. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
4f29dcdcec |
chore(deps): bump rustfs-uring to 0.2.1 (#4731)
chore(deps): bump rustfs-uring to 0.2.1 from crates.io rustfs-uring 0.2.1 routes the driver's runtime diagnostics through `tracing` with structured fields instead of `eprintln!` (rustfs/uring#13). No public API change — UringDriver::probe_and_start_sharded, read_at, and read_at_direct keep their signatures — and the read path / cancel-safety ownership model are untouched, so this is a drop-in patch bump of the version requirement plus the lockfile entry. 0.2.1 adds a `tracing` dependency; it is recorded in the rustfs-uring lock entry. tracing is already in the workspace graph, so nothing new is pulled in. The Cargo.lock change is scoped to the rustfs-uring package only; unrelated lockfile drift is left for its own change. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
ffca98cdbf |
fix(ecstore): harden io_uring integration (#4726)
* fix(ecstore): close the fd-cache open-then-insert race with a generation guard (rustfs/backlog#1176) pread_uring's miss path opened a descriptor on the blocking pool and only then inserted it into the moka cache. moka's invalidations cover only entries present at call time, so a heal/delete commit that invalidated between the open and the insert could not stop the just-opened stale inode from being cached afterwards — serving the pre-heal/pre-delete inode for up to the TTL and defeating the heal. Add an invalidation generation to FdCache, bumped by invalidate_exact and invalidate_under before they touch moka. The read path snapshots the generation before opening and inserts via insert_if_fresh, which refuses the insert if the generation moved during the open and, with a post-insert re-check, removes the entry if an invalidation raced the insert itself. Reads that never miss are unaffected. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): invalidate the fd cache on the primary object-delete paths (rustfs/backlog#1175) The fd-cache invalidation contract was only wired into DiskAPI::delete, rename_file and rename_data, but object deletion almost never goes through LocalDisk::delete — DeleteObject(s) reach delete_version, delete_versions -> delete_versions_internal, and delete_paths, all of which remove a version's data dir (move_to_trash / rename_all staging) with no invalidation. A cached io_uring descriptor kept the deleted part.N inode readable for up to the TTL, so a GET in that window could still return deleted data. Invalidate every cached fd under the removed data dir at each site: in delete_version and delete_versions_internal the data_dir uuid and object path are in hand (invalidate_cached_fds_under(volume, "{path}/{uuid}")); delete_paths invalidates under each removed path. A later rollback that restores a data dir just causes the next read to re-open it. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): close remaining fd-cache invalidation gaps (rustfs/backlog#1177) Three residual paths could keep serving a stale descriptor: - delete_volume removed the whole bucket tree (remove_dir_all/remove_dir) with no invalidation, and the cache-hit read path skips the volume-access check, so a cached fd kept a removed object readable. Add invalidate_cached_fds_for_volume (a per-volume moka predicate) and call it after the bucket is removed. - A retired LocalDisk instance (renew_disk on reconnect builds a fresh one) kept its populated cache alive while still referenced by in-flight ops, so invalidations through the new instance never reached it. close() now clears the backend's cache via clear_cached_fds. - rename_data's post-commit rollback (a commit-metadata fsync failure under strict durability) restored the old data dir without dropping fds cached during the committed window; the streaming branch now invalidates the dst part fds on those rollback paths. The inline branch's rollback runs inside spawn_blocking and is left to the TTL backstop. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): narrow the io_uring latch classes to match StdBackend (rustfs/backlog#1171) The runtime degradation classification reused the probe-time restriction errnos, which the driver's C7 contract explicitly warns against, so a single per-file error could latch a whole disk off io_uring: - is_io_uring_unsupported no longer includes EACCES: at read time on an already-open fd it is per-file (an LSM hooks security_file_permission on every read) and StdBackend hits the same denial, so falling back masks nothing and a full-disk latch would be wrong. ENOSYS and EPERM (seccomp/LSM applied after startup) remain. EOPNOTSUPP is now classified per-path by the caller. - pread_uring_direct's read-error arm now mirrors StdBackend: an O_DIRECT-shape error (EINVAL/EOPNOTSUPP) latches only direct_uring.supported so eligible reads take StdBackend's aligned path, instead of over-latching the whole io_uring backend or never latching a read-side EINVAL at all. - try_new only negative-caches genuine restriction-class probe failures in URING_UNSUPPORTED_DISKS; an unexpected (possibly transient) probe failure now falls back without latching, so the next reconnect re-probes. Co-Authored-By: heihutu <heihutu@gmail.com> * feat(ecstore): log when a disk latches io_uring off at runtime (rustfs/backlog#1172) A probe-gated gray release was flying blind: the permanent per-disk `active` latch flipped with no log and no metric, so the only message operators ever saw was the startup "io_uring read backend enabled" line — which stayed true on dashboards even after the very first read latched the disk back to StdBackend forever. Add latch_active_off, which flips the latch with `swap` and logs the true->false transition exactly once at warn with a dedicated event constant, disk root, and errno. Both the buffered and O_DIRECT read paths use it. A fallback/latch metric counter and periodic export of the driver StatsSnapshot (cq_overflow, cancel_already) remain as follow-ups that need rustfs_io_metrics plumbing. Co-Authored-By: heihutu <heihutu@gmail.com> * chore(audit): correct the stale rustfs-uring license-allow rationale (rustfs/backlog#1181) The dependency-review allow said rustfs-uring is "pulled as a git dependency", but ecstore now pins it from crates.io. Update the rationale and scope the allow to the exact pinned version (pkg:cargo/rustfs-uring@0.1.0) so a future version bump forces a conscious re-review of the license/provenance claim instead of being waved through on an outdated justification. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): offload io_uring driver teardown off the tokio worker (rustfs/backlog#1170) UringBackend held Arc<UringDriver> and had no Drop, so when the last LocalDisk reference dropped in async context (disk reconnect via renew_disk, or shutdown), UringDriver's own Drop ran on that thread — sending Shutdown and joining each shard thread, which can block up to the bounded-drain timeout (5s) on a hung / D-state disk, stalling a tokio worker. Wrap the driver in ManuallyDrop (deref is transparent, so read call sites are unchanged) and add a Drop that takes the Arc and, when a runtime is present, drops it on a blocking thread so the potentially-blocking join never runs on a runtime worker. Off-runtime it drops inline. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): restore StdBackend read parity on the uring paths (rustfs/backlog#1173) Two byte-for-byte parity breaks against StdBackend on the io_uring read paths: - A zero-length read on an fd-cache hit returned Ok(empty) without any bounds check, while StdBackend and the uring miss path return FileCorrupt for an offset past EOF. Fstat the cached descriptor on the length==0 path and match. - reclaim_read_range fadvise(DONTNEED)'d the raw unaligned [offset, offset+len) range, but fadvise only drops fully-covered pages, so the head partial page stayed resident — whereas StdBackend's mmap path reclaims the page-aligned superset. Bitrot shards' 32-byte block headers keep offsets off page boundaries, so this diverged on the common case. Page-align the reclaim window to match the mmap path exactly. (The third parity item from the audit — a failed reclaim fadvise failing the read — is already parity: StdBackend's mmap path propagates the same fadvise error with `?`, so no change is needed.) Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): bound worst-case in-flight memory by chunking huge uring reads (rustfs/backlog#1174) The driver's backpressure permits count operations, not bytes, and it zero-fills a full-size buffer per op, so a single unbounded read could pin ~length bytes per permit (128 permits x shards x up to ~2 GiB). ecstore passes a whole part's shard range as one pread_bytes with no upstream chunking. On the buffered path, split reads larger than URING_MAX_OP_LEN (128 MiB) into sequential chunks, awaited one at a time, so worst-case in-flight memory is bounded by permits x URING_MAX_OP_LEN per shard. The threshold is high enough that ordinary shard reads keep the single-op, zero-copy fast path unchanged. The O_DIRECT path (opt-in, alignment-constrained) is left for a follow-up. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): gate the io_uring fd cache on RLIMIT_NOFILE headroom (rustfs/backlog#1178) The fd cache holds up to FD_CACHE_CAPACITY (512) descriptors per disk, but try_new cannot know the disk count and nothing checked the process fd budget. On a bare-metal / non-systemd run with the common 1024 soft RLIMIT_NOFILE, two disks would already exhaust fds with EMFILE surfacing on reads and probes. Check the soft limit at try_new: enable the cache only with ample headroom (>= 16384), otherwise log a warning once and fall back to open-per-read. The packaged systemd unit sets 1,048,576, so tuned deployments are unaffected. Co-Authored-By: heihutu <heihutu@gmail.com> * test(ecstore): make io_uring test skips visible and gate non-vacuity (rustfs/backlog#1179) The ecstore io_uring tests degrade to a silent pass when io_uring is unavailable (bare `return`s or plain eprintlns), so a CI leg on a restricted runner never exercises the real UringBackend/FdCache/latch paths yet still goes green — an integration regression could merge unseen. Add uring_test_skip: it emits a grep-able `SKIP <name>` line and, when RUSTFS_URING_TESTS_MUST_RUN is set (a CI leg that guarantees io_uring, e.g. a seccomp=unconfined container), panics instead of skipping. Route the silent-skip sites through it. Wiring a dedicated CI leg that sets that env on a capable runner is tracked in the issue; this provides the enforcement mechanism. Co-Authored-By: heihutu <heihutu@gmail.com> * test(ecstore): cover delete_paths fd-cache invalidation (rustfs/backlog#1180) Add an end-to-end test that seeds the descriptor cache with a read, removes the part via disk.delete_paths (one of the primary object-delete entry points that does not go through LocalDisk::delete), and asserts the next read no longer returns the removed inode — pinning the invalidation added in #1175. The sharded cancel-routing half of #1180 is covered in the rustfs-uring PR. Co-Authored-By: heihutu <heihutu@gmail.com> * io_uring audit follow-ups: O_DIRECT chunking, inline invalidation, metrics, CI leg (backlog#1160) (#4729) * fix(ecstore): chunk large O_DIRECT reads too, bounding in-flight memory (rustfs/backlog#1174) The buffered read path already splits reads above URING_MAX_OP_LEN into sequential chunks; do the same for the O_DIRECT path, which was left for a follow-up. read_at_direct aligns each chunk's sub-range internally, and chunk sizes are a multiple of URING_MAX_OP_LEN so a boundary re-read is at most one block. Extract classify_direct_read_error so the single-op and chunked paths share one copy of the EINVAL/EOPNOTSUPP-vs-subsystem latch classification rather than duplicating it. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(ecstore): invalidate cached fds on the inline rename_data rollback (rustfs/backlog#1177) The streaming rename_data branch invalidates cached part fds on its post-commit rollback paths, but the inline branch runs its commit and rollback inside a single spawn_blocking closure where the async invalidate cannot be called, so it was left to the TTL backstop. Capture the closure's result instead of `??`-propagating it: on error (a commit-metadata fsync failure under strict durability rolls the committed rename back), invalidate the dst part paths at the async level before returning. Inline objects keep their data in xl.meta rather than separate part inodes, so this is largely defensive, but it removes the caveat and keeps the two branches consistent. Co-Authored-By: heihutu <heihutu@gmail.com> * feat(ecstore): export io_uring latch/fallback and driver stats metrics (rustfs/backlog#1172) Complete the gray-release observability. Beyond the warn log added earlier, emit metrics so a dashboard can answer "how much traffic is on io_uring vs falling back, and is any disk degrading": - rustfs_io_uring_latch_off_total — a disk latching io_uring off at runtime. - rustfs_io_uring_read_fallback_total — each io_uring -> StdBackend read fallback (latched-off short-circuit, O_DIRECT error, buffered error). - a low-frequency per-disk exporter of the driver StatsSnapshot as gauges (in_flight, cq_overflow, cancel_already), spawned in try_new. It holds only a Weak reference so it never keeps the driver alive, and drops any temporary strong reference on the blocking pool so a last-reference UringDriver::Drop join never runs on an async worker (rustfs/backlog#1170). submit_errors is deliberately not exported yet: it is a field added in the unreleased rustfs-uring 0.2.0, and ecstore still pins 0.1.0. It lands once the dependency is bumped (rustfs/backlog#1181). Co-Authored-By: heihutu <heihutu@gmail.com> * ci: add a real-io_uring integration leg on ubuntu-latest (rustfs/backlog#1179) The existing self-hosted sm-standard runners cannot guarantee io_uring is available (a container seccomp filter can block io_uring_setup), so the ecstore uring tests degrade to a silent skip and never exercise the real UringBackend/FdCache/latch paths in CI. Add a job on GitHub-hosted ubuntu-latest, which runs a recent kernel with no container seccomp filter, running the uring-named ecstore tests with RUSTFS_IO_URING_READ_ENABLE=true and RUSTFS_URING_TESTS_MUST_RUN=1 — the non-vacuity gate makes the leg fail rather than skip if io_uring is unavailable, so an integration regression can no longer merge green behind a vacuous pass. Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
ab60244d7d |
chore(deps): bump rustfs-uring to 0.2.0 (#4730)
chore(deps): bump rustfs-uring to 0.2.0 from crates.io rustfs-uring 0.2.0 is published on crates.io. It carries the read-driver hardening from the backlog#1160 adversarial audit (cancel-safety and graceful-degradation fixes on the Linux io_uring read path). The public API ecstore consumes is unchanged — UringDriver::probe_and_start_sharded, read_at, read_at_direct all keep their 0.1.0 signatures — so this is a drop-in bump of the version requirement plus the lockfile entry. The Cargo.lock change is intentionally scoped to the rustfs-uring package only; unrelated lockfile drift is left for its own change. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
2ebe8e561b |
fix(replication): allow loopback replication targets under an explicit test opt-in (#4725)
* fix(replication): allow loopback replication targets under an explicit test opt-in
Commit
|
||
|
Zhengchao An
|
05fae6f939 |
test(ecstore): add TierConfigMgr state-machine unit coverage (#4713)
* test(ecstore): unit-test TierConfigMgr add/edit/remove/verify state machine (backlog#1148 ilm-4) Covers the tier config state machine and persistence paths that previously had only 4 codec tests and none for tier_config.rs: - add: non-uppercase name, duplicate name, unsupported type, missing backend payload, and a regression anchor documenting that AWS-reserved names (STANDARD) are not currently rejected. - edit: unknown tier, missing-credentials rejection for RustFS and MinIO. - remove: idempotent unknown-tier no-op, in-use rejection, empty-backend success, force skips the in_use probe, and probe-error surfacing. - verify: unknown tier, healthy backend, unhealthy backend. - pure query helpers (empty/is_tier_valid/tier_type/get/list_tiers). - persistence: JSON marshal/unmarshal roundtrip, external tier-config.bin roundtrip for Azure and GCS payload mapping, truncated/unknown-format/ unknown-version rejection, legacy v1 version-word acceptance, and encode failure on missing payload. Tests are hermetic: error paths return before backend construction, and a MockWarmBackend injected into driver_cache exercises remove/verify without any real remote. Refs backlog#1155. Co-authored-by: overtrue <anzhengchao@gmail.com> * fix(tier): reject reserved names STANDARD/RRS in TierConfigMgr::add (backlog#1148 ilm-4) (#4721) |
||
|
Zhengchao An
|
846aa95c32 |
test(security): GHSA-named regression tests for 3p3x and r5qv (backlog#1151 sec-6) (#4707)
test(security): add GHSA-named regression tests for 3p3x and r5qv (backlog#1151 sec-6) Anchor the two fixed advisories to discoverable, named regression tests so `rg -i "ghsa|3p3x|r5qv"` finds a guard for each, and future fixes are forced to update pinned behavior (red -> green). GHSA-3p3x-734c-h5vx (constant-time WebDAV/FTPS secret comparison, rustfs#4403): - ftps_core.rs: new `assert_ftps_ghsa_3p3x_wrong_credentials_rejected` drives the `ct_eq` reject branch in FtpsAuthenticator::authenticate; asserts wrong password and unknown user are both rejected (530) and indistinguishable. - webdav_core.rs: the auth-failure block now sends a valid access key with a wrong secret (exercising the `ct_eq` branch, not just the unknown-access-key path) plus an unknown user, asserting both 401 and indistinguishable. - Module doc comments map advisory -> tests -> fix PR on both files. GHSA-r5qv-rc46-hv8q (internode RPC fail-closed, rustfs#4402): - http_auth.rs: renamed the default-fallback rejection test to `ghsa_r5qv_resolve_shared_secret_rejects_default_fallback` (and broadened it to cover default env secret + blank secrets), and added `ghsa_r5qv_verify_rpc_signature_fails_closed_on_missing_or_invalid_auth` pinning the exact advisory scenario (missing/forged/cross-URL signature is rejected; a correctly signed request still passes). File-level doc maps the advisory. Docs: new docs/testing/security-regressions.md with the advisory -> test mapping table and where each layer runs; linked from docs/testing/README.md. sec-14 will formalize the written policy in AGENTS.md. The unit-level ghsa_r5qv_* tests run in the default CI pass. The WebDAV/FTPS e2e live in the protocols suite (fixed ports, --test-threads=1); they cannot join the e2e-smoke profile and are wired into CI by sec-5. Refs: rustfs/backlog#1151 (sec-6), rustfs/backlog#1155 |
||
|
Zhengchao An
|
ce53feee87 |
test(lifecycle): regression test for expire/GET race (#3491) (#4706)
Adds a serial-lane ILM integration regression test that pins the local-first ordering contract established by rustfs#3491, where `expire_transitioned_object` deletes local metadata BEFORE any remote tier cleanup so a concurrent GET can never observe live local metadata pointing at an already-removed remote tier version (user-visible `NoSuchVersion`). `serial_tests::test_expire_transitioned_object_never_races_concurrent_get` in `crates/scanner/tests/lifecycle_integration_test.rs`: - Transitions an object to a mock warm tier, then runs a tight concurrent GET loop while calling `expire_transitioned_object`; every GET must return a full, correct body or a clean object/version-not-found -- never a tier-fetch failure. - Deterministic ordering assertion (revert-proof): immediately after expiry the remote tier object is still present and the mock recorded zero remote `remove` calls, proving remote cleanup is deferred to free-version recovery. Reverting to remote-first ordering turns this assertion red. Supporting changes: - Re-export `expire_transitioned_object` from `api::bucket::lifecycle`. - Record remote-tier mutating ops in the scanner test `MockWarmBackend`. - Update tier-ilm-debugging.md to reference the new regression test. Runs in the CI ILM Integration (serial) lane (ci.yml test-ilm-integration-serial), picked up by the existing binary(lifecycle_integration_test) filter. Refs: rustfs/backlog#1148 (ilm-2), rustfs/backlog#1155 |
||
|
Henry Guo
|
3f25426534 |
fix(ecstore): reject incomplete listing usage refreshes (#4698)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com> |
||
|
houseme
|
7437f99c45 |
fix(cache): key the object body cache on data_dir for write-uniqueness (#4703)
* refactor(object-data-cache): derive Default for ObjectDataCacheGetRequest The GET request literal is hand-listed field-by-field across ~13 test sites in two crates. Adding `mod_time_unix_nanos` in backlog#1111 had to touch every one and still missed a literal, producing a compile error caught only in a later CI lane. Derive `Default` and spread the engine-crate literals so the next field addition is absorbed rather than fanned out. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(cache): key the object body cache on data_dir for write-uniqueness The cache key's correctness rested on being write-unique, but its only write-scoped component was `mod_time` — a wall-clock timestamp that is not monotonic and can be absent. Two writes that collide on MD5 (same etag+size) and land on an equal mod_time (clock skew, same-tick, or absent) derived the same key, so a node that never saw the overwrite could serve the previous body for up to the TTL, and the same collision turned the fill-after-invalidation race into a serving bug. This is the store's strong-read-after-write guarantee leaning on a probabilistic argument. Add `data_dir` — the xl.meta directory UUID ecstore regenerates on every body write — as the primary write-unique anchor: - surface `data_dir: Option<Uuid>` on ObjectInfo, copied from FileInfo in the single GET-path constructor `from_file_info`; - carry it into ObjectDataCacheKey as `data_dir_u128` (held as u128 to keep the engine crate free of a uuid dependency), derived in the one planner site both the ecstore hook and the usecase layer share, so both produce an identical key by construction; - keep `mod_time` as a second anchor and `etag+size` as belt-and-braces; an absent data_dir falls back to the prior behavior — strict improvement, no regression. Two writes distinct only by data_dir now derive different keys even under an MD5 collision with identical mod_time — the case mod_time alone cannot cover. Blast radius is compiler-guarded: ObjectInfo derives Default and every real construction site uses `..Default::default()`, so only from_file_info and one full-literal test needed the field. The three P0 body_cache_hook_e2e regressions, engine (80), and app (36) suites pass unchanged; a mutation that severs the planner wiring fails planner_key_changes_with_data_dir. Refs: backlog#1111, backlog#1118 Co-Authored-By: heihutu <heihutu@gmail.com> * fix(cache): thread data_dir field through the merged mutation-hook test Merging main (which landed the object-mutation-hook work, backlog#1131) brought in a GetRequest test literal that predates the data_dir field. Spread it via `..Default::default()` — the derive(Default) added here means this is the last such hand-listed literal to need touching. Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
85fd824581 |
feat(object-data-cache): close write-side invalidation gaps and add an admin surface (#4694)
* feat(object-data-cache): close write/delete-side invalidation gaps The object data cache exposed only a single per-(bucket,object) invalidation primitive and no write-side ecstore hook, so several delete paths left dead bodies resident until TTL (hygiene/capacity, not stale-serving: lookups follow a fresh metadata quorum and cannot serve a gone object). This adds the missing primitives and wires them in. ODC-26 (backlog#1131): add an `ObjectMutationHook` trait beside the GET body hook, registered next to it at startup, and call it from the ecstore-internal delete paths (`apply_expiry_on_non_transitioned_objects`, `expire_transitioned_object` including the restored-copy branch, and `delete_object_versions`). The app impl is one `invalidate_object` call under a new `AfterLifecycleExpiry` reason. ODC-27 (backlog#1132): force prefix delete now invalidates the whole prefix, not just the prefix string. `store.delete_object(delete_prefix)` returns no deleted-name list, so this uses a new prefix primitive rather than the batch path. ODC-28 (backlog#1133): DeleteBucket now flushes the bucket via a new bucket-scope primitive (covers force and non-force, which share the delete_bucket call). ODC-C2 (backlog#1143): add `ObjectDataCache::clear()` and two admin handlers (GET stats, POST flush) routed through admin runtime_sources. The starshard identity index gains a single `remove_matching` full-scan API backing prefix/bucket/clear; it is documented as admin/delete-path only and never runs on the GET or fill hot path. New invalidation reasons and metric labels added; outcome (removed/noop) labelling kept correct for every new primitive. Also fixes a pre-existing broken intra-doc link in memory.rs. Co-Authored-By: heihutu <heihutu@gmail.com> * refactor(ecstore): extract the shared HookSlot behind both cache hooks This PR introduced object_mutation_hook.rs by mirroring body_cache_hook.rs, which left two process-global registration slots whose register/get/clear bodies were line-for-line identical except the trait type and the WARN string: a RwLock<Option<Arc<dyn _>>>, an Arc::ptr_eq "different instance" warning, the poison-recovery closure, and the same read-lock-and-clone read. Two copies of the same swap-vs-warn logic can drift apart under maintenance. Hoist it into a generic HookSlot<T: ?Sized> that owns the logic once. Each hook module keeps its `static HOOK: HookSlot<dyn XxxHook>` and its thin, unchanged public wrappers (register_/get_/clear_), so the crate's public surface and every call site are untouched — this is an internal consolidation, not a contract change. The load-bearing #1126 guarantee (newest registration wins, so a rebuilt AppContext is never stranded on a first-wins slot) previously had no direct test — the hook tests only covered register-then-notify. HookSlot now has its own unit tests including re_registration_swaps_to_the_latest_instance; mutation-testing confirms a first-wins regression fails exactly that test. No behavior change: the two hooks' existing tests, the P0 body_cache_hook_e2e regressions, and the app-layer mutation-hook tests all pass unchanged. Refs: backlog#1126, backlog#1131 Co-Authored-By: heihutu <heihutu@gmail.com> * fix(admin): register the object-data-cache routes in the policy inventory This PR added GET /object-data-cache/stats and POST /object-data-cache/flush but did not list them in the two registries that must account for every admin route: the route-policy inventory (route_policy.rs) and the route matrix (route_registration_test.rs). Their coverage tests — route_policy_inventory_covers_registered_routes and test_admin_route_matrix_matches_registered_routes — failed on CI because a registered route had no policy/matrix entry. These two tests are not part of `make pre-commit` (which runs fmt + arch + quick-check, not the full suite), so the gap passed local pre-commit and only surfaced in the CI Test-and-Lint lane. stats is a read (ServerInfoAdminAction, Sensitive); flush mutates (ConfigUpdateAdminAction, High) — matching the actions the handlers already enforce. The MinIO-alias matrix test is unaffected: these are native rustfs endpoints with no MinIO equivalent. Refs: backlog#1143 Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
f9874b591a |
[DO NOT MERGE until published] chore(ecstore): switch rustfs-uring to crates.io 0.1.0 (#4701)
* chore(ecstore): switch rustfs-uring from the git rev to crates.io 0.1.0
Prepared ahead of the rustfs-uring 0.1.0 crates.io release (rustfs/uring):
flip ecstore's dependency from the pinned git revision to the published
`0.1.0`, and drop the now-unneeded git-source allowance in deny.toml.
DO NOT MERGE until rustfs-uring 0.1.0 is on crates.io. Until then cargo
cannot resolve `rustfs-uring = "0.1.0"`, so this branch will not build and
CI will be red — that is expected, not a defect in the change.
After the crate is published:
1. `cargo update -p rustfs-uring --precise 0.1.0` to regenerate Cargo.lock
(deliberately not touched here — the registry entry, with its checksum,
cannot be written before the crate exists);
2. push the Cargo.lock;
3. CI goes green; merge.
No code change. The guard `scripts/check_no_tokio_io_uring.sh` is unaffected
(it bans only tokio's io-uring runtime feature, not an explicit rustfs-uring
dependency). `audit.yml`'s `allow-dependencies-licenses` for rustfs-uring is
left in place — it is a first-party Apache-2.0 crate either way.
Co-Authored-By: heihutu <heihutu@gmail.com>
* chore: refresh rustfs-uring lockfile
---------
Co-authored-by: heihutu <heihutu@gmail.com>
|
||
|
houseme
|
6780140318 |
fix(object-data-cache): make the GET key write-unique and dedup the lookup (#4693)
fix(object-data-cache): make GET body cache key write-unique and dedup lookups Address four object-data-cache GET-path findings (backlog#1107 batch): ODC-06 (backlog#1111): the cache key was content-unique, not write-unique. Extend ObjectDataCacheKey with the resolved version's modification time (i128 unix nanoseconds, None -> 0), derived once in the shared planner so the ecstore hook and the usecase layer produce an identical key. An unversioned overwrite advances mod_time, so a stale node can no longer serve old bytes for up to the TTL under an MD5 collision; etag + size stay as belt-and-braces. ODC-16 (backlog#1121): every cacheable GET planned and looked up twice (once in the ecstore hook, once in the usecase layer), double-counting hits, hit_bytes and lookups. GetObjectReader now carries a GetObjectBodySource marker (Unprobed / HookMissed / HookServed); the hook stamps it, and build_get_object_body_with_cache serves a hook-served body directly and skips its lookup whenever the hook already probed. One hook-served GET now records exactly one lookup. ODC-19 (backlog#1124): ENABLE=true with no explicit mode defaulted to HitOnly, which never fills and keeps a permanent 0% hit rate. Default to FillBufferedOnly, log the resolved mode at startup, and warn when HitOnly is selected explicitly. ODC-24 (backlog#1129): max_entry_bytes above the in-memory GET fill limits was silently inert. Clamp the planner's size eligibility to min(max_entry_bytes, seek-support threshold, 64 MiB buffer cap) so ineligible sizes plan SkipTooLarge instead of being reported eligible, and warn at startup when the excess is inert. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
Zhengchao An
|
d0ca14d8df |
test(ecstore): cover post-commit hard-crash in rename_data crash harness (#4689)
The rename_data crash-consistency harness (backlog#935/#878) only armed pre-commit crash points (AfterDataRename, AfterBackupBeforeMetaCommit), which assert the *old* version survives. The other half of the "partial commit -> old or new, never mixed" invariant — a hard power loss *after* the xl.meta commit rename must leave the *new* version — had no crash-point coverage. The existing after-metadata-commit failpoint only exercises the graceful in-process rollback (restores old), a different path from a hard crash (no rollback runs, commit stays on disk). Add a RenameDataCrashPoint::AfterMetaCommit injection right after the commit rename (cfg(test), compiles to a const-false no-op in production, like the existing points) and overwrite/fresh x strict/relaxed tests asserting the object reads back as the new version. The fresh case pins that the point is genuinely post-commit: a pre-commit misplacement would leave no readable object and fail the assertion. cargo test -p rustfs-ecstore crash_consistency: 9 passed. clippy -D warnings clean; fmt and arch guards clean. |
||
|
houseme
|
d715cb5c34 |
refactor(ecstore): single-source the bitrot read/verify path (backlog#1159) (#4697)
P-A (`read_appending`) and P-C (the in-memory fast path) each copied the hashed-read logic, so `BitrotReader` ended up with the hash verification, the short-read error, and the scratch-buffer fill written three times across `read` and `read_appending`'s two branches. That is patch-on-patch: a change to the bitrot contract would have to be made in three places and kept in sync by hand. Collapse the duplication onto three single-source pieces: - `split_and_verify` — a free function that splits `[hash][data]`, verifies (unless skip_verify), and returns the data slice plus the hash time. Free rather than a method so it can run while `self` is borrowed for the block. - `read_scratch_block` — the single-pass fill of `self.buf` with the short-read-to-UnexpectedEof contract. - `short_shard_read` / `begin_read` — the shared error and preamble. `read` and `read_appending` now differ only in what they must: how the block is acquired (caller's slice vs `try_take_block` vs scratch fill) and where the verified shard lands (`copy_from_slice` vs `extend_from_slice`). This also fixes a latent inconsistency the duplication hid: the old `read` did `copy_from_slice` *before* verifying, so on a hash mismatch it left the corrupt bytes in the caller's buffer before returning the error, while `read_appending` verified first. Both now verify before writing, so a shard that fails the hash never reaches the caller's buffer on either method — the stronger of the two behaviors. Net -19 lines; behavior otherwise unchanged. Verified: `erasure::` 215 passed, 0 failed (including the fast-path equivalence and corrupt-shard tests, and the existing `test_bitrot_read_hash_mismatch`); clippy --all-targets -D warnings clean. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
6f05a740b3 |
refactor(ecstore): extract the shared io_uring read preamble (backlog#1145) (#4696)
`pread_uring` and `pread_uring_direct` accumulated across seven rounds (#4632/#4635/#4645/#4649/#4653/#4658/#4662) and each carried its own copy of the same open preamble: resolve the bucket path, run the volume access check, resolve the object path, and check the path length. The only real difference is what happens after — one opens buffered and errors as `DiskError`, the other opens `O_DIRECT` and errors as `DirectOpenError` so it can latch an O_DIRECT refusal. Extract that shared sequence into `resolve_uring_object_path`, a blocking helper called inside both `spawn_blocking` closures. Each caller keeps exactly what differs — its open flags, its error type, its metadata/bounds/align handling — and no longer restates the resolution. No behavior change: the same checks run in the same order and produce the same errors (the O_DIRECT path still maps them through `DirectOpenError::Disk`). Net -12 lines. Verified on a real Linux host (16-core, real io_uring): clippy -p rustfs-ecstore --all-targets -D warnings clean; disk::local tests 144 passed, 0 failed — including the io_uring, O_DIRECT, fd-cache, and page-cache-reclaim cases that exercise both read paths. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
c2362bca14 |
perf(ecstore): slice in-memory shards instead of copying them twice (#4687)
* perf(ecstore): slice in-memory shards instead of copying them twice (backlog#1159)
The GET path reads a shard out of the page cache into a `Bytes`, then
`open_disk_reader` erased it behind `Box<dyn AsyncRead>` by wrapping it in
a `Cursor`. Downstream, `BitrotReader` could only get it back by copying:
once out of the `Cursor` into its scratch buffer, and once from there into
the caller's buffer. CPU profiling of a cached 1 MiB GET (device reads = 0)
attributed 8.23% of the whole server to `Cursor::poll_read` alone — a copy
of data that was already sitting in memory.
Keep the source concrete instead of erasing it. `ShardReader` is an enum of
`InMemory(Cursor<Bytes>)` and `Stream(Box<dyn AsyncRead ...>)`, and the new
`ShardSource::try_take_block(n)` lets an in-memory source hand over the
`[hash][data]` block as a slice. `read_appending` uses it to verify the hash
on the slice and `extend_from_slice` the shard straight into the caller's
buffer: one copy instead of two.
`try_take_block` defaults to `None`, so a streaming source keeps the old
path byte for byte, along with its short-read and EOF semantics. A source
that cannot serve `n` bytes declines rather than truncating, so a partial
block still becomes UnexpectedEof rather than a short shard. The hash is
still checked before anything is appended, so a corrupt shard never reaches
the caller's buffer on either path. The deferred parity reader opens its
source lazily and stays on the streaming path; parity is only read when a
data shard fails.
Tests gate equivalence and non-vacuity:
* `try_take_block` fires for `Cursor<Bytes>`, advances the position exactly
as a read of the same length would, declines when fewer than `n` bytes
remain, and returns `None` for a non-`Bytes` source — without this the
equivalence test below would silently compare one path against itself;
* both paths return identical bytes for the same shard;
* a corrupt shard fails on the fast path too, appending nothing.
Verified: clippy --tests -D warnings clean; `erasure::` 215 passed, 0 failed;
`set_disk::core::io_primitives` 49 and `io_support::` 22 pass.
Stacked on #4681 (`read_appending`), which this builds on.
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(ecstore): implement ShardSource for Cursor<&[u8]> used by the erasure bench
`crates/ecstore/benches/erasure_benchmark.rs` builds
`BitrotReader<Cursor<&[u8]>>`, which the new `ShardSource` bound on
`ParallelReader`/`decode` does not accept. `cargo clippy --tests` does not
compile bench targets, so this only surfaced in CI's `--all-targets` run.
A borrowed slice carries no `Bytes` to hand out, so it takes the default
`try_take_block` and keeps the old streaming copy path — no behavior change.
Verified with the same target set CI uses:
`cargo clippy -p rustfs-ecstore --all-targets -- -D warnings` clean.
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
|
||
|
Zhengchao An
|
13f8768e7f |
feat(ecstore): make replication timing intervals env-overridable for tests (#4680)
Introduce RUSTFS_REPL_HEALTH_CHECK_INTERVAL_MS, RUSTFS_REPL_MRF_FLUSH_INTERVAL_MS and RUSTFS_REPL_RESYNC_POLL_MAX_MS so tests and operators can shorten the replication background loops. Defaults are unchanged; invalid values fall back with a warn and values below 10ms are clamped to avoid busy-spin. Add replication_fast_env() e2e helper (backlog#1147 repl-4). |
||
|
houseme
|
5f1a475c56 |
perf(ecstore): stop zeroing pooled shard buffers on the GET path (backlog#1159) (#4681)
`ShardBufferPool::take` handed out a `resize(len, 0)`-ed buffer, and the
reader then overwrote every byte of it. CPU profiling of a cached 1 MiB
GET (device reads = 0, so all cost is CPU) attributed 4.81% of the whole
server to that memset — a buffer pool exists to reuse an allocation, and
memsetting it gives the saving straight back.
The zeroing was load-bearing only because `BitrotReader::read` takes
`&mut [u8]`, which must be initialized. But the reader never reads what
the caller put there, and never returns a partially filled buffer: both
the hashed and the no-hash path either fill the whole shard or fail with
UnexpectedEof, and a hash mismatch is an error rather than a short read.
So the initialization bought nothing observable.
Add `BitrotReader::read_appending(&mut Vec<u8>, want)`, which appends into
the buffer's spare capacity instead of demanding initialized bytes:
* hashed path — unchanged single copy, `extend_from_slice(data)` in place
of `copy_from_slice` into a pre-zeroed buffer, and only after the hash
verifies, so corrupt bytes never reach the caller's buffer;
* no-hash path — `read_buf` writes straight into the spare capacity and
advances the length only over bytes the reader actually wrote, so an
uninitialized tail can never be exposed.
`ShardBufferPool::take` now yields an empty buffer with capacity, and
`read_shard` no longer needs to `truncate`. `read` keeps its old signature
for the remaining callers.
Four tests gate the contract rather than the call:
* `read_appending` is byte-for-byte identical to `read` on both paths;
* a truncated shard is UnexpectedEof, never a partially filled buffer;
* bytes that fail the bitrot hash never reach the caller's buffer;
* `want > shard_size` is rejected;
plus the pool test now asserts the allocation is reused (same pointer) and
never zeroed.
Verified: `erasure::` 213 passed, 0 failed; on a real Linux host
`erasure::` 209 and `disk::local::` 143 pass serially, and the failures
seen in a parallel full-suite run reproduce identically on unmodified main
(they are ENOSPC from a full root filesystem plus pre-existing flakes).
Not claimed: an end-to-end throughput number. The A/B on the bench host was
too noisy to attribute (one rep pair was not fully cached, and its root
filesystem filled mid-run); what is measured is that the removed memset was
4.81% of GET CPU in the pre-change profile.
Co-authored-by: heihutu <heihutu@gmail.com>
|
||
|
Zhengchao An
|
80ddd8fa7e |
fix(ecstore): roll back delete on disks that staged then errored (#4676)
fix(ecstore): roll back delete on disks that staged then errored (backlog#1158) #4300 rolls back a failed delete only on disks that returned Ok, skipping any disk that staged its rollback backup, applied the delete, and then errored -- leaving that disk deleted while its peers are restored. On rollback, fan the undo out to every online disk instead; the disk-side restore_delete_rollback is already idempotent (Ok no-op when nothing was staged), so unstaged disks are unaffected. The err.is_some() skip now applies only to the success/cleanup path. Covers both single-object and batch delete. Refs backlog#1158. |
||
|
houseme
|
8039a4ceae |
test(cache): end-to-end regressions for the body-cache hook P0s (#4675)
fix(ecstore): make body-cache hook re-registrable + add e2e regressions ODC-21 (backlog#1126): the GET body-cache hook lived in a first-wins OnceLock. When AppContext is rebuilt (config reload, test re-init) a fresh ObjectDataCacheAdapter is constructed and re-registered, but the OnceLock kept ecstore's GET probe pointed at adapter #1 while every usecase-layer fill and invalidation targeted adapter #2 — silently degrading the feature to a 0% hit rate with no error, log, or metric, and stranding entries in the unreachable cache until their TTL. Replace the slot with RwLock<Option<Arc<dyn GetObjectBodyCacheHook>>> so re-registration atomically swaps to the newest adapter, and log at WARN when a swap replaces a *different* instance (Arc::ptr_eq). RwLock over ArcSwapOption because arc-swap's RefCnt is impl<T> (Sized, thin *mut T) and cannot hold an Arc<dyn Trait> without a sized newtype wrapper; the probe reads the slot once per full-object GET but only clones an Arc, negligible next to the metadata quorum fan-out already done before the probe. Add a test-only clear_get_object_body_cache_hook so tests register/unregister deterministically. With the hook now re-registrable, add true end-to-end regressions that drive get_object_reader (not the full_object_plaintext_len predicate) against a real erasure-coded, genuinely-compressed object via the blackbox make_local_set_disks harness, with a stand-in hook playing the app-layer cache (the injection point production uses; the adapter itself lives above ecstore). These close the gap the predicate-only tests left — a caller that opens a new shortcut serving the cached body directly, the original form of both P0s: - backlog#1108: a raw_data_movement_read must yield the STORED (compressed) bytes, never the cached plaintext. - backlog#1109: a compressed cache hit must publish the DECOMPRESSED length as object_info.size (the UploadPartCopy invariant), with the streamed length matching. - backlog#1146: a restore read (restore_request.days) must serve STORED bytes, not the cache. Mutation-verified each e2e test bites: dropping the raw_data_movement_read gate serves plaintext (fails #1108); removing the hit-site size republication publishes 2972 vs 660000 (fails #1109); dropping the restore gate serves plaintext (fails #1146). Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
f83f9ada13 |
fix(ecstore): reclaim page cache after io_uring reads (#4662)
fix(ecstore): io_uring reads must reclaim the page cache like StdBackend (backlog#1145) `StdBackend::pread_bytes` calls `fadvise(DONTNEED)` over the range it just read whenever `should_reclaim_file_cache_after_read(length)` holds — `RUSTFS_OBJECT_FILE_CACHE_RECLAIM_READ_ENABLE` (on by default) above a 4 MiB threshold. Large object reads are usually cold, and leaving them resident evicts everything else, so this is a deliberate policy rather than a side effect of how StdBackend happens to read. `pread_uring` and `pread_uring_direct` never did it. Enabling io_uring on a disk therefore turned the policy off silently: shard reads at or above the threshold stayed resident and grew the page cache without bound. This is the same class of mistake as the O_DIRECT loss fixed earlier — copying the read itself while dropping the policy attached to it. It also badly distorts any benchmark. An end-to-end warp GET A/B on a 16-core host (4 MiB objects, conc 64) showed io_uring at 8782 MiB/s against StdBackend's 116 MiB/s. That is not a 76x speedup: with io_uring the run issued *zero* device reads and grew the page cache, while StdBackend read 2950 MiB from the device and shrank it. The two legs were not running the same policy. (Disabling the mmap read path changed nothing — 1.01x — so the mmap-vs-pread difference was not the cause either.) Both io_uring read paths now reclaim the range they read, with the same gate, the same range, and the same error handling as StdBackend. O_DIRECT should leave nothing resident anyway, but a filesystem that quietly buffered the read still has to honour the policy, so `pread_uring_direct` reclaims too. The test measures the policy, not the call: it reads an 8 MiB file through each backend and asks `mincore(2)` how much stayed resident. Crucially it also asserts the reclaim-disabled case leaves the range resident — without that gate, a backend that never reclaimed would pass silently. Verified: with the fix removed, the test fails with "uring: reclaim is on ... 2048/2048 pages remain"; with the fix it passes. Verified on a real Linux host (16-core, real io_uring): clippy --tests -D warnings clean; disk::local tests 143 passed, 0 failed. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
Henry Guo
|
99eef032c6 |
fix(ecstore): restore Windows async read trait import (#4652)
* fix(ecstore): restore Windows async read trait import * fix(ecstore): gate async read import to non-Unix |
||
|
houseme
|
15b8b13698 |
feat(ecstore): cache part-file descriptors for io_uring reads (#4658)
* feat(ecstore): run one io_uring ring per shard on each disk (backlog#1145)
A buffered read that hits the page cache completes inline inside
`io_uring_enter`, so the thread driving a ring performs that read's
memcpy. One ring per disk therefore capped cache-hit reads at a single
core's memory bandwidth: measured on a 16-core host, one driver thread sat
pinned at 100% CPU while throughput stayed flat at ~5 GB/s regardless of
read size, against 50 GB/s for the blocking-pool baseline.
rustfs/uring#6 taught the driver to hold N independent rings, each with its
own thread, pending table, backpressure semaphore, and eventfd. Wire it up:
`UringBackend::try_new` now calls `probe_and_start_sharded`, and
`RUSTFS_IO_URING_SHARDS` selects the count per disk.
The default is a quarter of the available parallelism clamped to `1..=4`,
because the cost is `disks × shards` driver threads (each normally blocked
in `poll(2)`). Any override is clamped to `1..=16`, so a mistyped value can
neither disable the driver (0) nor spawn threads without bound; an
unparseable value falls back to the default.
Effect (warm page cache, 16-core, rustfs/uring's concurrent_pread_bench):
1 MiB, conc 8: 1 shard 4911 MB/s -> 8 shards 47361 MB/s (9.6x);
the blocking-pool baseline is 50662 MB/s
64 KiB, conc 32: StdBackend 153678 IOPS, p999 3030 us
8 shards 345402 IOPS, p999 897 us
64 KiB, conc 128: StdBackend 135155 IOPS, p999 10716 us
8 shards 389047 IOPS, p999 4092 us
Sharding removes the throughput deficit *and* keeps io_uring's tail-latency
advantage, rather than trading one for the other.
Unchanged: io_uring read stays gray-off by default
(`RUSTFS_IO_URING_READ_ENABLE`), reads are byte-for-byte identical to
StdBackend, the per-disk degradation latches and probe cache (backlog#1101)
and the O_DIRECT tiered fallback (backlog#1102) all still apply. Rings stay
per-disk, so a stalled disk cannot starve another disk's rings
(backlog#1055). Bumps the rustfs-uring pin to the merged #6 commit.
Verified on a real Linux host (16-core, real io_uring): cargo clippy
--tests -D warnings clean; disk::local tests 132 passed, 0 failed —
including the existing io_uring and O_DIRECT cases now running on the
sharded driver, plus a new test covering the shard-count default, override,
and clamping.
Co-Authored-By: heihutu <heihutu@gmail.com>
* feat(ecstore): cache part-file descriptors for io_uring reads (backlog#1145)
`pread_uring` opened the file on the blocking pool for every read, so each
read paid a `spawn_blocking` round trip — the very thread hop io_uring
exists to avoid. Sharding the driver (backlog#1145) removed the previous
ceiling and left this as the binding cost. Measured on a 16-core host with
a 4-shard driver, warm page cache:
64 KiB, conc 8: 143942 -> 263054 IOPS (+83%), p999 240 -> 65 us
64 KiB, conc 32: 150128 -> 204876 IOPS (+36%), p999 2508 -> 871 us
64 KiB, conc 128: 129172 -> 361287 IOPS (+180%), p999 15329 -> 3046 us
1 MiB, conc 32: 33875 -> 42301 IOPS (+25%)
At 64 KiB / conc 128 the open is what masked io_uring entirely: with it,
io_uring beat StdBackend by 3.5%; without it, by 189%.
Add a bounded per-disk descriptor cache used by the io_uring read path.
A hit takes no `open` and no `spawn_blocking`, so the read never leaves the
runtime worker.
Why caching a part-file descriptor is safe:
* only `<object>/<data_dir>/part.N` reaches this backend's `pread_bytes`;
`xl.meta` — the one path replaced in place — is read through `read_all`
/ `read_metadata` and never gets here;
* part files are never rewritten in place. A replacement is always
write-new-tmp then `rename`, which swaps the inode, so a cached
descriptor can never observe a torn shard.
Why invalidation is nevertheless REQUIRED: heal reuses the existing
version's `data_dir` and renames a rebuilt shard onto the SAME part path.
A cached descriptor would keep serving the pre-heal (corrupt) inode,
defeating the heal and eroding read quorum. `delete` likewise unlinks a
part that a cached descriptor would keep readable. So `rename_data`,
`rename_file`, and `delete` all call the new
`LocalIoBackend::invalidate_cached_fds` after they mutate, and a 5s TTL
bounds the blast radius should a future mutation path forget to.
Two preamble checks the miss path runs are not silently lost on a hit:
* bounds — the driver only short-reads at EOF (it resubmits otherwise),
so `bytes.len() != length` is exactly the old `meta.len() < end_offset`
check, and now yields the same `FileCorrupt`;
* volume access — skipped while an entry is live. An unreachable disk
keeps serving already-open descriptors for at most the TTL, after which
the re-open re-runs the check. Disk health is tracked independently of
this per-read probe.
Scope: buffered io_uring reads only. The O_DIRECT path keeps opening per
read (its reads are >= 4 MiB, so the open is a small fraction), and
StdBackend is untouched — it must take the blocking hop for the pread
regardless, so caching would buy it only 2-6% while carrying the same
invalidation risk. `RUSTFS_IO_URING_FD_CACHE=false` restores open-per-read.
Verified on a real Linux host (16-core, real io_uring): clippy --tests
-D warnings clean; disk::local tests 135 passed, 0 failed. The new
heal-staleness test first asserts a read still returns the PRE-heal bytes —
proving the cache is live and the hazard real — then that invalidation makes
the healed shard visible. A second test drives `rename_file` and `delete`
through `LocalDisk` to prove those paths actually invalidate, and a unit
test pins prefix invalidation to component boundaries (`a/b` must not drop
`a/bc`).
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: heihutu <heihutu@gmail.com>
|
||
|
Zhengchao An
|
aaffd3ede8 | fix(ecstore): align Deep verifier shard geometry (#4656) | ||
|
houseme
|
8c76efead2 |
fix(cache): stop the GET body-cache hook from bypassing ReadPlan (#4654)
* fix(cache): gate GET body-cache hook to preserve ReadPlan output The ecstore GET body-cache hook serves cached full-object plaintext directly, bypassing ReadPlan/ReadTransform. That is only sound when the normal read path returns that same plaintext byte-for-byte. Two probe conditions were missing, plus two usecase-layer planner gaps. ODC-01 (backlog#1108): raw/data-movement reads. ReadPlan::build returns the STORED representation for raw_data_movement_read (e.g. compressed bytes, length = oi.size), but the cache holds the post-decompression body. Decommission (raw_data_movement_read: true) would receive decompressed plaintext where raw compressed bytes are required, silently corrupting the destination pool. ODC-02 (backlog#1109): compressed objects. ReadTransform::Compressed rewrites object_info.size to the decompressed length; on a hook hit object_info is returned unchanged, so object_info.size is the compressed size while the stream carries the decompressed body. UploadPartCopy then uses src_info.size as the copy length and truncates the part. Fix: gate the hook probe with should_probe_body_cache_hook, refusing raw_data_movement_read, data_movement, and compressed objects, mirroring the conditions get_small_object_direct_memory_decision already applies. ODC-33 (backlog#1138): build_get_object_body_cache_plan lacked the is_remote() exclusion the ecstore hook enforces; add it so transitioned (remote-tier) objects are excluded uniformly. ODC-C1 (backlog#1142): zero-length bodies save no I/O (ecstore returns an empty body before the hook probe) yet the planner admitted them; change the guard to response_content_length <= 0 so they plan Skip, mirroring should_buffer_get_object_in_memory_with_threshold. Tests: body_cache_hook_gate_tests (4) cover plain-probe plus raw/data-movement/compressed skips; planner gains plan_skips_remote_transitioned_objects and plan_skips_zero_length_objects. Co-Authored-By: heihutu <heihutu@gmail.com> * fix(cache): allow compressed bodies via a fail-closed read allow-list The body-cache hook probe was gated by a deny-list that refused compressed objects outright, which cost the cache every compressed body — a growing share of stored data. Replace it with an allow-list that returns the exact plaintext length a hit may serve, or None. full_object_plaintext_len() answers a single question: would the normal ReadPlan produce this object's complete plaintext, and under which size? Compressed objects now qualify, and the hit site publishes the returned length as object_info.size, reproducing the contract ReadTransform:: Compressed establishes. A hit whose body length disagrees is refused and falls through to the erasure read. This also closes a gate the deny-list only covered by accident: a restore read forces ReadPlan down the Plain branch, so a compressed object yields STORED bytes under its compressed size. Refusing compressed objects hid that; admitting them exposes it, so restore reads are refused explicitly. Being fail-closed, a newly added ReadPlan branch bypasses the cache by default rather than silently serving the wrong representation — the structural defect behind both backlog#1108 and backlog#1109. Refs: backlog#1108, backlog#1109 Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
GatewayJ
|
a269f8df05 | fix(ecstore): require write quorum for metadata early stop (#4300) | ||
|
houseme
|
fe682e16e9 |
feat(ecstore): run one io_uring ring per shard on each disk (backlog#1145) (#4653)
A buffered read that hits the page cache completes inline inside
`io_uring_enter`, so the thread driving a ring performs that read's
memcpy. One ring per disk therefore capped cache-hit reads at a single
core's memory bandwidth: measured on a 16-core host, one driver thread sat
pinned at 100% CPU while throughput stayed flat at ~5 GB/s regardless of
read size, against 50 GB/s for the blocking-pool baseline.
rustfs/uring#6 taught the driver to hold N independent rings, each with its
own thread, pending table, backpressure semaphore, and eventfd. Wire it up:
`UringBackend::try_new` now calls `probe_and_start_sharded`, and
`RUSTFS_IO_URING_SHARDS` selects the count per disk.
The default is a quarter of the available parallelism clamped to `1..=4`,
because the cost is `disks × shards` driver threads (each normally blocked
in `poll(2)`). Any override is clamped to `1..=16`, so a mistyped value can
neither disable the driver (0) nor spawn threads without bound; an
unparseable value falls back to the default.
Effect (warm page cache, 16-core, rustfs/uring's concurrent_pread_bench):
1 MiB, conc 8: 1 shard 4911 MB/s -> 8 shards 47361 MB/s (9.6x);
the blocking-pool baseline is 50662 MB/s
64 KiB, conc 32: StdBackend 153678 IOPS, p999 3030 us
8 shards 345402 IOPS, p999 897 us
64 KiB, conc 128: StdBackend 135155 IOPS, p999 10716 us
8 shards 389047 IOPS, p999 4092 us
Sharding removes the throughput deficit *and* keeps io_uring's tail-latency
advantage, rather than trading one for the other.
Unchanged: io_uring read stays gray-off by default
(`RUSTFS_IO_URING_READ_ENABLE`), reads are byte-for-byte identical to
StdBackend, the per-disk degradation latches and probe cache (backlog#1101)
and the O_DIRECT tiered fallback (backlog#1102) all still apply. Rings stay
per-disk, so a stalled disk cannot starve another disk's rings
(backlog#1055). Bumps the rustfs-uring pin to the merged #6 commit.
Verified on a real Linux host (16-core, real io_uring): cargo clippy
--tests -D warnings clean; disk::local tests 132 passed, 0 failed —
including the existing io_uring and O_DIRECT cases now running on the
sharded driver, plus a new test covering the shard-count default, override,
and clamping.
Co-authored-by: heihutu <heihutu@gmail.com>
|
||
|
houseme
|
baf7e899b9 |
fix(ecstore): bound every walk filesystem call by the stall budget (#4650)
The listing path no longer has a wrapper-level total timeout, so any filesystem call the walk makes without a stall bound would have no liveness bound at all. The first pass covered list_dir and read_metadata but left four reads unbounded: the volume access() probe and fs::metadata() in walk_dir, the xl.meta read in its dir-object branch, and is_empty_dir() in scan_dir. Split the helper so reads that do not yield a Result go through the same budget, and route the four remaining calls through it. A hung drive now fails those reads with DiskError::Timeout instead of parking the walk until the merge consumer's own stall timeout aborts it. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
8b233cdd94 |
fix(ecstore): keep O_DIRECT for eligible reads via native io_uring (backlog#1102) (#4649)
fix(ecstore): keep O_DIRECT for eligible reads via native io_uring (rustfs/backlog#1102)
Wire ecstore's io_uring read backend to rustfs-uring's native aligned
O_DIRECT read (`read_at_direct`, merged in rustfs/uring#3), so an
O_DIRECT-eligible read keeps BOTH io_uring's async submission AND
O_DIRECT's page-cache bypass instead of trading one for the other.
`UringBackend::pread_bytes` now selects the read shape by a tiered,
per-disk-latched ladder — never a blanket downgrade:
1. io_uring latched off for this disk (backlog#1101) -> StdBackend.
2. O_DIRECT-eligible + native path usable -> pread_uring_direct: open
the file O_DIRECT, probe the device alignment once (cached), and
read via read_at_direct. Best path: async + no cache pollution.
3. O_DIRECT-eligible but the fs refused O_DIRECT earlier (latched)
-> StdBackend aligned path (which itself degrades to buffered).
4. non-O_DIRECT read -> buffered pread_uring.
Latching keeps a failure from being re-attempted per-read: an fs that
refuses O_DIRECT (EINVAL/EOPNOTSUPP on open) latches the native direct
path off for that disk; a restriction-class errno from the read latches
io_uring off wholesale (backlog#1101). Any other error falls back for
that one read without masking a genuine data problem as a permanent
downgrade. The alignment comes from the existing probe_direct_io_align
(statx STATX_DIOALIGN, default 4096) and is cached in a per-disk
OnceLock so the probe runs at most once.
This replaces the interim guard that routed every O_DIRECT-eligible read
to StdBackend (which disabled io_uring on the hottest shard reads); the
native path is now the default and StdBackend is only a fallback tier.
Bumps the rustfs-uring pin to the merged #3 commit.
Verified on a real Linux host (16-core, real io_uring + O_DIRECT):
cargo check --tests, clippy -D warnings, and disk::local tests all pass
(128 passed, 0 failed), including uring_preserves_o_direct_for_eligible_reads
across unaligned ranges.
Co-authored-by: heihutu <heihutu@gmail.com>
|
||
|
houseme
|
399461c33e |
fix(ecstore): bound listing walks by drive stall, not total duration (#4647)
Foreground S3 listings wrapped the entire walk_dir stream in a single wall-clock timeout (RUSTFS_DRIVE_WALKDIR_TIMEOUT_SECS, default 5s). That budget measured how much data the walk had to produce rather than whether the drive was still answering, so a healthy but large prefix on slow media returned 500 InternalError / "Io error: timeout" to the client. The timer also kept running while the walk was blocked writing to a slow consumer, charging merge-side backpressure to the producer. WalkDirOptions::stall_timeout_ms already carried the right semantics but was only honored by the remote-disk RPC walk; LocalDisk::walk_dir ignored it entirely. A single-drive deployment therefore had no way to distinguish a hung drive from a big directory. Teach LocalDisk::walk_dir to bound each individual drive read with the stall budget, defaulting it from RUSTFS_DRIVE_WALKDIR_STALL_TIMEOUT_SECS when the caller does not pin one, and let the foreground listing path skip the wrapper-level total timeout. A walk that keeps making progress now runs to completion; a drive that stops answering still fails with DiskError::Timeout. Time spent blocked on the consumer stays outside the budget. Heal and rebalance walks already skipped the total timeout and previously ran unbounded on local drives. Give them an explicit, generous 60s stall budget so this change does not tighten them from "no bound" to the 5s default. Fixes #4644 Refs #2999, #3001 Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
0ad4a26056 |
fix(ecstore): keep O_DIRECT for eligible reads when io_uring is enabled (backlog#1102) (#4645)
fix(ecstore): keep O_DIRECT for eligible reads when io_uring is enabled (rustfs/backlog#1102) UringBackend::pread_uring opens the file buffered, so with io_uring enabled an O_DIRECT-eligible read (RUSTFS_OBJECT_DIRECT_IO_READ_ENABLE + length >= the threshold) silently lost O_DIRECT and polluted the page cache — a behavior change the moment io_uring was turned on for a disk that uses O_DIRECT. The io_uring backend is gray-off by default, so no deployment is affected, but the gap had to close before it can be enabled anywhere O_DIRECT is in use. Route O_DIRECT-eligible reads back through StdBackend's aligned path (which itself falls back to buffered when the filesystem rejects O_DIRECT). A native aligned O_DIRECT read in the driver remains part of rustfs/backlog#1102. Adds uring_preserves_o_direct_for_eligible_reads: with BOTH flags on and the threshold at 1, unaligned ranges still return exactly the requested bytes. Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
Zhengchao An
|
dc3099bf0f |
feat(ecstore): run bucket operations on the store's own instance context (#4642)
backlog#1052 S7 — the final piece: full bucket-namespace isolation between embedded servers in one process. Server B's requests resolved B's own ECStore (per-server dispatch landed earlier), but the store's bucket operations still went through ambient process facades, so both servers effectively operated on the FIRST server's disks and metadata: - LocalPeerS3Client::local_disks_for_pools() called all_local_disk() (the ambient disk registry = the first published store's context), so list/make/delete/heal bucket scanned and wrote the wrong volumes. - BucketMetadata::save() persisted through the ambient object handle, so a second server's bucket metadata landed in the first server's .rustfs.sys; set/remove/get/created_at all used the ambient metadata system. Now the whole chain is bound to the owning store's InstanceContext: - S3PeerSys/LocalPeerS3Client gain *_with_instance_ctx constructors and operate on that context's registered disks; ECStore::new (and the test store builder) pass the store's context. The legacy constructors keep the bootstrap default. - BucketMetadata::save_with_store persists through an explicit store; BucketMetadataSys::persist_and_set uses the system's own api handle. - metadata_sys gains instance-scoped variants (get_in / created_at_in / set_bucket_metadata_in / remove_bucket_metadata_in) that resolve the context's metadata system and fall back to the ambient default before the instance cell is initialized (early startup, unchanged behavior). - The store's bucket handlers (make/get_info/list/delete + the table-bucket delete guard and emptiness check) use the per-context variants and this instance's disks. Acceptance (e2e): two embedded servers with different credentials are now isolated end to end — each authenticates only its own key, neither sees the other's buckets or objects, and both data planes stay intact. The embedded module doc drops the shared-IAM caveat. 579 ecstore bucket/metadata/peer regressions plus the embedded basic and deferred-IAM e2e stay green. |
||
|
houseme
|
e6e4aef45b |
test(ecstore): cover the io_uring per-disk probe cache hit (backlog#1101) (#4641)
test(ecstore): cover the io_uring per-disk probe cache hit (rustfs/backlog#1101) Follow-up to #4635. The per-disk negative probe cache was implemented but had no dedicated test. Add uring_probe_cache_skips_known_unsupported_disk: a root recorded in URING_UNSUPPORTED_DISKS makes try_new return None without a fresh probe. Completes the #1101 acceptance ("cache hit does not re-probe"). Co-authored-by: heihutu <heihutu@gmail.com> |
||
|
houseme
|
d1daa97ac5 |
fix(ecstore): keep local path startup fail-fast under Kubernetes (#4640)
The startup topology convergence auto-detection (added in #4631) checked Kubernetes before endpoint style, so a local path deployment (single or multi-drive, non-distributed) running inside Kubernetes was classified as orchestrated with an effectively unbounded wait window. Path-style endpoints have no hostnames to resolve, so the wait never actually triggers and runtime behavior is unchanged, but the startup log then advertised mode=orchestrated / wait_timeout=MAX for a purely local deployment, which is misleading and contradicts the documented policy (local path endpoints -> fail-fast). Order the auto-detection by endpoint style first: only distributed URL endpoints, which resolve hostnames, pick orchestrated (Kubernetes) or bounded (otherwise); local path endpoints stay fail-fast regardless of the platform. Update the doc comment to match and add a policy case locking Kubernetes + local path to fail-fast. Co-authored-by: heihutu <heihutu@gmail.com> |