feat(ecstore): run one io_uring ring per shard on each disk (backlog#1145) (#4653)

A buffered read that hits the page cache completes inline inside
`io_uring_enter`, so the thread driving a ring performs that read's
memcpy. One ring per disk therefore capped cache-hit reads at a single
core's memory bandwidth: measured on a 16-core host, one driver thread sat
pinned at 100% CPU while throughput stayed flat at ~5 GB/s regardless of
read size, against 50 GB/s for the blocking-pool baseline.

rustfs/uring#6 taught the driver to hold N independent rings, each with its
own thread, pending table, backpressure semaphore, and eventfd. Wire it up:
`UringBackend::try_new` now calls `probe_and_start_sharded`, and
`RUSTFS_IO_URING_SHARDS` selects the count per disk.

The default is a quarter of the available parallelism clamped to `1..=4`,
because the cost is `disks × shards` driver threads (each normally blocked
in `poll(2)`). Any override is clamped to `1..=16`, so a mistyped value can
neither disable the driver (0) nor spawn threads without bound; an
unparseable value falls back to the default.

Effect (warm page cache, 16-core, rustfs/uring's concurrent_pread_bench):

  1 MiB, conc 8:    1 shard  4911 MB/s -> 8 shards 47361 MB/s (9.6x);
                    the blocking-pool baseline is 50662 MB/s
  64 KiB, conc 32:  StdBackend 153678 IOPS, p999 3030 us
                    8 shards   345402 IOPS, p999  897 us
  64 KiB, conc 128: StdBackend 135155 IOPS, p999 10716 us
                    8 shards    389047 IOPS, p999  4092 us

Sharding removes the throughput deficit *and* keeps io_uring's tail-latency
advantage, rather than trading one for the other.

Unchanged: io_uring read stays gray-off by default
(`RUSTFS_IO_URING_READ_ENABLE`), reads are byte-for-byte identical to
StdBackend, the per-disk degradation latches and probe cache (backlog#1101)
and the O_DIRECT tiered fallback (backlog#1102) all still apply. Rings stay
per-disk, so a stalled disk cannot starve another disk's rings
(backlog#1055). Bumps the rustfs-uring pin to the merged #6 commit.

Verified on a real Linux host (16-core, real io_uring): cargo clippy
--tests -D warnings clean; disk::local tests 132 passed, 0 failed —
including the existing io_uring and O_DIRECT cases now running on the
sharded driver, plus a new test covering the shard-count default, override,
and clamping.

Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
houseme
2026-07-10 15:41:11 +08:00
committed by GitHub
parent baf7e899b9
commit fe682e16e9
2 changed files with 66 additions and 4 deletions
+1 -1
View File
@@ -145,7 +145,7 @@ metrics = { workspace = true }
# scripts/check_no_tokio_io_uring.sh allows an explicit io-uring integration;
# only the tokio "io-uring" runtime feature is banned.
[target.'cfg(target_os = "linux")'.dependencies]
rustfs-uring = { git = "https://github.com/rustfs/uring", rev = "f577ba0bfd6c3d74f9bdc54b573b8fcb45d6bc22" }
rustfs-uring = { git = "https://github.com/rustfs/uring", rev = "719b245f9bb24e838de8639c19858df2690f3ae1" }
[dev-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "test-util"] }