mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 08:18:18 +00:00
houseme
60ad15a7f9
* fix(obs): remove the dial9 task-dump switch that could never do anything Measured on a bench host (Linux x86_64) against the code merged in #4663: with `RUSTFS_RUNTIME_DIAL9_TASK_DUMP_ENABLED=true`, a `dial9-taskdump` build, and `--cfg tokio_taskdump`, dial9 recorded **zero** TaskDump events. dial9 captures a task dump only for futures it wrapped itself — those spawned through `dial9_tokio_telemetry::spawn`, which is where `TaskDumped<F>` gets applied. `tokio::spawn` gets no wrapper, and RustFS spawns with `tokio::spawn` throughout. Same workload, same binary, only the spawner changed: tokio::spawn -> 0 dumps dial9::spawn -> 14709 dumps, all with callchains Upstream documents this (README line 151) and tracks the doc gap at dial9-rs/dial9#477. I did not read it before wiring `with_task_dumps` in #4663, and so shipped exactly the kind of lying configuration knob that PR set out to delete. Remove it: the two environment variables, the config fields, the `with_task_dumps` call, and the `dial9-taskdump` feature — whose only effect was to constrain the build to Linux while recording nothing. Re-adding it only makes sense together with migrating the paths under investigation to dial9's spawner. Tracked as D9-16 in rustfs/backlog#1157. Also drop the `--cfg tokio_taskdump` requirement from the Makefile. Measured: dumps are captured with and without it (14709 vs 14674, within noise), and upstream never asked for it. That requirement was mine, invented and untested. Cargo.lock loses tokio's `backtrace` dependency, which `tokio/taskdump` pulled in. Co-Authored-By: heihutu <heihutu@gmail.com> * docs(obs): replace guessed dial9 retention numbers with measured ones Three corrections, all to claims I wrote in #4663 without measuring them. "Under a high poll rate that budget can wrap in minutes" was a guess. Measured on a single-node 4-drive cluster under warp mixed (66 MiB/s, 110 obj/s, 32 concurrent): 13023 events/s, 0.16 MiB/s, so the default 1 GiB budget wraps after roughly 108 minutes. Even at ten times the throughput that is ~11 minutes. State the measured rate and how to scale it instead. dial9 was described as the tool for drive stalls. It is not. RustFS does disk I/O on the blocking pool and through io_uring, never on an async worker, so a slow drive never lengthens a poll. Injecting 200 ms of latency on one of four drives cut throughput by 64% and left the poll distribution unchanged (polls >= 5 ms: 49 -> 56; p999: 2.67 ms -> 2.75 ms). Enabling dial9's CPU and sched profilers does not help: sched events are per-worker only, and the CPU profiler samples on-CPU while a stalled drive is an off-CPU wait. Say so plainly, and point at the `rustfs_io_*` metrics instead. What dial9 *is* good for, on the same traces: single polls of 418-625 ms with no fault injected at all — real worker stalls nothing else in the obs stack surfaces. Lead with that. Also link the two upstream issues filed for the gaps we documented: dial9-rs/dial9#658 (writer death unobservable) and #659 (worker-s3 CVEs). Measurements: rustfs/backlog#1157 (D9-11, D9-13, D9-18). Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com>
75 lines
3.2 KiB
Makefile
75 lines
3.2 KiB
Makefile
## —— Local Native Build using build-rustfs.sh script (Recommended) --------------------------------
|
|
|
|
.PHONY: build
|
|
build: ## Build RustFS binary (includes console by default)
|
|
@echo "🔨 Building RustFS using build-rustfs.sh script..."
|
|
./build-rustfs.sh
|
|
|
|
.PHONY: build-dev
|
|
build-dev: ## Build RustFS in Development mode
|
|
@echo "🔨 Building RustFS in development mode..."
|
|
./build-rustfs.sh --dev
|
|
|
|
.PHONY: build-musl
|
|
build-musl: ## Build x86_64 musl version
|
|
@echo "🔨 Building rustfs for x86_64-unknown-linux-musl..."
|
|
@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
|
|
./build-rustfs.sh --platform x86_64-unknown-linux-musl
|
|
|
|
.PHONY: build-gnu
|
|
build-gnu: ## Build x86_64 GNU version
|
|
@echo "🔨 Building rustfs for x86_64-unknown-linux-gnu..."
|
|
@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
|
|
./build-rustfs.sh --platform x86_64-unknown-linux-gnu
|
|
|
|
.PHONY: build-musl-arm64
|
|
build-musl-arm64: ## Build aarch64 musl version
|
|
@echo "🔨 Building rustfs for aarch64-unknown-linux-musl..."
|
|
@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
|
|
./build-rustfs.sh --platform aarch64-unknown-linux-musl
|
|
|
|
.PHONY: build-gnu-arm64
|
|
build-gnu-arm64: ## Build aarch64 GNU version
|
|
@echo "🔨 Building rustfs for aarch64-unknown-linux-gnu..."
|
|
@echo "💡 On macOS/Windows, use 'make build-docker' or 'make docker-dev' instead"
|
|
./build-rustfs.sh --platform aarch64-unknown-linux-gnu
|
|
|
|
|
|
## —— Profiling build (dial9 Tokio runtime telemetry) ------------------------------------------
|
|
|
|
# dial9 hooks Tokio's unstable runtime instrumentation, so it needs
|
|
# `--cfg tokio_unstable`. That flag is deliberately absent from
|
|
# .cargo/config.toml: it is not free, and release binaries do not carry it.
|
|
# Setting RUSTFLAGS here replaces (never appends to) the config-file value, and
|
|
# crates/obs/build.rs fails the build if the feature and the flag disagree.
|
|
#
|
|
# There are no task-dump or S3-upload features — see the notes in
|
|
# crates/obs/Cargo.toml for why.
|
|
DIAL9_FEATURES ?= dial9
|
|
DIAL9_RUSTFLAGS ?= --cfg tokio_unstable
|
|
|
|
.PHONY: build-profiling
|
|
build-profiling: ## Build RustFS with dial9 Tokio runtime telemetry (diagnostic builds only)
|
|
@echo "🔬 Building RustFS with dial9 telemetry (features: $(DIAL9_FEATURES))..."
|
|
@echo "⚠️ Diagnostic build: telemetry writes trace segments to disk continuously."
|
|
RUSTFLAGS="$(DIAL9_RUSTFLAGS)" cargo build --release --bin rustfs --features $(DIAL9_FEATURES)
|
|
|
|
.PHONY: build-cross-all
|
|
build-cross-all: core-deps ## Build binaries for all architectures
|
|
@echo "🔧 Building all target architectures..."
|
|
@echo "💡 On macOS/Windows, use 'make docker-dev' for reliable multi-arch builds"
|
|
@echo "🔨 Generating protobuf code..."
|
|
cargo run --bin gproto || true
|
|
|
|
@echo "🔨 Building rustfs for x86_64-unknown-linux-musl..."
|
|
./build-rustfs.sh --platform x86_64-unknown-linux-musl
|
|
|
|
@echo "🔨 Building rustfs for x86_64-unknown-linux-gnu..."
|
|
./build-rustfs.sh --platform x86_64-unknown-linux-gnu
|
|
|
|
@echo "🔨 Building rustfs for aarch64-unknown-linux-musl..."
|
|
./build-rustfs.sh --platform aarch64-unknown-linux-musl
|
|
|
|
@echo "🔨 Building rustfs for aarch64-unknown-linux-gnu..."
|
|
./build-rustfs.sh --platform aarch64-unknown-linux-gnu
|