Commit Graph

22 Commits

Author SHA1 Message Date
houseme 56179210ab chore(deps): simplify dependency features (#4890)
* chore(deps): remove redundant dependency features

Remove manifest feature entries that are implied by other requested features in the same dependency declaration.

Verified that the resolved Cargo feature graph is unchanged after the cleanup.

Co-Authored-By: heihutu <heihutu@gmail.com>

* chore(deps): narrow tokio and reqwest features

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-16 05:20:43 +00:00
houseme f3a7a4b0da chore(deps): localize workspace dependency features (#4888)
Move workspace-level dependency feature lists into the member crates that consume each dependency while keeping required default-features flags at the workspace root.

Also refresh starshard to 2.2.2 via cargo update and cargo upgrade --exclude ratelimit.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-16 03:55:27 +00:00
Zhengchao An 763f246f8a test(ecstore): add shared MockWarmBackend test utility for lifecycle and tier tests (#4716)
* test(ecstore): extract shared MockWarmBackend into a test-util feature (backlog#1148 ilm-6)

The tier/lifecycle integration tests carried two byte-for-byte copies of an
in-memory WarmBackend mock — one in crates/scanner/tests and one in
rustfs/src/app — plus duplicated register_mock_tier and polling helpers. Both
implemented the same ecstore WarmBackend trait.

Consolidate them into ecstore behind a new `test-util` feature, exposed via the
`rustfs_ecstore::api::tier::test_util` facade:

- MockWarmBackend: in-memory WarmBackend with an operation log (for ordering
  assertions such as "local delete precedes remote remove") and fault injection
  (FaultConfig): unreachable, HTTP 5xx, credential rejection, injected latency,
  plus external_remove to simulate an out-of-band remote deletion.
- register_mock_tier / register_mock_tier_backend: register the mock into any
  TierConfigMgr handle (the global manager used by scanner tests or a
  per-instance one used by the app tests).
- xl.meta transition assertion helpers: read_transition_meta,
  assert_transition_meta_consistent (cross-shard consistency of the
  status/tier/remote-key/remote-version-id tuple plus free-version count), and
  free_version_count.
- polling helpers: wait_for_remote_absence, wait_for_object_count,
  wait_for_free_version_absence.

Both existing copies now consume this single definition; `rg 'struct
MockWarmBackend'` collapses to one. The feature is enabled only from
[dev-dependencies], so it never links into the production binary (resolver 3).

Designed for downstream ilm-8 (restore lifecycle) and ilm-11 (tier fault
injection matrix). Coordinates with #4706 (ilm-2), which adds op-logging to the
scanner mock — that op-logging is now part of this shared surface, so #4706
should rebase onto it.

Refs rustfs/backlog#1148 (ilm-6), rustfs/backlog#1155.

* test(ecstore): fix shared MockWarmBackend usage after main merge

- Access stored objects via MockWarmBackend::contains() instead of the now
  private inner objects map (fixes E0609 after the shared test-util refactor).
- Drop dead ReadCloser/ReaderImpl/DiskAPI imports and the unused
  transition_api test re-exports the mock extraction left behind.
- Reword the scanner/rustfs test-util dependency comments so they no longer
  embed the literal rustfs_ecstore:: path that trips the ECStore
  architecture-migration guard.
2026-07-12 00:20:45 +08:00
houseme 2247823200 fix(runtime): remove tokio io-uring feature and add regression guard (#4364)
Drop the [target.'cfg(target_os = "linux")'.dependencies] tokio
"io-uring" feature from 6 crates and the rustfs binary. Because
.cargo/config.toml enables --cfg tokio_unstable globally, this feature
switched every Linux build's file I/O onto tokio's io_uring runtime
backend. Restricted Linux environments (Docker default seccomp, gVisor,
proot, old kernels) reject io_uring_setup with EACCES/ENOSYS, which
tokio surfaced as PermissionDenied and RustFS reported as
DiskAccessDenied at startup.

Add scripts/check_no_tokio_io_uring.sh so the feature cannot silently
return: it fails on any tokio dependency line enabling "io-uring", while
still allowing a future application-level io-uring crate dependency.
Wire it into make pre-commit/pre-pr/dev-check and CI.

Tracking: rustfs/backlog#890 (parent rustfs/backlog#897)

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-07 17:47:59 +08:00
Zhengchao An 8a0617865b refactor(replication): route app storage through ecstore (#4251) 2026-07-04 02:00:28 +08:00
Zhengchao An 18d89d1e31 refactor(replication): route metadata contracts through crate (#4160) 2026-07-02 08:48:42 +08:00
escapecode 7a8514bdfa feat(sftp): add macOS and Windows platform support (#3372)
The session watchdog now selects its detection method per
platform. It previously probed kernel TCP state through a
Linux-only procfs path, so on macOS every healthy idle session
was killed within a minute, and on Windows the watchdog never
spawned at all, leaving wedged sessions with no cleanup. Linux
keeps its fast-kill watchdog unchanged. Other platforms get a
silence-only backstop that kills a session only at the
documented 30-minute idle ceiling.

The host-key loader now has a Windows arm. It loads OpenSSH
format host keys from the configured directory and logs a
one-time warning to restrict NTFS ACLs on the key directory,
the same operator-managed approach FTPS, WebDAV, KMS, and IAM
already use on Windows. Startup previously aborted with
UnsupportedPlatform because the Unix mode-bit permission check
has no Windows equivalent. tokio's io-uring feature is now
enabled only in Linux builds. io-uring is a Linux kernel
interface and enabling it unconditionally broke the Windows
build.

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 18:08:23 +08:00
安正超 69549634ea refactor(config): migrate scanner config consumer (#3356) 2026-06-11 17:30:26 +08:00
安正超 87968275a7 refactor(storage): route maintenance inventory reads (#3337) 2026-06-11 10:32:48 +08:00
houseme dcbffb084f chore(deps): refresh workspace deps and linux fs_type gating (#3030)
* chore(deps): refresh workspace deps and linux fs_type gating

- refresh workspace dependency pins and lockfile updates

- remove now-unused crate dependency entries in multiple Cargo.toml files

- enable profiling export defaults in config and scripts/run.sh

- gate os::fs_type module/function/tests to Linux to avoid non-Linux dead_code warnings

* fix(utils): simplify fs_type linux gating

- keep fs_type module-level linux cfg in os::mod

- remove redundant linux cfg on get_fs_type and test module

* chore(deps): bump s3s git revision

- update workspace s3s dependency to rev 507e1312b211c3ddc214b03875d6fabd15d22ed5

- refresh Cargo.lock source entry for s3s

* chore(dev): allow mysql_async git source and env overrides

- allow mysql_async git source in deny.toml allow-git list

- make scripts/run.sh core env vars overrideable via existing shell env

* fix(utils): import get_fs_type in fs_type tests

- add explicit super::get_fs_type import in fs_type test module

- fix Linux E0425 unresolved function errors in unit tests

* chore(dev): tune run script observability defaults

- make profiling export env overrideable in scripts/run.sh

- set RUSTFS_OBS_SAMPLE_RATIO default from 2.0 to 1.0

- update allow-git review window comments in deny.toml

* test(obs): stabilize profiling env alias tests
2026-05-20 15:03:07 +00:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012)
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
houseme f77ccd5b23 chore: move mcp crate to standalone repository (#2578) 2026-04-17 12:42:26 +00:00
houseme 6ce24f3b63 feat(lifecycle): improve ILM compatibility and scanner runtime config (#2534)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-16 10:28:03 +00:00
houseme 28edfd6190 fix(storage): harden offline drive fail-fast paths (#2564)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-16 09:21:45 +00:00
heihutu 3c28f0a0ba feat(metrics): migrate system monitoring from rustfs-obs to rustfs-metrics (#2242)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-20 18:52:33 +08:00
evan slack b3da8ae269 feat(scanner): Add dynamic throttling presets (#2095)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: GatewayJ <835269233@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: weisd <im@weisd.in>
2026-03-10 16:12:56 +08:00
houseme 024a3107d8 build(deps): bump the dependencies group with 4 updates (#1955) 2026-02-25 19:21:20 +08:00
houseme 0b870d6301 build(deps): bump the dependencies group with 19 updates (#1745) 2026-02-07 12:22:14 +08:00
houseme 07cf2feaad fix(pprof): Fixed the problem that pprof crate does not support the window platform (#1681)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-01 00:12:55 +08:00
likewu 087f58b7c8 fix(lifecycle): lifecycle fixes (#1625)
Signed-off-by: likewu <likewu@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-31 19:14:47 +08:00
houseme 2ee81496b0 fix: deduplicate disks in capacity calculation to prevent inflation (#1656) 2026-01-30 00:03:21 +08:00
weisd 6631407416 feat: Add RustFS Scanner Module and Multiple Bug Fixes (#1579) 2026-01-22 13:39:38 +08:00