Commit Graph

17 Commits

Author SHA1 Message Date
houseme b44e82fef1 fix(notify): restore webhook HTTPS target initialization (#5060)
Restore the workspace reqwest default feature stack for RustFS outbound HTTPS clients, while keeping per-crate extra APIs such as json, stream, and multipart explicit. Lazily initialize the notification runtime from admin target access when RUSTFS_NOTIFY_ENABLE=true is already effective, and add regression coverage for HTTPS webhook custom CA handling and target-list visibility.

Fixes #5052.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-20 19:55:07 +08:00
houseme 56179210ab chore(deps): simplify dependency features (#4890)
* chore(deps): remove redundant dependency features

Remove manifest feature entries that are implied by other requested features in the same dependency declaration.

Verified that the resolved Cargo feature graph is unchanged after the cleanup.

Co-Authored-By: heihutu <heihutu@gmail.com>

* chore(deps): narrow tokio and reqwest features

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-16 05:20:43 +00:00
houseme f3a7a4b0da chore(deps): localize workspace dependency features (#4888)
Move workspace-level dependency feature lists into the member crates that consume each dependency while keeping required default-features flags at the workspace root.

Also refresh starshard to 2.2.2 via cargo update and cargo upgrade --exclude ratelimit.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-16 03:55:27 +00:00
houseme 2247823200 fix(runtime): remove tokio io-uring feature and add regression guard (#4364)
Drop the [target.'cfg(target_os = "linux")'.dependencies] tokio
"io-uring" feature from 6 crates and the rustfs binary. Because
.cargo/config.toml enables --cfg tokio_unstable globally, this feature
switched every Linux build's file I/O onto tokio's io_uring runtime
backend. Restricted Linux environments (Docker default seccomp, gVisor,
proot, old kernels) reject io_uring_setup with EACCES/ENOSYS, which
tokio surfaced as PermissionDenied and RustFS reported as
DiskAccessDenied at startup.

Add scripts/check_no_tokio_io_uring.sh so the feature cannot silently
return: it fails on any tokio dependency line enabling "io-uring", while
still allowing a future application-level io-uring crate dependency.
Wire it into make pre-commit/pre-pr/dev-check and CI.

Tracking: rustfs/backlog#890 (parent rustfs/backlog#897)

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-07 17:47:59 +08:00
Zhengchao An e1c1d8edcb test: expand snapshot coverage (#4024) 2026-06-29 07:56:37 +08:00
houseme 0a00d8d500 perf(server): lighten internode data-plane stack (#3735)
* refactor(server): split internode dispatch scaffold

* test(server): cover internode dispatch prefix split

* refactor(server): name internode stack boundaries

* perf(server): skip internode request logging layer

* perf(server): skip internode trace layer

* perf(server): use lite internode request context

* feat(metrics): track internode rpc duration

* feat(ecstore): add put object stage summary logs

* test(metrics): update internode descriptor expectations

* fix(server): tighten internode path matching

* fix(pr): address review follow-up comments

* style(ecstore): simplify commit tail duration field

* refactor(ecstore): group put stage summary fields

* refactor(ecstore): inline put stage summary log

* fix(s3): return storage class for object attributes

* merge: sync latest main and resolve object attributes conflict

* fmt

* fix(server): remove duplicate rpc imports

* build(deps): bump memmap2 for RUSTSEC-2026-0186

* fix(s3select): align object_store with datafusion

* chore(deps): prune workspace dependencies

* perf(fuzz): optimize CI runtime with build/run split and matrix parallelization

  Separate fuzz harness compilation from execution to eliminate redundant
  builds across targets. Introduce matrix-based parallel execution for
  PR smoke and nightly fuzz jobs.

  Changes:

  - Split CI workflow into `fuzz-build` (compile once) and matrix run jobs
    (`pr-fuzz-smoke`, `nightly-fuzz-corpus`) that execute targets in parallel
  - Add `BUILD_ONLY` mode to run_ci_targets.sh / run_nightly_targets.sh
  - Add run_single_target.sh for matrix jobs (no build phase)
  - Optimize `local_metadata` fuzz target: reduce prefix iterations from
    8-10 (4 functions each) to 5 critical prefixes (parser-only), cutting
    per-iteration cost by ~3-5x
  - Move archive path validation (`validate_extract_relative_path`,
    `normalize_extract_entry_key`) from `rustfs` to `rustfs-utils::path`,
    eliminating `rustfs` binary crate dependency from fuzz harness
  - Remove `rustfs` from fuzz/Cargo.toml (drops significant transitive deps)
  - Add unit tests for archive path validation in rustfs-utils
  - Update fuzz/README.md with new workflow and script documentation

  Expected CI improvement: PR smoke wall-clock from ~120min (frequent
  timeout) to ~40min; nightly from ~180min to ~60min.

* refactor(fuzz): consolidate scripts and fix prefix test alignment

Replace three duplicated shell scripts (run_ci_targets.sh,
run_nightly_targets.sh, run_single_target.sh) with a single
parameterized run.sh that supports BUILD_ONLY, SKIP_BUILD, and
MAX_TOTAL_TIME environment variables.

Fix local_metadata fuzz target prefix testing: replace always-true
'len > 0' guard with lengths aligned to xl.meta binary layout
(4/5/8/12 bytes for magic+version+header fields). Remove redundant
empty-slice test.

Hoist RUSTFLAGS to workflow top-level env to eliminate per-job
duplication. Update README with unified script documentation.

Net: -118 lines, zero functionality loss.

* perf(fuzz): optimize CI runtime with build/run split and matrix parallelization

Restructure fuzz CI workflow to eliminate redundant compilation and
run targets in parallel via matrix strategy.

Workflow changes:
- Split into fuzz-build (compile once) and matrix run jobs
- PR smoke: 3 targets parallel, 60s each, timeout 30min (was 120min)
- Nightly: 3 targets parallel, 300s each, timeout 60min (was 180min)
- Pass compiled harness via actions/artifact between jobs
- Hoist RUSTFLAGS to workflow top-level env

Script consolidation:
- Replace 3 duplicated scripts with single parameterized run.sh
- Supports BUILD_ONLY, SKIP_BUILD, MAX_TOTAL_TIME env vars

Target optimizations:
- Remove rustfs binary crate from fuzz dependencies (was pulling
  979 transitive deps); move archive path validation to rustfs-utils
- Optimize local_metadata: reduce prefix iterations from 8-10x4
  calls to 5 prefixes with parser-only (no decompress), aligned
  with xl.meta binary layout (4/5/8/12 bytes)
- Add unit tests for archive path validation in rustfs-utils
- Update fuzz/README.md with unified script documentation

Expected: PR smoke wall-clock from ~120min (frequent timeout)
to ~40min; nightly from ~180min to ~60min.

* fix(rpc): resolve internode metrics via app context

* fmt

* ci: speed up fuzz smoke artifact restore

---------

Signed-off-by: houseme <housemecn@gmail.com>
2026-06-23 21:36:39 +08:00
houseme 53ff19dd2d fix(kms): harden local master key storage (#3589) 2026-06-19 08:50:35 +08:00
escapecode 7a8514bdfa feat(sftp): add macOS and Windows platform support (#3372)
The session watchdog now selects its detection method per
platform. It previously probed kernel TCP state through a
Linux-only procfs path, so on macOS every healthy idle session
was killed within a minute, and on Windows the watchdog never
spawned at all, leaving wedged sessions with no cleanup. Linux
keeps its fast-kill watchdog unchanged. Other platforms get a
silence-only backstop that kills a session only at the
documented 30-minute idle ceiling.

The host-key loader now has a Windows arm. It loads OpenSSH
format host keys from the configured directory and logs a
one-time warning to restrict NTFS ACLs on the key directory,
the same operator-managed approach FTPS, WebDAV, KMS, and IAM
already use on Windows. Startup previously aborted with
UnsupportedPlatform because the Unix mode-bit permission check
has no Windows equivalent. tokio's io-uring feature is now
enabled only in Linux builds. io-uring is a Linux kernel
interface and enabling it unconditionally broke the Windows
build.

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 18:08:23 +08:00
安正超 03eb10b07f feat: add KMS config redaction safeguards (#3303) 2026-06-09 14:08:58 +00:00
houseme f77ccd5b23 chore: move mcp crate to standalone repository (#2578) 2026-04-17 12:42:26 +00:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
唐小鸭 7a42af922d Refactor: refactor SSE layer and KMS subsystem (#1703)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-04 16:10:33 +08:00
heihutu db253c01a9 refactor: replace chrono with jiff for time handling (#1582)
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-22 17:24:56 +08:00
houseme 1a4e95e940 chore: remove unused dependencies to optimize build (#1117) 2025-12-11 18:13:26 +08:00
Copilot b7964081ce Fix KMS configuration synchronization across cluster nodes (#855)
* Initial plan

* Add KMS configuration persistence to cluster storage

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Apply code formatting to KMS configuration changes

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* add comment

* fix fmt

* fix

* Fix overlapping dependabot cargo configurations

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* improve code for comment and replace  `Once_Cell` to `std::sync::OnceLock`

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com>
2025-11-16 00:05:03 +08:00
houseme 9b7f4d477a Fix Tokio Runtime Initialization: Remove Private API Usage and Ensure IO Enabled (#587)
* fix: remove code

* improve code for tokio runtime config

* improve code for main

* fix: add tokio enable_all

* upgrade version

* improve for Cargo.toml
2025-09-24 22:23:31 +08:00
guojidan 9ddf6a011d feature: support kms && encryt (#573)
* feat(kms): implement key management service with local and vault backends

Signed-off-by: junxiang Mu <1948535941@qq.com>

* feat(kms): enhance security with zeroize for sensitive data and improve key management

Signed-off-by: junxiang Mu <1948535941@qq.com>

* remove Hashi word

Signed-off-by: junxiang Mu <1948535941@qq.com>

* refactor: remove unused request structs from kms handlers

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-22 17:53:05 +08:00