Commit Graph

35 Commits

Author SHA1 Message Date
Zhengchao An e468648f0f fix(utils): map remaining Linux uapi filesystem magic values
Map the remaining stable Linux uapi filesystem magic constants and keep values without a stable uapi source explicitly unknown.
2026-07-09 04:52:43 +08:00
Zhengchao An 68c3278efd test(utils): unignore drive stats platform test (#4549) 2026-07-09 03:02:13 +08:00
Zhengchao An 276a4f24c0 test(utils): extend Linux fs type mappings (#4556)
test(utils): extend linux fs type mappings
2026-07-09 03:01:57 +08:00
安正超 d933d5e938 refactor: move ecstore rebalance flow modules (#3658) 2026-06-20 17:56:11 +08:00
houseme c479f3d0cb fix(ecstore): gate rustix fs diagnostics on windows (#3267)
* fix(ecstore): gate rustix fs diagnostics on windows

* fix(ecstore): improve windows disk validation diagnostics

* build(deps): bump workspace dependency versions
2026-06-08 00:05:55 +00:00
houseme 5d74637968 fix(utils): tolerate bavail greater than bfree on Linux (#3119)
* fix(utils): tolerate bavail greater than bfree on Linux

Treat f_bavail > f_bfree as a compatibility edge case instead of fatal corruption during startup disk info probing.

- keep corruption checks for other invalid counter relationships
- log a warning and clamp reserved blocks to 0 when bavail exceeds bfree
- add linux unit coverage for the compatibility and overflow paths

Refs #3025

* fix(utils): cap bavail to bfree for Linux statfs edge

* fix(utils): rate-limit Linux statfs compatibility warning

* test(utils): cover Linux statfs capacity math

Expand Linux statfs capacity math tests around normal, equal, zero, and fully-free block counter combinations.

Refs #3025

* fix(utils): avoid statfs warning guard reallocations

* test(utils): align statfs reserved block expectations

* fmt

* fix(utils): avoid warning guard path allocation

Check the warn-once set before allocating a PathBuf so repeated bavail/bfree compatibility warnings stay low-overhead.

Refs #3025
2026-05-29 11:20:47 +00:00
weisd f9475e10dc fix(replication): preserve multipart pending state (#3058)
* Preserve multipart replication recovery state

Multipart uploads previously only scheduled replication after completion, leaving no persisted pending state for scanner recovery if the initial async work was lost. Persist the same pending replication metadata during multipart initialization and let completion evaluate the object metadata that was actually stored.

The scanner heal path also treated ordinary pending objects as delete-replication candidates. Restrict that path to delete markers and version purge state so pending objects remain eligible for object replication heal.

Constraint: Bucket replication recovery depends on persisted object metadata after the async queue is unavailable.
Rejected: Rely only on immediate completion-time scheduling | it cannot recover after process restart or worker loss.
Confidence: high
Scope-risk: moderate
Directive: Keep multipart upload initialization aligned with single PUT replication metadata semantics.
Tested: cargo fmt --all --check
Tested: cargo clippy --workspace --all-features --all-targets -- -D warnings
Tested: LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 make pre-commit
Tested: Runtime replication outage check confirmed multipart xl.meta stores PENDING status and timestamp.

* fix(replication): preserve version purge scanner state

Role-derived replication configs need target-scoped status strings before scanner heal can build per-target purge status. The duplicated replication-status assignment left version purge status unset, so scanner recovery could lose the target-level purge state.

Constraint: Scanner heal derives per-target purge decisions from version_purge_status_internal.
Rejected: Leave the duplicate as a harmless cleanup | it changes recovery behavior for role-only configs with version purge state.
Confidence: high
Scope-risk: narrow
Directive: Keep role-derived replication and version purge internal status mapping symmetric.
Tested: cargo fmt --all --check
Tested: cargo test -p rustfs-ecstore heal -- --nocapture

---------

Co-authored-by: wly <wlywly0735@126.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-24 14:24:00 +00:00
houseme 6264be437c fix(storage): add scoped timeout policy and startup fs guardrail (#3056)
* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

* style: fix cargo fmt formatting in disk_store.rs

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

Extend the TimeoutHealthAction introduced in #2996 to read_metadata,
list_dir, and disk_info operations when RUSTFS_NETWORK_MOUNT_MODE=true.
Also raises all drive operation timeouts to 60s (explicit per-operation
overrides still take precedence).

Closes #2790

* feat(startup): add unsupported filesystem policy guardrail

* chore(deps): refresh lockfile and dependency pins

* feat(ecstore): add scoped timeout health-action policy

* docs(config): document drive timeout health-action policy

* refactor(ecstore): cache timeout health policy per disk wrapper

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends (#2838)

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

* style: fix cargo fmt formatting in disk_store.rs

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

Extend the TimeoutHealthAction introduced in #2996 to read_metadata,
list_dir, and disk_info operations when RUSTFS_NETWORK_MOUNT_MODE=true.
Also raises all drive operation timeouts to 60s (explicit per-operation
overrides still take precedence).

Closes #2790

* fix(utils): map verified Linux filesystem magic values (#3051)

* fix(utils): cover sha256 checksum validation (#3052)

* fix(utils): cover sha256 checksum validation

* docs: clarify sha256 checksum validation

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>

* refactor(config): replace network mount mode with timeout profile preset

* fix(review): align fallback defaults and extend fs-type detection

* fix(review): cache timeout profile and restore probe timeout semantics

* refactor(ecstore): cache timeout health policy lookup

* perf(ecstore): cache active probe timeout per monitor task

---------

Co-authored-by: mistik <mistiklord4@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-22 06:37:30 +00:00
安正超 0d94437788 fix(utils): map verified Linux filesystem magic values (#3051) 2026-05-22 02:37:50 +00:00
houseme dcbffb084f chore(deps): refresh workspace deps and linux fs_type gating (#3030)
* chore(deps): refresh workspace deps and linux fs_type gating

- refresh workspace dependency pins and lockfile updates

- remove now-unused crate dependency entries in multiple Cargo.toml files

- enable profiling export defaults in config and scripts/run.sh

- gate os::fs_type module/function/tests to Linux to avoid non-Linux dead_code warnings

* fix(utils): simplify fs_type linux gating

- keep fs_type module-level linux cfg in os::mod

- remove redundant linux cfg on get_fs_type and test module

* chore(deps): bump s3s git revision

- update workspace s3s dependency to rev 507e1312b211c3ddc214b03875d6fabd15d22ed5

- refresh Cargo.lock source entry for s3s

* chore(dev): allow mysql_async git source and env overrides

- allow mysql_async git source in deny.toml allow-git list

- make scripts/run.sh core env vars overrideable via existing shell env

* fix(utils): import get_fs_type in fs_type tests

- add explicit super::get_fs_type import in fs_type test module

- fix Linux E0425 unresolved function errors in unit tests

* chore(dev): tune run script observability defaults

- make profiling export env overrideable in scripts/run.sh

- set RUSTFS_OBS_SAMPLE_RATIO default from 2.0 to 1.0

- update allow-git review window comments in deny.toml

* test(obs): stabilize profiling env alias tests
2026-05-20 15:03:07 +00:00
安正超 11054263c1 fix(utils): map common Linux filesystem magic values (#3023) 2026-05-20 04:00:32 +00:00
安正超 4728abcff1 fix(security): document unsafe and TLS overrides (#2835) 2026-05-06 15:09:02 +00:00
houseme 1e9c75a201 feat(ecstore): enforce local disk topology guardrails and expose device ids (#2679) 2026-04-25 06:02:02 +00:00
likewu 087f58b7c8 fix(lifecycle): lifecycle fixes (#1625)
Signed-off-by: likewu <likewu@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-31 19:14:47 +08:00
houseme 90ed75a3dc refactor(utils/os): Optimize Windows OS utilities and add safety comments (#1671)
Co-authored-by: weisd <weishidavip@163.com>
2026-01-30 16:13:42 +08:00
houseme 2ee81496b0 fix: deduplicate disks in capacity calculation to prevent inflation (#1656) 2026-01-30 00:03:21 +08:00
houseme d251b9fb35 fix: unify path handling to use S3-standard forward slashes on all platforms (#1555)
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: LeonWang0735 <wlywly0735@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-26 18:49:21 +08:00
安正超 4890fb25c1 fix: listobjects v2 pagination (#1607) 2026-01-25 09:30:08 +08:00
Jan S 68c5c0b834 Use POSIX statvfs, since statfs is not designed to be portable (#1495) 2026-01-14 16:03:32 +08:00
houseme 04f441361e replace winapi to windows crate (#1455) 2026-01-10 02:15:08 +08:00
Jan S d06397cf4a fix: try casting available blocks to a u64 on FreeBSD and OpenBSD (#1360)
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-04 11:06:14 +08:00
安正超 357cced49c Replace prints with logs and fix grammar (#437)
* refactor: replace print statements with proper logging and fix grammar

- Fix English grammar errors in existing log messages
- Add tracing imports where needed
- Improve log message clarity and consistency
- Follow project logging best practices using tracing crate

* fix: resolve clippy warnings and format code

- Fix unused import warnings by making test imports conditional with #[cfg(test)]
- Fix unused variable warning by prefixing with underscore
- Run cargo fmt to fix formatting issues
- Ensure all code passes clippy checks with -D warnings flag

* refactor: move tracing::debug import into test module

Move the tracing::debug import from file-level #[cfg(test)] into the test module itself for better code organization and consistency with other test modules

* Checkpoint before follow-up message

Co-authored-by: anzhengchao <anzhengchao@gmail.com>

* refactor: move tracing::debug import into test module in user_agent.rs

Complete the refactoring by moving the tracing::debug import from file-level #[cfg(test)] into the test module for consistency across all test files

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-08-21 05:49:40 +08:00
shiro.lee 2c0c7fafa3 fix: Optimized io::ErrorKind::NotFound error handling during Windows system startup (#181) 2025-07-12 06:54:21 +08:00
houseme 2e14b32ccd chore: Add copyright and license headers (#23)
* chore: Add copyright and license headers

This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.

* cargo fmt

* fix

* fmt

* fix clippy
2025-07-02 15:07:47 +08:00
houseme 7b891750e8 fix 2025-06-27 23:21:27 +08:00
houseme 1efcce0001 fix 2025-06-27 22:59:34 +08:00
houseme fafba030c2 fix 2025-06-27 19:37:27 +08:00
houseme 749537664f fix:Apply suggestions from clippy 1.88 2025-06-27 18:16:29 +08:00
weisd 7e17d7d729 refactor: improve object listing and version handling
- Refactor find_version_index to accept Uuid directly instead of string
- Split from_meta_cache_entries_sorted into separate methods for versions and infos
- Add support for after_version_id filtering in version listing
- Fix S3 error code for replication configuration not found
- Improve error handling and logging levels
- Clean up import statements and remove debug code
2025-06-24 11:10:23 +08:00
likewu 364c3098b2 Fix/main (#508)
* fix error

* fix

* fix cargo

* fix openssl cfg

* fix clippy

* fix clippy

* fix

* fix clippy

* fix clippy
2025-06-24 10:27:41 +08:00
houseme e6b019c29d Merge branch 'main' of github.com:rustfs/s3-rustfs into feature/observability-metrics
# Conflicts:
#	.github/workflows/build.yml
#	.github/workflows/ci.yml
#	Cargo.lock
#	Cargo.toml
#	appauth/src/token.rs
#	crates/config/src/config.rs
#	crates/event-notifier/examples/simple.rs
#	crates/event-notifier/src/global.rs
#	crates/event-notifier/src/lib.rs
#	crates/event-notifier/src/notifier.rs
#	crates/event-notifier/src/store.rs
#	crates/filemeta/src/filemeta.rs
#	crates/notify/examples/webhook.rs
#	crates/utils/Cargo.toml
#	ecstore/Cargo.toml
#	ecstore/src/cmd/bucket_replication.rs
#	ecstore/src/config/com.rs
#	ecstore/src/disk/error.rs
#	ecstore/src/disk/mod.rs
#	ecstore/src/set_disk.rs
#	ecstore/src/store_api.rs
#	ecstore/src/store_list_objects.rs
#	iam/Cargo.toml
#	iam/src/manager.rs
#	policy/Cargo.toml
#	rustfs/src/admin/rpc.rs
#	rustfs/src/main.rs
#	rustfs/src/storage/mod.rs
2025-06-19 13:16:48 +08:00
Nugine c413465645 fix(utils): ignore failed test 2025-06-13 18:56:44 +08:00
overtrue 4252377249 fix: replace Error::new(ErrorKind::Other) with Error::other() for clippy compliance 2025-06-10 22:15:52 +08:00
weisd 91c099e35f add Error test, fix clippy 2025-06-09 11:29:23 +08:00
weisd 9384b831ec ecstore update ec/disk/error 2025-06-06 01:13:51 +08:00