c55c7a6373
feat: add docker usage for rustfs mcp ( #365 )
2025-08-08 17:18:20 +08:00
d987686c14
feat(lifecycle): Implement object lifecycle management functionality ( #358 )
...
* feat(lifecycle): Implement object lifecycle management functionality
Add a lifecycle module to automatically handle object expiration and transition during scanning
Modify the file metadata cache module to be publicly visible to support lifecycle operations
Adjust the scanning interval to a shorter time for testing lifecycle rules
Implement the parsing and execution logic for S3 lifecycle configurations
Add integration tests to verify the lifecycle expiration functionality
Update dependencies to support the new lifecycle features
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix cargo dependencies
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix fmt
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: houseme <housemecn@gmail.com >
2025-08-08 10:51:02 +08:00
48a9707110
fix: add tokio-test ( #363 )
...
* fix: add tokio-test
* fix: "called `unwrap` on `v` after checking its variant with `is_some`"
= help: try using `if let` or `match`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_unwrap
= note: `-D clippy::unnecessary-unwrap` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::unnecessary_unwrap)]`
* fmt
* set toolchain 1.88.0
* fmt
* fix: cliip
2025-08-08 10:23:22 +08:00
b89450f54d
replace make with just ( #349 )
2025-08-07 22:37:05 +08:00
e0c99bced4
chore: add tls log and removing unused crates ( #359 )
...
* chore: add tls log
* improve code for http
* improve code dependencies for `cargo.toml` and removing unused crates
* modify name
* improve code
* fix
* Update crates/config/src/constants/env.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* improve code
* fix
* add `is_enabled` and `is_disabled`
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-08-07 19:02:09 +08:00
c42fbed3d2
fix: Fixed an issue where the list_objects_v2 API did not return dire… ( #352 )
...
* fix: Fixed an issue where the list_objects_v2 API did not return directory names when they conflicted with file names in the same bucket (e.g., test/ vs. test.txt, aaa/ vs. aaa.csv) (#335 )
* fix: adjusted the order of directory listings
2025-08-07 11:05:05 +08:00
9aba89a12c
fix: miss inline metadata ( #345 )
2025-08-06 11:45:23 +08:00
1b88714d27
Fix: fix data integrity check
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-08-06 11:03:29 +08:00
b119894425
perf: avoid transmitting parity shards when the object is good ( #322 )
2025-08-02 14:37:43 +08:00
6c37e1cb2a
refactor: replace lazy_static with LazyLock ( #318 )
...
* refactor: replace `lazy_static` with `LazyLock`
Replace `lazy_static` with `LazyLock`.
Compile time may reduce a little.
See https://github.com/rust-lang-nursery/lazy-static.rs/issues/214
* fmt
* fix
2025-07-31 14:25:39 +08:00
c5df1f92c2
refactor: replace lazy_static with LazyLock and notify crate registry create_targets_from_config ( #311 )
...
* improve code for notify
* improve code for logger and fix typo (#272 )
* Add GNU to build.yml (#275 )
* fix unzip error
* fix url change error
fix url change error
* Simplify user experience and integrate console and endpoint
Simplify user experience and integrate console and endpoint
* Add gnu to build.yml
* upgrade version
* feat: add `cargo clippy --fix --allow-dirty` to pre-commit command (#282 )
Resolves #277
- Add --fix flag to automatically fix clippy warnings
- Add --allow-dirty flag to run on dirty Git trees
- Improves code quality in pre-commit workflow
* fix: the issue where preview fails when the path length exceeds 255 characters (#280 )
* fix
* fix: improve Windows build support and CI/CD workflow (#283 )
- Fix Windows zip command issue by using PowerShell Compress-Archive
- Add Windows support for OSS upload with ossutil
- Replace Chinese comments with English in build.yml
- Fix bash syntax error in package_zip function
- Improve code formatting and consistency
- Update various configuration files for better cross-platform support
Resolves Windows build failures in GitHub Actions.
* fix: update link in README.md leading to a 404 error (#285 )
* add rustfs.spec for rustfs (#103 )
add support on loongarch64
* improve cargo.lock
* build(deps): bump the dependencies group with 5 updates (#289 )
Bumps the dependencies group with 5 updates:
| Package | From | To |
| --- | --- | --- |
| [hyper-util](https://github.com/hyperium/hyper-util ) | `0.1.15` | `0.1.16` |
| [rand](https://github.com/rust-random/rand ) | `0.9.1` | `0.9.2` |
| [serde_json](https://github.com/serde-rs/json ) | `1.0.140` | `1.0.141` |
| [strum](https://github.com/Peternator7/strum ) | `0.27.1` | `0.27.2` |
| [sysinfo](https://github.com/GuillaumeGomez/sysinfo ) | `0.36.0` | `0.36.1` |
Updates `hyper-util` from 0.1.15 to 0.1.16
- [Release notes](https://github.com/hyperium/hyper-util/releases )
- [Changelog](https://github.com/hyperium/hyper-util/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper-util/compare/v0.1.15...v0.1.16 )
Updates `rand` from 0.9.1 to 0.9.2
- [Release notes](https://github.com/rust-random/rand/releases )
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand/compare/rand_core-0.9.1...rand_core-0.9.2 )
Updates `serde_json` from 1.0.140 to 1.0.141
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.140...v1.0.141 )
Updates `strum` from 0.27.1 to 0.27.2
- [Release notes](https://github.com/Peternator7/strum/releases )
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Peternator7/strum/compare/v0.27.1...v0.27.2 )
Updates `sysinfo` from 0.36.0 to 0.36.1
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md )
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.36.0...v0.36.1 )
---
updated-dependencies:
- dependency-name: hyper-util
dependency-version: 0.1.16
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: rand
dependency-version: 0.9.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: serde_json
dependency-version: 1.0.141
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: strum
dependency-version: 0.27.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: sysinfo
dependency-version: 0.36.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* improve code for logger
* improve
* upgrade
* refactor: 优化构建工作流，统一 latest 文件处理和简化制品上传 (#293 )
* Refactor: DatabaseManagerSystem as global
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix: fmt
Signed-off-by: junxiang Mu <1948535941@qq.com >
* Test: add e2e_test for s3select
Signed-off-by: junxiang Mu <1948535941@qq.com >
* Test: add test script for e2e
Signed-off-by: junxiang Mu <1948535941@qq.com >
* improve code for registry and intergation
* improve code for registry `create_targets_from_config`
* fix
* Feature up/ilm (#305 )
* fix
* fix
* fix
* fix delete-marker expiration. add api_restore.
* fix
* time retry object upload
* lock file
* make fmt
* fix
* restore object
* fix
* fix
* serde-rs-xml -> quick-xml
* fix
* checksum
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* transfer lang to english
* upgrade clap version from 4.5.41 to 4.5.42
* refactor: replace `lazy_static` with `LazyLock`
* add router
* fix: modify comment
* improve code
* fix typos
* fix
* fix: modify name and fmt
* improve code for registry
* fix test
---------
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: shiro.lee <69624924+shiroleeee@users.noreply.github.com >
Co-authored-by: Marco Orlandin <mipnamic@mipnamic.net >
Co-authored-by: zhangwenlong <zhangwenlong@loongson.cn >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: likewu <likewu@126.com >
2025-07-30 19:02:10 +08:00
4f1770d3fe
feat:add mcp integration ( #300 )
...
* add list_buckets mcp server
* add list_objects mcp
* add upload object mcp
* add get object mcp
* add list_buckets mcp server
* fix: resolve clippy warnings in rustfs-mcp-server
* fix: rename mcp package
* fix
* fix:remove useless comment
* feat:add mcp doc
2025-07-30 14:25:01 +08:00
d56cee26db
Feature up/ilm ( #305 )
...
* fix
* fix
* fix
* fix delete-marker expiration. add api_restore.
* fix
* time retry object upload
* lock file
* make fmt
* fix
* restore object
* fix
* fix
* serde-rs-xml -> quick-xml
* fix
* checksum
* fix
* fix
* fix
* fix
* fix
* fix
* fix
2025-07-29 14:21:19 +08:00
56fd8132e9
fix:#303 returns empty when querying an empty or not dir ( #304 )
2025-07-28 16:17:40 +08:00
35daa74430
Merge pull request #302 from guojidan/lock
...
Lock: add transactional
2025-07-28 12:00:44 +08:00
dc156fb4cd
Fix: clippy
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-28 11:38:42 +08:00
de905a878c
Cargo: use workspace dependence
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-28 11:02:40 +08:00
f3252f989b
Test: Add e2e test case for lock transactional
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-28 11:00:10 +08:00
01a2afca9a
lock: Add transactional
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-28 10:59:43 +08:00
a4fe68ad21
Merge pull request #301 from guojidan/improve-sql
...
s3Select: add unit test case
2025-07-28 09:56:10 +08:00
c03f86b23c
s3Select: add unit test case
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-28 09:19:47 +08:00
5667f324ae
Merge pull request #297 from guojidan/improve-sql
...
Test: Add e2e_test case for sql && add script for e2e_test
2025-07-25 17:16:41 +08:00
612404c47f
Test: add e2e_test for s3select
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-25 15:07:44 +08:00
85388262b3
Merge pull request #294 from guojidan/improve-sql
...
Refactor: DatabaseManagerSystem as global
2025-07-25 08:33:54 +08:00
25a4503285
fix: fmt
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-25 08:18:14 +08:00
addc964d56
Refactor: DatabaseManagerSystem as global
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 17:12:51 +08:00
0672b6dd3e
Chore: remove dirty file(cache.rs)
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 14:57:48 +08:00
77bc9af109
Update Cargo.toml
2025-07-24 14:14:12 +08:00
91b1c84430
rebase
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:18:05 +08:00
b667927216
fix fmt
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:28 +08:00
29795fac51
fix Cargo.toml
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:28 +08:00
2ce7e01f55
Chore: remove dirty file(heal)
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:27 +08:00
4fefd63a5b
rebase
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
2a8c46874d
fix: auto heal when xl.meta lose
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
b8b5511b68
fix: heal data part lose
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
bdaee228db
fix(ahm): adjust test expectations for missing xl.meta recovery scenario
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
d562620e99
fix: implement uses_data_dir method
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
69b0c828c9
fix: scanner add heal bucket
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
2bfd1efb9b
Fix: fix add heal_manager into scanner when scanner start
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
0854e6b921
Chore: rename init_heal_manager_with_channel
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
b907f4e61b
refactor(ahm): remove obsolete scanner/data_usage.rs after data usage refactor
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
6ec568459c
chore: update admin handlers, lockfile, and minor fixes
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:05 +08:00
ea210d52dc
refactor(heal): unify heal request interface, add disk field, update ahm/ecstore/common for erasure set healing
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:03 +08:00
3d3c6e4e06
chore(protos): update proto definitions, remove ns_scanner, fix codegen and formatting
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:12:49 +08:00
e7d0a8d4b9
feat: integrate global metrics system into AHM scanner
...
- Add global metrics system to common crate for cross-module usage
- Integrate global metrics collection into AHM scanner operations
- Update ECStore to use common metrics system instead of local implementation
- Add chrono dependency to AHM crate for timestamp handling
- Re-export IlmAction from common metrics in ECStore lifecycle module
- Update scanner methods to use global metrics for cycle, disk, and volume scans
- Maintain backward compatibility with local metrics collector
- Fix clippy warnings and ensure proper code formatting
This change enables unified metrics collection across the entire RustFS system,
allowing better monitoring and observability of scanner operations.
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:12:49 +08:00
7d3b2b774c
fix heal disk
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:12:49 +08:00
aed8f52423
refactor: integrate disk healing into erasure set healing
...
- Remove HealType::Disk and related disk-specific healing methods
- Integrate disk format healing into heal_erasure_set with include_format_heal option
- Update auto disk scanner to use ErasureSet heal type instead of Disk heal
- Fix disk status change event handling to use ErasureSet heal requests
- Add proper bucket list retrieval for auto healing scenarios
- Update data scanner to submit ErasureSet heal tasks for offline disks
- Remove duplicate healing logic between Disk and ErasureSet types
- Ensure all healing operations go through unified ErasureSet healing path
2025-07-24 12:12:49 +08:00
c49414f6ac
fix: resolve test conflicts and improve data scanner functionality
...
- Fix multi-threaded test conflicts in AHM heal integration tests
- Remove global environment sharing to prevent test state pollution
- Fix test_all_disk_method by clearing global disk map before test
- Improve data scanner and cache value implementations
- Update dependencies and resolve clippy warnings
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:12:49 +08:00
8e766b90cd
feat: implement heal channel mechanism for admin-ahm communication
...
- Add global unbounded channel in common crate for heal requests
- Implement channel processor in ahm to handle heal commands
- Add Start/Query/Cancel commands support via channel
- Integrate heal manager initialization in main.rs
- Replace direct MRF calls with channel-based heal requests in ecstore
- Support advanced heal options including pool_index and set_index
- Enable admin handlers to send heal requests via channel
2025-07-24 12:12:49 +08:00
3409cd8dff
feat(ahm): add HealingTracker support & complete fresh-disk healing
...
• Introduce ecstore HealingTracker into ahm crate; load/init/save tracker
• Re-implement heal_fresh_disk to use heal_erasure_set with tracker
• Enhance auto-disk scanner: detect unformatted disks via get_disk_id()
• Remove DataUsageCache handling for now
• Refactor imports & types, clean up duplicate constants
2025-07-24 12:12:49 +08:00
majinghe