088c4bda43
fix(ecstore): harden issue3031 multipart validation path ( #3106 )
...
* fix(ecstore): harden issue3031 multipart validation path
- clear stale multipart part destinations before rename fan-out
- add repeated part overwrite regression coverage
- reduce remote disk startup false-fault escalation to suspect-first
- refine remote locker diagnostics and lower scanner leader-lock log noise
- add a dedicated 4-node issue3031 docker validation script
* refactor(admin): inline console version json macro
- drop the unused serde_json::json import in admin console
- call serde_json::json! inline in version_handler
- keep the console version response behavior unchanged
* fix(remote-disk): recover suspect health on probe success
- record probe success during remote disk health checks so suspect drives recover
- use async_with_vars for the remote disk health probe test
- make the missing-listener test assert the state transition more robustly
2026-05-28 14:26:31 +00:00
247973f34c
fix(lifecycle): make transition worker resize nonblocking ( #3090 )
...
Co-authored-by: 季宏伟 <jihongwei@jihongweis-MacBook-Pro.local >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-28 01:36:08 +00:00
909f0d57fb
feat: improve degraded readiness reporting and shutdown handling ( #3089 )
...
* fix(server): unify runtime readiness and shutdown flow
* refactor(server): decouple readiness shared types
* refactor(server): keep readiness types crate-private
* refactor(server): await protocol shutdown handles
* fix(server): bypass cached startup readiness
* feat(health): expose degraded readiness reasons
* fix(health): preserve raw IAM readiness in degraded reports
2026-05-27 03:25:28 +00:00
ed2078e025
fix(ecstore): allow expired delete markers on locked buckets ( #3048 )
...
* fix: allow expired delete markers on locked buckets
* fix: reject zero-day del marker expiration
2026-05-22 01:39:53 +00:00
73bde843d6
refactor(s3): consolidate semantic boundaries and remove s3-common ( #3012 )
...
* refactor(common): introduce rustfs-data-usage core crate
* refactor(concurrency): migrate workers crate into concurrency
* refactor(crypto): migrate appauth token APIs into crypto
* fix docs urls
* remove unused crate
* refactor(data-usage): switch consumers to rustfs-data-usage
* chore(fmt): apply cargo fmt and lockfile sync
* refactor(common): remove data_usage compatibility re-export
* refactor(capacity): move capacity_scope to object-capacity
* refactor(io-metrics): relocate internode metrics from common
* refactor(common): decouple scanner report from madmin
* chore(fmt): normalize import ordering after pre-commit
* refactor(s3): split s3 types and ops crates
* refactor(s3): centralize event version and safe parsing
* refactor(s3): add op-event compatibility guardrails
* refactor(s3): add runtime op-event mismatch observability
* refactor(s3): extract delete event mapping helper
* refactor(s3): extract put event mapping helper
* refactor(s3): consolidate remaining event semantic helpers
* refactor(s3): add op-event coverage checks and observability alerts
* refactor(s3-ops): consolidate op-event semantic mapping
* refactor(scanner): remove last_minute wrapper module
* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
e65aebba53
fix(ecstore): guard transition worker max ( #3003 )
...
* fix(ecstore): guard transition worker max
* test: clarify env mutation safety
* test(ecstore): document env mutation concurrency contract
2026-05-19 02:25:51 +00:00
bd1e57293f
fix: harden lifecycle transition compensation and regression coverage ( #2995 )
...
* fix(ecstore): honor transition worker configuration
* fix(ecstore): add transition queue backpressure metrics
* fix(ecstore): schedule transition compensation on enqueue pressure
* fix(ecstore): log transition compensation scheduling
* test(rustfs): add transition compensation fault-injection coverage
* test(rustfs): cover delete after transition compensation
* test(scanner): cover cleanup after transition compensation
* test(rustfs): extend compensation transition coverage
* test(scanner): cover backfill idempotency after compensation
* test(scanner): cover noncurrent expiry after compensation
* test(rustfs): cover versioned delete after compensation
* test(rustfs): cover delete marker lifecycle after compensation
* test(scanner): extend versioned lifecycle compensation coverage
* test(scanner): model versioned delete after compensation
* test(scanner): clarify modeled versioned delete helper
* refactor(ecstore): optimize transition enqueue hot path
* refactor(ecstore): centralize transition runtime constants
* style(ecstore): apply rustfmt for transition timeout helper
* fix(ilm): align queue-full metric semantics
* refactor(ecstore): unify immediate enqueue failure handling
* refactor(ecstore): reuse transition worker env constant
* ci(actions): update setup action inputs
2026-05-18 12:50:43 +00:00
9b9e0db9f6
test(lifecycle): cover ILM process time aliases ( #2839 )
2026-05-06 21:12:22 +00:00
70be0804ee
fix: 2827 lifecycle days next midnight ( #2833 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-06 15:00:53 +00:00
2953558f41
fix(lifecycle): prevent eager date-expiry deletion on config update ( #2708 )
2026-04-28 10:26:14 +00:00
a68fe1601f
fix(ilm): harden signer failures and guard remote tier delete storms ( #2706 )
2026-04-27 12:39:02 +00:00
a77be8f89b
fix: some expect error ( #2622 )
...
Signed-off-by: likewu <likewu@126.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-04-21 03:27:50 +00:00
1511f9eacb
fix(lifecycle): correct delete replication fanout ( #2609 )
...
Co-authored-by: loverustfs <hello@rustfs.com >
2026-04-20 10:02:59 +00:00
478720d2ee
Centralize lifecycle state updates and fix systemd running status ( #2567 )
2026-04-17 03:20:11 +00:00
6ce24f3b63
feat(lifecycle): improve ILM compatibility and scanner runtime config ( #2534 )
...
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: cxymds <Cxymds@qq.com >
2026-04-16 10:28:03 +00:00
579b124726
lint: clippy rules or_fun_call ( #2561 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-04-16 02:48:39 +00:00
49366ee200
chore(lint): clippy rules redundant_clone ( #2554 )
2026-04-15 13:54:07 +00:00
0b99e02891
feat(ecstore): add stale multipart upload cleanup ( #2416 )
2026-04-07 18:57:07 +08:00
16db18216d
fix: populate tagging notification principalId and object metadata ( #2342 )
2026-03-30 14:32:39 +08:00
a236b0d01d
feat(ecstore): implement decommission and rebalance ( #2281 )
...
Co-authored-by: weisd <im@weisd.in >
Co-authored-by: houseme <housemecn@gmail.com >
2026-03-26 11:44:02 +08:00
41dcebda44
fix(tier): sweep transitioned copies from delete handlers ( #2287 )
2026-03-25 16:06:36 +08:00
28f57b228c
feat(s3): advance parity coverage ( #2278 )
2026-03-24 17:29:33 +08:00
dad9a7d708
fix(ecstore): honor lifecycle tag filters ( #2264 )
...
Co-authored-by: loverustfs <hello@rustfs.com >
2026-03-24 14:25:43 +08:00
236142a682
fix(ecstore): repair lifecycle transition and restore flows ( #2240 )
...
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: loverustfs <hello@rustfs.com >
Co-authored-by: weisd <im@weisd.in >
2026-03-23 12:29:13 +08:00
f11c307aec
fix(admin): avoid tier stats panic on missing tier ( #2238 )
...
Co-authored-by: heihutu <heihutu@gmail.com >
2026-03-20 20:57:33 +08:00
b9b7d86ae4
feat: improve legacy metadata and admin compatibility ( #2202 )
2026-03-18 21:05:09 +08:00
237c933f38
test(lifecycle): add prefix regression and zero-day checks ( #2201 )
2026-03-18 13:20:00 +08:00
c20b3c7f19
fix(ecstore): handle EODM rules without due date ( #2198 )
2026-03-18 12:54:41 +08:00
c66c6d97ec
fix(lifecycle): respect Filter.Prefix and safe delete marker expiry ( #2185 )
...
Signed-off-by: likewu <likewu@126.com >
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: likewu <likewu@126.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-03-17 18:45:38 +08:00
60aa47bf61
feat(storage): integrate S3Operation into OperationHelper for unified metrics and audit ( #2103 )
2026-03-08 17:57:33 +08:00
b4a633ebc6
fix(ecstore): set expiration header for put object via lifecycle prediction ( #2003 )
2026-02-28 16:21:58 +08:00
3f5ccb20fc
fix(s3): normalize GetObjectAttributes ETag XML response ( #2002 )
2026-02-28 14:53:53 +08:00
af6c32efac
refactor: improve code quality with safer error handling, trait decomposition, and dead code cleanup ( #1997 )
2026-02-28 01:19:47 +08:00
9d2b8822cf
refactor: stabilize heal format recovery integration tests ( #1984 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-02-27 15:26:19 +08:00
9786d9b004
metrics(scanner): Add metrics to scanner ( #1823 )
...
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: loverustfs <hello@rustfs.com >
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com >
2026-02-15 18:36:40 +08:00
087f58b7c8
fix(lifecycle): lifecycle fixes ( #1625 )
...
Signed-off-by: likewu <likewu@126.com >
Co-authored-by: loverustfs <hello@rustfs.com >
2026-01-31 19:14:47 +08:00
9285acba06
feat: object retention ( #1589 )
2026-01-24 22:12:45 +08:00
6631407416
feat: Add RustFS Scanner Module and Multiple Bug Fixes ( #1579 )
2026-01-22 13:39:38 +08:00
f795299d53
Optimization and collation of dependencies introduction processing ( #1493 )
2026-01-13 15:02:54 +08:00
978845b555
fix(lifecycle): Fix ObjectInfo fields and mod_time error handling ( #1088 )
...
Co-authored-by: loverustfs <hello@rustfs.com >
2025-12-11 07:17:35 +08:00
7e1a9e2ede
🔒 Upgrade Cryptography Libraries to Latest RC Versions ( #837 )
...
* fix
* chore: upgrade cryptography libraries to RC versions
- Upgrade aes-gcm to 0.11.0-rc.2 with rand_core support
- Upgrade chacha20poly1305 to 0.11.0-rc.2
- Upgrade argon2 to 0.6.0-rc.2 with std features
- Upgrade hmac to 0.13.0-rc.3
- Upgrade pbkdf2 to 0.13.0-rc.2
- Upgrade rsa to 0.10.0-rc.10
- Upgrade sha1 and sha2 to 0.11.0-rc.3
- Upgrade md-5 to 0.11.0-rc.3
These upgrades provide enhanced security features and performance
improvements while maintaining backward compatibility with existing
encryption workflows.
* add
* improve code
* fix
2025-11-11 21:10:03 +08:00
d934e3905b
Refactor telemetry initialization for non-production environments ( #789 )
...
* add dep `scopeguard`
* improve for tracing
* fix
* fix
* improve code for import
* add logger trace id
* fix
* fix
* fix
* fix
* fix
2025-11-05 00:55:08 +08:00
2aca1f77af
Fix/ilm ( #721 )
...
* fix tip remote tier error
* fix transitioned_object
* fix filemeta
* add GCS R2
* add aliyun tencent huaweicloud azure gcs r2 backend tier
* fix signer
* change azure to s3
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
2025-10-27 20:23:50 +08:00
12ecb36c6d
Fix collect ( #586 )
...
* fix: fix datausageinfo
Signed-off-by: junxiang Mu <1948535941@qq.com >
* feat(data-usage): implement local disk snapshot aggregation for data usage statistics
Signed-off-by: junxiang Mu <1948535941@qq.com >
* feat(scanner): improve data usage collection with local scan aggregation
Signed-off-by: junxiang Mu <1948535941@qq.com >
* refactor: improve object existence check and code style
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-09-24 02:48:23 -07:00
62a01f3801
Performance: improve ( #514 )
...
* Performance: improve
Signed-off-by: junxiang Mu <1948535941@qq.com >
* remove dirty
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix some err
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-09-11 19:48:28 +08:00
9d5ed1acac
Feature/scanner performance optimization ( #498 )
...
* Refactor: reimplement scanner
Signed-off-by: RustFS Developer <dandan@rustfs.com >
* comment lock
Signed-off-by: junxiang Mu <1948535941@qq.com >
* remove dirty file
Signed-off-by: junxiang Mu <1948535941@qq.com >
* Fix: fix rebase
* fix(scanner): Improve error handling and logging
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: RustFS Developer <dandan@rustfs.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: RustFS Developer <dandan@rustfs.com >
2025-09-08 18:35:45 +08:00
7462be983a
Feature up/ilm ( #470 )
...
* fix delete-marker expiration. add api_restore.
* time retry object upload
* lock file
* make fmt
* restore object
* serde-rs-xml -> quick-xml
* scanner_item prefix object_name
* object_path
* object_name
* fi version_purge_status
* old_dir None
Co-authored-by: houseme <housemecn@gmail.com >
2025-09-01 16:11:28 +08:00
d987686c14
feat(lifecycle): Implement object lifecycle management functionality ( #358 )
...
* feat(lifecycle): Implement object lifecycle management functionality
Add a lifecycle module to automatically handle object expiration and transition during scanning
Modify the file metadata cache module to be publicly visible to support lifecycle operations
Adjust the scanning interval to a shorter time for testing lifecycle rules
Implement the parsing and execution logic for S3 lifecycle configurations
Add integration tests to verify the lifecycle expiration functionality
Update dependencies to support the new lifecycle features
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix cargo dependencies
Signed-off-by: junxiang Mu <1948535941@qq.com >
* fix fmt
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: houseme <housemecn@gmail.com >
2025-08-08 10:51:02 +08:00
d56cee26db
Feature up/ilm ( #305 )
...
* fix
* fix
* fix
* fix delete-marker expiration. add api_restore.
* fix
* time retry object upload
* lock file
* make fmt
* fix
* restore object
* fix
* fix
* serde-rs-xml -> quick-xml
* fix
* checksum
* fix
* fix
* fix
* fix
* fix
* fix
* fix
2025-07-29 14:21:19 +08:00
ea210d52dc
refactor(heal): unify heal request interface, add disk field, update ahm/ecstore/common for erasure set healing
...
Signed-off-by: junxiang Mu <1948535941@qq.com >
2025-07-24 12:14:03 +08:00
houseme