gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-28 09:08:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,788 Commits 33 Branches 117 Tags
dfcff2db28e18a8d14d70d00529945ae9bc1ed4f
Commit Graph

49 Commits

Author SHA1 Message Date
Henry Guo a8327f8901 fix(scanner): account versioned delete markers in usage (#3904) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-26 18:34:21 +08:00
cxymds 0b9c8a7731 fix(heal): align scanner repair safety with quorum errors (#3887) 
* fix(heal): avoid task-level delete on repair failure

* fix(scanner): avoid recreate for scanner heal checks

* fix(heal): preserve typed quorum failures

* test(heal): satisfy clippy bool assertion
 2026-06-26 08:58:27 +08:00
cxymds 86e9362459 fix(heal): canonicalize scanner object-dir repairs (#3864) 2026-06-25 22:07:38 +08:00
cxymds 4cb28abfc9 fix(scanner): reduce non-actionable scan noise (#3840) 2026-06-25 14:35:10 +08:00
Zhengchao An a935854b32 refactor: centralize scanner runtime source helpers (#3827) 
* refactor: batch lifecycle runtime source handles

* refactor: centralize scanner runtime source helpers
 2026-06-24 22:05:03 +08:00
cxymds eef8f43251 fix: harden lock timeouts, health, and heal paths (#3815) 2026-06-24 15:48:06 +08:00
Henry Guo 7bdb25ae9d feat(scanner): define replication boundary contract (#3630) 
* feat(scanner): define replication boundary contract

* fix(scanner): propagate replication boundary metrics

* fix(scanner): preserve repair metadata on merge

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-23 21:35:36 +08:00
Zhengchao An d3796d6c10 refactor: remove external owner compat bridges (#3741) 2026-06-22 18:28:35 +08:00
安正超 cca9e83a8b refactor: use relative standalone compat consumers (#3734) 2026-06-22 15:07:58 +08:00
安正超 c13f42e9a0 refactor: wrap disk rpc compat trait methods (#3707) 2026-06-22 06:31:24 +08:00
安正超 7eb9a4759e refactor: wrap bucket compat trait methods (#3706) 2026-06-22 04:32:37 +08:00
安正超 6b3d96fde3 refactor: prune trait import compat re-exports (#3699) 2026-06-21 23:17:58 +08:00
安正超 bdb2461b55 refactor: clean scanner heal runtime boundaries (#3571) 2026-06-18 16:12:27 +08:00
安正超 acdf439371 refactor: clean external DTO consumers (#3566) 
* refactor: clean external DTO consumers

* fix: handle empty erasure shard recovery
 2026-06-18 14:22:44 +08:00
Henry Guo b387689f26 feat(heal): expose scanner-aware operations status (#3483) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-15 22:28:53 +08:00
Henry Guo d28aedfbb3 feat(scanner): expose replication repair kind metrics (#3476) 2026-06-15 16:52:39 +08:00
houseme e8012bd1ba refactor(logging): normalize admin telemetry and error messages (#3430) 2026-06-14 13:27:10 +08:00
houseme 7da10db852 refactor(logging): standardize heal and scanner events (#3414) 
* refactor(logging): standardize heal and scanner events

* chore(git): untrack local logging governance note

* chore(git): ignore local logging governance note
 2026-06-14 01:47:39 +08:00
Henry Guo 91f80a5585 feat(scanner): expose lifecycle transition status (#3326) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-10 16:45:02 +00:00
houseme b1339a98ed fix(scanner): delay deep verify for fresh objects (#3329) 
Gate scanner-triggered deep heal behind a short new-object cooldown and retry transient bitrot shard size mismatches during local verification.

Add targeted tests for cooldown mode selection and size-mismatch classification, and log when deep scans are downgraded or size-mismatch retries occur.
 2026-06-10 15:50:44 +00:00
Henry Guo ad9bf41fc8 feat(heal): expose scanner heal admission outcomes (#3292) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
 2026-06-09 03:39:57 +00:00
Henry Guo 6413df4b7f feat(replication): expose scanner repair outcomes (#3278) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
 2026-06-08 14:24:27 +00:00
cxymds 452c003341 fix(scanner): ignore missing rebalance metadata (#3282) 
* fix(ecstore): harden rebalance data movement

* fix(ecstore): preserve failed rebalance status

* fix(ecstore): avoid rebalance walkdir total timeout

* fix(ecstore): retry rebalance listing timeouts

* fix(ecstore): restrict data movement resume target

* refactor(ecstore): simplify multipart movement target

* fix(ecstore): restore resume target checks

* perf(ecstore): speed up rebalance bucket merges

* fix: keep rebalance listing alive on transient failures

* fix(scanner): ignore missing rebalance metadata

---------

Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-08 14:24:16 +00:00
cxymds 9504dff595 fix(ecstore): harden rebalance data movement (#3234) 
* fix(ecstore): harden rebalance data movement

* fix(ecstore): preserve failed rebalance status

* fix(ecstore): avoid rebalance walkdir total timeout

* fix(ecstore): retry rebalance listing timeouts

* fix(ecstore): restrict data movement resume target

* refactor(ecstore): simplify multipart movement target

* fix(ecstore): restore resume target checks

* perf(ecstore): speed up rebalance bucket merges

* fix: keep rebalance listing alive on transient failures

---------

Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-08 10:40:26 +00:00
Henry Guo f40dd2f93c fix(scanner): publish partial usage for compacted scans (#3277) 
* fix(scanner): publish partial usage for compacted scans

* fix(scanner): publish first partial usage immediately

* fix(scanner): skip startup delay for cold usage cache

* fix(scanner): tighten cold usage publish gate

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-08 06:18:35 +00:00
Henry Guo f49df41db9 fix(lifecycle): harden scanner ILM expiry accounting (#3257) 
* fix(lifecycle): harden scanner ILM expiry accounting

* fix(scanner): gate ILM action accounting on enqueue

* fix(metrics): avoid scanner source work argument list

* fix(scanner): gate local ILM accounting on enqueue

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-07 16:18:34 +00:00
Henry Guo b36a730e48 feat(scanner): expose checkpoint and source work status (#3230) 
* feat(scanner): expose checkpoint and source work status

* fix(scanner): count ignored checkpoints once per scan

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-06 10:47:39 +00:00
Henry Guo 0d445afda9 feat(scanner): add versioned scan checkpoints (#3220) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-05 13:49:41 +00:00
Henry Guo 542720a1f7 feat(scanner): add partial scan resume hints (#3207) 
* feat(scanner): add partial scan resume hints

* test(scanner): cover clearing scan resume hints

* fix(scanner): apply resume hint to combined child order

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-04 09:21:30 +00:00
Henry Guo 6850457707 feat(scanner): add runtime scanner controls and status (#3203) 
* feat(scanner): add runtime scanner controls and status

* fix(scanner): validate persisted scanner config

* docs(scanner): clarify start delay behavior

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-04 05:16:26 +00:00
Henry Guo ad1a489f75 feat(scanner): add scanner budget progress controls (#3185) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-03 14:37:58 +00:00
Henry Guo 1d46047d6f feat(scanner): expand scanner observability metrics (#3159) 
* feat(scanner): expand scanner observability metrics

* chore(scanner): align bucket-drive metric wording

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-06-02 01:29:53 +00:00
Henry Guo 76da2a48d0 feat(scanner): expose cycle observability controls (#3147) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-05-31 21:55:46 +00:00
Henry Guo 8e809f005d fix(scanner): support PBS subfolder alert threshold (#3129) 
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-05-30 11:03:13 +00:00
weisd 69345fe059 fix(scanner): preserve background heal compatibility (#3041) 2026-05-22 14:38:35 +08:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012) 
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
 2026-05-19 12:50:25 +00:00
weisd d4dcb2ac9d fix(scanner): avoid stalls after abandoned child listings (#2632) 2026-04-21 09:06:26 +00:00
houseme 6ce24f3b63 feat(lifecycle): improve ILM compatibility and scanner runtime config (#2534) 
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
 2026-04-16 10:28:03 +00:00
houseme 28edfd6190 fix(storage): harden offline drive fail-fast paths (#2564) 
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2026-04-16 09:21:45 +00:00
Tunglies 49366ee200 chore(lint): clippy rules redundant_clone (#2554) 2026-04-15 13:54:07 +00:00
weisd 27971f5c03 fix(scanner): preserve totals for compacted folder scans (#2530) 2026-04-15 07:32:13 +00:00
weisd 9ed6487e2f fix(scanner): continue scans after traversal errors (#2414) 2026-04-07 14:19:26 +08:00
evan slack b3da8ae269 feat(scanner): Add dynamic throttling presets (#2095) 
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: GatewayJ <835269233@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: weisd <im@weisd.in>
 2026-03-10 16:12:56 +08:00
LoganZ2 e73b17aff6 fix(scanner): skip recent IO-error objects (#1860) 
Signed-off-by: LoganZ2 <103290230+LoganZ2@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-02-27 22:25:52 +08:00
evan slack 9786d9b004 metrics(scanner): Add metrics to scanner (#1823) 
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
 2026-02-15 18:36:40 +08:00
evan slack 682b5bbb2f perf(scanner): Change DataUseageEntry from clone to borrow (#1757) 
Signed-off-by: evan slack <51209817+evanofslack@users.noreply.github.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-02-10 08:53:26 +08:00
houseme 0b870d6301 build(deps): bump the dependencies group with 19 updates (#1745) 2026-02-07 12:22:14 +08:00
houseme d251b9fb35 fix: unify path handling to use S3-standard forward slashes on all platforms (#1555) 
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: LeonWang0735 <wlywly0735@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-26 18:49:21 +08:00
weisd 6631407416 feat: Add RustFS Scanner Module and Multiple Bug Fixes (#1579) 2026-01-22 13:39:38 +08:00