d4ee44f49f
refactor(scanner): narrow cache storage bounds ( #3348 )
2026-06-11 13:52:35 +08:00
b36a730e48
feat(scanner): expose checkpoint and source work status ( #3230 )
...
* feat(scanner): expose checkpoint and source work status
* fix(scanner): count ignored checkpoints once per scan
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-06-06 10:47:39 +00:00
0d445afda9
feat(scanner): add versioned scan checkpoints ( #3220 )
...
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-06-05 13:49:41 +00:00
542720a1f7
feat(scanner): add partial scan resume hints ( #3207 )
...
* feat(scanner): add partial scan resume hints
* test(scanner): cover clearing scan resume hints
* fix(scanner): apply resume hint to combined child order
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-06-04 09:21:30 +00:00
6850457707
feat(scanner): add runtime scanner controls and status ( #3203 )
...
* feat(scanner): add runtime scanner controls and status
* fix(scanner): validate persisted scanner config
* docs(scanner): clarify start delay behavior
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-06-04 05:16:26 +00:00
cc07946782
feat(scanner): add cycle budget observability ( #3166 )
...
* feat(scanner): add cycle budget observability
* fix(scanner): clear cycle state after budget stop
* test(scanner): stabilize timeout-based scanner tests
* fix(scanner): keep cycle ILM counts scanner-only
* test(scanner): avoid test-only pending import
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
2026-06-03 03:32:41 +00:00
a14657f517
fix(scanner,data-usage): fix add() logic inversion and usize underflow in reduce_children_of ( #3142 )
...
* fix(scanner,data-usage): fix add() logic inversion and usize underflow in reduce_children_of
Bug #1 : add() had inverted logic — returned early when children were present,
making the recursive child traversal dead code. Leaf compaction path was
completely non-functional. Fixed by recursing into children when present and
collecting leaves only at leaf nodes.
Bug #2 : reduce_children_of used which underflows in debug
(panic) or release (wraps to usize::MAX, compacts entire cache). Fixed with
saturating_sub.
Both bugs existed identically in crates/scanner and crates/data-usage.
Added 5 tests covering add() subtree traversal, edge cases, and
reduce_children_of compaction behavior.
Closes rustfs/backlog#652
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com >
* fix: collect internal nodes as compaction candidates, not leaves
The previous fix for add() collected leaf nodes (children empty) as
compaction candidates. This is incorrect: total_children_rec returns 0
for leaves, so compacting them never decrements — the
compaction loop makes no progress.
Collect internal nodes (children non-empty) instead. Compacting an
internal node removes its subtree via delete_recursive, which actually
reduces the total child count. Leaf nodes are skipped since they have
no children to remove.
Updated tests to match the corrected semantics.
* refactor: rename leaves→candidates, add data-usage regression tests
- Rename misleading variable/parameter to in
add() and reduce_children_of() for both crates
- Add 4 regression tests in crates/data-usage covering add() candidate
selection and reduce_children_of() compaction + saturating_sub
* fix: address data usage compaction review feedback
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-31 21:52:53 +00:00
92104cb354
fix(scanner): reduce single-disk scanner churn ( #3135 )
...
* fix(scanner): slow single-disk default scans
* fix(scanner): reduce single-disk scanner churn
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-31 05:05:04 +00:00
73bde843d6
refactor(s3): consolidate semantic boundaries and remove s3-common ( #3012 )
...
* refactor(common): introduce rustfs-data-usage core crate
* refactor(concurrency): migrate workers crate into concurrency
* refactor(crypto): migrate appauth token APIs into crypto
* fix docs urls
* remove unused crate
* refactor(data-usage): switch consumers to rustfs-data-usage
* chore(fmt): apply cargo fmt and lockfile sync
* refactor(common): remove data_usage compatibility re-export
* refactor(capacity): move capacity_scope to object-capacity
* refactor(io-metrics): relocate internode metrics from common
* refactor(common): decouple scanner report from madmin
* chore(fmt): normalize import ordering after pre-commit
* refactor(s3): split s3 types and ops crates
* refactor(s3): centralize event version and safe parsing
* refactor(s3): add op-event compatibility guardrails
* refactor(s3): add runtime op-event mismatch observability
* refactor(s3): extract delete event mapping helper
* refactor(s3): extract put event mapping helper
* refactor(s3): consolidate remaining event semantic helpers
* refactor(s3): add op-event coverage checks and observability alerts
* refactor(s3-ops): consolidate op-event semantic mapping
* refactor(scanner): remove last_minute wrapper module
* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
9677320f23
fix(scanner): stabilize data usage cache persistence under slow metadata I/O ( #2594 )
2026-04-18 16:36:28 +00:00
478720d2ee
Centralize lifecycle state updates and fix systemd running status ( #2567 )
2026-04-17 03:20:11 +00:00
579b124726
lint: clippy rules or_fun_call ( #2561 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-04-16 02:48:39 +00:00
49366ee200
chore(lint): clippy rules redundant_clone ( #2554 )
2026-04-15 13:54:07 +00:00
b9b7d86ae4
feat: improve legacy metadata and admin compatibility ( #2202 )
2026-03-18 21:05:09 +08:00
ed18b3da75
Fix data usage cache and scanner ( #2074 )
2026-03-04 19:55:01 +08:00
e73b17aff6
fix(scanner): skip recent IO-error objects ( #1860 )
...
Signed-off-by: LoganZ2 <103290230+LoganZ2@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com >
Co-authored-by: loverustfs <hello@rustfs.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
2026-02-27 22:25:52 +08:00
682b5bbb2f
perf(scanner): Change DataUseageEntry from clone to borrow ( #1757 )
...
Signed-off-by: evan slack <51209817+evanofslack@users.noreply.github.com >
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: loverustfs <hello@rustfs.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-02-10 08:53:26 +08:00
dce117840c
refactor: NamespaceLock (nslock), AHM→Heal Crate, and Lock/Clippy Fixes ( #1664 )
...
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: weisd <2057561+weisd@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-01-30 13:13:41 +08:00
d251b9fb35
fix: unify path handling to use S3-standard forward slashes on all platforms ( #1555 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Signed-off-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com >
Co-authored-by: LeonWang0735 <wlywly0735@126.com >
Co-authored-by: loverustfs <hello@rustfs.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-01-26 18:49:21 +08:00
e5284a85ed
fix: Fixed detection warnings in rust v1.93.0 ( #1591 )
2026-01-23 13:09:41 +08:00
6631407416
feat: Add RustFS Scanner Module and Multiple Bug Fixes ( #1579 )
2026-01-22 13:39:38 +08:00
安正超