Commit Graph

89 Commits

Author SHA1 Message Date
安正超 afe46c01e7 refactor: prune consumer ecstore object aliases (#3622) 2026-06-19 21:33:00 +08:00
Henry Guo a689ff3c39 feat(scanner): wake cycles for dirty usage refresh (#3617) 2026-06-19 18:36:48 +08:00
安正超 54ffbedbc8 refactor: remove ecstore operation compat facades (#3608) 2026-06-19 17:01:12 +08:00
安正超 b14e49e84e refactor: narrow IAM and Swift compatibility surfaces (#3587)
* refactor: narrow IAM and Swift compatibility surfaces

* refactor: narrow heal and scanner compatibility surfaces (#3588)

* refactor: narrow RustFS runtime compatibility surfaces (#3591)
2026-06-19 03:06:57 +08:00
安正超 205f964dc5 refactor: tighten storage compat store_api aliases (#3582)
* refactor: collapse storage compat store_api modules

* chore: guard storage compat store_api aliases
2026-06-18 22:50:02 +08:00
安正超 c28fee0013 refactor: continue storage api contract cleanup (#3580)
* refactor: move delete object contracts to storage api

* refactor: narrow store api compatibility exports

* refactor: route table catalog test through storage compat
2026-06-18 22:42:02 +08:00
Henry Guo 80144ec886 feat(scanner): expose usage freshness status (#3577)
* feat(scanner): expose usage freshness status

* fix(scanner): reset dirty usage cycle state

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-18 21:14:55 +08:00
安正超 7b0cb9e725 refactor: prune storage compatibility re-export allowances (#3579)
* refactor: prune storage compatibility re-export allowances

* fix: reject whitespace-padded dot path segments
2026-06-18 21:14:24 +08:00
安正超 f3fcdd4ba2 refactor: narrow remaining storage compatibility exports (#3578) 2026-06-18 19:18:32 +08:00
安正超 bdb2461b55 refactor: clean scanner heal runtime boundaries (#3571) 2026-06-18 16:12:27 +08:00
安正超 0e7e2660bb refactor: clean remaining storage DTO imports (#3568) 2026-06-18 14:44:46 +08:00
安正超 acdf439371 refactor: clean external DTO consumers (#3566)
* refactor: clean external DTO consumers

* fix: handle empty erasure shard recovery
2026-06-18 14:22:44 +08:00
安正超 99941f7e7c refactor: clean external operation consumers (#3565) 2026-06-18 12:39:41 +08:00
Henry Guo 46e200c22b feat(scanner): prioritize dirty usage refresh work (#3538) 2026-06-18 11:03:23 +08:00
安正超 57403525ee refactor: move heal and namespace contracts (#3560) 2026-06-18 11:00:21 +08:00
安正超 e5cad7ed20 refactor: move object operation contracts (#3559) 2026-06-18 09:48:13 +08:00
安正超 53337a6f71 refactor: move HTTP range helper contracts (#3533) 2026-06-18 00:47:23 +08:00
Henry Guo a9aba323c6 fix(scanner): restore prompt bucket usage scans (#3516)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-17 21:49:56 +08:00
安正超 ed55857bca refactor: move bucket operations contract (#3507) 2026-06-17 09:10:38 +08:00
Henry Guo b387689f26 feat(heal): expose scanner-aware operations status (#3483)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-15 22:28:53 +08:00
Henry Guo d28aedfbb3 feat(scanner): expose replication repair kind metrics (#3476) 2026-06-15 16:52:39 +08:00
abdullahnah92 c2e0792f6f fix(scanner): skip startup delay when replication is active so failed… (#3455)
fix(scanner): skip startup delay when replication is active so failed objects heal promptly after restart

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-15 09:02:21 +08:00
安正超 84f7027e90 refactor: add object store resolver for standalone crates (#3437) 2026-06-14 18:02:39 +08:00
houseme e8012bd1ba refactor(logging): normalize admin telemetry and error messages (#3430) 2026-06-14 13:27:10 +08:00
houseme 7da10db852 refactor(logging): standardize heal and scanner events (#3414)
* refactor(logging): standardize heal and scanner events

* chore(git): untrack local logging governance note

* chore(git): ignore local logging governance note
2026-06-14 01:47:39 +08:00
Henry Guo 19ff378e9b fix(scanner): preserve newer usage snapshots (#3370)
* fix(scanner): preserve newer usage snapshots

* fix(table-catalog): require namespace locking backend

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-12 07:06:08 +08:00
安正超 a85cc0354c refactor(storage-api): remove namespace lock from StorageAPI (#3365)
* refactor(storage-api): remove namespace lock from StorageAPI

* test(scanner): wait for runtime budget cancellation

---------

Co-authored-by: loverustfs <hello@rustfs.com>
2026-06-11 22:42:12 +08:00
安正超 7146c893cb refactor(storage-api): remove legacy bucket DTO re-export (#3364) 2026-06-11 21:24:56 +08:00
安正超 b4524033e3 refactor(config): move global config accessors (#3360) 2026-06-11 19:16:02 +08:00
安正超 69549634ea refactor(config): migrate scanner config consumer (#3356) 2026-06-11 17:30:26 +08:00
安正超 d4ee44f49f refactor(scanner): narrow cache storage bounds (#3348) 2026-06-11 13:52:35 +08:00
安正超 87968275a7 refactor(storage): route maintenance inventory reads (#3337) 2026-06-11 10:32:48 +08:00
Henry Guo 91f80a5585 feat(scanner): expose lifecycle transition status (#3326)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-10 16:45:02 +00:00
houseme b1339a98ed fix(scanner): delay deep verify for fresh objects (#3329)
Gate scanner-triggered deep heal behind a short new-object cooldown and retry transient bitrot shard size mismatches during local verification.

Add targeted tests for cooldown mode selection and size-mismatch classification, and log when deep scans are downgraded or size-mismatch retries occur.
2026-06-10 15:50:44 +00:00
Henry Guo 62e9c5b94f feat(scanner): add adaptive pacing controls (#3315)
* feat(scanner): add adaptive pacing controls

* fix(scanner): bound pacing delay status precision

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-10 03:46:02 +00:00
Henry Guo e79c793803 feat(heal): add scanner-aware bitrot controls (#3297)
* feat(heal): add scanner-aware bitrot controls

* fix(config): register heal defaults

* test(admin): avoid heal config queue conflict

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-09 13:43:18 +00:00
Henry Guo ad9bf41fc8 feat(heal): expose scanner heal admission outcomes (#3292)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-09 03:39:57 +00:00
Henry Guo 6413df4b7f feat(replication): expose scanner repair outcomes (#3278)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-08 14:24:27 +00:00
cxymds 452c003341 fix(scanner): ignore missing rebalance metadata (#3282)
* fix(ecstore): harden rebalance data movement

* fix(ecstore): preserve failed rebalance status

* fix(ecstore): avoid rebalance walkdir total timeout

* fix(ecstore): retry rebalance listing timeouts

* fix(ecstore): restrict data movement resume target

* refactor(ecstore): simplify multipart movement target

* fix(ecstore): restore resume target checks

* perf(ecstore): speed up rebalance bucket merges

* fix: keep rebalance listing alive on transient failures

* fix(scanner): ignore missing rebalance metadata

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 14:24:16 +00:00
cxymds 9504dff595 fix(ecstore): harden rebalance data movement (#3234)
* fix(ecstore): harden rebalance data movement

* fix(ecstore): preserve failed rebalance status

* fix(ecstore): avoid rebalance walkdir total timeout

* fix(ecstore): retry rebalance listing timeouts

* fix(ecstore): restrict data movement resume target

* refactor(ecstore): simplify multipart movement target

* fix(ecstore): restore resume target checks

* perf(ecstore): speed up rebalance bucket merges

* fix: keep rebalance listing alive on transient failures

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 10:40:26 +00:00
Henry Guo f40dd2f93c fix(scanner): publish partial usage for compacted scans (#3277)
* fix(scanner): publish partial usage for compacted scans

* fix(scanner): publish first partial usage immediately

* fix(scanner): skip startup delay for cold usage cache

* fix(scanner): tighten cold usage publish gate

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 06:18:35 +00:00
Henry Guo f49df41db9 fix(lifecycle): harden scanner ILM expiry accounting (#3257)
* fix(lifecycle): harden scanner ILM expiry accounting

* fix(scanner): gate ILM action accounting on enqueue

* fix(metrics): avoid scanner source work argument list

* fix(scanner): gate local ILM accounting on enqueue

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-07 16:18:34 +00:00
Henry Guo 4ae070c8cc feat(scanner): expose scan partial source status (#3247)
* feat(scanner): expose scan partial source status

* fix(scanner): expose partial source in aggregated metrics

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-07 06:40:04 +00:00
Henry Guo b36a730e48 feat(scanner): expose checkpoint and source work status (#3230)
* feat(scanner): expose checkpoint and source work status

* fix(scanner): count ignored checkpoints once per scan

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-06 10:47:39 +00:00
Henry Guo 0d445afda9 feat(scanner): add versioned scan checkpoints (#3220)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-05 13:49:41 +00:00
Henry Guo 542720a1f7 feat(scanner): add partial scan resume hints (#3207)
* feat(scanner): add partial scan resume hints

* test(scanner): cover clearing scan resume hints

* fix(scanner): apply resume hint to combined child order

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-04 09:21:30 +00:00
Henry Guo 6850457707 feat(scanner): add runtime scanner controls and status (#3203)
* feat(scanner): add runtime scanner controls and status

* fix(scanner): validate persisted scanner config

* docs(scanner): clarify start delay behavior

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-04 05:16:26 +00:00
Henry Guo ad1a489f75 feat(scanner): add scanner budget progress controls (#3185)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-03 14:37:58 +00:00
Henry Guo cc07946782 feat(scanner): add cycle budget observability (#3166)
* feat(scanner): add cycle budget observability

* fix(scanner): clear cycle state after budget stop

* test(scanner): stabilize timeout-based scanner tests

* fix(scanner): keep cycle ILM counts scanner-only

* test(scanner): avoid test-only pending import

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-06-03 03:32:41 +00:00
Henry Guo 1d46047d6f feat(scanner): expand scanner observability metrics (#3159)
* feat(scanner): expand scanner observability metrics

* chore(scanner): align bucket-drive metric wording

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-02 01:29:53 +00:00