Commit Graph

3257 Commits

Author SHA1 Message Date
Henry Guo 66fd55a8e0 feat(scanner): expose pacing pressure status (#3319)
* feat(scanner): expose pacing pressure status

* fix(scanner): preserve merged pause pressure

* fix(scanner): default missing primary pressure

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-10 07:33:41 +00:00
安正超 bb5d9565a6 feat(storage-api): add bucket DTO contract (#3314)
* feat(storage-api): add bucket DTO contract

* ci(build): increase workflow timeout to 90 minutes

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-10 07:16:14 +00:00
houseme b9c924a6ed build(ci): update macOS runner and bump regex/uuid (#3318)
* ci(build): use macos-26-intel for x86_64 job

* build(deps): bump regex and uuid

* ci(build): disable cross for macos x86_64 job

---------

Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com>
2026-06-10 04:42:10 +00:00
houseme 68400933b5 chore(release): prepare 1.0.0-beta.8 (#3317)
* chore(release): prepare 1.0.0-beta.8

* chore(release): align release assets for 1.0.0-beta.8
2026-06-10 04:34:39 +00:00
安正超 0dc99a1abb docs: inventory KMS development defaults (#3304) 2026-06-10 12:46:08 +08:00
Henry Guo d7a7d1fc1e feat(table-catalog): add REST catalog alias (#3316)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-10 03:55:58 +00:00
Henry Guo 62e9c5b94f feat(scanner): add adaptive pacing controls (#3315)
* feat(scanner): add adaptive pacing controls

* fix(scanner): bound pacing delay status precision

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-10 03:46:02 +00:00
安正超 f80162b5c9 docs: decide config model boundary (#3307) 2026-06-10 10:02:53 +08:00
Henry Guo 2eafd9c602 feat(table-catalog): add credential response boundary (#3305)
* feat(table-catalog): add credential response boundary

* fix(table-catalog): enforce scoped catalog resources

* fix(table-catalog): reduce admin auth scope arguments

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-10 01:15:51 +00:00
安正超 a73c90c811 security: redact IAM and target debug secrets (#3306) 2026-06-10 09:03:50 +08:00
安正超 3795d44b86 feat(storage-api): add error code contract (#3313) 2026-06-10 09:03:29 +08:00
安正超 5fef105484 docs: define background controller contract (#3312) 2026-06-10 07:30:21 +08:00
安正超 bc95b9fb9e test: guard table catalog ingress unknown fields (#3308) 2026-06-09 20:06:36 +00:00
安正超 45391ac776 feat(storage-api): add contract crate (#3310) 2026-06-09 20:06:15 +00:00
安正超 f9a5e6d7e6 docs: inventory background services (#3311) 2026-06-10 04:05:33 +08:00
安正超 03eb10b07f feat: add KMS config redaction safeguards (#3303) 2026-06-09 14:08:58 +00:00
Henry Guo 7ec16e197c feat(table-catalog): add metadata maintenance control plane (#3302)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-09 22:08:28 +08:00
Julien Pervillé 992de65c58 feat(helm): add priorityClassName attribute (#3301)
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-09 22:07:41 +08:00
Maksim Vykhota c4d0d395af Refactor(CLI): avoid unnecessary unreachable! macro in CLI parsing (#3288)
* Refactor(CLI): avoid unnecessary `unreachable!` macro in CLI parsing

* Fix: clippy lint

---------

Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-09 22:01:51 +08:00
Henry Guo e79c793803 feat(heal): add scanner-aware bitrot controls (#3297)
* feat(heal): add scanner-aware bitrot controls

* fix(config): register heal defaults

* test(admin): avoid heal config queue conflict

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-09 13:43:18 +00:00
Henry Guo 0cdcd1eb7b feat(table-catalog): internalize catalog backing paths (#3295)
* feat(table-catalog): internalize catalog backing paths

* fix(table-catalog): clean internal catalog on bucket delete

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-09 12:01:25 +00:00
cxymds 55590e38fb fix(config): accept Kafka SASL keys in legacy admin (#3300) 2026-06-09 12:01:15 +00:00
安正超 f5bb034ec8 feat(kms): migrate KMS handlers to dedicated actions (#3298)
feat: migrate KMS handlers to dedicated actions
2026-06-09 12:01:02 +00:00
安正超 4fec606dc4 feat: add KMS action taxonomy (#3294) 2026-06-09 05:06:15 +00:00
Henry Guo ad9bf41fc8 feat(heal): expose scanner heal admission outcomes (#3292)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-09 03:39:57 +00:00
Henry Guo 51c26278a4 feat(table-catalog): support standard REST create commit (#3287)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-09 02:18:33 +00:00
安正超 b9a3a2245a feat: add admin route policy inventory (#3286)
* feat: add admin route policy inventory

* fix: align table route policy actions

* fix: align load table route policy action
2026-06-09 02:17:50 +00:00
houseme aabda41ae5 build: upgrade Rust baseline to 1.96.0 (#3291)
* build: upgrade Rust baseline to 1.96.0

* ci: pin Rust workflows to 1.96.0

* remove

* ci: pin setup action to Rust 1.96.0

* build: pin toolchain to Rust 1.96.0
2026-06-08 20:32:37 +00:00
houseme 9354b939ca refactor(zip): simplify archive extraction path (#3290)
* refactor(zip): simplify archive extraction path

* build(deps): align datafusion and http updates

* fix(test): restore pre-commit compatibility
2026-06-08 20:03:02 +00:00
Henry Guo 6413df4b7f feat(replication): expose scanner repair outcomes (#3278)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-08 14:24:27 +00:00
cxymds 452c003341 fix(scanner): ignore missing rebalance metadata (#3282)
* fix(ecstore): harden rebalance data movement

* fix(ecstore): preserve failed rebalance status

* fix(ecstore): avoid rebalance walkdir total timeout

* fix(ecstore): retry rebalance listing timeouts

* fix(ecstore): restrict data movement resume target

* refactor(ecstore): simplify multipart movement target

* fix(ecstore): restore resume target checks

* perf(ecstore): speed up rebalance bucket merges

* fix: keep rebalance listing alive on transient failures

* fix(scanner): ignore missing rebalance metadata

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 14:24:16 +00:00
Henry Guo 8c3e52efb8 feat(table-catalog): refine table catalog permissions (#3283)
* feat(table-catalog): refine table catalog permissions

* fix(policy): scope table admin resources

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-08 14:23:52 +00:00
唐小鸭 f7724d223b feat(rio): rio_v2 is compatible with minio for storing data. (#3115)
* Set up a compatibility layer for replacing old Rio components with new ones.

* fix(rio). compress range

* feat(rio). Add the experimental feature rio_v2 to support minio data at the binary level.

* feat(rio_v2): add sse-c test

* test compression component

* simple fix

* fix minlz encode

* fix metadata

* fix kms key cache error

* Update launch.json

* ci: set nix crate download user agent

* fix: gate obs pyroscope backend

* ignore minio test

* fix encrypt check

* fix

* fix

* fix

* Update object_usecase.rs

* Update ci.yml

* fix

* ci add rio-v2 test

* fix

* ci fix

* fix

* Reconstructed into a more reasonable compatibility mode

* fix

* fix

---------

Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-06-08 11:59:14 +00:00
cxymds 9504dff595 fix(ecstore): harden rebalance data movement (#3234)
* fix(ecstore): harden rebalance data movement

* fix(ecstore): preserve failed rebalance status

* fix(ecstore): avoid rebalance walkdir total timeout

* fix(ecstore): retry rebalance listing timeouts

* fix(ecstore): restrict data movement resume target

* refactor(ecstore): simplify multipart movement target

* fix(ecstore): restore resume target checks

* perf(ecstore): speed up rebalance bucket merges

* fix: keep rebalance listing alive on transient failures

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 10:40:26 +00:00
Michael 2fc1cf90e0 feat(helm): add option to disable log PVCs and mounts (#3189)
Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-08 10:35:12 +00:00
houseme ed73952cb6 perf(ecstore): improve erasure write diagnostics and single-block performance (#3280)
* docs(object-capacity): add localized crate docs

* fix(ecstore): improve quorum and transport diagnostics

* perf(ecstore): add safe single-block write fast path

* refactor(ecstore): collapse layered small write paths

* chore(docs): keep issue 662 design note tracked

* fix(docs): restore issue 662 design note

* chore(docs): keep issue 662 design local only

* feat(obs): add internode reliability metrics and dashboard

* feat(obs): extend internode diagnostics and service logging

* fix(docs): use AGENTS guide filename

* perf(ecstore): reuse owned buffer in small encode

* fix(ecstore): tighten small write diagnostics

---------

Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-08 09:45:56 +00:00
cxymds ddd35badad fix(ecstore): avoid offline disks on admin timeout (#3263)
* fix(ecstore): avoid offline disks on admin timeout

* fix(ecstore): handle poisoned admin cache locks

* fix(ecstore): recover poisoned admin cache on success

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-06-08 08:35:36 +00:00
Henry Guo f40dd2f93c fix(scanner): publish partial usage for compacted scans (#3277)
* fix(scanner): publish partial usage for compacted scans

* fix(scanner): publish first partial usage immediately

* fix(scanner): skip startup delay for cold usage cache

* fix(scanner): tighten cold usage publish gate

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 06:18:35 +00:00
Henry Guo 4751a9f4b9 feat(table-catalog): add recovery diagnostics (#3275)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 03:59:13 +00:00
安正超 3d0e6ce0da feat: add security governance policy contracts (#3271)
* feat: add security governance policy contracts

* fix: require signatures for release assets

* docs: update policy verification counts

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 02:42:03 +00:00
安正超 78df276d25 fix: return 503 on lock contention instead of 500 (#3274)
* fix(ecstore): skip hidden metadata in walk limit

* fix(ecstore): match walk limit to visible versions

* fix(ecstore): avoid decoding all versions for limit

* fix: return 503 on lock contention instead of 500

When concurrent PUTs to the same key contend for the namespace write
lock, lock timeout and conflict errors were wrapped as
StorageError::other(...) → StorageError::Io(...), which fell through
to S3ErrorCode::InternalError (500) in the error mapping.

Only QuorumNotReached was correctly mapped to ServiceUnavailable (503);
all other lock errors (Timeout, AlreadyLocked, etc.) became 500.

Fix: map_namespace_lock_error now returns StorageError::Lock(err) for
non-quorum lock errors, and StorageError::Lock is mapped to
S3ErrorCode::ServiceUnavailable in the HTTP error conversion.

Affected paths:
- crates/ecstore/src/set_disk/lock.rs
- crates/ecstore/src/store/object.rs
- crates/ecstore/src/store/bucket.rs (make_bucket, delete_bucket)
- rustfs/src/app/object_usecase.rs (copy_object)

Regression test: test_concurrent_put_same_key_never_returns_500

* test: fail on unexpected lock contention errors

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-08 02:20:21 +00:00
houseme c479f3d0cb fix(ecstore): gate rustix fs diagnostics on windows (#3267)
* fix(ecstore): gate rustix fs diagnostics on windows

* fix(ecstore): improve windows disk validation diagnostics

* build(deps): bump workspace dependency versions
2026-06-08 00:05:55 +00:00
Henry Guo c452dd8ad7 feat(table-catalog): add metadata maintenance cleanup (#3266)
* feat(table-catalog): add metadata maintenance dry run

* feat(table-catalog): add metadata-only maintenance delete

* fix(table-catalog): protect recent metadata cleanup candidates

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-08 00:05:41 +00:00
安正超 7a9bf707ee feat: add security governance contract types (#3270) 2026-06-07 23:14:11 +00:00
安正超 c03c0ebc36 test: add admin route matrix guard (#3268) 2026-06-08 05:30:18 +08:00
安正超 dee550a831 ci: add architecture migration rule checks (#3264)
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-07 16:23:38 +00:00
Henry Guo f49df41db9 fix(lifecycle): harden scanner ILM expiry accounting (#3257)
* fix(lifecycle): harden scanner ILM expiry accounting

* fix(scanner): gate ILM action accounting on enqueue

* fix(metrics): avoid scanner source work argument list

* fix(scanner): gate local ILM accounting on enqueue

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-07 16:18:34 +00:00
houseme cebea9a238 fix(ecstore): add disk validation diagnostics (#3265)
Log local endpoint disk-topology details during physical disk independence checks.
Include canonical paths, st_dev major:minor values, and resolved device ids in validation output to make Docker and bind-mount startup failures easier to diagnose.

Also assert the new diagnostics are present in the shared-disk regression test.
2026-06-07 15:32:38 +00:00
唐小鸭 74296761fa fix(replication): repair site replication setup (#3252)
* fix: repair site replication setup

* Optimized site status collection

---------

Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-07 13:19:50 +00:00
安正超 6f4d0b54a1 fix(ci): install ripgrep for script checks (#3260) 2026-06-07 22:03:25 +08:00