Commit Graph

2857 Commits

Author SHA1 Message Date
overtrue b6f135f0fe fix(ecstore): drop removed decommission test import 2026-08-22 23:28:14 +08:00
overtrue b07f0a92a8 merge: sync main into ILM decommission fix 2026-08-22 22:55:14 +08:00
Zhengchao An 9815694301 fix(ecstore): supervise decommission worker cleanup (#6372) 2026-08-22 14:33:24 +00:00
houseme 0e79106c2f fix(storageclass): use div_ceil for inline threshold to match shard size calc (#6390)
The inline_block threshold used floor division (DEFAULT_INLINE_OBJECT_BUDGET
/ data_shards) while shard_file_size uses ceiling division (div_ceil). For
EC 12:4 with 256KiB objects, this caused a 1-byte discrepancy:
- inline_block = 262144 / 12 = 21845 (floor)
- shard_file_size = 262144.div_ceil(12) = 21846 (ceil)
- should_inline(21846, 12, false) = false (wrong!)

Fix by using div_ceil for the inline_block calculation, so both sides
use the same rounding and the inline path is correctly triggered.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-22 22:05:20 +08:00
Zhengchao An 12a9e654b5 refactor(data-usage): rename ReplicationStats to ReplicationTargetUsage (#6345)
* refactor(data-usage): ReplicationStats -> ReplicationTargetUsage

Rename the data-usage crate's ReplicationStats to ReplicationTargetUsage.
Serde field names are byte-identical (only the Rust type name changed;
field identifiers that rmp encodes are untouched). An rmp round-trip test
guards against future drift.

Scanner test imports updated to match.

* style: cargo fmt
2026-08-22 13:56:58 +00:00
Zhengchao An 6b5e0feef6 test(e2e): wait for heal peers after node rejoin (#6359) 2026-08-22 13:52:08 +00:00
cxymds da90d02c15 test(ecstore): cover suspended-owner heal semantics (#6348)
* test(ecstore): cover suspended-owner heal semantics

* test(heal): cover suspended owner production path
2026-08-22 20:45:03 +08:00
唐小鸭 a930152d5a fix(admin): expose per-target disableProxy through remote target admin API (#6376)
The read-proxy selector already honors a target's disable_proxy flag
(PR #6172), but the admin API still rejected the field, so the only way
to set it was importing a MinIO-written bucket-targets.json.

- move disableProxy from REMOTE_TARGET_UNSUPPORTED_FIELDS to
  REMOTE_TARGET_WRITABLE_FIELDS (set-remote-target create accepts it)
- add TargetUpdateOp::Proxy so set-remote-target?update=true&proxy=true
  overlays only the proxy group (MinIO TargetUpdateType parity)
- bump REMOTE_TARGET_CAPABILITY_CONTRACT_VERSION 1 -> 2 and update the
  runtime capability pin tests
- keep edge/edgeSyncBeforeExpiry rejected (no implementation behind them)
- pin that a published TargetClient carries disable_proxy, the field the
  proxy-target selector consults

Refs rustfs/backlog#1950
2026-08-22 20:43:52 +08:00
cxymds f9d45e41e1 fix(quota): account compressed deletes by committed size (#6365) 2026-08-22 11:39:01 +00:00
overtrue 34a52a1e7d fix(ecstore): restore decommission test imports 2026-08-22 19:13:21 +08:00
cxymds 04e1ea227a fix(scanner): isolate corrupt cycle state (#6354)
* fix(scanner): isolate corrupt cycle state

* fix(scanner): preserve newer state during recovery reset

* fix(scanner): fence recovery reset state

* fix(scanner): reject terminal recovery epochs

* fix(scanner): reject trailing cycle state bytes

* fix(scanner): reject terminal leadership epochs

* fix(scanner): retain recovery wake notifications

* fix(scanner): recover from oversized markers
2026-08-22 11:11:48 +00:00
overtrue fe2d166cd3 fix(ecstore): distinguish active ILM target cleanup 2026-08-22 18:43:05 +08:00
overtrue c687f95260 fix(ecstore): preserve active ILM source journals 2026-08-22 18:43:05 +08:00
overtrue fe66aa509e test(ecstore): isolate durable ILM scenario stack 2026-08-22 18:43:05 +08:00
overtrue 918686bd4e test(ecstore): compile multi-source ILM recovery 2026-08-22 18:43:05 +08:00
overtrue ee7a82ec1f test(ecstore): cover durable ILM recovery boundaries 2026-08-22 18:43:05 +08:00
overtrue 3229ce05e3 fix(ecstore): repair durable ILM receipt recovery 2026-08-22 18:43:04 +08:00
overtrue 7428f3138c fix(ecstore): harden durable ILM cursor receipts 2026-08-22 18:43:04 +08:00
overtrue cf0a78ce11 fix(ecstore): avoid terminal receipt shadowing 2026-08-22 18:43:04 +08:00
overtrue 7bda7f4825 fix(ecstore): re-export durable ILM checkpoint 2026-08-22 18:43:04 +08:00
overtrue 20a8c443af fix(ecstore): anchor decommission ILM receipts 2026-08-22 18:43:04 +08:00
overtrue 948ee1fc52 fix(ecstore): close ILM receipt recovery gaps 2026-08-22 18:43:04 +08:00
overtrue ecd779b6f3 fix(ecstore): track ILM recovery across decommission 2026-08-22 18:43:04 +08:00
overtrue 619fae2512 fix(ecstore): verify ILM metadata before decommission 2026-08-22 18:43:04 +08:00
overtrue 4f9d22bda7 fix(ecstore): migrate ILM metadata during decommission 2026-08-22 18:43:04 +08:00
Zhengchao An 1a3be70d98 fix(ecstore): preserve remote delete error types (#6371) 2026-08-22 17:07:02 +08:00
cxymds 8679570c2a fix(heal): retain displaced task status (#6370) 2026-08-22 08:23:40 +00:00
Zhengchao An 7143697a5f fix(rpc): bound internode concurrency under multipart load (#6368) 2026-08-22 06:42:10 +00:00
cxymds a34310a58f fix(ecstore): fail closed on unresolved decommission entries (#6367)
* fix(ecstore): fail closed on unresolved decommission entries

* perf(ecstore): avoid successful listing name clone
2026-08-22 14:12:28 +08:00
cxymds 2e60029079 perf(ecstore): throttle decommission checkpoints (#6356) 2026-08-22 13:49:23 +08:00
houseme 2f0918f60b feat(disk): fsync dedicated blocking pool (default-off) (#6366) 2026-08-22 11:24:24 +08:00
Zhengchao An 98c4675617 refactor(e2e): consolidate duplicated helper functions into common.rs (#6355) 2026-08-22 01:10:14 +00:00
Zhengchao An eec34331de chore(madmin): remove dead trace structs, keep TraceType only (#6343)
chore(madmin): remove dead trace structs, keep TraceType bitflag helper only

TraceInfo, TraceInfoLegacy, TraceHTTPStats, TraceCallStats, TraceRequestInfo,
TraceResponseInfo, StorageStats, and OSStats are unreferenced outside trace.rs.
Trim to TraceType + its bitflag operations which are actively used by
service_commands.rs and profile_admin.rs.

-139 lines (215 -> 76 lines)
2026-08-22 01:01:24 +00:00
Zhengchao An 5d820df79c chore(obs): rename ReplicationStats to ReplicationMetricsSnapshot (#6344)
* chore(obs): ReplicationStats -> ReplicationMetricsSnapshot, BucketReplicationStats -> BucketReplicationMetricsSnapshot

Rename in-obs-crate ReplicationStats and BucketReplicationStats to
ReplicationMetricsSnapshot and BucketReplicationMetricsSnapshot respectively.
No serde impact (these types are Prometheus metric collectors, not serialized).
No external consumers found outside the obs crate.

* cleanup: remove #[serial] annotations from e2e_test, scanner, lifecycle, and object-capacity crates

Remove no-op #[serial] attributes (nextest ignores serial_test) and the
serial_test dependency from four crates. All tests already use temp_env
for env-var isolation, making #[serial] purely redundant.

Crates cleaned:
- e2e_test (37 annotations, 9 imports, removed serial_test dep)
- rustfs-scanner (115 annotations across 7 files, removed serial_test dep)
- rustfs-lifecycle (46 annotations, removed serial_test dep)
- rustfs-object-capacity (38 annotations, removed serial_test dep)

Also converted scanner/tests/lifecycle_integration_test.rs
with_forced_immediate_enqueue_timeout helper from unsafe raw
env::set_var/remove_var to temp_env::async_with_vars for proper
isolation, and added async_closure feature to scanner's temp-env dep.

* fix(lifecycle): restore #[serial] on 2 tests that read env vars without temp_env

eval_inner_expires_latest_object_after_days_due and
eval_inner_does_not_panic_on_many_equal_due_events call eval_inner()
which reads ENV_ILM_PROCESS_TIME via std::env::var(). Without #[serial]
they race with other tests that set these vars via temp_env.

* style: cargo fmt
2026-08-22 00:30:30 +00:00
Zhengchao An 37c5ce1399 docs(agents): streamline instruction routing (#6358) 2026-08-22 08:19:53 +08:00
Zhengchao An 61b5edf16e chore(ecstore): standardize 24 bare TODO/FIXME to tracked format (#6346) 2026-08-22 07:09:32 +08:00
Zhengchao An bc07cfd115 ci: harden test selection and nightly coverage (#6341) 2026-08-21 15:04:08 +00:00
cxymds adb90fc6e1 fix(scanner): defer usage publication during pool recovery (#6333)
* fix(scanner): defer usage publication during pool recovery

* fix(scanner): preserve metrics when publication is deferred

* fix(scanner): route test types through storage boundary

* fix(scanner): keep cache floor deferred during movement
2026-08-21 17:32:59 +08:00
cxymds cdfac5d7e3 fix(ecstore): avoid decommission walk deadline on backpressure (#6332)
fix(ecstore): bound decommission background walks
2026-08-21 17:32:12 +08:00
GatewayJ 23a0f6324c fix(iam): preserve MinIO permanent credentials in migration (#6328)
* fix(iam): preserve MinIO permanent credentials in migration

* test(iam): cover MinIO credential migration end to end
2026-08-21 15:34:25 +08:00
houseme 122a69df65 feat(ecstore): tune fdatasync group wait budget (#6327)
* feat(ecstore): tune fdatasync group wait budget

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ecstore): cover fdatasync wait budget contract

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-21 14:08:21 +08:00
Henry Guo 1aae680373 fix(lifecycle): honor bucket default retention (#6324)
* fix(lifecycle): honor bucket retention during scanner expiry

* fix(lifecycle): reject zero object lock retention

* test(lifecycle): cover malformed retention metadata

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-08-21 10:11:13 +08:00
houseme 4283591838 feat(ecstore): observe PUT commit lock admission (#6319)
* feat(ecstore): observe PUT commit lock admission

Co-Authored-By: heihutu <heihutu@gmail.com>

* update h2 v0.4.18

* test(e2e): box SSE-KMS negative errors

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-21 01:18:53 +00:00
Zhengchao An f2957a680d test: stabilize release-blocking full E2E checks (#6322) 2026-08-21 08:43:12 +08:00
Zhengchao An d22cb5d07a fix: resolve release-blocking integration failures (#6320)
* fix: resolve release-blocking integration failures

* fix: satisfy stable clippy lints

* fix: satisfy Rust 1.98 CI lints
2026-08-21 06:39:29 +08:00
houseme 762919b1ba perf(scanner): reduce per-object allocation churn (#6318)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-20 16:47:48 +00:00
houseme cee0d5cf9b perf(io-metrics): cache read version metric handles (#6317)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-20 16:39:26 +00:00
GatewayJ 205337151a fix(webdav): allow bucket-scoped root listings (#6298)
* fix(webdav): allow bucket-scoped root listings

* test(webdav): use public protocol export

* test(webdav): initialize identity inline

---------

Co-authored-by: cxymds <cxymds@gmail.com>
2026-08-21 00:15:33 +08:00
Henry Guo 105b6fbfde fix(scanner): honor explicit cycle cadence (#6313)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-08-20 23:35:03 +08:00
houseme b2e573c48b feat(ecstore): bound put commit lock admission (#6315)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-20 23:33:17 +08:00