Commit Graph

112 Commits

Author SHA1 Message Date
安正超 97dd825468 fix(ecstore): optimize ObjectInfo clone and fix critical TODOs (#3149)
* fix(ecstore): optimize ObjectInfo clone and fix critical TODOs

## ObjectInfo hot-path clone optimization (issue #653 item 2)
- Wrap user_defined (HashMap), user_tags (String), parts (Vec) in Arc
- Clone cost reduced from ~20 heap allocations to O(1) ref count bump
- Updated 11 downstream access sites with explicit deref where needed

## Critical TODO/FIXME fixes (issue #653 item 7)
- rpc/peer_s3_client.rs: descriptive error message for empty peer response
- set_disk/write.rs: concurrent rollback deletes via tokio::spawn + join_all
- store_list_objects.rs: resolved FIXME with explanation + 4 regression tests
- store/bucket.rs: namespace write locks for make_bucket and delete_bucket

Skipped TODOs (too risky without broader context):
- multipart.rs:30 nslock — causes lock timeout in existing tests
- bucket.rs:88 cached list_bucket — needs cache invalidation strategy
- bucket.rs:149 replication delete — needs replication subsystem integration

1134 tests pass (1 flaky under parallel execution, passes individually).

* fix: address PR #3149 review comments

- Preserve NamespaceLockQuorumUnavailable variant in bucket create/delete
- Fix cancellation test to actually test cancellation (keep senders open)
- Update ObjectInfo consumers in app/ for Arc-backed fields
- Update bucket_usecase.rs user_tags to Arc<String>

* fix: update ObjectInfo Arc consumers

* fix: address ecstore merge review comments

* fix: cancel blocked merge output sends

* fix(ecstore): satisfy ObjectInfo clone clippy
2026-06-01 00:44:58 +00:00
houseme 8577bd825e feat(admin): restore config admin compatibility (#3133)
* feat(admin): restore config admin compatibility

Co-authored-by: weisd <im@weisd.in>

* fix(admin): align config admin clean rebuild

Co-authored-by: weisd <im@weisd.in>

* fix(admin): align config history and peer signals

* fix(admin): harden config admin mutations

* fix(admin): tighten config review follow-ups

* perf(admin): reuse env snapshot in config render

* fix(ecstore): clean up config admin and listing error handling

Remove redundant is_all_volume_not_found check in list_merged, add
storage class encode/decode roundtrip tests, fresh boot integration
test, and config admin clean rebuild improvements.

Co-authored-by: hehutu <heihutu@gmail.com>

* fix(admin): sync global server config on mutation and reload

Change GLOBAL_SERVER_CONFIG from OnceLock to RwLock so config mutations
(set/del/restore/reload) are visible to readers without restart. Call
set_global_server_config after every store save and on snapshot reload.
Register storage_class as a dynamic config subsystem.

Co-authored-by: hehutu <heihutu@gmail.com>

* style: apply rustfmt to config and admin tests

Co-authored-by: hehutu <heihutu@gmail.com>

* fix(test): update signal_service test for storage_class dynamic subsystem

storage_class is now a valid dynamic config subsystem, so the
"requires object layer" test should expect "storage layer not initialized"
instead of "unsupported dynamic config subsystem".

Co-authored-by: hehutu <heihutu@gmail.com>

* fix(config): publish storage_class runtime config on dynamic reload

Change GLOBAL_STORAGE_CLASS from OnceLock to RwLock so runtime updates
are possible. apply_storage_class_runtime_config now actually publishes
the parsed config via set_global_storage_class instead of dropping it.

Addresses review feedback: storage_class was marked as dynamically
applied but the parsed result was discarded, so mc admin config set
returned config_applied=true while the runtime kept using stale parity
settings until restart.

Co-authored-by: hehutu <heihutu@gmail.com>

* fix(admin): harden config init, history ordering, and env redaction

- Change GLOBAL_SERVER_CONFIG from RwLock<Config> to RwLock<Option<Config>>
  initialized with None, preserving "not initialized" detection via None
- Move save_server_config_history before save_server_config_to_store in
  SetConfigKVHandler, DelConfigKVHandler, and SetConfigHandler so a
  restore point exists before mutations are persisted
- Redact sensitive env override values with *redacted* instead of
  silently omitting the line, improving admin visibility
- Add code comment explaining VolumeNotFound removal rationale in
  list_merged for listing paths

Co-authored-by: hehutu <heihutu@gmail.com>

* fix(config): keep in-memory config in sync after set/restore/reload

GLOBAL_SERVER_CONFIG was a OnceLock set once at startup and never
updated. After mc admin config set writes to the store, any fallback
to get_global_server_config() returned stale init-time data. Similarly,
reload_runtime_config_snapshot read from the store but discarded the
result.

- Replace OnceLock with RwLock for GLOBAL_SERVER_CONFIG and
  GLOBAL_STORAGE_CLASS so they can be updated at runtime
- Add set_global_server_config / set_global_storage_class setters
- Call set_global_server_config after every config save (set-kv,
  del-kv, set-config, restore-history)
- Re-apply dynamic subsystems (storage_class, audit_webhook,
  audit_mqtt) and signal peers in reload_runtime_config_snapshot
  and full-config operations
- Fix render_selected_config scope boundary check: track per-scope
  line count instead of checking global lines.is_empty()
- Include STORAGE_CLASS_SUB_SYS in is_dynamic_config_subsystem so
  apply_storage_class_runtime_config is reachable

Co-authored-by: hehutu <heihutu@gmail.com>

* fix(storageclass): use CLASS_RRS key in lookup_config for RRS parity

lookup_config used kvs.get(RRS) where RRS="REDUCED_REDUNDANCY", but the
admin config path writes the key as CLASS_RRS="rrs". This caused RRS
values to never be read back, always falling back to default parity.

- Changed kvs.get(RRS) to kvs.get(CLASS_RRS) in lookup_config
- Added regression tests verifying RRS read/write consistency

Co-authored-by: hehutu <heihutu@gmail.com>

* fix(config): add peer-side logging and don't swallow apply errors

- Add tracing::warn! in reload_dynamic_config_runtime_state and
  reload_runtime_config_snapshot when config read or subsystem apply
  fails, so on-host diagnostics show which signal failed and why
- Change `let _ = apply_dynamic_config_for_subsystem(...)` to
  `if let Err(err) = ... { warn!(...) }` in reload_runtime_config_snapshot
  so per-subsystem failures are logged instead of silently swallowed
- Remove weak test global_server_config_returns_none_before_init that
  had no meaningful assertion due to shared global state

Co-authored-by: hehutu <heihutu@gmail.com>

* style: apply rustfmt to config and storageclass tests

Co-authored-by: hehutu <heihutu@gmail.com>

---------

Co-authored-by: weisd <im@weisd.in>
Co-authored-by: hehutu <heihutu@gmail.com>
2026-05-31 11:50:13 +00:00
houseme 088c4bda43 fix(ecstore): harden issue3031 multipart validation path (#3106)
* fix(ecstore): harden issue3031 multipart validation path

- clear stale multipart part destinations before rename fan-out
- add repeated part overwrite regression coverage
- reduce remote disk startup false-fault escalation to suspect-first
- refine remote locker diagnostics and lower scanner leader-lock log noise
- add a dedicated 4-node issue3031 docker validation script

* refactor(admin): inline console version json macro

- drop the unused serde_json::json import in admin console
- call serde_json::json! inline in version_handler
- keep the console version response behavior unchanged

* fix(remote-disk): recover suspect health on probe success

- record probe success during remote disk health checks so suspect drives recover
- use async_with_vars for the remote disk health probe test
- make the missing-listener test assert the state transition more robustly
2026-05-28 14:26:31 +00:00
GatewayJ 247973f34c fix(lifecycle): make transition worker resize nonblocking (#3090)
Co-authored-by: 季宏伟 <jihongwei@jihongweis-MacBook-Pro.local>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-28 01:36:08 +00:00
houseme 909f0d57fb feat: improve degraded readiness reporting and shutdown handling (#3089)
* fix(server): unify runtime readiness and shutdown flow

* refactor(server): decouple readiness shared types

* refactor(server): keep readiness types crate-private

* refactor(server): await protocol shutdown handles

* fix(server): bypass cached startup readiness

* feat(health): expose degraded readiness reasons

* fix(health): preserve raw IAM readiness in degraded reports
2026-05-27 03:25:28 +00:00
weisd f9475e10dc fix(replication): preserve multipart pending state (#3058)
* Preserve multipart replication recovery state

Multipart uploads previously only scheduled replication after completion, leaving no persisted pending state for scanner recovery if the initial async work was lost. Persist the same pending replication metadata during multipart initialization and let completion evaluate the object metadata that was actually stored.

The scanner heal path also treated ordinary pending objects as delete-replication candidates. Restrict that path to delete markers and version purge state so pending objects remain eligible for object replication heal.

Constraint: Bucket replication recovery depends on persisted object metadata after the async queue is unavailable.
Rejected: Rely only on immediate completion-time scheduling | it cannot recover after process restart or worker loss.
Confidence: high
Scope-risk: moderate
Directive: Keep multipart upload initialization aligned with single PUT replication metadata semantics.
Tested: cargo fmt --all --check
Tested: cargo clippy --workspace --all-features --all-targets -- -D warnings
Tested: LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 make pre-commit
Tested: Runtime replication outage check confirmed multipart xl.meta stores PENDING status and timestamp.

* fix(replication): preserve version purge scanner state

Role-derived replication configs need target-scoped status strings before scanner heal can build per-target purge status. The duplicated replication-status assignment left version purge status unset, so scanner recovery could lose the target-level purge state.

Constraint: Scanner heal derives per-target purge decisions from version_purge_status_internal.
Rejected: Leave the duplicate as a harmless cleanup | it changes recovery behavior for role-only configs with version purge state.
Confidence: high
Scope-risk: narrow
Directive: Keep role-derived replication and version purge internal status mapping symmetric.
Tested: cargo fmt --all --check
Tested: cargo test -p rustfs-ecstore heal -- --nocapture

---------

Co-authored-by: wly <wlywly0735@126.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-24 14:24:00 +00:00
LeonWang0735 cbcfe625ef fix(replication): avoid skipping existing-object backfill for new targets (#2992)
* fix(replication): avoid skipping existing-object backfill for new targets

* optimize(replication): delete redundant line

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-23 04:53:08 +00:00
houseme ed2078e025 fix(ecstore): allow expired delete markers on locked buckets (#3048)
* fix: allow expired delete markers on locked buckets

* fix: reject zero-day del marker expiration
2026-05-22 01:39:53 +00:00
安正超 b530a9f0b2 fix(ecstore): remove stale bucket metadata parse TODO (#3021)
* fix(ecstore): remove stale bucket metadata parse TODO

* test: cover stored bucket target config parsing
2026-05-20 03:59:55 +00:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012)
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
安正超 e65aebba53 fix(ecstore): guard transition worker max (#3003)
* fix(ecstore): guard transition worker max

* test: clarify env mutation safety

* test(ecstore): document env mutation concurrency contract
2026-05-19 02:25:51 +00:00
houseme bd1e57293f fix: harden lifecycle transition compensation and regression coverage (#2995)
* fix(ecstore): honor transition worker configuration

* fix(ecstore): add transition queue backpressure metrics

* fix(ecstore): schedule transition compensation on enqueue pressure

* fix(ecstore): log transition compensation scheduling

* test(rustfs): add transition compensation fault-injection coverage

* test(rustfs): cover delete after transition compensation

* test(scanner): cover cleanup after transition compensation

* test(rustfs): extend compensation transition coverage

* test(scanner): cover backfill idempotency after compensation

* test(scanner): cover noncurrent expiry after compensation

* test(rustfs): cover versioned delete after compensation

* test(rustfs): cover delete marker lifecycle after compensation

* test(scanner): extend versioned lifecycle compensation coverage

* test(scanner): model versioned delete after compensation

* test(scanner): clarify modeled versioned delete helper

* refactor(ecstore): optimize transition enqueue hot path

* refactor(ecstore): centralize transition runtime constants

* style(ecstore): apply rustfmt for transition timeout helper

* fix(ilm): align queue-full metric semantics

* refactor(ecstore): unify immediate enqueue failure handling

* refactor(ecstore): reuse transition worker env constant

* ci(actions): update setup action inputs
2026-05-18 12:50:43 +00:00
安正超 5e0ca006f0 test(replication): cover ETag comparison edge cases (#2840) 2026-05-07 02:48:12 +00:00
Nikita Bakun 4d6171e996 fix(replication): handle version ID format mismatch with AWS S3 (#2829)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-06 21:30:07 +00:00
安正超 9b9e0db9f6 test(lifecycle): cover ILM process time aliases (#2839) 2026-05-06 21:12:22 +00:00
cxymds 70be0804ee fix: 2827 lifecycle days next midnight (#2833)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-06 15:00:53 +00:00
houseme 2953558f41 fix(lifecycle): prevent eager date-expiry deletion on config update (#2708) 2026-04-28 10:26:14 +00:00
houseme a68fe1601f fix(ilm): harden signer failures and guard remote tier delete storms (#2706) 2026-04-27 12:39:02 +00:00
weisd 334184b005 fix(replication): prevent target state loss across buckets (#2704) 2026-04-27 09:27:33 +00:00
weisd 468dc3aebd fix(replication): fan out single-bucket rules to all targets (#2701) 2026-04-27 06:51:19 +00:00
houseme 59f41eb86a feat(obs): improve metrics coverage and dashboard performance (#2682) 2026-04-25 18:51:29 +00:00
houseme 13b4500212 feat(obs): improve telemetry stack, replication metrics, and Grafana alignment (#2672)
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-24 13:50:17 +00:00
likewu a77be8f89b fix: some expect error (#2622)
Signed-off-by: likewu <likewu@126.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-21 03:27:50 +00:00
cxymds 1511f9eacb fix(lifecycle): correct delete replication fanout (#2609)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-20 10:02:59 +00:00
houseme 478720d2ee Centralize lifecycle state updates and fix systemd running status (#2567) 2026-04-17 03:20:11 +00:00
houseme 6ce24f3b63 feat(lifecycle): improve ILM compatibility and scanner runtime config (#2534)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-16 10:28:03 +00:00
Tunglies 579b124726 lint: clippy rules or_fun_call (#2561)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-16 02:48:39 +00:00
Tunglies e05d07494e chore(lint): clippy rules large_futures (#2555)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-16 02:10:13 +00:00
Tunglies 49366ee200 chore(lint): clippy rules redundant_clone (#2554) 2026-04-15 13:54:07 +00:00
安正超 2a3d127a86 revert: restore PutObjReader and remove PUT zero-copy chunk path (#2351 phase 2) (#2532) 2026-04-14 16:35:03 +08:00
houseme ec8f059506 feat: add timing and metrics for query execution phases (#2419)
Signed-off-by: Shreyan Jain <shreyan11@duck.com>
Co-authored-by: Shreyan Jain <shreyan11@duck.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-07 20:32:32 +08:00
weisd 0b99e02891 feat(ecstore): add stale multipart upload cleanup (#2416) 2026-04-07 18:57:07 +08:00
houseme 32bf8f5bf3 feat(storage): add direct chunk GET fast path (#2351)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-07 08:33:46 +08:00
weisd 15995aae14 feat(admin): complete site replication support (#2346) 2026-03-31 13:01:38 +08:00
houseme 16db18216d fix: populate tagging notification principalId and object metadata (#2342) 2026-03-30 14:32:39 +08:00
weisd d637c4d342 fix(object-lock): recover remaining s3 tests (#2294) 2026-03-26 12:11:34 +08:00
安正超 a236b0d01d feat(ecstore): implement decommission and rebalance (#2281)
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-26 11:44:02 +08:00
weisd 41dcebda44 fix(tier): sweep transitioned copies from delete handlers (#2287) 2026-03-25 16:06:36 +08:00
weisd 28f57b228c feat(s3): advance parity coverage (#2278) 2026-03-24 17:29:33 +08:00
cxymds dad9a7d708 fix(ecstore): honor lifecycle tag filters (#2264)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-03-24 14:25:43 +08:00
cxymds 236142a682 fix(ecstore): repair lifecycle transition and restore flows (#2240)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: weisd <im@weisd.in>
2026-03-23 12:29:13 +08:00
马登山 f11c307aec fix(admin): avoid tier stats panic on missing tier (#2238)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-20 20:57:33 +08:00
weisd 35e1f28f23 fix: support legacy bucket metadata decoding (#2227) 2026-03-19 12:43:53 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
马登山 237c933f38 test(lifecycle): add prefix regression and zero-day checks (#2201) 2026-03-18 13:20:00 +08:00
安正超 c20b3c7f19 fix(ecstore): handle EODM rules without due date (#2198) 2026-03-18 12:54:41 +08:00
马登山 c66c6d97ec fix(lifecycle): respect Filter.Prefix and safe delete marker expiry (#2185)
Signed-off-by: likewu <likewu@126.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: likewu <likewu@126.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-17 18:45:38 +08:00
LeonWang0735 7f1cdaedad feat(replication): add bandwidth-aware reporting for bucket replication metrics (#2141) 2026-03-15 09:03:10 +08:00
houseme f83bf95b04 feat(ecstore): Skip rustls provider install if already present (#2145)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-03-12 18:02:19 +08:00
GatewayJ 16946c5a54 fix: allow root to bypass bucket policy deny for policy management APIs (#2102)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-03-09 20:36:29 +08:00