f53eb4ad44
fix: reject invalid multipart part numbers ( #3091 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-27 08:09:59 +00:00
ceffe21f75
feat(tls): add inspect command for TLS layouts ( #3092 )
2026-05-27 07:23:58 +00:00
909f0d57fb
feat: improve degraded readiness reporting and shutdown handling ( #3089 )
...
* fix(server): unify runtime readiness and shutdown flow
* refactor(server): decouple readiness shared types
* refactor(server): keep readiness types crate-private
* refactor(server): await protocol shutdown handles
* fix(server): bypass cached startup readiness
* feat(health): expose degraded readiness reasons
* fix(health): preserve raw IAM readiness in degraded reports
2026-05-27 03:25:28 +00:00
658b8dea66
fix: unify runtime readiness publication and graceful shutdown flow ( #3087 )
...
* fix(server): unify runtime readiness and shutdown flow
* refactor(server): decouple readiness shared types
* refactor(server): keep readiness types crate-private
* refactor(server): await protocol shutdown handles
* fix(server): bypass cached startup readiness
2026-05-26 18:49:42 +00:00
0875f09a39
fix: rebuild wiped disks during admin heal ( #3084 )
...
* fix: rebuild wiped disks during admin heal
* Preserve forceStart heal admission semantics
* Address heal regression test review comments
* fix(heal): address admin heal review comments
* fix(heal): fix force-start dedup and test polling
* fix(heal): address unresolved review comments
* fix(heal): simplify dedup key for clippy
---------
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-26 21:54:29 +08:00
62d1f80dbf
fix(data-usage): refresh admin usage after object changes ( #3081 )
...
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-05-25 14:24:00 +00:00
f9475e10dc
fix(replication): preserve multipart pending state ( #3058 )
...
* Preserve multipart replication recovery state
Multipart uploads previously only scheduled replication after completion, leaving no persisted pending state for scanner recovery if the initial async work was lost. Persist the same pending replication metadata during multipart initialization and let completion evaluate the object metadata that was actually stored.
The scanner heal path also treated ordinary pending objects as delete-replication candidates. Restrict that path to delete markers and version purge state so pending objects remain eligible for object replication heal.
Constraint: Bucket replication recovery depends on persisted object metadata after the async queue is unavailable.
Rejected: Rely only on immediate completion-time scheduling | it cannot recover after process restart or worker loss.
Confidence: high
Scope-risk: moderate
Directive: Keep multipart upload initialization aligned with single PUT replication metadata semantics.
Tested: cargo fmt --all --check
Tested: cargo clippy --workspace --all-features --all-targets -- -D warnings
Tested: LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 make pre-commit
Tested: Runtime replication outage check confirmed multipart xl.meta stores PENDING status and timestamp.
* fix(replication): preserve version purge scanner state
Role-derived replication configs need target-scoped status strings before scanner heal can build per-target purge status. The duplicated replication-status assignment left version purge status unset, so scanner recovery could lose the target-level purge state.
Constraint: Scanner heal derives per-target purge decisions from version_purge_status_internal.
Rejected: Leave the duplicate as a harmless cleanup | it changes recovery behavior for role-only configs with version purge state.
Confidence: high
Scope-risk: narrow
Directive: Keep role-derived replication and version purge internal status mapping symmetric.
Tested: cargo fmt --all --check
Tested: cargo test -p rustfs-ecstore heal -- --nocapture
---------
Co-authored-by: wly <wlywly0735@126.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-24 14:24:00 +00:00
b0646be756
feat(s3select): improve SelectObjectContent streaming ( #3072 )
...
* feat(s3select): improve SelectObjectContent streaming
* fix(s3select): reject empty select expressions
* fix(s3select): address streaming review feedback
---------
Co-authored-by: loverustfs <hello@rustfs.com >
2026-05-24 14:23:47 +00:00
d74e6eb042
refactor(tls): centralize runtime foundation ( #3065 )
...
* refactor(targets): move notify net helpers from utils
* refactor(tls): centralize runtime foundation
* refactor(targets): move notify net helpers from utils
* refactor(tls): centralize runtime foundation
* feat(tls-runtime): add TLS debug state and admin handler
* refactor(tls-runtime): unify TLS debug consumer status view
* fix(tls): address PR3065 review feedback
* refactor(tls): align debug status payload types
* refactor(targets): harden TLS hot reload paths
* fix(targets): resolve review-4348251652 findings
* fix(targets): finalize tls runtime review follow-ups
* fix(targets): harden tls reload and review follow-ups
* fix(targets): align tls reload handling across targets
* fix(targets): finalize tls reload state and metrics updates
* chore(deps): trim unused TLS deps
* style(targets): normalize TLS reload formatting
* refactor(targets): introduce tls runtime adapter path
* chore: update workspace manifests for tls refactor
* fix(tls): stabilize material reload and audit workflow
* fix(targets): refresh tls fingerprint flow across sinks
* fix(tls): align runtime coordinator and http reader updates
* fix(sftp): simplify protocol error mapping
* fix(tls): harmonize material loading behavior
* fix(server): finalize tls material wiring in startup flow
* fix(protos): tighten tls generation cache and deps
2026-05-24 06:41:15 +00:00
20a4db7c9a
fix(tls): resolve RUSTFS_TLS_PATH startup regression ( #3059 )
...
* fix(tls): resolve RUSTFS_TLS_PATH startup regression
* fix(tls): adopt PR review follow-ups
2026-05-22 09:09:31 +00:00
6264be437c
fix(storage): add scoped timeout policy and startup fs guardrail ( #3056 )
...
* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends
* style: fix cargo fmt formatting in disk_store.rs
* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends
Extend the TimeoutHealthAction introduced in #2996 to read_metadata,
list_dir, and disk_info operations when RUSTFS_NETWORK_MOUNT_MODE=true.
Also raises all drive operation timeouts to 60s (explicit per-operation
overrides still take precedence).
Closes #2790
* feat(startup): add unsupported filesystem policy guardrail
* chore(deps): refresh lockfile and dependency pins
* feat(ecstore): add scoped timeout health-action policy
* docs(config): document drive timeout health-action policy
* refactor(ecstore): cache timeout health policy per disk wrapper
* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends (#2838 )
* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends
* style: fix cargo fmt formatting in disk_store.rs
* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends
Extend the TimeoutHealthAction introduced in #2996 to read_metadata,
list_dir, and disk_info operations when RUSTFS_NETWORK_MOUNT_MODE=true.
Also raises all drive operation timeouts to 60s (explicit per-operation
overrides still take precedence).
Closes #2790
* fix(utils): map verified Linux filesystem magic values (#3051 )
* fix(utils): cover sha256 checksum validation (#3052 )
* fix(utils): cover sha256 checksum validation
* docs: clarify sha256 checksum validation
---------
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
* refactor(config): replace network mount mode with timeout profile preset
* fix(review): align fallback defaults and extend fs-type detection
* fix(review): cache timeout profile and restore probe timeout semantics
* refactor(ecstore): cache timeout health policy lookup
* perf(ecstore): cache active probe timeout per monitor task
---------
Co-authored-by: mistik <mistiklord4@gmail.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
2026-05-22 06:37:30 +00:00
69345fe059
fix(scanner): preserve background heal compatibility ( #3041 )
2026-05-22 14:38:35 +08:00
ed2078e025
fix(ecstore): allow expired delete markers on locked buckets ( #3048 )
...
* fix: allow expired delete markers on locked buckets
* fix: reject zero-day del marker expiration
2026-05-22 01:39:53 +00:00
0985f0b37b
feat(internode): label transport operation metrics ( #3045 )
...
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-21 15:20:39 +00:00
dcbffb084f
chore(deps): refresh workspace deps and linux fs_type gating ( #3030 )
...
* chore(deps): refresh workspace deps and linux fs_type gating
- refresh workspace dependency pins and lockfile updates
- remove now-unused crate dependency entries in multiple Cargo.toml files
- enable profiling export defaults in config and scripts/run.sh
- gate os::fs_type module/function/tests to Linux to avoid non-Linux dead_code warnings
* fix(utils): simplify fs_type linux gating
- keep fs_type module-level linux cfg in os::mod
- remove redundant linux cfg on get_fs_type and test module
* chore(deps): bump s3s git revision
- update workspace s3s dependency to rev 507e1312b211c3ddc214b03875d6fabd15d22ed5
- refresh Cargo.lock source entry for s3s
* chore(dev): allow mysql_async git source and env overrides
- allow mysql_async git source in deny.toml allow-git list
- make scripts/run.sh core env vars overrideable via existing shell env
* fix(utils): import get_fs_type in fs_type tests
- add explicit super::get_fs_type import in fs_type test module
- fix Linux E0425 unresolved function errors in unit tests
* chore(dev): tune run script observability defaults
- make profiling export env overrideable in scripts/run.sh
- set RUSTFS_OBS_SAMPLE_RATIO default from 2.0 to 1.0
- update allow-git review window comments in deny.toml
* test(obs): stabilize profiling env alias tests
2026-05-20 15:03:07 +00:00
375482a4b6
refactor: converge storage io hot paths ( #3029 )
...
* refactor(issue-633): clarify layered io control policies
* refactor(issue-633): consolidate timeout and deadlock layers
* refactor(issue-633): align storage backpressure metadata
* refactor(issue-633): unify storage backpressure transitions
* refactor(issue-633): simplify watermark transition API
* test(issue-633): add storage backpressure transition test
* refactor(issue-633): align storage pipe meta shape
* refactor(issue-633): enrich storage monitor metadata
* refactor(issue-633): finalize storage backpressure convergence
* refactor(issue-633): complete scheduler layer convergence
* refactor(issue-633): reduce concurrency facade config duplication
* refactor(issue-633): migrate storage callsites to final policy names
* chore(issue-633): apply final pre-commit normalization
* refactor(issue-633): unify timeout wrapper dynamic size path
* refactor(issue-633): make concurrency policies copyable
* refactor(issue-633): converge storage io hot paths
* fix(issue-633): honor storage timeout min bound
* fix(storage): avoid timeout calc panic on huge sizes
* refactor(storage): consolidate timeout checks and test attrs
* fix(storage): harden io scheduler core config mapping
* refactor(storage): eliminate patch-on-patch patterns and dead code
- Remove trivial accessor methods on ConcurrencyConfig that just return pub fields
- Remove dead BackpressureEvent/BackpressureEventType types from concurrency crate
- Fix io_schedule test using wrong constructor (from_core_config -> from_scheduler_config)
- Update manager.rs to use config fields directly instead of removed accessors
* fix: adopt review feedback for config guards
* test: remove needless struct update defaults
* fix: harden timeout policy and preserve api alias
2026-05-20 12:00:23 +00:00
73bde843d6
refactor(s3): consolidate semantic boundaries and remove s3-common ( #3012 )
...
* refactor(common): introduce rustfs-data-usage core crate
* refactor(concurrency): migrate workers crate into concurrency
* refactor(crypto): migrate appauth token APIs into crypto
* fix docs urls
* remove unused crate
* refactor(data-usage): switch consumers to rustfs-data-usage
* chore(fmt): apply cargo fmt and lockfile sync
* refactor(common): remove data_usage compatibility re-export
* refactor(capacity): move capacity_scope to object-capacity
* refactor(io-metrics): relocate internode metrics from common
* refactor(common): decouple scanner report from madmin
* chore(fmt): normalize import ordering after pre-commit
* refactor(s3): split s3 types and ops crates
* refactor(s3): centralize event version and safe parsing
* refactor(s3): add op-event compatibility guardrails
* refactor(s3): add runtime op-event mismatch observability
* refactor(s3): extract delete event mapping helper
* refactor(s3): extract put event mapping helper
* refactor(s3): consolidate remaining event semantic helpers
* refactor(s3): add op-event coverage checks and observability alerts
* refactor(s3-ops): consolidate op-event semantic mapping
* refactor(scanner): remove last_minute wrapper module
* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
d6fc70eb12
feat(obs): expose key S3 usecase spans at info ( #3013 )
2026-05-19 11:00:51 +00:00
25c6bdf490
perf(filemeta): phase-1~3 rename_data metadata optimization ( #3011 )
...
* chore(perf): harden amd64 profiling benchmark flow
* fix(profiling): isolate bench buckets and map protobuf conflict
* perf: avoid blocking owned local writes
* style: format profile admin handler
* docs: clarify observability trace validation
* perf: reduce mkdir overhead on local writes
* perf: add rename_data meta microbenchmark
* perf(filemeta): fast-path data_dir decode in version meta
* perf(filemeta): collapse data-dir lookup into one scan
* perf(filemeta): reduce scan allocs and refresh meta bench
* perf(ecstore): skip mkdir path on read-only open
* perf(filemeta): single-pass unshared data-dir scan
* perf(filemeta): add two-key inline remove fast path
* perf(filemeta): compare remove-two keys by bytes first
* bench(ecstore): add remove_two-only micro benchmark
* bench(ecstore): stabilize rename_data meta benchmark timing
* bench(ecstore): align rename_data path with remove_two
* perf(filemeta): avoid uuid string alloc in remove_two
* perf(filemeta): add fast-path for empty inline data
* perf(filemeta): streamline add_version match branch
* perf(filemeta): fast-return remove_key on miss
* perf(filemeta): speed up add_version insertion lookup
* style(ecstore): normalize formatting in perf-tuning files
* refactor(filemeta): unify inline data removal paths
2026-05-19 10:20:24 +00:00
f695870626
feat(internode): add transport observability ( #3007 )
...
* docs: add internode data transport RFC
* feat: add internode operation metrics
* fix feedback
* fix(ci): fallback protoc token to github.token
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-05-19 07:16:05 +00:00
e65aebba53
fix(ecstore): guard transition worker max ( #3003 )
...
* fix(ecstore): guard transition worker max
* test: clarify env mutation safety
* test(ecstore): document env mutation concurrency contract
2026-05-19 02:25:51 +00:00
bd1e57293f
fix: harden lifecycle transition compensation and regression coverage ( #2995 )
...
* fix(ecstore): honor transition worker configuration
* fix(ecstore): add transition queue backpressure metrics
* fix(ecstore): schedule transition compensation on enqueue pressure
* fix(ecstore): log transition compensation scheduling
* test(rustfs): add transition compensation fault-injection coverage
* test(rustfs): cover delete after transition compensation
* test(scanner): cover cleanup after transition compensation
* test(rustfs): extend compensation transition coverage
* test(scanner): cover backfill idempotency after compensation
* test(scanner): cover noncurrent expiry after compensation
* test(rustfs): cover versioned delete after compensation
* test(rustfs): cover delete marker lifecycle after compensation
* test(scanner): extend versioned lifecycle compensation coverage
* test(scanner): model versioned delete after compensation
* test(scanner): clarify modeled versioned delete helper
* refactor(ecstore): optimize transition enqueue hot path
* refactor(ecstore): centralize transition runtime constants
* style(ecstore): apply rustfmt for transition timeout helper
* fix(ilm): align queue-full metric semantics
* refactor(ecstore): unify immediate enqueue failure handling
* refactor(ecstore): reuse transition worker env constant
* ci(actions): update setup action inputs
2026-05-18 12:50:43 +00:00
17ae9f34c7
fix(notify): accept case-insensitive filter rule names ( #2990 )
2026-05-18 03:41:48 +00:00
6e12289339
fix(runtime): finalize issue 2941 profiling cleanup ( #2983 )
...
* perf(runtime): narrow profiling support and upgrade starshard
* style(notify): normalize starshard imports
* perf(ecstore): reduce list_path_raw coordination overhead
* docs(scripts): add issue 2941 perf capture workflow
* fix(runtime): finalize issue 2941 profiling cleanup
* build(deps): bump quick-xml to 0.40.0
* chore(scripts): untrack local perf capture guide
* fix(scripts): honor label in perf capture output
2026-05-16 11:09:04 +00:00
9dcf8cb7ea
fix: stabilize rebalance start and listing ( #2961 )
...
* fix: unblock empty chunked admin rebalance start
Some admin clients send empty POST requests with chunked framing and no explicit content length. The admin rebalance route carries no request body, so normalize that known route before downstream validation sees a missing length.
Constraint: Keep normalization scoped to known empty-body admin routes
Rejected: Relax transfer-encoding handling for S3 routes | broader protocol risk
Confidence: high
Scope-risk: narrow
Tested: cargo test -p rustfs --lib server::layer::tests::
Tested: cargo fmt --all --check
Tested: git diff --check
* fix: keep rebalance listing from stalling
Large object migration was awaited inside the listing callback, so metacache readers could stop being drained while drive walk operations were still writing listing data. Move entry migration into bounded background tasks and wait for them after each set listing completes.
Constraint: Preserve existing rebalance cancellation and first-error propagation
Rejected: Only increase drive walk timeouts | masks callback backpressure and still fails for slower migrations
Confidence: medium
Scope-risk: moderate
Tested: cargo test -p rustfs-ecstore rebalance_unit_tests
Tested: cargo test -p rustfs-ecstore cache_value::metacache_set::tests
Tested: cargo test -p rustfs --lib server::layer::tests::
Tested: cargo clippy -p rustfs-ecstore --lib --all-features -- -D warnings
Tested: cargo clippy -p rustfs-ecstore --tests --all-features -- -D warnings
Tested: cargo fmt --all --check
Tested: git diff --check
---------
Co-authored-by: loverustfs <hello@rustfs.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: cxymds <Cxymds@qq.com >
2026-05-16 05:17:01 +00:00
6898e720dd
fix(security): harden proxy auth and default credentials ( #2981 )
...
* fix(security): harden proxy auth and default credentials
* fix(security): address proxy and credential feedback
2026-05-16 04:01:50 +00:00
738fb86611
fix(admin): surface access key policy errors ( #2970 )
...
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: loverustfs <hello@rustfs.com >
2026-05-15 08:26:38 +00:00
5cda460451
fix(sftp): preserve OPEN-time client attrs as object metadata ( #2913 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-15 00:32:06 +00:00
fc8322ed64
bucket policy notify & pba ( #2968 )
2026-05-14 14:31:40 +00:00
81754d80b3
refactor(targets): unify queue/connectivity handling and coverage ( #2953 )
...
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: marshawcoco <marshawcoco@gmail.com >
2026-05-14 04:31:23 +00:00
c1bcee327c
fix(notify): keep live listen events active when disabled ( #2952 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-13 17:10:14 +00:00
4ea805cbc0
fix: preserve pagination when max keys exceed limit ( #2943 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-13 12:48:38 +00:00
c5125b20bd
fix(storage): keep storage info RPC map encoded ( #2942 )
...
Co-authored-by: cxymds <Cxymds@qq.com >
2026-05-13 06:19:43 +00:00
d72b8cb88e
fix(server): fail fast when configured TLS parsing fails ( #2939 )
2026-05-13 04:16:39 +00:00
8ef2c0a5e1
fix(storage): sync transition tier config across peers ( #2918 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-11 15:23:29 +00:00
aba65a448c
fix(protocols): encode storage client request URIs ( #2911 )
2026-05-11 10:47:30 +00:00
bafff6d207
feat(targets): add check_mysql_server_available probe function ( #2884 )
...
Signed-off-by: JaySon-Huang <tshent@qq.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-10 11:53:45 +00:00
96b293bf8a
feat(sftp): add SFTPv3 protocol support ( #2875 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-10 03:48:42 +00:00
09c2d15057
fix(sse): Temporarily refactored the SSE design for ECStore ( #2813 )
...
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: cxymds <Cxymds@qq.com >
2026-05-09 14:06:35 +00:00
6275918d92
fix: empty-body requests without content length ( #2888 )
...
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: OmX <omx@oh-my-codex.dev >
2026-05-09 14:05:53 +00:00
5879d0b59d
fix(server): handle public health before s3 host parsing ( #2866 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-09 13:15:37 +00:00
81ad48dac2
feat(targets): add AMQP support for notify and audit ( #2879 )
...
Co-authored-by: Hyesook Yun <74169420+suk13574@users.noreply.github.com >
2026-05-09 01:56:26 +00:00
61bd5698bb
test(admin): cover pools list response serialization ( #2862 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-08 07:32:49 +00:00
fe978488a0
test(admin): cover pool used-size saturation ( #2863 )
2026-05-08 06:06:54 +00:00
9d85b7d23a
feat: enrich admin pools list response ( #2853 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-07 13:53:25 +00:00
5431b9273d
feat(targets): complete redis mysql postgres target wiring ( #2842 )
...
Signed-off-by: jaehanbyun <awbrg789@naver.com >
Signed-off-by: houseme <housemecn@gmail.com >
Signed-off-by: Gunther Xing <jiengup@gmail.com >
Signed-off-by: JaySon-Huang <tshent@qq.com >
Co-authored-by: jaehanbyun <awbrg789@naver.com >
Co-authored-by: Gunther Xing <jiengup@gmail.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
Co-authored-by: cxymds <Cxymds@qq.com >
Co-authored-by: JaySon <tshent@qq.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
2026-05-07 10:00:59 +00:00
b159d656cc
test(admin): cover POST content length compat layer ( #2844 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-07 09:51:09 +00:00
f1cd7c1345
feat(rustfs): add ftps/webdav defaults to info output ( #2845 )
2026-05-07 04:39:56 +00:00
06097a3c33
fix(admin): normalize empty admin POST content length ( #2843 )
2026-05-07 02:52:21 +00:00
3130670157
test(object-lock): cover default retention delete marker ( #2836 )
2026-05-06 21:14:00 +00:00
安正超