46d7f9e1f2
feat(get): harden codec streaming rollout ( #3981 )
...
* feat(get): consolidate GET performance optimization
Consolidated implementation of all GET performance optimizations into
a single, well-organized commit replacing the previous patch-on-patch
approach.
## Changes
### Configuration (set_disk/mod.rs)
- Consolidated all GET optimization flags into a single organized section
- Enabled by default: codec streaming, metadata early-stop, page cache reclaim
- Added codec streaming multipart flag (default: disabled)
- Added version-aware early-stop flag (default: disabled)
- Added adaptive duplex buffer sizing based on object size
- All flags use OnceLock caching with rollout percentage support
### Metadata Early-Stop (set_disk/read.rs)
- Delete marker early-stop when quorum agrees
- Version-aware early-stop for versioned GET requests
- MetadataQuorumAccumulator enhanced with:
- delete_marker_votes tracking
- requested_version_id and matching_version_votes tracking
- version_early_stop_decision() method
- 6 new tests for version early-stop scenarios
### Codec Streaming (erasure/coding/decode_reader.rs)
- DualInFlight (2-stripe lookahead) enabled by default
### Decode Pipeline (erasure/coding/decode.rs)
- Stripe prefetch count configuration
- Bitrot-decode overlap configuration
### Disk Layer (disk/local.rs)
- O_DIRECT read configuration constants (preparation)
### Metrics (io-metrics/lib.rs)
- BytesPool acquisition/return metrics
- Metadata phase duration with early-stop label
- Total duration with reader_path label
### Diagnostics (diagnostics/)
- Early-stop reason constants
- Pool tier/outcome label constants
### Observability (.docker/observability/)
- 3 Grafana dashboards for GET optimization monitoring
- Prometheus alert rules (6 alerts: 3 critical, 3 warning)
- Updated README.md and README_ZH.md with usage docs
### Config (config/src/constants/runtime.rs)
- Page cache reclaim read enabled by default
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| RUSTFS_GET_CODEC_STREAMING_ENABLE | true | Codec streaming base flag |
| RUSTFS_GET_CODEC_STREAMING_ROLLOUT_PCT | 100 | Codec streaming rollout % |
| RUSTFS_GET_CODEC_STREAMING_MULTIPART_ENABLE | false | Multipart codec streaming |
| RUSTFS_GET_METADATA_EARLY_STOP_ENABLE | true | Early-stop base flag |
| RUSTFS_GET_METADATA_EARLY_STOP_ROLLOUT_PCT | 100 | Early-stop rollout % |
| RUSTFS_GET_METADATA_VERSION_EARLY_STOP_ENABLE | false | Version-aware early-stop |
| RUSTFS_OBJECT_FILE_CACHE_RECLAIM_READ_ENABLE | true | Page cache reclaim |
| RUSTFS_OBJECT_DIRECT_IO_READ_ENABLE | false | O_DIRECT (preparation) |
| RUSTFS_GET_DECODE_STRIPE_PREFETCH_COUNT | 1 | Stripe prefetch |
| RUSTFS_GET_BITROT_DECODE_OVERLAP_ENABLE | false | Bitrot-decode overlap |
| RUSTFS_GET_CODEC_STREAMING_MAX_INFLIGHT | 2 | DualInFlight stripes |
## Rollback
All optimizations can be disabled via environment variables:
RUSTFS_GET_CODEC_STREAMING_ENABLE=false
RUSTFS_GET_METADATA_EARLY_STOP_ENABLE=false
RUSTFS_OBJECT_FILE_CACHE_RECLAIM_READ_ENABLE=false
Co-Authored-By: heihutu <heihutu@gmail.com >
* test(get): add stress test scripts for GET optimization validation
- quick-validate-get-optimization.sh: Quick 5-minute validation
- stress-test-get-optimization.sh: Full 30+ minute stress test
- README-stress-test.md: Usage documentation
Co-Authored-By: heihutu <heihutu@gmail.com >
* test(ecstore): align file cache reclaim defaults
* chore(deps): update redis and erasure codec
* test(ecstore): align decode fill policy default
* fix(get): wire codec streaming rollout gate
* perf(get): skip metrics-off codec timers
* test(get): capture codec streaming diagnostics
* test(get): add multipart fallback probe
* test(get): add encrypted fallback probe
* test(get): add compressed fallback probe
* test(get): add degraded read fallback probe
* test(get): cover remote fallback probe
* test(get): report warp request p99
* test(get): capture OTLP metric deltas
* perf(get): align codec streaming inflight default
* perf(get): reuse codec reader output buffers
* test(get): count codec reader fill starts
* perf(get): reuse codec reader fill worker
* perf(get): lazy init rustfs codec reconstruct
* test(get): cover rustfs codec source faults
* docs(get): record rustfs codec fallback scope
* feat(get): add multipart codec reader opt-in
* test(get): add multipart codec smoke option
* test(get): cover multipart codec degraded fallback
* perf(get): bound multipart codec eager setup
* test(get): satisfy codec hardening PR gate
---------
Co-authored-by: heihutu <heihutu@gmail.com >
2026-06-28 11:20:21 +08:00
27468ebfa9
feat(get): consolidate GET performance optimization ( #3972 )
...
* feat(get): consolidate GET performance optimization
Consolidated implementation of all GET performance optimizations into
a single, well-organized commit replacing the previous patch-on-patch
approach.
## Changes
### Configuration (set_disk/mod.rs)
- Consolidated all GET optimization flags into a single organized section
- Enabled by default: codec streaming, metadata early-stop, page cache reclaim
- Added codec streaming multipart flag (default: disabled)
- Added version-aware early-stop flag (default: disabled)
- Added adaptive duplex buffer sizing based on object size
- All flags use OnceLock caching with rollout percentage support
### Metadata Early-Stop (set_disk/read.rs)
- Delete marker early-stop when quorum agrees
- Version-aware early-stop for versioned GET requests
- MetadataQuorumAccumulator enhanced with:
- delete_marker_votes tracking
- requested_version_id and matching_version_votes tracking
- version_early_stop_decision() method
- 6 new tests for version early-stop scenarios
### Codec Streaming (erasure/coding/decode_reader.rs)
- DualInFlight (2-stripe lookahead) enabled by default
### Decode Pipeline (erasure/coding/decode.rs)
- Stripe prefetch count configuration
- Bitrot-decode overlap configuration
### Disk Layer (disk/local.rs)
- O_DIRECT read configuration constants (preparation)
### Metrics (io-metrics/lib.rs)
- BytesPool acquisition/return metrics
- Metadata phase duration with early-stop label
- Total duration with reader_path label
### Diagnostics (diagnostics/)
- Early-stop reason constants
- Pool tier/outcome label constants
### Observability (.docker/observability/)
- 3 Grafana dashboards for GET optimization monitoring
- Prometheus alert rules (6 alerts: 3 critical, 3 warning)
- Updated README.md and README_ZH.md with usage docs
### Config (config/src/constants/runtime.rs)
- Page cache reclaim read enabled by default
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| RUSTFS_GET_CODEC_STREAMING_ENABLE | true | Codec streaming base flag |
| RUSTFS_GET_CODEC_STREAMING_ROLLOUT_PCT | 100 | Codec streaming rollout % |
| RUSTFS_GET_CODEC_STREAMING_MULTIPART_ENABLE | false | Multipart codec streaming |
| RUSTFS_GET_METADATA_EARLY_STOP_ENABLE | true | Early-stop base flag |
| RUSTFS_GET_METADATA_EARLY_STOP_ROLLOUT_PCT | 100 | Early-stop rollout % |
| RUSTFS_GET_METADATA_VERSION_EARLY_STOP_ENABLE | false | Version-aware early-stop |
| RUSTFS_OBJECT_FILE_CACHE_RECLAIM_READ_ENABLE | true | Page cache reclaim |
| RUSTFS_OBJECT_DIRECT_IO_READ_ENABLE | false | O_DIRECT (preparation) |
| RUSTFS_GET_DECODE_STRIPE_PREFETCH_COUNT | 1 | Stripe prefetch |
| RUSTFS_GET_BITROT_DECODE_OVERLAP_ENABLE | false | Bitrot-decode overlap |
| RUSTFS_GET_CODEC_STREAMING_MAX_INFLIGHT | 2 | DualInFlight stripes |
## Rollback
All optimizations can be disabled via environment variables:
RUSTFS_GET_CODEC_STREAMING_ENABLE=false
RUSTFS_GET_METADATA_EARLY_STOP_ENABLE=false
RUSTFS_OBJECT_FILE_CACHE_RECLAIM_READ_ENABLE=false
Co-Authored-By: heihutu <heihutu@gmail.com >
* test(get): add stress test scripts for GET optimization validation
- quick-validate-get-optimization.sh: Quick 5-minute validation
- stress-test-get-optimization.sh: Full 30+ minute stress test
- README-stress-test.md: Usage documentation
Co-Authored-By: heihutu <heihutu@gmail.com >
* test(ecstore): align file cache reclaim defaults
* chore(deps): update redis and erasure codec
* test(ecstore): align decode fill policy default
* test(ecstore): align metadata early-stop default
* fix(ecstore): keep metadata early stop opt-in
---------
Co-authored-by: heihutu <heihutu@gmail.com >
2026-06-28 07:14:07 +08:00
bf03ff2869
feat(get): add limited opt-in rollout gates ( #3963 )
...
* feat(bench): harden cooled get ab harness
* feat(get): add limited opt-in rollout gates
* fix(get): tighten rollout gate fallbacks
2026-06-27 23:13:02 +08:00
20f56af09c
feat(get): tune output response handoff ( #3956 )
2026-06-27 21:46:38 +08:00
de86025f2c
feat(get): overlap read verify decode ( #3945 )
2026-06-27 15:14:44 +08:00
8994a8ff00
feat(get): add local-first shard reads ( #3924 )
...
* feat(get): observe shard locality cost
* feat(get): add local-first shard reads
----
Co-Authored-By: Heihutu <heihutu@gmail.com >
2026-06-27 01:08:36 +08:00
c0ddc14bb8
feat(get): observe shard locality cost ( #3922 )
...
* feat(get): observe shard locality cost
* fix(storage): avoid request counter underflow panic
* fix(storage): import put guard in concurrency tests
2026-06-26 23:50:37 +08:00
2c04807f05
feat(get): add guarded metadata early stop ( #3916 )
...
* feat(get): add v2 metrics compatibility harness
* feat(get): observe metadata fanout quorum (#3915 )
* feat(get): observe metadata fanout quorum
* fix(app): use storage ECStore type in app boundary
* feat(get): add guarded metadata early stop
2026-06-26 21:56:23 +08:00
5a78a9c416
feat(get): add v2 metrics compatibility harness ( #3913 )
...
* feat(get): add v2 metrics compatibility harness
* feat(get): observe metadata fanout quorum (#3915 )
* feat(get): observe metadata fanout quorum
* fix(app): use storage ECStore type in app boundary
----
Co-Authored-By: heihutu <heihutu@gmail.com >
2026-06-26 20:09:09 +08:00
0321e4c6ca
perf(get): reduce metrics and streaming handoff overhead ( #3907 )
2026-06-26 18:12:51 +08:00
7bf411f465
obs(get): add GET read pipeline metrics ( #3868 )
2026-06-26 04:02:02 +08:00
96b1f5c373
feat(storage): optimize gt1g get read path ( #3860 )
...
* feat(storage): tune gt1g get read path
* docs(ops): add gt1g get benchmark guide
* feat(scripts): add gt1g get fallback runner
* test(storage): trim gt1g get helper warnings
* test(storage): reduce gt1g get helper type complexity
2026-06-25 20:12:08 +08:00
efebcb66be
feat(perf): add large PUT tuning and encode optimization ( #3816 )
2026-06-24 19:04:04 +08:00
0a00d8d500
perf(server): lighten internode data-plane stack ( #3735 )
...
* refactor(server): split internode dispatch scaffold
* test(server): cover internode dispatch prefix split
* refactor(server): name internode stack boundaries
* perf(server): skip internode request logging layer
* perf(server): skip internode trace layer
* perf(server): use lite internode request context
* feat(metrics): track internode rpc duration
* feat(ecstore): add put object stage summary logs
* test(metrics): update internode descriptor expectations
* fix(server): tighten internode path matching
* fix(pr): address review follow-up comments
* style(ecstore): simplify commit tail duration field
* refactor(ecstore): group put stage summary fields
* refactor(ecstore): inline put stage summary log
* fix(s3): return storage class for object attributes
* merge: sync latest main and resolve object attributes conflict
* fmt
* fix(server): remove duplicate rpc imports
* build(deps): bump memmap2 for RUSTSEC-2026-0186
* fix(s3select): align object_store with datafusion
* chore(deps): prune workspace dependencies
* perf(fuzz): optimize CI runtime with build/run split and matrix parallelization
Separate fuzz harness compilation from execution to eliminate redundant
builds across targets. Introduce matrix-based parallel execution for
PR smoke and nightly fuzz jobs.
Changes:
- Split CI workflow into `fuzz-build` (compile once) and matrix run jobs
(`pr-fuzz-smoke`, `nightly-fuzz-corpus`) that execute targets in parallel
- Add `BUILD_ONLY` mode to run_ci_targets.sh / run_nightly_targets.sh
- Add run_single_target.sh for matrix jobs (no build phase)
- Optimize `local_metadata` fuzz target: reduce prefix iterations from
8-10 (4 functions each) to 5 critical prefixes (parser-only), cutting
per-iteration cost by ~3-5x
- Move archive path validation (`validate_extract_relative_path`,
`normalize_extract_entry_key`) from `rustfs` to `rustfs-utils::path`,
eliminating `rustfs` binary crate dependency from fuzz harness
- Remove `rustfs` from fuzz/Cargo.toml (drops significant transitive deps)
- Add unit tests for archive path validation in rustfs-utils
- Update fuzz/README.md with new workflow and script documentation
Expected CI improvement: PR smoke wall-clock from ~120min (frequent
timeout) to ~40min; nightly from ~180min to ~60min.
* refactor(fuzz): consolidate scripts and fix prefix test alignment
Replace three duplicated shell scripts (run_ci_targets.sh,
run_nightly_targets.sh, run_single_target.sh) with a single
parameterized run.sh that supports BUILD_ONLY, SKIP_BUILD, and
MAX_TOTAL_TIME environment variables.
Fix local_metadata fuzz target prefix testing: replace always-true
'len > 0' guard with lengths aligned to xl.meta binary layout
(4/5/8/12 bytes for magic+version+header fields). Remove redundant
empty-slice test.
Hoist RUSTFLAGS to workflow top-level env to eliminate per-job
duplication. Update README with unified script documentation.
Net: -118 lines, zero functionality loss.
* perf(fuzz): optimize CI runtime with build/run split and matrix parallelization
Restructure fuzz CI workflow to eliminate redundant compilation and
run targets in parallel via matrix strategy.
Workflow changes:
- Split into fuzz-build (compile once) and matrix run jobs
- PR smoke: 3 targets parallel, 60s each, timeout 30min (was 120min)
- Nightly: 3 targets parallel, 300s each, timeout 60min (was 180min)
- Pass compiled harness via actions/artifact between jobs
- Hoist RUSTFLAGS to workflow top-level env
Script consolidation:
- Replace 3 duplicated scripts with single parameterized run.sh
- Supports BUILD_ONLY, SKIP_BUILD, MAX_TOTAL_TIME env vars
Target optimizations:
- Remove rustfs binary crate from fuzz dependencies (was pulling
979 transitive deps); move archive path validation to rustfs-utils
- Optimize local_metadata: reduce prefix iterations from 8-10x4
calls to 5 prefixes with parser-only (no decompress), aligned
with xl.meta binary layout (4/5/8/12 bytes)
- Add unit tests for archive path validation in rustfs-utils
- Update fuzz/README.md with unified script documentation
Expected: PR smoke wall-clock from ~120min (frequent timeout)
to ~40min; nightly from ~180min to ~60min.
* fix(rpc): resolve internode metrics via app context
* fmt
* ci: speed up fuzz smoke artifact restore
---------
Signed-off-by: houseme <housemecn@gmail.com >
2026-06-23 21:36:39 +08:00
8d24d9133b
perf(put): comprehensive PUT performance optimization ( #3514 )
...
* perf(put): add eager path metrics and isolation tooling
* fix(decommission): persist progress adaptively (#3497 )
Persist decommission progress after either the existing time interval or a migrated-item threshold, and flush progress baselines after bucket and terminal-state saves.
Also stabilize the OIDC discovery mock used by the pre-commit gate.
* refactor: move bucket operations contract (#3507 )
* fix(s3): handle multipart flexible checksums (#3508 )
* fix(io-core): avoid blocking on pooled buffer return
* perf(put): add slow inflight diagnostics
* perf(put): fix 16KiB regression with threshold and pool bypass
- Lower SMALL_EAGER_PUT_MAX_SIZE from 256KB to 8KB so objects >8KiB
use the streaming BufReader path (matches baseline behavior)
- Add POOL_BYPASE_MAX_SIZE (16KiB) to bypass BytesPool for very small
objects, avoiding Small-tier Mutex contention under high concurrency
- Add read_small_put_body_exact_direct() for direct Vec<u8> allocation
- Fix stale test assertions to match new 8KB threshold
Root cause analysis: the 16KiB regression was primarily caused by
instrumentation overhead in set_disk.rs (4x Instant::now() + metrics
per PUT), not BytesPool contention. Lowering the threshold eliminates
the eager-path overhead for 16KiB+ objects.
* perf(put): gate stage metrics behind observability flag
Add put_stage_metrics_enabled() AtomicBool switch in io-metrics crate.
When disabled (default), record_put_object_path() and
record_put_object_stage_duration() are no-ops, avoiding unnecessary
histogram/counter macro overhead in the PUT hot path.
The flag is set to true during startup when OTEL metric export is
enabled (rustfs_obs::observability_metric_enabled() == true).
This eliminates the per-request metrics overhead that contributed
to the 16KiB PUT regression when metrics collection is not active.
* perf(put): comprehensive optimization - restore eager path, cache env, remove UUID
Change 1: Restore SMALL_EAGER_PUT_MAX_SIZE from 8KB to 1MB
- The try_lock() fix (d13a189e3) eliminates the blocking that caused
service health timeouts under 512KiB c64 load
- Eager path with BytesPool is now safe for objects up to 1MB
- Recovers the eager path benefit for 32KiB-256KiB objects
Change 2: Adjust POOL_BYPASE_MAX_SIZE from 16KB to 4KB
- With eager path restored to 1MB, objects 4KB-1MB benefit from pool reuse
- Only ≤4KB objects bypass the pool (allocation cost negligible)
Change 3: Cache RUSTFS_ERASURE_ENCODE_MAX_INFLIGHT_BYTES via OnceLock
- Eliminates per-encode std::env::var() syscall
- Env var still works (read once at first use)
Change 4: Replace Uuid::new_v4() with Uuid::nil() in Erasure construction
- _id field is unused in hot paths (documented in code)
- Eliminates CSPRNG syscall per PUT request
Change 5: Add concurrency-aware buffer sizing to PUT path
- Reuses get_concurrency_aware_buffer_size() from GET path
- Reduces buffer size under high concurrency (0.4x at >8 concurrent)
- Lowers memory pressure for >1MB streaming PUTs
* chore: add pyroscope feature flag and clean up imports
- Add pyroscope feature flag forwarding to rustfs-obs
- Remove unused allow(non_upper_case_globals) in globals.rs
- Sort imports and fix Cargo.toml formatting consistency
* style: fix import ordering and code formatting
- Sort imports alphabetically in globals.rs, encode.rs
- Fix indentation in erasure_coding encode/erasure
- Clean up HashReader formatting in object_usecase.rs
* fix(test): use tokio::test for request_logging_layer tests
The tests call tokio::spawn via RequestContextLayer, which requires a
Tokio runtime. Changed from #[test] + futures::executor::block_on to
#[tokio::test] + .await, and replaced tracing::subscriber::with_default
with tracing::subscriber::set_default to support async.
* fix(bench): normalize no-space throughput/latency parsing in to_bps/to_ms
When a benchmark tool prints throughput without a separator (e.g. 123MiB/s),
awk '{print $2}' returns empty because the whole string is one field,
causing to_bps to return N/A and losing valid measurements in CSV output.
Insert a space between number and unit via sed before awk field splitting.
Same fix applied to to_ms for latency values like '50ms'.
Also add TODO comment on PUT path noting that get_concurrency_aware_buffer_size
reads ACTIVE_GET_REQUESTS instead of PUT concurrency (PR #3514 review).
Refs: PR #3514 review comments by chatgpt-codex-connector
* fix(metrics): correct POOL_BYPASS comments and separate PUT vs generic stage metrics
- Fix 3 comment-code mismatches: POOL_BYPASS_MAX_SIZE is 4KiB, not 16KiB
- Add generic record_stage_duration() with separate histogram
(rustfs_internal_stage_duration_ms) for non-PUT paths
- Replace record_put_object_stage_duration with record_stage_duration in
metacache_set, store_list_objects, and bucket_lifecycle_ops to avoid
polluting PUT-specific dashboards with listing/lifecycle timings
- Fix flaky test: serialize tests mutating PUT_STAGE_METRICS_ENABLED with
METRICS_FLAG_LOCK mutex and explicitly set desired state at test start
Refs: PR #3514 review comments by chatgpt-codex-connector
* style: apply cargo fmt to metacache_set.rs
---------
Co-authored-by: cxymds <cxymds@gmail.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
2026-06-17 21:19:11 +08:00
ed73952cb6
perf(ecstore): improve erasure write diagnostics and single-block performance ( #3280 )
...
* docs(object-capacity): add localized crate docs
* fix(ecstore): improve quorum and transport diagnostics
* perf(ecstore): add safe single-block write fast path
* refactor(ecstore): collapse layered small write paths
* chore(docs): keep issue 662 design note tracked
* fix(docs): restore issue 662 design note
* chore(docs): keep issue 662 design local only
* feat(obs): add internode reliability metrics and dashboard
* feat(obs): extend internode diagnostics and service logging
* fix(docs): use AGENTS guide filename
* perf(ecstore): reuse owned buffer in small encode
* fix(ecstore): tighten small write diagnostics
---------
Co-authored-by: cxymds <Cxymds@qq.com >
2026-06-08 09:45:56 +00:00
29fbdc2dbf
feat(ecstore): add object lock diagnostics ( #3178 )
...
* feat(ecstore): add object lock diagnostics
Add configurable namespace lock diagnostics for object operations so production contention can be traced by operation, owner, and key.
Wrap object read/write lock acquisition in diagnostic guards across get, head, put, delete, copy, and multipart flows, and log slow acquisition and long hold durations behind new RUSTFS_OBJECT_LOCK_DIAG_* settings.
Verification:
- make pre-commit
* feat(obs): expose object lock diagnostics metrics
Add Prometheus metrics and Grafana panels for object namespace lock diagnostics, covering slow acquire counts, slow hold counts, acquire duration, hold duration, and the diagnostics-enabled state.
Adopt PR review feedback by keeping diagnostic guards alive through the guarded operation so long-hold warnings and metrics are emitted, reusing shared env helpers, and reducing default-path overhead when diagnostics are disabled.
Verification:
- cargo check -p rustfs-ecstore -p rustfs-io-metrics
- cargo test -p rustfs-ecstore store::object -- --nocapture
- make pre-commit
* perf(obs): reduce object lock diag overhead
Avoid repeated environment parsing on hot object-lock paths by caching the diagnostics-enabled flag, and stop allocating label strings for object lock metrics by recording static labels directly.
Strengthen io-metrics tests by using a local recorder and asserting that the expected object lock diagnostic counters, gauges, and histograms are emitted.
Verification:
- cargo check -p rustfs-ecstore -p rustfs-io-metrics
- cargo test -p rustfs-io-metrics -- --nocapture
- make pre-commit
2026-06-03 02:10:27 +00:00
522605a055
test(internode): cover adapter metrics validation ( #3071 )
...
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-05-23 15:23:05 +00:00
0985f0b37b
feat(internode): label transport operation metrics ( #3045 )
...
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-21 15:20:39 +00:00
73bde843d6
refactor(s3): consolidate semantic boundaries and remove s3-common ( #3012 )
...
* refactor(common): introduce rustfs-data-usage core crate
* refactor(concurrency): migrate workers crate into concurrency
* refactor(crypto): migrate appauth token APIs into crypto
* fix docs urls
* remove unused crate
* refactor(data-usage): switch consumers to rustfs-data-usage
* chore(fmt): apply cargo fmt and lockfile sync
* refactor(common): remove data_usage compatibility re-export
* refactor(capacity): move capacity_scope to object-capacity
* refactor(io-metrics): relocate internode metrics from common
* refactor(common): decouple scanner report from madmin
* chore(fmt): normalize import ordering after pre-commit
* refactor(s3): split s3 types and ops crates
* refactor(s3): centralize event version and safe parsing
* refactor(s3): add op-event compatibility guardrails
* refactor(s3): add runtime op-event mismatch observability
* refactor(s3): extract delete event mapping helper
* refactor(s3): extract put event mapping helper
* refactor(s3): consolidate remaining event semantic helpers
* refactor(s3): add op-event coverage checks and observability alerts
* refactor(s3-ops): consolidate op-event semantic mapping
* refactor(scanner): remove last_minute wrapper module
* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
c90bfe2b23
fix(ecstore): harden runtime read-path quorum handling ( #2872 )
2026-05-08 09:56:39 +00:00
9c0141fbdf
docs(io-metrics): fix misleading metrics links ( #2849 )
2026-05-07 20:00:19 +08:00
50d03ef021
perf(memory): add reclaim signals and cache controls ( #2689 )
2026-04-26 16:42:35 +00:00
59f41eb86a
feat(obs): improve metrics coverage and dashboard performance ( #2682 )
2026-04-25 18:51:29 +00:00
116db4f5d9
refactor(metrics): unify process sampling and split network IO ( #2590 )
2026-04-18 15:30:44 +00:00
1cbf156559
refactor(obs): migrate metrics runtime/schema and tighten migration guards ( #2584 )
2026-04-18 07:51:15 +00:00
96fb06f48e
refactor(metrics): separate console stream from exports ( #2583 )
2026-04-17 18:21:08 +00:00
ffcf18f5f3
fix(cache): wire trusted-proxy cache and remove stale cache traces ( #2581 )
2026-04-17 16:57:21 +00:00
f77ccd5b23
chore: move mcp crate to standalone repository ( #2578 )
2026-04-17 12:42:26 +00:00
478720d2ee
Centralize lifecycle state updates and fix systemd running status ( #2567 )
2026-04-17 03:20:11 +00:00
49366ee200
chore(lint): clippy rules redundant_clone ( #2554 )
2026-04-15 13:54:07 +00:00
642d83f0e4
revert: remove #2351 chunk I/O and object-io crate (phase 7) ( #2543 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-04-15 10:54:41 +08:00
6d476ae9a1
revert: remove BlockReadable trait and chunk-based I/O from rio, ecstore, disk layers ( #2533 )
2026-04-14 19:09:02 +08:00
8abf683178
fix(io-metrics): remove dead GET chunk fast path metrics ( #2531 )
2026-04-14 15:47:57 +08:00
458086dc80
fix(get): remove GET chunk fast path ( #2507 )
2026-04-12 23:12:23 +08:00
b8c45fc9e3
fix(get-object): harden GET fast path against mid-stream regressions ( #2472 )
2026-04-10 21:38:29 +08:00
064e21062d
fix(capacity): harden scope registry, scan symlink guard, and test temp dir cleanup ( #2432 )
...
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
2026-04-08 20:58:17 +08:00
79ffecbf14
refactor(storage): remove object cache plumbing ( #2422 )
...
Co-authored-by: heihutu <heihutu@gmail.com >
2026-04-07 22:00:53 +08:00
32bf8f5bf3
feat(storage): add direct chunk GET fast path ( #2351 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: cxymds <Cxymds@qq.com >
2026-04-07 08:33:46 +08:00
bd36cf3588
test(filemeta): cover legacy delete marker decoding ( #2333 )
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-31 22:08:45 +08:00
dd9e093dcc
perf(capacity): tune default capacity settings, sync docs, and fix refresh/metrics correctness ( #2336 )
...
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-03-30 05:33:56 +08:00
7172e151de
fix: address correctness, safety, and concurrency issues ( #2327 )
...
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-03-30 00:30:57 +08:00
houseme