feat(get): consolidate GET performance optimization (#3972)

* feat(get): consolidate GET performance optimization

Consolidated implementation of all GET performance optimizations into
a single, well-organized commit replacing the previous patch-on-patch
approach.

## Changes

### Configuration (set_disk/mod.rs)
- Consolidated all GET optimization flags into a single organized section
- Enabled by default: codec streaming, metadata early-stop, page cache reclaim
- Added codec streaming multipart flag (default: disabled)
- Added version-aware early-stop flag (default: disabled)
- Added adaptive duplex buffer sizing based on object size
- All flags use OnceLock caching with rollout percentage support

### Metadata Early-Stop (set_disk/read.rs)
- Delete marker early-stop when quorum agrees
- Version-aware early-stop for versioned GET requests
- MetadataQuorumAccumulator enhanced with:
  - delete_marker_votes tracking
  - requested_version_id and matching_version_votes tracking
  - version_early_stop_decision() method
- 6 new tests for version early-stop scenarios

### Codec Streaming (erasure/coding/decode_reader.rs)
- DualInFlight (2-stripe lookahead) enabled by default

### Decode Pipeline (erasure/coding/decode.rs)
- Stripe prefetch count configuration
- Bitrot-decode overlap configuration

### Disk Layer (disk/local.rs)
- O_DIRECT read configuration constants (preparation)

### Metrics (io-metrics/lib.rs)
- BytesPool acquisition/return metrics
- Metadata phase duration with early-stop label
- Total duration with reader_path label

### Diagnostics (diagnostics/)
- Early-stop reason constants
- Pool tier/outcome label constants

### Observability (.docker/observability/)
- 3 Grafana dashboards for GET optimization monitoring
- Prometheus alert rules (6 alerts: 3 critical, 3 warning)
- Updated README.md and README_ZH.md with usage docs

### Config (config/src/constants/runtime.rs)
- Page cache reclaim read enabled by default

## Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| RUSTFS_GET_CODEC_STREAMING_ENABLE | true | Codec streaming base flag |
| RUSTFS_GET_CODEC_STREAMING_ROLLOUT_PCT | 100 | Codec streaming rollout % |
| RUSTFS_GET_CODEC_STREAMING_MULTIPART_ENABLE | false | Multipart codec streaming |
| RUSTFS_GET_METADATA_EARLY_STOP_ENABLE | true | Early-stop base flag |
| RUSTFS_GET_METADATA_EARLY_STOP_ROLLOUT_PCT | 100 | Early-stop rollout % |
| RUSTFS_GET_METADATA_VERSION_EARLY_STOP_ENABLE | false | Version-aware early-stop |
| RUSTFS_OBJECT_FILE_CACHE_RECLAIM_READ_ENABLE | true | Page cache reclaim |
| RUSTFS_OBJECT_DIRECT_IO_READ_ENABLE | false | O_DIRECT (preparation) |
| RUSTFS_GET_DECODE_STRIPE_PREFETCH_COUNT | 1 | Stripe prefetch |
| RUSTFS_GET_BITROT_DECODE_OVERLAP_ENABLE | false | Bitrot-decode overlap |
| RUSTFS_GET_CODEC_STREAMING_MAX_INFLIGHT | 2 | DualInFlight stripes |

## Rollback

All optimizations can be disabled via environment variables:
RUSTFS_GET_CODEC_STREAMING_ENABLE=false
RUSTFS_GET_METADATA_EARLY_STOP_ENABLE=false
RUSTFS_OBJECT_FILE_CACHE_RECLAIM_READ_ENABLE=false

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(get): add stress test scripts for GET optimization validation

- quick-validate-get-optimization.sh: Quick 5-minute validation
- stress-test-get-optimization.sh: Full 30+ minute stress test
- README-stress-test.md: Usage documentation

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ecstore): align file cache reclaim defaults

* chore(deps): update redis and erasure codec

* test(ecstore): align decode fill policy default

* test(ecstore): align metadata early-stop default

* fix(ecstore): keep metadata early stop opt-in

---------

Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
houseme
2026-06-28 07:14:07 +08:00
committed by GitHub
parent 512418cda9
commit 27468ebfa9
22 changed files with 2886 additions and 71 deletions
+10
View File
@@ -94,6 +94,12 @@ pub(crate) const GET_METADATA_EARLY_STOP_REASON_NOT_FOUND: &str = "not_found";
pub(crate) const GET_METADATA_EARLY_STOP_REASON_UNSAFE_REQUEST: &str = "unsafe_request";
pub(crate) const GET_METADATA_EARLY_STOP_REASON_VALID_QUORUM: &str = "valid_quorum";
pub(crate) const GET_METADATA_EARLY_STOP_REASON_VERSION_NOT_FOUND: &str = "version_not_found";
pub(crate) const GET_METADATA_EARLY_STOP_REASON_VERSION_MATCH_QUORUM: &str = "version_match_quorum";
/// Early-stop active state labels
pub(crate) const EARLY_STOP_ACTIVE_HIT: &str = "hit";
pub(crate) const EARLY_STOP_ACTIVE_MISS: &str = "miss";
pub(crate) const EARLY_STOP_ACTIVE_DISABLED: &str = "disabled";
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub(crate) enum GetObjectFailureReason {
@@ -296,5 +302,9 @@ mod tests {
assert_eq!(GET_METADATA_EARLY_STOP_REASON_UNSAFE_REQUEST, "unsafe_request");
assert_eq!(GET_METADATA_EARLY_STOP_REASON_VALID_QUORUM, "valid_quorum");
assert_eq!(GET_METADATA_EARLY_STOP_REASON_VERSION_NOT_FOUND, "version_not_found");
assert_eq!(GET_METADATA_EARLY_STOP_REASON_VERSION_MATCH_QUORUM, "version_match_quorum");
assert_eq!(EARLY_STOP_ACTIVE_HIT, "hit");
assert_eq!(EARLY_STOP_ACTIVE_MISS, "miss");
assert_eq!(EARLY_STOP_ACTIVE_DISABLED, "disabled");
}
}
+1
View File
@@ -14,3 +14,4 @@
pub(crate) mod admin_server_info;
pub(crate) mod get;
pub(crate) mod pool;
+30
View File
@@ -0,0 +1,30 @@
// Copyright 2024 RustFS Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! BytesPool metric label constants.
//!
//! These constants are used when recording pool acquisition and return
//! metrics to avoid string allocations and ensure label consistency.
/// BytesPool tier labels
pub const POOL_TIER_SMALL: &str = "small";
pub const POOL_TIER_MEDIUM: &str = "medium";
pub const POOL_TIER_LARGE: &str = "large";
pub const POOL_TIER_XLARGE: &str = "xlarge";
/// BytesPool outcome labels
pub const POOL_OUTCOME_HIT: &str = "hit";
pub const POOL_OUTCOME_MISS: &str = "miss";
pub const POOL_OUTCOME_RECYCLED: &str = "recycled";
pub const POOL_OUTCOME_DROPPED: &str = "dropped";