mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-03 11:57:43 +00:00
Zhengchao An
c82ee6be58
feat(api): wire opt-in per-client S3 API rate limiting (backlog#1191) RustFS shipped three rate-limiter implementations and none was wired to any request path: the tower layer never returned 429 (its over-limit branch passed requests through) and was never instantiated, the console env switches only logged, and the Swift token bucket was never called. Replace them with one working, default-off implementation: - Rewrite rustfs/src/server/rate_limit.rs as a sharded per-client-IP token-bucket limiter (32 mutex shards instead of one global RwLock write per request), bounded at 100k tracked IPs with lossless refilled-idle sweeps, returning 429 + Retry-After + x-ratelimit-* headers and an S3-style XML body. - Key on trusted-proxy-validated ClientInfo.real_ip, else the socket peer address; never read spoofable X-Forwarded-For/X-Real-IP headers. Requests without a resolvable identity fail open. The echoed request id is charset-gated to prevent reflected XML injection. - Wire the layer once at startup via option_layer between CatchPanicLayer and ReadinessGateLayer (external stack only), gated by new RUSTFS_API_RATE_LIMIT_ENABLE/_RPM/_BURST constants; health and profiling probes, internode RPC/gRPC, and the console are exempt. - Make RUSTFS_CONSOLE_RATE_LIMIT_ENABLE/_RPM actually enforce by reusing the same limiter core through an axum middleware. - Delete the dead Swift ratelimit module, its isolated tests, and the stale logging-guardrail entry; keep the live SwiftError 429 mapping. - Add unit tests (exhaustion/recovery with injected time, concurrency, cap eviction, spoofed-header and fail-open behavior, env matrix) and e2e tests proving 429 + Retry-After on the real server and zero behavior change with default configuration.
75 lines
2.2 KiB
Rust
75 lines
2.2 KiB
Rust
// Copyright 2024 RustFS Team
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
#[cfg(feature = "constants")]
|
|
pub mod constants;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::api::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::app::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::body_limits::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::capacity::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::compress::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::console::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::drive::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::env::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::heal::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::health::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::internode::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::object::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::profiler::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::protocols::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::proxy::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::quota::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::runtime::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::scanner::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::targets::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::tls::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::workload::*;
|
|
#[cfg(feature = "constants")]
|
|
pub use constants::zero_copy::*;
|
|
#[cfg(feature = "constants")]
|
|
pub mod oidc {
|
|
pub use super::constants::oidc::*;
|
|
}
|
|
#[cfg(feature = "audit")]
|
|
pub mod audit;
|
|
#[cfg(feature = "notify")]
|
|
pub mod notify;
|
|
#[cfg(feature = "observability")]
|
|
pub mod observability;
|
|
#[cfg(feature = "opa")]
|
|
pub mod opa;
|
|
#[cfg(feature = "server-config-model")]
|
|
pub mod server_config;
|