mirror of
https://github.com/rustfs/rustfs.git
synced 2026-08-31 01:09:23 +00:00
feat(api): wire opt-in per-client S3 API rate limiting (429 + Retry-After) (#4895)
feat(api): wire opt-in per-client S3 API rate limiting (backlog#1191) RustFS shipped three rate-limiter implementations and none was wired to any request path: the tower layer never returned 429 (its over-limit branch passed requests through) and was never instantiated, the console env switches only logged, and the Swift token bucket was never called. Replace them with one working, default-off implementation: - Rewrite rustfs/src/server/rate_limit.rs as a sharded per-client-IP token-bucket limiter (32 mutex shards instead of one global RwLock write per request), bounded at 100k tracked IPs with lossless refilled-idle sweeps, returning 429 + Retry-After + x-ratelimit-* headers and an S3-style XML body. - Key on trusted-proxy-validated ClientInfo.real_ip, else the socket peer address; never read spoofable X-Forwarded-For/X-Real-IP headers. Requests without a resolvable identity fail open. The echoed request id is charset-gated to prevent reflected XML injection. - Wire the layer once at startup via option_layer between CatchPanicLayer and ReadinessGateLayer (external stack only), gated by new RUSTFS_API_RATE_LIMIT_ENABLE/_RPM/_BURST constants; health and profiling probes, internode RPC/gRPC, and the console are exempt. - Make RUSTFS_CONSOLE_RATE_LIMIT_ENABLE/_RPM actually enforce by reusing the same limiter core through an axum middleware. - Delete the dead Swift ratelimit module, its isolated tests, and the stale logging-guardrail entry; keep the live SwiftError 429 mapping. - Add unit tests (exhaustion/recovery with injected time, concurrency, cap eviction, spoofed-header and fail-open behavior, env matrix) and e2e tests proving 429 + Retry-After on the real server and zero behavior change with default configuration.
This commit is contained in:
@@ -16,7 +16,7 @@
|
||||
|
||||
#![cfg(feature = "swift")]
|
||||
|
||||
use rustfs_protocols::swift::{quota, ratelimit, slo, symlink, sync, tempurl, versioning};
|
||||
use rustfs_protocols::swift::{quota, slo, symlink, sync, tempurl, versioning};
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// Test sync configuration parsing
|
||||
@@ -92,14 +92,6 @@ fn test_symlink_detection() {
|
||||
let _is_symlink = symlink::is_symlink(&metadata);
|
||||
}
|
||||
|
||||
/// Test rate limit parsing
|
||||
#[test]
|
||||
fn test_rate_limit_parsing() {
|
||||
let rl = ratelimit::RateLimit::parse("100/60").unwrap();
|
||||
assert_eq!(rl.limit, 100);
|
||||
assert_eq!(rl.window_seconds, 60);
|
||||
}
|
||||
|
||||
/// Test quota structure
|
||||
#[test]
|
||||
fn test_quota_structure() {
|
||||
|
||||
Reference in New Issue
Block a user