Files
rustfs/crates
Zhengchao An 97edb2e5cf feat(api): add opt-in per-bucket dimension to the S3 API rate limiter (#4949)
Follow-up to #4895 (backlog#1191 deferred sub-item). The client-IP
dimension gives per-client fairness; this adds a collective per-bucket
budget so one hot bucket cannot monopolize the server regardless of how
many client IPs the traffic is spread across.

- Generalize RateLimiter over its key type with a Borrow-based check()
  so &str lookups against String bucket keys stay allocation-free on
  the hit path; client-IP call sites are unchanged in behavior.
- RateLimitLayer now carries optional client and bucket limiters; a
  request must pass every configured dimension, and rejections report
  which one tripped via a new 'dimension' metric label.
- Bucket extraction mirrors s3s host routing: virtual-hosted-style
  resolves the Host/authority prefix against the same expanded domain
  set (with port variants) the s3s router uses; otherwise the first
  path segment. Admin and table-catalog namespaces are never buckets.
- New env vars RUSTFS_API_RATE_LIMIT_BUCKET_RPM/_BURST (default 0 =
  dimension off) under the existing enable switch; bucket-only
  configurations (client RPM 0) are supported.
- Bucket names are attacker-chosen, so the bounded-shards design
  (100k keys, lossless idle sweeps, most-idle eviction) is the memory
  defense; a test floods 10k random names and asserts the cap holds.
- Unit tests for extraction, shared bucket budgets across IPs, both-
  dimensions interaction, and the extended env matrix; e2e test proves
  bucket-only throttling on the real server with an unrelated bucket
  unaffected.
2026-07-17 19:01:18 +08:00
..