gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-27 08:38:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
1.0.0-beta.10
rustfs/crates/config
T
History
Zhengchao An 78469aa63b fix(get): bound GET disk-read admission with a hard cap instead of unbounded permit bypass (#4935) 
Refs https://github.com/rustfs/backlog/issues/1317

Previously, when the primary disk-read permit pool stayed saturated past `RUSTFS_OBJECT_DISK_PERMIT_WAIT_TIMEOUT` (default 5s) — which a handful of slow clients can cause because a permit is held for the whole body transfer — the GET path set `disk_permit = None` and continued reading with no admission token at all. That made the disk-read concurrency limit unbounded under exactly the overload it is meant to protect against: any number of GETs could pile onto the disks simultaneously.

This replaces the permit-less bypass with a bounded overflow lane and a hard cap:

- A new bounded degraded semaphore (`RUSTFS_OBJECT_DISK_DEGRADED_READ_CAP`, default mirrors the primary cap) is consulted only after the primary wait times out. A GET takes one degraded permit without blocking, or is rejected with `SlowDown`/503 once that lane is also full. The total number of GETs performing disk-active reads is therefore hard-capped at `primary_cap + degraded_cap`, and no GET ever reads without holding an admission token.
- Admission is centralized in `ConcurrencyManager::admit_disk_read`, returning `Primary`/`Degraded`/`Unbounded`/`Rejected`. `Degraded` and `Rejected` are counted in metrics (`rustfs.get_object.disk_permit.degraded.total`, `rustfs.get_object.disk_permit.hard_reject.total`), replacing the removed `rustfs.get_object.disk_permit.bypass.total`.
- A primary cap of `0` (disk-read throttling disabled) is preserved as the only intentional permit-less path via `Unbounded`, so that degenerate-but-served configuration is not turned into all-503. The `primary_wait == 0` wait-forever opt-out is likewise unchanged.

Healthy GETs are unaffected: concurrency at or below the primary cap is admitted immediately from the primary pool exactly as before, with no new latency or rejection. The rejection is surfaced before response headers are constructed, so it is a clean pre-header 503, never a post-header 504 masquerade. Degraded-lane GETs use the identical streaming reader and forward-progress stall timeout as primary GETs, so a slow but progressing large download is not killed. Owned permits (primary or degraded) are held by `DiskReadPermitReader` and released on body EOF or client drop/cancel, so tokens are always returned.

Body stall timeout already resets on forward progress and post-header failures already surface as body errors, so no timeout-split changes were needed here.

Scope: this PR implements the minimal safe correctness core (eliminate unbounded bypass + hard cap + SlowDown). The two-level weighted-fair + aging scheduler (small setup cap feeding a size-aware data-producer fair queue) and the strictly-bounded producer/client buffer with cancellation propagation from the issue plan remain follow-ups. The black-box slow-client soak is deferred to the unbuilt facility in https://github.com/rustfs/backlog/issues/1325 rather than faked.

Tests (white-box, virtual clock via `start_paused`): degrade-then-hard-reject with token release; 100 concurrent GETs against primary=1/degraded=1 never exceed 2 simultaneous admissions with every request either admitted or explicitly rejected; disabled cap serves unbounded; zero-wait blocks on the primary lane. Reverting the hard cap makes the concurrency invariant fail.
2026-07-17 08:30:39 +08:00
..
src
fix(get): bound GET disk-read admission with a hard cap instead of unbounded permit bypass (#4935)
2026-07-17 08:30:39 +08:00
AGENTS.md
docs: scope AGENTS instructions by directory (#2083)
2026-03-05 17:25:37 +08:00
Cargo.toml
chore(deps): localize workspace dependency features (#4888)
2026-07-16 03:55:27 +00:00
README.md
fix(admin): support external OIDC browser redirects (#4280)
2026-07-09 18:07:49 +08:00

README.md

RustFS

RustFS Config - Configuration Management

Configuration management and validation module for RustFS distributed object storage

CI 📖 Documentation · 🐛 Bug Reports · 💬 Discussions

📖 Overview

RustFS Config provides configuration management and validation capabilities for the RustFS distributed object storage system. For the complete RustFS experience, please visit the main RustFS repository.

✨ Features

  • Multi-format configuration support (TOML, YAML, JSON, ENV)
  • Environment variable integration and override
  • Configuration validation and type safety
  • Hot-reload capabilities for dynamic updates
  • Default value management and fallbacks
  • Secure credential handling and encryption

📚 Documentation

For comprehensive documentation, examples, and usage guides, please visit the main RustFS repository.

Environment Variable Naming Conventions

RustFS uses a flat naming style for top-level configuration: environment variables are RUSTFS_* without nested module segments.

Examples:

  • RUSTFS_REGION
  • RUSTFS_ADDRESS
  • RUSTFS_VOLUMES
  • RUSTFS_LICENSE
  • RUSTFS_LICENSE_PUBLIC_KEY

Current guidance:

  • Prefer module-specific names only when they are not top-level product configuration.
  • Renamed variables must keep backward-compatible aliases until before beta.
  • Alias usage must emit deprecation warnings and be treated as transitional only.
  • Deprecated example:
    • RUSTFS_ENABLE_SCANNER -> RUSTFS_SCANNER_ENABLED
    • RUSTFS_ENABLE_HEAL -> RUSTFS_HEAL_ENABLED
    • RUSTFS_DATA_SCANNER_START_DELAY_SECS -> RUSTFS_SCANNER_START_DELAY_SECS

License environment variables

  • RUSTFS_LICENSE contains the signed license token.
  • RUSTFS_LICENSE_PUBLIC_KEY contains the RSA public key used to verify signed license tokens.

CORS environment variables

  • RUSTFS_CORS_ALLOWED_ORIGINS defaults to empty, so the S3 endpoint emits no generic CORS headers unless configured. Set * for wildcard origins without credentials, or a comma-separated allow-list for credentialed explicit origins.
  • RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS defaults to * for the console service.

Browser redirect environment variables

  • RUSTFS_BROWSER_REDIRECT_URL sets the externally reachable browser origin used for OIDC callback, console success redirect, and logout fallback URLs. Configure it to the public scheme and authority without a path, for example https://console.example.com. In load-balancer deployments, keep OIDC authorize and callback requests on the same backend node because the in-flight OIDC state is local to the RustFS node.

Scanner environment aliases

  • RUSTFS_SCANNER_SPEED (canonical, also accepts MINIO_SCANNER_SPEED)
  • RUSTFS_SCANNER_DELAY (canonical)
  • RUSTFS_SCANNER_MAX_WAIT_SECS (canonical)
  • RUSTFS_SCANNER_CYCLE (canonical, also accepts MINIO_SCANNER_CYCLE)
  • RUSTFS_SCANNER_START_DELAY_SECS (canonical)
  • RUSTFS_DATA_SCANNER_START_DELAY_SECS (deprecated alias for compatibility)
  • RUSTFS_SCANNER_IDLE_MODE (canonical)
  • RUSTFS_SCANNER_CACHE_SAVE_TIMEOUT_SECS (canonical)
  • RUSTFS_SCANNER_CYCLE_MAX_DURATION_SECS (canonical)
  • RUSTFS_SCANNER_CYCLE_MAX_OBJECTS (canonical)
  • RUSTFS_SCANNER_CYCLE_MAX_DIRECTORIES (canonical)

Mmap read environment aliases

  • RUSTFS_OBJECT_MMAP_READ_ENABLE (canonical)
  • RUSTFS_OBJECT_ZERO_COPY_ENABLE (deprecated alias for compatibility)

Health compatibility switches

  • RUSTFS_HEALTH_ENDPOINT_ENABLE
    • controls canonical /health, /health/live, and /health/ready endpoint exposure.
  • RUSTFS_HEALTH_MINIMAL_RESPONSE_ENABLE
    • enables minimal payload mode for GET health responses (status, ready only).
  • RUSTFS_HEALTH_READINESS_CACHE_TTL_MS
    • TTL for readiness cache evaluation.
  • RUSTFS_HEALTH_COMPAT_BUSY_CHECK_ENABLE
    • enables busy protection behavior for health probes.
    • default is false.
  • RUSTFS_HEALTH_COMPAT_BUSY_MAX_ACTIVE_REQUESTS
    • max active HTTP requests; health probes return 429 when active requests reach or exceed this value.
    • 0 disables thresholding even if busy protection is enabled.
  • RUSTFS_HEALTH_COMPAT_KMS_READY_CHECK_ENABLE
    • enables KMS readiness enforcement for /health/ready.
    • default is false.

Drive timeout environment variables

  • RUSTFS_DRIVE_METADATA_TIMEOUT_SECS
  • RUSTFS_DRIVE_DISK_INFO_TIMEOUT_SECS
  • RUSTFS_DRIVE_LIST_DIR_TIMEOUT_SECS
  • RUSTFS_DRIVE_WALKDIR_TIMEOUT_SECS
  • RUSTFS_DRIVE_WALKDIR_STALL_TIMEOUT_SECS

Legacy compatibility fallback:

  • RUSTFS_DRIVE_MAX_TIMEOUT_DURATION This legacy variable is treated as a deprecated fallback for the operation-specific drive timeout variables above when a canonical variable is unset.

Drive timeout health-action policy:

  • RUSTFS_DRIVE_TIMEOUT_HEALTH_ACTION
    • mark_failure (default): timeout marks failure and may transition drive runtime state.
    • ignore_scanner: timeout does not mark failure for scanner-sensitive operations (walk_dir, read_metadata, list_dir, disk_info).

Drive timeout profile preset:

  • RUSTFS_DRIVE_TIMEOUT_PROFILE
    • default (default): keep current timeout defaults.
    • high_latency: use 60s default timeout for scanner-sensitive operations when no per-operation timeout override is set (read_metadata, disk_info, list_dir, walk_dir, walk_dir_stall).
  • Precedence:
    • Explicit per-operation timeout env (RUSTFS_DRIVE_*_TIMEOUT_SECS) takes highest precedence.
    • Then RUSTFS_DRIVE_MAX_TIMEOUT_DURATION legacy fallback.
    • Then the profile-derived default (default or high_latency).

Startup filesystem boundary policy

  • RUSTFS_UNSUPPORTED_FS_POLICY controls startup behavior when RustFS detects local endpoint filesystems that are outside the supported production boundary.
    • warn (default): log warning and continue startup.
    • fail: abort startup with an error.

RustFS production guidance remains direct-attached local POSIX filesystems. Network-mounted filesystems (for example nfs, cifs, smb2, and fuse.*) are treated as unsupported by this startup guard.

📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Reference in New Issue View Git Blame Copy Permalink