Chris 641c4b3493 chore(release): merge 1.0.1 back into main and refresh installation references (#8314)
* fix(ci): include pagination regression in full E2E selection

* fix(deps): replace yanked yoke-derive release

* fix(scanner): expose pause backlog replica diagnostics (#8258)

* fix(scanner): expose pause backlog replica diagnostics

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(connect): stabilize runtime profile lease cancellation

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(connect): tolerate delayed schedule startup in CI

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(e2e): retry quota reads during usage warmup

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): avoid meta-bucket incarnation self-deadlock

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(test): use persisted incarnation in heal fixture

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* fix(usage): reconcile stale counters after lifecycle expiration (#8108)

* fix(usage): reconcile stale counters after lifecycle expiration

* fix(usage): account lifecycle expiry during continuous writes

* test(usage): run lifecycle usage scenarios on one scanner store

* test(usage): use a Windows-representable pre-mutation offset

* fix(usage): harden expiry accounting recovery and quota checks

Borrow expiry receipt bucket names and avoid allocating a map key on cache hits. Cover cancelled receipts, durable snapshot recovery, and legacy quota admission after scanner confirmation. Use representable timestamp offsets in the quota regression.

Refs rustfs/backlog#2689

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(usage): recover stale persisted counts through the scanner

Seed incorrect complete usage for empty and retained-object buckets, then run the real scanner and publication consumer without further object mutations. Verify durable and admin usage over two cycles instead of writing a corrected snapshot in the test.

Refs rustfs/backlog#2689
Refs rustfs/backlog#2691

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(e2e): bound delimiter pagination fixture concurrency

The 120-second smoke timeout expired after 1018 of 1200 serial fixture PUTs, before LIST ran. Prepare the same objects with at most eight concurrent requests and await every PUT. Retain the timeout and strengthen exact prefix, KeyCount, empty Contents, and continuation-token assertions with phase diagnostics.

Refs rustfs/backlog#2689

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(upgrade): establish a persisted previous-release baseline

Seed the pinned previous-release cluster and restart it once with its data intact before replacing any node. Require every old writer to pass the strict readiness probe and preserve the seed through both mixed phases and the final current cluster. Keep InternalError fail-fast behavior and all existing compatibility deadlines and assertions.

Refs rustfs/backlog#2689
Refs rustfs/backlog#2384

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): bound cancellation metadata persistence waits

Use the system-bucket incarnation boundary now supplied by main PR #8268. Bound the three cancellation waits that previously hung during pool.bin persistence, retaining their remote-generation, target-cohort, and durable-state assertions.

Refs rustfs/backlog#2697
Refs rustfs/backlog#2689

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: Chris <anzhengchao@gmail.com>
Co-authored-by: Hauser <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* test(e2e): retain startup and shutdown failure diagnostics

* fix(test): supply CPU workload for sampler regression

* fix(ci): locate security chain scripts in the workspace

* fix(usage): recover historical counters with generation fencing (#8273)

* fix(usage): recover historical counters during continued writes

Use newer converged scanner snapshots to reconcile stale absolute usage
baselines while preserving concurrent mutation and expiry receipt fences.
Cover durable publication, admin and quota reads, and legacy generations.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(usage): fence snapshots and move preserved cache entries

Apply the cached scanner generation floor before every reconciliation path
and retain it even when an older snapshot happens to match core counts.
Move preserved usage entries instead of cloning their histogram maps under
the cache lock, retaining expiry receipt identity and cancellation fences.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* test(scanner): verify checkpoint takeover and repair dispatch (#8275)

* test(scanner): cover checkpoint handoff and repair dispatch

Drive runtime budget expiry, partial-cycle persistence, leadership claims,
and stale checkpoint rejection between real disk-backed fixture scans.
Verify that a metadata repair beyond the first bounded prefix is saved in
the scanner ledger and dispatched by the MRF consumer.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* ci: isolate scanner fixtures and refresh full e2e membership

Reserve nextest capacity for the real-disk scanner publication and MRF
admission fixtures. Bind both platform membership checks to the reviewed
pagination deadline test added on main.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(scanner): consolidate checkpoint fixture lifecycle

Keep one durable control store across timeout and leadership transitions,
and inject generation advancement into the shared checkpoint scenario.
Check the actual saved metadata path so late-write rejection also proves
that existing checkpoint bytes remain intact.

Centralize MRF fixture isolation and reuse nextest process isolation when
the startup environment already satisfies the test contract.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* fix(obs): distinguish allocator counters from live memory (#8274)

* fix(obs): distinguish allocator counters from live memory

Preserve count/counter semantics and mark requested-byte attribution unavailable when live statistics or sampling are missing. Document sustained multipart memory diagnosis.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(obs): parse allocator statistics from one node

Resolve each statistic before interpreting its shape, avoiding unsupported-field tree scans and mixing data across wrapper scopes. Preserve unavailable-statistics policy and add precedence regressions.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): isolate late parity recovery from metadata hedges

Use the existing object-scoped hedge timer barrier in exact-count recovery fixtures. Preserve payload and total-read assertions and verify that the omitted parity disk is read only during late refresh.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* fix(usage): combine identical snapshot retention branches

* fix(test): await HTTP sender readiness in Top RPC fixture

* [release/1.0.1] Gate multipart copy through write admission (#8284)

Gate multipart copy through write admission

Make UploadPartCopy acquire the shared foreground write admission permit before lifecycle locks or source readers so server-side multipart copy cannot bypass the same backpressure used by UploadPart. Document the shared queue semantics and add focused coverage for saturation, cancellation, lock ordering, and disabled admission.

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* Gate multipart copy through write admission (#8283)

Make UploadPartCopy acquire the shared foreground write admission permit before lifecycle locks or source readers so server-side multipart copy cannot bypass the same backpressure used by UploadPart. Document the shared queue semantics and add focused coverage for saturation, cancellation, lock ordering, and disabled admission.

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* fix: add UploadPart OOM validation guardrails (#8287)

* docs(release): validate candidates on release branch

* fix(scanner): validate checkpoints against global cycle fence (#8278)

## Related Issues

Related to rustfs/backlog#2701.

## Summary of Changes

Route scanner checkpoint cycle and leader validation through the global store while retaining the owning set for cache persistence, CAS revisions and publication admission.

## Verification

Two independent final-diff source reviews found no issues across correctness, concurrency and durability, test coverage, compatibility, performance and simplicity on head `8b8fe51d092090b053f552ae283960e2e306be33`. Root approval `5373624714` is bound to that exact head. Regression tests cover real two-pool routing, stale fences, post-save rejection and CAS conflicts; their reported local execution belongs to the PR author, not this merge operation. Current required CI remains pending, and this authorized admin squash does not establish CI or runtime acceptance.

## Impact

Restores checkpoint progress when global cycle and leader state differ from a set-scoped view. No format, retry, timeout, assertion or scanner-policy changes are introduced by this diff. The three prior main scanner failures remain unproved repaired.

## Additional Notes

Full validation must run on the resulting exact main revision. Reverting this patch restores the earlier set-scoped fence lookup and its checkpoint rejection behavior.

* fix(ci): restore E2E membership and pagination timeouts (#8281)

* ci: locate the auto-testing checkout for lanes that run evidence from a subdirectory (#8279)

## Related Issues

Follow-up to #8229.

## Summary of Changes

Locate the private auto-testing checkout from the lane root or the workspace root so the nested security checkout can record functional-chain evidence.

## Verification

The exact PR head b66129ab9f passed one mechanical correctness and simplicity review, nine real-Git layout and provenance checks, and sixteen existing evidence/envelope tests. The baseline sibling layout failed with git exit 128; the corrected layout succeeded while revision mismatches and missing checkouts stayed rejected. Current PR checks are completed with successful or skipped conclusions, including the aggregate.

## Impact

Both lane and private-script revision checks remain intact. No time limits, assertions, production behavior, or evidence validation requirements change. The synthetic layout checks do not execute the actual scheduled security suite; integrated main CI and release acceptance remain separate gates.

## Additional Notes

Approved review 5374559393 is bound to the exact head above. Reverting the single-file change restores the previous checkout lookup.

* fix: add UploadPart OOM validation guardrails

Add a Docker validation harness for backlog#2704 so the ordinary UploadPart
low-concurrency memory workload can be reproduced with comparable case metadata,
process/cgroup sampling, TLS and metrics toggles, cache-env controls, and write
reclaim/direct-write experiments.

Warn when operators set the unrecognized RUSTFS_OBJECT_CACHE_* variables that
appeared in the reporter compose file. The variables are reported but remain
ignored, so startup does not silently change object data cache behavior.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: overtrue <anzhengchao@gmail.com>
Co-authored-by: AL <allan.bednarowski@gmail.com>
Co-authored-by: hector <42570491+majinghe@users.noreply.github.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* fix(scanner): pass fence store to checkpoint fixture

* fix(s3): queue bucket operations and restore strict Clippy checks (#8290)

* fix(s3): queue concurrent bucket creation and deletion

Keep eight active bucket transactions and bound admission waiting to 128 requests and 30 seconds. Preserve detached transaction ownership and return Retry-After with overload responses.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): restore strict Clippy compatibility on Rust 1.99

Use try_update without changing atomic ordering or overflow behavior. Keep
recursive storage futures boxed once at each frame and remove the redundant
async-recursion macro, including its non-recursive SQL planner use. Remove
needless closure borrows and orphaned dependency entries.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* chore: refresh dependencies and atomic update APIs

Update workspace dependencies and the lockfile. Replace deprecated
atomic fetch_update aliases with try_update while preserving closures
and memory ordering.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* improve

* fix(scanner): diagnose and verify pause backlog recovery (#8293)

* fix(scanner): diagnose and verify pause backlog recovery

Expose the retained replica snapshot and claimed membership in abnormal
admin status responses. Keep diagnostics off metrics updates and verify
single-pool recovery and conflicting-proof preservation across 24 sets.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(scanner): move replica snapshots into diagnostics

Consume the terminal admin read snapshot in a single state match and move
membership, revision, and error buffers into the response. Verify buffer
handoff and the unchanged JSON contract without altering ledger authority.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* fix(heal): wait for held legacy responsibility in replay test

* ci: remove Docker Hub description sync

* fix: emit NextPartNumberMarker only when ListParts is truncated

ListPartsInfo.next_part_number_marker was a non-optional usize that
defaulted to 0 and was only assigned when the response was truncated.
The S3 serializer then emitted it unconditionally as Some(0), causing
AWS SDK paginators to loop infinitely on part_number_marker=0 instead
of terminating.

Change the field to Option<usize> (None by default) and set it only
inside the is_truncated branch. The S3 output layer now uses
.and_then() so NextPartNumberMarker is absent when IsTruncated=false,
matching AWS S3 behavior.

Fixes #8208

(cherry picked from commit 44de803a38)

* fix(s3): honor sparse ListParts markers and verify termination

Resume part listings at the first part above the numeric marker, even
when that marker is absent. Use binary search over the sorted part
numbers and retain the existing exact-tail empty-slice path.

Add storage, XML, and real AWS SDK paginator regressions for empty and
terminal pages, sparse markers, and multipart completion integrity.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
(cherry picked from commit 673031eea1)

* fix(storage): publish delete rollback backups atomically

Stage rollback metadata outside the rollback directory and publish it only after the full write succeeds. A short write must not leave a backup that quorum rollback can rename over acknowledged version history.

Add an isolated real short-write regression and register the backported ListParts SDK test in the smoke and Linux full inventories.

* test(e2e): register paginator regression in Darwin inventory

* fix(release): install yq before Helm template checks

* chore(release): align installation references for 1.0.1

---------

Co-authored-by: Hauser <housemecn@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: Peder Bergan <pederbe@users.noreply.github.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: AL <allan.bednarowski@gmail.com>
Co-authored-by: hector <42570491+majinghe@users.noreply.github.com>
Co-authored-by: Chapman <touch65536@gmail.com>
2026-10-03 14:40:52 +08:00
2025-12-18 20:13:24 +08:00
2025-07-08 09:04:37 +08:00
2025-08-07 22:37:05 +08:00

RustFS

RustFS is a high-performance, distributed object storage system built in Rust.

CI Build and Push Docker Images GitHub commit activity Github Last Commit Discord Featured|HelloGitHub

rustfs%2Frustfs | Trendshift

Getting Started · Docs · Bug reports · Discussions

English | 简体中文 | Deutsch | Español | français | 日本語 | 한국어 | Portuguese | Русский

RustFS is a high-performance, distributed object storage system built in Rust—one of the most loved programming languages worldwide. RustFS combines the simplicity of MinIO with the memory safety and raw performance of Rust. It offers broad S3 API compatibility for supported features, is completely open-source, and is optimized for data lakes, AI, and big data workloads.

Unlike other storage systems, RustFS is released under the permissible Apache 2.0 license, avoiding the restrictions of AGPL. With Rust as its foundation, RustFS delivers superior speed and secure distributed features for next-generation object storage.

Feature & Status

  • High Performance: Built with Rust to ensure maximum speed and resource efficiency.
  • Distributed Architecture: Scalable and fault-tolerant design suitable for large-scale deployments.
  • S3 Compatibility: Seamless integration with common S3-compatible applications and tools; current coverage is tracked in the S3 compatibility matrix.
  • OpenStack Swift API: Native support for Swift protocol with Keystone authentication.
  • OpenStack Keystone Integration: Native support for OpenStack Keystone authentication with X-Auth-Token headers.
  • Data Lake Support: Optimized for high-throughput big data and AI workloads.
  • Open Source: Licensed under Apache 2.0, encouraging unrestricted community contributions and commercial usage.
  • User-Friendly: Designed with simplicity in mind for easy deployment and management.

Status legend: ✅ Available — shipped and covered by CI gates; 🧪 Preview — shipped behind an opt-in flag or with a bounded compatibility claim.

Feature Status Feature Status
S3 Core Features ✅ Available Distributed Mode ✅ Available
Upload / Download ✅ Available Single Node Mode ✅ Available
Versioning ✅ Available Bitrot Protection ✅ Available
Object Lock (WORM) ✅ Available Healing & Scanner ✅ Available
Server-Side Encryption ✅ Available Pool Expansion / Decommission ✅ Available
RustFS KMS ✅ Available Bucket Replication ✅ Available
Lifecycle Management (ILM) ✅ Available Site Replication ✅ Available
ILM Tiering (Remote S3) ✅ Available Bucket Quota ✅ Available
S3 Select ✅ Available Event Notifications ✅ Available
S3 Tables (Iceberg REST) 🧪 Preview Audit Logging ✅ Available
IAM / Policies ✅ Available Logging & Observability ✅ Available
OIDC / SSO ✅ Available Web Console ✅ Available
Keystone Auth ✅ Available K8s Helm Charts ✅ Available
Swift API ✅ Available FTPS / WebDAV ✅ Available
Multi-Tenancy ✅ Available SFTP ✅ Available
MinIO On-Disk Compatibility 🧪 Preview

Notes:

  • RustFS KMS: Vault (KV2 / Transit) and AWS KMS backends are supported for production. The Local and Static backends are for development and testing only. See KMS backend security properties.
  • Swift API / SFTP: opt-in cargo features (--features swift, --features sftp, or full). FTPS and WebDAV are enabled in the default build.
  • S3 Tables: ships as an Iceberg REST Catalog with automated PyIceberg and DuckDB coverage; other engines and vendor profiles carry bounded claims listed in the S3 Tables support matrix.
  • MinIO On-Disk Compatibility: gated behind the rio-v2 feature and not part of the default build. Objects MinIO encrypted are not readable by RustFS. See MinIO file-format interoperability.

RustFS vs MinIO Performance

Stress Test Environment:

Type Parameter Remark
CPU 2 Core Intel Xeon (Sapphire Rapids) Platinum 8475B, 2.7/3.2 GHz
Memory 4GB
Network 15Gbps
Drive 40GB x 4 IOPS 3800 / Drive

https://github.com/user-attachments/assets/2e4979b5-260c-4f2c-ac12-c87fd558072a

RustFS vs Other Object Storage

Feature RustFS Other Object Storage
Console Experience Powerful Console
Comprehensive management interface.
Basic / Limited Console
Often overly simple or lacking critical features.
Language & Safety Rust-based
Memory safety by design.
Go or C-based
Potential for memory GC pauses or leaks.
Data Sovereignty No Telemetry / Full Compliance
Guards against unauthorized cross-border data egress. Compliant with GDPR (EU/UK), CCPA (US), and APPI (Japan).
Potential Risk
Possible legal exposure and unwanted data telemetry.
Licensing Permissive Apache 2.0
Business-friendly, no "poison pill" clauses.
Restrictive AGPL v3
Risk of license traps and intellectual property pollution.
Compatibility S3-Compatible Core
Works with common S3-compatible clients, with coverage tracked in the compatibility matrix.
Variable Compatibility
May lack support for local cloud vendors or specific APIs.
Edge & IoT Strong Edge Support
Ideal for secure, innovative edge devices.
Weak Edge Support
Often too heavy for edge gateways.
Risk Profile Enterprise Risk Mitigation
Clear IP rights and safe for commercial use.
Legal Risks
Intellectual property ambiguity and usage restrictions.

Staying ahead

Star RustFS on GitHub and be instantly notified of new releases.

Quickstart

Important

Pool expansion notice:

  • A single-node single-drive (SNSD) deployment is supported only as a standalone local path. It cannot expand in place or be added as a Pool. To move to a multi-drive topology, create a new deployment and migrate data through S3.
  • Keep an existing multi-drive Pool's endpoints and Erasure Set width unchanged; expand by appending a new Pool. With ellipsis-based expansion, every Pool argument must contain an ellipsis expression and expand to at least two drive endpoints.
  • Single-node multi-drive Pools and multi-node Pools with one drive per node are allowed, subject to valid Erasure Set geometry and EC settings; acceptance does not guarantee host-failure tolerance.

These topology rules follow MinIO, but automatic parity selection differs between the projects. See the Pool layout compatibility and regression tests before expanding a deployment.

To get started with RustFS, follow these steps:

1. One-click Installation (Option 1)

curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh

2. Docker Quick Start (Option 2)

The RustFS container runs as a non-root user rustfs (UID/GID 10001:10001). If you bind-mount host directories with Docker or Compose, every mounted path must be writable by that user, otherwise startup may fail with permission denied errors. This applies to data directories, log directories, and TLS certificate directories when RUSTFS_TLS_PATH is enabled.

# Create data and logs directories
mkdir -p data logs

# Change the owner of these directories
chown -R 10001:10001 data logs

# Using latest version
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

# Using specific version
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.1

If you use podman instead of docker, you can install the RustFS with the below command

# Create data and logs directories
mkdir -p data logs

# Run the container (podman will automatically set the folders ownership)
podman run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data:Z,U -v $(pwd)/logs:/logs:Z,U rustfs/rustfs:latest

If you enable TLS with a bind-mounted certificate directory, prepare that mount the same way:

mkdir -p certs
chown -R 10001:10001 certs

You can also use Docker Compose. Using the docker-compose-simple.yml file in the root directory:

docker compose -f docker-compose-simple.yml up -d

Before running Compose with host bind mounts:

  • Ensure every mounted host path is writable by 10001:10001.
  • If you enable TLS, ensure the certificate mount for /opt/tls is also readable by 10001:10001.
  • If matching host ownership is not practical, run the rustfs service with user: "<host-uid>:<host-gid>" instead.
  • docker-compose-simple.yml includes a volume-permission-helper service for named volumes. docker-compose-simple.yml relies on you to prepare bind-mounted host paths in advance.

Similarly, you can run the command with podman

podman compose -f docker-compose-simple.yml up -d

Webhook notification quick start (Docker):

docker run -d --name rustfs -p 9000:9000 \
  -e RUSTFS_NOTIFY_ENABLE=true \
  -e RUSTFS_NOTIFY_WEBHOOK_ENABLE_PRIMARY=on \
  -e RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY=http://<host-ip>:3020/webhook \
  -e RUSTFS_NOTIFY_WEBHOOK_QUEUE_DIR_PRIMARY=/tmp/rustfs-events \
  -e RUSTFS_OUTBOUND_ALLOW_ORIGINS=http://<host-ip>:3020 \
  rustfs/rustfs:latest

Notes:

  • RUSTFS_NOTIFY_ENABLE=true enables the global notify module switch.
  • For ARN arn:rustfs:sqs::primary:webhook, use instance-scoped env vars with _PRIMARY.
  • If queue dir is omitted, the official image maps /opt/rustfs/events into the persistent /data volume so it is writable by the runtime user and pending events survive container recreation. Override queue_dir when using another deployment layout.
  • RUSTFS_NOTIFY_WEBHOOK_SKIP_TLS_VERIFY_PRIMARY defaults to false; enabling it skips webhook TLS certificate verification, allows MITM attacks, and emits a startup warning. Prefer RUSTFS_NOTIFY_WEBHOOK_CLIENT_CA_PRIMARY for private CAs.
  • Since 1.0.0-beta.11, webhook endpoints on private or container networks (Docker Compose service names, host.docker.internal, RFC 1918 addresses) are blocked unless their exact scheme://host:port origin is listed in RUSTFS_OUTBOUND_ALLOW_ORIGINS (the origin only, without the path). See Outbound Connection Policy.

NOTE: We recommend reviewing the docker-compose.yml file before running. It defines several services including Grafana, Prometheus, and Jaeger, which are helpful for RustFS observability. If you wish to start Redis or Nginx containers, you can specify the corresponding profiles.

3. Build from Source (Option 3) - Advanced Users

For developers who want to build RustFS Docker images from source with multi-architecture support:

# Build multi-architecture images locally
./docker-buildx.sh

# Build a single-platform image locally
./docker-buildx.sh -p linux/amd64

# Build and push to registry
./docker-buildx.sh --push

# Build specific version
./docker-buildx.sh --release 1.0.1 --push

# Build for custom registry
./docker-buildx.sh --registry your-registry.com --namespace yourname --push

The docker-buildx.sh script supports:

  • Multi-architecture builds: linux/amd64, linux/arm64
  • Automatic version detection: Uses git tags or commit hashes
  • Registry flexibility: Supports Docker Hub, GitHub Container Registry, etc.
  • Build optimization: Includes caching and parallel builds

You can also use Make targets for convenience:

make docker-buildx                    # Build locally
make docker-buildx-push               # Build and push
make docker-buildx-version VERSION=1.0.1  # Build specific version
make help-docker                      # Show all Docker-related commands

Heads-up (macOS cross-compilation): macOS keeps the default ulimit -n at 256, so cargo zigbuild or ./build-rustfs.sh --platform ... may fail with ProcessFdQuotaExceeded when targeting Linux. The build script attempts to raise the limit automatically, but if you still see the warning, run ulimit -n 4096 (or higher) in your shell before building.

4. Build with Helm Chart (Option 4) - Cloud Native

Follow the instructions in the Helm Chart README to install RustFS on a Kubernetes cluster.

For scanner pacing, cycle budgets, bitrot cadence, lifecycle transition status, and single-node single-disk idle CPU tuning, see Scanner Runtime Controls. For repeatable scanner-pressure validation, see Scanner Benchmark Runbook. For drive timeout knobs on slow storage — including the walk stall budget that governs ListObjects on large prefixes — see Drive Timeout Tuning.

5. Nix Flake (Option 5)

If you have Nix with flakes enabled:

# Run directly without installing
nix run github:rustfs/rustfs

# Build the binary
nix build github:rustfs/rustfs
./result/bin/rustfs --help

# Or from a local checkout
nix build
nix run

The flake also exports a NixOS module and the RustFS rc client. Add the module to your system and provide credentials through runtime files (for example, sops-nix or agenix) so secrets are never stored in the Nix store:

imports = [ inputs.rustfs.nixosModules.rustfs ];

services.rustfs = {
  enable = true;
  accessKeyFile = "/run/secrets/rustfs-access-key";
  secretKeyFile = "/run/secrets/rustfs-secret-key";
  volumes = [ "/var/lib/rustfs" ];
};

Install the S3-compatible client with nix profile install github:rustfs/rustfs#rustfs-client (the executable is named rc), or use inputs.rustfs.packages.${pkgs.system}.rustfs-client in a system configuration.

6. X-CMD (Option 6)

If you are an x-cmd user:

# Run directly without installing
x rustfs

# Download the binary and install it to the global environment
x env use rustfs
rustfs --help

Accessing RustFS

  1. Access the Console: Open your web browser and navigate to http://localhost:9001 to access the RustFS console.
    • Default credentials: rustfsadmin / rustfsadmin
  2. Create a Bucket: Use the console to create a new bucket for your objects.
  3. Upload Objects: You can upload files directly through the console or use S3-compatible APIs/clients to interact with your RustFS instance.

NOTE: To access the RustFS instance via https, please refer to the TLS Configuration Docs.

OIDC Roles Claim (Microsoft Entra ID)

RustFS supports mapping an OIDC claim containing role values into the existing authorization pipeline. The roles_claim setting is optional: when unset or empty, only the groups claim contributes to authorization (same as older RustFS releases). For Microsoft Entra ID app roles, set roles_claim=roles so both console admin checks and bucket IAM policies can evaluate those roles.

Example environment configuration (opt-in roles claim):

RUSTFS_IDENTITY_OPENID_ENABLE=on
RUSTFS_IDENTITY_OPENID_CONFIG_URL="https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration"
RUSTFS_IDENTITY_OPENID_CLIENT_ID="<client-id>"
RUSTFS_IDENTITY_OPENID_CLIENT_SECRET="<client-secret>"
RUSTFS_IDENTITY_OPENID_SCOPES="openid,profile,email"
RUSTFS_IDENTITY_OPENID_GROUPS_CLAIM="groups"
RUSTFS_IDENTITY_OPENID_ROLES_CLAIM="roles"

Policy condition example (evaluate app roles directly with jwt:roles; when roles_claim is configured, RustFS also merges those values into jwt:groups for backward compatibility with older policies):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["admin:*"],
      "Resource": ["arn:aws:s3:::*"],
      "Condition": {
        "ForAnyValue:StringEquals": {
          "jwt:roles": ["RustFS.ConsoleAdmin"]
        }
      }
    }
  ]
}

Documentation

For detailed documentation, including configuration options, API references, and advanced usage, please visit our Documentation.

Getting Help

If you have any questions or need assistance:

  • Check the FAQ for common issues and solutions.
  • Join our GitHub Discussions to ask questions and share your experiences.
  • Open an issue on our GitHub Issues page for bug reports or feature requests.

Contact

Contributors

RustFS is a community-driven project, and we appreciate all contributions. Check out the Contributors page to see the amazing people who have helped make RustFS better.

RustFS contributors

Star History

RustFS star history chart

License

Apache 2.0

RustFS is a trademark of RustFS, Inc. All other trademarks are the property of their respective owners.

S
Description
2.3x faster than MinIO for 4KB object payloads. RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.
Readme Apache-2.0 157 MiB
Languages
Rust 94.3%
Shell 3.1%
Python 2.5%