* feat(rio): wire XXHash3/64/128 and SHA-512 into ChecksumType (S2) Add the AWS 2026-04 additional checksum algorithms as base types in rustfs-rio's ChecksumType, covering every dispatch site (key, raw_byte_len, hasher, Display, from_string_with_obj_type, BASE_CHECKSUM_TYPES) so no path silently strips them. Derive BASE_TYPE_MASK from BASE_CHECKSUM_TYPES as the single source of truth, allocate the new base-type bits append-only above bit 9 to preserve the on-disk varint format, and add streaming hashers whose digest uses the S3 canonical big-endian encoding (seed 0). The new algorithms are COMPOSITE-only: an explicit FULL_OBJECT request is rejected and they are never routed through add_part()/can_merge(). A round-trip guardrail test asserts every base type survives all dispatch sites, failing loudly if a future algorithm is added but a match arm or the mask is forgotten. Refs rustfs/backlog#1254 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * test(rio): pin XXHash/SHA-512 digests to official vectors, big-endian (S3) Lock the byte order and seed of the new algorithms against the OFFICIAL upstream xxHash / SHA-512 empty-input test vectors (XXH3-64, XXH64, XXH3-128, SHA-512), in big-endian, so the stored and echoed checksum is byte-for-byte identical to what AWS SDKs (awscrt) compute — the interop correctness this feature hinges on. Add a non-empty regression lock (official "fox" vectors) that also asserts the encoded field is the standard-base64 of the raw digest. Refs rustfs/backlog#1255 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * test(rio): lock on-disk checksum round-trip and forward-compat degrade (S8) Cover the xl.meta varint (de)serialization for the new algorithms: to_bytes() -> read_checksums() must recover the value under the Display key for XXHASH3/64/128 and SHA512. Pin the rolling-upgrade contract that a node reading a future, unknown base-type bit degrades safely — skips the entry and returns without panicking or mis-decoding a length. Combined with the append-only bit allocation from S2, this protects mixed-version clusters. Refs rustfs/backlog#1260 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * feat(head): echo XXHash/SHA-512 additional checksums on HeadObject (S5) HeadObject with x-amz-checksum-mode: ENABLED now returns the XXHash3/64/128 and SHA-512 checksums that S3 stored, closing the head_object gap in #4800. s3s HeadObjectOutput has no typed field for these, so they are emitted as raw response headers via response.headers (the same mechanism RustFS already uses for tagging-count), keyed by ChecksumType::key(). The existing five typed algorithms are unchanged. Also carries the Cargo.lock update for the xxhash-rust dependency introduced in S2. Refs rustfs/backlog#1257 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * fix(checksums): fail-closed on unknown checksum algorithm (S7) A. Harden unknown/unsupported checksum algorithms to fail closed instead of panicking. ChecksumMode::base() in the outbound S3 client (crates/ecstore/src/client/checksum.rs) previously did `panic!("enum err.")` for any mode without a concrete base algorithm (e.g. a bare ChecksumFullObject flag); it now falls back to ChecksumNone. Added unit tests proving base() never panics and hasher() returns Err for unsupported modes. rustfs-checksums FromStr already returns Err on unknown names; added a regression test asserting garbage/unknown names fail closed. B. Extend rustfs-checksums ChecksumAlgorithm with the AWS 2026-04 additional algorithms Sha512/Xxhash3/Xxhash64/Xxhash128. Updated FromStr, as_str, into_impl, name constants, the x-amz-checksum-* header constants and the HttpChecksum impls. Byte order/seed matches the server-side rustfs-rio spec: xxh3/xxh64 as u64 big-endian (8 bytes, seed 0), xxh128 as u128 big-endian (16 bytes), sha512 via sha2::Sha512. Added tests validating each digest against a direct library computation. MD5 stays intentionally rejected (PR #4513) and is left untouched. C. crates/ecstore/src/client/checksum.rs ChecksumMode is enumset repr="u8" with 7 variants already consuming 7 bits; adding the 4 new algorithms would overflow u8 and require a breaking repr change, so ChecksumMode is left unchanged. The new algorithms are available through the rustfs-checksums ChecksumAlgorithm path. Refs rustfs/backlog#1259 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * feat(get,put): echo XXHash/SHA-512 checksums on GetObject and PutObject (S5-GET, S4) Complete the additional-checksum round-trip so AWS SDKs can verify integrity on download and confirm it on upload: - GetObject with x-amz-checksum-mode: ENABLED now returns XXHash3/64/128 and SHA-512 checksums (the download-side path SDKs auto-verify). The values flow from build_get_object_checksums through GetObjectOutputContext into finalize_get_object_response and are emitted after wrap_response_with_cors. - PutObject echoes the server-computed additional checksum on its response, captured at the want_checksum set points before opts is moved. Both reuse a single centralized helper, inject_additional_checksum_headers, which HeadObject now also uses. This is the ONLY place that emits these headers, so when s3s gains typed fields for these algorithms the migration is one spot (fill the typed field, drop the insert) with no risk of duplicate headers. The five s3s-typed algorithms are unchanged. Trailing-checksum PUT echo (value lands after the body) is left for e2e coverage in S10. Refs rustfs/backlog#1257 rustfs/backlog#1256 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * feat(multipart): support XXHash/SHA-512 composite multipart checksums (S9) Make multipart uploads work end-to-end for the composite-only algorithms (XXHash3/64/128, SHA-512): - complete_part_checksum previously returned the outer None for any algorithm outside the five typed ones, which failed CompleteMultipartUpload with InvalidPart. It now accepts any valid base type with no double-check value (Some(None)) — mirroring the missing-value path of the typed algorithms — since s3s CompletePart has no field to carry a client-supplied per-part value and the part was already verified server-side at UploadPart. Genuinely unset/invalid types are still rejected. - The existing COMPOSITE assembly (Checksum::new_from_data over the concatenated per-part raw digests; full_object_requested() is false so add_part() is correctly bypassed) already works for these algorithms via the S2 wiring. A rio test locks the assembly and that add_part refuses them. - UploadPart and CompleteMultipartUpload echo the new-algorithm checksum on their responses via the shared inject_additional_checksum_headers helper (now pub(crate)), since s3s has no typed output field. Refs rustfs/backlog#1261 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * feat(rio): add MD5 as an additional checksum (x-amz-checksum-md5) (S6) Wire MD5 into ChecksumType as an additional (flexible) checksum, distinct from the legacy Content-MD5 / ETag path: header x-amz-checksum-md5, 16-byte digest, COMPOSITE-only, md-5 hasher. Pinned to the official empty-input MD5 vector. Thanks to the single-source-of-truth wiring from S2, every dispatch site (GetObject/HeadObject/PutObject echo, multipart complete_part_checksum and the COMPOSITE assembly) picks MD5 up automatically via base()/key()/the catch-all arm — no handler changes needed. Tests are extended to cover MD5 across them. Coordination with #4513: that PR made the OUTBOUND rustfs-checksums client reject "md5" so it could never silently fall back to CRC32. This change is on the server-side rio path and never falls back — it implements MD5 correctly rather than substituting another algorithm — so the #4513 intent is preserved, and the outbound client keeps rejecting md5 (S7). Refs rustfs/backlog#1258 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * perf(rio): drop per-request to_uppercase alloc in checksum parsing (S11) from_string_with_obj_type ran alg.to_uppercase() on every checksummed request, allocating a String just to compare against a fixed set of algorithm names. Replace it with eq_ignore_ascii_case, which is allocation-free and, for the ASCII algorithm names involved, exactly equivalent. A test locks that case-insensitivity, the CRC64NVME full-object assumption, composite-only FULL_OBJECT rejection, and unknown/empty handling are all unchanged. The other S11 notes are intentionally not acted on: the Phase-0 header scan is N/A (we chose full support over rejection, so there is no reject guard), and parallelizing the serialized hash passes is deferred pending a measured need. Refs rustfs/backlog#1263 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * refactor(checksums): collapse 5 duplicated response-checksum loops into one Review of the accumulated commits found the same "iterate decrypted checksums, match five typed algorithms, drop the rest" loop copy-pasted across five response paths (GetObject, HeadObject, GetObjectAttributes object-level and part-level, CompleteMultipartUpload). That was patch-on-patch duplication. Collapse it into a single source of truth: - rustfs-rio gains ChecksumType::is_s3s_typed() — the one place that defines the five-typed vs additional-algorithm split. - object_usecase gains ResponseChecksums + classify_response_checksums(), which performs the typed/extra split once. All five call sites now destructure its result; additional_checksum_echo_pairs() also uses is_s3s_typed() instead of a hand-rolled five-way comparison. Behaviour is unchanged (GetObjectAttributes still cannot surface the additional algorithms — an s3s XML-body limitation, now documented in one spot). One pass over the map; extra pairs pushed only when a new-algorithm checksum is present. Refs rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * test(checksums): unit tests for classifier/echo helpers + fix unused import Add direct unit tests for the refactored single-source-of-truth helpers: - rio ChecksumType::is_s3s_typed() — exhaustive typed-vs-additional split, and that flags (FULL_OBJECT/MULTIPART) on a base type don't change classification. - object_usecase classify_response_checksums() — typed fields vs `extra` headers, the checksum-type marker, and empty input. - additional_checksum_echo_pairs() — echo pair only for additional algorithms, none for the five typed ones, none for None. - inject_additional_checksum_headers() — writes all pairs; empty is a no-op. Also drop the now-unused AMZ_CHECKSUM_TYPE import in multipart_usecase.rs left by the classifier refactor (would fail the -D warnings gate). Refs rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * style(rio): fix typo flagged by CI (mis-decoding -> decoding a wrong length) The Typos CI check flagged "mis-decoding" (it reads "mis" as a word). Reword the S8 forward-compat comment; no code change. Refs rustfs/backlog#1260 Co-Authored-By: heihutu <heihutu@gmail.com> * test(e2e): integration test for XXHash/SHA-512/MD5 additional checksums (S10) Permanent verify-on-write integration test in the e2e suite for the AWS 2026-04 additional algorithms. aws_sdk_s3 has no typed builder for these, so the x-amz-checksum-<algo> header is injected via mutate_request (value from rustfs-rio, byte-for-byte identical to awscrt). Uses a client with automatic checksum calculation disabled (request_checksum_calculation=WhenRequired) so the injected header is the only checksum on the wire. For each of XXHash3/64/128, SHA-512 and MD5: a correct value is accepted and the object stored intact; a mismatched value is rejected with BadDigest and nothing is stored. Verified passing locally (1 passed) alongside a boto3+awscrt round-trip that additionally confirms the HEAD/GET header echo (14/14). Refs rustfs/backlog#1262 rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> * style(get): allow too_many_arguments on finalize_get_object_response The classifier refactor added an extra_checksum_headers parameter, pushing finalize_get_object_response to 8 args and tripping clippy::too_many_arguments under CI's `-D warnings`. Add the same #[allow] the sibling GET helpers already carry; no behavior change. Refs rustfs/backlog#1252 Co-Authored-By: heihutu <heihutu@gmail.com> --------- Co-authored-by: heihutu <heihutu@gmail.com>
RustFS is a high-performance, distributed object storage system built in Rust.
Getting Started · Docs · Bug reports · Discussions
English | 简体中文 | Deutsch | Español | français | 日本語 | 한국어 | Portuguese | Русский
RustFS is a high-performance, distributed object storage system built in Rust—one of the most loved programming languages worldwide. RustFS combines the simplicity of MinIO with the memory safety and raw performance of Rust. It offers broad S3 API compatibility for supported features, is completely open-source, and is optimized for data lakes, AI, and big data workloads.
Unlike other storage systems, RustFS is released under the permissible Apache 2.0 license, avoiding the restrictions of AGPL. With Rust as its foundation, RustFS delivers superior speed and secure distributed features for next-generation object storage.
Feature & Status
- High Performance: Built with Rust to ensure maximum speed and resource efficiency.
- Distributed Architecture: Scalable and fault-tolerant design suitable for large-scale deployments.
- S3 Compatibility: Seamless integration with common S3-compatible applications and tools; current coverage is tracked in the S3 compatibility matrix.
- OpenStack Swift API: Native support for Swift protocol with Keystone authentication.
- OpenStack Keystone Integration: Native support for OpenStack Keystone authentication with X-Auth-Token headers.
- Data Lake Support: Optimized for high-throughput big data and AI workloads.
- Open Source: Licensed under Apache 2.0, encouraging unrestricted community contributions and commercial usage.
- User-Friendly: Designed with simplicity in mind for easy deployment and management.
| Feature | Status | Feature | Status |
|---|---|---|---|
| S3 Core Features | ✅ Available | Bitrot Protection | ✅ Available |
| Upload / Download | ✅ Available | Single Node Mode | ✅ Available |
| Versioning | ✅ Available | Bucket Replication | ✅ Available |
| Logging | ✅ Available | Lifecycle Management | 🚧 Under Testing |
| Event Notifications | ✅ Available | Distributed Mode | 🚧 Under Testing |
| K8s Helm Charts | ✅ Available | RustFS KMS | 🚧 Under Testing |
| Keystone Auth | ✅ Available | Multi-Tenancy | ✅ Available |
| Swift API | ✅ Available | Swift Metadata Ops | 🚧 Partial |
RustFS vs MinIO Performance
Stress Test Environment:
| Type | Parameter | Remark |
|---|---|---|
| CPU | 2 Core | Intel Xeon (Sapphire Rapids) Platinum 8475B, 2.7/3.2 GHz |
| Memory | 4GB | |
| Network | 15Gbps | |
| Drive | 40GB x 4 | IOPS 3800 / Drive |
https://github.com/user-attachments/assets/2e4979b5-260c-4f2c-ac12-c87fd558072a
RustFS vs Other Object Storage
| Feature | RustFS | Other Object Storage |
|---|---|---|
| Console Experience | Powerful Console Comprehensive management interface. |
Basic / Limited Console Often overly simple or lacking critical features. |
| Language & Safety | Rust-based Memory safety by design. |
Go or C-based Potential for memory GC pauses or leaks. |
| Data Sovereignty | No Telemetry / Full Compliance Guards against unauthorized cross-border data egress. Compliant with GDPR (EU/UK), CCPA (US), and APPI (Japan). |
Potential Risk Possible legal exposure and unwanted data telemetry. |
| Licensing | Permissive Apache 2.0 Business-friendly, no "poison pill" clauses. |
Restrictive AGPL v3 Risk of license traps and intellectual property pollution. |
| Compatibility | S3-Compatible Core Works with common S3-compatible clients, with coverage tracked in the compatibility matrix. |
Variable Compatibility May lack support for local cloud vendors or specific APIs. |
| Edge & IoT | Strong Edge Support Ideal for secure, innovative edge devices. |
Weak Edge Support Often too heavy for edge gateways. |
| Risk Profile | Enterprise Risk Mitigation Clear IP rights and safe for commercial use. |
Legal Risks Intellectual property ambiguity and usage restrictions. |
Staying ahead
Star RustFS on GitHub and be instantly notified of new releases.
Quickstart
To get started with RustFS, follow these steps:
1. One-click Installation (Option 1)
curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh
2. Docker Quick Start (Option 2)
The RustFS container runs as a non-root user rustfs (UID/GID 10001:10001). If you bind-mount host directories with Docker or Compose, every mounted path must be writable by that user, otherwise startup may fail with permission denied errors. This applies to data directories, log directories, and TLS certificate directories when RUSTFS_TLS_PATH is enabled.
# Create data and logs directories
mkdir -p data logs
# Change the owner of these directories
chown -R 10001:10001 data logs
# Using latest version
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
# Using specific version
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:1.0.0-beta.8
If you use podman instead of docker, you can install the RustFS with the below command
# Create data and logs directories
mkdir -p data logs
# Run the container (podman will automatically set the folders ownership)
podman run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data:Z,U -v $(pwd)/logs:/logs:Z,U rustfs/rustfs:latest
If you enable TLS with a bind-mounted certificate directory, prepare that mount the same way:
mkdir -p certs
chown -R 10001:10001 certs
You can also use Docker Compose. Using the docker-compose-simple.yml file in the root directory:
docker compose -f docker-compose-simple.yml up -d
Before running Compose with host bind mounts:
- Ensure every mounted host path is writable by
10001:10001. - If you enable TLS, ensure the certificate mount for
/opt/tlsis also readable by10001:10001. - If matching host ownership is not practical, run the
rustfsservice withuser: "<host-uid>:<host-gid>"instead. docker-compose-simple.ymlincludes avolume-permission-helperservice for named volumes.docker-compose-simple.ymlrelies on you to prepare bind-mounted host paths in advance.
Similarly, you can run the command with podman
podman compose -f docker-compose-simple.yml up -d
Webhook notification quick start (Docker):
docker run -d --name rustfs -p 9000:9000 \
-e RUSTFS_NOTIFY_ENABLE=true \
-e RUSTFS_NOTIFY_WEBHOOK_ENABLE_PRIMARY=on \
-e RUSTFS_NOTIFY_WEBHOOK_ENDPOINT_PRIMARY=http://<host-ip>:3020/webhook \
-e RUSTFS_NOTIFY_WEBHOOK_QUEUE_DIR_PRIMARY=/tmp/rustfs-events \
rustfs/rustfs:latest
Notes:
RUSTFS_NOTIFY_ENABLE=trueenables the global notify module switch.- For ARN
arn:rustfs:sqs::primary:webhook, use instance-scoped env vars with_PRIMARY. - If queue dir is omitted, default is
/opt/rustfs/events; ensure it is writable by the container runtime user. RUSTFS_NOTIFY_WEBHOOK_SKIP_TLS_VERIFY_PRIMARYdefaults tofalse; enabling it skips webhook TLS certificate verification, allows MITM attacks, and emits a startup warning. PreferRUSTFS_NOTIFY_WEBHOOK_CLIENT_CA_PRIMARYfor private CAs.
NOTE: We recommend reviewing the docker-compose.yml file before running. It defines several services including Grafana, Prometheus, and Jaeger, which are helpful for RustFS observability. If you wish to start Redis or Nginx containers, you can specify the corresponding profiles.
3. Build from Source (Option 3) - Advanced Users
For developers who want to build RustFS Docker images from source with multi-architecture support:
# Build multi-architecture images locally
./docker-buildx.sh --build-arg RELEASE=latest
# Build and push to registry
./docker-buildx.sh --push
# Build specific version
./docker-buildx.sh --release v1.0.0 --push
# Build for custom registry
./docker-buildx.sh --registry your-registry.com --namespace yourname --push
The docker-buildx.sh script supports:
- Multi-architecture builds:
linux/amd64,linux/arm64 - Automatic version detection: Uses git tags or commit hashes
- Registry flexibility: Supports Docker Hub, GitHub Container Registry, etc.
- Build optimization: Includes caching and parallel builds
You can also use Make targets for convenience:
make docker-buildx # Build locally
make docker-buildx-push # Build and push
make docker-buildx-version VERSION=v1.0.0 # Build specific version
make help-docker # Show all Docker-related commands
Heads-up (macOS cross-compilation): macOS keeps the default
ulimit -nat 256, socargo zigbuildor./build-rustfs.sh --platform ...may fail withProcessFdQuotaExceededwhen targeting Linux. The build script attempts to raise the limit automatically, but if you still see the warning, runulimit -n 4096(or higher) in your shell before building.
4. Build with Helm Chart (Option 4) - Cloud Native
Follow the instructions in the Helm Chart README to install RustFS on a Kubernetes cluster.
For scanner pacing, cycle budgets, bitrot cadence, lifecycle transition status, and single-node single-disk idle CPU tuning, see Scanner Runtime Controls. For repeatable scanner-pressure validation, see Scanner Benchmark Runbook.
5. Nix Flake (Option 5)
If you have Nix with flakes enabled:
# Run directly without installing
nix run github:rustfs/rustfs
# Build the binary
nix build github:rustfs/rustfs
./result/bin/rustfs --help
# Or from a local checkout
nix build
nix run
6. X-CMD (Option 6)
If you are an x-cmd user:
# Run directly without installing
x rustfs
# Download the binary and install it to the global environment
x env use rustfs
rustfs --help
Accessing RustFS
- Access the Console: Open your web browser and navigate to
http://localhost:9001to access the RustFS console.- Default credentials:
rustfsadmin/rustfsadmin
- Default credentials:
- Create a Bucket: Use the console to create a new bucket for your objects.
- Upload Objects: You can upload files directly through the console or use S3-compatible APIs/clients to interact with your RustFS instance.
NOTE: To access the RustFS instance via https, please refer to the TLS Configuration Docs.
OIDC Roles Claim (Microsoft Entra ID)
RustFS supports mapping an OIDC claim containing role values into the existing
authorization pipeline. The roles_claim setting is optional: when unset or
empty, only the groups claim contributes to authorization (same as older
RustFS releases). For Microsoft Entra ID app roles, set roles_claim=roles so
both console admin checks and bucket IAM policies can evaluate those roles.
Example environment configuration (opt-in roles claim):
RUSTFS_IDENTITY_OPENID_ENABLE=on
RUSTFS_IDENTITY_OPENID_CONFIG_URL="https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration"
RUSTFS_IDENTITY_OPENID_CLIENT_ID="<client-id>"
RUSTFS_IDENTITY_OPENID_CLIENT_SECRET="<client-secret>"
RUSTFS_IDENTITY_OPENID_SCOPES="openid,profile,email"
RUSTFS_IDENTITY_OPENID_GROUPS_CLAIM="groups"
RUSTFS_IDENTITY_OPENID_ROLES_CLAIM="roles"
Policy condition example (evaluate app roles directly with jwt:roles; when
roles_claim is configured, RustFS also merges those values into jwt:groups
for backward compatibility with older policies):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["admin:*"],
"Resource": ["arn:aws:s3:::*"],
"Condition": {
"ForAnyValue:StringEquals": {
"jwt:roles": ["RustFS.ConsoleAdmin"]
}
}
}
]
}
Documentation
For detailed documentation, including configuration options, API references, and advanced usage, please visit our Documentation.
Getting Help
If you have any questions or need assistance:
- Check the FAQ for common issues and solutions.
- Join our GitHub Discussions to ask questions and share your experiences.
- Open an issue on our GitHub Issues page for bug reports or feature requests.
Links
- Documentation - The manual you should read
- Changelog - What we broke and fixed
- GitHub Discussions - Where the community lives
- Discord - Chat with the RustFS community
Contact
- Bugs: GitHub Issues
- Business: hello@rustfs.com
- Jobs: jobs@rustfs.com
- General Discussion: GitHub Discussions
- Contributing: CONTRIBUTING.md
Contributors
RustFS is a community-driven project, and we appreciate all contributions. Check out the Contributors page to see the amazing people who have helped make RustFS better.
Star History
License
RustFS is a trademark of RustFS, Inc. All other trademarks are the property of their respective owners.