mirror of
https://github.com/rustfs/rustfs.git
synced 2026-09-05 19:55:37 +00:00
唐小鸭
8bf569899a
* fix(ecstore): persist merged checksum type for full-object multipart complete_multipart_upload built the object-level checksum record from a ChecksumType copied before the MULTIPART / INCLUDES_MULTIPART flags were merged in. ChecksumType::merge takes &mut self, so the merge updated the local variable while the copy already inside the Checksum struct stayed behind. The composite branch rebuilt the Checksum from the merged type and was unaffected; the full-object branch never rebuilt it, so those flags never reached disk. rustfs_rio::read_checksums only sets its multipart flag and only emits the "x-amz-checksum-type" = "FULL_OBJECT" entry inside its MULTIPART branch, so a full-object multipart object read back as non-multipart with no type entry, and GetObject and HeadObject answered with no x-amz-checksum-type header at all where AWS returns FULL_OBJECT. Hand the full-object branch the merged type instead of rebuilding the Checksum: the value must stay the running merge produced by add_part, because hashing the concatenated part digests would yield the COMPOSITE value, a different number than the one the client sent. The serialization now lives in multipart_object_checksum_record so both shapes are covered by unit tests. Records written by earlier builds carry the bare algorithm type with no MULTIPART flags and no trailing part block; they keep reading back to the same checksum value, and the FULL_OBJECT reader arm predates this change so older peers parse the new record shape correctly too. Found while root-causing rustfs#6825. * fix(s3): reject contradicting multipart checksum type as client error A CompleteMultipartUpload declaring an x-amz-checksum-type that contradicts the type recorded at CreateMultipartUpload answered 500 InternalError, telling the caller to retry a request that can only ever fail. The storage layer does refuse the combination, but through a generic error that maps to InternalError. Validate the header against the recorded type in the usecase, where the upload metadata returned by get_multipart_info is already in hand, and answer InvalidRequest naming both types, matching AWS. The storage-layer check stays as a backstop for non-HTTP callers. Uploads created without a checksum algorithm record no type, so there is nothing to contradict and the header is left alone rather than newly rejected. Replication is unaffected: replication_put_object_options already excludes x-amz-checksum-type from the metadata it forwards. * test(e2e): cover full-object multipart checksum type round-trip Adds an end-to-end test that a CRC32 FULL_OBJECT multipart upload reports x-amz-checksum-type: FULL_OBJECT and the unsuffixed full-object value on both GetObject and HeadObject, and one that a CompleteMultipartUpload contradicting the recorded type is rejected as InvalidRequest while leaving the upload intact. Extends the existing CRC64NVME multipart test with the same checksum-type assertion. * fix(s3): keep checksum-type validation off the s3s error macro The s3s footprint ratchet (scripts/check_s3s_footprint.sh) counts s3_error! invocation lines and is lower-only: new code must route through the gateway abstractions rather than widen the direct s3s surface the s3gate migration is shrinking. Raise the contradiction through ApiError::invalid_request instead. The response is byte-for-byte identical -- From<ApiError> for S3Error carries the InvalidRequest code and the message through unchanged -- and the usecase already returns ApiError elsewhere, so this is the idiomatic path rather than a way around the counter. The explanatory comment deliberately says "the s3s error macro" instead of naming the macro: the ratchet counts raw matches, so spelling it out in a comment tripped the same check.
RustFS is a high-performance distributed object storage software built using Rust
Getting Started · Docs · Bug reports · Discussions
English | Simplified Chinese
RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. Along with MinIO, it shares a range of advantages such as simplicity, broad S3 API compatibility for supported features, open-source nature, support for data lakes, AI, and big data. Furthermore, it has a better and more user-friendly open-source license in comparison to other storage systems, being constructed under the Apache license. As Rust serves as its foundation, RustFS provides faster speed and safer distributed features for high-performance object storage.
Features
- High Performance: Built with Rust, ensuring speed and efficiency.
- Distributed Architecture: Scalable and fault-tolerant design for large-scale deployments.
- S3 Compatibility: Integration with common S3-compatible applications; current coverage is tracked in the S3 compatibility matrix.
- Data Lake Support: Optimized for big data and AI workloads.
- Open Source: Licensed under Apache 2.0, encouraging community contributions and transparency.
- User-Friendly: Designed with simplicity in mind, making it easy to deploy and manage.
RustFS vs MinIO
Stress test server parameters
| Type | parameter | Remark |
|---|---|---|
| CPU | 2 Core | Intel Xeon(Sapphire Rapids) Platinum 8475B , 2.7/3.2 GHz |
| Memory | 4GB | |
| Network | 15Gbp | |
| Driver | 40GB x 4 | IOPS 3800 / Driver |
https://github.com/user-attachments/assets/2e4979b5-260c-4f2c-ac12-c87fd558072a
RustFS vs Other object storage
| RustFS | Other object storage |
|---|---|
| Powerful Console | Simple and useless Console |
| Developed based on Rust language, memory is safer | Developed in Go or C, with potential issues like memory GC/leaks |
| Does not report logs to third-party countries | Reporting logs to other third countries may violate national security laws |
| Licensed under Apache, more business-friendly | AGPL V3 License and other License, polluted open source and License traps, infringement of intellectual property rights |
| S3-compatible core, with coverage tracked in the compatibility matrix | Variable S3 support and local cloud vendor coverage |
| Rust-based development, strong support for secure and innovative devices | Poor support for edge gateways and secure innovative devices |
| Stable commercial prices, free community support | High pricing, with costs up to $250,000 for 1PiB |
| No risk | Intellectual property risks and risks of prohibited uses |
Quickstart
To get started with RustFS, follow these steps:
-
One-click installation script (Option 1)
curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh -
Docker Quick Start (Option 2)
# Docker Hub (recommended)
docker run -d -p 9000:9000 -v /data:/data rustfs/rustfs:latest
# Alternative using Podman
podman run -d -p 9000:9000 -v /data:/data rustfs/rustfs:latest
- Access the Console: Open your web browser and navigate to
http://localhost:9001to access the RustFS console, default username and password isrustfsadmin. - Create a Bucket: Use the console to create a new bucket for your objects.
- Upload Objects: You can upload files directly through the console or use S3-compatible APIs to interact with your RustFS instance.
Documentation
For detailed documentation, including configuration options, API references, and advanced usage, please visit our Documentation.
Getting Help
If you have any questions or need assistance, you can:
- Check the FAQ for common issues and solutions.
- Join our GitHub Discussions to ask questions and share your experiences.
- Open an issue on our GitHub Issues page for bug reports or feature requests.
Links
- Documentation - The manual you should read
- Changelog - What we broke and fixed
- GitHub Discussions - Where the community lives
Contact
- Bugs: GitHub Issues
- Business: hello@rustfs.com
- Jobs: jobs@rustfs.com
- General Discussion: GitHub Discussions
- Contributing: CONTRIBUTING.md
Contributors
RustFS is a community-driven project, and we appreciate all contributions. Check out the Contributors page to see the amazing people who have helped make RustFS better.
License
RustFS is a trademark of RustFS, Inc. All other trademarks are the property of their respective owners.