Commit Graph

37 Commits

Author SHA1 Message Date
houseme 8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00
houseme 2924b4e463 Restore globals and add unified TLS/mTLS loading from RUSTFS_TLS_PATH (#1309)
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2025-12-30 21:55:43 +08:00
Jitter cd6a26bc3a fix(net): resolve 1GB upload hang and macos build (Issue #1001 regression) (#1035) 2025-12-07 18:05:51 +08:00
0xdx2 7c6cbaf837 feat: enhance error handling and add precondition checks for object o… (#1008) 2025-12-06 20:39:03 +08:00
Copilot 601f3456bc Fix large file upload freeze with adaptive buffer sizing (#869)
* Initial plan

* Fix large file upload freeze by increasing StreamReader buffer size

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Add comprehensive documentation for large file upload freeze fix

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* upgrade s3s version

* Fix compilation error: use BufReader instead of non-existent StreamReader::with_capacity

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Update documentation with correct BufReader implementation

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* add tokio feature `io-util`

* Implement adaptive buffer sizing based on file size

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* Constants are managed uniformly and fmt code

* fix

* Fix: Trigger self-heal on read when shards missing from rejoined nodes (#871)

* Initial plan

* Fix: Trigger self-heal when missing shards detected during read

- Added proactive heal detection in get_object_with_fileinfo
- When reading an object, now checks if any shards are missing even if read succeeds
- Sends low-priority heal request to reconstruct missing shards on rejoined nodes
- This fixes the issue where data written during node outage is not healed when node rejoins

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* Unify CRC implementations to crc-fast (#873)

* Initial plan

* Replace CRC libraries with unified crc-fast implementation

Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

* fix

* fix: replace low to Normal

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2025-11-17 23:15:20 +08:00
houseme 7e1a9e2ede 🔒 Upgrade Cryptography Libraries to Latest RC Versions (#837)
* fix

* chore: upgrade cryptography libraries to RC versions

- Upgrade aes-gcm to 0.11.0-rc.2 with rand_core support
- Upgrade chacha20poly1305 to 0.11.0-rc.2
- Upgrade argon2 to 0.6.0-rc.2 with std features
- Upgrade hmac to 0.13.0-rc.3
- Upgrade pbkdf2 to 0.13.0-rc.2
- Upgrade rsa to 0.10.0-rc.10
- Upgrade sha1 and sha2 to 0.11.0-rc.3
- Upgrade md-5 to 0.11.0-rc.3

These upgrades provide enhanced security features and performance
improvements while maintaining backward compatibility with existing
encryption workflows.

* add

* improve code

* fix
2025-11-11 21:10:03 +08:00
安正超 dd47fcf2a8 fix: restore localized samples in tests (#749)
* fix: restore required localized examples

* style: fix formatting issues
2025-10-29 13:16:31 +08:00
weisd 6a59c0a474 fix: multipart upload checksum validation (#712)
* fix multipart upload checksum
2025-10-24 18:23:32 +08:00
weisd a65856bdf4 Fix CRC32C Checksum Implementation and Enhance Authentication System (#678)
* fix: get_condition_values

* fix checksum crc32c

* fix clippy
2025-10-21 21:28:00 +08:00
weisd cd1e244c68 Refactor: Introduce content checksums and improve multipart/object metadata handling (#671)
* feat:  adapt to s3s typed etag support

* refactor: move replication struct to rustfs_filemeta, fix filemeta transition bug

* add head_object checksum, filter object metadata output

* fix multipart checksum

* fix multipart checksum

* add content md5,sha256 check

* fix test

* fix cargo

---------

Co-authored-by: overtrue <anzhengchao@gmail.com>
2025-10-20 23:46:13 +08:00
weisd 90f21a9102 refactor: Reimplement bucket replication system with enhanced architecture (#590)
* feat:refactor replication

* use aws sdk for replication client

* refactor/replication

* merge main

* fix lifecycle test
2025-09-26 14:27:53 +08:00
guojidan ef0dbaaeb5 feat(encryption): add managed encryption support for SSE-S3 and SSE-KMS (#583)
Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-24 02:09:04 -07:00
guojidan 9ddf6a011d feature: support kms && encryt (#573)
* feat(kms): implement key management service with local and vault backends

Signed-off-by: junxiang Mu <1948535941@qq.com>

* feat(kms): enhance security with zeroize for sensitive data and improve key management

Signed-off-by: junxiang Mu <1948535941@qq.com>

* remove Hashi word

Signed-off-by: junxiang Mu <1948535941@qq.com>

* refactor: remove unused request structs from kms handlers

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: junxiang Mu <1948535941@qq.com>
2025-09-22 17:53:05 +08:00
guojidan 9d5ed1acac Feature/scanner performance optimization (#498)
* Refactor: reimplement scanner

Signed-off-by: RustFS Developer <dandan@rustfs.com>

* comment lock

Signed-off-by: junxiang Mu <1948535941@qq.com>

* remove dirty file

Signed-off-by: junxiang Mu <1948535941@qq.com>

* Fix: fix rebase

* fix(scanner): Improve error handling and logging

Signed-off-by: junxiang Mu <1948535941@qq.com>

---------

Signed-off-by: RustFS Developer <dandan@rustfs.com>
Signed-off-by: junxiang Mu <1948535941@qq.com>
Co-authored-by: RustFS Developer <dandan@rustfs.com>
2025-09-08 18:35:45 +08:00
安正超 d552210b59 feat: translate chinese to english (#402)
* Checkpoint before follow-up message

Co-authored-by: anzhengchao <anzhengchao@gmail.com>

* Translate project documentation and comments from Chinese to English

Co-authored-by: anzhengchao <anzhengchao@gmail.com>

* Fix typo: "unparseable" to "unparsable" in version test comment

Co-authored-by: anzhengchao <anzhengchao@gmail.com>

* Refactor compression test code with minor syntax improvements

Co-authored-by: anzhengchao <anzhengchao@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-08-14 00:19:01 +08:00
houseme 2e14b32ccd chore: Add copyright and license headers (#23)
* chore: Add copyright and license headers

This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.

* cargo fmt

* fix

* fmt

* fix clippy
2025-07-02 15:07:47 +08:00
houseme 7bd8431006 cargo fmt 2025-06-27 18:53:51 +08:00
houseme 749537664f fix:Apply suggestions from clippy 1.88 2025-06-27 18:16:29 +08:00
weisd 34ec13e532 change objectinfo metadata use HashMap
add copyobject compresss support

fix listobject size use actual_size
2025-06-23 16:24:11 +08:00
weisd e145586b65 refactor: reorganize RPC modules into unified structure 2025-06-19 17:06:54 +08:00
weisd 5b1b527851 fix bitrot 2025-06-18 18:22:30 +08:00
loverustfs 65ff5faada Merge pull request #480 from rustfs/feat/compress
feat: add object compression support
2025-06-17 20:20:45 +08:00
Nugine 4a786618d4 refactor(rio): HttpReader use StreamReader 2025-06-17 16:22:55 +08:00
weisd fa8ac29e76 optimize the code 2025-06-17 15:59:31 +08:00
weisd c48ebd5149 feat: add compress support 2025-06-17 15:06:40 +08:00
weisd 52342f2f8e feat(grpc): walk_dir http
fix(ecstore): rebalance loop
2025-06-16 10:32:03 +08:00
Nugine 048727f183 feat(rio): reuse http connections 2025-06-15 16:56:05 +08:00
weisd e8a59d7c07 move disk::utils to crates::utils 2025-06-11 15:55:40 +08:00
weisd 2eeb9dbcbc fix cargo test error, delete unnecessary files 2025-06-11 00:35:16 +08:00
overtrue 4fa8cb7ef0 fix(rio): resolve infinite loop in bitrot tests 2025-06-10 23:21:07 +08:00
weisd 754ffd0ff2 update ec share size
update bitrot
2025-06-10 16:41:34 +08:00
weisd f85ef06783 merge main 2025-06-09 15:31:11 +08:00
weisd 91c099e35f add Error test, fix clippy 2025-06-09 11:29:23 +08:00
weisd b51ee48699 update filereader/writer todo 2025-06-06 18:04:51 +08:00
weisd db355bb26b todo 2025-06-06 11:35:27 +08:00
weisd 9384b831ec ecstore update ec/disk/error 2025-06-06 01:13:51 +08:00
weisd 7fe0cc74d2 add rio/filemeta 2025-06-04 14:21:34 +08:00