Commit Graph

16 Commits

Author SHA1 Message Date
Zhengchao An 027456032f fix(ecstore): enforce the NAME_MAX segment budget on the write path too (#5826)
#5804 added the on-disk segment budget to check_bucket_and_object_names, but PUT validates through check_put_object_args, which has its own checks and never calls it. An over-NAME_MAX key therefore still reached the disk layer and came back to the client as ENAMETOOLONG → InternalError 500, exactly the behavior #5785 reported.

Caught by re-running the acceptance suite against the locked build 4b2d79f5d, which contains #5804: S3-003 still failed with a 512-byte key.

Multipart is unaffected — check_new_multipart_args and check_multipart_object_args both route through check_object_args → check_bucket_and_object_names, which already carries the budget.

Verification: new test pins the same boundaries on check_put_object_args (255 ok / 256 rejected, byte-based via CJK, multi-segment long keys ok, __XLDIR__ budget for directory keys); cargo test -p rustfs-ecstore --lib -- bucket::utils 17 passed; cargo clippy -p rustfs-ecstore --all-targets clean; make pre-commit green.
2026-08-08 05:30:15 +08:00
Zhengchao An 766afe12fb fix(ecstore): reject over-NAME_MAX key segments up front; classify irreconcilable parity as corrupt metadata (#5804)
fix(ecstore): reject over-NAME_MAX object key segments up front and classify irreconcilable parity as corrupt metadata

Two defects found during release acceptance and the backlog#1776 investigation:

Object keys with any path segment longer than 255 bytes could never be stored (each segment maps to one on-disk directory entry), but the failure surfaced only when the disk layer hit ENAMETOOLONG, which leaked to clients as InternalError 500 (rustfs#5785). Validate the on-disk segment budget in check_bucket_and_object_names so such keys fail deterministically as ObjectNameInvalid (4xx) before any I/O. Directory-object keys (trailing '/') account for the __XLDIR__ suffix their final segment carries on disk.

object_quorum_from_meta conflated two very different no-quorum situations (rustfs#5801): stray or foreign metadata whose parity values are garbage produced the same retryable-looking ErasureReadQuorum (503) as a genuine partial outage, so clients retried unrecoverable reads and monitoring could not tell corruption from capacity loss. Now (a) parity counts outside [0, total_shards] are treated as invalid entries instead of being clamped to i32::MAX, which could poison common_parity's occurrence counting, and (b) when a full read quorum of disks answers but their parity values cannot be reconciled, the error is FileCorrupt — heal-actionable and non-retryable — while too-few-healthy-replies keeps returning ErasureReadQuorum.

Verification: 4 new unit tests (segment budget boundaries incl. byte-vs-char and __XLDIR__ budget; garbage parity sanitization; corrupt-vs-quorum classification), metadata::tests + utils::tests 62/62, set_disk+bucket suites 1214 passed with the single pre-existing heal_queue_marks_missing_versioning_state_as_missed cross-test flake also failing on a clean tree (not introduced here), clippy clean, make pre-commit green.
2026-08-07 22:13:54 +08:00
cxymds fa235e9018 fix(s3): list multipart uploads by bucket prefix (#5195)
* fix(s3): list multipart uploads by bucket prefix

* fix(s3): preserve exact-key crash reclamation
2026-07-24 21:28:06 +08:00
Zhengchao An e1fc4b12ea fix(api): descriptive InvalidArgument reason for Windows-unsupported object keys (#4947)
fix(api): return descriptive InvalidArgument reason for Windows-unsupported object keys

On Windows hosts object keys containing NTFS-reserved characters or
Win32-unaddressable path segments are rejected up front, but the client
only saw a bare "Invalid argument" (issue #3299). Attach an explicit
reason to these rejections and surface non-empty InvalidArgument reasons
through the S3 error message.
2026-07-17 19:00:52 +08:00
Zhengchao An c59d4e6f89 fix(ecstore): reject Windows-unreadable object name segments (#4225) 2026-07-03 12:21:04 +08:00
唐小鸭 eff656e086 fix(storage): restore legacy SSE-S3 read compatibility (#3584)
* Update .gitignore

* Fix. fixed SSE-S3 compatibility issues in large-scale testing

* fix

* fix(ecstore): reject whitespace bucket names

* Update replication_extension_test.rs

* style(ecstore): format bucket whitespace test

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <cxymds@gmail.com>
2026-06-23 21:35:17 +08:00
安正超 0cf7e5cf03 refactor: extract embedded startup control helpers (#3661) 2026-06-20 21:57:03 +08:00
安正超 7b0cb9e725 refactor: prune storage compatibility re-export allowances (#3579)
* refactor: prune storage compatibility re-export allowances

* fix: reject whitespace-padded dot path segments
2026-06-18 21:14:24 +08:00
houseme 87a3d107d6 test(property): add path validator invariants (#3541)
test(property): add path validator invariants for #3525
2026-06-18 02:14:24 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
Brayan Jules 7ce23c6b54 fix(ecstore): allow trailing slash in object names to match S3 behavior (#1996)
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-27 22:46:42 +08:00
houseme d251b9fb35 fix: unify path handling to use S3-standard forward slashes on all platforms (#1555)
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: LeonWang0735 <wlywly0735@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-26 18:49:21 +08:00
houseme f795299d53 Optimization and collation of dependencies introduction processing (#1493) 2026-01-13 15:02:54 +08:00
houseme 760cb1d734 Fix Windows Path Separator Handling in rustfs_utils (#1464)
Co-authored-by: reatang <tangtang1251@qq.com>
2026-01-11 19:53:51 +08:00
weisd ab752458ce Fix Path Traversal and Enhance Object Validation (#1387) 2026-01-05 15:57:15 +08:00
houseme 5826396cd0 refactor: Restructure project layout and clean up dependencies (#30)
This commit introduces a significant reorganization of the project structure to improve maintainability and clarity.

Key changes include:
- Adjusted the directory layout for a more logical module organization.
- Removed unused crate dependencies, reducing the overall project size and potentially speeding up build times.
- Updated import paths and configuration files to reflect the structural changes.
2025-07-02 19:33:12 +08:00