Commit Graph

71 Commits

Author SHA1 Message Date
Zhengchao An 92f83bfe15 Merge commit from fork
* fix(policy): quantify negated string conditions per value

ForAllValues:/ForAnyValue: negated string operators computed the positive
quantified match and then negated the aggregate. That yields NOT(all match)
and NOT(any match), which is the semantics of the *other* quantifier, so
ForAllValues:StringNotEquals and ForAnyValue:StringNotEquals were exactly
transposed. The same applied to StringNotEqualsIgnoreCase, StringNotLike,
ArnNotEquals and ArnNotLike.

Introduce an explicit Quantifier and push negation into the per-value
predicate for the qualified forms, so ForAllValues requires every request
value to satisfy the operator and ForAnyValue requires at least one.
Unqualified operators keep negating the aggregate, preserving AWS
single-valued-key semantics. Absent keys now follow AWS: ForAllValues is
vacuously satisfied, ForAnyValue is not.

A request value set that is fully contained in or fully disjoint from the
policy set cannot distinguish the two quantifiers, which is why the existing
cases missed this; the new tests use partially overlapping sets.

* fix(auth): keep request headers out of server-derived condition keys

get_condition_values folded every remaining request header into the policy
condition map. HeaderMap lowercases header names, and the server-derived keys
userid, username, principaltype, versionid and signatureversion are lowercase
too, so a header of the same name collided with them. The collision branch used
extend(), and non-quantified string operators match if ANY value in the vector
matches, so sending `userid: admin` was enough to satisfy a condition on
aws:userid. The jwt:/ldap: claim keys were reachable the same way whenever the
credential carried no such claim.

groups was worse than an append: the header loop ran before the cred.groups
block, which is gated on !args.contains_key("groups"), so a `groups:` header
both injected a value and suppressed the credential's real group list.

Resolve claims and group membership before merging headers, then skip any header
naming a key the server already derived or a well-known identity/context key.
The reserved set comes from KeyName so it tracks the key registry; s3:x-amz-*
keys stay mergeable because they mirror request headers by design.

* fix(access): gate the ListBucketVersions fallback on public-access checks

An anonymous request for ListObjectVersions that the bucket policy does not
grant directly falls back to re-checking the grant as s3:ListBucket. That
fallback returned Ok(()) straight away, skipping the two gates the direct grant
passes through: deny_anonymous_table_data_plane_if_needed and the
RestrictPublicBuckets check on the bucket's public-access block.

So a bucket whose policy allows anonymous s3:ListBucket kept serving anonymous
version listings after an operator enabled RestrictPublicBuckets, even though
the equivalent GetObject was correctly denied.

Fold the fallback into policy_allowed so both routes reach the same gates.

* fix(ftps): authorize MKD against the CreateBucket boundary

FTPS MKD creates a bucket but ran no authorization check, so any principal that
could open an FTPS session could create buckets regardless of policy. Every
other operation in this driver authorizes first — LIST, RETR, STOR, DELE and
RMD all call authorize_operation — and the WebDAV gateway checks
S3Action::CreateBucket on the equivalent path.

Add the matching check so MKD clears the same boundary as an S3 CreateBucket.
2026-07-26 06:14:01 +08:00
Henry Guo 906805568b feat(table-catalog): add durable backing state transfer (#4952)
* feat(table-catalog): add durable backing state transfer

* fix(table-catalog): reject orphaned migration entries

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
2026-07-18 10:45:32 +08:00
Zhengchao An 4ff7775ecb fix(admin): map service account validation errors (#4932) 2026-07-17 08:31:17 +08:00
Zhengchao An d5687e1693 fix(policy): compare NotResource in Statement equality (#4454)
Statement equality drives merge/dedup of policy statements. Omitting NotResource let semantically-distinct statements be treated as duplicates and dropped, which can shrink Deny coverage and escalate privileges. Compare not_resources too and add regression tests.

Refs rustfs/backlog#1028
2026-07-08 17:02:20 +08:00
Zhengchao An 1acd47f15e fix: SSE crash-loop DoS + credential reserved-char bypass (backlog#806) (#4404) 2026-07-08 10:05:51 +08:00
Zhengchao An 3f624bf06c refactor: centralize ecstore bucket runtime sources (#3810) 2026-06-24 09:41:11 +08:00
houseme e42c6df0e8 fix(runtime): remove high-impact unwrap paths (#3755)
* fix(runtime): remove high-impact unwrap paths

* fix(runtime): propagate managed SSE metadata errors

* fix(runtime): add typed OPA config errors

* fix(runtime): harden config and credential helpers

* fix(runtime): remove SSE hmac unwraps

* fix(runtime): complete SSE helper error propagation

* fix(trusted-proxies): avoid legacy global init panics

* test(credentials): allow deprecated rpc token check

* fix(storage): harden object lock retention parsing

* chore(checks): refresh layer dependency baseline

* chore(checks): refresh layer dependency baseline

* Update layer-dependency-baseline.txt

Signed-off-by: houseme <housemecn@gmail.com>

* test(context): avoid clone on copy boot time

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-06-23 15:12:47 +08:00
安正超 b1c6578df1 refactor: narrow test harness compatibility surfaces (#3592) 2026-06-19 07:10:52 +08:00
houseme d2a135b397 fix(admin): require admin auth for metrics (#3562)
fix: require admin auth for metrics
2026-06-18 12:39:16 +08:00
GatewayJ 3928117c8f fix(policy): preserve IAM policy readback shape (#3431) 2026-06-14 16:36:28 +08:00
Henry Guo 49a893f010 feat(table-catalog): add credential endpoint boundary (#3407)
* feat(table-catalog): add credential scope boundary

* feat(table-catalog): add credential endpoint boundary

* fix(table-catalog): remove redundant scope clone

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-13 21:26:59 +08:00
GatewayJ 42d9d5247d fix(policy): normalize IAM policy readback (#3402) 2026-06-13 18:47:08 +08:00
Henry Guo 2eafd9c602 feat(table-catalog): add credential response boundary (#3305)
* feat(table-catalog): add credential response boundary

* fix(table-catalog): enforce scoped catalog resources

* fix(table-catalog): reduce admin auth scope arguments

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-10 01:15:51 +00:00
安正超 4fec606dc4 feat: add KMS action taxonomy (#3294) 2026-06-09 05:06:15 +00:00
Henry Guo 8c3e52efb8 feat(table-catalog): refine table catalog permissions (#3283)
* feat(table-catalog): refine table catalog permissions

* fix(policy): scope table admin resources

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-08 14:23:52 +00:00
安正超 1838922f07 fix(security): add deny_unknown_fields to deserialization structs (#3198)
* fix(security): add deny_unknown_fields to deserialization structs

Prevent silent acceptance of malformed or adversarial payloads.

- policy: Policy, BucketPolicy, Statement, BPStatement, PrincipalObject
- notify: S3KeyFilter custom deserializer rejects unknown child elements
- update: VersionInfo (remote HTTP response)

26 policy + 82 notify tests pass.

* fix(update): keep version response forward compatible

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-03 15:33:28 +00:00
Henry Guo e0a03ce10b feat(policy): add table catalog admin actions (#3200)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-03 15:25:26 +00:00
overtrue 95836a0a4d fix: reject Snowball extract path traversal 2026-05-28 08:32:13 +08:00
安正超 095337100e test(policy): validate default policies (#2985) 2026-05-17 01:41:03 +00:00
GatewayJ e0729f5f4d fix(policy): align action-family validation and defaults (#2984)
* fix(policy): align action-family validation and defaults

* test(e2e): add accountinfo service-account roundtrip

* test(policy): add mixed action family cases
2026-05-16 11:19:04 +00:00
Henry Guo 4b36667ba1 fix(policy): avoid logging generated access keys (#2826)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-06 12:56:27 +00:00
安正超 b747e9817e fix(policy): preserve gateway ListBucket resources (#2710) 2026-04-28 14:09:42 +00:00
cxymds c698a0f2b6 fix(policy): allow AssumeRole in system policies (#2718) 2026-04-28 11:01:46 +00:00
安正超 159ddd5bac fix: honor bucket-scoped ListBucket policies with s3:prefix (#2707)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-27 14:13:22 +00:00
Henry Guo c717195de2 fix(admin): harden STS and KMS authorization checks (#2653)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-25 01:34:18 +00:00
cxymds 94f64acc87 fix(site-replication): sync IAM and bucket replication (#2671) 2026-04-25 01:33:43 +00:00
Ramakrishna Chilaka fefb308b35 feat(policy): implement BinaryEquals condition evaluation (#2626)
Signed-off-by: Ramakrishna Chilaka <49393831+RamakrishnaChilaka@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-24 07:08:11 +00:00
GatewayJ 16b9189e9b feat(oidc): add roles_claim and jwt:roles policy support (#2509)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-15 09:30:24 +08:00
安正超 67863630b2 fix(auth): reject ambiguous case-insensitive claim matches (#2386) 2026-04-04 08:36:14 +08:00
Logan Ye 2d91e2f580 fix(oidc): support case-insensitive claim name matching (#2362)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-03 21:45:56 +08:00
GatewayJ 3366bd2464 feat(iam,admin): prepared IAM auth, ExistingObjectTag, admin permission checks (#2315)
Signed-off-by: GatewayJ <835269233@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: GatewayJ <8352692332qq.com>
2026-03-29 19:18:16 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
LeonWang0735 7f1cdaedad feat(replication): add bandwidth-aware reporting for bucket replication metrics (#2141) 2026-03-15 09:03:10 +08:00
安正超 b2e8078971 fix(policy): avoid unicode panic in variable resolver (#2115) 2026-03-11 21:56:32 +08:00
安正超 b73059dcf2 fix(admin): allow non-consoleAdmin self password update (#2082) 2026-03-05 15:47:21 +08:00
安正超 7a83b818b8 fix(policy): address review feedback from #2018 (#2021)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-01 11:05:20 +08:00
安正超 7eb136faf0 feat(policy): add Service principal, ArnLike/IfExists conditions, and logging error ordering (#2018) 2026-03-01 08:44:42 +08:00
安正超 e7466eb1cc fix: policy StringNotEquals double negation and delete_objects version mapping (#2015) 2026-03-01 03:13:52 +08:00
安正超 af6c32efac refactor: improve code quality with safer error handling, trait decomposition, and dead code cleanup (#1997) 2026-02-28 01:19:47 +08:00
GatewayJ 62b51b5649 feat: admin permission check (#1783)
Signed-off-by: GatewayJ <835269233@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-25 11:58:30 +08:00
dependabot[bot] 045988e062 build(deps): bump the dependencies group with 15 updates (#1912)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-23 17:57:53 +08:00
安正超 4211652991 refactor(app): add application layer module entry (#1907) 2026-02-22 22:15:37 +08:00
Miguel Caballer Fernandez 7ae0415715 Increase ACCESS_KEY_MAX_LEN to 128 (#1870)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-18 22:00:16 +08:00
yxrxy da58f8e291 fix: Allow non-admin users to read bucket quota configuration. (#1759)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-10 11:23:21 +08:00
GatewayJ c8411fd62c fix: bucket policy id field serde (#1726) 2026-02-05 18:38:57 +08:00
GatewayJ e3a7eb2d3d fix: standart policy format (#1508) 2026-01-15 15:33:22 +08:00
GatewayJ 356dc7e0c2 feat: Add permission verification for account creation (#1401)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-06 21:47:18 +08:00
安正超 e7a3129be4 feat: s3 tests classification (#1392)
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-05 22:24:35 +08:00
LeonWang0735 f86761fae9 fix:allow NotResource-only policies in statement validation (#1364)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-05 13:07:42 +08:00
GatewayJ f5f6ea4a5c feat:policy Resources support string and array modes. (#1346)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-04 19:21:37 +08:00