Commit Graph

81 Commits

Author SHA1 Message Date
houseme b44e82fef1 fix(notify): restore webhook HTTPS target initialization (#5060)
Restore the workspace reqwest default feature stack for RustFS outbound HTTPS clients, while keeping per-crate extra APIs such as json, stream, and multipart explicit. Lazily initialize the notification runtime from admin target access when RUSTFS_NOTIFY_ENABLE=true is already effective, and add regression coverage for HTTPS webhook custom CA handling and target-list visibility.

Fixes #5052.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-20 19:55:07 +08:00
Henry Guo 906805568b feat(table-catalog): add durable backing state transfer (#4952)
* feat(table-catalog): add durable backing state transfer

* fix(table-catalog): reject orphaned migration entries

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
2026-07-18 10:45:32 +08:00
Zhengchao An 4ff7775ecb fix(admin): map service account validation errors (#4932) 2026-07-17 08:31:17 +08:00
houseme 56179210ab chore(deps): simplify dependency features (#4890)
* chore(deps): remove redundant dependency features

Remove manifest feature entries that are implied by other requested features in the same dependency declaration.

Verified that the resolved Cargo feature graph is unchanged after the cleanup.

Co-Authored-By: heihutu <heihutu@gmail.com>

* chore(deps): narrow tokio and reqwest features

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-16 05:20:43 +00:00
houseme f3a7a4b0da chore(deps): localize workspace dependency features (#4888)
Move workspace-level dependency feature lists into the member crates that consume each dependency while keeping required default-features flags at the workspace root.

Also refresh starshard to 2.2.2 via cargo update and cargo upgrade --exclude ratelimit.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-16 03:55:27 +00:00
Zhengchao An a7827ed91b test(policy): add property tests for the policy evaluation algebra (#4840)
crates/policy had zero property tests: the wildcard Action/Resource matching
and Deny-first evaluation in Policy::is_allowed — the algebra authorization
safety rests on — were guarded only by example-based tests.

Add crates/policy/tests/policy_eval_proptest.rs with three generated-input
invariants (pure evaluation, no IO or global state, parallel-safe):

- explicit_deny_anywhere_denies: a Deny matching the request denies it no
  matter how many broad Allow statements co-exist or at which index the Deny
  sits; a built-in sanity assertion first proves the Allows alone would permit,
  so the Deny is what flips the decision.
- wildcard_superset_implies_concrete_match: any request allowed by an
  exact-action/exact-resource policy is also allowed after widening to s3:* on
  bucket/* and to * on arn:aws:s3:::*, checked both on the built grant and on
  random probe requests (implication form).
- empty_policy_denies_everything: a statement-less policy and Policy::default()
  deny every non-owner request.

Statements are built from JSON exactly as production policies arrive via
PutPolicy. Generated names avoid wildcard metacharacters so the only wildcards
in play are the ones the properties introduce deliberately. proptest joins
crates/policy dev-deps following the filemeta/ecstore precedent.

Refs: backlog#1151 (sec-9)
2026-07-15 10:48:01 +00:00
Zhengchao An 776f7ee83f test(security): add bucket-policy x IAM priority conflict matrix (#4825)
The individual policy layers were tested in isolation, but the cross-layer
resolution — which layer wins when IAM and bucket policy disagree — had no
coverage. A priority error there is a data-exposure or data-lockout bug.

Add crates/policy/tests/bucket_iam_authz_matrix.rs, a pure table-driven test
(no globals, no IAM store, no server) that drives the real Policy::is_allowed
and BucketPolicy::is_allowed through a helper modeling the request-layer
orchestration in rustfs::storage::access::authorize_request (bucket explicit-Deny
gate -> IAM allow -> bucket allow fallback; anonymous = bucket policy alone). It
pins all four Allow/Deny quadrants plus the anonymous case, and adds intra-policy
Deny-beats-Allow checks for both layers.

The matrix surfaces the resolution invariant explicitly: a BUCKET explicit Deny
is a hard gate and always wins, but an IAM explicit Deny is SOFT — a bucket Allow
fallback overrides it, which diverges from AWS "an explicit Deny in any policy
always wins". The test characterizes the current (MinIO-lineage) behavior; if IAM
Deny is later hardened into a gate, iam_deny_x_bucket_allow flips to false and
must be updated deliberately.

Refs: backlog#1151 (sec-8)

Co-authored-by: houseme <housemecn@gmail.com>
2026-07-15 04:06:26 +00:00
Zhengchao An d5687e1693 fix(policy): compare NotResource in Statement equality (#4454)
Statement equality drives merge/dedup of policy statements. Omitting NotResource let semantically-distinct statements be treated as duplicates and dropped, which can shrink Deny coverage and escalate privileges. Compare not_resources too and add regression tests.

Refs rustfs/backlog#1028
2026-07-08 17:02:20 +08:00
Zhengchao An 1acd47f15e fix: SSE crash-loop DoS + credential reserved-char bypass (backlog#806) (#4404) 2026-07-08 10:05:51 +08:00
Zhengchao An 3f624bf06c refactor: centralize ecstore bucket runtime sources (#3810) 2026-06-24 09:41:11 +08:00
houseme e42c6df0e8 fix(runtime): remove high-impact unwrap paths (#3755)
* fix(runtime): remove high-impact unwrap paths

* fix(runtime): propagate managed SSE metadata errors

* fix(runtime): add typed OPA config errors

* fix(runtime): harden config and credential helpers

* fix(runtime): remove SSE hmac unwraps

* fix(runtime): complete SSE helper error propagation

* fix(trusted-proxies): avoid legacy global init panics

* test(credentials): allow deprecated rpc token check

* fix(storage): harden object lock retention parsing

* chore(checks): refresh layer dependency baseline

* chore(checks): refresh layer dependency baseline

* Update layer-dependency-baseline.txt

Signed-off-by: houseme <housemecn@gmail.com>

* test(context): avoid clone on copy boot time

---------

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Zhengchao An <anzhengchao@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-06-23 15:12:47 +08:00
安正超 b1c6578df1 refactor: narrow test harness compatibility surfaces (#3592) 2026-06-19 07:10:52 +08:00
houseme d2a135b397 fix(admin): require admin auth for metrics (#3562)
fix: require admin auth for metrics
2026-06-18 12:39:16 +08:00
GatewayJ 3928117c8f fix(policy): preserve IAM policy readback shape (#3431) 2026-06-14 16:36:28 +08:00
Henry Guo 49a893f010 feat(table-catalog): add credential endpoint boundary (#3407)
* feat(table-catalog): add credential scope boundary

* feat(table-catalog): add credential endpoint boundary

* fix(table-catalog): remove redundant scope clone

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-13 21:26:59 +08:00
GatewayJ 42d9d5247d fix(policy): normalize IAM policy readback (#3402) 2026-06-13 18:47:08 +08:00
Henry Guo 2eafd9c602 feat(table-catalog): add credential response boundary (#3305)
* feat(table-catalog): add credential response boundary

* fix(table-catalog): enforce scoped catalog resources

* fix(table-catalog): reduce admin auth scope arguments

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-10 01:15:51 +00:00
安正超 4fec606dc4 feat: add KMS action taxonomy (#3294) 2026-06-09 05:06:15 +00:00
Henry Guo 8c3e52efb8 feat(table-catalog): refine table catalog permissions (#3283)
* feat(table-catalog): refine table catalog permissions

* fix(policy): scope table admin resources

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-06-08 14:23:52 +00:00
安正超 1838922f07 fix(security): add deny_unknown_fields to deserialization structs (#3198)
* fix(security): add deny_unknown_fields to deserialization structs

Prevent silent acceptance of malformed or adversarial payloads.

- policy: Policy, BucketPolicy, Statement, BPStatement, PrincipalObject
- notify: S3KeyFilter custom deserializer rejects unknown child elements
- update: VersionInfo (remote HTTP response)

26 policy + 82 notify tests pass.

* fix(update): keep version response forward compatible

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-03 15:33:28 +00:00
Henry Guo e0a03ce10b feat(policy): add table catalog admin actions (#3200)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-03 15:25:26 +00:00
overtrue 95836a0a4d fix: reject Snowball extract path traversal 2026-05-28 08:32:13 +08:00
安正超 095337100e test(policy): validate default policies (#2985) 2026-05-17 01:41:03 +00:00
GatewayJ e0729f5f4d fix(policy): align action-family validation and defaults (#2984)
* fix(policy): align action-family validation and defaults

* test(e2e): add accountinfo service-account roundtrip

* test(policy): add mixed action family cases
2026-05-16 11:19:04 +00:00
Henry Guo 4b36667ba1 fix(policy): avoid logging generated access keys (#2826)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-06 12:56:27 +00:00
安正超 b747e9817e fix(policy): preserve gateway ListBucket resources (#2710) 2026-04-28 14:09:42 +00:00
cxymds c698a0f2b6 fix(policy): allow AssumeRole in system policies (#2718) 2026-04-28 11:01:46 +00:00
安正超 159ddd5bac fix: honor bucket-scoped ListBucket policies with s3:prefix (#2707)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-27 14:13:22 +00:00
Henry Guo c717195de2 fix(admin): harden STS and KMS authorization checks (#2653)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-25 01:34:18 +00:00
cxymds 94f64acc87 fix(site-replication): sync IAM and bucket replication (#2671) 2026-04-25 01:33:43 +00:00
Ramakrishna Chilaka fefb308b35 feat(policy): implement BinaryEquals condition evaluation (#2626)
Signed-off-by: Ramakrishna Chilaka <49393831+RamakrishnaChilaka@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-24 07:08:11 +00:00
GatewayJ 16b9189e9b feat(oidc): add roles_claim and jwt:roles policy support (#2509)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-15 09:30:24 +08:00
安正超 67863630b2 fix(auth): reject ambiguous case-insensitive claim matches (#2386) 2026-04-04 08:36:14 +08:00
Logan Ye 2d91e2f580 fix(oidc): support case-insensitive claim name matching (#2362)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-03 21:45:56 +08:00
GatewayJ 3366bd2464 feat(iam,admin): prepared IAM auth, ExistingObjectTag, admin permission checks (#2315)
Signed-off-by: GatewayJ <835269233@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: GatewayJ <8352692332qq.com>
2026-03-29 19:18:16 +08:00
heihutu 3c28f0a0ba feat(metrics): migrate system monitoring from rustfs-obs to rustfs-metrics (#2242)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-20 18:52:33 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
LeonWang0735 7f1cdaedad feat(replication): add bandwidth-aware reporting for bucket replication metrics (#2141) 2026-03-15 09:03:10 +08:00
安正超 b2e8078971 fix(policy): avoid unicode panic in variable resolver (#2115) 2026-03-11 21:56:32 +08:00
安正超 8c4735ff88 docs: scope AGENTS instructions by directory (#2083) 2026-03-05 17:25:37 +08:00
安正超 b73059dcf2 fix(admin): allow non-consoleAdmin self password update (#2082) 2026-03-05 15:47:21 +08:00
安正超 7a83b818b8 fix(policy): address review feedback from #2018 (#2021)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-01 11:05:20 +08:00
安正超 7eb136faf0 feat(policy): add Service principal, ArnLike/IfExists conditions, and logging error ordering (#2018) 2026-03-01 08:44:42 +08:00
安正超 e7466eb1cc fix: policy StringNotEquals double negation and delete_objects version mapping (#2015) 2026-03-01 03:13:52 +08:00
安正超 af6c32efac refactor: improve code quality with safer error handling, trait decomposition, and dead code cleanup (#1997) 2026-02-28 01:19:47 +08:00
GatewayJ 62b51b5649 feat: admin permission check (#1783)
Signed-off-by: GatewayJ <835269233@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-25 11:58:30 +08:00
dependabot[bot] 045988e062 build(deps): bump the dependencies group with 15 updates (#1912)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-23 17:57:53 +08:00
安正超 4211652991 refactor(app): add application layer module entry (#1907) 2026-02-22 22:15:37 +08:00
Miguel Caballer Fernandez 7ae0415715 Increase ACCESS_KEY_MAX_LEN to 128 (#1870)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-18 22:00:16 +08:00
yxrxy da58f8e291 fix: Allow non-admin users to read bucket quota configuration. (#1759)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-10 11:23:21 +08:00