Commit Graph

4775 Commits

Author SHA1 Message Date
houseme ce41adfa9b test(ilm): cover manual transition e2e gaps (#5219)
* test(ilm): cover manual transition queue pressure

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(protos): serialize msgpack-only env tests

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-25 07:00:24 +00:00
cxymds 59361ed786 feat(site-replication): make repair operations durable (#5217)
* feat(site-replication): make repair operations durable

* fix(site-replication): serialize durable repair execution
2026-07-25 10:35:37 +08:00
houseme dfb0a20048 test(rpc): add corrupt-json decode coverage (#5216)
* test(rpc): add corrupt-json decode coverage

* test(ilm): assert manual transition status-less contract
2026-07-25 01:03:59 +00:00
cxymds 7320d7fab2 fix(replication): make resync starts atomic (#5215) 2026-07-25 08:58:06 +08:00
Zhengchao An 28d19db9fc fix(console): handle apple touch icon paths (#5214) 2026-07-25 08:06:48 +08:00
dependabot[bot] e257573962 chore(deps): bump quinn-proto from 0.11.14 to 0.11.16 in /fuzz in the cargo group across 1 directory (#5210)
chore(deps): bump quinn-proto

Bumps the cargo group with 1 update in the /fuzz directory: [quinn-proto](https://github.com/quinn-rs/quinn).


Updates `quinn-proto` from 0.11.14 to 0.11.16
- [Release notes](https://github.com/quinn-rs/quinn/releases)
- [Commits](https://github.com/quinn-rs/quinn/compare/quinn-proto-0.11.14...quinn-proto-0.11.16)

---
updated-dependencies:
- dependency-name: quinn-proto
  dependency-version: 0.11.16
  dependency-type: indirect
  dependency-group: cargo
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-25 08:05:02 +08:00
cxymds 45b675c641 fix(replication): report authoritative backlog metrics (#5209) 2026-07-25 01:32:50 +08:00
cxymds 05caec0bd5 feat(replication): structure active check results (#5208) 2026-07-25 01:28:03 +08:00
cxymds eaa17e0441 feat(admin): add encrypted diagnostic archives (#5207) 2026-07-25 01:19:47 +08:00
cxymds 2b6cc0ee08 fix(admin): redact and seal config history (#5206) 2026-07-24 23:54:24 +08:00
cxymds 938f7296f9 fix(admin): report truthful diagnostic capabilities (#5205) 2026-07-24 23:33:45 +08:00
cxymds 866ac5073d fix(kms): validate configured backend cleanup (#5204)
test(kms): validate configured backend cleanup
2026-07-24 23:24:40 +08:00
houseme 187a060919 test(internode): pin msgpack gate rollback coverage (#5203)
Add test coverage for the msgpack-only request/fleet confirmation truth table and exact request JSON policy manifest.

Pin request and response rollback behavior so removing either gate restores JSON compatibility while both gates enter msgpack-only for eligible fields.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 15:16:23 +00:00
houseme cda443bd81 feat(ilm): reject overlapping manual transition runs (#5202)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 14:56:42 +00:00
houseme 44b1916103 fix(getobject): enrich stream error context (#5201)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 14:47:56 +00:00
Henry Guo 9d1b10144f perf(ecstore): avoid redundant leaf object scans (#5176)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-07-24 22:25:03 +08:00
cxymds d7f30fe0a2 fix(replication): persist sanitized resync errors (#5200) 2026-07-24 22:15:51 +08:00
houseme 20c4ea864a fix(ecstore): fix wide-prefix listing stalls and decode downstream logging (#5198)
* fix(ecstore): handle list stall and downstream decode logs

* docs(ecstore): keep wide-directory stall context for list_dir
2026-07-24 14:05:27 +00:00
houseme d1c2c42c90 fix(internode): align msgpack benchmark gates (#5197)
Update the internode gRPC benchmark P2 env output to require both the msgpack-only request flag and the fleet-confirmed gate before measuring a true msgpack-only phase.

Document the dual-gate rollback semantics and add a dry-run script test so the benchmark driver cannot silently regress to the single-flag flow.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 13:32:44 +00:00
cxymds fc43b149c5 fix(admin): restore complete config history snapshots (#5196) 2026-07-24 21:28:12 +08:00
cxymds fa235e9018 fix(s3): list multipart uploads by bucket prefix (#5195)
* fix(s3): list multipart uploads by bucket prefix

* fix(s3): preserve exact-key crash reclamation
2026-07-24 21:28:06 +08:00
houseme dd46de0945 fix(heal): report no-parity bitrot as unrecoverable (#5192)
Keep corrupted no-parity heal results on the integrity-failure path, preserve the object geometry in operator output, and document the recovery boundary for historical bad shards.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 20:20:40 +08:00
houseme bf6f0e5e81 test(internode): pin msgpack compatibility send sites (#5193)
test(internode): pin msgpack compat send sites

Add a checked test manifest for internode dual-encoded request and response payloads so node.proto _bin fields must be explicitly classified as msgpack-only eligible or always dual-write.

The manifest also pins the current JSON encoder call sites for request and response send paths, making future send-site drift visible before rollout.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 12:07:11 +00:00
houseme beb807ae2b perf(protos): cache internode msgpack-only flag (#5191)
Cache the internode msgpack-only env gate after the first read so metadata RPC send paths avoid repeated environment parsing. Keep a reset hook for tests that intentionally change the env in-process.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 11:54:14 +00:00
houseme d8426dc459 feat(ilm): gate durable manual transition mode (#5190)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 11:53:19 +00:00
cxymds f46ea6e14f fix(s3): enforce SSE-C copy key validation (#5185) 2026-07-24 19:11:36 +08:00
houseme fa26b6730d test(ilm): deny manual transition for non-admin (#5188)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 11:10:31 +00:00
houseme 7876319811 test(ilm): cover manual transition queue pressure (#5189)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 11:10:04 +00:00
houseme ea417b6a32 feat(ilm): trace manual transition runs (#5187)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 10:58:04 +00:00
houseme 4963412265 fix(internode): guard msgpack-only JSON fallback (#5180)
Keep internode JSON compatibility fields unless operators explicitly confirm fleet-wide msgpack-only readiness. This prevents a single legacy rollout flag from emptying JSON fields in mixed-version clusters where older peers may still read the legacy JSON payload.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 17:57:51 +08:00
houseme 8ac618e6c2 feat(ilm): count already transitioned manual runs (#5177)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 17:52:56 +08:00
GatewayJ 1c88aa43c1 fix(iam): virtualize OIDC service account parents (#5152)
* fix(iam): preserve OIDC service account policy boundary

* fix(iam): virtualize OIDC service account parents

* fix(iam): reject malformed OIDC policy boundaries

* test(iam): isolate federated policy regression

* fix(iam): keep OIDC replication envelope off claims
2026-07-24 17:41:52 +08:00
houseme a5a73610b6 fix(ecstore): self-verify no-parity bitrot writes (#5179)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 17:36:30 +08:00
cxymds 9eaf5fc8e3 fix(s3): complete CopyObject checksum support (#5178) 2026-07-24 16:57:11 +08:00
houseme 3132637294 feat(ilm): bound manual transition duration (#5174)
* feat(ilm): bound manual transition duration

Add maxDurationSeconds handling for manual transition runs so operators can bound long scoped scans without changing the existing enqueue_only job contract.

Co-Authored-By: heihutu <heihutu@gmail.com>

* docs(ilm): document manual transition run limits

Document the enqueue_only manual transition contract, secret-handling guidance, and best-effort duration budget while pinning the new duration flag in the e2e response model.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 07:54:52 +00:00
cxymds 358caa23cb fix(storage): expose truthful storage class capabilities (#5172) 2026-07-24 15:28:03 +08:00
houseme 6765aca3f9 feat(ilm): add manual transition run endpoint (#5171)
* feat(ilm): report manual transition backfill outcomes

Add scoped lifecycle transition backfill reporting for backlog #1478 and expose enqueue outcomes needed by #1479 without changing the existing scanner/compensation bool API.

Co-Authored-By: heihutu <heihutu@gmail.com>

* feat(admin): add manual transition run endpoint

Add a bounded POST /rustfs/admin/v3/ilm/transition/run API for backlog #1477 and cover the route, policy, query parsing, and partial status contract needed by #1481. Console operations from #1480 are intentionally left for a later client integration.

Co-Authored-By: heihutu <heihutu@gmail.com>

* feat(ilm): allow manual transition resume markers

Accept additive marker and versionMarker parameters on the bounded manual transition run API so clients can continue from a partial report without changing the existing default scan behavior.

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(ilm): harden manual transition partial reports

Preserve the null-version cursor contract, stop manual scans on enqueue pressure without skipping the failed object, and keep raw resume markers out of admin JSON responses.

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ilm): add manual transition e2e coverage

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ilm): keep transition enqueue hot path direct

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 15:13:22 +08:00
houseme 4133fbe0fc fix(storage): cover inline reader fallback controls (#5169)
* test(ecstore): cover inline fast path boundaries

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(storage): cover inline reader fallback controls

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(tier): keep commit fanout concurrent

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 06:08:17 +00:00
cxymds 6f6d8a4d3e fix(s3): persist multipart upload storage class (#5170)
Fixes rustfs/backlog#1464.
2026-07-24 13:47:25 +08:00
cxymds cb344a3c77 fix(s3): honor CopyObject replacement metadata (#5168)
Fixes rustfs/backlog#1463.
2026-07-24 12:33:33 +08:00
Zhengchao An 36e97aba26 fix(security): reject federated STS credentials that collide with the root principal (#5161)
A federated OIDC / AssumeRoleWithWebIdentity login derives parent_user from the external identity via session_identity() (username -> email -> sub -> oidc-user-unknown). At IAM request time a temporary credential whose parent_user equals the root access key is treated as owner (see rustfs/src/admin/auth.rs and the rustfs_iam owner resolution used by the policy engine and service-account paths). A federated identity whose display name happens to equal the root access key would therefore be silently granted full owner access on a string match alone.

Reject such issuance up front: immediately after parent_user is derived, and before credential generation, set_temp_user, and the site-replication hook, deny the binding when parent_user equals the root access key. Both federated flows funnel through the single DefaultFederatedSessionBinding::bind -> issue_credentials chokepoint, so this covers browser OIDC callback and AssumeRoleWithWebIdentity alike.

- rustfs/src/admin/service/federated_identity.rs: issue_credentials resolves the root access key via current_action_credentials() and rejects with FederatedSessionBindingError::InvalidRequest on collision. Added the pure helper parent_user_is_reserved(parent_user, root_access_key) so the collision decision is unit-testable without process globals; the comparison is exact and case-sensitive, mirroring the request-time owner comparison.

This is the federated-principal / root isolation pre-work (batch 0A, the immediate issuance-time block) from the OIDC review in rustfs/backlog#1437. Layer 2 (a signature-protected federated credential origin that forces is_owner=false at request time regardless of the parent_user string) is a separate follow-up. Already-issued colliding credentials continue on their existing TTL and should be revoked proactively. Legitimate federated identities are unaffected; only an exact collision with the configured root access key is denied.
2026-07-24 11:32:05 +08:00
cxymds a2fc6e15df fix(s3): honor CopyObject tagging directives (#5165)
Closes rustfs/backlog#1462.
2026-07-24 10:30:43 +08:00
houseme 0269c47bc6 fix(runtime): update codec and debug stack headroom (#5164)
* fix(runtime): raise debug worker stack headroom

Co-Authored-By: heihutu <heihutu@gmail.com>

* chore(deps): update codec to 8.0.2

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-24 09:27:49 +08:00
Zhengchao An d26adc29ca fix(security): pin OIDC discovery/token client to the outbound egress policy (#5159) 2026-07-23 17:13:33 +00:00
houseme 5131ba8271 chore(deps): refresh workspace dependencies (#5157)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-23 15:59:37 +00:00
houseme 14f31b797a feat(bench): capture node attribution metadata (#5158)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-23 15:56:48 +00:00
houseme 9f61bad94f chore(deps): update erasure codec and russh (#5155)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-23 18:13:33 +08:00
Zhengchao An 8e214104f3 perf(ecstore): add guarded encrypted multipart range seeks (#5145)
* perf(ecstore): seek encrypted multipart Range GETs to the covering part boundary on the Legacy rio v1 backend

Phase A of https://github.com/rustfs/backlog/issues/1316. Encrypted Range GETs on the default (non rio-v2) backend previously planned storage_offset=0, storage_length=oi.size and discarded the decrypted prefix, so a small Range on a large SSE object read, erasure-decoded, and decrypted the whole object. Eligible multipart objects now seek to the covering part boundary using only the per-part size/actual_size metadata facts; the multipart decrypt reader already handles streams starting at any part boundary, so rio and the on-disk format are untouched. Single-part objects, compressed payloads, defective parts tables, and zero-length ranges keep the previous full read, and RUSTFS_ENCRYPTED_RANGE_SEEK=false restores it globally. A new histogram rustfs_get_encrypted_range_read_amplification plus a full|part_seek path counter record the physical/plaintext amplification at the ReadPlan decision point.

* fix(ecstore): guard encrypted multipart range seeks

* fix(io-metrics): align encrypted range metric names with the rustfs_io_ prefix

Every other metric in rustfs-io-metrics uses the rustfs_io_ prefix; the
two encrypted-range-seek metrics were the only exception. Rename before
first release so dashboards never see the unprefixed names.
2026-07-23 08:33:54 +00:00
Zhengchao An 6bab9e421b fix(iam): route issuer-relative JWKS through config URL (#5150) 2026-07-23 13:46:51 +08:00
Zhengchao An d9e0a25174 fix(oidc): support separate discovery issuer (#5149) 2026-07-23 02:44:32 +00:00