Commit Graph

4 Commits

Author SHA1 Message Date
houseme d2b1003612 perf(storage): converge Wave 2 hot-path optimizations (#6065)
* perf(get): share inline shards and lock clients

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): converge PUT encoding on contiguous blocks

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(get): cache codec streaming gate config

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(sse): redact projected customer headers

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): collapse GET metadata snapshots

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(ecstore): reuse decode stripe scratch

Co-Authored-By: heihutu <heihutu@gmail.com>

* refactor(ecstore): trim decode scratch adapters

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ecstore): adapt transition checks to metadata snapshots

Co-Authored-By: heihutu <heihutu@gmail.com>

* perf(get): release metadata snapshots at ownership boundary

Co-Authored-By: heihutu <heihutu@gmail.com>

* refactor(ecstore): close cumulative fast-path findings

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(storage): preserve lock and header invariants

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(ecstore): adapt cumulative paths after rebase

Co-Authored-By: heihutu <heihutu@gmail.com>

* fix(rio-v2): adapt generated metadata fixture

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
2026-08-13 16:34:28 +08:00
Zhengchao An 60d8e8a20b refactor(kms): consolidate encryption metadata key constants into their shared home (#5995)
The shared module rustfs_utils::http::object_encryption_keys is the single source of truth for encryption metadata key names, but three call sites still carried their own copies or bare literals: crates/kms/src/service.rs (two private constants plus four bare x-rustfs-encryption-* literals on both the write and read path), rustfs/src/app/select_object.rs (six SELECT_* copies), and rustfs/src/storage/options.rs (two private prefix copies now imported from header_compat). All values are unchanged, so the change is a compiler-verified rename.

The reader-only x-rustfs-internal-server-side-encryption- family gets a named constant with the verified judgment recorded on it: no writer emits these keys anywhere in the repo (the SSE writer persists the MinIO-branded keys verbatim for interop), the two comments claiming the dual-key invariant writes this twin were wrong and are corrected, and the defensive redaction/strip readers are kept because removing them is risk-asymmetric.

rustfs-kms's rustfs-utils dependency now declares the http feature it uses instead of relying on feature unification from sibling crates.

Refs rustfs/backlog#1775, rustfs/backlog#1562.
2026-08-12 16:37:38 +00:00
唐小鸭 6333f21a2e feat(replication): SSE-C ciphertext passthrough replication (#5898)
Complete the encrypted-object replication series (backlog#1783, PR-C of
3, after #5872 and #5885): SSE-C objects replicate as ciphertext
passthrough — the source holds no customer key, so the stored bytes and
their encryption metadata travel verbatim and the replica decrypts only
with the original customer key, single-part and multipart.

- Sender: SSE-C objects read raw (raw_data_movement_read), transfer at
  ciphertext size, and range multipart parts over stored part sizes.
- Receiver: authorized replication PUTs restore the stored SSE-C keys
  from the transport headers (exact lowercase forms - the read-path
  check is case-sensitive), set ObjectOptions.preserve_ciphertext, and
  skip compression, bucket-default SSE, and sse_encryption behind one
  restore-derived gate. Multipart uses an internal session marker to
  store parts verbatim and strips it on complete.
- Convergence: the replication HEAD sends
  x-rustfs-source-replication-check; the target authorizes it as
  ReplicateObjectAction and skips SSE-C read validation for that
  request only, so keyless convergence HEADs see etag/size/mtime
  instead of 400 and SSE-C replicas stop re-driving forever.
- e2e: SSE-C contract flips to a key-gated readable replica (no-key and
  wrong-key GETs fail - the direct silent-plaintext detector); new
  multipart passthrough contract with ETag/marker/stability assertions.
2026-08-09 23:53:04 +08:00
唐小鸭 10c7476883 fix(replication): rebuild SSE metadata boundary for encrypted objects (#5872)
Groundwork for encrypted-object replication (backlog#1783, PR-A of 3):

- classify_replication_source_encryption: accept the AES256 marker that
  every stored SSE-C object carries; the SseC arm was unreachable.
- Fail closed on sealed material without an SSE marker (MinIO-written
  objects) instead of replicating ciphertext as plaintext.
- Replace the dead VALID_SSE_REPLICATION_HEADERS table with a transport
  map keyed by the metadata keys the SSE writer actually persists, shared
  via the new rustfs_utils::http::object_encryption_keys module.
- Structurally strip all encryption metadata from outbound replication
  (x-rustfs-encryption-* envelopes previously passed the filters).
- Skip decrypt_checksums for encrypted objects at the boundary so its
  is_multipart=false (a response-path contract) cannot misroute
  encrypted multipart objects once managed replication opens.
- Redact X-Rustfs-Replication-* SSE transport values in FileInfo Debug.

A reconciliation test pins that every key encryption_material_to_metadata
produces is either transport-mapped or stripped. All four SSE replication
e2e contracts still assert FAILED unchanged.
2026-08-09 03:05:11 +00:00