Commit Graph

10 Commits

Author SHA1 Message Date
Zhengchao An 2bf2ce1ff2 feat(admin): apply the requested key listing filters (#5608) 2026-08-02 14:03:57 +08:00
Zhengchao An da6fc5314d docs(kms): correct the static backend's MinIO compatibility claim (#5596)
* docs(kms): correct the static backend's MinIO compatibility claim

`StaticConfig` claimed the backend derives DEKs via "HMAC-SHA256 +
AES-256-GCM, matching the MinIO builtin/static KMS wire format". Neither
half holds: the configured key is used directly as the AES-256-GCM key
with no derivation step, and wrapped DEKs are serialized as RustFS's own
`DataKeyEnvelope` JSON. MinIO's KMS ciphertext uses a different shape,
which `is_data_key_envelope` explicitly classifies as foreign (see the
`minio_legacy` case in encryption/dek.rs).

The claim as written tells a migrating operator that MinIO-written
ciphertext will open here, which it will not. Restate what the backend
actually does and point at rustfs/backlog#1638 for the real interop work.

Also refresh the neighbouring ciphertext-format note in static_kms.rs,
which still described a raw `ciphertext || nonce` layout that the JSON
envelope replaced.

* ci(minio-interop): fix the dead test selector and guard against empty runs

The job selected its tests with `-p rustfs-ecstore -E
'binary(minio_generated_read_test)'`. #5435 moved those reader tests from
crates/ecstore/tests/minio_generated_read_test.rs into the `rustfs` crate
as a `#[cfg(test)] mod`, which removed that test binary; the selector has
selected zero interop tests since. Point it at the tests where they now
live, verified locally:

  cargo nextest list --run-ignored all -p rustfs --features rio-v2 \
    -E 'test(minio_generated_read_test::)'   # 4 tests, was 0

Add a guard step in front of the run. The old `binary(...)` form happened
to fail loudly once its binary disappeared, but the name-based form that
replaces it is a valid filterset even when it matches nothing, so a later
rename would silently reduce this job to a pass that asserts nothing. The
guard counts the selection and fails with an explicit reason; the count
comes from `filter-match.status`, since the JSON's top-level `test-count`
is the package total and ignores `-E`. `--no-tests=fail` on the run step
covers the same case if the guard is ever dropped.

Also record in the header what this job does and does not prove: MinIO
wrapped-DEK envelopes are still rejected by both envelope parsers, and
that work is tracked in rustfs/backlog#1638.
2026-08-02 05:31:30 +08:00
Zhengchao An a12043f49f fix(kms): page key listings so the deletion sweep sees every key (#5595)
* fix(kms): page the Local key listing so the deletion sweep sees every key

The Local backend answered every ListKeys with the first `limit` entries of
`read_dir` and a hardcoded `truncated: false`, so the deletion sweep ended
after one page: on a deployment with more keys than a page, expired key
material past the first page was never destroyed, and the lifecycle gauges
published that partial page as if it were the whole key set.

Listing now orders the key set by identifier and pages through it, with the
marker as an exclusive lower bound on the identifier rather than an index, so
a key added or removed between pages — including the marker key, which the
sweep itself destroys — cannot make the listing skip keys or restart. Only the
page is read from disk, so a list costs the requested limit rather than the
size of the key set.

The pagination arithmetic lives in a shared helper so the other self-paging
backends can adopt the same semantics, and the sweep now stops instead of
re-listing when a backend hands back the cursor it was just given.

* fix(kms): give ListKeys a defined zero-limit and cursor contract

`GET /rustfs/admin/v3/kms/keys?limit=0` reached the Vault KV2 and Vault
Transit backends as a page size of zero, where the page arithmetic indexed the
element before an empty page and aborted the request. Both backends also
resolved the marker by searching for it in the key list, so a marker naming a
key that had since been removed silently restarted the listing from the
beginning instead of resuming after it.

All four self-paging backends now share one contract: a zero limit is answered
as an empty, non-truncated page without reaching the backend at all, and the
marker is an exclusive lower bound on the key identifier rather than a position
in the list. The Vault backends read metadata only for the page they return,
so a list costs the requested limit instead of the whole key set, and KV2 now
applies the usage and status filters it previously accepted and ignored.
2026-08-02 05:27:46 +08:00
Zhengchao An 3c00ad6048 fix(kms): make the deletion waiting window non-bypassable (#5535) 2026-08-01 12:23:28 +00:00
Zhengchao An fd36bdfb1a feat(kms): allow key description and tag updates (#5546) 2026-08-01 06:32:47 +00:00
Zhengchao An 76b3c085b5 refactor(kms): fold the KmsClient layer into KmsBackend and complete lifecycle overrides (#5501) 2026-07-31 08:31:21 +00:00
Zhengchao An 342ee1df78 feat(kms): add backend capability discovery (#5485) 2026-07-31 06:09:43 +08:00
Zhengchao An 35a20622f1 feat(kms): add master key version to data key envelope contract (#5480)
* fix(kms): restore vault backend test compilation after timeout parameter

PR #5472 added an attempt_timeout parameter to VaultKmsClient::new while
PR #5474 landed tests still using the one-argument form, leaving
'cargo test -p rustfs-kms' unable to compile on main. Pass the same
30-second timeout the surrounding integration tests already use.

* feat(kms): add master key version to data key envelope contract

DataKeyEnvelope gains an optional master_key_version field recording
which KEK version wrapped the DEK, so rotation-aware backends can load
the matching historical material on decrypt. The field is skipped when
None, keeping envelopes from non-rotating backends byte-identical to
the historical seven-field JSON shape, and legacy envelopes without the
field deserialize to None. The envelope discriminator marker is
untouched, so mixed-format routing is unchanged in both directions.

Adds the KeyVersionNotFound typed error for version-addressed material
lookups that must fail closed instead of falling back to the current
version.

Refs rustfs/backlog#1565
2026-07-30 16:43:10 +00:00
唐小鸭 2216f00cfd fix(kms): unify persisted SSE data key envelopes (#5343)
* feat(kms): implement secure handling of static KMS secret keys and enhance encryption context validation

* feat: enhance local SSE DEK handling with JSON envelope format and versioning
2026-07-28 17:02:18 +08:00
唐小鸭 99e1f5fbd2 feat(kms): add static single-key backend (#5222) 2026-07-26 04:12:56 +08:00