Commit Graph

9 Commits

Author SHA1 Message Date
Zhengchao An 56a7e3b707 chore(s3-types): guard the EventName mask bit budget (#5582)
EventName::mask() turns a leaf variant's discriminant `v` into
`1 << (v - 1)`, so the enum can hold at most 64 variants in total —
compound "All" variants included, since they consume discriminants
even though they own no bit and push every later leaf further up the
range. The last variant is KmsServiceStopped at 61, leaving 3 slots.

Past the budget, `1u64 << (v - 1)` shifts by 64 or more: debug builds
panic with a shift overflow, release builds silently mask the shift
amount down and hand back a bit that already belongs to another event.
Either failure surfaces far from the line that added the variant.

The existing test_mask_bit_budget_is_not_exhausted only bounds the
events listed in the test-local ALL_EVENT_NAMES, so a variant nobody
remembered to list went unchecked. Add three layers instead:

- A const assertion on LAST_EVENT_NAME_VALUE, anchored on the last
  variant, that fails `cargo build` once the discriminant passes 64.
- An exhaustive-match tripwire next to ALL_EVENT_NAMES, so adding a
  variant breaks compilation with a non-exhaustive-patterns error that
  points at the budget notes.
- Tests pinning that discriminants stay dense and fully listed, that
  every leaf mask is non-zero and owns a unique bit (catching the
  release-mode wrapped-shift collision), and that the last variant
  still lands inside the u64.

Document the budget and the three guards on mask() so the next person
adding an event sees the remaining headroom.

Refs: rustfs/backlog#1572
2026-08-01 14:49:16 +00:00
Zhengchao An 0bd53becb5 feat(admin): emit KMS management audit events (#5554)
* feat(s3-types): add KMS service-control audit events

Configuration changes and service start/stop are management-plane actions
with no event name of their own, so they could not reach the audit
pipeline at all. Append three variants for them, following the existing
rule that KMS events are audit-only and live outside the `s3:` namespace,
so no bucket notification selector can expand to them.

* feat(admin): audit KMS management operations

Every KMS admin endpoint now builds an OperationContext from the
authenticated caller and hands it to the KMS layer, so the record the
manager already produces carries the principal, source address and
canonical request id instead of the internal placeholder.

A new adapter maps those records onto the server's existing AuditEntry
format and installs itself as the KMS audit sink at service assembly, so
KMS activity reaches the targets a deployment already operates. The
handlers emit directly for what the KMS layer cannot see: a request the
authorization gate rejects, and the endpoints with no context-aware KMS
entry point (data-key derivation and service control).

Only the failure class is recorded, never the error message, and the new
module joins the logging guardrail's checked files alongside the handlers
it serves.
2026-08-01 19:14:31 +08:00
Zhengchao An 5a6e850c67 feat(kms): wire OperationContext into an audit event contract (#5534) 2026-08-01 04:12:00 +00:00
houseme 78d6918c52 feat: extend hotpath coverage across crates (#5505)
Add opt-in hotpath feature surfaces to every workspace crate and wire the root rustfs feature passthrough for function, allocation, and CPU profiling.

Add a focused set of function-level measurements for scanner, heal, lock, target replay, IAM, KMS, Keystone, trusted proxy, and capacity paths without adding request-scoped primitive wrappers.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-31 05:42:19 +00:00
cxymds 6bc5fc77b5 fix(notify): emit noop event for missing deletes (#5381) 2026-07-28 17:05:10 +08:00
houseme f3a7a4b0da chore(deps): localize workspace dependency features (#4888)
Move workspace-level dependency feature lists into the member crates that consume each dependency while keeping required default-features flags at the workspace root.

Also refresh starshard to 2.2.2 via cargo update and cargo upgrade --exclude ratelimit.

Co-authored-by: heihutu <heihutu@gmail.com>
2026-07-16 03:55:27 +00:00
Zhengchao An 77eb91bf8d fix(s3-types): cover all removed events and fix scanner round-trip (#4512) 2026-07-09 01:07:06 +08:00
Zhengchao An 976a5d9713 fix(s3-types): give internal event names a mask base case to stop infinite recursion (backlog#965) (#4418)
EventName::mask() recursed forever for the three internal leaf events
(ObjectRemovedAbortMultipartUpload, ObjectCreatedCreateMultipartUpload,
ObjectRemovedDeleteObjects): their discriminants fall past
LAST_SINGLE_TYPE_VALUE so mask() took the compound branch, but expand()
returns vec![*self] for them, so mask() called itself with the same
value and overflowed the stack.

Detect the self-expanding leaf case (expand() yields exactly self) and
give it a dedicated bit derived from its discriminant. Those bits sit
above the single-type bits, so they never collide with each other or
with any compound 'All' mask.

Add exhaustive regression tests: every variant's mask() terminates, the
three internal masks are non-zero and mutually distinct and disjoint
from the single-type bits, and Everything covers all single-type bits.

Refs: https://github.com/rustfs/backlog/issues/965
2026-07-08 05:57:47 +00:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012)
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00