* refactor(rustfs): carve app/object out of object_usecase.rs — shared, extract, test_support children (backlog#1841 step 1)
Mechanical move-only split of rustfs/src/app/object_usecase.rs (19.7K lines). The file body moves to rustfs/src/app/object/mod.rs, and the first self-contained slices move into children: shared.rs (cross-cutting helpers: quota admission, response checksum injection, object-lock write validation, table-catalog mutation guard, deadlock request guard, proxy passthrough utilities), extract.rs (snowball auto-extract path incl. tar/pax helpers and execute_put_object_extract), and cfg(test) test_support.rs for cross-module test scaffolding. object_usecase.rs stays as a thin pub use facade so every existing crate::app::object_usecase:: path keeps working.
No behavior change: items move verbatim; the only source edits are visibility widenings required by the new module boundaries (private -> pub(super); pub(super) -> pub(crate) for the three helpers multipart_usecase and the app gating tests import). Guard scripts that pinned rustfs/src/app/object_usecase.rs now scan the rustfs/src/app/object tree, and the table_catalog source-text guard test concatenates the split files.
* refactor(rustfs): move the GetObject read path into app/object/get.rs (backlog#1841 step 2)
Move-only continuation of the object_usecase split: cold-fill orchestration, disk-permit admission, streaming readers and resume control, stream-buffer tuning, execute_get_object / execute_get_object_attributes, the GET replication proxy helpers, and their unit tests move from app/object/mod.rs into app/object/get.rs. Items keep their original text; cross-module call sites rely on the visibility widenings introduced in step 1.
* refactor(rustfs): move the PutObject and CopyObject paths into app/object (backlog#1841 step 3)
Move-only continuation: put.rs takes the PUT body admission and timeout readers, zero-copy and eager-commit machinery, execute_put_object, and the PUT unit tests; copy.rs takes the copy namespace/lifecycle lock helpers and execute_copy_object with its tests. Two source edits beyond visibility widenings: PutObjectChecksums fields become pub(super) (read by shared::apply_trailing_checksums across the new module boundary) and one relative super::storage_api call in the copy path becomes crate::app::storage_api since super now resolves to app::object. The table_catalog source-text guard concatenates the new files.
* refactor(rustfs): finish the object_usecase split — delete, head, restore modules (backlog#1841 step 4)
Move-only completion: delete.rs takes the delete helpers, cfg(test) delete hooks, and execute_delete_object/execute_delete_objects; head.rs takes execute_head_object with the HEAD replication proxy helpers; restore.rs takes execute_restore_object. app/object/mod.rs is now just the shared import prelude, module wiring, and the DefaultObjectUsecase struct with its constructors, accessors, and the execute_select_object_content delegation; the emptied tests module is gone. The delete re-export glob is cfg(test)-gated because its only cross-module consumers are the delete test hooks.
The table_catalog source-text guard now isolates the delete entrypoints from app/object/delete.rs, and doc/comment references that pointed at rustfs/src/app/object_usecase.rs internals now point at the per-operation modules.
Single-part encrypted objects in the legacy format could not serve range
reads without decrypting from byte 0: v1 frames are emitted per upstream
read, so no closed-form plaintext-to-ciphertext mapping exists. The v2
layout fixed the frame length (8218 ciphertext bytes per 8 KiB plaintext
frame), making the mapping closed-form.
Consume it:
- Single-part PUTs that encrypt locally under the v2 write switch stamp
the frame-layout marker, valued with the object's data_dir token -
ciphertext passthrough, data movement and copies mint a new data_dir
or strip the marker, so a re-homed marker disqualifies itself.
- The encrypted read plan seeks marked, uncompressed single-part objects
to frame_index * 8218 and decrypts from that frame: the frame index
rides the plan's sequence-number slot into DecryptReader::new_at_block,
whose nonce and AAD bind absolute indices. New metric path label
frame_seek.
- A lying marker fails closed: v2 authentication rejects bytes at a fake
frame boundary; plaintext is never served from the wrong offset.
Compressed objects and multipart sub-part seeks keep the conservative
paths (follow-up work); reading needs no switch - seekability follows
the marker.
* feat(madmin): add account and two-factor wire contract
Defines the self-service account and MFA API shapes in one place so the
console and the `rc` CLI decode identical payloads instead of each
carrying its own copy of the contract.
`AccountMutability` is part of the contract on purpose: a client needs to
know whether the server will accept a password change for this identity
before offering the control, rather than discovering it from a rejected
request.
* feat(s3-types): add IAM identity audit events
Adds `iam:Identity:CredentialChanged` and `iam:Identity:AuthChallenge`
so account and authentication activity reaches the audit pipeline in its
own namespace, the way the KMS events already do. Neither is reachable
from a bucket notification config.
Two variants for the whole surface rather than one per operation:
`mask()` gives every variant its own bit in a `u64`, and the budget is
nearly spent (63 of 64 used after this). The per-operation detail lives
in `AuditEntry::api.name` and the `iamOperation` tag, which is what a
SIEM filters on anyway. Splitting these further needs `mask()` widened
first.
* feat(iam): add two-factor authentication primitives
Implements the state machine behind TOTP enrollment and verification in
the IAM domain, so the admin handlers stay HTTP plumbing and the console
and CLI drive identical logic.
* `totp`: RFC 6238 over the workspace's existing hmac/sha1, pinned to the
published Appendix B vectors. SHA-1, 6 digits, 30s: the parameters every
mainstream authenticator app implements. Verification returns the
matched time step so the caller can burn it.
* `recovery`: ten single-use codes, 100 bits each, in a Crockford base32
alphabet without I/L/O/U. Stored as domain-separated SHA-256 digests —
a password KDF would have to run once per stored code on every attempt,
turning each guess into an attacker-controlled cost, and with uniform
100-bit input there is no dictionary for it to defend against.
* `challenge`: stateless HMAC tokens. A TTL cache would be node-local, so
a cluster without session affinity would issue on one node and verify
on another; nothing here needs replicating.
* `record`: two-phase enrollment, replay high-water mark, and lockout.
Pending enrollment never gates a login, so a mis-scanned QR cannot lock
an operator out, and re-configuring keeps the old factor working until
the new one is confirmed.
* `store`: one object per identity under `config/mfa/`, a sibling of
`config/iam/` so the IAM cache loader's startup walk does not sweep it
up. Optimistic `If-Match` writes; deliberately uncached, because a cache
would need cluster-wide invalidation to keep the replay mark and the
lockout counter honest.
* `qr`: server-side rendering, so neither client needs a QR encoder.
Enrollment is refused without `RUSTFS_IAM_MASTER_KEY`. A TOTP secret is
credential-equivalent, and one written in plaintext could be lifted off a
disk — worse than no second factor, because the user believes they have
one. IAM identities tolerate a missing master key for backward
compatibility; a new feature has no such history to honour.
Also adds `IamSys::revoke_sts_sessions_for_parent`, so a credential
rotation can invalidate the sessions minted under the old secret.
* feat(admin): add self-service account endpoints and the two-factor login gate
Adds the account surface (`/v3/account/*`), the second-factor endpoints,
the administrative reset (`/v3/user/mfa`), and `PUT
/v3/set-user-secret-key`, plus the gate on `AssumeRole`.
What the gate covers, and what it deliberately does not:
* `AssumeRole` is the only interactive login RustFS has, so it is where a
second factor can be enforced. With one enrolled it requires
`TokenCode`; without an enrollment the code path is unchanged, so
existing deployments are untouched.
* A request signed directly with a long-term access key stays ungated.
Gating it would break every script and CLI the moment a human enabled
2FA on their own account, and would add no protection: whoever holds
the secret key already has full access without presenting a code. This
is the division AWS draws; making 2FA meaningful for API access needs an
`aws:MultiFactorAuthPresent` policy condition, tracked separately.
`SerialNumber`/`TokenCode` are STS's own parameters, so an SDK or script
authenticates the same way the console does.
`caller_identity` resolves who a request acts as. The console signs with
a short-lived STS session, so "the caller" is almost never the key that
signed. It reports two separate capabilities: root cannot rotate its
secret (a process-wide `OnceLock` that also derives the internode RPC
secret) but *can* enroll a second factor — conflating the two would leave
the default deployment's console login unprotectable.
The self-service routes carry no admin action. Giving them one would be
wrong in both directions: it would stop an ordinary user from changing
their own password, and let any holder of that action change someone
else's. They gate on possession of the credential plus, for the
mutations, knowledge of the current secret — a signature only proves a
credential was used, so without that a hijacked tab could rewrite the
account's credentials or strip its second factor.
`set-user-secret-key` exists because the only prior way to change a
password was to re-POST the whole user through `add-user`, which rewrote
`status` and dropped the policy field — a password reset that silently
re-enabled a disabled account.
Wrong, replayed and malformed codes are indistinguishable on the wire;
the distinction survives only in the audit trail, where no submitted
value, secret or code is ever recorded.
* test(e2e): cover the two-factor lifecycle and its regressions
Unit tests cover the state machine at its edges; only an end-to-end test
proves the pieces are wired together and that the existing
authentication paths still behave.
Asserts, against a real server: enrollment is refused without a master
key; the full enroll/activate flow works with a genuine RFC 6238 code;
`AssumeRole` refuses without a factor and accepts a valid one; a recovery
code works exactly once; a direct SigV4 admin request keeps working with
a factor enrolled; `AssumeRole` for an unenrolled identity is unchanged;
and a password rotation invalidates the old secret.
The test computes TOTP codes itself rather than calling the server's
implementation — a shared helper could agree with a bug on both sides.
This suite caught a real defect during development: enrollment was
refused for root because its *password* is immutable, which would have
left the default deployment — an administrator signing into the console
as root — unable to protect the one login the feature exists for.
* docs(operations): document the two-factor authentication model
Records what the second factor protects and what it deliberately does
not, because several of the boundaries look like gaps until the
alternative is spelled out: why direct SigV4 access stays ungated, why
root credentials cannot be rotated at runtime, why secret keys cannot be
hashed in an S3 server, and why at-rest protection is mandatory for a
TOTP secret but optional for an IAM identity.
Also states the limitations plainly, including that GHSA-m77q-r63m-pj89
is unaffected: a holder of the root secret can still forge a session
token, 2FA claim included.
Placed alongside the other authentication and KMS security documents
rather than under a new `docs/security/`, which `.gitignore` excludes.
* fix(admin): route the new account handlers through the admin s3 facade
Two of the guardrails in the CI "Quick Checks" job rejected the previous
commits, so the required check would have gone red as soon as a maintainer
approved the workflow run.
`check_architecture_migration_rules.sh` requires everything under
`rustfs/src/admin` to reach `ECStore` through a domain module rather than
the root of `storage_api`. The MFA handler and the two `AssumeRole`
signatures now use `storage_api::runtime::ECStore`, which is where the
other ten admin handlers already take it from.
`check_s3s_footprint.sh` ratchets two counters that new code may not grow:
files referencing `s3s` and error-macro invocation lines. This branch added
four files and thirty-two lines to them. The ratchet is lower-only and its
header forbids raising a baseline to get green, so the construction moves
behind the facade instead: `storage_api::s3` now re-exports the request and
body types these handlers need and gains an `error` constructor over
`S3Error::with_message`. That is the same constructor the macro expands to
and the one `handlers/mod.rs`, `rebalance_internal_error` and
`invalid_object_lock_configuration` already call, so this is the existing
practice rather than a new one, and it keeps the `s3s` dependency in the
boundary file the s3gate migration replaces.
Every error code and message is carried over unchanged. In `sts.rs` only
the call site this branch added is converted; the sixteen that predate it
are left alone, because rewriting them would put unrelated churn in a
feature PR and push the counter below the baseline it is meant to hold.
* test(e2e): activate data usage regressions
* test(e2e): bind data usage selection to Linux listing
* test(e2e): refresh data usage Darwin selection
---------
Co-authored-by: houseme <housemecn@gmail.com>
* refactor(e2e/kms): replace fixed startup sleeps with KMS readiness probe
Replace 33 hard-coded sleep(3s) / sleep(2s) startup waits in KMS e2e tests
with an active readiness probe (wait_for_kms_ready) that polls the KMS
status endpoint with exponential backoff (200ms→1s, 5s budget).
This cuts per-test startup latency from a fixed 3s to ~200-500ms while
remaining robust against slow CI machines.
Non-startup sleeps (ILM polling loops, fault-recovery detection delays,
test-runner inter-test pauses) are left untouched.
* style: cargo fmt
* fix(kms): use .expect() instead of ? in test functions that return ()
7 call sites of wait_for_kms_ready() used ? in async test functions
that return () instead of Result. Changed to .expect("KMS ready").
* fix(kms): enforce readiness probe deadline
* fix(kms): validate readiness backend status
* chore(obs): ReplicationStats -> ReplicationMetricsSnapshot, BucketReplicationStats -> BucketReplicationMetricsSnapshot
Rename in-obs-crate ReplicationStats and BucketReplicationStats to
ReplicationMetricsSnapshot and BucketReplicationMetricsSnapshot respectively.
No serde impact (these types are Prometheus metric collectors, not serialized).
No external consumers found outside the obs crate.
* cleanup: remove #[serial] annotations from e2e_test, scanner, lifecycle, and object-capacity crates
Remove no-op #[serial] attributes (nextest ignores serial_test) and the
serial_test dependency from four crates. All tests already use temp_env
for env-var isolation, making #[serial] purely redundant.
Crates cleaned:
- e2e_test (37 annotations, 9 imports, removed serial_test dep)
- rustfs-scanner (115 annotations across 7 files, removed serial_test dep)
- rustfs-lifecycle (46 annotations, removed serial_test dep)
- rustfs-object-capacity (38 annotations, removed serial_test dep)
Also converted scanner/tests/lifecycle_integration_test.rs
with_forced_immediate_enqueue_timeout helper from unsafe raw
env::set_var/remove_var to temp_env::async_with_vars for proper
isolation, and added async_closure feature to scanner's temp-env dep.
* fix(lifecycle): restore #[serial] on 2 tests that read env vars without temp_env
eval_inner_expires_latest_object_after_days_due and
eval_inner_does_not_panic_on_many_equal_due_events call eval_inner()
which reads ENV_ILM_PROCESS_TIME via std::env::var(). Without #[serial]
they race with other tests that set these vars via temp_env.
* style: cargo fmt
houseme