gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-29 16:37:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
9cf276ed24ef22e6cf13a9acc686b9bd24c53925
rustfs/docs/operations/two-factor-auth.md
T
Sinan Eldem b93e7b2355 feat(admin): self-service account management and TOTP two-factor authentication (#6596) 
* feat(madmin): add account and two-factor wire contract

Defines the self-service account and MFA API shapes in one place so the
console and the `rc` CLI decode identical payloads instead of each
carrying its own copy of the contract.

`AccountMutability` is part of the contract on purpose: a client needs to
know whether the server will accept a password change for this identity
before offering the control, rather than discovering it from a rejected
request.

* feat(s3-types): add IAM identity audit events

Adds `iam:Identity:CredentialChanged` and `iam:Identity:AuthChallenge`
so account and authentication activity reaches the audit pipeline in its
own namespace, the way the KMS events already do. Neither is reachable
from a bucket notification config.

Two variants for the whole surface rather than one per operation:
`mask()` gives every variant its own bit in a `u64`, and the budget is
nearly spent (63 of 64 used after this). The per-operation detail lives
in `AuditEntry::api.name` and the `iamOperation` tag, which is what a
SIEM filters on anyway. Splitting these further needs `mask()` widened
first.

* feat(iam): add two-factor authentication primitives

Implements the state machine behind TOTP enrollment and verification in
the IAM domain, so the admin handlers stay HTTP plumbing and the console
and CLI drive identical logic.

* `totp`: RFC 6238 over the workspace's existing hmac/sha1, pinned to the
  published Appendix B vectors. SHA-1, 6 digits, 30s: the parameters every
  mainstream authenticator app implements. Verification returns the
  matched time step so the caller can burn it.
* `recovery`: ten single-use codes, 100 bits each, in a Crockford base32
  alphabet without I/L/O/U. Stored as domain-separated SHA-256 digests —
  a password KDF would have to run once per stored code on every attempt,
  turning each guess into an attacker-controlled cost, and with uniform
  100-bit input there is no dictionary for it to defend against.
* `challenge`: stateless HMAC tokens. A TTL cache would be node-local, so
  a cluster without session affinity would issue on one node and verify
  on another; nothing here needs replicating.
* `record`: two-phase enrollment, replay high-water mark, and lockout.
  Pending enrollment never gates a login, so a mis-scanned QR cannot lock
  an operator out, and re-configuring keeps the old factor working until
  the new one is confirmed.
* `store`: one object per identity under `config/mfa/`, a sibling of
  `config/iam/` so the IAM cache loader's startup walk does not sweep it
  up. Optimistic `If-Match` writes; deliberately uncached, because a cache
  would need cluster-wide invalidation to keep the replay mark and the
  lockout counter honest.
* `qr`: server-side rendering, so neither client needs a QR encoder.

Enrollment is refused without `RUSTFS_IAM_MASTER_KEY`. A TOTP secret is
credential-equivalent, and one written in plaintext could be lifted off a
disk — worse than no second factor, because the user believes they have
one. IAM identities tolerate a missing master key for backward
compatibility; a new feature has no such history to honour.

Also adds `IamSys::revoke_sts_sessions_for_parent`, so a credential
rotation can invalidate the sessions minted under the old secret.

* feat(admin): add self-service account endpoints and the two-factor login gate

Adds the account surface (`/v3/account/*`), the second-factor endpoints,
the administrative reset (`/v3/user/mfa`), and `PUT
/v3/set-user-secret-key`, plus the gate on `AssumeRole`.

What the gate covers, and what it deliberately does not:

* `AssumeRole` is the only interactive login RustFS has, so it is where a
  second factor can be enforced. With one enrolled it requires
  `TokenCode`; without an enrollment the code path is unchanged, so
  existing deployments are untouched.
* A request signed directly with a long-term access key stays ungated.
  Gating it would break every script and CLI the moment a human enabled
  2FA on their own account, and would add no protection: whoever holds
  the secret key already has full access without presenting a code. This
  is the division AWS draws; making 2FA meaningful for API access needs an
  `aws:MultiFactorAuthPresent` policy condition, tracked separately.

`SerialNumber`/`TokenCode` are STS's own parameters, so an SDK or script
authenticates the same way the console does.

`caller_identity` resolves who a request acts as. The console signs with
a short-lived STS session, so "the caller" is almost never the key that
signed. It reports two separate capabilities: root cannot rotate its
secret (a process-wide `OnceLock` that also derives the internode RPC
secret) but *can* enroll a second factor — conflating the two would leave
the default deployment's console login unprotectable.

The self-service routes carry no admin action. Giving them one would be
wrong in both directions: it would stop an ordinary user from changing
their own password, and let any holder of that action change someone
else's. They gate on possession of the credential plus, for the
mutations, knowledge of the current secret — a signature only proves a
credential was used, so without that a hijacked tab could rewrite the
account's credentials or strip its second factor.

`set-user-secret-key` exists because the only prior way to change a
password was to re-POST the whole user through `add-user`, which rewrote
`status` and dropped the policy field — a password reset that silently
re-enabled a disabled account.

Wrong, replayed and malformed codes are indistinguishable on the wire;
the distinction survives only in the audit trail, where no submitted
value, secret or code is ever recorded.

* test(e2e): cover the two-factor lifecycle and its regressions

Unit tests cover the state machine at its edges; only an end-to-end test
proves the pieces are wired together and that the existing
authentication paths still behave.

Asserts, against a real server: enrollment is refused without a master
key; the full enroll/activate flow works with a genuine RFC 6238 code;
`AssumeRole` refuses without a factor and accepts a valid one; a recovery
code works exactly once; a direct SigV4 admin request keeps working with
a factor enrolled; `AssumeRole` for an unenrolled identity is unchanged;
and a password rotation invalidates the old secret.

The test computes TOTP codes itself rather than calling the server's
implementation — a shared helper could agree with a bug on both sides.

This suite caught a real defect during development: enrollment was
refused for root because its *password* is immutable, which would have
left the default deployment — an administrator signing into the console
as root — unable to protect the one login the feature exists for.

* docs(operations): document the two-factor authentication model

Records what the second factor protects and what it deliberately does
not, because several of the boundaries look like gaps until the
alternative is spelled out: why direct SigV4 access stays ungated, why
root credentials cannot be rotated at runtime, why secret keys cannot be
hashed in an S3 server, and why at-rest protection is mandatory for a
TOTP secret but optional for an IAM identity.

Also states the limitations plainly, including that GHSA-m77q-r63m-pj89
is unaffected: a holder of the root secret can still forge a session
token, 2FA claim included.

Placed alongside the other authentication and KMS security documents
rather than under a new `docs/security/`, which `.gitignore` excludes.

* fix(admin): route the new account handlers through the admin s3 facade

Two of the guardrails in the CI "Quick Checks" job rejected the previous
commits, so the required check would have gone red as soon as a maintainer
approved the workflow run.

`check_architecture_migration_rules.sh` requires everything under
`rustfs/src/admin` to reach `ECStore` through a domain module rather than
the root of `storage_api`. The MFA handler and the two `AssumeRole`
signatures now use `storage_api::runtime::ECStore`, which is where the
other ten admin handlers already take it from.

`check_s3s_footprint.sh` ratchets two counters that new code may not grow:
files referencing `s3s` and error-macro invocation lines. This branch added
four files and thirty-two lines to them. The ratchet is lower-only and its
header forbids raising a baseline to get green, so the construction moves
behind the facade instead: `storage_api::s3` now re-exports the request and
body types these handlers need and gains an `error` constructor over
`S3Error::with_message`. That is the same constructor the macro expands to
and the one `handlers/mod.rs`, `rebalance_internal_error` and
`invalid_object_lock_configuration` already call, so this is the existing
practice rather than a new one, and it keeps the `s3s` dependency in the
boundary file the s3gate migration replaces.

Every error code and message is carried over unchanged. In `sts.rs` only
the call site this branch added is converted; the sixteen that predate it
are left alone, because rewriting them would put unrelated churn in a
feature PR and push the counter below the baseline it is meant to hold.
2026-08-26 09:35:29 +08:00

300 lines
14 KiB
Markdown
Raw Blame History

# Two-Factor Authentication
> Scope: the self-service account surface (`/rustfs/admin/v3/account/*`), the
> login gate on `AssumeRole`, and the administrative reset
> (`/rustfs/admin/v3/user/mfa`).
This document records what the second factor does and does not protect, and why.
The boundaries are deliberate; several of them look like gaps until the
alternative is spelled out.
## What is protected
TOTP gates **session minting**: the `AssumeRole` call that turns a long-term
credential into a short-lived STS session. That is the only interactive login
RustFS has — the Console holds nothing but an STS session, and obtains it by
signing an `AssumeRole` request with the access key the user typed.
When an identity has an active enrollment:
```
access key + secret key
│
▼
GET /v3/mfa/challenge (SigV4-signed; answers "is a factor needed?")
│ required: true
▼
POST / Action=AssumeRole
SerialNumber=<challenge>
TokenCode=<6-digit TOTP | recovery code>
│
▼
STS credentials, with the claim x-rustfs-mfa-verified: true
```
Without a `TokenCode`, `AssumeRole` fails with `AccessDenied` and a message
carrying the `MultiFactorAuthRequired` marker. Clients match on that marker to
prompt for a code rather than reporting a failed login — the password *was*
accepted.
`SerialNumber` and `TokenCode` are `AssumeRole`'s own parameters, so an SDK or a
script authenticates the same way the Console does, with no RustFS-specific
protocol.
## What is deliberately not protected
**A request signed directly with a long-term access key is not gated.** This is
the most important boundary in the design, and it is intentional:
- Gating it would break every script, SDK client and `rc` invocation the moment
a human enabled 2FA on their own account. An operator who turned on a security
feature would discover it by way of a production outage.
- It would add no protection. Whoever holds the secret key already has full
access to everything that identity can reach; they never need to present a
code, because they never need to mint a session.
This is the same division AWS draws: MFA gates `AssumeRole` and is enforced for
API calls through the `aws:MultiFactorAuthPresent` policy condition, not by
refusing signed requests.
**Consequence to state plainly:** 2FA raises the cost of a stolen *password*. It
does not contain a stolen *secret key*. Because in RustFS the password **is** the
S3 secret key (see below), those are the same string — so 2FA protects the
console login path against credential reuse and phishing, and nothing more,
until the policy-condition work lands.
The tracked follow-up is an `aws:MultiFactorAuthPresent` condition key populated
from the `x-rustfs-mfa-verified` session claim, which would let an operator write
a policy that denies administrative actions to a session that presented no
second factor. That is the mechanism that makes 2FA meaningful for API access.
**OIDC and Keystone sessions are not gated either.** Those identities are
authenticated by their provider; a RustFS-side TOTP enrollment would not be
consulted at login and would give a false impression of protection. MFA for a
federated identity belongs to its IdP. `CallerIdentity` reports such sessions as
`FederatedIdentity` and refuses enrollment.
**Service-account credentials cannot manage their parent's factor.** A machine
credential must not be able to take over the human identity it was minted from.
## Password reality: there is no password hash
RustFS is an S3 server. SigV4 requires the server to know the secret key itself
in order to recompute a request signature, so secret keys **cannot** be hashed —
not here, and not in any S3-compatible implementation. The "password" a user
types into the Console is their S3 secret key.
What protects it instead:
| Protection | Mechanism |
| --- | --- |
| At rest | `RUSTFS_IAM_MASTER_KEY` + `encrypt_stream_io` (Argon2id → AES-GCM / ChaCha20-Poly1305) |
| Length floor | `is_secret_key_valid` (`SECRET_KEY_MIN_LEN`) |
| Rotation | `POST /v3/account/password`, requiring the current secret |
| Session cleanup | Every STS session minted from the identity is revoked on rotation |
There is deliberately **no maximum length**. The previous Console capped
passwords at 40 characters, which was a client-side invention with no server
constraint behind it; capping password length is an anti-pattern.
## At-rest protection is mandatory for TOTP secrets
A TOTP secret is credential-equivalent: anyone holding it can mint valid codes
forever. So enrollment is **refused** when `RUSTFS_IAM_MASTER_KEY` is not
configured, rather than writing the secret in plaintext:
```
POST /v3/account/mfa/enroll → 501 NotImplemented
"two-factor authentication requires RUSTFS_IAM_MASTER_KEY to be configured
so the shared secret can be encrypted at rest"
```
IAM *identities* tolerate a missing master key for backward compatibility with
existing deployments. A new feature has no such history to honour, and a second
factor that can be lifted off a disk is worse than none, because the user
believes they have one.
`GET /v3/account/mfa` reports `enrollment_available: false` with the reason, so
the Console and `rc` explain the remedy instead of offering a control that fails.
## Root credentials cannot be changed at runtime
The root identity comes from `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` and lands
in a process-wide `OnceLock` (`crates/credentials/src/credentials.rs`). It cannot
be rotated while the server runs, and the account surface reports this as
`credentials_source: "env"` with `mutable.password: false`.
This is not merely a missing feature. The root secret key feeds three things:
1. **STS session token signing** (`root_credentials::token_signing_key`) — every
live session in the cluster is HMAC-signed with it.
2. **The internode RPC secret** (`derive_rpc_secret`), unless
`RUSTFS_RPC_SECRET` is set explicitly.
3. **Legacy IAM at-rest decryption** for blobs migrated from MinIO.
Rotating it at runtime would therefore invalidate every session cluster-wide and
break node-to-node authentication. Making root mutable is a separate piece of
work with those three couplings as prerequisites; it is not a side effect of
adding a profile page.
**Operational recommendation:** treat root as a bootstrap identity. Create a
built-in IAM user with the `consoleAdmin` policy for day-to-day administration.
That identity has a working password change and full 2FA support.
## Rate limiting, replay and expiry
| Control | Value | Where |
| --- | --- | --- |
| Failed attempts before lockout | 5 | `mfa/record.rs` |
| First lockout | 15 minutes, doubling per further run | `mfa/record.rs` |
| Lockout ceiling | 1 hour | so a sustained attack cannot deny the owner indefinitely |
| TOTP clock skew | ±1 step (±30s) | three codes valid at once, no more |
| TOTP replay | Consumed time step is a high-water mark; `step <= last_used` is refused | closes the ~90s window a captured code would otherwise have |
| Recovery code replay | `used_at` stamp, single use | |
| Login challenge TTL | 5 minutes | |
| Pending enrollment TTL | 10 minutes | an abandoned enrollment leaves no usable secret |
A wrong code, a replayed code and a malformed code are **indistinguishable** on
the wire: all three return `AccessDenied` with the same message. The distinction
survives only in the audit trail, so an operator can tell a guessing attempt from
a replay without an attacker learning that a captured code was genuine.
The lockout is stored in the record and updated under an optimistic
compare-and-set, so it holds across the cluster rather than per node.
## Storage
```
.rustfs.sys/config/mfa/<access-key>/totp.json (encrypted with the IAM master key)
```
A sibling of `config/iam/`, not a child: the IAM cache loader walks the whole
`config/iam/` tree on startup and buckets what it finds by first path segment, so
a new prefix under there would be swept into that walk for no benefit.
Records are **not cached**. Every verification reads from the store, because a
cache would need cluster-wide invalidation to keep the replay mark and the
lockout counter honest, and getting that wrong reopens exactly the holes this
design closes. Verifications are rare enough that the read is not worth
optimising.
Writes are read-modify-write under an `If-Match` precondition with bounded
retries — the same optimistic scheme the IAM lazy-rewrite path uses. It degrades
to a retry rather than to a distributed lock a crashed node would have to time
out.
## Login challenges are stateless
A challenge is `HMAC-SHA256(root_secret, "rustfs-mfa-challenge:v1" ‖ access_key ‖
issued_at)`, base64url-encoded with its payload.
The obvious alternative is a TTL cache, the way the OIDC flow stores its PKCE
verifiers. That store is node-local, which is fine for OIDC because the whole
authorization round trip returns to the node that started it. A second factor
does not: a cluster behind a load balancer without session affinity would issue
the challenge on one node and receive the code on another, and a node-local
challenge would fail there for reasons no operator could debug.
Statelessness costs nothing, because the challenge is not what makes the exchange
single-use — the consumed TOTP time step is.
## Authorization model
| Route | Gate |
| --- | --- |
| `GET /v3/account/info` | possession of the credential |
| `POST /v3/account/password` | credential **+ knowledge of the current secret** |
| `GET /v3/account/mfa` | possession of the credential |
| `POST /v3/account/mfa/enroll` | credential, and the credential kind must be mutable |
| `POST /v3/account/mfa/activate` | credential + a valid code from the pending secret |
| `POST /v3/account/mfa/disable` | credential + **a valid code and the account password** |
| `POST /v3/account/mfa/recovery-codes` | credential + a valid code |
| `GET /v3/mfa/challenge` | possession of the credential |
| `GET /v3/user/mfa` | `admin:GetUser` |
| `DELETE /v3/user/mfa` | `admin:EnableUser` |
| `PUT /v3/set-user-secret-key` | `admin:CreateUser` |
The self-service routes carry **no admin action**. Giving them one would be wrong
in both directions: it would stop an ordinary user from changing their own
password, and it would let any holder of that action change somebody else's.
They are registered as `CredentialOnly` in the route-policy matrix.
`POST /v3/account/password` and `POST /v3/account/mfa/disable` require a
proof-of-knowledge step because a signature only proves a credential was *used*.
The Console signs with a short-lived session, so without it a hijacked browser
tab could rewrite the account's credentials or strip its second factor.
### Why turning the factor off needs the password too
Requiring only a code would mean a single shoulder-surfed number, in a session
someone walked away from, is enough to remove the protection. Requiring the
password makes disabling the factor as hard as the thing the factor protects.
### Break-glass
`DELETE /v3/user/mfa` clears another identity's factor, for a user who lost both
their authenticator and their recovery codes. It is gated on `admin:EnableUser`
rather than a bespoke action, because that is the same capability that can
already re-enable a disabled account — anyone who can do that can already take
the identity over, so a separate action would be a distinction without a security
difference.
The record is deleted outright rather than disabled, so no stale lockout counter
survives to block the user's next enrollment. The acting administrator is
recorded in the audit entry.
## Recovery codes
Ten codes, `XXXX-XXXX-XXXX-XXXX-XXXX`, 100 bits of uniform randomness each, in a
Crockford base32 alphabet with `I`, `L`, `O` and `U` removed so a handwritten
code cannot be ambiguous.
Stored as domain-separated SHA-256 digests, **not** a password KDF. With 100 bits
of uniform randomness there is no dictionary to try and no human-chosen pattern
to exploit, so the attacks a slow KDF defends against do not apply — while a
memory-hard KDF would have to run once per stored code on every verification
attempt, turning each guess into an attacker-controlled multiple of that cost.
This is the standard treatment for high-entropy bearer tokens, and the same
reasoning is why there is no per-code salt.
Codes are returned in plaintext exactly once. Activation always replaces the set:
reusing a previous one would leave codes valid for a secret they were never
issued against. Disabling clears them, so no live bypass survives a factor the
user believes is gone.
## Audit
Two `EventName` variants carry the whole surface:
- `iam:Identity:CredentialChanged` — password rotation, enrollment, activation,
disable, recovery-code regeneration, administrative reset.
- `iam:Identity:AuthChallenge` — challenge issuance and second-factor
verification.
The per-operation detail lives in `api.name` and the `iamOperation` tag, which is
what a SIEM filters on. The enum is coarse because `EventName::mask()` gives every
variant its own bit in a `u64` and the budget is nearly spent — 63 of 64 used
after these two. Splitting these per-operation needs `mask()` widened first.
**Redaction:** no secret key, TOTP secret, provisioning URI, submitted code or
recovery code enters an audit entry — not even hashed, and not on the failure
paths where the submitted value would be the most tempting thing to record.
Failures are described by a closed set of static strings
(`AccountAuditFailure`), so no caller-supplied bytes can reach a log target
through this module.
## Known limitations
1. **2FA does not gate direct SigV4 access.** By design; see above. The fix is
the `aws:MultiFactorAuthPresent` policy condition.
2. **Root cannot rotate its own credentials at runtime.** By design; see above.
3. **GHSA-m77q-r63m-pj89 is unaffected.** STS session tokens are signed with the
root secret key, so anyone holding it can still forge a session token —
including one carrying `x-rustfs-mfa-verified`. 2FA does not close this; a
dedicated STS signing key does, and that advisory is tracked separately.
4. **Username changes are not supported for anyone.** The access key is the
primary key for policy mappings, group membership, service-account parents and
bucket-policy principals. A rename is a migration that orphans service
accounts and silently breaks bucket-policy ARNs, not an edit;
`mutable.username` is `false` for every identity.
Reference in New Issue View Git Blame Copy Permalink