mirror of
https://github.com/rustfs/rustfs.git
synced 2026-09-06 12:09:12 +00:00
feat(admin): add on-demand migration bucket admin API (#7076)
* feat(ecstore): add on-demand migration bucket config model Introduce OnDemandMigrationConfig (deny_unknown_fields, version 1) with typed validation, credential redaction, a secret-free Debug impl, and the OnceLock publish hook the runtime registers into. Exported through the api facade. * feat(ecstore): persist on-demand migration config in bucket metadata Store the config as a RustFS extension entry (on-demand-migration.json) with its update time in .metadata.bin, add the typed BucketMetadataSys accessor, and publish the config through the hook on every cache-install path alongside the durability sync. * refactor(ecstore): extract shared remote S3 client builder Move the aws_sdk_s3 client construction out of bucket_target_sys into bucket/remote_s3_client.rs: endpoint assembly, credential provider, path-style selection, custom CA / skip-TLS transports and the outbound SSRF gate now build from a neutral RemoteS3EndpointSpec so replication targets and the upcoming on-demand migration source client share one policy. Replication builds its client through From<&BucketTarget>; the gate keeps its relaxed semantics (private allowed, loopback only behind RUSTFS_REPLICATION_ALLOW_LOOPBACK_TARGET) verbatim. The builder also gains optional connect/read timeouts and a User-Agent suffix interceptor, both unset for replication. Refs rustfs/backlog#2149 * feat(ecstore): add on-demand migration SourceClient Add bucket/on_demand_migration/source_client.rs on top of the shared remote S3 builder: HEAD, ranged streaming GET, ListObjectsV2 with source-prefix mapping, GetObjectTagging and an admin probe. Every request carries the x-rustfs-/x-minio-source-proxy-request anti-loop markers and a RustFS-OnDemandMigration/<version> User-Agent suffix; SSE-C source objects are rejected as unsupported. SourceError classifies SDK failures (not found, access denied, throttled, timeout, connect, server error) with retryability and a stable metrics label. Debug output redacts credentials. Refs rustfs/backlog#2149 * docs(operations): point outbound policy at shared remote S3 client builder * chore: integrate ODM-01 and ODM-02 as B1 base (fix facade merge) * feat(admin): add on-demand migration bucket admin API Add the management plane for On-Demand Migration (ODM-07, rustfs/backlog#2154): PUT/GET/DELETE /v3/on-demand-migration/{bucket}, PUT ?dry-run=true, and a GET .../status skeleton. - PUT authorizes SetBucketOnDemandMigration, checks the bucket, the RUSTFS_ON_DEMAND_MIGRATION_ENABLED switch and the license, validates the ODM-01 config against local endpoints and replication targets, probes the source with SourceClient::probe(), then persists through the incarnation gate and asks peers to reload. Responses carry the redacted config and a probe summary; probe failures name only the error class. - GET answers 404 NoSuchConfiguration when unset; DELETE is idempotent (204). - New AdminAction variants admin:SetBucketOnDemandMigration and admin:GetBucketOnDemandMigration, route policy matrix rows, registration and MinIO alias coverage, and a doc row for the extra handler gates. - rustfs-madmin gains on_demand_migration wire types and client methods; golden fixtures under crates/madmin/fixtures/on_demand_migration/ are asserted byte-for-byte by both the handler and the client tests. Anonymous sources still map to a 400 naming source.credentials until the runtime slice adds the credential-less path. * refactor(admin): route on-demand migration handler errors through the s3 facade
This commit is contained in:
@@ -25,3 +25,14 @@ Router-level credential checks (`S3Router::check_access`) are bypassed only for:
|
||||
- console assets (`/favicon.ico`, `/rustfs/console...`), only while the console is enabled.
|
||||
|
||||
Every other admin route requires credentials at the router and a precise `AdminAction` or `S3Action` check in the handler (metrics routes, for example, authorize `GetMetricsAction`). The MinIO alias contract is specified in [minio-rustfs-router-compatibility.md](minio-rustfs-router-compatibility.md).
|
||||
|
||||
## Gated Bucket Feature Routes
|
||||
|
||||
Some bucket-scoped routes add gates after the `AdminAction` check. The gates are enforced in the handler, so they are invisible to the route matrix and listed here instead.
|
||||
|
||||
| Route | Actions | Extra gates after authorization |
|
||||
|---|---|---|
|
||||
| `PUT`/`DELETE /rustfs/admin/v3/on-demand-migration/{bucket}` (`?dry-run=true` validates and probes without saving) | `SetBucketOnDemandMigrationAction` (`admin:SetBucketOnDemandMigration`) | bucket must exist (`NoSuchBucket`); `PUT` also requires the `RUSTFS_ON_DEMAND_MIGRATION_ENABLED` module switch (`OnDemandMigrationDisabled`, 400) and the server license (`license_check()`, same mapping as object zip downloads); the source must answer HEAD + a one-key list (`OnDemandMigrationSourceUnreachable`, 400). Handler: `rustfs/src/admin/handlers/on_demand_migration.rs` |
|
||||
| `GET /rustfs/admin/v3/on-demand-migration/{bucket}` and `GET .../{bucket}/status` | `GetBucketOnDemandMigrationAction` (`admin:GetBucketOnDemandMigration`) | bucket must exist; reads work while the module switch is off so operators can inspect a disabled deployment; `GET` answers `NoSuchConfiguration` (404) when nothing is configured |
|
||||
|
||||
Responses on these routes carry the redacted configuration (`secret_key` and `session_token` replaced by `REDACTED`); the wire shape is pinned by the fixtures under `crates/madmin/fixtures/on_demand_migration/`, shared by the server handler tests and the `rustfs-madmin` client tests.
|
||||
|
||||
Reference in New Issue
Block a user