Zhengchao An
a5bde8b0af
* feat(ecstore): add on-demand migration bucket config model Introduce OnDemandMigrationConfig (deny_unknown_fields, version 1) with typed validation, credential redaction, a secret-free Debug impl, and the OnceLock publish hook the runtime registers into. Exported through the api facade. * feat(ecstore): persist on-demand migration config in bucket metadata Store the config as a RustFS extension entry (on-demand-migration.json) with its update time in .metadata.bin, add the typed BucketMetadataSys accessor, and publish the config through the hook on every cache-install path alongside the durability sync. * refactor(ecstore): extract shared remote S3 client builder Move the aws_sdk_s3 client construction out of bucket_target_sys into bucket/remote_s3_client.rs: endpoint assembly, credential provider, path-style selection, custom CA / skip-TLS transports and the outbound SSRF gate now build from a neutral RemoteS3EndpointSpec so replication targets and the upcoming on-demand migration source client share one policy. Replication builds its client through From<&BucketTarget>; the gate keeps its relaxed semantics (private allowed, loopback only behind RUSTFS_REPLICATION_ALLOW_LOOPBACK_TARGET) verbatim. The builder also gains optional connect/read timeouts and a User-Agent suffix interceptor, both unset for replication. Refs rustfs/backlog#2149 * feat(ecstore): add on-demand migration SourceClient Add bucket/on_demand_migration/source_client.rs on top of the shared remote S3 builder: HEAD, ranged streaming GET, ListObjectsV2 with source-prefix mapping, GetObjectTagging and an admin probe. Every request carries the x-rustfs-/x-minio-source-proxy-request anti-loop markers and a RustFS-OnDemandMigration/<version> User-Agent suffix; SSE-C source objects are rejected as unsupported. SourceError classifies SDK failures (not found, access denied, throttled, timeout, connect, server error) with retryability and a stable metrics label. Debug output redacts credentials. Refs rustfs/backlog#2149 * docs(operations): point outbound policy at shared remote S3 client builder * chore: integrate ODM-01 and ODM-02 as B1 base (fix facade merge) * feat(admin): add on-demand migration bucket admin API Add the management plane for On-Demand Migration (ODM-07, rustfs/backlog#2154): PUT/GET/DELETE /v3/on-demand-migration/{bucket}, PUT ?dry-run=true, and a GET .../status skeleton. - PUT authorizes SetBucketOnDemandMigration, checks the bucket, the RUSTFS_ON_DEMAND_MIGRATION_ENABLED switch and the license, validates the ODM-01 config against local endpoints and replication targets, probes the source with SourceClient::probe(), then persists through the incarnation gate and asks peers to reload. Responses carry the redacted config and a probe summary; probe failures name only the error class. - GET answers 404 NoSuchConfiguration when unset; DELETE is idempotent (204). - New AdminAction variants admin:SetBucketOnDemandMigration and admin:GetBucketOnDemandMigration, route policy matrix rows, registration and MinIO alias coverage, and a doc row for the extra handler gates. - rustfs-madmin gains on_demand_migration wire types and client methods; golden fixtures under crates/madmin/fixtures/on_demand_migration/ are asserted byte-for-byte by both the handler and the client tests. Anonymous sources still map to a 400 naming source.credentials until the runtime slice adds the credential-less path. * refactor(admin): route on-demand migration handler errors through the s3 facade
4.2 KiB
Admin Route Action Snapshot
Use this when: you add, move, or re-authorize an admin route and need to know where the route → handler → AdminAction contract is enforced.
Source of truth: rustfs/src/admin/route_policy.rs (the AdminRouteSpec matrix, checked by validate_admin_route_policy_specs), rustfs/src/admin/route_registration_test.rs (registration coverage), rustfs/src/admin/router.rs (dispatch and credential checks), rustfs/src/admin/handlers/*.rs (handler-level authorization calls).
This page is a pointer, not a route table. The machine-checked matrix in route_policy.rs lists every admin route with its AdminAction and RouteRiskLevel; routes that are registered but answered by policy instead of a handler are declared there too through DeferredRoutePolicyReason. The AdminRouteSpec type lives in crates/security-governance/src/admin_matrix.rs.
Prefix And Alias Contract
| Prefix | Behavior | Rule |
|---|---|---|
/rustfs/admin |
Canonical admin prefix used by make_admin_route (rustfs/src/admin/mod.rs) |
The only registered admin prefix |
/minio/admin |
Compatibility alias accepted by S3Router::is_match; canonicalize_admin_path rewrites it to /rustfs/admin immediately before route lookup (rustfs/src/admin/router.rs) |
Never register routes twice; preserve canonicalization |
/iceberg/v1 |
Table catalog prefix registered by register_table_catalog_route (rustfs/src/admin/handlers/table_catalog/routes.rs) and accepted by is_admin_path |
Stays outside /rustfs/admin; table actions are authorized per handler |
/health, /health/ready |
Public health endpoints, registered only when ENV_HEALTH_ENDPOINT_ENABLE allows |
Preserve the unauthenticated bypass |
/profile/cpu, /profile/memory |
Registered by the health handler but guarded by profile authorization | Never couple to health-endpoint enablement |
Public Exceptions
Router-level credential checks (S3Router::check_access) are bypassed only for:
- health routes, when they are registered;
- OIDC bootstrap paths matched by
is_oidc_path(providers,authorize/{provider_id},callback/{provider_id},logout); the bypass is path-based, so it applies to any method on those paths; - unsigned STS web-identity form posts to
/withapplication/x-www-form-urlencoded, which the STS handler validates itself; - console assets (
/favicon.ico,/rustfs/console...), only while the console is enabled.
Every other admin route requires credentials at the router and a precise AdminAction or S3Action check in the handler (metrics routes, for example, authorize GetMetricsAction). The MinIO alias contract is specified in minio-rustfs-router-compatibility.md.
Gated Bucket Feature Routes
Some bucket-scoped routes add gates after the AdminAction check. The gates are enforced in the handler, so they are invisible to the route matrix and listed here instead.
| Route | Actions | Extra gates after authorization |
|---|---|---|
PUT/DELETE /rustfs/admin/v3/on-demand-migration/{bucket} (?dry-run=true validates and probes without saving) |
SetBucketOnDemandMigrationAction (admin:SetBucketOnDemandMigration) |
bucket must exist (NoSuchBucket); PUT also requires the RUSTFS_ON_DEMAND_MIGRATION_ENABLED module switch (OnDemandMigrationDisabled, 400) and the server license (license_check(), same mapping as object zip downloads); the source must answer HEAD + a one-key list (OnDemandMigrationSourceUnreachable, 400). Handler: rustfs/src/admin/handlers/on_demand_migration.rs |
GET /rustfs/admin/v3/on-demand-migration/{bucket} and GET .../{bucket}/status |
GetBucketOnDemandMigrationAction (admin:GetBucketOnDemandMigration) |
bucket must exist; reads work while the module switch is off so operators can inspect a disabled deployment; GET answers NoSuchConfiguration (404) when nothing is configured |
Responses on these routes carry the redacted configuration (secret_key and session_token replaced by REDACTED); the wire shape is pinned by the fixtures under crates/madmin/fixtures/on_demand_migration/, shared by the server handler tests and the rustfs-madmin client tests.