feat(odm): enable on-demand migration by default (#7089)

* feat(odm): enable on-demand migration by default

The module switch RUSTFS_ON_DEMAND_MIGRATION_ENABLED now defaults to true,
so the feature is reachable without an opt-in; setting it to false still
keeps the module out of the read path entirely. A bucket without an
on-demand-migration.json is never resolved by the runtime and makes no
source call, so the flip changes nothing for unconfigured buckets.

The admin plane now reads the switch through the predicate published by
module_switches.rs instead of its own duplicated env constant; the
behaviour (an environment read per call) is unchanged.

* test(e2e): wire three on-demand migration cases into e2e-smoke

The PR smoke lane gains one case per user-visible contract: a GET miss
that pulls and persists, a HEAD miss that answers from the source and
stores nothing, and the admin config/status pair that must redact the
source secret. The HEAD case did not exist outside the nightly
real-source lane, so it is added to get_basic_test.

Measured on darwin: the lane goes from 168 tests in 101.98 s to 171
tests in 101.92 s, since the three cases overlap the lane's existing
work. The darwin selection digests for e2e-smoke and e2e-full are
regenerated; the e2e-full linux digest still needs a Linux runner.

* docs(changelog): record the on-demand migration feature
This commit is contained in:
Zhengchao An
2026-09-03 11:40:54 +08:00
committed by GitHub
parent 1747ed0292
commit d011ec9952
9 changed files with 79 additions and 21 deletions
@@ -12,9 +12,10 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//! Basic GET read-through scenarios (rustfs/backlog#2156): inline pull and
//! Basic read-through scenarios (rustfs/backlog#2156): inline pull and
//! local persistence, large-object passthrough with background backfill,
//! Range passthrough, source 404, `versionId` reads, and a disabled bucket.
//! Range passthrough, source 404, `versionId` reads, a disabled bucket, and
//! the HEAD passthrough that stores nothing (rustfs/backlog#2155).
//! Every source-side expectation is asserted on the fake source's journal.
use super::common::{BoxError, OdmSourceSpec, OdmTestEnv, SeedObject};
@@ -263,3 +264,35 @@ async fn get_after_disable_does_not_consult_the_source() -> TestResult {
env.assert_local_absent(bucket, key).await;
Ok(())
}
/// A HEAD miss is answered from the source but must not store anything: the
/// key stays absent locally, so a second HEAD consults the source again. This
/// is the smoke-lane guard for the HEAD passthrough (rustfs/backlog#2155).
#[tokio::test]
async fn head_miss_answers_from_the_source_without_persisting() -> TestResult {
let bucket = "odm-head-passthrough";
let env = configured_env(bucket, |_| {}).await?;
let key = "head/report.bin";
let body = payload(32 * 1024);
env.seed_source(SOURCE_BUCKET, &[SeedObject::new(key, body.clone())]);
let head = env.raw_object_request(http::Method::HEAD, bucket, key, &[]).await?;
assert_eq!(head.status, 200, "{}", String::from_utf8_lossy(&head.body));
assert_eq!(head.header(ODM_RESPONSE_HEADER), Some("source"), "a source answer is marked");
assert_eq!(head.header("content-length"), Some(body.len().to_string().as_str()));
assert!(head.body.is_empty(), "a HEAD answer carries no body");
assert_eq!(env.source.count_requests(Operation::HeadObject, key), 1);
assert_eq!(env.source.count_requests(Operation::GetObject, key), 0, "a HEAD must never pull the body");
env.assert_local_absent(bucket, key).await;
let again = env.raw_object_request(http::Method::HEAD, bucket, key, &[]).await?;
assert_eq!(again.status, 200, "{}", String::from_utf8_lossy(&again.body));
assert_eq!(
env.source.count_requests(Operation::HeadObject, key),
2,
"nothing was written back, so the second HEAD consults the source again"
);
assert_eq!(env.source.count_requests(Operation::GetObject, key), 0);
env.assert_local_absent(bucket, key).await;
Ok(())
}