* fix(odm): declare the remote client retry policy per consumer
The SDK retry policy was an inherited default: one logical call could cost
three wire requests, so the migration breaker counted logical calls on top
of a threefold amplification against a source that was already failing.
Make it an explicit RemoteS3EndpointSpec field. Replication targets declare
today's standard three attempts and keep their behaviour; the on-demand
migration source and its admin probe declare a disabled policy, so one
counted failure is exactly one source request and pull.rs owns the only
retry budget.
* fix(odm): count a stalled inline source as a source timeout
The inline tee wraps its source body in the idle guard, but the tee turns a
stalled source into an ordinary body read error, so the write-back reported
it as a local write failure. Hand commit_inline the guard so the pull is
counted under source_timeout instead.
The background pump now enforces the idle budget through the same guard
rather than a second copy of the timeout loop.
* test(odm): cover a stalled source body end to end
The fake target can now deliver a GetObject body in slices with a pause
between them, so the inline abort can be driven by a stalled source instead
of a truncated one. Two fault cases drop the workarounds they carried for
the SDK's retries: the scripted fault count and the observed source request
count now have to agree.
The operations guide records the retry and idle-timeout guarantees.
* test(odm): add the fault, concurrency, interaction and real-source e2e
Twenty-one scenarios for on-demand migration: source failures and the
circuit breaker, single-flight and the pull-concurrency limits, how a
pulled object composes with encryption, Object Lock, quota, events,
replication, delete markers and the disable switch, and two cases against
a real second RustFS as the source.
Every assertion carries a source-request-count dimension so a case cannot
pass by serving the client while quietly re-reading the source.
* ci(odm): route the slow on-demand-migration e2e to the nightly lane
The fault, concurrency and real-source modules wait out the 30 s breaker
window, drive 100-deep bursts, or start extra RustFS processes, so they
join e2e-nightly and are subtracted from the e2e-full merge budget. The
e2e-smoke selection is unchanged.
Zhengchao An