ci: raise CARGO_BUILD_JOBS to 3 on main pushes as a measured experiment (#5571)

The #5394 mitigation set it to 2 on the belief that three concurrent workspace
test links saturate the runner's overlay I/O. The sampler's cgroup v2 readings
show the pod has 14 CPUs and 28GB with a 2.1GB peak, so 2 throttles compilation
to a seventh of what is available and memory was never the constraint. The label
name sm-standard-4 had led everyone, including that mitigation and every
description written during this series, to assume four cores.

Raised to 3 for push events only. PRs keep 2, so the merge path is untouched
while the experiment runs.

Baseline over 17 non-cancelled samples at 2: median nextest/clippy step ratio
1.95, spread 1.85-2.06. Judging by that ratio rather than absolute wall clock is
what makes the signal usable — clippy runs on the same node at the same time and
is check-only for workspace members, so it never links the ~100 test binaries
this limit throttles, making it a control arm rather than a second measurement.

The criterion is the ratio dropping at least 10%, below about 1.76, with no 75m
timeout and no run showing three consecutive samples of rustc, collect2 or
rust-lld in D state. If it does not drop, the conclusion is that this limit is
not the bottleneck: fix it back at 2 and record the experiment. That is a
result, not a failure.

Kept at step level deliberately. rust-cache hashes CARGO/CC/CFLAGS/CXX/CMAKE/RUST
prefixed variables from process.env into its key, so promoting this to job level
would rotate every cache key on this lane.

Refs: rustfs/backlog#1598, rustfs/backlog#1601

.github/workflows/ci.yml

@@ -216,9 +216,29 @@ jobs:
 
       - name: Run nextest tests
         env:
-          # Three concurrent workspace test links saturate the self-hosted
-          # runner's overlay I/O and can wedge Cargo until the 75m timeout.
-          CARGO_BUILD_JOBS: "2"
+          # #5394 mitigation, now under a measured experiment (backlog#1601).
+          #
+          # 2 was chosen when three concurrent workspace test links were believed
+          # to saturate the runner's overlay I/O and wedge Cargo until the 75m
+          # timeout. cgroup v2 readings from the sampler show the pod actually
+          # has 14 CPUs and 28GB (peak use 2.1GB), so 2 throttles compilation to
+          # a seventh of what is available and memory was never the constraint —
+          # the label name "sm-standard-4" had led everyone, including the
+          # original mitigation, to assume 4 cores.
+          #
+          # Raised to 3 on main pushes only; PRs keep 2 so the merge path is
+          # untouched while the experiment runs. Baseline over 17 samples at 2:
+          # median nextest/clippy step ratio 1.95, spread 1.85-2.06. The gate-2
+          # criterion is that ratio dropping at least 10% (below ~1.76) with no
+          # 75m timeout and no run showing three consecutive samples of
+          # rustc/collect2/rust-lld in D state. If it does not, the conclusion is
+          # "this limit is not the bottleneck" — fix it back at 2 and record the
+          # experiment, which is a result, not a failure.
+          #
+          # Must stay step-level: rust-cache hashes CARGO/CC/CFLAGS/CXX/CMAKE/RUST
+          # prefixed variables from process.env into the cache key, so promoting
+          # this to job level would rotate every key on this lane.
+          CARGO_BUILD_JOBS: ${{ github.event_name == 'push' && '3' || '2' }}
         run: |
           mkdir -p artifacts/test-and-lint
           ./scripts/ci/resource_sampler.sh start nextest