Commit Graph

11 Commits

Author SHA1 Message Date
Chris An 191c95a469 ci: enforce affected validation and rehearse bounded shards (#9070)
* ci: enforce affected validation and rehearse bounded shards

* test(ci): keep regression output stable under forced color
2026-10-30 11:58:38 +08:00
Chris An 4fcbf60f10 ci: unify validation planning, artifacts and suite orchestration (#9059)
* ci: unify validation planning, artifacts and suite orchestration

* fix(ci): preserve JSON diagnostics for release checks

* fix(ci): keep manual attempts without plans visible

* fix(ci): distinguish nextest setup scripts from test results

* ci: reuse extracted archives and consolidate duplicate checks
2026-10-30 02:02:38 +08:00
Hauser e0444b519d feat(ecstore): add sampled batch CPU diagnostics (#9019)
* feat(ecstore): add sampled batch CPU diagnostics

Keep timing and request correlation in a separate default-off feature. Preserve the existing positioned-read backend, byte and error contracts.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): route batch diagnostics through storage facade

Keep the diagnostic implementation private to its owner and expose explicit feature-gated helpers through the existing storage boundary.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): retain diagnostic IDs across GET producer tasks

Capture request context before spawning the legacy producer. Keep the existing producer, cancellation and read-lock lifetime intact.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): avoid unused batch read index

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(obs): isolate local batch diagnostic JSON events

Omit span metadata and inferred parent IDs for diagnostic root events. Verify per-shard index order and document existing production metric timing.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(ecstore): accelerate receipt fixture in CI

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(heal): allocate stack for bucket recovery fixture

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(s3): allocate stack for conditional GET fixture

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-10-29 10:32:06 +08:00
Chris 9749447825 test(connect): retain CPU profile import bundle (#7820) 2026-09-14 10:54:25 +08:00
Chris 874d0d7b63 test(connect): retain CPU profile verification key (#7819) 2026-09-14 10:50:42 +08:00
Chris d2e2aa7d0b test(connect): verify profile cpu release artifacts (#7818) 2026-09-14 10:48:17 +08:00
Chris b391ec4f0e ci: retain top disk acceptance archive (#7812) 2026-09-14 10:29:28 +08:00
Chris a0bbcba2b9 test(connect): require native top.disk runner (#7809) 2026-09-14 10:24:28 +08:00
Chris dfcee45b98 test(connect): verify top.disk release artifacts (#7808) 2026-09-14 10:21:37 +08:00
Zhengchao An b7805caa58 ci: stop archiving every dependency's unpacked source in each cache (#5566)
The four consolidated ci keys plus the build keys still do not fit the
repository's fixed 10GB Actions cache quota, so LRU keeps evicting them:
measured demand is ci-dev 2331MB + ci-feat-proto 2310MB + ci-feat-rio 1951MB +
ci-uring 1317MB + two build legs at ~1429MB + cargo-deny 844MB, and main pushes
add two more build legs. That is roughly 14.4GB against 10.24GB. The symptom is
misleading: Cache Warm reports every job successful and the restores log "full
match: true", yet ci-feat-rio and ci-uring disappear from the cache list between
runs.

cache-all-crates was the wrong default for this repository. With it set to true,
rust-cache's cleanup returns before pruning ~/.cargo/registry/src and its config
archives the whole registry, so every cache carried the unpacked source tree of
every dependency — not, as the name suggests, just a few extra crates.

Setting it to false is rust-cache's own default and loses no coverage: the
package set comes from `cargo metadata --all-features`, a strict superset of any
single lane's feature closure; -sys crates are explicitly exempt from pruning,
since their source timestamps would otherwise trigger rebuilds; and everything
pruned is re-unpacked from the .crate files still in registry/cache, whose
mtimes crates.io normalises, so cargo fingerprints stay valid.

Applied to the setup composite and to audit.yml's own rust-cache. Cache Warm
now also reports the sizes of registry/src, registry/cache, registry/index,
~/.cargo/git and target/ to the step summary, immediately before rust-cache's
post step archives them, so the size of the effect is measured rather than
assumed.

The new sizes only appear once the cache key next rotates, since rust-cache
skips the save entirely on an exact key hit. Deliberately not forcing that by
bumping prefix-key: it would invalidate every family at once and produce a
repository-wide cold build.

Refs: rustfs/backlog#1598, rustfs/backlog#1600
2026-08-01 18:57:23 +08:00
Zhengchao An 3f4f31129e ci: narrow the io_uring lane and make the sampler attributable (#5540)
Two independent changes to the Test and Lint area.

The io_uring lane compiled 7 integration binaries to run none of their tests.
Job log 91309868055 shows the lib target reporting "18 passed; 3453 filtered
out" while every binary under crates/ecstore/tests/ reported "running 0 tests".
Adding --lib drops them from the build without changing the selected set.

The `uring_` filter itself must not be touched. libtest matches on substring, so
it also selects names containing `during_` — 6 of the 18 selected tests are such
incidental matches, and narrowing the filter to `io_uring` would silently drop
them. scripts/check_uring_lane_lib_only.sh asserts the precondition --lib
depends on: no test function whose name contains `uring_` may live under
crates/ecstore/tests/. A count floor would not do, because the dangerous case —
someone adding a matching test there — leaves the lib count unchanged and CI
green.

The resource sampler was measuring the wrong machine. The runners are ARC pods,
so /proc/loadavg, /proc/pressure/*, `free` and `df` are node-level and include
every other runner pod on the same Kubernetes node: one sample reported loadavg
6.67 with 832 threads node-wide while `ps` inside the pod showed about 10
processes. Judging a CARGO_BUILD_JOBS change on those numbers cannot work.

The sampler moves to scripts/ci/resource_sampler.sh and now records
/sys/fs/cgroup cpu.max, memory.max, memory.peak and the cpu/io/memory pressure
files, which are this pod's own. The node-level readings stay — co-tenancy is a
real cause of stalls, and a 9m57s plain `git checkout` was traced to it — but
are labelled NODE-LEVEL so nobody reads them as this job's load. Phase markers
are written on start, and clippy is now sampled too: it is the natural control
arm for a CARGO_BUILD_JOBS experiment, since --all-targets is check-only for
workspace members and never links the ~100 test binaries the limit throttles.

No numbers are changed in this commit. CARGO_BUILD_JOBS stays at 2 until there
is attributable data to change it on.

Refs: rustfs/backlog#1598, rustfs/backlog#1601
2026-08-01 11:32:00 +08:00