perf(microvm): hit Phase 2 boot/memory gates

t1→t2 hypervisor: 3224 ms → 264 ms (12×). 128 MiB guest now boots
cleanly without OOM, reverting the temporary 512 MiB bump.

Root causes:

1. build-microvm-image.sh staged the entire chroot into the cpio
   without dropping /boot, embedding a duplicate vmlinuz, Alpine's
   stock initramfs-virt, System.map, and a "boot → ." symlink loop
   (~25 MiB). Decompression peak exceeded 128 MiB. Strip /boot after
   the kernel is copied out, before the cpio. Initramfs drops from
   31 MiB → 11 MiB (53 MiB → 28 MiB uncompressed); total artifact
   42 MiB → 22 MiB.

2. vm/lifecycle.go burned 3 s of fixed sleep on every direct-boot
   launch: 2 s nbdkit pre-exec wait (no nbdkit on direct-boot) and
   1 s post-fork settle (QEMU binds QMP within ~10 ms when there is
   no firmware, ROM, or block layer). Both extracted to
   directBoot-aware helpers — nbdkitPreExecWait collapses to 0, and
   qemuStartSettleWait to 50 ms.

3. KernelCmdline only carried `console=ttyS0 virtio_mmio.device=…`.
   Added boot-time perf flags: quiet loglevel=3 (silence printk over
   115200-baud isa-serial), mitigations=off (KVM guest on trusted
   host, single-tenant LB workload), tsc=reliable, no_timer_check,
   reboot=t, i8042.{nopnp,noaux,nokbd} (microvm has no PS/2).

4. init.sh dropped modprobe virtio_mmio (built-in:
   CONFIG_VIRTIO_MMIO=y) and tightened polling: fw_cfg/iface waits go
   from 10 × 200 ms (up to 2 s) to 5 × 20 ms (up to 100 ms).

Phase 2 gates:
  - t1→t2 ≤ 300 ms       ✅ 264 ms / 281 ms
  - artifact ≤ 50 MiB    ✅ 22 MiB (gate already in build script)
  - ELBv2 e2e green      ✅ 32/33 (pre-existing client-VM failure,
                                   unrelated to microvm path)
This commit is contained in:
joshsiv-mulga
2026-05-17 19:41:20 +10:00
parent 509ea34927
commit a5efd45168
7 changed files with 84 additions and 24 deletions
+13 -5
View File
@@ -219,6 +219,19 @@ find "$CHROOT_DIR/usr/sbin" "$CHROOT_DIR/usr/bin" \
strip --strip-all "$bin" 2>/dev/null || true
done
# --- Copy vmlinuz out before stripping /boot from chroot ---
VMLINUZ_OUT="$BUILD_DIR/vmlinuz"
echo "[build-microvm-image] copying kernel: $VMLINUZ_OUT"
cp "$KERNEL_IMG" "$VMLINUZ_OUT"
# Drop /boot from the chroot — vmlinuz, Alpine's stock initramfs-virt, System.map
# and kernel config all live here (~25 MiB). None are needed at runtime: the
# guest already has the kernel loaded by QEMU's -kernel, and Alpine's initramfs
# is unused because we replace it with our own /init. Leaving /boot in the cpio
# nearly doubles uncompressed initramfs size and forces a 512 MiB guest just to
# survive decompression.
rm -rf "$CHROOT_DIR/boot"
# --- Build initramfs ---
INITRAMFS_OUT="$BUILD_DIR/initramfs.cpio.gz"
echo "[build-microvm-image] building initramfs: $INITRAMFS_OUT"
@@ -227,11 +240,6 @@ echo "[build-microvm-image] building initramfs: $INITRAMFS_OUT"
find . | cpio --quiet -o -H newc | gzip -9 > "$INITRAMFS_OUT"
)
# --- Copy vmlinuz ---
VMLINUZ_OUT="$BUILD_DIR/vmlinuz"
echo "[build-microvm-image] copying kernel: $VMLINUZ_OUT"
cp "$KERNEL_IMG" "$VMLINUZ_OUT"
# --- Log artifact sizes ---
echo ""
echo "[build-microvm-image] artifacts:"