mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 09:01:31 +00:00
feat(elbv2): QEMU microvm direct-boot for LB system instances (#228)
* feat(vm): direct-boot foundations + config/input extensions + image build pipeline
vm package (mulga-siv-62):
- Add KernelImage, Initrd, KernelCmdline, FwCfg/FwCfgEntry to vm.Config
- Emit -kernel/-initrd/-append/-fw_cfg args when fields set
- Relax drive invariant: accept kernel image in place of drives
- Add devices.go: IsMMIO predicate, NetDevice/BlkDevice/RngDevice helpers
- Refactor lifecycle.go + network_helpers.go call sites to use helpers
- shutdown.go: os.Remove fw_cfg tmpfiles after QEMU exits
- Add devices_test.go, microvm_test.go; extend vm_test.go
config/input extensions (mulga-siv-63):
- Add MicrovmConfig{ELBv2Enabled, ImagePath} to DaemonConfig
- ImagePath falls back to /usr/share/spinifex/microvm when empty
- Add commented [microvm] section to spinifex.toml template
- Add DirectBoot bool + NICs []NICConfig to SystemInstanceInput
image build pipeline (mulga-siv-64):
- scripts/build-microvm-image.sh: Alpine chroot build producing vmlinuz + initramfs.cpio.gz
- build/microvm/init.sh: fw_cfg-driven PID 1 init (mounts, modprobe, NIC config, lb-agent-env, ca-cert)
- build/microvm/inittab: BusyBox init with lb-agent respawn
- make build-microvm-image target
- setup.sh: install microvm/* to /usr/share/spinifex/microvm/
- .gitignore: exclude built vmlinuz + initramfs.cpio.gz artifacts
* feat(daemon): direct-boot launch path for microvm system instances
When SystemInstanceInput.DirectBoot=true:
- Skip GetAMIConfig and GenerateVolumes
- Build vm.Config directly (no buildBaseVMConfig, no PCIe root ports)
- MachineType=microvm,x-option-roms=off,pic=off,pit=off,rtc=on,acpi=on,isa-serial=on
- Resolve kernel/initrd from Microvm.ImagePath config field
- Serialise NICs to three fw_cfg tmpfiles (netcfg, lb-agent-env, ca-cert)
- Validate exactly one NICConfig.IsDefault=true before writing netcfg blob
- Shutdown path cleans up tmpfiles via existing Config.FwCfg iteration
PC-machine path (DirectBoot=false) unchanged.
Closes mulga-siv-65
* feat(elbv2): wire microvm.elbv2_enabled flag into ELBv2 service
When ELBv2Enabled=true:
- Drop lbVMUserData cloud-config; emit KEY=value LBAgentEnv blob instead
- Set SystemInstanceInput.DirectBoot=true; omit ImageID
- Populate NICs []NICConfig (index 0=primary ENI IsDefault, 1=mgmt, 2+=extra ENIs)
- Validate exactly one IsDefault NIC at service layer
When ELBv2Enabled=false: existing PC-machine path unchanged.
Adds buildLBAgentEnv, buildMicrovmNICs, subnetCIDRForIP, subnetGatewayIP helpers.
Tests in arn_test.go cover both flag-on and flag-off paths.
Closes mulga-siv-66
* test(daemon,vm): add unit tests for direct-boot helpers to meet diff coverage threshold
* feat(daemon): Phase 1 perf instrumentation + e2e runner microvm flag
- Log t1→t2 boot timing via slog on the direct-boot path
- measureQEMURSSMiB helper reads /proc/<pid>/status (tested)
- Gate constants: phase1BootGateMs=500, phase1RSSGateMiB=25
- run-lb-e2e.sh: --microvm flag, dual-path CI annotation, density test note
Closes mulga-siv-67
* fix(build): remove busybox-initscripts — not in Alpine 3.20, not needed
* fix(microvm): end-to-end fixes for direct-boot LB path
Serial output was silent (x-option-roms=off suppresses BIOS serial init);
fixed by removing it from the machine string and using -serial chardev:console0
to attach to the machine-created ISA serial device instead of adding a
duplicate isa-serial device (which would create ttyS1 while kernel uses ttyS0).
virtio-net interfaces were invisible to the kernel because QEMU's
auto-kernel-cmdline does not append virtio_mmio.device= params when -append
is used. Fixed by generating explicit virtio_mmio.device=0x200@0xfebNNNNN:N
params for each NIC in KernelCmdline, and modprobing virtio_mmio + virtio_net
in init.sh.
sys.micro bumped to 512 MiB — 128 MiB caused kernel OOM during initramfs
decompression (~68 MiB uncompressed).
BusyBox init ::respawn: entries do not source or export env files; fixed
inittab to use set -a; . /etc/conf.d/lb-agent; set +a before exec.
wireLBAgentConfig was documented as reading the CA cert but never set
elbv2Service.CACert; lb-agent TLS handshakes to AWSGW failed with
x509: certificate signed by unknown authority. Fixed by reading
d.config.NATS.CACert (same local CA that signs the AWSGW server cert)
and assigning to elbv2Service.CACert so writeFwCfgBlobs writes a
non-empty cacert blob into fw_cfg.
Adds vm_test TestExecute_MicrovmFileChardev and updates lifecycle_test
to assert SerialSocket is intentionally empty on direct-boot VMs.
* fix(build): auto-detect docker/podman when apk unavailable for microvm image build
* fix(build): create /dev/console and /dev/null as proper device nodes in initramfs
tar extraction without root silently demotes device files to 0-byte regular
files (mknod requires CAP_MKNOD). The kernel opens /dev/console when wiring
init's stdio to the serial console; a regular file there causes all init output
to be silently discarded to RAM — the VM appears stuck at 'Run /init as init
process' with no subsequent console output and lb-agent never heartbeats.
Fix: after rootfs population, wipe /dev and recreate console (c 5 1) and
null (c 1 3) via sudo mknod. devtmpfs provides all other devices at runtime
once init.sh mounts it.
* fix(microvm): fix mgmt NIC routing for all LB schemes
Two bugs prevented lb-agent from reaching the AWSGW via the mgmt interface:
1. RouteDst/RouteVia swapped — resolveMgmtRoute returns (gateway, target) but
buildMicrovmNICs assigned them as (Dst=gateway, Via=target), producing
'ip route add 10.15.8.1 via 192.168.1.33' instead of the correct
'ip route add 192.168.1.33 via 10.15.8.1'.
2. NIC[1] MAC and CIDR never filled — the daemon allocates mgmt IP/MAC
after buildMicrovmNICs returns, but didn't inject them into input.NICs[1]
before calling writeFwCfgBlobs, leaving NIC1_MAC= empty in the netcfg
blob and skipping interface setup in init.sh entirely.
Also extends mgmt routing to internet-facing scheme in the microvm path:
VPC SNAT is not reliable across all VPC topologies for control-plane
heartbeats; mgmt is always a direct L2 path to the host.
* feat(microvm): Phase 2 kernel stripping + boot optimisation
- cpu: switch to host,-pmu (disable PMU — not needed in microvm guest)
- machine type: make isa-serial configurable via microvm.isa_serial config
field; default true (on) preserves console log path
- build: strip unused kernel modules (USB, IDE, ATA, GPU, media, input,
HID, bluetooth, staging, sound, wireless) from initramfs before cpio
- build: add Phase 2 artifact size gate — fail if vmlinuz + initramfs.cpio.gz
exceeds MICROVM_ARTIFACT_MAX_MiB (default 50 MiB); override via env var
during development
- perf: add phase2BootGateMs=300 and phase2ArtifactGateMiB=50 constants
with canary tests to catch unintentional gate changes
- tests: cover buildMicrovmNICs nil-VPC and extra-ENI paths
- tests: cover microvmMachineType nil/true/false branches
* fix(build): use find-based virtio+fw_cfg module keep-list in initramfs strip
The directory-only keep-list (drivers/virtio, drivers/firmware) missed
virtio_net.ko which lives in drivers/net/, virtio_blk in drivers/block/,
and virtio-rng in drivers/char/hw_random/. Switch to find-based approach:
collect all virtio*.ko* and *fw_cfg*.ko* regardless of subdirectory, then
restore them into their original relative paths under kernel/.
Artifacts now: 42 MiB (vmlinuz 11 MiB + initramfs 31 MiB), gate passes.
* fix(build): keep failover modules in initramfs strip
virtio_net depends on net_failover/failover for live-migration support.
Stripping them left virtio_net with unresolved symbols
(net_failover_create/destroy), so the module failed to load and the
guest came up with only lo. init.sh then skipped every NIC ("no
interface found for MAC ..."), leaving lb-agent with no network path
to AWSGW and the LB stuck in provisioning.
Add *failover*.ko* to the keep-list. Image size unchanged at 42 MiB
(under the 50 MiB Phase 2 gate). ELBv2 e2e now reaches active and
2/2 targets healthy for both ALB and NLB internal LBs.
* fix(daemon): restore slog default after viperblock.New mutates it
viperblock.New unconditionally calls vb.SetDebug, which calls
slog.SetDefault with a TextHandler at LevelError — globally silencing
every slog.Info and slog.Warn in the caller process for the rest of
its life. Discovered while validating microvm Phase 2 (mulga-siv-68):
the daemon stopped emitting Info logs the moment the first customer
VM with volumes launched, hiding direct-boot timing,
LaunchSystemInstance completed, and CreateLoadBalancer completed
lines that we need for boot-time tuning.
Workaround in newViperblock: re-install an Info-level TextHandler on
os.Stderr after each viperblock.New call. Also drop the three explicit
vb.SetDebug(false) calls in prepareRootVolume / prepareEFIVolume /
prepareCloudInitVolume — they were no-ops (viperblock.New already
runs SetDebug internally) and each one re-triggered the global
mutation we are trying to undo.
Proper fix is in viperblock itself — tracked as mulga-siv-70: SetDebug
should not call slog.SetDefault. Until then, this restore call is the
shortest path to getting the daemon's observability back so we can
measure t1_to_t2_ms and tune Phase 2.
Preflight note: golangci-lint not available on this dev host, so the
project preflight target cannot run end-to-end. Verified with
go vet ./spinifex/handlers/ec2/instance/... (clean) and
go test ./spinifex/handlers/ec2/instance/... (pass).
* perf(microvm): hit Phase 2 boot/memory gates
t1→t2 hypervisor: 3224 ms → 264 ms (12×). 128 MiB guest now boots
cleanly without OOM, reverting the temporary 512 MiB bump.
Root causes:
1. build-microvm-image.sh staged the entire chroot into the cpio
without dropping /boot, embedding a duplicate vmlinuz, Alpine's
stock initramfs-virt, System.map, and a "boot → ." symlink loop
(~25 MiB). Decompression peak exceeded 128 MiB. Strip /boot after
the kernel is copied out, before the cpio. Initramfs drops from
31 MiB → 11 MiB (53 MiB → 28 MiB uncompressed); total artifact
42 MiB → 22 MiB.
2. vm/lifecycle.go burned 3 s of fixed sleep on every direct-boot
launch: 2 s nbdkit pre-exec wait (no nbdkit on direct-boot) and
1 s post-fork settle (QEMU binds QMP within ~10 ms when there is
no firmware, ROM, or block layer). Both extracted to
directBoot-aware helpers — nbdkitPreExecWait collapses to 0, and
qemuStartSettleWait to 50 ms.
3. KernelCmdline only carried `console=ttyS0 virtio_mmio.device=…`.
Added boot-time perf flags: quiet loglevel=3 (silence printk over
115200-baud isa-serial), mitigations=off (KVM guest on trusted
host, single-tenant LB workload), tsc=reliable, no_timer_check,
reboot=t, i8042.{nopnp,noaux,nokbd} (microvm has no PS/2).
4. init.sh dropped modprobe virtio_mmio (built-in:
CONFIG_VIRTIO_MMIO=y) and tightened polling: fw_cfg/iface waits go
from 10 × 200 ms (up to 2 s) to 5 × 20 ms (up to 100 ms).
Phase 2 gates:
- t1→t2 ≤ 300 ms ✅ 264 ms / 281 ms
- artifact ≤ 50 MiB ✅ 22 MiB (gate already in build script)
- ELBv2 e2e green ✅ 32/33 (pre-existing client-VM failure,
unrelated to microvm path)
* build(microvm): wire build into deploy + distro + release pipeline
Adds install-microvm Makefile target (with grouped-target staleness
detection on vmlinuz/initramfs.cpio.gz), wires it into dev-install.sh
and reset-dev-env.sh, makes 'make deploy' reinstall artifacts when
present, and makes 'make distro-amd64' stage build/microvm/* into the
release tarball staging dir. arm64 distro stays unchanged (no aarch64
microvm artifact yet — Phase 4).
Release workflow gains a setup-go step and a 'make build-microvm-image'
step before 'make distro-amd64' on the amd64 runner so the resulting
tarball includes the kernel + initramfs without a manual copy.
Precursor for the Phase 3 PC-machine LB removal (mulga-siv-69).
Closes mulga-siv-71.
* build(ansible): include microvm artifacts in build role tarball
Adds a build-microvm-image task to the ansible build role and stages
build/microvm/* into the role's release tarball so the install role's
setup.sh run picks them up via INSTALL_SPINIFEX_TARBALL. Skipped
gracefully when the build script is absent (older branches without
direct-boot LB wiring).
* feat(config): enable microvm direct-boot LB path by default
Uncomments the [nodes.<node>.daemon.microvm] section in the generated
spinifex.toml template and sets elbv2_enabled = true so all new
clusters bring up ELBv2 LBs via the direct-boot microvm path. Operators
can opt back to the legacy PC-machine path by flipping the flag to
false in the rendered config.
Required for end-to-end CI on this branch to exercise the direct-boot
path. Existing clusters' configs are unchanged (the template only
affects fresh 'spx admin init' renders).
* fix(microvm): bootstrap apk.static when no host rootfs tool exists
Self-hosted CI runners and the GitHub-hosted ubuntu-24.04 image often
have none of apk, docker, or podman installed. apk-tools is not in the
Debian 13 (trixie) repositories. Without one of these the rootfs build
fails up front with 'no rootfs build tool available'.
When all three are absent, fetch apk-tools-static from the configured
Alpine mirror, extract it to a tempdir, and prepend it to PATH so the
existing native-apk strategy below picks it up transparently. Requires
only curl + tar, both of which are present on every CI runner that can
already check out source.
No-op when apk, docker, or podman is already present.
Unblocks https://github.com/mulgadc/spinifex/actions/runs/25988830244
* revert: drop apk.static bootstrap in build-microvm-image
Reverts 0bd76b7. Self-hosted runners will get docker.io installed
out of band; existing Strategy 2 (docker run alpine:3.20) handles
the rootfs build without host sudo or unsigned package install.
* docs(workbook): drop lb-alpine import for microvm direct-boot LB
LBs now launch via direct-boot QEMU microvm using kernel + initramfs
bundled at /usr/share/spinifex/microvm/, so the legacy lb-alpine system
AMI import is no longer required.
* fix(daemon): own tap devices by daemon user so QEMU can attach without CAP_NET_ADMIN
When the daemon runs non-root (production: User=spinifex-daemon), child
QEMU processes inherit that uid but lack CAP_NET_ADMIN. The kernel's tun
attach check then requires the tap owner uid to match the caller's euid.
Without the `user` flag, `ip tuntap add` defaults to root:root and qemu's
TUNSETIFF fails with EPERM ("could not configure /dev/net/tun: Operation
not permitted").
Surfaced by the e2e-nightly cell-17 microvm LB launch on the ci-pseudo
runner where the kernel enforces strictly; legacy EC2 launches on
ci-single runners happened to attach successfully despite the same code
path. Adding `user $current group $gid` to `ip tuntap add` ensures the
tap is owned by whatever uid the daemon (and its qemu children) run as.
Root daemons skip the flag — they have CAP_NET_ADMIN ambient.
* fix(e2e): redirect lb probe chatter to stderr so stalls are visible
test_http_from / test_tcp_from echoed progress to stdout, which the
callers captured into *_OK vars via $(...). Result: 6+ minute retry
loops ran silently in CI, and the "$ok" comparison never matched
"true" because the captured value contained the full multi-line
chatter. Reachability pass/fail counters were also lost in the
subshell.
Redirect helper output to stderr, leave only the true/false flag on
stdout, and move pass/fail to the caller so they actually count.
* fix(microvm): don't /32 AdvertiseIP via mgmt for internet-facing single-node
buildMicrovmNICs overrode resolveMgmtRoute's deliberate empty return
for the single-node internet-facing case by falling back to
(MgmtBridgeIP, AdvertiseIP). That installed a /32 route to AdvertiseIP
via the mgmt NIC inside the LB microvm. On single-node where the
AWSGW host IP equals the WAN client IP, the /32 also catches reply
traffic for inbound HTTP/TCP to the LB's floating IP — replies leave
via the mgmt NIC with src=data-NIC IP, the host kernel drops the
asymmetric flow, and same-chassis ingress hangs.
resolveMgmtRoute already encodes the correct policy:
- multi-node any scheme → explicit MgmtRoute{Gateway,Target}
- single-node internal → fallback to (MgmtBridgeIP, AdvertiseIP)
- single-node internet-facing → empty (reach AWSGW via VPC SNAT)
Honor it. Matches the legacy lbVMUserData path, whose own comment
flagged the stolen-WAN-return-path symptom this introduced.
Test covers all three branches.
* feat(ansible): dev-deploy playbook for in-place artifact swap
Fills the gap between ansible-dev-install (skips when already installed)
and ansible-dev-reset (full teardown + reinstall + smoketest). Mirrors
Makefile `deploy` target: rebuild + replace binaries and microvm
artifacts on a running node, restart spinifex.target, wait for awsgw.
Skips setup.sh (no user/dir/systemd-unit changes) and smoketest so
iteration is fast. Use ansible-dev-reset when changing systemd units,
helper scripts, logrotate, or setup.sh itself.
Adds:
- roles/deploy/tasks/main.yml — install spx + nbdkit plugin + microvm
artifacts, with plugindir auto-detection via `nbdkit --dump-config`.
- playbooks/dev-deploy.yml — chains build + deploy roles.
- Makefile ansible-dev-deploy target.
- README playbook table + "when to use which" guidance.
* chore(microvm): remove legacy LB Alpine image build pipeline
Phase 3 microvm removal — strip the PC-machine LB image build
artifacts that direct-boot no longer needs. The microvm initramfs
already carries lb-agent + HAProxy via build-microvm-image, so the
lb.conf manifest, lb-setup.sh chroot script, build-lb-image target,
dev-install / reset-dev-env import steps, and ansible LB-image tasks
all become dead.
build-system-image stays — other manifests (e.g. GPU images) still
use it.
* refactor(microvm): remove legacy PC-machine LB boot path
Phase 3 — direct-boot becomes the only ELBv2 LB launch path.
- Drop SystemInstanceInput.DirectBoot / ImageID / UserData (every LB
caller set DirectBoot=true; legacy cloud-init userdata is gone).
- Drop ELBv2ServiceImpl.lbVMUserData, systemAMI* fields, SetSystemAMIFunc,
getSystemAMI, and the lazy-AMI wiring in daemon.startCluster.
- Collapse CreateLoadBalancer to a single direct-boot launch input.
- Collapse daemon.LaunchSystemInstance: drop AMI validation, ImageID
on RunInstancesInput, UserData base64 encode, GenerateVolumes branch.
buildDirectBootConfig hardcodes /usr/share/spinifex/microvm and
isa-serial=on (microvmMachineType no longer takes a flag).
- Drop MicrovmConfig from config.DaemonConfig and the
[daemon.microvm] section from the spinifex.toml template.
- Prune dead tests (lbVMUserData_*, PCMachine_Unchanged, AMI resolver
paths) and refresh stale comments that still mentioned lbVMUserData.
ARM hosts have no LB launch path until Phase 4 adds direct-boot on
virt — CreateLoadBalancer on aarch64 fails naturally at QEMU exec.
* docs: drop lb-alpine catalog entry from cluster setup guide
LB system instances no longer use an AMI — direct-boot microvm
artifacts ship in the spinifex package.
* feat(ui): remove LB image import banner + LB-AMI gating
Direct-boot microvm artifacts ship in the spinifex package, so the
LB system AMI no longer exists — the "Load balancer image not imported"
banner and the Create-button disable on the LB list / wizard pages
can never fire. Strip:
- lb-image-missing-banner.tsx component
- hasLbImage / isLbImage / LB_IMAGE_NAME / LB_MANAGED_BY_VALUE from
system-managed.ts (isSystemManagedImage stays — still used by the
EC2 Images pages to hide system AMIs)
- Banner gating + ec2ImagesQueryOptions imports from the LB list and
wizard pages
- The corresponding tests that asserted the banner appears
* revert: restore lb.conf + lb-setup.sh for mulga install pipeline
Phase 3 microvm removal (commit 6313d74) deleted these too early — the
mulga install/iso artifact builders (scripts/tofu-cluster/build-install
-artifacts.sh, scripts/tofu-iso/build-iso-artifacts.sh) still invoke
scripts/build-system-image.sh scripts/images/lb.conf, breaking e2e with
"ERROR: Manifest not found: scripts/images/lb.conf".
Restore both files verbatim so the legacy pipeline keeps building until
the mulga-side migration to microvm-only artifacts lands post-merge.
This commit is contained in:
Josh Siviour
@@ -31,6 +31,14 @@ jobs:
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: '1.26.3'
|
||||
|
||||
- name: Build microVM artifacts (lb-agent + kernel + initramfs)
|
||||
run: cd spinifex && make build-microvm-image
|
||||
|
||||
- name: Build amd64 Tarball
|
||||
env:
|
||||
VERSION: ${{ github.ref_name }}
|
||||
|
||||
@@ -17,6 +17,8 @@ go.work.sum
|
||||
# Build outputs
|
||||
bin/
|
||||
dist/
|
||||
build/microvm/vmlinuz
|
||||
build/microvm/initramfs.cpio.gz
|
||||
!spinifex/services/spinifexui/frontend/dist/
|
||||
tmp/
|
||||
*.test
|
||||
|
||||
@@ -79,8 +79,25 @@ ifndef IMAGE
|
||||
endif
|
||||
./scripts/build-system-image.sh scripts/images/$(IMAGE).conf
|
||||
|
||||
build-lb-image: ## Build LB Alpine image
|
||||
$(MAKE) build-system-image IMAGE=lb
|
||||
MICROVM_OUT_DIR := build/microvm
|
||||
MICROVM_ARTIFACTS := $(MICROVM_OUT_DIR)/vmlinuz $(MICROVM_OUT_DIR)/initramfs.cpio.gz
|
||||
MICROVM_INPUTS := scripts/build-microvm-image.sh $(MICROVM_OUT_DIR)/init.sh $(MICROVM_OUT_DIR)/inittab bin/lb-agent
|
||||
|
||||
# Grouped target — script writes both files in one run.
|
||||
$(MICROVM_ARTIFACTS) &: $(MICROVM_INPUTS)
|
||||
./scripts/build-microvm-image.sh
|
||||
|
||||
# Only triggers when bin/lb-agent is missing; preserves the artifact's mtime so
|
||||
# build-microvm-image stays correctly stale-aware.
|
||||
bin/lb-agent:
|
||||
$(MAKE) build-lb-agent
|
||||
|
||||
build-microvm-image: $(MICROVM_ARTIFACTS) ## Build microVM kernel + initramfs (incremental — skips when up to date)
|
||||
|
||||
install-microvm: $(MICROVM_ARTIFACTS) ## Install microVM artifacts to /usr/share/spinifex/microvm/
|
||||
sudo install -d /usr/share/spinifex/microvm
|
||||
sudo install -m 0644 $(MICROVM_OUT_DIR)/vmlinuz /usr/share/spinifex/microvm/vmlinuz
|
||||
sudo install -m 0644 $(MICROVM_OUT_DIR)/initramfs.cpio.gz /usr/share/spinifex/microvm/initramfs.cpio.gz
|
||||
|
||||
go_run:
|
||||
@echo -e "\n....Running $(GO_PROJECT_NAME)...."
|
||||
@@ -122,9 +139,18 @@ run:
|
||||
$(MAKE) go_build
|
||||
$(MAKE) go_run
|
||||
|
||||
# Fast iteration: build + install binary + restart all services
|
||||
# Fast iteration: build + install binary + restart all services.
|
||||
# Microvm artifacts are reinstalled when they already exist on disk — the rule's
|
||||
# input timestamps drive a rebuild only if anything actually changed. On a fresh
|
||||
# checkout (no build/microvm/vmlinuz yet) the install step is skipped; run
|
||||
# `make install-microvm` explicitly the first time.
|
||||
deploy: build
|
||||
sudo install -m 755 bin/spx /usr/local/bin/spx
|
||||
@if [ -f $(MICROVM_OUT_DIR)/vmlinuz ]; then \
|
||||
$(MAKE) install-microvm; \
|
||||
else \
|
||||
echo "[deploy] microvm artifacts absent — run 'make install-microvm' for first-time setup"; \
|
||||
fi
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart spinifex.target
|
||||
|
||||
@@ -206,6 +232,14 @@ distro-amd64:
|
||||
-f build/Dockerfile.distro \
|
||||
--output type=local,dest=dist/amd64/ \
|
||||
../
|
||||
@if [ -f $(MICROVM_OUT_DIR)/vmlinuz ] && [ -f $(MICROVM_OUT_DIR)/initramfs.cpio.gz ]; then \
|
||||
echo "[distro-amd64] staging microvm artifacts into tarball"; \
|
||||
mkdir -p dist/amd64/microvm; \
|
||||
cp $(MICROVM_OUT_DIR)/vmlinuz $(MICROVM_OUT_DIR)/initramfs.cpio.gz dist/amd64/microvm/; \
|
||||
else \
|
||||
echo "[distro-amd64] WARNING: microvm artifacts missing — tarball will not include them"; \
|
||||
echo "[distro-amd64] run 'make build-microvm-image' before 'make distro-amd64'"; \
|
||||
fi
|
||||
tar -czf dist/spinifex-$(VERSION)-linux-amd64.tar.gz -C dist/amd64 .
|
||||
sha256sum dist/spinifex-$(VERSION)-linux-amd64.tar.gz > dist/spinifex-$(VERSION)-linux-amd64.tar.gz.sha256
|
||||
|
||||
@@ -238,9 +272,12 @@ ansible-dev-install:
|
||||
ansible-dev-reset:
|
||||
cd scripts/ansible && ansible-playbook playbooks/dev-reset.yml
|
||||
|
||||
.PHONY: build build-ui build-installer build-lb-agent build-system-image build-lb-image go_build go_run preflight test test-cover test-race diff-coverage bench run \
|
||||
ansible-dev-deploy:
|
||||
cd scripts/ansible && ansible-playbook playbooks/dev-deploy.yml
|
||||
|
||||
.PHONY: build build-ui build-installer build-lb-agent build-system-image build-microvm-image install-microvm go_build go_run preflight test test-cover test-race diff-coverage bench run \
|
||||
deploy reinstall clean \
|
||||
install-system install-go install-aws quickinstall \
|
||||
lint fix govulncheck \
|
||||
distro distro-amd64 distro-arm64 distro-clean \
|
||||
ansible-dev-preflight ansible-dev-teardown ansible-dev-install ansible-dev-reset
|
||||
ansible-dev-preflight ansible-dev-teardown ansible-dev-install ansible-dev-reset ansible-dev-deploy
|
||||
|
||||
Executable
+56
@@ -0,0 +1,56 @@
|
||||
#!/bin/sh
|
||||
mount -t proc proc /proc
|
||||
mount -t sysfs sys /sys
|
||||
mount -t devtmpfs dev /dev
|
||||
|
||||
# virtio_mmio is built-in (CONFIG_VIRTIO_MMIO=y) so virtio-net devices are
|
||||
# already enumerated by the time PID 1 runs — no modprobe needed.
|
||||
# qemu_fw_cfg and virtio_net are =m in stock Alpine linux-virt, so load them.
|
||||
modprobe qemu_fw_cfg 2>/dev/null || modprobe fw_cfg_sysfs 2>/dev/null || true
|
||||
modprobe virtio_net 2>/dev/null || true
|
||||
|
||||
for i in 1 2 3 4 5; do
|
||||
[ -f /sys/firmware/qemu_fw_cfg/by_name/opt/spinifex/netcfg/raw ] && break
|
||||
sleep 0.02
|
||||
done
|
||||
|
||||
if [ ! -f /sys/firmware/qemu_fw_cfg/by_name/opt/spinifex/netcfg/raw ]; then
|
||||
echo "[init] FATAL: fw_cfg netcfg not available" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
. /sys/firmware/qemu_fw_cfg/by_name/opt/spinifex/netcfg/raw
|
||||
|
||||
for n in 0 1 2 3 4 5; do
|
||||
eval mac=\$NIC${n}_MAC
|
||||
[ -z "$mac" ] && continue
|
||||
|
||||
iface=""
|
||||
for attempt in 1 2 3 4 5; do
|
||||
iface=$(ip -o link | awk -v m="$mac" 'tolower($0) ~ tolower(m) {print $2}' | tr -d :)
|
||||
[ -n "$iface" ] && break
|
||||
sleep 0.02
|
||||
done
|
||||
|
||||
if [ -z "$iface" ]; then
|
||||
echo "[init] WARNING: no interface found for MAC $mac (NIC$n), skipping" >&2
|
||||
continue
|
||||
fi
|
||||
|
||||
eval cidr=\$NIC${n}_CIDR
|
||||
eval gw=\$NIC${n}_GW
|
||||
eval default=\$NIC${n}_DEFAULT
|
||||
eval rdst=\$NIC${n}_ROUTE_DST
|
||||
eval rvia=\$NIC${n}_ROUTE_VIA
|
||||
|
||||
ip link set "$iface" up
|
||||
[ -n "$cidr" ] && ip addr add "$cidr" dev "$iface"
|
||||
[ "$default" = "1" ] && [ -n "$gw" ] && ip route add default via "$gw" dev "$iface"
|
||||
[ -n "$rdst" ] && [ -n "$rvia" ] && ip route add "$rdst" via "$rvia" dev "$iface"
|
||||
done
|
||||
|
||||
cp /sys/firmware/qemu_fw_cfg/by_name/opt/spinifex/lb-agent-env/raw /etc/conf.d/lb-agent
|
||||
mkdir -p /etc/ssl/certs
|
||||
cp /sys/firmware/qemu_fw_cfg/by_name/opt/spinifex/ca-cert/raw /etc/ssl/certs/ca-certificates.crt
|
||||
|
||||
exec /sbin/init
|
||||
@@ -0,0 +1,4 @@
|
||||
::sysinit:/etc/rc.local
|
||||
::respawn:/bin/sh -c 'set -a; . /etc/conf.d/lb-agent; set +a; exec /usr/local/bin/lb-agent'
|
||||
::ctrlaltdel:/sbin/reboot
|
||||
::shutdown:/bin/umount -a -r
|
||||
@@ -60,7 +60,6 @@ spx admin images list
|
||||
NAME | DISTRO | VERSION | ARCH | BOOT
|
||||
debian-13-arm64 | debian | 13 | arm64 | bios
|
||||
debian-13-x86_64 | debian | 13 | x86_64 | bios
|
||||
lb-alpine-3.21.6-x86_64 | alpine | 3.21.6 | x86_64 | bios
|
||||
ubuntu-26.04-arm64 | ubuntu | 26.04 | arm64 | bios
|
||||
ubuntu-26.04-x86_64 | ubuntu | 26.04 | x86_64 | bios
|
||||
```
|
||||
|
||||
@@ -86,22 +86,16 @@ Or create a `main.tf` file and paste the full configuration below.
|
||||
|
||||
<!-- INCLUDE: main.tf lang:hcl -->
|
||||
|
||||
### Step 2. Install Load Balancer AMI
|
||||
### Step 2. Install Debian AMI
|
||||
|
||||
Next install the load balancer AMI images, which is used as the AMI disk image for launching the load-balancer and a requirement.
|
||||
|
||||
```bash
|
||||
spx admin images import --name lb-alpine-3.21.6-x86_64
|
||||
```
|
||||
|
||||
### Step 2.1 Install Debian AMI
|
||||
|
||||
Next, install the Debian 13 AMI which is used in the example to host the `nginx` webservers as an EC2 instance.
|
||||
Install the Debian 13 AMI which is used in the example to host the `nginx` webservers as an EC2 instance.
|
||||
|
||||
```bash
|
||||
spx admin images import --name debian-13-x86_64
|
||||
```
|
||||
|
||||
> **Note:** The load balancer itself runs as a direct-boot QEMU microvm using the kernel + initramfs bundled with the Spinifex distribution (`/usr/share/spinifex/microvm/`), so no separate LB AMI import is required.
|
||||
|
||||
### Step 3. Deploy
|
||||
|
||||
```bash
|
||||
|
||||
@@ -31,6 +31,7 @@ inventory and roles relative to CWD.
|
||||
| `playbooks/dev-teardown.yml` | Wipe state only (no reinstall) | `reset-dev-env.sh` §teardown |
|
||||
| `playbooks/dev-install.yml` | Build + install + init + smoketest on clean box | `dev-install.sh` |
|
||||
| `playbooks/dev-reset.yml` | Capture settings → teardown → build → install → init → smoketest | `reset-dev-env.sh` (full) |
|
||||
| `playbooks/dev-deploy.yml` | Rebuild + swap binaries/microvm artifacts + restart (no setup.sh, no smoketest) | `make deploy` |
|
||||
|
||||
Upcoming (not yet implemented):
|
||||
|
||||
@@ -44,6 +45,7 @@ ansible-playbook playbooks/dev-preflight.yml
|
||||
ansible-playbook playbooks/dev-teardown.yml
|
||||
ansible-playbook playbooks/dev-install.yml
|
||||
ansible-playbook playbooks/dev-reset.yml
|
||||
ansible-playbook playbooks/dev-deploy.yml
|
||||
```
|
||||
|
||||
Or via `make` (from `spinifex/`):
|
||||
@@ -53,8 +55,16 @@ make ansible-dev-preflight
|
||||
make ansible-dev-teardown
|
||||
make ansible-dev-install
|
||||
make ansible-dev-reset
|
||||
make ansible-dev-deploy
|
||||
```
|
||||
|
||||
### When to use which
|
||||
|
||||
- First-time / clean box → `ansible-dev-install`
|
||||
- Iterate on Go code, microvm initramfs, lb-agent → `ansible-dev-deploy` (fast)
|
||||
- Changed systemd units, helper scripts, logrotate, setup.sh → `ansible-dev-reset` (slow, full rebuild)
|
||||
- Need a clean slate without reinstall → `ansible-dev-teardown`
|
||||
|
||||
## Variable overrides
|
||||
|
||||
Defaults in `vars/defaults.yml`. Override per run with `-e`:
|
||||
@@ -74,7 +84,6 @@ Useful overrides:
|
||||
| `spinifex_region` | `ap-southeast-2` | Region passed to `spx admin init` |
|
||||
| `spinifex_az` | `{{ region }}a` | AZ passed to `spx admin init` |
|
||||
| `spinifex_external_mode` | `pool` | `pool` \| `nat` external networking |
|
||||
| `spinifex_build_lb_image` | `true` | Build + import LB system image after init |
|
||||
|
||||
`dev-reset.yml` automatically captures `region`, `az`, `external_mode`,
|
||||
pool range, gateway, prefix-len, nat gateway_ip from the existing
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
# In-place artifact swap: rebuilds + replaces binaries/microvm artifacts on
|
||||
# an already-installed node, then restarts spinifex.target. Use when iterating
|
||||
# on code without altering systemd units, users, dirs, or running the full
|
||||
# smoketest. For first-time or unit/script changes use dev-install / dev-reset.
|
||||
- name: Dev deploy — build + swap binaries + restart (no setup.sh, no smoketest)
|
||||
hosts: localhost
|
||||
gather_facts: true
|
||||
vars_files:
|
||||
- ../vars/defaults.yml
|
||||
|
||||
roles:
|
||||
- build
|
||||
- deploy
|
||||
@@ -11,6 +11,18 @@
|
||||
chdir: "{{ mulga_root }}/viperblock"
|
||||
changed_when: true
|
||||
|
||||
- name: build — detect microVM build script (present only on direct-boot branches)
|
||||
ansible.builtin.stat:
|
||||
path: "{{ spinifex_root }}/scripts/build-microvm-image.sh"
|
||||
register: build_microvm_script
|
||||
|
||||
- name: build — microVM kernel + initramfs (direct-boot LBs)
|
||||
ansible.builtin.command:
|
||||
cmd: make build-microvm-image
|
||||
chdir: "{{ spinifex_root }}"
|
||||
changed_when: true
|
||||
when: build_microvm_script.stat.exists
|
||||
|
||||
- name: build — verify build artifacts present
|
||||
ansible.builtin.stat:
|
||||
path: "{{ item }}"
|
||||
@@ -40,9 +52,15 @@
|
||||
mkdir -p "$STAGING/scripts"
|
||||
cp "{{ spinifex_root }}/build/scripts/"* "$STAGING/scripts/"
|
||||
cp "{{ spinifex_root }}/build/logrotate/spinifex" "$STAGING/logrotate-spinifex"
|
||||
# microVM artifacts ship only when the branch built them.
|
||||
if [ -f "{{ spinifex_root }}/build/microvm/vmlinuz" ] && [ -f "{{ spinifex_root }}/build/microvm/initramfs.cpio.gz" ]; then
|
||||
mkdir -p "$STAGING/microvm"
|
||||
cp "{{ spinifex_root }}/build/microvm/vmlinuz" "$STAGING/microvm/"
|
||||
cp "{{ spinifex_root }}/build/microvm/initramfs.cpio.gz" "$STAGING/microvm/"
|
||||
fi
|
||||
tar czf "{{ build_tarball_path }}" -C "$STAGING" .
|
||||
changed_when: true
|
||||
|
||||
- name: build — summary
|
||||
ansible.builtin.debug:
|
||||
msg: "Built spx + nbdkit plugin, tarball at {{ build_tarball_path }}"
|
||||
msg: "Built spx + nbdkit plugin{{ ' + microvm artifacts' if build_microvm_script.stat.exists else '' }}, tarball at {{ build_tarball_path }}"
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
---
|
||||
# In-place artifact swap for an already-installed spinifex node. Mirrors
|
||||
# `make deploy`: replace binaries + microvm artifacts, restart services.
|
||||
# Does NOT run setup.sh (no user/dir/systemd-unit changes) or smoketest.
|
||||
|
||||
- name: deploy — fail fast if spinifex isn't already installed
|
||||
ansible.builtin.stat:
|
||||
path: "{{ spinifex_config_file }}"
|
||||
register: deploy_installed_stat
|
||||
failed_when: not deploy_installed_stat.stat.exists
|
||||
|
||||
- name: deploy — assert config present
|
||||
ansible.builtin.assert:
|
||||
that: deploy_installed_stat.stat.exists
|
||||
fail_msg: |
|
||||
{{ spinifex_config_file }} missing — this playbook only updates an
|
||||
existing install. Run ansible-dev-install or ansible-dev-reset first.
|
||||
|
||||
- name: deploy — verify build artifacts present
|
||||
ansible.builtin.stat:
|
||||
path: "{{ item }}"
|
||||
register: deploy_artifacts
|
||||
loop:
|
||||
- "{{ spinifex_root }}/bin/spx"
|
||||
- "{{ mulga_root }}/viperblock/lib/nbdkit-viperblock-plugin.so"
|
||||
|
||||
- name: deploy — assert artifacts exist after build role
|
||||
ansible.builtin.assert:
|
||||
that: item.stat.exists
|
||||
fail_msg: "Build artifact missing: {{ item.item }} — build role should have produced it"
|
||||
loop: "{{ deploy_artifacts.results }}"
|
||||
loop_control:
|
||||
label: "{{ item.item }}"
|
||||
|
||||
- name: deploy — detect microvm artifacts (direct-boot branches only)
|
||||
ansible.builtin.stat:
|
||||
path: "{{ spinifex_root }}/build/microvm/{{ item }}"
|
||||
register: deploy_microvm_stat
|
||||
loop:
|
||||
- vmlinuz
|
||||
- initramfs.cpio.gz
|
||||
|
||||
- name: deploy — set microvm presence fact
|
||||
ansible.builtin.set_fact:
|
||||
deploy_microvm_present: "{{ deploy_microvm_stat.results | map(attribute='stat.exists') | min }}"
|
||||
|
||||
- name: deploy — detect nbdkit plugin directory
|
||||
ansible.builtin.shell: |
|
||||
set -e
|
||||
nbdkit --dump-config | awk -F= '/^plugindir=/{print $2; exit}'
|
||||
register: deploy_plugindir_cmd
|
||||
changed_when: false
|
||||
|
||||
- name: deploy — set plugin path fact
|
||||
ansible.builtin.set_fact:
|
||||
deploy_plugindir: "{{ deploy_plugindir_cmd.stdout | trim }}"
|
||||
|
||||
- name: deploy — assert plugindir resolved
|
||||
ansible.builtin.assert:
|
||||
that: deploy_plugindir | length > 0
|
||||
fail_msg: "Could not detect nbdkit plugin directory via `nbdkit --dump-config`"
|
||||
|
||||
- name: deploy — stop spinifex.target before binary swap
|
||||
become: true
|
||||
ansible.builtin.systemd:
|
||||
name: spinifex.target
|
||||
state: stopped
|
||||
failed_when: false
|
||||
|
||||
- name: deploy — install spx binary
|
||||
become: true
|
||||
ansible.builtin.copy:
|
||||
src: "{{ spinifex_root }}/bin/spx"
|
||||
dest: /usr/local/bin/spx
|
||||
mode: "0755"
|
||||
owner: root
|
||||
group: root
|
||||
|
||||
- name: deploy — install viperblock nbdkit plugin
|
||||
become: true
|
||||
ansible.builtin.copy:
|
||||
src: "{{ mulga_root }}/viperblock/lib/nbdkit-viperblock-plugin.so"
|
||||
dest: "{{ deploy_plugindir }}/nbdkit-viperblock-plugin.so"
|
||||
mode: "0755"
|
||||
owner: root
|
||||
group: root
|
||||
|
||||
- name: deploy — ensure microvm dir
|
||||
become: true
|
||||
ansible.builtin.file:
|
||||
path: /usr/share/spinifex/microvm
|
||||
state: directory
|
||||
mode: "0755"
|
||||
owner: root
|
||||
group: root
|
||||
when: deploy_microvm_present
|
||||
|
||||
- name: deploy — install microvm kernel
|
||||
become: true
|
||||
ansible.builtin.copy:
|
||||
src: "{{ spinifex_root }}/build/microvm/vmlinuz"
|
||||
dest: /usr/share/spinifex/microvm/vmlinuz
|
||||
mode: "0644"
|
||||
owner: root
|
||||
group: root
|
||||
when: deploy_microvm_present
|
||||
|
||||
- name: deploy — install microvm initramfs
|
||||
become: true
|
||||
ansible.builtin.copy:
|
||||
src: "{{ spinifex_root }}/build/microvm/initramfs.cpio.gz"
|
||||
dest: /usr/share/spinifex/microvm/initramfs.cpio.gz
|
||||
mode: "0644"
|
||||
owner: root
|
||||
group: root
|
||||
when: deploy_microvm_present
|
||||
|
||||
- name: deploy — daemon-reload (covers any unit changes shipped via setup.sh later)
|
||||
become: true
|
||||
ansible.builtin.systemd:
|
||||
daemon_reload: true
|
||||
|
||||
- name: deploy — start spinifex.target
|
||||
become: true
|
||||
ansible.builtin.systemd:
|
||||
name: spinifex.target
|
||||
state: started
|
||||
|
||||
- name: deploy — wait for awsgw to respond
|
||||
become: true
|
||||
become_user: "{{ spinifex_invoking_user }}"
|
||||
environment:
|
||||
AWS_PROFILE: spinifex
|
||||
ansible.builtin.command: aws ec2 describe-regions
|
||||
register: deploy_awsgw
|
||||
retries: 15
|
||||
delay: 2
|
||||
until: deploy_awsgw.rc == 0
|
||||
changed_when: false
|
||||
|
||||
- name: deploy — summary
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
Deployed:
|
||||
spx → /usr/local/bin/spx
|
||||
nbdkit plugin → {{ deploy_plugindir }}/nbdkit-viperblock-plugin.so
|
||||
microvm → {{ '/usr/share/spinifex/microvm/{vmlinuz,initramfs.cpio.gz}' if deploy_microvm_present else 'skipped (not built on this branch)' }}
|
||||
spinifex.target restarted; awsgw responding.
|
||||
@@ -74,19 +74,6 @@
|
||||
delay: 2
|
||||
changed_when: false
|
||||
|
||||
- name: init — build LB agent
|
||||
ansible.builtin.command:
|
||||
cmd: make build-lb-agent
|
||||
chdir: "{{ spinifex_root }}"
|
||||
changed_when: true
|
||||
when: spinifex_build_lb_image | bool
|
||||
|
||||
- name: init — import LB system image
|
||||
ansible.builtin.command:
|
||||
cmd: "{{ spinifex_root }}/scripts/build-system-image.sh {{ spinifex_lb_image_conf }} --import --quiet"
|
||||
changed_when: true
|
||||
when: spinifex_build_lb_image | bool
|
||||
|
||||
- name: init — summary
|
||||
ansible.builtin.debug:
|
||||
msg: |
|
||||
@@ -95,4 +82,3 @@
|
||||
external_mode={{ spinifex_external_mode }}
|
||||
spinifex.target: started
|
||||
CA: {{ spinifex_ca_trust_path }}
|
||||
LB image: {{ 'imported' if spinifex_build_lb_image else 'skipped' }}
|
||||
|
||||
@@ -79,10 +79,6 @@ mulga_root: "{{ lookup('pipe', 'readlink -f ' ~ playbook_dir ~ '/../../../..') }
|
||||
# Build output.
|
||||
build_tarball_path: /tmp/spinifex-local.tar.gz
|
||||
|
||||
# LB system image (built + imported at end of init).
|
||||
spinifex_build_lb_image: true
|
||||
spinifex_lb_image_conf: "{{ spinifex_root }}/scripts/images/lb.conf"
|
||||
|
||||
# Smoketest image — parameterised so tofu examples can share the same base.
|
||||
spinifex_smoketest_distro: debian
|
||||
spinifex_smoketest_version: "13"
|
||||
|
||||
Executable
+261
@@ -0,0 +1,261 @@
|
||||
#!/bin/bash
|
||||
# build-microvm-image.sh — build a minimal Alpine-based microVM kernel + initramfs
|
||||
# for use with QEMU microvm machine type and Spinifex lb-agent.
|
||||
#
|
||||
# Outputs:
|
||||
# $REPO_ROOT/build/microvm/vmlinuz
|
||||
# $REPO_ROOT/build/microvm/initramfs.cpio.gz
|
||||
#
|
||||
# Rootfs strategy (first match wins):
|
||||
# 1. apk available on host → install directly into a temp chroot (Alpine CI/CD)
|
||||
# 2. docker available → run alpine container, export filesystem
|
||||
# 3. podman available → same via podman
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
BUILD_DIR="$REPO_ROOT/build/microvm"
|
||||
INIT_SH="$BUILD_DIR/init.sh"
|
||||
INITTAB="$BUILD_DIR/inittab"
|
||||
|
||||
ALPINE_VERSION="${ALPINE_VERSION:-3.20}"
|
||||
ALPINE_MIRROR="${ALPINE_MIRROR:-https://dl-cdn.alpinelinux.org/alpine}"
|
||||
ALPINE_PACKAGES="busybox linux-virt haproxy iproute2 ca-certificates"
|
||||
|
||||
echo "[build-microvm-image] repo root: $REPO_ROOT"
|
||||
echo "[build-microvm-image] build dir: $BUILD_DIR"
|
||||
mkdir -p "$BUILD_DIR"
|
||||
|
||||
# --- Verify non-negotiable tools ---
|
||||
for tool in cpio gzip find; do
|
||||
if ! command -v "$tool" >/dev/null 2>&1; then
|
||||
echo "ERROR: required tool not found: $tool" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# --- Create chroot ---
|
||||
CHROOT_DIR=$(mktemp -d)
|
||||
cleanup() {
|
||||
echo "[build-microvm-image] cleaning up chroot: $CHROOT_DIR"
|
||||
rm -rf "$CHROOT_DIR"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# --- Populate rootfs ---
|
||||
# Strategy 1: native apk (Alpine hosts, CI)
|
||||
if command -v apk >/dev/null 2>&1; then
|
||||
echo "[build-microvm-image] rootfs: native apk"
|
||||
mkdir -p \
|
||||
"$CHROOT_DIR/etc/apk" \
|
||||
"$CHROOT_DIR/lib/apk/db" \
|
||||
"$CHROOT_DIR/var/cache/apk"
|
||||
cat > "$CHROOT_DIR/etc/apk/repositories" <<EOF
|
||||
${ALPINE_MIRROR}/v${ALPINE_VERSION}/main
|
||||
${ALPINE_MIRROR}/v${ALPINE_VERSION}/community
|
||||
EOF
|
||||
apk add \
|
||||
--root "$CHROOT_DIR" \
|
||||
--arch x86_64 \
|
||||
--no-cache \
|
||||
--allow-untrusted \
|
||||
--initdb \
|
||||
$ALPINE_PACKAGES
|
||||
|
||||
# Strategy 2/3: container runtime (dev machines)
|
||||
else
|
||||
CONTAINER_TOOL=""
|
||||
if command -v docker >/dev/null 2>&1 && docker info >/dev/null 2>&1; then
|
||||
CONTAINER_TOOL=docker
|
||||
elif command -v podman >/dev/null 2>&1; then
|
||||
CONTAINER_TOOL=podman
|
||||
else
|
||||
echo "ERROR: no rootfs build tool available." >&2
|
||||
echo " Install one of: apk (Alpine), docker, podman" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "[build-microvm-image] rootfs: $CONTAINER_TOOL export (alpine:${ALPINE_VERSION})"
|
||||
cid=$($CONTAINER_TOOL run -d \
|
||||
"alpine:${ALPINE_VERSION}" \
|
||||
sh -c "apk add --no-cache ${ALPINE_PACKAGES}")
|
||||
|
||||
exit_code=$($CONTAINER_TOOL wait "$cid")
|
||||
if [ "$exit_code" != "0" ]; then
|
||||
echo "ERROR: package installation failed in container (exit $exit_code)" >&2
|
||||
$CONTAINER_TOOL rm "$cid" >/dev/null 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
$CONTAINER_TOOL export "$cid" | tar -x -C "$CHROOT_DIR"
|
||||
$CONTAINER_TOOL rm "$cid" >/dev/null 2>&1
|
||||
fi
|
||||
|
||||
# --- Fix /dev device nodes ---
|
||||
# tar extraction without root silently creates 0-byte regular files for device
|
||||
# nodes (mknod requires CAP_MKNOD). The kernel opens /dev/console to wire
|
||||
# init's stdio to the serial console; a regular file there causes all init
|
||||
# output to be silently discarded. Recreate the two nodes needed before
|
||||
# devtmpfs mounts (which provides the rest at runtime).
|
||||
rm -rf "$CHROOT_DIR/dev"
|
||||
mkdir "$CHROOT_DIR/dev"
|
||||
sudo mknod -m 600 "$CHROOT_DIR/dev/console" c 5 1
|
||||
sudo mknod -m 666 "$CHROOT_DIR/dev/null" c 1 3
|
||||
|
||||
# --- Place init script ---
|
||||
echo "[build-microvm-image] installing init.sh as /init..."
|
||||
if [ ! -f "$INIT_SH" ]; then
|
||||
echo "ERROR: $INIT_SH not found — cannot build initramfs" >&2
|
||||
exit 1
|
||||
fi
|
||||
install -m 0755 "$INIT_SH" "$CHROOT_DIR/init"
|
||||
|
||||
# --- Place inittab ---
|
||||
if [ -f "$INITTAB" ]; then
|
||||
echo "[build-microvm-image] installing inittab..."
|
||||
install -m 0644 "$INITTAB" "$CHROOT_DIR/etc/inittab"
|
||||
fi
|
||||
|
||||
# --- Copy lb-agent binary ---
|
||||
LB_AGENT_BIN="$REPO_ROOT/bin/lb-agent"
|
||||
if [ -f "$LB_AGENT_BIN" ]; then
|
||||
echo "[build-microvm-image] copying lb-agent binary..."
|
||||
mkdir -p "$CHROOT_DIR/usr/local/bin"
|
||||
install -m 0755 "$LB_AGENT_BIN" "$CHROOT_DIR/usr/local/bin/lb-agent"
|
||||
else
|
||||
echo "[build-microvm-image] WARNING: $LB_AGENT_BIN not found — lb-agent will be absent from initramfs" >&2
|
||||
fi
|
||||
|
||||
# --- Create required directories for init ---
|
||||
mkdir -p \
|
||||
"$CHROOT_DIR/proc" \
|
||||
"$CHROOT_DIR/sys" \
|
||||
"$CHROOT_DIR/dev" \
|
||||
"$CHROOT_DIR/etc/ssl/certs" \
|
||||
"$CHROOT_DIR/etc/conf.d"
|
||||
|
||||
# --- Locate kernel ---
|
||||
KERNEL_IMG=$(find "$CHROOT_DIR/boot" -name "vmlinuz*" | sort | tail -1)
|
||||
if [ -z "$KERNEL_IMG" ]; then
|
||||
echo "ERROR: no vmlinuz found in $CHROOT_DIR/boot" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "[build-microvm-image] kernel image: $KERNEL_IMG"
|
||||
|
||||
KERNEL_VER=$(basename "$KERNEL_IMG" | sed 's/vmlinuz-//')
|
||||
KERNEL_CONFIG="$CHROOT_DIR/boot/config-${KERNEL_VER}"
|
||||
|
||||
# --- Assert fw_cfg module ---
|
||||
if ! find "$CHROOT_DIR/lib/modules" \
|
||||
-name "qemu_fw_cfg.ko*" \
|
||||
-o -name "fw_cfg_sysfs.ko*" \
|
||||
| grep -q .; then
|
||||
if ! grep -q "CONFIG_FW_CFG_SYSFS=y" "$KERNEL_CONFIG" 2>/dev/null; then
|
||||
echo "ERROR: qemu_fw_cfg module missing from initramfs — fw_cfg reads will fail at boot" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "[build-microvm-image] fw_cfg: built-in (CONFIG_FW_CFG_SYSFS=y)"
|
||||
else
|
||||
echo "[build-microvm-image] fw_cfg: module found in lib/modules"
|
||||
fi
|
||||
|
||||
# --- Strip kernel modules (Phase 2 keep-list approach) ---
|
||||
# Only modules needed by a virtio-mmio microvm guest are kept:
|
||||
# virtio*.ko* — virtio bus, net, blk, rng, console, and helpers
|
||||
# *fw_cfg*.ko* — QEMU fw_cfg sysfs driver (delivers boot blobs)
|
||||
# *failover*.ko* — net_failover + failover (virtio_net deps for live migration)
|
||||
# All other loadable modules are removed. Drivers compiled into the kernel
|
||||
# (CONFIG_VIRTIO_MMIO=y etc.) are unaffected — they live in vmlinuz, not here.
|
||||
echo "[build-microvm-image] stripping kernel modules to virtio+fw_cfg only..."
|
||||
for kver_dir in "$CHROOT_DIR/lib/modules"/*/; do
|
||||
kver="$(basename "$kver_dir")"
|
||||
kernel_dir="$kver_dir/kernel"
|
||||
[ -d "$kernel_dir" ] || continue
|
||||
|
||||
# Collect keeper modules and their relative paths from the kernel/ tree.
|
||||
tmp_keep=$(mktemp -d)
|
||||
find "$kernel_dir" -name "virtio*.ko.gz" -o -name "virtio*.ko" \
|
||||
-o -name "*fw_cfg*.ko.gz" -o -name "*fw_cfg*.ko" \
|
||||
-o -name "*failover*.ko.gz" -o -name "*failover*.ko" \
|
||||
2>/dev/null | while read -r mod; do
|
||||
rel="${mod#"$kernel_dir/"}"
|
||||
dest_dir="$tmp_keep/$(dirname "$rel")"
|
||||
mkdir -p "$dest_dir"
|
||||
cp "$mod" "$dest_dir/"
|
||||
done
|
||||
|
||||
# Replace kernel module tree with the keeper set.
|
||||
rm -rf "$kernel_dir"
|
||||
mkdir -p "$kernel_dir"
|
||||
# Restore each kept file into the same relative path.
|
||||
(cd "$tmp_keep" && find . -name "*.ko*" | while read -r f; do
|
||||
dest_dir="$kernel_dir/$(dirname "${f#./}")"
|
||||
mkdir -p "$dest_dir"
|
||||
cp "$f" "$dest_dir/"
|
||||
done)
|
||||
rm -rf "$tmp_keep"
|
||||
|
||||
# Regenerate module dependency map from the surviving modules.
|
||||
depmod -b "$CHROOT_DIR" "$kver" 2>/dev/null || true
|
||||
done
|
||||
|
||||
# --- Strip package-manager artifacts (not needed at runtime) ---
|
||||
echo "[build-microvm-image] stripping package manager artifacts..."
|
||||
rm -rf \
|
||||
"$CHROOT_DIR/lib/apk" \
|
||||
"$CHROOT_DIR/var/cache/apk" \
|
||||
"$CHROOT_DIR/etc/apk" \
|
||||
"$CHROOT_DIR/usr/share/man" \
|
||||
"$CHROOT_DIR/usr/share/doc" \
|
||||
"$CHROOT_DIR/usr/share/apk" \
|
||||
"$CHROOT_DIR/usr/include" \
|
||||
"$CHROOT_DIR/usr/lib/pkgconfig"
|
||||
|
||||
# --- Strip debug symbols from binaries ---
|
||||
echo "[build-microvm-image] stripping debug symbols from binaries..."
|
||||
find "$CHROOT_DIR/usr/sbin" "$CHROOT_DIR/usr/bin" \
|
||||
"$CHROOT_DIR/sbin" "$CHROOT_DIR/bin" \
|
||||
"$CHROOT_DIR/usr/local/bin" \
|
||||
-type f 2>/dev/null | while read -r bin; do
|
||||
strip --strip-all "$bin" 2>/dev/null || true
|
||||
done
|
||||
|
||||
# --- Copy vmlinuz out before stripping /boot from chroot ---
|
||||
VMLINUZ_OUT="$BUILD_DIR/vmlinuz"
|
||||
echo "[build-microvm-image] copying kernel: $VMLINUZ_OUT"
|
||||
cp "$KERNEL_IMG" "$VMLINUZ_OUT"
|
||||
|
||||
# Drop /boot from the chroot — vmlinuz, Alpine's stock initramfs-virt, System.map
|
||||
# and kernel config all live here (~25 MiB). None are needed at runtime: the
|
||||
# guest already has the kernel loaded by QEMU's -kernel, and Alpine's initramfs
|
||||
# is unused because we replace it with our own /init. Leaving /boot in the cpio
|
||||
# nearly doubles uncompressed initramfs size and forces a 512 MiB guest just to
|
||||
# survive decompression.
|
||||
rm -rf "$CHROOT_DIR/boot"
|
||||
|
||||
# --- Build initramfs ---
|
||||
INITRAMFS_OUT="$BUILD_DIR/initramfs.cpio.gz"
|
||||
echo "[build-microvm-image] building initramfs: $INITRAMFS_OUT"
|
||||
(
|
||||
cd "$CHROOT_DIR"
|
||||
find . | cpio --quiet -o -H newc | gzip -9 > "$INITRAMFS_OUT"
|
||||
)
|
||||
|
||||
# --- Log artifact sizes ---
|
||||
echo ""
|
||||
echo "[build-microvm-image] artifacts:"
|
||||
ls -lh "$VMLINUZ_OUT" "$INITRAMFS_OUT"
|
||||
echo ""
|
||||
|
||||
# --- Phase 2 artifact size gate (50 MiB total) ---
|
||||
ARTIFACT_MAX_MiB="${MICROVM_ARTIFACT_MAX_MiB:-50}"
|
||||
total_bytes=$(( $(stat -c %s "$VMLINUZ_OUT") + $(stat -c %s "$INITRAMFS_OUT") ))
|
||||
max_bytes=$(( ARTIFACT_MAX_MiB * 1024 * 1024 ))
|
||||
total_mib=$(( (total_bytes + 1048575) / 1048576 ))
|
||||
if [ "$total_bytes" -gt "$max_bytes" ]; then
|
||||
echo "ERROR: artifact size ${total_mib} MiB exceeds ${ARTIFACT_MAX_MiB} MiB gate" >&2
|
||||
echo " Set MICROVM_ARTIFACT_MAX_MiB=N to override during development." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "[build-microvm-image] artifact size gate: PASS (${total_mib} MiB / ${ARTIFACT_MAX_MiB} MiB)"
|
||||
echo ""
|
||||
echo "[build-microvm-image] done."
|
||||
@@ -91,9 +91,8 @@ if ! id -Gn 2>/dev/null | grep -qw spinifex; then
|
||||
newgrp spinifex
|
||||
fi
|
||||
|
||||
echo "=== Building and importing LB image ==="
|
||||
cd "$PROJECT_ROOT" && make build-lb-agent
|
||||
"$PROJECT_ROOT/scripts/build-system-image.sh" "$PROJECT_ROOT/scripts/images/lb.conf" --import --quiet
|
||||
echo "=== Building and installing microVM artifacts ==="
|
||||
cd "$PROJECT_ROOT" && make build-lb-agent install-microvm
|
||||
|
||||
echo "=== Done ==="
|
||||
echo "Services: sudo systemctl status spinifex.target"
|
||||
|
||||
@@ -268,10 +268,9 @@ sudo systemctl start spinifex.target
|
||||
# Wait for services to start
|
||||
sleep 5
|
||||
|
||||
# --- Build + import LB image (needs services running) ---
|
||||
echo "==> Building and importing LB image"
|
||||
cd "$PROJECT_ROOT" && make build-lb-agent
|
||||
"$PROJECT_ROOT/scripts/build-system-image.sh" "$PROJECT_ROOT/scripts/images/lb.conf" --import --quiet
|
||||
# --- Build + install microVM artifacts (kernel + initramfs for direct-boot LBs) ---
|
||||
echo "==> Building and installing microVM artifacts"
|
||||
cd "$PROJECT_ROOT" && make build-lb-agent install-microvm
|
||||
|
||||
# --- Smoke test ---
|
||||
sudo -u "$INVOKING_USER" "$SCRIPT_DIR/smoke-test.sh"
|
||||
|
||||
@@ -346,6 +346,13 @@ install_files() {
|
||||
$SUDO install -m 0755 "$EXTRACT_DIR/setup.sh" /usr/local/share/spinifex/setup.sh
|
||||
info " /usr/local/share/spinifex/setup.sh"
|
||||
fi
|
||||
|
||||
# microVM kernel + initramfs
|
||||
$SUDO install -d /usr/share/spinifex/microvm
|
||||
if [ -d "$EXTRACT_DIR/microvm" ]; then
|
||||
$SUDO cp "$EXTRACT_DIR/microvm/"* /usr/share/spinifex/microvm/
|
||||
info " /usr/share/spinifex/microvm/*"
|
||||
fi
|
||||
}
|
||||
|
||||
# --- Create directories ---
|
||||
|
||||
+20
-31
@@ -49,7 +49,6 @@ import (
|
||||
handlers_elbv2 "github.com/mulgadc/spinifex/spinifex/handlers/elbv2"
|
||||
"github.com/mulgadc/spinifex/spinifex/instancetypes"
|
||||
"github.com/mulgadc/spinifex/spinifex/objectstore"
|
||||
"github.com/mulgadc/spinifex/spinifex/tags"
|
||||
"github.com/mulgadc/spinifex/spinifex/types"
|
||||
"github.com/mulgadc/spinifex/spinifex/utils"
|
||||
"github.com/mulgadc/spinifex/spinifex/vm"
|
||||
@@ -1186,29 +1185,6 @@ func (d *Daemon) startCluster() error {
|
||||
// Wire system credentials + gateway URL for LB agent SigV4 auth.
|
||||
d.wireLBAgentConfig()
|
||||
|
||||
// Set up lazy system AMI discovery for LB VMs. The image may not exist
|
||||
// at daemon startup (imported later), so we resolve it at request time.
|
||||
if d.imageService != nil {
|
||||
imgSvc := d.imageService
|
||||
d.elbv2Service.SetSystemAMIFunc(func() (string, error) {
|
||||
imagesOut, imgErr := imgSvc.DescribeImages(&ec2.DescribeImagesInput{
|
||||
Filters: []*ec2.Filter{{
|
||||
Name: aws.String("tag:" + tags.ManagedByKey),
|
||||
Values: []*string{aws.String(tags.ManagedByELBv2)},
|
||||
}},
|
||||
}, utils.GlobalAccountID)
|
||||
if imgErr != nil {
|
||||
return "", fmt.Errorf("describe LB system images: %w", imgErr)
|
||||
}
|
||||
if len(imagesOut.Images) == 0 {
|
||||
return "", errors.New("LB system image not imported; run: spx admin images import --name lb-alpine-3.21.6-x86_64")
|
||||
}
|
||||
amiID := aws.StringValue(imagesOut.Images[0].ImageId)
|
||||
slog.Info("System AMI resolved for LB VMs", "amiId", amiID, "name", aws.StringValue(imagesOut.Images[0].Name))
|
||||
return amiID, nil
|
||||
})
|
||||
}
|
||||
|
||||
// System VMs (LB, NAT GW) use the dedicated sys.micro instance type.
|
||||
d.elbv2Service.SetSystemInstanceTypeFunc(func() string {
|
||||
return "sys.micro"
|
||||
@@ -2129,21 +2105,34 @@ func (d *Daemon) wireLBAgentConfig() {
|
||||
"awsgwBindIP", awsgwBindIP, "mgmtBridgeIP", d.mgmtBridgeIP, "advertiseIP", advertiseIP)
|
||||
}
|
||||
|
||||
// Pass mgmt route info so lbVMUserData can add a bootcmd route for
|
||||
// Pass mgmt route info so buildMicrovmNICs can add a host route for
|
||||
// internal LBs that reach the AWSGW via the management NIC.
|
||||
if d.mgmtRouteVia != "" {
|
||||
d.elbv2Service.MgmtRouteGateway = d.mgmtBridgeIP
|
||||
d.elbv2Service.MgmtRouteTarget = d.mgmtRouteVia
|
||||
}
|
||||
|
||||
// Always expose mgmtBridgeIP and advertiseIP so lbVMUserData can synthesize
|
||||
// a mgmt-NIC fallback route for internal-scheme LBs on single-node setups
|
||||
// (where MgmtRoute{Gateway,Target} stay empty because internet-facing LBs
|
||||
// reach AWSGW via VPC + EIP SNAT). Internal LBs have no EIP, so without
|
||||
// this fallback the agent has no return path and the LB stays in
|
||||
// provisioning forever.
|
||||
// Always expose mgmtBridgeIP and advertiseIP so buildMicrovmNICs can
|
||||
// synthesize a mgmt-NIC fallback route for internal-scheme LBs on
|
||||
// single-node setups (where MgmtRoute{Gateway,Target} stay empty because
|
||||
// internet-facing LBs reach AWSGW via VPC + EIP SNAT). Internal LBs have
|
||||
// no EIP, so without this fallback the agent has no return path and the
|
||||
// LB stays in provisioning forever.
|
||||
d.elbv2Service.MgmtBridgeIP = d.mgmtBridgeIP
|
||||
d.elbv2Service.AdvertiseIP = advertiseIP
|
||||
|
||||
// Read CA cert so direct-boot microvm guests can verify AWSGW TLS.
|
||||
// The NATS CA cert signs the AWSGW server cert (same local CA).
|
||||
if d.config.NATS.CACert != "" {
|
||||
if caBytes, err := os.ReadFile(d.config.NATS.CACert); err == nil {
|
||||
d.elbv2Service.CACert = string(caBytes)
|
||||
slog.Info("CA cert loaded for LB agent TLS", "path", d.config.NATS.CACert)
|
||||
} else {
|
||||
slog.Warn("Failed to read CA cert for LB agent TLS", "path", d.config.NATS.CACert, "err", err)
|
||||
}
|
||||
} else {
|
||||
slog.Warn("NATS CACert not configured — direct-boot LB VMs will not verify AWSGW TLS")
|
||||
}
|
||||
}
|
||||
|
||||
// resolveGPUModel maps a discovered GPU to an instance type model.
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
@@ -17,13 +19,11 @@ import (
|
||||
// Compile-time check that Daemon implements SystemInstanceLauncher.
|
||||
var _ handlers_elbv2.SystemInstanceLauncher = (*Daemon)(nil)
|
||||
|
||||
// LaunchSystemInstance creates and starts a system-managed VM. The VM is owned
|
||||
// by the system account (GlobalAccountID) and is not visible to customer
|
||||
// DescribeInstances calls.
|
||||
//
|
||||
// This is the internal equivalent of RunInstances — it reuses the same
|
||||
// instance service, resource manager, volume preparation, and QEMU launch
|
||||
// path, but skips the NATS request/response envelope and key pair validation.
|
||||
// LaunchSystemInstance creates and starts a system-managed VM (ELBv2 LB).
|
||||
// The VM is owned by the system account (GlobalAccountID), is not visible to
|
||||
// customer DescribeInstances calls, and always boots via direct kernel boot
|
||||
// (bundled vmlinuz+initramfs, fw_cfg-delivered config). There is no AMI,
|
||||
// volume, or cloud-init path.
|
||||
func (d *Daemon) LaunchSystemInstance(input *handlers_elbv2.SystemInstanceInput) (*handlers_elbv2.SystemInstanceOutput, error) {
|
||||
accountID := utils.GlobalAccountID
|
||||
// ENI account may differ from system account — the ENI is created under
|
||||
@@ -39,14 +39,6 @@ func (d *Daemon) LaunchSystemInstance(input *handlers_elbv2.SystemInstanceInput)
|
||||
return nil, fmt.Errorf("unknown instance type: %s", input.InstanceType)
|
||||
}
|
||||
|
||||
// Validate AMI
|
||||
if d.imageService == nil {
|
||||
return nil, fmt.Errorf("image service not initialized")
|
||||
}
|
||||
if _, err := d.imageService.GetAMIConfig(input.ImageID); err != nil {
|
||||
return nil, fmt.Errorf("AMI %s not found: %w", input.ImageID, err)
|
||||
}
|
||||
|
||||
// Allocate resources
|
||||
if err := d.resourceMgr.allocate(instanceType); err != nil {
|
||||
return nil, fmt.Errorf("insufficient capacity for %s: %w", input.InstanceType, err)
|
||||
@@ -57,7 +49,6 @@ func (d *Daemon) LaunchSystemInstance(input *handlers_elbv2.SystemInstanceInput)
|
||||
// customer-facing listings (Nodes page).
|
||||
runInput := &ec2.RunInstancesInput{
|
||||
InstanceType: aws.String(input.InstanceType),
|
||||
ImageId: aws.String(input.ImageID),
|
||||
MinCount: aws.Int64(1),
|
||||
MaxCount: aws.Int64(1),
|
||||
TagSpecifications: []*ec2.TagSpecification{
|
||||
@@ -72,10 +63,6 @@ func (d *Daemon) LaunchSystemInstance(input *handlers_elbv2.SystemInstanceInput)
|
||||
if input.SubnetID != "" {
|
||||
runInput.SubnetId = aws.String(input.SubnetID)
|
||||
}
|
||||
if input.UserData != "" {
|
||||
encoded := base64.StdEncoding.EncodeToString([]byte(input.UserData))
|
||||
runInput.UserData = aws.String(encoded)
|
||||
}
|
||||
|
||||
// Create VM via instance service
|
||||
instance, ec2Instance, err := d.instanceService.RunInstance(runInput)
|
||||
@@ -276,6 +263,14 @@ func (d *Daemon) LaunchSystemInstance(input *handlers_elbv2.SystemInstanceInput)
|
||||
instance.MgmtMAC = generateMgmtMAC(instance.ID)
|
||||
instance.MgmtIP = mgmtIP
|
||||
|
||||
// Inject the allocated MAC and CIDR into NIC[1] so writeFwCfgBlobs
|
||||
// carries real values — the ELBv2 service leaves them blank because
|
||||
// MAC/IP are only known after daemon allocation.
|
||||
if len(input.NICs) > 1 && input.NICs[1].MAC == "" {
|
||||
input.NICs[1].MAC = instance.MgmtMAC
|
||||
input.NICs[1].CIDR = instance.MgmtIP + "/24"
|
||||
}
|
||||
|
||||
tapName := vm.MgmtTapName(instance.ID)
|
||||
tapErr := d.networkPlumber.SetupTap(vm.TapSpec{Name: tapName, Bridge: mgmtBridge})
|
||||
if tapErr != nil {
|
||||
@@ -302,30 +297,25 @@ func (d *Daemon) LaunchSystemInstance(input *handlers_elbv2.SystemInstanceInput)
|
||||
}
|
||||
}
|
||||
|
||||
// Prepare volumes (root volume clone from AMI, cloud-init ISO, etc.)
|
||||
volumeInfos, err := d.instanceService.GenerateVolumes(runInput, instance)
|
||||
// Direct-boot (microvm) path: build the vm.Config directly with microvm
|
||||
// machine settings and fw_cfg blobs for network, lb-agent env, and CA cert.
|
||||
cfg, err := d.buildDirectBootConfig(instance.ID, input)
|
||||
if err != nil {
|
||||
d.cleanupFailedSystemInstance(instance, instanceType)
|
||||
return nil, fmt.Errorf("generate volumes: %w", err)
|
||||
}
|
||||
|
||||
instance.Instance.BlockDeviceMappings = make([]*ec2.InstanceBlockDeviceMapping, 0, len(volumeInfos))
|
||||
for _, vi := range volumeInfos {
|
||||
mapping := &ec2.InstanceBlockDeviceMapping{}
|
||||
mapping.SetDeviceName(vi.DeviceName)
|
||||
mapping.Ebs = &ec2.EbsInstanceBlockDevice{}
|
||||
mapping.Ebs.SetVolumeId(vi.VolumeId)
|
||||
mapping.Ebs.SetAttachTime(vi.AttachTime)
|
||||
mapping.Ebs.SetDeleteOnTermination(vi.DeleteOnTermination)
|
||||
mapping.Ebs.SetStatus("attached")
|
||||
instance.Instance.BlockDeviceMappings = append(instance.Instance.BlockDeviceMappings, mapping)
|
||||
return nil, fmt.Errorf("build direct-boot config: %w", err)
|
||||
}
|
||||
instance.Config = cfg
|
||||
instance.DirectBoot = true
|
||||
|
||||
// Launch QEMU VM
|
||||
t1 := time.Now()
|
||||
if err := d.vmMgr.Run(instance); err != nil {
|
||||
d.cleanupFailedSystemInstance(instance, instanceType)
|
||||
return nil, fmt.Errorf("launch instance: %w", err)
|
||||
}
|
||||
slog.Info("direct-boot timing",
|
||||
"instanceId", instance.ID,
|
||||
"t1_to_t2_ms", time.Since(t1).Milliseconds())
|
||||
|
||||
slog.Info("LaunchSystemInstance completed",
|
||||
"instanceId", instance.ID,
|
||||
@@ -418,3 +408,215 @@ func (d *Daemon) WaitForSystemInstance(instanceID string, timeout time.Duration)
|
||||
}
|
||||
return fmt.Errorf("instance %s did not reach running state within %s", instanceID, timeout)
|
||||
}
|
||||
|
||||
// buildDirectBootConfig constructs a vm.Config for a direct-boot microVM
|
||||
// launch. It does not call buildBaseVMConfig — the microvm machine type
|
||||
// requires no PCIe root ports, no UEFI/BIOS, and no drives. Runtime paths
|
||||
// (PID file, console log, serial socket) are filled in later by startQEMU.
|
||||
//
|
||||
// Network topology from input.NICs is serialised to three fw_cfg tmpfiles
|
||||
// (netcfg, lb-agent-env, ca-cert) under utils.RuntimeDir().
|
||||
func (d *Daemon) buildDirectBootConfig(instanceID string, input *handlers_elbv2.SystemInstanceInput) (vm.Config, error) {
|
||||
it := d.resourceMgr.instanceTypes[input.InstanceType]
|
||||
architecture := "x86_64"
|
||||
if it != nil && it.ProcessorInfo != nil && len(it.ProcessorInfo.SupportedArchitectures) > 0 && it.ProcessorInfo.SupportedArchitectures[0] != nil {
|
||||
architecture = *it.ProcessorInfo.SupportedArchitectures[0]
|
||||
}
|
||||
vcpus := 1
|
||||
memMiB := 128
|
||||
if it != nil {
|
||||
vcpus = int(instanceTypeVCPUs(it))
|
||||
memMiB = int(instanceTypeMemoryMiB(it))
|
||||
}
|
||||
|
||||
const imagePath = "/usr/share/spinifex/microvm"
|
||||
|
||||
fwCfg, err := d.writeFwCfgBlobs(instanceID, input)
|
||||
if err != nil {
|
||||
return vm.Config{}, fmt.Errorf("write fw_cfg blobs: %w", err)
|
||||
}
|
||||
|
||||
// Build kernel cmdline.
|
||||
//
|
||||
// QEMU microvm maps virtio-mmio slots at 0xfeb00000 + n*0x200 with IRQ 5+n.
|
||||
// Without virtio_mmio.device params the kernel never discovers the devices
|
||||
// (auto-kernel-cmdline does not append when -append is specified).
|
||||
//
|
||||
// Boot-time perf flags (Phase 2):
|
||||
// quiet loglevel=3 — suppress printk to ttyS0 (115200 baud serial
|
||||
// writes dominate boot time; warnings+errors only)
|
||||
// mitigations=off — skip spectre/meltdown microcode patching at
|
||||
// boot. Safe here: KVM guest on trusted host CPU,
|
||||
// single-tenant LB workload, no untrusted code in
|
||||
// the VM. Saves ~200-500ms.
|
||||
// tsc=reliable — skip TSC calibration loop on KVM (host already
|
||||
// vouched for invariant TSC)
|
||||
// no_timer_check — skip APIC/PIT cross-check; microvm has no PIT
|
||||
// reboot=t — triple-fault reboot; skip ACPI/keyboard probes
|
||||
// i8042.no{pnp,aux,kbd} — microvm has no PS/2; skip i8042 probe timeouts
|
||||
var sb strings.Builder
|
||||
sb.WriteString("console=ttyS0 quiet loglevel=3 mitigations=off tsc=reliable no_timer_check reboot=t i8042.nopnp i8042.noaux i8042.nokbd")
|
||||
for i := range input.NICs {
|
||||
fmt.Fprintf(&sb, " virtio_mmio.device=0x200@0x%x:%d",
|
||||
0xfeb00000+i*0x200, 5+i)
|
||||
}
|
||||
cmdline := sb.String()
|
||||
|
||||
machineType := microvmMachineType()
|
||||
|
||||
cfg := vm.Config{
|
||||
Name: instanceID,
|
||||
EnableKVM: true,
|
||||
NoGraphic: true,
|
||||
Architecture: architecture,
|
||||
CPUType: "host,-pmu",
|
||||
CPUCount: vcpus,
|
||||
Memory: memMiB,
|
||||
MachineType: machineType,
|
||||
KernelImage: filepath.Join(imagePath, "vmlinuz"),
|
||||
Initrd: filepath.Join(imagePath, "initramfs.cpio.gz"),
|
||||
KernelCmdline: cmdline,
|
||||
FwCfg: fwCfg,
|
||||
}
|
||||
|
||||
// Wire netdevs and devices for each NIC in declaration order.
|
||||
// Tap device creation happens later in startQEMU (host-side only here).
|
||||
allNICs := buildNICNetdevs(instanceID, input, machineType)
|
||||
cfg.NetDevs = append(cfg.NetDevs, allNICs.netdevs...)
|
||||
cfg.Devices = append(cfg.Devices, allNICs.devices...)
|
||||
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
// nicNetdevResult holds the QEMU netdev and device entries for all NICs.
|
||||
type nicNetdevResult struct {
|
||||
netdevs []vm.NetDev
|
||||
devices []vm.Device
|
||||
}
|
||||
|
||||
// microvmMachineType returns the QEMU -M string for the microvm machine.
|
||||
// isa-serial is kept on permanently for per-VM console log access — the
|
||||
// boot-time saving from turning it off is not worth losing console output.
|
||||
func microvmMachineType() string {
|
||||
return "microvm,pic=off,pit=off,rtc=on,acpi=on,isa-serial=on"
|
||||
}
|
||||
|
||||
// buildNICNetdevs produces QEMU -netdev and -device entries for each NIC in
|
||||
// input. The NIC order determines the netdev IDs: net0 = primary ENI,
|
||||
// net1 = mgmt (if present), net2+ = extra ENIs. Extra ENIs beyond the primary
|
||||
// are only included when corresponding ExtraENIs entries exist.
|
||||
func buildNICNetdevs(instanceID string, input *handlers_elbv2.SystemInstanceInput, machineType string) nicNetdevResult {
|
||||
var res nicNetdevResult
|
||||
|
||||
for i, nic := range input.NICs {
|
||||
netID := fmt.Sprintf("net%d", i)
|
||||
// Resolve the tap name from the corresponding ENI.
|
||||
tapName := tapNameForNIC(i, nic, instanceID, input)
|
||||
res.netdevs = append(res.netdevs, vm.NetDev{
|
||||
Value: fmt.Sprintf("tap,id=%s,ifname=%s,script=no,downscript=no", netID, tapName),
|
||||
})
|
||||
res.devices = append(res.devices, vm.NetDevice(machineType, netID, nic.MAC))
|
||||
}
|
||||
|
||||
return res
|
||||
}
|
||||
|
||||
// tapNameForNIC returns the Linux tap device name for a NIC at the given index.
|
||||
// Index 0 → primary ENI tap, index 1 → mgmt tap, index 2+ → extra ENI taps.
|
||||
func tapNameForNIC(idx int, _ handlers_elbv2.NICConfig, instanceID string, input *handlers_elbv2.SystemInstanceInput) string {
|
||||
switch idx {
|
||||
case 0:
|
||||
return vm.TapDeviceName(input.ENIID)
|
||||
case 1:
|
||||
return vm.MgmtTapName(instanceID)
|
||||
default:
|
||||
extraIdx := idx - 2
|
||||
if extraIdx < len(input.ExtraENIs) {
|
||||
return vm.TapDeviceName(input.ExtraENIs[extraIdx].ENIID)
|
||||
}
|
||||
return fmt.Sprintf("tap-unknown-%d", idx)
|
||||
}
|
||||
}
|
||||
|
||||
// writeFwCfgBlobs serialises NIC configuration, lb-agent env, and CA cert to
|
||||
// per-VM tmpfiles under utils.RuntimeDir(). Returns the fw_cfg entries for the
|
||||
// three blobs and an error if any write fails or the NIC default invariant is
|
||||
// violated.
|
||||
func (d *Daemon) writeFwCfgBlobs(instanceID string, input *handlers_elbv2.SystemInstanceInput) ([]vm.FwCfgEntry, error) {
|
||||
runtimeDir := utils.RuntimeDir()
|
||||
|
||||
netcfgPath := filepath.Join(runtimeDir, fmt.Sprintf("fwcfg-%s-netcfg.tmp", instanceID))
|
||||
lbenvPath := filepath.Join(runtimeDir, fmt.Sprintf("fwcfg-%s-lbenv.tmp", instanceID))
|
||||
cacertPath := filepath.Join(runtimeDir, fmt.Sprintf("fwcfg-%s-cacert.tmp", instanceID))
|
||||
|
||||
netcfg, err := buildNetcfgBlob(input.NICs)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := os.WriteFile(netcfgPath, []byte(netcfg), 0600); err != nil {
|
||||
return nil, fmt.Errorf("write netcfg blob: %w", err)
|
||||
}
|
||||
|
||||
if err := os.WriteFile(lbenvPath, []byte(input.LBAgentEnv), 0600); err != nil {
|
||||
_ = os.Remove(netcfgPath)
|
||||
return nil, fmt.Errorf("write lb-agent-env blob: %w", err)
|
||||
}
|
||||
|
||||
if err := os.WriteFile(cacertPath, []byte(input.CACert), 0600); err != nil {
|
||||
_ = os.Remove(netcfgPath)
|
||||
_ = os.Remove(lbenvPath)
|
||||
return nil, fmt.Errorf("write ca-cert blob: %w", err)
|
||||
}
|
||||
|
||||
slog.Debug("wrote fw_cfg blobs",
|
||||
"instanceId", instanceID,
|
||||
"netcfg", netcfgPath,
|
||||
"lbenv", lbenvPath,
|
||||
"cacert", cacertPath,
|
||||
)
|
||||
|
||||
return []vm.FwCfgEntry{
|
||||
{Name: "opt/spinifex/netcfg", File: netcfgPath},
|
||||
{Name: "opt/spinifex/lb-agent-env", File: lbenvPath},
|
||||
{Name: "opt/spinifex/ca-cert", File: cacertPath},
|
||||
}, nil
|
||||
}
|
||||
|
||||
// buildNetcfgBlob serialises a slice of NICConfig entries to the shell KEY=value
|
||||
// format consumed by the initramfs init script. Exactly one NIC must have
|
||||
// IsDefault=true; returns an error otherwise.
|
||||
func buildNetcfgBlob(nics []handlers_elbv2.NICConfig) (string, error) {
|
||||
defaultCount := 0
|
||||
for _, n := range nics {
|
||||
if n.IsDefault {
|
||||
defaultCount++
|
||||
}
|
||||
}
|
||||
if defaultCount != 1 {
|
||||
return "", fmt.Errorf("netcfg: exactly one NIC must have IsDefault=true, got %d", defaultCount)
|
||||
}
|
||||
|
||||
var sb strings.Builder
|
||||
for i, n := range nics {
|
||||
prefix := fmt.Sprintf("NIC%d", i)
|
||||
fmt.Fprintf(&sb, "%s_MAC=%s\n", prefix, n.MAC)
|
||||
if n.CIDR != "" {
|
||||
fmt.Fprintf(&sb, "%s_CIDR=%s\n", prefix, n.CIDR)
|
||||
}
|
||||
if n.Gateway != "" {
|
||||
fmt.Fprintf(&sb, "%s_GW=%s\n", prefix, n.Gateway)
|
||||
}
|
||||
if n.IsDefault {
|
||||
fmt.Fprintf(&sb, "%s_DEFAULT=1\n", prefix)
|
||||
} else {
|
||||
fmt.Fprintf(&sb, "%s_DEFAULT=0\n", prefix)
|
||||
}
|
||||
if n.RouteDst != "" {
|
||||
fmt.Fprintf(&sb, "%s_ROUTE_DST=%s\n", prefix, n.RouteDst)
|
||||
}
|
||||
if n.RouteVia != "" {
|
||||
fmt.Fprintf(&sb, "%s_ROUTE_VIA=%s\n", prefix, n.RouteVia)
|
||||
}
|
||||
}
|
||||
return sb.String(), nil
|
||||
}
|
||||
|
||||
@@ -1,10 +1,16 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/mulgadc/spinifex/spinifex/handlers/elbv2"
|
||||
"github.com/mulgadc/spinifex/spinifex/vm"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func newDaemonWithVMs(vms ...*vm.VM) *Daemon {
|
||||
@@ -66,3 +72,240 @@ func TestWaitForSystemInstance_Timeout(t *testing.T) {
|
||||
t.Fatal("expected timeout error")
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// buildNetcfgBlob
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestBuildNetcfgBlob_NoDefault(t *testing.T) {
|
||||
nics := []handlers_elbv2.NICConfig{
|
||||
{MAC: "02:aa:bb:cc:dd:01", CIDR: "10.0.1.5/24", IsDefault: false},
|
||||
}
|
||||
_, err := buildNetcfgBlob(nics)
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "exactly one NIC must have IsDefault=true")
|
||||
}
|
||||
|
||||
func TestBuildNetcfgBlob_TwoDefaults(t *testing.T) {
|
||||
nics := []handlers_elbv2.NICConfig{
|
||||
{MAC: "02:aa:bb:cc:dd:01", IsDefault: true},
|
||||
{MAC: "02:aa:bb:cc:dd:02", IsDefault: true},
|
||||
}
|
||||
_, err := buildNetcfgBlob(nics)
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "exactly one NIC must have IsDefault=true, got 2")
|
||||
}
|
||||
|
||||
func TestBuildNetcfgBlob_SingleNICMinimal(t *testing.T) {
|
||||
nics := []handlers_elbv2.NICConfig{
|
||||
{MAC: "02:aa:bb:cc:dd:01", IsDefault: true},
|
||||
}
|
||||
got, err := buildNetcfgBlob(nics)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, got, "NIC0_MAC=02:aa:bb:cc:dd:01\n")
|
||||
assert.Contains(t, got, "NIC0_DEFAULT=1\n")
|
||||
// Optional fields absent when empty.
|
||||
assert.NotContains(t, got, "NIC0_CIDR=")
|
||||
assert.NotContains(t, got, "NIC0_GW=")
|
||||
}
|
||||
|
||||
func TestBuildNetcfgBlob_FullNIC(t *testing.T) {
|
||||
nics := []handlers_elbv2.NICConfig{
|
||||
{
|
||||
MAC: "02:aa:bb:cc:dd:01",
|
||||
CIDR: "10.0.1.5/24",
|
||||
Gateway: "10.0.1.1",
|
||||
IsDefault: true,
|
||||
RouteDst: "10.20.0.5/32",
|
||||
RouteVia: "10.0.1.254",
|
||||
},
|
||||
}
|
||||
got, err := buildNetcfgBlob(nics)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, got, "NIC0_MAC=02:aa:bb:cc:dd:01\n")
|
||||
assert.Contains(t, got, "NIC0_CIDR=10.0.1.5/24\n")
|
||||
assert.Contains(t, got, "NIC0_GW=10.0.1.1\n")
|
||||
assert.Contains(t, got, "NIC0_DEFAULT=1\n")
|
||||
assert.Contains(t, got, "NIC0_ROUTE_DST=10.20.0.5/32\n")
|
||||
assert.Contains(t, got, "NIC0_ROUTE_VIA=10.0.1.254\n")
|
||||
}
|
||||
|
||||
func TestBuildNetcfgBlob_MultiNIC_DefaultFlags(t *testing.T) {
|
||||
nics := []handlers_elbv2.NICConfig{
|
||||
{MAC: "02:aa:bb:cc:dd:01", CIDR: "10.0.1.5/24", Gateway: "10.0.1.1", IsDefault: true},
|
||||
{MAC: "02:aa:bb:cc:dd:02", CIDR: "192.168.100.5/24", IsDefault: false},
|
||||
}
|
||||
got, err := buildNetcfgBlob(nics)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, got, "NIC0_DEFAULT=1\n")
|
||||
assert.Contains(t, got, "NIC1_DEFAULT=0\n")
|
||||
assert.Contains(t, got, "NIC1_MAC=02:aa:bb:cc:dd:02\n")
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// tapNameForNIC
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestTapNameForNIC_PrimaryENI(t *testing.T) {
|
||||
input := &handlers_elbv2.SystemInstanceInput{
|
||||
ENIID: "eni-0123456789abcdef0",
|
||||
}
|
||||
got := tapNameForNIC(0, handlers_elbv2.NICConfig{}, "i-abc123", input)
|
||||
// TapDeviceName strips "eni-" prefix → "tap0123456789abcde" (15 chars max)
|
||||
assert.True(t, strings.HasPrefix(got, "tap"), "expected tap prefix, got %q", got)
|
||||
assert.LessOrEqual(t, len(got), 15)
|
||||
}
|
||||
|
||||
func TestTapNameForNIC_MgmtNIC(t *testing.T) {
|
||||
input := &handlers_elbv2.SystemInstanceInput{ENIID: "eni-aaa"}
|
||||
got := tapNameForNIC(1, handlers_elbv2.NICConfig{}, "i-deadbeef", input)
|
||||
// MgmtTapName strips "i-" prefix → "mgdeadbeef" (10 chars)
|
||||
assert.True(t, strings.HasPrefix(got, "mg"), "expected mg prefix, got %q", got)
|
||||
assert.LessOrEqual(t, len(got), 15)
|
||||
}
|
||||
|
||||
func TestTapNameForNIC_ExtraENI(t *testing.T) {
|
||||
input := &handlers_elbv2.SystemInstanceInput{
|
||||
ENIID: "eni-primary",
|
||||
ExtraENIs: []handlers_elbv2.ExtraENIInput{
|
||||
{ENIID: "eni-extra0"},
|
||||
{ENIID: "eni-extra1"},
|
||||
},
|
||||
}
|
||||
got := tapNameForNIC(2, handlers_elbv2.NICConfig{}, "i-inst", input)
|
||||
assert.True(t, strings.HasPrefix(got, "tap"), "expected tap prefix for extra ENI, got %q", got)
|
||||
}
|
||||
|
||||
func TestTapNameForNIC_ExtraENI_OutOfRange(t *testing.T) {
|
||||
input := &handlers_elbv2.SystemInstanceInput{
|
||||
ENIID: "eni-primary",
|
||||
ExtraENIs: []handlers_elbv2.ExtraENIInput{},
|
||||
}
|
||||
// idx=2 → extraIdx=0 but no ExtraENIs → fallback name
|
||||
got := tapNameForNIC(2, handlers_elbv2.NICConfig{}, "i-inst", input)
|
||||
assert.Equal(t, fmt.Sprintf("tap-unknown-%d", 2), got)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// buildNICNetdevs
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestBuildNICNetdevs_SingleNIC(t *testing.T) {
|
||||
input := &handlers_elbv2.SystemInstanceInput{
|
||||
ENIID: "eni-000000000000000a",
|
||||
NICs: []handlers_elbv2.NICConfig{
|
||||
{MAC: "02:aa:bb:cc:dd:01", IsDefault: true},
|
||||
},
|
||||
}
|
||||
res := buildNICNetdevs("i-test", input, microvmMachineType())
|
||||
require.Len(t, res.netdevs, 1)
|
||||
require.Len(t, res.devices, 1)
|
||||
assert.Contains(t, res.netdevs[0].Value, "tap,id=net0,")
|
||||
// microvm machine type → virtio-net-device transport
|
||||
assert.Contains(t, res.devices[0].Value, "virtio-net-device,netdev=net0")
|
||||
assert.Contains(t, res.devices[0].Value, "02:aa:bb:cc:dd:01")
|
||||
}
|
||||
|
||||
func TestBuildNICNetdevs_TwoNICs(t *testing.T) {
|
||||
input := &handlers_elbv2.SystemInstanceInput{
|
||||
ENIID: "eni-000000000000000b",
|
||||
NICs: []handlers_elbv2.NICConfig{
|
||||
{MAC: "02:aa:bb:cc:dd:01", IsDefault: true},
|
||||
{MAC: "02:aa:bb:cc:dd:02", IsDefault: false},
|
||||
},
|
||||
}
|
||||
res := buildNICNetdevs("i-test2", input, microvmMachineType())
|
||||
require.Len(t, res.netdevs, 2)
|
||||
require.Len(t, res.devices, 2)
|
||||
assert.Contains(t, res.netdevs[0].Value, "id=net0,")
|
||||
assert.Contains(t, res.netdevs[1].Value, "id=net1,")
|
||||
assert.Contains(t, res.devices[0].Value, "netdev=net0")
|
||||
assert.Contains(t, res.devices[1].Value, "netdev=net1")
|
||||
}
|
||||
|
||||
func TestBuildNICNetdevs_EmptyNICs(t *testing.T) {
|
||||
input := &handlers_elbv2.SystemInstanceInput{ENIID: "eni-empty"}
|
||||
res := buildNICNetdevs("i-empty", input, microvmMachineType())
|
||||
assert.Empty(t, res.netdevs)
|
||||
assert.Empty(t, res.devices)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// microvmMachineType
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestMicrovmMachineType(t *testing.T) {
|
||||
mt := microvmMachineType()
|
||||
assert.Contains(t, mt, "microvm,")
|
||||
assert.Contains(t, mt, "isa-serial=on")
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// writeFwCfgBlobs
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestWriteFwCfgBlobs_HappyPath(t *testing.T) {
|
||||
// Use a temp dir as runtime dir so files land somewhere writable.
|
||||
tmpDir := t.TempDir()
|
||||
t.Setenv("XDG_RUNTIME_DIR", tmpDir)
|
||||
|
||||
d := &Daemon{}
|
||||
instanceID := "i-fwcfg-test"
|
||||
input := &handlers_elbv2.SystemInstanceInput{
|
||||
NICs: []handlers_elbv2.NICConfig{
|
||||
{MAC: "02:aa:bb:cc:dd:01", CIDR: "10.0.1.5/24", Gateway: "10.0.1.1", IsDefault: true},
|
||||
},
|
||||
LBAgentEnv: "LB_BACKEND=10.0.1.10:8080\n",
|
||||
CACert: "-----BEGIN CERTIFICATE-----\nfake\n-----END CERTIFICATE-----\n",
|
||||
}
|
||||
|
||||
entries, err := d.writeFwCfgBlobs(instanceID, input)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, entries, 3)
|
||||
|
||||
// Check names are as expected.
|
||||
assert.Equal(t, "opt/spinifex/netcfg", entries[0].Name)
|
||||
assert.Equal(t, "opt/spinifex/lb-agent-env", entries[1].Name)
|
||||
assert.Equal(t, "opt/spinifex/ca-cert", entries[2].Name)
|
||||
|
||||
// Verify files were written and contain expected content.
|
||||
netcfgData, err := os.ReadFile(entries[0].File)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, string(netcfgData), "NIC0_MAC=02:aa:bb:cc:dd:01")
|
||||
|
||||
lbenvData, err := os.ReadFile(entries[1].File)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "LB_BACKEND=10.0.1.10:8080\n", string(lbenvData))
|
||||
|
||||
cacertData, err := os.ReadFile(entries[2].File)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, string(cacertData), "BEGIN CERTIFICATE")
|
||||
|
||||
// Cleanup: remove the tmpfiles (proves they are real filesystem files).
|
||||
for _, e := range entries {
|
||||
assert.NoError(t, os.Remove(e.File))
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteFwCfgBlobs_InvalidNICs_NoDefault(t *testing.T) {
|
||||
tmpDir := t.TempDir()
|
||||
t.Setenv("XDG_RUNTIME_DIR", tmpDir)
|
||||
|
||||
d := &Daemon{}
|
||||
input := &handlers_elbv2.SystemInstanceInput{
|
||||
NICs: []handlers_elbv2.NICConfig{
|
||||
{MAC: "02:aa:bb:cc:dd:01", IsDefault: false},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := d.writeFwCfgBlobs("i-bad-nics", input)
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "IsDefault")
|
||||
|
||||
// No tmpfiles must be left behind.
|
||||
entries, _ := os.ReadDir(tmpDir)
|
||||
for _, e := range entries {
|
||||
assert.False(t, strings.HasPrefix(e.Name(), "fwcfg-i-bad-nics"),
|
||||
"unexpected tmpfile left behind: %s", e.Name())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"net"
|
||||
"os"
|
||||
"os/exec"
|
||||
"os/user"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
@@ -48,7 +49,20 @@ func (p *OVSNetworkPlumber) SetupTap(spec vm.TapSpec) error {
|
||||
}
|
||||
}
|
||||
|
||||
if out, err := sudoCommand("ip", "tuntap", "add", "dev", spec.Name, "mode", "tap").CombinedOutput(); err != nil {
|
||||
// When the daemon runs non-root (production: User=spinifex-daemon), child
|
||||
// QEMU processes inherit that uid without CAP_NET_ADMIN. The kernel's tun
|
||||
// attach check then requires the tap to be owned by the calling euid
|
||||
// (drivers/net/tun.c: tun_chr_open + TUNSETIFF perm check). Without the
|
||||
// `user` flag the tap defaults to root:root and qemu's TUNSETIFF fails
|
||||
// with EPERM ("could not configure /dev/net/tun: Operation not permitted").
|
||||
// Root daemons skip this — they have CAP_NET_ADMIN ambient.
|
||||
addArgs := []string{"tuntap", "add", "dev", spec.Name, "mode", "tap"}
|
||||
if os.Geteuid() != 0 {
|
||||
if u, err := user.Current(); err == nil {
|
||||
addArgs = append(addArgs, "user", u.Username, "group", u.Gid)
|
||||
}
|
||||
}
|
||||
if out, err := sudoCommand("ip", addArgs...).CombinedOutput(); err != nil {
|
||||
return fmt.Errorf("create tap %s: %s: %w", spec.Name, strings.TrimSpace(string(out)), err)
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Phase gate constants — changing these is a deliberate decision.
|
||||
const (
|
||||
phase1BootGateMs = 500
|
||||
phase1RSSGateMiB = 25
|
||||
|
||||
phase2BootGateMs = 300
|
||||
phase2ArtifactGateMiB = 50
|
||||
)
|
||||
|
||||
func TestBootGate_Phase1(t *testing.T) {
|
||||
if phase1BootGateMs != 500 {
|
||||
t.Errorf("Phase 1 boot gate changed: got %d, want 500", phase1BootGateMs)
|
||||
}
|
||||
if phase1RSSGateMiB != 25 {
|
||||
t.Errorf("Phase 1 RSS gate changed: got %d, want 25", phase1RSSGateMiB)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBootGate_Phase2(t *testing.T) {
|
||||
if phase2BootGateMs != 300 {
|
||||
t.Errorf("Phase 2 boot gate changed: got %d, want 300", phase2BootGateMs)
|
||||
}
|
||||
if phase2ArtifactGateMiB != 50 {
|
||||
t.Errorf("Phase 2 artifact gate changed: got %d, want 50", phase2ArtifactGateMiB)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMeasureQEMURSSMiB_Self(t *testing.T) {
|
||||
rss, err := measureQEMURSSMiB(os.Getpid())
|
||||
if err != nil {
|
||||
t.Fatalf("measureQEMURSSMiB self: %v", err)
|
||||
}
|
||||
if rss <= 0 {
|
||||
t.Errorf("expected RSS > 0, got %f", rss)
|
||||
}
|
||||
if rss > 1024 {
|
||||
t.Errorf("RSS %f MiB seems unreasonably high for a test process", rss)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMeasureQEMURSSMiB_InvalidPID(t *testing.T) {
|
||||
for _, pid := range []int{0, -1, -999} {
|
||||
_, err := measureQEMURSSMiB(pid)
|
||||
if err == nil {
|
||||
t.Errorf("expected error for pid=%d, got nil", pid)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMeasureQEMURSSMiB_NonExistentPID(t *testing.T) {
|
||||
_, err := measureQEMURSSMiB(999999999)
|
||||
if err == nil {
|
||||
t.Error("expected error for non-existent pid, got nil")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// measureQEMURSSMiB reads /proc/<pid>/status and returns RSS in MiB.
|
||||
func measureQEMURSSMiB(pid int) (float64, error) {
|
||||
if pid <= 0 {
|
||||
return 0, fmt.Errorf("invalid pid: %d", pid)
|
||||
}
|
||||
f, err := os.Open(fmt.Sprintf("/proc/%d/status", pid))
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("open /proc/%d/status: %w", pid, err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
scanner := bufio.NewScanner(f)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if !strings.HasPrefix(line, "VmRSS:") {
|
||||
continue
|
||||
}
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) < 2 {
|
||||
return 0, fmt.Errorf("unexpected VmRSS format: %q", line)
|
||||
}
|
||||
kb, err := strconv.ParseFloat(fields[1], 64)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("parse VmRSS kB: %w", err)
|
||||
}
|
||||
return kb / 1024, nil
|
||||
}
|
||||
return 0, fmt.Errorf("VmRSS not found in /proc/%d/status", pid)
|
||||
}
|
||||
@@ -67,5 +67,5 @@ func TestGenerateNetworkConfig_MgmtIPWithoutMAC(t *testing.T) {
|
||||
assert.NotContains(t, cfg, "mgmt0:", "mgmt NIC should not appear without MAC")
|
||||
}
|
||||
|
||||
// Route for multi-node is handled via bootcmd in lbVMUserData, not
|
||||
// in the network-config (Alpine cloud-init doesn't support v2 routes).
|
||||
// Route for multi-node LB mgmt traffic is handled via the fw_cfg netcfg
|
||||
// blob delivered to the LB microVM, not in the network-config.
|
||||
|
||||
@@ -149,8 +149,8 @@ func generateNetworkConfig(eniMAC, devMAC, mgmtMAC, mgmtIP string, extraENIMACs
|
||||
addresses:
|
||||
- "%s/24"
|
||||
`, mgmtMAC, mgmtIP)
|
||||
// Route for multi-node is added via bootcmd in lbVMUserData (Alpine
|
||||
// cloud-init does not support v2 routes under ethernets).
|
||||
// Route for multi-node LB mgmt traffic is delivered to the LB microVM
|
||||
// via the fw_cfg netcfg blob, not here.
|
||||
}
|
||||
|
||||
return cfg
|
||||
@@ -877,7 +877,20 @@ func (s *InstanceServiceImpl) newViperblock(volumeName string, size int, volumeC
|
||||
VolumeConfig: volumeConfig,
|
||||
}
|
||||
|
||||
return viperblock.New(&vbconfig, "s3", cfg)
|
||||
vb, err := viperblock.New(&vbconfig, "s3", cfg)
|
||||
restoreSlogDefault()
|
||||
return vb, err
|
||||
}
|
||||
|
||||
// restoreSlogDefault re-installs the daemon's Info-level slog handler after
|
||||
// viperblock.New mutates the global slog default via its SetDebug method
|
||||
// (see viperblock.go SetDebug — it calls slog.SetDefault with LevelError,
|
||||
// silencing every Info/Warn in the entire process). Tracked for proper fix
|
||||
// in viperblock as mulga-siv-70.
|
||||
func restoreSlogDefault() {
|
||||
slog.SetDefault(slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{
|
||||
Level: slog.LevelInfo,
|
||||
})))
|
||||
}
|
||||
|
||||
// prepareRootVolume handles creation/cloning of the root volume
|
||||
@@ -888,8 +901,6 @@ func (s *InstanceServiceImpl) prepareRootVolume(input *ec2.RunInstancesInput, im
|
||||
return errors.New(awserrors.ErrorServerInternal)
|
||||
}
|
||||
|
||||
vb.SetDebug(false)
|
||||
|
||||
// Initialize the backend
|
||||
err = vb.Backend.Init()
|
||||
if err != nil {
|
||||
@@ -996,8 +1007,6 @@ func (s *InstanceServiceImpl) prepareEFIVolume(imageId string, volumeConfig vipe
|
||||
return errors.New(awserrors.ErrorServerInternal)
|
||||
}
|
||||
|
||||
efiVb.SetDebug(false)
|
||||
|
||||
// Initialize the backend
|
||||
slog.Debug("Initializing EFI Viperblock store backend")
|
||||
err = efiVb.Backend.Init()
|
||||
@@ -1083,8 +1092,6 @@ func (s *InstanceServiceImpl) prepareCloudInitVolume(input *ec2.RunInstancesInpu
|
||||
return errors.New(awserrors.ErrorServerInternal)
|
||||
}
|
||||
|
||||
cloudInitVb.SetDebug(false)
|
||||
|
||||
// Initialize the backend
|
||||
slog.Debug("Initializing cloud-init Viperblock store backend")
|
||||
err = cloudInitVb.Backend.Init()
|
||||
|
||||
+221
-104
@@ -4,63 +4,16 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
"github.com/aws/aws-sdk-go/service/ec2"
|
||||
"github.com/aws/aws-sdk-go/service/elbv2"
|
||||
"github.com/mulgadc/spinifex/spinifex/config"
|
||||
handlers_ec2_vpc "github.com/mulgadc/spinifex/spinifex/handlers/ec2/vpc"
|
||||
"github.com/mulgadc/spinifex/spinifex/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// extractAgentEnv parses the cloud-config user-data into the env-var map that
|
||||
// cloud-init will write to /etc/conf.d/lb-agent. We don't pull in a YAML
|
||||
// dependency just for tests — the format is stable enough that line-based
|
||||
// parsing catches the things substring matching misses (misplaced keys,
|
||||
// broken indentation, content that escaped the agent-conf block).
|
||||
//
|
||||
// Returns the parsed env map plus the `runcmd:` section's body (raw lines)
|
||||
// so callers can assert on the launch command shape.
|
||||
func extractAgentEnv(t *testing.T, ud string) (envs map[string]string, runcmd []string) {
|
||||
t.Helper()
|
||||
require.True(t, strings.HasPrefix(ud, "#cloud-config\n"),
|
||||
"user-data must start with #cloud-config directive")
|
||||
|
||||
envs = map[string]string{}
|
||||
const agentPathMarker = " - path: /etc/conf.d/lb-agent"
|
||||
const contentMarker = " content: |"
|
||||
const envIndent = " " // 6 spaces under `content: |`
|
||||
|
||||
lines := strings.Split(ud, "\n")
|
||||
inAgentEnv := false
|
||||
inRuncmd := false
|
||||
for i := 0; i < len(lines); i++ {
|
||||
line := lines[i]
|
||||
switch {
|
||||
case line == agentPathMarker:
|
||||
require.Less(t, i+1, len(lines), "agent path entry truncated")
|
||||
require.Equal(t, contentMarker, lines[i+1],
|
||||
"agent path must be followed by `content: |` block")
|
||||
inAgentEnv = true
|
||||
i++ // skip the content marker
|
||||
case inAgentEnv:
|
||||
if !strings.HasPrefix(line, envIndent) {
|
||||
inAgentEnv = false
|
||||
i-- // re-process this line in case it starts another section
|
||||
continue
|
||||
}
|
||||
kv := strings.TrimPrefix(line, envIndent)
|
||||
k, v, ok := strings.Cut(kv, "=")
|
||||
require.True(t, ok, "agent env line missing '=': %q", line)
|
||||
envs[k] = v
|
||||
case line == "runcmd:":
|
||||
inRuncmd = true
|
||||
case inRuncmd:
|
||||
if !strings.HasPrefix(line, " ") {
|
||||
inRuncmd = false
|
||||
continue
|
||||
}
|
||||
runcmd = append(runcmd, line)
|
||||
}
|
||||
}
|
||||
return envs, runcmd
|
||||
}
|
||||
|
||||
func TestBuildLBArn(t *testing.T) {
|
||||
arn := buildLBArn("us-east-1", "123456789012", "my-alb", "50dc6c495c0c9188", LoadBalancerTypeApplication)
|
||||
assert.Equal(t, "arn:aws:elasticloadbalancing:us-east-1:123456789012:loadbalancer/app/my-alb/50dc6c495c0c9188", arn)
|
||||
@@ -86,70 +39,234 @@ func TestBuildListenerArn_NLB(t *testing.T) {
|
||||
assert.Equal(t, "arn:aws:elasticloadbalancing:eu-west-1:999888777666:listener/net/my-nlb/lbid123/listener456", arn)
|
||||
}
|
||||
|
||||
// TestLbVMUserData_Structural parses the generated cloud-config and asserts
|
||||
// the structure: agent env vars land in /etc/conf.d/lb-agent, runcmd starts
|
||||
// lb-agent via OpenRC (not the bare binary), the CA cert section — owned by
|
||||
// the instance service's cloud-init template — is absent, and no bootcmd is
|
||||
// emitted in the single-node default.
|
||||
func TestLbVMUserData_Structural(t *testing.T) {
|
||||
// TestBuildLBAgentEnv verifies that buildLBAgentEnv produces a KEY=value blob
|
||||
// containing all five env vars with the correct values.
|
||||
func TestBuildLBAgentEnv(t *testing.T) {
|
||||
svc := &ELBv2ServiceImpl{
|
||||
GatewayURL: "https://192.168.1.33:9999",
|
||||
GatewayURL: "https://10.0.0.1:9999",
|
||||
SystemAccessKey: "AKIAIOSFODNN7EXAMPLE",
|
||||
SystemSecretKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
|
||||
region: "ap-southeast-2",
|
||||
}
|
||||
ud, err := svc.lbVMUserData("lb-abc123", SchemeInternetFacing)
|
||||
env := svc.buildLBAgentEnv("lb-abc123")
|
||||
|
||||
lines := strings.Split(strings.TrimRight(env, "\n"), "\n")
|
||||
kvs := make(map[string]string, len(lines))
|
||||
for _, l := range lines {
|
||||
k, v, ok := strings.Cut(l, "=")
|
||||
require.True(t, ok, "env line missing '=': %q", l)
|
||||
kvs[k] = v
|
||||
}
|
||||
|
||||
assert.Equal(t, "lb-abc123", kvs["LB_LB_ID"])
|
||||
assert.Equal(t, "https://10.0.0.1:9999", kvs["LB_GATEWAY_URL"])
|
||||
assert.Equal(t, "AKIAIOSFODNN7EXAMPLE", kvs["LB_ACCESS_KEY"])
|
||||
assert.Equal(t, "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", kvs["LB_SECRET_KEY"])
|
||||
assert.Equal(t, "ap-southeast-2", kvs["LB_REGION"])
|
||||
}
|
||||
|
||||
// TestSubnetCIDRForIP and TestSubnetGatewayIP cover the CIDR helpers.
|
||||
func TestSubnetCIDRForIP(t *testing.T) {
|
||||
assert.Equal(t, "10.0.1.5/24", subnetCIDRForIP("10.0.1.5", "10.0.1.0/24"))
|
||||
assert.Equal(t, "172.16.2.10/20", subnetCIDRForIP("172.16.2.10", "172.16.0.0/20"))
|
||||
assert.Equal(t, "", subnetCIDRForIP("10.0.1.5", "bad-cidr"))
|
||||
}
|
||||
|
||||
func TestSubnetGatewayIP(t *testing.T) {
|
||||
assert.Equal(t, "10.0.1.1", subnetGatewayIP("10.0.1.0/24"))
|
||||
assert.Equal(t, "172.16.0.1", subnetGatewayIP("172.16.0.0/20"))
|
||||
assert.Equal(t, "", subnetGatewayIP("not-a-cidr"))
|
||||
}
|
||||
|
||||
// setupMicrovmTestService creates an ELBv2 service wired to a real VPC
|
||||
// service. The VPC has one subnet (10.0.1.0/24) pre-created.
|
||||
func setupMicrovmTestService(t *testing.T) (*ELBv2ServiceImpl, *handlers_ec2_vpc.VPCServiceImpl, string) {
|
||||
t.Helper()
|
||||
_, nc, _ := testutil.StartTestJetStream(t)
|
||||
testutil.StubVpcdSGResponder(t, nc)
|
||||
|
||||
vpcSvc, err := handlers_ec2_vpc.NewVPCServiceImplWithNATS(nil, nc)
|
||||
require.NoError(t, err)
|
||||
|
||||
envs, runcmd := extractAgentEnv(t, ud)
|
||||
assert.Equal(t, map[string]string{
|
||||
"LB_LB_ID": "lb-abc123",
|
||||
"LB_GATEWAY_URL": "https://192.168.1.33:9999",
|
||||
"LB_ACCESS_KEY": "AKIAIOSFODNN7EXAMPLE",
|
||||
"LB_SECRET_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
|
||||
"LB_REGION": "ap-southeast-2",
|
||||
}, envs)
|
||||
for k := range envs {
|
||||
assert.NotContains(t, k, "NATS", "NATS URL must not leak into agent config")
|
||||
cfg := &config.Config{
|
||||
Daemon: config.DaemonConfig{
|
||||
DevNetworking: true,
|
||||
},
|
||||
}
|
||||
elbv2Svc, err := NewELBv2ServiceImplWithNATS(cfg, nc)
|
||||
require.NoError(t, err)
|
||||
elbv2Svc.VPCService = vpcSvc
|
||||
elbv2Svc.GatewayURL = "https://10.20.0.5:9999"
|
||||
elbv2Svc.SystemAccessKey = "AKID"
|
||||
elbv2Svc.SystemSecretKey = "SECRET"
|
||||
elbv2Svc.region = "us-east-1"
|
||||
elbv2Svc.CACert = "-----BEGIN CERTIFICATE-----\nMOCK\n-----END CERTIFICATE-----\n"
|
||||
|
||||
// Agent is launched via OpenRC, never as a bare binary path.
|
||||
require.Len(t, runcmd, 1)
|
||||
assert.Contains(t, runcmd[0], `[ "rc-service", "lb-agent", "start" ]`)
|
||||
assert.NotContains(t, ud, "/usr/local/bin/lb-agent")
|
||||
vpcOut, err := vpcSvc.CreateVpc(&ec2.CreateVpcInput{
|
||||
CidrBlock: aws.String("10.0.0.0/16"),
|
||||
}, testAccountID)
|
||||
require.NoError(t, err)
|
||||
vpcID := *vpcOut.Vpc.VpcId
|
||||
|
||||
// CA cert is injected upstream by the instance service's template.
|
||||
assert.NotContains(t, ud, "ca_certs:")
|
||||
subnetOut, err := vpcSvc.CreateSubnet(&ec2.CreateSubnetInput{
|
||||
VpcId: aws.String(vpcID),
|
||||
CidrBlock: aws.String("10.0.1.0/24"),
|
||||
AvailabilityZone: aws.String("us-east-1a"),
|
||||
}, testAccountID)
|
||||
require.NoError(t, err)
|
||||
subnetID := *subnetOut.Subnet.SubnetId
|
||||
|
||||
// Single-node default: no MgmtRoute set means no bootcmd.
|
||||
assert.NotContains(t, ud, "bootcmd:")
|
||||
return elbv2Svc, vpcSvc, subnetID
|
||||
}
|
||||
|
||||
// TestLbVMUserData_MissingCredentials covers the three required-field error
|
||||
// paths (gateway URL, access key, secret key) in one table-driven test.
|
||||
func TestLbVMUserData_MissingCredentials(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
svc *ELBv2ServiceImpl
|
||||
}{
|
||||
{
|
||||
name: "missing GatewayURL",
|
||||
svc: &ELBv2ServiceImpl{SystemAccessKey: "AKID", SystemSecretKey: "SECRET"},
|
||||
},
|
||||
{
|
||||
name: "missing SystemAccessKey",
|
||||
svc: &ELBv2ServiceImpl{GatewayURL: "https://10.0.0.1:9999", SystemSecretKey: "SECRET"},
|
||||
},
|
||||
{
|
||||
name: "missing SystemSecretKey",
|
||||
svc: &ELBv2ServiceImpl{GatewayURL: "https://10.0.0.1:9999", SystemAccessKey: "AKID"},
|
||||
// TestCreateLoadBalancer_DeliversCACert asserts the CACert plumbed into the
|
||||
// service is forwarded to the launcher for fw_cfg delivery.
|
||||
func TestCreateLoadBalancer_DeliversCACert(t *testing.T) {
|
||||
svc, _, subnetID := setupMicrovmTestService(t)
|
||||
|
||||
mock := &mockSystemInstanceLauncher{
|
||||
launchResult: &SystemInstanceOutput{
|
||||
InstanceID: "i-microvm-test",
|
||||
PrivateIP: "10.0.1.5",
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
_, err := tt.svc.lbVMUserData("lb-test", SchemeInternetFacing)
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "missing system credentials")
|
||||
})
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
|
||||
_, err := svc.CreateLoadBalancer(&elbv2.CreateLoadBalancerInput{
|
||||
Name: aws.String("mv-alb"),
|
||||
Subnets: []*string{aws.String(subnetID)},
|
||||
}, testAccountID)
|
||||
require.NoError(t, err)
|
||||
|
||||
require.Len(t, mock.launchCalls, 1)
|
||||
assert.Equal(t, "-----BEGIN CERTIFICATE-----\nMOCK\n-----END CERTIFICATE-----\n",
|
||||
mock.launchCalls[0].CACert)
|
||||
}
|
||||
|
||||
// TestCreateLoadBalancer_Microvm_LBAgentEnv asserts all five lb-agent env vars
|
||||
// appear in LBAgentEnv with correct values.
|
||||
func TestCreateLoadBalancer_Microvm_LBAgentEnv(t *testing.T) {
|
||||
svc, _, subnetID := setupMicrovmTestService(t)
|
||||
|
||||
mock := &mockSystemInstanceLauncher{
|
||||
launchResult: &SystemInstanceOutput{InstanceID: "i-env-test", PrivateIP: "10.0.1.6"},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
|
||||
_, err := svc.CreateLoadBalancer(&elbv2.CreateLoadBalancerInput{
|
||||
Name: aws.String("env-alb"),
|
||||
Subnets: []*string{aws.String(subnetID)},
|
||||
}, testAccountID)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, mock.launchCalls, 1)
|
||||
|
||||
envBlob := mock.launchCalls[0].LBAgentEnv
|
||||
kvs := parseEnvBlob(t, envBlob)
|
||||
|
||||
assert.NotEmpty(t, kvs["LB_LB_ID"])
|
||||
assert.Equal(t, "https://10.20.0.5:9999", kvs["LB_GATEWAY_URL"])
|
||||
assert.Equal(t, "AKID", kvs["LB_ACCESS_KEY"])
|
||||
assert.Equal(t, "SECRET", kvs["LB_SECRET_KEY"])
|
||||
assert.Equal(t, "us-east-1", kvs["LB_REGION"])
|
||||
}
|
||||
|
||||
// TestCreateLoadBalancer_Microvm_NICs asserts NIC[0].IsDefault=true and
|
||||
// NIC[1].IsDefault=false, and that CIDR/Gateway are derived from the subnet.
|
||||
func TestCreateLoadBalancer_Microvm_NICs(t *testing.T) {
|
||||
svc, _, subnetID := setupMicrovmTestService(t)
|
||||
|
||||
mock := &mockSystemInstanceLauncher{
|
||||
launchResult: &SystemInstanceOutput{InstanceID: "i-nic-test", PrivateIP: "10.0.1.7"},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
|
||||
_, err := svc.CreateLoadBalancer(&elbv2.CreateLoadBalancerInput{
|
||||
Name: aws.String("nic-alb"),
|
||||
Subnets: []*string{aws.String(subnetID)},
|
||||
}, testAccountID)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, mock.launchCalls, 1)
|
||||
|
||||
nics := mock.launchCalls[0].NICs
|
||||
require.GreaterOrEqual(t, len(nics), 2, "must have at least primary ENI NIC and mgmt NIC")
|
||||
|
||||
// NIC[0]: primary VPC ENI — must be the default route owner.
|
||||
assert.True(t, nics[0].IsDefault, "NIC[0] must have IsDefault=true")
|
||||
assert.NotEmpty(t, nics[0].MAC, "NIC[0] MAC must be populated from ENI")
|
||||
assert.Contains(t, nics[0].CIDR, "/24", "NIC[0] CIDR must include prefix from subnet")
|
||||
assert.Equal(t, "10.0.1.1", nics[0].Gateway, "NIC[0] gateway must be subnet .1 address")
|
||||
|
||||
// NIC[1]: management NIC — daemon allocates IP/MAC.
|
||||
assert.False(t, nics[1].IsDefault, "NIC[1] must have IsDefault=false")
|
||||
}
|
||||
|
||||
// parseEnvBlob parses a KEY=value newline-separated blob into a map.
|
||||
func parseEnvBlob(t *testing.T, blob string) map[string]string {
|
||||
t.Helper()
|
||||
kvs := make(map[string]string)
|
||||
for line := range strings.SplitSeq(strings.TrimRight(blob, "\n"), "\n") {
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
k, v, ok := strings.Cut(line, "=")
|
||||
require.True(t, ok, "env blob line missing '=': %q", line)
|
||||
kvs[k] = v
|
||||
}
|
||||
return kvs
|
||||
}
|
||||
|
||||
// TestBuildMicrovmNICs_NilVPC verifies buildMicrovmNICs works when VPCService
|
||||
// is nil — CIDR and gateway remain empty, NIC structure is still correct.
|
||||
func TestBuildMicrovmNICs_NilVPC(t *testing.T) {
|
||||
svc := &ELBv2ServiceImpl{VPCService: nil}
|
||||
nics := svc.buildMicrovmNICs("10.0.1.5", "02:aa:bb:cc:dd:01", "subnet-abc", "eni-abc", SchemeInternal, nil, testAccountID)
|
||||
require.Len(t, nics, 2, "primary ENI NIC + mgmt NIC")
|
||||
assert.True(t, nics[0].IsDefault)
|
||||
assert.Equal(t, "02:aa:bb:cc:dd:01", nics[0].MAC)
|
||||
assert.Empty(t, nics[0].CIDR, "CIDR unknown when VPC service unavailable")
|
||||
assert.False(t, nics[1].IsDefault)
|
||||
}
|
||||
|
||||
// TestBuildMicrovmNICs_ExtraENIs verifies NIC[2+] are appended for multi-subnet ALBs.
|
||||
func TestBuildMicrovmNICs_ExtraENIs(t *testing.T) {
|
||||
svc := &ELBv2ServiceImpl{VPCService: nil}
|
||||
extras := []ExtraENIInput{
|
||||
{ENIID: "eni-extra1", ENIMac: "02:aa:bb:cc:dd:02", ENIIP: "10.0.2.5", SubnetID: "subnet-extra1"},
|
||||
}
|
||||
nics := svc.buildMicrovmNICs("10.0.1.5", "02:aa:bb:cc:dd:01", "subnet-abc", "eni-abc", SchemeInternal, extras, testAccountID)
|
||||
require.Len(t, nics, 3, "primary ENI + mgmt + 1 extra")
|
||||
assert.Equal(t, "02:aa:bb:cc:dd:02", nics[2].MAC)
|
||||
assert.False(t, nics[2].IsDefault)
|
||||
}
|
||||
|
||||
// TestBuildMicrovmNICs_MgmtRoute covers the same single-node/multi-node matrix
|
||||
// as TestLBVMUserData_InternalSingleNodeFallback: internet-facing single-node
|
||||
// must NOT emit a /32 via mgmt to AdvertiseIP because that steals the host's
|
||||
// WAN return path for reply traffic, breaking same-chassis ingress.
|
||||
func TestBuildMicrovmNICs_MgmtRoute(t *testing.T) {
|
||||
mk := func() *ELBv2ServiceImpl {
|
||||
return &ELBv2ServiceImpl{
|
||||
VPCService: nil,
|
||||
MgmtBridgeIP: "10.15.8.1",
|
||||
AdvertiseIP: "192.168.1.33",
|
||||
}
|
||||
}
|
||||
|
||||
internal := mk().buildMicrovmNICs("10.0.1.5", "02:aa:bb:cc:dd:01", "subnet-abc", "eni-abc", SchemeInternal, nil, testAccountID)
|
||||
require.Len(t, internal, 2)
|
||||
assert.Equal(t, "192.168.1.33", internal[1].RouteDst, "internal single-node forces /32 via mgmt")
|
||||
assert.Equal(t, "10.15.8.1", internal[1].RouteVia)
|
||||
|
||||
inet := mk().buildMicrovmNICs("10.0.1.5", "02:aa:bb:cc:dd:01", "subnet-abc", "eni-abc", SchemeInternetFacing, nil, testAccountID)
|
||||
require.Len(t, inet, 2)
|
||||
assert.Empty(t, inet[1].RouteDst, "internet-facing single-node must not /32 AdvertiseIP via mgmt")
|
||||
assert.Empty(t, inet[1].RouteVia)
|
||||
|
||||
multi := mk()
|
||||
multi.MgmtRouteGateway = "10.15.8.1"
|
||||
multi.MgmtRouteTarget = "10.15.8.100"
|
||||
nics := multi.buildMicrovmNICs("10.0.1.5", "02:aa:bb:cc:dd:01", "subnet-abc", "eni-abc", SchemeInternetFacing, nil, testAccountID)
|
||||
require.Len(t, nics, 2)
|
||||
assert.Equal(t, "10.15.8.100", nics[1].RouteDst, "multi-node uses explicit MgmtRouteTarget")
|
||||
assert.Equal(t, "10.15.8.1", nics[1].RouteVia)
|
||||
}
|
||||
|
||||
@@ -19,15 +19,15 @@ type SystemInstanceLauncher interface {
|
||||
TerminateSystemInstance(instanceID string) error
|
||||
}
|
||||
|
||||
// SystemInstanceInput describes the VM to launch.
|
||||
// SystemInstanceInput describes the direct-boot microVM to launch for an
|
||||
// ELBv2 load balancer. There is no AMI or cloud-init path — kernel and
|
||||
// initrd come from the bundled spinifex package and per-VM config is
|
||||
// delivered via QEMU fw_cfg blobs.
|
||||
type SystemInstanceInput struct {
|
||||
InstanceType string // e.g. "t3.nano"
|
||||
ImageID string // AMI ID
|
||||
InstanceType string // e.g. "sys.micro"
|
||||
SubnetID string // VPC subnet for networking
|
||||
UserData string // Cloud-init user data (plain text, will be base64-encoded)
|
||||
|
||||
// ENI fields — if ENIID is set, the VM uses this pre-created ENI instead
|
||||
// of auto-creating one. Used for ALB VMs that reuse the ALB's ENI.
|
||||
// ENI fields — the VM always uses a pre-created ENI (the ALB's primary ENI).
|
||||
ENIID string
|
||||
ENIMac string
|
||||
ENIIP string
|
||||
@@ -49,6 +49,17 @@ type SystemInstanceInput struct {
|
||||
// via the QEMU user-mode dev NIC (dev_networking mode only).
|
||||
// Each entry is a guest port (e.g. 80, 443). The host port is auto-assigned.
|
||||
HostfwdPorts []int
|
||||
|
||||
// NICs defines the network interfaces for the microVM.
|
||||
// Index 0 is the primary VPC ENI, index 1 is the management NIC, index 2+ are extra ENIs.
|
||||
NICs []NICConfig
|
||||
|
||||
// LBAgentEnv is a KEY=value blob written to /etc/conf.d/lb-agent inside
|
||||
// the guest via fw_cfg.
|
||||
LBAgentEnv string
|
||||
|
||||
// CACert holds PEM-encoded CA certificate bytes delivered to the guest via fw_cfg.
|
||||
CACert string
|
||||
}
|
||||
|
||||
// ExtraENIInput describes an additional pre-created ENI to attach to a
|
||||
@@ -60,6 +71,17 @@ type ExtraENIInput struct {
|
||||
SubnetID string
|
||||
}
|
||||
|
||||
// NICConfig describes a single network interface for a microVM direct-boot launch.
|
||||
// Index 0 is the primary VPC ENI, index 1 is the management NIC, index 2+ are extra ENIs.
|
||||
type NICConfig struct {
|
||||
MAC string // e.g. "02:0a:01:23:45:67"
|
||||
CIDR string // e.g. "10.0.1.5/24"
|
||||
Gateway string // e.g. "10.0.1.1"; empty for mgmt NIC
|
||||
IsDefault bool // true for exactly one NIC (primary VPC ENI)
|
||||
RouteDst string // specific host route dst, e.g. "10.20.0.5/32"
|
||||
RouteVia string // next-hop for RouteDst
|
||||
}
|
||||
|
||||
// SystemInstanceOutput contains the result of a successful launch.
|
||||
type SystemInstanceOutput struct {
|
||||
InstanceID string // e.g. "i-xxxxx"
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"maps"
|
||||
"net"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
@@ -64,45 +65,10 @@ func validateHealthCheckMatcher(m string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// lbVMUserData generates cloud-config user data for a load balancer VM.
|
||||
// Uses write_files to populate /etc/conf.d/lb-agent with the lb-id, gateway
|
||||
// URL, and system credentials for SigV4 auth. The CA cert is already injected
|
||||
// by the instance service's cloud-init template (same as regular EC2 VMs).
|
||||
// Cloud-init guarantees write_files runs before runcmd. The service is NOT
|
||||
// enabled at boot in the image — cloud-init is the sole trigger so the env
|
||||
// vars are always present before the agent starts.
|
||||
func (s *ELBv2ServiceImpl) lbVMUserData(lbID, scheme string) (string, error) {
|
||||
if s.GatewayURL == "" || s.SystemAccessKey == "" || s.SystemSecretKey == "" {
|
||||
return "", fmt.Errorf("missing system credentials: gatewayURL=%q accessKey=%q secretKey-set=%t",
|
||||
s.GatewayURL, s.SystemAccessKey, s.SystemSecretKey != "")
|
||||
}
|
||||
|
||||
cfg := fmt.Sprintf(`#cloud-config
|
||||
write_files:
|
||||
- path: /etc/conf.d/lb-agent
|
||||
content: |
|
||||
LB_LB_ID=%s
|
||||
LB_GATEWAY_URL=%s
|
||||
LB_ACCESS_KEY=%s
|
||||
LB_SECRET_KEY=%s
|
||||
LB_REGION=%s
|
||||
`, lbID, s.GatewayURL, s.SystemAccessKey, s.SystemSecretKey, s.region)
|
||||
|
||||
mgmtGW, mgmtTarget := s.resolveMgmtRoute(scheme)
|
||||
if mgmtGW != "" && mgmtTarget != "" {
|
||||
cfg += fmt.Sprintf(`bootcmd:
|
||||
- [ "ip", "route", "add", "%s/32", "via", "%s" ]
|
||||
`, mgmtTarget, mgmtGW)
|
||||
}
|
||||
|
||||
cfg += `runcmd:
|
||||
- [ "rc-service", "lb-agent", "start" ]
|
||||
`
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
// resolveMgmtRoute returns the (gateway, target) pair for the bootcmd host
|
||||
// route the lb-agent uses to reach AWSGW, or empty strings to skip the route.
|
||||
// resolveMgmtRoute returns the (gateway, target) pair for the host route the
|
||||
// lb-agent uses to reach AWSGW from inside the LB microVM, or empty strings
|
||||
// to skip the route. Consumed by buildMicrovmNICs to populate NIC[1]'s
|
||||
// RouteDst/RouteVia for the netcfg fw_cfg blob.
|
||||
//
|
||||
// Multi-node (AWSGW on dedicated mgmt IP): MgmtRoute{Gateway,Target} are set
|
||||
// by the daemon for both schemes — return them.
|
||||
@@ -141,16 +107,13 @@ type ELBv2ServiceImpl struct {
|
||||
MgmtRouteTarget string // AWSGW bind IP to route via mgmt NIC
|
||||
MgmtBridgeIP string // br-mgmt IP, populated whenever br-mgmt exists (single + multi node) for the internal-scheme fallback route
|
||||
AdvertiseIP string // AdvertiseIP / WAN gateway, populated whenever set; used as the internal-scheme fallback route target on single-node
|
||||
CACert string // PEM-encoded CA certificate delivered to microvm guests via fw_cfg
|
||||
nodeID string
|
||||
region string
|
||||
systemAMI string // AMI ID for system VMs (ALB VMs); resolved lazily via systemAMIFunc
|
||||
systemAMIFunc func() (string, error) // returns the current system AMI ID (queries image store)
|
||||
systemAMIMu sync.Mutex // guards lazy resolution of systemAMI
|
||||
systemAMIResolved bool // true once systemAMI has been resolved to a non-empty value
|
||||
systemInstanceType string // instance type for system VMs; resolved lazily via systemInstanceTypeFunc
|
||||
systemInstanceTypeFunc func() string // returns the smallest available instance type
|
||||
systemInstanceTypeMu sync.Mutex // guards lazy resolution of systemInstanceType
|
||||
systemInstanceTypeResolved bool // true once systemInstanceType has been resolved to a non-empty value
|
||||
systemInstanceType string // instance type for system VMs; resolved lazily via systemInstanceTypeFunc
|
||||
systemInstanceTypeFunc func() string // returns the smallest available instance type
|
||||
systemInstanceTypeMu sync.Mutex // guards lazy resolution of systemInstanceType
|
||||
systemInstanceTypeResolved bool // true once systemInstanceType has been resolved to a non-empty value
|
||||
ctx context.Context
|
||||
cancel context.CancelFunc
|
||||
hc *healthChecker
|
||||
@@ -194,39 +157,6 @@ func (s *ELBv2ServiceImpl) Close() {
|
||||
}
|
||||
}
|
||||
|
||||
// SetSystemAMIFunc sets a function that resolves the current system AMI ID.
|
||||
// This is called at request time so the AMI is discovered even if imported
|
||||
// after the daemon starts.
|
||||
func (s *ELBv2ServiceImpl) SetSystemAMIFunc(fn func() (string, error)) {
|
||||
s.systemAMIMu.Lock()
|
||||
defer s.systemAMIMu.Unlock()
|
||||
s.systemAMIFunc = fn
|
||||
s.systemAMI = ""
|
||||
s.systemAMIResolved = false
|
||||
}
|
||||
|
||||
// getSystemAMI returns the system AMI ID, resolving it lazily if needed.
|
||||
// Caches only successful resolutions so the resolver retries after errors.
|
||||
func (s *ELBv2ServiceImpl) getSystemAMI() (string, error) {
|
||||
s.systemAMIMu.Lock()
|
||||
defer s.systemAMIMu.Unlock()
|
||||
if s.systemAMIResolved {
|
||||
return s.systemAMI, nil
|
||||
}
|
||||
if s.systemAMIFunc == nil {
|
||||
return "", nil
|
||||
}
|
||||
ami, err := s.systemAMIFunc()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if ami != "" {
|
||||
s.systemAMI = ami
|
||||
s.systemAMIResolved = true
|
||||
}
|
||||
return ami, nil
|
||||
}
|
||||
|
||||
// SetSystemInstanceTypeFunc sets a function that resolves the smallest available
|
||||
// instance type. Called at request time so it adapts to node capacity.
|
||||
func (s *ELBv2ServiceImpl) SetSystemInstanceTypeFunc(fn func() string) {
|
||||
@@ -255,6 +185,111 @@ func (s *ELBv2ServiceImpl) getSystemInstanceType() string {
|
||||
return s.systemInstanceType
|
||||
}
|
||||
|
||||
// buildLBAgentEnv returns the KEY=value blob written to /etc/conf.d/lb-agent
|
||||
// via fw_cfg on the direct-boot path.
|
||||
func (s *ELBv2ServiceImpl) buildLBAgentEnv(lbID string) string {
|
||||
return fmt.Sprintf("LB_LB_ID=%s\nLB_GATEWAY_URL=%s\nLB_ACCESS_KEY=%s\nLB_SECRET_KEY=%s\nLB_REGION=%s\n",
|
||||
lbID, s.GatewayURL, s.SystemAccessKey, s.SystemSecretKey, s.region)
|
||||
}
|
||||
|
||||
// subnetCIDRForIP computes "ip/prefixlen" given a host IP and the subnet's
|
||||
// CIDR block (e.g. "10.0.1.0/24" → "10.0.1.5/24"). Returns empty string on
|
||||
// parse failure so the caller can log and continue without panicking.
|
||||
func subnetCIDRForIP(hostIP, subnetCIDR string) string {
|
||||
_, ipNet, err := net.ParseCIDR(subnetCIDR)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
ones, _ := ipNet.Mask.Size()
|
||||
return fmt.Sprintf("%s/%d", hostIP, ones)
|
||||
}
|
||||
|
||||
// subnetGatewayIP derives the gateway IP (network address +1) from a subnet
|
||||
// CIDR block. Returns empty string on parse failure.
|
||||
func subnetGatewayIP(subnetCIDR string) string {
|
||||
_, ipNet, err := net.ParseCIDR(subnetCIDR)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
gw := ipNet.IP.To4()
|
||||
if gw == nil {
|
||||
return ""
|
||||
}
|
||||
gwCopy := make(net.IP, len(gw))
|
||||
copy(gwCopy, gw)
|
||||
gwCopy[3]++
|
||||
return gwCopy.String()
|
||||
}
|
||||
|
||||
// buildMicrovmNICs constructs the NIC slice for a direct-boot microvm launch.
|
||||
//
|
||||
// NIC[0] is the primary VPC ENI (IsDefault=true): MAC/IP from the ENI record,
|
||||
// CIDR from ip+subnet prefix, gateway = subnet network address +1.
|
||||
//
|
||||
// NIC[1] is the management NIC (IsDefault=false): MAC and CIDR are left empty
|
||||
// here — the daemon injects them after allocating the per-instance mgmt IP.
|
||||
// RouteDst/RouteVia encode the host route the lb-agent needs to reach AWSGW
|
||||
// via the management interface (same logic as resolveMgmtRoute).
|
||||
//
|
||||
// NIC[2+] are extra VPC ENIs for multi-subnet ALBs.
|
||||
func (s *ELBv2ServiceImpl) buildMicrovmNICs(primaryIP, primaryMAC, primarySubnetID, _, scheme string, extraENIs []ExtraENIInput, accountID string) []NICConfig {
|
||||
// Resolve primary subnet CIDR for NIC[0] prefix and gateway.
|
||||
primaryCIDR := ""
|
||||
primaryGateway := ""
|
||||
if s.VPCService != nil && primarySubnetID != "" {
|
||||
if subnet, err := s.VPCService.GetSubnet(accountID, primarySubnetID); err == nil && subnet != nil {
|
||||
primaryCIDR = subnetCIDRForIP(primaryIP, subnet.CidrBlock)
|
||||
primaryGateway = subnetGatewayIP(subnet.CidrBlock)
|
||||
} else {
|
||||
slog.Warn("buildMicrovmNICs: could not look up primary subnet", "subnetId", primarySubnetID, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
nics := []NICConfig{
|
||||
// NIC[0]: primary VPC ENI — default route owner.
|
||||
{
|
||||
MAC: primaryMAC,
|
||||
CIDR: primaryCIDR,
|
||||
Gateway: primaryGateway,
|
||||
IsDefault: true,
|
||||
},
|
||||
}
|
||||
|
||||
// NIC[1]: management NIC — daemon fills MAC/CIDR after IP allocation.
|
||||
// RouteDst = AWSGW IP (destination), RouteVia = br-mgmt IP (next-hop).
|
||||
// Honor resolveMgmtRoute's deliberate empty return for single-node
|
||||
// internet-facing: adding a /32 to AdvertiseIP via mgmt steals the host's
|
||||
// WAN return path for reply traffic, breaking same-chassis ingress.
|
||||
mgmtRouteGW, mgmtRouteTarget := s.resolveMgmtRoute(scheme)
|
||||
nics = append(nics, NICConfig{
|
||||
IsDefault: false,
|
||||
RouteDst: mgmtRouteTarget, // AWSGW IP — destination to reach
|
||||
RouteVia: mgmtRouteGW, // br-mgmt IP — next-hop
|
||||
})
|
||||
|
||||
// NIC[2+]: extra ENIs for multi-subnet ALBs.
|
||||
for _, extra := range extraENIs {
|
||||
extraCIDR := ""
|
||||
extraGateway := ""
|
||||
if s.VPCService != nil && extra.SubnetID != "" {
|
||||
if subnet, err := s.VPCService.GetSubnet(accountID, extra.SubnetID); err == nil && subnet != nil {
|
||||
extraCIDR = subnetCIDRForIP(extra.ENIIP, subnet.CidrBlock)
|
||||
extraGateway = subnetGatewayIP(subnet.CidrBlock)
|
||||
} else {
|
||||
slog.Warn("buildMicrovmNICs: could not look up extra subnet", "subnetId", extra.SubnetID, "err", err)
|
||||
}
|
||||
}
|
||||
nics = append(nics, NICConfig{
|
||||
MAC: extra.ENIMac,
|
||||
CIDR: extraCIDR,
|
||||
Gateway: extraGateway,
|
||||
IsDefault: false,
|
||||
})
|
||||
}
|
||||
|
||||
return nics
|
||||
}
|
||||
|
||||
// resolveENIBindAddr looks up the private IP of the first ENI belonging to the ALB.
|
||||
// Returns empty string if VPC service is unavailable or no ENIs exist.
|
||||
func (s *ELBv2ServiceImpl) resolveENIBindAddr(lb *LoadBalancerRecord) string {
|
||||
@@ -671,15 +706,6 @@ func (s *ELBv2ServiceImpl) CreateLoadBalancer(input *elbv2.CreateLoadBalancerInp
|
||||
var hostPorts map[int]int
|
||||
var launchFailed bool
|
||||
if s.InstanceLauncher != nil && len(eniIDs) > 0 && len(subnets) > 0 {
|
||||
systemAMI, amiErr := s.getSystemAMI()
|
||||
if amiErr != nil {
|
||||
slog.Error("CreateLoadBalancer: cannot resolve LB system AMI", "lbId", lbID, "err", amiErr)
|
||||
return nil, errors.New(awserrors.ErrorServerInternal)
|
||||
}
|
||||
if systemAMI == "" {
|
||||
slog.Error("CreateLoadBalancer: LB system AMI not resolved", "lbId", lbID)
|
||||
return nil, errors.New(awserrors.ErrorServerInternal)
|
||||
}
|
||||
// Resolve MAC/IP for every ENI in one describe call.
|
||||
eniDetails := make(map[string]*ec2.NetworkInterface, len(eniIDs))
|
||||
if s.VPCService != nil {
|
||||
@@ -721,23 +747,27 @@ func (s *ELBv2ServiceImpl) CreateLoadBalancer(input *elbv2.CreateLoadBalancerInp
|
||||
})
|
||||
}
|
||||
|
||||
userData, udErr := s.lbVMUserData(lbID, scheme)
|
||||
if udErr != nil {
|
||||
slog.Error("CreateLoadBalancer: system credentials not configured — cannot launch ALB VM", "lbId", lbID, "err", udErr)
|
||||
var launchInput *SystemInstanceInput
|
||||
if s.GatewayURL == "" || s.SystemAccessKey == "" || s.SystemSecretKey == "" {
|
||||
slog.Error("CreateLoadBalancer: system credentials not configured — cannot launch ALB VM", "lbId", lbID)
|
||||
launchFailed = true
|
||||
} else {
|
||||
launchInput := &SystemInstanceInput{
|
||||
nics := s.buildMicrovmNICs(primaryIP, primaryMAC, subnets[0], eniIDs[0], scheme, extraENIInputs, accountID)
|
||||
launchInput = &SystemInstanceInput{
|
||||
InstanceType: s.getSystemInstanceType(),
|
||||
ImageID: systemAMI,
|
||||
SubnetID: subnets[0],
|
||||
UserData: userData,
|
||||
ENIID: eniIDs[0],
|
||||
ENIMac: primaryMAC,
|
||||
ENIIP: primaryIP,
|
||||
ExtraENIs: extraENIInputs,
|
||||
Scheme: scheme,
|
||||
AccountID: accountID,
|
||||
NICs: nics,
|
||||
LBAgentEnv: s.buildLBAgentEnv(lbID),
|
||||
CACert: s.CACert,
|
||||
}
|
||||
}
|
||||
if !launchFailed && launchInput != nil {
|
||||
// Dev-mode only: forward HTTP/HTTPS ports from host for local testing.
|
||||
// In production (VPC networking), traffic reaches the ALB VM's VPC IP directly.
|
||||
if s.config != nil && s.config.Daemon.DevNetworking {
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
package handlers_elbv2
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
@@ -1526,26 +1525,8 @@ func TestUpdateStoredConfig_SkipsWhenNoInstance(t *testing.T) {
|
||||
|
||||
// --- Service lifecycle and setter tests ---
|
||||
|
||||
// TestGetSystemAMI_ColdStart and TestGetSystemInstanceType_ColdStart pin the
|
||||
// no-resolver-wired path. TestGet*_RetryOn{Error,Empty} only exercise paths
|
||||
// after a resolver is set, so they don't cover the boot-time race where a
|
||||
// caller asks for the system AMI/instance type before LB-image import has
|
||||
// installed the resolver. Cold-start must return empty + no error rather
|
||||
// than panic on a nil func.
|
||||
|
||||
func TestGetSystemAMI_ColdStart(t *testing.T) {
|
||||
svc := setupTestService(t)
|
||||
|
||||
ami, err := svc.getSystemAMI()
|
||||
require.NoError(t, err)
|
||||
assert.Empty(t, ami)
|
||||
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-system-001", nil })
|
||||
ami, err = svc.getSystemAMI()
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "ami-system-001", ami)
|
||||
}
|
||||
|
||||
// TestGetSystemInstanceType_ColdStart pins the no-resolver-wired path —
|
||||
// cold-start must return empty rather than panic on a nil func.
|
||||
func TestGetSystemInstanceType_ColdStart(t *testing.T) {
|
||||
svc := setupTestService(t)
|
||||
|
||||
@@ -1555,38 +1536,6 @@ func TestGetSystemInstanceType_ColdStart(t *testing.T) {
|
||||
assert.Equal(t, "t3.micro", svc.getSystemInstanceType())
|
||||
}
|
||||
|
||||
func TestGetSystemAMI_RetryOnError(t *testing.T) {
|
||||
svc := setupTestService(t)
|
||||
|
||||
calls := 0
|
||||
svc.SetSystemAMIFunc(func() (string, error) {
|
||||
calls++
|
||||
if calls < 3 {
|
||||
return "", errors.New("LB system image not imported")
|
||||
}
|
||||
return "ami-system-001", nil
|
||||
})
|
||||
|
||||
// First two calls error — must NOT be cached
|
||||
_, err := svc.getSystemAMI()
|
||||
require.Error(t, err)
|
||||
_, err = svc.getSystemAMI()
|
||||
require.Error(t, err)
|
||||
assert.Equal(t, 2, calls)
|
||||
|
||||
// Third call succeeds — cached from now on
|
||||
ami, err := svc.getSystemAMI()
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "ami-system-001", ami)
|
||||
assert.Equal(t, 3, calls)
|
||||
|
||||
// Subsequent calls return cached value without re-invoking
|
||||
ami, err = svc.getSystemAMI()
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "ami-system-001", ami)
|
||||
assert.Equal(t, 3, calls)
|
||||
}
|
||||
|
||||
func TestGetSystemInstanceType_RetryOnEmpty(t *testing.T) {
|
||||
svc := setupTestService(t)
|
||||
|
||||
@@ -1610,21 +1559,6 @@ func TestGetSystemInstanceType_RetryOnEmpty(t *testing.T) {
|
||||
assert.Equal(t, 2, calls)
|
||||
}
|
||||
|
||||
func TestGetSystemAMI_Concurrent(t *testing.T) {
|
||||
svc := setupTestService(t)
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-system-001", nil })
|
||||
|
||||
var wg sync.WaitGroup
|
||||
for range 50 {
|
||||
wg.Go(func() {
|
||||
got, err := svc.getSystemAMI()
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, "ami-system-001", got)
|
||||
})
|
||||
}
|
||||
wg.Wait()
|
||||
}
|
||||
|
||||
func TestGetSystemInstanceType_Concurrent(t *testing.T) {
|
||||
svc := setupTestService(t)
|
||||
svc.SetSystemInstanceTypeFunc(func() string { return "t3.micro" })
|
||||
@@ -1639,54 +1573,6 @@ func TestGetSystemInstanceType_Concurrent(t *testing.T) {
|
||||
wg.Wait()
|
||||
}
|
||||
|
||||
func TestLBVMUserData_MgmtRoute(t *testing.T) {
|
||||
svc := setupTestService(t)
|
||||
svc.MgmtRouteGateway = "10.15.8.1"
|
||||
svc.MgmtRouteTarget = "10.15.8.100"
|
||||
svc.GatewayURL = "https://10.15.8.100:9999"
|
||||
svc.SystemAccessKey = "AK"
|
||||
svc.SystemSecretKey = "SK"
|
||||
|
||||
data, err := svc.lbVMUserData("lb-test1", SchemeInternetFacing)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, data, "bootcmd:")
|
||||
assert.Contains(t, data, `"10.15.8.100/32"`)
|
||||
assert.Contains(t, data, `"10.15.8.1"`)
|
||||
}
|
||||
|
||||
// TestLBVMUserData_InternalSingleNodeFallback covers the single-node case
|
||||
// where MgmtRoute{Gateway,Target} are empty (AWSGW reachable on advertiseIP
|
||||
// via VPC for internet-facing LBs) but an internal LB has no EIP and so
|
||||
// needs a forced mgmt-NIC route to AWSGW. Internet-facing in the same setup
|
||||
// must still get no bootcmd (would steal the host's WAN return path).
|
||||
func TestLBVMUserData_InternalSingleNodeFallback(t *testing.T) {
|
||||
mkSvc := func() *ELBv2ServiceImpl {
|
||||
svc := setupTestService(t)
|
||||
svc.GatewayURL = "https://192.168.1.33:9999"
|
||||
svc.SystemAccessKey = "AK"
|
||||
svc.SystemSecretKey = "SK"
|
||||
svc.MgmtBridgeIP = "10.15.8.1"
|
||||
svc.AdvertiseIP = "192.168.1.33"
|
||||
return svc
|
||||
}
|
||||
|
||||
internal, err := mkSvc().lbVMUserData("lb-int", SchemeInternal)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, internal, "bootcmd:")
|
||||
assert.Contains(t, internal, `"192.168.1.33/32"`)
|
||||
assert.Contains(t, internal, `"10.15.8.1"`)
|
||||
|
||||
inetFacing, err := mkSvc().lbVMUserData("lb-inet", SchemeInternetFacing)
|
||||
require.NoError(t, err)
|
||||
assert.NotContains(t, inetFacing, "bootcmd:", "internet-facing single-node must not get a /32 mgmt route")
|
||||
|
||||
noBridge := mkSvc()
|
||||
noBridge.MgmtBridgeIP = ""
|
||||
out, err := noBridge.lbVMUserData("lb-int", SchemeInternal)
|
||||
require.NoError(t, err)
|
||||
assert.NotContains(t, out, "bootcmd:", "no fallback when br-mgmt is absent")
|
||||
}
|
||||
|
||||
func TestIsCompatibleProtocol_UnknownListenerProtocol(t *testing.T) {
|
||||
assert.False(t, isCompatibleProtocol("SCTP", ProtocolTCP))
|
||||
assert.False(t, isCompatibleProtocol("", ProtocolHTTP))
|
||||
|
||||
@@ -1,13 +1,11 @@
|
||||
package handlers_elbv2
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
"github.com/aws/aws-sdk-go/service/ec2"
|
||||
"github.com/aws/aws-sdk-go/service/elbv2"
|
||||
"github.com/mulgadc/spinifex/spinifex/awserrors"
|
||||
"github.com/mulgadc/spinifex/spinifex/config"
|
||||
handlers_ec2_vpc "github.com/mulgadc/spinifex/spinifex/handlers/ec2/vpc"
|
||||
"github.com/mulgadc/spinifex/spinifex/testutil"
|
||||
@@ -160,7 +158,6 @@ func TestCreateLoadBalancer_MultiSubnet_AllENIsPassedToLauncher(t *testing.T) {
|
||||
},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-alb-test", nil })
|
||||
svc.GatewayURL = "https://10.0.0.1:9999"
|
||||
svc.SystemAccessKey = "AKID"
|
||||
svc.SystemSecretKey = "SECRET"
|
||||
@@ -281,7 +278,6 @@ func TestCreateLoadBalancer_InternetFacing_AllocatesPublicIP(t *testing.T) {
|
||||
},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-alb-test", nil })
|
||||
svc.GatewayURL = "https://10.0.0.1:9999"
|
||||
svc.SystemAccessKey = "AKID"
|
||||
svc.SystemSecretKey = "SECRET"
|
||||
@@ -330,7 +326,6 @@ func TestCreateLoadBalancer_Internal_NoPublicIP(t *testing.T) {
|
||||
},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-alb-test", nil })
|
||||
svc.GatewayURL = "https://10.0.0.1:9999"
|
||||
svc.SystemAccessKey = "AKID"
|
||||
svc.SystemSecretKey = "SECRET"
|
||||
@@ -371,7 +366,6 @@ func TestCreateLoadBalancer_NLB_Internal_NoPublicIP(t *testing.T) {
|
||||
},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-nlb-test", nil })
|
||||
svc.GatewayURL = "https://10.0.0.1:9999"
|
||||
svc.SystemAccessKey = "AKID"
|
||||
svc.SystemSecretKey = "SECRET"
|
||||
@@ -420,7 +414,6 @@ func TestDeleteLoadBalancer_TerminatesVM_WithPublicIP(t *testing.T) {
|
||||
terminateDone: make(chan struct{}),
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-alb-test", nil })
|
||||
svc.GatewayURL = "https://10.0.0.1:9999"
|
||||
svc.SystemAccessKey = "AKID"
|
||||
svc.SystemSecretKey = "SECRET"
|
||||
@@ -468,7 +461,6 @@ func TestDescribeLoadBalancers_InternetFacing_IncludesPublicIP(t *testing.T) {
|
||||
},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-alb-test", nil })
|
||||
svc.GatewayURL = "https://10.0.0.1:9999"
|
||||
svc.SystemAccessKey = "AKID"
|
||||
svc.SystemSecretKey = "SECRET"
|
||||
@@ -518,7 +510,6 @@ func TestDescribeLoadBalancers_Internal_NoPublicIP(t *testing.T) {
|
||||
},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-alb-test", nil })
|
||||
svc.GatewayURL = "https://10.0.0.1:9999"
|
||||
svc.SystemAccessKey = "AKID"
|
||||
svc.SystemSecretKey = "SECRET"
|
||||
@@ -557,7 +548,6 @@ func TestCreateLoadBalancer_LaunchFailure_SetsStateFailed(t *testing.T) {
|
||||
launchErr: assert.AnError,
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-alb-test", nil })
|
||||
svc.GatewayURL = "https://10.0.0.1:9999"
|
||||
svc.SystemAccessKey = "AKID"
|
||||
svc.SystemSecretKey = "SECRET"
|
||||
@@ -586,7 +576,6 @@ func TestCreateLoadBalancer_MissingCredentials_SetsStateFailed(t *testing.T) {
|
||||
},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) { return "ami-alb-test", nil })
|
||||
// Deliberately NOT setting credentials
|
||||
|
||||
out, err := svc.CreateLoadBalancer(&elbv2.CreateLoadBalancerInput{
|
||||
@@ -643,43 +632,6 @@ func TestENI_RequesterManagedFlag(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestCreateLoadBalancer_AMIResolverError_ReturnsServerInternal verifies that
|
||||
// when the launch path would fire (launcher + networking present) but the LB
|
||||
// system AMI cannot be resolved, CreateLoadBalancer returns ServerInternal and
|
||||
// does not persist a record or invoke the launcher.
|
||||
func TestCreateLoadBalancer_AMIResolverError_ReturnsServerInternal(t *testing.T) {
|
||||
svc, vpcSvc := setupTestServiceWithVPC(t)
|
||||
|
||||
subnets, err := vpcSvc.DescribeSubnets(&ec2.DescribeSubnetsInput{}, testAccountID)
|
||||
require.NoError(t, err)
|
||||
subnetID := *subnets.Subnets[0].SubnetId
|
||||
|
||||
mock := &mockSystemInstanceLauncher{
|
||||
launchResult: &SystemInstanceOutput{InstanceID: "i-should-not-launch"},
|
||||
}
|
||||
svc.InstanceLauncher = mock
|
||||
svc.SetSystemAMIFunc(func() (string, error) {
|
||||
return "", errors.New("LB system image not imported")
|
||||
})
|
||||
svc.GatewayURL = "https://10.0.0.1:9999"
|
||||
svc.SystemAccessKey = "AKID"
|
||||
svc.SystemSecretKey = "SECRET"
|
||||
|
||||
out, err := svc.CreateLoadBalancer(&elbv2.CreateLoadBalancerInput{
|
||||
Name: aws.String("ami-err-alb"),
|
||||
Subnets: []*string{aws.String(subnetID)},
|
||||
}, testAccountID)
|
||||
|
||||
require.Error(t, err)
|
||||
assert.Equal(t, awserrors.ErrorServerInternal, err.Error())
|
||||
assert.Nil(t, out)
|
||||
assert.Empty(t, mock.launchCalls, "launcher must not be invoked when AMI resolver errors")
|
||||
|
||||
lbs, err := svc.store.ListLoadBalancers()
|
||||
require.NoError(t, err)
|
||||
assert.Empty(t, lbs, "no LB record should be persisted when AMI resolver errors")
|
||||
}
|
||||
|
||||
// TestCreateLoadBalancer_AttachesSpecifiedSecurityGroupsToENI verifies that
|
||||
// when the caller supplies SecurityGroups on the LB input, the underlying ALB
|
||||
// ENI is created with those SGs (not the VPC default SG). Without this the
|
||||
|
||||
@@ -1 +1 @@
|
||||
import{n as e,t}from"./compiler-runtime.js";import{t as n}from"./useSuspenseQuery.js";import{t as r}from"./back-link.js";import{n as i}from"./ec22.js";import{t as a}from"./index.js";import{t as o}from"./page-heading.js";import{t as s}from"./state-badge.js";import{r as c}from"./system-managed.js";import{t as l}from"./ami-details.js";var u=t(),d=e();function f(){let e=(0,u.c)(14),{id:t}=a.useParams(),f;e[0]===t?f=e[1]:(f=i(t),e[0]=t,e[1]=f);let{data:p}=n(f),m=p?.Images?.[0];if(!m?.ImageId||c(m)){let t;return e[2]===Symbol.for(`react.memo_cache_sentinel`)?(t=(0,d.jsxs)(d.Fragment,{children:[(0,d.jsx)(r,{to:`/ec2/describe-images`,children:`Back to images`}),(0,d.jsx)(`p`,{className:`text-muted-foreground`,children:`Image not found.`})]}),e[2]=t):t=e[2],t}let h;e[3]===Symbol.for(`react.memo_cache_sentinel`)?(h=(0,d.jsx)(r,{to:`/ec2/describe-images`,children:`Back to images`}),e[3]=h):h=e[3];let g;e[4]===m.State?g=e[5]:(g=(0,d.jsx)(s,{state:m.State}),e[4]=m.State,e[5]=g);let _;e[6]!==m.ImageId||e[7]!==g?(_=(0,d.jsx)(o,{actions:g,subtitle:`EC2 Image Details`,title:m.ImageId}),e[6]=m.ImageId,e[7]=g,e[8]=_):_=e[8];let v;e[9]===m?v=e[10]:(v=(0,d.jsx)(l,{image:m,showExtendedDetails:!0}),e[9]=m,e[10]=v);let y;return e[11]!==_||e[12]!==v?(y=(0,d.jsxs)(d.Fragment,{children:[h,(0,d.jsxs)(`div`,{className:`space-y-6`,children:[_,v]})]}),e[11]=_,e[12]=v,e[13]=y):y=e[13],y}export{f as component};
|
||||
import{n as e,t}from"./compiler-runtime.js";import{t as n}from"./useSuspenseQuery.js";import{t as r}from"./back-link.js";import{n as i}from"./ec22.js";import{t as a}from"./index.js";import{t as o}from"./page-heading.js";import{t as s}from"./state-badge.js";import{t as c}from"./system-managed.js";import{t as l}from"./ami-details.js";var u=t(),d=e();function f(){let e=(0,u.c)(14),{id:t}=a.useParams(),f;e[0]===t?f=e[1]:(f=i(t),e[0]=t,e[1]=f);let{data:p}=n(f),m=p?.Images?.[0];if(!m?.ImageId||c(m)){let t;return e[2]===Symbol.for(`react.memo_cache_sentinel`)?(t=(0,d.jsxs)(d.Fragment,{children:[(0,d.jsx)(r,{to:`/ec2/describe-images`,children:`Back to images`}),(0,d.jsx)(`p`,{className:`text-muted-foreground`,children:`Image not found.`})]}),e[2]=t):t=e[2],t}let h;e[3]===Symbol.for(`react.memo_cache_sentinel`)?(h=(0,d.jsx)(r,{to:`/ec2/describe-images`,children:`Back to images`}),e[3]=h):h=e[3];let g;e[4]===m.State?g=e[5]:(g=(0,d.jsx)(s,{state:m.State}),e[4]=m.State,e[5]=g);let _;e[6]!==m.ImageId||e[7]!==g?(_=(0,d.jsx)(o,{actions:g,subtitle:`EC2 Image Details`,title:m.ImageId}),e[6]=m.ImageId,e[7]=g,e[8]=_):_=e[8];let v;e[9]===m?v=e[10]:(v=(0,d.jsx)(l,{image:m,showExtendedDetails:!0}),e[9]=m,e[10]=v);let y;return e[11]!==_||e[12]!==v?(y=(0,d.jsxs)(d.Fragment,{children:[h,(0,d.jsxs)(`div`,{className:`space-y-6`,children:[_,v]})]}),e[11]=_,e[12]=v,e[13]=y):y=e[13],y}export{f as component};
|
||||
File diff suppressed because one or more lines are too long
+2
-2
@@ -1,3 +1,3 @@
|
||||
import{n as e,t}from"./compiler-runtime.js";import{t as n}from"./error-banner.js";import{t as r}from"./useNavigate.js";import{t as i}from"./back-link.js";import{a as ee,i as a,n as o,r as s,t as c}from"./select.js";import{t as l}from"./cli-command-panel.js";import{c as u}from"./ec2.js";import{L as d}from"./index.js";import{t as f}from"./page-heading.js";import{a as p,r as m,t as h}from"./field.js";import{n as g,r as _,t as v}from"./zod.js";import{i as y}from"./ec23.js";import{t as b}from"./form-actions.js";var te=t(),x=e();function S(){let e=(0,te.c)(52),t=r(),S=u(),C;e[0]===Symbol.for(`react.memo_cache_sentinel`)?(C={resolver:v(y),defaultValues:{groupName:``,strategy:`spread`}},e[0]=C):C=e[0];let{control:w,handleSubmit:T,register:E,watch:D,formState:O}=_(C),{errors:k,isSubmitting:A}=O,j;e[1]!==S||e[2]!==t?(j=async e=>{t({to:`/ec2/describe-placement-groups/$id`,params:{id:(await S.mutateAsync(e)).PlacementGroup?.GroupId??``}})},e[1]=S,e[2]=t,e[3]=j):j=e[3];let M=j,N,P;e[4]===Symbol.for(`react.memo_cache_sentinel`)?(N=(0,x.jsx)(i,{to:`/ec2/describe-placement-groups`,children:`Back to placement groups`}),P=(0,x.jsx)(f,{title:`Create Placement Group`}),e[4]=N,e[5]=P):(N=e[4],P=e[5]);let F;e[6]===S.error?F=e[7]:(F=S.error&&(0,x.jsx)(n,{error:S.error,msg:`Failed to create placement group`}),e[6]=S.error,e[7]=F);let I;e[8]!==T||e[9]!==M?(I=T(M),e[8]=T,e[9]=M,e[10]=I):I=e[10];let L;e[11]===Symbol.for(`react.memo_cache_sentinel`)?(L=(0,x.jsx)(p,{children:(0,x.jsx)(`label`,{htmlFor:`groupName`,children:`Group Name`})}),e[11]=L):L=e[11];let R=!!k.groupName,z;e[12]===E?z=e[13]:(z=E(`groupName`),e[12]=E,e[13]=z);let B;e[14]!==R||e[15]!==z?(B=(0,x.jsx)(d,{"aria-invalid":R,id:`groupName`,placeholder:`my-placement-group`,...z}),e[14]=R,e[15]=z,e[16]=B):B=e[16];let V;e[17]===k.groupName?V=e[18]:(V=(0,x.jsx)(m,{errors:[k.groupName]}),e[17]=k.groupName,e[18]=V);let H;e[19]!==B||e[20]!==V?(H=(0,x.jsxs)(h,{children:[L,B,V]}),e[19]=B,e[20]=V,e[21]=H):H=e[21];let U;e[22]===Symbol.for(`react.memo_cache_sentinel`)?(U=(0,x.jsx)(p,{children:(0,x.jsx)(`label`,{htmlFor:`strategy`,children:`Strategy`})}),e[22]=U):U=e[22];let W;e[23]===k.strategy?W=e[24]:(W=e=>{let{field:t}=e;return(0,x.jsxs)(c,{onValueChange:e=>t.onChange(e),value:t.value??``,children:[(0,x.jsx)(a,{"aria-invalid":!!k.strategy,className:`w-full`,id:`strategy`,children:(0,x.jsx)(ee,{})}),(0,x.jsxs)(o,{children:[(0,x.jsx)(s,{value:`spread`,children:`spread`}),(0,x.jsx)(s,{value:`cluster`,children:`cluster`})]})]})},e[23]=k.strategy,e[24]=W);let G;e[25]!==w||e[26]!==W?(G=(0,x.jsx)(g,{control:w,name:`strategy`,render:W}),e[25]=w,e[26]=W,e[27]=G):G=e[27];let K;e[28]===k.strategy?K=e[29]:(K=(0,x.jsx)(m,{errors:[k.strategy]}),e[28]=k.strategy,e[29]=K);let q;e[30]!==G||e[31]!==K?(q=(0,x.jsxs)(h,{children:[U,G,K]}),e[30]=G,e[31]=K,e[32]=q):q=e[32];let J;e[33]===D?J=e[34]:(J=ne(D),e[33]=D,e[34]=J);let Y;e[35]===J?Y=e[36]:(Y=(0,x.jsx)(l,{commands:J}),e[35]=J,e[36]=Y);let X;e[37]===t?X=e[38]:(X=async()=>await t({to:`/ec2/describe-placement-groups`}),e[37]=t,e[38]=X);let Z;e[39]!==S.isPending||e[40]!==A||e[41]!==X?(Z=(0,x.jsx)(b,{isPending:S.isPending,isSubmitting:A,onCancel:X,pendingLabel:`Creating…`,submitLabel:`Create Placement Group`}),e[39]=S.isPending,e[40]=A,e[41]=X,e[42]=Z):Z=e[42];let Q;e[43]!==H||e[44]!==q||e[45]!==Y||e[46]!==Z||e[47]!==I?(Q=(0,x.jsxs)(`form`,{className:`max-w-4xl space-y-6`,onSubmit:I,children:[H,q,Y,Z]}),e[43]=H,e[44]=q,e[45]=Y,e[46]=Z,e[47]=I,e[48]=Q):Q=e[48];let $;return e[49]!==Q||e[50]!==F?($=(0,x.jsxs)(x.Fragment,{children:[N,P,F,Q]}),e[49]=Q,e[50]=F,e[51]=$):$=e[51],$}function ne(e){let t=e(`groupName`),n=typeof t==`string`?t:``,r=e(`strategy`),i=typeof r==`string`?r:``;return[{label:`Create Placement Group`,parts:[{type:`bin`,value:`AWS_PROFILE=spinifex aws ec2 create-placement-group`},{type:`flag`,value:` \\
|
||||
--group-name`},{type:`value`,value:` ${n||`<GroupName>`}`},{type:`flag`,value:` \\
|
||||
import{n as e,t}from"./compiler-runtime.js";import{t as n}from"./error-banner.js";import{t as r}from"./useNavigate.js";import{t as i}from"./back-link.js";import{a as ee,i as a,n as o,r as s,t as c}from"./select.js";import{t as l}from"./cli-command-panel.js";import{c as u}from"./ec2.js";import{L as d}from"./index.js";import{t as f}from"./page-heading.js";import{a as p,r as m,t as h}from"./field.js";import{n as g,r as _,t as v}from"./zod.js";import{i as y}from"./ec23.js";import{t as b}from"./form-actions.js";var te=t(),x=e();function S(){let e=(0,te.c)(52),t=r(),S=u(),C;e[0]===Symbol.for(`react.memo_cache_sentinel`)?(C={resolver:v(y),defaultValues:{groupName:``,strategy:`spread`}},e[0]=C):C=e[0];let{control:w,handleSubmit:T,register:E,watch:D,formState:O}=_(C),{errors:k,isSubmitting:A}=O,j;e[1]!==S||e[2]!==t?(j=async e=>{t({to:`/ec2/describe-placement-groups/$id`,params:{id:(await S.mutateAsync(e)).PlacementGroup?.GroupId??``}})},e[1]=S,e[2]=t,e[3]=j):j=e[3];let M=j,N,P;e[4]===Symbol.for(`react.memo_cache_sentinel`)?(N=(0,x.jsx)(i,{to:`/ec2/describe-placement-groups`,children:`Back to placement groups`}),P=(0,x.jsx)(f,{title:`Create Placement Group`}),e[4]=N,e[5]=P):(N=e[4],P=e[5]);let F;e[6]===S.error?F=e[7]:(F=S.error&&(0,x.jsx)(n,{error:S.error,msg:`Failed to create placement group`}),e[6]=S.error,e[7]=F);let I;e[8]!==T||e[9]!==M?(I=T(M),e[8]=T,e[9]=M,e[10]=I):I=e[10];let L;e[11]===Symbol.for(`react.memo_cache_sentinel`)?(L=(0,x.jsx)(p,{children:(0,x.jsx)(`label`,{htmlFor:`groupName`,children:`Group Name`})}),e[11]=L):L=e[11];let R=!!k.groupName,z;e[12]===E?z=e[13]:(z=E(`groupName`),e[12]=E,e[13]=z);let B;e[14]!==R||e[15]!==z?(B=(0,x.jsx)(d,{"aria-invalid":R,id:`groupName`,placeholder:`my-placement-group`,...z}),e[14]=R,e[15]=z,e[16]=B):B=e[16];let V;e[17]===k.groupName?V=e[18]:(V=(0,x.jsx)(m,{errors:[k.groupName]}),e[17]=k.groupName,e[18]=V);let H;e[19]!==B||e[20]!==V?(H=(0,x.jsxs)(h,{children:[L,B,V]}),e[19]=B,e[20]=V,e[21]=H):H=e[21];let U;e[22]===Symbol.for(`react.memo_cache_sentinel`)?(U=(0,x.jsx)(p,{children:(0,x.jsx)(`label`,{htmlFor:`strategy`,children:`Strategy`})}),e[22]=U):U=e[22];let W;e[23]===k.strategy?W=e[24]:(W=e=>{let{field:t}=e;return(0,x.jsxs)(c,{onValueChange:e=>t.onChange(e),value:t.value??``,children:[(0,x.jsx)(a,{"aria-invalid":!!k.strategy,className:`w-full`,id:`strategy`,children:(0,x.jsx)(ee,{})}),(0,x.jsxs)(o,{children:[(0,x.jsx)(s,{value:`spread`,children:`spread`}),(0,x.jsx)(s,{value:`cluster`,children:`cluster`})]})]})},e[23]=k.strategy,e[24]=W);let G;e[25]!==w||e[26]!==W?(G=(0,x.jsx)(g,{control:w,name:`strategy`,render:W}),e[25]=w,e[26]=W,e[27]=G):G=e[27];let K;e[28]===k.strategy?K=e[29]:(K=(0,x.jsx)(m,{errors:[k.strategy]}),e[28]=k.strategy,e[29]=K);let q;e[30]!==G||e[31]!==K?(q=(0,x.jsxs)(h,{children:[U,G,K]}),e[30]=G,e[31]=K,e[32]=q):q=e[32];let J;e[33]===D?J=e[34]:(J=ne(D),e[33]=D,e[34]=J);let Y;e[35]===J?Y=e[36]:(Y=(0,x.jsx)(l,{commands:J}),e[35]=J,e[36]=Y);let X;e[37]===t?X=e[38]:(X=async()=>await t({to:`/ec2/describe-placement-groups`}),e[37]=t,e[38]=X);let Z;e[39]!==S.isPending||e[40]!==A||e[41]!==X?(Z=(0,x.jsx)(b,{isPending:S.isPending,isSubmitting:A,onCancel:X,pendingLabel:`Creating…`,submitLabel:`Create Placement Group`}),e[39]=S.isPending,e[40]=A,e[41]=X,e[42]=Z):Z=e[42];let Q;e[43]!==H||e[44]!==q||e[45]!==Y||e[46]!==Z||e[47]!==I?(Q=(0,x.jsxs)(`form`,{className:`max-w-4xl space-y-6`,onSubmit:I,children:[H,q,Y,Z]}),e[43]=H,e[44]=q,e[45]=Y,e[46]=Z,e[47]=I,e[48]=Q):Q=e[48];let $;return e[49]!==Q||e[50]!==F?($=(0,x.jsxs)(x.Fragment,{children:[N,P,F,Q]}),e[49]=Q,e[50]=F,e[51]=$):$=e[51],$}function C(e){return`'${e.replaceAll(`'`,`'"'"'`)}'`}function ne(e){let t=e(`groupName`),n=typeof t==`string`?t:``,r=e(`strategy`),i=typeof r==`string`?r:``;return[{label:`Create Placement Group`,parts:[{type:`bin`,value:`AWS_PROFILE=spinifex aws ec2 create-placement-group`},{type:`flag`,value:` \\
|
||||
--group-name`},{type:`value`,value:` ${n?C(n):`<GroupName>`}`},{type:`flag`,value:` \\
|
||||
--strategy`},{type:`value`,value:` ${i||`spread`}`}]}]}export{S as component};
|
||||
@@ -1,14 +1,14 @@
|
||||
import{n as e}from"./compiler-runtime.js";import{t}from"./error-banner.js";import{t as n}from"./useSuspenseQuery.js";import{t as r}from"./useNavigate.js";import{t as i}from"./back-link.js";import{t as a}from"./cli-command-panel.js";import{x as o}from"./ec22.js";import{r as s}from"./elbv2.js";import{t as c}from"./page-heading.js";import{r as l,t as u}from"./zod.js";import{t as d}from"./form-actions.js";import{t as f}from"./target-group-form.js";import{r as p}from"./elbv23.js";var m=e();function h(){let e=r(),{data:h}=n(o),_=s(),v=h.Vpcs??[],y=l({resolver:u(p),defaultValues:{name:``,protocol:`HTTP`,port:80,vpcId:v[0]?.VpcId??``,healthCheck:{protocol:`HTTP`,path:`/`,port:`traffic-port`,intervalSeconds:30,timeoutSeconds:5,healthyThresholdCount:5,unhealthyThresholdCount:2,matcher:`200`},tags:[]}}),{handleSubmit:b,watch:x,formState:{isSubmitting:S}}=y;return(0,m.jsxs)(m.Fragment,{children:[(0,m.jsx)(i,{to:`/ec2/describe-target-groups`,children:`Back to target groups`}),(0,m.jsx)(c,{title:`Create target group`}),_.error&&(0,m.jsx)(t,{error:_.error,msg:`Failed to create target group`}),(0,m.jsxs)(`form`,{className:`max-w-4xl space-y-6`,onSubmit:b(async t=>{try{let n=(await _.mutateAsync(t)).TargetGroups?.[0]?.TargetGroupArn;e(n?{to:`/ec2/describe-target-groups/$id`,params:{id:encodeURIComponent(n)}}:{to:`/ec2/describe-target-groups`})}catch{}}),children:[(0,m.jsx)(f,{form:y,vpcs:v}),(0,m.jsx)(a,{commands:g(x)}),(0,m.jsx)(d,{isPending:_.isPending,isSubmitting:S,onCancel:async()=>await e({to:`/ec2/describe-target-groups`}),pendingLabel:`Creating…`,submitLabel:`Create target group`})]})]})}function g(e){let t=t=>{let n=e(t);return typeof n==`string`?n:``},n=t=>{let n=e(t);return typeof n==`number`&&Number.isFinite(n)?n:void 0},r=t(`name`),i=t(`protocol`)||`HTTP`,a=n(`port`),o=t(`vpcId`),s=t(`healthCheck.path`)||`/`,c=t(`healthCheck.port`)||`traffic-port`,l=n(`healthCheck.intervalSeconds`),u=n(`healthCheck.timeoutSeconds`),d=n(`healthCheck.healthyThresholdCount`),f=n(`healthCheck.unhealthyThresholdCount`),p=t(`healthCheck.matcher`)||`200`,m=[{type:`bin`,value:`AWS_PROFILE=spinifex aws elbv2 create-target-group`},{type:`flag`,value:` \\
|
||||
import{n as e}from"./compiler-runtime.js";import{t}from"./error-banner.js";import{t as n}from"./useSuspenseQuery.js";import{t as r}from"./useNavigate.js";import{t as i}from"./back-link.js";import{t as a}from"./cli-command-panel.js";import{x as o}from"./ec22.js";import{r as s}from"./elbv2.js";import{t as c}from"./page-heading.js";import{r as l,t as u}from"./zod.js";import{t as d}from"./form-actions.js";import{t as f}from"./target-group-form.js";import{r as p}from"./elbv23.js";var m=e();function h(){let e=r(),{data:h}=n(o),_=s(),v=h.Vpcs??[],y=l({resolver:u(p),defaultValues:{name:``,protocol:`HTTP`,port:80,vpcId:v[0]?.VpcId??``,healthCheck:{protocol:`HTTP`,path:`/`,port:`traffic-port`,intervalSeconds:30,timeoutSeconds:5,healthyThresholdCount:5,unhealthyThresholdCount:2,matcher:`200`},tags:[]}}),{handleSubmit:b,watch:x,formState:{isSubmitting:S}}=y;return(0,m.jsxs)(m.Fragment,{children:[(0,m.jsx)(i,{to:`/ec2/describe-target-groups`,children:`Back to target groups`}),(0,m.jsx)(c,{title:`Create target group`}),_.error&&(0,m.jsx)(t,{error:_.error,msg:`Failed to create target group`}),(0,m.jsxs)(`form`,{className:`max-w-4xl space-y-6`,onSubmit:b(async t=>{try{let n=(await _.mutateAsync(t)).TargetGroups?.[0]?.TargetGroupArn;e(n?{to:`/ec2/describe-target-groups/$id`,params:{id:encodeURIComponent(n)}}:{to:`/ec2/describe-target-groups`})}catch{}}),children:[(0,m.jsx)(f,{form:y,vpcs:v}),(0,m.jsx)(a,{commands:g(x)}),(0,m.jsx)(d,{isPending:_.isPending,isSubmitting:S,onCancel:async()=>await e({to:`/ec2/describe-target-groups`}),pendingLabel:`Creating…`,submitLabel:`Create target group`})]})]})}function g(e){let t=t=>{let n=e(t);return typeof n==`string`?n:``},n=t=>{let n=e(t);return typeof n==`number`&&Number.isFinite(n)?n:void 0},r=t(`name`),i=t(`protocol`)||`HTTP`,a=n(`port`),o=t(`vpcId`),s=t(`healthCheck.protocol`)||`HTTP`,c=t(`healthCheck.path`)||`/`,l=t(`healthCheck.port`)||`traffic-port`,u=n(`healthCheck.intervalSeconds`),d=n(`healthCheck.timeoutSeconds`),f=n(`healthCheck.healthyThresholdCount`),p=n(`healthCheck.unhealthyThresholdCount`),m=t(`healthCheck.matcher`)||`200`,h=[{type:`bin`,value:`AWS_PROFILE=spinifex aws elbv2 create-target-group`},{type:`flag`,value:` \\
|
||||
--name`},{type:`value`,value:` ${r||`<Name>`}`},{type:`flag`,value:` \\
|
||||
--protocol`},{type:`value`,value:` ${i}`},{type:`flag`,value:` \\
|
||||
--port`},{type:`value`,value:` ${a??80}`},{type:`flag`,value:` \\
|
||||
--target-type`},{type:`value`,value:` instance`}];return o&&m.push({type:`flag`,value:` \\
|
||||
--vpc-id`},{type:`value`,value:` ${o}`}),m.push({type:`flag`,value:` \\
|
||||
--health-check-protocol`},{type:`value`,value:` HTTP`},{type:`flag`,value:` \\
|
||||
--health-check-path`},{type:`value`,value:` ${s}`},{type:`flag`,value:` \\
|
||||
--health-check-port`},{type:`value`,value:` ${c}`},{type:`flag`,value:` \\
|
||||
--health-check-interval-seconds`},{type:`value`,value:` ${l??30}`},{type:`flag`,value:` \\
|
||||
--health-check-timeout-seconds`},{type:`value`,value:` ${u??5}`},{type:`flag`,value:` \\
|
||||
--healthy-threshold-count`},{type:`value`,value:` ${d??5}`},{type:`flag`,value:` \\
|
||||
--unhealthy-threshold-count`},{type:`value`,value:` ${f??2}`},{type:`flag`,value:` \\
|
||||
--matcher`},{type:`value`,value:` HttpCode=${p}`}),[{label:`Create Target Group`,parts:m}]}var _=h;export{_ as component};
|
||||
--target-type`},{type:`value`,value:` instance`}];return o&&h.push({type:`flag`,value:` \\
|
||||
--vpc-id`},{type:`value`,value:` ${o}`}),h.push({type:`flag`,value:` \\
|
||||
--health-check-protocol`},{type:`value`,value:` ${s}`},{type:`flag`,value:` \\
|
||||
--health-check-path`},{type:`value`,value:` ${c}`},{type:`flag`,value:` \\
|
||||
--health-check-port`},{type:`value`,value:` ${l}`},{type:`flag`,value:` \\
|
||||
--health-check-interval-seconds`},{type:`value`,value:` ${u??30}`},{type:`flag`,value:` \\
|
||||
--health-check-timeout-seconds`},{type:`value`,value:` ${d??5}`},{type:`flag`,value:` \\
|
||||
--healthy-threshold-count`},{type:`value`,value:` ${f??5}`},{type:`flag`,value:` \\
|
||||
--unhealthy-threshold-count`},{type:`value`,value:` ${p??2}`},{type:`flag`,value:` \\
|
||||
--matcher`},{type:`value`,value:` HttpCode=${m}`}),[{label:`Create Target Group`,parts:h}]}var _=h;export{_ as component};
|
||||
@@ -1 +1 @@
|
||||
import{n as e,t}from"./compiler-runtime.js";import{t as n}from"./useSuspenseQuery.js";import{r}from"./ec22.js";import{t as i}from"./page-heading.js";import{t as a}from"./state-badge.js";import{r as o}from"./system-managed.js";import{t as s}from"./list-card.js";var c=t(),l=e();function u(){let e=(0,c.c)(3),{data:t}=n(r),a;if(e[0]!==t.Images){let n=(t.Images??[]).filter(f),r;e[2]===Symbol.for(`react.memo_cache_sentinel`)?(r=(0,l.jsx)(i,{title:`EC2 Images`}),e[2]=r):r=e[2],a=(0,l.jsxs)(l.Fragment,{children:[r,n.length>0?(0,l.jsx)(`div`,{className:`space-y-4`,children:n.map(d)}):(0,l.jsx)(`p`,{className:`text-muted-foreground`,children:`No images found.`})]}),e[0]=t.Images,e[1]=a}else a=e[1];return a}function d(e){return e.ImageId?(0,l.jsx)(s,{badge:(0,l.jsx)(a,{state:e.State}),params:{id:e.ImageId},subtitle:e.ImageId??``,title:e.Name??e.ImageId??``,to:`/ec2/describe-images/$id`},e.ImageId):null}function f(e){return!o(e)}export{u as component};
|
||||
import{n as e,t}from"./compiler-runtime.js";import{t as n}from"./useSuspenseQuery.js";import{r}from"./ec22.js";import{t as i}from"./page-heading.js";import{t as a}from"./state-badge.js";import{t as o}from"./system-managed.js";import{t as s}from"./list-card.js";var c=t(),l=e();function u(){let e=(0,c.c)(3),{data:t}=n(r),a;if(e[0]!==t.Images){let n=(t.Images??[]).filter(f),r;e[2]===Symbol.for(`react.memo_cache_sentinel`)?(r=(0,l.jsx)(i,{title:`EC2 Images`}),e[2]=r):r=e[2],a=(0,l.jsxs)(l.Fragment,{children:[r,n.length>0?(0,l.jsx)(`div`,{className:`space-y-4`,children:n.map(d)}):(0,l.jsx)(`p`,{className:`text-muted-foreground`,children:`No images found.`})]}),e[0]=t.Images,e[1]=a}else a=e[1];return a}function d(e){return e.ImageId?(0,l.jsx)(s,{badge:(0,l.jsx)(a,{state:e.State}),params:{id:e.ImageId},subtitle:e.ImageId??``,title:e.Name??e.ImageId??``,to:`/ec2/describe-images/$id`},e.ImageId):null}function f(e){return!o(e)}export{u as component};
|
||||
+1
-1
@@ -1 +1 @@
|
||||
import{n as e,t}from"./compiler-runtime.js";import{t as n}from"./useSuspenseQuery.js";import{t as r}from"./link.js";import{t as i}from"./useNavigate.js";import{t as a}from"./button.js";import{r as o}from"./ec22.js";import{i as s}from"./elbv22.js";import{t as c}from"./page-heading.js";import{t as l}from"./state-badge.js";import{n as u}from"./system-managed.js";import{t as d}from"./lb-image-missing-banner.js";var f=t(),p=e();function m(e){return e?new Date(e).toLocaleString():``}function h(){let e=(0,f.c)(20),t=i(),{data:h}=n(s),{data:_}=n(o),v;e[0]===h.LoadBalancers?v=e[1]:(v=h.LoadBalancers??[],e[0]=h.LoadBalancers,e[1]=v);let y=v,b;e[2]===_.Images?b=e[3]:(b=_.Images??[],e[2]=_.Images,e[3]=b);let x;e[4]===b?x=e[5]:(x=u(b),e[4]=b,e[5]=x);let S=x,C=!S,w;e[6]===t?w=e[7]:(w=async()=>await t({to:`/ec2/create-load-balancer`}),e[6]=t,e[7]=w);let T;e[8]!==C||e[9]!==w?(T=(0,p.jsx)(c,{actions:(0,p.jsx)(a,{disabled:C,onClick:w,children:`Create Load Balancer`}),title:`Load Balancers`}),e[8]=C,e[9]=w,e[10]=T):T=e[10];let E;e[11]===S?E=e[12]:(E=!S&&(0,p.jsx)(d,{}),e[11]=S,e[12]=E);let D;e[13]!==y||e[14]!==t?(D=y.length>0?(0,p.jsx)(`div`,{className:`overflow-x-auto rounded-lg border bg-card`,children:(0,p.jsxs)(`table`,{className:`w-full text-sm`,children:[(0,p.jsx)(`thead`,{children:(0,p.jsxs)(`tr`,{className:`border-b text-left text-muted-foreground`,children:[(0,p.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`Name`}),(0,p.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`DNS Name`}),(0,p.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`State`}),(0,p.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`Type`}),(0,p.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`Scheme`}),(0,p.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`VPC`}),(0,p.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`Created`})]})}),(0,p.jsx)(`tbody`,{children:y.map(e=>{if(!e.LoadBalancerArn)return null;let n=e.LoadBalancerArn;return(0,p.jsxs)(`tr`,{className:`cursor-pointer border-b transition-colors last:border-0 hover:bg-accent`,onClick:async()=>await t({to:`/ec2/describe-load-balancers/$id`,params:{id:encodeURIComponent(n)}}),children:[(0,p.jsx)(`td`,{className:`px-4 py-2 font-medium`,children:(0,p.jsx)(r,{className:`text-primary hover:underline`,onClick:g,params:{id:encodeURIComponent(n)},to:`/ec2/describe-load-balancers/$id`,children:e.LoadBalancerName??n})}),(0,p.jsx)(`td`,{className:`px-4 py-2 font-mono text-xs`,children:e.DNSName??``}),(0,p.jsx)(`td`,{className:`px-4 py-2`,children:(0,p.jsx)(l,{state:e.State?.Code})}),(0,p.jsx)(`td`,{className:`px-4 py-2`,children:e.Type??``}),(0,p.jsx)(`td`,{className:`px-4 py-2`,children:e.Scheme??``}),(0,p.jsx)(`td`,{className:`px-4 py-2 font-mono text-xs`,children:e.VpcId??``}),(0,p.jsx)(`td`,{className:`px-4 py-2 text-xs`,children:m(e.CreatedTime)})]},n)})})]})}):(0,p.jsx)(`p`,{className:`text-muted-foreground`,children:`No load balancers found.`}),e[13]=y,e[14]=t,e[15]=D):D=e[15];let O;return e[16]!==T||e[17]!==E||e[18]!==D?(O=(0,p.jsxs)(p.Fragment,{children:[T,E,D]}),e[16]=T,e[17]=E,e[18]=D,e[19]=O):O=e[19],O}function g(e){return e.stopPropagation()}var _=h;export{_ as component};
|
||||
import{n as e,t}from"./compiler-runtime.js";import{t as n}from"./useSuspenseQuery.js";import{t as r}from"./link.js";import{t as i}from"./useNavigate.js";import{t as a}from"./button.js";import{i as o}from"./elbv22.js";import{t as s}from"./page-heading.js";import{t as c}from"./state-badge.js";var l=t(),u=e();function d(e){return e?new Date(e).toLocaleString():``}function f(){let e=(0,l.c)(10),t=i(),{data:f}=n(o),m;e[0]===f.LoadBalancers?m=e[1]:(m=f.LoadBalancers??[],e[0]=f.LoadBalancers,e[1]=m);let h=m,g;e[2]===t?g=e[3]:(g=(0,u.jsx)(s,{actions:(0,u.jsx)(a,{onClick:async()=>await t({to:`/ec2/create-load-balancer`}),children:`Create Load Balancer`}),title:`Load Balancers`}),e[2]=t,e[3]=g);let _;e[4]!==h||e[5]!==t?(_=h.length>0?(0,u.jsx)(`div`,{className:`overflow-x-auto rounded-lg border bg-card`,children:(0,u.jsxs)(`table`,{className:`w-full text-sm`,children:[(0,u.jsx)(`thead`,{children:(0,u.jsxs)(`tr`,{className:`border-b text-left text-muted-foreground`,children:[(0,u.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`Name`}),(0,u.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`DNS Name`}),(0,u.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`State`}),(0,u.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`Type`}),(0,u.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`Scheme`}),(0,u.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`VPC`}),(0,u.jsx)(`th`,{className:`px-4 py-2 font-medium`,children:`Created`})]})}),(0,u.jsx)(`tbody`,{children:h.map(e=>{if(!e.LoadBalancerArn)return null;let n=e.LoadBalancerArn;return(0,u.jsxs)(`tr`,{className:`cursor-pointer border-b transition-colors last:border-0 hover:bg-accent`,onClick:async()=>await t({to:`/ec2/describe-load-balancers/$id`,params:{id:encodeURIComponent(n)}}),children:[(0,u.jsx)(`td`,{className:`px-4 py-2 font-medium`,children:(0,u.jsx)(r,{className:`text-primary hover:underline`,onClick:p,params:{id:encodeURIComponent(n)},to:`/ec2/describe-load-balancers/$id`,children:e.LoadBalancerName??n})}),(0,u.jsx)(`td`,{className:`px-4 py-2 font-mono text-xs`,children:e.DNSName??``}),(0,u.jsx)(`td`,{className:`px-4 py-2`,children:(0,u.jsx)(c,{state:e.State?.Code})}),(0,u.jsx)(`td`,{className:`px-4 py-2`,children:e.Type??``}),(0,u.jsx)(`td`,{className:`px-4 py-2`,children:e.Scheme??``}),(0,u.jsx)(`td`,{className:`px-4 py-2 font-mono text-xs`,children:e.VpcId??``}),(0,u.jsx)(`td`,{className:`px-4 py-2 text-xs`,children:d(e.CreatedTime)})]},n)})})]})}):(0,u.jsx)(`p`,{className:`text-muted-foreground`,children:`No load balancers found.`}),e[4]=h,e[5]=t,e[6]=_):_=e[6];let v;return e[7]!==g||e[8]!==_?(v=(0,u.jsxs)(u.Fragment,{children:[g,_]}),e[7]=g,e[8]=_,e[9]=v):v=e[9],v}function p(e){return e.stopPropagation()}var m=f;export{m as component};
|
||||
@@ -1 +1 @@
|
||||
import{D as e,E as t,O as n,T as r,j as i,k as a,w as o}from"./SignatureV4.js";import{F as s,I as c,P as l}from"./ec2.js";var u=/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\/\d{1,2}$/,d=i().min(1,`Key name is required`).max(255,`Key name must be 255 characters or less`).regex(/^[\w\s._\-:/()#,@[\]+=&;{}!$*]+$/,`Key name contains invalid characters`),f=[`gp3`],p=a({imageId:i(`Please select an Image`),instanceType:i(`Please select an instance type`),keyName:i(`Please select a key pair`).min(1,`Key pair is required`),subnetId:i().optional(),placementGroupName:i().optional(),count:e(`Instance count must be a whole number`).min(1,`Instance count must be at least 1`),rootDeviceName:i().optional(),rootVolumeSize:n().int(`Volume size must be a whole number`).min(1,`Volume size must be at least 1 GiB`).max(16384,`Volume size must be at most 16384 GiB`).optional(),rootVolumeType:o(f).optional(),rootDeleteOnTermination:t().optional()}),m=a({keyName:d}),h=a({keyName:d,publicKeyMaterial:i().min(1,`Public key is required`).refine(e=>e.trim().length>0,`Public key cannot be empty`)}),g=a({size:n().int(`Size must be a whole number`).min(1,`Size must be at least 1 GiB`).max(16384,`Size must be at most 16384 GiB`),availabilityZone:i().min(1,`Availability zone is required`)}),_=a({size:n().int(`Size must be a whole number`).min(1,`Size must be at least 1 GiB`)}),v=a({volumeId:i().min(1,`Volume is required`),description:i().optional()});a({sourceSnapshotId:i().min(1,`Source snapshot is required`),sourceRegion:i().min(1,`Source region is required`),description:i().optional()}),a({volumeId:i().min(1,`Volume is required`),instanceId:i().min(1,`Instance is required`),device:i().optional()}),a({volumeId:i().min(1,`Volume is required`),instanceId:i().optional(),force:t().optional()}),a({instanceType:i().min(1,`Instance type is required`)}),a({name:i().min(1,`Name is required`),description:i().optional()});var y=a({vpcId:i().min(1,`VPC is required`),cidrBlock:i().min(1,`CIDR block is required`).regex(u,`Must be a valid CIDR block (e.g. 10.0.1.0/24)`),availabilityZone:i().optional()});a({cidrBlock:i().min(1,`CIDR block is required`).regex(u,`Must be a valid CIDR block (e.g. 10.0.0.0/16)`),name:i().optional()});var b=a({key:i().min(1,`Key is required`),value:i()}),x=a({mode:o([`vpc-only`,`vpc-and-more`]),namePrefix:i().optional(),autoGenerateNames:t(),cidrBlock:i().min(1,`CIDR block is required`).regex(u,`Must be a valid CIDR block (e.g. 10.0.0.0/16)`).refine(e=>c(e),`CIDR has invalid octets or prefix length (must be /16 to /28)`),tenancy:o([`default`,`dedicated`]),publicSubnetCount:n().int().min(0).max(1),privateSubnetCount:n().int().min(0).max(2),publicSubnetCidrs:r(i()),privateSubnetCidrs:r(i()),tags:r(b)}).superRefine((e,t)=>{if(e.mode!==`vpc-and-more`)return;let n=[];for(let[r,i]of e.publicSubnetCidrs.entries())c(i,16,28)?(l(e.cidrBlock,i)||t.addIssue({code:`custom`,message:`Subnet CIDR must be within the VPC CIDR range`,path:[`publicSubnetCidrs`,r]}),n.push({cidr:i,field:`publicSubnetCidrs`,index:r})):t.addIssue({code:`custom`,message:`Invalid subnet CIDR format or prefix length`,path:[`publicSubnetCidrs`,r]});for(let[r,i]of e.privateSubnetCidrs.entries())c(i,16,28)?(l(e.cidrBlock,i)||t.addIssue({code:`custom`,message:`Subnet CIDR must be within the VPC CIDR range`,path:[`privateSubnetCidrs`,r]}),n.push({cidr:i,field:`privateSubnetCidrs`,index:r})):t.addIssue({code:`custom`,message:`Invalid subnet CIDR format or prefix length`,path:[`privateSubnetCidrs`,r]});for(let e=0;e<n.length;e++)for(let r=e+1;r<n.length;r++){let i=n[e],a=n[r];i&&a&&s(i.cidr,a.cidr)&&t.addIssue({code:`custom`,message:`Subnet CIDRs must not overlap`,path:[a.field,a.index]})}}),S=a({groupName:i().min(1,`Group name is required`).max(255,`Group name must be 255 characters or less`),strategy:i().min(1,`Strategy is required`)}),C=a({groupName:i().min(1,`Group name is required`).max(255,`Group name must be 255 characters or less`),description:i().min(1,`Description is required`).max(255,`Description must be 255 characters or less`),vpcId:i().min(1,`VPC is required`)}),w=a({ipProtocol:i().min(1,`Protocol is required`),fromPort:n().int().min(-1).max(65535),toPort:n().int().min(-1).max(65535),cidrIp:i().min(1,`CIDR is required`).regex(u,`Must be a valid CIDR block (e.g. 0.0.0.0/0)`)});export{C as a,g as c,_ as d,w as f,S as i,x as l,p as n,v as o,m as r,y as s,f as t,h as u};
|
||||
import{D as e,E as t,O as n,T as r,j as i,k as a,w as o}from"./SignatureV4.js";import{F as s,I as c,P as l}from"./ec2.js";var u=/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\/\d{1,2}$/,d=i().min(1,`Key name is required`).max(255,`Key name must be 255 characters or less`).regex(/^[\w\s._\-:/()#,@[\]+=&;{}!$*]+$/,`Key name contains invalid characters`),f=[`gp3`],p=a({imageId:i(`Please select an Image`),instanceType:i(`Please select an instance type`),keyName:i(`Please select a key pair`).min(1,`Key pair is required`),subnetId:i().optional(),placementGroupName:i().optional(),count:e(`Instance count must be a whole number`).min(1,`Instance count must be at least 1`),rootDeviceName:i().optional(),rootVolumeSize:n().int(`Volume size must be a whole number`).min(1,`Volume size must be at least 1 GiB`).max(16384,`Volume size must be at most 16384 GiB`).optional(),rootVolumeType:o(f).optional(),rootDeleteOnTermination:t().optional()}),m=a({keyName:d}),h=a({keyName:d,publicKeyMaterial:i().min(1,`Public key is required`).refine(e=>e.trim().length>0,`Public key cannot be empty`)}),g=a({size:n().int(`Size must be a whole number`).min(1,`Size must be at least 1 GiB`).max(16384,`Size must be at most 16384 GiB`),availabilityZone:i().min(1,`Availability zone is required`)}),_=a({size:n().int(`Size must be a whole number`).min(1,`Size must be at least 1 GiB`)}),v=a({volumeId:i().min(1,`Volume is required`),description:i().optional()});a({sourceSnapshotId:i().min(1,`Source snapshot is required`),sourceRegion:i().min(1,`Source region is required`),description:i().optional()}),a({volumeId:i().min(1,`Volume is required`),instanceId:i().min(1,`Instance is required`),device:i().optional()}),a({volumeId:i().min(1,`Volume is required`),instanceId:i().optional(),force:t().optional()}),a({instanceType:i().min(1,`Instance type is required`)}),a({name:i().min(1,`Name is required`),description:i().optional()});var y=a({vpcId:i().min(1,`VPC is required`),cidrBlock:i().min(1,`CIDR block is required`).regex(u,`Must be a valid CIDR block (e.g. 10.0.1.0/24)`),availabilityZone:i().optional()});a({cidrBlock:i().min(1,`CIDR block is required`).regex(u,`Must be a valid CIDR block (e.g. 10.0.0.0/16)`),name:i().optional()});var b=a({key:i().min(1,`Key is required`),value:i()}),x=a({mode:o([`vpc-only`,`vpc-and-more`]),namePrefix:i().optional(),autoGenerateNames:t(),cidrBlock:i().min(1,`CIDR block is required`).regex(u,`Must be a valid CIDR block (e.g. 10.0.0.0/16)`).refine(e=>c(e),`CIDR has invalid octets or prefix length (must be /16 to /28)`),tenancy:o([`default`,`dedicated`]),publicSubnetCount:n().int().min(0).max(1),privateSubnetCount:n().int().min(0).max(2),publicSubnetCidrs:r(i()),privateSubnetCidrs:r(i()),tags:r(b)}).superRefine((e,t)=>{if(e.mode!==`vpc-and-more`)return;let n=[];for(let[r,i]of e.publicSubnetCidrs.entries())c(i,16,28)?(l(e.cidrBlock,i)||t.addIssue({code:`custom`,message:`Subnet CIDR must be within the VPC CIDR range`,path:[`publicSubnetCidrs`,r]}),n.push({cidr:i,field:`publicSubnetCidrs`,index:r})):t.addIssue({code:`custom`,message:`Invalid subnet CIDR format or prefix length`,path:[`publicSubnetCidrs`,r]});for(let[r,i]of e.privateSubnetCidrs.entries())c(i,16,28)?(l(e.cidrBlock,i)||t.addIssue({code:`custom`,message:`Subnet CIDR must be within the VPC CIDR range`,path:[`privateSubnetCidrs`,r]}),n.push({cidr:i,field:`privateSubnetCidrs`,index:r})):t.addIssue({code:`custom`,message:`Invalid subnet CIDR format or prefix length`,path:[`privateSubnetCidrs`,r]});for(let e=0;e<n.length;e++)for(let r=e+1;r<n.length;r++){let i=n[e],a=n[r];i&&a&&s(i.cidr,a.cidr)&&t.addIssue({code:`custom`,message:`Subnet CIDRs must not overlap`,path:[a.field,a.index]})}}),S=a({groupName:i().min(1,`Group name is required`).max(255,`Group name must be 255 characters or less`).regex(/^[\w\s._\-:/()#,@[\]+=&;{}!$*]+$/,`Group name contains invalid characters`),strategy:i().min(1,`Strategy is required`)}),C=a({groupName:i().min(1,`Group name is required`).max(255,`Group name must be 255 characters or less`),description:i().min(1,`Description is required`).max(255,`Description must be 255 characters or less`),vpcId:i().min(1,`VPC is required`)}),w=a({ipProtocol:i().min(1,`Protocol is required`),fromPort:n().int().min(-1).max(65535),toPort:n().int().min(-1).max(65535),cidrIp:i().min(1,`CIDR is required`).regex(u,`Must be a valid CIDR block (e.g. 0.0.0.0/0)`)});export{C as a,g as c,_ as d,w as f,S as i,x as l,p as n,v as o,m as r,y as s,f as t,h as u};
|
||||
@@ -1 +1 @@
|
||||
import{j as e,k as t}from"./SignatureV4.js";var n=t({userName:e().min(1,`User name is required`).max(64,`User name must be 64 characters or less`).regex(/^[\w+=,.@-]+$/,`User name contains invalid characters`),path:e().optional()}),r=t({policyName:e().min(1,`Policy name is required`).max(128,`Policy name must be 128 characters or less`),description:e().optional(),policyDocument:e().min(1,`Policy document is required`).refine(e=>{try{return JSON.parse(e),!0}catch{return!1}},{message:`Policy document must be valid JSON`})});export{n,r as t};
|
||||
import{j as e,k as t}from"./SignatureV4.js";var n=t({userName:e().min(1,`User name is required`).max(64,`User name must be 64 characters or less`).regex(/^[\w+=,.@-]+$/,`User name contains invalid characters`),path:e().optional()}),r=t({policyName:e().min(1,`Policy name is required`).max(128,`Policy name must be 128 characters or less`).regex(/^[\w+=,.@-]+$/,`Policy name contains invalid characters`),description:e().optional(),policyDocument:e().min(1,`Policy document is required`).refine(e=>{try{return JSON.parse(e),!0}catch{return!1}},{message:`Policy document must be valid JSON`})});export{n,r as t};
|
||||
File diff suppressed because one or more lines are too long
@@ -1 +0,0 @@
|
||||
import{n as e,t}from"./compiler-runtime.js";import{t as n}from"./system-managed.js";var r=t(),i=e();function a(){let e=(0,r.c)(1),t;return e[0]===Symbol.for(`react.memo_cache_sentinel`)?(t=(0,i.jsxs)(`div`,{className:`mb-6 max-w-4xl rounded-md border border-amber-500/50 bg-amber-500/10 p-4`,children:[(0,i.jsx)(`h2`,{className:`text-sm font-semibold`,children:`Load balancer image not imported`}),(0,i.jsx)(`p`,{className:`mt-1 text-sm text-muted-foreground`,children:`The load balancer system image has not been imported on this cluster. A cluster administrator must run the following command before load balancers can be created:`}),(0,i.jsxs)(`pre`,{className:`mt-2 overflow-x-auto rounded bg-muted px-3 py-2 font-mono text-xs`,children:[`spx admin images import --name `,n]})]}),e[0]=t):t=e[0],t}export{a as t};
|
||||
@@ -1 +1 @@
|
||||
var e=`lb-alpine-3.21.6-x86_64`;function t(e){return e.Tags?.some(e=>e.Key===`spinifex:managed-by`)??!1}function n(e){return e.Tags?.some(e=>e.Key===`spinifex:managed-by`&&e.Value===`elbv2`)??!1}function r(e){return e.some(n)}export{r as n,t as r,e as t};
|
||||
function e(e){return e.Tags?.some(e=>e.Key===`spinifex:managed-by`)??!1}export{e as t};
|
||||
File diff suppressed because one or more lines are too long
-19
@@ -1,19 +0,0 @@
|
||||
import { LB_IMAGE_NAME } from "@/lib/system-managed"
|
||||
|
||||
export function LbImageMissingBanner() {
|
||||
return (
|
||||
<div className="mb-6 max-w-4xl rounded-md border border-amber-500/50 bg-amber-500/10 p-4">
|
||||
<h2 className="text-sm font-semibold">
|
||||
Load balancer image not imported
|
||||
</h2>
|
||||
<p className="mt-1 text-sm text-muted-foreground">
|
||||
The load balancer system image has not been imported on this cluster. A
|
||||
cluster administrator must run the following command before load
|
||||
balancers can be created:
|
||||
</p>
|
||||
<pre className="mt-2 overflow-x-auto rounded bg-muted px-3 py-2 font-mono text-xs">
|
||||
spx admin images import --name {LB_IMAGE_NAME}
|
||||
</pre>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,13 +1,7 @@
|
||||
import type { Image } from "@aws-sdk/client-ec2"
|
||||
import { describe, expect, it } from "vitest"
|
||||
|
||||
import {
|
||||
LB_MANAGED_BY_VALUE,
|
||||
SYSTEM_MANAGED_TAG_KEY,
|
||||
hasLbImage,
|
||||
isLbImage,
|
||||
isSystemManagedImage,
|
||||
} from "./system-managed"
|
||||
import { SYSTEM_MANAGED_TAG_KEY, isSystemManagedImage } from "./system-managed"
|
||||
|
||||
describe("isSystemManagedImage", () => {
|
||||
it("returns true when image carries the managed-by tag", () => {
|
||||
@@ -30,49 +24,3 @@ describe("isSystemManagedImage", () => {
|
||||
expect(isSystemManagedImage({ ImageId: "ami-3" })).toBeFalsy()
|
||||
})
|
||||
})
|
||||
|
||||
describe("isLbImage", () => {
|
||||
it("returns true when tag value matches elbv2", () => {
|
||||
const image: Image = {
|
||||
ImageId: "ami-lb",
|
||||
Tags: [{ Key: SYSTEM_MANAGED_TAG_KEY, Value: LB_MANAGED_BY_VALUE }],
|
||||
}
|
||||
expect(isLbImage(image)).toBeTruthy()
|
||||
})
|
||||
|
||||
it("returns false for system-managed images owned by other components", () => {
|
||||
const image: Image = {
|
||||
ImageId: "ami-other",
|
||||
Tags: [{ Key: SYSTEM_MANAGED_TAG_KEY, Value: "other-service" }],
|
||||
}
|
||||
expect(isLbImage(image)).toBeFalsy()
|
||||
})
|
||||
|
||||
it("returns false for customer images", () => {
|
||||
expect(isLbImage({ ImageId: "ami-cust" })).toBeFalsy()
|
||||
})
|
||||
})
|
||||
|
||||
describe("hasLbImage", () => {
|
||||
it("returns true when any image in the list is the LB AMI", () => {
|
||||
const images: Image[] = [
|
||||
{ ImageId: "ami-cust", Tags: [{ Key: "Name", Value: "ubuntu" }] },
|
||||
{
|
||||
ImageId: "ami-lb",
|
||||
Tags: [{ Key: SYSTEM_MANAGED_TAG_KEY, Value: LB_MANAGED_BY_VALUE }],
|
||||
},
|
||||
]
|
||||
expect(hasLbImage(images)).toBeTruthy()
|
||||
})
|
||||
|
||||
it("returns false for an empty list", () => {
|
||||
expect(hasLbImage([])).toBeFalsy()
|
||||
})
|
||||
|
||||
it("returns false when no image carries the elbv2 tag", () => {
|
||||
const images: Image[] = [
|
||||
{ ImageId: "ami-cust", Tags: [{ Key: "Name", Value: "ubuntu" }] },
|
||||
]
|
||||
expect(hasLbImage(images)).toBeFalsy()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,30 +1,10 @@
|
||||
import type { Image } from "@aws-sdk/client-ec2"
|
||||
|
||||
// Tag key applied by Spinifex to platform-managed resources
|
||||
// (HAProxy VMs, their ENIs, the LB AMI). The value identifies the
|
||||
// owning component (e.g. "elbv2"). See spinifex/docs/TAG-CONVENTIONS.md.
|
||||
// (HAProxy VMs, their ENIs). The value identifies the owning component
|
||||
// (e.g. "elbv2"). See spinifex/docs/TAG-CONVENTIONS.md.
|
||||
export const SYSTEM_MANAGED_TAG_KEY = "spinifex:managed-by"
|
||||
|
||||
// Value of SYSTEM_MANAGED_TAG_KEY carried by the LB system AMI and its VMs.
|
||||
export const LB_MANAGED_BY_VALUE = "elbv2"
|
||||
|
||||
// Canonical name of the LB system image in utils.AvailableImages.
|
||||
// Kept in sync with spinifex/utils/images.go.
|
||||
export const LB_IMAGE_NAME = "lb-alpine-3.21.6-x86_64"
|
||||
|
||||
export function isSystemManagedImage(image: Image): boolean {
|
||||
return image.Tags?.some((tag) => tag.Key === SYSTEM_MANAGED_TAG_KEY) ?? false
|
||||
}
|
||||
|
||||
export function isLbImage(image: Image): boolean {
|
||||
return (
|
||||
image.Tags?.some(
|
||||
(tag) =>
|
||||
tag.Key === SYSTEM_MANAGED_TAG_KEY && tag.Value === LB_MANAGED_BY_VALUE,
|
||||
) ?? false
|
||||
)
|
||||
}
|
||||
|
||||
export function hasLbImage(images: Image[]): boolean {
|
||||
return images.some(isLbImage)
|
||||
}
|
||||
|
||||
-17
@@ -12,7 +12,6 @@ import {
|
||||
type CliCommand,
|
||||
type CommandPart,
|
||||
} from "@/components/cli-command-panel"
|
||||
import { LbImageMissingBanner } from "@/components/elbv2/lb-image-missing-banner"
|
||||
import { TargetGroupForm } from "@/components/elbv2/target-group-form"
|
||||
import { ErrorBanner } from "@/components/error-banner"
|
||||
import { FormActions } from "@/components/form-actions"
|
||||
@@ -27,14 +26,12 @@ import {
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select"
|
||||
import { hasLbImage } from "@/lib/system-managed"
|
||||
import { getNameTag } from "@/lib/utils"
|
||||
import {
|
||||
type LbWizardResult,
|
||||
useCreateLoadBalancerWizard,
|
||||
} from "@/mutations/elbv2"
|
||||
import {
|
||||
ec2ImagesQueryOptions,
|
||||
ec2SecurityGroupsQueryOptions,
|
||||
ec2SubnetsQueryOptions,
|
||||
ec2VpcsQueryOptions,
|
||||
@@ -85,10 +82,8 @@ export function CreateLoadBalancerPage() {
|
||||
const { data: subnetsData } = useSuspenseQuery(ec2SubnetsQueryOptions)
|
||||
const { data: sgsData } = useSuspenseQuery(ec2SecurityGroupsQueryOptions)
|
||||
const { data: tgsData } = useSuspenseQuery(elbv2TargetGroupsQueryOptions)
|
||||
const { data: imagesData } = useSuspenseQuery(ec2ImagesQueryOptions)
|
||||
const wizardMutation = useCreateLoadBalancerWizard()
|
||||
|
||||
const lbImageImported = hasLbImage(imagesData.Images ?? [])
|
||||
const [wizardResult, setWizardResult] = useState<LbWizardResult | null>(null)
|
||||
|
||||
const vpcs = vpcsData.Vpcs ?? []
|
||||
@@ -223,18 +218,6 @@ export function CreateLoadBalancerPage() {
|
||||
|
||||
const groupedSubnets = groupSubnetsByAz(vpcSubnets)
|
||||
|
||||
if (!lbImageImported) {
|
||||
return (
|
||||
<>
|
||||
<BackLink to="/ec2/describe-load-balancers">
|
||||
Back to load balancers
|
||||
</BackLink>
|
||||
<PageHeading title="Create load balancer" />
|
||||
<LbImageMissingBanner />
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
<BackLink to="/ec2/describe-load-balancers">
|
||||
|
||||
-8
@@ -2,12 +2,9 @@ import type { LoadBalancer } from "@aws-sdk/client-elastic-load-balancing-v2"
|
||||
import { useSuspenseQuery } from "@tanstack/react-query"
|
||||
import { Link, useNavigate } from "@tanstack/react-router"
|
||||
|
||||
import { LbImageMissingBanner } from "@/components/elbv2/lb-image-missing-banner"
|
||||
import { PageHeading } from "@/components/page-heading"
|
||||
import { StateBadge } from "@/components/state-badge"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { hasLbImage } from "@/lib/system-managed"
|
||||
import { ec2ImagesQueryOptions } from "@/queries/ec2"
|
||||
import { elbv2LoadBalancersQueryOptions } from "@/queries/elbv2"
|
||||
|
||||
function formatCreatedAt(createdAt: Date | undefined): string {
|
||||
@@ -20,17 +17,14 @@ function formatCreatedAt(createdAt: Date | undefined): string {
|
||||
export function DescribeLoadBalancersPage() {
|
||||
const navigate = useNavigate()
|
||||
const { data } = useSuspenseQuery(elbv2LoadBalancersQueryOptions)
|
||||
const { data: imagesData } = useSuspenseQuery(ec2ImagesQueryOptions)
|
||||
|
||||
const loadBalancers = data.LoadBalancers ?? []
|
||||
const lbImageImported = hasLbImage(imagesData.Images ?? [])
|
||||
|
||||
return (
|
||||
<>
|
||||
<PageHeading
|
||||
actions={
|
||||
<Button
|
||||
disabled={!lbImageImported}
|
||||
onClick={async () =>
|
||||
await navigate({ to: "/ec2/create-load-balancer" })
|
||||
}
|
||||
@@ -41,8 +35,6 @@ export function DescribeLoadBalancersPage() {
|
||||
title="Load Balancers"
|
||||
/>
|
||||
|
||||
{!lbImageImported && <LbImageMissingBanner />}
|
||||
|
||||
{loadBalancers.length > 0 ? (
|
||||
<div className="overflow-x-auto rounded-lg border bg-card">
|
||||
<table className="w-full text-sm">
|
||||
|
||||
+2
-30
@@ -1,4 +1,4 @@
|
||||
import type { Image, SecurityGroup, Subnet, Vpc } from "@aws-sdk/client-ec2"
|
||||
import type { SecurityGroup, Subnet, Vpc } from "@aws-sdk/client-ec2"
|
||||
import type { TargetGroup } from "@aws-sdk/client-elastic-load-balancing-v2"
|
||||
import { screen, waitFor } from "@testing-library/react"
|
||||
import userEvent from "@testing-library/user-event"
|
||||
@@ -90,22 +90,12 @@ const EXISTING_TG: TargetGroup = {
|
||||
Port: 80,
|
||||
VpcId: VPC_ID,
|
||||
}
|
||||
const LB_IMAGE: Image = {
|
||||
ImageId: "ami-lb",
|
||||
Name: "lb-alpine-3.21.6-x86_64",
|
||||
Tags: [{ Key: "spinifex:managed-by", Value: "elbv2" }],
|
||||
}
|
||||
|
||||
function seedClient(options: { lbImageImported?: boolean } = {}) {
|
||||
const { lbImageImported = true } = options
|
||||
function seedClient() {
|
||||
const qc = createTestQueryClient()
|
||||
qc.setQueryData(["ec2", "vpcs"], { Vpcs: VPCS })
|
||||
qc.setQueryData(["ec2", "subnets"], { Subnets: SUBNETS })
|
||||
qc.setQueryData(["ec2", "securityGroups"], { SecurityGroups: SGS })
|
||||
qc.setQueryData(["elbv2", "targetGroups"], { TargetGroups: [EXISTING_TG] })
|
||||
qc.setQueryData(["ec2", "images"], {
|
||||
Images: lbImageImported ? [LB_IMAGE] : [],
|
||||
})
|
||||
return qc
|
||||
}
|
||||
|
||||
@@ -268,22 +258,4 @@ describe("create-load-balancer route", () => {
|
||||
expect(screen.getByText(/arn:tg:new/)).toBeInTheDocument()
|
||||
expect(routerState.navigate).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it("blocks form render with import banner when LB AMI missing", () => {
|
||||
renderWithClient(
|
||||
<CreateLoadBalancerPage />,
|
||||
seedClient({ lbImageImported: false }),
|
||||
)
|
||||
|
||||
expect(
|
||||
screen.getByText("Load balancer image not imported"),
|
||||
).toBeInTheDocument()
|
||||
expect(
|
||||
screen.getByText(/spx admin images import --name lb-alpine/),
|
||||
).toBeInTheDocument()
|
||||
// Form inputs must not render
|
||||
expect(
|
||||
screen.queryByLabelText("Name", { selector: "#lb-name" }),
|
||||
).not.toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
-49
@@ -1,4 +1,3 @@
|
||||
import type { Image } from "@aws-sdk/client-ec2"
|
||||
import type { LoadBalancer } from "@aws-sdk/client-elastic-load-balancing-v2"
|
||||
import { screen } from "@testing-library/react"
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"
|
||||
@@ -79,12 +78,6 @@ const LBS: LoadBalancer[] = [
|
||||
},
|
||||
]
|
||||
|
||||
const LB_IMAGE: Image = {
|
||||
ImageId: "ami-lb",
|
||||
Name: "lb-alpine-3.21.6-x86_64",
|
||||
Tags: [{ Key: "spinifex:managed-by", Value: "elbv2" }],
|
||||
}
|
||||
|
||||
describe("describe-load-balancers list route", () => {
|
||||
beforeEach(() => sdk.reset())
|
||||
afterEach(() => vi.clearAllMocks())
|
||||
@@ -92,7 +85,6 @@ describe("describe-load-balancers list route", () => {
|
||||
it("renders load-balancer rows with resolved fields", () => {
|
||||
const qc = createTestQueryClient()
|
||||
qc.setQueryData(["elbv2", "loadBalancers"], { LoadBalancers: LBS })
|
||||
qc.setQueryData(["ec2", "images"], { Images: [LB_IMAGE] })
|
||||
|
||||
renderWithClient(<DescribeLoadBalancersPage />, qc)
|
||||
|
||||
@@ -100,9 +92,6 @@ describe("describe-load-balancers list route", () => {
|
||||
expect(screen.getByText("lb-one.example")).toBeInTheDocument()
|
||||
expect(screen.getByText("application")).toBeInTheDocument()
|
||||
expect(screen.getByText("internet-facing")).toBeInTheDocument()
|
||||
expect(
|
||||
screen.queryByText("Load balancer image not imported"),
|
||||
).not.toBeInTheDocument()
|
||||
expect(
|
||||
screen.getByRole("button", { name: "Create Load Balancer" }),
|
||||
).not.toBeDisabled()
|
||||
@@ -111,47 +100,9 @@ describe("describe-load-balancers list route", () => {
|
||||
it("shows empty state when no load balancers", () => {
|
||||
const qc = createTestQueryClient()
|
||||
qc.setQueryData(["elbv2", "loadBalancers"], { LoadBalancers: [] })
|
||||
qc.setQueryData(["ec2", "images"], { Images: [LB_IMAGE] })
|
||||
|
||||
renderWithClient(<DescribeLoadBalancersPage />, qc)
|
||||
|
||||
expect(screen.getByText("No load balancers found.")).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it("shows import banner and disables Create button when LB AMI missing", () => {
|
||||
const qc = createTestQueryClient()
|
||||
qc.setQueryData(["elbv2", "loadBalancers"], { LoadBalancers: [] })
|
||||
qc.setQueryData(["ec2", "images"], { Images: [] })
|
||||
|
||||
renderWithClient(<DescribeLoadBalancersPage />, qc)
|
||||
|
||||
expect(
|
||||
screen.getByText("Load balancer image not imported"),
|
||||
).toBeInTheDocument()
|
||||
expect(
|
||||
screen.getByText(/spx admin images import --name lb-alpine/),
|
||||
).toBeInTheDocument()
|
||||
expect(
|
||||
screen.getByRole("button", { name: "Create Load Balancer" }),
|
||||
).toBeDisabled()
|
||||
})
|
||||
|
||||
it("treats non-LB system images as missing", () => {
|
||||
const qc = createTestQueryClient()
|
||||
qc.setQueryData(["elbv2", "loadBalancers"], { LoadBalancers: [] })
|
||||
qc.setQueryData(["ec2", "images"], {
|
||||
Images: [
|
||||
{
|
||||
ImageId: "ami-customer",
|
||||
Tags: [{ Key: "Name", Value: "ubuntu" }],
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
renderWithClient(<DescribeLoadBalancersPage />, qc)
|
||||
|
||||
expect(
|
||||
screen.getByText("Load balancer image not imported"),
|
||||
).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
package vm
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// IsMMIO reports whether machineType is a memory-mapped I/O bus machine (e.g.
|
||||
// microvm). MMIO machines use virtio-*-device transports instead of
|
||||
// virtio-*-pci, because they have no PCI bus.
|
||||
func IsMMIO(machineType string) bool {
|
||||
return strings.HasPrefix(machineType, "microvm")
|
||||
}
|
||||
|
||||
// NetDevice returns the appropriate QEMU virtio-net device string for
|
||||
// machineType, wiring netdev and mac. mac is omitted when empty.
|
||||
func NetDevice(machineType, netdev, mac string) Device {
|
||||
var transport string
|
||||
if IsMMIO(machineType) {
|
||||
transport = "virtio-net-device"
|
||||
} else {
|
||||
transport = "virtio-net-pci"
|
||||
}
|
||||
if mac != "" {
|
||||
return Device{Value: fmt.Sprintf("%s,netdev=%s,mac=%s", transport, netdev, mac)}
|
||||
}
|
||||
return Device{Value: fmt.Sprintf("%s,netdev=%s", transport, netdev)}
|
||||
}
|
||||
|
||||
// BlkDevice returns the appropriate QEMU virtio-blk device string for
|
||||
// machineType. iothread and queues are omitted when empty/zero. bootIdx is
|
||||
// only emitted for PCI machines (MMIO has no bootindex concept).
|
||||
func BlkDevice(machineType, drive, iothread string, queues int, bootIdx int) Device {
|
||||
var b strings.Builder
|
||||
if IsMMIO(machineType) {
|
||||
b.WriteString("virtio-blk-device")
|
||||
} else {
|
||||
b.WriteString("virtio-blk-pci")
|
||||
}
|
||||
fmt.Fprintf(&b, ",drive=%s", drive)
|
||||
if iothread != "" {
|
||||
fmt.Fprintf(&b, ",iothread=%s", iothread)
|
||||
}
|
||||
if queues > 0 {
|
||||
fmt.Fprintf(&b, ",num-queues=%d", queues)
|
||||
}
|
||||
if !IsMMIO(machineType) && bootIdx > 0 {
|
||||
fmt.Fprintf(&b, ",bootindex=%d", bootIdx)
|
||||
}
|
||||
return Device{Value: b.String()}
|
||||
}
|
||||
|
||||
// RngDevice returns the appropriate QEMU virtio-rng device string for
|
||||
// machineType.
|
||||
func RngDevice(machineType string) Device {
|
||||
if IsMMIO(machineType) {
|
||||
return Device{Value: "virtio-rng-device"}
|
||||
}
|
||||
return Device{Value: "virtio-rng-pci"}
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
package vm
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestIsMMIO(t *testing.T) {
|
||||
tests := []struct {
|
||||
machineType string
|
||||
want bool
|
||||
}{
|
||||
{"microvm", true},
|
||||
{"microvm,x-option-roms=off", true},
|
||||
{"q35", false},
|
||||
{"virt", false},
|
||||
{"", false},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.machineType, func(t *testing.T) {
|
||||
assert.Equal(t, tt.want, IsMMIO(tt.machineType))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestNetDevice_PCI(t *testing.T) {
|
||||
d := NetDevice("q35", "net0", "02:00:00:aa:bb:cc")
|
||||
assert.Equal(t, "virtio-net-pci,netdev=net0,mac=02:00:00:aa:bb:cc", d.Value)
|
||||
}
|
||||
|
||||
func TestNetDevice_PCI_NoMAC(t *testing.T) {
|
||||
d := NetDevice("q35", "net0", "")
|
||||
assert.Equal(t, "virtio-net-pci,netdev=net0", d.Value)
|
||||
}
|
||||
|
||||
func TestNetDevice_MMIO(t *testing.T) {
|
||||
d := NetDevice("microvm", "net0", "02:00:00:aa:bb:cc")
|
||||
assert.Equal(t, "virtio-net-device,netdev=net0,mac=02:00:00:aa:bb:cc", d.Value)
|
||||
}
|
||||
|
||||
func TestNetDevice_MMIO_NoMAC(t *testing.T) {
|
||||
d := NetDevice("microvm,x-option-roms=off", "net0", "")
|
||||
assert.Equal(t, "virtio-net-device,netdev=net0", d.Value)
|
||||
}
|
||||
|
||||
func TestBlkDevice_PCI(t *testing.T) {
|
||||
d := BlkDevice("q35", "os", "ioth-os", 4, 1)
|
||||
assert.Equal(t, "virtio-blk-pci,drive=os,iothread=ioth-os,num-queues=4,bootindex=1", d.Value)
|
||||
}
|
||||
|
||||
func TestBlkDevice_PCI_NoBootIdx(t *testing.T) {
|
||||
d := BlkDevice("q35", "data", "ioth-data", 2, 0)
|
||||
assert.Equal(t, "virtio-blk-pci,drive=data,iothread=ioth-data,num-queues=2", d.Value)
|
||||
}
|
||||
|
||||
func TestBlkDevice_PCI_NoIOThread(t *testing.T) {
|
||||
d := BlkDevice("q35", "data", "", 0, 0)
|
||||
assert.Equal(t, "virtio-blk-pci,drive=data", d.Value)
|
||||
}
|
||||
|
||||
func TestBlkDevice_MMIO(t *testing.T) {
|
||||
d := BlkDevice("microvm", "os", "ioth-os", 4, 1)
|
||||
// MMIO has no bootindex
|
||||
assert.Equal(t, "virtio-blk-device,drive=os,iothread=ioth-os,num-queues=4", d.Value)
|
||||
}
|
||||
|
||||
func TestBlkDevice_MMIO_NoIOThread(t *testing.T) {
|
||||
d := BlkDevice("microvm,x-option-roms=off", "data", "", 0, 0)
|
||||
assert.Equal(t, "virtio-blk-device,drive=data", d.Value)
|
||||
}
|
||||
|
||||
func TestRngDevice_PCI(t *testing.T) {
|
||||
d := RngDevice("q35")
|
||||
assert.Equal(t, "virtio-rng-pci", d.Value)
|
||||
}
|
||||
|
||||
func TestRngDevice_MMIO(t *testing.T) {
|
||||
d := RngDevice("microvm")
|
||||
assert.Equal(t, "virtio-rng-device", d.Value)
|
||||
}
|
||||
|
||||
func TestRngDevice_MMIO_WithOptions(t *testing.T) {
|
||||
d := RngDevice("microvm,x-option-roms=off,pic=off")
|
||||
assert.Equal(t, "virtio-rng-device", d.Value)
|
||||
}
|
||||
+128
-74
@@ -10,6 +10,7 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
@@ -170,6 +171,28 @@ func (m *Manager) launch(instance *VM) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// nbdkitPreExecWait returns how long startQEMU sleeps before exec'ing QEMU to
|
||||
// give nbdkit time to bind its sockets. Direct-boot instances have no drives
|
||||
// and no nbdkit, so the wait collapses to zero and the boot-time budget is
|
||||
// preserved.
|
||||
func nbdkitPreExecWait(directBoot bool) time.Duration {
|
||||
if directBoot {
|
||||
return 0
|
||||
}
|
||||
return 2 * time.Second
|
||||
}
|
||||
|
||||
// qemuStartSettleWait returns the post-fork sleep that lets QEMU surface an
|
||||
// immediate-start crash (bad cmdline, missing kernel) before we attempt QMP.
|
||||
// Direct-boot binds the QMP socket within a few ms because there is no
|
||||
// firmware, ROM, or block layer setup, so a full second is unnecessary.
|
||||
func qemuStartSettleWait(directBoot bool) time.Duration {
|
||||
if directBoot {
|
||||
return 50 * time.Millisecond
|
||||
}
|
||||
return time.Second
|
||||
}
|
||||
|
||||
// startQEMU launches the QEMU process for instance and waits for startup to
|
||||
// confirm.
|
||||
func (m *Manager) startQEMU(instance *VM) error {
|
||||
@@ -191,78 +214,111 @@ func (m *Manager) startQEMU(instance *VM) error {
|
||||
consoleLogPath := filepath.Join(runtimeDir, fmt.Sprintf("console-%s.log", instance.ID))
|
||||
serialSocket := filepath.Join(runtimeDir, fmt.Sprintf("serial-%s.sock", instance.ID))
|
||||
|
||||
instance.Config = buildBaseVMConfig(instance.ID, pidFile, consoleLogPath, serialSocket, spec.Architecture, spec.VCPUs, spec.MemoryMiB)
|
||||
if instance.DirectBoot {
|
||||
// Direct-boot (microvm) path: vm.Config was pre-built by the launcher.
|
||||
// Only fill in the runtime-generated paths that are not known until now.
|
||||
// SerialSocket is intentionally omitted: microvm uses a file chardev
|
||||
// so kernel output is captured without a socket client.
|
||||
instance.Config.PIDFile = pidFile
|
||||
instance.Config.ConsoleLogPath = consoleLogPath
|
||||
} else {
|
||||
instance.Config = buildBaseVMConfig(instance.ID, pidFile, consoleLogPath, serialSocket, spec.Architecture, spec.VCPUs, spec.MemoryMiB)
|
||||
|
||||
instance.EBSRequests.Mu.Lock()
|
||||
drives, iothreads, devices, err := buildDrives(instance.EBSRequests.Requests, spec.VCPUs)
|
||||
instance.EBSRequests.Mu.Unlock()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
instance.Config.Drives = append(instance.Config.Drives, drives...)
|
||||
instance.Config.IOThreads = append(instance.Config.IOThreads, iothreads...)
|
||||
instance.Config.Devices = append(instance.Config.Devices, devices...)
|
||||
|
||||
if instance.ENIId != "" && m.deps.NetworkPlumber != nil {
|
||||
spec := VPCTapSpec(instance.ENIId, instance.ENIMac)
|
||||
if err := m.deps.NetworkPlumber.SetupTap(spec); err != nil {
|
||||
slog.Error("Failed to set up tap device", "eni", instance.ENIId, "err", err)
|
||||
return fmt.Errorf("setup tap device: %w", err)
|
||||
}
|
||||
tapName := spec.Name
|
||||
|
||||
instance.Config.NetDevs = append(instance.Config.NetDevs, NetDev{
|
||||
Value: fmt.Sprintf("tap,id=net0,ifname=%s,script=no,downscript=no", tapName),
|
||||
})
|
||||
instance.Config.Devices = append(instance.Config.Devices, Device{
|
||||
Value: fmt.Sprintf("virtio-net-pci,netdev=net0,mac=%s", instance.ENIMac),
|
||||
})
|
||||
slog.Info("VPC networking configured", "tap", tapName, "eni", instance.ENIId, "mac", instance.ENIMac)
|
||||
|
||||
if err := m.setupExtraENINICs(instance); err != nil {
|
||||
instance.EBSRequests.Mu.Lock()
|
||||
drives, iothreads, devices, err := buildDrives(instance.EBSRequests.Requests, spec.VCPUs, instance.Config.MachineType)
|
||||
instance.EBSRequests.Mu.Unlock()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
instance.Config.Drives = append(instance.Config.Drives, drives...)
|
||||
instance.Config.IOThreads = append(instance.Config.IOThreads, iothreads...)
|
||||
instance.Config.Devices = append(instance.Config.Devices, devices...)
|
||||
}
|
||||
|
||||
if m.deps.DevNetworking {
|
||||
m.appendDevHostfwdNIC(instance)
|
||||
if instance.DirectBoot {
|
||||
// Direct-boot (microvm) path: NetDevs and Devices are already set in
|
||||
// the pre-built Config. Only create host-side tap devices for VPC ENIs
|
||||
// so the kernel tap interfaces exist before QEMU opens them.
|
||||
if instance.ENIId != "" && m.deps.NetworkPlumber != nil {
|
||||
tapSpec := VPCTapSpec(instance.ENIId, instance.ENIMac)
|
||||
if err := m.deps.NetworkPlumber.SetupTap(tapSpec); err != nil {
|
||||
slog.Error("Failed to set up tap device (direct-boot)", "eni", instance.ENIId, "err", err)
|
||||
return fmt.Errorf("setup tap device: %w", err)
|
||||
}
|
||||
slog.Info("VPC tap configured (direct-boot)", "tap", tapSpec.Name, "eni", instance.ENIId, "mac", instance.ENIMac)
|
||||
for _, extra := range instance.ExtraENIs {
|
||||
extraSpec := VPCTapSpec(extra.ENIID, extra.ENIMac)
|
||||
if err := m.deps.NetworkPlumber.SetupTap(extraSpec); err != nil {
|
||||
slog.Error("Failed to set up extra ENI tap (direct-boot)", "eni", extra.ENIID, "err", err)
|
||||
return fmt.Errorf("setup tap device for extra ENI %s: %w", extra.ENIID, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
if instance.MgmtMAC != "" && m.deps.NetworkPlumber != nil {
|
||||
mgmtTap := MgmtTapName(instance.ID)
|
||||
mgmtBridge := "br-mgmt"
|
||||
if err := m.deps.NetworkPlumber.SetupTap(TapSpec{Name: mgmtTap, Bridge: mgmtBridge}); err != nil {
|
||||
slog.Error("Failed to set up mgmt tap (direct-boot)", "tap", mgmtTap, "err", err)
|
||||
return fmt.Errorf("setup mgmt tap: %w", err)
|
||||
}
|
||||
slog.Info("Mgmt tap configured (direct-boot)", "tap", mgmtTap, "mac", instance.MgmtMAC, "ip", instance.MgmtIP, "instanceId", instance.ID)
|
||||
}
|
||||
} else {
|
||||
sshDebugAddr, err := viperblock.FindFreePort()
|
||||
if err != nil {
|
||||
slog.Error("Failed to find free port", "err", err)
|
||||
return err
|
||||
}
|
||||
_, sshDebugPort, err := net.SplitHostPort(sshDebugAddr)
|
||||
if err != nil {
|
||||
slog.Error("Failed to parse port from address", "addr", sshDebugAddr, "err", err)
|
||||
return fmt.Errorf("parse port from %s: %w", sshDebugAddr, err)
|
||||
if instance.ENIId != "" && m.deps.NetworkPlumber != nil {
|
||||
spec := VPCTapSpec(instance.ENIId, instance.ENIMac)
|
||||
if err := m.deps.NetworkPlumber.SetupTap(spec); err != nil {
|
||||
slog.Error("Failed to set up tap device", "eni", instance.ENIId, "err", err)
|
||||
return fmt.Errorf("setup tap device: %w", err)
|
||||
}
|
||||
tapName := spec.Name
|
||||
|
||||
instance.Config.NetDevs = append(instance.Config.NetDevs, NetDev{
|
||||
Value: fmt.Sprintf("tap,id=net0,ifname=%s,script=no,downscript=no", tapName),
|
||||
})
|
||||
instance.Config.Devices = append(instance.Config.Devices, NetDevice(instance.Config.MachineType, "net0", instance.ENIMac))
|
||||
slog.Info("VPC networking configured", "tap", tapName, "eni", instance.ENIId, "mac", instance.ENIMac)
|
||||
|
||||
if err := m.setupExtraENINICs(instance); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if m.deps.DevNetworking {
|
||||
m.appendDevHostfwdNIC(instance)
|
||||
}
|
||||
} else {
|
||||
sshDebugAddr, err := viperblock.FindFreePort()
|
||||
if err != nil {
|
||||
slog.Error("Failed to find free port", "err", err)
|
||||
return err
|
||||
}
|
||||
_, sshDebugPort, err := net.SplitHostPort(sshDebugAddr)
|
||||
if err != nil {
|
||||
slog.Error("Failed to parse port from address", "addr", sshDebugAddr, "err", err)
|
||||
return fmt.Errorf("parse port from %s: %w", sshDebugAddr, err)
|
||||
}
|
||||
|
||||
bindIP := m.deps.BindHost
|
||||
if bindIP == "" || bindIP == "0.0.0.0" {
|
||||
bindIP = "127.0.0.1"
|
||||
}
|
||||
instance.Config.NetDevs = append(instance.Config.NetDevs, NetDev{
|
||||
Value: fmt.Sprintf("user,id=net0,hostfwd=tcp:%s:%s-:22", bindIP, sshDebugPort),
|
||||
})
|
||||
instance.Config.Devices = append(instance.Config.Devices, NetDevice(instance.Config.MachineType, "net0", ""))
|
||||
}
|
||||
|
||||
bindIP := m.deps.BindHost
|
||||
if bindIP == "" || bindIP == "0.0.0.0" {
|
||||
bindIP = "127.0.0.1"
|
||||
if instance.MgmtMAC != "" {
|
||||
mgmtTap := MgmtTapName(instance.ID)
|
||||
instance.Config.NetDevs = append(instance.Config.NetDevs, NetDev{
|
||||
Value: fmt.Sprintf("tap,id=mgmt0,ifname=%s,script=no,downscript=no", mgmtTap),
|
||||
})
|
||||
instance.Config.Devices = append(instance.Config.Devices, NetDevice(instance.Config.MachineType, "mgmt0", instance.MgmtMAC))
|
||||
slog.Info("Management NIC configured", "tap", mgmtTap, "mac", instance.MgmtMAC, "ip", instance.MgmtIP, "instanceId", instance.ID)
|
||||
}
|
||||
instance.Config.NetDevs = append(instance.Config.NetDevs, NetDev{
|
||||
Value: fmt.Sprintf("user,id=net0,hostfwd=tcp:%s:%s-:22", bindIP, sshDebugPort),
|
||||
})
|
||||
instance.Config.Devices = append(instance.Config.Devices, Device{
|
||||
Value: "virtio-net-pci,netdev=net0",
|
||||
})
|
||||
|
||||
instance.Config.Devices = append(instance.Config.Devices, RngDevice(instance.Config.MachineType))
|
||||
}
|
||||
|
||||
if instance.MgmtMAC != "" {
|
||||
mgmtTap := MgmtTapName(instance.ID)
|
||||
instance.Config.NetDevs = append(instance.Config.NetDevs, NetDev{
|
||||
Value: fmt.Sprintf("tap,id=mgmt0,ifname=%s,script=no,downscript=no", mgmtTap),
|
||||
})
|
||||
instance.Config.Devices = append(instance.Config.Devices, Device{
|
||||
Value: fmt.Sprintf("virtio-net-pci,netdev=mgmt0,mac=%s", instance.MgmtMAC),
|
||||
})
|
||||
slog.Info("Management NIC configured", "tap", mgmtTap, "mac", instance.MgmtMAC, "ip", instance.MgmtIP, "instanceId", instance.ID)
|
||||
}
|
||||
|
||||
instance.Config.Devices = append(instance.Config.Devices, Device{Value: "virtio-rng-pci"})
|
||||
|
||||
for i, addr := range instance.GPUPCIAddresses {
|
||||
xvga := "off"
|
||||
if instance.GPUXVGAEnabled {
|
||||
@@ -284,7 +340,7 @@ func (m *Manager) startQEMU(instance *VM) error {
|
||||
|
||||
// Wait briefly for nbdkit to start.
|
||||
// TODO: Improve, confirm nbdkit started for each volume.
|
||||
time.Sleep(2 * time.Second)
|
||||
time.Sleep(nbdkitPreExecWait(instance.DirectBoot))
|
||||
|
||||
processChan := make(chan int, 1)
|
||||
exitChan := make(chan int, 1)
|
||||
@@ -368,7 +424,9 @@ func (m *Manager) startQEMU(instance *VM) error {
|
||||
return fmt.Errorf("failed to start qemu")
|
||||
}
|
||||
|
||||
time.Sleep(1 * time.Second)
|
||||
// Catch immediate-start crashes (bad cmdline, missing kernel, etc.) before
|
||||
// attempting QMP.
|
||||
time.Sleep(qemuStartSettleWait(instance.DirectBoot))
|
||||
|
||||
select {
|
||||
case exitErr := <-exitChan:
|
||||
@@ -416,7 +474,8 @@ func (m *Manager) appendDevHostfwdNIC(instance *VM) {
|
||||
if bindIP == "" || bindIP == "0.0.0.0" {
|
||||
bindIP = "127.0.0.1"
|
||||
}
|
||||
netdevVal := fmt.Sprintf("user,id=dev0,hostfwd=tcp:%s:%s-:22", bindIP, sshDebugPort)
|
||||
var nb strings.Builder
|
||||
fmt.Fprintf(&nb, "user,id=dev0,hostfwd=tcp:%s:%s-:22", bindIP, sshDebugPort)
|
||||
|
||||
if instance.ExtraHostfwd != nil {
|
||||
for guestPort := range instance.ExtraHostfwd {
|
||||
@@ -435,17 +494,15 @@ func (m *Manager) appendDevHostfwdNIC(instance *VM) {
|
||||
slog.Warn("DEV_NETWORKING: failed to convert extra hostfwd port", "hostPort", hostPort, "err", convErr)
|
||||
continue
|
||||
}
|
||||
netdevVal += fmt.Sprintf(",hostfwd=tcp:%s:%s-:%d", bindIP, hostPort, guestPort)
|
||||
fmt.Fprintf(&nb, ",hostfwd=tcp:%s:%s-:%d", bindIP, hostPort, guestPort)
|
||||
instance.ExtraHostfwd[guestPort] = hostPortInt
|
||||
slog.Info("DEV_NETWORKING: extra hostfwd", "guestPort", guestPort, "hostPort", hostPort, "instanceId", instance.ID)
|
||||
}
|
||||
}
|
||||
|
||||
instance.Config.NetDevs = append(instance.Config.NetDevs, NetDev{Value: netdevVal})
|
||||
instance.Config.NetDevs = append(instance.Config.NetDevs, NetDev{Value: nb.String()})
|
||||
devMac := GenerateDevMAC(instance.ID)
|
||||
instance.Config.Devices = append(instance.Config.Devices, Device{
|
||||
Value: fmt.Sprintf("virtio-net-pci,netdev=dev0,mac=%s", devMac),
|
||||
})
|
||||
instance.Config.Devices = append(instance.Config.Devices, NetDevice(instance.Config.MachineType, "dev0", devMac))
|
||||
slog.Info("DEV_NETWORKING: added dev NIC with SSH hostfwd",
|
||||
"bindIP", bindIP, "port", sshDebugPort, "mac", devMac, "instanceId", instance.ID)
|
||||
}
|
||||
@@ -586,7 +643,7 @@ func buildBaseVMConfig(instanceID, pidFile, consoleLogPath, serialSocket, archit
|
||||
// buildDrives converts EBS volume requests into QEMU drive, iothread, and
|
||||
// device configurations. Returns an error if any non-EFI volume is missing
|
||||
// its NBDURI.
|
||||
func buildDrives(requests []types.EBSRequest, cpuCount int) ([]Drive, []IOThread, []Device, error) {
|
||||
func buildDrives(requests []types.EBSRequest, cpuCount int, machineType string) ([]Drive, []IOThread, []Device, error) {
|
||||
var drives []Drive
|
||||
var iothreads []IOThread
|
||||
var devices []Device
|
||||
@@ -611,10 +668,7 @@ func buildDrives(requests []types.EBSRequest, cpuCount int) ([]Drive, []IOThread
|
||||
|
||||
iothreadID := "ioth-os"
|
||||
iothreads = append(iothreads, IOThread{ID: iothreadID})
|
||||
devices = append(devices, Device{
|
||||
Value: fmt.Sprintf("virtio-blk-pci,drive=%s,iothread=%s,num-queues=%d,bootindex=1",
|
||||
drive.ID, iothreadID, cpuCount),
|
||||
})
|
||||
devices = append(devices, BlkDevice(machineType, drive.ID, iothreadID, cpuCount, 1))
|
||||
}
|
||||
|
||||
if v.CloudInit {
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/mulgadc/spinifex/spinifex/types"
|
||||
"github.com/stretchr/testify/assert"
|
||||
@@ -42,6 +43,7 @@ func TestBuildDrives(t *testing.T) {
|
||||
name string
|
||||
requests []types.EBSRequest
|
||||
cpuCount int
|
||||
machineType string
|
||||
wantDrives []Drive
|
||||
wantIOThreads []IOThread
|
||||
wantDevices []Device
|
||||
@@ -112,7 +114,11 @@ func TestBuildDrives(t *testing.T) {
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
drives, iothreads, devices, err := buildDrives(tt.requests, tt.cpuCount)
|
||||
machineType := tt.machineType
|
||||
if machineType == "" {
|
||||
machineType = "q35"
|
||||
}
|
||||
drives, iothreads, devices, err := buildDrives(tt.requests, tt.cpuCount, machineType)
|
||||
|
||||
if tt.wantErr != "" {
|
||||
require.Error(t, err)
|
||||
@@ -605,3 +611,170 @@ func TestAppendDevHostfwdNIC_ExtraHostfwd_HappyPath(t *testing.T) {
|
||||
require.Len(t, instance.Config.Devices, 1)
|
||||
assert.Equal(t, fmt.Sprintf("virtio-net-pci,netdev=dev0,mac=%s", GenerateDevMAC("i-extra-ok")), instance.Config.Devices[0].Value)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// startQEMU — DirectBoot branch
|
||||
//
|
||||
// We call startQEMU directly (same package) and force early returns via tap
|
||||
// errors so we cover the DirectBoot config-assignment lines and the tap setup
|
||||
// paths without spawning a real QEMU process or sleeping 2 s.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// directBootManager wires the minimum Deps for startQEMU to reach the
|
||||
// DirectBoot tap-setup block: InstanceTypes resolver + NetworkPlumber.
|
||||
func directBootManager(t *testing.T, plumber NetworkPlumber) *Manager {
|
||||
t.Helper()
|
||||
t.Setenv("XDG_RUNTIME_DIR", t.TempDir())
|
||||
resolver := fakeInstanceTypeResolver{
|
||||
"t3.nano": {Architecture: "x86_64", VCPUs: 1, MemoryMiB: 512},
|
||||
}
|
||||
return NewManagerWithDeps(Deps{
|
||||
InstanceTypes: resolver,
|
||||
NetworkPlumber: plumber,
|
||||
})
|
||||
}
|
||||
|
||||
// TestStartQEMU_DirectBoot_PrimaryTapError verifies that when the
|
||||
// NetworkPlumber fails to set up the primary ENI tap, startQEMU returns a
|
||||
// wrapped error. This also drives the DirectBoot config-assignment block
|
||||
// (PIDFile / ConsoleLogPath / SerialSocket) before the early return.
|
||||
func TestStartQEMU_DirectBoot_PrimaryTapError(t *testing.T) {
|
||||
tapErr := errors.New("tap setup failed")
|
||||
plumber := &fakeNetworkPlumber{setupErr: tapErr}
|
||||
m := directBootManager(t, plumber)
|
||||
|
||||
instance := &VM{
|
||||
ID: "i-db-tap-err",
|
||||
InstanceType: "t3.nano",
|
||||
DirectBoot: true,
|
||||
ENIId: "eni-000000000000aaaa",
|
||||
ENIMac: "02:aa:bb:cc:dd:01",
|
||||
}
|
||||
|
||||
err := m.startQEMU(instance)
|
||||
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "setup tap device")
|
||||
// PIDFile and ConsoleLogPath are set before the tap call.
|
||||
// SerialSocket is intentionally empty for direct-boot (file chardev used instead).
|
||||
assert.NotEmpty(t, instance.Config.PIDFile)
|
||||
assert.NotEmpty(t, instance.Config.ConsoleLogPath)
|
||||
assert.Empty(t, instance.Config.SerialSocket)
|
||||
// Exactly one SetupTap call (the primary ENI tap).
|
||||
require.Len(t, plumber.setupCalls, 1)
|
||||
}
|
||||
|
||||
// TestStartQEMU_DirectBoot_ExtraENITapError verifies that a tap failure on a
|
||||
// secondary ENI returns an error without starting QEMU.
|
||||
func TestStartQEMU_DirectBoot_ExtraENITapError(t *testing.T) {
|
||||
callCount := 0
|
||||
plumber := &fakeNetworkPlumber{}
|
||||
// Override SetupTap to fail only on the second call (extra ENI).
|
||||
realSetupCalls := &plumber.setupCalls
|
||||
_ = realSetupCalls
|
||||
|
||||
// Use a scripted plumber: success for primary, error for extra.
|
||||
scriptedPlumber := &scriptedNetworkPlumber{
|
||||
errs: []error{nil, errors.New("extra tap failed")},
|
||||
}
|
||||
m := directBootManager(t, scriptedPlumber)
|
||||
_ = callCount
|
||||
|
||||
instance := &VM{
|
||||
ID: "i-db-extra-tap-err",
|
||||
InstanceType: "t3.nano",
|
||||
DirectBoot: true,
|
||||
ENIId: "eni-000000000000bbbb",
|
||||
ENIMac: "02:aa:bb:cc:dd:02",
|
||||
ExtraENIs: []ExtraENI{
|
||||
{ENIID: "eni-000000000000cccc", ENIMac: "02:aa:bb:cc:dd:03"},
|
||||
},
|
||||
}
|
||||
|
||||
err := m.startQEMU(instance)
|
||||
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "extra ENI")
|
||||
assert.Len(t, scriptedPlumber.calls, 2)
|
||||
}
|
||||
|
||||
// TestStartQEMU_DirectBoot_MgmtTapError verifies that a tap failure on the
|
||||
// management NIC returns a wrapped error.
|
||||
func TestStartQEMU_DirectBoot_MgmtTapError(t *testing.T) {
|
||||
// No primary ENI → skips the ENI tap block; MgmtMAC set → enters mgmt block.
|
||||
plumber := &fakeNetworkPlumber{setupErr: errors.New("mgmt tap failed")}
|
||||
m := directBootManager(t, plumber)
|
||||
|
||||
instance := &VM{
|
||||
ID: "i-db-mgmt-err",
|
||||
InstanceType: "t3.nano",
|
||||
DirectBoot: true,
|
||||
MgmtMAC: "02:aa:bb:cc:dd:ff",
|
||||
}
|
||||
|
||||
err := m.startQEMU(instance)
|
||||
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "mgmt tap")
|
||||
require.Len(t, plumber.setupCalls, 1)
|
||||
assert.Equal(t, MgmtTapName("i-db-mgmt-err"), plumber.setupCalls[0].Name)
|
||||
assert.Equal(t, "br-mgmt", plumber.setupCalls[0].Bridge)
|
||||
}
|
||||
|
||||
// TestStartQEMU_DirectBoot_NoENI_NoMgmt_InstanceTypeNotFound verifies that
|
||||
// startQEMU returns an error when the instance type is absent from the resolver,
|
||||
// exercising the early-return before any DirectBoot config is written.
|
||||
func TestStartQEMU_DirectBoot_InstanceTypeNotFound(t *testing.T) {
|
||||
plumber := &fakeNetworkPlumber{}
|
||||
m := directBootManager(t, plumber)
|
||||
|
||||
instance := &VM{
|
||||
ID: "i-db-unknown-type",
|
||||
InstanceType: "x9.enormous",
|
||||
DirectBoot: true,
|
||||
}
|
||||
|
||||
err := m.startQEMU(instance)
|
||||
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "x9.enormous")
|
||||
assert.Empty(t, plumber.setupCalls)
|
||||
}
|
||||
|
||||
// scriptedNetworkPlumber returns errors from a pre-loaded slice, one per call.
|
||||
type scriptedNetworkPlumber struct {
|
||||
calls []TapSpec
|
||||
errs []error
|
||||
idx int
|
||||
}
|
||||
|
||||
func (p *scriptedNetworkPlumber) SetupTap(spec TapSpec) error {
|
||||
p.calls = append(p.calls, spec)
|
||||
if p.idx < len(p.errs) {
|
||||
err := p.errs[p.idx]
|
||||
p.idx++
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *scriptedNetworkPlumber) CleanupTap(_ string) error { return nil }
|
||||
|
||||
var _ NetworkPlumber = (*scriptedNetworkPlumber)(nil)
|
||||
|
||||
// TestNbdkitPreExecWait locks in the per-mode pre-exec sleep budget: direct-boot
|
||||
// is zero (no drives, no nbdkit) and the PC-machine path keeps the 2 s wait that
|
||||
// the rest of the launch flow depends on.
|
||||
func TestNbdkitPreExecWait(t *testing.T) {
|
||||
assert.Equal(t, time.Duration(0), nbdkitPreExecWait(true))
|
||||
assert.Equal(t, 2*time.Second, nbdkitPreExecWait(false))
|
||||
}
|
||||
|
||||
// TestQemuStartSettleWait locks in the per-mode post-fork settle: direct-boot
|
||||
// trims to 50 ms (QMP socket binds within a few ms when there is no firmware,
|
||||
// ROM, or block layer setup) and the PC-machine path retains the historical
|
||||
// 1 s buffer that catches bad-cmdline crashes before the QMP dial.
|
||||
func TestQemuStartSettleWait(t *testing.T) {
|
||||
assert.Equal(t, 50*time.Millisecond, qemuStartSettleWait(true))
|
||||
assert.Equal(t, time.Second, qemuStartSettleWait(false))
|
||||
}
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
package vm
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// TestMicroVMDirectBoot asserts that a microvm Config with direct-boot fields
|
||||
// set produces the expected QEMU args and does not include PCI-specific flags
|
||||
// such as -bios or pcie-root-port.
|
||||
func TestMicroVMDirectBoot(t *testing.T) {
|
||||
cfg := Config{
|
||||
CPUCount: 2,
|
||||
Memory: 512,
|
||||
Architecture: "x86_64",
|
||||
MachineType: "microvm,x-option-roms=off,pic=off,pit=off,rtc=on,acpi=on,isa-serial=on",
|
||||
KernelImage: "/path/vmlinuz",
|
||||
Initrd: "/path/initramfs.cpio.gz",
|
||||
KernelCmdline: "console=ttyS0 panic=1",
|
||||
FwCfg: []FwCfgEntry{
|
||||
{Name: "opt/spinifex/netcfg", File: "/tmp/nc.tmp"},
|
||||
},
|
||||
}
|
||||
|
||||
cmd, err := cfg.Execute()
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, cmd)
|
||||
|
||||
args := cmd.Args[1:]
|
||||
|
||||
// Machine type must be set
|
||||
assert.Equal(t, "microvm,x-option-roms=off,pic=off,pit=off,rtc=on,acpi=on,isa-serial=on", argValue(args, "-M"))
|
||||
|
||||
// Direct-boot flags must be present
|
||||
assert.Equal(t, "/path/vmlinuz", argValue(args, "-kernel"))
|
||||
assert.Equal(t, "/path/initramfs.cpio.gz", argValue(args, "-initrd"))
|
||||
assert.Equal(t, "console=ttyS0 panic=1", argValue(args, "-append"))
|
||||
assert.Equal(t, "name=opt/spinifex/netcfg,file=/tmp/nc.tmp", argValue(args, "-fw_cfg"))
|
||||
|
||||
// PCI-specific flags must NOT appear
|
||||
assert.False(t, argExists(args, "-bios"), "-bios must not appear for microvm direct-boot")
|
||||
for _, a := range args {
|
||||
assert.NotContains(t, a, "pcie-root-port", "pcie-root-port must not appear for microvm")
|
||||
}
|
||||
}
|
||||
@@ -86,9 +86,7 @@ func (m *Manager) setupExtraENINICs(instance *VM) error {
|
||||
instance.Config.NetDevs = append(instance.Config.NetDevs, NetDev{
|
||||
Value: fmt.Sprintf("tap,id=%s,ifname=%s,script=no,downscript=no", netID, extraTapName),
|
||||
})
|
||||
instance.Config.Devices = append(instance.Config.Devices, Device{
|
||||
Value: fmt.Sprintf("virtio-net-pci,netdev=%s,mac=%s", netID, extra.ENIMac),
|
||||
})
|
||||
instance.Config.Devices = append(instance.Config.Devices, NetDevice(instance.Config.MachineType, netID, extra.ENIMac))
|
||||
slog.Info("Extra VPC NIC configured",
|
||||
"tap", extraTapName, "eni", extra.ENIID, "mac", extra.ENIMac, "subnet", extra.SubnetID)
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package vm
|
||||
import (
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
@@ -333,6 +334,12 @@ func (m *Manager) shutdownAndUnmount(instance *VM) {
|
||||
slog.Error("Volume unmount failed", "id", instance.ID, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
for _, fw := range instance.Config.FwCfg {
|
||||
if err := os.Remove(fw.File); err != nil && !os.IsNotExist(err) {
|
||||
slog.Warn("Failed to remove fw_cfg temp file", "file", fw.File, "id", instance.ID, "err", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// cleanupTapDevices removes the primary VPC tap, every extra ENI tap, and
|
||||
|
||||
+46
-4
@@ -119,6 +119,12 @@ type VM struct {
|
||||
// filters out tagged VMs from customer-facing listings.
|
||||
ManagedBy string `json:"managed_by,omitempty"`
|
||||
|
||||
// DirectBoot signals that this VM was configured with a pre-built
|
||||
// vm.Config by the launcher (microvm direct-boot path). When true,
|
||||
// startQEMU skips buildBaseVMConfig and uses the existing Config
|
||||
// instead. False for all PC-machine VMs.
|
||||
DirectBoot bool `json:"direct_boot,omitempty"`
|
||||
|
||||
// GPUPCIAddresses holds the PCI addresses of all GPUs bound to this instance via
|
||||
// VFIO (e.g. ["0000:03:00.0"]). Empty for non-GPU instances. Single-GPU instances
|
||||
// have one entry; multi-GPU instances have one per assigned GPU.
|
||||
@@ -195,6 +201,17 @@ type Config struct {
|
||||
// UseUEFI requests OVMF firmware instead of the default SeaBIOS for x86_64 VMs.
|
||||
// Falls back silently to SeaBIOS if OVMF is not installed on the host.
|
||||
UseUEFI bool `json:"use_uefi,omitempty"`
|
||||
|
||||
KernelImage string `json:"kernel_image,omitempty"` // path to vmlinuz; emits -kernel when set
|
||||
Initrd string `json:"initrd,omitempty"` // path to initramfs; emits -initrd when set
|
||||
KernelCmdline string `json:"kernel_cmdline,omitempty"` // emits -append when set
|
||||
FwCfg []FwCfgEntry `json:"fw_cfg,omitempty"` // each emits -fw_cfg name=<name>,file=<path>
|
||||
}
|
||||
|
||||
// FwCfgEntry describes a single QEMU firmware configuration file entry.
|
||||
type FwCfgEntry struct {
|
||||
Name string `json:"name"`
|
||||
File string `json:"file"`
|
||||
}
|
||||
|
||||
func (cfg *Config) Execute() (*exec.Cmd, error) {
|
||||
@@ -230,10 +247,19 @@ func (cfg *Config) Execute() (*exec.Cmd, error) {
|
||||
args = append(args, "-display", "none")
|
||||
}
|
||||
|
||||
if cfg.SerialSocket != "" && cfg.ConsoleLogPath != "" {
|
||||
if IsMMIO(cfg.MachineType) && cfg.ConsoleLogPath != "" {
|
||||
// microvm with isa-serial=on: the machine creates the ISA serial device.
|
||||
// Use -chardev file so output is captured without a socket client, then
|
||||
// -serial chardev:console0 wires it to the existing device (not a new one).
|
||||
// Do NOT use -device isa-serial here — that adds a second device (ttyS1)
|
||||
// while the kernel talks to ttyS0.
|
||||
args = append(args, "-chardev", fmt.Sprintf("file,id=console0,path=%s", cfg.ConsoleLogPath))
|
||||
args = append(args, "-serial", "chardev:console0")
|
||||
} else if cfg.SerialSocket != "" && cfg.ConsoleLogPath != "" {
|
||||
chardevOpts := fmt.Sprintf("socket,id=console0,path=%s,server=on,wait=off,logfile=%s",
|
||||
cfg.SerialSocket, cfg.ConsoleLogPath)
|
||||
args = append(args, "-chardev", chardevOpts, "-serial", "chardev:console0")
|
||||
args = append(args, "-chardev", chardevOpts)
|
||||
args = append(args, "-serial", "chardev:console0")
|
||||
}
|
||||
|
||||
if cfg.CPUCount > 0 {
|
||||
@@ -252,8 +278,8 @@ func (cfg *Config) Execute() (*exec.Cmd, error) {
|
||||
args = append(args, "-object", fmt.Sprintf("iothread,id=%s", iot.ID))
|
||||
}
|
||||
|
||||
if len(cfg.Drives) == 0 {
|
||||
return nil, fmt.Errorf("at least one drive is required")
|
||||
if len(cfg.Drives) == 0 && cfg.KernelImage == "" {
|
||||
return nil, fmt.Errorf("at least one drive or a kernel image is required")
|
||||
}
|
||||
|
||||
for _, drive := range cfg.Drives {
|
||||
@@ -296,6 +322,22 @@ func (cfg *Config) Execute() (*exec.Cmd, error) {
|
||||
args = append(args, "-netdev", netdev.Value)
|
||||
}
|
||||
|
||||
if cfg.KernelImage != "" {
|
||||
args = append(args, "-kernel", cfg.KernelImage)
|
||||
}
|
||||
|
||||
if cfg.Initrd != "" {
|
||||
args = append(args, "-initrd", cfg.Initrd)
|
||||
}
|
||||
|
||||
if cfg.KernelCmdline != "" {
|
||||
args = append(args, "-append", cfg.KernelCmdline)
|
||||
}
|
||||
|
||||
for _, fw := range cfg.FwCfg {
|
||||
args = append(args, "-fw_cfg", fmt.Sprintf("name=%s,file=%s", fw.Name, fw.File))
|
||||
}
|
||||
|
||||
var qemuArchitecture string
|
||||
|
||||
switch cfg.Architecture {
|
||||
|
||||
+70
-2
@@ -3,6 +3,7 @@ package vm
|
||||
import (
|
||||
"os"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
@@ -34,9 +35,9 @@ func TestExecute(t *testing.T) {
|
||||
|
||||
cmd, err = cfg.Execute()
|
||||
|
||||
// Expect error, at least one drive required
|
||||
// Expect error, at least one drive or kernel image required
|
||||
assert.Error(t, err)
|
||||
assert.ErrorContains(t, err, "at least one drive is required")
|
||||
assert.ErrorContains(t, err, "at least one drive or a kernel image is required")
|
||||
assert.Nil(t, cmd)
|
||||
|
||||
cfg.Drives = []Drive{
|
||||
@@ -231,6 +232,49 @@ func argExists(args []string, flag string) bool {
|
||||
return slices.Contains(args, flag)
|
||||
}
|
||||
|
||||
// allArgValues returns all values following flag (every occurrence).
|
||||
func allArgValues(args []string, flag string) []string {
|
||||
var out []string
|
||||
for i, a := range args {
|
||||
if a == flag && i+1 < len(args) {
|
||||
out = append(out, args[i+1])
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestExecute_DriveAndKernelInvariant(t *testing.T) {
|
||||
base := Config{
|
||||
CPUCount: 2,
|
||||
Memory: 1024,
|
||||
Architecture: "x86_64",
|
||||
}
|
||||
|
||||
t.Run("drives-only is OK", func(t *testing.T) {
|
||||
cfg := base
|
||||
cfg.Drives = []Drive{{File: "disk.img", Format: "raw"}}
|
||||
cmd, err := cfg.Execute()
|
||||
assert.NoError(t, err)
|
||||
assert.NotNil(t, cmd)
|
||||
})
|
||||
|
||||
t.Run("kernel-only is OK", func(t *testing.T) {
|
||||
cfg := base
|
||||
cfg.KernelImage = "/boot/vmlinuz"
|
||||
cmd, err := cfg.Execute()
|
||||
assert.NoError(t, err)
|
||||
assert.NotNil(t, cmd)
|
||||
})
|
||||
|
||||
t.Run("both-empty returns error", func(t *testing.T) {
|
||||
cfg := base
|
||||
cmd, err := cfg.Execute()
|
||||
assert.Error(t, err)
|
||||
assert.ErrorContains(t, err, "at least one drive or a kernel image")
|
||||
assert.Nil(t, cmd)
|
||||
})
|
||||
}
|
||||
|
||||
func TestResetNodeLocalState(t *testing.T) {
|
||||
v := &VM{
|
||||
ID: "i-abc123",
|
||||
@@ -306,6 +350,30 @@ func TestExecute_SerialSocketAndConsoleLog(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
func TestExecute_MicrovmFileChardev(t *testing.T) {
|
||||
cfg := Config{
|
||||
CPUCount: 1,
|
||||
Memory: 512,
|
||||
Architecture: "x86_64",
|
||||
MachineType: "microvm,x-option-roms=off",
|
||||
ConsoleLogPath: "/var/log/console.log",
|
||||
KernelImage: "/boot/vmlinuz",
|
||||
}
|
||||
|
||||
cmd, err := cfg.Execute()
|
||||
assert.NoError(t, err)
|
||||
require.NotNil(t, cmd)
|
||||
|
||||
args := cmd.Args[1:]
|
||||
chardev := argValue(args, "-chardev")
|
||||
assert.Equal(t, "file,id=console0,path=/var/log/console.log", chardev)
|
||||
// -serial chardev:console0 attaches to the isa-serial=on device (ttyS0).
|
||||
assert.Equal(t, "chardev:console0", argValue(args, "-serial"))
|
||||
// No explicit isa-serial device addition (would create ttyS1, not ttyS0).
|
||||
devs := strings.Join(allArgValues(args, "-device"), " ")
|
||||
assert.NotContains(t, devs, "isa-serial")
|
||||
}
|
||||
|
||||
func TestExecute_ARM64_Q35(t *testing.T) {
|
||||
const uefiPath = "/usr/share/qemu-efi-aarch64/QEMU_EFI.fd"
|
||||
if _, err := os.Stat(uefiPath); err != nil {
|
||||
|
||||
+33
-9
@@ -12,6 +12,21 @@ set -e
|
||||
# ./tests/e2e/run-lb-e2e.sh # internal-only (single-node)
|
||||
# ./tests/e2e/run-lb-e2e.sh --peer <ip> # all 4 variants (multi-node, legacy)
|
||||
# ./tests/e2e/run-lb-e2e.sh --nodes <ip1> <ip2> ... # all 4 variants (multi-node, preferred)
|
||||
# ./tests/e2e/run-lb-e2e.sh --microvm # same as above but with microvm.elbv2_enabled=true
|
||||
#
|
||||
# CI dual-path: this script is run twice in CI for microvm validation:
|
||||
# 1. Default (microvm.elbv2_enabled=false) — baseline PC-machine path
|
||||
# 2. With --microvm flag — direct-boot microvm path
|
||||
# Both runs must pass identical assertions.
|
||||
#
|
||||
# Density test (manual, not CI-automated):
|
||||
# LB_COUNT=50 ./tests/e2e/run-lb-e2e.sh --microvm
|
||||
# Launches 50 LBs; asserts all reach Active; total QEMU fleet RSS logged.
|
||||
#
|
||||
# Customer-VM regression: run-e2e.sh (RunInstances + SSH) is run separately
|
||||
# with microvm.elbv2_enabled=true to confirm customer VMs are unaffected.
|
||||
# LaunchSystemInstance with DirectBoot=true is only reached by the ELBv2
|
||||
# service; RunInstances takes the PC-machine path unconditionally.
|
||||
|
||||
cd "$(dirname "$0")/../.."
|
||||
|
||||
@@ -31,13 +46,20 @@ fi
|
||||
# ==========================================================================
|
||||
PEER_NODE_IP=""
|
||||
ALL_NODE_IPS=()
|
||||
MICROVM_MODE=false
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--peer) PEER_NODE_IP="$2"; shift 2 ;;
|
||||
--nodes) shift; while [ $# -gt 0 ] && [[ "$1" != --* ]]; do ALL_NODE_IPS+=("$1"); shift; done ;;
|
||||
*) echo "Unknown arg: $1"; exit 1 ;;
|
||||
--peer) PEER_NODE_IP="$2"; shift 2 ;;
|
||||
--nodes) shift; while [ $# -gt 0 ] && [[ "$1" != --* ]]; do ALL_NODE_IPS+=("$1"); shift; done ;;
|
||||
--microvm) MICROVM_MODE=true; shift ;;
|
||||
*) echo "Unknown arg: $1"; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ "$MICROVM_MODE" = true ]; then
|
||||
export SPINIFEX_MICROVM_ELBV2_ENABLED=true
|
||||
echo "[run-lb-e2e] microvm mode: SPINIFEX_MICROVM_ELBV2_ENABLED=true"
|
||||
fi
|
||||
# --nodes supersedes --peer; fall back to --peer for backwards compat
|
||||
if [ ${#ALL_NODE_IPS[@]} -gt 0 ] && [ -z "$PEER_NODE_IP" ]; then
|
||||
PEER_NODE_IP="${ALL_NODE_IPS[1]:-}"
|
||||
@@ -776,7 +798,7 @@ if [ "$PEER_AVAILABLE" = true ]; then
|
||||
# test_http_from <node_ip> <url> <label>
|
||||
test_http_from() {
|
||||
local node_ip="$1" url="$2" label="$3"
|
||||
echo "Testing $label at $url from $node_ip..."
|
||||
echo "Testing $label at $url from $node_ip..." >&2
|
||||
local ok=false
|
||||
for attempt in $(seq 1 20); do
|
||||
local result
|
||||
@@ -788,17 +810,16 @@ if [ "$PEER_AVAILABLE" = true ]; then
|
||||
if echo "$result" | grep -q "instance_id"; then
|
||||
ok=true; break
|
||||
fi
|
||||
echo " Attempt $attempt/20: not yet responding..."
|
||||
echo " Attempt $attempt/20: not yet responding..." >&2
|
||||
sleep 5
|
||||
done
|
||||
if [ "$ok" = true ]; then pass "$label reachable"; else fail "$label unreachable at $url from $node_ip"; fi
|
||||
echo "$ok"
|
||||
}
|
||||
|
||||
# test_tcp_from <node_ip> <ip> <port> <label>
|
||||
test_tcp_from() {
|
||||
local node_ip="$1" target_ip="$2" port="$3" label="$4"
|
||||
echo "Testing $label at ${target_ip}:${port} from $node_ip..."
|
||||
echo "Testing $label at ${target_ip}:${port} from $node_ip..." >&2
|
||||
local ok=false
|
||||
for attempt in $(seq 1 20); do
|
||||
local probe
|
||||
@@ -808,16 +829,16 @@ if [ "$PEER_AVAILABLE" = true ]; then
|
||||
probe=$(peer_ssh "$node_ip" "echo '' | nc -w5 '$target_ip' '$port'" 2>/dev/null || true)
|
||||
fi
|
||||
if [ -n "$probe" ]; then ok=true; break; fi
|
||||
echo " Attempt $attempt/20..."
|
||||
echo " Attempt $attempt/20..." >&2
|
||||
sleep 5
|
||||
done
|
||||
if [ "$ok" = true ]; then pass "$label reachable"; else fail "$label unreachable at ${target_ip}:${port} from $node_ip"; fi
|
||||
echo "$ok"
|
||||
}
|
||||
|
||||
# --- ALB: local connectivity (from hosting node) ---
|
||||
ALB_URL="http://${ALB_PUBLIC_IP}:80"
|
||||
ALB_LOCAL_OK=$(test_http_from "$ALB_HOST" "$ALB_URL" "ALB inet (local)")
|
||||
if [ "$ALB_LOCAL_OK" = "true" ]; then pass "ALB inet (local) reachable"; else fail "ALB inet (local) unreachable at $ALB_URL from $ALB_HOST"; fi
|
||||
|
||||
# ALB traffic test from hosting node
|
||||
if [ "$ALB_LOCAL_OK" = "true" ]; then
|
||||
@@ -845,6 +866,7 @@ if [ "$PEER_AVAILABLE" = true ]; then
|
||||
# --- NLB: local connectivity (from hosting node) ---
|
||||
if [ -n "$NLB_PUBLIC_IP" ] && [ "$NLB_PUBLIC_IP" != "null" ]; then
|
||||
NLB_LOCAL_OK=$(test_tcp_from "$NLB_HOST" "$NLB_PUBLIC_IP" 9000 "NLB inet (local)")
|
||||
if [ "$NLB_LOCAL_OK" = "true" ]; then pass "NLB inet (local) reachable"; else fail "NLB inet (local) unreachable at ${NLB_PUBLIC_IP}:9000 from $NLB_HOST"; fi
|
||||
if [ "$NLB_LOCAL_OK" = "true" ]; then
|
||||
if [ "$NLB_HOST" = "$LOCAL_IP" ]; then
|
||||
run_tcp_traffic_test "$NLB_PUBLIC_IP" 9000 "NLB inet (local)"
|
||||
@@ -870,6 +892,7 @@ if [ "$PEER_AVAILABLE" = true ]; then
|
||||
# --- ALB: remote connectivity (from a different node) ---
|
||||
if [ -n "$ALB_REMOTE" ]; then
|
||||
ALB_REMOTE_OK=$(test_http_from "$ALB_REMOTE" "$ALB_URL" "ALB inet (remote)")
|
||||
if [ "$ALB_REMOTE_OK" = "true" ]; then pass "ALB inet (remote) reachable"; else fail "ALB inet (remote) unreachable at $ALB_URL from $ALB_REMOTE"; fi
|
||||
if [ "$ALB_REMOTE_OK" = "true" ]; then
|
||||
declare -A ALB_REMOTE_COUNTS
|
||||
ALB_REMOTE_TOTAL=0
|
||||
@@ -891,6 +914,7 @@ if [ "$PEER_AVAILABLE" = true ]; then
|
||||
# --- NLB: remote connectivity (from a different node) ---
|
||||
if [ -n "$NLB_REMOTE" ] && [ -n "$NLB_PUBLIC_IP" ] && [ "$NLB_PUBLIC_IP" != "null" ]; then
|
||||
NLB_REMOTE_OK=$(test_tcp_from "$NLB_REMOTE" "$NLB_PUBLIC_IP" 9000 "NLB inet (remote)")
|
||||
if [ "$NLB_REMOTE_OK" = "true" ]; then pass "NLB inet (remote) reachable"; else fail "NLB inet (remote) unreachable at ${NLB_PUBLIC_IP}:9000 from $NLB_REMOTE"; fi
|
||||
if [ "$NLB_REMOTE_OK" = "true" ]; then
|
||||
declare -A NLB_REMOTE_COUNTS
|
||||
NLB_REMOTE_TOTAL=0
|
||||
|
||||
Reference in New Issue
Block a user