8 Commits

Author SHA1 Message Date
Josh Siviour 1e1a51a8dc Allow test-link Lambda function environments (#1656)
* feat(lambda): deliver function environment to test-link guests

* fix(lambda): validate environment names before reserved keys
2026-11-08 08:55:22 +10:00
Josh Siviour 4be471c196 feat(lambda): boot test-link invocations under Firecracker (#1648)
* feat(lambda): deliver guest configuration on a disk the agent erases

* feat(lambda): boot test-link attempts under Firecracker with a kernel hard kill

* fix(lambda): detect the configuration disk by its missing fw_cfg entry

* test(lambda): cover the configuration-disk seal from the agent and state in-package reasons

* fix(lambda): release a Firecracker attempt whose supervisor never started
2026-11-06 11:18:24 +10:00
Josh Siviour ce29eb12bf feat(lambda): activate test-link functions and invoke a first function end to end (#1646)
* feat(lambda): activate test-link functions once the node serves the pinned image

In lambdatestlink builds the function owner moves a Pending function with
stored code to Active when this node's daemon serves the pinned runtime
image and the cluster has one node. The pass runs at start, after each
code-stored completion and periodically, and writes by compare-and-swap.
Release builds set no readiness, so nothing leaves Pending there.

* build: install the test-link Lambda daemon, image and initializer

install-lambda-testlink puts the outputs of build-lambda-testlink-image and
build-lambda-testlink in /usr/share/spinifex/microvm/lambda-testlink and
/usr/local/bin, where the tagged daemon and gateway read them, and restarts
the services. Controlled-test only; nothing default changes.

* fix(lambda): keep test-link functions the executor cannot run Pending

A function with environment variables or a code copy over the test-link
bound is refused by the executor on every Invoke, so readiness no longer
activates it and names the unsupported setting as its StateReason.

* build: drop install-lambda-testlink in favour of the deploy role

A target stop and restart can leave members running the old spx. The
Ansible deploy role already stops, waits, swaps and restarts any member
still on a stale binary, so the test-link install goes through it.

* docs: describe test-link activation and its install path in the Lambda contract README

* test(lambda): run the activation tests from an external package

The wake test now waits for the start pass before storing code, so only
the code-stored wake can activate the second function.

* test(lambda): mark the readiness test as in-package with its reason

* fix(lambda): refuse handlers AWS refuses and keep ones the test link cannot run Pending

* fix(lambda): stop reporting the test link as serving once it stops accepting

* test(lambda): enable readiness after the first function is stored so no wake is buffered

* revert(lambda): leave CreateFunction handler validation unchanged in this slice

* test(lambda): call ServeControl with the fixed frame limit from dev

* test(lambda): hold a daemon invocation while serving ends on listener failure
2026-11-06 07:43:11 +10:00
Josh Siviour 17ef43327e feat(lambda): carry 6 MiB test-link request and response payloads (#1644)
* feat(lambda): declare test-link control-socket payload and frame bounds

Replace the NATS-derived control-socket limits with a raw 6 MiB payload
bound and a frame bound that leaves 64 KiB beside a full base64 payload.
The daemon admits at most two connections from accept to close, closes
admitted connections on shutdown, and the gateway's dial stops at its
reply deadline. The control name moves to /3 so mixed limits never meet.

* fix(lambda): join test-link close-on-cancel callbacks before releasing

context.AfterFunc's stop does not wait for a callback already running, so
a connection's close could still be in flight after its admission slot
was released and Serve returned. Join a started callback first, and test
the join, a stalled peer at shutdown and cancellation racing completion.

* fix(lambda): refuse test-link requests whose guest frame cannot fit

The control frame and the guest invocation frame bound different fields,
so a request the control socket accepts could still overflow HostSend
after the grant started. The daemon now encodes the guest frame at its
longest deadline before Start and refuses one that does not fit.

* fix(lambda): check the test-link guest frame after taking the slot

Encode the guest frame at MinInt64, the longest int64 deadline, and only
once the busy slot is held, so a busy refusal never encodes the payload;
a fit refusal releases the slot. Bound every wait in the lifecycle and
guest-frame tests so a regression fails rather than hangs.

* test: cover test-link control frame bounds, admission and late dispatch

* test: prove 6 MiB test-link invoke through the gateway and real daemon refusals

* test: join held exchanges and check the echo runtime's own result

* chore(lambda): describe the executor request bound as the control frame

* test(lambda): prove test-link sender bounds, accepted connections and full lifetimes

The sender test uses a real presigned code URL, the longest function name
and qualifier, and the real frame limit; a 6 MiB + 1 payload is refused
before encoding, shown by bytes allocated. Admission reads the kernel socket
table, and the memory measurement covers both lifetimes through full replies.

* docs(lambda): document the /3 control socket and its bounds

* test(lambda): sample heap only across the measured lifetimes and join the sampler

The sampler now starts after the post-GC baseline and stops after both
replies are read, and cleanup joins it if the test fails first.

* test(lambda): check the /2 control socket cannot reach /3 in either direction
2026-11-05 22:57:51 +10:00
Josh Siviour 668f6341e3 feat: bind the runtime image digest to each test-link Invoke (#1641)
* fix: hold the test-link slot when the grant start outcome is unknown

* feat: bind the pinned runtime image digest to each test-link Invoke

The gateway carries the daemon's image digest pin and sends it in every
request; the daemon refuses before Start when it is missing or differs. The
control socket moves to a versioned name so mixed gateway and daemon builds
never connect.

* test: tie the mixed-version control test to the production socket name

The gateway logs a missing or malformed image pin once at startup.

* test: cover withdraw faults, image replacement and the digest's share of the request bound

* test: assert the at-limit request is exactly the probed size

* test: send the prepared digest so owner-refusal cases reach the owner
2026-11-05 21:51:58 +10:00
Josh Siviour 93d14a4a24 feat(lambda): deliver and revoke an execution-role session per test-link Invoke (#1635)
* feat(sts): revoke one Lambda environment session by scope and revision

* fix(sts): keep the access key out of Lambda session revoke errors

* feat(lambda): add the test-link control socket with peer UID checks

* fix(lambda): bound the test-link reply wait and keep cancellation sticky

* fix(lambda): cap the test-link write by the reply bound and refuse a zero bound

* test(lambda): bound the control-socket peer waits and surface peer errors

* feat(lambda): issue and revoke the execution session around each test-link Invoke

* feat(lambda): serve test-link requests on the control socket and deliver the execution credential to the guest

The daemon answers the gateway on the abstract control socket instead of
NATS and hands the guest the execution-role credential in its first frame.
Test-link builds negotiate spinifex-lambda-testlink/2 on both ends, so a
mismatched host and guest fail the handshake before any request is sent,
and the agent refuses a frame without a credential.

* test(lambda): run the composed test-link seam on real STS and a real control socket

The seam now issues and revokes sessions through the real STS service and
reaches the daemon's attempt steps over a real abstract control socket.
New cases show the guest's credential resolving as the role session with
its Lambda labels under the base ARN until Invoke returns, both orders of a
give-up revoke against the first frame, a failed revoke staying unknown
without the key in logs, a write error after full delivery being unknown
and revoked, and legacy NATS forgeries never reaching the caller.

* test(lambda): show the test-link daemon refuses only before the VM starts

A started process whose launch identity cannot be recorded replies unknown,
proven with a stub VM that is alive when its identity is read and an
injected write failure. A source check pins every refused reply in the
launch path before the VM start. The files the attempt leaves, its reply
and the log carry neither the credential nor the code URL.

* test(lambda): fail a delivered write at the control client and bound the seam waits

A test-only writer at the control client's write boundary forwards both
frames, waits until the daemon has read them, then fails the write; the
client reports an outcome that is not ErrNotSent. Seam tests now bound every
wait, open their gates and cancel their contexts at cleanup, and join the
inner exchange they start.

* docs(lambda): note the test-link ALPN protocol in the guest contract README

* test(sts): satisfy lint in the Lambda session revoke tests

* test(lambda): mark the new in-package test-link tests with their reason

* fix(lambda): report a refused test-link attempt with a failed revoke as unknown and run the handshake tests in CI
2026-11-05 15:21:39 +10:00
Josh Siviour 22cca42bf4 feat(lambda): authenticate the guest's control connection at startup (invocation disabled) (#1451)
* feat(lambda): open the authenticated control connection at guest startup

A normal lambda-agent build now reads its per-generation identity from
fw_cfg, unloads qemu_fw_cfg and refuses to start unless the fw_cfg tree is
gone, then dials the host under the pinned mutual TLS and holds the
connection. It accepts no invocations and never starts the bootstrap: any
data from the host fails closed until the message protocol exists.

The image loads qemu_fw_cfg and the virtio_net chain and configures the
guest interface from opt/spinifex/netcfg, as other system VMs do. The
lambdatestlink build also seals fw_cfg before starting the bootstrap.

* test(lambda): bound startup test waits and report host accept errors
2026-11-02 22:37:36 +10:00
Josh Siviour a541ee312d feat(lambda): add the runtime image, guest agent and authenticated TCP control connection (not wired) (#1448)
* feat(lambda): add the Lambda runtime image and guest agent

* fix(lambda): stop the guest agent after a timed-out invocation

* docs(lambda): declare the L-GUEST executor-to-agent contract

* refactor(vm): remove the unused virtio-serial port builder

Lambda guest control moves to TCP over the microVM's network, so no
caller needs SerialPorts. This reverts d7905107b.

* fix(lambda): drop the serial carrier from guest startup and keep virtio-net

run-agent no longer waits for a carrier device named on the kernel
command line, init loads virtio_net instead of virtio_console, and the
image keeps virtio_net's failover and net_failover dependencies. The
normal agent still refuses to start without an authenticated transport.

* feat(lambda): authenticate the guest's TCP control connection with pinned mutual TLS

hostlink loads the per-generation identity from fw_cfg and dials the host
over TLS 1.3 with ALPN spinifex-lambda-guest/1. VerifyPeer requires exactly
the pinned certificate, validates it, and checks its single URI SAN names
the expected scope and role. The L-GUEST contract now describes this
transport; message framing and fencing stay open, and the normal agent
still refuses to start.

* feat(lambda): add the host end of the guest's TCP mutual TLS connection

guestconn creates fresh host and guest identities for each environment
generation, produces the fw_cfg entries the guest reads, and accepts only
that generation's exact guest certificate. Peers on the untrusted bridge
that fail the handshake are closed and the listener keeps waiting until
its context ends; the cancel callback finishes before the deadline is
cleared so the listener can be reused.

The pinned-certificate check and the L-GUEST vocabulary move to
foundation/pki and contracts/lambda/v1 so both ends share them.

* docs(lambda): describe the shared connection vocabulary in the contract README
2026-11-02 22:09:58 +10:00