mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 17:11:30 +00:00
Josh Siviour
de64d6c63d
* feat(ecsgw): on-VM SigV4 client for the ECS agent→gateway channel
Adds internal/ecsgw, the twin of internal/eksgw: a TLS + SigV4 (service "ecs")
client the ecs-agent will use to reach the AWS gateway over HTTPS instead of
connecting to host NATS directly. Calls are AWS JSON 1.1 with an X-Amz-Target
action; the gateway relays onto ecs.* host-side, keeping NATS cluster-internal.
* feat(ecs): SubmitTaskStateChange gateway action → recordTaskState
Wires the AWS-API SubmitTaskStateChange action end to end (gateway action →
ecs.SubmitTaskStateChange NATS → daemon → Service.SubmitTaskStateChange) so a
gateway-routed agent reports task state without touching the NATS bus. The
handler maps the SDK input onto the same bus.TaskState shape and converges on
recordTaskState; the account is authoritative from the SigV4 caller, so an
instance cannot report state for another account's task.
* feat(ecs): durable assign KV inbox + PollAssignments gateway action
Scheduler writes task assigns into a per-instance KV inbox
(clusters/{cluster}/assignments/{instance}/{task}) instead of a
fire-and-forget core-NATS publish, and a new PollAssignments gateway
action lets a gateway-routed agent drain its inbox over TLS+SigV4.
At-least-once: an unacked assign is re-delivered on the next poll;
the agent acks accepted assigns (AckTaskIDs) and STOPPED transitions
reclaim stale inbox entries. The Poll response carries bus.Assign with
an RFC3339 time, so it is encoded with encoding/json (RawJSONActions)
rather than the jsonutil epoch-second marshaler.
* feat(ecs-agent): rewire control-plane off NATS onto the AWS gateway
The agent no longer connects to NATS. It registers, heartbeats, reports
task state, and receives assignments entirely through the AWS gateway
over TLS+SigV4, using seeded IAM credentials — keeping the NATS bus
host-internal and closing the trust-boundary gap (a tenant container
instance must not hold a shared bus token).
- controlPlane seam over ecsgw.Client replaces the NATS publisher:
Register (RegisterContainerInstance), SubmitTaskState
(SubmitTaskStateChange), PollAssignments.
- Heartbeat folds into a periodic idempotent re-register (refreshes
LastSeen); bus.Heartbeat is dropped from the agent.
- subscribeAssign (NATS subscribe) becomes a poll loop: drain the
inbox, dispatch each assign once, ack accepted taskIDs on the next
poll (at-least-once).
- config: ECS_ACCESS_KEY / ECS_SECRET_KEY / ECS_POLL_INTERVAL added,
ECS_NATS_URL removed; cloud-init initd seeds the IAM creds.
Tests move from the NATS publisher fake to a controlPlane fake.
* feat(ecs): cloud-init cred-seed + ecs-node bring-up script
ECS container instances are plain EC2 booted from the spinifex-ecs-node
AMI (AWS-faithful: no ECS-specific launch API). This renders the
cloud-init user-data that seeds /etc/spinifex-ecs/agent.env (gateway
URL/CA/region/cluster + IAM creds) and the gateway CA, then runs the
instance. The agent registers, heartbeats, polls assignments, and
reports task state entirely through the AWS gateway over TLS+SigV4 —
never NATS.
The guest dials the customer-facing gateway address (network advertise
IP), not the mgmt bridge, mirroring the EKS GatewayURL/AddonGatewayURL
split. Seeds the system key for now; scoped per-instance creds are a
follow-up (4g task-role endpoint).
* fix(ecs): reconcile dev merge — restore #418 ENI release + netns wiring
The 'Merge branch dev' resolution dropped code where Sprint 4d (#418,
awsvpc ENI-per-task) and the gateway control-plane rewrite (#419) touch
the same paths:
- recordTaskState STOPPED branch: reclaimTaskENI was clobbered by
reclaimAssignInbox; both are needed (release the task ENI AND drain the
KV inbox). Restores TestRecordTaskState_Awsvpc_ReleasesENIOnStopOnce.
- ecs-agent Agent: the netns field, its init, and the setupTaskNetns/
teardownTaskNetns wrappers were lost when the controlPlane Agent struct
won the merge, leaving taskrun.go calling undefined methods.
- eni_test assign assertion read the assign over NATS; #419 delivers it via
the durable KV inbox, so it now polls via PollAssignments.
- taskrun_test netns cases used the removed historyPub fake; switched to
fakeCP to match the controlPlane seam.
80 lines
3.2 KiB
Bash
Executable File
80 lines
3.2 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# ecs-node-bringup.sh — launch a Spinifex ECS container-instance VM.
|
|
#
|
|
# ECS container instances are plain EC2 instances booted from the spinifex-ecs-node
|
|
# AMI (Alpine + containerd + ecs-agent), AWS-faithful: there is no ECS-specific
|
|
# launch API. This script renders the cloud-init user-data that seeds the agent's
|
|
# control-plane config + IAM credentials into /etc/spinifex-ecs/agent.env and the
|
|
# gateway TLS CA into /etc/spinifex-ecs/gateway-ca.pem, then runs the instance.
|
|
#
|
|
# The agent then registers, heartbeats, polls assignments, and reports task state
|
|
# entirely through the AWS gateway over TLS+SigV4 — it never touches NATS.
|
|
#
|
|
# Usage:
|
|
# AWS_PROFILE=spinifex scripts/ecs-node-bringup.sh <cluster-name>
|
|
#
|
|
# Env overrides:
|
|
# GATEWAY_IP gateway/mgmt IP the guest dials (default 10.15.8.1)
|
|
# GATEWAY_CA host path to the gateway CA PEM (default /etc/spinifex/ca.pem)
|
|
# AMI_ID ECS-node AMI (default: latest spinifex-ecs-node)
|
|
# SUBNET_ID / SG_ID / KEY_NAME / INSTANCE_TYPE (default: first available / t3.small)
|
|
set -euo pipefail
|
|
|
|
CLUSTER="${1:?usage: ecs-node-bringup.sh <cluster-name>}"
|
|
PROFILE="${AWS_PROFILE:-spinifex}"
|
|
REGION="$(aws configure get region --profile "$PROFILE")"
|
|
GATEWAY_IP="${GATEWAY_IP:-10.15.8.1}"
|
|
GATEWAY_CA="${GATEWAY_CA:-/etc/spinifex/ca.pem}"
|
|
INSTANCE_TYPE="${INSTANCE_TYPE:-t3.small}"
|
|
|
|
awscli() { aws --profile "$PROFILE" "$@"; }
|
|
|
|
AMI_ID="${AMI_ID:-$(awscli ec2 describe-images \
|
|
--filters 'Name=name,Values=spinifex-ecs-node' \
|
|
--query 'sort_by(Images,&CreationDate)[-1].ImageId' --output text)}"
|
|
SUBNET_ID="${SUBNET_ID:-$(awscli ec2 describe-subnets \
|
|
--query 'Subnets[0].SubnetId' --output text)}"
|
|
SG_ID="${SG_ID:-$(awscli ec2 describe-security-groups \
|
|
--query 'SecurityGroups[0].GroupId' --output text)}"
|
|
KEY_NAME="${KEY_NAME:-$(awscli ec2 describe-key-pairs \
|
|
--query 'KeyPairs[0].KeyName' --output text)}"
|
|
|
|
ACCESS_KEY="$(aws configure get aws_access_key_id --profile "$PROFILE")"
|
|
SECRET_KEY="$(aws configure get aws_secret_access_key --profile "$PROFILE")"
|
|
|
|
echo "cluster=$CLUSTER ami=$AMI_ID subnet=$SUBNET_ID sg=$SG_ID key=$KEY_NAME gw=https://$GATEWAY_IP:9999" >&2
|
|
|
|
# Indent a file's content by six spaces for a cloud-init "content: |" block.
|
|
indent6() { sed 's/^/ /' "$1"; }
|
|
|
|
USERDATA="$(cat <<EOF
|
|
#cloud-config
|
|
bootcmd:
|
|
- rm -f /etc/resolv.conf
|
|
- printf 'nameserver 1.1.1.1\nnameserver 8.8.8.8\n' > /etc/resolv.conf
|
|
write_files:
|
|
- path: /etc/spinifex-ecs/agent.env
|
|
permissions: '0600'
|
|
content: |
|
|
ECS_GATEWAY_URL=https://$GATEWAY_IP:9999
|
|
ECS_GATEWAY_CA=/etc/spinifex-ecs/gateway-ca.pem
|
|
ECS_REGION=$REGION
|
|
ECS_CLUSTER=$CLUSTER
|
|
ECS_ACCESS_KEY=$ACCESS_KEY
|
|
ECS_SECRET_KEY=$SECRET_KEY
|
|
- path: /etc/spinifex-ecs/gateway-ca.pem
|
|
permissions: '0644'
|
|
content: |
|
|
$(indent6 "$GATEWAY_CA")
|
|
EOF
|
|
)"
|
|
|
|
INSTANCE_ID="$(awscli ec2 run-instances \
|
|
--image-id "$AMI_ID" --instance-type "$INSTANCE_TYPE" \
|
|
--key-name "$KEY_NAME" --subnet-id "$SUBNET_ID" --security-group-ids "$SG_ID" \
|
|
--user-data "$USERDATA" \
|
|
--tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=ecs-node-$CLUSTER}]" \
|
|
--query 'Instances[0].InstanceId' --output text)"
|
|
|
|
echo "$INSTANCE_ID"
|