mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 17:11:30 +00:00
Josh Siviour
471e95dac2
* feat(ecsgw): on-VM SigV4 client for the ECS agent→gateway channel
Adds internal/ecsgw, the twin of internal/eksgw: a TLS + SigV4 (service "ecs")
client the ecs-agent will use to reach the AWS gateway over HTTPS instead of
connecting to host NATS directly. Calls are AWS JSON 1.1 with an X-Amz-Target
action; the gateway relays onto ecs.* host-side, keeping NATS cluster-internal.
* feat(ecs): SubmitTaskStateChange gateway action → recordTaskState
Wires the AWS-API SubmitTaskStateChange action end to end (gateway action →
ecs.SubmitTaskStateChange NATS → daemon → Service.SubmitTaskStateChange) so a
gateway-routed agent reports task state without touching the NATS bus. The
handler maps the SDK input onto the same bus.TaskState shape and converges on
recordTaskState; the account is authoritative from the SigV4 caller, so an
instance cannot report state for another account's task.
* feat(ecs): durable assign KV inbox + PollAssignments gateway action
Scheduler writes task assigns into a per-instance KV inbox
(clusters/{cluster}/assignments/{instance}/{task}) instead of a
fire-and-forget core-NATS publish, and a new PollAssignments gateway
action lets a gateway-routed agent drain its inbox over TLS+SigV4.
At-least-once: an unacked assign is re-delivered on the next poll;
the agent acks accepted assigns (AckTaskIDs) and STOPPED transitions
reclaim stale inbox entries. The Poll response carries bus.Assign with
an RFC3339 time, so it is encoded with encoding/json (RawJSONActions)
rather than the jsonutil epoch-second marshaler.
* feat(ecs-agent): rewire control-plane off NATS onto the AWS gateway
The agent no longer connects to NATS. It registers, heartbeats, reports
task state, and receives assignments entirely through the AWS gateway
over TLS+SigV4, using seeded IAM credentials — keeping the NATS bus
host-internal and closing the trust-boundary gap (a tenant container
instance must not hold a shared bus token).
- controlPlane seam over ecsgw.Client replaces the NATS publisher:
Register (RegisterContainerInstance), SubmitTaskState
(SubmitTaskStateChange), PollAssignments.
- Heartbeat folds into a periodic idempotent re-register (refreshes
LastSeen); bus.Heartbeat is dropped from the agent.
- subscribeAssign (NATS subscribe) becomes a poll loop: drain the
inbox, dispatch each assign once, ack accepted taskIDs on the next
poll (at-least-once).
- config: ECS_ACCESS_KEY / ECS_SECRET_KEY / ECS_POLL_INTERVAL added,
ECS_NATS_URL removed; cloud-init initd seeds the IAM creds.
Tests move from the NATS publisher fake to a controlPlane fake.
* feat(ecs): cloud-init cred-seed + ecs-node bring-up script
ECS container instances are plain EC2 booted from the spinifex-ecs-node
AMI (AWS-faithful: no ECS-specific launch API). This renders the
cloud-init user-data that seeds /etc/spinifex-ecs/agent.env (gateway
URL/CA/region/cluster + IAM creds) and the gateway CA, then runs the
instance. The agent registers, heartbeats, polls assignments, and
reports task state entirely through the AWS gateway over TLS+SigV4 —
never NATS.
The guest dials the customer-facing gateway address (network advertise
IP), not the mgmt bridge, mirroring the EKS GatewayURL/AddonGatewayURL
split. Seeds the system key for now; scoped per-instance creds are a
follow-up (4g task-role endpoint).
* fix(ecs): reconcile dev merge — restore #418 ENI release + netns wiring
The 'Merge branch dev' resolution dropped code where Sprint 4d (#418,
awsvpc ENI-per-task) and the gateway control-plane rewrite (#419) touch
the same paths:
- recordTaskState STOPPED branch: reclaimTaskENI was clobbered by
reclaimAssignInbox; both are needed (release the task ENI AND drain the
KV inbox). Restores TestRecordTaskState_Awsvpc_ReleasesENIOnStopOnce.
- ecs-agent Agent: the netns field, its init, and the setupTaskNetns/
teardownTaskNetns wrappers were lost when the controlPlane Agent struct
won the merge, leaving taskrun.go calling undefined methods.
- eni_test assign assertion read the assign over NATS; #419 delivers it via
the durable KV inbox, so it now polls via PollAssignments.
- taskrun_test netns cases used the removed historyPub fake; switched to
fakeCP to match the controlPlane seam.
* feat(ecs): service controller + ELBv2 target-group registration (Sprint 4f)
Land the ECS Service controller on top of the 4e scheduler/RunTask flow:
- CreateService/UpdateService/DeleteService/DescribeServices/ListServices.
REPLICA only; DAEMON and serviceRegistries rejected in v1 (Q15).
- StartTask (explicit placement) and StopTask.
- Leader-driven replica reconciliation: a 10s scheduler tick converges each
service's running task count on desiredCount, launching the shortfall via
RunTask (tasks tagged group=service:{name}) and force-stopping the surplus.
- ELBv2 target-group registration: the scheduler registers a task's awsvpc ENI
IP on the RUNNING transition and deregisters on STOPPED via the existing
elbv2.RegisterTargets/DeregisterTargets NATS handlers (Q8, single-writer).
- forceStopTask consolidates the control-plane stop (STOPPED state, capacity +
ENI release, inbox drain, TG deregister); reused by StopTask, scale-in, and
the heartbeat reaper (the reaper now deregisters targets per Q13).
- Wire the 7 new actions through the gateway action map, NATS adapter, and
daemon subscriptions; add ServiceNotFoundException.
* fix(ecs-agent): anonymous pull for non-ECR images + normalize image refs
The agent minted an ECR authorization token via IMDS for every image, so
public docker.io refs failed with 'IMDS returned empty role name' on hosts
without an instance role. Gate ECR auth on .dkr.ecr. refs and pull all other
registries anonymously.
Bare image strings (nginx:alpine) were handed to containerd unmodified, which
parsed them as dummy://nginx:alpine and rejected the pull. Normalize refs to
their fully-qualified form (docker.io/library/nginx:alpine, :latest default)
before Pull and GetImage.
Adds StartTask/StopTask service-controller coverage.
* fix: address ECS 4f live-verification findings
ecs-node-bringup.sh defaulted GATEWAY_IP to the mgmt bridge (10.15.8.1), which
a VPC guest cannot route to — the agent could not reach the gateway. Default to
the host's WAN-bridge IP, which guests reach by NATing out through the WAN.
A target group not forwarded to by any listener reported its targets stuck in
'initial' forever (no health checker advances a detached TG). DescribeTargetHealth
now reports such targets as 'unused' with reason Target.NotInUse, matching AWS.
182 lines
8.5 KiB
Modula-2
182 lines
8.5 KiB
Modula-2
module github.com/mulgadc/spinifex
|
|
|
|
go 1.26.4
|
|
|
|
require (
|
|
github.com/aws/aws-sdk-go v1.55.8
|
|
github.com/charmbracelet/bubbles v1.0.0
|
|
github.com/charmbracelet/bubbletea v1.3.10
|
|
github.com/charmbracelet/lipgloss v1.1.0
|
|
github.com/containerd/containerd/v2 v2.3.2
|
|
github.com/distribution/reference v0.6.0
|
|
github.com/go-chi/chi/v5 v5.3.0
|
|
github.com/golang-jwt/jwt/v5 v5.3.1
|
|
github.com/google/uuid v1.6.0
|
|
github.com/insomniacslk/dhcp v0.0.0-20260407060928-11b94ed970f2
|
|
github.com/klauspost/cpuid/v2 v2.3.0
|
|
github.com/mulgadc/predastore v1.9.0
|
|
github.com/mulgadc/viperblock v1.9.0
|
|
github.com/nats-io/nats-server/v2 v2.14.2
|
|
github.com/nats-io/nats.go v1.52.0
|
|
github.com/opencontainers/runtime-spec v1.3.0
|
|
github.com/ovn-kubernetes/libovsdb v0.8.1
|
|
github.com/pelletier/go-toml/v2 v2.4.0
|
|
github.com/pterm/pterm v0.12.83
|
|
github.com/spf13/cobra v1.10.2
|
|
github.com/spf13/viper v1.21.0
|
|
github.com/stretchr/testify v1.11.1
|
|
go.uber.org/goleak v1.3.0
|
|
golang.org/x/crypto v0.53.0
|
|
golang.org/x/sync v0.21.0
|
|
golang.org/x/sys v0.46.0
|
|
gopkg.in/ini.v1 v1.67.3
|
|
gopkg.in/yaml.v3 v3.0.1
|
|
)
|
|
|
|
require (
|
|
atomicgo.dev/cursor v0.2.0 // indirect
|
|
atomicgo.dev/keyboard v0.2.9 // indirect
|
|
atomicgo.dev/schedule v0.1.0 // indirect
|
|
github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29 // indirect
|
|
github.com/Microsoft/hcsshim v0.15.0-rc.1 // indirect
|
|
github.com/antithesishq/antithesis-sdk-go v0.7.0 // indirect
|
|
github.com/armon/go-metrics v0.4.1 // indirect
|
|
github.com/atotto/clipboard v0.1.4 // indirect
|
|
github.com/aws/aws-sdk-go-v2 v1.42.0 // indirect
|
|
github.com/aws/smithy-go v1.27.1 // indirect
|
|
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
|
|
github.com/beorn7/perks v1.0.1 // indirect
|
|
github.com/boltdb/bolt v1.3.1 // indirect
|
|
github.com/buraksezer/consistent v0.10.0 // indirect
|
|
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
|
|
github.com/cenkalti/hub v1.0.2 // indirect
|
|
github.com/cenkalti/rpc2 v1.0.5 // indirect
|
|
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
|
github.com/charmbracelet/colorprofile v0.4.1 // indirect
|
|
github.com/charmbracelet/x/ansi v0.11.6 // indirect
|
|
github.com/charmbracelet/x/cellbuf v0.0.15 // indirect
|
|
github.com/charmbracelet/x/term v0.2.2 // indirect
|
|
github.com/clipperhouse/displaywidth v0.9.0 // indirect
|
|
github.com/clipperhouse/stringish v0.1.1 // indirect
|
|
github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
|
|
github.com/containerd/cgroups/v3 v3.1.3 // indirect
|
|
github.com/containerd/console v1.0.5 // indirect
|
|
github.com/containerd/containerd/api v1.11.1 // indirect
|
|
github.com/containerd/continuity v0.5.0 // indirect
|
|
github.com/containerd/errdefs v1.0.0 // indirect
|
|
github.com/containerd/errdefs/pkg v0.3.0 // indirect
|
|
github.com/containerd/fifo v1.1.0 // indirect
|
|
github.com/containerd/log v0.1.0 // indirect
|
|
github.com/containerd/platforms v1.0.0-rc.4 // indirect
|
|
github.com/containerd/plugin v1.1.0 // indirect
|
|
github.com/containerd/ttrpc v1.2.8 // indirect
|
|
github.com/containerd/typeurl/v2 v2.2.3 // indirect
|
|
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
|
|
github.com/dgraph-io/badger/v4 v4.9.2 // indirect
|
|
github.com/dgraph-io/ristretto/v2 v2.4.0 // indirect
|
|
github.com/dustin/go-humanize v1.0.1 // indirect
|
|
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
|
|
github.com/fatih/color v1.19.0 // indirect
|
|
github.com/felixge/httpsnoop v1.0.4 // indirect
|
|
github.com/fsnotify/fsnotify v1.9.0 // indirect
|
|
github.com/gabriel-vasile/mimetype v1.4.13 // indirect
|
|
github.com/go-logr/logr v1.4.3 // indirect
|
|
github.com/go-logr/stdr v1.2.2 // indirect
|
|
github.com/go-playground/locales v0.14.1 // indirect
|
|
github.com/go-playground/universal-translator v0.18.1 // indirect
|
|
github.com/go-playground/validator/v10 v10.30.1 // indirect
|
|
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
|
|
github.com/gogo/protobuf v1.3.2 // indirect
|
|
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
|
|
github.com/google/flatbuffers v25.12.19+incompatible // indirect
|
|
github.com/google/go-tpm v0.9.8 // indirect
|
|
github.com/gookit/color v1.6.0 // indirect
|
|
github.com/hashicorp/go-hclog v1.6.3 // indirect
|
|
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
|
|
github.com/hashicorp/go-metrics v0.5.4 // indirect
|
|
github.com/hashicorp/go-msgpack/v2 v2.1.5 // indirect
|
|
github.com/hashicorp/golang-lru v1.0.2 // indirect
|
|
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
|
|
github.com/hashicorp/raft v1.7.3 // indirect
|
|
github.com/hashicorp/raft-boltdb/v2 v2.3.1 // indirect
|
|
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
|
github.com/jmespath/go-jmespath v0.4.0 // indirect
|
|
github.com/josharian/native v1.1.0 // indirect
|
|
github.com/klauspost/compress v1.18.6 // indirect
|
|
github.com/klauspost/reedsolomon v1.14.1 // indirect
|
|
github.com/leodido/go-urn v1.4.0 // indirect
|
|
github.com/lithammer/fuzzysearch v1.1.8 // indirect
|
|
github.com/lucasb-eyer/go-colorful v1.3.0 // indirect
|
|
github.com/mattn/go-colorable v0.1.14 // indirect
|
|
github.com/mattn/go-isatty v0.0.20 // indirect
|
|
github.com/mattn/go-localereader v0.0.1 // indirect
|
|
github.com/mattn/go-runewidth v0.0.21 // indirect
|
|
github.com/mdlayher/packet v1.1.2 // indirect
|
|
github.com/mdlayher/socket v0.5.1 // indirect
|
|
github.com/minio/crc64nvme v1.1.1 // indirect
|
|
github.com/minio/highwayhash v1.0.4 // indirect
|
|
github.com/moby/locker v1.0.1 // indirect
|
|
github.com/moby/sys/mountinfo v0.7.2 // indirect
|
|
github.com/moby/sys/sequential v0.6.0 // indirect
|
|
github.com/moby/sys/signal v0.7.1 // indirect
|
|
github.com/moby/sys/user v0.4.0 // indirect
|
|
github.com/moby/sys/userns v0.1.0 // indirect
|
|
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
|
|
github.com/muesli/cancelreader v0.2.2 // indirect
|
|
github.com/muesli/termenv v0.16.0 // indirect
|
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
|
github.com/nats-io/jwt/v2 v2.8.2 // indirect
|
|
github.com/nats-io/nkeys v0.4.16 // indirect
|
|
github.com/nats-io/nuid v1.0.1 // indirect
|
|
github.com/opencontainers/go-digest v1.0.0 // indirect
|
|
github.com/opencontainers/image-spec v1.1.1 // indirect
|
|
github.com/pierrec/lz4/v4 v4.1.18 // indirect
|
|
github.com/pkg/errors v0.9.1 // indirect
|
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
|
|
github.com/prometheus/client_golang v1.23.2 // indirect
|
|
github.com/prometheus/client_model v0.6.2 // indirect
|
|
github.com/prometheus/common v0.67.5 // indirect
|
|
github.com/prometheus/procfs v0.20.1 // indirect
|
|
github.com/quic-go/quic-go v0.60.0 // indirect
|
|
github.com/rivo/uniseg v0.4.7 // indirect
|
|
github.com/sagikazarmark/locafero v0.12.0 // indirect
|
|
github.com/sirupsen/logrus v1.9.4 // indirect
|
|
github.com/spf13/afero v1.15.0 // indirect
|
|
github.com/spf13/cast v1.10.0 // indirect
|
|
github.com/spf13/pflag v1.0.10 // indirect
|
|
github.com/subosito/gotenv v1.6.0 // indirect
|
|
github.com/u-root/uio v0.0.0-20230220225925-ffce2a382923 // indirect
|
|
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
|
|
go.etcd.io/bbolt v1.4.3 // indirect
|
|
go.opencensus.io v0.24.0 // indirect
|
|
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
|
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 // indirect
|
|
go.opentelemetry.io/otel v1.44.0 // indirect
|
|
go.opentelemetry.io/otel/metric v1.44.0 // indirect
|
|
go.opentelemetry.io/otel/trace v1.44.0 // indirect
|
|
go.yaml.in/yaml/v2 v2.4.4 // indirect
|
|
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
|
golang.org/x/mod v0.36.0 // indirect
|
|
golang.org/x/net v0.56.0 // indirect
|
|
golang.org/x/telemetry v0.0.0-20260508192327-42602be52be6 // indirect
|
|
golang.org/x/term v0.44.0 // indirect
|
|
golang.org/x/text v0.38.0 // indirect
|
|
golang.org/x/time v0.15.0 // indirect
|
|
golang.org/x/tools v0.45.0 // indirect
|
|
golang.org/x/vuln v1.1.4 // indirect
|
|
google.golang.org/genproto/googleapis/rpc v0.0.0-20260618152121-87f3d3e198d3 // indirect
|
|
google.golang.org/grpc v1.81.1 // indirect
|
|
google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af // indirect
|
|
)
|
|
|
|
// go.opencensus.io -> grpc@v1.33.2 drags in the pre-split monolithic genproto,
|
|
// which still ships google.golang.org/genproto/googleapis/rpc/status and so
|
|
// collides with the split genproto/googleapis/rpc module that containerd/ttrpc
|
|
// needs ("ambiguous import"). The main module never imports the monolith, so a
|
|
// plain require is dropped by `go mod tidy`; pin it via replace to a post-split
|
|
// version (no rpc/status) so the fix survives tidy. Remove if grpc/opencensus
|
|
// stop requiring the monolith.
|
|
replace google.golang.org/genproto => google.golang.org/genproto v0.0.0-20260622175928-b703f567277d
|
|
|
|
tool golang.org/x/vuln/cmd/govulncheck
|