mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 17:11:30 +00:00
Josh Siviour
bc8ad05a0f
* feat: ochre-vector list-jobs subject Adds ochre.vector.listJobs to the daemon-side VectorService surface (both the in-process and NATS-forwarding implementations) so a caller can list every ingestion job for an account rather than looking one up by id. Widens jobGetter to expose JobStore.List, registers the subject on the daemon's post-connect subscription table, and updates the CLI test double to satisfy the widened VectorService interface. Also carries the jackc/puddle/v2 indirect dependency go.sum was missing. * feat: bedrock-agent knowledge-base + data-source CRUD Adds the emulated AWS bedrock-agent control-plane gateway service: KnowledgeBase/DataSource CRUD, StartIngestionJob/GetIngestionJob/ ListIngestionJobs, backed by new gateway-owned KBStore/DataSourceStore JetStream KV persistence (mirroring registry.go's CAS idiom) and the existing ochre-vector daemon surface for the bound vector index. One knowledge base binds exactly one vector index. embeddingModelArn maps to EmbeddingModel/Dimension; storageConfiguration and roleArn are accepted and stored verbatim but never acted on. CreateDataSource maps S3 + fixed-size chunking config onto a SourceSpec, converting AWS's maxTokens/overlapPercentage into internal ChunkSize/ChunkOverlap. DeleteKnowledgeBase cascades to every data source under it before dropping the bound index. Wires BedrockAgentKB/BedrockAgentDataSources/ BedrockAgentVector into the gateway and registers bedrock-agent in the supported-services and dispatch tables. Runtime (Retrieve/RetrieveAndGenerate) is out of scope here. * test: cover bedrock-agent routing and KB/DataSource store edges Adds end-to-end HTTP tests for BedrockAgent_Request's route table and per-route handler closures (all eleven operations, plus unknown-route/ missing-account/nil-dependency error paths), KBStore/DataSourceStore PurgeAll and no-JetStream-client error paths, and ListJobs coverage (vectorService, NATS round trip, per-account scoping) in the existing ochre-vector API test suite. * feat: bedrock-agent-runtime retrieve + retrieve-and-generate Adds the emulated bedrock-agent-runtime data plane: Retrieve resolves a knowledge base to its bound vector index and runs a similarity query; RetrieveAndGenerate runs the same retrieval, renders a RAG prompt (default or caller-supplied template), and calls Converse in-process for generation, returning citations built from the retrieved chunks and a session id. Recovers the full RetrievalFilter operator set from the raw request body since the vendored aws-sdk-go RetrievalFilter type only carries andAll/orAll, and fixes GetService never routing real signed bedrock-agent traffic (only bedrock-runtime and bedrock had discriminator rules; bedrock-agent's own knowledgebases paths had none). * fix: bedrock-agent KB delete idempotency + ingestion-job ownership CreateKnowledgeBase: a rollback DeleteIndex failure after a kb.Create claim failure is now logged at Error level with the orphaned index id instead of a bare Warn, while the caller still sees kb.Create's own error unchanged. DeleteKnowledgeBase: a bound index that DeleteIndex reports already gone (ErrIndexNotFound) no longer aborts before the KB record itself is removed, so a KB never dangles pointing at an index that no longer exists. Any other DeleteIndex error still aborts the cascade. GetIngestionJob: now verifies the addressed dataSourceId owns the job (Bucket+Prefix match against the data source's own SourceSpec, same best-effort check ListIngestionJobs already used), not just the knowledge base's bound index -- closing a path where a job could be read through the wrong data source id. * fix: default bedrock-agent-runtime retrieve to 5 results (AWS parity) * fix: thread DataSourceID through ingestion jobs JobRecord and IngestRequest now carry an exact DataSourceID instead of relying on Bucket+Prefix as a best-effort proxy. StartIngestionJob stamps it from the DataSource record; GetIngestionJob and ListIngestionJobs filter on it exactly, treating a job with no DataSourceID as belonging to no bedrock-agent data source. * fix: purge KB/DataSource records on appliance teardown Appliance.Teardown purged the index registry and job store but left the gateway-owned KB/DataSource KV records dangling, pointing at a deleted index. WithKBStores attaches those stores the same way WithStores does for registry/jobs, built before Connect so teardown still works on a broken appliance, and Teardown purges them too.
405 lines
14 KiB
YAML
405 lines
14 KiB
YAML
# Service interface manifest.
|
|
#
|
|
# Source of truth for "which services produce/consume which NATS subjects"
|
|
# and "which e2e suites cover which services". Consumed by
|
|
# tests/e2e/select-suites (Bead 4) to derive targeted suite selection from
|
|
# a PR diff.
|
|
#
|
|
# See docs/development/improvements/e2e-targeted-suite-selection.md.
|
|
#
|
|
# Editing rules
|
|
# -------------
|
|
# - subscribes / publishes: subjects this service receives / sends. Both
|
|
# lists use the literal subject string, or a "<prefix>.*" wildcard for
|
|
# pattern subscriptions (e.g. "ebs.snapshot.*").
|
|
# - depends_on: subjects this service requires from peers. Used by the
|
|
# selector to compute "if X changes, which suites might break".
|
|
# - Subjects must match what is actually referenced in source. The lint
|
|
# in Bead 5 greps `"(ec2|vpc|ebs|s3|iam|elbv2|spinifex)\.[a-zA-Z._-]+"`
|
|
# and fails if any subject is absent from this manifest.
|
|
|
|
version: 1
|
|
|
|
services:
|
|
|
|
awsgw:
|
|
path: spinifex/services/awsgw
|
|
# TLS frontend, SigV4 auth. Pure router: HTTP → NATS request/response.
|
|
# No subscribes; only publishes (via NATSRequest).
|
|
subscribes: []
|
|
publishes:
|
|
- "ec2.*"
|
|
- "elbv2.*"
|
|
- "acm.*"
|
|
- "iam.*"
|
|
- "rds.*"
|
|
- "ochre.vector.*"
|
|
depends_on:
|
|
- "ec2.*"
|
|
- "elbv2.*"
|
|
- "acm.*"
|
|
- "iam.*"
|
|
- "rds.*"
|
|
- "ochre.vector.*"
|
|
|
|
spinifex_daemon:
|
|
path: spinifex/services/spinifex
|
|
# Orchestrator. Owns most ec2.* + all elbv2.* + iam event publication.
|
|
# Translates VPC / EBS / S3 operations into vpc.* / ebs.* / s3.* and
|
|
# forwards to vpcd / viperblockd / predastore.
|
|
#
|
|
# Source layout: spinifex/services/spinifex (entry), spinifex/daemon
|
|
# (subscribe registration), spinifex/handlers/{ec2,elbv2,iam} (per-
|
|
# subject handler bodies). Selector treats all four paths as belonging
|
|
# to this service via the `additional_paths` field.
|
|
additional_paths:
|
|
- spinifex/daemon
|
|
- spinifex/handlers/ec2
|
|
- spinifex/handlers/elbv2
|
|
- spinifex/handlers/acm
|
|
- spinifex/handlers/iam
|
|
- spinifex/handlers/rds
|
|
- spinifex/handlers/ochrevector
|
|
subscribes:
|
|
# EC2 instance lifecycle.
|
|
- "ec2.RunInstances"
|
|
- "ec2.DescribeInstances"
|
|
- "ec2.DescribeStoppedInstances"
|
|
- "ec2.DescribeTerminatedInstances"
|
|
- "ec2.DescribeInstanceAttribute"
|
|
- "ec2.DescribeInstanceStatus"
|
|
- "ec2.DescribeInstanceTypes"
|
|
- "ec2.ModifyInstanceAttribute"
|
|
- "ec2.ModifyVolume"
|
|
- "ec2.start"
|
|
- "ec2.terminate"
|
|
- "ec2.launch"
|
|
- "ec2.cmd.*"
|
|
# EC2 VPC / network ops handled by daemon before delegation to vpcd.
|
|
- "ec2.CreateVpc"
|
|
- "ec2.DescribeVpcs"
|
|
- "ec2.CreateSubnet"
|
|
- "ec2.CreateInternetGateway"
|
|
- "ec2.CreateKeyPair"
|
|
- "ec2.CreateImage"
|
|
- "ec2.DescribePlacementGroups"
|
|
# Cluster coordination subjects.
|
|
- "ec2.ReserveClusterNode"
|
|
- "ec2.FinalizeClusterInstances"
|
|
- "spinifex.node.status"
|
|
- "spinifex.node.vms"
|
|
- "spinifex.nodes.discover"
|
|
- "spinifex.cluster.shutdown.progress"
|
|
# Coordinated cluster-shutdown phases (fan-out, no queue group).
|
|
- "spinifex.cluster.shutdown.gate"
|
|
- "spinifex.cluster.shutdown.drain"
|
|
- "spinifex.cluster.shutdown.storage"
|
|
- "spinifex.cluster.shutdown.persist"
|
|
- "spinifex.cluster.shutdown.infra"
|
|
- "spinifex.storage.config"
|
|
# ELBv2 control plane (handlers/elbv2/service_nats.go topic map).
|
|
- "elbv2.CreateLoadBalancer"
|
|
- "elbv2.DeleteLoadBalancer"
|
|
- "elbv2.DescribeLoadBalancers"
|
|
- "elbv2.CreateTargetGroup"
|
|
- "elbv2.DeleteTargetGroup"
|
|
- "elbv2.DescribeTargetGroups"
|
|
- "elbv2.RegisterTargets"
|
|
- "elbv2.DeregisterTargets"
|
|
- "elbv2.DescribeTargetHealth"
|
|
- "elbv2.CreateListener"
|
|
- "elbv2.DeleteListener"
|
|
- "elbv2.DescribeListeners"
|
|
# ACM-alike certificate store (handlers/acm/service_nats.go topic map).
|
|
- "acm.ImportCertificate"
|
|
- "acm.DescribeCertificate"
|
|
- "acm.ListCertificates"
|
|
- "acm.DeleteCertificate"
|
|
- "elbv2.AddListenerCertificates"
|
|
- "elbv2.RemoveListenerCertificates"
|
|
- "elbv2.DescribeListenerCertificates"
|
|
- "elbv2.DescribeSSLPolicies"
|
|
# RDS control plane (handlers/rds/subjects.go). One wildcard rather than
|
|
# 30 literals: every action subject is registered from a constant, so a
|
|
# per-subject list would be a hand-mirror of that file with nothing
|
|
# checking it.
|
|
- "rds.*"
|
|
# Layer-2 agent bus: register and health are wildcard subscriptions over
|
|
# {account}.{db-instance}, so the subject is built at runtime and only the
|
|
# prefix is declarable.
|
|
- "rds.bus.*"
|
|
# Ochre vector store tenant surface (handlers/ochrevector/api.go),
|
|
# gated behind config.OchreVectorConfig.Enabled (default off).
|
|
- "ochre.vector.createIndex"
|
|
- "ochre.vector.deleteIndex"
|
|
- "ochre.vector.listIndexes"
|
|
- "ochre.vector.ingest"
|
|
- "ochre.vector.describeJob"
|
|
- "ochre.vector.query"
|
|
- "ochre.vector.listJobs"
|
|
publishes:
|
|
# → vpcd
|
|
- "vpc.create"
|
|
- "vpc.delete"
|
|
- "vpc.create-subnet"
|
|
- "vpc.delete-subnet"
|
|
- "vpc.create-sg"
|
|
- "vpc.delete-sg"
|
|
- "vpc.update-sg"
|
|
- "vpc.create-port"
|
|
- "vpc.delete-port"
|
|
- "vpc.update-port-sgs"
|
|
- "vpc.igw-attach"
|
|
- "vpc.igw-detach"
|
|
- "vpc.add-nat-gateway"
|
|
- "vpc.delete-nat-gateway"
|
|
- "vpc.add-nat"
|
|
- "vpc.delete-nat"
|
|
- "vpc.dhcp.acquire"
|
|
- "vpc.dhcp.release"
|
|
# → viperblockd
|
|
- "ebs.mount"
|
|
- "ebs.unmount"
|
|
- "ebs.delete"
|
|
- "ebs.sync"
|
|
- "ebs.snapshot.*"
|
|
- "ebs.config"
|
|
- "ebs.config.*"
|
|
# → predastore (S3 / AMI / keypair store)
|
|
- "s3.*"
|
|
# IAM lifecycle event
|
|
- "iam.account.created"
|
|
# → the in-guest rds-agent: set-password and the like, published on the
|
|
# per-instance command subject and answered on its reply subject.
|
|
- "rds.bus.*"
|
|
depends_on:
|
|
- "vpc.*"
|
|
- "ebs.*"
|
|
- "s3.*"
|
|
|
|
vpcd:
|
|
path: spinifex/vpcd
|
|
# OVN-backed VPC / subnet / SG / NAT / IGW data plane.
|
|
subscribes:
|
|
- "vpc.create"
|
|
- "vpc.delete"
|
|
- "vpc.create-subnet"
|
|
- "vpc.delete-subnet"
|
|
- "vpc.create-sg"
|
|
- "vpc.delete-sg"
|
|
- "vpc.update-sg"
|
|
- "vpc.create-port"
|
|
- "vpc.delete-port"
|
|
- "vpc.update-port-sgs"
|
|
- "vpc.igw-attach"
|
|
- "vpc.igw-detach"
|
|
- "vpc.add-nat-gateway"
|
|
- "vpc.delete-nat-gateway"
|
|
- "vpc.add-nat"
|
|
- "vpc.delete-nat"
|
|
- "vpc.dhcp.acquire"
|
|
- "vpc.dhcp.release"
|
|
publishes:
|
|
- "vpc.port-status"
|
|
- "vpc.dhcp.lease-changed"
|
|
- "vpc.dhcp.owner-check"
|
|
depends_on: []
|
|
|
|
viperblockd:
|
|
path: spinifex/services/viperblockd
|
|
# EBS data plane. Backed by the viperblock storage engine in the
|
|
# mulga/viperblock submodule. Submodule pointer bump → selector forces
|
|
# "run all" (contract changes can be invisible at the spinifex source
|
|
# level).
|
|
subscribes:
|
|
- "ebs.mount"
|
|
- "ebs.unmount"
|
|
- "ebs.delete"
|
|
- "ebs.sync"
|
|
- "ebs.snapshot.*"
|
|
- "ebs.config"
|
|
- "ebs.config.*"
|
|
publishes:
|
|
- "ebs.mount.response"
|
|
- "ebs.unmount.response"
|
|
depends_on: []
|
|
|
|
predastore:
|
|
path: spinifex/services/predastore
|
|
# S3 / AMI / keypair storage. External submodule under mulga/predastore;
|
|
# this dir contains only the spinifex-side adapter. Submodule pointer
|
|
# bump → selector forces "run all".
|
|
subscribes:
|
|
- "s3.*"
|
|
publishes: []
|
|
depends_on: []
|
|
|
|
nats:
|
|
path: spinifex/services/nats
|
|
# NATS broker bootstrap / cluster config. Not a NATS pub/sub
|
|
# participant. Any change here is infra-path; selector forces
|
|
# "run all".
|
|
subscribes: []
|
|
publishes: []
|
|
depends_on: []
|
|
|
|
spinifexui:
|
|
path: spinifex/services/spinifexui
|
|
# Web UI. Talks to awsgw over HTTP, never NATS. Diffs limited to this
|
|
# dir do not need any e2e suite to run.
|
|
subscribes: []
|
|
publishes: []
|
|
depends_on: []
|
|
covers_no_suite: true
|
|
|
|
|
|
# Fixture → service mapping. Each Ensure* helper in
|
|
# tests/e2e/harness/fixtures.go exercises this set of services; the
|
|
# Bead 4 selector AST-scans test files for harness.Ensure* call sites
|
|
# and resolves coverage via this block.
|
|
fixtures:
|
|
|
|
EnsureKeyPair:
|
|
services: [spinifex_daemon, predastore]
|
|
|
|
EnsureAMI:
|
|
services: [spinifex_daemon, predastore, viperblockd]
|
|
|
|
EnsureDefaultVPC:
|
|
services: [vpcd, spinifex_daemon]
|
|
|
|
EnsureSubnet:
|
|
services: [vpcd, spinifex_daemon]
|
|
|
|
EnsureSG:
|
|
services: [vpcd, spinifex_daemon]
|
|
|
|
EnsureInstance:
|
|
services: [spinifex_daemon, viperblockd, vpcd]
|
|
|
|
EnsureVolume:
|
|
services: [spinifex_daemon, viperblockd]
|
|
|
|
EnsureSnapshot:
|
|
services: [spinifex_daemon, viperblockd, predastore]
|
|
|
|
EnsureNATGateway:
|
|
services: [vpcd, spinifex_daemon]
|
|
|
|
EnsureLoadBalancer:
|
|
services: [spinifex_daemon, awsgw]
|
|
|
|
|
|
suites:
|
|
|
|
e2e-cert:
|
|
path: tests/e2e/cert
|
|
covers: [awsgw, spinifex_daemon, vpcd, predastore]
|
|
|
|
e2e-lb:
|
|
path: tests/e2e/lb
|
|
covers: [awsgw, spinifex_daemon, vpcd, viperblockd, predastore]
|
|
# Exercises full ELBv2 control plane (handlers/elbv2) + data plane
|
|
# (LB agent on the system VM). ELBv2 is part of spinifex_daemon in
|
|
# this manifest; revisit if it splits into a standalone service.
|
|
|
|
e2e-single:
|
|
path: tests/e2e/single
|
|
covers: [awsgw, spinifex_daemon, vpcd, viperblockd, predastore]
|
|
# Full single-node lifecycle: VPC, AMI, keypair, instance, volume,
|
|
# snapshot, IAM. Currently runs all phases sequentially under one
|
|
# `TestSingleNode`. Bead 3 (single-fixture-migration) splits this
|
|
# into per-test fixture-driven coverage so the selector can pick
|
|
# subtests, not just whole suites.
|
|
|
|
e2e-iam:
|
|
path: tests/e2e/iam
|
|
covers: [awsgw, spinifex_daemon, vpcd, viperblockd, predastore]
|
|
# "Everything IAM": the IAM/STS control plane plus the instance-identity
|
|
# tests (instance-profile association + IMDS), which boot guest VMs — OVN
|
|
# via vpcd, AMI via predastore, root volume via viperblockd, lifecycle via
|
|
# spinifex_daemon. Single-mode, VM-booting leg.
|
|
|
|
e2e-ecr:
|
|
path: tests/e2e/ecr
|
|
covers: [awsgw, spinifex_daemon, predastore]
|
|
# ECR control plane (handlers/ecr, gateway/ecr*) + OCI data plane: blobs and
|
|
# manifests in predastore, control + auth bridge in awsgw, KV via
|
|
# spinifex_daemon. docker/crane/skopeo data-plane subtests skip when the
|
|
# client binary or registry-host resolution is absent.
|
|
|
|
e2e-bedrock:
|
|
path: tests/e2e/bedrock
|
|
covers: [awsgw]
|
|
# Ochre (AWS-Bedrock) control plane: static catalog list/get served
|
|
# straight from awsgw (gateway/bedrock.go, gateway/bedrock/) — no NATS
|
|
# hop to spinifex_daemon. Converse (Anthropic-direct / self-host vLLM)
|
|
# subtests are gated on backend availability and skip by default.
|
|
|
|
e2e-ecs:
|
|
path: tests/e2e/ecs
|
|
covers: [awsgw, spinifex_daemon, vpcd, viperblockd, predastore]
|
|
# ECS data plane end-to-end: cluster + a real container instance launched
|
|
# from the spinifex-ecs-node AMI (boots, registers over the gateway, runs
|
|
# tasks through containerd), standalone tasks in awsvpc/bridge/host modes,
|
|
# and an awsvpc service fronted by an ELBv2 target group. Boots a guest VM,
|
|
# so vpcd (OVN + awsvpc ENI), viperblockd (root volume), predastore (AMI)
|
|
# are all in the path.
|
|
|
|
e2e-rds:
|
|
path: tests/e2e/rds
|
|
covers: [awsgw, spinifex_daemon, vpcd, viperblockd, predastore]
|
|
# RDS create/describe/connect: CreateDBInstance boots a dual-NIC DB VM from
|
|
# the engine AMI (predastore) with a Viperblock data volume (viperblockd) and
|
|
# a customer ENI (vpcd), and the reconciler flips it to available on the
|
|
# in-guest agent's first healthy heartbeat over awsgw. Every test deletes the
|
|
# instances it creates, so the suite needs no gate beyond SPINIFEX_E2E.
|
|
|
|
e2e-ddil:
|
|
path: tests/e2e/ddil
|
|
covers: [awsgw, spinifex_daemon, vpcd, viperblockd, predastore]
|
|
# DDIL = degraded / disconnected / intermittent / low-bandwidth.
|
|
# Exercises the full daemon stack under network failure injection;
|
|
# any service touching the wire path is in scope.
|
|
|
|
# Suites below are bash-only today. Entries activate as the Go ports
|
|
# land (Bead 2 fixtures unblock new-shape ports for multinode etc.).
|
|
#
|
|
# e2e-multinode:
|
|
# path: spinifex/tests/e2e/multinode
|
|
# covers: [awsgw, spinifex_daemon, vpcd, viperblockd, predastore]
|
|
# e2e-reboot:
|
|
# path: spinifex/tests/e2e/reboot
|
|
# covers: [spinifex_daemon, viperblockd]
|
|
# e2e-tofu:
|
|
# path: spinifex/tests/e2e/tofu
|
|
# covers: [awsgw, spinifex_daemon, vpcd]
|
|
# e2e-bm:
|
|
# path: spinifex/tests/e2e/baremetal
|
|
# covers: [awsgw, spinifex_daemon, vpcd, viperblockd, predastore]
|
|
#
|
|
# e2e-natuplink activates once CI can provision an external_mode=nat
|
|
# node (setup-ovn.sh --nat-uplink + spx admin init --external-mode=nat).
|
|
# e2e-natuplink:
|
|
# path: spinifex/tests/e2e/natuplink
|
|
# covers: [awsgw, spinifex_daemon, vpcd, viperblockd, predastore]
|
|
|
|
|
|
# Special-case rules consumed by the selector (Bead 4). Hard-coded in
|
|
# the tool, listed here for visibility.
|
|
#
|
|
# infra_paths_force_all:
|
|
# - spinifex/services/nats/**
|
|
# - spinifex/tests/e2e/harness/**
|
|
# - spinifex/tests/e2e/lib/**
|
|
# - spinifex/.github/workflows/e2e-go.yml
|
|
# - spinifex/.github/scripts/**
|
|
# - spinifex/Makefile
|
|
# - mulga/scripts/tofu-cluster/**
|
|
#
|
|
# submodule_force_all:
|
|
# - mulga/predastore (submodule pointer)
|
|
# - mulga/viperblock (submodule pointer)
|
|
#
|
|
# many_services_force_all_threshold: 3 # ≥ N services touched → run all
|