mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-16 01:22:00 +00:00
Josh Siviour
835dbbaf1c
* feat: parameterise bedrock endpoint create/delete by account and pinned Widens EnsureEndpointInput/DescribeEndpointInput/DeleteEndpointInput with AccountID and (Ensure-only) Pinned, and keys the daemon's Ensure/Describe/Delete on the resolved account instead of the hardcoded GlobalAccountID. An empty AccountID still resolves to GlobalAccountID and Pinned defaults to false, so every existing caller (resolver, reaper, evictForCapacity, the NATS gateway client) is unchanged. This is stage 1 of provisioned throughput: it reuses the existing launch/capacity/readiness machinery for a dedicated, account-scoped, pinned endpoint without adding a parallel surface. * feat(bedrock): add provisioned-throughput CRUD and PT ARN Implements CreateProvisionedModelThroughput, Get, List, Update, and Delete over a new bedrock-provisioned KV store, plus the provisioned-model ARN format/parse. Create rejects provider-hosted models and requests a pinned, account-scoped endpoint from the daemon; Get derives status live from the pinned endpoint's state; Update honours name changes only and rejects a served-model swap; Delete tears down the pinned endpoint before removing the record. EndpointProvisioner is declared in gateway_bedrock with primitive-typed methods (handlers_bedrock already imports gateway_bedrock, so it can't be imported back for its EndpointService interface directly); a small adapter in handlers_bedrock bridges the two, always setting Pinned:true and the caller's real account on every Ensure. Does not yet wire a PT ARN into the Converse/InvokeModel inference path (next stage). * feat(bedrock): accept a provisioned-throughput ARN on Converse/InvokeModel Translates a PT ARN before resolve: the gateway inference entry parses it, loads the bedrock-provisioned record, and resolves against the pinned endpoint for the commitment's own account rather than the GlobalAccountID shorthand every bare modelId uses. Adds EndpointResolver.EndpointForAccount (gateway-side interface, implemented by DynamicEndpointResolver) so the seam stays import-cycle-safe, and threads it through vllm.go/invoke_llama.go's resolve call sites. A foreign-account or unknown PT ARN reads as ResourceNotFoundException, same as any other unresolvable model identifier. * fix(bedrock): accept a provisioned-throughput ARN on streaming inference ConverseStream and InvokeModelWithResponseStream built their router/adapter with a nil ProvisionedStore and the GlobalAccountID-shorthand provider, so a PT ARN was silently never translated on the streaming path even though the non-stream Converse/InvokeModel siblings already handled it. Thread the gateway's provisioned store through both entry points and give Router.ConverseStream / InvokeStreamRouter.InvokeModelWithResponseStream the same translate-before-resolve treatment Converse/InvokeModel already have. * fix(bedrock): surface pinned endpoints in the operator CLI Service.List only ever enumerated the GlobalAccountID key prefix, and 'endpoint describe' always looked up GlobalAccountID too, so a pinned, account-scoped endpoint held a GPU while reading ABSENT and missing from list. List now scans every account's records, and describe gains an optional --account flag (default empty, preserving GlobalAccountID resolution) to look up a specific account's endpoint. list/describe both render the endpoint's account and pinned status.