Commit Graph

2073 Commits

Author SHA1 Message Date
Josh Siviour abb817c7d5 feat: add admin Ochre catalog read and drop provider-direct model (#917)
Expose per-model availability and reason on an operator-only ListOchreCatalog
action for the console, keeping the tenant ListFoundationModels omission
intact. Remove the unbuilt provider-direct entry from the v1 catalog.
2026-08-27 21:13:44 +10:00
Josh Siviour d5d5d3defc refactor: move instance state onto kvstore typed stores (#916) 2026-08-27 20:59:25 +10:00
Josh Siviour 286e11da03 feat: recover kvstore buckets from stream loss (#915) 2026-08-27 16:19:41 +10:00
Julian Sommer 8336a73e4d fix: stop the leader-bucket test racing its own create advisory
The server queues the API audit advisory after the reply the caller
waits on, so subscribing once the bucket exists could still catch the
create the first call made. Watch from the start and account for it.
2026-08-27 16:12:22 +10:00
Julian Sommer 9133c48fdb fix: wait for the EC2 command outcome instead of racing the handler
The handler records its request point after it replies, so asserting as
soon as Request returns could read the metric before it landed. Poll for
the outcome the way the sibling drain test does.
2026-08-27 16:02:26 +10:00
Julian Sommer 4ae281bd1f fix: give reachable-agent RDS tests a realistic apply-params budget
The 40ms apply-params timeout was shared by the tests that expect the
agent to answer and those that expect it not to, so a parallel coverage
run could miss the reply. Only the unreachable-agent cases keep the
short budget.
2026-08-27 15:56:44 +10:00
Josh Siviour c9da4221b9 refactor: move stream-loss detection into kvutil (#914) 2026-08-27 15:52:48 +10:00
Josh Siviour fbbb9fb30b fix: route every running-state write through the local file (#912) 2026-08-27 15:20:48 +10:00
Julian Sommer c81abd1836 fix: add fipsboot guard to missing services 2026-08-27 15:05:49 +10:00
Julian Sommer 8c7e3dfa7b refactor: use bluebottle/pkg/safecast for the numeric conversions (#913) 2026-08-27 14:55:44 +10:00
Josh Siviour 68156c7348 feat: semantic topic detection for guardrails via embedding similarity (#908)
* feat: semantic topic detection for guardrails via embedding similarity

Guardrail topic policy previously only blocked on an exact word-boundary
match against a denied topic's name or listed examples, so a paraphrase or
synonym for a blocked topic sailed straight through. This adds an
embedding-similarity check: each denied topic's name, definition, and
examples are embedded and cached, incoming text is embedded and compared by
max cosine similarity against those phrase vectors, and a topic blocks when
that similarity clears a conservative threshold. The literal exact-match
check still runs first and always blocks on its own. If no embedder is
configured, or an embed call fails, the check falls back to the existing
literal match alone rather than ever failing open.

* fix: fail closed when guardrail topic embedding is unavailable

The topic policy's semantic check silently fell back to its literal-only
matcher whenever the embedder was configured but temporarily unreachable
(for example during model cold-start), so a paraphrase of a denied topic
could pass through unblocked while the embedding endpoint was down. Now that
case is distinguished from an unconfigured embedder: if literal matching
already caught the request, that block still wins outright. Otherwise, when
the embedder is wired but the embed call itself fails, the guardrail check
returns an error and the caller aborts the request rather than letting
unverified content through.
2026-08-27 14:54:42 +10:00
Julian Sommer 95afec6a76 refactor: use bluebottle/pkg/fipsboot for the FIPS 140-3 startup guard (#911) 2026-08-27 14:38:36 +10:00
Julian Sommer 97acfbbd80 refactor: use bluebottle/pkg/tlsconfig for the TLS curve allowlist (#910) 2026-08-27 14:31:29 +10:00
Josh Siviour fc1e950518 fix: keep a node's instances when its KV record is absent at boot (#909) 2026-08-27 14:26:31 +10:00
Josh Siviour f410b1b868 feat: converge DNS on change rather than on a timer (#902)
* feat: converge DNS on change rather than on a timer

The drift backstop now watches the four stores dnsDesiredSet reads: instance state, ELBv2 load balancers, and the per-account EKS and RDS buckets. Only the trigger changes; the leader election, computeBatch and the publish path are untouched.

The interval becomes the backstop for drift the buckets cannot report, such as a record edited directly in the zone object, rather than the path by which a lifecycle event reaches DNS.

* test: cover the DNS watch wiring and reconciler run path
2026-08-27 12:43:40 +10:00
Josh Siviour 63f47b202b feat: Ochre console — Guardrails page (#906)
* feat: add Ochre console Guardrails page

* fix: treat guardrail action BLOCKED without detected flag as blocked
2026-08-27 12:33:40 +10:00
Julian Sommer 347c2b2f80 fix: emit required IAM list members when nothing matches
ListUsers, ListRoles, ListInstanceProfiles and ListGroupsForUser built their
results from a nil slice when the filtering loop matched nothing. xmlutil
drops a nil slice entirely rather than emitting an empty element, so each
response omitted a member its output shape marks required.

Each operation's empty-bucket shortcut already returned an empty non-nil
slice, so only the filtering path was affected. Start those slices empty and
non-nil too.

ListAttachedGroupPolicies, ListAttachedRolePolicies, ListPolicies and
ListAttachedUserPolicies declare no required output members, so their nil
slices are legal and are left alone.
2026-08-27 11:51:35 +10:00
Julian Sommer fae6b08acc fix: conform DescribeImages BootMode and IAM list required members
DescribeImages projected BootMode through aws.String unconditionally, so an
AMI recording none emitted "", which is outside the model enum. RegisterImage
takes BootMode as an optional input, so this is the normal path for an AMI
registered without one, not stale data. Set the member only when present.

ListAccessKeys and ListGroups built their results from a nil slice when
nothing matched. xmlutil drops a nil slice entirely rather than emitting an
empty element, so the response lost a member the model marks required. Start
both slices empty and non-nil.

Integration conformance drops from 18 violations to 15; the remainder are the
EC2 AccessDenied error-code findings, which are unrelated.
2026-08-27 11:46:39 +10:00
Julian Sommer 23f7081a2a test: pin the ListVolumes/ListSnapshots page-size clamp
The conformance subtest asserting the clamp ran against a provider holding
no volumes, so len(resp.Volumes) <= MaxListResults held whatever PageSize
did. Removing the clamp branch left the suite green.

Add a unit test on PageSize covering above the cap, at it, below it, and at
or below zero, and give the conformance subtests a fixture so the claim they
keep, that an over-cap request returns a page rather than an error, can fail.
2026-08-27 11:34:40 +10:00
Julian Sommer 40a73a8bba fix: project instance status state through vm.EC2APIState
buildInstanceStatus read vm.EC2StateCodes directly, so a StateError instance
surfaced the non-AWS name "error" while DescribeInstances returned "stopped"
for the same instance.
2026-08-27 11:08:57 +10:00
Julian Sommer 35a00a14be fix: project Placement.AvailabilityZone for every instance (mulga-2ep36) 2026-08-27 10:58:05 +10:00
Julian Sommer 62c1378b79 fix: raise the Ubuntu minimum to 24.04 (mulga-ucvsq)
Ubuntu 22.04 was accepted by detect_os but the runtime apt list has never
resolved on jammy, which ships dhcpcd5 rather than dhcpcd-base. No CI cell
or E2E matrix installs on it, and the install docs already claim Ubuntu
26.04 and Debian 13 only, so drop the claim instead of carrying a
jammy-conditional package branch for an untested suite.
2026-08-27 10:49:46 +10:00
Julian Sommer d82ee71b54 refactor: drop duplicated bluebottle logic and hand-rolled token buckets (#905) 2026-08-27 10:29:06 +10:00
Josh Siviour db2bf8cf0d feat: Ochre console — Knowledge Bases page (mulga-0t256) (#901)
* feat: add Ochre console Knowledge Bases page

Read-only Knowledge Bases section for the Ochre console, plus the
one-time bedrock-agent client/query plumbing later Ochre pages reuse.

* feat: render retrieve tester chunks in a monospace scroll block
2026-08-27 09:45:46 +10:00
Julian Sommer 0362cdf371 refactor: drop the forked HTTP middleware for the shared one (#904) 2026-08-27 09:30:40 +10:00
Julian Sommer 2da27c4e2a fix: return InsufficientInstanceCapacity with a 5xx status (#903) 2026-08-27 09:14:30 +10:00
Julian Sommer 3bf0379daa fix: gate sts:AssumeRole on the caller's identity policy (#889) 2026-08-27 09:04:25 +10:00
Josh Siviour 6fd9b17f62 feat: add kvstore watch and the reconciler loop (#900)
Watch is used zero times in spinifex against 33 ticker loops. This adds the primitive on Bucket, which already owns the memoised handle, plus the watch/debounce/resync/re-establish loop, so each converted caller changes only its trigger.

Watches are established before the first pass: a change landing between the two would otherwise wait for the next resync, since the pass that would have seen it ran before the watch existed.
2026-08-27 08:53:21 +10:00
Josh Siviour 2d370081c3 refactor: route bedrock serving endpoints through kvstore (#899)
Service.bucket() resolved a JetStream client and did a get-or-create round trip on every KV operation; kvstore.Store memoises the handle per Service.

Add Store.Purge, since the endpoint records are deleted and re-created under the same key and Create is the single-writer claim: a delete marker is not the same starting state as an absent key.
2026-08-27 00:17:26 +10:00
Josh Siviour 8dddf9d55e refactor: convert the bedrock access, credential and usage stores to kvstore (#898) 2026-08-26 23:55:58 +10:00
Josh Siviour f0b650a83b refactor: convert the bedrock guardrail and provisioned stores to kvstore (#897) 2026-08-26 23:38:33 +10:00
Josh Siviour 526cd2d070 feat: enforce guardrail topicPolicy denied-topics at inference (#896) 2026-08-26 23:33:58 +10:00
Josh Siviour 70414154c5 refactor: convert the bedrock weights, prices and logging stores to kvstore (#895) 2026-08-26 23:23:25 +10:00
Josh Siviour e24fd42bdc refactor: convert the ochrevector appliance to kvstore (#894) 2026-08-26 23:02:13 +10:00
Josh Siviour e54b79b7bf fix: harden ochre ingest progress, job cancel, and retrieve error typing (#893)
Persist ingestion progress and bound per-document work with a timeout so a
job on a dead backend fails fast and visibly instead of sitting silently
IN_PROGRESS; add StopIngestionJob so a stuck job can be cancelled; map a
backend-down Retrieve/RetrieveAndGenerate error to 503 instead of an opaque
500.
2026-08-26 23:01:57 +10:00
Josh Siviour d1c9f5c6ae refactor: convert the ochrevector registry and kb stores to kvstore (#892) 2026-08-26 22:36:19 +10:00
Josh Siviour 9e450ac94b test: verify embedding sub-batching splits oversized input into bounded requests (#891) 2026-08-26 22:08:04 +10:00
Josh Siviour 5fc4eb0289 refactor: convert the acm certificate store to kvstore (#890) 2026-08-26 22:07:46 +10:00
Josh Siviour d43c0e4302 feat: add kvstore typed store and convert the ochrevector job store (#888) 2026-08-26 15:16:28 +10:00
Julian Sommer 46d8c03293 fix: pin predastore to the dev build that drops ExtractPolicyName
The bluebottle bump in #887 removed pkg/auth.ExtractPolicyName, which
predastore v1.18.0 still calls, breaking e2e-build on the runner.
2026-08-26 15:08:35 +10:00
Julian Sommer 08da638375 refactor: route policy ARN parsing through the shared bluebottle parser (#887) 2026-08-26 14:51:18 +10:00
Josh Siviour 155f869d23 refactor: convert placement group CAS loops to kvutil (#886) 2026-08-26 13:19:21 +10:00
Julian Sommer a865ffc48e release: update go.mod for v1.18.0 v1.18.0 2026-08-26 12:32:26 +10:00
Julian Sommer 2eef7b2b42 fix: use the sigv4 helper to redact session tokens from mismatch logs
The local helper masked x-amz-security-token by line prefix, which matched the
header form and missed the presigned query form entirely, leaking the token into
the exported logs on every signature mismatch. Defer to bluebottle's
RedactedCanonicalRequest, which knows the structure and masks both forms.
2026-08-26 12:17:43 +10:00
Julian Sommer 521e547c5e chore: bump cov req 2026-08-26 12:04:07 +10:00
Julian Sommer 9637978177 fix: pin the ISO cells' spinifex checkout to GIT_BRANCH
Cells 21 and 22 compile the checked-out spinifex tree, but their checkout
had no ref, so a scheduled run built main against siblings pinned to dev.
2026-08-26 11:43:28 +10:00
Julian Sommer 3196c0c993 test: move gateway authz tests to external test packages
Both files only ever called exported identifiers, so binding them to the
exported API needs just an import alias and qualified call sites. Clears
the test-package-check failure that surfaced once dev's base ref moved to
origin/main.
2026-08-26 11:33:55 +10:00
Julian Sommer e3d394864d fix: propagate KV read errors when releasing an ECS reservation
releaseReservation returned nil for every kv.Get error, so a NATS
timeout or connection drop was reported as a successful release and the
stopped task's CPU/memory/GPU capacity leaked on that instance. Narrow
the nil return to jetstream.ErrKeyNotFound, matching the pattern already
used in service.go and task_stop.go, and return anything else.
2026-08-26 11:33:51 +10:00
Josh Siviour 78a0dc7661 fix: stop reporting a placement group read failure as a successful (#885)
removal
2026-08-26 11:19:05 +10:00
Julian Sommer 8a300431a6 fix: support '?' and backslash escapes in EC2 filter wildcard matching (#884) 2026-08-26 11:08:19 +10:00