Files
xarmian 6fdd8816f3 feat(playbook): match free text to an active playbook via the decision provider (TASK-3120) (#1429)
* feat(server): POST /playbooks/match — typed-decision Choice over active playbooks (TASK-3120)

PLAN-3114 unit 5. A synchronous, read-only endpoint: given free text, asks
the workspace's typed-decision provider which active (non-draft,
non-deprecated) playbook it best matches, with a reserved "none" option so
unrelated text gets an honest answer instead of a forced pick — and, as a
side effect, keeps a workspace with exactly one active playbook askable
(decision.Question.Validate needs >=2 Choice options).

- decision.Runner gains a nil-safe Provider() accessor: this endpoint needs
  a plain synchronous Ask, not the owed-jobs pipeline Evaluate drives.
- No provider configured -> 404 decision_provider_unavailable (callers fall
  back to slug/trigger routing). A provider error or an out-of-set answer
  the provider was never offered -> 502 decision_provider_error, distinct
  from the 404 case so a caller can tell "try again" from "not configured".
  Zero active playbooks -> 200 with a stated reason, no provider call.
  Over the provider's 255-option ceiling (including "none") -> refused
  rather than silently dropping playbooks and answering from an
  incomplete set.
- listPlaybookItems is factored out of handleListPlaybooks so list and
  match share one visibility-scoped item source and can't drift on which
  playbooks a caller can see.
- 30s timeout on the synchronous provider call, tighter than the
  provider's own worst-case retry budget (~330s) since an HTTP caller
  blocks on this one directly, unlike the async decision tick.
- content_state_population_test.go's BUG-3033 sweep gets a new ruling: the
  PlaybookSummary call here feeds the provider prompt only, never the
  response, so there's no reader-facing text to mark.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* feat(cli): pad playbook match "<text>" (TASK-3120)

CLI + client-method half of PLAN-3114 unit 5 — POST /playbooks/match.
Table output prints the matched ref/title (or "none"), confidence, model,
and a probabilities table sorted descending; --format json passes the
response through unmodified. Read-only, side-effect-free.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* docs(skill): route to playbook match as rung 3, offer-only (TASK-3120)

PLAN-3114 unit 5's skill half. Exact invocation-slug match and trigger
match stay rungs 1-2; match is consulted only when the user is plainly
asking for a procedure/workflow and both of those missed — never on
ordinary conversation, since every call spends provider money. Any match
returned is offered, never auto-run, at any confidence: TASK-3137's replay
of this same provider's attention question set found no validated
threshold, so nothing here invents one for this question either. A 404
(no provider configured), a "none" choice, or any other error falls back
silently to today's slug/trigger-only behavior.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* chore(mcp): de-indent padItemToolDescription's continuation lines (TASK-3120)

Whitespace-only — git diff -w on this file is empty. The description's
continuation lines carried 18 leading spaces purely for visual alignment
under each action's "  name — " bullet in the Go source; that padding is
emitted verbatim into the tools/list JSON, where it conveys nothing an
LLM reading the tool description gets any value from.

De-indented by exactly 14 spaces on every line that had >= 18 (all 116 of
them, uniformly, since the file carries only three indentation levels:
0, 2, 18 — no deeper nesting to preserve a relative offset for).
Continuations now land at 4 (bullet's 2 + 2), so a reader can still tell
which action a line belongs to; a 1-space collapse would have inverted
that (continuation LESS indented than its own bullet).

tools/list: 55290 -> 53666 bytes (-1624). initialize: unchanged at 24530
(it carries no tool descriptions). Frees headroom against the 55296
budget for the pad_playbook.match action landing in the next commit —
budget constants themselves are untouched, per standing instruction.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* feat(mcp): pad_playbook.match action, ToolSurfaceVersion 0.39 -> 0.41 (TASK-3120)

PLAN-3114 unit 5's MCP half, landing on top of the whitespace-only
catalog_item.go trim (previous commit) so this stays within the
tools/list budget with room to spare.

- catalog_playbook.go: new `text` param, new `match` action
  (passThrough — the CLI's positional arg is named "text", matching
  this tool's param, so no custom shaping is needed) plus a
  description bullet in padPlaybookToolDescription.
- tool_surface.go: `match` marked read-only (nothing stored, no job
  enqueued).
- dispatch_http_routes.go: `playbook match` routeSpec (POST, flat
  {text} body) so the remote /mcp transport can dispatch it in-process.
- version.go: ToolSurfaceVersion 0.39 -> 0.41. 0.40 is reserved for
  PR #1427 (the 0.32/0.33 precedent — a gap is cheaper than a
  collision). 0.41's own entry: pure ADDITIVE bump, same disposition
  as 0.13/0.28/0.11 — nothing existing moved.
- instructions.md, README.md, CLAUDE.md: version headings + the
  pad_playbook bullet/table-row/changelog-sentence updated per the
  drift tests (TestInstructionsMDCoversEveryCatalogAction,
  TestReadmeTableCoversEveryCatalogAction, the two version-match
  tests). instructions.md's addition is intentionally terse ("`match`
  finds a playbook from text.") — it's embedded in the `initialize`
  handshake payload, which has its own 24576-byte budget and had only
  46 bytes of headroom; a fuller explanation lives in the tools/list
  param/action docs instead, which have budget to spare after the trim.
- catalog_readonly_test.go, dispatch_http_parity_test.go: new
  `pad_playbook.match` entries in three hand-built fixture maps
  (expected cmdPath ×2, a hand-rolled cmdhelp stub, and the shared
  parity-drive fixture input) so the existing bijection/parity/dispatch
  guards actually exercise the new action instead of skipping it.

Bytes measured (go test ./internal/mcp -run Budget -v):
  tools/list:  55290 (baseline) -> 53666 (after the trim commit) ->
               54805 (after this commit) — 491 bytes of headroom left
               against the 55296 budget.
  initialize:  24530 (baseline, unchanged by the trim) -> 24574 (after
               this commit) — 2 bytes of headroom against the 24576
               budget. Flagged in the implementer report: this is now
               as tight as tools/list was before the trim, and nothing
               in this unit addresses it.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* fix(server,decision): don't leak provider errors, classify oversized requests, fix none's wording (TASK-3120)

Lead's shallow-pass findings on cd7a41ed, applied after the MCP commit.

F1 (disclosure): the provider-error path answered with err.Error()
verbatim. typesafe.go's client surfaces the provider's raw 4xx body in
that string (APIError.Error()), which is the INSTANCE's own account
state with typesafe.ai — key validity, quota, billing wording — not
anything about the caller's request. This endpoint requires only
viewer access, so any workspace member could read it. Now logged
server-side in full (slog, workspace + err) and answered with a fixed
message plus a coarse details.reason ("timeout" for a context
deadline, "upstream" otherwise) via a new writeDecisionProviderError
helper. Test constructs a fake error containing a secret-looking
string and asserts it never reaches the response body.

F2 (misclassification): the request budget is 32000 tokens
(~80000 chars). At a couple hundred active playbooks with realistic
summaries, the Choice question's options alone can exceed it before
any state is even added — or leave no room for the input text. Both
were plain fmt.Errorf with no sentinel, so they fell into the generic
"provider failed" 502 path, telling a caller "try again" for a
request that will never fit no matter how many times it's retried.
decision/typesafe.go gains ErrRequestTooLarge (wrapped with %w onto
both pre-send refusals; existing message text unchanged, just
appended-to) alongside the existing ErrMaxTokensExceeded (the
provider's own after-the-fact refusal of a request fitState let
through) — both now map to a distinct 400 match_request_too_large.
New decision-package unit test builds a realistic 254-option Choice
(title + near-cap summary + trigger + slug per option, mirroring what
playbook-match could plausibly send) and confirms fitState refuses it
before the fake HTTP handler is ever called; server-side table test
covers both sentinels through the fake provider.

F3 (nit): the "none" option's description read "...listed above",
but Options is an unordered map — nothing puts them "above" it.
Reworded to "...any of the other options' procedures"; the pinned
exact-prompt test updated to match.

Also (lead's separate MCP-round instruction): decision.go's
maxChoiceOptions is now exported as MaxChoiceOptions, and the
server's over-limit refusal reads that constant instead of a copied
`255` literal, so the two ceilings cannot drift apart.

go test ./internal/decision/... ./internal/mcp/... ./internal/server/...
./cmd/pad/... ./internal/cli/... — all green. golangci-lint run ./...
— 0 issues. Full `go test ./...` run separately; final line in the
report.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* chore(mcp): cut pad_playbook.match's tools/list text to the approved +292 variant (TASK-3120)

The lead's shape-(a) approval was for the measured "+292 documented"
prototype, not the fuller wording ae38353c actually shipped
(+1139 bytes) — the trim commit exists to pay for the approved
action text, not to fund arbitrary new prose. Cut the `text` param to
one sentence (the 404-fallback advice lives once, in the action
bullet, not repeated in the param) and the `match` action bullet down
to four lines at consistent bullet+2 continuation indentation,
matching the rest of padPlaybookToolDescription's style (the previous
wording had drifted to 20-space continuation on its last two lines).

tools/list: 53666 (post-trim baseline) -> 54023 (+357, close to the
~300 target — wording differs from the original prototype's exact
text, so an exact match isn't expected). initialize: unchanged at
24574 (this file's bullet is tools/list-only; instructions.md is a
separate file, untouched here).

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* fix(server): don't overclaim writeDecisionProviderError's timeout reason (TASK-3120)

The doc comment said "timeout" covered a cancelled caller request too,
but context.Canceled is a distinct error from context.DeadlineExceeded
and errors.Is(err, context.DeadlineExceeded) does not match it — that
case actually falls through to "upstream", contradicting what the
comment claimed. Narrowed the comment to deadline-only (the preferred
fix per review): a cancelled caller's own request also never reads
this response body, so the distinction was moot as well as wrong.
Code was already correct; only the comment overclaimed.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* fix(cli,skill): pad playbook match needs -- before dash-led text (TASK-3120)

Codex round 2 finding on f31d5d64. Text starting with "-" (a dash-led
sentence, or literal text like "-ship it") is parsed as a flag by
cobra's default scanner, on both the CLI and stdio MCP (which shells
out to this same command). Pre-existing and class-wide — `pad item
search "-x"` fails the same way — so the general stdio fix across
every command is filed as BUG-3142 rather than done here; BuildCLIArgs
is untouched.

This command's own help now shows the workaround: an Example section
with `pad playbook match -- "-ship it"`. skills/pad/SKILL.md's rung-3
routing entry calls it the same way, with a one-line reason, and notes
the MCP form is unaffected (a structured `text` param, not a CLI
token).

New CLI test asserts BOTH directions: `playbook match "-ship it"`
(no --) fails at cobra's flag parser and never reaches the server;
`playbook match -- "-ship it"` succeeds and the server receives the
text with its leading dash intact. Ran both legs — see the test.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* fix(skill): local stdio MCP is not exempt from the leading-dash flag parse (TASK-3120)

Codex round 4 finding. The rung-3 routing entry claimed the MCP form of
match was "unaffected" by BUG-3142, but that's only true for remote
/mcp, which keeps `text` a structured param. Local stdio MCP shells
out to `pad playbook match`, turning `text` into a CLI positional via
BuildCLIArgs (dispatch.go:435) — the same flag parser, the same bug.
Corrected: remote /mcp is safe as-is; local stdio MCP still hits it and
should prefer the CLI's `--` form when the text starts with `-`.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd

* fix(cmdhelp): the example validator's flag scan must stop at "--" (TASK-3120)

CI red on #1429 (a2a5a853), Go/Go-PG/Nix all failing the same way:
TestRealPadTree_ExampleDriftValidator flagged the new `pad playbook
match -- "-ship it"` example's own dash-led TEXT ARGUMENT as an
unknown flag `--ship it`. Cobra stops flag parsing at a lone "--" —
that's the whole point of the token — but validateExample's scan
treated "--" as just another flag-shaped token: TrimLeft reduced it
to an empty name, which fell into the "skip, keep scanning" branch
instead of stopping the scan. Everything after "--" kept getting
flag-checked, so the terminated example's own positional text was
misread as a flag.

Fixed by checking `t == "--"` BEFORE the dash-stripping and breaking
the scan there, per review. A lone "-" (the stdin convention, e.g.
`pad item import -`) is a different token — it still reaches the
dash-stripping, still reduces to an empty name, and still just
`continue`s past — so it does not terminate the scan.

Three new unit tests in internal/cmdhelp cover all three legs: (a) a
dash-led positional after "--" produces no finding, (b) an unknown
flag BEFORE "--" is still caught — the break cannot hide drift ahead
of the terminator, and (c) a lone "-" does not stop the scan (an
unknown flag after it is still caught). Manually re-ran (a) with the
break removed to confirm it goes red without the fix, then restored
the real fix — ran, not just described: FAIL "a dash-led positional
after -- should not be reported as an unknown flag" without the break,
PASS with it.

The playbook match example itself is unchanged.

Claude-Session: https://claude.ai/code/session_01ByMjAhvdWgbAWWeGE9jwWd
2026-09-22 11:28:57 -04:00

712 lines
26 KiB
Go

package decision
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strconv"
"strings"
"time"
)
// defaultTypesafeEndpoint is typesafe.ai's System One endpoint.
const defaultTypesafeEndpoint = "https://api.typesafe.ai/v1/systemone"
// Token budget for one request.
//
// maxRequestTokens is MEASURED against jev-1.13.0, not taken from the vendor
// docs, which state no limit. Bisecting on the API's own
// usage.input_tokens: 32,476 tokens was ACCEPTED and the next step up
// (~33,600) was refused with HTTP 400 max_tokens_exceeded. The budget counts
// state AND questions together — input_tokens includes the questions — so a
// large question set leaves less room for state. 32,000 sits under the
// measured accepted figure with ~476 tokens of headroom, and independently
// matches the Cloudflare model card's published context window, which is the
// only published number and is therefore unlikely to be reduced below this.
// Receipts, including the full bisection, are on TASK-3116's trail.
const maxRequestTokens = 32000
// charsPerToken converts characters to an estimated token count for
// truncation.
//
// It is a deliberate UNDER-estimate of characters per token, which makes the
// resulting token estimate an OVER-estimate and truncation conservative. The
// bisection above measured 2.80 chars/token on number-dense filler; ordinary
// English prose runs nearer 4. Dense content — base64, hashes, minified JSON —
// tokenizes worse than either, so this cannot be a guarantee: it is a cheap
// bound that keeps normal item content well inside the budget. The real
// backstop for pathological input is the provider's own refusal, surfaced as
// [ErrMaxTokensExceeded] rather than swallowed, so a caller can shrink its
// state instead of reading a silent truncation.
const charsPerToken = 2.5
// Retry policy for transient provider failures.
const (
maxAttempts = 4
initialBackoff = 500 * time.Millisecond
maxBackoff = 8 * time.Second
maxRetryAfter = 30 * time.Second
requestTimeout = 60 * time.Second
maxErrBodyLength = 2048
)
// ErrMaxTokensExceeded reports that the provider refused the request because
// state plus questions exceeded its token budget.
//
// It is distinguished from ordinary validation failures because the remedy is
// different and mechanical: send less state. The provider signals it with
// HTTP 400 and a machine-readable body ({"detail":{"error_type":
// "max_tokens_exceeded"}}), so this does not rest on matching prose.
var ErrMaxTokensExceeded = errors.New("decision: provider token budget exceeded")
// ErrRequestTooLarge reports that fitState refused a request BEFORE it was
// sent — the questions alone exceed maxRequestTokens, or state has no room
// beside them (TASK-3120). Distinguished from [ErrMaxTokensExceeded], which
// is the PROVIDER's own refusal of a request fitState judged small enough to
// send: this one never reaches the network, so a caller mapping it does not
// need to distinguish "the provider is down" from "the request was too big
// to attempt" the way an opaque error would force it to. A caller that wants
// EITHER kind of "too big for one request" can errors.Is against both
// values; one that cares about the pre-send/provider-side distinction still
// can, since they remain separate values.
var ErrRequestTooLarge = errors.New("decision: request too large for the token budget")
// APIError is an error response from the provider. It is returned directly
// for a non-retryable status, and is reachable with errors.As through the
// retry wrapper when a retryable status persists past the last attempt.
//
// The body is carried verbatim (bounded) because the provider's refusal
// message is the only way a caller can fix the call. It is also printed by
// Error(), which is safe only because the provider's refusal bodies do not
// echo the request: measured 2026-09-21 against jev-1.13.0 with canary
// strings in the state, the instructions and the criteria, three refusal
// shapes (unknown question type, unknown model, and the budget refusal)
// returned fixed {"detail":{"error_type","message"}} bodies naming none of
// them. Receipts on TASK-3116's trail. A refusal shape not in that set is
// unmeasured.
type APIError struct {
StatusCode int
// ErrorType is the provider's machine-readable code from
// detail.error_type, when present.
ErrorType string
// Body is the response body, truncated to a bounded length.
Body string
// BodyReadErr is set when reading the response body failed partway.
// Body then holds whatever arrived before the failure, and callers must
// not treat it as the provider's complete message — in particular a
// missing ErrorType may mean "could not read" rather than "not that
// error". Keeping the read failure instead of discarding it is the
// difference between a caller knowing the evidence is partial and
// silently trusting a truncated body.
BodyReadErr error
}
func (e *APIError) Error() string {
var b strings.Builder
fmt.Fprintf(&b, "decision: provider returned %d", e.StatusCode)
if e.ErrorType != "" {
fmt.Fprintf(&b, " (%s)", e.ErrorType)
}
fmt.Fprintf(&b, ": %s", e.Body)
if e.BodyReadErr != nil {
fmt.Fprintf(&b, " [response body read failed partway: %v]", e.BodyReadErr)
}
return b.String()
}
// Unwrap exposes a partial-read failure to errors.Is and errors.As, so a
// caller cancelling mid-read sees errors.Is(err, context.Canceled) rather
// than having to inspect BodyReadErr by name.
func (e *APIError) Unwrap() error { return e.BodyReadErr }
// Is lets errors.Is(err, ErrMaxTokensExceeded) match the budget refusal
// without the caller inspecting the status code or the body.
func (e *APIError) Is(target error) bool {
return target == ErrMaxTokensExceeded && e.ErrorType == "max_tokens_exceeded"
}
// typesafeProvider is the typesafe.ai Jev backend.
type typesafeProvider struct {
apiKey string
model string
endpoint string
client *http.Client
// sleep waits for d or until ctx is done, whichever comes first,
// returning ctx.Err() in the latter case. Tests replace it to record the
// requested durations without spending wall clock.
//
// It takes a context because a plain time.Sleep here made a cancelled
// Ask keep waiting out the full backoff — up to maxBackoff — before
// issuing a request that could only fail. Cancellation has to be able to
// interrupt the WAIT, not merely be noticed after it.
sleep func(ctx context.Context, d time.Duration) error
}
func newTypesafe(apiKey, model string) *typesafeProvider {
return &typesafeProvider{
apiKey: apiKey,
model: model,
endpoint: defaultTypesafeEndpoint,
client: &http.Client{Timeout: requestTimeout},
sleep: ctxSleep,
}
}
// ctxSleep waits for d, or returns early with ctx.Err() if the context is done
// first.
func ctxSleep(ctx context.Context, d time.Duration) error {
t := time.NewTimer(d)
defer t.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-t.C:
return nil
}
}
func (p *typesafeProvider) Name() string { return ProviderTypesafe }
func (p *typesafeProvider) Model() string { return p.model }
// setEndpoint points the provider at a different base URL. Intended for tests
// standing up an httptest server; production callers leave the default.
func (p *typesafeProvider) setEndpoint(u string) { p.endpoint = u }
// wireQuestion is one question as the provider expects it. Criteria is
// `any` because its shape depends on the kind: a map for choice and noul, an
// ordered array for score.
type wireQuestion struct {
Type string `json:"type"`
Instructions string `json:"instructions"`
Criteria any `json:"criteria,omitempty"`
}
type wireRequest struct {
Model string `json:"model"`
State any `json:"state"`
Questions map[string]wireQuestion `json:"questions"`
}
// wireAnswer is one answer as the provider returns it. The value members are
// pointers so an ABSENT member is distinguishable from a zero one: a noul
// answer carries no confidence, and a response missing its `noul` would
// otherwise decode as 0.0, a confident "false" the provider never said.
// checkAnswer refuses any answer missing a member its kind requires.
type wireAnswer struct {
Type string `json:"type"`
Choice *string `json:"choice"`
Score *float64 `json:"score"`
Noul *float64 `json:"noul"`
Legend map[string]string `json:"legend"`
Probabilities map[string]float64 `json:"probabilities"`
Confidence *float64 `json:"confidence"`
}
type wireUsage struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
}
type wireResponse struct {
Model string `json:"model"`
Answers map[string]wireAnswer `json:"answers"`
Usage wireUsage `json:"usage"`
}
// wireError is the provider's error envelope. detail is `any` because the
// provider returns an object for the cases measured here and the shape is
// undocumented; errorTypeOf digs the code out without depending on it.
type wireError struct {
Detail any `json:"detail"`
}
// Ask implements [Provider].
func (p *typesafeProvider) Ask(ctx context.Context, state any, questions map[string]Question) (map[string]Answer, Usage, error) {
if len(questions) == 0 {
return nil, Usage{}, errors.New("decision: no questions asked")
}
wire := make(map[string]wireQuestion, len(questions))
for key, q := range questions {
if strings.TrimSpace(key) == "" {
return nil, Usage{}, errors.New("decision: question key is empty")
}
if err := q.Validate(); err != nil {
return nil, Usage{}, fmt.Errorf("question %q: %w", key, err)
}
wire[key] = toWire(q)
}
state, truncated, err := p.fitState(state, wire)
if err != nil {
return nil, Usage{}, err
}
body, err := json.Marshal(wireRequest{Model: p.model, State: state, Questions: wire})
if err != nil {
return nil, Usage{}, fmt.Errorf("decision: marshal request: %w", err)
}
resp, err := p.post(ctx, body)
if err != nil {
return nil, Usage{}, err
}
// Every answer is checked against the question it claims to answer
// before any of it reaches the caller: a missing answer, a wrong kind, a
// missing member, a value outside what was asked, or a number outside
// its range is refused, and the whole call fails (all-or-nothing, per
// [Provider.Ask]). Each of those would otherwise decode into a plausible
// value the provider never meant — see checkAnswer.
//
// The result is built from the QUESTIONS, not from the response, so an
// unsolicited extra answer is dropped rather than returned: the contract
// is one entry per question key.
answers := make(map[string]Answer, len(questions))
for key, q := range questions {
wa, ok := resp.Answers[key]
if !ok {
return nil, Usage{}, fmt.Errorf("decision: provider returned no answer for question %q", key)
}
if err := checkAnswer(q, wa); err != nil {
return nil, Usage{}, fmt.Errorf("decision: question %q: %w", key, err)
}
answers[key] = fromWire(wa)
}
return answers, Usage{
InputTokens: resp.Usage.InputTokens,
OutputTokens: resp.Usage.OutputTokens,
StateTruncated: truncated,
}, nil
}
// toWire converts a question to its provider representation. Criteria shape
// is per kind: option->description for choice, an ordered array for score,
// and a true/false pair (omitted when both are empty) for noul.
func toWire(q Question) wireQuestion {
w := wireQuestion{Type: string(q.Kind), Instructions: q.Instructions}
switch q.Kind {
case KindChoice:
w.Criteria = q.Options
case KindScore:
w.Criteria = q.Levels
case KindNoul:
if q.TrueDesc != "" || q.FalseDesc != "" {
w.Criteria = map[string]string{"true": q.TrueDesc, "false": q.FalseDesc}
}
}
return w
}
// fromWire converts a provider answer that checkAnswer has accepted, keeping
// only the members the answered kind populates. Copying a stray Score onto a
// choice answer would hand a consumer a number the provider never meant.
func fromWire(w wireAnswer) Answer {
a := Answer{Kind: Kind(w.Type)}
switch a.Kind {
case KindChoice:
a.Choice = *w.Choice
a.Probabilities = w.Probabilities
a.Confidence = w.Confidence
case KindScore:
a.Score = *w.Score
a.Legend = w.Legend
a.Probabilities = w.Probabilities
a.Confidence = w.Confidence
case KindNoul:
a.Noul = *w.Noul
}
return a
}
// checkAnswer refuses an answer that contradicts the question it answers.
//
// The population is every (kind, member) cell of [Answer], enumerated in
// the table on TASK-3116's trail (review round 2). Per kind:
//
// - every kind: Type must equal the question's kind.
// - choice: choice PRESENT and one of the options sent; confidence PRESENT
// and in [0,1]; probabilities NON-EMPTY, keyed only by options sent,
// each in [0,1].
// - score: score PRESENT and in [0, levels-1] (it is the probability-
// weighted mean over level indices, so it cannot leave that range);
// confidence PRESENT and in [0,1]; probabilities NON-EMPTY; probability
// and legend keys each the canonical decimal index of a level sent
// ("0".."n-1", so "01" and "9" are refused); probabilities in [0,1].
// - noul: noul PRESENT and in [0,1]. Members noul does not use are
// ignored rather than refused — fromWire drops them, so they cannot
// reach a caller.
//
// What it does NOT check: that probabilities sum to 1 (the provider rounds,
// and nothing downstream depends on the sum), or that the legend's NAMES
// match the levels sent (they are echoed text; a mismatch misnames, it
// cannot mis-index).
func checkAnswer(q Question, w wireAnswer) error {
if Kind(w.Type) != q.Kind {
return fmt.Errorf("asked a %s but the provider answered with a %s", q.Kind, w.Type)
}
switch q.Kind {
case KindChoice:
if w.Choice == nil {
return errors.New("choice answer carries no choice")
}
if _, ok := q.Options[*w.Choice]; !ok {
return fmt.Errorf("provider chose %q, which is not one of the options sent", *w.Choice)
}
if err := checkConfidence(w.Confidence); err != nil {
return err
}
return checkProbabilities(w.Probabilities, func(k string) bool {
_, ok := q.Options[k]
return ok
})
case KindScore:
if w.Score == nil {
return errors.New("score answer carries no score")
}
if hi := float64(len(q.Levels) - 1); *w.Score < 0 || *w.Score > hi {
return fmt.Errorf("score %v is outside [0, %v] for %d levels", *w.Score, hi, len(q.Levels))
}
if err := checkConfidence(w.Confidence); err != nil {
return err
}
isLevel := func(k string) bool {
n, err := strconv.Atoi(k)
return err == nil && n >= 0 && n < len(q.Levels) && strconv.Itoa(n) == k
}
for k := range w.Legend {
if !isLevel(k) {
return fmt.Errorf("legend key %q is not the index of a level sent", k)
}
}
return checkProbabilities(w.Probabilities, isLevel)
case KindNoul:
if w.Noul == nil {
return errors.New("noul answer carries no noul value")
}
if !inUnit(*w.Noul) {
return fmt.Errorf("noul %v is outside [0, 1]", *w.Noul)
}
}
return nil
}
func checkConfidence(c *float64) error {
if c == nil {
return errors.New("answer carries no confidence")
}
if !inUnit(*c) {
return fmt.Errorf("confidence %v is outside [0, 1]", *c)
}
return nil
}
func checkProbabilities(p map[string]float64, validKey func(string) bool) error {
if len(p) == 0 {
return errors.New("answer carries no probabilities")
}
for k, v := range p {
if !validKey(k) {
return fmt.Errorf("probability key %q is not an outcome that was asked", k)
}
if !inUnit(v) {
return fmt.Errorf("probability %v for %q is outside [0, 1]", v, k)
}
}
return nil
}
func inUnit(f float64) bool { return f >= 0 && f <= 1 }
// fitState shortens a STRING state, if needed, so that the request's
// estimated size fits the measured token budget. It reports whether it
// truncated. The estimate is bytes over [charsPerToken], which is a
// conservative heuristic rather than a bound — see that constant — so a
// fitted request can still be refused by the provider, and that refusal is
// surfaced as [ErrMaxTokensExceeded].
//
// Only a STRING state is truncated. Cutting a marshalled struct or map at a
// byte boundary would produce invalid JSON, and silently dropping fields
// would change what the questions are being asked ABOUT; a caller with a
// structured state that does not fit gets the provider's own refusal, which
// names the problem correctly.
func (p *typesafeProvider) fitState(state any, questions map[string]wireQuestion) (any, bool, error) {
// Reserve exactly what everything but the state costs, by marshalling
// the request with an empty string state: model name, questions and
// punctuation, whatever their size. A fixed allowance here let a long
// model name push a cut-to-the-limit request over the budget.
envelope, err := json.Marshal(wireRequest{Model: p.model, State: "", Questions: questions})
if err != nil {
return nil, false, fmt.Errorf("decision: marshal questions: %w", err)
}
budgetChars := int(float64(maxRequestTokens) * charsPerToken)
// remaining is what the state VALUE may occupy, its quotes included —
// the envelope's own "" is subtracted back out because the state's
// marshalled length counts it again.
remaining := budgetChars - (len(envelope) - 2)
if remaining < 0 {
return nil, false, fmt.Errorf("decision: questions alone exceed the %d-token budget: %w", maxRequestTokens, ErrRequestTooLarge)
}
s, ok := state.(string)
if !ok || marshalledLen(s) <= remaining {
return state, false, nil
}
// A string state that does not fit, with fewer than two bytes of room:
// not even "" fits, and returning it would still send a request one or
// two bytes over (review round 3). This sits AFTER the string check on
// purpose — a structured state is never cut, and one as small as `0`
// fits in a single byte (review round 4).
if remaining < 2 {
return nil, false, fmt.Errorf("decision: no room for the state beside the questions: %d byte(s) left, "+
"and even an empty string needs 2: %w", remaining, ErrRequestTooLarge)
}
// The budget is measured against the MARSHALLED length, not the raw
// string length, because encoding/json escapes <, > and & to six-byte
// \uXXXX sequences: a state of 80,000 '<' characters is 80,000 raw bytes
// and 480,000 on the wire (measured ratio exactly 6.00). Sizing the cut
// on raw length let such a state sail past the budget check and come back
// as max_tokens_exceeded, which is the very refusal truncation exists to
// avoid.
//
// Expansion is not uniform, so one division cannot place the cut. Start
// from the raw budget — marshalled length is never less than raw length,
// so that is a valid upper bound — then shrink by the expansion ratio
// actually observed at the current cut until it fits. `cut` strictly
// decreases every iteration, so this terminates.
//
// The first cut leaves room for the two quote bytes the marshalled
// string adds, so a state with no escapes fits on the first try instead
// of overshooting by two and being scaled back by the 5% margin below.
cut := runeBoundaryAtOrBefore(s, min(remaining-2, len(s)))
for cut > 0 {
got := marshalledLen(s[:cut])
if got <= remaining {
return s[:cut], true, nil
}
// Scale by the observed ratio, with a margin so a pathological
// distribution still converges, and always make progress.
next := int(float64(cut) * (float64(remaining) / float64(got)) * 0.95)
if next >= cut {
next = cut - 1
}
cut = runeBoundaryAtOrBefore(s, next)
}
// Nothing of the state fits beside the questions.
return "", true, nil
}
// marshalledLen is the number of bytes this string occupies as a JSON value
// — the bytes actually sent, which are what the token estimate is taken over.
func marshalledLen(s string) int {
b, err := json.Marshal(s)
if err != nil {
// json.Marshal cannot fail for a string; fall back to a
// deliberately pessimistic bound rather than reporting a small one.
return len(s) * 6
}
return len(b)
}
// runeBoundaryAtOrBefore returns the largest index <= i that is not a UTF-8
// continuation byte, so a cut there never SPLITS a rune: valid UTF-8 input
// yields a valid prefix. It does not repair input that was already invalid —
// a byte like 0xff counts as a boundary — and nothing here does: json.Marshal
// replaces invalid bytes with U+FFFD whether or not the state was cut, and
// marshalledLen measures the replaced form, so the budget still holds.
func runeBoundaryAtOrBefore(s string, i int) int {
if i < 0 {
return 0
}
// len(s) is a valid cut point (the whole string) and indexing s[len(s)]
// would panic, so return early rather than clamping into the loop.
if i >= len(s) {
return len(s)
}
for i > 0 && !isRuneStart(s[i]) {
i--
}
return i
}
// isRuneStart reports whether b can begin a UTF-8 sequence — i.e. it is not a
// continuation byte.
func isRuneStart(b byte) bool { return b&0xC0 != 0x80 }
// post sends the request, retrying transient failures with exponential
// backoff, and returns the decoded response.
func (p *typesafeProvider) post(ctx context.Context, body []byte) (*wireResponse, error) {
backoff := initialBackoff
var lastErr error
for attempt := 0; attempt < maxAttempts; attempt++ {
if attempt > 0 {
wait := backoff
if ra := retryAfterOf(lastErr); ra > 0 {
wait = ra
}
// The wait itself is cancellable; a cancelled context must not
// have to outlast the backoff before anyone notices.
if err := p.sleep(ctx, wait); err != nil {
return nil, err
}
if backoff *= 2; backoff > maxBackoff {
backoff = maxBackoff
}
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, p.endpoint, bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("decision: create request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+p.apiKey)
resp, err := p.client.Do(req)
if err != nil {
// A transport error may be transient; keep retrying, but never
// past a cancelled context.
if ctx.Err() != nil {
return nil, ctx.Err()
}
lastErr = fmt.Errorf("decision: call provider: %w", err)
continue
}
raw, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
retryAfter := parseRetryAfter(resp.Header.Get("Retry-After"))
status := resp.StatusCode
resp.Body.Close()
switch {
case status == http.StatusTooManyRequests || status == 529:
// Retryable by the provider's own contract.
lastErr = &transientError{APIError: apiErrorFrom(status, raw, readErr), retryAfter: retryAfter}
continue
case status >= 500:
lastErr = &transientError{APIError: apiErrorFrom(status, raw, readErr)}
continue
case status >= 400:
// Not retryable: the request itself is the problem. The body is
// surfaced verbatim, and max_tokens_exceeded is matchable via
// errors.Is without parsing prose. A body read that failed
// partway is carried on the error rather than dropped, so a
// caller can tell a missing ErrorType from an unread one.
apiErr := apiErrorFrom(status, raw, readErr)
return nil, &apiErr
}
if readErr != nil {
return nil, fmt.Errorf("decision: read provider response: %w", readErr)
}
var out wireResponse
if err := json.Unmarshal(raw, &out); err != nil {
return nil, fmt.Errorf("decision: decode provider response: %w", err)
}
if len(out.Answers) == 0 {
return nil, errors.New("decision: provider returned no answers")
}
return &out, nil
}
if lastErr == nil {
lastErr = errors.New("decision: provider retries exhausted")
}
return nil, fmt.Errorf("decision: provider failed after %d attempts: %w", maxAttempts, lastErr)
}
// transientError wraps a retryable provider response so the retry loop can
// carry a Retry-After hint without widening APIError.
type transientError struct {
APIError
retryAfter time.Duration
}
// Unwrap exposes the embedded APIError to errors.As.
//
// Without it, exhausted retries produce an error chain of
// fmt.Errorf -> *transientError and stop: errors.As(err, **APIError) returns
// FALSE, because *transientError is a different type and value-embedding does
// not make it one. A caller that retried four times into a 429 wall would then
// have no way to read the status or the error type off the failure — the
// information is in the chain and unreachable.
func (t *transientError) Unwrap() error { return &t.APIError }
// apiErrorFrom builds an APIError from a response, keeping a partial-read
// failure rather than discarding it.
func apiErrorFrom(status int, raw []byte, readErr error) APIError {
return APIError{
StatusCode: status,
ErrorType: errorTypeOf(raw),
Body: boundedBody(raw),
BodyReadErr: readErr,
}
}
func retryAfterOf(err error) time.Duration {
var t *transientError
if errors.As(err, &t) {
return t.retryAfter
}
return 0
}
// parseRetryAfter reads a Retry-After header in delta-seconds form, bounded
// so a hostile or mistaken value cannot park a request indefinitely.
func parseRetryAfter(v string) time.Duration {
v = strings.TrimSpace(v)
if v == "" {
return 0
}
secs, err := strconv.Atoi(v)
if err != nil || secs <= 0 {
return 0
}
d := time.Duration(secs) * time.Second
if d > maxRetryAfter {
return maxRetryAfter
}
return d
}
// errorTypeOf extracts detail.error_type from an error body, returning "" if
// the body is not that shape. It tolerates detail being a string or absent,
// since only the object form is measured and the rest is undocumented.
func errorTypeOf(raw []byte) string {
var we wireError
if err := json.Unmarshal(raw, &we); err != nil {
return ""
}
m, ok := we.Detail.(map[string]any)
if !ok {
return ""
}
s, _ := m["error_type"].(string)
return s
}
func boundedBody(raw []byte) string {
s := strings.TrimSpace(string(raw))
if len(s) > maxErrBodyLength {
return s[:maxErrBodyLength] + "...(truncated)"
}
return s
}
// levelIndex parses a score probability/legend key ("0", "1", ...) into its
// integer index. A key that is not an integer sorts last rather than
// panicking, so an unexpected vocabulary degrades instead of crashing.
func levelIndex(k string) int {
n, err := strconv.Atoi(k)
if err != nil {
return 1 << 30
}
return n
}