pad/cmd at cdbc69dd0e3836620d55fc01ff9584d0819ef662 - pad

mirror of https://github.com/PerpetualSoftware/pad.git synced 2026-10-25 22:10:25 +00:00

Files

T

xarmian 096584c349 feat(decision): pluggable typed-decision provider with a typesafe.ai backend (TASK-3116) (#1411 )

* feat(decision): pluggable typed-decision provider with a typesafe.ai backend

Unit 1 of PLAN-3114 (TASK-3116). Foundation only — no product behaviour, no
storage, no runner, no call sites.

internal/decision exposes three primitives (Choice, Score, Noul) answered in
one round trip, a pinned-model typesafe.ai backend, and one configuration
resolver. Unconfigured returns a nil Provider and no error, so every later
caller branches on nil and behaves exactly as today.

Three wire-shape facts are pinned to measurements rather than to the vendor
docs, which are silent or wrong on each. Receipts are on TASK-3116's trail.

- A Noul answer carries NO confidence and NO probabilities. Confidence is
therefore a *float64: 0.0 is not a neutral stand-in for absent, it is the
most extreme value the scale has, and a caller following the vendor's own
recommended banding would read every Noul answer as minimum confidence and
route all of them to a human.

- Score is the probability-weighted MEAN over level indices, not a level
index. Measured: probabilities {"0":0.86,"1":0.14} returns score 0.14.
Answer.TopLevel() is the correct way to ask which level was picked;
int-casting Score gives the wrong level for any distribution whose mass
does not sit on index 0. Score's request criteria is an ordered ARRAY and
its probability keys are stringified level INDICES, where Choice's are
option NAMES — same field, two vocabularies, decided by Kind.

- An over-budget request is refused with HTTP 400 and
{"detail":{"error_type":"max_tokens_exceeded"}}, not the 422 the task
contract predicted. It is matchable with errors.Is(err,
ErrMaxTokensExceeded) rather than by matching prose. 422 keeps its own
passthrough, with the body verbatim because the vendor's validation
messages name the offending question.

The token budget is measured, not cited: bisecting on the API's own
usage.input_tokens, 32,476 tokens was accepted and ~33,600 refused, and the
budget covers state AND questions together. maxRequestTokens is 32,000, under
the measured accepted figure. Truncation is a conservative char-based bound
and says so — it cannot be a guarantee for pathological input, which is why
the provider's own refusal is surfaced as a typed error instead of swallowed.

Config resolution has exactly ONE reader so that TASK-3121's instance-admin
setting adds a SOURCE rather than a second reader. The environment is passed
last because it overrides the admin setting (ruled day 73). That is why
ProviderNone exists: with an empty value meaning "this source has no
opinion", an environment that wants to disable what an admin enabled must say
so with a value. internal/config carries the config-FILE fields only and
documents why its env block deliberately does not read these three variables.

`pad server info` reports config.decision_provider as {name, model}, always an
object with name "none" when unconfigured rather than alternating between an
object and a bare string. It sits under `config` because that is what it
describes — this host's resolved settings; that command does not inspect
remote server internals, so against a remote server it is the CLI host's view.

Claude-Session: https://claude.ai/code/session_01BLg7wiBeSgXvAhWi23mT4Z

* test(decision): sweep the truncation cut offset; pin the measured constants; add a live-provider leg

Three additions, each closing a hole an instrument found rather than one I
reasoned my way to.

The rune-boundary assertion moves out of the truncation test and into
TestFitStateCutsOnRuneBoundaries, which sweeps the cut OFFSET by varying the
questions' size. A mutant deleting the rune-boundary loop entirely SURVIVED the
original single-offset assertion: that one offset happened to land on a
boundary, so the green could not go red. Eight offsets over two-byte runes now
land mid-rune several times and the mutant dies.

TestBudgetConstantsStayInsideTheMeasurement guards maxRequestTokens and
charsPerToken, whose correctness is a fact about the live provider that no
behavioural test can check. Without it, raising either to "use more of the
window" gives a green suite and a production 400.

TestLiveProviderAllThreePrimitives calls the real API, skipped unless
DECISION_LIVE=1, so CI and `make test` never reach it. It exists because the
httptest fake cannot answer its question: the fake replays bodies I wrote and
asserts shapes I chose, so an encoder and a fake sharing the same wrong field
name agree with each other and both pass. Run live, it confirms choice
probabilities are keyed by option name, score's legend echoes the levels sent,
score really is the probability-weighted mean of the level indices (asserted
against the live numbers, not a recording), usage.input_tokens decodes, and —
the assertion the whole pointer design rests on — a noul answer carries NO
confidence on the live wire.

Mutation matrix, after re-expressing the two mutants that did not compile
(a BUILD-FAIL is not a kill): 21 mutants, 21 DETECTED, each by its own NAMED
expected test, against a baseline stated green first. Harness restores by file
copy and verifies the restore before the next mutant.

Claude-Session: https://claude.ai/code/session_01BLg7wiBeSgXvAhWi23mT4Z

* wip(decision): round-1 review fixes — six findings fixed, matrix not yet re-run

WIP stand-down commit (Dave, day 73: account at its usage limit). Not pushed,
no PR. Gates and the mutation matrix have NOT been re-run against these fixes.

Six P2 findings from review round 1 (backend: opencode/gpt-5.6-luna), each
confirmed mechanically before accepting:

1. The backoff ignored context cancellation, so a cancelled Ask waited out the
full wait before a doomed retry. The sleep hook now takes a context and
returns its error; production installs ctxSleep (timer vs ctx.Done()).
2. A body read that failed partway was discarded, so a caller could not tell an
absent detail.error_type from an unread one. APIError.BodyReadErr keeps it.
3. Exhausted retries wrapped a *transientError, which is not an *APIError, so
errors.As could not reach the status. Verified false in a standalone
program; transientError now has Unwrap.
4. Truncation was sized on RAW string length while encoding/json escapes <, >
and & to six-byte sequences — measured expansion exactly 6.00x — so a state
of '<' passed the budget check and then drew max_tokens_exceeded. The cut is
now sized on marshalled length, shrinking by the observed expansion ratio.
5. An answer whose kind differed from its question's was accepted, and fromWire
then left the requested kind's member at a plausible zero: a noul question
answered with a score read as Noul == 0.0, a confident "false" the provider
never said. Ask now refuses a kind mismatch.
6. TopLevel ordered only on the parsed index, so distinct keys sharing an index
("1" and "01") or several unparseable keys sharing the sentinel were
tie-broken by randomised map iteration order. The sort is now a total order.

Two things found while fixing, not by the reviewer:
- runeBoundaryAtOrBefore indexed s[len(s)] and would panic; it returns early now.
- The first cancellation test never reached the backoff it claimed to test:
cancelling during the response makes http.Client.Do fail, so Ask returned
from the transport branch having never slept. Split into two instruments —
one pinning that Ask honours an interrupted wait, one pinning that the wait
installed in production is interruptible.

State: internal/decision tests pass (go test exit 0, run before the stand-down).
NOT done: re-running make test / lint / test-pg, and the 29-mutant matrix, which
was killed mid-run. It had mutated typesafe.go in place; the tree was restored
from the harness backup and all eight fix sites verified present before this
commit.

Claude-Session: https://claude.ai/code/session_01BLg7wiBeSgXvAhWi23mT4Z

* test(decision): make two mutants die by their own tests

M17 (missing-answer check removed) survived: the answer-kind check also
refuses a missing answer, since it decodes with an empty Kind, and also
names the key — so the test could not tell which check fired. It now pins
the "no answer" diagnosis, which is the difference a caller sees.

M23 (uncancellable ctxSleep) was only caught by go test's global timeout,
which panics the whole binary after 90s. The newTypesafe leg now runs
under its own 2s deadline and fails as a named test.