Files
pad/scripts
xarmian e598bc4f64 feat(decision): effort, effort_medium and high_stakes on the attention set, staged; drain-paced backfill (TASK-3921) (#2236)
* feat(decision): effort, effort_medium and high_stakes on the attention set, staged; one-shot backfill (TASK-3921)

- attention set gains effort (Score), effort_medium (Choice) and high_stakes
  (Noul) in the same call; needs_human_decision's text is unchanged.
- QuestionSet.Unsurfaced names the staged keys: WorkspaceFlags and
  SurfacedDecisions skip them (no flag, no state read, no counter); the
  decisions list and endpoint still return every row.
- attention chips only render keys that have a label.
- store: per-set enqueue body extracted (owedDecisionJobTx), OweDecisionJobs
  (chunked, under the installed debounce) and ItemsMissingDecisionKey.
- Runner.BackfillEffort owes the attention set once for eligible items with a
  needs_human_decision answer and no effort answer, marker
  effort_decisions_backfilled written after every chunk; hooked at the end of
  reconfigureDecisionsLocked (boot and later admin enable); makes no provider call.
- eval harness: titlebody now sends the live set; RESULTS.md rows pending.
- docs/api.md and tests updated.

Claude-Session: https://claude.ai/code/session_018oQ5EErQKHtBJK8yLpcDNF

* test(decision-eval): keep titlebody as the day-73 control, add titlebody-live; record the TASK-3957 and TASK-3921 runs

- titlebody again sends the day-73 pair (needs_human_decision + blocked), so it
  stays comparable with its 09-22 row as the model-drift control.
- titlebody-live sends the live attention set on the same {title, body} state:
  the TASK-3921 gate (needs_human_decision AUC 0.936, within .02 of 0.939).
- Every run prints the question keys it sends, before -dry too, so a row's
  call shape is in its own log.
- RESULTS.md: rows for the one-question call (TASK-3957) and the
  four-question call (TASK-3921).

Claude-Session: https://claude.ai/code/session_018oQ5EErQKHtBJK8yLpcDNF

* fix(decision): effort backfill lifecycle, staged keys off item reads, dry-run guard (TASK-3921; #r1 F1-F4)

- F1: startEffortBackfill checks stopped and does bg.Add under one
  decisionTick.mu hold; BackfillEffort/EffortBackfillCandidates take a ctx
  checked before each candidate read and each chunk; cancel returns ctx.Err()
  and never writes the marker. stopDecisionTick cancels the run.
- F2: reconfigure cancels a running backfill; only a non-nil runner starts a
  new one. A per-run generation replaces the atomic latch so a cancelled run
  cannot clobber its successor.
- F3: item show / --agent / MCP item get use Runner.ItemReadDecisions, which
  drops the set's Unsurfaced keys; the decisions endpoint stays raw. docs/api.md
  updated.
- F4: the file dry run refuses a path with a -wal/-shm sibling or under
  $HOME/.pad (refuseLiveDatabase, tested without the env var).

Claude-Session: https://claude.ai/code/session_018oQ5EErQKHtBJK8yLpcDNF

* test(decision): a chunk write that fails in the store writes no marker (TASK-3921; mutant M6)

The two failure tests fail the per-chunk test hook, so the path where
OweDecisionJobs itself returns an error was never exercised: a mutant that
ignores that error and marks anyway survived. The new test breaks the
decision_jobs table from the hook so the error comes back through the store,
asserts no marker, then restores the table and re-runs to completion.

Claude-Session: https://claude.ai/code/session_018oQ5EErQKHtBJK8yLpcDNF

* refactor(decision): drop the pre-marker cancel check; a cancel after the last chunk may mark (TASK-3921; M5, lead ruling)

Mutant M5 (that check removed) survived every test, and could not do
otherwise: a cancel can land there only after every chunk was owed, and the
marker means every candidate was ENQUEUED, not answered, so marking then is
true. The check is removed rather than kept as a guard no instrument can
turn red. The doc comments on BackfillEffort, cancelEffortBackfill and
startEffortBackfill now say a cancel before the last chunk writes no
marker and one after it may.

Claude-Session: https://claude.ai/code/session_018oQ5EErQKHtBJK8yLpcDNF

* docs(api): a backfill chunk in flight when decisions are disabled may still commit (TASK-3921; r2 L1, lead ruling bdce2116)

docs/api.md said that with no provider "no job row is ever written". That is
false for one window: a disable that lands while the one-shot effort
backfill is inside a chunk's write lets that chunk commit, and its jobs wait
until a provider returns. Ruled docs-only. The disable test's comment now
says it covers a backfill parked before its first chunk, not that window.

Sweep for the claim (CONVE-23): grep of docs, internal, cmd, scripts and
skills (.go, .md) for "no job row", "job row is", "no job is", "no jobs
are written/owed/enqueued" and "never written/owed/enqueued". Two hits are
about this mechanism: api.md:72 (fixed) and the test comment (narrowed).
Unchanged, still true:
- store/decisions.go:27 is about the WRITE DOORS, which still enqueue
  nothing without a resolver.
- runner_test.go:238 is about the nil-provider runner, which never starts a
  backfill.
- effort_backfill_lifecycle_test.go:128 covers a run parked before its
  first chunk.
The other hits are unrelated mechanisms.

Claude-Session: https://claude.ai/code/session_018oQ5EErQKHtBJK8yLpcDNF

* fix(decision): pace the effort backfill on drain; dry run opens a copy; docs eligibility (TASK-3921; r3 M1, L2, L4)

- M1: the backfill owes one tick pass of items per chunk (20 default, wired
  from the tick limit) and owes chunk k+1 only once chunk k's own jobs have
  drained: none claimable now, by ClaimDecisionJobs' own predicate
  (new Store.CountClaimableDecisionJobs). Debounced jobs do not hold it.
  The wait polls at the tick interval, honours ctx, and a cancel writes
  no marker; the marker still follows the last chunk being owed.
- L2: TestEffortBackfillDryRunOnFile copies the named file under GOTMPDIR
  and opens only the copy.
- L4: docs/api.md states the real eligibility and the drain pacing.

Claude-Session: https://claude.ai/code/session_018oQ5EErQKHtBJK8yLpcDNF

* docs(decision): the Unsurfaced comment names ItemReadDecisions (TASK-3921; r4 L2)

The field's comment listed WorkspaceFlags and SurfacedDecisions as the
readers that skip staged keys, but not ItemReadDecisions, which item show
(and so --agent and MCP item get) has used since the r1 F3 fix. Comment only.

Claude-Session: https://claude.ai/code/session_018oQ5EErQKHtBJK8yLpcDNF

* test(decision): break the table before the LAST chunk so the store-failure test pins M6 (TASK-3921; lead ruling)

Since the drain pacing (51c545850), the store-failure test broke the
decision_jobs table before chunk 1 of 3. Under mutant M6 (a failed chunk
write ignored) the drain wait that follows read the same broken table and
failed, so no marker was written by a different mechanism and the test stayed
green: an end-state assertion another path also produced (CONVE-12). Breaking
the table before the last chunk leaves no drain wait after it (the marker does
not wait for the last chunk to drain), so only the chunk's own error check can
keep the marker unwritten.

Claude-Session: https://claude.ai/code/session_018oQ5EErQKHtBJK8yLpcDNF
2026-10-19 13:05:05 -04:00
..