Files
pad/internal/store/items.go
T
xarmian 6a37512227 feat(server): outbox drain — webhooks delivered from the choke point (TASK-2714) (#1173)
* test(store): pin the events/1 taxonomy as an independent copy (TASK-2714)

TestCanonicalEventsAreFullyDeclared iterated kernelevents.Canonical() and
asserted each entry resolved something non-empty. That check cannot fail for
any table the compiler accepts: eventSpec requires both fields, so a corrupted
table — an entry deleted, an entry added, item.deleted quietly rebased onto the
ref-only payload — passed its own validation. A test that agrees with whatever
the table says is not a test of the table.

The sixteen name/subject/family triples are now written out as literals, so the
test DISAGREES with the table when the table moves. The wire strings behind the
name constants are pinned separately, because the triple map is keyed on
literals and a renamed constant would otherwise slip through as long as the
constant and the table moved together.

Ordered as this unit's first commit because TASK-2714 edits that table (the
handler-path bulk mapping): an independent copy earns its keep at the moment of
the edit, not before.

Mutation matrix, 4/4 caught: drop member.joined (17 -> 15 count mismatch and a
missing-name error), rehome item.deleted onto ref_only (family mismatch),
rename ItemMoved's wire string to item.move (constant leg), add an undeclared
item.frobnicated entry (count + undeclared-name + non-canonical legs). The
fourth reported "survived" on its first run because the sed never matched the
table's alignment — the mutation was verified present in the file before the
result was believed.

TASK-2714 requirement 4 (lead pass on #1172).

* feat(store): max-age prune for undispatched outbox rows (TASK-2714)

Requirement 3's missing half. PruneDispatchedOutbox filters on dispatched_at
IS NOT NULL, so a row that can never be delivered — a workspace whose only
webhook was deleted, an endpoint that 4xxs forever — is unreachable by it and
keeps its frozen payload indefinitely.

That matters because SPEC-3 makes payload privacy TEMPORAL. An outbox payload
is a frozen snapshot and account deletion's de-identify posture reaches only
live rows, so the retention window is the whole privacy claim; a window only
one of its two halves can close is not a window.

The trade is stated in the doc comment rather than left to be inferred:
at-least-once holds WITHIN the retention window and not past it, which is why
the caller's max-age must be far larger than any retry schedule. Deleting
rather than stamping the rows dispatched is deliberate — a dispatched stamp
would be a lie in the durable record, and this table is the only evidence of
what the kernel emitted.

Mutation matrix, 2/2 caught: drop the dispatched_at IS NULL clause (prunes the
aged DISPATCHED row too, handing retention two owners with different windows),
drop the occurred_at cutoff (prunes a young pending row a retry is still
owed). The test asserts its own premise — all three seeded rows are confirmed
present before the survivor checks, which would otherwise pass for a reason
unrelated to the prune.

No caller yet: the drain loop wires it up in the next commit.

* feat(events): derive SSE names from the taxonomy; retire item.updated_with_comment (TASK-2714)

SPEC-3 §"the choke point owns the canonical→surface name mapping". SSE's
snake_case vocabulary and the webhook dot-form vocabulary drifted because
nothing tied them together — each was hand-passed at its own call sites. This
ties them.

v1.5 pins what "derive" means: NAME derivation, not delivery path. SSE stays
direct-published at the mutation site, because it carries request-scoped
attribution (Actor / ActorName / Source) that a frozen outbox payload
deliberately does not hold; only its NAME now comes from the taxonomy. Moving
SSE behind the drain is TASK-2722.

- eventSpec gains an `sse` field — ONE table, not a second map, for the reason
  round 11 of the last unit established: a separate map can disagree with the
  first and fails open exactly when it matters. Empty is a real value (attachment,
  member and pack events have no SSE surface) and SurfaceSSE reports false for it,
  so silence can't be mistaken for a name.
- Several canonical events derive the SAME SSE name — status_changed and moved
  both surface as item_updated — because the SSE vocabulary is coarser than
  events/1 and the UI never distinguished them. The finer name is what the
  webhook wire and bindings get.
- The 12 canonical SSE publish sites take their names from derived package vars,
  resolved AT INIT. Every call site is a compile-time constant, so a missing
  surface is a startup panic rather than a per-request decision between "log and
  drop" and "publish under an empty name".
- handlers_item_links.go keeps the events.ItemUpdated literal, commented: link
  mutations are silent in events/1 (v1.5), so there is no canonical name to
  derive from. TASK-2723 carries link.created / link.removed.
- item.updated_with_comment retired (v1.2, Dave's ruling). One producer deleted;
  the events.ItemUpdatedWithComment constant deleted with it — it had no producer
  and no web consumer (grepped .go/.ts/.svelte), so leaving it would leave a name
  a future publisher could reach for.

The compat guard is what makes this a refactor rather than a wire change:
TestDerivedSSENamesMatchTheLegacyWireVocabulary asserts each derived name equals
the events.* constant clients are pinned to. A derivation producing
"item.created" or "item_deleted" would break the live UI while every other Go
test still passed.

Mutation matrix, 3/3 caught: rename item.deleted's SSE surface to item_deleted
(both the taxonomy test and the compat guard fail), split item.moved onto its own
SSE name (same), make SurfaceSSE return (spec.sse, ok) so no-surface events fail
open (the taxonomy test's silence leg names all four). Running total 9/9.

go test ./internal/server ./internal/store ./internal/events: all green.

* feat(webhooks): synchronous DeliverEvent seam with per-endpoint outcome (TASK-2714)

Requirements 1 and 2. Dispatch returns once its per-hook goroutines are
spawned and reports nothing, so a drain built on it would stamp rows
dispatched while the HTTP requests were still in flight — losing exactly the
events the outbox exists to make unlosable. DeliverEvent blocks and tallies.

- Delivery carries WorkspaceID / EventID / Event / OccurredAt / Payload.
  OccurredAt is the EVENT's timestamp, not dispatch time: SPEC-3 pins
  time-relative binding predicates to it, so stamping time.Now() would make
  every consumer's notion of when a mutation happened depend on how backed up
  the queue was. Payload is json.RawMessage — []byte would base64 the snapshot
  into a string that is valid JSON and completely unusable.
- WebhookPayload gains ID, the consumer dedupe key SPEC-3 §Delivery guarantees
  already told consumers to use. Before this, that instruction named a field
  nobody could see. omitempty, because the "webhook.test" ping is not a kernel
  event, has no outbox row, and must not invent an id.
- DeliveryOutcome counts rather than a status, because one event fans out to N
  endpoints and the answers differ. Three distinctions the drain branches on:
  Matched==0 is SUCCESS (a webhook-less workspace is owed nothing; reading it
  as undelivered would back up every event in every such workspace until
  retention deleted it); Permanent does not hold the event pending (re-sending
  to an endpoint that will reject it again costs the queue its progress);
  Transient does. Retryable() states the ack rule once instead of letting each
  caller re-derive it.
- A returned error is reserved for the SERVER's failures — listing hooks,
  marshalling. Those must not ack: nothing was attempted, so the event is
  still owed in full.
- Dispatch keeps its async shape for its one remaining caller and says so.
  deliver() now returns the outcome it always computed; the async path
  discards it.

Mutation matrix, 6/6 caught: stamp dispatch time instead of occurred_at; drop
the envelope id; pass the payload as []byte (base64); deliver asynchronously
and assume success (the synchronous leg names it exactly); count a permanent
rejection as transient; swallow a store failure into a zero outcome (the test
prints the outcome that would have acked an undelivered event).

Running total 15/15. go test ./internal/webhooks green.

* feat(store): batch_id correlation for handler-path bulk mutations (TASK-2714)

F2's write half. A lane-wide bulk action is a handler LOOP over per-item store
mutations with no enclosing transaction, so each member writes its own
canonical outbox row — which is what keeps SPEC-3's per-member binding
evaluation free, and also means that without a marker the drain would put 200
item.deleted events on the webhook wire for a 200-item lane archive: exactly
the flood TASK-1668's batch event exists to prevent.

RECORDED, NEVER INFERRED (SPEC-3 v1.5). The schema-free alternative was
grouping pending rows by workspace and a time window, which would fold two
unrelated single updates into somebody's bulk event whenever they landed in
the same tick. A wire event saying "these five items changed together" is only
true if something recorded that they did.

- migrations 082 / pgmigrations 060: nullable event_outbox.batch_id, no FK
  (a batch is not a row anywhere, it is a name the handler minted), plus a
  partial index on the pending set.
- store.MutationOption / WithEventBatch: variadic, because every existing call
  site is a single-item mutation with nothing to declare and making all of them
  pass a zero value would bury the one case that matters.
- The handler mints one id per bulk OPERATION, before the loop and
  unconditionally — deciding mid-loop whether a run "counts as" a batch would
  make the correlation depend on how far the loop got.

POPULATION CORRECTED: my escalation said four store methods; it is FIVE.
archive (DeleteItem), restore (RestoreItem), move (MoveItemWithPreCheck), field
update (UpdateItemWithPreCheck) and assign (UpdateItem) are the complete set of
mutating store calls handlers_items_bulk.go makes — restore was the one I
missed, which is CONVE-18's exact lesson arriving one level up. The test drives
all five rather than sampling, because the failure is per-method: a signature
that accepts the option and never threads it compiles, passes everything else,
and silently un-batches one of the six bulk verbs.

Mutation matrix, 5/5 caught across the four distinct emit sites: drop the stamp
on the update path (both Update legs fail), on delete, on restore, on move. The
delete mutation first read as SURVIVED — it had made the package fail to BUILD
(opt then unused), and the grep for test-level FAIL lines printed nothing. The
compiler catch is the stronger result, but the instrument mis-reported it, so
it was re-run with opt kept alive and the test named it directly.

go test ./internal/store ./internal/server green.

* test(server): anchor the SSE compat guard to the client's literal strings (TASK-2714)

The guard compared the derivation against events.* — the Go side. A
coordinated rename of the taxonomy AND the constants passes that, and is
exactly the change that breaks the browser: the client is pinned to the
STRINGS, in web/src/lib/services/sse.svelte.ts's ITEM_EVENTS.

The wanted column is now a literal copy of what the client listens for, with
the file named. events.* is asserted alongside as a second leg, so a drift
between the Go constants and the client is attributed rather than merely
reported. Same disagree-with-the-table principle as the taxonomy test, one
layer out: this file has to be edited by hand when the wire vocabulary
intentionally changes, and that edit is when someone goes and changes the
client too.

Mutation matrix, 2/2, each hitting only its own leg: rename events.ItemCreated
to the dot-form with the taxonomy untouched (drift leg fires), and make the
taxonomy publish the dot-form on SSE (browser leg fires). Running total 22/22.

Lead's catch on the day-49 review of commit 33662da0.

* feat(store): outbox claim protocol with lease and whole-batch claiming (TASK-2714)

F3. Every instance of a cloud deployment runs the drain, so an unclaimed
pending row is delivered once PER INSTANCE by construction. SPEC-3 permits
duplicates — consumers dedupe on the event id — but "occasionally, after a
crash" and "always, once per instance" are different promises, and only the
first is one a consumer can budget for.

- migrations 083 / pgmigrations 061: claimed_at + claimed_by, dialect-uniform
  conditional UPDATE (BUG-2415's orphan-GC protocol). Postgres FOR UPDATE SKIP
  LOCKED plus a separate SQLite path would be two implementations of one
  behaviour, only one of which runs where it matters.
- claimed_at doubles as the lease: an instance that dies between claiming and
  dispatching must not strand its rows, and at-least-once is exactly what makes
  re-claiming safe.
- BATCHES ARE CLAIMED WHOLE, past the limit. The limit is a throughput knob;
  letting it split a batch would make one bulk operation arrive as two wire
  events each reporting a partial member count.
- MarkOutboxAttemptFailed RELEASES the claim rather than letting it expire. A
  transient failure means the event is owed and nothing is in flight; on a
  single-instance deployment the lease would otherwise be the only reason a
  retry ever waited.

THE EXCLUSIVITY TEST WAS VACUOUS AND THE MATRIX CAUGHT IT. Removing the
availability predicate from the claim UPDATE left it green: the candidate query
already filters held rows, so single-threaded the end state is identical
(CONVE-12 — another mechanism produces it). That implementation double-claims
every row two instances select in the same moment, which is the entire bug.
claimOutboxIDs is now split out so a test can drive the arbiter with a
deliberately STALE candidate list, and the same mutation fails it by name.

Mutation matrix, 4/4: drop the UPDATE's availability predicate (survived the
first test, named by the race test); drop the batch expansion; keep the claim
on a failed attempt; and the vacuity finding above. Running total 26/26.

go test ./internal/store green.

* feat(server): the outbox drain — claim, fold, deliver, retain (TASK-2714)

The half of SPEC-3's choke point that turns stored events into delivered ones.
2a built the fill side; until this, the table filled and nothing read it.

NOT STARTED YET, deliberately: the hand-called dispatchWebhook sites are still
in place, so wiring the loop here would double-deliver every canonical event.
Starting it is the next commit, together with deleting them — the unit's
behaviour edge, kept as one reviewable diff.

- Two declared payload shapes for item.bulk_updated (SPEC-3 v1.6). The
  store-side single-tx producers know every member at write time and embed
  snapshots; the handler-path HEADER knows the operation, the shared delta and
  the member refs, with snapshots living on the members' own rows. Declared
  rather than loosened: stuffing placeholder snapshots to satisfy a
  single-shape check would be a lie in the durable record, and dropping the
  gate would drop it on the one event with two producers.
- EmitBulkHeaderEvent + bulkEventDelta: the delta is captured where it is
  KNOWN. By the time the drain sees member rows they carry post-mutation
  snapshots, and a diff of a snapshot against nothing is not a delta.
- The fold: header plus whatever member rows of that batch are still
  undispatched. Members whose header is not in this claim deliver
  individually — not a fallback, the defined behaviour for the window between
  the loop committing and the header landing. batch_id is on the wire so a
  consumer can tie the singles to the batch.
- Per-unit acking: a folded batch is many rows and ONE delivery, so a
  partially acked batch would re-deliver.
- Retention runs every tick, both halves. The undispatched one is the privacy
  bound; PruneDispatchedOutbox looks like it covers retention until you notice
  which rows it can never see.

TWO REAL BUGS THE TESTS FOUND, both in this commit's own code:

1. DEFAULTS APPLIED ONLY IN StartOutboxDrain. A tick reached directly ran with
   a ZERO undispatched max age, making the retention cutoff `now` and deleting
   the entire pending set on its first pass. Every test does this, and so
   would any future admin-triggered drain. Fixed by construction — one
   resolver both entry points call — with a refusal guard behind it.
2. THE GUARD'S FIRST TEST WAS VACUOUS AND PASSED WITH THE GUARD REMOVED.
   RFC3339 is second-granular, so a row written in the same second as a
   zero-window cutoff survives `occurred_at < cutoff` either way: the end
   state was reachable by another mechanism, and that mechanism was the clock.
   runOutboxRetention now returns its refusal so the test asserts the refusal
   rather than the survival, plus a positive control.

Mutation matrix, 7/7 after the instrument fix: ack regardless of outcome; ack
only when something succeeded (permanent failures would wedge the queue); fold
without acking its members; drop members that have no header instead of
delivering them; remove the retention guard; remove the resolver's max-age
default. Two mutations initially read as survivors — one had failed to build,
one met the vacuous test — and both are recorded above rather than counted as
passes. Running total 33/33.

go test ./internal/server ./internal/store green.

* feat(server): deliver canonical webhooks from the drain, not from the handlers (TASK-2714)

The unit's behaviour edge, kept as its own commit. The drain starts, and the
nine remaining hand-called dispatchWebhook sites go: comment.created,
comment.updated, item.created (x2 — plain and copy), item.updated,
item.deleted (x2), item.moved, item.bulk_updated. Each was verified to have an
outbox producer before its deletion, not assumed to.

The Server.dispatchWebhook helper goes with them — it had no production
callers left. Three copy tests used it as a probe and now call
s.webhooks.Dispatch directly, which is what it did.

WHAT CHANGES ON THE WIRE, stated plainly because "no behaviour change" would
be false here:

- TIMING. Deliveries were post-commit and inline; they are now up to one drain
  interval (5s default) later. In exchange a delivery survives a crash: the
  event is committed with the mutation it describes.
- THE DISJOINT-DELTA RULE ARRIVES (SPEC-3 v1.3, ruled in 2a). A bare status
  flip now emits item.status_changed ONLY, where the hand-call always emitted
  item.updated. A mixed update emits both. This was ruled while the webhook
  surface has no known consumers; it is the same grounding as the v1.2 fold.
- PAYLOADS. The envelope gains `id` (the dedupe key SPEC-3 already told
  consumers to use), and `timestamp` is now the event's occurred_at rather than
  dispatch time. Item snapshots come from the in-transaction read-back and are
  PII-scrubbed — the joined assignee name and email are gone, deliberately
  (see scrubItemPII: a frozen payload outlives account de-identification).
- item.bulk_updated carries batch_id, the shared delta, and the member
  snapshots folded in from the member rows.

Two copy tests needed real changes, not cosmetic ones: the DR-14 emission
matrix they assert (which workspace hears what) is unchanged, but nothing
arrives until a drain pass runs, and the fixture's own backlog — member joins,
filler items — would otherwise be reported as the copy's output. The observer
now drains once before the receivers are registered, which is what its
"baseline" has always meant, and drainWebhooks runs a pass before collecting.

go test ./internal/server ./internal/store ./internal/webhooks green.

* fix: codex round 1 — unbatched bulk verbs, member dedup, comment overclaims (TASK-2714)

THE P1, and it is CONVE-18 for the third time in this unit: batchID was
threaded into the bulk helpers' SIGNATURES but not passed at three of the six
store CALLS (set-priority/move-status via UpdateItemWithPreCheck, tag/untag via
UpdateItem, move via MoveItemWithPreCheck). Those verbs' member rows stayed
unbatched while the header was still written — N individual wire deliveries
plus a header claiming they were a batch.

My store-level test could not see it. It called the five store methods directly
with the option, so it proved the option WORKS and said nothing about whether
the handler passes it. TestBulkItems_EveryVerbStampsOneBatchID drives all seven
legs through the HTTP handler and asserts every row of the operation shares one
non-empty batch id with exactly one header. Reverting one stamp fails it by
name (set-priority and move-status both).

Writing that test also surfaced two legs that asserted nothing: untag and
assign were no-ops in the fixture (no such tag; nothing assigned), so no member
events existed at all. Both now perform real mutations, and the leg fails if
fewer than two rows appear.

Also from round 1:

- FOLD DEDUPS MEMBERS. The disjoint-delta rule means one member can write two
  or three rows (a move that also changes status emits item.moved AND
  item.status_changed), so the folded payload listed the same item repeatedly
  while `count` reported ITEMS — the wire event contradicting itself. Keeps the
  LAST snapshot per id; an unreadable snapshot is kept rather than dropped.
- BULK MOVE DELTA carries both collection and status when both were sent.
  bulkMoveCollection applies req.Status as a field override, so a
  move-with-status changes two things.
- FIVE COMMENT OVERCLAIMS, all mine or inherited and all now matching the code:
  the taxonomy package doc still said nothing drains the outbox; "every event
  produces exactly one payload shape" predates the batch event's second shape;
  two places said "the dispatcher runs item-level selectors against each member
  snapshot" when no binding engine exists and the dispatcher filters on event
  NAME only; my own retirement comment said this path emits "item.updated +
  comment.created" transactionally, when the item half is whichever slice moved
  (a status-only update emits status_changed) and the comment is a separate
  transaction; migration 083 described the claim as one statement doing both
  the select and the mark.

One finding recorded rather than fixed: affectedIDs counts rows TOUCHED, not
rows semantically changed, so an all-no-op operation writes a header with a
count and no members. Verified against origin/main — the webhook this replaces
fired on the identical condition with the identical count, so it is inherited,
and narrowing it is a wire change to count/item_ids that belongs with a
contract version rather than a delivery refactor.

Gates: build clean, make lint 0 issues, go test ./internal/... green,
make test-pg exit 0 / 3463 PASS / 0 FAIL with the new outbox tests verified
present in the Postgres run.

* fix: codex round 2 — batch correlation on the wire, prior_status survival (TASK-2714)

Round 2 was aimed at round 1's own fixes, and that is where both P1s were.

- BATCH_ID REACHED ONLY THE FOLDED HALF. A member delivered individually — the
  window this whole design accounts for — carried an item snapshot with no
  batch anywhere in it, while three comments claimed consumers could correlate
  the singles with the batch. They could not. batch_id is now an ENVELOPE field
  on every delivery of a batched event, singles included, which is the only
  place a consumer can read it for a member.
- FOLD DEDUP COULD DROP prior_status. A mixed update writes item.status_changed
  (carrying the transition) and item.updated (not); round 1's last-wins kept the
  later row and silently lost the one field a "nonterminal → terminal" binding
  needs, in exactly the case that produces both rows. The snapshot is still
  last-wins — every field IS fresher on the later row — but prior_status is
  carried forward, because it is envelope metadata only one of the two events
  ever has.
- Sibling scans deduped: a 100-row candidate slice from one batch ran the same
  query 100 times.
- The "only when something actually changed" comment on the bulk emission
  condition is corrected rather than left to be re-derived: the condition is
  that a row was TOUCHED without erroring. Untagging a tag nobody has succeeds
  on every row and changes nothing, so the header fires with a count while the
  store writes no member events. Same inherited asymmetry round 1 recorded;
  now the comment says it where the code is.
- MY OWN COUNTS WERE WRONG IN THREE PLACES, which is the number-discipline
  lesson landing on documentation instead of a report: the handler test said
  "six verbs" while driving seven legs and "three of the six verbs" for what
  was three CALL SITES across four verbs; the store test implied it covered the
  verbs when it covers entry points, and now says out loud that it is not
  sufficient alone — round 1's bug lived one layer above it.

Mutations, 2/2 on the new fixes: deliver singles with an empty batch id (the
member leg names it twice, once per member); revert the dedupe to plain
last-wins (the prior_status leg names it).

go test ./internal/... green.

* fix: codex round 3 — ack and release are conditioned on the claim (TASK-2714)

The P1, and it is round 2's area again: claim tokens were minted and never
checked. MarkOutboxDispatched and MarkOutboxAttemptFailed matched on the row id
alone, so once a lease expired, a slow pass could still reach rows a newer pass
legitimately owned — a late ack stamping a row the new holder is mid-delivery
on, and a late release CLEARING a live claim and handing the event to a third
pass.

Reachable, not theoretical: a workspace's endpoints are delivered sequentially,
each with three attempts and a 10s timeout, and the "well under a minute"
estimate behind the lease default is an estimate rather than a bound.

Both writes now carry the token and condition on claimed_by, and an empty token
is refused outright rather than matching NULL. OutboxEvent carries ClaimToken
so the drain never has to track it separately. A stale ack matches zero rows,
which is exactly right — the event has become the new claim's problem.

Also round 3, all P3:
- The fold's "embedded VERBATIM" claim now names its one exception: a deduped
  survivor is re-encoded to carry prior_status across. Non-duplicate members
  are untouched bytes.
- Four stale comments corrected where they live, not just where they were
  introduced: migration 081 still said nothing drained the table and webhooks
  fired from hand-calls; createItemChecked's summary still ended in "webhook
  dispatch"; handlers_watch_notify still called publishBulkItemsEvent "the
  SSE/webhook bulk path"; and two copies of the PII rationale said nothing
  drains or prunes, when the window is now bounded (bounded is not zero, which
  is why the scrub still does the work).

Mutations, 2/2: drop claimed_by from the ack (the stale-ack leg fires), drop it
from the release (the stale-release leg fires, naming the instance that took
the freed row). Both mutations were verified present in the file first — the
initial pair silently failed to apply and reported green, which is the third
instrument mis-report this unit.

Gates: go test ./internal/... green, make lint 0 issues.

* fix: codex round 4 — retention spares live claims, token refusal is unconditional (TASK-2714)

- RETENTION COULD DELETE A ROW MID-DELIVERY. Every instance runs retention, so
  one instance's prune could remove an old undispatched row another instance
  was actively delivering: the delivery would succeed while the ack matched
  zero rows, and a crash in that window loses an event the outbox had already
  committed. Live claims are now exempt, using the same lease predicate the
  claim itself uses. An EXPIRED claim stays fair game — that is what expiry
  means — and the test asserts both directions.
- THE EMPTY-TOKEN REFUSAL SAT BEHIND THE EMPTY-ID SHORT-CIRCUIT, so
  MarkOutboxDispatched("", nil) returned nil: a contract that depended on the
  argument it was not about. Token check first.
- The taxonomy comment claimed per-member events for ALL bulk mutations. True
  only of the handler path; the store-side single-transaction producers have no
  loop, and for those the snapshots INSIDE the payload are the only per-member
  view there is. Both mechanisms now named, since the distinction is visible in
  the payloads.
- ListPendingOutboxEvents is documented as the diagnostic reader. The drain
  claims; a reader finding two pending-row queries should not have to guess
  which one production uses.

Mutation, 1/1: drop the claim predicate from the prune (the new test names the
count). Verified applied before the result was read.

Round 4 also found a REGRESSION I am not fixing here because it is a fork:
create-with-parent webhooks carry a pre-link snapshot. CreateItem writes the
item.created row in its own transaction, SetParentLink runs in a separate one,
and main's hand-called webhook dispatched the RE-READ item — so the parent and
the post-link seq were visible then and are not now. Escalated with a
recommendation (emit item.updated from SetParentLink's own transaction, which
also covers the general case); it sits close enough to SPEC-3 v1.5's
link-silence ruling that it is not mine to infer.

go test ./internal/... green.

* fix: SetParentLink emits item.updated on its own transaction (TASK-2714)

Codex round 4's regression, ruled (a) by the lead with the F4 boundary made
mechanical rather than inferred (SPEC-3 v1.6): a mutation that writes the
ITEM'S OWN ROW emits item.updated; a relationship-graph link, which writes only
the links table, stays silent. A parent link advances seq and flips the
is_unparented bit, so it is on the emitting side of that line.

The regression it closes: createItemChecked calls SetParentLink AFTER
CreateItem has already committed item.created with a pre-link snapshot, then
re-reads the item for its response. Main's hand-called webhook dispatched that
re-read, so a consumer saw the parent and the post-link seq; under the drain
the frozen created row was all there was, with nothing to correct it.
created(pre-link) then updated(post-link) is a true history.

Placed in setParentLinkOnce, not in the shared setParentLinkTx:
UpdateItemWithParentLink reuses that core inside the item-update transaction
and already emits from the field diff, so the shared site would double-emit.

The snapshot comes from getItemTx, and the test enforces that rather than the
comment doing it alone — mutating the read to the pool's GetItem fails on the
seq assertion, because a different connection cannot see the uncommitted write
and would emit the pre-link row under a post-link event.

Mutations, 2/2: delete the emit (no event after linking); read the snapshot
from the pool (seq is the create's). Plus a control leg asserting the PARENT
emits nothing — the link does not write its row.

go test ./internal/... green.

* fix: codex round 5 — parent-only updates emit; the parent-emit claim is narrowed to the truth (TASK-2714)

Round 5 aimed at round 4's own fix and found two P1s in it. Four-for-four on
that angle now.

1. THE PARENT-ONLY UPDATE PATH STILL EMITTED NOTHING. SetParentLink's fix
   covers its own transaction; UpdateItemWithParentLink writes the hierarchy
   inside the ITEM-UPDATE transaction and emits from a snapshot DIFF — and a
   parent write leaves nothing in a snapshot to diff, since items.parent_id is
   legacy and untouched, the link lives in its own table, and seq/updated_at
   are excluded as metadata. So a fields_patch carrying only `parent` mutated
   the row and emitted zero events. The emitter now takes hierarchyChanged from
   the caller, which knows what it wrote; the diff cannot see it and must not
   have to. Covers set AND clear, with a control leg asserting a genuinely
   empty update still emits nothing — without it the fix could be "always
   emit", which would undo the disjoint-delta rule.

2. MY OWN ROUND-4 COMMENT AND TEST OVERCLAIMED. Both said the event carries a
   "post-link snapshot"; the payload is the item ROW, and the parent EDGE is
   not on it — IsUnparented is populated only by the local-first index
   queries, so the test's is_unparented assertion passed VACUOUSLY against an
   absent field. What the emit actually restores is the row change (a fresh
   seq and updated_at), which is exactly what main's hand-called webhook
   carried: it dispatched the handler's post-link re-read, the same scan. The
   comment now says that, and the test asserts the ABSENCE so the next reader
   cannot infer linkage data that has never been on this wire.

   Third instance this unit of the same shape: a partial verification written
   up as a complete one.

Also round 5, both comment-level:
- handlers_item_links.go said item-link mutations are silent in events/1. True
  per link TYPE, not per handler: parent crosses the "writes the item's own
  row" line and emits, blocks/blocked-by and implements do not. The comment
  now states the criterion and names the consequence — this handler publishes
  SSE for both kinds, so the SSE and events/1 pictures deliberately differ.
- EmitBulkHeaderEvent's guard said "a bulk operation that changed nothing is
  not an event" while being an empty-LIST guard. Corrected in place with the
  reason it stays: the webhook it replaced fired on the identical condition
  with the identical count, so narrowing it is a wire change for a contract
  version, not a fix.

Mutation, 1/1: drop the hierarchy force (the parent-only leg fails by name).
Verified applied before the result was read.

go test ./internal/... green.

* fix: codex round 6 — parent DETACH emits on every route (TASK-2714)

Round 6 aimed at round 5's fix and found two more in it. Five-for-five.

1. DETACH WAS SILENT ON TWO OF THREE ROUTES. Attach emitted from
   SetParentLink and from the update path, but ClearParentLink (its own
   transaction) and DeleteItemLink on a parent row (what DELETE /links/{id}
   actually calls) wrote the item's row and emitted nothing. A consumer's
   model would keep a parent the user had removed, with every attach route
   observable — the worst shape for this kind of gap, because the wire looks
   healthy. Routes are now enumerated in one test rather than sampled.

2. hierarchyChanged MEANT "PROVIDED", NOT "CHANGED". Clearing an
   already-unparented item deletes zero rows; round 5's flag still forced
   item.updated, putting an event on a public wire for a mutation that did not
   happen. clearParentLinkTx now reports whether it removed a link and the flag
   comes from that. The set branch stays unconditional — it is a
   DELETE-then-INSERT and bumps the row either way.

IMPLEMENTS IS FLAGGED, NOT DECIDED. It bumps the same row (so the v1.6
mechanical criterion would include it) but it is a relationship-graph link (so
v1.5's silence would exclude it). The contract does not resolve that case, and
inventing an answer inside a delivery refactor is how a public wire acquires an
event nobody ruled on. Recorded at the call site and raised with the lead.

Mutations, 2/2: drop the parent-detach emit from DeleteItemLink (the route's
leg fails); treat provided as changed on the clear branch (the no-op leg fails
by name). The second mutation first failed to BUILD — `removed` then unused,
which is the compiler catching it and my grep reading the empty result as a
pass — so it was re-run with the variable kept alive. Fourth instrument
mis-report this unit; all four are on the record.

Gates: go test ./internal/... green, make lint 0 issues, make test-pg exit 0
on the pre-round-6 tip (re-run pending on the final tip).

* fix: codex round 7 — the batch delta matches the mutation (TASK-2714)

Round 7 returned no P1s; the parent-detach work from round 6 came back clean on
transaction scope, lock ordering, error paths and duplicate emissions.

- BULK DELTA REPORTED THE REQUEST, NOT THE COMMIT. bulkEventDelta echoed raw
  request values while the mutation normalizes: bulkTagUpdate trims added tags
  and skips ones that go empty, and the store's assignment SET clause gives a
  NON-EMPTY id precedence over the clear flag (BUG-2566). So a request with
  both an id and clear=true announced a clear while the row was assigned —
  the delta describing the opposite of what committed. This is the one field
  of the batch payload the drain cannot derive, so nothing downstream corrects
  it: whatever it says is what a consumer believes. Now normalized the same
  way, including untag matching RAW because the mutation removes by exact
  match.
- THE implements COMMENT WAS WRONG, and it was mine from round 5: it listed
  implements with the silent link types when implements DOES bump the source
  row, exactly as parent does. Corrected in both places, and the case is
  stated as UNRESOLVED rather than settled — the mechanical criterion (writes
  the item's row) would have it emit, v1.5's relationship-link silence would
  not, and deciding it inside a delivery refactor would put an event nobody
  ruled on onto a public wire. With the lead.

Mutations, 2/2: give the clear flag precedence over a non-empty id (the
precedence leg fails, naming the row it would misdescribe); stop trimming
added tags (the trim leg fails). The first mutation initially failed to build
and was rewritten to compile before its result was believed.

Gates on the round-6 tip: go test ./internal/... green, make lint 0 issues,
make test-pg exit 0 / 3445 PASS / 0 FAIL with 13 of this unit's new test legs
verified present in the Postgres output. Re-run pending on the final tip.

* fix: codex round 8 — tag delta dedups, link SSE name derives (TASK-2714)

Two P2s, both small and both the same shape: a claim in a comment that the
code did not quite meet.

- THE TAG DELTA TRIMMED BUT DID NOT DEDUPE, while bulkTagUpdate does both — it
  skips a tag already in its `seen` set. So tag ["foo", " foo "] added one tag
  and advertised two, under a comment saying the delta is normalized "the same
  way the mutation does". Round 7 fixed half of that sentence; this fixes the
  other half.
- handlers_item_links.go PUBLISHED UNDER THE events.ItemUpdated LITERAL. The
  wire value happens to match, which is exactly why it was worth changing: it
  recreates the drift the central mapping exists to prevent, one rename away
  from being wrong. The name now derives like every other SSE site, and the
  comment separates the two facts a reader has to keep apart — the NAME
  derives, the events/1 EVENT still does not exist for relationship links.

Mutation, 1/1: drop the `seen` check from the delta's tag loop (the dedup test
names the duplicated value).

go test ./internal/... green.

* fix: codex round 9 — untag delta dedups, version restore derives its SSE name (TASK-2714)

Both are the same shape as round 8's, one layer further out.

- THE UNTAG DELTA STILL ECHOED DUPLICATES. bulkTagUpdate builds a removal SET,
  so ["foo","foo"] removes one tag; the delta advertised two. The two verbs
  normalize DIFFERENTLY — tag trims and dedups, untag dedups but matches raw,
  because removal is by exact string — and the delta now mirrors each side's
  own rule rather than applying one of them to both.
- handlers_item_versions.go PUBLISHED A RAW "item_updated" STRING. A second
  source of SSE vocabulary, and the harder kind to find: it does not even
  reference the events package, so a grep for events.ItemUpdated misses it.
  Now derived like every other site.

go test ./internal/... green.

Gates on the round-8 tip: make test-pg exit 0, 3447 PASS, 0 FAIL, with 150
lines of this unit's own test legs verified present in the Postgres output.
2026-08-20 23:33:26 -04:00

5351 lines
216 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
package store
import (
"database/sql"
"encoding/json"
"errors"
"fmt"
"regexp"
"sort"
"strings"
"time"
"github.com/PerpetualSoftware/pad/internal/diff"
"github.com/PerpetualSoftware/pad/internal/kernelevents"
"github.com/PerpetualSoftware/pad/internal/models"
)
// childLinkTypes lists the link types that establish a parent→child relationship
// for progress tracking. Both 'parent' and 'implements' links count as children.
var childLinkTypes = []string{"parent", "implements"}
// childLinkTypeSQL returns a SQL IN clause fragment like "'parent','implements'"
// for filtering item_links by child relationship types.
func childLinkTypeSQL() string {
quoted := make([]string, len(childLinkTypes))
for i, t := range childLinkTypes {
quoted[i] = "'" + t + "'"
}
return strings.Join(quoted, ",")
}
// ItemSearchResult holds FTS search results for items.
type ItemSearchResult struct {
Item models.Item `json:"item"`
Snippet string `json:"snippet"`
Rank float64 `json:"rank"`
}
// validateAssignmentScope checks that the assigned user and agent role belong to the
// same workspace as the item. This prevents cross-workspace assignment leaks.
func (s *Store) validateAssignmentScope(workspaceID string, assignedUserID, agentRoleID *string) error {
return s.validateAssignmentScopeQ(s.db, workspaceID, assignedUserID, agentRoleID)
}
// validateAssignmentScopeQ is validateAssignmentScope parameterized over the
// query surface so the same two checks can run inside a caller's transaction
// (createItemTx) instead of on an independent connection. Behaviour and error
// strings are identical to the *sql.DB form; only the connection the two
// existence probes run on differs.
//
// It deliberately re-issues the membership / agent-role probes as COUNT
// queries rather than calling IsWorkspaceMember / GetAgentRole, which are
// hard-wired to s.db. The predicates mirror those methods exactly, including
// GetAgentRole's `id = ? OR slug = ?` acceptance.
func (s *Store) validateAssignmentScopeQ(q rowQueryer, workspaceID string, assignedUserID, agentRoleID *string) error {
if assignedUserID != nil && *assignedUserID != "" {
var count int
if err := q.QueryRow(
s.q("SELECT COUNT(*) FROM workspace_members WHERE workspace_id = ? AND user_id = ?"),
workspaceID, *assignedUserID,
).Scan(&count); err != nil {
return fmt.Errorf("validate assigned user: %w", fmt.Errorf("check workspace membership: %w", err))
}
if count == 0 {
return fmt.Errorf("assigned user is not a member of this workspace")
}
}
if agentRoleID != nil && *agentRoleID != "" {
var count int
if err := q.QueryRow(
s.q("SELECT COUNT(*) FROM agent_roles WHERE workspace_id = ? AND (id = ? OR slug = ?)"),
workspaceID, *agentRoleID, *agentRoleID,
).Scan(&count); err != nil {
return fmt.Errorf("validate agent role: %w", fmt.Errorf("get agent role: %w", err))
}
if count == 0 {
return fmt.Errorf("agent role does not belong to this workspace")
}
}
return nil
}
// maxItemNumberRetries is the number of times CreateItem will retry when a
// concurrent insert claims the same workspace-global item_number.
const maxItemNumberRetries = 10
// nextWorkspaceSeqSubquery is the SQL fragment used to atomically compute
// the next workspace-scoped `seq` value inside an INSERT / UPDATE.
// Every items mutation (create / update / soft-delete / restore) stamps
// the new row's seq with `MAX(seq) + 1 WHERE workspace_id = ?`, which is
// the cursor mechanic for the local-first read model's delta sync
// (PLAN-1343 / DOC-1342 decision #1).
//
// Callers must append exactly one `workspaceID` arg for this fragment.
// SQLite is single-writer so the read-modify-write is naturally
// serialized; Postgres callers must additionally hold the workspace
// advisory lock acquired via acquireWorkspaceSeqLock so concurrent
// writes can't both read the same MAX(seq) and produce duplicates.
const nextWorkspaceSeqSubquery = "(SELECT COALESCE(MAX(seq), 0) + 1 FROM items WHERE workspace_id = ?)"
// unparentedItemPredicate is the canonical structural definition used by
// list filtering and local-first metadata. Archived/hidden targets still
// count because the relationship itself is what parents the source item;
// consequently the subquery intentionally does not join items.
const unparentedItemPredicate = "i.parent_id IS NULL AND NOT EXISTS (SELECT 1 FROM item_links unp WHERE unp.source_id = i.id AND unp.link_type IN ('parent', 'implements'))"
// nextTransitionSeqSubquery assigns a monotonic, insertion-ordered seq to each
// status_transitions row (global MAX+1 — cross-row dupes across items/workspaces
// are harmless since the ordering is only used WITHIN a single item's history).
// It's the precise tiebreak for "latest transition <= T" when created_at
// (second precision) ties (PLAN-1628 / TASK-1643). The inserts that use it run
// inside the workspace seq lock, so per-item assignment is serialized.
const nextTransitionSeqSubquery = "(SELECT COALESCE(MAX(seq), 0) + 1 FROM status_transitions)"
// acquireWorkspaceSeqLock takes a Postgres advisory transaction lock
// keyed on the workspace ID so concurrent seq-bumping mutations
// serialize. The lock auto-releases on COMMIT / ROLLBACK. On SQLite
// the single-writer rule already serializes writes, so this is a
// no-op there. Mirrors the existing advisory-lock pattern in
// tryCreateItem (which uses the same key for item_number assignment).
func (s *Store) acquireWorkspaceSeqLock(tx *sql.Tx, workspaceID string) error {
if s.dialect.Driver() != DriverPostgres {
return nil
}
if _, err := tx.Exec("SELECT pg_advisory_xact_lock(hashtext($1))", workspaceID); err != nil {
return fmt.Errorf("acquire workspace seq lock: %w", err)
}
return nil
}
// acquireWorkspaceParentLinkLock takes a Postgres advisory transaction lock
// that serializes ALL parent-edge-ADDING transactions within a workspace
// (BUG-2074). It is the outer guard that closes the N-hop parent-cycle gap the
// per-endpoint cycle walk cannot: BUG-2073's per-item sorted lock batch only
// covers the two endpoints of the edge being written, so a cycle closed via an
// edge on an item that NEITHER endpoint locks (e.g. an existing A->B and C->D
// cross-linked concurrently by SetParentLink(B,C) + SetParentLink(D,A), whose
// lock sets {B,C} and {D,A} are disjoint) still slips through — both cycle
// walks run on stale snapshots and both inserts commit, forming A->B->C->D->A.
//
// With this lock held by every edge-adder, no two parent-edge insertions can
// run concurrently in a workspace, so checkParentCycleQ always walks a
// consistent, non-racing ancestor snapshot and catches arbitrary N-hop cycles.
// Parent-link writes are rare, so serializing them per-workspace is an
// acceptable trade-off. Only edge-ADDING paths take this lock (the paths that
// call setParentLinkTx with a non-empty parentID); edge REMOVALS (clear /
// detach) and plain field updates can't create a cycle, so they don't request
// it — keeping the common UpdateItem path un-serialized.
//
// DISTINCT namespace from acquireWorkspaceSeqLock (bare hashtext(workspaceID))
// and from AcquireParentChildrenLocks (per-item 'pad:parent-children:<id>') so
// the three lock classes never alias on the same key.
//
// LOCK ORDERING (deadlock-freedom): callers acquire this OUTERMOST — before the
// workspace seq lock and before the per-item parent-children batch. The global
// order across every transaction is therefore:
//
// parent-link-cycle lock -> workspace seq lock -> parent-children batch
//
// No transaction takes any of these in the reverse relative order, and only
// edge-adding transactions take the cycle lock at all (a plain UpdateItem /
// CreateItem / clear-parent never requests it), so the classic AB/BA deadlock
// shape can't form.
//
// On SQLite the global BEGIN IMMEDIATE write lock already serializes every
// writer, so this is a no-op there (the N-hop race can't manifest on SQLite).
func (s *Store) acquireWorkspaceParentLinkLock(tx *sql.Tx, workspaceID string) error {
if s.dialect.Driver() != DriverPostgres {
return nil
}
if _, err := tx.Exec("SELECT pg_advisory_xact_lock(hashtext('pad:parent-link-cycle:' || $1))", workspaceID); err != nil {
return fmt.Errorf("acquire workspace parent-link lock: %w", err)
}
return nil
}
func (s *Store) CreateItem(workspaceID, collectionID string, input models.ItemCreate) (*models.Item, error) {
// Retry loop: if a concurrent insert claims the same item_number — or the
// same slug — we roll back and re-derive both on the next attempt.
//
// Re-deriving matters. Before TASK-2362 the slug was allocated ONCE,
// outside the transaction, and every retry re-submitted that same stale
// value: two concurrent creates of the same title had the loser burn all
// ten attempts on a slug the winner had already committed and then fail
// with a unique-constraint error. tryCreateItem now allocates inside the
// transaction, under the workspace lock, so each attempt sees the losing
// scan's outcome and picks the next free suffix.
var lastErr error
for attempt := 0; attempt < maxItemNumberRetries; attempt++ {
item, err := s.tryCreateItem(workspaceID, collectionID, input)
if err == nil {
return item, nil
}
lastErr = err
// Only retry on unique-constraint violations (item_number / slug).
// Everything else — including assignment-scope rejections — is
// returned as-is.
if !isUniqueViolation(err) {
return nil, err
}
}
return nil, fmt.Errorf("insert item after %d retries: %w", maxItemNumberRetries, lastErr)
}
// tryCreateItem attempts a single transactional insert of an item with the
// next available workspace-global item_number and a freshly-allocated unique
// slug. The item_number is computed atomically via a subquery in the INSERT to
// avoid races between concurrent inserts reading the same MAX(item_number).
//
// It is a thin BEGIN/COMMIT wrapper around createItemTx, which is also what
// the cross-workspace copy path calls with its own transaction — so the two
// creation paths share one implementation and cannot drift.
//
// Two deliberate behaviour changes from the pre-TASK-2362 shape, neither
// observable to any current caller (nothing in internal/server or cmd/pad
// string-matches these):
//
// - Each retry attempt now derives a fresh item id and timestamp, because
// both are generated inside createItemTx. Previously one id/timestamp was
// minted before the loop and re-submitted. The discarded id was never
// returned to anyone, and a retried attempt arguably SHOULD carry the
// time it actually succeeded.
// - The item is read back inside the transaction rather than after COMMIT.
// A read-back miss now rolls the create back with an error instead of
// returning (nil, nil) over a committed row — the old shape handed callers
// a nil item and a nil error for an item that existed.
func (s *Store) tryCreateItem(workspaceID, collectionID string, input models.ItemCreate) (*models.Item, error) {
tx, err := s.db.Begin()
if err != nil {
return nil, fmt.Errorf("insert item: %w", err)
}
defer tx.Rollback()
item, err := s.createItemTx(tx, workspaceID, collectionID, input)
if err != nil {
return nil, err
}
if err := tx.Commit(); err != nil {
return nil, fmt.Errorf("insert item: %w", err)
}
return item, nil
}
// insertItemTx is the write half of item creation: the items INSERT
// (item_number, workspace seq, content-flush watermarks), the initial
// item_versions row, wiki-link indexing + broken-title resolution, and the
// create-time status_transitions row. It neither begins nor commits — the
// caller owns the transaction boundary.
//
// Every side effect of an API-PATH creation lives in this one place. Its only
// caller is createItemTx, which both CreateItem and the cross-workspace copy
// path (PLAN-2357 / DR-9a) go through, so those two paths cannot drift.
//
// The "API-PATH" qualifier is load-bearing and was missing until TASK-2658.
// This comment used to read "every creation side effect", full stop, and it is
// not true: ImportWorkspace (export.go) writes items with its own INSERT INTO
// items and never reaches here. Two units in a row have now been misled by
// reading a scoped invariant as a global one — TASK-2657 shipped a P1 on the
// same mistake in this file family. If you are adding a creation side effect,
// grep `INSERT INTO items` and confirm which of the TWO write paths you need,
// rather than trusting that this is the only one.
func (s *Store) insertItemTx(tx *sql.Tx, id, workspaceID, collectionID, slug, ts, fields, tags, createdBy, source string, input models.ItemCreate) error {
var err error
// PostgreSQL: take an advisory lock keyed on the workspace to serialize
// item_number assignment. This eliminates the race between concurrent
// transactions reading the same MAX(item_number). The lock is released
// automatically when the transaction commits or rolls back.
// SQLite: single-writer by design, no advisory locks needed.
if s.dialect.Driver() == DriverPostgres {
// Use a hash of the workspace ID as the advisory lock key.
_, err = tx.Exec("SELECT pg_advisory_xact_lock(hashtext($1))", workspaceID)
if err != nil {
return fmt.Errorf("advisory lock: %w", err)
}
}
// Compute and insert the next item_number atomically within the lock.
// content_flushed_at + content_flushed_op_log_id are the op-log GC
// watermarks (TASK-1309). The id column is authoritative — sweeper
// uses strict id comparison to avoid second-granularity timestamp
// false positives. Both set iff content is non-empty:
// - timestamp = creation time (informational)
// - id = 0 (vacuously safe — there are no op-log rows yet, so
// "covers all rows up to id 0" never gates anything until the
// first op-log row arrives, at which point this item is
// non-dormant by virtue of having a recent row)
// Empty content → both NULL; sweeper treats NULL as "never flushed"
// and skips pruning.
var contentFlushedAt interface{}
var contentFlushedOpLogID interface{}
if input.Content != "" {
contentFlushedAt = ts
contentFlushedOpLogID = int64(0)
}
// Stamp any pad-attachment: references BEFORE the INSERT (see the
// ORDERING note on stampAttachmentRefsTx): the stamp's row locks
// make a concurrent GC claim block until this tx commits.
if err := stampAttachmentRefsTx(tx, s, workspaceID, input.Content, fields); err != nil {
return err
}
// The workspace advisory lock acquired above for item_number
// assignment ALSO serializes the seq subquery below — both read
// MAX(...) per workspace and would otherwise race in Postgres.
_, err = tx.Exec(s.q(`
INSERT INTO items (id, workspace_id, collection_id, title, slug, content, fields, tags,
pinned, sort_order, parent_id, assigned_user_id, agent_role_id, role_sort_order,
created_by, last_modified_by, source, item_number, created_at, updated_at,
content_flushed_at, content_flushed_op_log_id, seq)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 0, ?, ?, ?, 0, ?, ?, ?,
(SELECT COALESCE(MAX(item_number), 0) + 1 FROM items WHERE workspace_id = ?),
?, ?, ?, ?, `+nextWorkspaceSeqSubquery+`)
`), id, workspaceID, collectionID, input.Title, slug, input.Content, fields, tags,
s.dialect.BoolToInt(input.Pinned), input.ParentID, nullIfEmptyID(input.AssignedUserID), nullIfEmptyID(input.AgentRoleID),
createdBy, createdBy, source, workspaceID, ts, ts, contentFlushedAt, contentFlushedOpLogID, workspaceID)
if err != nil {
return err
}
// Create initial version if there's content
if input.Content != "" {
vid := newID()
// version_seq is a per-item monotonic tie-breaker (BUG-2270).
// COALESCE(MAX,0)+1 in the same tx; version creation is serialized
// per item under the item lock, so MAX+1 is race-safe.
_, err = tx.Exec(s.q(`
INSERT INTO item_versions (id, item_id, content, change_summary, created_by, source, is_diff, created_at, version_seq)
VALUES (?, ?, ?, '', ?, ?, ?, ?, (SELECT COALESCE(MAX(version_seq), 0) + 1 FROM item_versions WHERE item_id = ?))
`), vid, id, input.Content, createdBy, source, s.dialect.BoolToInt(false), ts, id)
if err != nil {
return fmt.Errorf("create initial version: %w", err)
}
}
// Index [[...]] wiki-links from the new content. Lives inside the
// same tx as the items INSERT so partial state never lands and a
// content rollback also rolls back the index rows. Empty content
// is fine — replaceWikiLinks short-circuits after deleting any
// prior rows (there are none on initial create). PLAN-1593 /
// TASK-1594.
if err := s.replaceWikiLinks(tx, id, workspaceID, input.Content); err != nil {
return fmt.Errorf("index wiki links: %w", err)
}
// Phase 2a (TASK-1595): flip any pre-existing broken `[[Title]]`
// rows that have been waiting for an item with this title to
// arrive. Cheap when no broken rows match (the common case).
// Without this, sources that mention the new item by title would
// stay broken until either their content is rewritten or the
// next migration-driven backfill — both rare.
collSlug, err := s.getCollectionSlugTx(tx, collectionID)
if err != nil {
return fmt.Errorf("lookup collection slug: %w", err)
}
if err := s.resolveBrokenTitleLinks(tx, id, workspaceID, collSlug, input.Title); err != nil {
return fmt.Errorf("resolve broken titles: %w", err)
}
// Seed the create-time "entered initial status" transition so an item
// created directly in a terminal done-field value (e.g. a retroactively
// logged "done" task, or an import) still counts as a completion in
// reports (PLAN-1628 / TASK-1637). Resolve the collection's done field
// in-tx; skip when it's unset at creation. The deterministic id keeps
// this idempotent with the backfill's create-seed for the same item.
var schemaJSON, settingsJSON string
if err := tx.QueryRow(s.q(`SELECT schema, settings FROM collections WHERE id = ?`), collectionID).Scan(&schemaJSON, &settingsJSON); err == nil {
doneKey := doneFieldKeyFromSchemaJSON(schemaJSON, settingsJSON)
if initial := extractFieldValue(fields, doneKey); initial != "" {
if _, err = tx.Exec(s.q(`
INSERT INTO status_transitions (id, item_id, workspace_id, collection_id, field_key, from_status, to_status, created_at, seq)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, `+nextTransitionSeqSubquery+`)
`), "create_"+id, id, workspaceID, collectionID, doneKey, "", initial, ts); err != nil {
return fmt.Errorf("record create-time status transition: %w", err)
}
}
}
return nil
}
// nullIfEmptyID maps a nil-or-empty ID pointer to SQL NULL. Nullable FK
// columns (assigned_user_id, agent_role_id) need this at bind time: a JSON
// client that blanks the field sends "", which validateAssignmentScope
// deliberately skips, and binding "" verbatim fails the FK instead of
// clearing the column (BUG-2566).
func nullIfEmptyID(p *string) any {
if p == nil || *p == "" {
return nil
}
return *p
}
// createItemTx is the tx-taking form of item creation (PLAN-2357 / DR-9a) and
// the single implementation behind BOTH CreateItem and the cross-workspace
// copy path. It performs the ENTIRE create pipeline — defaults,
// assignment-scope validation, workspace-scoped unique slug allocation,
// item_number, workspace seq, content-flush watermarks, the initial
// item_versions row, wiki-link indexing plus broken-title resolution in the
// destination workspace, and the create-time status_transitions row — inside a
// transaction the caller owns.
//
// It exists because CreateItem opens and commits its own transaction, so the
// cross-workspace copy path (which must create in B, remap attachments, write
// provenance and optionally archive the source atomically) cannot call it. A
// raw `INSERT INTO items` in its place would silently break version history,
// wiki-links, reporting, delta sync and slug uniqueness — none of which fail
// loudly. Making CreateItem go through this same function rather than a
// parallel copy is what keeps the two from drifting.
//
// CONTENT MUST ALREADY BE FINAL. Wiki-link indexing and the initial version
// row are written from input.Content as given, so a caller doing attachment-ref
// rewriting (DR-11) must rewrite BEFORE calling — otherwise the indexed body
// and the first version both carry the source workspace's attachment UUIDs.
//
// TRUST BOUNDARY — this helper TRUSTS its caller, matching the pre-extraction
// tryCreateItem:
// - collectionID is NOT checked to belong to workspaceID, nor to be live.
// DR-9 puts that on the caller, which re-reads and row-locks the
// destination collection (FOR UPDATE on Postgres) inside the same tx; a
// check here would be a second, weaker read of an already-pinned row.
// - input.ParentID is NOT checked to belong to workspaceID, and no parent
// cycle walk runs. Callers that set it must validate it; the copy path
// scrubs it to nil (DR-17 — the copy is unparented).
//
// What it does NOT trust: input.AssignedUserID and input.AgentRoleID are
// validated against workspaceID, exactly as CreateItem does.
//
// NO RETRY ON UNIQUE VIOLATION. CreateItem wraps this in a retry loop by
// re-running the whole transaction; a caller-owned transaction can't do that,
// because a failed statement poisons it (Postgres) and an internal retry would
// need a savepoint the caller can't see. Retrying is near-redundant anyway:
// the workspace advisory lock (Postgres) / BEGIN IMMEDIATE (SQLite) serializes
// item_number and slug allocation per workspace for the transaction's whole
// lifetime. A caller-owned transaction that wants the retry must re-run its
// own transaction.
//
// Returns the created item read back inside the tx, so the caller can consume
// its committed slug / item_number / seq (DR-14 fanout) without a second
// round-trip after COMMIT.
func (s *Store) createItemTx(tx *sql.Tx, workspaceID, collectionID string, input models.ItemCreate) (*models.Item, error) {
return s.createItemTxWithID(tx, newID(), workspaceID, collectionID, input)
}
// createItemTxWithID is createItemTx with the destination item's id supplied
// by the caller instead of minted inside.
//
// It exists for exactly one caller: CopyItemAcrossWorkspaces (PLAN-2357 /
// DR-9 / DR-11). The copy has to hand the attachment planner the destination
// item id BEFORE the item row exists, because every cloned attachment row must
// carry item_id from the outset — never transiently NULL, since a NULL-item_id
// row is a permanent un-reclaimable orphan (see AttachmentCopyRequest.DryRun's
// doc). Minting the id in the orchestration and passing it down is the only
// way to satisfy both that ordering and DR-9a's "the version row and the
// wiki-link index are built from the POST-rewrite content".
//
// An empty id is filled in, so a caller that has no opinion behaves exactly
// like createItemTx. The id is NOT validated for uniqueness here — the items
// primary key does that, and a collision (a caller re-using an id) surfaces as
// a unique violation that rolls the caller's transaction back.
func (s *Store) createItemTxWithID(tx *sql.Tx, id, workspaceID, collectionID string, input models.ItemCreate) (*models.Item, error) {
// Validate assignment scope before writing — parity with CreateItem, but
// read through the tx so it sees the caller's uncommitted membership /
// role writes and is serialized with them.
//
// On SQLite this (and the slug scan below) now runs under the db-wide
// BEGIN IMMEDIATE write lock, where pre-extraction CreateItem ran it on
// the pool before opening its transaction. Both probes are single indexed
// COUNT lookups and only run when an assignee / agent role is actually
// set, and the widening is the same tradeoff store.go's DSN comment
// already accepts for UpdateItem's in-lock slug-collision check.
if err := s.validateAssignmentScopeQ(tx, workspaceID, input.AssignedUserID, input.AgentRoleID); err != nil {
return nil, err
}
if id == "" {
id = newID()
}
ts := now()
fields := input.Fields
if fields == "" {
fields = "{}"
}
tags := input.Tags
if tags == "" {
tags = "[]"
}
createdBy := input.CreatedBy
if createdBy == "" {
createdBy = "user"
}
source := input.Source
if source == "" {
source = "web"
}
// Take the workspace advisory lock BEFORE allocating the slug (Postgres;
// no-op on SQLite, where BEGIN IMMEDIATE already holds the write lock from
// the transaction's first statement). The slug scan is a read-modify-write
// on the workspace's slug space, so it must run under the same lock that
// serializes the workspace's creators — otherwise two concurrent creates of
// the same title both scan "foo" as free and one fails the unique
// constraint. Same lock key insertItemTx takes below; advisory xact locks
// are re-entrant within a transaction, so taking it twice — or a third time
// from an outer orchestrator holding both workspaces' locks — is harmless.
if err := s.acquireWorkspaceSeqLock(tx, workspaceID); err != nil {
return nil, err
}
baseSlug := slugify(input.Title)
if baseSlug == "" {
baseSlug = "untitled"
}
slug, err := s.uniqueSlugQ(tx, "items", "workspace_id", workspaceID, baseSlug)
if err != nil {
return nil, fmt.Errorf("unique slug: %w", err)
}
if err := s.insertItemTx(tx, id, workspaceID, collectionID, slug, ts, fields, tags, createdBy, source, input); err != nil {
return nil, fmt.Errorf("insert item: %w", err)
}
item, err := s.getItemTx(tx, id)
if err != nil {
return nil, err
}
if item == nil {
return nil, fmt.Errorf("created item %s not readable in transaction", id)
}
// The choke point (SPEC-3 / TASK-2658): the item.created event is written
// to the outbox on THIS transaction, so it commits with the row it
// describes or not at all. Placed after the in-transaction read-back
// because the event must carry what the database actually holds, not what
// the caller asked for.
//
// A failure here fails the create. That is not incidental — an outbox
// write that degrades to best-effort would look durable while
// reintroducing exactly the lost-event window the outbox exists to close.
if err := s.emitItemEventTx(tx, kernelevents.ItemCreated, item, nil, ""); err != nil {
return nil, err
}
return item, nil
}
// getCollectionSlugTx reads collections.slug for a collection_id
// inside the supplied tx. Tiny helper used by the wiki-link cascade
// hooks that need the slug to build collection-qualified link keys.
func (s *Store) getCollectionSlugTx(tx *sql.Tx, collectionID string) (string, error) {
var slug string
if err := tx.QueryRow(s.q(`SELECT slug FROM collections WHERE id = ?`), collectionID).Scan(&slug); err != nil {
return "", err
}
return slug, nil
}
// IsUniqueViolation is the exported form of isUniqueViolation, for HTTP
// handlers that have to turn a store error into a 409 without re-implementing
// the heuristic.
//
// Exported in TASK-2365 rather than duplicated at the call site: the
// cross-workspace copy endpoint needs exactly this test, and a second copy of
// the same two magic strings is a place for the two to drift (Codex round 8).
// It is a string match rather than a SQLSTATE/driver-type check because
// internal/store is driver-agnostic and both drivers sit behind database/sql;
// the strings are stable parts of each engine's user-facing error text.
func IsUniqueViolation(err error) bool { return isUniqueViolation(err) }
// isUniqueViolation checks whether an error is a unique constraint violation.
// Works for both SQLite (UNIQUE constraint failed) and PostgreSQL (duplicate key).
func isUniqueViolation(err error) bool {
if err == nil {
return false
}
msg := err.Error()
return strings.Contains(msg, "UNIQUE constraint failed") ||
strings.Contains(msg, "duplicate key value violates unique constraint")
}
func (s *Store) GetItem(id string) (*models.Item, error) {
return s.GetItemQ(s.db, id)
}
// GetItemQ is GetItem parameterized over its executor, so the same read can
// run against the pool or on an in-flight transaction's connection (see
// Queryer). getItemTx and the cross-workspace copy's authorization callback
// are the transaction-side callers.
func (s *Store) GetItemQ(q Queryer, id string) (*models.Item, error) {
item, err := s.getItemScanQ(q, id, false)
if err != nil {
return nil, fmt.Errorf("get item: %w", err)
}
return item, nil
}
// getItemTx is the in-transaction variant of GetItem. Used by
// UpdateItemWithPreCheck to re-read the parent under the workspace +
// parent-children locks so the invariant precheck classifies the
// transition against a snapshot that's stable for the rest of the tx
// (Codex round-3 P2). Soft-deleted items are excluded, matching
// GetItem's contract.
func (s *Store) getItemTx(tx *sql.Tx, id string) (*models.Item, error) {
item, err := s.getItemScanQ(tx, id, false)
if err != nil {
return nil, fmt.Errorf("get item (tx): %w", err)
}
return item, nil
}
// getItemScanQ is the one item-row scan behind GetItem, getItemTx and
// GetItemIncludeDeleted — identical SELECT and hydration, differing only in
// executor and in whether soft-deleted rows are visible. (nil, nil) on no row.
func (s *Store) getItemScanQ(q Queryer, id string, includeDeleted bool) (*models.Item, error) {
var item models.Item
var createdAt, updatedAt string
var deletedAt *string
var pinned bool
query := `
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.content, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at, i.deleted_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, '')
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.id = ?`
if !includeDeleted {
query += ` AND i.deleted_at IS NULL`
}
err := q.QueryRow(s.q(query), id).Scan(
&item.ID, &item.WorkspaceID, &item.CollectionID, &item.Title, &item.Slug,
&item.Content, &item.Fields, &item.Tags,
&pinned, &item.SortOrder, &item.ParentID, &item.AssignedUserID, &item.AgentRoleID, &item.RoleSortOrder,
&item.CreatedBy, &item.LastModifiedBy, &item.Source,
&item.ItemNumber, &item.Seq, &createdAt, &updatedAt, &deletedAt,
&item.CollectionSlug, &item.CollectionName, &item.CollectionIcon, &item.CollectionPrefix,
&item.AssignedUserName, &item.AssignedUserEmail,
&item.AgentRoleName, &item.AgentRoleSlug, &item.AgentRoleIcon,
)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
item.Pinned = pinned
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
item.DeletedAt = parseTimePtr(deletedAt)
hydrateItemComputedMetadata(&item)
return &item, nil
}
func (s *Store) GetItemBySlug(workspaceID, slug string) (*models.Item, error) {
var id string
err := s.db.QueryRow(s.q(`
SELECT id FROM items
WHERE workspace_id = ? AND slug = ? AND deleted_at IS NULL
`), workspaceID, slug).Scan(&id)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("get item by slug: %w", err)
}
return s.GetItem(id)
}
// GetItemByRef looks up an item by its PREFIX-NUMBER reference (e.g. "IDEA-15").
// Since item numbers are workspace-unique, this first tries an exact prefix match
// and falls back to a number-only lookup. This allows old refs to still resolve
// after an item has been moved to a different collection (e.g. PLAN-42 still
// finds the item even after it became TASK-42).
func (s *Store) GetItemByRef(workspaceID, prefix string, number int) (*models.Item, error) {
var id string
// Try exact prefix + number match first
err := s.db.QueryRow(s.q(`
SELECT i.id FROM items i
JOIN collections c ON c.id = i.collection_id
WHERE i.workspace_id = ? AND c.prefix = ? AND i.item_number = ? AND i.deleted_at IS NULL
`), workspaceID, prefix, number).Scan(&id)
if err == nil {
return s.GetItem(id)
}
if err != nil && err != sql.ErrNoRows {
return nil, fmt.Errorf("get item by ref: %w", err)
}
// Fallback: item numbers are workspace-unique, so look up by number alone.
// This handles the case where an item was moved to a different collection
// but is still being referenced by its old prefix.
err = s.db.QueryRow(s.q(`
SELECT id FROM items
WHERE workspace_id = ? AND item_number = ? AND deleted_at IS NULL
`), workspaceID, number).Scan(&id)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("get item by number: %w", err)
}
return s.GetItem(id)
}
// ResolveItem looks up an item by UUID, PREFIX-NUMBER ref (e.g. "IDEA-15"),
// or slug. UUID is tried first, then ref, then slug.
func (s *Store) ResolveItem(workspaceID, identifier string) (*models.Item, error) {
// Try UUID lookup first (8-4-4-4-12 hex format)
if isUUID(identifier) {
item, err := s.GetItem(identifier)
if err != nil {
return nil, err
}
if item != nil && item.WorkspaceID == workspaceID {
return item, nil
}
}
// Try PREFIX-NUMBER ref
if prefix, number, ok := parseItemRef(identifier); ok {
item, err := s.GetItemByRef(workspaceID, prefix, number)
if err != nil {
return nil, err
}
if item != nil {
return item, nil
}
}
// Fall back to slug lookup
return s.GetItemBySlug(workspaceID, identifier)
}
// isUUID checks if a string looks like a UUID (8-4-4-4-12 hex).
func isUUID(s string) bool {
if len(s) != 36 {
return false
}
for i, c := range s {
if i == 8 || i == 13 || i == 18 || i == 23 {
if c != '-' {
return false
}
} else if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')) {
return false
}
}
return true
}
// ResolveItemIncludeDeleted is like ResolveItem but includes soft-deleted items.
func (s *Store) ResolveItemIncludeDeleted(workspaceID, slugOrRef string) (*models.Item, error) {
// UUID lookup first, mirroring ResolveItem — but include-deleted so a
// bulk restore (TASK-1674) can resolve archived rows by id.
if isUUID(slugOrRef) {
item, err := s.GetItemIncludeDeleted(slugOrRef)
if err != nil {
return nil, err
}
if item != nil && item.WorkspaceID == workspaceID {
return item, nil
}
}
if prefix, number, ok := parseItemRef(slugOrRef); ok {
var item models.Item
var createdAt, updatedAt string
var deletedAt *string
var pinned bool
err := s.db.QueryRow(s.q(`
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.content, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at, i.deleted_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, '')
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.workspace_id = ? AND c.prefix = ? AND i.item_number = ?
`), workspaceID, prefix, number).Scan(
&item.ID, &item.WorkspaceID, &item.CollectionID, &item.Title, &item.Slug,
&item.Content, &item.Fields, &item.Tags,
&pinned, &item.SortOrder, &item.ParentID, &item.AssignedUserID, &item.AgentRoleID, &item.RoleSortOrder,
&item.CreatedBy, &item.LastModifiedBy, &item.Source,
&item.ItemNumber, &item.Seq, &createdAt, &updatedAt, &deletedAt,
&item.CollectionSlug, &item.CollectionName, &item.CollectionIcon, &item.CollectionPrefix,
&item.AssignedUserName, &item.AssignedUserEmail,
&item.AgentRoleName, &item.AgentRoleSlug, &item.AgentRoleIcon,
)
if err == nil {
item.Pinned = pinned
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
item.DeletedAt = parseTimePtr(deletedAt)
hydrateItemComputedMetadata(&item)
return &item, nil
}
if err != sql.ErrNoRows {
return nil, fmt.Errorf("resolve ref (include deleted): %w", err)
}
}
return s.GetItemBySlugIncludeDeleted(workspaceID, slugOrRef)
}
// parseItemRef parses "PREFIX-123" into ("PREFIX", 123, true).
// Returns false if the string is not a valid item ref.
// Case-insensitive: "task-5", "Task-5", and "TASK-5" all parse to ("TASK", 5, true).
func parseItemRef(s string) (string, int, bool) {
s = strings.ToUpper(s)
idx := strings.LastIndex(s, "-")
if idx <= 0 || idx == len(s)-1 {
return "", 0, false
}
prefix := s[:idx]
// Prefix must be all uppercase letters
for _, c := range prefix {
if c < 'A' || c > 'Z' {
return "", 0, false
}
}
numStr := s[idx+1:]
num := 0
for _, c := range numStr {
if c < '0' || c > '9' {
return "", 0, false
}
num = num*10 + int(c-'0')
}
if num == 0 {
return "", 0, false
}
return prefix, num, true
}
// parseItemNumber parses a bare numeric string (e.g. "843") into a positive
// item number. Returns false for empty strings, non-digit input, zero, or
// values exceeding a sane upper bound (999999 — items_workspace_number is
// workspace-global so this comfortably fits any real workspace).
//
// Used by Search() to support "type a number, get the item" — a workspace
// has at most one item with any given item_number (unique index on
// (workspace_id, item_number)) so this resolves to a single direct hit.
// See BUG-910.
func parseItemNumber(s string) (int, bool) {
s = strings.TrimSpace(s)
if s == "" {
return 0, false
}
num := 0
for _, c := range s {
if c < '0' || c > '9' {
return 0, false
}
num = num*10 + int(c-'0')
if num > 999999 {
return 0, false
}
}
if num == 0 {
return 0, false
}
return num, true
}
// GetItemIncludeDeleted finds an item by id including soft-deleted
// items. Used by code paths that need to act on records the user
// already owns even though the parent item has been moved to trash —
// the most common case is the Settings → Storage attachment list,
// where attachments survive a soft-deleted parent (so the user can
// see what's still consuming quota and decide whether to delete the
// blob). The visibility check still keys off the (still-set)
// collection_id, so soft-deleting an item doesn't escalate access.
func (s *Store) GetItemIncludeDeleted(id string) (*models.Item, error) {
return s.GetItemIncludeDeletedQ(s.db, id)
}
// GetItemIncludeDeletedQ is GetItemIncludeDeleted parameterized over its
// executor (see Queryer).
func (s *Store) GetItemIncludeDeletedQ(q Queryer, id string) (*models.Item, error) {
item, err := s.getItemScanQ(q, id, true)
if err != nil {
return nil, fmt.Errorf("get item (include deleted): %w", err)
}
return item, nil
}
// GetItemsByIDsIncludeDeleted fetches items (soft-deleted included) for the
// given IDs in ONE `WHERE id IN (...)` query, keyed by item ID. It replaces
// the per-row GetItemIncludeDeleted N+1 the dashboard's recent-activity
// enrichment used to run once per activity row (BUG-2002). The rich-text body
// is omitted — the only consumers read title/slug/ref/collection + the
// visibility inputs, never the markdown body. IDs with no matching row are
// simply absent from the map.
func (s *Store) GetItemsByIDsIncludeDeleted(ids []string) (map[string]*models.Item, error) {
result := make(map[string]*models.Item, len(ids))
if len(ids) == 0 {
return result, nil
}
placeholders := make([]string, len(ids))
args := make([]any, len(ids))
for i, id := range ids {
placeholders[i] = "?"
args[i] = id
}
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, '', i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at, i.deleted_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, '')
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.id IN (%s)
`, strings.Join(placeholders, ","))), args...)
if err != nil {
return nil, fmt.Errorf("get items by ids (include deleted): %w", err)
}
defer rows.Close()
for rows.Next() {
var item models.Item
var createdAt, updatedAt string
var deletedAt *string
var pinned bool
if err := rows.Scan(
&item.ID, &item.WorkspaceID, &item.CollectionID, &item.Title, &item.Slug,
&item.Content, &item.Fields, &item.Tags,
&pinned, &item.SortOrder, &item.ParentID, &item.AssignedUserID, &item.AgentRoleID, &item.RoleSortOrder,
&item.CreatedBy, &item.LastModifiedBy, &item.Source,
&item.ItemNumber, &item.Seq, &createdAt, &updatedAt, &deletedAt,
&item.CollectionSlug, &item.CollectionName, &item.CollectionIcon, &item.CollectionPrefix,
&item.AssignedUserName, &item.AssignedUserEmail,
&item.AgentRoleName, &item.AgentRoleSlug, &item.AgentRoleIcon,
); err != nil {
return nil, err
}
item.Pinned = pinned
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
item.DeletedAt = parseTimePtr(deletedAt)
hydrateItemComputedMetadata(&item)
itemCopy := item
result[item.ID] = &itemCopy
}
return result, rows.Err()
}
// GetItemBySlugIncludeDeleted finds an item by slug including soft-deleted items.
// Used for restore operations where the item is archived.
func (s *Store) GetItemBySlugIncludeDeleted(workspaceID, slug string) (*models.Item, error) {
var item models.Item
var createdAt, updatedAt string
var deletedAt *string
var pinned bool
err := s.db.QueryRow(s.q(`
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.content, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at, i.deleted_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, '')
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.workspace_id = ? AND i.slug = ?
`), workspaceID, slug).Scan(
&item.ID, &item.WorkspaceID, &item.CollectionID, &item.Title, &item.Slug,
&item.Content, &item.Fields, &item.Tags,
&pinned, &item.SortOrder, &item.ParentID, &item.AssignedUserID, &item.AgentRoleID, &item.RoleSortOrder,
&item.CreatedBy, &item.LastModifiedBy, &item.Source,
&item.ItemNumber, &item.Seq, &createdAt, &updatedAt, &deletedAt,
&item.CollectionSlug, &item.CollectionName, &item.CollectionIcon, &item.CollectionPrefix,
&item.AssignedUserName, &item.AssignedUserEmail,
&item.AgentRoleName, &item.AgentRoleSlug, &item.AgentRoleIcon,
)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("get item by slug (include deleted): %w", err)
}
item.Pinned = pinned
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
item.DeletedAt = parseTimePtr(deletedAt)
hydrateItemComputedMetadata(&item)
return &item, nil
}
func (s *Store) ListItems(workspaceID string, params models.ItemListParams) ([]models.Item, error) {
// Non-nil empty CollectionIDs means "no visible collections" — return
// empty results immediately, unless ItemIDs are also provided (item-level
// grants may still allow access to specific items even without full
// collection access).
if params.CollectionIDs != nil && len(params.CollectionIDs) == 0 && len(params.ItemIDs) == 0 {
return nil, nil
}
// When search is specified, use FTS. Whitespace-only input is treated as
// "no search filter" (would otherwise sanitize to empty and crash SQLite
// FTS5 with "syntax error near \"\"" — see BUG-818).
if strings.TrimSpace(params.Search) != "" {
return s.listItemsFTS(workspaceID, params)
}
// NoContent swaps the rich-text body column for an empty literal so
// count/summary scans (e.g. the dashboard builder) don't load every
// item's full markdown. scanItems still scans the same column count;
// item.Content just comes back empty (BUG-2002).
contentCol := "i.content"
if params.NoContent {
contentCol = "''"
}
query := `
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, ` + contentCol + `, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''), i.deleted_at
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.workspace_id = ?
`
args := []interface{}{workspaceID}
if !params.IncludeArchived {
query += " AND i.deleted_at IS NULL"
}
if params.CollectionSlug != "" {
query += " AND c.slug = ?"
args = append(args, params.CollectionSlug)
}
if len(params.CollectionIDs) > 0 && len(params.ItemIDs) > 0 {
// Guest with both collection-level and item-level grants:
// item must be in a fully-granted collection OR be a specifically granted item
collPlaceholders := make([]string, len(params.CollectionIDs))
for i, id := range params.CollectionIDs {
collPlaceholders[i] = "?"
args = append(args, id)
}
itemPlaceholders := make([]string, len(params.ItemIDs))
for i, id := range params.ItemIDs {
itemPlaceholders[i] = "?"
args = append(args, id)
}
query += " AND (i.collection_id IN (" + strings.Join(collPlaceholders, ",") + ") OR i.id IN (" + strings.Join(itemPlaceholders, ",") + "))"
} else if len(params.CollectionIDs) > 0 {
placeholders := make([]string, len(params.CollectionIDs))
for i, id := range params.CollectionIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.collection_id IN (" + strings.Join(placeholders, ",") + ")"
} else if len(params.ItemIDs) > 0 {
// Guest with only item-level grants (no collection-level grants)
placeholders := make([]string, len(params.ItemIDs))
for i, id := range params.ItemIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.id IN (" + strings.Join(placeholders, ",") + ")"
}
if params.Tag != "" {
tagExpr, tagArg := s.dialect.JSONArrayContains("i.tags", params.Tag)
query += " AND " + tagExpr
args = append(args, tagArg)
}
if params.ParentID != "" {
query += " AND i.parent_id = ?"
args = append(args, params.ParentID)
}
if params.Unparented {
query += " AND " + unparentedItemPredicate
}
if params.AssignedUserID != "" {
query += " AND i.assigned_user_id = ?"
args = append(args, params.AssignedUserID)
}
if params.AgentRoleID != "" {
query += " AND (i.agent_role_id = ? OR ar.slug = ?)"
args = append(args, params.AgentRoleID, params.AgentRoleID)
}
// Parent link filter via item_links. Joins items so we ignore links pointing
// to a soft-deleted parent — slug/ref filtering already rejects deleted
// parents upstream, but raw-UUID input bypasses that path. See BUG-734 /
// Codex review on PR #259.
if params.ParentLinkID != "" {
query += " AND EXISTS (SELECT 1 FROM item_links il JOIN items p ON p.id = il.target_id AND p.deleted_at IS NULL WHERE il.source_id = i.id AND il.link_type = 'parent' AND il.target_id = ?)"
args = append(args, params.ParentLinkID)
}
// Field filters — supports comma-separated values as OR
for key, value := range params.Fields {
// Sanitize the key to prevent SQL injection — field names must be
// alphanumeric/underscore only (user-controlled from query params).
if !isValidFieldKey(key) {
continue
}
jsonExpr := s.dialect.JSONExtractText("i.fields", key)
if strings.Contains(value, ",") {
values := strings.Split(value, ",")
placeholders := make([]string, len(values))
for i, v := range values {
placeholders[i] = "?"
args = append(args, strings.TrimSpace(v))
}
query += " AND " + jsonExpr + " IN (" + strings.Join(placeholders, ",") + ")"
} else {
query += " AND " + jsonExpr + " = ?"
args = append(args, value)
}
}
// Non-terminal filter (BUG-2001): keep only items whose resolved done
// field is NOT one of their collection's terminal options. Evaluated
// per-collection so custom status vocabularies work.
if params.NonTerminal {
clause, ntArgs := s.nonTerminalFilter(workspaceID, "i")
query += " AND " + clause
args = append(args, ntArgs...)
}
// Sorting
query += buildItemSort(params.Sort, s.dialect)
// Pagination
if params.Limit > 0 {
query += " LIMIT ?"
args = append(args, params.Limit)
if params.Offset > 0 {
query += " OFFSET ?"
args = append(args, params.Offset)
}
}
rows, err := s.db.Query(s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("list items: %w", err)
}
defer rows.Close()
return scanItems(rows)
}
// ListWorkspaceTags returns the distinct tags used across a workspace's items
// with the number of (non-archived) items carrying each, ordered by count
// desc then tag asc.
//
// collectionIDs / itemIDs are the same permission filters ListItems takes and
// carry identical semantics: nil collectionIDs means no restriction (admins /
// owners); a non-nil empty collectionIDs with no itemIDs means "no visible
// collections" and returns an empty result. When both are non-empty (a guest
// with collection- and item-level grants) the filters are OR'd, matching
// ListItems exactly so tag counts can never leak items the caller can't see.
func (s *Store) ListWorkspaceTags(workspaceID string, collectionIDs, itemIDs []string) ([]models.TagCount, error) {
if collectionIDs != nil && len(collectionIDs) == 0 && len(itemIDs) == 0 {
return []models.TagCount{}, nil
}
fromExpr, valueExpr := s.dialect.JSONArrayElements("i.tags", "je")
// COUNT(DISTINCT i.id), not COUNT(*): the contract is "items carrying the
// tag". The unnest produces one row per array element, so an item with
// duplicate tags (e.g. ["ux","ux"] — the write path doesn't enforce
// per-item uniqueness) would otherwise be counted twice.
query := `
SELECT ` + valueExpr + ` AS tag, COUNT(DISTINCT i.id) AS cnt
FROM items i, ` + fromExpr + `
WHERE i.workspace_id = ? AND i.deleted_at IS NULL`
args := []interface{}{workspaceID}
if len(collectionIDs) > 0 && len(itemIDs) > 0 {
collPlaceholders := make([]string, len(collectionIDs))
for i, id := range collectionIDs {
collPlaceholders[i] = "?"
args = append(args, id)
}
itemPlaceholders := make([]string, len(itemIDs))
for i, id := range itemIDs {
itemPlaceholders[i] = "?"
args = append(args, id)
}
query += " AND (i.collection_id IN (" + strings.Join(collPlaceholders, ",") + ") OR i.id IN (" + strings.Join(itemPlaceholders, ",") + "))"
} else if len(collectionIDs) > 0 {
placeholders := make([]string, len(collectionIDs))
for i, id := range collectionIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.collection_id IN (" + strings.Join(placeholders, ",") + ")"
} else if len(itemIDs) > 0 {
placeholders := make([]string, len(itemIDs))
for i, id := range itemIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.id IN (" + strings.Join(placeholders, ",") + ")"
}
query += " GROUP BY " + valueExpr + " ORDER BY cnt DESC, tag ASC"
rows, err := s.db.Query(s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("list workspace tags: %w", err)
}
defer rows.Close()
tags := []models.TagCount{}
for rows.Next() {
var tc models.TagCount
if err := rows.Scan(&tc.Tag, &tc.Count); err != nil {
return nil, fmt.Errorf("scan tag count: %w", err)
}
tags = append(tags, tc)
}
return tags, rows.Err()
}
// ItemIndexParams is the trimmed parameter set for ListItemsIndex.
// It deliberately omits sort/search/pagination/field-filter knobs that the
// "skinny projection" endpoint doesn't expose — the local-first read model
// fetches the entire workspace once and does its own client-side filtering.
type ItemIndexParams struct {
// CollectionSlug optionally restricts to a single collection by slug.
CollectionSlug string
// CollectionIDs is the permission filter for visible collections.
// nil = unfiltered. A non-nil empty slice means "no visible collections"
// and (combined with empty ItemIDs) returns an empty result immediately,
// matching ListItems semantics.
CollectionIDs []string
// ItemIDs additionally allows specific items through (item-level grants
// for guests / restricted members).
ItemIDs []string
// IncludeArchived returns soft-deleted items when true.
IncludeArchived bool
// IncludeUnparentedMetadata projects the structural is_unparented bit.
// Server handlers set this only for unrestricted callers.
IncludeUnparentedMetadata bool
}
// ListItemsIndex returns the skinny-projection of items in a workspace —
// every column EXCEPT i.content. Used by the local-first read model
// (PLAN-1343) so the client can hydrate an in-memory + IndexedDB index
// without paying the rich-text body cost.
//
// Deterministic sort: updated_at DESC, id ASC (stable tiebreaker so cursors
// over equal-timestamp items are reproducible).
func (s *Store) ListItemsIndex(workspaceID string, params ItemIndexParams) ([]models.Item, error) {
// Mirror ListItems: a non-nil empty CollectionIDs without item-level grants
// means "no visible collections" — return empty immediately.
if params.CollectionIDs != nil && len(params.CollectionIDs) == 0 && len(params.ItemIDs) == 0 {
return nil, nil
}
// `i.deleted_at` is in the projection so the local-first client
// (PLAN-1343 / TASK-1355) can distinguish archived rows hydrated
// with `IncludeArchived=true` from live rows. When the flag is
// false, the WHERE clause filters them out anyway; when it's
// true, the field is populated for the soft-deleted subset and
// nil for live rows. Mirrors the projection of
// ListItemsChangesSince which has always carried this column.
unparentedCol := "NULL"
if params.IncludeUnparentedMetadata {
unparentedCol = "(" + unparentedItemPredicate + ")"
}
query := `
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at, i.deleted_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''), ` + unparentedCol + `
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.workspace_id = ?
`
args := []interface{}{workspaceID}
if !params.IncludeArchived {
query += " AND i.deleted_at IS NULL"
}
if params.CollectionSlug != "" {
query += " AND c.slug = ?"
args = append(args, params.CollectionSlug)
}
if len(params.CollectionIDs) > 0 && len(params.ItemIDs) > 0 {
collPlaceholders := make([]string, len(params.CollectionIDs))
for i, id := range params.CollectionIDs {
collPlaceholders[i] = "?"
args = append(args, id)
}
itemPlaceholders := make([]string, len(params.ItemIDs))
for i, id := range params.ItemIDs {
itemPlaceholders[i] = "?"
args = append(args, id)
}
query += " AND (i.collection_id IN (" + strings.Join(collPlaceholders, ",") + ") OR i.id IN (" + strings.Join(itemPlaceholders, ",") + "))"
} else if len(params.CollectionIDs) > 0 {
placeholders := make([]string, len(params.CollectionIDs))
for i, id := range params.CollectionIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.collection_id IN (" + strings.Join(placeholders, ",") + ")"
} else if len(params.ItemIDs) > 0 {
placeholders := make([]string, len(params.ItemIDs))
for i, id := range params.ItemIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.id IN (" + strings.Join(placeholders, ",") + ")"
}
// Deterministic sort: most-recently-updated first, with id as a stable
// secondary key so equal-timestamp rows have a reproducible order.
query += " ORDER BY i.updated_at DESC, i.id ASC"
rows, err := s.db.Query(s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("list items index: %w", err)
}
defer rows.Close()
return scanItemsIndex(rows)
}
// ItemChangesParams is the parameter set for ListItemsChangesSince.
// Mirrors the visibility-filter half of ItemIndexParams (CollectionIDs
// / ItemIDs) plus the cursor-specific knobs Since / Limit.
type ItemChangesParams struct {
// CollectionIDs is the permission filter for visible collections.
// nil = unfiltered. A non-nil empty slice (with empty ItemIDs)
// short-circuits to an empty result, matching ListItemsIndex
// semantics.
CollectionIDs []string
// ItemIDs is the item-level grant set (guests / restricted
// members can see specific items even outside their collection
// scope).
ItemIDs []string
// Since is the exclusive seq lower bound (returns rows where
// `seq > since`).
Since int64
// Limit caps the returned slice. <=0 means use the default cap
// (DefaultItemChangesLimit); values above MaxItemChangesLimit
// are clamped.
Limit int
// IncludeUnparentedMetadata mirrors ItemIndexParams. Restricted callers
// leave it false so the optional bit is omitted from every delta row.
IncludeUnparentedMetadata bool
}
// DefaultItemChangesLimit is the default cap on /items-changes
// responses. 5,000 is enough to drain a typical workspace in one
// round-trip; clients that need more re-page via the cursor.
const DefaultItemChangesLimit = 5000
// MaxItemChangesLimit clamps any caller-supplied limit so a runaway
// `?limit=999999` poll can't materialize the entire workspace
// (think: months-offline tab).
const MaxItemChangesLimit = 50000
// ListItemsChangesSince returns the skinny-projection of items that
// have mutated (create / update / soft-delete / restore) since the
// given seq cursor, in ascending seq order. Soft-deleted rows ARE
// included so delta-sync clients can drop tombstoned items from
// their local index — the scan populates models.Item.DeletedAt for
// every row so the caller can distinguish upserts from deletes.
//
// Auth: same shape as ListItemsIndex — CollectionIDs / ItemIDs gate
// visibility. A delta from `since=0` over a fully-permitted scope
// matches the /items-index payload modulo ordering (changes is seq
// ASC, index is updated_at DESC).
//
// Returns at most Limit rows (default DefaultItemChangesLimit,
// capped at MaxItemChangesLimit). When truncated, the caller's
// next poll should pass the returned cursor's MAX(seq) as `since`
// to resume.
func (s *Store) ListItemsChangesSince(workspaceID string, params ItemChangesParams) ([]models.Item, error) {
if params.CollectionIDs != nil && len(params.CollectionIDs) == 0 && len(params.ItemIDs) == 0 {
return nil, nil
}
limit := params.Limit
if limit <= 0 {
limit = DefaultItemChangesLimit
}
if limit > MaxItemChangesLimit {
limit = MaxItemChangesLimit
}
// Same column list as scanItemsIndex plus deleted_at so the caller
// can distinguish upserts from tombstones. There is no
// `i.deleted_at IS NULL` filter here — that's the whole point of
// the delta: soft-deleted rows propagate so clients can remove
// them from their local index.
unparentedCol := "NULL"
if params.IncludeUnparentedMetadata {
unparentedCol = "(" + unparentedItemPredicate + ")"
}
query := `
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at, i.deleted_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''), ` + unparentedCol + `
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.workspace_id = ? AND i.seq > ?
`
args := []interface{}{workspaceID, params.Since}
if len(params.CollectionIDs) > 0 && len(params.ItemIDs) > 0 {
collPlaceholders := make([]string, len(params.CollectionIDs))
for i, id := range params.CollectionIDs {
collPlaceholders[i] = "?"
args = append(args, id)
}
itemPlaceholders := make([]string, len(params.ItemIDs))
for i, id := range params.ItemIDs {
itemPlaceholders[i] = "?"
args = append(args, id)
}
query += " AND (i.collection_id IN (" + strings.Join(collPlaceholders, ",") + ") OR i.id IN (" + strings.Join(itemPlaceholders, ",") + "))"
} else if len(params.CollectionIDs) > 0 {
placeholders := make([]string, len(params.CollectionIDs))
for i, id := range params.CollectionIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.collection_id IN (" + strings.Join(placeholders, ",") + ")"
} else if len(params.ItemIDs) > 0 {
placeholders := make([]string, len(params.ItemIDs))
for i, id := range params.ItemIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.id IN (" + strings.Join(placeholders, ",") + ")"
}
// Ascending seq is the canonical delta order: the next poll passes
// the response's MAX(seq) as `since` and resumes with no gap or
// overlap. The supporting index is (workspace_id, seq DESC)
// (TASK-1352 migration) — the engine can still use it for ASC
// scans, just walked in reverse.
query += " ORDER BY i.seq ASC LIMIT ?"
args = append(args, limit)
rows, err := s.db.Query(s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("list items changes: %w", err)
}
defer rows.Close()
return scanItemsChanges(rows)
}
// scanItemsChanges scans rows from ListItemsChangesSince (skinny
// projection + deleted_at so callers can distinguish tombstones).
func scanItemsChanges(rows *sql.Rows) ([]models.Item, error) {
var items []models.Item
for rows.Next() {
var item models.Item
var createdAt, updatedAt string
var deletedAt *string
var isUnparented sql.NullBool
var pinned bool
if err := rows.Scan(
&item.ID, &item.WorkspaceID, &item.CollectionID, &item.Title, &item.Slug,
&item.Fields, &item.Tags,
&pinned, &item.SortOrder, &item.ParentID, &item.AssignedUserID, &item.AgentRoleID, &item.RoleSortOrder,
&item.CreatedBy, &item.LastModifiedBy, &item.Source,
&item.ItemNumber, &item.Seq, &createdAt, &updatedAt, &deletedAt,
&item.CollectionSlug, &item.CollectionName, &item.CollectionIcon, &item.CollectionPrefix,
&item.AssignedUserName, &item.AssignedUserEmail,
&item.AgentRoleName, &item.AgentRoleSlug, &item.AgentRoleIcon, &isUnparented,
); err != nil {
return nil, err
}
item.Pinned = pinned
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
item.DeletedAt = parseTimePtr(deletedAt)
if isUnparented.Valid {
v := isUnparented.Bool
item.IsUnparented = &v
}
hydrateItemComputedMetadata(&item)
items = append(items, item)
}
return items, rows.Err()
}
// MovedOutRow is a minimal "this item left your view" signal returned
// by ListMovedOutSince — id + seq only, no title/fields/target so a
// caller learns an item disappeared from a collection they can see
// without leaking any data from the (invisible) destination.
type MovedOutRow struct {
ID string
Seq int64
}
// ListMovedOutSince finds items that the main /items-changes delta drops
// because they moved OUT of the caller's visible scope: their CURRENT
// collection is one the caller can't see (so the seq>since row is
// filtered out), yet item_collection_moves records that they left a
// collection the caller CAN see at a seq in this delta window. The
// caller has read access to that source collection, so signalling "id X
// left your view" is within their scope — and the returned row carries
// only id+seq, never any destination data (BUG-1675).
//
// Keyed on the MOVE's seq (item_collection_moves.seq), which is written
// in the SAME transaction as the move and never changes on later edits —
// so the tombstone fires once, pages deterministically, and can't be
// raced or lost by the best-effort activity log (Codex rounds 1–3). The
// per-move rows also handle multi-hop moves: only a move whose
// from-collection is visible qualifies, and MIN(seq) picks the earliest
// such visibility loss so the cursor can't skip it.
//
// collLevelVisibleIDs is the set of collections the caller browses at the
// collection level (same set used to filter the main delta).
// grantedItemIDs are excluded: an item the caller holds a direct grant on
// stays visible across a move (the grant transcends collection), so the
// main delta still delivers it and it must NOT be tombstoned.
//
// Returns nil for unrestricted callers (empty visible scope) — full
// members never lose visibility on a move, so this path is moot for them.
func (s *Store) ListMovedOutSince(
workspaceID string,
since int64,
limit int,
collLevelVisibleIDs []string,
grantedItemIDs []string,
) ([]MovedOutRow, error) {
if len(collLevelVisibleIDs) == 0 {
return nil, nil
}
if limit <= 0 {
limit = DefaultItemChangesLimit
}
if limit > MaxItemChangesLimit {
limit = MaxItemChangesLimit
}
// Earliest in-window move OUT of a visible source collection, per
// item that is currently NOT visible to the caller. Fully SQL +
// indexed — no JSON parsing, no Go-side filtering.
query := `
SELECT m.item_id, MIN(m.seq) AS move_seq
FROM item_collection_moves m
JOIN items i ON i.id = m.item_id
WHERE m.workspace_id = ? AND m.seq > ?
`
args := []interface{}{workspaceID, since}
vis := make([]string, len(collLevelVisibleIDs))
for i, id := range collLevelVisibleIDs {
vis[i] = "?"
args = append(args, id)
}
// Left a collection the caller CAN see...
query += " AND m.from_collection_id IN (" + strings.Join(vis, ",") + ")"
// ...and now lives in one they CAN'T.
vis2 := make([]string, len(collLevelVisibleIDs))
for i, id := range collLevelVisibleIDs {
vis2[i] = "?"
args = append(args, id)
}
query += " AND i.collection_id NOT IN (" + strings.Join(vis2, ",") + ")"
if len(grantedItemIDs) > 0 {
gph := make([]string, len(grantedItemIDs))
for i, id := range grantedItemIDs {
gph[i] = "?"
args = append(args, id)
}
query += " AND i.id NOT IN (" + strings.Join(gph, ",") + ")"
}
query += " GROUP BY m.item_id ORDER BY move_seq ASC LIMIT ?"
args = append(args, limit)
rows, err := s.db.Query(s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("list moved-out: %w", err)
}
defer rows.Close()
var out []MovedOutRow
for rows.Next() {
var r MovedOutRow
if err := rows.Scan(&r.ID, &r.Seq); err != nil {
return nil, err
}
out = append(out, r)
}
return out, rows.Err()
}
// MaxItemSeq returns the largest items.seq across the workspace, or 0
// if the workspace has no items. This is the cursor floor for the
// local-first read model (PLAN-1343 / TASK-1353): /items-index hands
// it back when its filtered result set is empty so the client can
// poll /items-changes?since=<cursor> against the workspace's true
// current position instead of restarting from 0.
//
// Soft-deleted items DO contribute to MAX(seq) — the seq column
// bumps on tombstone writes (DeleteItem) so a client's cursor must
// move past those events for the next /items-changes scan to skip
// them. Filtering by `deleted_at IS NULL` here would silently regress
// the cursor whenever the most recent mutation was a delete.
func (s *Store) MaxItemSeq(workspaceID string) (int64, error) {
var seq int64
err := s.db.QueryRow(s.q(`SELECT COALESCE(MAX(seq), 0) FROM items WHERE workspace_id = ?`), workspaceID).Scan(&seq)
if err != nil {
return 0, fmt.Errorf("max item seq: %w", err)
}
return seq, nil
}
// ItemCheckboxProgress is the per-item count of markdown checkboxes
// (`- [ ]` / `- [x]`) extracted from item content. Used by the
// collection page to render checklist progress badges without
// shipping the rich-text body over the wire (PLAN-1343 Phase 1 /
// TASK-1349).
type ItemCheckboxProgress struct {
ItemID string `json:"item_id"`
Total int `json:"total"`
Done int `json:"done"`
}
// checkboxCountSQL is the SQL fragment used to count `- [ ]` and
// `- [x]` markers inside item content. Implemented identically on
// SQLite and PostgreSQL via the LENGTH/REPLACE arithmetic trick —
// both dialects support LENGTH and REPLACE on TEXT, and integer
// division is identical.
//
// The `i.deleted_at` clause is appended dynamically in
// CollectionCheckboxProgress so callers can request progress for
// archived rows (matches /items-index's include_archived semantics).
const checkboxCountSQL = `
SELECT i.id,
(LENGTH(i.content) - LENGTH(REPLACE(i.content, '- [ ]', ''))) / 5
+ (LENGTH(i.content) - LENGTH(REPLACE(i.content, '- [x]', ''))) / 5 AS total,
(LENGTH(i.content) - LENGTH(REPLACE(i.content, '- [x]', ''))) / 5 AS done
FROM items i
WHERE i.workspace_id = ?
AND i.collection_id = ?
AND i.content LIKE '%- [%]%'
`
// CollectionCheckboxProgress returns the per-item checkbox totals for
// every item in a collection whose content has at least one
// `- [ ]` / `- [x]` marker. The query computes counts server-side via
// LENGTH/REPLACE arithmetic so the wire payload stays small (three
// ints per non-zero item) — much cheaper than shipping every item's
// rich-text body just so the client can grep for checkboxes.
//
// includeArchived controls whether soft-deleted items contribute
// rows. The default (false) matches the pre-existing client-side
// parse for the un-toggled view. With the page's Archived toggle
// on, the collection page renders archived items too — passing
// true preserves their progress badges (per Codex round 2 [P2] on
// PR #491).
//
// Items with no markers, or with non-positive totals after subtracting
// done from open, are filtered out. Result order is unspecified.
func (s *Store) CollectionCheckboxProgress(workspaceID, collectionID string, includeArchived bool) ([]ItemCheckboxProgress, error) {
query := checkboxCountSQL
if !includeArchived {
query += " AND i.deleted_at IS NULL"
}
rows, err := s.db.Query(s.q(query), workspaceID, collectionID)
if err != nil {
return nil, fmt.Errorf("collection checkbox progress: %w", err)
}
defer rows.Close()
var result []ItemCheckboxProgress
for rows.Next() {
var p ItemCheckboxProgress
if err := rows.Scan(&p.ItemID, &p.Total, &p.Done); err != nil {
return nil, err
}
// Skip rows with no checkboxes — the LIKE filter is a fast
// preliminary check, but item bodies can contain the substring
// inside a code block or other context that doesn't end up as
// a markdown checkbox; the per-row Total accounts for that.
if p.Total <= 0 {
continue
}
result = append(result, p)
}
return result, rows.Err()
}
// scanItemsIndex scans rows from ListItemsIndex (skinny projection — no
// i.content column).
func scanItemsIndex(rows *sql.Rows) ([]models.Item, error) {
var items []models.Item
for rows.Next() {
var item models.Item
var createdAt, updatedAt string
var deletedAt *string
var isUnparented sql.NullBool
var pinned bool
if err := rows.Scan(
&item.ID, &item.WorkspaceID, &item.CollectionID, &item.Title, &item.Slug,
&item.Fields, &item.Tags,
&pinned, &item.SortOrder, &item.ParentID, &item.AssignedUserID, &item.AgentRoleID, &item.RoleSortOrder,
&item.CreatedBy, &item.LastModifiedBy, &item.Source,
&item.ItemNumber, &item.Seq, &createdAt, &updatedAt, &deletedAt,
&item.CollectionSlug, &item.CollectionName, &item.CollectionIcon, &item.CollectionPrefix,
&item.AssignedUserName, &item.AssignedUserEmail,
&item.AgentRoleName, &item.AgentRoleSlug, &item.AgentRoleIcon, &isUnparented,
); err != nil {
return nil, err
}
item.Pinned = pinned
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
if deletedAt != nil {
t := parseTime(*deletedAt)
item.DeletedAt = &t
}
if isUnparented.Valid {
v := isUnparented.Bool
item.IsUnparented = &v
}
hydrateItemComputedMetadata(&item)
items = append(items, item)
}
return items, rows.Err()
}
func (s *Store) listItemsFTS(workspaceID string, params models.ItemListParams) ([]models.Item, error) {
var query string
var args []interface{}
var ftsRank string
if s.dialect.Driver() == DriverPostgres {
// PostgreSQL: search_vector lives on the items table (aliased as "i").
ftsMatch := s.dialect.FTSMatch("i", "search_vector")
ftsRank = s.dialect.FTSRank("i", "search_vector")
query = fmt.Sprintf(`
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.content, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''), i.deleted_at
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.workspace_id = ? AND i.deleted_at IS NULL
AND %s
`, ftsMatch)
// PG FTSMatch consumes TWO args: the raw user query AND its
// hyphen-sanitized form, OR-combined inside the SQL fragment so
// that hyphenated terms like `task-five` match titles indexed as
// `task-five-distinctive` while preserving `BUG-842`-style
// matches (BUG-842).
args = []interface{}{workspaceID, params.Search, sanitizePGFTSQuery(params.Search)}
} else {
// SQLite: uses FTS5 virtual table "items_fts".
ftsMatch := s.dialect.FTSMatch("items_fts", "search_vector")
ftsRank = s.dialect.FTSRank("items_fts", "search_vector")
query = fmt.Sprintf(`
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.content, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''), i.deleted_at
FROM items i
JOIN items_fts fts ON i.rowid = fts.rowid
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.workspace_id = ? AND i.deleted_at IS NULL
AND %s
`, ftsMatch)
// Wrap each whitespace-delimited token in double quotes so FTS5 treats
// hyphens (and other special chars like AND/OR/NOT/(/)) as literals
// rather than boolean operators. Without this, `?search=TASK-5` raises
// "no such column: 5" — see BUG-818. Postgres handles raw input via
// the OR-combined plainto_tsquery in the dialect (BUG-842).
args = []interface{}{workspaceID, sanitizeFTSQuery(params.Search)}
}
if params.CollectionSlug != "" {
query += " AND c.slug = ?"
args = append(args, params.CollectionSlug)
}
// Parent link filter — mirrors the non-FTS path so combining
// `parent=<UUID>&search=<q>` doesn't silently drop the parent constraint
// (and, by extension, the soft-deleted-parent rejection from BUG-734).
if params.ParentLinkID != "" {
query += " AND EXISTS (SELECT 1 FROM item_links il JOIN items p ON p.id = il.target_id AND p.deleted_at IS NULL WHERE il.source_id = i.id AND il.link_type = 'parent' AND il.target_id = ?)"
args = append(args, params.ParentLinkID)
}
if len(params.CollectionIDs) > 0 && len(params.ItemIDs) > 0 {
collPlaceholders := make([]string, len(params.CollectionIDs))
for i, id := range params.CollectionIDs {
collPlaceholders[i] = "?"
args = append(args, id)
}
itemPlaceholders := make([]string, len(params.ItemIDs))
for i, id := range params.ItemIDs {
itemPlaceholders[i] = "?"
args = append(args, id)
}
query += " AND (i.collection_id IN (" + strings.Join(collPlaceholders, ",") + ") OR i.id IN (" + strings.Join(itemPlaceholders, ",") + "))"
} else if len(params.CollectionIDs) > 0 {
placeholders := make([]string, len(params.CollectionIDs))
for i, id := range params.CollectionIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.collection_id IN (" + strings.Join(placeholders, ",") + ")"
} else if len(params.ItemIDs) > 0 {
placeholders := make([]string, len(params.ItemIDs))
for i, id := range params.ItemIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND i.id IN (" + strings.Join(placeholders, ",") + ")"
}
// Filter parity with the non-FTS path. Without these, `?search=...` combined
// with any of these filter params silently drops the filter and over-returns
// items. See BUG-812.
if params.Tag != "" {
tagExpr, tagArg := s.dialect.JSONArrayContains("i.tags", params.Tag)
query += " AND " + tagExpr
args = append(args, tagArg)
}
if params.ParentID != "" {
query += " AND i.parent_id = ?"
args = append(args, params.ParentID)
}
if params.Unparented {
query += " AND " + unparentedItemPredicate
}
if params.AssignedUserID != "" {
query += " AND i.assigned_user_id = ?"
args = append(args, params.AssignedUserID)
}
if params.AgentRoleID != "" {
query += " AND (i.agent_role_id = ? OR ar.slug = ?)"
args = append(args, params.AgentRoleID, params.AgentRoleID)
}
// Field filters — supports comma-separated values as OR. Field keys are
// user-controlled (query params), so isValidFieldKey gates SQL composition.
for key, value := range params.Fields {
if !isValidFieldKey(key) {
continue
}
jsonExpr := s.dialect.JSONExtractText("i.fields", key)
if strings.Contains(value, ",") {
values := strings.Split(value, ",")
placeholders := make([]string, len(values))
for i, v := range values {
placeholders[i] = "?"
args = append(args, strings.TrimSpace(v))
}
query += " AND " + jsonExpr + " IN (" + strings.Join(placeholders, ",") + ")"
} else {
query += " AND " + jsonExpr + " = ?"
args = append(args, value)
}
}
// Non-terminal filter (BUG-2001) — parity with the non-FTS path so a
// `search + non_terminal` combination hides terminal items per each
// collection's own terminal_options.
if params.NonTerminal {
clause, ntArgs := s.nonTerminalFilter(workspaceID, "i")
query += " AND " + clause
args = append(args, ntArgs...)
}
// SQLite bm25(): more negative = more relevant → ASC (default).
// PostgreSQL ts_rank(): higher = more relevant → DESC.
// PG FTSRank embeds the same OR-combined plainto_tsquery as FTSMatch
// and consumes TWO args (raw + hyphen-sanitized) — BUG-842.
if s.dialect.Driver() == DriverPostgres {
query += " ORDER BY " + ftsRank + " DESC"
args = append(args, params.Search, sanitizePGFTSQuery(params.Search))
} else {
query += " ORDER BY " + ftsRank
}
if params.Limit > 0 {
query += " LIMIT ?"
args = append(args, params.Limit)
}
rows, err := s.db.Query(s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("search items: %w", err)
}
defer rows.Close()
return scanItems(rows)
}
func (s *Store) UpdateItem(id string, input models.ItemUpdate, opts ...MutationOption) (*models.Item, error) {
return s.UpdateItemWithPreCheck(id, input, nil, opts...)
}
// UpdateConflictError is returned by UpdateItem when the caller supplied
// ItemUpdate.ExpectedUpdatedAt and it no longer matches the item's current
// updated_at — another writer changed the row first (TASK-2022,
// optimistic concurrency). The check runs under the same write lock as the
// mutation, so a matching timestamp is a genuine guarantee that nothing
// slipped in between. The handler maps this to a pad-structured-error/v1
// conflict envelope (HTTP 409, code "update_conflict").
type UpdateConflictError struct {
ItemID string
ExpectedUpdatedAt string
ActualUpdatedAt time.Time
}
func (e *UpdateConflictError) Error() string {
return fmt.Sprintf(
"item %s was modified by another writer (expected updated_at %s, actual %s)",
e.ItemID, e.ExpectedUpdatedAt, e.ActualUpdatedAt.UTC().Format(time.RFC3339),
)
}
// mergeFieldsPatch applies a shallow JSON-merge-patch (RFC 7396 semantics,
// one level deep) of `patch` onto the item's current fields JSON (IDEA-1480
// / TASK-2022). A key mapped to nil (JSON null) DELETES that key; any other
// value sets it; every key absent from the patch is preserved verbatim.
// Returns the re-marshaled fields JSON.
//
// Pad fields are flat scalars (select/text/date/number/checkbox), so a
// shallow merge is the entire contract — we deliberately do NOT recurse
// into nested objects the way full RFC 7396 would, because no Pad field is
// itself an object whose sub-keys need independent patching.
func mergeFieldsPatch(currentJSON string, patch map[string]interface{}) (string, error) {
m := map[string]interface{}{}
if currentJSON != "" && currentJSON != "{}" {
if err := json.Unmarshal([]byte(currentJSON), &m); err != nil {
return "", fmt.Errorf("parse current fields for merge: %w", err)
}
}
for k, v := range patch {
if v == nil {
delete(m, k)
continue
}
m[k] = v
}
out, err := json.Marshal(m)
if err != nil {
return "", fmt.Errorf("marshal merged fields: %w", err)
}
return string(out), nil
}
// ParentLinkUpdate describes an optional parent-link mutation to apply
// ATOMICALLY inside UpdateItem's transaction (BUG-2013). The handler used
// to run SetParentLink/ClearParentLink as a separate write AFTER the field
// update committed, so a failing link write left the item half-updated and
// returned a 500. Folding the mutation into the same tx makes the pair
// all-or-nothing.
//
// - Provided=false → no parent-link change (the common case).
// - Provided=true, ParentID!="" → set the parent to ParentID.
// - Provided=true, ParentID=="" → clear the parent link.
type ParentLinkUpdate struct {
Provided bool
ParentID string
WorkspaceID string
CreatedBy string
}
// UpdateItemWithPreCheck is UpdateItem with an optional pre-mutation
// hook that runs inside the same transaction (and, on Postgres, holds
// the same workspace advisory lock) as the update itself. Callers can
// use the hook to enforce cross-row invariants whose decision must be
// atomic with the write — e.g. the open-children guard (IDEA-1494)
// needs the children-list query and the parent's status flip to share
// a tx so a concurrent child insert / child status change can't slip
// between them.
//
// The hook receives the transaction and the freshly-read existing
// item. Returning a non-nil error rolls the tx back and surfaces the
// error verbatim — callers can return a sentinel and `errors.Is` it
// in the handler.
//
// Pass a nil precheck for the standard, unchecked update path.
func (s *Store) UpdateItemWithPreCheck(
id string,
input models.ItemUpdate,
precheck func(tx *sql.Tx, existing *models.Item) error,
opts ...MutationOption,
) (*models.Item, error) {
return s.UpdateItemWithParentLink(id, input, precheck, nil, opts...)
}
// UpdateItemWithParentLink is UpdateItemWithPreCheck plus an OPTIONAL
// parent-link mutation applied inside the same transaction (BUG-2013).
// When parentLink is non-nil and Provided, the SetParentLink/ClearParentLink
// write runs after the field update but BEFORE commit — so if the link write
// fails (cycle, DB error), the field update rolls back too. No more
// 500-with-half-the-patch-applied.
//
// Pass a nil parentLink for the standard update path (equivalent to
// UpdateItemWithPreCheck).
//
// BUG-2073: wrapped in retryOnParentSetChanged so that if a concurrent
// reparent moves the item's parent set during lock acquisition, the whole
// transaction rolls back and retries from a fresh read (the retry folds the
// moved parent into the initial sorted lock batch, avoiding an out-of-order
// grab). The body commits nothing before the locks are held, so a retry can't
// leave partial state.
func (s *Store) UpdateItemWithParentLink(
id string,
input models.ItemUpdate,
precheck func(tx *sql.Tx, existing *models.Item) error,
parentLink *ParentLinkUpdate,
opts ...MutationOption,
) (*models.Item, error) {
opt := newMutationOptions(opts)
return retryOnParentSetChanged(func() (*models.Item, error) {
return s.updateItemWithParentLinkOnce(id, input, precheck, parentLink, opt)
})
}
func (s *Store) updateItemWithParentLinkOnce(
id string,
input models.ItemUpdate,
precheck func(tx *sql.Tx, existing *models.Item) error,
parentLink *ParentLinkUpdate,
opt mutationOptions,
) (*models.Item, error) {
existing, err := s.GetItem(id)
if err != nil {
return nil, err
}
if existing == nil {
return nil, nil
}
// Validate assignment scope before writing
if err := s.validateAssignmentScope(existing.WorkspaceID, input.AssignedUserID, input.AgentRoleID); err != nil {
return nil, err
}
// mutSignal accumulates the race-free status/assignment delta this call
// produces (TASK-2533 / models.ItemMutationSignal). Populated below,
// right alongside the status_transitions write and the final in-tx
// re-read, then attached to the returned item only if something
// actually changed.
var mutSignal models.ItemMutationSignal
tx, err := s.db.Begin()
if err != nil {
return nil, err
}
defer tx.Rollback()
// BUG-2074: when this update ADDS a parent edge (parentLink sets a
// non-empty ParentID), take the workspace-scoped parent-link cycle lock
// FIRST — before the seq lock and the parent-children batch — so the
// cycle walk in setParentLinkTx (applied later in this tx) runs against a
// consistent, non-racing ancestor snapshot and catches N-hop cycles that
// the per-endpoint lock set can't cover. Acquired only for edge-ADDING
// updates so plain field updates / status flips / clear-parent stay off
// this serialization point. Outermost acquisition keeps the global order
// cycle -> seq -> parent-children (see acquireWorkspaceParentLinkLock).
if parentLink != nil && parentLink.Provided && parentLink.ParentID != "" {
if err := s.acquireWorkspaceParentLinkLock(tx, existing.WorkspaceID); err != nil {
return nil, err
}
}
// Serialize concurrent seq assignments per workspace on Postgres
// (no-op on SQLite). Held until COMMIT / ROLLBACK.
if err := s.acquireWorkspaceSeqLock(tx, existing.WorkspaceID); err != nil {
return nil, err
}
// IDEA-1494 round 2: also acquire the parent-children advisory
// lock for THIS item (as a potential parent) AND for its own
// parent (when it is itself a child). That gives the open-children
// guard a tight serialization:
//
// - A parent's UpdateItem precheck holds `pad:parent-children:<parent_id>`
// while reading the children list and writing the parent.
// - A child's UpdateItem holds the same key for its parent
// while it writes itself.
//
// Result: a child status-flip that would invalidate the parent's
// guard cannot interleave between the parent's children-read and
// the parent's status-write. SQLite gets this for free from
// BEGIN IMMEDIATE; Postgres needs the explicit advisory lock.
//
// Lock ordering: workspace lock → THIS item's parent lock → THIS
// item's own children lock. Both lock keys are namespaced under
// `pad:parent-children:` so they only contend on the parent ID;
// acquiring two distinct keys in a fixed order can't deadlock.
//
// BUG-2013: when this update also re-parents the item, fold the NEW
// parent's key into this same sorted acquisition so setParentLinkTx's
// later (idempotent) re-lock never introduces an out-of-order grab.
var extraLockKeys []string
if parentLink != nil && parentLink.Provided && parentLink.ParentID != "" {
extraLockKeys = append(extraLockKeys, parentLink.ParentID)
}
if err := s.acquireParentChildrenLocksForUpdate(tx, id, extraLockKeys...); err != nil {
return nil, err
}
// IDEA-1494 round 2: run the caller's invariant check (if any)
// AFTER the locks are held but BEFORE any mutation. Closing the
// guard-vs-write TOCTOU window relies on this ordering — the
// precheck's view of `items` / `item_links` is the same one the
// UPDATE below will write against because every concurrent
// UpdateItem on this parent (or on any of its children) blocks
// on the same advisory key.
//
// Codex round-3 P2: re-read the item INSIDE the tx (after locks)
// and pass that fresh snapshot to the precheck. The pre-tx
// `existing` above was loaded without holding the workspace seq
// lock or the parent-children lock — a concurrent writer could
// have flipped the parent's done-field between that read and
// here, which would mis-classify the transition (false-fire or
// false-skip). The post-lock re-read sees what the UPDATE will
// write against.
// TASK-2022 / IDEA-1494: the optimistic-concurrency guard, the
// open-children precheck, and the field-level merge all need the item's
// state as seen UNDER the write lock — the pre-tx `existing` was read
// before the locks, so a concurrent writer could have superseded it.
//
// TASK-2533 codex round 2 finding 4: this used to be conditional
// (precheck != nil || ExpectedUpdatedAt != "" || FieldsPatch != nil),
// which left `existing` as the STALE pre-tx snapshot for any update
// that touched none of those three — including a plain assignment
// change. The status-transition capture below defended against this
// itself with its OWN separate re-read (conditional on precheck ==
// nil), but the LastMutation assignment-delta capture (further down)
// used `existing.AssignedUserID` directly with no such guard: a
// concurrent OTHER transaction's assignment change landing between
// this transaction's pre-tx read and its lock acquisition could get
// misattributed to THIS transaction (spurious/duplicate
// AssignmentChanged for an update that never touched assignment at
// all), or the reverse (a real change this transaction DID make
// compared against the wrong prior value). Re-reading UNCONDITIONALLY
// here — once, right after the locks are held and before any SET-
// clause building or the UPDATE itself — closes that for every
// existing.* comparison in this function at once, not just the ones
// that happen to remember to guard themselves. The extra SELECT is
// one row, under locks this function already holds; correctness here
// is worth more than skipping it in the common case.
{
fresh, ferr := s.getItemTx(tx, id)
if ferr != nil {
return nil, fmt.Errorf("re-read item under lock: %w", ferr)
}
if fresh == nil {
// Item was deleted between the pre-tx read and the post-lock
// re-read. Treat as not-found and let the handler surface a 404.
return nil, nil
}
existing = fresh
}
// Optimistic-concurrency guard runs FIRST — before the open-children
// precheck — so a caller who lost the race gets the promised
// update_conflict envelope, not a semantic (e.g. open_children)
// rejection for a transition attempted against a row that is no longer
// the one they read (Codex round 2). Parsed as RFC3339 and compared with
// time.Equal so a round-tripped `updated_at` matches regardless of
// zone/format.
if input.ExpectedUpdatedAt != "" {
expected, perr := time.Parse(time.RFC3339, input.ExpectedUpdatedAt)
if perr != nil {
return nil, fmt.Errorf("invalid expected_updated_at %q: %w", input.ExpectedUpdatedAt, perr)
}
if !existing.UpdatedAt.Equal(expected) {
return nil, &UpdateConflictError{
ItemID: id,
ExpectedUpdatedAt: input.ExpectedUpdatedAt,
ActualUpdatedAt: existing.UpdatedAt,
}
}
}
// IDEA-1494 round 2: run the caller's invariant check (open-children
// guard) against the same in-tx snapshot the UPDATE will write. Closing
// the guard-vs-write TOCTOU window relies on this ordering — every
// concurrent UpdateItem on this parent (or its children) blocks on the
// same advisory key, so the precheck's view of `items` / `item_links`
// is the one the UPDATE below mutates.
if precheck != nil {
if err := precheck(tx, existing); err != nil {
return nil, err
}
}
// Field-level merge (IDEA-1480): fold the caller's field patch onto the
// locked row's current fields and hand the result to the rest of the
// function as if it were a full `fields` replace. Because the base is the
// in-tx snapshot, two concurrent single-field patches serialize behind
// the workspace/parent locks and can't clobber each other. The handler
// guarantees Fields and FieldsPatch are never both set.
if input.FieldsPatch != nil {
merged, mErr := mergeFieldsPatch(existing.Fields, input.FieldsPatch)
if mErr != nil {
return nil, mErr
}
input.Fields = &merged
}
ts := now()
// Create version if content is changing
if input.Content != nil && *input.Content != existing.Content {
createdBy := input.LastModifiedBy
if createdBy == "" {
createdBy = "user"
}
// VersionSource takes precedence so the per-version-row
// attribution can differ from the (persisted) item source.
// See ItemUpdate.VersionSource doc comment + TASK-1267.
source := input.VersionSource
if source == "" {
source = input.Source
}
if source == "" {
source = "web"
}
// ForceVersion (e.g. a version restore) and a title change both bypass
// the per-(actor, source) throttle so a bracketing snapshot is always
// written when content changes — otherwise a throttled write would move
// items.content forward with no version anchoring the reverse-patch chain.
// ForceVersion can mint same-second versions; item_versions ordering
// uses version_seq (a per-item monotonic counter, BUG-2270) to break
// second-precision created_at ties, so rapid forced versions resolve
// in deterministic insertion order. See migration 076/054.
forceVersion := input.ForceVersion || (input.Title != nil && *input.Title != existing.Title)
shouldVersion := forceVersion
if !shouldVersion {
shouldVersion, err = s.shouldCreateItemVersion(id, createdBy, source)
if err != nil {
return nil, fmt.Errorf("check version throttle: %w", err)
}
}
if shouldVersion {
vid := newID()
versionContent := existing.Content
isDiff := false
patch := diff.CreateReversePatch(existing.Content, *input.Content)
if diff.IsDiffSmaller(patch, existing.Content) {
versionContent = patch
isDiff = true
}
// version_seq is a per-item monotonic tie-breaker (BUG-2270):
// same-second versions (a restore plus rapid edits) tie on
// created_at, so COALESCE(MAX,0)+1 gives a deterministic order.
// Race-safe because version creation is serialized per item
// under the item lock (this runs in the update's own tx).
_, err = tx.Exec(s.q(`
INSERT INTO item_versions (id, item_id, content, change_summary, created_by, source, is_diff, created_at, version_seq)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, (SELECT COALESCE(MAX(version_seq), 0) + 1 FROM item_versions WHERE item_id = ?))
`), vid, id, versionContent, input.ChangeSummary, createdBy, source, s.dialect.BoolToInt(isDiff), ts, id)
if err != nil {
return nil, fmt.Errorf("create version: %w", err)
}
}
}
// Build update query. Every mutation bumps seq to MAX(seq)+1 per
// workspace; the local-first read model uses that as a cursor (see
// nextWorkspaceSeqSubquery / PLAN-1343).
sets := []string{"updated_at = ?", "seq = " + nextWorkspaceSeqSubquery}
args := []interface{}{ts, existing.WorkspaceID}
if input.Title != nil {
sets = append(sets, "title = ?")
args = append(args, *input.Title)
baseSlug := slugify(*input.Title)
if baseSlug == "" {
baseSlug = "untitled"
}
newSlug, err := s.uniqueSlugExcluding("items", "workspace_id", existing.WorkspaceID, baseSlug, id)
if err != nil {
return nil, fmt.Errorf("unique slug: %w", err)
}
sets = append(sets, "slug = ?")
args = append(args, newSlug)
}
if input.Content != nil {
sets = append(sets, "content = ?")
args = append(args, *input.Content)
// Bump the human-readable timestamp on every content
// update — this is informational and never gates GC.
sets = append(sets, "content_flushed_at = ?")
args = append(args, ts)
// Op-log GC watermark policy (TASK-1309 round 5 [P1]):
// only advance content_flushed_op_log_id from server-driven
// full-content writes (CLI / MCP / applier-direct-write /
// version-restore / PruneAndApply). Browser collab-snapshot
// PATCHes are NOT eligible because they can't prove their
// markdown captures every peer op:
//
// Tab A's Y.Doc is at op N. Peer B's op N+1 commits to
// the op-log. Tab A's 5s flush PATCHes stale markdown
// derived from its op-N view. If we advanced the watermark
// to MAX(op-log.id) = N+1 here, the sweeper would later
// prune op N+1 — the only durable copy of peer B's edit.
//
// VersionSource == "collab-snapshot" is the marker the
// HTTP handler sets for browser flushes (TASK-1267). All
// other content writes either rebuild content from full
// op-log state (PruneAndApply) or replace it wholesale
// (CLI / version restore) — those CAN safely stamp the
// watermark to MAX(op-log.id) at write time.
//
// **Cursor-gated browser flush** (TASK-1319). Browser
// flushes carry an OpLogCursor recording the highest
// op-log id their Y.Doc has applied. When that cursor
// equals the current MAX(item_yjs_updates.id), the
// flusher has demonstrably captured every persisted op
// and we advance the watermark to that id. When the
// cursor is below MAX, peer ops outside the flusher's
// view exist; the watermark stays put so the GC sweeper
// cannot delete them. When the cursor is missing (older
// clients, malformed bodies) we behave as before — no
// advancement.
if input.VersionSource != "collab-snapshot" {
sets = append(sets, "content_flushed_op_log_id = (SELECT COALESCE(MAX(id), 0) FROM item_yjs_updates WHERE item_id = ?)")
args = append(args, id)
} else if input.OpLogCursor != nil {
// Conditional advance: the SQL UPDATE stamps the
// caller's cursor IFF that cursor still matches the
// current MAX(op-log.id) at COMMIT time. A peer op
// that lands between the client computing its
// cursor and this UPDATE running causes MAX to be
// strictly greater than the cursor, the predicate
// fails, and the watermark expression evaluates to
// the existing column value (a no-op). Never
// regresses, never over-advances.
sets = append(sets,
"content_flushed_op_log_id = CASE "+
"WHEN ? = (SELECT COALESCE(MAX(id), 0) FROM item_yjs_updates WHERE item_id = ?) "+
"THEN ? "+
"ELSE content_flushed_op_log_id "+
"END")
args = append(args, *input.OpLogCursor, id, *input.OpLogCursor)
}
}
if input.Fields != nil {
// IDEA-1486: normalize the empty-string sentinel to a valid JSON
// object before writing. After the NOT NULL DEFAULT '{}'
// hardening, Postgres rejects "" at JSONB type-validation and
// SQLite would silently store invalid JSON. Same boundary
// normalization as CreateItem (items.go:103-110) and the
// IDEA-1484 precedent at collections.go:248. Shape validation
// (object vs. array vs. primitive) is handled at the handler
// boundary by ItemUpdate.UnmarshalJSON (BUG-1144).
fields := *input.Fields
if fields == "" {
fields = "{}"
}
sets = append(sets, "fields = ?")
args = append(args, fields)
}
if input.Tags != nil {
// IDEA-1486: same empty-string coercion as fields above, but
// tags is array-shaped so the default is "[]". Mirrors
// CreateItem at items.go:107-110.
tags := *input.Tags
if tags == "" {
tags = "[]"
}
sets = append(sets, "tags = ?")
args = append(args, tags)
}
if input.Pinned != nil {
sets = append(sets, "pinned = ?")
args = append(args, s.dialect.BoolToInt(*input.Pinned))
}
if input.SortOrder != nil {
sets = append(sets, "sort_order = ?")
args = append(args, *input.SortOrder)
}
if input.ParentID != nil {
sets = append(sets, "parent_id = ?")
args = append(args, *input.ParentID)
}
// An explicit empty string clears the assignment, same as
// ClearAssignedUser / ClearAgentRole: it's what a JSON client sends
// when a user blanks the field, validateAssignmentScope already
// treats "" as "nothing to validate", and binding it verbatim would
// hit the FK instead of writing NULL (BUG-2566).
if input.AssignedUserID != nil && *input.AssignedUserID != "" {
sets = append(sets, "assigned_user_id = ?")
args = append(args, *input.AssignedUserID)
} else if input.ClearAssignedUser || (input.AssignedUserID != nil && *input.AssignedUserID == "") {
sets = append(sets, "assigned_user_id = NULL")
}
if input.AgentRoleID != nil && *input.AgentRoleID != "" {
sets = append(sets, "agent_role_id = ?")
args = append(args, *input.AgentRoleID)
} else if input.ClearAgentRole || (input.AgentRoleID != nil && *input.AgentRoleID == "") {
sets = append(sets, "agent_role_id = NULL")
}
if input.LastModifiedBy != "" {
sets = append(sets, "last_modified_by = ?")
args = append(args, input.LastModifiedBy)
}
if input.Source != "" {
sets = append(sets, "source = ?")
args = append(args, input.Source)
}
// Capture the pre-update status BEFORE the UPDATE runs, so the
// transition log below records an accurate from_status. `existing` is
// now UNCONDITIONALLY the locked, in-tx snapshot (see the re-read
// above, widened by TASK-2533 codex round 2 finding 4 to cover every
// existing.* comparison in this function, not just this one) — no
// separate defensive re-read needed here anymore. Reading here,
// before the UPDATE, is essential: a re-read after the UPDATE would
// see the new status and the hop would vanish.
var statusBefore, doneKey string
if input.Fields != nil {
doneKey = s.doneFieldKey(existing.CollectionID)
statusBefore = extractFieldValue(existing.Fields, doneKey)
}
// Stamp pad-attachment: references carried by the new content /
// fields BEFORE the UPDATE (see the ORDERING note on
// stampAttachmentRefsTx — the stamp's row locks make a concurrent
// GC claim wait out this tx). Covers every funnel into this core:
// item PATCH, the collab-snapshot flush, version restore, and bulk
// updates. input.Fields is already the RESOLVED blob here (a
// fields_patch was merged into it above).
{
var refTexts []string
if input.Content != nil {
refTexts = append(refTexts, *input.Content)
}
if input.Fields != nil {
refTexts = append(refTexts, *input.Fields)
}
if len(refTexts) > 0 {
if err := stampAttachmentRefsTx(tx, s, existing.WorkspaceID, refTexts...); err != nil {
return nil, err
}
}
}
args = append(args, id)
query := fmt.Sprintf("UPDATE items SET %s WHERE id = ?", strings.Join(sets, ", "))
_, err = tx.Exec(s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("update item: %w", err)
}
// Durable restore boundary (BUG-2264): stamp last_restore_seq with the seq
// this UPDATE just assigned. A second statement (not a SET on the UPDATE
// above) because seq is computed there via nextWorkspaceSeqSubquery, so
// `last_restore_seq = seq` in the same statement would read the OLD seq; a
// follow-up UPDATE in the SAME tx reads the freshly-committed-to-row value.
// Atomic with the content write + op-log prune (all in this one tx), so the
// boundary can never be torn from the restore it fences.
if input.MarkRestoreBoundary {
if _, err = tx.Exec(s.q("UPDATE items SET last_restore_seq = seq WHERE id = ?"), id); err != nil {
return nil, fmt.Errorf("stamp restore boundary: %w", err)
}
}
// Record a structured status transition when the fields blob was part
// of this update AND the `status` value actually changed. Written in
// the same tx as the item UPDATE so the transition log can never
// diverge from the item's persisted status, and — unlike the activity
// feed — NOT debounced, so every hop (open → in-progress → done) is its
// own row. This is the canonical timestamp source for the Reports
// completed-throughput and cycle-time series (PLAN-1628 / TASK-1637).
if input.Fields != nil {
newStatus := extractFieldValue(*input.Fields, doneKey)
// Record any change in the done-field value, INCLUDING a clear
// (X → ""). input.Fields is the full merged blob (CLI/handler merge
// before write), so newStatus == "" genuinely means the field was
// cleared, not omitted — recording it keeps the as-of-T reconstruction
// accurate (an item cleared back to no-status reads as open).
if newStatus != statusBefore {
if _, err = tx.Exec(s.q(`
INSERT INTO status_transitions (id, item_id, workspace_id, collection_id, field_key, from_status, to_status, created_at, seq)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, `+nextTransitionSeqSubquery+`)
`), newID(), id, existing.WorkspaceID, existing.CollectionID, doneKey, statusBefore, newStatus, ts); err != nil {
return nil, fmt.Errorf("record status transition: %w", err)
}
mutSignal.StatusChanged = true
mutSignal.StatusFieldKey = doneKey
mutSignal.FromStatus = statusBefore
mutSignal.ToStatus = newStatus
}
}
// Title rename — cascade to title-form backlinks. Fires whether
// content changed or not. ORDER MATTERS: cascade runs BEFORE
// replaceWikiLinks(self) so the pre-existing wl rows pointing
// at self via target_item_id=renamedItemID are still present
// when cascade does its SELECT. Codex round 6 finding 2 caught
// the original order (re-index self → cascade) silently
// breaking the self-ref cascade on title+content combined
// updates: re-indexing self first would delete the self-row
// that cascade needs to find. Function early-returns when
// oldTitle == newTitle so title-shaped-but-unchanged updates
// pay nothing. PLAN-1593 / TASK-1595.
if input.Title != nil && *input.Title != existing.Title {
// excludeSelf=true when the caller also supplied new content
// — they're authoritatively rewriting the renamed item's own
// body and the cascade should respect that. Self-refs in
// title-only renames still get cascade-rewritten so stale
// `[[Old Title]]` literals in unmodified content don't go
// broken. Mirrors documents.go::updateLinksInTx's pattern.
excludeSelf := input.Content != nil
if err := s.cascadeTitleRename(tx, id, existing.WorkspaceID, existing.Title, *input.Title, excludeSelf); err != nil {
return nil, fmt.Errorf("cascade title rename: %w", err)
}
}
// Re-index [[...]] wiki-links if the content was part of this
// update (regardless of whether the new content equals the old —
// the caller already paid the UPDATE cost so the delete-then-insert
// is cheap and keeps the index consistent if a previous reparse
// left stale rows). When `input.Content == nil` the content
// wasn't touched, so the existing rows remain valid and we skip
// work. PLAN-1593 / TASK-1594.
//
// Read items.content fresh from the DB instead of using
// *input.Content directly: cascadeTitleRename above may have
// rewritten the renamed item's own content if it contained
// self-references (`[[oldTitle]]` → `[[newTitle]]`), and we
// want the index to reflect that post-cascade state. Without
// the fresh read, this re-index would overwrite the cascade-
// rewritten rows back to whatever the user submitted, undoing
// the cascade's effect.
if input.Content != nil {
currentContent := *input.Content
if input.Title != nil && *input.Title != existing.Title {
if err := tx.QueryRow(s.q(`SELECT content FROM items WHERE id = ?`), id).Scan(&currentContent); err != nil {
return nil, fmt.Errorf("re-read self content after cascade: %w", err)
}
}
if err := s.replaceWikiLinks(tx, id, existing.WorkspaceID, currentContent); err != nil {
return nil, fmt.Errorf("index wiki links: %w", err)
}
}
// BUG-2013: apply the parent-link mutation INSIDE this tx, after the
// field write. A failure here (cycle detected, DB error) rolls the
// whole transaction back, so the caller can never observe the field
// update committed while the parent link write failed. The new
// parent's advisory lock was already folded into the acquisition
// above, so setParentLinkTx's re-lock is a no-op.
// PROVIDED IS NOT CHANGED (codex round 6). Clearing an already-unparented
// item deletes zero rows; forcing item.updated for it would describe a
// mutation that did not happen. The set branch always writes — it is a
// DELETE-then-INSERT that bumps the row either way.
var hierarchyChanged bool
if parentLink != nil && parentLink.Provided {
if parentLink.ParentID != "" {
if _, err := s.setParentLinkTx(tx, parentLink.WorkspaceID, id, parentLink.ParentID, parentLink.CreatedBy); err != nil {
return nil, err
}
hierarchyChanged = true
} else {
removed, err := s.clearParentLinkTx(tx, id, existing.WorkspaceID)
if err != nil {
return nil, err
}
hierarchyChanged = removed
}
}
// Read the updated row WITHIN the tx, BEFORE commit (BUG-2264). A
// post-commit re-read (s.GetItem) can fail AFTER a successful commit —
// making a committed update look failed to the caller (version-restore
// treats that as "roll back the room" and diverges) — and can observe a
// concurrent writer's later seq. getItemTx sees exactly this tx's own write
// (identical SQL to GetItem); a read failure here rolls the tx back cleanly,
// so a nil error is an unambiguous "this update committed, with this seq".
updated, err := s.getItemTx(tx, id)
if err != nil {
return nil, err
}
if updated == nil {
return nil, nil
}
// Assignment delta, read from the same in-tx before/after snapshots
// used for the status delta above — `existing` is now UNCONDITIONALLY
// this transaction's locked, in-tx pre-write view (TASK-2533 codex
// round 2 finding 4), `updated` is this transaction's own committed
// write. Comparing the two committed values (rather than re-deriving
// "after" from input.AssignedUserID / ClearAssignedUser) sidesteps
// having to duplicate that tri-state set/clear/untouched logic here.
beforeAssignee, afterAssignee := "", ""
if existing.AssignedUserID != nil {
beforeAssignee = *existing.AssignedUserID
}
if updated.AssignedUserID != nil {
afterAssignee = *updated.AssignedUserID
}
if beforeAssignee != afterAssignee {
mutSignal.AssignmentChanged = true
mutSignal.FromAssignedUserID = beforeAssignee
mutSignal.ToAssignedUserID = afterAssignee
}
if mutSignal.StatusChanged || mutSignal.AssignmentChanged {
sig := mutSignal
updated.LastMutation = &sig
}
// The choke point (SPEC-3 / TASK-2658). Emitted here — after the in-tx
// read-back and after the status/assignment deltas are computed, before
// COMMIT — because this is the first point where BOTH facts the event
// needs are known: what the row now holds, and what changed to get there.
// The prior status comes from the same in-tx before/after comparison that
// wrote the status_transitions row, so the event and the transition log
// cannot disagree about what happened.
if err := s.emitItemUpdateEventsTx(tx, existing, updated, mutSignal.StatusChanged, mutSignal.FromStatus, doneKey, opt.batchID, hierarchyChanged); err != nil {
return nil, err
}
if err := tx.Commit(); err != nil {
return nil, err
}
return updated, nil
}
// DeleteItem soft-deletes the item by stamping deleted_at and bumping
// the workspace-scoped seq so delta-sync clients see the tombstone.
// The seq bump uses the same MAX(seq)+1 subquery the other mutations
// rely on; the advisory lock keeps concurrent Postgres writes from
// racing on it.
func (s *Store) DeleteItem(id string, opts ...MutationOption) error {
opt := newMutationOptions(opts)
// Look up the workspace before the write so we can key the
// advisory lock and the seq subquery. The lookup tolerates
// already-deleted items (we still need to short-circuit cleanly
// in that case) by reading the include-deleted variant.
existing, err := s.GetItemIncludeDeleted(id)
if err != nil {
return err
}
if existing == nil {
return sql.ErrNoRows
}
tx, err := s.db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
if err := s.acquireWorkspaceSeqLock(tx, existing.WorkspaceID); err != nil {
return err
}
// SPEC-3 §Bindings requires item.deleted to carry the FINAL PRE-ARCHIVE
// state — that snapshot is the only thing keeping a deleted item
// addressable to binding predicates, since they never consult the live
// store. Read it here: in-tx, under the seq lock already held, and before
// the UPDATE while the row is still live (getItemTx filters archived rows,
// so after the write it would return nil).
//
// In-tx rather than reusing the pre-tx `existing` read above: that read
// happened before the lock, so a concurrent update landing in between
// would make the event describe a state that was never the final one.
preArchive, err := s.getItemTx(tx, id)
if err != nil {
return fmt.Errorf("read pre-archive snapshot: %w", err)
}
ts := now()
result, err := tx.Exec(s.q(`
UPDATE items SET deleted_at = ?, updated_at = ?, seq = `+nextWorkspaceSeqSubquery+`
WHERE id = ? AND deleted_at IS NULL
`), ts, ts, existing.WorkspaceID, id)
if err != nil {
return fmt.Errorf("delete item: %w", err)
}
rows, _ := result.RowsAffected()
if rows == 0 {
return sql.ErrNoRows
}
// A re-delete of an already-archived item must announce nothing: the
// mutation did not happen, and an event for it would be a lie the outbox
// exists not to tell.
//
// In the CURRENT code order, the zero-row return above is what produces
// that: the UPDATE carries `deleted_at IS NULL`, so a second delete
// matches nothing and this line is never reached.
//
// The `preArchive != nil` check is therefore NOT a second re-delete guard:
// it is unreachable for that case today. What it IS: the guard that keeps
// this correct if the order or the predicate ever changes — moving this
// emit above the zero-row return leaves the re-delete test green, because
// getItemTx filters archived rows and `preArchive` is already nil by then.
//
// It also covers the ordinary case where the pre-archive read found no
// live row at all, which must not emit an event for an item that was not
// there to archive.
if preArchive != nil {
if err := s.emitItemEventTx(tx, kernelevents.ItemDeleted, preArchive, nil, opt.batchID); err != nil {
return err
}
}
return tx.Commit()
}
// RestoreItem un-archives a soft-deleted item and bumps the
// workspace-scoped seq so delta-sync clients re-materialize the row.
// Same lock + subquery shape as DeleteItem.
func (s *Store) RestoreItem(id string, opts ...MutationOption) (*models.Item, error) {
opt := newMutationOptions(opts)
// BUG-2073: retry if the item's parent set moves during lock acquisition.
return retryOnParentSetChanged(func() (*models.Item, error) {
return s.restoreItemOnce(id, opt)
})
}
func (s *Store) restoreItemOnce(id string, opt mutationOptions) (*models.Item, error) {
existing, err := s.GetItemIncludeDeleted(id)
if err != nil {
return nil, err
}
if existing == nil {
return nil, sql.ErrNoRows
}
tx, err := s.db.Begin()
if err != nil {
return nil, err
}
defer tx.Rollback()
if err := s.acquireWorkspaceSeqLock(tx, existing.WorkspaceID); err != nil {
return nil, err
}
// Codex round-3 P1 / round-4 P1: restoring an item resurrects it
// as a (potentially non-terminal) child of EVERY parent it's
// linked to (one item can have both a `parent` and an
// `implements` link). Lock ALL of those parents' children-keys
// via the canonical sorted-multi-lock helper so concurrent
// UpdateItemWithPreCheck callers on any of them see this
// resurrection in their post-lock snapshots.
//
// Pre-fix this called AcquireParentChildrenLock for a single
// LIMIT 1 row — a multi-parent child would have left another
// parent's precheck racing the resurrection.
//
// BUG-2073: route through the shared acquireParentChildrenLocksForUpdate
// helper (rather than an inline read-then-lock) so RestoreItem also holds
// the restored item's OWN (id) lock and re-reads the parent set under it —
// closing the read-then-lock window this path shared with UpdateItem.
if err := s.acquireParentChildrenLocksForUpdate(tx, id); err != nil {
return nil, err
}
// BUG-2629: re-assert the item's attachment references at the moment it
// becomes live again. While archived, the live AttachmentReferenced scan
// can't see this item's refs, so the orphan GC may have let their
// last_referenced_at go stale; a claim racing this restore keys on that
// stamp (not the live scan). Stamp BEFORE the deleted_at clear, in this
// tx, per stampAttachmentRefsTx's ORDERING note: the stamp's row-lock
// makes a concurrent claim block until commit and then re-evaluate
// against the fresh stamp — refusing. (Prevention only: a blob already
// reclaimed is gone, and the stamp matches zero rows — see BUG-2629.)
if err := stampAttachmentRefsTx(tx, s, existing.WorkspaceID, existing.Content, existing.Fields); err != nil {
return nil, err
}
ts := now()
result, err := tx.Exec(s.q(`
UPDATE items SET deleted_at = NULL, updated_at = ?, seq = `+nextWorkspaceSeqSubquery+`
WHERE id = ? AND deleted_at IS NOT NULL
`), ts, existing.WorkspaceID, id)
if err != nil {
return nil, fmt.Errorf("restore item: %w", err)
}
rows, _ := result.RowsAffected()
if rows == 0 {
return nil, sql.ErrNoRows
}
// item.restored carries the POST-restore snapshot (SPEC-3 v1.1). Read
// after the UPDATE, in-tx: the row is live again at this point, so
// getItemTx sees it, and the snapshot reflects the state a consumer will
// find if it goes looking. Restore was silent before v1.1 — an item could
// reappear with no observable event, which broke every consumer's model of
// what exists.
restored, err := s.getItemTx(tx, id)
if err != nil {
return nil, fmt.Errorf("read post-restore snapshot: %w", err)
}
if restored != nil {
if err := s.emitItemEventTx(tx, kernelevents.ItemRestored, restored, nil, opt.batchID); err != nil {
return nil, err
}
}
if err := tx.Commit(); err != nil {
return nil, err
}
return s.GetItem(id)
}
func (s *Store) SearchItems(workspaceID, query string) ([]ItemSearchResult, error) {
// Whitespace-only queries collapse to empty after FTS5 sanitization and
// would error on `MATCH ''`. Treat them as no-result rather than failing.
// See BUG-818.
if strings.TrimSpace(query) == "" {
return []ItemSearchResult{}, nil
}
var sqlQuery string
var args []interface{}
if s.dialect.Driver() == DriverPostgres {
// PostgreSQL: search_vector lives on the items table (aliased as "i").
ftsSnippet := s.dialect.FTSSnippet("i", 1, "i.content")
ftsMatch := s.dialect.FTSMatch("i", "search_vector")
ftsRank := s.dialect.FTSRank("i", "search_vector")
sqlQuery = fmt.Sprintf(`
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.content, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''),
%s as snippet,
%s as rank_score
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE %s
AND i.deleted_at IS NULL
`, ftsSnippet, ftsRank, ftsMatch)
// PG FTSSnippet, FTSRank, and FTSMatch each consume TWO "?" args
// (raw query + hyphen-sanitized query) for the OR-combined
// plainto_tsquery — see dialect.go and BUG-842.
sanitized := sanitizePGFTSQuery(query)
args = []interface{}{query, sanitized, query, sanitized, query, sanitized}
} else {
// SQLite: uses FTS5 virtual table "items_fts".
ftsSnippet := s.dialect.FTSSnippet("items_fts", 1, "i.content")
ftsMatch := s.dialect.FTSMatch("items_fts", "search_vector")
ftsRank := s.dialect.FTSRank("items_fts", "search_vector")
sqlQuery = fmt.Sprintf(`
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.content, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''),
%s as snippet,
%s as rank_score
FROM items_fts fts
JOIN items i ON i.rowid = fts.rowid
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE %s
AND i.deleted_at IS NULL
`, ftsSnippet, ftsRank, ftsMatch)
// Sanitize the user query so FTS5 special characters (hyphens, boolean
// operators) are treated as literals — see BUG-818.
args = []interface{}{sanitizeFTSQuery(query)}
}
if workspaceID != "" {
sqlQuery += " AND i.workspace_id = ?"
args = append(args, workspaceID)
}
if s.dialect.Driver() == DriverPostgres {
sqlQuery += " ORDER BY rank_score DESC LIMIT 50"
} else {
sqlQuery += " ORDER BY rank_score LIMIT 50"
}
rows, err := s.db.Query(s.q(sqlQuery), args...)
if err != nil {
return nil, fmt.Errorf("search items: %w", err)
}
defer rows.Close()
var results []ItemSearchResult
for rows.Next() {
var r ItemSearchResult
var createdAt, updatedAt string
var pinned bool
if err := rows.Scan(
&r.Item.ID, &r.Item.WorkspaceID, &r.Item.CollectionID, &r.Item.Title, &r.Item.Slug,
&r.Item.Content, &r.Item.Fields, &r.Item.Tags,
&pinned, &r.Item.SortOrder, &r.Item.ParentID, &r.Item.AssignedUserID, &r.Item.AgentRoleID, &r.Item.RoleSortOrder,
&r.Item.CreatedBy, &r.Item.LastModifiedBy,
&r.Item.Source, &r.Item.ItemNumber, &r.Item.Seq, &createdAt, &updatedAt,
&r.Item.CollectionSlug, &r.Item.CollectionName, &r.Item.CollectionIcon, &r.Item.CollectionPrefix,
&r.Item.AssignedUserName, &r.Item.AssignedUserEmail,
&r.Item.AgentRoleName, &r.Item.AgentRoleSlug, &r.Item.AgentRoleIcon,
&r.Snippet, &r.Rank,
); err != nil {
return nil, err
}
r.Item.Pinned = pinned
r.Item.CreatedAt = parseTime(createdAt)
r.Item.UpdatedAt = parseTime(updatedAt)
r.Item.ComputeRef()
r.Item.Content = "" // Don't include full content in search results
results = append(results, r)
}
return results, rows.Err()
}
// --- Item Links ---
func (s *Store) CreateItemLink(workspaceID string, input models.ItemLinkCreate, sourceID string) (*models.ItemLink, error) {
id := newID()
ts := now()
linkType, err := models.NormalizeItemLinkType(input.LinkType)
if err != nil {
return nil, err
}
if sourceID == input.TargetID {
return nil, fmt.Errorf("cannot link an item to itself")
}
createdBy := input.CreatedBy
if createdBy == "" {
createdBy = "user"
}
// BUG-2074: a `parent` link is the same graph edge SetParentLink writes and
// the only link_type checkParentCycleQ's ancestor walk follows. Route it
// through SetParentLink so it gets the FULL guarded parent-edge protocol:
// - single-parent DELETE-then-INSERT (so a child can't accumulate two
// `parent` rows — the append-only insert below would, and the cycle
// walk only follows ONE arbitrary parent per source, so a second row
// would let an N-hop cycle hide on the un-walked branch);
// - the workspace-scoped cycle lock + checkParentCycleQ under lock;
// - the errParentSetChanged retry wrapper.
// The plain append-only INSERT below is only safe for NON-parent link types
// (blocks / supersedes / implements / related / …), none of which the cycle
// walk follows.
if linkType == models.ItemLinkTypeParent {
return s.SetParentLink(workspaceID, sourceID, input.TargetID, createdBy)
}
tx, err := s.db.Begin()
if err != nil {
return nil, fmt.Errorf("begin tx: %w", err)
}
defer tx.Rollback()
// Structural relationships change the source item's local-first
// is_unparented metadata. Take the seq lock before the per-item child
// locks (the repository-wide lock order) so the link insert and source
// seq bump commit atomically without duplicate Postgres cursors.
if linkType == models.ItemLinkTypeImplements {
if err := s.acquireWorkspaceSeqLock(tx, workspaceID); err != nil {
return nil, err
}
}
// Codex round-3 P1: when this link puts `sourceID` into the
// children-set of `target` (i.e. linkType ∈ childLinkTypes), lock
// the target's parent-children key so a concurrent
// UpdateItemWithPreCheck on the target can't read 0 open children
// while we're about to attach a non-terminal one. Non-child link
// types (blocks, supersedes, …) don't affect the children-set so
// we skip the lock — keeps the common case lock-free.
//
// BUG-2073: ALSO lock the SOURCE item's key. Attaching sourceID as a
// child of target adds a parent to sourceID, so sourceID's own lock must
// be held for the "the child lock freezes an item's parent set" invariant
// that acquireParentChildrenLocksForUpdate / setParentLinkTx rely on to
// hold — otherwise a concurrent UpdateItem(sourceID) could miss this new
// parent on its post-lock re-read. Both keys go through the sorted helper,
// so the two-key grab stays deadlock-free.
//
// (The `parent` link_type never reaches here — it is routed through
// SetParentLink above for the full guarded parent-edge protocol; BUG-2074.
// The remaining child-link type that lands here is 'implements', which the
// cycle walk does not follow, so no cycle guard is needed.)
if isChildLinkType(linkType) {
if err := s.AcquireParentChildrenLocks(tx, sourceID, input.TargetID); err != nil {
return nil, err
}
}
if _, err := tx.Exec(s.q(`
INSERT INTO item_links (id, workspace_id, source_id, target_id, link_type, created_by, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)
`), id, workspaceID, sourceID, input.TargetID, linkType, createdBy, ts); err != nil {
return nil, fmt.Errorf("create item link: %w", err)
}
if linkType == models.ItemLinkTypeImplements {
if err := s.bumpStructuralLinkSourceTx(tx, workspaceID, sourceID); err != nil {
return nil, err
}
}
if err := tx.Commit(); err != nil {
return nil, fmt.Errorf("commit create item link: %w", err)
}
return s.getItemLink(id)
}
// getItemLink is the unfiltered post-insert readback used by CreateItemLink to
// hydrate the freshly-inserted row with collection/source/target metadata. It
// intentionally does NOT filter on items.deleted_at IS NULL: the only caller
// is the immediate readback after INSERT, and a delete race against either
// endpoint would otherwise cause the just-successful insert to return nil
// (Codex review on PR #259). User-facing surfaces all read links via
// GetItemLinks (plural) or GetParentForItem, both of which DO filter.
func (s *Store) getItemLink(id string) (*models.ItemLink, error) {
var link models.ItemLink
var createdAt string
var sourcePrefix, targetPrefix string
var sourceItemNumber, targetItemNumber sql.NullInt64
var sourceStatus, targetStatus sql.NullString
srcStatus := s.dialect.JSONExtractText("s.fields", "status")
tgtStatus := s.dialect.JSONExtractText("t.fields", "status")
err := s.db.QueryRow(s.q(fmt.Sprintf(`
SELECT l.id, l.workspace_id, l.source_id, l.target_id, l.link_type, l.created_by, l.created_at,
s.title, t.title, s.slug, t.slug, sc.slug, tc.slug, sc.prefix, tc.prefix,
s.item_number, t.item_number,
%s, %s
FROM item_links l
JOIN items s ON s.id = l.source_id
JOIN items t ON t.id = l.target_id
JOIN collections sc ON sc.id = s.collection_id
JOIN collections tc ON tc.id = t.collection_id
WHERE l.id = ?
`, srcStatus, tgtStatus)), id).Scan(
&link.ID, &link.WorkspaceID, &link.SourceID, &link.TargetID,
&link.LinkType, &link.CreatedBy, &createdAt,
&link.SourceTitle, &link.TargetTitle,
&link.SourceSlug, &link.TargetSlug,
&link.SourceCollectionSlug, &link.TargetCollectionSlug,
&sourcePrefix, &targetPrefix,
&sourceItemNumber, &targetItemNumber,
&sourceStatus, &targetStatus,
)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("get item link: %w", err)
}
link.CreatedAt = parseTime(createdAt)
if sourceItemNumber.Valid && sourcePrefix != "" {
link.SourceRef = fmt.Sprintf("%s-%d", sourcePrefix, sourceItemNumber.Int64)
}
if targetItemNumber.Valid && targetPrefix != "" {
link.TargetRef = fmt.Sprintf("%s-%d", targetPrefix, targetItemNumber.Int64)
}
if sourceStatus.Valid {
link.SourceStatus = sourceStatus.String
}
if targetStatus.Valid {
link.TargetStatus = targetStatus.String
}
return &link, nil
}
// GetItemLinks returns links where the given item is either source or target.
// Links pointing to or from soft-deleted items are filtered out so callers (e.g.
// `pad item related`, the lineage panel, the dashboard enrichment pass) don't
// surface dangling endpoints. The link rows themselves are preserved on disk —
// restoring a soft-deleted item resurrects its relationships automatically. See
// BUG-734.
func (s *Store) GetItemLinks(itemID string) ([]models.ItemLink, error) {
srcStatusExpr := s.dialect.JSONExtractText("s.fields", "status")
tgtStatusExpr := s.dialect.JSONExtractText("t.fields", "status")
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT l.id, l.workspace_id, l.source_id, l.target_id, l.link_type, l.created_by, l.created_at,
s.title, t.title, s.slug, t.slug, sc.slug, tc.slug, sc.prefix, tc.prefix,
s.item_number, t.item_number,
%s, %s
FROM item_links l
JOIN items s ON s.id = l.source_id AND s.deleted_at IS NULL
JOIN items t ON t.id = l.target_id AND t.deleted_at IS NULL
JOIN collections sc ON sc.id = s.collection_id
JOIN collections tc ON tc.id = t.collection_id
WHERE l.source_id = ? OR l.target_id = ?
ORDER BY l.created_at DESC
`, srcStatusExpr, tgtStatusExpr)), itemID, itemID)
if err != nil {
return nil, fmt.Errorf("get item links: %w", err)
}
defer rows.Close()
var links []models.ItemLink
for rows.Next() {
var link models.ItemLink
var createdAt string
var sourcePrefix, targetPrefix string
var sourceItemNumber, targetItemNumber sql.NullInt64
var sourceStatus, targetStatus sql.NullString
if err := rows.Scan(
&link.ID, &link.WorkspaceID, &link.SourceID, &link.TargetID,
&link.LinkType, &link.CreatedBy, &createdAt,
&link.SourceTitle, &link.TargetTitle,
&link.SourceSlug, &link.TargetSlug,
&link.SourceCollectionSlug, &link.TargetCollectionSlug,
&sourcePrefix, &targetPrefix,
&sourceItemNumber, &targetItemNumber,
&sourceStatus, &targetStatus,
); err != nil {
return nil, err
}
link.CreatedAt = parseTime(createdAt)
if sourceItemNumber.Valid && sourcePrefix != "" {
link.SourceRef = fmt.Sprintf("%s-%d", sourcePrefix, sourceItemNumber.Int64)
}
if targetItemNumber.Valid && targetPrefix != "" {
link.TargetRef = fmt.Sprintf("%s-%d", targetPrefix, targetItemNumber.Int64)
}
if sourceStatus.Valid {
link.SourceStatus = sourceStatus.String
}
if targetStatus.Valid {
link.TargetStatus = targetStatus.String
}
links = append(links, link)
}
return links, rows.Err()
}
// GetItemLinkByID returns a single item link by its ID, or nil if not found.
func (s *Store) GetItemLinkByID(id string) (*models.ItemLink, error) {
var link models.ItemLink
var createdAt string
err := s.db.QueryRow(s.q(`
SELECT id, workspace_id, source_id, target_id, link_type, created_by, created_at
FROM item_links WHERE id = ?
`), id).Scan(&link.ID, &link.WorkspaceID, &link.SourceID, &link.TargetID,
&link.LinkType, &link.CreatedBy, &createdAt)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("get item link by id: %w", err)
}
link.CreatedAt = parseTime(createdAt)
return &link, nil
}
func (s *Store) DeleteItemLink(id string) error {
tx, err := s.db.Begin()
if err != nil {
return fmt.Errorf("begin tx: %w", err)
}
defer tx.Rollback()
// Codex round-3 P1: peek the link's type + target before deleting
// so we can lock the target's parent-children key when this link
// participates in the children-set. Without this, a concurrent
// UpdateItemWithPreCheck on the target could read the child as
// still attached, decide the parent has no open children, and
// commit a terminal status while we orphan a non-terminal child.
//
// We DON'T lock for non-child link types (blocks, supersedes, …)
// — they don't affect the children-set, so contention there is
// unnecessary.
//
// BUG-2073: lock the SOURCE key too (not just the target) for child link
// types — detaching sourceID from target removes a parent from sourceID,
// so sourceID's own lock must be held for the "child lock freezes the
// parent set" invariant the update paths rely on. Both keys go through the
// sorted helper, so the grab stays deadlock-free.
var linkType, sourceID, targetID, workspaceID string
err = tx.QueryRow(s.q("SELECT link_type, source_id, target_id, workspace_id FROM item_links WHERE id = ?"), id).Scan(&linkType, &sourceID, &targetID, &workspaceID)
if err == sql.ErrNoRows {
return sql.ErrNoRows
}
if err != nil {
return fmt.Errorf("peek item link for delete: %w", err)
}
if linkType == models.ItemLinkTypeParent || linkType == models.ItemLinkTypeImplements {
if err := s.acquireWorkspaceSeqLock(tx, workspaceID); err != nil {
return err
}
}
if isChildLinkType(linkType) {
if err := s.AcquireParentChildrenLocks(tx, sourceID, targetID); err != nil {
return err
}
}
result, err := tx.Exec(s.q("DELETE FROM item_links WHERE id = ?"), id)
if err != nil {
return fmt.Errorf("delete item link: %w", err)
}
rows, _ := result.RowsAffected()
if rows == 0 {
return sql.ErrNoRows
}
if linkType == models.ItemLinkTypeParent || linkType == models.ItemLinkTypeImplements {
if err := s.bumpStructuralLinkSourceTx(tx, workspaceID, sourceID); err != nil {
return err
}
}
// item.updated for a PARENT detach, on this transaction (codex round 6).
//
// Same criterion as SetParentLink and the update path (SPEC-3 v1.6): the
// parent edge writes the source item's own row. Without this, DELETE
// /links/{id} on a parent link was the one detach route that stayed
// silent, so a consumer's model kept a parent the user had removed while
// every attach route was observable.
//
// IMPLEMENTS IS DELIBERATELY NOT INCLUDED HERE, and the asymmetry is
// flagged rather than resolved: it bumps the same row two lines above (so
// the mechanical criterion would include it) but it is a
// relationship-graph link (so v1.5's silence would exclude it). The
// contract does not currently decide that case, and inventing an answer
// inside a delivery refactor is how a public wire acquires an event nobody
// ruled on. Raised with the lead; tracked separately. The same note is on
// handlers_item_links.go, which is where a reader is likely to hit it
// first.
if linkType == models.ItemLinkTypeParent {
child, err := s.getItemTx(tx, sourceID)
if err != nil {
return err
}
if child != nil {
if err := s.emitItemEventTx(tx, kernelevents.ItemUpdated, child, nil, ""); err != nil {
return err
}
}
}
return tx.Commit()
}
// --- Phase Links ---
// SetParentLink sets the parent for an item. Since an item can belong to at most
// one parent, this deletes any existing parent link for the item first.
// Includes cycle detection to prevent A→B→A or deeper ancestor loops.
//
// Codex round-3 P1: acquires `pad:parent-children:<id>` for BOTH the
// old parent (if any) AND the new parent in sorted order. That makes
// a concurrent UpdateItemWithPreCheck on either parent block on the
// same key, closing the link-mutation TOCTOU gap — without this, the
// guard could read 0 open children while this method was about to
// attach a non-terminal child.
func (s *Store) SetParentLink(workspaceID, itemID, parentID, createdBy string) (*models.ItemLink, error) {
// BUG-2073: retry if the item's parent moved during lock acquisition
// (setParentLinkTx re-reads under the child lock and signals a rollback).
return retryOnParentSetChanged(func() (*models.ItemLink, error) {
return s.setParentLinkOnce(workspaceID, itemID, parentID, createdBy)
})
}
func (s *Store) setParentLinkOnce(workspaceID, itemID, parentID, createdBy string) (*models.ItemLink, error) {
tx, err := s.db.Begin()
if err != nil {
return nil, fmt.Errorf("begin tx: %w", err)
}
defer tx.Rollback()
// BUG-2074: serialize all parent-edge additions in this workspace so the
// cycle walk below runs against a consistent snapshot and can't miss an
// N-hop cycle closed by a concurrent add on items neither endpoint locks.
// Acquired OUTERMOST (before setParentLinkTx's parent-children batch) to
// keep the global lock order cycle -> seq -> parent-children.
if err := s.acquireWorkspaceParentLinkLock(tx, workspaceID); err != nil {
return nil, err
}
if err := s.acquireWorkspaceSeqLock(tx, workspaceID); err != nil {
return nil, err
}
id, err := s.setParentLinkTx(tx, workspaceID, itemID, parentID, createdBy)
if err != nil {
return nil, err
}
// item.updated for the CHILD, on this transaction (TASK-2714, codex rounds
// 4-5; SPEC-3 v1.6 clarification).
//
// A parent link writes the item's OWN row — it advances seq — and that is
// the criterion the contract uses to divide two rulings that look
// adjacent: a mutation touching the item's row emits item.updated, while a
// relationship-graph link (blocks / blocked-by), which writes only the
// links table, stays silent under v1.5.
//
// WHAT THE PAYLOAD DOES AND DOES NOT CARRY, stated because round 4's
// version of this comment claimed more than the code delivers (round 5
// caught it): the snapshot is the ITEM ROW, and the parent EDGE is not on
// it. items.parent_id is legacy and untouched here, and IsUnparented is
// populated only by the local-first index queries. So this event reports
// that the row changed — a fresh seq and updated_at — not the linkage
// itself. That is not a shortfall against the behaviour it restores: the
// hand-called webhook this replaced dispatched the handler's post-link
// re-read, which is the SAME scan and carried the same fields. A consumer
// wanting the edge reads the item, exactly as before.
//
// The regression it closes: createItemChecked calls this AFTER CreateItem
// has committed item.created with a pre-link snapshot. Without an event
// here the frozen created row — with the stale seq — was the only thing on
// the wire, and nothing corrected it.
//
// The snapshot MUST come from getItemTx: a pool read takes a different
// connection, cannot see this uncommitted write, and would emit the
// pre-link seq under a post-link event.
//
// EMITTED HERE RATHER THAN IN setParentLinkTx, which is the shared core.
// UpdateItemWithParentLink reuses that core inside the item-update
// transaction and already emits its own item events from the field diff;
// putting the emit in the shared function would double-emit on that path.
child, err := s.getItemTx(tx, itemID)
if err != nil {
return nil, err
}
if child != nil {
if err := s.emitItemEventTx(tx, kernelevents.ItemUpdated, child, nil, ""); err != nil {
return nil, err
}
}
if err := tx.Commit(); err != nil {
return nil, fmt.Errorf("commit parent link: %w", err)
}
// Return the full link with enriched fields. Use the unfiltered readback
// helper so that a delete race against either endpoint between commit and
// readback doesn't cause the successful insert to surface as nil.
return s.getItemLink(id)
}
// setParentLinkTx performs the lock acquisition, cycle check, and the
// DELETE-then-INSERT of the parent link entirely within the caller's
// transaction, returning the new link's id. It is the shared core of the
// public SetParentLink (which opens its own tx) AND UpdateItemWithParentLink
// (which reuses the item-update tx so the field write and the link write
// commit or roll back together — closing the partial-commit window from
// BUG-2013).
//
// Lock acquisition routes through AcquireParentChildrenLocks, so re-acquiring
// keys the enclosing tx already holds (as UpdateItemWithParentLink does after
// pre-locking the new parent) is an idempotent no-op rather than a deadlock.
func (s *Store) setParentLinkTx(tx *sql.Tx, workspaceID, itemID, parentID, createdBy string) (string, error) {
// Find the existing parent (if any) so we can fold it into the initial
// lock batch. The DELETE below targets link_type='parent' specifically,
// which matches what the guard's children query treats as the parent
// edge (childLinkTypes includes 'parent'); other child-link types
// like 'implements' aren't displaced by this method so we don't
// need their old parent here. This read is best-effort (pre-lock); it is
// re-verified under the child lock below.
oldParentID, err := s.readParentLinkTarget(tx, itemID)
if err != nil {
return "", err
}
// BUG-2073 race 1 (cycle): acquire the CHILD's own (itemID) lock in
// addition to the old + new parent keys, all in ONE sorted batch. Before
// this fix SetParentLink locked only the old+new parents, so concurrent
// SetParentLink(A,B) and SetParentLink(B,A) locked disjoint keys ({B} vs
// {A}), both cycle walks passed on stale snapshots, and both inserts
// committed — forming an A↔B cycle. With itemID folded in, the two calls
// both contend on {A,B}, serialize, and the loser's cycle walk (run under
// the lock, below) observes the committed edge and rejects. Sorted
// acquisition keeps the multi-key grab deadlock-free.
if err := s.AcquireParentChildrenLocks(tx, itemID, oldParentID, parentID); err != nil {
return "", err
}
// BUG-2073 race 2 (stale old parent): oldParentID was read BEFORE the
// locks were held. A concurrent reparent of THIS child can commit in the
// window before we acquired the child's lock, moving the real old parent.
// Now that we hold the child (itemID) lock the parent edge is frozen, so
// re-read it and verify. If it moved to a parent we did NOT lock, we can't
// safely acquire that key now: it may sort before a key we already hold,
// which would violate AcquireParentChildrenLocks' canonical sorted order
// and could deadlock. Instead we signal errParentSetChanged so the
// tx-owning caller rolls back (releasing every lock) and retries from a
// fresh read — on the retry the moved parent is folded into the INITIAL
// sorted batch, keeping acquisition deadlock-free. This mismatch can only
// happen on the public SetParentLink path; UpdateItemWithParentLink holds
// the child lock from its own acquisition, so its re-read always matches.
reOldParentID, err := s.readParentLinkTarget(tx, itemID)
if err != nil {
return "", err
}
if reOldParentID != oldParentID {
return "", errParentSetChanged
}
// Cycle detection: walk the ancestor chain from parentID to ensure itemID
// is not an ancestor. Run this AFTER the parent-children locks are held (Codex
// review, PR #868): checking before the lock lets two concurrent reparents
// each pass on a stale ancestry snapshot, block on the lock, then both insert
// — closing the loop (A→B→C→A). Under the lock the walk reads via the tx, so
// it sees the edge the just-unblocked peer committed and catches the cycle.
// BUG-2074: cycles closed via an edge on an item NEITHER endpoint locks
// (e.g. concurrent SetParentLink(B,C) + SetParentLink(D,A) completing
// A→B→C→D→A, whose per-endpoint lock sets {B,C} and {D,A} are disjoint)
// used to slip past this per-endpoint walk. The tx-owning callers now hold
// the workspace-scoped parent-link cycle lock (acquireWorkspaceParentLinkLock,
// taken outermost in setParentLinkOnce / updateItemWithParentLinkOnce), which
// serializes ALL parent-edge additions in the workspace — so this walk runs
// against a snapshot no concurrent add can mutate, catching arbitrary N-hop
// cycles.
if err := s.checkParentCycleQ(tx, itemID, parentID); err != nil {
return "", err
}
// Delete existing parent link for this item (if any). Targeting by
// source_id detaches the child from whatever parent it ACTUALLY has —
// whose lock we now hold via the re-read above.
if _, err := tx.Exec(s.q(`DELETE FROM item_links WHERE source_id = ? AND link_type = 'parent'`), itemID); err != nil {
return "", fmt.Errorf("delete existing parent link: %w", err)
}
// Insert new parent link
id := newID()
now := time.Now().UTC().Format(time.RFC3339)
if _, err := tx.Exec(s.q(`
INSERT INTO item_links (id, workspace_id, source_id, target_id, link_type, created_by, created_at)
VALUES (?, ?, ?, ?, 'parent', ?, ?)
`), id, workspaceID, itemID, parentID, createdBy, now); err != nil {
return "", fmt.Errorf("insert parent link: %w", err)
}
if err := s.bumpStructuralLinkSourceTx(tx, workspaceID, itemID); err != nil {
return "", err
}
return id, nil
}
// rowQueryer is the minimal surface checkParentCycleQ needs from either
// *sql.DB or *sql.Tx, so the cycle walk can run either unlocked (public
// SetParentLink) or inside an in-flight transaction (UpdateItem's atomic
// parent-link path, which must read item_links under the same tx/locks
// it writes with).
type rowQueryer interface {
QueryRow(query string, args ...any) *sql.Row
}
// readParentLinkTarget returns the target_id of an item's `parent` link, or
// "" when it has none. Parameterized over rowQueryer so it can read either
// unlocked or inside an in-flight transaction — the parent-link paths call it
// twice (once best-effort before locking, once under the child lock to catch a
// reparent that landed during the lock-acquisition window; BUG-2073).
func (s *Store) readParentLinkTarget(q rowQueryer, itemID string) (string, error) {
var target sql.NullString
if err := q.QueryRow(s.q(`
SELECT target_id FROM item_links
WHERE source_id = ? AND link_type = 'parent'
LIMIT 1
`), itemID).Scan(&target); err != nil && err != sql.ErrNoRows {
return "", fmt.Errorf("lookup existing parent: %w", err)
}
return target.String, nil
}
// checkParentCycleQ walks the ancestor chain from parentID and returns an
// error if itemID is found (which would create a cycle). Parameterized over
// the queryer so the walk can execute inside a transaction: reading via the
// same tx that holds the parent-children locks keeps the cycle decision
// consistent with the DELETE/INSERT that follows it.
func (s *Store) checkParentCycleQ(q rowQueryer, itemID, parentID string) error {
visited := map[string]bool{itemID: true}
current := parentID
for {
if visited[current] {
return fmt.Errorf("cannot set parent: would create a cycle")
}
visited[current] = true
// Look up the parent of current
var targetID sql.NullString
err := q.QueryRow(s.q(`
SELECT target_id FROM item_links
WHERE source_id = ? AND link_type = 'parent'
`), current).Scan(&targetID)
if err != nil || !targetID.Valid {
break // no parent — no cycle
}
current = targetID.String
}
return nil
}
// ClearParentLink removes the parent link for an item.
//
// Codex round-3 P1: runs in a tx and acquires `pad:parent-children:<old>`
// before the DELETE so a concurrent UpdateItemWithPreCheck on the old
// parent blocks until this commit. Detaching a child is materially
// similar to attaching one — the parent's children-set changes either
// way and the guard must see a consistent view.
func (s *Store) ClearParentLink(itemID string) error {
// BUG-2073: retry if the item's parent moved during lock acquisition.
_, err := retryOnParentSetChanged(func() (struct{}, error) {
return struct{}{}, s.clearParentLinkOnce(itemID)
})
return err
}
func (s *Store) clearParentLinkOnce(itemID string) error {
tx, err := s.db.Begin()
if err != nil {
return fmt.Errorf("begin tx: %w", err)
}
defer tx.Rollback()
workspaceID, err := s.itemWorkspaceIDTx(tx, itemID)
if err != nil {
return err
}
if err := s.acquireWorkspaceSeqLock(tx, workspaceID); err != nil {
return err
}
removed, err := s.clearParentLinkTx(tx, itemID, workspaceID)
if err != nil {
return err
}
// Same criterion as SetParentLink (SPEC-3 v1.6): a parent DETACH writes
// the item's own row, so it emits. Removing this half would leave the
// attach observable and the detach silent — a consumer's model would keep
// a parent the user removed (codex round 6).
if removed {
child, err := s.getItemTx(tx, itemID)
if err != nil {
return err
}
if child != nil {
if err := s.emitItemEventTx(tx, kernelevents.ItemUpdated, child, nil, ""); err != nil {
return err
}
}
}
return tx.Commit()
}
// clearParentLinkTx removes the item's parent link within the caller's
// transaction. Shared by the public ClearParentLink (own tx) and
// UpdateItemWithParentLink (item-update tx), so a cleared parent commits
// atomically with the field write it accompanied (BUG-2013).
// Returns whether a link was actually REMOVED. Callers need the distinction:
// clearing an already-unparented item deletes zero rows and changes nothing, so
// forcing an item.updated for it would put an event on the wire describing a
// mutation that did not happen (codex round 6).
func (s *Store) clearParentLinkTx(tx *sql.Tx, itemID, workspaceID string) (bool, error) {
// Best-effort pre-lock read of the current parent, re-verified under lock.
oldParentID, err := s.readParentLinkTarget(tx, itemID)
if err != nil {
return false, fmt.Errorf("lookup parent for clear: %w", err)
}
// BUG-2073: fold the CHILD's own (itemID) lock into the batch alongside
// the old parent, in ONE sorted acquisition. The child lock serializes
// concurrent parent mutations of this item (SetParentLink/ClearParentLink/
// UpdateItemWithParentLink all take it), so a detach can't race a reparent.
if err := s.AcquireParentChildrenLocks(tx, itemID, oldParentID); err != nil {
return false, err
}
// Re-read the parent under the child lock (BUG-2073 race 2): a concurrent
// reparent may have committed in the window before we held itemID's lock.
// Now the parent edge is frozen; if it moved to a parent we did NOT lock,
// signal errParentSetChanged so the tx-owning caller rolls back and retries
// from a fresh read (see setParentLinkTx for the rationale — acquiring the
// moved key here would risk an out-of-order grab).
reOldParentID, err := s.readParentLinkTarget(tx, itemID)
if err != nil {
return false, fmt.Errorf("lookup parent for clear: %w", err)
}
if reOldParentID != oldParentID {
return false, errParentSetChanged
}
result, err := tx.Exec(s.q(`DELETE FROM item_links WHERE source_id = ? AND link_type = 'parent'`), itemID)
if err != nil {
return false, fmt.Errorf("clear parent link: %w", err)
}
rows, err := result.RowsAffected()
if err != nil {
return false, fmt.Errorf("clear parent link rows affected: %w", err)
}
if rows > 0 {
if err := s.bumpStructuralLinkSourceTx(tx, workspaceID, itemID); err != nil {
return false, err
}
}
return rows > 0, nil
}
// bumpStructuralLinkSourceTx advances the source row whenever a parent or
// implements edge changes. Callers must already hold the workspace seq lock
// on Postgres. Keeping the bump in the link transaction makes index/delta
// metadata and the committed relationship indivisible.
func (s *Store) bumpStructuralLinkSourceTx(tx *sql.Tx, workspaceID, sourceID string) error {
result, err := tx.Exec(s.q(`
UPDATE items
SET updated_at = ?, seq = `+nextWorkspaceSeqSubquery+`
WHERE id = ? AND workspace_id = ?
`), now(), workspaceID, sourceID, workspaceID)
if err != nil {
return fmt.Errorf("bump structural link source seq: %w", err)
}
rows, err := result.RowsAffected()
if err != nil {
return fmt.Errorf("bump structural link source seq rows affected: %w", err)
}
if rows == 0 {
return sql.ErrNoRows
}
return nil
}
func (s *Store) itemWorkspaceIDTx(tx *sql.Tx, itemID string) (string, error) {
var workspaceID string
if err := tx.QueryRow(s.q(`SELECT workspace_id FROM items WHERE id = ?`), itemID).Scan(&workspaceID); err != nil {
return "", fmt.Errorf("lookup item workspace: %w", err)
}
return workspaceID, nil
}
// GetParentForItem returns the parent link for an item, or nil if it has no parent.
// A parent link pointing to a soft-deleted item is treated as no parent — the
// breadcrumb / lineage UI shouldn't show a deleted ancestor. See BUG-734.
func (s *Store) GetParentForItem(itemID string) (*models.ItemLink, error) {
sStatusExpr := s.dialect.JSONExtractText("s.fields", "status")
tStatusExpr := s.dialect.JSONExtractText("t.fields", "status")
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT l.id, l.workspace_id, l.source_id, l.target_id, l.link_type, l.created_by, l.created_at,
s.title, t.title, s.slug, t.slug, sc.slug, tc.slug, sc.prefix, tc.prefix,
s.item_number, t.item_number,
%s, %s
FROM item_links l
JOIN items s ON s.id = l.source_id AND s.deleted_at IS NULL
JOIN items t ON t.id = l.target_id AND t.deleted_at IS NULL
JOIN collections sc ON sc.id = s.collection_id
JOIN collections tc ON tc.id = t.collection_id
WHERE l.source_id = ? AND l.link_type IN (%s)
`, sStatusExpr, tStatusExpr, childLinkTypeSQL())), itemID)
if err != nil {
return nil, fmt.Errorf("get parent for item: %w", err)
}
defer rows.Close()
if !rows.Next() {
return nil, nil
}
var link models.ItemLink
var createdAt string
var sourcePrefix, targetPrefix string
var sourceItemNumber, targetItemNumber sql.NullInt64
var sourceStatus, targetStatus sql.NullString
if err := rows.Scan(
&link.ID, &link.WorkspaceID, &link.SourceID, &link.TargetID,
&link.LinkType, &link.CreatedBy, &createdAt,
&link.SourceTitle, &link.TargetTitle,
&link.SourceSlug, &link.TargetSlug,
&link.SourceCollectionSlug, &link.TargetCollectionSlug,
&sourcePrefix, &targetPrefix,
&sourceItemNumber, &targetItemNumber,
&sourceStatus, &targetStatus,
); err != nil {
return nil, fmt.Errorf("scan parent link: %w", err)
}
link.CreatedAt = parseTime(createdAt)
if sourceItemNumber.Valid && sourcePrefix != "" {
link.SourceRef = fmt.Sprintf("%s-%d", sourcePrefix, sourceItemNumber.Int64)
}
if targetItemNumber.Valid && targetPrefix != "" {
link.TargetRef = fmt.Sprintf("%s-%d", targetPrefix, targetItemNumber.Int64)
}
if sourceStatus.Valid {
link.SourceStatus = sourceStatus.String
}
if targetStatus.Valid {
link.TargetStatus = targetStatus.String
}
return &link, nil
}
// GetParentMap returns a map of item ID -> parent item ID for all parent links
// in a workspace. Used for efficient batch lookups (e.g., dashboard, list enrichment).
//
// Links whose source or target item is soft-deleted are excluded so that
// dashboard orphan-detection (handlers_dashboard.go) and similar enrichment
// passes don't treat a task whose parent has been archived as still parented.
// See BUG-734.
func (s *Store) GetParentMap(workspaceID string) (map[string]string, error) {
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT il.source_id, il.target_id FROM item_links il
JOIN items s ON s.id = il.source_id AND s.deleted_at IS NULL
JOIN items t ON t.id = il.target_id AND t.deleted_at IS NULL
WHERE il.workspace_id = ? AND il.link_type IN (%s)
`, childLinkTypeSQL())), workspaceID)
if err != nil {
return nil, fmt.Errorf("get parent map: %w", err)
}
defer rows.Close()
m := make(map[string]string)
for rows.Next() {
var sourceID, targetID string
if err := rows.Scan(&sourceID, &targetID); err != nil {
return nil, err
}
m[sourceID] = targetID
}
return m, rows.Err()
}
// LineageRef is a skinny projection of a parent item — only the fields
// parent-link enrichment decorates onto children (title, ref, slug, and the
// collection info needed for the visibility filter). Fetched in one batch
// query instead of a full-row GetItem per parent. See BUG-2003.
type LineageRef struct {
ID string
Title string
Ref string
Slug string
CollectionID string
CollectionSlug string
}
// GetItemLineageByIDs fetches skinny parent-lineage projections for the given
// item IDs in a single `WHERE id IN (...)` query. Soft-deleted items are
// excluded. IDs with no matching (or soft-deleted) row are simply absent from
// the returned map — parent enrichment is best-effort decoration, so a missing
// parent must not fail the caller.
//
// This replaces the per-parent GetItem N+1 in enrichItemsWithParent (BUG-2003):
// callers scope the ID slice to only the parents of the returned items, then
// hydrate all of them in one round-trip.
func (s *Store) GetItemLineageByIDs(ids []string) (map[string]LineageRef, error) {
result := make(map[string]LineageRef, len(ids))
if len(ids) == 0 {
return result, nil
}
placeholders := make([]string, len(ids))
args := make([]any, len(ids))
for i, id := range ids {
placeholders[i] = "?"
args[i] = id
}
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT i.id, i.title, i.slug, i.item_number, i.collection_id, c.slug, c.prefix
FROM items i
JOIN collections c ON c.id = i.collection_id
WHERE i.id IN (%s) AND i.deleted_at IS NULL
`, strings.Join(placeholders, ","))), args...)
if err != nil {
return nil, fmt.Errorf("get item lineage by ids: %w", err)
}
defer rows.Close()
for rows.Next() {
var ref LineageRef
var itemNumber *int
var prefix string
if err := rows.Scan(&ref.ID, &ref.Title, &ref.Slug, &itemNumber, &ref.CollectionID, &ref.CollectionSlug, &prefix); err != nil {
return nil, err
}
if prefix != "" && itemNumber != nil {
ref.Ref = fmt.Sprintf("%s-%d", prefix, *itemNumber)
}
result[ref.ID] = ref
}
return result, rows.Err()
}
// --- Child Item Progress ---
// GetItemProgress counts total and done child items linked to a parent via item_links.
// "Done" means the child item's done field (resolved from its collection's
// board_group_by, defaulting to status) matches one of that field's terminal
// options. Children from any collection count toward progress, and each
// child is evaluated against its own collection's done rules.
func (s *Store) GetItemProgress(parentItemID string) (total int, done int, err error) {
filters := s.childrenDoneFiltersForParent(parentItemID)
doneExpr, doneArgs := s.buildChildrenDoneExpr(filters, "i")
args := append(doneArgs, parentItemID)
err = s.db.QueryRow(s.q(fmt.Sprintf(`
SELECT COUNT(*),
COUNT(CASE WHEN %s THEN 1 END)
FROM items i
JOIN item_links il ON il.source_id = i.id AND il.link_type IN (%s) AND il.target_id = ?
WHERE i.deleted_at IS NULL
`, doneExpr, childLinkTypeSQL())), args...).Scan(&total, &done)
if err != nil {
return 0, 0, fmt.Errorf("get item progress: %w", err)
}
return total, done, nil
}
// collectionDoneFilter describes how to evaluate "done" for a single child
// collection: which JSON key to read, and which values count as terminal.
type collectionDoneFilter struct {
collectionID string
doneKey string
values []string
}
// childrenDoneFiltersForParent returns a filter per distinct child-item
// collection under the given parent. Each filter carries the child
// collection's resolved done field (honoring board_group_by) and terminal
// values so the caller can build a per-collection OR clause that evaluates
// each child against its own done rules.
//
// Soft-deleted collections are intentionally INCLUDED: progress-counting
// callers count items regardless of their collection's deleted_at, so
// excluding the collection here would leave those items without a
// matching per-collection clause and cause them to always evaluate as
// non-terminal. The collection row still carries valid schema + settings
// until a hard delete cascades, so the done rules remain applicable.
func (s *Store) childrenDoneFiltersForParent(parentItemID string) []collectionDoneFilter {
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT DISTINCT c.id, c.schema, c.settings
FROM items i
JOIN collections c ON c.id = i.collection_id
JOIN item_links il ON il.source_id = i.id AND il.link_type IN (%s) AND il.target_id = ?
WHERE i.deleted_at IS NULL
`, childLinkTypeSQL())), parentItemID)
if err != nil {
return nil
}
defer rows.Close()
return scanCollectionDoneFilters(rows)
}
// doneFiltersForWorkspace returns a done-filter per collection in the
// workspace. Used by cross-collection queries (e.g. agent-role
// breakdowns) that need to evaluate "is done?" for every item regardless
// of which collection it belongs to.
//
// Includes soft-deleted collections: callers (e.g. GetRoleBreakdown)
// count items in the workspace without filtering by collection
// deleted_at, so excluding soft-deleted collections here would leave
// their items without a matching per-collection clause and cause them
// to always register as non-terminal.
func (s *Store) doneFiltersForWorkspace(workspaceID string) []collectionDoneFilter {
rows, err := s.db.Query(
s.q(`SELECT id, schema, settings FROM collections WHERE workspace_id = ?`),
workspaceID,
)
if err != nil {
return nil
}
defer rows.Close()
return scanCollectionDoneFilters(rows)
}
// childrenDoneFiltersForCollection is the batch version: it gathers one
// filter per distinct child-item collection across all parent→child links
// for parents in a given (workspace, collectionSlug).
//
// Includes soft-deleted child collections for the same reason as
// childrenDoneFiltersForParent — callers count items regardless of their
// collection's deleted_at, and we want items from soft-deleted
// collections to still be evaluated against their own done rules.
//
// includeArchived mirrors the same flag in GetAllItemProgress: when true,
// the parent-row join does NOT filter out archived parents. This matters
// because if a child collection's only parent links point to archived
// parents, the collection would be absent from the filter map under the
// live-only predicate — causing those children to fall back to default
// done semantics and producing wrong done counts in the main query.
func (s *Store) childrenDoneFiltersForCollection(workspaceID, collectionSlug string, includeArchived bool) []collectionDoneFilter {
parentDeletedFilter := "AND p.deleted_at IS NULL"
if includeArchived {
parentDeletedFilter = ""
}
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT DISTINCT c.id, c.schema, c.settings
FROM items t
JOIN collections c ON c.id = t.collection_id
JOIN item_links il ON il.source_id = t.id AND il.link_type IN (%s)
JOIN items p ON p.id = il.target_id %s
JOIN collections pc ON pc.id = p.collection_id AND pc.slug = ?
WHERE p.workspace_id = ?
AND t.deleted_at IS NULL
`, childLinkTypeSQL(), parentDeletedFilter)), collectionSlug, workspaceID)
if err != nil {
return nil
}
defer rows.Close()
return scanCollectionDoneFilters(rows)
}
// scanCollectionDoneFilters consumes rows yielding (id, schema, settings)
// and resolves each into a collectionDoneFilter.
//
// When a collection's schema fails to parse we still emit a filter — one
// that falls back to the `status` field and the global default terminal
// list. Silently skipping the collection would leave its items without a
// matching per-collection clause in buildChildrenDoneExpr, so they'd
// always register as non-terminal in progress / role / starred queries
// (a malformed schema on one collection would skew counts on every
// parent-progress computation).
func scanCollectionDoneFilters(rows *sql.Rows) []collectionDoneFilter {
var filters []collectionDoneFilter
for rows.Next() {
var id, schemaJSON, settingsJSON string
if err := rows.Scan(&id, &schemaJSON, &settingsJSON); err != nil {
continue
}
var schema models.CollectionSchema
if err := json.Unmarshal([]byte(schemaJSON), &schema); err != nil {
// Malformed schema → emit a default-fallback filter so the
// collection's items still get evaluated against the status
// column + global default terminals. This matches pre-TASK-604
// behavior for those items.
filters = append(filters, collectionDoneFilter{
collectionID: id,
doneKey: "status",
values: models.DefaultTerminalStatuses,
})
continue
}
var settings models.CollectionSettings
if settingsJSON != "" {
_ = json.Unmarshal([]byte(settingsJSON), &settings)
}
key, values := models.TerminalValuesForDoneField(schema, settings)
filters = append(filters, collectionDoneFilter{
collectionID: id,
doneKey: key,
values: values,
})
}
return filters
}
// buildChildrenDoneExpr compiles a set of per-collection done filters into
// a single SQL boolean expression plus ordered args. `itemAlias` is the
// item-table alias in the outer query (e.g. "i" for GetItemProgress, "t"
// for GetAllItemProgress).
//
// Expression shape:
//
// ((<alias>.collection_id = ? AND LOWER(COALESCE(<field_A>, '')) IN (?,?)) OR
// (<alias>.collection_id = ? AND LOWER(COALESCE(<field_B>, '')) IN (?,?)))
//
// The `<field_X>` JSON extract uses scalar text extraction; this works
// because DoneFieldKey in the models package only resolves done fields to
// `select` typed columns (see that function's doc). multi_select-backed
// done fields would store their values as a JSON array and scalar IN
// matching would silently miss them — hence the upstream restriction.
//
// If no filters were constructed (no child collections discovered, or all
// of their schemas failed to parse), falls back to checking <alias>.status
// against the global default terminal list — mirroring the legacy behavior
// so dashboards for untyped collections keep working.
func (s *Store) buildChildrenDoneExpr(filters []collectionDoneFilter, itemAlias string) (string, []any) {
if len(filters) == 0 {
statusExpr := s.dialect.JSONExtractText(itemAlias+".fields", "status")
placeholders, args := models.DefaultTerminalStatusPlaceholders()
return fmt.Sprintf("LOWER(COALESCE(%s, '')) IN (%s)", statusExpr, placeholders), args
}
clauses := make([]string, 0, len(filters))
args := make([]any, 0, len(filters)*4)
for _, f := range filters {
fieldExpr := s.dialect.JSONExtractText(itemAlias+".fields", f.doneKey)
placeholders := make([]string, len(f.values))
args = append(args, f.collectionID)
for i, v := range f.values {
placeholders[i] = "?"
args = append(args, strings.ToLower(v))
}
clauses = append(clauses, fmt.Sprintf(
"(%s.collection_id = ? AND LOWER(COALESCE(%s, '')) IN (%s))",
itemAlias, fieldExpr, strings.Join(placeholders, ","),
))
}
return "(" + strings.Join(clauses, " OR ") + ")", args
}
// nonTerminalFilter builds a WHERE fragment (plus ordered args) that keeps
// only items whose resolved done-field value is NOT one of their
// collection's terminal options. It reuses the same per-collection done
// machinery as parent-progress (doneFiltersForWorkspace +
// buildChildrenDoneExpr): buildChildrenDoneExpr yields an expression that
// is TRUE when an item is terminal, so negating it selects the
// non-terminal set.
//
// Each collection is evaluated against its OWN terminal_options (with the
// global DefaultTerminalStatuses fallback for schemas that declare none),
// so collections with custom status vocabularies are handled correctly
// rather than against a hardcoded global allowlist (BUG-2001). This is the
// server-side default that both the CLI (`pad item list` with no --status/
// --all) and the MCP `pad_item.action=list` inherit.
//
// itemAlias is the item-table alias in the outer query (e.g. "i").
func (s *Store) nonTerminalFilter(workspaceID, itemAlias string) (string, []any) {
filters := s.doneFiltersForWorkspace(workspaceID)
doneExpr, args := s.buildChildrenDoneExpr(filters, itemAlias)
return "NOT " + doneExpr, args
}
// ItemProgress holds child item completion counts for a single parent item.
type ItemProgress struct {
ItemID string `json:"item_id"`
Total int `json:"total"`
Done int `json:"done"`
}
// GetAllItemProgress returns child item completion counts for every item in
// the given collection within a workspace.
//
// includeArchived controls whether soft-deleted parent items contribute rows.
// When false (the default for /plans-progress) only live parents are returned.
// When true (used by /child-progress with include_archived=true) archived
// parents also appear — matching the archived-toggle semantics on the
// collection page (mirrors CollectionCheckboxProgress's includeArchived param).
func (s *Store) GetAllItemProgress(workspaceID, collectionSlug string, includeArchived bool) ([]ItemProgress, error) {
filters := s.childrenDoneFiltersForCollection(workspaceID, collectionSlug, includeArchived)
doneExpr, doneArgs := s.buildChildrenDoneExpr(filters, "t")
args := append(doneArgs, workspaceID, collectionSlug)
parentDeletedFilter := "AND p.deleted_at IS NULL"
if includeArchived {
parentDeletedFilter = ""
}
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT p.id,
COUNT(t.id),
COUNT(CASE WHEN t.id IS NOT NULL AND %s THEN 1 END)
FROM items p
JOIN collections pc ON pc.id = p.collection_id
LEFT JOIN item_links il ON il.link_type IN (%s) AND il.target_id = p.id
LEFT JOIN items t ON t.id = il.source_id
AND t.deleted_at IS NULL
WHERE p.workspace_id = ?
AND pc.slug = ?
%s
GROUP BY p.id
`, doneExpr, childLinkTypeSQL(), parentDeletedFilter)), args...)
if err != nil {
return nil, fmt.Errorf("get all item progress: %w", err)
}
defer rows.Close()
var result []ItemProgress
for rows.Next() {
var ip ItemProgress
if err := rows.Scan(&ip.ItemID, &ip.Total, &ip.Done); err != nil {
return nil, fmt.Errorf("scan item progress: %w", err)
}
result = append(result, ip)
}
if result == nil {
result = []ItemProgress{}
}
return result, rows.Err()
}
// GetChildItems returns all non-deleted child items linked to the given parent
// via item_links. Returns children from any collection.
func (s *Store) GetChildItems(parentItemID string) ([]models.Item, error) {
return s.getChildItems(s.db, parentItemID)
}
// GetChildItemsTx is the in-transaction variant of GetChildItems. The
// underlying query is the same as GetChildItems; using a *sql.Tx ties
// the read to the caller's transaction so it sees the same snapshot
// the subsequent UPDATE will write against (IDEA-1494 R2).
//
// Atomicity vs. concurrent child mutations is provided by the caller's
// transaction-scoped locking:
//
// - SQLite: db-wide BEGIN IMMEDIATE write lock (set globally via
// `_txlock=immediate`) serializes all writers, so any concurrent
// child insert / child update blocks until this tx commits or
// rolls back. No additional locking is needed.
// - Postgres: the caller is expected to hold a parent-keyed advisory
// lock (see AcquireParentChildrenLocks below) so concurrent
// mutations on the same parent's children are serialized against
// this read.
//
// FOR UPDATE is intentionally NOT used — the underlying SELECT carries
// DISTINCT (necessary because item_links can carry both `parent` and
// the legacy `plan` link_type for the same edge), and Postgres rejects
// `SELECT DISTINCT … FOR UPDATE`. The advisory-lock pattern sidesteps
// that constraint while still giving us a serialized snapshot.
func (s *Store) GetChildItemsTx(tx *sql.Tx, parentItemID string) ([]models.Item, error) {
if tx == nil {
return s.GetChildItems(parentItemID)
}
return s.getChildItems(tx, parentItemID)
}
// acquireParentChildrenLocksForUpdate is the in-tx helper UpdateItem
// uses to serialize itself against the open-children guard
// (IDEA-1494 R2 / R4). It acquires the parent-children advisory lock
// for:
//
// 1. EVERY parent the item is currently a child of (via item_links
// of type ∈ childLinkTypes — `parent` AND `implements`).
// childLinkTypes is the inclusion rule GetChildItems walks, so
// this set is exactly the parents whose guard precheck could see
// this item as a child.
// 2. THIS item itself, as a parent — so any concurrent precheck
// running against this item's own children list waits.
//
// Codex round-4 P1: pre-fix this helper used `LIMIT 1` and only
// locked one parent. A child of TWO parents (one via `parent`, one
// via `implements`) would let a status flip race against the
// un-locked parent's precheck. The query now returns ALL distinct
// parent target_ids and we lock every one.
//
// All keys (parents + self) are funnelled through
// AcquireParentChildrenLocks so they're acquired in a single,
// canonical sorted order — round-4 P2's deadlock-avoidance contract.
// No call site outside this helper takes pad:parent-children:* locks
// in any other order.
//
// extraKeys lets a caller fold additional parent IDs into the SAME sorted
// batch — UpdateItemWithParentLink passes the NEW parent when an atomic
// parent-link change accompanies the field write. Acquiring the new parent
// in this initial sorted acquisition (rather than later, inside
// setParentLinkTx) preserves the canonical lock ordering and keeps the
// combined update deadlock-free.
//
// BUG-2073: the parent set is read BEFORE the locks are held, so a concurrent
// reparent of this item can commit before we acquire the item's own (itemID)
// key and add a parent we didn't lock. Once itemID is held the parent set is
// frozen, so we re-read it; if a new parent appeared, we signal
// errParentSetChanged (rather than acquiring it out of the canonical sorted
// order) so the tx-owning caller rolls back and retries — on the retry the new
// parent is included in the INITIAL sorted batch, keeping acquisition
// deadlock-free. Every tx-owning caller wraps its body in
// retryOnParentSetChanged.
func (s *Store) acquireParentChildrenLocksForUpdate(tx *sql.Tx, itemID string, extraKeys ...string) error {
if s.dialect.Driver() != DriverPostgres {
return nil
}
parentIDs, err := s.listParentChildLockKeys(tx, itemID)
if err != nil {
return err
}
keys := append(parentIDs, itemID)
keys = append(keys, extraKeys...)
if err := s.AcquireParentChildrenLocks(tx, keys...); err != nil {
return err
}
// Re-read the parent set under the now-held itemID lock. Any parent that
// appeared during the acquisition window is not covered by the locks we
// took, so bail out for a retry rather than close the open-children guard
// serialization gap with an out-of-order grab.
reParentIDs, err := s.listParentChildLockKeys(tx, itemID)
if err != nil {
return err
}
if newKeys := keysNotIn(keys, reParentIDs); len(newKeys) > 0 {
return errParentSetChanged
}
return nil
}
// errParentSetChanged is the retry sentinel for BUG-2073: a parent-children
// lock acquisition re-read the item's parent set under its own lock and found
// it had moved during the acquisition window. Acquiring the newly-appeared key
// in-place could violate AcquireParentChildrenLocks' canonical sorted order, so
// the tx-owning caller instead rolls back (releasing every advisory lock) and
// retries from a fresh read via retryOnParentSetChanged. Because the item's own
// lock is always in the batch, the parent set is frozen once acquired, so a
// retry converges in one extra attempt in the overwhelmingly common case.
var errParentSetChanged = errors.New("parent-children lock set changed during acquisition; retry")
// maxParentLockRetries bounds retryOnParentSetChanged so a pathological stream
// of concurrent reparents of the same item can't spin forever. Reaching the
// cap surfaces the sentinel as a real error rather than corrupting state.
const maxParentLockRetries = 8
// retryOnParentSetChanged runs fn, retrying (up to maxParentLockRetries) while
// it returns errParentSetChanged. fn MUST open and own its own transaction and
// roll it back on any error (the standard `defer tx.Rollback()` pattern), so
// each attempt starts from a clean slate with all advisory locks released.
func retryOnParentSetChanged[T any](fn func() (T, error)) (T, error) {
var zero T
for attempt := 0; attempt < maxParentLockRetries; attempt++ {
v, err := fn()
if errors.Is(err, errParentSetChanged) {
continue
}
return v, err
}
return zero, fmt.Errorf("parent-children lock set kept changing after %d attempts: %w", maxParentLockRetries, errParentSetChanged)
}
// keysNotIn returns the entries of want that are not already present in have.
// Used to detect parent lock keys that appeared on a post-lock re-read
// (BUG-2073).
func keysNotIn(have, want []string) []string {
if len(want) == 0 {
return nil
}
seen := make(map[string]struct{}, len(have))
for _, k := range have {
seen[k] = struct{}{}
}
var out []string
for _, k := range want {
if k == "" {
continue
}
if _, ok := seen[k]; ok {
continue
}
seen[k] = struct{}{} // dedupe within want too
out = append(out, k)
}
return out
}
// listParentChildLockKeys returns every target_id this item is the
// `source_id` of under a childLinkTypes link — i.e. every parent
// whose children-set includes this item. Used wherever we need to
// lock all of an item's parents at once (UpdateItem, RestoreItem,
// link-mutation paths).
//
// IMPORTANT: this MUST stay in lockstep with childLinkTypes (the
// inclusion rule GetChildItems uses). If a new link type joins the
// children-set, both the query here and the read query must add it
// together so lock coverage matches read coverage.
func (s *Store) listParentChildLockKeys(tx *sql.Tx, itemID string) ([]string, error) {
rows, err := tx.Query(s.q(fmt.Sprintf(`
SELECT DISTINCT target_id FROM item_links
WHERE source_id = ? AND link_type IN (%s)
`, childLinkTypeSQL())), itemID)
if err != nil {
return nil, fmt.Errorf("list parent lock keys: %w", err)
}
defer rows.Close()
var out []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, fmt.Errorf("scan parent lock key: %w", err)
}
if id == "" || id == itemID {
continue
}
out = append(out, id)
}
return out, rows.Err()
}
// AcquireParentChildrenLocks is the CANONICAL helper for taking
// `pad:parent-children:<id>` advisory locks. Every call site that
// needs to serialize against the open-children guard MUST go through
// this function — UpdateItemWithPreCheck precheck, MoveItemWithPreCheck
// precheck, RestoreItem, SetParentLink, ClearParentLink,
// CreateItemLink (child-link types), DeleteItemLink (child-link
// types). Ad-hoc single-key acquisition outside this helper is
// FORBIDDEN — two call sites taking distinct keys in different
// orders WILL deadlock under contention (the classic AB/BA shape).
//
// The contract this helper enforces:
//
// 1. Deduplicate. Repeated IDs in the input collapse to one lock.
// 2. Drop empties. "" / nil entries don't get locked.
// 3. Acquire in canonical sorted order (string-sort by ID).
// Two concurrent callers that share any subset of IDs always
// grab the overlap in the same order → no deadlock.
//
// SQLite is a no-op because the global BEGIN IMMEDIATE write lock
// (set via _txlock=immediate in store.go) already serializes every
// writer; advisory locks would add no protection there.
//
// Per Codex round-3 P1 (link-mutations bypass) + round-4 P2
// (lock-order asymmetry). If you find yourself writing
// `pg_advisory_xact_lock(... 'pad:parent-children:' ...)` anywhere
// outside this helper, route it through here instead.
func (s *Store) AcquireParentChildrenLocks(tx *sql.Tx, parentItemIDs ...string) error {
if s.dialect.Driver() != DriverPostgres {
return nil
}
seen := make(map[string]struct{}, len(parentItemIDs))
keys := make([]string, 0, len(parentItemIDs))
for _, id := range parentItemIDs {
if id == "" {
continue
}
if _, ok := seen[id]; ok {
continue
}
seen[id] = struct{}{}
keys = append(keys, id)
}
sort.Strings(keys)
for _, k := range keys {
if _, err := tx.Exec("SELECT pg_advisory_xact_lock(hashtext('pad:parent-children:' || $1))", k); err != nil {
return fmt.Errorf("acquire parent-children lock %q: %w", k, err)
}
}
return nil
}
// isChildLinkType reports whether the given link type is one the
// open-children guard counts toward the children-set (i.e. matches
// the inclusion rule baked into `childLinkTypes` and used by
// GetChildItems via `childLinkTypeSQL()`). Single source of truth so
// link-writer lock acquisition can't drift from the read query's set.
func isChildLinkType(linkType string) bool {
for _, t := range childLinkTypes {
if t == linkType {
return true
}
}
return false
}
// childQueryer is the small surface the children-list query needs from
// either *sql.DB or *sql.Tx. Lets getChildItems serve both the unlocked
// and tx-bound paths from one implementation.
type childQueryer interface {
Query(query string, args ...any) (*sql.Rows, error)
}
func (s *Store) getChildItems(q childQueryer, parentItemID string) ([]models.Item, error) {
rows, err := q.Query(s.q(fmt.Sprintf(`
SELECT DISTINCT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.content, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''), i.deleted_at
FROM items i
JOIN collections c ON c.id = i.collection_id
JOIN item_links il ON il.source_id = i.id AND il.link_type IN (%s) AND il.target_id = ?
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.deleted_at IS NULL
ORDER BY i.sort_order ASC, i.created_at ASC
`, childLinkTypeSQL())), parentItemID)
if err != nil {
return nil, fmt.Errorf("get child items: %w", err)
}
defer rows.Close()
return scanItems(rows)
}
// GetChildItemsForParents returns the live child items for each of the given
// parent item IDs, grouped by parent ID, in ONE query — collapsing the
// per-parent GetChildItems N+1 the dashboard used to run once per active plan
// (BUG-2002). The rich-text body (content) is omitted from the projection:
// every dashboard consumer of plan children reads only structured fields
// (status/priority) + identity, never the markdown body, so we skip the heavy
// column.
//
// Ordering within each parent's slice matches GetChildItems (sort_order ASC,
// created_at ASC). A child linked to more than one requested parent appears
// under each. Parents with no live children are simply absent from the map.
func (s *Store) GetChildItemsForParents(parentIDs []string) (map[string][]models.Item, error) {
result := make(map[string][]models.Item, len(parentIDs))
if len(parentIDs) == 0 {
return result, nil
}
placeholders := make([]string, len(parentIDs))
args := make([]any, len(parentIDs))
for i, id := range parentIDs {
placeholders[i] = "?"
args[i] = id
}
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT DISTINCT il.target_id,
i.id, i.workspace_id, i.collection_id, i.title, i.slug, '', i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''), i.deleted_at
FROM items i
JOIN collections c ON c.id = i.collection_id
JOIN item_links il ON il.source_id = i.id AND il.link_type IN (%s) AND il.target_id IN (%s)
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.deleted_at IS NULL
ORDER BY i.sort_order ASC, i.created_at ASC
`, childLinkTypeSQL(), strings.Join(placeholders, ","))), args...)
if err != nil {
return nil, fmt.Errorf("get child items for parents: %w", err)
}
defer rows.Close()
for rows.Next() {
var parentID string
var item models.Item
var createdAt, updatedAt string
var deletedAt *string
var pinned bool
if err := rows.Scan(
&parentID,
&item.ID, &item.WorkspaceID, &item.CollectionID, &item.Title, &item.Slug,
&item.Content, &item.Fields, &item.Tags,
&pinned, &item.SortOrder, &item.ParentID, &item.AssignedUserID, &item.AgentRoleID, &item.RoleSortOrder,
&item.CreatedBy, &item.LastModifiedBy, &item.Source,
&item.ItemNumber, &item.Seq, &createdAt, &updatedAt,
&item.CollectionSlug, &item.CollectionName, &item.CollectionIcon, &item.CollectionPrefix,
&item.AssignedUserName, &item.AssignedUserEmail,
&item.AgentRoleName, &item.AgentRoleSlug, &item.AgentRoleIcon,
&deletedAt,
); err != nil {
return nil, err
}
item.Pinned = pinned
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
item.DeletedAt = parseTimePtr(deletedAt)
hydrateItemComputedMetadata(&item)
result[parentID] = append(result[parentID], item)
}
return result, rows.Err()
}
// BlocksEdge is a skinny projection of a `blocks` link plus the blocker
// (source) essentials the dashboard needs to decide whether a blocked item
// should be flagged: the blocker's visibility input (collection ID), its
// done-state input (fields), and its title/status for the reason string.
// Fetched for a whole workspace in one query instead of the per-item
// GetItemLinks + per-link GetItem N+1 the dashboard used to run (BUG-2002).
type BlocksEdge struct {
TargetID string // the blocked item (the `blocks` link's target)
SourceID string // the blocker (the `blocks` link's source)
SourceTitle string
SourceCollectionID string
SourceFields string
}
// GetBlocksEdges returns every `blocks` link in the workspace whose source and
// target items are both live (non-deleted), newest link first. The
// created_at DESC ordering matches GetItemLinks so callers replicating the
// dashboard's "first active blocker wins" selection pick the same blocker the
// per-item path did. Replaces the dashboard's per-non-done-item
// GetItemLinks + per-link GetItem N+1 (BUG-2002).
func (s *Store) GetBlocksEdges(workspaceID string) ([]BlocksEdge, error) {
rows, err := s.db.Query(s.q(`
SELECT l.target_id, l.source_id, s.title, s.collection_id, s.fields
FROM item_links l
JOIN items s ON s.id = l.source_id AND s.deleted_at IS NULL
JOIN items t ON t.id = l.target_id AND t.deleted_at IS NULL
WHERE l.workspace_id = ? AND l.link_type = 'blocks'
ORDER BY l.created_at DESC
`), workspaceID)
if err != nil {
return nil, fmt.Errorf("get blocks edges: %w", err)
}
defer rows.Close()
var edges []BlocksEdge
for rows.Next() {
var e BlocksEdge
if err := rows.Scan(&e.TargetID, &e.SourceID, &e.SourceTitle, &e.SourceCollectionID, &e.SourceFields); err != nil {
return nil, err
}
edges = append(edges, e)
}
return edges, rows.Err()
}
// PopulateHasChildren sets HasChildren=true on items that have at least one
// child linked via parent link_type. Operates in-place on the slice.
func (s *Store) PopulateHasChildren(items []models.Item) {
if len(items) == 0 {
return
}
// Build ID list and index
ids := make([]string, len(items))
idx := make(map[string]int, len(items))
for i, item := range items {
ids[i] = item.ID
idx[item.ID] = i
}
// Batch query: which of these IDs are targets of a parent link?
placeholders := make([]string, len(ids))
args := make([]any, len(ids))
for i, id := range ids {
placeholders[i] = "?"
args[i] = id
}
query := fmt.Sprintf(`
SELECT DISTINCT il.target_id FROM item_links il
JOIN items child ON child.id = il.source_id AND child.deleted_at IS NULL
WHERE il.link_type IN (%s) AND il.target_id IN (%s)
`, childLinkTypeSQL(), strings.Join(placeholders, ","))
rows, err := s.db.Query(s.q(query), args...)
if err != nil {
return // best-effort; don't fail the whole request
}
defer rows.Close()
for rows.Next() {
var targetID string
if err := rows.Scan(&targetID); err != nil {
continue
}
if i, ok := idx[targetID]; ok {
items[i].HasChildren = true
}
}
}
// MoveItem moves an item to a different collection within the same workspace.
// It updates the collection_id and fields JSON. The item_number is preserved
// because numbering is workspace-global — the number stays the same, only the
// collection prefix changes (e.g. IDEA-42 → BUG-42).
//
// The move also bumps the workspace-scoped seq so delta-sync clients
// see the collection change (PLAN-1343 / TASK-1352). Without it a
// client polling /items-changes?since=cursor would render the item
// under its old collection until a full refresh.
func (s *Store) MoveItem(itemID, targetCollectionID, newFieldsJSON string) (*models.Item, error) {
return s.MoveItemWithPreCheck(itemID, targetCollectionID, newFieldsJSON, nil)
}
// MoveItemWithPreCheck is MoveItem with the same precheck escape hatch
// UpdateItemWithPreCheck offers. Codex round-3 P1: a `pad item move
// ... --field status=done` writes a terminal done-field value through
// MoveItem, bypassing the open-children guard wired into the regular
// UpdateItem path. This variant runs the caller's invariant check
// inside the move's transaction, after acquiring the workspace seq
// lock + the parent-children lock for this item's own parent (it CAN
// itself be a parent — children stay attached across collection
// changes — so we lock for itself too, matching
// acquireParentChildrenLocksForUpdate's shape).
//
// The precheck receives a fresh in-tx snapshot of the item, same as
// UpdateItemWithPreCheck (the pre-tx `existing` is replaced).
func (s *Store) MoveItemWithPreCheck(
itemID, targetCollectionID, newFieldsJSON string,
precheck func(tx *sql.Tx, existing *models.Item) error,
opts ...MutationOption,
) (*models.Item, error) {
opt := newMutationOptions(opts)
// BUG-2073: retry if the item's parent set moves during lock acquisition.
return retryOnParentSetChanged(func() (*models.Item, error) {
return s.moveItemWithPreCheckOnce(itemID, targetCollectionID, newFieldsJSON, precheck, opt)
})
}
func (s *Store) moveItemWithPreCheckOnce(
itemID, targetCollectionID, newFieldsJSON string,
precheck func(tx *sql.Tx, existing *models.Item) error,
opt mutationOptions,
) (*models.Item, error) {
existing, err := s.GetItem(itemID)
if err != nil {
return nil, err
}
if existing == nil {
return nil, sql.ErrNoRows
}
tx, err := s.db.Begin()
if err != nil {
return nil, err
}
defer tx.Rollback()
if err := s.acquireWorkspaceSeqLock(tx, existing.WorkspaceID); err != nil {
return nil, err
}
if err := s.acquireParentChildrenLocksForUpdate(tx, itemID); err != nil {
return nil, err
}
if precheck != nil {
freshExisting, ferr := s.getItemTx(tx, itemID)
if ferr != nil {
return nil, fmt.Errorf("re-read item under lock: %w", ferr)
}
if freshExisting == nil {
return nil, sql.ErrNoRows
}
if err := precheck(tx, freshExisting); err != nil {
return nil, err
}
existing = freshExisting
}
// Capture the pre-move status under lock before the UPDATE, mirroring the
// UpdateItemWithPreCheck path: `existing` is the fresh in-tx snapshot when
// a precheck ran, otherwise re-read so a concurrent write doesn't make
// from_status stale. The done field resolves against the TARGET collection
// (where the item now lives and which reports group by); for a move that
// also crosses to a collection with a different done field, the old value
// read through that key may be empty, which correctly reads as "entered".
moveDoneKey := s.doneFieldKey(targetCollectionID)
oldFields := existing.Fields
// preMove is the item as it stands UNDER THE LOCK, immediately before the
// UPDATE. It is what the event decisions below compare against, and it is
// deliberately not `existing`: `existing` is only refreshed in-tx on the
// precheck path, so on the no-precheck path it is the PRE-LOCK pool read
// and every field on it — CollectionID included — may already be stale.
//
// That matters twice over for TASK-2658. A stale CollectionID makes the
// item.moved decision wrong outright. And itemUpdatedSliceChanged
// documents a precondition that BOTH snapshots come from getItemTx,
// because a pool read and an in-tx read can render join-populated fields
// differently and the diff would report those differences as changes —
// comparing `existing` against the post-move snapshot violated exactly
// that precondition (Codex round 1, P2).
preMove := existing
if precheck == nil {
// Read it once, and treat a failure as a failure. The previous shape
// tolerated a read error by silently keeping the pre-lock value, which
// only ever degraded from_status; now it would also silently decide
// which events to emit, so a degraded read is no longer an acceptable
// outcome. Under a held lock on a row we just resolved live, an error
// or a missing row means something is genuinely wrong.
fresh, ferr := s.getItemTx(tx, itemID)
if ferr != nil {
return nil, fmt.Errorf("re-read item under lock: %w", ferr)
}
if fresh == nil {
return nil, sql.ErrNoRows
}
oldFields = fresh.Fields
preMove = fresh
}
// A move's field OVERRIDES can inject a brand-new pad-attachment:
// reference into the rewritten fields blob — this write bypasses the
// UpdateItem core, so stamp here too, BEFORE the UPDATE (BUG-2415,
// codex rounds 1 #2 and 3; see the ORDERING note on
// stampAttachmentRefsTx).
if err := stampAttachmentRefsTx(tx, s, existing.WorkspaceID, newFieldsJSON); err != nil {
return nil, err
}
moveTS := time.Now().UTC().Format(time.RFC3339)
_, err = tx.Exec(s.q(`
UPDATE items
SET collection_id = ?, fields = ?, updated_at = ?, seq = `+nextWorkspaceSeqSubquery+`
WHERE id = ? AND deleted_at IS NULL`),
targetCollectionID, newFieldsJSON, moveTS, existing.WorkspaceID, itemID)
if err != nil {
return nil, fmt.Errorf("move item: %w", err)
}
// A move can carry a status-changing field override (e.g.
// `pad item move ... --field status=done`), which rewrites `fields`
// outside the UpdateItemWithPreCheck path. Record the transition here
// too so status_transitions stays the canonical source for reports
// (PLAN-1628 / TASK-1637). collection_id reflects the TARGET collection
// the item now lives in. Same tx, not debounced.
oldStatus := extractFieldValue(oldFields, moveDoneKey)
newStatus := extractFieldValue(newFieldsJSON, moveDoneKey)
// Record any done-field change, including a clear (X → "") — see the
// UpdateItemWithPreCheck hook for the rationale.
if newStatus != oldStatus {
if _, err = tx.Exec(s.q(`
INSERT INTO status_transitions (id, item_id, workspace_id, collection_id, field_key, from_status, to_status, created_at, seq)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, `+nextTransitionSeqSubquery+`)
`), newID(), itemID, existing.WorkspaceID, targetCollectionID, moveDoneKey, oldStatus, newStatus, moveTS); err != nil {
return nil, fmt.Errorf("record status transition on move: %w", err)
}
}
// Durable cross-collection move record (BUG-1675), written in the
// SAME tx as the move so /items-changes' moved-out tombstone can't
// depend on the best-effort post-commit activity row. Skip same-
// collection no-ops (move callers reject those upstream, but guard
// anyway). The seq is the value the UPDATE just assigned — read it
// back under the still-held lock so the tombstone seq matches what
// /items-changes sees for the item.
if targetCollectionID != existing.CollectionID {
var moveSeq int64
if err = tx.QueryRow(s.q(`SELECT seq FROM items WHERE id = ?`), itemID).Scan(&moveSeq); err != nil {
return nil, fmt.Errorf("read post-move seq: %w", err)
}
if _, err = tx.Exec(s.q(`
INSERT INTO item_collection_moves (id, workspace_id, item_id, from_collection_id, to_collection_id, seq, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)
`), newID(), existing.WorkspaceID, itemID, existing.CollectionID, targetCollectionID, moveSeq, moveTS); err != nil {
return nil, fmt.Errorf("record collection move: %w", err)
}
}
// The choke point for the move path, applying the same disjoint-delta rule
// as the update path (SPEC-3 v1.3). A move is not automatically ONE event:
// item.moved owns the location slice, item.status_changed owns the status
// field — which this path can change, since it accepts a new fields blob —
// and item.updated owns whatever else moved. Emitting only item.moved
// would silently drop a status transition that a binding is watching for,
// which is precisely the gap the rule exists to close.
moved, err := s.getItemTx(tx, itemID)
if err != nil {
return nil, fmt.Errorf("read post-move snapshot: %w", err)
}
if moved != nil {
if targetCollectionID != preMove.CollectionID {
if err := s.emitItemEventTx(tx, kernelevents.ItemMoved, moved, nil, opt.batchID); err != nil {
return nil, err
}
}
if newStatus != oldStatus {
if err := s.emitItemEventTx(tx, kernelevents.ItemStatusChanged, moved, &oldStatus, opt.batchID); err != nil {
return nil, err
}
}
// itemUpdatedSliceChanged already excludes collection_id and the
// derived collection_*/ref keys, so a pure relocation does not leak
// into item.updated's slice.
otherChanged, cerr := itemUpdatedSliceChanged(preMove, moved, moveDoneKey)
if cerr != nil {
return nil, cerr
}
if otherChanged {
if err := s.emitItemEventTx(tx, kernelevents.ItemUpdated, moved, nil, opt.batchID); err != nil {
return nil, err
}
}
}
if err := tx.Commit(); err != nil {
return nil, err
}
result, err := s.GetItem(itemID)
if err != nil || result == nil {
return result, err
}
// Same race-free-delta rationale as updateItemWithParentLinkOnce
// (TASK-2533): oldStatus/newStatus were captured under the write lock
// above, so this is safe to attach post-commit.
if newStatus != oldStatus {
result.LastMutation = &models.ItemMutationSignal{
StatusChanged: true,
StatusFieldKey: moveDoneKey,
FromStatus: oldStatus,
ToStatus: newStatus,
}
}
return result, nil
}
// --- Helpers ---
// validSortField matches safe field names (alphanumeric + underscore, starting with a letter).
var validSortField = regexp.MustCompile(`^[a-zA-Z][a-zA-Z0-9_]*$`)
func buildItemSort(sort string, dialect Dialect) string {
if sort == "" {
return " ORDER BY i.pinned DESC, i.updated_at DESC"
}
var parts []string
for _, seg := range strings.Split(sort, ",") {
seg = strings.TrimSpace(seg)
tokens := strings.SplitN(seg, ":", 2)
col := tokens[0]
dir := "ASC"
if len(tokens) == 2 && strings.ToUpper(tokens[1]) == "DESC" {
dir = "DESC"
}
switch col {
case "title":
parts = append(parts, fmt.Sprintf("i.title %s", dir))
case "created_at":
parts = append(parts, fmt.Sprintf("i.created_at %s", dir))
case "updated_at":
parts = append(parts, fmt.Sprintf("i.updated_at %s", dir))
case "sort_order":
parts = append(parts, fmt.Sprintf("i.sort_order %s", dir))
default:
// For field-based sorting, use dialect JSON extract — validate the field name
// to prevent SQL injection via crafted sort parameters.
if !validSortField.MatchString(col) {
continue // skip invalid field names
}
parts = append(parts, fmt.Sprintf("%s %s", dialect.JSONExtractText("i.fields", col), dir))
}
}
if len(parts) == 0 {
return " ORDER BY i.pinned DESC, i.updated_at DESC"
}
return " ORDER BY " + strings.Join(parts, ", ")
}
// shouldCreateItemVersion mirrors ShouldCreateVersion but queries item_versions.
func (s *Store) shouldCreateItemVersion(itemID, actor, source string) (bool, error) {
var createdBy, src, createdAt string
err := s.db.QueryRow(s.q(`
SELECT created_by, source, created_at
FROM item_versions
WHERE item_id = ?
ORDER BY created_at DESC, version_seq DESC
LIMIT 1
`), itemID).Scan(&createdBy, &src, &createdAt)
if err == sql.ErrNoRows {
return true, nil // No versions yet
}
if err != nil {
return false, err
}
// Actor or source changed — always snapshot
if createdBy != actor || src != source {
return true, nil
}
// Throttle
lastTime := parseTime(createdAt)
return time.Since(lastTime) >= VersionThrottleInterval, nil
}
// ListItemVersionsResolved returns versions with full content (diffs resolved).
// Requires the current item content to reconstruct diff-based versions.
//
// Unbounded: every version is read and every reverse patch applied. Callers
// that only need the newest N should use ListItemVersionsResolvedPage, which
// bounds BOTH the read and the patch walk (BUG-2608). This form remains
// correct — and required — where an arbitrary version must be located, since
// the chain can only be walked from current content backwards.
func (s *Store) ListItemVersionsResolved(itemID, currentContent string) ([]models.Version, error) {
return s.ListItemVersionsResolvedPage(itemID, currentContent, 0)
}
// ListItemVersionsResolvedPage is ListItemVersionsResolved bounded to the
// newest `limit` versions (limit <= 0 means unbounded).
//
// The bound is cheap ONLY because it takes the newest N. Versions are stored
// as REVERSE patches, so reconstructing any version means starting from the
// item's current content and walking backwards through everything newer — a
// window at the newest end is exactly the prefix of that walk, while an older
// window would still require walking everything above it. That asymmetry is
// why this offers a limit and not an offset (BUG-2608).
func (s *Store) ListItemVersionsResolvedPage(itemID, currentContent string, limit int) ([]models.Version, error) {
versions, err := s.ListItemVersionsPage(itemID, limit)
if err != nil {
return nil, err
}
// Resolve diffs: walk from newest to oldest, applying reverse patches.
content := currentContent
for i := range versions {
if !versions[i].IsDiff {
content = versions[i].Content
continue
}
resolved, applyErr := diff.ApplyPatch(content, versions[i].Content)
if applyErr != nil {
versions[i].Content = fmt.Sprintf("[patch error: %v]", applyErr)
versions[i].IsDiff = false
continue
}
versions[i].Content = resolved
versions[i].IsDiff = false
content = resolved
}
return versions, nil
}
// GetItemVersionResolved returns a single version with its diff resolved to
// full content. Reverse-patch versions can only be reconstructed by walking the
// chain from current content newest→oldest, so this resolves the whole chain and
// returns the requested row. Used by the timeline's lazy "resolve on expand" path
// (BUG-1612) — the paginated timeline serves raw patch text, so the card fetches
// real content only when a diff version is expanded. Returns nil if not found.
func (s *Store) GetItemVersionResolved(itemID, versionID, currentContent string) (*models.Version, error) {
versions, err := s.ListItemVersionsResolved(itemID, currentContent)
if err != nil {
return nil, err
}
for i := range versions {
if versions[i].ID == versionID {
return &versions[i], nil
}
}
return nil, nil
}
// ListItemVersionsBeforeTime returns versions for an item created before the given time,
// ordered newest-first, limited to `limit` results. Used for cursor-based timeline pagination.
//
// When beforeID is empty (first page / no cursor), the secondary id tie-breaker
// is omitted. See ListCommentsBeforeTime for the rationale (BUG-1086).
func (s *Store) ListItemVersionsBeforeTime(itemID string, before time.Time, beforeID string, limit int) ([]models.Version, error) {
ts := before.Format(time.RFC3339)
const selectCols = `id, item_id, content, change_summary, created_by, source, is_diff, created_at`
// Deliberately keeps `id DESC` (NOT version_seq DESC). This is a keyset-
// paginated query and the cursor below filters on id (`created_at = ? AND
// id < ?`); the ORDER-BY key MUST match the cursor key or a same-second
// row at a page boundary can be skipped forever. This path also feeds the
// UUID-keyed MERGED timeline (handlers_timeline.go re-sorts merged events
// by id and derives the next before_id from them) and does NOT reconstruct
// diffs, so version_seq ordering would neither be honored nor needed here.
// BUG-2270's same-second determinism lives in the diff-RECONSTRUCTION
// paths instead (ListItemVersions / ListItemVersionsResolved /
// shouldCreateItemVersion), which are ordered by version_seq DESC.
const orderLimit = `ORDER BY created_at DESC, id DESC LIMIT ?`
var rows *sql.Rows
var err error
if beforeID == "" {
rows, err = s.db.Query(s.q(`
SELECT `+selectCols+`
FROM item_versions
WHERE item_id = ? AND created_at < ?
`+orderLimit), itemID, ts, limit)
} else {
rows, err = s.db.Query(s.q(`
SELECT `+selectCols+`
FROM item_versions
WHERE item_id = ? AND (created_at < ? OR (created_at = ? AND id < ?))
`+orderLimit), itemID, ts, ts, beforeID, limit)
}
if err != nil {
return nil, err
}
defer rows.Close()
var versions []models.Version
for rows.Next() {
var v models.Version
var createdAt string
var isDiff bool
if err := rows.Scan(&v.ID, &v.DocumentID, &v.Content, &v.ChangeSummary, &v.CreatedBy, &v.Source, &isDiff, &createdAt); err != nil {
return nil, err
}
v.IsDiff = isDiff
v.CreatedAt = parseTime(createdAt)
versions = append(versions, v)
}
return versions, rows.Err()
}
// ListItemVersions returns all versions for an item.
func (s *Store) ListItemVersions(itemID string) ([]models.Version, error) {
return s.ListItemVersionsPage(itemID, 0)
}
// ListItemVersionsPage returns an item's versions newest-first, bounded to
// `limit` rows (limit <= 0 means unbounded). Raw rows — reverse-patch versions
// still carry patch text, not content; see ListItemVersionsResolvedPage.
func (s *Store) ListItemVersionsPage(itemID string, limit int) ([]models.Version, error) {
query := `
SELECT id, item_id, content, change_summary, created_by, source, is_diff, created_at
FROM item_versions
WHERE item_id = ?
ORDER BY created_at DESC, version_seq DESC
`
args := []interface{}{itemID}
if limit > 0 {
query += " LIMIT ?"
args = append(args, limit)
}
rows, err := s.db.Query(s.q(query), args...)
if err != nil {
return nil, err
}
defer rows.Close()
var versions []models.Version
for rows.Next() {
var v models.Version
var createdAt string
var isDiff bool
if err := rows.Scan(&v.ID, &v.DocumentID, &v.Content, &v.ChangeSummary, &v.CreatedBy, &v.Source, &isDiff, &createdAt); err != nil {
return nil, err
}
v.IsDiff = isDiff
v.CreatedAt = parseTime(createdAt)
versions = append(versions, v)
}
return versions, rows.Err()
}
func scanItems(rows *sql.Rows) ([]models.Item, error) {
var items []models.Item
for rows.Next() {
var item models.Item
var createdAt, updatedAt string
var deletedAt *string
var pinned bool
if err := rows.Scan(
&item.ID, &item.WorkspaceID, &item.CollectionID, &item.Title, &item.Slug,
&item.Content, &item.Fields, &item.Tags,
&pinned, &item.SortOrder, &item.ParentID, &item.AssignedUserID, &item.AgentRoleID, &item.RoleSortOrder,
&item.CreatedBy, &item.LastModifiedBy, &item.Source,
&item.ItemNumber, &item.Seq, &createdAt, &updatedAt,
&item.CollectionSlug, &item.CollectionName, &item.CollectionIcon, &item.CollectionPrefix,
&item.AssignedUserName, &item.AssignedUserEmail,
&item.AgentRoleName, &item.AgentRoleSlug, &item.AgentRoleIcon,
&deletedAt,
); err != nil {
return nil, err
}
item.Pinned = pinned
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
item.DeletedAt = parseTimePtr(deletedAt)
hydrateItemComputedMetadata(&item)
items = append(items, item)
}
return items, rows.Err()
}
// ItemsModifiedSince returns items in a workspace that were updated after the
// given timestamp. Used for incremental sync on tab resume. Also returns IDs of
// items that were deleted (hard-deleted or archived) since the timestamp.
//
// The updated list includes both active AND recently archived items (those with
// deleted_at at/after since). This lets the frontend update archived views
// correctly — an item that was just archived needs its full data to appear in
// archived views, not just its ID in the deleted list.
//
// The cursor comparison is INCLUSIVE of its own second, deliberately (BUG-2539).
// items.updated_at / items.deleted_at are written at RFC3339 whole-second
// precision (store.now()), while the caller's cursor is a unix-MILLISECOND value
// — normally the previous response's server_time. Formatting that cursor for
// comparison truncates it DOWN to the second, so with a strict `>` every change
// that landed in the cursor's own second compares equal and is dropped. It is
// dropped PERMANENTLY, because the caller then advances its cursor past that
// second and no later query reaches back for it: a bulk archive ~450ms after a
// page seeded its cursor left the item rendering as live indefinitely.
//
// `>=` against the truncated second makes the endpoint AT-LEAST-ONCE at the
// second boundary: rows written inside the cursor's second come back again, and
// come back on every sync whose cursor stays inside that second, so a burst of
// rapid syncs can repeat them more than once. That is the cheap direction to be
// wrong in, because the payload is server STATE rather than a delta to apply on
// top of what the caller has — every in-tree consumer overwrites its row from
// it, so a repeat is a no-op rather than a double-apply. A future consumer that
// treats these rows as increments would not be safe here. Sub-second storage would be
// the other fix, but items timestamps are second-precision throughout and
// mixing formats in one column breaks the lexicographic ordering these string
// comparisons rely on ("…20.451Z" sorts BEFORE "…20Z"), so precision is a
// migration, not a one-line change.
func (s *Store) ItemsModifiedSince(workspaceID string, since time.Time) (updated []models.Item, deletedIDs []string, err error) {
sinceStr := since.UTC().Truncate(time.Second).Format(time.RFC3339)
// Fetch updated items: active items modified since the timestamp,
// PLUS items archived since the timestamp (so archived views can update).
query := s.q(`
SELECT i.id, i.workspace_id, i.collection_id, i.title, i.slug, i.content, i.fields, i.tags,
i.pinned, i.sort_order, i.parent_id, i.assigned_user_id, i.agent_role_id, i.role_sort_order,
i.created_by, i.last_modified_by, i.source,
i.item_number, i.seq, i.created_at, i.updated_at,
c.slug, c.name, c.icon, c.prefix,
COALESCE(au.name, ''), COALESCE(au.email, ''),
COALESCE(ar.name, ''), COALESCE(ar.slug, ''), COALESCE(ar.icon, ''), i.deleted_at
FROM items i
JOIN collections c ON c.id = i.collection_id
LEFT JOIN users au ON au.id = i.assigned_user_id
LEFT JOIN agent_roles ar ON ar.id = i.agent_role_id
WHERE i.workspace_id = ?
AND i.updated_at >= ?
AND (i.deleted_at IS NULL OR i.deleted_at >= ?)
ORDER BY i.updated_at ASC
`)
rows, err := s.db.Query(query, workspaceID, sinceStr, sinceStr)
if err != nil {
return nil, nil, err
}
defer rows.Close()
updated, err = scanItems(rows)
if err != nil {
return nil, nil, err
}
// Fetch IDs of items deleted since the timestamp.
delQuery := s.q(`
SELECT id FROM items
WHERE workspace_id = ?
AND deleted_at IS NOT NULL
AND deleted_at >= ?
`)
delRows, err := s.db.Query(delQuery, workspaceID, sinceStr)
if err != nil {
return updated, nil, err
}
defer delRows.Close()
for delRows.Next() {
var id string
if err := delRows.Scan(&id); err != nil {
return updated, nil, err
}
deletedIDs = append(deletedIDs, id)
}
return updated, deletedIDs, delRows.Err()
}
// ItemCollectionRef is a minimal item reference with collection ID, used for
// filtering deleted items by collection visibility.
type ItemCollectionRef struct {
ID string
CollectionID string
}
// GetDeletedItemsWithCollection returns minimal item info (ID + CollectionID)
// for soft-deleted items, used to filter deleted item IDs by collection visibility.
//
// Despite the name, this is just GetItemCollectionRefs under the hood — the
// underlying query has no deleted_at filter, so it works for any item state.
// The name is kept for this call site's existing meaning (its caller only
// ever passes already-known-deleted IDs); BUG-1928 needed the same
// state-agnostic lookup for live-or-deleted item grants, hence the rename
// of the shared implementation to the more accurate GetItemCollectionRefs.
func (s *Store) GetDeletedItemsWithCollection(workspaceID string, itemIDs []string) ([]ItemCollectionRef, error) {
return s.GetItemCollectionRefs(workspaceID, itemIDs)
}
// GetItemCollectionRefs returns minimal item info (ID + CollectionID) for the
// given item IDs, scoped to the workspace. State-agnostic: the query has no
// deleted_at filter, so it resolves live and soft-deleted items alike. Used
// wherever a caller needs an item_id → collection_id mapping without paying
// for a full item fetch (e.g. bulk visibility filtering).
func (s *Store) GetItemCollectionRefs(workspaceID string, itemIDs []string) ([]ItemCollectionRef, error) {
if len(itemIDs) == 0 {
return nil, nil
}
placeholders := make([]string, len(itemIDs))
args := []interface{}{workspaceID}
for i, id := range itemIDs {
placeholders[i] = "?"
args = append(args, id)
}
rows, err := s.db.Query(s.q(fmt.Sprintf(`
SELECT id, collection_id FROM items
WHERE workspace_id = ? AND id IN (%s)
`, strings.Join(placeholders, ","))), args...)
if err != nil {
return nil, fmt.Errorf("get item collection refs: %w", err)
}
defer rows.Close()
var results []ItemCollectionRef
for rows.Next() {
var r ItemCollectionRef
if err := rows.Scan(&r.ID, &r.CollectionID); err != nil {
return nil, fmt.Errorf("scan item collection ref: %w", err)
}
results = append(results, r)
}
return results, rows.Err()
}
// WorkspaceHasAgentActivity reports whether any non-deleted item VISIBLE
// to the caller was created via an agent surface — direct CLI invocation
// or the Remote MCP transport. Used by the dashboard to auto-hide the
// "connect an agent" banner once a workspace's agent loop is wired up.
//
// "Agent activity" is the union of two source values:
//
// - source='cli': set by both the direct `pad` CLI and the
// HTTPHandlerDispatcher used by the Remote MCP transport, which
// deliberately mirrors CLI attribution (see dispatch_http_test.go).
// Today, this single value covers both surfaces.
// - source='mcp': reserved for future code paths that may want to
// distinguish MCP attribution from CLI. Included here defensively so
// the query keeps working if attribution is later split.
//
// Visibility filtering matches the dashboard's existing model (see
// handleGetDashboard): an item counts when its collection is in
// collectionIDs OR its id is in itemIDs (union — guest item-level grants
// can expose items in otherwise-hidden collections). Pass nil for both to
// run unfiltered (full-visibility caller). A non-nil empty
// collectionIDs slice with no itemIDs means "no visible collections" and
// returns false without hitting the DB — symmetric with ListItems.
//
// Backed by EXISTS so it short-circuits on the first match.
func (s *Store) WorkspaceHasAgentActivity(workspaceID string, collectionIDs, itemIDs []string) (bool, error) {
// Symmetric early-exit with ListItems: caller signaled no visibility.
if collectionIDs != nil && len(collectionIDs) == 0 && len(itemIDs) == 0 {
return false, nil
}
query := `
SELECT EXISTS(
SELECT 1 FROM items
WHERE workspace_id = ? AND source IN ('cli', 'mcp') AND deleted_at IS NULL
`
args := []interface{}{workspaceID}
if len(collectionIDs) > 0 && len(itemIDs) > 0 {
collPlaceholders := make([]string, len(collectionIDs))
for i, id := range collectionIDs {
collPlaceholders[i] = "?"
args = append(args, id)
}
itemPlaceholders := make([]string, len(itemIDs))
for i, id := range itemIDs {
itemPlaceholders[i] = "?"
args = append(args, id)
}
query += " AND (collection_id IN (" + strings.Join(collPlaceholders, ",") + ") OR id IN (" + strings.Join(itemPlaceholders, ",") + "))"
} else if len(collectionIDs) > 0 {
placeholders := make([]string, len(collectionIDs))
for i, id := range collectionIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND collection_id IN (" + strings.Join(placeholders, ",") + ")"
} else if len(itemIDs) > 0 {
placeholders := make([]string, len(itemIDs))
for i, id := range itemIDs {
placeholders[i] = "?"
args = append(args, id)
}
query += " AND id IN (" + strings.Join(placeholders, ",") + ")"
}
query += ")"
var has bool
if err := s.db.QueryRow(s.q(query), args...).Scan(&has); err != nil {
return false, fmt.Errorf("workspace has agent activity: %w", err)
}
return has, nil
}
// WorkspaceHasUserCreatedItems reports whether ANY non-deleted item
// in the workspace was created by something other than template
// seeding. Used by the agent bootstrap to compute the
// `needs_onboarding` flag — true when zero user-created items exist,
// false the moment the user (or an agent on their behalf) creates
// the first real item. PLAN-1496 / TASK-1504.
//
// "User-created" is the inverse of "template seed": items written
// during workspace init via SeedCollectionsFromTemplate carry
// source="template" + created_by="system". Everything else — CLI,
// web UI, MCP, future surfaces — is treated as user activity. The
// query filters on `source != 'template'` rather than enumerating
// the user-side values so new surfaces (mcp, api, etc.) are
// included automatically without code changes here.
//
// Visibility filtering is intentionally omitted: needs_onboarding
// is a workspace-level state signal, not a per-user view. Two
// callers reading bootstrap concurrently should see the same answer
// regardless of their individual collection-access grants — the
// flag describes whether the WORKSPACE has been activated, not
// whether the calling user has done so. Server already gates the
// bootstrap endpoint on workspace membership, so unauthorized
// callers never reach this code path.
//
// Backed by EXISTS so it short-circuits on the first match.
func (s *Store) WorkspaceHasUserCreatedItems(workspaceID string) (bool, error) {
const query = `
SELECT EXISTS(
SELECT 1 FROM items
WHERE workspace_id = ?
AND deleted_at IS NULL
AND (source IS NULL OR source != 'template')
)
`
var has bool
if err := s.db.QueryRow(s.q(query), workspaceID).Scan(&has); err != nil {
return false, fmt.Errorf("workspace has user-created items: %w", err)
}
return has, nil
}
func hydrateItemComputedMetadata(item *models.Item) {
if item == nil {
return
}
item.ComputeRef()
item.CodeContext = models.ExtractItemCodeContext(item.Fields)
item.Convention = models.ExtractItemConventionMetadata(item.Fields)
item.ImplementationNotes = models.ExtractItemImplementationNotes(item.Fields)
item.DecisionLog = models.ExtractItemDecisionLog(item.Fields)
}