Files
xarmian 402f79e016 feat(store,server,web): collection kernel traits — de-hardcode conventions/playbooks slugs (TASK-2657, BUG-2702) (#1171)
Implements SPEC-5 §Collection traits (approved v1.1) — the first unit of
PLAN-2656 phase 0. Three kernel behaviors were keyed on the literal collection
slugs "conventions" and "playbooks": what the agent bootstrap loads, which
items route by invocation slug, and which items export as portable artifacts.
Collections now DECLARE those behaviors and the kernel resolves them from the
declarations.

Fixes the KERNEL half of BUG-2702, which stays open for the rest (see below).
A slug is not a stable identifier —
UpdateCollection re-slugs on any name change, and renaming a collection is a
documented onboarding step (TASK-1510) — so renaming either collection silently
detached all three behaviors from it, with the items still present and no error
anywhere. Measured on origin/main before the fix: conventions and
convention_index dropped 1 -> 0, playbooks 1 -> 0, and GET /playbooks/{slug}
went 200 -> 404, so `/pad ship` stopped resolving with no sign the playbook
still existed. Both halves are locked by regression tests observed failing on
unfixed code.

BUG-2702 is NOT fully closed here, deliberately. Every kernel behavior follows
the trait, and library activation on the MCP dispatcher and CLI was converted
too — but the pack's own dedicated web routes (/conventions, /playbooks list and
detail, /library) still address their collection by literal slug and render
empty after a rename. Filed as BUG-2705 with the route paths and the likely fix
shape; 2702 closes when that lands. Degradation there is bounded: no data loss,
and the collection stays usable at its own /[collection] route and in the
sidebar.

SPEC-5 was amended to v1.1 BEFORE any code, per the spec tree's own discipline:
bootstrap_include becomes a LIST of {mode, filter, key} because v1.0 could not
express convention_index at all; the conventions filter is now normative and
includes status=active, which v1.0's shorthand omitted and which the
implementation does enforce (implementing v1.0 literally would have leaked
draft conventions into every agent's boot payload); v1 filters are field-
equality maps with query/1 named as the widening path, since SPEC-2 is phase 1
and PLAN-2656 forbids growing toward it; and invocation_field is constrained to
the literal `invocation_slug`, because any other field name falls outside the
partial unique indexes in migrations/054 and pgmigrations/033 that are the real
uniqueness guard.

Traits get their own column rather than a key inside the schema JSON. The
schema column is overwritten wholesale on update and every client rebuilds it
fields-only, so a traits key stored there is destroyed by one ordinary
collection edit — measured during this task, not assumed. Trait authority
cannot rest on a value an unrelated UI save deletes. UpdateCollection writes
traits only when explicitly supplied, so pre-existing clients leave them alone;
an explicit "{}" still clears.

Bootstrap keeps its three payload keys as first-party views fed from the
declarations, and gains a generic bootstrap_includes array for any other
declared key — so the boot surface is genuinely generic rather than three
hardcoded payloads, and no consumer breaks.

Existing workspaces are backfilled slug-keyed in both dialects, guarded on
traits='{}' so a re-run cannot clobber a workspace's own declarations. The
backfill inherits today's blind spot (a workspace that renamed the collection
before upgrading is not reached) but cannot do worse than the status quo, which
is itself slug-keyed; from the backfill forward the hazard is structurally gone.

Malformed declarations are refused at create and update rather than stored:
an unparseable blob degrades to "declares nothing", which is silently the wrong
behavior instead of a loud error (SPEC-0 L6).

Web groups agent-facing collections by bootstrap_include presence, replacing a
hardcoded two-slug array repeated at five call sites.

Not done, deliberately: no MCP catalog change (traits are first-party kernel
declarations, no agent needs to set them, and the separate column means
pad_collection.update passes through harmlessly — no ToolSurfaceVersion bump);
bootstrap's collections[] projection does not carry traits (PLAN-1410 trimmed
that payload and nothing consumes them there); prefix.go's NormalizeSlug is
untouched (a pure function with no workspace context, and de-hardcoding two of
its six slugs would make it less coherent, not more).

Eight Codex review rounds found nineteen real defects, all fixed here. The
serious one:
bootstrap_include filter keys FAIL OPEN. The item store's field-filter path
drops any key its sanitizer rejects, removing the predicate rather than matching
nothing, so a declaration filtering on `"stat us"` would narrow nothing and ship
every convention — drafts included — to every agent at boot, defeating the
status=active guarantee this change makes normative. Filter keys are now
validated against the store's own sanitizer shape and pinned by a cross-package
agreement test, since models cannot import store and a future divergence would
silently reopen it. SPEC-5 amended to v1.2 with the rule and its fail-open /
fail-closed asymmetry. Also fixed: an unknown declared artifact_kind reached
artifact.Encode and surfaced as a 500 (now a 400 at the export boundary, since
SPEC-5 permits unknown kinds as legal non-round-tripping declarations); and
workspace import validated traits as JSON only, so an archive could persist a
declaration that degrades to "declares nothing" (now validated, degrading to
"{}" with a warning rather than refusing an import that may be the only copy).

Later rounds found more, and several were defects this change itself created.
A hidden collection could SHADOW a visible one: resolution used to name exactly
one collection, so with several declaring, resolving across all of them and
rejecting afterwards on visibility made a visible playbook unreachable behind a
hidden one — candidates are now filtered by visibility before selection, in both
playbook resolution and artifact import. Importing a pre-traits archive produced
an INERT workspace: the migration backfill cannot reach rows inserted long after
it ran, so conventions/playbooks arrived declaring nothing, and canonical
declarations are now inferred from the slug when a collection declares none
(never overriding declarations that survived the round trip). The generic
include path had no L4 boot budget and is now capped with an overflow count.
Trait parsing claimed to be strict but json.Decoder ignores trailing bytes, so
`{...} garbage` parsed cleanly. First-party payload keys are now mode-pinned,
since their projections have fixed shapes and declaring the other mode would be
silently ignored. Duplicate artifact_kind / invocation_field declarations are
refused at the collection API, and a conflicting archive warns on import.

Agent-facing text was updated with the rest, not after it: SKILL.md,
instructions.md and the MCP catalog said the literal slugs, which is exactly the
artifact an agent acts on. ToolSurfaceVersion 0.24 -> 0.25 for the
pad_library.activate behaviour change.

Trait uniqueness is a documented BEST-EFFORT gate, not an invariant, by lead
ruling. The gate reads then writes without a lock, import bypasses it, and a
rename can mint a duplicate without touching that path. The database-level
enforcement (partial unique indexes on the extracted traits) cannot ship first:
existing deployments can already hold duplicates via rename-then-reseed, so the
index would fail the migration on precisely the databases that most need
repairing. TASK-2710 carries the de-duplication pass and the indexes; SPEC-5
v1.3 records the deferral and the reason. L6's requirement that conflicts fail
loud is met by the refusal plus the warning — the mechanism is deferred, the
principle is not.

Gates: build · make lint 0 issues · go test ./internal/... · make test-pg ·
svelte-check 0 errors · vitest 99 files / 1734 tests. Mutation-verified across
four matrices, 20 mutations, 19 caught; the survivor is a seeding path whose
trait-vs-slug difference is unreachable today (SeedCollectionsFromTemplate
creates any missing template collection before it seeds items), recorded on the
task trail rather than papered over with a test that proves nothing.

Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
2026-08-20 13:26:05 -04:00

200 lines
8.4 KiB
Go

package models
import (
"encoding/json"
"errors"
"time"
)
type FieldDef struct {
Key string `json:"key"`
Label string `json:"label"`
Type string `json:"type"` // text, number, select, multi_select, date, checkbox, url, relation, json
Options []string `json:"options,omitempty"`
TerminalOptions []string `json:"terminal_options,omitempty"` // for select fields: which options represent a terminal/finalized state
Default any `json:"default,omitempty"`
Required bool `json:"required,omitempty"`
Computed bool `json:"computed,omitempty"`
Collection string `json:"collection,omitempty"` // for relation type
Suffix string `json:"suffix,omitempty"` // for number type display
Pattern string `json:"pattern,omitempty"` // optional ECMAScript-style regex applied to text values; empty = no pattern check
UniqueScope string `json:"unique_scope,omitempty"` // "workspace_collection" enforces uniqueness within a collection (non-empty values only); empty = no uniqueness
}
type CollectionSchema struct {
Fields []FieldDef `json:"fields"`
}
// QuickAction defines a prompt template that can be triggered from the UI.
type QuickAction struct {
Label string `json:"label"` // display label for the button
Prompt string `json:"prompt"` // prompt template with {ref}, {title}, {status}, etc.
Scope string `json:"scope"` // "item" or "collection"
Icon string `json:"icon,omitempty"` // optional emoji/icon
}
type CollectionSettings struct {
Layout string `json:"layout,omitempty"` // fields-primary, content-primary, balanced
DefaultView string `json:"default_view,omitempty"` // list, board, table
BoardGroupBy string `json:"board_group_by,omitempty"`
ListSortBy string `json:"list_sort_by,omitempty"`
ListGroupBy string `json:"list_group_by,omitempty"`
QuickActions []QuickAction `json:"quick_actions,omitempty"`
ContentTemplate string `json:"content_template,omitempty"` // markdown template for new items
}
type Collection struct {
ID string `json:"id"`
WorkspaceID string `json:"workspace_id"`
Name string `json:"name"`
Slug string `json:"slug"`
Icon string `json:"icon"`
Description string `json:"description"`
Schema string `json:"schema"` // JSON string in DB, parsed via methods
Settings string `json:"settings"` // JSON string in DB
// Traits is the collection's kernel-trait declaration set (SPEC-5), a
// JSON string in the DB parsed via ParseCollectionTraits. Its own column
// rather than a key inside Schema on purpose: Schema is overwritten
// wholesale on update and every client rebuilds it fields-only, so a
// traits key stored there is destroyed by any ordinary collection edit.
// See collection_traits.go for the full rationale. TASK-2657.
Traits string `json:"traits"`
Prefix string `json:"prefix"`
SortOrder int `json:"sort_order"`
IsDefault bool `json:"is_default"`
IsSystem bool `json:"is_system"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DeletedAt *time.Time `json:"deleted_at,omitempty"`
// Computed (not stored)
ItemCount int `json:"item_count"`
ActiveItemCount int `json:"active_item_count"`
}
type CollectionCreate struct {
Name string `json:"name"`
Slug string `json:"slug,omitempty"`
Prefix string `json:"prefix,omitempty"`
Icon string `json:"icon,omitempty"`
Description string `json:"description,omitempty"`
Schema string `json:"schema,omitempty"`
Settings string `json:"settings,omitempty"`
Traits string `json:"traits,omitempty"`
IsDefault bool `json:"is_default,omitempty"`
IsSystem bool `json:"is_system,omitempty"`
}
// FieldMigration describes a bulk update to apply to existing items when
// a collection schema changes (e.g. renaming select options).
type FieldMigration struct {
Field string `json:"field"` // field key to migrate
RenameOptions map[string]string `json:"rename_options,omitempty"` // old_value → new_value
}
type CollectionUpdate struct {
Name *string `json:"name,omitempty"`
Prefix *string `json:"prefix,omitempty"`
Icon *string `json:"icon,omitempty"`
Description *string `json:"description,omitempty"`
Schema *string `json:"schema,omitempty"`
Settings *string `json:"settings,omitempty"`
// Traits, when non-nil, replaces the collection's kernel-trait
// declarations. Nil leaves them untouched — which is the case for every
// client that rebuilds a collection's schema/settings blobs without
// knowing traits exist, and is why traits survive an ordinary edit.
Traits *string `json:"traits,omitempty"`
SortOrder *int `json:"sort_order,omitempty"`
Migrations []FieldMigration `json:"migrations,omitempty"`
// ExpectedUpdatedAt, when non-empty, opts into optimistic-concurrency
// control (BUG-2265, mirroring the item pattern from IDEA-1480): the
// store re-reads the row's updated_at under the workspace write lock and
// rejects the write with a conflict error when it no longer matches the
// RFC3339 timestamp the caller last read. Empty = last-write-wins (the
// legacy behaviour, unchanged for CLI / MCP / API callers that don't send
// it). Never persisted; consumed by UpdateCollection only.
ExpectedUpdatedAt string `json:"expected_updated_at,omitempty"`
}
// ErrInvalidSettingsType is returned by CollectionUpdate.UnmarshalJSON
// and CollectionCreate.UnmarshalJSON when the inbound `settings` value is
// neither a JSON object nor a JSON-encoded string. IDEA-1488: handler-
// layer shape validation ceiling, paired with the IDEA-1484 NOT NULL
// floor on collections.settings. Mirrors item.go's
// ErrInvalidFieldsType / ErrInvalidTagsType and view.go's
// ErrInvalidConfigType.
var ErrInvalidSettingsType = errors.New(`"settings" must be a JSON object or a JSON-encoded string`)
// UnmarshalJSON for CollectionCreate accepts `settings` either as the
// canonical JSON-encoded string shape (matches models.Collection.Settings
// storage) OR as the natural nested object shape any reasonable HTTP
// client would send. Mirrors ItemCreate.UnmarshalJSON; see
// flexJSONToString in item.go for the shape contract.
func (c *CollectionCreate) UnmarshalJSON(data []byte) error {
type alias CollectionCreate
aux := struct {
Settings json.RawMessage `json:"settings,omitempty"`
Traits json.RawMessage `json:"traits,omitempty"`
*alias
}{alias: (*alias)(c)}
if err := json.Unmarshal(data, &aux); err != nil {
return err
}
if settingsStr, err := flexJSONToString(aux.Settings, '{', ErrInvalidSettingsType); err != nil {
return err
} else if settingsStr != nil {
c.Settings = *settingsStr
}
// `traits` gets the same both-shapes treatment as `settings`: the field
// is a JSON string in storage, but any reasonable client sends a nested
// object. TASK-2657.
if traitsStr, err := flexJSONToString(aux.Traits, '{', ErrInvalidTraitsType); err != nil {
return err
} else if traitsStr != nil {
c.Traits = *traitsStr
}
return nil
}
// UnmarshalJSON for CollectionUpdate accepts `settings` either as the
// canonical JSON-encoded string shape OR as the natural nested object
// shape. Mirrors ItemUpdate.UnmarshalJSON; the struct field stays
// `*string` and downstream consumers are unchanged.
//
// IDEA-1488: closes the shape-validation gap that IDEA-1484's NOT NULL
// floor doesn't cover. After this lands, the writer-side store coercion
// at collections.go:248 only sees empty-string-or-valid-object input.
func (u *CollectionUpdate) UnmarshalJSON(data []byte) error {
type alias CollectionUpdate
aux := struct {
Settings json.RawMessage `json:"settings,omitempty"`
Traits json.RawMessage `json:"traits,omitempty"`
*alias
}{alias: (*alias)(u)}
if err := json.Unmarshal(data, &aux); err != nil {
return err
}
settingsStr, err := flexJSONToString(aux.Settings, '{', ErrInvalidSettingsType)
if err != nil {
return err
}
u.Settings = settingsStr
// Absent `traits` stays nil, which UpdateCollection reads as "leave the
// declarations alone" — the case for every client that predates traits.
// TASK-2657.
traitsStr, err := flexJSONToString(aux.Traits, '{', ErrInvalidTraitsType)
if err != nil {
return err
}
u.Traits = traitsStr
return nil
}