Files
pad/internal/models/collection.go
T
xarmian 9d563aaf09 fix(store): collection settings must be a JSON object at every write door (BUG-3985) (#2260)
* fix(store): the collection store doors refuse non-object settings (BUG-3985)

The HTTP decoder already refuses collection settings that are not a JSON
object (IDEA-1488). CreateCollection (via createCollectionTx, which app
provisioning also uses) and UpdateCollection did not, so a Go caller that
never passes through the decoder could store [], "x", 7, null or non-JSON
text. Both now refuse such a value with models.ErrSettingsNotObject
(introduced by BUG-3976) before writing; "" stays the empty sentinel
written as "{}".

Workspace import is unchanged: it already repairs these values to "{}"
(coerceJSONForImport), the IDEA-1488 log-and-coerce posture for restoring
data. No current internal caller writes a non-object value.

Claude-Session: https://claude.ai/code/session_01EZ6hZzQiytCeBxXrtCWcVd

* fix(server): whitespace-only collection settings are refused over HTTP too (BUG-3985)

flexJSONToString passes a whitespace-only settings string through as the
"" sentinel, so it reached the store. Before the store doors refused it, a
PATCH with {"settings":" "} replaced the stored settings with " " (invalid
JSON on SQLite); a create repaired it to an object. Both now answer 400
validation_error with ErrSettingsNotObject and write nothing; the new HTTP
test pins both doors.

Comments that called the create arm unreachable over HTTP, or named
workspace import as a source of non-object stored settings (import repairs
them to {}), are corrected.

Claude-Session: https://claude.ai/code/session_01EZ6hZzQiytCeBxXrtCWcVd

* fix(models): whitespace-only collection settings decode as the "" sentinel (BUG-3985)

The previous commit let the store refuse whitespace-only settings, which
turned a create that used to answer 201 (repaired to an object) into a 400.
Instead, CollectionCreate and CollectionUpdate now read a whitespace-only
settings string as "", so create keeps its repair and a PATCH behaves
exactly as "" does: the settings become {} with tracks_work and
awaits_human carried. The store refusal stays as defense in depth.

Settings only. The other fields decoded by flexJSONToString (traits, item
fields and tags, view config) are BUG-3992.

The HTTP tests pin "" and two whitespace shapes on both doors, and the
create handler's comment is restored to "unreachable over HTTP" with the
reason updated.

Claude-Session: https://claude.ai/code/session_01EZ6hZzQiytCeBxXrtCWcVd
2026-10-20 18:11:13 -04:00

359 lines
16 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
package models
import (
"encoding/json"
"errors"
"strings"
"time"
)
type FieldDef struct {
Key string `json:"key"`
Label string `json:"label"`
Type string `json:"type"` // text, number, select, multi_select, date, checkbox, url, relation, multi_relation, json
Options []string `json:"options,omitempty"`
TerminalOptions []string `json:"terminal_options,omitempty"` // for select fields: which options represent a terminal/finalized state
// AbandonedOptions (BUG-2347) is the subset of TerminalOptions that
// closes an item WITHOUT delivering it (cancelled, rejected, overturned,
// withdrawn…). Completed-work views (changelog, standup, report
// throughput) count terminal − abandoned. When a field declares none,
// the global NegativeTerminals names are the fallback, so a collection
// that never sets it behaves as before. Validated as a subset of
// TerminalOptions on collection create/update.
AbandonedOptions []string `json:"abandoned_options,omitempty"`
Default any `json:"default,omitempty"`
Required bool `json:"required,omitempty"`
Computed bool `json:"computed,omitempty"`
Collection string `json:"collection,omitempty"` // for relation type
Suffix string `json:"suffix,omitempty"` // for number type display
Pattern string `json:"pattern,omitempty"` // optional ECMAScript-style regex applied to text values; empty = no pattern check
UniqueScope string `json:"unique_scope,omitempty"` // "workspace_collection" enforces uniqueness within a collection (non-empty values only); empty = no uniqueness
}
// RelationFieldTypes is every field type whose VALUE names other items
// (PLAN-2857 U4).
//
// It exists to be enumerated rather than grepped. Before U4 the only such type
// was the literal "relation", written out at ~20 sites across 8 Go files with
// no constant behind it, so "which code decides relation behaviour?" was a
// question only a string search could answer — and the answer was wrong the
// moment a second relation type existed. A third one joins here and the field-
// level predicates below come along with it.
//
// NOT every one of those sites is a field-level predicate: the ones that walk
// VALUES (migration, default resolution) have to decide array handling for
// themselves, and widening them with IsRelation alone would silently treat an
// array as a scalar. Those sites name their types explicitly on purpose.
func RelationFieldTypes() []string {
return []string{"relation", "multi_relation"}
}
// IsRelation reports whether this field's value names other items — a single
// one for "relation", an ordered list for "multi_relation".
//
// Use it for FIELD-level questions: does this field have a target collection,
// should it be hydrated, does a rename touch it. For VALUE-level work, ask
// IsMultiRelation too, because the shapes differ.
func (fd FieldDef) IsRelation() bool {
return fd.Type == "relation" || fd.Type == "multi_relation"
}
// IsMultiRelation reports whether this field's value is an ORDERED LIST of
// references rather than a single one.
func (fd FieldDef) IsMultiRelation() bool {
return fd.Type == "multi_relation"
}
type CollectionSchema struct {
Fields []FieldDef `json:"fields"`
}
// QuickAction defines a prompt template that can be triggered from the UI.
type QuickAction struct {
Label string `json:"label"` // display label for the button
Prompt string `json:"prompt"` // prompt template with {ref}, {title}, {status}, etc.
Scope string `json:"scope"` // "item" or "collection"
Icon string `json:"icon,omitempty"` // optional emoji/icon
}
type CollectionSettings struct {
Layout string `json:"layout,omitempty"` // fields-primary, content-primary, balanced
DefaultView string `json:"default_view,omitempty"` // list, board, table
BoardGroupBy string `json:"board_group_by,omitempty"`
ListSortBy string `json:"list_sort_by,omitempty"`
ListGroupBy string `json:"list_group_by,omitempty"`
QuickActions []QuickAction `json:"quick_actions,omitempty"`
ContentTemplate string `json:"content_template,omitempty"` // markdown template for new items
// TracksWork: false makes the collection REFERENCE material (PLAN-3535):
// its items leave progress, close blocking, Insights and open-work
// counts. nil (absent) means work. Read it through CollectionTracksWork.
TracksWork *bool `json:"tracks_work,omitempty"`
// settings.awaits_human (IDEA-3906) is deliberately NOT a field here:
// read it through CollectionAwaitsHumanJSON, which reads the one key, so
// a stored non-boolean (an import is not validated) cannot fail this
// struct's Unmarshal and turn every other setting into its default.
}
type Collection struct {
ID string `json:"id"`
WorkspaceID string `json:"workspace_id"`
Name string `json:"name"`
Slug string `json:"slug"`
Icon string `json:"icon"`
Description string `json:"description"`
Schema string `json:"schema"` // JSON string in DB, parsed via methods
Settings string `json:"settings"` // JSON string in DB
// Traits is the collection's kernel-trait declaration set (SPEC-5), a
// JSON string in the DB parsed via ParseCollectionTraits. Its own column
// rather than a key inside Schema on purpose: Schema is overwritten
// wholesale on update and every client rebuilds it fields-only, so a
// traits key stored there is destroyed by any ordinary collection edit.
// See collection_traits.go for the full rationale. TASK-2657.
Traits string `json:"traits"`
Prefix string `json:"prefix"`
SortOrder int `json:"sort_order"`
IsDefault bool `json:"is_default"`
IsSystem bool `json:"is_system"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DeletedAt *time.Time `json:"deleted_at,omitempty"`
// Computed (not stored)
ItemCount int `json:"item_count"`
ActiveItemCount int `json:"active_item_count"`
// Warnings is set only on a create/update RESPONSE, never stored or read
// back (BUG-2896). Additive and omitempty, so a clean write is
// byte-identical to before.
Warnings *CollectionWriteWarnings `json:"warnings,omitempty"`
}
// CollectionWriteWarnings names what a collection write normalised.
type CollectionWriteWarnings struct {
// CollapsedDuplicateKeys names each JSON member that the request's
// `traits` repeated (at any depth). The traits were stored as the parsed
// declaration's own encoding, so SQLite's unique index (json_extract, the
// FIRST occurrence) and the resolver (Go, the LAST) read one declaration.
CollapsedDuplicateKeys []string `json:"collapsed_duplicate_keys,omitempty"`
// Orphaned (TASK-2188) names each field or select option an UPDATE's
// schema removed that live items still hold, with how many. The values
// stay stored: a removed field's value is kept as an undeclared key and
// comes back if a field with that key is declared again, and a removed
// option's value stays on its items. Nothing was refused.
Orphaned []OrphanedValue `json:"orphaned,omitempty"`
}
// OrphanedValue is one removal a schema edit made that left values behind:
// a field (Option empty) or one select option, and how many live items still
// hold it.
type OrphanedValue struct {
Field string `json:"field"`
Option string `json:"option,omitempty"`
Items int `json:"items"`
}
// CanonicalTraitsIfDuplicated returns raw unchanged unless some object in it
// repeats a member name, in which case it returns the parsed declaration's own
// encoding and the repeated member (BUG-2896). raw must already have passed
// ParseCollectionTraits. Rewriting only on a repeat keeps every other blob's
// bytes as the caller sent them.
func CanonicalTraitsIfDuplicated(raw string) (out, dupKey string, err error) {
key, dup := FirstDuplicateJSONKey([]byte(raw))
if !dup {
return raw, "", nil
}
parsed, err := ParseCollectionTraits(raw)
if err != nil {
return raw, "", err
}
canon, err := parsed.JSON()
if err != nil {
return raw, "", err
}
return canon, key, nil
}
type CollectionCreate struct {
Name string `json:"name"`
Slug string `json:"slug,omitempty"`
Prefix string `json:"prefix,omitempty"`
Icon string `json:"icon,omitempty"`
Description string `json:"description,omitempty"`
Schema string `json:"schema,omitempty"`
Settings string `json:"settings,omitempty"`
Traits string `json:"traits,omitempty"`
IsDefault bool `json:"is_default,omitempty"`
IsSystem bool `json:"is_system,omitempty"`
// TracksWork, when non-nil, sets settings.tracks_work (PLAN-3535),
// overriding the same key inside Settings. Nil: the settings' own value,
// else the default (DefaultTracksWork).
TracksWork *bool `json:"tracks_work,omitempty"`
// AwaitsHuman, when non-nil, sets settings.awaits_human (IDEA-3906),
// overriding the same key inside Settings. Nil: the settings' own value,
// else absent (false).
AwaitsHuman *bool `json:"awaits_human,omitempty"`
// Source is how the collection was created ('web' | 'cli' | 'mcp'), set by
// the server from the request's auth shape, never from the body (BUG-3447).
// Empty leaves the column's '' (provenance unknown, never agent).
Source string `json:"-"`
}
// ValidCollectionSource returns source when it is one the server writes
// ('web' | 'cli' | 'mcp') and ” (provenance unknown, never agent) otherwise:
// the gate a bundle import's value passes before it is stored (BUG-3447).
func ValidCollectionSource(source string) string {
switch source {
case "web", "cli", "mcp":
return source
}
return ""
}
// FieldMigration describes a bulk update to apply to existing items when
// a collection schema changes (e.g. renaming select options).
type FieldMigration struct {
Field string `json:"field"` // field key to migrate
RenameOptions map[string]string `json:"rename_options,omitempty"` // old_value → new_value
}
type CollectionUpdate struct {
Name *string `json:"name,omitempty"`
Prefix *string `json:"prefix,omitempty"`
Icon *string `json:"icon,omitempty"`
Description *string `json:"description,omitempty"`
Schema *string `json:"schema,omitempty"`
Settings *string `json:"settings,omitempty"`
// Traits, when non-nil, replaces the collection's kernel-trait
// declarations. Nil leaves them untouched — which is the case for every
// client that rebuilds a collection's schema/settings blobs without
// knowing traits exist, and is why traits survive an ordinary edit.
Traits *string `json:"traits,omitempty"`
SortOrder *int `json:"sort_order,omitempty"`
// TracksWork, when non-nil, sets settings.tracks_work and nothing else in
// settings (PLAN-3535): the store merges it into the stored (or the
// supplied) settings under its own read, so a client can flip it without
// a read-modify-write of the whole blob. An older server ignores the
// member, so a caller that must know it landed checks the response.
TracksWork *bool `json:"tracks_work,omitempty"`
// AwaitsHuman, when non-nil, sets settings.awaits_human and nothing else
// in settings (IDEA-3906), merged under the store's own read exactly like
// TracksWork above.
AwaitsHuman *bool `json:"awaits_human,omitempty"`
Migrations []FieldMigration `json:"migrations,omitempty"`
// ExpectedUpdatedAt, when non-empty, opts into optimistic-concurrency
// control (BUG-2265, mirroring the item pattern from IDEA-1480): the
// store re-reads the row's updated_at under the workspace write lock and
// rejects the write with a conflict error when it no longer matches the
// RFC3339 timestamp the caller last read. Empty = last-write-wins (the
// legacy behaviour, unchanged for CLI / MCP / API callers that don't send
// it). Never persisted; consumed by UpdateCollection only.
ExpectedUpdatedAt string `json:"expected_updated_at,omitempty"`
}
// ErrInvalidSettingsType is returned by CollectionUpdate.UnmarshalJSON
// and CollectionCreate.UnmarshalJSON when the inbound `settings` value is
// neither a JSON object nor a JSON-encoded string. IDEA-1488: handler-
// layer shape validation ceiling, paired with the IDEA-1484 NOT NULL
// floor on collections.settings. Mirrors item.go's
// ErrInvalidFieldsType / ErrInvalidTagsType and view.go's
// ErrInvalidConfigType.
var ErrInvalidSettingsType = errors.New(`"settings" must be a JSON object or a JSON-encoded string`)
// UnmarshalJSON for CollectionCreate accepts `settings` either as the
// canonical JSON-encoded string shape (matches models.Collection.Settings
// storage) OR as the natural nested object shape any reasonable HTTP
// client would send. Mirrors ItemCreate.UnmarshalJSON; see
// flexJSONToString in item.go for the shape contract.
func (c *CollectionCreate) UnmarshalJSON(data []byte) error {
type alias CollectionCreate
aux := struct {
Settings json.RawMessage `json:"settings,omitempty"`
Traits json.RawMessage `json:"traits,omitempty"`
*alias
}{alias: (*alias)(c)}
if err := json.Unmarshal(data, &aux); err != nil {
return err
}
if settingsStr, err := flexJSONToString(aux.Settings, '{', ErrInvalidSettingsType); err != nil {
return err
} else if settingsStr != nil {
c.Settings = *settingsSentinel(settingsStr)
}
// `traits` gets the same both-shapes treatment as `settings`: the field
// is a JSON string in storage, but any reasonable client sends a nested
// object. TASK-2657.
if traitsStr, err := flexJSONToString(aux.Traits, '{', ErrInvalidTraitsType); err != nil {
return err
} else if traitsStr != nil {
c.Traits = *traitsStr
}
return nil
}
// settingsSentinel maps a whitespace-only settings string to the "" sentinel.
// flexJSONToString passes such a string through untrimmed, and the store
// would refuse it as not an object; a client that sends it means "" (BUG-3985).
// Settings only: the helper's other fields are BUG-3992.
func settingsSentinel(s *string) *string {
if s != nil && *s != "" && strings.TrimSpace(*s) == "" {
empty := ""
return &empty
}
return s
}
// UnmarshalJSON for CollectionUpdate accepts `settings` either as the
// canonical JSON-encoded string shape OR as the natural nested object
// shape. Mirrors ItemUpdate.UnmarshalJSON; the struct field stays
// `*string` and downstream consumers are unchanged.
//
// IDEA-1488: closes the shape-validation gap that IDEA-1484's NOT NULL
// floor doesn't cover. After this lands, the writer-side store coercion
// at collections.go:248 only sees empty-string-or-valid-object input.
func (u *CollectionUpdate) UnmarshalJSON(data []byte) error {
type alias CollectionUpdate
aux := struct {
Settings json.RawMessage `json:"settings,omitempty"`
Traits json.RawMessage `json:"traits,omitempty"`
*alias
}{alias: (*alias)(u)}
if err := json.Unmarshal(data, &aux); err != nil {
return err
}
settingsStr, err := flexJSONToString(aux.Settings, '{', ErrInvalidSettingsType)
if err != nil {
return err
}
u.Settings = settingsSentinel(settingsStr)
// Absent `traits` stays nil, which UpdateCollection reads as "leave the
// declarations alone" — the case for every client that predates traits.
// TASK-2657.
traitsStr, err := flexJSONToString(aux.Traits, '{', ErrInvalidTraitsType)
if err != nil {
return err
}
u.Traits = traitsStr
return nil
}
// ArchivedCollection is one archived (soft-deleted) collection, as the
// restore surfaces list it (TASK-2189). ItemCount counts its live items: an
// archive hides them behind the collection's deleted_at and deletes none.
type ArchivedCollection struct {
ID string `json:"id"`
Name string `json:"name"`
Slug string `json:"slug"`
Prefix string `json:"prefix"`
Icon string `json:"icon,omitempty"`
ItemCount int `json:"item_count"`
ArchivedAt string `json:"archived_at"`
}