mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-11 21:39:01 +00:00
ec71903be7
* feat(store): NOT NULL hardening on items.fields/tags + views.config (IDEA-1486) Paired ship of IDEA-1486 (sibling-table JSONB NOT NULL hardening) and IDEA-1488 (handler-layer shape validation for ViewUpdate/CollectionUpdate). Generalizes the IDEA-1484 / collections.settings precedent (PR #562) to the remaining nullable JSON columns and closes the shape-validation gap that NOT NULL alone doesn't cover. Schema layer (IDEA-1486 floor): - migrations/056_items_jsonb_not_null.sql: rebuild items with fields TEXT NOT NULL DEFAULT '{}' and tags TEXT NOT NULL DEFAULT '[]', preserving all 7 indexes, recreating the 3 items_fts triggers, and rebuilding the FTS5 index. Foreign-keys-off / on bookends are lifted outside the IDEA-1485 atomic-tx wrapper. - migrations/057_views_config_not_null.sql: rebuild views with config TEXT NOT NULL DEFAULT '{}'. - pgmigrations/035 + 036: SET NOT NULL + SET DEFAULT on the three JSONB columns. Split per-table to mirror the SQLite per-table file granularity. Store layer (IDEA-1486 floor): - items.go UpdateItem and views.go UpdateView normalize "" -> "{}" / "[]" before writing. Same boundary pattern as CreateItem and the IDEA-1484 precedent at collections.go:248. - export.go ImportWorkspace coerces empty-string AND malformed JSON at import time on items.fields, items.tags, and collections.settings. Malformed input is coerce-and-log via slog.Warn (length only, never raw value) so legacy bundles don't fail-stop on one bad row. - remapFieldIDs early-returns "{}" on empty input so the second-pass UPDATE can't write "" verbatim. - Migrated the existing fmt.Printf at export.go:329 to slog.Warn for consistency. Handler layer (IDEA-1488 ceiling): - ViewCreate / ViewUpdate UnmarshalJSON via flexJSONToString with new ErrInvalidConfigType sentinel. - CollectionCreate / CollectionUpdate UnmarshalJSON with new ErrInvalidSettingsType sentinel. - handlers_views.go and handlers_collections.go surface both sentinels as 400 with the domain-level message (mirrors the BUG-1144 precedent at handlers_items.go:641). Tests: - internal/store/items_views_jsonb_test.go: store-coercion + import coercion + log-and-coerce-on-malformed + SQLite schema introspection (7 indexes + 3 FTS triggers + items_fts virtual table survival) + Postgres NOT NULL enforcement + migration re-apply idempotency + item_links round-trip after rebuild. - internal/server/handlers_views_collections_jsonb_test.go: PATCH/POST flexible-shape coverage for views.config and collections.settings, including domain-level 400 message assertions that the response does not leak Go unmarshal internals. Refs: IDEA-1486, IDEA-1488, IDEA-1484 (precedent), IDEA-1485 (substrate). * fix(store,models): codex R1 follow-ups for IDEA-1486 / IDEA-1488 Three concrete defects surfaced by codex R1 against the initial paired ship. All three close holes that defeated parts of the original contract. P1.1: migration 056 missed the playbook invocation_slug unique index. - migrations/056_items_jsonb_not_null.sql: recreate the partial UNIQUE index idx_items_invocation_slug_per_collection from migration 054 verbatim after the other 7 indexes. Without it, the application-layer pre-check in handlers_items.go:checkUniqueFields would be a TOCTOU race with no DB-level guard — the original index that 054 explicitly added as the actual uniqueness backstop would be silently dropped during the items rebuild. - items_views_jsonb_test.go: the schema-introspection test now asserts 8 indexes, not 7. Verified via `grep -rn "ON items(" migrations/` that no other items-touching indexes were missed. P1.2: flexJSONToString didn't validate inner content of JSON-encoded strings. Pre-fix, `{"config": "[]"}` / `{"settings": "not json"}` / `{"fields": "[]"}` / `{"tags": "{}"}` slipped past the shape validators because the `case '"'` branch unmarshalled the envelope and returned the inner string verbatim — bypassing the whole point of IDEA-1488. - models/item.go: after unmarshalling the JSON-encoded string, validate that the trimmed inner content's first byte matches expectedStart ('{' / '[') AND parses as JSON. Empty inner strings still pass through to the store-layer empty-string coercion (IDEA-1486 floor), so legacy "" → default normalization is preserved. - The pre-existing ItemUpdate fields/tags path inherits the same tightening because it routes through this helper — covered by new test file handlers_items_jsonb_inner_shape_test.go. - Parallel handler tests for views.config and collections.settings added to handlers_views_collections_jsonb_test.go. P2: coerceJSONForImport accepted JSON null as well-formed. - store/export.go: json.Unmarshal("null", &m) returns err=nil with m staying nil; the prior code returned the raw "null" string verbatim, which lands as JSONB null on Postgres (satisfies NOT NULL since SQL NULL ≠ JSONB null) or text "null" on SQLite. The non-nil check on the unmarshalled value routes JSON null to the existing log-and-coerce path with the rest of the malformed shapes. - items_views_jsonb_test.go: extended import test with an item carrying fields=null / tags=null; expects both coerced to "{}" / "[]" and the structured slog.Warn emitted. Verified: make test (SQLite) and the full ./... suite against the existing port-5445 Postgres container both pass cleanly. Refs: IDEA-1486, IDEA-1488, codex R1 review. * fix(store,server): codex R2 follow-ups for IDEA-1486 / IDEA-1488 Two defects surfaced by codex R2. P1 is a real ship-breaker; P2 closes a parity gap that R1 missed. P1: migration backfill normalized only SQL NULL, not malformed/wrong- shape JSON. The four new migrations originally wrote `WHERE x IS NULL`. Rows with fields = '' / 'null' / '[]' / 'not json' all survived the filter, then violated the post-migration NOT NULL+shape contract. Concrete ship- breaker on SQLite: 056 recreates the partial UNIQUE index on json_extract(fields, '$.invocation_slug') from migration 054, and json_extract errors on rows whose fields fails json_valid — a single bad row breaks CREATE INDEX mid-migration. Toggle-verified: with the NULL-only WHERE, the new SQLite test fails at exactly that CREATE INDEX with "SQL logic error: malformed JSON (1)". Widened the backfill clauses: - migrations/056: UPDATE items WHERE fields IS NULL OR json_valid(fields)=0 OR json_type(fields)!='object' (same trio for tags with 'array'). - migrations/057: same trio for views.config. - pgmigrations/035: WHERE fields IS NULL OR jsonb_typeof(fields)!='object'. JSONB rejects invalid JSON on write so the json_valid leg isn't needed on Postgres; only the shape check matters. - pgmigrations/036: same shape check on views.config. Regression tests in internal/store/items_views_jsonb_test.go: - TestItemsViewsJSONB_SQLiteBackfillRepairsMalformedShapes: applies migrations through 053 (skipping 054 which would itself error on malformed rows), seeds every observable shape pathology — SQL NULL, empty string, JSON null literal, wrong-shape JSON, non-JSON garbage — then applies 055/056/057. Asserts every malformed row is repaired AND the partial UNIQUE index actually fires on duplicate invocation_slug post-rebuild (proving the CREATE INDEX path executed end-to-end). - TestItemsViewsJSONB_PostgresBackfillRepairsMalformedShapes: parallel Postgres coverage; seeds JSONB null / array / primitive via direct ::jsonb cast and asserts the widened WHERE clause repairs each. P2: handleCreateItem didn't unwrap ErrInvalidFieldsType/ErrInvalidTagsType. R1's flexJSONToString tightening propagated the sentinels through every UnmarshalJSON path, but handleCreateItem (POST /items) still returned 'invalid JSON: <wrapped>' from decodeJSON. PATCH and the view/collection POST/PATCH handlers already unwrapped — POST was the outlier. - internal/server/handlers_items.go: mirror the PATCH-side errors.Is handling at the POST path. Brief, three-line diff. - handlers_items_jsonb_inner_shape_test.go: new TestCreateItem_ JSONEncodedStringInnerShapeValidated covers POST with fields=`[]`, fields=42, tags=`{}`, tags={"x":1}, plus a valid positive control. Asserts no "invalid JSON:" wrapper and presence of the sentinel message verbatim. Backfill-pattern audit (codex R2's grep prompt): only 055 / pg-034 (collections.settings, already shipped) exhibits the same NULL-only WHERE gap. Per the brief: NOT touched — retroactive repair belongs to a separate IDEA. Other NULL-only backfills (043/pg-023's oauth_providers, 044/pg-024's expires_at) handle their respective shapes correctly or aren't JSON columns. Verified: make test (SQLite) clean. Full ./... suite against the existing port-5445 Postgres container clean (one unrelated flake in internal/collab passed on rerun). Refs: IDEA-1486, IDEA-1488, codex R2 review.
208 lines
5.5 KiB
Go
208 lines
5.5 KiB
Go
package store
|
|
|
|
import (
|
|
"database/sql"
|
|
"fmt"
|
|
|
|
"github.com/PerpetualSoftware/pad/internal/models"
|
|
)
|
|
|
|
// CreateView adds a new saved view.
|
|
func (s *Store) CreateView(workspaceID string, input models.ViewCreate) (*models.View, error) {
|
|
id := newID()
|
|
ts := now()
|
|
|
|
slug := input.Slug
|
|
if slug == "" {
|
|
slug = slugify(input.Name)
|
|
}
|
|
slug, err := s.uniqueSlug("views", "workspace_id", workspaceID, slug)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("generate slug: %w", err)
|
|
}
|
|
|
|
config := input.Config
|
|
if config == "" {
|
|
config = "{}"
|
|
}
|
|
|
|
viewType := input.ViewType
|
|
if viewType == "" {
|
|
viewType = "list"
|
|
}
|
|
|
|
_, err = s.db.Exec(s.q(`
|
|
INSERT INTO views (id, workspace_id, collection_id, name, slug, view_type, config, sort_order, is_default, created_at, updated_at)
|
|
VALUES (?, ?, ?, ?, ?, ?, ?, 0, ?, ?, ?)`),
|
|
id, workspaceID, input.CollectionID, input.Name, slug, viewType, config, s.dialect.BoolToInt(false), ts, ts,
|
|
)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("insert view: %w", err)
|
|
}
|
|
|
|
return s.GetView(id)
|
|
}
|
|
|
|
// GetView returns a single view by ID.
|
|
func (s *Store) GetView(id string) (*models.View, error) {
|
|
var v models.View
|
|
var collectionID *string
|
|
var isDefault bool
|
|
var createdAt, updatedAt string
|
|
|
|
err := s.db.QueryRow(s.q(`
|
|
SELECT id, workspace_id, collection_id, name, slug, view_type, config, sort_order, is_default, created_at, updated_at
|
|
FROM views
|
|
WHERE id = ?`), id).Scan(
|
|
&v.ID, &v.WorkspaceID, &collectionID, &v.Name, &v.Slug, &v.ViewType,
|
|
&v.Config, &v.SortOrder, &isDefault, &createdAt, &updatedAt,
|
|
)
|
|
if err == sql.ErrNoRows {
|
|
return nil, nil
|
|
}
|
|
if err != nil {
|
|
return nil, fmt.Errorf("get view: %w", err)
|
|
}
|
|
|
|
v.CollectionID = collectionID
|
|
v.IsDefault = isDefault
|
|
v.CreatedAt = parseTime(createdAt)
|
|
v.UpdatedAt = parseTime(updatedAt)
|
|
return &v, nil
|
|
}
|
|
|
|
// GetViewBySlug returns a view by its workspace-scoped slug.
|
|
func (s *Store) GetViewBySlug(workspaceID, slug string) (*models.View, error) {
|
|
var v models.View
|
|
var collectionID *string
|
|
var isDefault bool
|
|
var createdAt, updatedAt string
|
|
|
|
err := s.db.QueryRow(s.q(`
|
|
SELECT id, workspace_id, collection_id, name, slug, view_type, config, sort_order, is_default, created_at, updated_at
|
|
FROM views
|
|
WHERE workspace_id = ? AND slug = ?`), workspaceID, slug).Scan(
|
|
&v.ID, &v.WorkspaceID, &collectionID, &v.Name, &v.Slug, &v.ViewType,
|
|
&v.Config, &v.SortOrder, &isDefault, &createdAt, &updatedAt,
|
|
)
|
|
if err == sql.ErrNoRows {
|
|
return nil, nil
|
|
}
|
|
if err != nil {
|
|
return nil, fmt.Errorf("get view by slug: %w", err)
|
|
}
|
|
|
|
v.CollectionID = collectionID
|
|
v.IsDefault = isDefault
|
|
v.CreatedAt = parseTime(createdAt)
|
|
v.UpdatedAt = parseTime(updatedAt)
|
|
return &v, nil
|
|
}
|
|
|
|
// ListViews returns all views for a collection within a workspace.
|
|
func (s *Store) ListViews(workspaceID, collectionID string) ([]models.View, error) {
|
|
rows, err := s.db.Query(s.q(`
|
|
SELECT id, workspace_id, collection_id, name, slug, view_type, config, sort_order, is_default, created_at, updated_at
|
|
FROM views
|
|
WHERE workspace_id = ? AND collection_id = ?
|
|
ORDER BY sort_order ASC, created_at ASC`), workspaceID, collectionID)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("list views: %w", err)
|
|
}
|
|
defer rows.Close()
|
|
|
|
var views []models.View
|
|
for rows.Next() {
|
|
var v models.View
|
|
var collID *string
|
|
var isDefault bool
|
|
var createdAt, updatedAt string
|
|
|
|
if err := rows.Scan(
|
|
&v.ID, &v.WorkspaceID, &collID, &v.Name, &v.Slug, &v.ViewType,
|
|
&v.Config, &v.SortOrder, &isDefault, &createdAt, &updatedAt,
|
|
); err != nil {
|
|
return nil, fmt.Errorf("scan view: %w", err)
|
|
}
|
|
v.CollectionID = collID
|
|
v.IsDefault = isDefault
|
|
v.CreatedAt = parseTime(createdAt)
|
|
v.UpdatedAt = parseTime(updatedAt)
|
|
views = append(views, v)
|
|
}
|
|
return views, rows.Err()
|
|
}
|
|
|
|
// UpdateView modifies an existing view.
|
|
func (s *Store) UpdateView(id string, input models.ViewUpdate) (*models.View, error) {
|
|
ts := now()
|
|
|
|
// Build dynamic SET clause
|
|
sets := []string{"updated_at = ?"}
|
|
args := []interface{}{ts}
|
|
|
|
if input.Name != nil {
|
|
sets = append(sets, "name = ?")
|
|
args = append(args, *input.Name)
|
|
}
|
|
if input.ViewType != nil {
|
|
sets = append(sets, "view_type = ?")
|
|
args = append(args, *input.ViewType)
|
|
}
|
|
if input.Config != nil {
|
|
// IDEA-1486: normalize the empty-string sentinel to a valid JSON
|
|
// object before writing. After the NOT NULL DEFAULT '{}'
|
|
// hardening (migration 057 / pgmigrations 036), Postgres rejects
|
|
// "" at JSONB type-validation and SQLite would silently store
|
|
// invalid JSON. Mirrors CreateView at views.go:24-27 and the
|
|
// IDEA-1484 precedent at collections.go:248. Shape validation
|
|
// (object vs. array vs. primitive) is handled at the handler
|
|
// boundary by ViewUpdate.UnmarshalJSON (IDEA-1488).
|
|
config := *input.Config
|
|
if config == "" {
|
|
config = "{}"
|
|
}
|
|
sets = append(sets, "config = ?")
|
|
args = append(args, config)
|
|
}
|
|
if input.SortOrder != nil {
|
|
sets = append(sets, "sort_order = ?")
|
|
args = append(args, *input.SortOrder)
|
|
}
|
|
|
|
args = append(args, id)
|
|
|
|
query := "UPDATE views SET "
|
|
for i, s := range sets {
|
|
if i > 0 {
|
|
query += ", "
|
|
}
|
|
query += s
|
|
}
|
|
query += " WHERE id = ?"
|
|
|
|
result, err := s.db.Exec(s.q(query), args...)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("update view: %w", err)
|
|
}
|
|
n, _ := result.RowsAffected()
|
|
if n == 0 {
|
|
return nil, sql.ErrNoRows
|
|
}
|
|
|
|
return s.GetView(id)
|
|
}
|
|
|
|
// DeleteView removes a view by ID.
|
|
func (s *Store) DeleteView(id string) error {
|
|
result, err := s.db.Exec(s.q("DELETE FROM views WHERE id = ?"), id)
|
|
if err != nil {
|
|
return fmt.Errorf("delete view: %w", err)
|
|
}
|
|
n, _ := result.RowsAffected()
|
|
if n == 0 {
|
|
return sql.ErrNoRows
|
|
}
|
|
return nil
|
|
}
|