Files
pad/internal/server/handlers_views.go
T
xarmian c72fe5a663 feat(items): add unparented filtering contract (#926)
* feat(items): add unparented filtering contract

* fix(items): preserve unparented projection state

* fix(views): preserve reserved filter on reset

* fix(items): resync projection scope changes

* fix(items): address PR 926 review findings

- localIndex: fetch snapshot before clearing store/cache in resyncProjectionScope (no data-loss window on fetch failure)
- items: degrade to committed item when post-parent-link readback fails instead of 500
- items: treat unparented=<non-true> as a field filter so a schema field named unparented still filters
- persistence: delete dead persistCursor
- mark validateUnparentedListRequest canonical; cross-reference the 3 early-feedback copies

* fix(items): resync race + purge safety per Codex review (round 1)

- resyncProjectionScope: merge-reconcile instead of blunt clear so a higher-seq
  upsert/delta racing the snapshot fetch is preserved (not erased) and the cursor
  never regresses below it
- recheck generation after persistWipe so a sign-out/403 purge during the wipe
  can't resurrect purged rows via persistDelta
- snapshot rows authoritatively replace local copies (drop is_unparented on
  downgrade); mergeRow's projection-preservation is bypassed for resync

* fix(items): sanitize projection bit on preserved racing rows per Codex review (round 2)

When a projection resync lands a restricted snapshot, strip is_unparented from
any racing higher-seq row kept by the seq guards — the old scope no longer grants
it. Keep the row itself (dropping it would reintroduce the racing-mutation data
loss; server 403 enforces real visibility).

* fix(items): transactional cache replace in resync per Codex review (round 3)

Replace wipe()+persistDelta() in resyncProjectionScope with a single
persistReplace() transaction (clear + write in one tx). Avoids the
deleteDatabase() onblocked cross-tab hang where a pending delete stalls the
following reopen+write indefinitely, wedging the resync promise. wipe() stays
for the sign-out / schema-mismatch full-teardown paths.

* fix(items): drop-and-replay resync reconciliation per Codex review (round 4)

Rework resyncProjectionScope: drop every row absent from the authoritative
snapshot (not just older-than-cursor ones) and pin the cursor to the snapshot
cursor. A post-snapshot mutation the client can still see is re-fetched by the
next /items-changes?since=cursor under the NEW scope, so visible rows return and
old-scope-hidden rows stay gone — no old-scope row survives the resync, and
nothing is permanently lost. Present-in-snapshot racing edits are still kept
(is_unparented stripped under a restricted scope).

* fix(items): continue delta poll after resync so replay actually fires (round 5)

The drop-and-replay resync (round 4) pins the cursor to the snapshot cursor so
post-snapshot mutations re-fetch under the new scope — but both poll loops broke
out / returned immediately after the resync, so the replay never ran until an
unrelated sync/reload. Both callers now continue the loop from the pinned cursor;
resync already aligned the scope so the branch can't re-fire, and the existing
50-iteration cap bounds it.

* fix(items): keep pendingResync set until replay catches up (round 6)

resyncProjectionScope cleared pendingResync after installing the snapshot but
before the pinned-cursor replay drained. If that replay later failed or hit the
50-page cap, pendingResync stayed false and the next bootstrap() no-opped with
racing mutations still missing. Let the reconcile loop's caughtUp logic own the
flag instead.

* fix(items): set pendingResync when any resync begins (round 7)

Round 6 removed the premature clear but only the bootstrap path pre-sets
pendingResync; a page deltaSync resync ran with it false, so a failed/capped
replay there wouldn't trigger a bootstrap resume. Set pendingResync=true at the
start of resyncProjectionScope so any caller marks catch-up pending; the
reconcile loop clears it on caughtUp.

* fix(items): fence stale optimistic writes + epoch-guard resync catch-up (round 8)

Adds a resync-epoch + fenced-id mechanism to close the last two race classes:

- fencedIds: a resync records the ids it dropped (hidden under the new scope).
  upsert() refuses a fenced id, so a stale old-scope create/update response
  resolving after the resync can't resurrect a now-hidden row that no new-scope
  delta would evict (P1). An authoritative applyDelta re-add un-fences; the next
  resync recomputes the set (re-upgrade clears it). Self-contained in the store —
  no epoch threading through the optimistic callers.
- scopeEpoch: bumped when a resync installs a new snapshot. Both reconcile loops
  capture it before each /items-changes and skip treating a response that raced a
  concurrent resync as caught-up, so a stale in-flight delta can't clear
  pendingResync without validating the pinned cursor (P2).

Regression test covers fence → reject stale upsert → authoritative re-add
un-fences → later edits accepted.

* fix(items): bump scope epoch before resync fetch (round 9 P2)

scopeEpoch advanced only after listIndex() returned, so a reconcile response
racing the fetch saw the old epoch and could clear the pendingResync the resync
set at start. Bump the epoch before the network await instead.
2026-07-13 22:46:55 -04:00

351 lines
9.6 KiB
Go

package server
import (
"database/sql"
"encoding/json"
"errors"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/PerpetualSoftware/pad/internal/models"
)
// handleListViews returns all saved views for a collection.
func (s *Server) handleListViews(w http.ResponseWriter, r *http.Request) {
workspaceID, ok := s.getWorkspaceID(w, r)
if !ok {
return
}
collSlug := chi.URLParam(r, "collSlug")
coll, err := s.store.GetCollectionBySlug(workspaceID, collSlug)
if err != nil {
writeInternalError(w, err)
return
}
if coll == nil {
writeError(w, http.StatusNotFound, "not_found", "Collection not found")
return
}
// Check collection visibility
visibleIDs, visErr := s.visibleCollectionIDs(r, workspaceID)
if visErr != nil {
writeInternalError(w, visErr)
return
}
if !isCollectionVisible(coll.ID, visibleIDs) {
writeError(w, http.StatusNotFound, "not_found", "Collection not found")
return
}
views, err := s.store.ListViews(workspaceID, coll.ID)
if err != nil {
writeInternalError(w, err)
return
}
if views == nil {
views = []models.View{}
}
if visibleIDs != nil {
stripReservedUnparentedFromViews(views)
}
writeJSON(w, http.StatusOK, views)
}
// handleCreateView creates a new saved view for a collection.
func (s *Server) handleCreateView(w http.ResponseWriter, r *http.Request) {
if !requireMinRole(w, r, "editor") {
return
}
workspaceID, ok := s.getWorkspaceID(w, r)
if !ok {
return
}
collSlug := chi.URLParam(r, "collSlug")
coll, err := s.store.GetCollectionBySlug(workspaceID, collSlug)
if err != nil {
writeInternalError(w, err)
return
}
if coll == nil {
writeError(w, http.StatusNotFound, "not_found", "Collection not found")
return
}
// Check collection visibility and edit permission (grant-aware)
visibleIDs, visErr := s.visibleCollectionIDs(r, workspaceID)
if visErr != nil {
writeInternalError(w, visErr)
return
}
if !isCollectionVisible(coll.ID, visibleIDs) {
writeError(w, http.StatusNotFound, "not_found", "Collection not found")
return
}
if !s.requireEditPermission(w, r, workspaceID, "", coll.ID) {
return
}
var input models.ViewCreate
if err := decodeJSON(r, &input); err != nil {
// IDEA-1488: surface the domain-level error from
// ViewCreate.UnmarshalJSON without the "invalid JSON: ..."
// wrapper from decodeJSON, so callers see a clean message
// naming the field (mirrors handlers_items.go:641 precedent).
if errors.Is(err, models.ErrInvalidConfigType) {
writeError(w, http.StatusBadRequest, "bad_request", models.ErrInvalidConfigType.Error())
return
}
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
if input.Name == "" {
writeError(w, http.StatusBadRequest, "bad_request", "Name is required")
return
}
if visibleIDs != nil {
input.Config = stripReservedUnparentedViewFilter(input.Config)
}
input.CollectionID = &coll.ID
view, err := s.store.CreateView(workspaceID, input)
if err != nil {
writeInternalError(w, err)
return
}
writeJSON(w, http.StatusCreated, view)
}
// requireViewVisible looks up a view by ID, verifies it belongs to the
// workspace, and checks that its collection is visible. Returns the view
// or writes an error and returns nil.
func (s *Server) requireViewVisible(w http.ResponseWriter, r *http.Request, workspaceID, viewID string) *models.View {
view, err := s.store.GetView(viewID)
if err != nil || view == nil || view.WorkspaceID != workspaceID {
writeError(w, http.StatusNotFound, "not_found", "View not found")
return nil
}
if view.CollectionID != nil {
visibleIDs, visErr := s.visibleCollectionIDs(r, workspaceID)
if visErr != nil {
writeInternalError(w, visErr)
return nil
}
if !isCollectionVisible(*view.CollectionID, visibleIDs) {
writeError(w, http.StatusNotFound, "not_found", "View not found")
return nil
}
}
return view
}
// requireViewEditable is like requireViewVisible but also checks edit permission
// on the view's collection (grant-aware for guests/restricted members).
func (s *Server) requireViewEditable(w http.ResponseWriter, r *http.Request, workspaceID, viewID string) *models.View {
view := s.requireViewVisible(w, r, workspaceID, viewID)
if view == nil {
return nil
}
if view.CollectionID != nil {
if !s.requireEditPermission(w, r, workspaceID, "", *view.CollectionID) {
return nil
}
}
return view
}
// handleUpdateView modifies an existing saved view.
func (s *Server) handleUpdateView(w http.ResponseWriter, r *http.Request) {
workspaceID, ok := s.getWorkspaceID(w, r)
if !ok {
return
}
viewID := chi.URLParam(r, "viewID")
existingView := s.requireViewEditable(w, r, workspaceID, viewID)
if existingView == nil {
return
}
var input models.ViewUpdate
if err := decodeJSON(r, &input); err != nil {
// IDEA-1488: surface the domain-level error from
// ViewUpdate.UnmarshalJSON without the "invalid JSON: ..."
// wrapper from decodeJSON, so callers see a clean message
// naming the field (mirrors handlers_items.go:641 precedent).
if errors.Is(err, models.ErrInvalidConfigType) {
writeError(w, http.StatusBadRequest, "bad_request", models.ErrInvalidConfigType.Error())
return
}
writeError(w, http.StatusBadRequest, "bad_request", err.Error())
return
}
visibleIDs, visErr := s.visibleCollectionIDs(r, workspaceID)
if visErr != nil {
writeInternalError(w, visErr)
return
}
if visibleIDs != nil && input.Config != nil {
config, err := preserveReservedUnparentedViewFilter(existingView.Config, *input.Config)
if err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "Config must be valid JSON")
return
}
input.Config = &config
}
view, err := s.store.UpdateView(viewID, input)
if err != nil {
if err == sql.ErrNoRows {
writeError(w, http.StatusNotFound, "not_found", "View not found")
return
}
writeInternalError(w, err)
return
}
if visibleIDs != nil {
view.Config = stripReservedUnparentedViewFilter(view.Config)
}
writeJSON(w, http.StatusOK, view)
}
// handleDeleteView removes a saved view.
func (s *Server) handleDeleteView(w http.ResponseWriter, r *http.Request) {
workspaceID, ok := s.getWorkspaceID(w, r)
if !ok {
return
}
viewID := chi.URLParam(r, "viewID")
if s.requireViewEditable(w, r, workspaceID, viewID) == nil {
return
}
if err := s.store.DeleteView(viewID); err != nil {
if err == sql.ErrNoRows {
writeError(w, http.StatusNotFound, "not_found", "View not found")
return
}
writeInternalError(w, err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// reservedUnparentedViewField is the saved-view pseudo-field the web phase
// uses to persist the structural filter. It is not a collection-schema field.
// Restricted and public consumers must never receive it because evaluating it
// would expose whether hidden structural relationships exist.
const reservedUnparentedViewField = "$unparented"
func stripReservedUnparentedFromViews(views []models.View) {
for i := range views {
views[i].Config = stripReservedUnparentedViewFilter(views[i].Config)
}
}
func stripReservedUnparentedViewFilter(config string) string {
if config == "" {
return config
}
var doc map[string]any
if err := json.Unmarshal([]byte(config), &doc); err != nil {
return config
}
rawFiltersValue, hasFilters := doc["filters"]
if !hasFilters {
return config
}
rawFilters, ok := rawFiltersValue.([]any)
if !ok {
// View config is intentionally flexible, but the filter evaluator only
// accepts arrays. Drop malformed shapes rather than returning their raw
// contents to restricted/public callers, where a nested reserved field
// could otherwise bypass the element-wise sanitizer below.
delete(doc, "filters")
b, err := json.Marshal(doc)
if err != nil {
return "{}"
}
return string(b)
}
filtered := make([]any, 0, len(rawFilters))
changed := false
for _, raw := range rawFilters {
filter, ok := raw.(map[string]any)
if ok && filter["field"] == reservedUnparentedViewField {
changed = true
continue
}
filtered = append(filtered, raw)
}
if !changed {
return config
}
doc["filters"] = filtered
b, err := json.Marshal(doc)
if err != nil {
return config
}
return string(b)
}
// preserveReservedUnparentedViewFilter applies a restricted caller's visible
// config replacement without letting that round-trip delete an unrestricted
// $unparented filter which the caller was never allowed to see. New reserved
// filters from the submitted config are stripped first; only entries already
// present in the stored config are restored.
func preserveReservedUnparentedViewFilter(existingConfig, submittedConfig string) (string, error) {
reserved := reservedUnparentedFilters(existingConfig)
// Match Store.UpdateView's valid reset sentinel. Normalizing before the
// merge keeps restricted behavior identical whether or not the stored
// config contains a hidden reserved filter.
if submittedConfig == "" {
submittedConfig = "{}"
}
sanitized := stripReservedUnparentedViewFilter(submittedConfig)
if len(reserved) == 0 {
return sanitized, nil
}
var doc map[string]any
if err := json.Unmarshal([]byte(sanitized), &doc); err != nil {
return "", err
}
filters, _ := doc["filters"].([]any)
doc["filters"] = append(filters, reserved...)
b, err := json.Marshal(doc)
if err != nil {
return "", err
}
return string(b), nil
}
func reservedUnparentedFilters(config string) []any {
var doc map[string]any
if err := json.Unmarshal([]byte(config), &doc); err != nil {
return nil
}
filters, ok := doc["filters"].([]any)
if !ok {
return nil
}
reserved := make([]any, 0, 1)
for _, raw := range filters {
filter, ok := raw.(map[string]any)
if ok && filter["field"] == reservedUnparentedViewField {
reserved = append(reserved, raw)
}
}
return reserved
}