refactor(bootstrap): slim Collections projection — drop id/timestamps/settings, parse schema inline (TASK-1412) (#535)

Introduces BootstrapCollection — a purpose-built projection for the
bootstrap response that replaces []models.Collection on
AgentBootstrap.Collections. Drops fields the /pad skill never reads:

  - id, workspace_id           — agent addresses collections by slug
  - created_at, updated_at,     — irrelevant at context-load time
    deleted_at
  - settings                    — quick_actions + view defaults are
                                  web-UI chat-prompt config

The remaining schema string is delivered as a nested JSON object
(json.RawMessage) rather than a JSON-encoded string, killing the
backslash-escape overhead so the agent sees real {}/[] structure
instead of double-encoded quotes. json.Valid() gates the emission
so a future migration leaving non-JSON in the column can't break
agent-side json.Unmarshal — invalid/empty schemas are simply
omitted (omitempty).

Measured against the bootstrapSizeBudget fixture (TASK-1411):

                   before        after        delta
  collections      8,848 b       3,979 b      -4,869 b (-55%)
  total bootstrap  13,861 b      8,992 b      -4,869 b (-35%)

Budget tightened from 16 KiB to 11 KiB to lock in the win. Later
PLAN-1410 PRs (TASK-1413's dedup + dashboard caps, TASK-1417's
final measurement) tighten further.

Wire-shape change details:

  - BuildAgentBootstrap holds collections as []models.Collection
    through the visibility-restricted role+count recompute (which
    keys lookups by Collection.ID), then projects to []BootstrapCollection
    at the end of that section. ID-keyed recompute logic is preserved
    verbatim — only the final wire shape changes.
  - printBootstrapMarkdown was already reading {slug, name, prefix}
    via its own anonymous struct; those three are preserved.
  - No web-UI consumers exist for /agent/bootstrap (grep confirms),
    so no client-side churn.

Out of scope (handled by later PLAN-1410 PRs):

  - Dedup'ing top-level recent_activity, dropping convention slug,
    capping dashboard.attention/recent_activity (TASK-1413).
  - ToolSurfaceVersion bump 0.3 → 0.4 (TASK-1418, after all shape
    changes land).

Parent: PLAN-1410.

internal/server/handlers_bootstrap.go

@@ -4,6 +4,7 @@ import (
 	"encoding/json"
 	"net/http"
 	"sort"
+	"strings"
 	"time"
 
 	"github.com/PerpetualSoftware/pad/internal/models"
@@ -22,7 +23,7 @@ import (
 type AgentBootstrap struct {
 	Workspace      AgentBootstrapWorkspace      `json:"workspace"`
 	User           AgentBootstrapUser           `json:"user"`
-	Collections    []models.Collection          `json:"collections"`
+	Collections    []BootstrapCollection        `json:"collections"`
 	Conventions    []AgentBootstrapConvention   `json:"conventions"`
 	Roles          []models.AgentRole           `json:"roles"`
 	Playbooks      []AgentBootstrapPlaybookMeta `json:"playbooks"`
@@ -30,6 +31,69 @@ type AgentBootstrap struct {
 	RecentActivity []DashboardActivity          `json:"recent_activity"`
 }
 
+// BootstrapCollection is the lightweight collection projection delivered
+// in the agent bootstrap response. Distinct from models.Collection: drops
+// fields the agent never reads (id, workspace_id, created_at, updated_at,
+// deleted_at) and the web-UI `settings` blob (quick-action prompts,
+// default views, group-by hints) so the per-invocation payload stays
+// tight. The `schema` field is delivered as a nested JSON object rather
+// than a JSON-encoded string so the agent sees real `{}`/`[]` structure
+// instead of backslash-escaped quotes — roughly 25% byte reduction on
+// the schema field alone, more when escape-heavy.
+//
+// Fields preserved are exactly what the /pad skill consumes:
+//   - slug / name / prefix / icon / description — addressing + listing
+//   - schema — drives `pad item create/update --field key=value` validation
+//   - item_count / active_item_count — surface-area counts in greetings
+//   - is_default / is_system — distinguish template seeds from custom collections
+//   - sort_order — preserves the workspace's authored ordering
+//
+// PLAN-1410 / TASK-1412. Pairs with the bootstrapSizeBudget benchmark
+// added in TASK-1411 — landing this projection tightens the budget by
+// ~25-30% of the collections section.
+type BootstrapCollection struct {
+	Slug            string          `json:"slug"`
+	Name            string          `json:"name"`
+	Prefix          string          `json:"prefix"`
+	Icon            string          `json:"icon,omitempty"`
+	Description     string          `json:"description,omitempty"`
+	Schema          json.RawMessage `json:"schema,omitempty"`
+	SortOrder       int             `json:"sort_order"`
+	IsDefault       bool            `json:"is_default"`
+	IsSystem        bool            `json:"is_system"`
+	ItemCount       int             `json:"item_count"`
+	ActiveItemCount int             `json:"active_item_count"`
+}
+
+// projectBootstrapCollection converts a models.Collection into the slim
+// bootstrap projection. The `schema` field is emitted as a nested JSON
+// object when the stored string is valid JSON; an empty or malformed
+// schema is omitted (omitempty + nil RawMessage) so the response never
+// carries garbage that would break agent-side json.Unmarshal.
+//
+// json.Valid() is cheap (single byte-stream pass, no allocation) and
+// guarantees the wire shape stays parseable even if a future migration
+// or buggy write leaves a non-JSON value in the column. Defensive only:
+// every collection created via the API today writes valid JSON.
+func projectBootstrapCollection(c models.Collection) BootstrapCollection {
+	out := BootstrapCollection{
+		Slug:            c.Slug,
+		Name:            c.Name,
+		Prefix:          c.Prefix,
+		Icon:            c.Icon,
+		Description:     c.Description,
+		SortOrder:       c.SortOrder,
+		IsDefault:       c.IsDefault,
+		IsSystem:        c.IsSystem,
+		ItemCount:       c.ItemCount,
+		ActiveItemCount: c.ActiveItemCount,
+	}
+	if s := strings.TrimSpace(c.Schema); s != "" && json.Valid([]byte(s)) {
+		out.Schema = json.RawMessage(s)
+	}
+	return out
+}
+
 // AgentBootstrapWorkspace is the minimal workspace projection (slug + name
 // + id) the agent needs to address the workspace in subsequent calls.
 type AgentBootstrapWorkspace struct {
@@ -157,9 +221,13 @@ func (s *Server) BuildAgentBootstrap(workspaceID string, user *models.User, r *h
 		}
 	}
 
-	// Collections — keep the same shape ListCollections returns, filtered
-	// by visibility so a guest only sees collections they're permitted
-	// into. Mirrors handleListCollections.
+	// Collections — load and apply visibility filtering. We hold these
+	// in their full models.Collection shape through the role/count
+	// recompute below (which keys lookups by Collection.ID), then
+	// project to BootstrapCollection at the end of this section. The
+	// projection drops id/workspace_id/timestamps/settings and parses
+	// the schema string into a nested object — see BootstrapCollection
+	// godoc + PLAN-1410 / TASK-1412.
 	collections, err := s.store.ListCollections(workspaceID)
 	if err != nil {
 		return nil, err
@@ -173,7 +241,6 @@ func (s *Server) BuildAgentBootstrap(workspaceID string, user *models.User, r *h
 		}
 		collections = filtered
 	}
-	out.Collections = collections
 
 	// Build the (collectionIDs, itemIDs) tuple the sub-queries should
 	// project through. When the caller has item-level grants, switch to
@@ -189,7 +256,7 @@ func (s *Server) BuildAgentBootstrap(workspaceID string, user *models.User, r *h
 	// Conventions — only the always-on, active set, restricted by the
 	// caller's authorized view. A guest with a grant to one specific
 	// convention item gets only that item, not the whole always-on set.
-	conventionsCollVisible := visibleIDs == nil || isCollectionSlugVisible(out.Collections, "conventions")
+	conventionsCollVisible := visibleIDs == nil || isCollectionSlugVisible(collections, "conventions")
 	if conventionsCollVisible {
 		convs, cerr := s.collectAlwaysOnConventions(workspaceID, subCollIDs, subItemIDs)
 		if cerr != nil {
@@ -242,8 +309,12 @@ func (s *Server) BuildAgentBootstrap(workspaceID string, user *models.User, r *h
 		// callers see a self-consistent number rather than a leaked
 		// full-workspace value. Full members keep the store-side
 		// active_item_count untouched.
-		for i := range out.Collections {
-			c := &out.Collections[i]
+		//
+		// We mutate the local `collections` slice (models.Collection
+		// shape) here, before the bootstrap projection below — keyed
+		// by Collection.ID, which the projection drops.
+		for i := range collections {
+			c := &collections[i]
 			c.ItemCount = collItemCounts[c.ID]
 			c.ActiveItemCount = collItemCounts[c.ID]
 		}
@@ -254,10 +325,17 @@ func (s *Server) BuildAgentBootstrap(workspaceID string, user *models.User, r *h
 	}
 	out.Roles = roles
 
+	// Project collections into the slim bootstrap shape now that
+	// counts (above) have been recomputed for restricted callers.
+	out.Collections = make([]BootstrapCollection, 0, len(collections))
+	for _, c := range collections {
+		out.Collections = append(out.Collections, projectBootstrapCollection(c))
+	}
+
 	// Playbooks (metadata only) — restricted to the caller's authorized
 	// view. A guest granted one specific playbook item sees that one,
 	// not the whole collection.
-	playbooksCollVisible := visibleIDs == nil || isCollectionSlugVisible(out.Collections, "playbooks")
+	playbooksCollVisible := visibleIDs == nil || isCollectionSlugVisible(collections, "playbooks")
 	if playbooksCollVisible {
 		playbooks, perr := s.collectPlaybookMetadata(workspaceID, subCollIDs, subItemIDs)
 		if perr != nil {

internal/server/handlers_bootstrap_test.go

@@ -131,16 +131,16 @@ func TestBootstrapIncludesPlaybookMetadata(t *testing.T) {
 // keeps later shape-change PRs honest: each one tightens this constant
 // once the win is measured against the fixture.
 //
-// Initial budget (TASK-1411): the seeded fixture currently encodes to
-// ~13.8 KB; the budget is set at 16 KiB to give ~2 KB headroom for
-// minor non-shape variance (e.g. an extra timestamp millisecond, schema
-// reordering) without admitting silent regressions. Later PLAN-1410 PRs
-// tighten this constant once the win is measured against the fixture —
-// TASK-1417's measurement step records the post-trim actual.
+// Budget history (each line is a PLAN-1410 PR that landed a win):
 //
-// The constant intentionally lives next to the test that consumes it so
-// PRs touching the bootstrap shape see the budget in the diff.
-const bootstrapSizeBudget = 16 * 1024
+//	TASK-1411 — 16 KiB (baseline benchmark added; fixture at 13,861 bytes)
+//	TASK-1412 — 11 KiB (slim BootstrapCollection projection; fixture at 8,992 bytes — collections section dropped from 8,848 to 3,979 bytes)
+//
+// The next PRs in PLAN-1410 tighten this further (TASK-1413's dedup +
+// caps; TASK-1417's final measurement). The constant intentionally
+// lives next to the test that consumes it so PRs touching the bootstrap
+// shape see the budget in the diff.
+const bootstrapSizeBudget = 11 * 1024
 
 // TestBootstrapSizeBudget locks in a payload-size budget for the
 // bootstrap response so future regressions are caught at PR time.