mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-10-26 06:19:49 +00:00
ca158c22d8dfe942e327dc2accc163cbf6dcc7fb
384 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 xarmian
|
ca158c22d8 |
feat(apps): app install staging: apps/1 manifest, fetch policy, pending reservations, preview (TASK-3397, U8a) (#1770)
SPEC-6 U8a (DOC-3371 §1, §2 steps 1-5, §9). Strict apps/1 parsing; an https/no-redirect fetch policy screened at dial time with an admin private-origin list; pending-install reservations under the owner lock and the instance advisory lock, holding the full charge until the fetch ends; the owner preview with every artifact normalized by the importer's own code (normalizeArtifact, prepareCreateFields, both extracted behaviour-preserving and pinned by a parity golden); apps_enabled and the admin apps settings; a background sweep of expired pending installs; migration 118/092. The preview is advisory: it may refuse what the import would accept, never the reverse; U8b re-checks atomically. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
e5262160e1 |
feat(oauth): install clients, service tokens and the issuance barrier for installed apps (TASK-3394, SPEC-6 U5a) (#1769)
* feat(store): install clients, the issuance barrier and app_token_bindings (TASK-3394) * feat(oauth): install-client service tokens, in-process app-token introspection, public introspection hardening, A0 token gate (TASK-3394) * test(store): the issuance barrier serializes with disable, rotate and re-enable on Postgres (TASK-3394) * test(server): pin each install-token refusal by its own hint; bearer introspection of another token (TASK-3394) * fix(oauth): a grant carries the epoch read before client auth; the install-caller check reads credentials as fosite does (TASK-3394, codex r1) * fix(oauth): classify the token request from the body fosite parses, multipart included (TASK-3394, codex r2) * test(server): account for the install-client token rules' form readers in the body-reader census (TASK-3394) * chore(store): renumber the install-client migrations to 117 / pg 091 after #1768 (TASK-3394) |
||
|
xarmian
|
82395bde1c |
feat(import): mark rows a workspace import wrote, author not verified (BUG-3379) (#1755)
* feat(import): mark rows a workspace import wrote, author not verified (BUG-3379) Workspace import keeps the export's comment authors, version kinds and attachment uploader ids verbatim (BUG-3372 ruling (1)), and nothing distinguished those rows from authenticated authorship, so an importer could present a fabricated trail. Nothing stored could tell them apart either: user_id is NULL after an account deletion too, the export supplies the timestamps, and import writes no activity. - Migrations 107 / 081 add imported (INTEGER 0/1) to comments, item_versions and attachments. Only the import doors set it (JSON and bundle; a derived thumbnail or transform inherits its original's). - Comments, versions and attachment rows serialise an omitempty imported: true. - An imported attachment's uploaded_by is never resolved to a local name: the member route sends X-Pad-Attachment-Imported: 1 instead. That also closes an importer naming a real member as uploader by putting their id in the manifest. - The web marks them: "· imported" beside a comment or reply author, an Imported chip on a history row (and an imported row never merges with a native one), and "imported" in place of the uploader in the viewer. - Rows imported before this are not marked: there is nothing to backfill from. - MCP tool surface 0.64 (additive): the flag reaches comment and attachment responses on both transports. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(import): keep the imported mark through recovery, autosave runs and copies (BUG-3379) Codex review round 1: - An imported version claiming recovery or system lost its marker in the history; it is kept. - The History tab's autosave row had no marker; it shows the chip. - The server's autosave burst grouping could fold an imported snapshot into a native one (both often have no user id); it no longer does. - A cross-workspace copy carried imported=1 onto a clone the copier uploaded, hiding its real local uploader; the clone is not imported. - A store test inserting attachments by column list is updated for the new column. - The 0.64 changelog names the MCP responses that actually carry the flag: comments, attachment list and full history (not history summaries, attachment show or the attachment resource). Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(import): a local edit never lands inside an imported version (BUG-3379) Codex review round 2: the version throttle ignored the flag, so a local edit within the hour, by the same kind and source as the newest imported version, wrote no version and read as part of the imported change. An imported predecessor now always starts a new version. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
a2e1c5d9ca |
fix(oauth): narrow current-only wildcard connections to their current workspaces (BUG-3338) (#1752)
Connected Apps let a user turn include_future_workspaces off under the wildcard, but the wildcard is live and the flag was never read, so those connections kept reaching workspaces joined later. A startup step, next to BackfillOAuthConnections, turns each such connection into a specific list of its user's live memberships and clears the wildcard: the same result as "Limit to my current workspaces". Narrowing-only (it writes rows only for workspaces the wildcard already reached and only ever clears the wildcard), idempotent (a narrowed row no longer matches, and no door stores the split combination since BUG-3338), one transaction, and it logs how many connections it narrowed, how many rows it wrote, and how many were left empty. A Go step rather than a numbered migration because the migration runner has no per-migration hook to log a count. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
Matt Faltyn
|
5614980edc |
Make database restore preserve live data on failure (#1703)
Stage and normalize SQLite backups before atomic publication, checkpoint committed live journals safely, and stop PostgreSQL restores on SQL errors. Cover recovery failures and compatibility with real-engine regressions. |
||
|
xarmian
|
2bd0862af2 |
fix(auth): platform administration refuses an API token (BUG-3361) (#1742)
* fix(auth): platform administration refuses an API token (BUG-3361)
BREAKING for anyone driving /api/v1/admin/* or admin account creation with
a personal access token: those now answer 403 session_required. Use a
`pad auth login` CLI session, or on self-host set PAD_ADMIN_API_TOKENS=allow.
An admin's PAT carried full platform authority. codex reproduced the chain
during BUG-3336's review: register an account with a chosen password,
promote it to admin, sign in as it, and reset any account's password. A
leaked admin token therefore escalated to interactive sessions for any
account.
- refuseAPITokenForAdmin guards the whole /api/v1/admin route group, so an
admin route added later is refused by construction.
TestBUG3361_EveryAdminRouteRefusesAnAdminPAT walks the router to hold it.
- handleRegister refuses admin-create signup with an API token.
- PAD_ADMIN_API_TOKENS=allow is the self-host escape hatch. It is read at
startup, logs a WARN, and is never honoured in cloud mode. Documented in
docs/deployment.md, Security.
- A browser session and a CLI session (padsess_) are unaffected. So is the
Pad Cloud sidecar, which authenticates with the cloud secret, never a
PAT. /audit-log already refused a PAT (BUG-3334).
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test: drive platform administration with the admin's session, not its API token (BUG-3361)
CI on the first push: 70 E2E specs and TestMCPCapability_UpgradeChangesNothingWhileOff
called /api/v1/admin/* or admin account creation with an admin PAT, exactly
the use this change refuses. Our own harness was the first operator script
the breaking change broke.
- E2E: global setup keeps the session token login already returns as
fixture.adminSessionToken (padsess_ is not User-Agent bound when sent as
a Bearer). The seven admin call sites use it: account creation in
permission-walk, workspace-tabs, workspace-access-live, no-commerce and
account-delete; the admin user search; and PUT /admin/mcp. Every other
call keeps the PAT, including the MCP client config in plan-2310.
- mcp_capability_test: the admin drives /admin with a CLI session and keeps
its PAT for /mcp, which takes one.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test(e2e): the admin console specs use the admin's session (BUG-3361)
CI on
|
||
|
xarmian
|
484ca668a3 |
feat(server): revoking access re-checks live connections immediately (#1744)
* feat(server): access kicks re-check live connections now, not on the next tick (TASK-3365 PR1) Dave's ruling on TASK-3345 #3. A per-instance registry indexes live collab, /events and /events/stream connections by user and workspace; a kick runs the revalidation each already performs on its 60s tick immediately (the tick stays as the backstop). Kicks travel the watch bus as the server-internal KindAccessInvalidated, which watchStreamDelivers drops, and the existing workspace_access_changed publication kicks too, so the doors that already publish it are covered. Kicks are non-blocking sends that coalesce per connection. PR2 wires the remaining access and credential doors. * fix(server): access kicks ride their own transport, reach watch streams, install idempotently (TASK-3365 PR1) Codex r1: kicks on the client watch bus took sequence ids, replay slots and client queue space, so they could evict real notifications and open gaps. They now ride internal/accesskick (Redis pub/sub on its own channel, or local with one instance), and the access-changed helpers kick directly, so nothing internal touches the client bus and the internal kind is gone. Watch streams register under a wildcard workspace so workspace kicks reach them. SetAccessKickTransport is idempotent and replacement stops the old subscription. A partial revoke kicks too. Tests make the first tick deterministic. * fix(server): access kicks never wait on Redis on a request path (TASK-3365 PR1) Codex r2: the shared Redis client ignores a context deadline on command I/O, so a stalled Redis held each publish ~5s, serially per recipient, on the request that revoked access. Local connections are now kicked at once and remote publication goes through one bounded background worker (dropped when full; other instances fall back to the tick); the transport uses its own client clone with context timeouts enabled; lifecycle changes send one workspace kick, and a complete revoke no longer kicks twice. * fix(server): access-kick worker cancels on removal; transport owns and closes its client (TASK-3365 PR1) Codex r3: the worker published with context.Background, so removing a stalled transport left the worker and queue alive; it now has its own cancellable context. The transport's cloned client is closed when the server replaces or clears it, and the clone gets its own push-notification processor instead of registering on the source client's. |
||
|
xarmian
|
38c591673f |
fix(search): scope workspace search to visible items (#1723)
* fix(search): scope workspace search to visible items (BUG-3331) /api/v1/search is mounted outside RequireWorkspaceAccess and decides access itself. A caller with no visibility in a named workspace reached the store with an empty permission filter, which the store read as no filter at all. Store: the permission filter is fail-closed. SearchParams gains an explicit Unrestricted flag; empty CollectionIDs and ItemIDs (nil or not) now match nothing unless it is set. One helper applies the predicate on every query path (ref lookup, number lookup, FTS, count, facets) and adds a match-nothing predicate when a restricted filter is empty. Handler: searchAccessFor resolves each workspace the way RequireWorkspaceAccess does (fresh-install window, user-less workspace token limited to its own workspace, consent allow-list, cookie-only admin bypass, full and restricted members, grants for non-members), for both a named workspace and the unscoped fan-out. No access is an empty result, not a 403. Collections: a caller who sees a collection only through an item grant gets item_count and active_item_count computed from the granted items, not the whole collection. Co-Authored-By: Claude <noreply@anthropic.com> * test(search): cover the restricted number lookup, total, mixed fan-out and active count (BUG-3331) Codex r1 (gpt-6-luna, max scrutiny) found no P0/P1. Its P2 (a restricted member's direct grant to a soft-deleted collection now hides that collection's items from search while the item list still shows them) is search being the stricter of two doors; the item list's side is BUG-3333. Its test gaps are closed here: the bare-number lookup and the filtered total for an item-grant guest, a fan-out over workspaces with different access, and the active count following a closed granted item. Also: the named-workspace consent check is visible in handleSearch again (the allow-list coverage guard scans the handler; searchAccessFor still re-checks), and the now-unused removeString is gone. Co-Authored-By: Claude <noreply@anthropic.com> * fix(search): check the consent allow-list first in searchAccessFor (BUG-3331) Codex r2 (no P0/P1): the allow-list check came after the no-user branch, the reverse of RequireWorkspaceAccess, so a context with a mismatching allow-list and a matching legacy workspace token would have read its workspace. No current auth flow produces that combination, but the order now matches the middleware and the coverage claim is exact. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
xarmian
|
26fd8a6a46 |
feat(mcp): ChatGPT tools declare their OAuth scope; a missing scope answers the sign-in challenge (TASK-3321 U2c) (#1720)
* feat(mcp): ChatGPT tools declare their OAuth scope; a missing scope answers the sign-in challenge (TASK-3321 U2c) Every ChatGPT catalog tool now carries securitySchemes, an oauth2 scheme naming the one scope it needs: pad:read for a read tool and pad:write for anything else. The Apps SDK reads it at the tool's top level and keeps _meta.securitySchemes as a mirror. mcp-go's Tool marshals a fixed set of keys, so the tool carries the _meta copy, and WithChatGPTToolSchemes adds the top-level copy to a single tools/list JSON response on the ChatGPT mount. Every other request and response passes through byte for byte. When a write tool is called with a read-only grant, the handler answers before dispatching anything: an isError result whose _meta["mcp/www_authenticate"] is a Bearer challenge with error="insufficient_scope" and an error_description, naming the protected-resource document of the mount the request arrived on (server.MCPResourceMetadataURLFromContext). That is what makes ChatGPT offer to reconnect with the wider grant. The wire golden is built through the production transport (newChatGPTTransport). Only the ChatGPT golden moved; /mcp is byte-identical. ChatGPTToolSurfaceVersion stays 0.1: it is unstable and unpublished. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(mcp): ChatGPT tools/list is a JSON body with top-level securitySchemes when asked the way ChatGPT asks (TASK-3321 U2c) Streamable HTTP upgrades a response to an event stream only by flushing notifications mid-request (the remote transport has no event store), and WithChatGPTToolSchemes hands the transport a writer that cannot flush. So tools/list requested with Accept: application/json, text/event-stream is a JSON body, and every tool carries securitySchemes at the top level. One leg pins that through the production transport, and another pins that the buffered writer never implements http.Flusher. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
1339916b9c |
feat(chatgpt): serve OpenAI's domain challenge at /.well-known/openai-apps-challenge (TASK-3321 G4) (#1717)
OpenAI verifies a plugin's domain by fetching the token its dashboard issued from /.well-known/openai-apps-challenge on the MCP host, and the body must be the token alone. PAD_OPENAI_APPS_CHALLENGE (toml openai_apps_challenge) resolves into config.MCPEndpoints, so it reaches the server through the existing SetMCPConfig binding. A value with inner whitespace or a control character, or over 1024 bytes, is refused into Problems() and logged at startup rather than served. The route answers 200 text/plain with exactly the token only on cloud, only with a token set, and only on the MCP URL's host. Every other case is the U0a JSON 404, byte-identical to an unowned well-known path. It is not behind the MCP or OAuth availability gates, because verification happens before the plugin server is switched on. Co-authored-by: Claude <noreply@anthropic.com> |
||
|
xarmian
|
6118f7065c |
feat(mcp): mount the ChatGPT catalog at /mcp/chatgpt, OAuth-only, off by default (TASK-3321 U2b) (#1719)
The ChatGPT projection (U1) gets its own transport at <MCP URL>/chatgpt, behind PAD_CHATGPT_MCP_ENABLED=true and served only while OAuth is available. It is its own OAuth resource (U2a's second canonical): its path-aware protected-resource document is /.well-known/oauth-protected-resource/mcp/chatgpt, its 401s name that document, and tokens do not cross between the two mounts. Personal access tokens are refused there with a 401 naming OAuth-only, ahead of the PAT branch that would accept them without the audience check. The consent screen names the ChatGPT surface when the requested resource is the ChatGPT one. The mount is wired only while the door versions content (ChatGPTDoorVersionsContent, held to the behaviour by a test) and every catalog tool has a response shape (ChatGPTCatalogReady); otherwise startup logs and skips it. /mcp is byte-identical. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
2dc5ac7058 |
feat(mcp): ChatGPT item_history names who made each change; email-shaped names are masked (TASK-3321) (#1716)
#1714 put actor_name on every history summary row. The ChatGPT item_history allow-list now keeps it, and the description says "who" again. Per the lead's ruling, a display name that is (or contains) an email address reaches ChatGPT as "a Pad user" in every person field (author, agent_name, actor_name, assigned_user(_name), owner_username, assigned_users, and user.name), never as the local part, which would be a new identifier. Other names (workspace, collection, role) are untouched. The leak sweep now runs twice, the second time with an owner whose display name is an email address. It fails with the mask off, as does the history leg. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
5c53aae306 |
feat(oauth): a second canonical resource, with every token bound to one (TASK-3321 U2a) (#1715)
* feat(oauth): a second canonical resource, with every token bound to one (TASK-3321 U2a) Ruling (i) on TASK-3321: the ChatGPT catalog is served at its own MCP URL, which is a canonical OAuth resource of its own, and its tokens are not interchangeable with /mcp's. - config derives ChatGPTResourceURL from the /mcp resource: its MCP path plus /chatgpt, so https://mcp.getpad.dev becomes https://mcp.getpad.dev/mcp/chatgpt. - The authorization server issues tokens for AllowedAudience plus AdditionalAudiences. A request must name exactly ONE of them, and every requested audience must be the same resource. A request for both, or for a foreign one, is refused before a code is issued, and the client's hydrated audience must include the one requested. A request naming no resource is still bound to /mcp's, the documented default. - MCPBearerAuth checks the MOUNT's canonical. A route stamps it with WithMCPResource (never from the request); an unstamped route is /mcp's. A /mcp token is refused at the ChatGPT mount and the reverse. Nothing is mounted yet (U2b). The cross-refusal test covers both directions and the no-resource token, and goes red when the middleware ignores the mount or the strategy allows two resources. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(oauth): a refreshed token keeps its one resource (TASK-3321 U2a review) Codex confirmed fosite's refresh flow restores the original granted audience; this pins it end to end: refreshing a ChatGPT token (with no resource, or asking for /mcp's) yields a token refused at /mcp and accepted at its own mount, and the test fails if no refresh succeeded at all. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
d3b1209d04 |
feat(mcp): ChatGPT responses are minimized through per-tool allow-lists (TASK-3321 U3) (#1713)
* feat(mcp): ChatGPT responses are minimized through per-tool allow-lists (TASK-3321 U3)
Ruling R2: ChatGPT tool responses carry refs, titles, statuses, fields,
names and user-meaningful dates, and no UUIDs, emails, counters or
internal codes. Before this, the workspace overview carried the caller's
email, every full item carried assigned_user_email, and search, create,
update, restore and get_playbook returned raw items.
Each tool's response goes through an ALLOW-LIST shape in the ChatGPT
handler: a key nobody listed is dropped, so a new field cannot reach
ChatGPT by default. The text fallback is rewritten from the projected
value. Relation values in fields become refs, and an id no ref replaced
is dropped. Comment id and parent_id stay as reply_to handles, and seq
stays for expected_seq. list_collections' schema is parsed to an object.
A plan-limit refusal loses its upgrade text and billing link.
Surface fixes: ready_items drops a limit the server ignored; item_history
no longer claims "who"; archive_item answers {ref, archived: true}; the
set_workspace exclusion reason is corrected (#1710 wired the bootstrap).
/mcp is unchanged: its goldens do not move.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* fix(mcp): ChatGPT errors and non-JSON results are minimized too (TASK-3321 U3 review)
Codex round 1, four P2s:
- Error results passed through unprojected. They now keep only code,
message, hint, field, expected and workspace slug and name, with ids in
the prose masked; a source Go error becomes a plain failure. A raw
validation hint that quotes an unresolved id was the case that
reproduced it; the error-path test fails without the projection.
- A non-JSON success passed through. It now fails closed.
- archive_item echoed the caller's ref even when it was an id. An
id-shaped ref is no longer echoed.
- The sweep now checks the text fallback with the same rules, and covers
error results.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* fix(mcp): mask ids in an error's field and expected too, and pin available_workspaces (TASK-3321 U3 review)
Codex round 2: field and expected were kept but not masked, and nothing
pinned that available_workspaces (slug, name, default) survives the error
projection. The envelope test now sets an id in every text member and fails
when only message and hint are masked.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
|
||
|
xarmian
|
0c0874dc10 |
feat(mcp): pad_item.history rows name who made the change (ToolSurface 0.63) (#1714)
Every pad_item.action=history summary row, on both transports, gains an omitempty actor_name: the display name of the user who made the change. Rows said only created_by "user" or "agent", although the server joins the name on every version read. Empty for rows with no user (recovery writes, versions from before migration 103). ADDITIVE, on the v0.53 and v0.28 grounds; no name, enum or param moved. The CLI's projection, which stdio returns as it stands, moves into toItemVersionSummaries so it can be tested. Each transport has a leg that fails without its copy. Requested by the lead as its own PR before TASK-3321 U2, so the ChatGPT item_history tool can say who changed an item again. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
bc2a982591 |
feat: every content change through the ChatGPT door is versioned first (TASK-3321 U1b) (#1711)
* feat(items): every content change through the ChatGPT door is versioned first (TASK-3321 U1b)
Ruling (b) on TASK-3321: ChatGPT's update_item is non-destructive because
every content change can be undone from History. An in-process context
marker (server.WithChatGPTSurface, unreachable from the wire) marks the
ChatGPT door. For a content write through it, the update handler sets
ForceVersion past the 1h throttle, labels the version "chatgpt", and sets
RefusePendingEdits: a tab's unflushed edits refuse the write with 409
content_pending_flush rather than be missing from the version and lost to
an undo. That refusal speaks to a chat user ("being edited in Pad right
now; ... Try again shortly") and names no API override. Field-only updates
write no version. A plain API body may not claim the chatgpt label. With a
tab open and caught up, the version is written through BUG-3327's
ExternalContent.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* feat(mcp): the ChatGPT catalog marks its door, so update_item versions every content change (TASK-3321 U1b)
Every ChatGPT tool call now reaches its source handler with
server.WithChatGPTSurface on the context. The HTTP dispatcher builds its
in-process request from that context, and the item update handler saves a
version before every content change from that door. The equivalence test
now checks the one intended difference: every ChatGPT dispatch carries the
marker (all 21 tools fail without it) and no /mcp dispatch does.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test(items): a collab-snapshot flush cannot wear the chatgpt label; simplify the strip (TASK-3321 U1b review)
Codex round 1 suspected a ?source=collab-snapshot flush could keep a
body's chatgpt label. It cannot: the collab-snapshot branch overwrites the
label first. That made the strip's collab guard dead code, so it is
removed, and the case is pinned by a test leg.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
|
||
|
xarmian
|
3901101c81 |
fix(mcp): remote pad_set_workspace embeds the workspace bootstrap (BUG-3326, ToolSurface 0.62) (#1710)
* fix(mcp): remote pad_set_workspace embeds the workspace bootstrap (BUG-3326) registerRemoteMCP passed no BootstrapFetcher, so over /mcp the tool returned only a not_persisted note: remote clients started sessions without the conventions and playbooks stdio clients get. HTTPResourceFetcher gains Bootstrap, delegating to its existing in-process bootstrap resource read, which runs as the token's user through the dispatcher's scope/consent perimeter. registerRemoteMCP builds the fetcher once and hands it to both Register and RegisterResources. Tests go through wireMCP over /mcp with PAT bearers: the embed is present for a member, and absent for a non-member while the owner of that workspace gets it. Co-Authored-By: Claude <noreply@anthropic.com> * chore(mcp): ToolSurfaceVersion 0.62 for the remote set_workspace bootstrap embed (BUG-3326) ADDITIVE bump (v0.53 / v0.28 grounds): remote pad_set_workspace gains the `bootstrap` member stdio always carried. No name, enum or param moved; the tools/list goldens are unchanged, the initialize goldens move only the version and the instructions header. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
xarmian
|
39a5259551 |
feat(mcp): the ChatGPT catalog, a one-tool-per-operation projection of /mcp (TASK-3321 U1, stacked on #1707) (#1708)
* feat(mcp): the ChatGPT catalog, a one-tool-per-operation projection of /mcp (TASK-3321 U1) A second tool surface for OpenAI's plugin directory: 22 tools, each naming the /mcp (tool, action) it stands for and executed by that tool's own fan-out handler, so validation, dispatch and every server rule are the ones /mcp runs. An entry adds only its outward shape: a name, a description, a narrowed parameter list (types and enums from the source), its own hints, Fixed inputs (get_item always asks for the agent view), and the irreversibility justification the submission form asks for. Guards: - every /mcp operation and pad_set_workspace is an entry or an exclusion with a reason, never both or neither, with no stale names; - the hints match the ruling, written out independently; - the validator refuses each defect it exists for; - every exposed param has a ChatGPT-own description, and no description uses /mcp's action vocabulary; - each ChatGPT tool dispatches exactly as its source (CLI path, args and dispatch input) through a recording dispatcher over the real cmdhelp doc; - an undeclared param is refused before dispatch; - a new golden pair pins the ChatGPT initialize and tools/list. /mcp is byte-identical: its golden files do not move. NewServer gains optional Instructions and Experimental overrides, defaulting to /mcp's. Not mounted: U2 serves it at its own URL behind a setting, after U1b (update_item's forced version row, which its description already promises) and U3 (response minimization). Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(mcp): ChatGPT catalog keeps field number literals; set_workspace is an exclusion (TASK-3321 U1 review) Codex round 1: - P1: the projection re-marshalled the DECODED arguments into RawArguments, so a fields number had already lost its literal (9007199254740993 -> ...992, 1.10 -> 1.1) before the source handler's BUG-3217 path. The caller's raw bytes now go through, with the action and Fixed inputs spliced in; a test fails on the old code. - P2: set_workspace promised an overview it cannot give. On the shared server it persists nothing, and the remote transport has no BootstrapFetcher, so it returns only a not-persisted note. It is now an exclusion with that reason, and get_workspace_overview is the connect step. - P2: link_type routing among the link commands is one disclosed operation (relate two items); noted at the entry. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(mcp): every ChatGPT tool but list_workspaces requires the workspace (TASK-3321 U1) There is no session workspace on the shared ChatGPT server, so an optional slug would fall back to nothing. Pins the lead's check on #1708; red when item_history's workspace is made optional. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
eb987af709 |
feat(mcp): pad_workspace destructive + open-world (invite emails), pad_playbook open-world (TASK-3321 U0b, ToolSurface 0.61) (#1707)
* feat(mcp): pad_workspace is destructive and open-world, because invite sends email (TASK-3321 U0b) pad_workspace advertised destructiveHint:false (all writes "additive") and every tool advertised openWorldHint:false. But its invite action sends an email to an address the caller names: a send cannot be taken back, which is what the MCP spec and OpenAI's plugin review mean by destructive, and an arbitrary recipient is outside the server's own data, which is what openWorldHint means. invite leaves additiveWriteActions. A new openWorldActions set, guarded by a stale-entry test, sets openWorldHint per tool. The wire goldens move for pad_workspace only, on both transports. ToolSurfaceVersion is now 0.61 (an annotation-only behaviour bump on the v0.20 grounds). Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * feat(mcp): pad_playbook is open-world, because match sends text to the decision provider (TASK-3321 U0b review) Codex round 1, P2: pad_playbook.match posts the caller's free text to the configured decision provider, a third-party model API. It stays read-only and non-destructive, but it is open-world. Open-world is about where data goes, not whether anything is written, so the stale-entry test no longer forbids a read in openWorldActions. The goldens now move for pad_workspace and pad_playbook only, and the 0.61 entry names both. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
cb99879d53 |
feat(materialize): recover unsaved editor sessions into items.content (TASK-2198 U4) (#1683)
* feat(materialize): recover unsaved editor sessions into items.content in the background (TASK-2198 U4) Wires the U3 supervisor in. A collab room reclaimed at the end of its grace TTL with content-bearing rows above the flush watermark (T3), or a dormant pending item found by a one-minute sweep (T1), is queued; one worker goroutine materializes the item's whole content-bearing op-log with the whole-workspace link index and lands it through store.MaterializeFlush, under the collab item lock with no room open. The write is a flush: cursor == MAX(op-log id), rows pending, no set-aside rows, checked in the write's own transaction; watermark to the cursor only; seq bumped; one version row attributed to system/recovery; the op-log is never pruned. PAD_MATERIALIZE=off disables it entirely. A per-item in-memory failure budget (3 failures, doubling backoff, reset by a new op-log row) keeps untrusted input from being retried in a loop. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): recovery watermark is the cursor, not MAX at UPDATE time (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): a second recovery still writes its own version row (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * docs(mcp): op-log recovery bounds the stale-body window; tool surface v0.60 (TASK-2198 U4) content_state applied_pending_flush now clears on its own after the item's room closes, unless recovery fails or PAD_MATERIALIZE=off. Every sentence that called the window unbounded, or said nothing server-side moves op-log content into items.content, is corrected (CLAUDE.md, instructions.md, README, docs/deployment.md, docs/backup.md, code comments). BEHAVIOR bump to 0.60; goldens regenerated. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): count ErrNoMemoryCap silently against the per-item budget (TASK-2198 U4) U3 now refuses every job with ErrNoMemoryCap when no memory cap can be established, and warns once itself. The recovery worker counts it like any failure but logs it at debug, not per item. Its log lines go through a logger seam so the test can tell. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * style: gofmt (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * feat(materialize): one log line per budget exhaustion; count set-aside skips on the sweep line (TASK-2198 U4) An item that uses up its failure budget logs exactly one WARN naming the item, the failure count and the last error kind (DEBUG when the kind is no_memory_cap, which the supervisor already warned about once), and logs again only after op-log growth resets the budget and it is exhausted anew. The schema-version skip goes through the same line. Items skipped for set-aside rows are counted and reported on a new single sweep summary line, written only when it has news; never per item. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): keyset-page the recovery sweep so no candidate starves (TASK-2198 U4, codex r1) The sweep read the oldest 200 candidates from the start every tick, and the room check and the failure budget filtered them afterwards. Once 200 sat in backoff or exhausted, every sweep returned the same 200 and the 201st was never queued: filtering above a bound is starvation. Each tick now reads one page strictly after an in-memory keyset cursor (newest op-log created_at, item id) and wraps to the start after a short page, so every candidate is offered within ceil(N/limit)+1 ticks, whatever sorts ahead of it. Keyset over NOT IN: no parameter-count limit, and it also covers the room check, which an exclusion list would not. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(materialize): the sweep cursor wraps; docs for the exhaustion line, the set-aside count and paging (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * docs(deployment): reflow the recovery budget paragraph (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(materialize): recovery strips the follows-title marker as a tab's save does (TASK-2198 U4, BUG-3315) Since #1688 a tab loads a link that follows its target's title with the follows-title marker in the link's title attribute, so the marker lives in Y.Docs that recovery replays. Three fixture cases (a follows-title link; the same link after a rename; a URL edited to an external address) are generated from documents that really hold the marker, and the existing 8 are unchanged. A new leg recovers each through the production path (whole-workspace index) and through the runner with a restricted and an empty index, and asserts no 'pad-follows-title' reaches the body. It goes red in all 7 ways when the bundle's marker strips are disabled. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): a recovery version row carries #1691's history data (TASK-2198 U4) Rebased onto PLAN-2348 U2 (#1691), which adds user_id, line counts and a create marker to version rows. Recovery writes its row through the same UpdateItem insert, so it gets them automatically: no user_id (a system row), not a create, and line counts equal to diff.LineCounts of the stale and recovered bodies. The leg pins that. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
d78ff9bdc5 |
feat(materialize): worker supervisor with per-OS memory cap (TASK-2198 U3) (#1682)
* feat(materialize): parent-side worker supervisor with per-OS memory cap (TASK-2198 U3) The Supervisor runs materializer jobs in ONE long-lived child process (`pad __materialize-worker`), started lazily on the first job and fed jobs serially. A job past PAD_MATERIALIZE_TIMEOUT (default 2s) kills the child (goja's Interrupt, sent as timeout_ms a tenth earlier, is only the soft layer); a dead child is respawned on the next job after an exponential backoff (1s doubling, capped at 5 min, reset by a child that lived a minute). Every stop is logged with its reason (deadline, memory, exit, protocol, start, canceled, closed); every start logs the effective timeout, memory limit and cap mechanism. The child's stderr goes to the log, pid-tagged and bounded, never near the protocol. A job whose child died is never retried here. Memory cap (PAD_MATERIALIZE_MEM_LIMIT, default 2GiB, bytes or KiB/MiB/GiB, clamped to 1536MiB-16GiB): - Linux: RLIMIT_AS set on the child by prlimit before its first byte of input; the child only runs its own startup uncapped. - Windows: Job Object with JOB_OBJECT_LIMIT_PROCESS_MEMORY; the child is created suspended, assigned, then resumed: no uncapped window. - macOS: parent-side RSS watchdog every 100ms (proc_info syscall, ps fallback). - Other: deadline kill only. Not wired into the server yet (U4). Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * ci(materialize): run the supervisor on the real macOS and Windows caps (TASK-2198 U3) native-smoke now builds the real materializer bundle (as the go job does) and runs the supervisor, memory-cap and deadline tests on macos-latest (proc_info RSS watchdog) and windows-latest (suspended start + Job Object). PAD_MATERIALIZE_TEST_NO_SKIP turns the cap tests' skips into failures there, so a cap that stops running cannot pass as skipped. Tests: a real Windows processGone (OpenProcess + GetExitCodeProcess), RSS samplers for darwin (the production proc_info path) and Windows (K32GetProcessMemoryInfo), and an allocation helper that writes noise into every byte so macOS memory compression cannot hide it from the watchdog. The worker-loop tests share one loaded Runner through serveWorker (RunWorker's loop, split out); TestWorkerRoundTrip still goes through RunWorker end to end. go test -race -count=3 on the package: 714s -> 413s. docs: the materializer section says the worker and its knobs are inert until U4 wires it in. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): a worker never outlives its supervising process (TASK-2198 U3, codex r1) If the server died mid-job (SIGKILL, OOM kill, crash) its worker ran on with no deadline kill and, on macOS, no memory watchdog. Two layers: - Linux: Pdeathsig = SIGKILL. The fork runs on a fresh goroutine (startUnlocked), never locked to its thread, so it cannot land on a thread the runtime later retires (golang/go#27505); Go's forkExec re-checks getppid if the parent is already gone. - Every Unix (the only layer on macOS): the worker command calls ExitWhenOrphaned before the bundle loads, polling its parent pid every 200ms against PAD_MATERIALIZE_PARENT_PID and exiting (status 70) when it changes. It writes nothing. - Windows was already covered by the Job Object's KILL_ON_JOB_CLOSE. TestSupervisorOrphanedWorkerExits kills a helper process that supervises a worker hung in a job and requires the worker gone within 1s: 0s on Linux, 105ms with the ppid watch alone; with both layers removed the worker outlives its parent by 5s. It matches native-smoke's TestSupervisor.* selection, so it runs on macOS and Windows too. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): relative memory cap, refuse when uncappable, GOMAXPROCS=2; nix-safe ps test (TASK-2198 U3) The absolute Linux RLIMIT_AS failed on CI: a 64 MiB allocation died under a 1600 MiB cap, because the runtime there reserved more address space than on the machine the floor was measured on. The cap is now RELATIVE on every OS: once the worker has loaded and before it is sent any job, its usage is read as a baseline and the cap is set to baseline + PAD_MATERIALIZE_MEM_LIMIT — address space on Linux (/proc/<pid>/stat vsize, prlimit, read back), committed memory on Windows (the Job Object's per-process limit, set after the suspended start and load), resident memory on macOS (the watchdog's threshold). Every start logs baseline, limit and effective cap. Lead ruling, REFUSE: if the baseline probe fails or is implausible (0, or above MaxPlausibleBaseline = 8 GiB, ~4.8x the largest measured 1704 MiB), onBaselineProbeFailure kills the worker without sending it the job and fails the job with ErrNoMemoryCap; one WARN until a probe succeeds again. An OS with no cap mechanism never starts a worker and refuses every job with ErrNoMemoryCap, one WARN naming runtime.GOOS. The worker runs with GOMAXPROCS=2: measured loaded-worker baseline 1357 MiB vs 1597 MiB at this host's default (8) and 1704 MiB at 64. MinMemLimit drops from 1536 MiB (an artefact of the absolute cap) to 256 MiB: the corpus grows the worker by at most +10 MiB of address space and +18 MiB resident, and a 4,000-deep nested document by +129..194 MiB. A cgo worker's "pthread_create failed" at the cap is now classified as a memory death. Nix: TestParsePSRSS parses fixed strings only; the live `ps` leg is TestPSRSSLive and skips when there is no `ps` (the Nix sandbox has none; ps is only macOS's fallback), unless PAD_MATERIALIZE_TEST_NO_SKIP is set. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): Windows kills on the job's memory notification; steady thread set before the baseline (TASK-2198 U3) Windows (CI on c1866957): the Job Object refused the 1 GiB commit (VirtualAlloc errno=1455), but the worker did not die and the job ran to its deadline. The cap no longer depends on the runtime choosing to crash: the job is associated with an I/O completion port, and JOB_OBJECT_MSG_PROCESS_MEMORY_LIMIT for the worker's pid kills it at once (ErrMemoryLimit). errno=1455 on stderr is a second signal. On every OS a cap mechanism that sees the limit hit calls onLimit, and the kill is reported as "reached its <mechanism> cap". Linux (CI's cgo test binary at GOMAXPROCS=64): threads created after the baseline ate the growth budget. The worker now brings its thread set to steady state before readiness (warmThreads: an allocating goroutine per P, then a GC). Measured on a cgo build: threads added after the baseline are 0 at GOMAXPROCS=2 with or without it; at 64, +3 threads and +272 MiB of address space during the corpus without it, +0 and +0 MiB with it. 256 MiB stays the floor (largest growth at GOMAXPROCS=2: +193 MiB, a 4,000-deep nested document; the corpus adds +0 MiB). The cap test's second leg no longer uses GOMAXPROCS=64 (which also changed thread creation) to stand in for a bigger host: the helper inflates its own baseline before readiness, with never-touched address space on Linux, committed memory on Windows and resident memory on macOS. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * feat(materialize): the worker stops after an idle timeout; the next job restarts it (TASK-2198 U3) Dave's ruling on the worker process: GO, with an idle exit. A loaded worker holds ~200 MB resident, which an install that rarely materializes should not pay for all day. PAD_MATERIALIZE_IDLE_TIMEOUT: default 5m, clamped to 30s-24h, 0 = never. After each job the supervisor (re)arms a timer. When it fires, the stop runs only while holding the job slot, so it never takes a worker with a job in flight. It re-checks the idle time, forgets the worker WITHOUT recording a death (backoff is untouched), and closes the worker's stdin so its read loop ends with status 0, with a kill after a grace if it does not go. The stop is logged at INFO, reason=idle. A job that arrives meanwhile waits for the slot, finds no worker, and starts a new one with no backoff wait. Legs: the idle exit after the timeout (INFO, status 0, no WARN); the next job respawns and succeeds with no wait; an idle stop leaves the backoff streak alone (crash, idle, crash waits 2s, not 4s); a job arriving during the stop runs on a new worker; a timer that fires mid-job does not stop that worker; NeverIdle arms no timer; env parsing and clamps. Mutations, each red on its own: recording a death (3 legs), logging as a kill (1), not holding the slot (1), never arming (3). Docs: the knob beside the other two, and `idle` in the stop reasons. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): Close does not return while an idle-stopping worker still runs (TASK-2198 U3, codex r1) An idle stop forgets the worker before it has exited, so a Close in that window found no worker and returned with it still alive. The supervisor now tracks the worker an idle stop is ending, and Close kills and reaps it. A leg pins it, and goes red without the fix. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): an idle stop that Close ended says so (TASK-2198 U3, codex r2) The idle stop stored its kill reason unconditionally, so it overwrote the 'closed' of a Close that had already killed the worker, and its stop line said idle. It now claims the reason only if none is set, and the line reports reason=closed. The Close-during-idle-stop leg asserts it. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
a61f50421f |
fix: a move refused for a value only the caller can supply gets a way forward on web, CLI and MCP (BUG-3200) (#1690)
* fix(web): a pane-menu move refused for a value only the user can supply opens the copy dialog's picker, preset to that collection (BUG-3200) * fix(cli): item move refused as missing_required_fields prints what each field needs, from the copy preflight (BUG-3200) * fix(mcp): a remote move refused for a destination value names the field parameter that supplies it (BUG-3200) * fix(web): fence and classify the copy dialog's preset continuation; the handoff is behind stillOnSource (BUG-3200, full vitest) * test(web): the handed-off move keeps the item's id and ref and sends no copy (BUG-3200, lead check) |
||
|
xarmian
|
fa2755e183 |
feat(materialize): headless op-log materializer bundle, goja runner and worker (TASK-2198 U2) (#1678)
* feat(materialize): headless op-log materializer bundle, goja runner and worker (TASK-2198 U2)
Builds the editor's own JS (Yjs -> y-tiptap -> Tiptap schema -> tiptap-markdown
-> the tab's flush pipeline) into one IIFE, embeds it, and runs it in goja. Not
wired into the server yet (U3 supervisor, U4 write path).
- web/src/lib/collab/materializer/: bundle entry (extension list mirrors
Editor.svelte's collab branch via the U1 shared modules), op-log replay
mirroring wsProvider's MESSAGE_SYNC arm, linkedom DOM shim and polyfills
from the spike, and a vendored patch of prosemirror-markdown
MarkdownSerializerState.atBlank (quadratic under goja; equivalent).
- web/scripts/build-materializer.mjs (esbuild) -> web/build-materializer/,
run by `npm run build`; embedded as pad.MaterializerJS.
- internal/materialize: Runner (Go-native atob/btoa/TextEncoder/TextDecoder/
getRandomValues, schema-version check, ctx interrupt with a clean VM after),
and RunWorker's length-prefixed JSON protocol behind the hidden
`pad __materialize-worker` command.
- Parity: vitest drives two real Editor.svelte mounts and compares
materialize() with the tab's flush on 50 seeded cases; the same cases are
committed as testdata/corpus.json and replayed in goja (TestCorpusParity).
- CI Go jobs, release tests, Docker and Nix produce the bundle; linkedom and
esbuild are new devDependencies.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test(materialize): pin the atBlank patch's equivalence under V8 too (TASK-2198 U2, codex r1)
Codex r1 claimed upstream's /(^|\n)$/ is true for a trailing CR in JS. It is
not (checked in V8: false for "\r" and "a\u2028", as the patch returns),
but the only equivalence test ran under goja and the patch header cited a
V8-side test that did not exist. This adds it, with the line-terminator edge
cases, and it fails when the patch checks CR instead of LF.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* build(nix): vendorHash for goja (TASK-2198 U2)
Taken from the Nix job's recompute on
|
||
|
xarmian
|
a2e7768035 |
fix(comments): a deleted comment with replies leaves a tombstone (BUG-3252) (#1675)
* fix(comments): a deleted comment with replies leaves a tombstone (BUG-3252) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(comments): backdate so a tombstone moving updated_at is seen (BUG-3252) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * chore(mcp,store): v0.59 instructions heading, NUL baseline for comments.deleted_at (BUG-3252) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(comments): reactions wait for a tombstoning delete; unreact on a tombstone answers 409 (BUG-3252, codex r1) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(comments): unreact racing a tombstoning delete answers 409, not 404 (BUG-3252, codex r2) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
a90f75fde2 |
fix(server): sign protected share-page images on every install (BUG-3305) (#1673)
* fix(server): sign protected share-page images on every install (BUG-3305)
A password or require-auth share link on a self-hosted instance showed a
placeholder for every embedded image. The key that signs those image refs
was set only in the cloud block, and only with PAD_MCP_PUBLIC_URL set, so
it was nil on every self-host, and the mint side omitted the refs.
The key is now set right after server.New on every install, from the
deployment key every running install has (EnsureEncryptionKey generates
one on SQLite and refuses to start Postgres without one). The MAC stays
domain-separated from the key's other uses. The cloud-only site is gone.
Deliberate behaviour change: after upgrade, self-hosted protected share
pages render their images. So does a cloud instance without
PAD_MCP_PUBLIC_URL, which had the same gap.
Comments that gave the old reason for a missing key ("self-host without
an encryption key") are corrected; the no-key branch stays as the
defensive floor, with its tests.
Evidence: a live boot of the built binary as a plain self-host. A
password share resolves with a signed image ref; the signed asset
answers 200 and the unsigned one 404. The same run on main's binary has
no ref.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
* docs(server): say how the share-asset MAC stays apart from the key's other uses (BUG-3305 codex r1)
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
|
||
|
xarmian
|
b510d67696 |
fix(oauth): sweep expired token, code and PKCE rows (BUG-3301) (#1671)
* fix(oauth): sweep expired token, code and PKCE rows (BUG-3301) Nothing deleted OAuth rows by age. Each refresh leaves the chain's previous access and refresh rows inactive and inserts a fresh pair, measured at +1 access and +1 refresh row per refresh (48 rows a day for one active client at the 1 h access lifetime), forever, on Pad Cloud and on any https self-host that turns MCP on. The token reaper now also runs SweepExpiredOAuthRows every hour. A row goes once its requested_at (its own issuance, measured) is older than its token's lifespan plus 24 h: - refresh and authorization codes: the refresh lifespan. A replayed rotated token or a reused code keeps revoking the family for as long as the token could have been valid, because fosite checks "inactive" before "expired". - access: the access lifespan. Nothing reads an expired access row. - PKCE: the code lifespan. Only abandoned authorizations leave one. The lifespans come from the running OAuth server (defaults when OAuth is not built). No row a live chain or a concurrent exchange could use is ever below a cutoff. Deletes run 500 rows per statement with a 20 ms pause between batches. Measured on SQLite with 50,000 rows and a concurrent token write every 5 ms: with no pause, one write waited 1.55 s; with the pause, the worst write took 52 ms. The per-tick cap is 200 batches per table. Connected Apps and the MCP resume count now drop a chain whose refresh token has been expired for the grace period, so their "upper bound" comments say how far. The deployment guide's "never swept" gap is gone. Tests: store cutoffs, zero cutoffs, and batches with the cap, on SQLite and Postgres. A server end-to-end test through /oauth/token: the live chain survives and still refreshes; an aged-out chain leaves Connected Apps and the resume count; a swept token is refused without revoking the family; a retained rotated token still revokes it. Mutants: inverted age comparison (store tests fail), refresh cutoff at now (e2e fails: the live chain's refresh answers invalid_grant). Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz * fix(oauth): code retention adds the code lifespan; Capped means "may remain" (BUG-3301, codex r1) - A code's requested_at is when it was authorized, but the first refresh token it yields starts at the exchange, up to a code lifespan later. The code cutoff now adds the code lifespan to the refresh lifespan, so reuse detection lasts as long as that refresh token could be valid. The comment no longer claims "the chain's lifetime": a chain renews forever. - Capped says the cap was used, so rows MAY remain; the last full batch may have been the end. Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz * fix(oauth): a non-expiring lifespan skips its tables (BUG-3301, codex r2) fosite reads a negative lifespan as "never expires"; the cutoff math turned -1 into "now minus the grace", which would delete a live non-expiring token after a day. A table whose retention depends on a negative lifespan now gets a zero cutoff, which the store skips (codes depend on both the code and refresh lifespans). Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz |
||
|
xarmian
|
22bfe8cc89 |
feat(mcp): off-cloud Host allowlist on the non-API MCP paths (TASK-2319, PLAN-2310 U3) (#1665)
* feat(mcp): off-cloud Host allowlist on the non-API MCP paths (TASK-2319, PLAN-2310 U3) Off cloud, /mcp, /oauth/* and /.well-known/oauth-* answer 421 Misdirected Request (JSON) when Host does not name the configured origin, MCP URL or auth-server URL. Matching is case-insensitive with the port defaulted from the configured scheme; a loopback origin admits every loopback spelling on its port; an empty Host is refused. It runs after the availability gate (MCP off still answers 404 whatever the Host) and before auth, audit and the OAuth group's rate limiter. The /api/v1 routes are not wrapped: they keep the API perimeter, and the in-process dispatcher sends them an empty Host. Cloud is unchanged. mcp-go's DisableLocalhostProtection stays, with its comment rewritten. The e2e harness sets PAD_URL to its base URL. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * feat(mcp): honour X-Forwarded-Host from trusted proxies; 421 names the fix (TASK-2319 lead review) Behind a proxy that rewrites Host to the backend's name (Azure Application Gateway and App Service, some ingresses) every MCP request would get a 421. The allowlist now checks the first X-Forwarded-Host entry when the TCP peer is in PAD_TRUSTED_PROXIES, the same trust test cliAuthScheme and TrustedProxyRealIP use; from any other peer the header is ignored, so DNS-rebinding protection is unchanged. The 421 names the received host and its header, the configured origin, and the proxy fix, and the first refusal per host is logged at WARN behind one overall limiter (5, then 1 a minute; at most 1024 hosts remembered). Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 |
||
|
xarmian
|
ae7f77d9d8 |
feat(mcp): construct MCP off cloud, mount always, gate per request (TASK-2317, PLAN-2310 U2) (#1663)
* feat(mcp): construct MCP off cloud, mount always, gate per request (TASK-2317, PLAN-2310 U2) The remote MCP capability (transport, in-process dispatcher and resource fetcher, audit writer, session tracker) is now constructed on every install by wireMCP, and the OAuth server, claim secret and connections backfill whenever the resolved auth-server URL is https. Nothing is served until Server.mcpAvailable() / oauthAvailable() say so, per request: every DR-5 route is mounted always and answers the JSON 404 while unavailable, before auth, audit or rate limiting. The two audit-history routes are unchanged. - Discovery takes its URLs from configuration only. The four "https://"+r.Host fallbacks and rewriteMcpSubdomain are gone; with nothing configured the documents answer 404, and the /mcp 401 is a bare Bearer realm="pad" unless OAuth is available. - resource_metadata follows RFC 9728 section 3.1, so a resource with a path (a self-host's <origin>/mcp) points at a route that exists. Byte-identical for a path-less resource (cloud). - An explicitly set PAD_MCP_PUBLIC_URL / PAD_AUTH_SERVER_URL keeps its historical spelling (TrimRight "/"), so the OAuth audience of every existing deployment is byte-identical and issued tokens keep validating. - Protected share-link image signing gets its own key field, set exactly where the claim secret used to be (cloud with PAD_MCP_PUBLIC_URL), so building OAuth on an https self-host changes nothing on share pages (lead ruling B; BUG-3305 is the deliberate fix). - The session and setup payloads keep mcp_public_url cloud-only until the connect modal gains its PAT path (DR-8, unit 6). - The claim routes answer 404 instead of 412 claim_disabled where OAuth is not available; the auth-server metadata answers 404 instead of 503. Tests: the DR-5 route table across the eight acceptance states and the upgrade test (no mcp_enabled row, https origin: every gated route 404, the audit routes unchanged, a valid dispatcher claim refused; the control leg reaches every route and the same claim succeeds), both over the production wiring; an https self-host OAuth flow for audience <origin>/mcp; a PAT call over http that writes an audit row. Release gate (PLAN-2310): must not ship without units 3, 4 and 7. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * docs(mcp): the availability gates run inside the API perimeter on /api/v1 routes (TASK-2317 codex r1) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 |
||
|
xarmian
|
eb167bea00 |
feat(mcp): MCP addressing from config; mcp_enabled setting with an env lock (TASK-3302, PLAN-2310 U1) (#1662)
* feat(mcp): resolve MCP addressing from config; mcp_enabled setting with an env lock (TASK-3302)
PLAN-2310 unit 1 (DR-2, DR-3, the data behind DR-7). No route changes:
nothing is gated or exposed by this unit, and U2 wires the predicate
into the routes.
Addressing (internal/config/mcp.go, ResolveMCPEndpoints):
- The origin is cfg.URL (PAD_URL, toml url, --url), else PUBLIC_URL,
never the Host:Port fallback and never a request's Host.
- The MCP URL, which is also the OAuth audience, is PAD_MCP_PUBLIC_URL,
else <origin>/mcp. The auth-server URL is PAD_AUTH_SERVER_URL, else the
origin.
- A value must be an absolute http(s) URL without user info, query or
fragment; an origin also has no path. An unusable value never fails
startup (PAD_URL also serves the CLI). It is logged, leaves its field
empty, and keeps MCP unavailable. An unusable override does not fall
back to the origin.
- HTTPS() is the auth-server URL's scheme: https means OAuth plus PATs,
http means PATs only.
The setting (internal/server/handlers_admin_mcp.go):
- platform_settings.mcp_enabled, default off, read per request with no
cache (the webmcp_enabled precedent). A read error counts as off.
- PAD_MCP_ENABLED (true/false/1/0/yes/no/on/off) forces it and locks the
toggle. An unparseable value forces nothing and is logged. Cloud is
always on.
- mcpAvailable() is DR-1's predicate: cloud, or the setting on and the
addressing usable.
- GET/PUT /api/v1/admin/mcp, admin-only. GET reports enabled, source
(setting|environment|cloud), locked and the readiness: origin, MCP URL,
auth-server URL, https, auth methods, problems, state on|off|blocked
with the reason, and the resume counts. PUT answers 409
set_by_environment under the env lock (the decision-provider shape),
403 managed_by_operator on cloud, and 400 without {enabled}. It is
audit-logged as settings_changed.
- The resume counts (internal/store/mcp_resume.go) count live OAuth
chains the way the Connected Apps page does (request_id with an active
access or refresh row), and PATs the way /mcp accepts them (has a
user_id; expiry parsed as a time, as ValidateToken does).
Tests:
- config resolution table (11 cases); PAD_MCP_ENABLED parsing and Load.
- Store counts: a rotated chain counted once, a revoked chain excluded, a
refresh-only chain counted; the legacy and expired PATs excluded.
- Router tests: default off (the upgrade case); https on; http PAT-only;
blocked without or with an unusable origin; the env lock with a stored
value underneath; cloud; admin-only and body validation; a setting read
error is off, with a control leg.
- Mutants, each BUILD OK and killed by the test aimed at it: env override
ignored; availability ignoring addressing; PUT skipping the env lock;
origin path allowed; PUT allowed on cloud; read error counted as on
(error branch only); invalid override falling back; expired PATs
counted; refresh-only chains missed.
Docs for the variables and the endpoint land with PLAN-2310 U7.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
* fix(mcp): canonical MCP URLs; PAT count mirrors /mcp's user check; one setting read per response (TASK-3302)
Codex round 1 on
|
||
|
xarmian
|
237333681b |
feat(attachments): attach binds an unattached upload to an item, so its share renders it (TASK-2247) (#1652)
The July audit's share-page broken-image finding was fixed by BUG-2389 and
TASK-2637: the share byte endpoint serves variants of attachments the
shared item OWNS. What was left: an image uploaded with
`pad attachment upload -` (item_id NULL) and then embedded is never owned,
so its share shows the placeholder. A census of this workspace found 4
such refs out of 13, all from that path.
Lead ruling (option C): fix it explicitly, with no implicit binding on
content writes.
- POST /workspaces/{ws}/attachments/{id}/attach {item}. It binds a live,
unattached original in this workspace, and its live variants, to an
item. Every UPDATE requires item_id IS NULL, so an attached row is never
moved, and the bind is exclusive with the never-attached GC claim. The
item is pinned like CreateAttachmentForLiveItem.
- Authorization: the caller can see and edit the item, and uploaded the
attachment or is a workspace owner. An attached row answers 409 to a
caller who can see its owner, and the shared 404 to anyone else.
Restricted members and guests cannot read unattached rows, so they get
the 404. That closes the case where a guest binds an id they know to an
item they can edit and publishes it through its share.
- The attachment_attach capability. `pad attachment attach <id> <item-ref>`
refuses against a server that does not advertise it, because an older
build's bare 404 would read as "attachment not found".
- The `upload -` help text, the attachment group help, both skill copies
and CLAUDE.md say an unattached image does not render on share pages.
- No MCP surface: pad_attachment has no upload.
Tests: 12 server/store tests, including an end-to-end share leg (404
before, variant bytes after), and 9/9 mutants killed. CLI capability
gate: 2 tests, mutant killed. e2e: the share renders the image after
attach; with attach made a no-op it goes red at "attached: no
placeholder".
Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz
|
||
|
xarmian
|
c29af5731c |
test(mcp): pin the initialize and tools/list wire bytes on both transports (TASK-2306 unit 1) (#1610)
Golden files for the initialize and tools/list results, from the production bindings: the real `pad mcp serve` subcommand for stdio, and registerRemoteMCP behind NewRemoteTransport for remote /mcp. The remote registration is extracted from the cloud branch of serverStartCmd into registerRemoteMCP, a mechanical move, so the test drives the binding rather than assembling its own server. It turns 'behavior-neutral' into a checked claim for the alias/funnel work that follows and for any SDK bump. Re-injecting BUG-2302 (destructiveHint true on read-only tools) fails both transports on the exact line. serverInfo.version is the only normalised field. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 |
||
|
xarmian
|
9b7e0832ac |
fix(comments): deleting a comment that has replies answers 409 comment_has_replies instead of 500 (BUG-3252) (#1608)
* fix(comments): deleting a comment that has replies answers 409 comment_has_replies instead of 500 (BUG-3252)
comments.parent_id references comments(id) with no ON DELETE, so deleting a
parent with replies failed the foreign key inside the store. Every door said
"internal error", the web delete button included, and nothing was deleted.
Per the lead's ruling (reversible with Dave's veto), the store now refuses
by name. The DELETE carries a NOT EXISTS on replies, so the refusal is part of
the statement itself, and a zero-row result is told apart by counting.
Nothing is deleted. Every door reports it:
- Both server routes answer 409 comment_has_replies with details
{comment_id, reply_count}.
- The CLI root writes a structured marker, so stdio MCP reports the code
and hint rather than server_error.
- Remote MCP passes it through the 409 allow-list with the same hint.
- The web timeline shows the server's message in place of "An internal
error occurred".
MCP 0.58 (behavior bump, on the v0.45 grounds). Cascade versus tombstone
stays an open product decision; this refusal is the floor under either.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
* fix(comments): a delete whose replies vanish between statements retries instead of reporting not-found (BUG-3252, codex r2)
Under Postgres READ COMMITTED the conditional DELETE and the reply count take
separate snapshots, so a reply the DELETE saw could be gone by the count, and
a comment that still existed was reported missing. The zero-row branch now
rechecks that the comment exists and, if it does, runs the conditional
delete again, at most three times.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
* test(comments): pin the replies refusal on Postgres (BUG-3252)
The server and MCP tests run on SQLite, so the conditional DELETE had never
executed on Postgres. This test runs there (make test-pg). Against the bare
DELETE mutant it fails with comments_parent_id_fkey, which is the original
defect.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
|
||
|
xarmian
|
0f1069e9df |
feat(comments): agents can edit and delete comments from the CLI and every MCP transport (TASK-2695) (#1607)
Web users could edit and delete comments since PLAN-1662. Agents, who write
most comments, could only append corrections.
- Server: new item-scoped routes, PATCH/DELETE
/workspaces/{ws}/items/{ref}/comments/{id}. They answer 404 unless the
comment is on {ref}, then run the existing handlers unchanged (edit is
author-only, delete needs item-edit). GET /server/capabilities advertises
them as item_scoped_comment_writes.
- CLI: `pad item comment-edit <ref> <comment-id> [message|--stdin]` and
`pad item comment-delete <ref> <comment-id>`. Both refuse to send to a
server without the capability, whose bare 404 would otherwise read as
"comment not found". `pad item comments` now prints each comment's id,
plus "edited" once its body has changed.
- models.Comment serialises a derived `edited` bool, true when
updated_at > created_at (the web timeline's rule).
- MCP 0.57: pad_item edit-comment / delete-comment and a comment_id param,
on stdio, remote /mcp and WebMCP. All three use the item-scoped routes,
so the check that the comment is on ref happens on the server for every
transport. Remote packages a delete's 204 as the CLI's JSON.
Found along the way, and out of scope: deleting a comment that has replies
answers 500, because parent_id has no ON DELETE. Filed as BUG-3252.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
|
||
|
xarmian
|
2d51308d39 |
fix(collab): a schema-version rebuild sets unflushed edits aside instead of deleting them (BUG-3244 A) (#1598)
* wip(collab): the schema-version rebuild sets unflushed edits aside and the item stays marked (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip(server): set-aside rows refuse every content write and restore unless overwrite_pending_edits, which discards and reports them (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip: set-aside read/discard endpoints and pad item set-aside; census and migrated-table entries (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip(store): set-aside edits travel in the workspace bundle; the migrate gate asks the op-log only (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip: set-aside advice branches at every door that names a remedy; mark-only doors share one helper (BUG-3244 A, ruling 1) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(cli): set-aside notices name the right remedy (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * docs: the rebuild sets unflushed edits aside; freeze reason, API docs and op-log prose follow (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(web): identity-gate rows re-reviewed for the set-aside reason threading (BUG-3244 A) The edits in each covered unit are a pure pendingEditsReason(e) argument or string branch, or a reason passed to the dialog; no await, write or fence changed. Three anonymous-unit matchers follow the new call text, and the keepalive failure row lists the pure classifier in may. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(server): a restore override counts the set-aside rows it discarded (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * fix(store): the rebuild sets aside the rows its DELETE returned, so an append between choosing and clearing is not lost (BUG-3244 A, codex r1) On Postgres each statement takes its own snapshot and the setup lock does not stop a connected peer appending, so a row committed between the old INSERT ... SELECT copy and the DELETE was deleted without being set aside. The kept rows are now chosen from DELETE ... RETURNING. A test hook in the gap drives a Postgres leg; a mutant that chooses before the gap loses the row. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(web): playbook save row re-reviewed after the BUG-3230 U3 rebase (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * fix(store): the set-aside move's first statement is the DELETE, so SQLite never upgrades a stale read snapshot (BUG-3244 A, codex r2) Reading the watermark first took a read snapshot on SQLite; a concurrent append would then make the DELETE fail with SQLITE_BUSY_SNAPSHOT instead of waiting. The watermark is now read after the DELETE, where a later value is only more accurate. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * feat(mcp): ToolSurfaceVersion 0.56, content_state gains superseded_set_aside (BUG-3244 A) The value reaches pad_item get / full list, pad_playbook run and the item resource's line above the body, and its remedy differs from applied_pending_flush (opening the item will not restore the edits). An additive bump with a version.go entry; README, CLAUDE.md and the MCP instructions follow. Read/discard of set-aside rows stays off the catalog. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(server): after a bumped rebuild the next open is granted the seed once and the item stays superseded (BUG-3244 A x BUG-3240) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 |
||
|
xarmian
|
b459892d8b |
fix(search): the collection filter resolves exact-first per workspace in scope, so a collection named task is no longer shadowed by tasks (BUG-2659, BUG-2839) (#1576)
* fix(search): the collection filter resolves exact-first per workspace in scope, so a collection named `task` is no longer shadowed by `tasks` (BUG-2659, BUG-2839) /search matched its collection filter as a literal slug, so shorthand found nothing and both clients normalised it first, which rewrote `task` to `tasks` even where a `task` collection existed. The handler now resolves the filter with resolveItemCollectionSlug in every workspace in scope and filters the store on the resulting collection IDs; one slug cannot express that, since the same input can resolve differently per workspace. The capability endpoint advertises search_collection_resolution, and the CLI sends the name as typed only to a server that does. The in-process remote MCP mapper stops normalising outright. Also corrects cascadeTitleRename's doc comment, which named links.ReplaceTitle for a cascade that rewrites through links.RewriteBracketsAt (BUG-2839). * chore(mcp): tool surface 0.55 — pad_item.search's collection resolves exact-first per workspace (BUG-2659) Lead-ruled BEHAVIOR bump on the v0.36/v0.40 grounds: in a workspace holding both task and tasks, the same call now answers from task. |
||
|
xarmian
|
9059862431 |
fix(session): a headless disarm holds for as long as its session lives: an unverifiable owner fails closed and is kept, and the file names the session, not the exiting command (BUG-2771, BUG-3227) (#1574)
* ci(smoke): a headless disarm on Windows must survive later reads (BUG-2771) Added before the fix, so the first CI run shows it red against the pre-fix binary. * fix(session): an arm-state file whose session cannot be verified is kept and fails closed, so a disarm is never forgotten (BUG-2771) The unknown owner verdict (every headless file on Windows; on unix an unexaminable /proc entry or socket) was treated as dead: the next read reaped the file and auto_arm re-armed over an explicit disarm. It now resolves to LocalArmUnverifiable, which is not armed and not reaped. `pad session arm` reports the real state, `status` explains it, and a new `pad session reset` removes the kept file so auto_arm decides again. * ci(smoke): the headless-disarm step covers unix too, owned by the step's shell via PAD_SESSION_PID (BUG-3227) Pushed on top of the BUG-2771 fix but before BUG-3227's, so this run shows the unix path red pre-fix while Windows turns green. * fix(session): a headless arm or disarm is owned by the session process, not the exiting command, and a write no session can own is refused (BUG-3227) A headless arm-state file recorded os.Getpid(), the pid of the short-lived `pad session arm`/`disarm` itself, so on unix it was dead on the next read and reaped: an explicit disarm never held. It now records CaptureSessionOwner() (PAD_SESSION_PID, then CLAUDE_PID), as the session registry does. With no session process to name, the owner is the command, and arm/disarm refuse rather than report a consent change that did not happen. * docs(session): arm and disarm --help say how a headless session is identified, and that with no owner they exit non-zero (BUG-3227) |
||
|
xarmian
|
e422d6b183 | fix(cli,web): the CLI activity surfaces apply the change-pill display rule through a Go twin sharing one fixture with the web, and the admin audit log expands a row to its full record (BUG-2789) (#1570) | ||
|
xarmian
|
547527d7dc |
fix(server): Content-Disposition carries a non-ASCII or unquotable filename in filename* with an ASCII fallback, so NBSP and zero-width names no longer reach clients as Go escape text (BUG-3190) (#1569)
* fix(server): Content-Disposition carries a non-ASCII or unquotable filename in filename* with an ASCII fallback, so NBSP and zero-width names no longer reach clients as Go escape text (BUG-3190) * test(server): the Content-Disposition population carries characters the extended form must encode, and the doc comment survives gofmt (BUG-3190) |
||
|
xarmian
|
bd21b97c70 |
refactor(store): delete the NUL SUSPECT path, now that the predicate sees a NUL behind a repeated key (BUG-3220) (#1561)
The suspect class held pre-filter matches the NUL predicate cleared, and migrate-to-pg cast each one on the destination, because one shape in it (a NUL shadowed by a literal duplicate key) was invisible to every layer. Since BUG-2812 the token walk reports that shape as an ordinary violation and the ordinary repair fixes it, so the only suspects left were doubled-backslash literals, which PostgreSQL accepts. ScanNUL judges stored bytes, so the same holds for any earlier release's rows (the PostgreSQL 12/17 x UTF8/SQL_ASCII/LATIN1 matrix is on the trail). Deleted: NULSuspect and the Suspects bucket, the destination oracle (CheckJSONBAcceptable and its classification), RepairSuspectValue and its buckets, the preflight's destination cast (preflightNULForMigration no longer takes a destination), the suspect heading in scan-nul/repair-nul, the exported RepairJSONEscapes whose only other caller was the suspect repair, and the docs paragraph. MigratedTables and nulKeyPredicate move out of the deleted file. |
||
|
xarmian
|
c31fb06c86 |
fix(store): the migrate-to-pg preflight refuses a stored value that is not valid UTF-8, and scan-nul / repair-nul report and repair it (BUG-3222) (#1559)
* fix(store): the migrate-to-pg preflight refuses a stored value that is not valid UTF-8, and scan-nul / repair-nul report and repair it (BUG-3222) PostgreSQL refuses a text or jsonb value holding invalid UTF-8 (SQLSTATE 22021 under a UTF8 database); SQLite stores it silently. A legacy row of that class failed `pad db migrate-to-pg` partway through the copy, leaving the destination half-populated, and the NUL preflight never looked for it. The destination cast on NUL suspects caught a sliver of the class by coincidence (a value that also mentioned the escape), which BUG-3220 is about to delete. The NUL census now reads every protected column in full as a BLOB and reports invalid UTF-8 as a violation kind (merged with a NUL finding on the same row), so scan-nul lists it, repair-nul replaces each invalid sequence with U+FFFD, and the preflight refuses on it through the path it already had (migrated tables only; nothing written first). Measured on a copy of a 306 MB live database: scan-nul 0.51 s -> 1.52-1.62 s, peak RSS +10 MB. * fix(store): a value with both defects on an incomplete-key row is counted once (merged by rowid), and the JSON and SQLSTATE wording is exact (BUG-3222 codex r1) * test(store): no protected table may declare a column that shadows SQLite's rowid, which the NUL scan merges on (BUG-3222 codex r2) |
||
|
xarmian
|
7376023ff2 |
fix(server): the request-body gate reads tokens with jsontext, so a NUL behind a repeated key or beside 1e999 is refused, and a repeated request member is refused on every body (BUG-2812) (#1550)
* fix(server): the request-body gate reads tokens with jsontext, so a NUL behind a repeated key or beside 1e999 is refused, and a repeated request member is refused on every body (BUG-2812) The body gate and the store's JSON-column guard decoded into map[string]any and walked the tree. The tree is a lossy model: it keeps one value per repeated key, so a NUL in a shadowed copy passed (and the typed decode could merge it into storage), and a number no float64 holds made the decode fail, so the gate answered "no NUL" for a body the typed decode then accepted. Walking that tree also cost about 2x on any body that merely mentioned an escape. - internal/jsonscan wraps github.com/go-json-experiment/json/jsontext (pinned v0.0.0-20260820222146-c27c302e5fc3, BSD-3, The Go Authors). It is the module's only importer; its doc comment names the exit to stdlib encoding/json/jsontext. A differential test pins its acceptance to encoding/json v1's, which found that jsontext refuses a lone surrogate v1 accepts; that case falls back to v1's unquote. - scanRequestBody reads every body once, recording a NUL and a repeated request member separately and reading to the end, so malformed input still reports nothing. Repeats count in the request's own structure, and not in caller data under JSON-encoded keys (BUG-2896 collapses those). Names are case-folded where the destination is a struct: decodeJSONBytes reflects its destination's TOP-LEVEL type, so a struct folds its members, a slice of structs each element's, and a map (admin settings, admin user limits) none. Below the top level only an exact repeat counts. decodeJSONBytes and both bundle paths refuse either. - textguard's document walk and artifactIsBindableText read tokens too, so every layer refuses the shadowed NUL together: the one textguard.KnownGaps case moves into Corpus. - The store's suspect path stays, redundant but correct, by ruling; its deletion is its own unit. Its instruments now pin what is true: no refused value is a suspect, and the remaining suspects are the doubled-backslash class the destination accepts. - The map-model walk survives only as the differential's oracle. * fix(server): update vendorHash and the Postgres-only suspect instruments for the token scan (BUG-2812) The Postgres-backed legs of the suspect path fired when the shared predicate learned to see a NUL behind a repeated key: that value is now a violation everywhere, and the store's own guard refuses it before the destination cast. The preflight legs now assert that, the unmigrated- table fixture uses a doubled-backslash literal (still a suspect), and PostgreSQL's native refusal stays measured by TestNativePostgresAgreesWithTheCorpus. vendorHash is the value this PR's Nix job recomputed. |
||
|
xarmian
|
25d76af0ba |
fix(fields): field writes keep every number's literal, so an integer above 2^53 is no longer rounded by a write that never named it (BUG-3202) (#1545)
* fix(fields): field writes keep every number's literal, so an integer above 2^53 is no longer rounded by a write that never named it (BUG-3202) Every door that writes an item's fields decoded the blob into map[string]any, turning each JSON number into a float64, so the re-encode rounded integers above 2^53 (9007199254740993 stored as ...992). The stored blob was decoded that way too, so a fields_patch to `status` rewrote every large integer in the row. - models.DecodeFieldsJSON / DecodeJSONKeepingNumbers decode with UseNumber. They accept and refuse exactly what json.Unmarshal does (trailing data, out-of-float64-range numbers); only the representation changes. - Used at every fields decode that feeds a write: create, full-fields update, the fields object on the wire, fields_patch (request and store merge), the carried-blank-relation normalisation, the Append/Set metadata helpers, move, bulk move, cross-workspace copy and its preflight, and the move/copy field_overrides (models.FieldValues). - items.validateFieldType accepts a json.Number that parses as a finite number. items.coerceValue and the CLI's parseFieldFlag keep a numeric string that is a JSON number literal as that literal. - Read-only decoders are unchanged (display, hydration, guards). Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * fix(fields): compare field values by value, so an equal number in another spelling is no change and two integers that share a float64 are (BUG-3202) Field writes now keep a number's literal, so one value can be stored as different text (1e3 vs 1000). Three places compared field values: - item.updated's change detector compared the fields blob's TEXT whenever it was not masking a string done key: an equal-value rewrite, or a key-reorder-only rewrite, emitted item.updated. - the activity `changes` diff and the BUG-3163 carry check compared float64s: equal spellings were fine, but two different integers above 2^53 compared equal, so such a change went unrecorded and a changed reserved value passed as carried. models.CanonicalJSONNumbers gives every number one exact canonical spelling. All three decode keeping literals and compare canonical forms. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m |
||
|
xarmian
|
8d81f3a75c |
fix(timeline): structured entries get a persisted id instead of a positional one that renumbers when a sibling moves (BUG-2788) (#1540)
* fix(timeline): structured entries get a persisted id instead of a positional one that renumbers when a sibling moves (BUG-2788) An implementation note or decision with no usable id, or with an id an earlier entry already claims, fell to the timeline's "<prefix>-idx-<i>" fallback. That id is the cursor's tie-breaker and the client's list key, and it changed whenever an earlier entry was inserted or removed, or when the first of two duplicates was deleted, so an entry could be skipped or shown twice across a page boundary. models.StructuredEntryIDKey is now the one rule for what a stored id means in the timeline, and models.EnsureStructuredEntryIDs persists a minted id on exactly the entries that rule leaves without one (first holder of a duplicate keeps it). It runs in both append helpers, in workspace import, and in a startup backfill that is silent by ruling (no seq, updated_at, activity or event); the Q1 census that licenses the silence is in its comment. A repaired entry's timeline id changes once, from the positional form to its persisted id, and never again. No MCP surface change. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * fix(timeline): the entry-id repair touches only what the timeline reads, keeps values byte-stable, and its backfill compares the fields it read (BUG-2788 review round 1) - Skip a kind whose array the timeline's typed extraction rejects (a non-object element or non-string id hides the whole kind there), so the repaired population is exactly the positional one. - Leave a blob that is not exactly one JSON value untouched, instead of dropping its trailing bytes; re-encode without HTML escaping. - The backfill's UPDATE compares the fields text it read as well as seq, so a write that changed fields without moving seq is never overwritten. - Import's comment no longer claims byte-verbatim storage for a repaired blob. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * fix(timeline): the entry-id repair leaves a blob the timeline cannot parse at all (BUG-2788 review round 2) The timeline parses the whole fields blob with a plain Unmarshal first, which fails on a number outside float64's range and then shows no entries; the repair decoded with UseNumber and would still have minted ids there. It now requires that same first-stage parse. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m |
||
|
xarmian
|
70f7c0c15c |
fix(watchevents): a resume from a cursor inside an abandoned id space is refused on every arm, instead of being served the new space's ids or told it is caught up (BUG-2743, BUG-2728) (#1518)
* fix(watchevents): a resume from a cursor inside an abandoned id space is refused on every arm, instead of being served the new space's ids or told it is caught up (BUG-2743, BUG-2728) Every arm that detects a watch-stream id-space restart (the epoch arm and both counter-backward arms) now records the old space's high-water mark, and replaySince refuses any cursor at or below it. The boundary is never cleared: climbing past the old peak does not make a cursor below it any less ambiguous. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * docs(BUG-2743): the counter-reset arm's prose states the boundary's cost and which id the resume check compares (codex round 1) Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * fix(watch): pad watch --stream prints a PAD (resync) line on sync_required, so the agent reading it learns it missed notifications instead of carrying on (BUG-2743 follow-through) Clearing the cursor was the process's whole response; the consumer of its stdout could not see a cursor and was never told. The line goes through the same consent gate as a notification, and repeats with no notification between them are folded into one. Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * docs(plugin): tell the agent what a PAD (resync) monitor line means and what to do with it (BUG-2743 follow-through) Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m * docs(BUG-2743): the resync line says the stream MAY have missed notifications, since sync_required means the span is unvouched, not certainly lossy (codex round 3) Claude-Session: https://claude.ai/code/session_01A3rXV7GDLAhBe9mL4V6z2m |
||
|
xarmian
|
a231d0b65a |
fix(BUG-2367): a move or copy that would reopen, close or un-abandon an item must name the status instead of inheriting whatever survived migration (#1516)
Before this, every migrate door carried whichever done-field value survived
migration, or injected the destination default. A done task moved into
ideas silently reopened as `new`, and an abandoned item could land counted
as shipped.
Now every door (single move, bulk move, cross-workspace copy and its
preflight) compares the item's CloseState (open / done / abandoned, derived
from IsTerminalItem + IsAbandonedItem) before and after. A change is
refused unless the caller names the destination done field. Only the state
is compared, so two different values that both mean done carry without
asking. A destination with no done select field is exempt.
- move: 400 state_change_requires_value, details {field, options, from, to}
- bulk move: the same refusal in failed[]; `move` + `status` counts as supplied
- copy: 400 validation_error with the same sentence, after the unique check,
in the preflight's order
- preflight: a needs_value row with reason `state_change`, which the web
dialog's existing picker renders
- CLI: `hint: pad item move REF COLL --field status=<a|b|c>`
ToolSurfaceVersion 0.52 (behaviour bump). Missing needs_value routes on the
pane-menu move, CLI move and MCP are BUG-3200.
Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N
|
||
|
xarmian
|
fd95b79694 |
fix(BUG-2367): moves and copies never carry into a computed field, and a carried unique-field collision is dropped and named instead of 500ing or storing a duplicate (#1514)
MigrateFields matched fields by key and type only. On every migrate door (single move, bulk move, cross-workspace copy and its preflight): - a value was carried into a destination field marked `computed`. It is now dropped, with preflight reason `target_computed`. - a value colliding on a destination `unique_scope` field answered 500 on single move (invocation_slug, which has an index), leaked the SQL error through bulk move, and stored a duplicate on an unindexed unique field. Provenance decides now, as TASK-2878 did for relation referents: a CARRIED collision is dropped and reported as `not_unique`, and an override or schema default that collides is refused with 409 conflict. A required field emptied by the drop is refused as required. One store check (UniqueFieldConflictsQ / DropCarriedUniqueCollisionsQ) serves all four doors, including the tx-bound copy. The report names the value, and names the item holding it only when the caller may see that item. The activity row always uses the holder-free sentence. Reporting, all additive and omitempty: warnings.not_unique on the move response, the copy response and bulk updated[] rows; `detail` on the preflight's dropped row; CLI copy dry run and result, plus ` dropped:` lines on `item move` stdout; web copy dialog, move and copy toasts, and the timeline move card. ToolSurfaceVersion 0.51 (behaviour bump). The relation leg of the original report was already closed by TASK-2878. Item 4 (terminal/abandoned state change) is PR 2. Out of scope and filed separately: BUG-3200 (missing needs_value routes) and BUG-3202 (float64 precision; the migrateCopyFields comment now points there). Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N |
||
|
xarmian
|
03ff8fc43a |
fix(BUG-3186): the CLI turns an attachment name into a path Windows can store, on Windows only; stored names stay as given (#1498)
A stored attachment name keeps < > : " | ? * (legal and common on Linux and macOS: "Meeting: notes.pdf"). On Windows those are refused, and a colon is worse: "a.svg:x.txt" names an NTFS alternate data stream, so the bytes land in a stream on an empty "a.svg". - attachments.WindowsPathComponent, beside WindowsDeviceName: reserved characters and C0 controls become "_", trailing dots and spaces are trimmed, and a device stem gets the "_" prefix. It is for the boundary where a name becomes a path, never for the stored name. - The CLI's safeLocalFilename applies it only when it RUNS on Windows (safeLocalFilenameFor(runtime.GOOS, ...), so it is testable on any host). Each saver applies its own filesystem's rules, as browsers do. The accepted edge is written down there: a Linux CLI writing onto an NTFS mount keeps the colon. Not changed, by lead ruling on the trail: the Content-Disposition name (it stays faithful; the server cannot know the client filesystem) and the bundle tar path. Import recomputes that path from the manifest and skips unmatched blobs silently, so changing it would make older servers drop bytes. A cross-version contract comment on bundleAttachmentPath says so. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX |
||
|
xarmian
|
c50f0585ad |
fix(collab): WebSocket dials get their own per-user bucket and a per-user socket cap, and the editor jitters its reconnects (BUG-1308) (#1499)
* fix(collab): WebSocket dials get their own per-user bucket and a per-user socket cap; the web provider jitters its reconnects (BUG-1308) A collab dial drew on the general per-user API bucket (burst 60), so 100 dials from one user got 60 through, and a user's own dials and REST calls refused each other. Dials now draw on a CollabDial bucket (5/s, burst 50), and how many sockets a principal may hold is bounded separately by PAD_COLLAB_MAX_PER_USER (default 50), refused with 429 collab_limit_exceeded and Retry-After. Both defaults are sized from the reconnect-storm measurement on BUG-1308's trail (checkpoint 2). A browser cannot read a refused handshake's status, so the provider's backoff is what spaces a refused herd out: each step is now jittered to between half and all of itself. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * docs(collab): PAD_COLLAB_MAX_PER_USER and the collab dial bucket, in the deployment docs, samples and load-test usage (BUG-1308) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * fix(collab): only a WebSocket upgrade is a dial; per-user cap pinned within one workspace; dial-bucket keying stated correctly (BUG-1308, codex round 1) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * docs(collab): a PAD_COLLAB_MAX_PER_USER below 1 means unlimited, as the SSE knobs do (BUG-1308, codex round 2) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N * docs(collab): the struct field agrees, 0 or less is unlimited (BUG-1308, codex round 3) Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N |
||
|
xarmian
|
80ec51c18c |
fix(BUG-3185): an item export's file name whose stem is a Windows device gets a "_" prefix, on the server and in the CLI (#1495)
`pad item export` saves to "<slug>.pad.md", so an item slugged nul, con, aux, prn, com1-9 or lpt1-9 named a Windows device, and the CLI on Windows wrote the artifact nowhere. - attachments.WindowsDeviceName is exported (a rename, no behaviour change): the one device rule from BUG-2822 (#1493), where the stem ends at the first "." or ":". Its doc says not to write a second one. - artifactExportFilename prefixes "_" to a device name, so the Content-Disposition both clients save under is safe. - The CLI applies the same rule to the path it will write (defaultExportPath): it is the process writing the file, an older server still suggests the device name, and the "<ref>.pad.md" fallback never passed through the server. Population (every Content-Disposition and client default name): the bundle export ("<slug>-export.tar.gz", whose stem is never bare), the account export (a fixed name) and the edit temp/recovery files (the "pad-edit-" prefix) are safe; attachments are #1493. Import has no mirror: nothing it creates is named from content. Claude-Session: https://claude.ai/code/session_01Vf8uZmYdsrQqJtbAmjUFSX |
||
|
xarmian
|
1db30daa7f |
fix(project): standup and changelog rows show the value of the item's own done field, not a blank read from a literal status (BUG-2640) (#1477)
Both copies of the completed-work listing (the server's listTerminalItemsSince and the CLI's listCompletedWorkSince) already resolve each collection's done field per collection when querying. The rows rendering those items read the literal "status", so an item from a collection whose done field is something else (board_group_by: stage) was included with a blank Status. The resolved field now travels beside the items as a collection-id map, and the four row builders (server and CLI, standup and changelog) read it through completedWorkValue, falling back to status. The JSON key stays "status"; only the value it reads moves. Dashboard and item-list views that read the literal status to decide active, in-progress and stalled ask a different question, and are left alone. Claude-Session: https://claude.ai/code/session_01RYiqbqXmxQMoKHsGm8B77N |