mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-10-25 05:49:50 +00:00
4532eecffdeac4eeff1cd159ff1353df1ccac07b
399 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 xarmian
|
a23edba856 |
feat: built-in update count in the bootstrap, and accepted updates record replaced fields (TASK-3462 U3a) (#1868)
* test(server): the bootstrap counts built-ins with an update on offer (TASK-3462 U3a, red) * feat(server): bootstrap carries builtin_updates, a count of built-ins with an update on offer (TASK-3462 U3a) update_available plus diverged, among the items the caller sees in full: the library listing's own visibility, through one shared helper. omitempty, so a workspace with nothing on offer is byte-identical to before. * feat(mcp): agents mention built-in updates once and never apply them; tool surface 0.71 (TASK-3462 U3a) The skill (both copies) and the MCP server instructions describe the new builtin_updates bootstrap count: mention it once per session, and leave each update for a person to accept. ADDITIVE bump to 0.71, since the bootstrap every door returns gained a member and the instructions changed. Golden wire bytes regenerated: only the version and the instructions text moved. * test(server): an accepted built-in update records the field values it replaced (TASK-3462 U3a, red) * feat(server): an accepted built-in update names the field values it replaced (TASK-3462 U3a) Version history keeps bodies only, so the summary of the update now lists each non-body field whose value it changes, as key: old → new (JSON, bounded), including typed convention metadata. Fields the update leaves unchanged are not listed. * test(server): every template seed and every library activation reads current at once (TASK-3462 U3a) The guard the night-43 review ran by hand: the registry's hash must equal what each seeding door stores. A seed that stores drifted text turns it red (update_available straight after seeding). * test(server): replaced-fields summary ignores numerically equal values and truncates on a rune boundary (TASK-3462 U3a, codex r1, red) * fix(server): replaced-fields summary compares canonical numbers and cuts on a rune boundary (TASK-3462 U3a, codex r1) |
||
|
xarmian
|
90a304765f |
fix(cli): copy --dry-run previews a big number with every digit (BUG-3456) (#1858)
The copy preflight decoded carried field values with plain json.Unmarshal into `any`, so a number above 2^53 became a float64 and the dry run previewed 9007199254740992 for a stored 9007199254740993. The copy itself keeps the stored digits (server-side), so the preview misled exactly where a user would check. Decode with models.DecodeJSONKeepingNumbers, as BUG-3448 did for item show and the MCP resource. The mutating copy's response is left on json.Unmarshal: it prints no carried values (the item's fields are a string), and a decode error there would misreport a committed copy. Tests: the client keeps the digits, and the rendered dry run through the real client shows them; on the old decode it renders "= 9007199254740992". Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
3aff8e388f |
fix: pad keeps an edited /pad skill and never downgrades a newer one (BUG-3466) (#1857)
* test: pad init keeps an edited skill and does not downgrade a newer one (BUG-3466) * fix: pad keeps an edited /pad skill and never downgrades a newer one; skill files carry a stamp, and --force replaces (BUG-3466) * test: workspace init updates an older unedited skill, agent update reports a keep, --list names --force for a newer skill, and semver numbers have no size limit (BUG-3466, codex r1) * fix: workspace init routes installed skills through the decision, agent update and --list report kept files with --force, and semver numbers compare at any size (BUG-3466, codex r1) * test: --list flags an untracked edited skill, and agent update counts an untracked current one (BUG-3466, codex r2) * fix: --list asks the decision for local skills, and every skill present after a write is recorded and counted (BUG-3466, codex r2) * test: invalid SemVer never refuses as a downgrade; the frozen list pins a released text (BUG-3466, codex r3) * fix: the frozen legacy list holds released output only, and SemVer is validated strictly before comparing (BUG-3466, codex r3) * test: every unstamped release's skill output must be in the frozen list; the release workflow (which has the tags) fails loudly otherwise (BUG-3466, lead) * test: whitespace after the stamp does not hide it; pinned real copies and per-tag formatter checks guard the frozen list independently (BUG-3466, codex r5) * fix: whitespace after the stamp no longer hides it (BUG-3466, codex r5) * test: a newer pad's identical text reads as current, not as a downgrade (BUG-3466, codex r6 declined with reason) * chore: regenerate the legacy skill hashes with v0.18.0-rc.8 (BUG-3466) |
||
|
xarmian
|
b3cf4f5292 |
fix(server): an ordinary open backlog fills Up next and pad project ready (BUG-3453) (#1854)
A moved-in backlog of open tasks and bugs with no priority set, bugs in their initial status `new`, answered "No ready items found": only the literal status `open` counted, and an open item outside an active plan needed a high/critical priority (BUG-1082's "without flooding", though the list is capped at three). Such items now fill the slots the existing tiers leave empty, from a fallback tier below every item admitted before, so a workspace that filled three slots gets the same list. The tier admits items whose collection is not a system collection and declares `priority` or `severity` (lead-ruled; across every built-in template that is Tasks, Features, Backlog and Bugs), whose done field is not terminal, that are not in progress and not blocked. It ranks by priority, else severity, then age. `ready`'s help and the MCP `ready` description said it returns the full actionable backlog; it returns the same capped top three as `next`, and now says so. The empty state says why and points at `pad project stale`. ListCollectionsMinimal also returns is_system. ToolSurfaceVersion 0.70 (behavior bump on the v0.49 grounds). Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
c17865462f |
feat: built-ins record their origin; an item made from one can be offered Pad's current text (TASK-3462 U1+U2) (#1852)
* feat: built-ins record their origin, and an item made from one can be offered Pad's current text (TASK-3462 U1+U2)
U1: every convention and playbook Pad ships has a stable key and a content
hash; template seeding and the new server-side POST /library/activate record
{key, seed hash, seed text} in item_builtin_origin (migration 128 / pg 102).
CLI, remote MCP and the web Library page activate through the endpoint;
seeded and activated copies of an entry store the same fields. Export/import
carry the origin; copies get none; a request body cannot claim one.
U2: GET /items/{ref}/builtin derives current / update_available / diverged /
unknown_origin / unknown_entry; POST /items/{ref}/builtin/update takes the
library text through the ordinary PATCH path (expected_seq required, status
and title untouched, a forced version row keeps a diverged item's edits);
GET /builtins lists states in one join, showing only fully visible
collections.
* fix: the origin's seed text is NUL-protected (trigger migration 129); the MCP activate test asserts the endpoint call; activation follows the declared kind server-side (TASK-3462)
* fix: the built-in seed moves in the update's own transaction; a NUL-bearing imported origin is skipped; the listing computes visibility once; numbers hash by value (TASK-3462, codex r1)
* fix: on the applier path the built-in seed is recorded only after the apply confirms; a library change that only drops a field is offered (TASK-3462, codex r2)
* fix: a field the library added that the user already set reads diverged; an update clears convention metadata the library dropped (TASK-3462, codex r3)
* test: the lifecycle test finds the imported item by slug (the copy shares its title); ScanNUL's cost figures count the two new columns (TASK-3462)
* test(e2e): Library activate posts the entry to /library/activate and the item records its built-in origin (TASK-3462)
|
||
|
xarmian
|
0f932b5880 |
fix(mcp): remote routes answer a missing workspace with no_workspace and the list (TASK-2314) (#1846)
Every remote route that needs a workspace and lacks one (routeTable mappers, special routes, link routes, the assign/role resolvers) now answers the structured no_workspace envelope with available_workspaces and a hint, the one stdio has always given, instead of validation_failed "workspace is required". Mappers return one sentinel the dispatcher converts centrally; special routes call noWorkspaceResult directly. The single-workspace default now runs before the assign/role resolvers, so a one-workspace caller who omits workspace is not refused. The no_workspace hint is never empty. ToolSurfaceVersion 0.69 (behavior bump on the v0.42/v0.45 grounds). Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
05c039f131 |
fix: agents load items with --agent; --format markdown stays body-verbatim and names an empty body (TASK-3451) (#1842)
* test: markdown stays body-verbatim and names an empty body; no agent-facing text loads an item with it (TASK-3451) * fix: skills and playbooks load items with --agent; --format markdown stays body-verbatim and names an empty body on stderr (TASK-3451) * test: the guard covers every flag spelling and agent-read doc; a stale empty body is not claimed empty (TASK-3451, codex r1) * fix(cli): the empty-body hint is not printed over a stale body (TASK-3451, codex r1) * fix: remaining skill load lines use --agent; the guard exempts only round-trip descriptions and covers quoted refs and the read alias (TASK-3451, codex r2) * fix(skill): the convention-body fallback loads with --agent (TASK-3451, codex r3) |
||
|
xarmian
|
c204dd3683 |
fix(cli): json fields print as JSON, metadata labelled for the item's kind; MCP 0.68 (BUG-3448) (#1838)
* test(cli): json fields print as JSON, the metadata block is labelled for the item's kind, the update summary sizes structured values (BUG-3448) * fix(cli): json fields print as JSON, the metadata block is named for the item's kind, the update summary sizes structured values (BUG-3448) * test: large numbers, null, the MCP resource and unguessed labels (BUG-3448, codex r1) * fix: numbers keep their digits, null prints as null, the MCP item resource prints json fields as JSON, metadata labels are not guessed (BUG-3448, codex r1) * test(cli): the agent projection keeps large numbers (BUG-3448, codex r2) * fix(cli): the agent projection keeps large numbers (BUG-3448, codex r2) * chore(mcp): ToolSurfaceVersion 0.68, a BEHAVIOR bump: the item resource renders json fields as JSON (BUG-3448, lead ruling) |
||
|
xarmian
|
a91fcce7f9 |
fix: blank workspaces accept the library's own triggers and scopes (BUG-3446) (#1836)
* fix: a blank workspace's conventions and playbooks take the library's own triggers and scopes (BUG-3446) The blank template seeds Conventions with trigger [always] / scope [all] (Playbooks [manual] / [all]) and nothing widened them, so all 18 library conventions were refused on every door: CLI and MCP library activate, the web Activate button, and the onboard playbook's literal B3 create. A create in a system collection declaring the convention or playbook artifact kind now adds a missing trigger/scope value to the select's options when the LIBRARY itself uses that word (collections.LibraryOptionVocabulary, computed from the library). The write goes through UpdateCollection with its concurrency token, ahead of validation, and is reported in warnings.options_added (CLI: stderr) and in the created activity's metadata, which the item timeline renders. Any other value is refused as before. Editors may trigger it (lead ruling). The onboard playbook's B3 now says how an owner adds an invented trigger. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * chore(mcp): ToolSurfaceVersion 0.67 for BUG-3446 (additive) Lead ruling: a create in a system conventions/playbooks collection now adds a library trigger/scope word instead of refusing it, and its result gains an omitempty warnings.options_added. No tool name, enum or param moved; tools/list golden bytes are unchanged, initialize moves for the version and the instructions note. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
8c8deeeb89 |
fix: an invite asks an existing user instead of adding them; decline; CLI invitations (BUG-2136 U1) (#1831)
* test(server): an invite asks an existing user; re-invite replaces; decline by id and by code (BUG-2136, red)
* fix(server): an invite asks an existing user instead of adding them; a re-invite replaces the pending invitation; decline by id and by code (BUG-2136)
* feat(cli,mcp): pad workspace invitations/accept/decline; invite output says pending; ToolSurfaceVersion 0.66 (BUG-2136)
* test(e2e): invitees accept their invitation before the specs use the membership (BUG-2136)
* test(store): concurrent re-invites of one address leave one pending invitation (BUG-2136)
* test(store): an accept that loses to a decline adds no membership (BUG-2136, codex r1, red)
* test(server): an unverified invitee can decline by code, as they can accept (BUG-2136, codex r1, red)
* fix: an accept that loses to a decline or replace adds no membership; decline by code passes the verified-email gate; honest oracle comment (BUG-2136, codex r1)
* fix(server): the verified-email exemption matches exactly /invitations/{code}/accept|decline (BUG-2136, codex r2)
* test(server): drop the plan-limit harness's now-unused newUser (BUG-2136)
|
||
|
xarmian
|
f6938b33a5 |
feat(webhooks): owner webhooks keep per-endpoint delivery state; 408/425/429 are retried (TASK-3409) (#1809)
* fix(webhooks): 408, 425 and 429 are transient on the owner and app delivery paths, through one shared RetryableStatus (TASK-3409) * feat(webhooks): owner hooks keep per-endpoint delivery state: a retry goes only to endpoints still owed, a 4xx is final for the event, and endpoints still owed when retention gives up are counted as dropped (GET /webhooks, pad webhook list) (TASK-3409) * fix(webhooks): an owner endpoint deactivated while an event is still owed to it is recorded and counted as dropped, not silently skipped (TASK-3409, codex r1) * fix(store): the automatic deactivation settles the endpoint's owed deliveries as dropped and counts them, in its own transaction (TASK-3409, codex r2) |
||
|
xarmian
|
78f3c08240 |
test(e2e): a real installed app, end to end over https (TASK-3415) (#1804)
* feat(e2e): a test-build-only door for the app origin's CA (TASK-3415) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * test(e2e): a real install end to end over an https front and app origin (TASK-3415) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * test(e2e): the install review waits for a real manifest fetch (TASK-3415) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * chore: ignore the e2etest pad binary (TASK-3415) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * test(e2e): discovered https endpoints, awaited teardown, OS-assigned ports, an explicit timeout (codex r1 on TASK-3415) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * test(e2e): teardown survives a spawn failure, pad's port retries, the front drops hop-by-hop headers, cert setup cleans up (codex r2 on TASK-3415) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
41be1bf113 |
feat(apps): human-facing JSON names the app that wrote an item, comment or version (TASK-3413, SPEC-6 U9c) (#1787)
* feat(apps): human-facing JSON names the app that wrote an item, comment or version (TASK-3413, SPEC-6 U9c) Items (the creating app), comments and version rows carry via_app (the install id) and via_app_name (the bot's name, else the origin; kept after uninstall) on the web API, the MCP get / list summary / history summary projections, and the TS types, with WorkspaceApp naming the members apps array. ToolSurface 0.65, additive. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): attribution on search, /changes, playbook show and restore; chunked lookup (codex r1 on U9c) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): attribution on the role board, a comment edit's response and the account export (codex r2 on U9c) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * docs: README tool catalog at v0.65 (TASK-3413) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
82c45f24cd |
feat(apps): app item actions and context codes (TASK-3414, SPEC-6 U11) (#1784)
* feat(apps): app item actions and context codes (TASK-3414, SPEC-6 U11) [wip] Item actions are written as app_item_actions rows at provisioning, kept in step by a reviewed upgrade (a changed action gets a new revision, a removed one is retired), deleted at uninstall and backfilled at start. A viewer's click mints a single-use, two-minute context code; the app redeems it with its service token. Mint and redeem take the install, item and action rows in that order (FOR SHARE on Postgres), re-check the install, the item's current companion collection, the action's revision and the viewer's current visibility (the human item read's rule), and refuse with one 404 body. Redeem consumes the code first. The frozen TS type ItemAppAction and api.items.appActions / mintAppAction ship for U9. Migrations 125 / pg 099. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * test(apps): U11 mint, redeem, refusal and revocation tests; redeem in the read census; NUL baseline (TASK-3414) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * test(apps): invalidating mutations wait for a redeem holding the section 6 locks (TASK-3414) Deterministic: a hook pauses the redeem with its locks held while disable, an upgrade's action change, a move or a delete is started; each stays blocked until the redeem commits. On Postgres, dropping the install lock lets disable through, dropping the item lock lets move and delete through. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * test(apps): U11 mutant kills: epoch, deactivation, companion moves, release, admin non-member, backfill and workspace guards (TASK-3414) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): redeem at the token's workspace, service tokens only; no pool reads under the context locks; normalized base URL (codex r1 on U11, TASK-3414) Redeem moves to POST /api/app/v1/context/redeem (§6's path), outside /workspaces/{ws}: the workspace is the token's, injected before the same per-route stack, so no workspace-path refusal can answer before the code is consumed. It refuses a delegated token with the same 404. The viewer re-admission reads the user, role, membership and grants on the mint or redeem transaction (GetUserQ, ViewerRoleQ, UserHasGrantsInWorkspaceQ). A one-connection pool runs mint and redeem for a full member, a restricted member and a guest without hanging; switching any of those reads back to the pool deadlocks it. The link-out URL is built from the validated origin, refused before a code is written when base_url does not normalize. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): a delegated redeem attempt consumes the code before it is refused (codex r2 on U11, TASK-3414) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): build the context code expiry from its constant (lint, TASK-3414) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
aadf3c09d6 |
feat(apps): app-owned webhooks, held until redeem hands out the secret (TASK-3408, SPEC-6 U10a) (#1778)
* feat(apps): app-owned webhooks, held until redeem hands out the secret (TASK-3408, SPEC-6 U10a)
One webhooks row per install (app_install_id), created from the manifest
at provisioning, kept in step by a reviewed upgrade, deleted by uninstall.
Its signing secret is handed to the app only by redeem, which rotates it
and stamps secret_delivered_at; until then the hook is HELD (lead ruling,
day 86) and the owner sees "needs a new install code to activate".
Installs provisioned before U10 are backfilled at start, held.
App hooks stay off every owner door: ListWebhooks (the owner list and the
owner dispatcher), the scoped get/test and delete, and the plan count.
Events resolve to THIS install's companions only (via_app).
GET /workspaces/{ws}/apps/{installID} and the lifecycle responses carry
the hook's status. Delivery is U10b.
Migrations 122 / pg 096.
Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS
* fix(apps): the webhook backfill re-decides under the install lock from the stored manifest (codex r1, TASK-3408)
Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS
* test(apps): NUL census baseline for the app webhook columns (TASK-3408)
Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS
|
||
|
xarmian
|
ca158c22d8 |
feat(apps): app install staging: apps/1 manifest, fetch policy, pending reservations, preview (TASK-3397, U8a) (#1770)
SPEC-6 U8a (DOC-3371 §1, §2 steps 1-5, §9). Strict apps/1 parsing; an https/no-redirect fetch policy screened at dial time with an admin private-origin list; pending-install reservations under the owner lock and the instance advisory lock, holding the full charge until the fetch ends; the owner preview with every artifact normalized by the importer's own code (normalizeArtifact, prepareCreateFields, both extracted behaviour-preserving and pinned by a parity golden); apps_enabled and the admin apps settings; a background sweep of expired pending installs; migration 118/092. The preview is advisory: it may refuse what the import would accept, never the reverse; U8b re-checks atomically. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
e5262160e1 |
feat(oauth): install clients, service tokens and the issuance barrier for installed apps (TASK-3394, SPEC-6 U5a) (#1769)
* feat(store): install clients, the issuance barrier and app_token_bindings (TASK-3394) * feat(oauth): install-client service tokens, in-process app-token introspection, public introspection hardening, A0 token gate (TASK-3394) * test(store): the issuance barrier serializes with disable, rotate and re-enable on Postgres (TASK-3394) * test(server): pin each install-token refusal by its own hint; bearer introspection of another token (TASK-3394) * fix(oauth): a grant carries the epoch read before client auth; the install-caller check reads credentials as fosite does (TASK-3394, codex r1) * fix(oauth): classify the token request from the body fosite parses, multipart included (TASK-3394, codex r2) * test(server): account for the install-client token rules' form readers in the body-reader census (TASK-3394) * chore(store): renumber the install-client migrations to 117 / pg 091 after #1768 (TASK-3394) |
||
|
xarmian
|
82395bde1c |
feat(import): mark rows a workspace import wrote, author not verified (BUG-3379) (#1755)
* feat(import): mark rows a workspace import wrote, author not verified (BUG-3379) Workspace import keeps the export's comment authors, version kinds and attachment uploader ids verbatim (BUG-3372 ruling (1)), and nothing distinguished those rows from authenticated authorship, so an importer could present a fabricated trail. Nothing stored could tell them apart either: user_id is NULL after an account deletion too, the export supplies the timestamps, and import writes no activity. - Migrations 107 / 081 add imported (INTEGER 0/1) to comments, item_versions and attachments. Only the import doors set it (JSON and bundle; a derived thumbnail or transform inherits its original's). - Comments, versions and attachment rows serialise an omitempty imported: true. - An imported attachment's uploaded_by is never resolved to a local name: the member route sends X-Pad-Attachment-Imported: 1 instead. That also closes an importer naming a real member as uploader by putting their id in the manifest. - The web marks them: "· imported" beside a comment or reply author, an Imported chip on a history row (and an imported row never merges with a native one), and "imported" in place of the uploader in the viewer. - Rows imported before this are not marked: there is nothing to backfill from. - MCP tool surface 0.64 (additive): the flag reaches comment and attachment responses on both transports. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(import): keep the imported mark through recovery, autosave runs and copies (BUG-3379) Codex review round 1: - An imported version claiming recovery or system lost its marker in the history; it is kept. - The History tab's autosave row had no marker; it shows the chip. - The server's autosave burst grouping could fold an imported snapshot into a native one (both often have no user id); it no longer does. - A cross-workspace copy carried imported=1 onto a clone the copier uploaded, hiding its real local uploader; the clone is not imported. - A store test inserting attachments by column list is updated for the new column. - The 0.64 changelog names the MCP responses that actually carry the flag: comments, attachment list and full history (not history summaries, attachment show or the attachment resource). Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(import): a local edit never lands inside an imported version (BUG-3379) Codex review round 2: the version throttle ignored the flag, so a local edit within the hour, by the same kind and source as the newest imported version, wrote no version and read as part of the imported change. An imported predecessor now always starts a new version. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
a2e1c5d9ca |
fix(oauth): narrow current-only wildcard connections to their current workspaces (BUG-3338) (#1752)
Connected Apps let a user turn include_future_workspaces off under the wildcard, but the wildcard is live and the flag was never read, so those connections kept reaching workspaces joined later. A startup step, next to BackfillOAuthConnections, turns each such connection into a specific list of its user's live memberships and clears the wildcard: the same result as "Limit to my current workspaces". Narrowing-only (it writes rows only for workspaces the wildcard already reached and only ever clears the wildcard), idempotent (a narrowed row no longer matches, and no door stores the split combination since BUG-3338), one transaction, and it logs how many connections it narrowed, how many rows it wrote, and how many were left empty. A Go step rather than a numbered migration because the migration runner has no per-migration hook to log a count. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
Matt Faltyn
|
5614980edc |
Make database restore preserve live data on failure (#1703)
Stage and normalize SQLite backups before atomic publication, checkpoint committed live journals safely, and stop PostgreSQL restores on SQL errors. Cover recovery failures and compatibility with real-engine regressions. |
||
|
xarmian
|
2bd0862af2 |
fix(auth): platform administration refuses an API token (BUG-3361) (#1742)
* fix(auth): platform administration refuses an API token (BUG-3361)
BREAKING for anyone driving /api/v1/admin/* or admin account creation with
a personal access token: those now answer 403 session_required. Use a
`pad auth login` CLI session, or on self-host set PAD_ADMIN_API_TOKENS=allow.
An admin's PAT carried full platform authority. codex reproduced the chain
during BUG-3336's review: register an account with a chosen password,
promote it to admin, sign in as it, and reset any account's password. A
leaked admin token therefore escalated to interactive sessions for any
account.
- refuseAPITokenForAdmin guards the whole /api/v1/admin route group, so an
admin route added later is refused by construction.
TestBUG3361_EveryAdminRouteRefusesAnAdminPAT walks the router to hold it.
- handleRegister refuses admin-create signup with an API token.
- PAD_ADMIN_API_TOKENS=allow is the self-host escape hatch. It is read at
startup, logs a WARN, and is never honoured in cloud mode. Documented in
docs/deployment.md, Security.
- A browser session and a CLI session (padsess_) are unaffected. So is the
Pad Cloud sidecar, which authenticates with the cloud secret, never a
PAT. /audit-log already refused a PAT (BUG-3334).
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test: drive platform administration with the admin's session, not its API token (BUG-3361)
CI on the first push: 70 E2E specs and TestMCPCapability_UpgradeChangesNothingWhileOff
called /api/v1/admin/* or admin account creation with an admin PAT, exactly
the use this change refuses. Our own harness was the first operator script
the breaking change broke.
- E2E: global setup keeps the session token login already returns as
fixture.adminSessionToken (padsess_ is not User-Agent bound when sent as
a Bearer). The seven admin call sites use it: account creation in
permission-walk, workspace-tabs, workspace-access-live, no-commerce and
account-delete; the admin user search; and PUT /admin/mcp. Every other
call keeps the PAT, including the MCP client config in plan-2310.
- mcp_capability_test: the admin drives /admin with a CLI session and keeps
its PAT for /mcp, which takes one.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test(e2e): the admin console specs use the admin's session (BUG-3361)
CI on
|
||
|
xarmian
|
484ca668a3 |
feat(server): revoking access re-checks live connections immediately (#1744)
* feat(server): access kicks re-check live connections now, not on the next tick (TASK-3365 PR1) Dave's ruling on TASK-3345 #3. A per-instance registry indexes live collab, /events and /events/stream connections by user and workspace; a kick runs the revalidation each already performs on its 60s tick immediately (the tick stays as the backstop). Kicks travel the watch bus as the server-internal KindAccessInvalidated, which watchStreamDelivers drops, and the existing workspace_access_changed publication kicks too, so the doors that already publish it are covered. Kicks are non-blocking sends that coalesce per connection. PR2 wires the remaining access and credential doors. * fix(server): access kicks ride their own transport, reach watch streams, install idempotently (TASK-3365 PR1) Codex r1: kicks on the client watch bus took sequence ids, replay slots and client queue space, so they could evict real notifications and open gaps. They now ride internal/accesskick (Redis pub/sub on its own channel, or local with one instance), and the access-changed helpers kick directly, so nothing internal touches the client bus and the internal kind is gone. Watch streams register under a wildcard workspace so workspace kicks reach them. SetAccessKickTransport is idempotent and replacement stops the old subscription. A partial revoke kicks too. Tests make the first tick deterministic. * fix(server): access kicks never wait on Redis on a request path (TASK-3365 PR1) Codex r2: the shared Redis client ignores a context deadline on command I/O, so a stalled Redis held each publish ~5s, serially per recipient, on the request that revoked access. Local connections are now kicked at once and remote publication goes through one bounded background worker (dropped when full; other instances fall back to the tick); the transport uses its own client clone with context timeouts enabled; lifecycle changes send one workspace kick, and a complete revoke no longer kicks twice. * fix(server): access-kick worker cancels on removal; transport owns and closes its client (TASK-3365 PR1) Codex r3: the worker published with context.Background, so removing a stalled transport left the worker and queue alive; it now has its own cancellable context. The transport's cloned client is closed when the server replaces or clears it, and the clone gets its own push-notification processor instead of registering on the source client's. |
||
|
xarmian
|
38c591673f |
fix(search): scope workspace search to visible items (#1723)
* fix(search): scope workspace search to visible items (BUG-3331) /api/v1/search is mounted outside RequireWorkspaceAccess and decides access itself. A caller with no visibility in a named workspace reached the store with an empty permission filter, which the store read as no filter at all. Store: the permission filter is fail-closed. SearchParams gains an explicit Unrestricted flag; empty CollectionIDs and ItemIDs (nil or not) now match nothing unless it is set. One helper applies the predicate on every query path (ref lookup, number lookup, FTS, count, facets) and adds a match-nothing predicate when a restricted filter is empty. Handler: searchAccessFor resolves each workspace the way RequireWorkspaceAccess does (fresh-install window, user-less workspace token limited to its own workspace, consent allow-list, cookie-only admin bypass, full and restricted members, grants for non-members), for both a named workspace and the unscoped fan-out. No access is an empty result, not a 403. Collections: a caller who sees a collection only through an item grant gets item_count and active_item_count computed from the granted items, not the whole collection. Co-Authored-By: Claude <noreply@anthropic.com> * test(search): cover the restricted number lookup, total, mixed fan-out and active count (BUG-3331) Codex r1 (gpt-6-luna, max scrutiny) found no P0/P1. Its P2 (a restricted member's direct grant to a soft-deleted collection now hides that collection's items from search while the item list still shows them) is search being the stricter of two doors; the item list's side is BUG-3333. Its test gaps are closed here: the bare-number lookup and the filtered total for an item-grant guest, a fan-out over workspaces with different access, and the active count following a closed granted item. Also: the named-workspace consent check is visible in handleSearch again (the allow-list coverage guard scans the handler; searchAccessFor still re-checks), and the now-unused removeString is gone. Co-Authored-By: Claude <noreply@anthropic.com> * fix(search): check the consent allow-list first in searchAccessFor (BUG-3331) Codex r2 (no P0/P1): the allow-list check came after the no-user branch, the reverse of RequireWorkspaceAccess, so a context with a mismatching allow-list and a matching legacy workspace token would have read its workspace. No current auth flow produces that combination, but the order now matches the middleware and the coverage claim is exact. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
xarmian
|
26fd8a6a46 |
feat(mcp): ChatGPT tools declare their OAuth scope; a missing scope answers the sign-in challenge (TASK-3321 U2c) (#1720)
* feat(mcp): ChatGPT tools declare their OAuth scope; a missing scope answers the sign-in challenge (TASK-3321 U2c) Every ChatGPT catalog tool now carries securitySchemes, an oauth2 scheme naming the one scope it needs: pad:read for a read tool and pad:write for anything else. The Apps SDK reads it at the tool's top level and keeps _meta.securitySchemes as a mirror. mcp-go's Tool marshals a fixed set of keys, so the tool carries the _meta copy, and WithChatGPTToolSchemes adds the top-level copy to a single tools/list JSON response on the ChatGPT mount. Every other request and response passes through byte for byte. When a write tool is called with a read-only grant, the handler answers before dispatching anything: an isError result whose _meta["mcp/www_authenticate"] is a Bearer challenge with error="insufficient_scope" and an error_description, naming the protected-resource document of the mount the request arrived on (server.MCPResourceMetadataURLFromContext). That is what makes ChatGPT offer to reconnect with the wider grant. The wire golden is built through the production transport (newChatGPTTransport). Only the ChatGPT golden moved; /mcp is byte-identical. ChatGPTToolSurfaceVersion stays 0.1: it is unstable and unpublished. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(mcp): ChatGPT tools/list is a JSON body with top-level securitySchemes when asked the way ChatGPT asks (TASK-3321 U2c) Streamable HTTP upgrades a response to an event stream only by flushing notifications mid-request (the remote transport has no event store), and WithChatGPTToolSchemes hands the transport a writer that cannot flush. So tools/list requested with Accept: application/json, text/event-stream is a JSON body, and every tool carries securitySchemes at the top level. One leg pins that through the production transport, and another pins that the buffered writer never implements http.Flusher. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
1339916b9c |
feat(chatgpt): serve OpenAI's domain challenge at /.well-known/openai-apps-challenge (TASK-3321 G4) (#1717)
OpenAI verifies a plugin's domain by fetching the token its dashboard issued from /.well-known/openai-apps-challenge on the MCP host, and the body must be the token alone. PAD_OPENAI_APPS_CHALLENGE (toml openai_apps_challenge) resolves into config.MCPEndpoints, so it reaches the server through the existing SetMCPConfig binding. A value with inner whitespace or a control character, or over 1024 bytes, is refused into Problems() and logged at startup rather than served. The route answers 200 text/plain with exactly the token only on cloud, only with a token set, and only on the MCP URL's host. Every other case is the U0a JSON 404, byte-identical to an unowned well-known path. It is not behind the MCP or OAuth availability gates, because verification happens before the plugin server is switched on. Co-authored-by: Claude <noreply@anthropic.com> |
||
|
xarmian
|
6118f7065c |
feat(mcp): mount the ChatGPT catalog at /mcp/chatgpt, OAuth-only, off by default (TASK-3321 U2b) (#1719)
The ChatGPT projection (U1) gets its own transport at <MCP URL>/chatgpt, behind PAD_CHATGPT_MCP_ENABLED=true and served only while OAuth is available. It is its own OAuth resource (U2a's second canonical): its path-aware protected-resource document is /.well-known/oauth-protected-resource/mcp/chatgpt, its 401s name that document, and tokens do not cross between the two mounts. Personal access tokens are refused there with a 401 naming OAuth-only, ahead of the PAT branch that would accept them without the audience check. The consent screen names the ChatGPT surface when the requested resource is the ChatGPT one. The mount is wired only while the door versions content (ChatGPTDoorVersionsContent, held to the behaviour by a test) and every catalog tool has a response shape (ChatGPTCatalogReady); otherwise startup logs and skips it. /mcp is byte-identical. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
2dc5ac7058 |
feat(mcp): ChatGPT item_history names who made each change; email-shaped names are masked (TASK-3321) (#1716)
#1714 put actor_name on every history summary row. The ChatGPT item_history allow-list now keeps it, and the description says "who" again. Per the lead's ruling, a display name that is (or contains) an email address reaches ChatGPT as "a Pad user" in every person field (author, agent_name, actor_name, assigned_user(_name), owner_username, assigned_users, and user.name), never as the local part, which would be a new identifier. Other names (workspace, collection, role) are untouched. The leak sweep now runs twice, the second time with an owner whose display name is an email address. It fails with the mask off, as does the history leg. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
5c53aae306 |
feat(oauth): a second canonical resource, with every token bound to one (TASK-3321 U2a) (#1715)
* feat(oauth): a second canonical resource, with every token bound to one (TASK-3321 U2a) Ruling (i) on TASK-3321: the ChatGPT catalog is served at its own MCP URL, which is a canonical OAuth resource of its own, and its tokens are not interchangeable with /mcp's. - config derives ChatGPTResourceURL from the /mcp resource: its MCP path plus /chatgpt, so https://mcp.getpad.dev becomes https://mcp.getpad.dev/mcp/chatgpt. - The authorization server issues tokens for AllowedAudience plus AdditionalAudiences. A request must name exactly ONE of them, and every requested audience must be the same resource. A request for both, or for a foreign one, is refused before a code is issued, and the client's hydrated audience must include the one requested. A request naming no resource is still bound to /mcp's, the documented default. - MCPBearerAuth checks the MOUNT's canonical. A route stamps it with WithMCPResource (never from the request); an unstamped route is /mcp's. A /mcp token is refused at the ChatGPT mount and the reverse. Nothing is mounted yet (U2b). The cross-refusal test covers both directions and the no-resource token, and goes red when the middleware ignores the mount or the strategy allows two resources. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(oauth): a refreshed token keeps its one resource (TASK-3321 U2a review) Codex confirmed fosite's refresh flow restores the original granted audience; this pins it end to end: refreshing a ChatGPT token (with no resource, or asking for /mcp's) yields a token refused at /mcp and accepted at its own mount, and the test fails if no refresh succeeded at all. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
d3b1209d04 |
feat(mcp): ChatGPT responses are minimized through per-tool allow-lists (TASK-3321 U3) (#1713)
* feat(mcp): ChatGPT responses are minimized through per-tool allow-lists (TASK-3321 U3)
Ruling R2: ChatGPT tool responses carry refs, titles, statuses, fields,
names and user-meaningful dates, and no UUIDs, emails, counters or
internal codes. Before this, the workspace overview carried the caller's
email, every full item carried assigned_user_email, and search, create,
update, restore and get_playbook returned raw items.
Each tool's response goes through an ALLOW-LIST shape in the ChatGPT
handler: a key nobody listed is dropped, so a new field cannot reach
ChatGPT by default. The text fallback is rewritten from the projected
value. Relation values in fields become refs, and an id no ref replaced
is dropped. Comment id and parent_id stay as reply_to handles, and seq
stays for expected_seq. list_collections' schema is parsed to an object.
A plan-limit refusal loses its upgrade text and billing link.
Surface fixes: ready_items drops a limit the server ignored; item_history
no longer claims "who"; archive_item answers {ref, archived: true}; the
set_workspace exclusion reason is corrected (#1710 wired the bootstrap).
/mcp is unchanged: its goldens do not move.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* fix(mcp): ChatGPT errors and non-JSON results are minimized too (TASK-3321 U3 review)
Codex round 1, four P2s:
- Error results passed through unprojected. They now keep only code,
message, hint, field, expected and workspace slug and name, with ids in
the prose masked; a source Go error becomes a plain failure. A raw
validation hint that quotes an unresolved id was the case that
reproduced it; the error-path test fails without the projection.
- A non-JSON success passed through. It now fails closed.
- archive_item echoed the caller's ref even when it was an id. An
id-shaped ref is no longer echoed.
- The sweep now checks the text fallback with the same rules, and covers
error results.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* fix(mcp): mask ids in an error's field and expected too, and pin available_workspaces (TASK-3321 U3 review)
Codex round 2: field and expected were kept but not masked, and nothing
pinned that available_workspaces (slug, name, default) survives the error
projection. The envelope test now sets an id in every text member and fails
when only message and hint are masked.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
|
||
|
xarmian
|
0c0874dc10 |
feat(mcp): pad_item.history rows name who made the change (ToolSurface 0.63) (#1714)
Every pad_item.action=history summary row, on both transports, gains an omitempty actor_name: the display name of the user who made the change. Rows said only created_by "user" or "agent", although the server joins the name on every version read. Empty for rows with no user (recovery writes, versions from before migration 103). ADDITIVE, on the v0.53 and v0.28 grounds; no name, enum or param moved. The CLI's projection, which stdio returns as it stands, moves into toItemVersionSummaries so it can be tested. Each transport has a leg that fails without its copy. Requested by the lead as its own PR before TASK-3321 U2, so the ChatGPT item_history tool can say who changed an item again. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
bc2a982591 |
feat: every content change through the ChatGPT door is versioned first (TASK-3321 U1b) (#1711)
* feat(items): every content change through the ChatGPT door is versioned first (TASK-3321 U1b)
Ruling (b) on TASK-3321: ChatGPT's update_item is non-destructive because
every content change can be undone from History. An in-process context
marker (server.WithChatGPTSurface, unreachable from the wire) marks the
ChatGPT door. For a content write through it, the update handler sets
ForceVersion past the 1h throttle, labels the version "chatgpt", and sets
RefusePendingEdits: a tab's unflushed edits refuse the write with 409
content_pending_flush rather than be missing from the version and lost to
an undo. That refusal speaks to a chat user ("being edited in Pad right
now; ... Try again shortly") and names no API override. Field-only updates
write no version. A plain API body may not claim the chatgpt label. With a
tab open and caught up, the version is written through BUG-3327's
ExternalContent.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* feat(mcp): the ChatGPT catalog marks its door, so update_item versions every content change (TASK-3321 U1b)
Every ChatGPT tool call now reaches its source handler with
server.WithChatGPTSurface on the context. The HTTP dispatcher builds its
in-process request from that context, and the item update handler saves a
version before every content change from that door. The equivalence test
now checks the one intended difference: every ChatGPT dispatch carries the
marker (all 21 tools fail without it) and no /mcp dispatch does.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test(items): a collab-snapshot flush cannot wear the chatgpt label; simplify the strip (TASK-3321 U1b review)
Codex round 1 suspected a ?source=collab-snapshot flush could keep a
body's chatgpt label. It cannot: the collab-snapshot branch overwrites the
label first. That made the strip's collab guard dead code, so it is
removed, and the case is pinned by a test leg.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
|
||
|
xarmian
|
3901101c81 |
fix(mcp): remote pad_set_workspace embeds the workspace bootstrap (BUG-3326, ToolSurface 0.62) (#1710)
* fix(mcp): remote pad_set_workspace embeds the workspace bootstrap (BUG-3326) registerRemoteMCP passed no BootstrapFetcher, so over /mcp the tool returned only a not_persisted note: remote clients started sessions without the conventions and playbooks stdio clients get. HTTPResourceFetcher gains Bootstrap, delegating to its existing in-process bootstrap resource read, which runs as the token's user through the dispatcher's scope/consent perimeter. registerRemoteMCP builds the fetcher once and hands it to both Register and RegisterResources. Tests go through wireMCP over /mcp with PAT bearers: the embed is present for a member, and absent for a non-member while the owner of that workspace gets it. Co-Authored-By: Claude <noreply@anthropic.com> * chore(mcp): ToolSurfaceVersion 0.62 for the remote set_workspace bootstrap embed (BUG-3326) ADDITIVE bump (v0.53 / v0.28 grounds): remote pad_set_workspace gains the `bootstrap` member stdio always carried. No name, enum or param moved; the tools/list goldens are unchanged, the initialize goldens move only the version and the instructions header. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
xarmian
|
39a5259551 |
feat(mcp): the ChatGPT catalog, a one-tool-per-operation projection of /mcp (TASK-3321 U1, stacked on #1707) (#1708)
* feat(mcp): the ChatGPT catalog, a one-tool-per-operation projection of /mcp (TASK-3321 U1) A second tool surface for OpenAI's plugin directory: 22 tools, each naming the /mcp (tool, action) it stands for and executed by that tool's own fan-out handler, so validation, dispatch and every server rule are the ones /mcp runs. An entry adds only its outward shape: a name, a description, a narrowed parameter list (types and enums from the source), its own hints, Fixed inputs (get_item always asks for the agent view), and the irreversibility justification the submission form asks for. Guards: - every /mcp operation and pad_set_workspace is an entry or an exclusion with a reason, never both or neither, with no stale names; - the hints match the ruling, written out independently; - the validator refuses each defect it exists for; - every exposed param has a ChatGPT-own description, and no description uses /mcp's action vocabulary; - each ChatGPT tool dispatches exactly as its source (CLI path, args and dispatch input) through a recording dispatcher over the real cmdhelp doc; - an undeclared param is refused before dispatch; - a new golden pair pins the ChatGPT initialize and tools/list. /mcp is byte-identical: its golden files do not move. NewServer gains optional Instructions and Experimental overrides, defaulting to /mcp's. Not mounted: U2 serves it at its own URL behind a setting, after U1b (update_item's forced version row, which its description already promises) and U3 (response minimization). Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(mcp): ChatGPT catalog keeps field number literals; set_workspace is an exclusion (TASK-3321 U1 review) Codex round 1: - P1: the projection re-marshalled the DECODED arguments into RawArguments, so a fields number had already lost its literal (9007199254740993 -> ...992, 1.10 -> 1.1) before the source handler's BUG-3217 path. The caller's raw bytes now go through, with the action and Fixed inputs spliced in; a test fails on the old code. - P2: set_workspace promised an overview it cannot give. On the shared server it persists nothing, and the remote transport has no BootstrapFetcher, so it returns only a not-persisted note. It is now an exclusion with that reason, and get_workspace_overview is the connect step. - P2: link_type routing among the link commands is one disclosed operation (relate two items); noted at the entry. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(mcp): every ChatGPT tool but list_workspaces requires the workspace (TASK-3321 U1) There is no session workspace on the shared ChatGPT server, so an optional slug would fall back to nothing. Pins the lead's check on #1708; red when item_history's workspace is made optional. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
eb987af709 |
feat(mcp): pad_workspace destructive + open-world (invite emails), pad_playbook open-world (TASK-3321 U0b, ToolSurface 0.61) (#1707)
* feat(mcp): pad_workspace is destructive and open-world, because invite sends email (TASK-3321 U0b) pad_workspace advertised destructiveHint:false (all writes "additive") and every tool advertised openWorldHint:false. But its invite action sends an email to an address the caller names: a send cannot be taken back, which is what the MCP spec and OpenAI's plugin review mean by destructive, and an arbitrary recipient is outside the server's own data, which is what openWorldHint means. invite leaves additiveWriteActions. A new openWorldActions set, guarded by a stale-entry test, sets openWorldHint per tool. The wire goldens move for pad_workspace only, on both transports. ToolSurfaceVersion is now 0.61 (an annotation-only behaviour bump on the v0.20 grounds). Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * feat(mcp): pad_playbook is open-world, because match sends text to the decision provider (TASK-3321 U0b review) Codex round 1, P2: pad_playbook.match posts the caller's free text to the configured decision provider, a third-party model API. It stays read-only and non-destructive, but it is open-world. Open-world is about where data goes, not whether anything is written, so the stale-entry test no longer forbids a read in openWorldActions. The goldens now move for pad_workspace and pad_playbook only, and the 0.61 entry names both. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
cb99879d53 |
feat(materialize): recover unsaved editor sessions into items.content (TASK-2198 U4) (#1683)
* feat(materialize): recover unsaved editor sessions into items.content in the background (TASK-2198 U4) Wires the U3 supervisor in. A collab room reclaimed at the end of its grace TTL with content-bearing rows above the flush watermark (T3), or a dormant pending item found by a one-minute sweep (T1), is queued; one worker goroutine materializes the item's whole content-bearing op-log with the whole-workspace link index and lands it through store.MaterializeFlush, under the collab item lock with no room open. The write is a flush: cursor == MAX(op-log id), rows pending, no set-aside rows, checked in the write's own transaction; watermark to the cursor only; seq bumped; one version row attributed to system/recovery; the op-log is never pruned. PAD_MATERIALIZE=off disables it entirely. A per-item in-memory failure budget (3 failures, doubling backoff, reset by a new op-log row) keeps untrusted input from being retried in a loop. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): recovery watermark is the cursor, not MAX at UPDATE time (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): a second recovery still writes its own version row (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * docs(mcp): op-log recovery bounds the stale-body window; tool surface v0.60 (TASK-2198 U4) content_state applied_pending_flush now clears on its own after the item's room closes, unless recovery fails or PAD_MATERIALIZE=off. Every sentence that called the window unbounded, or said nothing server-side moves op-log content into items.content, is corrected (CLAUDE.md, instructions.md, README, docs/deployment.md, docs/backup.md, code comments). BEHAVIOR bump to 0.60; goldens regenerated. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): count ErrNoMemoryCap silently against the per-item budget (TASK-2198 U4) U3 now refuses every job with ErrNoMemoryCap when no memory cap can be established, and warns once itself. The recovery worker counts it like any failure but logs it at debug, not per item. Its log lines go through a logger seam so the test can tell. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * style: gofmt (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * feat(materialize): one log line per budget exhaustion; count set-aside skips on the sweep line (TASK-2198 U4) An item that uses up its failure budget logs exactly one WARN naming the item, the failure count and the last error kind (DEBUG when the kind is no_memory_cap, which the supervisor already warned about once), and logs again only after op-log growth resets the budget and it is exhausted anew. The schema-version skip goes through the same line. Items skipped for set-aside rows are counted and reported on a new single sweep summary line, written only when it has news; never per item. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): keyset-page the recovery sweep so no candidate starves (TASK-2198 U4, codex r1) The sweep read the oldest 200 candidates from the start every tick, and the room check and the failure budget filtered them afterwards. Once 200 sat in backoff or exhausted, every sweep returned the same 200 and the 201st was never queued: filtering above a bound is starvation. Each tick now reads one page strictly after an in-memory keyset cursor (newest op-log created_at, item id) and wraps to the start after a short page, so every candidate is offered within ceil(N/limit)+1 ticks, whatever sorts ahead of it. Keyset over NOT IN: no parameter-count limit, and it also covers the room check, which an exclusion list would not. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(materialize): the sweep cursor wraps; docs for the exhaustion line, the set-aside count and paging (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * docs(deployment): reflow the recovery budget paragraph (TASK-2198 U4) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(materialize): recovery strips the follows-title marker as a tab's save does (TASK-2198 U4, BUG-3315) Since #1688 a tab loads a link that follows its target's title with the follows-title marker in the link's title attribute, so the marker lives in Y.Docs that recovery replays. Three fixture cases (a follows-title link; the same link after a rename; a URL edited to an external address) are generated from documents that really hold the marker, and the existing 8 are unchanged. A new leg recovers each through the production path (whole-workspace index) and through the runner with a restricted and an empty index, and asserts no 'pad-follows-title' reaches the body. It goes red in all 7 ways when the bundle's marker strips are disabled. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(store): a recovery version row carries #1691's history data (TASK-2198 U4) Rebased onto PLAN-2348 U2 (#1691), which adds user_id, line counts and a create marker to version rows. Recovery writes its row through the same UpdateItem insert, so it gets them automatically: no user_id (a system row), not a create, and line counts equal to diff.LineCounts of the stale and recovered bodies. The leg pins that. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
d78ff9bdc5 |
feat(materialize): worker supervisor with per-OS memory cap (TASK-2198 U3) (#1682)
* feat(materialize): parent-side worker supervisor with per-OS memory cap (TASK-2198 U3) The Supervisor runs materializer jobs in ONE long-lived child process (`pad __materialize-worker`), started lazily on the first job and fed jobs serially. A job past PAD_MATERIALIZE_TIMEOUT (default 2s) kills the child (goja's Interrupt, sent as timeout_ms a tenth earlier, is only the soft layer); a dead child is respawned on the next job after an exponential backoff (1s doubling, capped at 5 min, reset by a child that lived a minute). Every stop is logged with its reason (deadline, memory, exit, protocol, start, canceled, closed); every start logs the effective timeout, memory limit and cap mechanism. The child's stderr goes to the log, pid-tagged and bounded, never near the protocol. A job whose child died is never retried here. Memory cap (PAD_MATERIALIZE_MEM_LIMIT, default 2GiB, bytes or KiB/MiB/GiB, clamped to 1536MiB-16GiB): - Linux: RLIMIT_AS set on the child by prlimit before its first byte of input; the child only runs its own startup uncapped. - Windows: Job Object with JOB_OBJECT_LIMIT_PROCESS_MEMORY; the child is created suspended, assigned, then resumed: no uncapped window. - macOS: parent-side RSS watchdog every 100ms (proc_info syscall, ps fallback). - Other: deadline kill only. Not wired into the server yet (U4). Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * ci(materialize): run the supervisor on the real macOS and Windows caps (TASK-2198 U3) native-smoke now builds the real materializer bundle (as the go job does) and runs the supervisor, memory-cap and deadline tests on macos-latest (proc_info RSS watchdog) and windows-latest (suspended start + Job Object). PAD_MATERIALIZE_TEST_NO_SKIP turns the cap tests' skips into failures there, so a cap that stops running cannot pass as skipped. Tests: a real Windows processGone (OpenProcess + GetExitCodeProcess), RSS samplers for darwin (the production proc_info path) and Windows (K32GetProcessMemoryInfo), and an allocation helper that writes noise into every byte so macOS memory compression cannot hide it from the watchdog. The worker-loop tests share one loaded Runner through serveWorker (RunWorker's loop, split out); TestWorkerRoundTrip still goes through RunWorker end to end. go test -race -count=3 on the package: 714s -> 413s. docs: the materializer section says the worker and its knobs are inert until U4 wires it in. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): a worker never outlives its supervising process (TASK-2198 U3, codex r1) If the server died mid-job (SIGKILL, OOM kill, crash) its worker ran on with no deadline kill and, on macOS, no memory watchdog. Two layers: - Linux: Pdeathsig = SIGKILL. The fork runs on a fresh goroutine (startUnlocked), never locked to its thread, so it cannot land on a thread the runtime later retires (golang/go#27505); Go's forkExec re-checks getppid if the parent is already gone. - Every Unix (the only layer on macOS): the worker command calls ExitWhenOrphaned before the bundle loads, polling its parent pid every 200ms against PAD_MATERIALIZE_PARENT_PID and exiting (status 70) when it changes. It writes nothing. - Windows was already covered by the Job Object's KILL_ON_JOB_CLOSE. TestSupervisorOrphanedWorkerExits kills a helper process that supervises a worker hung in a job and requires the worker gone within 1s: 0s on Linux, 105ms with the ppid watch alone; with both layers removed the worker outlives its parent by 5s. It matches native-smoke's TestSupervisor.* selection, so it runs on macOS and Windows too. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): relative memory cap, refuse when uncappable, GOMAXPROCS=2; nix-safe ps test (TASK-2198 U3) The absolute Linux RLIMIT_AS failed on CI: a 64 MiB allocation died under a 1600 MiB cap, because the runtime there reserved more address space than on the machine the floor was measured on. The cap is now RELATIVE on every OS: once the worker has loaded and before it is sent any job, its usage is read as a baseline and the cap is set to baseline + PAD_MATERIALIZE_MEM_LIMIT — address space on Linux (/proc/<pid>/stat vsize, prlimit, read back), committed memory on Windows (the Job Object's per-process limit, set after the suspended start and load), resident memory on macOS (the watchdog's threshold). Every start logs baseline, limit and effective cap. Lead ruling, REFUSE: if the baseline probe fails or is implausible (0, or above MaxPlausibleBaseline = 8 GiB, ~4.8x the largest measured 1704 MiB), onBaselineProbeFailure kills the worker without sending it the job and fails the job with ErrNoMemoryCap; one WARN until a probe succeeds again. An OS with no cap mechanism never starts a worker and refuses every job with ErrNoMemoryCap, one WARN naming runtime.GOOS. The worker runs with GOMAXPROCS=2: measured loaded-worker baseline 1357 MiB vs 1597 MiB at this host's default (8) and 1704 MiB at 64. MinMemLimit drops from 1536 MiB (an artefact of the absolute cap) to 256 MiB: the corpus grows the worker by at most +10 MiB of address space and +18 MiB resident, and a 4,000-deep nested document by +129..194 MiB. A cgo worker's "pthread_create failed" at the cap is now classified as a memory death. Nix: TestParsePSRSS parses fixed strings only; the live `ps` leg is TestPSRSSLive and skips when there is no `ps` (the Nix sandbox has none; ps is only macOS's fallback), unless PAD_MATERIALIZE_TEST_NO_SKIP is set. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): Windows kills on the job's memory notification; steady thread set before the baseline (TASK-2198 U3) Windows (CI on c1866957): the Job Object refused the 1 GiB commit (VirtualAlloc errno=1455), but the worker did not die and the job ran to its deadline. The cap no longer depends on the runtime choosing to crash: the job is associated with an I/O completion port, and JOB_OBJECT_MSG_PROCESS_MEMORY_LIMIT for the worker's pid kills it at once (ErrMemoryLimit). errno=1455 on stderr is a second signal. On every OS a cap mechanism that sees the limit hit calls onLimit, and the kill is reported as "reached its <mechanism> cap". Linux (CI's cgo test binary at GOMAXPROCS=64): threads created after the baseline ate the growth budget. The worker now brings its thread set to steady state before readiness (warmThreads: an allocating goroutine per P, then a GC). Measured on a cgo build: threads added after the baseline are 0 at GOMAXPROCS=2 with or without it; at 64, +3 threads and +272 MiB of address space during the corpus without it, +0 and +0 MiB with it. 256 MiB stays the floor (largest growth at GOMAXPROCS=2: +193 MiB, a 4,000-deep nested document; the corpus adds +0 MiB). The cap test's second leg no longer uses GOMAXPROCS=64 (which also changed thread creation) to stand in for a bigger host: the helper inflates its own baseline before readiness, with never-touched address space on Linux, committed memory on Windows and resident memory on macOS. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * feat(materialize): the worker stops after an idle timeout; the next job restarts it (TASK-2198 U3) Dave's ruling on the worker process: GO, with an idle exit. A loaded worker holds ~200 MB resident, which an install that rarely materializes should not pay for all day. PAD_MATERIALIZE_IDLE_TIMEOUT: default 5m, clamped to 30s-24h, 0 = never. After each job the supervisor (re)arms a timer. When it fires, the stop runs only while holding the job slot, so it never takes a worker with a job in flight. It re-checks the idle time, forgets the worker WITHOUT recording a death (backoff is untouched), and closes the worker's stdin so its read loop ends with status 0, with a kill after a grace if it does not go. The stop is logged at INFO, reason=idle. A job that arrives meanwhile waits for the slot, finds no worker, and starts a new one with no backoff wait. Legs: the idle exit after the timeout (INFO, status 0, no WARN); the next job respawns and succeeds with no wait; an idle stop leaves the backoff streak alone (crash, idle, crash waits 2s, not 4s); a job arriving during the stop runs on a new worker; a timer that fires mid-job does not stop that worker; NeverIdle arms no timer; env parsing and clamps. Mutations, each red on its own: recording a death (3 legs), logging as a kill (1), not holding the slot (1), never arming (3). Docs: the knob beside the other two, and `idle` in the stop reasons. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): Close does not return while an idle-stopping worker still runs (TASK-2198 U3, codex r1) An idle stop forgets the worker before it has exited, so a Close in that window found no worker and returned with it still alive. The supervisor now tracks the worker an idle stop is ending, and Close kills and reaps it. A leg pins it, and goes red without the fix. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(materialize): an idle stop that Close ended says so (TASK-2198 U3, codex r2) The idle stop stored its kill reason unconditionally, so it overwrote the 'closed' of a Close that had already killed the worker, and its stop line said idle. It now claims the reason only if none is set, and the line reports reason=closed. The Close-during-idle-stop leg asserts it. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
a61f50421f |
fix: a move refused for a value only the caller can supply gets a way forward on web, CLI and MCP (BUG-3200) (#1690)
* fix(web): a pane-menu move refused for a value only the user can supply opens the copy dialog's picker, preset to that collection (BUG-3200) * fix(cli): item move refused as missing_required_fields prints what each field needs, from the copy preflight (BUG-3200) * fix(mcp): a remote move refused for a destination value names the field parameter that supplies it (BUG-3200) * fix(web): fence and classify the copy dialog's preset continuation; the handoff is behind stillOnSource (BUG-3200, full vitest) * test(web): the handed-off move keeps the item's id and ref and sends no copy (BUG-3200, lead check) |
||
|
xarmian
|
fa2755e183 |
feat(materialize): headless op-log materializer bundle, goja runner and worker (TASK-2198 U2) (#1678)
* feat(materialize): headless op-log materializer bundle, goja runner and worker (TASK-2198 U2)
Builds the editor's own JS (Yjs -> y-tiptap -> Tiptap schema -> tiptap-markdown
-> the tab's flush pipeline) into one IIFE, embeds it, and runs it in goja. Not
wired into the server yet (U3 supervisor, U4 write path).
- web/src/lib/collab/materializer/: bundle entry (extension list mirrors
Editor.svelte's collab branch via the U1 shared modules), op-log replay
mirroring wsProvider's MESSAGE_SYNC arm, linkedom DOM shim and polyfills
from the spike, and a vendored patch of prosemirror-markdown
MarkdownSerializerState.atBlank (quadratic under goja; equivalent).
- web/scripts/build-materializer.mjs (esbuild) -> web/build-materializer/,
run by `npm run build`; embedded as pad.MaterializerJS.
- internal/materialize: Runner (Go-native atob/btoa/TextEncoder/TextDecoder/
getRandomValues, schema-version check, ctx interrupt with a clean VM after),
and RunWorker's length-prefixed JSON protocol behind the hidden
`pad __materialize-worker` command.
- Parity: vitest drives two real Editor.svelte mounts and compares
materialize() with the tab's flush on 50 seeded cases; the same cases are
committed as testdata/corpus.json and replayed in goja (TestCorpusParity).
- CI Go jobs, release tests, Docker and Nix produce the bundle; linkedom and
esbuild are new devDependencies.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* test(materialize): pin the atBlank patch's equivalence under V8 too (TASK-2198 U2, codex r1)
Codex r1 claimed upstream's /(^|\n)$/ is true for a trailing CR in JS. It is
not (checked in V8: false for "\r" and "a\u2028", as the patch returns),
but the only equivalence test ran under goja and the patch header cited a
V8-side test that did not exist. This adds it, with the line-terminator edge
cases, and it fails when the patch checks CR instead of LF.
Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC
* build(nix): vendorHash for goja (TASK-2198 U2)
Taken from the Nix job's recompute on
|
||
|
xarmian
|
a2e7768035 |
fix(comments): a deleted comment with replies leaves a tombstone (BUG-3252) (#1675)
* fix(comments): a deleted comment with replies leaves a tombstone (BUG-3252) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * test(comments): backdate so a tombstone moving updated_at is seen (BUG-3252) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * chore(mcp,store): v0.59 instructions heading, NUL baseline for comments.deleted_at (BUG-3252) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(comments): reactions wait for a tombstoning delete; unreact on a tombstone answers 409 (BUG-3252, codex r1) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC * fix(comments): unreact racing a tombstoning delete answers 409, not 404 (BUG-3252, codex r2) Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |
||
|
xarmian
|
a90f75fde2 |
fix(server): sign protected share-page images on every install (BUG-3305) (#1673)
* fix(server): sign protected share-page images on every install (BUG-3305)
A password or require-auth share link on a self-hosted instance showed a
placeholder for every embedded image. The key that signs those image refs
was set only in the cloud block, and only with PAD_MCP_PUBLIC_URL set, so
it was nil on every self-host, and the mint side omitted the refs.
The key is now set right after server.New on every install, from the
deployment key every running install has (EnsureEncryptionKey generates
one on SQLite and refuses to start Postgres without one). The MAC stays
domain-separated from the key's other uses. The cloud-only site is gone.
Deliberate behaviour change: after upgrade, self-hosted protected share
pages render their images. So does a cloud instance without
PAD_MCP_PUBLIC_URL, which had the same gap.
Comments that gave the old reason for a missing key ("self-host without
an encryption key") are corrected; the no-key branch stays as the
defensive floor, with its tests.
Evidence: a live boot of the built binary as a plain self-host. A
password share resolves with a signed image ref; the signed asset
answers 200 and the unsigned one 404. The same run on main's binary has
no ref.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
* docs(server): say how the share-asset MAC stays apart from the key's other uses (BUG-3305 codex r1)
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
|
||
|
xarmian
|
b510d67696 |
fix(oauth): sweep expired token, code and PKCE rows (BUG-3301) (#1671)
* fix(oauth): sweep expired token, code and PKCE rows (BUG-3301) Nothing deleted OAuth rows by age. Each refresh leaves the chain's previous access and refresh rows inactive and inserts a fresh pair, measured at +1 access and +1 refresh row per refresh (48 rows a day for one active client at the 1 h access lifetime), forever, on Pad Cloud and on any https self-host that turns MCP on. The token reaper now also runs SweepExpiredOAuthRows every hour. A row goes once its requested_at (its own issuance, measured) is older than its token's lifespan plus 24 h: - refresh and authorization codes: the refresh lifespan. A replayed rotated token or a reused code keeps revoking the family for as long as the token could have been valid, because fosite checks "inactive" before "expired". - access: the access lifespan. Nothing reads an expired access row. - PKCE: the code lifespan. Only abandoned authorizations leave one. The lifespans come from the running OAuth server (defaults when OAuth is not built). No row a live chain or a concurrent exchange could use is ever below a cutoff. Deletes run 500 rows per statement with a 20 ms pause between batches. Measured on SQLite with 50,000 rows and a concurrent token write every 5 ms: with no pause, one write waited 1.55 s; with the pause, the worst write took 52 ms. The per-tick cap is 200 batches per table. Connected Apps and the MCP resume count now drop a chain whose refresh token has been expired for the grace period, so their "upper bound" comments say how far. The deployment guide's "never swept" gap is gone. Tests: store cutoffs, zero cutoffs, and batches with the cap, on SQLite and Postgres. A server end-to-end test through /oauth/token: the live chain survives and still refreshes; an aged-out chain leaves Connected Apps and the resume count; a swept token is refused without revoking the family; a retained rotated token still revokes it. Mutants: inverted age comparison (store tests fail), refresh cutoff at now (e2e fails: the live chain's refresh answers invalid_grant). Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz * fix(oauth): code retention adds the code lifespan; Capped means "may remain" (BUG-3301, codex r1) - A code's requested_at is when it was authorized, but the first refresh token it yields starts at the exchange, up to a code lifespan later. The code cutoff now adds the code lifespan to the refresh lifespan, so reuse detection lasts as long as that refresh token could be valid. The comment no longer claims "the chain's lifetime": a chain renews forever. - Capped says the cap was used, so rows MAY remain; the last full batch may have been the end. Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz * fix(oauth): a non-expiring lifespan skips its tables (BUG-3301, codex r2) fosite reads a negative lifespan as "never expires"; the cutoff math turned -1 into "now minus the grace", which would delete a live non-expiring token after a day. A table whose retention depends on a negative lifespan now gets a zero cutoff, which the store skips (codes depend on both the code and refresh lifespans). Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz |
||
|
xarmian
|
22bfe8cc89 |
feat(mcp): off-cloud Host allowlist on the non-API MCP paths (TASK-2319, PLAN-2310 U3) (#1665)
* feat(mcp): off-cloud Host allowlist on the non-API MCP paths (TASK-2319, PLAN-2310 U3) Off cloud, /mcp, /oauth/* and /.well-known/oauth-* answer 421 Misdirected Request (JSON) when Host does not name the configured origin, MCP URL or auth-server URL. Matching is case-insensitive with the port defaulted from the configured scheme; a loopback origin admits every loopback spelling on its port; an empty Host is refused. It runs after the availability gate (MCP off still answers 404 whatever the Host) and before auth, audit and the OAuth group's rate limiter. The /api/v1 routes are not wrapped: they keep the API perimeter, and the in-process dispatcher sends them an empty Host. Cloud is unchanged. mcp-go's DisableLocalhostProtection stays, with its comment rewritten. The e2e harness sets PAD_URL to its base URL. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * feat(mcp): honour X-Forwarded-Host from trusted proxies; 421 names the fix (TASK-2319 lead review) Behind a proxy that rewrites Host to the backend's name (Azure Application Gateway and App Service, some ingresses) every MCP request would get a 421. The allowlist now checks the first X-Forwarded-Host entry when the TCP peer is in PAD_TRUSTED_PROXIES, the same trust test cliAuthScheme and TrustedProxyRealIP use; from any other peer the header is ignored, so DNS-rebinding protection is unchanged. The 421 names the received host and its header, the configured origin, and the proxy fix, and the first refusal per host is logged at WARN behind one overall limiter (5, then 1 a minute; at most 1024 hosts remembered). Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 |
||
|
xarmian
|
ae7f77d9d8 |
feat(mcp): construct MCP off cloud, mount always, gate per request (TASK-2317, PLAN-2310 U2) (#1663)
* feat(mcp): construct MCP off cloud, mount always, gate per request (TASK-2317, PLAN-2310 U2) The remote MCP capability (transport, in-process dispatcher and resource fetcher, audit writer, session tracker) is now constructed on every install by wireMCP, and the OAuth server, claim secret and connections backfill whenever the resolved auth-server URL is https. Nothing is served until Server.mcpAvailable() / oauthAvailable() say so, per request: every DR-5 route is mounted always and answers the JSON 404 while unavailable, before auth, audit or rate limiting. The two audit-history routes are unchanged. - Discovery takes its URLs from configuration only. The four "https://"+r.Host fallbacks and rewriteMcpSubdomain are gone; with nothing configured the documents answer 404, and the /mcp 401 is a bare Bearer realm="pad" unless OAuth is available. - resource_metadata follows RFC 9728 section 3.1, so a resource with a path (a self-host's <origin>/mcp) points at a route that exists. Byte-identical for a path-less resource (cloud). - An explicitly set PAD_MCP_PUBLIC_URL / PAD_AUTH_SERVER_URL keeps its historical spelling (TrimRight "/"), so the OAuth audience of every existing deployment is byte-identical and issued tokens keep validating. - Protected share-link image signing gets its own key field, set exactly where the claim secret used to be (cloud with PAD_MCP_PUBLIC_URL), so building OAuth on an https self-host changes nothing on share pages (lead ruling B; BUG-3305 is the deliberate fix). - The session and setup payloads keep mcp_public_url cloud-only until the connect modal gains its PAT path (DR-8, unit 6). - The claim routes answer 404 instead of 412 claim_disabled where OAuth is not available; the auth-server metadata answers 404 instead of 503. Tests: the DR-5 route table across the eight acceptance states and the upgrade test (no mcp_enabled row, https origin: every gated route 404, the audit routes unchanged, a valid dispatcher claim refused; the control leg reaches every route and the same claim succeeds), both over the production wiring; an https self-host OAuth flow for audience <origin>/mcp; a PAT call over http that writes an audit row. Release gate (PLAN-2310): must not ship without units 3, 4 and 7. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * docs(mcp): the availability gates run inside the API perimeter on /api/v1 routes (TASK-2317 codex r1) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 |
||
|
xarmian
|
eb167bea00 |
feat(mcp): MCP addressing from config; mcp_enabled setting with an env lock (TASK-3302, PLAN-2310 U1) (#1662)
* feat(mcp): resolve MCP addressing from config; mcp_enabled setting with an env lock (TASK-3302)
PLAN-2310 unit 1 (DR-2, DR-3, the data behind DR-7). No route changes:
nothing is gated or exposed by this unit, and U2 wires the predicate
into the routes.
Addressing (internal/config/mcp.go, ResolveMCPEndpoints):
- The origin is cfg.URL (PAD_URL, toml url, --url), else PUBLIC_URL,
never the Host:Port fallback and never a request's Host.
- The MCP URL, which is also the OAuth audience, is PAD_MCP_PUBLIC_URL,
else <origin>/mcp. The auth-server URL is PAD_AUTH_SERVER_URL, else the
origin.
- A value must be an absolute http(s) URL without user info, query or
fragment; an origin also has no path. An unusable value never fails
startup (PAD_URL also serves the CLI). It is logged, leaves its field
empty, and keeps MCP unavailable. An unusable override does not fall
back to the origin.
- HTTPS() is the auth-server URL's scheme: https means OAuth plus PATs,
http means PATs only.
The setting (internal/server/handlers_admin_mcp.go):
- platform_settings.mcp_enabled, default off, read per request with no
cache (the webmcp_enabled precedent). A read error counts as off.
- PAD_MCP_ENABLED (true/false/1/0/yes/no/on/off) forces it and locks the
toggle. An unparseable value forces nothing and is logged. Cloud is
always on.
- mcpAvailable() is DR-1's predicate: cloud, or the setting on and the
addressing usable.
- GET/PUT /api/v1/admin/mcp, admin-only. GET reports enabled, source
(setting|environment|cloud), locked and the readiness: origin, MCP URL,
auth-server URL, https, auth methods, problems, state on|off|blocked
with the reason, and the resume counts. PUT answers 409
set_by_environment under the env lock (the decision-provider shape),
403 managed_by_operator on cloud, and 400 without {enabled}. It is
audit-logged as settings_changed.
- The resume counts (internal/store/mcp_resume.go) count live OAuth
chains the way the Connected Apps page does (request_id with an active
access or refresh row), and PATs the way /mcp accepts them (has a
user_id; expiry parsed as a time, as ValidateToken does).
Tests:
- config resolution table (11 cases); PAD_MCP_ENABLED parsing and Load.
- Store counts: a rotated chain counted once, a revoked chain excluded, a
refresh-only chain counted; the legacy and expired PATs excluded.
- Router tests: default off (the upgrade case); https on; http PAT-only;
blocked without or with an unusable origin; the env lock with a stored
value underneath; cloud; admin-only and body validation; a setting read
error is off, with a control leg.
- Mutants, each BUILD OK and killed by the test aimed at it: env override
ignored; availability ignoring addressing; PUT skipping the env lock;
origin path allowed; PUT allowed on cloud; read error counted as on
(error branch only); invalid override falling back; expired PATs
counted; refresh-only chains missed.
Docs for the variables and the endpoint land with PLAN-2310 U7.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
* fix(mcp): canonical MCP URLs; PAT count mirrors /mcp's user check; one setting read per response (TASK-3302)
Codex round 1 on
|
||
|
xarmian
|
237333681b |
feat(attachments): attach binds an unattached upload to an item, so its share renders it (TASK-2247) (#1652)
The July audit's share-page broken-image finding was fixed by BUG-2389 and
TASK-2637: the share byte endpoint serves variants of attachments the
shared item OWNS. What was left: an image uploaded with
`pad attachment upload -` (item_id NULL) and then embedded is never owned,
so its share shows the placeholder. A census of this workspace found 4
such refs out of 13, all from that path.
Lead ruling (option C): fix it explicitly, with no implicit binding on
content writes.
- POST /workspaces/{ws}/attachments/{id}/attach {item}. It binds a live,
unattached original in this workspace, and its live variants, to an
item. Every UPDATE requires item_id IS NULL, so an attached row is never
moved, and the bind is exclusive with the never-attached GC claim. The
item is pinned like CreateAttachmentForLiveItem.
- Authorization: the caller can see and edit the item, and uploaded the
attachment or is a workspace owner. An attached row answers 409 to a
caller who can see its owner, and the shared 404 to anyone else.
Restricted members and guests cannot read unattached rows, so they get
the 404. That closes the case where a guest binds an id they know to an
item they can edit and publishes it through its share.
- The attachment_attach capability. `pad attachment attach <id> <item-ref>`
refuses against a server that does not advertise it, because an older
build's bare 404 would read as "attachment not found".
- The `upload -` help text, the attachment group help, both skill copies
and CLAUDE.md say an unattached image does not render on share pages.
- No MCP surface: pad_attachment has no upload.
Tests: 12 server/store tests, including an end-to-end share leg (404
before, variant bytes after), and 9/9 mutants killed. CLI capability
gate: 2 tests, mutant killed. e2e: the share renders the image after
attach; with attach made a no-op it goes red at "attached: no
placeholder".
Claude-Session: https://claude.ai/code/session_01NidgPePY7UvHcEJCMkD2Qz
|
||
|
xarmian
|
c29af5731c |
test(mcp): pin the initialize and tools/list wire bytes on both transports (TASK-2306 unit 1) (#1610)
Golden files for the initialize and tools/list results, from the production bindings: the real `pad mcp serve` subcommand for stdio, and registerRemoteMCP behind NewRemoteTransport for remote /mcp. The remote registration is extracted from the cloud branch of serverStartCmd into registerRemoteMCP, a mechanical move, so the test drives the binding rather than assembling its own server. It turns 'behavior-neutral' into a checked claim for the alias/funnel work that follows and for any SDK bump. Re-injecting BUG-2302 (destructiveHint true on read-only tools) fails both transports on the exact line. serverInfo.version is the only normalised field. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 |
||
|
xarmian
|
9b7e0832ac |
fix(comments): deleting a comment that has replies answers 409 comment_has_replies instead of 500 (BUG-3252) (#1608)
* fix(comments): deleting a comment that has replies answers 409 comment_has_replies instead of 500 (BUG-3252)
comments.parent_id references comments(id) with no ON DELETE, so deleting a
parent with replies failed the foreign key inside the store. Every door said
"internal error", the web delete button included, and nothing was deleted.
Per the lead's ruling (reversible with Dave's veto), the store now refuses
by name. The DELETE carries a NOT EXISTS on replies, so the refusal is part of
the statement itself, and a zero-row result is told apart by counting.
Nothing is deleted. Every door reports it:
- Both server routes answer 409 comment_has_replies with details
{comment_id, reply_count}.
- The CLI root writes a structured marker, so stdio MCP reports the code
and hint rather than server_error.
- Remote MCP passes it through the 409 allow-list with the same hint.
- The web timeline shows the server's message in place of "An internal
error occurred".
MCP 0.58 (behavior bump, on the v0.45 grounds). Cascade versus tombstone
stays an open product decision; this refusal is the floor under either.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
* fix(comments): a delete whose replies vanish between statements retries instead of reporting not-found (BUG-3252, codex r2)
Under Postgres READ COMMITTED the conditional DELETE and the reply count take
separate snapshots, so a reply the DELETE saw could be gone by the count, and
a comment that still existed was reported missing. The zero-row branch now
rechecks that the comment exists and, if it does, runs the conditional
delete again, at most three times.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
* test(comments): pin the replies refusal on Postgres (BUG-3252)
The server and MCP tests run on SQLite, so the conditional DELETE had never
executed on Postgres. This test runs there (make test-pg). Against the bare
DELETE mutant it fails with comments_parent_id_fkey, which is the original
defect.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
|
||
|
xarmian
|
0f1069e9df |
feat(comments): agents can edit and delete comments from the CLI and every MCP transport (TASK-2695) (#1607)
Web users could edit and delete comments since PLAN-1662. Agents, who write
most comments, could only append corrections.
- Server: new item-scoped routes, PATCH/DELETE
/workspaces/{ws}/items/{ref}/comments/{id}. They answer 404 unless the
comment is on {ref}, then run the existing handlers unchanged (edit is
author-only, delete needs item-edit). GET /server/capabilities advertises
them as item_scoped_comment_writes.
- CLI: `pad item comment-edit <ref> <comment-id> [message|--stdin]` and
`pad item comment-delete <ref> <comment-id>`. Both refuse to send to a
server without the capability, whose bare 404 would otherwise read as
"comment not found". `pad item comments` now prints each comment's id,
plus "edited" once its body has changed.
- models.Comment serialises a derived `edited` bool, true when
updated_at > created_at (the web timeline's rule).
- MCP 0.57: pad_item edit-comment / delete-comment and a comment_id param,
on stdio, remote /mcp and WebMCP. All three use the item-scoped routes,
so the check that the comment is on ref happens on the server for every
transport. Remote packages a delete's 204 as the CLI's JSON.
Found along the way, and out of scope: deleting a comment that has replies
answers 500, because parent_id has no ON DELETE. Filed as BUG-3252.
Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9
|
||
|
xarmian
|
2d51308d39 |
fix(collab): a schema-version rebuild sets unflushed edits aside instead of deleting them (BUG-3244 A) (#1598)
* wip(collab): the schema-version rebuild sets unflushed edits aside and the item stays marked (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip(server): set-aside rows refuse every content write and restore unless overwrite_pending_edits, which discards and reports them (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip: set-aside read/discard endpoints and pad item set-aside; census and migrated-table entries (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip(store): set-aside edits travel in the workspace bundle; the migrate gate asks the op-log only (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * wip: set-aside advice branches at every door that names a remedy; mark-only doors share one helper (BUG-3244 A, ruling 1) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(cli): set-aside notices name the right remedy (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * docs: the rebuild sets unflushed edits aside; freeze reason, API docs and op-log prose follow (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(web): identity-gate rows re-reviewed for the set-aside reason threading (BUG-3244 A) The edits in each covered unit are a pure pendingEditsReason(e) argument or string branch, or a reason passed to the dialog; no await, write or fence changed. Three anonymous-unit matchers follow the new call text, and the keepalive failure row lists the pure classifier in may. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(server): a restore override counts the set-aside rows it discarded (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * fix(store): the rebuild sets aside the rows its DELETE returned, so an append between choosing and clearing is not lost (BUG-3244 A, codex r1) On Postgres each statement takes its own snapshot and the setup lock does not stop a connected peer appending, so a row committed between the old INSERT ... SELECT copy and the DELETE was deleted without being set aside. The kept rows are now chosen from DELETE ... RETURNING. A test hook in the gap drives a Postgres leg; a mutant that chooses before the gap loses the row. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(web): playbook save row re-reviewed after the BUG-3230 U3 rebase (BUG-3244 A) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * fix(store): the set-aside move's first statement is the DELETE, so SQLite never upgrades a stale read snapshot (BUG-3244 A, codex r2) Reading the watermark first took a read snapshot on SQLite; a concurrent append would then make the DELETE fail with SQLITE_BUSY_SNAPSHOT instead of waiting. The watermark is now read after the DELETE, where a later value is only more accurate. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * feat(mcp): ToolSurfaceVersion 0.56, content_state gains superseded_set_aside (BUG-3244 A) The value reaches pad_item get / full list, pad_playbook run and the item resource's line above the body, and its remedy differs from applied_pending_flush (opening the item will not restore the edits). An additive bump with a version.go entry; README, CLAUDE.md and the MCP instructions follow. Read/discard of set-aside rows stays off the catalog. Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 * test(server): after a bumped rebuild the next open is granted the seed once and the item stays superseded (BUG-3244 A x BUG-3240) Claude-Session: https://claude.ai/code/session_016PXRzXhokr61CLbbu3Txj9 |
||
|
xarmian
|
b459892d8b |
fix(search): the collection filter resolves exact-first per workspace in scope, so a collection named task is no longer shadowed by tasks (BUG-2659, BUG-2839) (#1576)
* fix(search): the collection filter resolves exact-first per workspace in scope, so a collection named `task` is no longer shadowed by `tasks` (BUG-2659, BUG-2839) /search matched its collection filter as a literal slug, so shorthand found nothing and both clients normalised it first, which rewrote `task` to `tasks` even where a `task` collection existed. The handler now resolves the filter with resolveItemCollectionSlug in every workspace in scope and filters the store on the resulting collection IDs; one slug cannot express that, since the same input can resolve differently per workspace. The capability endpoint advertises search_collection_resolution, and the CLI sends the name as typed only to a server that does. The in-process remote MCP mapper stops normalising outright. Also corrects cascadeTitleRename's doc comment, which named links.ReplaceTitle for a cascade that rewrites through links.RewriteBracketsAt (BUG-2839). * chore(mcp): tool surface 0.55 — pad_item.search's collection resolves exact-first per workspace (BUG-2659) Lead-ruled BEHAVIOR bump on the v0.36/v0.40 grounds: in a workspace holding both task and tasks, the same call now answers from task. |