mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-11 13:28:57 +00:00
cbf2dd29c80958bf665933be69757e950523acdb
256 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 xarmian
|
2e9ace4194 |
docs: nine overclaims across code, metrics, docs and the CLI (BUG-2739, codex round 5)
A cross-artifact pass, which is the angle that keeps paying on this family. Every item below was a statement of mine that was false or unsupported; the code did not change. WRONG FACTS: - Watch epochs are opaque UUIDs, not numeric generations. I had copied internal/events' wording, where they ARE numeric — the distinction is the subject of internal/idspace's package comment. - undecodable_message was described as proof a notification was missed. The instance knows only that something it could not read arrived on its channel; it cannot tell whether that was ours. It stops vouching BECAUSE it cannot tell, which is a different and weaker claim. Corrected in four places. - The failover-cost paragraph said every SSE client on the instance reconciles. Wrong twice: a watch-bus resubscription ends the WATCH stream's coverage (activity coverage is per-workspace), and the one client that uses that stream today — pad watch --stream — answers sync_required by clearing its cursor and keeping the connection open, so it issues no request at all. Verified in cmd_watch.go rather than assumed. - The midstream/reset ratio is not fan-out in aggregate: the announcement counter also carries gaps and slow-subscriber drops and coalesces per connection. Only a reset observed in isolation reads that way. - 'The watch stream's only signal was a later non-contiguous notification' is true for a client HOLDING A STREAM OPEN. A reconnecting client was always covered, because a resume asks the shared counter instead of local state. Scoped in the doc and in the test header. - The dropped-confirmation fallback said coverage still ends. Usually, not necessarily: with no traffic during the outage nothing was lost, and if the drops continue through whatever would expose the hole and the stream goes quiet, nothing ever does — BUG-2727's boundary. Named both. - The new Observer Close warning was overbroad: reports run on the receive goroutine only on the RedisBus receive path, while a ResumeGap runs on the caller's and MemoryBus has no such goroutine. The rule stays unconditional, since a callback cannot tell which case it is in, but it now says why. STALE AFTER THIS BRANCH: - metrics.go's WatchSequenceResetsTotal comment listed two reset reasons. - observer_test.go said 'both reset reasons'. - The constructor comment named Channel() after the loop moved to ChannelWithSubscriptions, and did not say the Receive beneath it is load-bearing for that loop having no skip-the-first flag. It does now, and names the test that fails if it goes away. - cmd_watch.go's sync_required cause list predated BUG-2739 (and did not mention the mid-stream delivery BUG-2730 added). Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
b7ae022b6f |
refactor(events): every way of subscribing hands back the gap signal (BUG-2730, codex round 5)
Subscribe allocated and raised a gap channel its callers could not read, which round 5 called dead work. The read is right and the disposition is the other one: an interface method whose subscribers CANNOT be told they missed something is a silent under-delivery waiting for its first production caller, and internal/watchevents' Subscribe already returns the signal, so the asymmetry was the defect rather than the allocation. Subscribe now returns it too, on all three implementations. No production caller changes — the handlers use SubscribeIfAllowed and SubscribeAndReplaySince — so this is a test-call-site sweep plus one signature. |
||
|
xarmian
|
6afe683389 |
fix(events): do not arm reset detection where interleave is ordinary traffic (BUG-2736)
Codex round 9, from the 3am-operator angle. Six findings; one of them was a regression this diff would have shipped in the DEFAULT configuration, and the review framed it as a log-volume problem. THE REGRESSION. Phase 1 publishes with a two-call INCR-then-PUBLISH, so on any multi-instance deployment two publishers interleave routinely and a lower ID arrives after a higher one as ordinary traffic. main has no counter-backwards detection at all; this diff added it. Armed unconditionally, it would have fired on that ordinary interleave, dropped EVERY workspace's replay buffer, and resynced every client -- in phase 1, which is where every deployment sits until an operator flips phase 2. The check is now armed only once an epoch has been adopted. What that costs is stated rather than hidden: a genuine counter reset on a never-flipped deployment goes undetected, which is exactly the behaviour before this change and precisely the case phase 2 exists to fix. The new test asserts the gate, and also asserts what is NOT claimed -- the interleaved workspace's own buffer still holds ids out of order, so a cursor at the higher one reads as foreign. That is pre-existing, unchanged here, and strictly less harmful than a global drop; it is asserted rather than described so a future change to since() surfaces there. THE REST ARE THE OPERATOR'S SIGNALS, which were unreadable: - The effective phase was invisible. pad_event_sequence_resets_total cannot be interpreted without it -- a counter_backward rate is expected on phase 1 and an anomaly on phase 2 -- and the setting can arrive from an env var, a TOML file, or neither. It is now on the startup line as id_space_phase. - An unparseable PAD_EVENTS_PUBLISH_EPOCH was silently ignored, so an operator who typed "yes" believed they had flipped. Ignoring it stays the right behaviour; being silent about it does not. - Both publish-failure logs said only "failed to publish". They now say what the operator needs, which differs by phase: phase 1 may or may not have reached subscribers, and phase 2's script is atomic so it did not half-execute, but a lost reply means it may have published anyway -- do not re-publish by hand. - Adopting an epoch with empty buffers is the moment the documented residual becomes possible on that replica, and it happened silently. It now logs at INFO -- not a reset count, deliberately, since counting it would give the reset metric a per-deploy baseline. Declined with reasons: a cause label on the resume-gap counter and a publish failure counter are both pre-existing shapes rather than anything this diff changed, and the straggler log is already bounded by the recovery window. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
736a8c48f7 |
docs: thirteen claims about code that had moved under them (BUG-2736)
Codex round 5, cross-artifact consistency. Every one was a claim in a comment, help string, doc, or test name that the code no longer supported — and this diff created most of them by moving the code. The ones that would have misled an operator: - pad_event_sequence_resets_total documented ONE reason in both the Go doc comment and the Prometheus help text, and the deployment table said the same. It has emitted three since this branch. An operator reading the help string to build an alert would have alerted on a third of the signal. - deployment.md said every published message carries an epoch prefix. Phase 1 publishes bare JSON — which is the entire point of having two phases. - deployment.md said the first flipped message reaches each replica and every resuming client gets sync_required. A replica learns the epoch only from a message it RECEIVES, so only replicas subscribed to a workspace with traffic see it; and a replica with empty buffers adopts without dropping or counting, deliberately. - deployment.md said a restart's IDs cannot collide. internal/idspace documents a bounded case — the earlier process publishing more than 2^20 events per millisecond of its life. Stated as the bound it is, with the backwards-clock direction named as the safe one. - cmd_watch.go described sync_required as eviction-only. It has had four other causes since BUG-2731 and gained a fifth here. The ones that would have misled the next person editing this code: - bus.go said the Redis half was unwritten and a reset counter could still merge two ID spaces. It is written, three commits back on this branch. - bus.go and watchevents.go said in-memory IDs restart from 1. They count from an incarnation base. - redis_bus.go described this bus's epoch as an opaque uuid equivalent to the watch bus's, twice, after round 3 made it a Redis-minted generation. Only the watch bus still uses uuids. - observer.go said counter_backward happens only during mixed-version rolls. Phase 1's two-call publish produces it in steady state too. - redisns.go said the publish script spans four keys (it is five here now, plus a two-key assign script), and its hand-kept reserved-name inventory never gained event_epoch or event_epoch_gen — so a namespace equal to either would have nested one installation inside another's keyspace unrefused. - A test comment referenced idIncarnationShift, which moved to internal/idspace.Shift when the package was extracted. - Two tests called themselves process-restart tests while constructing successive buses in one process. They test bus incarnations; the comment now says so and says why that is the equivalent thing. And one reasoning error rather than a stale fact: the counter-backwards branch justified raising the floor by asserting the arriving ID is necessarily in the SAME numeric space. It is not — a phase-1 counter reset publishes low IDs with no epoch to explain them, which is a NEW space we cannot see. The behaviour is unchanged and still correct (the lead's day-52 ruling: raise unconditionally, prefer a loud bounded resync loop to a silent skip), but it now says what it actually knows, which is nothing, and names the cost on a real phase-1 reset. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
d393126d80 |
test(events): close the gaps a tests-as-production-code pass found (BUG-2736)
Codex round 4, on the tests themselves. Eight findings, all real; three of them were behaviours in this diff with no test at all. NO TEST AT ALL: - The real receive path. Every reconciliation test drove fanOutFromRedis directly and the publish tests read the wire with a raw subscriber, so a regression that decoded the epoch correctly and then handed 0 to the fan-out would have passed all of them -- reconciliation silently never running in production. Now driven through Subscribe/Publish and back through Redis, with the mutation checked. - The atomic script's ordering claim. Every phase-2 test published once or ran the script sequentially, so a two-call INCR-then-PUBLISH implementation passed them all -- and that ordering is load-bearing, because the receive path reads a descending id as a counter reset. 300 concurrent publishes now assert arrival order equals id order; verified to FAIL 5 of 5 against a two-call implementation and pass 3 of 3 against the script, so the instrument discriminates rather than merely being green. - The TOML tag. The env-var test proved PAD_EVENTS_PUBLISH_EPOCH reaches the field and said nothing about the toml:"events_publish_epoch" tag -- the exact form the rollback procedure warns about, since a file value outlives an unset env var. PASSING FOR THE WRONG REASON: - The production config wiring was still unexecuted: passing an empty config.Config at both RunE call sites compiled and passed everything. The source-text guard that already counts those call sites now also requires them to pass the loaded config. - The phase-2 wire assertions accepted a well-formed payload with an empty event body. They now assert the body survives. - The Redis metrics subtest had no served-resume control, so a bus that refused every resume would have passed. It now round-trips a publish through Redis first. - TestResumeGapIsReportedForBothWaysOfNotServing never proved ws-warm HAD a buffer, so its second half could silently duplicate its first. - internal/server's cold-resume tests still sent a literal 4200, which the incarnation guard now answers before the handler's no-buffer path is reached. My own round-1 sweep of this class stopped at four packages and never looked at internal/server: reviewer-named instances are a sample (team CONVE-18), and so, evidently, are self-named ones. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
a9544a57ba |
test(events): repair the resume tests the base guard made vacuous (BUG-2736)
Codex round 1 named three sites; the class was six, across four packages. Every MemoryBus test that spelled out a cursor as a small literal now passes through the incarnation guard before reaching the branch it is named for. The worst were the two that exist precisely to distinguish branches: the both-ways-of-not-serving observer test would have gone green with BOTH of its branches deleted, and the watch bus's eviction test would have gone green with eviction deleted. Cursors are now base-relative or read back from what the bus issued. Where the test has only the EventBus interface and no access to the base, the cursor is derived from a published event's id instead. Mutation-checked in the direction that matters: with the no-buffer branch, the coverage check, and the eviction check each made inert in turn, the tests named for them fail. Before this commit they did not. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
4a6a748c85 |
feat(events): identify the shared Redis ID space, behind a two-phase flip (BUG-2736)
The activity event counter lives in Redis and is shared by every instance, so no instance can compute an identity for it the way MemoryBus computes its own incarnation base. If that counter is ever reset -- evicted under maxmemory, deleted by hand, a fresh Redis after a restore -- IDs start again from 1, and a replica buffering the old sequence cannot tell the new 101 from the old 101. It merges two ID spaces into one replay buffer and answers a resume across the boundary as though nothing was missed. Numeric detection alone cannot see it. By the time the new sequence passes the replica's high-water mark it looks like ordinary progress -- which is the case the epoch exists for, and the high-water check is what catches the OTHER case (a publisher that never learned the epoch), so both are kept. So the identity travels WITH each message, as an opaque token in a "<epoch>|<id>|<json>" prefix. A prefix rather than an envelope field: an older instance would unmarshal an envelope object SILENTLY -- no matching keys, no error, a zero-valued Event delivered to its clients -- and fails loudly on the prefix instead. TWO PHASES, because the failure is asymmetric. Every instance ACCEPTS both wire forms from this release; only emission is gated, on PAD_EVENTS_PUBLISH_EPOCH. Phase 1 rolls the binary everywhere publishing the historical bare JSON; phase 2 sets the flag and rolls again. Flipping before every instance is upgraded is the one direction that LOSES events rather than resyncing: a pre-phase-1 binary cannot parse the prefix at all. Rollback is symmetric and safe. docs/deployment.md carries the procedure both ways, what the reset counters should read during each roll, and what remains unfixed. Phase 2 also moves ID assignment into one atomic script. The two-call INCR-then-PUBLISH lets two instances interleave, so a receiving instance can append 6 before 5 -- a window older than this change, and already wrong, but load-bearing here because counter-backwards detection reads a descending ID as a reset. The script carries a dedupe token for the same reason internal/watchevents' does: go-redis retries a command whose REPLY was lost, so a publish can happen AND return an error, and the retry would deliver a second copy that looks perfectly valid. THE COUNTER-BACKWARDS FLOOR STAYS, and the earlier hope that this unit would delete it was wrong. Its trigger is mixed-VERSION ordering -- an older binary assigning and publishing in two calls -- not mixed-FORMAT payloads, so publish-old-until-flip removes the format window only. It lives for as long as a deployment can run two publisher versions at once, which is every rolling upgrade, and the code now says so where it fires. THE ASYMMETRY WITH MemoryBus IS DECLARED IN BOTH BUSES, in both packages: an opaque epoch where the counter is shared, a numeric base where one process owns it. They are not two spellings of one idea and must not be symmetrized. A numeric base for Redis would close more -- it would refuse cross-incarnation cursors, which the epoch cannot -- and is deferred rather than rejected: at the flip, IDs would jump to ~1.8e18 in one step and every un-flipped publisher's message would read as a massive backwards jump, dropping every buffer across the whole roll. It is a candidate follow-on once the flip has soaked. What this does NOT fix is stated in the code and the docs rather than implied: the client cursor is still a bare integer with no epoch, so an old and a new ID of the same value remain indistinguishable TO A RESUME even though the buffers can no longer mix them. The flip is read inside newObservedEventBus, which now takes the whole Config. As a hand-picked argument at the two RunE call sites it was untested wiring: replacing it with `false` compiled, passed the entire tree, and left the deployment silently on phase 1 -- indistinguishable from a correct phase-1 deployment, since phase 1 is the default. Mutation-checked in both directions, because a helper that ignores its config and hardcodes either value would pass a one-directional test. Also: the epoch and dedupe keys join the namespace assertions (an epoch shared between two installations is a cross-feed with teeth -- each would read the other's ID-space changes as its own), and this package's four-key EVAL is now recorded on BUG-2724's cluster deferral, which had one call site and now has two. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
c017ad359d |
fix(events): give each in-memory bus incarnation its own ID space (BUG-2736)
Both in-process buses assigned Last-Event-ID values from a counter that restarted at 1 on every process start. A client holding cursor 2 from a previous incarnation could reconnect to a restarted server, pass every coverage check BUG-2731 added, and be replayed the NEW space's 3, 4, 5 as though they followed the OLD space's 2 -- silently missing everything the dead space held above 2. Nothing local could tell the two 2s apart. The cursor carries no epoch, and in internal/events per-workspace IDs are non-consecutive by construction, so "did we issue this ID?" was numerically undecidable. The four adjacent levers were checked rather than assumed: comparing in memory has nothing to compare against; persisting the counter makes single-process Pad carry durable event-bus state and still resets on data loss; refusing cursors we did not issue is the undecidable one; and a nonce on a second channel is unavailable because EventSource echoes Last-Event-ID and nothing else, and cannot rewrite its URL on an automatic reconnect. So the ID space's identity goes in the ID's VALUE while its FORMAT is unchanged: still a bare int64, still ParseInt on the way back. internal/idspace mints a base of processStartUnixMilli<<20 and each bus counts up from it. Two incarnations can only collide if the earlier process published more than 2^20 events per millisecond of its own lifetime -- a deterministic bound, not the probabilistic one BUG-2736's body rules out. A CAS makes bases strictly increasing within a process too, which the clock alone does not do for two buses constructed in the same millisecond. A backwards clock step degrades in the SAFE direction: a lower base puts old cursors ABOVE the new buffer's newest ID, so they are refused rather than answered wrongly. The overflow bound is computed, not estimated: the last start instant that fits is 2248-09-26T15:10:22Z. Each bus then answers the resume question exactly instead of inferring it: a non-zero cursor at or below this incarnation's base was issued by a dead space. That is strictly stronger than the coverage check alone, which serves the ADJACENT cursor on reasoning that only holds within one ID space. In internal/watchevents the check lives in one helper both entry points call. Written inline in EventsSince it was absent from SubscribeAndReplaySince -- the path the SSE handler actually uses -- so the component was fixed and its wiring was not (team CONVE-19). A test now drives both. web's ItemEvent no longer declares `id?: number`. Nothing read it, which is the only reason it was harmless; a base of ~1.8e18 is past JavaScript's MAX_SAFE_INTEGER, so the first reader would have silently got a rounded number. Defused while still unread. Tests that spelled out IDs now read back what the bus assigned -- a literal 1 is a cursor from a dead space, which turned two negative controls into their own opposite. The two watchevents guards (cold buffer, dead incarnation) are tested separately, because a single test covering both would keep passing with either deleted. The Redis half is not here. Its counter is shared across processes, so identifying its ID space needs an epoch travelling with each message; that is the next commit on this branch. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
9f88e94832 |
fix(events): a resume must not be answered from coverage we never had (BUG-2731)
internal/events answered a Last-Event-ID resume with an empty-but-non-nil
slice whenever the workspace's replay buffer could not speak to the span
being asked about. The SSE handler reads that as "caught up", so the client
sat on a live stream believing it was current while everything between its
cursor and now was silently gone.
COVERAGE. replayBuffer gains knownFrom: the lowest event ID from which this
instance's coverage of a workspace can be vouched for. A resume from below
it answers nil, which the handler already turns into sync_required. Covers
a buffer that does not exist (cold start, restart, scale-up, or simply the
first connection to a workspace on this instance), a buffer that exists but
starts above the cursor — NOT full and NOT empty, reachable on any
multi-instance deployment with no eviction and no restart — and a non-zero
cursor from a previous incarnation of a single process.
knownFrom here means RECEIVING-continuity, never ID-contiguity, and the
defining comment says so with the measurement attached.
internal/watchevents has a field of the same name that ALSO detects holes
by noticing a non-consecutive ID; porting that would have been a serious
regression, because this bus has a global counter and per-workspace
buffers, so a workspace's buffer holds non-consecutive IDs by construction
(four publishes alternating across two workspaces measure as W=[1 4],
X=[2 3]). An ID-contiguity check would fire on nearly every append and turn
every resume into sync_required — the false-positive inversion of this bug.
LIFECYCLE. Coverage now ends where it really ends:
- a stopped workspace subscription drops its replay buffer. Keeping it
"in case they come back" looks like a free win and is the bug: events
published elsewhere never enter it while it goes on looking complete.
- subscriptions are generation-numbered, so a straggler from an ended
subscription cannot re-create a buffer and vouch for coverage that
ended with it — including the case where the workspace has already been
resubscribed under the stale goroutine.
- a pub/sub reconnect ends that workspace's coverage. PubSub.Channel
resubscribes transparently, so a Redis failover left a hole the buffer
had no idea about; the loop reads pubsub.Receive instead. It must
RECOVER rather than exit — returning on a transient error would leave
an instance publishing fine and receiving nothing — and it drops ONE
workspace's buffer, since a dropped subscription says nothing about any
other channel.
Subscribers are indexed by workspace because the replay buffers moved under
the same mutex (necessary for the straggler race): scanning every local
subscriber under that lock would make one hot workspace the serialization
point for every other workspace's fan-out and every resume.
Also removes Publish's local-counter fallback on a failed INCR, which
minted an ID from a process-local space and published it — every receiving
instance reads that as the counter having been reset. It bought nothing:
this bus has no local fan-out path, so an event that does not reach Redis
reaches no subscriber here either.
SIBLING. internal/watchevents had the identical cold-resume defect on its
MemoryBus — its RedisBus guards it, MemoryBus reached the buffer directly —
so a single-process instance answered a post-restart resume as caught up.
Found by a cross-artifact review pass; the guard goes in `since` so both
implementations inherit it, and is tested through SubscribeAndReplaySince
as well as EventsSince because that is the path the handler uses.
Refs BUG-2731
|
||
|
xarmian
|
bb003dd6bb |
fix: five claims the final comment-truth round found (BUG-2724, BUG-2726)
The bounded process the lead set: N rounds, an author prune pass, one final comment-truth round. This is that round's output, and the loop stops here. Two were mechanisms I had wrong, and both are the kind a reader would reuse without re-deriving: - "Different Redis DB numbers do not help" was half true. Ordinary keys ARE DB-scoped, so two installations on different DBs keep separate presence registries; it is pub/sub that ignores DBs entirely, which is why the buses cross-feed regardless. Stating it as "does not help" made the namespace look like the only fix for a problem it only half is. - A namespace cutover's client resync was attributed to the epoch check. That check needs an OLD epoch to compare against and a freshly namespaced bus has none — the resync comes from the cold replay-buffer coverage check instead (knownFrom is zero, so every resume falls below it). Same honest outcome, different mechanism, and the mechanism is what someone reasoning about a cutover would use. Three were stale or over-general after earlier changes: the admission comment still said the global limit is passed to the bus as 0 (that parameter is gone), `pad watch --help` and the plugin monitor description lumped a missing .pad.toml's hourly retry in with the 5s-to-5min backoff, and CLAUDE.md said clients must back off without the browser exception docs/deployment.md spells out. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
461c5a3e3d |
refactor: prune the claim surface, and turn one prose claim into a test
The review could not converge on this diff's comments because each round of corrections re-expanded the surface it was reviewing — rounds 16 and 17 found errors inside 15 and 16's fixes. That is a production rate being measured, not a backlog being drained, so the treatment is to write fewer claims rather than review the same ones again. PRUNED, ~135 comment lines: process narration. "An earlier version said X", "found by mutation testing", "codex round N caught this", the scoreboards. Every one of those is already in a commit message, which is where the archaeology belongs; in the source they are claims a future reader has to verify, about a past that no longer exists. KEPT, because they earn it and a reader would otherwise re-derive them: metric semantics, reachability boundaries, what a test does and does not discriminate, why the obvious alternative was rejected, and the hazards that cannot be enforced in code. MOVED TO A TEST, per the rule this run earned the hard way: a comment asserting countable behaviour belongs in the suite. Two test comments in internal/watchevents relied on "this constructor waits for its SUBSCRIBE to be confirmed" — prose, and the same assumption applied to the OTHER bus (which subscribes asynchronously) is what made a namespace test flake. It is now asserted with no polling and no sleep, and the mutation that removes the wait fails it. That rule generalises and is why round 17's find mattered: "counts every unservable resume" was prose, so its falseness could hide a real metric gap. Prose is for claims that cannot be asserted. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
35e564298b |
fix: seven more prose claims, one real metric gap, and a flaky test of my own (codex round 17)
The prose angle again, and it is still finding things — which is itself the finding: this diff's comment density is generating wrong beliefs faster than the review is removing them, in the one dimension where the defect is a reader's understanding rather than the program's behaviour. Everything below was a claim I wrote. ONE WAS A REAL GAP, not just wording. pad_watchevents_resume_gaps_total was documented as counting every unservable resume, and counted only the half decided by the shared counter. The LOCAL half — a cursor below what this instance can vouch for, from a hole or a cold start — returns nil from replaySince, becomes sync_required for the client, and reported nothing. Now counted, on the deferred path so it fires with the lock released. Its test needed a second pass to be an instrument: the first version arranged a hole and asserted the counter moved, but the shared counter disagreed too, so resumeOutrunsLocalView reported and the mutation survived. It now sets the counter to AGREE with what the instance has seen, which is the only arrangement that isolates the local path. The prose corrections, swept by grep rather than by instance this time: - MemoryBus's comment said a single-process deployment never wires an observer. cmd_server wires one, deliberately — that is what makes the drop counter meaningful there, which is a claim I had just added elsewhere. - "Every write path works with Redis down" was too strong in three places. Push answers 503 for an unresolvable targeted push and 502 push_unconfirmed on publish failure — the paths whose job IS cross-instance delivery. - Presence-failure consequences were stated as certainties in four more places after round 16 fixed one. A failure means an error was REPORTED; Redis can fail a pipeline after applying it. - The deployment metrics table still described pad_eventbus_publish_total as "Events published" after the Help string had been corrected to attempts. - The reserved-namespace rationale called prefix nesting a "collision". It is nesting; an exact collision would need the namespace to match a workspace UUID. Refused anyway, and now for the reason that is true. - A presence cutover was described as stranding one renewal interval of stale entries. It is the full 90s TTL — three intervals. AND A FLAKE OF MY OWN, caught by the full suite rather than by the targeted runs: the activity-bus namespace test asserted subscription state immediately, but that bus subscribes ASYNCHRONOUSLY (the watch bus waits for confirmation; the two differ). It now polls, and the asymmetry is named in both tests so the next reader does not assume symmetry the way I did. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
2fa1316853 |
fix: three claims round 15's corrections got wrong or missed (codex round 16)
Reviewing the corrections found three more, which is the honest shape of this: the prose angle keeps paying because the errors are in prose. - My round-15 correction said the Redis counters "stay at zero" on a single-process binary. That is wrong for one of them: pad_watchevents_notifications_dropped_total moves there, because MemoryBus has the same slow-subscriber drop and is wired to the same observer. So the comment was wrong before AND after, in opposite directions. It now says which counters are Redis-only by construction (everything sequence-related — MemoryBus assigns contiguous ids and has no subscription to lose) and which are not, and a test pins both halves. - "The three keyspaces cannot drift" survived in cmd_server.go. Round 15 fixed the copy in redisns.go and not this one — a two-member class, fixed one member, which is team CONVE-18 for the second time in this branch. - The presence-failure consequences were stated as certainties. Redis can fail a pipeline or a script AFTER it applied, so a failure means the operation reported an error, not that it did not happen. Now phrased as what a failure risks. Codex's reserved-namespace audit came back complete: the set covers every current suffix root (watchevents:pub: is covered by watchevents), every configuration path goes through Parse, all three production constructors receive the parsed value, and the exact-match controls do not reject names that merely contain a reserved word. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
9d54f24626 |
fix(server,cli): the half of round 12's fix I missed (BUG-2726)
Codex round 13, unanchored, found that my previous commit fixed one of the two refusal paths on /api/v1/events. The admission check moved above the SSE headers; the PER-WORKSPACE check stayed below them, so half the 429s on that endpoint still carried the JSON error envelope under Content-Type: text/event-stream — the exact defect the commit said it fixed. Team CONVE-18 in its own shape: the reviewer named one instance, I fixed that instance, and the class had two members. The enumeration I owed was "how many ways can this handler refuse", and it takes ten seconds to read. Every refusal is now above the header block, with a line saying nothing below it refuses. The contract test made the same omission and is the reason this reached another round: it drove the admission bound on both endpoints and never the per-workspace one, so it agreed with a handler that was half fixed. It now enumerates all three refusal paths, and the mutation that reintroduces the defect fails it by name. Also from round 13: `pad project watch`'s 429 message named the two knobs that cover both streams and omitted PAD_SSE_MAX_PER_WORKSPACE, which is the one most likely to be the cause on a busy workspace — true as far as it went, and pointing the reader away from the answer. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
3e3170e915 |
fix(server,cli,docs): the consumer contract, per codex round 12 (BUG-2726)
An angle no earlier round took: what does a CLIENT see. Two of the five
findings were about consumers I had never opened.
- `pad project watch` returned "event stream returned 429: {json}" and
exited, which sends the reader looking for a bug rather than at a
limit. It now says what happened and which knobs govern it, and names
the fact that those knobs cover this stream and the agent watch stream
together. It still exits rather than backing off — it is interactive,
and a human can decide — unlike the unattended monitor, which already
folds 429 into its ladder.
- Both endpoints now answer a refusal through one helper: same status,
same code, same message, plus `Retry-After`. `/api/v1/events` was
setting `Content-Type: text/event-stream` BEFORE the admission check,
so its 429 carried the JSON error envelope under an SSE content type —
a different contract from its sibling's for the same refusal. Admission
moved above the headers, which is where it belonged anyway.
- The anonymous-caller rule was documented as if it applied to both
endpoints. It applies to `/api/v1/events` only; the watch stream
requires a resolved user and answers 401 without one.
- docs/architecture.md described one SSE endpoint and one bus. It now has
the table: two streams, two buses, different scopes and consumers, one
shared connection budget, one Redis namespace.
FILED, not fixed: the web UI's `EventSource` cannot see a 429 or a
`Retry-After` — the spec exposes neither to the page — so a refused
browser tab reconnects at a constant rate while the CLI backs off. That
asymmetry means reaching the limit sheds load from the population that
respects it and not from the one that grows fastest under it. No
server-side change closes it; the fix is a client-side reconnect wrapper.
BUG-2733, and docs/deployment.md warns operators to size the limit with
it in mind.
Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X
|
||
|
xarmian
|
ec70f13608 |
refactor: act on codex round 8's scope review (BUG-2727)
The "should this be here at all" angle, which my own record says I do not reliably ask of my own work. Six findings; one was a real inconsistency, the rest were claims that needed stating rather than code that needed removing. REMOVED: the presence observer's interface, adapter type and constructor, in favour of a plain callback. One method, one production consumer — and the same diff already uses bare callbacks for RedisHealth and the stream gauge, so this was inconsistent with itself. internal/watchevents keeps an interface because it reports five distinct conditions; one does not earn one. TRIMMED: .env.example's per-variable prose down to the upgrade-relevant facts plus a pointer at docs/deployment.md, which is canonical. The same policy was restated in seven artifacts and that is a drift surface. KEPT, with the reason written where a reader will ask: - The receive-loop-exit counter is expected to stay at zero, and that is what it is for — a should-never-fire alarm on a state undetectable from outside the process (an instance that publishes fine, answers health checks and receives nothing). BUG-2727 filed the silent return as the defect, and a log line nobody greps is not the same artifact as a counter somebody alerts on. - The prober's synchronous first probe duplicates cmd_server's dial-time ping. Deliberate: reusing that result would couple this type to its caller's startup sequence for one round trip that runs once per process. The consequence is now stated too — because the dial-time ping is FATAL, the prober's "unreachable at startup" branch cannot fire in the shipped binary. - The keyspace wiring guard parses source and will break on a rename. The alternative on offer needs three packages' constructors collapsed into one API. A guard that costs a one-line update after a deliberate rename beats an invariant with no enforcement, which is what the package comment alone amounts to. RAISED WITH THE LEAD, not decided here: events.EventBus.Publish's global limit parameter is now dead in production, since the handler passes 0 and the process-wide gate owns that bound. Removing it is the clean seam and it is an interface change in a shared package, which is a structural call. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
a790810bd6 |
docs: close the cross-artifact gaps codex round 6 found (BUG-2724, BUG-2726, BUG-2727)
The angle no earlier round probed: which artifacts a human or agent CONSUMES should have changed and did not. Five, and the pattern is the one my own record keeps naming — the caveat existed in the artifacts I was editing and not in the ones that get read. - .env.example had neither new variable and still described PAD_SSE_MAX_CONNECTIONS with its old single-endpoint meaning. It is the file an operator copies; docs/deployment.md being right does not help someone who never opens it. - docs/deployment.md called the readiness endpoint /health/ready. The route is /api/v1/health/ready, so every instruction to go read the new redis block pointed at a 404. Corrected there and in four code comments, and the Health Check section now actually shows the three endpoints, the healthy payload, and the degraded one — it previously demonstrated only /api/v1/health, which is the build-info endpoint and says nothing about readiness. - CLAUDE.md listed /api/v1/events and not /api/v1/events/stream at all, so the endpoint this unit bounds was undocumented in the file agents read first. Added, with the limits and the 429 contract. - `pad watch --stream --help` said silence means "no workspace linked or padd unreachable". A capacity refusal now produces the same silence through the same backoff, so the help was enumerating a set that had quietly grown. - The plugin skill told agents "silence means nothing changed" — now false in the same way, and worse, because an agent repeats it to a user as though the quiet were evidence. Rewritten to say what silence does and does not prove. The plugin monitor description had the same enumeration and got the same fix. Checked rather than assumed: there are two SKILL.md files, and only the plugin copy carries a notifications section — the embedded one has no monitor guidance to correct. NOT changed, and raised with the lead instead: deploy/k8s/deployment.yaml points both probes at /api/v1/health, so the readiness endpoint is never consumed. Fixing it is right but it changes rollout behaviour for anyone using the shipped manifest (a database blip would start pulling pods from the load balancer), which is a deployment-posture call rather than part of this unit. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
9afedbe1a0 |
fix(server,metrics,watchevents): seven codex round-4 findings — operator and next-author angle (BUG-2727, BUG-2724)
Round 4 read the diff as the operator of a running deployment and as the author of the next change. Five findings were claims my own text made that the code does not support, which is the failure mode this angle is for. 1. The degradation list said Redis loss costs "cross-instance activity events". It costs ALL of them: events.RedisBus.Publish logs its failure and returns without a local fan-out, so subscribers on the originating instance stop receiving too. A responder told only about cross-instance delivery would have looked elsewhere. Corrected in the health payload, both prober log lines, and the docs. 2. config.go promised that connected clients resync after a namespace change. True of the watch stream, false of the activity stream, whose cold replay buffer answers a resume as "caught up" (BUG-2731). The docs already carried the asymmetry; the comment did not, and the comment is what the next author reads. 3. Resume-detected gaps were counted nowhere. They are the only gap shape that is always USER-VISIBLE — the client gets sync_required — so an incident reading pad_watchevents_sequence_gaps_total would have missed the failure mode with the clearest symptom. New pad_watchevents_resume_gaps_total, kept separate rather than folded in because the two are diagnosed differently: one is a delivery fault, the other is any cursor this instance cannot vouch for. 4. The presence-failure metric's doc said every failure leaves sessions unlisted and untargetable. Two of the four ops fail in the OPPOSITE direction — a failed deregister leaves a dead session listed, so a push aimed at it is accepted and reaches nobody — and a generic alert on the total would send a responder the wrong way. Now documented per op, in the code and in the docs table. 5. The go-redis log bridge levels everything at WARN, and the comment justified that with "benign reconnect chatter" I had never enumerated. Enumerated now: the stream carries genuine failures, state changes and informational fallbacks with no severity attached. WARN stays — INFO would bury the dropped-message line the bridge exists for, and classifying by message TEXT would make Pad's log levels depend on go-redis's prose — and a component=go-redis field makes it routable instead. 6. internal/redisns centralizes key construction but cannot stop a future contributor wiring one bus with a different Keys than another: every package compiles, every unit test passes, and the deployment runs split across two keyspaces while looking configured. Adds a wiring drift guard that reads cmd_server.go and fails if the three constructors do not share one Parse-produced value. The rule was already written down in a package comment; this is its enforcement step. 7. The limits are per-process and the startup log, log fields and gauge Help called them "global". Renamed to per-instance / per-principal throughout, with the no-shared-counter caveat in the startup line. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
2b33184ef1 |
fix(metrics,watchevents,server): three codex round-1 findings (BUG-2727)
1. pad_redis_up was registered unconditionally, so a deployment with no Redis exported a permanent 0 — which reads as "Redis is down" to anything scraping it and would have every single-process binary alerting on a dependency it does not have. It now registers only inside the PAD_REDIS_URL branch, matching /health/ready, which already omitted its redis block on the same condition. My own field comment claimed the absent behaviour while the code did the opposite. 2. The receive loop could report a false exit during shutdown: Close cancels the context AND closes the pubsub, and Go picks between ready select cases at random. A context re-check makes the outcome independent of that. Scope stated honestly, because it is narrower than the finding implies. With the guard removed, 200 Close cycles under publish traffic produced zero false exits — and removing it AND reversing Close's ordering still produced none, because Close waits on the receive goroutine and the goroutine observes the cancelled context either way. So no test fails if these three lines are deleted, and both the code comment and the test doc say so rather than implying coverage that does not exist. It is kept as defence against a future reordering, not as a fix for observed behaviour. 3. Corrupt session entries returned a list error without incrementing the failure counter, so pad_session_presence_failures_total under-reported precisely the case an operator is least likely to find another way — a dead Redis is obvious, a corrupt row is not. Both corrupt shapes now count. The non-string arm is unreachable through MGET (Redis answers nil for a key holding a non-string value, verified), so it is annotated as defensive and the test says no leg drives it instead of quietly covering only the reachable one. Test-power notes are measured, not asserted: the Close test catches removal of the select's ctx case (mutation-verified) and does not discriminate the guard. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
0877b260c1 |
feat(redis): namespace every Redis keyspace from one shared config value (BUG-2724)
Every Redis key and channel Pad uses was flat — pad:events:, pad:event_seq, pad:watchevents*, pad:session:* — so two Pad installations pointed at one Redis endpoint cross-feed each other's notifications and merge each other's session-presence registries. Different logical DB numbers do not help: Redis pub/sub is not namespaced by DB at all. The exposure is narrow but real. Delivery is filtered per caller on user id, and user ids are per-installation UUIDs, so cross-feed needs the same id in both installations — a CLONED database, such as a staging environment restored from a production dump. For that case it is a genuine cross-tenant leak: foreign sessions listed in the picker, and a private push deliverable across installations. Fixed the way internal/watchevents' existing ruling demanded: not by one package growing a prefix the others lack, but through internal/redisns — one value parsed in cmd/pad/cmd_server.go and passed into all three constructors. The three cannot drift because there is nothing to drift from, and the operator rule is stateable in one sentence for every keyspace. PAD_REDIS_NAMESPACE defaults to empty, which reproduces the historical names byte for byte, so an existing deployment keeps addressing its own replay buffers, counters and presence entries across the upgrade. Tests assert both directions per keyspace — present under the namespace AND absent under the historical names — because an implementation that wrote both would still cross-feed while passing a one-directional test. Namespaces are validated at startup, and a colon is rejected specifically: it is Pad's own separator, so namespace "a:events" would build pad:a:events:<ws> and collide with installation "a"'s channel — reintroducing the cross-feed through the mechanism meant to fix it. Names are built through a function rather than assembled from a literal at each site, and redisns' doc says why: "pad:" also begins Pad's OAuth SCOPE values (pad:read / pad:write / pad:admin) in four files, so a grep-driven prefix sweep would break authorization. Not included, deliberately: hash tags for Redis Cluster. BUG-2724's trail recommended shipping them alongside on cost-sharing grounds; that premise is falsified by publishScript, which spans four keys in one EVAL and fails CROSSSLOT exactly as presence's MGET does. There is no cheap half, and no cluster client here to exercise tagged keys against, so they would ship untested by construction. Cluster stays documented as unsupported and the future unit is named on the trail. Renaming is a CUTOVER for the buses (the seq and epoch keys carry Last-Event-ID meaning, so connected clients resync) and free for presence (90s TTL). Both stated in docs/deployment.md and at the constructors. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
720b792176 |
feat(server,config): bound the watch-events stream, with one budget across both SSE endpoints (BUG-2726)
GET /api/v1/events/stream had no concurrent-connection limit of any kind. PAD_SSE_MAX_* gated only /api/v1/events, and the API rate limiter caps how FAST connections are opened, not how many are HELD — so one authenticated user could hold arbitrarily many streams, each costing a goroutine, a bus subscription and, since BUG-2698, a presence registration in shared Redis. The bound is a process-wide admission gate rather than a second per-bus limit. Each bus can bound its own subscribers atomically and events.EventBus already does, but neither can bound the two together, and a held connection costs the same process resources whichever endpoint opened it. A global limit on one bus would have let a user exhaust the machine through the other while every configured limit still read as satisfied. So PAD_SSE_MAX_CONNECTIONS now covers BOTH endpoints and is passed to the events bus as 0. That is a deliberate re-point of an existing knob, ruled rather than assumed: an operator who tuned it for one endpoint is now bounding both and may reach the limit sooner. A knob that silently bounded half the connections it named is the worse failure — invisible — where this one announces itself and is tunable. A startup log line reports the effective limits and which endpoints each covers, so the change is visible without reading release notes. New PAD_SSE_MAX_PER_USER (default 50) applies to both endpoints. The global bound alone lets one user exhaust the process for everyone, which the per-workspace limit cannot prevent — the watch stream has no workspace to count against. Per-workspace stays /api/v1/events-only for the same reason. Refusal is 429 sse_limit_exceeded, matching the existing endpoint. The CLI monitor folds any non-200 into its backoff ladder (linear, 5s base, 5min cap, reset on connect), verified rather than assumed, so a refused stream backs off instead of spinning. Deliberately NOT a registry-side cap: PR #1175 added one in review round 17 and removed it in round 21, because it bounded one of three resources a held stream consumes, was never hard (admitted renewals must bypass it), and cost delivered_sessions its honesty. The admission check is upstream of all of that — refusing costs one connection instead of making a live session untargetable. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
8dea9abca3 |
feat(watchevents,metrics): operational observability for the Redis notification bus (BUG-2727)
The watch bus detects four conditions an operator would want to alert on — a notification dropped for a slow local subscriber, a gap in the received id sequence, an id-space reset, and the receive loop stopping — and until now reported all four to slog and nowhere else. Log lines are not alertable without someone already looking, and the last of the four was not even logged: the loop returned silently, leaving an instance that publishes fine and receives nothing indistinguishable from a quiet workspace. Adds watchevents.Observer, an adapter seam rather than a bus wrapper. The events.EventBus wrapper shape does not work here: every condition is detected on the RECEIVE path, inside the bus, and is invisible at the Bus interface — a wrapper can count publishes and subscribers, but not a notification that never arrived. Two corrections to BUG-2727's filing, both verified against go-redis v9.22.0 rather than assumed: - Its proposed fix — "re-subscribe rather than exiting where the cause is recoverable" — would be dead code. PubSub.Channel's message channel is closed ONLY on pool.ErrClosed; every other receive error is retried indefinitely, and a health-check goroutine pings every 3s and reconnects on failure. So go-redis already does the re-subscribing. The exit gets an ERROR log and a counter instead, which is what the condition actually needs. - The genuinely silent path is go-redis DROPPING messages when a subscription's 100-deep buffer stays full past its 60s send timeout, logged only through go-redis's own logger. Pad cannot count that directly, so it is reported by its CONSEQUENCE (a sequence gap) and its cause is made visible by routing go-redis's logger into slog. Observer's doc comment states that boundary, so a gap is not misread as evidence of any particular cause. Session presence gets the same treatment for the same reason: it is fail-soft everywhere by design, so its failures have no user-visible signal beyond a push that quietly reaches fewer sessions than it should. The renew counter is deliberately NOT throttled where its log line is — throttling the metric would make it under-report during the incident it exists for. Tests assert the CONDITION increments the counter, not that the counter exists, and each asserts its own premise first (a healthy subscriber reports nothing; contiguous ids report nothing; a cold start reports nothing) so a bus that reported on every notification could not pass. The receive-loop test drives the real closed-client condition rather than calling the reporter. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
ea139272ce |
fix(server,watchevents): shared session presence + honest push acceptance (BUG-2698, BUG-2699) (#1175)
Two coupled defects in the push path, fixed as one unit because 2699's honest-acceptance signature is the substrate 2698's fix reports through. BUG-2699 — Bus.Publish reports acceptance. The endpoint returned 200 pushed:true for a publish that was dropped, because Publish returned nothing and swallowed every failure. An error is two outcomes and they are kept apart: ErrBusClosed proves nothing was published (503 unavailable, safe to resend), while any other error means UNCONFIRMED — go-redis retries a command whose reply was lost, which is why the publish script already carries a dedupe token — and gets 502 push_unconfirmed, deliberately off the web client's safe-to-resend list. MemoryBus was the worse case, not the exempt one: neither implementation checked `closed`, and the in-process one dropped silently with no log at all. Seven production call sites, not the six the item named; the six best-effort producers discard through one named helper, and an AST-based test fails when a new producer publishes directly. BUG-2698 — RedisSessionPresence. A session-targeted push was resolved against the answering replica's presence registry, and the handler skips the publish when the target is absent, so a POST landing on A for a session held on B dropped the instruction and answered delivered_sessions:0. Fixed at the REGISTRY rather than the gate: a shared registry makes the snapshot right, which makes the picker complete and restores the gate's original premise, so the existing skip becomes correct for the reason it was written. Entry and index are written atomically under a TTL renewed by a goroutine that lives exactly as long as the connection; a crashed process stops renewing and Redis clears it. Staleness is unchanged and now stated in full: ~30s for a dropped client, ~90s for a dead instance. delivered_sessions becomes nullable — null means published-but-uncountable, never zero — documented as three states at every consumer. 35 Codex review rounds. Notable: a per-user registry cap was added and then removed after three consecutive rounds found defects inside it and a fourth was asked whether it belonged in this PR at all; a context bound was documented, disproved by its own test (go-redis does not apply a command context to connection establishment — 5.0s measured against a 150ms ctx), and rewritten to say what is true. Every fix was mutation-checked; one instrument was deleted for passing on broken code and one for not asserting its own premise. Filed rather than folded in: BUG-2724 (Redis keyspace namespacing + Cluster), BUG-2725 (delivered_sessions is an estimate with error in both directions), BUG-2726 (no concurrent-connection limit on the watch stream), BUG-2727 (Redis absent from readiness/metrics; silent subscriber loss), BUG-2728 (epoch-reset resume lead). Gates: build · make lint 0 issues · go test ./... (25 pkgs) · svelte-check 0 errors · vitest 1738 passed · CI 7/7 including Go (PostgreSQL) and Nix. |
||
|
xarmian
|
6a37512227 |
feat(server): outbox drain — webhooks delivered from the choke point (TASK-2714) (#1173)
* test(store): pin the events/1 taxonomy as an independent copy (TASK-2714)
TestCanonicalEventsAreFullyDeclared iterated kernelevents.Canonical() and
asserted each entry resolved something non-empty. That check cannot fail for
any table the compiler accepts: eventSpec requires both fields, so a corrupted
table — an entry deleted, an entry added, item.deleted quietly rebased onto the
ref-only payload — passed its own validation. A test that agrees with whatever
the table says is not a test of the table.
The sixteen name/subject/family triples are now written out as literals, so the
test DISAGREES with the table when the table moves. The wire strings behind the
name constants are pinned separately, because the triple map is keyed on
literals and a renamed constant would otherwise slip through as long as the
constant and the table moved together.
Ordered as this unit's first commit because TASK-2714 edits that table (the
handler-path bulk mapping): an independent copy earns its keep at the moment of
the edit, not before.
Mutation matrix, 4/4 caught: drop member.joined (17 -> 15 count mismatch and a
missing-name error), rehome item.deleted onto ref_only (family mismatch),
rename ItemMoved's wire string to item.move (constant leg), add an undeclared
item.frobnicated entry (count + undeclared-name + non-canonical legs). The
fourth reported "survived" on its first run because the sed never matched the
table's alignment — the mutation was verified present in the file before the
result was believed.
TASK-2714 requirement 4 (lead pass on #1172).
* feat(store): max-age prune for undispatched outbox rows (TASK-2714)
Requirement 3's missing half. PruneDispatchedOutbox filters on dispatched_at
IS NOT NULL, so a row that can never be delivered — a workspace whose only
webhook was deleted, an endpoint that 4xxs forever — is unreachable by it and
keeps its frozen payload indefinitely.
That matters because SPEC-3 makes payload privacy TEMPORAL. An outbox payload
is a frozen snapshot and account deletion's de-identify posture reaches only
live rows, so the retention window is the whole privacy claim; a window only
one of its two halves can close is not a window.
The trade is stated in the doc comment rather than left to be inferred:
at-least-once holds WITHIN the retention window and not past it, which is why
the caller's max-age must be far larger than any retry schedule. Deleting
rather than stamping the rows dispatched is deliberate — a dispatched stamp
would be a lie in the durable record, and this table is the only evidence of
what the kernel emitted.
Mutation matrix, 2/2 caught: drop the dispatched_at IS NULL clause (prunes the
aged DISPATCHED row too, handing retention two owners with different windows),
drop the occurred_at cutoff (prunes a young pending row a retry is still
owed). The test asserts its own premise — all three seeded rows are confirmed
present before the survivor checks, which would otherwise pass for a reason
unrelated to the prune.
No caller yet: the drain loop wires it up in the next commit.
* feat(events): derive SSE names from the taxonomy; retire item.updated_with_comment (TASK-2714)
SPEC-3 §"the choke point owns the canonical→surface name mapping". SSE's
snake_case vocabulary and the webhook dot-form vocabulary drifted because
nothing tied them together — each was hand-passed at its own call sites. This
ties them.
v1.5 pins what "derive" means: NAME derivation, not delivery path. SSE stays
direct-published at the mutation site, because it carries request-scoped
attribution (Actor / ActorName / Source) that a frozen outbox payload
deliberately does not hold; only its NAME now comes from the taxonomy. Moving
SSE behind the drain is TASK-2722.
- eventSpec gains an `sse` field — ONE table, not a second map, for the reason
round 11 of the last unit established: a separate map can disagree with the
first and fails open exactly when it matters. Empty is a real value (attachment,
member and pack events have no SSE surface) and SurfaceSSE reports false for it,
so silence can't be mistaken for a name.
- Several canonical events derive the SAME SSE name — status_changed and moved
both surface as item_updated — because the SSE vocabulary is coarser than
events/1 and the UI never distinguished them. The finer name is what the
webhook wire and bindings get.
- The 12 canonical SSE publish sites take their names from derived package vars,
resolved AT INIT. Every call site is a compile-time constant, so a missing
surface is a startup panic rather than a per-request decision between "log and
drop" and "publish under an empty name".
- handlers_item_links.go keeps the events.ItemUpdated literal, commented: link
mutations are silent in events/1 (v1.5), so there is no canonical name to
derive from. TASK-2723 carries link.created / link.removed.
- item.updated_with_comment retired (v1.2, Dave's ruling). One producer deleted;
the events.ItemUpdatedWithComment constant deleted with it — it had no producer
and no web consumer (grepped .go/.ts/.svelte), so leaving it would leave a name
a future publisher could reach for.
The compat guard is what makes this a refactor rather than a wire change:
TestDerivedSSENamesMatchTheLegacyWireVocabulary asserts each derived name equals
the events.* constant clients are pinned to. A derivation producing
"item.created" or "item_deleted" would break the live UI while every other Go
test still passed.
Mutation matrix, 3/3 caught: rename item.deleted's SSE surface to item_deleted
(both the taxonomy test and the compat guard fail), split item.moved onto its own
SSE name (same), make SurfaceSSE return (spec.sse, ok) so no-surface events fail
open (the taxonomy test's silence leg names all four). Running total 9/9.
go test ./internal/server ./internal/store ./internal/events: all green.
* feat(webhooks): synchronous DeliverEvent seam with per-endpoint outcome (TASK-2714)
Requirements 1 and 2. Dispatch returns once its per-hook goroutines are
spawned and reports nothing, so a drain built on it would stamp rows
dispatched while the HTTP requests were still in flight — losing exactly the
events the outbox exists to make unlosable. DeliverEvent blocks and tallies.
- Delivery carries WorkspaceID / EventID / Event / OccurredAt / Payload.
OccurredAt is the EVENT's timestamp, not dispatch time: SPEC-3 pins
time-relative binding predicates to it, so stamping time.Now() would make
every consumer's notion of when a mutation happened depend on how backed up
the queue was. Payload is json.RawMessage — []byte would base64 the snapshot
into a string that is valid JSON and completely unusable.
- WebhookPayload gains ID, the consumer dedupe key SPEC-3 §Delivery guarantees
already told consumers to use. Before this, that instruction named a field
nobody could see. omitempty, because the "webhook.test" ping is not a kernel
event, has no outbox row, and must not invent an id.
- DeliveryOutcome counts rather than a status, because one event fans out to N
endpoints and the answers differ. Three distinctions the drain branches on:
Matched==0 is SUCCESS (a webhook-less workspace is owed nothing; reading it
as undelivered would back up every event in every such workspace until
retention deleted it); Permanent does not hold the event pending (re-sending
to an endpoint that will reject it again costs the queue its progress);
Transient does. Retryable() states the ack rule once instead of letting each
caller re-derive it.
- A returned error is reserved for the SERVER's failures — listing hooks,
marshalling. Those must not ack: nothing was attempted, so the event is
still owed in full.
- Dispatch keeps its async shape for its one remaining caller and says so.
deliver() now returns the outcome it always computed; the async path
discards it.
Mutation matrix, 6/6 caught: stamp dispatch time instead of occurred_at; drop
the envelope id; pass the payload as []byte (base64); deliver asynchronously
and assume success (the synchronous leg names it exactly); count a permanent
rejection as transient; swallow a store failure into a zero outcome (the test
prints the outcome that would have acked an undelivered event).
Running total 15/15. go test ./internal/webhooks green.
* feat(store): batch_id correlation for handler-path bulk mutations (TASK-2714)
F2's write half. A lane-wide bulk action is a handler LOOP over per-item store
mutations with no enclosing transaction, so each member writes its own
canonical outbox row — which is what keeps SPEC-3's per-member binding
evaluation free, and also means that without a marker the drain would put 200
item.deleted events on the webhook wire for a 200-item lane archive: exactly
the flood TASK-1668's batch event exists to prevent.
RECORDED, NEVER INFERRED (SPEC-3 v1.5). The schema-free alternative was
grouping pending rows by workspace and a time window, which would fold two
unrelated single updates into somebody's bulk event whenever they landed in
the same tick. A wire event saying "these five items changed together" is only
true if something recorded that they did.
- migrations 082 / pgmigrations 060: nullable event_outbox.batch_id, no FK
(a batch is not a row anywhere, it is a name the handler minted), plus a
partial index on the pending set.
- store.MutationOption / WithEventBatch: variadic, because every existing call
site is a single-item mutation with nothing to declare and making all of them
pass a zero value would bury the one case that matters.
- The handler mints one id per bulk OPERATION, before the loop and
unconditionally — deciding mid-loop whether a run "counts as" a batch would
make the correlation depend on how far the loop got.
POPULATION CORRECTED: my escalation said four store methods; it is FIVE.
archive (DeleteItem), restore (RestoreItem), move (MoveItemWithPreCheck), field
update (UpdateItemWithPreCheck) and assign (UpdateItem) are the complete set of
mutating store calls handlers_items_bulk.go makes — restore was the one I
missed, which is CONVE-18's exact lesson arriving one level up. The test drives
all five rather than sampling, because the failure is per-method: a signature
that accepts the option and never threads it compiles, passes everything else,
and silently un-batches one of the six bulk verbs.
Mutation matrix, 5/5 caught across the four distinct emit sites: drop the stamp
on the update path (both Update legs fail), on delete, on restore, on move. The
delete mutation first read as SURVIVED — it had made the package fail to BUILD
(opt then unused), and the grep for test-level FAIL lines printed nothing. The
compiler catch is the stronger result, but the instrument mis-reported it, so
it was re-run with opt kept alive and the test named it directly.
go test ./internal/store ./internal/server green.
* test(server): anchor the SSE compat guard to the client's literal strings (TASK-2714)
The guard compared the derivation against events.* — the Go side. A
coordinated rename of the taxonomy AND the constants passes that, and is
exactly the change that breaks the browser: the client is pinned to the
STRINGS, in web/src/lib/services/sse.svelte.ts's ITEM_EVENTS.
The wanted column is now a literal copy of what the client listens for, with
the file named. events.* is asserted alongside as a second leg, so a drift
between the Go constants and the client is attributed rather than merely
reported. Same disagree-with-the-table principle as the taxonomy test, one
layer out: this file has to be edited by hand when the wire vocabulary
intentionally changes, and that edit is when someone goes and changes the
client too.
Mutation matrix, 2/2, each hitting only its own leg: rename events.ItemCreated
to the dot-form with the taxonomy untouched (drift leg fires), and make the
taxonomy publish the dot-form on SSE (browser leg fires). Running total 22/22.
Lead's catch on the day-49 review of commit
|
||
|
xarmian
|
402f79e016 |
feat(store,server,web): collection kernel traits — de-hardcode conventions/playbooks slugs (TASK-2657, BUG-2702) (#1171)
Implements SPEC-5 §Collection traits (approved v1.1) — the first unit of
PLAN-2656 phase 0. Three kernel behaviors were keyed on the literal collection
slugs "conventions" and "playbooks": what the agent bootstrap loads, which
items route by invocation slug, and which items export as portable artifacts.
Collections now DECLARE those behaviors and the kernel resolves them from the
declarations.
Fixes the KERNEL half of BUG-2702, which stays open for the rest (see below).
A slug is not a stable identifier —
UpdateCollection re-slugs on any name change, and renaming a collection is a
documented onboarding step (TASK-1510) — so renaming either collection silently
detached all three behaviors from it, with the items still present and no error
anywhere. Measured on origin/main before the fix: conventions and
convention_index dropped 1 -> 0, playbooks 1 -> 0, and GET /playbooks/{slug}
went 200 -> 404, so `/pad ship` stopped resolving with no sign the playbook
still existed. Both halves are locked by regression tests observed failing on
unfixed code.
BUG-2702 is NOT fully closed here, deliberately. Every kernel behavior follows
the trait, and library activation on the MCP dispatcher and CLI was converted
too — but the pack's own dedicated web routes (/conventions, /playbooks list and
detail, /library) still address their collection by literal slug and render
empty after a rename. Filed as BUG-2705 with the route paths and the likely fix
shape; 2702 closes when that lands. Degradation there is bounded: no data loss,
and the collection stays usable at its own /[collection] route and in the
sidebar.
SPEC-5 was amended to v1.1 BEFORE any code, per the spec tree's own discipline:
bootstrap_include becomes a LIST of {mode, filter, key} because v1.0 could not
express convention_index at all; the conventions filter is now normative and
includes status=active, which v1.0's shorthand omitted and which the
implementation does enforce (implementing v1.0 literally would have leaked
draft conventions into every agent's boot payload); v1 filters are field-
equality maps with query/1 named as the widening path, since SPEC-2 is phase 1
and PLAN-2656 forbids growing toward it; and invocation_field is constrained to
the literal `invocation_slug`, because any other field name falls outside the
partial unique indexes in migrations/054 and pgmigrations/033 that are the real
uniqueness guard.
Traits get their own column rather than a key inside the schema JSON. The
schema column is overwritten wholesale on update and every client rebuilds it
fields-only, so a traits key stored there is destroyed by one ordinary
collection edit — measured during this task, not assumed. Trait authority
cannot rest on a value an unrelated UI save deletes. UpdateCollection writes
traits only when explicitly supplied, so pre-existing clients leave them alone;
an explicit "{}" still clears.
Bootstrap keeps its three payload keys as first-party views fed from the
declarations, and gains a generic bootstrap_includes array for any other
declared key — so the boot surface is genuinely generic rather than three
hardcoded payloads, and no consumer breaks.
Existing workspaces are backfilled slug-keyed in both dialects, guarded on
traits='{}' so a re-run cannot clobber a workspace's own declarations. The
backfill inherits today's blind spot (a workspace that renamed the collection
before upgrading is not reached) but cannot do worse than the status quo, which
is itself slug-keyed; from the backfill forward the hazard is structurally gone.
Malformed declarations are refused at create and update rather than stored:
an unparseable blob degrades to "declares nothing", which is silently the wrong
behavior instead of a loud error (SPEC-0 L6).
Web groups agent-facing collections by bootstrap_include presence, replacing a
hardcoded two-slug array repeated at five call sites.
Not done, deliberately: no MCP catalog change (traits are first-party kernel
declarations, no agent needs to set them, and the separate column means
pad_collection.update passes through harmlessly — no ToolSurfaceVersion bump);
bootstrap's collections[] projection does not carry traits (PLAN-1410 trimmed
that payload and nothing consumes them there); prefix.go's NormalizeSlug is
untouched (a pure function with no workspace context, and de-hardcoding two of
its six slugs would make it less coherent, not more).
Eight Codex review rounds found nineteen real defects, all fixed here. The
serious one:
bootstrap_include filter keys FAIL OPEN. The item store's field-filter path
drops any key its sanitizer rejects, removing the predicate rather than matching
nothing, so a declaration filtering on `"stat us"` would narrow nothing and ship
every convention — drafts included — to every agent at boot, defeating the
status=active guarantee this change makes normative. Filter keys are now
validated against the store's own sanitizer shape and pinned by a cross-package
agreement test, since models cannot import store and a future divergence would
silently reopen it. SPEC-5 amended to v1.2 with the rule and its fail-open /
fail-closed asymmetry. Also fixed: an unknown declared artifact_kind reached
artifact.Encode and surfaced as a 500 (now a 400 at the export boundary, since
SPEC-5 permits unknown kinds as legal non-round-tripping declarations); and
workspace import validated traits as JSON only, so an archive could persist a
declaration that degrades to "declares nothing" (now validated, degrading to
"{}" with a warning rather than refusing an import that may be the only copy).
Later rounds found more, and several were defects this change itself created.
A hidden collection could SHADOW a visible one: resolution used to name exactly
one collection, so with several declaring, resolving across all of them and
rejecting afterwards on visibility made a visible playbook unreachable behind a
hidden one — candidates are now filtered by visibility before selection, in both
playbook resolution and artifact import. Importing a pre-traits archive produced
an INERT workspace: the migration backfill cannot reach rows inserted long after
it ran, so conventions/playbooks arrived declaring nothing, and canonical
declarations are now inferred from the slug when a collection declares none
(never overriding declarations that survived the round trip). The generic
include path had no L4 boot budget and is now capped with an overflow count.
Trait parsing claimed to be strict but json.Decoder ignores trailing bytes, so
`{...} garbage` parsed cleanly. First-party payload keys are now mode-pinned,
since their projections have fixed shapes and declaring the other mode would be
silently ignored. Duplicate artifact_kind / invocation_field declarations are
refused at the collection API, and a conflicting archive warns on import.
Agent-facing text was updated with the rest, not after it: SKILL.md,
instructions.md and the MCP catalog said the literal slugs, which is exactly the
artifact an agent acts on. ToolSurfaceVersion 0.24 -> 0.25 for the
pad_library.activate behaviour change.
Trait uniqueness is a documented BEST-EFFORT gate, not an invariant, by lead
ruling. The gate reads then writes without a lock, import bypasses it, and a
rename can mint a duplicate without touching that path. The database-level
enforcement (partial unique indexes on the extracted traits) cannot ship first:
existing deployments can already hold duplicates via rename-then-reseed, so the
index would fail the migration on precisely the databases that most need
repairing. TASK-2710 carries the de-duplication pass and the indexes; SPEC-5
v1.3 records the deferral and the reason. L6's requirement that conflicts fail
loud is met by the refusal plus the warning — the mechanism is deferred, the
principle is not.
Gates: build · make lint 0 issues · go test ./internal/... · make test-pg ·
svelte-check 0 errors · vitest 99 files / 1734 tests. Mutation-verified across
four matrices, 20 mutations, 19 caught; the survivor is a seeding path whose
trait-vs-slug difference is unreachable today (SeedCollectionsFromTemplate
creates any missing template collection before it seeds items), recorded on the
task trail rather than papered over with a test that proves nothing.
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
|
||
|
David Barkhausen
|
5784d907c0 |
feat(cli): PAD_TOKEN environment override for stored credentials (#879) (#1160)
* feat(cli): PAD_TOKEN environment override for stored credentials (#879) Layer 1 of #879: if PAD_TOKEN is set, the CLI uses it as the bearer token and skips the credential-store lookup — gh's GH_TOKEN convention. Reads never write credentials.json, so a read-only override sidesteps the multi-agent identity contention completely; the store is never touched under the override. Per the acceptance grounding notes: - NewClientFromURL resolves PAD_TOKEN before the per-server store lookup (the single token-attachment chokepoint). - whoami no longer lies under the override: it skips the store short-circuit and reports the effective identity via a real /me fetch, with an 'Auth: PAD_TOKEN environment override' line. - auth login/logout print a gh-style stderr notice when the override is active. logout additionally pins its server-side session invalidation to the STORED token — an unpinned Logout() after the constructor change would have invalidated the env token's session — and skips the server call when there is no stored session. - pad init's status line and server info's report disclose the override (env_token_override field; the auth probe uses the token every other command would use). Zero behaviour change when PAD_TOKEN is unset. Token minting stays web-only; a minimal 'pad token' CLI is offered as a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): review round 1 — init fails on a rejected PAD_TOKEN; login shortcut skipped under the override; logout asymmetry documented Per the PR #1160 round-1 review: - Bug 1: pad init's auth step no longer falls back to stored credentials when a set PAD_TOKEN is rejected — it fails with the distinct rejected-token message (mirroring whoami), which also makes the status line's override disclosure truthful. Test drives the real padInitCmd flow and asserts the stored identity is never consulted. - Bug 2: login's 'Already logged in as <stored user>' shortcut is skipped when the override is active — it reads the store, and firing it right after envTokenNotice contradicted the notice. A second test pins the unchanged no-override shortcut behaviour. - Doc ask: the deliberate logout asymmetry (the env token's own session is never invalidated; its lifecycle belongs to the minter, GH_TOKEN posture) is now stated in env_token.go's doc comment and the README PAD_TOKEN section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
xarmian
|
25c7cd20f5 |
feat(watchevents): Redis-backed bus so watch notifications cross instances (BUG-2651) (#1167)
* feat(watchevents): Redis-backed bus so watch notifications cross instances (BUG-2651) internal/watchevents shipped MemoryBus only, so in a multi-instance deployment a notification published on instance A never reached a stream held open on instance B — watches appeared to work and silently dropped. Bus was an interface from day one for exactly this; adding RedisBus changed no producer and no consumer. NOT A MECHANICAL PORT of internal/events.RedisBus. Three deliberate divergences, each documented at the point someone diffing the two files would call it a mistake: - ONE channel and ONE replay buffer, because this package has exactly one logical stream by contract (DOC-2479 DR-2: all per-caller filtering happens in the consumer). Most of the template's bookkeeping — per- workspace counts, subscriptions, buffers — has nothing to key on here. - EAGER subscription for the bus's lifetime, not lazily on first local subscriber. The replay buffer fills from the RECEIVE path, so a lazily torn-down subscription stops filling it at precisely the moment before a Last-Event-ID resume — for one harness monitor holding one stream, that makes resume structurally useless. The template can afford lazy because per-workspace means N idle subscriptions; here it is one. - ONE mutex across subscriber membership and the replay buffer, held through the whole local fan-out. The template uses two and offers only separate Subscribe + EventsSince, which cannot provide SubscribeAndReplaySince's guarantee. Copying its locking would have handed back the double-delivery window this package's interface exists to close. Publish fails CLOSED when INCR fails, where the template falls back to a local counter. Two instances falling back at once mint ids from independent counters into a shared stream, and replayBuffer.since() reasons on monotonicity — so the damage is silent replay corruption, not a visible error. INCR and PUBLISH share a connection anyway, so the fallback mostly lets a doomed publish proceed carrying a poisoned id. Both load-bearing tests were VACUOUS as first written; the mutation matrix is the only reason I know: - the concurrency test's producer finished before the subscriber joined, so the channel leg was never exercised and a split-lock mutant survived 50 iterations. Now paced, with a both-legs-non-empty precondition that fails a run which never approached the boundary, plus a dedicated detector (600 attempts, 8/8 kills, 0.02s after switching the drain to non-blocking — exact, because the duplicate is already buffered when the call returns). - the fail-closed test asserted nothing was delivered, which is true of the fallback too: Publish never delivers locally, so with Redis down neither policy delivers. Rewritten around a go-redis ProcessHook that records attempted commands, which is where the policies actually differ (INCR-then-stop vs INCR-then-PUBLISH). Also corrects session_presence.go, which told the next person these two had to be fixed together. Delivery is now cross-instance; the registry's under-report is unchanged, so the remaining defect is a picker that under-reports rather than a push that lies. The PLAN-2558 S3 gate stays, for that reason instead of the old one. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): make id assignment and publish atomic; close the bus on shutdown (Codex round 1) P1 — INCR and PUBLISH as two client calls are not order-preserving, and the failure is concrete: A gets id 1 and is descheduled, B gets id 2 and publishes, A publishes 1. Every subscriber receives 2 before 1, the replay buffer appends in ARRIVAL order, and replayBuffer.since() reasons on monotonicity — so a resume from 2 hits the sinceID > newestID branch and answers 'gap too large', turning a healthy reconnect into a spurious sync_required, while a resume from 1 silently skips the late arrival. Fixed at the source with a Lua script: Redis runs it atomically on its single thread, so INCR and PUBLISH for one instance both complete before another's script begins, and publish order equals id order globally with no coordination on our side. The id rides as a '<id>|<json>' prefix rather than being edited into the JSON from Lua; the id is digits and the FIRST '|' separates, so a '|' in the body is unambiguous. A pleasant consequence: there is no longer a window where an id exists but the publish has not happened, so the fail-closed decision and the publish decision became the same decision. P2 — Stop() never closed the watch bus. That was survivable for MemoryBus, whose Close only drops channels; RedisBus holds a receive goroutine and a Redis subscription from construction, so it leaked both for the process's life. Closed after bg.Wait(), so a background producer cannot publish into a bus already tearing down. nits, all real, all in artifacts someone reads: - 'exactly-once delivery' was simply wrong. Redis pub/sub is at-most-once and the local send is deliberately non-blocking. The property the round trip actually buys is NO DOUBLE DELIVERY to the publishing instance; the comment now says that and names the replay buffer as the bounded recovery mechanism for the rest. - the Bus interface comment still said only MemoryBus existed. - cmd_server.go's session-presence note still claimed the same caveat as 'the watch bus directly above', which had just stopped applying. - session_presence.go now says delivery is fixed WHEN PAD_REDIS_URL is set, rather than unconditionally. Tests: the fail-closed assertion moved from 'nothing was delivered' — still true under the two-call version — to 'no bare INCR or PUBLISH was issued', which is what distinguishes atomic from not. Mutation-verified by splitting the script back into two calls. Added a decode round-trip test covering the new wire format, a '|' inside the body, and four malformed payloads, since that decoder consumes bytes from a channel any holder of the Redis credentials can publish to. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents,server): correct the targeted-push claim; close the bus before HTTP shutdown (Codex round 2) P2 — I claimed cross-instance DELIVERY was fixed. Half true, and the false half was mine to catch: handlers_push.go gates a session-targeted push on the LOCAL presence registry and skips the publish entirely when the id is not there, so a POST landing on A for a session held on B still delivers nothing. The bus would carry it; the gate means it never reaches the bus. Broadcast pushes and every other notification kind ARE fixed. I asserted that behaviour from reading the bus and session_presence.go without reading the push handler — the exact thing I hold myself to not doing. Corrected in all three places the claim was made (the package doc, session_presence.go, and the KindPush comment), with the correction recorded rather than quietly overwritten. The gate's own justification is now stale too, and worth more than a tweak: 'a target this instance cannot see is a guaranteed no-op' was TRUE under MemoryBus and is FALSE under RedisBus, where another instance may hold that session. Left in place deliberately — publishing unconditionally would fix delivery and immediately make delivered_sessions=0 a lie in the other direction, which is a question about what that field promises. It belongs with the shared-state SessionPresence that PLAN-2558 S3 already gates on: fixing the registry makes the snapshot right, and then the skip is correct again for its original reason. Both open halves collapse into that one implementation. P2 — the watch bus was closed only in Server.Stop(), which runs AFTER http.Server.Shutdown. The event bus is closed before Shutdown precisely so its SSE handlers unblock; the watch stream is the same shape, so an open one would have held Shutdown to its full 30s deadline. Now closed alongside eventBus, with the Stop() close kept as the path for other callers — both implementations are idempotent. nit — MemoryBus and RedisBus disagreed after Close: RedisBus handed a late Subscribe an already-closed channel, MemoryBus registered one nobody would ever close, so a consumer racing shutdown blocked forever. MemoryBus now matches, and its Close is idempotent, which the CLI's double close relies on. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): report a missed notification as a replay gap (Codex round 3) P2 — a divergence MemoryBus structurally cannot have. It assigns every id itself, so its replay buffer is contiguous and the only gap it can report is eviction. RedisBus receives ids over at-most-once pub/sub, so a blipped subscription can miss 101 and receive 102: the buffer holds a hole, is nowhere near full, and replayBuffer.since() answers a resume from 100 with just [102]. The consumer loses a nudge and is never told. RedisBus now tracks the id at which the sequence resumed after the most recent hole, and answers nil — the same signal eviction already gives, which the SSE handler already turns into sync_required — for a resume that would have to span it. Resumes that do not span it still replay normally, and sinceID=0 is treated as a fresh subscriber rather than a resume, so a hole nobody spanned is not turned into a spurious resync. The atomic publish script is what makes this readable: publish order is id order globally, so a non-consecutive id means MISSED, not reordered. Mutation-verified by disabling the check; the test fails on both the spanning resumes and would have failed the over-broad version too (it asserts the non-spanning resumes still work). Two residuals documented rather than fixed, both because the fix is the same shared-state SessionPresence that PLAN-2558 S3 gates on: - delivered_sessions is now wrong in BOTH directions for a broadcast push — the count is local while delivery is global, so a replica can report 1 while two sessions receive it, or 0 while a remote one does. No local arithmetic fixes that; it is asking one replica what all of them are doing. - the Redis channel and counter names are not deployment-scoped, so two installations sharing a Redis endpoint cross-feed (and picking different logical DBs does not help — pub/sub ignores them). Left flat to match internal/events rather than giving one of the two buses a prefix the other lacks; the rule is one Redis endpoint per installation, and relaxing it should cover both buses at once. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): a cold-started replica must report a gap too (Codex round 4) P1 — the round-3 hole check only fired BETWEEN two received messages, so it never fired for the first one. A replica restarting while Redis is already at 101 has an empty buffer; its first received message is 102, nothing looks like a hole, and a client reconnecting to that replica with Last-Event-ID 100 was handed [102] — skipping 101 exactly as silently as the case round 3 fixed, by a different route. Replaced contiguousFrom with knownFrom: the lowest id from which this instance's buffer is contiguous. SET on the first append (before which this instance knows nothing) and RESET on every hole (before which it no longer knows anything usable). One variable, both failures. The boundary is pinned in both directions, which is what stops this being an over-broad 'always gap after a restart': a resume from exactly the id before our first (101 when we started at 102) IS contiguous with our view and replays normally. Mutation-verified by disabling the cold-start arm. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): idempotent publish, confirmed subscription, and real Redis tests (Codex round 5) P2 — go-redis retries a command whose reply is lost to a network error, and the publish script was not idempotent: the same notification would be published twice under two different ids. Both copies look valid — ordered, distinct — so nothing downstream could tell them apart, and on the push path a duplicate is a duplicate DISPATCH into an agent harness. The script now takes a caller-generated token and SET NX's it, so a retry carrying the same arguments returns 0 without publishing. TWO THINGS THIS UNIT OWES ITS TESTS, both found within minutes of each other and both invisible to the hermetic ones: 1. The idempotency script shipped indexing ARGV[3] while Publish passed two arguments. Caught by re-reading, which is not a control worth relying on for the next Lua edit. 2. NewRedisBus returned before go-redis had established the subscription, so notifications published in that window were lost to this instance, silently. Surfaced as a test flake; the production shape is a rolling deploy, where a replica takes traffic before its subscription is live. The constructor now waits for the confirmation (bounded, and a failure is logged rather than fatal since Channel() re-subscribes on reconnect). So miniredis is now a test dependency, and the round-trip tests it enables cover what fanOutLocally-driven tests structurally cannot: the channel name, the KEYS/ARGV mapping, the id prefix wire format, the shared counter across two buses, cross-instance delivery (the actual bug), the dedupe token, and Close tearing down the SERVER-side subscription rather than just local channels. Verified by restoring the ARGV[3] bug: the round-trip test fails on it. The two findings I am NOT fixing here are unchanged and documented where the reasoning is met — the targeted-push gate and delivered_sessions are both consequences of the per-process presence registry, and both are closed by the shared-state SessionPresence that PLAN-2558 S3 gates on, not by anything in this package. make vuln: 0 vulnerabilities in imported packages. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): survive a Redis counter reset without replaying stale ids (Codex round 6) P2 — pad:watchevents_seq has no TTL but can still vanish: evicted under maxmemory, dropped by a FLUSHDB, or restored from an older snapshot. Ids then restart at 1 while this instance's ring still holds the hundreds. Keeping both is what corrupts replay — the two id spaces are not comparable, so a resume from 2 in the NEW space would be handed the stale 99/100/101 as though they were newer. A backwards id now drops the replay buffer and re-anchors knownFrom. Every resume from the old space then exceeds the newest id held and gets nil — the resync signal that is the only honest answer once the ids stopped meaning what the client thinks they mean — while clients in the new space keep working immediately. The test asserts BOTH halves, which is what makes it a detector rather than a description: a build that logged the reset and kept the buffer passes 'the old resume reports a gap' and fails 'the new resume never returns a pre-reset entry'. Mutation-verified on exactly that. Hardened while I was here: the epoch-reset path REBUILDS the buffer at runtime, so a bus constructed with a non-positive replay size would have turned a counter reset into a panic (newReplayBuffer(0)'s first append indexes a zero-length slice) rather than a resync. The constructor now normalizes. MemoryBus has the same trap for a caller passing 0; left alone as pre-existing and off this path, but named in the comment rather than silently fixed or silently ignored. nit — this file's header still claimed there was no miniredis dependency and no round-trip coverage, which the previous commit made false. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(watchevents): actually correct the hermetic test header (Codex round 7) The previous commit's message claimed this fix. It did not contain it: the edit ran as one of two scripts in a single command, its assertion failed with a traceback, and the second script's success is what I read. The header kept saying there was no miniredis dependency and no round-trip coverage — both false since two commits ago, in the file a reader consults to find out what IS covered. That is the adjacent-success-signal failure exactly: a success line from the step next to the one I cared about. The tell was in the output and I walked past it, then asserted the change in a commit message. Recording it here rather than quietly fixing, because a commit that claims a change it does not make is worse than one that omits it. Verified this time by reading the file back and grepping for the stale phrases: zero. Round 7's other three findings are the documented residuals re-raised for the third time — the targeted-push gate, delivered_sessions, and the unnamespaced Redis keys. All three are dispositioned at the line a reader meets them, all three are consequences of the per-process SessionPresence registry or of matching internal/events' existing convention, and none is fixable inside this package. They stay open, on the record, and with the lead. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(watchevents,cli): correct pad push --help; document the reset-window residual (Codex round 8) nit, and the one that stings — cmd_push.go's Long help still said pushes go over the 'in-memory watch-events bus'. That is the text a user reads when they run pad push --help, and it has been false since this branch's first commit. I have a standing pre-push step to grep the artifacts a CONSUMER reads for exactly this, and I ran it as a code search (watchevents.New) rather than a prose search, so --help never came up. The help now distinguishes broadcast (reaches every instance) from session-targeted (still resolved against the handling server) and names the bug. P2 — the counter-reset handling fires when the first post-reset notification ARRIVES, so there is a window between Redis losing the counter and the next publish in which this instance still replays old ids to a reconnecting client. Documented as accepted rather than closed: nothing local can detect the reset earlier (the counter is in Redis and we learn of it by receiving something), and the two shapes that would — a GET per resume, or a background poller — put network I/O on a latency-sensitive path or spend a goroutine and a round trip per tick forever against a condition measured in years. The exposure is redelivery of notifications the client already has, bounded by the window and self-healing on the next publish. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): a replica that has received nothing must not answer 'caught up' (Codex round 9) P1 — the coverage check was skipped entirely while knownFrom was still 0, so a bus that had received NOTHING answered any cursor with an empty-but-non-nil replay, which the SSE handler reads as caught-up. The scenario is a restart, not an exotic one: replica B comes up while Redis is at 100, id 101 is published before B's subscription is live, and a client reconnects to B with Last-Event-ID 100 before 102 arrives. B says caught-up, then delivers 102 live, and 101 is gone with nothing to tell anyone. The principle the code now follows: having received nothing is strictly LESS knowledge than 'contiguous from X', so it must produce at least as strong a signal. A non-zero cursor against an empty bus is a gap. Both sides pinned, because the over-broad version is a real risk here — answering every fresh connection with a resync would be its own bug. A sinceID of 0 is not a resume and still gets an empty replay rather than a gap. Mutation-verified on the new arm. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(watchevents,cli): name the trailing-gap and shutdown trades (Codex round 10) Two findings that are decisions rather than defects, so both are documented at the line where the reasoning is met and taken to the plan instead of being settled unilaterally after ten review rounds. P1 as reported — the TRAILING gap. Everything the coverage bookkeeping does reasons about what this instance HAS received; it cannot see a notification missed at the END of the sequence. Hold 100, miss 101 to a disconnect, and a client resuming from 100 before 102 arrives is told caught-up. The hole only becomes visible when 102 lands, which is too late for that connection. What would reveal it is a GET of the sequence key: a value above lastAppendedID means ids exist we never saw, and a value BELOW it reveals the counter reset documented last round — one mechanism, both open windows. It is not done here because it is product-visible in the other direction: INCR happens before the message propagates, so the counter legitimately runs ahead of every instance for microseconds after each publish, and a strict comparison turns ordinary in-flight traffic into spurious sync_required responses with no principled tolerance to pick. A resync is recoverable and a lost nudge is not, which is the argument for doing it — but that is a call about how chatty the resync path should be. P2 — closing the watch bus before Shutdown drains handlers means a push already in flight can publish into a closed bus and still return 200 with pushed:true. Closing after would instead hold every shutdown to its 30s deadline on any open stream. eventBus already makes the same trade the same way; naming it rather than inheriting it silently. The honest fix is Bus.Publish reporting the drop so the handler can, which is an interface change and a different unit. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * feat(watchevents): close the trailing gap with a settle-window authority check (lead ruling) Lead's ruling on BUG-2651: a silently lost nudge is unbounded staleness, a spurious resync costs one redundant fetch, so the gap must not survive — and don't pick a magnitude tolerance, because the reason the counter legitimately runs ahead is in-flight propagation, which is TIME-bounded while a genuinely missed message never arrives. So the discriminator is time. On a resume (and only on a resume), read the shared counter: if it disagrees with this instance's high-water mark, wait out one settle window and read again. In-flight ids land during the beat and the resume proceeds normally; missed ones never do and the resume is answered with a gap. That converts an unprincipled 'how many ids behind is too many' threshold into a principled propagation bound. The same read also catches the counter having gone BACKWARDS, so the counter-reset window documented last round is closed by the same mechanism rather than needing its own — the arrival-time reset handling stays, because it is what repairs the instance's own state and what covers a bus with no reconnecting clients. Ordering matters and is documented at the call: the check runs WITHOUT the mutex (it sleeps and does network I/O, neither of which may happen inside the lock fan-out needs) and BEFORE subscribing rather than between subscribe and replay, which would reopen the double-delivery window SubscribeAndReplaySince exists to close. Nothing is lost by waiting first — fanOutLocally buffers regardless of subscribers. An unreadable counter falls back to local knowledge rather than failing closed: turning a Redis hiccup into a resync for every reconnecting client at once is a worse failure than the one being guarded against. EventsSince deliberately does NOT do this and says so — it is the local primitive the Bus interface already describes as being for tests and non-resuming callers, and making it sleep and hit the network would surprise every one of them. Five tests, each pinning a different half: the missed tail reports a gap; a current instance does NOT (the control that stops this being 'always resync'); an id arriving mid-settle is tolerated; an unreadable counter falls back; a fresh subscriber neither waits nor gets a gap. Mutation-verified twice — disabling the check, and removing the settle beat — each killed by the test that names it. Also filed at the lead's direction, so the two remaining cross-instance defects have tracked homes rather than only comments: BUG-2698 (targeted push resolved against local presence, plus the delivered_sessions inaccuracy — one shared-state SessionPresence closes both) and BUG-2699 (push returns 200 pushed:true for a dropped publish; Bus.Publish reports nothing, and fixing it is an interface change). Every disposition comment now cites its item. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): compare two FRESH reads, not one stale snapshot (Codex round 11) P1 — the settle beat re-read only the local side, so the comparison was against a counter SNAPSHOT taken before the wait. Id 2 arrives during the beat while id 3 is published and missed: the stale remote is still 2, the check declares convergence, and 3 is silently lost — the exact failure this whole mechanism exists to prevent, reintroduced inside it. P2 — the same staleness in the other direction. A GET can land just before a publish completes and report a value BELOW what this instance already holds; that never matches, so a client who had missed nothing got a full resync. Both are one defect: agreement between the authority and this instance has to be evaluated on two FRESH reads or it is not agreement. Now re-reads both sides after the beat, and treats any remaining disagreement as a gap in either direction — still behind means ids never reached us, still ahead means the counter was reset under us and our buffer belongs to a dead id space. Two tests, one per direction, each mutation-verified against the re-read-locally-only version: the second counter advance must produce a gap, and the raced read must NOT produce a resync. Without the second test the fix could have been 'always report a gap', which passes the first. Documented the cost side of the lead's ruling while I was in here: the condition is agreement, so a resume during CONTINUOUS publishing across the whole settle window can disagree every time and resync. Bounded by this stream being low-volume by design and resumes only happening on reconnect; if a workload makes it chatty, the answer is a longer window, not a magnitude threshold. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(watchevents): an absent sequence key is zero, not unreadable (Codex round 12) P2 — the counter key can DISAPPEAR after this bus has seen ids (FLUSHDB, eviction). Reading redis.Nil as 'unreadable' meant falling back to local knowledge and cheerfully replaying an id space the authority no longer has — while the next publish starts again at 1 and collides with it. Absent is a VALUE. Returning zero-and-readable makes the case fall out of the ordinary comparison with no special branch: an instance holding 101 disagrees with an authority at 0, does not converge, and the resume is answered with a gap. A genuinely fresh deployment still agrees at zero and is not resynced — which is the control leg, and the reason 'absent means gap' would have been the wrong fix: it passes the first test while resyncing every first connection on a new install. P1 as reported — the equality fast path returning without settling — is not closed, and the comment now says why rather than leaving it to be re-found. A notification published AFTER that read and missed by this instance is invisible to any check made here, and settling anyway would not close it: the same race exists in the instant after the function returns. The check's honest scope is what was missed BEFORE the resume. A message missed after it is a property of at-most-once pub/sub with no per-connection ack, and the real answer is a durable stream (Redis Streams with consumer groups), not a longer wait. Mutation-verified: restoring redis.Nil to the unreadable branch fails the disappearing-counter test. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * feat(watchevents): epoch marker, so a reset that caught up is still a reset (Codex round 13) P2 — numeric detection is blind to a reset that has already climbed past this instance's high-water mark. Hold 100, lose the connection, the counter resets and ids 1-101 are published, and the only one that reaches us is 101 — the perfect contiguous successor of 100. Every arithmetic check passes, the buffer quietly mixes two id spaces, and a client resuming from OLD 100 is handed NEW 101 having silently missed the new space's 1-100. No amount of comparing numbers fixes that, because the question is not 'is this bigger' but 'is this the same sequence'. The publish script now mints an epoch once per id space (SET NX, so every publisher can offer one and the first wins) and carries it on every message; a change drops the buffer and re-anchors. The subtle half, and the one the first attempt got wrong: after an epoch change the cold-start rule must NOT admit its usual contiguous-with-our-view cursor. Within an epoch, a client at n.ID-1 is genuinely adjacent to our first id. Across one it is ambiguous — id spaces overlap, so that cursor may be the OLD sequence's n.ID-1, a different notification entirely — and admitting it hands them the new epoch's id as though it followed theirs, which is exactly the failure the epoch exists to prevent. Letting it back in one line later would have been a poor joke. The test caught it; the control leg (a cursor genuinely inside the new epoch is still served) is what stops the fix becoming 'resync everyone forever after any reset'. Wire format changed to <epoch>|<id>|<json>. Free of compat cost, checked rather than assumed: redis_bus.go does not exist on origin/main, so no released build produces or consumes the old shape. The numeric backward check stays — it covers a counter reset where the epoch key survived (eviction picks keys individually), and it is what repairs an instance with no reconnecting clients at all. Mutation-verified: ignoring the epoch change fails the new test. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(watchevents): the wire format comments say <epoch>|<id>|<json> (Codex round 14) Three comments still described the pre-epoch format. Worth more than a tidy-up: a maintainer following them would conclude the epoch prefix is vestigial and remove it, which reintroduces exactly the cross-epoch replay corruption round 13 existed to fix. The publishScript comment now also says outright that the epoch is not decoration and points at redisWatchEpochKey before anyone considers it removable. Verified by grepping for the old shape rather than by trusting the edits — zero remaining, which is the check I owed after getting this wrong in round 7. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * chore(nix): update vendorHash for the miniredis test dependency (BUG-2651) CI's Nix job failed on a fixed-output hash mismatch, and it is neither a flake nor a surprise once seen: nix/package.nix pins the vendored module set, and adding miniredis (plus gopher-lua, its Lua interpreter) to go.mod changed it. Regenerated per the procedure the file itself documents — build and read the 'got:' line. Run on CI rather than locally because this box has no nix; the hash is a content hash of the module set determined by go.mod/go.sum, so the same inputs produce it in either place. Worth naming as a gate lesson rather than just fixing: my pre-merge matrix had build, lint, test, test-pg, vuln and Codex, and none of them can see this. A dependency change has a SEVENTH consumer — the Nix packaging — and the only thing that checks it is the CI job that just did. Adding a dependency means checking the packaging, not only the security scan. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V |
||
|
xarmian
|
449ac109e9 |
fix(server,mcp,cli): refuse system-metadata keys in fields_patch + retry-hostile error code (BUG-2627 part 2, BUG-2675) (#1166)
* fix(server,mcp,cli): refuse system-metadata keys in fields_patch + retry-hostile error code (BUG-2627 part 2, BUG-2675) Part 2 of BUG-2627 closes the door that mints the defect parts 1 and 3 dealt with: `--field implementation_notes=<json>` stored the entries as a JSON-ENCODED STRING, which is invisible to every reader and — since part 3's guard — disables `pad item note` on that item until the row is repaired. Refused SERVER-SIDE in `fields_patch`, not at the CLI as the item's scope line proposed. The deviation is deliberate and recorded on the trail: the CLI is one of three clients, and all three lower a user field-setter into the same key (`pad item update --field` at cmd_item.go, the MCP `field` param via dispatch_http_advanced.go on remote, and stdio by shelling out to that CLI). One gate closes all three; a CLI-only refusal would have left remote MCP writing the key. Both call sites were read, and the CLI's lowering is now pinned by a test rather than left as an assumption. Scope, stated because it is deliberate: this closes UPDATE only. The full `fields` blob stays open because that door is SHARED — `pad item note` / `decide` / `github link` send one, and so does convention activation via BuildConventionItemFields -> ItemCreate. Closing it would break the system writers the gate exists to protect. Item create therefore remains a mint site, tracked with the rest of that surface in BUG-2685. The refusal message is per-key: implementation_notes -> `pad item note`, decision_log -> `pad item decide`, github_pr -> the GitHub link flow, and `convention` refuses WITHOUT naming a command, because none writes it. PATTE-135 wants a remedy that works in the failing state; a single "use pad item note" line would have been wrong for three of the four keys. BUG-2675 rides along on one ToolSurfaceVersion bump, as ruled. The append refusal from part 3 reached MCP agents as `server_error` — not our fault, and not transient, so agents could reasonably retry a failure that is deterministic forever. New closed-set code `stored_state_unreadable`, emitted on BOTH transports: HTTP classifies the sentinel error directly, stdio via a `pad-structured-error/v1:` marker the CLI now writes for its own local refusal (the first marker generated without an upstream APIError). v0.16-then-v0.17 is what a one-transport fix costs. Also here: - items.ReservedOverrideKeys -> ReservedFieldKeysIn. The second caller passes a patch, not an override map, and the old doc comment said fields_patch was an open exposure — true until this commit. - `Extract* returns nil for THREE reasons` -> FOUR. The comment listed four; the count was corrected everywhere except the code. - Consumer-read artifacts updated where the claim is ACTED on, not only where it is documented: instructions.md (incl. a "do not retry this code" section), the catalog `field` param description, `pad item update --help`, README. Gates: build · make lint · go test ./... · make test-pg · Codex. Eleven-mutation matrix run against the new tests; every one killed by an assertion (two were rewritten after killing by compile error / surviving, which proves nothing). Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(server,mcp): honest remedy when the stored value is already unreadable; name the MCP-facing code (Codex round 1) Three findings from the pre-push review, all real: P2 — the refusal named `pad item note` unconditionally, but on an item whose stored value is ALREADY undecodable that command refuses too (part 3's guard). The caller was routed in a circle: field write refused -> run the note -> refused -> back again. That is exactly the failure PATTE-135 exists to prevent, and my own trail had reasoned the remedy was safe on the strength of the HEALTHY case only. The message now inspects the item's stored value and, when the key is unparseable, says so and points at the one action that works in that state (inspection), noting that the repair needs a full `fields` write no CLI flag exposes. P2 — two doc claims were false where an actor reads them. The catalog said reserved keys are refused "on every action that accepts field", which includes CREATE, and create is deliberately NOT gated; and both the catalog and instructions.md named `validation_error` (the HTTP code) where an MCP client actually receives `validation_failed`. Both corrected, and the create exception is now stated rather than implied by omission — an agent that reads only "refused on update" will otherwise assume create is fine, which is how a hole gets used. nit — the destructive-downstream sentence claimed every reserved key becomes unreadable and trips an append guard. True only for the two append-backed keys; github_pr and convention are simply overwritten. The clause is now per-key, because a confident wrong explanation is worse than a vague right one. Two more mutations run against the new branch: always-readable (the circular remedy returns) and never-readable (the working remedy disappears) — both killed by assertions. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(models,mcp,cli): one appendability predicate, per-key docs, stdio hint parity (Codex round 2) Five findings, all real. P2 — the message's readability check and the guard it describes were two different decodes. Mine unmarshalled into []json.RawMessage; the guard uses []ItemImplementationNote. A stored `[1]` passed mine and fails the guard, so the message would again have prescribed a command that refuses — the same circularity round 1 caught, through a narrower door. Replaced with models.StructuredFieldIsAppendable, which ASKS the guard rather than re-deriving it, plus an agreement test over 12 shapes x 2 keys that compares the predicate against the real Append* helpers. Verified by restoring the RawMessage version: the table catches it on `[1]`. P2 — stdio lost the new code's hint. Remote MCP told the agent retrying is pointless and how to inspect; stdio got the code with an empty hint, because the CLI's marker envelope carried none and the classifier parsed none. Both fixed, with the hint hoisted into paired constants (the same duplication StructuredErrorMarker already uses) and the test comparing the two TRANSPORTS' envelopes rather than either against a literal. P2 — doc text was still false for `convention`: the catalog, the instructions and `--help` all said reserved keys are maintained by note/decide/the GitHub flow, which is true of three of the four. Each key now names its own writer, and `convention` names library activation. Also dropped the `malformed_override` advertisement — that is the SERVER's code; an MCP client sees validation_failed for both refusals. nit — the classification test called structuredAppendErrorResult directly, so deleting either dispatcher call site left it green. Added dispatcher-level tests driving the real server + store, asserting the code, the hint, and that the item's stored fields are byte-identical afterwards. Mutation-verified by reverting the note call site. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(items,models,mcp): github_pr stays writable through fields_patch; no nil-map panic (Codex round 3) P1 — the gate refused `github_pr`, and that was wrong. My model was "system writers use the full fields blob, user setters use fields_patch", which holds for three of the four reserved keys and fails for this one: `pad github link` needs a local git checkout and the `gh` CLI, so it is excluded from remote MCP BY NAME, and internal/mcp/dispatch_http.go's noRemoteEquivalent map tells remote agents in so many words to use `item update --field github_pr=...` instead. For that audience the patch door is not a bypass of the writer — it IS the writer. So the refusal deleted a documented capability from remote agents, and answered with a message naming a command they cannot run: the same circular remedy round 1 caught, aimed this time at the people the gate was meant to help. items.PatchRefusedFieldKeysIn now exempts the key and records the rule being applied — refuse a raw write where a real writer exists — rather than the list it produces. Whether remote agents should get a proper PR-link action, so the key can be closed too, is a product question and is left as one. P2 — the hint told agents to read the bad value with `pad_item action=get`. They cannot: stripDuplicatedFieldsKeys removes implementation_notes and decision_log from every MCP response's fields blob, and the top-level arrays come from the extractor, which returns nil for exactly this shape. The value is invisible on the whole surface. The hint now says so and routes to a human, who can read it with `pad item show --format json`. P2 — `fields` holding a literal `null` unmarshals into a NIL map with no error, and both Append* helpers assign into what they get back, so `pad item note` PANICKED ("assignment to entry in nil map") instead of appending. Reproduced, fixed in parseMutableItemFields, and pinned by a test that fails on a panic rather than taking the process down. An absent blob and a null blob mean the same thing to every caller. Pre-existing, but it sits in the function family this bug is about and the message was about to recommend the command that panics. nit — README claimed a "closed eight-code taxonomy" (17 codes, and I had just added one) and read as if create lowers into fields_patch. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(models,mcp): predicate matches the append on malformed blobs; stop promising a broken workaround (Codex round 4) P1 — round 3 exempted `github_pr` from the update gate on the strength of noRemoteEquivalent's documented workaround. That workaround does not work: ingestFieldKVP (remote) and parseFieldFlag (CLI, and so stdio) both store a `field` value as a STRING, so the PR data lands double-encoded and no link appears — the BUG-2627 shape one key over. Filed as BUG-2696 with the three candidate fixes; NOT folded in, because the narrowest of them changes how every field value is typed. The exemption stands regardless: refusing would leave remote agents with strictly less than a broken door. What changes is what we may PROMISE. The catalog, instructions.md, version.go and README said "this is how you link a PR"; they now say the door is open and broken, and to hand PR linking to a human. Advertising a capability that isn't there is the failure mode this whole unit keeps circling. P2 — StructuredFieldIsAppendable returned TRUE when the whole fields blob was unparseable, on the reasoning that a broken outer blob is a different problem. True of the cause, irrelevant to the caller: the Append* helpers bail on that same parse, so the message again named a command that fails. It now returns false, which is simply the honest answer to the question asked, and the agreement table grew a malformed-outer-blob leg — the gap that let the disagreement through. P2 — the message claimed a raw field write always stores something Pad cannot read back. That holds for the CLI and MCP (a `--field` value is typed by schema lookup and these keys are in no schema) but not for a direct REST caller sending a valid array, who is refused for ownership reasons alone. Reworded to say both parts. nit — a misplaced parenthetical in the README read as if item CREATE lowers into fields_patch. It does not; it sends the full blob. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(mcp,models): stop the remote hint advertising the broken PR workaround; classify an unparseable blob as retry-hostile (Codex round 5) P1 — I corrected four artifacts that pointed agents at the github_pr field write and missed the fifth: noRemoteEquivalent's own text, which IS the message a remote agent receives when it calls `github link`, and which Codex had quoted at me in round 3 to establish the workaround existed. The nearest artifact to the actor was the one I did not open. Both entries now say there is no working remote path and name BUG-2696, with a test pinning the negative so a future edit cannot quietly reinstate the advice while the write is still broken. P2 — a fields blob that will not parse at all produced a bare parse error, so `note` / `decide` reached agents as `server_error`: transient- looking, and therefore retried, for a failure that is as deterministic as the per-key one BUG-2675 exists for. Both Append* helpers now wrap that parse failure in ErrStructuredFieldUnreadable, which both transports already classify, and the malformed-blob test asserts the sentinel rather than just an error. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(mcp,cli): qualify what an agent can actually see when the state is unreadable (Codex round 6 nit) Round 5 widened stored_state_unreadable to cover a fields blob that fails to parse outright, which made half of its own hint false: MCP's normalization strips a broken structured KEY (so `get` hides it), but leaves an unparseable BLOB as a raw string (so `get` shows it). The hint and instructions.md asserted the first case for both. Now stated per layer, in the two paired constants and the instructions. The reason it is worth the words rather than being cut: an agent told 'you cannot see this' does not look, and would have missed a value that was in fact right there in the response it already had. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(mcp): classify the move/copy reserved-key refusal as validation on stdio too (Codex round 7) P2 — carried over from v0.22, surfaced because THIS bump documents the two reserved-key refusals as agreeing across transports. The move/copy message ("Field(s) reserved for system metadata and not settable here") matched none of the stdio validation patterns, so the same deterministic 400 arrived as validation_failed on remote and server_error on stdio — and server_error reads as transient, so an agent retries a refusal that can never pass. One pattern added, plus a test that drives both real classifiers with the real server message text for both refusals, so a reworded message that stops matching fails here rather than in the field. nit — the github_pr exemption is UPDATE-only; move and copy still refuse it, because there the argument is BUG-2674's (an override reintroduces the key the migration just dropped), not this one's. The catalog and instructions said "not refused" without that qualifier. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(mcp): cover the copy path's own refusal wording in the stdio classifier (Codex round 8) P2 — round 7 fixed the MOVE wording; the copy path words the same class of refusal differently ("Destination collection has no field(s): ..."), so it kept arriving as server_error on stdio and validation_failed on remote. Third message in one family, and the round-7 test used the move text for every case, which is why it missed this. The parity table now carries all three real messages plus a control leg using one the pattern list already covered — without it the table could pass by matching everything. Recorded in the pattern list's comment rather than left implicit: matching prose is a stopgap, the structural fix is the pad-structured-error/v1 marker that carries the code instead of inferring it, and until a refusal emits one, this test is where a new wording has to be added. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * test(mcp): use the real upstream codes in the parity fixtures (Codex round 9 nit) The copy legs carried `validation_error` where the handlers actually emit `malformed_override` and `invalid_override`. The 400 branch ignores the body code today, so the test passed either way — which is exactly why the fixture mattered: it was quietly recording a wrong contract, and a future code-aware classifier would regress against a table that agrees with it. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(mcp): the upstream code is not forwarded to MCP clients (Codex round 10 nit) The catalog said the server's own code (validation_error / malformed_override) appears in the MCP message. It does not: the 400 branch emits code=validation_failed with a fixed "Validation failed." message and the server's text in the HINT, discarding the finer-grained code. Reworded to say what an agent actually receives, and to say that telling the two refusals apart means reading the message. Also carried the update-only qualifier on the github_pr exemption into the README, matching the catalog and instructions. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * docs(items): state the exemption predicate, not the exemption list (lead ruling) The lead's ruling on the github_pr reversal: make the REASON what the code says, so the next key added to reserved metadata is evaluated against 'does this audience have a real writer?' rather than pattern-matched onto a list that happened to be wrong for one key. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V |
||
|
xarmian
|
de96cce900 |
fix(items,server,web): reserved metadata survives a move; referential metadata travels only within its context (BUG-2674) (#1165)
* fix(items,server): reserved metadata survives a move, and dropped fields are reported (BUG-2674)
Moving an item destroyed its implementation notes, decision log and linked-PR
metadata. Well-formed data, on a routine documented operation, silently, with a
success message.
Reproduced before the fix: a note written through `pad item note` — correct
shape, visible on every surface — was gone after `pad item move`, leaving
fields as `{"status":"new"}`.
## Why it happened
items.MigrateFields drops every key absent from the TARGET schema. The reserved
keys — implementation_notes, decision_log, github_pr, convention — are system
metadata that NO collection schema declares; each renders from its own dedicated
surface rather than as a generic field. So they are absent from every targetDefs
and were dropped on every move.
That blindness is structural, not incidental: any code path reasoning about
fields BY CONSULTING A SCHEMA cannot see these keys. It is the shared root of
this bug and of BUG-2627, where the CLI types a --field value by schema lookup
and these keys fall through to a raw string.
## The enumeration comes first, deliberately
Before this there were four constants and exactly ONE non-test consumer treating
them as a set — an inline || chain in a CLI display path. Naming the set inline
again here would have created the SECOND hand-maintained list, which is the
generator pattern behind both bugs reproduced inside its own fix: the next
reserved field lands in the constants, gets wired into whichever surface
prompted it, and silently misses the other.
So models.IsReservedItemField is now the single place that knows, MigrateFields
consults it, and the CLI's || chain is converted to it — the only way it is
provably THE list rather than A list. (formatChangeValue keeps its per-key
switch: it needs to know WHICH reserved key it has, to say "notes" vs "entries",
not whether the key is reserved.)
`convention` is IN the set, settled with evidence rather than by the principle
alone: 35 of 36 conventions in a live workspace do not store the key at all, and
the one that does holds a blob that is a redundant mirror of the alias keys
beside it. No user types a `convention` object — ApplyItemConventionMetadata
writes it, via library activation and the web form. System-stamped.
## Contract
System-minted non-referential data carries; anything dropped is reported.
PLAN-2357 DR-17 settled the analogous case — tags carry because "there is no
workspace-scoped foreign key to break, so dropping them would lose information
for no safety reason". These are the same shape: inert JSON with nothing that
could dangle in a destination. The plan's carry list simply never considered
them, so there was no deliberate semantics to defer to. DR-17's own heading is
"None of this may be silent."
## The reporting half
MigrateResult.Dropped has always existed and the single-move handler has always
thrown it away, so the only record of a field disappearing was the field being
gone. It now rides the move's audit metadata — not the response body, which is
the bare item and would break every consumer, and the activity timeline is where
someone asking "what happened to my item" looks. Joined into one string because
that map is map[string]string and a raw array renders as a Go map literal in the
timeline (BUG-2628).
## Verified
Unit: reserved keys carry with their payload INTACT (asserted on the value, not
merely the key — a carry that re-encoded or zeroed it would pass a presence
check), and bypass schema matching entirely, so a target declaring
`implementation_notes` as `text` cannot coerce them. Mutants run: guard removed
-> both new tests fail; carried-but-also-reported-dropped -> the not-dropped
assertion fails; carry-everything -> the control leg fails alongside three
pre-existing tests.
Live, against a server built from this branch: the note survives the move
byte-identical, and the move's activity metadata carries
`dropped_fields: "priority, status"` for the values the target schema genuinely
has no home for.
## Known scope limit
The BULK move path still discards its Dropped list — a reporting gap only, since
the carry-through lives in MigrateFields and bulk inherits it. Threading the list
out crosses two function boundaries whose signatures serve every bulk operation,
so it is a refactor of the bulk dispatch's return contract rather than a line.
Filed as BUG-2683 rather than smuggled in here.
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
* fix(server,web): close the four gaps Codex round 1 found on the carry-through (BUG-2674)
Round 1 raised no P1 and four P2s. Three were real defects introduced or exposed
by the carry-through; one was a genuine overclaim in the previous commit. All
four closed here, each mutation-verified rather than asserted.
## A schema may no longer declare a reserved key
MigrateFields carries these keys by identity, but every caller then validates
against the target schema — and ValidateFieldsDetailed iterates schema.Fields,
so it DOES see a declared key. A target declaring implementation_notes as `text`
would receive the carried array and reject it, turning a move that previously
destroyed the notes into one that fails outright. That is a worse failure than
the one being fixed: loud, but it blocks an operation that used to work.
The gate already existed — validateNoReservedFieldKeys, with its
grandfathering — and listed only parent/plan. The four metadata keys join it,
sourced from models.ReservedItemFieldKeys() so the two lists cannot drift.
Forbidding the declaration is the honest fix; coercing the value, or skipping
validation for a key the schema genuinely declares, would be guessing at which
meaning the author wanted.
The web's RESERVED_FIELD_KEYS gains the same four, preserving the existing
deliberate asymmetry (the client lowercases and is therefore stricter than the
server's exact match) so the UI steers authors away before the 400.
## The copy preflight no longer under-reports
`carried` is built by walking the DESTINATION SCHEMA, and these keys are declared
by no schema anywhere — so after the carry-through they appeared in NEITHER
bucket. A copy of an item whose content is its notes would report "nothing
carries over" while in fact retaining them. Before the carry-through they at
least showed under `dropped`, accurately. Reporting in neither is a regression
in the preflight's honesty, which is the same defect class as the move that
reported nothing.
They are now appended to `carried` after the schema-ordered entries, marked
`type: "system"` with a rendered label since they have no author-supplied one.
The bucket's doc comment says so: a client must no longer assume every `carried`
entry resolves to a destination FieldDef.
## The audit report now reaches a human
The previous commit claimed the activity timeline is where someone asks "what
happened to my item" — true, and the timeline renderer ignored the key, so the
report existed only for API and CLI consumers. Stored-but-invisible is not
reported. TimelineActivityCard renders the dropped keys on a move.
## Test aliasing
The "untouched" assertions compared the result against the SAME objects passed
in, so an in-place mutation would change both sides and DeepEqual would stay
true. The expectations are now independent deep copies — the only thing that
makes "untouched" mean untouched.
## Mutants, each run
Preflight pass removed -> the carried assertion fails. Timeline block disabled
-> the render assertion fails. Timeline action guard dropped -> the non-move
negative leg fails (a presence-only test would have passed it). Reserved-set
helper returning everything -> the IsReservedItemField control leg fails.
## Not fixed here
Codex's remaining observation — that a cross-workspace copy now carries
github_pr into a workspace whose repository it does not describe, and leaves a
convention blob detectable on an item outside the conventions collection — is a
product question about what a copy MEANS, not a defect in this mechanism. Raised
for a ruling rather than decided inside a bug fix.
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
* fix(items,server): referential system metadata travels only within its context (BUG-2674)
Lead ruling on the copy-semantics fork Codex round 1 raised. It does not add an
exception to the carry rule — it applies the qualifier the rule already had.
The contract was "system-minted NON-REFERENTIAL data carries". github_pr is
referential: it names a repository that is a property of the SOURCE workspace's
project, and it hydrates into code_context and renders as a live PR link. Carried
into another workspace that link is a false statement about the destination's
project, not preserved information. implementation_notes and decision_log
describe the item's own history and are true wherever the item is.
So the rule stays one sentence: non-referential system data carries everywhere;
referential system data carries only where its referent's context still holds.
## Scope is a required argument
MigrateFields takes items.MigrateScope. Required rather than defaulted because
BOTH wrong answers lose something: SameWorkspace on a cross-workspace copy
carries a PR link into a workspace it does not describe, and CrossWorkspace on
an ordinary move DROPS metadata from an item whose repo context never changed. A
caller that must name its scope cannot pick one by omission.
The two move handlers pass SameWorkspace as a property of the endpoint, not a
guess — a move changes an item's COLLECTION and cannot change its workspace.
The copy and its preflight COMPUTE it by comparing workspace ids rather than
assuming cross-workspace, because that endpoint accepts a target_workspace equal
to the source; hardcoding would drop a github_pr from a same-workspace duplicate.
Both sides use the same helper, or the preview promises a carry the copy drops —
the DR-6 divergence the shared endpoint exists to prevent.
## The drop is reported, with a reason that explains itself
PLAN-2357 DR-17: "None of this may be silent." It would be perverse to
reintroduce a silent drop inside this fix's own new branch.
The preflight reports it as `referent_not_portable` rather than the generic
`no_target_field`. That generic reason would be actively misleading here: no
schema declares these keys ANYWHERE, so "the destination has no such field" is
equally true of the source and explains nothing about why the value is being
left behind.
## Verified
Mutants run: scope ignored (always carry) -> the cross-workspace leg fails;
generic reason on the preflight drop -> the reason assertion fails. The
same-workspace leg and the non-referential-sibling leg are what stop an
implementation that ignores scope in EITHER direction from passing — each half
alone is satisfiable by a constant.
Gates re-run for THIS commit: lint 0 · go test ./... 0 · make test-pg 0 (3282).
Web gates NOT re-run and not claimed: this commit touches no web file (the web
half of BUG-2674 shipped in
|
||
|
xarmian
|
bc68b84848 |
fix(cli,mcp): send raw collection slug so an alias can't shadow a real collection (BUG-2630) (#1162)
* fix(cli,mcp): send raw collection slug so an alias can't shadow a real collection (BUG-2630)
The client-side alias map (collections.NormalizeSlug) rewrote seven hardcoded
singulars ("task", "plan", …) to their plurals BEFORE the request. In a
workspace whose collection slug IS one of those singulars, the user's exact
name was rewritten away and their create/list/move landed in a DIFFERENT
collection — silently, with a success message naming the wrong one.
Fix, per lead ruling on the BUG-2630 trail, split by transport:
CLI (real HTTP, may hit a pre-resolver server) — Option 2, one shared helper
cli.WithCollectionAliasFallback: send the RAW slug first (the server's
exact-match-first resolver from BUG-2578 wins, so an exact name is never
shadowed), and retry with the alias ONLY on a collection-not-found error, only
when the alias differs. Keying on collection-not-found is load-bearing: a
request to a collection that exists but fails for another reason is never
retried into the alias (that would recreate the bug). Both the schema fetch and
the create funnel through the helper so typed --field values parse against — and
the item lands in — one collection. On a genuine double-miss the error names the
RAW slug the user typed (collection "widget" not found), not the alias.
MCP remote transport (in-process ServeHTTP against the SAME binary, which always
carries the resolver — no version skew) — drop client-side normalization
entirely and send raw. Also removed the dormant expandPath collection
normalization: no routeSpec uses a {collection}/{target_collection} path
placeholder, so the branch was dead code in the area this fixes.
Search is deliberately out of scope (filed BUG-2659): its collection is a global
c.slug=? FILTER, not a path — a miss returns 200 + zero results, not
collection-not-found, so the retry can't key on it; and handleSearch is
cross-workspace, so the per-workspace resolver has no single workspace to run
against. Cross-workspace copy is excluded too (DR-13 forbids auto-retrying the
copy mutation).
Verified live against a real server: create/list/move into a singular collection
that collides with its plural now land in the named singular; shorthand still
resolves; genuine misses error naming the raw slug. New MCP integration test
reproduces the original shadow (item → PLANS-1) when normalization is restored.
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
* fix(server,cli): own collection resolution server-side + capability-gate the CLI retry (BUG-2630 Codex r1)
Addresses all three Codex round-1 findings, via the lead's ruling that
dissolves the earlier "retry vs archived-protection" tension by making the
server the sole owner of resolution semantics.
Finding #2 (MCP lost the legacy abbreviations t/i/p/d and phase/phases -> plans,
which the server's ±s resolver did not cover): fold the legacy alias map into
collectionSlugCandidates as a LAST-resort candidate. Exact-match-first and the
archived-claims refusal run for the input and every structural candidate before
the alias is reached, so it never shadows or redirects around a real/archived
collection. Now every client can send the raw slug — including the MCP transport
that can't retry — and lose nothing.
Finding #1 (the client retry re-opened the archived/hidden redirect the server
deliberately refused, because not_found can't be told from absent): add a
collection_resolution capability flag to GET /server/capabilities and gate the
CLI retry on it. Happy path unchanged (raw slug, one request). On
collection-not-found ONLY, the client probes capabilities once (cached): if the
server advertises resolution, its not-found is authoritative — the slug is
absent, archived, or hidden — so the client does NOT retry. Only an older server
that lacks the flag (or 404s the endpoint) triggers the legacy alias retry,
which is non-regressive there since old servers never had the protection. The
probe fails safe toward retry. This makes the follow-up distinct-error-code bug
unnecessary.
Finding #3 (double-fail masked a substantive alias error as "collection not
found"): the helper now surfaces a substantive alias-attempt error verbatim, and
only collapses to the raw-named not-found when the alias ALSO 404s.
Verified live against a resolving server: create/list/move into a singular that
collides with its plural land in the named singular; the abbreviation `i`
resolves to `ideas`; and after archiving `plan`, `create plan` honestly fails
("collection \"plan\" not found") instead of being retried into a live `plans`.
Gates: make lint 0 issues; go test ./... green; make test-pg green.
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
* fix(cli): fail-closed capability probe + always-retry the schema lookup (BUG-2630 Codex r2)
P1: the capability probe cached ANY failure as "no resolver", so a single
transient blip (timeout/5xx) permanently re-enabled the alias retry and could
bypass the archived/hidden protection on a resolving server. Now the probe
distinguishes a DEFINITIVE verdict (HTTP 200 with the flag, or a clean 404 =
legacy build) from an INDETERMINATE one (transport error / 5xx): only definitive
verdicts are cached, and an indeterminate probe fails CLOSED (trusts the
not-found, no retry) without caching, so the next call re-probes. A genuine old
server still returns a clean 404, so its retry is unaffected. Renamed the
predicate to CollectionNotFoundIsAuthoritative to name what it actually decides.
P2: the create schema lookup hits exact-match-only GetCollection, which does NOT
resolve slugs server-side, so capability-gating it made `create task
--field amount=3` 404 the schema fetch, skip the retry, and send amount as the
string "3". The schema lookup now always retries the alias (nil gate),
restoring typed-field parsing against an aliased collection's schema. Best-effort
as before: a genuine miss still degrades to string fields.
New client test covers the probe: definitive verdicts cache (one probe), and a
transient failure fails closed AND re-probes on the next call (mutation-verified).
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
* docs(cli): note fail-closed-on-indeterminate as a deliberate safety asymmetry (BUG-2630)
Per lead review: make explicit in CollectionNotFoundIsAuthoritative's doc that
failing closed on an indeterminate capability probe is deliberate — a recoverable
alias-shorthand miss is the safer side of the trade vs a retry doing an
un-undoable wrong-write. Comment-only.
Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V
|
||
|
杨成锴
|
22c5a858a1 |
fix(server): stop counting disabled conventions as completed work (#1152)
Merged after two codex review rounds (converged) on top of the community-loop supply-chain/static review. Review found two narrow follow-ups — the guest item-grant leg of the grouped terminal query keeps pre-PR over-matching semantics, and the standup/changelog display layer hardcodes `status` — both pre-existing edges, filed internally as follow-up work. Thanks @asjdf for a well-tested fix, and for honoring the per-collection terminal_options contract on both the CLI and server paths. |
||
|
xarmian
|
052c971785 |
feat(plugin): consent-gated push monitor + tri-state arm/disarm + envelope (PLAN-2613 S3, TASK-2618) (#1150)
* feat(plugin): consent-gated push monitor + tri-state arm/disarm + envelope (PLAN-2613 S3, TASK-2618) The plugin layer of the push-consent gate. S2 built the CLI arm/disarm/status verbs and the arm-state file; S3 makes the monitor existence itself the gate (D1) and adds the tri-state, the envelope, and the connect ritual. - Tri-state arm-state file: a session can be explicitly ARMED, explicitly DISARMED, or absent. `pad session disarm` now writes a session-scoped OFF marker (not a file removal), so a within-session disconnect wins even in an auto_arm=true repo — the disconnect verb must not be a lie there. The marker dies with the session (same liveness), so across sessions auto_arm remains the standing contract. ResolveAnnouncedArmed folds the tri-state over auto_arm; the monitor announces its result. - Gated monitors (monitors.json): the single always-on monitor is replaced by two — an `always` auto-arm monitor and an `on-skill-invoke:connect` manual monitor — both running scripts/pad-monitor.sh. The wrapper gates on a new hidden `pad session should-arm`, dedupes concurrent monitors with a liveness-aware per-session lockfile, and carries the reconnect loop so an in-session disarm stops the stream on its next reconnect. No consent → the monitor exits → nothing listening. - D5 envelope: a push notification carries the verbatim direction-with-authority framing (confirm in-session before anything destructive/irreversible); item- change kinds stay a light informational label. - /pad:connect + /pad:disconnect skills; /pad:status gains a one-line connection header from `pad session status`. /pad:connect runs the workspace's on-session-start playbooks on the first connect only (D8), tracked by a Booted flag carried forward across arm/disarm. plugin 0.2.1 → 0.3.0. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(plugin): address Codex R1 on S3 (disarm stops active stream, fail-closed local state) - HIGH-1: a within-session disarm now stops an ACTIVE stream, not just the next reconnect. The monitor re-checks consent every 2s while streaming and cancels the connection when it flips to not-armed, then exits (D1's whole- stream-behind-consent gate at the top of the loop), so the plugin wrapper keeps it dead. Fixes /pad:disconnect being a lie for an idle SSE that might never naturally reconnect. - HIGH-2: a corrupt/unreadable local arm-state file now fails CLOSED (LocalArmError -> not armed) instead of falling through to auto_arm, so a corrupted disarm marker can't silently re-arm an auto_arm repo. It is not reaped (reaping would re-arm on the next read); it is session-keyed and a re-arm overwrites it. - Shell wrapper: an empty (mid-startup) lock pid is treated as live so two monitors can't both steal the lock; INT/TERM now exit (a trap otherwise resumes the loop and reconnects without a lock). - Docs: plugin/skills/pad describes the new push-envelope line format; connect/status skills distinguish "consent set (armed)" from the server's observed connection counts rather than claiming "Connected". Bounded/safe-direction residuals documented in code: the reap TOCTOU and the Booted carry-forward race (both fail-closed / benign), and lock pid-reuse (dedupe only, fails toward not-streaming). Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(plugin): address Codex R2 on S3 (disarm-watcher timing, semantic corruption fail-closed) - HIGH-1: the disarm-watcher now starts BEFORE the connection is opened, so a disarm during connection/header negotiation cancels the request too (the request is built on streamCtx). streamWatchEvents also re-checks consent before delivering each notification and stops the stream if it was withdrawn, so no push is printed after a disarm even within the poll window. - HIGH-2: a syntactically-valid but semantically-garbage arm-state file (e.g. {} or {"pid":1}) now fails CLOSED via a well-formedness check (StartedAt + PID must be present, as our writer always stamps them) before liveness or reaping — so it can't be judged owner-dead, reaped, and re-armed through auto_arm, nor mistaken for a live headless arm naming init. - LOW: the cleanup trap uses condition 0 (portable) rather than the EXIT name. The disconnect skill note reflects the ~2s active-stream drop. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(plugin): /pad:disconnect always disarms, never gated on a linked workspace (Codex R3) Consent is session-scoped (keyed by the messaging socket, not the workspace), so a session that connected in one repo must be able to disconnect from anywhere — including a directory with no .pad.toml. The old precondition let a session move to an unlinked directory, "disconnect", and keep receiving pushes. Verified: `pad session disarm` from an unlinked cwd disarms the socket-keyed session state; should-arm then reports not-armed back in the original repo. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(cli): enforce the Armed != Disarmed writer invariant in arm-state validation (Codex R4) armStateWellFormed checked only StartedAt + PID, so a well-stamped file that violated the writer invariant — both armed and disarmed false (or both true) — passed validation and, since SessionArmState only branches on Disarmed, resolved to LocalArmOn and armed. The writer always sets exactly one of the two; require it, so a neither/both file fails closed (LocalArmError). Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V |
||
|
xarmian
|
e40df6b31c |
feat(cli): pad session arm/disarm/status + consent config resolution (PLAN-2613 S2, TASK-2617) (#1149)
* feat(cli): pad session arm/disarm/status + consent config resolution (PLAN-2613 S2, TASK-2617) The S2 CLI contract S3's plugin skills and S4's web composer build against. S1 gated push delivery on a server-side armed bit declared at stream connect; nothing decided WHETHER to arm or sent the declaration. S2 adds both, defaulting off everywhere. - ResolveAutoArm (internal/cli/arm_consent.go): pure consent resolver. .pad.toml [push] auto_arm is the only per-repo enabler (D4); a per-user config auto_arm=false vetoes it (deny-wins); default off. Config surfaces: PadToml.Push.AutoArm + config.Config.Push.AutoArm (*bool, unset != false), both nil-safe. - Wire contract: StreamSessionIdentity.Armed sends ?armed=true on the event stream — S1's server gate finally has a sender. The monitor announces armed = live local arm OR resolved auto_arm, so a repo opt-in works end to end with a safe default-off skew. - Verbs pad session arm/disarm/status: arm/disarm manage a per-session local arm-state file; status reports the resolved local/auto decision plus the server's own armed/connected counts (new Client.ListSessions), degrading gracefully when padd is unreachable. - Arm-state file (session_arm_state.go): keyed per session by CLAUDE_CODE_MESSAGING_SOCKET (cwd fallback for headless, secondary to auto_arm). Mandatory liveness — a dead-owner file (socket vanished / pid gone) reads as disarmed and is reaped, so a crashed session can never arm a future monitor. Local client state only; the server's armed bit stays the sole delivery authority. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(cli): address Codex R1 on push-consent (fail-closed config, owner-identity liveness) - HIGH-1: user config.toml read now fails CLOSED. config.LoadPushConfigAutoArm reads the [push] auto_arm value strictly — absent → no opinion, but present-but-unparseable → error — and ResolveAutoArmFromDisk refuses to auto-arm when it can't confirm the user's veto (was: swallowed by the lenient config.Load and treated as no-opinion). - HIGH-2: arm-state liveness now checks owner IDENTITY, not just presence. Socket-keyed files record the socket's mtime and require an exact match, so a reused socket path can't revive a stale file. Headless files record a Linux /proc start-time token (portable fallback documented) to reject a reused pid. - MED-1: arm-state writes are atomic (temp + rename) and reaping is non-destructive (re-checks staleness before removing) — a concurrent re-arm is never clobbered. - MED-2: pad session status applies the .pad.toml URL override, so it queries the same server the monitor connects to. - LOW: malformed arm-state files are now reaped (safe now that writes are atomic — a corrupt file can't be a torn in-progress write). Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V * fix(cli): address Codex R2 on push-consent (atomic config write, stronger owner identity) - HIGH-1: Config.Save() is now atomic (temp + rename), so a monitor reconnecting while `pad configure` rewrites config.toml can't read a truncated/partial file, miss a [push] auto_arm=false veto, and arm. - finding 2: socket owner identity now uses inode+device (unix) as the primary signal, with mtime as the non-unix fallback — a rebound socket or a lingering stale node at the same path gets a new inode and is rejected, closing the mtime-collision / reused-node gaps. - finding 3: headless liveness fails closed when a proc-start token was recorded but can't be re-verified (was: fell back to bare pid-liveness, which a reused pid passes); zombies (state 'Z') now report not-alive. - finding 5: `pad session status` applies an explicit --url override too, not just the .pad.toml one. - finding 4 (connect-time TOCTOU): documented as an accepted, bounded residual — a disarm racing an in-flight connect is corrected on the next reconnect; fully closing it needs S3's server-side disarm-on-open signal. Claude-Session: https://claude.ai/code/session_017jD6t1zjxGSq47SQpZfp1V |
||
|
xarmian
|
625cab9984 |
fix: bound item history and stop resolving bodies nobody reads (BUG-2608) (#1147)
* fix: bound item history and stop resolving bodies nobody reads (BUG-2608)
Item history was unbounded on every surface, and summary mode paid for what it
discarded: the endpoint resolved EVERY version by walking the item's whole
reverse-patch chain, and both the CLI and the MCP dispatcher then projected
that away to metadata. An item edited under collab records a version every few
seconds while someone types, so this is routinely hundreds of full-content
reconstructions per history call, for output that shows none of them.
Two independent fixes, because they address different costs.
SUMMARY SKIPS THE WALK. `?summary=true` returns metadata from the raw rows and
never resolves a patch. That is the dominant win: the resolution was pure waste
for every caller except --full. Content and is_diff are cleared TOGETHER — an
empty body still claiming to be a reverse patch would tell a consumer to
resolve something that is not there.
LIMIT BOUNDS THE WINDOW, newest-first. That direction is not a preference: with
reverse patches, reconstructing any version means walking back from current
content, so a newest-end window is the cheap prefix of that walk while an older
one still pays for everything above it. That is also why there is deliberately
no offset — it would advertise a pagination whose later pages cost the same as
no bound at all.
Absent limit stays UNBOUNDED on the endpoint, following the item-list
precedent (maxItemListQueryLimit: "a zero/absent limit is left unbounded — this
only clamps an explicit oversized request"). The defaults live on the CLIENTS,
where a token budget is actually known: `pad item history` defaults to 50 with
--limit to change it, and the MCP catalog action injects 50 (max 300, the same
pair list and backlinks already use). A server that truncates a request nobody
bounded is a silent-truncation trap for third-party API consumers.
The MCP default goes in the CATALOG action rather than either dispatcher, so it
reaches BOTH transports — HTTP reads it off the input, and stdio receives it as
the CLI's new --limit through BuildCLIArgs. ToolSurfaceVersion 0.20 -> 0.21
with a changelog entry, plus instructions.md and README, per the 2304-family
contract discipline. Additive param bump: `limit` already existed, nothing
changed shape, and a v0.20 consumer that sends no limit now gets the newest 50
instead of all — which is the fix, not a break in it.
The restore and single-version-expand paths still resolve the FULL chain, and a
test pins that: bounding their walk would strand exactly the old versions those
paths exist to reach.
Eight mutations, each failing only the leg it targets. Three fixture problems
surfaced that way and are worth naming, because each made a test that could not
fail:
- force_version in a PATCH body does nothing (`json:"-"` on ItemUpdate), so
the throttle collapsed six edits into one version; varying the source per
edit is what actually records them.
- an 8-byte body is cheaper stored whole than as a patch, so no version was
ever is_diff=true and the is_diff assertion was inert. The fixture now uses
a body large enough that the store really stores patches.
- the cmdhelp test fixture lacked the new --limit flag, so BuildCLIArgs
silently dropped it. Verified against the REAL cmdhelp tree that the flag
is present and typed int, so the fixture mirrors the CLI rather than
flattering it.
* docs: bring CLAUDE.md to v0.21 and name why the two result caps differ (BUG-2608)
Codex round 1, both findings.
CLAUDE.md still described the MCP surface as v0.20 — stale because of my own
bump, in the document every agent working this repo reads first. README and
instructions.md are held to the version by a test; CLAUDE.md is not, which is
exactly why it drifts.
The cap "mismatch" (MCP max 300, endpoint clamp 500) is deliberate layering,
not an oversight — item lists have the identical split (300 in the catalog,
1000 at the endpoint) because the two answer different questions: an agent
token budget is only knowable in the catalog, while the endpoint's clamp is a
server-resource ceiling on what any caller may ask for. But nothing said so
anywhere, so a reader comparing the two numbers had no way to tell design from
accident — which is precisely the report Codex filed. Now stated at the
constant and in CLAUDE.md, including why the versions ceiling is LOWER than the
list one (resolving a version can cost a patch application per row, not just a
row read) and why an absent limit is left unbounded at the endpoint.
* fix+test: honest truncation notice, armed fixtures, and the residual named (BUG-2608)
Codex round 2, both findings, and the second is the more useful one.
CLI TRUNCATION NOTICE was wrong in both directions: it compared the response
length against the requested limit, so an item with exactly N versions was
reported as truncated, and a --limit above the server's ceiling was clamped
there and reported as complete. It now asks for ONE MORE row than it shows and
reports truncation only when that extra row comes back. The one case this still
cannot detect — an ask above the server's own ceiling, where the probe row is
clamped away with everything else — is stated in the code rather than papered
over by hardcoding the server's constant in the CLI.
UNDER-ARMED FIXTURES. The unbounded test seeded 5 versions, so a server quietly
defaulting to 50 would have passed the assertion that denies exactly that; it
now seeds 60. The clamp test seeded 2 and could not observe a clamp at all;
the clamp is now asserted directly against a parseItemVersionsLimit function
extracted for the purpose, over the inputs a URL can really carry (absent, 0,
negative, unparseable, either side of the ceiling).
That extraction replaced my own first attempt, which was worse than no test: it
re-implemented the clamp arithmetic in the test body and asserted the result
against itself. It could not have failed.
THE RESIDUAL, NAMED RATHER THAN IMPLIED. Codex's sharpest point is that the
summary tests cannot detect "resolve everything, then clear the fields" —
verified by mutation: pointing the summary branch at the resolving reader
leaves every handler test green, because the response is byte-identical either
way. So the performance claim does not rest on them. It rests on the handler's
summary branch calling ListItemVersionsPage (one reviewable line) plus a new
store test proving that reader really returns unresolved rows rather than
quietly resolving them — mutation-verified from the other side by making the
resolver a passthrough. The test file says all of this, including that an
end-to-end assertion would need a patch-application counter in the production
path, and why that is not worth it when the cost of being wrong is performance
rather than correctness.
* fix(cli): don't resolve for table output, guard the probe overflow, finish the CLAUDE.md bump (BUG-2608)
Codex round 3, four findings.
--full was treated as "content needed" regardless of output format, but the
table path prints no bodies at any setting — so `pad item history --full`
without --format json made the server walk the entire patch chain to build
content the CLI then dropped. That is the exact waste this bug is about,
reintroduced through the flag meant to opt into it. Content is now resolved
only when it will actually be shown.
The limit+1 probe overflowed at MaxInt: it wrapped negative, the client omitted
the parameter, and a request the user bounded came back unbounded — the
opposite of the ask. Guarded.
The truncation notice's documented blind spot was understated: it is AT the
server ceiling as well as above it, since the probe row is clamped away with
everything else. Wording corrected rather than resolved — the CLI still does
not duplicate the server's constant, because a copied ceiling goes stale
silently and asking for hundreds of versions is already opting out of a bound.
Two more CLAUDE.md sites still called v0.19 current; I fixed only the first on
the previous pass. That document describes the contract in three places and I
updated one, which is its own small lesson about grepping for every instance of
a claim rather than the first.
Live-verified against a real server: --limit 2 truncates and says so, --limit 4
on a 4-version item says nothing (the false positive Codex found), --limit 0
returns everything, --full --format json carries bodies, and the default JSON
shape carries metadata only.
* fix(server): clamp an out-of-range limit instead of treating it as unbounded (BUG-2608)
Codex round 4. `?limit=9223372036854775808` made strconv.Atoi return ErrRange,
which my parser lumped in with 'unparseable' and mapped to 0 — unbounded. So an
absurdly large number defeated the very ceiling the function exists to impose,
which is worse than no clamp at all: the bound looked enforced and was not.
Atoi hands back the saturated value alongside ErrRange, so a range-positive
input is now clamped to the ceiling — it is an oversized ASK, not a malformed
one. Range-negative still falls through to unbounded, matching a plain
negative.
Regression cases cover both overflow directions, and the mutation that
reinstates the old behaviour fails on the exact input.
This is the input-domain lesson again: I enumerated absent, zero, negative and
non-numeric, and stopped at what a person would plausibly type rather than at
what the TYPE admits.
* docs(cli): say in --help that a large --limit is capped server-side (BUG-2608)
Codex round 6, and the third time this ceiling came up — which is the tell that
the disposition was in the wrong place. I had documented it twice in CODE
comments, where the person affected never looks. `--help` says what the flag
does, and it said 'show only the newest N versions (0 = all)', which is false
for an N above the server's cap.
Now stated in both the flag help and the command's long description, including
that the truncation notice cannot detect that case and that --limit 0 is the
way to ask for a complete history.
Deliberately not naming the number: a constant duplicated into help text goes
stale silently, and 'capped server-side' is the part a user needs in order to
act. Behaviour unchanged — this is the artifact catching up with it, which is
the same correction the BUG-2301 sentinel comment and the instructions.md
overclaim both came down to.
* test: seed versions through the store so the fixture stops tripping the rate limiter (BUG-2608)
CI caught this and it is my defect, not a flake. Arming the unbounded test
above the plausible default meant seeding 60 versions, and the fixture did that
with 60 HTTP PATCHes in a burst — which trips the server's rate limiter. It
passed locally and in the Go job and failed under Nix, which is the signature
of a test that was always going to fail intermittently rather than one that
broke.
Seeding now goes through the store. That is not a weakening: versions are
recorded by the store on either path, and the endpoint under test is the READ
side, so seeding underneath the transport costs the assertions nothing while
removing a burst the server is entitled to refuse.
The three things that make this fixture work are now stated where someone would
otherwise undo them by accident — the large body (a small one is stored whole,
so no version is ever is_diff and every diff assertion goes vacuous), the
rotating source (the throttle collapses same-(actor, source) bursts into one
version), and the store-not-HTTP seeding with the rate-limit reason attached.
Re-verified after the change: the fixture still records more than 50 versions
and still produces reverse-patch rows, and the default-cap mutation now uses
the REALISTIC default of 50 rather than the 3 I first tested with — the old
5-version fixture could only have caught an implausibly small cap.
|
||
|
xarmian
|
9c155ac185 |
fix(cli): gate promptAndBootstrap on canPromptForConfig() (BUG-2597) (#1119)
Third member of the BUG-2577 family (offerSkillInstall #1111, installInteractive #1116): promptAndBootstrap — the legacy --cli-prompt admin bootstrap — guarded its prompts on stdin-only term.IsTerminal, so a pty-backed harness with a redirected stdout got " Email: " printed into the pipe and then blocked on the read. Swap to canPromptForConfig() (stdin AND stdout) with the family's boundary comment; the BUG-988 refuse-with-headless-hint behavior is unchanged. The error message no longer blames stdin specifically ("not running in an interactive terminal") since the widened gate can fire when stdin IS a terminal and stdout isn't; the existing non-TTY test's assertion updated to match. Claude-Session: https://claude.ai/code/session_018qREYgDd6Ag1X1SDmqhyFM |
||
|
xarmian
|
4a2c4c1a39 |
fix(cli): suppress pad agent install's dangling (Y/n) prompt in non-interactive contexts (BUG-2593) (#1116)
installInteractive gated its prompt on cli.IsTerminal() (stdin only), so a pty-backed harness whose stdin looks like a char device — with nobody able to answer — got "Install /pad skill for all N? (Y/n): " printed and then hung at readChoice. Same shape and same fix as offerSkillInstall's BUG-2577 (PR #1111): swap to canPromptForConfig() (stdin AND stdout), document the both-pty undetectable boundary, keep the auto-install behavior unchanged. Test mirrors #1111's offerSkillInstall test and pins the closed-stdin no-prompt path; the discriminating pty-stdin case is live-verified on the trail (pre-fix binary prints the prompt and hangs to a 10s kill, fixed binary installs silently and exits 0 — identical undriven-pty harness). Claude-Session: https://claude.ai/code/session_018qREYgDd6Ag1X1SDmqhyFM |
||
|
xarmian
|
2580c2c8bb |
fix(cli): gate pad init's Step-4 login on canPromptForConfig() (BUG-2592) (#1115)
* fix(cli): gate pad init's Step-4 login on canPromptForConfig() (BUG-2592) A configured-but-unauthenticated non-interactive `pad init` fell into doBrowserLogin and blocked on the poll wait (wall-clock-bounded since BUG-2572, still minutes of hang nobody can complete) instead of failing fast — Step 3 has had this exact gate since init.go:205, and cmd_workspace.go got it in PR #1111 (BUG-2538). The gate sits AFTER the saved-credentials check so a headless run with valid stored credentials proceeds untouched. Remedy text per the corrected trail ruling (the r1 constraint was refuted by r2): piped `pad auth login --interactive` IS a working non-interactive login (doInteractiveLogin reads a plain bufio.Reader, piped-bytes-safe since BUG-1886), so the message points there — and deliberately not at pad init's --email/--name/--password, which only fire when SetupRequired. Claude-Session: https://claude.ai/code/session_018qREYgDd6Ag1X1SDmqhyFM * docs(plugin): pad init no longer hangs in the session-expired case — update the three claims + plugin 0.2.1 The BUG-2592 gate makes three plugin-skill passages stale (same shape as PR #1111's codex r3 self-invalidation): capture and onboard said `pad init` can still hang on the browser flow when configured-but- unauthenticated, and the pad skill's whoami-guidance said the same at its "not a safer probe" sentence. All three now state the fixed truth, live-verified this session: fixed binary fails fast in 0.1s with the piped-login remedy; pre-fix control binary hangs to the timeout kill in the identical sandbox state; the remedy itself (piped `pad auth login --interactive`) logs in and restores credentials. Plugin 0.2.1 — text reaches nobody without a bump (version-pinned at install). Claude-Session: https://claude.ai/code/session_018qREYgDd6Ag1X1SDmqhyFM |
||
|
xarmian
|
ef903f0b22 |
feat(cli,mcp): --clear-parent / clear_parent to detach an item's parent (BUG-2078) (#1113)
* feat(cli,mcp): add --clear-parent / clear_parent to detach an item's parent (BUG-2078)
The server has honoured a present-but-empty "parent" key in fields_patch
as "clear the link" since BUG-2013, but neither the CLI (--parent ""
silently no-ops) nor MCP (parent is a plain string with the usual
"empty means not provided" convention) could reach it. Mirrors the
clear_assigned_user/clear_agent_role shape from IDEA-2584: a boolean
that carries its destructive meaning in its name and survives the trip
to local stdio MCP via BuildCLIArgs' snake_case-to-flag mapping.
Bumps ToolSurfaceVersion 0.18 -> 0.19 and updates the drift-pinned docs
(instructions.md, README.md) accordingly.
* test(cli,mcp): cover --clear-parent / clear_parent on both transports (BUG-2078)
CLI: --clear-parent sends fields_patch{"parent":""}; is absent when not
passed; conflicts with --parent and refuses without issuing a PATCH;
item create pins the deliberate create/update asymmetry.
MCP: clear_parent detaches through the real store+server (not a
recording handler) so the assertion is "item ends up unparented", not
just "payload shaped correctly"; clear_parent=false is inert; a plain
empty `parent` string stays a no-op (control leg); a simultaneous
parent + clear_parent is refused via both the direct param and the
--field-lifted route.
* fix(cli,mcp): close --clear-parent bypass via --field parent/plan aliases (BUG-2078, codex r1 P1)
extractParentLink (internal/server/handlers_items.go) resolves the parent
link from either a "parent" or a "plan" key in fields_patch, with no
early exit, so the later key in its own loop wins. The clear_parent
conflict check only covered one path each on the two client surfaces:
- CLI: the check ran BEFORE the --field overlay and only compared
against --parent's own value, so `--clear-parent --field parent=X`
(or `--field plan=X`) reached the wire unrejected — the --field loop
ran after clearParent's own `patch["parent"] = ""` and silently
overwrote it.
- MCP HTTP dispatcher: the check ran after the --field overlay (correct
ordering) but only inspected `patch["parent"]`, missing the "plan"
alias route.
Both surfaces now run the clear_parent check after every patch-building
step (named flags, --field overlay, column lift) and check both
"parent" and "plan" for a competing non-empty value.
* fix(cli,mcp): refuse --clear-parent/clear_parent when schema shadows "parent"/"plan" (BUG-2078, codex r2 #2)
extractParentLink (internal/server/handlers_items.go ~L606-610) is a
pre-existing, deliberate policy: it skips hierarchy handling entirely
when a collection's schema declares its own field literally named
"parent" or "plan", letting the value fall through as an ordinary
field write instead. Once {"parent":""} reaches the server it can no
longer distinguish clear-hierarchy intent from a legitimate
blank-my-schema-field write, so a client-side clear_parent request
against a shadowed collection used to report success while silently
blanking the data field AND leaving the real hierarchy link untouched
-- reproduced empirically before this guard existed.
The ambiguity is created at the surface that accepted the clear
request, so that surface refuses rather than pushing the decision
server-side (server-side refusal would also break legitimate blanking
of a real schema field).
CLI: the check is free -- collSchema is already fetched for --field
type parsing whenever any field change (including a bare
--clear-parent) happens.
MCP HTTP dispatcher: adds one conditional collection lookup, paid only
when clear_parent=true -- the common update path fetches no schema
today and doesn't start.
* docs: sync repo CLAUDE.md tool-surface contract to v0.19 (BUG-2078, codex r3 P2)
CLAUDE.md's MCP tool-surface prose still said "currently v0.18" and its
changelog omitted clear_parent -- a consumed-artifact gap, same rule as
the SKILL.md case: the doc a diff invalidates ships with the diff.
Synced three spots (intro paragraph, Tools bullet, ToolSurfaceVersion
stability-contract changelog) to v0.19, matching internal/mcp/version.go's
in-code entry's wording, plus the schema-shadow refusal (BUG-2078's
second follow-up commit) at the same level of detail the changelog
already gives the parent/plan alias conflict-refusal.
Grepped the rest of CLAUDE.md for any other 0.18/tool-surface reference
-- none found outside these three lines.
* docs: add schema-shadow refusal to version.go's v0.19 changelog entry (BUG-2078, codex r3 follow-up)
The in-code changelog is the canonical source; it was missing the
codex r2 schema-shadow refusal that a later commit added, which is
why CLAUDE.md and version.go briefly disagreed. Completes version.go
instead of letting CLAUDE.md drift ahead of it.
|
||
|
xarmian
|
ac05d8a2b1 |
fix(cli): fail fast and quiet on non-interactive workspace init (BUG-2538, BUG-2577) (#1111)
* Fail fast and quiet on non-interactive `pad workspace init` BUG-2538: initCmd drove runBrowserSetup/doBrowserLogin unconditionally when the instance needed first-run setup or login, blocking a non-interactive caller (script, CI, headless agent) on a browser handoff nobody can complete. Gate both branches on canPromptForConfig(), mirroring the precedent already used by `pad init` (init.go:205-206), and fail fast with a hint pointing at `pad init --email/--name/--password` or `pad auth setup`/`pad auth login`. BUG-2577: offerSkillInstall (shared by workspace init and workspace link) printed a "(Y/n): " prompt even when the answer would be auto-defaulted rather than read, because it gated on cli.IsTerminal() (stdin only). Switch to canPromptForConfig() (stdin AND stdout), which is the same predicate now used for BUG-2538 and the more robust of the two checks already in the codebase. Claude-Session: https://claude.ai/code/session_01BhQoeaWXxJbvw86ezzK8dt * Fix wrong remedy in BUG-2538's !Authenticated error message codex r1: the !Authenticated branch suggested `pad init --email/--name/--password`, but those headless flags only bootstrap the first admin account and only fire when SetupRequired — for an already-set-up-but-unauthenticated instance, `pad init` falls through to its own ungated Step 4 re-auth (BUG-2592), so the suggestion relocated the hang instead of avoiding it. Drop the pad-init suggestion in this branch only; point at `pad auth login` and note there's no non-interactive login path yet. SetupRequired branch is unchanged — its pad-init suggestion is correct for that state. Claude-Session: https://claude.ai/code/session_01BhQoeaWXxJbvw86ezzK8dt * Fix two more inaccurate remedies flagged by codex r2 1. SetupRequired branch: `pad init --email/--name/--password` silently eats the caller's workspace name/--template — pad init creates its own CWD-named workspace as a side effect, so a re-run of the original `pad workspace init <name> --template <t>` short-circuits on the link pad init just made with no signal <name>/<t> were ignored. Switch the remedy to `pad auth setup --email/--name/--password`, which bootstraps the admin account only (no workspace side effects), then re-run the original command. 2. !Authenticated branch: the "no non-interactive login path exists" claim was false — `pad auth login --interactive` reads email/password off a plain, TTY-ungated bufio.Reader (doInteractiveLogin, cmd_auth.go:554+; BUG-1886 made it piped-bytes-safe), so it works fine when credentials are piped in. Reworded to point at it and dropped the incorrect BUG-2592 reference (that bug tracks pad init's ungated Step 4, not a missing login mechanism). TestWorkspaceInitNonTTYSetupRequired's assertion updated from "pad init" to "pad auth setup" to match; TestWorkspaceInitNonTTYNotAuthenticated needed no change (still asserts "pad auth login"). Claude-Session: https://claude.ai/code/session_01BhQoeaWXxJbvw86ezzK8dt * Update skill docs invalidated by the non-interactive fast-fail fix codex r3: BUG-2538/BUG-2577 made this diff's own docs stale. Four files (skills/pad/SKILL.md, plugin/skills/pad/SKILL.md, plugin/skills/onboard/SKILL.md, plugin/skills/capture/SKILL.md) still say non-interactive `pad workspace init` on a configured-but- unauthenticated machine "blocks for minutes with no non-interactive fallback" — that was true pre-fix (per BUG-2541's verification) and is false now. Reworded the WHY without dropping the underlying do-not-run-blind guidance: an agent's tool call is always non-interactive, so it now gets a fast, actionable error instead of a hang, but the error still just says a human needs an interactive terminal — `pad auth whoami` remains the right check to run instead. Where the docs' `pad init` claims are about the still-unfixed session-expired path (BUG-2592, this diff's Step-4 sibling, left untouched), those claims are unchanged and now cite BUG-2592 explicitly. skills/INSTALL.md:24 updated separately (P3): notes the non-interactive silent-install branch of `pad workspace init`'s skill offer, alongside the existing interactive-prompt description. Docs only, no Go changes — go build/test and embed.go's //go:embed skills/pad/SKILL.md still resolve; no test asserts the old wording. Claude-Session: https://claude.ai/code/session_01BhQoeaWXxJbvw86ezzK8dt |
||
|
xarmian
|
7d5d3bd672 |
fix(cli): give the CLI auth poll loop its own wall-clock timeout (BUG-2572) (#1109)
* fix(cli): bound pollAndSaveCLIAuth with its own wall-clock timeout (BUG-2572) pollAndSaveCLIAuth had no wall-clock limit of its own — the ~5m bound users rely on was purely the server-side session TTL, so an unreachable server after session creation left the poll loop spinning forever on Ctrl-C alone. Add a 20m timer (matching the longer of the two server TTLs, since this helper is shared by both the plain login and first-run setup flows) plus a consecutive-transient-error bound so a permanently unreachable server fails fast with a network-shaped error instead of waiting out the full timeout. Claude-Session: https://claude.ai/code/session_01BhQoeaWXxJbvw86ezzK8dt * fix(cli): make poll-error headline accurate for HTTP-error servers (BUG-2572 r2) The consecutive-error bail-out message claimed "could not reach server", but client.get returns an error for both transport failures and non-2xx HTTP responses, so a server that's reachable but persistently returning 500 got misreported as unreachable. Bailing out fast is still correct for that case; only the headline was wrong. Switch to a cause-neutral message and let the wrapped error carry the specifics (codex round 2). Claude-Session: https://claude.ai/code/session_01BhQoeaWXxJbvw86ezzK8dt |
||
|
xarmian
|
d7da237198 |
feat(mcp,cli): clear_assigned_user / clear_agent_role — a discoverable unassign (IDEA-2584) (#1107)
* feat(mcp,cli): clear_assigned_user / clear_agent_role — a discoverable unassign (IDEA-2584)
v0.16 and v0.17 made unassigning WORK. Nothing advertised it. The params
that do it — `assigned_user_id` / `agent_role_id` — were never in the
catalog, so an agent reading the tool schema to find out how saw only
`assign` (a name) and reached for `assign: ""`, which is a no-op and
deliberately stays one. The capability existed with no name an agent
could find.
`clear_assigned_user` / `clear_agent_role` booleans on `pad_item`, backed
by new `--clear-assigned-user` / `--clear-agent-role` bareword flags on
`pad item update`.
WHY BOOLEANS rather than declaring the existing string params. Two
reasons, and the second decided it:
1. An empty DECLARED string is inert everywhere else on this tool
(title, content, comment, tags), so a client that pads optional
params with "" instead of omitting them is harmless today. Giving
one a destructive meaning would turn that same client into one that
silently unassigns every item it touches. A boolean carries its
meaning in its name and can't be tripped that way.
2. Only a boolean can REACH local stdio. BuildCLIArgs emits the CLI's
real flags, so a catalog param with no flag behind it is dropped
before dispatch — declaring `assigned_user_id` would have left the
direct form remote-only, i.e. would not have closed the gap this
change exists to close. That fact reframed the design fork and is
what the ruling turned on.
Server-side this is WIRING, not new semantics:
models.ItemUpdate.ClearAssignedUser / ClearAgentRole already existed and
the store has honoured them since BUG-2566, on the same branch as the
empty-string form. The older forms keep working and are NOT deprecated;
they're just not what the schema advertises.
UPDATE ONLY, deliberately asymmetric with create, and recorded in-place
at both the flag registration and the catalog description so a
symmetry-minded reader meets the reasoning before the "fix": clearing at
create is a request to not-set something never set, whose only honest
behaviour is a no-op — it teaches a wrong affordance and pads every
create call's schema. A test fails if someone adds them there.
CLI precedence is the OPPOSITE of the --field lift's, deliberately: an
explicit `--clear-assigned-user` beats `--assign`, because that
combination is a contradiction the user typed and the reading that
cannot silently assign somebody is the safer one. Tested.
The dispatcher forwards the booleans VERBATIM rather than only-when-true.
A `&& b` guard would read as the thing protecting a param-padding client
and would be lying: what makes `false` inert is the store. Same call I
made on #1106's `len(patch) > 0` — a guard that reads as load-bearing
while doing nothing is worse than none.
ToolSurfaceVersion 0.17 -> 0.18, ADDITIVE bump per the v0.5 / v0.6
precedent: no existing tool, action or param changed shape.
Consumed artifacts moved in the same commit, which is the whole point of
this change — the schema IS the deliverable: catalog_item.go (the schema
agents read, plus an `assign` description that now says where to find the
clear), instructions.md (leads with the boolean, mentions the older forms
as still-working), version.go, README, CLAUDE.md.
VERIFIED LIVE, five legs, both transports:
CLI --clear-assigned-user -> assigned=None, role intact
CLI --clear-agent-role -> role=None
stdio clear_assigned_user:false -> assignment SURVIVES and the
update still applied (title
changed) — the control that
makes the boolean safe to
declare at all
stdio clear_assigned_user:true -> assigned=None
stdio clear_agent_role:true -> role=None
Three mutations, each failing only its own tests: dropping the dispatcher
forwarding; hardcoding true in the dispatcher (fails the false-control);
dropping the CLI flag wiring.
go test ./cmd/pad ./internal/mcp — pass. gofmt clean.
Closes IDEA-2584.
Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd
* fix(mcp,cli): refuse a simultaneous set-and-clear (codex round 1)
Codex found a real bug, and the more useful half of the finding is that
MY OWN TEST FOR IT WAS VACUOUS.
The store's branch order is `if AssignedUserID != "" { set } else if
ClearAssignedUser { clear }`. So `--assign wren --clear-assigned-user`
assigned Wren and the clear evaporated. My in-place comment claimed the
opposite ("an explicit clear wins"), and the test I wrote to prove it
asserted `body["clear_assigned_user"] == true` — that the FLAG was set,
not that the item ended up unassigned. The flag was set. The behaviour
was backwards. A test that asserts a field is present says nothing about
which field wins.
Both surfaces now REFUSE the contradiction rather than silently resolving
it. Rejecting beats picking a winner here: the store already picks one
silently, which is the bug; and a caller who typed both wants to be told,
not guessed at. Precedent in the same command family — `item list`
already makes `--parent` and `--unparented` mutually exclusive.
PLACEMENT IS THE LOAD-BEARING PART, and I got it wrong first. There are
two routes to a competing value: `--assign` / `assigned_user_id`, which
resolve early, and `field: ["assigned_user_id=<uuid>"]`, which reaches
the payload via liftFieldsToColumns LATER. My first version checked
between them and its comment asserted the lift "has already" run — it
hadn't. That version rejects the direct case and lets the lifted case
through: a half-fix that reads as complete. The check now runs after
both, in the CLI after --assign/--role resolution and the lift, in the
dispatcher immediately before the body marshal.
That mutation is now a test: moving the dispatcher check back to the
pre-lift view fails ONLY the two `lifted …` subtests and passes the
direct one — the exact shape of the bug I nearly shipped.
Tests assert the OUTCOME, not the message: a refused conflict must leave
the item's assignment AND role untouched, and the CLI must issue no PATCH
at all. An error string alone wouldn't prove the write didn't happen.
Agent-facing text moved with it (the consumed-artifact step): both
catalog descriptions, instructions.md, and the v0.18 version entry now
say the combination is refused. An agent that pairs them gets a
structured refusal, so the schema has to say so.
go test ./cmd/pad ./internal/mcp — pass. gofmt clean.
Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd
|
||
|
xarmian
|
847ee73327 |
fix(cli): lift assigned_user_id / agent_role_id onto their columns (BUG-2583) (#1106)
* fix(cli): lift assigned_user_id / agent_role_id onto their columns (BUG-2583)
`pad item update TASK-9 --field assigned_user_id=<uuid>` wrote the pair
into the item's FIELDS JSON BLOB while the column stayed stale, and then
printed "Updated TASK-9". Two defects in one line: a success message for
a write that did nothing the caller asked for, and a blob key shadowing a
real column's name, so the CLI surface diverged from store/HTTP/MCP
truth. The empty-string case was the same defect wearing a worse hat —
it was the only route an agent had to unassign an item.
Blast radius beyond the CLI: local stdio MCP (`pad mcp serve` — Claude
Desktop, Cursor, Windsurf) dispatches through ExecDispatcher, which
shells out to this CLI. So TASK-2571's fix reached the remote /mcp
transport only, and the transport most agents actually use still could
not unassign. This closes that half.
`cmd/pad/cmd_item.go` now lifts `columnFieldKeys` out of the --field map
onto the column pointers, on CREATE and UPDATE both, mirroring
internal/mcp/dispatch_http.go's liftFieldsToColumns — including its
INVARIANT, which is the part that matters: only keys with defined
clear-to-NULL semantics for "" belong in the list, and `tags` never does
(an empty write corrupts a JSONB column rather than clearing it). A test
fails if anyone adds it.
Two compat changes, ruled separately by the lead:
Q1 non-empty values move to the COLUMN and stop writing the blob key.
Accepted: relying on the old behaviour is relying on a shadowing
defect.
Q2 empty values clear the column. Falls out of the lift, inheriting
BUG-2566's store semantics.
`agent_role_id` gets identical treatment. Existing stray blob keys are
left alone per the ruling — this stops minting new ones; a sweep would
be its own change.
Precedence is explicit and tested: `--assign` / `--role` win over a
lifted --field value, matching liftFieldsToColumns' "caller-supplied
top-level values win". It is delivered by the ORDER of two blocks in the
command, which is exactly the kind of thing that gets reordered by
accident, so there is a test whose only job is to fail when it does.
A non-string --field value is deliberately NOT lifted: a collection that
genuinely declares a field with one of these names makes parseFieldFlag
return a typed value, which cannot address a column. It stays in the
blob — today's behaviour and the only lossless option.
ToolSurfaceVersion 0.16 -> 0.17, and v0.16's transport-scope paragraph
now points forward rather than claiming a limitation that no longer
holds. Behaviour-only bump again, same grounds as v0.16 and v0.9. The
CLI's own marker, CmdhelpVersion, deliberately does NOT move: its
contract is flag/arg SCHEMAS, and no flag or argument changed shape.
instructions.md — the text agents receive at handshake — drops the
"remote only" caveat it carried since TASK-2571. That file is the reason
this PR exists in the shape it does: it is the artifact the actor reads,
and it was the one place the previous PR overclaimed.
VERIFIED LIVE against a running server, with a negative control, because
the claim is about a transport rather than a function:
legs, fixed binary
--field assigned_user_id= -> column CLEARED, blob clean
--field assigned_user_id=<uuid> -> column SET, blob clean
--field agent_role_id= / <uuid> -> same, sibling column untouched
stdio MCP tools/call pad_item
action=update field=["assigned_user_id="]
-> column CLEARED, blob clean
control, PRE-FIX binary, same server + same item + same JSON-RPC bytes
-> column UNCHANGED, blob polluted
with {"assigned_user_id":""}
Six unit tests in cmd/pad/item_column_fields_test.go, four mutations each
failing only its own test (no lift; drop non-strings; flip the
lift/assign precedence; add `tags` to the list). One assertion was
rewritten after mutation testing showed it was VACUOUS: `len(fields_patch)
!= 0` passes whether the key is absent or present-and-empty, so it now
asserts key PRESENCE — confirmed by mutating `omitempty` off the model
field and watching the old form stay green. The redundant `len(patch) > 0`
guard that assertion was meant to cover is gone too; `omitempty` already
does that job, and a guard that reads as load-bearing while doing nothing
is worse than no guard.
go test ./cmd/pad ./internal/mcp — pass. gofmt clean.
Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd
* test(cli): cover the create half of the column lift (BUG-2583)
Codex came back CLEAN, but the review reminded me I'd changed `item
create` and only tested it through `liftColumnFields` directly — no test
asserted what create actually puts on the wire. That's the weaker half to
leave uncovered, not the stronger one: on update a wrong write contradicts
a visible prior value, while on create the column-named key is simply
baked into the blob at birth with nothing to contradict it.
The assertion has to parse rather than index, because ItemCreate.Fields is
a JSON-encoded STRING and not a nested object — a body["fields"]["…"]
lookup would have been vacuous in a way that looks fine.
Mutation-tested like the rest: neutralizing the create-side lift fails
this test and only this test.
Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd
* docs(mcp): say WHICH form of the unassign works on which transport (codex round 2)
Codex round 2, and it is the same class of defect as the previous PR's
round 2 — an overclaim in the artifact agents actually read. My
instructions.md said "works on BOTH transports" of two forms that do not
behave the same:
field: ["assigned_user_id="] clears on BOTH transports
assigned_user_id: "" clears on REMOTE ONLY
The direct params are not declared in pad_item's schema. They reach the
remote mapper only by riding the verbatim input map; on stdio,
BuildCLIArgs drops unknown keys, so the call does nothing.
VERIFIED, not accepted on the reviewer's word, and the verification
corrected my own first reading. My initial probe appeared to show the
stdio call CORRUPTING the fields blob — but that blob key was leftover
state from the earlier pre-fix control leg, not something the probe
wrote. Re-run against a freshly created item, the two forms separate
cleanly:
before assigned=b6786b13... fields={priority,status}
after assigned_user_id:"" assigned=b6786b13... fields={priority,status} (clean no-op)
after field:["assigned_user_id="] assigned=None fields={priority,status} (cleared)
So the stdio behaviour of the direct param is a DROP, not a corruption —
worth stating precisely, because "it corrupts the blob" would have sent
the next reader hunting a bug that isn't there. (Identity-doc rule: a
guessed mechanism stated as the reason is a claim, not a hedge.)
instructions.md now leads with the form that works everywhere and names
the remote-only limitation of the other; version.go and CLAUDE.md say the
same. IDEA-2584 — declare the params properly — is the fix that would
collapse this distinction, and is now cited from all three.
Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd
* fix(cli): don't lift a field the collection actually DECLARES (codex round 3)
Nothing reserves `assigned_user_id` or `agent_role_id` as field names, so a
collection may legally declare a field with one of those keys. For that
collection `--field assigned_user_id=foo` means the DECLARED field — and
the lift I just added would redirect it to the assignment column while
dropping the value the user set. Two wrongs from one line: the intended
write vanishes and an unintended one happens.
liftColumnFields is now schema-aware and never lifts a declared key. Cheap
to do here because both call sites already fetch the collection schema for
parseFieldFlag. The check is PER-KEY — an undeclared sibling still lifts,
so one collision doesn't disable the feature — and a schema-fetch failure
degrades toward lifting, matching how the rest of --field handling degrades.
This makes the CLI deliberately STRICTER than the MCP dispatcher it
otherwise mirrors. liftFieldsToColumns has the identical collision and
can't make the same check as written: it builds its fields map straight
from the tool input without fetching a schema. Filed as IDEA-2587 rather
than fixed here, because closing it costs a round-trip on a hot path while
the CLI fix was free — and recorded so the divergence is KNOWN, in the safe
direction, rather than something a later reader "fixes" by loosening the
CLI to match.
The old non-string branch stays as belt-and-braces: parseFieldFlag only
returns a non-string for a declared field, which the new check already
catches, but if that stops being true a non-string still can't address a
column.
Mutation-tested: ignoring the schema declaration fails the new test and
only that test.
Claude-Session: https://claude.ai/code/session_01QCMLhHQBrMHVML3YKdm4Cd
|
||
|
David Barkhausen
|
b9381bf5f1 |
feat(cli): markdown output on the remaining list surfaces; broaden ANSI stripping (#1080)
Completes #898 and fixes #1076. Markdown on the seven surfaces left out of #1070, so `--format markdown` is now honestly global and the flag help collapses to "table, json, markdown": - `item comments`, `item deps`, `project activity`, `attachment list`, `library list`, `role list`, `workspace members`. Two of those are not tabular, and markdown follows the terminal shape rather than forcing a table onto them: - `item comments` keeps the attribution-line-then-body form, and the body is emitted VERBATIM. A comment body is authored as markdown; escaping it would turn its lists and code fences into literal text. Only the attribution line, which we construct, is sanitized. - `item deps` keeps its two sections as `## Blocks` / `## Blocked by` lists. Colour carried the direction in the terminal (yellow out, red in); headings carry it here. New shared spine: `cli.RenderMarkdownTable(w, headers, rows)`. Every cell is escaped, and ragged rows are padded or truncated to the header width so a short or long row can't shift the column count and break the table. Wiring a surface is now naming columns and mapping rows. #1076 — ANSI stripping covered only SGR (`ESC[…m`), so non-SGR CSI sequences, OSC-8 hyperlinks, and stray C0 controls survived, both in the table width maths and in markdown output whose doc comment promised escape-free text. Replaced `sgrPattern` with `ansiPattern` + `stripANSI` covering OSC, CSI, two-character Fe escapes, and stray C0/DEL, with TAB/LF/CR deliberately preserved for callers that normalize them. `displayWidth` now uses it too: a control sequence is zero-width, so counting it was a column-alignment bug of the same family. Tests: 12 stripping cases, 4 table-helper cases (including ragged rows), 4 renderer cases for the two non-tabular surfaces, and the routing test extended to 8 subtests — one per surface, driven through cobra against an httptest server. Also covers the two gaps named in #1076: `item starred` and the scoped `item list <collection>` path. Each new guard was proven by mutating the source and watching it fail, not just by passing. Gates: go build ./... PASS; go vet PASS; gofmt clean; golangci-lint 0 issues. Both touched packages show the same 6+2 pre-existing Windows failures as clean main under an identical sandboxed run. |
||
|
xarmian
|
c84cf7437c |
feat(sessions): announce session identity on the event stream (PLAN-2558 S2, TASK-2560) (#1094)
* feat(sessions): announce session identity on the event stream (TASK-2560) PLAN-2558 S2. S1 gave the presence registry a count of anonymous uuids; this makes each row nameable, which is what S3 needs for an honest empty state and S5 needs for a target picker. A monitor now announces itself when it opens the stream: X-Pad-Session-Label (the working directory's basename) and X-Pad-Session-Pid. The server sanitizes both and stores them on the LiveSession; GET /api/v1/sessions returns them. TRANSPORT. The task body sketched "the stream connect carries it" without picking a mechanism and explicitly left the call open. Headers, because a query param would put the label and pid into every access-log line (this server logs path= for each request) and any proxy log in front of it — which is the same "don't let local detail travel further than it needs to" the privacy line below is about — and a separate registration POST would need its own correlation to the connection it describes, plus a matching lifecycle, when the registry entry already lives and dies with the stream. Headers ride the request that exists and sit alongside Last-Event-ID, already doing this job on this endpoint. Cost, written into the code rather than discovered later: a browser EventSource cannot set headers, so a future web-tab consumer needs a deliberate query-param fallback or a fetch-based SSE reader. PRIVACY. The basename crosses, never the full cwd — "/home/dave/Dev/ docapp" additionally hands over a home directory and usually an account name for no gain — and messaging_socket_path never leaves the machine. Pinned by a test rather than by the implementation being one line. WHAT THIS DELIBERATELY DOESN'T DO: read ~/.pad/sessions/. The task framed S2 as giving `pad session register` its first consumer, and the monitor cannot honestly be one. Registry entries are written by whatever process ran that command — a different pid — and the only matchable fields are pid and cwd, so two agent sessions in one checkout are indistinguishable and "pick the newest" is a coin flip that would put a confident wrong name in the S5 picker. Process ancestry settles it exactly and is platform-specific (this binary ships for macOS and Windows). The monitor's own cwd basename and pid are never wrong and answer the question the label exists to answer; correlating a stream to the agent session that spawned it needs an identifier the harness passes down, which is worth doing when something needs it and worth not faking until then. Also moves S1's STALENESS doc block, which sat above LiveSession.Label where it read as documenting the name rather than the whole entry. Tests: sanitizer units (whitespace collapse, control-char stripping, rune-not-byte truncation), header wiring, the end-to-end labelled session, the unannounced-client compatibility leg (a pre-S2 monitor must still register and still stream), a hostile-input leg over the wire, the client's omit-when-unset behaviour, and the basename promise. Measured rather than assumed: Go's server answers 400 to a header value containing a control byte before any handler runs (verified with a raw socket, since Go's own client refuses to send one and the two refusals are indistinguishable from a normal client test). So that arm of the sanitizer is unreachable over HTTP; it stays as defence in depth for the next caller in, and both the comment and the wire test say so instead of the test quietly passing because the transport refused the input. Mutation-tested four ways, each revert grep-verified: handler ignoring the parsed identity, monitor sending the full cwd, dropping the truncation, and the client always setting the headers. Refs TASK-2560, PLAN-2558 * fix(cli): sanitize the session label client-side per Codex review (round 1) Codex round 1's only finding, and it is a bigger deal than a missing label. Unix directory names may contain control bytes — "doc\napp" is a legal directory — and Go's http.Client REFUSES to send a request whose header value holds one: Do returns "invalid header field value" and nothing is transmitted. In the monitor that is indistinguishable from an unreachable padd, so the retry loop backs off and tries again, forever, printing nothing by contract. A user who named a directory that way would simply stop receiving notifications, with no signal anywhere. The server cannot defend against a request that never arrives. Reproduced before fixing, with a real directory and a real client, rather than reasoned about from the error message. Sanitizing in NewWatchEventsStreamRequest rather than in monitorSessionIdentity: the invariant is "this function never builds an unsendable request", which belongs at the point where a value becomes a header, not at one caller. The client's cap (256 runes) is deliberately looser than and independent of the server's (64): the server decides what a label should look like, the client only has to keep the request sane, and neither has to track the other to stay correct. The regression test does the ROUND TRIP instead of inspecting the header, because the header contents were never the bug — http.Header.Set stores anything, so an assertion on the value passes against the broken version too. Only attempting the request tells the two apart. Mutation-verified: reverting the sanitizer fails the test with exactly the "invalid header field value" error from the field report. |
||
|
xarmian
|
21001bc4c3 |
feat(sessions): session-presence registry + GET /api/v1/sessions (PLAN-2558 S1) (#1091)
* feat(sessions): session-presence registry + GET /api/v1/sessions (PLAN-2558 S1)
Slice 1 of PLAN-2558 (IDEA-2544 Phase 3, web-UI push). The server can now
answer "is anything actually listening right now?" for the calling user.
WHY. `pad push` (Phase 1,
|
||
|
xarmian
|
da6ce642da |
feat(push): pad push — user-authored instruction dispatch to agent sessions (IDEA-2544 Phase 1) (#1090)
* feat(push): add pad push <ref> -m vertical (IDEA-2544 Phase 1)
Self-addressed, human-to-harness dispatch over the existing watch-events
bus/stream: CLI -> POST .../items/{itemSlug}/push -> a new KindPush
Notification (carrying the generalized TargetUserID addressed-to field
KindAsk will later share) -> watchNotificationVisible delivers it back
to the pushing user's own connected monitor sessions. Transient,
fire-and-forget by design (no migration, no durable inbox) since
assignment already covers the durable-notification case and this is
meant to be the explicit, no-inference dispatch verb instead.
* docs(plugin): document the push notification contract (IDEA-2544 Phase 1)
Push is the one notification kind that IS an instruction rather than a
passive fact, so it gets its own lead bullet in the plugin skill's
notification-etiquette section (ahead of the read-only/park default,
which it explicitly lifts) and a mention in the monitor's description.
The embed-source skills/pad/SKILL.md has no notification section to
mirror this into (the two files diverge by design) and is left
untouched.
* fix(push): reject over-long push messages instead of unbounded Summary
Comments truncate their notification Summary to a preview (the full
body is still fetchable), but a push message IS the payload — silently
truncating it would corrupt the instruction with nothing to recover it
from. Add maxPushMessageLen (4096, measured post-collapse) and reject
anything over it with a 400 rather than truncating; state the same
bound in `pad push --help` so it's discoverable before a 400, not only
from one.
* fix(push): close the watch-fallthrough leak, disambiguate SKILL.md exceptions
Codex round 1 P1: watchNotificationVisible's push branch only returned
early on a MATCH — a non-target caller fell through to the watch-map
check below it, so anyone holding an unconditional (or predicated)
watch on the item received every push addressed to every OTHER user,
instruction text included. Push is addressed private dispatch, not an
item-level fact watchers have a legitimate claim on (unlike assignment,
which watchers are expected to see per `pad watch --help`) — the branch
now returns unconditionally for KindPush, gating strictly on
TargetUserID and never reaching the watch-map fallback either way.
Pinned explicitly since Phase 4's session targeting is expected to
inherit this same exclusivity.
Also (codex P2): reworded the SKILL.md notification-etiquette bullets —
the new push exception and the pre-existing assignment/ask exception
literally contradicted each other ("the ONE narrow exception" claimed
singularity after push had already claimed exception status). Now
explicitly enumerated as the first and second exceptions to the
never-write rule.
* test(cli): pin that PushItem inherits X-Pad-Agent (BUG-2542 rebase)
Verified, not assumed: PushItem builds its request via c.post ->
c.newRequest like every other mutating client method (CreateWatch
included), so the attribution fix's client.agentName wiring covers it
for free with zero code changes needed on this branch. Adds a live
httptest assertion rather than trusting the code-path read alone —
the same shape as TestClientSendsResolvedAgentHeader, scoped to
PushItem specifically since that's the one method this PR added.
* fix(push): disambiguate workspace in the monitor line and skill contract
Codex round 2 P1: the watch-events stream is user-scoped ACROSS every
workspace a caller has watches in, but formatMonitorLine printed only
ItemRef/Kind/Actor/Summary and dropped the Workspace field the wire
payload already carried — a session linked to workspace A receiving a
notification for workspace B would resolve the wrong item (or 404) with
no signal in the line that anything was off.
Fixed universally, not push-only: grepped plugin/ and skills/ for
anything parsing "PAD ..." lines and found none — the Claude Code
plugin host ingests the stdout line as free-text notification prose,
formatMonitorLine's only real consumer is its own fmt.Println, so there
is no wire-format consumer a workspace prefix could break. The
ambiguity predates push (any watched item across workspaces already had
it); push just makes the consequence sharper because it carries an
instruction rather than a passive fact.
SKILL.md's push bullet now tells the agent to resolve with
`pad --workspace <workspace> item show <ref>` using the slug read off
the notification line, not a bare `pad item show <ref>`.
* fix(push): respect --format json instead of hardcoding plain text
Codex round 2 P2: pushCmd's RunE ignored the global format flag and
always printed "Pushed <ref>", silently discarding --format json.
- server.pushResponse replaces the bare map the handler wrote before —
a typed {ref, workspace, pushed, message} shape, with workspace
resolved to the CANONICAL slug via s.getWorkspace (not merely echoed
from whatever the URL contained), matching the same disambiguation
need the round-2 P1 fix addressed for the monitor line.
- cli.PushItem now returns (*PushResult, error) instead of discarding
the response body.
- pushCmd checks formatFlag == "json" and calls cli.PrintJSON, mirroring
runCreateWatch's existing pattern.
internal/cli/agent_identity_test.go's TestPushItemSendsResolvedAgentHeader
needed a one-line update for PushItem's new two-value return — caught by
`go vet ./...`, not `go build ./...` (which doesn't compile test files);
folding vet into my own pre-flight going forward.
|
||
|
xarmian
|
212d59e7c6 |
fix(cli,server): make agent attribution actually happen (BUG-2542) (#1088)
* fix(cli,server): make agent attribution actually happen (BUG-2542)
Agent CLI writes were recorded as the human whose credentials they used.
Three independent defects, each verified by reading the path AND by
probing a live instance — the item deliberately held the mechanism open,
so none of this is inherited.
1. THE HEADER WAS NEVER SENT. actorFromRequest sets actor="agent" on one
signal: the X-Pad-Agent header. The only code that sets it took the
value from `agent_name` in .pad.toml and nowhere else — no
environment detection, no session detection. This repo's .pad.toml
has only `workspace`, so the header has never been sent from here and
every agent write has looked human. ResolveAgentName now resolves
.pad.toml → $PAD_AGENT → detected runtime.
2. ITEM CREATE DISCARDED THE ACTOR. createItemChecked called
actorFromRequest and kept only the source (`_, src :=`), never
setting input.CreatedBy, so store.CreateItem fell through to its
"user" default — even for an agent that DID send the header.
Comments have always stamped it correctly; item creation silently did
not, which made the skill's own contract false on its own terms.
3. SINGLE-ITEM PATCH NEVER STAMPED LastModifiedBy. Bulk ops do
(handlers_items_bulk.go); the single-item path did not, so an item
edited only by agents read as human-edited.
Only entries VERIFIED against a live session belong in the runtime
detection table, so it has exactly one: Claude Code exports CLAUDECODE=1
to child processes, confirmed by reading a pad subprocess's environment
inside one. Guessing at Cursor/Windsurf/Aider variable names would put
unverified claims in a shipped binary and misattribute silently when
wrong; those set $PAD_AGENT until someone confirms a signature.
WHAT THIS DOES NOT DO, stated in the code and the skill rather than left
for someone to assume: the header is client-supplied and self-declared.
An agent that omits it is indistinguishable from the human it borrows
credentials from, and a human running `! pad ...` inside an agent's
terminal inherits that environment and is attributed to the agent. This
makes the trail HONEST, not VERIFIED — it is not a basis for
machine-verifiable human-approval provenance, which needs a channel the
agent cannot author at all. The incident behind this item is exactly
that distinction: an agent's relay of a human's words was recorded
indistinguishably from the human typing them.
Contract corrected in both skill copies, since the item's first question
was which of contract and behavior was wrong. It was the contract: it
promised automatic agent attribution that only ever applied to
workspaces that had opted in.
Tests, each mutation-tested against its own defect reverted alone:
- TestResolveAgentName — precedence plus the negative that makes it mean
something: a plain human shell must still resolve to "". Fails 2/5
reverted.
- TestItemAttribution_AgentVsHuman — agent and human legs for create,
update and the create-stamp-survives-edit invariant. Fails on the
create stamp reverted; fails 2/2 on the update stamp reverted.
The update leg deliberately uses the OTHER writer: insertItemTx seeds
last_modified_by FROM created_by, so a same-writer edit passes whether
or not the PATCH stamps anything — the first version of this test did
exactly that and passed its own counterfactual. Caught only because
each fix was reverted separately.
- TestItemAttribution_ExplicitBodyValueWins — an explicit body value
still beats the header.
End-to-end on a live instance through the real CLI, no .pad.toml opt-in:
agent session → created_by/last_modified_by/comment all `agent`; same
binary with CLAUDECODE stripped → all `user`.
Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag
* fix(server): artifact import wrote a UUID into created_by (BUG-2542)
Found by Codex while reviewing the attribution fix. handleImportArtifact
set `input.CreatedBy = u.ID`, which is the wrong DOMAIN for the field
rather than merely the wrong value: created_by holds the role — "user"
or "agent" — and consumers compare it against those literals
(CommentThread.svelte, TimelineVersionCard.svelte). An imported item
matched neither and rendered as neither.
It also would have defeated the fix in the parent commit at this path: a
non-empty CreatedBy suppresses the actor stamp, so imports would have
kept a UUID while every other create path started recording the actor.
The line contradicted the comment directly above it, which said Source
was being left blank precisely so createItemChecked could stamp it "like
every other create path". Now both fields are left blank and stamped
together.
The user's identity has its own home — the items.created_by_user_id
column — which no create path currently populates. That is a separate
gap and is not widened into this change.
Claude-Session: https://claude.ai/code/session_01QGbUKZBAZoWdEgiTNWsXag
* fix: close the remaining attribution bypasses Codex found (BUG-2542)
Review found no P1s and three P2 families beyond the artifact-import bug
already fixed in
|
||
|
xarmian
|
ec7fd027fc |
feat(server,cli): watches, user-scoped event stream, plugin monitor command — PLAN-2469 Phase 1 (TASK-2533) (#1082)
* feat(store): race-free status/assignment mutation signal (TASK-2533)
Adds models.Item.LastMutation (ItemMutationSignal), populated inside the
SAME transaction that already writes status_transitions / assigned_user_id
in UpdateItemWithParentLink and MoveItemWithPreCheck. This is the
foundation for TASK-2533's watch-notification pipeline: a before/after
snapshot taken in the HTTP handler layer would race concurrent writers of
the same item, so the signal is computed where the authoritative diff
already happens, in-transaction.
* feat(store): watches table migration, both drivers (TASK-2533)
watches(id, workspace_id, user_id, item_id, predicate, created_at) per
DOC-2479's subscription-table design: durable, server-side subscriptions
that survive both the plugin-monitor process and a padd restart.
uq_watches_user_item makes `pad watch <ref>` idempotent (re-watching
upserts the predicate). Wires watches into the workspace-purge child-delete
list, mirroring item_stars.
* feat(watchevents): add in-process notification bus (TASK-2533)
New package: a global (not per-workspace) in-process pub/sub bus carrying
watch-worthy Notifications (status-change / assignment / comment; ask
reserved in the enum with no producer yet — see the follow-up server
commit). Bus is an interface specifically so a Redis-backed implementation
can slot in later without touching any caller; only MemoryBus exists today.
Package doc comment states the single-process/multi-instance limitation
explicitly, mirroring internal/events' shape.
* feat(store): watches CRUD (TASK-2533)
models.Watch + Store.CreateWatch (upsert on user+item)/GetWatchByUserItem/
ListWatchesForUser (unscoped by workspace — a watch is personal, and the
event-stream handler needs every watch a caller holds across all their
workspaces)/DeleteWatch.
* feat(server): watch/nudge event stream + CRUD endpoints (TASK-2533)
GET /api/v1/events/stream (DOC-2479): a user-scoped, cross-workspace SSE
stream, filtered server-side to the caller's watches (with optional
--until field=value predicate) plus "addressed to you" — narrowed to
assignment-to-you only for Phase 1, confirmed with the dispatcher: this
codebase has neither a Collection.Kind field nor any user->active-role
binding to ground DOC-2479's "human-gate-shaped collection targets your
active role" half mechanically. watchevents.KindAsk stays in the wire
enum with no producer. `pad session register` is the natural future hook
for a session-carried role identity.
POST/DELETE .../items/{slug}/watch, GET /api/v1/watches (unscoped,
mirrors /auth/tokens' shape for a personal, not workspace, resource).
Producer wiring (TASK-2533 audit) publishes from every live mutation path
that can produce a LastMutation signal or a new comment: handleUpdateItem
(incl. its collab sub-paths and the comment-attached-to-update path, which
bypasses handleCreateComment entirely), handleMoveItem, handleCreateComment,
item creation with an initial assignee, and the bulk-items loop (covers
archive/restore/move/set-priority/tag/untag/assign uniformly via one call
site). Named, not silent, bypasses: import bundle, status_transitions
backfill, workspace restore/purge — none are live human-facing mutations.
Known Phase-1 tradeoff, flagged not fixed: bulk mutations are NOT batched
into one notification the way the existing SSE/webhook bulk path is — a
bulk-assign of N items surfaces N individual notifications. Each is still
correctly scoped by the recipient's own watches/addressed-to-you filter
(a narrower audience than the workspace-wide SSE firehose the existing
batching protects), so this is a noise-discipline tradeoff, not a leak.
* feat(cli): pad watch + pad session register (TASK-2533)
pad watch <ref> [--until field=value] creates/upserts a durable watch;
pad watch list / pad watch remove <ref> are the hygiene companions the
dispatcher asked to be included explicitly rather than silently added.
pad watch --stream --for-session is the plugin-monitor command: one
stdout line per matching event ("PAD TASK-214 -> kind (actor): summary"),
silent on startup with no .pad.toml (hourly retry) or an unreachable padd
(backoff retry) per DOC-2479's noise-discipline contract. The retry/
backoff math and line formatting are pure, unit-tested functions; the
actual sleep loop is not (per the dispatcher's ask).
pad session register writes ~/.pad/sessions/<pid>.json (pid, cwd,
CLAUDE_CODE_MESSAGING_SOCKET when set) -- forward-looking infra for
Phase 3's live-sessions/presence surface; nothing consumes it yet in
Phase 1/2.
* fix(server): comment replies never published a watch notification (TASK-2533)
Codex round 1 finding 2 (verified real, not a false positive):
handleCreateReply is a SEPARATE code path from handleCreateComment — it
calls store.CreateComment directly via POST .../comments/{id}/replies,
not POST .../comments — and was missing the watch-notification hook
entirely. A reply to a comment on a watched item produced zero
notification. Same kind=comment publish as the top-level path, plus a
regression test covering the reply route specifically.
* fix(server): re-check current access before serving/delivering watches (TASK-2533)
Codex round 1 finding 1: ListWatchesForUser filtered only by user_id — a
watch row survives a revoked workspace membership or grant (nothing
deletes it), so GET /api/v1/watches and the event-stream's notification
filter could keep leaking item title/ref, workspace slug, actor, and
summary for access the caller no longer has.
Adds Store.ListWatchesForUser's ItemCollectionID column (needed for the
visibility check) and server.filterWatchesByCurrentAccess, which mirrors
computeSSEVisibility's RBAC resolution (handlers_events.go) — admin
bypass, VisibleCollectionIDs for member/guest full-collection access,
GuestVisibleResources for item-level grants — grouped by workspace since
a caller's watches can span many, unlike a single SSE connection scoped
to one. Fails closed on any lookup error.
Wired into handleListWatches here; the event-stream's loadWatchPredicates
call site picks up the same filter in the next commit, which also
restructures that function's Subscribe/replay sequence and therefore
touches the same lines.
* fix(watchevents): atomic ID assignment + subscribe-and-replay (TASK-2533)
Codex round 1, findings 3 and 4 (same subsystem, fixed together):
Finding 4 — sequence assignment and replay-buffer insertion happened
under SEPARATE locks in MemoryBus.Publish. Two concurrent Publish calls
could append to the ring buffer out of ID order, corrupting since()'s
ordering assumptions (it walks the ring oldest→newest assuming monotonic
IDs). Fixed by unifying seq assignment, buffer append, and the
subscriber-list snapshot under one lock; the (already non-blocking)
fan-out send still happens after releasing it.
Finding 3 — GET /api/v1/events/stream called Subscribe() and, later
(when resuming via Last-Event-ID), EventsSince() as two separate calls.
A Notification published in the window between them landed in BOTH the
replay result and the live channel, double-delivering it. Bus gains
SubscribeAndReplaySince(sinceID), which atomically subscribes and reads
the replay buffer under the SAME lock; the stream handler now uses it
whenever a Last-Event-ID is present (this commit carries that call-site
change, plus the finding-1 loadWatchPredicates filter wiring from the
previous commit — both land in the same lines of this function).
Adds a concurrent-publish ID-ordering test and a subscribe-then-
concurrent-publish no-duplicate test, both run with -race.
* fix(cli): monitor silent-start ordering + sync_required handling (TASK-2533)
Codex round 1, findings 5 and 6:
Finding 5 (P1) — runWatchMonitor called getClient() once, before the
loop and before the .pad.toml check. getClient() -> getConfiguredConfig()
os.Exit(1)s when unconfigured with no TTY, or launches an INTERACTIVE
configuration wizard when one is attached — either way a direct violation
of DOC-2479's silent-start contract, which requires "not ready yet" to be
a silent retry, never a crash or a prompt. Adds monitorClient(), which
builds the client the same way but returns a plain error instead of
exiting or prompting; client construction now happens INSIDE the loop,
after the .pad.toml gate, on every iteration, and its failure folds into
the existing padd-unreachable backoff path.
Finding 6 (P2) — streamWatchEvents ignored "sync_required" (the server's
signal that the requested Last-Event-ID was evicted from its replay
buffer), so a stale cursor got resent on every reconnect forever. Now
clears the cursor on sync_required so the next reconnect is a fresh,
non-resuming subscription instead.
Both covered by tests that assert the goroutine returns promptly on
context cancellation (proving no os.Exit / no blocking prompt was hit,
since the test process itself is still running to observe the return)
and that streamWatchEvents clears/re-tracks the cursor correctly around
sync_required.
* fix(server): uniform current-access gate for watch AND addressed-to-you delivery (TASK-2533)
Codex round 2, findings 1 and 2 — same subsystem (watch/nudge delivery
access control), fixed together; finding 2 explicitly falsifies finding
1's fix's own admin-bypass argument, so this replaces that reasoning
rather than patching around it.
Finding 1 (confirmed real): VisibleCollectionIDs / GuestVisibleCollectionIDs
deliberately over-widen for navigation — a collection ID is included if the
caller has an item grant on ANY item inside it, explicitly leaving
item-level narrowing to the caller (their own doc comments say so).
computeWatchAccessVisibility used that over-wide set directly as the
"fully visible" gate, so a guest granted item A was treated as having full
access to A's WHOLE collection, including an ungranted sibling item B.
Fixed by building the "genuinely full access" set from
GuestVisibleResources' fullCollectionIDs (populated only from direct
collection_grants, never widened by an item grant) + GetMemberCollectionAccess
/ ListSystemCollectionIDs for an actual member — exactly computeSSEVisibility's
own fullCollSet construction, not an approximation of it.
Finding 2 (confirmed real): the addressed-to-you (KindAssignment) branch in
watchNotificationVisible returned true unconditionally, with NO access
check. validateAssignmentScope (internal/store/items.go) only checks
WORKSPACE membership, never collection access, so an item can be assigned
to a "specific"-access member whose granted collections don't include it
at all — an ordinary assignment, no revocation timing required. Fixed by
gating EVERY notification kind — watch-matched and addressed-to-you alike —
through the SAME watchAccessVisibility check before either branch runs.
watchevents.Notification gains CollectionID so the check has what it needs
without a second lookup; the stream handler resolves it lazily per
workspace via a small connection-scoped cache (workspaces aren't known in
advance for addressed-to-you the way watch workspaces are), cleared on the
same reval tick that reloads the watches map.
This also required replacing computeWatchAccessVisibility's admin-bypass
argument, not just its code: "every call site filters the caller's OWN
watches" stopped being a sufficient justification once addressed-to-you
(which is fundamentally about *this* caller's own assignment activity
across every workspace) shares the same gate — a bearer-borne admin token
unconditionally trusted for that is exactly BUG-1616's blast radius. Now
mirrors computeSSEVisibility's cookie-vs-bearer distinction exactly.
Tests: guest-with-item-grant no longer sees a sibling item's watch or
stream notification (filter-level and HTTP/SSE-level); an assignment
outside a restricted member's granted collections is denied at both
levels; addressed-to-you is proven still gated (denied with no access,
visible once granted) as a pure unit test.
* fix(store): always re-read existing under lock, not just for precheck/patch updates (TASK-2533)
Codex round 2 finding 4, verified real: updateItemWithParentLinkOnce's
`existing` snapshot was only refreshed under the write lock when precheck
!= nil, ExpectedUpdatedAt != "", or FieldsPatch != nil — any update
touching none of those (e.g. a plain title-only PATCH) kept the STALE
pre-tx `existing` for the rest of the function, including the
LastMutation assignment-delta comparison added in TASK-2533's first
round. A concurrent OTHER transaction's assignment change landing between
this transaction's pre-tx read and its lock acquisition would get
misattributed to THIS transaction: a title-only update could report a
spurious, wrongly-attributed AssignmentChanged for a transition it never
made, duplicating the one the other transaction already reported
correctly (or missing a real one, depending on interleaving).
The status-transition capture already defended against exactly this with
its own separate conditional re-read; the assignment-delta capture added
later did not replicate that guard. Fixed by making the re-read
unconditional — once, right after the locks are held, before any SET-
clause building or the UPDATE itself — so every existing.* comparison in
this function is race-free by construction, not by each caller
remembering to guard itself. Also removes the now-redundant duplicate
re-read the status code had of its own.
Reproduces the exact race deterministically using UpdateItemWithPreCheck's
precheck hook as a synchronization point (TX2's assignment change blocks
mid-transaction while TX1's title-only update races its own pre-tx read
against it) — the new test fails reliably against the pre-fix code and
passes reliably (including under -race, and in Postgres mode) against
the fix.
* fix(watchevents): send under the same lock Unsubscribe/Close use (TASK-2533)
Codex round 2 finding 3, confirmed real and high-severity: Publish
snapshotted subscriber channels under the lock, released it, and only
then sent to them. A concurrent Unsubscribe or Close could close one of
those channels in the window between the snapshot and the send — a send
on a closed channel PANICS in Go, which crashes the whole padd process,
not just one subscriber's connection. The reasoning for releasing the
lock before sending ("a slow subscriber would stall everyone else") didn't
hold up: the send is already non-blocking (select/default — a full
channel is dropped-and-logged, never awaited), so holding the lock
through it costs nothing and closes the window structurally.
Adds a hammer test (many iterations of concurrent Publish / Subscribe /
Unsubscribe / Close, short-lived churned channels, recover()-wrapped so a
regression fails cleanly instead of crashing the whole `go test` run) that
reproduces "send on closed channel" dozens of times per run against the
pre-fix code (plus an independent -race detection) and passes cleanly,
repeatedly, against the fix.
* fix(server): re-fetch the user, not just the vis map, on each reval tick (TASK-2533)
Codex round 3, confirmed real: watchVisCache captured *models.User ONCE
at connect time (newWatchVisCache) and never re-fetched it; reset()
cleared only the per-workspace visibility map. computeSSEVisibility's own
doc comment explains why it re-fetches the user fresh on every call —
"so mid-stream role changes (admin demotion, user.disabled flips) take
effect on the next tick" — and the round-2 commit claimed to mirror that
"exactly," but only carried over the collection/bearer logic, not the
re-fetch itself. Net effect: a demoted or disabled admin kept fullAccess
on an open stream (both watch-matched and addressed-to-you delivery,
since both go through this same cache) until reconnect.
Adds watchVisCache.refreshUser, called by both the constructor and
reset() so the cadence matches computeSSEVisibility's actual cadence in
handlers_events.go (that function is invoked once at connect and again
only on each membershipCheck tick — never per event — so "per cache
reset" here is the same cadence, not a narrower one). Deliberately fails
CLOSED (not open-to-stale like computeSSEVisibility's own transient-error
fallback) on a fetch error, a deleted user, or a disabled user — a nudge
stream's wrong failure mode is delivering a fact to someone who
shouldn't see it, not a dropped UI update, so this trades
computeSSEVisibility's availability-leaning fallback for a stricter one
and says so in the comment rather than repeating the "mirrors exactly"
claim the fix falsified.
Tests: a unit-level pair (mirroring handlers_events_revalidation_test.go's
existing admin-demotion/disable coverage of the analogous SSE gap
exactly) proves an admin loses fullAccess after a demotion + reset(),
and a disabled user is denied outright; an HTTP/SSE-level test proves a
live stream stops delivering entirely once its connected user is
disabled and a reval tick passes. All three reproduce the bug reliably
against the pre-fix code and pass cleanly against the fix.
The HTTP-level test deliberately runs serially (not t.Parallel()): it
mutates the package-level watchListRevalInterval var, which every other
parallel watch-stream test in this package also reads via its own
ticker — writing to it from a t.Parallel() test raced against those
reads under -race (misattributed by the race detector to a whole
cluster of unrelated concurrently-running tests before this was
diagnosed). Full server package -race pass is clean after the fix.
* fix(server): decouple vis-cache reset from watch-list reload success (TASK-2533)
Codex round 4, confirmed real: on a reval tick, if ListWatchesForUser
errored, the handler's `continue` skipped visCache.reset() entirely —
the two were coupled, with reset() only reachable on the reload's
success path. A demoted or disabled user's stale identity/visibility
(round 3's fix) stayed live for exactly as long as that UNRELATED query
kept failing, so the round-3 leak reopens for the duration of any
watch-list reload error.
Fixed by running visCache.reset() first, unconditionally, before
attempting the watch-list reload. On a reload failure, the stale watch
list is kept (its own staleness is already bounded by
watchListRevalInterval's "eventually consistent" contract) but is now
gated by the FRESH visCache regardless — a demoted/disabled user is
denied via visCache even while the watch list itself lags a tick.
Chose this over dropping all delivery for the tick (the other option the
finding offered) because tying stream availability to an unrelated
query's transient health seemed like the wrong tradeoff; the comment at
the call site states this choice explicitly.
Adds a watchPredicatesLoadFault test seam on *Server (mirrors the
existing restoreAckFault pattern) so the reload failure can be forced
deterministically without breaking the DB connection for the whole test.
Reproduces the exact bug: forces the reload to fail on every tick while
concurrently disabling the connected user, and asserts addressed-to-you
delivery (which depends only on visCache, never the watch list) is
denied anyway. Fails reliably against a reverted (pre-fix, coupled)
version of the reval branch and passes cleanly against the fix.
Full server package -race pass, full suite (SQLite + Postgres) pass,
lint clean — this is the pre-PR verification matrix; round 5 will be a
narrow re-verify of this fix only.
* fix(server): bound stale watch set under persistent reload failure; atomic test seam (TASK-2533)
Codex round 5, two P2s, both confirmed real:
Finding 1 — `watches = fresh` only ran on the reload's success path, so
under a PERSISTENT (not single-tick) reload failure the watch set stayed
live indefinitely: a dead watch (removed, item deleted) kept matching
forever, and a watch created during the outage was silently missed
forever — visCache (round 4) gates current ACCESS, not whether a watch
still legitimately exists, so it couldn't catch this on its own. Fixed
by tracking consecutive reload failures and clearing the watch set once
maxConsecutiveWatchReloadFailures (3 ticks) is crossed, failing closed
on watch-matched delivery specifically while addressed-to-you delivery
(visCache-only, unaffected either way) continues throughout. Updated the
tradeoff comment at the call site so the "eventually-consistent" claim
now matches the bounded, not unbounded, behavior it actually describes.
Finding 2 — the watchPredicatesLoadFault test seam was a plain `func()
error` field, written by a test AFTER the SSE stream's background
goroutine was already running and reading it on every reval tick:
genuinely racy, unlike restoreAckFault's own use of the identical field
shape, which is set once, synchronously, before the single HTTP request
that reads it — goroutine creation's happens-before edge makes THAT
usage safe without any extra synchronization. Verified restoreAckFault
does not share the flaw and left it untouched. Fixed the watch seam with
atomic.Pointer[func() error] instead.
Test for finding 1: forces maxConsecutiveWatchReloadFailures+1
consecutive reload failures via the (now-atomic) fault seam and asserts
watch-matched delivery is suppressed once the bound is crossed while
addressed-to-you keeps delivering, then clears the fault and confirms
watch-matched delivery resumes on the next successful reload — a bounded
outage response, not a one-way ratchet. Fails reliably against the
bound disabled, passes cleanly restored.
This is the (re-run) pre-PR verification matrix per the dispatcher:
SQLite + Postgres + full-suite -race + lint + gofmt, all clean. Round 6
is a narrow re-verify of these two fixes only.
* test(store): bound the concurrent mutation-signal test's wait (TASK-2533)
CI-triage follow-up: PR #1082's plain Postgres step hit go test's default
10-minute per-binary timeout. Investigated whether any store test added by
this branch scales with runner slowness (lock-wait defaults, sleep-based
polling, transaction-hold durations):
- Watches CRUD tests (8): 0.63-0.80s each under Postgres, isolated and in
the full 741-test package run.
- Mutation-signal tests (6), including the precheck-hook two-transaction
race test: 0.48-0.80s each; the race test held at 0.48-0.51s across 10
consecutive runs (no variance) and across the full-package run.
- Full store package under Postgres: 279.17s and 277.12s across two runs
on this branch, matching the ~275s/297s baseline team-lead measured
locally and on PR #1081 — no reproducible slowdown from anything this
branch adds.
No pathological test found locally. The one test with genuine
cross-goroutine DB lock contention (TestLastMutation_AssignmentDelta_
NotMisattributedUnderConcurrentWrite) had an unbounded wg.Wait() as its
only unbounded wait — TX2's release was already unconditional (fixed 50ms
sleep, not gated on TX1's progress), so there's no deadlock risk, but
there was no ceiling on how long legitimate lock contention could
stretch it under a slow/shared runner. Replaced with a bounded 10s wait
that fails fast with a diagnostic instead of silently consuming
test-binary budget if it's ever exceeded. Verified the regression test
still fails reliably (5/5) against a revert of the round-2 fix it guards.
Could not reproduce the CI timeout locally; likely the pre-existing
~297s CI baseline (already noted as close to the 10-minute ceiling)
plus environmental variance on the shared runner, not a specific test
this branch adds.
|
||
|
David Barkhausen
|
f900b0aefb |
fix(cli): address review on markdown list output
All three requested changes from @xarmian's review of #1070, plus both nits. 1. Escape backslashes before pipes in escapeMarkdownCell. A title containing "\|" became "\|", which GFM reads as an escaped backslash followed by a LIVE pipe, so the row still gained a column. Backslash-first turns it into "\\|". Confirmed the bug with a failing test before fixing it. 2. Sanitize the group headings. Extracted SanitizeMarkdownText (SGR strip + newline collapse) and ran the collection icon and name through it, so a newline in a collection name can no longer inject a second "## " heading. Sanitizing happens per part, before joining, because it trims and would otherwise eat the separating space. Pipes are deliberately not escaped outside a table. 3. Tightened the --format help to the precise enumeration: "markdown on: item list/starred, collection list, item show, project changelog" per option (a) on #898. Nits: - `item starred --format markdown` on an empty result now says "No starred items." rather than the shared renderer's "No items found."; the empty check moved above the format branch so both paths agree. - Added format_markdown_routing_test.go: three end-to-end tests driving `item list` and `collection list` through cobra against an httptest server, asserting the markdown branch is actually reached and that the table and markdown paths don't leak into each other. Proven by disabling the markdown branch and watching the test fail. Follows the item_open_test.go pattern, with USERPROFILE set alongside HOME since os.UserHomeDir reads USERPROFILE on Windows — worth noting, as tests that set only HOME are why part of the credential-store suite fails there. Gates: go build ./... PASS; go vet PASS; gofmt clean; golangci-lint 0 issues; all markdown tests PASS. Both touched packages show the same 6+2 pre-existing Windows failures as clean main under an identical sandboxed run. |
||
|
David Barkhausen
|
4ab7b10b35 |
feat(cli): markdown output for the list commands
Implements `--format markdown` on the list commands that lacked it, so the format is honestly global rather than honestly-partial (#898, the option (a) follow-up to #851). - `pad item list` — grouped `## Icon Name (N)` sections with a table each when listing across collections (mirroring the table layout), a single table when scoped to one collection. Heading style matches `project changelog`. - `pad item starred` — single table. - `pad collection list` — Name / Slug / Items / Default. - `--format` help no longer carries the "markdown on select commands" caveat. The markdown renderers deliberately do NOT reuse the colorized helpers (ColorizedStatus, PriorityColor, Dim): markdown goes to a file, a PR body or an agent's context, never a terminal, so raw values go in and the reader's renderer styles them. Every cell is escaped — an unescaped `|` in a title silently adds a column and corrupts the row. Refs #898 |