mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-10 23:15:40 +00:00
bc87aee6b520fc0ef2b7041cf63483f2cec2a124
1483 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 xarmian
|
bc87aee6b5 |
fix(docs,metrics): repair the metrics table I split, and bound the fan-out claim (BUG-2739, codex round 19)
Two real findings from an unconstrained fresh-eyes pass. THE TABLE WAS BROKEN. Round 11's rollout note was inserted BETWEEN two rows of the metrics table, so every row after it — eight of them, including all the pad_event_* counters and the presence failures — rendered as plain pipe-delimited text rather than a table. A documentation change that silently breaks the page it documents is worse than the omission it fixed, and no gate in this repo renders Markdown. The note now sits after the table, and a check across the whole file confirms no blank line or prose splits any of its eight tables. THE FAN-OUT CLAIM WAS STILL TOO STRONG, in both the metric help and the docs row: 'each moves pad_watchevents_midstream_resyncs_total once per such subscriber'. The gap signal is capacity-1 and coalescing, so a second cause firing before a client has acted on the first adds no announcement. It is AT MOST one per subscriber, and reading the fan-out off the two counters needs a reset observed in isolation against idle clients. Round 5 narrowed the aggregate version of this claim and left the per-event one standing. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
603c7f77c3 |
docs(watchevents): bring the shared Bus contract up to what RedisBus now does (BUG-2739, codex round 18)
A parity round asked whether MemoryBus and RedisBus are still describable by one contract. They are — MemoryBus has no transport to lose anything in, so its sequence is contiguous by construction and it raises only the per-subscriber signal — but the interface DOCS had drifted: Subscribe listed the per-instance causes as 'a hole in what it received from Redis', and SubscribeAndReplaySince's nil-replay list predated both new causes. The interface is what a caller reads, so it now names the two scopes explicitly, says the per-instance causes are all RedisBus's because they are all transport faults, and warns that the absence of an instance-wide signal is evidence about which bus you have and nothing else. SubscribeAndReplaySince's wording also gets the distinction this branch has been correcting everywhere else: a nil replay means the span cannot be vouched for, NOT that something was necessarily lost. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
3895fc4f0c |
test(watchevents): record the startup guard's one unordered window, with the measurement (BUG-2739, codex round 17)
A mutation-coverage audit named a concrete killer for every new test — and for the startup control it named a way to SURVIVE instead: nothing orders the initial subscribe confirmation against newCutterBus attaching the observer, so a mutated constructor could in principle slip past unobserved. The window is real and does not open. Measured with the mutation applied rather than argued: 50/50 caught plain, 30/30 caught under -race, which perturbs scheduling hard enough to surface a genuine race. The asymmetry is why — the confirmation needs a round trip through a TCP proxy into a goroutine that has not started, while SetObserver is two field writes on the goroutine that just returned from the constructor. Closing it properly would mean a construction-time observer seam, widening production API for a test. The measurement is the better trade, and it is now written where a future flake will be read as this window opening rather than as noise. The audit found the other ten tests each killable by a named mutation, which matches the matrix run here — with one difference worth noting: it derived them from the code, and the matrix ran them. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
bc6d3541cb |
refactor(watchevents,docs): order the startup test, move operator prose out of the code (BUG-2739, codex round 16)
A future-maintainer round, three of whose four findings were fair. THE REAL DEFECT: TestNoCoverageIsDroppedAtStartup slept 500ms and hoped the receive goroutine had had its chance. No happens-before, so it could miss the regression or turn scheduler-sensitive. It now publishes and waits for DELIVERY instead: the pub/sub channel is FIFO, so a startup confirmation — if the constructor stopped consuming it — is queued AHEAD of that notification and has necessarily been processed by the time it comes out the other end. Deterministic, strictly stronger, and 0.00s instead of 0.50s. Re-verified against its mutation: removing the constructor's Receive still fails it 3/3. COMMENT ACCRETION: dropCoverage had 76 lines of commentary over 30 of code, including a threat model and a per-message cost breakdown that are operator decisions. Those moved to docs/deployment.md, where operators actually read, and the code keeps the invariants and the one design question a maintainer will ask (why not gate on the shared counter). 45 lines now, and nothing was deleted — only relocated to the artifact whose audience it was written for. THE TIME BOMB: the rollout note said 'this paragraph expires at the next tag' with nothing enforcing it. A claim about release state that goes stale silently is exactly what this branch has spent nine rounds removing, so it now carries the three commands to re-derive it instead of asking to be trusted. DECLINED: extracting fanOutLocally's switch into a coverage-state transition helper. The accretion is real and predates this branch — the switch, its four fields and their reset duplication are the existing design, to which this added one arm. A state-machine refactor of the receive path is its own change with its own review and its own mutation matrix; folding it into a bug fix at round 16 is how a fix's blast radius stops matching its claim. Worth filing if a third reviewer raises it. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
d1501d9cf2 |
fix(watchevents): keep backward detection alive across a coverage drop (BUG-2739, codex round 13)
A regression this branch introduced, found by a round asked to ignore documentation and look only at runtime behaviour. Backward-counter detection asks 'did an id arrive at or below what we already hold', and it read lastAppendedID — which dropCoverage sets to 0. So a counter reset happening DURING the outage that caused the drop arrived afterwards looking like an ordinary cold start, and counter_backward was never reported. The pairing is not exotic: a Redis restarted from a stale snapshot drops every connection (the resubscription) and restores watchevents_seq to an older value (the reset) in the same event, which is precisely the incident an operator would be staring at. Fixed with a highWaterID that survives dropCoverage and is discarded only with the epoch it belongs to, plus a cold-start arm that recognises an id at or below it. It changes NO coverage decision — both roads reach knownFrom = n.ID over an emptied buffer — so this is the operator's signal only, which docs/deployment.md leans on per migration phase. Writing it exposed a second obligation the same mark creates: every site that learns of a reset must REBASE it onto the new space, or one incident becomes a run of false ones as the restarted sequence climbs below the old peak. Two such sites, and the pre-existing backward arm is one of them. Mutation matrix, 4 applied and grep-verified, all 4 caught — but only after two of them survived a first pass and showed the tests stopped one step short: both the stale-mark failures are invisible until a LATER coverage drop, and the epoch arm's own epochJustChanged flag suppresses detection for exactly the one notification the first version asserted on. The extended legs say so in place. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
2f1896b5fb |
test(watchevents): a diagnostic that described go-redis behaviour it does not have (BUG-2739, codex round 12)
TestCloseDoesNotLookLikeAResubscription's failure message told the reader that closing the PubSub emits unsubscribe confirmations and that the Kind filter is what separates them. Neither half is true: PubSub.Close in the pinned go-redis v9.22.0 closes the connection without sending UNSUBSCRIBE, so no confirmation is emitted at all — checked in the module source. The message now points at what a failure would actually mean. The filter's own comment gains the same fact, which strengthens rather than weakens it: nothing in this bus produces an unsubscribe confirmation by either road, so it is defence, and it is kept because a Kind switch that names only the case it handles is a switch waiting to mis-handle the others. Third time this run that I have described a mechanism I had not read — the first two were the go-redis channel bound (checked, and correct) and the initial-confirmation timing (checked by mutation, and correct). This one was neither checked nor correct, and it was sitting in a message a future maintainer would read while debugging. The rest of the Go-language angle came back clean: type switch, nil handling, shadowing, defer ordering, select spin, goroutine leaks. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
988bea658c |
docs(deployment): state the watch reset metric has no released contract (BUG-2739, codex round 11)
A mixed-version-fleet round raised two things — the label spelling, and the
metric widening from 'the ID space changed' to 'replay coverage was dropped'
with two new reasons — as a rollout hazard where old and new instances report
two shapes under one name.
Both collapse to one fact, verified rather than assumed: the entire metric was
introduced by
|
||
|
xarmian
|
c84418579e |
docs(watchevents): cost out the malformed-message flood rather than waving at it (BUG-2739, codex round 10)
A security/resource-exhaustion round re-raised the undecodable arm as a flood vector, with more detail than round 8's threat-model note answered. 'Out of threat model' is not the same as 'free', so the costs are now itemised where the arm lives. Three of the four per-message costs are self-bounding: the subscriber pass is a non-blocking send onto a capacity-1 flag that is already raised, so a flood collapses to nothing there; the metric increment IS the alarm an operator wants, at full rate; and the receive loop is serial, so buffer allocations are one at a time against a GC rather than a growing heap. LOG VOLUME is the genuine residual — one ERROR line per message — and bounding it needs a threshold, which is the deployment decision this unit keeps declining to make on the operator's behalf. Payload size is deliberately not capped, and the reason is that a cap here would do nothing: go-redis has read the whole message into memory before decodePayload sees it. That bound belongs to the Redis deployment (proto-max-bulk-len) and to who holds PUBLISH. The round found no log injection (the attacker-controlled fragment is %q-escaped), no per-message goroutine leak, and no retained unbounded structure. No code change. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
720182a445 |
docs(watchevents): sweep the undecodable overclaim as a class, with its boundary (BUG-2739, codex round 9)
Third round running on the same claim, each time in a place the previous
instance fix had not looked — which is CONVE-18's failure mode in miniature,
committed by me twice after writing the sweep into two commit messages.
THE POPULATION, enumerated this time instead of chased: grep for
'undecodable' across the tree. Within internal/watchevents and the artifacts
this branch owns, three sites still said a notification was LOST — the test's
headline sentence (contradicted by its own closing paragraph eight lines
below), the receive-loop comment ('the id of what we missed is unknown BY
DEFINITION'), and the deployment table's trailing summary ('part of it was
simply missed'). All three now say what the instance actually knows:
something arrived that it could not read, it cannot tell whether that was
ours, and it stops vouching BECAUSE it cannot tell — a claim about our own
evidence rather than about the stream.
THE SEARCH BOUNDARY: internal/watchevents, internal/metrics and
docs/deployment.md — the artifacts this change owns. internal/events carries
the same wording for its own undecodable arm (observer.go, bus.go, the
pad_event_* Help string and its docs row) and is NOT touched here. Whether
that wording is equally overclaimed is an open question about a package this
PR otherwise leaves alone, not a cleared one.
Round 9 also read the branch's commits in order for cross-commit interaction
and found none.
Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X
|
||
|
xarmian
|
90f4191b45 |
docs(watchevents): record why dropCoverage is not counter-gated (BUG-2739, codex round 8)
An adversarial design round argued the reset should be deferred until counter divergence is established, so a harmless reconnect does not discard a buffer that was still complete. The argument is right about the COST and wrong about the remedy, and the reasoning is now on dropCoverage rather than in a review transcript. The remedy needs a shared-counter read on the receive path. A resubscription means the Redis connection just failed, so that read either fails — and resumeOutrunsLocalView answers false on a failed read by design, proceeding on local knowledge — or blocks the single receive goroutine on network I/O and stalls delivery for every subscriber. Ending coverage locally is the record that survives Redis being unreadable, which is why it is the one kept. The cost is stated rather than glossed: when nothing was published during the outage, a complete buffer is discarded and a client reconnecting before the next notification is answered sync_required instead of replayed. That is this package's standing trade, and it is bounded by the next notification. Also recorded: why dropping at all, given the resume path and the gap arm each catch a real loss independently — both need something to happen, a reachable Redis or a later publish, and the live subscriber on a stream that goes quiet has neither. And the threat model for the undecodable arm, where a channel writer can force a resync: anyone who can publish there can already publish forged notifications, so the realistic cause is two installations sharing a Redis without PAD_REDIS_NAMESPACE (BUG-2724). Rate-limiting it would need a threshold, and a threshold is a deployment decision — the same reason BUG-2738 is not fixed here. No code change. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
e7ba44136e |
test(watchevents,metrics): cover the three behaviours nothing exercised (BUG-2739, codex round 7)
A 'what has NO test at all' pass, which is a different question from 'is each fix pinned' and found three gaps after six rounds of checking individual fixes. 1. The two new reset reasons never reached the metrics adapter. Now covered — and with LITERAL label strings rather than the exported constants, which is the point rather than a style choice: every other assertion in that file derives its expected label from the same constant the code emits, so the pair moves together and a rename stays green while operator dashboards break. A metric label is a wire format with consumers outside this repo. counter_backward is pinned literally too, since its spelling changed on this branch and nothing would have noticed a revert. Mutation-verified: putting the plural back fails the test on two legs, including the emit-both-spellings shim a 'compatibility' fix would produce. 2. A resubscription on an IDLE bus — nothing published yet, so no bookkeeping to clear — was exercised by nothing, and it is the case that matters most: an idle instance's subscribers are precisely the ones a flap can strand, because no later notification is coming to expose anything. Caught the plausible optimisation (skip dropCoverage when nothing was covered), which would silently reintroduce this bug for the quietest instances. 3. Close reporting a spurious resubscription had no test. The invariant is now pinned. Its comment ALSO records what it does not do: deleting the Kind filter leaves it green, because Close leaves the loop through ctx.Done() rather than reading an unsubscribe confirmation, and this bus never calls pubsub.Unsubscribe. The filter is labelled as defence where it lives, in the same form as the knownFrom reset. Found by running the mutation on a comment that claimed otherwise. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
76aa111e53 |
docs(metrics,deployment): finish the undecodable wording and answer the label question in the artifact (BUG-2739, codex round 6)
Round 6's end-to-end failover trace matched the comments step for step, so both findings were prose again. The struct comment above WatchSequenceResetsTotal still grouped undecodable_message with subscription_resumed as 'we simply lost part of it'. Only the second is demonstrable: for the first, the instance knows an unreadable message arrived and cannot tell whether it was ours. Round 5 corrected the exported Help string and the docs but not this one — the same claim in a fourth place, which is what a class-wide sweep is supposed to prevent. The counter_backward rename has now been raised three rounds running, each time because the answer lives in a commit message and a reviewer reads artifacts. It is answered where an operator with a broken dashboard would look: no tagged release emitted the plural, so there is nothing to migrate. Re-derived at 40b0db06 rather than restated. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
2e9ace4194 |
docs: nine overclaims across code, metrics, docs and the CLI (BUG-2739, codex round 5)
A cross-artifact pass, which is the angle that keeps paying on this family. Every item below was a statement of mine that was false or unsupported; the code did not change. WRONG FACTS: - Watch epochs are opaque UUIDs, not numeric generations. I had copied internal/events' wording, where they ARE numeric — the distinction is the subject of internal/idspace's package comment. - undecodable_message was described as proof a notification was missed. The instance knows only that something it could not read arrived on its channel; it cannot tell whether that was ours. It stops vouching BECAUSE it cannot tell, which is a different and weaker claim. Corrected in four places. - The failover-cost paragraph said every SSE client on the instance reconciles. Wrong twice: a watch-bus resubscription ends the WATCH stream's coverage (activity coverage is per-workspace), and the one client that uses that stream today — pad watch --stream — answers sync_required by clearing its cursor and keeping the connection open, so it issues no request at all. Verified in cmd_watch.go rather than assumed. - The midstream/reset ratio is not fan-out in aggregate: the announcement counter also carries gaps and slow-subscriber drops and coalesces per connection. Only a reset observed in isolation reads that way. - 'The watch stream's only signal was a later non-contiguous notification' is true for a client HOLDING A STREAM OPEN. A reconnecting client was always covered, because a resume asks the shared counter instead of local state. Scoped in the doc and in the test header. - The dropped-confirmation fallback said coverage still ends. Usually, not necessarily: with no traffic during the outage nothing was lost, and if the drops continue through whatever would expose the hole and the stream goes quiet, nothing ever does — BUG-2727's boundary. Named both. - The new Observer Close warning was overbroad: reports run on the receive goroutine only on the RedisBus receive path, while a ResumeGap runs on the caller's and MemoryBus has no such goroutine. The rule stays unconditional, since a callback cannot tell which case it is in, but it now says why. STALE AFTER THIS BRANCH: - metrics.go's WatchSequenceResetsTotal comment listed two reset reasons. - observer_test.go said 'both reset reasons'. - The constructor comment named Channel() after the loop moved to ChannelWithSubscriptions, and did not say the Receive beneath it is load-bearing for that loop having no skip-the-first flag. It does now, and names the test that fails if it goes away. - cmd_watch.go's sync_required cause list predated BUG-2739 (and did not mention the mid-stream delivery BUG-2730 added). Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
c9f0dc60db |
docs(watchevents): an observer must not call Close either (BUG-2739, codex round 4)
The Observer contract named ONE forbidden call — a bus method that reports, because that is unbounded mutual recursion. Reports fire on the receive goroutine and Close waits on that goroutine through wg.Wait(), so an observer that closes the bus from a callback waits on itself forever. Naming only the first hazard reads as though the second were allowed. Not a lock-ordering problem and not fixable by one: the goroutine is the resource being waited for. The round's other finding — the receive path mutating state and reporting metrics after Close, because Close makes both select cases ready and Go picks at random — is PRE-EXISTING and filed as BUG-2741 rather than folded in. Two of its three sites predate this unit; the third is dropCoverage. Filing states the population and the search boundary per CONVE-18: three locked sites in redis_bus.go, internal/watchevents only, internal/events not searched. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
b27407c7ab |
test(watchevents): wait on the expected reset REASON, and pin the startup control with its mutation (BUG-2739, codex round 3)
waitForResets accepted any reset, so an unrelated one could satisfy the wait and let a test run its real assertions before the condition under test had happened — a wait that measures nothing and leaves what follows racing. It is reason-specific now, and its failure message names what it wanted and what it saw. The same round argued TestNoCoverageIsDroppedAtStartup could not do its job, because the initial subscribe confirmation would land before SetObserver and Subscribe are called and so go unrecorded. Ran the mutation instead of arguing the timing: with the constructor's pubsub.Receive removed, the test fails 3/3 with got map[subscription_resumed:1]. go-redis does not deliver the confirmation until the ChannelWithSubscriptions goroutine reads it, which is after the constructor returns. The reasoning and the 3/3 result are now on the test, so the next reader does not have to re-derive it — including the part codex was right about, that this test PASSES on unfixed main, because it is a control rather than a regression test. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
5d5450a43d |
docs(watchevents,deployment): name the failover cost and the dropped-confirmation road (BUG-2739, codex round 2)
Two operator-angle findings, both real and neither a code defect. A subscription confirmation goes through the SAME bounded channel as messages — go-redis v9.22.0 initAllChan handles `case *Subscription, *Message:` identically, chanSize 100, chanSendTimeout 1 minute — so under sustained load the resubscription marker can be dropped like any message. Checked in the library rather than argued. Coverage still ends by the other road: a full channel means traffic, the outage left a hole in the ids, and the gap arm raises it on the next message consumed. The operator gets a less specific label for the same truth. BUG-2727's standing boundary (a drop whose hole no later notification exposes) is unchanged in both directions. And detection is not free: a resubscription ends coverage for the whole instance, so every connected SSE client reconciles at once — up to PAD_SSE_MAX_CONNECTIONS of them, since per-connection coalescing smooths repeats within a wave and not the wave itself. Named in the deployment doc with the ratio that measures it, because an operator meeting this for the first time during a failover should not have to derive it. Round 2 also re-raised the counter_backward rename and the half-open connection. The first is answered by the ancestry evidence in 40b0db06 — nothing released carries either spelling. The second is BUG-2738, already named in this doc as a surviving residual and rulings-first per the lead. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
3b2df81032 |
docs(deployment): the activity stream cannot detect ID-sequence holes (BUG-2739, codex round 1)
The rewritten paragraph claimed both streams now detect the same three things, ID-sequence holes included. They do not, and the paragraph directly below it said so — its per-workspace IDs come from a counter shared across workspaces, so holes in them are the normal state and no arithmetic on them means anything. That is why pad_watchevents_sequence_gaps_total has no pad_event_* counterpart, which is now stated where an operator looking for the missing counter would look. What BUG-2739 actually equalises is the two DIRECT detections: a pub/sub resubscription and an undecodable message. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
48d3c90b07 |
docs(watchevents): score the knownFrom reset honestly — it is defence, not a tested line (BUG-2739)
The previous commit's test comment claimed the EventsSince leg was what catches a dropCoverage that skips `knownFrom = 0`. It is not: that mutation survives every assertion in the file, because emptying the buffer on the line above already makes replayBuffer.since answer nil for any sinceID > 0, and the next notification takes the cold-start arm and overwrites knownFrom anyway. Not every mutation is a fair test, and a survivor is a question rather than a verdict. The answer here is that the line is unobservable today and worth keeping regardless — without it, dropCoverage's correctness would rest on another type's internal guard in another file, for a reason unrelated to coverage. That is now said on the line itself, in the form the receive loop's own ctx.Err() defence uses: NO TEST FAILS IF IT IS DELETED. The EventsSince leg is kept too, with its claim corrected to what it actually does: it pins the local-view invariant independently of the shared-counter check, which is what answers when a Redis read fails — the condition a flap produces. Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X |
||
|
xarmian
|
b24a1bceb9 |
test(watchevents): close two mutation survivors in the BUG-2739 tests
Both survived because the assertion was rescued by a mechanism unrelated to
the fix — CONVE-12's shape, found by mutation rather than by reading.
- dropCoverage skipping `knownFrom = 0` survived everything. The resume
path consults the shared counter BEFORE local state, and that check
refuses the cursor on its own once lastAppendedID is back to 0. The new
leg asserts through EventsSince, which answers from local state only and
deliberately skips the authority check. Not a mutation-only concern: the
authority check answers false when the Redis read fails, which is exactly
when a flap is happening.
- dropCoverage keeping the replay buffer survived everything. sinceID == 0
is 'everything you have buffered' and does not go through the coverage
guard, so a subscriber arriving after the outage was handed the pre-flap
notifications and would believe itself caught up at the newest of them —
with the flap's lost ids just above, nothing later non-contiguous to
expose them, and no signal, since it arrived after the signalled set was
taken.
Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X
|
||
|
xarmian
|
a3a98b249a |
fix(watchevents,metrics): unify the reset label on counter_backward (BUG-2739)
The two buses spelled the same condition one letter apart: internal/watchevents emitted counter_backwards, internal/events emits counter_backward. Same metric family, same meaning — so an operator writing one alert expression across both gets silence from one of them. Singular wins because it is the majority and the documented one: internal/events' constant, both metric help strings, and docs/deployment.md (the reasons table, the ID-space migration section, and the phase notes) all say counter_backward. watchevents' plural, added in BUG-2727, is the lone deviation. CONTRACT-SAFE, and this is the load-bearing half rather than a nicety, since renaming an emitted metric label ordinarily breaks any alert built on it. Nothing released carries either spelling. Re-derived in this session rather than carried from the ruling's date: git describe --tags --abbrev=0 origin/main -> v0.14.0 git rev-list --count v0.14.0..origin/main -> 128 git merge-base --is-ancestor |
||
|
xarmian
|
5184aba852 |
fix(watchevents): detect the two holes the watch bus could not see (BUG-2739)
The watch bus learned of a hole ONLY when a later notification arrived with a
non-contiguous id. So a Redis flap that lost the NEWEST notification, on a
stream that then went quiet, left every connected CLI silently stale
indefinitely: nothing later ever arrived to be non-consecutive with. The
activity bus has detected both of these directly since BUG-2731; this ports
them.
Two conditions now end this instance's coverage:
- a pub/sub RESUBSCRIPTION. go-redis reconnects and re-subscribes silently,
and whatever was published during the outage never reaches us. Requires
ChannelWithSubscriptions, which surfaces the confirmations Channel hides.
- an UNDECODABLE message. It is not enough that this bus's ids are
consecutive by construction so the gap arm would catch it next time —
that detection needs a next time, and the case that matters is an
undecodable newest message on a quiet stream.
NO "SKIP THE FIRST CONFIRMATION" FLAG, which is the one place a port of
internal/events' loop would have been wrong. That package's receive loop is
handed a fresh PubSub nobody has read from, so its initial confirmation
arrives on the channel and must be skipped. Ours does not:
NewRedisBusWithKeys calls pubsub.Receive before the goroutine starts and that
Receive consumes the initial confirmation — verified with a probe, which saw
zero subscriptions on the channel at startup. Copying the flag would have
swallowed the first GENUINE resubscription, i.e. shipped this bug wearing a
fix. TestNoCoverageIsDroppedAtStartup is the enforcement for that dependency,
not a comment: it fails if the constructor's Receive is ever removed.
dropCoverage resets replay, lastAppendedID and knownFrom TOGETHER. Clearing
the buffer and knownFrom while leaving lastAppendedID stale makes the next
notification read as contiguous, so no arm of fanOutLocally's switch fires,
knownFrom is never re-established, and replaySince refuses every resume on
that instance forever — correct-looking and permanently broken. The recovery
test was written before the refusal test for exactly that reason: a bricked
bus refuses too, so asserting only the refusal cannot tell them apart.
epochJustChanged is deliberately not set: both conditions are a hole in our
view of the SAME id space, so the cold-start arm's ordinary knownFrom = n.ID
is right. The +1 exists only for the ambiguity between two id spaces.
Live subscribers are told through signalAllLocked, which BUG-2730 left in
place for this shape — so the client holding the stream open across the flap
gets sync_required mid-stream, which is the whole point of the unit.
tcpCutter is ported from internal/events' reconnect test for the reason its
header gives: nothing short of a real severed connection produces a
resubscription, so testing the decision logic alone would leave the wiring
claim unproven (CONVE-19).
docs/deployment.md's paragraph stating this asymmetry as a known gap is
rewritten rather than deleted, and now names both surviving residuals:
BUG-2735 (a message lost in transit with the connection intact) and BUG-2738
(a half-open connection, which nothing here can see because go-redis's
pub/sub health check writes without reading).
Claude-Session: https://claude.ai/code/session_01JVDBKbgn3Xt7ndW1YoYd8X
|
||
|
dependabot[bot]
|
ff969d23e0 |
chore(deps)(deps): bump the go-minor-and-patch group across 1 directory with 4 updates (#1180)
* chore(deps)(deps): bump the go-minor-and-patch group across 1 directory with 4 updates Bumps the go-minor-and-patch group with 4 updates in the / directory: [github.com/go-chi/chi/v5](https://github.com/go-chi/chi), [github.com/mark3labs/mcp-go](https://github.com/mark3labs/mcp-go), [golang.org/x/crypto](https://github.com/golang/crypto) and [modernc.org/sqlite](https://gitlab.com/cznic/sqlite). Updates `github.com/go-chi/chi/v5` from 5.3.1 to 5.3.2 - [Release notes](https://github.com/go-chi/chi/releases) - [Changelog](https://github.com/go-chi/chi/blob/master/CHANGELOG.md) - [Commits](https://github.com/go-chi/chi/compare/v5.3.1...v5.3.2) Updates `github.com/mark3labs/mcp-go` from 0.57.0 to 0.58.0 - [Release notes](https://github.com/mark3labs/mcp-go/releases) - [Commits](https://github.com/mark3labs/mcp-go/compare/v0.57.0...v0.58.0) Updates `golang.org/x/crypto` from 0.54.0 to 0.55.0 - [Commits](https://github.com/golang/crypto/compare/v0.54.0...v0.55.0) Updates `modernc.org/sqlite` from 1.56.0 to 1.57.0 - [Changelog](https://gitlab.com/cznic/sqlite/blob/master/CHANGELOG.md) - [Commits](https://gitlab.com/cznic/sqlite/compare/v1.56.0...v1.57.0) --- updated-dependencies: - dependency-name: github.com/go-chi/chi/v5 dependency-version: 5.3.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: go-minor-and-patch - dependency-name: github.com/mark3labs/mcp-go dependency-version: 0.58.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-minor-and-patch - dependency-name: golang.org/x/crypto dependency-version: 0.55.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-minor-and-patch - dependency-name: modernc.org/sqlite dependency-version: 1.57.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-minor-and-patch ... Signed-off-by: dependabot[bot] <support@github.com> * nix: vendorHash for the go-minor-and-patch bump The four module updates change the vendored dep set; hash taken from the fixed-output derivation mismatch on this PR's own Nix run (the in-branch fix the day-26 batch established on #1041). Claude-Session: https://claude.ai/code/session_01HvAuiZ7JaWyCqqyV99LyWt --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xarmian <xarmian@gmail.com> |
||
|
dependabot[bot]
|
0ccf178e91 |
chore(deps)(deps-dev): bump jsdom from 26.1.0 to 29.1.1 in /web (#1141)
* chore(deps)(deps-dev): bump jsdom from 26.1.0 to 29.1.1 in /web Bumps [jsdom](https://github.com/jsdom/jsdom) from 26.1.0 to 29.1.1. - [Release notes](https://github.com/jsdom/jsdom/releases) - [Commits](https://github.com/jsdom/jsdom/compare/v26.1.0...v29.1.1) --- updated-dependencies: - dependency-name: jsdom dependency-version: 29.1.1 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * test(a11y): emulate the :modal-unsupported engine explicitly under jsdom 29 jsdom 26 threw on the :modal pseudo-class, so the fallback-path tests in viewerBackdrop.svelte.test.ts ran their premise on the bare environment for free. jsdom 29 PARSES :modal but never matches it (the setup-jsdom showModal polyfill sets no top-layer state), which the module's probe reads as a supporting engine — flipping three tests off the path their titles name and silently shifting a fourth. The unsupported engine is now emulated the same way the supporting one always was: mockModalUnsupported() throws SyntaxError from every probe the module makes (querySelector, querySelectorAll, Element.matches), per the re-take note on TASK-2586. Claude-Session: https://claude.ai/code/session_01HvAuiZ7JaWyCqqyV99LyWt --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: xarmian <xarmian@gmail.com> |
||
|
dependabot[bot]
|
ac0f1180a9 |
chore(deps)(deps): bump the npm-minor-and-patch group across 1 directory with 17 updates (#1140)
Bumps the npm-minor-and-patch group with 16 updates in the /web directory: | Package | From | To | | --- | --- | --- | | [@dagrejs/dagre](https://github.com/dagrejs/dagre) | `3.1.0` | `3.1.1` | | [@tiptap/core](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/core) | `3.29.2` | `3.30.1` | | [@tiptap/extension-bubble-menu](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/extension-bubble-menu) | `3.29.2` | `3.30.1` | | [@tiptap/extension-code-block-lowlight](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/extension-code-block-lowlight) | `3.29.2` | `3.30.1` | | [@tiptap/extension-collaboration](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/extension-collaboration) | `3.29.2` | `3.30.1` | | [@tiptap/extension-collaboration-caret](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/extension-collaboration-caret) | `3.29.2` | `3.30.1` | | [@tiptap/extension-placeholder](https://github.com/ueberdosis/tiptap/tree/HEAD/packages-deprecated/extension-placeholder) | `3.29.2` | `3.30.1` | | [@tiptap/extension-table](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/extension-table) | `3.29.2` | `3.30.1` | | [@tiptap/extension-task-item](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/extension-task-item) | `3.29.2` | `3.30.1` | | [@tiptap/extension-task-list](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/extension-task-list) | `3.29.2` | `3.30.1` | | [@tiptap/pm](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/pm) | `3.29.2` | `3.30.1` | | [@tiptap/starter-kit](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/starter-kit) | `3.29.2` | `3.30.1` | | [@tiptap/suggestion](https://github.com/ueberdosis/tiptap/tree/HEAD/packages/suggestion) | `3.29.2` | `3.30.1` | | [@sveltejs/vite-plugin-svelte](https://github.com/sveltejs/vite-plugin-svelte/tree/HEAD/packages/vite-plugin-svelte) | `7.2.0` | `7.3.0` | | [svelte](https://github.com/sveltejs/svelte/tree/HEAD/packages/svelte) | `5.56.8` | `5.56.9` | | [svelte-check](https://github.com/sveltejs/language-tools) | `4.7.5` | `4.7.6` | Updates `@dagrejs/dagre` from 3.1.0 to 3.1.1 - [Release notes](https://github.com/dagrejs/dagre/releases) - [Changelog](https://github.com/dagrejs/dagre/blob/master/changelog.md) - [Commits](https://github.com/dagrejs/dagre/compare/v3.1.0...v3.1.1) Updates `@tiptap/core` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/core/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/core) Updates `@tiptap/extension-bubble-menu` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/extension-bubble-menu/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/extension-bubble-menu) Updates `@tiptap/extension-code-block-lowlight` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/extension-code-block-lowlight/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/extension-code-block-lowlight) Updates `@tiptap/extension-collaboration` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/extension-collaboration/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/extension-collaboration) Updates `@tiptap/extension-collaboration-caret` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/extension-collaboration-caret/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/extension-collaboration-caret) Updates `@tiptap/extension-link` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/extension-link/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/extension-link) Updates `@tiptap/extension-placeholder` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages-deprecated/extension-placeholder/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages-deprecated/extension-placeholder) Updates `@tiptap/extension-table` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/extension-table/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/extension-table) Updates `@tiptap/extension-task-item` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/extension-task-item) Updates `@tiptap/extension-task-list` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/extension-task-list) Updates `@tiptap/pm` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/pm/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/pm) Updates `@tiptap/starter-kit` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/starter-kit/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/starter-kit) Updates `@tiptap/suggestion` from 3.29.2 to 3.30.1 - [Release notes](https://github.com/ueberdosis/tiptap/releases) - [Changelog](https://github.com/ueberdosis/tiptap/blob/main/packages/suggestion/CHANGELOG.md) - [Commits](https://github.com/ueberdosis/tiptap/commits/v3.30.1/packages/suggestion) Updates `@sveltejs/vite-plugin-svelte` from 7.2.0 to 7.3.0 - [Release notes](https://github.com/sveltejs/vite-plugin-svelte/releases) - [Changelog](https://github.com/sveltejs/vite-plugin-svelte/blob/main/packages/vite-plugin-svelte/CHANGELOG.md) - [Commits](https://github.com/sveltejs/vite-plugin-svelte/commits/@sveltejs/vite-plugin-svelte@7.3.0/packages/vite-plugin-svelte) Updates `svelte` from 5.56.8 to 5.56.9 - [Release notes](https://github.com/sveltejs/svelte/releases) - [Changelog](https://github.com/sveltejs/svelte/blob/main/packages/svelte/CHANGELOG.md) - [Commits](https://github.com/sveltejs/svelte/commits/svelte@5.56.9/packages/svelte) Updates `svelte-check` from 4.7.5 to 4.7.6 - [Release notes](https://github.com/sveltejs/language-tools/releases) - [Commits](https://github.com/sveltejs/language-tools/compare/svelte-check@4.7.5...svelte-check@4.7.6) --- updated-dependencies: - dependency-name: "@dagrejs/dagre" dependency-version: 3.1.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: npm-minor-and-patch - dependency-name: "@sveltejs/vite-plugin-svelte" dependency-version: 7.3.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/core" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/extension-bubble-menu" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/extension-code-block-lowlight" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/extension-collaboration" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/extension-collaboration-caret" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/extension-link" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/extension-placeholder" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/extension-table" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/extension-task-item" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/extension-task-list" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/pm" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/starter-kit" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: "@tiptap/suggestion" dependency-version: 3.30.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-minor-and-patch - dependency-name: svelte dependency-version: 5.56.9 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm-minor-and-patch - dependency-name: svelte-check dependency-version: 4.7.6 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm-minor-and-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
xarmian
|
a1373cfcc3 |
Merge pull request #1179 from PerpetualSoftware/fix/live-subscriber-gap-honesty
fix(events): tell a live subscriber when its stream has a hole (BUG-2730) |
||
|
xarmian
|
d075b39a73 |
test(events): the Redis atomicity test now exercises its interleaving (BUG-2730, codex round 20)
The test added one round earlier published only before the call and after it returned, which proves nothing about the guarantee it is named for: a version that released the lock between registering the subscriber and reading the buffer would pass it. events.RedisBus gets the same afterSubscribeRegister seam MemoryBus and watchevents.RedisBus already carry, and the test attempts a fan-out from inside the critical section — the one moment the guarantee is observable. Worth recording how the mutation went, because the first one was not a fair test either. Unlocking and immediately re-locking around the buffer read SURVIVED: Go hands the mutex straight back to the barging goroutine, so the window never opened and the "mutant" was not the wrong implementation, just a slower right one. The mutation that discriminates is the genuine two-critical-section structure — register, release, read under a fresh acquire — which is what the old handler did and what the guarantee forbids. That one fails. |
||
|
xarmian
|
0eb274fed8 |
test: close the mutation gaps a coverage audit named (BUG-2730, codex round 18)
Round 18 walked every behavioural change in the diff, named the smallest edit that would break it, and listed the ones no test caught. Twelve. All but two are now covered, and each new test was verified against the mutation it exists for: - the activity bus's gap channel coalescing (the watch twin had it, this one did not) - Redis-backed atomic subscribe-and-replay, which reaches the guarantee by a different mechanism than MemoryBus and would break alone - a resuming client being held to the per-workspace limit, so the new API is not a second door past a bound the fresh path enforces - the resume-gap report on the new path, with a fresh-subscription control so it cannot fire on every subscribe and still pass - the Redis drop metric, asserted per DROPPED SUBSCRIBER with two slow subscribers, so a report hoisted out of the fan-out loop halves the count and fails - the gap channel surviving Unsubscribe, since closing it would make a consumer's select spin - every subscribe API returning a non-nil signal - the watch handler incrementing the WATCH counter (countMidStreamResync takes a bool to choose, which is the kind of argument that gets passed the wrong way round), with both wrong-counter legs asserted - the production cooldown, which every handler test overrides, so nothing else would notice it set to zero - the wrapper's gauge on the atomic-resume path, which the previous assertion checked only for non-nil-ness Two left uncovered deliberately: an interleaving test at the handler level for the atomic API (the bus-level tests carry that guarantee and the handler cannot arrange the interleaving), and the same for the handler choosing the atomic call over subscribe-plus-EventsSince. Two existing tests were also repaired rather than kept green by luck: the ordering stress test demanded every published event and a slow reader legitimately loses some, and the new Redis atomicity test resumed against a workspace the bus was not covering. |
||
|
xarmian
|
d6832d6604 |
docs(bus): state the gap seam's invariant for the third cause (BUG-2730, codex round 17)
Asked what a maintainer adding a third reason a subscriber can be signalled would get wrong: nothing stopped them routing a cause with DIFFERENT remediation through the same payload-free channel, passing every test, and silently losing the distinction. Stated as a precondition on both signalGap implementations, with the enforcement named — the channel carries no payload, so a distinction it cannot represent cannot be lost downstream, because it cannot be put in. Coalescing makes the same point from the other side: two causes between two reads become one signal, so per-cause handling was always undecidable here. A condition needing different handling belongs on its own seam. A typed reason was the alternative and is not worth it for two causes with identical remediation — it would buy a field nobody reads and reintroduce a which-reason-wins question that coalescing has no good answer to. |
||
|
xarmian
|
d6480c1f02 |
revert(sse): remove the ordering barrier; its failure mode is worse than the problem (BUG-2730, codex round 16)
Round 16 found the third defect in a row inside the previous round's fix: the gap branch reset gapDrainBudget to the CURRENT queue depth on every signal, so a producer refilling faster than a slow client drains could re-raise the coalesced gap before the budget reached zero and the announcement would never fire — the exact starvation the budget was introduced to prevent, one level up. Rounds 13, 15 and 16 each found a defect in the fix from the round before. That pattern is the signal to stop patching and reassess, so I reassessed the barrier itself rather than fixing it a third time. What it prevented: a client receiving sync_required and then events queued before the hole, whose IDs re-establish a cursor below it. Bounded and self-correcting — the client was told to reconcile, and a later reconnect from such a cursor is refused by the coverage check and told again. What it risked: never announcing at all, on the connection type this whole unit exists for. Unbounded silence. A mechanism whose own failure class is worse than the one it fixes should not ship, so the barrier, its drain budget and its predicate are gone. The announcer and its cooldown stay: they answer a real feedback loop and they latch rather than drop, and their binding to both handlers is tested. The residual ordering behaviour is now documented in docs/deployment.md under what a client should do with sync_required, and in a comment at the gap branch — stated rather than left for a reader to find, which is the same posture as the rest of this unit. |
||
|
xarmian
|
7c03beb24e |
fix(sse): bound the ordering barrier by a count, not by the channel emptying (BUG-2730, codex round 15)
The barrier shipped one commit ago with a comment asserting it could not starve. That was wrong, and wrong in the way that matters: it waited for len(ch) == 0, which never happens while a publisher refills faster than a slow client drains — and the subscriber this whole signal exists for is precisely a slow one on a busy workspace. The announcement it was supposed to make could be deferred indefinitely. The wait is now bounded by the queue depth captured when the gap was latched, decremented once per event taken off the channel. Once that many have gone out, every event that predated the hole has been delivered and anything still queued arrived after it, so the ordering guarantee is satisfied and the announcement goes. Terminating by construction, and exact rather than a timeout. The decrement counts filtered events too — an invisible event occupied a queue slot like any other. An honest note on the instrument, because the first one was no good. I wrote an end-to-end test with a goroutine publishing continuously and it PASSED against the unbounded version: under most schedulings the channel does briefly empty, so the scenario is not reliably reproducible through the handler. The bound is therefore a named predicate, gapReadyToAnnounce, with the starvation case asserted directly — latched, budget spent, channel refilled — where it cannot be scheduled away. The end-to-end test stays for the ordering claim, which it does discriminate. |
||
|
xarmian
|
3a00783557 |
fix(sse): queued events go out before the gap announcement (BUG-2730, codex round 13)
Reading both handlers as state machines: the event channel and the gap channel are two arms of one select, so with both ready Go picks at random. Announcing first and then draining events the subscriber queued BEFORE the hole is the wrong order twice over — the client is told its position is untrustworthy and then immediately handed IDs that re-establish one, below the hole; and on an ID-space change those queued events belong to the space that was just abandoned. The gap is now latched and the announcement waits for an empty channel. Nothing is discarded to achieve it, and that restraint is the load-bearing part on the watch stream: a queued one-shot PUSH cannot be recovered by any reconcile, so dropping it to make the cursor tidy would destroy the only copy. Draining cannot starve the announcement either — one event per iteration, re-checked at the top, so it lands on the first iteration with nothing queued, immediately when the channel was already empty. Pinned by asserting the ORDER of the frames with twenty events queued ahead of the gap. Without the barrier that fails on the first or third event, roughly half the time per run. |
||
|
xarmian
|
fa3710d9da |
docs: scope the metric correlations to the causes that produce them (BUG-2730, codex round 12)
A cross-artifact pass over every claim in the comments, help strings and deployment doc found two, both mine and both the same shape — a correlation stated as general when it holds for one cause: The watch drop metric and the doc row above it pointed operators at pad_event_midstream_resyncs_total, while watch announcements increment pad_watchevents_midstream_resyncs_total. Following either reference led to the wrong series. "drops >= announcements" and "the reset ratio is the fan-out" are each true of one cause and not of the others. A watch sequence gap announces to every subscriber without moving the drop counter; and the no-buffer coverage loss, which the previous round added deliberately, announces while moving NO cause counter at all — there was no coverage to end, but the subscribers still have a hole. That last one is the interesting case to leave written down, because an operator seeing announcements with every cause counter flat would otherwise reasonably conclude the metric was broken. Both counters' descriptions now say ANNOUNCEMENTS rather than clients told, and enumerate which causes correlate how. |
||
|
xarmian
|
b3c5ba95f5 |
style: gofmt the struct-field alignment the new Server field broke
Caught by make lint, after I chained the commit onto the same line as the gate and shipped it on a failing exit code. Same shape as the rule about never piping a gate: read the exit status before the commit runs, not alongside it. |
||
|
xarmian
|
a82bbd6b4f |
test(server): the rate limit has to be tested where it is BOUND (BUG-2730, codex round 11)
A one-survivor pass on the added tests: the announcer was tested directly and each handler test injected a single gap, so a handler that bypassed the limiter entirely and emitted sync_required straight from `case <-gaps:` passed everything — reopening the exact feedback loop the limiter exists to prevent. The same CONVE-19 shape as the wrapper: the component was vouched for, the binding was not. Both handlers now drive a burst through one connection and assert both halves of the bound: exactly ONE announcement inside the window, and one MORE after it. The second leg matters as much as the first — a handler that discarded the extras rather than latching them would satisfy the first and be this fix's own defect one layer up. The cooldown becomes a Server field so the test can narrow it. An integration test that waited out five real seconds per assertion would not have been written, which is how the gap got here. |
||
|
xarmian
|
d54f5236e8 |
docs: say what a client should DO with sync_required (BUG-2730, codex round 10)
Read as a third-party client author with only the wire contract, the frame was ambiguous: an empty id: retires the cursor but does not close the connection or request a reconnect, and the doc described recovery only for the web activity client. Now stated for both endpoints, including the part that is a limitation rather than an instruction: on the watch stream, watch-matched notifications can be re-derived by re-reading the items, but one-shot PUSHES cannot. They are not stored as recoverable state and there is no backfill endpoint, so a push missed during a hole is missed permanently. That endpoint is best-effort for pushes by design, and sync_required on it means the position is untrustworthy, not that a refetch makes the client whole. Also stated: keep the connection open. A client that redials on every sync_required turns one delta into a reconnect storm. |
||
|
xarmian
|
1e839331db |
fix(events): concurrent publishes must deliver in ID order (BUG-2730, codex round 9)
The composition angle found the duplicate this unit did not close. SubscribeAndReplaySince shuts the window between subscribing and reading the replay, but MemoryBus.Publish still assigned the ID under replayMu and fanned out after releasing it — so two concurrent publishes could take N and N+1 and deliver in the other order. A subscriber that sees N+1 then N has a cursor that REGRESSED, and its next reconnect replays N+1 a second time. Same symptom as the window this unit is about, reached by a different route, and closing one while leaving the other open would have been a half-answer. Pre-existing rather than introduced here, and MemoryBus was the outlier: events.RedisBus holds its mutex across append and fan-out, and watchevents.MemoryBus holds its single mutex across both. This one now holds replayMu through the fan-out, which costs an O(subscribers) walk inside a lock a resume read also wants — the trade RedisBus has always made. Pinned by a stress test rather than a seam, because the fix is precisely that there is no longer a point between the two halves to pause at. It fails against the unserialized version on the first of five runs. |
||
|
xarmian
|
8799e7d0cb |
docs: correct the comments this change made wrong (BUG-2730, codex round 7)
A next-maintainer read of every comment against the code it describes found nine, most of them made stale by this branch: - the watch observer and its fan-out still said a subscriber holding a stream open is told nothing about a sequence gap, which is the exact sentence this unit exists to falsify - the events interface described the gap signal as only a full-channel drop, omitting the coverage-loss scope that reaches the same channel - both SubscribeAndReplaySince doc comments still described a two-value return and an eviction-only nil - the InstrumentedBus header said it wraps without changing the interface or its implementations, in a diff that changes both - the SSE handler said a restarted Redis counter is undetectable, which BUG-2736 fixed; what stays silent is narrower And three correctness points about the new metrics, all conceded: - drops and mid-stream announcements are NOT one-to-one. Coalescing and the 5s latch turn a burst on one connection into a single announcement, so the counter measures announcements, not clients, and a large ratio means one client far behind rather than many affected. - the announcement counter increments before the write. Stated rather than changed: counting after would lose every announcement to a client that vanished mid-write, which is the population most worth seeing. - the doc said a connection is told at most once per five seconds. Only the MID-STREAM announcement is bounded; the resume signal is not, and never needed to be. A pass stripping review-history attribution from comments was reverted rather than shipped: it churned 50 files, and the surrounding code uses that attribution style throughout, so removing it here would have made this diff the inconsistent one. |
||
|
xarmian
|
6ce542782d |
docs: say what each stream actually detects, not what the pair does (BUG-2730, codex round 6)
An end-to-end trace of a pub/sub flap found the deployment doc claiming, for BOTH streams, that a reconnect or an undecodable message produces a mid-stream sync_required. True of the activity bus, which subscribes with ChannelWithSubscriptions and ends the workspace's coverage on either. False of the watch bus, which uses a plain Channel() and discards an undecodable payload with a log line — it learns of a hole only when a later notification arrives non-contiguous, so a flap that loses the newest notification with nothing published after it leaves a connected CLI silently stale. That gap is real and pre-existing (BUG-2731 was an activity-bus unit); filed as BUG-2739 rather than folded in, because widening DETECTION is a different claim from announcing what is already detected, and the watch bus's single replay buffer makes "end coverage" a decision rather than a copy. The doc now states the asymmetry and names the item. Also from the same round, both mine: a comment in the activity fan-out still said the drop was silent and that no bus had a channel to a live consumer, three lines above the code that signals one; and two metric descriptions still pointed operators at pad_*_resume_gaps_total for mid-stream signals, which the previous commit deliberately moved to pad_*_midstream_resyncs_total. |
||
|
xarmian
|
b7ae022b6f |
refactor(events): every way of subscribing hands back the gap signal (BUG-2730, codex round 5)
Subscribe allocated and raised a gap channel its callers could not read, which round 5 called dead work. The read is right and the disposition is the other one: an interface method whose subscribers CANNOT be told they missed something is a silent under-delivery waiting for its first production caller, and internal/watchevents' Subscribe already returns the signal, so the asymmetry was the defect rather than the allocation. Subscribe now returns it too, on all three implementations. No production caller changes — the handlers use SubscribeIfAllowed and SubscribeAndReplaySince — so this is a test-call-site sweep plus one signature. |
||
|
xarmian
|
d936464736 |
fix(events): bound the mid-stream signal, and stop it moving existing alerts (BUG-2730, codex round 4)
Three findings from the operator-at-3am angle, all real. A pub/sub outage on a workspace with a subscriber but NO replay buffer yet was silent. dropWorkspaceCoverage returned early before telling anyone, on the reasoning that there was no coverage to end — true of the BUFFER, and beside the point for the SUBSCRIBER, which has the largest possible hole and the least evidence of it. Live subscribers are now signalled on that path while the reset metric stays suppressed: the metric measures coverage endings, the signal measures clients who may have missed something, and those are different questions. The gap channel coalesces, which bounds the queue but not the loop: once the handler consumes a signal the next drop re-arms it, so a slow client could be answered with a delta sync, made slower, and answered again. Both handlers now share a gapAnnouncer that allows one announcement per connection per 5 seconds — a delta-sync round trip, not a tuning knob — and LATCHES rather than drops, so a gap inside the window is announced when the window closes. Suppressing it would be this fix's own defect one layer up. Folding mid-stream signals into pad_*_resume_gaps_total silently changed what every existing alert on those counters measures, and a mixed-version fleet would have reported two populations under one name for the length of a rollout. They go back to counting resumes; the new population gets pad_event_midstream_resyncs_total and pad_watchevents_midstream_resyncs_total, which count CLIENTS TOLD rather than causes — one instance-wide coverage loss moves them once per subscriber while the reset counter moves once, and that ratio is the fan-out an operator wants when judging a storm. |
||
|
xarmian
|
b9dff0072e |
test: close the coverage gaps codex round 3 named (BUG-2730)
Round 3 reviewed the added tests as production code and found four things, all real: - the atomicity test never asserted the replay CARRIED anything, so an implementation that always answered "cannot vouch" and delivered everything live would have satisfied "never both" by never replaying. It now seeds two events and resumes from the first. - both handler tests would have passed for a handler that emitted sync_required and then closed the stream. The activity one now proves an ordinary event still arrives afterwards; the watch one, which has no cheap ordinary event to publish, proves the handler did not return. - the refused-connection test's end state also holds on origin/main, so it argues for nothing about the new ordering. It is a regression test for the defect that ordering introduced, and the comment now says so. - whole paths had no test: the activity RedisBus's drop, coverage-drop and ID-space-reset signalling; the watch bus's epoch-change and counter-backward arms; the InstrumentedBus delegation; the metrics adapter; and the mid-stream counter increment. All covered now. The wrapper tests matter most of the three new files: it is the only implementation that does not originate a gap channel, and in production the bus IS wrapped, so a wrapper returning nil there would have disabled the whole fix while every bus-level test stayed green (CONVE-19). |
||
|
xarmian
|
b2277641cf |
fix(events): a refused connection is not a sync_required (BUG-2730, codex round 1)
Moving the Last-Event-ID parse above the subscribe — which is what makes the atomic subscribe-and-replay possible — put the unreadable-cursor increment on the wrong side of the per-workspace admission check. A connection refused with 429 is sent nothing, and was still counted in pad_event_resume_gaps_total, whose population is signals SENT. Counted at the emission site instead, which is where it effectively was before the parse moved. Pinned with both legs: refused must not count, admitted-with-an-unreadable-cursor must. |
||
|
xarmian
|
db8c5b76ed |
docs(deployment): sync_required is not only a resume answer (BUG-2730)
The signal's documented meaning was resume-shaped in every place it appeared, while the fix widens it to a live subscriber told mid-stream that it has a hole. A widened signal whose docs still state the narrow meaning is a half-shipped contract. Adds a subsection stating both situations and what a client does with each, and corrects the two resume-gap counters' descriptions: they count SIGNALS, not resumes, so a deploy with no reconnects at all can now move them. Documents the new pad_event_events_dropped_total, including that a deploy which starts reporting it may simply be the first that could. |
||
|
xarmian
|
1cee8e615c |
test(server): the quiet-control leg could not see a gap announced at connect (BUG-2730)
A mutation that made the activity handler announce a gap unconditionally at connect SURVIVED both tests. The control asserted the absence only AFTER publishing an ordinary event, and waitForFrameWithEvent reads past every frame that is not the one it wants — so the spurious sync_required went by unexamined and the window that followed was genuinely empty. The absence is now asserted first, before anything else is published, and the wait for the ordinary event refuses a sync_required instead of reading past it. Both legs fail on the mutation now. |
||
|
xarmian
|
af99762815 | test(server): the gap signal must reach the wire, not just the bus (BUG-2730, CONVE-19) | ||
|
xarmian
|
fa2ddf3f22 | test(events): pin the drop signal, its metric, and the subscribe-replay atomicity (BUG-2730) | ||
|
xarmian
|
9f23364b43 | test(watchevents): pin both gap-signal scopes with their counterfactuals (BUG-2730) | ||
|
xarmian
|
4269bdd4cc |
fix(watchevents): the same hole, told to the stream holding it open (BUG-2730)
fanOutLocally already DETECTED a gap in the received notification sequence, logged the exact id range, and raised knownFrom so a client RECONNECTING across it was honestly told sync_required. A client holding the stream OPEN across the same gap was told nothing: it stayed connected, kept receiving everything after the hole, and never saw what went missing. The instance knew; the one consumer whose correctness depended on it did not. Subscribers now carry the same capacity-1 coalescing gap channel the activity bus grew, raised from two different scopes: - per-INSTANCE, to every live subscriber, when this instance discovers a hole in what it received — a sequence gap, a counter that went backwards, an epoch change. Instance-scoped is the whole scope: the ids never arrived HERE, other instances may have them, and every subscriber registered at the moment of detection is exactly the set that was connected across the hole. - per-SUBSCRIBER, to one connection, when its channel was full. The watch SSE handler answers with sync_required mid-stream. The pad CLI monitor already clears its cursor on that event, so the client half needed no change. Refs BUG-2730. |
||
|
xarmian
|
b5615cc1b5 |
fix(events): a live subscriber is told when it has a hole (BUG-2730)
The activity bus dropped an event for a subscriber whose 64-deep channel was full, logged it, and continued. The subscriber was told nothing, so a later delivered event advanced its Last-Event-ID past the dropped ids — after which no replica would ever replay them, because every replica agrees that cursor is current. Only a full reconciliation corrected it. Every subscriber now carries a capacity-1 coalescing gap channel. It is raised when the bus cannot hand that connection an event, and when this instance's coverage of a workspace ends under it (a pub/sub flap, an undecodable message, an ID-space reset) — that second case is per- instance and reaches every subscriber of the affected workspace, because the gap is theirs collectively. The SSE handler selects on it and emits sync_required mid-stream, the same signal a resume we cannot serve gets: the web client already answers it with an incremental /changes delta, so the distinction a new event name would express is one the client would act on identically. Also folded in, because they are the same defect surface: - Subscribe-and-replay is now ONE critical section on both bus implementations (SubscribeAndReplaySince), closing the duplicate window where an event published between the two steps landed in both the replay set and the live channel. MemoryBus takes b.mu across the buffer append and the fan-out to get it; the lock order is b.mu then b.replayMu, everywhere. - internal/events gains the drop report its sibling has had since BUG-2699 (Observer.EventDropped, pad_event_events_dropped_total). The drops were log-only, so the condition this fix makes honest was not countable before it. Both resume-gap counters' help text now names what they actually count — sync_required signals, including the mid-stream ones — rather than resumes alone. Refs BUG-2730. |
||
|
xarmian
|
2a121c32d5 |
Merge pull request #1178 from PerpetualSoftware/fix/events-idspace-migration
fix(events): give every event ID space an identity, behind a two-phase flip (BUG-2736) |