mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-23 11:03:41 +00:00
xarmian
3ba5dcc836
The conditions the coverage fix added are invisible from outside the bus,
so nothing could tell an operator they were happening. A resume the bus
refused is a nil return — a wrapper can SEE that nil but not why, or that
it happened at all, without reimplementing the coverage rules it wraps —
and a coverage reset is detected on the receive path, which no caller
invokes.
That is why this is an Observer seam and not more of
metrics.InstrumentedBus, which already wraps this bus. The split is by what
each can see, and the package doc says so, because "why is this
instrumented twice" has an obvious wrong answer.
Three counters:
- pad_event_resume_gaps_total. Expect a step around a deploy and a return
to baseline; each instance starts with no coverage. It counts RESUMES,
not clients, so a deploy nobody reconnects through does not move it. A
rate that does NOT settle is the evidence against this fix's central
claim — that the syncs it adds are only the warranted ones — which is
what makes the claim falsifiable in production rather than only in
tests.
- pad_event_sequence_resets_total{reason}. One reason today,
subscription_resumed. Labelled from the start rather than shipped bare,
because BUG-2736's ID-space work adds reasons an operator diagnoses
completely differently, and an alert built on an unlabelled total would
have to be rewritten when they arrive.
- pad_event_receive_loop_exits_total. Unlike the watch stream's twin this
does NOT stay at zero — one loop per workspace, so it fires whenever a
workspace's last local subscriber leaves. Read as a rate.
The resume counter is reported in ONE place, on the way out of EventsSince,
so both ways of failing to serve — no buffer at all, and a buffer refusing
the span — reach it structurally rather than by remembering to increment at
each return. That is the exact shape the watch stream's own gap counter got
wrong. internal/watchevents.MemoryBus also gains the reporting its RedisBus
already had, so that counter stops reading zero on single-process
deployments for a path that genuinely resyncs clients.
Wired on MemoryBus too, not only RedisBus: a single-process deployment
restarts, and the cold-buffer gap is as real there.
newObservedEventBus exists so the WIRING is testable (CONVE-19). Inline in
the command's RunE, the SetObserver call was a claim no test could reach —
the events test attaches its own observer and so does the metrics test, so
both pass with the production line deleted. Extracting it also had to be
done without hollowing out the BUG-2724 keyspace guard, which reads
cmd_server.go: it now checks both that the constructor takes the shared
value and that every call to the helper passes it, with the helper's own
declaration excluded from the call count.
Refs BUG-2731