mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-26 13:39:47 +00:00
rcourtman
487ab8a7c4
## Why A heap profile of the long-running dogfood install (77.6 MB live after a forced GC) shows the alert history projection holding 36 MB (47%). It keeps one merged alert per occurrence for the event log's 90-day retention: 1,895 occurrences there, with resolved snapshots averaging about 20 KB. Reads made it much worse (#1803): - every history read deep-copied the whole projection before applying its limit, so the attention summary's 500-row read copied all 1,895 occurrences; - every read with a `since` (the 30-day alert-quality window, date-filtered history) built a fresh projection and re-decoded every snapshot in its window. That churn is why the same install holds about 158 MB of Go heap for 78 MB of live data. ## What changed - One full-history fold, built and extended as before. Each occurrence also keeps a small mark per folded event (time, durable ID, and the `LastSeen` it contributed) and the index of its latest acknowledged snapshot. - A `since` read is answered from that fold. The window starts at each occurrence's first event that the log's own `occurred_at >= since` text comparison admits. `LastSeen` and acknowledgement come from that point on, and occurrences are ordered by their first in-window event, exactly as a walk of only the window would produce. - Overlay, sort and limit run on lightweight rows under the projection lock. Only returned rows are copied, once, by `canonicalizeAlertHistoryForOutput`. - The alerts contract notes the windowed-read rule and the pinning tests. ## Measured (pulse-dev, log shaped like the dogfood install's: about 1,900 occurrences, 40k events) | Read | main | this PR | |---|---|---| | 30-day window | 68.9 ms, 19.8 MB allocated | 5.2 ms, 2.8 MB | | Latest 500 | 12.7 ms, 8.5 MB | 3.6 ms, 2.0 MB | | Full history | 20.8 ms, 12.9 MB | 13.4 ms, 6.9 MB | The 36 MB the fold itself holds is unchanged. The new marks add about 56 bytes per lifecycle event, about 2 MB at 40k events. ## Checks - `TestWindowedHistoryMatchesAWalkOfOnlyTheWindow` compares every window and limit against an independent walk of only the window (the original algorithm). It covers 25 random histories with acknowledgements on both sides of the boundary, out-of-order `LastSeen`, uneven fractional seconds where RFC3339Nano text order and time order disagree, same-instant events, a history clear, and live overlays. - `TestWindowedHistoryReusesTheFullHistoryFold`, `TestLimitedHistoryReadCopiesOnlyReturnedRows` and `TestHistoryReadResultsDoNotAliasTheSharedFold` pin reuse, copying and isolation. - Negative controls: ignoring the window, building a fresh fold per `since` read, and copying every row before the limit each fail their test. - Removing the per-row clone did not fail the aliasing test, because the output copy already isolates results. That double copy was dropped, and the test guards isolation end to end. - Vet and the whole `internal/alerts/...` tree pass on pulse-dev. ## Not established How much held heap this saves on the dogfood install itself. That needs this build running there. Related to #1803