Canary state was left behind by every router delete, growing without bound under
router churn, because nothing enumerated the stores that hold per-router state.
The gossip type registry looked like that enumeration but is not: it drives the
epoch sweep, and canaries deliberately sit out that sweep, since a canary carries
the epoch that detects a change. Opting out of the sweep silently opted them out
of delete cleanup as well.
- splits the two lifecycles. The gossip type registry keeps driving the epoch
sweep; a separate registry takes every store keyed by router id, which is a
different membership for a different reason
- registers link gossip, link metrics, canary gossip, the canary listener's own
map and the link index, so one call site tears down all of them
- tests that every per-router store is registered. Without that the registry is a
convention, and a convention is what was already being broken
- replaces the canary listener's sync.Map with a map under a mutex, so a read and
the delete that follows it cannot interleave
The cleanup is not ordered against a create of the same router id, and does not
try to be. Router ids are assigned at enrollment; the one path that lets an id be
chosen is deprecated.
Paul Lorenz
3b59bfb95e