fix(mesh): re-evaluate data plane every 10s and add opt-in auto-recreate (#1184)

* fix(mesh): re-evaluate data plane every 10s and add opt-in auto-recreate

MeshService.dataPlaneStatus was written exactly once at boot in
setupMeshNetwork() and never re-evaluated. After the operator removed
sencho_mesh at runtime (or it was recreated externally, or Sencho was
disconnected from it), /api/health and the dashboard banner kept
returning the stale boot-time discriminator until the next process
restart.

Adds a 10s revalidator that inspects the current Docker truth in one
network-inspect call and transitions dataPlaneStatus to reflect it.
Short-circuits in not_started / not_in_docker / subnet_invalid
(states that cannot change within this process) and in concurrent
ticks. Transitions are idempotent on stable state, so the timer can
tick indefinitely on a healthy mesh without log noise.

New 'not_found' reason value for the network-was-removed-at-runtime
case. Existing reasons (subnet_mismatch, subnet_overlap, attach_failed)
also surface from the revalidator when their underlying conditions
arise post-boot. transitionDataPlane keeps message and subnet fields
fresh across consecutive observations even when reason is unchanged,
so /api/health never reports stale numbers (e.g. two consecutive
subnet_mismatch observations against different external subnets).

Adds an opt-in mesh_auto_recreate global setting (default off). When
on, the revalidator additionally calls attemptInPlaceRecreate() after
surfacing not_found. The helper hard-prefers the boot-chosen subnet
(this.meshSubnet) and never iterates candidates, because changing the
subnet here would invalidate every existing extra_hosts override on
disk. A real conflict on the original subnet is reported as
subnet_overlap and preserved during the 60s recreate throttle window
so the operator-actionable reason is not flapped back to not_found
between attempts.

Self-attachment is checked via Name match (operator --hostname X
matches container Name /X) or full container-ID prefix for hex
HOSTNAMEs >= 12 chars (Docker default short ID). Non-hex HOSTNAMEs
cannot collide with container IDs at all so a Name miss is conclusive;
short hex HOSTNAMEs preserve the prior status as 'unknown' rather than
risking a false-positive prefix match.

Frontend surfaces:
- types/mesh.ts: 'not_found' added to MeshDataPlaneReason.
- MeshDataPlaneBanner: 'not_found' headline copy.
- Settings > System > Mesh data plane: TogglePill bound to
  mesh_auto_recreate, default off, helper text explains the tradeoff.

Backend coverage in backend/src/__tests__/mesh-data-plane-revalidate.test.ts
(25 cases): short-circuits, idempotent stable-state, recovery from
subnet_mismatch / subnet_overlap, transition to not_found / subnet_mismatch
/ attach_failed, transient-Docker anti-flap, re-entrancy guard, name
match path, ID-prefix path, short hex hostname ambiguity, non-hex
hostname certainty, transition message refresh on observation drift,
auto-recreate off (default), auto-recreate success with senchoIp
preservation, auto-recreate overlap classification with no subnet
drift, throttle window preserves classified reason, throttle release.
Lifecycle test covers timer wiring in start()/stop().

Existing mesh-setup-error-classification suite (27 cases) still green.

Resolves: F-4 in the v1.0 audit tracker.

* fix(mesh): address Codex review of PR #1184

Three findings from the independent review:

BLOCKER: attemptInPlaceRecreate() called recordSetupFailure() on
create / attach failures, which clears this.senchoIp. The next
revalidator tick's attachment check is guarded on senchoIp, so with
it null the check is skipped and the snapshot path can silently flip
the status back to ok against a network where Sencho is in fact not
attached. Also: a later successful recreate would call
ensureSelfAttached() with senchoIp null, which short-circuits, so
the network gets recreated without binding Sencho.

Replaced the recordSetupFailure() calls in attemptInPlaceRecreate
with a new recordRecreateFailure() that uses transitionDataPlane and
preserves senchoIp. Added two tests: create-fails-then-succeeds
(verifies senchoIp survives the failure and the later retry binds
Sencho correctly) and create-succeeds-attach-fails-then-next-tick
(verifies the snapshot path surfaces attach_failed on the next tick
instead of falsely reporting ok).

SHOULD-FIX 1: single-key POST /api/settings wrote String(value)
without re-validating against the per-key schema, so an allowlisted
enum-shaped key like mesh_auto_recreate could persist arbitrary
strings ('banana', 'true') that the bulk PATCH would later refuse.
Routed the single-key path through SettingsPatchSchema.safeParse so
both write paths validate identically. Added regression tests for
an invalid mesh_auto_recreate value, a valid mesh_auto_recreate
write, and an out-of-range numeric value.

SHOULD-FIX 2: the new Mesh data plane subsection lived inside a
section the registry exposes to non-admins, who would see the toggle
and only learn it was admin-only after the save 403'd. Gated the
subsection on `isAdmin` from useAuth so non-admins do not see the
control. The other system controls keep their existing visibility
pattern (read-only for non-admins).

71/71 backend tests green (revalidate + mesh-setup + settings-routes).
276/276 frontend tests green. tsc clean on backend + frontend.

docs/features/sencho-mesh.mdx

@@ -165,6 +165,8 @@ services:
 
 Sencho pins itself to `<network address> + 2` (so `10.42.0.2` for the example above; the bridge's `+1` gateway sits between). The `sencho_mesh` network is created with an IPRange covering the upper half of the subnet, so Docker's auto-allocation hands meshed workloads `<network> + 128` and up by default and never grabs Sencho's static IP while Sencho is offline. Each node is configured independently; the alias registry pushes the correct local IP to each node's override file.
 
+Sencho re-checks the mesh data plane every 10 seconds and updates `/api/health` and the dashboard banner with the current state, so removing or recreating `sencho_mesh` while Sencho is running is reflected within one tick instead of waiting for a restart. The reconciler is report-only by default: if the network disappears, Sencho surfaces `not_found` and waits for the operator to restart. To have Sencho rebuild the network at the same subnet on the next tick, flip **Settings → System → Mesh data plane → Auto-recreate mesh network** on. Auto-recreate never iterates the subnet candidate list (it would invalidate every existing `extra_hosts` override on disk); if the original subnet is no longer free, Sencho reports `subnet_overlap` and stops. Repeat failures are throttled to one attempt per minute.
+
 ## Security and trust boundaries
 
 **Authentication.** Cross-node traffic rides an authenticated WebSocket between Sencho instances. Pilot Agent nodes authenticate with the long-lived JWT issued at enrollment (stored at `/app/data/pilot.jwt` on the agent). Distributed API Proxy nodes authenticate with the [Node Token](/features/multi-node#add-a-remote-node-distributed-api-proxy) generated on the remote. Restricted API token scopes (read-only, deploy-only) cannot carry mesh traffic; only a full-admin Node Token can.