test(targets,notify,audit): share a builder MockTarget testkit (#6717)

* test(targets): ship a builder MockTarget testkit and retire the in-crate Target mocks

Adds crates/targets/src/testkit.rs with a builder-style MockTarget implementing Target<E> for every E: PluginEvent, with orthogonal off-by-default knobs: disabled/active override, health delay plus health-started signal plus a drop-guard counter proving a cancelled probe future was dropped, an init failure budget (usize::MAX = always fail) plus blocking init plus an init counter, a close counter/signal/semaphore gate with a runtime block toggle, a save counter plus save failure budget, caller-supplied store and failed-store handles, and a shared final-failure counter. Clones and clone_dyn share all counters, so an observer clone keeps watching a target after it is boxed into a runtime.

The module is gated as #[cfg(any(test, feature = "test-support"))]: in-crate unit tests get it via cfg(test), and downstream test suites opt in through the new off-by-default test-support cargo feature (test-support = [], activating no dependencies).

Migrates the five in-crate duplicate mocks onto it: the plugin.rs registry-factory TestTarget, the runtime/adapter.rs lifecycle TestTarget (init/close/store knobs), the runtime/mod.rs TestTarget plus HealthDropGuard (close gating and health-probe tests), the target/mod.rs MoveTestTarget (folded into the test_support helper constructors used by the NATS JetStream failed-store tests), and the target/mod.rs StoreBackedTarget (the default send_from_store purge test; the mock deliberately does not override send_from_store or handle_terminal_failure). The forced init failure now uses TargetError::Initialization instead of the old adapter mock's Configuration; the adapter derives its redacted failure summary from the target id alone, so the migrated assertions are unchanged.

Leak guard: testkit unit tests assert the crate manifest still declares default = [] and that test-support = [] stays a pure cfg gate, complementing the compile-level cfg gate that keeps the mock out of production builds.

Part of rustfs/backlog#1846 (cluster 3, step 1).

* test(notify,audit): migrate the Target mocks onto the shared testkit

Retires the hand-written Target mocks in crates/notify and crates/audit in favor of rustfs_targets::testkit::MockTarget: notify's notifier.rs TestTarget/DeferredTestTarget/ClosableTestTarget, lifecycle.rs BlockingInitTarget/RetryInitTarget (rebuilt as observed MockTarget templates cloned by their plugin-descriptor factories, sharing the init signal, close counter, and single-failure init budget across generations), runtime_view.rs TestTarget, runtime_facade.rs TestTarget, and audit's pipeline.rs MockTarget, system.rs TestTarget, registry.rs CloseTestTarget, plus TestTarget and FailingTarget in audit/tests/pipeline_layer_test.rs. Removing the two integration-test mocks also removes their respelled PluginEvent bound (E: Send + Sync + 'static + Clone + Serialize + DeserializeOwned), which the plugin-contract rules require to be spelled only via PluginEvent.

lifecycle.rs ReplayTarget stays bespoke on purpose: its generation tags, mpsc observation channels, gated send_raw delivery, and ObservedQueueStore model the replay pipeline itself and would contort a general-purpose mock. The other bespoke mocks named out of scope in PR-3a (ProgrammedTarget, ClassifyingTarget, the ReloadableTargetTls fakes) are likewise untouched.

New testkit knobs, each defaulted off and unit-tested: with_id (rename a clone while keeping the shared counters, for factory templates), with_first_save_gate (the first save notifies entered and waits on release; several mocks may share one pair), with_health_gate (is_active waits on a release handle after notifying health_started), with_delivery_snapshot (fixed snapshot overriding the store-derived default), with_close_failures (close-failure budget, default TargetError::Storage) with with_close_failure_error to shape the variant (audit's registry test pins TargetError::Unknown), and an always-on is_enabled call counter exposed as enabled_call_count (notifier's generation tests count dispatcher selections through it).

Both crates enable the testkit through a dev-dependency on rustfs-targets with the test-support feature; the feature stays out of default and activates no dependencies, so production builds are unchanged.

Part of rustfs/backlog#1846 (cluster 3, step 2).

crates/audit/src/pipeline.rs

@@ -564,88 +564,21 @@ impl AuditRuntimeFacade {
 mod tests {
     use super::AuditPipeline;
     use crate::{AuditEntry, AuditError, AuditRegistry};
-    use async_trait::async_trait;
-    use rustfs_targets::arn::TargetID;
-    use rustfs_targets::store::{Key, Store};
-    use rustfs_targets::target::{EntityTarget, QueuedPayload, QueuedPayloadMeta};
-    use rustfs_targets::{StoreError, Target, TargetError};
+    use rustfs_targets::testkit::MockTarget;
     use std::sync::Arc;
     use tokio::sync::{Mutex, Notify};
 
-    /// Mock target whose `save()` outcome is fixed at construction so tests can
-    /// force full-success / full-failure / partial-failure fan-outs.
-    #[derive(Clone)]
-    struct MockTarget {
-        id: TargetID,
-        fail: bool,
-        health_gate: Option<(Arc<Notify>, Arc<Notify>)>,
-    }
-
-    impl MockTarget {
-        fn new(id: &str, fail: bool) -> Self {
-            Self {
-                id: TargetID::new(id.to_string(), "webhook".to_string()),
-                fail,
-                health_gate: None,
-            }
-        }
-
-        fn with_health_gate(mut self, started: Arc<Notify>, release: Arc<Notify>) -> Self {
-            self.health_gate = Some((started, release));
-            self
-        }
-    }
-
-    #[async_trait]
-    impl<E> Target<E> for MockTarget
-    where
-        E: rustfs_targets::PluginEvent,
-    {
-        fn id(&self) -> TargetID {
-            self.id.clone()
-        }
-
-        async fn is_active(&self) -> Result<bool, TargetError> {
-            if let Some((started, release)) = &self.health_gate {
-                started.notify_one();
-                release.notified().await;
-            }
-            Ok(true)
-        }
-
-        async fn save(&self, _event: Arc<EntityTarget<E>>) -> Result<(), TargetError> {
-            if self.fail {
-                Err(TargetError::Configuration("forced save failure".to_string()))
-            } else {
-                Ok(())
-            }
-        }
-
-        async fn send_raw_from_store(&self, _key: Key, _body: Vec<u8>, _meta: QueuedPayloadMeta) -> Result<(), TargetError> {
-            Ok(())
-        }
-
-        async fn close(&self) -> Result<(), TargetError> {
-            Ok(())
-        }
-
-        fn store(&self) -> Option<&(dyn Store<QueuedPayload, Error = StoreError, Key = Key> + Send + Sync)> {
-            None
-        }
-
-        fn clone_dyn(&self) -> Box<dyn Target<E> + Send + Sync> {
-            Box::new(self.clone())
-        }
-
-        fn is_enabled(&self) -> bool {
-            true
-        }
+    /// Builds a mock target whose `save()` outcome is fixed at construction so tests can force
+    /// full-success / full-failure / partial-failure fan-outs.
+    fn mock_target(id: &str, fail: bool) -> MockTarget {
+        let target = MockTarget::new(id, "webhook");
+        if fail { target.with_save_failures(usize::MAX) } else { target }
     }
 
     fn pipeline_with(targets: Vec<MockTarget>) -> AuditPipeline {
         let mut registry = AuditRegistry::new();
         for target in targets {
-            registry.add_target(target.id.to_string(), Box::new(target));
+            registry.add_target(target.target_id().to_string(), Box::new(target));
         }
         AuditPipeline::new(Arc::new(Mutex::new(registry)))
     }
@@ -658,7 +591,7 @@ mod tests {
     // dispatch must return Err rather than swallowing the failures as Ok.
     #[tokio::test]
     async fn dispatch_returns_err_when_all_targets_fail() {
-        let pipeline = pipeline_with(vec![MockTarget::new("a:webhook", true), MockTarget::new("b:webhook", true)]);
+        let pipeline = pipeline_with(vec![mock_target("a:webhook", true), mock_target("b:webhook", true)]);
         let result = pipeline.dispatch(entry()).await;
         assert!(matches!(result, Err(AuditError::Target(_))), "expected Err, got {result:?}");
     }
@@ -667,13 +600,13 @@ mod tests {
     // so dispatch reports success (degradation is logged, not propagated).
     #[tokio::test]
     async fn dispatch_returns_ok_on_partial_failure() {
-        let pipeline = pipeline_with(vec![MockTarget::new("ok:webhook", false), MockTarget::new("bad:webhook", true)]);
+        let pipeline = pipeline_with(vec![mock_target("ok:webhook", false), mock_target("bad:webhook", true)]);
         pipeline.dispatch(entry()).await.expect("partial success should return Ok");
     }
 
     #[tokio::test]
     async fn dispatch_returns_ok_when_all_targets_succeed() {
-        let pipeline = pipeline_with(vec![MockTarget::new("a:webhook", false), MockTarget::new("b:webhook", false)]);
+        let pipeline = pipeline_with(vec![mock_target("a:webhook", false), mock_target("b:webhook", false)]);
         pipeline.dispatch(entry()).await.expect("all-success should return Ok");
     }
 
@@ -686,9 +619,10 @@ mod tests {
 
     #[tokio::test]
     async fn health_probe_does_not_hold_the_registry_lock() {
-        let started = Arc::new(Notify::new());
         let release = Arc::new(Notify::new());
-        let pipeline = pipeline_with(vec![MockTarget::new("blocked", false).with_health_gate(started.clone(), release.clone())]);
+        let target = mock_target("blocked", false).with_health_gate(release.clone());
+        let started = target.health_started();
+        let pipeline = pipeline_with(vec![target]);
         let registry = Arc::clone(&pipeline.registry);
         let snapshot_task = tokio::spawn(async move { pipeline.snapshot_target_health().await });
         started.notified().await;
@@ -706,14 +640,14 @@ mod tests {
     // whole-batch loss instead of returning Ok.
     #[tokio::test]
     async fn dispatch_batch_returns_err_when_all_targets_fail() {
-        let pipeline = pipeline_with(vec![MockTarget::new("a:webhook", true)]);
+        let pipeline = pipeline_with(vec![mock_target("a:webhook", true)]);
         let result = pipeline.dispatch_batch(vec![entry(), entry()]).await;
         assert!(matches!(result, Err(AuditError::Target(_))), "expected Err, got {result:?}");
     }
 
     #[tokio::test]
     async fn dispatch_batch_returns_ok_when_all_targets_succeed() {
-        let pipeline = pipeline_with(vec![MockTarget::new("a:webhook", false), MockTarget::new("b:webhook", false)]);
+        let pipeline = pipeline_with(vec![mock_target("a:webhook", false), mock_target("b:webhook", false)]);
         pipeline
             .dispatch_batch(vec![entry(), entry()])
             .await