Adds branch-coverage tests for eight packages whose target functions were
measured at 0% before this change. Every named target was verified to move
by running each package's coverage with and without the new file.
- internal/ai/eval: all 36 Scenario constructors and the four PatrolScenario
constructors 0% -> 100%. These are catalog-invariant tests, not literal
echoes: unique names, populated required fields, runnable assertions, tool
references checked against the agentcapabilities registry, and exact
assertion-count deltas across the env-gated conditional appends. A parity
test scans scenarios.go itself, so adding a constructor without registering
it in the table now fails rather than silently going untested.
- cmd/pulse-control-plane: nine MSP and tenant-runtime print helpers
0% -> 100%, covering the nil, empty-slice and optional-field arms.
- internal/ai/memory: RemediationLog GetByID, MarkRolledBack and
GetRollbackable 0% -> 100%, pinning the overwrite-vs-preserve contract on
RollbackInfo and each falsy arm of the rollbackable predicate.
- internal/alerts/config: AlertConfig.UnmarshalJSON 0% -> 90% and
NormalizeAlertConfigAliases 52.9% -> 94.1%.
- internal/config: RunMigrationIfNeeded 0% -> 100%, copyFile 0% -> 88.9%.
- internal/mock: AvailabilityFixtures, FixtureGraph.SupplementalChanges and
generateMockHostRate 0% -> 100%.
- internal/api: testProxmoxPlatformConnection 0% -> 100% through its injected
connect func, so no network is involved.
- internal/servicediscovery: needsDeepScan 0% -> 100% across every return arm
including the confidence boundary.
No source file is modified. Adversarial review found no rejects; two findings
were acted on before committing, replacing a circular catalog-count assertion
with the real source-parity scan and reducing an AllPatrolScenarios test that
compared the function against the same constructors it calls to the ordering
and completeness signal that is actually independent.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no source or contract change
Contract-Neutral: test-only branch coverage, no source or contract change
Five new branch-coverage tests taking previously unreached functions from zero
to covered, with no source or existing test touched.
cmd/pulse-control-plane: the MSP status failure lookup across exact, absent,
case-differing and substring inputs, and the status state ordering proved
deterministic across repeated runs over the same map so a random map iteration
order cannot pass by luck.
internal/ai/eval: the signal coverage assertion driven through both the lazy
quality evaluation path and the supplied-quality path, at the minimum rate
boundary and either side of it, asserting the formatted message rather than
only the pass or fail outcome, plus the approval write command builder.
internal/api: the setup script server name derivation including the fallback
arm, the patrol autonomy level validator, the approval risk assessment across
every level it can return, and the two typed error messages.
internal/hostagent: the token already exists classifier over each phrase it
recognises, a wrapped error carrying the phrase, and near-miss messages that
must not match.
internal/monitoring: the limited temperature buffer including its truncation
boundary and its aliasing behaviour, and the mock series generators asserted on
determinism under a fixed seed and on their value bounds.
Reported for follow-up, not fixed here: the node escalation arm in the approval
risk assessment is unreachable for the current command set, generatePlateauSeries
panics on a negative point count and leaves its tail unclamped, and the limited
temperature buffer returns a slice that aliases its internal storage.
Contract-Neutral: test-only branch coverage, no contract surface touched
Manifest-backed MCP tools, prompts, and resources with surface affordance contracts; agent capability manifest and governance projection; API contract tests and capability route projection; operations-loop and intelligence-funnel telemetry; release-control subsystem documentation, registry, and tooling; licensing and configuration.
- Add unit tests for internal/ai/eval package
- Validate configuration, retry logic, and custom SSE parsing
- Enables coverage for eval framework without requiring live Pulse server
Refactor patrol eval runner to use a dual approach:
1. Poll GET /api/ai/patrol/status until Running=false (primary signal)
2. Best-effort SSE stream connection for tool event visibility
Changes:
- Add status polling loop with configurable timeout
- Make SSE stream optional (may not connect in time)
- Add Completed flag to PatrolRunResult
- Improve assertion error messages
- Add new scenarios and assertions
This is more reliable than relying solely on SSE stream which
may timeout waiting for headers during slow patrol initialization.
Add comprehensive patrol evaluation framework:
- patrol.go: Runner for patrol scenarios with streaming support
- patrol_assertions.go: Assertions for tool usage, findings, timing
- patrol_scenarios.go: Scenarios for basic, investigation, finding quality
- eval_test.go: Unit tests for patrol eval runner
Scenarios:
- patrol-basic: Verifies patrol completes with tools and findings
- patrol-investigation: Ensures investigation before reporting
- patrol-finding-quality: Validates finding structure and evidence
Run with: go run ./cmd/eval -scenario patrol
- Add retry logic for transient failures (phantom, stream, empty response)
- Add environment variable overrides for infrastructure naming
- Add JSON report output per scenario
- Expand assertions with new validation types
- Add more comprehensive test scenarios
- Add docs/EVAL.md with usage documentation
The eval harness now better handles flaky AI responses and provides
detailed reports for debugging.
rcourtman