Six structural strengthenings to certctl QA documentation surface, raising
acquisition-readiness QA-doc score 4.0 -> 4.7. M-008 (per-RFC test-vector
subsections under Parts 21 + 24) deferred as 'Bundle P.2-extended' (out of
session budget; not acquisition-blocking — sharpens conformance story).
P.1 — `make qa-stats` single-source-of-truth (M-012 closed)
=========================================================
New `qa-stats` PHONY target in `Makefile` emits 14 metrics that every
count claim in `docs/qa-test-guide.md` and `docs/testing-guide.md` is
derived from: backend test files / Test functions / t.Run subtests,
frontend test files, fuzz targets, t.Skip sites, qa_test.go Part_ subtests,
testing-guide.md Parts, and unique seed IDs (mc-* / ag-* / iss-* / tgt-* /
nst-*). Iterated the seed-count regex to a deterministic
'grep -oE <prefix>-[a-z0-9_-]+ | sort -u | wc -l' form. Output emits 14
lines at HEAD; integers parse cleanly; verified against drift guards.
P.2 — CI drift guards (M-011 closed)
=========================================================
Two new CI steps in `.github/workflows/ci.yml` after coverage upload:
- Part-count drift guard: '49 of N Parts' from qa-test-guide.md vs
'^## Part N:' header count in testing-guide.md. Fails on mismatch.
- Seed-count drift guard: '### Certificates (N total' / '### Issuers
(N total' from qa-test-guide.md vs unique mc-* / iss-* IDs in
seed_demo.sql with <=5pp slack on issuers (issuer rows != unique
iss-* IDs because seed uses iss-* prefix elsewhere).
Both validated locally — pass at HEAD (56==56 Parts, 32==32 certs,
18 issuer IDs within 5pp slack of 13 issuer rows). YAML lint clean.
P.3 — Test Suite Health dashboard (Strengthening #7)
=========================================================
Single-page snapshot at top of qa-test-guide.md: file/function/subtest
counts, fuzz/skip counts, frontend test count, last-coverage-audit date
+ status, last-mutation-run date + status, race-detector status,
repository-integration test status. Designed for first-look auditor /
acquirer / new-engineer scanning.
P.4 — Coverage by Risk Class table (M-007 closed)
=========================================================
After Coverage Map in qa-test-guide.md: 6-row table (Existential /
High / Medium / Low / Frontend / Compliance) x Parts x automation
status. Cross-references each row to coverage-matrix.md. Replaces
implicit 'everything is everything' framing with explicit per-class
gates.
P.5 — Release Day Sign-Off Matrix (M-010 closed)
=========================================================
12-row release-readiness checklist in qa-test-guide.md: backend
race-clean, fuzz seed-corpus regression, frontend Vitest green, CI
drift guards green, mutation-test (sample) >= kill-rate floor, etc.
Each row cites verification command + gate value. Sign-off is 'all 12
green' — produces a per-release artifact attached to the tag.
P.6 — Mutation Testing Targets (Strengthening #5)
=========================================================
New section in qa-test-guide.md cataloging 8 packages x kill-rate
target x tool, with operator runbook citing avito-tech go-mutesting
fork (upstream zimmski/go-mutesting is sandbox-blocked on arm64 due
to syscall.Dup2). Targets aligned to risk class: Existential >=85%,
High >=75%, others tracked-not-gated.
P.7 — Per-Connector Failure-Mode Matrix (M-009 closed, condensed)
=========================================================
New 'Part 9.0 Per-Connector Failure-Mode Matrix' in
docs/testing-guide.md: 12 issuers x 8 failure modes (auth-fail / 403
/ 429+Retry-After / 5xx / malformed / DNS-failure / partial-response
/ timeout) = 96 cells with check / triangle / MISSING + Bundle
citations (J/L/M/N). Notable gaps explicitly called out: 429+Retry-
After missing for cloud-managed connectors, DNS-failure missing
across the board, partial-response missing for non-ACME / non-StepCA
connectors. Each gap is a follow-on-bundle candidate.
Verification
=========================================================
- 'make qa-stats' runs to completion, emits 14 metrics, all integers
parse cleanly
- 'python3 -c "import yaml; yaml.safe_load(...)"' clean on ci.yml
- Both CI drift guards executed locally — both PASS at HEAD
- git diff --stat: 5 files changed, +249 / -1
Audit deliverables
=========================================================
- gap-backlog.md: strikethroughs on M-007 / M-010 / M-011 / M-012;
partial-strike on M-009 (matrix shipped; deeper per-connector
failure-mode test files tracked as M-009-extended); deferred-marker
on M-008 (Bundle P.2-extended); Bundle P closure-log entry
- closure-plan.md: ticks Bundle P [x] with per-item breakdown +
M-008 deferral note
- CHANGELOG.md: full Bundle P [unreleased] entry above Bundle O
- testing-guide.md: new Part 9.0 Per-Connector Failure-Mode Matrix
- qa-test-guide.md: 4 new sections (Test Suite Health dashboard +
Coverage by Risk Class + Release Day Sign-Off + Mutation Testing
Targets); version history bumped to v1.3
- Makefile: new qa-stats PHONY target
- ci.yml: 2 new drift-guard steps after coverage upload
Closes: M-007, M-010, M-011, M-012
Closes (condensed): M-009 (matrix shipped; deeper test files = M-009-extended)
Deferred: M-008 (Bundle P.2-extended; not acquisition-blocking)
Bundle: P (QA Doc Strengthening)
CI on the bundle-9 merge (run #24962543332) failed golangci-lint with 16
staticcheck ST1018 'string literal contains the Unicode format character
U+202X, consider using the \u202X escape sequence' hits — across the
two test files we added (internal/validation/unicode_test.go +
internal/connector/issuer/local/bundle9_coverage_test.go).
Mechanical sweep, byte-identical at runtime:
internal/validation/unicode_test.go (13 + 1 hits cleared)
RTL/LTR overrides U+202A..U+202E + U+2066..U+2069 (lines 39-47)
zero-width U+200B..U+200D + U+2060 (lines 67-70)
additional U+202E in TestValidateUnicodeSafe_ErrorMentionsByteOffset
internal/connector/issuer/local/bundle9_coverage_test.go (3 hits)
U+202E in TestValidateCSRUnicode_RejectsDNSNameRTL
U+200B in TestValidateCSRUnicode_RejectsEmailZeroWidth
U+202E in TestValidateCSRUnicode_RejectsAdditionalSAN
The strings now use Go \uXXXX escape sequences. Identical UTF-8 bytes
hit ValidateUnicodeSafe at runtime — every test passes unchanged
locally. The file-header comment in unicode_test.go that promised this
convention is now actually honored.
Verification: staticcheck -checks=ST1018 returns clean across the two
packages. go test -count=1 -short still green.
Pre-commit gate added to prevent recurrence:
Makefile: new 'verify' aggregate target runs gofmt + go vet +
golangci-lint run + go test -short — same set CI enforces. Run
'make verify' before every commit going forward.
cowork/CLAUDE.md: new 'Pre-commit verification gate' paragraph in
Operating Rules. Documents make verify as the canonical gate;
explains WHY (Bundle-9 shipped green-on-vet / red-on-CI because
ST1018 only fires under golangci-lint's staticcheck, not vet);
documents the staticcheck-only fallback for disk-constrained
sandboxes.
This commit changes only:
- 2 test source files (\uXXXX escapes, no behavior change)
- Makefile (1 new target, 1 .PHONY entry, 1 help line)
- cowork/CLAUDE.md (1 new operating-rule paragraph)
Wire all remaining dashboard views to real API: agent detail page
with heartbeat status and capabilities, audit trail with time range/
actor/resource filters, notifications with grouped-by-cert view and
read/unread state, policies with severity summary bar, new issuers
and targets list views. Add GitHub Actions CI with parallel Go and
Frontend jobs. Update Makefile with test-cover and frontend-build
targets.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
shankar0123