M-001/M-006: strip HTTP auth from EST/SCEP + fail-loud SCEP preflight

Closes CWE-306 (missing authentication for critical function) for SCEP
via a fail-loud startup gate, and aligns EST/SCEP HTTP dispatch with
their respective RFCs. CRL/OCSP remain unauthenticated under
.well-known/pki/* per RFC 5280 §5 / RFC 6960 / RFC 8615. Option (D):
no mTLS in this milestone.

- RFC 7030 §3.2.3 (EST auth is deployment-specific) and §4.1.1
  (/cacerts explicitly anonymous): EST paths served unauthenticated;
  CSR-signature + profile policy enforce identity inside ESTService.
- RFC 8894 §3.2: SCEP authenticates via the challengePassword
  PKCS#10 attribute (OID 1.2.840.113549.1.9.7), not an HTTP credential.
  HTTP dispatch is unauthenticated; preflightSCEPChallengePassword
  refuses to start when CERTCTL_SCEP_ENABLED=true without
  CERTCTL_SCEP_CHALLENGE_PASSWORD. SCEPService.PKCSReq enforces the
  same invariant defense-in-depth and compares with
  crypto/subtle.ConstantTimeCompare.

cmd/server/main.go:
- Extract buildFinalHandler(apiHandler, noAuthHandler, webDir,
  dashboardEnabled); route /.well-known/est/*, /scep, /scep/*,
  /.well-known/pki/crl/{id}, /.well-known/pki/ocsp/{id}/{serial},
  and health probes through noAuthHandler (RequestID +
  structuredLogger + Recovery only).
- Add preflightSCEPChallengePassword fail-loud gate; startup log
  emits challenge_password_set boolean for operator visibility.

cmd/server/finalhandler_test.go (new, 314 lines, 27 subtests):
- TestBuildFinalHandler_Dispatch (20) + TestBuildFinalHandler_NoDashboard
  (7) pin the dispatch surface: EST 4-endpoint, SCEP exact +
  trailing-slash + query-string, PKI CRL+OCSP, health, /api/v1/*
  authenticated, /assets/* file server, SPA fallback.

internal/api/router/router.go, internal/config/config.go:
- Router-level comments explain why EST/SCEP/PKI dispatchers sit
  outside the authenticated mux; SCEP challenge password config
  plumbed through.

docs/architecture.md:
- New EST Authentication subsection (RFC 7030 §3.2.3 + §4.1.1,
  buildFinalHandler + noAuthHandler references).
- Rewrite SCEP Authentication subsection; replaces pre-existing
  factually-incorrect "any value accepted" claim with CWE-306
  preflight, service-layer defense-in-depth, and
  crypto/subtle.ConstantTimeCompare.
- Top-level Authentication section: qualify /api/v1/* scope on API
  clients bullet; add standards-based-endpoints bullet referencing
  the 27-subtest regression harness.

docs/compliance-soc2.md:
- CC6.1: scope API Key Authentication to /api/v1/*; add
  standards-based endpoints bullet citing RFCs and CWE-306 closure.
- CC6.3: scope API Key Policy to /api/v1/* with cross-reference to
  CC6.1.
- Evidence Locations augmented with buildFinalHandler,
  preflightSCEPChallengePassword, scep.go defense path, regression
  harness, and OpenAPI security:[] overrides.

api/openapi.yaml: verified already correct (global bearerAuth
default overridden with security:[] on /cacerts, /simpleenroll,
/simplereenroll, /csrattrs, /scep GET+POST, /crl/{issuer_id},
/ocsp/{issuer_id}/{serial}); no edits needed.

cmd/server/main.go

@@ -725,55 +725,14 @@ func main() {
 		middleware.Recovery,
 	)
 
+	dashboardEnabled := false
 	if _, err := os.Stat(webDir + "/index.html"); err == nil {
-		fileServer := http.FileServer(http.Dir(webDir))
-		finalHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-			path := r.URL.Path
-			// Health/ready and auth/info bypass auth middleware.
-			// Health/ready: Docker/K8s health probes don't carry Bearer tokens.
-			// auth/info: React app calls this before login to detect auth mode.
-			if path == "/health" || path == "/ready" || path == "/api/v1/auth/info" {
-				noAuthHandler.ServeHTTP(w, r)
-				return
-			}
-			// RFC 5280 CRL and RFC 6960 OCSP live under /.well-known/pki/ and
-			// MUST be served unauthenticated — relying parties (browsers,
-			// OpenSSL, OCSP stapling sidecars, mTLS clients) cannot present
-			// certctl Bearer tokens. See router.RegisterPKIHandlers.
-			if len(path) >= 16 && path[:16] == "/.well-known/pki" {
-				noAuthHandler.ServeHTTP(w, r)
-				return
-			}
-			// All other API and EST routes go through the full middleware stack (with auth)
-			if (len(path) >= 8 && path[:8] == "/api/v1/") ||
-				(len(path) >= 16 && path[:16] == "/.well-known/est") {
-				apiHandler.ServeHTTP(w, r)
-				return
-			}
-			// Try to serve static files (JS, CSS, assets)
-			if len(path) > 8 && path[:8] == "/assets/" {
-				fileServer.ServeHTTP(w, r)
-				return
-			}
-			// SPA fallback: serve index.html for all other routes
-			http.ServeFile(w, r, webDir+"/index.html")
-		})
+		dashboardEnabled = true
+	}
+	finalHandler = buildFinalHandler(apiHandler, noAuthHandler, webDir, dashboardEnabled)
+	if dashboardEnabled {
 		logger.Info("dashboard available at /", "web_dir", webDir)
 	} else {
-		// No dashboard: route health/auth-info and /.well-known/pki without
-		// auth, everything else through full stack.
-		finalHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-			path := r.URL.Path
-			if path == "/health" || path == "/ready" || path == "/api/v1/auth/info" {
-				noAuthHandler.ServeHTTP(w, r)
-				return
-			}
-			if len(path) >= 16 && path[:16] == "/.well-known/pki" {
-				noAuthHandler.ServeHTTP(w, r)
-				return
-			}
-			apiHandler.ServeHTTP(w, r)
-		})
 		logger.Info("dashboard directory not found, serving API only")
 	}
 
@@ -858,3 +817,95 @@ func preflightSCEPChallengePassword(enabled bool, challengePassword string) erro
 	}
 	return nil
 }
+
+// buildFinalHandler builds the outer HTTP dispatch handler that routes incoming
+// requests to either the authenticated apiHandler chain or the unauthenticated
+// noAuthHandler chain based on URL path prefix. Extracted from main() so the
+// dispatch logic can be unit tested without booting the full server stack
+// (see cmd/server/finalhandler_test.go).
+//
+// Dispatch rules (M-001, audit 2026-04-19, option D):
+//
+//   - /health, /ready, /api/v1/auth/info           → no-auth (probes + login detection)
+//   - /.well-known/pki/*                           → no-auth (RFC 5280 CRL, RFC 6960 OCSP)
+//   - /.well-known/est/*                           → no-auth (RFC 7030 §3.2.3)
+//   - /scep, /scep/*                               → no-auth (RFC 8894 §3.2, CSR challengePassword)
+//   - /api/v1/*                                    → auth (Bearer token required)
+//   - /assets/*                                    → static file server (dashboard only)
+//   - anything else                                → SPA index.html fallback (dashboard only)
+//                                                    OR apiHandler (no dashboard)
+//
+// EST/SCEP clients (IoT devices, 802.1X supplicants, MDM endpoints, network
+// appliances) cannot present certctl Bearer tokens, so those endpoints must be
+// reachable without the Auth middleware. Authentication is instead enforced by
+// CSR signature verification, profile policy gates, and for SCEP the
+// challengePassword shared secret (fail-loud gated by preflightSCEPChallengePassword
+// above).
+//
+// webDir must point to a directory containing index.html + assets/ when
+// dashboardEnabled is true; it is ignored otherwise.
+func buildFinalHandler(apiHandler, noAuthHandler http.Handler, webDir string, dashboardEnabled bool) http.Handler {
+	var fileServer http.Handler
+	if dashboardEnabled {
+		fileServer = http.FileServer(http.Dir(webDir))
+	}
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		path := r.URL.Path
+
+		// Health/ready and auth/info bypass auth middleware.
+		// Health/ready: Docker/K8s health probes don't carry Bearer tokens.
+		// auth/info: React app calls this before login to detect auth mode.
+		if path == "/health" || path == "/ready" || path == "/api/v1/auth/info" {
+			noAuthHandler.ServeHTTP(w, r)
+			return
+		}
+
+		// RFC 5280 CRL and RFC 6960 OCSP live under /.well-known/pki/ and MUST
+		// be served unauthenticated — relying parties (browsers, OpenSSL, OCSP
+		// stapling sidecars, mTLS clients) cannot present certctl Bearer tokens.
+		if strings.HasPrefix(path, "/.well-known/pki") {
+			noAuthHandler.ServeHTTP(w, r)
+			return
+		}
+
+		// RFC 7030 EST endpoints ride the no-auth middleware chain (M-001,
+		// option D, audit 2026-04-19). Trust boundary is CSR signature + profile
+		// policy, not HTTP Bearer. /.well-known/est/cacerts is explicitly
+		// anonymous per RFC 7030 §4.1.1.
+		if strings.HasPrefix(path, "/.well-known/est") {
+			noAuthHandler.ServeHTTP(w, r)
+			return
+		}
+
+		// RFC 8894 SCEP rides the no-auth chain (M-001, option D). SCEP clients
+		// authenticate via the challengePassword attribute in the PKCS#10 CSR,
+		// not via HTTP Bearer tokens. preflightSCEPChallengePassword refuses to
+		// start the server if SCEP is enabled without a non-empty shared secret.
+		if path == "/scep" || strings.HasPrefix(path, "/scep/") {
+			noAuthHandler.ServeHTTP(w, r)
+			return
+		}
+
+		// Authenticated API routes — full middleware stack including Auth.
+		if strings.HasPrefix(path, "/api/v1/") {
+			apiHandler.ServeHTTP(w, r)
+			return
+		}
+
+		if !dashboardEnabled {
+			// No dashboard: everything non-special falls through to the
+			// authenticated handler (preserves pre-M-001 behavior for API-only
+			// deployments).
+			apiHandler.ServeHTTP(w, r)
+			return
+		}
+
+		// Dashboard-present: serve static assets directly, SPA fallback for
+		// everything else.
+		if strings.HasPrefix(path, "/assets/") {
+			fileServer.ServeHTTP(w, r)
+			return
+		}
+		http.ServeFile(w, r, webDir+"/index.html")
+	})
+}