fix(audit): ship streaming NDJSON audit export endpoint (HIGH-9 / HIGH-11)

Audit 2026-05-10 HIGH-9 + HIGH-11 closure. HIGH-10 deferred to v3.

HIGH-9 (verification only): Fix 01's CRIT-1 router-gate sweep already
wraps every role-mgmt route with rbacGate. Verified via grep:
  - GET    /api/v1/auth/roles                          → auth.role.list
  - POST   /api/v1/auth/roles                          → auth.role.create
  - GET    /api/v1/auth/roles/{id}                     → auth.role.list
  - PUT    /api/v1/auth/roles/{id}                     → auth.role.edit
  - DELETE /api/v1/auth/roles/{id}                     → auth.role.delete
  - POST   /api/v1/auth/roles/{id}/permissions         → auth.role.edit
  - DELETE /api/v1/auth/roles/{id}/permissions/{perm}  → auth.role.edit
  - POST   /api/v1/auth/keys/{id}/roles                → auth.role.assign
  - DELETE /api/v1/auth/keys/{id}/roles/{role_id}      → auth.role.revoke
Defense-in-depth invariant restored: privilege check fires at BOTH
router and service layers; AST-level coverage is pinned by
TestRouterRBACGateCoverage (Fix 01's CI guard).

HIGH-11: ship GET /api/v1/audit/export — streaming NDJSON audit export
gated by audit.export. Pre-fix, the permission was seeded into r-admin
and r-auditor (migration 000031) but no endpoint enforced it; r-auditor's
claim was misleading capability advertisement. Post-fix:

  - internal/api/handler/audit.go::ExportAudit emits one JSON event per
    line as application/x-ndjson — the de-facto compliance-archive
    format consumed by SIEMs (Splunk universal forwarder, Elastic
    Filebeat, Vector).
  - Required from/to (RFC3339) bounded to a 90-day max window;
    optional category filter (cert_lifecycle/auth/config); optional
    limit capped at 100k rows.
  - Content-Disposition: attachment; filename="certctl-audit-<from>_to_<to>.ndjson"
    so curl + browser downloads land with a sensible filename.
  - Recursively self-audits: every successful export emits an
    audit.export row capturing actor + range + category + row count
    so compliance reviewers can see who pulled which evidence and when.
  - Service layer: AuditService.ExportEventsByFilter reuses the
    existing repository.AuditFilter (From/To/EventCategory already
    supported); no SQL duplication.
  - OpenAPI parity exception added for the streaming-shape route
    (matches the ACME/SCEP/EST precedent at
    internal/api/router/openapi_parity_test.go::SpecParityExceptions).

Regression matrix in audit_export_test.go (7 cases):
  - TestExportAudit_StreamsNDJSONLines (happy path; pins content-type +
    content-disposition + JSON-per-line shape + recursive self-audit)
  - TestExportAudit_RejectsRangeBeyond90Days (100-day window → 400)
  - TestExportAudit_RejectsMissingFromOrTo (3 cases)
  - TestExportAudit_RejectsInvalidCategory (unknown enum → 400)
  - TestExportAudit_AcceptsValidCategoryFilter (auth filter passes through)
  - TestExportAudit_RejectsNonGET (POST → 405)
  - TestExportAudit_RejectsToBeforeFrom (inverted range → 400)

The auditor role's surface is now complete (read + export). The
handler interface is extended with ExportEventsByFilter +
RecordEventWithCategory; mockAuditService satisfies both with a
self-audit trace (lastAuditAction / lastAuditCategory / lastAuditActor).

HIGH-10 (scope + expiry on assignRoleRequest): DEFERRED to v3.
Schema column already exists (ActorRole.ExpiresAt); load-bearing wire
remains v3 work. Documented carve-out at HIGH-10's annotation.

Refs: cowork/auth-bundles-audit-2026-05-10.md HIGH-9 HIGH-11
Spec: cowork/auth-bundles-fixes-2026-05-10/12-high-9-10-11-role-mgmt-cleanup.md
This commit is contained in:
shankar0123
2026-05-10 21:36:01 +00:00
parent b81588e717
commit 8bd85af400
6 changed files with 436 additions and 0 deletions
+162
View File
@@ -2,11 +2,16 @@ package handler
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"net/http"
"strconv"
"strings"
"time"
"github.com/certctl-io/certctl/internal/api/middleware"
"github.com/certctl-io/certctl/internal/auth"
"github.com/certctl-io/certctl/internal/domain"
)
@@ -20,6 +25,18 @@ type AuditService interface {
// empty string returns all categories. Used by the auditor role
// (filtered to "auth" via /v1/audit?category=auth).
ListAuditEventsByCategory(ctx context.Context, eventCategory string, page, perPage int) ([]domain.AuditEvent, int64, error)
// ExportEventsByFilter returns audit events matching a
// (from, to, eventCategory) filter, capped at maxRows. Audit
// 2026-05-10 HIGH-11 closure — backs the new
// GET /api/v1/audit/export endpoint that makes the `audit.export`
// permission load-bearing.
ExportEventsByFilter(ctx context.Context, from, to time.Time, eventCategory string, maxRows int) ([]domain.AuditEvent, error)
// RecordEventWithCategory is needed by the export handler so it
// can recursively self-audit each export call (operator-visible
// proof that compliance evidence pulls happened + by whom + over
// what range). The bare-string actor type is the existing wire
// shape used by every other Phase 8 caller.
RecordEventWithCategory(ctx context.Context, actor string, actorType domain.ActorType, action, eventCategory, resourceType, resourceID string, details map[string]interface{}) error
}
// AuditHandler handles HTTP requests for audit event operations.
@@ -124,3 +141,148 @@ func (h AuditHandler) GetAuditEvent(w http.ResponseWriter, r *http.Request) {
JSON(w, http.StatusOK, event)
}
// ExportAudit streams an NDJSON export of audit events for compliance
// evidence collection. Gated by the `audit.export` permission (already
// seeded into r-admin + r-auditor by migration 000031).
//
// Audit 2026-05-10 HIGH-11 closure — pre-fix, the permission existed
// in the catalogue + role grants but no endpoint enforced it; r-auditor's
// "audit.export" claim was misleading capability advertisement. This
// endpoint makes the permission load-bearing and the auditor role's
// surface complete.
//
// GET /api/v1/audit/export?from=<RFC3339>&to=<RFC3339>&category=<cat>
//
// Constraints:
// - from + to are required, RFC3339 format.
// - to - from MUST be ≤ 90 days (compliance window).
// - category optional: cert_lifecycle | auth | config.
// - max 50,000 rows per export (operator-tunable via query param
// up to 100,000); larger exports require operator-side pagination
// by date range.
//
// Response: application/x-ndjson, one event per line. Newline-delimited
// JSON is the de-facto compliance-archive format consumed by SIEMs
// (Splunk universal forwarder, Elastic Filebeat, Vector, etc.).
//
// The export itself is recursively audited: every successful export
// emits an `audit.export` event capturing actor, range, category, and
// row count so the audit log itself records who pulled which compliance
// evidence and when.
func (h AuditHandler) ExportAudit(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
Error(w, http.StatusMethodNotAllowed, "Method not allowed")
return
}
requestID := middleware.GetRequestID(r.Context())
q := r.URL.Query()
fromStr := q.Get("from")
toStr := q.Get("to")
if fromStr == "" || toStr == "" {
ErrorWithRequestID(w, http.StatusBadRequest,
"`from` and `to` query params are required (RFC3339 format)",
requestID)
return
}
from, err := time.Parse(time.RFC3339, fromStr)
if err != nil {
ErrorWithRequestID(w, http.StatusBadRequest,
"`from` must be RFC3339 (e.g. 2026-04-01T00:00:00Z)",
requestID)
return
}
to, err := time.Parse(time.RFC3339, toStr)
if err != nil {
ErrorWithRequestID(w, http.StatusBadRequest,
"`to` must be RFC3339 (e.g. 2026-05-01T00:00:00Z)",
requestID)
return
}
if !to.After(from) {
ErrorWithRequestID(w, http.StatusBadRequest,
"`to` must be after `from`",
requestID)
return
}
const maxWindow = 90 * 24 * time.Hour
if to.Sub(from) > maxWindow {
ErrorWithRequestID(w, http.StatusBadRequest,
fmt.Sprintf("range exceeds 90-day max (got %s); paginate by narrower date range", to.Sub(from)),
requestID)
return
}
category := q.Get("category")
if category != "" {
switch category {
case domain.EventCategoryCertLifecycle, domain.EventCategoryAuth, domain.EventCategoryConfig:
// ok
default:
ErrorWithRequestID(w, http.StatusBadRequest,
"Invalid category — allowed: cert_lifecycle, auth, config",
requestID)
return
}
}
maxRows := 50000
if lim := q.Get("limit"); lim != "" {
if parsed, err := strconv.Atoi(lim); err == nil && parsed > 0 && parsed <= 100000 {
maxRows = parsed
}
}
events, err := h.svc.ExportEventsByFilter(r.Context(), from, to, category, maxRows)
if err != nil {
ErrorWithRequestID(w, http.StatusInternalServerError,
"Failed to export audit events",
requestID)
return
}
w.Header().Set("Content-Type", "application/x-ndjson")
w.Header().Set("Content-Disposition",
fmt.Sprintf(`attachment; filename="certctl-audit-%s_to_%s.ndjson"`,
from.UTC().Format("2006-01-02"), to.UTC().Format("2006-01-02")))
w.WriteHeader(http.StatusOK)
enc := json.NewEncoder(w)
for i := range events {
if err := enc.Encode(&events[i]); err != nil {
// Mid-stream encode error — connection probably closed by
// client. Logged + abandoned; the partial response is
// already on the wire and rolling back the headers isn't
// possible.
slog.WarnContext(r.Context(), "audit export: encode failed mid-stream",
"err", err, "rows_written", i, "rows_total", len(events))
return
}
}
// Recursively self-audit the export. The audit row captures actor,
// from, to, category, and row count so compliance reviewers can see
// who pulled which evidence and when. Best-effort (the data is
// already on the wire); failure logs WARN per the HIGH-6 closure.
actorID, _ := r.Context().Value(auth.ActorIDKey{}).(string)
if actorID == "" {
actorID = "unknown"
}
if err := h.svc.RecordEventWithCategory(r.Context(),
actorID, domain.ActorTypeUser,
"audit.export", domain.EventCategoryAuth,
"audit", "export",
map[string]interface{}{
"from": from.UTC().Format(time.RFC3339),
"to": to.UTC().Format(time.RFC3339),
"category": category,
"rows": len(events),
}); err != nil {
slog.WarnContext(r.Context(), "audit.export self-audit failed (export already streamed)",
"actor_id", actorID, "rows", len(events), "err", err)
}
}
+189
View File
@@ -0,0 +1,189 @@
package handler
import (
"bufio"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/certctl-io/certctl/internal/domain"
)
// Audit 2026-05-10 HIGH-11 closure — pin the streaming NDJSON audit
// export endpoint. Pre-fix, the `audit.export` permission was seeded
// into r-admin + r-auditor (migration 000031) but no endpoint enforced
// it; the auditor role's claim was misleading capability advertisement.
// Post-fix, GET /api/v1/audit/export gates on `audit.export`, streams
// audit rows as line-delimited JSON, bounded to a 90-day window, and
// recursively self-audits each export call.
// exportMockSvc extends mockAuditService with explicit hooks for the
// HIGH-11 export path.
type exportMockSvc struct {
mockAuditService
exportFn func(from, to time.Time, eventCategory string, maxRows int) ([]domain.AuditEvent, error)
}
func (m *exportMockSvc) ExportEventsByFilter(_ context.Context, from, to time.Time, eventCategory string, maxRows int) ([]domain.AuditEvent, error) {
if m.exportFn != nil {
return m.exportFn(from, to, eventCategory, maxRows)
}
return nil, nil
}
func TestExportAudit_StreamsNDJSONLines(t *testing.T) {
events := []domain.AuditEvent{
{ID: "ev-1", Action: "cert.issue", Actor: "alice", Timestamp: time.Now()},
{ID: "ev-2", Action: "cert.revoke", Actor: "bob", Timestamp: time.Now()},
{ID: "ev-3", Action: "auth.role.grant", Actor: "alice", Timestamp: time.Now()},
}
mockSvc := &exportMockSvc{
exportFn: func(from, to time.Time, _ string, _ int) ([]domain.AuditEvent, error) {
return events, nil
},
}
h := NewAuditHandler(mockSvc)
req := httptest.NewRequest(http.MethodGet,
"/api/v1/audit/export?from=2026-04-01T00:00:00Z&to=2026-05-01T00:00:00Z", nil)
w := httptest.NewRecorder()
h.ExportAudit(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d; want 200; body=%s", w.Code, w.Body.String())
}
if ct := w.Header().Get("Content-Type"); ct != "application/x-ndjson" {
t.Errorf("Content-Type = %q; want application/x-ndjson", ct)
}
if cd := w.Header().Get("Content-Disposition"); !strings.HasPrefix(cd, "attachment;") {
t.Errorf("Content-Disposition = %q; want attachment;...", cd)
}
scanner := bufio.NewScanner(strings.NewReader(w.Body.String()))
count := 0
for scanner.Scan() {
line := scanner.Text()
if line == "" {
continue
}
var got domain.AuditEvent
if err := json.Unmarshal([]byte(line), &got); err != nil {
t.Errorf("line %d not valid JSON: %v; line=%s", count, err, line)
}
count++
}
if count != len(events) {
t.Errorf("scanned %d NDJSON lines; want %d", count, len(events))
}
// Self-audit leg: the export must emit an audit.export row for the
// recursive trail.
if mockSvc.lastAuditAction != "audit.export" {
t.Errorf("lastAuditAction = %q; want audit.export (recursive self-audit)", mockSvc.lastAuditAction)
}
if mockSvc.lastAuditCategory != domain.EventCategoryAuth {
t.Errorf("lastAuditCategory = %q; want %q", mockSvc.lastAuditCategory, domain.EventCategoryAuth)
}
}
func TestExportAudit_RejectsRangeBeyond90Days(t *testing.T) {
mockSvc := &exportMockSvc{}
h := NewAuditHandler(mockSvc)
// 100-day window — must reject.
req := httptest.NewRequest(http.MethodGet,
"/api/v1/audit/export?from=2026-01-01T00:00:00Z&to=2026-04-15T00:00:00Z", nil)
w := httptest.NewRecorder()
h.ExportAudit(w, req)
if w.Code != http.StatusBadRequest {
t.Errorf("status = %d; want 400 for >90d range", w.Code)
}
if !strings.Contains(w.Body.String(), "90-day") {
t.Errorf("body = %q; want it to mention the 90-day cap", w.Body.String())
}
}
func TestExportAudit_RejectsMissingFromOrTo(t *testing.T) {
mockSvc := &exportMockSvc{}
h := NewAuditHandler(mockSvc)
cases := []string{
"/api/v1/audit/export",
"/api/v1/audit/export?from=2026-04-01T00:00:00Z",
"/api/v1/audit/export?to=2026-04-30T00:00:00Z",
}
for _, url := range cases {
req := httptest.NewRequest(http.MethodGet, url, nil)
w := httptest.NewRecorder()
h.ExportAudit(w, req)
if w.Code != http.StatusBadRequest {
t.Errorf("URL %q: status = %d; want 400 (missing from/to)", url, w.Code)
}
}
}
func TestExportAudit_RejectsInvalidCategory(t *testing.T) {
mockSvc := &exportMockSvc{}
h := NewAuditHandler(mockSvc)
req := httptest.NewRequest(http.MethodGet,
"/api/v1/audit/export?from=2026-04-01T00:00:00Z&to=2026-04-30T00:00:00Z&category=zzz_unknown", nil)
w := httptest.NewRecorder()
h.ExportAudit(w, req)
if w.Code != http.StatusBadRequest {
t.Errorf("status = %d; want 400 for invalid category", w.Code)
}
}
func TestExportAudit_AcceptsValidCategoryFilter(t *testing.T) {
captured := struct {
category string
}{}
mockSvc := &exportMockSvc{
exportFn: func(_, _ time.Time, eventCategory string, _ int) ([]domain.AuditEvent, error) {
captured.category = eventCategory
return []domain.AuditEvent{}, nil
},
}
h := NewAuditHandler(mockSvc)
req := httptest.NewRequest(http.MethodGet,
"/api/v1/audit/export?from=2026-04-01T00:00:00Z&to=2026-04-30T00:00:00Z&category=auth", nil)
w := httptest.NewRecorder()
h.ExportAudit(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status = %d; want 200; body=%s", w.Code, w.Body.String())
}
if captured.category != domain.EventCategoryAuth {
t.Errorf("captured.category = %q; want %q", captured.category, domain.EventCategoryAuth)
}
}
func TestExportAudit_RejectsNonGET(t *testing.T) {
mockSvc := &exportMockSvc{}
h := NewAuditHandler(mockSvc)
req := httptest.NewRequest(http.MethodPost,
"/api/v1/audit/export?from=2026-04-01T00:00:00Z&to=2026-04-30T00:00:00Z", nil)
w := httptest.NewRecorder()
h.ExportAudit(w, req)
if w.Code != http.StatusMethodNotAllowed {
t.Errorf("status = %d; want 405 for POST", w.Code)
}
}
func TestExportAudit_RejectsToBeforeFrom(t *testing.T) {
mockSvc := &exportMockSvc{}
h := NewAuditHandler(mockSvc)
req := httptest.NewRequest(http.MethodGet,
"/api/v1/audit/export?from=2026-05-01T00:00:00Z&to=2026-04-01T00:00:00Z", nil)
w := httptest.NewRecorder()
h.ExportAudit(w, req)
if w.Code != http.StatusBadRequest {
t.Errorf("status = %d; want 400 (to before from)", w.Code)
}
}
@@ -18,6 +18,10 @@ type mockAuditService struct {
listFunc func(page, perPage int) ([]domain.AuditEvent, int64, error)
listByCatFunc func(category string, page, perPage int) ([]domain.AuditEvent, int64, error)
getFunc func(id string) (*domain.AuditEvent, error)
// HIGH-11 self-audit trace — last RecordEventWithCategory call.
lastAuditActor string
lastAuditAction string
lastAuditCategory string
}
func (m *mockAuditService) ListAuditEvents(_ context.Context, page, perPage int) ([]domain.AuditEvent, int64, error) {
@@ -44,6 +48,32 @@ func (m *mockAuditService) GetAuditEvent(_ context.Context, id string) (*domain.
return nil, nil
}
// ExportEventsByFilter satisfies the Audit 2026-05-10 HIGH-11 interface
// extension. The test mock just defers to the existing list helpers
// (no separate export-specific test fixture needed for the bundles that
// don't exercise export).
func (m *mockAuditService) ExportEventsByFilter(_ context.Context, _, _ time.Time, eventCategory string, _ int) ([]domain.AuditEvent, error) {
if m.listFunc != nil {
events, _, err := m.listFunc(1, 50000)
if err != nil {
return nil, err
}
return events, nil
}
return nil, nil
}
// RecordEventWithCategory satisfies the Audit 2026-05-10 HIGH-11
// interface extension (the export handler self-audits each call).
// Tests that don't care about the audit row trace can leave the field
// nil; tests that do can read m.lastAuditAction etc. after the call.
func (m *mockAuditService) RecordEventWithCategory(_ context.Context, actor string, _ domain.ActorType, action, eventCategory, _, _ string, _ map[string]interface{}) error {
m.lastAuditActor = actor
m.lastAuditAction = action
m.lastAuditCategory = eventCategory
return nil
}
func TestListAuditEvents_Success(t *testing.T) {
events := []domain.AuditEvent{
{
@@ -144,6 +144,15 @@ var SpecParityExceptions = map[string]string{
"POST /api/v1/auth/breakglass/credentials": "Auth Bundle 2 Phase 7.5 — set/rotate password; gated auth.breakglass.admin.",
"POST /api/v1/auth/breakglass/credentials/{actor_id}/unlock": "Auth Bundle 2 Phase 7.5 — clear lockout state; gated auth.breakglass.admin.",
"DELETE /api/v1/auth/breakglass/credentials/{actor_id}": "Auth Bundle 2 Phase 7.5 — remove credential; gated auth.breakglass.admin.",
// Audit 2026-05-10 HIGH-11 — streaming NDJSON audit export. Like
// other streaming wire-protocol surfaces (ACME, SCEP, EST), the
// response is line-oriented application/x-ndjson rather than a
// single JSON object; documenting it as a regular OpenAPI operation
// would misrepresent the streaming shape. The contract is documented
// in docs/operator/security.md::audit-export and the handler doc
// comment.
"GET /api/v1/audit/export": "Audit 2026-05-10 HIGH-11 — streaming NDJSON audit export; gated audit.export. Documented inline at internal/api/handler/audit.go::ExportAudit.",
}
func TestRouter_OpenAPIParity(t *testing.T) {
+8
View File
@@ -627,6 +627,14 @@ func (r *Router) RegisterHandlers(reg HandlerRegistry) {
// Audit routes: /api/v1/audit
r.Register("GET /api/v1/audit", rbacGate(reg.Checker, "audit.read", reg.Audit.ListAuditEvents))
// Audit 2026-05-10 HIGH-11 closure — `audit.export` permission was
// already seeded into r-admin + r-auditor (migration 000031), but
// no endpoint enforced it pre-fix; r-auditor's claim was misleading
// capability advertisement. The export endpoint makes the grant
// load-bearing. Register `/audit/export` BEFORE `/audit/{id}` so
// Go's net/http stdlib routing gives the more specific path
// precedence over the catch-all.
r.Register("GET /api/v1/audit/export", rbacGate(reg.Checker, "audit.export", reg.Audit.ExportAudit))
r.Register("GET /api/v1/audit/{id}", rbacGate(reg.Checker, "audit.read", reg.Audit.GetAuditEvent))
// Bundle CRL/OCSP-Responder Phase 5: admin observability for the
+38
View File
@@ -247,6 +247,44 @@ func (s *AuditService) ListAuditEventsByCategory(ctx context.Context, eventCateg
return result, total, nil
}
// ExportEventsByFilter returns audit events matching a date-range +
// optional category filter without pagination — the export handler
// uses this to stream NDJSON for compliance evidence collection.
//
// Audit 2026-05-10 HIGH-11 closure: pre-fix, the `audit.export`
// permission was seeded into r-admin and r-auditor (migration 000031)
// but no endpoint enforced it — misleading capability advertisement.
// This method is the service-layer building block for the new
// GET /api/v1/audit/export endpoint.
//
// Bounded callers: the handler enforces a max 90-day range + max-rows
// cap before invoking this; the service-layer method itself is
// permissive so future callers (compliance-job runner, MCP tool) can
// reuse the helper without duplicating the bound enforcement.
func (s *AuditService) ExportEventsByFilter(ctx context.Context, from, to time.Time, eventCategory string, maxRows int) ([]domain.AuditEvent, error) {
if maxRows <= 0 {
maxRows = 50000
}
filter := &repository.AuditFilter{
EventCategory: eventCategory,
From: from,
To: to,
Page: 1,
PerPage: maxRows,
}
events, err := s.auditRepo.List(ctx, filter)
if err != nil {
return nil, fmt.Errorf("failed to list audit events for export: %w", err)
}
out := make([]domain.AuditEvent, 0, len(events))
for _, e := range events {
if e != nil {
out = append(out, *e)
}
}
return out, nil
}
// GetAuditEvent returns a single audit event (handler interface method).
func (s *AuditService) GetAuditEvent(ctx context.Context, id string) (*domain.AuditEvent, error) {
filter := &repository.AuditFilter{