acme-server: orders + authorizations + finalize + cert download (Phase 2/7)

Closes the issuance loop in trust_authenticated mode (commits ec88a61
+ 44a85d6 wired the foundation + JWS-verified account resource).
After this commit, an ACME client running against a profile with
acme_auth_mode='trust_authenticated' end-to-end-issues a real cert:

  POST /acme/profile/<id>/new-order      → 201 + order URL (status=ready)
  POST /acme/profile/<id>/order/<oid>    → POST-as-GET fetch
  POST /acme/profile/<id>/order/<oid>/finalize  → 200 + status=valid + cert URL
  POST /acme/profile/<id>/cert/<cid>     → 200 + PEM chain

Profiles with acme_auth_mode='challenge' get the same code path with
authz/challenge rows in `pending` state until Phase 3's validators
wire up. The mode is read from the bound profile's column at request
time, NOT cached at server start — operators flipping the column via
SQL take effect on the next order without restart.

Architecture (the load-bearing part):
  - Finalize routes through service.CertificateService.Create — the
    canonical certctl issuance entry point that wraps the
    managed_certificates row insert + audit row in s.tx.WithinTx.
    RenewalPolicy / CertificateProfile / per-issuer-type Prometheus
    metrics / audit rows all apply uniformly to ACME-issued certs via
    the same code path that already serves EST/SCEP/agent/REST issuance.
  - Identifier validation runs BEFORE order creation. Rejected
    identifiers return RFC 7807 with per-identifier subproblems and
    create no order row.
  - Source stamp on managed_certificates: domain.CertificateSourceACME.
    Operators bulk-revoke ACME-issued certs by filtering on Source=ACME.
  - 3-step atomicity boundary documented in code + this commit msg:
    (A) WithinTx-A marks order processing + audit row.
    (B) IssuerConnector.IssueCertificate + CertificateService.Create
        (each in its own WithinTx — Create wraps cert row + audit
        atomically).
    (C) WithinTx-C creates certificate_versions row + transitions order
        to valid + sets certificate_id + audit row.
    The brief window between B and C can leave a managed_certificates
    row whose order is still in `processing`. Phase 5's GC scheduler
    reconciles. Documented inline.

What ships:
  - internal/api/acme/order.go: OrderResponseJSON + AuthorizationResponseJSON
    + ChallengeResponseJSON + NewOrderRequest + FinalizeRequest wire
    shapes; ValidateIdentifiers (Phase 2 syntactic checks, dns-only);
    CSRMatchesIdentifiers (RFC 8555 §7.4 strict equality, case-folded).
  - internal/domain/acme.go: ACMEOrder + ACMEAuthorization + ACMEChallenge
    + ACMEIdentifier + ACMEProblem domain types + closed status enums
    for each (order: pending|ready|processing|valid|invalid; authz:
    pending|valid|invalid|deactivated|expired|revoked; challenge:
    pending|processing|valid|invalid; challenge type: http-01|dns-01|
    tls-alpn-01).
  - internal/domain/profile.go: new ACMEAuthMode field reading from
    certificate_profiles.acme_auth_mode (added in migration 25).
  - internal/domain/certificate.go: new CertificateSourceACME enum value.
  - internal/repository/postgres/profile.go: extended SELECT/scanProfile
    to read the per-profile acme_auth_mode column with a COALESCE
    default of trust_authenticated.
  - internal/repository/postgres/acme.go: full order/authz/challenge
    CRUD (CreateOrderWithTx + GetOrderByID + UpdateOrderWithTx +
    CreateAuthzWithTx + GetAuthzByID + ListAuthzsByOrder +
    ListChallengesByAuthz + CreateChallengeWithTx) with proper
    sql.NullTime + JSONB handling. scanACMEOrder /
    scanACMEAuthz / scanACMEChallenge helpers.
  - internal/service/acme.go: extended ACMERepo interface; new
    SetIssuancePipeline wires certificateService + certificateRepo +
    issuerRegistry. CreateOrder (auth-mode-dispatched: trust_authenticated
    auto-marks order ready + authz valid + 1 placeholder http-01
    challenge valid; challenge mode keeps everything pending). LookupOrder
    (with account-ownership assertion). LookupAuthz. ListAuthzsByOrder.
    FinalizeOrder (3-step atomicity boundary as above; CSR-vs-order
    SAN strict-equality check before issuance; persists FinalizeOrderResult
    {Order, CertID}). LookupCertificate. randIDSuffix + base32encode
    helpers for the human-readable acme-ord-* / acme-authz-* /
    acme-chall-* prefixes (CLAUDE.md "TEXT primary keys with human-
    readable prefixes" architecture decision). 8 new per-op metrics.
  - internal/service/acme_test.go: extended fakeACMERepo with Phase 2
    interface stubs; new orderTrackingRepo for observable persistence;
    2 new tests asserting trust_authenticated → auto-ready/valid and
    challenge → stays-pending.
  - internal/api/handler/acme.go: NewOrder + Order + OrderFinalize +
    Authz + Cert handler methods. orderURL / authzURL / certURL /
    challengeURLBuilder helpers; marshalOrderForResponse fetches
    per-order authzs to populate the URL list. parseOptionalTime for
    notBefore / notAfter.
  - internal/api/handler/acme_handler_test.go: extended mockACMEService
    with Phase 2 method stubs; 4 new handler tests (NewOrder happy +
    rejected-identifier + OrderFinalize bad-CSR + Cert happy).
  - internal/api/router/router.go: 10 new Register calls (5 per-profile
    + 5 shorthand) for new-order, order/{ord_id}, order/{ord_id}/finalize,
    authz/{authz_id}, cert/{cert_id}.
  - internal/api/router/openapi_parity_test.go + api/openapi-handler-exceptions.yaml:
    10 new exception entries.
  - cmd/server/main.go: SetIssuancePipeline at startup, threading
    certificateService + certificateRepo + issuerRegistry into ACMEService.
  - docs/acme-server.md: phase status updated; endpoints table grows
    5 rows for new-order/order/finalize/authz/cert (per-profile +
    shorthand variants); new section "Finalize routing through
    CertificateService.Create" documenting the 3-step atomicity
    boundary + the actor-string convention `acme:<account-id>`.

Tests: ACME package + service + handler + router + config + domain
all green under -short. New cases:
  - TestCreateOrder_TrustAuthenticated_AutoReady (asserts auto-ready
    transition + valid-status authz/challenge + audit row + metric bump).
  - TestCreateOrder_ChallengeMode_StaysPending (asserts pending-status
    cascading authz/challenge for challenge mode).
  - TestACMEHandler_NewOrder_HappyPath (asserts 201 + Location +
    finalize URL shape).
  - TestACMEHandler_NewOrder_RejectedIdentifier (asserts 400 + RFC 7807
    rejectedIdentifier + per-identifier subproblems for type=ip).
  - TestACMEHandler_OrderFinalize_BadCSR (asserts 400 + badCSR for
    non-base64 CSR field).
  - TestACMEHandler_Cert_HappyPath (asserts 200 + PEM content-type +
    PEM chain in body).

Engineering history: cowork/WORKSPACE-CHANGELOG.md "ACME-Server-2".
This commit is contained in:
shankar0123
2026-05-03 13:46:10 +00:00
parent a05a7d3dad
commit c351bba41a
15 changed files with 2179 additions and 28 deletions
+353
View File
@@ -6,6 +6,7 @@ package postgres
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"time"
@@ -232,6 +233,358 @@ func (r *ACMERepository) UpdateAccountStatusWithTx(ctx context.Context, q reposi
return nil
}
// --- Phase 2 — order / authz / challenge CRUD --------------------------
// CreateOrderWithTx inserts an acme_orders row. Used by ACMEService.CreateOrder
// during new-order processing; the transaction also creates the per-identifier
// authz + challenge rows + audit row in the same WithinTx.
func (r *ACMERepository) CreateOrderWithTx(ctx context.Context, q repository.Querier, order *domain.ACMEOrder) error {
if order.OrderID == "" || order.AccountID == "" {
return fmt.Errorf("acme: create order: missing required field")
}
now := time.Now().UTC()
if order.CreatedAt.IsZero() {
order.CreatedAt = now
}
order.UpdatedAt = now
identifiersJSON, err := jsonMarshalACME(order.Identifiers)
if err != nil {
return fmt.Errorf("acme: marshal identifiers: %w", err)
}
var (
notBefore, notAfter interface{}
errBlob interface{}
certID interface{}
)
if order.NotBefore != nil {
notBefore = *order.NotBefore
}
if order.NotAfter != nil {
notAfter = *order.NotAfter
}
if order.Error != nil {
b, err := jsonMarshalACME(order.Error)
if err != nil {
return fmt.Errorf("acme: marshal error: %w", err)
}
errBlob = b
}
if order.CertificateID != "" {
certID = order.CertificateID
}
_, err = q.ExecContext(ctx, `
INSERT INTO acme_orders (
order_id, account_id, identifiers, status, expires_at,
not_before, not_after, error, csr_pem, certificate_id,
created_at, updated_at
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12)
`,
order.OrderID, order.AccountID, identifiersJSON, string(order.Status),
order.ExpiresAt, notBefore, notAfter, errBlob, order.CSRPEM, certID,
order.CreatedAt, order.UpdatedAt,
)
if err != nil {
return fmt.Errorf("acme: insert order: %w", err)
}
return nil
}
// GetOrderByID retrieves an order. Returns sql.ErrNoRows-wrapped
// repository.ErrNotFound on miss.
func (r *ACMERepository) GetOrderByID(ctx context.Context, orderID string) (*domain.ACMEOrder, error) {
row := r.db.QueryRowContext(ctx, `
SELECT order_id, account_id, identifiers, status, expires_at,
not_before, not_after, error, COALESCE(csr_pem, ''),
COALESCE(certificate_id, ''), created_at, updated_at
FROM acme_orders
WHERE order_id = $1
`, orderID)
return scanACMEOrder(row)
}
// UpdateOrderWithTx persists changes to an order's mutable fields.
// Used by FinalizeOrder to transition status / set csr_pem /
// certificate_id / error.
func (r *ACMERepository) UpdateOrderWithTx(ctx context.Context, q repository.Querier, order *domain.ACMEOrder) error {
order.UpdatedAt = time.Now().UTC()
var (
errBlob interface{}
certID interface{}
)
if order.Error != nil {
b, err := jsonMarshalACME(order.Error)
if err != nil {
return fmt.Errorf("acme: marshal error: %w", err)
}
errBlob = b
}
if order.CertificateID != "" {
certID = order.CertificateID
}
res, err := q.ExecContext(ctx, `
UPDATE acme_orders SET
status = $2,
error = $3,
csr_pem = $4,
certificate_id = $5,
updated_at = $6
WHERE order_id = $1
`, order.OrderID, string(order.Status), errBlob, order.CSRPEM, certID, order.UpdatedAt)
if err != nil {
return fmt.Errorf("acme: update order: %w", err)
}
n, err := res.RowsAffected()
if err != nil {
return fmt.Errorf("acme: update order rows: %w", err)
}
if n == 0 {
return fmt.Errorf("order not found: %w", repository.ErrNotFound)
}
return nil
}
// CreateAuthzWithTx inserts an acme_authorizations row.
func (r *ACMERepository) CreateAuthzWithTx(ctx context.Context, q repository.Querier, authz *domain.ACMEAuthorization) error {
if authz.AuthzID == "" || authz.OrderID == "" {
return fmt.Errorf("acme: create authz: missing required field")
}
now := time.Now().UTC()
if authz.CreatedAt.IsZero() {
authz.CreatedAt = now
}
authz.UpdatedAt = now
idJSON, err := jsonMarshalACME(authz.Identifier)
if err != nil {
return fmt.Errorf("acme: marshal identifier: %w", err)
}
_, err = q.ExecContext(ctx, `
INSERT INTO acme_authorizations (
authz_id, order_id, identifier, status, expires_at, wildcard,
created_at, updated_at
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
`,
authz.AuthzID, authz.OrderID, idJSON, string(authz.Status),
authz.ExpiresAt, authz.Wildcard, authz.CreatedAt, authz.UpdatedAt,
)
if err != nil {
return fmt.Errorf("acme: insert authz: %w", err)
}
return nil
}
// GetAuthzByID returns the authz row + its child challenges.
func (r *ACMERepository) GetAuthzByID(ctx context.Context, authzID string) (*domain.ACMEAuthorization, error) {
row := r.db.QueryRowContext(ctx, `
SELECT authz_id, order_id, identifier, status, expires_at, wildcard, created_at, updated_at
FROM acme_authorizations WHERE authz_id = $1
`, authzID)
authz, err := scanACMEAuthz(row)
if err != nil {
return nil, err
}
chs, err := r.ListChallengesByAuthz(ctx, authzID)
if err != nil {
return nil, fmt.Errorf("acme: list challenges: %w", err)
}
authz.Challenges = chs
return authz, nil
}
// ListAuthzsByOrder returns the per-order authz rows (without challenges
// — callers needing challenges call GetAuthzByID per entry).
func (r *ACMERepository) ListAuthzsByOrder(ctx context.Context, orderID string) ([]*domain.ACMEAuthorization, error) {
rows, err := r.db.QueryContext(ctx, `
SELECT authz_id, order_id, identifier, status, expires_at, wildcard, created_at, updated_at
FROM acme_authorizations WHERE order_id = $1
ORDER BY created_at ASC
`, orderID)
if err != nil {
return nil, fmt.Errorf("acme: list authzs: %w", err)
}
defer rows.Close()
var out []*domain.ACMEAuthorization
for rows.Next() {
a, err := scanACMEAuthz(rows)
if err != nil {
return nil, err
}
out = append(out, a)
}
return out, rows.Err()
}
// CreateChallengeWithTx inserts an acme_challenges row.
func (r *ACMERepository) CreateChallengeWithTx(ctx context.Context, q repository.Querier, ch *domain.ACMEChallenge) error {
if ch.ChallengeID == "" || ch.AuthzID == "" || ch.Type == "" || ch.Token == "" {
return fmt.Errorf("acme: create challenge: missing required field")
}
if ch.CreatedAt.IsZero() {
ch.CreatedAt = time.Now().UTC()
}
var (
validatedAt interface{}
errBlob interface{}
)
if ch.ValidatedAt != nil {
validatedAt = *ch.ValidatedAt
}
if ch.Error != nil {
b, err := jsonMarshalACME(ch.Error)
if err != nil {
return fmt.Errorf("acme: marshal error: %w", err)
}
errBlob = b
}
_, err := q.ExecContext(ctx, `
INSERT INTO acme_challenges (
challenge_id, authz_id, type, status, token, validated_at, error, created_at
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
`,
ch.ChallengeID, ch.AuthzID, string(ch.Type), string(ch.Status), ch.Token,
validatedAt, errBlob, ch.CreatedAt,
)
if err != nil {
return fmt.Errorf("acme: insert challenge: %w", err)
}
return nil
}
// ListChallengesByAuthz returns the challenge rows for an authz.
func (r *ACMERepository) ListChallengesByAuthz(ctx context.Context, authzID string) ([]domain.ACMEChallenge, error) {
rows, err := r.db.QueryContext(ctx, `
SELECT challenge_id, authz_id, type, status, token, validated_at, error, created_at
FROM acme_challenges WHERE authz_id = $1
ORDER BY created_at ASC
`, authzID)
if err != nil {
return nil, fmt.Errorf("acme: list challenges: %w", err)
}
defer rows.Close()
var out []domain.ACMEChallenge
for rows.Next() {
ch, err := scanACMEChallenge(rows)
if err != nil {
return nil, err
}
out = append(out, *ch)
}
return out, rows.Err()
}
// scanACMEOrder parses an acme_orders row.
func scanACMEOrder(row interface{ Scan(...interface{}) error }) (*domain.ACMEOrder, error) {
var (
o domain.ACMEOrder
identifiers []byte
statusStr string
notBefore sql.NullTime
notAfter sql.NullTime
errBlob sql.NullString
)
err := row.Scan(
&o.OrderID, &o.AccountID, &identifiers, &statusStr, &o.ExpiresAt,
&notBefore, &notAfter, &errBlob, &o.CSRPEM, &o.CertificateID,
&o.CreatedAt, &o.UpdatedAt,
)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, fmt.Errorf("order not found: %w", repository.ErrNotFound)
}
return nil, fmt.Errorf("acme: scan order: %w", err)
}
o.Status = domain.ACMEOrderStatus(statusStr)
if err := jsonUnmarshalACME(identifiers, &o.Identifiers); err != nil {
return nil, fmt.Errorf("acme: unmarshal identifiers: %w", err)
}
if notBefore.Valid {
t := notBefore.Time
o.NotBefore = &t
}
if notAfter.Valid {
t := notAfter.Time
o.NotAfter = &t
}
if errBlob.Valid && errBlob.String != "" {
var p domain.ACMEProblem
if err := jsonUnmarshalACME([]byte(errBlob.String), &p); err == nil {
o.Error = &p
}
}
return &o, nil
}
// scanACMEAuthz parses an acme_authorizations row.
func scanACMEAuthz(row interface{ Scan(...interface{}) error }) (*domain.ACMEAuthorization, error) {
var (
a domain.ACMEAuthorization
identifier []byte
statusStr string
)
err := row.Scan(
&a.AuthzID, &a.OrderID, &identifier, &statusStr,
&a.ExpiresAt, &a.Wildcard, &a.CreatedAt, &a.UpdatedAt,
)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, fmt.Errorf("authz not found: %w", repository.ErrNotFound)
}
return nil, fmt.Errorf("acme: scan authz: %w", err)
}
a.Status = domain.ACMEAuthzStatus(statusStr)
if err := jsonUnmarshalACME(identifier, &a.Identifier); err != nil {
return nil, fmt.Errorf("acme: unmarshal authz identifier: %w", err)
}
return &a, nil
}
// scanACMEChallenge parses an acme_challenges row.
func scanACMEChallenge(row interface{ Scan(...interface{}) error }) (*domain.ACMEChallenge, error) {
var (
ch domain.ACMEChallenge
typeStr string
statusStr string
validatedAt sql.NullTime
errBlob sql.NullString
)
err := row.Scan(
&ch.ChallengeID, &ch.AuthzID, &typeStr, &statusStr,
&ch.Token, &validatedAt, &errBlob, &ch.CreatedAt,
)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil, fmt.Errorf("challenge not found: %w", repository.ErrNotFound)
}
return nil, fmt.Errorf("acme: scan challenge: %w", err)
}
ch.Type = domain.ACMEChallengeType(typeStr)
ch.Status = domain.ACMEChallengeStatus(statusStr)
if validatedAt.Valid {
t := validatedAt.Time
ch.ValidatedAt = &t
}
if errBlob.Valid && errBlob.String != "" {
var p domain.ACMEProblem
if err := jsonUnmarshalACME([]byte(errBlob.String), &p); err == nil {
ch.Error = &p
}
}
return &ch, nil
}
// jsonMarshalACME wraps encoding/json.Marshal — kept as a named
// helper so future per-column custom encoding is a one-line change.
func jsonMarshalACME(v interface{}) ([]byte, error) { return json.Marshal(v) }
// jsonUnmarshalACME tolerates empty input as a no-op (NULL JSONB
// columns scan into zero-length []byte; we treat that as "field
// absent" rather than an unmarshal error).
func jsonUnmarshalACME(data []byte, v interface{}) error {
if len(data) == 0 {
return nil
}
return json.Unmarshal(data, v)
}
// scanACMEAccount is the shared shape for the SELECT-by-X account
// queries above. Returns sql.ErrNoRows-wrapped repository.ErrNotFound
// on miss; any other scan failure surfaces verbatim.
+3
View File
@@ -28,6 +28,7 @@ func (r *ProfileRepository) List(ctx context.Context) ([]*domain.CertificateProf
SELECT id, name, description, allowed_key_algorithms, max_ttl_seconds,
allowed_ekus, required_san_patterns, spiffe_uri_pattern,
allow_short_lived, must_staple, required_csr_attributes,
COALESCE(acme_auth_mode, 'trust_authenticated'),
enabled, created_at, updated_at
FROM certificate_profiles
ORDER BY created_at DESC
@@ -59,6 +60,7 @@ func (r *ProfileRepository) Get(ctx context.Context, id string) (*domain.Certifi
SELECT id, name, description, allowed_key_algorithms, max_ttl_seconds,
allowed_ekus, required_san_patterns, spiffe_uri_pattern,
allow_short_lived, must_staple, required_csr_attributes,
COALESCE(acme_auth_mode, 'trust_authenticated'),
enabled, created_at, updated_at
FROM certificate_profiles
WHERE id = $1
@@ -213,6 +215,7 @@ func scanProfile(scanner interface {
&p.ID, &p.Name, &p.Description, &algJSON, &p.MaxTTLSeconds,
&ekuJSON, &sanJSON, &p.SPIFFEURIPattern,
&p.AllowShortLived, &p.MustStaple, &csrAttrsJSON,
&p.ACMEAuthMode,
&p.Enabled, &p.CreatedAt, &p.UpdatedAt,
)
if err != nil {