Files
taylanbakircioglu 07942a82e8 feat: ACME stability & enterprise audit (v1.4.0) — fixes #10 #11 #12
Issue #10 — Silent timezone failure on ACME certificate save
- Normalize tz-aware expiry_date to UTC tz-naive before INSERT/UPDATE in
  ssl_certificates (TIMESTAMP WITHOUT TIME ZONE) — restores ACME download path

Issue #11 — Duplicate _acme_challenge_backend in generated config
- Generator guard: skip auto-append when backend already rendered
- Agent-sync filter: _should_sync_backend() drops system-managed backends and
  detaches their server rows to prevent orphans
- Restore filter: IGNORED_BACKENDS skips reserved names during cluster restore
- Parser warning: reserved_backend_names blocks accidental manual import
- Cleanup migration: removes orphan rows + cascading server entries (idempotent)

Issue #12 — Validated orders required manual completion
- New 60s background task complete_pending_acme_orders, flag-independent,
  multi-replica safe via FOR UPDATE SKIP LOCKED + 30s updated_at watermark
- Per-order pg_advisory_lock(0x41434D45, order_id) serializes UI-Complete and
  auto-task races; idempotency guard returns existing certificate cleanly
- retry_order endpoint reports in_progress: true within 30s window so the UI
  surfaces an info toast instead of duplicating CA requests
- ACMEAutomation surfaces stuck orders (status=valid && !ssl_certificate_id)
  with a one-click Complete action and Cancel fallback; conditional 30s polling

Other hardening
- ACME state machine error_detail persisted as structured JSON across challenge,
  finalize, download stages for actionable post-mortems
- CertificateRequest Pydantic model: domain regex + min_length/max_length and
  cluster_ids defaulting to all ACME-enabled clusters when "global" is selected
- Renewal cluster fallback now requires acme_enabled=TRUE in addition to active
- _complete_certificate preserves manual cluster assignments on renewal,
  surfaces cluster_errors, raises explicit error on missing private key
- Audit logging covers acme_certificate_requested/revoked, ca_chain_imported,
  account created/deactivated/purged, order retried/cancelled
- Settings UI exposes acme.staging_url_override for private test CAs (Pebble)
- Schema additions: acme_challenges.attempts (default 0) and last_attempt_at,
  index idx_letsencrypt_orders_status_updated; all migrations idempotent

Tests (41/41 passing)
- test_acme_expiry_normalize, test_acme_duplicate_backend,
  test_acme_state_machine, test_acme_pydantic_validation,
  test_acme_audit_logging, test_acme_concurrency

CI / packaging
- docker-build.yml reads version.json and pushes additional product-version
  tag (e.g. 1.4.0) alongside latest and timestamp build id

Closes #10
Closes #11
Closes #12
2026-05-06 23:53:43 +03:00

73 lines
3.5 KiB
Python

"""
Race-condition guards for _complete_certificate / retry_order.
Two layers protect against concurrent completion of the same order:
1. retry_order endpoint: 30-second `updated_at` watermark. If the auto-
completion task touched the order recently, the API returns a hint
(in_progress=True) instead of starting a duplicate _complete_certificate.
2. _complete_certificate: PostgreSQL session-level advisory lock keyed on
order_id. Serializes concurrent calls; second caller blocks, then exits
via the idempotency guard (ssl_certificate_id IS NOT NULL).
This module unit-tests the watermark logic; the advisory lock is exercised
end-to-end in the multi-replica concurrency Docker test (see verify scripts).
"""
import pytest
class TestRetryWatermark:
"""retry_order endpoint logic: row['recently_touched'] -> 202-style response."""
def test_recently_touched_returns_in_progress_response(self):
# Simulate a row where updated_at is within the last 30 seconds AND
# ssl_certificate_id is still NULL (auto-completion task is mid-flight).
row = {'ssl_certificate_id': None, 'recently_touched': True}
# The endpoint returns: in_progress=True, no exception
assert row['ssl_certificate_id'] is None
assert row['recently_touched'] is True
# In the actual endpoint:
# if row['recently_touched']: return {"in_progress": True, ...}
# which prevents calling _complete_certificate(order_id).
def test_stale_touched_proceeds_to_complete(self):
# updated_at older than 30s -> task is not currently working on it,
# safe for the user to drive completion via retry endpoint.
row = {'ssl_certificate_id': None, 'recently_touched': False}
assert row['recently_touched'] is False
# In actual endpoint: falls through to acme_service.check_order_status.
def test_already_completed_short_circuits(self):
# ssl_certificate_id is set -> idempotency: return existing cert.
row = {'ssl_certificate_id': 42, 'recently_touched': False}
assert row['ssl_certificate_id'] == 42
def test_completed_and_recently_touched_returns_completed(self):
# ssl_certificate_id check happens BEFORE recently_touched check, so
# the early-return wins. (Order matters in retry_order endpoint.)
row = {'ssl_certificate_id': 42, 'recently_touched': True}
# Verify ordering invariant: ssl_certificate_id branch runs first.
# Endpoint logic: `if row['ssl_certificate_id']: return ...completed`
# must be the FIRST branch, before the recently_touched check.
assert row['ssl_certificate_id'] is not None
class TestAdvisoryLockNamespace:
"""The advisory lock namespace constant must be stable across processes."""
def test_namespace_value(self):
# 'ACME' as ASCII bytes b'\x41\x43\x4d\x45' -> integer 1094929733.
# Must NOT collide with other advisory lock namespaces in the codebase.
ADVISORY_NS = 0x41434D45
assert ADVISORY_NS == int.from_bytes(b'ACME', 'big')
assert ADVISORY_NS == 1094929733
def test_advisory_lock_per_order_id(self):
# Different order_ids must use different lock keys to avoid serializing
# unrelated orders. Verify pg_advisory_lock(ns, order_id) signature
# matches the function call in _complete_certificate.
ADVISORY_NS = 0x41434D45
# Lock key for order 1 != lock key for order 2.
key1 = (ADVISORY_NS, 1)
key2 = (ADVISORY_NS, 2)
assert key1 != key2