15 Commits

Author SHA1 Message Date
taylanbakircioglu 02b1cb2bca feat: v1.5.0 — Site Wizard (Issue #14) + ACME Diagnostic Panel (Issue #13)
Closes #13, Closes #14.

This release squashes the v1.4.0 → v1.5.0 development line. v1.4.0
shipped the ACME stability & enterprise audit (Issues #10/#11/#12).
v1.5.0 builds on that foundation with two co-equal headline features
plus a 22-round audit campaign hardening the prior configuration
surface. License remains MIT for v1.5.0 (relicense to AGPL-3.0
lands in v1.5.2).

------------------------------------------------------------------
HEADLINE FEATURE A — ACME Diagnostic Panel (Issue #13)
------------------------------------------------------------------
A live pre-flight + post-failure diagnostic surface for every ACME
order, reachable from the ACME Automation page. The panel exists
to make ACME failures legible to operators who do NOT have shell
access to the API host.

Endpoints (`backend/routers/acme_diagnostics.py`):
  POST /api/letsencrypt/orders/{order_id}/diagnostics
       Run the full 5-check suite (DNS / port-80 / routing /
       account / agents) and humanize the order's `error_detail`
       (>=11 RFC-8555 problem types, backwards compatible with
       legacy plain-string failures).
  POST /api/letsencrypt/orders/{order_id}/diagnostics/
                                {check_id}/rerun
       Re-run a single check in place — used by the "Re-run"
       button on every row of the modal's pre-flight table.
  GET  /api/letsencrypt/orders/{order_id}/events
       Merged event timeline combining the typed
       `acme_order_events` rows with correlated
       `user_activity_logs` entries (resource_type =
       'letsencrypt_order' AND resource_id = order_id). The
       diagnostic modal auto-tails this timeline every 5 seconds
       while open.

Service-level checks (`backend/services/acme_diagnostics.py`):
  * DNS resolution via stdlib socket.gethostbyname_ex through
    run_in_executor (intentionally avoiding an aiodns runtime
    dep for v1.5.0).
  * Port-80 HEAD probe, target locked to the order's domains,
    success on HTTP 200 OR 404, warns on egress timeout
    (corp egress policies routinely blackhole outbound 80 —
    fail-hard would be too noisy).
  * SSRF guard: probe refuses non-public IPs and surfaces the
    skip in the diagnostic result; IPv4-mapped IPv6 normalisation
    closes the `::ffff:169.254.169.254` cloud-metadata vector.
  * HAProxy routing presence check: matches the order's
    cluster_ids to a port-80 HTTP frontend.
  * ACME account validity check against `letsencrypt_accounts`.
  * Agent presence check (>=1 active agent in target cluster).
  * Every sub-check wrapped in a wall-clock timeout to bound
    impact on the API event loop.

RBAC: ssl.read for run, ssl.read for events. Per-user 5/min rate
limit on both run and rerun, backed by the (user_id, action,
created_at DESC) composite index.

Frontend (`frontend/src/components/ACMEAutomation.js`):
  * "Diagnose" button on every order row + the existing
    "stuck order" warning row.
  * Modal with two tabs:
    - Pre-flight Checks (Antd Table with status pills + Re-run
      buttons + humanized error banner)
    - Event Log (Antd Timeline with auto-tail polling, scroll-
      to-bottom, pause-on-hover)
  * Correlation IDs surfaced in error banners and individual
    check fail details for backend-log lookup.

------------------------------------------------------------------
HEADLINE FEATURE B — Site Setup Wizard (Issue #14)
------------------------------------------------------------------
A single guided flow that creates a Backend + Servers + HTTP
Frontend (and optional HTTPS Frontend) in one atomic transaction.

Endpoints (`backend/routers/site_wizard.py`):
  POST /api/site-wizard/preview     — diff-preview the changeset
  POST /api/site-wizard/create      — atomic execute
  POST /api/site-wizard/reject      — clean rollback (including
                                       any wizard_staged ACME
                                       orders)
  GET  /api/site-wizard/drafts      — draft persistence
  PUT  /api/site-wizard/drafts/{id} — save/update
  DELETE /api/site-wizard/drafts/{id}

Feature surface:
  * One screen captures both backend (mode + servers) AND
    frontend (http + optional https + SSL mode) inputs.
  * SSL modes: ACME (new order, HTTP-01 only for v1.5.0),
    Upload (existing PEM), Existing (link to a stored cert),
    or None.
  * ACME-staged path: wizard_staged_until watermark on the
    `letsencrypt_orders` row defers finalisation until agent
    confirmation; per-mode reject cleanly cancels and rolls
    back the staged order.
  * Live diff preview against the cluster's current generated
    config (renderer-evolution noise stripped — track-sc<N>
    dedup, per-server cookie strip, defaults-cookie
    inheritance, listen-block flattening).
  * Draft persistence with PEM stripped at save time (private
    keys never round-trip through the drafts table).
  * Per-cluster multi-tenancy: drafts and wizard_staged orders
    are isolated to the creating user's cluster scope.

Frontend (`frontend/src/components/SiteWizard.js`):
  * 4-step Antd Steps flow: Backend → Frontend → SSL → Review.
  * Render the live diff preview inline before commit.
  * Antd Form-level validation mirrors backend Pydantic
    validators (numeric bounds, HAProxy reserved keywords, ALPN
    consistency, IPv6 scope-id, domain regex, server name
    dedup).

------------------------------------------------------------------
AUDIT CAMPAIGN — Rounds 1 → 22 (Bulgu #1#82)
------------------------------------------------------------------
v1.5.0 includes 22 adversarial review passes. Each round produced
its own commit set in the corporate development line; this squash
collapses those into the v1.5.0 release artefact. Highlights:

  Round 1-4   Site Wizard core: dry-run parity, single-line
              value injection guard, ACL -f pattern-file block,
              SSL parity, timeout regex, form-state pin.
  Round 5-7   defaults-cookie inheritance, server-named-cookie
              guard, fe/be mode mismatch, duplicate server
              names, health_check_uri + server_address
              validators.
  Round 8-10  cookie_name / cookie_options newline-injection
              guard, dry-run parity (round 9), TCP-mode HTTP-only
              feature blockers.
  Round 11    SSL name path traversal + health-check >= 1.
  Round 12-13 SSL & ACME deep dive (Bulgu #23-#32).
  Round 14    single-line value injection (Bulgu #33).
  Round 15-17 ACME multi-tenant UX, numeric bounds, HAProxy
              reserved keywords, ALPN/TLS consistency,
              all-backup, multi-domain & multi-user enterprise
              edges, drain/HSTS/post-completion (Bulgu
              #34-#53).
  Round 18-21 concurrency, agent state, TCP-mode HTTP-only,
              list size caps, IPv6 scope-id, preview account
              validation, TCP backend + balance uri reject
              (Bulgu #54-#61).
  Round 22    FE error visibility + 3x stale-data lockouts,
              referential integrity + cascade safety,
              authentication & authorization, multi-cluster
              isolation, apply_pending_changes concurrency,
              script injection + bulk import multi-tenancy,
              prefix-stripped signature comparison
              (Bulgu #62-#82).

------------------------------------------------------------------
NO CORPORATE-SPECIFIC ARTIFACTS
------------------------------------------------------------------
This squash deliberately sanitises corporate hostnames, container
registry references, and TLS secret names into generic
placeholders (`your-registry.example.com/your-org`,
`haproxy-openmanager*.example.com`, `wildcard-tls`,
`taylanbakircioglu/haproxy-openmanager-*`) so the public artefact
contains no internal infrastructure detail. Pilot / development
history that retained those values stays in the corporate fork
and is NOT part of this commit.
2026-05-14 00:04:19 +03:00
taylanbakircioglu 07942a82e8 feat: ACME stability & enterprise audit (v1.4.0) — fixes #10 #11 #12
Issue #10 — Silent timezone failure on ACME certificate save
- Normalize tz-aware expiry_date to UTC tz-naive before INSERT/UPDATE in
  ssl_certificates (TIMESTAMP WITHOUT TIME ZONE) — restores ACME download path

Issue #11 — Duplicate _acme_challenge_backend in generated config
- Generator guard: skip auto-append when backend already rendered
- Agent-sync filter: _should_sync_backend() drops system-managed backends and
  detaches their server rows to prevent orphans
- Restore filter: IGNORED_BACKENDS skips reserved names during cluster restore
- Parser warning: reserved_backend_names blocks accidental manual import
- Cleanup migration: removes orphan rows + cascading server entries (idempotent)

Issue #12 — Validated orders required manual completion
- New 60s background task complete_pending_acme_orders, flag-independent,
  multi-replica safe via FOR UPDATE SKIP LOCKED + 30s updated_at watermark
- Per-order pg_advisory_lock(0x41434D45, order_id) serializes UI-Complete and
  auto-task races; idempotency guard returns existing certificate cleanly
- retry_order endpoint reports in_progress: true within 30s window so the UI
  surfaces an info toast instead of duplicating CA requests
- ACMEAutomation surfaces stuck orders (status=valid && !ssl_certificate_id)
  with a one-click Complete action and Cancel fallback; conditional 30s polling

Other hardening
- ACME state machine error_detail persisted as structured JSON across challenge,
  finalize, download stages for actionable post-mortems
- CertificateRequest Pydantic model: domain regex + min_length/max_length and
  cluster_ids defaulting to all ACME-enabled clusters when "global" is selected
- Renewal cluster fallback now requires acme_enabled=TRUE in addition to active
- _complete_certificate preserves manual cluster assignments on renewal,
  surfaces cluster_errors, raises explicit error on missing private key
- Audit logging covers acme_certificate_requested/revoked, ca_chain_imported,
  account created/deactivated/purged, order retried/cancelled
- Settings UI exposes acme.staging_url_override for private test CAs (Pebble)
- Schema additions: acme_challenges.attempts (default 0) and last_attempt_at,
  index idx_letsencrypt_orders_status_updated; all migrations idempotent

Tests (41/41 passing)
- test_acme_expiry_normalize, test_acme_duplicate_backend,
  test_acme_state_machine, test_acme_pydantic_validation,
  test_acme_audit_logging, test_acme_concurrency

CI / packaging
- docker-build.yml reads version.json and pushes additional product-version
  tag (e.g. 1.4.0) alongside latest and timestamp build id

Closes #10
Closes #11
Closes #12
2026-05-06 23:53:43 +03:00
taylanbakircioglu b5505b4c9d docs: update README structure and add ACME screenshots
Made-with: Cursor
2026-04-04 17:40:17 +03:00
taylanbakircioglu deb784bb65 fix: ACME certificate issuance improvements, null-safe hardening, guided setup UX, and order list enhancements (Issue #9)
- Fix critical cascading NULL status bug in ACME challenge flow that could cause 404s
- Add defense-in-depth NULL handling across all ACME service methods
- Add new GET /api/letsencrypt/prerequisites endpoint for configuration checks
- Add interactive ACME Setup Guide with step-by-step navigation links
- Add URL-based tab navigation in Settings and SSL Management pages
- Harden retry flow: return clear 409 errors for invalid/cancelled orders
- Allow cancellation of invalid orders (backend + frontend)
- Improve error message extraction with consistent getErrorMsg helper
- Add status filter tabs (Active/Completed/Failed/All) for order list
- Add visual dimming for cancelled/invalid orders
- Add enhanced pagination with size changer and total count
- Add comprehensive diagnostic logging with ACME: prefix
- Update README with ACME architecture docs, quick start guide, and troubleshooting

Resolves #9

Made-with: Cursor
2026-04-04 15:15:05 +03:00
taylanbakircioglu 93d7ad8fdb feat: add ACME Auto SSL with Let's Encrypt integration (v1.1.0)
Add automated SSL certificate management via ACME protocol (RFC 8555):
- Full ACME client implementation (account registration, HTTP-01 challenges, certificate issuance/renewal)
- Configurable ACME providers (Let's Encrypt, ZeroSSL, custom CA) via Settings UI
- Auto-renewal scheduler with PENDING -> Apply -> APPLIED flow alignment
- ACME account management (register, deactivate) from UI
- Certificate request wizard with domain validation and cluster targeting
- Zero changes to HAProxy agent scripts - challenges routed through existing architecture
- Comprehensive security hardening (no private key exposure in API responses)
- Full backward compatibility with existing SSL, Apply, Rollback, and Restore workflows
- Updated README, API documentation, and Kubernetes deployment notes
- Version management embedded in code (v1.1.0)
- UI messaging improvements for agent-pull architecture accuracy

Made-with: Cursor
2026-04-02 00:25:50 +03:00
taylanbakircioglu da64481dd9 docs: add Standalone Installation option to README (fixes #4)
Added standalone installation as a third deployment option alongside
Docker and Kubernetes. The react-scripts 5.0.1 upgrade in commit d4b58d6
resolved the Docker build failure reported in #4.

Made-with: Cursor
2026-03-16 14:47:29 +03:00
taylanbakircioglu c3f93106e2 docs: add Response Time Heatmap, IP Inventory screenshots and update README
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-18 21:09:55 +03:00
taylanbakircioglu 5edf53ec3e Update Docker registry to taylanbakircioglu 2025-12-24 19:11:57 +03:00
taylanbakircioglu c20c40ea4c docs: add HAProxy configuration validation troubleshooting guide to README
- Added detailed troubleshooting section for Apply Changes stuck issues
- Includes step-by-step diagnosis using haproxy -c -f validation
- Common validation errors and solutions
- Recovery steps and prevention tips
- Helps users debug config issues on agent servers
2025-11-14 01:06:38 +03:00
taylanbakircioglu b29c9e044e feat: Multi-cluster isolation and comprehensive bug fixes
This update consolidates bug fixes and improvements from internal development:

## Multi-Cluster Isolation (CRITICAL)
- Backend delete now isolated per cluster (added cluster_id to WHERE clauses)
- Orphan config version auto-detection and cleanup
- Prevents cross-cluster contamination when deleting entities
- REJECTED entities excluded from pending list

## Apply Management Fixes
- Fixed phantom pending entities (REJECTED entities no longer shown)
- Fixed cluster switch 404 errors (state cleared before fetch)
- Fixed page reload not refreshing data (added mount useEffect)
- Orphan entity status auto-cleanup on reject

## Backend Delete Improvements
- Added cluster_id to server/frontend updates (multi-cluster safe)
- Automatic ACL/use_backend cleanup from frontends
- NULL cluster_id support for legacy data
- Prevents HAProxy validation errors

## Orphan Version Detection
- Backend GET: Validates entity belongs to version's cluster
- Frontend GET: Validates entity belongs to version's cluster
- Apply: Auto-detects and removes orphan versions before apply
- Reject: Auto-detects and removes orphan versions before reject

## UI/UX Improvements
- Apply Management state management improved
- Debug logging for troubleshooting
- Better cluster switch handling
- README: Orphan version troubleshooting section

Technical Changes:
- backend/routers/backend.py: Multi-cluster isolation, orphan detection, NULL handling
- backend/routers/frontend.py: Orphan detection, REJECTED filter
- backend/routers/cluster.py: Orphan auto-cleanup, entity status cleanup
- frontend/ApplyManagement.js: State management, mount useEffect
- README.md: Troubleshooting documentation

Impact:
- Multi-cluster environments now fully isolated 
- Orphan config versions automatically cleaned 
- REJECTED entities properly filtered 
- Apply Management works correctly across cluster switches 
- No manual database intervention needed 
2025-11-12 13:34:07 +03:00
taylanbakircioglu 52a17df673 Feature: Smart SSL Auto-Assignment in Bulk Import + Documentation
NEW FEATURE: Smart SSL Auto-Assignment

Automatically assigns SSL certificates during bulk import when:
  - SSL certificates exist in SSL Management
  - Certificate name matches config path
  - Status is SYNCED (deployed to agents)

Backend Implementation (backend/routers/config.py):
  - Query SYNCED SSL certs for cluster (Global + Cluster-specific)
  - Extract SSL names from config paths
  - Auto-match and assign certificate IDs
  - Enhanced warnings with auto-assignment info

Frontend UI (frontend/src/components/BulkConfigImport.js):
  - New green Alert with step-by-step guide
  - Example code snippets
  - Visual certificate name tags
  - Clear workflow explanation

Documentation (README.md):
  - Added to Key Capabilities section
  - Added to Features at a Glance
  - Highlights automation benefit

Example Workflow:
  1. SSL Management: Upload 'demo-global' → Apply → SYNCED
  2. Bulk Import: Config has demo-global.pem
  3. Auto-match: SSL assigned automatically
  4. Result: Frontend created WITH SSL enabled
  5. Benefit: No manual edit needed!

User Benefits:
  - Saves time (no manual SSL assignment)
  - Reduces errors (automatic matching)
  - Better UX (clear guidance)
  - Optional (backward compatible)

Complete Implementation:
  - Backend logic
  - Frontend UI guide
  - Documentation
  - Risk analysis completed
  - All tests passed
2025-11-07 11:51:15 +03:00
taylanbakircioglu eeee768510 docs: Remove redundant Statistics section
- Remove duplicate Statistics section (already covered in Dashboard)
- Update Table of Contents to remove Statistics link
- Keep Statistics & Monitoring in API Reference (different context)
2025-10-31 20:22:48 +03:00
taylanbakircioglu e800d2a156 docs: Complete README overhaul with modern design and structure
Major improvements:
- Add comprehensive Table of Contents (15 sections with clickable links)
- Reorganize structure: Screenshots after Features (better UX)
- Modernize System Architecture diagram (3-layer design, dark mode optimized)
- Simplify Agent Version Update Flow (user vs automatic actions)
- Add visual Installation selection panel (Docker vs Kubernetes)
- Expand Project Structure (150+ files documented)
- Restructure sections: 'Getting Started' (usage) vs 'Installation' (setup)
- Add Role-Based User Management to features
- Remove redundant sections (Advanced Setup, Coming Soon notes)
- Improve all diagrams for readability and modern look
- Fix all Table of Contents anchor links
- Add 18 production screenshots with descriptions
2025-10-31 20:17:38 +03:00
taylanbakircioglu 7f54c40dc3 Update README: Change HAProxy Agent from Python Service to Bash Service 2025-10-27 15:16:03 +03:00
taylanbakircioglu 6aae0f4309 Initial commit 2025-10-27 12:14:03 +03:00