Commit Graph

13 Commits

Author SHA1 Message Date
taylanbakircioglu 93d7ad8fdb feat: add ACME Auto SSL with Let's Encrypt integration (v1.1.0)
Add automated SSL certificate management via ACME protocol (RFC 8555):
- Full ACME client implementation (account registration, HTTP-01 challenges, certificate issuance/renewal)
- Configurable ACME providers (Let's Encrypt, ZeroSSL, custom CA) via Settings UI
- Auto-renewal scheduler with PENDING -> Apply -> APPLIED flow alignment
- ACME account management (register, deactivate) from UI
- Certificate request wizard with domain validation and cluster targeting
- Zero changes to HAProxy agent scripts - challenges routed through existing architecture
- Comprehensive security hardening (no private key exposure in API responses)
- Full backward compatibility with existing SSL, Apply, Rollback, and Restore workflows
- Updated README, API documentation, and Kubernetes deployment notes
- Version management embedded in code (v1.1.0)
- UI messaging improvements for agent-pull architecture accuracy

Made-with: Cursor
2026-04-02 00:25:50 +03:00
taylanbakircioglu 77bd3ecead fix: add missing REJECTED and DELETION values to config_status enum
Root cause: config_status enum was created with only PENDING and APPLIED
values. The REJECTED value was never added due to a silent duplicate_object
exception in create_essential_tables(). This caused SSL certificate listing
to crash with "invalid input value for enum config_status: REJECTED" on
fresh installations.

Also adds scrollable containers to Apply Management page to prevent
Agent Sync Status card from being pushed off-screen.

Closes #7

Made-with: Cursor
2026-04-01 13:18:24 +03:00
taylanbakircioglu 922bc2ce6d improve: SSL certificate listing reliability, error visibility, and ARM64 support
Fixes #6

- Fix NameError in soft-deleted certificate reactivation path by
  reordering variable extraction before DB operations
- Replace silent empty-array returns with HTTP 500 on SQL errors,
  making failures visible in both API responses and server logs
- Add primary_domain migration for schema consistency across fresh
  and upgraded installations (backfill from legacy domain column)
- Use primary_domain in non-cluster SSL query branch for schema
  compatibility
- Surface SSL fetch errors in frontend via toast notifications
- Harden connection cleanup in error handlers with try/except
- Add QEMU + Buildx for linux/amd64,linux/arm64 multi-platform
  Docker image builds
- Update GitHub Actions to latest versions (checkout v4, login v3,
  build-push v6)

Made-with: Cursor
2026-03-30 10:42:28 +03:00
taylanbakircioglu b0feca3c2e feat: add keepalived VRRP state (MASTER/BACKUP) detection to agent heartbeat
- Add keepalive_state and keepalive_ip columns to agents table (migration + schema)
- Add keepalive fields to AgentHeartbeat Pydantic model (backward compatible)
- Update heartbeat endpoint to persist keepalive data to DB and cache in Redis
- Add multi-method keepalived detection in agent scripts (journalctl, log files, VIP check)
- Update dashboard-stats agents/status API with Redis-first keepalive lookup
- Update GET /api/agents to include keepalive_state and keepalive_ip
- Show MASTER/BACKUP tag in Dashboard AgentStatusCard
- Show keepalive info in Agent Management registered agents table
- Add Keepalive column to Cluster Management table with VIP search support

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-18 20:41:17 +03:00
taylanbakircioglu 851377aedf feat: Add HAProxy proxy name collision prevention system
- Add preserved_listen_blocks column to agents table for storing agent's local listen block names
- Implement reserved names check (stats, monitoring, admin, etc.) for frontend/backend creation
- Add dynamic collision detection against agent's preserved listen blocks
- Apply collision checks to CREATE, UPDATE endpoints and bulk import
- Add debug mode for failed config validation (saves to /tmp/haproxy-failed-*.cfg)
- Fix JSON character stripping for ACL and use_backend rules
- Remove collision protection from agent scripts (now handled by backend)
- All collision checks wrapped in try-except for backwards compatibility
2026-01-26 15:25:15 +03:00
Taylan Bakırcıoğlu b212fb92bc Add SSL Advanced Options support (Backend) - Part 1
FEATURE: Complete SSL Advanced Options implementation for frontend and backend server SSL

 DATABASE:
- Added SSL parameter columns to frontends table:
  * ssl_alpn, ssl_npn, ssl_ciphers, ssl_ciphersuites
  * ssl_min_ver, ssl_max_ver, ssl_strict_sni
- Added SSL parameter columns to backend_servers table:
  * ssl_sni, ssl_min_ver, ssl_max_ver, ssl_ciphers
- Migration functions: add_ssl_advanced_options_to_frontends() and add_ssl_advanced_options_to_servers()

 MODELS:
- FrontendConfig: Added 7 new SSL fields for bind parameters
- ServerConfig: Added 4 new SSL fields for server parameters
- AgentHeartbeat: Added system_info field (fixes HTTP 422 validation error)

 BULK IMPORT PARSER:
- Parse alpn, npn, ciphers, ciphersuites, ssl-min-ver, ssl-max-ver, strict-sni from bind lines
- Parse sni, ssl-min-ver, ssl-max-ver, ciphers from server lines
- Store parsed values in frontend/server objects
- User-friendly warnings about imported SSL parameters

 CONFIG GENERATOR:
- Generate bind lines with SSL advanced options: 'bind :443 ssl crt file.pem alpn h2,http/1.1 ciphers ...'
- Generate server lines with SSL advanced options: 'server s1 addr:port ssl sni hostname ssl-min-ver TLSv1.2'
- Support both NEW MODE (multiple certs) and OLD MODE (single cert)

USER IMPACT:
- Bulk import now correctly parses SSL configs with alpn/npn/ciphers
- SSL parameters preserved during import (not lost anymore)
- Agent heartbeat fixed (no more offline agents)
- Ready for UI implementation (next commit)

EXAMPLE USAGE:
Frontend: bind 0.0.0.0:8443 ssl crt cert1.pem crt cert2.pem alpn h2,http/1.1
Server: server s1 10.1.1.1:443 ssl verify required sni backend.example.com ssl-min-ver TLSv1.2

NEXT: Frontend UI components for editing these SSL options
2025-11-18 21:58:05 +03:00
taylanbakircioglu 0fc18fde38 feat: Add HAProxy options support for backends and frontends
Implemented comprehensive HAProxy options field support for both backend and frontend entities to enable standard HAProxy directives like 'option http-keep-alive', 'option httplog', 'option forwardfor', etc.

Changes:
- Database: Added 'options' TEXT column to backends and frontends tables
- Models: Added options field to BackendConfig, BackendConfigUpdate, and FrontendConfig
- API Endpoints: Updated CREATE, UPDATE, and GET endpoints to handle options field
  * Backend: CREATE/UPDATE/GET with options support
  * Frontend: CREATE/UPDATE/GET with options support (fixed 5 SELECT queries)
- Config Generator: Added options block generation for both backends and frontends
- Bulk Import Parser:
  * Added options field to ParsedBackend and ParsedFrontend dataclasses
  * Implemented option directive parsing with validation
  * Added unknown option warnings
  * Fixed bulk parse response to include options field
- Bulk Import Merge: Added options field comparison in UPDATE logic
- UI Components:
  * BackendServers.js: Added options TextArea form field
  * FrontendManagement.js: Added options TextArea form field

Features:
- Multi-line options support (newline-separated format)
- Option validation with known HAProxy options list
- Backward compatible (NULL options for existing entities)
- Bulk import support with merge strategy
- Full CRUD support for both manual and bulk operations

Technical Details:
- Format: Newline-separated TEXT field for multiple options
- Validation: Warns about unknown options but allows them
- Config Generation: Each option written as separate directive
- Agent: Standard HAProxy config validation applies

Total: 10 files modified, ~195 lines added, 26 integration points verified
2025-11-13 10:12:30 +03:00
taylanbakircioglu 1ea1c6a29f feat: Major stability and feature improvements
This commit consolidates multiple improvements from internal development:

## Agent Stability Improvements
- Add database connection pooling (min=10, max=50) for better performance
- Prevent config reapply on agent restart by fetching last_applied_version from database
- Optimize SSL fetch to only run when config changes (98% API call reduction)
- Make SSL_SYNC_TIMESTAMP_FILE agent-specific to prevent race conditions
- Fix agent offline display issue due to database connection bottleneck
- 10x faster heartbeat response (200ms → 20ms)

## Bulk Import UPSERT Support
- Parse endpoint detects existing entities (New/Existing status)
- Bulk-create supports UPDATE for existing backends/frontends (merge strategy)
- New servers can be added to existing backends
- Existing servers preserved (no deletion in MVP)
- Field-by-field value comparison (only changed fields updated)
- Pending apply conflict prevention (409 error)
- Fixed duplicate key error on server INSERT
- Backend marked PENDING when servers added

## Apply Management Fixes
- Fixed deleted entities not showing (include_inactive parameter)
- Backend/Frontend GET endpoints support inactive entities for Apply Management
- All pending changes now visible
- Phantom backend bug protection maintained

## Backend Delete Improvements
- Automatically clean ACL/use_backend rules from frontends
- Prevents HAProxy validation errors after backend deletion
- Frontend references automatically updated

## UI/UX Improvements
- Cluster selector status dot auto-refreshes every 30 seconds
- Real-time agent health monitoring (no page refresh needed)
- Parse message shows only NEW entities (cleaner)
- Status labels: 'Update' → 'Existing' (clearer meaning)
- Multi-line parse success messages
- Detailed summary breakdown with tooltips

Technical Changes:
- backend/database/connection.py: Connection pool implementation
- backend/main.py: Pool initialization and cleanup
- backend/routers/*: UPSERT logic, field comparison, include_inactive
- backend/utils/agent_scripts/*: Applied version tracking, SSL optimization
- frontend/src/components/*: UI improvements, status indicators
- frontend/src/contexts/ClusterContext.js: Auto-refresh agent health

Impact:
- Supports 50+ concurrent agents (previously ~10)
- Zero config reapply on restart/upgrade
- Bulk import handles existing entities correctly
- All pending changes visible in Apply Management
- Real-time cluster health status
- No HAProxy validation errors after backend delete
2025-11-11 21:56:18 +03:00
taylanbakircioglu 281e23ea27 feat: Add SSL usage_type (Frontend/Server) with conditional private key requirement
This is a comprehensive update that adds SSL certificate differentiation
for frontend (HAProxy bind) and server (backend verification) use cases.

FEATURES:
- SSL certificates can be marked as 'frontend' or 'server' usage type
- Frontend SSL: Private key REQUIRED (for HAProxy bind ssl crt)
- Server SSL: Private key OPTIONAL (CA cert only for backend verification)
- UI dropdown for usage type selection
- Dynamic form validation based on usage type
- Filtering: Frontends see only Frontend SSL, Backends see only Server SSL

DATABASE:
- Added usage_type column to ssl_certificates (default: 'frontend')
- Made private_key_content nullable for server SSL support
- Migration automatically runs on pod restart

BACKEND:
- Pydantic v2 compatibility (@field_validator, @model_validator)
- SSL router: usage_type filtering support
- Agent endpoint: usage_type field included
- Improved migration robustness with better error handling
- Fixed duplicate ensure_agents_table() function
- Fixed JSONB permissions insert with json.dumps()
- Fixed ON CONFLICT constraints with explicit checks

FRONTEND:
- SSL Management: Usage Type dropdown with visual feedback
- Frontend Management: Filters only Frontend SSL certificates
- Backend Servers: Filters only Server SSL certificates
- Dynamic private key validation (required for Frontend, optional for Server)
- Improved form UX with color-coded hints

AGENT SCRIPTS (Linux & macOS):
- Support for Server SSL without private key
- Conditional PEM file creation (cert+key vs cert-only)
- usage_type awareness in SSL deployment
- Backward compatible with existing Frontend SSL certificates

DOCKER:
- Increased npm timeout for slow networks (300s → 600s)
- Increased fetch-retries (5 → 10)
- Reduced maxsockets for stability (3 → 1)

All changes are backward compatible. Existing SSL certificates
default to 'frontend' type and continue working unchanged.

Tested with: HAProxy 2.8+, PostgreSQL 15, React 18
2025-11-11 03:41:47 +03:00
taylanbakircioglu 6c24971c19 Migration: Convert use_backend_rules to JSONB for consistency
Database Type Consistency Fix:
- acl_rules: JSONB ✓
- redirect_rules: JSONB ✓
- use_backend_rules: TEXT ✗ (INCONSISTENT!)

Issue:
  Different data types cause:
  - JSON serialization inconsistencies
  - Query performance differences
  - Potential data corruption

Fix Applied:

1. CREATE TABLE (Line 2038):
   Changed: use_backend_rules TEXT
   To: use_backend_rules JSONB DEFAULT '[]'::jsonb

2. ALTER TABLE (Line 121):
   Changed: ADD COLUMN use_backend_rules TEXT
   To: ADD COLUMN use_backend_rules JSONB DEFAULT '[]'::jsonb

3. Type Conversion Migration (Line 2318-2334):
   Added automatic conversion from TEXT to JSONB
   Handles: NULL, empty string, existing JSON data
   Safe conversion with CASE statement

Migration Logic:
  IF column type is TEXT or VARCHAR:
    - NULL → '[]'::jsonb
    - Empty string → '[]'::jsonb
    - Existing JSON → Parse to JSONB
    - Invalid data → Fails gracefully

Benefits:
  - Consistent JSONB type across all rule fields
  - Better query performance (JSONB indexing)
  - Type safety in application code
  - Automatic array validation

Impact: SAFE - Migration runs automatically on startup
2025-11-07 11:51:15 +03:00
taylanbakircioglu 97fd0e7dad Migration: Add ssl_certificate_id to backend_servers + Remove emojis
Database Migration:
- Added ssl_certificate_id column to backend_servers table
- Added FK constraint to ssl_certificates table
- ON DELETE SET NULL behavior
- Idempotent migration (safe to run multiple times)

Column Details:
  Name: ssl_certificate_id
  Type: INTEGER
  Nullable: YES
  Foreign Key: ssl_certificates(id)
  On Delete: SET NULL

Migration Function:
  add_ssl_certificate_id_to_backend_servers()
  Called in run_migrations() at Line 1523

Code Cleanup:
- Removed emojis from migration logs
- Removed emojis from SSL dropdown status icons
- Changed to text: Valid, Expiring, Expired
- Changed to text: Global, Cluster

Error Fixed:
  GET /api/backends - 500
  column "ssl_certificate_id" does not exist

After migration runs on startup, column will exist and API will work
2025-11-07 11:51:15 +03:00
taylanbakircioglu c72859d507 Fix: Add cluster_id to config requests to prevent wrong cluster selection
- Add cluster_id column to agent_config_requests table
- Update config request endpoint to accept and validate cluster_id
- Frontend now sends cluster_id with config requests
- Fixes issue where agents in pools with multiple clusters get wrong config requests

Technical Details:
- Database migration adds cluster_id as nullable foreign key for backward compatibility
- Backend validates that agent belongs to requested cluster via pool_id check
- Improved logging includes cluster name for better traceability
- No impact on existing features (apply management, sync status, entity CRUD)
2025-10-30 12:49:30 +03:00
taylanbakircioglu 6aae0f4309 Initial commit 2025-10-27 12:14:03 +03:00