8 Commits

Author	SHA1	Message	Date
mustafa.ulukaya	ef26860df9	feat(logging): unified request/response log with configurable retention (v1.11.0) ... Until now the only record of what happened was `user_activity_logs`, which stores non-GET 2xx operations with no bodies. When something failed you could see that a counter went up, never what was sent or what came back. This adds one queryable timeline covering both directions: - inbound: every API call, including GETs and including 4xx/5xx, with the user, client IP, status, duration and — redacted, size-capped — the request and response bodies. - outbound: every HTTP call the backend makes, tagged with who it went to (ACME/Let's Encrypt, Cloudflare, GoDaddy, HAProxy stats, agents, the ACME diagnostics probe). Outbound rows inherit the inbound request's id, so one operator action and the CA/DNS calls it triggered read as a single trace: opening a failed "Request Certificate" shows the exact POST /acme/new-order and the CA's 429 underneath. Implementation notes: - Capture is a pure-ASGI middleware that TEES the request and response streams rather than draining them. `await request.body()` inside a BaseHTTPMiddleware would consume the receive channel and break the raw-body agent heartbeat handler. Registered last so it is outermost: it then sees the final client-visible response and seeds correlation_id_context before the error handler reads it. - Rows are written by a batching background writer with a bounded queue, so the request path never awaits the database and a saturated logger drops rows visibly (surfaced on the page) instead of blocking. Redaction runs on the writer, off the request coroutine. - Secrets never land: headers are an allowlist with Authorization/Cookie kept only as a presence marker; body keys and value shapes are redacted (passwords, tokens, api_token, API keys, private-key PEMs, JWTs); the ACME JWS request body is never stored, because a stored protected+signature pair is a replayable credential — a summary is logged instead; DNS-provider errors record only the exception type; the ACME HTTP-01 challenge endpoint is excluded so key_authorization is never captured. - Retention is operator-configurable in Settings -> Request Log: separate day counts for successful and failed rows (7 / 30) plus a hard row cap (500k), whichever is reached first. Pruned in batches under a Postgres advisory lock, with the day counts bound as parameters, never interpolated. - New permissions requestlog.read / requestlog.manage. super_admin and security_admin get both, operator gets read, viewer gets neither. Schema: one new table (request_logs) plus its settings seed, SCHEMA_VERSION 10 -> 11, auto-migrated. No existing table altered, no agent or rendered-config change. Kill switches: REQUEST_LOG_ENABLED=false (middleware never registered) or the `enabled` toggle in Settings. Tests: 245 new (7 backend files + 1 frontend), full suite 1655 backend + 17 frontend passing.	2026-08-11 02:36:03 +03:00
taylanbakircioglu	a1192e602d	feat: HA / VIP (Keepalived) management from the UI (#27) ... Manage highly-available virtual IPs backed by Keepalived (VRRP) directly from the OpenManager UI — no more SSHing into nodes to install/configure Keepalived by hand. Builds on the agent pull-architecture: define the VIP centrally, click Apply, and the agents converge. Highlights: - New "HA / VIP" tab: create a virtual IP, pick a per-node interface, select which pool nodes participate (MASTER/BACKUP roles + priorities); live MASTER/BACKUP/FAULT per node. - On Apply, agents install & configure Keepalived (unicast VRRP, cloud-safe default) across the major distros (Debian/Ubuntu, RHEL/CentOS/Alma/Rocky, Fedora, SUSE/openSUSE, Alpine) with a HAProxy health-check, so the VIP fails over automatically when HAProxy drops. - Single-node (a managed floating IP without failover) and multi-node VRRP failover both work. - VIP changes ride the standard Apply Management flow with the standard "View Change" diff. - Approval-gated deletion (safety): deleting a running VIP is staged for approval and the VIP keeps running, untouched, until you approve it — an agent never tears a VIP down without an explicit human approval. Per-VIP Diagnostics view; opt-in package uninstall (only on nodes where OpenManager installed it). A node already running a hand-managed Keepalived is detected and never overwritten ("externally managed"). - Fully opt-in and backward compatible: nodes/clusters without a VIP are unaffected. Adds vip_instances + vip_members tables (idempotent SCHEMA_VERSION bump; existing data and passwords unaffected) and a `vip` RBAC permission group. - Also includes a HAProxy config-generator robustness fix: auto-inject a stick-table when a frontend uses a stick counter (track-sc / sc_*_rate) but declares none. On-prem / L2 (VRRP) scope; the UI notes the cloud caveat.	2026-06-07 01:52:20 +03:00
taylanbakircioglu	bd6a31cb0d	feat: v1.6.0 — Multi-Factor Authentication (Issue #18) ... Adds opt-in TOTP-based Multi-Factor Authentication that is fully backwards compatible with existing logins. Operators choose to enable MFA per account; nothing changes for users who do not opt in. Highlights ========== * RFC 6238 TOTP (6 digits, 30s period, SHA1) with ±30s skew tolerance, compatible with Microsoft / Google Authenticator, Authy, Duo, 1Password. * Per-step replay protection (`mfa_last_used_totp_step`) so a captured code cannot be reused inside the same window. * Fernet-encrypted TOTP secrets at rest, key resolution via `MFA_ENCRYPTION_KEY` env (HKDF-derived from `SECRET_KEY` as fallback). * 10 single-use, bcrypt-hashed backup codes per user, formatted `XXXX-YYYY` from a confusion-free alphabet (no 0/O/1/I/L). * Two-step login flow: `POST /api/auth/login` returns `mfa_required` + `mfa_token`, then `POST /api/auth/login/mfa-verify` accepts a TOTP code OR a backup code. JWT is minted only after MFA succeeds. * Self-service: users enable / disable MFA from their own row in the Users page; admins reset (single user or bulk) but never enable on behalf of someone else (matches AWS IAM / GitHub / Google Workspace). * Bulk emergency reset CLI: `scripts/admin-mfa-reset-all.sh`. Security hardening ================== * Atomic transactions with `SELECT … FOR UPDATE` on `mfa_pending_logins` and `users` rows so concurrent verify / enroll calls cannot race. * `/api/mfa/enroll/start` refuses re-enrollment when MFA is already on (prevents silent secret rotation via a stolen JWT). * Pydantic `ValidationError` messages are sanitized before reaching the audit log so request bodies (TOTP / backup codes in flight) never appear in plaintext. * Slowapi rate limits are per-USER, not per-IP, with a trusted-proxy XFF strategy so a single ingress address cannot exhaust the bucket for thousands of operators (`MFA_TRUSTED_PROXY_CIDRS`, `MFA_RATE_LIMIT_*` env-overridable). * Login query now scopes to `is_active = TRUE` so a soft-deleted row with the same username can no longer occlude the active user (also closes a small account-enumeration side channel). Database ======== Additive migrations (idempotent `ADD COLUMN IF NOT EXISTS`, `CREATE TABLE IF NOT EXISTS`): - users: mfa_enabled, mfa_method, mfa_secret_encrypted, mfa_enrolled_at, mfa_last_used_at, mfa_last_used_totp_step - mfa_backup_codes (user_id ON DELETE CASCADE) - mfa_pending_logins (user_id ON DELETE CASCADE, challenge_token, attempts, expires_at) - mfa_pending_enrollments (user_id ON DELETE CASCADE) Frontend ======== * Login page becomes a 3-phase state machine (credentials → MFA → submitting); legacy single-step login is preserved for users who haven't enrolled. * New MFAEnrollModal (3-step wizard: QR + secret → verify → backup codes) using `qrcode.react`. * Users page shows MFA column + per-row enable/disable/reset actions. Admins viewing other users with MFA off see a non-actionable info icon explaining that only the user themselves can enable MFA. Deployment ========== * `MFA_ENCRYPTION_KEY` is added to `k8s/manifests/03-secrets.yaml` as a placeholder; `SECRET_KEY` is also placeholder-ized so both are injected by the existing pipeline pattern (sed-replace + apply). * No new build-time env vars are required for the frontend. The SPA uses `window.location.host` for `/api/*` and is routed by the existing nginx ingress configuration. * `frontend/.dockerignore` ensures host `.env*` files cannot bleed into the production bundle. Tests ===== * New unit suites: - `test_mfa_service.py` (TOTP, encryption, backup codes) - `test_mfa_backwards_compat.py` (regression — non-MFA flow unchanged) - `test_mfa_rate_limits.py` (env override + dataclass immutability) - `test_mfa_rate_limit_key.py` (JWT key, trusted-proxy XFF, fallbacks) * All existing 1000+ unit tests continue to pass. Documentation ============= * README MFA section (overview, day-to-day operations, emergency reset CLI, env variables, rate-limit tuning). * `scripts/README.md` documents the bulk reset script. Issue: #18	2026-05-19 04:35:16 +03:00
taylanbakircioglu	02b1cb2bca	feat: v1.5.0 — Site Wizard (Issue #14) + ACME Diagnostic Panel (Issue #13) ... Closes #13, Closes #14. This release squashes the v1.4.0 → v1.5.0 development line. v1.4.0 shipped the ACME stability & enterprise audit (Issues #10/#11/#12). v1.5.0 builds on that foundation with two co-equal headline features plus a 22-round audit campaign hardening the prior configuration surface. License remains MIT for v1.5.0 (relicense to AGPL-3.0 lands in v1.5.2). ------------------------------------------------------------------ HEADLINE FEATURE A — ACME Diagnostic Panel (Issue #13) ------------------------------------------------------------------ A live pre-flight + post-failure diagnostic surface for every ACME order, reachable from the ACME Automation page. The panel exists to make ACME failures legible to operators who do NOT have shell access to the API host. Endpoints (`backend/routers/acme_diagnostics.py`): POST /api/letsencrypt/orders/{order_id}/diagnostics Run the full 5-check suite (DNS / port-80 / routing / account / agents) and humanize the order's `error_detail` (>=11 RFC-8555 problem types, backwards compatible with legacy plain-string failures). POST /api/letsencrypt/orders/{order_id}/diagnostics/ {check_id}/rerun Re-run a single check in place — used by the "Re-run" button on every row of the modal's pre-flight table. GET /api/letsencrypt/orders/{order_id}/events Merged event timeline combining the typed `acme_order_events` rows with correlated `user_activity_logs` entries (resource_type = 'letsencrypt_order' AND resource_id = order_id). The diagnostic modal auto-tails this timeline every 5 seconds while open. Service-level checks (`backend/services/acme_diagnostics.py`): * DNS resolution via stdlib socket.gethostbyname_ex through run_in_executor (intentionally avoiding an aiodns runtime dep for v1.5.0). * Port-80 HEAD probe, target locked to the order's domains, success on HTTP 200 OR 404, warns on egress timeout (corp egress policies routinely blackhole outbound 80 — fail-hard would be too noisy). * SSRF guard: probe refuses non-public IPs and surfaces the skip in the diagnostic result; IPv4-mapped IPv6 normalisation closes the `::ffff:169.254.169.254` cloud-metadata vector. * HAProxy routing presence check: matches the order's cluster_ids to a port-80 HTTP frontend. * ACME account validity check against `letsencrypt_accounts`. * Agent presence check (>=1 active agent in target cluster). * Every sub-check wrapped in a wall-clock timeout to bound impact on the API event loop. RBAC: ssl.read for run, ssl.read for events. Per-user 5/min rate limit on both run and rerun, backed by the (user_id, action, created_at DESC) composite index. Frontend (`frontend/src/components/ACMEAutomation.js`): * "Diagnose" button on every order row + the existing "stuck order" warning row. * Modal with two tabs: - Pre-flight Checks (Antd Table with status pills + Re-run buttons + humanized error banner) - Event Log (Antd Timeline with auto-tail polling, scroll- to-bottom, pause-on-hover) * Correlation IDs surfaced in error banners and individual check fail details for backend-log lookup. ------------------------------------------------------------------ HEADLINE FEATURE B — Site Setup Wizard (Issue #14) ------------------------------------------------------------------ A single guided flow that creates a Backend + Servers + HTTP Frontend (and optional HTTPS Frontend) in one atomic transaction. Endpoints (`backend/routers/site_wizard.py`): POST /api/site-wizard/preview — diff-preview the changeset POST /api/site-wizard/create — atomic execute POST /api/site-wizard/reject — clean rollback (including any wizard_staged ACME orders) GET /api/site-wizard/drafts — draft persistence PUT /api/site-wizard/drafts/{id} — save/update DELETE /api/site-wizard/drafts/{id} Feature surface: * One screen captures both backend (mode + servers) AND frontend (http + optional https + SSL mode) inputs. * SSL modes: ACME (new order, HTTP-01 only for v1.5.0), Upload (existing PEM), Existing (link to a stored cert), or None. * ACME-staged path: wizard_staged_until watermark on the `letsencrypt_orders` row defers finalisation until agent confirmation; per-mode reject cleanly cancels and rolls back the staged order. * Live diff preview against the cluster's current generated config (renderer-evolution noise stripped — track-sc<N> dedup, per-server cookie strip, defaults-cookie inheritance, listen-block flattening). * Draft persistence with PEM stripped at save time (private keys never round-trip through the drafts table). * Per-cluster multi-tenancy: drafts and wizard_staged orders are isolated to the creating user's cluster scope. Frontend (`frontend/src/components/SiteWizard.js`): * 4-step Antd Steps flow: Backend → Frontend → SSL → Review. * Render the live diff preview inline before commit. * Antd Form-level validation mirrors backend Pydantic validators (numeric bounds, HAProxy reserved keywords, ALPN consistency, IPv6 scope-id, domain regex, server name dedup). ------------------------------------------------------------------ AUDIT CAMPAIGN — Rounds 1 → 22 (Bulgu #1 → #82) ------------------------------------------------------------------ v1.5.0 includes 22 adversarial review passes. Each round produced its own commit set in the corporate development line; this squash collapses those into the v1.5.0 release artefact. Highlights: Round 1-4 Site Wizard core: dry-run parity, single-line value injection guard, ACL -f pattern-file block, SSL parity, timeout regex, form-state pin. Round 5-7 defaults-cookie inheritance, server-named-cookie guard, fe/be mode mismatch, duplicate server names, health_check_uri + server_address validators. Round 8-10 cookie_name / cookie_options newline-injection guard, dry-run parity (round 9), TCP-mode HTTP-only feature blockers. Round 11 SSL name path traversal + health-check >= 1. Round 12-13 SSL & ACME deep dive (Bulgu #23-#32). Round 14 single-line value injection (Bulgu #33). Round 15-17 ACME multi-tenant UX, numeric bounds, HAProxy reserved keywords, ALPN/TLS consistency, all-backup, multi-domain & multi-user enterprise edges, drain/HSTS/post-completion (Bulgu #34-#53). Round 18-21 concurrency, agent state, TCP-mode HTTP-only, list size caps, IPv6 scope-id, preview account validation, TCP backend + balance uri reject (Bulgu #54-#61). Round 22 FE error visibility + 3x stale-data lockouts, referential integrity + cascade safety, authentication & authorization, multi-cluster isolation, apply_pending_changes concurrency, script injection + bulk import multi-tenancy, prefix-stripped signature comparison (Bulgu #62-#82). ------------------------------------------------------------------ NO CORPORATE-SPECIFIC ARTIFACTS ------------------------------------------------------------------ This squash deliberately sanitises corporate hostnames, container registry references, and TLS secret names into generic placeholders (`your-registry.example.com/your-org`, `haproxy-openmanager*.example.com`, `wildcard-tls`, `taylanbakircioglu/haproxy-openmanager-*`) so the public artefact contains no internal infrastructure detail. Pilot / development history that retained those values stays in the corporate fork and is NOT part of this commit.	2026-05-14 00:04:19 +03:00
taylanbakircioglu	c2766eadbd	feat: dark mode support, UI improvements, and version bump to v1.2.0 ... - Full dark mode support across all pages with lightbulb toggle in header - Theme preference persisted in localStorage across sessions - Ant Design 5 token-based theming (40+ components updated) - Recharts dark mode: axes, grids, tooltips adapt to theme - Login page redesigned with product-consistent blue-gray palette - Overscroll bounce background matches dark theme - All Servers search with multi-field filtering - Bulk Config Import UI streamlined with collapsible guidelines - ConfigProvider moved above AppContent for correct token resolution Made-with: Cursor	2026-04-03 01:16:05 +03:00
taylanbakircioglu	199ade8ce5	Fix: Browser cache causing phantom deleted entities across all pages ... 🐛 Critical Browser Cache Bug - System-Wide Fix: - Fixed deleted entities reappearing on normal page refresh - Hard refresh (Cmd+Shift+R) worked, normal refresh showed stale cache data - Applied cache-busting to ALL entity fetch operations across entire application 🔧 Cache-Control Headers Added to 10 Components: 1. BackendServers.js - fetchBackends(), fetchFrontends(), fetchSSLCertificates() 2. FrontendManagement.js - fetchFrontends(), fetchBackends(), fetchSSLCertificates() 3. SSLManagement.js - fetchCertificates() 4. ApplyManagement.js - fetchPendingChanges() (4 API calls: frontends, backends, WAF, SSL) 5. WAFManagement.js - fetchFrontends() 6. AgentManagement.js - fetchAgents(), fetchPools() 7. PoolManagement.js - fetchPools(), fetchPoolAgents() 8. UserManagement.js - fetchUsers() 9. Configuration.js - fetchAgents() 10. ClusterContext.js - fetchClusters() Headers Applied: 'Cache-Control': 'no-cache, no-store, must-revalidate' 'Pragma': 'no-cache' 'Expires': '0' (some components) 🎯 Impact Analysis - SAFE Changes: ✅ Only GET requests affected (POST/PUT/DELETE unchanged) ✅ Response format unchanged (only headers added to request) ✅ No breaking changes to existing functionality ✅ Performance impact minimal (entities change frequently anyway) 🛡️ Protected Against Cache: - Deleted backends/frontends won't reappear - Deleted agents won't show in lists - SSL certificates always fresh - User list always current - Cluster/Pool data always accurate 🔍 Testing Performed: - Backend API verified: Only active backends returned (is_active=TRUE) - SSL API verified: Returns 4 certificates correctly - All axios.get calls now have cache-control headers - No linter errors ✅ Root Cause Solved: Browser/Axios caching GET responses → Stale data on normal refresh Solution: Force fresh data from API on every request Impact: Phantom entities bug completely resolved across entire application	2025-11-07 11:51:14 +03:00
taylanbakircioglu	1133fbe229	security: Fix critical RBAC vulnerability in user management ... Critical security fixes: - Add admin-only checks for user CRUD operations - Add admin-only checks for role CRUD operations - Add admin-only checks for role assignment operations - Add permission check for agent script generation - Fix auth_middleware to include is_admin flag in user context - Hide user/role management buttons from non-admin users in UI - Add 'View Only' labels for viewer users Security improvements: - Prevent viewer users from creating/editing/deleting users - Prevent viewer users from creating/editing/deleting roles - Prevent viewer users from assigning roles to users - Backend API endpoints now properly check admin status - Frontend UI now hides admin-only actions from viewers Public release changes: - Remove company-specific registry URLs from build-images.sh - Update registry to generic example: your-registry.example.com Affected endpoints: - POST /api/users (create user) - admin only - PUT /api/users/{id} (update user) - admin only - DELETE /api/users/{id} (delete user) - admin only - POST /api/roles (create role) - admin only - PUT /api/roles/{id} (update role) - admin only - DELETE /api/roles/{id} (delete role) - admin only - POST /api/users/{id}/roles (assign roles) - admin only - POST /api/agents/generate-install-script - permission check	2025-11-04 13:44:49 +03:00
taylanbakircioglu	6aae0f4309	Initial commit	2025-10-27 12:14:03 +03:00