mirror of
https://github.com/taylanbakircioglu/haproxy-openmanager.git
synced 2026-09-12 05:48:58 +00:00
taylanbakircioglu
bd6a31cb0d
Adds opt-in TOTP-based Multi-Factor Authentication that is fully
backwards compatible with existing logins. Operators choose to enable
MFA per account; nothing changes for users who do not opt in.
Highlights
==========
* RFC 6238 TOTP (6 digits, 30s period, SHA1) with ±30s skew tolerance,
compatible with Microsoft / Google Authenticator, Authy, Duo, 1Password.
* Per-step replay protection (`mfa_last_used_totp_step`) so a captured
code cannot be reused inside the same window.
* Fernet-encrypted TOTP secrets at rest, key resolution via
`MFA_ENCRYPTION_KEY` env (HKDF-derived from `SECRET_KEY` as fallback).
* 10 single-use, bcrypt-hashed backup codes per user, formatted
`XXXX-YYYY` from a confusion-free alphabet (no 0/O/1/I/L).
* Two-step login flow: `POST /api/auth/login` returns `mfa_required`
+ `mfa_token`, then `POST /api/auth/login/mfa-verify` accepts a TOTP
code OR a backup code. JWT is minted only after MFA succeeds.
* Self-service: users enable / disable MFA from their own row in the
Users page; admins reset (single user or bulk) but never enable on
behalf of someone else (matches AWS IAM / GitHub / Google Workspace).
* Bulk emergency reset CLI: `scripts/admin-mfa-reset-all.sh`.
Security hardening
==================
* Atomic transactions with `SELECT … FOR UPDATE` on `mfa_pending_logins`
and `users` rows so concurrent verify / enroll calls cannot race.
* `/api/mfa/enroll/start` refuses re-enrollment when MFA is already on
(prevents silent secret rotation via a stolen JWT).
* Pydantic `ValidationError` messages are sanitized before reaching the
audit log so request bodies (TOTP / backup codes in flight) never
appear in plaintext.
* Slowapi rate limits are per-USER, not per-IP, with a trusted-proxy
XFF strategy so a single ingress address cannot exhaust the bucket
for thousands of operators (`MFA_TRUSTED_PROXY_CIDRS`,
`MFA_RATE_LIMIT_*` env-overridable).
* Login query now scopes to `is_active = TRUE` so a soft-deleted row
with the same username can no longer occlude the active user
(also closes a small account-enumeration side channel).
Database
========
Additive migrations (idempotent `ADD COLUMN IF NOT EXISTS`,
`CREATE TABLE IF NOT EXISTS`):
- users: mfa_enabled, mfa_method, mfa_secret_encrypted,
mfa_enrolled_at, mfa_last_used_at, mfa_last_used_totp_step
- mfa_backup_codes (user_id ON DELETE CASCADE)
- mfa_pending_logins (user_id ON DELETE CASCADE, challenge_token,
attempts, expires_at)
- mfa_pending_enrollments (user_id ON DELETE CASCADE)
Frontend
========
* Login page becomes a 3-phase state machine
(credentials → MFA → submitting); legacy single-step login is
preserved for users who haven't enrolled.
* New MFAEnrollModal (3-step wizard: QR + secret → verify → backup
codes) using `qrcode.react`.
* Users page shows MFA column + per-row enable/disable/reset actions.
Admins viewing other users with MFA off see a non-actionable info
icon explaining that only the user themselves can enable MFA.
Deployment
==========
* `MFA_ENCRYPTION_KEY` is added to `k8s/manifests/03-secrets.yaml` as
a placeholder; `SECRET_KEY` is also placeholder-ized so both are
injected by the existing pipeline pattern (sed-replace + apply).
* No new build-time env vars are required for the frontend. The SPA
uses `window.location.host` for `/api/*` and is routed by the
existing nginx ingress configuration.
* `frontend/.dockerignore` ensures host `.env*` files cannot bleed
into the production bundle.
Tests
=====
* New unit suites:
- `test_mfa_service.py` (TOTP, encryption, backup codes)
- `test_mfa_backwards_compat.py` (regression — non-MFA flow unchanged)
- `test_mfa_rate_limits.py` (env override + dataclass immutability)
- `test_mfa_rate_limit_key.py` (JWT key, trusted-proxy XFF, fallbacks)
* All existing 1000+ unit tests continue to pass.
Documentation
=============
* README MFA section (overview, day-to-day operations, emergency
reset CLI, env variables, rate-limit tuning).
* `scripts/README.md` documents the bulk reset script.
Issue: #18
HAProxy OpenManager - Utility Scripts
This directory contains utility scripts for managing, monitoring, and troubleshooting HAProxy OpenManager.
🧹 Cleanup Scripts
cleanup-cluster-entities.sh
Purpose: Clean all frontends and backends from a specific cluster
Usage:
# Interactive mode (prompts for cluster)
./scripts/cleanup-cluster-entities.sh
# Direct cluster specification
./scripts/cleanup-cluster-entities.sh "demo-cluster"
./scripts/cleanup-cluster-entities.sh 1
Features:
- Interactive cluster selection (by name or ID)
- Safety confirmation prompts
- Automatic HAProxy config apply
- Comprehensive status reporting
- Preserves cluster (only deletes entities)
cleanup-soft-deleted.sh
Purpose: Permanently delete soft-deleted entities from database
Usage:
./scripts/cleanup-soft-deleted.sh
Features:
- Database health check
- Dry run preview
- Double confirmation (yes + CONFIRM)
- Verification after cleanup
- Detailed entity deletion report
🔧 Agent Scripts
fix-agent-status.sh
Fix agents stuck in 'upgrading' status
update-agent-version.sh
Update agent script versions
🔍 Monitoring & Debugging Scripts
check-agent-logs.sh
Check agent log files for errors
check-agent-stats.sh
Verify agent statistics collection
check-haproxy-stats-socket.sh
Test HAProxy stats socket connectivity
debug-agent-stats-function.sh
Debug agent stats collection functions
test-agent-stats-sending.sh
Test agent stats sending functionality
test-real-heartbeat.sh
Test agent heartbeat functionality
test-stats-parser.py
Test HAProxy stats parsing
🏗️ Build & Test Scripts
test-build.sh
Run build tests for the project
🔐 MFA Admin Scripts
admin-mfa-reset-all.sh
Purpose: Emergency — disable Multi-Factor Authentication for every user in one call. Use only when there is a mass loss of authenticator devices / inherited platform without working operators (Issue #18, v1.6.0).
Usage:
# Interactive prompts ask for the admin Bearer token + reason
./scripts/admin-mfa-reset-all.sh
# Non-interactive (still requires double confirmation typed at the keyboard)
API_URL=https://hap.example.com \
ADMIN_TOKEN=eyJhbGciOi... \
./scripts/admin-mfa-reset-all.sh
Features:
- Calls
POST /api/mfa/admin-reset-all(requiresusers.is_admin = TRUE) - Double confirmation: type
yes, thenRESET ALL MFAexactly - Required reason is recorded in
user_activity_logs(action='mfa.disabled.admin_bulk_reset') - Deletes every backup code and invalidates pending MFA challenges
Safety:
- Irreversible — all users must re-enroll MFA afterwards
- All other authentication (password, JWT, roles) is unaffected
🚨 Emergency Use Cases
1. Cluster Migration/Cleanup:
# Clean old cluster completely
./scripts/cleanup-cluster-entities.sh "old-cluster"
# Clean soft-deleted entities
./scripts/cleanup-soft-deleted.sh
2. Database Maintenance:
# Regular cleanup of soft-deleted entities
./scripts/cleanup-soft-deleted.sh
3. Agent Issues:
# Fix stuck agents
./scripts/fix-agent-status.sh
# Debug stats problems
./scripts/debug-agent-stats-function.sh
4. MFA Outage (mass lost authenticators):
# Disable MFA for every user, then ask them to re-enroll
./scripts/admin-mfa-reset-all.sh
⚠️ Safety Notes
- All cleanup scripts require admin authentication
- Cluster entity cleanup preserves the cluster itself
- Soft-delete cleanup is permanent and cannot be undone
- Always use dry-run features when available
- Test in development environment first