mirror of
https://github.com/taylanbakircioglu/haproxy-openmanager.git
synced 2026-09-12 05:48:58 +00:00
taylanbakircioglu
bd6a31cb0d
Adds opt-in TOTP-based Multi-Factor Authentication that is fully
backwards compatible with existing logins. Operators choose to enable
MFA per account; nothing changes for users who do not opt in.
Highlights
==========
* RFC 6238 TOTP (6 digits, 30s period, SHA1) with ±30s skew tolerance,
compatible with Microsoft / Google Authenticator, Authy, Duo, 1Password.
* Per-step replay protection (`mfa_last_used_totp_step`) so a captured
code cannot be reused inside the same window.
* Fernet-encrypted TOTP secrets at rest, key resolution via
`MFA_ENCRYPTION_KEY` env (HKDF-derived from `SECRET_KEY` as fallback).
* 10 single-use, bcrypt-hashed backup codes per user, formatted
`XXXX-YYYY` from a confusion-free alphabet (no 0/O/1/I/L).
* Two-step login flow: `POST /api/auth/login` returns `mfa_required`
+ `mfa_token`, then `POST /api/auth/login/mfa-verify` accepts a TOTP
code OR a backup code. JWT is minted only after MFA succeeds.
* Self-service: users enable / disable MFA from their own row in the
Users page; admins reset (single user or bulk) but never enable on
behalf of someone else (matches AWS IAM / GitHub / Google Workspace).
* Bulk emergency reset CLI: `scripts/admin-mfa-reset-all.sh`.
Security hardening
==================
* Atomic transactions with `SELECT … FOR UPDATE` on `mfa_pending_logins`
and `users` rows so concurrent verify / enroll calls cannot race.
* `/api/mfa/enroll/start` refuses re-enrollment when MFA is already on
(prevents silent secret rotation via a stolen JWT).
* Pydantic `ValidationError` messages are sanitized before reaching the
audit log so request bodies (TOTP / backup codes in flight) never
appear in plaintext.
* Slowapi rate limits are per-USER, not per-IP, with a trusted-proxy
XFF strategy so a single ingress address cannot exhaust the bucket
for thousands of operators (`MFA_TRUSTED_PROXY_CIDRS`,
`MFA_RATE_LIMIT_*` env-overridable).
* Login query now scopes to `is_active = TRUE` so a soft-deleted row
with the same username can no longer occlude the active user
(also closes a small account-enumeration side channel).
Database
========
Additive migrations (idempotent `ADD COLUMN IF NOT EXISTS`,
`CREATE TABLE IF NOT EXISTS`):
- users: mfa_enabled, mfa_method, mfa_secret_encrypted,
mfa_enrolled_at, mfa_last_used_at, mfa_last_used_totp_step
- mfa_backup_codes (user_id ON DELETE CASCADE)
- mfa_pending_logins (user_id ON DELETE CASCADE, challenge_token,
attempts, expires_at)
- mfa_pending_enrollments (user_id ON DELETE CASCADE)
Frontend
========
* Login page becomes a 3-phase state machine
(credentials → MFA → submitting); legacy single-step login is
preserved for users who haven't enrolled.
* New MFAEnrollModal (3-step wizard: QR + secret → verify → backup
codes) using `qrcode.react`.
* Users page shows MFA column + per-row enable/disable/reset actions.
Admins viewing other users with MFA off see a non-actionable info
icon explaining that only the user themselves can enable MFA.
Deployment
==========
* `MFA_ENCRYPTION_KEY` is added to `k8s/manifests/03-secrets.yaml` as
a placeholder; `SECRET_KEY` is also placeholder-ized so both are
injected by the existing pipeline pattern (sed-replace + apply).
* No new build-time env vars are required for the frontend. The SPA
uses `window.location.host` for `/api/*` and is routed by the
existing nginx ingress configuration.
* `frontend/.dockerignore` ensures host `.env*` files cannot bleed
into the production bundle.
Tests
=====
* New unit suites:
- `test_mfa_service.py` (TOTP, encryption, backup codes)
- `test_mfa_backwards_compat.py` (regression — non-MFA flow unchanged)
- `test_mfa_rate_limits.py` (env override + dataclass immutability)
- `test_mfa_rate_limit_key.py` (JWT key, trusted-proxy XFF, fallbacks)
* All existing 1000+ unit tests continue to pass.
Documentation
=============
* README MFA section (overview, day-to-day operations, emergency
reset CLI, env variables, rate-limit tuning).
* `scripts/README.md` documents the bulk reset script.
Issue: #18
152 lines
3.7 KiB
Markdown
152 lines
3.7 KiB
Markdown
# HAProxy OpenManager - Utility Scripts
|
|
|
|
This directory contains utility scripts for managing, monitoring, and troubleshooting HAProxy OpenManager.
|
|
|
|
## 🧹 Cleanup Scripts
|
|
|
|
### cleanup-cluster-entities.sh
|
|
**Purpose:** Clean all frontends and backends from a specific cluster
|
|
|
|
**Usage:**
|
|
```bash
|
|
# Interactive mode (prompts for cluster)
|
|
./scripts/cleanup-cluster-entities.sh
|
|
|
|
# Direct cluster specification
|
|
./scripts/cleanup-cluster-entities.sh "demo-cluster"
|
|
./scripts/cleanup-cluster-entities.sh 1
|
|
```
|
|
|
|
**Features:**
|
|
- Interactive cluster selection (by name or ID)
|
|
- Safety confirmation prompts
|
|
- Automatic HAProxy config apply
|
|
- Comprehensive status reporting
|
|
- Preserves cluster (only deletes entities)
|
|
|
|
### cleanup-soft-deleted.sh
|
|
**Purpose:** Permanently delete soft-deleted entities from database
|
|
|
|
**Usage:**
|
|
```bash
|
|
./scripts/cleanup-soft-deleted.sh
|
|
```
|
|
|
|
**Features:**
|
|
- Database health check
|
|
- Dry run preview
|
|
- Double confirmation (yes + CONFIRM)
|
|
- Verification after cleanup
|
|
- Detailed entity deletion report
|
|
|
|
## 🔧 Agent Scripts
|
|
|
|
### fix-agent-status.sh
|
|
Fix agents stuck in 'upgrading' status
|
|
|
|
### update-agent-version.sh
|
|
Update agent script versions
|
|
|
|
## 🔍 Monitoring & Debugging Scripts
|
|
|
|
### check-agent-logs.sh
|
|
Check agent log files for errors
|
|
|
|
### check-agent-stats.sh
|
|
Verify agent statistics collection
|
|
|
|
### check-haproxy-stats-socket.sh
|
|
Test HAProxy stats socket connectivity
|
|
|
|
### debug-agent-stats-function.sh
|
|
Debug agent stats collection functions
|
|
|
|
### test-agent-stats-sending.sh
|
|
Test agent stats sending functionality
|
|
|
|
### test-real-heartbeat.sh
|
|
Test agent heartbeat functionality
|
|
|
|
### test-stats-parser.py
|
|
Test HAProxy stats parsing
|
|
|
|
## 🏗️ Build & Test Scripts
|
|
|
|
### test-build.sh
|
|
Run build tests for the project
|
|
|
|
## 🔐 MFA Admin Scripts
|
|
|
|
### admin-mfa-reset-all.sh
|
|
**Purpose:** Emergency — disable Multi-Factor Authentication for **every** user
|
|
in one call. Use only when there is a mass loss of authenticator devices /
|
|
inherited platform without working operators (Issue #18, v1.6.0).
|
|
|
|
**Usage:**
|
|
```bash
|
|
# Interactive prompts ask for the admin Bearer token + reason
|
|
./scripts/admin-mfa-reset-all.sh
|
|
|
|
# Non-interactive (still requires double confirmation typed at the keyboard)
|
|
API_URL=https://hap.example.com \
|
|
ADMIN_TOKEN=eyJhbGciOi... \
|
|
./scripts/admin-mfa-reset-all.sh
|
|
```
|
|
|
|
**Features:**
|
|
- Calls `POST /api/mfa/admin-reset-all` (requires `users.is_admin = TRUE`)
|
|
- Double confirmation: type `yes`, then `RESET ALL MFA` exactly
|
|
- Required reason is recorded in `user_activity_logs`
|
|
(`action='mfa.disabled.admin_bulk_reset'`)
|
|
- Deletes every backup code and invalidates pending MFA challenges
|
|
|
|
**Safety:**
|
|
- Irreversible — all users must re-enroll MFA afterwards
|
|
- All other authentication (password, JWT, roles) is unaffected
|
|
|
|
## 🚨 Emergency Use Cases
|
|
|
|
**1. Cluster Migration/Cleanup:**
|
|
```bash
|
|
# Clean old cluster completely
|
|
./scripts/cleanup-cluster-entities.sh "old-cluster"
|
|
|
|
# Clean soft-deleted entities
|
|
./scripts/cleanup-soft-deleted.sh
|
|
```
|
|
|
|
**2. Database Maintenance:**
|
|
```bash
|
|
# Regular cleanup of soft-deleted entities
|
|
./scripts/cleanup-soft-deleted.sh
|
|
```
|
|
|
|
**3. Agent Issues:**
|
|
```bash
|
|
# Fix stuck agents
|
|
./scripts/fix-agent-status.sh
|
|
|
|
# Debug stats problems
|
|
./scripts/debug-agent-stats-function.sh
|
|
```
|
|
|
|
**4. MFA Outage (mass lost authenticators):**
|
|
```bash
|
|
# Disable MFA for every user, then ask them to re-enroll
|
|
./scripts/admin-mfa-reset-all.sh
|
|
```
|
|
|
|
## ⚠️ Safety Notes
|
|
|
|
- All cleanup scripts require admin authentication
|
|
- Cluster entity cleanup preserves the cluster itself
|
|
- Soft-delete cleanup is permanent and cannot be undone
|
|
- Always use dry-run features when available
|
|
- Test in development environment first
|
|
|
|
## 🔗 Related Documentation
|
|
|
|
- [Testing Guide](../TESTING.md)
|
|
- [Impact Analysis](../IMPACT_ANALYSIS.md)
|
|
- [Configuration Guide](../CONFIG.md)
|