Files
haproxy-openmanager/scripts/README.md
T
taylanbakircioglu bd6a31cb0d feat: v1.6.0 — Multi-Factor Authentication (Issue #18)
Adds opt-in TOTP-based Multi-Factor Authentication that is fully
backwards compatible with existing logins. Operators choose to enable
MFA per account; nothing changes for users who do not opt in.

Highlights
==========

* RFC 6238 TOTP (6 digits, 30s period, SHA1) with ±30s skew tolerance,
  compatible with Microsoft / Google Authenticator, Authy, Duo, 1Password.
* Per-step replay protection (`mfa_last_used_totp_step`) so a captured
  code cannot be reused inside the same window.
* Fernet-encrypted TOTP secrets at rest, key resolution via
  `MFA_ENCRYPTION_KEY` env (HKDF-derived from `SECRET_KEY` as fallback).
* 10 single-use, bcrypt-hashed backup codes per user, formatted
  `XXXX-YYYY` from a confusion-free alphabet (no 0/O/1/I/L).
* Two-step login flow: `POST /api/auth/login` returns `mfa_required`
  + `mfa_token`, then `POST /api/auth/login/mfa-verify` accepts a TOTP
  code OR a backup code. JWT is minted only after MFA succeeds.
* Self-service: users enable / disable MFA from their own row in the
  Users page; admins reset (single user or bulk) but never enable on
  behalf of someone else (matches AWS IAM / GitHub / Google Workspace).
* Bulk emergency reset CLI: `scripts/admin-mfa-reset-all.sh`.

Security hardening
==================

* Atomic transactions with `SELECT … FOR UPDATE` on `mfa_pending_logins`
  and `users` rows so concurrent verify / enroll calls cannot race.
* `/api/mfa/enroll/start` refuses re-enrollment when MFA is already on
  (prevents silent secret rotation via a stolen JWT).
* Pydantic `ValidationError` messages are sanitized before reaching the
  audit log so request bodies (TOTP / backup codes in flight) never
  appear in plaintext.
* Slowapi rate limits are per-USER, not per-IP, with a trusted-proxy
  XFF strategy so a single ingress address cannot exhaust the bucket
  for thousands of operators (`MFA_TRUSTED_PROXY_CIDRS`,
  `MFA_RATE_LIMIT_*` env-overridable).
* Login query now scopes to `is_active = TRUE` so a soft-deleted row
  with the same username can no longer occlude the active user
  (also closes a small account-enumeration side channel).

Database
========

Additive migrations (idempotent `ADD COLUMN IF NOT EXISTS`,
`CREATE TABLE IF NOT EXISTS`):

  - users: mfa_enabled, mfa_method, mfa_secret_encrypted,
    mfa_enrolled_at, mfa_last_used_at, mfa_last_used_totp_step
  - mfa_backup_codes (user_id ON DELETE CASCADE)
  - mfa_pending_logins (user_id ON DELETE CASCADE, challenge_token,
    attempts, expires_at)
  - mfa_pending_enrollments (user_id ON DELETE CASCADE)

Frontend
========

* Login page becomes a 3-phase state machine
  (credentials → MFA → submitting); legacy single-step login is
  preserved for users who haven't enrolled.
* New MFAEnrollModal (3-step wizard: QR + secret → verify → backup
  codes) using `qrcode.react`.
* Users page shows MFA column + per-row enable/disable/reset actions.
  Admins viewing other users with MFA off see a non-actionable info
  icon explaining that only the user themselves can enable MFA.

Deployment
==========

* `MFA_ENCRYPTION_KEY` is added to `k8s/manifests/03-secrets.yaml` as
  a placeholder; `SECRET_KEY` is also placeholder-ized so both are
  injected by the existing pipeline pattern (sed-replace + apply).
* No new build-time env vars are required for the frontend. The SPA
  uses `window.location.host` for `/api/*` and is routed by the
  existing nginx ingress configuration.
* `frontend/.dockerignore` ensures host `.env*` files cannot bleed
  into the production bundle.

Tests
=====

* New unit suites:
  - `test_mfa_service.py` (TOTP, encryption, backup codes)
  - `test_mfa_backwards_compat.py` (regression — non-MFA flow unchanged)
  - `test_mfa_rate_limits.py` (env override + dataclass immutability)
  - `test_mfa_rate_limit_key.py` (JWT key, trusted-proxy XFF, fallbacks)
* All existing 1000+ unit tests continue to pass.

Documentation
=============

* README MFA section (overview, day-to-day operations, emergency
  reset CLI, env variables, rate-limit tuning).
* `scripts/README.md` documents the bulk reset script.

Issue: #18
2026-05-19 04:35:16 +03:00

3.7 KiB

HAProxy OpenManager - Utility Scripts

This directory contains utility scripts for managing, monitoring, and troubleshooting HAProxy OpenManager.

🧹 Cleanup Scripts

cleanup-cluster-entities.sh

Purpose: Clean all frontends and backends from a specific cluster

Usage:

# Interactive mode (prompts for cluster)
./scripts/cleanup-cluster-entities.sh

# Direct cluster specification
./scripts/cleanup-cluster-entities.sh "demo-cluster"
./scripts/cleanup-cluster-entities.sh 1

Features:

  • Interactive cluster selection (by name or ID)
  • Safety confirmation prompts
  • Automatic HAProxy config apply
  • Comprehensive status reporting
  • Preserves cluster (only deletes entities)

cleanup-soft-deleted.sh

Purpose: Permanently delete soft-deleted entities from database

Usage:

./scripts/cleanup-soft-deleted.sh

Features:

  • Database health check
  • Dry run preview
  • Double confirmation (yes + CONFIRM)
  • Verification after cleanup
  • Detailed entity deletion report

🔧 Agent Scripts

fix-agent-status.sh

Fix agents stuck in 'upgrading' status

update-agent-version.sh

Update agent script versions

🔍 Monitoring & Debugging Scripts

check-agent-logs.sh

Check agent log files for errors

check-agent-stats.sh

Verify agent statistics collection

check-haproxy-stats-socket.sh

Test HAProxy stats socket connectivity

debug-agent-stats-function.sh

Debug agent stats collection functions

test-agent-stats-sending.sh

Test agent stats sending functionality

test-real-heartbeat.sh

Test agent heartbeat functionality

test-stats-parser.py

Test HAProxy stats parsing

🏗️ Build & Test Scripts

test-build.sh

Run build tests for the project

🔐 MFA Admin Scripts

admin-mfa-reset-all.sh

Purpose: Emergency — disable Multi-Factor Authentication for every user in one call. Use only when there is a mass loss of authenticator devices / inherited platform without working operators (Issue #18, v1.6.0).

Usage:

# Interactive prompts ask for the admin Bearer token + reason
./scripts/admin-mfa-reset-all.sh

# Non-interactive (still requires double confirmation typed at the keyboard)
API_URL=https://hap.example.com \
ADMIN_TOKEN=eyJhbGciOi... \
  ./scripts/admin-mfa-reset-all.sh

Features:

  • Calls POST /api/mfa/admin-reset-all (requires users.is_admin = TRUE)
  • Double confirmation: type yes, then RESET ALL MFA exactly
  • Required reason is recorded in user_activity_logs (action='mfa.disabled.admin_bulk_reset')
  • Deletes every backup code and invalidates pending MFA challenges

Safety:

  • Irreversible — all users must re-enroll MFA afterwards
  • All other authentication (password, JWT, roles) is unaffected

🚨 Emergency Use Cases

1. Cluster Migration/Cleanup:

# Clean old cluster completely
./scripts/cleanup-cluster-entities.sh "old-cluster"

# Clean soft-deleted entities
./scripts/cleanup-soft-deleted.sh

2. Database Maintenance:

# Regular cleanup of soft-deleted entities
./scripts/cleanup-soft-deleted.sh

3. Agent Issues:

# Fix stuck agents
./scripts/fix-agent-status.sh

# Debug stats problems
./scripts/debug-agent-stats-function.sh

4. MFA Outage (mass lost authenticators):

# Disable MFA for every user, then ask them to re-enroll
./scripts/admin-mfa-reset-all.sh

⚠️ Safety Notes

  • All cleanup scripts require admin authentication
  • Cluster entity cleanup preserves the cluster itself
  • Soft-delete cleanup is permanent and cannot be undone
  • Always use dry-run features when available
  • Test in development environment first