A user running the API on a 2-core/4GB host reported slow-feeling API
responses (Issue #35 follow-up). Review of the hot paths found no
pathological defect; the dominant factors are the single uvicorn worker
(one core serves all requests) and the constant agent-poll baseline
(4 requests per agent every 30s). Two zero-risk improvements:
- backend/Dockerfile: CMD now honors UVICORN_WORKERS, falling back to
WEB_CONCURRENCY and then 1. Flagless uvicorn natively honors
WEB_CONCURRENCY, so the fallback keeps any deployment that relied on it
byte-for-byte compatible; with the final default of 1 worker uvicorn runs
in-process exactly as before. >1 enables the multiprocess supervisor so
multi-core hosts can use all cores. Background tasks are already
multi-replica safe (FOR UPDATE SKIP LOCKED / advisory locks), as exercised
by the k8s HPA deployment (2-10 replicas). `exec` keeps uvicorn as PID 1
(clean SIGTERM, verified ~1s docker stop with 2 workers).
docker-compose.yml passes UVICORN_WORKERS through as empty-when-unset so a
user-set WEB_CONCURRENCY is never overridden; .env.template documents it.
- routers/agent.py heartbeat (by-name endpoint): the agent's
status/version/upgrade_status were read with three separate single-column
SELECTs against the same row; now one SELECT. Identical values and None
semantics (single consistent snapshot instead of three reads); saves two
round-trips per heartbeat per agent every 30s. The legacy by-id heartbeat
endpoint is untouched; the heartbeat API contract is unchanged for agents
of every version.
- README: new "Performance Tuning" section (worker/replica scaling, and how
to use the X-Response-Time header plus "Slow request detected" logs to
pinpoint slow endpoints).
Verification: full backend suite in docker green (1063 passed, 151 skipped;
also re-run by the runtime image build); worker-count expansion matrix
(unset->1, UVICORN_WORKERS=2->2, WEB_CONCURRENCY=3->3, both->UVICORN_WORKERS,
empty->fallback) all correct; default run confirmed single-process with
uvicorn as PID 1 and healthy API; UVICORN_WORKERS=2 confirmed parent + 2
workers, healthy API, clean shutdown; live heartbeats verified for
register + existing-agent paths AND degraded agents (no stats socket /
haproxy stopped / garbage stats CSV / unknown backend in server_statuses):
all return 200, agent row updates correctly, zero backend errors.
No schema, API, or agent changes.
Supersedes the v1.6.4 stable pin (1.30.2-alpine) with the mainline
patched release. No config, schema, or behavior changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bundled nginx reverse proxy was flagged for the nginx 'poolslip' advisory
(affected: mainline <=1.31.0; fixed: stable 1.30.2+ / mainline 1.31.1+). The
config-level mitigation (named capture groups instead of $1/$2 in rewrite) does
not apply — the product's nginx config (nginx/nginx.conf and the k8s configmap)
has no rewrite capture-group directives, only prefix locations + proxy_pass. So
the fix is the version: pin nginx:alpine -> nginx:1.30.2-alpine in
docker-compose.yml and k8s/manifests/10-nginx.yaml.
No config/schema/behavior change (nginx only reverse-proxies). Version bumped to
1.6.4 across all layers. Verified in Docker: nginx -v=1.30.2; nginx -t OK on both
the compose and production configmap configs; all proxied routes work through
nginx; no nginx errors.
Adds opt-in TOTP-based Multi-Factor Authentication that is fully
backwards compatible with existing logins. Operators choose to enable
MFA per account; nothing changes for users who do not opt in.
Highlights
==========
* RFC 6238 TOTP (6 digits, 30s period, SHA1) with ±30s skew tolerance,
compatible with Microsoft / Google Authenticator, Authy, Duo, 1Password.
* Per-step replay protection (`mfa_last_used_totp_step`) so a captured
code cannot be reused inside the same window.
* Fernet-encrypted TOTP secrets at rest, key resolution via
`MFA_ENCRYPTION_KEY` env (HKDF-derived from `SECRET_KEY` as fallback).
* 10 single-use, bcrypt-hashed backup codes per user, formatted
`XXXX-YYYY` from a confusion-free alphabet (no 0/O/1/I/L).
* Two-step login flow: `POST /api/auth/login` returns `mfa_required`
+ `mfa_token`, then `POST /api/auth/login/mfa-verify` accepts a TOTP
code OR a backup code. JWT is minted only after MFA succeeds.
* Self-service: users enable / disable MFA from their own row in the
Users page; admins reset (single user or bulk) but never enable on
behalf of someone else (matches AWS IAM / GitHub / Google Workspace).
* Bulk emergency reset CLI: `scripts/admin-mfa-reset-all.sh`.
Security hardening
==================
* Atomic transactions with `SELECT … FOR UPDATE` on `mfa_pending_logins`
and `users` rows so concurrent verify / enroll calls cannot race.
* `/api/mfa/enroll/start` refuses re-enrollment when MFA is already on
(prevents silent secret rotation via a stolen JWT).
* Pydantic `ValidationError` messages are sanitized before reaching the
audit log so request bodies (TOTP / backup codes in flight) never
appear in plaintext.
* Slowapi rate limits are per-USER, not per-IP, with a trusted-proxy
XFF strategy so a single ingress address cannot exhaust the bucket
for thousands of operators (`MFA_TRUSTED_PROXY_CIDRS`,
`MFA_RATE_LIMIT_*` env-overridable).
* Login query now scopes to `is_active = TRUE` so a soft-deleted row
with the same username can no longer occlude the active user
(also closes a small account-enumeration side channel).
Database
========
Additive migrations (idempotent `ADD COLUMN IF NOT EXISTS`,
`CREATE TABLE IF NOT EXISTS`):
- users: mfa_enabled, mfa_method, mfa_secret_encrypted,
mfa_enrolled_at, mfa_last_used_at, mfa_last_used_totp_step
- mfa_backup_codes (user_id ON DELETE CASCADE)
- mfa_pending_logins (user_id ON DELETE CASCADE, challenge_token,
attempts, expires_at)
- mfa_pending_enrollments (user_id ON DELETE CASCADE)
Frontend
========
* Login page becomes a 3-phase state machine
(credentials → MFA → submitting); legacy single-step login is
preserved for users who haven't enrolled.
* New MFAEnrollModal (3-step wizard: QR + secret → verify → backup
codes) using `qrcode.react`.
* Users page shows MFA column + per-row enable/disable/reset actions.
Admins viewing other users with MFA off see a non-actionable info
icon explaining that only the user themselves can enable MFA.
Deployment
==========
* `MFA_ENCRYPTION_KEY` is added to `k8s/manifests/03-secrets.yaml` as
a placeholder; `SECRET_KEY` is also placeholder-ized so both are
injected by the existing pipeline pattern (sed-replace + apply).
* No new build-time env vars are required for the frontend. The SPA
uses `window.location.host` for `/api/*` and is routed by the
existing nginx ingress configuration.
* `frontend/.dockerignore` ensures host `.env*` files cannot bleed
into the production bundle.
Tests
=====
* New unit suites:
- `test_mfa_service.py` (TOTP, encryption, backup codes)
- `test_mfa_backwards_compat.py` (regression — non-MFA flow unchanged)
- `test_mfa_rate_limits.py` (env override + dataclass immutability)
- `test_mfa_rate_limit_key.py` (JWT key, trusted-proxy XFF, fallbacks)
* All existing 1000+ unit tests continue to pass.
Documentation
=============
* README MFA section (overview, day-to-day operations, emergency
reset CLI, env variables, rate-limit tuning).
* `scripts/README.md` documents the bulk reset script.
Issue: #18
- Backend/frontend now pull pre-built images from Docker Hub by default
- Build sections commented out (can be uncommented for local builds)
- Removed dev volume mount (./backend:/app) for production use
- Backend healthcheck changed from curl to python urllib (curl not in slim image)
- Added nginx /health endpoint for compose healthcheck
Made-with: Cursor
taylanbakircioglu