Bulk import / manual edit silently dropped `filter spoe engine ...` (Coraza WAF)
and frontend `log-format` because the parser recognised only a fixed directive
set. The regenerated config then missed the SPOE engine, so HAProxy failed with
"unable to find SPOE engine 'coraza' used by the send-spoe-group".
- parser: capture `filter` + `log-format`/`log-format-sd` into new ParsedFrontend fields
- db: additive nullable `log_format` + `filters` TEXT columns on frontends (SCHEMA_VERSION 8->9)
- generator: new `filter` bucket flushed before http-request rules so `filter` precedes
`send-spoe-group`; `log-format` kept in prelude
- bulk import: preview dict, change-detection, persist (create + merge-update); cluster-aware
SPOE pre-flight advisories (missing-filter + host-prerequisite) surfaced in the UI
- manual CRUD: full round-trip (get/create/update) incl. React form fields (no null-wipe)
- reject/rollback: restore the new columns; restore path + wizard helper kept in parity
- backend `option spop-check` recognised (suppresses spurious warning for coraza-spoa)
- tests: test_spoe_filter_import.py; full suite green (1079 passed)
When a backend block in haproxy.cfg does not specify explicit timeout
values, the bulk import was injecting hardcoded defaults (connect 10s,
server 60s, queue 60s) into the database. These then appeared in the
generated config and overrode the agent's defaults section. Now, only
explicitly declared timeouts are stored; omitted ones remain NULL so
the agent's existing defaults section stays in effect.
- Add 'random' and 'first' options to backend balance method selector
- Add balance method validation in config parser with warning for unknown methods
- Update API documentation with all supported balance algorithms
CRITICAL FIXES:
1. Agent Heartbeat Validation Error (HTTP 422)
- Added missing 'system_info' field to AgentHeartbeat model
- Agents were sending system_info but backend model didn't accept it
- No agent script update needed - agents already send this field
2. Bulk Import SSL Parser Enhancement
- Fixed parsing of multiple SSL certificates with alpn/npn parameters
- Example: 'bind :443 ssl crt cert1.pem crt cert2.pem crt cert3.pem alpn h2,http/1.1'
- Old parser stopped at first whitespace after crt path
- New parser extracts all crt paths even with alpn/npn/ciphers after them
- Added user-friendly warning for SSL parameters (alpn, npn, ciphers) that won't be imported
TECHNICAL DETAILS:
- backend/models/agent.py: Added system_info: Optional[Dict[str, Any]]
- backend/utils/haproxy_config_parser.py: Enhanced SSL bind parsing logic
* Parse bind line by splitting and iterating through parts
* Extract all crt paths before hitting SSL parameters
* Detect and warn about alpn, npn, ciphers, ciphersuites parameters
* Inform user these advanced options should be configured manually
USER IMPACT:
- Agents will come online after backend deployment (no reinstall needed)
- Bulk import will correctly parse configs with multiple SSL certs + alpn
- Clear warnings shown in UI about SSL parameters not imported
- Add use-service skip to backend parser (consistent with frontend)
- Add use-service preservation to backend bulk import merge strategy
- Remove use-service merge from frontend PUT endpoint (allows user deletion)
- Ensures frontend/backend full consistency for use-service directives
Changes:
1. Backend parser now skips use-service directives during bulk import
2. Backend bulk import preserves manually-added use-service directives
3. Frontend PUT no longer prevents use-service deletion by users
Impact:
- Prevents loss of manually configured services during bulk imports
- Allows users to freely add/remove use-service directives via UI
- Frontend and Backend now have identical use-service handling logic
Related fixes:
- Bug 2: use-service deleted on bulk import (now preserved)
- Bug 3: use-service cannot be deleted via UI (now deletable)
CRITICAL FIX: Prevent 'option httpchk' duplication in HAProxy config by implementing 3-layer validation:
1. BULK IMPORT PARSER:
- Frontend: Filter out 'option httpchk' with warning (not applicable to frontends)
- Backend: Already filtering 'option httpchk' (handled by health_check_uri field)
2. BACKEND API:
- Backend create/update: Auto-filter 'option httpchk' from options field
- Frontend create/update: Auto-filter 'option httpchk' from options field
- Added filter_httpchk_from_options() helper function in both routers
3. FRONTEND UI:
- Backend modal: Real-time warning when 'option httpchk' is typed
- Frontend modal: Real-time warning when 'option httpchk' is typed
- Warning messages guide users to use proper fields instead
Changes:
- backend/utils/haproxy_config_parser.py: Added httpchk filtering for frontend parsing
- backend/routers/backend.py: Added filter function + applied to create/update
- backend/routers/frontend.py: Added filter function + applied to create/update
- frontend/src/components/BackendServers.js: Added dynamic warning for httpchk
- frontend/src/components/FrontendManagement.js: Added dynamic warning for httpchk
User Experience:
✅ Bulk Import: Automatically filters httpchk, shows warning in preview
✅ Manual Entry: Shows real-time warning, auto-filters on save
✅ No Config Duplication: 'option httpchk' never appears twice in generated config
Impact: Users can safely paste or type 'option httpchk' without breaking HAProxy config. System automatically filters it and guides users to use the Health Check URI field instead.
Implemented comprehensive HAProxy options field support for both backend and frontend entities to enable standard HAProxy directives like 'option http-keep-alive', 'option httplog', 'option forwardfor', etc.
Changes:
- Database: Added 'options' TEXT column to backends and frontends tables
- Models: Added options field to BackendConfig, BackendConfigUpdate, and FrontendConfig
- API Endpoints: Updated CREATE, UPDATE, and GET endpoints to handle options field
* Backend: CREATE/UPDATE/GET with options support
* Frontend: CREATE/UPDATE/GET with options support (fixed 5 SELECT queries)
- Config Generator: Added options block generation for both backends and frontends
- Bulk Import Parser:
* Added options field to ParsedBackend and ParsedFrontend dataclasses
* Implemented option directive parsing with validation
* Added unknown option warnings
* Fixed bulk parse response to include options field
- Bulk Import Merge: Added options field comparison in UPDATE logic
- UI Components:
* BackendServers.js: Added options TextArea form field
* FrontendManagement.js: Added options TextArea form field
Features:
- Multi-line options support (newline-separated format)
- Option validation with known HAProxy options list
- Backward compatible (NULL options for existing entities)
- Bulk import support with merge strategy
- Full CRUD support for both manual and bulk operations
Technical Details:
- Format: Newline-separated TEXT field for multiple options
- Validation: Warns about unknown options but allows them
- Config Generation: Each option written as separate directive
- Agent: Standard HAProxy config validation applies
Total: 10 files modified, ~195 lines added, 26 integration points verified
🐛 Critical Bug Fix:
- Fixed 'list' object has no attribute 'strip' error
- Error occurred in _validate_parsed_config() at line 847
- use_backend_rules is now a list, not a string
🔧 Technical Details:
- Changed from: frontend.use_backend_rules.strip()
- Changed to: bool(frontend.use_backend_rules)
- Simple boolean check works for both list and None types
✅ Impact:
- Bulk import parsing now works without errors
- Config validation properly handles list-based use_backend_rules
- All warning messages display correctly
Error was:
'list' object has no attribute 'strip'
at _validate_parsed_config line 847
Fix applied:
Line 847-848: Use bool() instead of .strip() for list validation
🐛 Critical Bug Fixes:
- Fixed duplicate 'acl' prefix in generated config (was: 'acl acl Name ...')
- Fixed duplicate 'use_backend' prefix in generated config
- Added use_backend directive parsing from bulk import configs
- Fixed redirect_rules list handling (was causing .strip() error)
🔧 Parser Improvements:
- Added use_backend rules parsing (stored as list like ACL rules)
- Changed use_backend_rules field from str to list for consistency
- Parser now captures all use_backend directives with conditions
🎯 Config Generation Improvements:
- Smart prefix detection: only add 'acl' if not already present
- Smart prefix detection: only add 'use_backend' if not already present
- Support both legacy (string) and new (list) format for rules
- Proper JSON parsing with fallback to newline-separated format
✅ HAProxy Validation:
- Generated config now passes HAProxy validation (haproxy -c -f)
- ACL and use_backend directives in correct HAProxy format
- Routing rules properly linked with ACL conditions
Example parsed config:
acl Elasticsearch hdr(host) -i baremetal-elastic.burgan.com.tr
use_backend Elasticsearch if Elasticsearch
Tested with full config including multiple ACLs and routing rules.
taylanbakircioglu