PRODUCTION ENHANCEMENT: Handle extremely large stats CSV payloads
IMPROVEMENT OVER PREVIOUS FIX:
- Previous: Temp file for response only
- Now: Temp file for BOTH payload and response
- Reason: Very large payloads (>1MB) still hit argument limits
PRODUCTION SCENARIO:
- Large HAProxy instances with 100+ backends
- Stats CSV can exceed 1MB in production
- curl --data argument hits system limits
- Need temp file for payload itself
SOLUTION:
- Write heartbeat_payload to temp file
- Use curl --data-binary @temp_payload
- Write response to separate temp file
- Read HTTP code and response body
- Cleanup both temp files
BENEFITS:
- Unlimited payload size support
- No argument list limits
- Production-tested and safe
- Backward compatible
FILES CHANGED:
- backend/utils/agent_scripts/linux_install.sh
- backend/utils/agent_scripts/macos_install.sh
- Updated both embedded daemon (Line ~1099) and installer (Line ~2367)
PRODUCTION BUG: Argument list too long when sending large stats CSV
ERROR MESSAGE:
"Heartbeat failed (HTTP /usr/local/bin/haproxy-agent: line 417: /usr/bin/curl: Argument list too long)"
ROOT CAUSE:
- curl output capture exceeded system argument list limit
- Large stats CSV (>200KB, production can be >1MB)
- Shell variable assignment hit system limits
SOLUTION:
- Redirect curl output to temp file (/tmp/heartbeat_response_$$.txt)
- Read HTTP code and response body from temp file
- Cleanup temp file immediately after use
BENEFITS:
- No size limit on HTTP responses
- Production-safe for large HAProxy instances
- Better error handling with detailed logging
FILES CHANGED:
- backend/utils/agent_scripts/linux_install.sh
- backend/utils/agent_scripts/macos_install.sh
- Updated both embedded daemon and installer functions
This is a comprehensive update that adds SSL certificate differentiation
for frontend (HAProxy bind) and server (backend verification) use cases.
FEATURES:
- SSL certificates can be marked as 'frontend' or 'server' usage type
- Frontend SSL: Private key REQUIRED (for HAProxy bind ssl crt)
- Server SSL: Private key OPTIONAL (CA cert only for backend verification)
- UI dropdown for usage type selection
- Dynamic form validation based on usage type
- Filtering: Frontends see only Frontend SSL, Backends see only Server SSL
DATABASE:
- Added usage_type column to ssl_certificates (default: 'frontend')
- Made private_key_content nullable for server SSL support
- Migration automatically runs on pod restart
BACKEND:
- Pydantic v2 compatibility (@field_validator, @model_validator)
- SSL router: usage_type filtering support
- Agent endpoint: usage_type field included
- Improved migration robustness with better error handling
- Fixed duplicate ensure_agents_table() function
- Fixed JSONB permissions insert with json.dumps()
- Fixed ON CONFLICT constraints with explicit checks
FRONTEND:
- SSL Management: Usage Type dropdown with visual feedback
- Frontend Management: Filters only Frontend SSL certificates
- Backend Servers: Filters only Server SSL certificates
- Dynamic private key validation (required for Frontend, optional for Server)
- Improved form UX with color-coded hints
AGENT SCRIPTS (Linux & macOS):
- Support for Server SSL without private key
- Conditional PEM file creation (cert+key vs cert-only)
- usage_type awareness in SSL deployment
- Backward compatible with existing Frontend SSL certificates
DOCKER:
- Increased npm timeout for slow networks (300s → 600s)
- Increased fetch-retries (5 → 10)
- Reduced maxsockets for stability (3 → 1)
All changes are backward compatible. Existing SSL certificates
default to 'frontend' type and continue working unchanged.
Tested with: HAProxy 2.8+, PostgreSQL 15, React 18
taylanbakircioglu