Commit Graph

3 Commits

Author SHA1 Message Date
Alphaeus Mote e2090cc6ad Add Python dependency and start CSV-based data generation
DEPENDENCY INSTALLER UPDATES:

- Added Python 3 installation check and installer

- Python is required for fast CSV data generation

- Updated step numbers from [1/4] to [1/5]

- Python installed before PostgreSQL client

- Supports Ubuntu, Debian, CentOS, RHEL, Fedora, Arch, macOS

DEPLOY SCRIPT UPDATES:

- Added automatic dependency checking (Step 3/8)

- Checks for python3, psql, and docker

- Automatically runs install-dependencies.sh if missing

- Updated all step numbers to reflect 8 total steps

- Step 3: Check dependencies (NEW)

- Step 4: Fix permissions (was Step 3)

- Step 5: Start containers (was Step 4)

- Step 6: Initialize schema (was Step 5)

- Step 7: Generate data (was Step 6)

- Step 8: Verify deployment (was Step 7)

CSV DATA GENERATION (IN PROGRESS):

- Created scripts/generate_csv_data.py

- Python-based CSV generator for bulk import

- Much faster than SQL INSERT statements

- Generates realistic data with proper relationships

- Partial implementation (customers, accounts, merchants, cards)

- Next: Complete transactions and analytics data

RATIONALE:

- Current bash-based data generation has transaction commit issues

- Data shows as generated but doesn't persist (0 rows after completion)

- CSV + COPY command is PostgreSQL best practice for bulk loading

- 10-100x faster than INSERT statements

- More reliable - atomic COPY operations

- Easier to debug - can inspect CSV files
2025-10-24 12:43:39 -04:00
Alphaeus Mote 75fa7e0159 Fix container startup timing and pgAdmin health check issues
DEPLOY SCRIPT IMPROVEMENTS:

- Start PostgreSQL first with --no-deps flag

- Wait for PostgreSQL to be healthy (increased to 120 seconds)

- Start pgAdmin separately after PostgreSQL is ready

- Added note that pgAdmin takes 30-60 seconds to initialize

- pgAdmin startup no longer blocks database operations

DOCKER COMPOSE FIX:

- Removed health check condition from pgAdmin depends_on

- Changed from 'condition: service_healthy' to simple dependency

- pgAdmin can start independently without waiting for PostgreSQL health

- Prevents 'Container is unhealthy' errors during startup

TROUBLESHOOTING SECTION:

- Added troubleshooting tips to deploy.sh output

- Command to check pgAdmin status: docker logs business_analytics_ui

- Instructions to wait for pgAdmin initialization

- Look for 'Listening at: http://[::]:80' in logs

BENEFITS:

- No more 'Container is unhealthy' errors

- Database operations can proceed while pgAdmin initializes

- Clear user expectations about pgAdmin startup time

- Better error handling and troubleshooting guidance
2025-10-24 12:22:35 -04:00
Alphaeus Mote 3e927fc5a8 Add master deployment script for one-command setup
NEW SCRIPT: deploy.sh - Master deployment automation

- Tears down existing containers and volumes

- Fixes file permissions automatically

- Starts fresh containers

- Initializes database schema

- Generates test data

- Verifies deployment

- IDEMPOTENT: Safe to run multiple times

FEATURES:

- Beautiful colored output with progress indicators

- Confirmation prompt before destructive operations

- Waits for PostgreSQL to be healthy before proceeding

- Comprehensive access information at completion

- Useful commands reference

DOCUMENTATION:

- Updated README.md with Option A (one-command) and Option B (manual)

- Updated QUICKSTART.md with super quick start section

- Manual steps now in collapsible section

USER EXPERIENCE:

- Clone repo + run deploy.sh = DONE

- No more complex multi-step setup

- Perfect for demos and quick testing

- Rebuilds from scratch every time (no stale data)
2025-10-24 12:18:06 -04:00