mirror of
https://github.com/freedbygrace/SQL.git
synced 2026-07-26 11:28:16 +00:00
aa803bd3bd
Major expansion from fraud detection to comprehensive business analytics: DATABASE CHANGES: - Renamed database from 'fraud_detection' to 'business_analytics' - Renamed user from 'fraud_analyst' to 'data_analyst' - Expanded from 20 to 39 tables across 4 business models NEW MODELS (19 tables): 1. Customer Analytics (5 tables): - customer_segments, customer_lifetime_value, churn_predictions - customer_satisfaction, engagement_metrics 2. Sales & Revenue Analytics (6 tables): - product_catalog, sales_transactions, sales_targets - sales_performance, revenue_forecasts 3. KPI & Metrics (8 tables): - kpi_definitions, daily_metrics, monthly_summaries - trend_analysis, dashboard_snapshots - report_definitions, report_executions, data_quality_checks DATA GENERATION: - Extended generate_data.sh with 6 new steps (now 15 total) - Added CLV calculations for all customers - Added churn predictions based on transaction recency - Added 30K customer satisfaction surveys - Added 1M sales transactions linked to 24 products - Added 90 days of daily KPI metrics - Added 24 months of business summaries SQL EXERCISES (3 new levels): - Level 2: Customer Analytics (10 exercises + 3 challenges) - Level 3: Sales & Revenue Analysis (12 exercises + 3 challenges) - Level 4: KPI Dashboards & Metrics (12 exercises + 3 challenges) DOCUMENTATION: - Updated README.md with business analytics focus - Updated QUICKSTART.md with new data generation steps - Updated SETUP_COMPLETE.md with 39-table architecture - Added DATA_MODELS.md with complete model specifications - Added WHATS_NEW.md with migration guide SEED DATA: - Added 8 customer segments (VIP, High Value, etc.) - Added 24 products across 5 categories - Added 16 KPI definitions across 4 categories - Added 8 standard report definitions All changes maintain idempotency and backward compatibility with existing fraud detection functionality.
7.2 KiB
7.2 KiB
Quick Start Guide
🚀 Get Up and Running in 5 Minutes
Step 1: Start Docker Containers (1 minute)
# From the project root directory
docker-compose up -d
What this does:
- Starts PostgreSQL 16 database
- Starts DB-UI web interface
- Creates network and volumes
Verify it's running:
docker-compose ps
You should see both business_analytics_db and business_analytics_ui running.
Step 2: Initialize Database Schema (1 minute)
# Make script executable (first time only)
chmod +x scripts/setup-database.sh
# Run setup
./scripts/setup-database.sh
What this does:
- Creates all 39 tables across 4 business models
- Sets up indexes and constraints
- Loads reference data (countries, merchant categories, customer segments, products, KPIs, etc.)
Expected output:
✓ PostgreSQL is ready
✓ Creating tables, indexes, and constraints
✓ Loading reference data
✓ Database setup completed successfully!
Step 3: Generate Test Data (15-30 minutes)
# Make script executable (first time only)
chmod +x data/generate_data.sh
# Run data generation
./data/generate_data.sh
What this does:
- Generates 100,000 customers
- Creates 150,000 accounts
- Generates 5,000,000 transactions
- Creates fraud patterns and alerts
- Generates customer analytics (CLV, churn, satisfaction)
- Creates sales data (1M sales records)
- Generates KPI metrics (90 days of daily metrics)
⏱️ Time estimate:
- Fast machine (SSD, 16GB RAM): ~15 minutes
- Average machine: ~20-25 minutes
- Slower machine: ~30 minutes
You can monitor progress: The script shows progress for each step:
[1/15] Loading geographic reference data...
[2/15] Generating Customers...
[3/15] Generating Accounts...
...
[10/15] Generating Customer Lifetime Value data...
[11/15] Generating Churn Predictions...
[12/15] Generating Customer Satisfaction data...
[13/15] Generating Sales Transactions...
[14/15] Generating Daily Metrics...
[15/15] Generating Monthly Summaries...
Step 4: Access the Database
Option A: DB-UI Web Interface (Recommended for Beginners)
- Open your browser to: http://localhost:3000
- You'll see the database tables in the sidebar
- Click any table to browse data
- Use the "Custom SQL" tab to run queries
Features:
- Visual table browser
- SQL query editor with syntax highlighting
- Export results to CSV
- Schema introspection
Option B: Command Line (psql)
docker exec -it business_analytics_db psql -U data_analyst -d business_analytics
Quick commands:
-- List all tables
\dt
-- Describe a table
\d customers
-- Run a query
SELECT COUNT(*) FROM transactions;
-- Exit
\q
Option C: Your Favorite SQL Client
Connection Details:
Host: localhost
Port: 5432
Database: business_analytics
Username: data_analyst
Password: SecurePass123!
Popular clients:
- DBeaver (free, cross-platform)
- pgAdmin (free, PostgreSQL-specific)
- DataGrip (paid, JetBrains)
- TablePlus (paid, macOS/Windows)
🎓 Your First Queries
1. Check Data Counts
-- How many customers?
SELECT COUNT(*) FROM customers;
-- How many transactions?
SELECT COUNT(*) FROM transactions;
-- How many sales records?
SELECT COUNT(*) FROM sales_transactions;
-- How many KPIs are being tracked?
SELECT COUNT(*) FROM kpi_definitions WHERE is_active = TRUE;
2. Customer Analytics: High-Value Customers
SELECT
c.customer_id,
c.first_name,
c.last_name,
clv.clv_score,
cs.segment_name
FROM customers c
JOIN customer_lifetime_value clv ON c.customer_id = clv.customer_id
JOIN customer_segments cs ON clv.segment_id = cs.segment_id
WHERE cs.segment_name IN ('VIP', 'High Value')
ORDER BY clv.clv_score DESC
LIMIT 10;
3. Sales Analytics: Top Products
SELECT
p.product_name,
p.product_category,
COUNT(st.transaction_id) as sales_count,
SUM(st.total_amount) as total_revenue
FROM product_catalog p
JOIN sales_transactions st ON p.product_id = st.product_id
GROUP BY p.product_id, p.product_name, p.product_category
ORDER BY total_revenue DESC
LIMIT 10;
4. KPI Dashboard: Current Status
SELECT
kd.kpi_name,
kd.kpi_category,
dm.metric_value,
kd.target_value,
dm.status
FROM kpi_definitions kd
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
WHERE dm.metric_date = CURRENT_DATE
AND kd.is_active = TRUE
ORDER BY kd.kpi_category, kd.kpi_name;
5. Fraud Detection: Flagged Transactions
SELECT
t.transaction_id,
t.amount,
t.fraud_score,
t.flagged_reason,
c.first_name,
c.last_name
FROM transactions t
JOIN accounts a ON t.account_id = a.account_id
JOIN customers c ON a.customer_id = c.customer_id
WHERE t.is_flagged = TRUE
ORDER BY t.fraud_score DESC
LIMIT 10;
📚 Next Steps
Start Learning SQL
- Begin with basics:
exercises/01-basic-queries/README.md - Progress through levels: Work through exercises 01-06
- Practice fraud detection:
exercises/06-fraud-detection/README.md
Explore the Data
-- What countries are represented?
SELECT country_name, COUNT(*) as customer_count
FROM customers c
JOIN countries co ON c.country_id = co.country_id
GROUP BY country_name
ORDER BY customer_count DESC;
-- What are the top merchant categories?
SELECT mc.category_name, COUNT(*) as transaction_count
FROM transactions t
JOIN merchants m ON t.merchant_id = m.merchant_id
JOIN merchant_categories mc ON m.category_id = mc.category_id
GROUP BY mc.category_name
ORDER BY transaction_count DESC;
-- How many fraud cases by type?
SELECT ft.fraud_name, COUNT(*) as case_count
FROM fraud_cases fc
JOIN fraud_types ft ON fc.fraud_type_id = ft.fraud_type_id
GROUP BY ft.fraud_name
ORDER BY case_count DESC;
🔧 Troubleshooting
Database won't start
# Check logs
docker-compose logs postgres
# Restart containers
docker-compose restart
Can't connect to database
# Check if PostgreSQL is ready
docker exec fraud_detection_db pg_isready -U fraud_analyst
# Check port is not in use
netstat -an | grep 5432
Data generation fails
# Check disk space
df -h
# Check memory
free -h
# Try with smaller dataset
# Edit data/generate_data.sh and reduce:
NUM_CUSTOMERS=10000
NUM_TRANSACTIONS=500000
Reset everything
# Stop and remove everything
docker-compose down -v
# Start fresh
docker-compose up -d
./scripts/setup-database.sh
./data/generate_data.sh
💡 Tips
- Use DB-UI for exploration - Great for browsing and understanding the schema
- Use psql for practice - Best for learning SQL commands
- Start simple - Begin with basic SELECT queries before complex joins
- Check the exercises - They're designed to build your skills progressively
- Experiment - The database is yours to explore and learn from!
🎯 Learning Goals
After completing this tutorial, you'll be able to:
- ✅ Write complex SQL queries
- ✅ Understand database relationships
- ✅ Detect fraud patterns in data
- ✅ Use window functions and CTEs
- ✅ Optimize queries with indexes
- ✅ Investigate financial crimes
Ready to start? Head to exercises/01-basic-queries/README.md! 🚀