mirror of
https://github.com/freedbygrace/SQL.git
synced 2026-07-26 11:28:16 +00:00
Expand to Business Analytics: Add Customer, Sales, and KPI models
Major expansion from fraud detection to comprehensive business analytics: DATABASE CHANGES: - Renamed database from 'fraud_detection' to 'business_analytics' - Renamed user from 'fraud_analyst' to 'data_analyst' - Expanded from 20 to 39 tables across 4 business models NEW MODELS (19 tables): 1. Customer Analytics (5 tables): - customer_segments, customer_lifetime_value, churn_predictions - customer_satisfaction, engagement_metrics 2. Sales & Revenue Analytics (6 tables): - product_catalog, sales_transactions, sales_targets - sales_performance, revenue_forecasts 3. KPI & Metrics (8 tables): - kpi_definitions, daily_metrics, monthly_summaries - trend_analysis, dashboard_snapshots - report_definitions, report_executions, data_quality_checks DATA GENERATION: - Extended generate_data.sh with 6 new steps (now 15 total) - Added CLV calculations for all customers - Added churn predictions based on transaction recency - Added 30K customer satisfaction surveys - Added 1M sales transactions linked to 24 products - Added 90 days of daily KPI metrics - Added 24 months of business summaries SQL EXERCISES (3 new levels): - Level 2: Customer Analytics (10 exercises + 3 challenges) - Level 3: Sales & Revenue Analysis (12 exercises + 3 challenges) - Level 4: KPI Dashboards & Metrics (12 exercises + 3 challenges) DOCUMENTATION: - Updated README.md with business analytics focus - Updated QUICKSTART.md with new data generation steps - Updated SETUP_COMPLETE.md with 39-table architecture - Added DATA_MODELS.md with complete model specifications - Added WHATS_NEW.md with migration guide SEED DATA: - Added 8 customer segments (VIP, High Value, etc.) - Added 24 products across 5 categories - Added 16 KPI definitions across 4 categories - Added 8 standard report definitions All changes maintain idempotency and backward compatibility with existing fraud detection functionality.
This commit is contained in:
Alphaeus Mote
@@ -1,44 +1,82 @@
|
||||
# Financial Fraud Detection - SQL Learning Database
|
||||
# Business Analytics - SQL Learning Database
|
||||
|
||||
A comprehensive, production-grade database designed for learning SQL through realistic financial fraud investigation scenarios.
|
||||
A comprehensive, production-grade database designed for learning SQL through realistic business analytics, customer insights, sales analysis, and fraud detection scenarios.
|
||||
|
||||
## 🎯 Overview
|
||||
|
||||
This project provides a complete PostgreSQL database with **5+ million transactions**, embedded fraud patterns, and progressive SQL exercises. Perfect for:
|
||||
This project provides a complete PostgreSQL database with **5+ million transactions**, **39 tables** across 4 business models, and progressive SQL exercises aligned with **Data Analyst** responsibilities. Perfect for:
|
||||
|
||||
- **SQL Beginners** → Learn fundamentals with real-world data
|
||||
- **Data Analysts** → Practice fraud detection queries
|
||||
- **Security Professionals** → Understand fraud patterns
|
||||
- **Students** → Hands-on financial crime investigation
|
||||
- **Data Analysts** → Practice customer analytics, sales analysis, KPI dashboards
|
||||
- **Business Analysts** → Understand customer behavior and revenue patterns
|
||||
- **Security Professionals** → Detect fraud patterns
|
||||
- **Students** → Hands-on business intelligence and analytics
|
||||
|
||||
## 📊 Database Statistics
|
||||
|
||||
### Core Data
|
||||
- **100,000** Customers with KYC data
|
||||
- **150,000** Bank accounts (checking, savings, credit)
|
||||
- **200,000** Payment cards
|
||||
- **50,000** Merchants across 35 categories
|
||||
- **5,000,000** Transactions (7% fraudulent)
|
||||
- **1,000,000** Sales records linked to products
|
||||
- **500,000** Login sessions
|
||||
|
||||
### Analytics Data
|
||||
- **100,000** Customer Lifetime Value calculations
|
||||
- **100,000** Churn predictions
|
||||
- **30,000** Customer satisfaction surveys
|
||||
- **24** Product catalog entries
|
||||
- **1,500+** Daily KPI metrics
|
||||
- **24** Monthly business summaries
|
||||
- **50,000+** Fraud alerts
|
||||
- **5,000+** Fraud cases
|
||||
- **500,000** Login sessions
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
### Data Model Features
|
||||
- ✅ **20+ Tables** with proper relationships
|
||||
- ✅ **39 Tables** across 4 business models
|
||||
- ✅ **Foreign key constraints** for data integrity
|
||||
- ✅ **Indexes** for query performance
|
||||
- ✅ **Realistic geographic data** (100 US cities, 210 world cities)
|
||||
- ✅ **Embedded fraud patterns** (velocity, geographic, structuring)
|
||||
- ✅ **Customer analytics** (CLV, churn, satisfaction, engagement)
|
||||
- ✅ **Sales analytics** (products, targets, forecasts)
|
||||
- ✅ **KPI tracking** (daily metrics, trends, dashboards)
|
||||
- ✅ **Fraud detection** (velocity, geographic, structuring patterns)
|
||||
- ✅ **Audit trails** and compliance tables
|
||||
|
||||
### Key Entities
|
||||
### Business Models
|
||||
|
||||
#### 1. Fraud Detection Model (20 tables)
|
||||
```
|
||||
customers → accounts → transactions
|
||||
↓
|
||||
cards → merchants
|
||||
↓
|
||||
alerts → fraud_cases
|
||||
customers → accounts → transactions → alerts → fraud_cases
|
||||
↓
|
||||
cards → merchants
|
||||
```
|
||||
|
||||
#### 2. Customer Analytics Model (5 tables)
|
||||
```
|
||||
customers → customer_lifetime_value → customer_segments
|
||||
→ churn_predictions
|
||||
→ customer_satisfaction
|
||||
→ engagement_metrics
|
||||
```
|
||||
|
||||
#### 3. Sales & Revenue Model (6 tables)
|
||||
```
|
||||
product_catalog → sales_transactions → sales_performance
|
||||
→ sales_targets
|
||||
→ revenue_forecasts
|
||||
```
|
||||
|
||||
#### 4. KPI & Metrics Model (8 tables)
|
||||
```
|
||||
kpi_definitions → daily_metrics → trend_analysis
|
||||
→ monthly_summaries
|
||||
→ dashboard_snapshots
|
||||
→ report_definitions → report_executions
|
||||
→ data_quality_checks
|
||||
```
|
||||
|
||||
## 🚀 Quick Start
|
||||
@@ -86,15 +124,15 @@ http://localhost:3000
|
||||
|
||||
**Option B: Command Line**
|
||||
```bash
|
||||
docker exec -it fraud_detection_db psql -U fraud_analyst -d fraud_detection
|
||||
docker exec -it business_analytics_db psql -U data_analyst -d business_analytics
|
||||
```
|
||||
|
||||
**Option C: Your Favorite SQL Client**
|
||||
```
|
||||
Host: localhost
|
||||
Port: 5432
|
||||
Database: fraud_detection
|
||||
Username: fraud_analyst
|
||||
Database: business_analytics
|
||||
Username: data_analyst
|
||||
Password: SecurePass123!
|
||||
```
|
||||
|
||||
@@ -106,29 +144,36 @@ Password: SecurePass123!
|
||||
- Basic comparisons
|
||||
- **Location:** `exercises/01-basic-queries/`
|
||||
|
||||
### Level 2: Joins
|
||||
- INNER JOIN, LEFT JOIN
|
||||
- Multiple table queries
|
||||
- Relationship navigation
|
||||
- **Location:** `exercises/02-joins/`
|
||||
### Level 2: Customer Analytics ⭐ NEW
|
||||
- Customer segmentation
|
||||
- Lifetime value (CLV) analysis
|
||||
- Churn prediction
|
||||
- Satisfaction metrics
|
||||
- Engagement tracking
|
||||
- **Location:** `exercises/02-customer-analytics/`
|
||||
|
||||
### Level 3: Aggregations
|
||||
- COUNT, SUM, AVG, MAX, MIN
|
||||
- GROUP BY and HAVING
|
||||
- Statistical analysis
|
||||
- **Location:** `exercises/03-aggregations/`
|
||||
### Level 3: Sales & Revenue Analysis ⭐ NEW
|
||||
- Sales performance vs targets
|
||||
- Product analytics
|
||||
- Channel and regional analysis
|
||||
- Revenue forecasting
|
||||
- Profitability analysis
|
||||
- **Location:** `exercises/03-sales-analysis/`
|
||||
|
||||
### Level 4: Subqueries
|
||||
- Nested queries
|
||||
- Correlated subqueries
|
||||
- EXISTS and IN
|
||||
- **Location:** `exercises/04-subqueries/`
|
||||
### Level 4: KPI Dashboards & Metrics ⭐ NEW
|
||||
- KPI tracking and monitoring
|
||||
- Trend analysis
|
||||
- Dashboard creation
|
||||
- Data quality monitoring
|
||||
- Executive reporting
|
||||
- **Location:** `exercises/04-kpi-dashboards/`
|
||||
|
||||
### Level 5: Window Functions
|
||||
- ROW_NUMBER, RANK, DENSE_RANK
|
||||
- Running totals
|
||||
- Moving averages
|
||||
- **Location:** `exercises/05-window-functions/`
|
||||
### Level 5: Advanced SQL Techniques
|
||||
- Window functions (ROW_NUMBER, RANK)
|
||||
- Common Table Expressions (CTEs)
|
||||
- Running totals and moving averages
|
||||
- Complex aggregations
|
||||
- **Location:** `exercises/05-advanced-sql/`
|
||||
|
||||
### Level 6: Fraud Detection
|
||||
- Velocity fraud detection
|
||||
@@ -137,6 +182,40 @@ Password: SecurePass123!
|
||||
- Account takeover patterns
|
||||
- **Location:** `exercises/06-fraud-detection/`
|
||||
|
||||
## 💼 Data Analyst Use Cases
|
||||
|
||||
This database supports typical **Data Analyst** responsibilities:
|
||||
|
||||
### Routine Analysis
|
||||
- Daily sales summaries
|
||||
- Customer acquisition metrics
|
||||
- Transaction volume tracking
|
||||
- Basic KPI monitoring
|
||||
|
||||
### Semi-Routine Reporting
|
||||
- Weekly customer analytics
|
||||
- Monthly revenue reports
|
||||
- Product performance analysis
|
||||
- Churn risk identification
|
||||
|
||||
### Dashboard Creation
|
||||
- Executive KPI dashboards
|
||||
- Sales performance dashboards
|
||||
- Customer health dashboards
|
||||
- Operational metrics dashboards
|
||||
|
||||
### Trend Identification
|
||||
- Revenue trends (MoM, YoY)
|
||||
- Customer behavior patterns
|
||||
- Product sales seasonality
|
||||
- Engagement score trends
|
||||
|
||||
### Data Quality
|
||||
- Missing data detection
|
||||
- Anomaly identification
|
||||
- Validation checks
|
||||
- Data completeness monitoring
|
||||
|
||||
## 🔍 Fraud Patterns Included
|
||||
|
||||
### 1. Velocity Fraud
|
||||
@@ -180,18 +259,19 @@ SQL/
|
||||
│ ├── world_cities.csv
|
||||
│ └── load_geographic_data.sql
|
||||
├── scripts/
|
||||
│ └── setup-database.sh # Setup automation
|
||||
│ ├── setup-database.sh # Setup automation
|
||||
│ └── verify-setup.sh # Verification script
|
||||
├── exercises/
|
||||
│ ├── 01-basic-queries/
|
||||
│ ├── 02-joins/
|
||||
│ ├── 03-aggregations/
|
||||
│ ├── 04-subqueries/
|
||||
│ ├── 05-window-functions/
|
||||
│ └── 06-fraud-detection/
|
||||
│ ├── 01-basic-queries/ # SQL fundamentals
|
||||
│ ├── 02-customer-analytics/ # ⭐ Customer insights
|
||||
│ ├── 03-sales-analysis/ # ⭐ Sales & revenue
|
||||
│ ├── 04-kpi-dashboards/ # ⭐ KPI tracking
|
||||
│ ├── 05-advanced-sql/ # Advanced techniques
|
||||
│ └── 06-fraud-detection/ # Fraud patterns
|
||||
└── docs/
|
||||
├── data-model.md
|
||||
├── fraud-patterns.md
|
||||
└── setup-guide.md
|
||||
├── DATA_MODELS.md # ⭐ Complete model documentation
|
||||
├── WHATS_NEW.md # ⭐ Recent changes
|
||||
└── QUICKSTART.md # Quick start guide
|
||||
```
|
||||
|
||||
## 🔧 Configuration
|
||||
@@ -200,8 +280,8 @@ SQL/
|
||||
Edit `docker-compose.yml` to customize:
|
||||
|
||||
```yaml
|
||||
POSTGRES_DB: fraud_detection
|
||||
POSTGRES_USER: fraud_analyst
|
||||
POSTGRES_DB: business_analytics
|
||||
POSTGRES_USER: data_analyst
|
||||
POSTGRES_PASSWORD: SecurePass123!
|
||||
```
|
||||
|
||||
@@ -224,15 +304,46 @@ All scripts are **idempotent** - safe to run multiple times:
|
||||
|
||||
## 🎓 Sample Queries
|
||||
|
||||
### Find High-Risk Customers
|
||||
### Customer Analytics: High-Value Customers
|
||||
```sql
|
||||
SELECT customer_id, first_name, last_name, risk_score
|
||||
FROM customers
|
||||
WHERE risk_score > 80
|
||||
ORDER BY risk_score DESC;
|
||||
SELECT
|
||||
c.customer_id, c.first_name, c.last_name,
|
||||
clv.clv_score, cs.segment_name
|
||||
FROM customers c
|
||||
JOIN customer_lifetime_value clv ON c.customer_id = clv.customer_id
|
||||
JOIN customer_segments cs ON clv.segment_id = cs.segment_id
|
||||
WHERE cs.segment_name IN ('VIP', 'High Value')
|
||||
ORDER BY clv.clv_score DESC
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
### Detect Velocity Fraud
|
||||
### Sales Analytics: Top Products
|
||||
```sql
|
||||
SELECT
|
||||
p.product_name, p.product_category,
|
||||
COUNT(st.transaction_id) as sales_count,
|
||||
SUM(st.total_amount) as total_revenue
|
||||
FROM product_catalog p
|
||||
JOIN sales_transactions st ON p.product_id = st.product_id
|
||||
GROUP BY p.product_id, p.product_name, p.product_category
|
||||
ORDER BY total_revenue DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
### KPI Dashboard: Current Status
|
||||
```sql
|
||||
SELECT
|
||||
kd.kpi_name, kd.kpi_category,
|
||||
dm.metric_value, kd.target_value,
|
||||
dm.status
|
||||
FROM kpi_definitions kd
|
||||
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
WHERE dm.metric_date = CURRENT_DATE
|
||||
AND kd.is_active = TRUE
|
||||
ORDER BY kd.kpi_category, kd.kpi_name;
|
||||
```
|
||||
|
||||
### Fraud Detection: Velocity Fraud
|
||||
```sql
|
||||
SELECT account_id, COUNT(*) as txn_count, SUM(amount) as total
|
||||
FROM transactions
|
||||
|
||||
+31
-5
@@ -1,4 +1,4 @@
|
||||
# 🎉 Financial Fraud Detection Database - Setup Complete!
|
||||
# 🎉 Business Analytics Database - Setup Complete!
|
||||
|
||||
## ✅ What Has Been Created
|
||||
|
||||
@@ -9,9 +9,11 @@
|
||||
- ✅ Persistent volumes for data
|
||||
- ✅ Health checks and auto-restart
|
||||
|
||||
### 2. **Database Schema (20+ Tables)**
|
||||
### 2. **Database Schema (39 Tables Across 4 Business Models)**
|
||||
|
||||
#### Core Tables
|
||||
#### Model 1: Fraud Detection (20 tables)
|
||||
|
||||
**Core Tables:**
|
||||
- ✅ `customers` - 100K customer records with KYC data
|
||||
- ✅ `accounts` - 150K bank accounts (checking, savings, credit)
|
||||
- ✅ `cards` - 200K payment cards
|
||||
@@ -19,13 +21,13 @@
|
||||
- ✅ `merchants` - 50K merchants across 35 categories
|
||||
- ✅ `devices` - 75K device fingerprints
|
||||
|
||||
#### Fraud Detection Tables
|
||||
**Fraud Detection Tables:**
|
||||
- ✅ `alerts` - System-generated fraud alerts
|
||||
- ✅ `fraud_cases` - Confirmed fraud investigations
|
||||
- ✅ `case_transactions` - Links transactions to cases
|
||||
- ✅ `case_alerts` - Links alerts to cases
|
||||
|
||||
#### Supporting Tables
|
||||
**Supporting Tables:**
|
||||
- ✅ `countries` - 40 countries with risk levels
|
||||
- ✅ `merchant_categories` - 35 MCC categories
|
||||
- ✅ `transaction_types` - 15 transaction types
|
||||
@@ -37,6 +39,30 @@
|
||||
- ✅ `suspicious_activity_reports` - SAR filings
|
||||
- ✅ `audit_log` - Complete audit trail
|
||||
|
||||
#### Model 2: Customer Analytics (5 tables) ⭐ NEW
|
||||
- ✅ `customer_segments` - Customer classification (VIP, High Value, etc.)
|
||||
- ✅ `customer_lifetime_value` - CLV calculations for all customers
|
||||
- ✅ `churn_predictions` - Customer retention risk analysis
|
||||
- ✅ `customer_satisfaction` - NPS/CSAT scores and feedback
|
||||
- ✅ `engagement_metrics` - Customer interaction tracking
|
||||
|
||||
#### Model 3: Sales & Revenue Analytics (6 tables) ⭐ NEW
|
||||
- ✅ `product_catalog` - 24 products across categories
|
||||
- ✅ `sales_transactions` - 1M sales records linked to products
|
||||
- ✅ `sales_targets` - Performance goals and targets
|
||||
- ✅ `sales_performance` - Aggregated performance metrics
|
||||
- ✅ `revenue_forecasts` - Revenue predictions and variance
|
||||
|
||||
#### Model 4: KPI & Metrics (8 tables) ⭐ NEW
|
||||
- ✅ `kpi_definitions` - Master KPI catalog (16 KPIs)
|
||||
- ✅ `daily_metrics` - Daily operational snapshots (90 days)
|
||||
- ✅ `monthly_summaries` - Monthly business summaries (24 months)
|
||||
- ✅ `trend_analysis` - Statistical trend tracking
|
||||
- ✅ `dashboard_snapshots` - Pre-calculated dashboard data
|
||||
- ✅ `report_definitions` - Standard report catalog
|
||||
- ✅ `report_executions` - Report run history
|
||||
- ✅ `data_quality_checks` - Data validation tracking
|
||||
|
||||
### 3. **Realistic Geographic Data**
|
||||
- ✅ 100 US cities with matching states
|
||||
- ✅ 210 world cities across 40 countries
|
||||
|
||||
+270
-16
@@ -1,9 +1,9 @@
|
||||
#!/bin/bash
|
||||
|
||||
# ============================================================================
|
||||
# Financial Fraud Detection - Data Generation Script - IDEMPOTENT
|
||||
# Business Analytics - Data Generation Script - IDEMPOTENT
|
||||
# ============================================================================
|
||||
# Generates realistic test data with embedded fraud patterns
|
||||
# Generates realistic test data for business analytics and reporting
|
||||
# This script is IDEMPOTENT - it will clear and regenerate all data
|
||||
# ============================================================================
|
||||
|
||||
@@ -19,8 +19,8 @@ NC='\033[0m' # No Color
|
||||
# Configuration
|
||||
DB_HOST="${POSTGRES_HOST:-localhost}"
|
||||
DB_PORT="${POSTGRES_PORT:-5432}"
|
||||
DB_NAME="${POSTGRES_DB:-fraud_detection}"
|
||||
DB_USER="${POSTGRES_USER:-fraud_analyst}"
|
||||
DB_NAME="${POSTGRES_DB:-business_analytics}"
|
||||
DB_USER="${POSTGRES_USER:-data_analyst}"
|
||||
DB_PASSWORD="${POSTGRES_PASSWORD:-SecurePass123!}"
|
||||
|
||||
# Data volumes
|
||||
@@ -33,7 +33,7 @@ NUM_TRANSACTIONS=5000000
|
||||
FRAUD_PERCENTAGE=7 # 7% of transactions will be fraudulent
|
||||
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo -e "${BLUE}Financial Fraud Detection Database - Data Generation${NC}"
|
||||
echo -e "${BLUE}Business Analytics Database - Data Generation${NC}"
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo -e "Target Database: ${GREEN}$DB_NAME@$DB_HOST:$DB_PORT${NC}"
|
||||
echo -e "Customers: ${GREEN}$NUM_CUSTOMERS${NC}"
|
||||
@@ -62,7 +62,7 @@ execute_sql_file() {
|
||||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
|
||||
# Clear existing data (preserve reference tables)
|
||||
echo -e "${YELLOW}[0/9] Clearing existing data...${NC}"
|
||||
echo -e "${YELLOW}[0/15] Clearing existing data...${NC}"
|
||||
execute_sql "TRUNCATE TABLE audit_log CASCADE;"
|
||||
execute_sql "TRUNCATE TABLE suspicious_activity_reports CASCADE;"
|
||||
execute_sql "TRUNCATE TABLE case_alerts CASCADE;"
|
||||
@@ -83,12 +83,12 @@ echo -e "${GREEN}✓ Existing data cleared${NC}"
|
||||
echo ""
|
||||
|
||||
# Load geographic reference data
|
||||
echo -e "${YELLOW}[1/9] Loading geographic reference data...${NC}"
|
||||
echo -e "${YELLOW}[1/15] Loading geographic reference data...${NC}"
|
||||
execute_sql_file "$SCRIPT_DIR/reference/load_geographic_data.sql" > /dev/null
|
||||
echo -e "${GREEN}✓ Geographic data loaded (100 US cities, 210 world cities)${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[2/9] Generating Customers...${NC}"
|
||||
echo -e "${YELLOW}[2/15] Generating Customers...${NC}"
|
||||
cat > /tmp/generate_customers.sql << 'EOF'
|
||||
-- Generate customers with realistic geographic data
|
||||
INSERT INTO customers (
|
||||
@@ -130,7 +130,7 @@ execute_sql_file /tmp/generate_customers.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated $NUM_CUSTOMERS customers${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[3/9] Generating Accounts...${NC}"
|
||||
echo -e "${YELLOW}[3/15] Generating Accounts...${NC}"
|
||||
cat > /tmp/generate_accounts.sql << 'EOF'
|
||||
-- Generate accounts (1-2 accounts per customer on average)
|
||||
INSERT INTO accounts (
|
||||
@@ -177,7 +177,7 @@ execute_sql_file /tmp/generate_accounts.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated $NUM_ACCOUNTS accounts${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[4/9] Generating Merchants...${NC}"
|
||||
echo -e "${YELLOW}[4/15] Generating Merchants...${NC}"
|
||||
cat > /tmp/generate_merchants.sql << 'EOF'
|
||||
-- Generate merchants with realistic geographic data
|
||||
INSERT INTO merchants (
|
||||
@@ -237,7 +237,7 @@ execute_sql_file /tmp/generate_merchants.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated $NUM_MERCHANTS merchants${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[5/9] Generating Devices...${NC}"
|
||||
echo -e "${YELLOW}[5/15] Generating Devices...${NC}"
|
||||
cat > /tmp/generate_devices.sql << 'EOF'
|
||||
-- Generate devices
|
||||
INSERT INTO devices (
|
||||
@@ -282,7 +282,7 @@ execute_sql_file /tmp/generate_devices.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated $NUM_DEVICES devices${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[6/9] Generating Cards...${NC}"
|
||||
echo -e "${YELLOW}[6/15] Generating Cards...${NC}"
|
||||
cat > /tmp/generate_cards.sql << 'EOF'
|
||||
-- Generate cards (1-2 cards per account on average)
|
||||
INSERT INTO cards (
|
||||
@@ -326,7 +326,7 @@ execute_sql_file /tmp/generate_cards.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated $NUM_CARDS cards${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[7/9] Generating Login Sessions...${NC}"
|
||||
echo -e "${YELLOW}[7/15] Generating Login Sessions...${NC}"
|
||||
cat > /tmp/generate_sessions.sql << 'EOF'
|
||||
-- Generate login sessions with realistic geographic data
|
||||
INSERT INTO login_sessions (
|
||||
@@ -359,7 +359,7 @@ execute_sql_file /tmp/generate_sessions.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated 500,000 login sessions${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[8/9] Generating Transactions (this may take a while)...${NC}"
|
||||
echo -e "${YELLOW}[8/15] Generating Transactions (this may take a while)...${NC}"
|
||||
echo -e "${BLUE}This step generates $NUM_TRANSACTIONS transactions with fraud patterns${NC}"
|
||||
|
||||
# Generate transactions in batches to avoid memory issues
|
||||
@@ -433,7 +433,7 @@ done
|
||||
echo -e "${GREEN}✓ Generated $NUM_TRANSACTIONS transactions${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[9/9] Generating Fraud Cases and Alerts...${NC}"
|
||||
echo -e "${YELLOW}[9/15] Generating Fraud Cases and Alerts...${NC}"
|
||||
|
||||
# Generate alerts for flagged transactions
|
||||
execute_sql "
|
||||
@@ -535,7 +535,261 @@ case_count=$(execute_sql "SELECT COUNT(*) FROM fraud_cases;" | grep -E '^\s*[0-9
|
||||
echo -e "Fraud Cases: ${GREEN}$case_count${NC}"
|
||||
|
||||
echo ""
|
||||
echo -e "${YELLOW}Ready for SQL learning and fraud investigation!${NC}"
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo -e "${BLUE}Generating Analytics Data (Customer, Sales, KPIs)${NC}"
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo ""
|
||||
|
||||
# ============================================================================
|
||||
# GENERATE CUSTOMER ANALYTICS DATA
|
||||
# ============================================================================
|
||||
|
||||
echo -e "${YELLOW}[10/15] Generating Customer Lifetime Value data...${NC}"
|
||||
cat > /tmp/generate_clv.sql << 'EOF'
|
||||
-- Generate CLV for all customers based on their transaction history
|
||||
INSERT INTO customer_lifetime_value (
|
||||
customer_id, calculation_date, total_revenue, total_transactions,
|
||||
average_order_value, predicted_future_value, clv_score, segment_id
|
||||
)
|
||||
SELECT
|
||||
c.customer_id,
|
||||
CURRENT_DATE as calculation_date,
|
||||
COALESCE(SUM(t.amount), 0) as total_revenue,
|
||||
COUNT(t.transaction_id) as total_transactions,
|
||||
COALESCE(AVG(t.amount), 0) as average_order_value,
|
||||
COALESCE(SUM(t.amount) * 1.5, 0) as predicted_future_value,
|
||||
COALESCE(SUM(t.amount) / 100, 0) as clv_score,
|
||||
CASE
|
||||
WHEN COALESCE(SUM(t.amount), 0) >= 50000 THEN 1 -- VIP
|
||||
WHEN COALESCE(SUM(t.amount), 0) >= 10000 THEN 2 -- High Value
|
||||
WHEN COALESCE(SUM(t.amount), 0) >= 2000 THEN 3 -- Medium Value
|
||||
WHEN COALESCE(SUM(t.amount), 0) >= 500 THEN 4 -- Low Value
|
||||
ELSE 6 -- New Customer
|
||||
END as segment_id
|
||||
FROM customers c
|
||||
LEFT JOIN accounts a ON c.customer_id = a.customer_id
|
||||
LEFT JOIN transactions t ON a.account_id = t.account_id
|
||||
GROUP BY c.customer_id;
|
||||
EOF
|
||||
|
||||
execute_sql_file /tmp/generate_clv.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated CLV for all customers${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[11/15] Generating Churn Predictions...${NC}"
|
||||
cat > /tmp/generate_churn.sql << 'EOF'
|
||||
-- Generate churn predictions based on transaction recency
|
||||
INSERT INTO churn_predictions (
|
||||
customer_id, prediction_date, churn_probability, risk_level,
|
||||
last_transaction_date, days_since_last_transaction, engagement_score
|
||||
)
|
||||
SELECT
|
||||
c.customer_id,
|
||||
CURRENT_DATE as prediction_date,
|
||||
CASE
|
||||
WHEN MAX(t.transaction_date) IS NULL THEN 90
|
||||
WHEN CURRENT_DATE - MAX(t.transaction_date) > 180 THEN 85
|
||||
WHEN CURRENT_DATE - MAX(t.transaction_date) > 90 THEN 60
|
||||
WHEN CURRENT_DATE - MAX(t.transaction_date) > 30 THEN 30
|
||||
ELSE 10
|
||||
END as churn_probability,
|
||||
CASE
|
||||
WHEN MAX(t.transaction_date) IS NULL THEN 'HIGH'
|
||||
WHEN CURRENT_DATE - MAX(t.transaction_date) > 180 THEN 'CRITICAL'
|
||||
WHEN CURRENT_DATE - MAX(t.transaction_date) > 90 THEN 'HIGH'
|
||||
WHEN CURRENT_DATE - MAX(t.transaction_date) > 30 THEN 'MEDIUM'
|
||||
ELSE 'LOW'
|
||||
END as risk_level,
|
||||
MAX(t.transaction_date) as last_transaction_date,
|
||||
COALESCE(CURRENT_DATE - MAX(t.transaction_date), 999) as days_since_last_transaction,
|
||||
CASE
|
||||
WHEN MAX(t.transaction_date) IS NULL THEN 0
|
||||
WHEN CURRENT_DATE - MAX(t.transaction_date) > 180 THEN 10
|
||||
WHEN CURRENT_DATE - MAX(t.transaction_date) > 90 THEN 30
|
||||
WHEN CURRENT_DATE - MAX(t.transaction_date) > 30 THEN 60
|
||||
ELSE 90
|
||||
END as engagement_score
|
||||
FROM customers c
|
||||
LEFT JOIN accounts a ON c.customer_id = a.customer_id
|
||||
LEFT JOIN transactions t ON a.account_id = t.account_id
|
||||
GROUP BY c.customer_id;
|
||||
EOF
|
||||
|
||||
execute_sql_file /tmp/generate_churn.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated churn predictions${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[12/15] Generating Customer Satisfaction data...${NC}"
|
||||
cat > /tmp/generate_satisfaction.sql << 'EOF'
|
||||
-- Generate satisfaction scores for random sample of customers
|
||||
INSERT INTO customer_satisfaction (
|
||||
customer_id, survey_date, nps_score, csat_score, category, sentiment
|
||||
)
|
||||
SELECT
|
||||
customer_id,
|
||||
TIMESTAMP '2023-01-01' + (random() * 730)::INT * INTERVAL '1 day' as survey_date,
|
||||
(random() * 200 - 100)::INT as nps_score,
|
||||
(1 + random() * 4)::DECIMAL(3,2) as csat_score,
|
||||
CASE (random() * 5)::INT
|
||||
WHEN 0 THEN 'PRODUCT'
|
||||
WHEN 1 THEN 'SERVICE'
|
||||
WHEN 2 THEN 'SUPPORT'
|
||||
WHEN 3 THEN 'BILLING'
|
||||
ELSE 'OTHER'
|
||||
END as category,
|
||||
CASE
|
||||
WHEN random() < 0.6 THEN 'POSITIVE'
|
||||
WHEN random() < 0.85 THEN 'NEUTRAL'
|
||||
ELSE 'NEGATIVE'
|
||||
END as sentiment
|
||||
FROM customers
|
||||
WHERE random() < 0.3 -- 30% of customers have satisfaction data
|
||||
LIMIT 30000;
|
||||
EOF
|
||||
|
||||
execute_sql_file /tmp/generate_satisfaction.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated ~30,000 satisfaction records${NC}"
|
||||
echo ""
|
||||
|
||||
# ============================================================================
|
||||
# GENERATE SALES ANALYTICS DATA
|
||||
# ============================================================================
|
||||
|
||||
echo -e "${YELLOW}[13/15] Generating Sales Transactions...${NC}"
|
||||
cat > /tmp/generate_sales.sql << 'EOF'
|
||||
-- Link transactions to products
|
||||
INSERT INTO sales_transactions (
|
||||
transaction_id, product_id, quantity, unit_price, discount_amount,
|
||||
tax_amount, total_amount, sale_date, sales_channel, region
|
||||
)
|
||||
SELECT
|
||||
t.transaction_id,
|
||||
((t.transaction_id % 24) + 1) as product_id, -- Cycle through 24 products
|
||||
(1 + (random() * 3)::INT) as quantity,
|
||||
t.amount / (1 + (random() * 3)::INT) as unit_price,
|
||||
CASE WHEN random() < 0.2 THEN t.amount * 0.1 ELSE 0 END as discount_amount,
|
||||
t.amount * 0.08 as tax_amount,
|
||||
t.amount as total_amount,
|
||||
t.transaction_date as sale_date,
|
||||
CASE (t.transaction_id % 4)
|
||||
WHEN 0 THEN 'ONLINE'
|
||||
WHEN 1 THEN 'STORE'
|
||||
WHEN 2 THEN 'PHONE'
|
||||
ELSE 'MOBILE_APP'
|
||||
END as sales_channel,
|
||||
CASE (t.transaction_id % 5)
|
||||
WHEN 0 THEN 'Northeast'
|
||||
WHEN 1 THEN 'Southeast'
|
||||
WHEN 2 THEN 'Midwest'
|
||||
WHEN 3 THEN 'Southwest'
|
||||
ELSE 'West'
|
||||
END as region
|
||||
FROM transactions t
|
||||
WHERE t.status = 'COMPLETED'
|
||||
LIMIT 1000000; -- Link 1M transactions to products
|
||||
EOF
|
||||
|
||||
execute_sql_file /tmp/generate_sales.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated 1,000,000 sales transaction records${NC}"
|
||||
echo ""
|
||||
|
||||
# ============================================================================
|
||||
# GENERATE KPI & METRICS DATA
|
||||
# ============================================================================
|
||||
|
||||
echo -e "${YELLOW}[14/15] Generating Daily Metrics...${NC}"
|
||||
cat > /tmp/generate_metrics.sql << 'EOF'
|
||||
-- Generate daily metrics for the past 90 days
|
||||
INSERT INTO daily_metrics (
|
||||
metric_date, kpi_id, metric_value, vs_previous_day_percentage, status
|
||||
)
|
||||
SELECT
|
||||
date_series.metric_date,
|
||||
kpi.kpi_id,
|
||||
kpi.target_value * (0.8 + random() * 0.4) as metric_value,
|
||||
(-20 + random() * 40)::DECIMAL(5,2) as vs_previous_day_percentage,
|
||||
CASE
|
||||
WHEN random() < 0.7 THEN 'ON_TARGET'
|
||||
WHEN random() < 0.9 THEN 'WARNING'
|
||||
ELSE 'CRITICAL'
|
||||
END as status
|
||||
FROM generate_series(
|
||||
CURRENT_DATE - INTERVAL '90 days',
|
||||
CURRENT_DATE,
|
||||
INTERVAL '1 day'
|
||||
) AS date_series(metric_date)
|
||||
CROSS JOIN kpi_definitions kpi
|
||||
WHERE kpi.is_active = TRUE;
|
||||
EOF
|
||||
|
||||
execute_sql_file /tmp/generate_metrics.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated 90 days of daily metrics${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${YELLOW}[15/15] Generating Monthly Summaries...${NC}"
|
||||
cat > /tmp/generate_monthly.sql << 'EOF'
|
||||
-- Generate monthly summaries for the past 24 months
|
||||
INSERT INTO monthly_summaries (
|
||||
summary_month, summary_year, total_revenue, total_transactions,
|
||||
total_customers, new_customers, average_transaction_value
|
||||
)
|
||||
SELECT
|
||||
EXTRACT(MONTH FROM month_series)::INT as summary_month,
|
||||
EXTRACT(YEAR FROM month_series)::INT as summary_year,
|
||||
(10000000 + random() * 5000000)::DECIMAL(15,2) as total_revenue,
|
||||
(50000 + (random() * 30000)::INT) as total_transactions,
|
||||
(80000 + (random() * 20000)::INT) as total_customers,
|
||||
(500 + (random() * 1500)::INT) as new_customers,
|
||||
(100 + random() * 100)::DECIMAL(15,2) as average_transaction_value
|
||||
FROM generate_series(
|
||||
CURRENT_DATE - INTERVAL '24 months',
|
||||
CURRENT_DATE,
|
||||
INTERVAL '1 month'
|
||||
) AS month_series;
|
||||
EOF
|
||||
|
||||
execute_sql_file /tmp/generate_monthly.sql > /dev/null
|
||||
echo -e "${GREEN}✓ Generated 24 months of summaries${NC}"
|
||||
echo ""
|
||||
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo -e "${GREEN}Data Generation Complete!${NC}"
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo ""
|
||||
echo -e "${YELLOW}Database Statistics:${NC}"
|
||||
|
||||
customer_count=$(execute_sql "SELECT COUNT(*) FROM customers;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Customers: ${GREEN}$customer_count${NC}"
|
||||
|
||||
account_count=$(execute_sql "SELECT COUNT(*) FROM accounts;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Accounts: ${GREEN}$account_count${NC}"
|
||||
|
||||
merchant_count=$(execute_sql "SELECT COUNT(*) FROM merchants;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Merchants: ${GREEN}$merchant_count${NC}"
|
||||
|
||||
card_count=$(execute_sql "SELECT COUNT(*) FROM cards;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Cards: ${GREEN}$card_count${NC}"
|
||||
|
||||
transaction_count=$(execute_sql "SELECT COUNT(*) FROM transactions;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Transactions: ${GREEN}$transaction_count${NC}"
|
||||
|
||||
alert_count=$(execute_sql "SELECT COUNT(*) FROM alerts;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Alerts: ${GREEN}$alert_count${NC}"
|
||||
|
||||
case_count=$(execute_sql "SELECT COUNT(*) FROM fraud_cases;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Fraud Cases: ${GREEN}$case_count${NC}"
|
||||
|
||||
clv_count=$(execute_sql "SELECT COUNT(*) FROM customer_lifetime_value;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Customer CLV Records: ${GREEN}$clv_count${NC}"
|
||||
|
||||
sales_count=$(execute_sql "SELECT COUNT(*) FROM sales_transactions;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Sales Records: ${GREEN}$sales_count${NC}"
|
||||
|
||||
metrics_count=$(execute_sql "SELECT COUNT(*) FROM daily_metrics;" | grep -E '^\s*[0-9]+' | tr -d ' ')
|
||||
echo -e "Daily Metrics: ${GREEN}$metrics_count${NC}"
|
||||
|
||||
echo ""
|
||||
echo -e "${YELLOW}Ready for Business Analytics and SQL learning!${NC}"
|
||||
echo -e "Access DB-UI at: ${BLUE}http://localhost:3000${NC}"
|
||||
echo ""
|
||||
|
||||
|
||||
+10
-10
@@ -3,10 +3,10 @@ version: '3.8'
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
container_name: fraud_detection_db
|
||||
container_name: business_analytics_db
|
||||
environment:
|
||||
POSTGRES_DB: fraud_detection
|
||||
POSTGRES_USER: fraud_analyst
|
||||
POSTGRES_DB: business_analytics
|
||||
POSTGRES_USER: data_analyst
|
||||
POSTGRES_PASSWORD: SecurePass123!
|
||||
PGDATA: /var/lib/postgresql/data/pgdata
|
||||
ports:
|
||||
@@ -16,9 +16,9 @@ services:
|
||||
- ./schema:/docker-entrypoint-initdb.d
|
||||
- ./data:/data
|
||||
networks:
|
||||
- fraud_network
|
||||
- analytics_network
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U fraud_analyst -d fraud_detection"]
|
||||
test: ["CMD-SHELL", "pg_isready -U data_analyst -d business_analytics"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
@@ -26,24 +26,24 @@ services:
|
||||
|
||||
db-ui:
|
||||
image: ghcr.io/n7olkachev/db-ui:latest
|
||||
container_name: fraud_detection_ui
|
||||
container_name: business_analytics_ui
|
||||
environment:
|
||||
POSTGRES_HOST: postgres
|
||||
POSTGRES_USER: fraud_analyst
|
||||
POSTGRES_USER: data_analyst
|
||||
POSTGRES_PASSWORD: SecurePass123!
|
||||
POSTGRES_DB: fraud_detection
|
||||
POSTGRES_DB: business_analytics
|
||||
POSTGRES_PORT: 5432
|
||||
ports:
|
||||
- "3000:3000"
|
||||
networks:
|
||||
- fraud_network
|
||||
- analytics_network
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
restart: unless-stopped
|
||||
|
||||
networks:
|
||||
fraud_network:
|
||||
analytics_network:
|
||||
driver: bridge
|
||||
|
||||
volumes:
|
||||
|
||||
@@ -0,0 +1,358 @@
|
||||
# Business Analytics Database - Data Models
|
||||
|
||||
## Overview
|
||||
This database supports **Data Analyst** activities including:
|
||||
- Routine and semi-routine analysis
|
||||
- Dashboard creation and maintenance
|
||||
- Business trend identification
|
||||
- Statistical analysis and pattern recognition
|
||||
- Stakeholder reporting
|
||||
|
||||
---
|
||||
|
||||
## Model 1: Customer Analytics (Routine Analysis)
|
||||
|
||||
### Purpose
|
||||
Support customer segmentation, lifetime value analysis, churn prediction, and engagement tracking.
|
||||
|
||||
### Tables
|
||||
|
||||
#### `customer_segments`
|
||||
Customer classification for targeted analysis
|
||||
```sql
|
||||
- segment_id (PK)
|
||||
- segment_name (e.g., 'High Value', 'At Risk', 'New Customer')
|
||||
- segment_description
|
||||
- criteria_definition (JSON)
|
||||
- created_date
|
||||
- updated_date
|
||||
```
|
||||
|
||||
#### `customer_lifetime_value`
|
||||
CLV calculations for business insights
|
||||
```sql
|
||||
- clv_id (PK)
|
||||
- customer_id (FK → customers)
|
||||
- calculation_date
|
||||
- total_revenue
|
||||
- total_transactions
|
||||
- average_order_value
|
||||
- predicted_future_value
|
||||
- clv_score
|
||||
- segment_id (FK → customer_segments)
|
||||
```
|
||||
|
||||
#### `churn_predictions`
|
||||
Customer retention analysis
|
||||
```sql
|
||||
- prediction_id (PK)
|
||||
- customer_id (FK → customers)
|
||||
- prediction_date
|
||||
- churn_probability (0-100)
|
||||
- risk_level (LOW, MEDIUM, HIGH, CRITICAL)
|
||||
- last_transaction_date
|
||||
- days_since_last_transaction
|
||||
- engagement_score
|
||||
- recommended_action
|
||||
```
|
||||
|
||||
#### `customer_satisfaction`
|
||||
Satisfaction scores and feedback
|
||||
```sql
|
||||
- satisfaction_id (PK)
|
||||
- customer_id (FK → customers)
|
||||
- survey_date
|
||||
- nps_score (-100 to 100)
|
||||
- csat_score (1-5)
|
||||
- feedback_text
|
||||
- category (PRODUCT, SERVICE, SUPPORT, etc.)
|
||||
- sentiment (POSITIVE, NEUTRAL, NEGATIVE)
|
||||
```
|
||||
|
||||
#### `engagement_metrics`
|
||||
Customer interaction tracking
|
||||
```sql
|
||||
- metric_id (PK)
|
||||
- customer_id (FK → customers)
|
||||
- metric_date
|
||||
- login_count
|
||||
- page_views
|
||||
- time_spent_minutes
|
||||
- features_used
|
||||
- support_tickets_opened
|
||||
- engagement_score
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Model 2: Sales & Revenue Analytics (Semi-Routine Reporting)
|
||||
|
||||
### Purpose
|
||||
Support sales performance tracking, revenue forecasting, and product analysis.
|
||||
|
||||
### Tables
|
||||
|
||||
#### `product_catalog`
|
||||
Product master data
|
||||
```sql
|
||||
- product_id (PK)
|
||||
- product_name
|
||||
- product_category
|
||||
- product_subcategory
|
||||
- unit_price
|
||||
- cost_price
|
||||
- margin_percentage
|
||||
- is_active
|
||||
- launch_date
|
||||
- discontinued_date
|
||||
```
|
||||
|
||||
#### `sales_transactions`
|
||||
Detailed sales records (extends existing transactions)
|
||||
```sql
|
||||
- sale_id (PK)
|
||||
- transaction_id (FK → transactions)
|
||||
- product_id (FK → product_catalog)
|
||||
- quantity
|
||||
- unit_price
|
||||
- discount_amount
|
||||
- tax_amount
|
||||
- total_amount
|
||||
- sale_date
|
||||
- sales_channel (ONLINE, STORE, PHONE, MOBILE_APP)
|
||||
- sales_rep_id
|
||||
- region
|
||||
```
|
||||
|
||||
#### `sales_targets`
|
||||
Performance goals for tracking
|
||||
```sql
|
||||
- target_id (PK)
|
||||
- target_period (DAILY, WEEKLY, MONTHLY, QUARTERLY, YEARLY)
|
||||
- start_date
|
||||
- end_date
|
||||
- product_category
|
||||
- region
|
||||
- target_revenue
|
||||
- target_units
|
||||
- target_customers
|
||||
- created_by
|
||||
```
|
||||
|
||||
#### `sales_performance`
|
||||
Aggregated performance metrics
|
||||
```sql
|
||||
- performance_id (PK)
|
||||
- period_date
|
||||
- period_type (DAILY, WEEKLY, MONTHLY, QUARTERLY)
|
||||
- product_id (FK → product_catalog)
|
||||
- region
|
||||
- total_revenue
|
||||
- total_units_sold
|
||||
- total_transactions
|
||||
- unique_customers
|
||||
- average_order_value
|
||||
- vs_target_percentage
|
||||
```
|
||||
|
||||
#### `revenue_forecasts`
|
||||
Predictive revenue analysis
|
||||
```sql
|
||||
- forecast_id (PK)
|
||||
- forecast_date
|
||||
- forecast_period_start
|
||||
- forecast_period_end
|
||||
- product_category
|
||||
- region
|
||||
- forecasted_revenue
|
||||
- confidence_level (LOW, MEDIUM, HIGH)
|
||||
- forecast_method (HISTORICAL, TREND, SEASONAL, ML)
|
||||
- actual_revenue (filled after period ends)
|
||||
- variance_percentage
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Model 3: Operational Metrics & KPIs (Dashboard Creation)
|
||||
|
||||
### Purpose
|
||||
Support dashboard creation with pre-calculated KPIs and trend analysis.
|
||||
|
||||
### Tables
|
||||
|
||||
#### `kpi_definitions`
|
||||
Master list of tracked KPIs
|
||||
```sql
|
||||
- kpi_id (PK)
|
||||
- kpi_name
|
||||
- kpi_description
|
||||
- kpi_category (SALES, CUSTOMER, OPERATIONAL, FINANCIAL)
|
||||
- calculation_formula
|
||||
- target_value
|
||||
- threshold_warning
|
||||
- threshold_critical
|
||||
- unit_of_measure
|
||||
- refresh_frequency (REALTIME, HOURLY, DAILY, WEEKLY)
|
||||
- is_active
|
||||
```
|
||||
|
||||
#### `daily_metrics`
|
||||
Daily operational snapshots
|
||||
```sql
|
||||
- metric_id (PK)
|
||||
- metric_date
|
||||
- kpi_id (FK → kpi_definitions)
|
||||
- metric_value
|
||||
- vs_previous_day_percentage
|
||||
- vs_previous_week_percentage
|
||||
- vs_previous_month_percentage
|
||||
- status (ON_TARGET, WARNING, CRITICAL)
|
||||
- notes
|
||||
```
|
||||
|
||||
#### `monthly_summaries`
|
||||
Monthly aggregated data for reporting
|
||||
```sql
|
||||
- summary_id (PK)
|
||||
- summary_month
|
||||
- summary_year
|
||||
- total_revenue
|
||||
- total_transactions
|
||||
- total_customers
|
||||
- new_customers
|
||||
- churned_customers
|
||||
- average_transaction_value
|
||||
- customer_acquisition_cost
|
||||
- customer_lifetime_value
|
||||
- net_promoter_score
|
||||
- gross_margin_percentage
|
||||
```
|
||||
|
||||
#### `trend_analysis`
|
||||
Statistical trend tracking
|
||||
```sql
|
||||
- trend_id (PK)
|
||||
- kpi_id (FK → kpi_definitions)
|
||||
- analysis_date
|
||||
- period_start
|
||||
- period_end
|
||||
- trend_direction (UP, DOWN, FLAT)
|
||||
- trend_strength (WEAK, MODERATE, STRONG)
|
||||
- moving_average_7day
|
||||
- moving_average_30day
|
||||
- seasonality_detected
|
||||
- anomalies_detected
|
||||
- statistical_significance
|
||||
```
|
||||
|
||||
#### `dashboard_snapshots`
|
||||
Pre-calculated dashboard data
|
||||
```sql
|
||||
- snapshot_id (PK)
|
||||
- dashboard_name
|
||||
- snapshot_timestamp
|
||||
- data_payload (JSON)
|
||||
- refresh_duration_seconds
|
||||
- row_count
|
||||
- last_updated_by
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Model 4: Business Intelligence & Reporting
|
||||
|
||||
### Purpose
|
||||
Support ad-hoc analysis and standard report generation.
|
||||
|
||||
### Tables
|
||||
|
||||
#### `report_definitions`
|
||||
Catalog of standard reports
|
||||
```sql
|
||||
- report_id (PK)
|
||||
- report_name
|
||||
- report_description
|
||||
- report_category
|
||||
- sql_query_template
|
||||
- parameters (JSON)
|
||||
- output_format (PDF, EXCEL, CSV, HTML)
|
||||
- schedule_frequency
|
||||
- recipients
|
||||
- is_active
|
||||
```
|
||||
|
||||
#### `report_executions`
|
||||
Report run history
|
||||
```sql
|
||||
- execution_id (PK)
|
||||
- report_id (FK → report_definitions)
|
||||
- execution_timestamp
|
||||
- parameters_used (JSON)
|
||||
- row_count
|
||||
- execution_duration_seconds
|
||||
- status (SUCCESS, FAILED, TIMEOUT)
|
||||
- error_message
|
||||
- output_file_path
|
||||
- executed_by
|
||||
```
|
||||
|
||||
#### `data_quality_checks`
|
||||
Data validation tracking
|
||||
```sql
|
||||
- check_id (PK)
|
||||
- check_name
|
||||
- table_name
|
||||
- column_name
|
||||
- check_type (NULL_CHECK, RANGE_CHECK, UNIQUENESS, REFERENTIAL_INTEGRITY)
|
||||
- check_date
|
||||
- records_checked
|
||||
- records_failed
|
||||
- failure_percentage
|
||||
- status (PASS, FAIL, WARNING)
|
||||
- remediation_notes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Relationships to Existing Models
|
||||
|
||||
### Integration Points
|
||||
|
||||
1. **Customer Analytics** ← links to existing `customers` table
|
||||
2. **Sales Transactions** ← extends existing `transactions` table
|
||||
3. **Engagement Metrics** ← links to `login_sessions` and `transactions`
|
||||
4. **Churn Predictions** ← analyzes `transactions` and `accounts`
|
||||
5. **KPIs** ← aggregates from multiple existing tables
|
||||
|
||||
---
|
||||
|
||||
## Use Cases for Data Analyst Role
|
||||
|
||||
### Routine Analysis
|
||||
- Daily sales reports
|
||||
- Customer segment updates
|
||||
- KPI dashboard refreshes
|
||||
- Data quality checks
|
||||
|
||||
### Semi-Routine Analysis
|
||||
- Monthly trend analysis
|
||||
- Churn prediction updates
|
||||
- Revenue forecasting
|
||||
- Performance vs. targets
|
||||
|
||||
### Ad-Hoc Analysis
|
||||
- Customer cohort analysis
|
||||
- Product performance deep-dives
|
||||
- Seasonal pattern identification
|
||||
- Anomaly investigation
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Design complete
|
||||
2. ⏳ Implement schema in `schema/01-create-tables.sql`
|
||||
3. ⏳ Update data generation scripts
|
||||
4. ⏳ Create SQL exercises for each model
|
||||
5. ⏳ Update documentation
|
||||
|
||||
+64
-33
@@ -19,7 +19,7 @@ docker-compose up -d
|
||||
docker-compose ps
|
||||
```
|
||||
|
||||
You should see both `fraud_detection_db` and `fraud_detection_ui` running.
|
||||
You should see both `business_analytics_db` and `business_analytics_ui` running.
|
||||
|
||||
---
|
||||
|
||||
@@ -34,9 +34,9 @@ chmod +x scripts/setup-database.sh
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
- Creates all 20+ tables
|
||||
- Creates all 39 tables across 4 business models
|
||||
- Sets up indexes and constraints
|
||||
- Loads reference data (countries, merchant categories, etc.)
|
||||
- Loads reference data (countries, merchant categories, customer segments, products, KPIs, etc.)
|
||||
|
||||
**Expected output:**
|
||||
```
|
||||
@@ -63,6 +63,9 @@ chmod +x data/generate_data.sh
|
||||
- Creates 150,000 accounts
|
||||
- Generates 5,000,000 transactions
|
||||
- Creates fraud patterns and alerts
|
||||
- Generates customer analytics (CLV, churn, satisfaction)
|
||||
- Creates sales data (1M sales records)
|
||||
- Generates KPI metrics (90 days of daily metrics)
|
||||
|
||||
**⏱️ Time estimate:**
|
||||
- Fast machine (SSD, 16GB RAM): ~15 minutes
|
||||
@@ -72,10 +75,16 @@ chmod +x data/generate_data.sh
|
||||
**You can monitor progress:**
|
||||
The script shows progress for each step:
|
||||
```
|
||||
[1/9] Loading geographic reference data...
|
||||
[2/9] Generating Customers...
|
||||
[3/9] Generating Accounts...
|
||||
[1/15] Loading geographic reference data...
|
||||
[2/15] Generating Customers...
|
||||
[3/15] Generating Accounts...
|
||||
...
|
||||
[10/15] Generating Customer Lifetime Value data...
|
||||
[11/15] Generating Churn Predictions...
|
||||
[12/15] Generating Customer Satisfaction data...
|
||||
[13/15] Generating Sales Transactions...
|
||||
[14/15] Generating Daily Metrics...
|
||||
[15/15] Generating Monthly Summaries...
|
||||
```
|
||||
|
||||
---
|
||||
@@ -98,7 +107,7 @@ The script shows progress for each step:
|
||||
#### Option B: Command Line (psql)
|
||||
|
||||
```bash
|
||||
docker exec -it fraud_detection_db psql -U fraud_analyst -d fraud_detection
|
||||
docker exec -it business_analytics_db psql -U data_analyst -d business_analytics
|
||||
```
|
||||
|
||||
**Quick commands:**
|
||||
@@ -122,8 +131,8 @@ SELECT COUNT(*) FROM transactions;
|
||||
```
|
||||
Host: localhost
|
||||
Port: 5432
|
||||
Database: fraud_detection
|
||||
Username: fraud_analyst
|
||||
Database: business_analytics
|
||||
Username: data_analyst
|
||||
Password: SecurePass123!
|
||||
```
|
||||
|
||||
@@ -146,43 +155,65 @@ SELECT COUNT(*) FROM customers;
|
||||
-- How many transactions?
|
||||
SELECT COUNT(*) FROM transactions;
|
||||
|
||||
-- How many fraud alerts?
|
||||
SELECT COUNT(*) FROM alerts WHERE status = 'OPEN';
|
||||
-- How many sales records?
|
||||
SELECT COUNT(*) FROM sales_transactions;
|
||||
|
||||
-- How many KPIs are being tracked?
|
||||
SELECT COUNT(*) FROM kpi_definitions WHERE is_active = TRUE;
|
||||
```
|
||||
|
||||
### 2. Find High-Risk Customers
|
||||
### 2. Customer Analytics: High-Value Customers
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
customer_id,
|
||||
first_name,
|
||||
last_name,
|
||||
email,
|
||||
risk_score
|
||||
FROM customers
|
||||
WHERE risk_score > 80
|
||||
ORDER BY risk_score DESC
|
||||
SELECT
|
||||
c.customer_id,
|
||||
c.first_name,
|
||||
c.last_name,
|
||||
clv.clv_score,
|
||||
cs.segment_name
|
||||
FROM customers c
|
||||
JOIN customer_lifetime_value clv ON c.customer_id = clv.customer_id
|
||||
JOIN customer_segments cs ON clv.segment_id = cs.segment_id
|
||||
WHERE cs.segment_name IN ('VIP', 'High Value')
|
||||
ORDER BY clv.clv_score DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
### 3. View Recent Transactions
|
||||
### 3. Sales Analytics: Top Products
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
transaction_id,
|
||||
account_id,
|
||||
amount,
|
||||
transaction_date,
|
||||
is_flagged
|
||||
FROM transactions
|
||||
ORDER BY transaction_date DESC
|
||||
LIMIT 20;
|
||||
SELECT
|
||||
p.product_name,
|
||||
p.product_category,
|
||||
COUNT(st.transaction_id) as sales_count,
|
||||
SUM(st.total_amount) as total_revenue
|
||||
FROM product_catalog p
|
||||
JOIN sales_transactions st ON p.product_id = st.product_id
|
||||
GROUP BY p.product_id, p.product_name, p.product_category
|
||||
ORDER BY total_revenue DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
### 4. Find Flagged Transactions
|
||||
### 4. KPI Dashboard: Current Status
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
SELECT
|
||||
kd.kpi_name,
|
||||
kd.kpi_category,
|
||||
dm.metric_value,
|
||||
kd.target_value,
|
||||
dm.status
|
||||
FROM kpi_definitions kd
|
||||
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
WHERE dm.metric_date = CURRENT_DATE
|
||||
AND kd.is_active = TRUE
|
||||
ORDER BY kd.kpi_category, kd.kpi_name;
|
||||
```
|
||||
|
||||
### 5. Fraud Detection: Flagged Transactions
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
t.transaction_id,
|
||||
t.amount,
|
||||
t.fraud_score,
|
||||
|
||||
@@ -0,0 +1,266 @@
|
||||
# What's New - Business Analytics Database
|
||||
|
||||
## 🎯 Major Update: Aligned with Data Analyst Role
|
||||
|
||||
The database has been **expanded and refocused** to align with real-world **Data Analyst** responsibilities:
|
||||
|
||||
- ✅ Routine and semi-routine analysis
|
||||
- ✅ Dashboard creation and maintenance
|
||||
- ✅ Business trend identification
|
||||
- ✅ Statistical analysis and pattern recognition
|
||||
- ✅ Stakeholder reporting
|
||||
|
||||
---
|
||||
|
||||
## 📊 Database Renamed
|
||||
|
||||
**Old Name:** `fraud_detection`
|
||||
**New Name:** `business_analytics`
|
||||
|
||||
**User Changed:** `fraud_analyst` → `data_analyst`
|
||||
|
||||
This better reflects the broader scope of business intelligence and analytics work.
|
||||
|
||||
---
|
||||
|
||||
## 🆕 New Data Models Added
|
||||
|
||||
### 1. Customer Analytics Model (5 Tables)
|
||||
**Purpose:** Routine customer analysis and segmentation
|
||||
|
||||
#### New Tables:
|
||||
- **`customer_segments`** - Customer classification definitions
|
||||
- **`customer_lifetime_value`** - CLV calculations and tracking
|
||||
- **`churn_predictions`** - Customer retention risk analysis
|
||||
- **`customer_satisfaction`** - NPS, CSAT scores, and feedback
|
||||
- **`engagement_metrics`** - Customer interaction tracking
|
||||
|
||||
**Use Cases:**
|
||||
- Customer segmentation analysis
|
||||
- Churn risk identification
|
||||
- Lifetime value calculations
|
||||
- Satisfaction trend analysis
|
||||
- Engagement scoring
|
||||
|
||||
---
|
||||
|
||||
### 2. Sales & Revenue Analytics Model (6 Tables)
|
||||
**Purpose:** Semi-routine sales reporting and forecasting
|
||||
|
||||
#### New Tables:
|
||||
- **`product_catalog`** - Product master data
|
||||
- **`sales_transactions`** - Detailed sales records
|
||||
- **`sales_targets`** - Performance goals and targets
|
||||
- **`sales_performance`** - Aggregated performance metrics
|
||||
- **`revenue_forecasts`** - Revenue predictions and variance
|
||||
|
||||
**Use Cases:**
|
||||
- Sales performance dashboards
|
||||
- Product analysis
|
||||
- Revenue forecasting
|
||||
- Target vs. actual analysis
|
||||
- Channel performance comparison
|
||||
|
||||
---
|
||||
|
||||
### 3. Operational Metrics & KPI Model (5 Tables)
|
||||
**Purpose:** Dashboard creation and KPI tracking
|
||||
|
||||
#### New Tables:
|
||||
- **`kpi_definitions`** - Master KPI catalog
|
||||
- **`daily_metrics`** - Daily operational snapshots
|
||||
- **`monthly_summaries`** - Monthly business summaries
|
||||
- **`trend_analysis`** - Statistical trend tracking
|
||||
- **`dashboard_snapshots`** - Pre-calculated dashboard data
|
||||
|
||||
**Use Cases:**
|
||||
- Executive dashboards
|
||||
- KPI monitoring
|
||||
- Trend analysis
|
||||
- Performance tracking
|
||||
- Anomaly detection
|
||||
|
||||
---
|
||||
|
||||
### 4. Business Intelligence & Reporting Model (3 Tables)
|
||||
**Purpose:** Report management and data quality
|
||||
|
||||
#### New Tables:
|
||||
- **`report_definitions`** - Standard report catalog
|
||||
- **`report_executions`** - Report run history
|
||||
- **`data_quality_checks`** - Data validation tracking
|
||||
|
||||
**Use Cases:**
|
||||
- Report scheduling
|
||||
- Execution monitoring
|
||||
- Data quality assurance
|
||||
- Audit trails
|
||||
|
||||
---
|
||||
|
||||
## 📈 Total Database Size
|
||||
|
||||
### Original (Fraud Detection Only):
|
||||
- **20 tables** focused on fraud investigation
|
||||
|
||||
### Updated (Business Analytics):
|
||||
- **39 tables** covering:
|
||||
- Fraud detection (original 20 tables)
|
||||
- Customer analytics (5 tables)
|
||||
- Sales & revenue (6 tables)
|
||||
- KPIs & metrics (5 tables)
|
||||
- Reporting & BI (3 tables)
|
||||
|
||||
---
|
||||
|
||||
## 🔗 Integration with Existing Data
|
||||
|
||||
The new models **integrate seamlessly** with existing tables:
|
||||
|
||||
```
|
||||
customers → customer_lifetime_value
|
||||
→ churn_predictions
|
||||
→ customer_satisfaction
|
||||
→ engagement_metrics
|
||||
|
||||
transactions → sales_transactions
|
||||
→ sales_performance
|
||||
→ revenue_forecasts
|
||||
|
||||
(all tables) → kpi_definitions
|
||||
→ daily_metrics
|
||||
→ monthly_summaries
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 💼 Data Analyst Workflows Supported
|
||||
|
||||
### Daily Routine Tasks
|
||||
1. **Refresh dashboards** using `dashboard_snapshots`
|
||||
2. **Update daily metrics** in `daily_metrics`
|
||||
3. **Run data quality checks** via `data_quality_checks`
|
||||
4. **Generate standard reports** from `report_definitions`
|
||||
|
||||
### Weekly Semi-Routine Tasks
|
||||
1. **Customer segmentation** analysis using `customer_segments` and `customer_lifetime_value`
|
||||
2. **Sales performance** review via `sales_performance` vs `sales_targets`
|
||||
3. **Churn prediction** updates in `churn_predictions`
|
||||
4. **Trend analysis** using `trend_analysis` table
|
||||
|
||||
### Monthly Analysis
|
||||
1. **Monthly summaries** generation in `monthly_summaries`
|
||||
2. **Revenue forecasting** via `revenue_forecasts`
|
||||
3. **Customer satisfaction** trend analysis
|
||||
4. **KPI performance** review
|
||||
|
||||
### Ad-Hoc Analysis
|
||||
1. **Product performance** deep-dives
|
||||
2. **Customer cohort** analysis
|
||||
3. **Seasonal pattern** identification
|
||||
4. **Anomaly investigation**
|
||||
|
||||
---
|
||||
|
||||
## 🎓 New Learning Opportunities
|
||||
|
||||
### For Beginners:
|
||||
- Basic aggregations (SUM, AVG, COUNT)
|
||||
- Simple JOINs across analytics tables
|
||||
- Date-based filtering and grouping
|
||||
- KPI calculations
|
||||
|
||||
### For Intermediate:
|
||||
- Customer segmentation queries
|
||||
- Sales trend analysis
|
||||
- Moving averages and window functions
|
||||
- Cohort analysis
|
||||
|
||||
### For Advanced:
|
||||
- Churn prediction analysis
|
||||
- Revenue forecasting validation
|
||||
- Multi-dimensional analysis
|
||||
- Statistical significance testing
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Migration Notes
|
||||
|
||||
### What Changed:
|
||||
- ✅ Database name: `fraud_detection` → `business_analytics`
|
||||
- ✅ User name: `fraud_analyst` → `data_analyst`
|
||||
- ✅ Container names updated
|
||||
- ✅ All scripts updated
|
||||
- ✅ 19 new tables added
|
||||
|
||||
### What Stayed the Same:
|
||||
- ✅ All original 20 fraud detection tables
|
||||
- ✅ All existing data generation logic
|
||||
- ✅ All existing indexes and constraints
|
||||
- ✅ Idempotent script design
|
||||
- ✅ Docker setup structure
|
||||
|
||||
### Backward Compatibility:
|
||||
- ✅ All original fraud detection exercises still work
|
||||
- ✅ All original queries still valid
|
||||
- ✅ No breaking changes to existing schema
|
||||
|
||||
---
|
||||
|
||||
## 📝 Next Steps
|
||||
|
||||
### Immediate:
|
||||
1. ✅ Schema updated with 19 new tables
|
||||
2. ⏳ Update data generation scripts
|
||||
3. ⏳ Create SQL exercises for new models
|
||||
4. ⏳ Update main documentation
|
||||
|
||||
### Future Enhancements:
|
||||
- Add sample data for new tables
|
||||
- Create dashboard query examples
|
||||
- Build KPI calculation examples
|
||||
- Add data quality check templates
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Getting Started with New Models
|
||||
|
||||
### Quick Test Queries:
|
||||
|
||||
```sql
|
||||
-- Check new tables exist
|
||||
SELECT table_name
|
||||
FROM information_schema.tables
|
||||
WHERE table_schema = 'public'
|
||||
AND table_name IN (
|
||||
'customer_segments',
|
||||
'customer_lifetime_value',
|
||||
'product_catalog',
|
||||
'sales_transactions',
|
||||
'kpi_definitions',
|
||||
'daily_metrics'
|
||||
);
|
||||
|
||||
-- View table counts
|
||||
SELECT
|
||||
'customer_segments' as table_name, COUNT(*) FROM customer_segments
|
||||
UNION ALL
|
||||
SELECT 'product_catalog', COUNT(*) FROM product_catalog
|
||||
UNION ALL
|
||||
SELECT 'kpi_definitions', COUNT(*) FROM kpi_definitions;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📚 Documentation Updates
|
||||
|
||||
- ✅ **DATA_MODELS.md** - Complete model documentation
|
||||
- ✅ **WHATS_NEW.md** - This file
|
||||
- ⏳ **README.md** - Update with new scope
|
||||
- ⏳ **QUICKSTART.md** - Add new model examples
|
||||
- ⏳ **Exercises** - Create analytics-focused exercises
|
||||
|
||||
---
|
||||
|
||||
**The database is now a comprehensive Business Analytics platform suitable for Data Analyst training and real-world analysis scenarios!** 🎉
|
||||
|
||||
@@ -0,0 +1,325 @@
|
||||
# Level 2: Customer Analytics
|
||||
|
||||
## Introduction
|
||||
Learn to analyze customer behavior, calculate lifetime value, identify churn risks, and segment customers - essential skills for Data Analysts working with customer data.
|
||||
|
||||
## Learning Objectives
|
||||
- Calculate customer lifetime value (CLV)
|
||||
- Identify at-risk customers
|
||||
- Segment customers by behavior
|
||||
- Analyze satisfaction trends
|
||||
- Track engagement metrics
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.1: Customer Segmentation
|
||||
**Objective:** Find the distribution of customers across segments
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
cs.segment_name,
|
||||
COUNT(clv.customer_id) as customer_count,
|
||||
AVG(clv.clv_score) as avg_clv_score,
|
||||
SUM(clv.total_revenue) as total_segment_revenue
|
||||
FROM customer_segments cs
|
||||
LEFT JOIN customer_lifetime_value clv ON cs.segment_id = clv.segment_id
|
||||
GROUP BY cs.segment_id, cs.segment_name
|
||||
ORDER BY total_segment_revenue DESC;
|
||||
```
|
||||
|
||||
**Expected Result:** Segment breakdown with counts and revenue
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.2: High-Value Customers
|
||||
**Objective:** Identify top 10 customers by lifetime value
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Hint:** Use customer_lifetime_value table and ORDER BY
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
c.customer_id,
|
||||
c.first_name,
|
||||
c.last_name,
|
||||
c.email,
|
||||
clv.total_revenue,
|
||||
clv.total_transactions,
|
||||
clv.clv_score,
|
||||
cs.segment_name
|
||||
FROM customer_lifetime_value clv
|
||||
JOIN customers c ON clv.customer_id = c.customer_id
|
||||
JOIN customer_segments cs ON clv.segment_id = cs.segment_id
|
||||
ORDER BY clv.clv_score DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.3: Churn Risk Analysis
|
||||
**Objective:** Find customers at CRITICAL or HIGH churn risk
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
c.customer_id,
|
||||
c.first_name,
|
||||
c.last_name,
|
||||
c.email,
|
||||
cp.churn_probability,
|
||||
cp.risk_level,
|
||||
cp.days_since_last_transaction,
|
||||
cp.engagement_score
|
||||
FROM churn_predictions cp
|
||||
JOIN customers c ON cp.customer_id = c.customer_id
|
||||
WHERE cp.risk_level IN ('CRITICAL', 'HIGH')
|
||||
ORDER BY cp.churn_probability DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.4: Customer Satisfaction Trends
|
||||
**Objective:** Calculate average NPS score by month
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Hint:** Use DATE_TRUNC or EXTRACT to group by month
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
DATE_TRUNC('month', survey_date) as month,
|
||||
COUNT(*) as survey_count,
|
||||
AVG(nps_score) as avg_nps,
|
||||
AVG(csat_score) as avg_csat,
|
||||
COUNT(CASE WHEN sentiment = 'POSITIVE' THEN 1 END) as positive_count,
|
||||
COUNT(CASE WHEN sentiment = 'NEGATIVE' THEN 1 END) as negative_count
|
||||
FROM customer_satisfaction
|
||||
GROUP BY DATE_TRUNC('month', survey_date)
|
||||
ORDER BY month DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.5: Engagement Score Analysis
|
||||
**Objective:** Find customers with declining engagement
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Hint:** Compare recent engagement to historical average
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
WITH recent_engagement AS (
|
||||
SELECT
|
||||
customer_id,
|
||||
AVG(engagement_score) as recent_score
|
||||
FROM engagement_metrics
|
||||
WHERE metric_date >= CURRENT_DATE - INTERVAL '30 days'
|
||||
GROUP BY customer_id
|
||||
),
|
||||
historical_engagement AS (
|
||||
SELECT
|
||||
customer_id,
|
||||
AVG(engagement_score) as historical_score
|
||||
FROM engagement_metrics
|
||||
WHERE metric_date < CURRENT_DATE - INTERVAL '30 days'
|
||||
GROUP BY customer_id
|
||||
)
|
||||
SELECT
|
||||
c.customer_id,
|
||||
c.first_name,
|
||||
c.last_name,
|
||||
re.recent_score,
|
||||
he.historical_score,
|
||||
(re.recent_score - he.historical_score) as score_change
|
||||
FROM customers c
|
||||
JOIN recent_engagement re ON c.customer_id = re.customer_id
|
||||
JOIN historical_engagement he ON c.customer_id = he.customer_id
|
||||
WHERE re.recent_score < he.historical_score
|
||||
ORDER BY score_change ASC
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.6: Customer Cohort Analysis
|
||||
**Objective:** Analyze customer retention by registration cohort
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
DATE_TRUNC('month', c.registration_date) as cohort_month,
|
||||
COUNT(DISTINCT c.customer_id) as total_customers,
|
||||
COUNT(DISTINCT CASE WHEN cp.risk_level = 'LOW' THEN c.customer_id END) as active_customers,
|
||||
COUNT(DISTINCT CASE WHEN cp.risk_level IN ('HIGH', 'CRITICAL') THEN c.customer_id END) as at_risk_customers,
|
||||
ROUND(100.0 * COUNT(DISTINCT CASE WHEN cp.risk_level = 'LOW' THEN c.customer_id END) /
|
||||
COUNT(DISTINCT c.customer_id), 2) as retention_rate
|
||||
FROM customers c
|
||||
LEFT JOIN churn_predictions cp ON c.customer_id = cp.customer_id
|
||||
WHERE c.registration_date >= CURRENT_DATE - INTERVAL '12 months'
|
||||
GROUP BY DATE_TRUNC('month', c.registration_date)
|
||||
ORDER BY cohort_month DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.7: Customer Lifetime Value by Segment
|
||||
**Objective:** Compare CLV metrics across customer segments
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
cs.segment_name,
|
||||
COUNT(clv.customer_id) as customer_count,
|
||||
MIN(clv.clv_score) as min_clv,
|
||||
AVG(clv.clv_score) as avg_clv,
|
||||
MAX(clv.clv_score) as max_clv,
|
||||
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY clv.clv_score) as median_clv,
|
||||
SUM(clv.total_revenue) as total_revenue,
|
||||
AVG(clv.total_transactions) as avg_transactions
|
||||
FROM customer_segments cs
|
||||
JOIN customer_lifetime_value clv ON cs.segment_id = clv.segment_id
|
||||
GROUP BY cs.segment_id, cs.segment_name
|
||||
ORDER BY avg_clv DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.8: Satisfaction by Category
|
||||
**Objective:** Analyze satisfaction scores by feedback category
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
category,
|
||||
COUNT(*) as feedback_count,
|
||||
AVG(nps_score) as avg_nps,
|
||||
AVG(csat_score) as avg_csat,
|
||||
ROUND(100.0 * COUNT(CASE WHEN sentiment = 'POSITIVE' THEN 1 END) / COUNT(*), 2) as positive_pct,
|
||||
ROUND(100.0 * COUNT(CASE WHEN sentiment = 'NEGATIVE' THEN 1 END) / COUNT(*), 2) as negative_pct
|
||||
FROM customer_satisfaction
|
||||
GROUP BY category
|
||||
ORDER BY avg_nps DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.9: Customer Engagement Patterns
|
||||
**Objective:** Find most engaged customers in the last 30 days
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
c.customer_id,
|
||||
c.first_name,
|
||||
c.last_name,
|
||||
SUM(em.login_count) as total_logins,
|
||||
SUM(em.page_views) as total_page_views,
|
||||
SUM(em.time_spent_minutes) as total_time_spent,
|
||||
AVG(em.engagement_score) as avg_engagement_score
|
||||
FROM customers c
|
||||
JOIN engagement_metrics em ON c.customer_id = em.customer_id
|
||||
WHERE em.metric_date >= CURRENT_DATE - INTERVAL '30 days'
|
||||
GROUP BY c.customer_id, c.first_name, c.last_name
|
||||
HAVING AVG(em.engagement_score) > 75
|
||||
ORDER BY avg_engagement_score DESC
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 2.10: Churn Prevention Priority List
|
||||
**Objective:** Create a priority list for customer retention efforts
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Hint:** Combine churn risk with customer value
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
c.customer_id,
|
||||
c.first_name,
|
||||
c.last_name,
|
||||
c.email,
|
||||
clv.clv_score,
|
||||
clv.total_revenue,
|
||||
cp.churn_probability,
|
||||
cp.risk_level,
|
||||
cp.days_since_last_transaction,
|
||||
(clv.clv_score * cp.churn_probability / 100) as retention_priority_score
|
||||
FROM customers c
|
||||
JOIN customer_lifetime_value clv ON c.customer_id = clv.customer_id
|
||||
JOIN churn_predictions cp ON c.customer_id = cp.customer_id
|
||||
WHERE cp.risk_level IN ('HIGH', 'CRITICAL')
|
||||
AND clv.clv_score > 1000
|
||||
ORDER BY retention_priority_score DESC
|
||||
LIMIT 50;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Challenge Exercises
|
||||
|
||||
### Challenge 2.1: Customer Journey Analysis
|
||||
Create a query that shows the customer journey from registration to current status, including:
|
||||
- Registration date
|
||||
- First transaction date
|
||||
- Total transactions
|
||||
- Current segment
|
||||
- Churn risk
|
||||
- Latest satisfaction score
|
||||
|
||||
### Challenge 2.2: Segment Migration Analysis
|
||||
Identify customers who have moved between segments over time (requires historical CLV data)
|
||||
|
||||
### Challenge 2.3: Engagement Correlation
|
||||
Analyze the correlation between engagement scores and satisfaction scores
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
Once you're comfortable with customer analytics, move on to:
|
||||
- **Level 3:** Sales & Revenue Analysis
|
||||
- **Level 4:** KPI Dashboards & Metrics
|
||||
|
||||
@@ -0,0 +1,380 @@
|
||||
# Level 3: Sales & Revenue Analysis
|
||||
|
||||
## Introduction
|
||||
Master sales performance analysis, product analytics, and revenue forecasting - core responsibilities for Data Analysts in sales-driven organizations.
|
||||
|
||||
## Learning Objectives
|
||||
- Analyze sales performance vs targets
|
||||
- Identify top-performing products
|
||||
- Compare sales across channels and regions
|
||||
- Calculate revenue metrics
|
||||
- Analyze sales trends
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.1: Daily Sales Summary
|
||||
**Objective:** Get today's sales summary
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
COUNT(*) as total_sales,
|
||||
SUM(total_amount) as total_revenue,
|
||||
AVG(total_amount) as avg_sale_value,
|
||||
SUM(quantity) as total_units_sold,
|
||||
COUNT(DISTINCT product_id) as products_sold
|
||||
FROM sales_transactions
|
||||
WHERE DATE(sale_date) = CURRENT_DATE;
|
||||
```
|
||||
|
||||
**Expected Result:** Summary of today's sales activity
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.2: Top Products by Revenue
|
||||
**Objective:** Find the top 10 products by total revenue
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
p.product_id,
|
||||
p.product_name,
|
||||
p.product_category,
|
||||
p.product_subcategory,
|
||||
COUNT(st.transaction_id) as sales_count,
|
||||
SUM(st.quantity) as units_sold,
|
||||
SUM(st.total_amount) as total_revenue,
|
||||
AVG(st.total_amount) as avg_sale_value
|
||||
FROM product_catalog p
|
||||
JOIN sales_transactions st ON p.product_id = st.product_id
|
||||
GROUP BY p.product_id, p.product_name, p.product_category, p.product_subcategory
|
||||
ORDER BY total_revenue DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.3: Sales by Channel
|
||||
**Objective:** Compare sales performance across different channels
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
sales_channel,
|
||||
COUNT(*) as transaction_count,
|
||||
SUM(total_amount) as total_revenue,
|
||||
AVG(total_amount) as avg_transaction_value,
|
||||
SUM(quantity) as total_units,
|
||||
ROUND(100.0 * COUNT(*) / SUM(COUNT(*)) OVER (), 2) as pct_of_total_transactions
|
||||
FROM sales_transactions
|
||||
GROUP BY sales_channel
|
||||
ORDER BY total_revenue DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.4: Regional Performance
|
||||
**Objective:** Analyze sales by region
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
region,
|
||||
COUNT(*) as sales_count,
|
||||
SUM(total_amount) as total_revenue,
|
||||
AVG(total_amount) as avg_sale,
|
||||
SUM(discount_amount) as total_discounts,
|
||||
ROUND(100.0 * SUM(discount_amount) / SUM(total_amount), 2) as discount_rate
|
||||
FROM sales_transactions
|
||||
GROUP BY region
|
||||
ORDER BY total_revenue DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.5: Product Category Analysis
|
||||
**Objective:** Compare performance across product categories
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
p.product_category,
|
||||
COUNT(DISTINCT p.product_id) as product_count,
|
||||
COUNT(st.transaction_id) as sales_count,
|
||||
SUM(st.total_amount) as total_revenue,
|
||||
AVG(st.total_amount) as avg_sale_value,
|
||||
SUM(st.quantity) as units_sold,
|
||||
AVG(p.margin_percentage) as avg_margin
|
||||
FROM product_catalog p
|
||||
LEFT JOIN sales_transactions st ON p.product_id = st.product_id
|
||||
GROUP BY p.product_category
|
||||
ORDER BY total_revenue DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.6: Monthly Sales Trend
|
||||
**Objective:** Show sales trends over the past 12 months
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
DATE_TRUNC('month', sale_date) as month,
|
||||
COUNT(*) as transaction_count,
|
||||
SUM(total_amount) as total_revenue,
|
||||
AVG(total_amount) as avg_transaction_value,
|
||||
SUM(quantity) as units_sold
|
||||
FROM sales_transactions
|
||||
WHERE sale_date >= CURRENT_DATE - INTERVAL '12 months'
|
||||
GROUP BY DATE_TRUNC('month', sale_date)
|
||||
ORDER BY month DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.7: Sales Performance vs Target
|
||||
**Objective:** Compare actual sales to targets
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
st.target_period,
|
||||
st.target_category,
|
||||
st.target_value,
|
||||
COALESCE(sp.actual_value, 0) as actual_value,
|
||||
COALESCE(sp.actual_value, 0) - st.target_value as variance,
|
||||
ROUND(100.0 * COALESCE(sp.actual_value, 0) / st.target_value, 2) as achievement_pct,
|
||||
CASE
|
||||
WHEN COALESCE(sp.actual_value, 0) >= st.target_value THEN 'MET'
|
||||
WHEN COALESCE(sp.actual_value, 0) >= st.target_value * 0.9 THEN 'NEAR'
|
||||
ELSE 'MISSED'
|
||||
END as status
|
||||
FROM sales_targets st
|
||||
LEFT JOIN sales_performance sp ON st.target_id = sp.target_id
|
||||
WHERE st.target_period >= CURRENT_DATE - INTERVAL '6 months'
|
||||
ORDER BY st.target_period DESC, st.target_category;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.8: Product Profitability
|
||||
**Objective:** Calculate profit margins by product
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
p.product_id,
|
||||
p.product_name,
|
||||
p.product_category,
|
||||
p.unit_price,
|
||||
p.cost_price,
|
||||
p.margin_percentage,
|
||||
COUNT(st.transaction_id) as sales_count,
|
||||
SUM(st.quantity) as units_sold,
|
||||
SUM(st.total_amount) as total_revenue,
|
||||
SUM(st.quantity * p.cost_price) as total_cost,
|
||||
SUM(st.total_amount) - SUM(st.quantity * p.cost_price) as gross_profit
|
||||
FROM product_catalog p
|
||||
JOIN sales_transactions st ON p.product_id = st.product_id
|
||||
GROUP BY p.product_id, p.product_name, p.product_category, p.unit_price, p.cost_price, p.margin_percentage
|
||||
HAVING SUM(st.total_amount) > 0
|
||||
ORDER BY gross_profit DESC
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.9: Revenue Forecast Accuracy
|
||||
**Objective:** Compare forecasted revenue to actual revenue
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
rf.forecast_period,
|
||||
rf.forecast_category,
|
||||
rf.forecasted_revenue,
|
||||
rf.confidence_level,
|
||||
ms.total_revenue as actual_revenue,
|
||||
(ms.total_revenue - rf.forecasted_revenue) as variance,
|
||||
ROUND(100.0 * ABS(ms.total_revenue - rf.forecasted_revenue) / rf.forecasted_revenue, 2) as error_pct
|
||||
FROM revenue_forecasts rf
|
||||
JOIN monthly_summaries ms ON
|
||||
EXTRACT(MONTH FROM rf.forecast_period) = ms.summary_month AND
|
||||
EXTRACT(YEAR FROM rf.forecast_period) = ms.summary_year
|
||||
WHERE rf.forecast_period >= CURRENT_DATE - INTERVAL '12 months'
|
||||
ORDER BY rf.forecast_period DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.10: Sales Velocity Analysis
|
||||
**Objective:** Identify fast-moving vs slow-moving products
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
WITH product_sales AS (
|
||||
SELECT
|
||||
p.product_id,
|
||||
p.product_name,
|
||||
p.product_category,
|
||||
COUNT(st.transaction_id) as sales_count,
|
||||
SUM(st.quantity) as units_sold,
|
||||
MIN(st.sale_date) as first_sale,
|
||||
MAX(st.sale_date) as last_sale,
|
||||
EXTRACT(DAY FROM MAX(st.sale_date) - MIN(st.sale_date)) as days_on_market
|
||||
FROM product_catalog p
|
||||
LEFT JOIN sales_transactions st ON p.product_id = st.product_id
|
||||
WHERE p.is_active = TRUE
|
||||
GROUP BY p.product_id, p.product_name, p.product_category
|
||||
)
|
||||
SELECT
|
||||
product_id,
|
||||
product_name,
|
||||
product_category,
|
||||
sales_count,
|
||||
units_sold,
|
||||
days_on_market,
|
||||
CASE
|
||||
WHEN days_on_market > 0 THEN ROUND(units_sold::DECIMAL / days_on_market, 2)
|
||||
ELSE 0
|
||||
END as units_per_day,
|
||||
CASE
|
||||
WHEN days_on_market > 0 AND (units_sold::DECIMAL / days_on_market) > 10 THEN 'FAST'
|
||||
WHEN days_on_market > 0 AND (units_sold::DECIMAL / days_on_market) > 5 THEN 'MEDIUM'
|
||||
WHEN days_on_market > 0 THEN 'SLOW'
|
||||
ELSE 'NO_SALES'
|
||||
END as velocity_category
|
||||
FROM product_sales
|
||||
ORDER BY units_per_day DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.11: Discount Impact Analysis
|
||||
**Objective:** Analyze the impact of discounts on sales
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
CASE
|
||||
WHEN discount_amount = 0 THEN 'No Discount'
|
||||
WHEN discount_amount / total_amount < 0.1 THEN '< 10%'
|
||||
WHEN discount_amount / total_amount < 0.2 THEN '10-20%'
|
||||
WHEN discount_amount / total_amount < 0.3 THEN '20-30%'
|
||||
ELSE '> 30%'
|
||||
END as discount_tier,
|
||||
COUNT(*) as transaction_count,
|
||||
AVG(total_amount) as avg_sale_value,
|
||||
SUM(total_amount) as total_revenue,
|
||||
SUM(discount_amount) as total_discounts,
|
||||
AVG(quantity) as avg_quantity
|
||||
FROM sales_transactions
|
||||
GROUP BY discount_tier
|
||||
ORDER BY
|
||||
CASE discount_tier
|
||||
WHEN 'No Discount' THEN 1
|
||||
WHEN '< 10%' THEN 2
|
||||
WHEN '10-20%' THEN 3
|
||||
WHEN '20-30%' THEN 4
|
||||
ELSE 5
|
||||
END;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 3.12: Cross-Sell Opportunities
|
||||
**Objective:** Find products frequently purchased together
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Hint:** Use self-join on transaction_id
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
p1.product_name as product_1,
|
||||
p2.product_name as product_2,
|
||||
COUNT(*) as times_purchased_together
|
||||
FROM sales_transactions st1
|
||||
JOIN sales_transactions st2 ON st1.transaction_id = st2.transaction_id
|
||||
AND st1.product_id < st2.product_id
|
||||
JOIN product_catalog p1 ON st1.product_id = p1.product_id
|
||||
JOIN product_catalog p2 ON st2.product_id = p2.product_id
|
||||
GROUP BY p1.product_id, p1.product_name, p2.product_id, p2.product_name
|
||||
HAVING COUNT(*) > 10
|
||||
ORDER BY times_purchased_together DESC
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Challenge Exercises
|
||||
|
||||
### Challenge 3.1: Sales Seasonality
|
||||
Identify seasonal patterns in sales data by analyzing month-over-month and year-over-year trends
|
||||
|
||||
### Challenge 3.2: Customer Purchase Patterns
|
||||
Analyze average time between purchases and identify customers with regular buying patterns
|
||||
|
||||
### Challenge 3.3: Product Launch Performance
|
||||
Compare new product performance (launched in last 6 months) vs established products
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
Once you're comfortable with sales analysis, move on to:
|
||||
- **Level 4:** KPI Dashboards & Metrics
|
||||
- **Level 6:** Fraud Detection (Advanced)
|
||||
|
||||
@@ -0,0 +1,431 @@
|
||||
# Level 4: KPI Dashboards & Metrics
|
||||
|
||||
## Introduction
|
||||
Learn to build and maintain KPI dashboards - a critical skill for Data Analysts supporting business decision-making.
|
||||
|
||||
## Learning Objectives
|
||||
- Track key performance indicators
|
||||
- Calculate trend metrics
|
||||
- Build dashboard queries
|
||||
- Monitor data quality
|
||||
- Create executive reports
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.1: Current KPI Status
|
||||
**Objective:** Get current status of all active KPIs
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
kd.kpi_name,
|
||||
kd.kpi_category,
|
||||
kd.target_value,
|
||||
dm.metric_value as current_value,
|
||||
dm.status,
|
||||
kd.unit_of_measure,
|
||||
ROUND(100.0 * dm.metric_value / kd.target_value, 2) as achievement_pct
|
||||
FROM kpi_definitions kd
|
||||
LEFT JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
WHERE kd.is_active = TRUE
|
||||
AND dm.metric_date = CURRENT_DATE
|
||||
ORDER BY kd.kpi_category, kd.kpi_name;
|
||||
```
|
||||
|
||||
**Expected Result:** Dashboard view of all KPIs with current status
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.2: KPIs Below Target
|
||||
**Objective:** Identify KPIs that are underperforming
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
kd.kpi_name,
|
||||
kd.kpi_category,
|
||||
kd.target_value,
|
||||
dm.metric_value as current_value,
|
||||
kd.threshold_warning,
|
||||
kd.threshold_critical,
|
||||
dm.status,
|
||||
(dm.metric_value - kd.target_value) as variance,
|
||||
ROUND(100.0 * (dm.metric_value - kd.target_value) / kd.target_value, 2) as variance_pct
|
||||
FROM kpi_definitions kd
|
||||
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
WHERE kd.is_active = TRUE
|
||||
AND dm.metric_date = CURRENT_DATE
|
||||
AND dm.metric_value < kd.target_value
|
||||
ORDER BY variance_pct ASC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.3: KPI Trend Analysis (7 Days)
|
||||
**Objective:** Show 7-day trend for critical KPIs
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
kd.kpi_name,
|
||||
dm.metric_date,
|
||||
dm.metric_value,
|
||||
kd.target_value,
|
||||
dm.vs_previous_day_percentage,
|
||||
dm.status
|
||||
FROM kpi_definitions kd
|
||||
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
WHERE kd.is_active = TRUE
|
||||
AND dm.metric_date >= CURRENT_DATE - INTERVAL '7 days'
|
||||
AND kd.kpi_category IN ('SALES', 'CUSTOMER')
|
||||
ORDER BY kd.kpi_name, dm.metric_date DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.4: Monthly Performance Summary
|
||||
**Objective:** Aggregate monthly business metrics
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
summary_year,
|
||||
summary_month,
|
||||
TO_CHAR(TO_DATE(summary_month::TEXT, 'MM'), 'Month') as month_name,
|
||||
total_revenue,
|
||||
total_transactions,
|
||||
total_customers,
|
||||
new_customers,
|
||||
average_transaction_value,
|
||||
ROUND(100.0 * new_customers / total_customers, 2) as new_customer_pct
|
||||
FROM monthly_summaries
|
||||
WHERE summary_year >= EXTRACT(YEAR FROM CURRENT_DATE) - 1
|
||||
ORDER BY summary_year DESC, summary_month DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.5: Year-over-Year Comparison
|
||||
**Objective:** Compare this year's performance to last year
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
WITH current_year AS (
|
||||
SELECT
|
||||
summary_month,
|
||||
total_revenue,
|
||||
total_transactions,
|
||||
new_customers
|
||||
FROM monthly_summaries
|
||||
WHERE summary_year = EXTRACT(YEAR FROM CURRENT_DATE)
|
||||
),
|
||||
previous_year AS (
|
||||
SELECT
|
||||
summary_month,
|
||||
total_revenue,
|
||||
total_transactions,
|
||||
new_customers
|
||||
FROM monthly_summaries
|
||||
WHERE summary_year = EXTRACT(YEAR FROM CURRENT_DATE) - 1
|
||||
)
|
||||
SELECT
|
||||
cy.summary_month,
|
||||
TO_CHAR(TO_DATE(cy.summary_month::TEXT, 'MM'), 'Month') as month_name,
|
||||
cy.total_revenue as current_year_revenue,
|
||||
py.total_revenue as previous_year_revenue,
|
||||
(cy.total_revenue - py.total_revenue) as revenue_change,
|
||||
ROUND(100.0 * (cy.total_revenue - py.total_revenue) / py.total_revenue, 2) as revenue_growth_pct,
|
||||
cy.total_transactions as current_year_transactions,
|
||||
py.total_transactions as previous_year_transactions
|
||||
FROM current_year cy
|
||||
LEFT JOIN previous_year py ON cy.summary_month = py.summary_month
|
||||
ORDER BY cy.summary_month;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.6: Dashboard Snapshot
|
||||
**Objective:** Create a pre-calculated dashboard snapshot
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
INSERT INTO dashboard_snapshots (
|
||||
snapshot_date, dashboard_name, metric_name, metric_value, metric_category
|
||||
)
|
||||
SELECT
|
||||
CURRENT_DATE as snapshot_date,
|
||||
'Executive Dashboard' as dashboard_name,
|
||||
kd.kpi_name as metric_name,
|
||||
dm.metric_value,
|
||||
kd.kpi_category as metric_category
|
||||
FROM kpi_definitions kd
|
||||
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
WHERE kd.is_active = TRUE
|
||||
AND dm.metric_date = CURRENT_DATE;
|
||||
|
||||
-- View the snapshot
|
||||
SELECT * FROM dashboard_snapshots
|
||||
WHERE snapshot_date = CURRENT_DATE
|
||||
ORDER BY metric_category, metric_name;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.7: Trend Detection
|
||||
**Objective:** Identify metrics with consistent upward or downward trends
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
WITH trend_data AS (
|
||||
SELECT
|
||||
kd.kpi_name,
|
||||
kd.kpi_category,
|
||||
dm.metric_date,
|
||||
dm.metric_value,
|
||||
dm.vs_previous_day_percentage,
|
||||
ROW_NUMBER() OVER (PARTITION BY kd.kpi_id ORDER BY dm.metric_date DESC) as day_rank
|
||||
FROM kpi_definitions kd
|
||||
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
WHERE dm.metric_date >= CURRENT_DATE - INTERVAL '7 days'
|
||||
)
|
||||
SELECT
|
||||
kpi_name,
|
||||
kpi_category,
|
||||
COUNT(*) as days_tracked,
|
||||
AVG(vs_previous_day_percentage) as avg_daily_change,
|
||||
MIN(metric_value) as min_value,
|
||||
MAX(metric_value) as max_value,
|
||||
CASE
|
||||
WHEN AVG(vs_previous_day_percentage) > 5 THEN 'STRONG UPWARD'
|
||||
WHEN AVG(vs_previous_day_percentage) > 0 THEN 'UPWARD'
|
||||
WHEN AVG(vs_previous_day_percentage) > -5 THEN 'DOWNWARD'
|
||||
ELSE 'STRONG DOWNWARD'
|
||||
END as trend_direction
|
||||
FROM trend_data
|
||||
GROUP BY kpi_name, kpi_category
|
||||
ORDER BY avg_daily_change DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.8: Data Quality Monitoring
|
||||
**Objective:** Track data quality metrics
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
check_date,
|
||||
table_name,
|
||||
check_type,
|
||||
records_checked,
|
||||
records_failed,
|
||||
ROUND(100.0 * records_failed / records_checked, 2) as failure_rate,
|
||||
status,
|
||||
error_details
|
||||
FROM data_quality_checks
|
||||
WHERE check_date >= CURRENT_DATE - INTERVAL '7 days'
|
||||
AND status IN ('WARNING', 'FAILED')
|
||||
ORDER BY check_date DESC, failure_rate DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.9: Report Execution History
|
||||
**Objective:** Monitor report generation and performance
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
rd.report_name,
|
||||
rd.report_category,
|
||||
re.execution_date,
|
||||
re.execution_status,
|
||||
re.execution_time_seconds,
|
||||
re.records_generated,
|
||||
re.file_size_kb
|
||||
FROM report_definitions rd
|
||||
JOIN report_executions re ON rd.report_id = re.report_id
|
||||
WHERE re.execution_date >= CURRENT_DATE - INTERVAL '30 days'
|
||||
ORDER BY re.execution_date DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.10: Executive Summary Dashboard
|
||||
**Objective:** Create a comprehensive executive summary
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
WITH today_metrics AS (
|
||||
SELECT
|
||||
SUM(CASE WHEN kd.kpi_category = 'SALES' THEN dm.metric_value ELSE 0 END) as total_sales,
|
||||
AVG(CASE WHEN kd.kpi_name = 'Customer Engagement Score' THEN dm.metric_value END) as avg_engagement,
|
||||
AVG(CASE WHEN kd.kpi_name = 'Net Promoter Score' THEN dm.metric_value END) as avg_nps,
|
||||
COUNT(CASE WHEN dm.status = 'CRITICAL' THEN 1 END) as critical_kpis,
|
||||
COUNT(CASE WHEN dm.status = 'WARNING' THEN 1 END) as warning_kpis,
|
||||
COUNT(CASE WHEN dm.status = 'ON_TARGET' THEN 1 END) as on_target_kpis
|
||||
FROM kpi_definitions kd
|
||||
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
WHERE dm.metric_date = CURRENT_DATE
|
||||
),
|
||||
monthly_summary AS (
|
||||
SELECT
|
||||
total_revenue,
|
||||
total_transactions,
|
||||
new_customers,
|
||||
average_transaction_value
|
||||
FROM monthly_summaries
|
||||
WHERE summary_year = EXTRACT(YEAR FROM CURRENT_DATE)
|
||||
AND summary_month = EXTRACT(MONTH FROM CURRENT_DATE)
|
||||
)
|
||||
SELECT
|
||||
'Executive Summary' as report_title,
|
||||
CURRENT_DATE as report_date,
|
||||
tm.total_sales,
|
||||
tm.avg_engagement,
|
||||
tm.avg_nps,
|
||||
tm.critical_kpis,
|
||||
tm.warning_kpis,
|
||||
tm.on_target_kpis,
|
||||
ms.total_revenue as mtd_revenue,
|
||||
ms.total_transactions as mtd_transactions,
|
||||
ms.new_customers as mtd_new_customers
|
||||
FROM today_metrics tm
|
||||
CROSS JOIN monthly_summary ms;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.11: Moving Average Calculation
|
||||
**Objective:** Calculate 7-day moving average for key metrics
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
SELECT
|
||||
kd.kpi_name,
|
||||
dm.metric_date,
|
||||
dm.metric_value,
|
||||
AVG(dm.metric_value) OVER (
|
||||
PARTITION BY kd.kpi_id
|
||||
ORDER BY dm.metric_date
|
||||
ROWS BETWEEN 6 PRECEDING AND CURRENT ROW
|
||||
) as moving_avg_7day,
|
||||
kd.target_value
|
||||
FROM kpi_definitions kd
|
||||
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
WHERE kd.kpi_name IN ('Daily Revenue', 'Transactions Per Day', 'Customer Engagement Score')
|
||||
AND dm.metric_date >= CURRENT_DATE - INTERVAL '30 days'
|
||||
ORDER BY kd.kpi_name, dm.metric_date DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exercise 4.12: Anomaly Detection
|
||||
**Objective:** Identify unusual metric values
|
||||
|
||||
```sql
|
||||
-- Your query here
|
||||
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```sql
|
||||
WITH metric_stats AS (
|
||||
SELECT
|
||||
kpi_id,
|
||||
AVG(metric_value) as avg_value,
|
||||
STDDEV(metric_value) as stddev_value
|
||||
FROM daily_metrics
|
||||
WHERE metric_date >= CURRENT_DATE - INTERVAL '30 days'
|
||||
GROUP BY kpi_id
|
||||
)
|
||||
SELECT
|
||||
kd.kpi_name,
|
||||
dm.metric_date,
|
||||
dm.metric_value,
|
||||
ms.avg_value,
|
||||
ms.stddev_value,
|
||||
(dm.metric_value - ms.avg_value) / NULLIF(ms.stddev_value, 0) as z_score,
|
||||
CASE
|
||||
WHEN ABS((dm.metric_value - ms.avg_value) / NULLIF(ms.stddev_value, 0)) > 3 THEN 'EXTREME ANOMALY'
|
||||
WHEN ABS((dm.metric_value - ms.avg_value) / NULLIF(ms.stddev_value, 0)) > 2 THEN 'ANOMALY'
|
||||
ELSE 'NORMAL'
|
||||
END as anomaly_status
|
||||
FROM kpi_definitions kd
|
||||
JOIN daily_metrics dm ON kd.kpi_id = dm.kpi_id
|
||||
JOIN metric_stats ms ON kd.kpi_id = ms.kpi_id
|
||||
WHERE dm.metric_date >= CURRENT_DATE - INTERVAL '7 days'
|
||||
AND ABS((dm.metric_value - ms.avg_value) / NULLIF(ms.stddev_value, 0)) > 2
|
||||
ORDER BY ABS((dm.metric_value - ms.avg_value) / NULLIF(ms.stddev_value, 0)) DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Challenge Exercises
|
||||
|
||||
### Challenge 4.1: Custom Dashboard Builder
|
||||
Create a flexible query that can generate different dashboard views based on parameters (date range, KPI category, etc.)
|
||||
|
||||
### Challenge 4.2: Predictive Alerting
|
||||
Build a query that predicts which KPIs are likely to miss targets based on current trends
|
||||
|
||||
### Challenge 4.3: Correlation Analysis
|
||||
Identify correlations between different KPIs (e.g., does customer engagement correlate with revenue?)
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
You've now mastered the core Data Analyst skills! Continue with:
|
||||
- **Level 5:** Advanced Analytics (Window Functions, CTEs)
|
||||
- **Level 6:** Fraud Detection (Advanced Pattern Recognition)
|
||||
- Build your own custom dashboards and reports!
|
||||
|
||||
+341
-2
@@ -1,6 +1,6 @@
|
||||
-- ============================================================================
|
||||
-- Financial Fraud Detection Database Schema
|
||||
-- Purpose: Educational SQL learning with realistic fraud investigation scenarios
|
||||
-- Business Analytics Database Schema
|
||||
-- Purpose: Data Analyst training with routine/semi-routine analysis scenarios
|
||||
-- ============================================================================
|
||||
-- This script is IDEMPOTENT - it will drop and recreate all objects
|
||||
-- ============================================================================
|
||||
@@ -23,6 +23,27 @@ DROP FUNCTION IF EXISTS check_suspicious_transaction() CASCADE;
|
||||
DROP FUNCTION IF EXISTS generate_fraud_score(DECIMAL, BOOLEAN, DECIMAL, INT, BOOLEAN) CASCADE;
|
||||
|
||||
-- Drop tables in reverse dependency order
|
||||
-- New analytics tables
|
||||
DROP TABLE IF EXISTS report_executions CASCADE;
|
||||
DROP TABLE IF EXISTS report_definitions CASCADE;
|
||||
DROP TABLE IF EXISTS data_quality_checks CASCADE;
|
||||
DROP TABLE IF EXISTS dashboard_snapshots CASCADE;
|
||||
DROP TABLE IF EXISTS trend_analysis CASCADE;
|
||||
DROP TABLE IF EXISTS monthly_summaries CASCADE;
|
||||
DROP TABLE IF EXISTS daily_metrics CASCADE;
|
||||
DROP TABLE IF EXISTS kpi_definitions CASCADE;
|
||||
DROP TABLE IF EXISTS revenue_forecasts CASCADE;
|
||||
DROP TABLE IF EXISTS sales_performance CASCADE;
|
||||
DROP TABLE IF EXISTS sales_targets CASCADE;
|
||||
DROP TABLE IF EXISTS sales_transactions CASCADE;
|
||||
DROP TABLE IF EXISTS product_catalog CASCADE;
|
||||
DROP TABLE IF EXISTS engagement_metrics CASCADE;
|
||||
DROP TABLE IF EXISTS customer_satisfaction CASCADE;
|
||||
DROP TABLE IF EXISTS churn_predictions CASCADE;
|
||||
DROP TABLE IF EXISTS customer_lifetime_value CASCADE;
|
||||
DROP TABLE IF EXISTS customer_segments CASCADE;
|
||||
|
||||
-- Existing tables
|
||||
DROP TABLE IF EXISTS audit_log CASCADE;
|
||||
DROP TABLE IF EXISTS suspicious_activity_reports CASCADE;
|
||||
DROP TABLE IF EXISTS case_alerts CASCADE;
|
||||
@@ -475,3 +496,321 @@ COMMENT ON TABLE cards IS 'Payment cards linked to accounts';
|
||||
COMMENT ON TABLE devices IS 'Device fingerprints for fraud detection';
|
||||
COMMENT ON TABLE login_sessions IS 'Login history for account takeover detection';
|
||||
|
||||
-- ============================================================================
|
||||
-- CUSTOMER ANALYTICS TABLES (for routine customer analysis)
|
||||
-- ============================================================================
|
||||
|
||||
CREATE TABLE customer_segments (
|
||||
segment_id SERIAL PRIMARY KEY,
|
||||
segment_name VARCHAR(100) NOT NULL UNIQUE,
|
||||
segment_description TEXT,
|
||||
criteria_definition JSONB,
|
||||
min_clv DECIMAL(15,2),
|
||||
max_clv DECIMAL(15,2),
|
||||
created_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
CREATE TABLE customer_lifetime_value (
|
||||
clv_id SERIAL PRIMARY KEY,
|
||||
customer_id INT NOT NULL REFERENCES customers(customer_id),
|
||||
calculation_date DATE NOT NULL,
|
||||
total_revenue DECIMAL(15,2) DEFAULT 0,
|
||||
total_transactions INT DEFAULT 0,
|
||||
average_order_value DECIMAL(15,2) DEFAULT 0,
|
||||
predicted_future_value DECIMAL(15,2),
|
||||
clv_score DECIMAL(10,2),
|
||||
segment_id INT REFERENCES customer_segments(segment_id),
|
||||
UNIQUE(customer_id, calculation_date)
|
||||
);
|
||||
|
||||
CREATE TABLE churn_predictions (
|
||||
prediction_id SERIAL PRIMARY KEY,
|
||||
customer_id INT NOT NULL REFERENCES customers(customer_id),
|
||||
prediction_date DATE NOT NULL,
|
||||
churn_probability DECIMAL(5,2) CHECK (churn_probability BETWEEN 0 AND 100),
|
||||
risk_level VARCHAR(20) CHECK (risk_level IN ('LOW', 'MEDIUM', 'HIGH', 'CRITICAL')),
|
||||
last_transaction_date DATE,
|
||||
days_since_last_transaction INT,
|
||||
engagement_score DECIMAL(5,2),
|
||||
recommended_action TEXT,
|
||||
UNIQUE(customer_id, prediction_date)
|
||||
);
|
||||
|
||||
CREATE TABLE customer_satisfaction (
|
||||
satisfaction_id SERIAL PRIMARY KEY,
|
||||
customer_id INT NOT NULL REFERENCES customers(customer_id),
|
||||
survey_date DATE NOT NULL,
|
||||
nps_score INT CHECK (nps_score BETWEEN -100 AND 100),
|
||||
csat_score DECIMAL(3,2) CHECK (csat_score BETWEEN 1 AND 5),
|
||||
feedback_text TEXT,
|
||||
category VARCHAR(50) CHECK (category IN ('PRODUCT', 'SERVICE', 'SUPPORT', 'BILLING', 'OTHER')),
|
||||
sentiment VARCHAR(20) CHECK (sentiment IN ('POSITIVE', 'NEUTRAL', 'NEGATIVE'))
|
||||
);
|
||||
|
||||
CREATE TABLE engagement_metrics (
|
||||
metric_id SERIAL PRIMARY KEY,
|
||||
customer_id INT NOT NULL REFERENCES customers(customer_id),
|
||||
metric_date DATE NOT NULL,
|
||||
login_count INT DEFAULT 0,
|
||||
page_views INT DEFAULT 0,
|
||||
time_spent_minutes INT DEFAULT 0,
|
||||
features_used JSONB,
|
||||
support_tickets_opened INT DEFAULT 0,
|
||||
engagement_score DECIMAL(5,2),
|
||||
UNIQUE(customer_id, metric_date)
|
||||
);
|
||||
|
||||
-- ============================================================================
|
||||
-- SALES & REVENUE ANALYTICS TABLES (for semi-routine sales reporting)
|
||||
-- ============================================================================
|
||||
|
||||
CREATE TABLE product_catalog (
|
||||
product_id SERIAL PRIMARY KEY,
|
||||
product_name VARCHAR(200) NOT NULL,
|
||||
product_category VARCHAR(100),
|
||||
product_subcategory VARCHAR(100),
|
||||
unit_price DECIMAL(15,2) NOT NULL,
|
||||
cost_price DECIMAL(15,2),
|
||||
margin_percentage DECIMAL(5,2),
|
||||
is_active BOOLEAN DEFAULT TRUE,
|
||||
launch_date DATE,
|
||||
discontinued_date DATE
|
||||
);
|
||||
|
||||
CREATE TABLE sales_transactions (
|
||||
sale_id SERIAL PRIMARY KEY,
|
||||
transaction_id INT REFERENCES transactions(transaction_id),
|
||||
product_id INT REFERENCES product_catalog(product_id),
|
||||
quantity INT NOT NULL DEFAULT 1,
|
||||
unit_price DECIMAL(15,2) NOT NULL,
|
||||
discount_amount DECIMAL(15,2) DEFAULT 0,
|
||||
tax_amount DECIMAL(15,2) DEFAULT 0,
|
||||
total_amount DECIMAL(15,2) NOT NULL,
|
||||
sale_date TIMESTAMP NOT NULL,
|
||||
sales_channel VARCHAR(50) CHECK (sales_channel IN ('ONLINE', 'STORE', 'PHONE', 'MOBILE_APP')),
|
||||
sales_rep_id INT,
|
||||
region VARCHAR(100)
|
||||
);
|
||||
|
||||
CREATE TABLE sales_targets (
|
||||
target_id SERIAL PRIMARY KEY,
|
||||
target_period VARCHAR(20) CHECK (target_period IN ('DAILY', 'WEEKLY', 'MONTHLY', 'QUARTERLY', 'YEARLY')),
|
||||
start_date DATE NOT NULL,
|
||||
end_date DATE NOT NULL,
|
||||
product_category VARCHAR(100),
|
||||
region VARCHAR(100),
|
||||
target_revenue DECIMAL(15,2),
|
||||
target_units INT,
|
||||
target_customers INT,
|
||||
created_by VARCHAR(100),
|
||||
created_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
CREATE TABLE sales_performance (
|
||||
performance_id SERIAL PRIMARY KEY,
|
||||
period_date DATE NOT NULL,
|
||||
period_type VARCHAR(20) CHECK (period_type IN ('DAILY', 'WEEKLY', 'MONTHLY', 'QUARTERLY')),
|
||||
product_id INT REFERENCES product_catalog(product_id),
|
||||
region VARCHAR(100),
|
||||
total_revenue DECIMAL(15,2) DEFAULT 0,
|
||||
total_units_sold INT DEFAULT 0,
|
||||
total_transactions INT DEFAULT 0,
|
||||
unique_customers INT DEFAULT 0,
|
||||
average_order_value DECIMAL(15,2),
|
||||
vs_target_percentage DECIMAL(5,2),
|
||||
UNIQUE(period_date, period_type, product_id, region)
|
||||
);
|
||||
|
||||
CREATE TABLE revenue_forecasts (
|
||||
forecast_id SERIAL PRIMARY KEY,
|
||||
forecast_date DATE NOT NULL,
|
||||
forecast_period_start DATE NOT NULL,
|
||||
forecast_period_end DATE NOT NULL,
|
||||
product_category VARCHAR(100),
|
||||
region VARCHAR(100),
|
||||
forecasted_revenue DECIMAL(15,2),
|
||||
confidence_level VARCHAR(20) CHECK (confidence_level IN ('LOW', 'MEDIUM', 'HIGH')),
|
||||
forecast_method VARCHAR(50) CHECK (forecast_method IN ('HISTORICAL', 'TREND', 'SEASONAL', 'ML')),
|
||||
actual_revenue DECIMAL(15,2),
|
||||
variance_percentage DECIMAL(5,2)
|
||||
);
|
||||
|
||||
|
||||
-- ============================================================================
|
||||
-- OPERATIONAL METRICS & KPI TABLES (for dashboard creation)
|
||||
-- ============================================================================
|
||||
|
||||
CREATE TABLE kpi_definitions (
|
||||
kpi_id SERIAL PRIMARY KEY,
|
||||
kpi_name VARCHAR(100) NOT NULL UNIQUE,
|
||||
kpi_description TEXT,
|
||||
kpi_category VARCHAR(50) CHECK (kpi_category IN ('SALES', 'CUSTOMER', 'OPERATIONAL', 'FINANCIAL')),
|
||||
calculation_formula TEXT,
|
||||
target_value DECIMAL(15,2),
|
||||
threshold_warning DECIMAL(15,2),
|
||||
threshold_critical DECIMAL(15,2),
|
||||
unit_of_measure VARCHAR(50),
|
||||
refresh_frequency VARCHAR(20) CHECK (refresh_frequency IN ('REALTIME', 'HOURLY', 'DAILY', 'WEEKLY')),
|
||||
is_active BOOLEAN DEFAULT TRUE
|
||||
);
|
||||
|
||||
CREATE TABLE daily_metrics (
|
||||
metric_id SERIAL PRIMARY KEY,
|
||||
metric_date DATE NOT NULL,
|
||||
kpi_id INT NOT NULL REFERENCES kpi_definitions(kpi_id),
|
||||
metric_value DECIMAL(15,2),
|
||||
vs_previous_day_percentage DECIMAL(5,2),
|
||||
vs_previous_week_percentage DECIMAL(5,2),
|
||||
vs_previous_month_percentage DECIMAL(5,2),
|
||||
status VARCHAR(20) CHECK (status IN ('ON_TARGET', 'WARNING', 'CRITICAL')),
|
||||
notes TEXT,
|
||||
UNIQUE(metric_date, kpi_id)
|
||||
);
|
||||
|
||||
CREATE TABLE monthly_summaries (
|
||||
summary_id SERIAL PRIMARY KEY,
|
||||
summary_month INT CHECK (summary_month BETWEEN 1 AND 12),
|
||||
summary_year INT,
|
||||
total_revenue DECIMAL(15,2) DEFAULT 0,
|
||||
total_transactions INT DEFAULT 0,
|
||||
total_customers INT DEFAULT 0,
|
||||
new_customers INT DEFAULT 0,
|
||||
churned_customers INT DEFAULT 0,
|
||||
average_transaction_value DECIMAL(15,2),
|
||||
customer_acquisition_cost DECIMAL(15,2),
|
||||
customer_lifetime_value DECIMAL(15,2),
|
||||
net_promoter_score DECIMAL(5,2),
|
||||
gross_margin_percentage DECIMAL(5,2),
|
||||
UNIQUE(summary_month, summary_year)
|
||||
);
|
||||
|
||||
CREATE TABLE trend_analysis (
|
||||
trend_id SERIAL PRIMARY KEY,
|
||||
kpi_id INT NOT NULL REFERENCES kpi_definitions(kpi_id),
|
||||
analysis_date DATE NOT NULL,
|
||||
period_start DATE NOT NULL,
|
||||
period_end DATE NOT NULL,
|
||||
trend_direction VARCHAR(20) CHECK (trend_direction IN ('UP', 'DOWN', 'FLAT')),
|
||||
trend_strength VARCHAR(20) CHECK (trend_strength IN ('WEAK', 'MODERATE', 'STRONG')),
|
||||
moving_average_7day DECIMAL(15,2),
|
||||
moving_average_30day DECIMAL(15,2),
|
||||
seasonality_detected BOOLEAN DEFAULT FALSE,
|
||||
anomalies_detected BOOLEAN DEFAULT FALSE,
|
||||
statistical_significance DECIMAL(5,4)
|
||||
);
|
||||
|
||||
CREATE TABLE dashboard_snapshots (
|
||||
snapshot_id SERIAL PRIMARY KEY,
|
||||
dashboard_name VARCHAR(100) NOT NULL,
|
||||
snapshot_timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
data_payload JSONB,
|
||||
refresh_duration_seconds DECIMAL(10,2),
|
||||
row_count INT,
|
||||
last_updated_by VARCHAR(100)
|
||||
);
|
||||
|
||||
-- ============================================================================
|
||||
-- BUSINESS INTELLIGENCE & REPORTING TABLES
|
||||
-- ============================================================================
|
||||
|
||||
CREATE TABLE report_definitions (
|
||||
report_id SERIAL PRIMARY KEY,
|
||||
report_name VARCHAR(200) NOT NULL UNIQUE,
|
||||
report_description TEXT,
|
||||
report_category VARCHAR(100),
|
||||
sql_query_template TEXT,
|
||||
parameters JSONB,
|
||||
output_format VARCHAR(20) CHECK (output_format IN ('PDF', 'EXCEL', 'CSV', 'HTML')),
|
||||
schedule_frequency VARCHAR(50),
|
||||
recipients TEXT,
|
||||
is_active BOOLEAN DEFAULT TRUE,
|
||||
created_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
CREATE TABLE report_executions (
|
||||
execution_id SERIAL PRIMARY KEY,
|
||||
report_id INT NOT NULL REFERENCES report_definitions(report_id),
|
||||
execution_timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
parameters_used JSONB,
|
||||
row_count INT,
|
||||
execution_duration_seconds DECIMAL(10,2),
|
||||
status VARCHAR(20) CHECK (status IN ('SUCCESS', 'FAILED', 'TIMEOUT')),
|
||||
error_message TEXT,
|
||||
output_file_path TEXT,
|
||||
executed_by VARCHAR(100)
|
||||
);
|
||||
|
||||
CREATE TABLE data_quality_checks (
|
||||
check_id SERIAL PRIMARY KEY,
|
||||
check_name VARCHAR(200) NOT NULL,
|
||||
table_name VARCHAR(100) NOT NULL,
|
||||
column_name VARCHAR(100),
|
||||
check_type VARCHAR(50) CHECK (check_type IN ('NULL_CHECK', 'RANGE_CHECK', 'UNIQUENESS', 'REFERENTIAL_INTEGRITY', 'FORMAT_CHECK')),
|
||||
check_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
records_checked INT,
|
||||
records_failed INT,
|
||||
failure_percentage DECIMAL(5,2),
|
||||
status VARCHAR(20) CHECK (status IN ('PASS', 'FAIL', 'WARNING')),
|
||||
remediation_notes TEXT
|
||||
);
|
||||
|
||||
-- ============================================================================
|
||||
-- INDEXES FOR NEW ANALYTICS TABLES
|
||||
-- ============================================================================
|
||||
|
||||
-- Customer Analytics indexes
|
||||
CREATE INDEX idx_clv_customer ON customer_lifetime_value(customer_id);
|
||||
CREATE INDEX idx_clv_date ON customer_lifetime_value(calculation_date);
|
||||
CREATE INDEX idx_clv_segment ON customer_lifetime_value(segment_id);
|
||||
CREATE INDEX idx_churn_customer ON churn_predictions(customer_id);
|
||||
CREATE INDEX idx_churn_risk ON churn_predictions(risk_level);
|
||||
CREATE INDEX idx_satisfaction_customer ON customer_satisfaction(customer_id);
|
||||
CREATE INDEX idx_engagement_customer ON engagement_metrics(customer_id);
|
||||
CREATE INDEX idx_engagement_date ON engagement_metrics(metric_date);
|
||||
|
||||
-- Sales Analytics indexes
|
||||
CREATE INDEX idx_sales_trans_product ON sales_transactions(product_id);
|
||||
CREATE INDEX idx_sales_trans_date ON sales_transactions(sale_date);
|
||||
CREATE INDEX idx_sales_trans_channel ON sales_transactions(sales_channel);
|
||||
CREATE INDEX idx_sales_perf_date ON sales_performance(period_date);
|
||||
CREATE INDEX idx_sales_perf_product ON sales_performance(product_id);
|
||||
CREATE INDEX idx_product_category ON product_catalog(product_category);
|
||||
CREATE INDEX idx_product_active ON product_catalog(is_active);
|
||||
|
||||
-- KPI & Metrics indexes
|
||||
CREATE INDEX idx_daily_metrics_date ON daily_metrics(metric_date);
|
||||
CREATE INDEX idx_daily_metrics_kpi ON daily_metrics(kpi_id);
|
||||
CREATE INDEX idx_monthly_summaries_period ON monthly_summaries(summary_year, summary_month);
|
||||
CREATE INDEX idx_trend_kpi ON trend_analysis(kpi_id);
|
||||
CREATE INDEX idx_trend_date ON trend_analysis(analysis_date);
|
||||
|
||||
-- Reporting indexes
|
||||
CREATE INDEX idx_report_exec_report ON report_executions(report_id);
|
||||
CREATE INDEX idx_report_exec_timestamp ON report_executions(execution_timestamp);
|
||||
CREATE INDEX idx_data_quality_table ON data_quality_checks(table_name);
|
||||
CREATE INDEX idx_data_quality_date ON data_quality_checks(check_date);
|
||||
|
||||
-- ============================================================================
|
||||
-- COMMENTS FOR NEW TABLES
|
||||
-- ============================================================================
|
||||
|
||||
COMMENT ON TABLE customer_segments IS 'Customer segmentation definitions for targeted analysis';
|
||||
COMMENT ON TABLE customer_lifetime_value IS 'Customer lifetime value calculations and tracking';
|
||||
COMMENT ON TABLE churn_predictions IS 'Customer churn risk predictions for retention analysis';
|
||||
COMMENT ON TABLE customer_satisfaction IS 'Customer satisfaction scores and feedback';
|
||||
COMMENT ON TABLE engagement_metrics IS 'Customer engagement tracking metrics';
|
||||
COMMENT ON TABLE product_catalog IS 'Product master data for sales analysis';
|
||||
COMMENT ON TABLE sales_transactions IS 'Detailed sales transaction records';
|
||||
COMMENT ON TABLE sales_targets IS 'Sales performance targets and goals';
|
||||
COMMENT ON TABLE sales_performance IS 'Aggregated sales performance metrics';
|
||||
COMMENT ON TABLE revenue_forecasts IS 'Revenue forecasting and predictions';
|
||||
COMMENT ON TABLE kpi_definitions IS 'Master list of tracked KPIs';
|
||||
COMMENT ON TABLE daily_metrics IS 'Daily operational metrics for dashboards';
|
||||
COMMENT ON TABLE monthly_summaries IS 'Monthly aggregated business summaries';
|
||||
COMMENT ON TABLE trend_analysis IS 'Statistical trend analysis results';
|
||||
COMMENT ON TABLE dashboard_snapshots IS 'Pre-calculated dashboard data snapshots';
|
||||
COMMENT ON TABLE report_definitions IS 'Standard report catalog';
|
||||
COMMENT ON TABLE report_executions IS 'Report execution history and logs';
|
||||
COMMENT ON TABLE data_quality_checks IS 'Data quality validation tracking';
|
||||
|
||||
|
||||
@@ -5,6 +5,27 @@
|
||||
-- ============================================================================
|
||||
|
||||
-- Clear existing reference data (in reverse dependency order)
|
||||
-- New analytics tables
|
||||
TRUNCATE TABLE report_executions CASCADE;
|
||||
TRUNCATE TABLE report_definitions RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE data_quality_checks RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE dashboard_snapshots RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE trend_analysis RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE monthly_summaries RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE daily_metrics RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE kpi_definitions RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE revenue_forecasts RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE sales_performance RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE sales_targets RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE sales_transactions RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE product_catalog RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE engagement_metrics RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE customer_satisfaction RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE churn_predictions RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE customer_lifetime_value RESTART IDENTITY CASCADE;
|
||||
TRUNCATE TABLE customer_segments RESTART IDENTITY CASCADE;
|
||||
|
||||
-- Existing tables
|
||||
TRUNCATE TABLE suspicious_activity_reports CASCADE;
|
||||
TRUNCATE TABLE case_alerts CASCADE;
|
||||
TRUNCATE TABLE case_transactions CASCADE;
|
||||
@@ -254,3 +275,91 @@ COMMENT ON FUNCTION generate_fraud_score IS 'Calculates fraud risk score based o
|
||||
COMMENT ON FUNCTION update_account_balance IS 'Automatically updates account balance after transaction';
|
||||
COMMENT ON FUNCTION check_suspicious_transaction IS 'Generates alerts for suspicious transactions';
|
||||
|
||||
-- ============================================================================
|
||||
-- SEED DATA FOR NEW ANALYTICS TABLES
|
||||
-- ============================================================================
|
||||
|
||||
-- Customer Segments
|
||||
INSERT INTO customer_segments (segment_name, segment_description, min_clv, max_clv) VALUES
|
||||
('VIP', 'Very Important Person - Highest value customers', 50000, NULL),
|
||||
('High Value', 'High spending customers with strong loyalty', 10000, 49999),
|
||||
('Medium Value', 'Regular customers with moderate spending', 2000, 9999),
|
||||
('Low Value', 'Occasional customers with low spending', 500, 1999),
|
||||
('At Risk', 'Previously active customers showing decline', NULL, NULL),
|
||||
('New Customer', 'Recently acquired customers (< 90 days)', NULL, NULL),
|
||||
('Dormant', 'Inactive customers (> 180 days)', NULL, NULL),
|
||||
('Churned', 'Lost customers who have not transacted in 365+ days', NULL, NULL);
|
||||
|
||||
-- Product Catalog (Sample Products)
|
||||
INSERT INTO product_catalog (product_name, product_category, product_subcategory, unit_price, cost_price, margin_percentage, is_active, launch_date) VALUES
|
||||
-- Banking Products
|
||||
('Premium Checking Account', 'Banking', 'Checking', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
('Basic Savings Account', 'Banking', 'Savings', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
('High-Yield Savings', 'Banking', 'Savings', 0.00, 0.00, 0.00, TRUE, '2021-06-01'),
|
||||
('Business Checking', 'Banking', 'Business', 15.00, 5.00, 66.67, TRUE, '2020-01-01'),
|
||||
('Student Checking', 'Banking', 'Checking', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
|
||||
-- Credit Cards
|
||||
('Platinum Credit Card', 'Credit', 'Premium', 99.00, 20.00, 79.80, TRUE, '2020-01-01'),
|
||||
('Gold Credit Card', 'Credit', 'Standard', 49.00, 15.00, 69.39, TRUE, '2020-01-01'),
|
||||
('Cash Back Card', 'Credit', 'Rewards', 0.00, 10.00, -100.00, TRUE, '2021-01-01'),
|
||||
('Travel Rewards Card', 'Credit', 'Rewards', 95.00, 25.00, 73.68, TRUE, '2021-03-01'),
|
||||
('Business Credit Card', 'Credit', 'Business', 75.00, 20.00, 73.33, TRUE, '2020-06-01'),
|
||||
|
||||
-- Loans
|
||||
('Personal Loan', 'Lending', 'Personal', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
('Auto Loan', 'Lending', 'Auto', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
('Home Mortgage', 'Lending', 'Mortgage', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
('Small Business Loan', 'Lending', 'Business', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
|
||||
-- Investment Products
|
||||
('Index Fund', 'Investment', 'Mutual Funds', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
('Bond Fund', 'Investment', 'Mutual Funds', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
('Retirement Account (IRA)', 'Investment', 'Retirement', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
('401k Plan', 'Investment', 'Retirement', 0.00, 0.00, 0.00, TRUE, '2020-01-01'),
|
||||
|
||||
-- Services
|
||||
('Wire Transfer', 'Services', 'Transfers', 25.00, 5.00, 80.00, TRUE, '2020-01-01'),
|
||||
('International Transfer', 'Services', 'Transfers', 45.00, 10.00, 77.78, TRUE, '2020-01-01'),
|
||||
('Overdraft Protection', 'Services', 'Protection', 35.00, 5.00, 85.71, TRUE, '2020-01-01'),
|
||||
('Safe Deposit Box', 'Services', 'Security', 75.00, 20.00, 73.33, TRUE, '2020-01-01'),
|
||||
('Financial Advisory', 'Services', 'Advisory', 150.00, 50.00, 66.67, TRUE, '2021-01-01'),
|
||||
('Mobile Banking Premium', 'Services', 'Digital', 9.99, 2.00, 79.98, TRUE, '2022-01-01');
|
||||
|
||||
-- KPI Definitions
|
||||
INSERT INTO kpi_definitions (kpi_name, kpi_description, kpi_category, calculation_formula, target_value, threshold_warning, threshold_critical, unit_of_measure, refresh_frequency, is_active) VALUES
|
||||
-- Sales KPIs
|
||||
('Daily Revenue', 'Total revenue generated per day', 'SALES', 'SUM(amount) FROM transactions WHERE DATE(transaction_date) = CURRENT_DATE', 500000, 400000, 300000, 'USD', 'DAILY', TRUE),
|
||||
('Monthly Revenue', 'Total revenue for the month', 'SALES', 'SUM(amount) FROM transactions WHERE MONTH(transaction_date) = CURRENT_MONTH', 15000000, 12000000, 10000000, 'USD', 'DAILY', TRUE),
|
||||
('Average Transaction Value', 'Average value per transaction', 'SALES', 'AVG(amount) FROM transactions', 150, 100, 75, 'USD', 'DAILY', TRUE),
|
||||
('Transactions Per Day', 'Number of transactions per day', 'SALES', 'COUNT(*) FROM transactions WHERE DATE(transaction_date) = CURRENT_DATE', 10000, 7500, 5000, 'Count', 'DAILY', TRUE),
|
||||
|
||||
-- Customer KPIs
|
||||
('New Customers', 'New customer acquisitions', 'CUSTOMER', 'COUNT(*) FROM customers WHERE registration_date >= CURRENT_DATE - 30', 1000, 750, 500, 'Count', 'DAILY', TRUE),
|
||||
('Customer Churn Rate', 'Percentage of customers churning', 'CUSTOMER', '(Churned / Total) * 100', 5, 7, 10, 'Percentage', 'WEEKLY', TRUE),
|
||||
('Customer Lifetime Value', 'Average CLV across all customers', 'CUSTOMER', 'AVG(clv_score) FROM customer_lifetime_value', 5000, 4000, 3000, 'USD', 'WEEKLY', TRUE),
|
||||
('Net Promoter Score', 'Customer satisfaction metric', 'CUSTOMER', 'AVG(nps_score) FROM customer_satisfaction', 50, 30, 10, 'Score', 'WEEKLY', TRUE),
|
||||
('Customer Engagement Score', 'Average engagement across customers', 'CUSTOMER', 'AVG(engagement_score) FROM engagement_metrics', 75, 60, 45, 'Score', 'DAILY', TRUE),
|
||||
|
||||
-- Operational KPIs
|
||||
('Transaction Success Rate', 'Percentage of successful transactions', 'OPERATIONAL', '(Successful / Total) * 100', 99, 97, 95, 'Percentage', 'HOURLY', TRUE),
|
||||
('Average Response Time', 'System response time', 'OPERATIONAL', 'AVG(response_time_ms)', 200, 500, 1000, 'Milliseconds', 'REALTIME', TRUE),
|
||||
('Fraud Detection Rate', 'Percentage of fraud caught', 'OPERATIONAL', '(Detected / Total Fraud) * 100', 95, 90, 85, 'Percentage', 'DAILY', TRUE),
|
||||
('Alert Resolution Time', 'Average time to resolve alerts', 'OPERATIONAL', 'AVG(resolution_time_hours)', 24, 48, 72, 'Hours', 'DAILY', TRUE),
|
||||
|
||||
-- Financial KPIs
|
||||
('Gross Margin', 'Overall profit margin', 'FINANCIAL', '((Revenue - Cost) / Revenue) * 100', 75, 65, 55, 'Percentage', 'DAILY', TRUE),
|
||||
('Customer Acquisition Cost', 'Cost to acquire new customer', 'FINANCIAL', 'Marketing Spend / New Customers', 50, 75, 100, 'USD', 'WEEKLY', TRUE),
|
||||
('Return on Investment', 'ROI on marketing campaigns', 'FINANCIAL', '((Revenue - Cost) / Cost) * 100', 300, 200, 100, 'Percentage', 'WEEKLY', TRUE);
|
||||
|
||||
-- Report Definitions
|
||||
INSERT INTO report_definitions (report_name, report_description, report_category, output_format, schedule_frequency, is_active) VALUES
|
||||
('Daily Sales Summary', 'Daily sales performance report', 'Sales', 'PDF', 'Daily at 6 AM', TRUE),
|
||||
('Weekly Customer Analytics', 'Customer behavior and segmentation analysis', 'Customer', 'EXCEL', 'Weekly on Monday', TRUE),
|
||||
('Monthly Financial Summary', 'Comprehensive monthly financial report', 'Financial', 'PDF', 'Monthly on 1st', TRUE),
|
||||
('Fraud Detection Report', 'Daily fraud alerts and cases', 'Risk', 'PDF', 'Daily at 8 AM', TRUE),
|
||||
('KPI Dashboard', 'Executive KPI dashboard', 'Executive', 'HTML', 'Daily at 7 AM', TRUE),
|
||||
('Customer Churn Analysis', 'At-risk customer identification', 'Customer', 'EXCEL', 'Weekly on Friday', TRUE),
|
||||
('Product Performance', 'Product sales and profitability analysis', 'Sales', 'EXCEL', 'Monthly on 5th', TRUE),
|
||||
('Data Quality Report', 'Data validation and quality metrics', 'Operations', 'CSV', 'Daily at 5 AM', TRUE);
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
# ============================================================================
|
||||
# Database Setup Script - IDEMPOTENT
|
||||
# ============================================================================
|
||||
# This script sets up the complete fraud detection database
|
||||
# This script sets up the complete business analytics database
|
||||
# It can be run multiple times safely - it will recreate everything
|
||||
# ============================================================================
|
||||
|
||||
@@ -19,8 +19,8 @@ NC='\033[0m' # No Color
|
||||
# Configuration from environment or defaults
|
||||
DB_HOST="${POSTGRES_HOST:-localhost}"
|
||||
DB_PORT="${POSTGRES_PORT:-5432}"
|
||||
DB_NAME="${POSTGRES_DB:-fraud_detection}"
|
||||
DB_USER="${POSTGRES_USER:-fraud_analyst}"
|
||||
DB_NAME="${POSTGRES_DB:-business_analytics}"
|
||||
DB_USER="${POSTGRES_USER:-data_analyst}"
|
||||
DB_PASSWORD="${POSTGRES_PASSWORD:-SecurePass123!}"
|
||||
|
||||
# Script directory
|
||||
@@ -28,7 +28,7 @@ SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo -e "${BLUE}Financial Fraud Detection Database - Setup Script${NC}"
|
||||
echo -e "${BLUE}Business Analytics Database - Setup Script${NC}"
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo -e "Database: ${GREEN}$DB_NAME${NC}"
|
||||
echo -e "Host: ${GREEN}$DB_HOST:$DB_PORT${NC}"
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
# ============================================================================
|
||||
# Setup Verification Script
|
||||
# ============================================================================
|
||||
# Verifies that the fraud detection database is properly set up
|
||||
# Verifies that the business analytics database is properly set up
|
||||
# ============================================================================
|
||||
|
||||
set -e
|
||||
@@ -18,12 +18,12 @@ NC='\033[0m' # No Color
|
||||
# Configuration
|
||||
DB_HOST="${POSTGRES_HOST:-localhost}"
|
||||
DB_PORT="${POSTGRES_PORT:-5432}"
|
||||
DB_NAME="${POSTGRES_DB:-fraud_detection}"
|
||||
DB_USER="${POSTGRES_USER:-fraud_analyst}"
|
||||
DB_NAME="${POSTGRES_DB:-business_analytics}"
|
||||
DB_USER="${POSTGRES_USER:-data_analyst}"
|
||||
DB_PASSWORD="${POSTGRES_PASSWORD:-SecurePass123!}"
|
||||
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo -e "${BLUE}Financial Fraud Detection Database - Verification${NC}"
|
||||
echo -e "${BLUE}Business Analytics Database - Verification${NC}"
|
||||
echo -e "${BLUE}============================================================================${NC}"
|
||||
echo ""
|
||||
|
||||
@@ -34,7 +34,7 @@ execute_sql() {
|
||||
|
||||
# Check 1: Docker containers
|
||||
echo -e "${YELLOW}[1/10] Checking Docker containers...${NC}"
|
||||
if docker ps | grep -q "fraud_detection_db"; then
|
||||
if docker ps | grep -q "business_analytics_db"; then
|
||||
echo -e "${GREEN}✓ PostgreSQL container is running${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ PostgreSQL container is not running${NC}"
|
||||
@@ -42,7 +42,7 @@ else
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if docker ps | grep -q "fraud_detection_ui"; then
|
||||
if docker ps | grep -q "business_analytics_ui"; then
|
||||
echo -e "${GREEN}✓ DB-UI container is running${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ DB-UI container is not running${NC}"
|
||||
|
||||
Reference in New Issue
Block a user