flowfish/docs/02-feature-list.md

Flowfish - Detailed Feature List

This document describes all Flowfish platform features in detail, organized by phase.

📋 Table of Contents

• Phase 1: MVP (Minimum Viable Product)
  • 1.X. L7 Application Level Service Map
• Phase 2: Advanced Features
• Phase 3: Enterprise Features

---

Phase 1: MVP (Minimum Viable Product)

Target Timeline: 0-3 months
Status: Design Phase
Goal: Complete core platform infrastructure and core features

1.1. Inspektor Gadget Integration and Data Collection

1.1.1. Inspektor Gadget Installation

• ✅ Deployed to Kubernetes/OpenShift cluster as a DaemonSet
• ✅ Automatic loading of eBPF programs on each node
• ✅ Kernel compatibility checks and version management
• ✅ Resource limits and requests definition
• ✅ Health checks and liveness probes

1.1.2. Gadget Modules

Network Traffic Gadget:

• TCP/UDP connection tracking
• Source/destination IP and port information
• Packet count and byte transfer metrics
• Connection duration tracking

DNS Gadget:

• DNS query and response logging
• Query type (A, AAAA, CNAME, etc.)
• Response code tracking
• Latency measurement

TCP Connections Gadget:

• TCP connection lifecycle (SYN, ACK, FIN)
• Connection state tracking
• Retransmission count
• Window size monitoring

Process Events Gadget:

• Process creation and termination
• Process tree (parent-child relationship)
• Binary path and arguments
• User and group information

Syscall Tracking Gadget:

• Network-related syscalls (socket, connect, bind, listen)
• File-related syscalls (open, read, write)
• Syscall frequency and latency
• Error rate tracking

File Access Gadget:

• File open/read/write events
• File path and permissions
• Access patterns
• Security-sensitive file monitoring

1.1.3. Data Collection Control

• ✅ No data collection by default (privacy-first)
• ✅ Active only when the user starts an analysis
• ✅ Scope-based collection (cluster, namespace, workload selection)
• ✅ Start/stop analysis control
• ✅ Resource usage monitoring and throttling

1.2. Kubernetes Resource Discovery and Communication Detection

1.2.1. Workload Discovery

Pod Discovery:

• Pod name, namespace, labels, annotations
• Container list and image information
• Node placement information
• Owner reference (Deployment, StatefulSet, etc.)
• Pod IP and port bindings
• Resource requests and limits

Deployment Discovery:

• Deployment name, namespace, labels
• Replica count (desired vs actual)
• Selector labels
• Strategy (RollingUpdate, Recreate)
• Deployment conditions and status

StatefulSet Discovery:

• StatefulSet name, namespace, labels
• Replica count and pod naming pattern
• Persistent Volume Claims
• Service name association
• Update strategy

Service Discovery:

• Service name, namespace, labels
• Service type (ClusterIP, NodePort, LoadBalancer)
• Cluster IP and external IP
• Port mappings (port, targetPort, protocol)
• Selector labels
• Endpoint addresses

1.2.2. Communication Discovery

Connection Tracking:

• Source workload (pod/deployment/service)
• Destination workload
• Source IP:Port → Destination IP:Port
• Protocol (TCP, UDP, HTTP, HTTPS, gRPC, etc.)
• First seen timestamp
• Last seen timestamp
• Request count and frequency (requests/second)
• Average payload size (bytes)
• Average latency (milliseconds) - optional
• Namespace crossing detection
• Cluster crossing detection (multi-cluster)

Metadata Enrichment:

• Kubernetes labels and annotations
• Service mesh labels (Istio, Linkerd)
• Custom tags
• Application tier (frontend, backend, database)
• Environment (production, staging, development)

1.2.3. Risk & Importance Scoring

Risk Factors:

• Cross-namespace communication (medium risk)
• External egress traffic (high risk)
• Unknown destination IP (high risk)
• Privileged port usage (<1024)
• Unexpected protocol usage
• High request frequency

Risk Levels:

• 🟢 Low (0-30): Normal internal communication
• 🟡 Medium (31-60): Cross-namespace, elevated ports
• 🔴 High (61-100): External, unknown, privileged

Importance Factors:

• Request frequency
• Number of dependent services
• Critical service labels
• Production environment
• SLA tier

1.2.4. Pod & Deployment Annotations

• Full annotation support, including automatic merge of Deployment/StatefulSet annotations into pods
• Annotation filtering for internal Kubernetes prefixes

1.3. Real-Time Dependency Map

1.3.1. Graph Data Model

Vertex (Node) Types:

• Pod node
• Deployment node
• StatefulSet node
• Service node
• Namespace node (grouping)

Edge (Relationship) Types:

• COMMUNICATES_WITH (directed edge)
• PART_OF (pod → deployment/statefulset)
• EXPOSES (service → deployment/statefulset)
• RUNS_ON (pod → node) - optional

Node Properties:

• Name, namespace, type
• Labels, annotations
• Status (running, pending, failed)
• Resource usage (CPU, memory)
• Creation timestamp

Edge Properties:

• Port, protocol
• Request count
• Latency (avg, p50, p95, p99)
• First/last seen
• Data transfer (bytes in/out)
• Error rate
• Risk score

1.3.2. Graph Visualization

Cytoscape.js Integration:

• Node rendering (different shapes: pod=circle, deployment=square, service=diamond)
• Node coloring (namespace-based, status-based, risk-based)
• Edge rendering (thickness by request count, color by protocol)
• Edge animation (live traffic flow)
• Layout algorithms:
  • Hierarchical (tier-based: frontend → backend → database)
  • Force-directed (physics simulation)
  • Circular
  • Grid
  • Manual positioning

Interactivity:

• Node click → Detail panel
• Node hover → Quick info tooltip
• Edge click → Communication details
• Multi-select (Ctrl+Click)
• Zoom and pan
• Minimap navigator
• Search and filtering

Export:

• PNG/JPG image export
• SVG vector export
• JSON export
• PDF export

1.4. Cluster and Namespace Management

1.4.1. Multi-Cluster Support

Cluster Management:

• Add cluster (Kubernetes, OpenShift)
• Connection types:
  • In-cluster (platform runs inside the cluster itself)
  • Kubeconfig file
  • Service Account token
  • API server URL + credentials
• Cluster health monitoring
• Cluster metadata:
  • Name, description
  • Kubernetes version
  • Node count
  • Pod count
  • Namespace count

Cluster Selector:

• Top navigation bar dropdown
• Quick cluster switching
• Default cluster configuration
• Cluster-specific dashboards

1.4.2. Namespace Management

• Namespace list and filtering
• Namespace metadata display
• Namespace-based access control
• Namespace label management
• Cross-namespace communication tracking

1.5. Analysis Wizard

Step 1: Scope Selection

Selectable Levels:

Cluster-level:

• Cover all namespaces
• Cluster-wide visibility
• Use case: Full platform analysis

Namespace-level:

• Select specific namespace(s)
• Multiple namespace selection
• Use case: Application-specific analysis

Workload-level:

• Specific Deployment(s)
• Specific StatefulSet(s)
• Specific Pod(s)
• Use case: Targeted troubleshooting

Label-based (Advanced):

• Label selector (app=frontend, tier=backend)
• Multiple label combinations
• Exclude labels
• Use case: Complex filtering

Hierarchical Selection:

• When a namespace is selected → all workloads under it are included automatically
• When a Deployment is selected → replica pods are included automatically

Step 2: Gadget Module Selection

Module List:

• ☑️ Network Traffic (TCP/UDP connections)
• ☑️ DNS Queries
• ☑️ TCP Connection State
• ☑️ Process Events
• ☑️ Syscall Tracking
• ☑️ File Access

Per Module:

• Enable/Disable toggle
• Configuration options:
  • Sampling rate (1/1, 1/10, 1/100)
  • Filter rules (port ranges, protocols)
  • Data retention period
• Estimated resource usage display
• Recommended presets (Light, Medium, Heavy)

Step 3: Time & Profile Settings

Analysis Modes:

Continuous Analysis:

• Runs continuously after start
• Real-time data collection
• Stopped with Stop button
• Use case: Production monitoring

Time-bound Analysis:

• Start time, end time selection
• Duration: 5 min, 15 min, 1 hour, 4 hours, 24 hours
• Automatic stop
• Use case: Troubleshooting, incident analysis

Scheduled Analysis (Periodic):

• Scheduling via cron expression
• Daily, weekly, monthly presets
• Timezone selection
• Use case: Regular audits, compliance

Baseline Creation Mode (Profile Creation):

• Learn “normal” traffic profile over a defined period
• Minimum duration: 24 hours
• Recommended: 7 days
• Use case: Anomaly detection baseline

Profile Settings:

• Select existing baseline
• Create baseline
• Update baseline
• Compare with baseline

Step 4: Output & Integration

Dashboard Selection:

• ☑️ Main Dashboard
• ☑️ Application Dependency Dashboard
• ☑️ Traffic & Behavior Dashboard
• ☑️ Security & Risk Dashboard
• ☑️ Change Timeline Dashboard

LLM Analysis:

• ☑️ Enable LLM anomaly detection
• LLM provider selection:
  • OpenAI GPT-4
  • Azure OpenAI
  • Anthropic Claude
  • Custom endpoint (OpenAI-compatible)
• API key configuration
• Analysis frequency (15 min, 30 min, 1 hour)
• Prompt customization

Alarm & Webhook:

• ☑️ Enable alerting
• Webhook URL configuration
• Alert conditions:
  • New connections
  • Lost connections
  • Anomaly detected
  • Risk score threshold
  • Traffic spike (% increase)
• Alert format (JSON, Slack, Teams, PagerDuty)

Report Settings:

• ☑️ Auto-generate reports
• Report format (PDF, HTML, JSON)
• Report frequency (daily, weekly, monthly)
• Email recipients
• Storage location (S3, Azure Blob, local)

1.6. Core Dashboards

1.6.1. Main Dashboard (Home/Overview)

Metrics:

• Total application count (pods, deployments, services)
• Total connection count (active vs historical)
• Active anomaly count
• Active change event count
• Most communicating services (top 10)
• Highest risk score applications (top 10)
• Cluster health overview
• Namespace distribution pie chart

Widgets:

• Real-time traffic rate (requests/second)
• Total data transfer (MB/s in/out)
• Protocol distribution (TCP, UDP, HTTP, gRPC)
• Top source namespaces
• Top destination namespaces
• Recent alerts timeline

1.6.2. Live Map Dashboard

Main View:

• Interactive graph visualization
• Real-time updates (5-second refresh)
• Smooth animations
• Node clustering by namespace
• Edge bundling for clarity

Controls:

• Layout selection (dropdown)
• Filter panel (slide-out)
• Search box
• Zoom controls (+/-)
• Fit to screen button
• Lock/unlock node positions
• Show/hide labels
• Show/hide edge labels

Filter Options:

• Namespace multi-select
• Workload type (pod, deployment, service)
• Protocol filter
• Risk level filter
• Request count threshold
• Latency threshold
• Time range

Detail Panel:

• Selected node information
• Incoming connections list
• Outgoing connections list
• Resource usage charts
• Labels and annotations
• Logs link (integration with logging system)
• Metrics link (integration with Prometheus/Grafana)

1.7. User Management and RBAC

1.7.1. User Management

User CRUD:

• Create user (username, email, password)
• Update user profile
• Delete user (soft delete)
• User status (active, inactive, locked)
• Password reset (self-service + admin)
• User groups

User Profile:

• First name, last name
• Email address
• Avatar/profile picture
• Timezone
• Language preference
• Notification preferences

1.7.2. Role-Based Access Control

Predefined Roles:

Super Admin:

• Full system access
• User management
• Cluster management
• System configuration
• Audit log access
• LLM configuration
• Billing & usage monitoring

Platform Admin:

• Cluster management (add/remove)
• Namespace management
• Integration configuration
• Dashboard management
• User role assignment (except Super Admin)
• Audit log access (read-only)

Security Analyst:

• View all dashboards
• Anomaly detection management
• Change detection management
• Risk score configuration
• Security reports
• Alert configuration
• Export data (CSV, JSON)

Developer (Read-Only):

• View dashboards (limited)
• View dependency maps
• View application inventory
• View own namespace only
• No configuration access
• No export capability

Custom Roles:

• Create custom roles
• Permission matrix selection
• Role naming and description
• Role assignment

1.7.3. Permissions

Granular Permissions:

• clusters.view, clusters.create, clusters.edit, clusters.delete
• namespaces.view, namespaces.create, namespaces.edit, namespaces.delete
• analyses.view, analyses.create, analyses.edit, analyses.delete, analyses.execute
• dependencies.view, dependencies.export
• anomalies.view, anomalies.manage
• changes.view, changes.manage
• users.view, users.create, users.edit, users.delete
• roles.view, roles.create, roles.edit, roles.delete
• settings.view, settings.edit
• audit.view

1.8. Authentication

1.8.1. OAuth 2.0 / SSO

Supported Providers:

• Google OAuth
• Microsoft Azure AD / Entra ID
• Okta
• Auth0
• Keycloak
• GitHub
• GitLab

Configuration:

• Client ID and Client Secret
• Authorization endpoint
• Token endpoint
• User info endpoint
• Scope configuration
• Redirect URI

Flow:

1. User clicks "Login with [Provider]"
2. Redirect to provider
3. User authenticates
4. Callback to Flowfish with auth code
5. Exchange code for access token
6. Fetch user info
7. Create/update user in Flowfish
8. Generate JWT token
9. Redirect to dashboard

1.8.2. Kubernetes Service Account

In-Cluster Authentication:

• Service Account token mount
• Automatic user creation from SA
• Namespace-based role mapping
• Label-based permission mapping

External Cluster:

• Kubeconfig import
• Token extraction
• User mapping based on cluster role
• Certificate authentication

1.8.3. JWT Token Management

• Access token (short-lived, 1 hour)
• Refresh token (long-lived, 7 days)
• Token revocation
• Token blacklist (Redis)
• Session management

1.8.4. API Key Management

• API key creation, expiration, and revocation for programmatic access alongside JWT authentication

1.9. Core API Endpoints

Authentication:

• POST /api/v1/auth/login
• POST /api/v1/auth/logout
• POST /api/v1/auth/refresh
• GET /api/v1/auth/me

Clusters:

• GET /api/v1/clusters
• POST /api/v1/clusters
• GET /api/v1/clusters/{id}
• PUT /api/v1/clusters/{id}
• DELETE /api/v1/clusters/{id}

Namespaces:

• GET /api/v1/clusters/{id}/namespaces
• GET /api/v1/clusters/{id}/namespaces/{namespace}

Workloads:

• GET /api/v1/clusters/{id}/workloads
• GET /api/v1/clusters/{id}/pods
• GET /api/v1/clusters/{id}/deployments
• GET /api/v1/clusters/{id}/statefulsets
• GET /api/v1/clusters/{id}/services

Analyses:

• GET /api/v1/analyses
• POST /api/v1/analyses
• GET /api/v1/analyses/{id}
• PUT /api/v1/analyses/{id}
• DELETE /api/v1/analyses/{id}
• POST /api/v1/analyses/{id}/start
• POST /api/v1/analyses/{id}/stop

Communications:

• GET /api/v1/communications
• GET /api/v1/communications/{id}

Dependencies:

• GET /api/v1/dependencies/graph
• GET /api/v1/dependencies/map

---

1.X. L7 Application Level Service Map (Grafana Beyla)

1.X.1. Beyla eBPF Agent Deployment

• ✅ Grafana Beyla deployed as DaemonSet for L7 traffic capture
• ✅ HTTP, gRPC, and DNS protocol interception via eBPF
• ✅ Cluster management integration (install/upgrade/uninstall scripts)
• ✅ Health status monitoring per cluster
• ✅ Cluster scale presets (small/medium/large/xlarge) for resource limits

1.X.2. flowfish-l7-collector

• ✅ In-cluster OTLP HTTP receiver (bridges Beyla push to Flowfish pull)
• ✅ Circular event buffer with cursor-based pagination
• ✅ K8s metadata enrichment (IP → workload resolution)
• ✅ Pull API for Flowfish central platform

1.X.3. L7 Ingestion Pipeline

• ✅ L7 Ingestion Service with K8s API Service Proxy polling
• ✅ Two-layer event transformation (collector → ingestion → RabbitMQ)
• ✅ Per-analysis enrichment (analysis_id, cluster_id)
• ✅ L7 capture filters (namespace, service, HTTP method/status/path)
• ✅ Isolated RabbitMQ exchanges and queues (flowfish.l7.*)

1.X.4. L7 Data Storage

• ✅ ClickHouse tables: l7_http_flows, l7_grpc_flows, l7_dns_flows
• ✅ 5-minute aggregation materialized view for HTTP
• ✅ Neo4j L7Workload nodes and L7_COMMUNICATES_WITH relationships
• ✅ 30-day TTL on ClickHouse L7 tables

1.X.5. L7 Service Map UI

• ✅ Interactive graph visualization with 16 layout algorithms
• ✅ Protocol-aware edge coloring (HTTP/gRPC/DNS)
• ✅ Namespace grouping and filtering
• ✅ Edge click for communication details (latency, error rates, request counts)
• ✅ API gateway chain resolution (A → APISIX → B)
• ✅ Network type classification (Pod-Network, Service-Network, Node-Network, Internal, External) with L4-consistent coloring
• ✅ IP metadata enrichment: Node IP → cluster-infra, Service VIP → service name, unresolved IP → CIDR classification
• ✅ Dependency guard: "Dim System Namespaces" mode (opacity 0.15) instead of hiding — edges always preserved
• ✅ Multi-protocol backend filtering (protocols=http,grpc with backward-compatible protocol parameter)
• ✅ Error breakdown tooltip on stats bar (HTTP status code distribution)
• ✅ Error breakdown panel in workload drawer (4xx/5xx distribution per workload)
• ✅ Configurable edge limit (200/500/1k/2k/5k) with alert banner when truncated
• ✅ Node visual indicators: network type icon, owner kind tag, external badge

1.X.6. Analysis Wizard L7 Support

• ✅ L4/L7/Both analysis level selection
• ✅ L7 protocol cards with volume/impact indicators
• ✅ L7 capture filters configuration
• ✅ Protocol rate estimation and data volume warnings
• ✅ Agent health pre-check (Inspector Gadget + Beyla)

1.X.7. Integration Hub L7 Support

• ✅ Network Level (L3/L4) and Application Level (L7) toggle
• ✅ Unified L4+L7 dependency view — analysis_level=both runs the L4 and L7 endpoints in parallel (Promise.allSettled) and renders Network/Application tabs in the Preview step plus an L4/L7 toggle in the Integration Code step
• ✅ Shared identification surface — both L4 and L7 summary endpoints accept annotation_key/value, label_key/value, owner_name (alias for L7 workload_name), and pod_name; values support fnmatch globs (*, ?, [seq])
• ✅ is_matched flag on L7 summary results so callers can distinguish workloads matched by the filter from their immediate neighbours
• ✅ L7-specific code snippets (cURL, Python, JavaScript, Pipeline)

1.X.7a. L7 HTTP Path Visibility (v2.7.0+, Audit v4)

• ✅ Real HTTP paths in Service Map and Integration Hub — event_transformer._extract_http_path walks the OpenTelemetry HTTP semantic-convention lookup chain (url.path → http.route → url.full → http.target → http.url) so Beyla CLIENT spans (which only emit url.full) no longer collapse to /
• ✅ Relative-path guard against malformed url.full values — falls through to the next attribute when urlsplit returns a bare token, preventing garbage paths from poisoning the per-path edge MERGE key
• ✅ Per-path edge model — Neo4j L7_COMMUNICATES_WITH MERGE key extended to (analysis_id, http_method, http_path); each distinct endpoint between two workloads materialises as its own edge with stable request_count / error_count / latency counters
• ✅ Dedup migration cyphers (_MIGRATE_OUT_CYPHER, _MIGRATE_IN_CYPHER) preserve per-path granularity when re-pointing edges off namespace='unknown' placeholder nodes
• ✅ l7_comm_method and l7_comm_path relationship property indexes (Neo4j 4.3+) keep summary/graph queries fast even with high path cardinality
• ✅ Frontend canvas edge bundling — ServiceMap.tsx groups per-path edges by (source, target, protocol) for rendering, exposes the full paths array in edge data, and shows a N paths suffix on bundled edge labels
• ✅ Distinct-peer aggregation in get_l7_dependency_summary — inbound_count / outbound_count count peers, not edges, so per-path multiplication doesn't inflate dependency counts in the UI
• ✅ Integration Hub Edges Table rowKey extended with (method, path, cluster) to eliminate React rowKey collisions under per-path multiplication

1.X.8. L7 API Endpoints

• GET /api/v1/l7/dependencies/graph
• GET /api/v1/l7/communications
• GET /api/v1/l7/communications/stats
• GET /api/v1/l7/communications/error-stats
• GET /api/v1/l7/dependencies/summary
• GET /api/v1/l7/dependencies/tree-summary
• GET /api/v1/l7/events/http
• GET /api/v1/l7/events/grpc
• GET /api/v1/l7/events/dns
• GET /api/v1/l7/events/stats
• GET /api/v1/l7/events/histogram
• GET /api/v1/l7/events/traces (distributed tracing — recent traces list)
• GET /api/v1/l7/events/traces/{trace_id} (distributed tracing — span detail)
• GET /api/v1/dependencies/unified-summary
• GET /api/v1/settings/beyla
• PUT /api/v1/settings/beyla

1.X.9. L7 Distributed Tracing (W3C OpenTelemetry)

• ✅ Beyla passive mode (ebpf.track_request_headers: true) reads existing traceparent headers without injecting (no extra kernel privileges, no request mutation)
• ✅ Trace context (trace_id, span_id, parent_span_id, span_name, span_kind) persisted in ClickHouse l7_http_flows, l7_grpc_flows, l7_dns_flows with bloom-filter index for fast trace_id lookup
• ✅ Trace properties (last_trace_id, last_span_id, trace_count) stamped on Neo4j L7_COMMUNICATES_WITH relationships
• ✅ Cross-cluster SAME_WORKLOAD periodic matcher with 3 confidence layers: trace_id (high), exact name (medium), hostname (medium) — bridges external placeholder nodes to local cluster nodes
• ✅ Service Map: edge-click opens trace drawer for traced edges; cluster badges visible on nodes
• ✅ Trace Explorer page (/discovery/trace-explorer) with workload and trace_id search, recent traces table, waterfall detail view
• ✅ TraceWaterfall component with parent/child indentation, span_kind, protocol coloring, error highlighting, and sampling/filter warnings
• ✅ Feature-flagged: L7_TRACING_ENABLED env var on writers (default off in production manifests, on in local-test)
• ✅ Graceful schema fallback: writer falls back to legacy INSERT when trace columns are not yet migrated, preventing deploy-order races

---

Phase 2: Advanced Features

Target Timeline: 4-6 months
Status: Planning Phase
Goal: Advanced analysis and integration features

2.1. Historical Dependency Map and Timeline

2.1.1. Historical Map

Time Travel Capability:

• Time slider control (drag to select time)
• Date/time picker (specific timestamp)
• Time range selection (from - to)
• Playback mode (animation)
• Speed control (1x, 2x, 5x, 10x)

Historical Data Storage:

• ClickHouse time-series data
• Graph snapshots (daily, weekly, monthly)
• Snapshot comparison
• Diff visualization (added/removed/changed)

Use Cases:

• Incident investigation: "What was the network topology at 03:00 AM?"
• Change impact analysis: "What changed after deployment?"
• Capacity planning: "How did traffic grow over last 3 months?"
• Compliance audit: "Show all communications in Q1 2024"

2.1.2. Timeline Dashboard

Visualization:

• Horizontal timeline with events
• Event types:
  • New connection detected
  • Connection lost
  • Traffic spike
  • Anomaly detected
  • Deployment event
  • Configuration change
• Event markers with tooltips
• Event filtering by type
• Event search

Integration:

• Kubernetes events (deployments, rollouts, failures)
• Git commits (CI/CD integration)
• Change tickets (Jira, ServiceNow)
• Incident tickets
• Maintenance windows

2.2. Change Detection

2.2.1. Change Types

Connection Changes:

• ➕ New connections (added)
• ➖ Lost connections (removed)
• 🔄 Modified connections (port/protocol change)
• 📈 Traffic increase (>50% increase)
• 📉 Traffic decrease (>50% decrease)

Workload Changes:

• New pods deployed
• Pods terminated
• Deployment scaled up/down
• Service configuration changed
• Labels/annotations modified

Network Changes:

• New external endpoints
• DNS resolution changes
• Certificate changes
• Network policy updates

2.2.2. Change Detection Rules

Rule Configuration:

• Threshold settings (traffic increase %, connection count)
• Time window (compare last 1 hour vs previous 1 hour)
• Baseline comparison (current vs baseline)
• Whitelist (ignore expected changes)
• Blacklist (always alert on specific changes)

Notification:

• Real-time alerts
• Daily digest email
• Slack/Teams notification
• PagerDuty integration
• Webhook POST

2.2.3. Change Dashboard

Metrics:

• Total changes today/week/month
• Changes by type (pie chart)
• Changes by namespace (bar chart)
• Change trend (line chart)
• Top changing workloads

Change List:

• Sortable table
• Filter by type, namespace, time
• Drill-down to details
• Mark as reviewed
• Add comments

2.3. Anomaly Detection (LLM Integration)

2.3.1. LLM Configuration

Supported LLM Providers:

• OpenAI (GPT-4, GPT-4-turbo)
• Azure OpenAI Service
• Anthropic Claude (3.5 Sonnet, Opus)
• AWS Bedrock
• Google Vertex AI
• Self-hosted (OpenAI-compatible API)

Configuration Parameters:

• API endpoint URL
• API key / credentials
• Model selection
• Temperature (0.0 - 1.0)
• Max tokens
• Request timeout
• Rate limiting
• Cost tracking

2.3.2. Anomaly Detection Pipeline

Data Preparation:

1. Baseline data export (JSON format)
2. Recent data export (last 15 min / 1 hour)
3. Change diff calculation
4. Context enrichment (labels, metadata, previous incidents)

LLM Prompt Structure:

System: You are a Kubernetes network security analyst...

Baseline Network Communication (Last 7 days):
[JSON data]

Recent Network Communication (Last 15 minutes):
[JSON data]

Changes Detected:
[Diff data]

Task: Analyze the recent communication pattern and identify any anomalies, 
security concerns, or unexpected behaviors. For each anomaly, provide:
1. Anomaly score (0-100)
2. Severity (low, medium, high, critical)
3. Description
4. Affected workloads
5. Recommended action
6. Confidence level (%)

LLM Response Parsing:

• JSON response extraction
• Anomaly score normalization
• Severity mapping
• Action item extraction
• Confidence threshold filtering (>70%)

2.3.3. Anomaly Types

Network Anomalies:

• Unknown destination IP (external)
• Unusual port usage
• Protocol violation
• Unexpected connection direction (ingress from untrusted)
• DNS exfiltration patterns
• Beaconing behavior (periodic external connections)

Behavioral Anomalies:

• Traffic spike (sudden increase)
• Traffic drop (unexpected decrease)
• Latency anomaly (sudden slowdown)
• Error rate increase
• Retry storm
• Connection churn (frequent connect/disconnect)

Security Anomalies:

• Privilege escalation attempt
• Lateral movement
• Data exfiltration (high outbound traffic)
• Cryptomining pattern (CPU + network)
• Unauthorized service access
• Compliance violation

2.3.4. Anomaly Dashboard

Overview:

• Active anomalies count
• Anomalies by severity (pie chart)
• Anomaly trend (last 24 hours)
• Top affected namespaces
• Top affected workloads

Anomaly List:

• Sortable table (severity, score, time)
• Filter by severity, type, namespace
• Status: new, investigating, resolved, false-positive
• Assign to user
• Add comments and notes
• Link to incident ticket

Anomaly Details:

• Anomaly score and severity
• AI-generated description
• Affected workloads (graph visualization)
• Related connections
• Timeline (when detected, how long)
• LLM reasoning (show prompt + response)
• Recommended actions
• Playbook suggestions
• Similar past anomalies

2.4. Import / Export

2.4.1. Export Formats

Format 1: CSV (Human-Readable)

source_cluster,source_namespace,source_workload,source_type,destination_cluster,destination_namespace,destination_workload,destination_type,port,protocol,request_count,avg_latency_ms,first_seen,last_seen,risk_score
cluster-prod,frontend,web-app,deployment,cluster-prod,backend,api-service,service,8080,HTTP,125430,45.3,2024-01-15T10:00:00Z,2024-01-15T18:00:00Z,25

Fields:

• Source: cluster, namespace, workload name, workload type
• Destination: cluster, namespace, workload name, workload type
• Communication: port, protocol
• Metrics: request_count, avg_latency_ms, bytes_transferred
• Timestamps: first_seen, last_seen
• Analysis: risk_score, importance_score

Use Cases:

• Excel analysis
• Data science / ML
• Reporting
• Compliance audit

Format 2: Graph JSON (System Format)

{
  "version": "1.0",
  "exported_at": "2024-01-15T18:30:00Z",
  "cluster_id": "cluster-prod",
  "snapshot_id": "snapshot-20240115-183000",
  "nodes": [
    {
      "id": "node-1",
      "type": "deployment",
      "name": "web-app",
      "namespace": "frontend",
      "labels": {"app": "web", "tier": "frontend"},
      "metadata": {...}
    },
    ...
  ],
  "edges": [
    {
      "id": "edge-1",
      "source": "node-1",
      "target": "node-2",
      "type": "COMMUNICATES_WITH",
      "port": 8080,
      "protocol": "HTTP",
      "metrics": {...}
    },
    ...
  ],
  "metadata": {
    "total_nodes": 150,
    "total_edges": 420,
    "time_range": {...}
  }
}

Properties:

• Neo4j-compatible format
• Full node and edge details
• Metadata and metrics
• Versioned format
• Re-importable

2.4.2. Export Operations

Manual Export:

• Export button in UI
• Format selection (CSV / Graph JSON)
• Scope selection:
  • Current view
  • Specific namespace
  • Entire cluster
  • Time range
• Download immediately

Scheduled Export:

• Cron schedule configuration
• Format selection
• Destination:
  • Local file system
  • S3 bucket
  • Azure Blob Storage
  • Google Cloud Storage
  • FTP/SFTP server
• Retention policy (keep last N files)
• Notification on completion

API Export:

• GET /api/v1/export/csv?cluster_id=X&namespace=Y&time_range=...
• GET /api/v1/export/graph?cluster_id=X&format=json
• Streaming response for large datasets
• Compression support (gzip)
• Authentication via API key

2.4.3. Import Operations

Single File Import:

• Upload CSV or Graph JSON
• Validation:
  • Schema validation
  • Data type checking
  • Duplicate detection
  • Missing reference checking
• Preview before import
• Import mode selection:
  • Merge (add to existing)
  • Overwrite (replace existing)
  • Create new snapshot
• Progress indicator
• Error handling and rollback

Batch Import:

• Upload multiple files
• Zip archive support
• Parallel processing
• Batch validation
• Batch import report (success/failure per file)

Import Options:

• Timestamp handling:
  • Use original timestamps
  • Use import timestamp
  • Offset timestamps
• Conflict resolution:
  • Keep existing
  • Overwrite with imported
  • Create duplicate with suffix
  • Prompt user
• Namespace mapping (rename on import)
• Label filtering (import only matching labels)

2.4.4. Snapshot Versioning

Snapshot Management:

• Automatic snapshot creation:
  • Daily snapshots (keep 30 days)
  • Weekly snapshots (keep 12 weeks)
  • Monthly snapshots (keep 12 months)
• Manual snapshot creation
• Snapshot naming and tagging
• Snapshot metadata:
  • Creation time
  • Cluster state
  • Node/edge counts
  • Storage size
• Snapshot comparison tool
• Snapshot restore

Version Control:

• Import creates new version
• Version numbering (v1.0, v1.1, v2.0)
• Version tags (production, staging, backup)
• Version diff tool
• Version rollback

2.5. Risk Scores

2.5.1. Risk Scoring Model

Risk Factors (Weighted):

• External communication (weight: 40)
  • Internet egress: +40
  • Unknown IP: +30
  • Public cloud IPs: +20
• Cross-namespace communication (weight: 20)
  • Different namespace: +20
  • System namespace: +30
• Port usage (weight: 15)
  • Privileged ports (<1024): +15
  • Non-standard ports: +10
• Protocol (weight: 10)
  • Unencrypted (HTTP, FTP): +10
  • Unknown protocol: +15
• Request patterns (weight: 10)
  • Very high frequency: +10
  • Irregular patterns: +15
• Security context (weight: 5)
  • Privileged pod: +5
  • Root user: +5

Calculation:

Risk Score = Σ (Factor Value × Factor Weight) / Total Weight × 100
Range: 0-100

Risk Levels:

• 🟢 Low (0-30): Normal, expected traffic
• 🟡 Medium (31-60): Review recommended
• 🟠 High (61-80): Action required
• 🔴 Critical (81-100): Immediate attention

2.5.2. Importance Scoring

Importance Factors:

• Request volume (requests/hour)
• Number of dependents (how many services depend on this)
• Criticality labels (critical=true, tier=1)
• SLA tier (platinum, gold, silver)
• Production environment
• Customer-facing service

Importance Levels:

• ⬇️ Low: <5 dependents, non-prod
• ➡️ Medium: 5-20 dependents, staging
• ⬆️ High: 20-50 dependents, prod
• 🔥 Critical: >50 dependents, customer-facing

2.5.3. Risk Dashboard

Metrics:

• Total services by risk level (stacked bar)
• High-risk services list
• Risk trend over time
• Risk by namespace
• Risk by protocol
• Risk by destination type

Risk Mitigation:

• Recommended network policies
• Port restrictions
• TLS enforcement recommendations
• Firewall rules

2.6. Baseline Creation

2.6.1. Baseline Creation

Learning Period:

• Minimum: 24 hours
• Recommended: 7 days
• Maximum: 30 days

Data Collection:

• All communications
• Request patterns (frequency, volume)
• Latency patterns
• Error rates
• Active hours (time-of-day patterns)
• Day-of-week patterns

Baseline Profiles:

• Per namespace
• Per workload
• Per environment (prod, staging, dev)

2.6.2. Baseline Storage

Profile Data:

{
  "baseline_id": "baseline-2024-01-15",
  "created_at": "2024-01-15T00:00:00Z",
  "learning_period_days": 7,
  "namespace": "backend",
  "workload": "api-service",
  "communication_patterns": [
    {
      "destination": "database-service",
      "port": 5432,
      "protocol": "PostgreSQL",
      "avg_requests_per_hour": 1250,
      "std_dev": 180,
      "avg_latency_ms": 12.5,
      "active_hours": [8, 9, 10, ..., 22]
    }
  ],
  "allowed_destinations": [...],
  "typical_protocols": [...],
  "typical_ports": [...]
}

2.6.3. Baseline Comparison

Deviation Detection:

• Request volume deviation (>2 standard deviations)
• New destinations (not in baseline)
• New protocols (not in baseline)
• New ports (not in baseline)
• Latency spike (>3x baseline average)
• Off-hours activity (outside baseline active hours)

Comparison Dashboard:

• Current vs baseline (side-by-side)
• Deviation score per workload
• Top deviations list
• Deviation timeline
• Drill-down to specific deviations

2.7. Advanced Filtering and Layered View

2.7.1. Advanced Filters

Multi-dimensional Filtering:

• Namespace (multi-select, regex)
• Workload type (pod, deployment, service, statefulset)
• Labels (key-value pairs, AND/OR logic)
• Annotations (key-value pairs)
• Node (which k8s node)
• Risk level (low, medium, high, critical)
• Protocol (TCP, UDP, HTTP, HTTPS, gRPC, etc.)
• Port range (1-1024, 8000-9000)
• Request count range (>1000, <100)
• Latency range (>100ms, <10ms)
• Time range (custom date/time picker)

Filter Presets:

• Save current filters as preset
• Load saved preset
• Share preset with team
• System presets:
  • High-risk communications
  • External traffic only
  • Cross-namespace only
  • Production only
  • Slow connections (high latency)

2.7.2. Logical View (Tier-based)

Application Tiers:

• Frontend tier (web, mobile-api)
• Backend tier (api, microservices)
• Data tier (database, cache)
• Integration tier (message queue, event bus)
• External tier (3rd party APIs, CDN)

Auto-classification:

• Based on labels (tier=frontend)
• Based on naming (web-, api-, db-*)
• Based on connections (talks to DB → backend)
• Manual override

Visualization:

• Hierarchical layout (top-to-bottom: frontend → backend → data)
• Tier grouping (collapse/expand)
• Inter-tier connections emphasized
• Intra-tier connections dimmed

2.7.3. Physical View

Node-based Layout:

• Group pods by Kubernetes node
• Show node capacity (CPU, memory)
• Show pod distribution
• Highlight cross-node traffic
• Show node-to-node bandwidth

Namespace-based Layout:

• Group by namespace (boundaries)
• Show namespace network policies
• Highlight cross-namespace traffic
• Show namespace quotas

Zone/Region Layout (Multi-cluster):

• Group by availability zone
• Group by region
• Show cross-zone/region traffic
• Show latency by distance

2.8. Multi-Cluster / Multi-Domain Support

2.8.1. Domain Management

Domain Definition:

• Domain name (e.g., "Production US-East", "Staging EU")
• Cluster list (multiple clusters per domain)
• Description and metadata
• Domain tags

Domain Types:

• Single-cluster domain
• Multi-cluster domain (same region)
• Multi-region domain
• Hybrid domain (on-prem + cloud)

2.8.2. Cross-Domain Communication

Discovery:

• Cross-cluster service calls (via Istio, Linkerd)
• External endpoints shared between clusters
• DNS-based routing between clusters
• Ingress/egress gateway tracking

Visualization:

• Domain boundaries on graph
• Cross-domain edges (thicker, different color)
• Domain selector filter
• Multi-domain view (all domains on one graph)

2.8.3. Unified Dashboard

Views:

• Isolated view (single cluster)
• Domain view (all clusters in domain)
• Global view (all domains)

Aggregation:

• Total metrics across clusters
• Per-cluster breakdown
• Cross-cluster traffic summary
• Domain-level risk scores

2.9. Webhook and SIEM Integration

2.9.1. Webhook Configuration

Webhook Events:

• Anomaly detected
• High-risk connection detected
• Change detected (new/lost connection)
• Analysis started/completed
• Import completed
• Threshold breach (custom metrics)

Webhook Payload:

{
  "event_type": "anomaly_detected",
  "event_id": "evt-12345",
  "timestamp": "2024-01-15T18:30:00Z",
  "cluster_id": "cluster-prod",
  "namespace": "backend",
  "severity": "high",
  "anomaly": {
    "score": 85,
    "description": "Unusual external connection detected",
    "affected_workload": "api-service",
    "destination_ip": "203.0.113.45",
    "recommended_action": "Block IP and investigate"
  },
  "metadata": {...}
}

Webhook Delivery:

• HTTP POST request
• Retry logic (exponential backoff)
• Timeout configuration
• Signature verification (HMAC)
• Custom headers support

2.9.2. SIEM Integration

Supported SIEM Platforms:

• Splunk (HTTP Event Collector)
• Elasticsearch (Logstash compatible)
• Azure Sentinel (Log Analytics API)
• IBM QRadar (syslog)
• ArcSight (CEF format)
• Generic syslog

Log Forwarding:

• Real-time event streaming
• Batch log export
• CEF (Common Event Format)
• JSON format
• Syslog format

Event Types:

• Network communication events
• Anomaly events
• Change events
• Security events
• Audit events

2.10. Integration Hub

• Four-step guided wizard (Integration Type → Configure → Preview → Integration Code) with two integration paths: Dependency Analysis and Blast Radius Gate
• Configure step: analysis selection, service identification method (Annotation, Label, Namespace + Deployment / Workload, Pod Name, Advanced), search depth, live test query. Namespace lives as an orthogonal field that applies to every method (including L7).
• Preview step: per-service dependency breakdown with expandable matched services, downstream/callers by category, hop distance visualization for multi-depth queries, and aggregate summary. For analysis_level=both selections, the Preview renders two tabs — Network Dependencies (L4) (by_category grouping) and Application Dependencies (L7) (by_protocol grouping). v2.6.0+
• Integration Code step: tabbed snippet generation (Pipeline YAML, cURL, Python, JavaScript, Blast Radius). BOTH-mode adds a top-level L4/L7 radio toggle that swaps the active snippet set between Network and Application without duplicating tabs. v2.6.0+
• BOTH-mode dual query: Test Query fans out to GET /communications/dependencies/summary (L4), GET /l7/dependencies/summary (L7), and GET /l7/dependencies/tree-summary (L7) in parallel via Promise.allSettled. L4 covers every selected analysis, L7 queries the first analysis with a toast warning when additional analyses are dropped. Partial failure (e.g. L7 endpoint down) still enables the Preview — the failing tab carries an inline error banner. v2.6.0+
• Dedicated pipeline platforms with platform-specific templates: Azure DevOps, GitHub Actions, GitLab CI, Jenkins, and Other (Generic)
• Blast Radius tab: pre-deployment risk assessment snippets with cross-link to Blast Radius Oracle page
• Theme-aware UI with Ant Design token-based colors for light/dark mode consistency

---

Phase 3: Enterprise Features

Target Timeline: 7-9 months
Status: Conceptual Phase
Goal: Enterprise-ready features and advanced AI/ML capabilities

3.1. Policy Simulation (What-If Analysis)

3.1.1. Network Policy Simulation

Simulation Wizard:

• Current policy view
• Proposed policy editor (YAML)
• Validation (syntax check)
• Impact analysis:
  • Which connections will be blocked
  • Which workloads will be affected
  • Risk assessment (breaking changes)
• Side-by-side comparison

Test Scenarios:

• Block all external traffic
• Block cross-namespace traffic
• Allow-list specific ports
• Deny-list specific IPs
• Require mTLS for specific services

Simulation Results:

• Blocked connections count
• Affected workloads list
• Estimated impact (low, medium, high)
• Recommended adjustments
• Rollback plan

3.1.2. Deployment Impact Simulation

What-If Questions:

• "What if I delete this deployment?"
• "What if I scale down this service?"
• "What if I update this service's port?"
• "What if I move this pod to another namespace?"

Impact Analysis:

• Dependent services list
• Communication breakdown
• Cascading failures prediction
• Alternative routing suggestions

3.2. Change Simulation (CAP Integration)

3.2.1. Change Impact Assessment

Enterprise process integration for change simulation

Supported Change Types:

• Application Changes:

  • Container image update (version change)
  • Environment variable changes
  • Resource limits update (CPU, memory)
  • Replica count change (scaling)
  • ConfigMap/Secret updates

• Configuration Changes:

  • Service port changes
  • Ingress rule updates
  • Volume mount changes
  • Deployment strategy changes

• Infrastructure Changes:

  • Node maintenance (drain/cordon)
  • Namespace migration
  • Storage class changes
  • Network policy updates

Impact Analysis Dimensions:

1. Dependency Impact:

• Upstream Dependencies:

  • Services that call this service
  • Expected impact on each upstream service
  • Traffic volume that will be affected
  • Alternative routing availability

• Downstream Dependencies:

  • Services called by this service
  • Compatibility check with downstream versions
  • API version compatibility
  • Breaking changes detection

2. Traffic Impact:

• Current traffic patterns (requests/sec)
• Expected traffic interruption duration
• Peak traffic hours conflict check
• Capacity reduction during change
• Estimated downtime window

3. Data Impact:

• StatefulSet data migration needs
• PVC mount point changes
• Database schema compatibility
• Data backup requirements
• Data consistency checks

4. Risk Assessment:

• Risk Score (0-100):

  • Change complexity: 30 points
  • Number of affected services: 25 points
  • Production environment: 20 points
  • Peak hour deployment: 15 points
  • Rollback difficulty: 10 points

• Risk Categories:

  • 🟢 Low Risk (0-30): Minor config changes, non-prod
  • 🟡 Medium Risk (31-60): Version updates, moderate impact
  • 🟠 High Risk (61-80): Major version jumps, many dependents
  • 🔴 Critical Risk (81-100): Breaking changes, production-critical services

5. Blast Radius:

• Direct impact: Immediate affected services
• Indirect impact: 2nd and 3rd degree dependencies
• Total affected pods count
• Total affected namespaces
• Cross-cluster impact (if multi-cluster)
• Estimated user impact (if available)

3.2.2. Change Advisory Process (CAP) Workflow

CAP Integration:

Change Request Creation:

Change Request:
  ID: CR-2024-001234
  Title: "Update payment-service to v2.5.0"
  Type: Application Update
  Priority: Normal / High / Emergency
  Requester: john.doe@company.com
  Planned Window: 2024-01-20 02:00-04:00 UTC
  
  Current State:
    - Service: payment-service
    - Version: v2.3.1
    - Replicas: 5
    - Dependencies: 12 upstream, 8 downstream
  
  Proposed Change:
    - New Version: v2.5.0
    - Configuration: [env var updates]
    - Resource Changes: [CPU +20%]
  
  Impact Assessment:
    - Risk Score: 65 (High)
    - Affected Services: 12
    - Estimated Downtime: 2 minutes
    - Traffic Impact: 15,000 req/min
    - Rollback Time: < 5 minutes

Approval Workflow:

1. Change Request Submitted
   ↓
2. Automated Impact Analysis (Flowfish)
   - Dependency graph analysis
   - Risk scoring
   - Blast radius calculation
   ↓
3. Pre-Approval Checks
   - No conflicting changes in same window
   - Maintenance window validation
   - Resource availability check
   ↓
4. Stakeholder Notification
   - Notify owners of affected services
   - Auto-generated impact report
   - Request for comments
   ↓
5. Approval Process
   - Technical Lead: Review impact
   - Security Team: Security implications (if high risk)
   - Change Manager: Final approval
   ↓
6. Scheduled Execution
   - Pre-change validation
   - Execute change
   - Post-change validation
   ↓
7. Post-Implementation Review
   - Success/failure status
   - Actual vs predicted impact
   - Lessons learned

Approval Matrix:

Risk Level	Approver 1	Approver 2	Approver 3	Emergency Override
Low	Tech Lead	-	-	Tech Lead
Medium	Tech Lead	Team Manager	-	Tech Lead + Manager
High	Tech Lead	Security Lead	Change Manager	C-Level
Critical	Tech Lead	Security Lead	Change Manager + C-Level	Emergency Board

3.2.3. Change Simulation Dashboard

Pre-Change Analysis View:

┌─────────────────────────────────────────────────────────────┐
│ Change Request: CR-2024-001234                              │
│ Status: Pending Approval                                    │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│ ┌─────────────────────────────────────────────────────┐    │
│ │ Change Summary                                       │    │
│ │                                                      │    │
│ │ Service: payment-service                            │    │
│ │ Type: Image Update                                   │    │
│ │ From: v2.3.1 → To: v2.5.0                          │    │
│ │ Scheduled: 2024-01-20 02:00 UTC                     │    │
│ │ Risk Score: 65 (High Risk) 🟠                       │    │
│ └─────────────────────────────────────────────────────┘    │
│                                                             │
│ ┌─────────────────────────────────────────────────────┐    │
│ │ Impact Analysis                                      │    │
│ │                                                      │    │
│ │ Directly Affected: 12 services                      │    │
│ │ Indirectly Affected: 23 services                    │    │
│ │ Total Pods Affected: 67                             │    │
│ │ Estimated Downtime: 2-3 minutes                     │    │
│ │ Traffic Impact: 15,000 req/min                      │    │
│ │                                                      │    │
│ │ [View Dependency Graph] [View Traffic Impact]      │    │
│ └─────────────────────────────────────────────────────┘    │
│                                                             │
│ ┌─────────────────────────────────────────────────────┐    │
│ │ Affected Services (Top 5)                           │    │
│ │                                                      │    │
│ │ 1. 🔴 checkout-service (Critical Dependency)        │    │
│ │    Impact: API breaking change detected            │    │
│ │    Mitigation: Update checkout-service first       │    │
│ │                                                      │    │
│ │ 2. 🟡 order-service (Medium Impact)                 │    │
│ │    Impact: Increased latency expected              │    │
│ │    Mitigation: Monitor performance                 │    │
│ │                                                      │    │
│ │ 3. 🟢 notification-service (Low Impact)             │    │
│ │    Impact: Minimal, async communication            │    │
│ │                                                      │    │
│ │ ... [View All 12 Affected Services]                 │    │
│ └─────────────────────────────────────────────────────┘    │
│                                                             │
│ ┌─────────────────────────────────────────────────────┐    │
│ │ Risk Factors                                         │    │
│ │                                                      │    │
│ │ ✓ Breaking API changes detected                     │    │
│ │ ✓ Production environment                            │    │
│ │ ✓ High traffic service (>10K req/min)              │    │
│ │ ⚠ Scheduled during business hours                   │    │
│ │ ✓ Multiple dependencies affected                    │    │
│ │ ✓ Rollback plan validated                           │    │
│ └─────────────────────────────────────────────────────┘    │
│                                                             │
│ ┌─────────────────────────────────────────────────────┐    │
│ │ Recommendations                                      │    │
│ │                                                      │    │
│ │ 🔸 Consider deploying to staging first              │    │
│ │ 🔸 Update checkout-service API compatibility        │    │
│ │ 🔸 Schedule during low-traffic window (02:00 UTC)   │    │
│ │ 🔸 Prepare rollback script (automated)              │    │
│ │ 🔸 Enable canary deployment (10% traffic)           │    │
│ │ 🔸 Monitor error rates closely                      │    │
│ └─────────────────────────────────────────────────────┘    │
│                                                             │
│ ┌─────────────────────────────────────────────────────┐    │
│ │ Approval Status                                      │    │
│ │                                                      │    │
│ │ ✅ Tech Lead: Approved (John Smith)                 │    │
│ │ ⏳ Security Lead: Pending Review                    │    │
│ │ ⏳ Change Manager: Pending Approval                 │    │
│ │                                                      │    │
│ │ Required: 3/3 approvals                             │    │
│ └─────────────────────────────────────────────────────┘    │
│                                                             │
│ [ Simulate Again ] [ Request Approval ] [ Edit Change ]    │
└─────────────────────────────────────────────────────────────┘

3.2.4. Integration with CAP Tools

ServiceNow Integration:

• Automatic Change Request creation
• Impact assessment attachment
• Approval workflow synchronization
• Status updates (approved/rejected/completed)
• CMDB data enrichment

Jira Integration:

• Change ticket creation
• Custom fields mapping
• Approval workflow via Jira
• Comment synchronization
• Change history tracking

PagerDuty Integration:

• Change event creation
• On-call engineer notification
• Incident correlation
• Change rollback alerts

Slack/Teams Integration:

• Change approval notifications
• Interactive approval buttons
• Impact summary in channels
• Stakeholder mentions
• Post-change status updates

3.2.5. Change Validation Checks

Pre-Change Validation:

• ✅ Image exists in registry
• ✅ Configuration syntax valid
• ✅ Resource limits within quota
• ✅ No conflicting changes scheduled
• ✅ Rollback plan prepared
• ✅ Monitoring alerts configured

Post-Change Validation:

• ✅ All pods running
• ✅ Health checks passing
• ✅ Error rate within threshold (<1%)
• ✅ Latency within SLA (p95 < 500ms)
• ✅ Dependent services responding
• ✅ No increase in 5xx errors

Automated Rollback Triggers:

• Error rate > 5%
• Pod crash loop
• Health check failures > 3
• Latency spike > 200% baseline
• Manual rollback request
• Timeout (change not completing in 30 min)

3.2.6. Change History & Analytics

Change Tracking:

• All changes logged
• Before/after state snapshots
• Actual vs predicted impact comparison
• Success/failure rate
• Average approval time
• Rollback frequency

Analytics Dashboard:

• Changes per week/month
• Success rate by service
• Risk distribution
• Most frequently changed services
• Change velocity trends
• Downtime caused by changes
• MTTR (Mean Time To Recover)

Compliance Reports:

• All changes audit trail
• Approval evidence
• Change window compliance
• Emergency change justifications
• SOX/SOC2 compliance reports

3.2. Universal Data Ingestion & Enrichment

3.2.1. Multi-Source Data Ingestion

Extended Data Sources:

Beyond eBPF, Flowfish collects data from many sources for full visibility:

Infrastructure Data Sources:

• ✅ eBPF (Inspektor Gadget): Network, process, syscall events (primary)
• ✅ Kubernetes Events: Pod crashes, deployments, scaling events
• ✅ Kubernetes Metrics: Resource usage (CPU, memory, disk, network)
• ✅ Node Metrics: Node health, capacity, conditions
• ✅ CNI Plugins: Calico, Cilium, Weave network policies and flows

Application Data Sources:

• ✅ Prometheus Metrics: Application-level metrics (RED, USE, golden signals)
• ✅ Service Mesh: Istio, Linkerd telemetry (L7 metrics, mTLS status)
• ✅ APM Traces: Jaeger, Zipkin distributed traces
• ✅ Application Logs: Structured logs (JSON) for error correlation
• ✅ Custom Metrics: StatsD, OpenMetrics endpoint scraping

External Data Sources:

• ✅ Cloud Provider APIs: AWS, Azure, GCP (load balancers, managed services)
• ✅ CI/CD Systems: GitLab, Jenkins, ArgoCD (deployment events)
• ✅ Incident Management: PagerDuty, Opsgenie (incident correlation)
• ✅ Configuration Management: Git commits, Terraform changes

3.2.2. Dependency Graph with Provenance

Provenance Tracking:

Full provenance for each dependency edge:

{
  "edge": {
    "source": "payment-service",
    "target": "database-postgres",
    "type": "COMMUNICATES_WITH"
  },
  "provenance": {
    "discovered_by": "inspektor-gadget",
    "detection_method": "eBPF TCP tracking",
    "first_seen": "2024-01-15T10:30:00Z",
    "confidence_score": 0.99,
    "data_sources": [
      {
        "source": "eBPF network tracer",
        "timestamp": "2024-01-15T10:30:00Z",
        "sample_count": 15420
      },
      {
        "source": "Istio telemetry",
        "timestamp": "2024-01-15T10:32:00Z",
        "tls_verified": true
      }
    ],
    "validated_by": "john.doe@company.com",
    "validated_at": "2024-01-16T09:00:00Z"
  }
}

Provenance Features:

• Multi-Source Verification: Validate the same dependency from different sources
• Confidence Scoring: Confidence score per dependency (0.0-1.0)
• Historical Tracking: First detection, changes, validations
• Manual Validation: Users can approve/reject dependencies

3.3. Disaster Recovery Posture Assessment

3.3.1. Stateful Workload DR Checks

Automatic DR Posture Discovery:

Flowfish automatically detects stateful applications and evaluates DR posture.

DR Metrics:

RPO (Recovery Point Objective):

• Backup frequency is detected
• Last backup time is checked
• Calculated RPO: Time elapsed since last backup

RTO (Recovery Time Objective):

• Restore time estimates
• Automated failover capability
• Calculated RTO: Estimated restore duration

Parity Check:

• Primary vs replica data consistency
• Replication lag monitoring
• Split-brain detection

DR Posture Dashboard:

Overall DR Score: 75/100 🟡 (Needs Improvement)

Critical Issues:
🔴 postgres-primary: No backup in 36 hours
   RPO: >36h (Target: 1h)
   Action: Enable automated backups

🔴 redis-cache: No replication configured
   RTO: >2h (Target: 5min)
   Action: Configure Redis Sentinel

Integration:

• Velero: Kubernetes backup/restore status
• Stash: Backup job status
• Cloud Provider: EBS snapshots, Azure Disk snapshots

3.4. Advanced Governance Automation

3.4.1. CI/CD Pipeline Integration

Pre-Deployment Checks:

Flowfish runs automated pre-deployment checks integrated with CI/CD:

Policy Checks:

• ✅ Network Policy Coverage: Is there a network policy for the new service?
• ✅ Security Context: Compliant with pod security standards?
• ✅ Resource Limits: Are CPU/memory limits defined?
• ✅ Breaking Changes: Any API breaking changes?
• ✅ Dependency Health: Are dependent services healthy?

Admission Controller (Webhook):

Deployment-time validation via Kubernetes admission webhook:

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: flowfish-validator
webhooks:
- name: validate.flowfish.io
  rules:
  - operations: ["CREATE", "UPDATE"]
    apiGroups: ["apps"]
    resources: ["deployments", "statefulsets"]

GitOps Integration:

Pre-sync policy checks with ArgoCD, FluxCD:

# ArgoCD Pre-Sync Hook
apiVersion: batch/v1
kind: Job
metadata:
  name: flowfish-presync-check
  annotations:
    argocd.argoproj.io/hook: PreSync

CI/CD Platform Plugins:

GitHub Actions:

- name: Flowfish Policy Check
  uses: flowfish/github-action@v1
  with:
    manifests: k8s/*.yaml
    fail-on: critical

3.4.2. Policy as Code

Custom Policy Definition:

apiVersion: policy.flowfish.io/v1
kind: GovernancePolicy
metadata:
  name: production-standards
spec:
  rules:
  - name: require-network-policy
    severity: high
    condition: deployment.networkPolicy.exists == true
    
  - name: dependency-health-check
    severity: critical
    condition: all dependencies.health == "healthy"

Policy Enforcement:

• Blocking: Deployment blocked on critical policy violations
• Warning: High/Medium violations produce warnings
• Audit: All violations are logged

3.5. Explainable AI & Natural Language Queries

3.5.1. Natural Language Query Interface

Conversational Interface:

Users can ask questions in natural language:

Example Queries:

"Show me all external connections from payment service"
→ Executes graph query, returns results with evidence

"Which services depend on the database?"
→ Upstream dependency query

"What changed in the last 24 hours?"
→ Change detection timeline

"Why is checkout-service slow today?"
→ Performance analysis + anomaly correlation

"What happens if I delete redis-cache?"
→ Impact simulation query

"Find all services without network policies"
→ Governance compliance query

3.5.2. Grounded AI Responses

Explainable AI with Evidence:

Every AI answer is supported by evidence:

User: "Is payment-service secure?"

AI Response:
Security Assessment: ⚠️ Medium Risk (55/100)

Findings:
1. ✅ Network Policy: Configured correctly
   Evidence: NetworkPolicy 'payment-policy' exists
   Source: Kubernetes API, verified 2024-01-20
   
2. 🔴 Unencrypted Connection: HTTP to internal-api
   Evidence: eBPF traced 1,200 HTTP connections
   Source: Inspektor Gadget network tracer
   Recommendation: Enable TLS

Confidence: 95%
Data Sources: eBPF, Kubernetes API, Istio

Key Features:

• ✅ Evidence-Based: Source shown for every claim
• ✅ Source Attribution: Which data source it came from
• ✅ Confidence Scores: AI confidence level
• ✅ Actionable: Concrete recommendations

3.5.3. AI-Assisted Troubleshooting

Interactive Debugging:

User: "Payment API is failing, help me debug"

AI: Let me investigate...

Step 1: Checking payment-service health
✅ Pods: 5/5 running, healthy

Step 2: Analyzing recent changes
⚠️ Found: payment-service updated v2.3→v2.5, 2 hours ago

Step 3: Checking dependencies
🔴 Issue: checkout-service returning 502 errors
   Started: 2 hours ago (matches deployment)

Root Cause: API version mismatch after upgrade

Recommendations:
1. Rollback payment-service to v2.3
2. Update checkout-service to v2/validate endpoint

Would you like me to create rollback CR?
[ Yes ] [ No ]

3.6. Advanced AI/ML-Based Analysis

3.6.1. Predictive Analytics

Traffic Forecasting:

• Request volume prediction (next 1 hour, 24 hours, 7 days)
• Resource usage prediction
• Capacity planning recommendations
• Scaling suggestions

Anomaly Prediction:

• Predict potential anomalies based on patterns
• Pre-emptive alerting
• Root cause prediction

3.2.2. Recommendation Engine

Automated Recommendations:

• Network policy suggestions (least-privilege)
• Service mesh configuration
• Resource allocation (CPU, memory)
• Scaling recommendations
• Security hardening steps

ML Models:

• Time-series forecasting (Prophet, LSTM)
• Clustering (DBSCAN for similar services)
• Classification (normal vs anomalous)
• Reinforcement learning (optimal policy suggestions)

3.3. Automated Remediation Suggestions

3.3.1. Auto-Remediation

Remediatable Issues:

• High-risk external connections → Block via NetworkPolicy
• Unencrypted traffic → Enforce TLS
• Cross-namespace violations → Update policies
• Resource exhaustion → Scale up

Remediation Workflow:

1. Issue detected
2. Remediation action suggested
3. User approval required (or auto-approve for low-risk)
4. Action executed (kubectl apply)
5. Verification (did it work?)
6. Rollback if failed

3.3.2. Playbook Automation

Runbook Integration:

• Pre-defined playbooks for common scenarios
• Triggered by specific anomalies/changes
• Step-by-step execution
• Human-in-the-loop approval gates
• Audit log of all actions

3.4. Compliance and Audit Reports

3.4.1. Compliance Frameworks

Supported Standards:

• PCI DSS (network segmentation)
• HIPAA (data flow auditing)
• SOC 2 (access controls)
• ISO 27001 (security monitoring)
• GDPR (data locality)

Compliance Checks:

• Automated compliance scanning
• Policy violations detection
• Compliance score per namespace
• Remediation guidance
• Evidence collection for audits

3.4.2. Audit Reports

Report Types:

• Daily activity report
• Weekly security summary
• Monthly compliance report
• Quarterly trend analysis
• Annual review report
• Custom reports (user-defined)

Report Contents:

• Executive summary
• Key metrics and trends
• Anomalies and incidents
• Changes and deployments
• Compliance status
• Recommendations
• Detailed appendices

Report Formats:

• PDF (formatted, printable)
• HTML (web-viewable)
• JSON (machine-readable)
• Excel (data analysis)

Distribution:

• Email delivery
• Slack/Teams notification
• S3/Blob storage upload
• Report portal (web-based)

3.5. Advanced Visualization

3.5.1. 3D Graph Visualization

3D Layout:

• Three.js based rendering
• Z-axis for time dimension or hierarchy
• Orbital controls (rotate, zoom, pan)
• Node stacking (multiple pods in one view)
• Enhanced visual appeal

3.5.2. Logical Layer View

Application Layers:

• Layer 7: Application (microservices)
• Layer 6: Presentation (API gateways)
• Layer 5: Session (service mesh)
• Layer 4: Transport (load balancers)
• Layer 3: Network (ingress/egress)
• Layer 2: Data Link (CNI)
• Layer 1: Physical (nodes)

Layer-based Filtering:

• Show only specific layers
• Highlight layer boundaries
• Layer-to-layer communication emphasis

3.5.3. Heat Maps

Heatmap Types:

• Traffic intensity heatmap (request volume)
• Latency heatmap (response time)
• Error rate heatmap
• Time-of-day heatmap (hourly patterns)
• Day-of-week heatmap

Visualization:

• Color gradients (green → yellow → red)
• Interactive hover (show exact values)
• Exportable as image

3.6. Custom Dashboard Builder

3.6.1. Dashboard Customization

Widget Library:

• Metrics cards (single value)
• Time-series charts (line, area)
• Bar charts, pie charts
• Tables (sortable, filterable)
• Graph visualizations
• Heatmaps
• Text/markdown widgets
• Image widgets

Drag-and-Drop:

• Visual editor
• Grid layout system
• Widget resizing
• Widget positioning
• Widget duplication
• Widget templates

3.6.2. Dashboard Sharing

Sharing Options:

• Public link (read-only, time-limited)
• Team sharing (specific users/groups)
• Embed code (iframe)
• PDF export
• Schedule snapshot emails

Dashboard Templates:

• Pre-built templates library
• Community-contributed templates
• Import/export templates (JSON)

---

📊 Feature Comparison Matrix

Feature	Phase 1 (MVP)	Phase 2 (Advanced)	Phase 3 (Enterprise)
Data Collection	✅ eBPF/Inspektor Gadget	✅	✅
Automatic Discovery	✅ Real-time	✅ Real-time + Historical	✅
Graph Visualization	✅ Basic (2D)	✅ Advanced (filtering)	✅ 3D + Layers
Analysis Wizard	✅ 4-step	✅	✅
Dashboards	✅ Main + Live Map	✅ + 5 advanced	✅ + Custom builder
RBAC	✅ 4 roles	✅ + Custom roles	✅
Authentication	✅ OAuth/SSO, K8s SA	✅	✅
API Key Management	✅	✅	✅
Pod & Deployment Annotations	✅	✅	✅
Integration Hub	❌	✅	✅
Multi-Cluster	❌	✅	✅
Change Detection	❌	✅	✅
Anomaly Detection	❌	✅ LLM	✅ LLM + ML models
Import/Export	❌	✅ CSV + JSON	✅
Risk Scores	❌	✅	✅
Baseline	❌	✅	✅
What-If Analysis	❌	❌	✅
Change Simulation (CAP)	❌	❌	✅
Universal Data Ingestion	✅ eBPF	✅ + Prometheus	✅ + Service Mesh, APM, Logs
Dependency Provenance	❌	⚠️ Basic	✅ Full multi-source
DR Posture Checks	❌	❌	✅ RPO/RTO/Parity
Governance Automation	❌	❌	✅ CI/CD + Admission Control
Natural Language Queries	❌	❌	✅
Explainable AI	❌	⚠️ Basic	✅ Grounded responses
Auto-Remediation	❌	❌	✅
Compliance Reports	❌	❌	✅
Predictive Analytics	❌	❌	✅
SIEM Integration	❌	✅	✅
CAP Tool Integration	❌	❌	✅ ServiceNow, Jira
API	✅ Core	✅ Advanced	✅ Full

---

🎯 Success Criteria

Phase 1 (MVP) Success Criteria

• Inspektor Gadget deployed and running successfully
• Data collection works on clusters with at least 1000 pods
• Graph visualization can render 500 nodes + 1000 edges
• Real-time updates run with <5 second latency
• Four user roles implemented and working
• OAuth SSO integration works with at least one provider
• Analysis wizard can create analyses in 4 steps

Phase 2 Success Criteria

• Historical data stored and queryable for 30 days
• Change detection operates at 95% accuracy
• LLM anomaly detection responds within 10 seconds
• Import/export can handle 10MB+ files
• Multi-cluster can manage 5+ clusters
• Webhook can send 1000+ events/hour

Phase 3 Success Criteria

• What-if simulation returns results in <30 seconds
• Change simulation (CAP) 95%+ accuracy
• Predictive analytics 80%+ accuracy
• DR posture assessment for 100+ stateful workloads
• Policy as Code with 50+ custom policies
• Natural language queries 90%+ intent recognition
• Grounded AI responses 95%+ evidence-backed
• CI/CD integration with 5+ platforms (GitHub, GitLab, Jenkins, ArgoCD, FluxCD)
• Compliance reports can be generated for 10 different frameworks
• Custom dashboard supports 50+ widgets

---

Last Updated: January 2025
Version: 1.0.0
Status: Design Documentation