27 Commits

Author SHA1 Message Date
taylanbakircioglu 613ed0bf99 fix(security): patch CRITICAL Dependabot findings (python-jose 3.4.0, jspdf 4.2.1)
- python-jose[cryptography] 3.3.0 -> 3.4.0 (CVE-2024-33663 algorithm
  confusion) in backend + api-gateway. App uses HS256 only; encode/decode
  API unchanged; 3.3.0<->3.4.0 tokens cross-verifiable (rolling-deploy safe).
- jspdf 4.0.0 -> 4.2.1 (CVE-2026-31938 HTML injection in new-window paths)
  in frontend; @babel/runtime bumped 7.28.4 -> 7.28.6 (required by jspdf
  4.2.1). App uses only new jsPDF()/autoTable()/doc.save() — vulnerable
  output path not used.

Verified: jose roundtrip + cross-version, frontend npm build, backend &
frontend docker images, and Kubernetes (kind) jose 3.4.0 + frontend serve.
No feature/UI behavior change; backward compatible.
2026-06-03 01:35:06 +03:00
taylanbakircioglu 7a5d4904a9 feat: sync v2.7.0 per-path L7 + v2.8.0 (rabbitmq resilience, version bump)
Bring the public mirror up to date with the latest developments:

- L7 HTTP Path Visibility & Per-Path Edge Model (v2.7.0): real HTTP
  paths in the Service Map / Integration Hub / exports instead of "/".
  Touches event_transformer (path extraction), graph-writer per-path
  MERGE key + indexes, graph-query per-path counting, and the frontend.
- Beyla route decoration to fix the `/*/*/*/*/*` star-explosion in
  API-gateway namespaces (low-cardinality route labelling).
- RabbitMQ queue-declaration resilience: graph-writer and
  timeseries-writer tolerate a pre-existing queue whose arguments differ
  from the declaration (bind to the existing queue instead of crashing).
- Version bump to 2.8.0 + CHANGELOG.

Example hostnames/annotations use placeholder values (example.com).
2026-06-02 23:49:10 +03:00
taylanbakircioglu 6e503368f7 feat: L7 (Application Level) observability — Service Map, Trace Explorer, APM, Beyla
- Grafana Beyla DaemonSet for kernel-level HTTP/gRPC/DNS capture (passive,
  zero application changes, W3C traceparent header propagation)
- flowfish-l7-collector in-cluster bridge: OTLP receiver + buffered pull API
- L7 Ingestion Service: K8s service-proxy poll → enrich → RabbitMQ
- ClickHouse l7_http_flows / l7_grpc_flows / l7_dns_flows + APM RED MVs
- Neo4j L7Workload nodes + SAME_WORKLOAD cross-cluster bridges
- New pages: Service Map, Trace Explorer, APM Services List, APM Service Detail
- Analysis Wizard now supports L4 / L7 / Both modes with HTTP/gRPC/DNS picks
- Integration Hub gains L7 dependency summary + tree-summary integrations
- Multi-Cluster Management: dual-agent install (Inspector Gadget L4 + Beyla L7),
  runtime OpenShift detection so SCCs auto-install with kubectl too
- ServiceMap edge → Trace Explorer drill-down with virtual_trace_id correlation
- Docs: new L7 architecture diagram, README L7 sections, 3 new screenshots
2026-05-14 10:09:15 +03:00
taylanbakircioglu 3783763aa8 feat: configurable CIDR ranges, improved map drawer & noise filtering
- Add Settings UI for managing SDN pod CIDR ranges with backend API
- Propagate network config from Settings through orchestrator/gRPC to ingestion service
- Improve map drawer: correct Service node labels, simplify IP display
- Add aggregated workload drawer with pod details table, cluster badges
- Filter sdn-infrastructure noise entries in Integration Hub dependency summaries
- Maintain full backward compatibility with hardcoded CIDR defaults

Made-with: Cursor
2026-04-09 17:12:14 +03:00
taylanbakircioglu e2ed026ee4 fix: clear stale gadget warnings when analysis is restarted
When an analysis is stopped and restarted, the previous run's
gadget_errors and has_gadget_warnings were persisted in output_config
and never cleared, causing a false "Some gadgets failed to start"
warning in the UI even when all gadgets started successfully.

Made-with: Cursor
2026-04-07 14:35:47 +03:00
taylanbakircioglu b27cb93795 feat: make gadget upgrade script interactive and safer, apply pending fixes
Upgrade script now interactively prompts for namespace, target version,
registry, memory limit, and buffer length with validation at each step.
Also includes pending fixes: sync gRPC method, migration dollar-quoting,
and double-v prefix in gadget version display.

Made-with: Cursor
2026-04-07 14:08:31 +03:00
taylanbakircioglu 38a5411eb8 feat: Gadget memory optimization, continuous analysis controls, v0.50.1 upgrade support, Flowfish v2.5.0
- Reduce Inspektor Gadget memory limit from 12Gi to 6Gi with buffer optimization (16384->8192)
- Add InspektorGadgetHighMemory (>3Gi) and CriticalMemory (>5Gi) Prometheus alerts
- Add configurable event ingestion rate limiting via Settings UI (ingestion_rate_limit_per_second)
- Update /analysis-limits/defaults endpoint to read from DB with Pydantic fallback
- Orchestrator fetches rate limit via isolated HTTP call (avoids cross-thread async issues)
- Ingestion service uses session-based rate limit with >0 comparison (fixes or-operator semantic bug)
- Add gadget_version (field 22) and max_events_per_second (field 23) to protobuf StartCollectionRequest
- Sync backend/proto with proto/ (add GadgetError message to backend copy)
- Upgrade Inspektor Gadget from v0.48.0 to v0.50.1 across all references
- Dynamic OCI image tagging: cluster-specific gadget_version flows through gRPC to ingestion service
- Fix get_cluster_sync SQL to include gadget_version column
- Fix NULL gadget_version causing protobuf TypeError (row[10] or '' pattern)
- Add GET /clusters/{id}/gadget-upgrade-script endpoint with cluster-specific parameters
- Add GadgetUpgradeModal and semver-based upgrade badge to Cluster Management page
- Add upgrade available indicator to Dashboard OperationsTab
- Include supported_gadget_version in GET /clusters response
- Update cluster_validator minimum supported version from v0.18 to v0.46.0
- Add ingestion_rate_limit_per_second to migration seed data and auto_stop_monitor fallback
- Improve AnalysisWizard Rolling Window and Recurring mode descriptions
- Add GADGET_SUPPORTED_VERSION and GADGET_MIN_SUPPORTED_VERSION to backend config
- Fix hardcoded "1.0.0" version in backend root/info endpoints to use __full_version__
- Replace grep -oP with POSIX grep -oE in upgrade script (macOS compatibility)
- Add ConfigMap events-buffer-length optimization step to upgrade script
- Add GADGET_VERSION sed substitution to prepare-manifests.sh
- Bump Flowfish version from 2.4.0 to 2.5.0
- Update pipeline GADGET_VERSION variable and all documentation references

Made-with: Cursor
2026-04-07 13:07:04 +03:00
taylanbakircioglu 4620eb4a4a feat: server-side event histogram for Events Timeline visualization
Replace client-side event bucketing (limited to 50 paginated events) with
server-side ClickHouse aggregation using toStartOfInterval(). This fixes
the timeline showing a single bar despite multiple analysis runs.

Backend:
- Add query_event_histogram() to timeseries-query with epoch-aligned
  bucket generation, capability_checks security filter, and count()-based
  empty table detection
- Add GET /events/histogram endpoints through all layers (timeseries-query,
  backend repository, service, router with analysis_id validation)

Frontend:
- Add getEventHistogram RTK Query hook with EventHistogram cache tag
- Replace client-side timelineData bucketing with server-side histogram
- Stacked bar colors (sorted by eventTypeConfig order for consistency)
- Clickable bars for time-range filtering (zoom-in)
- Loading skeleton, smart time labels (DD MMM HH:mm for multi-day),
  full event type legend filtered by presence, total event count in header
- event_types filter synced with histogram query

Documentation:
- Update CURRENT_ARCHITECTURE, DATA_SOURCES, ARCHITECTURE_DETAILED,
  services/README, and OpenAPI spec with /events/histogram endpoint

Made-with: Cursor
2026-04-06 15:47:49 +03:00
taylanbakircioglu 01c69fc8cd fix: dark mode compatibility for Change Detection and Login pages
- Anomalies card: dark-aware gradient background
- Compare Runs summary cards: dark-aware background colors
- Login page: dark mode CSS for inputs, background, and card shadow
- Login version: use dynamic APP_VERSION instead of hardcoded 1.0.0
- Config: clean up comment in dns_search_domains field

Made-with: Cursor
2026-04-02 15:16:24 +03:00
taylanbakircioglu 99f44f854c feat: DNS search domain normalization to deduplicate graph nodes
Kubernetes DNS search domain expansion creates multiple NXDOMAIN queries
for the same external domain (e.g. auth.docker.io.cluster.local,
auth.docker.io.svc.cluster.local). Without normalization, each variant
becomes a separate Neo4j node, inflating Public connections on the Map.

Backend (graph-writer):
- Rewrite _normalize_dns_name with TLD-safe 4-step algorithm
- Strip .svc.cluster.local, .<ns>.svc.cluster.local, .cluster.local
- Support custom search domains via DNS_SEARCH_DOMAINS env var
- Apply normalization in process_dns_query and process_sni_event
- Fix NXDOMAIN vertex overriding NOERROR resolved IP data
- Add KNOWN_TLDS/MULTI_LEVEL_TLDS for .com.tr/.co.uk safety checks

Frontend (NetworkExplorer):
- Add "Hide Search Domain Artifacts" toggle on DNS Queries tab
- Detect NXDOMAIN entries ending with .cluster.local
- Show artifact count, toggle hidden by default

Raw ClickHouse data and API responses are NOT modified — only the
dependency graph (Neo4j) is deduplicated. Full debugging data preserved.

Tests: 31 cases covering normalization, edge cases, and integration.
Made-with: Cursor
2026-04-02 14:34:00 +03:00
taylanbakircioglu 582e3eba68 feat: per-service dependency breakdown in summary API
Restructure /dependencies/summary response to provide per-service
downstream/callers breakdowns instead of a single aggregated view.

Backend:
- Unified loop in format_dependency_summary handles single and multi-service
  identically, fixing replica data loss bug for single-service queries
- Each matched_service now carries its own downstream/callers DependencySummaryGroup
- New top-level summary object with globally deduplicated aggregate counts
  (split downstream_critical_count and callers_critical_count)
- hop_count added to _compact_service (only when > 1 to avoid depth=1 noise)
- _dedup_entries now preserves entry with lowest hop_count on collision
- multi_service flag now correctly checks collapsed workload count

Frontend:
- Updated TypeScript interfaces (DependencySummary, MatchedService with groups)
- IntegrationHub Preview: per-service Collapse for multi-service, summary stats
- DependencyCategoryGroup: conditional hop_count column for multi-depth queries
- Snippet builders updated to parse matched_services and summary structure

API & Docs:
- OpenAPI spec: new DependencySummary schema, depth parameter descriptions
- README, feature-list, DATA_ARCHITECTURE docs updated
- Fixed pre-existing doc bug (default traversal depth 3 -> 1)

Made-with: Cursor
2026-03-30 01:08:39 +03:00
taylanbakircioglu 84988333cb fix: use platform-aware architecture detection in Dockerfiles for multi-arch builds
Made-with: Cursor
2026-03-29 22:21:09 +03:00
taylanbakircioglu 67fc39aeab fix: normalize kind field to prevent null vs empty string inconsistency
Made-with: Cursor
2026-03-29 20:41:49 +03:00
taylanbakircioglu 03fafabf7e fix: restore Cypher precision for exact annotation keys and optimize glob loop
Made-with: Cursor
2026-03-29 20:38:16 +03:00
taylanbakircioglu 7c43e4c1b1 feat: annotation wildcard matching and openshift noise filter
Made-with: Cursor
2026-03-29 20:32:59 +03:00
taylanbakircioglu dd7039afd0 fix: add query-layer defense-in-depth for kind resolution and annotation filtering
Made-with: Cursor
2026-03-29 19:02:23 +03:00
taylanbakircioglu f7d72e0c2f fix: improve AI Hub summary output quality (dedup, kind, annotations)
Made-with: Cursor
2026-03-27 20:23:32 +03:00
taylanbakircioglu 3717445302 fix: exempt DNS events from aggressive destination filter
Made-with: Cursor
2026-03-26 10:18:54 +03:00
taylanbakircioglu be698d0b0c feat: add aggressive/conservative strategy for exclusion filter
Made-with: Cursor
2026-03-26 09:29:45 +03:00
taylanbakircioglu 7247f7a0f7 feat: add system pod exclusion filter for analyses
Made-with: Cursor
2026-03-25 19:01:15 +03:00
taylanbakircioglu 4508a756bb docs: translate all documentation to English, update API docs with AI Integration Hub and annotation features
Made-with: Cursor
2026-03-25 14:27:37 +03:00
taylanbakircioglu 3f35f00b36 feat: add scheduled analysis, rolling window retention & cluster safety
Made-with: Cursor
2026-03-25 12:26:25 +03:00
taylanbakircioglu 42f2dc2a50 feat: improve UI quality, data filtering, and wizard UX
Made-with: Cursor
2026-03-24 17:40:21 +03:00
taylanbakircioglu 3caa9d9b71 feat: improve AI Integration Hub with multi-service aggregation and UX fixes
Made-with: Cursor
2026-03-24 16:38:35 +03:00
taylanbakircioglu 16d249e7a3 feat: fix annotation ingestion via gRPC and enrich annotation display across UI
Made-with: Cursor
2026-03-24 15:04:21 +03:00
taylanbakircioglu fdc2ec4b7d feat: add pod annotations support, dependency stream API, and AI Integration Hub
Made-with: Cursor
2026-03-24 13:25:38 +03:00
taylanbakircioglu d7ca50b387 feat: Flowfish - eBPF-powered multi-cluster Kubernetes observability platform
Multi-cluster dependency mapping, real-time network monitoring,
impact analysis, and CI/CD integration capabilities.

Made-with: Cursor
2026-03-29 21:43:20 +03:00