gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-01 09:48:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(obs): Update observability docker compose stack (#2010)

Browse Source
This commit is contained in:
evan slack
2026-02-28 14:03:50 -05:00
committed by GitHub
parent fe884eabfc
commit fd1b903531
13 changed files with 319 additions and 161 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.docker/observability/README.md
+30 -30
View File
@@ -15,28 +15,28 @@ The stack is composed of the following best-in-class open-source components:
## Architecture ## Architecture
1. **Telemetry Collection**: Applications send OTLP (OpenTelemetry Protocol) data (Metrics, Logs, Traces) to the **OpenTelemetry Collector**. 1. **Telemetry Collection**: Applications send OTLP (OpenTelemetry Protocol) data (Metrics, Logs, Traces) to the **OpenTelemetry Collector**.
2. **Processing & Exporting**: The Collector processes the data (batching, memory limiting) and exports it to the respective backends: 2. **Processing & Exporting**: The Collector processes the data (batching, memory limiting) and exports it to the respective backends:
- **Traces** -> **Tempo** (Primary) & **Jaeger** (Secondary/Optional) - **Traces** -> **Tempo** (Primary) & **Jaeger** (Secondary/Optional)
- **Metrics** -> **Prometheus** (via scraping the Collector's exporter) - **Metrics** -> **Prometheus** (via scraping the Collector's exporter)
- **Logs** -> **Loki** - **Logs** -> **Loki**
3. **Visualization**: **Grafana** connects to all backends (Prometheus, Tempo, Loki, Jaeger) to provide a unified dashboard experience. 3. **Visualization**: **Grafana** connects to all backends (Prometheus, Tempo, Loki, Jaeger) to provide a unified dashboard experience.
## Features ## Features
- **Full Persistence**: All data (Metrics, Logs, Traces) is persisted to Docker volumes, ensuring no data loss on restart. - **Full Persistence**: All data (Metrics, Logs, Traces) is persisted to Docker volumes, ensuring no data loss on restart.
- **Correlation**: Seamless navigation between Metrics, Logs, and Traces in Grafana. - **Correlation**: Seamless navigation between Metrics, Logs, and Traces in Grafana.
- Jump from a Metric spike to relevant Traces. - Jump from a Metric spike to relevant Traces.
- Jump from a Trace to relevant Logs. - Jump from a Trace to relevant Logs.
- **High Performance**: Optimized configurations for batching, compression, and memory management. - **High Performance**: Optimized configurations for batching, compression, and memory management.
- **Standardized Protocols**: Built entirely on OpenTelemetry standards. - **Standardized Protocols**: Built entirely on OpenTelemetry standards.
## Quick Start ## Quick Start
### Prerequisites ### Prerequisites
- Docker - Docker
- Docker Compose - Docker Compose
### Deploy ### Deploy
@@ -48,12 +48,12 @@ docker compose up -d
### Access Dashboards ### Access Dashboards
| Service | URL | Credentials | Description | | Service | URL | Credentials | Description |
| :--- | :--- | :--- | :--- | | :------------- | :----------------------------------------------- | :---------------- | :----------------------------- |
| **Grafana** | [http://localhost:3000](http://localhost:3000) | `admin` / `admin` | Main visualization hub. | | **Grafana** | [http://localhost:3000](http://localhost:3000) | `admin` / `admin` | Main visualization hub. |
| **Prometheus** | [http://localhost:9090](http://localhost:9090) | - | Metric queries and status. | | **Prometheus** | [http://localhost:9090](http://localhost:9090) | - | Metric queries and status. |
| **Jaeger UI** | [http://localhost:16686](http://localhost:16686) | - | Secondary trace visualization. | | **Jaeger UI** | [http://localhost:16686](http://localhost:16686) | - | Secondary trace visualization. |
| **Tempo** | [http://localhost:3200](http://localhost:3200) | - | Tempo status/metrics. | | **Tempo** | [http://localhost:3200](http://localhost:3200) | - | Tempo status/metrics. |
## Configuration ## Configuration
@@ -61,10 +61,10 @@ docker compose up -d
Data is stored in the following Docker volumes: Data is stored in the following Docker volumes:
- `prometheus-data`: Prometheus metrics - `prometheus-data`: Prometheus metrics
- `tempo-data`: Tempo traces (WAL and Blocks) - `tempo-data`: Tempo traces (WAL and Blocks)
- `loki-data`: Loki logs (Chunks and Rules) - `loki-data`: Loki logs (Chunks and Rules)
- `jaeger-data`: Jaeger traces (Badger DB) - `jaeger-data`: Jaeger traces (Badger DB)
To clear all data: To clear all data:
@@ -74,12 +74,12 @@ docker compose down -v
### Customization ### Customization
- **Prometheus**: Edit `prometheus.yml` to add scrape targets or alerting rules. - **Prometheus**: Edit `prometheus.yml` to add scrape targets or alerting rules.
- **Grafana**: Dashboards and datasources are provisioned from the `grafana/` directory. - **Grafana**: Dashboards and datasources are provisioned from the `grafana/` directory.
- **Collector**: Edit `otel-collector-config.yaml` to modify pipelines, processors, or exporters. - **Collector**: Edit `otel-collector-config.yaml` to modify pipelines, processors, or exporters.
## Troubleshooting ## Troubleshooting
- **Service Health**: Check the health of services using `docker compose ps`. - **Service Health**: Check the health of services using `docker compose ps`.
- **Logs**: View logs for a specific service using `docker compose logs -f <service_name>`. - **Logs**: View logs for a specific service using `docker compose logs -f <service_name>`.
- **Otel Collector**: Check `http://localhost:13133` for health status and `http://localhost:1888/debug/pprof/` for profiling. - **Otel Collector**: Check `http://localhost:13133` for health status and `http://localhost:1888/debug/pprof/` for profiling.
.docker/observability/docker-compose.yml
+113 -60
View File
@@ -13,93 +13,149 @@
# limitations under the License. # limitations under the License.
services: services:
rustfs:
security_opt:
- "no-new-privileges:true"
image: rustfs/rustfs:latest
container_name: rustfs-server
ports:
- "9000:9000" # S3 API port
- "9001:9001" # Console port
environment:
- RUSTFS_VOLUMES=/data/rustfs
- RUSTFS_ADDRESS=0.0.0.0:9000
- RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001
- RUSTFS_CONSOLE_ENABLE=true
- RUSTFS_CORS_ALLOWED_ORIGINS=*
- RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS=*
- RUSTFS_ACCESS_KEY=rustfsadmin
- RUSTFS_SECRET_KEY=rustfsadmin
- RUSTFS_OBS_LOGGER_LEVEL=info
- RUSTFS_OBS_ENDPOINT=http://otel-collector:4318
volumes:
- rustfs-data:/data/rustfs
networks:
- otel-network
restart: unless-stopped
healthcheck:
test:
[
"CMD",
"sh",
"-c",
"curl -f http://127.0.0.1:9000/health && curl -f http://127.0.0.1:9001/rustfs/console/health",
]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
depends_on:
otel-collector:
condition: service_started
rustfs-init:
image: alpine
container_name: rustfs-init
volumes:
- rustfs-data:/data
networks:
- otel-network
command: >
sh -c "
chown -R 10001:10001 /data &&
echo 'Volume Permissions fixed' &&
exit 0
"
restart: no
# --- Tracing --- # --- Tracing ---
tempo-init:
image: busybox:latest
command: [ "sh", "-c", "chown -R 10001:10001 /var/tempo" ]
volumes:
- ./tempo-data:/var/tempo
user: root
networks:
- otel-network
restart: "no"
tempo: tempo:
image: grafana/tempo:latest image: grafana/tempo:latest
user: "10001" container_name: tempo
command: [ "-config.file=/etc/tempo.yaml" ] command: ["-config.file=/etc/tempo.yaml"]
volumes: volumes:
- ./tempo.yaml:/etc/tempo.yaml:ro - ./tempo.yaml:/etc/tempo.yaml:ro
- ./tempo-data:/var/tempo - tempo-data:/var/tempo
ports: ports:
- "3200:3200" # tempo - "3200:3200" # tempo
- "4317" # otlp grpc - "4317" # otlp grpc
- "4318" # otlp http - "4318" # otlp http
restart: unless-stopped
networks: networks:
- otel-network - otel-network
restart: unless-stopped
depends_on:
- redpanda
healthcheck: healthcheck:
test: [ "CMD-SHELL", "wget --spider -q http://localhost:3200/metrics || exit 1" ] test: ["CMD", "wget", "--spider", "-q", "http://localhost:3200/ready"]
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 5 retries: 3
start_period: 40s start_period: 15s
redpanda:
image: redpandadata/redpanda:latest # for tempo ingest
container_name: redpanda
ports:
- "9092:9092"
networks:
- otel-network
restart: unless-stopped
command: >
redpanda start --overprovisioned
--mode=dev-container
--kafka-addr=PLAINTEXT://0.0.0.0:9092
--advertise-kafka-addr=PLAINTEXT://redpanda:9092
jaeger: jaeger:
image: jaegertracing/jaeger:latest image: jaegertracing/jaeger:latest
container_name: jaeger
environment: environment:
- TZ=Asia/Shanghai
- SPAN_STORAGE_TYPE=badger - SPAN_STORAGE_TYPE=badger
- BADGER_EPHEMERAL=false - BADGER_EPHEMERAL=false
- BADGER_DIRECTORY_VALUE=/badger/data - BADGER_DIRECTORY_VALUE=/badger/data
- BADGER_DIRECTORY_KEY=/badger/key - BADGER_DIRECTORY_KEY=/badger/key
- COLLECTOR_OTLP_ENABLED=true - COLLECTOR_OTLP_ENABLED=true
volumes: volumes:
- ./jaeger-data:/badger - ./jaeger.yaml:/etc/jaeger/config.yml
- jaeger-data:/badger
ports: ports:
- "16686:16686" # Web UI - "16686:16686" # Web UI
- "14269:14269" # Admin/Metrics - "14269:14269" # Admin/Metrics
- "4317" - "4317" # otlp grpc
- "4318" - "4318" # otlp http
command: ["--config", "/etc/jaeger/config.yml"]
networks: networks:
- otel-network - otel-network
restart: unless-stopped
healthcheck: healthcheck:
test: [ "CMD-SHELL", "wget --spider -q http://localhost:14269 || exit 1" ] test: ["CMD", "wget", "--spider", "-q", "http://localhost:14269"]
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 5 retries: 3
start_period: 20s start_period: 15s
# --- Metrics --- # --- Metrics ---
prometheus: prometheus:
image: prom/prometheus:latest image: prom/prometheus:latest
environment: container_name: prometheus
- TZ=Asia/Shanghai
volumes: volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml:ro - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
- ./prometheus-data:/prometheus - prometheus-data:/prometheus
ports: ports:
- "9090:9090" - "9090:9090"
command: command:
- '--config.file=/etc/prometheus/prometheus.yml' - "--config.file=/etc/prometheus/prometheus.yml"
- '--web.enable-otlp-receiver' - "--web.enable-otlp-receiver" # Enable OTLP
- '--web.enable-remote-write-receiver' - "--web.enable-remote-write-receiver" # Enable remote write
- '--enable-feature=promql-experimental-functions' - "--enable-feature=promql-experimental-functions" # Enable info()
- '--storage.tsdb.min-block-duration=2h' - "--storage.tsdb.retention.time=30d"
- '--storage.tsdb.max-block-duration=2h'
- '--log.level=info'
- '--storage.tsdb.retention.time=30d'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/usr/share/prometheus/console_libraries'
- '--web.console.templates=/usr/share/prometheus/consoles'
restart: unless-stopped restart: unless-stopped
networks: networks:
- otel-network - otel-network
healthcheck: healthcheck:
test: [ "CMD-SHELL", "wget --spider -q http://localhost:9090/-/healthy || exit 1" ] test: ["CMD", "wget", "--spider", "-q", "http://localhost:9090/-/healthy"]
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 3 retries: 3
@@ -108,18 +164,18 @@ services:
loki: loki:
image: grafana/loki:latest image: grafana/loki:latest
environment: container_name: loki
- TZ=Asia/Shanghai
volumes: volumes:
- ./loki-config.yaml:/etc/loki/local-config.yaml:ro - ./loki.yaml:/etc/loki/loki.yaml:ro
- ./loki-data:/loki - loki-data:/loki
ports: ports:
- "3100:3100" - "3100:3100"
command: -config.file=/etc/loki/local-config.yaml command: -config.file=/etc/loki/loki.yaml
networks: networks:
- otel-network - otel-network
restart: unless-stopped
healthcheck: healthcheck:
test: [ "CMD-SHELL", "wget --spider -q http://localhost:3100/metrics || exit 1" ] test: ["CMD", "wget", "--spider", "-q", "http://localhost:3100/ready"]
interval: 15s interval: 15s
timeout: 10s timeout: 10s
retries: 5 retries: 5
@@ -129,8 +185,6 @@ services:
otel-collector: otel-collector:
image: otel/opentelemetry-collector-contrib:latest image: otel/opentelemetry-collector-contrib:latest
environment:
- TZ=Asia/Shanghai
volumes: volumes:
- ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml:ro - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml:ro
ports: ports:
@@ -143,52 +197,51 @@ services:
- "55679:55679" # zpages - "55679:55679" # zpages
networks: networks:
- otel-network - otel-network
restart: unless-stopped
depends_on: depends_on:
- tempo - tempo
- jaeger - jaeger
- prometheus - prometheus
- loki - loki
healthcheck: healthcheck:
test: [ "CMD-SHELL", "wget --spider -q http://localhost:13133 || exit 1" ] test: ["CMD", "wget", "--spider", "-q", "http://localhost:13133"]
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 3 retries: 3
start_period: 20s
# --- Visualization --- # --- Visualization ---
grafana: grafana:
image: grafana/grafana:latest image: grafana/grafana:latest
container_name: grafana
ports: ports:
- "3000:3000" - "3000:3000"
volumes:
- ./grafana/provisioning:/etc/grafana/provisioning
- ./grafana/dashboards:/var/lib/grafana/dashboards
- ./grafana-data:/var/lib/grafana
environment: environment:
- GF_SECURITY_ADMIN_PASSWORD=admin - GF_SECURITY_ADMIN_PASSWORD=admin
- GF_SECURITY_ADMIN_USER=admin - GF_SECURITY_ADMIN_USER=admin
- TZ=Asia/Shanghai volumes:
- GF_INSTALL_PLUGINS=grafana-pyroscope-datasource - ./grafana/provisioning:/etc/grafana/provisioning:ro
- GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/var/lib/grafana/dashboards/home.json - ./grafana/dashboards:/etc/grafana/dashboards:ro
restart: unless-stopped - grafana-data:/var/lib/grafana
networks: networks:
- otel-network - otel-network
restart: unless-stopped
depends_on: depends_on:
- prometheus - prometheus
- tempo - tempo
- loki - loki
healthcheck: healthcheck:
test: [ "CMD-SHELL", "wget --spider -q http://localhost:3000/api/health || exit 1" ] test: ["CMD", "wget", "--spider", "-q", "http://localhost:3000/api/health"]
interval: 10s interval: 10s
timeout: 5s timeout: 5s
retries: 3 retries: 3
volumes: volumes:
prometheus-data: rustfs-data:
tempo-data: tempo-data:
loki-data:
jaeger-data: jaeger-data:
prometheus-data:
loki-data:
grafana-data: grafana-data:
networks: networks:
.docker/observability/grafana-data/.gitignore
-1
View File
@@ -1 +0,0 @@
*
.docker/observability/grafana/provisioning/dashboards/dashboard.yml
+11
View File
@@ -0,0 +1,11 @@
apiVersion: 1
providers:
- name: "default"
orgId: 1
folder: ""
type: file
disableDeletion: false
updateIntervalSeconds: 10
options:
path: /etc/grafana/dashboards
.docker/observability/grafana/provisioning/datasources/datasources.yaml
+78
View File
@@ -0,0 +1,78 @@
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
uid: prometheus
url: http://prometheus:9090
access: proxy
isDefault: true
editable: false
jsonData:
httpMethod: GET
exemplarTraceIdDestinations:
- name: trace_id
datasourceUid: tempo
- name: Tempo
type: tempo
uid: tempo
access: proxy
url: http://tempo:3200
isDefault: false
editable: false
jsonData:
httpMethod: GET
serviceMap:
datasourceUid: prometheus
tracesToLogs:
datasourceUid: loki
tags: [ 'job', 'instance', 'pod', 'namespace', 'service.name' ]
mappedTags: [ { key: 'service.name', value: 'app' } ]
spanStartTimeShift: '-1h'
spanEndTimeShift: '1h'
filterByTraceID: true
filterBySpanID: false
tracesToMetrics:
datasourceUid: prometheus
tags: [ { key: 'service.name' }, { key: 'job' } ]
queries:
- name: 'Service-Level Latency'
query: 'sum(rate(traces_spanmetrics_latency_bucket{$$__tags}[5m])) by (le)'
- name: 'Service-Level Calls'
query: 'sum(rate(traces_spanmetrics_calls_total{$$__tags}[5m]))'
- name: 'Service-Level Errors'
query: 'sum(rate(traces_spanmetrics_calls_total{status_code="ERROR", $$__tags}[5m]))'
nodeGraph:
enabled: true
- name: Loki
type: loki
uid: loki
url: http://loki:3100
basicAuth: false
isDefault: false
editable: false
jsonData:
derivedFields:
- datasourceUid: tempo
matcherRegex: 'trace_id=(\w+)'
name: 'TraceID'
url: '$${__value.raw}'
- name: Jaeger
type: jaeger
uid: jaeger
url: http://jaeger:16686
access: proxy
isDefault: false
editable: false
jsonData:
tracesToLogs:
datasourceUid: loki
tags: [ 'job', 'instance', 'pod', 'namespace', 'service.name' ]
mappedTags: [ { key: 'service.name', value: 'app' } ]
spanStartTimeShift: '1s'
spanEndTimeShift: '-1s'
filterByTraceID: true
filterBySpanID: false
.docker/observability/jaeger-data/.gitignore
-1
View File
@@ -1 +0,0 @@
*
.docker/observability/jaeger-config.yaml → .docker/observability/jaeger.yaml
+26 -42
View File
@@ -13,12 +13,15 @@
# limitations under the License. # limitations under the License.
service: service:
extensions: [ jaeger_storage, jaeger_query, remote_sampling, healthcheckv2 ] extensions: [jaeger_storage, jaeger_query]
pipelines: pipelines:
traces: traces:
receivers: [ otlp, jaeger, zipkin ] receivers: [otlp]
processors: [ batch, adaptive_sampling ] processors: [batch]
exporters: [ jaeger_storage_exporter ] exporters: [jaeger_storage_exporter, spanmetrics]
metrics/spanmetrics:
receivers: [spanmetrics]
exporters: [prometheus]
telemetry: telemetry:
resource: resource:
service.name: jaeger service.name: jaeger
@@ -31,60 +34,41 @@ service:
host: 0.0.0.0 host: 0.0.0.0
port: 8888 port: 8888
logs: logs:
level: info level: DEBUG
extensions: extensions:
healthcheckv2:
use_v2: true
http:
jaeger_query: jaeger_query:
storage: storage:
traces: badger_store traces: some_storage
ui: metrics: some_metrics_storage
config_file: ./cmd/jaeger/config-ui.json
log_access: true
max_clock_skew_adjust: 0s
grpc:
endpoint: 0.0.0.0:16685
http:
endpoint: 0.0.0.0:16686
jaeger_storage: jaeger_storage:
backends: backends:
badger_store: some_storage:
badger: memory:
ephemeral: false max_traces: 100000
directory_key: /badger/key metric_backends:
directory_value: /badger/data some_metrics_storage:
span_store_ttl: 72h prometheus:
endpoint: http://prometheus:9090
normalize_calls: true
normalize_duration: true
remote_sampling: connectors:
adaptive: spanmetrics:
sampling_store: badger_store
initial_sampling_probability: 0.1
http:
grpc:
receivers: receivers:
otlp: otlp:
protocols: protocols:
grpc: grpc:
endpoint: "0.0.0.0:4317"
http: http:
endpoint: "0.0.0.0:4318"
jaeger:
protocols:
grpc:
thrift_binary:
thrift_compact:
thrift_http:
zipkin:
processors: processors:
batch: batch:
adaptive_sampling:
exporters: exporters:
jaeger_storage_exporter: jaeger_storage_exporter:
trace_storage: badger_store trace_storage: some_storage
prometheus:
endpoint: "0.0.0.0:8889"
.docker/observability/loki-data/.gitignore
-1
View File
@@ -1 +0,0 @@
*
.docker/observability/loki-config.yaml → .docker/observability/loki.yaml
+7 -12
View File
@@ -11,12 +11,11 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
auth_enabled: false auth_enabled: false
server: server:
http_listen_port: 3100 http_listen_port: 3100
grpc_listen_port: 9096 grpc_listen_port: 9095
log_level: info log_level: info
grpc_server_max_concurrent_streams: 1000 grpc_server_max_concurrent_streams: 1000
@@ -39,12 +38,6 @@ query_range:
enabled: true enabled: true
max_size_mb: 100 max_size_mb: 100
limits_config:
metric_aggregation_enabled: true
max_line_size: 256KB
max_line_size_truncate: false
allow_structured_metadata: true
schema_config: schema_config:
configs: configs:
- from: 2020-10-24 - from: 2020-10-24
@@ -54,15 +47,17 @@ schema_config:
index: index:
prefix: index_ prefix: index_
period: 24h period: 24h
row_shards: 16
limits_config:
reject_old_samples: true
reject_old_samples_max_age: 168h
allow_structured_metadata: true
max_line_size: 256KB
pattern_ingester: pattern_ingester:
enabled: true enabled: true
metric_aggregation: metric_aggregation:
loki_address: localhost:3100 loki_address: localhost:3100
ruler:
alertmanager_url: http://localhost:9093
frontend: frontend:
encoding: protobuf encoding: protobuf
.docker/observability/prometheus-data/.gitignore
-1
View File
@@ -1 +0,0 @@
*
.docker/observability/tempo-data/.gitignore
-1
View File
@@ -1 +0,0 @@
*
.docker/observability/tempo.yaml
+52 -8
View File
@@ -12,21 +12,44 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
partition_ring_live_store: true
stream_over_http_enabled: true
server: server:
http_listen_port: 3200 http_listen_port: 3200
log_level: info log_level: info
distributor: distributor:
ingester_write_path_enabled: false
kafka_write_path_enabled: true
receivers: receivers:
otlp: otlp:
protocols: protocols:
grpc: grpc:
endpoint: "0.0.0.0:4317" endpoint: "tempo:4317"
http: http:
endpoint: "0.0.0.0:4318" endpoint: "tempo:4318"
#log_received_spans:
# enabled: true
# log_discarded_spans:
# enabled: true
ingester: backend_scheduler:
max_block_duration: 5m # cut the headblock when this much time passes. this is being set for demo purposes and should probably be left alone normally provider:
compaction:
compaction:
block_retention: 1h
backend_worker:
backend_scheduler_addr: localhost:3200
compaction:
block_retention: 1h
ring:
kvstore:
store: memberlist
querier:
query_live_store: true
metrics_generator: metrics_generator:
registry: registry:
@@ -38,13 +61,34 @@ metrics_generator:
remote_write: remote_write:
- url: http://prometheus:9090/api/v1/write - url: http://prometheus:9090/api/v1/write
send_exemplars: true send_exemplars: true
traces_storage:
path: /var/tempo/generator/traces query_frontend:
rf1_after: "1999-01-01T00:00:00Z"
mcp_server:
enabled: true
storage: storage:
trace: trace:
backend: local # backend configuration to use backend: local
wal: wal:
path: /var/tempo/wal # where to store the wal locally path: /var/tempo/wal
local: local:
path: /var/tempo/blocks path: /var/tempo/blocks
overrides:
defaults:
metrics_generator:
processors: ["span-metrics", "service-graphs", "local-blocks"]
generate_native_histograms: both
ingest:
enabled: true
kafka:
address: redpanda:9092
topic: tempo-ingest
block_builder:
consume_cycle_duration: 30s
usage_report:
reporting_enabled: false
docker-compose.yml
+2 -4
View File
@@ -195,9 +195,7 @@ services:
command: command:
- "--config.file=/etc/prometheus/prometheus.yml" - "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus" - "--storage.tsdb.path=/prometheus"
- "--web.console.libraries=/usr/share/prometheus/console_libraries" - "--storage.tsdb.retention.time=30d"
- "--web.console.templates=/usr/share/prometheus/consoles"
- "--storage.tsdb.retention.time=200h"
- "--web.enable-lifecycle" - "--web.enable-lifecycle"
- "--web.enable-otlp-receiver" - "--web.enable-otlp-receiver"
- "--web.enable-remote-write-receiver" - "--web.enable-remote-write-receiver"
@@ -236,7 +234,7 @@ services:
volumes: volumes:
- grafana_data:/var/lib/grafana - grafana_data:/var/lib/grafana
- ./.docker/observability/grafana/provisioning:/etc/grafana/provisioning:ro - ./.docker/observability/grafana/provisioning:/etc/grafana/provisioning:ro
- ./.docker/observability/grafana/dashboards:/var/lib/grafana/dashboards:ro - ./.docker/observability/grafana/dashboards:/etc/grafana/dashboards:ro
networks: networks:
- rustfs-network - rustfs-network
restart: unless-stopped restart: unless-stopped