gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 16:28:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

merge main

Browse Source
This commit is contained in:
weisd
2025-04-14 22:38:58 +08:00
parent b9fb5e7e84 08fe5192fb
commit 38b7bbbc49
161 changed files with 12615 additions and 4296 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.docker/Dockerfile.devenv
+4 -4
View File
@@ -7,12 +7,12 @@ RUN sed -i s@http://.*archive.ubuntu.com@http://repo.huaweicloud.com@g /etc/apt/
RUN apt-get clean && apt-get update && apt-get install wget git curl unzip gcc pkg-config libssl-dev lld libdbus-1-dev libwayland-dev libwebkit2gtk-4.1-dev libxdo-dev -y
# install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v27.0/protoc-27.0-linux-x86_64.zip \
&& unzip protoc-27.0-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc && mv protoc3/include/* /usr/local/include/ && rm -rf protoc-27.0-linux-x86_64.zip protoc3
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v30.2/protoc-30.2-linux-x86_64.zip \
&& unzip protoc-30.2-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc && mv protoc3/include/* /usr/local/include/ && rm -rf protoc-30.2-linux-x86_64.zip protoc3
# install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v24.3.25/Linux.flatc.binary.g++-13.zip \
RUN wget https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc && rm -rf Linux.flatc.binary.g++-13.zip
.docker/Dockerfile.rockylinux9.3
+4 -4
View File
@@ -13,13 +13,13 @@ RUN dnf makecache
RUN yum install wget git unzip gcc openssl-devel pkgconf-pkg-config -y
# install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v27.0/protoc-27.0-linux-x86_64.zip \
&& unzip protoc-27.0-linux-x86_64.zip -d protoc3 \
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v30.2/protoc-30.2-linux-x86_64.zip \
&& unzip protoc-30.2-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc \
&& rm -rf protoc-27.0-linux-x86_64.zip protoc3
&& rm -rf protoc-30.2-linux-x86_64.zip protoc3
# install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v24.3.25/Linux.flatc.binary.g++-13.zip \
RUN wget https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc \
&& rm -rf Linux.flatc.binary.g++-13.zip
.docker/Dockerfile.ubuntu22.04
+4 -4
View File
@@ -7,12 +7,12 @@ RUN sed -i s@http://.*archive.ubuntu.com@http://repo.huaweicloud.com@g /etc/apt/
RUN apt-get clean && apt-get update && apt-get install wget git curl unzip gcc pkg-config libssl-dev lld libdbus-1-dev libwayland-dev libwebkit2gtk-4.1-dev libxdo-dev -y
# install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v27.0/protoc-27.0-linux-x86_64.zip \
&& unzip protoc-27.0-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc && mv protoc3/include/* /usr/local/include/ && rm -rf protoc-27.0-linux-x86_64.zip protoc3
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v30.2/protoc-30.2-linux-x86_64.zip \
&& unzip protoc-30.2-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc && mv protoc3/include/* /usr/local/include/ && rm -rf protoc-30.2-linux-x86_64.zip protoc3
# install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v24.3.25/Linux.flatc.binary.g++-13.zip \
RUN wget https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc && rm -rf Linux.flatc.binary.g++-13.zip
.docker/observability/README.md
+101
View File
@@ -0,0 +1,101 @@
# Observability
This directory contains the observability stack for the application. The stack is composed of the following components:
- Prometheus v3.2.1
- Grafana 11.6.0
- Loki 3.4.2
- Jaeger 2.4.0
- Otel Collector 0.120.0 #0.121.0 remove loki
## Prometheus
Prometheus is a monitoring and alerting toolkit. It scrapes metrics from instrumented jobs, either directly or via an
intermediary push gateway for short-lived jobs. It stores all scraped samples locally and runs rules over this data to
either aggregate and record new time series from existing data or generate alerts. Grafana or other API consumers can be
used to visualize the collected data.
## Grafana
Grafana is a multi-platform open-source analytics and interactive visualization web application. It provides charts,
graphs, and alerts for the web when connected to supported data sources.
## Loki
Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is
designed to be very cost-effective and easy to operate. It does not index the contents of the logs, but rather a set of
labels for each log stream.
## Jaeger
Jaeger is a distributed tracing system released as open source by Uber Technologies. It is used for monitoring and
troubleshooting microservices-based distributed systems, including:
- Distributed context propagation
- Distributed transaction monitoring
- Root cause analysis
- Service dependency analysis
- Performance / latency optimization
## Otel Collector
The OpenTelemetry Collector offers a vendor-agnostic implementation on how to receive, process, and export telemetry
data. It removes the need to run, operate, and maintain multiple agents/collectors in order to support open-source
observability data formats (e.g. Jaeger, Prometheus, etc.) sending to one or more open-source or commercial back-ends.
## How to use
To deploy the observability stack, run the following command:
```bash
docker-compose -f docker-compose.yml -f docker-compose.override.yml up -d
```
To access the Grafana dashboard, navigate to `http://localhost:3000` in your browser. The default username and password
are `admin` and `admin`, respectively.
To access the Jaeger dashboard, navigate to `http://localhost:16686` in your browser.
To access the Prometheus dashboard, navigate to `http://localhost:9090` in your browser.
## How to stop
To stop the observability stack, run the following command:
```bash
docker-compose -f docker-compose.yml -f docker-compose.override.yml down
```
## How to remove data
To remove the data generated by the observability stack, run the following command:
```bash
docker-compose -f docker-compose.yml -f docker-compose.override.yml down -v
```
## How to configure
To configure the observability stack, modify the `docker-compose.override.yml` file. The file contains the following
```yaml
services:
prometheus:
environment:
- PROMETHEUS_CONFIG_FILE=/etc/prometheus/prometheus.yml
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
grafana:
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
volumes:
- ./grafana/provisioning:/etc/grafana/provisioning
```
The `prometheus` service mounts the `prometheus.yml` file to `/etc/prometheus/prometheus.yml`. The `grafana` service
mounts the `grafana/provisioning` directory to `/etc/grafana/provisioning`. You can modify these files to configure the
observability stack.
.docker/observability/config/obs.toml
+33
View File
@@ -0,0 +1,33 @@
[observability]
endpoint = "http://localhost:4317" # Default is "http://localhost:4317" if not specified
use_stdout = false # Output with stdout, true output, false no output
sample_ratio = 2.0
meter_interval = 30
service_name = "rustfs"
service_version = "0.1.0"
environments = "production"
logger_level = "info"
[sinks]
[sinks.kafka] # Kafka sink is disabled by default
enabled = false
bootstrap_servers = "localhost:9092"
topic = "logs"
batch_size = 100 # Default is 100 if not specified
batch_timeout_ms = 1000 # Default is 1000ms if not specified
[sinks.webhook]
enabled = false
endpoint = "http://localhost:8080/webhook"
auth_token = ""
batch_size = 100 # Default is 3 if not specified
batch_timeout_ms = 1000 # Default is 100ms if not specified
[sinks.file]
enabled = true
path = "/root/data/logs/app.log"
batch_size = 10
batch_timeout_ms = 1000 # Default is 8192 bytes if not specified
[logger]
queue_capacity = 10
.docker/observability/docker-compose.yml
+65
View File
@@ -0,0 +1,65 @@
services:
otel-collector:
image: otel/opentelemetry-collector-contrib:0.120.0
environment:
- TZ=Asia/Shanghai
volumes:
- ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
ports:
- 1888:1888
- 8888:8888
- 8889:8889
- 13133:13133
- 4317:4317
- 4318:4318
- 55679:55679
networks:
- otel-network
jaeger:
image: jaegertracing/jaeger:2.4.0
environment:
- TZ=Asia/Shanghai
ports:
- "16686:16686"
- "14317:4317"
- "14318:4318"
networks:
- otel-network
prometheus:
image: prom/prometheus:v3.2.1
environment:
- TZ=Asia/Shanghai
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
ports:
- "9090:9090"
networks:
- otel-network
loki:
image: grafana/loki:3.4.2
environment:
- TZ=Asia/Shanghai
volumes:
- ./loki-config.yaml:/etc/loki/local-config.yaml
ports:
- "3100:3100"
command: -config.file=/etc/loki/local-config.yaml
networks:
- otel-network
grafana:
image: grafana/grafana:11.6.0
ports:
- "3000:3000" # Web UI
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
- TZ=Asia/Shanghai
networks:
- otel-network
networks:
otel-network:
driver: bridge
name: "network_otel_config"
driver_opts:
com.docker.network.enable_ipv6: "true"
.docker/observability/jaeger-config.yaml
+87
View File
@@ -0,0 +1,87 @@
service:
extensions: [ jaeger_storage, jaeger_query, remote_sampling, healthcheckv2 ]
pipelines:
traces:
receivers: [ otlp, jaeger, zipkin ]
processors: [ batch, adaptive_sampling ]
exporters: [ jaeger_storage_exporter ]
telemetry:
resource:
service.name: jaeger
metrics:
level: detailed
readers:
- pull:
exporter:
prometheus:
host: 0.0.0.0
port: 8888
logs:
level: debug
# TODO Initialize telemetry tracer once OTEL released new feature.
# https://github.com/open-telemetry/opentelemetry-collector/issues/10663
extensions:
healthcheckv2:
use_v2: true
http:
# pprof:
# endpoint: 0.0.0.0:1777
# zpages:
# endpoint: 0.0.0.0:55679
jaeger_query:
storage:
traces: some_store
traces_archive: another_store
ui:
config_file: ./cmd/jaeger/config-ui.json
# The maximum duration that is considered for clock skew adjustments.
# Defaults to 0 seconds, which means it's disabled.
max_clock_skew_adjust: 0s
jaeger_storage:
backends:
some_store:
memory:
max_traces: 100000
another_store:
memory:
max_traces: 100000
remote_sampling:
# You can either use file or adaptive sampling strategy in remote_sampling
# file:
# path: ./cmd/jaeger/sampling-strategies.json
adaptive:
sampling_store: some_store
initial_sampling_probability: 0.1
http:
grpc:
receivers:
otlp:
protocols:
grpc:
http:
jaeger:
protocols:
grpc:
thrift_binary:
thrift_compact:
thrift_http:
zipkin:
processors:
batch:
# Adaptive Sampling Processor is required to support adaptive sampling.
# It expects remote_sampling extension with `adaptive:` config to be enabled.
adaptive_sampling:
exporters:
jaeger_storage_exporter:
trace_storage: some_store
.docker/observability/loki-config.yaml
+63
View File
@@ -0,0 +1,63 @@
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9096
log_level: debug
grpc_server_max_concurrent_streams: 1000
common:
instance_addr: 127.0.0.1
path_prefix: /tmp/loki
storage:
filesystem:
chunks_directory: /tmp/loki/chunks
rules_directory: /tmp/loki/rules
replication_factor: 1
ring:
kvstore:
store: inmemory
query_range:
results_cache:
cache:
embedded_cache:
enabled: true
max_size_mb: 100
limits_config:
metric_aggregation_enabled: true
schema_config:
configs:
- from: 2020-10-24
store: tsdb
object_store: filesystem
schema: v13
index:
prefix: index_
period: 24h
pattern_ingester:
enabled: true
metric_aggregation:
loki_address: localhost:3100
ruler:
alertmanager_url: http://localhost:9093
frontend:
encoding: protobuf
# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
# Statistics help us better understand how Loki is used, and they show us performance
# levels for most users. This helps us prioritize features and documentation.
# For more information on what's sent, look at
# https://github.com/grafana/loki/blob/main/pkg/analytics/stats.go
# Refer to the buildReport method to see what goes into a report.
#
# If you would like to disable reporting, uncomment the following lines:
#analytics:
# reporting_enabled: false
.docker/observability/otel-collector-config.yaml
+57
View File
@@ -0,0 +1,57 @@
receivers:
otlp:
protocols:
grpc: # OTLP gRPC 接收器
endpoint: 0.0.0.0:4317
http: # OTLP HTTP 接收器
endpoint: 0.0.0.0:4318
processors:
batch: # 批处理处理器，提升吞吐量
timeout: 5s
send_batch_size: 1000
memory_limiter:
check_interval: 1s
limit_mib: 512
exporters:
otlp/traces: # OTLP 导出器，用于跟踪数据
endpoint: "jaeger:4317" # Jaeger 的 OTLP gRPC 端点
tls:
insecure: true # 开发环境禁用 TLS，生产环境需配置证书
prometheus: # Prometheus 导出器，用于指标数据
endpoint: "0.0.0.0:8889" # Prometheus 刮取端点
namespace: "rustfs" # 指标前缀
send_timestamps: true # 发送时间戳
# enable_open_metrics: true
loki: # Loki 导出器，用于日志数据
# endpoint: "http://loki:3100/otlp/v1/logs"
endpoint: "http://loki:3100/loki/api/v1/push"
tls:
insecure: true
extensions:
health_check:
pprof:
zpages:
service:
extensions: [ health_check, pprof, zpages ] # 启用扩展
pipelines:
traces:
receivers: [ otlp ]
processors: [ memory_limiter,batch ]
exporters: [ otlp/traces ]
metrics:
receivers: [ otlp ]
processors: [ batch ]
exporters: [ prometheus ]
logs:
receivers: [ otlp ]
processors: [ batch ]
exporters: [ loki ]
telemetry:
logs:
level: "info" # Collector 日志级别
metrics:
address: "0.0.0.0:8888" # Collector 自身指标暴露
.docker/observability/prometheus.yml
+11
View File
@@ -0,0 +1,11 @@
global:
scrape_interval: 5s # 刮取间隔
scrape_configs:
- job_name: 'otel-collector'
static_configs:
- targets: ['otel-collector:8888'] # 从 Collector 刮取指标
- job_name: 'otel-metrics'
static_configs:
- targets: ['otel-collector:8889'] # 应用指标
.github/actions/setup/action.yml
+2 -7
View File
@@ -20,16 +20,11 @@ runs:
shell: bash
run: |
sudo apt update
sudo apt install -y \
lld \
libdbus-1-dev \
libwayland-dev \
libwebkit2gtk-4.1-dev \
libxdo-dev
sudo apt install -y musl-tools build-essential lld libdbus-1-dev libwayland-dev libwebkit2gtk-4.1-dev libxdo-dev
- uses: arduino/setup-protoc@v3
with:
version: "27.0"
version: "30.2"
- uses: Nugine/setup-flatc@v1
with:
.github/workflows/build.yml
+47 -24
View File
@@ -3,11 +3,11 @@ name: Build
on:
workflow_dispatch:
schedule:
- cron: '0 0 * * 0' # at midnight of each sunday
- cron: "0 0 * * 0" # at midnight of each sunday
push:
branches:
- main
tags: [ 'v*', '*' ]
tags: [ "v*", "*" ]
jobs:
build-rustfs:
@@ -16,9 +16,10 @@ jobs:
strategy:
matrix:
variant:
- { profile: dev, target: x86_64-unknown-linux-gnu, glibc: "default" }
# - { profile: dev, target: x86_64-unknown-linux-musl, glibc: "default" }
- { profile: release, target: x86_64-unknown-linux-musl, glibc: "default" }
- { profile: release, target: x86_64-unknown-linux-gnu, glibc: "default" }
- { profile: release, target: x86_64-unknown-linux-gnu, glibc: "2.31" }
# - { profile: release, target: x86_64-unknown-linux-gnu, glibc: "2.31" }
steps:
- uses: actions/checkout@v4
@@ -51,13 +52,13 @@ jobs:
ARTIFACT_NAME="${ARTIFACT_NAME}-glibc${{ matrix.variant.glibc }}"
fi
echo "artifact_name=${ARTIFACT_NAME}" >> $GITHUB_OUTPUT
# Determine binary path
bin_path="target/artifacts/rustfs.${{ matrix.variant.profile }}.${{ matrix.variant.target }}.bin"
if [ -f "target/artifacts/rustfs.${{ matrix.variant.profile }}.${{ matrix.variant.target }}.glibc${{ matrix.variant.glibc }}.bin" ]; then
bin_path="target/artifacts/rustfs.${{ matrix.variant.profile }}.${{ matrix.variant.target }}.glibc${{ matrix.variant.glibc }}.bin"
fi
# Create package
mkdir -p ${ARTIFACT_NAME}
cp "$bin_path" ${ARTIFACT_NAME}/rustfs
@@ -69,6 +70,16 @@ jobs:
name: ${{ steps.package.outputs.artifact_name }}
path: ${{ steps.package.outputs.artifact_name }}.zip
retention-days: 7
- name: Upload to Aliyun OSS
uses: JohnGuan/oss-upload-action@main
with:
key-id: ${{ secrets.ALICLOUDOSS_KEY_ID }}
key-secret: ${{ secrets.ALICLOUDOSS_KEY_SECRET }}
region: oss-cn-beijing
bucket: rustfs-artifacts
assets: |
${{ steps.package.outputs.artifact_name }}.zip:/artifacts/rustfs/${{ steps.package.outputs.artifact_name }}.zip
${{ steps.package.outputs.artifact_name }}.zip:/artifacts/rustfs/${{ steps.package.outputs.artifact_name }}.latest.zip
build-rustfs-gui:
runs-on: ubuntu-latest
@@ -88,32 +99,34 @@ jobs:
name: "rustfs-${{ matrix.variant.profile }}-${{ matrix.variant.target }}"
- name: Display structure of downloaded files
run: |
ls -R
ls -R
unzip -o -j "rustfs-${{ matrix.variant.profile }}-${{ matrix.variant.target }}.zip" -d ./cli/rustfs-gui/embedded-rustfs/
ls -la cli/rustfs-gui/embedded-rustfs
- name: Cache dioxus-cli
uses: actions/cache@v4
# - name: Cache dioxus-cli
# uses: actions/cache@v4
# with:
# path: ~/.cargo/bin/dx
# key: ${{ runner.os }}-dioxus-cli-${{ hashFiles('**/Cargo.lock') }}
# restore-keys: |
# ${{ runner.os }}-dioxus-cli-
#
# - name: Install dioxus-cli
# run: |
# if [ ! -f ~/.cargo/bin/dx ]; then
# cargo install dioxus-cli
# fi
- uses: taiki-e/cache-cargo-install-action@v2
with:
path: ~/.cargo/bin/dx
key: ${{ runner.os }}-dioxus-cli-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-dioxus-cli-
- name: Install dioxus-cli
run: |
if [ ! -f ~/.cargo/bin/dx ]; then
cargo install dioxus-cli
fi
tool: dioxus-cli
- name: Build and Bundle rustfs-gui
run: |
ls -la
release_path="target/${{ matrix.variant.target }}"
mkdir -p ${release_path}
cd cli/rustfs-gui
ls -la embedded-rustfs
# Configure the linker based on the target
case "${{ matrix.target }}" in
"x86_64-unknown-linux-gnu")
@@ -140,7 +153,7 @@ jobs:
# Validating Environment Variables (for Debugging)
echo "CC for ${{ matrix.target }}: $CC_${{ matrix.target }}"
echo "Linker for ${{ matrix.target }}: $CARGO_TARGET_${{ matrix.target }}_LINKER"
if [[ "${{ matrix.variant.target }}" == *"apple-darwin"* ]]; then
dx bundle --platform macos --package-types "macos" --package-types "dmg" --package-types "ios" --release --profile release --out-dir ../../${release_path}
elif [[ "${{ matrix.variant.target }}" == *"windows-msvc"* ]]; then
@@ -159,6 +172,16 @@ jobs:
name: ${{ steps.package.outputs.gui_artifact_name }}
path: ${{ steps.package.outputs.gui_artifact_name }}.zip
retention-days: 7
- name: Upload to Aliyun OSS
uses: JohnGuan/oss-upload-action@main
with:
key-id: ${{ secrets.ALICLOUDOSS_KEY_ID }}
key-secret: ${{ secrets.ALICLOUDOSS_KEY_SECRET }}
region: oss-cn-beijing
bucket: rustfs-artifacts
assets: |
${{ steps.package.outputs.gui_artifact_name }}.zip:/artifacts/rustfs/${{ steps.package.outputs.gui_artifact_name }}.zip
${{ steps.package.outputs.gui_artifact_name }}.zip:/artifacts/rustfs/${{ steps.package.outputs.gui_artifact_name }}.latest.zip
merge:
runs-on: ubuntu-latest
@@ -167,5 +190,5 @@ jobs:
- uses: actions/upload-artifact/merge@v4
with:
name: rustfs-packages
pattern: 'rustfs-*'
pattern: "rustfs-*"
delete-merged: true
.gitignore
+4 -1
View File
@@ -8,4 +8,7 @@
.devcontainer
rustfs/static/*
vendor
cli/rustfs-gui/embedded-rustfs/rustfs
cli/rustfs-gui/embedded-rustfs/rustfs
deploy/config/obs.toml
*.log
deploy/certs/*
Cargo.lock
Generated
+2738 -554
View File
File diff suppressed because it is too large Load Diff
Cargo.toml
+66 -38
View File
@@ -11,6 +11,10 @@ members = [
"iam", # Identity and Access Management
"crypto", # Cryptography and security features
"cli/rustfs-gui", # Graphical user interface client
"crates/obs", # Observability utilities
"s3select/api",
"s3select/query",
"appauth",
]
resolver = "2"
@@ -29,12 +33,21 @@ all = "warn"
[workspace.dependencies]
madmin = { path = "./madmin" }
async-trait = "0.1.86"
backon = "1.3.0"
bytes = "1.9.0"
bytesize = "1.3.0"
atoi = "2.0.0"
async-recursion = "1.1.1"
async-trait = "0.1.88"
atomic_enum = "0.3.0"
axum = "0.8.3"
axum-extra = "0.10.1"
axum-server = { version = "0.7.2", features = ["tls-rustls"] }
backon = "1.5.0"
bytes = "1.10.1"
bytesize = "1.3.3"
chrono = { version = "0.4.40", features = ["serde"] }
clap = { version = "4.5.31", features = ["derive", "env"] }
clap = { version = "4.5.35", features = ["derive", "env"] }
config = "0.15.11"
datafusion = "46.0.0"
derive_builder = "0.20.2"
dioxus = { version = "0.6.3", features = ["router"] }
dirs = "6.0.0"
ecstore = { path = "./ecstore" }
@@ -42,78 +55,93 @@ flatbuffers = "24.12.23"
futures = "0.3.31"
futures-util = "0.3.31"
common = { path = "./common/common" }
policy = {path = "./policy"}
policy = { path = "./policy" }
hex = "0.4.3"
hyper = "1.6.0"
hyper-util = { version = "0.1.10", features = [
hyper-util = { version = "0.1.11", features = [
"tokio",
"server-auto",
"server-graceful",
] }
http = "1.2.0"
http = "1.3.1"
http-body = "1.0.1"
humantime = "2.1.0"
keyring = { version = "3.6.1", features = ["apple-native", "windows-native", "sync-secret-service"] }
humantime = "2.2.0"
jsonwebtoken = "9.3.1"
keyring = { version = "3.6.2", features = ["apple-native", "windows-native", "sync-secret-service"] }
lock = { path = "./common/lock" }
lazy_static = "1.5.0"
libsystemd = { version = "0.7" }
local-ip-address = "0.6.3"
matchit = "0.8.4"
md-5 = "0.10.6"
mime = "0.3.17"
netif = "0.1.6"
opentelemetry = { version = "0.29.1" }
opentelemetry-appender-tracing = { version = "0.29.1", features = ["experimental_use_tracing_span_context", "experimental_metadata_attributes"] }
opentelemetry_sdk = { version = "0.29" }
opentelemetry-stdout = { version = "0.29.0" }
opentelemetry-otlp = { version = "0.29" }
opentelemetry-prometheus = { version = "0.29.1" }
opentelemetry-semantic-conventions = { version = "0.29.0", features = ["semconv_experimental"] }
pin-project-lite = "0.2"
prometheus = "0.14.0"
# pin-utils = "0.1.0"
prost = "0.13.4"
prost-build = "0.13.4"
prost-types = "0.13.4"
prost = "0.13.5"
prost-build = "0.13.5"
prost-types = "0.13.5"
protobuf = "3.7"
protos = { path = "./common/protos" }
rand = "0.8.5"
reqwest = { version = "0.12.12", default-features = false, features = ["rustls-tls", "charset", "http2", "macos-system-configuration", "stream","blocking"] }
rfd = { version = "0.15.2", default-features = false, features = ["xdg-portal", "tokio"] }
rdkafka = { version = "0.37", features = ["tokio"] }
reqwest = { version = "0.12.15", default-features = false, features = ["rustls-tls", "charset", "http2", "macos-system-configuration", "stream", "json", "blocking"] }
rfd = { version = "0.15.3", default-features = false, features = ["xdg-portal", "tokio"] }
rmp = "0.8.14"
rmp-serde = "1.3.0"
rust-embed = "8.6.0"
s3s = { git = "https://github.com/Nugine/s3s.git", rev = "ab139f72fe768fb9d8cecfe36269451da1ca9779", default-features = true, features = [
"tower",
] }
s3s-policy = { git = "https://github.com/Nugine/s3s.git", rev = "ab139f72fe768fb9d8cecfe36269451da1ca9779" }
shadow-rs = { version = "0.38.0", default-features = false }
serde = { version = "1.0.217", features = ["derive"] }
serde_json = "1.0.138"
rustfs-obs = { path = "crates/obs", version = "0.0.1" }
rust-embed = "8.7.0"
rustls = { version = "0.23.26" }
rustls-pki-types = "1.11.0"
rustls-pemfile = "2.2.0"
s3s = { git = "https://github.com/Nugine/s3s.git", rev = "3ad13ace7af703c3c8afc99cf19f4c18c82603a3" }
s3s-policy = { git = "https://github.com/Nugine/s3s.git", rev = "3ad13ace7af703c3c8afc99cf19f4c18c82603a3" }
shadow-rs = { version = "0.38.1", default-features = false }
serde = { version = "1.0.219", features = ["derive"] }
serde_json = "1.0.140"
serde_urlencoded = "0.7.1"
sha2 = "0.10.8"
tempfile = "3.16.0"
thiserror = "2.0.11"
time = { version = "0.3.37", features = [
snafu = "0.8.5"
tempfile = "3.19.1"
test-case = "3.3.1"
thiserror = "2.0.12"
time = { version = "0.3.41", features = [
"std",
"parsing",
"formatting",
"macros",
"serde",
] }
tokio = { version = "1.43.0", features = ["fs", "rt-multi-thread"] }
tonic = { version = "0.12.3", features = ["gzip"] }
tonic-build = "0.12.3"
tonic-reflection = "0.12"
tokio = { version = "1.44.2", features = ["fs", "rt-multi-thread"] }
tonic = { version = "0.13.0", features = ["gzip"] }
tonic-build = "0.13.0"
tokio-rustls = { version = "0.26", default-features = false }
tokio-stream = "0.1.17"
tokio-util = { version = "0.7.14", features = ["io", "compat"] }
tower = { version = "0.5.2", features = ["timeout"] }
tower-http = { version = "0.6.2", features = ["cors"] }
tracing = "0.1.41"
tracing-core = "0.1.33"
tracing-error = "0.2.1"
tracing-subscriber = { version = "0.3.19", features = ["env-filter", "time"] }
tracing-appender = "0.2.3"
tracing-opentelemetry = "0.30"
transform-stream = "0.3.1"
url = "2.5.4"
uuid = { version = "1.15.1", features = [
uuid = { version = "1.16.0", features = [
"v4",
"fast-rng",
"macro-diagnostics",
] }
log = "0.4.25"
axum = "0.7.9"
md-5 = "0.10.6"
workers = { path = "./common/workers" }
test-case = "3.3.1"
zip = "2.2.3"
[profile.wasm-dev]
inherits = "dev"
Dockerfile.obs
+17
View File
@@ -0,0 +1,17 @@
FROM alpine:latest
# RUN apk add --no-cache <package-name>
# 如果 rustfs 有依赖，可以在这里添加，例如：
# RUN apk add --no-cache openssl
# RUN apk add --no-cache bash # 安装 Bash
WORKDIR /app
COPY ./target/x86_64-unknown-linux-musl/release/rustfs /app/rustfs
RUN chmod +x /app/rustfs
EXPOSE 9000
EXPOSE 9002
CMD ["/app/rustfs"]
README.md
+80 -12
View File
@@ -1,30 +1,32 @@
# How to compile RustFS
| Must package | Version |
|--------------|---------|
| Rust | 1.8.5 |
| protoc | 27.0 |
| flatc | 24.0+ |
| Must package | Version | download link |
|--------------|---------|----------------------------------------------------------------------------------------------------------------------------------|
| Rust | 1.8.5 | https://www.rust-lang.org/tools/install |
| protoc | 30.2 | [protoc-30.2-linux-x86_64.zip](https://github.com/protocolbuffers/protobuf/releases/download/v30.2/protoc-30.2-linux-x86_64.zip) |
| flatc | 24.0+ | [Linux.flatc.binary.g++-13.zip](https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip) |
Download Links:
https://github.com/google/flatbuffers/releases/download/v24.3.25/Linux.flatc.binary.g++-13.zip
https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip
https://github.com/protocolbuffers/protobuf/releases/download/v27.0/protoc-27.0-linux-x86_64.zip
https://github.com/protocolbuffers/protobuf/releases/download/v30.2/protoc-30.2-linux-x86_64.zip
Or use Docker:
```yml
- uses: arduino/setup-protoc@v3
with:
version: "27.0"
version: "30.2"
- uses: Nugine/setup-flatc@v1
with:
version: "24.3.25"
version: "25.2.10"
```
# How to add Console web
1. wget [http://dl.rustfs.com/console/console.latest.tar.gz](https://dl.rustfs.com/console/rustfs-console-latest.zip)
1. `wget https://dl.rustfs.com/artifacts/console/rustfs-console-latest.zip`
2. mkdir in this repos folder `./rustfs/static`
@@ -32,14 +34,34 @@ Or use Docker:
# Star RustFS
Add Env infomation:
Add Env Information:
```
export RUST_LOG="rustfs=debug,ecstore=debug,s3s=debug,iam=debug"
export RUSTFS_VOLUMES="./target/volume/test"
export RUSTFS_ADDRESS="0.0.0.0:9000"
export RUSTFS_CONSOLE_ENABLE=true
export RUSTFS_CONSOLE_ADDRESS="0.0.0.0:9001"
# 具体路径修改为配置文件真实路径，obs.example.toml 仅供参考 其中`RUSTFS_OBS_CONFIG` 和下面变量二选一
export RUSTFS_OBS_CONFIG="./deploy/config/obs.example.toml"
# 如下变量需要必须参数都有值才可以，以及会覆盖配置文件`obs.example.toml`中的值
export RUSTFS__OBSERVABILITY__ENDPOINT=http://localhost:4317
export RUSTFS__OBSERVABILITY__USE_STDOUT=true
export RUSTFS__OBSERVABILITY__SAMPLE_RATIO=2.0
export RUSTFS__OBSERVABILITY__METER_INTERVAL=30
export RUSTFS__OBSERVABILITY__SERVICE_NAME=rustfs
export RUSTFS__OBSERVABILITY__SERVICE_VERSION=0.1.0
export RUSTFS__OBSERVABILITY__ENVIRONMENT=develop
export RUSTFS__OBSERVABILITY__LOGGER_LEVEL=info
export RUSTFS__SINKS__FILE__ENABLED=true
export RUSTFS__SINKS__FILE__PATH="./deploy/logs/rustfs.log"
export RUSTFS__SINKS__WEBHOOK__ENABLED=false
export RUSTFS__SINKS__WEBHOOK__ENDPOINT=""
export RUSTFS__SINKS__WEBHOOK__AUTH_TOKEN=""
export RUSTFS__SINKS__KAFKA__ENABLED=false
export RUSTFS__SINKS__KAFKA__BOOTSTRAP_SERVERS=""
export RUSTFS__SINKS__KAFKA__TOPIC=""
export RUSTFS__LOGGER__QUEUE_CAPACITY=10
```
You need replace your real data folder:
@@ -47,3 +69,49 @@ You need replace your real data folder:
```
./rustfs /data/rustfs
```
## How to deploy the observability stack
The OpenTelemetry Collector offers a vendor-agnostic implementation on how to receive, process, and export telemetry
data. It removes the need to run, operate, and maintain multiple agents/collectors in order to support open-source
observability data formats (e.g. Jaeger, Prometheus, etc.) sending to one or more open-source or commercial back-ends.
1. Enter the `.docker/observability` directory,
2. Run the following command:
```bash
docker-compose -f docker-compose.yml up -d
```
3. Access the Grafana dashboard by navigating to `http://localhost:3000` in your browser. The default username and
password are `admin` and `admin`, respectively.
4. Access the Jaeger dashboard by navigating to `http://localhost:16686` in your browser.
5. Access the Prometheus dashboard by navigating to `http://localhost:9090` in your browser.
## Create a new Observability configuration file
#### 1. Enter the `deploy/config` directory,
#### 2. Copy `obs.toml.example` to `obs.toml`
#### 3. Modify the `obs.toml` configuration file
##### 3.1. Modify the `endpoint` value to the address of the OpenTelemetry Collector
##### 3.2. Modify the `service_name` value to the name of the service
##### 3.3. Modify the `service_version` value to the version of the service
##### 3.4. Modify the `environment` value to the environment of the service
##### 3.5. Modify the `meter_interval` value to export interval
##### 3.6. Modify the `sample_ratio` value to the sample ratio
##### 3.7. Modify the `use_stdout` value to export to stdout
##### 3.8. Modify the `logger_level` value to the logger level
appauth/Cargo.toml
+19
View File
@@ -0,0 +1,19 @@
[package]
name = "appauth"
edition.workspace = true
license.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true
[dependencies]
base64-simd = "0.8.0"
common.workspace = true
hex-simd = "0.8.0"
rand.workspace = true
rsa = "0.9.8"
serde.workspace = true
serde_json.workspace = true
[lints]
workspace = true
appauth/src/lib.rs
+1
View File
@@ -0,0 +1 @@
pub mod token;
appauth/src/token.rs
+110
View File
@@ -0,0 +1,110 @@
use common::error::Result;
use rsa::Pkcs1v15Encrypt;
use rsa::{
pkcs8::{DecodePrivateKey, DecodePublicKey},
RsaPrivateKey, RsaPublicKey,
};
use serde::{Deserialize, Serialize};
#[derive(Serialize, Deserialize, Debug, Default, Clone)]
pub struct Token {
pub name: String, // 应用ID
pub expired: u64, // 到期时间 (UNIX时间戳)
}
// 公钥生成Token
// [token] Token对象
// [key] 公钥字符串
// 返回base64处理的加密字符串
pub fn gencode(token: &Token, key: &str) -> Result<String> {
let data = serde_json::to_vec(token)?;
let public_key = RsaPublicKey::from_public_key_pem(key)?;
let encrypted_data = public_key.encrypt(&mut rand::thread_rng(), Pkcs1v15Encrypt, &data)?;
Ok(base64_simd::URL_SAFE_NO_PAD.encode_to_string(&encrypted_data))
}
// 私钥解析Token
// [token] base64处理的加密字符串
// [key] 私钥字符串
// 返回Token对象
pub fn parse(token: &str, key: &str) -> Result<Token> {
let encrypted_data = base64_simd::URL_SAFE_NO_PAD.decode_to_vec(token.as_bytes())?;
let private_key = RsaPrivateKey::from_pkcs8_pem(key)?;
let decrypted_data = private_key.decrypt(Pkcs1v15Encrypt, &encrypted_data)?;
let res: Token = serde_json::from_slice(&decrypted_data)?;
Ok(res)
}
pub fn parse_license(license: &str) -> Result<Token> {
parse(license, TEST_PRIVATE_KEY)
// match parse(license, TEST_PRIVATE_KEY) {
// Ok(token) => {
// if token.expired > SystemTime::now().duration_since(UNIX_EPOCH)?.as_secs() {
// Ok(token)
// } else {
// Err("Token expired".into())
// }
// }
// Err(e) => Err(e),
// }
}
static TEST_PRIVATE_KEY:&str ="-----BEGIN PRIVATE KEY-----\nMIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCj86SrJIuxSxR6\nBJ/dlJEUIj6NeBRnhLQlCDdovuz61+7kJXVcxaR66w4m8W7SLEUP+IlPtnn6vmiG\n7XMhGNHIr7r1JsEVVLhZmL3tKI66DEZl786ZhG81BWqUlmcooIPS8UEPZNqJXLuz\nVGhxNyVGbj/tV7QC2pSISnKaixc+nrhxvo7w56p5qrm9tik0PjTgfZsUePkoBsSN\npoRkAauS14MAzK6HGB75CzG3dZqXUNWSWVocoWtQbZUwFGXyzU01ammsHQDvc2xu\nK1RQpd1qYH5bOWZ0N0aPFwT0r59HztFXg9sbjsnuhO1A7OiUOkc6iGVuJ0wm/9nA\nwZIBqzgjAgMBAAECggEAPMpeSEbotPhNw2BrllE76ec4omPfzPJbiU+em+wPGoNu\nRJHPDnMKJbl6Kd5jZPKdOOrCnxfd6qcnQsBQa/kz7+GYxMV12l7ra+1Cnujm4v0i\nLTHZvPpp8ZLsjeOmpF3AAzsJEJgon74OqtOlVjVIUPEYKvzV9ijt4gsYq0zfdYv0\nhrTMzyrGM4/UvKLsFIBROAfCeWfA7sXLGH8JhrRAyDrtCPzGtyyAmzoHKHtHafcB\nuyPFw/IP8otAgpDk5iiQPNkH0WwzAQIm12oHuNUa66NwUK4WEjXTnDg8KeWLHHNv\nIfN8vdbZchMUpMIvvkr7is315d8f2cHCB5gEO+GWAQKBgQDR/0xNll+FYaiUKCPZ\nvkOCAd3l5mRhsqnjPQ/6Ul1lAyYWpoJSFMrGGn/WKTa/FVFJRTGbBjwP+Mx10bfb\ngUg2GILDTISUh54fp4zngvTi9w4MWGKXrb7I1jPkM3vbJfC/v2fraQ/r7qHPpO2L\nf6ZbGxasIlSvr37KeGoelwcAQQKBgQDH3hmOTS2Hl6D4EXdq5meHKrfeoicGN7m8\noQK7u8iwn1R9zK5nh6IXxBhKYNXNwdCQtBZVRvFjjZ56SZJb7lKqa1BcTsgJfZCy\nnI3Uu4UykrECAH8AVCVqBXUDJmeA2yE+gDAtYEjvhSDHpUfWxoGHr0B/Oqk2Lxc/\npRy1qV5fYwKBgBWSL/hYVf+RhIuTg/s9/BlCr9SJ0g3nGGRrRVTlWQqjRCpXeFOO\nJzYqSq9pFGKUggEQxoOyJEFPwVDo9gXqRcyov+Xn2kaXl7qQr3yoixc1YZALFDWY\nd1ySBEqQr0xXnV9U/gvEgwotPRnjSzNlLWV2ZuHPtPtG/7M0o1H5GZMBAoGAKr3N\nW0gX53o+my4pCnxRQW+aOIsWq1a5aqRIEFudFGBOUkS2Oz+fI1P1GdrRfhnnfzpz\n2DK+plp/vIkFOpGhrf4bBlJ2psjqa7fdANRFLMaAAfyXLDvScHTQTCcnVUAHQPVq\n2BlSH56pnugyj7SNuLV6pnql+wdhAmRN2m9o1h8CgYAbX2juSr4ioXwnYjOUdrIY\n4+ERvHcXdjoJmmPcAm4y5NbSqLXyU0FQmplNMt2A5LlniWVJ9KNdjAQUt60FZw/+\nr76LdxXaHNZghyx0BOs7mtq5unSQXamZ8KixasfhE9uz3ij1jXjG6hafWkS8/68I\nuWbaZqgvy7a9oPHYlKH7Jg==\n-----END PRIVATE KEY-----\n";
#[cfg(test)]
mod tests {
use super::*;
use rsa::{
pkcs8::{EncodePrivateKey, EncodePublicKey, LineEnding},
RsaPrivateKey,
};
use std::time::{SystemTime, UNIX_EPOCH};
#[test]
fn test_gencode_and_parse() {
let mut rng = rand::thread_rng();
let bits = 2048;
let private_key = RsaPrivateKey::new(&mut rng, bits).expect("Failed to generate private key");
let public_key = RsaPublicKey::from(&private_key);
let private_key_pem = private_key.to_pkcs8_pem(LineEnding::LF).unwrap();
let public_key_pem = public_key.to_public_key_pem(LineEnding::LF).unwrap();
let token = Token {
name: "test_app".to_string(),
expired: SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs() + 3600, // 1 hour from now
};
let encoded = gencode(&token, &public_key_pem).expect("Failed to encode token");
let decoded = parse(&encoded, &private_key_pem).expect("Failed to decode token");
assert_eq!(token.name, decoded.name);
assert_eq!(token.expired, decoded.expired);
}
#[test]
fn test_parse_invalid_token() {
let private_key_pem = RsaPrivateKey::new(&mut rand::thread_rng(), 2048)
.expect("Failed to generate private key")
.to_pkcs8_pem(LineEnding::LF)
.unwrap();
let invalid_token = "invalid_base64_token";
let result = parse(invalid_token, &private_key_pem);
assert!(result.is_err());
}
#[test]
fn test_gencode_with_invalid_key() {
let token = Token {
name: "test_app".to_string(),
expired: SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs() + 3600, // 1 hour from now
};
let invalid_key = "invalid_public_key";
let result = gencode(&token, invalid_key);
assert!(result.is_err());
}
}
cli/rustfs-gui/Cargo.toml
-1
View File
@@ -10,7 +10,6 @@ version.workspace = true
chrono = { workspace = true }
dioxus = { workspace = true, features = ["router"] }
dirs = { workspace = true }
futures-util = { workspace = true }
hex = { workspace = true }
keyring = { workspace = true }
lazy_static = { workspace = true }
common/common/Cargo.toml
+1 -1
View File
@@ -11,5 +11,5 @@ async-trait.workspace = true
lazy_static.workspace = true
scopeguard = "1.2.0"
tokio.workspace = true
tonic.workspace = true
tonic = { workspace = true }
tracing-error.workspace = true
common/protos/Cargo.toml
+1 -1
View File
@@ -13,7 +13,7 @@ flatbuffers = { workspace = true }
prost = { workspace = true }
protobuf = { workspace = true }
tokio = { workspace = true }
tonic = { workspace = true, features = ["transport", "tls"] }
tonic = { workspace = true, features = ["transport"] }
tower = { workspace = true }
[build-dependencies]
common/protos/build.rs
+1 -1
View File
@@ -8,7 +8,7 @@ use std::{
type AnyError = Box<dyn std::error::Error>;
const ENV_OUT_DIR: &str = "OUT_DIR";
const VERSION_PROTOBUF: Version = Version(27, 0, 0); // 27.0
const VERSION_PROTOBUF: Version = Version(30, 2, 0); // 30.2.0
const VERSION_FLATBUFFERS: Version = Version(24, 3, 25); // 24.3.25
/// Build protos if the major version of `flatc` or `protoc` is greater
/// or lesser than the expected version.
common/protos/src/generated/flatbuffers_generated/models.rs
+104 -103
View File
@@ -1,10 +1,9 @@
// automatically generated by the FlatBuffers compiler, do not modify
// @generated
use core::mem;
use core::cmp::Ordering;
use core::mem;
extern crate flatbuffers;
use self::flatbuffers::{EndianScalar, Follow};
@@ -12,112 +11,114 @@ use self::flatbuffers::{EndianScalar, Follow};
#[allow(unused_imports, dead_code)]
pub mod models {
use core::mem;
use core::cmp::Ordering;
use core::cmp::Ordering;
use core::mem;
extern crate flatbuffers;
use self::flatbuffers::{EndianScalar, Follow};
extern crate flatbuffers;
use self::flatbuffers::{EndianScalar, Follow};
pub enum PingBodyOffset {}
#[derive(Copy, Clone, PartialEq)]
pub enum PingBodyOffset {}
#[derive(Copy, Clone, PartialEq)]
pub struct PingBody<'a> {
pub _tab: flatbuffers::Table<'a>,
}
impl<'a> flatbuffers::Follow<'a> for PingBody<'a> {
type Inner = PingBody<'a>;
#[inline]
unsafe fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
Self { _tab: flatbuffers::Table::new(buf, loc) }
}
}
impl<'a> PingBody<'a> {
pub const VT_PAYLOAD: flatbuffers::VOffsetT = 4;
pub const fn get_fully_qualified_name() -> &'static str {
"models.PingBody"
}
#[inline]
pub unsafe fn init_from_table(table: flatbuffers::Table<'a>) -> Self {
PingBody { _tab: table }
}
#[allow(unused_mut)]
pub fn create<'bldr: 'args, 'args: 'mut_bldr, 'mut_bldr, A: flatbuffers::Allocator + 'bldr>(
_fbb: &'mut_bldr mut flatbuffers::FlatBufferBuilder<'bldr, A>,
args: &'args PingBodyArgs<'args>
) -> flatbuffers::WIPOffset<PingBody<'bldr>> {
let mut builder = PingBodyBuilder::new(_fbb);
if let Some(x) = args.payload { builder.add_payload(x); }
builder.finish()
}
#[inline]
pub fn payload(&self) -> Option<flatbuffers::Vector<'a, u8>> {
// Safety:
// Created from valid Table for this object
// which contains a valid value in this slot
unsafe { self._tab.get::<flatbuffers::ForwardsUOffset<flatbuffers::Vector<'a, u8>>>(PingBody::VT_PAYLOAD, None)}
}
}
impl flatbuffers::Verifiable for PingBody<'_> {
#[inline]
fn run_verifier(
v: &mut flatbuffers::Verifier, pos: usize
) -> Result<(), flatbuffers::InvalidFlatbuffer> {
use self::flatbuffers::Verifiable;
v.visit_table(pos)?
.visit_field::<flatbuffers::ForwardsUOffset<flatbuffers::Vector<'_, u8>>>("payload", Self::VT_PAYLOAD, false)?
.finish();
Ok(())
}
}
pub struct PingBodyArgs<'a> {
pub payload: Option<flatbuffers::WIPOffset<flatbuffers::Vector<'a, u8>>>,
}
impl<'a> Default for PingBodyArgs<'a> {
#[inline]
fn default() -> Self {
PingBodyArgs {
payload: None,
pub struct PingBody<'a> {
pub _tab: flatbuffers::Table<'a>,
}
}
}
pub struct PingBodyBuilder<'a: 'b, 'b, A: flatbuffers::Allocator + 'a> {
fbb_: &'b mut flatbuffers::FlatBufferBuilder<'a, A>,
start_: flatbuffers::WIPOffset<flatbuffers::TableUnfinishedWIPOffset>,
}
impl<'a: 'b, 'b, A: flatbuffers::Allocator + 'a> PingBodyBuilder<'a, 'b, A> {
#[inline]
pub fn add_payload(&mut self, payload: flatbuffers::WIPOffset<flatbuffers::Vector<'b , u8>>) {
self.fbb_.push_slot_always::<flatbuffers::WIPOffset<_>>(PingBody::VT_PAYLOAD, payload);
}
#[inline]
pub fn new(_fbb: &'b mut flatbuffers::FlatBufferBuilder<'a, A>) -> PingBodyBuilder<'a, 'b, A> {
let start = _fbb.start_table();
PingBodyBuilder {
fbb_: _fbb,
start_: start,
impl<'a> flatbuffers::Follow<'a> for PingBody<'a> {
type Inner = PingBody<'a>;
#[inline]
unsafe fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
Self {
_tab: flatbuffers::Table::new(buf, loc),
}
}
}
}
#[inline]
pub fn finish(self) -> flatbuffers::WIPOffset<PingBody<'a>> {
let o = self.fbb_.end_table(self.start_);
flatbuffers::WIPOffset::new(o.value())
}
}
impl core::fmt::Debug for PingBody<'_> {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
let mut ds = f.debug_struct("PingBody");
ds.field("payload", &self.payload());
ds.finish()
}
}
} // pub mod models
impl<'a> PingBody<'a> {
pub const VT_PAYLOAD: flatbuffers::VOffsetT = 4;
pub const fn get_fully_qualified_name() -> &'static str {
"models.PingBody"
}
#[inline]
pub unsafe fn init_from_table(table: flatbuffers::Table<'a>) -> Self {
PingBody { _tab: table }
}
#[allow(unused_mut)]
pub fn create<'bldr: 'args, 'args: 'mut_bldr, 'mut_bldr, A: flatbuffers::Allocator + 'bldr>(
_fbb: &'mut_bldr mut flatbuffers::FlatBufferBuilder<'bldr, A>,
args: &'args PingBodyArgs<'args>,
) -> flatbuffers::WIPOffset<PingBody<'bldr>> {
let mut builder = PingBodyBuilder::new(_fbb);
if let Some(x) = args.payload {
builder.add_payload(x);
}
builder.finish()
}
#[inline]
pub fn payload(&self) -> Option<flatbuffers::Vector<'a, u8>> {
// Safety:
// Created from valid Table for this object
// which contains a valid value in this slot
unsafe {
self._tab
.get::<flatbuffers::ForwardsUOffset<flatbuffers::Vector<'a, u8>>>(PingBody::VT_PAYLOAD, None)
}
}
}
impl flatbuffers::Verifiable for PingBody<'_> {
#[inline]
fn run_verifier(v: &mut flatbuffers::Verifier, pos: usize) -> Result<(), flatbuffers::InvalidFlatbuffer> {
use self::flatbuffers::Verifiable;
v.visit_table(pos)?
.visit_field::<flatbuffers::ForwardsUOffset<flatbuffers::Vector<'_, u8>>>("payload", Self::VT_PAYLOAD, false)?
.finish();
Ok(())
}
}
pub struct PingBodyArgs<'a> {
pub payload: Option<flatbuffers::WIPOffset<flatbuffers::Vector<'a, u8>>>,
}
impl<'a> Default for PingBodyArgs<'a> {
#[inline]
fn default() -> Self {
PingBodyArgs { payload: None }
}
}
pub struct PingBodyBuilder<'a: 'b, 'b, A: flatbuffers::Allocator + 'a> {
fbb_: &'b mut flatbuffers::FlatBufferBuilder<'a, A>,
start_: flatbuffers::WIPOffset<flatbuffers::TableUnfinishedWIPOffset>,
}
impl<'a: 'b, 'b, A: flatbuffers::Allocator + 'a> PingBodyBuilder<'a, 'b, A> {
#[inline]
pub fn add_payload(&mut self, payload: flatbuffers::WIPOffset<flatbuffers::Vector<'b, u8>>) {
self.fbb_
.push_slot_always::<flatbuffers::WIPOffset<_>>(PingBody::VT_PAYLOAD, payload);
}
#[inline]
pub fn new(_fbb: &'b mut flatbuffers::FlatBufferBuilder<'a, A>) -> PingBodyBuilder<'a, 'b, A> {
let start = _fbb.start_table();
PingBodyBuilder {
fbb_: _fbb,
start_: start,
}
}
#[inline]
pub fn finish(self) -> flatbuffers::WIPOffset<PingBody<'a>> {
let o = self.fbb_.end_table(self.start_);
flatbuffers::WIPOffset::new(o.value())
}
}
impl core::fmt::Debug for PingBody<'_> {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
let mut ds = f.debug_struct("PingBody");
ds.field("payload", &self.payload());
ds.finish()
}
}
} // pub mod models
common/protos/src/generated/proto_gen/node_service.rs
+786 -3056
View File
File diff suppressed because it is too large Load Diff
crates/obs/Cargo.toml
+53
View File
@@ -0,0 +1,53 @@
[package]
name = "rustfs-obs"
edition.workspace = true
license.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true
[lints]
workspace = true
[features]
default = ["file"]
kafka = ["dep:rdkafka"]
webhook = ["dep:reqwest"]
file = []
[dependencies]
async-trait = { workspace = true }
chrono = { workspace = true }
config = { workspace = true }
opentelemetry = { workspace = true }
opentelemetry-appender-tracing = { workspace = true, features = ["experimental_use_tracing_span_context", "experimental_metadata_attributes"] }
opentelemetry_sdk = { workspace = true, features = ["rt-tokio"] }
opentelemetry-stdout = { workspace = true }
opentelemetry-otlp = { workspace = true, features = ["grpc-tonic", "gzip-tonic"] }
opentelemetry-prometheus = { workspace = true }
opentelemetry-semantic-conventions = { workspace = true, features = ["semconv_experimental"] }
prometheus = { workspace = true }
serde = { workspace = true }
tracing = { workspace = true, features = ["std", "attributes"] }
tracing-core = { workspace = true }
tracing-error = { workspace = true }
tracing-opentelemetry = { workspace = true }
tracing-subscriber = { workspace = true, features = ["registry", "std", "fmt", "env-filter", "tracing-log", "time", "local-time", "json"] }
tokio = { workspace = true, features = ["sync", "fs", "rt-multi-thread"] }
rdkafka = { workspace = true, features = ["tokio"], optional = true }
reqwest = { workspace = true, optional = true, default-features = false }
serde_json = { workspace = true }
thiserror = { workspace = true }
local-ip-address = { workspace = true }
[dev-dependencies]
chrono = { workspace = true }
opentelemetry = { workspace = true }
opentelemetry_sdk = { workspace = true, features = ["rt-tokio"] }
opentelemetry-stdout = { workspace = true }
opentelemetry-otlp = { workspace = true, features = ["grpc-tonic"] }
opentelemetry-semantic-conventions = { workspace = true, features = ["semconv_experimental"] }
tokio = { workspace = true, features = ["full"] }
tracing = { workspace = true, features = ["std", "attributes"] }
tracing-subscriber = { workspace = true, features = ["registry", "std", "fmt"] }
crates/obs/examples/config.toml
+33
View File
@@ -0,0 +1,33 @@
[observability]
endpoint = "http://localhost:4317" # Default is "http://localhost:4317" if not specified
use_stdout = false # Output with stdout, true output, false no output
sample_ratio = 1
meter_interval = 30
service_name = "rustfs_obs"
service_version = "0.1.0"
environments = "develop"
logger_level = "debug"
[sinks]
[sinks.kafka]
enabled = false
bootstrap_servers = "localhost:9092"
topic = "logs"
batch_size = 100 # Default is 100 if not specified
batch_timeout_ms = 1000 # Default is 1000ms if not specified
[sinks.webhook]
enabled = false
endpoint = "http://localhost:8080/webhook"
auth_token = ""
batch_size = 100 # Default is 3 if not specified
batch_timeout_ms = 1000 # Default is 100ms if not specified
[sinks.file]
enabled = true
path = "logs/app.log"
batch_size = 100
batch_timeout_ms = 1000 # Default is 8192 bytes if not specified
[logger]
queue_capacity = 10000
crates/obs/examples/server.rs
+88
View File
@@ -0,0 +1,88 @@
use opentelemetry::global;
use rustfs_obs::{get_logger, init_obs, load_config, log_info, BaseLogEntry, ServerLogEntry};
use std::collections::HashMap;
use std::time::{Duration, SystemTime};
use tracing::{info, instrument};
use tracing_core::Level;
#[tokio::main]
async fn main() {
let obs_conf = Some("crates/obs/examples/config.toml".to_string());
let config = load_config(obs_conf);
let (_logger, _guard) = init_obs(config.clone()).await;
let span = tracing::span!(Level::INFO, "main");
let _enter = span.enter();
info!("Program starts");
// Simulate the operation
tokio::time::sleep(Duration::from_millis(100)).await;
run(
"service-demo".to_string(),
"object-demo".to_string(),
"user-demo".to_string(),
"service-demo".to_string(),
)
.await;
info!("Program ends");
}
#[instrument(fields(bucket, object, user))]
async fn run(bucket: String, object: String, user: String, service_name: String) {
let start_time = SystemTime::now();
info!("Log module initialization is completed service_name: {:?}", service_name);
// Record Metrics
let meter = global::meter("rustfs");
let request_duration = meter.f64_histogram("s3_request_duration_seconds").build();
request_duration.record(
start_time.elapsed().unwrap().as_secs_f64(),
&[opentelemetry::KeyValue::new("operation", "run")],
);
let base_entry = BaseLogEntry::new()
.message(Some("run logger api_handler info".to_string()))
.request_id(Some("request_id".to_string()))
.timestamp(chrono::DateTime::from(start_time))
.tags(Some(HashMap::default()));
let server_entry = ServerLogEntry::new(Level::INFO, "api_handler".to_string())
.with_base(base_entry)
.user_id(Some(user.clone()))
.add_field("operation".to_string(), "login".to_string())
.add_field("bucket".to_string(), bucket.clone())
.add_field("object".to_string(), object.clone());
let result = get_logger().lock().await.log_server_entry(server_entry).await;
info!("Logging is completed {:?}", result);
put_object("bucket".to_string(), "object".to_string(), "user".to_string()).await;
info!("Logging is completed");
tokio::time::sleep(Duration::from_secs(2)).await;
info!("Program run ends");
}
#[instrument(fields(bucket, object, user))]
async fn put_object(bucket: String, object: String, user: String) {
let start_time = SystemTime::now();
info!("Starting put_object operation time: {:?}", start_time);
let meter = global::meter("rustfs");
let request_duration = meter.f64_histogram("s3_request_duration_seconds").build();
request_duration.record(
start_time.elapsed().unwrap().as_secs_f64(),
&[opentelemetry::KeyValue::new("operation", "put_object")],
);
info!(
"Starting PUT operation content: bucket = {}, object = {}, user = {},start_time = {}",
bucket,
object,
user,
start_time.elapsed().unwrap().as_secs_f64()
);
let result = log_info("put_object logger info", "put_object").await;
info!("put_object is completed {:?}", result);
// Simulate the operation
tokio::time::sleep(Duration::from_millis(100)).await;
info!("PUT operation completed");
}
crates/obs/src/config.rs
+271
View File
@@ -0,0 +1,271 @@
use crate::global::{ENVIRONMENT, LOGGER_LEVEL, METER_INTERVAL, SAMPLE_RATIO, SERVICE_NAME, SERVICE_VERSION, USE_STDOUT};
use config::{Config, Environment, File, FileFormat};
use serde::Deserialize;
use std::env;
/// OpenTelemetry Configuration
/// Add service name, service version, environment
/// Add interval time for metric collection
/// Add sample ratio for trace sampling
/// Add endpoint for metric collection
/// Add use_stdout for output to stdout
/// Add logger level for log level
#[derive(Debug, Deserialize, Clone)]
pub struct OtelConfig {
pub endpoint: String,
pub use_stdout: Option<bool>,
pub sample_ratio: Option<f64>,
pub meter_interval: Option<u64>,
pub service_name: Option<String>,
pub service_version: Option<String>,
pub environment: Option<String>,
pub logger_level: Option<String>,
}
// 辅助函数：从环境变量中提取可观测性配置
fn extract_otel_config_from_env() -> OtelConfig {
OtelConfig {
endpoint: env::var("RUSTFS_OBSERVABILITY_ENDPOINT").unwrap_or_else(|_| "".to_string()),
use_stdout: env::var("RUSTFS_OBSERVABILITY_USE_STDOUT")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(USE_STDOUT)),
sample_ratio: env::var("RUSTFS_OBSERVABILITY_SAMPLE_RATIO")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(SAMPLE_RATIO)),
meter_interval: env::var("RUSTFS_OBSERVABILITY_METER_INTERVAL")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(METER_INTERVAL)),
service_name: env::var("RUSTFS_OBSERVABILITY_SERVICE_NAME")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(SERVICE_NAME.to_string())),
service_version: env::var("RUSTFS_OBSERVABILITY_SERVICE_VERSION")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(SERVICE_VERSION.to_string())),
environment: env::var("RUSTFS_OBSERVABILITY_ENVIRONMENT")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(ENVIRONMENT.to_string())),
logger_level: env::var("RUSTFS_OBSERVABILITY_LOGGER_LEVEL")
.ok()
.and_then(|v| v.parse().ok())
.or(Some(LOGGER_LEVEL.to_string())),
}
}
impl Default for OtelConfig {
fn default() -> Self {
extract_otel_config_from_env()
}
}
/// Kafka Sink Configuration - Add batch parameters
#[derive(Debug, Deserialize, Clone, Default)]
pub struct KafkaSinkConfig {
pub enabled: bool,
pub bootstrap_servers: String,
pub topic: String,
pub batch_size: Option<usize>, // Batch size, default 100
pub batch_timeout_ms: Option<u64>, // Batch timeout time, default 1000ms
}
/// Webhook Sink Configuration - Add Retry Parameters
#[derive(Debug, Deserialize, Clone, Default)]
pub struct WebhookSinkConfig {
pub enabled: bool,
pub endpoint: String,
pub auth_token: String,
pub max_retries: Option<usize>, // Maximum number of retry times, default 3
pub retry_delay_ms: Option<u64>, // Retry the delay cardinality, default 100ms
}
/// File Sink Configuration - Add buffering parameters
#[derive(Debug, Deserialize, Clone)]
pub struct FileSinkConfig {
pub enabled: bool,
pub path: String,
pub buffer_size: Option<usize>, // Write buffer size, default 8192
pub flush_interval_ms: Option<u64>, // Refresh interval time, default 1000ms
pub flush_threshold: Option<usize>, // Refresh threshold, default 100 logs
}
impl FileSinkConfig {
pub fn get_default_log_path() -> String {
let temp_dir = env::temp_dir().join("rustfs");
if let Err(e) = std::fs::create_dir_all(&temp_dir) {
eprintln!("Failed to create log directory: {}", e);
return "rustfs/rustfs.log".to_string();
}
temp_dir
.join("rustfs.log")
.to_str()
.unwrap_or("rustfs/rustfs.log")
.to_string()
}
}
impl Default for FileSinkConfig {
fn default() -> Self {
FileSinkConfig {
enabled: true,
path: env::var("RUSTFS_SINKS_FILE_PATH")
.ok()
.filter(|s| !s.trim().is_empty())
.unwrap_or_else(Self::get_default_log_path),
buffer_size: Some(8192),
flush_interval_ms: Some(1000),
flush_threshold: Some(100),
}
}
}
/// Sink configuration collection
#[derive(Debug, Deserialize, Clone)]
pub struct SinkConfig {
pub kafka: Option<KafkaSinkConfig>,
pub webhook: Option<WebhookSinkConfig>,
pub file: Option<FileSinkConfig>,
}
impl Default for SinkConfig {
fn default() -> Self {
SinkConfig {
kafka: None,
webhook: None,
file: Some(FileSinkConfig::default()),
}
}
}
///Logger Configuration
#[derive(Debug, Deserialize, Clone)]
pub struct LoggerConfig {
pub queue_capacity: Option<usize>,
}
impl Default for LoggerConfig {
fn default() -> Self {
LoggerConfig {
queue_capacity: Some(10000),
}
}
}
/// Overall application configuration
/// Add observability, sinks, and logger configuration
///
/// Observability: OpenTelemetry configuration
/// Sinks: Kafka, Webhook, File sink configuration
/// Logger: Logger configuration
///
/// # Example
/// ```
/// use rustfs_obs::AppConfig;
/// use rustfs_obs::load_config;
///
/// let config = load_config(None);
/// ```
#[derive(Debug, Deserialize, Clone)]
pub struct AppConfig {
pub observability: OtelConfig,
pub sinks: SinkConfig,
pub logger: Option<LoggerConfig>,
}
// 为 AppConfig 实现 Default
impl AppConfig {
pub fn new() -> Self {
Self {
observability: OtelConfig::default(),
sinks: SinkConfig::default(),
logger: Some(LoggerConfig::default()),
}
}
}
impl Default for AppConfig {
fn default() -> Self {
Self::new()
}
}
const DEFAULT_CONFIG_FILE: &str = "obs";
/// Loading the configuration file
/// Supports TOML, YAML and .env formats, read in order by priority
///
/// # Parameters
/// - `config_dir`: Configuration file path
///
/// # Returns
/// Configuration information
///
/// # Example
/// ```
/// use rustfs_obs::AppConfig;
/// use rustfs_obs::load_config;
///
/// let config = load_config(None);
/// ```
pub fn load_config(config_dir: Option<String>) -> AppConfig {
let config_dir = if let Some(path) = config_dir {
// If a path is provided, check if it's empty
if path.is_empty() {
// If empty, use the default config file name
DEFAULT_CONFIG_FILE.to_string()
} else {
// Use the provided path
let path = std::path::Path::new(&path);
if path.extension().is_some() {
// If path has extension, use it as is (extension will be added by Config::builder)
path.with_extension("").to_string_lossy().into_owned()
} else {
// If path is a directory, append the default config file name
path.to_string_lossy().into_owned()
}
}
} else {
// If no path provided, use current directory + default config file
match env::current_dir() {
Ok(dir) => dir.join(DEFAULT_CONFIG_FILE).to_string_lossy().into_owned(),
Err(_) => {
eprintln!("Warning: Failed to get current directory, using default config file");
DEFAULT_CONFIG_FILE.to_string()
}
}
};
// Log using proper logging instead of println when possible
println!("Using config file base: {}", config_dir);
let app_config = Config::builder()
.add_source(File::with_name(config_dir.as_str()).format(FileFormat::Toml).required(false))
.add_source(File::with_name(config_dir.as_str()).format(FileFormat::Yaml).required(false))
.add_source(
Environment::default()
.prefix("RUSTFS")
.prefix_separator("__")
.separator("__")
.with_list_parse_key("volumes")
.try_parsing(true),
)
.build()
.unwrap_or_default();
match app_config.try_deserialize::<AppConfig>() {
Ok(app_config) => {
println!("Parsed AppConfig: {:?}", app_config);
app_config
}
Err(e) => {
println!("Failed to deserialize config: {}", e);
AppConfig::default()
}
}
}
crates/obs/src/entry/args.rs
+74
View File
@@ -0,0 +1,74 @@
use crate::entry::ObjectVersion;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
/// Args - defines the arguments for API operations
/// Args is used to define the arguments for API operations.
///
/// # Example
/// ```
/// use rustfs_obs::Args;
/// use std::collections::HashMap;
///
/// let args = Args::new()
/// .set_bucket(Some("my-bucket".to_string()))
/// .set_object(Some("my-object".to_string()))
/// .set_version_id(Some("123".to_string()))
/// .set_metadata(Some(HashMap::new()));
/// ```
#[derive(Debug, Clone, Serialize, Deserialize, Default, Eq, PartialEq)]
pub struct Args {
#[serde(rename = "bucket", skip_serializing_if = "Option::is_none")]
pub bucket: Option<String>,
#[serde(rename = "object", skip_serializing_if = "Option::is_none")]
pub object: Option<String>,
#[serde(rename = "versionId", skip_serializing_if = "Option::is_none")]
pub version_id: Option<String>,
#[serde(rename = "objects", skip_serializing_if = "Option::is_none")]
pub objects: Option<Vec<ObjectVersion>>,
#[serde(rename = "metadata", skip_serializing_if = "Option::is_none")]
pub metadata: Option<HashMap<String, String>>,
}
impl Args {
/// Create a new Args object
pub fn new() -> Self {
Args {
bucket: None,
object: None,
version_id: None,
objects: None,
metadata: None,
}
}
/// Set the bucket
pub fn set_bucket(mut self, bucket: Option<String>) -> Self {
self.bucket = bucket;
self
}
/// Set the object
pub fn set_object(mut self, object: Option<String>) -> Self {
self.object = object;
self
}
/// Set the version ID
pub fn set_version_id(mut self, version_id: Option<String>) -> Self {
self.version_id = version_id;
self
}
/// Set the objects
pub fn set_objects(mut self, objects: Option<Vec<ObjectVersion>>) -> Self {
self.objects = objects;
self
}
/// Set the metadata
pub fn set_metadata(mut self, metadata: Option<HashMap<String, String>>) -> Self {
self.metadata = metadata;
self
}
}
crates/obs/src/entry/audit.rs
+453
View File
@@ -0,0 +1,453 @@
use crate::{BaseLogEntry, LogRecord, ObjectVersion};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::collections::HashMap;
/// API details structure
/// ApiDetails is used to define the details of an API operation
///
/// The `ApiDetails` structure contains the following fields:
/// - `name` - the name of the API operation
/// - `bucket` - the bucket name
/// - `object` - the object name
/// - `objects` - the list of objects
/// - `status` - the status of the API operation
/// - `status_code` - the status code of the API operation
/// - `input_bytes` - the input bytes
/// - `output_bytes` - the output bytes
/// - `header_bytes` - the header bytes
/// - `time_to_first_byte` - the time to first byte
/// - `time_to_first_byte_in_ns` - the time to first byte in nanoseconds
/// - `time_to_response` - the time to response
/// - `time_to_response_in_ns` - the time to response in nanoseconds
///
/// The `ApiDetails` structure contains the following methods:
/// - `new` - create a new `ApiDetails` with default values
/// - `set_name` - set the name
/// - `set_bucket` - set the bucket
/// - `set_object` - set the object
/// - `set_objects` - set the objects
/// - `set_status` - set the status
/// - `set_status_code` - set the status code
/// - `set_input_bytes` - set the input bytes
/// - `set_output_bytes` - set the output bytes
/// - `set_header_bytes` - set the header bytes
/// - `set_time_to_first_byte` - set the time to first byte
/// - `set_time_to_first_byte_in_ns` - set the time to first byte in nanoseconds
/// - `set_time_to_response` - set the time to response
/// - `set_time_to_response_in_ns` - set the time to response in nanoseconds
///
/// # Example
/// ```
/// use rustfs_obs::ApiDetails;
/// use rustfs_obs::ObjectVersion;
///
/// let api = ApiDetails::new()
/// .set_name(Some("GET".to_string()))
/// .set_bucket(Some("my-bucket".to_string()))
/// .set_object(Some("my-object".to_string()))
/// .set_objects(vec![ObjectVersion::new_with_object_name("my-object".to_string())])
/// .set_status(Some("OK".to_string()))
/// .set_status_code(Some(200))
/// .set_input_bytes(100)
/// .set_output_bytes(200)
/// .set_header_bytes(Some(50))
/// .set_time_to_first_byte(Some("100ms".to_string()))
/// .set_time_to_first_byte_in_ns(Some("100000000ns".to_string()))
/// .set_time_to_response(Some("200ms".to_string()))
/// .set_time_to_response_in_ns(Some("200000000ns".to_string()));
/// ```
#[derive(Debug, Serialize, Deserialize, Clone, Default, PartialEq, Eq)]
pub struct ApiDetails {
#[serde(rename = "name", skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
#[serde(rename = "bucket", skip_serializing_if = "Option::is_none")]
pub bucket: Option<String>,
#[serde(rename = "object", skip_serializing_if = "Option::is_none")]
pub object: Option<String>,
#[serde(rename = "objects", skip_serializing_if = "Vec::is_empty", default)]
pub objects: Vec<ObjectVersion>,
#[serde(rename = "status", skip_serializing_if = "Option::is_none")]
pub status: Option<String>,
#[serde(rename = "statusCode", skip_serializing_if = "Option::is_none")]
pub status_code: Option<i32>,
#[serde(rename = "rx")]
pub input_bytes: i64,
#[serde(rename = "tx")]
pub output_bytes: i64,
#[serde(rename = "txHeaders", skip_serializing_if = "Option::is_none")]
pub header_bytes: Option<i64>,
#[serde(rename = "timeToFirstByte", skip_serializing_if = "Option::is_none")]
pub time_to_first_byte: Option<String>,
#[serde(rename = "timeToFirstByteInNS", skip_serializing_if = "Option::is_none")]
pub time_to_first_byte_in_ns: Option<String>,
#[serde(rename = "timeToResponse", skip_serializing_if = "Option::is_none")]
pub time_to_response: Option<String>,
#[serde(rename = "timeToResponseInNS", skip_serializing_if = "Option::is_none")]
pub time_to_response_in_ns: Option<String>,
}
impl ApiDetails {
/// Create a new `ApiDetails` with default values
pub fn new() -> Self {
ApiDetails {
name: None,
bucket: None,
object: None,
objects: Vec::new(),
status: None,
status_code: None,
input_bytes: 0,
output_bytes: 0,
header_bytes: None,
time_to_first_byte: None,
time_to_first_byte_in_ns: None,
time_to_response: None,
time_to_response_in_ns: None,
}
}
/// Set the name
pub fn set_name(mut self, name: Option<String>) -> Self {
self.name = name;
self
}
/// Set the bucket
pub fn set_bucket(mut self, bucket: Option<String>) -> Self {
self.bucket = bucket;
self
}
/// Set the object
pub fn set_object(mut self, object: Option<String>) -> Self {
self.object = object;
self
}
/// Set the objects
pub fn set_objects(mut self, objects: Vec<ObjectVersion>) -> Self {
self.objects = objects;
self
}
/// Set the status
pub fn set_status(mut self, status: Option<String>) -> Self {
self.status = status;
self
}
/// Set the status code
pub fn set_status_code(mut self, status_code: Option<i32>) -> Self {
self.status_code = status_code;
self
}
/// Set the input bytes
pub fn set_input_bytes(mut self, input_bytes: i64) -> Self {
self.input_bytes = input_bytes;
self
}
/// Set the output bytes
pub fn set_output_bytes(mut self, output_bytes: i64) -> Self {
self.output_bytes = output_bytes;
self
}
/// Set the header bytes
pub fn set_header_bytes(mut self, header_bytes: Option<i64>) -> Self {
self.header_bytes = header_bytes;
self
}
/// Set the time to first byte
pub fn set_time_to_first_byte(mut self, time_to_first_byte: Option<String>) -> Self {
self.time_to_first_byte = time_to_first_byte;
self
}
/// Set the time to first byte in nanoseconds
pub fn set_time_to_first_byte_in_ns(mut self, time_to_first_byte_in_ns: Option<String>) -> Self {
self.time_to_first_byte_in_ns = time_to_first_byte_in_ns;
self
}
/// Set the time to response
pub fn set_time_to_response(mut self, time_to_response: Option<String>) -> Self {
self.time_to_response = time_to_response;
self
}
/// Set the time to response in nanoseconds
pub fn set_time_to_response_in_ns(mut self, time_to_response_in_ns: Option<String>) -> Self {
self.time_to_response_in_ns = time_to_response_in_ns;
self
}
}
/// Entry - audit entry logs
/// AuditLogEntry is used to define the structure of an audit log entry
///
/// The `AuditLogEntry` structure contains the following fields:
/// - `base` - the base log entry
/// - `version` - the version of the audit log entry
/// - `deployment_id` - the deployment ID
/// - `event` - the event
/// - `entry_type` - the type of audit message
/// - `api` - the API details
/// - `remote_host` - the remote host
/// - `user_agent` - the user agent
/// - `req_path` - the request path
/// - `req_host` - the request host
/// - `req_claims` - the request claims
/// - `req_query` - the request query
/// - `req_header` - the request header
/// - `resp_header` - the response header
/// - `access_key` - the access key
/// - `parent_user` - the parent user
/// - `error` - the error
///
/// The `AuditLogEntry` structure contains the following methods:
/// - `new` - create a new `AuditEntry` with default values
/// - `new_with_values` - create a new `AuditEntry` with version, time, event and api details
/// - `with_base` - set the base log entry
/// - `set_version` - set the version
/// - `set_deployment_id` - set the deployment ID
/// - `set_event` - set the event
/// - `set_entry_type` - set the entry type
/// - `set_api` - set the API details
/// - `set_remote_host` - set the remote host
/// - `set_user_agent` - set the user agent
/// - `set_req_path` - set the request path
/// - `set_req_host` - set the request host
/// - `set_req_claims` - set the request claims
/// - `set_req_query` - set the request query
/// - `set_req_header` - set the request header
/// - `set_resp_header` - set the response header
/// - `set_access_key` - set the access key
/// - `set_parent_user` - set the parent user
/// - `set_error` - set the error
///
/// # Example
/// ```
/// use rustfs_obs::AuditLogEntry;
/// use rustfs_obs::ApiDetails;
/// use std::collections::HashMap;
///
/// let entry = AuditLogEntry::new()
/// .set_version("1.0".to_string())
/// .set_deployment_id(Some("123".to_string()))
/// .set_event("event".to_string())
/// .set_entry_type(Some("type".to_string()))
/// .set_api(ApiDetails::new())
/// .set_remote_host(Some("remote-host".to_string()))
/// .set_user_agent(Some("user-agent".to_string()))
/// .set_req_path(Some("req-path".to_string()))
/// .set_req_host(Some("req-host".to_string()))
/// .set_req_claims(Some(HashMap::new()))
/// .set_req_query(Some(HashMap::new()))
/// .set_req_header(Some(HashMap::new()))
/// .set_resp_header(Some(HashMap::new()))
/// .set_access_key(Some("access-key".to_string()))
/// .set_parent_user(Some("parent-user".to_string()))
/// .set_error(Some("error".to_string()));
#[derive(Debug, Serialize, Deserialize, Clone, Default)]
pub struct AuditLogEntry {
#[serde(flatten)]
pub base: BaseLogEntry,
pub version: String,
#[serde(rename = "deploymentid", skip_serializing_if = "Option::is_none")]
pub deployment_id: Option<String>,
pub event: String,
// Class of audit message - S3, admin ops, bucket management
#[serde(rename = "type", skip_serializing_if = "Option::is_none")]
pub entry_type: Option<String>,
pub api: ApiDetails,
#[serde(rename = "remotehost", skip_serializing_if = "Option::is_none")]
pub remote_host: Option<String>,
#[serde(rename = "userAgent", skip_serializing_if = "Option::is_none")]
pub user_agent: Option<String>,
#[serde(rename = "requestPath", skip_serializing_if = "Option::is_none")]
pub req_path: Option<String>,
#[serde(rename = "requestHost", skip_serializing_if = "Option::is_none")]
pub req_host: Option<String>,
#[serde(rename = "requestClaims", skip_serializing_if = "Option::is_none")]
pub req_claims: Option<HashMap<String, Value>>,
#[serde(rename = "requestQuery", skip_serializing_if = "Option::is_none")]
pub req_query: Option<HashMap<String, String>>,
#[serde(rename = "requestHeader", skip_serializing_if = "Option::is_none")]
pub req_header: Option<HashMap<String, String>>,
#[serde(rename = "responseHeader", skip_serializing_if = "Option::is_none")]
pub resp_header: Option<HashMap<String, String>>,
#[serde(rename = "accessKey", skip_serializing_if = "Option::is_none")]
pub access_key: Option<String>,
#[serde(rename = "parentUser", skip_serializing_if = "Option::is_none")]
pub parent_user: Option<String>,
#[serde(rename = "error", skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
}
impl AuditLogEntry {
/// Create a new `AuditEntry` with default values
pub fn new() -> Self {
AuditLogEntry {
base: BaseLogEntry::new(),
version: String::new(),
deployment_id: None,
event: String::new(),
entry_type: None,
api: ApiDetails::new(),
remote_host: None,
user_agent: None,
req_path: None,
req_host: None,
req_claims: None,
req_query: None,
req_header: None,
resp_header: None,
access_key: None,
parent_user: None,
error: None,
}
}
/// Create a new `AuditEntry` with version, time, event and api details
pub fn new_with_values(version: String, time: DateTime<Utc>, event: String, api: ApiDetails) -> Self {
let mut base = BaseLogEntry::new();
base.timestamp = time;
AuditLogEntry {
base,
version,
deployment_id: None,
event,
entry_type: None,
api,
remote_host: None,
user_agent: None,
req_path: None,
req_host: None,
req_claims: None,
req_query: None,
req_header: None,
resp_header: None,
access_key: None,
parent_user: None,
error: None,
}
}
/// Set the base log entry
pub fn with_base(mut self, base: BaseLogEntry) -> Self {
self.base = base;
self
}
/// Set the version
pub fn set_version(mut self, version: String) -> Self {
self.version = version;
self
}
/// Set the deployment ID
pub fn set_deployment_id(mut self, deployment_id: Option<String>) -> Self {
self.deployment_id = deployment_id;
self
}
/// Set the event
pub fn set_event(mut self, event: String) -> Self {
self.event = event;
self
}
/// Set the entry type
pub fn set_entry_type(mut self, entry_type: Option<String>) -> Self {
self.entry_type = entry_type;
self
}
/// Set the API details
pub fn set_api(mut self, api: ApiDetails) -> Self {
self.api = api;
self
}
/// Set the remote host
pub fn set_remote_host(mut self, remote_host: Option<String>) -> Self {
self.remote_host = remote_host;
self
}
/// Set the user agent
pub fn set_user_agent(mut self, user_agent: Option<String>) -> Self {
self.user_agent = user_agent;
self
}
/// Set the request path
pub fn set_req_path(mut self, req_path: Option<String>) -> Self {
self.req_path = req_path;
self
}
/// Set the request host
pub fn set_req_host(mut self, req_host: Option<String>) -> Self {
self.req_host = req_host;
self
}
/// Set the request claims
pub fn set_req_claims(mut self, req_claims: Option<HashMap<String, Value>>) -> Self {
self.req_claims = req_claims;
self
}
/// Set the request query
pub fn set_req_query(mut self, req_query: Option<HashMap<String, String>>) -> Self {
self.req_query = req_query;
self
}
/// Set the request header
pub fn set_req_header(mut self, req_header: Option<HashMap<String, String>>) -> Self {
self.req_header = req_header;
self
}
/// Set the response header
pub fn set_resp_header(mut self, resp_header: Option<HashMap<String, String>>) -> Self {
self.resp_header = resp_header;
self
}
/// Set the access key
pub fn set_access_key(mut self, access_key: Option<String>) -> Self {
self.access_key = access_key;
self
}
/// Set the parent user
pub fn set_parent_user(mut self, parent_user: Option<String>) -> Self {
self.parent_user = parent_user;
self
}
/// Set the error
pub fn set_error(mut self, error: Option<String>) -> Self {
self.error = error;
self
}
}
impl LogRecord for AuditLogEntry {
fn to_json(&self) -> String {
serde_json::to_string(self).unwrap_or_else(|_| String::from("{}"))
}
fn get_timestamp(&self) -> DateTime<Utc> {
self.base.timestamp
}
}
crates/obs/src/entry/base.rs
+92
View File
@@ -0,0 +1,92 @@
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::collections::HashMap;
/// Base log entry structure shared by all log types
/// This structure is used to serialize log entries to JSON
/// and send them to the log sinks
/// This structure is also used to deserialize log entries from JSON
/// This structure is also used to store log entries in the database
/// This structure is also used to query log entries from the database
///
/// The `BaseLogEntry` structure contains the following fields:
/// - `timestamp` - the timestamp of the log entry
/// - `request_id` - the request ID of the log entry
/// - `message` - the message of the log entry
/// - `tags` - the tags of the log entry
///
/// The `BaseLogEntry` structure contains the following methods:
/// - `new` - create a new `BaseLogEntry` with default values
/// - `message` - set the message
/// - `request_id` - set the request ID
/// - `tags` - set the tags
/// - `timestamp` - set the timestamp
///
/// # Example
/// ```
/// use rustfs_obs::BaseLogEntry;
/// use chrono::{DateTime, Utc};
/// use std::collections::HashMap;
///
/// let timestamp = Utc::now();
/// let request = Some("req-123".to_string());
/// let message = Some("This is a log message".to_string());
/// let tags = Some(HashMap::new());
///
/// let entry = BaseLogEntry::new()
/// .timestamp(timestamp)
/// .request_id(request)
/// .message(message)
/// .tags(tags);
/// ```
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq, Default)]
pub struct BaseLogEntry {
#[serde(rename = "time")]
pub timestamp: DateTime<Utc>,
#[serde(rename = "requestID", skip_serializing_if = "Option::is_none")]
pub request_id: Option<String>,
#[serde(rename = "message", skip_serializing_if = "Option::is_none")]
pub message: Option<String>,
#[serde(rename = "tags", skip_serializing_if = "Option::is_none")]
pub tags: Option<HashMap<String, Value>>,
}
impl BaseLogEntry {
/// Create a new BaseLogEntry with default values
pub fn new() -> Self {
BaseLogEntry {
timestamp: Utc::now(),
request_id: None,
message: None,
tags: None,
}
}
/// Set the message
pub fn message(mut self, message: Option<String>) -> Self {
self.message = message;
self
}
/// Set the request ID
pub fn request_id(mut self, request_id: Option<String>) -> Self {
self.request_id = request_id;
self
}
/// Set the tags
pub fn tags(mut self, tags: Option<HashMap<String, Value>>) -> Self {
self.tags = tags;
self
}
/// Set the timestamp
pub fn timestamp(mut self, timestamp: DateTime<Utc>) -> Self {
self.timestamp = timestamp;
self
}
}
crates/obs/src/entry/mod.rs
+144
View File
@@ -0,0 +1,144 @@
pub(crate) mod args;
pub(crate) mod audit;
pub(crate) mod base;
pub(crate) mod unified;
use serde::de::Error;
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use tracing_core::Level;
/// ObjectVersion is used across multiple modules
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq)]
pub struct ObjectVersion {
#[serde(rename = "name")]
pub object_name: String,
#[serde(rename = "versionId", skip_serializing_if = "Option::is_none")]
pub version_id: Option<String>,
}
impl ObjectVersion {
/// Create a new ObjectVersion object
pub fn new() -> Self {
ObjectVersion {
object_name: String::new(),
version_id: None,
}
}
/// Create a new ObjectVersion with object name
pub fn new_with_object_name(object_name: String) -> Self {
ObjectVersion {
object_name,
version_id: None,
}
}
/// Set the object name
pub fn set_object_name(mut self, object_name: String) -> Self {
self.object_name = object_name;
self
}
/// Set the version ID
pub fn set_version_id(mut self, version_id: Option<String>) -> Self {
self.version_id = version_id;
self
}
}
impl Default for ObjectVersion {
fn default() -> Self {
Self::new()
}
}
/// Log kind/level enum
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Default)]
pub enum LogKind {
#[serde(rename = "INFO")]
#[default]
Info,
#[serde(rename = "WARNING")]
Warning,
#[serde(rename = "ERROR")]
Error,
#[serde(rename = "FATAL")]
Fatal,
}
/// Trait for types that can be serialized to JSON and have a timestamp
/// This trait is used by `ServerLogEntry` to convert the log entry to JSON
/// and get the timestamp of the log entry
/// This trait is implemented by `ServerLogEntry`
///
/// # Example
/// ```
/// use rustfs_obs::LogRecord;
/// use chrono::{DateTime, Utc};
/// use rustfs_obs::ServerLogEntry;
/// use tracing_core::Level;
///
/// let log_entry = ServerLogEntry::new(Level::INFO, "api_handler".to_string());
/// let json = log_entry.to_json();
/// let timestamp = log_entry.get_timestamp();
/// ```
pub trait LogRecord {
fn to_json(&self) -> String;
fn get_timestamp(&self) -> chrono::DateTime<chrono::Utc>;
}
/// Wrapper for `tracing_core::Level` to implement `Serialize` and `Deserialize`
/// for `ServerLogEntry`
/// This is necessary because `tracing_core::Level` does not implement `Serialize`
/// and `Deserialize`
/// This is a workaround to allow `ServerLogEntry` to be serialized and deserialized
/// using `serde`
///
/// # Example
/// ```
/// use rustfs_obs::SerializableLevel;
/// use tracing_core::Level;
///
/// let level = Level::INFO;
/// let serializable_level = SerializableLevel::from(level);
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SerializableLevel(pub Level);
impl From<Level> for SerializableLevel {
fn from(level: Level) -> Self {
SerializableLevel(level)
}
}
impl From<SerializableLevel> for Level {
fn from(serializable_level: SerializableLevel) -> Self {
serializable_level.0
}
}
impl Serialize for SerializableLevel {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
serializer.serialize_str(self.0.as_str())
}
}
impl<'de> Deserialize<'de> for SerializableLevel {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
match s.as_str() {
"TRACE" => Ok(SerializableLevel(Level::TRACE)),
"DEBUG" => Ok(SerializableLevel(Level::DEBUG)),
"INFO" => Ok(SerializableLevel(Level::INFO)),
"WARN" => Ok(SerializableLevel(Level::WARN)),
"ERROR" => Ok(SerializableLevel(Level::ERROR)),
_ => Err(D::Error::custom("unknown log level")),
}
}
}
crates/obs/src/entry/unified.rs
+287
View File
@@ -0,0 +1,287 @@
use crate::{AuditLogEntry, BaseLogEntry, LogKind, LogRecord, SerializableLevel};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use tracing_core::Level;
/// Server log entry with structured fields
/// ServerLogEntry is used to log structured log entries from the server
///
/// The `ServerLogEntry` structure contains the following fields:
/// - `base` - the base log entry
/// - `level` - the log level
/// - `source` - the source of the log entry
/// - `user_id` - the user ID
/// - `fields` - the structured fields of the log entry
///
/// The `ServerLogEntry` structure contains the following methods:
/// - `new` - create a new `ServerLogEntry` with specified level and source
/// - `with_base` - set the base log entry
/// - `user_id` - set the user ID
/// - `fields` - set the fields
/// - `add_field` - add a field
///
/// # Example
/// ```
/// use rustfs_obs::ServerLogEntry;
/// use tracing_core::Level;
///
/// let entry = ServerLogEntry::new(Level::INFO, "test_module".to_string())
/// .user_id(Some("user-456".to_string()))
/// .add_field("operation".to_string(), "login".to_string());
/// ```
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct ServerLogEntry {
#[serde(flatten)]
pub base: BaseLogEntry,
pub level: SerializableLevel,
pub source: String,
#[serde(rename = "userId", skip_serializing_if = "Option::is_none")]
pub user_id: Option<String>,
#[serde(skip_serializing_if = "Vec::is_empty", default)]
pub fields: Vec<(String, String)>,
}
impl ServerLogEntry {
/// Create a new ServerLogEntry with specified level and source
pub fn new(level: Level, source: String) -> Self {
ServerLogEntry {
base: BaseLogEntry::new(),
level: SerializableLevel(level),
source,
user_id: None,
fields: Vec::new(),
}
}
/// Set the base log entry
pub fn with_base(mut self, base: BaseLogEntry) -> Self {
self.base = base;
self
}
/// Set the user ID
pub fn user_id(mut self, user_id: Option<String>) -> Self {
self.user_id = user_id;
self
}
/// Set fields
pub fn fields(mut self, fields: Vec<(String, String)>) -> Self {
self.fields = fields;
self
}
/// Add a field
pub fn add_field(mut self, key: String, value: String) -> Self {
self.fields.push((key, value));
self
}
}
impl LogRecord for ServerLogEntry {
fn to_json(&self) -> String {
serde_json::to_string(self).unwrap_or_else(|_| String::from("{}"))
}
fn get_timestamp(&self) -> DateTime<Utc> {
self.base.timestamp
}
}
/// Console log entry structure
/// ConsoleLogEntry is used to log console log entries
/// The `ConsoleLogEntry` structure contains the following fields:
/// - `base` - the base log entry
/// - `level` - the log level
/// - `console_msg` - the console message
/// - `node_name` - the node name
/// - `err` - the error message
///
/// The `ConsoleLogEntry` structure contains the following methods:
/// - `new` - create a new `ConsoleLogEntry`
/// - `new_with_console_msg` - create a new `ConsoleLogEntry` with console message and node name
/// - `with_base` - set the base log entry
/// - `set_level` - set the log level
/// - `set_node_name` - set the node name
/// - `set_console_msg` - set the console message
/// - `set_err` - set the error message
///
/// # Example
/// ```
/// use rustfs_obs::ConsoleLogEntry;
///
/// let entry = ConsoleLogEntry::new_with_console_msg("Test message".to_string(), "node-123".to_string());
/// ```
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ConsoleLogEntry {
#[serde(flatten)]
pub base: BaseLogEntry,
pub level: LogKind,
pub console_msg: String,
pub node_name: String,
#[serde(skip)]
pub err: Option<String>,
}
impl ConsoleLogEntry {
/// Create a new ConsoleLogEntry
pub fn new() -> Self {
ConsoleLogEntry {
base: BaseLogEntry::new(),
level: LogKind::Info,
console_msg: String::new(),
node_name: String::new(),
err: None,
}
}
/// Create a new ConsoleLogEntry with console message and node name
pub fn new_with_console_msg(console_msg: String, node_name: String) -> Self {
ConsoleLogEntry {
base: BaseLogEntry::new(),
level: LogKind::Info,
console_msg,
node_name,
err: None,
}
}
/// Set the base log entry
pub fn with_base(mut self, base: BaseLogEntry) -> Self {
self.base = base;
self
}
/// Set the log level
pub fn set_level(mut self, level: LogKind) -> Self {
self.level = level;
self
}
/// Set the node name
pub fn set_node_name(mut self, node_name: String) -> Self {
self.node_name = node_name;
self
}
/// Set the console message
pub fn set_console_msg(mut self, console_msg: String) -> Self {
self.console_msg = console_msg;
self
}
/// Set the error message
pub fn set_err(mut self, err: Option<String>) -> Self {
self.err = err;
self
}
}
impl Default for ConsoleLogEntry {
fn default() -> Self {
Self::new()
}
}
impl LogRecord for ConsoleLogEntry {
fn to_json(&self) -> String {
serde_json::to_string(self).unwrap_or_else(|_| String::from("{}"))
}
fn get_timestamp(&self) -> DateTime<Utc> {
self.base.timestamp
}
}
/// Unified log entry type
/// UnifiedLogEntry is used to log different types of log entries
///
/// The `UnifiedLogEntry` enum contains the following variants:
/// - `Server` - a server log entry
/// - `Audit` - an audit log entry
/// - `Console` - a console log entry
///
/// The `UnifiedLogEntry` enum contains the following methods:
/// - `to_json` - convert the log entry to JSON
/// - `get_timestamp` - get the timestamp of the log entry
///
/// # Example
/// ```
/// use rustfs_obs::{UnifiedLogEntry, ServerLogEntry};
/// use tracing_core::Level;
///
/// let server_entry = ServerLogEntry::new(Level::INFO, "test_module".to_string());
/// let unified = UnifiedLogEntry::Server(server_entry);
/// ```
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type")]
pub enum UnifiedLogEntry {
#[serde(rename = "server")]
Server(ServerLogEntry),
#[serde(rename = "audit")]
Audit(Box<AuditLogEntry>),
#[serde(rename = "console")]
Console(ConsoleLogEntry),
}
impl LogRecord for UnifiedLogEntry {
fn to_json(&self) -> String {
match self {
UnifiedLogEntry::Server(entry) => entry.to_json(),
UnifiedLogEntry::Audit(entry) => entry.to_json(),
UnifiedLogEntry::Console(entry) => entry.to_json(),
}
}
fn get_timestamp(&self) -> DateTime<Utc> {
match self {
UnifiedLogEntry::Server(entry) => entry.get_timestamp(),
UnifiedLogEntry::Audit(entry) => entry.get_timestamp(),
UnifiedLogEntry::Console(entry) => entry.get_timestamp(),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_base_log_entry() {
let base = BaseLogEntry::new()
.request_id(Some("req-123".to_string()))
.message(Some("Test message".to_string()));
assert_eq!(base.request_id, Some("req-123".to_string()));
assert_eq!(base.message, Some("Test message".to_string()));
}
#[test]
fn test_server_log_entry() {
let entry = ServerLogEntry::new(Level::INFO, "test_module".to_string())
.user_id(Some("user-456".to_string()))
.add_field("operation".to_string(), "login".to_string());
assert_eq!(entry.level.0, Level::INFO);
assert_eq!(entry.source, "test_module");
assert_eq!(entry.user_id, Some("user-456".to_string()));
assert_eq!(entry.fields.len(), 1);
assert_eq!(entry.fields[0], ("operation".to_string(), "login".to_string()));
}
#[test]
fn test_unified_log_entry_json() {
let server_entry = ServerLogEntry::new(Level::INFO, "test_source".to_string());
let unified = UnifiedLogEntry::Server(server_entry);
let json = unified.to_json();
assert!(json.contains("test_source"));
}
}
crates/obs/src/global.rs
+89
View File
@@ -0,0 +1,89 @@
use crate::telemetry::OtelGuard;
use std::sync::{Arc, Mutex};
use tokio::sync::{OnceCell, SetError};
use tracing::{error, info};
pub(crate) const USE_STDOUT: bool = true;
pub(crate) const SERVICE_NAME: &str = "RustFS";
pub(crate) const SAMPLE_RATIO: f64 = 1.0;
pub(crate) const METER_INTERVAL: u64 = 60;
pub(crate) const SERVICE_VERSION: &str = "0.1.0";
pub(crate) const ENVIRONMENT: &str = "production";
pub(crate) const LOGGER_LEVEL: &str = "info";
/// Global guard for OpenTelemetry tracing
static GLOBAL_GUARD: OnceCell<Arc<Mutex<OtelGuard>>> = OnceCell::const_new();
/// Error type for global guard operations
#[derive(Debug, thiserror::Error)]
pub enum GuardError {
#[error("Failed to set global guard: {0}")]
SetError(#[from] SetError<Arc<Mutex<OtelGuard>>>),
#[error("Global guard not initialized")]
NotInitialized,
}
/// Set the global guard for OpenTelemetry
///
/// # Arguments
/// * `guard` - The OtelGuard instance to set globally
///
/// # Returns
/// * `Ok(())` if successful
/// * `Err(GuardError)` if setting fails
///
/// # Example
/// ```rust
/// use rustfs_obs::{init_telemetry, load_config, set_global_guard};
///
/// async fn init() -> Result<(), Box<dyn std::error::Error>> {
/// let config = load_config(None);
/// let guard = init_telemetry(&config.observability).await?;
/// set_global_guard(guard)?;
/// Ok(())
/// }
/// ```
pub fn set_global_guard(guard: OtelGuard) -> Result<(), GuardError> {
info!("Initializing global OpenTelemetry guard");
GLOBAL_GUARD.set(Arc::new(Mutex::new(guard))).map_err(GuardError::SetError)
}
/// Get the global guard for OpenTelemetry
///
/// # Returns
/// * `Ok(Arc<Mutex<OtelGuard>>)` if guard exists
/// * `Err(GuardError)` if guard not initialized
///
/// # Example
/// ```rust
/// use rustfs_obs::get_global_guard;
///
/// async fn trace_operation() -> Result<(), Box<dyn std::error::Error>> {
/// let guard = get_global_guard()?;
/// let _lock = guard.lock().unwrap();
/// // Perform traced operation
/// Ok(())
/// }
/// ```
pub fn get_global_guard() -> Result<Arc<Mutex<OtelGuard>>, GuardError> {
GLOBAL_GUARD.get().cloned().ok_or(GuardError::NotInitialized)
}
/// Try to get the global guard for OpenTelemetry
///
/// # Returns
/// * `Some(Arc<Mutex<OtelGuard>>)` if guard exists
/// * `None` if guard not initialized
pub fn try_get_global_guard() -> Option<Arc<Mutex<OtelGuard>>> {
GLOBAL_GUARD.get().cloned()
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_get_uninitialized_guard() {
let result = get_global_guard();
assert!(matches!(result, Err(GuardError::NotInitialized)));
}
}
crates/obs/src/lib.rs
+94
View File
@@ -0,0 +1,94 @@
/// # obs
///
/// `obs` is a logging and observability library for Rust.
/// It provides a simple and easy-to-use interface for logging and observability.
/// It is built on top of the `log` crate and `opentelemetry` crate.
///
/// ## Features
/// - Structured logging
/// - Distributed tracing
/// - Metrics collection
/// - Log processing worker
/// - Multiple sinks
/// - Configuration-based setup
/// - Telemetry guard
/// - Global logger
/// - Log levels
/// - Log entry types
/// - Log record
/// - Object version
/// - Local IP address
///
/// ## Usage
///
/// ```rust
/// use rustfs_obs::{AppConfig, init_obs};
///
/// let config = AppConfig::default();
/// let (logger, guard) = init_obs(config);
/// ```
mod config;
mod entry;
mod global;
mod logger;
mod sink;
mod telemetry;
mod utils;
mod worker;
pub use config::load_config;
pub use config::{AppConfig, OtelConfig};
pub use entry::args::Args;
pub use entry::audit::{ApiDetails, AuditLogEntry};
pub use entry::base::BaseLogEntry;
pub use entry::unified::{ConsoleLogEntry, ServerLogEntry, UnifiedLogEntry};
pub use entry::{LogKind, LogRecord, ObjectVersion, SerializableLevel};
pub use global::{get_global_guard, set_global_guard, try_get_global_guard, GuardError};
pub use logger::{ensure_logger_initialized, log_debug, log_error, log_info, log_trace, log_warn, log_with_context};
pub use logger::{get_global_logger, init_global_logger, locked_logger, start_logger};
pub use logger::{log_init_state, InitLogStatus};
pub use logger::{LogError, Logger};
pub use sink::Sink;
use std::sync::Arc;
pub use telemetry::init_telemetry;
pub use telemetry::{get_global_registry, metrics};
use tokio::sync::Mutex;
pub use utils::{get_local_ip, get_local_ip_with_default};
pub use worker::start_worker;
/// Initialize the observability module
///
/// # Parameters
/// - `config`: Configuration information
///
/// # Returns
/// A tuple containing the logger and the telemetry guard
///
/// # Example
/// ```
/// use rustfs_obs::{AppConfig, init_obs};
///
/// let config = AppConfig::default();
/// let (logger, guard) = init_obs(config);
/// ```
pub async fn init_obs(config: AppConfig) -> (Arc<Mutex<Logger>>, telemetry::OtelGuard) {
let guard = init_telemetry(&config.observability);
let sinks = sink::create_sinks(&config).await;
let logger = init_global_logger(&config, sinks).await;
(logger, guard)
}
/// Get the global logger instance
/// This function returns a reference to the global logger instance.
///
/// # Returns
/// A reference to the global logger instance
///
/// # Example
/// ```
/// use rustfs_obs::get_logger;
///
/// let logger = get_logger();
/// ```
pub fn get_logger() -> &'static Arc<Mutex<Logger>> {
get_global_logger()
}
crates/obs/src/logger.rs
+535
View File
@@ -0,0 +1,535 @@
use crate::global::{ENVIRONMENT, SERVICE_NAME, SERVICE_VERSION};
use crate::{AppConfig, AuditLogEntry, BaseLogEntry, ConsoleLogEntry, OtelConfig, ServerLogEntry, Sink, UnifiedLogEntry};
use std::sync::Arc;
use std::time::SystemTime;
use tokio::sync::mpsc::{self, Receiver, Sender};
use tokio::sync::{Mutex, OnceCell};
use tracing_core::Level;
// Add the global instance at the module level
static GLOBAL_LOGGER: OnceCell<Arc<Mutex<Logger>>> = OnceCell::const_new();
/// Server log processor
#[derive(Debug)]
pub struct Logger {
sender: Sender<UnifiedLogEntry>, // Log sending channel
queue_capacity: usize,
}
impl Logger {
/// Create a new Logger instance
/// Returns Logger and corresponding Receiver
pub fn new(config: &AppConfig) -> (Self, Receiver<UnifiedLogEntry>) {
// Get queue capacity from configuration, or use default values 10000
let queue_capacity = config.logger.as_ref().and_then(|l| l.queue_capacity).unwrap_or(10000);
let (sender, receiver) = mpsc::channel(queue_capacity);
(Logger { sender, queue_capacity }, receiver)
}
/// get the queue capacity
/// This function returns the queue capacity.
/// # Returns
/// The queue capacity
/// # Example
/// ```
/// use rustfs_obs::Logger;
/// async fn example(logger: &Logger) {
/// let _ = logger.get_queue_capacity();
/// }
/// ```
pub fn get_queue_capacity(&self) -> usize {
self.queue_capacity
}
/// Log a server entry
#[tracing::instrument(skip(self), fields(log_source = "logger_server"))]
pub async fn log_server_entry(&self, entry: ServerLogEntry) -> Result<(), LogError> {
self.log_entry(UnifiedLogEntry::Server(entry)).await
}
/// Log an audit entry
#[tracing::instrument(skip(self), fields(log_source = "logger_audit"))]
pub async fn log_audit_entry(&self, entry: AuditLogEntry) -> Result<(), LogError> {
self.log_entry(UnifiedLogEntry::Audit(Box::new(entry))).await
}
/// Log a console entry
#[tracing::instrument(skip(self), fields(log_source = "logger_console"))]
pub async fn log_console_entry(&self, entry: ConsoleLogEntry) -> Result<(), LogError> {
self.log_entry(UnifiedLogEntry::Console(entry)).await
}
/// Asynchronous logging of unified log entries
#[tracing::instrument(skip(self), fields(log_source = "logger"))]
#[tracing::instrument(level = "error", skip_all)]
pub async fn log_entry(&self, entry: UnifiedLogEntry) -> Result<(), LogError> {
// Extract information for tracing based on entry type
match &entry {
UnifiedLogEntry::Server(server) => {
tracing::Span::current()
.record("log_level", server.level.0.as_str())
.record("log_message", server.base.message.as_deref().unwrap_or("log message not set"))
.record("source", &server.source);
// Generate tracing event based on log level
match server.level.0 {
Level::ERROR => {
tracing::error!(target: "server_logs", message = %server.base.message.as_deref().unwrap_or(""));
}
Level::WARN => {
tracing::warn!(target: "server_logs", message = %server.base.message.as_deref().unwrap_or(""));
}
Level::INFO => {
tracing::info!(target: "server_logs", message = %server.base.message.as_deref().unwrap_or(""));
}
Level::DEBUG => {
tracing::debug!(target: "server_logs", message = %server.base.message.as_deref().unwrap_or(""));
}
Level::TRACE => {
tracing::trace!(target: "server_logs", message = %server.base.message.as_deref().unwrap_or(""));
}
}
}
UnifiedLogEntry::Audit(audit) => {
tracing::info!(
target: "audit_logs",
event = %audit.event,
api = %audit.api.name.as_deref().unwrap_or("unknown"),
message = %audit.base.message.as_deref().unwrap_or("")
);
}
UnifiedLogEntry::Console(console) => {
let level_str = match console.level {
crate::LogKind::Info => "INFO",
crate::LogKind::Warning => "WARN",
crate::LogKind::Error => "ERROR",
crate::LogKind::Fatal => "FATAL",
};
tracing::info!(
target: "console_logs",
level = %level_str,
node = %console.node_name,
message = %console.console_msg
);
}
}
// Send logs to async queue with improved error handling
match self.sender.try_send(entry) {
Ok(_) => Ok(()),
Err(mpsc::error::TrySendError::Full(entry)) => {
// Processing strategy when queue is full
tracing::warn!("Log queue full, applying backpressure");
match tokio::time::timeout(std::time::Duration::from_millis(500), self.sender.send(entry)).await {
Ok(Ok(_)) => Ok(()),
Ok(Err(_)) => Err(LogError::SendFailed("Channel closed")),
Err(_) => Err(LogError::Timeout("Queue backpressure timeout")),
}
}
Err(mpsc::error::TrySendError::Closed(_)) => Err(LogError::SendFailed("Logger channel closed")),
}
}
/// Write log with context information
/// This function writes log messages with context information.
///
/// # Parameters
/// - `message`: Message to be logged
/// - `source`: Source of the log
/// - `request_id`: Request ID
/// - `user_id`: User ID
/// - `fields`: Additional fields
///
/// # Returns
/// Result indicating whether the operation was successful
///
/// # Example
/// ```
/// use tracing_core::Level;
/// use rustfs_obs::Logger;
///
/// async fn example(logger: &Logger) {
/// let _ = logger.write_with_context("This is an information message", "example",Level::INFO, Some("req-12345".to_string()), Some("user-6789".to_string()), vec![("endpoint".to_string(), "/api/v1/data".to_string())]).await;
/// }
pub async fn write_with_context(
&self,
message: &str,
source: &str,
level: Level,
request_id: Option<String>,
user_id: Option<String>,
fields: Vec<(String, String)>,
) -> Result<(), LogError> {
let base = BaseLogEntry::new().message(Some(message.to_string())).request_id(request_id);
let server_entry = ServerLogEntry::new(level, source.to_string())
.user_id(user_id)
.fields(fields)
.with_base(base);
self.log_server_entry(server_entry).await
}
/// Write log
/// This function writes log messages.
/// # Parameters
/// - `message`: Message to be logged
/// - `source`: Source of the log
/// - `level`: Log level
///
/// # Returns
/// Result indicating whether the operation was successful
///
/// # Example
/// ```
/// use rustfs_obs::Logger;
/// use tracing_core::Level;
///
/// async fn example(logger: &Logger) {
/// let _ = logger.write("This is an information message", "example", Level::INFO).await;
/// }
/// ```
pub async fn write(&self, message: &str, source: &str, level: Level) -> Result<(), LogError> {
self.write_with_context(message, source, level, None, None, Vec::new()).await
}
/// Shutdown the logger
/// This function shuts down the logger.
///
/// # Returns
/// Result indicating whether the operation was successful
///
/// # Example
/// ```
/// use rustfs_obs::Logger;
///
/// async fn example(logger: Logger) {
/// let _ = logger.shutdown().await;
/// }
/// ```
pub async fn shutdown(self) -> Result<(), LogError> {
drop(self.sender); //Close the sending end so that the receiver knows that there is no new message
Ok(())
}
}
/// Log error type
/// This enum defines the error types that can occur when logging.
/// It is used to provide more detailed error information.
/// # Example
/// ```
/// use rustfs_obs::LogError;
/// use thiserror::Error;
///
/// LogError::SendFailed("Failed to send log");
/// LogError::Timeout("Operation timed out");
/// ```
#[derive(Debug, thiserror::Error)]
pub enum LogError {
#[error("Failed to send log: {0}")]
SendFailed(&'static str),
#[error("Operation timed out: {0}")]
Timeout(&'static str),
}
/// Start the log module
/// This function starts the log module.
/// It initializes the logger and starts the worker to process logs.
/// # Parameters
/// - `config`: Configuration information
/// - `sinks`: A vector of Sink instances
/// # Returns
/// The global logger instance
/// # Example
/// ```
/// use rustfs_obs::{AppConfig, start_logger};
///
/// let config = AppConfig::default();
/// let sinks = vec![];
/// let logger = start_logger(&config, sinks);
/// ```
pub fn start_logger(config: &AppConfig, sinks: Vec<Arc<dyn Sink>>) -> Logger {
let (logger, receiver) = Logger::new(config);
tokio::spawn(crate::worker::start_worker(receiver, sinks));
logger
}
/// Initialize the global logger instance
/// This function initializes the global logger instance and returns a reference to it.
/// If the logger has been initialized before, it will return the existing logger instance.
///
/// # Parameters
/// - `config`: Configuration information
/// - `sinks`: A vector of Sink instances
///
/// # Returns
/// A reference to the global logger instance
///
/// # Example
/// ```
/// use rustfs_obs::{AppConfig,init_global_logger};
///
/// let config = AppConfig::default();
/// let sinks = vec![];
/// let logger = init_global_logger(&config, sinks);
/// ```
pub async fn init_global_logger(config: &AppConfig, sinks: Vec<Arc<dyn Sink>>) -> Arc<Mutex<Logger>> {
let logger = Arc::new(Mutex::new(start_logger(config, sinks)));
GLOBAL_LOGGER.set(logger.clone()).expect("Logger already initialized");
logger
}
/// Get the global logger instance
///
/// This function returns a reference to the global logger instance.
///
/// # Returns
/// A reference to the global logger instance
///
/// # Example
/// ```
/// use rustfs_obs::get_global_logger;
///
/// let logger = get_global_logger();
/// ```
pub fn get_global_logger() -> &'static Arc<Mutex<Logger>> {
GLOBAL_LOGGER.get().expect("Logger not initialized")
}
/// Get the global logger instance with a lock
/// This function returns a reference to the global logger instance with a lock.
/// It is used to ensure that the logger is thread-safe.
///
/// # Returns
/// A reference to the global logger instance with a lock
///
/// # Example
/// ```
/// use rustfs_obs::locked_logger;
///
/// async fn example() {
/// let logger = locked_logger().await;
/// }
/// ```
pub async fn locked_logger() -> tokio::sync::MutexGuard<'static, Logger> {
get_global_logger().lock().await
}
/// Initialize with default empty logger if needed (optional)
/// This function initializes the logger with a default empty logger if needed.
/// It is used to ensure that the logger is initialized before logging.
///
/// # Returns
/// A reference to the global logger instance
///
/// # Example
/// ```
/// use rustfs_obs::ensure_logger_initialized;
///
/// let logger = ensure_logger_initialized();
/// ```
pub fn ensure_logger_initialized() -> &'static Arc<Mutex<Logger>> {
if GLOBAL_LOGGER.get().is_none() {
let config = AppConfig::default();
let sinks = vec![];
let logger = Arc::new(Mutex::new(start_logger(&config, sinks)));
let _ = GLOBAL_LOGGER.set(logger);
}
GLOBAL_LOGGER.get().unwrap()
}
/// Log information
/// This function logs information messages.
///
/// # Parameters
/// - `message`: Message to be logged
/// - `source`: Source of the log
///
/// # Returns
/// Result indicating whether the operation was successful
///
/// # Example
/// ```
/// use rustfs_obs::log_info;
///
/// async fn example() {
/// let _ = log_info("This is an information message", "example").await;
/// }
/// ```
pub async fn log_info(message: &str, source: &str) -> Result<(), LogError> {
get_global_logger().lock().await.write(message, source, Level::INFO).await
}
/// Log error
/// This function logs error messages.
/// # Parameters
/// - `message`: Message to be logged
/// - `source`: Source of the log
/// # Returns
/// Result indicating whether the operation was successful
/// # Example
/// ```
/// use rustfs_obs::log_error;
///
/// async fn example() {
/// let _ = log_error("This is an error message", "example").await;
/// }
pub async fn log_error(message: &str, source: &str) -> Result<(), LogError> {
get_global_logger().lock().await.write(message, source, Level::ERROR).await
}
/// Log warning
/// This function logs warning messages.
/// # Parameters
/// - `message`: Message to be logged
/// - `source`: Source of the log
/// # Returns
/// Result indicating whether the operation was successful
///
/// # Example
/// ```
/// use rustfs_obs::log_warn;
///
/// async fn example() {
/// let _ = log_warn("This is a warning message", "example").await;
/// }
/// ```
pub async fn log_warn(message: &str, source: &str) -> Result<(), LogError> {
get_global_logger().lock().await.write(message, source, Level::WARN).await
}
/// Log debug
/// This function logs debug messages.
/// # Parameters
/// - `message`: Message to be logged
/// - `source`: Source of the log
/// # Returns
/// Result indicating whether the operation was successful
///
/// # Example
/// ```
/// use rustfs_obs::log_debug;
///
/// async fn example() {
/// let _ = log_debug("This is a debug message", "example").await;
/// }
/// ```
pub async fn log_debug(message: &str, source: &str) -> Result<(), LogError> {
get_global_logger().lock().await.write(message, source, Level::DEBUG).await
}
/// Log trace
/// This function logs trace messages.
/// # Parameters
/// - `message`: Message to be logged
/// - `source`: Source of the log
///
/// # Returns
/// Result indicating whether the operation was successful
///
/// # Example
/// ```
/// use rustfs_obs::log_trace;
///
/// async fn example() {
/// let _ = log_trace("This is a trace message", "example").await;
/// }
/// ```
pub async fn log_trace(message: &str, source: &str) -> Result<(), LogError> {
get_global_logger().lock().await.write(message, source, Level::TRACE).await
}
/// Log with context information
/// This function logs messages with context information.
/// # Parameters
/// - `message`: Message to be logged
/// - `source`: Source of the log
/// - `level`: Log level
/// - `request_id`: Request ID
/// - `user_id`: User ID
/// - `fields`: Additional fields
/// # Returns
/// Result indicating whether the operation was successful
/// # Example
/// ```
/// use tracing_core::Level;
/// use rustfs_obs::log_with_context;
///
/// async fn example() {
/// let _ = log_with_context("This is an information message", "example", Level::INFO, Some("req-12345".to_string()), Some("user-6789".to_string()), vec![("endpoint".to_string(), "/api/v1/data".to_string())]).await;
/// }
/// ```
pub async fn log_with_context(
message: &str,
source: &str,
level: Level,
request_id: Option<String>,
user_id: Option<String>,
fields: Vec<(String, String)>,
) -> Result<(), LogError> {
get_global_logger()
.lock()
.await
.write_with_context(message, source, level, request_id, user_id, fields)
.await
}
/// Log initialization status
#[derive(Debug)]
pub struct InitLogStatus {
pub timestamp: SystemTime,
pub service_name: String,
pub version: String,
pub environment: String,
}
impl Default for InitLogStatus {
fn default() -> Self {
Self {
timestamp: SystemTime::now(),
service_name: String::from(SERVICE_NAME),
version: SERVICE_VERSION.to_string(),
environment: ENVIRONMENT.to_string(),
}
}
}
impl InitLogStatus {
pub fn new_config(config: &OtelConfig) -> Self {
let config = config.clone();
let environment = config.environment.unwrap_or(ENVIRONMENT.to_string());
let version = config.service_version.unwrap_or(SERVICE_VERSION.to_string());
Self {
timestamp: SystemTime::now(),
service_name: String::from(SERVICE_NAME),
version,
environment,
}
}
pub async fn init_start_log(config: &OtelConfig) -> Result<(), LogError> {
let status = Self::new_config(config);
log_init_state(Some(status)).await
}
}
/// Log initialization details during system startup
pub async fn log_init_state(status: Option<InitLogStatus>) -> Result<(), LogError> {
let status = status.unwrap_or_default();
let base_entry = BaseLogEntry::new()
.timestamp(chrono::DateTime::from(status.timestamp))
.message(Some(format!(
"Service initialization started - {} v{} in {}",
status.service_name, status.version, status.environment
)))
.request_id(Some("system_init".to_string()));
let server_entry = ServerLogEntry::new(Level::INFO, "system_initialization".to_string())
.with_base(base_entry)
.user_id(Some("system".to_string()));
get_global_logger().lock().await.log_server_entry(server_entry).await?;
Ok(())
}
crates/obs/src/sink.rs
+497
View File
@@ -0,0 +1,497 @@
use crate::{AppConfig, LogRecord, UnifiedLogEntry};
use async_trait::async_trait;
use std::sync::Arc;
use tokio::fs::OpenOptions;
use tokio::io;
use tokio::io::AsyncWriteExt;
/// Sink Trait definition, asynchronously write logs
#[async_trait]
pub trait Sink: Send + Sync {
async fn write(&self, entry: &UnifiedLogEntry);
}
#[cfg(feature = "kafka")]
/// Kafka Sink Implementation
pub struct KafkaSink {
producer: rdkafka::producer::FutureProducer,
topic: String,
batch_size: usize,
batch_timeout_ms: u64,
entries: Arc<tokio::sync::Mutex<Vec<UnifiedLogEntry>>>,
last_flush: Arc<std::sync::atomic::AtomicU64>,
}
#[cfg(feature = "kafka")]
impl KafkaSink {
/// Create a new KafkaSink instance
pub fn new(producer: rdkafka::producer::FutureProducer, topic: String, batch_size: usize, batch_timeout_ms: u64) -> Self {
// Create Arc-wrapped values first
let entries = Arc::new(tokio::sync::Mutex::new(Vec::with_capacity(batch_size)));
let last_flush = Arc::new(std::sync::atomic::AtomicU64::new(
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64,
));
let sink = KafkaSink {
producer: producer.clone(),
topic: topic.clone(),
batch_size,
batch_timeout_ms,
entries: entries.clone(),
last_flush: last_flush.clone(),
};
// Start background flusher
tokio::spawn(Self::periodic_flush(producer, topic, entries, last_flush, batch_timeout_ms));
sink
}
/// Add a getter method to read the batch_timeout_ms field
#[allow(dead_code)]
pub fn batch_timeout(&self) -> u64 {
self.batch_timeout_ms
}
/// Add a method to dynamically adjust the timeout if needed
#[allow(dead_code)]
pub fn set_batch_timeout(&mut self, new_timeout_ms: u64) {
self.batch_timeout_ms = new_timeout_ms;
}
async fn periodic_flush(
producer: rdkafka::producer::FutureProducer,
topic: String,
entries: Arc<tokio::sync::Mutex<Vec<UnifiedLogEntry>>>,
last_flush: Arc<std::sync::atomic::AtomicU64>,
timeout_ms: u64,
) {
loop {
tokio::time::sleep(tokio::time::Duration::from_millis(timeout_ms / 2)).await;
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64;
let last = last_flush.load(std::sync::atomic::Ordering::Relaxed);
if now - last >= timeout_ms {
let mut batch = entries.lock().await;
if !batch.is_empty() {
Self::send_batch(&producer, &topic, batch.drain(..).collect()).await;
last_flush.store(now, std::sync::atomic::Ordering::Relaxed);
}
}
}
}
async fn send_batch(producer: &rdkafka::producer::FutureProducer, topic: &str, entries: Vec<UnifiedLogEntry>) {
for entry in entries {
let payload = match serde_json::to_string(&entry) {
Ok(p) => p,
Err(e) => {
eprintln!("Failed to serialize log entry: {}", e);
continue;
}
};
let span_id = entry.get_timestamp().to_rfc3339();
let _ = producer
.send(
rdkafka::producer::FutureRecord::to(topic).payload(&payload).key(&span_id),
std::time::Duration::from_secs(5),
)
.await;
}
}
}
#[cfg(feature = "kafka")]
#[async_trait]
impl Sink for KafkaSink {
async fn write(&self, entry: &UnifiedLogEntry) {
let mut batch = self.entries.lock().await;
batch.push(entry.clone());
let should_flush_by_size = batch.len() >= self.batch_size;
let should_flush_by_time = {
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64;
let last = self.last_flush.load(std::sync::atomic::Ordering::Relaxed);
now - last >= self.batch_timeout_ms
};
if should_flush_by_size || should_flush_by_time {
// Existing flush logic
let entries_to_send: Vec<UnifiedLogEntry> = batch.drain(..).collect();
let producer = self.producer.clone();
let topic = self.topic.clone();
self.last_flush.store(
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64,
std::sync::atomic::Ordering::Relaxed,
);
tokio::spawn(async move {
KafkaSink::send_batch(&producer, &topic, entries_to_send).await;
});
}
}
}
#[cfg(feature = "kafka")]
impl Drop for KafkaSink {
fn drop(&mut self) {
// Perform any necessary cleanup here
// For example, you might want to flush any remaining entries
let producer = self.producer.clone();
let topic = self.topic.clone();
let entries = self.entries.clone();
let last_flush = self.last_flush.clone();
tokio::spawn(async move {
let mut batch = entries.lock().await;
if !batch.is_empty() {
KafkaSink::send_batch(&producer, &topic, batch.drain(..).collect()).await;
last_flush.store(
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64,
std::sync::atomic::Ordering::Relaxed,
);
}
});
eprintln!("Dropping KafkaSink with topic: {}", self.topic);
}
}
#[cfg(feature = "webhook")]
/// Webhook Sink Implementation
pub struct WebhookSink {
endpoint: String,
auth_token: String,
client: reqwest::Client,
max_retries: usize,
retry_delay_ms: u64,
}
#[cfg(feature = "webhook")]
impl WebhookSink {
pub fn new(endpoint: String, auth_token: String, max_retries: usize, retry_delay_ms: u64) -> Self {
WebhookSink {
endpoint,
auth_token,
client: reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()
.unwrap_or_else(|_| reqwest::Client::new()),
max_retries,
retry_delay_ms,
}
}
}
#[cfg(feature = "webhook")]
#[async_trait]
impl Sink for WebhookSink {
async fn write(&self, entry: &UnifiedLogEntry) {
let mut retries = 0;
let url = self.endpoint.clone();
let entry_clone = entry.clone();
let auth_value = reqwest::header::HeaderValue::from_str(format!("Bearer {}", self.auth_token.clone()).as_str()).unwrap();
while retries < self.max_retries {
match self
.client
.post(&url)
.header(reqwest::header::AUTHORIZATION, auth_value.clone())
.json(&entry_clone)
.send()
.await
{
Ok(response) if response.status().is_success() => {
return;
}
_ => {
retries += 1;
if retries < self.max_retries {
tokio::time::sleep(tokio::time::Duration::from_millis(
self.retry_delay_ms * (1 << retries), // Exponential backoff
))
.await;
}
}
}
}
eprintln!("Failed to send log to webhook after {} retries", self.max_retries);
}
}
#[cfg(feature = "webhook")]
impl Drop for WebhookSink {
fn drop(&mut self) {
// Perform any necessary cleanup here
// For example, you might want to log that the sink is being dropped
eprintln!("Dropping WebhookSink with URL: {}", self.endpoint);
}
}
#[cfg(feature = "file")]
/// File Sink Implementation
pub struct FileSink {
path: String,
buffer_size: usize,
writer: Arc<tokio::sync::Mutex<io::BufWriter<tokio::fs::File>>>,
entry_count: std::sync::atomic::AtomicUsize,
last_flush: std::sync::atomic::AtomicU64,
flush_interval_ms: u64, // Time between flushes
flush_threshold: usize, // Number of entries before flush
}
#[cfg(feature = "file")]
impl FileSink {
/// Create a new FileSink instance
pub async fn new(
path: String,
buffer_size: usize,
flush_interval_ms: u64,
flush_threshold: usize,
) -> Result<Self, io::Error> {
// check if the file exists
let file_exists = tokio::fs::metadata(&path).await.is_ok();
// if the file not exists, create it
if !file_exists {
tokio::fs::create_dir_all(std::path::Path::new(&path).parent().unwrap()).await?;
tracing::debug!("File does not exist, creating it. Path: {:?}", path)
}
let file = if file_exists {
// If the file exists, open it in append mode
tracing::debug!("FileSink: File exists, opening in append mode.");
OpenOptions::new().append(true).create(true).open(&path).await?
} else {
// If the file does not exist, create it
tracing::debug!("FileSink: File does not exist, creating a new file.");
// Create the file and write a header or initial content if needed
OpenOptions::new().create(true).truncate(true).write(true).open(&path).await?
};
let writer = io::BufWriter::with_capacity(buffer_size, file);
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64;
Ok(FileSink {
path,
buffer_size,
writer: Arc::new(tokio::sync::Mutex::new(writer)),
entry_count: std::sync::atomic::AtomicUsize::new(0),
last_flush: std::sync::atomic::AtomicU64::new(now),
flush_interval_ms,
flush_threshold,
})
}
#[allow(dead_code)]
async fn initialize_writer(&mut self) -> io::Result<()> {
let file = tokio::fs::File::create(&self.path).await?;
// Use buffer_size to create a buffer writer with a specified capacity
let buf_writer = io::BufWriter::with_capacity(self.buffer_size, file);
// Replace the original writer with the new Mutex
self.writer = Arc::new(tokio::sync::Mutex::new(buf_writer));
Ok(())
}
// Get the current buffer size
#[allow(dead_code)]
pub fn buffer_size(&self) -> usize {
self.buffer_size
}
// How to dynamically adjust the buffer size
#[allow(dead_code)]
pub async fn set_buffer_size(&mut self, new_size: usize) -> io::Result<()> {
if self.buffer_size != new_size {
self.buffer_size = new_size;
// Reinitialize the writer directly, without checking is_some()
self.initialize_writer().await?;
}
Ok(())
}
// Check if flushing is needed based on count or time
fn should_flush(&self) -> bool {
// Check entry count threshold
if self.entry_count.load(std::sync::atomic::Ordering::Relaxed) >= self.flush_threshold {
return true;
}
// Check time threshold
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64;
let last = self.last_flush.load(std::sync::atomic::Ordering::Relaxed);
now - last >= self.flush_interval_ms
}
}
#[cfg(feature = "file")]
#[async_trait]
impl Sink for FileSink {
async fn write(&self, entry: &UnifiedLogEntry) {
let line = format!("{:?}\n", entry);
let mut writer = self.writer.lock().await;
if let Err(e) = writer.write_all(line.as_bytes()).await {
eprintln!(
"Failed to write log to file {}: {},entry timestamp:{:?}",
self.path,
e,
entry.get_timestamp()
);
return;
}
// Only flush periodically to improve performance
// Logic to determine when to flush could be added here
// Increment the entry count
self.entry_count.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
// Check if we should flush
if self.should_flush() {
if let Err(e) = writer.flush().await {
eprintln!("Failed to flush log file {}: {}", self.path, e);
return;
}
// Reset counters
self.entry_count.store(0, std::sync::atomic::Ordering::Relaxed);
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_millis() as u64;
self.last_flush.store(now, std::sync::atomic::Ordering::Relaxed);
}
}
}
#[cfg(feature = "file")]
impl Drop for FileSink {
fn drop(&mut self) {
let writer = self.writer.clone();
let path = self.path.clone();
tokio::task::spawn_blocking(move || {
let rt = tokio::runtime::Runtime::new().unwrap();
rt.block_on(async {
let mut writer = writer.lock().await;
if let Err(e) = writer.flush().await {
eprintln!("Failed to flush log file {}: {}", path, e);
}
});
});
}
}
/// Create a list of Sink instances
pub async fn create_sinks(config: &AppConfig) -> Vec<Arc<dyn Sink>> {
let mut sinks: Vec<Arc<dyn Sink>> = Vec::new();
#[cfg(feature = "kafka")]
{
match &config.sinks.kafka {
Some(sink_kafka) => {
if sink_kafka.enabled {
match rdkafka::config::ClientConfig::new()
.set("bootstrap.servers", &sink_kafka.bootstrap_servers)
.set("message.timeout.ms", "5000")
.create()
{
Ok(producer) => {
sinks.push(Arc::new(KafkaSink::new(
producer,
sink_kafka.topic.clone(),
sink_kafka.batch_size.unwrap_or(100),
sink_kafka.batch_timeout_ms.unwrap_or(1000),
)));
}
Err(e) => {
tracing::error!("Failed to create Kafka producer: {}", e);
}
}
} else {
tracing::info!("Kafka sink is disabled in the configuration");
}
}
_ => {
tracing::info!("Kafka sink is not configured or disabled");
}
}
}
#[cfg(feature = "webhook")]
{
match &config.sinks.webhook {
Some(sink_webhook) => {
if sink_webhook.enabled {
sinks.push(Arc::new(WebhookSink::new(
sink_webhook.endpoint.clone(),
sink_webhook.auth_token.clone(),
sink_webhook.max_retries.unwrap_or(3),
sink_webhook.retry_delay_ms.unwrap_or(100),
)));
} else {
tracing::info!("Webhook sink is disabled in the configuration");
}
}
_ => {
tracing::info!("Webhook sink is not configured or disabled");
}
}
}
#[cfg(feature = "file")]
{
// let config = config.clone();
match &config.sinks.file {
Some(sink_file) => {
tracing::info!("File sink is enabled in the configuration");
let path = if sink_file.enabled {
sink_file.path.clone()
} else {
"rustfs.log".to_string()
};
tracing::debug!("FileSink: Using path: {}", path);
sinks.push(Arc::new(
FileSink::new(
path.clone(),
sink_file.buffer_size.unwrap_or(8192),
sink_file.flush_interval_ms.unwrap_or(1000),
sink_file.flush_threshold.unwrap_or(100),
)
.await
.unwrap(),
));
}
_ => {
tracing::info!("File sink is not configured or disabled");
}
}
}
sinks
}
crates/obs/src/telemetry.rs
+319
View File
@@ -0,0 +1,319 @@
use crate::global::{ENVIRONMENT, LOGGER_LEVEL, METER_INTERVAL, SAMPLE_RATIO, SERVICE_NAME, SERVICE_VERSION, USE_STDOUT};
use crate::{get_local_ip_with_default, OtelConfig};
use opentelemetry::trace::TracerProvider;
use opentelemetry::{global, KeyValue};
use opentelemetry_appender_tracing::layer;
use opentelemetry_otlp::WithExportConfig;
use opentelemetry_sdk::logs::SdkLoggerProvider;
use opentelemetry_sdk::{
metrics::{MeterProviderBuilder, PeriodicReader, SdkMeterProvider},
trace::{RandomIdGenerator, Sampler, SdkTracerProvider},
Resource,
};
use opentelemetry_semantic_conventions::{
attribute::{DEPLOYMENT_ENVIRONMENT_NAME, NETWORK_LOCAL_ADDRESS, SERVICE_VERSION as OTEL_SERVICE_VERSION},
SCHEMA_URL,
};
use prometheus::{Encoder, Registry, TextEncoder};
use std::io::IsTerminal;
use std::sync::Arc;
use tokio::sync::{Mutex, OnceCell};
use tracing::{info, warn};
use tracing_error::ErrorLayer;
use tracing_opentelemetry::{MetricsLayer, OpenTelemetryLayer};
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt, EnvFilter, Layer};
/// A guard object that manages the lifecycle of OpenTelemetry components.
///
/// This struct holds references to the created OpenTelemetry providers and ensures
/// they are properly shut down when the guard is dropped. It implements the RAII
/// (Resource Acquisition Is Initialization) pattern for managing telemetry resources.
///
/// When this guard goes out of scope, it will automatically shut down:
/// - The tracer provider (for distributed tracing)
/// - The meter provider (for metrics collection)
/// - The logger provider (for structured logging)
///
/// # Example
///
/// ```
/// use rustfs_obs::{init_telemetry, OtelConfig};
///
/// let config = OtelConfig::default();
/// let otel_guard = init_telemetry(&config);
///
/// // The guard is kept alive for the duration of the application
/// // When it's dropped, all telemetry components are properly shut down
/// drop(otel_guard);
/// ```
#[derive(Debug)]
pub struct OtelGuard {
tracer_provider: SdkTracerProvider,
meter_provider: SdkMeterProvider,
logger_provider: SdkLoggerProvider,
}
impl Drop for OtelGuard {
fn drop(&mut self) {
if let Err(err) = self.tracer_provider.shutdown() {
eprintln!("Tracer shutdown error: {:?}", err);
}
if let Err(err) = self.meter_provider.shutdown() {
eprintln!("Meter shutdown error: {:?}", err);
}
if let Err(err) = self.logger_provider.shutdown() {
eprintln!("Logger shutdown error: {:?}", err);
}
}
}
/// Global registry for Prometheus metrics
static GLOBAL_REGISTRY: OnceCell<Arc<Mutex<Registry>>> = OnceCell::const_new();
/// Get the global registry instance
/// This function returns a reference to the global registry instance.
///
/// # Returns
/// A reference to the global registry instance
///
/// # Example
/// ```
/// use rustfs_obs::get_global_registry;
///
/// let registry = get_global_registry();
/// ```
pub fn get_global_registry() -> Arc<Mutex<Registry>> {
GLOBAL_REGISTRY.get().unwrap().clone()
}
/// Prometheus metric endpoints
/// This function returns a string containing the Prometheus metrics.
/// The metrics are collected from the global registry.
/// The function is used to expose the metrics via an HTTP endpoint.
///
/// # Returns
/// A string containing the Prometheus metrics
///
/// # Example
/// ```
/// use rustfs_obs::metrics;
///
/// async fn main() {
/// let metrics = metrics().await;
/// println!("{}", metrics);
/// }
/// ```
pub async fn metrics() -> String {
let encoder = TextEncoder::new();
// Get a reference to the registry for reading metrics
let registry = get_global_registry().lock().await.to_owned();
let metric_families = registry.gather();
if metric_families.is_empty() {
warn!("No metrics available in Prometheus registry");
} else {
info!("Metrics collected: {} families", metric_families.len());
}
let mut buffer = Vec::new();
encoder.encode(&metric_families, &mut buffer).unwrap();
String::from_utf8(buffer).unwrap_or_else(|_| "Error encoding metrics".to_string())
}
/// create OpenTelemetry Resource
fn resource(config: &OtelConfig) -> Resource {
let config = config.clone();
Resource::builder()
.with_service_name(config.service_name.unwrap_or(SERVICE_NAME.to_string()))
.with_schema_url(
[
KeyValue::new(OTEL_SERVICE_VERSION, config.service_version.unwrap_or(SERVICE_VERSION.to_string())),
KeyValue::new(DEPLOYMENT_ENVIRONMENT_NAME, config.environment.unwrap_or(ENVIRONMENT.to_string())),
KeyValue::new(NETWORK_LOCAL_ADDRESS, get_local_ip_with_default()),
],
SCHEMA_URL,
)
.build()
}
/// Creates a periodic reader for stdout metrics
fn create_periodic_reader(interval: u64) -> PeriodicReader<opentelemetry_stdout::MetricExporter> {
PeriodicReader::builder(opentelemetry_stdout::MetricExporter::default())
.with_interval(std::time::Duration::from_secs(interval))
.build()
}
/// Initialize Meter Provider
fn init_meter_provider(config: &OtelConfig) -> SdkMeterProvider {
let mut builder = MeterProviderBuilder::default().with_resource(resource(config));
// If endpoint is empty, use stdout output
if config.endpoint.is_empty() {
builder = builder.with_reader(create_periodic_reader(config.meter_interval.unwrap_or(METER_INTERVAL)));
} else {
// If endpoint is not empty, use otlp output
let exporter = opentelemetry_otlp::MetricExporter::builder()
.with_tonic()
.with_endpoint(&config.endpoint)
.with_temporality(opentelemetry_sdk::metrics::Temporality::default())
.build()
.unwrap();
builder = builder.with_reader(
PeriodicReader::builder(exporter)
.with_interval(std::time::Duration::from_secs(config.meter_interval.unwrap_or(METER_INTERVAL)))
.build(),
);
// If use_stdout is true, output to stdout at the same time
if config.use_stdout.unwrap_or(USE_STDOUT) {
builder = builder.with_reader(create_periodic_reader(config.meter_interval.unwrap_or(METER_INTERVAL)));
}
}
let registry = Registry::new();
// Set global registry
GLOBAL_REGISTRY.set(Arc::new(Mutex::new(registry.clone()))).unwrap();
// Create Prometheus exporter
let prometheus_exporter = opentelemetry_prometheus::exporter().with_registry(registry).build().unwrap();
// Build meter provider
let meter_provider = builder.with_reader(prometheus_exporter).build();
global::set_meter_provider(meter_provider.clone());
meter_provider
}
/// Initialize Tracer Provider
fn init_tracer_provider(config: &OtelConfig) -> SdkTracerProvider {
let sample_ratio = config.sample_ratio.unwrap_or(SAMPLE_RATIO);
let sampler = if sample_ratio > 0.0 && sample_ratio < 1.0 {
Sampler::TraceIdRatioBased(sample_ratio)
} else {
Sampler::AlwaysOn
};
let builder = SdkTracerProvider::builder()
.with_sampler(sampler)
.with_id_generator(RandomIdGenerator::default())
.with_resource(resource(config));
let tracer_provider = if config.endpoint.is_empty() {
builder
.with_batch_exporter(opentelemetry_stdout::SpanExporter::default())
.build()
} else {
let exporter = opentelemetry_otlp::SpanExporter::builder()
.with_tonic()
.with_endpoint(&config.endpoint)
.build()
.unwrap();
if config.use_stdout.unwrap_or(USE_STDOUT) {
builder
.with_batch_exporter(exporter)
.with_batch_exporter(opentelemetry_stdout::SpanExporter::default())
} else {
builder.with_batch_exporter(exporter)
}
.build()
};
global::set_tracer_provider(tracer_provider.clone());
tracer_provider
}
/// Initialize Telemetry
pub fn init_telemetry(config: &OtelConfig) -> OtelGuard {
let tracer_provider = init_tracer_provider(config);
let meter_provider = init_meter_provider(config);
// Initialize logger provider based on configuration
let logger_provider = {
let mut builder = SdkLoggerProvider::builder().with_resource(resource(config));
if config.endpoint.is_empty() {
// Use stdout exporter when no endpoint is configured
builder = builder.with_simple_exporter(opentelemetry_stdout::LogExporter::default());
} else {
// Configure OTLP exporter when endpoint is provided
let exporter = opentelemetry_otlp::LogExporter::builder()
.with_tonic()
.with_endpoint(&config.endpoint)
.build()
.unwrap();
builder = builder.with_batch_exporter(exporter);
// Add stdout exporter if requested
if config.use_stdout.unwrap_or(USE_STDOUT) {
builder = builder.with_batch_exporter(opentelemetry_stdout::LogExporter::default());
}
}
builder.build()
};
let config = config.clone();
let logger_level = config.logger_level.unwrap_or(LOGGER_LEVEL.to_string());
let logger_level = logger_level.as_str();
// Setup OpenTelemetryTracingBridge layer
let otel_layer = {
// Filter to prevent infinite telemetry loops
// This blocks events from OpenTelemetry and its dependent libraries (tonic, reqwest, etc.)
// from being sent back to OpenTelemetry itself
let filter_otel = match logger_level {
"trace" | "debug" => {
info!("OpenTelemetry tracing initialized with level: {}", logger_level);
EnvFilter::new(logger_level)
}
_ => {
let mut filter = EnvFilter::new(logger_level);
for directive in ["hyper", "tonic", "h2", "reqwest"] {
filter = filter.add_directive(format!("{}=off", directive).parse().unwrap());
}
filter
}
};
layer::OpenTelemetryTracingBridge::new(&logger_provider).with_filter(filter_otel)
};
let tracer = tracer_provider.tracer(config.service_name.unwrap_or(SERVICE_NAME.to_string()));
let registry = tracing_subscriber::registry()
.with(switch_level(logger_level))
.with(OpenTelemetryLayer::new(tracer))
.with(MetricsLayer::new(meter_provider.clone()))
.with(otel_layer);
// Configure formatting layer
let enable_color = std::io::stdout().is_terminal();
let fmt_layer = tracing_subscriber::fmt::layer()
.with_ansi(enable_color)
.with_thread_names(true)
.with_file(true)
.with_line_number(true);
// Creating a formatting layer with explicit type to avoid type mismatches
let fmt_layer = fmt_layer.with_filter(
EnvFilter::new(logger_level).add_directive(
format!("opentelemetry={}", if config.endpoint.is_empty() { logger_level } else { "off" })
.parse()
.unwrap(),
),
);
registry.with(ErrorLayer::default()).with(fmt_layer).init();
if !config.endpoint.is_empty() {
info!(
"OpenTelemetry telemetry initialized with OTLP endpoint: {}, logger_level: {}",
config.endpoint, logger_level
);
}
OtelGuard {
tracer_provider,
meter_provider,
logger_provider,
}
}
/// Switch log level
fn switch_level(logger_level: &str) -> tracing_subscriber::filter::LevelFilter {
match logger_level {
"error" => tracing_subscriber::filter::LevelFilter::ERROR,
"warn" => tracing_subscriber::filter::LevelFilter::WARN,
"info" => tracing_subscriber::filter::LevelFilter::INFO,
"debug" => tracing_subscriber::filter::LevelFilter::DEBUG,
"trace" => tracing_subscriber::filter::LevelFilter::TRACE,
_ => tracing_subscriber::filter::LevelFilter::OFF,
}
}
crates/obs/src/utils.rs
+42
View File
@@ -0,0 +1,42 @@
use local_ip_address::{local_ip, local_ipv6};
use std::net::{IpAddr, Ipv4Addr};
/// Get the IP address of the machine
///
/// Priority is given to trying to get the IPv4 address, and if it fails, try to get the IPv6 address.
/// If both fail to retrieve, None is returned.
///
/// # Returns
///
/// * `Some(IpAddr)` - Native IP address (IPv4 or IPv6)
/// * `None` - Unable to obtain any native IP address
pub fn get_local_ip() -> Option<IpAddr> {
local_ip().ok().or_else(|| local_ipv6().ok())
}
/// Get the IP address of the machine as a string
///
/// If the IP address cannot be obtained, returns "127.0.0.1" as the default value.
///
/// # Returns
///
/// * `String` - Native IP address (IPv4 or IPv6) as a string, or the default value
pub fn get_local_ip_with_default() -> String {
get_local_ip()
.unwrap_or_else(|| IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1))) // Provide a safe default value
.to_string()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_get_local_ip() {
match get_local_ip() {
Some(ip) => println!("the ip address of this machine:{}", ip),
None => println!("Unable to obtain the IP address of the machine"),
}
assert!(get_local_ip().is_some());
}
}
crates/obs/src/worker.rs
+13
View File
@@ -0,0 +1,13 @@
use crate::{sink::Sink, UnifiedLogEntry};
use std::sync::Arc;
use tokio::sync::mpsc::Receiver;
/// Start the log processing worker thread
pub async fn start_worker(receiver: Receiver<UnifiedLogEntry>, sinks: Vec<Arc<dyn Sink>>) {
let mut receiver = receiver;
while let Some(entry) = receiver.recv().await {
for sink in &sinks {
sink.write(&entry).await;
}
}
}
crypto/Cargo.toml
+1 -1
View File
@@ -14,7 +14,7 @@ aes-gcm = { version = "0.10.3", features = ["std"], optional = true }
argon2 = { version = "0.5.3", features = ["std"], optional = true }
cfg-if = "1.0.0"
chacha20poly1305 = { version = "0.10.1", optional = true }
jsonwebtoken = "9.3.0"
jsonwebtoken = { workspace = true }
pbkdf2 = { version = "0.12.2", optional = true }
rand = { workspace = true, optional = true }
sha2 = { version = "0.10.8", optional = true }
data/README.md
+1
View File
@@ -0,0 +1 @@
## Observability Docker Compose
deploy/README.md
+35
View File
@@ -0,0 +1,35 @@
# RustFS Deploy
This directory contains the deployment scripts and configurations for the project.
The deployment process is divided into two main parts: the RustFS binary and the RustFS console. The RustFS binary is
responsible for the core functionality of the system, while the RustFS console provides a web-based interface for
managing and monitoring the system.
# Directory Structure
```text
|--data // data directory
| |--vol1 // volume 1 not created
| |--vol2 // volume 2 not created
| |--vol3 // volume 3 not created
| |--vol4 // volume 4 not created
| |--README.md // data directory readme
|--logs // log directory
| |--rustfs.log // RustFS log
| |--README.md // logs directory readme
|--build
| |--rustfs.run.md // deployment script for RustFS
| |--rustfs.run-zh.md // deployment script for RustFS in Chinese
| |--rustfs.service // systemd service file
| |--rustfs-zh.service.md // systemd service file in Chinese
|--certs
| |--README.md // certs readme
| |--rustfs_tls_cert.pem // API cert.pem
| |--rustfs_tls_key.pem // API key.pem
| |--rustfs_console_tls_cert.pem // console cert.pem
| |--rustfs_console_tls_key.pem // console key.pem
|--config
| |--obs.example.yaml // example config
| |--rustfs.env // env config
| |--rustfs-zh.env // env config in Chinese
```
deploy/build/rustfs-zh.service
+102
View File
@@ -0,0 +1,102 @@
[Unit]
Description=RustFS Object Storage Server
# 定义服务的描述，说明这是一个 RustFS 对象存储服务器，显示在 systemctl status 中。
Documentation=https://rustfs.com/docs/
# 提供服务的官方文档链接，方便管理员查阅，占位符需替换为实际 URL。
After=network-online.target
# 指定服务在 network-online.target（网络就绪）之后启动，确保网络可用。
Wants=network-online.target
# 表示服务希望依赖 network-online.target，但不是强依赖，即使网络未就绪也尝试启动。
# If you're using a database, you'll need to add the corresponding dependencies
# 如果服务依赖数据库，可以添加数据库相关的依赖项（当前为注释，未启用）。
# After=postgresql.service
# 示例：若依赖 PostgreSQL，则在 PostgreSQL 服务后启动（当前未启用）。
# Requires=postgresql.service
# 示例：若强制依赖 PostgreSQL，则要求其启动成功（当前未启用）。
[Service]
Type=notify
# 服务类型为 notify，表示服务通过 sd_notify 通知 systemd 其状态（如就绪）。
NotifyAccess=main
# 指定只有主进程可以发送通知给 systemd，避免子进程干扰。
User=rustfs
# 以 rustfs 用户身份运行服务，需预先创建此用户，提升安全性。
Group=rustfs
# 以 rustfs 组身份运行服务，与 User 配合使用。
# working directory
WorkingDirectory=/opt/rustfs
# 设置服务的工作目录为 /opt/rustfs，影响相对路径的解析。
# 定义环境变量配置，用于传递给服务程序。
Environment=RUSTFS_ACCESS_KEY=rustfsadmin
# 设置访问密钥为 rustfsadmin，用于 RustFS 的认证。
Environment=RUSTFS_SECRET_KEY=rustfsadmin
# 设置秘密密钥为 rustfsadmin，与访问密钥配套使用。
ExecStart=/usr/local/bin/rustfs \
--address 0.0.0.0:9000 \
--volumes /data/rustfs/vol1,/data/rustfs/vol2 \
--obs-config /etc/rustfs/obs.yaml \
--console-enable \
--console-address 0.0.0.0:9002
# 定义启动命令，运行 /usr/local/bin/rustfs，带参数：
# --address 0.0.0.0:9000：服务监听所有接口的 9000 端口。
# --volumes：指定存储卷路径为 /data/rustfs/vol1 和 /data/rustfs/vol2。
# --obs-config：指定配置文件路径为 /etc/rustfs/obs.yaml。
# --console-enable：启用控制台功能。
# --console-address 0.0.0.0:9002：控制台监听所有接口的 9002 端口。
# 定义环境变量配置，用于传递给服务程序，推荐使用且简洁
# rustfs 示例文件 详见: `../config/rustfs-zh.env`
EnvironmentFile=-/etc/default/rustfs
ExecStart=/usr/local/bin/rustfs $RUSTFS_VOLUMES $RUSTFS_OPTS
# resource constraints
LimitNOFILE=1048576
# 设置文件描述符上限为 1048576，支持高并发连接。
LimitNPROC=32768
# 设置进程数上限为 32768，限制子进程数量。
TasksMax=infinity
# 允许服务创建无限数量的线程（谨慎使用，可能耗尽资源）。
# restart the policy
Restart=always
# 服务异常退出时总是重启，提高可用性。
RestartSec=10s
# 重启前等待 10 秒，避免频繁重启导致资源浪费。
# graceful exit configuration
TimeoutStartSec=30s
# 启动超时时间为 30 秒，若超时则认为启动失败。
TimeoutStopSec=30s
# 停止超时时间为 30 秒，若超时则强制停止。
# security settings
NoNewPrivileges=true
# 禁止服务提升权限，增强安全性。
ProtectSystem=full
# 保护系统目录（如 /usr、/boot、/etc）为只读，防止服务修改。
ProtectHome=true
# 保护用户主目录（如 /home、/root），禁止服务访问。
PrivateTmp=true
# 为服务提供私有 /tmp 目录，隔离临时文件。
PrivateDevices=true
# 禁止服务访问硬件设备（如 /dev），提升安全性。
ProtectClock=true
# 保护系统时钟，禁止服务修改时间。
ProtectKernelTunables=true
# 保护内核参数（/proc/sys），禁止服务修改。
ProtectKernelModules=true
# 禁止服务加载或卸载内核模块。
ProtectControlGroups=true
# 保护控制组（cgroups），禁止服务修改。
RestrictSUIDSGID=true
# 禁止服务使用 SUID/SGID 文件，提升安全性。
RestrictRealtime=true
# 禁止服务使用实时调度，防止资源滥用。
ReadWritePaths=/data/rustfs
# 允许服务对 /data/rustfs 目录读写，限制其他路径访问。
[Install]
WantedBy=multi-user.target
# 服务在多用户模式下自动启动，配合 systemctl enable 使用。
deploy/build/rustfs.run-zh.md
+90
View File
@@ -0,0 +1,90 @@
# RustFS 服务安装配置教程
## 1. 准备工作
### 1.1 创建系统用户
```bash
# 创建 rustfs 系统用户和用户组，禁止登录shell
sudo useradd -r -s /sbin/nologin rustfs
```
### 1.2 创建必要目录
```bash
# 创建程序目录
sudo mkdir -p /opt/rustfs
# 创建数据目录
sudo mkdir -p /data/rustfs/{vol1,vol2}
# 创建配置目录
sudo mkdir -p /etc/rustfs
# 设置目录权限
sudo chown -R rustfs:rustfs /opt/rustfs /data/rustfs
sudo chmod 755 /opt/rustfs /data/rustfs
```
## 2. 安装 RustFS
```bash
# 复制 RustFS 二进制文件
sudo cp rustfs /usr/local/bin/
sudo chmod +x /usr/local/bin/rustfs
# 复制配置文件
sudo cp obs.yaml /etc/rustfs/
sudo chown -R rustfs:rustfs /etc/rustfs
```
## 3. 配置 Systemd 服务
```bash
# 复制服务单元文件
sudo cp rustfs.service /etc/systemd/system/
# 重新加载 systemd 配置
sudo systemctl daemon-reload
```
## 4. 服务管理
### 4.1 启动服务
```bash
sudo systemctl start rustfs
```
### 4.2 查看服务状态
```bash
sudo systemctl status rustfs
```
### 4.3 启用开机自启
```bash
sudo systemctl enable rustfs
```
### 4.4 查看服务日志
```bash
# 查看实时日志
sudo journalctl -u rustfs -f
# 查看今天的日志
sudo journalctl -u rustfs --since today
```
## 5. 验证安装
```bash
# 检查服务端口
ss -tunlp | grep 9000
ss -tunlp | grep 9002
# 测试服务可用性
curl -I http://localhost:9000
```
deploy/build/rustfs.run.md
+90
View File
@@ -0,0 +1,90 @@
# RustFS Service Installation Guide
## 1. Prerequisites
### 1.1 Create System User
```bash
# Create rustfs system user and group without login shell
sudo useradd -r -s /sbin/nologin rustfs
```
### 1.2 Create Required Directories
```bash
# Create program directory
sudo mkdir -p /opt/rustfs
# Create data directories
sudo mkdir -p /data/rustfs/{vol1,vol2}
# Create configuration directory
sudo mkdir -p /etc/rustfs
# Set directory permissions
sudo chown -R rustfs:rustfs /opt/rustfs /data/rustfs
sudo chmod 755 /opt/rustfs /data/rustfs
```
## 2. Install RustFS
```bash
# Copy RustFS binary
sudo cp rustfs /usr/local/bin/
sudo chmod +x /usr/local/bin/rustfs
# Copy configuration file
sudo cp obs.yaml /etc/rustfs/
sudo chown -R rustfs:rustfs /etc/rustfs
```
## 3. Configure Systemd Service
```bash
# Copy service unit file
sudo cp rustfs.service /etc/systemd/system/
# Reload systemd configuration
sudo systemctl daemon-reload
```
## 4. Service Management
### 4.1 Start Service
```bash
sudo systemctl start rustfs
```
### 4.2 Check Service Status
```bash
sudo systemctl status rustfs
```
### 4.3 Enable Auto-start
```bash
sudo systemctl enable rustfs
```
### 4.4 View Service Logs
```bash
# View real-time logs
sudo journalctl -u rustfs -f
# View today's logs
sudo journalctl -u rustfs --since today
```
## 5. Verify Installation
```bash
# Check service ports
ss -tunlp | grep 9000
ss -tunlp | grep 9002
# Test service availability
curl -I http://localhost:9000
```
deploy/build/rustfs.service
+62
View File
@@ -0,0 +1,62 @@
[Unit]
Description=RustFS Object Storage Server
Documentation=https://rustfs.com/docs/
After=network-online.target
Wants=network-online.target
# If you're using a database, you'll need to add the corresponding dependencies
# After=postgresql.service
# Requires=postgresql.service
[Service]
Type=notify
NotifyAccess=main
User=rustfs
Group=rustfs
# working directory
WorkingDirectory=/opt/rustfs
# environment variable configuration and main program (Option 1: Directly specify arguments)
Environment=RUSTFS_ACCESS_KEY=rustfsadmin
Environment=RUSTFS_SECRET_KEY=rustfsadmin
ExecStart=/usr/local/bin/rustfs \
--address 0.0.0.0:9000 \
--volumes /data/rustfs/vol1,/data/rustfs/vol2 \
--obs-config /etc/rustfs/obs.yaml \
--console-enable \
--console-address 0.0.0.0:9002
# environment variable configuration (Option 2: Use environment variables)
# rustfs example file see: `../config/rustfs.env`
EnvironmentFile=-/etc/default/rustfs
ExecStart=/usr/local/bin/rustfs $RUSTFS_VOLUMES $RUSTFS_OPTS
# resource constraints
LimitNOFILE=1048576
LimitNPROC=32768
TasksMax=infinity
# restart the policy
Restart=always
RestartSec=10s
# graceful exit configuration
TimeoutStartSec=30s
TimeoutStopSec=30s
# security settings
NoNewPrivileges=true
ProtectSystem=full
ProtectHome=true
PrivateTmp=true
PrivateDevices=true
ProtectClock=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictSUIDSGID=true
RestrictRealtime=true
ReadWritePaths=/data/rustfs
[Install]
WantedBy=multi-user.target
deploy/certs/README.md
+38
View File
@@ -0,0 +1,38 @@
## Certs
### Generate a self-signed certificate
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes
```
### Generate a self-signed certificate with a specific subject
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes \
-subj "/C=US/ST=California/L=San Francisco/O=My Company/CN=mydomain.com"
```
### Generate a self-signed certificate with a specific subject and SAN
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes \
-subj "/C=US/ST=California/L=San Francisco/O=My Company/CN=mydomain.com" \
-addext "subjectAltName=DNS:mydomain.com,DNS:www.mydomain.com"
```
### Generate a self-signed certificate with a specific subject and SAN (multiple SANs)
```bash
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes \
-subj "/C=US/ST=California/L=San Francisco/O=My Company/CN=mydomain.com" \
-addext "subjectAltName=DNS:mydomain.com,DNS:www.mydomain.com,DNS:api.mydomain.com"
```
### TLS File
```text
rustfs_public.crt #api cert.pem
rustfs_private.key #api key.pem
```
deploy/config/.example.env
+27
View File
@@ -0,0 +1,27 @@
OBSERVABILITY__ENDPOINT=http://localhost:4317
OBSERVABILITY__USE_STDOUT=true
OBSERVABILITY__SAMPLE_RATIO=2.0
OBSERVABILITY__METER_INTERVAL=30
OBSERVABILITY__SERVICE_NAME=rustfs
OBSERVABILITY__SERVICE_VERSION=0.1.0
OBSERVABILITY__ENVIRONMENT=develop
OBSERVABILITY__LOGGER_LEVEL=debug
SINKS__KAFKA__ENABLED=false
SINKS__KAFKA__BOOTSTRAP_SERVERS=localhost:9092
SINKS__KAFKA__TOPIC=logs
SINKS__KAFKA__BATCH_SIZE=100
SINKS__KAFKA__BATCH_TIMEOUT_MS=1000
SINKS__WEBHOOK__ENABLED=false
SINKS__WEBHOOK__ENDPOINT=http://localhost:8080/webhook
SINKS__WEBHOOK__AUTH_TOKEN=
SINKS__WEBHOOK__BATCH_SIZE=100
SINKS__WEBHOOK__BATCH_TIMEOUT_MS=1000
SINKS__FILE__ENABLED=true
SINKS__FILE__PATH=./deploy/logs/rustfs.log
SINKS__FILE__BATCH_SIZE=10
SINKS__FILE__BATCH_TIMEOUT_MS=1000
LOGGER__QUEUE_CAPACITY=10
deploy/config/obs-zh.example.toml
+33
View File
@@ -0,0 +1,33 @@
[observability]
endpoint = "http://localhost:4317" # 可观测性数据上报的终端地址，默认为"http://localhost:4317"
use_stdout = true # 是否将日志输出到标准输出
sample_ratio = 2.0 # 采样率，表示每 2 条数据采样 1 条
meter_interval = 30 # 指标收集间隔，单位为秒
service_name = "rustfs" # 服务名称，用于标识当前服务
service_version = "0.1.0" # 服务版本号
environments = "develop" # 运行环境，如开发环境 (develop)
logger_level = "debug" # 日志级别，可选 debug/info/warn/error 等
[sinks]
[sinks.kafka] # Kafka 接收器配置
enabled = false # 是否启用 Kafka 接收器，默认禁用
bootstrap_servers = "localhost:9092" # Kafka 服务器地址
topic = "logs" # Kafka 主题名称
batch_size = 100 # 批处理大小，每次发送的消息数量
batch_timeout_ms = 1000 # 批处理超时时间，单位为毫秒
[sinks.webhook] # Webhook 接收器配置
enabled = false # 是否启用 Webhook 接收器
endpoint = "http://localhost:8080/webhook" # Webhook 接收地址
auth_token = "" # 认证令牌
batch_size = 100 # 批处理大小
batch_timeout_ms = 1000 # 批处理超时时间，单位为毫秒
[sinks.file] # 文件接收器配置
enabled = true # 是否启用文件接收器
path = "./deploy/logs/rustfs.log" # 日志文件路径
batch_size = 10 # 批处理大小
batch_timeout_ms = 1000 # 批处理超时时间，单位为毫秒
[logger] # 日志器配置
queue_capacity = 10 # 日志队列容量，表示可以缓存的日志条数
deploy/config/obs.example.toml
+33
View File
@@ -0,0 +1,33 @@
[observability]
endpoint = "http://localhost:4317" # Default is "http://localhost:4317" if not specified
use_stdout = false # Output with stdout, true output, false no output
sample_ratio = 2.0
meter_interval = 30
service_name = "rustfs"
service_version = "0.1.0"
environment = "develop"
logger_level = "info"
[sinks]
[sinks.kafka] # Kafka sink is disabled by default
enabled = false
bootstrap_servers = "localhost:9092"
topic = "logs"
batch_size = 100 # Default is 100 if not specified
batch_timeout_ms = 1000 # Default is 1000ms if not specified
[sinks.webhook]
enabled = false
endpoint = "http://localhost:8080/webhook"
auth_token = ""
batch_size = 100 # Default is 3 if not specified
batch_timeout_ms = 1000 # Default is 100ms if not specified
[sinks.file]
enabled = true
path = "./deploy/logs/rustfs.log"
batch_size = 100
batch_timeout_ms = 1000 # Default is 8192 bytes if not specified
[logger]
queue_capacity = 10000
deploy/config/rustfs-zh.env
+26
View File
@@ -0,0 +1,26 @@
# RustFS 管理员用户名
RUSTFS_ROOT_USER=rustfsadmin
# RustFS 管理员密码
RUSTFS_ROOT_PASSWORD=rustfsadmin
# 数据卷配置示例路径：deploy/data/rustfs.env
# RustFS 数据卷存储路径，支持多卷配置，vol1 到 vol4
RUSTFS_VOLUMES="./deploy/deploy/vol{1...4}"
# RustFS 服务启动参数，指定监听地址和端口
RUSTFS_OPTS="--address 0.0.0.0:9000"
# RustFS 服务监听地址和端口
RUSTFS_ADDRESS="0.0.0.0:9000"
# 是否启用 RustFS 控制台功能
RUSTFS_CONSOLE_ENABLE=true
# RustFS 控制台监听地址和端口
RUSTFS_CONSOLE_ADDRESS="0.0.0.0:9002"
# RustFS 服务端点地址，用于客户端访问
RUSTFS_SERVER_ENDPOINT="http://127.0.0.1:9000"
# RustFS 服务域名配置
RUSTFS_SERVER_DOMAINS=127.0.0.1:9002
# RustFS 许可证内容
RUSTFS_LICENSE="license content"
# 可观测性配置文件路径：deploy/config/obs.example.toml
RUSTFS_OBS_CONFIG=/etc/default/obs.toml
# TLS 证书目录路径：deploy/certs
RUSTFS_TLS_PATH=/etc/default/tls
deploy/config/rustfs.env
+26
View File
@@ -0,0 +1,26 @@
# RustFS administrator username
RUSTFS_ROOT_USER=rustfsadmin
# RustFS administrator password
RUSTFS_ROOT_PASSWORD=rustfsadmin
# Data volume configuration example path: deploy/data/rustfs.env
# RustFS data volume storage paths, supports multiple volumes from vol1 to vol4
RUSTFS_VOLUMES="./deploy/deploy/vol{1...4}"
# RustFS service startup parameters, specifying listen address and port
RUSTFS_OPTS="--address 0.0.0.0:9000"
# RustFS service listen address and port
RUSTFS_ADDRESS="0.0.0.0:9000"
# Enable RustFS console functionality
RUSTFS_CONSOLE_ENABLE=true
# RustFS console listen address and port
RUSTFS_CONSOLE_ADDRESS="0.0.0.0:9002"
# RustFS service endpoint for client access
RUSTFS_SERVER_ENDPOINT="http://127.0.0.1:9000"
# RustFS service domain configuration
RUSTFS_SERVER_DOMAINS=127.0.0.1:9002
# RustFS license content
RUSTFS_LICENSE="license content"
# Observability configuration file path: deploy/config/obs.example.toml
RUSTFS_OBS_CONFIG=/etc/default/obs.toml
# TLS certificates directory path: deploy/certs
RUSTFS_TLS_PATH=/etc/default/tls
deploy/data/README.md
+1
View File
@@ -0,0 +1 @@
# Data Volumes
deploy/logs/README.md
+1
View File
@@ -0,0 +1 @@
# RustFS Logs
docker-compose-obs.yaml
+148
View File
@@ -0,0 +1,148 @@
services:
otel-collector:
image: otel/opentelemetry-collector-contrib:0.120.0
environment:
- TZ=Asia/Shanghai
volumes:
- ./.docker/observability/otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
ports:
- 1888:1888
- 8888:8888
- 8889:8889
- 13133:13133
- 4317:4317
- 4318:4318
- 55679:55679
networks:
- rustfs-network
jaeger:
image: jaegertracing/jaeger:2.4.0
environment:
- TZ=Asia/Shanghai
ports:
- "16686:16686"
- "14317:4317"
- "14318:4318"
networks:
- rustfs-network
prometheus:
image: prom/prometheus:v3.2.1
environment:
- TZ=Asia/Shanghai
volumes:
- ./.docker/observability/prometheus.yml:/etc/prometheus/prometheus.yml
ports:
- "9090:9090"
networks:
- rustfs-network
loki:
image: grafana/loki:3.4.2
environment:
- TZ=Asia/Shanghai
volumes:
- ./.docker/observability/loki-config.yaml:/etc/loki/local-config.yaml
ports:
- "3100:3100"
command: -config.file=/etc/loki/local-config.yaml
networks:
- rustfs-network
grafana:
image: grafana/grafana:11.6.0
ports:
- "3000:3000" # Web UI
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
- TZ=Asia/Shanghai
networks:
- rustfs-network
node1:
build:
context: .
dockerfile: Dockerfile.obs
container_name: node1
environment:
- RUSTFS_VOLUMES=/root/data/target/volume/test{1...4}
- RUSTFS_ADDRESS=0.0.0.0:9000
- RUSTFS_CONSOLE_ENABLE=true
- RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9002
- RUSTFS_OBS_CONFIG=/etc/observability/config.obs.toml
platform: linux/amd64
ports:
- "9001:9000" # 映射宿主机的 9001 端口到容器的 9000 端口
- "9101:9002"
volumes:
- ./data:/root/data # 将当前路径挂载到容器内的 /root/data
- ./.docker/observability/config/obs.toml:/etc/observability/config.obs.toml
networks:
- rustfs-network
node2:
build:
context: .
dockerfile: Dockerfile.obs
container_name: node2
environment:
- RUSTFS_VOLUMES=/root/data/target/volume/test{1...4}
- RUSTFS_ADDRESS=0.0.0.0:9000
- RUSTFS_CONSOLE_ENABLE=true
- RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9002
- RUSTFS_OBS_CONFIG=/etc/observability/config.obs.toml
platform: linux/amd64
ports:
- "9002:9000" # 映射宿主机的 9002 端口到容器的 9000 端口
- "9102:9002"
volumes:
- ./data:/root/data
- ./.docker/observability/config/obs.toml:/etc/observability/config.obs.toml
networks:
- rustfs-network
node3:
build:
context: .
dockerfile: Dockerfile.obs
container_name: node3
environment:
- RUSTFS_VOLUMES=/root/data/target/volume/test{1...4}
- RUSTFS_ADDRESS=0.0.0.0:9000
- RUSTFS_CONSOLE_ENABLE=true
- RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9002
- RUSTFS_OBS_CONFIG=/etc/observability/config.obs.toml
platform: linux/amd64
ports:
- "9003:9000" # 映射宿主机的 9003 端口到容器的 9000 端口
- "9103:9002"
volumes:
- ./data:/root/data
- ./.docker/observability/config/obs.toml:/etc/observability/config.obs.toml
networks:
- rustfs-network
node4:
build:
context: .
dockerfile: Dockerfile.obs
container_name: node4
environment:
- RUSTFS_VOLUMES=/root/data/target/volume/test{1...4}
- RUSTFS_ADDRESS=0.0.0.0:9000
- RUSTFS_CONSOLE_ENABLE=true
- RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9002
- RUSTFS_OBS_CONFIG=/etc/observability/config.obs.toml
platform: linux/amd64
ports:
- "9004:9000" # 映射宿主机的 9004 端口到容器的 9000 端口
- "9104:9002"
volumes:
- ./data:/root/data
- ./.docker/observability/config/obs.toml:/etc/observability/config.obs.toml
networks:
- rustfs-network
networks:
rustfs-network:
driver: bridge
name: "network_rustfs_config"
driver_opts:
com.docker.network.enable_ipv6: "true"
e2e_test/Cargo.toml
+1 -1
View File
@@ -22,7 +22,7 @@ protos.workspace = true
rmp-serde.workspace = true
serde.workspace = true
serde_json.workspace = true
tonic = { version = "0.12.3", features = ["gzip"] }
tonic = { workspace = true }
tokio = { workspace = true }
tower.workspace = true
url.workspace = true
ecstore/Cargo.toml
+2 -2
View File
@@ -32,7 +32,7 @@ s3s.workspace = true
http.workspace = true
highway = "1.3.0"
url.workspace = true
uuid = { version = "1.15.1", features = ["v4", "fast-rng", "serde"] }
uuid = { workspace = true, features = ["v4", "fast-rng", "serde"] }
reed-solomon-erasure = { version = "6.0.0", features = ["simd-accel"] }
transform-stream = "0.3.1"
lazy_static.workspace = true
@@ -44,7 +44,7 @@ path-absolutize = "3.1.1"
protos.workspace = true
rmp.workspace = true
rmp-serde.workspace = true
tokio-util = { version = "0.7.13", features = ["io", "compat"] }
tokio-util = { workspace = true, features = ["io", "compat"] }
crc32fast = "1.4.2"
siphasher = "1.0.1"
base64-simd = "0.8.0"
ecstore/src/config/storageclass.rs
+1 -7
View File
@@ -189,13 +189,7 @@ pub fn lookup_config(kvs: &KVS, set_drive_count: usize) -> Result<Config> {
validate_parity_inner(standard.parity, rrs.parity, set_drive_count)?;
let optimize = {
if let Ok(ev) = env::var(OPTIMIZE_ENV) {
Some(ev)
} else {
None
}
};
let optimize = { env::var(OPTIMIZE_ENV).ok() };
let inline_block = {
if let Ok(ev) = env::var(INLINE_BLOCK_ENV) {
ecstore/src/disk/error.rs
+14
View File
@@ -1,4 +1,5 @@
use std::io::{self, ErrorKind};
use std::path::PathBuf;
use tracing::error;
@@ -345,6 +346,19 @@ pub fn os_err_to_file_err(e: io::Error) -> Error {
}
}
#[derive(Debug, thiserror::Error)]
pub struct FileAccessDeniedWithContext {
pub path: PathBuf,
#[source]
pub source: std::io::Error,
}
impl std::fmt::Display for FileAccessDeniedWithContext {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "访问文件 '{}' 被拒绝: {}", self.path.display(), self.source)
}
}
pub fn is_unformatted_disk(err: &Error) -> bool {
matches!(err.downcast_ref::<DiskError>(), Some(DiskError::UnformattedDisk))
}
ecstore/src/disk/local.rs
+61 -26
View File
@@ -11,16 +11,20 @@ use super::{
};
use crate::bitrot::bitrot_verify;
use crate::bucket::metadata_sys::{self};
use crate::bucket::versioning::VersioningApi;
use crate::bucket::versioning_sys::BucketVersioningSys;
use crate::cache_value::cache::{Cache, Opts, UpdateFn};
use crate::disk::error::{
convert_access_error, is_err_os_not_exist, is_sys_err_handle_invalid, is_sys_err_invalid_arg, is_sys_err_is_dir,
is_sys_err_not_dir, map_err_not_exists, os_err_to_file_err,
is_sys_err_not_dir, map_err_not_exists, os_err_to_file_err, FileAccessDeniedWithContext,
};
use crate::disk::os::{check_path_length, is_empty_dir};
use crate::disk::STORAGE_FORMAT_FILE;
use crate::file_meta::{get_file_info, read_xl_meta_no_data, FileInfoOpts};
use crate::global::{GLOBAL_IsErasureSD, GLOBAL_RootDiskThreshold};
use crate::heal::data_scanner::{has_active_rules, scan_data_folder, ScannerItem, ShouldSleepFn, SizeSummary};
use crate::heal::data_scanner::{
lc_has_active_rules, rep_has_active_rules, scan_data_folder, ScannerItem, ShouldSleepFn, SizeSummary,
};
use crate::heal::data_scanner_metric::{ScannerMetric, ScannerMetrics};
use crate::heal::data_usage_cache::{DataUsageCache, DataUsageEntry};
use crate::heal::error::{ERR_IGNORE_FILE_CONTRIB, ERR_SKIP_FILE};
@@ -51,7 +55,6 @@ use path_absolutize::Absolutize;
use std::collections::{HashMap, HashSet};
use std::fmt::Debug;
use std::io::SeekFrom;
use std::os::unix::fs::MetadataExt;
use std::sync::atomic::{AtomicU32, Ordering};
use std::sync::Arc;
use std::time::{Duration, SystemTime};
@@ -308,7 +311,19 @@ impl LocalDisk {
// })
// }
#[allow(unreachable_code)]
#[allow(unused_variables)]
pub async fn move_to_trash(&self, delete_path: &PathBuf, recursive: bool, immediate_purge: bool) -> Result<()> {
if recursive {
remove_all(delete_path).await?;
} else {
remove(delete_path).await?;
}
return Ok(());
// TODO: 异步通知 检测硬盘空间 清空回收站
let trash_path = self.get_object_path(super::RUSTFS_META_TMP_DELETED_BUCKET, Uuid::new_v4().to_string().as_str())?;
if let Some(parent) = trash_path.parent() {
if !parent.exists() {
@@ -347,7 +362,6 @@ impl LocalDisk {
return Ok(());
}
// TODO: 异步通知 检测硬盘空间 清空回收站
Ok(())
}
@@ -383,7 +397,10 @@ impl LocalDisk {
kind => {
if kind.to_string() != "directory not empty" {
warn!("delete_file remove_dir {:?} err {}", &delete_path, kind.to_string());
return Err(Error::from(err));
return Err(Error::new(FileAccessDeniedWithContext {
path: delete_path.clone(),
source: err,
}));
}
}
}
@@ -395,7 +412,10 @@ impl LocalDisk {
ErrorKind::NotFound => (),
_ => {
warn!("delete_file remove_file {:?} err {:?}", &delete_path, &err);
return Err(Error::from(err));
return Err(Error::new(FileAccessDeniedWithContext {
path: delete_path.clone(),
source: err,
}));
}
}
}
@@ -472,13 +492,14 @@ impl LocalDisk {
let meta = f.metadata().await?;
if meta.is_dir() {
return Err(Error::new(DiskError::FileNotFound));
// fix use io::Error
return Err(std::io::Error::new(ErrorKind::NotFound, "is dir").into());
}
let meta = f.metadata().await.map_err(os_err_to_file_err)?;
if meta.is_dir() {
return Err(Error::new(DiskError::FileNotFound));
return Err(std::io::Error::new(ErrorKind::NotFound, "is dir").into());
}
let size = meta.len() as usize;
@@ -743,12 +764,10 @@ impl LocalDisk {
.await
.map_err(os_err_to_file_err)?;
// let mut data = Vec::new();
// let n = file.read_to_end(&mut data).await?;
let meta = file.metadata().await?;
let file_size = meta.len() as usize;
bitrot_verify(Box::new(file), meta.size() as usize, part_size, algo, sum.to_vec(), shard_size).await
bitrot_verify(Box::new(file), file_size, part_size, algo, sum.to_vec(), shard_size).await
}
async fn scan_dir<W: AsyncWrite + Unpin>(
@@ -816,8 +835,6 @@ impl LocalDisk {
// 第一层过滤
for item in entries.iter_mut() {
// warn!("walk_dir get entry {:?}", &entry);
let entry = item.clone();
// check limit
if opts.limit > 0 && *objs_returned >= opts.limit {
@@ -857,7 +874,10 @@ impl LocalDisk {
let metadata = self
.read_metadata(self.get_object_path(bucket, format!("{}/{}", &current, &entry).as_str())?)
.await?;
let name = entry.trim_end_matches(STORAGE_FORMAT_FILE).trim_end_matches(SLASH_SEPARATOR);
// 用strip_suffix只删除一次
let entry = entry.strip_suffix(STORAGE_FORMAT_FILE).unwrap_or_default().to_owned();
let name = entry.trim_end_matches(SLASH_SEPARATOR);
let name = decode_dir_object(format!("{}/{}", &current, &name).as_str());
out.write_obj(&MetaCacheEntry {
@@ -887,7 +907,6 @@ impl LocalDisk {
let mut dir_stack: Vec<String> = Vec::with_capacity(5);
for entry in entries.iter() {
//
if opts.limit > 0 && *objs_returned >= opts.limit {
return Ok(());
}
@@ -1217,7 +1236,7 @@ impl DiskAPI for LocalDisk {
.join(path)
.join(fi.data_dir.map_or("".to_string(), |dir| dir.to_string()))
.join(format!("part.{}", part.number));
let err = match self
let err = (self
.bitrot_verify(
&part_path,
erasure.shard_file_size(part.size),
@@ -1225,11 +1244,8 @@ impl DiskAPI for LocalDisk {
&checksum_info.hash,
erasure.shard_size(erasure.block_size),
)
.await
{
Ok(_) => None,
Err(err) => Some(err),
};
.await)
.err();
resp.results[i] = conv_part_err_to_int(&err);
if resp.results[i] == CHECK_PART_UNKNOWN {
if let Some(err) = err {
@@ -2283,6 +2299,7 @@ impl DiskAPI for LocalDisk {
Ok(info)
}
#[tracing::instrument(level = "info", skip_all)]
async fn ns_scanner(
&self,
cache: &DataUsageCache,
@@ -2296,18 +2313,30 @@ impl DiskAPI for LocalDisk {
// must befor metadata_sys
let Some(store) = new_object_layer_fn() else { return Err(Error::msg("errServerNotInitialized")) };
let mut cache = cache.clone();
// Check if the current bucket has a configured lifecycle policy
if let Ok((lc, _)) = metadata_sys::get_lifecycle_config(&cache.info.name).await {
if lc_has_active_rules(&lc, "") {
cache.info.life_cycle = Some(lc);
}
}
// Check if the current bucket has replication configuration
if let Ok((rcfg, _)) = metadata_sys::get_replication_config(&cache.info.name).await {
if has_active_rules(&rcfg, "", true) {
if rep_has_active_rules(&rcfg, "", true) {
// TODO: globalBucketTargetSys
}
}
let vcfg = match BucketVersioningSys::get(&cache.info.name).await {
Ok(vcfg) => Some(vcfg),
Err(_) => None,
};
let loc = self.get_disk_location();
let disks = store.get_disks(loc.pool_idx.unwrap(), loc.disk_idx.unwrap()).await?;
let disk = Arc::new(LocalDisk::new(&self.endpoint(), false).await?);
let disk_clone = disk.clone();
let mut cache = cache.clone();
cache.info.updates = Some(updates.clone());
let mut data_usage_info = scan_data_folder(
&disks,
@@ -2316,8 +2345,9 @@ impl DiskAPI for LocalDisk {
Box::new(move |item: &ScannerItem| {
let mut item = item.clone();
let disk = disk_clone.clone();
let vcfg = vcfg.clone();
Box::pin(async move {
if item.path.ends_with(&format!("{}{}", SLASH_SEPARATOR, STORAGE_FORMAT_FILE)) {
if !item.path.ends_with(&format!("{}{}", SLASH_SEPARATOR, STORAGE_FORMAT_FILE)) {
return Err(Error::from_string(ERR_SKIP_FILE));
}
let stop_fn = ScannerMetrics::log(ScannerMetric::ScanObject);
@@ -2358,7 +2388,12 @@ impl DiskAPI for LocalDisk {
}
};
let versioned = false;
let versioned = if let Some(vcfg) = vcfg.as_ref() {
vcfg.versioned(item.object_path().to_str().unwrap_or_default())
} else {
false
};
let mut obj_deleted = false;
for info in obj_infos.iter() {
let done = ScannerMetrics::time(ScannerMetric::ApplyVersion);
ecstore/src/endpoints.rs
+4 -3
View File
@@ -1,4 +1,4 @@
use tracing::warn;
use tracing::{instrument, warn};
use crate::{
disk::endpoint::{Endpoint, EndpointType},
@@ -407,7 +407,7 @@ pub struct PoolEndpoints {
pub platform: String,
}
/// list of list of endpoints
/// list of endpoints
#[derive(Debug, Clone, Default)]
pub struct EndpointServerPools(pub Vec<PoolEndpoints>);
@@ -532,6 +532,8 @@ impl EndpointServerPools {
nodes
}
#[instrument]
pub fn hosts_sorted(&self) -> Vec<Option<XHost>> {
let (mut peers, local) = self.peers();
@@ -604,7 +606,6 @@ impl EndpointServerPools {
#[cfg(test)]
mod test {
use super::*;
use std::path::Path;
ecstore/src/global.rs
+1 -1
View File
@@ -61,7 +61,7 @@ pub fn get_global_deployment_id() -> Option<String> {
pub fn set_global_endpoints(eps: Vec<PoolEndpoints>) {
GLOBAL_Endpoints
.set(EndpointServerPools::from(eps))
.expect("GLOBAL_Endpoints set faild")
.expect("GLOBAL_Endpoints set failed")
}
pub fn get_global_endpoints() -> EndpointServerPools {
ecstore/src/heal/data_scanner.rs
+124 -36
View File
@@ -18,7 +18,10 @@ use super::{
data_usage_cache::{DataUsageCache, DataUsageEntry, DataUsageHash},
heal_commands::{HealScanMode, HEAL_DEEP_SCAN, HEAL_NORMAL_SCAN},
};
use crate::heal::data_usage::DATA_USAGE_ROOT;
use crate::{
bucket::{versioning::VersioningApi, versioning_sys::BucketVersioningSys},
heal::data_usage::DATA_USAGE_ROOT,
};
use crate::{
cache_value::metacache_set::{list_path_raw, ListPathRawOptions},
config::{
@@ -49,7 +52,7 @@ use common::error::{Error, Result};
use lazy_static::lazy_static;
use rand::Rng;
use rmp_serde::{Deserializer, Serializer};
use s3s::dto::{ReplicationConfiguration, ReplicationRuleStatus};
use s3s::dto::{BucketLifecycleConfiguration, ExpirationStatus, LifecycleRule, ReplicationConfiguration, ReplicationRuleStatus};
use serde::{Deserialize, Serialize};
use tokio::{
sync::{
@@ -352,7 +355,7 @@ impl CurrentScannerCycle {
let str_len = rmp::decode::read_str_len(&mut cur)?;
// ！！！ Vec::with_capacity(str_len) 失败，vec!正常
// ！！！Vec::with_capacity(str_len) 失败，vec! 正常
let mut field_buff = vec![0u8; str_len as usize];
cur.read_exact(&mut field_buff)?;
@@ -412,7 +415,7 @@ pub struct ScannerItem {
pub prefix: String,
pub object_name: String,
pub replication: Option<ReplicationConfiguration>,
// todo: lifecycle
pub lifecycle: Option<BucketLifecycleConfiguration>,
// typ: fs::Permissions,
pub heal: Heal,
pub debug: bool,
@@ -435,7 +438,7 @@ impl ScannerItem {
pub async fn apply_versions_actions(&self, fivs: &[FileInfo]) -> Result<Vec<ObjectInfo>> {
let obj_infos = self.apply_newer_noncurrent_version_limit(fivs).await?;
if obj_infos.len() >= SCANNER_EXCESS_OBJECT_VERSIONS.load(Ordering::SeqCst).try_into().unwrap() {
if obj_infos.len() >= SCANNER_EXCESS_OBJECT_VERSIONS.load(Ordering::SeqCst) as usize {
// todo
}
@@ -444,12 +447,7 @@ impl ScannerItem {
cumulative_size += obj_info.size;
}
if cumulative_size
>= SCANNER_EXCESS_OBJECT_VERSIONS_TOTAL_SIZE
.load(Ordering::SeqCst)
.try_into()
.unwrap()
{
if cumulative_size >= SCANNER_EXCESS_OBJECT_VERSIONS_TOTAL_SIZE.load(Ordering::SeqCst) as usize {
//todo
}
@@ -457,22 +455,31 @@ impl ScannerItem {
}
pub async fn apply_newer_noncurrent_version_limit(&self, fivs: &[FileInfo]) -> Result<Vec<ObjectInfo>> {
let done = ScannerMetrics::time(ScannerMetric::ApplyNonCurrent);
let mut object_infos = Vec::new();
for info in fivs.iter() {
object_infos.push(info.to_object_info(&self.bucket, &self.object_path().to_string_lossy(), false));
// let done = ScannerMetrics::time(ScannerMetric::ApplyNonCurrent);
let versioned = match BucketVersioningSys::get(&self.bucket).await {
Ok(vcfg) => vcfg.versioned(self.object_path().to_str().unwrap_or_default()),
Err(_) => false,
};
let mut object_infos = Vec::with_capacity(fivs.len());
if self.lifecycle.is_none() {
for info in fivs.iter() {
object_infos.push(info.to_object_info(&self.bucket, &self.object_path().to_string_lossy(), versioned));
}
return Ok(object_infos);
}
done().await;
// done().await;
Ok(object_infos)
}
pub async fn apply_actions(&self, _oi: &ObjectInfo, _size_s: &SizeSummary) -> (bool, usize) {
pub async fn apply_actions(&self, oi: &ObjectInfo, _size_s: &SizeSummary) -> (bool, usize) {
let done = ScannerMetrics::time(ScannerMetric::Ilm);
//todo: lifecycle
done().await;
(false, 0)
(false, oi.size)
}
}
@@ -551,6 +558,7 @@ impl FolderScanner {
true
}
#[tracing::instrument(level = "info", skip_all)]
async fn scan_folder(&mut self, folder: &CachedFolder, into: &mut DataUsageEntry) -> Result<()> {
let this_hash = hash_path(&folder.name);
let was_compacted = into.compacted;
@@ -564,8 +572,18 @@ impl FolderScanner {
let (_, prefix) = path_to_bucket_object_with_base_path(&self.root, &folder.name);
// Todo: lifeCycle
let active_life_cycle = if let Some(lc) = self.old_cache.info.life_cycle.as_ref() {
if lc_has_active_rules(lc, &prefix) {
self.old_cache.info.life_cycle.clone()
} else {
None
}
} else {
None
};
let replication_cfg = if self.old_cache.info.replication.is_some()
&& has_active_rules(self.old_cache.info.replication.as_ref().unwrap(), &prefix, true)
&& rep_has_active_rules(self.old_cache.info.replication.as_ref().unwrap(), &prefix, true)
{
self.old_cache.info.replication.clone()
} else {
@@ -596,7 +614,7 @@ impl FolderScanner {
continue;
}
if !sub_path.is_dir() {
if sub_path.is_dir() {
let h = hash_path(ent_name.to_str().unwrap());
if h == this_hash {
continue;
@@ -641,6 +659,7 @@ impl FolderScanner {
.unwrap_or_default(),
debug: self.data_usage_scanner_debug,
replication: replication_cfg.clone(),
lifecycle: active_life_cycle.clone(),
heal: Heal::default(),
};
@@ -684,16 +703,11 @@ impl FolderScanner {
}
let should_compact = self.new_cache.info.name != folder.name
&& existing_folders.len() + new_folders.len() >= DATA_SCANNER_COMPACT_AT_FOLDERS.try_into().unwrap()
|| existing_folders.len() + new_folders.len() >= DATA_SCANNER_FORCE_COMPACT_AT_FOLDERS.try_into().unwrap();
&& (existing_folders.len() + new_folders.len() >= DATA_SCANNER_COMPACT_AT_FOLDERS as usize
|| existing_folders.len() + new_folders.len() >= DATA_SCANNER_FORCE_COMPACT_AT_FOLDERS as usize);
let total_folders = existing_folders.len() + new_folders.len();
if total_folders
> SCANNER_EXCESS_FOLDERS
.load(std::sync::atomic::Ordering::SeqCst)
.try_into()
.unwrap()
{
if total_folders > SCANNER_EXCESS_FOLDERS.load(Ordering::SeqCst) as usize {
let _prefix_name = format!("{}/", folder.name.trim_end_matches('/'));
// todo: notification
}
@@ -941,7 +955,7 @@ impl FolderScanner {
let _ = list_path_raw(rx, lopts).await;
if *found_objs.read().await {
let this = CachedFolder {
let this: CachedFolder = CachedFolder {
name: k.clone(),
parent: this_hash.clone(),
object_heal_prob_div: 1,
@@ -957,7 +971,7 @@ impl FolderScanner {
if !into.compacted && self.new_cache.info.name != folder.name {
let mut flat = self.new_cache.size_recursive(&this_hash.key()).unwrap_or_default();
flat.compacted = true;
let compact = if flat.objects < DATA_SCANNER_COMPACT_LEAST_OBJECT.try_into().unwrap() {
let compact = if flat.objects < DATA_SCANNER_COMPACT_LEAST_OBJECT as usize {
true
} else {
// Compact if we only have objects as children...
@@ -1002,6 +1016,7 @@ impl FolderScanner {
Ok(())
}
#[tracing::instrument(level = "info", skip_all)]
async fn send_update(&mut self) {
if SystemTime::now().duration_since(self.last_update).unwrap() < Duration::from_secs(60) {
return;
@@ -1013,8 +1028,9 @@ impl FolderScanner {
}
}
#[tracing::instrument(level = "info", skip(into, folder_scanner))]
async fn scan(folder: &CachedFolder, into: &mut DataUsageEntry, folder_scanner: &mut FolderScanner) {
let mut dst = if !into.compacted {
let mut dst = if into.compacted {
DataUsageEntry::default()
} else {
into.clone()
@@ -1034,7 +1050,74 @@ async fn scan(folder: &CachedFolder, into: &mut DataUsageEntry, folder_scanner:
}
}
pub fn has_active_rules(config: &ReplicationConfiguration, prefix: &str, recursive: bool) -> bool {
fn lc_get_prefix(rule: &LifecycleRule) -> String {
if let Some(p) = &rule.prefix {
return p.to_string();
} else if let Some(filter) = &rule.filter {
if let Some(p) = &filter.prefix {
return p.to_string();
} else if let Some(and) = &filter.and {
if let Some(p) = &and.prefix {
return p.to_string();
}
}
}
"".into()
}
pub fn lc_has_active_rules(config: &BucketLifecycleConfiguration, prefix: &str) -> bool {
if config.rules.is_empty() {
return false;
}
for rule in config.rules.iter() {
if rule.status == ExpirationStatus::from_static(ExpirationStatus::DISABLED) {
continue;
}
let rule_prefix = lc_get_prefix(rule);
if !prefix.is_empty() && !rule_prefix.is_empty() {
if !prefix.starts_with(&rule_prefix) && !rule_prefix.starts_with(prefix) {
continue;
}
}
if let Some(e) = &rule.noncurrent_version_expiration {
if let Some(true) = e.noncurrent_days.map(|d| d > 0) {
return true;
}
if let Some(true) = e.newer_noncurrent_versions.map(|d| d > 0) {
return true;
}
}
if rule.noncurrent_version_transitions.is_some() {
return true;
}
if let Some(true) = rule.expiration.as_ref().map(|e| e.date.is_some()) {
return true;
}
if let Some(true) = rule.expiration.as_ref().map(|e| e.days.is_some()) {
return true;
}
if let Some(Some(true)) = rule.expiration.as_ref().map(|e| e.expired_object_delete_marker.map(|m| m)) {
return true;
}
if let Some(true) = rule.transitions.as_ref().map(|t| !t.is_empty()) {
return true;
}
if rule.transitions.is_some() {
return true;
}
}
false
}
pub fn rep_has_active_rules(config: &ReplicationConfiguration, prefix: &str, recursive: bool) -> bool {
if config.rules.is_empty() {
return false;
}
@@ -1092,8 +1175,14 @@ pub async fn scan_data_folder(
root: base_path,
get_size: get_size_fn,
old_cache: cache.clone(),
new_cache: DataUsageCache::default(),
update_cache: DataUsageCache::default(),
new_cache: DataUsageCache {
info: cache.info.clone(),
..Default::default()
},
update_cache: DataUsageCache {
info: cache.info.clone(),
..Default::default()
},
data_usage_scanner_debug: false,
heal_object_select: 0,
scan_mode: heal_scan_mode,
@@ -1121,8 +1210,7 @@ pub async fn scan_data_folder(
if s.scan_folder(&folder, &mut root).await.is_err() {
close_disk().await;
}
s.new_cache
.force_compact(DATA_SCANNER_COMPACT_AT_CHILDREN.try_into().unwrap());
s.new_cache.force_compact(DATA_SCANNER_COMPACT_AT_CHILDREN as usize);
s.new_cache.info.last_update = Some(SystemTime::now());
s.new_cache.info.next_cycle = cache.info.next_cycle;
close_disk().await;
ecstore/src/heal/data_usage_cache.rs
+5 -4
View File
@@ -10,7 +10,7 @@ use http::HeaderMap;
use path_clean::PathClean;
use rand::Rng;
use rmp_serde::Serializer;
use s3s::dto::ReplicationConfiguration;
use s3s::dto::{BucketLifecycleConfiguration, ReplicationConfiguration};
use serde::{Deserialize, Serialize};
use std::collections::{HashMap, HashSet};
use std::hash::{DefaultHasher, Hash, Hasher};
@@ -347,7 +347,8 @@ pub struct DataUsageCacheInfo {
pub next_cycle: u32,
pub last_update: Option<SystemTime>,
pub skip_healing: bool,
// todo: life_cycle
#[serde(skip)]
pub life_cycle: Option<BucketLifecycleConfiguration>,
// pub life_cycle:
#[serde(skip)]
pub updates: Option<Sender<DataUsageEntry>>,
@@ -380,7 +381,7 @@ impl DataUsageCache {
let mut retries = 0;
while retries < 5 {
let path = Path::new(BUCKET_META_PREFIX).join(name);
warn!("Loading data usage cache from backend: {}", path.display());
// warn!("Loading data usage cache from backend: {}", path.display());
match store
.get_object_reader(
RUSTFS_META_BUCKET,
@@ -588,7 +589,7 @@ impl DataUsageCache {
Some(e) => e,
None => return,
};
if top_e.children.len() > DATA_SCANNER_FORCE_COMPACT_AT_FOLDERS.try_into().unwrap() {
if top_e.children.len() > <u64 as TryInto<usize>>::try_into(DATA_SCANNER_FORCE_COMPACT_AT_FOLDERS).unwrap() {
self.reduce_children_of(&hash_path(&self.info.name), limit, true);
}
if self.cache.len() <= limit {
ecstore/src/heal/heal_ops.rs
+1 -4
View File
@@ -406,10 +406,7 @@ impl HealSequence {
async fn traverse_and_heal(h: Arc<HealSequence>) {
let buckets_only = false;
let result = match Self::heal_items(h.clone(), buckets_only).await {
Ok(_) => None,
Err(err) => Some(err),
};
let result = (Self::heal_items(h.clone(), buckets_only).await).err();
let _ = h.traverse_and_heal_done_tx.read().await.send(result).await;
}
ecstore/src/io.rs
+6 -2
View File
@@ -140,10 +140,14 @@ impl<R> EtagReader<R> {
impl<R: AsyncRead + Unpin> AsyncRead for EtagReader<R> {
fn poll_read(mut self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll<tokio::io::Result<()>> {
let befor_size = buf.filled().len();
match Pin::new(&mut self.inner).poll_read(cx, buf) {
Poll::Ready(Ok(())) => {
let bytes = buf.filled();
self.md5.update(bytes);
if buf.filled().len() > befor_size {
let bytes = &buf.filled()[befor_size..];
self.md5.update(bytes);
}
Poll::Ready(Ok(()))
}
ecstore/src/peer.rs
+1 -6
View File
@@ -80,12 +80,7 @@ impl S3PeerSys {
let mut futures = Vec::with_capacity(self.clients.len());
for client in self.clients.iter() {
// client_clon
futures.push(async move {
match client.get_bucket_info(bucket, &BucketOptions::default()).await {
Ok(_) => None,
Err(err) => Some(err),
}
});
futures.push(async move { (client.get_bucket_info(bucket, &BucketOptions::default()).await).err() });
}
let errs = join_all(futures).await;
ecstore/src/set_disk.rs
+10 -11
View File
@@ -1235,7 +1235,7 @@ impl SetDisks {
let shallow_versions: Vec<Vec<FileMetaShallowVersion>> = metadata_shallow_versions.iter().flatten().cloned().collect();
let read_quorum = (fileinfos.len() + 1) / 2;
let read_quorum = fileinfos.len().div_ceil(2);
let versions = merge_file_meta_versions(read_quorum, false, 1, &shallow_versions);
let meta = FileMeta {
versions,
@@ -3020,7 +3020,7 @@ impl SetDisks {
});
// Calc usage
let before = cache.info.last_update;
let cache = match disk.clone().ns_scanner(&cache, tx, heal_scan_mode, None).await {
let mut cache = match disk.clone().ns_scanner(&cache, tx, heal_scan_mode, None).await {
Ok(cache) => cache,
Err(_) => {
if cache.info.last_update > before {
@@ -3030,6 +3030,9 @@ impl SetDisks {
continue;
}
};
cache.info.updates = None;
let _ = task.await;
let mut root = DataUsageEntry::default();
if let Some(r) = cache.root() {
root = cache.flatten(&r);
@@ -3046,7 +3049,6 @@ impl SetDisks {
entry: root,
})
.await;
let _ = task.await;
let _ = cache.save(&cache_name.to_string_lossy()).await;
}
}
@@ -5090,7 +5092,7 @@ fn is_object_dang_ling(
});
if !valid_meta.is_valid() {
let data_blocks = (meta_arr.len() + 1) / 2;
let data_blocks = meta_arr.len().div_ceil(2);
if not_found_parts_errs > data_blocks {
return Ok(valid_meta);
}
@@ -5103,7 +5105,7 @@ fn is_object_dang_ling(
}
if valid_meta.deleted {
let data_blocks = (errs.len() + 1) / 2;
let data_blocks = errs.len().div_ceil(2);
if not_found_meta_errs > data_blocks {
return Ok(valid_meta);
}
@@ -5318,7 +5320,7 @@ async fn disks_with_all_parts(
if let Some(data) = &meta.data {
let checksum_info = meta.erasure.get_checksum_info(meta.parts[0].number);
let data_len = data.len();
let verify_err = match bitrot_verify(
let verify_err = (bitrot_verify(
Box::new(Cursor::new(data.clone())),
data_len,
meta.erasure.shard_file_size(meta.size),
@@ -5326,11 +5328,8 @@ async fn disks_with_all_parts(
checksum_info.hash,
meta.erasure.shard_size(meta.erasure.block_size),
)
.await
{
Ok(_) => None,
Err(err) => Some(err),
};
.await)
.err();
if let Some(vec) = data_errs_by_part.get_mut(&0) {
if index < vec.len() {
ecstore/src/store.rs
+12 -11
View File
@@ -608,7 +608,7 @@ impl ECStore {
let mut ress = Vec::new();
// join_all结果跟输入顺序一致
// join_all 结果跟输入顺序一致
for (i, res) in results.into_iter().enumerate() {
let index = i;
@@ -740,7 +740,7 @@ impl ECStore {
let cancel_clone = cancel.clone();
let all_buckets_clone = all_buckets.clone();
futures.push(async move {
let (tx, mut rx) = mpsc::channel(100);
let (tx, mut rx) = mpsc::channel(1);
let task = tokio::spawn(async move {
loop {
match rx.recv().await {
@@ -951,6 +951,7 @@ impl ECStore {
}
}
#[tracing::instrument(level = "info", skip(all_buckets, updates))]
async fn update_scan(
all_merged: Arc<RwLock<DataUsageCache>>,
results: Arc<RwLock<Vec<DataUsageCache>>>,
@@ -972,7 +973,7 @@ async fn update_scan(
}
w.merge(info);
}
if w.info.last_update > *last_update && w.root().is_none() {
if (last_update.is_none() || w.info.last_update > *last_update) && w.root().is_some() {
let _ = updates.send(w.dui(&w.info.name, &all_buckets)).await;
*last_update = w.info.last_update;
}
@@ -1010,7 +1011,7 @@ pub async fn all_local_disk() -> Vec<DiskStore> {
.collect()
}
// init_local_disks 初始化本地磁盘，server启动前必须初始化成功
// init_local_disks 初始化本地磁盘，server 启动前必须初始化成功
pub async fn init_local_disks(endpoint_pools: EndpointServerPools) -> Result<()> {
let opt = &DiskOption {
cleanup: true,
@@ -1279,7 +1280,7 @@ impl StorageAPI for ECStore {
// TODO: replication opts.srdelete_op
// 删除meta
// 删除 meta
self.delete_all(RUSTFS_META_BUCKET, format!("{}/{}", BUCKET_META_PREFIX, bucket).as_str())
.await?;
Ok(())
@@ -1483,7 +1484,7 @@ impl StorageAPI for ECStore {
// results.push(jh.await.unwrap());
// }
// 记录pool Index 对应的objects pool_idx -> objects idx
// 记录 pool Index 对应的 objects pool_idx -> objects idx
let mut pool_obj_idx_map = HashMap::new();
let mut orig_index_map = HashMap::new();
@@ -1533,9 +1534,9 @@ impl StorageAPI for ECStore {
if !pool_obj_idx_map.is_empty() {
for (i, sets) in self.pools.iter().enumerate() {
// 取pool idx 对应的 objects index
// 取 pool idx 对应的 objects index
if let Some(objs) = pool_obj_idx_map.get(&i) {
// 取对应obj,理论上不会none
// 取对应 obj，理论上不会 none
// let objs: Vec<ObjectToDelete> = obj_idxs.iter().filter_map(|&idx| objects.get(idx).cloned()).collect();
if objs.is_empty() {
@@ -1544,10 +1545,10 @@ impl StorageAPI for ECStore {
let (pdel_objs, perrs) = sets.delete_objects(bucket, objs.clone(), opts.clone()).await?;
// 同时存入不可能为none
// 同时存入不可能为 none
let org_indexes = orig_index_map.get(&i).unwrap();
// perrs的顺序理论上跟obj_idxs顺序一致
// perrs 的顺序理论上跟 obj_idxs 顺序一致
for (i, err) in perrs.into_iter().enumerate() {
let obj_idx = org_indexes[i];
@@ -1580,7 +1581,7 @@ impl StorageAPI for ECStore {
let object = utils::path::encode_dir_object(object);
let object = object.as_str();
// 查询在哪个pool
// 查询在哪个 pool
let (mut pinfo, errs) = self
.get_pool_info_existing_with_opts(bucket, object, &opts)
.await
ecstore/src/store_list_objects.rs
+5 -11
View File
@@ -25,7 +25,7 @@ use std::io::ErrorKind;
use std::sync::Arc;
use tokio::sync::broadcast::{self, Receiver as B_Receiver};
use tokio::sync::mpsc::{self, Receiver, Sender};
use tracing::{error, warn};
use tracing::error;
use uuid::Uuid;
const MAX_OBJECT_LIST: i32 = 1000;
@@ -248,7 +248,7 @@ impl ECStore {
..Default::default()
};
warn!("list_objects_generic opts {:?}", &opts);
// warn!("list_objects_generic opts {:?}", &opts);
// use get
if !opts.prefix.is_empty() && opts.limit == 1 && opts.marker.is_none() {
@@ -698,7 +698,7 @@ impl ECStore {
futures.push(async move {
let mut ask_disks = get_list_quorum(&opts.ask_disks, set.set_drive_count as i32);
if ask_disks == -1 {
let new_disks = get_quorum_disks(&disks, &infos, (disks.len() + 1) / 2);
let new_disks = get_quorum_disks(&disks, &infos, disks.len().div_ceil(2));
if !new_disks.is_empty() {
disks = new_disks;
} else {
@@ -1156,13 +1156,7 @@ async fn merge_entry_channels(
if let Some(entry) = &best {
let mut versions = Vec::with_capacity(to_merge.len() + 1);
let mut has_xl = {
if let Ok(meta) = entry.clone().xl_meta() {
Some(meta)
} else {
None
}
};
let mut has_xl = { entry.clone().xl_meta().ok() };
if let Some(x) = &has_xl {
versions.push(x.versions.clone());
@@ -1229,7 +1223,7 @@ impl SetDisks {
let mut ask_disks = get_list_quorum(&opts.ask_disks, self.set_drive_count as i32);
if ask_disks == -1 {
let new_disks = get_quorum_disks(&disks, &infos, (disks.len() + 1) / 2);
let new_disks = get_quorum_disks(&disks, &infos, disks.len().div_ceil(2));
if !new_disks.is_empty() {
disks = new_disks;
ask_disks = 1;
ecstore/src/utils/os/linux.rs
+3 -2
View File
@@ -151,7 +151,7 @@ fn read_drive_stats(stats_file: &str) -> Result<IOStats> {
fn read_stat(file_name: &str) -> Result<Vec<u64>> {
// 打开文件
let path = Path::new(file_name);
let file = File::open(&path)?;
let file = File::open(path)?;
// 创建一个 BufReader
let reader = io::BufReader::new(file);
@@ -161,7 +161,8 @@ fn read_stat(file_name: &str) -> Result<Vec<u64>> {
if let Some(line) = reader.lines().next() {
let line = line?;
// 分割行并解析为 u64
for token in line.trim().split_whitespace() {
// https://rust-lang.github.io/rust-clippy/master/index.html#trim_split_whitespace
for token in line.split_whitespace() {
let ui64: u64 = token.parse()?;
stats.push(ui64);
}
ecstore/src/utils/os/windows.rs
+2 -1
View File
@@ -1,7 +1,8 @@
#![allow(unsafe_code)] // TODO: audit unsafe code
use super::IOStats;
use crate::{disk::Info, error::Result};
use crate::disk::Info;
use common::error::Result;
use std::io::{Error, ErrorKind};
use std::mem;
use std::os::windows::ffi::OsStrExt;
iam/Cargo.toml
+1 -2
View File
@@ -11,7 +11,6 @@ workspace = true
[dependencies]
tokio.workspace = true
log.workspace = true
time = { workspace = true, features = ["serde-human-readable"] }
serde = { workspace = true, features = ["derive", "rc"] }
ecstore = { path = "../ecstore" }
@@ -27,7 +26,7 @@ itertools = "0.14.0"
futures.workspace = true
rand.workspace = true
base64-simd = "0.8.0"
jsonwebtoken = "9.3.0"
jsonwebtoken = { workspace = true }
tracing.workspace = true
madmin.workspace = true
lazy_static.workspace = true
iam/src/cache.rs
+6 -6
View File
@@ -6,12 +6,12 @@ use std::{
};
use arc_swap::{ArcSwap, AsRaw, Guard};
use log::warn;
use policy::{
auth::UserIdentity,
policy::{Args, PolicyDoc},
};
use time::OffsetDateTime;
use tracing::warn;
use crate::store::{GroupInfo, MappedPolicy};
@@ -63,7 +63,7 @@ impl Cache {
let mut new = CacheEntity::clone(&cur);
op(&mut new);
// 使用cas原子替换内容
// 使用 cas 原子替换内容
let prev = target.compare_and_swap(&*cur, Arc::new(new));
let swapped = Self::ptr_eq(&*cur, &*prev);
if swapped {
@@ -112,8 +112,8 @@ impl CacheInner {
// todo!()
// }
// /// 如果是临时用户，返回Ok(Some(partent_name)))
// /// 如果不是临时用户，返回Ok(None)
// /// 如果是临时用户，返回 Ok(Some(partent_name)))
// /// 如果不是临时用户，返回 Ok(None)
// fn is_temp_user(&self, user_name: &str) -> crate::Result<Option<&str>> {
// let user = self
// .get_user(user_name)
@@ -126,8 +126,8 @@ impl CacheInner {
// }
// }
// /// 如果是临时用户，返回Ok(Some(partent_name)))
// /// 如果不是临时用户，返回Ok(None)
// /// 如果是临时用户，返回 Ok(Some(partent_name)))
// /// 如果不是临时用户，返回 Ok(None)
// fn is_service_account(&self, user_name: &str) -> crate::Result<Option<&str>> {
// let user = self
// .get_user(user_name)
iam/src/lib.rs
+2 -1
View File
@@ -1,12 +1,12 @@
use common::error::{Error, Result};
use ecstore::store::ECStore;
use error::Error as IamError;
use log::debug;
use manager::IamCache;
use policy::auth::Credentials;
use std::sync::{Arc, OnceLock};
use store::object::ObjectStore;
use sys::IamSys;
use tracing::{debug, instrument};
pub mod cache;
pub mod error;
@@ -51,6 +51,7 @@ pub fn get_global_action_cred() -> Option<Credentials> {
GLOBAL_ACTIVE_CRED.get().cloned()
}
#[instrument(skip(ecstore))]
pub async fn init_iam_sys(ecstore: Arc<ECStore>) -> Result<()> {
debug!("init iam system");
let s = IamCache::new(ObjectStore::new(ecstore)).await;
iam/src/manager.rs
+2 -2
View File
@@ -11,7 +11,6 @@ use crate::{
use common::error::{Error, Result};
use ecstore::config::error::is_err_config_not_found;
use ecstore::utils::{crypto::base64_encode, path::path_join_buf};
use log::{debug, warn};
use madmin::{AccountStatus, AddOrUpdateUserReq, GroupDesc};
use policy::{
arn::ARN,
@@ -40,6 +39,7 @@ use tokio::{
},
};
use tracing::error;
use tracing::warn;
const IAM_FORMAT_FILE: &str = "format.json";
const IAM_FORMAT_VERSION_1: i32 = 1;
@@ -135,7 +135,7 @@ where
}
async fn load(self: Arc<Self>) -> Result<()> {
debug!("load iam to cache");
// debug!("load iam to cache");
self.api.load_all(&self.cache).await?;
self.last_timestamp
.store(OffsetDateTime::now_utc().unix_timestamp(), Ordering::Relaxed);
iam/src/store/object.rs
+6 -6
View File
@@ -24,7 +24,7 @@ use serde::{de::DeserializeOwned, Serialize};
use std::{collections::HashMap, sync::Arc};
use tokio::sync::broadcast::{self, Receiver as B_Receiver};
use tokio::sync::mpsc::{self, Sender};
use tracing::{debug, info, warn};
use tracing::{info, warn};
lazy_static! {
pub static ref IAM_CONFIG_PREFIX: String = format!("{}/iam", RUSTFS_CONFIG_PREFIX);
@@ -133,7 +133,7 @@ impl ObjectStore {
}
async fn list_iam_config_items(&self, prefix: &str, ctx_rx: B_Receiver<bool>, sender: Sender<StringOrErr>) {
debug!("list iam config items, prefix: {}", &prefix);
// debug!("list iam config items, prefix: {}", &prefix);
// todo, 实现walk，使用walk
@@ -675,11 +675,11 @@ impl Store for ObjectStore {
async fn load_all(&self, cache: &Cache) -> Result<()> {
let listed_config_items = self.list_all_iamconfig_items().await?;
let mut policy_docs_cache = CacheEntity::new(get_default_policyes());
if let Some(policies_list) = listed_config_items.get(POLICIES_LIST_KEY) {
let mut policies_list = policies_list.clone();
let mut policy_docs_cache = CacheEntity::new(get_default_policyes());
loop {
if policies_list.len() < 32 {
let policy_docs = self.load_policy_doc_concurrent(&policies_list).await?;
@@ -712,10 +712,10 @@ impl Store for ObjectStore {
policies_list = policies_list.split_off(32);
}
cache.policy_docs.store(Arc::new(policy_docs_cache.update_load_time()));
}
cache.policy_docs.store(Arc::new(policy_docs_cache.update_load_time()));
let mut user_items_cache = CacheEntity::default();
// users
iam/src/sys.rs
+12
View File
@@ -121,6 +121,18 @@ impl<T: Store> IamSys<T> {
// TODO: notification
}
pub async fn get_role_policy(&self, arn_str: &str) -> Result<(ARN, String)> {
let Some(arn) = ARN::parse(arn_str).ok() else {
return Err(Error::msg("Invalid ARN"));
};
let Some(policy) = self.roles_map.get(&arn) else {
return Err(Error::msg("No such role"));
};
Ok((arn, policy.clone()))
}
pub async fn delete_user(&self, name: &str, _notify: bool) -> Result<()> {
self.store.delete_user(name, UserType::Reg).await
// TODO: notification
madmin/Cargo.toml
+1
View File
@@ -18,3 +18,4 @@ serde.workspace = true
serde_json.workspace = true
time.workspace = true
tracing.workspace = true
s3s.workspace = true
madmin/src/user.rs
+40
View File
@@ -1,6 +1,9 @@
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use time::OffsetDateTime;
use crate::BackendInfo;
#[derive(Debug, Serialize, Deserialize, Default, PartialEq, Eq)]
pub enum AccountStatus {
#[serde(rename = "enabled")]
@@ -221,3 +224,40 @@ impl UpdateServiceAccountReq {
Ok(())
}
}
#[derive(Serialize, Deserialize, Debug, Default)]
pub struct AccountInfo {
pub account_name: String,
pub server: BackendInfo,
pub policy: serde_json::Value, // Use iam/policy::parse to parse the result, to be done by the caller.
pub buckets: Vec<BucketAccessInfo>,
}
#[derive(Serialize, Deserialize, Debug, Default)]
pub struct BucketAccessInfo {
pub name: String,
pub size: u64,
pub objects: u64,
pub object_sizes_histogram: HashMap<String, u64>,
pub object_versions_histogram: HashMap<String, u64>,
pub details: Option<BucketDetails>,
pub prefix_usage: HashMap<String, u64>,
#[serde(rename = "expiration", with = "time::serde::rfc3339::option")]
pub created: Option<OffsetDateTime>,
pub access: AccountAccess,
}
#[derive(Serialize, Deserialize, Debug, Default)]
pub struct BucketDetails {
pub versioning: bool,
pub versioning_suspended: bool,
pub locking: bool,
pub replication: bool,
// pub tagging: Option<Tagging>,
}
#[derive(Serialize, Deserialize, Debug, Default)]
pub struct AccountAccess {
pub read: bool,
pub write: bool,
}
policy/Cargo.toml
+1 -2
View File
@@ -11,7 +11,6 @@ workspace = true
[dependencies]
tokio.workspace = true
log.workspace = true
time = { workspace = true, features = ["serde-human-readable"] }
serde = { workspace = true, features = ["derive", "rc"] }
serde_json.workspace = true
@@ -25,7 +24,7 @@ itertools = "0.14.0"
futures.workspace = true
rand.workspace = true
base64-simd = "0.8.0"
jsonwebtoken = "9.3.0"
jsonwebtoken = { workspace = true }
tracing.workspace = true
madmin.workspace = true
lazy_static.workspace = true
policy/src/policy/action.rs
+229 -8
View File
@@ -54,6 +54,7 @@ pub enum Action {
AdminAction(AdminAction),
StsAction(StsAction),
KmsAction(KmsAction),
None,
}
impl Action {
@@ -69,6 +70,7 @@ impl From<&Action> for &str {
Action::AdminAction(s) => s.into(),
Action::StsAction(s) => s.into(),
Action::KmsAction(s) => s.into(),
Action::None => "",
}
}
}
@@ -232,35 +234,254 @@ pub enum S3Action {
PutObjectFanOutAction,
}
// #[derive(Serialize, Deserialize, Hash, PartialEq, Eq, Clone, EnumString, IntoStaticStr, Debug, Copy)]
// #[serde(try_from = "&str", into = "&str")]
// pub enum AdminAction {
// #[strum(serialize = "admin:*")]
// AllActions,
// #[strum(serialize = "admin:Profiling")]
// ProfilingAdminAction,
// #[strum(serialize = "admin:ServerTrace")]
// TraceAdminAction,
// #[strum(serialize = "admin:ConsoleLog")]
// ConsoleLogAdminAction,
// #[strum(serialize = "admin:ServerInfo")]
// ServerInfoAdminAction,
// #[strum(serialize = "admin:OBDInfo")]
// HealthInfoAdminAction,
// #[strum(serialize = "admin:TopLocksInfo")]
// TopLocksAdminAction,
// #[strum(serialize = "admin:LicenseInfo")]
// LicenseInfoAdminAction,
// #[strum(serialize = "admin:BandwidthMonitor")]
// BandwidthMonitorAction,
// #[strum(serialize = "admin:InspectData")]
// InspectDataAction,
// #[strum(serialize = "admin:Prometheus")]
// PrometheusAdminAction,
// #[strum(serialize = "admin:ListServiceAccounts")]
// ListServiceAccountsAdminAction,
// #[strum(serialize = "admin:CreateServiceAccount")]
// CreateServiceAccountAdminAction,
// }
// AdminAction - admin policy action.
#[derive(Serialize, Deserialize, Hash, PartialEq, Eq, Clone, EnumString, IntoStaticStr, Debug, Copy)]
#[serde(try_from = "&str", into = "&str")]
pub enum AdminAction {
#[strum(serialize = "admin:*")]
AllActions,
#[strum(serialize = "admin:Heal")]
HealAdminAction,
#[strum(serialize = "admin:Decommission")]
DecommissionAdminAction,
#[strum(serialize = "admin:Rebalance")]
RebalanceAdminAction,
#[strum(serialize = "admin:StorageInfo")]
StorageInfoAdminAction,
#[strum(serialize = "admin:Prometheus")]
PrometheusAdminAction,
#[strum(serialize = "admin:DataUsageInfo")]
DataUsageInfoAdminAction,
#[strum(serialize = "admin:ForceUnlock")]
ForceUnlockAdminAction,
#[strum(serialize = "admin:TopLocksInfo")]
TopLocksAdminAction,
#[strum(serialize = "admin:Profiling")]
ProfilingAdminAction,
#[strum(serialize = "admin:ServerTrace")]
TraceAdminAction,
#[strum(serialize = "admin:ConsoleLog")]
ConsoleLogAdminAction,
#[strum(serialize = "admin:KMSCreateKey")]
KMSCreateKeyAdminAction,
#[strum(serialize = "admin:KMSKeyStatus")]
KMSKeyStatusAdminAction,
#[strum(serialize = "admin:ServerInfo")]
ServerInfoAdminAction,
#[strum(serialize = "admin:OBDInfo")]
HealthInfoAdminAction,
#[strum(serialize = "admin:TopLocksInfo")]
TopLocksAdminAction,
#[strum(serialize = "admin:LicenseInfo")]
LicenseInfoAdminAction,
#[strum(serialize = "admin:BandwidthMonitor")]
BandwidthMonitorAction,
#[strum(serialize = "admin:InspectData")]
InspectDataAction,
#[strum(serialize = "admin:Prometheus")]
PrometheusAdminAction,
#[strum(serialize = "admin:ListServiceAccounts")]
ListServiceAccountsAdminAction,
#[strum(serialize = "admin:ServerUpdate")]
ServerUpdateAdminAction,
#[strum(serialize = "admin:ServiceRestart")]
ServiceRestartAdminAction,
#[strum(serialize = "admin:ServiceStop")]
ServiceStopAdminAction,
#[strum(serialize = "admin:ServiceFreeze")]
ServiceFreezeAdminAction,
#[strum(serialize = "admin:ConfigUpdate")]
ConfigUpdateAdminAction,
#[strum(serialize = "admin:CreateUser")]
CreateUserAdminAction,
#[strum(serialize = "admin:DeleteUser")]
DeleteUserAdminAction,
#[strum(serialize = "admin:ListUsers")]
ListUsersAdminAction,
#[strum(serialize = "admin:EnableUser")]
EnableUserAdminAction,
#[strum(serialize = "admin:DisableUser")]
DisableUserAdminAction,
#[strum(serialize = "admin:GetUser")]
GetUserAdminAction,
#[strum(serialize = "admin:SiteReplicationAdd")]
SiteReplicationAddAction,
#[strum(serialize = "admin:SiteReplicationDisable")]
SiteReplicationDisableAction,
#[strum(serialize = "admin:SiteReplicationRemove")]
SiteReplicationRemoveAction,
#[strum(serialize = "admin:SiteReplicationResync")]
SiteReplicationResyncAction,
#[strum(serialize = "admin:SiteReplicationInfo")]
SiteReplicationInfoAction,
#[strum(serialize = "admin:SiteReplicationOperation")]
SiteReplicationOperationAction,
#[strum(serialize = "admin:CreateServiceAccount")]
CreateServiceAccountAdminAction,
#[strum(serialize = "admin:UpdateServiceAccount")]
UpdateServiceAccountAdminAction,
#[strum(serialize = "admin:RemoveServiceAccount")]
RemoveServiceAccountAdminAction,
#[strum(serialize = "admin:ListServiceAccounts")]
ListServiceAccountsAdminAction,
#[strum(serialize = "admin:ListTemporaryAccounts")]
ListTemporaryAccountsAdminAction,
#[strum(serialize = "admin:AddUserToGroup")]
AddUserToGroupAdminAction,
#[strum(serialize = "admin:RemoveUserFromGroup")]
RemoveUserFromGroupAdminAction,
#[strum(serialize = "admin:GetGroup")]
GetGroupAdminAction,
#[strum(serialize = "admin:ListGroups")]
ListGroupsAdminAction,
#[strum(serialize = "admin:EnableGroup")]
EnableGroupAdminAction,
#[strum(serialize = "admin:DisableGroup")]
DisableGroupAdminAction,
#[strum(serialize = "admin:CreatePolicy")]
CreatePolicyAdminAction,
#[strum(serialize = "admin:DeletePolicy")]
DeletePolicyAdminAction,
#[strum(serialize = "admin:GetPolicy")]
GetPolicyAdminAction,
#[strum(serialize = "admin:AttachUserOrGroupPolicy")]
AttachPolicyAdminAction,
#[strum(serialize = "admin:UpdatePolicyAssociation")]
UpdatePolicyAssociationAction,
#[strum(serialize = "admin:ListUserPolicies")]
ListUserPoliciesAdminAction,
#[strum(serialize = "admin:SetBucketQuota")]
SetBucketQuotaAdminAction,
#[strum(serialize = "admin:GetBucketQuota")]
GetBucketQuotaAdminAction,
#[strum(serialize = "admin:SetBucketTarget")]
SetBucketTargetAction,
#[strum(serialize = "admin:GetBucketTarget")]
GetBucketTargetAction,
#[strum(serialize = "admin:ReplicationDiff")]
ReplicationDiff,
#[strum(serialize = "admin:ImportBucketMetadata")]
ImportBucketMetadataAction,
#[strum(serialize = "admin:ExportBucketMetadata")]
ExportBucketMetadataAction,
#[strum(serialize = "admin:SetTier")]
SetTierAction,
#[strum(serialize = "admin:ListTier")]
ListTierAction,
#[strum(serialize = "admin:ExportIAM")]
ExportIAMAction,
#[strum(serialize = "admin:ImportIAM")]
ImportIAMAction,
#[strum(serialize = "admin:ListBatchJobs")]
ListBatchJobsAction,
#[strum(serialize = "admin:DescribeBatchJob")]
DescribeBatchJobAction,
#[strum(serialize = "admin:StartBatchJob")]
StartBatchJobAction,
#[strum(serialize = "admin:CancelBatchJob")]
CancelBatchJobAction,
#[strum(serialize = "admin:*")]
AllAdminActions,
}
impl AdminAction {
// IsValid - checks if action is valid or not.
pub fn is_valid(&self) -> bool {
matches!(
self,
AdminAction::HealAdminAction
| AdminAction::DecommissionAdminAction
| AdminAction::RebalanceAdminAction
| AdminAction::StorageInfoAdminAction
| AdminAction::PrometheusAdminAction
| AdminAction::DataUsageInfoAdminAction
| AdminAction::ForceUnlockAdminAction
| AdminAction::TopLocksAdminAction
| AdminAction::ProfilingAdminAction
| AdminAction::TraceAdminAction
| AdminAction::ConsoleLogAdminAction
| AdminAction::KMSCreateKeyAdminAction
| AdminAction::KMSKeyStatusAdminAction
| AdminAction::ServerInfoAdminAction
| AdminAction::HealthInfoAdminAction
| AdminAction::LicenseInfoAdminAction
| AdminAction::BandwidthMonitorAction
| AdminAction::InspectDataAction
| AdminAction::ServerUpdateAdminAction
| AdminAction::ServiceRestartAdminAction
| AdminAction::ServiceStopAdminAction
| AdminAction::ServiceFreezeAdminAction
| AdminAction::ConfigUpdateAdminAction
| AdminAction::CreateUserAdminAction
| AdminAction::DeleteUserAdminAction
| AdminAction::ListUsersAdminAction
| AdminAction::EnableUserAdminAction
| AdminAction::DisableUserAdminAction
| AdminAction::GetUserAdminAction
| AdminAction::SiteReplicationAddAction
| AdminAction::SiteReplicationDisableAction
| AdminAction::SiteReplicationRemoveAction
| AdminAction::SiteReplicationResyncAction
| AdminAction::SiteReplicationInfoAction
| AdminAction::SiteReplicationOperationAction
| AdminAction::CreateServiceAccountAdminAction
| AdminAction::UpdateServiceAccountAdminAction
| AdminAction::RemoveServiceAccountAdminAction
| AdminAction::ListServiceAccountsAdminAction
| AdminAction::ListTemporaryAccountsAdminAction
| AdminAction::AddUserToGroupAdminAction
| AdminAction::RemoveUserFromGroupAdminAction
| AdminAction::GetGroupAdminAction
| AdminAction::ListGroupsAdminAction
| AdminAction::EnableGroupAdminAction
| AdminAction::DisableGroupAdminAction
| AdminAction::CreatePolicyAdminAction
| AdminAction::DeletePolicyAdminAction
| AdminAction::GetPolicyAdminAction
| AdminAction::AttachPolicyAdminAction
| AdminAction::UpdatePolicyAssociationAction
| AdminAction::ListUserPoliciesAdminAction
| AdminAction::SetBucketQuotaAdminAction
| AdminAction::GetBucketQuotaAdminAction
| AdminAction::SetBucketTargetAction
| AdminAction::GetBucketTargetAction
| AdminAction::ReplicationDiff
| AdminAction::ImportBucketMetadataAction
| AdminAction::ExportBucketMetadataAction
| AdminAction::SetTierAction
| AdminAction::ListTierAction
| AdminAction::ExportIAMAction
| AdminAction::ImportIAMAction
| AdminAction::ListBatchJobsAction
| AdminAction::DescribeBatchJobAction
| AdminAction::StartBatchJobAction
| AdminAction::CancelBatchJobAction
| AdminAction::AllAdminActions
)
}
}
#[derive(Serialize, Deserialize, Hash, PartialEq, Eq, Clone, EnumString, IntoStaticStr, Debug, Copy)]
policy/src/policy/policy.rs
+2 -2
View File
@@ -240,7 +240,7 @@ fn get_values_from_claims(claims: &HashMap<String, Value>, claim_name: &str) ->
(s, false)
}
fn get_policies_from_claims(claims: &HashMap<String, Value>, policy_claim_name: &str) -> (HashSet<String>, bool) {
pub fn get_policies_from_claims(claims: &HashMap<String, Value>, policy_claim_name: &str) -> (HashSet<String>, bool) {
get_values_from_claims(claims, policy_claim_name)
}
@@ -401,7 +401,7 @@ pub mod default {
effect: Effect::Allow,
actions: ActionSet({
let mut hash_set = HashSet::new();
hash_set.insert(Action::AdminAction(AdminAction::AllActions));
hash_set.insert(Action::AdminAction(AdminAction::AllAdminActions));
hash_set
}),
not_actions: ActionSet(Default::default()),
rustfs/Cargo.toml
+35 -32
View File
@@ -16,66 +16,70 @@ workspace = true
[dependencies]
madmin.workspace = true
log.workspace = true
api = { path = "../s3select/api" }
appauth = { version = "0.0.1", path = "../appauth" }
atoi = { workspace = true }
atomic_enum = { workspace = true }
axum.workspace = true
axum-extra = { workspace = true }
axum-server = { workspace = true }
async-trait.workspace = true
bytes.workspace = true
chrono = { workspace = true }
clap.workspace = true
crypto = { path = "../crypto" }
datafusion = { workspace = true }
common.workspace = true
const-str = { version = "0.6.1", features = ["std", "proc"] }
ecstore.workspace = true
policy.workspace =true
policy.workspace = true
flatbuffers.workspace = true
futures.workspace = true
futures-util.workspace = true
h2 = "0.4.7"
hyper.workspace = true
hyper-util.workspace = true
http.workspace = true
http-body.workspace = true
iam = { path = "../iam" }
lock.workspace = true
local-ip-address = { workspace = true }
matchit = { workspace = true }
mime.workspace = true
netif.workspace = true
mime_guess = "2.0.5"
pin-project-lite.workspace = true
prost.workspace = true
prost-types.workspace = true
protos.workspace = true
protobuf.workspace = true
query = { path = "../s3select/query" }
rmp-serde.workspace = true
rustfs-obs = { workspace = true }
rustls.workspace = true
rustls-pemfile.workspace = true
rustls-pki-types.workspace = true
rust-embed = { workspace = true, features = ["interpolate-folder-path"] }
s3s.workspace = true
serde.workspace = true
serde_json.workspace = true
serde_urlencoded = { workspace = true }
shadow-rs.workspace = true
tracing.workspace = true
time = { workspace = true, features = ["parsing", "formatting", "serde"] }
tokio-util = { version = "0.7.13", features = ["io", "compat"] }
tokio-util.workspace = true
tokio = { workspace = true, features = [
"rt-multi-thread",
"macros",
"net",
"signal",
] }
tokio-rustls.workspace = true
lazy_static.workspace = true
tokio-stream.workspace = true
tonic = { version = "0.12.3", features = ["gzip"] }
tonic-reflection.workspace = true
tonic = { workspace = true }
tower.workspace = true
tracing-error.workspace = true
tracing-subscriber.workspace = true
transform-stream.workspace = true
uuid = "1.15.1"
url.workspace = true
axum.workspace = true
matchit = "0.8.6"
shadow-rs.workspace = true
const-str = { version = "0.6.1", features = ["std", "proc"] }
atoi = "2.0.0"
serde_urlencoded = "0.7.1"
crypto = { path = "../crypto" }
iam = { path = "../iam" }
jsonwebtoken = "9.3.0"
tower-http = { version = "0.6.2", features = ["cors"] }
mime_guess = "2.0.5"
rust-embed = { workspace = true, features = ["interpolate-folder-path"] }
local-ip-address = { workspace = true }
chrono = { workspace = true }
tower-http.workspace = true
uuid = { workspace = true }
[target.'cfg(target_os = "linux")'.dependencies]
libsystemd.workspace = true
[build-dependencies]
prost-build.workspace = true
@@ -87,14 +91,13 @@ futures-util.workspace = true
# uuid = { version = "1.8.0", features = ["v4", "fast-rng", "serde"] }
ecstore = { path = "../ecstore" }
s3s.workspace = true
clap = { version = "4.5.31", features = ["derive", "env"] }
tracing-subscriber = { version = "0.3.19", features = ["env-filter", "time"] }
hyper-util = { version = "0.1.10", features = [
clap = { workspace = true }
hyper-util = { workspace = true, features = [
"tokio",
"server-auto",
"server-graceful",
] }
transform-stream = "0.3.1"
transform-stream = { workspace = true }
netif = "0.1.6"
shadow-rs.workspace = true
# pin-utils = "0.1.0"
rustfs/README.md
+30
View File
@@ -0,0 +1,30 @@
rustfs/
├── Cargo.toml
├── src/
│ ├── main.rs # 主入口
│ ├── admin/
│ │ └── mod.rs # 管理接口
│ ├── auth/
│ │ └── mod.rs # 认证模块
│ ├── config/
│ │ ├── mod.rs # 配置模块
│ │ └── options.rs # 命令行参数
│ ├── console/
│ │ ├── mod.rs # 控制台模块
│ │ └── server.rs # 控制台服务器
│ ├── grpc/
│ │ └── mod.rs # gRPC 服务
│ ├── license/
│ │ └── mod.rs # 许可证管理
│ ├── logging/
│ │ └── mod.rs # 日志管理
│ ├── server/
│ │ ├── mod.rs # 服务器实现
│ │ ├── connection.rs # 连接处理
│ │ ├── service.rs # 服务实现
│ │ └── state.rs # 状态管理
│ ├── storage/
│ │ ├── mod.rs # 存储模块
│ │ └── fs.rs # 文件系统实现
│ └── utils/
│ └── mod.rs # 工具函数
rustfs/src/admin/handlers.rs
+184 -40
View File
@@ -1,11 +1,11 @@
use super::router::Operation;
use ::policy::policy::action::{Action, S3Action};
use ::policy::policy::resource::Resource;
use ::policy::policy::statement::BPStatement;
use ::policy::policy::{ActionSet, BucketPolicy, Effect, ResourceSet};
use crate::auth::check_key_valid;
use crate::auth::get_condition_values;
use crate::auth::get_session_token;
use bytes::Bytes;
use common::error::Error as ec_Error;
use ecstore::admin_server_info::get_server_info;
use ecstore::bucket::versioning_sys::BucketVersioningSys;
use ecstore::global::GLOBAL_ALlHealState;
use ecstore::heal::data_usage::load_data_usage_from_backend;
use ecstore::heal::heal_commands::HealOpts;
@@ -14,14 +14,22 @@ use ecstore::metrics_realtime::{collect_local_metrics, CollectMetricsOpts, Metri
use ecstore::new_object_layer_fn;
use ecstore::peer::is_reserved_or_invalid_bucket;
use ecstore::store::is_valid_object_prefix;
use ecstore::store_api::BucketOptions;
use ecstore::store_api::StorageAPI;
use ecstore::utils::path::path_join;
use futures::{Stream, StreamExt};
use http::{HeaderMap, Uri};
use hyper::StatusCode;
use iam::get_global_action_cred;
use iam::store::MappedPolicy;
use madmin::metrics::RealtimeMetrics;
use madmin::utils::parse_duration;
use matchit::Params;
use policy::policy::action::Action;
use policy::policy::action::S3Action;
use policy::policy::default::DEFAULT_POLICIES;
use policy::policy::Args;
use policy::policy::BucketPolicy;
use s3s::header::CONTENT_TYPE;
use s3s::stream::{ByteStream, DynByteStream};
use s3s::{s3_error, Body, S3Error, S3Request, S3Response, S3Result};
@@ -40,7 +48,7 @@ use tokio_stream::wrappers::ReceiverStream;
use tracing::{error, info, warn};
pub mod group;
pub mod policy;
pub mod policys;
pub mod pools;
pub mod service_account;
pub mod sts;
@@ -59,49 +67,187 @@ pub struct AccountInfoHandler {}
#[async_trait::async_trait]
impl Operation for AccountInfoHandler {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
warn!("handle AccountInfoHandler");
let Some(cred) = req.credentials else { return Err(s3_error!(InvalidRequest, "get cred failed")) };
warn!("AccountInfoHandler cread {:?}", &cred);
let Some(store) = new_object_layer_fn() else {
return Err(S3Error::with_message(S3ErrorCode::InternalError, "Not init".to_string()));
};
// test policy
let Some(input_cred) = req.credentials else {
return Err(s3_error!(InvalidRequest, "get cred failed"));
};
let mut s3_all_act = HashSet::with_capacity(1);
s3_all_act.insert(Action::S3Action(S3Action::AllActions));
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key).await?;
let mut all_res = HashSet::with_capacity(1);
all_res.insert(Resource::S3("*".to_string()));
let Ok(iam_store) = iam::get() else { return Err(s3_error!(InvalidRequest, "iam not init")) };
let bucket_policy = BucketPolicy {
id: "".into(),
version: "2012-10-17".to_owned(),
statements: vec![BPStatement {
sid: "".into(),
effect: Effect::Allow,
actions: ActionSet(s3_all_act.clone()),
resources: ResourceSet(all_res),
let default_claims = HashMap::new();
let claims = cred.claims.as_ref().unwrap_or(&default_claims);
let cred_clone = cred.clone();
let conditions = get_condition_values(&req.headers, &cred_clone);
let cred_clone = Arc::new(cred_clone);
let conditions = Arc::new(conditions);
let is_allow = Box::new({
let iam_clone = Arc::clone(&iam_store);
let cred_clone = Arc::clone(&cred_clone);
let conditions = Arc::clone(&conditions);
move |name: String| {
let iam_clone = Arc::clone(&iam_clone);
let cred_clone = Arc::clone(&cred_clone);
let conditions = Arc::clone(&conditions);
async move {
let (mut rd, mut wr) = (false, false);
if !iam_clone
.is_allowed(&Args {
account: &cred_clone.access_key,
groups: &cred_clone.groups,
action: Action::S3Action(S3Action::ListBucketAction),
bucket: &name,
conditions: &conditions,
is_owner: owner,
object: "",
claims,
deny_only: false,
})
.await
{
rd = true
}
if !iam_clone
.is_allowed(&Args {
account: &cred_clone.access_key,
groups: &cred_clone.groups,
action: Action::S3Action(S3Action::GetBucketLocationAction),
bucket: &name,
conditions: &conditions,
is_owner: owner,
object: "",
claims,
deny_only: false,
})
.await
{
rd = true
}
if !iam_clone
.is_allowed(&Args {
account: &cred_clone.access_key,
groups: &cred_clone.groups,
action: Action::S3Action(S3Action::PutObjectAction),
bucket: &name,
conditions: &conditions,
is_owner: owner,
object: "",
claims,
deny_only: false,
})
.await
{
wr = true
}
(rd, wr)
}
}
});
let account_name = if cred.is_temp() || cred.is_service_account() {
cred.parent_user.clone()
} else {
cred.access_key.clone()
};
let claims_args = Args {
account: "",
groups: &None,
action: Action::None,
bucket: "",
conditions: &HashMap::new(),
is_owner: false,
object: "",
claims,
deny_only: false,
};
let role_arn = claims_args.get_role_arn();
// TODO: get_policies_from_claims(claims);
let Some(admin_cred) = get_global_action_cred() else {
return Err(S3Error::with_message(
S3ErrorCode::InternalError,
"get_global_action_cred failed".to_string(),
));
};
let mut effective_policy: policy::policy::Policy = Default::default();
if account_name == admin_cred.access_key {
for (name, p) in DEFAULT_POLICIES.iter() {
if *name == "consoleAdmin" {
effective_policy = p.clone();
break;
}
}
} else if let Some(arn) = role_arn {
let (_, policy_name) = iam_store
.get_role_policy(arn)
.await
.map_err(|e| S3Error::with_message(S3ErrorCode::InternalError, e.to_string()))?;
let policies = MappedPolicy::new(&policy_name).to_slice();
effective_policy = iam_store.get_combined_policy(&policies).await;
} else {
let policies = iam_store
.policy_db_get(&account_name, &cred.groups)
.await
.map_err(|e| S3Error::with_message(S3ErrorCode::InternalError, format!("get policy failed: {}", e)))?;
effective_policy = iam_store.get_combined_policy(&policies).await;
};
let policy_str = serde_json::to_string(&effective_policy)
.map_err(|_e| S3Error::with_message(S3ErrorCode::InternalError, "parse policy failed"))?;
let mut account_info = madmin::AccountInfo {
account_name,
server: store.backend_info().await,
policy: serde_json::Value::String(policy_str),
..Default::default()
};
// TODO: bucket policy
let buckets = store
.list_bucket(&BucketOptions {
cached: true,
..Default::default()
}],
};
})
.await
.map_err(|e| S3Error::with_message(S3ErrorCode::InternalError, e.to_string()))?;
// let policy = bucket_policy
// .marshal_msg()
// .map_err(|_e| S3Error::with_message(S3ErrorCode::InternalError, "parse policy failed"))?;
for bucket in buckets.iter() {
let (rd, wr) = is_allow(bucket.name.clone()).await;
if rd || wr {
// TODO: BucketQuotaSys
// TODO: other attributes
account_info.buckets.push(madmin::BucketAccessInfo {
name: bucket.name.clone(),
details: Some(madmin::BucketDetails {
versioning: BucketVersioningSys::enabled(bucket.name.as_str()).await,
versioning_suspended: BucketVersioningSys::suspended(bucket.name.as_str()).await,
..Default::default()
}),
created: bucket.created,
access: madmin::AccountAccess { read: rd, write: wr },
..Default::default()
});
}
}
let backend_info = store.backend_info().await;
let info = AccountInfo {
account_name: cred.access_key,
server: backend_info,
policy: bucket_policy,
};
let data = serde_json::to_vec(&info)
let data = serde_json::to_vec(&account_info)
.map_err(|_e| S3Error::with_message(S3ErrorCode::InternalError, "parse accountInfo failed"))?;
let mut header = HeaderMap::new();
@@ -126,8 +272,6 @@ pub struct ServerInfoHandler {}
#[async_trait::async_trait]
impl Operation for ServerInfoHandler {
async fn call(&self, _req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
warn!("handle ServerInfoHandler");
let info = get_server_info(true).await;
let data = serde_json::to_vec(&info)
rustfs/src/admin/handlers/policy.rs → rustfs/src/admin/handlers/policys.rs
+1 -2
View File
@@ -1,5 +1,3 @@
use std::collections::HashMap;
use crate::admin::{router::Operation, utils::has_space_be};
use http::{HeaderMap, StatusCode};
use iam::{error::is_err_no_such_user, get_global_action_cred, store::MappedPolicy};
@@ -8,6 +6,7 @@ use policy::policy::Policy;
use s3s::{header::CONTENT_TYPE, s3_error, Body, S3Error, S3ErrorCode, S3Request, S3Response, S3Result};
use serde::Deserialize;
use serde_urlencoded::from_bytes;
use std::collections::HashMap;
use tracing::warn;
#[derive(Debug, Deserialize, Default)]
rustfs/src/admin/handlers/pools.rs
+5 -1
View File
@@ -4,6 +4,7 @@ use matchit::Params;
use s3s::{header::CONTENT_TYPE, s3_error, Body, S3Error, S3ErrorCode, S3Request, S3Response, S3Result};
use serde::Deserialize;
use serde_urlencoded::from_bytes;
use tokio::sync::broadcast;
use tracing::warn;
use crate::{admin::router::Operation, storage::error::to_s3_error};
@@ -159,6 +160,9 @@ impl Operation for StartDecommission {
let pools: Vec<&str> = query.pool.split(",").collect();
let mut pools_indices = Vec::with_capacity(pools.len());
// TODO: ctx
let (_ctx_tx, ctx_rx) = broadcast::channel::<bool>(1);
for pool in pools.iter() {
let idx = {
if is_byid {
@@ -188,7 +192,7 @@ impl Operation for StartDecommission {
}
if !pools_indices.is_empty() {
store.decommission(pools_indices).await.map_err(to_s3_error)?;
store.decommission(ctx_rx, pools_indices).await.map_err(to_s3_error)?;
}
Ok(S3Response::new((StatusCode::OK, Body::default())))
rustfs/src/admin/handlers/service_account.rs
+153 -24
View File
@@ -1,4 +1,5 @@
use crate::admin::utils::has_space_be;
use crate::auth::{get_condition_values, get_session_token};
use crate::{admin::router::Operation, auth::check_key_valid};
use http::HeaderMap;
use hyper::StatusCode;
@@ -12,7 +13,8 @@ use madmin::{
ServiceAccountInfo, UpdateServiceAccountReq,
};
use matchit::Params;
use policy::policy::Policy;
use policy::policy::action::{Action, AdminAction};
use policy::policy::{Args, Policy};
use s3s::S3ErrorCode::InvalidRequest;
use s3s::{header::CONTENT_TYPE, s3_error, Body, S3Error, S3ErrorCode, S3Request, S3Response, S3Result};
use serde::Deserialize;
@@ -29,7 +31,8 @@ impl Operation for AddServiceAccount {
return Err(s3_error!(InvalidRequest, "get cred failed"));
};
let (cred, _owner) = check_key_valid(&req.headers, &req_cred.access_key).await?;
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &req_cred.access_key).await?;
let mut input = req.input;
let body = match input.store_all_unlimited().await {
@@ -89,6 +92,25 @@ impl Operation for AddServiceAccount {
let Ok(iam_store) = iam::get() else { return Err(s3_error!(InvalidRequest, "iam not init")) };
let deny_only = cred.access_key == target_user || cred.parent_user == target_user;
if !iam_store
.is_allowed(&Args {
account: &cred.access_key,
groups: &cred.groups,
action: Action::AdminAction(AdminAction::CreateServiceAccountAdminAction),
bucket: "",
conditions: &get_condition_values(&req.headers, &cred),
is_owner: owner,
object: "",
claims: cred.claims.as_ref().unwrap_or(&HashMap::new()),
deny_only,
})
.await
{
return Err(s3_error!(AccessDenied, "access denied"));
}
if target_user != cred.access_key {
let has_user = iam_store.get_user(&target_user).await;
if has_user.is_none() && target_user != sys_cred.access_key {
@@ -210,7 +232,31 @@ impl Operation for UpdateServiceAccount {
update_req
.validate()
.map_err(|e| S3Error::with_message(InvalidRequest, e.to_string()))?;
// TODO: is_allowed
let Some(input_cred) = req.credentials else {
return Err(s3_error!(InvalidRequest, "get cred failed"));
};
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key).await?;
if !iam_store
.is_allowed(&Args {
account: &cred.access_key,
groups: &cred.groups,
action: Action::AdminAction(AdminAction::UpdateServiceAccountAdminAction),
bucket: "",
conditions: &get_condition_values(&req.headers, &cred),
is_owner: owner,
object: "",
claims: cred.claims.as_ref().unwrap_or(&HashMap::new()),
deny_only: false,
})
.await
{
return Err(s3_error!(AccessDenied, "access denied"));
}
let sp = {
if let Some(policy) = update_req.new_policy {
let sp = Policy::parse_config(policy.as_bytes()).map_err(|e| {
@@ -278,7 +324,36 @@ impl Operation for InfoServiceAccount {
s3_error!(InternalError, "get service account failed")
})?;
// TODO: is_allowed
let Some(input_cred) = req.credentials else {
return Err(s3_error!(InvalidRequest, "get cred failed"));
};
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key).await?;
if !iam_store
.is_allowed(&Args {
account: &cred.access_key,
groups: &cred.groups,
action: Action::AdminAction(AdminAction::ListServiceAccountsAdminAction),
bucket: "",
conditions: &get_condition_values(&req.headers, &cred),
is_owner: owner,
object: "",
claims: cred.claims.as_ref().unwrap_or(&HashMap::new()),
deny_only: false,
})
.await
{
let user = if cred.parent_user.is_empty() {
&cred.access_key
} else {
&cred.parent_user
};
if user != &svc_account.parent_user {
return Err(s3_error!(AccessDenied, "access denied"));
}
}
let implied_policy = if let Some(policy) = session_policy.as_ref() {
policy.version.is_empty() && policy.statements.is_empty()
@@ -357,27 +432,55 @@ impl Operation for ListServiceAccount {
return Err(s3_error!(InvalidRequest, "get cred failed"));
};
let (cred, _owner) = check_key_valid(&req.headers, &input_cred.access_key).await.map_err(|e| {
debug!("check key failed: {e:?}");
s3_error!(InternalError, "check key failed")
})?;
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key)
.await
.map_err(|e| {
debug!("check key failed: {e:?}");
s3_error!(InternalError, "check key failed")
})?;
let target_account = if let Some(user) = query.user {
if user != input_cred.access_key {
user
} else if cred.parent_user.is_empty() {
input_cred.access_key
} else {
cred.parent_user
// let target_account = if let Some(user) = query.user {
// if user != input_cred.access_key {
// user
// } else if cred.parent_user.is_empty() {
// input_cred.access_key
// } else {
// cred.parent_user
// }
// } else if cred.parent_user.is_empty() {
// input_cred.access_key
// } else {
// cred.parent_user
// };
let Ok(iam_store) = iam::get() else { return Err(s3_error!(InvalidRequest, "iam not init")) };
let target_account = if query.user.as_ref().is_some_and(|v| v != &cred.access_key) {
if !iam_store
.is_allowed(&Args {
account: &cred.access_key,
groups: &cred.groups,
action: Action::AdminAction(AdminAction::UpdateServiceAccountAdminAction),
bucket: "",
conditions: &get_condition_values(&req.headers, &cred),
is_owner: owner,
object: "",
claims: cred.claims.as_ref().unwrap_or(&HashMap::new()),
deny_only: false,
})
.await
{
return Err(s3_error!(AccessDenied, "access denied"));
}
query.user.unwrap_or_default()
} else if cred.parent_user.is_empty() {
input_cred.access_key
cred.access_key
} else {
cred.parent_user
};
let Ok(iam_store) = iam::get() else { return Err(s3_error!(InvalidRequest, "iam not init")) };
let service_accounts = iam_store.list_service_accounts(&target_account).await.map_err(|e| {
debug!("list service account failed: {e:?}");
s3_error!(InternalError, "list service account failed")
@@ -415,10 +518,13 @@ impl Operation for DeleteServiceAccount {
return Err(s3_error!(InvalidRequest, "get cred failed"));
};
let (_cred, _owner) = check_key_valid(&req.headers, &input_cred.access_key).await.map_err(|e| {
debug!("check key failed: {e:?}");
s3_error!(InternalError, "check key failed")
})?;
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key)
.await
.map_err(|e| {
debug!("check key failed: {e:?}");
s3_error!(InternalError, "check key failed")
})?;
let query = {
if let Some(query) = req.uri.query() {
@@ -436,7 +542,7 @@ impl Operation for DeleteServiceAccount {
let Ok(iam_store) = iam::get() else { return Err(s3_error!(InvalidRequest, "iam not init")) };
let _svc_account = match iam_store.get_service_account(&query.access_key).await {
let svc_account = match iam_store.get_service_account(&query.access_key).await {
Ok((res, _)) => Some(res),
Err(err) => {
if is_err_no_such_service_account(&err) {
@@ -447,7 +553,30 @@ impl Operation for DeleteServiceAccount {
}
};
// TODO: is_allowed
if !iam_store
.is_allowed(&Args {
account: &cred.access_key,
groups: &cred.groups,
action: Action::AdminAction(AdminAction::RemoveServiceAccountAdminAction),
bucket: "",
conditions: &get_condition_values(&req.headers, &cred),
is_owner: owner,
object: "",
claims: cred.claims.as_ref().unwrap_or(&HashMap::new()),
deny_only: false,
})
.await
{
let user = if cred.parent_user.is_empty() {
&cred.access_key
} else {
&cred.parent_user
};
if svc_account.is_some_and(|v| &v.parent_user != user) {
return Err(s3_error!(InvalidRequest, "service account not exist"));
}
}
iam_store.delete_service_account(&query.access_key).await.map_err(|e| {
debug!("delete service account failed, e: {:?}", e);
rustfs/src/admin/handlers/sts.rs
+3 -2
View File
@@ -42,12 +42,13 @@ impl Operation for AssumeRoleHandle {
let Some(user) = req.credentials else { return Err(s3_error!(InvalidRequest, "get cred failed")) };
let session_token = get_session_token(&req.headers);
let session_token = get_session_token(&req.uri, &req.headers);
if session_token.is_some() {
return Err(s3_error!(InvalidRequest, "AccessDenied1"));
}
let (cred, _owner) = check_key_valid(&req.headers, &user.access_key).await?;
let (cred, _owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &user.access_key).await?;
// // TODO: 判断权限, 不允许sts访问
if cred.is_temp() || cred.is_service_account() {
rustfs/src/admin/handlers/user.rs
+54 -21
View File
@@ -1,9 +1,13 @@
use std::str::from_utf8;
use std::{collections::HashMap, str::from_utf8};
use http::{HeaderMap, StatusCode};
use iam::get_global_action_cred;
use madmin::{AccountStatus, AddOrUpdateUserReq};
use matchit::Params;
use policy::policy::{
action::{Action, AdminAction},
Args,
};
use s3s::{header::CONTENT_TYPE, s3_error, Body, S3Error, S3ErrorCode, S3Request, S3Response, S3Result};
use serde::Deserialize;
use serde_urlencoded::from_bytes;
@@ -11,7 +15,7 @@ use tracing::warn;
use crate::{
admin::{router::Operation, utils::has_space_be},
auth::check_key_valid,
auth::{check_key_valid, get_condition_values, get_session_token},
};
#[derive(Debug, Deserialize, Default)]
@@ -39,7 +43,8 @@ impl Operation for AddUser {
return Err(s3_error!(InvalidRequest, "get cred failed"));
};
let (cred, _owner) = check_key_valid(&req.headers, &input_cred.access_key).await?;
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key).await?;
let ak = query.access_key.as_deref().unwrap_or_default();
@@ -62,8 +67,6 @@ impl Operation for AddUser {
let args: AddOrUpdateUserReq = serde_json::from_slice(&body)
.map_err(|e| S3Error::with_message(S3ErrorCode::InternalError, format!("unmarshal body err {}", e)))?;
warn!("add user args {:?}", args);
if args.secret_key.is_empty() {
return Err(s3_error!(InvalidArgument, "access key is empty"));
}
@@ -88,13 +91,24 @@ impl Operation for AddUser {
return Err(s3_error!(InvalidArgument, "access key is not utf8"));
}
// let check_deny_only = if ak == cred.access_key {
// true
// } else {
// false
// };
// TODO: is_allowed
let deny_only = ak == cred.access_key;
let conditions = get_condition_values(&req.headers, &cred);
if !iam_store
.is_allowed(&Args {
account: &cred.access_key,
groups: &cred.groups,
action: Action::AdminAction(AdminAction::CreateUserAdminAction),
bucket: "",
conditions: &conditions,
is_owner: owner,
object: "",
claims: cred.claims.as_ref().unwrap_or(&HashMap::new()),
deny_only,
})
.await
{
return Err(s3_error!(AccessDenied, "access denied"));
}
iam_store
.create_user(ak, &args)
@@ -112,8 +126,6 @@ pub struct SetUserStatus {}
#[async_trait::async_trait]
impl Operation for SetUserStatus {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
warn!("handle SetUserStatus");
let query = {
if let Some(query) = req.uri.query() {
let input: AddUserQuery =
@@ -164,8 +176,6 @@ pub struct ListUsers {}
#[async_trait::async_trait]
impl Operation for ListUsers {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
warn!("handle ListUsers");
let query = {
if let Some(query) = req.uri.query() {
let input: BucketQuery =
@@ -213,8 +223,6 @@ pub struct RemoveUser {}
#[async_trait::async_trait]
impl Operation for RemoveUser {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
warn!("handle RemoveUser");
let query = {
if let Some(query) = req.uri.query() {
let input: AddUserQuery =
@@ -246,7 +254,8 @@ impl Operation for RemoveUser {
return Err(s3_error!(InvalidRequest, "get cred failed"));
};
let (cred, _owner) = check_key_valid(&req.headers, &input_cred.access_key).await?;
let (cred, _owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key).await?;
let sys_cred = get_global_action_cred()
.ok_or_else(|| S3Error::with_message(S3ErrorCode::InternalError, "get_global_action_cred failed"))?;
@@ -275,8 +284,6 @@ pub struct GetUserInfo {}
#[async_trait::async_trait]
impl Operation for GetUserInfo {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
warn!("handle GetUserInfo");
let query = {
if let Some(query) = req.uri.query() {
let input: AddUserQuery =
@@ -295,6 +302,32 @@ impl Operation for GetUserInfo {
let Ok(iam_store) = iam::get() else { return Err(s3_error!(InvalidRequest, "iam not init")) };
let Some(input_cred) = req.credentials else {
return Err(s3_error!(InvalidRequest, "get cred failed"));
};
let (cred, owner) =
check_key_valid(get_session_token(&req.uri, &req.headers).unwrap_or_default(), &input_cred.access_key).await?;
let deny_only = ak == cred.access_key;
let conditions = get_condition_values(&req.headers, &cred);
if !iam_store
.is_allowed(&Args {
account: &cred.access_key,
groups: &cred.groups,
action: Action::AdminAction(AdminAction::GetUserAdminAction),
bucket: "",
conditions: &conditions,
is_owner: owner,
object: "",
claims: cred.claims.as_ref().unwrap_or(&HashMap::new()),
deny_only,
})
.await
{
return Err(s3_error!(AccessDenied, "access denied"));
}
let info = iam_store
.get_user_info(ak)
.await
rustfs/src/admin/mod.rs
+7 -6
View File
@@ -6,10 +6,11 @@ pub mod utils;
use common::error::Result;
// use ecstore::global::{is_dist_erasure, is_erasure};
use handlers::{
group, policy, pools,
group, policys, pools,
service_account::{AddServiceAccount, DeleteServiceAccount, InfoServiceAccount, ListServiceAccount, UpdateServiceAccount},
sts, user,
};
use hyper::Method;
use router::{AdminOperation, S3Router};
use rpc::regist_rpc_route;
@@ -231,35 +232,35 @@ fn regist_user_route(r: &mut S3Router<AdminOperation>) -> Result<()> {
r.insert(
Method::GET,
format!("{}{}", ADMIN_PREFIX, "/v3/list-canned-policies").as_str(),
AdminOperation(&policy::ListCannedPolicies {}),
AdminOperation(&policys::ListCannedPolicies {}),
)?;
// info-canned-policy?name=xxx
r.insert(
Method::GET,
format!("{}{}", ADMIN_PREFIX, "/v3/info-canned-policy").as_str(),
AdminOperation(&policy::InfoCannedPolicy {}),
AdminOperation(&policys::InfoCannedPolicy {}),
)?;
// add-canned-policy?name=xxx
r.insert(
Method::PUT,
format!("{}{}", ADMIN_PREFIX, "/v3/add-canned-policy").as_str(),
AdminOperation(&policy::AddCannedPolicy {}),
AdminOperation(&policys::AddCannedPolicy {}),
)?;
// remove-canned-policy?name=xxx
r.insert(
Method::DELETE,
format!("{}{}", ADMIN_PREFIX, "/v3/remove-canned-policy").as_str(),
AdminOperation(&policy::RemoveCannedPolicy {}),
AdminOperation(&policys::RemoveCannedPolicy {}),
)?;
// set-user-or-group-policy?policyName=xxx&userOrGroup=xxx&isGroup=xxx
r.insert(
Method::PUT,
format!("{}{}", ADMIN_PREFIX, "/v3/set-user-or-group-policy").as_str(),
AdminOperation(&policy::SetPolicyForUserOrGroup {}),
AdminOperation(&policys::SetPolicyForUserOrGroup {}),
)?;
Ok(())
rustfs/src/admin/router.rs
+4 -2
View File
@@ -67,14 +67,16 @@ where
uri.path().starts_with(ADMIN_PREFIX) || uri.path().starts_with(RPC_PREFIX)
}
async fn call(&self, req: S3Request<Body>) -> S3Result<S3Response<(StatusCode, Body)>> {
async fn call(&self, req: S3Request<Body>) -> S3Result<S3Response<Body>> {
let uri = format!("{}|{}", &req.method, req.uri.path());
// warn!("get uri {}", &uri);
if let Ok(mat) = self.router.at(&uri) {
let op: &T = mat.value;
return op.call(req, mat.params).await;
let mut resp = op.call(req, mat.params).await?;
resp.status = Some(resp.output.0);
return Ok(resp.map_output(|x| x.1));
}
return Err(s3_error!(NotImplemented));
rustfs/src/auth.rs
+25 -10
View File
@@ -1,6 +1,7 @@
use std::collections::HashMap;
use http::HeaderMap;
use http::Uri;
use iam::error::Error as IamError;
use iam::get_global_action_cred;
use iam::sys::SESSION_POLICY_NAME;
@@ -48,7 +49,7 @@ impl S3Auth for IAMAuth {
}
// check_key_valid checks the key is valid or not. return the user's credentials and if the user is the owner.
pub async fn check_key_valid(header: &HeaderMap, access_key: &str) -> S3Result<(auth::Credentials, bool)> {
pub async fn check_key_valid(session_token: &str, access_key: &str) -> S3Result<(auth::Credentials, bool)> {
let Some(mut cred) = get_global_action_cred() else {
return Err(S3Error::with_message(
S3ErrorCode::InternalError,
@@ -88,7 +89,7 @@ pub async fn check_key_valid(header: &HeaderMap, access_key: &str) -> S3Result<(
cred = u.credentials;
}
let claims = check_claims_from_token(header, &cred)
let claims = check_claims_from_token(session_token, &cred)
.map_err(|e| S3Error::with_message(S3ErrorCode::InternalError, format!("check claims failed {}", e)))?;
cred.claims = if !claims.is_empty() { Some(claims) } else { None };
@@ -105,23 +106,21 @@ pub async fn check_key_valid(header: &HeaderMap, access_key: &str) -> S3Result<(
Ok((cred, owner))
}
pub fn check_claims_from_token(header: &HeaderMap, cred: &auth::Credentials) -> S3Result<HashMap<String, Value>> {
let token = get_session_token(header).unwrap_or_default();
pub fn check_claims_from_token(token: &str, cred: &auth::Credentials) -> S3Result<HashMap<String, Value>> {
if !token.is_empty() && cred.access_key.is_empty() {
return Err(s3_error!(InvalidRequest, "no access key"));
}
if token.is_empty() && cred.is_temp() && !cred.is_service_account() {
return Err(s3_error!(InvalidRequest, "invalid token"));
return Err(s3_error!(InvalidRequest, "invalid token1"));
}
if !token.is_empty() && !cred.is_temp() {
return Err(s3_error!(InvalidRequest, "invalid token"));
return Err(s3_error!(InvalidRequest, "invalid token2"));
}
if !cred.is_service_account() && cred.is_temp() && token != cred.session_token {
return Err(s3_error!(InvalidRequest, "invalid token"));
return Err(s3_error!(InvalidRequest, "invalid token3"));
}
if cred.is_temp() && cred.is_expired() {
@@ -149,8 +148,10 @@ pub fn check_claims_from_token(header: &HeaderMap, cred: &auth::Credentials) ->
Ok(HashMap::new())
}
pub fn get_session_token(hds: &HeaderMap) -> Option<&str> {
hds.get("x-amz-security-token").map(|v| v.to_str().unwrap_or_default())
pub fn get_session_token<'a>(uri: &'a Uri, hds: &'a HeaderMap) -> Option<&'a str> {
hds.get("x-amz-security-token")
.map(|v| v.to_str().unwrap_or_default())
.or_else(|| get_query_param(uri.query().unwrap_or_default(), "x-amz-security-token"))
}
pub fn get_condition_values(header: &HeaderMap, cred: &auth::Credentials) -> HashMap<String, Vec<String>> {
@@ -295,3 +296,17 @@ pub fn get_condition_values(header: &HeaderMap, cred: &auth::Credentials) -> Has
args
}
pub fn get_query_param<'a>(query: &'a str, param_name: &str) -> Option<&'a str> {
let param_name = param_name.to_lowercase();
for pair in query.split('&') {
let mut parts = pair.splitn(2, '=');
if let (Some(key), Some(value)) = (parts.next(), parts.next()) {
if key.to_lowercase() == param_name {
return Some(value);
}
}
}
None
}
rustfs/src/config/mod.rs
+59 -6
View File
@@ -1,11 +1,39 @@
use clap::Parser;
use const_str::concat;
use ecstore::global::DEFAULT_PORT;
use std::string::ToString;
shadow_rs::shadow!(build);
/// Default Access Key
/// Default value: rustfsadmin
/// Environment variable: RUSTFS_ACCESS_KEY
/// Command line argument: --access-key
/// Example: RUSTFS_ACCESS_KEY=rustfsadmin
/// Example: --access-key rustfsadmin
pub const DEFAULT_ACCESS_KEY: &str = "rustfsadmin";
/// Default Secret Key
/// Default value: rustfsadmin
/// Environment variable: RUSTFS_SECRET_KEY
/// Command line argument: --secret-key
/// Example: RUSTFS_SECRET_KEY=rustfsadmin
/// Example: --secret-key rustfsadmin
pub const DEFAULT_SECRET_KEY: &str = "rustfsadmin";
/// Default configuration file for observability
/// Default value: config/obs.toml
/// Environment variable: RUSTFS_OBS_CONFIG
/// Command line argument: --obs-config
/// Example: RUSTFS_OBS_CONFIG=config/obs.toml
/// Example: --obs-config config/obs.toml
/// Example: --obs-config /etc/rustfs/obs.toml
pub const DEFAULT_OBS_CONFIG: &str = "config/obs.toml";
/// Default TLS key for rustfs
/// This is the default key for TLS.
pub(crate) const RUSTFS_TLS_KEY: &str = "rustfs_private.key";
/// Default TLS cert for rustfs
/// This is the default cert for TLS.
pub(crate) const RUSTFS_TLS_CERT: &str = "rustfs_public.crt";
#[allow(clippy::const_is_empty)]
const SHORT_VERSION: &str = {
@@ -31,7 +59,7 @@ const LONG_VERSION: &str = concat!(
concat!("git status :\n", build::GIT_STATUS_FILE),
);
#[derive(Debug, Parser)]
#[derive(Debug, Parser, Clone)]
#[command(version = SHORT_VERSION, long_version = LONG_VERSION)]
pub struct Opt {
/// DIR points to a directory on a filesystem.
@@ -42,6 +70,7 @@ pub struct Opt {
#[arg(long, default_value_t = format!("0.0.0.0:{}", DEFAULT_PORT), env = "RUSTFS_ADDRESS")]
pub address: String,
/// Domain name used for virtual-hosted-style requests.
#[arg(long, env = "RUSTFS_SERVER_DOMAINS")]
pub server_domains: Vec<String>,
@@ -53,13 +82,37 @@ pub struct Opt {
#[arg(long, default_value_t = DEFAULT_SECRET_KEY.to_string(), env = "RUSTFS_SECRET_KEY")]
pub secret_key: String,
/// Domain name used for virtual-hosted-style requests.
#[arg(long, env = "RUSTFS_DOMAIN_NAME")]
pub domain_name: Option<String>,
#[arg(long, default_value_t = false, env = "RUSTFS_CONSOLE_ENABLE")]
pub console_enable: bool,
#[arg(long, default_value_t = format!("127.0.0.1:{}", 9002), env = "RUSTFS_CONSOLE_ADDRESS")]
pub console_address: String,
/// rustfs endpoint for console
#[arg(long, env = "RUSTFS_CONSOLE_FS_ENDPOINT")]
pub console_fs_endpoint: Option<String>,
/// Observability configuration file
/// Default value: config/obs.toml
#[arg(long, default_value_t = DEFAULT_OBS_CONFIG.to_string(), env = "RUSTFS_OBS_CONFIG")]
pub obs_config: String,
/// tls path for rustfs api and console.
#[arg(long, env = "RUSTFS_TLS_PATH")]
pub tls_path: Option<String>,
#[arg(long, env = "RUSTFS_LICENSE")]
pub license: Option<String>,
}
// lazy_static::lazy_static! {
// pub(crate) static ref OPT: OnceLock<Opt> = OnceLock::new();
// }
// pub fn init_config(opt: Opt) {
// OPT.set(opt).expect("Failed to set global config");
// }
// pub fn get_config() -> &'static Opt {
// OPT.get().expect("Global config not initialized")
// }
rustfs/src/console.rs
+178 -31
View File
@@ -1,19 +1,25 @@
use crate::config::{RUSTFS_TLS_CERT, RUSTFS_TLS_KEY};
use crate::license::get_license;
use axum::{
body::Body,
extract::Host,
http::{Response, StatusCode},
response::IntoResponse,
routing::get,
Router,
};
use axum_extra::extract::Host;
use axum_server::tls_rustls::RustlsConfig;
use http::Uri;
use mime_guess::from_path;
use rust_embed::RustEmbed;
use serde::Serialize;
use shadow_rs::shadow;
use std::net::{Ipv4Addr, SocketAddr, ToSocketAddrs};
use std::net::{Ipv4Addr, SocketAddr};
use std::sync::OnceLock;
use tracing::info;
use std::time::Duration;
use tokio::signal;
use tracing::{debug, error, info, instrument};
shadow!(build);
@@ -153,31 +159,85 @@ pub(crate) fn init_console_cfg(local_ip: Ipv4Addr, port: u16) {
// host.parse::<SocketAddr>().is_ok() || host.parse::<IpAddr>().is_ok()
// }
async fn license_handler() -> impl IntoResponse {
let license = get_license().unwrap_or_default();
Response::builder()
.header("content-type", "application/json")
.status(StatusCode::OK)
.body(Body::from(serde_json::to_string(&license).unwrap_or_default()))
.unwrap()
}
fn _is_private_ip(ip: std::net::IpAddr) -> bool {
match ip {
std::net::IpAddr::V4(ip) => {
let octets = ip.octets();
// 10.0.0.0/8
octets[0] == 10 ||
// 172.16.0.0/12
(octets[0] == 172 && (octets[1] >= 16 && octets[1] <= 31)) ||
// 192.168.0.0/16
(octets[0] == 192 && octets[1] == 168)
}
std::net::IpAddr::V6(_) => false,
}
}
#[allow(clippy::const_is_empty)]
async fn config_handler(Host(host): Host) -> impl IntoResponse {
let host_with_port = if host.contains(':') { host } else { format!("{}:80", host) };
#[instrument(fields(host))]
async fn config_handler(uri: Uri, Host(host): Host) -> impl IntoResponse {
let scheme = uri.scheme().map(|s| s.as_str()).unwrap_or("http");
let is_addr = host_with_port
.to_socket_addrs()
.map(|addrs| {
addrs.into_iter().find(|v| {
if let SocketAddr::V4(ipv4) = v {
!ipv4.ip().is_private() && !ipv4.ip().is_loopback() && !ipv4.ip().is_unspecified()
} else {
false
}
})
})
.unwrap_or_default();
// 从 uri 中获取 host，如果没有则使用 Host extractor 的值
let host = uri.host().unwrap_or(host.as_str());
let host = if host.contains(':') {
let (host, _) = host.split_once(':').unwrap_or((host, "80"));
host
} else {
host
};
// 将当前配置复制一份
let mut cfg = CONSOLE_CONFIG.get().unwrap().clone();
let url = if let Some(addr) = is_addr {
format!("http://{}:{}", addr.ip(), cfg.port)
} else {
let (host, _) = host_with_port.split_once(':').unwrap_or_default();
format!("http://{}:{}", host, cfg.port)
};
let url = format!("{}://{}:{}", scheme, host, cfg.port);
// // 如果指定入口, 直接使用
// let url = if let Some(endpoint) = &config::get_config().console_fs_endpoint {
// debug!("axum Using rustfs endpoint address: {}", endpoint);
// endpoint.clone()
// } else {
// let host_with_port = if host.contains(':') {
// host.clone()
// } else {
// format!("{}:80", host)
// };
// // 尝试解析为 socket address，但不强制要求一定要是 IP 地址
// let socket_addr = host_with_port.to_socket_addrs().ok().and_then(|mut addrs| addrs.next());
// debug!("axum Using host with port: {}, Socket address: {:?}", host_with_port, socket_addr);
// match socket_addr {
// Some(addr) if addr.ip().is_ipv4() => {
// let ipv4 = addr.ip().to_string();
// // 如果是私有 IP、环回地址或未指定地址，保留原始域名
// if is_private_ip(addr.ip()) || addr.ip().is_loopback() || addr.ip().is_unspecified() {
// let (host, _) = host_with_port.split_once(':').unwrap_or((&host, "80"));
// debug!("axum Using private IPv4 address: {}", host);
// format!("http://{}:{}", host, cfg.port)
// } else {
// debug!("axum Using public IPv4 address");
// format!("http://{}:{}", ipv4, cfg.port)
// }
// }
// _ => {
// // 如果不是有效的 IPv4 地址，保留原始域名
// let (host, _) = host_with_port.split_once(':').unwrap_or((&host, "80"));
// debug!("axum Using domain address: {}", host);
// format!("http://{}:{}", host, cfg.port)
// }
// }
// };
cfg.api.base_url = format!("{}{}", url, RUSTFS_ADMIN_PREFIX);
cfg.s3.endpoint = url;
@@ -189,18 +249,105 @@ async fn config_handler(Host(host): Host) -> impl IntoResponse {
.unwrap()
}
pub async fn start_static_file_server(addrs: &str, local_ip: Ipv4Addr, access_key: &str, secret_key: &str) {
// 创建路由
pub async fn start_static_file_server(
addrs: &str,
local_ip: Ipv4Addr,
access_key: &str,
secret_key: &str,
tls_path: Option<String>,
) {
// Create a route
let app = Router::new()
.route("/license", get(license_handler))
.route("/config.json", get(config_handler))
.nest_service("/", get(static_handler));
let listener = tokio::net::TcpListener::bind(addrs).await.unwrap();
let local_addr = listener.local_addr().unwrap();
.fallback_service(get(static_handler));
let local_addr: SocketAddr = addrs.parse().expect("Failed to parse socket address");
info!("WebUI: http://{}:{} http://127.0.0.1:{}", local_ip, local_addr.port(), local_addr.port());
info!(" RootUser: {}", access_key);
info!(" RootPass: {}", secret_key);
axum::serve(listener, app).await.unwrap();
// Check and start the HTTPS/HTTP server
match start_server(addrs, local_addr, tls_path, app.clone()).await {
Ok(_) => info!("Server shutdown gracefully"),
Err(e) => error!("Server error: {}", e),
}
}
async fn start_server(addrs: &str, local_addr: SocketAddr, tls_path: Option<String>, app: Router) -> std::io::Result<()> {
let tls_path = tls_path.unwrap_or_default();
let key_path = format!("{}/{}", tls_path, RUSTFS_TLS_KEY);
let cert_path = format!("{}/{}", tls_path, RUSTFS_TLS_CERT);
let has_tls_certs = tokio::try_join!(tokio::fs::metadata(&key_path), tokio::fs::metadata(&cert_path)).is_ok();
debug!("Console TLS certs: {:?}", has_tls_certs);
if has_tls_certs {
debug!("Found TLS certificates, starting with HTTPS");
match tokio::try_join!(tokio::fs::read(&key_path), tokio::fs::read(&cert_path)) {
Ok((key_data, cert_data)) => {
match RustlsConfig::from_pem(cert_data, key_data).await {
Ok(config) => {
let handle = axum_server::Handle::new();
// create a signal off listening task
let handle_clone = handle.clone();
tokio::spawn(async move {
shutdown_signal().await;
handle_clone.graceful_shutdown(Some(Duration::from_secs(10)));
});
info!("Starting HTTPS server...");
axum_server::bind_rustls(local_addr, config)
.handle(handle.clone())
.serve(app.into_make_service())
.await
.map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))?;
Ok(())
}
Err(e) => {
error!("Failed to create TLS config: {}", e);
start_http_server(addrs, app).await
}
}
}
Err(e) => {
error!("Failed to read TLS certificates: {}", e);
start_http_server(addrs, app).await
}
}
} else {
debug!("TLS certificates not found at {} and {}", key_path, cert_path);
start_http_server(addrs, app).await
}
}
async fn start_http_server(addrs: &str, app: Router) -> std::io::Result<()> {
debug!("Starting HTTP server...");
let listener = tokio::net::TcpListener::bind(addrs).await?;
axum::serve(listener, app)
.with_graceful_shutdown(shutdown_signal())
.await
.map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))
}
async fn shutdown_signal() {
let ctrl_c = async {
signal::ctrl_c().await.expect("failed to install Ctrl+C handler");
};
#[cfg(unix)]
let terminate = async {
signal::unix::signal(signal::unix::SignalKind::terminate())
.expect("failed to install signal handler")
.recv()
.await;
};
#[cfg(not(unix))]
let terminate = std::future::pending::<()>();
tokio::select! {
_ = ctrl_c => {
info!("shutdown_signal ctrl_c")
},
_ = terminate => {
info!("shutdown_signal terminate")
},
}
}
rustfs/src/license.rs
+62
View File
@@ -0,0 +1,62 @@
use appauth::token::Token;
use common::error::{Error, Result};
use std::sync::OnceLock;
use std::time::SystemTime;
use std::time::UNIX_EPOCH;
use tracing::error;
use tracing::info;
lazy_static::lazy_static! {
static ref LICENSE: OnceLock<Token> = OnceLock::new();
}
pub fn init_license(license: Option<String>) {
if license.is_none() {
error!("License is None");
return;
}
let license = license.unwrap();
let token = appauth::token::parse_license(&license).unwrap_or_default();
LICENSE.set(token).unwrap_or_else(|_| {
error!("Failed to set license");
});
}
pub fn get_license() -> Option<Token> {
LICENSE.get().cloned()
}
#[allow(unreachable_code)]
pub fn license_check() -> Result<()> {
return Ok(());
let inval_license = LICENSE.get().map(|token| {
if token.expired < SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs() {
error!("License expired");
return Err(Error::from_string("Incorrect license, please contact RustFS.".to_string()));
}
info!("License is valid ! expired at {}", token.expired);
Ok(())
});
// let inval_license = config::get_config().license.as_ref().map(|license| {
// if license.is_empty() {
// error!("License is empty");
// return Err(Error::from_string("Incorrect license, please contact RustFS.".to_string()));
// }
// let token = appauth::token::parse_license(license)?;
// if token.expired < SystemTime::now().duration_since(UNIX_EPOCH)?.as_secs() {
// error!("License expired");
// return Err(Error::from_string("Incorrect license, please contact RustFS.".to_string()));
// }
// info!("License is valid ! expired at {}", token.expired);
// Ok(())
// });
if inval_license.is_none() || inval_license.is_some_and(|v| v.is_err()) {
return Err(Error::from_string("Incorrect license, please contact RustFS.".to_string()));
}
Ok(())
}

Some files were not shown because too many files have changed in this diff Show More