mirror of
https://github.com/rustfs/rustfs.git
synced 2026-07-26 08:18:18 +00:00
docs(scanner): document runtime scanner controls (#3339)
* docs(scanner): document runtime scanner controls * docs(scanner): split English and Chinese README --------- Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com> Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com>
This commit is contained in:
@@ -210,6 +210,10 @@ make help-docker # Show all Docker-related commands
|
||||
|
||||
Follow the instructions in the [Helm Chart README](https://charts.rustfs.com/) to install RustFS on a Kubernetes cluster.
|
||||
|
||||
For scanner pacing, cycle budgets, bitrot cadence, lifecycle transition status,
|
||||
and single-node single-disk idle CPU tuning, see
|
||||
[Scanner Runtime Controls](docs/operations/scanner-runtime-controls.md).
|
||||
|
||||
### 5\. Nix Flake (Option 5)
|
||||
|
||||
If you have [Nix with flakes enabled](https://nixos.wiki/wiki/Flakes#Enable_flakes):
|
||||
|
||||
+10
-18
@@ -1,36 +1,28 @@
|
||||
# RustFS Scanner
|
||||
|
||||
RustFS Scanner 提供了数据完整性检查、健康监控和存储分析等扫描功能。
|
||||
RustFS Scanner is the background maintenance scan loop. It handles usage
|
||||
accounting, lifecycle expiry and transition admission, bucket replication repair
|
||||
admission, scanner-driven heal/bitrot checks, and namespace alerts.
|
||||
|
||||
## 功能特性
|
||||
For operator-facing runtime controls, status fields, and tuning workflows, see
|
||||
[Scanner Runtime Controls](../../docs/operations/scanner-runtime-controls.md).
|
||||
|
||||
- 数据完整性扫描
|
||||
- 健康监控
|
||||
- 存储分析
|
||||
- 可扩展的扫描框架
|
||||
Chinese documentation is available in [README.zh-CN.md](README.zh-CN.md).
|
||||
|
||||
## 使用示例
|
||||
## Development
|
||||
|
||||
```rust
|
||||
use rustfs_scanner::ScannerError;
|
||||
|
||||
// TODO: 添加使用示例
|
||||
```
|
||||
|
||||
## 开发
|
||||
|
||||
### 构建
|
||||
### Build
|
||||
|
||||
```bash
|
||||
cargo build --package rustfs-scanner
|
||||
```
|
||||
|
||||
### 测试
|
||||
### Test
|
||||
|
||||
```bash
|
||||
cargo test --package rustfs-scanner
|
||||
```
|
||||
|
||||
## 许可证
|
||||
## License
|
||||
|
||||
Apache License 2.0
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# RustFS Scanner
|
||||
|
||||
RustFS Scanner 是后台维护扫描循环,负责用量统计、生命周期过期和分层转移准入、桶复制修复准入、scanner 发起的 heal/bitrot 检查,以及命名空间告警。
|
||||
|
||||
面向运维人员的运行时控制项、状态字段和调参流程,请参考
|
||||
[Scanner Runtime Controls](../../docs/operations/scanner-runtime-controls.md)。
|
||||
|
||||
英文文档请参考 [README.md](README.md)。
|
||||
|
||||
## 开发
|
||||
|
||||
### 构建
|
||||
|
||||
```bash
|
||||
cargo build --package rustfs-scanner
|
||||
```
|
||||
|
||||
### 测试
|
||||
|
||||
```bash
|
||||
cargo test --package rustfs-scanner
|
||||
```
|
||||
|
||||
## 许可证
|
||||
|
||||
Apache License 2.0
|
||||
@@ -0,0 +1,219 @@
|
||||
# Scanner Runtime Controls
|
||||
|
||||
This document describes the runtime controls and status fields for the RustFS
|
||||
data scanner. It is written for operators who need to reduce scanner pressure,
|
||||
diagnose slow scan progress, or confirm that background lifecycle, replication,
|
||||
heal, bitrot, and usage work is still moving.
|
||||
|
||||
## What the scanner does
|
||||
|
||||
The scanner is the background maintenance loop that walks stored objects and
|
||||
feeds several subsystems:
|
||||
|
||||
- usage accounting and data usage cache updates;
|
||||
- lifecycle expiry and transition admission;
|
||||
- bucket replication repair admission;
|
||||
- scanner-originated heal and bitrot checks;
|
||||
- namespace alerts for excessive versions, retained version size, and folder
|
||||
fan-out.
|
||||
|
||||
Slowing the scanner can reduce idle CPU and disk pressure, but it also delays
|
||||
the maintenance work above. Prefer using the status fields below before changing
|
||||
cycle or pacing values.
|
||||
|
||||
## Configuration Sources
|
||||
|
||||
Scanner runtime config is resolved in this order:
|
||||
|
||||
1. Environment variables.
|
||||
2. Persisted admin config for the `scanner` subsystem.
|
||||
3. Built-in defaults or speed preset-derived values.
|
||||
|
||||
Bitrot cycle resolution is slightly different because the canonical persistent
|
||||
key belongs to the `heal` subsystem:
|
||||
|
||||
1. `RUSTFS_SCANNER_BITROT_CYCLE_SECS`.
|
||||
2. `heal.bitrot_cycle`.
|
||||
3. Legacy compatibility key `scanner.bitrot_cycle`.
|
||||
4. Built-in default.
|
||||
|
||||
The `/v3/scanner/status` response reports each effective runtime value with a
|
||||
`source` of `env`, `config`, `scanner_compat_config`, or `default`.
|
||||
|
||||
## Runtime Controls
|
||||
|
||||
| Persistent key | Environment variable | Unit | Default | Effect |
|
||||
|---|---|---:|---:|---|
|
||||
| `scanner.speed` | `RUSTFS_SCANNER_SPEED` | preset | `default` | Selects the base pacing preset: `fastest`, `fast`, `default`, `slow`, or `slowest`. |
|
||||
| `scanner.delay` | `RUSTFS_SCANNER_DELAY` | factor | preset-derived | Overrides the sleep multiplier. Valid range is `0` through `10000`. |
|
||||
| `scanner.max_wait` | `RUSTFS_SCANNER_MAX_WAIT_SECS` | seconds | preset-derived | Caps one scanner sleep. |
|
||||
| `scanner.cycle` | `RUSTFS_SCANNER_CYCLE` | seconds | preset-derived | Sets the interval between scanner cycles. |
|
||||
| `scanner.start_delay` | `RUSTFS_SCANNER_START_DELAY_SECS` | seconds | unset | Sets startup delay and, for compatibility, the cycle interval when `scanner.cycle` is unset. |
|
||||
| `scanner.cycle_max_duration` | `RUSTFS_SCANNER_CYCLE_MAX_DURATION_SECS` | seconds | `0` | Caps one cycle's runtime. `0` disables this budget. |
|
||||
| `scanner.cycle_max_objects` | `RUSTFS_SCANNER_CYCLE_MAX_OBJECTS` | objects | `0` | Caps objects processed by one cycle. `0` disables this budget. |
|
||||
| `scanner.cycle_max_directories` | `RUSTFS_SCANNER_CYCLE_MAX_DIRECTORIES` | directories | `0` | Caps directories entered by one cycle. `0` disables this budget. |
|
||||
| `heal.bitrot_cycle` | `RUSTFS_SCANNER_BITROT_CYCLE_SECS` | seconds | `2592000` | Controls periodic deep bitrot scans. `false`, `off`, `no`, or `disabled` disables periodic deep scans; `0`, `true`, `on`, or `yes` runs deep mode every scanner cycle. |
|
||||
| `scanner.idle_mode` | `RUSTFS_SCANNER_IDLE_MODE` | boolean | `true` | Enables scanner sleeps and cooperative throttling. |
|
||||
| `scanner.cache_save_timeout` | `RUSTFS_SCANNER_CACHE_SAVE_TIMEOUT_SECS` | seconds | `30` | Timeout for saving scanner cache; runtime enforces a minimum of `1`. |
|
||||
| `scanner.max_concurrent_set_scans` | `RUSTFS_SCANNER_MAX_CONCURRENT_SET_SCANS` | count | `0` | Caps concurrent set-level scanner tasks. `0` keeps topology-derived concurrency. |
|
||||
| `scanner.max_concurrent_disk_scans` | `RUSTFS_SCANNER_MAX_CONCURRENT_DISK_SCANS` | count | `0` | Caps concurrent disk bucket walks per set. `0` keeps disk-count-derived concurrency. |
|
||||
| `scanner.yield_every_n_objects` | `RUSTFS_SCANNER_YIELD_EVERY_N_OBJECTS` | objects | `128` | Controls how often object loops yield to the async runtime. `0` disables this extra yield. |
|
||||
| `scanner.alert_excess_versions` | `RUSTFS_SCANNER_ALERT_EXCESS_VERSIONS` | versions | `100` | Version count threshold for scanner alerts. |
|
||||
| `scanner.alert_excess_version_size` | `RUSTFS_SCANNER_ALERT_EXCESS_VERSION_SIZE` | bytes | `1099511627776` | Retained version byte threshold for scanner alerts. |
|
||||
| `scanner.alert_excess_folders` | `RUSTFS_SCANNER_ALERT_EXCESS_FOLDERS` | folders | `65538` | Direct subfolder threshold for scanner alerts. |
|
||||
|
||||
The `fastest`, `fast`, `default`, `slow`, and `slowest` presets set the base
|
||||
sleep multiplier, maximum wait, and cycle interval. Use `scanner.delay`,
|
||||
`scanner.max_wait`, and `scanner.cycle` when the preset is close but one axis
|
||||
needs a precise override.
|
||||
|
||||
## Status Endpoint
|
||||
|
||||
The scanner status route is:
|
||||
|
||||
```text
|
||||
GET /v3/scanner/status
|
||||
```
|
||||
|
||||
The request must be authenticated with an admin identity that has
|
||||
`ServerInfoAdminAction`. The JSON response has two top-level objects:
|
||||
|
||||
- `runtime_config`: the effective runtime controls and their value sources.
|
||||
- `metrics`: scanner work, pressure, checkpoint, lifecycle, replication, heal,
|
||||
bitrot, and alert counters.
|
||||
|
||||
Example fields to inspect:
|
||||
|
||||
```text
|
||||
runtime_config.speed.value
|
||||
runtime_config.delay.value
|
||||
runtime_config.max_wait_seconds.value
|
||||
runtime_config.cycle_interval_seconds.value
|
||||
runtime_config.bitrot_cycle_seconds.value
|
||||
metrics.pacing_pressure.primary_pressure
|
||||
metrics.pacing_pressure.last_cycle_budget_limited
|
||||
metrics.lifecycle_transition.current_queued
|
||||
metrics.lifecycle_transition.scanner_missed
|
||||
metrics.source_work
|
||||
metrics.scan_checkpoint
|
||||
```
|
||||
|
||||
## Reading Pacing Pressure
|
||||
|
||||
`metrics.pacing_pressure.primary_pressure` summarizes the highest-priority
|
||||
scanner pressure signal:
|
||||
|
||||
| Value | Meaning | Usual response |
|
||||
|---|---|---|
|
||||
| `queued_scans` | Set or disk scan queues are backing up. | Lower scanner concurrency or increase pacing delay if user traffic is affected. |
|
||||
| `cycle_budget` | The last cycle stopped because a runtime/object/directory budget was reached. | Check `last_cycle_partial_reason` and `last_cycle_partial_source`; increase the specific budget if scans need to finish sooner. |
|
||||
| `throttle_pause` | Scanner sleeps or cooperative yields were observed. | Expected when `idle_mode` is enabled; inspect pause ratios before tuning. |
|
||||
| `active_scans` | Scanner work is active but not currently queued or budget-limited. | Usually healthy; correlate with CPU/disk metrics. |
|
||||
| `none` | No current scanner pressure was observed. | No scanner pacing action needed. |
|
||||
|
||||
The ratio fields are fractions of the last cycle duration:
|
||||
|
||||
- `last_cycle_throttle_sleep_ratio`
|
||||
- `last_cycle_yield_ratio`
|
||||
- `last_cycle_total_pause_ratio`
|
||||
|
||||
If CPU is high but pause ratios are already high, increasing `scanner.delay` or
|
||||
`scanner.max_wait` may have limited value. Check active paths, source work, and
|
||||
disk activity before changing the cycle interval.
|
||||
|
||||
## Reading Source Work
|
||||
|
||||
`metrics.source_work`, `metrics.current_cycle_source_work`, and
|
||||
`metrics.last_cycle_source_work` group scanner work by source:
|
||||
|
||||
- `usage`
|
||||
- `lifecycle`
|
||||
- `bucket_replication`
|
||||
- `site_replication`
|
||||
- `heal`
|
||||
- `bitrot`
|
||||
- `alerts`
|
||||
|
||||
Each source has `checked`, `queued`, `executed`, `failed`, `skipped`, and
|
||||
`missed` counters. `missed` means the scanner found work but could not admit it
|
||||
to the downstream queue. `skipped` means the work was intentionally merged or
|
||||
deduplicated.
|
||||
|
||||
Use these counters to decide whether scan progress is limited by scanner pacing
|
||||
or by a downstream subsystem such as lifecycle transition, replication repair,
|
||||
or heal admission.
|
||||
|
||||
## Reading Lifecycle Transition Status
|
||||
|
||||
`metrics.lifecycle_transition` focuses on scanner-driven lifecycle transition
|
||||
work:
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `current_queue_capacity` | Current transition queue capacity. |
|
||||
| `current_queued` | Transition tasks currently queued. |
|
||||
| `current_active` | Transition tasks currently being processed. |
|
||||
| `current_workers` | Transition worker count. |
|
||||
| `queue_full` | Queue-full observations in the transition state. |
|
||||
| `queue_send_timeout` | Send timeouts for transition queue admission. |
|
||||
| `compensation_scheduled` | Buckets scheduled for transition compensation. |
|
||||
| `compensation_running` | Transition compensation tasks currently running. |
|
||||
| `scanner_queued` | Scanner transition tasks admitted to the queue. |
|
||||
| `scanner_missed` | Scanner transition tasks that could not be admitted. |
|
||||
| `completed` | Transition worker completions. |
|
||||
| `failed` | Transition worker failures. |
|
||||
|
||||
When `scanner_missed` or `queue_full` rises, scanner lifecycle work is finding
|
||||
transition candidates faster than the transition queue can accept them. That is
|
||||
a downstream transition pressure signal, not just a scanner walk pressure signal.
|
||||
|
||||
## Tuning Workflow
|
||||
|
||||
For symptoms where a mostly idle single-node, single-disk deployment has
|
||||
sustained CPU usage while the scanner is enabled:
|
||||
|
||||
1. Read `/v3/scanner/status`.
|
||||
2. Check `metrics.pacing_pressure.primary_pressure`.
|
||||
3. Check `runtime_config.delay`, `runtime_config.max_wait_seconds`, and
|
||||
`runtime_config.cycle_interval_seconds` to confirm the active values and
|
||||
their sources.
|
||||
4. Check `metrics.current_cycle_objects_scanned`,
|
||||
`metrics.current_cycle_directories_scanned`, and active paths to confirm the
|
||||
scanner is the active work.
|
||||
5. If `primary_pressure` is `throttle_pause` and pause ratios are low, raise
|
||||
`scanner.delay` first.
|
||||
6. If individual sleeps are too short, raise `scanner.max_wait`.
|
||||
7. If each scan cycle finishes but starts too often, raise `scanner.cycle`.
|
||||
8. If scans must be broken into bounded chunks, set one of the cycle budgets:
|
||||
`scanner.cycle_max_duration`, `scanner.cycle_max_objects`, or
|
||||
`scanner.cycle_max_directories`.
|
||||
9. Recheck `pacing_pressure`, source work, and lifecycle transition status after
|
||||
one or more scanner cycles.
|
||||
|
||||
Do not rely only on a longer cycle interval if lifecycle, replication, heal, or
|
||||
bitrot work must keep moving. Use source work and transition status to confirm
|
||||
that background maintenance is still making progress.
|
||||
|
||||
## Helm
|
||||
|
||||
The Helm chart exposes the scanner environment variables under
|
||||
`config.rustfs.scanner`. Example:
|
||||
|
||||
```yaml
|
||||
config:
|
||||
rustfs:
|
||||
scanner:
|
||||
speed: "slow"
|
||||
delay: "30"
|
||||
max_wait_secs: "15"
|
||||
cycle_secs: "3600"
|
||||
cycle_max_duration_secs: "1800"
|
||||
cycle_max_objects: "1000000"
|
||||
cycle_max_directories: "100000"
|
||||
idle_mode: "true"
|
||||
yield_every_n_objects: "128"
|
||||
bitrot_cycle_secs: "2592000"
|
||||
```
|
||||
|
||||
Use `extraEnv` for experimental or unrelated environment variables that are not
|
||||
represented by chart values.
|
||||
@@ -34,9 +34,22 @@ RustFS helm chart supports **standalone and distributed mode**. For standalone m
|
||||
| config.rustfs.metrics.enabled | bool | `false` | Toggle metrics export. |
|
||||
| config.rustfs.metrics.endpoint | string | `""` | Dedicated metrics endpoint. |
|
||||
| config.rustfs.scanner.speed | string | `""` | Scanner speed preset: `fastest`, `fast`, `default`, `slow`, `slowest`. |
|
||||
| config.rustfs.scanner.delay | string | `""` | Override scanner sleep multiplier with `RUSTFS_SCANNER_DELAY` (`0` through `10000`). |
|
||||
| config.rustfs.scanner.max_wait_secs | string | `""` | Override maximum scanner sleep in seconds with `RUSTFS_SCANNER_MAX_WAIT_SECS`. |
|
||||
| config.rustfs.scanner.cycle_secs | string | `""` | Override scanner cycle interval in seconds with `RUSTFS_SCANNER_CYCLE`. |
|
||||
| config.rustfs.scanner.start_delay_secs | string | `""` | Override scanner cycle interval in seconds with `RUSTFS_SCANNER_START_DELAY_SECS`. |
|
||||
| config.rustfs.scanner.cycle_max_duration_secs | string | `""` | Cap one scanner cycle's runtime in seconds with `RUSTFS_SCANNER_CYCLE_MAX_DURATION_SECS` (`0` disables). |
|
||||
| config.rustfs.scanner.cycle_max_objects | string | `""` | Cap objects processed by one scanner cycle with `RUSTFS_SCANNER_CYCLE_MAX_OBJECTS` (`0` disables). |
|
||||
| config.rustfs.scanner.cycle_max_directories | string | `""` | Cap directories entered by one scanner cycle with `RUSTFS_SCANNER_CYCLE_MAX_DIRECTORIES` (`0` disables). |
|
||||
| config.rustfs.scanner.bitrot_cycle_secs | string | `""` | Override periodic deep bitrot cycle with `RUSTFS_SCANNER_BITROT_CYCLE_SECS`; `false`, `off`, `no`, or `disabled` disables it. |
|
||||
| config.rustfs.scanner.idle_mode | string | `""` | Override scanner idle throttling flag (`RUSTFS_SCANNER_IDLE_MODE`). |
|
||||
| config.rustfs.scanner.cache_save_timeout_secs | string | `""` | Override scanner cache save timeout in seconds with `RUSTFS_SCANNER_CACHE_SAVE_TIMEOUT_SECS` (minimum `1`). |
|
||||
| config.rustfs.scanner.max_concurrent_set_scans | string | `""` | Cap concurrent scanner set tasks with `RUSTFS_SCANNER_MAX_CONCURRENT_SET_SCANS` (`0` keeps topology-derived concurrency). |
|
||||
| config.rustfs.scanner.max_concurrent_disk_scans | string | `""` | Cap concurrent scanner disk bucket walks per set with `RUSTFS_SCANNER_MAX_CONCURRENT_DISK_SCANS` (`0` keeps disk-count-derived concurrency). |
|
||||
| config.rustfs.scanner.yield_every_n_objects | string | `""` | Yield to the async runtime every N scanned objects with `RUSTFS_SCANNER_YIELD_EVERY_N_OBJECTS` (`0` disables extra yield). |
|
||||
| config.rustfs.scanner.alert_excess_versions | string | `""` | Set version count threshold for scanner alerts with `RUSTFS_SCANNER_ALERT_EXCESS_VERSIONS`. |
|
||||
| config.rustfs.scanner.alert_excess_version_size | string | `""` | Set retained version byte threshold for scanner alerts with `RUSTFS_SCANNER_ALERT_EXCESS_VERSION_SIZE`. |
|
||||
| config.rustfs.scanner.alert_excess_folders | string | `""` | Set direct subfolder threshold for scanner alerts with `RUSTFS_SCANNER_ALERT_EXCESS_FOLDERS`. |
|
||||
| config.rustfs.obs_endpoint.enabled | bool | `false` | Whether to send metrics/logs/traces/profilings to remote endpoint, eg, OLTP. |
|
||||
| config.rustfs.obs_endpoint.base_endpoint | string | `""` | Root OTLP/HTTP endpoint, e.g. http://otel-collector:4318. |
|
||||
| config.rustfs.obs_endpoint.use_stdout | bool | `false` | Whether to output logs to stdout in addition the OLTP. |
|
||||
@@ -166,6 +179,10 @@ uer. `ClusterIssuer` or `Issuer`. |
|
||||
| gatewayApi.existingGateway.name | string | `""` | The existing gateway name, instead of creating a new one. |
|
||||
| gatewayApi.existingGateway.namespace | string | `""` | The namespace of the existing gateway, if not the local namespace. |
|
||||
|
||||
Scanner values map directly to scanner environment variables. For tuning
|
||||
workflow and `/v3/scanner/status` interpretation, see
|
||||
[Scanner Runtime Controls](../docs/operations/scanner-runtime-controls.md).
|
||||
|
||||
---
|
||||
|
||||
**NOTE**:
|
||||
|
||||
@@ -73,18 +73,57 @@ data:
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- with .Values.config.rustfs.scanner }}
|
||||
{{- if .speed }}
|
||||
{{- if ne (toString .speed) "" }}
|
||||
RUSTFS_SCANNER_SPEED: {{ .speed | quote }}
|
||||
{{- end }}
|
||||
{{- if .start_delay_secs }}
|
||||
{{- if ne (toString .delay) "" }}
|
||||
RUSTFS_SCANNER_DELAY: {{ .delay | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .max_wait_secs) "" }}
|
||||
RUSTFS_SCANNER_MAX_WAIT_SECS: {{ .max_wait_secs | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .cycle_secs) "" }}
|
||||
RUSTFS_SCANNER_CYCLE: {{ .cycle_secs | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .start_delay_secs) "" }}
|
||||
RUSTFS_SCANNER_START_DELAY_SECS: {{ .start_delay_secs | quote }}
|
||||
{{- end }}
|
||||
{{- if .idle_mode }}
|
||||
{{- if ne (toString .cycle_max_duration_secs) "" }}
|
||||
RUSTFS_SCANNER_CYCLE_MAX_DURATION_SECS: {{ .cycle_max_duration_secs | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .cycle_max_objects) "" }}
|
||||
RUSTFS_SCANNER_CYCLE_MAX_OBJECTS: {{ .cycle_max_objects | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .cycle_max_directories) "" }}
|
||||
RUSTFS_SCANNER_CYCLE_MAX_DIRECTORIES: {{ .cycle_max_directories | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .bitrot_cycle_secs) "" }}
|
||||
RUSTFS_SCANNER_BITROT_CYCLE_SECS: {{ .bitrot_cycle_secs | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .idle_mode) "" }}
|
||||
RUSTFS_SCANNER_IDLE_MODE: {{ .idle_mode | quote }}
|
||||
{{- end }}
|
||||
{{- if .cache_save_timeout_secs }}
|
||||
{{- if ne (toString .cache_save_timeout_secs) "" }}
|
||||
RUSTFS_SCANNER_CACHE_SAVE_TIMEOUT_SECS: {{ .cache_save_timeout_secs | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .max_concurrent_set_scans) "" }}
|
||||
RUSTFS_SCANNER_MAX_CONCURRENT_SET_SCANS: {{ .max_concurrent_set_scans | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .max_concurrent_disk_scans) "" }}
|
||||
RUSTFS_SCANNER_MAX_CONCURRENT_DISK_SCANS: {{ .max_concurrent_disk_scans | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .yield_every_n_objects) "" }}
|
||||
RUSTFS_SCANNER_YIELD_EVERY_N_OBJECTS: {{ .yield_every_n_objects | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .alert_excess_versions) "" }}
|
||||
RUSTFS_SCANNER_ALERT_EXCESS_VERSIONS: {{ .alert_excess_versions | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .alert_excess_version_size) "" }}
|
||||
RUSTFS_SCANNER_ALERT_EXCESS_VERSION_SIZE: {{ .alert_excess_version_size | quote }}
|
||||
{{- end }}
|
||||
{{- if ne (toString .alert_excess_folders) "" }}
|
||||
RUSTFS_SCANNER_ALERT_EXCESS_FOLDERS: {{ .alert_excess_folders | quote }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if .Values.config.rustfs.timeout_profile }}
|
||||
RUSTFS_DRIVE_TIMEOUT_PROFILE: {{ .Values.config.rustfs.timeout_profile | quote }}
|
||||
|
||||
@@ -85,12 +85,38 @@ config:
|
||||
scanner:
|
||||
# Scanner speed preset: fastest|fast|default|slow|slowest
|
||||
speed: ""
|
||||
# Override scanner sleep multiplier (0-10000)
|
||||
delay: ""
|
||||
# Override maximum scanner sleep in seconds
|
||||
max_wait_secs: ""
|
||||
# Override scanner cycle interval in seconds
|
||||
cycle_secs: ""
|
||||
# Override start delay in seconds (optional)
|
||||
start_delay_secs: ""
|
||||
# Cap one scanner cycle's runtime in seconds (0 disables)
|
||||
cycle_max_duration_secs: ""
|
||||
# Cap objects processed by one scanner cycle (0 disables)
|
||||
cycle_max_objects: ""
|
||||
# Cap directories entered by one scanner cycle (0 disables)
|
||||
cycle_max_directories: ""
|
||||
# Periodic deep bitrot cycle in seconds; false/off/no/disabled disables
|
||||
bitrot_cycle_secs: ""
|
||||
# Enable/disable scanner sleeps for throttling
|
||||
idle_mode: ""
|
||||
# Timeout for scanner cache saves in seconds (minimum 1 second)
|
||||
cache_save_timeout_secs: ""
|
||||
# Cap concurrent scanner set tasks (0 keeps topology-derived concurrency)
|
||||
max_concurrent_set_scans: ""
|
||||
# Cap concurrent scanner disk bucket walks per set (0 keeps disk-count-derived concurrency)
|
||||
max_concurrent_disk_scans: ""
|
||||
# Yield to the async runtime every N scanned objects (0 disables extra yield)
|
||||
yield_every_n_objects: ""
|
||||
# Version count threshold for scanner alerts
|
||||
alert_excess_versions: ""
|
||||
# Retained version byte threshold for scanner alerts
|
||||
alert_excess_version_size: ""
|
||||
# Direct subfolder threshold for scanner alerts
|
||||
alert_excess_folders: ""
|
||||
# Drive timeout profile preset: default|high_latency
|
||||
# - default: keep current timeout defaults.
|
||||
# - high_latency: use higher timeout defaults for scanner-sensitive operations.
|
||||
|
||||
@@ -51,6 +51,42 @@ grep -Eq '^[[:space:]]*replicas:[[:space:]]*1[[:space:]]*$' <<<"$rolling_output"
|
||||
scaled_to_zero_output=$(render_standalone_deployment --set replicaCount=0)
|
||||
grep -Eq '^[[:space:]]*replicas:[[:space:]]*0[[:space:]]*$' <<<"$scaled_to_zero_output"
|
||||
|
||||
scanner_config_output=$(render_chart \
|
||||
--set config.rustfs.scanner.speed=slow \
|
||||
--set config.rustfs.scanner.delay=30 \
|
||||
--set config.rustfs.scanner.max_wait_secs=15 \
|
||||
--set config.rustfs.scanner.cycle_secs=3600 \
|
||||
--set config.rustfs.scanner.start_delay_secs=60 \
|
||||
--set config.rustfs.scanner.cycle_max_duration_secs=1800 \
|
||||
--set config.rustfs.scanner.cycle_max_objects=0 \
|
||||
--set config.rustfs.scanner.cycle_max_directories=100000 \
|
||||
--set config.rustfs.scanner.bitrot_cycle_secs=0 \
|
||||
--set config.rustfs.scanner.idle_mode=false \
|
||||
--set config.rustfs.scanner.cache_save_timeout_secs=30 \
|
||||
--set config.rustfs.scanner.max_concurrent_set_scans=2 \
|
||||
--set config.rustfs.scanner.max_concurrent_disk_scans=1 \
|
||||
--set config.rustfs.scanner.yield_every_n_objects=128 \
|
||||
--set config.rustfs.scanner.alert_excess_versions=100 \
|
||||
--set config.rustfs.scanner.alert_excess_version_size=1099511627776 \
|
||||
--set config.rustfs.scanner.alert_excess_folders=65538)
|
||||
grep -q 'RUSTFS_SCANNER_SPEED: "slow"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_DELAY: "30"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_MAX_WAIT_SECS: "15"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_CYCLE: "3600"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_START_DELAY_SECS: "60"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_CYCLE_MAX_DURATION_SECS: "1800"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_CYCLE_MAX_OBJECTS: "0"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_CYCLE_MAX_DIRECTORIES: "100000"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_BITROT_CYCLE_SECS: "0"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_IDLE_MODE: "false"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_CACHE_SAVE_TIMEOUT_SECS: "30"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_MAX_CONCURRENT_SET_SCANS: "2"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_MAX_CONCURRENT_DISK_SCANS: "1"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_YIELD_EVERY_N_OBJECTS: "128"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_ALERT_EXCESS_VERSIONS: "100"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_ALERT_EXCESS_VERSION_SIZE: "1099511627776"' <<<"$scanner_config_output"
|
||||
grep -q 'RUSTFS_SCANNER_ALERT_EXCESS_FOLDERS: "65538"' <<<"$scanner_config_output"
|
||||
|
||||
# Fail-closed credential checks. Rendering must fail when no credentials,
|
||||
# existingSecret, or allowInsecureDefaults override is supplied.
|
||||
default_render_status=0
|
||||
|
||||
Reference in New Issue
Block a user