# Concurrent Download Tool (example)

This tool downloads multiple URLs concurrently and saves files to a target directory.

Saved filename format:

`<nanoseconds>_<index>_<original_filename>`

All downloaded files are written into one output directory.

## Environment variables

- `DOWNLOAD_URLS` (required): comma-separated URLs.
- `DOWNLOAD_OUTPUT_DIR` (optional): output directory, default `target/tmp/concurrent_downloads`.
- `DOWNLOAD_CONCURRENCY` (optional): max concurrent downloads, default `8`.
- `DOWNLOAD_REPEAT` (optional): repeat count per URL, default `1`.
- `DOWNLOAD_MAX_RETRIES` (optional): retry count per task after first failure, default `0`.
- `DOWNLOAD_RETRY_BACKOFF_MS` (optional): fixed backoff between retries, default `200`.

## Statistics output

After run, the tool prints:

- total tasks
- succeeded
- failed
- total bytes
- elapsed ms
- throughput bps
- total attempts
- retried tasks
- retry attempts
- latency p50 ms
- latency p95 ms
- failure details (`[index] url => error`) when failures exist

If any task fails, the tool exits with an error after printing the summary.

Retry is triggered only for recoverable cases:

- network/request timeout/connect errors
- HTTP `429`
- HTTP `5xx`

## Compile check

```bash
cargo build -p rustfs --example concurrent_download_tool
```

## Manual run example

The commands below are for manual execution only.
The tool is an example binary, so it is built on demand and never runs as part of the test suites.

```bash
DOWNLOAD_URLS="http://127.0.0.1:9001/demo/google-cloud-aiplugin-1.46.1-253.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=HAXVOTZK9MLBJT8KWI4E%2F20260329%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260329T105159Z&X-Amz-Expires=86400&X-Amz-Security-Token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJwYXJlbnQiOiJydXN0ZnNhZG1pbiIsImV4cCI6MTc3NDgyMDgyMX0.tYhQoPRcg0Ysx4KVw9ez7ZpYxsqGgqomtsP_iaeTsKzoii8EVNt74BZm2wbUjXW-FbGXc1pqEYX6wZ5Ncpk9Iw&X-Amz-Signature=15f47b19832f53b34f9e0fe1862d53d71660bbf8f1a512669bb2d041ac8d0697&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject" \
DOWNLOAD_OUTPUT_DIR="/Users/zhi/Documents/code/rust/rustfs/rustfs/target/tmp/concurrent_downloads" \
DOWNLOAD_CONCURRENCY="40" \
DOWNLOAD_REPEAT="40" \
DOWNLOAD_MAX_RETRIES="2" \
DOWNLOAD_RETRY_BACKOFF_MS="300" \
cargo run -p rustfs --example concurrent_download_tool
```

