mirror of https://github.com/rustfs/rustfs.git synced 2026-07-26 08:18:18 +00:00

Files

T

Zhengchao An eb392f24d6 chore(scripts): add scripts index and archive one-shot scripts (#4822 )

chore(scripts): index scripts/ and archive 29 one-shot scripts

backlog#1153 infra-13. scripts/ had 80+ unlabelled top-level entries
mixing CI gates with finished one-shot issue-validation scripts.

- scripts/README.md — one index row per entry with status
  (ci-gate / dev-tool / archived), purpose, and wiring; subdirectories
  get one row each. run_scanner_benchmarks.sh is annotated
  "disposition owned by backlog perf-10" and deliberately untouched.
- git mv 29 confirmed-stale one-shot entries to scripts/archive/:
  11 issue-scoped validation/perf-capture scripts, the 5-script
  backlog#706 large-PUT breakdown family, the 4-file GET-optimization
  stress suite, 2 gt1g one-shots, and 7 other orphaned one-shots.
  Evidence: a whole-tree boundary-aware reference census showed zero
  references from CI/Makefiles/docs/code for every moved entry (or
  references only from other scripts inside the same archived set);
  re-run after the move shows zero dangling references.
- docs/testing/README.md links the index.

Moves only — no script content changed.

2026-07-15 03:43:26 +00:00

3.3 KiB

Raw Permalink Blame History

GET Optimization Stress Test Scripts

Quick Start

Prerequisites

warp installed (https://github.com/minio/warp)
mc configured with access to the RustFS server
RustFS server running with GET optimizations enabled

Quick Validation (5 minutes)

./scripts/quick-validate-get-optimization.sh localhost:9000

This runs basic functional tests:

Data integrity verification
Concurrent GET stability
Early-stop behavior validation

Full Stress Test (30+ minutes)

./scripts/stress-test-get-optimization.sh localhost:9000 ./stress-results

This runs comprehensive tests:

Data correctness validation (1KB, 1MB, 10MB objects)
Concurrent GET stress test (16, 64, 256 concurrency)
Mixed read/write workload
Early-stop behavior under load

Environment Variables

Variable	Default	Description
`WARP_ACCESS_KEY`	rustfsadmin	S3 access key
`WARP_SECRET_KEY`	rustfsadmin	S3 secret key
`MC_ALIAS`	rustfs	mc alias for the server
`TEST_BUCKET`	auto-generated	Test bucket name
`TEST_DURATION`	300s	Duration for stress tests
`CONCURRENCY`	64	Default concurrency level

Test Scenarios

1. Data Correctness Validation

Verifies that GET returns correct data with early-stop enabled:

Uploads random data
Downloads and compares MD5 hash
Tests different object sizes (1KB, 1MB, 10MB)

2. Concurrent GET Stress Test

Tests performance under high concurrency:

Object sizes: 1KiB, 1MiB, 4MiB, 10MiB
Concurrency levels: 16, 64, 256
Duration: configurable (default 300s)

3. Mixed Read/Write Stress Test

Tests stability under concurrent read/write:

50% reads, 50% writes
1MB objects
64 concurrent operations

4. Early-Stop Behavior Validation

Verifies early-stop works correctly:

100 sequential reads of the same object
Verifies data size matches expected
Checks for any download failures

Expected Results

Success Criteria

Data Correctness: 100% pass rate (no data corruption)
Concurrent GET: No errors, consistent latency
Mixed Workload: No deadlocks or data corruption
Early-Stop: All reads return correct data size

Performance Baselines

Object Size	Expected Throughput	Expected p95 Latency
1KiB	> 5 MiB/s	< 10ms
1MiB	> 500 MiB/s	< 20ms
4MiB	> 1000 MiB/s	< 30ms
10MiB	> 2000 MiB/s	< 50ms

Troubleshooting

Common Issues

warp not found: Install with go install github.com/minio/warp@latest
mc not configured: Run mc alias set rustfs http://localhost:9000 admin password
Connection refused: Verify RustFS is running and accessible
Permission denied: Check S3 credentials

Debug Mode

Enable debug logging:

RUST_LOG=rustfs_ecstore::bucket::lifecycle=debug ./scripts/stress-test-get-optimization.sh

Output Files

File	Description
`test-config.txt`	Test configuration
`correctness-results.txt`	Data correctness results
`early-stop-results.txt`	Early-stop validation results
`get-*.json`	Concurrent GET performance (warp JSON)
`mixed-*.json`	Mixed workload performance (warp JSON)
`correctness-errors.log`	Data correctness errors (if any)
`early-stop-errors.log`	Early-stop errors (if any)

3.3 KiB Raw Permalink Blame History