fix: restore localized samples in tests (#749)

* fix: restore required localized examples

* style: fix formatting issues
This commit is contained in:
安正超
2025-10-29 13:16:31 +08:00
committed by GitHub
parent 64ba52bc1e
commit dd47fcf2a8
41 changed files with 1294 additions and 3312 deletions
+130 -144
View File
@@ -1,267 +1,253 @@
# KMS End-to-End Tests
本目录包含 RustFS KMS (Key Management Service) 的端到端集成测试,用于验证完整的 KMS 功能流程。
This directory contains the integration suites used to validate the full RustFS KMS (Key Management Service) workflow.
## 📁 测试文件说明
## 📁 Test Overview
### `kms_local_test.rs`
本地KMS后端的端到端测试,包含:
- 自动启动和配置本地KMS后端
- 通过动态配置API配置KMS服务
- 测试SSE-C(客户端提供密钥)加密流程
- 验证S3兼容的对象加密/解密操作
- 密钥生命周期管理测试
End-to-end coverage for the local KMS backend:
- Auto-start and configure the local backend
- Configure KMS through the dynamic configuration API
- Verify SSE-C (client-provided keys)
- Exercise S3-compatible encryption/decryption
- Validate key lifecycle management
### `kms_vault_test.rs`
Vault KMS后端的端到端测试,包含:
- 自动启动Vault开发服务器
- 配置Vault transit engine和密钥
- 通过动态配置API配置KMS服务
- 测试完整的Vault KMS集成
- 验证Token认证和加密操作
End-to-end coverage for the Vault backend:
- Launch a Vault dev server automatically
- Configure the transit engine and encryption keys
- Configure KMS via the dynamic configuration API
- Run the full Vault integration flow
- Validate token authentication and encryption operations
### `kms_comprehensive_test.rs`
**完整的KMS功能测试套件**(当前因AWS SDK API兼容性问题暂时禁用),包含:
- **Bucket加密配置**: SSE-S3SSE-KMS默认加密设置
- **完整的SSE加密模式测试**:
- SSE-S3: S3管理的服务端加密
- SSE-KMS: KMS管理的服务端加密
- SSE-C: 客户端提供密钥的服务端加密
- **对象操作测试**: 上传、下载、验证三种SSE模式
- **分片上传测试**: 多部分上传支持所有SSE模式
- **对象复制测试**: 不同SSE模式间的复制操作
- **完整KMS API管理**:
- 密钥生命周期管理(创建、列表、描述、删除、取消删除)
- 直接加密/解密操作
- 数据密钥生成和操作
- KMS服务管理(启动、停止、状态查询)
**Full KMS capability suite** (currently disabled because of AWS SDK compatibility issues):
- **Bucket encryption configuration**: SSE-S3 and SSE-KMS defaults
- **All SSE encryption modes**:
- SSE-S3 (S3-managed server-side encryption)
- SSE-KMS (KMS-managed server-side encryption)
- SSE-C (client-provided keys)
- **Object operations**: upload, download, and validation for every SSE mode
- **Multipart uploads**: cover each SSE mode
- **Object replication**: cross-mode replication scenarios
- **Complete KMS API management**:
- Key lifecycle (create, list, describe, delete, cancel delete)
- Direct encrypt/decrypt operations
- Data key generation and handling
- KMS service lifecycle (start, stop, status)
### `kms_integration_test.rs`
综合性KMS集成测试,包含:
- 多后端兼容性测试
- KMS服务生命周期测试
- 错误处理和恢复测试
- **注意**: 当前因AWS SDK API兼容性问题暂时禁用
Broad integration tests that exercise:
- Multiple backends
- KMS lifecycle management
- Error handling and recovery
- **Note**: currently disabled because of AWS SDK compatibility gaps
## 🚀 如何运行测试
## 🚀 Running Tests
### 前提条件
### Prerequisites
1. **系统依赖**
1. **System dependencies**
```bash
# macOS
brew install vault awscurl
# Ubuntu/Debian
apt-get install vault
pip install awscurl
```
2. **构建RustFS**
2. **Build RustFS**
```bash
# 在项目根目录
cargo build
```
### 运行单个测试
### Run individual suites
#### 本地KMS测试
#### Local backend
```bash
cd crates/e2e_test
cargo test test_local_kms_end_to_end -- --nocapture
```
#### Vault KMS测试
#### Vault backend
```bash
cd crates/e2e_test
cargo test test_vault_kms_end_to_end -- --nocapture
```
#### 高可用性测试
#### High availability
```bash
cd crates/e2e_test
cargo test test_vault_kms_high_availability -- --nocapture
```
#### 完整功能测试(开发中)
#### Comprehensive features (disabled)
```bash
cd crates/e2e_test
# 注意:以下测试因AWS SDK API兼容性问题暂时禁用
# Disabled due to AWS SDK compatibility gaps
# cargo test test_comprehensive_kms_functionality -- --nocapture
# cargo test test_sse_modes_compatibility -- --nocapture
# cargo test test_sse_modes_compatibility -- --nocapture
# cargo test test_kms_api_comprehensive -- --nocapture
```
### 运行所有KMS测试
### Run all KMS suites
```bash
cd crates/e2e_test
cargo test kms -- --nocapture
```
### 串行运行(避免端口冲突)
### Run serially (avoid port conflicts)
```bash
cd crates/e2e_test
cargo test kms -- --nocapture --test-threads=1
```
## 🔧 测试配置
## 🔧 Configuration
### 环境变量
### Environment variables
```bash
# 可选:自定义端口(默认使用9050
# Optional: custom RustFS port (default 9050)
export RUSTFS_TEST_PORT=9050
# 可选:自定义Vault端口(默认使用8200
# Optional: custom Vault port (default 8200)
export VAULT_TEST_PORT=8200
# 可选:启用详细日志
# Optional: enable verbose logging
export RUST_LOG=debug
```
### 依赖的二进制文件路径
### Required binaries
测试会自动查找以下二进制文件:
- `../../target/debug/rustfs` - RustFS服务器
- `vault` - Vault (需要在PATH)
- `/Users/dandan/Library/Python/3.9/bin/awscurl` - AWS签名工具
Tests look for:
- `../../target/debug/rustfs` RustFS server
- `vault` Vault CLI (must be on PATH)
- `/Users/dandan/Library/Python/3.9/bin/awscurl` AWS SigV4 helper
## 📋 测试流程说明
## 📋 Test Flow
### Local KMS测试流程
1. **环境准备**:创建临时目录,设置KMS密钥存储路径
2. **启动服务**:启动RustFS服务器,启用KMS功能
3. **等待就绪**:检查端口监听和S3 API响应
4. **配置KMS**:通过awscurl发送配置请求到admin API
5. **启动KMS**:激活KMS服务
6. **功能测试**
- 创建测试存储桶
- 测试SSE-C加密(客户端提供密钥)
- 验证对象加密/解密
7. **清理**:终止进程,清理临时文件
### Local backend
1. **Prepare environment** create temporary directories and key storage paths
2. **Start RustFS** launch the server with KMS enabled
3. **Wait for readiness** confirm the port listener and S3 API
4. **Configure KMS** send configuration via awscurl to the admin API
5. **Start KMS** activate the KMS service
6. **Exercise functionality**
- Create a test bucket
- Run SSE-C encryption with client-provided keys
- Validate encryption/decryption behavior
7. **Cleanup** stop processes and remove temporary files
### Vault KMS测试流程
1. **启动Vault**:使用开发模式启动Vault服务器
2. **配置Vault**
- 启用transit secrets engine
- 创建加密密钥(rustfs-master-key
3. **启动RustFS**:启用KMS功能的RustFS服务器
4. **配置KMS**:通过API配置Vault后端,包含:
- Vault地址和Token认证
- Transit engine配置
- 密钥路径设置
5. **功能测试**:完整的加密/解密流程测试
6. **清理**:终止所有进程
### Vault backend
1. **Launch Vault** start the dev-mode server
2. **Configure Vault**
- Enable the transit secrets engine
- Create the `rustfs-master-key`
3. **Start RustFS** run the server with KMS enabled
4. **Configure KMS** point RustFS at Vault (address, token, transit config, key path)
5. **Exercise functionality** complete the encryption/decryption workflow
6. **Cleanup** stop all services
## 🛠️ 故障排除
## 🛠️ Troubleshooting
### 常见问题
### Common issues
**Q: 测试失败 "RustFS server failed to become ready"**
```
A: 检查端口是否被占用:
**Q: `RustFS server failed to become ready`**
```bash
lsof -i :9050
kill -9 <PID> # 如果有进程占用端口
kill -9 <PID> # Free the port if necessary
```
**Q: Vault服务启动失败**
```
A: 确保Vault已安装且在PATH中:
**Q: Vault fails to start**
```bash
which vault
vault version
```
**Q: awscurl认证失败**
```
A: 检查awscurl路径是否正确:
**Q: awscurl authentication fails**
```bash
ls /Users/dandan/Library/Python/3.9/bin/awscurl
# 或安装到不同路径:
# Or install elsewhere
pip install awscurl
which awscurl # 然后更新测试中的路径
which awscurl # Update the path in tests accordingly
```
**Q: 测试超时**
```
A: 增加等待时间或检查日志:
**Q: Tests time out**
```bash
RUST_LOG=debug cargo test test_local_kms_end_to_end -- --nocapture
```
### 调试技巧
### Debug tips
1. **查看详细日志**
1. **Enable verbose logs**
```bash
RUST_LOG=rustfs_kms=debug,rustfs=info cargo test -- --nocapture
```
2. **保留临时文件**
修改测试代码,注释掉清理部分,检查生成的配置文件
2. **Keep temporary files** comment out cleanup logic to inspect generated configs
3. **单步调试**
在测试中添加 `std::thread::sleep` 来暂停执行,手动检查服务状态
3. **Pause execution** add `std::thread::sleep` for manual inspection during tests
4. **端口检查**
4. **Monitor ports**
```bash
# 测试运行时检查端口状态
netstat -an | grep 9050
curl http://127.0.0.1:9050/minio/health/ready
```
## 📊 测试覆盖范围
## 📊 Coverage
### 功能覆盖
- ✅ KMS服务动态配置
- ✅ 本地和Vault后端支持
- ✅ AWS S3兼容加密接口
- ✅ 密钥管理和生命周期
- ✅ 错误处理和恢复
- ✅ 高可用性场景
### Functional
- ✅ Dynamic KMS configuration
- ✅ Local and Vault backends
- ✅ AWS S3-compatible encryption APIs
- ✅ Key lifecycle management
- ✅ Error handling and recovery paths
- ✅ High-availability behavior
### 加密模式覆盖
- ✅ SSE-C (Server-Side Encryption with Customer-Provided Keys)
- ✅ SSE-S3 (Server-Side Encryption with S3-Managed Keys)
- ✅ SSE-KMS (Server-Side Encryption with KMS-Managed Keys)
### Encryption modes
- ✅ SSE-C (customer-provided)
- ✅ SSE-S3 (S3-managed)
- ✅ SSE-KMS (KMS-managed)
### S3操作覆盖
- ✅ 对象上传/下载 (SSE-C模式)
- 🚧 分片上传 (需要AWS SDK兼容性修复)
- 🚧 对象复制 (需要AWS SDK兼容性修复)
- 🚧 Bucket加密配置 (需要AWS SDK兼容性修复)
### S3 operations
- ✅ Object upload/download (SSE-C)
- 🚧 Multipart uploads (pending AWS SDK fixes)
- 🚧 Object replication (pending AWS SDK fixes)
- 🚧 Bucket encryption defaults (pending AWS SDK fixes)
### KMS API覆盖
- ✅ 基础密钥管理 (创建、列表)
- 🚧 完整密钥生命周期 (需要AWS SDK兼容性修复)
- 🚧 直接加密/解密操作 (需要AWS SDK兼容性修复)
- 🚧 数据密钥生成和解密 (需要AWS SDK兼容性修复)
- ✅ KMS服务管理 (配置、启动、停止、状态)
### KMS API
- ✅ Basic key management (create/list)
- 🚧 Full key lifecycle (pending AWS SDK fixes)
- 🚧 Direct encrypt/decrypt (pending AWS SDK fixes)
- 🚧 Data key operations (pending AWS SDK fixes)
- ✅ Service lifecycle (configure/start/stop/status)
### 认证方式覆盖
- ✅ Vault Token认证
- 🚧 Vault AppRole认证
### Authentication
- ✅ Vault token auth
- 🚧 Vault AppRole auth
## 🔄 持续集成
## 🔄 CI Integration
这些测试设计为可在CI/CD环境中运行:
Designed to run inside CI/CD pipelines:
```yaml
# GitHub Actions 示例
- name: Run KMS E2E Tests
run: |
# 安装依赖
sudo apt-get update
sudo apt-get install -y vault
pip install awscurl
# 构建并测试
cargo build
cd crates/e2e_test
cargo test kms -- --nocapture --test-threads=1
```
## 📚 相关文档
## 📚 References
- [KMS 配置文档](../../../../docs/kms/README.md) - KMS功能完整文档
- [动态配置API](../../../../docs/kms/http-api.md) - REST API接口说明
- [故障排除指南](../../../../docs/kms/troubleshooting.md) - 常见问题解决
- [KMS configuration guide](../../../../docs/kms/README.md)
- [Dynamic configuration API](../../../../docs/kms/http-api.md)
- [Troubleshooting](../../../../docs/kms/troubleshooting.md)
---
*这些测试确保KMS功能的稳定性和可靠性,为生产环境部署提供信心。*
*These suites ensure KMS stability and reliability, building confidence for production deployments.*
+11 -11
View File
@@ -547,9 +547,9 @@ pub async fn test_multipart_upload_with_config(
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let total_size = config.total_size();
info!("🧪 开始分片上传测试 - {:?}", config.encryption_type);
info!("🧪 Starting multipart upload test - {:?}", config.encryption_type);
info!(
" 对象: {}, 分片: {}, 每片: {}MB, 总计: {}MB",
" Object: {}, parts: {}, part size: {} MB, total: {} MB",
config.object_key,
config.total_parts,
config.part_size / (1024 * 1024),
@@ -589,7 +589,7 @@ pub async fn test_multipart_upload_with_config(
let create_multipart_output = create_request.send().await?;
let upload_id = create_multipart_output.upload_id().unwrap();
info!("📋 创建分片上传,ID: {}", upload_id);
info!("📋 Created multipart upload, ID: {}", upload_id);
// Step 2: Upload parts
let mut completed_parts = Vec::new();
@@ -598,7 +598,7 @@ pub async fn test_multipart_upload_with_config(
let end = std::cmp::min(start + config.part_size, total_size);
let part_data = &test_data[start..end];
info!("📤 上传分片 {} ({:.2}MB)", part_number, part_data.len() as f64 / (1024.0 * 1024.0));
info!("📤 Uploading part {} ({:.2} MB)", part_number, part_data.len() as f64 / (1024.0 * 1024.0));
let mut upload_request = s3_client
.upload_part()
@@ -625,7 +625,7 @@ pub async fn test_multipart_upload_with_config(
.build(),
);
debug!("分片 {} 上传完成,ETag: {}", part_number, etag);
debug!("Part {} uploaded with ETag {}", part_number, etag);
}
// Step 3: Complete multipart upload
@@ -633,7 +633,7 @@ pub async fn test_multipart_upload_with_config(
.set_parts(Some(completed_parts))
.build();
info!("🔗 完成分片上传");
info!("🔗 Completing multipart upload");
let complete_output = s3_client
.complete_multipart_upload()
.bucket(bucket)
@@ -643,10 +643,10 @@ pub async fn test_multipart_upload_with_config(
.send()
.await?;
debug!("完成分片上传,ETag: {:?}", complete_output.e_tag());
debug!("Multipart upload finalized with ETag {:?}", complete_output.e_tag());
// Step 4: Download and verify
info!("📥 下载文件并验证");
info!("📥 Downloading object for verification");
let mut get_request = s3_client.get_object().bucket(bucket).key(&config.object_key);
// Add encryption headers for SSE-C GET
@@ -680,7 +680,7 @@ pub async fn test_multipart_upload_with_config(
assert_eq!(downloaded_data.len(), total_size);
assert_eq!(&downloaded_data[..], &test_data[..]);
info!("分片上传测试通过 - {:?}", config.encryption_type);
info!("Multipart upload test passed - {:?}", config.encryption_type);
Ok(())
}
@@ -700,7 +700,7 @@ pub async fn test_all_multipart_encryption_types(
bucket: &str,
base_object_key: &str,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
info!("🧪 测试所有加密类型的分片上传");
info!("🧪 Testing multipart uploads for every encryption type");
let part_size = 5 * 1024 * 1024; // 5MB per part
let total_parts = 2;
@@ -718,7 +718,7 @@ pub async fn test_all_multipart_encryption_types(
test_multipart_upload_with_config(s3_client, bucket, &config).await?;
}
info!("所有加密类型的分片上传测试通过");
info!("Multipart uploads succeeded for every encryption type");
Ok(())
}
@@ -201,7 +201,7 @@ async fn test_step3_multipart_upload_with_sse_s3() -> Result<(), Box<dyn std::er
let total_parts = 2;
let total_size = part_size * total_parts;
// 生成测试数据
// Generate test data
let test_data: Vec<u8> = (0..total_size).map(|i| ((i / 1000) % 256) as u8).collect();
info!(
@@ -275,43 +275,43 @@ async fn test_step3_multipart_upload_with_sse_s3() -> Result<(), Box<dyn std::er
.send()
.await?;
debug!("完成加密分片上传,ETag: {:?}", complete_output.e_tag());
debug!("Encrypted multipart upload completed with ETag {:?}", complete_output.e_tag());
// 步骤 4HEAD 请求检查元数据
info!("📋 检查对象元数据");
// Step 4: HEAD request to inspect metadata
info!("📋 Inspecting object metadata");
let head_response = s3_client.head_object().bucket(TEST_BUCKET).key(object_key).send().await?;
debug!("HEAD 响应 SSE: {:?}", head_response.server_side_encryption());
debug!("HEAD 响应 元数据:{:?}", head_response.metadata());
debug!("HEAD response SSE: {:?}", head_response.server_side_encryption());
debug!("HEAD response metadata: {:?}", head_response.metadata());
// 步骤 5GET 请求下载并验证
info!("📥 下载加密文件并验证");
// Step 5: GET request to download and verify
info!("📥 Downloading encrypted object for verification");
let get_response = s3_client.get_object().bucket(TEST_BUCKET).key(object_key).send().await?;
debug!("GET 响应 SSE: {:?}", get_response.server_side_encryption());
debug!("GET response SSE: {:?}", get_response.server_side_encryption());
// 🎯 关键验证:GET 响应必须包含 SSE-S3 加密头
// 🎯 Critical check: GET response must include SSE-S3 headers
assert_eq!(
get_response.server_side_encryption(),
Some(&aws_sdk_s3::types::ServerSideEncryption::Aes256)
);
// 验证数据完整性
// Verify data integrity
let downloaded_data = get_response.body.collect().await?.into_bytes();
assert_eq!(downloaded_data.len(), total_size);
assert_eq!(&downloaded_data[..], &test_data[..]);
kms_env.base_env.delete_test_bucket(TEST_BUCKET).await?;
info!("步骤 3 通过:分片上传 + SSE-S3 加密功能正常");
info!("Step 3 passed: multipart upload with SSE-S3 encryption");
Ok(())
}
/// 步骤 4:测试更大的分片上传(测试流式加密)
/// Step 4: test larger multipart uploads (streaming encryption)
#[tokio::test]
#[serial]
async fn test_step4_large_multipart_upload_with_encryption() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
init_logging();
info!("🧪 步骤 4:测试大文件分片上传加密");
info!("🧪 Step 4: test large-file multipart encryption");
let mut kms_env = LocalKMSTestEnvironment::new().await?;
let _default_key_id = kms_env.start_rustfs_for_local_kms().await?;
@@ -321,18 +321,18 @@ async fn test_step4_large_multipart_upload_with_encryption() -> Result<(), Box<d
kms_env.base_env.create_test_bucket(TEST_BUCKET).await?;
let object_key = "test-large-multipart-encrypted";
let part_size = 6 * 1024 * 1024; // 6MB per part (大于 1MB 加密块大小)
let total_parts = 3; // 总共 18MB
let part_size = 6 * 1024 * 1024; // 6 MB per part (greater than the 1 MB encryption chunk)
let total_parts = 3; // 18 MB total
let total_size = part_size * total_parts;
info!(
"🗂️ 生成大文件测试数据:{} parts,每个 {}MB,总计 {}MB",
"🗂️ Generated large-file test data: {} parts, {} MB each, {} MB total",
total_parts,
part_size / (1024 * 1024),
total_size / (1024 * 1024)
);
// 生成大文件测试数据(使用复杂模式便于验证)
// Generate large test data (complex pattern for validation)
let test_data: Vec<u8> = (0..total_size)
.map(|i| {
let part_num = i / part_size;
@@ -341,9 +341,9 @@ async fn test_step4_large_multipart_upload_with_encryption() -> Result<(), Box<d
})
.collect();
info!("🔐 开始大文件分片上传(SSE-S3 加密)");
info!("🔐 Starting large-file multipart upload (SSE-S3 encryption)");
// 创建分片上传
// Create multipart upload
let create_multipart_output = s3_client
.create_multipart_upload()
.bucket(TEST_BUCKET)
@@ -353,9 +353,9 @@ async fn test_step4_large_multipart_upload_with_encryption() -> Result<(), Box<d
.await?;
let upload_id = create_multipart_output.upload_id().unwrap();
info!("📋 创建大文件加密分片上传,ID: {}", upload_id);
info!("📋 Created large encrypted multipart upload, ID: {}", upload_id);
// 上传各个分片
// Upload each part
let mut completed_parts = Vec::new();
for part_number in 1..=total_parts {
let start = (part_number - 1) * part_size;
@@ -363,7 +363,7 @@ async fn test_step4_large_multipart_upload_with_encryption() -> Result<(), Box<d
let part_data = &test_data[start..end];
info!(
"🔐 上传大文件加密分片 {} ({:.2}MB)",
"🔐 Uploading encrypted large-file part {} ({:.2} MB)",
part_number,
part_data.len() as f64 / (1024.0 * 1024.0)
);
@@ -386,15 +386,15 @@ async fn test_step4_large_multipart_upload_with_encryption() -> Result<(), Box<d
.build(),
);
debug!("大文件加密分片 {} 上传完成,ETag: {}", part_number, etag);
debug!("Large encrypted part {} uploaded with ETag {}", part_number, etag);
}
// 完成分片上传
// Complete the multipart upload
let completed_multipart_upload = aws_sdk_s3::types::CompletedMultipartUpload::builder()
.set_parts(Some(completed_parts))
.build();
info!("🔗 完成大文件加密分片上传");
info!("🔗 Completing large encrypted multipart upload");
let complete_output = s3_client
.complete_multipart_upload()
.bucket(TEST_BUCKET)
@@ -404,40 +404,40 @@ async fn test_step4_large_multipart_upload_with_encryption() -> Result<(), Box<d
.send()
.await?;
debug!("完成大文件加密分片上传,ETag: {:?}", complete_output.e_tag());
debug!("Large encrypted multipart upload completed with ETag {:?}", complete_output.e_tag());
// 下载并验证
info!("📥 下载大文件并验证");
// Download and verify
info!("📥 Downloading large object for verification");
let get_response = s3_client.get_object().bucket(TEST_BUCKET).key(object_key).send().await?;
// 验证加密头
// Verify encryption headers
assert_eq!(
get_response.server_side_encryption(),
Some(&aws_sdk_s3::types::ServerSideEncryption::Aes256)
);
// 验证数据完整性
// Verify data integrity
let downloaded_data = get_response.body.collect().await?.into_bytes();
assert_eq!(downloaded_data.len(), total_size);
// 逐字节验证数据(对于大文件更严格)
// Validate bytes individually (stricter for large files)
for (i, (&actual, &expected)) in downloaded_data.iter().zip(test_data.iter()).enumerate() {
if actual != expected {
panic!("大文件数据在第{i}字节不匹配:实际={actual}, 期待={expected}");
panic!("Large file mismatch at byte {i}: actual={actual}, expected={expected}");
}
}
kms_env.base_env.delete_test_bucket(TEST_BUCKET).await?;
info!("步骤 4 通过:大文件分片上传加密功能正常");
info!("Step 4 passed: large-file multipart encryption succeeded");
Ok(())
}
/// 步骤 5:测试所有加密类型的分片上传
/// Step 5: test multipart uploads for every encryption mode
#[tokio::test]
#[serial]
async fn test_step5_all_encryption_types_multipart() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
init_logging();
info!("🧪 步骤 5:测试所有加密类型的分片上传");
info!("🧪 Step 5: test multipart uploads for every encryption mode");
let mut kms_env = LocalKMSTestEnvironment::new().await?;
let _default_key_id = kms_env.start_rustfs_for_local_kms().await?;
@@ -450,8 +450,8 @@ async fn test_step5_all_encryption_types_multipart() -> Result<(), Box<dyn std::
let total_parts = 2;
let total_size = part_size * total_parts;
// 测试 SSE-KMS
info!("🔐 测试 SSE-KMS 分片上传");
// Test SSE-KMS
info!("🔐 Testing SSE-KMS multipart upload");
test_multipart_encryption_type(
&s3_client,
TEST_BUCKET,
@@ -463,8 +463,8 @@ async fn test_step5_all_encryption_types_multipart() -> Result<(), Box<dyn std::
)
.await?;
// 测试 SSE-C
info!("🔐 测试 SSE-C 分片上传");
// Test SSE-C
info!("🔐 Testing SSE-C multipart upload");
test_multipart_encryption_type(
&s3_client,
TEST_BUCKET,
@@ -477,7 +477,7 @@ async fn test_step5_all_encryption_types_multipart() -> Result<(), Box<dyn std::
.await?;
kms_env.base_env.delete_test_bucket(TEST_BUCKET).await?;
info!("步骤 5 通过:所有加密类型的分片上传功能正常");
info!("Step 5 passed: multipart uploads succeeded for every encryption mode");
Ok(())
}
@@ -487,7 +487,7 @@ enum EncryptionType {
SSEC,
}
/// 辅助函数:测试特定加密类型的分片上传
/// Helper: test multipart uploads for a specific encryption type
async fn test_multipart_encryption_type(
s3_client: &aws_sdk_s3::Client,
bucket: &str,
@@ -497,10 +497,10 @@ async fn test_multipart_encryption_type(
total_parts: usize,
encryption_type: EncryptionType,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
// 生成测试数据
// Generate test data
let test_data: Vec<u8> = (0..total_size).map(|i| ((i * 7) % 256) as u8).collect();
// 准备 SSE-C 所需的密钥(如果需要)
// Prepare SSE-C keys when required
let (sse_c_key, sse_c_md5) = if matches!(encryption_type, EncryptionType::SSEC) {
let key = "01234567890123456789012345678901";
let key_b64 = base64::Engine::encode(&base64::engine::general_purpose::STANDARD, key);
@@ -510,9 +510,9 @@ async fn test_multipart_encryption_type(
(None, None)
};
info!("📋 创建分片上传 - {:?}", encryption_type);
info!("📋 Creating multipart upload - {:?}", encryption_type);
// 创建分片上传
// Create multipart upload
let mut create_request = s3_client.create_multipart_upload().bucket(bucket).key(object_key);
create_request = match encryption_type {
@@ -526,7 +526,7 @@ async fn test_multipart_encryption_type(
let create_multipart_output = create_request.send().await?;
let upload_id = create_multipart_output.upload_id().unwrap();
// 上传分片
// Upload parts
let mut completed_parts = Vec::new();
for part_number in 1..=total_parts {
let start = (part_number - 1) * part_size;
@@ -541,7 +541,7 @@ async fn test_multipart_encryption_type(
.part_number(part_number as i32)
.body(aws_sdk_s3::primitives::ByteStream::from(part_data.to_vec()));
// SSE-C 需要在每个 UploadPart 请求中包含密钥
// SSE-C requires the key on each UploadPart request
if matches!(encryption_type, EncryptionType::SSEC) {
upload_request = upload_request
.sse_customer_algorithm("AES256")
@@ -558,10 +558,10 @@ async fn test_multipart_encryption_type(
.build(),
);
debug!("{:?} 分片 {} 上传完成", encryption_type, part_number);
debug!("{:?} part {} uploaded", encryption_type, part_number);
}
// 完成分片上传
// Complete the multipart upload
let completed_multipart_upload = aws_sdk_s3::types::CompletedMultipartUpload::builder()
.set_parts(Some(completed_parts))
.build();
@@ -575,10 +575,10 @@ async fn test_multipart_encryption_type(
.send()
.await?;
// 下载并验证
// Download and verify
let mut get_request = s3_client.get_object().bucket(bucket).key(object_key);
// SSE-C 需要在 GET 请求中包含密钥
// SSE-C requires the key on GET requests
if matches!(encryption_type, EncryptionType::SSEC) {
get_request = get_request
.sse_customer_algorithm("AES256")
@@ -588,7 +588,7 @@ async fn test_multipart_encryption_type(
let get_response = get_request.send().await?;
// 验证加密头
// Verify encryption headers
match encryption_type {
EncryptionType::SSEKMS => {
assert_eq!(
@@ -601,11 +601,11 @@ async fn test_multipart_encryption_type(
}
}
// 验证数据完整性
// Verify data integrity
let downloaded_data = get_response.body.collect().await?.into_bytes();
assert_eq!(downloaded_data.len(), total_size);
assert_eq!(&downloaded_data[..], &test_data[..]);
info!("✅ {:?} 分片上传测试通过", encryption_type);
info!("✅ {:?} multipart upload test passed", encryption_type);
Ok(())
}
+21 -21
View File
@@ -346,7 +346,7 @@ impl KMSTestSuite {
/// Run the complete test suite
pub async fn run_test_suite(&self) -> Vec<TestResult> {
init_logging();
info!("🚀 开始KMS统一测试套件");
info!("🚀 Starting unified KMS test suite");
let start_time = Instant::now();
let mut results = Vec::new();
@@ -359,17 +359,17 @@ impl KMSTestSuite {
.filter(|test| !self.config.include_critical_only || test.is_critical)
.collect();
info!("📊 测试计划: {} 个测试将被执行", tests_to_run.len());
info!("📊 Test plan: {} test(s) scheduled", tests_to_run.len());
for (i, test) in tests_to_run.iter().enumerate() {
info!(" {}. {} ({})", i + 1, test.name, test.category.as_str());
}
// Execute tests
for (i, test_def) in tests_to_run.iter().enumerate() {
info!("🧪 执行测试 {}/{}: {}", i + 1, tests_to_run.len(), test_def.name);
info!(" 📝 描述: {}", test_def.description);
info!(" 🏷️ 分类: {}", test_def.category.as_str());
info!(" ⏱️ 预计时间: {:?}", test_def.estimated_duration);
info!("🧪 Running test {}/{}: {}", i + 1, tests_to_run.len(), test_def.name);
info!(" 📝 Description: {}", test_def.description);
info!(" 🏷️ Category: {}", test_def.category.as_str());
info!(" ⏱️ Estimated duration: {:?}", test_def.estimated_duration);
let test_start = Instant::now();
let result = self.run_single_test(test_def).await;
@@ -377,11 +377,11 @@ impl KMSTestSuite {
match result {
Ok(_) => {
info!("测试通过: {} ({:.2}s)", test_def.name, test_duration.as_secs_f64());
info!("Test passed: {} ({:.2}s)", test_def.name, test_duration.as_secs_f64());
results.push(TestResult::success(test_def.name.clone(), test_def.category.clone(), test_duration));
}
Err(e) => {
error!("测试失败: {} ({:.2}s): {}", test_def.name, test_duration.as_secs_f64(), e);
error!("Test failed: {} ({:.2}s): {}", test_def.name, test_duration.as_secs_f64(), e);
results.push(TestResult::failure(
test_def.name.clone(),
test_def.category.clone(),
@@ -393,7 +393,7 @@ impl KMSTestSuite {
// Add delay between tests to avoid resource conflicts
if i < tests_to_run.len() - 1 {
debug!("⏸️ 等待2秒后执行下一个测试...");
debug!("⏸️ Waiting two seconds before the next test...");
sleep(Duration::from_secs(2)).await;
}
}
@@ -408,22 +408,22 @@ impl KMSTestSuite {
async fn run_single_test(&self, test_def: &TestDefinition) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
// This is a placeholder for test dispatch logic
// In a real implementation, this would dispatch to actual test functions
warn!("⚠️ 测试函数 '{}' 在统一运行器中尚未实现,跳过", test_def.name);
warn!("⚠️ Test '{}' is not implemented in the unified runner; skipping", test_def.name);
Ok(())
}
/// Print comprehensive test summary
fn print_test_summary(&self, results: &[TestResult], total_duration: Duration) {
info!("📊 KMS测试套件总结");
info!("⏱️ 总执行时间: {:.2}秒", total_duration.as_secs_f64());
info!("📈 总测试数量: {}", results.len());
info!("📊 KMS test suite summary");
info!("⏱️ Total duration: {:.2} seconds", total_duration.as_secs_f64());
info!("📈 Total tests: {}", results.len());
let passed = results.iter().filter(|r| r.success).count();
let failed = results.iter().filter(|r| !r.success).count();
info!("通过: {}", passed);
info!("失败: {}", failed);
info!("📊 成功率: {:.1}%", (passed as f64 / results.len() as f64) * 100.0);
info!("Passed: {}", passed);
info!("Failed: {}", failed);
info!("📊 Success rate: {:.1}%", (passed as f64 / results.len() as f64) * 100.0);
// Summary by category
let mut category_summary: std::collections::HashMap<TestCategory, (usize, usize)> = std::collections::HashMap::new();
@@ -435,7 +435,7 @@ impl KMSTestSuite {
}
}
info!("📊 分类汇总:");
info!("📊 Category summary:");
for (category, (total, passed_count)) in category_summary {
info!(
" 🏷️ {}: {}/{} ({:.1}%)",
@@ -448,7 +448,7 @@ impl KMSTestSuite {
// List failed tests
if failed > 0 {
warn!("失败的测试:");
warn!("Failing tests:");
for result in results.iter().filter(|r| !r.success) {
warn!(
" - {}: {}",
@@ -479,7 +479,7 @@ async fn test_kms_critical_suite() -> Result<(), Box<dyn std::error::Error + Sen
return Err(format!("Critical test suite failed: {failed_count} tests failed").into());
}
info!("所有关键测试通过");
info!("All critical tests passed");
Ok(())
}
@@ -494,13 +494,13 @@ async fn test_kms_full_suite() -> Result<(), Box<dyn std::error::Error + Send +
let failed_count = results.iter().filter(|r| !r.success).count();
let success_rate = ((total_tests - failed_count) as f64 / total_tests as f64) * 100.0;
info!("📊 完整测试套件结果: {:.1}% 成功率", success_rate);
info!("📊 Full suite success rate: {:.1}%", success_rate);
// Allow up to 10% failure rate for non-critical tests
if success_rate < 90.0 {
return Err(format!("Test suite success rate too low: {success_rate:.1}%").into());
}
info!("完整测试套件通过");
info!("Full test suite succeeded");
Ok(())
}