mirror of
https://github.com/Grace-Solutions/PSMinIO.git
synced 2026-07-26 06:48:13 +00:00
Create comprehensive PowerShell threading documentation and patterns
� PERMANENT THREADING KNOWLEDGE BASE:
✅ CREATED COMPREHENSIVE DOCUMENTATION:
• docs/POWERSHELL-THREADING-RULES.md - Complete threading guide
• Enhanced ThreadSafeProgressCollector with usage examples
• Added to project structure documentation
• Saved to AI memory for future reference
� CRITICAL RULES DOCUMENTED:
• PowerShell cmdlets can ONLY call Write-* methods from main thread
• Background threads must QUEUE updates, never call Write-* directly
• ProcessQueuedUpdates() must ONLY be called from main thread
• Common mistakes and error symptoms clearly identified
� DESIGN PATTERNS PROVIDED:
• Periodic processing pattern (every 1 second)
• Completion-based processing pattern
• Manual processing points pattern
• Correct Task.WaitAll usage with timeouts
� IMPLEMENTATION CHECKLIST:
• Pre-implementation checklist for new cmdlets
• Debugging tips and error identification
• Code examples for correct and incorrect patterns
• Testing guidelines for threading compliance
� PREVENTS FUTURE REGRESSIONS:
• Clear documentation of the 'golden rule'
• Examples of common threading violations
• Patterns for all background operation types
• Reference for all future PowerShell module development
This ensures we never repeat the same threading mistakes!
This commit is contained in:
PSMinIO Developer
@@ -0,0 +1,175 @@
|
||||
# PowerShell Cmdlet Threading Rules and Design Patterns
|
||||
|
||||
## 🚨 CRITICAL THREADING RULE
|
||||
|
||||
**PowerShell cmdlets can ONLY call Write-Progress, Write-Verbose, Write-Object, Write-Error from the main cmdlet thread.**
|
||||
|
||||
**❌ NEVER call these methods from:**
|
||||
- Background threads (`Task.Run`, `Task.Factory.StartNew`)
|
||||
- Async callbacks (`async/await` continuations)
|
||||
- Timer callbacks
|
||||
- Event handlers from other threads
|
||||
- Any thread other than the main cmdlet thread
|
||||
|
||||
**✅ ONLY call these methods from:**
|
||||
- `BeginProcessing()` override
|
||||
- `ProcessRecord()` override
|
||||
- `EndProcessing()` override
|
||||
- Main cmdlet thread execution
|
||||
|
||||
## 🔧 DESIGN PATTERN: ThreadSafeProgressCollector
|
||||
|
||||
### Problem
|
||||
When performing background operations (file uploads, downloads, processing), you need to report progress from background threads, but PowerShell doesn't allow direct calls to Write-Progress from those threads.
|
||||
|
||||
### Solution
|
||||
Use the **ThreadSafeProgressCollector** pattern:
|
||||
|
||||
```csharp
|
||||
// Background thread - QUEUE updates (thread-safe)
|
||||
_progressCollector.QueueProgressUpdate(activityId, "Processing", "Status", percentage);
|
||||
_progressCollector.QueueVerboseMessage("Processing file: {0}", fileName);
|
||||
|
||||
// Main thread - PROCESS queued updates
|
||||
_progressCollector.ProcessQueuedUpdates(); // Only call from main thread!
|
||||
```
|
||||
|
||||
### Implementation Pattern
|
||||
|
||||
```csharp
|
||||
public class MyLongRunningCmdlet : PSCmdlet
|
||||
{
|
||||
private ThreadSafeProgressCollector _progressCollector;
|
||||
|
||||
protected override void BeginProcessing()
|
||||
{
|
||||
_progressCollector = new ThreadSafeProgressCollector(this);
|
||||
}
|
||||
|
||||
protected override void ProcessRecord()
|
||||
{
|
||||
// Start background work
|
||||
var tasks = new List<Task>();
|
||||
|
||||
for (int i = 0; i < workItems.Count; i++)
|
||||
{
|
||||
var task = Task.Run(() =>
|
||||
{
|
||||
// ✅ CORRECT: Queue from background thread
|
||||
_progressCollector.QueueProgressUpdate(1, "Processing", $"Item {i}", progress);
|
||||
_progressCollector.QueueVerboseMessage("Processing item {0}", i);
|
||||
|
||||
// ❌ WRONG: Never call directly from background thread
|
||||
// WriteProgress(...); // This will cause threading errors!
|
||||
// WriteVerbose(...); // This will cause threading errors!
|
||||
});
|
||||
tasks.Add(task);
|
||||
}
|
||||
|
||||
// ✅ CORRECT: Process updates from main thread with periodic updates
|
||||
while (!Task.WaitAll(tasks.ToArray(), 1000)) // 1 second intervals
|
||||
{
|
||||
_progressCollector.ProcessQueuedUpdates(); // Safe on main thread
|
||||
}
|
||||
|
||||
// Final processing
|
||||
_progressCollector.ProcessQueuedUpdates();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🚨 COMMON MISTAKES TO AVOID
|
||||
|
||||
### ❌ Mistake 1: Direct calls from background threads
|
||||
```csharp
|
||||
Task.Run(() =>
|
||||
{
|
||||
WriteProgress(...); // THREADING ERROR!
|
||||
WriteVerbose(...); // THREADING ERROR!
|
||||
});
|
||||
```
|
||||
|
||||
### ❌ Mistake 2: ProcessQueuedUpdates from background threads
|
||||
```csharp
|
||||
Task.Run(() =>
|
||||
{
|
||||
_progressCollector.QueueProgressUpdate(...);
|
||||
_progressCollector.ProcessQueuedUpdates(); // THREADING ERROR!
|
||||
});
|
||||
```
|
||||
|
||||
### ❌ Mistake 3: Async/await callbacks
|
||||
```csharp
|
||||
await SomeAsyncOperation().ContinueWith(task =>
|
||||
{
|
||||
WriteProgress(...); // THREADING ERROR!
|
||||
});
|
||||
```
|
||||
|
||||
## ✅ CORRECT PATTERNS
|
||||
|
||||
### Pattern 1: Periodic Processing
|
||||
```csharp
|
||||
var tasks = StartBackgroundTasks();
|
||||
while (!Task.WaitAll(tasks, 1000))
|
||||
{
|
||||
_progressCollector.ProcessQueuedUpdates(); // Every 1 second
|
||||
}
|
||||
_progressCollector.ProcessQueuedUpdates(); // Final update
|
||||
```
|
||||
|
||||
### Pattern 2: Completion-based Processing
|
||||
```csharp
|
||||
var tasks = StartBackgroundTasks();
|
||||
Task.WaitAll(tasks);
|
||||
_progressCollector.ProcessQueuedUpdates(); // After completion
|
||||
```
|
||||
|
||||
### Pattern 3: Manual Processing Points
|
||||
```csharp
|
||||
foreach (var item in items)
|
||||
{
|
||||
ProcessItemInBackground(item); // Queues updates
|
||||
_progressCollector.ProcessQueuedUpdates(); // Process after each item
|
||||
}
|
||||
```
|
||||
|
||||
## 🎯 ERROR SYMPTOMS
|
||||
|
||||
If you violate these rules, you'll see errors like:
|
||||
- "The WriteObject and WriteError methods cannot be called from outside the overrides..."
|
||||
- "WriteProgress can only be called from within the same thread"
|
||||
- Cmdlet failures with threading exceptions
|
||||
- Progress bars not updating or appearing
|
||||
|
||||
## 📋 CHECKLIST FOR NEW CMDLETS
|
||||
|
||||
Before implementing background operations:
|
||||
|
||||
- [ ] Are you using ThreadSafeProgressCollector?
|
||||
- [ ] Are all Write* calls from main thread only?
|
||||
- [ ] Are you calling ProcessQueuedUpdates() only from main thread?
|
||||
- [ ] Do you have periodic progress processing for long operations?
|
||||
- [ ] Have you tested with verbose output enabled?
|
||||
- [ ] Have you tested with progress bars enabled?
|
||||
|
||||
## 🔧 DEBUGGING TIPS
|
||||
|
||||
1. **Enable verbose logging** to see threading violations
|
||||
2. **Test with progress bars** - they're most sensitive to threading issues
|
||||
3. **Use Task.WaitAll with timeouts** for periodic processing
|
||||
4. **Never call ProcessQueuedUpdates from callbacks**
|
||||
5. **Queue everything from background threads, process from main thread**
|
||||
|
||||
## 📚 RELATED PATTERNS
|
||||
|
||||
- **File Upload/Download**: Use periodic processing during Task.WaitAll
|
||||
- **Multipart Operations**: Queue from parallel tasks, process periodically
|
||||
- **Long-running Operations**: Process updates every 1-2 seconds
|
||||
- **Collection Processing**: Process after each item or batch
|
||||
|
||||
## 🎯 REMEMBER
|
||||
|
||||
**The golden rule: Background threads QUEUE, main thread PROCESSES.**
|
||||
|
||||
This pattern ensures PowerShell cmdlet compliance while providing real-time progress updates to users.
|
||||
@@ -42,6 +42,7 @@ PSMinIO/
|
||||
│ ├── USAGE.md # Comprehensive usage guide
|
||||
│ ├── RELEASE-NOTES.md # Release notes
|
||||
│ ├── POWERSHELL-GALLERY-RELEASE.md # Gallery release guide
|
||||
│ ├── POWERSHELL-THREADING-RULES.md # Critical PowerShell threading patterns
|
||||
│ └── PROJECT-STRUCTURE.md # This file
|
||||
│
|
||||
├── Artifacts/ # Build artifacts
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
# Check what parameters are available for New-MinIOObject
|
||||
Import-Module ".\Module\PSMinIO\PSMinIO.psd1" -Force
|
||||
|
||||
Write-Host "=== New-MinIOObject Parameter Information ===" -ForegroundColor Cyan
|
||||
|
||||
# Get command info
|
||||
$cmdInfo = Get-Command New-MinIOObject
|
||||
|
||||
Write-Host "`nParameter Sets:" -ForegroundColor Yellow
|
||||
foreach ($paramSet in $cmdInfo.ParameterSets) {
|
||||
Write-Host " $($paramSet.Name):" -ForegroundColor Green
|
||||
foreach ($param in $paramSet.Parameters) {
|
||||
if ($param.Name -notin @('Verbose', 'Debug', 'ErrorAction', 'WarningAction', 'InformationAction', 'ErrorVariable', 'WarningVariable', 'InformationVariable', 'OutVariable', 'OutBuffer', 'PipelineVariable', 'WhatIf', 'Confirm')) {
|
||||
$mandatory = if ($param.IsMandatory) { " (Mandatory)" } else { "" }
|
||||
$aliases = if ($param.Aliases.Count -gt 0) { " [Aliases: $($param.Aliases -join ', ')]" } else { "" }
|
||||
Write-Host " - $($param.Name)$mandatory$aliases" -ForegroundColor White
|
||||
}
|
||||
}
|
||||
Write-Host ""
|
||||
}
|
||||
|
||||
Write-Host "All Parameters:" -ForegroundColor Yellow
|
||||
foreach ($param in $cmdInfo.Parameters.Values) {
|
||||
if ($param.Name -notin @('Verbose', 'Debug', 'ErrorAction', 'WarningAction', 'InformationAction', 'ErrorVariable', 'WarningVariable', 'InformationVariable', 'OutVariable', 'OutBuffer', 'PipelineVariable', 'WhatIf', 'Confirm')) {
|
||||
$aliases = if ($param.Aliases.Count -gt 0) { " [Aliases: $($param.Aliases -join ', ')]" } else { "" }
|
||||
$paramSets = $param.ParameterSets.Keys -join ', '
|
||||
Write-Host " $($param.Name)$aliases - Sets: $paramSets" -ForegroundColor White
|
||||
}
|
||||
}
|
||||
@@ -1,156 +0,0 @@
|
||||
# Test script for uploading 50 test files to demonstrate enhanced upload functionality
|
||||
# This tests FileInfo[] support, multi-layer progress tracking, and BucketDirectory features
|
||||
|
||||
param(
|
||||
[string]$TestBucketName = "psminiotest-50files-$(Get-Date -Format 'yyyyMMdd-HHmmss')",
|
||||
[string]$TestDirectory = "TestFiles50",
|
||||
[switch]$Cleanup,
|
||||
[switch]$Verbose
|
||||
)
|
||||
|
||||
# Set verbose preference if requested
|
||||
if ($Verbose) {
|
||||
$VerbosePreference = 'Continue'
|
||||
}
|
||||
|
||||
Write-Host "=== PSMinIO 50-File Upload Test ===" -ForegroundColor Cyan
|
||||
Write-Host "Testing enhanced upload functionality with multi-layer progress tracking" -ForegroundColor Green
|
||||
|
||||
try {
|
||||
# Import the module
|
||||
Write-Host "`n1. Loading PSMinIO module..." -ForegroundColor Yellow
|
||||
Import-Module ".\Module\PSMinIO\PSMinIO.psd1" -Force
|
||||
|
||||
# Connect to MinIO
|
||||
Write-Host "2. Connecting to MinIO..." -ForegroundColor Yellow
|
||||
Connect-MinIO -Endpoint "https://api.s3.gracesolution.info" -AccessKey "T34Wg85SAwezUa3sk3m4" -SecretKey "PxEmnbQoQJTJsDocSEV6mSSscDpJMiJCayPv93xe"
|
||||
Write-Host "✅ Connected successfully!" -ForegroundColor Green
|
||||
|
||||
# Create test directory and files
|
||||
Write-Host "`n3. Creating test files..." -ForegroundColor Yellow
|
||||
if (Test-Path $TestDirectory) {
|
||||
Remove-Item $TestDirectory -Recurse -Force
|
||||
}
|
||||
New-Item -ItemType Directory -Path $TestDirectory -Force | Out-Null
|
||||
|
||||
# Create 50 test files with varying sizes and content
|
||||
$testFiles = @()
|
||||
for ($i = 1; $i -le 50; $i++) {
|
||||
$fileName = "testfile-{0:D3}.txt" -f $i
|
||||
$filePath = Join-Path $TestDirectory $fileName
|
||||
|
||||
# Create files with different sizes (1KB to 100KB)
|
||||
$fileSize = Get-Random -Minimum 1024 -Maximum 102400
|
||||
$content = "Test file $i`n" + ("X" * ($fileSize - 20))
|
||||
|
||||
Set-Content -Path $filePath -Value $content -NoNewline
|
||||
$testFiles += Get-Item $filePath
|
||||
}
|
||||
|
||||
Write-Host "✅ Created 50 test files (total size: $((($testFiles | Measure-Object Length -Sum).Sum / 1MB).ToString('F2')) MB)" -ForegroundColor Green
|
||||
|
||||
# Create test bucket
|
||||
Write-Host "`n4. Creating test bucket: $TestBucketName" -ForegroundColor Yellow
|
||||
New-MinIOBucket -BucketName $TestBucketName
|
||||
Write-Host "✅ Bucket created successfully!" -ForegroundColor Green
|
||||
|
||||
# Test 1: Upload all files using FileInfo[] parameter
|
||||
Write-Host "`n5. Test 1: Uploading 50 files using FileInfo[] parameter..." -ForegroundColor Yellow
|
||||
Write-Host " This will demonstrate multi-layer progress tracking:" -ForegroundColor Cyan
|
||||
Write-Host " • Layer 1: Collection progress (overall files)" -ForegroundColor Cyan
|
||||
Write-Host " • Layer 2: File progress (current file)" -ForegroundColor Cyan
|
||||
Write-Host " • Layer 3: Transfer progress (bytes)" -ForegroundColor Cyan
|
||||
|
||||
$uploadStart = Get-Date
|
||||
$uploadResults = New-MinIOObject -BucketName $TestBucketName -Files $testFiles -BucketDirectory "batch-upload" -PassThru -Verbose
|
||||
$uploadDuration = (Get-Date) - $uploadStart
|
||||
|
||||
Write-Host "✅ Upload completed in $($uploadDuration.TotalSeconds.ToString('F2')) seconds!" -ForegroundColor Green
|
||||
Write-Host " Average speed: $(($uploadResults | Measure-Object TotalSize -Sum).Sum / $uploadDuration.TotalSeconds / 1MB | ForEach-Object { $_.ToString('F2') }) MB/s" -ForegroundColor Green
|
||||
Write-Host " Files uploaded: $($uploadResults.Count)" -ForegroundColor Green
|
||||
|
||||
# Test 2: Upload using Directory parameter with BucketDirectory
|
||||
Write-Host "`n6. Test 2: Uploading directory with BucketDirectory parameter..." -ForegroundColor Yellow
|
||||
|
||||
$dirUploadStart = Get-Date
|
||||
$dirUploadResults = New-MinIOObject -BucketName $TestBucketName -Directory (Get-Item $TestDirectory) -BucketDirectory "directory-upload/nested/structure" -PassThru -Verbose
|
||||
$dirUploadDuration = (Get-Date) - $dirUploadStart
|
||||
|
||||
Write-Host "✅ Directory upload completed in $($dirUploadDuration.TotalSeconds.ToString('F2')) seconds!" -ForegroundColor Green
|
||||
Write-Host " Files uploaded: $($dirUploadResults.Count)" -ForegroundColor Green
|
||||
|
||||
# Test 3: Verify uploads by listing objects
|
||||
Write-Host "`n7. Verifying uploads..." -ForegroundColor Yellow
|
||||
$allObjects = Get-MinIOObject -BucketName $TestBucketName
|
||||
|
||||
$batchObjects = $allObjects | Where-Object { $_.Name -like "batch-upload/*" }
|
||||
$dirObjects = $allObjects | Where-Object { $_.Name -like "directory-upload/*" }
|
||||
|
||||
Write-Host "✅ Verification complete:" -ForegroundColor Green
|
||||
Write-Host " Total objects in bucket: $($allObjects.Count)" -ForegroundColor Green
|
||||
Write-Host " Batch upload objects: $($batchObjects.Count)" -ForegroundColor Green
|
||||
Write-Host " Directory upload objects: $($dirObjects.Count)" -ForegroundColor Green
|
||||
|
||||
# Test 4: Test with filters (create subdirectories first)
|
||||
Write-Host "`n8. Test 3: Testing directory upload with filters..." -ForegroundColor Yellow
|
||||
|
||||
# Create subdirectories with different file types
|
||||
$subDir1 = Join-Path $TestDirectory "SubDir1"
|
||||
$subDir2 = Join-Path $TestDirectory "SubDir2"
|
||||
New-Item -ItemType Directory -Path $subDir1 -Force | Out-Null
|
||||
New-Item -ItemType Directory -Path $subDir2 -Force | Out-Null
|
||||
|
||||
# Create some .log and .json files
|
||||
for ($i = 1; $i -le 5; $i++) {
|
||||
Set-Content -Path (Join-Path $subDir1 "logfile$i.log") -Value "Log entry $i"
|
||||
Set-Content -Path (Join-Path $subDir2 "config$i.json") -Value "{`"test`": $i}"
|
||||
}
|
||||
|
||||
# Upload only .log files using inclusion filter
|
||||
$filterStart = Get-Date
|
||||
$filterResults = New-MinIOObject -BucketName $TestBucketName -Directory (Get-Item $TestDirectory) -Recursive -InclusionFilter { $_.Extension -eq ".log" } -BucketDirectory "filtered-upload" -PassThru -Verbose
|
||||
$filterDuration = (Get-Date) - $filterStart
|
||||
|
||||
Write-Host "✅ Filtered upload completed in $($filterDuration.TotalSeconds.ToString('F2')) seconds!" -ForegroundColor Green
|
||||
Write-Host " Log files uploaded: $($filterResults.Count)" -ForegroundColor Green
|
||||
|
||||
# Summary
|
||||
Write-Host "`n=== TEST SUMMARY ===" -ForegroundColor Cyan
|
||||
Write-Host "✅ All tests completed successfully!" -ForegroundColor Green
|
||||
Write-Host "Features tested:" -ForegroundColor Yellow
|
||||
Write-Host " • FileInfo[] parameter with 50 files" -ForegroundColor White
|
||||
Write-Host " • Multi-layer progress tracking (3 layers)" -ForegroundColor White
|
||||
Write-Host " • BucketDirectory parameter for nested structures" -ForegroundColor White
|
||||
Write-Host " • Directory parameter with recursive upload" -ForegroundColor White
|
||||
Write-Host " • InclusionFilter for selective file upload" -ForegroundColor White
|
||||
Write-Host " • PassThru parameter for upload results" -ForegroundColor White
|
||||
Write-Host " • Thread-safe progress reporting" -ForegroundColor White
|
||||
|
||||
Write-Host "`nPerformance metrics:" -ForegroundColor Yellow
|
||||
Write-Host " • Batch upload: $($uploadDuration.TotalSeconds.ToString('F2'))s for $($uploadResults.Count) files" -ForegroundColor White
|
||||
Write-Host " • Directory upload: $($dirUploadDuration.TotalSeconds.ToString('F2'))s for $($dirUploadResults.Count) files" -ForegroundColor White
|
||||
Write-Host " • Filtered upload: $($filterDuration.TotalSeconds.ToString('F2'))s for $($filterResults.Count) files" -ForegroundColor White
|
||||
|
||||
# Cleanup option
|
||||
if ($Cleanup) {
|
||||
Write-Host "`n9. Cleaning up..." -ForegroundColor Yellow
|
||||
Remove-MinIOBucket -BucketName $TestBucketName -Force
|
||||
Remove-Item $TestDirectory -Recurse -Force
|
||||
Write-Host "✅ Cleanup completed!" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host "`nTest bucket '$TestBucketName' and files preserved for inspection." -ForegroundColor Cyan
|
||||
Write-Host "Use -Cleanup parameter to automatically clean up test resources." -ForegroundColor Cyan
|
||||
}
|
||||
|
||||
} catch {
|
||||
Write-Host "❌ Test failed: $($_.Exception.Message)" -ForegroundColor Red
|
||||
Write-Host "Stack trace: $($_.ScriptStackTrace)" -ForegroundColor Red
|
||||
} finally {
|
||||
# Clean up test files if they exist
|
||||
if (Test-Path $TestDirectory) {
|
||||
Write-Host "`nCleaning up local test files..." -ForegroundColor Yellow
|
||||
Remove-Item $TestDirectory -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host "`n=== Test Complete ===" -ForegroundColor Cyan
|
||||
@@ -1,117 +0,0 @@
|
||||
# Test script for current upload functionality (single file version)
|
||||
param(
|
||||
[string]$TestBucketName = "psminiotest-current-$(Get-Date -Format 'yyyyMMdd-HHmmss')",
|
||||
[switch]$Cleanup,
|
||||
[switch]$Verbose
|
||||
)
|
||||
|
||||
if ($Verbose) {
|
||||
$VerbosePreference = 'Continue'
|
||||
}
|
||||
|
||||
Write-Host "=== PSMinIO Current Upload Test ===" -ForegroundColor Cyan
|
||||
Write-Host "Testing current upload functionality" -ForegroundColor Green
|
||||
|
||||
try {
|
||||
# Import the module
|
||||
Write-Host "`n1. Loading PSMinIO module..." -ForegroundColor Yellow
|
||||
Import-Module ".\Module\PSMinIO\PSMinIO.psd1" -Force
|
||||
|
||||
# Connect to MinIO
|
||||
Write-Host "2. Connecting to MinIO..." -ForegroundColor Yellow
|
||||
Connect-MinIO -Endpoint "https://api.s3.gracesolution.info" -AccessKey "T34Wg85SAwezUa3sk3m4" -SecretKey "PxEmnbQoQJTJsDocSEV6mSSscDpJMiJCayPv93xe"
|
||||
Write-Host "✅ Connected successfully!" -ForegroundColor Green
|
||||
|
||||
# Create test files
|
||||
Write-Host "`n3. Creating test files..." -ForegroundColor Yellow
|
||||
$testDir = "TestFilesCurrent"
|
||||
if (Test-Path $testDir) {
|
||||
Remove-Item $testDir -Recurse -Force
|
||||
}
|
||||
New-Item -ItemType Directory -Path $testDir -Force | Out-Null
|
||||
|
||||
# Create 10 test files
|
||||
$testFiles = @()
|
||||
for ($i = 1; $i -le 10; $i++) {
|
||||
$fileName = "testfile-{0:D3}.txt" -f $i
|
||||
$filePath = Join-Path $testDir $fileName
|
||||
$content = "Test file $i - $(Get-Date)`n" + ("Content line $i`n" * 10)
|
||||
Set-Content -Path $filePath -Value $content
|
||||
$testFiles += Get-Item $filePath
|
||||
}
|
||||
|
||||
Write-Host "✅ Created 10 test files" -ForegroundColor Green
|
||||
|
||||
# Create test bucket
|
||||
Write-Host "`n4. Creating test bucket: $TestBucketName" -ForegroundColor Yellow
|
||||
New-MinIOBucket -BucketName $TestBucketName
|
||||
Write-Host "✅ Bucket created successfully!" -ForegroundColor Green
|
||||
|
||||
# Test uploading files one by one (current functionality)
|
||||
Write-Host "`n5. Uploading files individually..." -ForegroundColor Yellow
|
||||
$uploadResults = @()
|
||||
$uploadStart = Get-Date
|
||||
|
||||
foreach ($file in $testFiles) {
|
||||
Write-Host " Uploading: $($file.Name)" -ForegroundColor Cyan
|
||||
$result = New-MinIOObject -BucketName $TestBucketName -File $file -BucketDirectory "individual-uploads" -PassThru -Verbose
|
||||
$uploadResults += $result
|
||||
}
|
||||
|
||||
$uploadDuration = (Get-Date) - $uploadStart
|
||||
Write-Host "✅ Upload completed in $($uploadDuration.TotalSeconds.ToString('F2')) seconds!" -ForegroundColor Green
|
||||
Write-Host " Files uploaded: $($uploadResults.Count)" -ForegroundColor Green
|
||||
|
||||
# Verify uploads
|
||||
Write-Host "`n6. Verifying uploads..." -ForegroundColor Yellow
|
||||
$objects = Get-MinIOObject -BucketName $TestBucketName
|
||||
Write-Host "✅ Found $($objects.Count) objects in bucket" -ForegroundColor Green
|
||||
|
||||
# Test with different bucket directories
|
||||
Write-Host "`n7. Testing nested bucket directories..." -ForegroundColor Yellow
|
||||
$nestedResults = @()
|
||||
foreach ($file in $testFiles[0..2]) { # Upload first 3 files to nested structure
|
||||
$result = New-MinIOObject -BucketName $TestBucketName -File $file -BucketDirectory "level1/level2/level3" -PassThru -Verbose
|
||||
$nestedResults += $result
|
||||
}
|
||||
Write-Host "✅ Nested directory upload completed: $($nestedResults.Count) files" -ForegroundColor Green
|
||||
|
||||
# Final verification
|
||||
Write-Host "`n8. Final verification..." -ForegroundColor Yellow
|
||||
$allObjects = Get-MinIOObject -BucketName $TestBucketName
|
||||
$individualObjects = $allObjects | Where-Object { $_.Name -like "individual-uploads/*" }
|
||||
$nestedObjects = $allObjects | Where-Object { $_.Name -like "level1/level2/level3/*" }
|
||||
|
||||
Write-Host "✅ Final verification complete:" -ForegroundColor Green
|
||||
Write-Host " Total objects: $($allObjects.Count)" -ForegroundColor Green
|
||||
Write-Host " Individual uploads: $($individualObjects.Count)" -ForegroundColor Green
|
||||
Write-Host " Nested uploads: $($nestedObjects.Count)" -ForegroundColor Green
|
||||
|
||||
# Summary
|
||||
Write-Host "`n=== TEST SUMMARY ===" -ForegroundColor Cyan
|
||||
Write-Host "✅ Current functionality test completed!" -ForegroundColor Green
|
||||
Write-Host "Features tested:" -ForegroundColor Yellow
|
||||
Write-Host " • Single file upload with File parameter" -ForegroundColor White
|
||||
Write-Host " • BucketDirectory parameter for nested structures" -ForegroundColor White
|
||||
Write-Host " • PassThru parameter for upload results" -ForegroundColor White
|
||||
Write-Host " • Multiple individual uploads" -ForegroundColor White
|
||||
Write-Host " • Nested directory structure creation" -ForegroundColor White
|
||||
|
||||
Write-Host "`nNote: Enhanced features (FileInfo[], Directory, Multi-layer progress) require updated DLL" -ForegroundColor Yellow
|
||||
|
||||
if ($Cleanup) {
|
||||
Write-Host "`n9. Cleaning up..." -ForegroundColor Yellow
|
||||
Remove-MinIOBucket -BucketName $TestBucketName -Force
|
||||
Write-Host "✅ Cleanup completed!" -ForegroundColor Green
|
||||
}
|
||||
|
||||
} catch {
|
||||
Write-Host "❌ Test failed: $($_.Exception.Message)" -ForegroundColor Red
|
||||
Write-Host "Stack trace: $($_.ScriptStackTrace)" -ForegroundColor Red
|
||||
} finally {
|
||||
if (Test-Path $testDir) {
|
||||
Remove-Item $testDir -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host "`n=== Test Complete ===" -ForegroundColor Cyan
|
||||
@@ -1,176 +0,0 @@
|
||||
#!/usr/bin/env pwsh
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Final comprehensive test of the new PSMinIO implementation
|
||||
|
||||
.DESCRIPTION
|
||||
Tests all core functionality of the rebuilt PSMinIO module:
|
||||
- Connection management
|
||||
- Bucket operations
|
||||
- Object upload/download with progress
|
||||
- Performance metrics
|
||||
- Error handling
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param()
|
||||
|
||||
# Import the module
|
||||
Write-Output "=== PSMinIO Final Implementation Test ==="
|
||||
Write-Output "Importing PSMinIO module..."
|
||||
Import-Module "$PSScriptRoot\..\Module\PSMinIO\PSMinIO.psd1" -Force
|
||||
|
||||
try {
|
||||
# Test 1: Connection
|
||||
Write-Output "`n1. Testing Connection..."
|
||||
Connect-MinIO -Endpoint "https://api.s3.gracesolution.info" -AccessKey "T34Wg85SAwezUa3sk3m4" -SecretKey "PxEmnbQoQJTJsDocSEV6mSSscDpJMiJCayPv93xe" -TestConnection -Verbose
|
||||
|
||||
# Get the connection from session variable to verify
|
||||
$connection = Get-Variable -Name "MinIOConnection" -ValueOnly -ErrorAction SilentlyContinue
|
||||
if ($connection -and $connection.Status -eq 'Connected') {
|
||||
Write-Output "✅ Connection successful!"
|
||||
Write-Output " Status: $($connection.Status)"
|
||||
Write-Output " Endpoint: $($connection.Configuration.Endpoint)"
|
||||
} else {
|
||||
throw "Connection failed"
|
||||
}
|
||||
|
||||
# Test 2: List Buckets
|
||||
Write-Output "`n2. Testing Bucket Listing..."
|
||||
$buckets = Get-MinIOBucket -Verbose
|
||||
|
||||
if ($buckets) {
|
||||
Write-Output "✅ Found $($buckets.Count) buckets"
|
||||
$buckets | Select-Object Name, CreationDate | Format-Table -AutoSize
|
||||
} else {
|
||||
Write-Output "⚠️ No buckets found"
|
||||
}
|
||||
|
||||
# Test 3: Create Test Bucket
|
||||
Write-Output "`n3. Testing Bucket Creation..."
|
||||
$testBucket = "psminiotest-$(Get-Date -Format 'yyyyMMdd-HHmmss')"
|
||||
$bucketResult = New-MinIOBucket -BucketName $testBucket -PassThru -Verbose
|
||||
|
||||
if ($bucketResult) {
|
||||
Write-Output "✅ Bucket created: $($bucketResult.Name)"
|
||||
} else {
|
||||
throw "Bucket creation failed"
|
||||
}
|
||||
|
||||
# Test 4: Test Bucket Exists
|
||||
Write-Output "`n4. Testing Bucket Existence Check..."
|
||||
$exists = Test-MinIOBucketExists -BucketName $testBucket -Verbose
|
||||
|
||||
if ($exists) {
|
||||
Write-Output "✅ Bucket existence confirmed: $exists"
|
||||
} else {
|
||||
throw "Bucket existence check failed"
|
||||
}
|
||||
|
||||
# Test 5: Create and Upload Test File
|
||||
Write-Output "`n5. Testing File Upload with Progress..."
|
||||
$testFile = Join-Path $env:TEMP "psminiotest-$(Get-Date -Format 'yyyyMMddHHmmss').txt"
|
||||
$testContent = @"
|
||||
PSMinIO Final Test File
|
||||
======================
|
||||
Created: $(Get-Date)
|
||||
Implementation: Custom REST API
|
||||
Features: Real progress reporting, performance metrics
|
||||
Test data: $('A' * 2000)
|
||||
"@
|
||||
|
||||
$testContent | Out-File -FilePath $testFile -Encoding UTF8
|
||||
$fileInfo = Get-Item $testFile
|
||||
Write-Output " Test file created: $($fileInfo.Length) bytes"
|
||||
|
||||
$uploadResult = New-MinIOObject -BucketName $testBucket -File $fileInfo -PassThru -Verbose
|
||||
|
||||
if ($uploadResult -and $uploadResult.Success) {
|
||||
Write-Output "✅ Upload successful!"
|
||||
Write-Output " Object: $($uploadResult.ObjectName)"
|
||||
Write-Output " Size: $($uploadResult.TotalSizeFormatted)"
|
||||
Write-Output " Duration: $($uploadResult.DurationFormatted)"
|
||||
Write-Output " Speed: $($uploadResult.AverageSpeedFormatted)"
|
||||
} else {
|
||||
throw "File upload failed"
|
||||
}
|
||||
|
||||
# Test 6: List Objects
|
||||
Write-Output "`n6. Testing Object Listing..."
|
||||
$objects = Get-MinIOObject -BucketName $testBucket -Verbose
|
||||
|
||||
if ($objects -and $objects.Count -gt 0) {
|
||||
Write-Output "✅ Found $($objects.Count) objects"
|
||||
$objects | Select-Object Name, Size, LastModified | Format-Table -AutoSize
|
||||
} else {
|
||||
throw "Object listing failed"
|
||||
}
|
||||
|
||||
# Test 7: Download File with Progress
|
||||
Write-Output "`n7. Testing File Download with Progress..."
|
||||
$downloadFile = Join-Path $env:TEMP "psminiotest-download-$(Get-Date -Format 'yyyyMMddHHmmss').txt"
|
||||
$downloadResult = Get-MinIOObjectContent -BucketName $testBucket -ObjectName $fileInfo.Name -LocalPath $downloadFile -PassThru -Verbose
|
||||
|
||||
if ($downloadResult -and $downloadResult.Success) {
|
||||
Write-Output "✅ Download successful!"
|
||||
Write-Output " Object: $($downloadResult.ObjectName)"
|
||||
Write-Output " Size: $($downloadResult.TotalSizeFormatted)"
|
||||
Write-Output " Duration: $($downloadResult.DurationFormatted)"
|
||||
Write-Output " Speed: $($downloadResult.AverageSpeedFormatted)"
|
||||
} else {
|
||||
throw "File download failed"
|
||||
}
|
||||
|
||||
# Test 8: Verify Content
|
||||
Write-Output "`n8. Testing Content Verification..."
|
||||
$originalContent = Get-Content $testFile -Raw -ErrorAction SilentlyContinue
|
||||
$downloadedContent = Get-Content $downloadFile -Raw -ErrorAction SilentlyContinue
|
||||
|
||||
if ($originalContent -and $downloadedContent -and ($originalContent -eq $downloadedContent)) {
|
||||
Write-Output "✅ File content verification PASSED!"
|
||||
} else {
|
||||
throw "File content verification FAILED!"
|
||||
}
|
||||
|
||||
# Test Summary
|
||||
Write-Output "`n=== Test Summary ==="
|
||||
Write-Output "✅ All tests PASSED!"
|
||||
Write-Output ""
|
||||
Write-Output "New PSMinIO Implementation Features Verified:"
|
||||
Write-Output " ✓ Custom REST API client (no MinIO SDK dependency)"
|
||||
Write-Output " ✓ Real progress reporting during transfers"
|
||||
Write-Output " ✓ Performance metrics (duration, speed)"
|
||||
Write-Output " ✓ Synchronous operations optimized for PowerShell"
|
||||
Write-Output " ✓ Enhanced error handling and logging"
|
||||
Write-Output " ✓ AWS S3 signature v4 authentication"
|
||||
Write-Output " ✓ Comprehensive cmdlet functionality"
|
||||
Write-Output ""
|
||||
Write-Output "Architecture Benefits:"
|
||||
Write-Output " • No async/await compatibility issues"
|
||||
Write-Output " • Reduced dependencies (removed 8+ DLLs)"
|
||||
Write-Output " • True progress from HTTP streams"
|
||||
Write-Output " • Built-in performance monitoring"
|
||||
Write-Output " • PowerShell-native design patterns"
|
||||
|
||||
} catch {
|
||||
Write-Output "❌ Test failed: $($_.Exception.Message)"
|
||||
Write-Output "Error details: $($_.Exception.ToString())"
|
||||
exit 1
|
||||
} finally {
|
||||
# Cleanup
|
||||
Write-Output "`n=== Cleanup ==="
|
||||
|
||||
# Remove test files
|
||||
if (Test-Path $testFile -ErrorAction SilentlyContinue) {
|
||||
Remove-Item $testFile -Force -ErrorAction SilentlyContinue
|
||||
Write-Output "Removed test file: $testFile"
|
||||
}
|
||||
|
||||
if (Test-Path $downloadFile -ErrorAction SilentlyContinue) {
|
||||
Remove-Item $downloadFile -Force -ErrorAction SilentlyContinue
|
||||
Write-Output "Removed download file: $downloadFile"
|
||||
}
|
||||
|
||||
Write-Output "Note: Test bucket '$testBucket' left for manual cleanup"
|
||||
Write-Output "Cleanup completed."
|
||||
}
|
||||
@@ -1,183 +0,0 @@
|
||||
#!/usr/bin/env pwsh
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Test script for the new PSMinIO implementation with custom REST API
|
||||
|
||||
.DESCRIPTION
|
||||
This script demonstrates the new PSMinIO module capabilities including:
|
||||
- Connection management with custom REST API
|
||||
- Real progress reporting during transfers
|
||||
- Performance metrics in result objects
|
||||
- Enhanced error handling and logging
|
||||
|
||||
.PARAMETER Endpoint
|
||||
MinIO server endpoint (e.g., https://minio.example.com:9000)
|
||||
|
||||
.PARAMETER AccessKey
|
||||
MinIO access key
|
||||
|
||||
.PARAMETER SecretKey
|
||||
MinIO secret key
|
||||
|
||||
.PARAMETER TestBucket
|
||||
Name of test bucket to create (default: psminiotest)
|
||||
|
||||
.EXAMPLE
|
||||
.\Test-NewImplementation.ps1 -Endpoint "https://play.min.io" -AccessKey "Q3AM3UQ867SPQQA43P2F" -SecretKey "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$Endpoint,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$AccessKey,
|
||||
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$SecretKey,
|
||||
|
||||
[Parameter()]
|
||||
[string]$TestBucket = "psminiotest-$(Get-Date -Format 'yyyyMMdd-HHmmss')"
|
||||
)
|
||||
|
||||
# Import the module
|
||||
Write-Host "Importing PSMinIO module..." -ForegroundColor Green
|
||||
Import-Module "$PSScriptRoot\..\Module\PSMinIO\PSMinIO.psd1" -Force
|
||||
|
||||
try {
|
||||
# Test 1: Connection
|
||||
Write-Host "`n=== Test 1: Connection ===" -ForegroundColor Yellow
|
||||
Write-Host "Connecting to MinIO server: $Endpoint" -ForegroundColor Cyan
|
||||
|
||||
$connection = Connect-MinIO -Endpoint $Endpoint -AccessKey $AccessKey -SecretKey $SecretKey -TestConnection -PassThru -Verbose
|
||||
Write-Host "✅ Connection successful!" -ForegroundColor Green
|
||||
Write-Host "Connection Status: $($connection.Status)" -ForegroundColor White
|
||||
|
||||
# Test 2: List Buckets
|
||||
Write-Host "`n=== Test 2: List Buckets ===" -ForegroundColor Yellow
|
||||
Write-Host "Listing existing buckets..." -ForegroundColor Cyan
|
||||
|
||||
$buckets = Get-MinIOBucket -Verbose
|
||||
Write-Host "✅ Found $($buckets.Count) buckets" -ForegroundColor Green
|
||||
|
||||
if ($buckets.Count -gt 0) {
|
||||
Write-Host "First few buckets:" -ForegroundColor White
|
||||
$buckets | Select-Object -First 3 | Format-Table Name, CreationDate, @{Name="Age"; Expression={(Get-Date) - $_.CreationDate}}
|
||||
}
|
||||
|
||||
# Test 3: Create Test Bucket
|
||||
Write-Host "`n=== Test 3: Create Test Bucket ===" -ForegroundColor Yellow
|
||||
Write-Host "Creating test bucket: $TestBucket" -ForegroundColor Cyan
|
||||
|
||||
$bucketResult = New-MinIOBucket -Name $TestBucket -PassThru -Verbose
|
||||
Write-Host "✅ Bucket created successfully!" -ForegroundColor Green
|
||||
Write-Host "Bucket: $($bucketResult.Name)" -ForegroundColor White
|
||||
|
||||
# Test 4: Test Bucket Exists
|
||||
Write-Host "`n=== Test 4: Test Bucket Exists ===" -ForegroundColor Yellow
|
||||
Write-Host "Testing if bucket exists..." -ForegroundColor Cyan
|
||||
|
||||
$exists = Test-MinIOBucketExists -Name $TestBucket -Verbose
|
||||
Write-Host "✅ Bucket exists: $exists" -ForegroundColor Green
|
||||
|
||||
# Test 5: Create Test File
|
||||
Write-Host "`n=== Test 5: Create and Upload Test File ===" -ForegroundColor Yellow
|
||||
$testFile = "$env:TEMP\psminiotest-$(Get-Date -Format 'yyyyMMddHHmmss').txt"
|
||||
$testContent = @"
|
||||
PSMinIO Test File
|
||||
================
|
||||
Created: $(Get-Date)
|
||||
Endpoint: $Endpoint
|
||||
Bucket: $TestBucket
|
||||
|
||||
This is a test file created by the PSMinIO test script.
|
||||
The new implementation uses a custom REST API client for better PowerShell compatibility.
|
||||
|
||||
Features demonstrated:
|
||||
- Real progress reporting
|
||||
- Performance metrics
|
||||
- Enhanced error handling
|
||||
- Synchronous operations optimized for PowerShell
|
||||
|
||||
Test data: $('A' * 100)
|
||||
"@
|
||||
|
||||
Write-Host "Creating test file: $testFile" -ForegroundColor Cyan
|
||||
$testContent | Out-File -FilePath $testFile -Encoding UTF8
|
||||
$fileInfo = Get-Item $testFile
|
||||
Write-Host "✅ Test file created: $($fileInfo.Length) bytes" -ForegroundColor Green
|
||||
|
||||
# Test 6: Upload File
|
||||
Write-Host "`n=== Test 6: Upload File with Progress ===" -ForegroundColor Yellow
|
||||
Write-Host "Uploading test file..." -ForegroundColor Cyan
|
||||
|
||||
$uploadResult = New-MinIOObject -BucketName $TestBucket -File $fileInfo -PassThru -Verbose
|
||||
Write-Host "✅ Upload completed!" -ForegroundColor Green
|
||||
Write-Host "Upload Result:" -ForegroundColor White
|
||||
$uploadResult | Format-List ObjectName, TotalSizeFormatted, DurationFormatted, AverageSpeedFormatted, Success
|
||||
|
||||
# Test 7: List Objects
|
||||
Write-Host "`n=== Test 7: List Objects ===" -ForegroundColor Yellow
|
||||
Write-Host "Listing objects in test bucket..." -ForegroundColor Cyan
|
||||
|
||||
$objects = Get-MinIOObject -BucketName $TestBucket -Verbose
|
||||
Write-Host "✅ Found $($objects.Count) objects" -ForegroundColor Green
|
||||
|
||||
if ($objects.Count -gt 0) {
|
||||
Write-Host "Objects in bucket:" -ForegroundColor White
|
||||
$objects | Format-Table Name, @{Name="Size"; Expression={$_.Size}}, LastModified
|
||||
}
|
||||
|
||||
# Test 8: Download File
|
||||
Write-Host "`n=== Test 8: Download File with Progress ===" -ForegroundColor Yellow
|
||||
$downloadFile = "$env:TEMP\psminiotest-download-$(Get-Date -Format 'yyyyMMddHHmmss').txt"
|
||||
Write-Host "Downloading file to: $downloadFile" -ForegroundColor Cyan
|
||||
|
||||
$downloadResult = Get-MinIOObjectContent -BucketName $TestBucket -ObjectName $fileInfo.Name -LocalPath $downloadFile -PassThru -Verbose
|
||||
Write-Host "✅ Download completed!" -ForegroundColor Green
|
||||
Write-Host "Download Result:" -ForegroundColor White
|
||||
$downloadResult | Format-List ObjectName, TotalSizeFormatted, DurationFormatted, AverageSpeedFormatted, Success
|
||||
|
||||
# Verify download
|
||||
$downloadedContent = Get-Content $downloadFile -Raw
|
||||
$originalContent = Get-Content $testFile -Raw
|
||||
if ($downloadedContent -eq $originalContent) {
|
||||
Write-Host "✅ File content verification passed!" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host "❌ File content verification failed!" -ForegroundColor Red
|
||||
}
|
||||
|
||||
Write-Host "`n=== Test Summary ===" -ForegroundColor Yellow
|
||||
Write-Host "✅ All tests completed successfully!" -ForegroundColor Green
|
||||
Write-Host "New PSMinIO implementation is working correctly with:" -ForegroundColor White
|
||||
Write-Host " - Custom REST API client" -ForegroundColor White
|
||||
Write-Host " - Real progress reporting" -ForegroundColor White
|
||||
Write-Host " - Performance metrics" -ForegroundColor White
|
||||
Write-Host " - Enhanced error handling" -ForegroundColor White
|
||||
Write-Host " - PowerShell-optimized synchronous operations" -ForegroundColor White
|
||||
|
||||
} catch {
|
||||
Write-Host "❌ Test failed: $($_.Exception.Message)" -ForegroundColor Red
|
||||
Write-Host "Stack trace:" -ForegroundColor Red
|
||||
Write-Host $_.ScriptStackTrace -ForegroundColor Red
|
||||
} finally {
|
||||
# Cleanup
|
||||
Write-Host "`n=== Cleanup ===" -ForegroundColor Yellow
|
||||
|
||||
# Remove test files
|
||||
if (Test-Path $testFile) {
|
||||
Remove-Item $testFile -Force
|
||||
Write-Host "Removed test file: $testFile" -ForegroundColor Gray
|
||||
}
|
||||
|
||||
if (Test-Path $downloadFile) {
|
||||
Remove-Item $downloadFile -Force
|
||||
Write-Host "Removed download file: $downloadFile" -ForegroundColor Gray
|
||||
}
|
||||
|
||||
# Note: We're not removing the test bucket to avoid issues with the test environment
|
||||
Write-Host "Note: Test bucket '$TestBucket' was left for manual cleanup" -ForegroundColor Gray
|
||||
|
||||
Write-Host "Cleanup completed." -ForegroundColor Gray
|
||||
}
|
||||
@@ -1,186 +0,0 @@
|
||||
# PSMinIO Comprehensive Test Script
|
||||
# Tests all major functionality against Grace Solution S3 instance
|
||||
|
||||
param(
|
||||
[string]$TestDirectory = "C:\Temp\PSMinIOTest",
|
||||
[string]$TestBucket = "psminiotest-$(Get-Date -Format 'yyyyMMdd-HHmmss')"
|
||||
)
|
||||
|
||||
# Test configuration
|
||||
$S3Url = "https://api.s3.gracesolution.info"
|
||||
$AccessKey = "T34Wg85SAwezUa3sk3m4"
|
||||
$SecretKey = "PxEmnbQoQJTJsDocSEV6mSSscDpJMiJCayPv93xe"
|
||||
|
||||
Write-Verbose "=== PSMinIO Comprehensive Test Suite ==="
|
||||
Write-Verbose "S3 Endpoint: $S3Url"
|
||||
Write-Verbose "Test Bucket: $TestBucket"
|
||||
Write-Verbose "Test Directory: $TestDirectory"
|
||||
|
||||
# Create test directory and files
|
||||
Write-Verbose "Setting up test environment..."
|
||||
if (Test-Path $TestDirectory) {
|
||||
Remove-Item $TestDirectory -Recurse -Force
|
||||
}
|
||||
New-Item -ItemType Directory -Path $TestDirectory -Force | Out-Null
|
||||
|
||||
# Create test files of various sizes
|
||||
$SmallFile = Join-Path $TestDirectory "small-test.txt"
|
||||
$MediumFile = Join-Path $TestDirectory "medium-test.txt"
|
||||
$LargeFile = Join-Path $TestDirectory "large-test.txt"
|
||||
|
||||
"This is a small test file for PSMinIO testing." | Out-File $SmallFile -Encoding UTF8
|
||||
1..1000 | ForEach-Object { "Line $_ - Medium test file content for PSMinIO testing with more data." } | Out-File $MediumFile -Encoding UTF8
|
||||
1..10000 | ForEach-Object { "Line $_ - Large test file content for PSMinIO multipart testing with substantial data to trigger chunked operations." } | Out-File $LargeFile -Encoding UTF8
|
||||
|
||||
$testFiles = Get-ChildItem $TestDirectory
|
||||
Write-Verbose "Created test files: $($testFiles.Count) files, Total size: $([math]::Round(($testFiles | Measure-Object Length -Sum).Sum / 1KB, 2)) KB"
|
||||
|
||||
$testResults = @()
|
||||
|
||||
try {
|
||||
# Test 1: Connection
|
||||
Write-Verbose "Testing connection..."
|
||||
$connection = Connect-MinIO -Url $S3Url -AccessKey $AccessKey -SecretKey $SecretKey -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "Connection"
|
||||
Status = "Success"
|
||||
Details = "Connected to $S3Url"
|
||||
Result = $connection
|
||||
}
|
||||
|
||||
# Test 2: List existing buckets
|
||||
Write-Verbose "Testing bucket listing..."
|
||||
$buckets = Get-MinIOBucket -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "List Buckets"
|
||||
Status = "Success"
|
||||
Details = "Found $($buckets.Count) existing buckets"
|
||||
Result = $buckets
|
||||
}
|
||||
|
||||
# Test 3: Create test bucket
|
||||
Write-Verbose "Testing bucket creation..."
|
||||
$newBucket = New-MinIOBucket -BucketName $TestBucket -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "Create Bucket"
|
||||
Status = "Success"
|
||||
Details = "Created bucket: $($newBucket.Name)"
|
||||
Result = $newBucket
|
||||
}
|
||||
|
||||
# Test 4: Verify bucket exists
|
||||
Write-Verbose "Testing bucket existence check..."
|
||||
$bucketExists = Test-MinIOBucketExists -BucketName $TestBucket -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "Bucket Exists"
|
||||
Status = "Success"
|
||||
Details = "Bucket exists: $bucketExists"
|
||||
Result = $bucketExists
|
||||
}
|
||||
|
||||
# Test 5: Upload small file (single part)
|
||||
Write-Verbose "Testing single file upload..."
|
||||
$uploadResult = New-MinIOObject -BucketName $TestBucket -Files (Get-Item $SmallFile) -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "Single File Upload"
|
||||
Status = "Success"
|
||||
Details = "Uploaded small file: $($uploadResult.Count) objects"
|
||||
Result = $uploadResult
|
||||
}
|
||||
|
||||
# Test 6: Upload medium file (single part)
|
||||
Write-Verbose "Testing medium file upload..."
|
||||
$uploadResult2 = New-MinIOObject -BucketName $TestBucket -Files (Get-Item $MediumFile) -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "Medium File Upload"
|
||||
Status = "Success"
|
||||
Details = "Uploaded medium file: $($uploadResult2.Count) objects"
|
||||
Result = $uploadResult2
|
||||
}
|
||||
|
||||
# Test 7: Upload large file (multipart)
|
||||
Write-Verbose "Testing multipart upload..."
|
||||
$multipartResult = New-MinIOObjectMultipart -BucketName $TestBucket -FilePath (Get-Item $LargeFile) -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "Multipart Upload"
|
||||
Status = "Success"
|
||||
Details = "Multipart upload completed: $($multipartResult.Count) objects"
|
||||
Result = $multipartResult
|
||||
}
|
||||
|
||||
# Test 8: List objects in bucket
|
||||
Write-Verbose "Testing object listing..."
|
||||
$objects = Get-MinIOObject -BucketName $TestBucket -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "List Objects"
|
||||
Status = "Success"
|
||||
Details = "Found $($objects.Count) objects in bucket"
|
||||
Result = $objects
|
||||
}
|
||||
|
||||
# Test 9: Download small file
|
||||
Write-Verbose "Testing single file download..."
|
||||
$downloadPath = Join-Path $TestDirectory "downloaded-small.txt"
|
||||
$downloadResult = Get-MinIOObjectContent -BucketName $TestBucket -ObjectName "small-test.txt" -LocalPath $downloadPath -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "Single File Download"
|
||||
Status = "Success"
|
||||
Details = "Downloaded file: $($downloadResult.FilePath)"
|
||||
Result = $downloadResult
|
||||
}
|
||||
|
||||
# Test 10: Download large file (multipart)
|
||||
Write-Verbose "Testing multipart download..."
|
||||
$downloadPath2 = Join-Path $TestDirectory "downloaded-large.txt"
|
||||
$multipartDownload = Get-MinIOObjectContentMultipart -BucketName $TestBucket -ObjectName "large-test.txt" -DestinationPath (New-Object System.IO.FileInfo $downloadPath2) -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "Multipart Download"
|
||||
Status = "Success"
|
||||
Details = "Multipart download completed: $($multipartDownload.FilePath)"
|
||||
Result = $multipartDownload
|
||||
}
|
||||
|
||||
# Test 11: Generate presigned URL
|
||||
Write-Verbose "Testing presigned URL generation..."
|
||||
$presignedUrl = Get-MinIOPresignedUrl -BucketName $TestBucket -ObjectName "small-test.txt" -Expiration (New-TimeSpan -Hours 1) -Verbose
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "Presigned URL"
|
||||
Status = "Success"
|
||||
Details = "Generated presigned URL"
|
||||
Result = $presignedUrl
|
||||
}
|
||||
|
||||
} catch {
|
||||
$testResults += [PSCustomObject]@{
|
||||
Test = "ERROR"
|
||||
Status = "Failed"
|
||||
Details = $_.Exception.Message
|
||||
Result = $_.ScriptStackTrace
|
||||
}
|
||||
} finally {
|
||||
# Cleanup
|
||||
Write-Verbose "Cleaning up test resources..."
|
||||
|
||||
try {
|
||||
# Remove local test directory
|
||||
if (Test-Path $TestDirectory) {
|
||||
Remove-Item $TestDirectory -Recurse -Force
|
||||
Write-Verbose "Removed local test directory"
|
||||
}
|
||||
} catch {
|
||||
Write-Warning "Cleanup warning: $($_.Exception.Message)"
|
||||
}
|
||||
}
|
||||
|
||||
# Output test results
|
||||
Write-Output "PSMinIO Comprehensive Test Results:"
|
||||
Write-Output "=================================="
|
||||
$testResults | Format-Table Test, Status, Details -AutoSize
|
||||
Write-Output ""
|
||||
Write-Output "Test bucket '$TestBucket' left for manual cleanup"
|
||||
Write-Output "Total tests: $($testResults.Count)"
|
||||
Write-Output "Successful: $(($testResults | Where-Object Status -eq 'Success').Count)"
|
||||
Write-Output "Failed: $(($testResults | Where-Object Status -eq 'Failed').Count)"
|
||||
|
||||
# Return results for further processing
|
||||
return $testResults
|
||||
@@ -1,123 +0,0 @@
|
||||
# PSMinIO Individual Test Commands
|
||||
# Copy and paste these commands one by one to test functionality
|
||||
|
||||
# Test Configuration
|
||||
$S3Url = "https://api.s3.gracesolution.info"
|
||||
$AccessKey = "T34Wg85SAwezUa3sk3m4"
|
||||
$SecretKey = "PxEmnbQoQJTJsDocSEV6mSSscDpJMiJCayPv93xe"
|
||||
$TestBucket = "psminiotest-$(Get-Date -Format 'yyyyMMdd-HHmmss')"
|
||||
|
||||
Write-Output "Test Configuration:"
|
||||
Write-Output "S3 Endpoint: $S3Url"
|
||||
Write-Output "Test Bucket: $TestBucket"
|
||||
Write-Output ""
|
||||
|
||||
# ===== STEP 1: IMPORT MODULE =====
|
||||
Write-Output "1. Import PSMinIO Module:"
|
||||
Import-Module "..\Module\PSMinIO\PSMinIO.psd1" -Force -Verbose
|
||||
|
||||
# ===== STEP 2: CONNECT TO S3 =====
|
||||
Write-Output "`n2. Connect to S3:"
|
||||
$connection = Connect-MinIO -Url $S3Url -AccessKey $AccessKey -SecretKey $SecretKey -Verbose
|
||||
$connection
|
||||
|
||||
# ===== STEP 3: LIST EXISTING BUCKETS =====
|
||||
Write-Output "`n3. List existing buckets:"
|
||||
$buckets = Get-MinIOBucket -Verbose
|
||||
$buckets | Format-Table Name, CreationDate, Region -AutoSize
|
||||
|
||||
# ===== STEP 4: CREATE TEST BUCKET =====
|
||||
Write-Output "`n4. Create test bucket:"
|
||||
$newBucket = New-MinIOBucket -BucketName $TestBucket -Verbose
|
||||
$newBucket
|
||||
|
||||
# ===== STEP 5: VERIFY BUCKET EXISTS =====
|
||||
Write-Output "`n5. Verify bucket exists:"
|
||||
$bucketExists = Test-MinIOBucketExists -BucketName $TestBucket -Verbose
|
||||
Write-Output "Bucket exists: $bucketExists"
|
||||
|
||||
# ===== STEP 6: CREATE TEST FILES =====
|
||||
Write-Output "`n6. Create test files:"
|
||||
$TestDirectory = "C:\Temp\PSMinIOTest"
|
||||
if (Test-Path $TestDirectory) { Remove-Item $TestDirectory -Recurse -Force }
|
||||
New-Item -ItemType Directory -Path $TestDirectory -Force | Out-Null
|
||||
|
||||
# Small file (< 1KB)
|
||||
$SmallFile = Join-Path $TestDirectory "small-test.txt"
|
||||
"This is a small test file for PSMinIO testing." | Out-File $SmallFile -Encoding UTF8
|
||||
|
||||
# Medium file (~50KB)
|
||||
$MediumFile = Join-Path $TestDirectory "medium-test.txt"
|
||||
1..1000 | ForEach-Object { "Line $_ - Medium test file content for PSMinIO testing with more data." } | Out-File $MediumFile -Encoding UTF8
|
||||
|
||||
# Large file (~500KB)
|
||||
$LargeFile = Join-Path $TestDirectory "large-test.txt"
|
||||
1..10000 | ForEach-Object { "Line $_ - Large test file content for PSMinIO multipart testing with substantial data to trigger chunked operations." } | Out-File $LargeFile -Encoding UTF8
|
||||
|
||||
Get-ChildItem $TestDirectory | Select-Object Name, @{Name="Size(KB)";Expression={[math]::Round($_.Length/1KB,2)}}
|
||||
|
||||
# ===== STEP 7: UPLOAD SMALL FILE (SINGLE PART) =====
|
||||
Write-Output "`n7. Upload small file (single part):"
|
||||
$uploadResult1 = New-MinIOObject -BucketName $TestBucket -Files (Get-Item $SmallFile) -Verbose
|
||||
$uploadResult1
|
||||
|
||||
# ===== STEP 8: UPLOAD MEDIUM FILE (SINGLE PART) =====
|
||||
Write-Output "`n8. Upload medium file (single part):"
|
||||
$uploadResult2 = New-MinIOObject -BucketName $TestBucket -Files (Get-Item $MediumFile) -Verbose
|
||||
$uploadResult2
|
||||
|
||||
# ===== STEP 9: UPLOAD LARGE FILE (MULTIPART) =====
|
||||
Write-Output "`n9. Upload large file (multipart):"
|
||||
$multipartResult = New-MinIOObjectMultipart -BucketName $TestBucket -FilePath (Get-Item $LargeFile) -Verbose
|
||||
$multipartResult
|
||||
|
||||
# ===== STEP 10: LIST OBJECTS IN BUCKET =====
|
||||
Write-Output "`n10. List objects in bucket:"
|
||||
$objects = Get-MinIOObject -BucketName $TestBucket -Verbose
|
||||
$objects | Format-Table Key, @{Name="Size(KB)";Expression={[math]::Round($_.Size/1KB,2)}}, LastModified -AutoSize
|
||||
|
||||
# ===== STEP 11: DOWNLOAD SMALL FILE =====
|
||||
Write-Output "`n11. Download small file:"
|
||||
$downloadPath1 = Join-Path $TestDirectory "downloaded-small.txt"
|
||||
$downloadResult1 = Get-MinIOObjectContent -BucketName $TestBucket -ObjectName "small-test.txt" -LocalPath $downloadPath1 -Verbose
|
||||
$downloadResult1
|
||||
|
||||
# ===== STEP 12: DOWNLOAD LARGE FILE (MULTIPART) =====
|
||||
Write-Output "`n12. Download large file (multipart):"
|
||||
$downloadPath2 = Join-Path $TestDirectory "downloaded-large.txt"
|
||||
$multipartDownload = Get-MinIOObjectContentMultipart -BucketName $TestBucket -ObjectName "large-test.txt" -DestinationPath (New-Object System.IO.FileInfo $downloadPath2) -Verbose
|
||||
$multipartDownload
|
||||
|
||||
# ===== STEP 13: GENERATE PRESIGNED URL =====
|
||||
Write-Output "`n13. Generate presigned URL:"
|
||||
$presignedUrl = Get-MinIOPresignedUrl -BucketName $TestBucket -ObjectName "small-test.txt" -Expiration (New-TimeSpan -Hours 1) -Verbose
|
||||
$presignedUrl
|
||||
|
||||
# ===== STEP 14: VERIFY DOWNLOADED FILES =====
|
||||
Write-Output "`n14. Verify downloaded files:"
|
||||
Write-Output "Original files:"
|
||||
Get-ChildItem $TestDirectory -Filter "*test.txt" | Select-Object Name, @{Name="Size(KB)";Expression={[math]::Round($_.Length/1KB,2)}}
|
||||
|
||||
Write-Output "`nDownloaded files:"
|
||||
Get-ChildItem $TestDirectory -Filter "downloaded-*" | Select-Object Name, @{Name="Size(KB)";Expression={[math]::Round($_.Length/1KB,2)}}
|
||||
|
||||
# ===== STEP 15: COMPARE FILE CONTENTS =====
|
||||
Write-Output "`n15. Compare file contents:"
|
||||
$originalSmall = Get-Content $SmallFile
|
||||
$downloadedSmall = Get-Content $downloadPath1
|
||||
Write-Output "Small file content match: $(($originalSmall -join '') -eq ($downloadedSmall -join ''))"
|
||||
|
||||
$originalLarge = Get-Content $LargeFile
|
||||
$downloadedLarge = Get-Content $downloadPath2
|
||||
Write-Output "Large file content match: $(($originalLarge -join '') -eq ($downloadedLarge -join ''))"
|
||||
|
||||
# ===== CLEANUP INSTRUCTIONS =====
|
||||
Write-Output "`n16. Cleanup (manual):"
|
||||
Write-Output "Test bucket created: $TestBucket"
|
||||
Write-Output "Local test directory: $TestDirectory"
|
||||
Write-Output ""
|
||||
Write-Output "To clean up:"
|
||||
Write-Output "1. Remove local directory: Remove-Item '$TestDirectory' -Recurse -Force"
|
||||
Write-Output "2. Remove S3 bucket objects and bucket manually from S3 console"
|
||||
Write-Output ""
|
||||
Write-Output "=== ALL TESTS COMPLETED ==="
|
||||
@@ -1,104 +0,0 @@
|
||||
#!/usr/bin/env pwsh
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Quick complete test of PSMinIO functionality
|
||||
|
||||
.DESCRIPTION
|
||||
Tests all core functionality quickly to verify the implementation works
|
||||
#>
|
||||
|
||||
# Import the module
|
||||
Import-Module "$PSScriptRoot\..\Module\PSMinIO\PSMinIO.psd1" -Force
|
||||
|
||||
Write-Output "=== PSMinIO Complete Functionality Test ==="
|
||||
|
||||
try {
|
||||
# Test 1: Connection
|
||||
Write-Output "1. Testing Connection..."
|
||||
Connect-MinIO -Endpoint "https://api.s3.gracesolution.info" -AccessKey "T34Wg85SAwezUa3sk3m4" -SecretKey "PxEmnbQoQJTJsDocSEV6mSSscDpJMiJCayPv93xe" -TestConnection
|
||||
Write-Output "✅ Connection successful!"
|
||||
|
||||
# Test 2: List Buckets
|
||||
Write-Output "2. Testing Bucket Listing..."
|
||||
$buckets = Get-MinIOBucket
|
||||
Write-Output "✅ Found $($buckets.Count) buckets"
|
||||
|
||||
# Test 3: Create Test Bucket
|
||||
Write-Output "3. Testing Bucket Creation..."
|
||||
$testBucket = "psminiotest-$(Get-Date -Format 'yyyyMMdd-HHmmss')"
|
||||
$bucketResult = New-MinIOBucket -BucketName $testBucket -PassThru
|
||||
Write-Output "✅ Bucket created: $($bucketResult.Name)"
|
||||
|
||||
# Test 4: Test Bucket Exists
|
||||
Write-Output "4. Testing Bucket Existence..."
|
||||
$exists = Test-MinIOBucketExists -BucketName $testBucket
|
||||
Write-Output "✅ Bucket exists: $exists"
|
||||
|
||||
# Test 5: Upload File
|
||||
Write-Output "5. Testing File Upload..."
|
||||
$testFile = Join-Path $env:TEMP "psminiotest-complete.txt"
|
||||
$testContent = @"
|
||||
PSMinIO Complete Test File
|
||||
=========================
|
||||
Created: $(Get-Date)
|
||||
Implementation: Custom REST API
|
||||
Test data: $('A' * 1000)
|
||||
"@
|
||||
$testContent | Out-File -FilePath $testFile -Encoding UTF8
|
||||
$fileInfo = Get-Item $testFile
|
||||
$uploadResult = New-MinIOObject -BucketName $testBucket -File $fileInfo -PassThru
|
||||
Write-Output "✅ Upload successful: $($uploadResult.ObjectName) ($($uploadResult.TotalSizeFormatted))"
|
||||
|
||||
# Test 6: List Objects
|
||||
Write-Output "6. Testing Object Listing..."
|
||||
$objects = Get-MinIOObject -BucketName $testBucket
|
||||
Write-Output "✅ Found $($objects.Count) objects"
|
||||
|
||||
# Test 7: Download File
|
||||
Write-Output "7. Testing File Download..."
|
||||
$downloadFile = Join-Path $env:TEMP "psminiotest-complete-download.txt"
|
||||
$downloadResult = Get-MinIOObjectContent -BucketName $testBucket -ObjectName $fileInfo.Name -LocalPath $downloadFile -PassThru
|
||||
Write-Output "✅ Download successful: $($downloadResult.TotalSizeFormatted)"
|
||||
|
||||
# Test 8: Verify Content
|
||||
Write-Output "8. Testing Content Verification..."
|
||||
$originalContent = Get-Content $testFile -Raw
|
||||
$downloadedContent = Get-Content $downloadFile -Raw
|
||||
if ($originalContent -eq $downloadedContent) {
|
||||
Write-Output "✅ Content verification PASSED!"
|
||||
} else {
|
||||
Write-Output "❌ Content verification FAILED!"
|
||||
}
|
||||
|
||||
Write-Output ""
|
||||
Write-Output "=== ALL TESTS PASSED! ==="
|
||||
Write-Output "🎉 PSMinIO Custom REST API Implementation is fully functional!"
|
||||
Write-Output ""
|
||||
Write-Output "Features Verified:"
|
||||
Write-Output " ✓ Connection management with AWS S3 signature v4"
|
||||
Write-Output " ✓ Bucket operations (create, list, exists)"
|
||||
Write-Output " ✓ Object upload with progress tracking"
|
||||
Write-Output " ✓ Object download with progress tracking"
|
||||
Write-Output " ✓ Object listing and filtering"
|
||||
Write-Output " ✓ Performance metrics and timing"
|
||||
Write-Output " ✓ Content integrity verification"
|
||||
Write-Output ""
|
||||
Write-Output "Architecture Benefits:"
|
||||
Write-Output " • No MinIO SDK dependency"
|
||||
Write-Output " • No async/await compatibility issues"
|
||||
Write-Output " • Real progress reporting from HTTP streams"
|
||||
Write-Output " • Built-in performance monitoring"
|
||||
Write-Output " • Reduced dependencies (3 DLLs vs 10+)"
|
||||
|
||||
} catch {
|
||||
Write-Output "❌ Test failed: $($_.Exception.Message)"
|
||||
exit 1
|
||||
} finally {
|
||||
# Cleanup
|
||||
if (Test-Path $testFile -ErrorAction SilentlyContinue) {
|
||||
Remove-Item $testFile -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
if (Test-Path $downloadFile -ErrorAction SilentlyContinue) {
|
||||
Remove-Item $downloadFile -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
}
|
||||
@@ -1,70 +0,0 @@
|
||||
#!/usr/bin/env pwsh
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Quick test of PSMinIO upload and download functionality
|
||||
|
||||
.DESCRIPTION
|
||||
Tests the new PSMinIO implementation with file upload and download
|
||||
#>
|
||||
|
||||
# Import the module
|
||||
Import-Module "$PSScriptRoot\..\Module\PSMinIO\PSMinIO.psd1" -Force
|
||||
|
||||
# Connect to MinIO
|
||||
Write-Verbose "Connecting to MinIO..." -Verbose
|
||||
Connect-MinIO -Endpoint "https://api.s3.gracesolution.info" -AccessKey "T34Wg85SAwezUa3sk3m4" -SecretKey "PxEmnbQoQJTJsDocSEV6mSSscDpJMiJCayPv93xe" -Verbose
|
||||
|
||||
# Create test file
|
||||
$testFile = Join-Path $env:TEMP "psminiotest.txt"
|
||||
$testContent = @"
|
||||
PSMinIO Test File
|
||||
================
|
||||
Created: $(Get-Date)
|
||||
Test data: $('A' * 1000)
|
||||
"@
|
||||
|
||||
Write-Verbose "Creating test file: $testFile" -Verbose
|
||||
$testContent | Out-File -FilePath $testFile -Encoding UTF8
|
||||
$fileInfo = Get-Item $testFile
|
||||
Write-Verbose "Test file created: $($fileInfo.Length) bytes" -Verbose
|
||||
|
||||
# Upload file
|
||||
$testBucket = "psminiotest-20250711-142415"
|
||||
Write-Verbose "Uploading file to bucket: $testBucket" -Verbose
|
||||
$uploadResult = New-MinIOObject -BucketName $testBucket -File $fileInfo -PassThru -Verbose
|
||||
|
||||
Write-Output "Upload Result:"
|
||||
if ($uploadResult) {
|
||||
$uploadResult | Format-List ObjectName, TotalSizeFormatted, DurationFormatted, AverageSpeedFormatted, Success
|
||||
}
|
||||
|
||||
# List objects
|
||||
Write-Verbose "Listing objects in bucket..." -Verbose
|
||||
$objects = Get-MinIOObject -BucketName $testBucket -Verbose
|
||||
$objects | Format-Table Name, Size, LastModified
|
||||
|
||||
# Download file
|
||||
$downloadFile = Join-Path $env:TEMP "psminiotest-download.txt"
|
||||
Write-Verbose "Downloading file to: $downloadFile" -Verbose
|
||||
$downloadResult = Get-MinIOObjectContent -BucketName $testBucket -ObjectName $fileInfo.Name -LocalPath $downloadFile -PassThru -Verbose
|
||||
|
||||
Write-Output "Download Result:"
|
||||
if ($downloadResult) {
|
||||
$downloadResult | Format-List ObjectName, TotalSizeFormatted, DurationFormatted, AverageSpeedFormatted, Success
|
||||
}
|
||||
|
||||
# Verify content
|
||||
$originalContent = Get-Content $testFile -Raw -ErrorAction SilentlyContinue
|
||||
$downloadedContent = Get-Content $downloadFile -Raw -ErrorAction SilentlyContinue
|
||||
|
||||
if ($originalContent -and $downloadedContent -and ($originalContent -eq $downloadedContent)) {
|
||||
Write-Output "✅ File content verification PASSED!"
|
||||
} else {
|
||||
Write-Output "❌ File content verification FAILED!"
|
||||
}
|
||||
|
||||
# Cleanup
|
||||
Remove-Item $testFile -Force -ErrorAction SilentlyContinue
|
||||
Remove-Item $downloadFile -Force -ErrorAction SilentlyContinue
|
||||
|
||||
Write-Output "Test completed!"
|
||||
@@ -1,74 +0,0 @@
|
||||
# Test XML Parsing for Multipart Upload Response
|
||||
# This script tests the XML parsing logic without modifying the codebase
|
||||
|
||||
# Sample XML response from your S3 server
|
||||
$xmlResponse = @'
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Bucket>xylem</Bucket><Key>Win10ISO.zip</Key><UploadId>MzBlZDdmNjUtZjNjYi00YzhhLThhMWQtNzViYTAyNWIzYjFhLmJiYjBmNzcxLWZiODEtNGFhYy1iYWQwLTk3M2EyNDIwMjRkM3gxNzUyNTQwMTc4NzE4NjczNjY1</UploadId></InitiateMultipartUploadResult>
|
||||
'@
|
||||
|
||||
Write-Output "=== XML Parsing Test ==="
|
||||
Write-Output "Testing XML response parsing logic"
|
||||
Write-Output ""
|
||||
|
||||
# Parse the XML
|
||||
$doc = [System.Xml.Linq.XDocument]::Parse($xmlResponse)
|
||||
|
||||
Write-Output "1. Basic XML parsing:"
|
||||
Write-Output " Root element: $($doc.Root.Name)"
|
||||
Write-Output " Root namespace: $($doc.Root.GetDefaultNamespace())"
|
||||
Write-Output ""
|
||||
|
||||
# Test method 1: Without namespace (current broken method)
|
||||
Write-Output "2. Method 1 - Without namespace (broken):"
|
||||
$uploadId1 = $doc.Descendants("UploadId") | Select-Object -First 1 | ForEach-Object { $_.Value }
|
||||
Write-Output " UploadId found: $($uploadId1 ?? 'NULL')"
|
||||
Write-Output ""
|
||||
|
||||
# Test method 2: With namespace (fixed method)
|
||||
Write-Output "3. Method 2 - With namespace (fixed):"
|
||||
$ns = $doc.Root.GetDefaultNamespace()
|
||||
if ($ns) {
|
||||
$uploadId2 = $doc.Descendants([System.Xml.Linq.XName]::Get("UploadId", $ns.NamespaceName)) | Select-Object -First 1 | ForEach-Object { $_.Value }
|
||||
} else {
|
||||
$uploadId2 = $doc.Descendants("UploadId") | Select-Object -First 1 | ForEach-Object { $_.Value }
|
||||
}
|
||||
Write-Output " Namespace: $($ns.NamespaceName)"
|
||||
Write-Output " UploadId found: $($uploadId2 ?? 'NULL')"
|
||||
Write-Output ""
|
||||
|
||||
# Test method 3: Alternative approach
|
||||
Write-Output "4. Method 3 - Alternative approach:"
|
||||
$uploadId3 = $doc.Descendants() | Where-Object { $_.Name.LocalName -eq "UploadId" } | Select-Object -First 1 | ForEach-Object { $_.Value }
|
||||
Write-Output " UploadId found: $($uploadId3 ?? 'NULL')"
|
||||
Write-Output ""
|
||||
|
||||
# Show all elements
|
||||
Write-Output "5. All elements in the XML:"
|
||||
$doc.Descendants() | ForEach-Object {
|
||||
Write-Output " Element: $($_.Name.LocalName) = $($_.Value)"
|
||||
}
|
||||
Write-Output ""
|
||||
|
||||
# Test the exact logic that should work
|
||||
Write-Output "6. Recommended fix logic:"
|
||||
$ns = $doc.Root.GetDefaultNamespace()
|
||||
if ($ns -and $ns.NamespaceName) {
|
||||
$uploadIdFinal = $doc.Descendants([System.Xml.Linq.XName]::Get("UploadId", $ns.NamespaceName)) | Select-Object -First 1 | ForEach-Object { $_.Value }
|
||||
} else {
|
||||
$uploadIdFinal = $doc.Descendants("UploadId") | Select-Object -First 1 | ForEach-Object { $_.Value }
|
||||
}
|
||||
|
||||
Write-Output " Final UploadId: $($uploadIdFinal ?? 'NULL')"
|
||||
Write-Output " Success: $($uploadIdFinal -ne $null -and $uploadIdFinal -ne '')"
|
||||
Write-Output ""
|
||||
|
||||
if ($uploadIdFinal) {
|
||||
Write-Output "✅ XML parsing would work with namespace handling!"
|
||||
Write-Output " The UploadId '$uploadIdFinal' was successfully extracted."
|
||||
} else {
|
||||
Write-Output "❌ XML parsing still has issues."
|
||||
}
|
||||
|
||||
Write-Output ""
|
||||
Write-Output "=== Test Complete ==="
|
||||
@@ -1,202 +0,0 @@
|
||||
# Test script for the new New-MinIOZipArchive cmdlet
|
||||
# Demonstrates comprehensive zip functionality with progress tracking
|
||||
|
||||
param(
|
||||
[string]$TestDirectory = "TestZipFiles",
|
||||
[switch]$Cleanup,
|
||||
[switch]$Verbose
|
||||
)
|
||||
|
||||
if ($Verbose) {
|
||||
$VerbosePreference = 'Continue'
|
||||
}
|
||||
|
||||
Write-Host "=== PSMinIO Zip Archive Test ===" -ForegroundColor Cyan
|
||||
Write-Host "Testing New-MinIOZipArchive cmdlet with comprehensive functionality" -ForegroundColor Green
|
||||
|
||||
try {
|
||||
# Import the module
|
||||
Write-Host "`n1. Loading PSMinIO module..." -ForegroundColor Yellow
|
||||
Import-Module ".\Module\PSMinIO\PSMinIO.psd1" -Force
|
||||
|
||||
# Create test directory and files
|
||||
Write-Host "2. Creating test files..." -ForegroundColor Yellow
|
||||
if (Test-Path $TestDirectory) {
|
||||
Remove-Item $TestDirectory -Recurse -Force
|
||||
}
|
||||
New-Item -ItemType Directory -Path $TestDirectory -Force | Out-Null
|
||||
|
||||
# Create main directory files
|
||||
for ($i = 1; $i -le 10; $i++) {
|
||||
$fileName = "document-{0:D2}.txt" -f $i
|
||||
$filePath = Join-Path $TestDirectory $fileName
|
||||
$content = "Document $i`n" + ("Sample content line $i`n" * (Get-Random -Minimum 5 -Maximum 20))
|
||||
Set-Content -Path $filePath -Value $content
|
||||
}
|
||||
|
||||
# Create subdirectories with different file types
|
||||
$subDir1 = Join-Path $TestDirectory "Reports"
|
||||
$subDir2 = Join-Path $TestDirectory "Data"
|
||||
$subDir3 = Join-Path $TestDirectory "Logs"
|
||||
New-Item -ItemType Directory -Path $subDir1, $subDir2, $subDir3 -Force | Out-Null
|
||||
|
||||
# Add files to subdirectories
|
||||
for ($i = 1; $i -le 5; $i++) {
|
||||
Set-Content -Path (Join-Path $subDir1 "report$i.txt") -Value "Report $i content"
|
||||
Set-Content -Path (Join-Path $subDir2 "data$i.csv") -Value "ID,Name,Value`n$i,Item$i,$(Get-Random -Minimum 100 -Maximum 1000)"
|
||||
Set-Content -Path (Join-Path $subDir3 "app$i.log") -Value "$(Get-Date) - Log entry $i"
|
||||
}
|
||||
|
||||
$allFiles = Get-ChildItem $TestDirectory -Recurse -File
|
||||
Write-Host "✅ Created test structure: $($allFiles.Count) files in multiple directories" -ForegroundColor Green
|
||||
|
||||
# Test 1: Create zip from FileInfo array
|
||||
Write-Host "`n3. Test 1: Creating zip from FileInfo array..." -ForegroundColor Yellow
|
||||
$mainFiles = Get-ChildItem $TestDirectory -File
|
||||
$zipFile1 = [System.IO.FileInfo]"test-files-array.zip"
|
||||
|
||||
Write-Host " Creating zip with $($mainFiles.Count) files using Files parameter set" -ForegroundColor Cyan
|
||||
$result1 = New-MinIOZipArchive -DestinationPath $zipFile1 -Path $mainFiles -CompressionLevel Optimal -Verbose
|
||||
|
||||
Write-Host "✅ Files array zip created:" -ForegroundColor Green
|
||||
Write-Host " Files: $($result1.FileCount)" -ForegroundColor White
|
||||
Write-Host " Original size: $([math]::Round($result1.TotalUncompressedSize / 1KB, 2)) KB" -ForegroundColor White
|
||||
Write-Host " Compressed size: $([math]::Round($result1.TotalCompressedSize / 1KB, 2)) KB" -ForegroundColor White
|
||||
Write-Host " Compression: $($result1.CompressionEfficiency.ToString('F1'))%" -ForegroundColor White
|
||||
Write-Host " Duration: $($result1.Duration.TotalSeconds.ToString('F2'))s" -ForegroundColor White
|
||||
|
||||
# Test 2: Create zip from directory (non-recursive)
|
||||
Write-Host "`n4. Test 2: Creating zip from directory (non-recursive)..." -ForegroundColor Yellow
|
||||
$zipFile2 = [System.IO.FileInfo]"test-directory-flat.zip"
|
||||
|
||||
Write-Host " Creating zip from directory (top-level files only)" -ForegroundColor Cyan
|
||||
$result2 = New-MinIOZipArchive -DestinationPath $zipFile2 -Directory (Get-Item $TestDirectory) -CompressionLevel Fastest -Verbose
|
||||
|
||||
Write-Host "✅ Directory flat zip created:" -ForegroundColor Green
|
||||
Write-Host " Files: $($result2.FileCount)" -ForegroundColor White
|
||||
Write-Host " Compression: $($result2.CompressionEfficiency.ToString('F1'))%" -ForegroundColor White
|
||||
|
||||
# Test 3: Create zip from directory (recursive)
|
||||
Write-Host "`n5. Test 3: Creating zip from directory (recursive)..." -ForegroundColor Yellow
|
||||
$zipFile3 = [System.IO.FileInfo]"test-directory-recursive.zip"
|
||||
|
||||
Write-Host " Creating zip from directory (recursive, all subdirectories)" -ForegroundColor Cyan
|
||||
$result3 = New-MinIOZipArchive -DestinationPath $zipFile3 -Directory (Get-Item $TestDirectory) -Recursive -IncludeBaseDirectory -CompressionLevel Optimal -Verbose
|
||||
|
||||
Write-Host "✅ Directory recursive zip created:" -ForegroundColor Green
|
||||
Write-Host " Files: $($result3.FileCount)" -ForegroundColor White
|
||||
Write-Host " Compression: $($result3.CompressionEfficiency.ToString('F1'))%" -ForegroundColor White
|
||||
|
||||
# Test 4: Create zip with file filtering
|
||||
Write-Host "`n6. Test 4: Creating zip with file filtering..." -ForegroundColor Yellow
|
||||
$zipFile4 = [System.IO.FileInfo]"test-filtered-logs.zip"
|
||||
|
||||
Write-Host " Creating zip with only .log files using InclusionFilter" -ForegroundColor Cyan
|
||||
$result4 = New-MinIOZipArchive -DestinationPath $zipFile4 -Directory (Get-Item $TestDirectory) -Recursive -InclusionFilter { $_.Extension -eq ".log" } -CompressionLevel Optimal -Verbose
|
||||
|
||||
Write-Host "✅ Filtered zip created:" -ForegroundColor Green
|
||||
Write-Host " Log files: $($result4.FileCount)" -ForegroundColor White
|
||||
Write-Host " Compression: $($result4.CompressionEfficiency.ToString('F1'))%" -ForegroundColor White
|
||||
|
||||
# Test 5: Append to existing zip (Update mode)
|
||||
Write-Host "`n7. Test 5: Appending to existing zip..." -ForegroundColor Yellow
|
||||
$csvFiles = Get-ChildItem $TestDirectory -Recurse -Filter "*.csv"
|
||||
|
||||
Write-Host " Appending $($csvFiles.Count) CSV files to existing zip" -ForegroundColor Cyan
|
||||
$result5 = New-MinIOZipArchive -DestinationPath $zipFile4 -Path $csvFiles -Mode Update -CompressionLevel Optimal -Verbose
|
||||
|
||||
Write-Host "✅ Files appended to zip:" -ForegroundColor Green
|
||||
Write-Host " Total files now: $($result5.FileCount)" -ForegroundColor White
|
||||
|
||||
# Test 6: Create zip with custom base path
|
||||
Write-Host "`n8. Test 6: Creating zip with custom base path..." -ForegroundColor Yellow
|
||||
$zipFile6 = [System.IO.FileInfo]"test-custom-basepath.zip"
|
||||
|
||||
Write-Host " Creating zip with custom base path (flattened structure)" -ForegroundColor Cyan
|
||||
$result6 = New-MinIOZipArchive -DestinationPath $zipFile6 -Directory (Get-Item $TestDirectory) -Recursive -BasePath $TestDirectory -CompressionLevel Optimal -Verbose
|
||||
|
||||
Write-Host "✅ Custom base path zip created:" -ForegroundColor Green
|
||||
Write-Host " Files: $($result6.FileCount)" -ForegroundColor White
|
||||
|
||||
# Test 7: Get zip archive information
|
||||
Write-Host "`n9. Test 7: Reading zip archive information..." -ForegroundColor Yellow
|
||||
$zipFiles = Get-ChildItem "*.zip"
|
||||
|
||||
foreach ($zipFile in $zipFiles) {
|
||||
Write-Host " Reading archive: $($zipFile.Name)" -ForegroundColor Cyan
|
||||
|
||||
# Basic archive info
|
||||
$archiveInfo = Get-MinIOZipArchive -ZipFile $zipFile -Verbose
|
||||
Write-Host " Entries: $($archiveInfo.EntryCount)" -ForegroundColor White
|
||||
Write-Host " Size: $([math]::Round($archiveInfo.TotalUncompressedSize / 1KB, 2)) KB -> $([math]::Round($archiveInfo.TotalCompressedSize / 1KB, 2)) KB" -ForegroundColor White
|
||||
Write-Host " Compression: $($archiveInfo.CompressionEfficiency.ToString('F1'))%" -ForegroundColor White
|
||||
|
||||
# Detailed entries for one archive
|
||||
if ($zipFile.Name -eq "test-directory-recursive.zip") {
|
||||
Write-Host " Getting detailed entries for recursive archive..." -ForegroundColor Cyan
|
||||
$detailedInfo = Get-MinIOZipArchive -ZipFile $zipFile -IncludeEntries -Verbose
|
||||
Write-Host " Entry details: $($detailedInfo.Entries.Count) entries" -ForegroundColor White
|
||||
|
||||
# Show first few entries
|
||||
$detailedInfo.Entries | Select-Object -First 3 | ForEach-Object {
|
||||
Write-Host " $($_.FullName) ($([math]::Round($_.Length / 1KB, 2)) KB)" -ForegroundColor Gray
|
||||
}
|
||||
}
|
||||
|
||||
# Validate integrity for one archive
|
||||
if ($zipFile.Name -eq "test-files-array.zip") {
|
||||
Write-Host " Validating archive integrity..." -ForegroundColor Cyan
|
||||
$validatedInfo = Get-MinIOZipArchive -ZipFile $zipFile -ValidateIntegrity -Verbose
|
||||
$validStatus = if ($validatedInfo.IsValid) { "✅ Valid" } else { "❌ Invalid" }
|
||||
Write-Host " Validation: $validStatus (took $($validatedInfo.ValidationDuration.TotalMilliseconds.ToString('F0'))ms)" -ForegroundColor White
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host "✅ Archive reading tests completed!" -ForegroundColor Green
|
||||
|
||||
# Summary
|
||||
Write-Host "`n=== TEST SUMMARY ===" -ForegroundColor Cyan
|
||||
Write-Host "✅ All zip archive tests completed successfully!" -ForegroundColor Green
|
||||
Write-Host "New-MinIOZipArchive features tested:" -ForegroundColor Yellow
|
||||
Write-Host " • FileInfo destination path parameter" -ForegroundColor White
|
||||
Write-Host " • FileInfo[] parameter with Files parameter set" -ForegroundColor White
|
||||
Write-Host " • Directory parameter with recursive and non-recursive modes" -ForegroundColor White
|
||||
Write-Host " • File filtering with InclusionFilter ScriptBlock" -ForegroundColor White
|
||||
Write-Host " • Append mode (Update) for adding files to existing zips" -ForegroundColor White
|
||||
Write-Host " • Custom base path for entry name control" -ForegroundColor White
|
||||
Write-Host " • Multiple compression levels (Optimal, Fastest)" -ForegroundColor White
|
||||
Write-Host " • Comprehensive progress tracking and metrics" -ForegroundColor White
|
||||
Write-Host " • Always returns result objects (no PassThru needed)" -ForegroundColor White
|
||||
|
||||
Write-Host "Get-MinIOZipArchive features tested:" -ForegroundColor Yellow
|
||||
Write-Host " • Basic archive information reading" -ForegroundColor White
|
||||
Write-Host " • Detailed entry information with IncludeEntries" -ForegroundColor White
|
||||
Write-Host " • Archive integrity validation" -ForegroundColor White
|
||||
Write-Host " • Proper disposal handling with OpenRead" -ForegroundColor White
|
||||
Write-Host " • Comprehensive metrics and compression statistics" -ForegroundColor White
|
||||
|
||||
Write-Host "`nZip files created:" -ForegroundColor Yellow
|
||||
foreach ($zipFile in $zipFiles) {
|
||||
$sizeKB = [math]::Round($zipFile.Length / 1KB, 2)
|
||||
Write-Host " • $($zipFile.Name) ($sizeKB KB)" -ForegroundColor White
|
||||
}
|
||||
|
||||
if ($Cleanup) {
|
||||
Write-Host "`n10. Cleaning up..." -ForegroundColor Yellow
|
||||
Remove-Item "*.zip" -Force -ErrorAction SilentlyContinue
|
||||
Write-Host "✅ Cleanup completed!" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host "`nZip files preserved for inspection. Use -Cleanup to remove them." -ForegroundColor Cyan
|
||||
}
|
||||
|
||||
} catch {
|
||||
Write-Host "❌ Test failed: $($_.Exception.Message)" -ForegroundColor Red
|
||||
Write-Host "Stack trace: $($_.ScriptStackTrace)" -ForegroundColor Red
|
||||
} finally {
|
||||
# Clean up test directory
|
||||
if (Test-Path $TestDirectory) {
|
||||
Remove-Item $TestDirectory -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host "`n=== Test Complete ===" -ForegroundColor Cyan
|
||||
@@ -8,6 +8,21 @@ namespace PSMinIO.Utils
|
||||
/// <summary>
|
||||
/// Thread-safe progress data collector that accumulates progress updates from background threads
|
||||
/// and allows the main thread to safely report them to PowerShell
|
||||
///
|
||||
/// CRITICAL THREADING RULE:
|
||||
/// PowerShell cmdlets can ONLY call Write-Progress, Write-Verbose, Write-Object, Write-Error
|
||||
/// from the main cmdlet thread - NEVER from background threads!
|
||||
///
|
||||
/// USAGE PATTERN:
|
||||
/// - Background threads: Call QueueProgressUpdate(), QueueVerboseMessage() (thread-safe)
|
||||
/// - Main thread ONLY: Call ProcessQueuedUpdates() to display queued updates
|
||||
///
|
||||
/// EXAMPLE:
|
||||
/// Task.Run(() => {
|
||||
/// collector.QueueProgressUpdate(1, "Processing", "Status", 50); // ✅ Safe
|
||||
/// // WriteProgress(...); // ❌ THREADING ERROR!
|
||||
/// });
|
||||
/// collector.ProcessQueuedUpdates(); // ✅ Only from main thread
|
||||
/// </summary>
|
||||
public class ThreadSafeProgressCollector
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user