PSMinIO/docs/POWERSHELL-THREADING-RULES.md

PowerShell Cmdlet Threading Rules and Design Patterns

🚨 CRITICAL THREADING RULE

PowerShell cmdlets can ONLY call Write-Progress, Write-Verbose, Write-Object, Write-Error from the main cmdlet thread.

❌ NEVER call these methods from:

• Background threads (Task.Run, Task.Factory.StartNew)
• Async callbacks (async/await continuations)
• Timer callbacks
• Event handlers from other threads
• Any thread other than the main cmdlet thread

✅ ONLY call these methods from:

• BeginProcessing() override
• ProcessRecord() override
• EndProcessing() override
• Main cmdlet thread execution

🔧 DESIGN PATTERN: ThreadSafeProgressCollector

Problem

When performing background operations (file uploads, downloads, processing), you need to report progress from background threads, but PowerShell doesn't allow direct calls to Write-Progress from those threads.

Solution

Use the ThreadSafeProgressCollector pattern:

// Background thread - QUEUE updates (thread-safe)
_progressCollector.QueueProgressUpdate(activityId, "Processing", "Status", percentage);
_progressCollector.QueueVerboseMessage("Processing file: {0}", fileName);

// Main thread - PROCESS queued updates
_progressCollector.ProcessQueuedUpdates(); // Only call from main thread!

Implementation Pattern

public class MyLongRunningCmdlet : PSCmdlet
{
    private ThreadSafeProgressCollector _progressCollector;

    protected override void BeginProcessing()
    {
        _progressCollector = new ThreadSafeProgressCollector(this);
    }

    protected override void ProcessRecord()
    {
        // Start background work
        var tasks = new List<Task>();
        
        for (int i = 0; i < workItems.Count; i++)
        {
            var task = Task.Run(() =>
            {
                // ✅ CORRECT: Queue from background thread
                _progressCollector.QueueProgressUpdate(1, "Processing", $"Item {i}", progress);
                _progressCollector.QueueVerboseMessage("Processing item {0}", i);
                
                // ❌ WRONG: Never call directly from background thread
                // WriteProgress(...); // This will cause threading errors!
                // WriteVerbose(...);  // This will cause threading errors!
            });
            tasks.Add(task);
        }

        // ✅ CORRECT: Process updates from main thread with periodic updates
        while (!Task.WaitAll(tasks.ToArray(), 1000)) // 1 second intervals
        {
            _progressCollector.ProcessQueuedUpdates(); // Safe on main thread
        }
        
        // Final processing
        _progressCollector.ProcessQueuedUpdates();
    }
}

🚨 COMMON MISTAKES TO AVOID

❌ Mistake 1: Direct calls from background threads

Task.Run(() =>
{
    WriteProgress(...); // THREADING ERROR!
    WriteVerbose(...);  // THREADING ERROR!
});

❌ Mistake 2: ProcessQueuedUpdates from background threads

Task.Run(() =>
{
    _progressCollector.QueueProgressUpdate(...);
    _progressCollector.ProcessQueuedUpdates(); // THREADING ERROR!
});

❌ Mistake 3: Async/await callbacks

await SomeAsyncOperation().ContinueWith(task =>
{
    WriteProgress(...); // THREADING ERROR!
});

✅ CORRECT PATTERNS

Pattern 1: Periodic Processing

var tasks = StartBackgroundTasks();
while (!Task.WaitAll(tasks, 1000))
{
    _progressCollector.ProcessQueuedUpdates(); // Every 1 second
}
_progressCollector.ProcessQueuedUpdates(); // Final update

Pattern 2: Completion-based Processing

var tasks = StartBackgroundTasks();
Task.WaitAll(tasks);
_progressCollector.ProcessQueuedUpdates(); // After completion

Pattern 3: Manual Processing Points

foreach (var item in items)
{
    ProcessItemInBackground(item); // Queues updates
    _progressCollector.ProcessQueuedUpdates(); // Process after each item
}

🎯 ERROR SYMPTOMS

If you violate these rules, you'll see errors like:

• "The WriteObject and WriteError methods cannot be called from outside the overrides..."
• "WriteProgress can only be called from within the same thread"
• Cmdlet failures with threading exceptions
• Progress bars not updating or appearing

📋 CHECKLIST FOR NEW CMDLETS

Before implementing background operations:

• Are you using ThreadSafeProgressCollector?
• Are all Write* calls from main thread only?
• Are you calling ProcessQueuedUpdates() only from main thread?
• Do you have periodic progress processing for long operations?
• Have you tested with verbose output enabled?
• Have you tested with progress bars enabled?

🔧 DEBUGGING TIPS

1. Enable verbose logging to see threading violations
2. Test with progress bars - they're most sensitive to threading issues
3. Use Task.WaitAll with timeouts for periodic processing
4. Never call ProcessQueuedUpdates from callbacks
5. Queue everything from background threads, process from main thread

📚 RELATED PATTERNS

• File Upload/Download: Use periodic processing during Task.WaitAll
• Multipart Operations: Queue from parallel tasks, process periodically
• Long-running Operations: Process updates every 1-2 seconds
• Collection Processing: Process after each item or batch

🎯 REMEMBER

The golden rule: Background threads QUEUE, main thread PROCESSES.

This pattern ensures PowerShell cmdlet compliance while providing real-time progress updates to users.