gsadmin/PSMinIO
1
0
Fork 0
mirror of https://github.com/Grace-Solutions/PSMinIO.git synced 2026-07-26 06:48:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Comprehensive README update with all new functionality

Browse Source 
📚 COMPLETE README OVERHAUL:

✅ ENHANCED FEATURE OVERVIEW:
  • Updated key features with bucket directory management
  • Added high-performance multipart operations section
  • Comprehensive bucket directory ecosystem description
  • Enterprise security and management features

🚀 EXPANDED QUICK START GUIDE:
  • Connection setup with SSL options
  • Bucket management examples
  • Folder management operations
  • File upload operations (single, multiple, directory, multipart)
  • File download operations (simple and multipart)
  • Advanced object listing and filtering

 COMPLETE CMDLET DOCUMENTATION:
   Organized by functional categories
   Added New-MinIOBucketFolder and Remove-MinIOBucketFolder
   Updated descriptions with latest capabilities
   Clear categorization: Connection, Buckets, Folders, Objects, Performance, Archives, URLs, Security

 ADVANCED EXAMPLES SECTION:
   Comprehensive bucket directory management examples
   High-performance multipart operations with optimal settings
   Advanced object listing and filtering techniques
   Security and policy management examples
   Archive operations with progress tracking

 PERFORMANCE FEATURES HIGHLIGHT:
   Multi-threaded operations
   Intelligent progress tracking with 3-layer visibility
   Optimized multipart operations
   Resume capability
   Memory efficiency
   Connection management
   Performance monitoring

 PROFESSIONAL PRESENTATION:
   Emoji-enhanced section headers for better readability
   Code examples with syntax highlighting
   Clear feature categorization
   Enterprise-focused language
   Comprehensive but concise descriptions

Ready for GitHub push with complete documentation!
This commit is contained in:
PSMinIO Developer
2025-07-14 23:26:01 -04:00
parent 048c4f11f2
commit a4b8e314c9
1 changed files with 164 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+164 -45
View File
@@ -1,19 +1,38 @@
# PSMinIO
A comprehensive PowerShell module for MinIO object storage operations, built on the official [Minio .NET SDK](https://www.nuget.org/packages/Minio). Provides full-featured object storage management with enterprise-grade capabilities.
A comprehensive PowerShell module for MinIO object storage operations with enterprise-grade features. Built with custom REST API implementation for optimal PowerShell compatibility and performance.
## Features
## 🚀 Key Features
- **🚀 High Performance**: Built for .NET Standard 2.0, compatible with PowerShell 5.1+ and PowerShell 7+
- **📦 Complete Object Management**: Upload, download, list, and delete objects with advanced filtering and sorting
- **🗂️ Flexible Directory Support**: Create nested folder structures with automatic directory creation
- **⚡ Chunked Operations**: Large file uploads and downloads with resume capability and progress tracking
- **🔒 Security & Policy Management**: Comprehensive bucket policy and access control management
- **📊 Advanced Object Listing**: Filter by prefix, sort by multiple criteria, limit results, and exclude directories
- **⏱️ Timing & Performance Metrics**: Detailed timing information and transfer speed reporting
- **🔄 Progress Reporting**: Multi-layer progress tracking for file collections and chunked operations
- **📝 Professional Logging**: Clean, timestamped logging with configurable verbosity levels
- **🛡️ Robust Error Handling**: Graceful handling of network issues and edge cases
### **📦 Complete Object Management**
- **Single & Multipart Uploads**: Automatic chunking for large files with resume capability
- **Parallel Downloads**: High-performance multipart downloads with progress tracking
- **Bucket Directory Management**: Create, manage, and remove nested folder structures
- **Advanced Object Listing**: Filter, sort, and paginate with comprehensive metadata
### **⚡ High-Performance Operations**
- **Intelligent Progress Tracking**: Real-time 3-layer progress bars for concurrent operations
- **Optimized Multipart Uploads**: Configurable chunk sizes and parallel upload limits
- **Performance Monitoring**: Transfer speeds, timing metrics, and completion statistics
- **Memory Efficient**: Streaming operations for large files without memory bloat
### **🗂️ Bucket Directory Ecosystem**
- **Multi-Level Folder Creation**: Support for complex nested directory structures
- **Path Format Flexibility**: Handle Windows (`\Folder\Sub`) and Unix (`/Folder/Sub`) paths
- **Recursive Operations**: Create or remove entire directory trees
- **Automatic Sanitization**: Clean and normalize all path inputs
### **🔒 Enterprise Security & Management**
- **Bucket Policy Management**: Complete CRUD operations for access policies
- **Presigned URL Generation**: Secure temporary access with configurable expiration
- **Connection Management**: Secure credential handling with SSL/TLS support
- **Comprehensive Error Handling**: Detailed error reporting and recovery options
### **📊 Advanced Features**
- **ZIP Archive Integration**: Create and extract archives with progress tracking
- **Metadata Management**: Custom headers and content type handling
- **Professional Logging**: Timestamped verbose output with intelligent formatting
- **PowerShell Integration**: Full ShouldProcess support with -WhatIf and -Confirm
## Installation
@@ -45,48 +64,138 @@ Import-Module .\Module\PSMinIO\PSMinIO.psd1
Import-Module .\Module\PSMinIO\PSMinIO.psd1
```
## Quick Start
## 🚀 Quick Start
### **Connection Setup**
```powershell
# Connect to MinIO server
$connection = Connect-MinIO -Endpoint "https://minio.example.com" -AccessKey "your-access-key" -SecretKey "your-secret-key"
Connect-MinIO -Endpoint "https://minio.example.com" -AccessKey "your-access-key" -SecretKey "your-secret-key"
# Create a bucket
New-MinIOBucket -BucketName 'my-data-bucket' -Verbose
# Upload files with automatic directory creation
New-MinIOObject -BucketName 'my-data-bucket' -Files "document.pdf" -BucketDirectory "documents/2025/january"
# List objects with advanced filtering
Get-MinIOObject -BucketName 'my-data-bucket' -Prefix "documents/" -SortBy "Size" -Descending -MaxObjects 10
# Download with timing information
Get-MinIOObjectContent -BucketName 'my-data-bucket' -ObjectName 'documents/2025/january/document.pdf' -FilePath 'C:\Downloads\document.pdf'
# Chunked upload for large files
New-MinIOObjectChunked -BucketName 'my-data-bucket' -Files "large-video.mp4" -ChunkSize 10MB -BucketDirectory "media/videos"
# Connect with SSL verification disabled (development only)
Connect-MinIO -Endpoint "https://minio.local" -AccessKey "admin" -SecretKey "password" -SkipCertificateValidation
```
## Available Cmdlets
### **Bucket Management**
```powershell
# List all buckets
Get-MinIOBucket
### Connection Management
# Create a new bucket
New-MinIOBucket -BucketName "my-bucket" -Region "us-east-1"
# Check if bucket exists
Test-MinIOBucketExists -BucketName "my-bucket"
```
### **Folder Management**
```powershell
# Create nested folder structure
New-MinIOBucketFolder -BucketName "my-bucket" -FolderPath "Documents/Projects/2024" -Recursive
# Remove folder and contents
Remove-MinIOBucketFolder -BucketName "my-bucket" -FolderPath "OldData" -Recursive -Force
```
### **File Upload Operations**
```powershell
# Simple file upload
New-MinIOObject -BucketName "my-bucket" -Path "C:\file.txt"
# Upload to specific bucket directory
New-MinIOObject -BucketName "my-bucket" -Path "C:\file.txt" -BucketDirectory "Documents/2024"
# Upload multiple files with progress
New-MinIOObject -BucketName "my-bucket" -Path @("file1.txt", "file2.txt") -Verbose
# Upload directory recursively
New-MinIOObject -BucketName "my-bucket" -Directory "C:\MyFolder" -Recursive -BucketDirectory "Backup"
# Large file multipart upload
New-MinIOObjectMultipart -BucketName "my-bucket" -FilePath "C:\large-file.zip" -ChunkSize 64MB -MaxParallelUploads 4 -BucketDirectory "Archives"
```
### **File Download Operations**
```powershell
# Simple download
Get-MinIOObjectContent -BucketName "my-bucket" -ObjectName "file.txt" -LocalPath "C:\Downloads\file.txt"
# Multipart download for large files
Get-MinIOObjectContentMultipart -BucketName "my-bucket" -ObjectName "large-file.zip" -LocalPath "C:\Downloads\" -ChunkSize 32MB
```
### **Object Listing & Management**
```powershell
# List all objects
Get-MinIOObject -BucketName "my-bucket"
# Advanced filtering and sorting
Get-MinIOObject -BucketName "my-bucket" -Prefix "documents/" -SortBy "LastModified" -Descending -MaxKeys 100 -ExcludeDirectories
```
## 📋 Available Cmdlets
### **🔌 Connection Management**
- **`Connect-MinIO`** - Establish connection to MinIO server with SSL support and certificate validation options
### Bucket Operations
- **`Get-MinIOBucket`** - List all buckets with optional statistics
- **`New-MinIOBucket`** - Create new buckets with region support
- **`Remove-MinIOBucket`** - Delete buckets with safety confirmations
- **`Test-MinIOBucketExists`** - Check bucket existence
### **🗂️ Bucket Management**
- **`Get-MinIOBucket`** - List all buckets with filtering and metadata
- **`New-MinIOBucket`** - Create new buckets with region specification
- **`Test-MinIOBucketExists`** - Check bucket existence efficiently
### Object Operations
- **`Get-MinIOObject`** - Advanced object listing with filtering, sorting, and pagination
- Filter by prefix or exact object name
- Sort by Name, Size, LastModified, or ETag (ascending/descending)
- Limit results with MaxObjects
- Exclude directories with ObjectsOnly
- **`New-MinIOObject`** - Upload files with directory support
- Single file or multiple file uploads
- Automatic nested directory creation
### **📁 Bucket Directory Management**
- **`New-MinIOBucketFolder`** - Create nested folder structures with multi-level support
- **`Remove-MinIOBucketFolder`** - Remove folders with recursive deletion options
### **📦 Object Operations**
- **`Get-MinIOObject`** - List objects with advanced filtering, sorting, and pagination
- **`New-MinIOObject`** - Upload single/multiple files with directory support and progress tracking
- **`Get-MinIOObjectContent`** - Download objects with automatic directory creation
### **⚡ High-Performance Operations**
- **`New-MinIOObjectMultipart`** - Large file uploads with chunking, parallelization, and resume capability
- **`Get-MinIOObjectContentMultipart`** - High-speed downloads with parallel chunk processing
### **📦 Archive Management**
- **`New-MinIOZipArchive`** - Create ZIP archives with progress tracking and metadata
- **`Get-MinIOZipArchive`** - Extract and analyze ZIP archive contents
### **🔗 URL Management**
- **`New-MinIOPresignedUrl`** - Generate secure temporary URLs for object access
- **`Get-MinIOPresignedUrl`** - Retrieve presigned URLs with configurable expiration
### **🔒 Security & Policy Management**
- **`Get-MinIOBucketPolicy`** - Retrieve bucket access policies
- **`Set-MinIOBucketPolicy`** - Configure bucket access policies
- **`Remove-MinIOBucketPolicy`** - Remove bucket access policies
## 💡 Advanced Examples
### **📁 Comprehensive Bucket Directory Management**
```powershell
# Create complex nested structure
New-MinIOBucketFolder -BucketName "enterprise-data" -FolderPath "Departments/IT/Projects/2024/Q1" -Recursive -Verbose
# Upload files to specific directories
New-MinIOObject -BucketName "enterprise-data" -Path "C:\Reports\*.pdf" -BucketDirectory "Departments/Finance/Reports/2024"
# Multipart upload to nested directory
New-MinIOObjectMultipart -BucketName "enterprise-data" -FilePath "C:\Backups\database.bak" -BucketDirectory "Backups/Database/2024" -ChunkSize 128MB -MaxParallelUploads 6
# Clean up old directories
Remove-MinIOBucketFolder -BucketName "enterprise-data" -FolderPath "Departments/IT/Projects/2023" -Recursive -Force
```
### **⚡ High-Performance Multipart Operations**
```powershell
# Large file upload with optimal settings
New-MinIOObjectMultipart -BucketName "media-storage" -FilePath "C:\Videos\4K-movie.mkv" -ChunkSize 256MB -MaxParallelUploads 8 -ContentType "video/x-matroska" -Verbose
# Resume interrupted upload
$uploadResult = New-MinIOObjectMultipart -BucketName "media-storage" -FilePath "C:\Videos\4K-movie.mkv" -ResumeUploadId "abc123..." -CompletedParts $previousParts
# High-speed download with chunking
Get-MinIOObjectContentMultipart -BucketName "media-storage" -ObjectName "4K-movie.mkv" -LocalPath "C:\Downloads\" -ChunkSize 128MB -MaxParallelDownloads 6
```
- Progress tracking and timing information
- **`New-MinIOObjectChunked`** - Chunked uploads for large files
- Configurable chunk sizes (1MB minimum)
@@ -144,7 +253,17 @@ See the [scripts/examples](./scripts/examples/) directory for comprehensive usag
- **.NET Framework**: 4.7.2+ (for PowerShell 5.1) or .NET Core/.NET 5+ (for PowerShell 7+)
- **MinIO Server**: Compatible with MinIO and Amazon S3 APIs
## Contributing
## 🚀 Performance Features
- **🔄 Multi-threaded Operations**: Parallel uploads and downloads for maximum throughput
- **🧠 Intelligent Progress Tracking**: Real-time 3-layer progress bars with concurrent chunk visibility
- **⚡ Optimized Multipart Operations**: Configurable chunk sizes (1MB to 5GB) with parallel processing
- **🔄 Resume Capability**: Interrupted transfers can be resumed from the last completed chunk
- **💾 Memory Efficiency**: Streaming operations minimize memory usage for large files
- **🌐 Connection Management**: Efficient HTTP connection handling with SSL/TLS support
- **📊 Performance Monitoring**: Transfer speeds, timing metrics, and completion statistics
## 🤝 Contributing
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)