gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-04 19:25:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: simplify crates README files and enforce PR-only workflow (#110)

Browse Source 
* docs: simplify all crates README files

- Remove extensive code examples and detailed documentation
- Convert to minimal module introductions with core feature lists
- Direct users to main RustFS repository for comprehensive docs
- Updated 20 crate README files for consistency and brevity

Files updated:
- crates/rio/README.md (415→15 lines)
- crates/s3select-api/README.md (592→15 lines)
- crates/s3select-query/README.md (658→15 lines)
- crates/signer/README.md (407→15 lines)
- crates/utils/README.md (395→15 lines)
- crates/workers/README.md (463→15 lines)
- crates/zip/README.md (408→15 lines)

* docs: restore original headers in crates README files

- Add back RustFS logo image and CI badges
- Restore formatted headers and structured layout
- Keep simplified content with module introductions
- Maintain consistent documentation structure across all crates

All 20 crate README files now have proper headers while keeping
the simplified content that directs users to the main repository.

* rules: enforce PR-only workflow for main branch

- Strengthen rule that ALL changes must go through pull requests
- Explicitly forbid direct commits to main branch under any circumstances
- Add comprehensive PR requirements and enforcement guidelines
- Clarify that PRs are the ONLY way to merge to main branch
- Add requirement for PR approval before merging
- Include enforcement mechanisms for branch protection
This commit is contained in:
安正超
2025-07-08 21:10:07 +08:00
committed by GitHub
parent 49f480d346
commit 3dbcaaa221
22 changed files with 227 additions and 8348 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/filemeta/README.md
+10 -235
View File
@@ -3,7 +3,7 @@
# RustFS FileMeta - File Metadata Management
<p align="center">
<strong>High-performance file metadata management for RustFS distributed object storage</strong>
<strong>Advanced file metadata management and indexing module for RustFS distributed object storage</strong>
</p>
<p align="center">
@@ -17,246 +17,21 @@
## 📖 Overview
**RustFS FileMeta** is the metadata management module for the [RustFS](https://rustfs.com) distributed object storage system. It provides efficient storage, retrieval, and management of file metadata, supporting features like versioning, tagging, and extended attributes with high performance and reliability.
> **Note:** This is a core submodule of RustFS that provides essential metadata management capabilities for the distributed object storage system. For the complete RustFS experience, please visit the [main RustFS repository](https://github.com/rustfs/rustfs).
**RustFS FileMeta** provides advanced file metadata management and indexing capabilities for the [RustFS](https://rustfs.com) distributed object storage system. For the complete RustFS experience, please visit the [main RustFS repository](https://github.com/rustfs/rustfs).
## ✨ Features
### 📝 Metadata Management
- **File Information**: Complete file metadata including size, timestamps, and checksums
- **Object Versioning**: Version-aware metadata management
- **Extended Attributes**: Custom metadata and tagging support
- **Inline Metadata**: Optimized storage for small metadata
### 🚀 Performance Features
- **FlatBuffers Serialization**: Zero-copy metadata serialization
- **Efficient Storage**: Optimized metadata storage layout
- **Fast Lookups**: High-performance metadata queries
- **Batch Operations**: Bulk metadata operations
### 🔧 Advanced Capabilities
- **Schema Evolution**: Forward and backward compatible metadata schemas
- **Compression**: Metadata compression for space efficiency
- **Validation**: Metadata integrity verification
- **Migration**: Seamless metadata format migration
## 📦 Installation
Add this to your `Cargo.toml`:
```toml
[dependencies]
rustfs-filemeta = "0.1.0"
```
## 🔧 Usage
### Basic Metadata Operations
```rust
use rustfs_filemeta::{FileInfo, XLMeta};
use std::collections::HashMap;
fn main() -> Result<(), Box<dyn std::error::Error>> {
// Create file metadata
let mut file_info = FileInfo::new();
file_info.name = "example.txt".to_string();
file_info.size = 1024;
file_info.mod_time = chrono::Utc::now();
// Add custom metadata
let mut user_defined = HashMap::new();
user_defined.insert("author".to_string(), "john@example.com".to_string());
user_defined.insert("department".to_string(), "engineering".to_string());
file_info.user_defined = user_defined;
// Create XL metadata
let xl_meta = XLMeta::new(file_info);
// Serialize metadata
let serialized = xl_meta.serialize()?;
// Deserialize metadata
let deserialized = XLMeta::deserialize(&serialized)?;
println!("File: {}, Size: {}", deserialized.file_info.name, deserialized.file_info.size);
Ok(())
}
```
### Advanced Metadata Management
```rust
use rustfs_filemeta::{XLMeta, FileInfo, VersionInfo};
async fn advanced_metadata_example() -> Result<(), Box<dyn std::error::Error>> {
// Create versioned metadata
let mut xl_meta = XLMeta::new(FileInfo::default());
// Set version information
xl_meta.set_version_info(VersionInfo {
version_id: "v1.0.0".to_string(),
is_latest: true,
delete_marker: false,
restore_ongoing: false,
});
// Add checksums
xl_meta.add_checksum("md5", "d41d8cd98f00b204e9800998ecf8427e");
xl_meta.add_checksum("sha256", "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855");
// Set object tags
xl_meta.set_tags(vec![
("Environment".to_string(), "Production".to_string()),
("Owner".to_string(), "DataTeam".to_string()),
]);
// Set retention information
xl_meta.set_retention_info(
chrono::Utc::now() + chrono::Duration::days(365),
"GOVERNANCE".to_string(),
);
// Validate metadata
xl_meta.validate()?;
Ok(())
}
```
### Inline Metadata Operations
```rust
use rustfs_filemeta::{InlineMetadata, MetadataSize};
fn inline_metadata_example() -> Result<(), Box<dyn std::error::Error>> {
// Create inline metadata for small files
let mut inline_meta = InlineMetadata::new();
// Set basic properties
inline_meta.set_content_type("text/plain");
inline_meta.set_content_encoding("gzip");
inline_meta.set_cache_control("max-age=3600");
// Add custom headers
inline_meta.add_header("x-custom-field", "custom-value");
inline_meta.add_header("x-app-version", "1.2.3");
// Check if metadata fits inline storage
if inline_meta.size() <= MetadataSize::INLINE_THRESHOLD {
println!("Metadata can be stored inline");
} else {
println!("Metadata requires separate storage");
}
// Serialize for storage
let bytes = inline_meta.to_bytes()?;
// Deserialize from storage
let restored = InlineMetadata::from_bytes(&bytes)?;
Ok(())
}
```
## 🏗️ Architecture
### Metadata Storage Layout
```
FileMeta Architecture:
┌─────────────────────────────────────────────────────────────┐
│ Metadata API Layer │
├─────────────────────────────────────────────────────────────┤
│ XL Metadata │ Inline Metadata │ Version Info │
├─────────────────────────────────────────────────────────────┤
│ FlatBuffers Serialization │
├─────────────────────────────────────────────────────────────┤
│ Compression │ Validation │ Migration │
├─────────────────────────────────────────────────────────────┤
│ Storage Backend Integration │
└─────────────────────────────────────────────────────────────┘
```
### Metadata Types
| Type | Use Case | Storage | Performance |
|------|----------|---------|-------------|
| XLMeta | Large objects with rich metadata | Separate file | High durability |
| InlineMeta | Small objects with minimal metadata | Embedded | Fastest access |
| VersionMeta | Object versioning information | Version-specific | Version-aware |
## 🧪 Testing
Run the test suite:
```bash
# Run all tests
cargo test
# Run serialization benchmarks
cargo bench
# Test metadata validation
cargo test validation
# Test schema migration
cargo test migration
```
## 🚀 Performance
FileMeta is optimized for high-performance metadata operations:
- **Serialization**: Zero-copy FlatBuffers serialization
- **Storage**: Compact binary format reduces I/O
- **Caching**: Intelligent metadata caching
- **Batch Operations**: Efficient bulk metadata processing
## 📋 Requirements
- **Rust**: 1.70.0 or later
- **Platforms**: Linux, macOS, Windows
- **Memory**: Minimal memory footprint
- **Storage**: Compatible with RustFS storage backend
## 🌍 Related Projects
This module is part of the RustFS ecosystem:
- [RustFS Main](https://github.com/rustfs/rustfs) - Core distributed storage system
- [RustFS ECStore](../ecstore) - Erasure coding storage engine
- [RustFS Utils](../utils) - Utility functions
- [RustFS Proto](../protos) - Protocol definitions
- High-performance metadata storage and retrieval
- Advanced indexing with full-text search capabilities
- File attribute management and custom metadata
- Version tracking and history management
- Distributed metadata replication
- Real-time metadata synchronization
## 📚 Documentation
For comprehensive documentation, visit:
- [RustFS Documentation](https://docs.rustfs.com)
- [FileMeta API Reference](https://docs.rustfs.com/filemeta/)
## 🔗 Links
- [Documentation](https://docs.rustfs.com) - Complete RustFS manual
- [Changelog](https://github.com/rustfs/rustfs/releases) - Release notes and updates
- [GitHub Discussions](https://github.com/rustfs/rustfs/discussions) - Community support
## 🤝 Contributing
We welcome contributions! Please see our [Contributing Guide](https://github.com/rustfs/rustfs/blob/main/CONTRIBUTING.md) for details.
For comprehensive documentation, examples, and usage guides, please visit the main [RustFS repository](https://github.com/rustfs/rustfs).
## 📄 License
Licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/rustfs/rustfs/blob/main/LICENSE) for details.
---
<p align="center">
<strong>RustFS</strong> is a trademark of RustFS, Inc.<br>
All other trademarks are the property of their respective owners.
</p>
<p align="center">
Made with ❤️ by the RustFS Team
</p>
This project is licensed under the Apache License 2.0 - see the [LICENSE](../../LICENSE) file for details.