gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-06 13:27:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: implement Docker improvements and binary build scripts (#191)

Browse Source 
* feat: implement Docker improvements and binary build scripts

This commit transforms the RustFS Docker build system to follow MinIO's best practices:

## 🏗️ Binary Build Script (build-rustfs.sh)
- Create independent binary compilation script for multi-platform builds
- Support x86_64 and aarch64 Linux musl targets
- Include checksum generation and optional binary signing
- Support cross-compilation and upload functionality
- Automated target installation and environment setup

## 🐳 Docker Improvements
- Rewrite Dockerfiles to download precompiled binaries instead of building from source
- Follow MinIO's approach for security and binary verification
- Add comprehensive LABEL metadata (version, build-date, vcs-ref)
- Implement proper environment variable management
- Add signature verification with minisign (commented for future use)
- Include static curl download for minimal runtime dependencies

## 🚀 Enhanced Build Script (docker-buildx.sh)
- Inspired by MinIO's docker-buildx.sh for consistency and reliability
- Support multiple platforms with proper build arguments
- Auto-detect git versions and pass metadata to containers
- Improved error messages with helpful troubleshooting hints
- Cleanup and cache management between builds

## 🛠️ Supporting Scripts
- scripts/download-static-curl.sh: Download statically compiled curl
- scripts/setup-test-binaries.sh: Create test binaries for local development

## 📋 Key Benefits
- Faster Docker builds (download vs compile)
- Better security with signature verification
- Consistent with industry standards (MinIO approach)
- Proper multi-platform support
- Enhanced metadata and traceability
- Independent binary distribution capability

* feat: update Docker files to use Aliyun OSS for binary downloads

* feat: merge stash with OSS binary download improvements

- Remove old build_rustfs.sh script
- Keep Aliyun OSS download URLs for binary retrieval
- Maintain Docker build improvements from stash
- Resolve merge conflicts between stash and OSS updates

* feat: improve build-rustfs.sh with auto platform detection

- Auto-detect current platform using uname (like old build_rustfs.sh)
- Default to building for current platform only
- Add --all-platforms flag for cross-compilation to Linux musl targets
- Support macOS (darwin) and Linux platforms
- Auto-enable cross compilation when needed
- Provide better usage examples and platform detection info

This makes the script much more user-friendly by default while
maintaining flexibility for cross-compilation scenarios.

* refactor: simplify build-rustfs.sh for CI/CD pipeline usage

- Remove cross-compilation complexity (each CI runner builds natively)
- Focus on single platform builds per runner
- Remove --all-platforms and --cross options
- Simplify to match CI/CD workflow where:
  * Linux x86_64 runner builds Linux x86_64 binary
  * Linux ARM64 runner builds Linux ARM64 binary
  * macOS x86_64 runner builds macOS x86_64 binary
  * macOS ARM64 runner builds macOS ARM64 binary
- Keep signing and upload functionality for release CI
- Make the script's purpose and usage clearer

This aligns with the user's understanding that build scripts should
focus on native compilation for the current platform only.

* feat: update download server domain to dl.rustfs.com

- Update Dockerfile to use dl.rustfs.com/dev/ for development binaries
- Update Dockerfile.release to use dl.rustfs.com/release/ for release binaries
- Update docker-buildx.sh error messages with new URLs
- Update build-rustfs.sh upload target to dl.rustfs.com
- Update test scripts to reference new domain
- Clean up remaining git conflict markers

This centralizes all binary downloads through the official
dl.rustfs.com domain instead of direct OSS access.

* fix: correct dl.rustfs.com path structure to include /artifacts/rustfs/

- Update all download URLs to use correct path structure:
  * Dev: https://dl.rustfs.com/artifacts/rustfs/dev/
  * Release: https://dl.rustfs.com/artifacts/rustfs/release/
- Test confirmed both paths return HTTP 200 with application/zip content-type
- Update Dockerfile, Dockerfile.release, docker-buildx.sh, and build-rustfs.sh
- Update test scripts with correct base path

The dl.rustfs.com domain requires the /artifacts/rustfs/ prefix
to access the binary files correctly.

* feat: refactor Dockerfile to download binaries from GitHub Releases

- Changed binary download source from dl.rustfs.com to GitHub Releases
- Added support for latest release auto-detection via GitHub API
- Enhanced error handling with detailed messages and helpful links
- Added optional checksum verification using SHA256SUMS
- Improved architecture support for amd64 and arm64
- Removed unnecessary minisign installation
- Added jq dependency for JSON parsing

* feat: consolidate Docker build to use single Dockerfile

- Removed Dockerfile.release and use unified Dockerfile instead
- Updated docker-buildx.sh to use single Dockerfile with build args
- Both latest and release variants now use GitHub Releases
- Simplified build process and reduced maintenance overhead
- Updated error messages to point to GitHub releases

* chore: remove unused Dockerfile.obs

- Removed Dockerfile.obs as it's no longer needed
- Simplified Docker build configuration

* feat: unify Docker prebuild variants to use GitHub Releases

- Updated .docker/alpine/Dockerfile.prebuild to download from GitHub Releases
- Updated .docker/ubuntu/Dockerfile.prebuild to download from GitHub Releases
- All prebuild variants now consistently use GitHub Releases as binary source
- Added checksum verification for all prebuild variants
- Updated .docker/README.md to reflect unified GitHub Releases approach
- Improved error handling and user guidance in all prebuild Dockerfiles

* feat: major Docker structure simplification and consolidation

## 🎯 Simplified Docker Structure

Moved from complex multi-directory structure to clean root-level organization:

### Before:
- Dockerfile (production)
- .docker/alpine/Dockerfile.prebuild (duplicate)
- .docker/alpine/Dockerfile.source
- .docker/ubuntu/Dockerfile.prebuild (duplicate)
- .docker/ubuntu/Dockerfile.source
- .docker/ubuntu/Dockerfile.dev

### After:
- Dockerfile (production - Alpine + GitHub Releases)
- Dockerfile.source (source build - Ubuntu + cross-compilation)
- Dockerfile.dev (development - Ubuntu + full toolchain)

## 🔧 Key Changes

- **Eliminated Duplicates**: Removed redundant prebuild variants
- **Moved Core Files**: Dockerfile.{source,dev} now in root directory
- **Unified Configuration**: cargo.config.toml moved to root
- **Updated References**: Fixed all GitHub Actions and docker-compose paths
- **Simplified CI Matrix**: Reduced from 5 to 3 Docker variants

## 📦 Preserved Valuable Diversity

- **Production**: Alpine-based for minimal size
- **Source**: Ubuntu-based with cross-compilation support
- **Development**: Ubuntu-based with full development tools

## 🚀 Benefits

- ✅ Cleaner project structure
- ✅ Easier maintenance and navigation
- ✅ Reduced CI/CD complexity
- ✅ Faster build matrix execution
- ✅ Maintained functionality and flexibility

* chore: remove duplicate cargo.config.toml from .docker directory

The file is now in the root directory and no longer needed in .docker/

* fix: update all references to removed Dockerfile files

- Updated .docker/compose/README.md to reference Dockerfile.source instead of Dockerfile.obs
- Updated docker-compose.yml to use Dockerfile.source instead of Dockerfile.dev
- Updated scripts/build-docker-multiarch.sh to use Dockerfile.source for devenv builds
- Updated .github/workflows/docker.yml to use Dockerfile.source for dev builds
- Updated Makefile to use Dockerfile.source for init-devenv target
- Updated .docker/README.md to remove references to non-existent Dockerfile.dev
- Ensured all Docker configurations consistently use the unified Dockerfile structure

* chore: remove unnecessary console static assets download

- Remove obsolete download steps from build.yml and performance.yml
- Console static assets are already embedded via rust-embed in rustfs/static/
- The download from dl.rustfs.com is no longer needed as project contains complete console assets
- This improves build reliability and reduces external dependencies
- Replaced with verification steps that confirm embedded assets are present

* feat: update Makefile and README.md for new Docker build system

- Updated Makefile to use unified Docker build system:
  - Replace references to non-existent Dockerfile.ubuntu22.04 and Dockerfile.rockylinux9.3
  - Add new docker-buildx targets using docker-buildx.sh script
  - Deprecate old docker-build-multiarch targets with warnings
  - Add docker-build-production and docker-build-source targets
  - Update help-docker with new command structure

- Updated README.md with docker-buildx.sh usage:
  - Add comprehensive Docker build from source section
  - Document multi-architecture build capabilities
  - Include both script and Make target examples
  - Show registry flexibility and build optimization features
  - Update step numbers in quickstart guide

- Improve developer experience with clear documentation and updated tooling
- Maintain backward compatibility with deprecation warnings

* feat: integrate console assets download into build-rustfs.sh

- Added console download functionality to build-rustfs.sh:
  - New flags: --download-console, --force-console-update, --console-version
  - Intelligent detection of existing console assets
  - Retry logic with fallback error handling
  - Consistent with Docker build asset management

- Updated scripts to use unified build process:
  - scripts/static.sh: Now uses build-rustfs.sh for console downloads
  - scripts/run.sh: Uses build-rustfs.sh instead of direct curl
  - scripts/run.ps1: Updated with guidance for Windows users

- Benefits:
  - Unified asset management across all build processes
  - Consistent version handling and retry logic
  - Eliminates duplicate download logic
  - Better error handling and user feedback
  - Preparation for CI/CD integration

- Removed unused download-static-curl.sh script

This change centralizes console asset management and prepares for
streamlined CI/CD processes where build-rustfs.sh becomes the
single point of truth for binary and asset builds.

* fix: update PowerShell script to use unified console asset management

- Updated scripts/run.ps1 to use build-rustfs.sh for console asset downloads
- Added guidance for Windows users to use the unified build script
- Maintains consistency across all platform-specific scripts

* feat: add binary verification to build script

- Add verify_binary function to test built binaries
- Test --help and --version commands
- Verify binary structure with readelf/otool
- Add --skip-verification option for cross-compilation
- Include verification status in build output
- Automatic error handling if verification fails

* feat: add platform selection support to build script

- Add --platform parameter to build-rustfs.sh for target platform selection
- Implement cross-compilation support with automatic 'cross' tool detection
- Auto-enable --skip-verification for cross-compilation scenarios
- Update all Makefile build targets to use unified build-rustfs.sh script
- Add helpful error messages and suggestions for cross-compilation failures
- Update help documentation with platform selection examples
- Improve build consistency across different architectures

* feat: modernize CI/CD build process with build-rustfs.sh

- Replace manual cargo build commands with unified build-rustfs.sh script
- Simplify matrix configuration by removing cross-compilation flags
- Ensure consistency between local and CI/CD builds
- Automatic cross-compilation tool detection and installation
- Built-in binary verification for quality assurance
- Unified console asset management
- Better error handling and suggestions

Benefits:
- Consistent build process across all environments
- Automatic detection and handling of cross-compilation scenarios
- Built-in quality checks with binary verification
- Reduced CI/CD configuration complexity
- Better maintainability with single source of truth for build logic

* feat: optimize CI/CD workspace path management

- Add WORKSPACE_DIR environment variable to cache github.workspace
- Set default working-directory at job level for consistency
- Use explicit workspace paths in critical operations
- Improve reliability and maintainability of CI/CD paths
- Ensure consistent behavior across different GitHub Actions environments

Benefits:
- More explicit and reliable path handling
- Better maintainability with centralized workspace reference
- Reduced risk of path-related issues in CI/CD
- Consistent working directory across all job steps

* refactor: simplify CI/CD path management - remove redundant workspace references

- Remove unnecessary WORKSPACE_DIR environment variable
- Remove redundant defaults.run.working-directory setting
- Use relative paths since GITHUB_WORKSPACE is the default working directory
- Follow GitHub Actions best practices by leveraging default behavior

As per GitHub Actions documentation, GITHUB_WORKSPACE is already the default
working directory, so explicit specification is unnecessary in most cases.

* docs: update Docker README to reflect current project state

- Fix directory structure: remove non-existent nginx/ directory
- Correct base OS: Dockerfile.source uses Debian Bookworm, not Ubuntu 22.04
- Add docker-buildx.sh script documentation
- Update Docker tag examples to match actual CI/CD workflows
- Add CI/CD integration section explaining automated builds
- Document build variants and manual build options
- Reflect current project architecture and tooling

These updates ensure the documentation accurately represents the current
Docker build system and CI/CD workflows.

* fix: update Docker command in rustfs README

- Replace quay.io registry with Docker Hub (rustfs/rustfs:latest)
- Remove separate console port 9001, console now runs on main port 9000
- Add both Docker and Podman examples for user choice
- Fix console access URL to use unified port

This aligns with the recent console port consolidation changes
and the project's move to Docker Hub as the primary registry.

* wip

* fix: remove unnecessary entrypoint.sh and fix Docker paths

* Update Dockerfile

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* cleanup: remove unused DOCKERFILE_PATH variable from Makefile

* feat: update Docker build to use dl.rustfs.com for binary downloads

- Replace GitHub releases download with dl.rustfs.com
- Add CHANNEL parameter support (release/dev)
- Update docker-buildx.sh to support channel-specific builds
- Improve error messages with new download URLs
- Support both latest and specific version downloads
- Add channel validation in build script

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
安正超
2025-07-14 19:15:46 +08:00
committed by GitHub
parent 04d1c8724d
commit 36e3efb5a5
27 changed files with 1225 additions and 1252 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.docker/README.md
+140 -98
View File
@@ -1,157 +1,199 @@
# RustFS Docker Images
This directory contains organized Dockerfile configurations for building RustFS container images across multiple platforms and system versions.
This directory contains Docker configuration files and supporting infrastructure for building and running RustFS container images.
## 📁 Directory Structure
```
.docker/
├── alpine/ # Alpine Linux variants
│ ├── Dockerfile.prebuild # Alpine + pre-built binaries
│ └── Dockerfile.source # Alpine + source compilation
├── ubuntu/ # Ubuntu variants
│ ├── Dockerfile.prebuild # Ubuntu + pre-built binaries
│ ├── Dockerfile.source # Ubuntu + source compilation
│ └── Dockerfile.dev # Ubuntu + development environment
└── cargo.config.toml # Rust cargo configuration
rustfs/
├── Dockerfile # Production image (Alpine + GitHub Releases)
├── Dockerfile.source # Source build (Debian + cross-compilation)
├── cargo.config.toml # Rust cargo configuration
├── docker-buildx.sh # Multi-architecture build script
└── .docker/ # Supporting infrastructure
├── observability/ # Monitoring and observability configs
├── compose/ # Docker Compose configurations
├── mqtt/ # MQTT broker configs
└── openobserve-otel/ # OpenObserve + OpenTelemetry configs
```
## 🎯 Image Variants
### Production Images
### Core Images
| Variant | Base OS | Build Method | Size | Use Case |
|---------|---------|--------------|------|----------|
| `production` (default) | Alpine 3.18 | Pre-built | Smallest | Production deployment |
| `alpine` | Alpine 3.18 | Pre-built | Small | Explicit Alpine choice |
| `alpine-source` | Alpine 3.18 | Source build | Small | Custom Alpine builds |
| `ubuntu` | Ubuntu 22.04 | Pre-built | Medium | Ubuntu environments |
| `ubuntu-source` | Ubuntu 22.04 | Source build | Medium | Full Ubuntu compatibility |
### Development Images
| Variant | Base OS | Features | Use Case |
|---------|---------|----------|----------|
| `ubuntu-dev` | Ubuntu 22.04 | Full toolchain + dev tools | Interactive development |
| Image | Base OS | Build Method | Size | Use Case |
|-------|---------|--------------|------|----------|
| `production` (default) | Alpine 3.18 | GitHub Releases | Smallest | Production deployment |
| `source` | Debian Bookworm | Source build | Medium | Custom builds with cross-compilation |
| `dev` | Debian Bookworm | Development tools | Large | Interactive development |
## 🚀 Usage Examples
### Quick Start (Production)
```bash
# Default production image (Alpine + pre-built)
# Default production image (Alpine + GitHub Releases)
docker run -p 9000:9000 rustfs/rustfs:latest
# Specific version with production variant
docker run -p 9000:9000 rustfs/rustfs:1.2.3-production
# Explicit Alpine variant
docker run -p 9000:9000 rustfs/rustfs:latest-alpine
# Ubuntu-based production
docker run -p 9000:9000 rustfs/rustfs:latest-ubuntu
# Specific version
docker run -p 9000:9000 rustfs/rustfs:1.2.3
```
### Complete Tag Strategy Examples
```bash
# Stable Releases
docker run rustfs/rustfs:1.2.3 # Main version (production)
docker run rustfs/rustfs:1.2.3-production # Explicit production variant
docker run rustfs/rustfs:1.2.3-alpine # Explicit Alpine variant
docker run rustfs/rustfs:1.2.3-alpine-source # Alpine source build
docker run rustfs/rustfs:latest # Latest stable
docker run rustfs/rustfs:1.2.3 # Main version (production)
docker run rustfs/rustfs:1.2.3-production # Explicit production variant
docker run rustfs/rustfs:1.2.3-source # Source build variant
docker run rustfs/rustfs:latest # Latest stable
# Prerelease Versions
docker run rustfs/rustfs:1.3.0-alpha.2 # Specific alpha version
docker run rustfs/rustfs:1.3.0-alpha.2-alpine # Alpha with Alpine
docker run rustfs/rustfs:alpha # Latest alpha
docker run rustfs/rustfs:beta # Latest beta
docker run rustfs/rustfs:rc # Latest release candidate
docker run rustfs/rustfs:1.3.0-alpha.2 # Specific alpha version
docker run rustfs/rustfs:alpha # Latest alpha
docker run rustfs/rustfs:beta # Latest beta
docker run rustfs/rustfs:rc # Latest release candidate
# Development Versions
docker run rustfs/rustfs:dev # Latest development
docker run rustfs/rustfs:dev-13e4a0b # Specific commit
docker run rustfs/rustfs:dev-alpine # Development Alpine
docker run rustfs/rustfs:dev # Latest main branch development
docker run rustfs/rustfs:dev-13e4a0b # Specific commit
docker run rustfs/rustfs:dev-latest # Latest development
docker run rustfs/rustfs:main-latest # Main branch latest
```
### Development Environment
```bash
# Start development container
docker run -it -v $(pwd):/app -p 9000:9000 rustfs/rustfs:latest-ubuntu-dev
docker run -it -v $(pwd):/workspace -p 9000:9000 rustfs/rustfs:latest-dev
# Inside container:
cd /app
cargo build --release
cargo run
# Build from source locally
docker build -f Dockerfile.source -t rustfs:custom .
# Development with hot reload
docker-compose up rustfs-dev
```
## 🏗️ Build Arguments
## 🏗️ Build Arguments and Scripts
### Using docker-buildx.sh (Recommended)
For multi-architecture builds, use the provided script:
```bash
# Build latest version for all architectures
./docker-buildx.sh
# Build and push to registry
./docker-buildx.sh --push
# Build specific version
./docker-buildx.sh --release v1.2.3
# Build and push specific version
./docker-buildx.sh --release v1.2.3 --push
```
### Manual Docker Builds
All images support dynamic version selection:
```bash
# Build with specific version
docker build \
--build-arg VERSION="1.0.0" \
--build-arg BUILD_TYPE="release" \
-f .docker/alpine/Dockerfile.prebuild \
-t rustfs:1.0.0-alpine .
# Build production image with latest release
docker build --build-arg RELEASE="latest" -t rustfs:latest .
# Build from source with specific target
docker build -f Dockerfile.source \
--build-arg TARGETPLATFORM="linux/amd64" \
-t rustfs:source .
# Development build
docker build -f Dockerfile.source -t rustfs:dev .
```
## 🌐 Multi-Platform Support
## 🔧 Binary Download Sources
All images support multiple architectures:
### Unified GitHub Releases
- `linux/amd64` (Intel/AMD 64-bit)
- `linux/arm64` (ARM 64-bit, Apple Silicon, etc.)
The production image downloads from GitHub Releases for reliability and transparency:
## ⚡ Build Speed Optimizations
- ✅ **production** → GitHub Releases API with automatic latest detection
- ✅ **Checksum verification** → SHA256SUMS validation when available
- ✅ **Multi-architecture** → Supports amd64 and arm64
### Docker Build Optimizations
### Source Build
- **Multi-layer caching**: GitHub Actions cache + Registry cache
- **Parallel matrix builds**: All 5 variants build simultaneously
- **Multi-platform builds**: amd64/arm64 built in parallel
- **BuildKit features**: Advanced caching and inline cache
The source variant compiles from source code with advanced features:
### Rust Compilation Optimizations
- 🔧 **Cross-compilation** → Supports multiple target platforms via `TARGETPLATFORM`
- ⚡ **Build caching** → sccache for faster compilation
- 🎯 **Optimized builds** → Release optimizations with LTO and symbol stripping
- **sccache**: Distributed compilation cache for Rust builds
- **Parallel compilation**: Uses all available CPU cores (`-j $(nproc)`)
- **Optimized cargo config**: Sparse registry protocol, fast linker (lld)
- **Dependency caching**: Separate Docker layers for dependencies vs. source code
- **Release optimizations**: LTO, strip symbols, optimized codegen
## 📋 Architecture Support
### Cache Strategy
All variants support multi-architecture builds:
```yaml
# GitHub Actions cache
cache-from: type=gha,scope=docker-{variant}
cache-to: type=gha,mode=max,scope=docker-{variant}
- **linux/amd64** (x86_64)
- **linux/arm64** (aarch64)
# Registry cache (persistent across runs)
cache-from: type=registry,ref=ghcr.io/rustfs/rustfs:buildcache-{variant}
cache-to: type=registry,ref=ghcr.io/rustfs/rustfs:buildcache-{variant}
Architecture is automatically detected during build using Docker's `TARGETARCH` build argument.
## 🔐 Security Features
- **Checksum Verification**: Production image verifies SHA256SUMS when available
- **Non-root User**: All images run as user `rustfs` (UID 1000)
- **Minimal Runtime**: Production image only includes necessary dependencies
- **Secure Defaults**: No hardcoded credentials or keys
## 🛠️ Development Workflow
For local development and testing:
```bash
# Quick development setup
docker-compose up rustfs-dev
# Custom source build
docker build -f Dockerfile.source -t rustfs:custom .
# Run with development tools
docker run -it -v $(pwd):/workspace rustfs:custom bash
```
### Build Performance Comparison
## 🚀 CI/CD Integration
| Build Type | Time (Est.) | Cache Hit | Cache Miss |
|------------|-------------|-----------|-----------|
| Production (Alpine pre-built) | ~2-3 min | ~1 min | ~2 min |
| Alpine pre-built | ~2-3 min | ~1 min | ~2 min |
| Alpine source | ~8-12 min | ~3-5 min | ~10 min |
| Ubuntu pre-built | ~3-4 min | ~1-2 min | ~3 min |
| Ubuntu source | ~10-15 min | ~4-6 min | ~12 min |
The project uses GitHub Actions for automated multi-architecture Docker builds:
## 📋 Build Matrix
### Automated Builds
| Trigger | Version Format | Download Path | Image Tags |
|---------|---------------|---------------|------------|
| `push main` | `dev-{sha}` | `artifacts/rustfs/dev/` | `dev-{sha}-{variant}`, `dev-{variant}`, `dev` |
| `push 1.2.3` | `1.2.3` | `artifacts/rustfs/release/` | `1.2.3-{variant}`, `1.2.3`, `latest-{variant}`, `latest` |
| `push 1.3.0-alpha.2` | `1.3.0-alpha.2` | `artifacts/rustfs/release/` | `1.3.0-alpha.2-{variant}`, `alpha-{variant}`, `alpha` |
| `push 1.3.0-beta.1` | `1.3.0-beta.1` | `artifacts/rustfs/release/` | `1.3.0-beta.1-{variant}`, `beta-{variant}`, `beta` |
| `push 1.3.0-rc.1` | `1.3.0-rc.1` | `artifacts/rustfs/release/` | `1.3.0-rc.1-{variant}`, `rc-{variant}`, `rc` |
- **Tags**: Automatic builds triggered on version tags (e.g., `v1.2.3`)
- **Main Branch**: Development builds with `dev-latest` and `main-latest` tags
- **Pull Requests**: Test builds without registry push
### Build Variants
Each build creates three image variants:
- `rustfs/rustfs:v1.2.3` (production - Alpine-based)
- `rustfs/rustfs:v1.2.3-source` (source build - Debian-based)
- `rustfs/rustfs:v1.2.3-dev` (development - Debian-based with tools)
### Manual Builds
Trigger custom builds via GitHub Actions:
```bash
# Use workflow_dispatch to build specific versions
# Available options: latest, main-latest, dev-latest, v1.2.3, dev-abc123
```
## 📦 Supporting Infrastructure
The `.docker/` directory contains supporting configuration files:
- **observability/** - Prometheus, Grafana, OpenTelemetry configs
- **compose/** - Multi-service Docker Compose setups
- **mqtt/** - MQTT broker configurations
- **openobserve-otel/** - Log aggregation and tracing setup
See individual README files in each subdirectory for specific usage instructions.
.docker/alpine/Dockerfile.prebuild
-142
View File
@@ -1,142 +0,0 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Multi-stage Alpine build for minimal runtime image
FROM rust:1.88-alpine AS builder
# Build arguments for dynamic artifact download
ARG VERSION=""
ARG BUILD_TYPE="release"
ARG TARGETARCH
# Install build dependencies
RUN apk add --no-cache \
musl-dev \
pkgconfig \
openssl-dev \
openssl-libs-static \
curl \
unzip \
bash \
wget \
ca-certificates
# Install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v31.1/protoc-31.1-linux-x86_64.zip \
&& unzip protoc-31.1-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc \
&& mv protoc3/include/* /usr/local/include/ && rm -rf protoc-31.1-linux-x86_64.zip protoc3
# Install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc \
&& rm -rf Linux.flatc.binary.g++-13.zip
# Download pre-built binary (VERSION is required)
RUN if [ -z "$VERSION" ]; then \
echo "❌ ERROR: VERSION build argument is required"; \
echo "Please provide VERSION (e.g., main-latest, latest, v1.0.0, dev-abc123)"; \
exit 1; \
fi; \
\
# Map TARGETARCH to our naming convention
case "${TARGETARCH}" in \
amd64) ARCH="x86_64" ;; \
arm64) ARCH="aarch64" ;; \
*) echo "❌ Unsupported architecture: ${TARGETARCH}" && exit 1 ;; \
esac; \
\
# Handle VERSION with different patterns
if [[ "$VERSION" == "main-latest" ]]; then \
DOWNLOAD_PATH="artifacts/rustfs/dev"; \
FILENAME="rustfs-linux-${ARCH}-main-latest.zip"; \
elif [[ "$VERSION" == "latest" ]]; then \
DOWNLOAD_PATH="artifacts/rustfs/release"; \
FILENAME="rustfs-linux-${ARCH}-latest.zip"; \
elif [[ "$VERSION" == dev-* ]]; then \
DOWNLOAD_PATH="artifacts/rustfs/dev"; \
FILENAME="rustfs-linux-${ARCH}-dev-${VERSION#dev-}.zip"; \
else \
DOWNLOAD_PATH="artifacts/rustfs/release"; \
FILENAME="rustfs-linux-${ARCH}-v${VERSION}.zip"; \
fi; \
\
# Download the binary
DOWNLOAD_URL="https://dl.rustfs.com/${DOWNLOAD_PATH}/${FILENAME}"; \
echo "🔽 Downloading RustFS binary from: ${DOWNLOAD_URL}"; \
\
# Download with clear error handling
if ! curl -fsSL --connect-timeout 30 --max-time 120 -o /tmp/rustfs.zip "${DOWNLOAD_URL}"; then \
echo "❌ Failed to download binary from: ${DOWNLOAD_URL}"; \
echo "💡 Please ensure the binary exists or trigger a build first"; \
echo "💡 Available options:"; \
echo " - For main-latest: Push to main branch or run build workflow"; \
echo " - For latest: Create a release tag"; \
echo " - For dev-xxx: Run build workflow with specific commit"; \
exit 1; \
fi; \
\
# Extract binary
if ! unzip -o /tmp/rustfs.zip -d /tmp >/dev/null 2>&1; then \
echo "❌ Failed to extract downloaded binary"; \
exit 1; \
fi; \
\
# Install binary
mv /tmp/rustfs /usr/local/bin/rustfs; \
chmod +x /usr/local/bin/rustfs; \
rm -rf /tmp/*; \
\
echo "✅ Successfully downloaded and installed RustFS binary (${VERSION})"
# Final Alpine runtime image
FROM alpine:3.18
RUN apk add --no-cache \
ca-certificates \
tzdata \
bash
# Create rustfs user for security
RUN addgroup -g 1000 rustfs && \
adduser -D -u 1000 -G rustfs rustfs
WORKDIR /app
# Copy binary from builder
COPY --from=builder /usr/local/bin/rustfs /app/rustfs
RUN chmod +x /app/rustfs && chown rustfs:rustfs /app/rustfs
# Create data directories
RUN mkdir -p /data && chown -R rustfs:rustfs /data /app
# Switch to non-root user
USER rustfs
# Environment variables
ENV RUSTFS_ACCESS_KEY=rustfsadmin \
RUSTFS_SECRET_KEY=rustfsadmin \
RUSTFS_ADDRESS=":9000" \
RUSTFS_CONSOLE_ENABLE=true \
RUSTFS_VOLUMES=/data \
RUST_LOG=warn
EXPOSE 9000
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:9000/health || exit 1
CMD ["/app/rustfs"]
.docker/alpine/Dockerfile.source
-126
View File
@@ -1,126 +0,0 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Multi-stage Alpine build from source
FROM rust:1.88-alpine AS builder
# Install build dependencies
RUN apk add --no-cache \
musl-dev \
pkgconfig \
openssl-dev \
openssl-libs-static \
curl \
unzip \
bash \
wget \
ca-certificates \
git
# Install sccache for Rust compilation caching
RUN wget https://github.com/mozilla/sccache/releases/download/v0.8.1/sccache-v0.8.1-x86_64-unknown-linux-musl.tar.gz \
&& tar -xzf sccache-v0.8.1-x86_64-unknown-linux-musl.tar.gz \
&& mv sccache-v0.8.1-x86_64-unknown-linux-musl/sccache /usr/local/bin/ \
&& chmod +x /usr/local/bin/sccache \
&& rm -rf sccache-v0.8.1-x86_64-unknown-linux-musl.tar.gz sccache-v0.8.1-x86_64-unknown-linux-musl
# Set up sccache environment
ENV RUSTC_WRAPPER=sccache \
SCCACHE_DIR=/tmp/sccache \
SCCACHE_CACHE_SIZE=2G
# Install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v31.1/protoc-31.1-linux-x86_64.zip \
&& unzip protoc-31.1-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc \
&& mv protoc3/include/* /usr/local/include/ && rm -rf protoc-31.1-linux-x86_64.zip protoc3
# Install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc \
&& rm -rf Linux.flatc.binary.g++-13.zip
WORKDIR /usr/src/rustfs
# Copy cargo configuration for optimized builds
COPY .docker/cargo.config.toml ./.cargo/config.toml
# Copy cargo files for dependency caching
COPY Cargo.toml Cargo.lock ./
COPY */Cargo.toml ./*/
# Create dummy main.rs files for dependency compilation
RUN find . -name "Cargo.toml" -not -path "./Cargo.toml" | \
xargs -I {} dirname {} | \
xargs -I {} sh -c 'mkdir -p {}/src && echo "fn main() {}" > {}/src/main.rs'
# Configure cargo for optimized builds
ENV CARGO_NET_GIT_FETCH_WITH_CLI=true \
CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse \
CARGO_INCREMENTAL=0 \
CARGO_PROFILE_RELEASE_DEBUG=false \
CARGO_PROFILE_RELEASE_SPLIT_DEBUGINFO=off \
CARGO_PROFILE_RELEASE_STRIP=symbols
# Build dependencies only (cache layer) with optimizations
RUN cargo build --release --target x86_64-unknown-linux-musl -j $(nproc)
# Copy source code
COPY . .
# Build the actual application with optimizations
RUN sccache --start-server 2>/dev/null || true && \
cargo build --release --target x86_64-unknown-linux-musl --bin rustfs -j $(nproc) && \
sccache --show-stats || true
# Final Alpine runtime image
FROM alpine:3.18
RUN apk add --no-cache \
ca-certificates \
tzdata \
bash
# Create rustfs user for security
RUN addgroup -g 1000 rustfs && \
adduser -D -u 1000 -G rustfs rustfs
WORKDIR /app
# Copy binary from builder
COPY --from=builder /usr/src/rustfs/target/x86_64-unknown-linux-musl/release/rustfs /app/rustfs
RUN chmod +x /app/rustfs && chown rustfs:rustfs /app/rustfs
# Create data directories
RUN mkdir -p /data && chown -R rustfs:rustfs /data /app
# Switch to non-root user
USER rustfs
# Environment variables
ENV RUSTFS_ACCESS_KEY=rustfsadmin \
RUSTFS_SECRET_KEY=rustfsadmin \
RUSTFS_ADDRESS=":9000" \
RUSTFS_CONSOLE_ENABLE=true \
RUSTFS_VOLUMES=/data \
RUST_LOG=warn
EXPOSE 9000
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:9000/health || exit 1
CMD ["/app/rustfs"]
.docker/cargo.config.toml
-55
View File
@@ -1,55 +0,0 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Build optimization settings
[build]
# Use all available CPU cores for parallel compilation
jobs = 0 # 0 = use all cores
[target.x86_64-unknown-linux-musl]
# Use lld linker for faster linking
linker = "lld"
[target.x86_64-unknown-linux-gnu]
# Use lld linker for faster linking
linker = "lld"
[target.aarch64-unknown-linux-gnu]
# Use lld linker for faster linking
linker = "lld"
[profile.release]
# Optimize for size and speed
codegen-units = 1
lto = true
panic = "abort"
strip = true
debug = false
[profile.dev]
# Faster incremental builds during development
incremental = true
debug = true
[registry]
# Use sparse registry protocol for faster dependency resolution
default = "sparse+https://index.crates.io/"
[source.crates-io]
# Use sparse registry protocol
replace-with = "sparse+https://index.crates.io/"
[net]
# Use git CLI for better performance with private repos
git-fetch-with-cli = true
.docker/compose/README.md
+1 -1
View File
@@ -25,7 +25,7 @@ This directory contains specialized Docker Compose configurations and their asso
- **`docker-compose.observability.yaml`** - **Observability Focus**
- Specialized setup for testing observability features
- Includes OpenTelemetry, Jaeger, Prometheus, Loki, Grafana
- Uses `../../Dockerfile.obs` for builds
- Uses `../../Dockerfile.source` for builds
- Perfect for observability development
## 🚀 Usage Examples
.docker/compose/docker-compose.observability.yaml
+4 -4
View File
@@ -73,7 +73,7 @@ services:
node1:
build:
context: ../..
dockerfile: Dockerfile.obs
dockerfile: Dockerfile.source
container_name: node1
environment:
- RUSTFS_VOLUMES=http://node{1...4}:9000/root/data/target/volume/test{1...4}
@@ -90,7 +90,7 @@ services:
node2:
build:
context: ../..
dockerfile: Dockerfile.obs
dockerfile: Dockerfile.source
container_name: node2
environment:
- RUSTFS_VOLUMES=http://node{1...4}:9000/root/data/target/volume/test{1...4}
@@ -107,7 +107,7 @@ services:
node3:
build:
context: ../..
dockerfile: Dockerfile.obs
dockerfile: Dockerfile.source
container_name: node3
environment:
- RUSTFS_VOLUMES=http://node{1...4}:9000/root/data/target/volume/test{1...4}
@@ -124,7 +124,7 @@ services:
node4:
build:
context: ../..
dockerfile: Dockerfile.obs
dockerfile: Dockerfile.source
container_name: node4
environment:
- RUSTFS_VOLUMES=http://node{1...4}:9000/root/data/target/volume/test{1...4}
.docker/ubuntu/Dockerfile.dev
-96
View File
@@ -1,96 +0,0 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Ubuntu-based development environment
# Provides full development toolchain for building RustFS from source
FROM ubuntu:22.04
ENV LANG=C.UTF-8
ENV DEBIAN_FRONTEND=noninteractive
# Use faster mirrors for better build performance
RUN sed -i s@http://.*archive.ubuntu.com@http://repo.huaweicloud.com@g /etc/apt/sources.list
# Install development dependencies
RUN apt-get clean && apt-get update && apt-get install -y \
wget \
git \
curl \
unzip \
gcc \
pkg-config \
libssl-dev \
lld \
libdbus-1-dev \
libwayland-dev \
libwebkit2gtk-4.1-dev \
libxdo-dev \
ca-certificates \
bash \
vim \
nano \
htop \
tree \
&& rm -rf /var/lib/apt/lists/*
# Install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v31.1/protoc-31.1-linux-x86_64.zip \
&& unzip protoc-31.1-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc \
&& mv protoc3/include/* /usr/local/include/ && rm -rf protoc-31.1-linux-x86_64.zip protoc3
# Install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc && rm -rf Linux.flatc.binary.g++-13.zip
# Install rust for development
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"
# Install additional Rust tools for development
RUN /root/.cargo/bin/cargo install \
cargo-watch \
cargo-nextest \
cargo-audit \
cargo-outdated
# Copy cargo config for Chinese users
COPY .docker/cargo.config.toml /root/.cargo/config.toml
# Create development user
RUN groupadd -g 1000 rustfs && \
useradd -d /app -g rustfs -u 1000 -s /bin/bash rustfs
WORKDIR /app
# Create data directories for testing
RUN mkdir -p /data && chown -R rustfs:rustfs /data /app
# Environment variables for development
ENV RUSTFS_ACCESS_KEY=devadmin \
RUSTFS_SECRET_KEY=devadmin \
RUSTFS_ADDRESS=":9000" \
RUSTFS_CONSOLE_ENABLE=true \
RUSTFS_VOLUMES=/data \
RUST_LOG=debug \
RUST_BACKTRACE=1
EXPOSE 9000
# Development mode: keep container alive for interactive development
CMD echo "RustFS Development Environment" && \
echo "Source code should be mounted at /app" && \
echo "Use 'cargo build' to build, 'cargo run' to run" && \
exec bash -c "while true; do sleep 1; done"
.docker/ubuntu/Dockerfile.prebuild
-154
View File
@@ -1,154 +0,0 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Multi-purpose Ubuntu-based Dockerfile
# Can be used as development environment or ubuntu runtime variant
FROM ubuntu:22.04
# Build arguments for dynamic artifact download (when used as runtime)
ARG VERSION=""
ARG BUILD_TYPE="release"
ARG TARGETARCH
ENV LANG=C.UTF-8
ENV DEBIAN_FRONTEND=noninteractive
# Use faster mirrors for better build performance
RUN sed -i s@http://.*archive.ubuntu.com@http://repo.huaweicloud.com@g /etc/apt/sources.list
# Install dependencies
RUN apt-get clean && apt-get update && apt-get install -y \
wget \
git \
curl \
unzip \
gcc \
pkg-config \
libssl-dev \
lld \
libdbus-1-dev \
libwayland-dev \
libwebkit2gtk-4.1-dev \
libxdo-dev \
ca-certificates \
bash \
&& rm -rf /var/lib/apt/lists/*
# Install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v31.1/protoc-31.1-linux-x86_64.zip \
&& unzip protoc-31.1-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc \
&& mv protoc3/include/* /usr/local/include/ && rm -rf protoc-31.1-linux-x86_64.zip protoc3
# Install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc && rm -rf Linux.flatc.binary.g++-13.zip
# Install rust (for development use)
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"
# Copy cargo config for Chinese users
COPY .docker/cargo.config.toml /root/.cargo/config.toml
# Download pre-built binary (VERSION is required)
RUN if [ -z "$VERSION" ]; then \
echo "❌ ERROR: VERSION build argument is required"; \
echo "Please provide VERSION (e.g., main-latest, latest, v1.0.0, dev-abc123)"; \
exit 1; \
fi; \
\
# Map TARGETARCH to our naming convention
case "${TARGETARCH}" in \
amd64) ARCH="x86_64" ;; \
arm64) ARCH="aarch64" ;; \
*) echo "❌ Unsupported architecture: ${TARGETARCH}" && exit 1 ;; \
esac; \
\
# Handle VERSION with different patterns
if [[ "$VERSION" == "main-latest" ]]; then \
DOWNLOAD_PATH="artifacts/rustfs/dev"; \
FILENAME="rustfs-linux-${ARCH}-main-latest.zip"; \
elif [[ "$VERSION" == "latest" ]]; then \
DOWNLOAD_PATH="artifacts/rustfs/release"; \
FILENAME="rustfs-linux-${ARCH}-latest.zip"; \
elif [[ "$VERSION" == dev-* ]]; then \
DOWNLOAD_PATH="artifacts/rustfs/dev"; \
FILENAME="rustfs-linux-${ARCH}-dev-${VERSION#dev-}.zip"; \
else \
DOWNLOAD_PATH="artifacts/rustfs/release"; \
FILENAME="rustfs-linux-${ARCH}-v${VERSION}.zip"; \
fi; \
\
# Download the binary
DOWNLOAD_URL="https://dl.rustfs.com/${DOWNLOAD_PATH}/${FILENAME}"; \
echo "🔽 Downloading RustFS binary from: ${DOWNLOAD_URL}"; \
\
# Download with clear error handling
if ! curl -fsSL --connect-timeout 30 --max-time 120 -o /tmp/rustfs.zip "${DOWNLOAD_URL}"; then \
echo "❌ Failed to download binary from: ${DOWNLOAD_URL}"; \
echo "💡 Please ensure the binary exists or trigger a build first"; \
echo "💡 Available options:"; \
echo " - For main-latest: Push to main branch or run build workflow"; \
echo " - For latest: Create a release tag"; \
echo " - For dev-xxx: Run build workflow with specific commit"; \
exit 1; \
fi; \
\
# Extract binary
if ! unzip -o /tmp/rustfs.zip -d /tmp >/dev/null 2>&1; then \
echo "❌ Failed to extract downloaded binary"; \
exit 1; \
fi; \
\
# Install binary
mv /tmp/rustfs /usr/local/bin/rustfs; \
chmod +x /usr/local/bin/rustfs; \
rm -rf /tmp/*; \
\
echo "✅ Successfully downloaded and installed RustFS binary (${VERSION})"
# Create rustfs user for security
RUN groupadd -g 1000 rustfs && \
useradd -d /app -g rustfs -u 1000 -s /bin/bash rustfs
WORKDIR /app
# Create data directories
RUN mkdir -p /data && chown -R rustfs:rustfs /data /app
# Environment variables
ENV RUSTFS_ACCESS_KEY=rustfsadmin \
RUSTFS_SECRET_KEY=rustfsadmin \
RUSTFS_ADDRESS=":9000" \
RUSTFS_CONSOLE_ENABLE=true \
RUSTFS_VOLUMES=/data \
RUST_LOG=warn
EXPOSE 9000
# Health check (only if rustfs binary exists)
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD if [ -f /usr/local/bin/rustfs ]; then wget --no-verbose --tries=1 --spider http://localhost:9000/health || exit 1; else exit 0; fi
# Default command: if rustfs binary exists, run it; otherwise, keep container alive for development
CMD if [ -f /usr/local/bin/rustfs ]; then \
echo "Starting RustFS server..."; \
exec /usr/local/bin/rustfs; \
else \
echo "Running in development mode..."; \
echo "RustFS source code should be mounted at /app"; \
exec bash -c "while true; do sleep 1; done"; \
fi
.docker/ubuntu/Dockerfile.source
-155
View File
@@ -1,155 +0,0 @@
# Multi-stage Dockerfile for RustFS
# Supports cross-compilation for amd64 and arm64 architectures
ARG TARGETPLATFORM
ARG BUILDPLATFORM
# Build stage
FROM --platform=$BUILDPLATFORM rust:1.88-bookworm AS builder
# Install required build dependencies
RUN apt-get update && apt-get install -y \
wget \
git \
curl \
unzip \
gcc \
pkg-config \
libssl-dev \
lld \
&& rm -rf /var/lib/apt/lists/*
# Install sccache for Rust compilation caching
RUN wget https://github.com/mozilla/sccache/releases/download/v0.8.1/sccache-v0.8.1-x86_64-unknown-linux-gnu.tar.gz \
&& tar -xzf sccache-v0.8.1-x86_64-unknown-linux-gnu.tar.gz \
&& mv sccache-v0.8.1-x86_64-unknown-linux-gnu/sccache /usr/local/bin/ \
&& chmod +x /usr/local/bin/sccache \
&& rm -rf sccache-v0.8.1-x86_64-unknown-linux-gnu.tar.gz sccache-v0.8.1-x86_64-unknown-linux-gnu
# Set up sccache environment
ENV RUSTC_WRAPPER=sccache \
SCCACHE_DIR=/tmp/sccache \
SCCACHE_CACHE_SIZE=2G
# Install cross-compilation tools for ARM64
RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
apt-get update && \
apt-get install -y gcc-aarch64-linux-gnu && \
rm -rf /var/lib/apt/lists/*; \
fi
# Install protoc
RUN wget https://github.com/protocolbuffers/protobuf/releases/download/v31.1/protoc-31.1-linux-x86_64.zip \
&& unzip protoc-31.1-linux-x86_64.zip -d protoc3 \
&& mv protoc3/bin/* /usr/local/bin/ && chmod +x /usr/local/bin/protoc \
&& mv protoc3/include/* /usr/local/include/ && rm -rf protoc-31.1-linux-x86_64.zip protoc3
# Install flatc
RUN wget https://github.com/google/flatbuffers/releases/download/v25.2.10/Linux.flatc.binary.g++-13.zip \
&& unzip Linux.flatc.binary.g++-13.zip \
&& mv flatc /usr/local/bin/ && chmod +x /usr/local/bin/flatc && rm -rf Linux.flatc.binary.g++-13.zip
# Set up Rust targets based on platform
RUN case "$TARGETPLATFORM" in \
"linux/amd64") rustup target add x86_64-unknown-linux-gnu ;; \
"linux/arm64") rustup target add aarch64-unknown-linux-gnu ;; \
*) echo "Unsupported platform: $TARGETPLATFORM" && exit 1 ;; \
esac
# Set up environment for cross-compilation
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc
ENV CC_aarch64_unknown_linux_gnu=aarch64-linux-gnu-gcc
ENV CXX_aarch64_unknown_linux_gnu=aarch64-linux-gnu-g++
WORKDIR /usr/src/rustfs
# Copy cargo configuration for optimized builds
COPY .docker/cargo.config.toml ./.cargo/config.toml
# Copy Cargo files for dependency caching
COPY Cargo.toml Cargo.lock ./
COPY */Cargo.toml ./*/
# Create dummy main.rs files for dependency compilation
RUN find . -name "Cargo.toml" -not -path "./Cargo.toml" | \
xargs -I {} dirname {} | \
xargs -I {} sh -c 'mkdir -p {}/src && echo "fn main() {}" > {}/src/main.rs'
# Configure cargo for optimized builds
ENV CARGO_NET_GIT_FETCH_WITH_CLI=true \
CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse \
CARGO_INCREMENTAL=0 \
CARGO_PROFILE_RELEASE_DEBUG=false \
CARGO_PROFILE_RELEASE_SPLIT_DEBUGINFO=off \
CARGO_PROFILE_RELEASE_STRIP=symbols
# Build dependencies only (cache layer) with optimizations
RUN sccache --start-server 2>/dev/null || true && \
case "$TARGETPLATFORM" in \
"linux/amd64") cargo build --release --target x86_64-unknown-linux-gnu -j $(nproc) ;; \
"linux/arm64") cargo build --release --target aarch64-unknown-linux-gnu -j $(nproc) ;; \
esac
# Copy source code
COPY . .
# Generate protobuf code
RUN cargo run --bin gproto
# Build the actual application with optimizations
RUN sccache --start-server 2>/dev/null || true && \
case "$TARGETPLATFORM" in \
"linux/amd64") \
cargo build --release --target x86_64-unknown-linux-gnu --bin rustfs -j $(nproc) && \
cp target/x86_64-unknown-linux-gnu/release/rustfs /usr/local/bin/rustfs \
;; \
"linux/arm64") \
cargo build --release --target aarch64-unknown-linux-gnu --bin rustfs -j $(nproc) && \
cp target/aarch64-unknown-linux-gnu/release/rustfs /usr/local/bin/rustfs \
;; \
esac && \
sccache --show-stats || true
# Runtime stage - Ubuntu minimal for better compatibility
FROM ubuntu:22.04
# Install runtime dependencies
RUN apt-get update && apt-get install -y \
ca-certificates \
tzdata \
wget \
&& rm -rf /var/lib/apt/lists/*
# Create rustfs user and group
RUN groupadd -g 1000 rustfs && \
useradd -d /app -g rustfs -u 1000 -s /bin/bash rustfs
WORKDIR /app
# Create data directories
RUN mkdir -p /data/rustfs{0,1,2,3} && \
chown -R rustfs:rustfs /data /app
# Copy binary from builder stage
COPY --from=builder /usr/local/bin/rustfs /app/rustfs
RUN chmod +x /app/rustfs && chown rustfs:rustfs /app/rustfs
# Switch to non-root user
USER rustfs
# Expose ports
EXPOSE 9000
# Environment variables
ENV RUSTFS_ACCESS_KEY=rustfsadmin \
RUSTFS_SECRET_KEY=rustfsadmin \
RUSTFS_ADDRESS=":9000" \
RUSTFS_CONSOLE_ENABLE=true \
RUSTFS_VOLUMES=/data \
RUST_LOG=warn
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:9000/health || exit 1
# Set default command
CMD ["/app/rustfs"]