39 Commits

Author SHA1 Message Date
Zhengchao An 61dbaf60d3 ci: install zip and aws CLI on self-hosted runners (#4897) 2026-07-16 15:23:51 +08:00
Zhengchao An 4310b55238 ci: add schedule-failure-issue composite action and wire nightly alerts (#4671)
Scheduled workflow failures previously went unnoticed (15 consecutive red
weekly s3-tests sweeps, silent perf nightly failures). Add a reusable
composite action that opens a tracking issue titled
"[scheduled-failure] <workflow name>" — or appends a comment to the
existing open one (dedupe key = workflow name) — with the run URL, run
attempt, and failed job names, labeled "infrastructure".

Wire it into the scheduled paths of e2e-s3tests, mint, fuzz (nightly) and
performance-ab via a final alert-on-failure job gated by
"always() && github.event_name == 'schedule' &&
contains(needs.*.result, 'failure')", with issues:write scoped to that
job only. e2e-s3tests and mint previously had no permissions key; they now
get workflow-level contents:read, reducing every other job's token.

Add a dispatch-only drill workflow that forces a job failure and runs the
action through the exact consumer wiring, for end-to-end verification.
The ci-7 e2e nightly does not exist yet; it is recorded as a pending
consumer in the action README.

Refs: rustfs/backlog#1149 (ci-8), rustfs/backlog#1155
2026-07-10 20:53:44 +08:00
Zhengchao An 7ae9697d42 docs: close final architecture audit gaps (#3966) 2026-06-28 00:07:42 +08:00
安正超 6f4d0b54a1 fix(ci): install ripgrep for script checks (#3260) 2026-06-07 22:03:25 +08:00
安正超 0dbf0b13a8 fix(lock): align distributed acquisition retries (#3177)
* fix(lock): align distributed acquisition retries

* fix(lock): retry remote lock RPC timeouts

* fix(lock): quiet retryable acquisition logs

* fix(lock): enforce attempt acquisition deadline

* fix(ci): install unzip in setup action

* fix(lock): handle acquisition review followups

* fix(ci): use nextest install-action shorthand

* fix(lock): preserve hard failures on attempt timeout

* fix(lock): retry attempt timeout when quorum remains possible

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-03 01:58:47 +00:00
安正超 d817bd4449 fix(ci): stabilize Rust setup action (#3163)
* fix(ecstore): restore windows endpoint validation build

* fix(ci): install unzip for protoc setup

* fix(ci): use nextest install-action shorthand

* fix(ci): restore cargo-nextest action ref

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-03 01:51:30 +00:00
Henry Guo 69dcf9e6cb fix(tooling): harden internode transport benchmark setup (#3037)
* refactor(config): centralize internode transport constants

* fix(bench): guard all ripgrep calls behind dry-run check

Move require_cmd rg and metrics collection inside the non-dry-run
path so that --dry-run works on hosts without rg installed.

* feat(tooling): cross-platform protoc setup for Linux and macOS

Make install-protoc.sh support Linux (x86_64, aarch64) alongside
macOS, and bump CI protoc from 29.3 to 33.1 to match the version
required by the gproto build script.

* fix(bench): record internode baseline error counts

* fix(skill): correct YAML frontmatter formatting for release-version-bump

* chore(ci): bump protoc version to 34.1

* fix(tooling): bump protoc 33.1 to 34.1 in install script, restore SKILL.md description

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-21 05:47:32 +00:00
Henry Guo e929814edc feat(ecstore): add internode transport boundary and TCP baseline runner (#3010)
* docs: add internode data transport RFC

* feat: add internode operation metrics

* fix feedback

* fix(ci): fallback protoc token to github.token

* feat(ecstore): add internode transport boundary and baseline runner

* feat(internode): harden data transport baseline

* Revert "feat(internode): harden data transport baseline"

This reverts commit 5b8d6b8aa4.

* fix(internode): address baseline review comments

* fix(ci): pin setup-protoc to stable release

* fix(ci): install protoc via apt on linux

* fix(ci): restore protoc install for macos and windows

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-19 22:41:34 +00:00
安正超 1c8fdfddf4 chore(ci): pin RustFS setup-protoc release (#3016)
* chore(ci): use RustFS setup-protoc fork

* chore(ci): pin RustFS setup-protoc release
2026-05-19 15:28:51 +00:00
Henry Guo f695870626 feat(internode): add transport observability (#3007)
* docs: add internode data transport RFC

* feat: add internode operation metrics

* fix feedback

* fix(ci): fallback protoc token to github.token

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-19 07:16:05 +00:00
houseme bd1e57293f fix: harden lifecycle transition compensation and regression coverage (#2995)
* fix(ecstore): honor transition worker configuration

* fix(ecstore): add transition queue backpressure metrics

* fix(ecstore): schedule transition compensation on enqueue pressure

* fix(ecstore): log transition compensation scheduling

* test(rustfs): add transition compensation fault-injection coverage

* test(rustfs): cover delete after transition compensation

* test(scanner): cover cleanup after transition compensation

* test(rustfs): extend compensation transition coverage

* test(scanner): cover backfill idempotency after compensation

* test(scanner): cover noncurrent expiry after compensation

* test(rustfs): cover versioned delete after compensation

* test(rustfs): cover delete marker lifecycle after compensation

* test(scanner): extend versioned lifecycle compensation coverage

* test(scanner): model versioned delete after compensation

* test(scanner): clarify modeled versioned delete helper

* refactor(ecstore): optimize transition enqueue hot path

* refactor(ecstore): centralize transition runtime constants

* style(ecstore): apply rustfmt for transition timeout helper

* fix(ilm): align queue-full metric semantics

* refactor(ecstore): unify immediate enqueue failure handling

* refactor(ecstore): reuse transition worker env constant

* ci(actions): update setup action inputs
2026-05-18 12:50:43 +00:00
安正超 1977d19c29 ci: force Node.js 24 for JS setup actions (#2424) 2026-04-07 23:49:43 +08:00
loverustfs 1e35edf079 chore(ci): restore workflows before 8e0aeb4 (#1212) 2025-12-20 07:50:49 +08:00
loverustfs 8e0aeb4fdc Optimize ci ubicloud (#1208) 2025-12-19 23:22:45 +08:00
houseme 9d553620cf remove linux dep and upgrade Protocol Buffers and FlatBuffers (#853) 2025-11-14 12:50:55 +08:00
houseme 2c7366038e modify protobuf version from to 2025-08-13 01:01:50 +08:00
houseme 1cc6dfde87 modify protobuf version from 31.1 to 31.0 2025-08-13 00:58:22 +08:00
安正超 6ef6f188e5 fix: Restore working build configuration from 4fb4b353 (#204)
* fix: Resolve zstd-sys Zig compilation issues

- Remove specific Zig version constraint in action.yml to use default version
- Clean up duplicate environment variable settings in build-rustfs.sh
- Add CARGO_TARGET_*_LINKER environment variables for better cross-compilation support
- Optimize build configuration for consistent cross-platform compilation

Fixes compilation issues with zstd-sys when using Zig cross-compilation.
Aligns with previously working configuration that uses default Zig version.

* fix: Restore working build configuration from 4fb4b353

- Restore matrix.cross parameter to differentiate cross-compilation
- Use simple cargo zigbuild instead of complex build-rustfs.sh script
- Remove unnecessary zstd dependencies from action.yml
- Restore console asset download step
- Use correct target directory path for packaging
- Align with known working configuration from commit 4fb4b353

This reverts to the proven working build approach that successfully
performed cross-platform compilation.

* fix: Align build-rustfs.sh with working version logic

- Simplify build logic to match working version 4fb4b353
- Use exact same build commands as the working build.yml:
  * cargo build for native compilation
  * cargo zigbuild for Linux ARM64 cross-compilation
  * cross build for Windows ARM64 cross-compilation
- Remove complex environment variable setup that caused conflicts
- Add touch rustfs/build.rs to match working version
- Use -p rustfs --bins flag consistent with working version

This ensures build-rustfs.sh (if used) follows the proven working approach.
2025-07-14 20:22:29 +08:00
安正超 ccad91a4a9 fix: resolve zstd-sys compilation issues with zig cross-compilation (#203)
- Update to mlugg/setup-zig@v2 for better stability and features
- Use Zig 0.13.0 for improved musl target support
- Add system zstd libraries (libzstd-dev, zstd) to Ubuntu dependencies
- Configure environment variables for zstd-sys to use pkg-config
- Enable pkg-config feature for zstd dependency to prefer system library
- Add proper C/C++ compiler configuration for musl targets

Fixes the 'error: unable to parse target query x86_64-unknown-linux-musl: UnknownOperatingSystem'
compilation error in zstd-sys during cross-compilation.
2025-07-14 20:01:52 +08:00
Nugine 58c5a633e2 ci: fix cache (#165) 2025-07-10 23:50:26 +08:00
Nugine a3b5445824 ci: use nextest (#148)
* ci: use nextest

* add doctests back
2025-07-10 11:33:12 +08:00
overtrue aa65766312 fix: api rate limit 2025-07-09 09:16:04 +08:00
overtrue 660f004cfd fix: api rate limit 2025-07-09 09:11:46 +08:00
overtrue 49f480d346 fix: resolve GitHub Actions build failures and optimize cross-compilation
- Remove invalid github-token parameter from arduino/setup-protoc action
- Fix cross-compilation RUSTFLAGS issue by conditionally setting target-cpu=native
- Update workflow tag triggers from v* to * for non-v prefixed tags
- Optimize Zig and cargo-zigbuild installation using official actions

This resolves build failures in aarch64-unknown-linux-musl target where
zig was receiving invalid x86_64 CPU flags during cross-compilation.
2025-07-08 20:21:11 +08:00
安正超 9862677fcf fix: restore Zig and cargo-zigbuild caching in GitHub Actions setup action (#92)
* fix: restore Zig and cargo-zigbuild caching in GitHub Actions setup action

Use mlugg/setup-zig and taiki-e/cache-cargo-install-action to speed up cross-compilation tool installation and avoid repeated downloads. All comments and code are in English.

* fix: use correct taiki-e/install-action for cargo-zigbuild

Use taiki-e/install-action@cargo-zigbuild instead of taiki-e/cache-cargo-install-action@v2 to match the original implementation from PR #77.

* refactor: remove explicit Zig version to use latest stable
2025-07-07 23:15:40 +08:00
安正超 1ecd5a87d9 feat: optimize GitHub Actions workflows with performance improvements (#77)
* feat: optimize GitHub Actions workflows with performance improvements

- Rename workflows with more descriptive names
- Add unified setup action for consistent environment setup
- Optimize caching strategy with Swatinem/rust-cache@v2
- Implement skip-check mechanism to avoid duplicate builds
- Simplify matrix builds with better include/exclude logic
- Add intelligent build strategy checks
- Optimize Docker multi-arch builds
- Improve artifact naming and retention
- Add performance testing with benchmark support
- Enhance security audit with dependency scanning
- Change Chinese comments to English for better maintainability

Performance improvements:
- CI testing: ~35 min (42% faster)
- Build release: ~60 min (50% faster)
- Docker builds: ~45 min (50% faster)
- Security audit: ~8 min (47% faster)

* fix: correct secrets context usage in GitHub Actions workflow

- Move environment variables to job level to fix secrets access issue
- Fix unrecognized named-value 'secrets' error in if condition
- Ensure OSS upload step can properly check for required secrets

* fix: resolve GitHub API rate limit by adding authentication token

- Add github-token input to setup action to authenticate GitHub API requests
- Pass GITHUB_TOKEN to all setup action usages to avoid rate limiting
- Fix arduino/setup-protoc@v3 API access issues in CI/CD workflows
- Ensure protoc installation can successfully access GitHub releases API
2025-07-07 12:38:17 +08:00
houseme 2e14b32ccd chore: Add copyright and license headers (#23)
* chore: Add copyright and license headers

This commit adds the Apache 2.0 license and a copyright notice to the header of all source files. This ensures that the licensing and copyright information is clearly stated within the codebase.

* cargo fmt

* fix

* fmt

* fix clippy
2025-07-02 15:07:47 +08:00
Nugine e82a69c9ca fix(ci): refactor ci check 2025-06-13 18:12:19 +08:00
houseme d66525a22f refactor(deps): centralize crate versions in root Cargo.toml (#448)
* chore(ci): upgrade protoc from 30.2 to 31.1

- Update protoc version in GitHub Actions setup workflow
- Use arduino/setup-protoc@v3 to install the latest protoc version
- Ensure compatibility with current project requirements
- Improve proto file compilation performance and stability

This upgrade aligns our development environment with the latest protobuf standards.

* modify package version

* refactor(deps): centralize crate versions in root Cargo.toml

- Move all dependency versions to workspace.dependencies section
- Standardize AWS SDK and related crates versions
- Update tokio, bytes, and futures crates to latest stable versions
- Ensure consistent version use across all workspace members
- Implement workspace inheritance for common dependencies

This change simplifies dependency management and ensures version consistency across the project.

* fix

* modify
2025-06-07 21:52:59 +08:00
houseme f8f58a8355 test 2025-05-10 00:52:41 +08:00
houseme 2a9a60197b # Add aarch64-apple-darwin Build Target Support
Added ARM64 macOS (Apple Silicon) build target support to the CI/CD pipeline by:

1. Including `aarch64-apple-darwin` as a new build variant in the build matrix
2. Adding proper exclusion rules to ensure the target only runs on macOS runners
3. Ensuring compatibility with the existing build scripts and packaging process

This change enables native builds for Apple Silicon Macs, improving performance for users with M1/M2/M3/M4 processors while maintaining the same artifact organization and deployment process.
2025-05-10 00:15:05 +08:00
houseme f5a97b63b9 chore(ci): optimize build workflow and update protoc version
- Update protoc version from 27.0 to 30.2 for better compatibility
- Improve build workflow parameters handling
2025-04-10 11:49:44 +08:00
houseme 96625b42ff improve code for build 2025-04-07 17:47:43 +08:00
houseme 212ec9ca69 improve 2025-04-07 17:24:13 +08:00
houseme 8de029bd7a update 2025-04-07 17:13:19 +08:00
houseme 5204cb67d8 Change build target to x86_64-unknown-linux-musl and update system dependencies
- Change build target from `x86_64-unknown-linux-gnu` to `x86_64-unknown-linux-musl`
- Default install `x86_64-unknown-linux-musl` toolchain in setup action
- Add `musl-tools` and `build-essential` to system dependencies
2025-04-07 16:55:14 +08:00
Nugine 09b855a3a9 ci: fix setup 2025-03-02 04:08:40 +08:00
Nugine 0a01113c83 ci: fix e2e tests 2025-02-26 15:14:02 +08:00
Nugine e87cc87cbf ci: build for old glibc (#232)
* build(deps): shadow_rs

* ci(build): build for old glibc

* ci: simplify all
2025-02-22 20:25:16 +08:00