Commit Graph

12 Commits

Author SHA1 Message Date
Zhengchao An 2b31bda6d1 ci: clear the checkout token in every job that does not push (#5547)
actions/checkout writes its token into .git/config as an http extraheader,
where it stays for the rest of the job. That matters more here than usual:
pull_request jobs run on self-hosted runners and execute the PR's own build.rs,
proc-macros and tests, any of which can read that file. Test and Lint holds
actions: write on top of that, so its token can cancel runs and delete the
Actions caches the whole pipeline now depends on — it was given
persist-credentials: false when that permission was added, and this extends the
same treatment to the other 45 checkouts.

Two are exempt because the token IS the credential the job needs.
helm-package's publish job pushes to rustfs/helm with it; clearing it would
break chart publishing. nix-flake-update is exempt pending verification: it
passes FLAKE_UPDATE_TOKEN to create-pull-request directly rather than reusing
.git/config, so it very likely does not need persistence, but that is unproven
and a broken weekly bot is not worth the guess. Both carry a
persist-credentials-exempt comment saying which.

scripts/security/check_persist_credentials.sh requires every checkout to either
clear its credentials or carry that comment, so the decision stays visible in
review rather than being an omission nobody notices.

Refs: rustfs/backlog#1598, rustfs/backlog#1602
2026-08-01 11:48:47 +08:00
Zhengchao An 6c99d4fe22 ci: stop the weekly flake.lock bot from running the whole pipeline (#5539)
nix-flake-update opens a PR every Sunday that changes flake.lock and nothing
else. flake.lock is consumed only by Nix packaging — cargo never reads it — yet
it was in no paths filter, so each of those PRs ran the full Continuous
Integration pipeline and the merge then ran Build and Release too. Added to all
four lists that have to agree: ci.yml's push and pull_request paths-ignore,
build.yml's push paths-ignore, and ci-docs-only.yml's paths.

The cron stays. flake.lock still has consumers — anyone running `nix build` or
`nix develop` — so stopping the updates would remove the workflow's output, not
just its CI cost.

Those four lists drifting is a silent failure, so scripts/check_ci_paths_sync.sh
now asserts the pair that matters: ci.yml's pull_request paths-ignore must equal
ci-docs-only.yml's paths. An entry present only in the first means a PR touching
those files triggers neither workflow, nobody reports "Test and Lint" or "Quick
Checks", and the PR waits on a required check forever. It also asserts both
companion job names still exist, since renaming one produces the same hang. The
push list is not compared: no required check is reported for push events.

Also in this cleanup:

- nix-flake-update's GITHUB_TOKEN drops to contents: read. The branch push and
  the pull request are both made by update-flake-lock with the
  FLAKE_UPDATE_TOKEN PAT, so the write scopes were an unused repo-write
  credential on an unattended weekly job.
- build.yml's build_docker dispatch input is documented as advisory. docker.yml
  triggers on workflow_run and requires the triggering event to be a tag push,
  so a manual dispatch never reaches it whatever this input says.
- The nine disabled workflow files get a banner saying so. Their
  disabled_manually state lives in GitHub's UI and is invisible when reading the
  file, which has already misled one audit into treating dead workflows as live.

Refs: rustfs/backlog#1598, rustfs/backlog#1603
2026-08-01 11:28:14 +08:00
Zhengchao An e2b2bdcc34 ci: bound every job's runtime and stop pasting inputs into shell (#5537)
Three hardening changes with no effect on what any workflow produces.

Declare timeout-minutes on the 25 jobs that lacked it. GitHub's default is 360
minutes, and this repository has a history of runners stalling intermittently
(#5394) plus a measured 9m57s plain `git checkout` under node-level I/O
contention, so one wedged job could hold a runner for six hours out of a pool of
roughly 15-21. Budgets follow what the jobs actually do: 10 minutes for
echo-only and guard-script jobs, 30 for anything calling the GitHub API,
uploading release assets or pushing over the network.
scripts/security/check_job_timeouts.sh keeps it that way, checking only jobs
that declare runs-on so reusable-workflow callers are not flagged.

Pass workflow inputs and workflow_run fields through env instead of `${{ }}`
interpolation in run blocks. A git ref name may contain `$(...)` — any string
without a space is a legal tag — and interpolation pastes it into the script
where bash evaluates it. The worst instance was helm-package's final commit
message: it is built from the triggering tag name inside the job that holds the
cross-repository push token with rustfs/helm already checked out. Also converted
in build.yml, docker.yml and performance-ab.yml; the last is currently disabled,
but a disabled workflow can be re-enabled. Not touched: helm-package's
`contains(head_branch, '.')` tag test, since GitHub expressions have no regex
and this repository's tags carry no `v` prefix, so rewriting the condition would
change which builds publish a chart.

Give audit.yml a scheduled-failure alert and run it daily. A scheduled
cargo-deny failure usually means the dependency tree just matched a newly
published RustSec advisory — the most important signal this workflow produces,
and until now it was visible only to whoever happened to open the Actions tab.
coverage.yml and e2e-replication-nightly.yml already use this ci-8 mechanism.
The cron moves from weekly to daily so a new advisory against an unchanged tree
surfaces within a day instead of seven; the check list is untouched, since
splitting it into a light daily run and a weekly full run would create runs
where sources, bans and licenses go unverified.

Refs: rustfs/backlog#1598, rustfs/backlog#1602
2026-08-01 11:25:26 +08:00
Zhengchao An 5f1759eb3c ci: cancel PR workflows on close (#4323) 2026-07-06 22:38:00 +08:00
Zhengchao An 7ae9697d42 docs: close final architecture audit gaps (#3966) 2026-06-28 00:07:42 +08:00
houseme 9e6d0c7292 chore(deps): refresh workflow actions and crate pins (#3696) 2026-06-21 20:29:17 +08:00
houseme aabda41ae5 build: upgrade Rust baseline to 1.96.0 (#3291)
* build: upgrade Rust baseline to 1.96.0

* ci: pin Rust workflows to 1.96.0

* remove

* ci: pin setup action to Rust 1.96.0

* build: pin toolchain to Rust 1.96.0
2026-06-08 20:32:37 +00:00
唐小鸭 f7724d223b feat(rio): rio_v2 is compatible with minio for storing data. (#3115)
* Set up a compatibility layer for replacing old Rio components with new ones.

* fix(rio). compress range

* feat(rio). Add the experimental feature rio_v2 to support minio data at the binary level.

* feat(rio_v2): add sse-c test

* test compression component

* simple fix

* fix minlz encode

* fix metadata

* fix kms key cache error

* Update launch.json

* ci: set nix crate download user agent

* fix: gate obs pyroscope backend

* ignore minio test

* fix encrypt check

* fix

* fix

* fix

* Update object_usecase.rs

* Update ci.yml

* fix

* ci add rio-v2 test

* fix

* ci fix

* fix

* Reconstructed into a more reasonable compatibility mode

* fix

* fix

---------

Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-06-08 11:59:14 +00:00
houseme 28bac7fbd6 chore(release): prepare 1.0.0-beta.6 (#3104)
* chore(release): prepare 1.0.0-beta.6

* ci(nix): harden flaky crate fetch handling

* ci(nix): drop magic cache and force fallback

* ci(nix): set explicit user-agent for crate fetch

* ci(nix): adopt determinate nix workflow stack

* ci(nix): add nix user-agent suffix for fetches

* ci(nix): add flakehub cache and align determinate actions

* ci(nix): pin determinate actions to release tags

* ci(nix): disable flakehub auth path in CI cache

* ci(nix): restore stable magic cache baseline

* ci(nix): trust local magic cache substituter

* ci(nix): stop forcing Node24 for JS actions

* ci(nix): drop manual localhost cache config

* ci(nix): adopt latest determinate flakehub stack

* ci(nix): record latest determinate workflow state
2026-05-28 09:21:16 +00:00
houseme cd2cd74314 ci: force Node24 in Nix workflows with pinned actions (#2966) 2026-05-14 11:13:42 +00:00
houseme 593a58c161 refactor(obs): optimize logging with custom RollingAppender and improved cleanup (#2151)
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-13 13:20:27 +08:00
houseme 1554e7e76a ci: optimize and translate nix workflow (#1821) 2026-02-15 00:37:57 +08:00