Compare commits

...

1 Commits

Author SHA1 Message Date
hector 82fb2811c5 ci: build macOS and Windows on a weekly schedule
Keep the 01:13 UTC cron on Linux and add a Sunday 16:00 UTC cron
for macOS and Windows. Select that subset from the firing cron so a
delayed runner cannot guess the weekday.

Co-authored-by: hector <majinghe@users.noreply.github.com>
2026-11-11 01:46:03 +00:00
6 changed files with 95 additions and 17 deletions
+3 -2
View File
@@ -360,14 +360,15 @@
"cadence": "nightly",
"timezone": "UTC",
"schedule": [
"13 1 * * *"
"13 1 * * *",
"0 16 * * 0"
],
"owner": "@overtrue",
"policy": "active",
"monitor": "workflow",
"max_age_hours": 36,
"result_owner": "build.yml",
"disposition": "Builds artifacts only; public release writes belong to release-publish.yml.",
"disposition": "Daily cron builds Linux; Sunday 16:00 UTC builds macOS and Windows. Public release writes belong to release-publish.yml.",
"failure_owner": "workflow",
"origin": "baseline",
"alert_contract": "scheduled-validation"
+1 -1
View File
@@ -80,6 +80,6 @@ Rows using `macos-latest-xlarge` or a label starting with `sm-`, `dind-`, `ubicl
## Release stages
The [candidate release pipeline](../../docs/operations/release-pipeline.md) builds Linux first and defers paid desktop capacity until acceptance. Daily development builds also select Linux; use manual platform selection for desktop development artifacts. All six platforms remain required before publication. The pipeline records and promotes the exact tested bytes. Builder workflows have no public-release credentials. Keep periodic development builds, PR verification and field-test chains separate when their contracts differ; do not re-enable disabled workflows as part of a source-file migration.
The [candidate release pipeline](../../docs/operations/release-pipeline.md) builds Linux first and defers paid desktop capacity until acceptance. The daily `build.yml` schedule selects Linux; the Sunday 16:00 UTC schedule selects macOS and Windows. Manual dispatch can still select any platform list. All six platforms remain required before publication. The pipeline records and promotes the exact tested bytes. Builder workflows have no public-release credentials. Keep periodic development builds, PR verification and field-test chains separate when their contracts differ; do not re-enable disabled workflows as part of a source-file migration.
Larger runners remain billed for public repositories; see [GitHub runner billing](https://docs.github.com/en/actions/concepts/runners/larger-runners#billing). Actual spending comes from organization billing, not runner labels or an estimated build duration.
+27 -8
View File
@@ -12,7 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# Nightly development artifacts and reusable immutable-candidate builds.
# Scheduled development artifacts and reusable immutable-candidate builds.
# The daily cron builds Linux. Sunday 16:00 UTC builds macOS and Windows.
# Public release writes live exclusively in release-publish.yml. Candidate callers
# select Linux or desktop and pin release inputs before any matrix starts.
@@ -30,7 +31,8 @@ on:
required: true
type: string
schedule:
- cron: "13 1 * * *" # Nightly 01:13 UTC
- cron: "13 1 * * *" # Daily Linux, 01:13 UTC
- cron: "0 16 * * 0" # Weekly macOS and Windows, Sunday 16:00 UTC (Monday 00:00 Asia/Shanghai)
workflow_dispatch:
inputs:
publish:
@@ -219,16 +221,33 @@ jobs:
env:
# via env, not interpolation: a dispatch input is free-form text and
# would otherwise be pasted into the script for bash to evaluate.
# Daily development builds cover Linux. Desktop remains available on
# manual dispatch and is required after candidate field acceptance.
# A direct schedule has no platforms input. Choose its subset from
# github.event.schedule, which is fixed when that cron fires, so a
# queued job cannot switch subsets because the runner clock moved.
# Callers that pass platforms keep that list, including a scheduled
# caller: inputs.platforms is set even though event_name stays schedule.
RAW_PLATFORMS: ${{ inputs.platforms || 'linux' }}
HAS_PLATFORM_INPUT: ${{ inputs.platforms && 'true' || 'false' }}
EVENT_NAME: ${{ github.event_name }}
EVENT_SCHEDULE: ${{ github.event.schedule }}
run: |
set -euo pipefail
selected="$RAW_PLATFORMS"
selected="$(echo "${selected}" | tr -d '[:space:]')"
if [[ -z "${selected}" ]]; then
selected="all"
if [[ "${EVENT_NAME}" == "schedule" && "${HAS_PLATFORM_INPUT}" != "true" ]]; then
if [[ "${EVENT_SCHEDULE}" == "13 1 * * *" ]]; then
selected="linux"
elif [[ "${EVENT_SCHEDULE}" == "0 16 * * 0" ]]; then
selected="desktop"
else
echo "Unrecognized schedule cron: ${EVENT_SCHEDULE}" >&2
exit 1
fi
else
selected="${RAW_PLATFORMS}"
selected="$(echo "${selected}" | tr -d '[:space:]')"
if [[ -z "${selected}" ]]; then
selected="all"
fi
fi
# linux-aarch64-* need a real aarch64 host (placement rule: hardware)
+1 -1
View File
@@ -10,7 +10,7 @@ RustFS builds the final product version before publication. A candidate is ident
| `release-complete.yml` | Successful candidate run, accepted Linux manifest SHA-256, backlog acceptance report | Desktop archives, DEB/RPM, OCI images, Helm chart, full certification and sealed manifest | None |
| `release-publish.yml` | Successful completion run and approved final manifest SHA-256 | Annotated tag, GitHub Release, R2 files, registry images, stable pointers and Helm index | Explicit publication |
The builder workflows (`build.yml`, `package.yml`, `docker.yml`, `helm-package.yml`) are reusable stages. Only `release-publish.yml` has release write credentials. Development builds retain their artifact names and download channel. Daily development builds now select Linux only; macOS/Windows development artifacts are refreshed by an explicit manual build, and both remain mandatory for a completed release candidate. Tag pushes no longer start a second compilation.
The builder workflows (`build.yml`, `package.yml`, `docker.yml`, `helm-package.yml`) are reusable stages. Only `release-publish.yml` has release write credentials. Development builds retain their artifact names and download channel. The daily schedule selects Linux only; the Sunday 16:00 UTC schedule selects macOS and Windows. Manual dispatch can still select any platform list, and both desktop targets remain mandatory for a completed release candidate. Tag pushes no longer start a second compilation.
## Freeze and build
+2 -2
View File
@@ -87,7 +87,7 @@ Scheduled lanes never block a PR. Their workflow-local gate fails the run. The v
|---|---|---|---|---|
| `ci.yml` (nightly) | full matrix, including the schedule/dispatch-only rio-v2 jobs `build-rustfs-debug-binary-rio-v2` and `e2e-tests-rio-v2` | strict aggregate; the full E2E lane runs on dispatch, merge groups, and main pushes | yes | dispatch `ci.yml` |
| `cache-warm.yml` (every six hours, plus main pushes that change `Cargo.lock`, `rust-toolchain.toml` or the setup action) | `warm-ci-dev`, `warm-ci-feat-rio`, `warm-ci-feat-proto`, `warm-ci-uring` | no verdict; sole writer of the `rust-cache` keys that `ci.yml` restores | yes | dispatch `cache-warm.yml` |
| `build.yml` (nightly) | `build-rustfs` over the six-target platform matrix in `prepare-platform-matrix` (four Linux, macOS aarch64, Windows x86_64) | build/package integrity | yes | dispatch `build.yml` with an exact platform set |
| `build.yml` (nightly Linux, weekly desktop) | `build-rustfs` over the `prepare-platform-matrix` subset for the firing cron: four Linux targets at 01:13 UTC, and macOS aarch64 plus Windows x86_64 at Sunday 16:00 UTC | build/package integrity | yes | dispatch `build.yml` with an exact platform set |
| `e2e-replication-nightly.yml` (nightly) | `repl-nightly`, `cluster-nightly`, `protocols-nightly` | three independent gates; JUnit, membership listing, server logs | yes | `python3 scripts/e2e_binary.py build --bins`, then `python3 scripts/e2e_binary.py run -- cargo nextest run --profile e2e-repl-nightly -p e2e_test`; `--features e2e-test-hooks` on both steps with `--profile e2e-nightly`; `--features ftps,webdav,sftp` on both steps with `-j 1 --profile e2e-protocols` |
| `e2e-distributed.yml` (nightly) | `distributed` | fail-closed 4-node 4-disk S3, durability, replication, movement, fault, and direct/rolling upgrade gate; JUnit, membership listing, per-node server logs | yes | download the pinned previous release as in the workflow, export `RUSTFS_UPGRADE_SOURCE_BINARY`, run `python3 scripts/e2e_binary.py build --bins --features e2e-test-hooks`, then `python3 scripts/e2e_binary.py run --features e2e-test-hooks -- cargo nextest run --profile e2e-distributed -p e2e_test` |
| `e2e-s3tests.yml` (weekly) | `s3tests` (single and distributed, four shards each), `upstream-head-canary` | compatibility gate; report, JUnit, node IDs, server logs | yes | `scripts/s3-tests/run.sh` against an existing single or distributed target |
@@ -122,7 +122,7 @@ The versioned plan is `.config/release-validation-plan.json` (`schema_version=1`
| Trigger | Workflow / job | Result |
|---|---|---|
| Nightly schedule, dispatch | `build.yml` `build-rustfs` (development build over the full platform matrix) | build artifacts and the uncertified dev download channel; no release publication |
| Nightly Linux schedule, weekly desktop schedule, or dispatch | `build.yml` `build-rustfs` (development build of that schedule's platform subset, or the dispatch platform list) | build artifacts and the uncertified dev download channel; no release publication |
| Valid release or preview tag | `build.yml` `build-rustfs`, `release-certification`, `create-release`, `upload-release-assets`, `publish-release`, `promote-release-artifacts` | staged binaries, draft release, checksummed assets, then certified publish and promotion of the same bytes |
| Successful non-preview release-tag build (`workflow_run`) | `docker.yml` `release-certification`, `build-docker`, `scan-docker-image` | multi-architecture images staged, verified against the certification and promoted, plus vulnerability report |
| Successful release-tag build (`workflow_run`) | `package.yml` `release-certification`, `package`, `promote-packages` | DEB/RPM packages with wrapped-binary verification, staged then promoted with checksums |
+61 -3
View File
@@ -343,11 +343,69 @@ class PromotionTests(unittest.TestCase):
class WorkflowContractTests(unittest.TestCase):
def test_nightly_avoids_desktop_and_manual_builds_keep_platform_selection(self):
def test_schedule_cron_selects_linux_or_desktop_without_dropping_manual_platforms(self):
value = workflow("build.yml")
selection = value["jobs"]["prepare-platform-matrix"]["steps"][-1]
self.assertEqual(selection["env"]["RAW_PLATFORMS"], "${{ inputs.platforms || 'linux' }}")
self.assertEqual([item["cron"] for item in value["on"]["schedule"]], ["13 1 * * *", "0 16 * * 0"])
self.assertNotIn("push", value["on"])
self.assertNotIn("pull_request", value["on"])
self.assertEqual(value["on"]["workflow_dispatch"]["inputs"]["platforms"]["default"], "all")
selection = value["jobs"]["prepare-platform-matrix"]["steps"][-1]
env = selection["env"]
self.assertEqual(env["RAW_PLATFORMS"], "${{ inputs.platforms || 'linux' }}")
self.assertEqual(env["HAS_PLATFORM_INPUT"], "${{ inputs.platforms && 'true' || 'false' }}")
self.assertEqual(env["EVENT_NAME"], "${{ github.event_name }}")
self.assertEqual(env["EVENT_SCHEDULE"], "${{ github.event.schedule }}")
linux = [
("linux-x86_64-musl", "sm-standard-4"),
("linux-aarch64-musl", "ubicloud-standard-16-arm"),
("linux-x86_64-gnu", "sm-standard-4"),
("linux-aarch64-gnu", "ubicloud-standard-16-arm"),
]
desktop = [("macos-aarch64", "macos-latest-xlarge"), ("windows-x86_64", "windows-latest")]
def select(event, raw, schedule="", has_input="false"):
with tempfile.TemporaryDirectory() as temp:
output = Path(temp) / "github-output"
environment = {**os.environ, "EVENT_NAME": event, "EVENT_SCHEDULE": schedule,
"RAW_PLATFORMS": raw, "HAS_PLATFORM_INPUT": has_input, "GITHUB_OUTPUT": str(output)}
result = subprocess.run(["bash", "-c", selection["run"]], env=environment, capture_output=True, text=True)
parsed = {}
if output.exists():
for line in output.read_text().splitlines():
key, item = line.split("=", 1)
parsed[key] = item
return result, parsed
def rows(parsed):
return [(item["target_id"], item["os"]) for item in json.loads(parsed["matrix"])["include"]]
daily, daily_out = select("schedule", "all", "13 1 * * *")
self.assertEqual(daily.returncode, 0, daily.stderr)
self.assertEqual(rows(daily_out), linux)
self.assertEqual(daily_out["selected"], ",".join(item[0] for item in linux))
weekly, weekly_out = select("schedule", "linux", "0 16 * * 0")
self.assertEqual(weekly.returncode, 0, weekly.stderr)
self.assertEqual(rows(weekly_out), desktop)
self.assertEqual(weekly_out["selected"], ",".join(item[0] for item in desktop))
for schedule in ("", "13 1 * * 1", "0 16 * * 7", "13 1 extra stuff"):
rejected, _ = select("schedule", "all", schedule)
self.assertNotEqual(rejected.returncode, 0, schedule)
self.assertIn("Unrecognized schedule cron", rejected.stderr)
caller, caller_out = select("schedule", "linux", "0 16 * * 0", has_input="true")
self.assertEqual(caller.returncode, 0, caller.stderr)
self.assertEqual(rows(caller_out), linux)
dispatched, dispatched_out = select("workflow_dispatch", "all", has_input="true")
self.assertEqual(dispatched.returncode, 0, dispatched.stderr)
self.assertEqual(dispatched_out["selected"], "all")
self.assertEqual(rows(dispatched_out), linux + desktop)
custom, custom_out = select("workflow_dispatch", "linux-x86_64-musl,macos-aarch64", has_input="true")
self.assertEqual(custom.returncode, 0, custom.stderr)
self.assertEqual(custom_out["selected"], "linux-x86_64-musl,macos-aarch64")
self.assertEqual(rows(custom_out), [("linux-x86_64-musl", "sm-standard-4"), ("macos-aarch64", "macos-latest-xlarge")])
for event, raw, expected in (("workflow_call", "linux", linux), ("workflow_call", "desktop", desktop)):
called, called_out = select(event, raw, has_input="true")
self.assertEqual(called.returncode, 0, called.stderr)
self.assertEqual(rows(called_out), expected)
def test_main_push_cannot_cancel_the_scheduled_oidc_gate(self):
value = workflow("oidc-keycloak.yml")