ci: add schedule-failure-issue composite action and wire nightly alerts (#4671)

Scheduled workflow failures previously went unnoticed (15 consecutive red weekly s3-tests sweeps, silent perf nightly failures). Add a reusable composite action that opens a tracking issue titled "[scheduled-failure] <workflow name>" — or appends a comment to the existing open one (dedupe key = workflow name) — with the run URL, run attempt, and failed job names, labeled "infrastructure". Wire it into the scheduled paths of e2e-s3tests, mint, fuzz (nightly) and performance-ab via a final alert-on-failure job gated by "always() && github.event_name == 'schedule' && contains(needs.*.result, 'failure')", with issues:write scoped to that job only. e2e-s3tests and mint previously had no permissions key; they now get workflow-level contents:read, reducing every other job's token. Add a dispatch-only drill workflow that forces a job failure and runs the action through the exact consumer wiring, for end-to-end verification. The ci-7 e2e nightly does not exist yet; it is recorded as a pending consumer in the action README. Refs: rustfs/backlog#1149 (ci-8), rustfs/backlog#1155
2026-08-26 05:56:50 +00:00 · 2026-07-10 20:53:44 +08:00
parent fd18b1df49
commit 4310b55238
7 changed files with 329 additions and 0 deletions
@@ -0,0 +1,62 @@
+# Copyright 2024 RustFS Team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Manual drill for the scheduled-failure alerting path (backlog#1149 ci-8).
+#
+# Forces a job failure and then runs .github/actions/schedule-failure-issue
+# through the exact `needs` + `always() && contains(needs.*.result,
+# 'failure')` wiring used by the real consumers (e2e-s3tests, mint, fuzz,
+# performance-ab). Dispatch it twice to verify both the issue-creation and
+# the dedupe-comment paths, then close the resulting
+# "[scheduled-failure] Schedule Failure Alert Drill" issue.
+#
+# The run itself is expected to end red (the forced failure); only the
+# alert-on-failure job result matters.
+
+name: Schedule Failure Alert Drill
+
+on:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  forced-failure:
+    name: Forced failure
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    steps:
+      - name: Fail on purpose
+        run: |
+          echo "Deliberate failure so alert-on-failure exercises the real consumer wiring."
+          exit 1
+
+  alert-on-failure:
+    name: Alert on scheduled failure
+    needs: [forced-failure]
+    # Mirrors the consumer wiring, minus the schedule-event guard (this
+    # workflow is dispatch-only by design).
+    if: always() && contains(needs.*.result, 'failure')
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    permissions:
+      contents: read
+      issues: write
+    steps:
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
+      - name: Open or update failure-tracking issue
+        uses: ./.github/actions/schedule-failure-issue
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}